summaryrefslogtreecommitdiff
path: root/regtest
AgeCommit message (Collapse)AuthorFilesLines
2013-12-15regtest: Add a command line option to create-report command to not launch ↵Carlos Garcia Campos2-3/+7
the browser
2013-12-15regtest: Return an exist status code depending on whether the command succeededCarlos Garcia Campos8-10/+20
2013-12-06regtest: Limit the stderr files to ~1MBCarlos Garcia Campos1-1/+5
Some tests send a lot of information to stderr, usually due to parsing errors in buggy documents. More than 1MB of stderr output is diffcult to hanlde and in most cases it's redundant with a lot of duplicated messages. This patch reduced the size of the refs dir for the complete test suite by 1GB.
2013-12-06regtest: Save checksum results sorted in md5 filesCarlos Garcia Campos1-1/+1
We are using os.listdir() to get the list of test results that returns files in arbitrary order.
2013-12-06regtest: Show also the tests expected to crash/fail to run but don't failCarlos Garcia Campos1-1/+7
2013-12-06regtest: Improve readability of test resultsCarlos Garcia Campos1-20/+47
Show a summary of tests failed per backend with the percentages and use a new line for every test in the result instead of using a comma separated line.
2013-11-29regtest: Do not buffer stderr outputCarlos Garcia Campos1-10/+30
Some buggy documents can produce a huge stderr output because of parsing errors or whatever. We could give a file directly to Popen to write the stderr file, but we only want to create the file when there's output, because it's what we use to know whether the command produced output or not. So, instead of buffering the whole output and then write it to the file, now we read from the pipe while the command is running, writing the output in chunks to the file. This improves a lot the memory consumption when running some tests.
2013-11-29regtest: Read test results in chunks to get the md5 digestCarlos Garcia Campos1-7/+12
Some backends can generate huge results, like huge postscript files that we don't want to load in memory to get the md5. So, instead of creating thr md5 object with the entire file, we feed it with chunks of data using the update method. This improves a lot the memory consumption and performance as well.
2013-11-29regtest: Remove unused method _check_exit_status2Carlos Garcia Campos1-20/+0
It was used when the backends ran in parallel odd and even pages, but it's no longer used since threads support was added.
2013-11-29regtest: Do not store the current line in Printer but only its lengthCarlos Garcia Campos1-5/+5
We are not using the line text anymore, but only the length.
2013-11-27regtest: Do not consider docs with no refs as skippedCarlos Garcia Campos1-17/+23
Handle them differently as new docs, and show them in the summary suggesting to use create-refs command to include them in the test suite.
2013-11-27regtest: Use number of tests run to show the progressCarlos Garcia Campos3-112/+118
Instead of using the number of document tested. We don't really need to set a number to a particular document, we only want to know the progress of the whole process. This ensures that the progress is shown in the correct order. It also simplifies the code a bit. Also improved the output formatting to make it easier to read.
2012-12-29regtest: Print test results when the test has finishedCarlos Garcia Campos2-15/+28
To make sure the result corresponds to the test now that we are using multiple threads.
2012-12-08regtest: Use the number of cpus as default number of worker threadsCarlos Garcia Campos1-3/+5
2012-12-08regtest: Remove trailing whitespacesCarlos Garcia Campos3-13/+13
2012-12-06implement parallel testing using Python's Queue classAdam Reichold6-34/+104
2012-10-21regstest: Add print_test_result_ln() and use it for failed test resultsCarlos Garcia Campos2-5/+8
To make sure they are always shown.
2012-10-21regtest: Do not rewrite lines in verbose modeCarlos Garcia Campos1-1/+1
2012-09-23regtest: Reduce the noise of the default output when running testsCarlos Garcia Campos8-29/+137
Show permanent information only about failed tests, without the details about the failing pages. Previous verbose output is available passing --verbose command line output.
2012-04-01regtest: remove debug print to show bisect exit status in Bisect.pyCarlos Garcia Campos1-1/+0
2012-04-01regtest: Fix crash when creating HTML report for a single testCarlos Garcia Campos1-4/+1
2012-04-01regtest: Don't show results for crashed or failed to run tests in HTML reportCarlos Garcia Campos1-0/+2
2012-03-11regtest: Add find-regression command to run git bisect automaticallyCarlos Garcia Campos4-0/+339
2012-03-10regtest: Add information about skipped tests in the summaryCarlos Garcia Campos1-3/+9
2012-03-03regtest: Ignore backends with no results when creating html reportCarlos Garcia Campos1-0/+5
2012-03-03regtest: Add --pretty-diff option to create-report commandCarlos Garcia Campos2-3/+129
It includes a pretty-diff link to make a bit easier to check differences in test results. For images it creates a html with javascript that toggles actual and expected image, using the same code than WebKit layout test results. For text files it uses HtmlDiff class from python difflib. It's an option disabled by default because pretty diff for text files is very slow.
2012-02-25regtest: Add create-report command to generate html report of test resultsCarlos Garcia Campos3-1/+273
2012-02-25regtest: Use diff.png extension for image diff filesCarlos Garcia Campos4-5/+17
So that they are recognized as images
2012-01-07regtest: read stderr output before calling wait to fix a possible deadlockThomas Freitag1-4/+4
In the way the scripts starts it subprocesses, stderr becomes buffered for the subprocesses. And unforunately, when the buffer limit is reached, the subprocess suspends it work until it can print again on stderr. That's why the python script runs into a deadlock when the subprocess produces a lot of error messages. A small rearrange of the commands, first read the stderr output and then wait that the subprocess ends, will remove this deadlock.
2011-11-06regtest: Skip tests with results when creating refsCarlos Garcia Campos2-2/+5
Not only tests that have md5, but also crashed or failed tests.
2011-11-01regtest: Add --update-refs command line option to run-tests commandCarlos Garcia Campos3-3/+37
It allows to update the references of failing tests.
2011-09-20Update .gitignoreAdrian Johnson1-0/+1
2011-09-19regtest: render cairo at 72ppiAdrian Johnson1-2/+2
For consistency with splash. It is faster and easier to compare with splash results.
2011-09-19regtest: Ignore checksums of crashed, failed and stderr filesCarlos Garcia Campos1-3/+3
They shouldn't have a checksum, but it might happen with md5 files generated with previous versions.
2011-09-18Render at 72 instead of 150, it's faster :DAlbert Astals Cid1-2/+2
2011-09-18regtest: Do not create checksums for crashed, failed and stderr filesCarlos Garcia Campos1-1/+11
2011-09-18regtest: Fix checksum check for tests containing spaces in filenameCarlos Garcia Campos1-1/+1
2011-09-14Make it work with Python3Albert Astals Cid9-38/+38
2011-09-13regtest: Add a way to skip filesCarlos Garcia Campos4-5/+39
A new command line option --skip has been added to give a file with the list of test to skip. When --skip is not used, it look for a file named Skipped in the tests dir. Lines starting with '#' are considered comments and are ignored.
2011-09-13regtest: Limit the number of arguments to 1Carlos Garcia Campos3-26/+26
It's easier to run poppler-regtest more than once if you need to run different tests.
2011-09-13regtest: Sort tests before create-refs/run-tests and show progressCarlos Garcia Campos3-23/+53
2011-09-13Add initial poppler regressions test programCarlos Garcia Campos14-0/+1025
generated by cgit v1.2.3 (git 2.39.1) at 2024-05-25 18:14:25 +0000