summaryrefslogtreecommitdiff
path: root/regtest/TestRun.py
AgeCommit message (Collapse)AuthorFilesLines
2014-11-07regtest: Update references also for tests that no longer crashCarlos Garcia Campos1-0/+2
We are currently updating the refs only when we have checksums to compare, but if a test used to crash there's no md5 file in refs.
2014-10-26regtest: Limit the number of worker threads to the number of documents to testCarlos Garcia Campos1-9/+18
We are always spawning all the threads even if the documents to test is less than the worker threads. Also optimize the case of running only one test to not spwn any thread.
2014-10-26regtest: Allow to run groups of tests individuallyCarlos Garcia Campos1-2/+24
Now it's possible to pass more than one argument to run-tests command and optionally the docs directory. When more than one test is passed and the docs directory is not provided, the common base path of all passed tests is used as docs directory. The tests passed can be documents or directories, using absolute paths or paths relative to the docs directory. This also allows us to update the refs for a group of tests.
2013-12-15regtest: Return an exist status code depending on whether the command succeededCarlos Garcia Campos1-0/+2
2013-12-06regtest: Show also the tests expected to crash/fail to run but don't failCarlos Garcia Campos1-1/+7
2013-12-06regtest: Improve readability of test resultsCarlos Garcia Campos1-20/+47
Show a summary of tests failed per backend with the percentages and use a new line for every test in the result instead of using a comma separated line.
2013-11-27regtest: Do not consider docs with no refs as skippedCarlos Garcia Campos1-17/+23
Handle them differently as new docs, and show them in the summary suggesting to use create-refs command to include them in the test suite.
2013-11-27regtest: Use number of tests run to show the progressCarlos Garcia Campos1-62/+74
Instead of using the number of document tested. We don't really need to set a number to a particular document, we only want to know the progress of the whole process. This ensures that the progress is shown in the correct order. It also simplifies the code a bit. Also improved the output formatting to make it easier to read.
2012-12-29regtest: Print test results when the test has finishedCarlos Garcia Campos1-9/+9
To make sure the result corresponds to the test now that we are using multiple threads.
2012-12-08regtest: Remove trailing whitespacesCarlos Garcia Campos1-5/+5
2012-12-06implement parallel testing using Python's Queue classAdam Reichold1-12/+46
2012-10-21regstest: Add print_test_result_ln() and use it for failed test resultsCarlos Garcia Campos1-5/+5
To make sure they are always shown.
2012-09-23regtest: Reduce the noise of the default output when running testsCarlos Garcia Campos1-17/+18
Show permanent information only about failed tests, without the details about the failing pages. Previous verbose output is available passing --verbose command line output.
2012-03-10regtest: Add information about skipped tests in the summaryCarlos Garcia Campos1-3/+9
2011-11-01regtest: Add --update-refs command line option to run-tests commandCarlos Garcia Campos1-1/+1
It allows to update the references of failing tests.
2011-09-14Make it work with Python3Albert Astals Cid1-16/+16
2011-09-13regtest: Add a way to skip filesCarlos Garcia Campos1-2/+7
A new command line option --skip has been added to give a file with the list of test to skip. When --skip is not used, it look for a file named Skipped in the tests dir. Lines starting with '#' are considered comments and are ignored.
2011-09-13regtest: Sort tests before create-refs/run-tests and show progressCarlos Garcia Campos1-11/+10
2011-09-13Add initial poppler regressions test programCarlos Garcia Campos1-0/+156