PUTKA (Preverjanje Učinkovitosti, Temeljitosti in Korektnosti Algoritmov) is an automated judge system for testing programs. It was originally conceived as an aid for conducting youth programming classes at ZRI Ljubljana (ZRI), but quickly gained support for competitions.
A publicly-accessible deployment of Putka with over 200 tasks is available at http://www.putka.si; the website is geared to everyone motivated to test their knowledge of solving algorithmic challenges. Registration and access to some basic tasks are enabled for everyone, but most other tasks and forum support are restricted to ZRI students.
Authors
Version 4.0
- User interface: Žiga Ham, Mitja Trampuš
- Testing environment: Jan Berčič, Mitja Trampuš
- Thanks go also to: dr. Jelko Urbančič and Nino Bašić (testing and feedback), Jure Ham (graphical design) and Andrej Veber (IT support, hosting)
History
The project started in 2004 as part of a programming summer school at ZRI Ljubljana. It has been rewritten from scratch multiple times, utilising experiences gained from earlier versions. These earlier versions also benefited from valuable contributions by Tomaž Hočevar, Andrej Bukošek, Domen Blenkuš, Peter Koželj and Igor Lalič.
References
- Putka has been used extensively in programming classes at ZRI Ljubljana since 2007.
- In the period from 2008 to 2011, it has been used to run the national programming competition, organised by the Association for Technical Culture of Slovenia (ZOTKS).
- The system has been tested through a large number of public and internal competitions.
- To date, the system has judged over 100.000 submissions.
Technical Characteristics
The system comprises three main components which can, as necessary, run on different host machines: the user interface (responsible for generating the user-facing website), testers (which run submitted programs in a secure controlled environment and grade them), and the manager (responsible for controlling the testers and dispatching test jobs to them). Owing to historical usage in many different contexts (programming classes, competitions with different rule systems), the system is very configurable and flexible.
- The user interface supports multiple languages, full ACL-based content access control and different competition rule systems.
- With a minimum of configuration, Putka supports "classic" tasks, tasks with multiple correct solutions, interactive tasks, manual task grading and more. The grading style is, in all details, specified by a test script - this is usually only one or two lines of code, but can be as complex as needed.
- The testers intercept all system calls during execution and filter them using language- and/or task-dependent rule sets.
