Changelog and Versions
For reference, all previous releases of EUTelescope are listed here. Note that all versions below the section “unmaintained” are considered outdated and will no longer be supported.
- GBL: General Broken Lines can now be used for alignment and track fitting. An example with steering files (datura-noDUT) is included.
- Improved processor output verbosity: instead of every processor creating arbitrary amounts of console output every X events, you can now easily control the verbosity using the PrintEventNumber processor (see examples).
- Optimized code efficiency, especially in histogram filling routines
- New tool for plotting the histograms produced during your analysis: Using “pyroplot” (Python ROOT plotter, located in tools/pyroplot) you can easily generate a single pdf file containing all or a selection of histograms from arbitrary root files or even compare histograms between files. See the tools/pyroplot/README and online documentation for details. NOTE: due to its dependency on rootpy, pyroplot requires Python 2.6 or later.
- includes fixes for treatment of non-zero suppressed data, provided by M. Kofarago (see issue https://eutelescope.web.cern.ch/issue/parts-code-treating-non-zero-suppressed-data-have-bugs)
- includes fix for hotpixel removal in clustering step, provided by M. Kofarago (see issue https://eutelescope.web.cern.ch/issue/hotpixel-killer-doesnt-work-if-called-dut)
- fixes edge issue in APIX clustering processor as suggested by J. Lange (see https://eutelescope.web.cern.ch/forum/missing-hits-row-0-dut-after-offline-reco)
- fixes bug in ExhaustiveTrackFinde affecting number of reconstructed tracks.
- EUTelPreAlignment: Fix bug, FixedPlane parameter didn’t accept anything else than zero.
- ApplyAlignment: fixed bug with ‘missing’ reference vector collection when use of refvec was explicitly disabled; this also removes any warnings of missing reference vectors for this case.
- GBL tracking bugfix
- Change CMSPixelDecoder external reference to the new github repository.
- fix crash due to non-existing reference hit vector in EUTelMille
- fix for concatenation function for Python versions < 2.6; fixes issue https://eutelescope.web.cern.ch/issue/concatenation-older-python reported by C. Nellist
- minor modifications and updates to some READMEs
- Improved histogram output (binning, description, axis labels)
- EUTelMille now allows to include custom pede commands into the generated pede steering file directly via processor option.
- New jobsub example for using ATLAS FEI4 as DUT in Anemone telescope
- EUTelHitmaker now assigns uncertainties to hits (taken from information in GEAR file for now)
- moved external libraries to ./external
- updated and added examples (using jobsub framework)
- New ApplyAlignment processor (EUTelProcessorApplyAlignment, needed for 6D alignment using GBL)
- use reference vectors for coordinate transformations by default to ensure consistency between processors
- removed obsolete pysub scripts (see notes on updating)
- Code cleanup: fixed many compiler warnings/errors
- Code cleanup: removed obsolete code and unused processor parameters
- pysub (a job submission script collection used as default in EUTelescope versions prior to 0.8) has been removed for being obsolete: its functionality was replaced by the more flexible and user-friendlier jobsub script. See the jobsub README file for information on how to transfer your analysis templates to jobsub.
- the reference vector collection initialized in the hitmaker step is now used for coordinate transformation calculations by default in all processors (options “UseReferenceCollection” and “ApplyToReferenceCollection”); if, for some reason, you disabled this feature explicitly, please make sure that this is done consistently throughout your analysis. This should not affect you if e.g. you based your analysis on any recent examples.
- changed verbosity: many processors do not print information on every event anymore. This is now done through the PrintEventNumber processor (see “datura-noDUT” example)
- some processor options have changed or have been added. Update them through “Marlin -u” command and search for any parameter that is prefixed with the comment “<!– Sorry, this parameter isn’t a default parameter for this processor: description and type lost!! –>”. Such parameters are either renamed (e.g. for clarity) or have been removed altogether from the processor. In the former case you should search for the parameter under its new name in the updated steering file to adjust its setting. In the latter case you can just safely remove the parameter from your steering file.
For up-to-date details, check http://eutelescope.web.cern.ch/project/issues
- EUTelMille: AlignMode = 1 does not allow to use the ExcludedPlanes parameter; for details see http://eutelescope.web.cern.ch/issue/excluding-planes-alignment-does-not-work
- Clustering: with MessageLevel set to DEBUG, the clustering step crashes; for details see http://eutelescope.web.cern.ch/issue/seg-fault-clustering-step-message-level-debug
- DafFitter: known issues with excluding planes, see http://eutelescope.web.cern.ch/issue/daffitter-excluding-planes
- The fitter/analysis step of the datura-noDUT example requires more than 3GB of memory to be available to the Marlin process; sufficient physical RAM and a 64bit kernel are therefore required to run the full analysis chain as provided.
- jobsub: running jobsub directly (i.e. using ‘./jobsub’ and not ‘python jobsub’) requires the symbolic link ‘python2’ to be present in the PATH; this is set up on most systems by default and is needed to ensure that the right interpreter is being called on systems with both python 2.X and 3.X installed.
- GBL: the processors using GBL are in an early stage and might still be a bit “rough” around the edges; please report any issues you encounter in the forums or file a bug report.
- GBL: Applying “6D” alignment (with 6 degrees of freedom) requires using the EUTelApplyAlignmentProcessor together with a reference vector collection (created through EUTelHitmaker and loaded before applying the alignment) for consistent treatment of alignment shifts and rotations. The 3D alignment (x,y,xy), however, can be applied using with or without reference vector collections.
- SPEED: histogram filling is not very efficiently handled in some processors; this issue is already partially addressed in this release. Still, when you are batch processing many files and do not care about the histograms produced in intermediate steps, consider to switch the histogram filling off via processor parameters in the steering files. For details see http://eutelescope.web.cern.ch/issue/very-inefficient-histogram-handling-processors
- SPEED and STORAGE EFFICIENCY: EUTelClusteringProcessor creates “original_zsdata” collections for Mimosa planes which are required by some processors and therefore cannot be easily removed; this leads to (storage) overhead, for details see http://eutelescope.web.cern.ch/issue/data-duplication-output-eutelclusteringprocessor
- Maintenance and stability release in preparation for version 1.0.
- New lean and flexible job submission tool jobsub.
- All examples now use CTest framework for automated regression tests.
- Includes processors for CMS Pixel chips (PSI46xxx).
- Updated and improved documentation including examples.
- jobsub: fixes loading of the argparse module provided when using Python versions older than 2.7
- jobsub: fixes backtrace on certain errors with Python versions < 2.7
- fixed some typos in documentation
- New PedeSteeringAdditionalCmds parameter in EUTelMille: allows to add arbitrary parameters to the generated pede steering file
- Updated example analysis datura-noDUT
- Support for ILCSoft installer > 01-17-02
- jobsub now linked to ./bin directory and therefore present in PATH
- Fixed name definition of reference hit collection in EUTelHitmaker
- Fixed bug in jobsub concerning case-sensitivity for command line supplied options
- Added user-requested concatenation feature to jobsub
- Fixed memory leaks in EUTelTestFitter and EUTelHotPixelKiller
- Fixed sensor id reporting in EUTelDUTHistograms
- Fixed option parameter parsing in jobsub
- Reduced CPU load of jobsub during Marlin output parsing
- Run ranges passed to jobsub are no longer processed out-of-order
- Jobsub uses line buffered output (reduces output delays) if stdbuf tool is available
- Fixed compilation on newer GCC versions.
- Improved error handling and documentation of jobsub.
- Removed legacy code, fixed various bugs and improved code quality (e.g. fixed compiler warnings).
- Revised producer console output messages and verbosity.
- Updated and improved documentation.
For up-to-date details, check http://eutelescope.web.cern.ch/project/issues
- EXAMPLE datura-noDUT: this example is based on an analysis in progress which uses some features only available in the development version (trunk) of EUTelescope. The results produced with these steering files therefore differ slightly between releases.
- SPEED: histogram filling is not very efficiently handled in some processors; when you are batch processing many files and do not care about the histograms produced in intermediate steps, consider to switch the histogram filling off via processor parameters in the steering files. For details see http://eutelescope.web.cern.ch/issue/very-inefficient-histogram-handling-processors
- SPEED and STORAGE EFFICIENCY: EUTelClusteringProcessor creates “original_zsdata” collections for Mimosa planes which are required by some processors and therefore cannot be easily removed; this leads to (storage) overhead, for details see http://eutelescope.web.cern.ch/issue/data-duplication-output-eutelclusteringprocessor
- Update for the installation documentation.
- Recommended ilcsoft release v01-14-01.
- Update to the APIXTbTrackTuple dumper.
- The root tuple now contains information about the angles at which tracks enter the DUT plane.
- Major update to the EUTelMille processor. Multiple fixes turned out to be a major rewrite of the processor.
- The 3D shift + 3 angle rotation alignment has been validated in the ATLAS IBL testbeams.
- Performance tuned for semi-automatic usage, however, one has to be carefull with the alignment level residual cuts and sensor resolution values set in the align-tmp.xml template, the Millepede II module (pede) is very sensitive to both. Major changes to the active plane identification. This was formerly done by a hit z-coordinate proximity to the gear file nominal position of the plane. It does not work well for planes with high tilts realtive to the beam and it does not work at all for sensors sharing the same z-coordinate (due to the specific hardware setup). There are two methods which have been implemented to address this issue. The first one is based on the hit-to-cluster-to-PlaneID lookup (works only for data clusters). The second one is based on linear algebra of finding a shortest distance from a hit to the plane (as it was defined in the gear file). For the track fitted hits only the latter approach is applicable.
- An obsolete LCIO method set IsReferencePointPCA is commented out for compatibility with new ILCSoft release (v01-12)
- Minor: 2D histograms added to EUTelDafFitter.
- Inconsistency with the GEAR library found! Use only GEAR v01-17, where the sensor (plane) tilts are treated properly. If you are using ilcsoft v01-11, then the GEAR library v01-17 is included.
- New processor EUTelAPIXHotPixelKiller is introduced which dumps hot pixels into an event-like record. This information is used in PreAlignment (hitmaker step) and alignment steps. The hot pixels are not excluded from any event record, only a mask condition is applied to prevent the hotpixels from entering (pre)-alignment pixel selection. Otherwise the logic of the processors did not change.
- Several fixes to the PreAlign processor, increased range of the histogram-arrays to allow larger displacement between the sensors.
- Installation documentation fixed.
- Recommended installation path consists of using ilcinstall scripts from version v01-11 and installing all packages for ilcsoft release v01-10. The ilcinstall release v01-11 contains improved CMake scripts assisting with the installation/compilation of EUDAQ and Millepede II. The ilcsoft version v01-10 should be used as much as possible untill there is a crucial bug fix or classes logic upgrade, which is incompatible with other packages in v01-10.
- New CMake approach implemented for ilcsoft v01-11 release. It is still backward compatible with ilcsoft v01-10.
- EUTelTestFitter fixed for a tilted sensor with the true z-coordinate of the fitted hit.
- Deterministic Annealing Filter (DAF fitter) introduced.
- Caution: new EUDAQ and Millepede II reference rules in the ilcinstall script.
- The installation guide page updated accordingly.
- New CMake approach implemented.
- Bug fix in the clustering (EUTelClusteringProcessor:DFF).
- Redefine the source of the center of gravity of a cluster (EUTelHitMaker)
- Intermediate pre-release
- Includes (merges) with branch “engels” to adopt new CMake scheme.
- Small improvements to the full_analysis_chain scripts.
- Bug fixes in the EUTelDUTHistograms class.
- Extend the template “template_ibl/fitter-tmp.xml” to have Telescope sensors (6x Mimosa 26) and a set of DUT sensors (APIX as an example).
- DUTHistograms done in one execution step.
- New release of the ILCSoft v01-10.
- Dependency on the latest GEAR library (!) after 30.11.2010.
- Three rotation angles added. Can work with tilted sensors now.
- Fix the python scripts to be able to run on GRID.
- Bug fix in the correlator processor to be able to manage sensors with any pitch size (also different in X and Y) and rotation (this time only in XY plane), also sensor flip.
- Intermediate release.
- Intermediate release.
- Final optimisations to the EUTelTestFitter by Filip Zarnecki, for more details see his talk at the EUDET-JRA1 meeting: EUDET-JRA1 TestFitter analytical fit.
- EUTelTestFitter updates by Filip Zarnecki, fixes are related to the non zero beam slope and the new UseSlope logic, significant performance improvement bug fix in the correlator processor due to sometimes confusing sensor addressing (by ID, by Z position, by order in the gear file).
- Changed the safety factor in the ApplyAlignment step to 1.0, otherwise the memory allocation fails.
- Major update. Introduce a (version of a) correlation band calculator. It is used to get an estimate of the sensors offset. To be applied in the HitMaker in order to have all sensors “preAligned”. It makes life for the Millepede II (called in the EUTelMille) easier, since it is supposed to work anyway only with small shift/rotations.
- Bug fix in EUTelMille: do not implement the estimated Z position (set the difference to = 0), otherwise the alignment constants could (rarely) get very large.
- Bug fix in the zsFixedFrame: now able to cope with initially empty status collections.
- Update DEPFET.
- A final decoding for S3Bsystem and for test version of DCD readout has been implemented.
- Bug fix, uncomment one of the chi2 safety checks in the EUTelTestFitter processor in order to maintain previous functionality.
- Add few more correlation histograms to the EUTelCorrelator processor.
- Few major changes in the data processing flow: Running the HotPixel processor in the converter step: this has to be done on special Off-Beam runs. It dumps the hotpixel collection into a dedicated db file (slcio format), the hitpixel-db.slcio is read at the clustering step of the hot pixel map and reduces the combinatorics by a large fraction (the amount depends on the threshold which defines a pixel being hot or not).
- Several control cards are added to the fitter processor, they are: UseSlope - turn on the optimisation for the track candidates search. SlopeDistanceMax - defines a tube, which is parallel to axis Z, to reduce the combinatorics of all possible hit combinations when trying to form a new track candidate. SlopeXLimit, SlopeYLimit - define the allowed deviations between the hit pairs constituting a track candidate.
- Several bug fixes: variable initialisation issues.
- Optimisation of the status collection (speedup for the Digital Fixed Frame Clustering). Fix of the cluster center position in the HitMaker, it has no effect on DFF, was buggy for the SparseClustering2 (SP2). The hit coordinates are expected to be identical for small clusters (smaller than defined by the DFF algorithm input settings). In principle the SP2 clustering algorithm could be used in some cases as the default one for M26 sensors.
- Millepede II based alignment processor fix introduced to be compatible with the official Millepede II versions. Backward compatibility is kept.
- Minor documentation fix.
- Millepede II based alignment processor extended to align tilted sensors (full 6D gears, 3 coordinate shifts and 3 angle rotations).
- Added APIX sensors description and clustering.
- Added FORTIS sensors description.
- Several minor updates: GRID submition scripts do not require pedestal runs any more (ERROR downgraded to a WARNING).
- Default configuration file config.cfg updated
- New: an example gear file for a CERN June 2010 APIX/FORTIS hardware setup.
- Several bug fixes.
- Documentation update.
- Capability to run the same processors on user devices too.
- Python based submission scripts.
- Histogramming package for a better and easier results visualization.
- Several minor updates.
- New: templates for the python scripts to analyse digital sensors (Mimosa 26).
- A gear file for the Mimosa-26 CERN September 2009 setup updated (only 6 M26 sensors).