Skip to content

Performance refactorings#267

Open
ericpohl wants to merge 10 commits intoADAPT:developfrom
ericpohl:develop
Open

Performance refactorings#267
ericpohl wants to merge 10 commits intoADAPT:developfrom
ericpohl:develop

Conversation

@ericpohl
Copy link
Contributor

Cache intermediate results to avoid excessive file I/O and XML parsing. This is particularly noticeable when doing a quick read to get the metadata from a LoggedData object (grower/farm/field, operation type, crop, date etc).

Refactorings break down as follows:

  • Cache the directory listing in ExtensionMethods.GetDirectoryFiles. Currently GetDirectoryFiles enumerates all files on every call to find a single file and can be called many times for  ISOTimeLog.GetTimeElement (each looking up a TLGnnnnn.xml), plus additional calls for .bin files in  TimeLogMapper.ReadTimeLog and FillDPDGeometryDefinitions. Every call re-enumerates the same directory. Create a simple dictionary to hold previous results, and clear at the end of each plugin.Import() and plugin.Export() call.
  • Enumerate external XML files once in ISO11783_TaskData.ReadXML. ProcessExternalNodes is called for 15 different XML element prefixes. Each call potentially checks the directory for matching files, but most datasets only have external files for TSK.
  • Cache parsed ISOTime results from ISOTimeLog.GetTimeElement. The method is called multiple times from 4 different callers; each call re-reads and re-parses the same TLGnnnnn.xml file.
  • Build a mapping from DDI to Representation in Representationmapper. Currently RepresentationMapper.Map() does a linear LINQ scan over ~600+ representations on every DDI lookup. With many timelogs and 10-20 data elements each, it can be hundreds of scans.
  • Pre-index DeviceProperties by DDI in DeviceHierarchyElement to avoid scanning DeviceProperties multiple times for different DDI's.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant