v2.0.0 Analyze Schema Improvements

Overview

This version gets a major version bump because the schema for the sqlite database has some improvements, that involved changing the schema, so this is a "breaking change". If you have SQL queries or other integration based on analyze databases then you may need to update them when you upgrade from v1.X to v2.X versions.

This version does not itself contain major new features, but it has some important fixes and prepares for upcoming work on analyzing content directory builds. The change in #44 is a substantial size improvement n the analyze database. There are also improvements to how scenes dependencies are represented. And the code is better generalized for representing non-AssetBundle content. It is recommended to upgrade even if you are only analyzing AssetBundle builds.

What's Changed

• [#44] Normalize refs table to deduplicate property strings by @SkowronskiAndrew in #80
• [#82] Rename assets/asset_dependencies tables to reflect AssetBundle/PreloadData origin by @SkowronskiAndrew in #84
• [#81] Fix scene/preload dependencies for Player and Scriptable Build Pipeline builds by @SkowronskiAndrew in #87
• Document project structure; split Archive and SerializedFile into their own libraries by @SkowronskiAndrew in #88
• [#68] Generalize AssetBundle to Archive in analyze schema and code by @SkowronskiAndrew in #89
• [#55] Show type name in build_report_packed_asset_contents_view by @SkowronskiAndrew in #90

Schema differences

If you have queries or tooling built on a pre-2.0 analyze database, use the rename maps below to update them.

1. Table / view renames

Archive generalization (issue #68)

UnityDataTools now analyzes more than AssetBundles (Player, Content Archive, ContentDirectory builds), so AssetBundle-centric names for the generic archive concept were generalized. Names genuinely specific to the AssetBundle object were intentionally left unchanged.

Before	After
asset_bundles (table)	archives
serialized_files.asset_bundle	serialized_files.archive
object_view.asset_bundle (and *_asset_bundle aliases: material_/shader_/texture_/dep_)	object_view.archive (and *_archive)
in_bundles	in_archives
build_report_archive_contents.assetbundle	build_report_archive_contents.archive
build_report_archive_contents.assetbundle_content	build_report_archive_contents.archive_content

AssetBundle/PreloadData table clarification (issue #82)

The vaguely-named assets/asset_dependencies tables were renamed to reflect what actually populates them (the AssetBundle object's m_Container/m_PreloadTable and PreloadData). They are mostly empty for Player/ContentDirectory builds.

Before	After
assets (table)	assetbundle_assets
asset_view (view)	assetbundle_asset_view
asset_dependencies (table)	preload_dependencies
asset_dependencies_view (view)	preload_dependencies_view
index asset_dependencies_object	preload_dependencies_object
index asset_dependencies_dependency	preload_dependencies_dependency

Unchanged (genuinely AssetBundle-object-specific): assetbundle_assets, assetbundle_asset_view, preload_dependencies, AssetBundleHandler.

2. Human-readable types in build reports (issue #55)

• build_report_packed_asset_contents_view.type now shows the type name (e.g. Texture2D) instead of the numeric Class ID.
• The underlying build_report_packed_asset_info.type column still stores the numeric class id.

Full Changelog: v1.3.6...v2.0.0

v1.3.6 - UnityFileSystemApi update, find-refs fix, test coverage

What's Changed

• [#72] Add find-refs test coverage and fix its database/null-ref bugs by @SkowronskiAndrew in #79

• find-refs command fixed and now supports --stdout

• Update UnityFileSystemApi to 6000.6.0a7 so that UnityDataTools works with built output from Unity 6.6. It continues to support output from older versions of Unity.

• Testing: Reorganization of existing Unity project and add a second LeadingEdge Unity project that uses newer feature, e.g. serialized dictionaries and content directory builds. Resulting builds are added to source control and now used to test the dump and find-refs command. More usage of this test data will come in follow work.

Full Changelog: v1.3.5...v1.3.6

v1.3.5 - Usability improvements, initial Unity 6.6 support

What's Changed

Performance and CRC correctness

#70 includes a significant performance fix related to the calculation of CRCs. There are also correctness fixes for the CRC calculation. However investigation discovered another existing flaw with CRC that could be relevant when comparing objects between different analyze databases, That will require its own separate fix, tracked in #74 .

Usability improvements

• Improve dump command: add --stdout, refactors, more tests by @SkowronskiAndrew in #69
• [#49] Improve analyze file specification, CLI help and docs improvements by @SkowronskiAndrew in #76
• [#77] Detect missing TypeTrees up front in analyze and dump by @SkowronskiAndrew in #78

The support for "--stdout" makes it easier to use the dump command interactively and for piping in scripts.

The new support for file and path specification for analyze makes it easier to analyze individual files, and to analyze Build Report files along with the build output.

Initial support for content directory builds

New type being introduced in Unity 6.6. More support will come in follow up releases.

• [#70] Fix analyze CRC for cah:/ ContentDirectory resources; add --skip-crc by @SkowronskiAndrew in #73
• Document ContentLayout.json and publish its schema by @SkowronskiAndrew in #75

Documentation improvement

• Docs: Addressables local→remote dependency example by @timt-unity3d in #67
• Also doc improvements included in #77

Compatibility

These changes should be pretty backward compatible for existing scripts and tools based on past version of UnityDataTool.
However the fixes for CRC included in #70 can mean that the calculated CRC values for objects will now be different from the value calculated on the same objects in the older version of UnityDataTool.

Full Changelog: v1.3.4...v1.3.5

v1.3.4

What's Changed

• Add type filtering to dump command (Issue #52) by @Ijtihed in #63
• Improve archive Command and update UnityFileSystemAPI by @SkowronskiAndrew in #62
• Remove unnecessary files from the release zip files (Issue #61)

New Contributors

• @Ijtihed made their first contribution in #63

Full Changelog: v1.3.3...v1.3.4

v1.3.3 - Support for Extracted TypeTrees

What's Changed

• Added support for specifying an extracted type tree data file #60. This permits use of UnityDataTool to analyze the output of the new feature for reducing build size (available starting in Unity 6.5 + Addressables 2.9). See Addressable AssetBundle memory considerations for details.
• Update UnityFileSystem (dlls, so and dylib) to the version shipped with 6000.5.0a8 - this version supports version 23 of the serialized file format and has the API needed to load extracted type tree files.

New Contributors

• @paulb-unity made their first contribution in #60

Full Changelog: 1.3.2...v1.3.3

1.3.2

What's Changed

• serializedFile metadata command (Unity version, platform and typetree info) #59 The "-f json" version prints out a lot more details than the text version.
• serializedfile objectlist and externalrefs subcommands now work for files that don't have typetrees. Rather than loading the file with UnityFileSystem the serialized file is parsed directly.
• Clearer error messages for Analyze in some cases when UnityDataTool fails to open a file because it does not have TypeTrees.
• New UnityBinaryFormat assembly with the code to recognize Unity file formats and to parse the header/metadata section of a binary SerializedFile

Full Changelog: v1.3.1...1.3.2

v1.3.1

What's Changed

• Improve Error Handling with Serialized File Detection by #58
  • Archives, YAML files detected with clear error messages
  • Reliable way to detect if a file is a Unity SerializedFile despite lack of a distinctive file extension or "magic prefix"
  • Added serialized-file header subcommand for looking at the header of serialized files
  • Added testing and examples of different versions of the serialized file format.
• Addressables BuildLayout.json Analyze fixes by #56
• Fix error handling and reporting in analyze command (Issue #48) by #57

Full Changelog: v1.3.0...v1.3.1

v1.3.0

What's Changed

• Analyze command: MonoScript and BuildReport support
• New serialized-file command
• Doc improvements

See #46 for details of new functionality.

Full Changelog: v1.2.1...v1.3.0

v1.2.1

This release has the same functionality as 1.2.0

But it contains a fix for an SQL statement that was discovered when fixing the unit tests, and that might also cause problems when using the v1.2.0 build. See #39 for details.

It also includes improved test coverage, and cleanup in the source code white space and minor "style" cleanup.

v1.2.0

This release adds support to the analyze command to reading Addressables Build Layout files.

See Aaddressables Build Reports for details.