ntfs_forensics tables ntfs_indx_data and ntfs_file_data not returning results

[mike-myers-tob]

observed behavior:

osquery> SELECT * FROM ntfs_part_data;
+--------------------+---------+------------------------------+
| device             | address | description                  |
+--------------------+---------+------------------------------+
| \\.\PhysicalDrive0 | 0       | Safety Table                 |
| \\.\PhysicalDrive0 | 1       | Unallocated                  |
| \\.\PhysicalDrive0 | 2       | GPT Header                   |
| \\.\PhysicalDrive0 | 3       | Partition Table              |
| \\.\PhysicalDrive0 | 4       | Basic data partition         |
| \\.\PhysicalDrive0 | 5       | EFI system partition         |
| \\.\PhysicalDrive0 | 6       | Microsoft reserved partition |
| \\.\PhysicalDrive0 | 7       | Basic data partition         |
| \\.\PhysicalDrive0 | 8       | Unallocated                  |
+--------------------+---------+------------------------------+
osquery> SELECT * FROM ntfs_indx_data WHERE device="\\.\PhysicalDrive0" AND partition=4;
osquery> SELECT * FROM ntfs_indx_data WHERE device="\\.\PhysicalDrive0" AND partition=4 AND parent_path="\";
unable to open file \
osquery> SELECT * FROM ntfs_indx_data WHERE device="\\.\PhysicalDrive0" AND partition=5 AND parent_path="\";
unable to open file \
osquery> SELECT * FROM ntfs_indx_data WHERE device="\\.\PhysicalDrive0" AND partition=7 AND parent_path="\";
unable to open file \
osquery> SELECT * FROM ntfs_indx_data WHERE device="\\.\PhysicalDrive0" AND partition=7 AND parent_path="/";

expected behavior: shown in example

osquery version: the release binary from this repo
Name and version of OS tested on: Windows 10

[alessandrogario]

I have improved constraint handling for that table, and it is now capable of listing all disks and partitions in case there is no device or partition condition in the WHERE clause.

The table now only needs one mandatory parameter (either parent_inode or parent_path) rather than three and will print an error if it is missing (first query was being silently discarded). Sadly, I could only print it to the status log (LOG(WARNING) << ...), but in the future we could expand the custom error messages feature we implemented for writable extension tables to the ::generate() methods as well.

The other queries didn't work because it seems like specifying the root path as a constraint is not yet implemented.

[GarretReece]

Digging into the original issue, the cause appears to be GUID Partition Tables--Windows 10 Pro uses MBR, which Sleuth Kit handles just fine, but Windows 10 Enterprise uses GPT and that doesn't work as well. Fortunately it seems that SleuthKit can still open the file system just fine if we use the raw image and offset to the file system, rather than attempting to open the partition.

[GarretReece]

Debugging messages when attempting to open the partitions one at a time on the system:

osquery> select * from ntfs_file_data WHERE path="/ProgramData/osquery"
    ...> ;
W0803 15:53:50.953084 17900 ntfsfileinfotable.cpp:186] unable to open filesystem: Invalid magic value (Not a NTFS file system (magic))
W0803 15:53:50.959982 17900 ntfsfileinfotable.cpp:186] unable to open filesystem: Invalid magic value (Not a NTFS file system (invalid sector size 0)))
W0803 15:53:50.965960 17900 ntfsfileinfotable.cpp:186] unable to open filesystem: Invalid magic value (Not a NTFS file system (invalid sector size 0)))
W0803 15:53:50.972040 17900 ntfsfileinfotable.cpp:186] unable to open filesystem: Invalid magic value (Not a NTFS file system (magic))
W0803 15:53:50.977958 17900 ntfsfileinfotable.cpp:186] unable to open filesystem: Invalid magic value (Not a NTFS file system (magic))
W0803 15:53:50.993963 17900 ntfsfileinfotable.cpp:186] unable to open filesystem: Invalid magic value (Not a NTFS file system (invalid MFT entry size))
W0803 15:53:51.000965 17900 ntfsfileinfotable.cpp:186] unable to open filesystem: Invalid magic value (Not a NTFS file system (magic))
W0803 15:53:51.006969 17900 ntfsfileinfotable.cpp:186] unable to open filesystem: Invalid magic value (Not a NTFS file system (volume size is 0))

Further info:

$> .\mmls.exe -B \\.\PhysicalDrive0
GUID Partition Table (EFI)
Offset Sector: 0
Units are in 512-byte sectors

      Slot      Start        End          Length       Size    Description
000:  Meta      0000000000   0000000000   0000000001   0512B   Safety Table
001:  -------   0000000000   0000002047   0000002048   1024K   Unallocated
002:  Meta      0000000001   0000000001   0000000001   0512B   GPT Header
003:  Meta      0000000002   0000000033   0000000032   0016K   Partition Table
004:  000       0000002048   0000616447   0000614400   0300M   Basic data partition
005:  001       0000616448   0001640447   0001024000   0500M   EFI system partition
006:  002       0001640448   0001902591   0000262144   0128M   Microsoft reserved partition
007:  003       0001902592   0500117503   0498214912   0237G   Basic data partition
008:  -------   0500117504   0500118191   0000000688   0344K   Unallocated

$> .\fsstat.exe -o 0001902592 \\.\PhysicalDrive0
Cannot determine file system type

Manual inspection of the GPT verified that the displayed values match the on-disk contents.

[mike-myers-tob]

What is needed to close this issue? Is it a limitation of SleuthKit that we should document, and file an issue upstream with them?

[alessandrogario]

@GarretReece Any update on this issue? I'm moving this back to the Up next for the time being