btrfs-progs: docs: clarify number represention in on-disk-format tables

Added 0x prefix to HEX numbers and transform some tables to new format.

Pull-request: #881
Signed-off-by: Yuwei Han <hrx@bupt.moe>
[ Fix RST grammar errors ]
Signed-off-by: Qu Wenruo <wqu@suse.com>

Author: hanyuwei70