Skip to content

Add parquet merge and download functionality #160

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
axelstudios wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Add parquet merge and download functionality #160

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions .editorconfig
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Editor configuration, see https://editorconfig.org
root = true

[*]
charset = utf-8
indent_style = space
indent_size = 2
insert_final_newline = true
trim_trailing_whitespace = true

[*.ts]
quote_type = single

[*.md]
max_line_length = off
2 changes: 1 addition & 1 deletion .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,4 @@ _site/
vendor/

.DS_Store

/.idea/
8 changes: 5 additions & 3 deletions Gemfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
source 'https://rubygems.org'

gem "jekyll", "~> 4.3" # installed by `gem jekyll`
gem "jekyll", "~> 4.4.1" # installed by `gem jekyll`
# gem "webrick" # required when using Ruby >= 3 and Jekyll <= 4.2.2

gem "just-the-docs", "0.4.1" # pinned to the current release
# gem "just-the-docs" # always download the latest release
gem "jekyll-remote-theme", "~> 0.4.3"

gem "jekyll-seo-tag", "~> 2.8.0"
gem "jekyll-include-cache", "~> 0.2.1"
64 changes: 39 additions & 25 deletions Gemfile.lock
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,49 +1,60 @@
GEM
remote: https://rubygems.org/
specs:
addressable (2.8.6)
public_suffix (>= 2.0.2, < 6.0)
addressable (2.8.7)
public_suffix (>= 2.0.2, < 7.0)
base64 (0.2.0)
bigdecimal (3.1.9)
colorator (1.1.0)
concurrent-ruby (1.2.3)
concurrent-ruby (1.3.5)
csv (3.3.2)
em-websocket (0.5.3)
eventmachine (>= 0.12.9)
http_parser.rb (~> 0)
eventmachine (1.2.7)
ffi (1.16.3-x64-mingw-ucrt)
ffi (1.17.2-x64-mingw-ucrt)
forwardable-extended (2.6.0)
google-protobuf (4.26.1-x64-mingw-ucrt)
google-protobuf (4.30.2-x64-mingw-ucrt)
bigdecimal
rake (>= 13)
http_parser.rb (0.8.0)
i18n (1.14.4)
i18n (1.14.7)
concurrent-ruby (~> 1.0)
jekyll (4.3.3)
jekyll (4.4.1)
addressable (~> 2.4)
base64 (~> 0.2)
colorator (~> 1.0)
csv (~> 3.0)
em-websocket (~> 0.5)
i18n (~> 1.0)
jekyll-sass-converter (>= 2.0, < 4.0)
jekyll-watch (~> 2.0)
json (~> 2.6)
kramdown (~> 2.3, >= 2.3.1)
kramdown-parser-gfm (~> 1.0)
liquid (~> 4.0)
mercenary (>= 0.3.6, < 0.5)
mercenary (~> 0.3, >= 0.3.6)
pathutil (~> 0.9)
rouge (>= 3.0, < 5.0)
safe_yaml (~> 1.0)
terminal-table (>= 1.8, < 4.0)
webrick (~> 1.7)
jekyll-include-cache (0.2.1)
jekyll (>= 3.7, < 5.0)
jekyll-remote-theme (0.4.3)
addressable (~> 2.0)
jekyll (>= 3.5, < 5.0)
jekyll-sass-converter (>= 1.0, <= 3.0.0, != 2.0.0)
rubyzip (>= 1.3.0, < 3.0)
jekyll-sass-converter (3.0.0)
sass-embedded (~> 1.54)
jekyll-seo-tag (2.8.0)
jekyll (>= 3.8, < 5.0)
jekyll-watch (2.2.1)
listen (~> 3.0)
just-the-docs (0.4.1)
jekyll (>= 3.8.5)
jekyll-seo-tag (>= 2.0)
rake (>= 12.3.1)
kramdown (2.4.0)
rexml
json (2.10.2)
kramdown (2.5.1)
rexml (>= 3.3.9)
kramdown-parser-gfm (1.1.0)
kramdown (~> 2.0)
liquid (4.0.4)
Expand All @@ -53,27 +64,30 @@ GEM
mercenary (0.4.0)
pathutil (0.16.2)
forwardable-extended (~> 2.6)
public_suffix (5.0.4)
rake (13.1.0)
public_suffix (6.0.1)
rake (13.2.1)
rb-fsevent (0.11.2)
rb-inotify (0.10.1)
rb-inotify (0.11.1)
ffi (~> 1.0)
rexml (3.2.6)
rouge (4.2.1)
rexml (3.4.1)
rouge (4.5.1)
rubyzip (2.4.1)
safe_yaml (1.0.5)
sass-embedded (1.72.0-x64-mingw-ucrt)
google-protobuf (>= 3.25, < 5.0)
sass-embedded (1.86.3-x64-mingw-ucrt)
google-protobuf (~> 4.30)
terminal-table (3.0.2)
unicode-display_width (>= 1.1.1, < 3)
unicode-display_width (2.5.0)
webrick (1.8.1)
unicode-display_width (2.6.0)
webrick (1.9.1)

PLATFORMS
x64-mingw-ucrt

DEPENDENCIES
jekyll (~> 4.3)
just-the-docs (= 0.4.1)
jekyll (~> 4.4.1)
jekyll-include-cache (~> 0.2.1)
jekyll-remote-theme (~> 0.4.3)
jekyll-seo-tag (~> 2.8.0)

BUNDLED WITH
2.5.7
4 changes: 2 additions & 2 deletions _config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
title: ComStock
description: ComStock Documentation
# theme: just-the-docs
remote_theme: pmarsceill/just-the-docs
url: https://github.com/NREL/ComStock.github.io
logo: "/assets/images/ComStock_color_hort-transparent.png"
Expand Down Expand Up @@ -29,9 +28,10 @@ callouts:
othermedia:
color: grey-dk
opacity: 0.1
plugins:
- jekyll-remote-theme

# Google Analytics Tracking (optional)
# Supports a CSV of tracking ID strings (eg. "UA-1234567-89,G-1AB234CDE5")
ga_tracking: G-D7EWHC7BS0
ga_tracking_anonymize_ip: true # Use GDPR compliant Google Analytics settings (true/nil by default)

13 changes: 13 additions & 0 deletions assets/js/parquet-merge-CuJnH4Sl.js
View file
Open in desktop

Large diffs are not rendered by default.

12 changes: 10 additions & 2 deletions docs/resources/resources.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ This section provides lessons for understanding certain capabilities and functio
- [Perform an analysis by blending ComStock and local data]({{ site.baseurl }}{% link docs/resources/tutorials/local_segmentation_study.md %})

## How-to Guides
This section provides a collection of step-by-step guides for using the ComStock dataset to answer a given question.
This section provides a collection of step-by-step guides for using the ComStock dataset to answer a given question.

- [Access the ComStock datasets programmatically]({{ site.baseurl }}{% link docs/resources/how_to_guides/example_scripts.md %})<span class="label label-green">UPDATE</span>
- [Filter the building characteristics dashboard for a county]({{ site.baseurl }}{% link docs/resources/how_to_guides/characteristics_dashboard.md %})<span class="label label-blue">NEW</span>
Expand Down Expand Up @@ -48,6 +48,14 @@ These documents provide explanations focusing on the *how* and *why* of various
- [Metadata and Annual Results Aggregate File Discrepancy in 2024 Release 2]({{ site.baseurl }}{% link docs/resources/explanations/aggregate_file_discrepancy_known_issue.md %})<span class="label label-blue">NEW</span>


## Download and Merge Parquet Data
<div id="parquet-merge"></div>

{% assign bundle = site.static_files | where_exp:"f","f.name contains 'parquet-merge-'" | first %}

<script src="{{ bundle.path | relative_url }}"></script>


## References
These documents describe various aspects of ComStock, including the baseline and upgrade model documentation, as well as geographic clustering methodology.

Expand Down Expand Up @@ -96,4 +104,4 @@ systems, and the presence of cooking equipment to separate buildings into catego

**May 2024**

</details>
</details>