Skip to content

Commit 5677951

Browse files
enryHsayalaruano
enryH
and
sayalaruano
authored
Check that report files are unchanged (for now in streamlit) (#132)
* ✨ add current main state of streamlit report Python files * 🙈 hide unformatted html files for now (could use prettier etc before writing to file) * ✨ use git diff to detect changes (should not fail atm) * 🎨 sort for now only src and docs folder * 🐛 only one folder supported translated to run `isort --check-only --diff src` * 🐛 fix import in home section ( no duplication and sorted) * 🐛 sort imports so it's not depend on random (set), statements after imports * 🙈 show changes to streamlit folders unter tests/report_examples * 🐛 some paths are not yet relative... * 🎨 do not format tests folder for now * 🐛 fix path (which is still absolute) * 🐛 Correct relative paths for excel dataframes * ⏪️ Revert changes of excel telative pahs to avoid CICD error --------- Co-authored-by: sayalaruano <sebasar1245@gmail.com>
1 parent f90e4a5 commit 5677951

File tree

14 files changed

+555
-8
lines changed

14 files changed

+555
-8
lines changed

‎.github/workflows/cdci.yml‎

Lines changed: 16 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,8 +20,12 @@ jobs:
2020
- uses: actions/checkout@v4
2121
- uses: psf/black@stable
2222
with:
23-
jupyter: true
23+
# ! for now (format output files of vuegen for streamlit)
24+
src: "./src"
2425
- uses: isort/isort-action@v1
26+
with:
27+
# ! should be removed once all imports are correctly sorted
28+
sort-paths: src
2529
- name: Set up Python ${{ matrix.python-version }}
2630
uses: actions/setup-python@v5
2731
with:
@@ -67,6 +71,7 @@ jobs:
6771
run: |
6872
cd docs
6973
vuegen -dir example_data/Earth_microbiome_vuegen_demo_notebook
74+
# here we are testing that the generated config from the -dir call works
7075
vuegen -c example_data/Earth_microbiome_vuegen_demo_notebook/Earth_microbiome_vuegen_demo_notebook_config.yaml
7176
# repeat for easier inspection on GitHub:
7277
- name: quarto html report
@@ -104,6 +109,16 @@ jobs:
104109
cd docs
105110
vuegen -dir example_data/Earth_microbiome_vuegen_demo_notebook -rt jupyter
106111
vuegen -c example_data/Earth_microbiome_vuegen_demo_notebook/Earth_microbiome_vuegen_demo_notebook_config.yaml -rt jupyter
112+
- name: check for changes in report files
113+
run: |
114+
# write streamlit report to test folder
115+
vuegen -dir docs/example_data/Basic_example_vuegen_demo_notebook -output_dir tests/report_examples/Basic_example_vuegen_demo_notebook
116+
# Check for changes
117+
if git diff tests/report_examples | grep .; then
118+
echo "Error: One or more protected files have been modified."
119+
exit 1
120+
fi
121+
107122
108123
publish:
109124
name: Publish package to PyPI

‎.gitignore‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -116,7 +116,8 @@ cython_debug/
116116
# Temporary files
117117
logs/
118118
vuegen/logs/
119-
streamlit_report/
119+
./streamlit_report/
120+
!tests/report_examples
120121
quarto_report/
121122
output_docker/
122123

@@ -129,4 +130,4 @@ docs/images/Graphical_abstract/
129130
docs/images/Nfcore_module_figure
130131
docs/presentations/
131132
basic_example_vuegen_demo_notebook_config.yaml
132-
earth_microbiome_vuegen_demo_notebook_config.yaml
133+
earth_microbiome_vuegen_demo_notebook_config.yaml

‎src/vuegen/streamlit_reportview.py‎

Lines changed: 14 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,13 @@
99

1010
from . import report as r
1111
from . import table_utils
12-
from .utils import create_folder, generate_footer, get_relative_file_path, is_url
12+
from .utils import (
13+
create_folder,
14+
generate_footer,
15+
get_relative_file_path,
16+
is_url,
17+
sort_imports,
18+
)
1319
from .utils.variables import make_valid_identifier
1420

1521

@@ -368,7 +374,6 @@ def _generate_home_section(
368374
all_components, subsection_imports, _ = self._combine_components(
369375
home_section.components
370376
)
371-
372377
try:
373378
# Create folder for the home page
374379
home_dir_path = Path(output_dir) / "Home"
@@ -381,9 +386,11 @@ def _generate_home_section(
381386

382387
# Create the home page content
383388
home_content = []
384-
home_content.append("import streamlit as st")
385-
if subsection_imports:
386-
home_content.extend(subsection_imports)
389+
subsection_imports.append("import streamlit as st")
390+
subsection_imports = set(subsection_imports)
391+
subsection_imports, _ = sort_imports(subsection_imports)
392+
393+
home_content.extend(subsection_imports)
387394
if self.report.description:
388395
home_content.append(
389396
self._format_text(text=self.report.description, type="paragraph")
@@ -517,6 +524,8 @@ def _combine_components(self, components: list[dict]) -> tuple[list, list, bool]
517524
all_contents.extend(content)
518525
# remove duplicates
519526
all_imports = list(set(all_imports))
527+
all_imports, setup_statements = sort_imports(all_imports)
528+
all_imports.extend(setup_statements)
520529
return all_contents, all_imports, has_chatbot
521530

522531
def _generate_subsection(self, subsection) -> tuple[List[str], List[str]]:

‎tests/report_examples/.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
*.html

‎tests/report_examples/Basic_example_vuegen_demo_notebook/streamlit_report/sections/Dataframes/All_Formats.py‎

Lines changed: 146 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,146 @@
1+
from st_aggrid import AgGrid, GridOptionsBuilder
2+
from vuegen import table_utils
3+
import pandas as pd
4+
import streamlit as st
5+
df_index = 1
6+
7+
st.markdown('''<h3 style='text-align: center; color: #023558;'>All Formats</h3>''', unsafe_allow_html=True)
8+
st.markdown('''<h4 style='text-align: center; color: #2b8cbe;'>Phyla Correlation Network Csv</h4>''', unsafe_allow_html=True)
9+
df = pd.read_csv('docs/example_data/Basic_example_vuegen_demo_notebook/2_Dataframes/1_All_formats/1_phyla_correlation_network_csv.csv')
10+
11+
12+
# Displays a DataFrame using AgGrid with configurable options.
13+
grid_builder = GridOptionsBuilder.from_dataframe(df)
14+
grid_builder.configure_default_column(editable=True, groupable=True, filter=True)
15+
grid_builder.configure_side_bar(filters_panel=True, columns_panel=True)
16+
grid_builder.configure_selection(selection_mode="multiple")
17+
grid_builder.configure_pagination(enabled=True, paginationAutoPageSize=False, paginationPageSize=20)
18+
grid_options = grid_builder.build()
19+
20+
AgGrid(df, gridOptions=grid_options, enable_enterprise_modules=True)
21+
22+
# Button to download the df
23+
df_csv = df.to_csv(sep=',', header=True, index=False).encode('utf-8')
24+
st.download_button(
25+
label="Download dataframe as CSV",
26+
data=df_csv,
27+
file_name=f"dataframe_{df_index}.csv",
28+
mime='text/csv',
29+
key=f"download_button_{df_index}")
30+
df_index += 1
31+
st.markdown('''<h4 style='text-align: center; color: #2b8cbe;'>Abundance Table Example Xls</h4>''', unsafe_allow_html=True)
32+
selected_sheet = 0
33+
sheet_names = table_utils.get_sheet_names("/home/runner/work/vuegen/vuegen/docs/example_data/Basic_example_vuegen_demo_notebook/2_Dataframes/1_All_formats/2_abundance_table_example_xls.xls")
34+
selected_sheet = st.selectbox("Select a sheet to display", options=sheet_names)
35+
36+
df = pd.read_excel('/home/runner/work/vuegen/vuegen/docs/example_data/Basic_example_vuegen_demo_notebook/2_Dataframes/1_All_formats/2_abundance_table_example_xls.xls', sheet_name=selected_sheet)
37+
38+
39+
# Displays a DataFrame using AgGrid with configurable options.
40+
grid_builder = GridOptionsBuilder.from_dataframe(df)
41+
grid_builder.configure_default_column(editable=True, groupable=True, filter=True)
42+
grid_builder.configure_side_bar(filters_panel=True, columns_panel=True)
43+
grid_builder.configure_selection(selection_mode="multiple")
44+
grid_builder.configure_pagination(enabled=True, paginationAutoPageSize=False, paginationPageSize=20)
45+
grid_options = grid_builder.build()
46+
47+
AgGrid(df, gridOptions=grid_options, enable_enterprise_modules=True)
48+
49+
# Button to download the df
50+
df_csv = df.to_csv(sep=',', header=True, index=False).encode('utf-8')
51+
st.download_button(
52+
label="Download dataframe as CSV",
53+
data=df_csv,
54+
file_name=f"dataframe_{df_index}.csv",
55+
mime='text/csv',
56+
key=f"download_button_{df_index}")
57+
df_index += 1
58+
st.markdown('''<h4 style='text-align: center; color: #2b8cbe;'>Sample Info Example Txt</h4>''', unsafe_allow_html=True)
59+
df = pd.read_table('docs/example_data/Basic_example_vuegen_demo_notebook/2_Dataframes/1_All_formats/3_sample_info_example_txt.txt')
60+
61+
62+
# Displays a DataFrame using AgGrid with configurable options.
63+
grid_builder = GridOptionsBuilder.from_dataframe(df)
64+
grid_builder.configure_default_column(editable=True, groupable=True, filter=True)
65+
grid_builder.configure_side_bar(filters_panel=True, columns_panel=True)
66+
grid_builder.configure_selection(selection_mode="multiple")
67+
grid_builder.configure_pagination(enabled=True, paginationAutoPageSize=False, paginationPageSize=20)
68+
grid_options = grid_builder.build()
69+
70+
AgGrid(df, gridOptions=grid_options, enable_enterprise_modules=True)
71+
72+
# Button to download the df
73+
df_csv = df.to_csv(sep=',', header=True, index=False).encode('utf-8')
74+
st.download_button(
75+
label="Download dataframe as CSV",
76+
data=df_csv,
77+
file_name=f"dataframe_{df_index}.csv",
78+
mime='text/csv',
79+
key=f"download_button_{df_index}")
80+
df_index += 1
81+
st.markdown('''<h4 style='text-align: center; color: #2b8cbe;'>Sample Info Example Parquet</h4>''', unsafe_allow_html=True)
82+
df = pd.read_parquet('docs/example_data/Basic_example_vuegen_demo_notebook/2_Dataframes/1_All_formats/4_sample_info_example_parquet.parquet')
83+
84+
85+
# Displays a DataFrame using AgGrid with configurable options.
86+
grid_builder = GridOptionsBuilder.from_dataframe(df)
87+
grid_builder.configure_default_column(editable=True, groupable=True, filter=True)
88+
grid_builder.configure_side_bar(filters_panel=True, columns_panel=True)
89+
grid_builder.configure_selection(selection_mode="multiple")
90+
grid_builder.configure_pagination(enabled=True, paginationAutoPageSize=False, paginationPageSize=20)
91+
grid_options = grid_builder.build()
92+
93+
AgGrid(df, gridOptions=grid_options, enable_enterprise_modules=True)
94+
95+
# Button to download the df
96+
df_csv = df.to_csv(sep=',', header=True, index=False).encode('utf-8')
97+
st.download_button(
98+
label="Download dataframe as CSV",
99+
data=df_csv,
100+
file_name=f"dataframe_{df_index}.csv",
101+
mime='text/csv',
102+
key=f"download_button_{df_index}")
103+
df_index += 1
104+
st.markdown('''<h4 style='text-align: center; color: #2b8cbe;'>Example Xlsx</h4>''', unsafe_allow_html=True)
105+
selected_sheet = 0
106+
df = pd.read_excel('/home/runner/work/vuegen/vuegen/docs/example_data/Basic_example_vuegen_demo_notebook/2_Dataframes/1_All_formats/5_example_xlsx.xlsx', sheet_name=selected_sheet)
107+
108+
109+
# Displays a DataFrame using AgGrid with configurable options.
110+
grid_builder = GridOptionsBuilder.from_dataframe(df)
111+
grid_builder.configure_default_column(editable=True, groupable=True, filter=True)
112+
grid_builder.configure_side_bar(filters_panel=True, columns_panel=True)
113+
grid_builder.configure_selection(selection_mode="multiple")
114+
grid_builder.configure_pagination(enabled=True, paginationAutoPageSize=False, paginationPageSize=20)
115+
grid_options = grid_builder.build()
116+
117+
AgGrid(df, gridOptions=grid_options, enable_enterprise_modules=True)
118+
119+
# Button to download the df
120+
df_csv = df.to_csv(sep=',', header=True, index=False).encode('utf-8')
121+
st.download_button(
122+
label="Download dataframe as CSV",
123+
data=df_csv,
124+
file_name=f"dataframe_{df_index}.csv",
125+
mime='text/csv',
126+
key=f"download_button_{df_index}")
127+
df_index += 1
128+
footer = '''<style type="text/css">
129+
.footer {
130+
position: relative;
131+
left: 0;
132+
width: 100%;
133+
text-align: center;
134+
}
135+
</style>
136+
<footer class="footer">
137+
This report was generated with
138+
<a href="https://github.com/Multiomics-Analytics-Group/vuegen" target="_blank">
139+
<img src="https://raw.githubusercontent.com/Multiomics-Analytics-Group/vuegen/main/docs/images/vuegen_logo.svg" alt="VueGen" width="65px">
140+
</a>
141+
| Copyright 2025 <a href="https://github.com/Multiomics-Analytics-Group" target="_blank">
142+
Multiomics Network Analytics Group (MoNA)
143+
</a>
144+
</footer>'''
145+
146+
st.markdown(footer, unsafe_allow_html=True)

‎tests/report_examples/Basic_example_vuegen_demo_notebook/streamlit_report/sections/Home/Homepage.py‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
import requests
2+
import streamlit as st
3+
st.markdown('''<p style='text-align: center; color: #000000;'>A general description of the report.
4+
</p>''', unsafe_allow_html=True)
5+
st.markdown('''<h4 style='text-align: center; color: #2b8cbe;'>Description</h4>''', unsafe_allow_html=True)
6+
7+
with open('docs/example_data/Basic_example_vuegen_demo_notebook/description.md', 'r') as markdown_file:
8+
markdown_content = markdown_file.read()
9+
10+
st.markdown(markdown_content, unsafe_allow_html=True)
11+
12+
footer = '''<style type="text/css">
13+
.footer {
14+
position: relative;
15+
left: 0;
16+
width: 100%;
17+
text-align: center;
18+
}
19+
</style>
20+
<footer class="footer">
21+
This report was generated with
22+
<a href="https://github.com/Multiomics-Analytics-Group/vuegen" target="_blank">
23+
<img src="https://raw.githubusercontent.com/Multiomics-Analytics-Group/vuegen/main/docs/images/vuegen_logo.svg" alt="VueGen" width="65px">
24+
</a>
25+
| Copyright 2025 <a href="https://github.com/Multiomics-Analytics-Group" target="_blank">
26+
Multiomics Network Analytics Group (MoNA)
27+
</a>
28+
</footer>'''
29+
30+
st.markdown(footer, unsafe_allow_html=True)

‎tests/report_examples/Basic_example_vuegen_demo_notebook/streamlit_report/sections/Html/All_Html.py‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
import requests
2+
import streamlit as st
3+
4+
st.markdown('''<h3 style='text-align: center; color: #023558;'>All Html</h3>''', unsafe_allow_html=True)
5+
st.markdown('''<h4 style='text-align: center; color: #2b8cbe;'>Plot</h4>''', unsafe_allow_html=True)
6+
7+
with open('docs/example_data/Basic_example_vuegen_demo_notebook/4_Html/1_All_html/1_plot.html', 'r', encoding='utf-8') as html_file:
8+
html_content = html_file.read()
9+
10+
st.components.v1.html(html_content, height=600, scrolling=True)
11+
12+
st.markdown('''<h4 style='text-align: center; color: #2b8cbe;'>Ckg Network</h4>''', unsafe_allow_html=True)
13+
14+
with open('docs/example_data/Basic_example_vuegen_demo_notebook/4_Html/1_All_html/2_ckg_network.html', 'r') as html_file:
15+
html_content = html_file.read()
16+
17+
18+
st.markdown(f"<p style='text-align: center; color: black;'> <b>Number of nodes:</b> 33 </p>", unsafe_allow_html=True)
19+
st.markdown(f"<p style='text-align: center; color: black;'> <b>Number of relationships:</b> 35 </p>", unsafe_allow_html=True)
20+
21+
# Streamlit checkbox for controlling the layout
22+
control_layout = st.checkbox('Add panel to control layout', value=True)
23+
net_html_height = 1200 if control_layout else 630
24+
# Load HTML into HTML component for display on Streamlit
25+
st.components.v1.html(html_content, height=net_html_height)
26+
27+
st.markdown('''<h4 style='text-align: center; color: #2b8cbe;'>Multiqc Report</h4>''', unsafe_allow_html=True)
28+
29+
with open('docs/example_data/Basic_example_vuegen_demo_notebook/4_Html/1_All_html/3_multiqc_report.html', 'r', encoding='utf-8') as html_file:
30+
html_content = html_file.read()
31+
32+
st.components.v1.html(html_content, height=600, scrolling=True)
33+
34+
footer = '''<style type="text/css">
35+
.footer {
36+
position: relative;
37+
left: 0;
38+
width: 100%;
39+
text-align: center;
40+
}
41+
</style>
42+
<footer class="footer">
43+
This report was generated with
44+
<a href="https://github.com/Multiomics-Analytics-Group/vuegen" target="_blank">
45+
<img src="https://raw.githubusercontent.com/Multiomics-Analytics-Group/vuegen/main/docs/images/vuegen_logo.svg" alt="VueGen" width="65px">
46+
</a>
47+
| Copyright 2025 <a href="https://github.com/Multiomics-Analytics-Group" target="_blank">
48+
Multiomics Network Analytics Group (MoNA)
49+
</a>
50+
</footer>'''
51+
52+
st.markdown(footer, unsafe_allow_html=True)

‎tests/report_examples/Basic_example_vuegen_demo_notebook/streamlit_report/sections/Markdown/All_Markdown.py‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
import requests
2+
import streamlit as st
3+
4+
st.markdown('''<h3 style='text-align: center; color: #023558;'>All Markdown</h3>''', unsafe_allow_html=True)
5+
st.markdown('''<h4 style='text-align: center; color: #2b8cbe;'>Readme</h4>''', unsafe_allow_html=True)
6+
7+
with open('docs/example_data/Basic_example_vuegen_demo_notebook/5_Markdown/1_All_markdown/README.md', 'r') as markdown_file:
8+
markdown_content = markdown_file.read()
9+
10+
st.markdown(markdown_content, unsafe_allow_html=True)
11+
12+
footer = '''<style type="text/css">
13+
.footer {
14+
position: relative;
15+
left: 0;
16+
width: 100%;
17+
text-align: center;
18+
}
19+
</style>
20+
<footer class="footer">
21+
This report was generated with
22+
<a href="https://github.com/Multiomics-Analytics-Group/vuegen" target="_blank">
23+
<img src="https://raw.githubusercontent.com/Multiomics-Analytics-Group/vuegen/main/docs/images/vuegen_logo.svg" alt="VueGen" width="65px">
24+
</a>
25+
| Copyright 2025 <a href="https://github.com/Multiomics-Analytics-Group" target="_blank">
26+
Multiomics Network Analytics Group (MoNA)
27+
</a>
28+
</footer>'''
29+
30+
st.markdown(footer, unsafe_allow_html=True)

0 commit comments

Comments
 (0)