Create guide on recommended tools for viewing and adding data

[nicopace]

Context & Goal

We need to create a comprehensive guide that helps users understand our recommended toolstack for interacting with data—specifically for viewing (querying/visualization) and adding (uploading/importing) data.

This guide will establish our "opinionated" stance on these tools, outlining where they excel, where they fall short, and how they fit into our ecosystem.

Proposed Guide Outline

How to "Get Data"

A high-level overview of the data retrieval flow and the general philosophy of how data moves through our system.

2Tool Recommendations & Comparison

We want to provide clear guidance on the tools we officially support or are highly opinionated about. This section should detail the strengths and weaknesses of each:

• Existing / Supported Tools:

  • Windmill: Our core engine for automated workflows and heavy lifting. We use it for Data Upload and Raw Database exploration.
  • Apache Superset: Great for heavy-duty business intelligence, dashboarding, and deep-dive visualization.

• Potential / Evaluated Tools:

  • Metabase: Highly user-friendly for non-technical users to query and visualize data. (Note: We should highlight how data cleaning fits in as a crucial prerequisite step before using Metabase).
  • Supabase: An excellent potential tool for managing backend data, offering an instant Postgres API and user management.

3. Clarifying "Adding Data"

We need to explicitly document how users should upload or import new datasets.

• The Windmill Dataset Importer: This is our primary mechanism, but the current implementation and documentation need an update.
• We must point users to the existing dataset uploader script and update its reference:
• Reference link to update: [Dataset Uploader Documentation](https://docs.guardianconnector.net/reference/gc-toolkit/gc-scripts-hub/dataset-uploader)

[rudokemper]

I wouldn't talk about Supabase, I don't think that's useful to anybody reading this.

We should also include tools that we might recommend that are not part of the stack. Like QGIS or GeoLibre for spatial data.