[WIP] New site: calypr.org (Reopen PR #41)

docs/.nav.yml

@@ -0,0 +1,4 @@
+nav:
+  - Home: index.md
+  - Calypr: calypr/
+  - Tools: tools/

docs/calypr/.nav.yml

@@ -0,0 +1,7 @@
+title: Calypr
+nav:
+  - Quick Start Guide: quick-start.md
+  - Data: data/
+  - Project Management: project-management/
+  - Analysis: analysis/
+  - Website: website/

docs/calypr/analysis/.nav.yml

@@ -0,0 +1,2 @@
+nav:
+  - Data Querying + Gen3 SDK: query.md

docs/workflows/query.md → docs/calypr/analysis/query.md

@@ -12,20 +12,20 @@ Gen3 supports API access to Files and Metadata, allowing users to download and q
 ## 1. Dependency and Credentials 
 
 
-Prior to installing, check a profile credentials. 
+Prior to querying, ensure your DRS remotes are configured.
 Test:
 ```bash
-g3t ping 
+git drs remote list
 ```
-- will return a list of projects that a profile has access to.
+- will return a list of configured DRS remotes and projects you have access to.
 
 - For new setup or renew of gen3 credentials - Follow steps to configure/re-configure a profile with credentials: 
 
-    - Download an API Key from the [Profile page](https://calypr.ohsu.edu.org/identity) and save it to `~/.gen3/credentials.json`
+    - Download an API Key from the [Profile page](https://calypr-public.ohsu.edu/Profile) and save it to `~/.gen3/credentials.json`
 
-    ![Gen3 Profile page](../images/api-key.png)
+    ![Gen3 Profile page](../../images/api-key.png)
 
-    ![Gen3 Credentials](../images/credentials.png)
+    ![Gen3 Credentials](../../images/credentials.png)
 
 
 ## 2. Install

docs/calypr/data/.nav.yml

@@ -0,0 +1,5 @@
+nav:
+  - FHIR for Researchers: introduction.md
+  - Adding FHIR Metadata: adding-metadata.md
+  - Managing Metadata: managing-metadata.md
+  - Integrating Your Data: integration.md

docs/workflows/metadata.md → docs/calypr/data/adding-metadata.md

@@ -1,22 +1,21 @@
 # Adding FHIR metadata
 
-
 ## Background
 
 Adding files to a project is a two-step process:
 
-1. Adding file metadata entries to the manifest (see [adding files](add-files.md))
+1. Adding file metadata entries to the manifest
 2. Creating FHIR-compliant metadata using the manifest
 
-This page will guide you through the second step of generating FHIR metadata in your g3t project. To understand the FHIR data model, see [FHIR for Researchers](../data-model/introduction.md)
+This page will guide you through the second step of generating FHIR metadata in your `git-drs` project. To understand the FHIR data model, see [FHIR for Researchers](introduction.md)
 
-## Generating FHIR Data using g3t
+## Generating FHIR Data using git-drs
 
-To submit metadata from the manifest to the platform, that metadata needs to be converted into FHIR standard. We will use the file metadata entries we had created during the `g3t add` on our data files.
+To submit metadata from the manifest to the platform, that metadata needs to be converted into FHIR standard. We will use the file metadata entries we had created during the `git drs add` on our data files.
 
 ### Creating metadata files using the manifest
 
-Using the file metadata entries created by the `g3t add` command, `g3t meta init` creates FHIR-compliant metadata files in the `META/` directory, where each file corresponds to a [FHIR resource](https://build.fhir.org/resourcelist.html). At a minimum, this directory will create:
+Using the file metadata entries created by the `git drs add` command, `forge meta init` creates FHIR-compliant metadata files in the `META/` directory, where each file corresponds to a [FHIR resource](https://build.fhir.org/resourcelist.html). At a minimum, this directory will create:
 
 | File                     | Contents                   |
 |--------------------------|----------------------------|
@@ -32,19 +31,18 @@ Depending on if a `patient` or `specimen` flag was specified, other resources ca
 * measurements (Observation)
 
 
-* This command will create a skeleton metadata file for each file added to the project using the `patient`, `specimen`, `task`, and/or `observation` flags specified by the `g3t add` command.  
+* This command will create a skeleton metadata file for each file added to the project using the `patient`, `specimen`, `task`, and/or `observation` flags specified by the `git drs add` command.  
 * You can edit the metadata to map additional fields.
 * The metadata files can be created at any time, but the system will validate them before the changes are committed.
 * **Note:** If an existing file is modified, it won't get automatically staged
   - For instance, if `DocumentReference.json` is already created and it has to be updated to reflect an additional file, this change is not automatically staged.
-  - Make sure to either `git add META/` or use the `-a` flag in `g3t commit` to ensure that your FHIR metadata changes are staged.
+  - Make sure to either `git add META/` to ensure that your FHIR metadata changes are staged.
 
 ### Example
 To add a cram file that's associated with a subject, sample, and particular task
 
 ```sh
-g3t add myfile.cram  --patient P0 --specimen P0-BoneMarrow --task_id P0-Sequencing
-g3t meta init
+git add myfile.cram  --patient P0 --specimen P0-BoneMarrow --task_id P0-Sequencing
 ```
 
 This will produce metadata with the following relationships:
@@ -54,8 +52,8 @@ This will produce metadata with the following relationships:
 When the project is committed, the system will validate new or changed records. You may validate the metadata on demand by:
 
 ```sh
-$ g3t  meta validate --help
-Usage: g3t meta validate [OPTIONS] DIRECTORY
+$ forge meta validate --help
+Usage: forge meta validate [OPTIONS] DIRECTORY
 
   Validate FHIR data in DIRECTORY.
 
@@ -73,16 +71,15 @@ All FHIR metadata is housed in the `META/` directory. The convention of using a
 
 ## Supplying your own FHIR metadata
 
-In some cases, it might be useful to supply your own FHIR metadata without using `g3t add` to create any file metadata. In that case, adding metadata would take on the following flow:
+In some cases, it might be useful to supply your own FHIR metadata without using `git drs add` to create any file metadata. In that case, adding metadata would take on the following flow:
 
 1. Initialize your project
 2. Copy external FHIR data as `.ndjson` files to your `META/` directory
 3. `git add META/`
-4. `g3t commit -m "supplying FHIR metadata"`
+4. `git commit -m "supplying FHIR metadata"`
 
 This process would be useful for individuals who want to use the system to track relations between metadata but might not necessarily want to connect their actual data files to the system.
 
 ## Next Steps
 
-* See the  <a href="/workflows/tabular/">tabular metadata section</a> for more information on working with metadata.
-* See the  <a href="/workflows/commit-push/">commit and push section</a> for more information on publishing.
+See the [data management section](managing-metadata.md) for more information on working with metadata and publishing.

docs/calypr/data/git-drs.md

@@ -0,0 +1,34 @@
+# Git-DRS
+
+!!! note
+    The tools listed here are under development and may be subject to change.
+
+## Overview
+
+Use case: As an analyst, in order to share data with collaborators, I need a way to create a project, upload files and associate those files with metadata. The system should be capable of adding files in an incremental manner.
+
+The following guide details the steps a data contributor must take to submit a project to the CALYPR data commons.
+
+### Core Concepts
+
+> In a Gen3 data commons, a semantic distinction is made between two types of data: "data files" and "metadata". [more](https://gen3.org/resources/user/dictionary/#understanding-data-representation-in-gen3)
+
+*   **Data File**: Information like tabulated data values in a spreadsheet or a fastq/bam file containing DNA sequences. The contents are not exposed to the API as queryable properties.
+*   **Metadata**: Variables that help to organize or convey additional information about corresponding data files so they can be queried.
+
+## 1. Setup
+
+CALYPR project management is handled using standard Git workflows. you will need the **Large File Storage (LFS)** plugin to track genomic data files and the **Git-DRS** plugin to interface with CALYPR's storage and indexing systems.
+
+Visit the [Quick Start Guide](../quick-start.md) for detailed, OS-specific installation instructions for these tools.
+
+| Tool | Purpose |
+| :--- | :--- |
+| **git-drs** | Manages large file tracking, storage, and DRS indexing. |
+| **forge** | Handles metadata validation, transformation (ETL), and publishing. |
+| **data-client** | Administrative tool for managing [collaborators and access requests](../../tools/data-client/access_requests.md). |
+{: .caption }
+
+## Git DRS Workflows
+
+For complete Git DRS documentation including project initialization, file management, and upload workflows, see the [Git DRS Quick Start](../../tools/git-drs/quickstart.md).

docs/data-model/integration.md → docs/calypr/data/integration.md

@@ -5,9 +5,9 @@ Converting tabular data (CSV, TSV, spreadsheet, database table) into FHIR (Fast
 
 As you create a upload files, you can tag them with identifiers which by default will create minimal, skeleton graph.
 
-You can retrieve that data using the g3t command line tool, and update the metadata to create a more complete graph representing your study.
+You can retrieve that data using the [git-drs](../../tools/git-drs/index.md) command line tool, and update the metadata using [forge](../../tools/forge/index.md) to create a more complete graph representing your study.
 
-You may choose to work with the data in it's "native" json format, or convert it to a tabular format for integration.  The system will re-convert tabular data back to json for submittal.
+You may choose to work with the data in its "native" JSON format, or convert it to a tabular format for integration. The system will re-convert tabular data back to JSON for submittal.
 
 The process of integrating your data into the graph involves several steps:
 
@@ -24,12 +24,12 @@ The process of integrating your data into the graph involves several steps:
     * Normalize Data: Split the spreadsheet data into FHIR-compliant resources.
 
 * Step 4: Utilize provided FHIR Tooling or Libraries
-    * FHIR Tooling: Use `g3t  meta dataframe ` and associated libraries to support data conversion and validation.
-    * Validation: Use `g3t  meta validate` to validate the transformed data against FHIR specifications to ensure compliance and accuracy.
+    * FHIR Tooling: Use `forge meta` and associated libraries to support data conversion and validation.
+    * Validation: Use `forge validate` to validate the transformed data against FHIR specifications to ensure compliance and accuracy.
 
 * Step 5: Import into FHIR-Compatible System
-    * Load Data: Use `g3t commit` to load the transformed data into the calypr system.
-    * Testing and Verification: Use `g3t push` to ensure your data appears correctly in the portal and analysis tools.
+    * Load Data: Use `git commit` and `git push` to manage your local data state.
+    * Testing and Verification: Ensure your data appears correctly in the portal and analysis tools after a successful push.
 
 * Step 6: Iterate and Refine
     * Review and Refine: Check for any discrepancies or issues during the import process. Refine the conversion process as needed.
@@ -76,7 +76,7 @@ Identifiers in FHIR references typically include the following components: [see
 
 > A string, typically numeric or alphanumeric, that is associated with a single object or entity within a given system. Typically, identifiers are used to connect content in resources to external content available in other frameworks or protocols.
 
-System: Indicates the system or namespace to which the identifier belongs. By default the namespace is `http://calypr.ohsu.edu.org/<project-id>`.
+System: Indicates the system or namespace to which the identifier belongs. By default the namespace is `http://calypr-public.ohsu.edu/<project-id>`.
 
 Value: The actual value of the identifier within the specified system. For instance, a lab controlled subject identifier or a specimen identifier.
 
@@ -109,4 +109,4 @@ By using identifiers in references, FHIR ensures that data can be accurately lin
 > A reference to a document of any kind for any purpose. [see more](https://hl7.org/fhir/documentreference.html)
 
 
-See the  <a href="/workflows/metadata/">metadata workflow section</a> for more information on how to create and upload metadata.
+See the [data management section](managing-metadata.md) for more information on how to create and upload metadata.

docs/data-model/introduction.md → docs/calypr/data/introduction.md

@@ -5,6 +5,12 @@ Given all of the intricacies healthcare and experimental data, we use Fast Healt
 
 ## What is FHIR?
 
+> In a Gen3 data commons, a semantic distinction is made between two types of data: "data files" and "metadata". [more](https://gen3.org/resources/user/dictionary/#understanding-data-representation-in-gen3)
+
+A "data file" could be information like tabulated data values in a spreadsheet or a fastq/bam file containing DNA sequences. The contents of the file are not exposed to the API as queryable properties, so the file must be downloaded to view its content.
+
+"Metadata" are variables that help to organize or convey additional information about corresponding data files so that they can be queried via the Gen3 data commons’ API or viewed in the Gen3 data commons’ data exploration tool. In a Gen3 data dictionary, variable names are termed "properties", and data contributors provide the values for these pre-defined properties in their data submissions.
+
 In an era where healthcare information is abundant yet diverse and often siloed, FHIR emerges as a standard, empowering research analysts to navigate, aggregate, and interpret health data seamlessly. This guide aims to unravel the intricacies of FHIR, equipping research analysts with the knowledge and tools needed to harness the potential of interoperable healthcare data for insightful analysis and impactful research outcomes in the context of CALYPR collaborations.
 
 ## Graph Model
@@ -21,7 +27,7 @@ The following "file focused" example illustrates how CALYPR uses FHIR resources
 
 Examine [resource](https://www.hl7.org/fhir/resource.html) definitions [here](http://www.hl7.org/fhir/resource.html):
 
-* Details on [uploaded files](https://calypr.github.io/workflows/upload/) are captured as [DocumentReference](http://www.hl7.org/fhir/documentreference.html)
+* Details on uploaded files are captured as [DocumentReference](http://www.hl7.org/fhir/documentreference.html)
 
 * DocumentReference.[subject](https://www.hl7.org/fhir/documentreference-definitions.html#DocumentReference.subject) indicates who or what the document is about:  
   * Can simply point to the [ResearchStudy](https://hl7.org/fhir/researchstudy.html), to indicate the file is part of the study
@@ -31,6 +37,4 @@ Examine [resource](https://www.hl7.org/fhir/resource.html) definitions [here](ht
 
 Each resource has at least one study controlled [official](https://hl7.org/fhir/codesystem-identifier-use.html#identifier-use-official) [Identifier](https://hl7.org/fhir/datatypes.html#Identifier).  Child resources have [Reference](http://www.hl7.org/fhir/references.html) fields to point to their parent.
 
-
 <img src="/images/fhir-graph-model.png" width="100%">
-