Skip to content

OpenScienceComputing/esip2025

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

19 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

ESIP2025 Demo

  • download from Sciop
  • upload to Source.coop
  • visualize
  1. Install Coiled Install conda if not already installed
    conda create -n coiled coiled -y
    conda activate coiled
  2. Set a TOKEN that give you access to the ESIP-LAB Coiled account (for today only) using one of these options:
    export DASK_COILED__TOKEN=0d98de5376c148cfa1a42506cabb91b4-b680b641f13363e926760686e736175e59ecf1d5
    
    set DASK_COILED__TOKEN=0d98de5376c148cfa1a42506cabb91b4-b680b641f13363e926760686e736175e59ecf1d5
    
    coiled login --token 0d98de5376c148cfa1a42506cabb91b4-b680b641f13363e926760686e736175e59ecf1d5
    
  3. To start Jupyterlab for downloading and exploring data use this:
    coiled notebook start --region us-west-2 --vm-type m5.large --software esip2025 --workspace esip-lab --disk-size 50GB
  4. Open a terminal in Jupyterlab and type:
    git clone https://github.com/OpenScienceComputing/esip2025.git
    
  5. Run the notebooks

Creating your own repos on Source

  1. Request permission to create repos on Source.coop
  2. Create a repo and get the AWS credentials for the repo
  3. Specify the AWS credentials as environment variables
    export AWS_ACCESS_KEY_ID=xxxxxxxxxxxxxxx
    export AWS_SECRET_ACCESS_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
  4. Start Jupyterlab for downloading, exploring data and pushing to Source.coop:
    coiled notebook start --region us-west-2 --vm-type m5.large --software esip2025 --workspace esip-lab --disk-size 50GB --env AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID --env AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY --env AWS_REQUEST_CHECKSUM_CALCULATION=WHEN_REQUIRED 

Creating the Coiled software environment

coiled env create --name esip2025 --workspace esip-lab --conda environment.yml

About

Tools for moving data from Sciops (bittorrent) to Source.coop (AWS bucket)

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published