Form Bio Platform Documentation
Form Bio Platform Documentation
🛠️

Form Bio CLI SDK

Form Bio CLI/SDK

formbio is a high-level command line tool for managing Form Bio workflows and projects.

⚠️
The Form Bio CLI SDK is under active development and subject to change.

Installing and Updating the formbio CLI tool

To install the latest version of the formbio CLI tool or update your existing installation with the latest you can run the following depending on your operating system.

Windows


# Install via PowerShell 
iwr https://storage.googleapis.com/formbio-go-releases/public/latest/install/windows | iex

MacOS

# Install via command line
curl -sSL https://storage.googleapis.com/formbio-go-releases/public/latest/install/macos | bash

sudo mkdir -p /usr/local/bin/

# Make it available on your $PATH (only need to do this the first time you install)
sudo ln -s $HOME/.local/bin/formbio /usr/local/bin/formbio

Linux

# Install via command line
curl -sSL https://storage.googleapis.com/formbio-go-releases/public/latest/install/linux | bash

# Make it available on your $PATH (only need to do this the first time you install)
sudo ln -s $HOME/.local/bin/formbio /usr/local/bin/formbio

Commands

‣
formbio --help
$ formbio --help
manages Formbio workflows, storage and resources.

Usage:
  formbio [command]

Available Commands:
  auth        Runs authentication subcommands
  completion  Generate the autocompletion script for the specified shell
  help        Help about any command
  storage     Runs storage subcommands
  version     Version will output the current build information
  workflow    Runs workflow subcommands

Flags:
      --config string   Select your config file (default is $HOME/.formbio.yaml).
      --debug           Print debug log lines in the CLI console.
  -h, --help            help for formbio

Use "formbio [command] --help" for more information about a command.
‣
formbio auth --help 
$ formbio auth --help
Runs authentication subcommands

Usage:
  formbio auth [command]

Available Commands:
  login       Authenticates with the Form Bio platform for running CLI commands
  logout      Logs the current user out
  whoami      Prints the current user's email

Flags:
  -h, --help   help for auth

Global Flags:
      --config string   Select your config file (default is $HOME/.formbio.yaml).
      --debug           Print debug log lines in the CLI console.

Use "formbio auth [command] --help" for more information about a command.
‣
formbio storage --help
$ formbio storage --help
Runs storage subcommands

Usage:
  formbio storage [command]

Available Commands:
  cp          Copies and object or tree of objects
  ls          Lists the stored objects for a specific project

Flags:
  -h, --help   help for storage

Global Flags:
      --config string   Select your config file (default is $HOME/.formbio.yaml).
      --debug           Print debug log lines in the CLI console.

Use "formbio storage [command] --help" for more information about a command.
‣
formbio workflow run --help
$ formbio workflow run --help
Runs a Nextflow workflow through the Form Bio platform

Usage:
  formbio workflow run --org org --project project --repo repo --version version --run-name runName  -- [--paramName <param-value>] [--paramName2 <param-value>] [flags]

Examples:
formbio workflow run --org=org-production --project=projectA --repo=formbio/nf-examplepipeline --version=main --run-name="testInputs" -- --input "formbio://org-slug/project-slug/sequencing" --batchFile "formbio://org-slug/project-slug/design.batch"

Flags:
      --config string                  If --workflow-container is true, a stringified JSON object containing container config values (e.g. '{"inputs": ["filePath1"], "outputs": ["outputs/"]}')
      --config-file string             If --workflow-container is true, the location of a .json file containing container-config options
  -h, --help                           help for run
      --nextflow-params string         Nextflow pipeline params JSON e.g. --nextflow-params='{"input": "gs://test-bucket/test.txt", "batchInput": "gs://test-bucket/test.batch"}'
      --nextflow-version string        The version of Nextflow that you want to use for your workflow run.
      --org string                     The slug of the organization you are accessing.
      --project string                 The slug of the project you are accessing.
      --repo string                    The repository you want to source for your Nextflow script (e.g. formbio/nf-proteingetinfo).
      --require-schema-validation      If set to true, a workflow run will not be started unless the inputs can be validated against the workflow schema. (default true)
      --run-name string                The name of the pipeline run that it will be indexed under.
      --set-default-inputs             If set to true, empty workflow inputs will be auto-filled with default values (if available). (default true)
      --skip-workflow-results string   Skip running workflow, copy provided workflow results to output folder. Must be in "org://project/workflow_run_id" format.
      --timeout duration               Timeout for running Nextflow on google life sciences. (default 720h0m0s)
      --version string                 Github branch or tag to run (e.g. main or v1.0.2).
      --workflow string                The name of the workflow you are running.
      --workflow-container             If set to true, workflow container will run directly rather than use an uploaded workflow. Must include --config or --config-file

Global Flags:
      --debug   Print debug log lines in the CLI console.

Authentication

Login

‣
formbio auth login --help
$ formbio auth login --help
Authenticates with the Form Bio platform for running CLI commands

Usage:
  formbio auth login [flags]

Flags:
  -h, --help   help for login

Global Flags:
      --config string   Select your config file (default is $HOME/.formbio.yaml).
      --debug           Print debug log lines in the CLI console.

Example usage:

# logs the user in 
formbio auth login
‣
Output
Authenticating Form Bio CLI...
Log in at https://form.bio/auth/login-token/, then copy/paste your Form Bio CLI API Token below...
Enter the CLI API Token:


// User's browser will open to the Form Bio platform where they can
// 1. Log in with email and password
// 2. Log in with Google Signin

// Once user pastes in the CLI API Token, they will see this message:
Authenticated as [user@email.com]
To switch users, use:
        formbio auth logout
# logs the user in when they are already authenticated
formbio auth login
‣
Output
Already authenticated as [user@email.com] - to switch users, use:
        formbio auth logout

Logout

‣
formbio auth logout --help
$ formbio auth logout --help
Logs the current user out

Usage:
  formbio auth logout [flags]

Flags:
      --env string   The Bantha environment you want to access [local|dev|sbx|stg|prod].
  -h, --help         help for logout

Global Flags:
      --config string   Select your config file (default is $HOME/.formbio.yaml).
      --debug           Print debug log lines in the CLI console.

Example usage:

# logs the user out
formbio auth logout
‣
Output
You have successfully logged out.

Whoami

‣
formbio auth whoami --help
$ formbio auth whoami --help
Prints the current user's email

Usage:
  formbio auth whoami [flags]

Aliases:
  whoami, who

Flags:
      --env string   The Bantha environment you want to access [local|dev|sbx|stg|prod].
  -h, --help         help for whoami

Global Flags:
      --config string   Select your config file (default is $HOME/.formbio.yaml).
      --debug           Print debug log lines in the CLI console.

Example usage:

# list currently authenticated user
formbio auth whoami
‣
Output
You are currently authenticated as Form Bio user user@email.com

Data Upload and Download

List

‣
formbio storage ls --help
$ formbio storage ls --help
Lists the stored objects for a specific organization and project. The path should be in the format "formbio://<organization>/<project>/<prefix>".

Usage:
  formbio storage ls [-r | --recursive] <path> [flags]

Examples:
  # list the root level objects for a project
  formbio storage ls --env=prod formbio://my-organization/my-project/

  # recursively list the objects in a specific directory
  formbio storage ls --env=prod -r formbio://my-organization/my-project/workflow-input-files/

Flags:
      --env string   The Bantha environment you want to access [local|dev|sbx|stg|prod].
  -h, --help         help for ls
  -r, --recursive    List objects recursively, default is non-recursive.

Global Flags:
      --config string   Select your config file (default is $HOME/.formbio.yaml).
      --debug           Print debug log lines in the CLI console.

Example usage:

# list files in top level of project space
formbio storage ls \
   formbio://my-organization/my-project/
‣
Output
Authenticating on the "prod" environment
Authenticated as [my_user@formbio.com] on the prod environment
Looking up stored objects

total objects: 3

formbio://my-organization/my-project/:
            2022-06-23T23:54:20Z  formbio://my-organization/my-project/logs/
            2022-10-07T05:18:53Z  formbio://my-organization/my-project/my_file.txt
# list files recursively in the "logs" directory
formbio storage ls -r \
   formbio://my-organization/my-project/logs/
‣
Output
Authenticating on the "prod" environment
Authenticated as [my_user@formbio.com] on the prod environment
Looking up stored objects

total objects: 2

formbio://my-organization/my-project/logs/:
                                  formbio://my-organization/my-project/logs/end/

formbio://my-organization/my-project/logs/end/:
                                  formbio://my-organization/my-project/logs/end/of/

formbio://my-organization/my-project/logs/end/of/:
                                  formbio://my-organization/my-project/logs/end/of/the/

formbio://my-organization/my-project/logs/end/of/the/:
                                  formbio://my-organization/my-project/logs/end/of/the/rainbow/

formbio://my-organization/my-project/logs/end/of/the/rainbow/:
    338361  2022-10-07T05:21:09Z  formbio://my-organization/my-project/logs/end/of/the/rainbow/pot-of-gold.png

Upload / Download

‣
formbio storage cp --help
Copies and object or tree of objects from source to target, used for downloading, uploading, and copying, where source and target can be either local paths or formbio URLs in the format "formbio://<organization>/<project>/<prefix>".

Usage:
  formbio storage cp [-r | --recursive] [-c | --continue] <source> <target> [flags]

Examples:
  # upload single file to a directory
  formbio storage cp --env=prod myfile.txt formbio://my-organization/my-project/

  # download a directory into the current local directory
  formbio storage cp --env=prod -r formbio://my-organization/my-project/workflow-input-files/ .

  # copy a directory from one project into another
  formbio storage cp --env=prod -r formbio://my-organization/project-a/my-dataset/ formbio://my-organization/project-b/my-dataset/

Flags:
  -c, --continue     To resume an uploaded previously partially completed, use the continue flag.
      --env string   The Bantha environment you want to access [local|dev|sbx|stg|prod].
  -h, --help         help for cp
  -r, --recursive    To copy a tree of objects, specify a recursive copy.

Global Flags:
      --config string   Select your config file (default is $HOME/.formbio.yaml).
      --debug           Print debug log lines in the CLI console.

Example usage:

# download a single file from the project into mylocaldir
formbio storage cp \
  formbio://my-organization/my-project/logs/my_file.txt \
  mylocaldir/
‣
Output
Authenticating on the "prod" environment
Authenticated as [user@formbio.com] on the prod environment
Starting download

Preparing Download   █████████████████████████████████████████████████ 100% | 0s
 SUCCESS  logs/my_file.txt                                                                                                                                                                                                  
DownloadingFiles     █████████████████████████████████████████████████ 100% | 0s

Files Downloaded:      1/1
Total Data Downloaded: 104 KiB/104 KiB
Download complete
# upload a single file to the project root
formbio storage cp \
  mylocaldir/my_file.txt \
  formbio://my-organization/my-project/
‣
Output
Authenticating on the "prod" environment
Authenticated as [user@formbio.com] on the prod environment
Starting upload

Preparing Upload     █████████████████████████████████████████████████ 100% | 1s
 SUCCESS  formbio://my-organization/my-project/my_file.txt
Uploading Files      █████████████████████████████████████████████████ 100% | 1s

Files Uploaded:      1/1
Total Data Uploaded: 104 KiB/104 KiB
Upload complete
# upload a local directory to a project subdirectory
formbio storage cp -r \
  mylocaldir \
  formbio://my-organization/my-project/subdir/
‣
Output
Authenticating on the "prod" environment
Authenticated as [user@formbio.com] on the prod environment
Starting upload

Preparing Upload     █████████████████████████████████████████████████ 100% | 2s
 SUCCESS  formbio://my-organization/my-project/logs/subdir/mylocaldir/my_file1.txt
 SUCCESS  formbio://my-organization/my-project/logs/subdir/mylocaldir/my_file2.txt
Uploading Files      █████████████████████████████████████████████████ 100% | 1s

Files Uploaded:      2/2
Total Data Uploaded: 209 KiB/209 KiB
Upload complete
# resume a partially completed upload
formbio storage cp -c \
  one_gig_file.txt \
  formbio://my-organization/my-project/large-files-dir/
‣
Output
Authenticating on the "prod" environment
Authenticated as [user@formbio.com] on the prod environment
Starting upload

Preparing Upload     █████████████████████████████████████████████████ 100% | 1s
Uploading Files      ███████████████████████████████████████            79% | 0s
# downloads files that match the wildcard
# workaround for wildcard support which will come soon:
# formbio storage cp formbio://path/*.txt .
formbio storage ls \
  formbio://my-organization/my-project/logs | grep txt |rev| cut -f 1 -d ' ' | rev | awk '{print "formbio storage cp",$1,"."}' |sh
‣
Output
 SUCCESS  ./example.txt                                                                                                                                            
Downloading    ███████████████████████████████████████████████████████ 100% | 1s

Files Downloaded:      1/1
Total Data Downloaded: 5.00 MiB/5.00 MiB

 SUCCESS  ./example2.txt                                                                                                                                        
Downloading    ███████████████████████████████████████████████████████ 100% | 0s

Files Downloaded:      1/1
Total Data Downloaded: 9 B/9 B

Create Directory

# use the -p (--parents) flag to create several nested directories at once
formbio storage mkdir \
    -p \
    formbio://my-organization/my-project/existing_dir/new_dir/also_new
‣
Output
Authenticating on the "prod" environment
Authenticated as [user@formbio.com] on the prod environment
Successfully created new directory

Run Workflows on Form Bio

Launch workflow run

‣
formbio workflow run --help
$ ./bin/formbio workflow run --help
Runs a Nextflow workflow through the Form Bio platform

Usage:
  formbio workflow run --org org --project project --repo repo --version version --run-name runName  -- [--paramName <param-value>] [--paramName2 <param-value>] [flags]

Examples:
formbio workflow run --org=org-production --project=projectA --repo=formbio/nf-examplepipeline --version=main --run-name="testInputs" -- --input "formbio://org-slug/project-slug/sequencing" --batchFile "formbio://org-slug/project-slug/design.batch"

Flags:
      --config string                  If --workflow-container is true, a stringified JSON object containing container config values (e.g. '{"inputs": ["filePath1"], "outputs": ["outputs/"]}')
      --config-file string             If --workflow-container is true, the location of a .json file containing container-config options
  -h, --help                           help for run
      --nextflow-params string         Nextflow pipeline params JSON e.g. --nextflow-params='{"input": "gs://test-bucket/test.txt", "batchInput": "gs://test-bucket/test.batch"}'
      --nextflow-version string        The version of Nextflow that you want to use for your workflow run.
      --org string                     The slug of the organization you are accessing.
      --project string                 The slug of the project you are accessing.
      --repo string                    The repository you want to source for your Nextflow script (e.g. formbio/nf-proteingetinfo).
      --require-schema-validation      If set to false, a workflow run will submit even if the inputs fail to validate against the workflow schema. (default true)
      --response-format string         Default response format is human-readable. This command also supports "JSON" format.
      --run-name string                The name of the pipeline run that it will be indexed under.
      --set-default-inputs             If set to false, empty workflow inputs will not be auto-filled with available default values. (default true)
      --skip-workflow-results string   Skip running workflow, copy provided workflow results to output folder. Must be in "org://project/workflow_run_id" format.
      --timeout duration               Timeout for running Nextflow on google life sciences. (default 720h0m0s)
      --version string                 Github branch or tag to run (e.g. main or v1.0.2).
      --workflow string                The name of the workflow you are running.
      --workflow-container             If set to true, workflow container will run directly rather than use an uploaded workflow. Must include --config or --config-file

Global Flags:
      --debug   Print debug log lines in the CLI console.

Example command:

# For a given Form Bio Org / Project e.g. Org=form-bio-solutions / Project=rna-seq 
# https://form.bio/organizations/form-bio-solutions/projects/rna-seq/home/
# for JSON output, use the --response-format=JSON flag
$ formbio workflow run \
    --env prod \
    --org form-bio-solutions \
    --project rna-seq \
    --repo nf-rnaseq \
    --workflow expressiondetective \
    --version v1.0.0  \
    --run-name 'runRNASEq' \
    -- \
    --design=formbio://form-bio-solutions/rna-seq/SRP045500.design.txt \
    --input=formbio://form-bio-solutions/rna-seq/pipeline-outputs/getncbi/2023-05-06T00_24_44Z_90a32859-6a83-4fec-882d-38df4147a4aa/output/ \
    --genomeid=GRCh38 \
    --genoannotver=latest \
    --version=rnaseq_v1.0.0 \
    --trim=true \
    --cancer=false \
    --aligner=star \
    --repoDir=/seqprg \
    --markdups=skip \
    --stranded=U \
    --readorigin=F \
    --sourceType=cts \
    --orientation=I \
    --genomes_bucket=gs://bioinfo-workflow-references/genomes \
    --taxid=9606

Creating the workflow...
Your run output is available at: 
https://form.bio/organizations/form-bio-solutions/projects/rna-seqUsage:
  formbio workflow run --org org --project project --repo repo --version version --runName runName  -- [--paramName <param-value>] [--paramName2 <param-value>] [flags]

Get workflow run

‣
formbio workflow-run get --help
$ ./bin/formbio workflow run --help
gets details, including status, about the requested workflow run

Usage:
  formbio workflow-run get <workflow_run_id> [flags]

Examples:
# to get the details about a workflow run
formbio workflow-run get 9dc2d02a-b8eb-438c-9bb7-2f61fdf5066e

# to get the details about a workflow run in JSON format
formbio workflow-run get --response-format=JSON 9dc2d02a-b8eb-438c-9bb7-2f61fdf5066e

Flags:
  -h, --help                     help for get
      --response-format string   Default response format is human-readable. This command also supports "JSON" format.

Global Flags:
      --debug           Print debug log lines in the CLI console.

Example command:

# for JSON output, use the --response-format=JSON flag
$ formbio workflow-run get 9dc2d02a-b8eb-438c-9bb7-2f61fdf5066e

┌───────────────────────────────────────────────────┐
| ID         | 9dc2d02a-b8eb-438c-9bb7-2f61fdf5066e |
| Name       | runRNASEq                            |
| Repository | expressiondetective                  |
| User       | A. N. Nother                         |
| Status     | FAILED                               |
| Created    | 2024-02-29 21:10:56                  |
| Updated    | 2024-02-29 21:15:11                  |
└───────────────────────────────────────────────────┘

# Web links for this workflow run

To view workflow run:
https://formbio.com/organizations/form-bio-solutions/projects/rna-seq/workflow-runs/9dc2d02a-b8eb-438c-9bb7-2f61fdf5066e/

To view workflow run output files in data manager:
(may not work immediately)
https://formbio.com/organizations/form-bio-solutions/projects/rna-seq/vault/?path=pipeline-outputs%2Fexpressiondetective%2F2024-02-29T21_10_56Z_9dc2d02a-b8eb-438c-9bb7-2f61fdf5066e


# CLI commands for this workflow run

To get details about this workflow run:
formbio workflow-run get a9bc9dae-742b-496b-a4b4-4ee4daf9c95b

To list the workflow output directories and files:
(may not work immediately)
formbio storage ls formbio://form-bio-solutions/rna-seq/pipeline-outputs/developerdecoy/2024-02-29T21_10_56Z_a9bc9dae-742b-496b-a4b4-4ee4daf9c95b/

Legal Notices

The formbio CLI tool uses the following Open Source software libraries.

  1. github.com/gosimple/slug@1.12.0
  2. https://github.com/hashicorp/errwrap/tree/v1.1.0
  3. https://github.com/hashicorp/go-multierror/tree/v1.1.1
  4. github.com/hashicorp/hcl/json/parser@1.0.0

← Previous

Add link here

Next →

Add link here