We will outline the main data retrieval workflow and functions using a case study based on two public sets of data:
Before accessing data you will need to connect to a cBioPortal database and set your base URL for the R session. In this example we will use data from the public cBioPortal database instance (https://www.cbioportal.org). You do not need a token to access this public website. If you are using a private instance of cBioPortal (like MSK’s institutional database), you will need to acquire a token and save it to your .Renviron file.
Note: If you are a MSK researcher working on IMPACT, you should connect to MSK’s cBioPortal instance to get the most up to date IMPACT data, and you must follow MSK-IMPACT publication guidelines when using the data.
To set the database url for your current R session use the
set_cbioportal_db() function. To set it to the public
instance you can either provide the full URL to the function, or just
public as a shortcut. This function will both check your
connection to the database and set the url
(www.cbioportal.org/api) as your base url to connect to for
all future API calls during your session.
set_cbioportal_db("public")
#> ✔ You are successfully connected!
#> ✔ base_url for this R session is now set to "www.cbioportal.org/api"You can use test_cbioportal_db at any time throughout
your session to check your connection. This can be helpful when
troubleshooting issues with your API calls.
Now that we are successfully connected, we may want to view all
studies available for our chosen database to find the correct
study_id corresponding to the data we want to pull. All
studies have a unique identifier in the database. You can view all
studies available in your database with the following:
all_studies <- available_studies()
all_studies
#> # A tibble: 391 × 13
#> studyId name description publicStudy
#> <chr> <chr> <chr> <lgl>
#> 1 acc_tcga Adre… "TCGA Adre… TRUE
#> 2 laml_tcga Acut… "TCGA Acut… TRUE
#> 3 blca_tcga Blad… "TCGA Blad… TRUE
#> 4 brca_tcga Brea… "TCGA Brea… TRUE
#> 5 kirc_tcga Kidn… "TCGA Kidn… TRUE
#> 6 cesc_tcga Cerv… "TCGA Cerv… TRUE
#> 7 chol_tcga Chol… "TCGA Chol… TRUE
#> 8 kich_tcga Kidn… "TCGA Kidn… TRUE
#> 9 coadread… Colo… "TCGA Colo… TRUE
#> 10 dlbc_tcga Lymp… "TCGA Lymp… TRUE
#> # ℹ 381 more rows
#> # ℹ 9 more variables: groups <chr>,
#> # status <int>, importDate <chr>,
#> # allSampleCount <int>,
#> # readPermission <lgl>,
#> # cancerTypeId <chr>,
#> # referenceGenome <chr>, pmid <chr>, …By inspecting this data frame, we see the unique
study_id for the NMIBC data set is
"blca_nmibc_2017" and the unique study_id for
the prostate cancer data set is "prad_msk_2019". To get
more information on our studies we can do the following:
Note: the transpose function t() is just used here
to better view results
all_studies %>%
filter(studyId %in% c("blca_nmibc_2017", "prad_msk_2019"))
#> # A tibble: 2 × 13
#> studyId name description publicStudy
#> <chr> <chr> <chr> <lgl>
#> 1 prad_msk_… Pros… MSK-IMPACT… TRUE
#> 2 blca_nmib… Nonm… IMPACT seq… TRUE
#> # ℹ 9 more variables: groups <chr>,
#> # status <int>, importDate <chr>,
#> # allSampleCount <int>,
#> # readPermission <lgl>,
#> # cancerTypeId <chr>,
#> # referenceGenome <chr>, pmid <chr>,
#> # citation <chr>More in-depth information about the study can be found with
get_study_info()
get_study_info("blca_nmibc_2017") %>%
t()
#> [,1]
#> name "Nonmuscle Invasive Bladder Cancer (MSK Eur Urol 2017)"
#> description "IMPACT sequencing of 105 High Risk Nonmuscle Invasive Bladder Cancer samples."
#> publicStudy "TRUE"
#> pmid "28583311"
#> citation "Pietzak et al. Eur Urol 2017"
#> groups "PUBLIC"
#> status "0"
#> importDate "2023-07-19 17:59:06"
#> allSampleCount "105"
#> sequencedSampleCount "105"
#> cnaSampleCount "105"
#> mrnaRnaSeqSampleCount "0"
#> mrnaRnaSeqV2SampleCount "0"
#> mrnaMicroarraySampleCount "0"
#> miRnaSampleCount "0"
#> methylationHm27SampleCount "0"
#> rppaSampleCount "0"
#> massSpectrometrySampleCount "0"
#> completeSampleCount "0"
#> readPermission "TRUE"
#> treatmentCount "0"
#> studyId "blca_nmibc_2017"
#> cancerTypeId "blca"
#> cancerType.name "Bladder Urothelial Carcinoma"
#> cancerType.dedicatedColor "Yellow"
#> cancerType.shortName "BLCA"
#> cancerType.parent "bladder"
#> cancerType.cancerTypeId "blca"
#> referenceGenome "hg19"get_study_info("prad_msk_2019") %>%
t()
#> [,1]
#> name "Prostate Cancer (MSK, Cell Metab 2020)"
#> description "MSK-IMPACT Sequencing of 18 prostate cancer tumor/normal pairs."
#> publicStudy "TRUE"
#> pmid "31564440"
#> citation "Granlund et al. Cell Metab 2020"
#> groups "PUBLIC"
#> status "0"
#> importDate "2023-06-20 12:45:47"
#> allSampleCount "18"
#> sequencedSampleCount "18"
#> cnaSampleCount "18"
#> mrnaRnaSeqSampleCount "0"
#> mrnaRnaSeqV2SampleCount "0"
#> mrnaMicroarraySampleCount "0"
#> miRnaSampleCount "0"
#> methylationHm27SampleCount "0"
#> rppaSampleCount "0"
#> massSpectrometrySampleCount "0"
#> completeSampleCount "0"
#> readPermission "TRUE"
#> treatmentCount "0"
#> studyId "prad_msk_2019"
#> cancerTypeId "prostate"
#> cancerType.name "Prostate"
#> cancerType.dedicatedColor "Cyan"
#> cancerType.shortName "PROSTATE"
#> cancerType.parent "tissue"
#> cancerType.cancerTypeId "prostate"
#> referenceGenome "hg19"Lastly, it is important to know what genomic data is available for our studies. Not all studies in your database will have data available on all types of genomic information. For example, it is common for studies not to provide data on fusions/structural variants.
We can check available genomic data with
available_profiles().
available_profiles(study_id = "blca_nmibc_2017")
#> # A tibble: 3 × 8
#> molecularAlterationType datatype name
#> <chr> <chr> <chr>
#> 1 COPY_NUMBER_ALTERATION DISCRETE Putati…
#> 2 MUTATION_EXTENDED MAF Mutati…
#> 3 STRUCTURAL_VARIANT SV Struct…
#> # ℹ 5 more variables: description <chr>,
#> # showProfileInAnalysisTab <lgl>,
#> # patientLevel <lgl>,
#> # molecularProfileId <chr>,
#> # studyId <chr>available_profiles(study_id = "prad_msk_2019")
#> # A tibble: 3 × 8
#> molecularAlterationType datatype name
#> <chr> <chr> <chr>
#> 1 COPY_NUMBER_ALTERATION DISCRETE Putati…
#> 2 MUTATION_EXTENDED MAF Mutati…
#> 3 STRUCTURAL_VARIANT SV Struct…
#> # ℹ 5 more variables: description <chr>,
#> # showProfileInAnalysisTab <lgl>,
#> # patientLevel <lgl>,
#> # molecularProfileId <chr>,
#> # studyId <chr>Luckily, in this example our studies have mutation, copy number
alteration and fusion (structural variant) data available. Each of these
data types has a unique molecular profile ID. The molecular profile ID
usually takes the form of <study_id>_mutations,
<study_id>_structural_variants,
<study_id>_cna.
Now that we have inspected our studies and confirmed the genomic data that is available, we will pull the data into our R environment. We will show two ways to do this:
get_genetics_by_study())get_genetics_by_sample())Pulling by study will give us genomic data for all genes/panels included in the study. These functions can only pull data one study ID at a time and will return all genomic data available for that study. Pulling by study ID can be efficient, and a good way to ensure you have all genomic information available in cBioPortal for a particular study.
If you are working across multiple studies, or only need a subset of samples from one or multiple studies, you may chose to pull by sample IDs instead of study ID. When you pull by sample IDs you can pull specific samples across multiple studies, but must also specify the studies they belong to. You may also pass a specific list of genes for which to return information. If you don’t specify a list of genes the function will default to returning all available gene data for each sample.
To pull by study ID, we can pull each data type individually.
mut_blca <- get_mutations_by_study(study_id = "blca_nmibc_2017")
#> ℹ Returning all data for the "blca_nmibc_2017_mutations" molecular profile in the "blca_nmibc_2017" study
cna_blca<- get_cna_by_study(study_id = "blca_nmibc_2017")
#> ℹ Returning all data for the "blca_nmibc_2017_cna" molecular profile in the "blca_nmibc_2017" study
fus_blca <- get_fusions_by_study(study_id = "blca_nmibc_2017")
#> ℹ Returning all data for the "blca_nmibc_2017_structural_variants" molecular profile in the "blca_nmibc_2017" study
mut_prad <- get_mutations_by_study(study_id = "prad_msk_2019")
#> ℹ Returning all data for the "prad_msk_2019_mutations" molecular profile in the "prad_msk_2019" study
cna_prad <- get_cna_by_study(study_id = "prad_msk_2019")
#> ℹ Returning all data for the "prad_msk_2019_cna" molecular profile in the "prad_msk_2019" study
fus_prad <- get_fusions_by_study(study_id = "prad_msk_2019")
#> ℹ Returning all data for the "prad_msk_2019_structural_variants" molecular profile in the "prad_msk_2019" studyOr we can pull all genomic data at the same time with
get_genetics_by_study()
all_genomic_blca <- get_genetics_by_study("blca_nmibc_2017")
#> ℹ Returning all data for the "blca_nmibc_2017_mutations" molecular profile in the "blca_nmibc_2017" study
#> ℹ Returning all data for the "blca_nmibc_2017_cna" molecular profile in the "blca_nmibc_2017" study
#> ℹ Returning all data for the "blca_nmibc_2017_structural_variants" molecular profile in the "blca_nmibc_2017" study
all_genomic_prad <- get_genetics_by_study("prad_msk_2019")
#> ℹ Returning all data for the "prad_msk_2019_mutations" molecular profile in the "prad_msk_2019" study
#> ℹ Returning all data for the "prad_msk_2019_cna" molecular profile in the "prad_msk_2019" study
#> ℹ Returning all data for the "prad_msk_2019_structural_variants" molecular profile in the "prad_msk_2019" studyall_equal(mut_blca, all_genomic_blca$mutation)
#> [1] TRUE
all_equal(cna_blca, all_genomic_blca$cna)
#> [1] TRUE
all_equal(fus_blca, all_genomic_blca$structural_variant)
#> [1] TRUEFinally, we can join the two studies together
When we pull by sample IDs, we can pull specific samples across
multiple studies. In the above example, we can pull from both studies at
the same time for a select set of samples using the
sample_study_pairs argument in
get_genetics_by_sample().
Let’s pull data for the first 10 samples in each study. We first need to construct our dataframe to pass to the function:
Note: you can also run
available_samples(sample_list_id = <sample list ID>)
to pull sample IDs by a specific sample list ID (see
available_sample_lists()), or
available_patients() to pull patient IDs
s1 <- available_samples("blca_nmibc_2017") %>%
select(sampleId, patientId, studyId) %>%
head(10)
s2 <- available_samples("prad_msk_2019") %>%
select(sampleId, patientId, studyId) %>%
head(10)
df_pairs <- bind_rows(s1, s2) %>%
select(-patientId)We need to rename the columns as per the functions documentation.
Now we pass this to get_genetics_by_sample()
all_genomic <- get_genetics_by_sample(sample_study_pairs = df_pairs)
#> Joining with `by = join_by(study_id)`
#> The following parameters were used in
#> query:
#> Study ID: "blca_nmibc_2017" and
#> "prad_msk_2019"
#> Molecular Profile ID:
#> blca_nmibc_2017_mutations and
#> prad_msk_2019_mutations
#> Genes: "All available genes"
#> Joining with `by = join_by(study_id)`
#> The following parameters were used in
#> query:
#> Study ID: "blca_nmibc_2017" and
#> "prad_msk_2019"
#> Molecular Profile ID: blca_nmibc_2017_cna
#> and prad_msk_2019_cna
#> Genes: "All available genes"
#> Joining with `by = join_by(study_id)`
#> The following parameters were used in
#> query:
#> Study ID: "blca_nmibc_2017" and
#> "prad_msk_2019"
#> Molecular Profile ID:
#> blca_nmibc_2017_structural_variants and
#> prad_msk_2019_structural_variants
#> Genes: "All available genes"
mut_sample <- all_genomic$mutationLike with querying by study ID, you can also pull data individually by genomic data type:
mut_only <- get_mutations_by_sample(sample_study_pairs = df_pairs)
#> Joining with `by = join_by(study_id)`
#> The following parameters were used in
#> query:
#> Study ID: "blca_nmibc_2017" and
#> "prad_msk_2019"
#> Molecular Profile ID:
#> blca_nmibc_2017_mutations and
#> prad_msk_2019_mutations
#> Genes: "All available genes"
identical(mut_only, mut_sample)
#> [1] TRUELet’s compare these results with the ones we got from pulling by study:
# filter to our subset used in sample query
mut_study_subset <- mut_study %>%
filter(sampleId %in% df_pairs$sample_id)
# arrange to compare
mut_study_subset <- mut_study_subset %>%
arrange(desc(sampleId))%>%
arrange(desc(entrezGeneId))
mut_sample <- mut_sample %>%
arrange(desc(sampleId)) %>%
arrange(desc(entrezGeneId)) %>%
# reorder so columns in same order
select(names(mut_study_subset))
all.equal(mut_study_subset, mut_sample)
#> [1] TRUEBoth results are equal.
Note: some studies also have copy number segments data available that can be pulled by study ID or sample ID:
seg_blca <- get_segments_by_study("blca_nmibc_2017")
#> ℹ Returning all "copy number segmentation" data for the "blca_nmibc_2017" study
# To pull alongside other genomic data types, use the `return_segments` argument
all_genomic_blca <- get_genetics_by_study("blca_nmibc_2017", return_segments = TRUE)
#> ℹ Returning all data for the "blca_nmibc_2017_mutations" molecular profile in the "blca_nmibc_2017" study
#> ℹ Returning all data for the "blca_nmibc_2017_cna" molecular profile in the "blca_nmibc_2017" study
#> ℹ Returning all data for the "blca_nmibc_2017_structural_variants" molecular profile in the "blca_nmibc_2017" study
#> ℹ Returning all "copy number segmentation" data for the "blca_nmibc_2017" studyWhen pulling by sample IDs, we can also limit our results to a
specific set of genes by passing a vector of Entrez Gene IDs or Hugo
Symbols to the gene argument, or a specified panel by
passing a panel ID to the panel argument (see
available_gene_panels() for supported panels). This can be
useful if, for example, we want to pull all IMPACT gene results for two
studies but one of the two uses a much larger panel. In that case, we
can limit our query to just the genes for which we want results:
by_hugo <- get_mutations_by_sample(sample_study_pairs = df_pairs, genes = "TP53")
#> Joining with `by = join_by(study_id)`
#> The following parameters were used in
#> query:
#> Study ID: "blca_nmibc_2017" and
#> "prad_msk_2019"
#> Molecular Profile ID:
#> blca_nmibc_2017_mutations and
#> prad_msk_2019_mutations
#> Genes: "TP53"
by_gene_id <- get_mutations_by_sample(sample_study_pairs = df_pairs, genes = 7157)
#> Joining with `by = join_by(study_id)`
#> The following parameters were used in
#> query:
#> Study ID: "blca_nmibc_2017" and
#> "prad_msk_2019"
#> Molecular Profile ID:
#> blca_nmibc_2017_mutations and
#> prad_msk_2019_mutations
#> Genes: 7157
identical(by_hugo, by_gene_id)
#> [1] TRUEget_mutations_by_sample(
sample_study_pairs = df_pairs,
panel = "IMPACT468") %>%
head()
#> Joining with `by = join_by(study_id)`
#> The following parameters were used in
#> query:
#> Study ID: "blca_nmibc_2017" and
#> "prad_msk_2019"
#> Molecular Profile ID:
#> blca_nmibc_2017_mutations and
#> prad_msk_2019_mutations
#> Genes: "IMPACT468"
#> # A tibble: 6 × 28
#> hugoGeneSymbol entrezGeneId
#> <chr> <int>
#> 1 SMAD4 4089
#> 2 TERT 7015
#> 3 ERBB4 2066
#> 4 CUL3 8452
#> 5 PBRM1 55193
#> 6 APC 324
#> # ℹ 26 more variables:
#> # uniqueSampleKey <chr>,
#> # uniquePatientKey <chr>,
#> # molecularProfileId <chr>,
#> # sampleId <chr>, patientId <chr>,
#> # studyId <chr>, center <chr>,
#> # mutationStatus <chr>, …You can also pull clinical data by study ID, sample ID, or patient ID. Pulling by sample ID will pull all sample-level characteristics (e.g. sample site, tumor stage at sampling time and other variables collected at time of sampling that may be available). Pulling by patient ID will pull all patient-level characteristics (e.g. age, sex, etc.). Pulling by study ID will pull all sample and patient-level characteristics at once.
You can explore what clinical data is available a study using:
attr_blca <- available_clinical_attributes("blca_nmibc_2017")
attr_prad <- available_clinical_attributes("prad_msk_2019")
attr_prad
#> # A tibble: 13 × 7
#> displayName description datatype
#> <chr> <chr> <chr>
#> 1 Cancer Type Cancer Type STRING
#> 2 Cancer Type Detai… Cancer Typ… STRING
#> 3 Fraction Genome A… Fraction G… NUMBER
#> 4 Gene Panel Gene Panel. STRING
#> 5 Mutation Count Mutation C… NUMBER
#> 6 Oncotree Code Oncotree C… STRING
#> 7 Sample Class The sample… STRING
#> 8 Number of Samples… Number of … STRING
#> 9 Sample Type The type o… STRING
#> 10 Sex Sex STRING
#> 11 Somatic Status Somatic St… STRING
#> 12 Specimen Preserva… The method… STRING
#> 13 TMB (nonsynonymou… TMB (nonsy… NUMBER
#> # ℹ 4 more variables:
#> # patientAttribute <lgl>,
#> # priority <chr>,
#> # clinicalAttributeId <chr>,
#> # studyId <chr>There are a select set available for both studies:
The below pulls data at the sample level:
clinical_blca <- get_clinical_by_sample(sample_id = s1$sampleId,
study_id = "blca_nmibc_2017",
clinical_attribute = in_both)
clinical_prad <- get_clinical_by_sample(sample_id = s2$sampleId,
study_id = "prad_msk_2019",
clinical_attribute = in_both)
all_clinical <- bind_rows(clinical_blca, clinical_prad)
all_clinical %>%
select(-contains("unique")) %>%
head()
#> # A tibble: 6 × 5
#> sampleId patientId studyId
#> <chr> <chr> <chr>
#> 1 P-0001453-T01-IM3 P-0001453 blca_nmibc_…
#> 2 P-0001453-T01-IM3 P-0001453 blca_nmibc_…
#> 3 P-0001453-T01-IM3 P-0001453 blca_nmibc_…
#> 4 P-0001453-T01-IM3 P-0001453 blca_nmibc_…
#> 5 P-0001453-T01-IM3 P-0001453 blca_nmibc_…
#> 6 P-0001453-T01-IM3 P-0001453 blca_nmibc_…
#> # ℹ 2 more variables:
#> # clinicalAttributeId <chr>,
#> # value <chr>The below pulls data at the patient level:
p1 <- available_patients("blca_nmibc_2017")
clinical_blca <- get_clinical_by_patient(patient_id = s1$patientId,
study_id = "blca_nmibc_2017",
clinical_attribute = in_both)
clinical_prad <- get_clinical_by_patient(patient_id = s2$patientId,
study_id = "prad_msk_2019",
clinical_attribute = in_both)
all_clinical <- bind_rows(clinical_blca, clinical_prad)
all_clinical %>%
select(-contains("unique")) %>%
head()
#> # A tibble: 6 × 4
#> patientId studyId clinicalAttributeId
#> <chr> <chr> <chr>
#> 1 P-0001453 blca_nmib… SAMPLE_COUNT
#> 2 P-0001453 blca_nmib… SEX
#> 3 P-0002166 blca_nmib… SAMPLE_COUNT
#> 4 P-0002166 blca_nmib… SEX
#> 5 P-0003238 blca_nmib… SAMPLE_COUNT
#> 6 P-0003238 blca_nmib… SEX
#> # ℹ 1 more variable: value <chr>Like with the genomic data pull functions, you can also pull clinical data by a data frame of sample ID - study ID pairs, or a data frame of patient ID - study ID pairs. Below, we will pull by patient ID - study ID pairs.
First, we construct the data frame of pairs to pass:
df_pairs <- bind_rows(s1, s2) %>%
select(-sampleId)
df_pairs <- df_pairs %>%
select(patientId, studyId)Now we pass this data frame to
get_genetics_by_patient()
all_patient_clinical <- get_clinical_by_patient(patient_study_pairs = df_pairs,
clinical_attribute = in_both)
all_patient_clinical %>%
select(-contains("unique"))
#> # A tibble: 34 × 4
#> patientId studyId clinicalAttributeId
#> <chr> <chr> <chr>
#> 1 P-0001453 blca_nmi… SAMPLE_COUNT
#> 2 P-0001453 blca_nmi… SEX
#> 3 P-0002166 blca_nmi… SAMPLE_COUNT
#> 4 P-0002166 blca_nmi… SEX
#> 5 P-0003238 blca_nmi… SAMPLE_COUNT
#> 6 P-0003238 blca_nmi… SEX
#> 7 P-0003257 blca_nmi… SAMPLE_COUNT
#> 8 P-0003257 blca_nmi… SEX
#> 9 P-0003261 blca_nmi… SAMPLE_COUNT
#> 10 P-0003261 blca_nmi… SEX
#> # ℹ 24 more rows
#> # ℹ 1 more variable: value <chr>