# Developing-Heart-Local - Notebooks

**Pages:** 209

---

## Variables¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/Xenium/no2_integration_and_cell-type-annotation/no6_manual-annotation_ventricularCM.html

**Contents:**
- Variables¶
- Read in data¶
- Markers¶
- 1st round annotation¶
- Subset and reclustering¶
- Transfer annotation to the original adata¶
- Save¶

15516 rows × 8 columns

--> cluster 1 has lower total counts and smaller cell area, and no specific gene expression --> annotate as "unassigned"

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os

import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 

import scanpy.external as sce
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/session_info/main.py:213: UserWarning: The '__version__' attribute is deprecated and will be removed in MarkupSafe 3.1. Use feature detection, or `importlib.metadata.version("markupsafe")`, instead.
  mod_version = _find_version(mod.__version__)
```

Example 4 (python):
```python
-----
matplotlib          3.10.1
numpy               2.1.3
pandas              2.2.3
scanpy              1.11.1
seaborn             0.13.2
session_info        v1.0.1
-----
```

Example 5 (python):
```python
Cython              3.1.1
PIL                 11.1.0
anndata             0.11.4
asciitree           NA
asttokens           NA
charset_normalizer  3.4.1
cloudpickle         3.1.1
comm                0.2.2
cycler              0.12.1
cython              3.1.1
cython_runtime      NA
dask                2024.11.2
dateutil            2.9.0.post0
debugpy             1.8.13
decorator           5.2.1
exceptiongroup      1.2.2
executing           2.2.0
h5py                3.13.0
igraph              0.11.8
importlib_metadata  NA
ipykernel           6.29.5
jedi                0.19.2
jinja2              3.1.6
joblib              1.4.2
kiwisolver          1.4.8
legacy_api_wrap     NA
leidenalg           0.10.2
llvmlite            0.44.0
markupsafe          3.0.2
mpl_toolkits        NA
msgpack             1.1.0
natsort             8.4.0
numba               0.61.0
numcodecs           0.13.1
packaging           24.2
parso               0.8.4
patsy               1.0.1
platformdirs        4.3.7
prompt_toolkit      3.0.50
psutil              7.0.0
pure_eval           0.2.3
pyarrow             19.0.1
pydev_ipython       NA
pydevconsole        NA
pydevd              3.2.3
pydevd_file_utils   NA
pydevd_plugins      NA
pydevd_tracing      NA
pygments            2.19.1
pyparsing           3.2.3
pytz                2025.2
scipy               1.15.2
six                 1.17.0
sklearn             1.5.2
stack_data          0.6.3
statsmodels         0.14.4
tblib               3.1.0
texttable           1.7.0
threadpoolctl       3.6.0
tlz                 1.0.0
toolz               1.0.0
tornado             6.4.2
traitlets           5.14.3
typing_extensions   NA
wcwidth             0.2.13
yaml                6.0.2
zarr                2.18.3
zipp                NA
zmq                 26.3.0
zoneinfo            NA
```

Example 6 (unknown):
```unknown
-----
IPython             8.34.0
jupyter_client      8.6.3
jupyter_core        5.7.2
-----
Python 3.10.14 | packaged by conda-forge | (main, Mar 20 2024, 12:45:18) [GCC 12.3.0]
Linux-4.18.0-553.53.1.el8_10.x86_64-x86_64-with-glibc2.28
-----
Session information updated at 2025-06-09 21:18
```

Example 7 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/home/kk837/rfs/rfs-teichlab-nfs-iCNyzSAaucw/kk837/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 8 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 9 (python):
```python
adata_dir = '/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium/subsets'
sample_id = 'C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat'
celltype = 'VentricularCardiomyocytes'
```

Example 10 (unknown):
```unknown
# figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk/Aug2024'
# table_dir = '/nfs/team205/heart/fetal_ms_figures/tables/Aug2024'
```

Example 11 (unknown):
```unknown
latent_space = 'pca_harmony'
```

Example 12 (python):
```python
# read in
adata = sc.read_h5ad(f'{adata_dir}/{sample_id}_5K_{celltype}_lognorm.h5ad')
adata
```

Example 13 (unknown):
```unknown
AnnData object with n_obs × n_vars = 188122 × 5001
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_3.0', 'multi_celltypes_coarse', 'leiden_0.2', 'leiden_0.4', 'leiden_0.6', 'leiden_1'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1', 'n_cells', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'dendrogram_pca_harmony_leiden_1.0', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1_colors', 'log1p', 'multi_celltypes_coarse_colors', 'pca', 'pca_harmony', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.2_colors', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.3_colors', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.4_colors', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.5_colors', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_0.8_colors', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.0_colors', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_1.5_colors', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_2.0_colors', 'pca_harmony_leiden_3.0', 'pca_harmony_leiden_3.0_colors', 'tissue_block_id_colors', 'umap'
    obsm: 'X_pca', 'X_pca_harmony', 'X_umap_pca_harmony', 'spatial'
    varm: 'PCs'
    obsp: 'pca_harmony_connectivities', 'pca_harmony_distances'
```

Example 14 (python):
```python
adata.X.data[:5]
```

Example 15 (unknown):
```unknown
array([2.4982107, 2.4982107, 2.4982107, 2.4982107, 3.5404415],
      dtype=float32)
```

Example 16 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=['total_counts','n_genes_by_counts','cell_area','tissue_block_id'],
                    wspace=0.2,cmap='RdPu',vmax='p100')
```

Example 17 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=['multi_celltypes_coarse',"conf_score_midmod2fine","celltypist_midmod2fine",],
                    wspace=0.2,cmap='RdPu',vmax='p100')
resolutions_list = [0.2,0.4,0.6,1]
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                    color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                    wspace=0.2)
```

Example 18 (python):
```python
%%time
# leiden clustering
resolutions_list = [# 0.2,0.4,0.6,1, # already done
                    1.2,1.5,2]
for resolution in resolutions_list:
    sc.tl.leiden(adata,neighbors_key=latent_space,
                     resolution=resolution,key_added=f'leiden_{str(resolution)}',n_iterations=2)
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                wspace=0.3)
```

Example 19 (unknown):
```unknown
<timed exec>:5: FutureWarning: In the future, the default backend for leiden will be igraph instead of leidenalg.

 To achieve the future defaults please pass: flavor="igraph" and n_iterations=2.  directed must also be False to work with igraph's implementation.
```

Example 20 (unknown):
```unknown
CPU times: user 3min 38s, sys: 1.89 s, total: 3min 39s
Wall time: 3min 40s
```

Example 21 (python):
```python
# read in markers identified using single-nucleus data
markers_df = pd.read_csv(f'/home/kk837/rfs/rfs-teichlab-nfs-iCNyzSAaucw/kk837/notebooks/Foetal/VisiumHD/re-seq_mpp0p3/sc_{celltype}_finegrain_markers.csv')

mask1 = markers_df['group_mean']>0.1
mask2 = markers_df['logfoldchanges']>0.1
markers_df = markers_df[mask1&mask2]

markers_df
```

Example 22 (python):
```python
markers = {}
n_top = 10
for c in set(markers_df['group']):
    markers_df_sub = markers_df[markers_df['group']==c]
    markers_df_sub.sort_values('logfoldchanges',ascending=False)
    markers[c]=list(markers_df_sub[:n_top]['names'])

# manually add genes
markers['VentricularConductionSystemProximal'] = markers['VentricularConductionSystemProximal']+['TBX3','PRDM16','NIBAN1','PHACTR1','FGF18'] # 'PRDM16','NIBAN1','PHACTR1','FGF18' are the markers compared to VCSdistal
markers['VentricularConductionSystemDistal'] = markers['VentricularConductionSystemDistal']+['SEMA3A','IRX2','IRX3','OPCML','NTN1','EPHA4','GABRB2'] 
markers['Cycling'] = ['MKI67'] 

genes = []
for g in markers.values():
    genes = genes + [x for x in g if (x in list(adata.var_names))&(x not in genes)]
for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in list(adata.var_names)]
len(genes)
```

Example 23 (python):
```python
sc.pl.embedding(adata,basis=f"X_umap_{latent_space}",color=genes, size=10,
               ncols=6, wspace=0.2,cmap='RdPu',vmax='p100')
```

Example 24 (unknown):
```unknown
resolution_sel = 1
```

Example 25 (python):
```python
markers_others = {
    'CM': ['TTN', 'TNNT2','MYH6',
        'MYH7', 'MYL2', 'FHL2', # vCN
            'NPPA', 'MYL7', 'MYL4'], #aCM
    'FB': ['DCN', 'GCN', 'PDGFRA','COL1A1','COL1A2'],
    'EC': ['VWF', 'PECAM1', 'CDH5','RGCC', 'FABP5',],
    'Peri': ['RGS5', 'ABCC9', 'KCNJ8'],
    'SMC': ['MYH11', 'TAGLN', 'ACTA2'],
    'Meso': ['MSLN', 'WT1', 'BNC1'],
    'Neuro': ['PLP1', 'NRXN1', 'NRXN3','PRPH', 'NEFL', 'NEFM', 'NEFH', 'STMN2', 'SST', 'SLC10A4', 'SLC18A3', 'TH', 'NPY'],
    'Adipo': ['GPAM', 'FASN', 'LEP'],
    'Myelo': ['CD14', 'C1QA', 'CD68','LYVE1','TIMD4'],
    'Lymph': ['CD8A', 'IL7R', 'CD40LG'],
    'B': ['CD19','SDC1','IGHG1','IGHA1'],
    'Mast': ['KIT', 'CPA3'],
}

for key in markers_others.keys():
    markers_others[key]=[x for x in markers_others[key] if x in adata.var_names]

sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
sc.pl.dotplot(adata,
              markers_others, 
              groupby=f'leiden_{str(resolution_sel)}',
              dendrogram=True,
              standard_scale="var",
              color_map="Reds",
              swap_axes=False,
             )
```

Example 26 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: CM, FB, EC, etc.
```

Example 27 (python):
```python
sc.tl.dendrogram(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.pl.dotplot(adata,
              var_names=markers,
              groupby=f'leiden_{str(resolution_sel)}',
              standard_scale='var',
              dendrogram=True
             )
```

Example 28 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: VentricularConductionSystemProximal, VentricularCardiomyocytesCycling, VentricularConductionSystemDistal, etc.
```

Example 29 (python):
```python
print(adata.X.data[:5])
adata.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata, dendrogram=True, standard_scale='var', return_fig=True)
dp.add_totals().show()
```

Example 30 (unknown):
```unknown
[2.4982107 2.4982107 2.4982107 2.4982107 3.5404415]
```

Example 31 (python):
```python
sc.pl.violin(adata,keys=['total_counts','cell_area'],groupby=f'leiden_{str(resolution_sel)}')
```

Example 32 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=f'leiden_{str(resolution_sel)}',legend_loc='on data',
                )
```

Example 33 (unknown):
```unknown
'VentricularCardiomyocytes'
```

Example 34 (python):
```python
# annotate
adata.obs['tmp_annotation'] = adata.obs[f'leiden_{str(resolution_sel)}'].copy()
adata.obs.replace({'tmp_annotation':{
    '0':'VentricularCardiomyocytes',
    '1':'unassigned',
    '2':'VentricularCardiomyocytes',
    '3':'VentricularCardiomyocytes',
    '4':'VentricularCardiomyocytes',
    '5':'VentricularCardiomyocytesCycling',
    '6':'VentricularCardiomyocytes',
    '7':'VentricularCardiomyocytes',
    '8':'VentricularCardiomyocytes',
    '9':'VentricularCardiomyocytes',
    '10':'VentricularCardiomyocytes',
}},inplace=True)
adata.obs['tmp_annotation'].value_counts()
```

Example 35 (python):
```python
/tmp/ipykernel_66828/1515001482.py:3: FutureWarning: The behavior of Series.replace (and DataFrame.replace) with CategoricalDtype is deprecated. In a future version, replace will only be used for cases that preserve the categories. To change the categories, use ser.cat.rename_categories instead.
  adata.obs.replace({'tmp_annotation':{
```

Example 36 (unknown):
```unknown
tmp_annotation
VentricularCardiomyocytes           135389
unassigned                           39920
VentricularCardiomyocytesCycling     12813
Name: count, dtype: int64
```

Example 37 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color='tmp_annotation'
                )
```

Example 38 (python):
```python
adata_sub = adata[adata.obs['tmp_annotation'].isin(['VentricularCardiomyocytes'])]
```

Example 39 (python):
```python
# drop leiden clustering results
resolutions_list = [0.2,0.4,0.6,1, 1.2,1.5,2]
for resolution in resolutions_list:
    adata_sub.obs.drop(f'leiden_{str(resolution)}',axis=1,inplace=True)
adata_sub
```

Example 40 (unknown):
```unknown
AnnData object with n_obs × n_vars = 135389 × 5001
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_3.0', 'multi_celltypes_coarse', 'tmp_annotation'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1', 'n_cells', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'dendrogram_pca_harmony_leiden_1.0', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1_colors', 'log1p', 'multi_celltypes_coarse_colors', 'pca', 'pca_harmony', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.2_colors', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.3_colors', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.4_colors', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.5_colors', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_0.8_colors', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.0_colors', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_1.5_colors', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_2.0_colors', 'pca_harmony_leiden_3.0', 'pca_harmony_leiden_3.0_colors', 'tissue_block_id_colors', 'umap', 'leiden_1.2', 'leiden_1.5', 'leiden_2', 'leiden_1.2_colors', 'leiden_1.5_colors', 'leiden_2_colors', 'dendrogram_leiden_1', 'rank_genes_groups', 'tmp_annotation_colors'
    obsm: 'X_pca', 'X_pca_harmony', 'X_umap_pca_harmony', 'spatial'
    varm: 'PCs'
    obsp: 'pca_harmony_connectivities', 'pca_harmony_distances'
```

Example 41 (python):
```python
## Remove mid-grain markers from the feature
# read in markers
midgrain_markers = pd.read_csv('/home/kk837/rfs/rfs-teichlab-nfs-iCNyzSAaucw/kk837/notebooks/Foetal/VisiumHD/re-seq_mpp0p3/sc_midgrain_markers.csv',
                               index_col=0)
# remove the markers of celltype of interest
midgrain_markers = midgrain_markers[midgrain_markers['group']!=celltype]
# remove
mask = [x not in list(midgrain_markers['names']) for x in adata_sub.var_names]
bdata = adata_sub[:,mask]
bdata
```

Example 42 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 135389 × 4535
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_3.0', 'multi_celltypes_coarse', 'tmp_annotation'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1', 'n_cells', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'dendrogram_pca_harmony_leiden_1.0', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1_colors', 'log1p', 'multi_celltypes_coarse_colors', 'pca', 'pca_harmony', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.2_colors', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.3_colors', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.4_colors', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.5_colors', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_0.8_colors', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.0_colors', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_1.5_colors', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_2.0_colors', 'pca_harmony_leiden_3.0', 'pca_harmony_leiden_3.0_colors', 'tissue_block_id_colors', 'umap', 'leiden_1.2', 'leiden_1.5', 'leiden_2', 'leiden_1.2_colors', 'leiden_1.5_colors', 'leiden_2_colors', 'dendrogram_leiden_1', 'rank_genes_groups', 'tmp_annotation_colors'
    obsm: 'X_pca', 'X_pca_harmony', 'X_umap_pca_harmony', 'spatial'
    varm: 'PCs'
    obsp: 'pca_harmony_connectivities', 'pca_harmony_distances'
```

Example 43 (python):
```python
'''
# remove cell cycle genes
cell_cycle_genes = ['MCM5','PCNA','TYMS','FEN1','MCM2','MCM4','RRM1','UNG','GINS2','MCM6','CDCA7','DTL','PRIM1','UHRF1','MLF1IP','HELLS','RFC2','RPA2','NASP',
                'RAD51AP1','GMNN','WDR76','SLBP','CCNE2','UBR7','POLD3','MSH2','ATAD2','RAD51','RRM2','CDC45','CDC6','EXO1','TIPIN','DSCC1','BLM','CASP8AP2',
                'USP1','CLSPN','POLA1','CHAF1B','BRIP1','E2F8','HMGB2','CDK1','NUSAP1','UBE2C','BIRC5','TPX2','TOP2A','NDC80','CKS2','NUF2','CKS1B','MKI67',
                'TMPO','CENPF','TACC3','FAM64A','SMC4','CCNB2','CKAP2L','CKAP2','AURKB','BUB1','KIF11','ANP32E','TUBB4B','GTSE1','KIF20B','HJURP','CDCA3','HN1',
                'CDC20','TTK','CDC25C','KIF2C','RANGAP1','NCAPD2','DLGAP5','CDCA2','CDCA8','ECT2','KIF23','HMMR','AURKA','PSRC1','ANLN','LBR','CKAP5','CENPE','CTCF',
                'NEK2','G2E3','GAS2L3','CBX5','CENPA']
mask = [x not in cell_cycle_genes for x in bdata.var_names]
bdata = bdata[:,mask]
bdata.shape
'''
```

Example 44 (python):
```python
# HVGs, using the object which doesn't have midgrain markers
sc.pp.highly_variable_genes(bdata, flavor='seurat', min_mean=0.0125, max_mean=3, min_disp=0.5)
print(f'number of hvgs: {sum(bdata.var.highly_variable)}')
# PCA using the degected HVGs
sc.tl.pca(bdata, use_highly_variable=True)
adata_sub.obsm['X_pca'] = bdata.obsm['X_pca'].copy()
# harmony integration
sce.pp.harmony_integrate(adata_sub, 'tissue_block_id', basis='X_pca', adjusted_basis='X_pca_harmony')
# create neighbour graph
latent_space = 'pca_harmony'
sc.pp.neighbors(adata_sub,n_neighbors=15,use_rep=f"X_{latent_space}",key_added=latent_space)
# visualisation with umap
sc.tl.umap(adata_sub,min_dist=0.5,neighbors_key=latent_space)
adata_sub.obsm[f"X_umap_{latent_space}"]=adata_sub.obsm["X_umap"]
del adata_sub.obsm["X_umap"]
```

Example 45 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_highly_variable_genes.py:698: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  adata.uns["hvg"] = {"flavor": flavor}
```

Example 46 (unknown):
```unknown
number of hvgs: 631
```

Example 47 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_pca/__init__.py:438: FutureWarning: Argument `use_highly_variable` is deprecated, consider using the mask argument. Use_highly_variable=True can be called through mask_var="highly_variable". Use_highly_variable=False can be called through mask_var=None
  warn(msg, FutureWarning)
2025-06-09 21:27:35,158 - harmonypy - INFO - Computing initial centroids with sklearn.KMeans...
2025-06-09 21:27:47,956 - harmonypy - INFO - sklearn.KMeans initialization complete.
2025-06-09 21:27:48,553 - harmonypy - INFO - Iteration 1 of 10
2025-06-09 21:28:34,773 - harmonypy - INFO - Iteration 2 of 10
2025-06-09 21:29:29,154 - harmonypy - INFO - Converged after 2 iterations
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm
```

Example 48 (python):
```python
%%time
# reclustering
# leiden clustering
resolutions_list = [0.2,0.4,0.6,1, 1.2,1.5,2]
for resolution in resolutions_list:
    sc.tl.leiden(adata_sub,neighbors_key=latent_space,
                 resolution=resolution,key_added=f'leiden_{str(resolution)}',n_iterations=2)
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                wspace=0.3)
```

Example 49 (unknown):
```unknown
CPU times: user 5min 22s, sys: 2.48 s, total: 5min 24s
Wall time: 5min 25s
```

Example 50 (python):
```python
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",size=10,
                color=genes,
               ncols=6, wspace=0.2,cmap='RdPu',vmax='p100')
```

Example 51 (python):
```python
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color=["conf_score_midmod2fine","celltypist_midmod2fine",],
                    wspace=0.2,cmap='RdPu',vmax='p100')
```

Example 52 (python):
```python
# umap per celltype to highlight
group_col = 'celltypist_midmod2fine'
groups_to_plot = list(set(adata_sub.obs[group_col]))

plt.rcParams['axes.grid'] = False
fig,axs=plt.subplots(nrows=2,ncols=4,figsize=(15,8))
# Flatten the axs array for easier iteration in case of multiple rows and columns
axs_flat = axs.flatten()

for i,grp in enumerate(groups_to_plot):
    sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                    color=[group_col], groups=grp, 
               size=10, frameon=False,
               show=False, ax=axs_flat[i], legend_loc=None, legend_fontsize=5,
               title=grp
              )
# Turn off any remaining subplots
for j in range(len(groups_to_plot), len(axs_flat)):
    fig.delaxes(axs_flat[j])

# fig.suptitle('Main Title for the Figure')
fig.tight_layout()
```

Example 53 (unknown):
```unknown
sub_resolution_sel = 2
```

Example 54 (python):
```python
markers_others = {
    'CM': ['TTN', 'TNNT2','MYH6',
        'MYH7', 'MYL2', 'FHL2', # vCN
            'NPPA', 'MYL7', 'MYL4'], #aCM
    'FB': ['DCN', 'GCN', 'PDGFRA','COL1A1','COL1A2'],
    'EC': ['VWF', 'PECAM1', 'CDH5','RGCC', 'FABP5',],
    'Peri': ['RGS5', 'ABCC9', 'KCNJ8'],
    'SMC': ['MYH11', 'TAGLN', 'ACTA2'],
    'Meso': ['MSLN', 'WT1', 'BNC1'],
    'Neuro': ['PLP1', 'NRXN1', 'NRXN3','PRPH', 'NEFL', 'NEFM', 'NEFH', 'STMN2', 'SST', 'SLC10A4', 'SLC18A3', 'TH', 'NPY'],
    'Adipo': ['GPAM', 'FASN', 'LEP'],
    'Myelo': ['CD14', 'C1QA', 'CD68','LYVE1','TIMD4'],
    'Lymph': ['CD8A', 'IL7R', 'CD40LG'],
    'B': ['CD19','SDC1','IGHG1','IGHA1'],
    'Mast': ['KIT', 'CPA3'],
}

for key in markers_others.keys():
    markers_others[key]=[x for x in markers_others[key] if x in adata.var_names]

sc.tl.dendrogram(adata_sub, groupby=f'leiden_{str(sub_resolution_sel)}')
sc.pl.dotplot(adata_sub,
              markers_others, 
              groupby=f'leiden_{str(sub_resolution_sel)}',
              dendrogram=True,
              standard_scale="var",
              color_map="Reds",
              swap_axes=False,
             )
```

Example 55 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: CM, FB, EC, etc.
```

Example 56 (python):
```python
sc.tl.dendrogram(adata_sub,groupby=f'leiden_{str(sub_resolution_sel)}')
sc.pl.dotplot(adata_sub,
              var_names=markers,
              groupby=f'leiden_{str(sub_resolution_sel)}',
              standard_scale='var',
              dendrogram=True
             )
```

Example 57 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: VentricularConductionSystemProximal, VentricularCardiomyocytesCycling, VentricularConductionSystemDistal, etc.
```

Example 58 (python):
```python
# DEG analysis
print(adata_sub.X.data[:5])
adata_sub.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata_sub,groupby=f'leiden_{str(sub_resolution_sel)}')
sc.tl.dendrogram(adata_sub, groupby=f'leiden_{str(sub_resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata_sub, dendrogram=True, standard_scale='var', return_fig=True)
dp.add_totals().show()
```

Example 59 (unknown):
```unknown
[2.4982107 2.4982107 2.4982107 2.4982107 3.5404415]
```

Example 60 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:458: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "names"] = self.var_names[global_indices]
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:460: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "scores"] = scores[global_indices]
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:463: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "pvals"] = pvals[global_indices]
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:473: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "pvals_adj"] = pvals_adj[global_indices]
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:484: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "logfoldchanges"] = np.log2(
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:458: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "names"] = self.var_names[global_indices]
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:460: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "scores"] = scores[global_indices]
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:463: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "pvals"] = pvals[global_indices]
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:473: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "pvals_adj"] = pvals_adj[global_indices]
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:484: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "logfoldchanges"] = np.log2(
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:458: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "names"] = self.var_names[global_indices]
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:460: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "scores"] = scores[global_indices]
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:463: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "pvals"] = pvals[global_indices]
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:473: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "pvals_adj"] = pvals_adj[global_indices]
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:484: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "logfoldchanges"] = np.log2(
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:458: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "names"] = self.var_names[global_indices]
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:460: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "scores"] = scores[global_indices]
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:463: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "pvals"] = pvals[global_indices]
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:473: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "pvals_adj"] = pvals_adj[global_indices]
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:484: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "logfoldchanges"] = np.log2(
```

Example 61 (python):
```python
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color=f'leiden_{str(sub_resolution_sel)}',legend_loc='on data',
                wspace=0.3)
```

Example 62 (python):
```python
# annotate
adata_sub.obs['fine_grain_manual'] = adata_sub.obs[f'leiden_{str(sub_resolution_sel)}'].copy()
adata_sub.obs.replace({'fine_grain_manual':{
    '0':'VentricularCardiomyocytes',
    '1':'VentricularCardiomyocytes',
    '2':'VentricularCardiomyocytes',
    '3':'VentricularCardiomyocytesCycling',
    '4':'VentricularCardiomyocytes',
    '5':'VentricularConductionSystemDistal',
    '6':'VentricularCardiomyocytes',
    '7':'VentricularCardiomyocytes',
    '8':'VentricularCardiomyocytes',
    '9':'VentricularCardiomyocytes',
    '10':'VentricularCardiomyocytes',
    '11':'VentricularCardiomyocytes',
    '12':'VentricularCardiomyocytes',
    '13':'VentricularCardiomyocytesCycling',
    '14':'?SkeletalMuscle',
    '15':'VentricularCardiomyocytesCycling',
    '16':'?ThyroidCells',
    '17':'VentricularCardiomyocytes',
    '18':'VentricularCardiomyocytes',
    '19':'VentricularCardiomyocytes',
    '20':'VentricularCardiomyocytes',
    '21':'VentricularCardiomyocytes',
    '22':'VentricularCardiomyocytes',
    '23':'VentricularCardiomyocytes',
    '24':'VentricularCardiomyocytes',
    '25':'VentricularCardiomyocytes',
    '26':'VentricularCardiomyocytes',
    '27':'VentricularCardiomyocytes',
}},inplace=True)
adata_sub.obs['fine_grain_manual'].value_counts()
```

Example 63 (python):
```python
/tmp/ipykernel_66828/1113603435.py:3: FutureWarning: The behavior of Series.replace (and DataFrame.replace) with CategoricalDtype is deprecated. In a future version, replace will only be used for cases that preserve the categories. To change the categories, use ser.cat.rename_categories instead.
  adata_sub.obs.replace({'fine_grain_manual':{
```

Example 64 (unknown):
```unknown
fine_grain_manual
VentricularCardiomyocytes            104353
VentricularCardiomyocytesCycling      17703
VentricularConductionSystemDistal      9595
?SkeletalMuscle                        2891
?ThyroidCells                           847
Name: count, dtype: int64
```

Example 65 (unknown):
```unknown
'VentricularCardiomyocytes'
```

Example 66 (python):
```python
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color='fine_grain_manual',
                wspace=0.3)
```

Example 67 (python):
```python
adata.obs['fine_grain_manual'] = adata.obs['tmp_annotation'].astype('str').copy()
adata.obs.loc[adata_sub.obs_names,'fine_grain_manual'] = adata_sub.obs['fine_grain_manual']
```

Example 68 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color='fine_grain_manual'
                )
```

Example 69 (python):
```python
for tissue_id in ['C194-HEA-0-FFPE-1', 'Hst45-HEA-0-FFPE-1']:
    slide = adata[adata.obs['tissue_block_id']==tissue_id]
    sc.pl.spatial(slide,color=['fine_grain_manual'],# groups='SinoatrialNodePacemakerCells',
                  spot_size=60,vmax='p100')
```

Example 70 (python):
```python
/tmp/ipykernel_66828/4023824168.py:3: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],# groups='SinoatrialNodePacemakerCells',
```

Example 71 (python):
```python
/tmp/ipykernel_66828/4023824168.py:3: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],# groups='SinoatrialNodePacemakerCells',
```

Example 72 (python):
```python
for ct in set(adata.obs['fine_grain_manual']):
    for tissue_id in ['C194-HEA-0-FFPE-1', 'Hst45-HEA-0-FFPE-1']:
        slide = adata[adata.obs['tissue_block_id']==tissue_id]
        sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
                      spot_size=30,vmax='p100')
```

Example 73 (python):
```python
/tmp/ipykernel_66828/3828796421.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 74 (python):
```python
/tmp/ipykernel_66828/3828796421.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 75 (python):
```python
/tmp/ipykernel_66828/3828796421.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 76 (python):
```python
/tmp/ipykernel_66828/3828796421.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 77 (python):
```python
/tmp/ipykernel_66828/3828796421.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 78 (python):
```python
/tmp/ipykernel_66828/3828796421.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 79 (python):
```python
/tmp/ipykernel_66828/3828796421.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 80 (python):
```python
/tmp/ipykernel_66828/3828796421.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 81 (python):
```python
/tmp/ipykernel_66828/3828796421.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 82 (python):
```python
/tmp/ipykernel_66828/3828796421.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 83 (python):
```python
/tmp/ipykernel_66828/3828796421.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 84 (python):
```python
/tmp/ipykernel_66828/3828796421.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 85 (python):
```python
# save
adata.write(f'{adata_dir}/{sample_id}_5K_{celltype}_lognorm.h5ad')
print(f'{adata_dir}/{sample_id}_5K_{celltype}_lognorm.h5ad')
```

Example 86 (unknown):
```unknown
/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium/subsets/C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat_5K_VentricularCardiomyocytes_lognorm.h5ad
```

---

## Variables¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/Xenium/no1_per-sample-processing/C194-HEA-0-FFPE-1/no2_celltypist_prediction.html

**Contents:**
- Variables¶
- Read in data¶
- Normalise the counts¶
- CellTypist prediction: coarse-grain¶
- CellTypist prediction: mid-grain¶
- celltypist prediction: fine-grain¶
- Save¶

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os

import celltypist
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
-----
celltypist          1.6.3
matplotlib          3.10.1
numpy               2.1.3
pandas              2.2.3
scanpy              1.11.1
seaborn             0.13.2
session_info        v1.0.1
-----
```

Example 4 (python):
```python
PIL                 11.1.0
anndata             0.11.4
asciitree           NA
asttokens           NA
certifi             2025.01.31
charset_normalizer  3.4.1
cloudpickle         3.1.1
comm                0.2.2
cycler              0.12.1
cython_runtime      NA
dask                2024.11.2
dateutil            2.9.0.post0
debugpy             1.8.13
decorator           5.2.1
exceptiongroup      1.2.2
executing           2.2.0
h5py                3.13.0
idna                3.10
igraph              0.11.8
importlib_metadata  NA
ipykernel           6.29.5
jedi                0.19.2
jinja2              3.1.6
joblib              1.4.2
kiwisolver          1.4.8
legacy_api_wrap     NA
leidenalg           0.10.2
llvmlite            0.44.0
markupsafe          3.0.2
mpl_toolkits        NA
msgpack             1.1.0
natsort             8.4.0
numba               0.61.0
numcodecs           0.13.1
packaging           24.2
parso               0.8.4
patsy               1.0.1
platformdirs        4.3.7
prompt_toolkit      3.0.50
psutil              7.0.0
pure_eval           0.2.3
pyarrow             19.0.1
pydev_ipython       NA
pydevconsole        NA
pydevd              3.2.3
pydevd_file_utils   NA
pydevd_plugins      NA
pydevd_tracing      NA
pygments            2.19.1
pyparsing           3.2.3
pytz                2025.2
requests            2.32.3
scipy               1.15.2
six                 1.17.0
sklearn             1.5.2
stack_data          0.6.3
statsmodels         0.14.4
tblib               3.1.0
texttable           1.7.0
threadpoolctl       3.6.0
tlz                 1.0.0
toolz               1.0.0
tornado             6.4.2
traitlets           5.14.3
typing_extensions   NA
urllib3             2.3.0
wcwidth             0.2.13
yaml                6.0.2
zarr                2.18.3
zipp                NA
zmq                 26.3.0
zoneinfo            NA
```

Example 5 (unknown):
```unknown
-----
IPython             8.34.0
jupyter_client      8.6.3
jupyter_core        5.7.2
-----
Python 3.10.14 | packaged by conda-forge | (main, Mar 20 2024, 12:45:18) [GCC 12.3.0]
Linux-4.18.0-553.45.1.el8_10.x86_64-x86_64-with-glibc2.28
-----
Session information updated at 2025-05-28 10:28
```

Example 6 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 7 (python):
```python
sample_id = 'C194-HEA-0-FFPE-1'
path_adata = f'/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium/{sample_id}_5K_filtered_raw.h5ad'
celltypist_model_dir = '/rfs/project/rfs-iCNyzSAaucw/kk837/notebooks/Foetal/Xenium/5K/celltypist_models'
```

Example 8 (unknown):
```unknown
os.getcwd()
```

Example 9 (unknown):
```unknown
'/rfs/project/rfs-iCNyzSAaucw/kk837/notebooks/Foetal/Xenium/5K/C194-HEA-0-FFPE-1'
```

Example 10 (python):
```python
adata = sc.read_h5ad(path_adata)
adata
```

Example 11 (unknown):
```unknown
AnnData object with n_obs × n_vars = 144916 × 5001
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts'
    obsm: 'spatial'
```

Example 12 (python):
```python
sc.pp.filter_genes(adata, min_cells=3)
```

Example 13 (python):
```python
adata.shape
```

Example 14 (unknown):
```unknown
(144916, 5001)
```

Example 15 (python):
```python
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)
```

Example 16 (python):
```python
adata.X.data[:5]
```

Example 17 (unknown):
```unknown
array([4.2093534, 2.6576407, 2.6576407, 3.3151019, 3.3151019],
      dtype=float32)
```

Example 18 (python):
```python
# coarse-grained
predictions_b2c = celltypist.annotate(adata, 
                                  model = f'{celltypist_model_dir}/global2coarse.pkl', 
                                  majority_voting = False)

# add to anndata
adata = predictions_b2c.to_adata(insert_labels = True, insert_conf = True)
adata.obs.rename(columns={'predicted_labels':'celltypist_coarse',
                              'conf_score':'conf_score_coarse'
                             },inplace=True)
adata.obs.head()
```

Example 19 (unknown):
```unknown
🔬 Input data has 144916 cells and 5001 genes
🔗 Matching reference genes in the model
🧬 4975 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
```

Example 20 (python):
```python
sc.pl.spatial(adata,color=['celltypist_coarse','conf_score_coarse'],cmap='RdPu',spot_size=20,vmax='p99',ncols=1)
```

Example 21 (python):
```python
# celltypist confidence
plt.figure(figsize=(6, 2))
sns.histplot(adata.obs[f'conf_score_coarse'], kde=False, bins=100)
# plt.yscale('log')
plt.title('coarse_grained')
plt.show()
```

Example 22 (python):
```python
# check
conf_thresh = 0.5 # starting from relaxed threshold
pd.crosstab(adata.obs['celltypist_coarse'],adata.obs[f'conf_score_coarse']<conf_thresh)
```

Example 23 (python):
```python
# finalised coarse-grained labels
conf_thresh = 0.5
adata.obs['celltypist_coarse_fin'] = adata.obs['celltypist_coarse'].astype('str').copy()
low_quality_cells = adata.obs_names[adata.obs[f'conf_score_coarse']<conf_thresh]
adata.obs.loc[low_quality_cells,'celltypist_coarse_fin'] = 'unassigned'
adata.obs['celltypist_coarse_fin'].value_counts()
```

Example 24 (unknown):
```unknown
celltypist_coarse_fin
Cardiomyocytes    73158
Mesenchymal       44487
Endothelium       12863
Neural             5842
Leukocytes         4566
Epicardium         3420
unassigned          580
Name: count, dtype: int64
```

Example 25 (python):
```python
sc.pl.spatial(adata,color=['celltypist_coarse_fin'],cmap='RdPu',spot_size=20,vmax='p99',ncols=1)
```

Example 26 (unknown):
```unknown
... storing 'celltypist_coarse_fin' as categorical
```

Example 27 (python):
```python
adata.obs['celltypist_coarse2midmod'] = np.nan
adata.obs['conf_score_coarse2midmod'] = np.nan

for celltype in adata.obs['celltypist_coarse_fin'].unique():
    if celltype not in ['unassigned', 'Epicardium']: # 'Epicardium' is also 'Epicardium' in mid-grained
        print(celltype)
        # subset
        adata_sub = adata[adata.obs['celltypist_coarse_fin']==celltype]
        # predict
        predictions = celltypist.annotate(adata_sub, 
                                      model = f'{celltypist_model_dir}/coarse2midmod_{celltype}.pkl', 
                                      majority_voting = False)
        # add to anndata
        adata_sub = predictions.to_adata(insert_labels = True, insert_conf = True)
        adata.obs.loc[adata_sub.obs_names,'celltypist_coarse2midmod'] = adata_sub.obs['predicted_labels']
        adata.obs.loc[adata_sub.obs_names,'conf_score_coarse2midmod'] = adata_sub.obs['conf_score']
        print('')
```

Example 28 (unknown):
```unknown
🔬 Input data has 44487 cells and 5001 genes
🔗 Matching reference genes in the model
```

Example 29 (unknown):
```unknown
Mesenchymal
```

Example 30 (unknown):
```unknown
🧬 4958 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
🔬 Input data has 5842 cells and 5001 genes
🔗 Matching reference genes in the model
🧬 4802 features used for prediction
```

Example 31 (unknown):
```unknown
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
```

Example 32 (unknown):
```unknown
Cardiomyocytes
```

Example 33 (unknown):
```unknown
🔬 Input data has 73158 cells and 5001 genes
🔗 Matching reference genes in the model
🧬 4867 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
🔬 Input data has 4566 cells and 5001 genes
🔗 Matching reference genes in the model
```

Example 34 (unknown):
```unknown
🧬 4947 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
🔬 Input data has 12863 cells and 5001 genes
🔗 Matching reference genes in the model
🧬 4916 features used for prediction
```

Example 35 (unknown):
```unknown
Endothelium
```

Example 36 (unknown):
```unknown
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
```

Example 37 (python):
```python
sc.pl.spatial(adata,color=['celltypist_coarse2midmod','conf_score_coarse2midmod'],
              cmap='RdPu',spot_size=20,vmax='p99',ncols=1)
```

Example 38 (unknown):
```unknown
... storing 'celltypist_coarse2midmod' as categorical
```

Example 39 (python):
```python
# check
conf_thresh = 0.8
pd.crosstab(adata.obs['celltypist_coarse2midmod'],adata.obs[f'conf_score_coarse2midmod']<conf_thresh)
```

Example 40 (python):
```python
# finalised mid-grained labels
conf_thresh = 0.8
adata.obs['celltypist_mid_fin'] = adata.obs['celltypist_coarse_fin'].astype('str').copy()
adata.obs.replace({'celltypist_mid_fin':{'Epicardium':'EpicardialCells'}},inplace=True)
passed = adata.obs_names[adata.obs['conf_score_coarse2midmod']>=conf_thresh]
low_quality_cells = adata.obs_names[adata.obs[f'conf_score_coarse2midmod']<conf_thresh]

adata.obs.loc[passed,'celltypist_mid_fin'] = adata.obs.loc[passed,'celltypist_coarse2midmod'].copy()
adata.obs.loc[low_quality_cells,'celltypist_mid_fin'] = 'unassigned'
adata.obs['celltypist_mid_fin'].value_counts()
```

Example 41 (unknown):
```unknown
celltypist_mid_fin
VentricularCardiomyocytes      57761
Fibroblasts                    28224
AtrialCardiomyocytes           15042
MuralCells                      7747
BloodVesselEndothelialCells     7183
PericardialCells                5402
unassigned                      4630
Glia                            4142
EpicardialCells                 3420
EndocardialCells                3158
LymphoidCells                   2727
LymphaticEndothelialCells       2287
MyeloidCells                    1626
Neurons                         1567
Name: count, dtype: int64
```

Example 42 (python):
```python
sc.pl.spatial(adata,color=['celltypist_mid_fin'],
              cmap='RdPu',spot_size=20,vmax='p99',ncols=1)
```

Example 43 (unknown):
```unknown
... storing 'celltypist_mid_fin' as categorical
```

Example 44 (python):
```python
adata.obs['celltypist_midmod2fine'] = np.nan
adata.obs['conf_score_midmod2fine'] = np.nan

for celltype in adata.obs['celltypist_mid_fin'].unique():
    if celltype not in ['unassigned', 'LymphaticEndothelialCells']: # 'LymphaticEndothelialCells' is also 'LymphaticEndothelialCells' in fine-grained
        print(celltype)
        # subset
        adata_sub = adata[adata.obs['celltypist_mid_fin']==celltype]
        # predict
        predictions = celltypist.annotate(adata_sub, 
                                      model = f'{celltypist_model_dir}/midmod2fine_{celltype}.pkl', 
                                      majority_voting = False)
        # add to anndata
        adata_sub = predictions.to_adata(insert_labels = True, insert_conf = True)
        adata.obs.loc[adata_sub.obs_names,'celltypist_midmod2fine'] = adata_sub.obs['predicted_labels']
        adata.obs.loc[adata_sub.obs_names,'conf_score_midmod2fine'] = adata_sub.obs['conf_score']
        print('')
```

Example 45 (unknown):
```unknown
🔬 Input data has 28224 cells and 5001 genes
🔗 Matching reference genes in the model
```

Example 46 (unknown):
```unknown
Fibroblasts
```

Example 47 (unknown):
```unknown
🧬 4923 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
🔬 Input data has 5402 cells and 5001 genes
🔗 Matching reference genes in the model
🧬 4811 features used for prediction
```

Example 48 (unknown):
```unknown
PericardialCells
```

Example 49 (unknown):
```unknown
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
🔬 Input data has 7747 cells and 5001 genes
🔗 Matching reference genes in the model
🧬 4846 features used for prediction
```

Example 50 (unknown):
```unknown
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
🔬 Input data has 4142 cells and 5001 genes
🔗 Matching reference genes in the model
🧬 4706 features used for prediction
```

Example 51 (unknown):
```unknown
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
🔬 Input data has 1567 cells and 5001 genes
🔗 Matching reference genes in the model
```

Example 52 (unknown):
```unknown
🧬 4674 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
```

Example 53 (unknown):
```unknown
VentricularCardiomyocytes
```

Example 54 (unknown):
```unknown
🔬 Input data has 57761 cells and 5001 genes
🔗 Matching reference genes in the model
🧬 4822 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
🔬 Input data has 1626 cells and 5001 genes
🔗 Matching reference genes in the model
🧬 4875 features used for prediction
```

Example 55 (unknown):
```unknown
MyeloidCells
```

Example 56 (unknown):
```unknown
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
🔬 Input data has 2727 cells and 5001 genes
🔗 Matching reference genes in the model
🧬 4889 features used for prediction
```

Example 57 (unknown):
```unknown
LymphoidCells
```

Example 58 (unknown):
```unknown
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
🔬 Input data has 7183 cells and 5001 genes
🔗 Matching reference genes in the model
🧬 4833 features used for prediction
```

Example 59 (unknown):
```unknown
BloodVesselEndothelialCells
```

Example 60 (unknown):
```unknown
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
🔬 Input data has 15042 cells and 5001 genes
🔗 Matching reference genes in the model
🧬 4709 features used for prediction
```

Example 61 (unknown):
```unknown
AtrialCardiomyocytes
```

Example 62 (unknown):
```unknown
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
🔬 Input data has 3158 cells and 5001 genes
🔗 Matching reference genes in the model
🧬 4675 features used for prediction
```

Example 63 (unknown):
```unknown
EndocardialCells
```

Example 64 (unknown):
```unknown
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
🔬 Input data has 3420 cells and 5001 genes
🔗 Matching reference genes in the model
🧬 4829 features used for prediction
```

Example 65 (unknown):
```unknown
EpicardialCells
```

Example 66 (unknown):
```unknown
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
```

Example 67 (python):
```python
sc.pl.spatial(adata,color=['celltypist_midmod2fine','conf_score_midmod2fine'],
              cmap='RdPu',spot_size=20,vmax='p99',ncols=1)
```

Example 68 (unknown):
```unknown
... storing 'celltypist_midmod2fine' as categorical
```

Example 69 (python):
```python
sc.pl.spatial(adata,color=['celltypist_midmod2fine'],
              groups=['SinoatrialNodePacemakerCells','AtrioventricularNodePacemakerCells'],
              cmap='RdPu',spot_size=20,vmax='p99',ncols=3)
```

Example 70 (python):
```python
sc.pl.spatial(adata,color=['celltypist_midmod2fine'],
              groups=['MacrophagesCX3CR1pos','MacrophagesLYVE1pos'],
              cmap='RdPu',spot_size=20,vmax='p99',ncols=3)
```

Example 71 (python):
```python
adata.write(f'/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium/{sample_id}_5K_filtered_lognorm.h5ad')
```

Example 72 (python):
```python
print(f'/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium/{sample_id}_5K_filtered_lognorm.h5ad')
```

Example 73 (unknown):
```unknown
/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium/C194-HEA-0-FFPE-1_5K_filtered_lognorm.h5ad
```

---

## Subcluster¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/RNA/scnRNA_Integration-Annotation/Epicardium/PericardiumAndEpicardium.html

**Contents:**
- Subcluster¶

**Examples:**

Example 1 (unknown):
```unknown
#Added the keys for age and sex
```

Example 2 (python):
```python
# Import packages, initial settings

import numpy as np
import matplotlib.pyplot as plt
import scanpy as sc
import pandas as pd
import anndata
import scrublet as scr
import seaborn as sns
import magic
import pybiomart
import six
import sys
sys.modules['sklearn.externals.six'] = six
import scanorama
import harmonypy
import bbknn
import gseapy
from gseapy.plot import barplot, dotplot
import scvelo as scv
import scvi

plt.rcParams['pdf.fonttype'] = 42
plt.rcParams['ps.fonttype'] = 42

sc.settings.verbosity = 3  # verbosity: errors (0), warnings (1), info (2), hints (3)

from matplotlib.colors import LinearSegmentedColormap
cmap = LinearSegmentedColormap.from_list(name='gene_cmap', colors=['lightgrey', 'thistle', 'red', 'darkred']) 

sc.settings.set_figure_params(dpi=80, color_map='viridis', vector_friendly=False,  dpi_save=300, facecolor='white')
sc.logging.print_version_and_date()
#sc.logging.print_versions()
```

Example 3 (unknown):
```unknown
Global seed set to 0
```

Example 4 (unknown):
```unknown
Running Scanpy 1.8.1, on 2023-09-01 22:02.
```

Example 5 (python):
```python
adata=sc.read('./Objects/Feb28ObjectRaw.h5ad')
print(adata.shape)
```

Example 6 (unknown):
```unknown
(297473, 36601)
```

Example 7 (python):
```python
cell_cycle_genes = ['MCM5','PCNA','TYMS','FEN1','MCM2','MCM4','RRM1','UNG','GINS2','MCM6','CDCA7','DTL','PRIM1','UHRF1','MLF1IP','HELLS','RFC2','RPA2','NASP','RAD51AP1','GMNN','WDR76','SLBP','CCNE2','UBR7','POLD3','MSH2','ATAD2','RAD51','RRM2','CDC45','CDC6','EXO1','TIPIN','DSCC1','BLM','CASP8AP2','USP1','CLSPN','POLA1','CHAF1B','BRIP1','E2F8','HMGB2','CDK1','NUSAP1','UBE2C','BIRC5','TPX2','TOP2A','NDC80','CKS2','NUF2','CKS1B','MKI67','TMPO','CENPF','TACC3','FAM64A','SMC4','CCNB2','CKAP2L','CKAP2','AURKB','BUB1','KIF11','ANP32E','TUBB4B','GTSE1','KIF20B','HJURP','CDCA3','HN1','CDC20','TTK','CDC25C','KIF2C','RANGAP1','NCAPD2','DLGAP5','CDCA2','CDCA8','ECT2','KIF23','HMMR','AURKA','PSRC1','ANLN','LBR','CKAP5','CENPE','CTCF','NEK2','G2E3','GAS2L3','CBX5','CENPA']
s_genes = cell_cycle_genes[:43]
s_genes = [x for x in s_genes if x in adata.var_names]
print('S genes:', s_genes, '\n')
g2m_genes = cell_cycle_genes[43:]
g2m_genes = [x for x in g2m_genes if x in adata.var_names]
print('G2 genes:', g2m_genes, '\n')
cell_cycle_genes = [x for x in cell_cycle_genes if x in adata.var_names]

hb_genes1 = ['HBB','HBD','HBG1','HBG2','HBE1','HBZ','HBM','HBA2','HBA1', 'HBQ1']
hb_genes1 = [x for x in hb_genes1 if x in adata.var_names]
print('Hemoglobin genes set 1:', hb_genes1, '\n')

hb_genes2 = ['AHSP','GYPA','RHAG','GFI1B','KEL','EPB42','ALAS2','TSPO2','GYPB','GYPE','SPTA1','RHCE','KLF1','RHD','SLC4A1','SNX22','SQSTM1','SOD1','FTH1','MAP1LC3B','SLC25A37','TRIM58','HMBS']
hb_genes2 = [x for x in hb_genes2 if x in adata.var_names]
print('Hemoglobin genes set 2:', hb_genes2, '\n')

mito_genes=['MT-ND1','MT-ND2','MT-CO1','MT-CO2','MT-ATP8','MT-ATP6','MT-CO3','MT-ND3','MT-ND4L', 'MT-ND4', 'MT-ND5', 'MT-ND6', 'MT-CYB']
mito_genes = [x for x in mito_genes if x in adata.var_names]
print('Mitochondrial genes:', mito_genes, '\n')

ribo_genes = ['RPLP0','RPLP1','RPLP2','RPL3','RPL3L','RPL4','RPL5','RPL6','RPL7','RPL7A','RPL7L1','RPL8','RPL9','RPL10','RPL10A','RPL10L','RPL11','RPL12','RPL13','RPL13A','RPL14','RPL15','RPL17','RPL18','RPL18A','RPL19','RPL21','RPL22','RPL22L1','RPL23','RPL23A','RPL24','RPL26','RPL26L1','RPL27','RPL27A','RPL28','RPL29','RPL30','RPL31','RPL32','RPL34','RPL35','RPL35A','RPL36','RPL36A','RPL36AL','RPL37','RPL37A','RPL38','RPL39','RPL39L','UBA52','RPL41','RPSA','RPS2','RPS3','RPS3A','RPS4X','RPS4Y1','RPS4Y2','RPS5','RPS6','RPS7','RPS8','RPS9','RPS10','RPS11','RPS12','RPS13','RPS14','RPS15','RPS15A','RPS16','RPS17','RPS18','RPS19','RPS20','RPS21','RPS23','RPS24','RPS25','RPS26','RPS27','RPS27A','RPS27L','RPS28','RPS29','FAU']
ribo_genes = [x for x in ribo_genes if x in adata.var_names]
print('Ribo genes:', ribo_genes, '\n')

stress_genes=['FOS','FOSB','JUN','JUNB','JUND','ZFP36','SOCS3','CEBPD','CEBPB','ATF3','EGR1','HSPA1A','HSPA1B','HSP90AA1','HSP90AB1','HSPA8','HSPB1','HSPE1','IER3','IER2','BTG1','BTG2','DUSP1']
stress_genes = [x for x in stress_genes if x in adata.var_names]
print('Tissue dissociation stress associated genes:', stress_genes, '\n')

FoxProteins=['FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1']
chrY_genes=['SRY', 'RPS4Y1', 'ZFY', 'ZFY-AS1', 'LINC00278', 'PCDH11Y', 'TTTY2B', 'TBL1Y', 'PRKY', 'USP9Y', 'DDX3Y', 'UTY', 'TMSB4Y', 'NLGN4Y', 'TTTY14', 'KDM5D', 'TTTY10', 'EIF1AY', 'PRORY', 'LINC00266-4P']
MembraneProteins1=['AADAT','AAK1','ABCA12','ABCA7','ABCB1','ABCB11','ABCB4','ABCB6','ABCC1','ABCC3','ABCC4','ABCC5','ABCC6','ABCG2','ABHD3','ABI1','ABLIM3','AC004691.2','AC004922.1','AC005324.3','AC005726.1','AC008403.1','AC138696.1','ACIN1','ACKR3','ACLY','ACOT13','ACOT9','ACTN1','ACTR10','ACTR6','ADA','ADAM10','ADAM12','ADAM22','ADAM28','ADAM33','ADAMTS1','ADAMTS2','ADAMTSL4','ADAP1','ADCK5','ADD1','ADD2','ADGRB2','ADGRD1','ADGRG2','ADGRG7','ADGRL2','ADH1A','ADH1B','ADH1C','ADH7','ADK','ADORA1','ADRA1B','ADRM1','ADSS','AFAP1L2','AFDN','AFG1L','AGER','AGRN','AHNAK','AHNAK2','AIFM2','AKAP10','AKAP11','AKAP12','AKR1B10','AKR1B15','AL132780.3','AL139300.1','AL355315.1','AL358113.1','AL645922.1','ALDH3A1','ALK','ALOX15B','ALOXE3','ALPI','ALPP','ALPPL2','ALS2CR12','AMACR','AMDHD1','AMER1','AMER2','AMIGO2','AMOT','AMOTL1','AMOTL2','AMPH','ANK2','ANK3','ANKEF1','ANKFN1','ANKIB1','ANKLE2','ANKRA2','ANKRD13A','ANKRD35','ANKRD42','ANKRD45','ANKS1B','ANO1','ANO10','ANO2','ANO6','ANPEP','ANXA1','ANXA13','ANXA2','ANXA3','AP2M1','APBB1','APBB1IP','APC','APLP1','APMAP','APOC3','APOD','APOLD1','AQP1','AQP10','AQP3','AQP4','AQP5','ARAP1','ARF1','ARHGAP18','ARHGAP19','ARHGAP21','ARHGAP22','ARHGAP23','ARHGAP24','ARHGAP28','ARHGAP29','ARHGAP33','ARHGAP36','ARHGAP40','ARHGAP9','ARHGDIG','ARHGEF1','ARHGEF11','ARHGEF12','ARHGEF15','ARHGEF26','ARHGEF28','ARHGEF40','ARHGEF5','ARID2','ARID4A','ARID5B','ARL13A','ARL15','ARL4C','ARMC12','ARMCX5','ARPC1A','ARPC5','ARRB2','ARRDC4','ARVCF','ASAP1','ASB5','ASGR1','ASGR2','ASH2L','ASIC1','ASIC4','ASPM','ASPSCR1','ATG13','ATG3','ATG7','ATIC','ATOX1','ATP10D','ATP12A','ATP13A3','ATP1B3','ATP23','ATP2B1','ATP2B4','ATP2C2','ATP6V0A2','ATP6V1H','ATP8A2','ATP8B1','ATXN10','ATXN3','ATXN7L3','AVEN','AVPI1','AXIN2','AXL','B2M','B3GALT2','BACE1','BAG3','BAIAP2','BAIAP2L1','BANK1','BASP1','BBOF1','BCAR1','BCAR3','BCL2L12','BCORL1','BCR','BDKRB2','BEND6','BFSP1','BFSP2','BICD2','BIN2','BLNK','BLVRB','BMPR2','BMX','BOC','BOLL','BORCS5','BORCS8-MEF2B','BRK1','BRPF1','BRSK1','BSDC1','BST2','BTK','BTN2A1','BTRC','BUB1B-PAK6','BVES','BZW2','C10orf90','C11orf42','C11orf52','C12orf73','C12orf76','C14orf37','C14orf93','C16orf74','C17orf58','C17orf62','C18orf21','C19orf33','C19orf38','C19orf48','C19orf71','C1orf116','C1orf189','C1orf35','C1orf54','C1orf68','C1orf74','C1QBP','C2orf72','C3orf14','C3orf20','C3orf38','C4orf19','C4orf22','C7orf43','C8orf33','C8orf37','C9orf153','C9orf3','CA14','CA9','CABLES2','CACNA1E','CACNG4','CADM1','CALCRL','CALD1','CALML5','CALN1','CAMK2A','CAMK2B','CAMK2D','CAMK2G','CAP2','CAPS','CASKIN2','CASP4','CAST','CAV3','CAVIN1','CAVIN2','CAVIN3','CAVIN4','CBWD1','CBWD2','CBWD3','CBWD5','CBWD6','CC2D1A','CCBE1','CCDC107','CCDC112','CCDC115','CCDC124','CCDC136','CCDC144NL','CCDC15','CCDC154','CCDC170','CCDC24','CCDC28A','CCDC59','CCDC62','CCDC66','CCDC69','CCDC8','CCDC83','CCDC85C','CCDC93','CCDC97','CCNT2','CCNYL1','CCR3','CCR4','CCT3','CD109','CD163','CD164L2','CD200R1','CD226','CD248','CD274','CD2AP','CD300A','CD300LB','CD33','CD34','CD37','CD38','CD3E','CD3G','CD4','CD44','CD46','CD47','CD48','CD79B','CD81','CD84','CD86','CD8A','CD8B','CD9','CD93','CD99L2','CDAN1','CDC42EP1','CDC42EP3','CDC42EP4','CDC42SE1','CDCA4','CDH1','CDH13','CDH15','CDH17','CDH2','CDH26','CDH3','CDH4','CDH5','CDH6','CDH8','CDIPT','CDK16','CDK5','CDK5RAP2','CDK7','CDRT15','CDV3','CEACAM5','CELSR1','CENPJ','CEP162','CEP295','CEP55','CEP57L1','CEP89','CERCAM','CFAP157','CFAP206','CFAP58','CFAP65','CFB','CFL2','CFLAR','CGN','CGNL1','CHCHD7','CHIC2','CHMP2A','CHRM3','CHRNA1','CHRNA5','CHRNB4','CHRND','CIB1','CIB3','CIP2A','CKAP5','CLCA2','CLCA4','CLCN5','CLCN6','CLDN11','CLDN18','CLDN2','CLDN23','CLDN3','CLDN4','CLDN7','CLDN9','CLEC2A','CLEC4F','CLEC5A','CLECL1','CLIC2','CLIC5','CLTB','CLUL1','CMTM4','CMTM7','CMTR2','CMYA5','CNGA1','CNKSR2','CNNM1','CNNM4','CNP','CNR1','CNTD1','COBL','COBLL1','COG3','COL16A1','COL17A1','COL4A5','COL6A2','COLEC12','COLQ','COMMD4','COMTD1','COPRS','COQ7','CORIN','CORO1B','CORO2A','CORO2B','COX6B2','CPEB3','CPPED1','CPZ','CRB3','CREB5','CRIP1','CRIP2','CRK','CRLF3','CROCC','CRTC1','CRYAB','CRYBG3','CRYL1','CSDE1','CSF1','CSF1R','CSPG4','CST6','CTNNA1','CTNNB1','CTNNBIP1','CTNND1','CTNND2','CTSF','CTSV','CTTN','CTXN1','CUEDC1','CUTA','CX3CL1','CX3CR1','CXADR','CXCR2','CYB5D2','CYBRD1','CYFIP2','CYP20A1','CYP24A1','CYTH2','CYTH4','DAAM1','DAB2','DAG1','DAGLB','DAW1','DBN1','DCAF13','DCAF16','DCBLD2','DCHS1','DCHS2','DCP2','DCT','DCTN6','DCUN1D2','DDAH1','DDB2','DDIAS','DDIT4L','DDR1','DDR2','DDX58','DDX60L','DENND6B','DFFA','DGKG','DGKH','DHDDS','DHPS','DHRS7B','DIAPH1','DIAPH3','DIO2','DIS3L','DLG1','DLG2','DLG5','DLGAP3','DLL3','DMWD','DNAAF1','DNAAF4','DNAH17','DNAJA4','DNAJB13','DNAJB4','DNAJC18','DNAJC5','DNAJC6','DNAJC9','DOC2A','DOCK4','DOK3','DONSON','DPEP1','DPH1','DPPA5','DPYSL2','DQX1','DRD1','DRP2','DSC3','DSEL','DSG2','DSP','DTNA','DUS1L','DUSP15','DXO','DYSF','ECSCR','EDA2R','EDNRA','EDNRB','EEF1AKMT1','EEF2','EFCAB1','EFCAB11','EFCAB2','EFCAB7','EFHC2','EFNB1','EFR3A','EFR3B','EGFR','EHBP1','EHD1','EHD2','EHD3','EHD4','EIF2B2','EIF5','EIF5B','ELFN1','ELMO3','ELN','ELOC','EMP3','ENAH','ENC1','ENG','ENKD1','ENO1','ENO2','ENO3','ENOX1','ENOX2','EPB41','EPB41L1','EPB41L2','EPB41L3','EPB41L4A','EPB41L4B','EPB41L5','EPCAM','EPHA2','EPHA3','EPHB1','EPHB2','EPN1','EPOR','EPS8L2','ERAP1','ERBB2','ERBB3','ERBIN','ERC1','ERCC4','ERMN','ESCO2','ETNK1','ETS2','EVA1A','EVI2B','EXOC1','EXOC3L1','EXOC7','EZR','F11R','F2RL1','F2RL3','FAAP20','FABP2','FABP3','FABP5','FAIM','FAM126A','FAM126B','FAM129A','FAM129B','FAM13A','FAM163B','FAM171A1','FAM193A','FAM205A','FAM210B','FAM222A','FAM45A','FAM47E','FAM47E-STBD1','FAM72A','FAM72B','FAM72C','FAM72D','FANCG','FAS','FASN','FBLIM1','FBLN2','FBLN5','FBLN7','FBP2','FBRSL1','FBXL2','FBXO16','FBXO38','FBXO41','FBXW11','FCAR','FCGBP','FCGR1A','FCGR1B','FCGR2A','FCHSD2','FFAR4','FGD1','FGD5','FGFR2','FGR','FHIT','FHL1','FIBCD1','FIGN','FILIP1','FILIP1L','FKBP2','FLAD1','FLCN','FLNA','FLNB','FLNC','FLOT1','FLOT2','FLRT3','FLT1','FLT4','FLVCR1','FMN2','FMNL2','FMNL3','FNBP1','FOXA2','FOXN3','FP565260.3','FRMD3','FRMD7','FRMD8','FRS2','FSCN1','FTCD','FUT2','FXYD3','FYB1','FYN','FZD1','FZD2','FZD4','FZD6','GAB1','GAB2','GABRA1','GABRA2','GABRA3','GABRA5','GABRG1','GAD1','GADL1','GAGE1','GAGE10','GAGE12C','GAGE12D','GAGE12E','GAGE12F','GAGE12G','GAGE12H','GAGE12J','GAGE13','GAGE2A','GAGE2E','GALNT10','GALR1','GAP43','GAPDH','GAPT','GAPVD1','GAS2','GAS2L1','GAS7','GAS8','GBP4','GCA','GCC1','GCLM','GCSAML-AS1','GFI1B','GFOD1','GFRA3','GHR','GID8','GINM1','GIPC1','GJA1','GJB3','GJB4','GJB6','GJB7','GJC1','GJC2','GLO1','GLP2R','GLRX','GLUL','GMIP','GNAI2','GNAQ','GNAS','GNB1','GNB2','GNB3','GNB4','GNG2','GP9','GPAT3','GPATCH1','GPBP1','GPBP1L1','GPC1','GPC3','GPC4','GPI','GPR1','GPR142','GPR156','GPR157','GPR160','GPR173','GPR20','GPR27','GPR50','GPR63','GPRC5A','GPRIN1','GPRIN2','GPX8','GRAMD1A','GRAMD1B','GRAMD1C','GRAP2','GRB7','GRID2IP','GRIK5','GRK3','GRK5','GRPR','GSDMA','GTF2F1','GTSE1','GUCA1A','GYPA','GYPC','HAPLN3','HARBI1','HAS1','HAUS4','HAUS7','HCAR2','HCAR3','HCFC2','HCK','HCLS1','HCST','HDAC8','HEG1','HELZ','HENMT1','HERC2','HERPUD1','HFE','HIF1A','HIF3A','HINT1','HIP1R','HLA-A','HLA-DPB1','HMGCS1','HMOX1','HOMER3','HOXC5','HOXD4','HPS3','HRASLS5','HRH1','HS6ST3','HSH2D','HSPB1','HSPG2','HTR1B','HTR7','HTRA1','HUNK','HYDIN','HYI','HYLS1','ICAM1','ICAM2','ICOS','ICOSLG','IFFO2','IFIT5','IFNGR1','IFRD1','IFT122','IGFL2','IGSF11','IGSF23','IKZF5','IL10RA','IL12RB1','IL12RB2','IL13','IL13RA2','IL16','IL17RB','IL1R1','IL1RL1','IL21R','IL4R','IL6ST','IL7R','ILDR2','ILK','INPP5A','INSR','INTS12','IP6K2','IPCEF1','IQGAP1','IQGAP2','IREB2','IRF2','IRS1','ISLR','ITGA2B','ITGA3','ITGA4','ITGA7','ITGA8','ITGA9','ITGB1','ITGB2','ITGB3','ITGB4','ITGB5','ITGB6','ITGB7','ITGB8','ITK','ITM2A','ITPKA','ITSN1','ITSN2','IYD','JAG1','JAK2','JAK3','JAM2','JAML','JCAD','JMJD4','JPT2','JUP','KANK1','KANK3','KANSL2','KARS','KATNB1','KCNAB1','KCNB2','KCNH8','KCNJ15','KCNK13','KCNK17','KCNK6','KCNK7','KCNK9','KCNN2','KCNS3','KCNU1','KDF1','KDR','KEL','KIAA0556','KIAA0895','KIAA1324','KIAA1522','KIAA1958','KIF17','KIF19','KIF21A','KIF21B','KIF26A','KIF26B','KISS1R','KIT','KLC1','KLC2','KLF9','KLHDC7B','KLHL41','KLHL6','KLHL7','KLK7','KLRC1','KLRC2','KLRF1','KLRK1','KMT2D','KNDC1','KNSTRN','KNTC1','KPTN','KRT75','KRT77','KRTAP25-1','L1CAM','L3MBTL1','LAIR1','LAMA4','LAMC1','LAMP2','LAMTOR1','LAPTM4A','LASP1','LAT','LAT2','LAX1','LCP1','LCTL','LDB2','LDLRAD3','LENG1','LGALS3','LGALS4','LGALS8','LGR6','LHFPL2','LHX6','LIMA1','LINGO1','LIPJ','LMCD1','LMOD2','LMTK2','LMTK3','LNX1','LNX2','LONRF1','LONRF3','LOXHD1','LPAR1','LPAR3','LPAR4','LPIN1','LPIN2','LRCH2','LRP11','LRP12','LRRC15','LRRC26','LRRC34','LRRC45','LRRC69','LRRC7','LRRC9','LRRFIP2','LRRIQ3','LSMEM2','LSR','LTB4R2','LUZP2','LY6K','LYN','LYPD1','LYST','LZTS1','LZTS2','MACF1','MADD','MAGI1','MAGI3','MAMDC2','MAP1S','MAP2','MAP2K1','MAP2K4','MAP2K7','MAP3K12','MAP3K21','MAP3K6','MAP3K7','MAP4','MAP4K1','MAP4K5','MAP6','MAPK15','MAPK8IP1','MAPT','MARCH10','MARCH7','MARCKS','MARK2','MARVELD2','MAS1L','MAT2B','MBP','MC5R','MCAM','MCEMP1','MCF2L2','MCMBP','MCTS1','MDGA1','ME1','MEF2B','MEIOC','MEPCE','MERTK','MET','METAP1','METAP2','METTL15','METTL25','METTL9','MFAP3L','MFSD1','MFSD2A','MGAT4C','MIB1','MICAL2','MICAL3','MICALL1','MICB','MIEN1','MIGA2','MISP','MKLN1','MKRN3','MLF2','MLKL','MLST8','MMACHC','MMP10','MMP15','MMP24','MMP25','MOAP1','MPHOSPH8','MPHOSPH9','MPI','MPIG6B','MPL','MPP1','MPP5','MPP7','MPZL1','MPZL2','MR1','MRGPRF','MRNIP','MRPL42','MRPL44','MRPL46','MRPS18B','MS4A1','MS4A12','MS4A14','MS4A15','MSN','MSRA','MTCP1','MTFR1L','MTHFR','MTM1','MTPN','MTSS1','MUC1','MUC20','MVD','MYH11','MYH9','MYL4','MYLK','MYO10','MYO1B','MYO1C','MYO1E','MYO1G','MYO3A','MYO5B','MYO9A','MYOF','N4BP2','NAA35','NAB1','NAIF1','NAMPT','NAP1L2','NAPG','NARFL','NCAM1','NCAM2','NCK1','NCKAP1L','NCKAP5','NCKIPSD','NCOA1','NCS1','NDC1','NECTIN2','NECTIN3','NECTIN4','NEDD1','NEDD9','NEO1','NEU1','NEXN','NF2','NFE2L2','NFKBID','NGEF']
MembraneProteins2=['NGFR','NHS','NHSL2','NID2','NINJ2','NKTR','NKX2-8','NLGN1','NLGN3','NLK','NLRX1','NMB','NMBR','NME1','NME2','NMT1','NMT2','NMUR2','NOS1','NOTCH2','NPFFR2','NPL','NPNT','NPR1','NPTN','NPTXR','NR2C1','NRAS','NRCAM','NRP2','NRSN1','NRXN1','NSD1','NSFL1C','NT5E','NTNG1','NTNG2','NTSR1','NUDCD3','NUDT16L1','NUDT9','NUMB','NUP35','NUS1','OAS3','OBSCN','OCA2','OCLN','OCM','OCM2','ODC1','ODR4','OGT','OLFM4','OLIG2','OLR1','OPHN1','OPRK1','OR10P1','OR1N2','OR2C3','OR6B2','OR6B3','OR6C6','OR8S1','OSBPL10','OSBPL6','OSBPL7','OSCAR','OSCP1','OSGEP','OSR2','OTOP2','OTULIN','OXGR1','P3H3','PACSIN2','PACSIN3','PAFAH1B2','PAG1','PAIP1','PAK1','PAK4','PAK6','PALLD','PALM','PALM2','PALM2-AKAP2','PALM3','PALMD','PANX1','PANX2','PAQR8','PARD3','PARD3B','PARD6A','PARD6G','PARP6','PATJ','PAWR','PCBD2','PCDH1','PCDH17','PCDH7','PCDHA1','PCDHA11','PCDHA12','PCDHA4','PCDHA5','PCDHB1','PCDHB11','PCDHB12','PCDHB16','PCDHB4','PCDHB5','PCDHGA1','PCDHGA10','PCDHGA11','PCDHGA12','PCDHGA2','PCDHGA3','PCDHGA4','PCDHGA5','PCDHGA6','PCDHGA7','PCDHGA8','PCDHGA9','PCDHGB1','PCDHGB2','PCDHGB3','PCDHGB4','PCDHGB5','PCDHGB6','PCDHGB7','PCDHGC3','PCDHGC4','PCDHGC5','PCLO','PCMTD1','PDAP1','PDCD7','PDE3A','PDE4A','PDE4D','PDE9A','PDGFC','PDGFRA','PDHX','PDLIM1','PDLIM5','PDPN','PDZD2','PDZD8','PEAR1','PEBP1','PFDN1','PGAP3','PGLYRP2','PGLYRP4','PGRMC2','PHACTR1','PHACTR4','PHF20L1','PHGDH','PHLDB2','PHLDB3','PHOSPHO1','PHPT1','PI3','PI4K2A','PI4KA','PIEZO2','PIGB','PIGN','PIGW','PIGX','PIK3AP1','PIK3CG','PIK3R6','PIP4K2A','PIP4K2C','PIP5K1A','PIRT','PITPNM3','PKD2L2','PKN1','PKN2','PKP1','PKP2','PKP3','PKP4','PLAA','PLAUR','PLCB1','PLCB2','PLCD3','PLCD4','PLEKHA4','PLEKHA6','PLEKHA7','PLEKHG3','PLEKHG4','PLEKHG6','PLEKHH2','PLEKHO2','PLIN4','PLPP1','PLPPR5','PLS3','PLSCR1','PLVAP','PLXNA3','PLXNA4','PNCK','PNISR','PODXL','POLE','POU2F3','PPEF1','PPFIBP1','PPIP5K1','PPL','PPM1A','PPM1F','PPP1CA','PPP1CB','PPP1R12B','PPP1R16A','PPP1R16B','PPP1R18','PPP1R27','PPP1R9A','PPP2R1B','PPP3R1','PPP3R2','PPP4C','PPP6R3','PPTC7','PRAME','PRC1','PRDX6','PRIMA1','PRKCA','PRKCE','PRKCH','PRKCZ','PRKD1','PROM2','PROSER2','PRR15','PRR32','PRR7','PRRC2A','PRRG1','PRRT2','PRRT3','PRRT4','PRTG','PRX','PSCA','PSD','PSD2','PSEN1','PSMC5','PSMC6','PSMD9','PSTPIP2','PTAFR','PTCHD4','PTK2B','PTK6','PTPN12','PTPN21','PTPN22','PTPN4','PTPRB','PTPRG','PTPRK','PTPRM','PTPRR','PTPRS','PTX3','PURG','PVALB','PVR','PXDC1','PXK','PYGL','PYGO2','PYM1','RAB13','RAB23','RAB25','RAB26','RAB27B','RAB3B','RAB3GAP2','RAB43','RAB8A','RABEPK','RAD51C','RAD52','RAI2','RALA','RALGAPA2','RAPGEF2','RAPH1','RAPSN','RASA4','RASA4B','RASAL1','RASAL2','RASAL3','RASD1','RASD2','RASGRP1','RASGRP2','RASGRP4','RASSF2','RASSF3','RASSF4','RBM46','RC3H1','RCCD1','RCSD1','RD3L','RDX','RECK','RELL1','REPS1','RET','RFLNB','RFPL1','RFPL2','RFPL3','RGP1','RGS10','RGS14','RGS19','RGS3','RGS7BP','RGS9BP','RHCE','RHD','RHEX','RHOA','RHOBTB2','RHOJ','RHOXF2','RHOXF2B','RIC8A','RIC8B','RIF1','RILPL1','RIMBP3','RIMBP3B','RIMBP3C','RIMS2','RIOK2','RIOK3','RIPK1','RND1','RNF114','RNF145','RNF146','RNF149','RNF165','RNF166','RNF39','ROBO1','ROBO3','ROBO4','ROM1','RP2','RPE','RPEL1','RPGRIP1L','RPIA','RPL36A','RPL36A-HNRNPH2','RPL36AL','RPSA','RPUSD1','RRAD','RRP12','RRP7A','RSBN1L','RSC1A1','RSPH3','RTL3','RTL4','RTN4R','RTN4RL1','RYR2','S100A13','S100A14','S100A16','S100A2','S100A3','S100A4','S100A6','S100A9','SAMD4A','SAMHD1','SAMSN1','SAPCD1','SASH3','SC5D','SCAMP1','SCAMP4','SCEL','SCHIP1','SCIN','SCN1A','SCN5A','SCN8A','SCN9A','SCNN1D','SCNN1G','SCRIB','SCUBE3','SDC2','SDC4','SDE2','SDHB','SEC16B','SEMA3C','SEMA3D','SEMA4B','SEMA4F','SEMA6C','SEMA6D','SEPT2','SEPT3','SERINC2','SERINC5','SERPINB3','SFSWAP','SGCE','SGK494','SH3BGR','SH3D21','SHANK1','SHANK3','SHARPIN','SHISA6','SHKBP1','SHROOM2','SHTN1','SIGLEC10','SIGLEC14','SIGLEC6','SIGLEC7','SIN3B','SIPA1','SIPA1L1','SIPA1L3','SKAP1','SLAMF6','SLC10A5','SLC12A2','SLC13A3','SLC13A5','SLC14A1','SLC16A1','SLC16A10','SLC16A2','SLC16A3','SLC16A7','SLC16A9','SLC17A5','SLC19A1','SLC19A3','SLC1A5','SLC1A6','SLC22A16','SLC23A3','SLC24A4','SLC26A9','SLC27A1','SLC28A1','SLC2A1','SLC2A11','SLC2A12','SLC2A14','SLC2A2','SLC2A3','SLC30A1','SLC31A2','SLC34A1','SLC34A2','SLC35F5','SLC35G2','SLC36A4','SLC38A5','SLC38A6','SLC38A7','SLC39A10','SLC39A14','SLC39A2','SLC39A6','SLC3A2','SLC40A1','SLC41A3','SLC43A1','SLC43A3','SLC44A2','SLC45A4','SLC46A1','SLC46A3','SLC4A1AP','SLC4A2','SLC4A7','SLC4A8','SLC5A12','SLC5A3','SLC5A6','SLC5A7','SLC6A6','SLC7A14','SLC7A2','SLC7A5','SLC7A6','SLC8A1','SLC9A1','SLC9A2','SLC9A3','SLC9A3R1','SLC9A3R2','SLC9A4','SLC9A5','SLC9B2','SLC9C2','SLCO1A2','SLCO1B3','SLCO4A1','SLCO4C1','SLCO5A1','SLIT1','SLIT3','SLITRK6','SLK','SLX4','SMAGP','SMAP1','SMOC1','SMPD2','SMPX','SNAP23','SNAP25','SNRK','SNTB2','SNTG2','SNX15','SNX32','SNX9','SOCS3','SOCS5','SOD1','SORBS1','SORCS1','SORT1','SOX11','SPAG16','SPANXN4','SPART','SPATA12','SPATC1L','SPERT','SPG11','SPHK1','SPINK13','SPINT1','SPINT2','SPN','SPPL2B','SPPL3','SPRED3','SPRYD3','SPTBN2','SRC','SRCIN1','SRD5A3','SRPK1','SRPRB','SRRM3','SSFA2','SSH1','SSH3','SSTR1','SSTR5','SSX2IP','ST8SIA5','STAB1','STAC','STAC2','STAMBP','STAMBPL1','STARD4','STAT2','STEAP4','STIL','STIP1','STK10','STK16','STK17A','STK17B','STOM','STOML2','STOML3','STX18','STX4','STXBP4','STYK1','SUCLG1','SUCLG2','SUGT1','SUMF2','SUPT6H','SUSD2','SVIL','SWAP70','SYCP2','SYK','SYMPK','SYNC','SYNGR3','SYNPO2L','SYPL2','SYT4','SYTL2','SYVN1','TACC2','TACR2','TACSTD2','TASP1','TAZ','TBC1D10A','TBC1D2','TBC1D24','TBC1D30','TBC1D9B','TBXA2R','TCAF2','TCAF2C','TCHP','TCIM','TDG','TDO2','TDP1','TEC','TEDC2','TEK','TES','TEX14','TEX9','TGFBR1','TGFBR2','TGM1','TGM2','THBS1','THBS2','THEGL','THY1','TIAM1','TICAM1','TIGD7','TIMD4','TJP1','TJP2','TJP3','TLN1','TLR4','TM2D2','TM4SF20','TM4SF4','TM4SF5','TMC5','TMED8','TMEM101','TMEM107','TMEM108','TMEM121B','TMEM139','TMEM141','TMEM164','TMEM176B','TMEM183A','TMEM186','TMEM2','TMEM218','TMEM222','TMEM229A','TMEM245','TMEM251','TMEM254','TMEM258','TMEM266','TMEM268','TMEM50A','TMEM63B','TMPRSS15','TMPRSS2','TMTC2','TNFRSF10D','TNFRSF11A','TNFRSF12A','TNFRSF21','TNFRSF25','TNFRSF8','TNFRSF9','TNFSF13B','TNK1','TNKS1BP1','TNNI2','TOGARAM2','TOLLIP','TOM1','TP53BP2','TPD52L1','TPRN','TRAF7','TRAIP','TRAPPC13','TRAT1','TRDN','TREML1','TRIB1','TRIM16','TRIM16L','TRIM38','TRIM4','TRIM44','TRIM54','TRIM71','TRIML2','TRIP6','TRMT1','TRMT10B','TRMT13','TRMT6','TRO','TRPM1','TRPM4','TRPM7','TRPV2','TRPV3','TSG101','TSGA13','TSHZ3','TSPAN15','TSPAN6','TSPO2','TSR2','TSSK4','TTC17','TTC23','TTC7B','TTLL12','TTLL5','TULP3','TWF2','TXNDC11','TXNIP','TYROBP','TYW3','UAP1','UBA3','UBA52','UBAC1','UBALD1','UBAP1','UBASH3B','UBE2A','UBE2B','UBE2C','UBE2D1','UBE2D2','UBE2D3','UBE2D4','UBE2K','UBE2QL1','UBE2S','UBE3C','UBQLN2','UBR2','UBTD1','UCP3','ULBP1','ULBP2','UNC5A','UNC79','UQCC1','USP14','USP15','USP2','USP20','USP21','USP4','USP53','UTP20','UTRN','VAMP5','VAMP8','VANGL1','VASP','VCAM1','VCPIP1','VIL1','VPS13B','VPS36','VPS50','VPS8','VSIG4','VTCN1','VWA5B1','VWA7','WAS','WASF2','WBP4','WDFY3','WDPCP','WDR1','WDR12','WDR13','WDR53','WDR6','WDR66','WIPF1','WWP1','XIAP','XIRP1','XIRP2','XPC','XPNPEP2','XPO6','XRN1','XYLT1','YES1','YIPF1','YIPF4','YOD1','YTHDC1','ZBED3','ZBTB33','ZBTB8OS','ZDHHC2','ZDHHC20','ZDHHC22','ZDHHC5','ZFP41','ZFP90','ZFYVE28','ZGPAT','ZMYND10','ZNF10','ZNF18','ZNF185','ZNF2','ZNF20','ZNF227','ZNF229','ZNF236','ZNF276','ZNF277','ZNF337','ZNF397','ZNF407','ZNF497','ZNF503','ZNF510','ZNF536','ZNF571','ZNF598','ZNF599','ZNF646','ZNF655','ZNF688','ZNF706','ZNF717','ZNF783','ZNF79','ZNF839','ZSCAN22','ZSCAN29','ZSWIM1','ZSWIM8','ZYX']
MembraneProteins=MembraneProteins1+MembraneProteins2

TFs1=['AC008770.3','AC023509.3','AC092835.1','AC138696.1','ADNP','ADNP2','AEBP1','AEBP2','AHCTF1','AHDC1','AHR','AHRR','AIRE','AKAP8','AKAP8L','AKNA','ALX1','ALX3','ALX4','ANHX','ANKZF1','AR','ARGFX','ARHGAP35','ARID2','ARID3A','ARID3B','ARID3C','ARID5A','ARID5B','ARNT','ARNT2','ARNTL','ARNTL2','ARX','ASCL1','ASCL2','ASCL3','ASCL4','ASCL5','ASH1L','ATF1','ATF2','ATF3','ATF4','ATF5','ATF6','ATF6B','ATF7','ATMIN','ATOH1','ATOH7','ATOH8','BACH1','BACH2','BARHL1','BARHL2','BARX1','BARX2','BATF','BATF2','BATF3','BAZ2A','BAZ2B','BBX','BCL11A','BCL11B','BCL6','BCL6B','BHLHA15','BHLHA9','BHLHE22','BHLHE23','BHLHE40','BHLHE41','BNC1','BNC2','BORCS8-MEF2B','BPTF','BRF2','BSX','C11orf95','CAMTA1','CAMTA2','CARF','CASZ1','CBX2','CC2D1A','CCDC169-SOHLH2','CCDC17','CDC5L','CDX1','CDX2','CDX4','CEBPA','CEBPB','CEBPD','CEBPE','CEBPG','CEBPZ','CENPA','CENPB','CENPBD1','CENPS','CENPT','CENPX','CGGBP1','CHAMP1','CHCHD3','CIC','CLOCK','CPEB1','CPXCR1','CREB1','CREB3','CREB3L1','CREB3L2','CREB3L3','CREB3L4','CREB5','CREBL2','CREBZF','CREM','CRX','CSRNP1','CSRNP2','CSRNP3','CTCF','CTCFL','CUX1','CUX2','CXXC1','CXXC4','CXXC5','DACH1','DACH2','DBP','DBX1','DBX2','DDIT3','DEAF1','DLX1','DLX2','DLX3','DLX4','DLX5','DLX6','DMBX1','DMRT1','DMRT2','DMRT3','DMRTA1','DMRTA2','DMRTB1','DMRTC2','DMTF1','DNMT1','DNTTIP1','DOT1L','DPF1','DPF3','DPRX','DR1','DRAP1','DRGX','DUX1','DUX3','DUX4','DUXA','DZIP1','E2F1','E2F2','E2F3','E2F4','E2F5','E2F6','E2F7','E2F8','E4F1','EBF1','EBF2','EBF3','EBF4','EEA1','EGR1','EGR2','EGR3','EGR4','EHF','ELF1','ELF2','ELF3','ELF4','ELF5','ELK1','ELK3','ELK4','EMX1','EMX2','EN1','EN2','EOMES','EPAS1','ERF','ERG','ESR1','ESR2','ESRRA','ESRRB','ESRRG','ESX1','ETS1','ETS2','ETV1','ETV2','ETV3','ETV3L','ETV4','ETV5','ETV6','ETV7','EVX1','EVX2','FAM170A','FAM200B','FBXL19','FERD3L','FEV','FEZF1','FEZF2','FIGLA','FIZ1','FLI1','FLYWCH1','FOS','FOSB','FOSL1','FOSL2','FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXN4','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1','GABPA','GATA1','GATA2','GATA3','GATA4','GATA5','GATA6','GATAD2A','GATAD2B','GBX1','GBX2','GCM1','GCM2','GFI1','GFI1B','GLI1','GLI2','GLI3','GLI4','GLIS1','GLIS2','GLIS3','GLMP','GLYR1','GMEB1','GMEB2','GPBP1','GPBP1L1','GRHL1','GRHL2','GRHL3','GSC','GSC2','GSX1','GSX2','GTF2B','GTF2I','GTF2IRD1','GTF2IRD2','GTF2IRD2B','GTF3A','GZF1','HAND1','HAND2','HBP1','HDX','HELT','HES1','HES2','HES3','HES4','HES5','HES6','HES7','HESX1','HEY1','HEY2','HEYL','HHEX','HIC1','HIC2','HIF1A','HIF3A','HINFP','HIVEP1','HIVEP2','HIVEP3','HKR1','HLF','HLX','HMBOX1','HMG20A','HMG20B','HMGA1','HMGA2','HMGN3','HMX1','HMX2','HMX3','HNF1A','HNF1B','HNF4A','HNF4G','HOMEZ','HOXA1','HOXA10','HOXA11','HOXA13','HOXA2','HOXA3','HOXA4','HOXA5','HOXA6','HOXA7','HOXA9','HOXB1','HOXB13','HOXB2','HOXB3','HOXB4','HOXB5','HOXB6','HOXB7','HOXB8','HOXB9','HOXC10','HOXC11','HOXC12','HOXC13','HOXC4','HOXC5','HOXC6','HOXC8','HOXC9','HOXD1','HOXD10','HOXD11','HOXD12','HOXD13','HOXD3','HOXD4','HOXD8','HOXD9','HSF1','HSF2','HSF4','HSF5','HSFX1','HSFX2','HSFY1','HSFY2','IKZF1','IKZF2','IKZF3','IKZF4','IKZF5','INSM1','INSM2','IRF1','IRF2','IRF3','IRF4','IRF5','IRF6','IRF7','IRF8','IRF9','IRX1','IRX2','IRX3','IRX4','IRX5','IRX6','ISL1','ISL2','ISX','JAZF1','JDP2','JRK','JRKL','JUN','JUNB','JUND','KAT7','KCMF1','KCNIP3','KDM2A','KDM2B','KDM5B','KIN','KLF1','KLF10','KLF11','KLF12','KLF13','KLF14','KLF15','KLF16','KLF17','KLF2','KLF3','KLF4','KLF5','KLF6','KLF7','KLF8','KLF9','KMT2A','KMT2B','L3MBTL1','L3MBTL3','L3MBTL4','LBX1','LBX2','LCOR','LCORL','LEF1','LEUTX','LHX1','LHX2','LHX3','LHX4','LHX5','LHX6','LHX8','LHX9','LIN28A','LIN28B','LIN54','LMX1A','LMX1B','LTF','LYL1','MAF','MAFA','MAFB','MAFF','MAFG','MAFK','MAX','MAZ','MBD1','MBD2','MBD3','MBD4','MBD6','MBNL2','MECOM','MECP2','MEF2A','MEF2B','MEF2C','MEF2D','MEIS1','MEIS2','MEIS3','MEOX1','MEOX2','MESP1','MESP2','MGA','MITF','MIXL1','MKX','MLX','MLXIP','MLXIPL','MNT','MNX1','MSANTD1','MSANTD3','MSANTD4','MSC','MSGN1','MSX1','MSX2','MTERF1','MTERF2','MTERF3','MTERF4','MTF1','MTF2','MXD1','MXD3','MXD4','MXI1','MYB','MYBL1','MYBL2','MYC','MYCL','MYCN','MYF5','MYF6','MYNN','MYOD1','MYOG','MYPOP','MYRF','MYRFL','MYSM1','MYT1','MYT1L','MZF1','NACC2','NAIF1','NANOG','NANOGNB','NANOGP8','NCOA1','NCOA2','NCOA3','NEUROD1','NEUROD2','NEUROD4','NEUROD6','NEUROG1','NEUROG2','NEUROG3','NFAT5','NFATC1','NFATC2','NFATC3','NFATC4','NFE2','NFE2L1','NFE2L2','NFE2L3','NFE4','NFIA','NFIB','NFIC','NFIL3','NFIX','NFKB1','NFKB2','NFX1','NFXL1','NFYA','NFYB','NFYC','NHLH1','NHLH2','NKRF','NKX1-1','NKX1-2','NKX2-1','NKX2-2','NKX2-3','NKX2-4','NKX2-5','NKX2-6','NKX2-8','NKX3-1','NKX3-2','NKX6-1','NKX6-2','NKX6-3','NME2','NOBOX','NOTO','NPAS1','NPAS2','NPAS3','NPAS4','NR0B1','NR1D1','NR1D2','NR1H2','NR1H3','NR1H4','NR1I2','NR1I3','NR2C1','NR2C2','NR2E1','NR2E3','NR2F1','NR2F2','NR2F6','NR3C1','NR3C2','NR4A1','NR4A2','NR4A3','NR5A1','NR5A2','NR6A1','NRF1','NRL','OLIG1','OLIG2','OLIG3','ONECUT1','ONECUT2','ONECUT3','OSR1','OSR2','OTP','OTX1','OTX2','OVOL1','OVOL2','OVOL3','PA2G4','PATZ1','PAX1','PAX2','PAX3','PAX4','PAX5','PAX6','PAX7','PAX8','PAX9','PBX1','PBX2','PBX3','PBX4','PCGF2','PCGF6','PDX1','PEG3','PGR','PHF1','PHF19 ','PHF20','PHF21A','PHOX2A','PHOX2B','PIN1','PITX1','PITX2','PITX3','PKNOX1','PKNOX2','PLAG1','PLAGL1','PLAGL2','PLSCR1','POGK','POU1F1','POU2AF1','POU2F1','POU2F2','POU2F3','POU3F1','POU3F2','POU3F3','POU3F4','POU4F1','POU4F2','POU4F3','POU5F1','POU5F1B','POU5F2','POU6F1','POU6F2','PPARA','PPARD','PPARG','PRDM1','PRDM10','PRDM12','PRDM13','PRDM14','PRDM15','PRDM16','PRDM2','PRDM4','PRDM5','PRDM6','PRDM8','PRDM9','PREB','PRMT3','PROP1','PROX1','PROX2','PRR12','PRRX1','PRRX2','PTF1A','PURA','PURB','PURG','RAG1','RARA','RARB','RARG','RAX','RAX2','RBAK','RBCK1','RBPJ','RBPJL','RBSN','REL','RELA','RELB','REPIN1','REST','REXO4','RFX1','RFX2','RFX3','RFX4','RFX5','RFX6','RFX7','RFX8','RHOXF1','RHOXF2','RHOXF2B','RLF','RORA','RORB','RORC','RREB1','RUNX1','RUNX2','RUNX3','RXRA','RXRB','RXRG','SAFB','SAFB2','SALL1','SALL2','SALL3','SALL4','SATB1','SATB2','SCMH1','SCML4','SCRT1','SCRT2','SCX','SEBOX','SETBP1','SETDB1','SETDB2','SGSM2','SHOX','SHOX2','SIM1','SIM2','SIX1','SIX2','SIX3','SIX4','SIX5','SIX6','SKI','SKIL','SKOR1','SKOR2','SLC2A4RG','SMAD1','SMAD3','SMAD4','SMAD5','SMAD9','SMYD3','SNAI1','SNAI2','SNAI3','SNAPC2','SNAPC4','SNAPC5','SOHLH1','SOHLH2','SON','SOX1','SOX10','SOX11']
TFs2=['SOX12','SOX13','SOX14','SOX15','SOX17','SOX18','SOX2','SOX21','SOX3','SOX30','SOX4','SOX5','SOX6','SOX7','SOX8','SOX9','SP1','SP100','SP110','SP140','SP140L','SP2','SP3','SP4','SP5','SP6','SP7','SP8','SP9','SPDEF','SPEN','SPI1','SPIB','SPIC','SPZ1','SRCAP','SREBF1','SREBF2','SRF','SRY','ST18','STAT1','STAT2','STAT3','STAT4','STAT5A','STAT5B','STAT6','T','TAL1','TAL2','TBP','TBPL1','TBPL2','TBR1','TBX1','TBX10','TBX15','TBX18','TBX19','TBX2','TBX20','TBX21','TBX22','TBX3','TBX4','TBX5','TBX6','TCF12','TCF15','TCF20','TCF21','TCF23','TCF24','TCF3','TCF4','TCF7','TCF7L1','TCF7L2','TCFL5','TEAD1','TEAD2','TEAD3','TEAD4','TEF','TERB1','TERF1','TERF2','TET1','TET2','TET3','TFAP2A','TFAP2B','TFAP2C','TFAP2D','TFAP2E','TFAP4','TFCP2','TFCP2L1','TFDP1','TFDP2','TFDP3','TFE3','TFEB','TFEC','TGIF1','TGIF2','TGIF2LX','TGIF2LY','THAP1','THAP10','THAP11','THAP12','THAP2','THAP3','THAP4','THAP5','THAP6','THAP7','THAP8','THAP9','THRA','THRB','THYN1','TIGD1','TIGD2','TIGD3','TIGD4','TIGD5','TIGD6','TIGD7','TLX1','TLX2','TLX3','TMF1','TOPORS','TP53','TP63','TP73','TPRX1','TRAFD1','TRERF1','TRPS1','TSC22D1','TSHZ1','TSHZ2','TSHZ3','TTF1','TWIST1','TWIST2','UBP1','UNCX','USF1','USF2','USF3','VAX1','VAX2','VDR','VENTX','VEZF1','VSX1','VSX2','WIZ','WT1','XBP1','XPA','YBX1','YBX2','YBX3','YY1','YY2','ZBED1','ZBED2','ZBED3','ZBED4','ZBED5','ZBED6','ZBED9','ZBTB1','ZBTB10','ZBTB11','ZBTB12','ZBTB14','ZBTB16','ZBTB17','ZBTB18','ZBTB2','ZBTB20','ZBTB21','ZBTB22','ZBTB24','ZBTB25','ZBTB26','ZBTB3','ZBTB32','ZBTB33','ZBTB34','ZBTB37','ZBTB38','ZBTB39','ZBTB4','ZBTB40','ZBTB41','ZBTB42','ZBTB43','ZBTB44','ZBTB45','ZBTB46','ZBTB47','ZBTB48','ZBTB49','ZBTB5','ZBTB6','ZBTB7A','ZBTB7B','ZBTB7C','ZBTB8A','ZBTB8B','ZBTB9','ZC3H8','ZEB1','ZEB2','ZFAT','ZFHX2','ZFHX3','ZFHX4','ZFP1','ZFP14','ZFP2','ZFP28','ZFP3','ZFP30','ZFP37','ZFP41','ZFP42','ZFP57','ZFP62','ZFP64','ZFP69','ZFP69B','ZFP82','ZFP90','ZFP91','ZFP92','ZFPM1','ZFPM2','ZFX','ZFY','ZGLP1','ZGPAT','ZHX1','ZHX2','ZHX3','ZIC1','ZIC2','ZIC3','ZIC4','ZIC5','ZIK1','ZIM2','ZIM3','ZKSCAN1','ZKSCAN2','ZKSCAN3','ZKSCAN4','ZKSCAN5','ZKSCAN7','ZKSCAN8','ZMAT1','ZMAT4','ZNF10','ZNF100','ZNF101','ZNF107','ZNF112','ZNF114','ZNF117','ZNF12','ZNF121','ZNF124','ZNF131','ZNF132','ZNF133','ZNF134','ZNF135','ZNF136','ZNF138','ZNF14','ZNF140','ZNF141','ZNF142','ZNF143','ZNF146','ZNF148','ZNF154','ZNF155','ZNF157','ZNF16','ZNF160','ZNF165','ZNF169','ZNF17','ZNF174','ZNF175','ZNF177','ZNF18','ZNF180','ZNF181','ZNF182','ZNF184','ZNF189','ZNF19','ZNF195','ZNF197','ZNF2','ZNF20','ZNF200','ZNF202','ZNF205','ZNF207','ZNF208','ZNF211','ZNF212','ZNF213','ZNF214','ZNF215','ZNF217','ZNF219','ZNF22','ZNF221','ZNF222','ZNF223','ZNF224','ZNF225','ZNF226','ZNF227','ZNF229','ZNF23','ZNF230','ZNF232','ZNF233','ZNF234','ZNF235','ZNF236','ZNF239','ZNF24','ZNF248','ZNF25','ZNF250','ZNF251','ZNF253','ZNF254','ZNF256','ZNF257','ZNF26','ZNF260','ZNF263','ZNF264','ZNF266','ZNF267','ZNF268','ZNF273','ZNF274','ZNF275','ZNF276','ZNF277','ZNF28','ZNF280A','ZNF280B','ZNF280C','ZNF280D','ZNF281','ZNF282','ZNF283','ZNF284','ZNF285','ZNF286A','ZNF286B','ZNF287','ZNF292','ZNF296','ZNF3','ZNF30','ZNF300','ZNF302','ZNF304','ZNF311','ZNF316','ZNF317','ZNF318','ZNF319','ZNF32','ZNF320','ZNF322','ZNF324','ZNF324B','ZNF326','ZNF329','ZNF331','ZNF333','ZNF334','ZNF335','ZNF337','ZNF33A','ZNF33B','ZNF34','ZNF341','ZNF343','ZNF345','ZNF346','ZNF347','ZNF35','ZNF350','ZNF354A','ZNF354B','ZNF354C','ZNF358','ZNF362','ZNF365','ZNF366','ZNF367','ZNF37A','ZNF382','ZNF383','ZNF384','ZNF385A','ZNF385B','ZNF385C','ZNF385D','ZNF391','ZNF394','ZNF395','ZNF396','ZNF397','ZNF398','ZNF404','ZNF407','ZNF408','ZNF41','ZNF410','ZNF414','ZNF415','ZNF416','ZNF417','ZNF418','ZNF419','ZNF420','ZNF423','ZNF425','ZNF426','ZNF428','ZNF429','ZNF43','ZNF430','ZNF431','ZNF432','ZNF433','ZNF436','ZNF438','ZNF439','ZNF44','ZNF440','ZNF441','ZNF442','ZNF443','ZNF444','ZNF445','ZNF446','ZNF449','ZNF45','ZNF451','ZNF454','ZNF460','ZNF461','ZNF462','ZNF467','ZNF468','ZNF469','ZNF470','ZNF471','ZNF473','ZNF474','ZNF479','ZNF48','ZNF480','ZNF483','ZNF484','ZNF485','ZNF486','ZNF487','ZNF488','ZNF490','ZNF491','ZNF492','ZNF493','ZNF496','ZNF497','ZNF500','ZNF501','ZNF502','ZNF503','ZNF506','ZNF507','ZNF510','ZNF511','ZNF512','ZNF512B','ZNF513','ZNF514','ZNF516','ZNF517','ZNF518A','ZNF518B','ZNF519','ZNF521','ZNF524','ZNF525','ZNF526','ZNF527','ZNF528','ZNF529','ZNF530','ZNF532','ZNF534','ZNF536','ZNF540','ZNF541','ZNF543','ZNF544','ZNF546','ZNF547','ZNF548','ZNF549','ZNF550','ZNF551','ZNF552','ZNF554','ZNF555','ZNF556','ZNF557','ZNF558','ZNF559','ZNF560','ZNF561','ZNF562','ZNF563','ZNF564','ZNF565','ZNF566','ZNF567','ZNF568','ZNF569','ZNF57','ZNF570','ZNF571','ZNF572','ZNF573','ZNF574','ZNF575','ZNF576','ZNF577','ZNF578','ZNF579','ZNF580','ZNF581','ZNF582','ZNF583','ZNF584','ZNF585A','ZNF585B','ZNF586','ZNF587','ZNF587B','ZNF589','ZNF592','ZNF594','ZNF595','ZNF596','ZNF597','ZNF598','ZNF599','ZNF600','ZNF605','ZNF606','ZNF607','ZNF608','ZNF609','ZNF610','ZNF611','ZNF613','ZNF614','ZNF615','ZNF616','ZNF618','ZNF619','ZNF620','ZNF621','ZNF623','ZNF624','ZNF625','ZNF626','ZNF627','ZNF628','ZNF629','ZNF630','ZNF639','ZNF641','ZNF644','ZNF645','ZNF646','ZNF648','ZNF649','ZNF652','ZNF653','ZNF654','ZNF655','ZNF658','ZNF66','ZNF660','ZNF662','ZNF664','ZNF665','ZNF667','ZNF668','ZNF669','ZNF670','ZNF671','ZNF672','ZNF674','ZNF675','ZNF676','ZNF677','ZNF678','ZNF679','ZNF680','ZNF681','ZNF682','ZNF683','ZNF684','ZNF687','ZNF688','ZNF689','ZNF69','ZNF691','ZNF692','ZNF695','ZNF696','ZNF697','ZNF699','ZNF7','ZNF70','ZNF700','ZNF701','ZNF703','ZNF704','ZNF705A','ZNF705B','ZNF705D','ZNF705E','ZNF705G','ZNF706','ZNF707','ZNF708','ZNF709','ZNF71','ZNF710','ZNF711','ZNF713','ZNF714','ZNF716','ZNF717','ZNF718','ZNF721','ZNF724','ZNF726','ZNF727','ZNF728','ZNF729','ZNF730','ZNF732','ZNF735','ZNF736','ZNF737','ZNF74','ZNF740','ZNF746','ZNF747','ZNF749','ZNF750','ZNF75A','ZNF75D','ZNF76','ZNF761','ZNF763','ZNF764','ZNF765','ZNF766','ZNF768','ZNF77','ZNF770','ZNF771','ZNF772','ZNF773','ZNF774','ZNF775','ZNF776','ZNF777','ZNF778','ZNF780A','ZNF780B','ZNF781','ZNF782','ZNF783','ZNF784','ZNF785','ZNF786','ZNF787','ZNF788','ZNF789','ZNF79','ZNF790','ZNF791','ZNF792','ZNF793','ZNF799','ZNF8','ZNF80','ZNF800','ZNF804A','ZNF804B','ZNF805','ZNF808','ZNF81','ZNF813','ZNF814','ZNF816','ZNF821','ZNF823','ZNF827','ZNF829','ZNF83','ZNF830','ZNF831','ZNF835','ZNF836','ZNF837','ZNF84','ZNF841','ZNF843','ZNF844','ZNF845','ZNF846','ZNF85','ZNF850','ZNF852','ZNF853','ZNF860','ZNF865','ZNF878','ZNF879','ZNF880','ZNF883','ZNF888','ZNF891','ZNF90','ZNF91','ZNF92','ZNF93','ZNF98','ZNF99','ZSCAN1','ZSCAN10','ZSCAN12','ZSCAN16','ZSCAN18','ZSCAN2','ZSCAN20','ZSCAN21','ZSCAN22','ZSCAN23','ZSCAN25','ZSCAN26','ZSCAN29','ZSCAN30','ZSCAN31','ZSCAN32','ZSCAN4','ZSCAN5A','ZSCAN5B','ZSCAN5C','ZSCAN9','ZUFSP','ZXDA','ZXDB','ZXDC','ZZZ3']
TFs=TFs1+TFs2

#TFs other, inclusive of the above list, not necessarily DNA binding proteins but affect transcription
TFs1Other=['AATF','ABL1','ABRA','AC008770.3','AC020915.1','AC023509.3','AC092835.1','AC105001.2','AC138696.1','ACOD1','ACTR3B','ACTR3C','ADAM8','ADAMTS17','ADAMTS19','ADAR','ADGRA1','ADGRG3','ADNP','ADNP2','AEBP1','AEBP2','AES','AFAP1 ','AFF1','AFF3','AFF4','AGER','AGT','AHCTF1','AHDC1','AHR','AHRR','AIM2','AIRE','AKAP8','AKAP8L','AKNA','AKT1','ALK','ALKBH2','ALX1','ALX3','ALX4','AMH','ANAPC2','ANHX','ANKRD30A','ANKZF1','ANXA3','ANXA4','AP5Z1','APBB3','APC2','APEX1','APOA4','APOBEC3A','APOBEC3B','APOBEC3G','APTX','AR','ARGFX','ARHGAP35','ARHGEF2','ARHGEF5','ARID1A','ARID1B','ARID2','ARID3A','ARID3B','ARID3C','ARID4A','ARID4B','ARID5A','ARID5B','ARNT','ARNT2','ARNTL','ARNTL2','ARRB1','ARRB2','ARX','ASCC1','ASCL1','ASCL2','ASCL3','ASCL4','ASCL5','ASH1L','ATF1','ATF2','ATF3','ATF4','ATF5','ATF6','ATF6B','ATF7','ATF7IP','ATM','ATMIN','ATOH1','ATOH7','ATOH8','ATR','ATRX','ATXN7','BACH1','BACH2','BAG1','BANF1','BANP','BARHL1','BARHL2','BARX1','BARX2','BATF','BATF2','BATF3','BAZ2A','BAZ2B','BBX','BCL10','BCL11A','BCL11B','BCL3','BCL6','BCL6B','BCLAF1','BCOR','BDP1','BEND4','BEX1','BHLHA15','BHLHA9','BHLHE22','BHLHE23','BHLHE40','BHLHE41','BHMG1','BIN1','BLM','BLZF1','BMI1','BMP2','BMP4','BMP7','BMPR1A','BNC1','BNC2','BOD1L1','BOLA1','BOLA2','BOLA2B','BOLA3','BORCS8-MEF2B','BPNT1','BPTF','BRCA1','BRD4','BRD7','BRD8','BRD9','BRF1','BRF2','BRMS1','BRPF1','BSX','BTAF1','BTBD8','BTG2','BTK','BTRC','BUD31','C11orf95','C14orf39','C17orf49','C1D','C20orf194','C3orf33','CACTIN','CAMK1D','CAMK2A','CAMTA1','CAMTA2','CAPN15','CAPN3','CARD11','CARD14','CARF','CARHSP1','CASP8AP2','CASZ1','CAT','CBFA2T2','CBFA2T3','CBFB','CBL','CBLL1','CBX2','CC2D1A','CC2D1B','CCDC124','CCDC169-SOHLH2','CCDC17','CCDC3','CCDC71','CCDC83','CCNT1','CCNT2','CD36','CD40','CD40LG','CDC5L','CDC6','CDK5RAP3','CDK7','CDK8','CDK9','CDKN1A','CDKN2A','CDX1','CDX2','CDX4','CEBPA','CEBPB','CEBPD','CEBPE','CEBPG','CEBPZ','CELF3','CENPA','CENPB','CENPBD1','CENPS','CENPT','CENPX','CERS2','CERS3','CERS4','CERS5','CERS6','CFLAR','CGGBP1','CHAMP1','CHCHD3','CHD1','CHD2','CHD3','CHD4','CHD6','CHD7','CHD9','CHP1','CHRAC1','CHUK','CHURC1','CIB1','CIC','CIITA','CIR1','CITED1','CITED2','CITED4','CIZ1','CLOCK','CLU','CMKLR1','CNBP','CNOT3','CNOT4','CNOT7','CNOT8','COMMD1','COMMD6','COMMD7','COPS2','COPS3','COPS4','CPEB1','CPSF4','CPSF4L','CPXCR1','CRAMP1','CREB1','CREB3','CREB3L1','CREB3L2','CREB3L3','CREB3L4','CREB5','CREBBP','CREBL2','CREBRF','CREBZF','CREG1','CREM','CRTC1','CRTC2','CRTC3','CRX','CSDC2','CSDE1','CSRNP1','CSRNP2','CSRNP3','CTBP1','CTBP2','CTCF','CTCFL','CTH','CTNNB1','CTNNBIP1','CUL1','CUL2','CUL3','CUL4A','CUL4B','CUL5','CUX1','CUX2','CXXC1','CXXC4','CXXC5','CYLD','CYP1B1','CYTL1','DAB2IP','DACH1','DACH2','DAP','DAXX','DBP','DBX1','DBX2','DCP1A','DDB1','DDB2','DDIT3','DDN','DDR2','DDRGK1','DDX3X','DDX5','DDX58','DEAF1','DEK','DEPDC1','DEPDC1B','DEPDC4','DEPDC5','DEPDC7','DEPTOR','DHX34','DHX36','DHX57','DISP3','DLX1','DLX2','DLX3','DLX4','DLX5','DLX6','DMAP1','DMBX1','DMC1','DMRT1','DMRT2','DMRT3','DMRTA1','DMRTA2','DMRTB1','DMRTC1','DMRTC2','DMTF1','DNA2','DNAJA3','DNAJC1','DNAJC2','DNAJC21','DNMT1','DNMT3A','DNMT3B','DNMT3L','DNTT','DNTTIP1','DOT1L','DPF1','DPF2','DPF3','DPRX','DR1','DRAP1','DRGX','DSP','DTX1','DUS3L','DUSP12','DUX1','DUX3','DUX4','DUXA','DVL1','DVL2','DVL3','DZIP1','DZIP1L','E2F1','E2F2','E2F3','E2F4','E2F5','E2F6','E2F7','E2F8','E4F1','EAF2','EBF1','EBF2','EBF3','EBF4','EDA','EDA2R','EDF1','EDN1','EEA1','EED','EGLN1','EGR1','EGR2','EGR3','EGR4','EHF','EID1','EID2','EIF2AK2','EIF2AK4','EIF3C','EIF3CL','EIF3K','ELF1','ELF2','ELF3','ELF4','ELF5','ELK1','ELK3','ELK4','ELMSAN1','EME1','EMX1','EMX2','EN1','EN2','ENO1','EOMES','EP300','EP400','EPAS1','EPHA5','EPM2AIP1','ERBIN','ERC1','ERCC3','ERCC4','ERF','ERG','ESR1','ESR2','ESRRA','ESRRB','ESRRG','ESX1','ETS1','ETS2','ETV1','ETV2','ETV3','ETV3L','ETV4','ETV5','ETV6','ETV7','EVX1','EVX2','EWSR1','EXO1','EXOC2','EXOG','EZH1','EZH2','FAAP24','FAM153C','FAM170A','FAM171B','FAM200A','FAM200B','FAN1','FANCA','FANCD2','FANK1','FARSA','FARSB','FBN1','FBXL19','FBXO41','FEN1','FER','FERD3L','FEV','FEZF1','FEZF2','FGD1','FHL2','FIGLA','FIZ1','FLI1','FLNA','FLYWCH1','FMNL2','FOS','FOSB','FOSL1','FOSL2','FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L2','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXN4','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1','FUBP1','FUBP3','FUS','FZD1','FZD2','FZD4','FZD6','GABPA','GABPB1','GABPB2','GADD45A','GAS6','GAS7','GATA1','GATA2','GATA3','GATA4','GATA5','GATA6','GATAD1','GATAD2A','GATAD2B','GBX1','GBX2','GCFC2','GCM1','GCM2','GEN1','GFI1','GFI1B','GLA','GLI1','GLI2','GLI3','GLI4','GLIS1','GLIS2','GLIS3','GLMP','GLYR1','GMEB1','GMEB2','GOLGB1','GON4L','GPATCH8','GPBP1','GPBP1L1','GPER1','GPR155','GREM1','GRHL1','GRHL2','GRHL3','GRM6','GSC','GSC2','GSX1','GSX2','GTF2A1','GTF2A1L','GTF2A2','GTF2B','GTF2E2','GTF2F1','GTF2F2','GTF2H1','GTF2H2','GTF2H2C','GTF2H3','GTF2H4','GTF2I','GTF2IRD1','GTF2IRD2','GTF2IRD2B','GTF3A','GTF3C1','GTF3C2','GULP1','GZF1','H1F0','H1FOO','H1FX','HAND1','HAND2','HAVCR2','HBP1','HCFC1','HCK','HDAC1','HDAC10','HDAC2','HDAC3','HDAC4','HDAC5','HDAC6','HDAC7','HDAC8','HDAC9','HDX','HELT','HELZ','HELZ2','HES1','HES2','HES3','HES4','HES5','HES6','HES7','HESX1','HEY1','HEY2','HEYL','HHEX','HIC1','HIC2','HIF1A','HIF3A','HINFP','HIPK2','HIRA','HIST1H1A','HIST1H1B','HIST1H1C','HIST1H1D','HIST1H1E','HIST1H1T','HIVEP1','HIVEP2','HIVEP3','HKR1','HLA-DQB1','HLA-DQB2','HLA-DRB1','HLA-DRB3','HLA-DRB5','HLF','HLTF','HLX','HMBOX1','HMG20A','HMG20B','HMGA1','HMGA2','HMGB1','HMGB2','HMGB3','HMGB4','HMGN1','HMGN3','HMGXB3','HMGXB4','HMOX1','HMX1','HMX2','HMX3','HNF1A','HNF1B','HNF4A','HNF4G','HNRNPA1','HNRNPA3','HNRNPAB','HNRNPCL2','HNRNPD','HNRNPDL','HNRNPK','HNRNPL','HNRNPU','HOMEZ','HOOK2','HOPX','HOXA1','HOXA10','HOXA11','HOXA13','HOXA2','HOXA3','HOXA4','HOXA5','HOXA6','HOXA7','HOXA9','HOXB1','HOXB13','HOXB2','HOXB3','HOXB4','HOXB5','HOXB6','HOXB7','HOXB8','HOXB9','HOXC10','HOXC11','HOXC12','HOXC13','HOXC4','HOXC5','HOXC6','HOXC8','HOXC9','HOXD1','HOXD10','HOXD11','HOXD12','HOXD13','HOXD3','HOXD4','HOXD8','HOXD9','HP1BP3','HR','HSBP1','HSF1','HSF2','HSF4','HSF5','HSFX1','HSFX2','HSFY1','HSFY2','HSPA1A','HSPA1B','ICAM1','ID1','ID2','ID3','ID4','IFI16','IFNB1','IGHM','IKBKB','IKBKG','IKZF1','IKZF2','IKZF3','IKZF4','IKZF5','IL10','IL1B','IL1RAP','IL4','IL5','IL6','INF2','ING4','INS','INSM1','INSM2','IQGAP1','IRAK1','IRAK2','IRAK3','IRF1','IRF2','IRF2BP1','IRF3','IRF4','IRF5','IRF6','IRF7','IRF8','IRF9','IRX1','IRX2','IRX3','IRX4','IRX5','IRX6','ISL1','ISL2','ISX','ITCH','ITGB2','ITGB3BP','IZUMO2','JADE1','JAK2','JARID2','JAZF1','JDP2','JMY','JRK','JRKL','JUN','JUNB','JUND','JUP','KANSL2','KAT2A','KAT2B','KAT5','KAT6A','KAT6B','KAT7','KAT8','KCMF1','KCNIP3','KDM1A','KDM2A','KDM2B','KDM3A','KDM5A','KDM5B','KDM5C','KDM5D','KEAP1','KIAA1549','KIAA1683','KIAA1841','KIAA1958','KIF15','KIN','KIT','KLF1','KLF10','KLF11','KLF12','KLF13','KLF14','KLF15','KLF16','KLF17','KLF2','KLF3','KLF4','KLF5','KLF6','KLF7','KLF8','KLF9','KMT2A','KMT2B','KMT2C','KMT2D','KNTC1','KRAS','KRBOX1','KRT13','KRTAP5-1','KRTAP5-9','KRTAP9-9','L3MBTL1','L3MBTL3','L3MBTL4','LARP1','LARP1B','LARP4','LARP4B','LARP6','LARP7','LBX1','LBX2','LCOR','LCORL','LEF1','LEUTX','LGALS9','LGR4','LHX1','LHX2','LHX3','LHX4','LHX5','LHX6','LHX8','LHX9','LIG1','LIG3','LIN28A','LIN28B','LIN54','LMO2','LMO4','LMX1A','LMX1B','LPP','LRP5','LRP6','LRP8','LRRFIP1','LRRFIP2','LSR','LTF','LTN1','LYAR','LYL1','LZTFL1','LZTR1','LZTS1','MACC1','MACF1','MAD2L2','MAEL','MAF','MAFA','MAFB','MAFF','MAFG','MAFK','MALT1','MAML1','MAML3','MAP2K5','MAP3K10','MAP3K13','MAP3K7','MAPK1','MAPK10','MAPK11','MAPK14','MAPK3','MAPK8','MAPK9','MATR3','MAVS','MAX','MAZ','MB21D1','MBD1','MBD2','MBD3','MBD4','MBD6','MBNL1','MBNL2','MBNL3','MBTPS2','MDFI','MDM2','MDM4','MECOM','MECP2','MED1','MED12','MED13','MED13L','MED14','MED15','MED16','MED17','MED21','MED23','MED24','MED6','MEF2A','MEF2B','MEF2C','MEF2D','MEIS1','MEIS2','MEIS3','MEN1','MEOX1','MEOX2','MESP1','MESP2','MET','MGA','MGMT','MID2','MIER1','MIER2','MIER3','MINK1','MIS18BP1','MITF','MIXL1','MKL1','MKL2','MKRN1','MKRN2','MKRN3','MKX','MLLT1','MLLT10','MLLT11','MLLT3','MLLT6','MLX','MLXIP','MLXIPL','MN1','MNAT1','MNT','MNX1','MORC3','MPG','MRPL28','MRRF','MSANTD1','MSANTD3','MSANTD4','MSC','MSGN1','MSH2','MSH3','MSH6','MSL3','MSRB2','MST1R','MSX1','MSX2','MTA1','MTA2','MTA3','MTDH','MTERF1','MTERF2','MTERF3','MTERF4','MTF1','MTF2','MTPN','MUS81','MXD1','MXD3','MXD4','MXI1','MYB','MYBBP1A','MYBL1','MYBL2','MYC','MYCBP','MYCL','MYCN','MYD88','MYEF2','MYF5','MYF6','MYNN','MYOCD','MYOD1','MYOG','MYPOP','MYRF','MYRFL','MYSM1','MYT1','MYT1L','MZF1','NAB2','NABP2','NACC2','NAIF1','NANOG','NANOGNB','NANOGP8','NCOA1','NCOA2','NCOA3','NCOA4','NCOA5','NCOA6','NCOA7','NCOR1','NCOR2','NDC80','NDN','NDP','NEB','NEIL1','NEUROD1','NEUROD2','NEUROD4','NEUROD6','NEUROG1','NEUROG2','NEUROG3','NFAM1','NFAT5','NFATC1','NFATC2','NFATC3','NFATC4','NFE2','NFE2L1','NFE2L2','NFE2L3','NFE4','NFIA','NFIB','NFIC','NFIL3','NFIX','NFKB1','NFKB2','NFKBIA','NFKBIB','NFKBID','NFKBIL1','NFKBIZ','NFRKB','NFX1','NFXL1','NFYA','NFYB','NFYC','NHLH1','NHLH2','NKRF','NKX1-1','NKX1-2','NKX2-1','NKX2-2','NKX2-3','NKX2-4','NKX2-5','NKX2-6','NKX2-8','NKX3-1','NKX3-2','NKX6-1','NKX6-2','NKX6-3','NLRC3','NLRC4','NLRC5','NLRP12','NLRP2B','NLRP3','NME2','NOBOX','NOC3L','NOC4L','NOCT','NOD1','NOD2','NODAL','NONO','NOP2','NOTCH1','NOTCH2','NOTCH3','NOTO','NPAS1','NPAS2','NPAS3','NPAS4','NPM1','NR0B1','NR0B2','NR1D1','NR1D2','NR1H2','NR1H3','NR1H4','NR1I2','NR1I3','NR2C1','NR2C2','NR2E1','NR2E3','NR2F1','NR2F2','NR2F6','NR3C1','NR3C2','NR4A1','NR4A2','NR4A3','NR5A1','NR5A2','NR6A1','NRBF2','NRF1','NRG1','NRIP1','NRIP3','NRK','NRL','NSD1','NSD2','NTRK1','NUCKS1','NUFIP1','NUP98','NUPL2']
TFs2Other=['OGG1','OLIG1','OLIG2','OLIG3','ONECUT1','ONECUT2','ONECUT3','OPRD1','OSR1','OSR2','OTOP3','OTP','OTULIN','OTX1','OTX2','OVOL1','OVOL2','OVOL3','PA2G4','PABPN1','PARK7','PARP1','PARP10','PARP12','PATZ1','PAWR','PAX1','PAX2','PAX3','PAX4','PAX5','PAX6','PAX7','PAX8','PAX9','PAXBP1','PAXIP1','PBRM1','PBX1','PBX2','PBX3','PBX4','PCBD1','PCBP1','PCBP2','PCBP3','PCGF2','PCGF6','PCNA','PCSK6','PDCD7','PDS5B','PDX1','PEG3','PELI1','PER1','PER2','PER3','PEX14','PFDN1','PGBD1','PGR','PGS1','PHB','PHB2','PHF1','PHF19 ','PHF20','PHF21A','PHF5A','PHOX2A','PHOX2B','PHTF1','PHTF2','PIAS1','PIAS2','PIAS3','PIAS4','PIDD1','PIKFYVE','PIM1','PIN1','PINK1','PITX1','PITX2','PITX3','PKHD1','PKHD1L1','PKNOX1','PKNOX2','PLA2G10','PLA2G1B','PLAG1','PLAGL1','PLAGL2','PLEK','PLEK2','PLEKHA4','PLPP3','PLSCR1','PLXNA1','PLXNA2','PLXNA3','PLXNA4','PLXNB1','PLXNB2','PLXNB3','PLXNC1','PLXND1','PML','PMS1','PNKP','POGK','POGZ','POLA1','POLB','POLE3','POLE4','POLG','POLH','POLI','POLK','POLL','POLM','POLN','POLQ','POLR1B','POLR2A','POLR2B','POLR2E','POLR2J','POLR2K','POLR2L','POLR3A','POLRMT','POT1','POU1F1','POU2AF1','POU2F1','POU2F2','POU2F3','POU3F1','POU3F2','POU3F3','POU3F4','POU4F1','POU4F2','POU4F3','POU5F1','POU5F1B','POU5F2','POU6F1','POU6F2','PPARA','PPARD','PPARG','PPARGC1A','PPARGC1B','PPP1R10','PPP1R13L','PPP1R21','PPP2R3B','PPP2R5B','PPP3CA','PPRC1','PQBP1','PRAM1','PRAP1','PRB3','PRDM1','PRDM10','PRDM11','PRDM12','PRDM13','PRDM14','PRDM15','PRDM16','PRDM2','PRDM4','PRDM5','PRDM6','PRDM7','PRDM8','PRDM9','PRDX3','PREB','PREX1','PREX2','PRH2','PRIM2','PRIMPOL','PRKCB','PRKCH','PRKCI','PRKCQ','PRKCZ','PRKD1','PRKD2','PRKN','PRMT2','PRMT3','PRNP','PROP1','PROX1','PROX2','PRR12','PRR3','PRRX1','PRRX2','PSMA6','PSMC5','PSMD10','PSMD11','PSMD12','PSPC1','PTCH1','PTEN','PTF1A','PTGIS','PTH','PTTG1','PUM3','PURA','PURB','PURG','PYCARD','PYDC1','PYDC2','QRICH1','RAB11FIP4','RAB7B','RAD21','RAD51','RAG1','RAI1','RAPGEF3','RAPGEF4','RAPGEF5','RARA','RARB','RARG','RAX','RAX2','RB1','RBAK','RBBP7','RBCK1','RBL1','RBL2','RBM10','RBM20','RBM22','RBM26','RBM27','RBM5','RBM6','RBPJ','RBPJL','RBSN','RC3H1','RC3H2','RCAN1','RCOR1','RCOR2','RCOR3','RECQL','REL','RELA','RELB','RELL2','RELN','REPIN1','RERE','REST','REV1','REXO4','RFC1','RFX1','RFX2','RFX3','RFX4','RFX5','RFX6','RFX7','RFX8','RFXANK','RFXAP','RGCC','RGS11','RGS6','RGS7','RGS9','RHEBL1','RHOXF1','RHOXF2','RHOXF2B','RING1','RIOK2','RIPK1','RIPK2','RIPK3','RIPK4','RLF','RLIM','RNASE2','RNASEH1','RNF113A','RNF113B','RNF114','RNF125','RNF138','RNF14','RNF166','RNF2','RNF219','RNF25','RNF31','RNF4','RNF41','RORA','RORB','RORC','RPA2','RPA4','RPL36A','RPL7','RPL7A','RPS27A','RPS6KA4','RPS6KA5','RREB1','RRN3','RUNX1','RUNX1T1','RUNX2','RUNX3','RWDD3','RXRA','RXRB','RXRG','RYBP','S100A12','S100A8','S100A9','SAFB','SAFB2','SALL1','SALL2','SALL3','SALL4','SART1','SATB1','SATB2','SAV1','SCAND1','SCAND2P','SCAPER','SCMH1','SCML1','SCML2','SCML4','SCOC','SCRT1','SCRT2','SCX','SEBOX','SEMA4A','SERTAD1','SERTAD2','SETBP1','SETD6','SETDB1','SETDB2','SF3A2','SF3A3','SFPQ','SFRP4','SFRP5','SFTPB','SGK1','SGK3','SGSM2','SHH','SHOX','SHOX2','SHPRH','SIGIRR','SIK1','SIM1','SIM2','SIN3A','SIN3B','SIRT1','SIRT2','SIRT3','SIRT6','SIVA1','SIX1','SIX2','SIX3','SIX4','SIX5','SIX6','SKI','SKIDA1','SKIL','SKOR1','SKOR2','SLC22A4','SLC26A10','SLC26A3','SLC2A4RG','SLC30A9','SLC39A10','SLC4A10','SMAD1','SMAD2','SMAD3','SMAD4','SMAD5','SMAD6','SMAD7','SMAD9','SMARCA1','SMARCA2','SMARCA4','SMARCA5','SMARCAL1','SMARCB1','SMARCC1','SMARCC2','SMARCD1','SMARCD2','SMARCD3','SMARCE1','SMC3','SMO','SMUG1','SMYD1','SMYD3','SNAI1','SNAI2','SNAI3','SNAPC2','SNAPC4','SNAPC5','SNRPC','SNURF','SNW1','SOGA3','SOHLH1','SOHLH2','SON','SORBS2','SOX1','SOX10','SOX11','SOX12','SOX13','SOX14','SOX15','SOX17','SOX18','SOX2','SOX21','SOX3','SOX30','SOX4','SOX5','SOX6','SOX7','SOX8','SOX9','SP1','SP100','SP110','SP140','SP140L','SP2','SP3','SP4','SP5','SP6','SP7','SP8','SP9','SPDEF','SPEN','SPHK1','SPI1','SPIB','SPIC','SPZ1','SRCAP','SREBF1','SREBF2','SRF','SRY','SSB','SSBP2','SSBP3','SSBP4','SSH1','SSH2','SSH3','SSRP1','SSX1','ST18','STAG1','STAT1','STAT2','STAT3','STAT4','STAT5A','STAT5B','STAT6','STK16','STK3','STK36','STRN','STRN3','SUB1','SUFU','SUMO1','SUPT16H','SUPT4H1','SUPT5H','SUPT6H','SUV39H1','SUV39H2','SUZ12','SYK','T','TAB1','TAB2','TAB3','TADA2A','TADA2B','TADA3','TAF1','TAF10','TAF11','TAF12','TAF13','TAF1A','TAF1B','TAF1C','TAF1L','TAF2','TAF3','TAF4','TAF4B','TAF5','TAF5L','TAF6','TAF6L','TAF7','TAF7L','TAF8','TAF9','TAF9B','TAL1','TAL2','TARDBP','TAX1BP1','TBL1Y','TBP','TBPL1','TBPL2','TBR1','TBX1','TBX10','TBX15','TBX18','TBX19','TBX2','TBX20','TBX21','TBX22','TBX3','TBX4','TBX5','TBX6','TCEA2','TCEAL1','TCEAL7','TCEAL8','TCF12','TCF15','TCF19','TCF20','TCF21','TCF23','TCF24','TCF25','TCF3','TCF4','TCF7','TCF7L1','TCF7L2','TCFL5','TDG','TDP1','TDP2','TEAD1','TEAD2','TEAD3','TEAD4','TEF','TERB1','TERF1','TERF2','TERF2IP','TESMIN','TET1','TET2','TET3','TFAM','TFAP2A','TFAP2B','TFAP2C','TFAP2D','TFAP2E','TFAP4','TFCP2','TFCP2L1','TFDP1','TFDP2','TFDP3','TFE3','TFEB','TFEC','TGFB1','TGFB1I1','TGIF1','TGIF2','TGIF2LX','TGIF2LY','TGS1','THAP1','THAP10','THAP11','THAP12','THAP2','THAP3','THAP4','THAP5','THAP6','THAP7','THAP8','THAP9','THRA','THRAP3','THRB','THYN1','TIA1','TICAM1','TIGD1','TIGD2','TIGD3','TIGD4','TIGD5','TIGD6','TIGD7','TIPARP','TIRAP','TLE1','TLE2','TLE3','TLE4','TLE6','TLR2','TLR3','TLR4','TLR9','TLX1','TLX2','TLX3','TMEM235','TMF1','TNF','TNFAIP3','TNFRSF11A','TNFRSF4','TNFSF11','TNFSF18','TNFSF4','TOE1','TONSL','TOP1','TOP2A','TOP2B','TOPORS','TOX','TOX2','TOX3','TOX4','TP53','TP53BP1','TP63','TP73','TPRX1','TRADD','TRAF1','TRAF2','TRAF3','TRAF4','TRAF5','TRAF6','TRAFD1','TRAK2','TRAPPC9','TRERF1','TREX1','TRIB1','TRIB3','TRIM13','TRIM14','TRIM15','TRIM21','TRIM22','TRIM23','TRIM24','TRIM25','TRIM26','TRIM27','TRIM28','TRIM29','TRIM3','TRIM31','TRIM32','TRIM33','TRIM34','TRIM37','TRIM38','TRIM40','TRIM5','TRIM52','TRIM62','TRIM8','TRIP11','TRIT1','TRMT1','TRMT1L','TRPS1','TRRAP','TSC2','TSC22D1','TSC22D2','TSC22D3','TSC22D4','TSHZ1','TSHZ2','TSHZ3','TSN','TSSK4','TTF1','TUB','TULP4','TUT1','TWIST1','TWIST2','TXK','U2AF1','U2AF1L4','UBA52','UBB','UBC','UBE2K','UBE2N','UBE2V1','UBN1','UBP1','UBR4','UBTF','UBTFL1','UBTFL6','UFL1','UHRF1','UHRF2','UNCX','UNG','UNK','UNKL','USF1','USF2','USF3','USP39','USP7','UTF1','VAV1','VAX1','VAX2','VDR','VEGFA','VENTX','VEZF1','VGLL1','VGLL2','VGLL3','VHLL','VPS72','VSX1','VSX2','WDHD1','WFS1','WIZ','WNT1','WNT10B','WNT2','WNT3A','WNT5A','WNT8B','WRN','WT1','WWOX','WWP2','WWTR1','XBP1','XCL1','XPA','YBX1','YBX2','YBX3','YEATS2','YEATS4','YOD1','YY1','YY2','ZBED1','ZBED2','ZBED3','ZBED4','ZBED5','ZBED6','ZBED6CL','ZBED8','ZBED9','ZBP1','ZBTB1','ZBTB10','ZBTB11','ZBTB12','ZBTB14','ZBTB16','ZBTB17','ZBTB18','ZBTB2','ZBTB20','ZBTB21','ZBTB22','ZBTB24','ZBTB25','ZBTB26','ZBTB3','ZBTB32','ZBTB33','ZBTB34','ZBTB37','ZBTB38','ZBTB39','ZBTB4','ZBTB40','ZBTB41','ZBTB42','ZBTB43','ZBTB44','ZBTB45','ZBTB46','ZBTB47','ZBTB48','ZBTB49','ZBTB5','ZBTB6','ZBTB7A','ZBTB7B','ZBTB7C','ZBTB8A','ZBTB8B','ZBTB9','ZC3H10','ZC3H11A','ZC3H12A','ZC3H13','ZC3H14','ZC3H15','ZC3H18','ZC3H3','ZC3H4','ZC3H6','ZC3H7A','ZC3H7B','ZC3H8','ZC3HAV1','ZCCHC11','ZCCHC6','ZDHHC11','ZDHHC19','ZEB1','ZEB2','ZFAND5','ZFAT','ZFHX2','ZFHX3','ZFHX4','ZFP1','ZFP14','ZFP2','ZFP28','ZFP3','ZFP30','ZFP36','ZFP36L1','ZFP36L2','ZFP37','ZFP41','ZFP42','ZFP57','ZFP62','ZFP64','ZFP69','ZFP69B','ZFP82','ZFP90','ZFP91','ZFP92','ZFPL1','ZFPM1','ZFPM2','ZFR','ZFR2','ZFX','ZFY','ZFYVE26','ZGLP1','ZGPAT','ZHX1','ZHX2','ZHX3','ZIC1','ZIC2','ZIC3','ZIC4','ZIC5','ZIK1','ZIM2','ZIM3','ZKSCAN1','ZKSCAN2','ZKSCAN3','ZKSCAN4','ZKSCAN5','ZKSCAN7','ZKSCAN8','ZMAT1','ZMAT2','ZMAT3','ZMAT4','ZMAT5','ZMYM1','ZMYM2','ZMYM3','ZMYM4','ZMYM5','ZMYM6','ZMYND10','ZNF10','ZNF100','ZNF101','ZNF106','ZNF107','ZNF112','ZNF114','ZNF117','ZNF12','ZNF121','ZNF124','ZNF131','ZNF132','ZNF133','ZNF134','ZNF135','ZNF136','ZNF137P','ZNF138','ZNF14','ZNF140','ZNF141','ZNF142','ZNF143','ZNF146','ZNF148','ZNF154','ZNF155','ZNF157','ZNF16','ZNF160','ZNF165','ZNF169','ZNF17','ZNF174','ZNF175','ZNF177','ZNF18','ZNF180','ZNF181','ZNF182','ZNF184','ZNF185','ZNF189','ZNF19','ZNF195','ZNF197','ZNF2','ZNF20','ZNF200','ZNF202','ZNF205','ZNF207','ZNF208','ZNF211','ZNF212','ZNF213','ZNF214','ZNF215','ZNF217','ZNF219','ZNF22','ZNF221','ZNF222','ZNF223','ZNF224','ZNF225','ZNF226','ZNF227','ZNF229','ZNF23','ZNF230','ZNF232','ZNF233','ZNF234','ZNF235','ZNF236','ZNF239','ZNF24','ZNF248','ZNF25','ZNF250','ZNF251','ZNF253','ZNF254','ZNF256','ZNF257','ZNF26','ZNF260','ZNF263','ZNF264','ZNF266','ZNF267','ZNF268','ZNF273','ZNF274','ZNF275','ZNF276','ZNF277','ZNF28','ZNF280A','ZNF280B','ZNF280C','ZNF280D','ZNF281','ZNF282','ZNF283','ZNF284','ZNF285','ZNF286A','ZNF286B','ZNF287','ZNF292','ZNF296','ZNF3','ZNF30','ZNF300','ZNF302','ZNF304','ZNF311','ZNF316','ZNF317','ZNF318','ZNF319','ZNF32','ZNF320','ZNF322','ZNF324','ZNF324B','ZNF326','ZNF329','ZNF330','ZNF331','ZNF333','ZNF334','ZNF335','ZNF337','ZNF33A','ZNF33B','ZNF34','ZNF341','ZNF343','ZNF345','ZNF346','ZNF347','ZNF35','ZNF350','ZNF354A','ZNF354B','ZNF354C','ZNF355P','ZNF358','ZNF362','ZNF365','ZNF366','ZNF367','ZNF37A','ZNF382','ZNF383','ZNF384','ZNF385A','ZNF385B','ZNF385C','ZNF385D','ZNF391','ZNF394','ZNF395','ZNF396','ZNF397','ZNF398','ZNF404','ZNF407','ZNF408','ZNF41','ZNF410','ZNF414','ZNF415','ZNF416','ZNF417','ZNF418','ZNF419','ZNF420','ZNF423','ZNF425','ZNF426','ZNF428','ZNF429','ZNF43','ZNF430','ZNF431','ZNF432','ZNF433','ZNF436','ZNF438','ZNF439','ZNF44','ZNF440','ZNF441','ZNF442','ZNF443','ZNF444','ZNF445','ZNF446','ZNF449','ZNF45','ZNF451','ZNF454','ZNF460','ZNF461','ZNF462','ZNF467','ZNF468','ZNF469','ZNF470','ZNF471','ZNF473','ZNF474','ZNF479','ZNF48','ZNF480','ZNF483','ZNF484','ZNF485','ZNF486','ZNF487','ZNF488','ZNF490','ZNF491','ZNF492','ZNF493','ZNF496','ZNF497','ZNF500','ZNF501','ZNF502','ZNF503','ZNF506','ZNF507','ZNF510','ZNF511','ZNF512','ZNF512B','ZNF513','ZNF514','ZNF516','ZNF517','ZNF518A','ZNF518B','ZNF519']
TFs3Other=['ZNF521','ZNF524','ZNF525','ZNF526','ZNF527','ZNF528','ZNF529','ZNF530','ZNF532','ZNF534','ZNF536','ZNF540','ZNF541','ZNF542P','ZNF543','ZNF544','ZNF546','ZNF547','ZNF548','ZNF549','ZNF550','ZNF551','ZNF552','ZNF554','ZNF555','ZNF556','ZNF557','ZNF558','ZNF559','ZNF560','ZNF561','ZNF562','ZNF563','ZNF564','ZNF565','ZNF566','ZNF567','ZNF568','ZNF569','ZNF57','ZNF570','ZNF571','ZNF572','ZNF573','ZNF574','ZNF575','ZNF576','ZNF577','ZNF578','ZNF579','ZNF580','ZNF581','ZNF582','ZNF583','ZNF584','ZNF585A','ZNF585B','ZNF586','ZNF587','ZNF587B','ZNF589','ZNF592','ZNF593','ZNF594','ZNF595','ZNF596','ZNF597','ZNF598','ZNF599','ZNF600','ZNF605','ZNF606','ZNF607','ZNF608','ZNF609','ZNF610','ZNF611','ZNF613','ZNF614','ZNF615','ZNF616','ZNF618','ZNF619','ZNF620','ZNF621','ZNF622','ZNF623','ZNF624','ZNF625','ZNF626','ZNF627','ZNF628','ZNF629','ZNF630','ZNF638','ZNF639','ZNF641','ZNF644','ZNF645','ZNF646','ZNF648','ZNF649','ZNF652','ZNF653','ZNF654','ZNF655','ZNF658','ZNF66','ZNF660','ZNF662','ZNF664','ZNF665','ZNF667','ZNF668','ZNF669','ZNF670','ZNF671','ZNF672','ZNF674','ZNF675','ZNF676','ZNF677','ZNF678','ZNF679','ZNF680','ZNF681','ZNF682','ZNF683','ZNF684','ZNF687','ZNF688','ZNF689','ZNF69','ZNF691','ZNF692','ZNF695','ZNF696','ZNF697','ZNF699','ZNF7','ZNF70','ZNF700','ZNF701','ZNF703','ZNF704','ZNF705A','ZNF705B','ZNF705D','ZNF705E','ZNF705G','ZNF706','ZNF707','ZNF708','ZNF709','ZNF71','ZNF710','ZNF711','ZNF713','ZNF714','ZNF716','ZNF717','ZNF718','ZNF720','ZNF721','ZNF724','ZNF726','ZNF727','ZNF728','ZNF729','ZNF73','ZNF730','ZNF732','ZNF735','ZNF736','ZNF737','ZNF738','ZNF74','ZNF740','ZNF746','ZNF747','ZNF749','ZNF750','ZNF75A','ZNF75D','ZNF76','ZNF761','ZNF763','ZNF764','ZNF765','ZNF766','ZNF768','ZNF77','ZNF770','ZNF771','ZNF772','ZNF773','ZNF774','ZNF775','ZNF776','ZNF777','ZNF778','ZNF780A','ZNF780B','ZNF781','ZNF782','ZNF783','ZNF784','ZNF785','ZNF786','ZNF787','ZNF788','ZNF789','ZNF79','ZNF790','ZNF791','ZNF792','ZNF793','ZNF799','ZNF8','ZNF80','ZNF800','ZNF804A','ZNF804B','ZNF805','ZNF808','ZNF81','ZNF812P','ZNF813','ZNF814','ZNF816','ZNF821','ZNF823','ZNF827','ZNF829','ZNF83','ZNF830','ZNF831','ZNF835','ZNF836','ZNF837','ZNF839','ZNF84','ZNF841','ZNF843','ZNF844','ZNF845','ZNF846','ZNF85','ZNF850','ZNF852','ZNF853','ZNF860','ZNF862','ZNF865','ZNF878','ZNF879','ZNF880','ZNF883','ZNF888','ZNF891','ZNF90','ZNF91','ZNF92','ZNF93','ZNF98','ZNF99','ZNFX1','ZRANB2','ZRSR1','ZRSR2','ZSCAN1','ZSCAN10','ZSCAN12','ZSCAN16','ZSCAN18','ZSCAN2','ZSCAN20','ZSCAN21','ZSCAN22','ZSCAN23','ZSCAN25','ZSCAN26','ZSCAN29','ZSCAN30','ZSCAN31','ZSCAN32','ZSCAN4','ZSCAN5A','ZSCAN5B','ZSCAN5C','ZSCAN5DP','ZSCAN9','ZUFSP','ZXDA','ZXDB','ZXDC','ZZZ3']
TFs4Other=['ATF2','ATF6B','ATN1','BHLHA15','BHLHB2','BHLHB4 ','BHLHB5','BHLHB9','C13ORF8','C20ORF194','CDX2','CRAMP1L','ELP3','ERVK3-1','ETF1','FLJ12895','FLJ20321','FLJ25476','FLJ35784','FOXM1','FOXO2','FOXR1','FOXR2','GATA1','GRLF1','KDM4C','KDM4E','KIAA0863','KIAA0961','KLRG1','LBXCOR1','LEF1','LHX7','LOC126295','LOC51058','LOC91661','MGC4728','MYOD1','MYOG','MYRF','NEUROD4','NEUROG1','NFAT2','NFAT4','NFE2L1','NR1A2','NR1B1','NR1B2','NR1B3','NR1C1','NR1C2','NR1C3','NR1F3','NR1I1','NR2A1','NR2B3','NR3A1','NR3A2','NR3B3','OTEX','PEPP-2','PHF2','POU2F1','POU2F2','POU4F3','POU5F1','POU6F1','PRDM3','PRKRIP1','PRKRIR','SETD1A','SMAD8','SMYD2','SOX12','SOX14','SOX15','TBX14','TCF1','TCF17','TCF2','TCF9','TEAD1','TPRXL','UBTFL1','WHSC1','ZBTB14','ZBTB18','ZBTB19','ZBTB21','ZBTB27','ZBTB28','ZBTB35','ZBTB49','ZFP161','ZFP28','ZKSCAN10','ZKSCAN11','ZKSCAN12','ZKSCAN13','ZKSCAN14','ZKSCAN16','ZKSCAN19','ZKSCAN21','ZKSCAN24','ZKSCAN6','ZKSCAN8','ZNF10','ZNF112','ZNF187','ZNF193','ZNF204P','ZNF238','ZNF271','ZNF295','ZNF305','ZNF323','ZNF434','ZNF499','ZNF542','ZNF545','ZNF553','ZNF56','ZNF625','ZNF643','ZNF698','ZNF705F','ZNF751','ZNF752','ZNF753','ZNF785','ZNF806','ZNF816','ZNF833P','ZNF906','ZSCAN1','ZSCAN25','ZSCAN26','ZSCAN31','ZSCAN32','ZSCAN5D','ZSCAN9']
TFsOther=TFs1Other+TFs2Other+TFs3Other+TFs4Other
```

Example 8 (unknown):
```unknown
S genes: ['MCM5', 'PCNA', 'TYMS', 'FEN1', 'MCM2', 'MCM4', 'RRM1', 'UNG', 'GINS2', 'MCM6', 'CDCA7', 'DTL', 'PRIM1', 'UHRF1', 'HELLS', 'RFC2', 'RPA2', 'NASP', 'RAD51AP1', 'GMNN', 'WDR76', 'SLBP', 'CCNE2', 'UBR7', 'POLD3', 'MSH2', 'ATAD2', 'RAD51', 'RRM2', 'CDC45', 'CDC6', 'EXO1', 'TIPIN', 'DSCC1', 'BLM', 'CASP8AP2', 'USP1', 'CLSPN', 'POLA1', 'CHAF1B', 'BRIP1', 'E2F8'] 

G2 genes: ['HMGB2', 'CDK1', 'NUSAP1', 'UBE2C', 'BIRC5', 'TPX2', 'TOP2A', 'NDC80', 'CKS2', 'NUF2', 'CKS1B', 'MKI67', 'TMPO', 'CENPF', 'TACC3', 'SMC4', 'CCNB2', 'CKAP2L', 'CKAP2', 'AURKB', 'BUB1', 'KIF11', 'ANP32E', 'TUBB4B', 'GTSE1', 'KIF20B', 'HJURP', 'CDCA3', 'CDC20', 'TTK', 'CDC25C', 'KIF2C', 'RANGAP1', 'NCAPD2', 'DLGAP5', 'CDCA2', 'CDCA8', 'ECT2', 'KIF23', 'HMMR', 'AURKA', 'PSRC1', 'ANLN', 'LBR', 'CKAP5', 'CENPE', 'CTCF', 'NEK2', 'G2E3', 'GAS2L3', 'CBX5', 'CENPA'] 

Hemoglobin genes set 1: ['HBB', 'HBD', 'HBG1', 'HBG2', 'HBE1', 'HBZ', 'HBM', 'HBA2', 'HBA1', 'HBQ1'] 

Hemoglobin genes set 2: ['AHSP', 'GYPA', 'RHAG', 'GFI1B', 'KEL', 'EPB42', 'ALAS2', 'TSPO2', 'GYPB', 'GYPE', 'SPTA1', 'RHCE', 'KLF1', 'RHD', 'SLC4A1', 'SNX22', 'SQSTM1', 'SOD1', 'FTH1', 'MAP1LC3B', 'SLC25A37', 'TRIM58', 'HMBS'] 

Mitochondrial genes: ['MT-ND1', 'MT-ND2', 'MT-CO1', 'MT-CO2', 'MT-ATP8', 'MT-ATP6', 'MT-CO3', 'MT-ND3', 'MT-ND4L', 'MT-ND4', 'MT-ND5', 'MT-ND6', 'MT-CYB'] 

Ribo genes: ['RPLP0', 'RPLP1', 'RPLP2', 'RPL3', 'RPL3L', 'RPL4', 'RPL5', 'RPL6', 'RPL7', 'RPL7A', 'RPL7L1', 'RPL8', 'RPL9', 'RPL10', 'RPL10A', 'RPL10L', 'RPL11', 'RPL12', 'RPL13', 'RPL13A', 'RPL14', 'RPL15', 'RPL17', 'RPL18', 'RPL18A', 'RPL19', 'RPL21', 'RPL22', 'RPL22L1', 'RPL23', 'RPL23A', 'RPL24', 'RPL26', 'RPL26L1', 'RPL27', 'RPL27A', 'RPL28', 'RPL29', 'RPL30', 'RPL31', 'RPL32', 'RPL34', 'RPL35', 'RPL35A', 'RPL36', 'RPL36A', 'RPL36AL', 'RPL37', 'RPL37A', 'RPL38', 'RPL39', 'RPL39L', 'UBA52', 'RPL41', 'RPSA', 'RPS2', 'RPS3', 'RPS3A', 'RPS4X', 'RPS4Y1', 'RPS4Y2', 'RPS5', 'RPS6', 'RPS7', 'RPS8', 'RPS9', 'RPS10', 'RPS11', 'RPS12', 'RPS13', 'RPS14', 'RPS15', 'RPS15A', 'RPS16', 'RPS17', 'RPS18', 'RPS19', 'RPS20', 'RPS21', 'RPS23', 'RPS24', 'RPS25', 'RPS26', 'RPS27', 'RPS27A', 'RPS27L', 'RPS28', 'RPS29', 'FAU'] 

Tissue dissociation stress associated genes: ['FOS', 'FOSB', 'JUN', 'JUNB', 'JUND', 'ZFP36', 'SOCS3', 'CEBPD', 'CEBPB', 'ATF3', 'EGR1', 'HSPA1A', 'HSPA1B', 'HSP90AA1', 'HSP90AB1', 'HSPA8', 'HSPB1', 'HSPE1', 'IER3', 'IER2', 'BTG1', 'BTG2', 'DUSP1']
```

Example 9 (python):
```python
#Define color palette
FineGrainPalette={
    'AtrialCardiomyocytesLeft':'#eac075',
    'AtrialCardiomyocytesRight':'#4c6b8a',
    'AtrialCardiomyocytesCycling':'#e94bc7',
    'VentricularCardiomyocytesLeftCompact':'#ab1c3c',
    'VentricularCardiomyocytesRightCompact':'#fdba45',
    'VentricularCardiomyocytesLeftTrabeculated':'#2b9200',
    'VentricularCardiomyocytesRightTrabeculated':'#0273a8',
    'VentricularCardiomyocytesCycling':'#e1acff',
    'SinoatrialNodeCardiomyocytes':'#854834',
    'AtrioventricularNodeCardiomyocytes':'#688500',
    'VentricularConductionSystemProximal':'#01ccc8',
    'VentricularConductionSystemDistal':'#b58fab',
    'GreatVesselAdventitialFibroblasts':'#ff7e97',
    'CoronaryVesselAdventitialFibroblasts':'#ff9b6e',
    'MyocardialInterstitialFibroblasts1':'#8a6800',
    'MyocardialInterstitialFibroblasts2':'#206601',
    'Myofibroblasts':'#1bdebf',
    'LymphNodeFibroblasticReticularCells':'#005aa0',
    'ValveInterstitialCells':'#e478ff',
    'GreatVesselSmoothMuscleCells':'#ff665f',
    'CoronarySmoothMuscleCells':'#d2cb18',
    'DuctusArteriosusSmoothMuscleCells':'#83d7b3',
    'CoronaryPericytes':'#9088ff',
    'PericardialCellsIntermediate':'#c95e00',
    'PericardialCellsFibrous':'#01e077',
    'PericardialCellsParietal':'#ff88e4',
    'GreatVesselArterialEndothelialCells':'#d4004b',
    'GreatVesselVenousEndothelialCells':'#8f4300',
    'CoronaryArterialEndothelialCells':'#89da5a',
    'CoronaryVenousEndothelialCells':'#006f86',
    'CoronaryCapillaryEndothelialCells':'#aa7dff',
    'EndocardialCells':'#ff8241',
    'EndocardialCushionCells':'#46603a',
    'ValveEndothelialCells':'#414eb6',
    'LymphaticEndothelialCells':'#8b3f60',
    'MesothelialEpicardialCells':'#0295fe',
    'EpicardiumDerivedCells':'#ff92c4',
    'NeuronPrecursors':'#d30038',
    'ChromaffinCells':'#726800',
    'SympatheticNeurons':'#01894b',
    'ParaympatheticNeurons':'#545191',
    'SchwannCellPrecursors':'#b0d276',
    'SchwannCells':'#b30079',
    'MonocytesMPOpos':'#c50066',
    'Monocytes':'#fab4b6',
    'MonocyteDerivedCells':'#d14008',
    'MacrophagesCX3CR1pos':'#eabe97',
    'MacrophagesTIMD4pos':'#b2a200',
    'MacrophagesLYVE1pos':'#4a6005',
    'MacrophagesATF3pos':'#3db833',
    'DendriticCellsType1':'#007859',
    'DendriticCellsMature':'#01a9db',
    'PlasmacytoidDendriticCells':'#2c54ab',
    'MastCells':'#d0bef2',
    'Megakaryocytes':'#94298f',
    'TCellsCD4pos':'#ff387c',
    'TCellsCD8pos':'#9e3429',
    'TregsCD4pos':'#b37700',
    'ProBCells':'#8dab00',
    'BCells':'#a8d1a1',
    'BCellsMS4A1pos':'#00d2f0',
    'NaturalKillerCells':'#abb4ff',
    'InnateLymphoidCells':'#72497f',
}
sc.pl.umap(adata, color='fine_grain',palette=FineGrainPalette)

MidGrainPalette={
    'AtrialCardiomyocytes':'#af3a4d',
    'VentricularCardiomyocytes':'#b1756c',
    'CardiacConductionSystem':'#d45633',
    'Fibroblasts':'#9b6e4e',
    'MuralCells':'#ffb92f',
    'PericardialCells':'#b4f03b',
    'BloodVesselEndothelialCells':'#72cc57',
    'EndocardialCells':'#436e43',
    'LymphaticEndothelialCells':'#85cca0',
    'EpicardialCells':'#5eabb0',
    'Neurons':'#7bbdff',
    'Glia':'#403c90',
    'MyeloidCells':'#420066',
    'LymphoidCells':'#f553f6',
}
sc.pl.umap(adata, color='mid_grain',palette=MidGrainPalette)

CoarseGrainPalette={
    'Cardiomyocytes':'#bc5950',
    'Mesenchymal':'#af9c4c',
    'Endothelium':'#599e58',
    'Epicardium':'#5eabb0',
    'Neural':'#5e75b1',
    'Leukocytes':'#9a52a0',
}
sc.pl.umap(adata, color='coarse_grain',palette=CoarseGrainPalette)

RegionGrainPalette={
    'aorta':'#ffa2bc',
    'apex':'#fe0049',
    'atria':'#732d00',
    'atrial septum':'#ffd54c',
    'base':'#837100',
    'ductus arteriosus':'#bdff8b',
    'whole sample':'#1b6c00',
    'heart without node':'#02e6c6',
    'left ventricle':'#01c1f5',
    'node':'#00466d',
    'outflow tract':'#020011',
    'pericardium':'#7a75ff',
    'pulmonary arches and branches':'#30003e',
    'right ventricle':'#ff88e1',
    'right ventricle and papillaries':'#98004f',
}
sc.pl.umap(adata, color='region',palette=RegionGrainPalette)
```

Example 10 (python):
```python
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
/tmp/ipykernel_61579/2162444827.py in <module>
     65     'InnateLymphoidCells':'#72497f',
     66 }
---> 67 sc.pl.umap(adata, color='fine_grain',palette=FineGrainPalette)
     68 
     69 MidGrainPalette={

~/miniconda3/envs/scanpynew/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py in umap(adata, **kwargs)
    657     tl.umap
    658     """
--> 659     return embedding(adata, 'umap', **kwargs)
    660 
    661 

~/miniconda3/envs/scanpynew/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py in embedding(adata, basis, color, gene_symbols, use_raw, sort_order, edges, edges_width, edges_color, neighbors_key, arrows, arrows_kwds, groups, components, layer, projection, scale_factor, color_map, cmap, palette, na_color, na_in_legend, size, frameon, legend_fontsize, legend_fontweight, legend_loc, legend_fontoutline, vmax, vmin, vcenter, norm, add_outline, outline_width, outline_color, ncols, hspace, wspace, title, show, save, ax, return_fig, **kwargs)
    250             groups=groups,
    251         )
--> 252         color_vector, categorical = _color_vector(
    253             adata,
    254             value_to_plot,

~/miniconda3/envs/scanpynew/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py in _color_vector(adata, values_key, values, palette, na_color)
   1267         color_map = {
   1268             k: to_hex(v)
-> 1269             for k, v in _get_palette(adata, values_key, palette=palette).items()
   1270         }
   1271         # If color_map does not have unique values, this can be slow as the

~/miniconda3/envs/scanpynew/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py in _get_palette(adata, values_key, palette)
   1233     values = pd.Categorical(adata.obs[values_key])
   1234     if palette:
-> 1235         _utils._set_colors_for_categorical_obs(adata, values_key, palette)
   1236     elif color_key not in adata.uns or len(adata.uns[color_key]) < len(
   1237         values.categories

~/miniconda3/envs/scanpynew/lib/python3.8/site-packages/scanpy/plotting/_utils.py in _set_colors_for_categorical_obs(adata, value_to_plot, palette)
    388         colors_list = [to_hex(x) for x in cmap(np.linspace(0, 1, len(categories)))]
    389     elif isinstance(palette, cabc.Mapping):
--> 390         colors_list = [to_hex(palette[k], keep_alpha=True) for k in categories]
    391     else:
    392         # check if palette is a list and convert it to a cycler, thus

~/miniconda3/envs/scanpynew/lib/python3.8/site-packages/scanpy/plotting/_utils.py in <listcomp>(.0)
    388         colors_list = [to_hex(x) for x in cmap(np.linspace(0, 1, len(categories)))]
    389     elif isinstance(palette, cabc.Mapping):
--> 390         colors_list = [to_hex(palette[k], keep_alpha=True) for k in categories]
    391     else:
    392         # check if palette is a list and convert it to a cycler, thus

KeyError: 'AtrioventricularNodePacemakerCells'
```

Example 11 (python):
```python
set(adata.obs.fine_grain)
```

Example 12 (unknown):
```unknown
{'AtrialCardiomyocytesCycling',
 'AtrialCardiomyocytesLeft',
 'AtrialCardiomyocytesRight',
 'AtrioventricularNodeCardiomyocytes',
 'BCells',
 'BCellsMS4A1pos',
 'ChromaffinCells',
 'CoronaryArterialEndothelialCells',
 'CoronaryCapillaryEndothelialCells',
 'CoronaryPericytes',
 'CoronarySmoothMuscleCells',
 'CoronaryVenousEndothelialCells',
 'CoronaryVesselAdventitialFibroblasts',
 'DendriticCellsMature',
 'DendriticCellsType1',
 'DuctusArteriosusSmoothMuscleCells',
 'EndocardialCells',
 'EndocardialCushionCells',
 'EpicardiumDerivedCells',
 'GreatVesselAdventitialFibroblasts',
 'GreatVesselArterialEndothelialCells',
 'GreatVesselSmoothMuscleCells',
 'GreatVesselVenousEndothelialCells',
 'InnateLymphoidCells',
 'LymphNodeFibroblasticReticularCells',
 'LymphaticEndothelialCells',
 'MacrophagesATF3pos',
 'MacrophagesCX3CR1pos',
 'MacrophagesLYVE1pos',
 'MacrophagesTIMD4pos',
 'MastCells',
 'Megakaryocytes',
 'MesothelialEpicardialCells',
 'MonocyteDerivedCells',
 'Monocytes',
 'MonocytesMPOpos',
 'MyocardialInterstitialFibroblasts1',
 'MyocardialInterstitialFibroblasts2',
 'Myofibroblasts',
 'NaturalKillerCells',
 'NeuronPrecursors',
 'ParaympatheticNeurons',
 'PericardialCellsFibrous',
 'PericardialCellsIntermediate',
 'PericardialCellsParietal',
 'PlasmacytoidDendriticCells',
 'ProBCells',
 'SchwannCellPrecursors',
 'SchwannCells',
 'SinoatrialNodeCardiomyocytes',
 'SympatheticNeurons',
 'TCellsCD4pos',
 'TCellsCD8pos',
 'TregsCD4pos',
 'ValveEndothelialCells',
 'ValveInterstitialCells',
 'VentricularCardiomyocytesCycling',
 'VentricularCardiomyocytesLeftCompact',
 'VentricularCardiomyocytesLeftTrabeculated',
 'VentricularCardiomyocytesRightCompact',
 'VentricularCardiomyocytesRightTrabeculated',
 'VentricularConductionSystemDistal',
 'VentricularConductionSystemProximal'}
```

Example 13 (python):
```python
set(adata.obs.mid_grain)
```

Example 14 (unknown):
```unknown
{'AtrialCardiomyocytes',
 'BloodVesselEndothelialCells',
 'CardiacConductionSystem',
 'EndocardialCells',
 'EpicardialCells',
 'Fibroblasts',
 'Glia',
 'LymphaticEndothelialCells',
 'LymphoidCells',
 'MuralCells',
 'MyeloidCells',
 'Neurons',
 'PericardialCells',
 'VentricularCardiomyocytes'}
```

Example 15 (python):
```python
set(adata.obs.coarse_grain)
```

Example 16 (unknown):
```unknown
{'Cardiomyocytes',
 'Endothelium',
 'Epicardium',
 'Leukocytes',
 'Mesenchymal',
 'Neural'}
```

Example 17 (python):
```python
#adata = adata[adata.obs['fine_grain'].isin(['CoronaryVesselAdventitialFibroblasts','MyocardialInterstitialFibroblasts1','MyocardialInterstitialFibroblasts2','Myofibroblasts','GreatVesselAdventitialFibroblasts'])].copy()
#adata = adata[adata.obs['fine_grain'].isin(['CoronaryArterialEndothelialCells','CoronaryCapillaryEndothelialCells','CoronaryVenousEndothelialCells','GreatVesselArterialEndothelialCells','GreatVesselVenousEndothelialCells'])].copy()
adata = adata[adata.obs['fine_grain'].isin(['PericardialCellsFibrous','PericardialCellsParietal','PericardialCellsIntermediate','EpicardiumDerivedCells','MesothelialEpicardialCells'])].copy()
#adata = adata[adata.obs['mid_grain'].isin(['PericardialCells'])].copy()
#adata=adata[adata.obs.mid_grain=='MuralCells',:].copy()
```

Example 18 (unknown):
```unknown
AnnData object with n_obs × n_vars = 20997 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score'
    var: 'gene_name_scRNA-0', 'gene_id'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'age_colors', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'cycling_colors', 'donor_colors', 'fine_grain_colors', 'heart_or_greatvessels_colors', 'hvg', 'kit_10x_colors', 'mid_grain_colors', 'modality_colors', 'neighbors', 'phase_colors', 'region_colors', 'sex_colors', 'trimester_colors', 'umap', 'week_colors'
    obsm: 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 19 (python):
```python
# preserve raw count
adata.layers["counts"] = adata.X.copy()

# normalise and log-transform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key='batch_key')
#sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5)

print(sum(adata.var.highly_variable))
sc.pl.highly_variable_genes(adata)

# freeze the state in `.raw`
adata.raw = adata

# subset hvgs
adata = adata[:, adata.var.highly_variable]

adata=adata.copy()

sc.tl.score_genes_cell_cycle(adata, s_genes=s_genes, g2m_genes=g2m_genes)
#sc.pp.regress_out(adata, ['S_score','G2M_score'])

#batch correction
scvi.model.SCVI.setup_anndata(adata, 
                              layer="counts", 
                              batch_key="batch_key",
                              categorical_covariate_keys=['phase'],
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo'])

vae = scvi.model.SCVI(adata, n_hidden = 128, n_latent = 50, n_layers = 3, dispersion = 'gene-batch')
vae.train(max_epochs=50)
    
# Convergence plot
x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
plt.plot(x, vae.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
plt.clf()

# add the latent space to adata
adata.obsm["X_scVI"] = vae.get_latent_representation()

sc.tl.score_genes(adata, stress_genes, score_name='stress_score')
sc.tl.score_genes(adata, hb_genes1, score_name='hb1_score')

sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata,min_dist=0.1)
#sc.tl.tsne(adata)
sc.pl.umap(adata, color=['log1p_total_counts','n_genes','n_genes_by_counts','total_counts','pct_counts_mt','pct_counts_ribo'],ncols=5)
sc.pl.umap(adata, color=['scrublet_score','stress_score', 'hb1_score','phase'], ncols=5,)

sc.pl.umap(adata, color=['donor','age','week','trimester','region','phase','cycling','cell_or_nuclei','modality','kit_10x','FACSgate','heart_or_greatvessels','coarse_grain','mid_grain','fine_grain'],wspace=0.8,ncols=5)
Genes=['TTN','TNNT2','MYOM1','LDB3','ALDH1A2','TNNT1','KRT19','UPK3B','BNC1','COL6A3','FBLN1','PCOLCE','LRRC17','PCDH18','EMCN','ECSCR','PECAM1','CDH5','ESAM', 'LAPTM5','PTPRC','LCP1','CD53','WAS','PHOX2B','NRXN1','STMN2','GRIK2','L1CAM']
sc.pl.umap(adata, color=Genes)
```

Example 20 (python):
```python
normalizing counts per cell
    finished (0:00:00)
extracting highly variable genes
    finished (0:00:09)
--> added
    'highly_variable', boolean vector (adata.var)
    'means', float vector (adata.var)
    'dispersions', float vector (adata.var)
    'dispersions_norm', float vector (adata.var)
1249
```

Example 21 (python):
```python
calculating cell cycle phase
computing score 'S_score'
WARNING: genes are not in var_names and ignored: ['MCM5', 'FEN1', 'MCM2', 'MCM4', 'RRM1', 'UNG', 'GINS2', 'MCM6', 'CDCA7', 'DTL', 'PRIM1', 'UHRF1', 'RFC2', 'RPA2', 'NASP', 'RAD51AP1', 'GMNN', 'WDR76', 'SLBP', 'CCNE2', 'UBR7', 'POLD3', 'MSH2', 'RAD51', 'CDC45', 'CDC6', 'EXO1', 'TIPIN', 'DSCC1', 'BLM', 'CASP8AP2', 'USP1', 'CLSPN', 'POLA1', 'CHAF1B', 'BRIP1', 'E2F8']
    finished: added
    'S_score', score of gene set (adata.obs).
    207 total control genes are used. (0:00:00)
computing score 'G2M_score'
WARNING: genes are not in var_names and ignored: ['CKS1B', 'TMPO', 'CKAP2L', 'BUB1', 'ANP32E', 'HJURP', 'CDC25C', 'KIF2C', 'RANGAP1', 'NCAPD2', 'CDCA2', 'KIF23', 'PSRC1', 'ANLN', 'LBR', 'CKAP5', 'CTCF', 'NEK2', 'G2E3', 'GAS2L3', 'CBX5', 'CENPA']
    finished: added
    'G2M_score', score of gene set (adata.obs).
    483 total control genes are used. (0:00:00)
-->     'phase', cell cycle phase (adata.obs)
```

Example 22 (unknown):
```unknown
GPU available: False, used: False
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
```

Example 23 (unknown):
```unknown
Epoch 50/50: 100%|███████████| 50/50 [01:19<00:00,  1.59s/it, loss=749, v_num=1]
```

Example 24 (python):
```python
computing score 'stress_score'
WARNING: genes are not in var_names and ignored: ['JUND', 'CEBPB', 'HSP90AA1', 'HSP90AB1', 'HSPA8', 'HSPB1', 'HSPE1', 'BTG1']
    finished: added
    'stress_score', score of gene set (adata.obs).
    236 total control genes are used. (0:00:00)
computing score 'hb1_score'
WARNING: genes are not in var_names and ignored: ['HBB', 'HBD', 'HBE1', 'HBZ', 'HBM', 'HBQ1']
    finished: added
    'hb1_score', score of gene set (adata.obs).
    146 total control genes are used. (0:00:00)
computing neighbors
    finished: added to `.uns['neighbors']`
    `.obsp['distances']`, distances for each pair of neighbors
    `.obsp['connectivities']`, weighted adjacency matrix (0:00:03)
computing UMAP
    finished: added
    'X_umap', UMAP coordinates (adata.obsm) (0:00:05)
```

Example 25 (unknown):
```unknown
... storing 'phase' as categorical
```

Example 26 (unknown):
```unknown
<Figure size 320x320 with 0 Axes>
```

Example 27 (python):
```python
set(adata.obs.fine_grain)
```

Example 28 (unknown):
```unknown
{'EpicardiumDerivedCells',
 'MesothelialEpicardialCells',
 'PericardialCellsFibrous',
 'PericardialCellsIntermediate',
 'PericardialCellsParietal'}
```

Example 29 (python):
```python
ClusterOfInterest='PericardialCells'
sc.tl.rank_genes_groups(adata, 'mid_grain', method='t-test', key_added = "t-test")
DEGenesForCluster = [x[ClusterOfInterest] for x in adata.uns['t-test']['names'][:50]]
sc.pl.dotplot(adata, DEGenesForCluster ,groupby='fine_grain',standard_scale='var',cmap='PRGn')
#sc.pl.umap(adata, color=DEGenesForCluster)
```

Example 30 (python):
```python
ranking genes
    finished: added to `.uns['t-test']`
    'names', sorted np.recarray to be indexed by group ids
    'scores', sorted np.recarray to be indexed by group ids
    'logfoldchanges', sorted np.recarray to be indexed by group ids
    'pvals', sorted np.recarray to be indexed by group ids
    'pvals_adj', sorted np.recarray to be indexed by group ids (0:00:00)
```

Example 31 (python):
```python
ClusterOfInterest='PericardialCellsParietal'
#sc.tl.rank_genes_groups(adata, 'fine_grain', method='t-test', key_added = "t-test")
DEGenesForCluster = [x[ClusterOfInterest] for x in adata.uns['t-test']['names'][:50]]
sc.pl.dotplot(adata, DEGenesForCluster ,groupby='fine_grain',standard_scale='var',cmap='PRGn')
#sc.pl.umap(adata, color=DEGenesForCluster)
```

Example 32 (python):
```python
FineGrainCategoriesOrder=['MesothelialEpicardialCells','EpicardiumDerivedCells','PericardialCellsFibrous','PericardialCellsIntermediate','PericardialCellsParietal']
sc.pl.dotplot(adata,  ['TPM1','ALDH1A2','EZR','CFI','KRT8','KRT18','KRT19','BNC1','MSLN','HAS1','PRG4','COL6A3','COL12A1','POSTN','VCAN','COL16A1','SBSPON','ITLN1','TBX18','KDR','FLRT3','CPB1','VEGFC','MEG3','GPC3','PCOLCE','SPARCL1','ITM2A','SERPINF1','TSHZ2','AUTS2','SETBP1','PID1','PTPRM','PDZRN3','CD9','CD63','CD81','COL1A1','COL1A2','COL3A1'] , var_group_positions=[(0,10),(11,15),(16,18),(19,22),(25,28),(29,34),(35,37)], var_group_labels=['epicardial','pericardial','mesothelial epicardial','epicardium derived','fibrous pericardial','parietal pericardial','exosomal'], var_group_rotation=90,groupby='fine_grain',categories_order=FineGrainCategoriesOrder,standard_scale='var',cmap='YlGn')
```

Example 33 (python):
```python
sc.pl.umap(adata, color='mid_grain')
sc.pl.umap(adata, color='fine_grain')
```

Example 34 (python):
```python
sc.pl.umap(adata, color=['COL2A1','TNMD','THBS4'])
```

Example 35 (python):
```python
#adata.write('./ObjectsToUpload/PericardiumAndEpicardium.h5ad')
```

Example 36 (python):
```python
sc.settings.set_figure_params(dpi=300, color_map='viridis', vector_friendly=False,  dpi_save=300, facecolor='white')
```

---

## Read in data¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/VisiumHD/no2_bin2cell_expand-label.html

**Contents:**
- Read in data¶
- Pre-processing¶
- Plot HE cell segmentation¶
- Select prob_thresh¶
- Expand labels¶
- Save the object¶

Let's slightly filter the object - require the genes to show up in three spots, and require the spots to have any information at all (the data is extremely sparse at this stage).

Visium HD suffers from variable bin sizing. When printing the chips, the 2um bins can have about 10% variability in their width/height. Inspecting the total counts per spot reveals a characteristic striped appearance, with some rows/columns capturing visibly fewer transcripts than others.

To overcome this, b2c.destripe() identifies a user-specified quantile (by default 0.99) of total counts for each row, then divides the counts of the spots in that row by that value. This procedure is then repeated for the columns. .obs[adjusted_counts_key] (by default "n_counts_adjusted") is obtained by multiplying the resulting per-spot factor by the global quantile of count totals, and the count matrix is by default rescaled to match it.

Prior to the adjustment, the count total has a visible striped appearance. The effect is diminished after the correction.

We can now load the resulting cell calls into the object. For any given bin, its corresponding pixel in the segmented image will be located and any potential label transferred over. It's important to inform b2c.insert_labels() of whether the segmented image was based on array (GEX visualisation of the grid) or spatial (rescaled H&E image) coordinates, which exact spatial representation matches the image, and what mpp was used.

Let's visualise the nuclear H&E segmentation on our selected region. The nuclei are identified successfully.

The package comes with a utility function to visualise StarDist segmentation on a fragment of the input image. As we can see, the calls are faithfully captured in the bins.

However, StarDist's H&E segmentation merely identifies nuclei, while there's more to a cell than just the nucleus. b2c.expand_labels() finds bins up to max_bin_distance (by default 2) bins away from a labelled nucleus, and joins them into the corresponding cell. In the event of a bin being equidistant from two nuclei, it's assigned to one based on the similarity of their gene expression profiles, as represented in PCA space.

it doesn't change --> go with 2

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import os

import bin2cell as b2c

#create directory for stardist input/output files
os.makedirs("stardist", exist_ok=True)
```

Example 2 (unknown):
```unknown
os.chdir('/nfs/team205/kk18/notebooks/Foetal/VisiumHD/re-seq_mpp0p3')
os.getcwd()
```

Example 3 (unknown):
```unknown
'/nfs/team205/kk18/notebooks/Foetal/VisiumHD/re-seq_mpp0p3'
```

Example 4 (unknown):
```unknown
!ls /lustre/scratch126/cellgen/team205/sharedData/ny1/24-11-26-visiumhd/HEA_FOET14880396/outs
```

Example 5 (unknown):
```unknown
binned_outputs	  metrics_summary.csv  probe_set.csv  web_summary.html
feature_slice.h5  molecule_info.h5     spatial
```

Example 6 (unknown):
```unknown
path = "/lustre/scratch126/cellgen/team205/sharedData/ny1/24-11-26-visiumhd/HEA_FOET14880396/outs/binned_outputs/square_002um"
spaceranger_image_path = '/lustre/scratch126/cellgen/team205/sharedData/ny1/24-11-26-visiumhd/HEA_FOET14880396/outs/spatial'
source_image_path = "/lustre/scratch126/cellgen/team205/kk18/VisiumHD/source_images/C194_HEA_0_FFPE_1_STAN_1EAF2_s3_2024_04_23_13_40_00.tiff"
```

Example 7 (python):
```python
adata = b2c.read_visium(path, 
                        spaceranger_image_path=spaceranger_image_path, 
                        source_image_path = source_image_path
                       )
adata.var_names_make_unique()
adata
```

Example 8 (unknown):
```unknown
anndata.py (1758): Variable names are not unique. To make them unique, call `.var_names_make_unique`.
anndata.py (1758): Variable names are not unique. To make them unique, call `.var_names_make_unique`.
```

Example 9 (unknown):
```unknown
AnnData object with n_obs × n_vars = 5032713 × 18085
    obs: 'in_tissue', 'array_row', 'array_col'
    var: 'gene_ids', 'feature_types', 'genome'
    uns: 'spatial'
    obsm: 'spatial'
```

Example 10 (python):
```python
sc.pp.filter_genes(adata, min_cells=3)
sc.pp.filter_cells(adata, min_counts=1)
adata
```

Example 11 (unknown):
```unknown
AnnData object with n_obs × n_vars = 5001851 × 18051
    obs: 'in_tissue', 'array_row', 'array_col', 'n_counts'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells'
    uns: 'spatial'
    obsm: 'spatial'
```

Example 12 (python):
```python
mpp = 0.3
b2c.scaled_he_image(adata, mpp=mpp, save_path="stardist/he.tiff")
```

Example 13 (unknown):
```unknown
Cropped spatial coordinates key: spatial_cropped_150_buffer
Image key: 0.3_mpp_150_buffer
```

Example 14 (python):
```python
b2c.destripe(adata,
             quantile=0.99, 
    counts_key='n_counts',
    factor_key='destripe_factor',
    adjusted_counts_key='n_counts_adjusted',
    adjust_counts=True)
```

Example 15 (unknown):
```unknown
_construct.py (149): Series.__getitem__ treating keys as positions is deprecated. In a future version, integer keys will always be treated as labels (consistent with DataFrame behavior). To access a value by position, use `ser.iloc[pos]`
```

Example 16 (python):
```python
#define a mask to easily pull out this region of the object in the future
mask = ((adata.obs['array_row'] >= 1450) & 
        (adata.obs['array_row'] <= 1550) & 
        (adata.obs['array_col'] >= 1400) & 
        (adata.obs['array_col'] <= 1600)
       )

bdata = adata[mask]
sc.pl.spatial(bdata, color=[None, "n_counts", "n_counts_adjusted"], color_map="OrRd",
              img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer")
```

Example 17 (unknown):
```unknown
anndata.py (1146): Trying to modify attribute `.var` of view, initializing view as actual.
anndata.py (1146): Trying to modify attribute `.var` of view, initializing view as actual.
```

Example 18 (python):
```python
adata.shape
```

Example 19 (unknown):
```unknown
(5001851, 18051)
```

Example 20 (python):
```python
prob_thresh_dict = {
    '0p01':0.01,
     '0p03':0.03,
    '0p05':0.05,
    '0p1':0.1,
    'default':'default'
}

for key,prob_thresh in prob_thresh_dict.items():
    print(f'### {key} ###')
    b2c.insert_labels(adata, 
                  labels_npz_path=f"stardist/he_{key}.npz", 
                  basis="spatial",
                  spatial_key="spatial_cropped_150_buffer",
                  mpp=mpp, 
                  labels_key=f"labels_he_{key}"
                 )

adata
```

Example 21 (unknown):
```unknown
### 0p01 ###
### 0p03 ###
### 0p05 ###
### 0p1 ###
### default ###
```

Example 22 (unknown):
```unknown
AnnData object with n_obs × n_vars = 5001851 × 18051
    obs: 'in_tissue', 'array_row', 'array_col', 'n_counts', 'destripe_factor', 'n_counts_adjusted', 'labels_he_0p01', 'labels_he_0p03', 'labels_he_0p05', 'labels_he_0p1', 'labels_he_default'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells'
    uns: 'spatial', 'bin2cell'
    obsm: 'spatial', 'spatial_cropped_150_buffer'
```

Example 23 (python):
```python
mask = ((adata.obs['array_row'] >= 1050) & 
        (adata.obs['array_row'] <= 1100) & 
        (adata.obs['array_col'] >= 1100) & 
        (adata.obs['array_col'] <= 1150)
       )

bdata = adata[mask]

for key in prob_thresh_dict.keys():
    #0 means unassigned
    bdata = bdata[bdata.obs[f'labels_he_{key}']>0]
    bdata.obs[f'labels_he_{key}'] = bdata.obs[f'labels_he_{key}'].astype(str)
    print(len(set(bdata.obs[f'labels_he_{key}'])))
    sc.pl.spatial(bdata, color=[None,f'labels_he_{key}'], 
                  img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer", legend_loc=None,
                  ncols=2
                 )
```

Example 24 (unknown):
```unknown
2683271253.py (12): Trying to modify attribute `.obs` of view, initializing view as actual.
```

Example 25 (unknown):
```unknown
2683271253.py (12): Trying to modify attribute `.obs` of view, initializing view as actual.
```

Example 26 (unknown):
```unknown
2683271253.py (12): Trying to modify attribute `.obs` of view, initializing view as actual.
```

Example 27 (unknown):
```unknown
2683271253.py (12): Trying to modify attribute `.obs` of view, initializing view as actual.
```

Example 28 (unknown):
```unknown
2683271253.py (12): Trying to modify attribute `.obs` of view, initializing view as actual.
```

Example 29 (python):
```python
prob_thresh_sel = '0p05' 
adata.obs['labels_he_sel'] = adata.obs[f'labels_he_{prob_thresh_sel}']
```

Example 30 (python):
```python
#the label viewer wants a crop of the processed image
#get the corresponding coordinates spanning the subset object
crop = b2c.get_crop(bdata, basis="spatial", spatial_key="spatial_cropped_150_buffer", mpp=mpp)

rendered = b2c.view_labels(image_path="stardist/he.tiff", 
                           labels_npz_path=f"stardist/he_{prob_thresh_sel}.npz", 
                           crop=crop
                          )
plt.imshow(rendered)
```

Example 31 (python):
```python
<matplotlib.image.AxesImage at 0x14c6e3bd1c30>
```

Example 32 (python):
```python
%%time
max_bin_distance_list = [1,2,3,4]
for max_bin_distance in max_bin_distance_list:
    print(max_bin_distance)
    b2c.expand_labels(adata, 
                      labels_key=f"labels_he_sel", 
                      expanded_labels_key=f"labels_he_expanded_{max_bin_distance}",
                      max_bin_distance=max_bin_distance
                     )
```

Example 33 (unknown):
```unknown
1
2
3
4
CPU times: user 5min 23s, sys: 14.2 s, total: 5min 37s
Wall time: 7min 18s
```

Example 34 (python):
```python
bdata = adata[mask]

for max_bin_distance in max_bin_distance_list:
    #0 means unassigned
    bdata = bdata[bdata.obs[f"labels_he_expanded_{max_bin_distance}"]>0]
    bdata.obs[f"labels_he_expanded_{max_bin_distance}"] = bdata.obs[f"labels_he_expanded_{max_bin_distance}"].astype(str)

    sc.pl.spatial(bdata, color=[None, f"labels_he_expanded_{max_bin_distance}"], 
                  img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer")
```

Example 35 (unknown):
```unknown
788255881.py (6): Trying to modify attribute `.obs` of view, initializing view as actual.
```

Example 36 (unknown):
```unknown
788255881.py (6): Trying to modify attribute `.obs` of view, initializing view as actual.
```

Example 37 (unknown):
```unknown
788255881.py (6): Trying to modify attribute `.obs` of view, initializing view as actual.
```

Example 38 (unknown):
```unknown
788255881.py (6): Trying to modify attribute `.obs` of view, initializing view as actual.
```

Example 39 (python):
```python
max_bin_distance_sel = 2
adata.obs['labels_he_expanded_sel'] = adata.obs[f"labels_he_expanded_{max_bin_distance_sel}"]
```

Example 40 (python):
```python
adata.write('/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/HEA_FOET14880396_b2c.h5ad')
```

---

## Functions¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/Visium/cell2location/wholeheart_without_DA/no2_mapping_OCT.html

**Contents:**
  - Loading packages¶
- Functions¶
- Read in metadata¶
- Data, parameters¶
- Loading visium data¶
- Filter out low quality spots¶
- Remove mitochondrial genes¶
- Add metadata and annotation¶
- Remove spots based on manual annotation¶
- Combine pcw13w-Atrial and pcw13w-Ventricle data to have whole heart in one sample¶
- Run per group: cell2location mapping¶

**Examples:**

Example 1 (python):
```python
import sys
import scanpy as sc
import anndata
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt 
import matplotlib as mpl

import cell2location
import scvi

from matplotlib import rcParams
rcParams['pdf.fonttype'] = 42 # enables correct plotting of text
import seaborn as sns
```

Example 2 (unknown):
```unknown
Global seed set to 0
```

Example 3 (python):
```python
import gc
import torch
```

Example 4 (python):
```python
def read_and_qc(sample_name, path):
    r""" This function reads the data for one 10X spatial experiment into the anndata object.
    It also calculates QC metrics. Modify this function if required by your workflow.

    :param sample_name: Name of the sample
    :param path: path to data
    """

    adata = sc.read_visium(path + str(sample_name),
                           count_file='filtered_feature_bc_matrix.h5', load_images=True)
    adata.var_names_make_unique(join='-')
    
    adata.obs['sample'] = sample_name
    
    # rename genes to ENSEMBL
    adata.var['SYMBOL'] = adata.var_names
    adata.var_names = adata.var['gene_ids']
    adata.var_names.name = None

    # Calculate QC metrics
    sc.pp.calculate_qc_metrics(adata, inplace=True)
    adata.var['mt'] = [gene.startswith('MT-') for gene in adata.var['SYMBOL']]
    adata.obs['mt_frac'] = adata[:, adata.var['mt'].tolist()].X.sum(1).A.squeeze()/adata.obs['total_counts']

    # add sample name to obs names
    adata.obs["sample"] = [str(i) for i in adata.obs['sample']]
    adata.obs_names = adata.obs["sample"] \
                          + '_' + adata.obs_names
    adata.obs.index.name = 'spot_id'

    return adata

def select_slide(adata, s, s_col='sample'):
    r""" This function selects the data for one slide from the spatial anndata object.

    :param adata: Anndata object with multiple spatial experiments
    :param s: name of selected experiment
    :param s_col: column in adata.obs listing experiment name for each location
    """

    slide = adata[adata.obs[s_col].isin([s]), :]
    s_keys = list(slide.uns['spatial'].keys())
    s_spatial = np.array(s_keys)[[s in k for k in s_keys]][0]

    slide.uns['spatial'] = {s_spatial: slide.uns['spatial'][s_spatial]}

    return slide
```

Example 5 (python):
```python
meta=pd.read_csv('/nfs/team205/heart/anndata_objects/Foetal/metadata/HeartTeamSamples_Mappeddata_20230208_for-cell2loc.csv',
                index_col=0)
meta
```

Example 6 (python):
```python
pd.crosstab(meta['donor'],meta['cell2loc-map_group'])
```

Example 7 (unknown):
```unknown
# select slides
# Healthy, OCT, and without DA
meta = meta[meta['cell2loc-map_group'].isin(['4PCW','5PCW','7PCW','13PCW','20PCW'])]

sampleIDs = list(meta['sangerID'])
len(sampleIDs)
```

Example 8 (unknown):
```unknown
results_folder = '/nfs/team205/heart/cell2location/foetal'

# create paths and names to results folders for reference regression and cell2location models
ref_run_name = f'{results_folder}/reference_signatures/ver28Feb2023/whole_woPeriCell_woDASMC'
run_name = f'{results_folder}/cell2location_map/ver28Feb2023'
```

Example 9 (unknown):
```unknown
visium_path='/nfs/team205/heart/mapped/spaceranger110/'

celltype_key='fine_grain'
```

Example 10 (python):
```python
# Read the data of the interested region into anndata objects
slides = []
for i in sampleIDs:
    slides.append(read_and_qc(i, path=visium_path))

# Combine anndata objects together
adata_vis = slides[0].concatenate(
    slides[1:],
    batch_key="sample",
    uns_merge="unique",
    batch_categories=sampleIDs,
    index_unique=None
)
```

Example 11 (unknown):
```unknown
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
```

Example 12 (python):
```python
# PLOT QC FOR EACH SAMPLE
fig, axs = plt.subplots(len(slides), 4, figsize=(15, 4*len(slides)-4))
for i, s in enumerate(adata_vis.obs['sample'].unique()):
    #fig.suptitle('Covariates for filtering')

    slide = select_slide(adata_vis, s)
    sns.distplot(slide.obs['total_counts'],
                 kde=False, ax = axs[i, 0])
    axs[i, 0].set_xlim(0, adata_vis.obs['total_counts'].max())
    axs[i, 0].set_xlabel(f'total_counts | {s}')

    sns.distplot(slide.obs['total_counts']\
                 [slide.obs['total_counts']<2000],
                 kde=False, bins=40, ax = axs[i, 1])
    axs[i, 1].set_xlim(0, 2000)
    axs[i, 1].set_xlabel(f'total_counts | {s}')

    sns.distplot(slide.obs['n_genes_by_counts'],
                 kde=False, bins=60, ax = axs[i, 2])
    axs[i, 2].set_xlim(0, adata_vis.obs['n_genes_by_counts'].max())
    axs[i, 2].set_xlabel(f'n_genes_by_counts | {s}')

    sns.distplot(slide.obs['n_genes_by_counts']\
                 [slide.obs['n_genes_by_counts']<1000],
                 kde=False, bins=60, ax = axs[i, 3])
    axs[i, 3].set_xlim(0, 1000)
    axs[i, 3].set_xlabel(f'n_genes_by_counts | {s}')

plt.tight_layout()
```

Example 13 (python):
```python
Trying to set attribute `._uns` of view, copying.
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/seaborn/distributions.py:2619: FutureWarning: `distplot` is a deprecated function and will be removed in a future version. Please adapt your code to use either `displot` (a figure-level function with similar flexibility) or `histplot` (an axes-level function for histograms).
Trying to set attribute `._uns` of view, copying.
Trying to set attribute `._uns` of view, copying.
Trying to set attribute `._uns` of view, copying.
Trying to set attribute `._uns` of view, copying.
Trying to set attribute `._uns` of view, copying.
Trying to set attribute `._uns` of view, copying.
Trying to set attribute `._uns` of view, copying.
Trying to set attribute `._uns` of view, copying.
Trying to set attribute `._uns` of view, copying.
Trying to set attribute `._uns` of view, copying.
Trying to set attribute `._uns` of view, copying.
Trying to set attribute `._uns` of view, copying.
Trying to set attribute `._uns` of view, copying.
Trying to set attribute `._uns` of view, copying.
Trying to set attribute `._uns` of view, copying.
Trying to set attribute `._uns` of view, copying.
Trying to set attribute `._uns` of view, copying.
```

Example 14 (python):
```python
# filter out low quality spots
sc.pp.filter_cells(adata_vis, min_counts = 500)
sc.pp.filter_cells(adata_vis, min_genes = 300)

# Spot number per slide
adata_vis.obs['sample'].value_counts()
```

Example 15 (unknown):
```unknown
HCAHeartST13162330    2806
HCAHeartST11605165    2429
HCAHeartST13162331    2338
HCAHeartST13162329    1777
HCAHeartST10317190    1337
HCAHeartST11605166    1321
HCAHeartST10298116    1219
HCAHeartST10298120    1170
HCAHeartST10298117    1126
HCAHeartST11605168    1082
HCAHeartST11605167    1044
HCAHeartST10298115    1025
HCAHeartST10317189    1004
HCAHeartST10317184     897
HCAHeartST10298114     872
HCAHeartST10317186     855
HCAHeartST10317185     824
HCAHeartST13162332     505
Name: sample, dtype: int64
```

Example 16 (python):
```python
# find mitochondria-encoded (MT) genes
adata_vis.var['MT_gene'] = [gene.startswith('MT-') for gene in adata_vis.var['SYMBOL']]
print(sum(adata_vis.var['MT_gene']))
    
# remove MT genes for spatial mapping (keeping their counts in the object)
adata_vis.obsm['MT'] = adata_vis[:, adata_vis.var['MT_gene'].values].X.toarray()
adata_vis = adata_vis[:, ~adata_vis.var['MT_gene'].values]
print(sum(adata_vis.var['MT_gene']))
```

Example 17 (unknown):
```unknown
meta.head()
```

Example 18 (python):
```python
adata_vis.obs = adata_vis.obs.reset_index().merge(meta, how='left', left_on='sample', right_on='sangerID').set_index('spot_id')
adata_vis
```

Example 19 (unknown):
```unknown
AnnData object with n_obs × n_vars = 23631 × 33525
    obs: 'in_tissue', 'array_row', 'array_col', 'sample', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'mt_frac', 'n_counts', 'n_genes', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'status', 'cell2loc-map_group'
    var: 'gene_ids', 'feature_types', 'genome', 'SYMBOL', 'mt', 'n_cells_by_counts-HCAHeartST10298114', 'mean_counts-HCAHeartST10298114', 'log1p_mean_counts-HCAHeartST10298114', 'pct_dropout_by_counts-HCAHeartST10298114', 'total_counts-HCAHeartST10298114', 'log1p_total_counts-HCAHeartST10298114', 'n_cells_by_counts-HCAHeartST10298115', 'mean_counts-HCAHeartST10298115', 'log1p_mean_counts-HCAHeartST10298115', 'pct_dropout_by_counts-HCAHeartST10298115', 'total_counts-HCAHeartST10298115', 'log1p_total_counts-HCAHeartST10298115', 'n_cells_by_counts-HCAHeartST10298116', 'mean_counts-HCAHeartST10298116', 'log1p_mean_counts-HCAHeartST10298116', 'pct_dropout_by_counts-HCAHeartST10298116', 'total_counts-HCAHeartST10298116', 'log1p_total_counts-HCAHeartST10298116', 'n_cells_by_counts-HCAHeartST10298117', 'mean_counts-HCAHeartST10298117', 'log1p_mean_counts-HCAHeartST10298117', 'pct_dropout_by_counts-HCAHeartST10298117', 'total_counts-HCAHeartST10298117', 'log1p_total_counts-HCAHeartST10298117', 'n_cells_by_counts-HCAHeartST10298120', 'mean_counts-HCAHeartST10298120', 'log1p_mean_counts-HCAHeartST10298120', 'pct_dropout_by_counts-HCAHeartST10298120', 'total_counts-HCAHeartST10298120', 'log1p_total_counts-HCAHeartST10298120', 'n_cells_by_counts-HCAHeartST10317184', 'mean_counts-HCAHeartST10317184', 'log1p_mean_counts-HCAHeartST10317184', 'pct_dropout_by_counts-HCAHeartST10317184', 'total_counts-HCAHeartST10317184', 'log1p_total_counts-HCAHeartST10317184', 'n_cells_by_counts-HCAHeartST10317185', 'mean_counts-HCAHeartST10317185', 'log1p_mean_counts-HCAHeartST10317185', 'pct_dropout_by_counts-HCAHeartST10317185', 'total_counts-HCAHeartST10317185', 'log1p_total_counts-HCAHeartST10317185', 'n_cells_by_counts-HCAHeartST10317186', 'mean_counts-HCAHeartST10317186', 'log1p_mean_counts-HCAHeartST10317186', 'pct_dropout_by_counts-HCAHeartST10317186', 'total_counts-HCAHeartST10317186', 'log1p_total_counts-HCAHeartST10317186', 'n_cells_by_counts-HCAHeartST10317189', 'mean_counts-HCAHeartST10317189', 'log1p_mean_counts-HCAHeartST10317189', 'pct_dropout_by_counts-HCAHeartST10317189', 'total_counts-HCAHeartST10317189', 'log1p_total_counts-HCAHeartST10317189', 'n_cells_by_counts-HCAHeartST10317190', 'mean_counts-HCAHeartST10317190', 'log1p_mean_counts-HCAHeartST10317190', 'pct_dropout_by_counts-HCAHeartST10317190', 'total_counts-HCAHeartST10317190', 'log1p_total_counts-HCAHeartST10317190', 'n_cells_by_counts-HCAHeartST11605165', 'mean_counts-HCAHeartST11605165', 'log1p_mean_counts-HCAHeartST11605165', 'pct_dropout_by_counts-HCAHeartST11605165', 'total_counts-HCAHeartST11605165', 'log1p_total_counts-HCAHeartST11605165', 'n_cells_by_counts-HCAHeartST11605166', 'mean_counts-HCAHeartST11605166', 'log1p_mean_counts-HCAHeartST11605166', 'pct_dropout_by_counts-HCAHeartST11605166', 'total_counts-HCAHeartST11605166', 'log1p_total_counts-HCAHeartST11605166', 'n_cells_by_counts-HCAHeartST11605167', 'mean_counts-HCAHeartST11605167', 'log1p_mean_counts-HCAHeartST11605167', 'pct_dropout_by_counts-HCAHeartST11605167', 'total_counts-HCAHeartST11605167', 'log1p_total_counts-HCAHeartST11605167', 'n_cells_by_counts-HCAHeartST11605168', 'mean_counts-HCAHeartST11605168', 'log1p_mean_counts-HCAHeartST11605168', 'pct_dropout_by_counts-HCAHeartST11605168', 'total_counts-HCAHeartST11605168', 'log1p_total_counts-HCAHeartST11605168', 'n_cells_by_counts-HCAHeartST13162329', 'mean_counts-HCAHeartST13162329', 'log1p_mean_counts-HCAHeartST13162329', 'pct_dropout_by_counts-HCAHeartST13162329', 'total_counts-HCAHeartST13162329', 'log1p_total_counts-HCAHeartST13162329', 'n_cells_by_counts-HCAHeartST13162330', 'mean_counts-HCAHeartST13162330', 'log1p_mean_counts-HCAHeartST13162330', 'pct_dropout_by_counts-HCAHeartST13162330', 'total_counts-HCAHeartST13162330', 'log1p_total_counts-HCAHeartST13162330', 'n_cells_by_counts-HCAHeartST13162331', 'mean_counts-HCAHeartST13162331', 'log1p_mean_counts-HCAHeartST13162331', 'pct_dropout_by_counts-HCAHeartST13162331', 'total_counts-HCAHeartST13162331', 'log1p_total_counts-HCAHeartST13162331', 'n_cells_by_counts-HCAHeartST13162332', 'mean_counts-HCAHeartST13162332', 'log1p_mean_counts-HCAHeartST13162332', 'pct_dropout_by_counts-HCAHeartST13162332', 'total_counts-HCAHeartST13162332', 'log1p_total_counts-HCAHeartST13162332', 'MT_gene'
    uns: 'spatial'
    obsm: 'spatial', 'MT'
```

Example 20 (python):
```python
adata_vis.obs.head()
```

Example 21 (python):
```python
# annotation
path_to_annot='/nfs/team205/heart/visium_annotations/foetal'
category_name='JC_edit'

# read in annotation csv file and concatenate
for i, ID in enumerate(sampleIDs):
    if f'{ID}_JC.csv' in os.listdir(path_to_annot):
    
        print(ID)
        annot_df = pd.read_csv(f'{path_to_annot}/{ID}_JC.csv')
        annot_df['spot_id']=ID+'_'+annot_df['Barcode']

        if i==0:
            annot_df_all=annot_df.copy()
        else:
            annot_df_all=pd.concat([annot_df_all,annot_df])
    else:
        print(f'{ID} is not annotated')

# add annotations to adata_vis
adata_vis.obs=pd.concat([adata_vis.obs,
                     annot_df_all.set_index('spot_id')[[category_name]].reindex(adata_vis.obs.index)],
                     axis=1)
# replace nan to 'none'
adata_vis.obs[category_name].fillna('none', inplace=True)

# minor modification
annotation_edit_dict = {
     'AV Ring':'AV ring',
    'Aorta':'Great vessels',
    'Coronary Artery':'Coronary artery',
    'Mitral Valve':'AV valves',
    'Mitral valve':'AV valves',
    'Myocardium - Ventricular - Trabecular':'Myocardium - Ventricular - Trabeculated',
    'Myocardim - Ventricular - Trabeculated':'Myocardium - Ventricular - Trabeculated',
    'Myocardim - Ventricular - Compact':'Myocardium - Ventricular - Compact',
    'unknown':'Unannotated',
    'Unknown':'Unannotated',
}
adata_vis.obs.replace({category_name:annotation_edit_dict}, inplace=True)
set(adata_vis.obs[category_name])
```

Example 22 (unknown):
```unknown
HCAHeartST10298114
HCAHeartST10298115
HCAHeartST10298116
HCAHeartST10298117
HCAHeartST10298120
HCAHeartST10317184
HCAHeartST10317185
HCAHeartST10317186
HCAHeartST10317189
HCAHeartST10317190
HCAHeartST11605165
HCAHeartST11605166
HCAHeartST11605167
HCAHeartST11605168
HCAHeartST13162329
HCAHeartST13162330
HCAHeartST13162331
HCAHeartST13162332
```

Example 23 (unknown):
```unknown
{'AV ring',
 'AV valves',
 'Aortic valve',
 'Cavity',
 'Connective tissue',
 'Coronary artery',
 'Epicardium',
 'Great vessels',
 'Myocardium - Atrial',
 'Myocardium - Papillary Muscle',
 'Myocardium - Ventricular - Compact',
 'Myocardium - Ventricular - Trabeculated',
 'Out_of_tissue',
 'Unannotated',
 'Valve apparatus',
 'none'}
```

Example 24 (python):
```python
# remove 'none' and 'no_tissue'
adata_vis = adata_vis[adata_vis.obs[category_name].isin(['none','Out_of_tissue'])==False]

# remove 'Cavity'
adata_vis = adata_vis[adata_vis.obs[category_name].isin(['Cavity'])==False]

# remove 'Unannotated'
adata_vis = adata_vis[adata_vis.obs[category_name].isin(['Unannotated'])==False]
```

Example 25 (python):
```python
# order annotations
annotations_order = ['Myocardium - Atrial',
                     'Myocardium - Ventricular - Compact',
                     'Myocardium - Ventricular - Trabeculated',
                    'Myocardium - Papillary Muscle',
                    'Epicardium',
                     'AV valves',
                     'Aortic valve',
                    'Valve apparatus',
                   'AV ring',
                    'Connective tissue',
                     'Great vessels',
                     'Coronary artery',
                     # 'Nerve',
                     ]
adata_vis.obs[category_name] = adata_vis.obs[category_name].astype('category')
adata_vis.obs[category_name].cat.reorder_categories(annotations_order,inplace=True)
```

Example 26 (unknown):
```unknown
Trying to set attribute `.obs` of view, copying.
/tmp/ipykernel_79813/3572501826.py:17: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
```

Example 27 (python):
```python
pd.crosstab(adata_vis.obs['cell2loc-map_group'],adata_vis.obs[category_name])
```

Example 28 (python):
```python
# check the slides for 'PCW-13w'
mask = adata_vis.obs['age']=='PCW-13w'
adata_vis[mask].obs['sample'].value_counts()
# 'HCAHeartST11605166': Atrium
# 'HCAHeartST11605165': Ventricle
```

Example 29 (unknown):
```unknown
HCAHeartST11605165    2428
HCAHeartST11605166    1120
Name: sample, dtype: int64
```

Example 30 (python):
```python
# combine
adata_vis.obs['sample_for_cell2loc']=adata_vis.obs['sample'].astype('str')
adata_vis.obs.replace({'sample_for_cell2loc':{
    'HCAHeartST11605165':'HCAHeartST11605165_HCAHeartST11605166',
    'HCAHeartST11605166':'HCAHeartST11605165_HCAHeartST11605166'
}},inplace=True)
adata_vis.obs['sample_for_cell2loc'].value_counts()
```

Example 31 (unknown):
```unknown
HCAHeartST11605165_HCAHeartST11605166    3548
HCAHeartST13162330                       2806
HCAHeartST13162331                       2338
HCAHeartST13162329                       1623
HCAHeartST10298116                       1187
HCAHeartST10298120                       1108
HCAHeartST10298117                       1085
HCAHeartST10298115                        991
HCAHeartST10317189                        944
HCAHeartST10317184                        836
HCAHeartST10298114                        812
HCAHeartST10317185                        704
HCAHeartST10317186                        693
HCAHeartST11605167                        692
HCAHeartST11605168                        639
HCAHeartST10317190                        609
HCAHeartST13162332                        413
Name: sample_for_cell2loc, dtype: int64
```

Example 32 (python):
```python
adata_vis.obs.columns
```

Example 33 (unknown):
```unknown
Index(['in_tissue', 'array_row', 'array_col', 'sample', 'n_genes_by_counts',
       'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts',
       'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes',
       'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'mt_frac',
       'n_counts', 'n_genes', 'sangerID', 'combinedID', 'donor', 'donor_type',
       'region', 'region_finest', 'age', 'gender', 'facility',
       'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'status',
       'cell2loc-map_group', 'JC_edit', 'sample_for_cell2loc'],
      dtype='object')
```

Example 34 (unknown):
```unknown
set(meta['cell2loc-map_group'])
```

Example 35 (unknown):
```unknown
{'13PCW', '20PCW', '4PCW', '5PCW', '7PCW'}
```

Example 36 (python):
```python
# convert adata_vis.var_names to symbol
adata_vis.var.set_index('SYMBOL',inplace=True)
adata_vis.var.head()
```

Example 37 (python):
```python
%%time

# read in reference signatures
inf_aver = pd.read_csv(f"{ref_run_name}/inf_aver.csv", index_col=0)

# find shared genes and subset both anndata and reference signatures
intersect = np.intersect1d(adata_vis.var_names, inf_aver.index)
adata_vis = adata_vis[:, intersect].copy()
inf_aver = inf_aver.loc[intersect, :].copy()
print(f'shape of inf_aver: {inf_aver.shape}')
print('reference cell types')
print(inf_aver.columns)

for group in set(meta['cell2loc-map_group']):
    print(f'####### {group} #######')
    
    # subset adata
    adata_vis_sub=adata_vis[adata_vis.obs['cell2loc-map_group']==group].copy()
    samples = list(set(adata_vis_sub.obs['sample_for_cell2loc']))
    samples.sort()
    print(f'samples in this run: {samples}')
    adata_vis_sub=adata_vis_sub.copy()
    # prepare anndata for cell2location model
    scvi.data.setup_anndata(adata=adata_vis_sub, batch_key="sample_for_cell2loc")
    scvi.data.view_anndata_setup(adata_vis_sub)

    # create and train the model
    mod = cell2location.models.Cell2location(
        adata_vis_sub, cell_state_df=inf_aver, 
        # the expected average cell abundance: tissue-dependent 
        # hyper-prior which can be estimated from paired histology:
        N_cells_per_location=20,
        # hyperparameter controlling normalisation of
        # within-experiment variation in RNA detection (using default here):
        detection_alpha=20
    ) 

    mod.train(max_epochs=20000, 
              # train using full data (batch_size=None)
              batch_size=None, 
              # use all data points in training because 
              # we need to estimate cell abundance at all locations
              train_size=1,
              use_gpu=True)

    # plot ELBO loss history during training, removing first 100 epochs from the plot
    mod.plot_history(1000)
    plt.legend(labels=['full data training']);

    # In this section, we export the estimated cell abundance (summary of the posterior distribution).
    adata_vis_sub = mod.export_posterior(
        adata_vis_sub, sample_kwargs={'num_samples': 1000, 'batch_size': mod.adata.n_obs, 'use_gpu': True}
    )

    # Save model
    mod.save(f"{run_name}/{group}", overwrite=True)

    # Save anndata object with results
    adata_vis_sub.write(f"{run_name}/{group}/sp.h5ad")

    # Examine reconstruction accuracy to assess if there are any issues with mapping
    # the plot should be roughly diagonal, strong deviations will signal problems
    mod.plot_QC()
    
    del mod,adata_vis_sub
    gc.collect()
    torch.cuda.empty_cache()
```

Example 38 (python):
```python
shape of inf_aver: (16314, 57)
reference cell types
Index(['AtrialCardiomyocytesLeft', 'AtrialCardiomyocytesRight',
       'VentricularCardiomyocytesLeftCompact',
       'VentricularCardiomyocytesRightCompact',
       'VentricularCardiomyocytesLeftTrabeculated',
       'VentricularCardiomyocytesRightTrabeculated',
       'SinoatrialNodeCardiomyocytes', 'AtrioventricularNodeCardiomyocytes',
       'VentricularConductionSystemProximal',
       'VentricularConductionSystemDistal',
       'GreatVesselAdventitialFibroblasts',
       'CoronaryVesselAdventitialFibroblasts',
       'MyocardialInterstitialFibroblasts1',
       'MyocardialInterstitialFibroblasts2', 'Myofibroblasts',
       'LymphNodeFibroblasticReticularCells', 'ValveInterstitialCells',
       'GreatVesselSmoothMuscleCells', 'CoronarySmoothMuscleCells',
       'CoronaryPericytes', 'GreatVesselArterialEndothelialCells',
       'GreatVesselVenousEndothelialCells', 'CoronaryArterialEndothelialCells',
       'CoronaryVenousEndothelialCells', 'CoronaryCapillaryEndothelialCells',
       'EndocardialCells', 'EndocardialCushionCells', 'ValveEndothelialCells',
       'LymphaticEndothelialCells', 'MesothelialEpicardialCells',
       'EpicardiumDerivedCells', 'NeuronPrecursors', 'ChromaffinCells',
       'SympatheticNeurons', 'ParaympatheticNeurons', 'SchwannCellPrecursors',
       'SchwannCells', 'MonocytesMPOpos', 'Monocytes', 'MonocyteDerivedCells',
       'MacrophagesCX3CR1pos', 'MacrophagesTIMD4pos', 'MacrophagesLYVE1pos',
       'MacrophagesATF3pos', 'DendriticCellsType1', 'DendriticCellsMature',
       'PlasmacytoidDendriticCells', 'MastCells', 'Megakaryocytes',
       'TCellsCD4pos', 'TCellsCD8pos', 'TregsCD4pos', 'ProBCells', 'BCells',
       'BCellsMS4A1pos', 'NaturalKillerCells', 'InnateLymphoidCells'],
      dtype='object')
####### 20PCW #######
samples in this run: ['HCAHeartST13162329', 'HCAHeartST13162330', 'HCAHeartST13162331']
INFO     Using batches from adata.obs["sample_for_cell2loc"]                                 
INFO     No label_key inputted, assuming all cells have same label                           
INFO     Using data from adata.X
```

Example 39 (unknown):
```unknown
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function setup_anndata is deprecated; Please use the model-specific setup_anndata methods instead. The global method will be removed in version 0.15.0.
```

Example 40 (python):
```python
INFO     Successfully registered anndata object containing 6767 cells, 16314 vars, 3 batches,
         1 labels, and 0 proteins. Also registered 0 extra categorical covariates and 0 extra
         continuous covariates.                                                              
INFO     Please do not further modify adata until model is trained.
```

Example 41 (unknown):
```unknown
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function view_anndata_setup is deprecated; This method will be removed in 0.15.0. Please avoid building any new dependencies on it.
```

Example 42 (unknown):
```unknown
Anndata setup with scvi-tools version 0.14.4.
```

Example 43 (unknown):
```unknown
Data Summary              
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┓
┃             Data             ┃ Count ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━┩
│            Cells             │ 6767  │
│             Vars             │ 16314 │
│            Labels            │   1   │
│           Batches            │   3   │
│           Proteins           │   0   │
│ Extra Categorical Covariates │   0   │
│ Extra Continuous Covariates  │   0   │
└──────────────────────────────┴───────┘
```

Example 44 (python):
```python
SCVI Data Registry              
┏━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃     Data      ┃    scvi-tools Location    ┃
┡━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│       X       │          adata.X          │
│ batch_indices │ adata.obs['_scvi_batch']  │
│    labels     │ adata.obs['_scvi_labels'] │
└───────────────┴───────────────────────────┘
```

Example 45 (python):
```python
Label Categories                        
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 46 (python):
```python
Batch Categories                                
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃         Source Location          ┃     Categories     ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['sample_for_cell2loc'] │ HCAHeartST13162329 │          0          │
│                                  │ HCAHeartST13162330 │          1          │
│                                  │ HCAHeartST13162331 │          2          │
└──────────────────────────────────┴────────────────────┴─────────────────────┘
```

Example 47 (unknown):
```unknown
GPU available: True, used: True
TPU available: False, using: 0 TPU cores
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/pytorch_lightning/trainer/configuration_validator.py:99: UserWarning: you passed in a val_dataloader but have no validation_step. Skipping val loop
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
```

Example 48 (unknown):
```unknown
Epoch 20000/20000: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20000/20000 [50:20<00:00,  6.62it/s, v_num=1, elbo_train=5.82e+7]
Sampling local variables, batch: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:44<00:00, 44.80s/it]
Sampling global variables, sample: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 999/999 [00:36<00:00, 27.29it/s]
```

Example 49 (unknown):
```unknown
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'sample' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'sangerID' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'combinedID' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'donor' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'donor_type' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'region' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'region_finest' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'age' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'gender' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'facility' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'cell_or_nuclei' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'modality' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'kit_10x' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'flushed' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'status' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'cell2loc-map_group' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'sample_for_cell2loc' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'feature_types' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'genome' as categorical
```

Example 50 (python):
```python
####### 5PCW #######
samples in this run: ['HCAHeartST11605167', 'HCAHeartST11605168']
INFO     Using batches from adata.obs["sample_for_cell2loc"]                                 
INFO     No label_key inputted, assuming all cells have same label                           
INFO     Using data from adata.X
```

Example 51 (unknown):
```unknown
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function setup_anndata is deprecated; Please use the model-specific setup_anndata methods instead. The global method will be removed in version 0.15.0.
```

Example 52 (python):
```python
INFO     Successfully registered anndata object containing 1331 cells, 16314 vars, 2 batches,
         1 labels, and 0 proteins. Also registered 0 extra categorical covariates and 0 extra
         continuous covariates.                                                              
INFO     Please do not further modify adata until model is trained.
```

Example 53 (unknown):
```unknown
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function view_anndata_setup is deprecated; This method will be removed in 0.15.0. Please avoid building any new dependencies on it.
```

Example 54 (unknown):
```unknown
Anndata setup with scvi-tools version 0.14.4.
```

Example 55 (unknown):
```unknown
Data Summary              
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┓
┃             Data             ┃ Count ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━┩
│            Cells             │ 1331  │
│             Vars             │ 16314 │
│            Labels            │   1   │
│           Batches            │   2   │
│           Proteins           │   0   │
│ Extra Categorical Covariates │   0   │
│ Extra Continuous Covariates  │   0   │
└──────────────────────────────┴───────┘
```

Example 56 (python):
```python
SCVI Data Registry              
┏━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃     Data      ┃    scvi-tools Location    ┃
┡━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│       X       │          adata.X          │
│ batch_indices │ adata.obs['_scvi_batch']  │
│    labels     │ adata.obs['_scvi_labels'] │
└───────────────┴───────────────────────────┘
```

Example 57 (python):
```python
Label Categories                        
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 58 (python):
```python
Batch Categories                                
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃         Source Location          ┃     Categories     ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['sample_for_cell2loc'] │ HCAHeartST11605167 │          0          │
│                                  │ HCAHeartST11605168 │          1          │
└──────────────────────────────────┴────────────────────┴─────────────────────┘
```

Example 59 (unknown):
```unknown
GPU available: True, used: True
TPU available: False, using: 0 TPU cores
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/pytorch_lightning/trainer/configuration_validator.py:99: UserWarning: you passed in a val_dataloader but have no validation_step. Skipping val loop
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
```

Example 60 (unknown):
```unknown
Epoch 20000/20000: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20000/20000 [32:32<00:00, 10.24it/s, v_num=1, elbo_train=1.7e+7]
Sampling local variables, batch: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:23<00:00, 23.56s/it]
Sampling global variables, sample: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 999/999 [00:22<00:00, 43.63it/s]
```

Example 61 (unknown):
```unknown
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'sample' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'sangerID' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'combinedID' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'donor' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'donor_type' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'region' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'region_finest' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'age' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'gender' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'facility' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'cell_or_nuclei' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'modality' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'kit_10x' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'flushed' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'status' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'cell2loc-map_group' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'sample_for_cell2loc' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'feature_types' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'genome' as categorical
```

Example 62 (python):
```python
####### 7PCW #######
samples in this run: ['HCAHeartST10298114', 'HCAHeartST10298115', 'HCAHeartST10298116', 'HCAHeartST10298117', 'HCAHeartST10298120', 'HCAHeartST10317184', 'HCAHeartST10317185', 'HCAHeartST10317186', 'HCAHeartST10317189', 'HCAHeartST10317190']
INFO     Using batches from adata.obs["sample_for_cell2loc"]                                 
INFO     No label_key inputted, assuming all cells have same label                           
INFO     Using data from adata.X
```

Example 63 (unknown):
```unknown
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function setup_anndata is deprecated; Please use the model-specific setup_anndata methods instead. The global method will be removed in version 0.15.0.
```

Example 64 (python):
```python
INFO     Successfully registered anndata object containing 8969 cells, 16314 vars, 10        
         batches, 1 labels, and 0 proteins. Also registered 0 extra categorical covariates   
         and 0 extra continuous covariates.                                                  
INFO     Please do not further modify adata until model is trained.
```

Example 65 (unknown):
```unknown
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function view_anndata_setup is deprecated; This method will be removed in 0.15.0. Please avoid building any new dependencies on it.
```

Example 66 (unknown):
```unknown
Anndata setup with scvi-tools version 0.14.4.
```

Example 67 (unknown):
```unknown
Data Summary              
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┓
┃             Data             ┃ Count ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━┩
│            Cells             │ 8969  │
│             Vars             │ 16314 │
│            Labels            │   1   │
│           Batches            │  10   │
│           Proteins           │   0   │
│ Extra Categorical Covariates │   0   │
│ Extra Continuous Covariates  │   0   │
└──────────────────────────────┴───────┘
```

Example 68 (python):
```python
SCVI Data Registry              
┏━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃     Data      ┃    scvi-tools Location    ┃
┡━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│       X       │          adata.X          │
│ batch_indices │ adata.obs['_scvi_batch']  │
│    labels     │ adata.obs['_scvi_labels'] │
└───────────────┴───────────────────────────┘
```

Example 69 (python):
```python
Label Categories                        
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 70 (python):
```python
Batch Categories                                
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃         Source Location          ┃     Categories     ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['sample_for_cell2loc'] │ HCAHeartST10298114 │          0          │
│                                  │ HCAHeartST10298115 │          1          │
│                                  │ HCAHeartST10298116 │          2          │
│                                  │ HCAHeartST10298117 │          3          │
│                                  │ HCAHeartST10298120 │          4          │
│                                  │ HCAHeartST10317184 │          5          │
│                                  │ HCAHeartST10317185 │          6          │
│                                  │ HCAHeartST10317186 │          7          │
│                                  │ HCAHeartST10317189 │          8          │
│                                  │ HCAHeartST10317190 │          9          │
└──────────────────────────────────┴────────────────────┴─────────────────────┘
```

Example 71 (unknown):
```unknown
GPU available: True, used: True
TPU available: False, using: 0 TPU cores
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/pytorch_lightning/trainer/configuration_validator.py:99: UserWarning: you passed in a val_dataloader but have no validation_step. Skipping val loop
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
```

Example 72 (unknown):
```unknown
Epoch 20000/20000: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20000/20000 [1:01:37<00:00,  5.41it/s, v_num=1, elbo_train=7.64e+7]
Sampling local variables, batch: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:46<00:00, 46.24s/it]
Sampling global variables, sample: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 999/999 [00:42<00:00, 23.48it/s]
```

Example 73 (unknown):
```unknown
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'sample' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'sangerID' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'combinedID' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'donor' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'donor_type' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'region' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'region_finest' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'age' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'gender' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'facility' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'cell_or_nuclei' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'modality' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'kit_10x' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'flushed' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'status' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'cell2loc-map_group' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'sample_for_cell2loc' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'feature_types' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'genome' as categorical
```

Example 74 (python):
```python
####### 13PCW #######
samples in this run: ['HCAHeartST11605165_HCAHeartST11605166']
INFO     Using batches from adata.obs["sample_for_cell2loc"]                                 
INFO     No label_key inputted, assuming all cells have same label                           
INFO     Using data from adata.X
```

Example 75 (unknown):
```unknown
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function setup_anndata is deprecated; Please use the model-specific setup_anndata methods instead. The global method will be removed in version 0.15.0.
```

Example 76 (python):
```python
INFO     Successfully registered anndata object containing 3548 cells, 16314 vars, 1 batches,
         1 labels, and 0 proteins. Also registered 0 extra categorical covariates and 0 extra
         continuous covariates.                                                              
INFO     Please do not further modify adata until model is trained.
```

Example 77 (unknown):
```unknown
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function view_anndata_setup is deprecated; This method will be removed in 0.15.0. Please avoid building any new dependencies on it.
```

Example 78 (unknown):
```unknown
Anndata setup with scvi-tools version 0.14.4.
```

Example 79 (unknown):
```unknown
Data Summary              
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┓
┃             Data             ┃ Count ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━┩
│            Cells             │ 3548  │
│             Vars             │ 16314 │
│            Labels            │   1   │
│           Batches            │   1   │
│           Proteins           │   0   │
│ Extra Categorical Covariates │   0   │
│ Extra Continuous Covariates  │   0   │
└──────────────────────────────┴───────┘
```

Example 80 (python):
```python
SCVI Data Registry              
┏━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃     Data      ┃    scvi-tools Location    ┃
┡━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│       X       │          adata.X          │
│ batch_indices │ adata.obs['_scvi_batch']  │
│    labels     │ adata.obs['_scvi_labels'] │
└───────────────┴───────────────────────────┘
```

Example 81 (python):
```python
Label Categories                        
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 82 (python):
```python
Batch Categories                                       
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┓
┃        Source Location         ┃             Categories              ┃ scvi-tools Encodin ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['sample_for_cell2loc │ HCAHeartST11605165_HCAHeartST116051 │         0          │
└────────────────────────────────┴─────────────────────────────────────┴────────────────────┘
```

Example 83 (unknown):
```unknown
GPU available: True, used: True
TPU available: False, using: 0 TPU cores
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/pytorch_lightning/trainer/configuration_validator.py:99: UserWarning: you passed in a val_dataloader but have no validation_step. Skipping val loop
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
```

Example 84 (unknown):
```unknown
Epoch 20000/20000: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20000/20000 [38:29<00:00,  8.66it/s, v_num=1, elbo_train=3.63e+7]
Sampling local variables, batch: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:29<00:00, 29.27s/it]
Sampling global variables, sample: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 999/999 [00:28<00:00, 35.40it/s]
```

Example 85 (unknown):
```unknown
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'sample' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'sangerID' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'combinedID' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'donor' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'donor_type' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'region' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'region_finest' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'age' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'gender' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'facility' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'cell_or_nuclei' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'modality' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'kit_10x' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'flushed' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'status' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'cell2loc-map_group' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'sample_for_cell2loc' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'feature_types' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'genome' as categorical
```

Example 86 (python):
```python
####### 4PCW #######
samples in this run: ['HCAHeartST13162332']
INFO     Using batches from adata.obs["sample_for_cell2loc"]                                 
INFO     No label_key inputted, assuming all cells have same label                           
INFO     Using data from adata.X                                                             
INFO     Successfully registered anndata object containing 413 cells, 16314 vars, 1 batches, 
         1 labels, and 0 proteins. Also registered 0 extra categorical covariates and 0 extra
         continuous covariates.                                                              
INFO     Please do not further modify adata until model is trained.
```

Example 87 (unknown):
```unknown
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function setup_anndata is deprecated; Please use the model-specific setup_anndata methods instead. The global method will be removed in version 0.15.0.
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function view_anndata_setup is deprecated; This method will be removed in 0.15.0. Please avoid building any new dependencies on it.
```

Example 88 (unknown):
```unknown
Anndata setup with scvi-tools version 0.14.4.
```

Example 89 (unknown):
```unknown
Data Summary              
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┓
┃             Data             ┃ Count ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━┩
│            Cells             │  413  │
│             Vars             │ 16314 │
│            Labels            │   1   │
│           Batches            │   1   │
│           Proteins           │   0   │
│ Extra Categorical Covariates │   0   │
│ Extra Continuous Covariates  │   0   │
└──────────────────────────────┴───────┘
```

Example 90 (python):
```python
SCVI Data Registry              
┏━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃     Data      ┃    scvi-tools Location    ┃
┡━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│       X       │          adata.X          │
│ batch_indices │ adata.obs['_scvi_batch']  │
│    labels     │ adata.obs['_scvi_labels'] │
└───────────────┴───────────────────────────┘
```

Example 91 (python):
```python
Label Categories                        
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 92 (python):
```python
Batch Categories                                
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃         Source Location          ┃     Categories     ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['sample_for_cell2loc'] │ HCAHeartST13162332 │          0          │
└──────────────────────────────────┴────────────────────┴─────────────────────┘
```

Example 93 (unknown):
```unknown
GPU available: True, used: True
TPU available: False, using: 0 TPU cores
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/pytorch_lightning/trainer/configuration_validator.py:99: UserWarning: you passed in a val_dataloader but have no validation_step. Skipping val loop
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
```

Example 94 (unknown):
```unknown
Epoch 20000/20000: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20000/20000 [30:21<00:00, 10.98it/s, v_num=1, elbo_train=5.22e+6]
Sampling local variables, batch: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:19<00:00, 19.73s/it]
Sampling global variables, sample: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 999/999 [00:20<00:00, 49.56it/s]
```

Example 95 (unknown):
```unknown
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'sample' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'sangerID' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'combinedID' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'donor' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'donor_type' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'region' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'region_finest' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'age' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'gender' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'facility' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'cell_or_nuclei' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'modality' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'kit_10x' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'flushed' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'status' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'cell2loc-map_group' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'sample_for_cell2loc' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'feature_types' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'genome' as categorical
```

Example 96 (unknown):
```unknown
CPU times: user 3h 14min 40s, sys: 28min 21s, total: 3h 43min 2s
Wall time: 3h 43min 2s
```

---

## Variables¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/VisiumHD/no6_clustering-umap_per-mid.html

**Contents:**
- Variables¶
- Read in data¶
- Save UMAP and leiden clustering results¶

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
-----
matplotlib          3.9.2
numpy               2.0.2
pandas              2.2.3
scanpy              1.10.3
seaborn             0.13.2
session_info        1.0.0
-----
```

Example 4 (python):
```python
PIL                 11.0.0
anndata             0.10.9
asttokens           NA
charset_normalizer  3.4.0
comm                0.2.2
cycler              0.12.1
cython_runtime      NA
dateutil            2.9.0.post0
debugpy             1.8.9
decorator           5.1.1
exceptiongroup      1.2.2
executing           2.1.0
google              NA
h5py                3.12.1
igraph              0.10.2
importlib_metadata  NA
importlib_resources NA
ipykernel           6.29.5
jedi                0.19.2
joblib              1.4.2
kiwisolver          1.4.7
legacy_api_wrap     NA
leidenalg           0.9.0
llvmlite            0.43.0
mpl_toolkits        NA
natsort             8.4.0
numba               0.60.0
packaging           24.2
parso               0.8.4
patsy               1.0.1
pexpect             4.9.0
platformdirs        4.3.6
prompt_toolkit      3.0.48
psutil              6.1.0
ptyprocess          0.7.0
pure_eval           0.2.3
pydev_ipython       NA
pydevconsole        NA
pydevd              3.2.3
pydevd_file_utils   NA
pydevd_plugins      NA
pydevd_tracing      NA
pygments            2.18.0
pyparsing           3.2.0
pytz                2024.2
scipy               1.13.1
six                 1.16.0
sklearn             1.5.2
stack_data          0.6.3
statsmodels         0.14.4
texttable           1.6.7
threadpoolctl       3.5.0
tornado             6.4.2
traitlets           5.14.3
typing_extensions   NA
wcwidth             0.2.13
yaml                6.0.2
zipp                NA
zmq                 26.2.0
zoneinfo            NA
```

Example 5 (unknown):
```unknown
-----
IPython             8.18.1
jupyter_client      8.6.3
jupyter_core        5.7.2
-----
Python 3.9.20 | packaged by conda-forge | (main, Sep 30 2024, 17:49:10) [GCC 13.3.0]
Linux-5.15.0-126-generic-x86_64-with-glibc2.35
-----
Session information updated at 2024-12-05 23:49
```

Example 6 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 7 (python):
```python
adata_dir = '/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects'
sample_id = 'HEA_FOET14880396'
```

Example 8 (unknown):
```unknown
os.getcwd()
```

Example 9 (unknown):
```unknown
'/nfs/team205/kk18/notebooks/Foetal/VisiumHD/re-seq_mpp0p3'
```

Example 10 (python):
```python
adata= sc.read_h5ad(f'{adata_dir}/{sample_id}_b2c_cells_filtered_lognorm.h5ad')
adata
```

Example 11 (unknown):
```unknown
AnnData object with n_obs × n_vars = 129803 × 18047
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'hvg', 'log1p', 'neighbors', 'pca', 'spatial', 'umap'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 12 (python):
```python
adata.X.data[:5]
```

Example 13 (unknown):
```unknown
array([1.27187559, 1.95641568, 1.75023687, 1.96760567, 2.7810601 ])
```

Example 14 (python):
```python
adata.obs['celltypist_mid_fin'].unique()
```

Example 15 (unknown):
```unknown
['LymphoidCells', 'MyeloidCells', 'unassigned', 'Fibroblasts', 'PericardialCells', ..., 'Glia', 'EpicardialCells', 'AtrialCardiomyocytes', 'EndocardialCells', 'VentricularCardiomyocytes']
Length: 14
Categories (14, object): ['AtrialCardiomyocytes', 'BloodVesselEndothelialCells', 'EndocardialCells', 'EpicardialCells', ..., 'Neurons', 'PericardialCells', 'VentricularCardiomyocytes', 'unassigned']
```

Example 16 (python):
```python
sc.pl.embedding(adata,
                    basis=f"X_umap",
                    color='celltypist_mid_fin',
                groups='unassigned',
                   ncols=4
                   )
```

Example 17 (python):
```python
adata_dir = '/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/subsets'
```

Example 18 (python):
```python
adata.uns['log1p']['base'] = None
```

Example 19 (python):
```python
%%time
for celltype in adata.obs['celltypist_mid_fin'].unique():
    if celltype == 'unassigned':
        continue
    
    print(f'##### {celltype} #####')
    # subset
    adata_sub = adata[adata.obs['celltypist_mid_fin']==celltype]
    sc.pp.filter_genes(adata_sub, min_cells=3)
    print(adata_sub.shape)
    # HVGs
    sc.pp.highly_variable_genes(adata_sub, flavor='seurat', min_mean=0.0125, max_mean=3, min_disp=0.5)
    print(f'number of hvgs: {sum(adata_sub.var.highly_variable)}')
    # PCA
    sc.tl.pca(adata_sub, use_highly_variable=True)
    # create neighbour graph
    sc.pp.neighbors(adata_sub,n_neighbors=15,use_rep='X_pca')
    # visualisation with umap
    sc.tl.umap(adata_sub,min_dist=0.5)
    sc.pl.embedding(adata_sub,basis='X_umap',color=['bin_count','total_counts','n_genes_by_counts',"conf_score_midmod2fine","celltypist_midmod2fine",],
                    wspace=0.2,cmap='RdPu',vmax='p100',ncols=5)
    # leiden clustering
    resolutions_list = [0.2,0.4,0.6,1,1.2,1.5,2]
    for resolution in resolutions_list:
        sc.tl.leiden(adata_sub,resolution=resolution,key_added=f'leiden_{str(resolution)}',n_iterations=2)
    sc.pl.embedding(adata_sub,basis='X_umap',
                    color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                    wspace=0.3)
    # save
    adata_sub.write(f'{adata_dir}/{sample_id}_{celltype}_b2c_cells_filtered_lognorm.h5ad')
    print('')
```

Example 20 (unknown):
```unknown
##### LymphoidCells #####
```

Example 21 (python):
```python
/software/cellgen/team205/kk18/envs/generic_env/lib/python3.9/site-packages/scanpy/preprocessing/_simple.py:283: ImplicitModificationWarning: Trying to modify attribute `.var` of view, initializing view as actual.
  adata.var["n_cells"] = number
```

Example 22 (unknown):
```unknown
(87, 10288)
number of hvgs: 2647
```

Example 23 (unknown):
```unknown
/software/cellgen/team205/kk18/envs/generic_env/lib/python3.9/site-packages/scanpy/preprocessing/_pca.py:377: FutureWarning: Argument `use_highly_variable` is deprecated, consider using the mask argument. Use_highly_variable=True can be called through mask_var="highly_variable". Use_highly_variable=False can be called through mask_var=None
  warn(msg, FutureWarning)
```

Example 24 (unknown):
```unknown
<timed exec>:24: FutureWarning: In the future, the default backend for leiden will be igraph instead of leidenalg.

 To achieve the future defaults please pass: flavor="igraph" and n_iterations=2.  directed must also be False to work with igraph's implementation.
```

Example 25 (unknown):
```unknown
##### MyeloidCells #####
```

Example 26 (python):
```python
/software/cellgen/team205/kk18/envs/generic_env/lib/python3.9/site-packages/scanpy/preprocessing/_simple.py:283: ImplicitModificationWarning: Trying to modify attribute `.var` of view, initializing view as actual.
  adata.var["n_cells"] = number
```

Example 27 (unknown):
```unknown
(526, 13502)
number of hvgs: 3182
```

Example 28 (unknown):
```unknown
/software/cellgen/team205/kk18/envs/generic_env/lib/python3.9/site-packages/scanpy/preprocessing/_pca.py:377: FutureWarning: Argument `use_highly_variable` is deprecated, consider using the mask argument. Use_highly_variable=True can be called through mask_var="highly_variable". Use_highly_variable=False can be called through mask_var=None
  warn(msg, FutureWarning)
```

Example 29 (unknown):
```unknown
##### Fibroblasts #####
```

Example 30 (python):
```python
/software/cellgen/team205/kk18/envs/generic_env/lib/python3.9/site-packages/scanpy/preprocessing/_simple.py:283: ImplicitModificationWarning: Trying to modify attribute `.var` of view, initializing view as actual.
  adata.var["n_cells"] = number
```

Example 31 (unknown):
```unknown
(20318, 17926)
number of hvgs: 3448
```

Example 32 (unknown):
```unknown
/software/cellgen/team205/kk18/envs/generic_env/lib/python3.9/site-packages/scanpy/preprocessing/_pca.py:377: FutureWarning: Argument `use_highly_variable` is deprecated, consider using the mask argument. Use_highly_variable=True can be called through mask_var="highly_variable". Use_highly_variable=False can be called through mask_var=None
  warn(msg, FutureWarning)
```

Example 33 (unknown):
```unknown
##### PericardialCells #####
```

Example 34 (python):
```python
/software/cellgen/team205/kk18/envs/generic_env/lib/python3.9/site-packages/scanpy/preprocessing/_simple.py:283: ImplicitModificationWarning: Trying to modify attribute `.var` of view, initializing view as actual.
  adata.var["n_cells"] = number
```

Example 35 (unknown):
```unknown
(3061, 15212)
number of hvgs: 3858
```

Example 36 (unknown):
```unknown
/software/cellgen/team205/kk18/envs/generic_env/lib/python3.9/site-packages/scanpy/preprocessing/_pca.py:377: FutureWarning: Argument `use_highly_variable` is deprecated, consider using the mask argument. Use_highly_variable=True can be called through mask_var="highly_variable". Use_highly_variable=False can be called through mask_var=None
  warn(msg, FutureWarning)
```

Example 37 (unknown):
```unknown
##### LymphaticEndothelialCells #####
```

Example 38 (python):
```python
/software/cellgen/team205/kk18/envs/generic_env/lib/python3.9/site-packages/scanpy/preprocessing/_simple.py:283: ImplicitModificationWarning: Trying to modify attribute `.var` of view, initializing view as actual.
  adata.var["n_cells"] = number
```

Example 39 (unknown):
```unknown
(1025, 14030)
number of hvgs: 3656
```

Example 40 (unknown):
```unknown
/software/cellgen/team205/kk18/envs/generic_env/lib/python3.9/site-packages/scanpy/preprocessing/_pca.py:377: FutureWarning: Argument `use_highly_variable` is deprecated, consider using the mask argument. Use_highly_variable=True can be called through mask_var="highly_variable". Use_highly_variable=False can be called through mask_var=None
  warn(msg, FutureWarning)
/software/cellgen/team205/kk18/envs/generic_env/lib/python3.9/site-packages/numpy/lib/_nanfunctions_impl.py:1424: RuntimeWarning: All-NaN slice encountered
  return _nanquantile_unchecked(
```

Example 41 (unknown):
```unknown
##### BloodVesselEndothelialCells #####
```

Example 42 (python):
```python
/software/cellgen/team205/kk18/envs/generic_env/lib/python3.9/site-packages/scanpy/preprocessing/_simple.py:283: ImplicitModificationWarning: Trying to modify attribute `.var` of view, initializing view as actual.
  adata.var["n_cells"] = number
```

Example 43 (unknown):
```unknown
(1765, 15377)
number of hvgs: 3724
```

Example 44 (unknown):
```unknown
/software/cellgen/team205/kk18/envs/generic_env/lib/python3.9/site-packages/scanpy/preprocessing/_pca.py:377: FutureWarning: Argument `use_highly_variable` is deprecated, consider using the mask argument. Use_highly_variable=True can be called through mask_var="highly_variable". Use_highly_variable=False can be called through mask_var=None
  warn(msg, FutureWarning)
```

Example 45 (unknown):
```unknown
##### MuralCells #####
```

Example 46 (python):
```python
/software/cellgen/team205/kk18/envs/generic_env/lib/python3.9/site-packages/scanpy/preprocessing/_simple.py:283: ImplicitModificationWarning: Trying to modify attribute `.var` of view, initializing view as actual.
  adata.var["n_cells"] = number
```

Example 47 (unknown):
```unknown
(10785, 17693)
number of hvgs: 3102
```

Example 48 (unknown):
```unknown
/software/cellgen/team205/kk18/envs/generic_env/lib/python3.9/site-packages/scanpy/preprocessing/_pca.py:377: FutureWarning: Argument `use_highly_variable` is deprecated, consider using the mask argument. Use_highly_variable=True can be called through mask_var="highly_variable". Use_highly_variable=False can be called through mask_var=None
  warn(msg, FutureWarning)
```

Example 49 (unknown):
```unknown
##### Neurons #####
```

Example 50 (python):
```python
/software/cellgen/team205/kk18/envs/generic_env/lib/python3.9/site-packages/scanpy/preprocessing/_simple.py:283: ImplicitModificationWarning: Trying to modify attribute `.var` of view, initializing view as actual.
  adata.var["n_cells"] = number
```

Example 51 (unknown):
```unknown
(589, 14494)
number of hvgs: 3715
```

Example 52 (unknown):
```unknown
/software/cellgen/team205/kk18/envs/generic_env/lib/python3.9/site-packages/scanpy/preprocessing/_pca.py:377: FutureWarning: Argument `use_highly_variable` is deprecated, consider using the mask argument. Use_highly_variable=True can be called through mask_var="highly_variable". Use_highly_variable=False can be called through mask_var=None
  warn(msg, FutureWarning)
```

Example 53 (unknown):
```unknown
##### Glia #####
```

Example 54 (python):
```python
/software/cellgen/team205/kk18/envs/generic_env/lib/python3.9/site-packages/scanpy/preprocessing/_simple.py:283: ImplicitModificationWarning: Trying to modify attribute `.var` of view, initializing view as actual.
  adata.var["n_cells"] = number
```

Example 55 (unknown):
```unknown
(588, 13981)
number of hvgs: 3625
```

Example 56 (unknown):
```unknown
/software/cellgen/team205/kk18/envs/generic_env/lib/python3.9/site-packages/scanpy/preprocessing/_pca.py:377: FutureWarning: Argument `use_highly_variable` is deprecated, consider using the mask argument. Use_highly_variable=True can be called through mask_var="highly_variable". Use_highly_variable=False can be called through mask_var=None
  warn(msg, FutureWarning)
```

Example 57 (unknown):
```unknown
##### EpicardialCells #####
```

Example 58 (python):
```python
/software/cellgen/team205/kk18/envs/generic_env/lib/python3.9/site-packages/scanpy/preprocessing/_simple.py:283: ImplicitModificationWarning: Trying to modify attribute `.var` of view, initializing view as actual.
  adata.var["n_cells"] = number
```

Example 59 (unknown):
```unknown
(603, 13050)
number of hvgs: 3578
```

Example 60 (unknown):
```unknown
/software/cellgen/team205/kk18/envs/generic_env/lib/python3.9/site-packages/scanpy/preprocessing/_pca.py:377: FutureWarning: Argument `use_highly_variable` is deprecated, consider using the mask argument. Use_highly_variable=True can be called through mask_var="highly_variable". Use_highly_variable=False can be called through mask_var=None
  warn(msg, FutureWarning)
```

Example 61 (unknown):
```unknown
##### AtrialCardiomyocytes #####
```

Example 62 (python):
```python
/software/cellgen/team205/kk18/envs/generic_env/lib/python3.9/site-packages/scanpy/preprocessing/_simple.py:283: ImplicitModificationWarning: Trying to modify attribute `.var` of view, initializing view as actual.
  adata.var["n_cells"] = number
```

Example 63 (unknown):
```unknown
(13904, 17650)
number of hvgs: 3349
```

Example 64 (unknown):
```unknown
/software/cellgen/team205/kk18/envs/generic_env/lib/python3.9/site-packages/scanpy/preprocessing/_pca.py:377: FutureWarning: Argument `use_highly_variable` is deprecated, consider using the mask argument. Use_highly_variable=True can be called through mask_var="highly_variable". Use_highly_variable=False can be called through mask_var=None
  warn(msg, FutureWarning)
```

Example 65 (unknown):
```unknown
##### EndocardialCells #####
```

Example 66 (python):
```python
/software/cellgen/team205/kk18/envs/generic_env/lib/python3.9/site-packages/scanpy/preprocessing/_simple.py:283: ImplicitModificationWarning: Trying to modify attribute `.var` of view, initializing view as actual.
  adata.var["n_cells"] = number
```

Example 67 (unknown):
```unknown
(1526, 14451)
number of hvgs: 3615
```

Example 68 (unknown):
```unknown
/software/cellgen/team205/kk18/envs/generic_env/lib/python3.9/site-packages/scanpy/preprocessing/_pca.py:377: FutureWarning: Argument `use_highly_variable` is deprecated, consider using the mask argument. Use_highly_variable=True can be called through mask_var="highly_variable". Use_highly_variable=False can be called through mask_var=None
  warn(msg, FutureWarning)
```

Example 69 (unknown):
```unknown
##### VentricularCardiomyocytes #####
```

Example 70 (python):
```python
/software/cellgen/team205/kk18/envs/generic_env/lib/python3.9/site-packages/scanpy/preprocessing/_simple.py:283: ImplicitModificationWarning: Trying to modify attribute `.var` of view, initializing view as actual.
  adata.var["n_cells"] = number
```

Example 71 (unknown):
```unknown
(61075, 17994)
number of hvgs: 2287
```

Example 72 (unknown):
```unknown
/software/cellgen/team205/kk18/envs/generic_env/lib/python3.9/site-packages/scanpy/preprocessing/_pca.py:377: FutureWarning: Argument `use_highly_variable` is deprecated, consider using the mask argument. Use_highly_variable=True can be called through mask_var="highly_variable". Use_highly_variable=False can be called through mask_var=None
  warn(msg, FutureWarning)
```

Example 73 (unknown):
```unknown
CPU times: user 4min 18s, sys: 49.1 s, total: 5min 7s
Wall time: 6min 48s
```

---

## Functions¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig4_Cardiomyocytes/AtrialCardiomyocytes/Visium_mapping/c2l_pt3_plot_FFPE_immature.html

**Contents:**
- Functions¶
- Read in metadata¶
- Data, parameters¶

2045 rows × 6 columns

**Examples:**

Example 1 (unknown):
```unknown
run='20Mar2024_fetal_FFPE'
```

Example 2 (python):
```python
import sys
import scanpy as sc
import anndata
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt 
import matplotlib as mpl

import cell2location
import scvi

from matplotlib import rcParams
rcParams['pdf.fonttype'] = 42 # enables correct plotting of text
import seaborn as sns
```

Example 3 (python):
```python
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm
Global seed set to 0
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/umap/distances.py:1063: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/umap/distances.py:1071: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/umap/distances.py:1086: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/umap/umap_.py:660: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
```

Example 4 (python):
```python
import gc
import torch
```

Example 5 (python):
```python
def read_and_qc(sample_name, path):
    r""" This function reads the data for one 10X spatial experiment into the anndata object.
    It also calculates QC metrics. Modify this function if required by your workflow.

    :param sample_name: Name of the sample
    :param path: path to data
    """

    adata = sc.read_visium(path + str(sample_name),
                           count_file='filtered_feature_bc_matrix.h5', load_images=True)
    adata.var_names_make_unique(join='-')
    
    adata.obs['sample'] = sample_name
    
    # rename genes to ENSEMBL
    adata.var['SYMBOL'] = adata.var_names
    adata.var_names = adata.var['gene_ids']
    adata.var_names.name = None

    # Calculate QC metrics
    sc.pp.calculate_qc_metrics(adata, inplace=True)
    adata.var['mt'] = [gene.startswith('MT-') for gene in adata.var['SYMBOL']]
    adata.obs['mt_frac'] = adata[:, adata.var['mt'].tolist()].X.sum(1).A.squeeze()/adata.obs['total_counts']

    # add sample name to obs names
    adata.obs["sample"] = [str(i) for i in adata.obs['sample']]
    adata.obs_names = adata.obs["sample"] \
                          + '_' + adata.obs_names
    adata.obs.index.name = 'spot_id'

    return adata

def select_slide(adata, s, s_col='sample'):
    r""" This function selects the data for one slide from the spatial anndata object.

    :param adata: Anndata object with multiple spatial experiments
    :param s: name of selected experiment
    :param s_col: column in adata.obs listing experiment name for each location
    """

    slide = adata[adata.obs[s_col].isin([s]), :]
    s_keys = list(slide.uns['spatial'].keys())
    s_spatial = np.array(s_keys)[[s in k for k in s_keys]][0]

    slide.uns['spatial'] = {s_spatial: slide.uns['spatial'][s_spatial]}

    return slide
```

Example 6 (python):
```python
meta=pd.read_csv('/nfs/team205/heart/anndata_objects/Foetal/metadata/HeartTeamSamples_Mappeddata_20230208_for-cell2loc.csv',
                index_col=0)
meta
```

Example 7 (python):
```python
pd.crosstab(meta['donor'],meta['cell2loc-map_group'])
```

Example 8 (unknown):
```unknown
# select slides
# Healthy FFPE
meta = meta[meta['cell2loc-map_group'].isin(['FFPE_16PCW'])]

sampleIDs = list(meta['sangerID'])
len(sampleIDs)
```

Example 9 (python):
```python
import os
results_folder = os.path.join(os.getcwd(),"analysis/Visium/cell2location_results")

ref_run_name = f'{results_folder}/reference_signatures/{run}'
run_name = f'{results_folder}/cell2location_map/{run}'
```

Example 10 (unknown):
```unknown
visium_path='/nfs/team205/heart/mapped/spaceranger200/'

celltype_key='fine_grain_modified'

vis_subset_key='cell2loc-map_group'
```

Example 11 (python):
```python
%%time
subset_values = os.listdir(os.path.join(results_folder, "cell2location_map", run))
print(f"Found these subsets {subset_values}")

for value in subset_values:
    print(f"###### {value} ######")
    adata_sub = sc.read(f"{run_name}/{value}/sp.h5ad")
    print("visium read in")
    adata_sub.obs[adata_sub.uns['mod']['factor_names']] = adata_sub.obsm['q05_cell_abundance_w_sf']
    IDs = meta[meta[vis_subset_key] == value].sangerID.tolist()
    print(f"{value}: {IDs}")

    for sample in IDs:
        try:
            slide = select_slide(adata_sub, sample)
        except IndexError as e:
            print(f"Sample {sample} not found in adata: {e}")
            continue  # Skip this sample if not found

        slide.obs[slide.uns['mod']['factor_names']] = slide.obsm['q05_cell_abundance_w_sf']
        list_of_cell_types = slide.uns['mod']['factor_names'].tolist()
        print("plotting HE image...")
        sc.pl.spatial(slide, color=None, size=0, img_key='hires', title=f"HE:{sample}", 
                      save=f"{run}__{value}__{sample}__HE.png", show=False)

        # Plotting abundances for all relevant cell states
        if list_of_cell_types:
            print("plotting abundances for all cell states...")
            sc.pl.spatial(slide, color=list_of_cell_types, cmap="RdPu", ncols=10, size=1.3, 
                          img_key='hires', vmin=0, vmax='p99.2', alpha_img=0.3, 
                          save=f"{run}__{value}__{sample}__all_cell_states_abundances.png", show=False)
```

Example 12 (unknown):
```unknown
Found these subsets ['FFPE_16PCW']
###### FFPE_16PCW ######
visium read in
FFPE_16PCW: ['BHF_F_Hea13295830', 'BHF_F_Hea13295831']
```

Example 13 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
```

Example 14 (unknown):
```unknown
plotting HE image...
```

Example 15 (python):
```python
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
File <timed exec>:22

File /lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:1003, in spatial(adata, basis, img, img_key, library_id, crop_coord, alpha_img, bw, size, scale_factor, spot_size, na_color, show, return_fig, save, **kwargs)
   1000     cmap_img = None
   1001 circle_radius = size * scale_factor * spot_size * 0.5
-> 1003 axs = embedding(
   1004     adata,
   1005     basis=basis,
   1006     scale_factor=scale_factor,
   1007     size=circle_radius,
   1008     na_color=na_color,
   1009     show=False,
   1010     save=False,
   1011     **kwargs,
   1012 )
   1013 if not isinstance(axs, list):
   1014     axs = [axs]

File /lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392, in embedding(adata, basis, color, gene_symbols, use_raw, sort_order, edges, edges_width, edges_color, neighbors_key, arrows, arrows_kwds, groups, components, dimensions, layer, projection, scale_factor, color_map, cmap, palette, na_color, na_in_legend, size, frameon, legend_fontsize, legend_fontweight, legend_loc, legend_fontoutline, colorbar_loc, vmax, vmin, vcenter, norm, add_outline, outline_width, outline_color, ncols, hspace, wspace, title, show, save, ax, return_fig, **kwargs)
    389         # if user did not set alpha, set alpha to 0.7
    390         kwargs['alpha'] = 0.7 if alpha is None else alpha
--> 392     cax = scatter(
    393         coords[:, 0],
    394         coords[:, 1],
    395         marker=".",
    396         c=color_vector,
    397         rasterized=settings._vector_friendly,
    398         norm=normalize,
    399         **kwargs,
    400     )
    402 # remove y and x ticks
    403 ax.set_yticks([])

File /lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/scanpy/plotting/_utils.py:1108, in circles(x, y, s, ax, marker, c, vmin, vmax, scale_factor, **kwargs)
   1105 # You can set `facecolor` with an array for each patch,
   1106 # while you can only set `facecolors` with a value for all.
   1107 if scale_factor != 1.0:
-> 1108     x = x * scale_factor
   1109     y = y * scale_factor
   1110 zipped = np.broadcast(x, y, s)

TypeError: can't multiply sequence by non-int of type 'float'
```

Example 16 (python):
```python
%%time

adata_dict={}

subset_values = os.listdir(os.path.join(results_folder, "cell2location_map", run))
print(f"Found these subsets {subset_values}")

for value in subset_values:
    print(f"###### {value} ######")
    adata_sub = sc.read(f"{run_name}/{value}/sp.h5ad")
    print("visium read in")
    adata_sub.obs[adata_sub.uns['mod']['factor_names']] = adata_sub.obsm['q05_cell_abundance_w_sf']
    IDs = meta[meta[vis_subset_key] == value].sangerID.tolist()
    print(f"{value}: {IDs}")

    for sample in IDs:
        try:
            slide = select_slide(adata_sub, sample)
        except IndexError as e:
            print(f"Sample {sample} not found in adata: {e}")
            continue  # Skip this sample if not found

        slide.obs[slide.uns['mod']['factor_names']] = slide.obsm['q05_cell_abundance_w_sf']
        adata_dict[sample]=slide
```

Example 17 (unknown):
```unknown
Found these subsets ['FFPE_16PCW']
###### FFPE_16PCW ######
visium read in
FFPE_16PCW: ['BHF_F_Hea13295830', 'BHF_F_Hea13295831']
```

Example 18 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
```

Example 19 (unknown):
```unknown
CPU times: user 793 ms, sys: 1.66 s, total: 2.45 s
Wall time: 3.88 s
```

Example 20 (python):
```python
adata_dict.keys()
```

Example 21 (unknown):
```unknown
dict_keys(['BHF_F_Hea13295830', 'BHF_F_Hea13295831'])
```

Example 22 (python):
```python
from tqdm import tqdm
atrial_data_dict={}
for sample,tmp_adata in tqdm(adata_dict.items()):
    print(f"sample: {sample},  age is {tmp_adata.obs.age.unique().tolist()[0]}")
    tmp_df=tmp_adata[tmp_adata.obs.JC_edit=="Myocardium - Atrial"].obs[['AtrialCardiomyocytesLeft_immature', 'AtrialCardiomyocytesLeft', 'AtrialCardiomyocytesRight','AtrialCardiomyocytesRight_immature']]

    if tmp_df.shape[0]>0:
        tmp_df["sample"]=sample
        tmp_df["age"]=tmp_adata.obs.age.unique().tolist()[0]
        atrial_data_dict[sample]=tmp_df
    else:
        print(f"no atrial myocardium spots annotated for sample: {sample}")
```

Example 23 (python):
```python
0%|                                                                                                                                                                                                                     | 0/2 [00:00<?, ?it/s]/tmp/ipykernel_266704/3397130741.py:8: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  tmp_df["sample"]=sample
/tmp/ipykernel_266704/3397130741.py:9: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  tmp_df["age"]=tmp_adata.obs.age.unique().tolist()[0]
/tmp/ipykernel_266704/3397130741.py:8: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  tmp_df["sample"]=sample
/tmp/ipykernel_266704/3397130741.py:9: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  tmp_df["age"]=tmp_adata.obs.age.unique().tolist()[0]
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 133.30it/s]
```

Example 24 (unknown):
```unknown
sample: BHF_F_Hea13295830,  age is PCW-16w
sample: BHF_F_Hea13295831,  age is PCW-16w
```

Example 25 (python):
```python
data=pd.concat(list(atrial_data_dict.values()))
data
```

Example 26 (unknown):
```unknown
data.to_csv("AtrialProportionsFFPE.csv")
```

Example 27 (unknown):
```unknown
# Assuming 'data' is your pandas DataFrame
# Calculate the total for each row
total = data['AtrialCardiomyocytesImmature'] + data['AtrialCardiomyocytesRight'] + data['AtrialCardiomyocytesLeft']

# Calculate the proportions for each column
data['AtrialCardiomyocytesImmature_prop'] = data['AtrialCardiomyocytesImmature'] / total
data['AtrialCardiomyocytesRight_prop'] = data['AtrialCardiomyocytesRight'] / total
data['AtrialCardiomyocytesLeft_prop'] = data['AtrialCardiomyocytesLeft'] / total

# The new DataFrame 'data' now includes the proportion columns
data
```

Example 28 (unknown):
```unknown
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
Cell In[17], line 3
      1 # Assuming 'data' is your pandas DataFrame
      2 # Calculate the total for each row
----> 3 total = data['AtrialCardiomyocytesImmature'] + data['AtrialCardiomyocytesRight'] + data['AtrialCardiomyocytesLeft']
      5 # Calculate the proportions for each column
      6 data['AtrialCardiomyocytesImmature_prop'] = data['AtrialCardiomyocytesImmature'] / total

NameError: name 'data' is not defined
```

Example 29 (python):
```python
import plotly.express as px
fig = px.scatter_ternary(data,
                         a="AtrialCardiomyocytesImmature_prop", b="AtrialCardiomyocytesLeft_prop", c="AtrialCardiomyocytesRight_prop",
                        color="age",
                        opacity=0.8)
fig.show()
```

---

## Functions¶

**URL:** http://127.0.0.1:9020/notebooks_html/Supplementary_materials/Epigenetic_stability/no3_entropy-score.html

**Contents:**
- Functions¶
- Parameters¶
- Read in RNA Milo nhoods data¶
- Read in ATAC Milo nhoods data¶
- Entropy of the diffusion of each RNA nhood in ATAC space¶
- Model entropy¶
- Assign fine-grained celltype to each nhood¶
- Entropy of the diffusion of each RNA nhood in ATAC space (selected cells)¶
- Save¶
- Plot scores on UMAP¶
- Plot entropy scores per cell type¶
- All mid-grained cell types¶
- Kruskal-Wallis Test for two groups¶
  - Atrial cardiomyocytes¶
  - Ventricular cardiomyocytes¶
  - Myeloid cells¶
- For figure plots¶
  - UMAPs¶
  - Bar plots¶

**Examples:**

Example 1 (python):
```python
import warnings
warnings.simplefilter("ignore", UserWarning)

import scanpy as sc
import pandas as pd
import numpy as np
import anndata
import os
from progressbar import progressbar

import matplotlib.pyplot as plt
import matplotlib.patches as mpatches
import seaborn as sns

import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 

from scipy.stats import pearsonr
```

Example 2 (python):
```python
import milopy.core as milo
import milopy.plot as milopl
import milopy.utils
```

Example 3 (python):
```python
import session_info
session_info.show()
```

Example 4 (python):
```python
-----
anndata             0.9.2
matplotlib          3.7.2
milopy              0.1.1
numpy               1.24.4
pandas              1.5.3
progressbar         4.3.2
scanpy              1.9.3
scipy               1.10.1
seaborn             0.12.2
session_info        1.0.0
-----
```

Example 5 (python):
```python
PIL                         10.0.0
anyio                       NA
arrow                       1.2.3
asttokens                   NA
attr                        23.1.0
attrs                       23.1.0
babel                       2.12.1
backcall                    0.2.0
backports                   NA
brotli                      1.0.9
certifi                     2023.11.17
cffi                        1.15.1
charset_normalizer          3.2.0
colorama                    0.4.6
comm                        0.1.4
cycler                      0.10.0
cython_runtime              NA
dateutil                    2.8.2
debugpy                     1.6.8
decorator                   5.1.1
defusedxml                  0.7.1
executing                   1.2.0
fastjsonschema              NA
fqdn                        NA
h5py                        3.9.0
idna                        3.4
igraph                      0.10.6
importlib_metadata          NA
importlib_resources         NA
ipykernel                   6.25.1
isoduration                 NA
jedi                        0.19.0
jinja2                      3.1.2
joblib                      1.3.2
json5                       NA
jsonpointer                 2.0
jsonschema                  4.19.0
jsonschema_specifications   NA
jupyter_events              0.7.0
jupyter_server              2.7.1
jupyterlab_server           2.24.0
kiwisolver                  1.4.5
leidenalg                   0.10.1
llvmlite                    0.40.1
markupsafe                  2.1.3
mpl_toolkits                NA
natsort                     8.4.0
nbformat                    5.9.2
numba                       0.57.1
overrides                   NA
packaging                   23.1
parso                       0.8.3
patsy                       0.5.3
pexpect                     4.8.0
pickleshare                 0.7.5
pkg_resources               NA
plotly                      5.16.1
prometheus_client           NA
prompt_toolkit              3.0.39
psutil                      5.9.5
ptyprocess                  0.7.0
pure_eval                   0.2.2
pydev_ipython               NA
pydevconsole                NA
pydevd                      2.9.5
pydevd_file_utils           NA
pydevd_plugins              NA
pydevd_tracing              NA
pygments                    2.16.1
pyparsing                   3.0.9
python_utils                NA
pythonjsonlogger            NA
pytz                        2023.3
referencing                 NA
requests                    2.31.0
rfc3339_validator           0.1.4
rfc3986_validator           0.1.1
rpds                        NA
rpy2                        3.5.14
ruamel                      NA
send2trash                  NA
six                         1.16.0
sklearn                     1.3.0
sniffio                     1.3.0
socks                       1.7.1
sphinxcontrib               NA
stack_data                  0.6.2
statsmodels                 0.14.0
texttable                   1.6.7
threadpoolctl               3.2.0
tornado                     6.3.3
traitlets                   5.9.0
typing_extensions           NA
tzlocal                     NA
uri_template                NA
urllib3                     2.0.4
wcwidth                     0.2.6
webcolors                   1.13
websocket                   1.6.2
yaml                        6.0.1
zipp                        NA
zmq                         25.1.1
zstandard                   0.19.0
```

Example 6 (unknown):
```unknown
-----
IPython             8.12.0
jupyter_client      8.3.0
jupyter_core        4.12.0
jupyterlab          4.0.5
notebook            7.0.2
-----
Python 3.8.17 (default, Jul  5 2023, 21:04:15) [GCC 11.2.0]
Linux-4.15.0-220-generic-x86_64-with-glibc2.17
-----
Session information updated at 2024-02-15 14:08
```

Example 7 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 8 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 9 (python):
```python
from scipy.stats import entropy
import numpy as np
from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model import LinearRegression
from sklearn.pipeline import make_pipeline

def calculate_entropy(df1, df2, relative_to_max=True):
    """
    Calculate the entropy-based diffusion scores for each group defined in df1 and apply it in df2.

    Parameters:
    - df1: DataFrame representing cell-by-cluster matrix for defining groups.
    - df2: DataFrame representing cell-by-cluster matrix for calculating entropy.

    Returns:
    A dictionary with group names as keys and their entropy scores as values.
    """
    scores = {}
    for group_name in progressbar(df1.columns):
        # Identify cells belonging to the current group
        group_indices = df1.index[df1[group_name] == 1].tolist()
        # Subset df2 for the current group
        group_df2 = df2.loc[group_indices]
        # Calculate the distribution of cells across clusters in df2
        cluster_distribution = group_df2.sum(axis=0) / group_df2.sum(axis=0).sum()
        # Calculate entropy for the group
        if relative_to_max:
            # Prepare maximum distribution
            # Create a DataFrame filled with zeros
            max_distr_df = pd.DataFrame(0, index=group_df2.index, columns=group_df2.columns)
            # Replace elements with 1
            col_indices = [i % max_distr_df.shape[1] for i in range(len(group_indices))]
            max_distr_df.values[range(len(group_indices)), col_indices] = 1
            # display(max_distr_df)
            # Calculate disctribution
            max_distribution = max_distr_df.sum(axis=0) / max_distr_df.sum(axis=0).sum()
            # print(max_distribution)
            scores[group_name] = entropy(pk=cluster_distribution,
                                         qk=max_distribution,
                                        )
        else:
            scores[group_name] = entropy(pk=cluster_distribution)

    return scores



def calculate_kl_divergence(df1, df2):
    """
    Calculate the Kullback-Leibler divergence for each group defined in df1 and apply it in df2.

    Parameters:
    - df1: DataFrame representing cell-by-cluster matrix for defining groups.
    - df2: DataFrame representing cell-by-cluster matrix for calculating KL divergence.

    Returns:
    A dictionary with group names as keys and their KL divergence as values.
    """
    scores = {}
    num_clusters = df2.shape[1]
    uniform_distribution = np.ones(num_clusters) / num_clusters

    for group_name in df1.columns:
        # Identify cells belonging to the current group
        group_indices = df1.index[df1[group_name] == 1].tolist()
        # Subset df2 for the current group
        group_df2 = df2.loc[group_indices]
        # Calculate the distribution of cells across clusters in df2
        cluster_distribution = group_df2.sum(axis=0) / group_df2.sum(axis=0).sum()
        # Calculate KL divergence for the group
        scores[group_name] = entropy(cluster_distribution, uniform_distribution)

    return scores

def model_entropy(y, x:list, degree=2):
    """
    Fit a trend to the nhood-wise entropy against the data size.

    Parameters:
    y (array-like): Numeric vector of dependent variable 
    x (list): list of numeric vector of independent variables (regressor)
    degree (int, optional): The degree of polynomials to be used in the loess fit.

    Returns:
    dict: A dictionary containing the fitted values and the model.
    """

    # Preparing the data for fitting
    X_list = []
    for xx in x:
        X_list.append(np.array(xx))
    Y = np.array(y)
    
    # Combine X1 and X2 into a single feature matrix
    if len(X_list)>1:
        X = np.column_stack(X_list)
    else:
        X = X_list[0]

    # Creating a polynomial regression model
    model = make_pipeline(PolynomialFeatures(degree), LinearRegression())
    model.fit(X, Y)

    # Predicting the fitted values
    fitted_values = model.predict(X)

    return {"fitted_values": fitted_values, "model": model}
```

Example 10 (python):
```python
# dir_to_adata_peakvi = '/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/PeakMatrix/mid_grain_mod'
path_to_gex_milo_nhoods = '/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/Milo_nhoods_gex/scVI_mid_grain_mod'
path_to_atac_milo_nhoods = '/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/Milo_nhoods/mid_grain_mod'
path_to_adata_gex = '/nfs/team205/heart/anndata_objects/Foetal/Feb28ObjectRaw_Multiome.h5ad'
celltype_col = 'mid_grain_mod'
```

Example 11 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk'
```

Example 12 (unknown):
```unknown
# order of mid grain (mod) cell types
celltypes = [
    'AtrialCardiomyocytes_plusCCS',
    'VentricularCardiomyocytes_plusCCS',
    'Fibroblasts',
    'MuralCells',
    'PericardialCells', 
    'EpicardialCells',
    'BloodVesselEndothelialCells',
    'EndocardialCells', 
     'Neurons',
    'Glia', 
     'MyeloidCells',
    'LymphoidCells',   
             ]
```

Example 13 (python):
```python
# nhood results
prefix = 'one-layer_added-cov-region_subsampled-n1000_'
suffix = '_milo_nhoods_result.csv'

rna_nhoods_res = {}
for file in os.listdir(path_to_gex_milo_nhoods):
    if (prefix in file)&(suffix in file):
        celltype = file.replace(prefix,'').replace(suffix,'')
        print(celltype)
        rna_nhoods_res[celltype] = pd.read_csv(f'{path_to_gex_milo_nhoods}/{prefix}{celltype}{suffix}',
                                           index_col=0
                                          )
        # rename nhood names
        rna_nhoods_res[celltype].index = celltype+'_'+rna_nhoods_res[celltype].index.astype('str')
rna_nhoods_res[celltype].head(n=3)
```

Example 14 (unknown):
```unknown
LymphoidCells
LymphaticEndothelialCells
EndocardialCells
AtrialCardiomyocytes_plusCCS
MuralCells
Neurons
VentricularCardiomyocytes_plusCCS
Fibroblasts
EpicardialCells
MyeloidCells
BloodVesselEndothelialCells
Glia
PericardialCells
```

Example 15 (python):
```python
prefix = 'one-layer_added-cov-region_subsampled-n1000_'
suffix = '_milo_cell-by-nhood.csv'

rna_cell_by_nhood_dfs = {}
for file in os.listdir(path_to_gex_milo_nhoods):
    if (prefix in file)&(suffix in file):
        celltype = file.replace(prefix,'').replace(suffix,'')
        print(celltype)
        rna_cell_by_nhood_dfs[celltype] = pd.read_csv(f'{path_to_gex_milo_nhoods}/{prefix}{celltype}{suffix}',
                                           index_col=0
                                          )
        # rename nhood names
        rna_cell_by_nhood_dfs[celltype].columns = celltype+'_'+rna_cell_by_nhood_dfs[celltype].columns.astype('str')
rna_cell_by_nhood_dfs[celltype].head(n=3)
```

Example 16 (unknown):
```unknown
VentricularCardiomyocytes_plusCCS
AtrialCardiomyocytes_plusCCS
MyeloidCells
PericardialCells
LymphoidCells
BloodVesselEndothelialCells
LymphaticEndothelialCells
Neurons
MuralCells
EpicardialCells
Glia
EndocardialCells
Fibroblasts
```

Example 17 (python):
```python
midfix = '_added-cov-region_subsampled-n1000'
suffix = '_milo_cell-by-nhood.csv'

atac_cell_by_nhood_dfs = {}
for file in os.listdir(path_to_atac_milo_nhoods):
    if (midfix in file)&(suffix in file):
        celltype = file.replace(midfix,'').replace(suffix,'')
        print(celltype)
        atac_cell_by_nhood_dfs[celltype] = pd.read_csv(f'{path_to_atac_milo_nhoods}/{celltype}{midfix}{suffix}',
                                           index_col=0
                                          )
        # rename nhood names
        atac_cell_by_nhood_dfs[celltype].columns = 'ATAC_'+celltype+'_'+atac_cell_by_nhood_dfs[celltype].columns.astype('str')

atac_cell_by_nhood_dfs[celltype].head(n=3)
```

Example 18 (unknown):
```unknown
PericardialCells
Glia
BloodVesselEndothelialCells
AtrialCardiomyocytes_plusCCS
MuralCells
VentricularCardiomyocytes_plusCCS
Neurons
Fibroblasts
EpicardialCells
MyeloidCells
LymphoidCells
EndocardialCells
LymphaticEndothelialCells
```

Example 19 (python):
```python
# for each celltype
for i,celltype in enumerate(celltypes):
    # entropy score
    res = calculate_entropy(df1 = rna_cell_by_nhood_dfs[celltype], 
                            df2 = atac_cell_by_nhood_dfs[celltype],
                            relative_to_max=False
                                                             )
    # add to nhood result dataframe
    rna_nhoods_res[celltype].loc[res.keys(),'entropy score'] = list(res.values())
    # checkign whether there's nan
    if rna_nhoods_res[celltype]['entropy score'].isna().sum()>0:
        print(f'{celltype} has nan in the result')

rna_nhoods_res[celltype].head(n=3)
```

Example 20 (unknown):
```unknown
100% (195 of 195) |######################| Elapsed Time: 0:00:00 Time:  0:00:00
100% (300 of 300) |######################| Elapsed Time: 0:00:00 Time:  0:00:00
100% (300 of 300) |######################| Elapsed Time: 0:00:00 Time:  0:00:00
100% (118 of 118) |######################| Elapsed Time: 0:00:00 Time:  0:00:00
100% (83 of 83) |########################| Elapsed Time: 0:00:00 Time:  0:00:00
100% (65 of 65) |########################| Elapsed Time: 0:00:00 Time:  0:00:00
100% (138 of 138) |######################| Elapsed Time: 0:00:00 Time:  0:00:00
100% (65 of 65) |########################| Elapsed Time: 0:00:00 Time:  0:00:00
100% (50 of 50) |########################| Elapsed Time: 0:00:00 Time:  0:00:00
100% (80 of 80) |########################| Elapsed Time: 0:00:00 Time:  0:00:00
100% (161 of 161) |######################| Elapsed Time: 0:00:00 Time:  0:00:00
100% (284 of 284) |######################| Elapsed Time: 0:00:00 Time:  0:00:00
```

Example 21 (python):
```python
for celltype in celltypes:
    nhood_sizes = np.array(rna_nhoods_res[celltype]['Nhood_size'])
    variances = np.array(rna_nhoods_res[celltype]['variance_scVI-latent'])
    entropy_scores = np.array(rna_nhoods_res[celltype]['entropy score'])
    fitted_values = model_entropy(y=entropy_scores, x=[nhood_sizes,variances], degree=1)['fitted_values']
    residuals = entropy_scores - fitted_values
    # add to the dataframe
    rna_nhoods_res[celltype]['entropy score, fitted'] = fitted_values.copy()
    rna_nhoods_res[celltype]['entropy score, residual'] = residuals.copy()
```

Example 22 (python):
```python
# Plotting original value with fitted line
# prepare figure
nrows = 2
ncols = 6
fig,axs=plt.subplots(nrows=nrows,ncols=ncols,figsize=(22,7)) 
for i,celltype in enumerate(celltypes):
    df = rna_nhoods_res[celltype]
    corr_res = pearsonr(df['Nhood_size'],df['entropy score'])
    axs[i//ncols,i%ncols].scatter(x=df['Nhood_size'],y=df['entropy score'])
    # Sort the sizes array and reorder the fitted values accordingly
    sorted_indices = np.argsort(df['Nhood_size'])
    sorted_sizes = np.array(df['Nhood_size'])[sorted_indices]
    sorted_fitted_values = df['entropy score, fitted'][sorted_indices]
    axs[i//ncols,i%ncols].plot(sorted_sizes,sorted_fitted_values,color='red')
    axs[i//ncols,i%ncols].set_title(f'{celltype}\nPearson corr coeff: {round(corr_res[0],2)}, pval: {round(corr_res[1],3)}')
    axs[i//ncols,i%ncols].set_xlabel('Nhood_size')
    axs[i//ncols,i%ncols].set_ylabel('entropy score')
fig.suptitle('before regressing out')
fig.tight_layout()

fig,axs=plt.subplots(nrows=nrows,ncols=ncols,figsize=(22,7)) 
for i,celltype in enumerate(celltypes):
    df = rna_nhoods_res[celltype]
    corr_res = pearsonr(df['variance_scVI-latent'],df['entropy score'])
    axs[i//ncols,i%ncols].scatter(x=df['variance_scVI-latent'],y=df['entropy score'])
    # Sort the sizes array and reorder the fitted values accordingly
    sorted_indices = np.argsort(df['variance_scVI-latent'])
    sorted_sizes = np.array(df['variance_scVI-latent'])[sorted_indices]
    sorted_fitted_values = df['entropy score, fitted'][sorted_indices]
    axs[i//ncols,i%ncols].plot(sorted_sizes,sorted_fitted_values,color='red')
    axs[i//ncols,i%ncols].set_title(f'{celltype}\nPearson corr coeff: {round(corr_res[0],2)}, pval: {round(corr_res[1],3)}')
    axs[i//ncols,i%ncols].set_xlabel('variance_scVI-latent')
    axs[i//ncols,i%ncols].set_ylabel('entropy score')
fig.suptitle('before regressing out')
fig.tight_layout()
```

Example 23 (python):
```python
# Plot after correction

# prepare figure
nrows = 2
ncols = 6
fig,axs=plt.subplots(nrows=nrows,ncols=ncols,figsize=(22,7)) 
for i,celltype in enumerate(celltypes):
    df = rna_nhoods_res[celltype]
    corr_res = pearsonr(df['Nhood_size'],df['entropy score, residual'])
    axs[i//ncols,i%ncols].scatter(x=df['Nhood_size'],y=df['entropy score, residual'])
    axs[i//ncols,i%ncols].set_title(f'{celltype}\nPearson corr coeff: {round(corr_res[0],2)}, pval: {round(corr_res[1],4)}')
    axs[i//ncols,i%ncols].set_xlabel('Nhood_size')
    axs[i//ncols,i%ncols].set_ylabel('entropy score, residual')
fig.suptitle('regressed out')
fig.tight_layout()

fig,axs=plt.subplots(nrows=nrows,ncols=ncols,figsize=(22,7)) 
for i,celltype in enumerate(celltypes):
    df = rna_nhoods_res[celltype]
    corr_res = pearsonr(df['variance_scVI-latent'],df['entropy score, residual'])
    axs[i//ncols,i%ncols].scatter(x=df['variance_scVI-latent'],y=df['entropy score, residual'])
    axs[i//ncols,i%ncols].set_title(f'{celltype}\nPearson corr coeff: {round(corr_res[0],2)}, pval: {round(corr_res[1],4)}')
    axs[i//ncols,i%ncols].set_xlabel('variance_scVI-latent')
    axs[i//ncols,i%ncols].set_ylabel('entropy score, residual')
fig.suptitle('regressed out')
fig.tight_layout()
```

Example 24 (python):
```python
purity_thresh = 0.6

# read in celltype info
ad = sc.read_h5ad(path_to_adata_gex,backed='r')
obs = ad.obs.copy()

# prepare figure
nrows = 2
ncols = 6
fig,axs=plt.subplots(nrows=nrows,ncols=ncols,figsize=(22,5)) 

rna_cell_by_nhood_dfs_sel = {}
# assign
for i,celltype in enumerate(celltypes):
    df = rna_cell_by_nhood_dfs[celltype].copy()
    # get cell number of each cell type in each nhood
    df.index = obs['fine_grain'].astype('str').reindex(df.index)
    df = df.reset_index().groupby('fine_grain').sum()
    # purity (proportion of the largest cell type)
    purity_arr = df.max(0)/df.sum(0)
    # get pure nhood
    pure_nhoods = list(purity_arr.index[purity_arr>purity_thresh])
    axs[i//ncols,i%ncols].hist(purity_arr,bins=50)
    axs[i//ncols,i%ncols].axvline(x=purity_thresh, color='red', linestyle='--', linewidth=1.5)
    axs[i//ncols,i%ncols].set_title(f'{celltype}\nprop of pure nhoods: {round(len(pure_nhoods)/len(purity_arr),3)}')
    # assign celltype
    df_pure = df[pure_nhoods]
    mask = df_pure == df_pure.max(0)
    assigned_pre = mask.reset_index().melt(id_vars=['fine_grain']).rename(columns={'value':'assigned',
                                                                               'variable':'nhood',
                                                                               'fine_grain':'assigned_fine_grain'
                                                                              })
    assigned_pre['purity_fine_grain'] = list(purity_arr[assigned_pre['nhood']])
    assigned = assigned_pre[assigned_pre['assigned']]
    # get cells of the assigned_celltype
    rna_cell_by_nhood_dfs_sel[celltype] = pd.DataFrame(0,
                                                       columns=rna_cell_by_nhood_dfs[celltype].columns,
                                                       index=rna_cell_by_nhood_dfs[celltype].index)
    for ind in assigned.index:
        celltype_assigned = assigned.loc[ind,'assigned_fine_grain']
        nh = assigned.loc[ind,'nhood']
        nh_cells = list(rna_cell_by_nhood_dfs[celltype].index[rna_cell_by_nhood_dfs[celltype][nh]==1])
        celltype_cells = list(obs.index[obs['fine_grain']==celltype_assigned])
        cells = list(set(nh_cells).intersection(celltype_cells))
        rna_cell_by_nhood_dfs_sel[celltype].loc[cells,nh] = 1
        rna_nhoods_res[celltype].loc[nh,'cells_of_assigned_celltype'] = ','.join(cells)
        rna_nhoods_res[celltype].loc[nh,'Nhood_size_sel'] = len(cells)
    
    # add to nhood result dataframe
    rna_nhoods_res[celltype].loc[assigned['nhood'],'assigned_fine_grain'] = list(assigned['assigned_fine_grain'])
    rna_nhoods_res[celltype].loc[assigned['nhood'],'purity_fine_grain'] = list(assigned['purity_fine_grain'])
    # sanity check
    if sum(rna_nhoods_res[celltype]['assigned_fine_grain'].isna()==False)/len(rna_nhoods_res[celltype])!=len(pure_nhoods)/len(purity_arr):
        print(f'{celltype}: something went wrong when adding to result dataframe')

fig.tight_layout()
rna_nhoods_res[celltype].head(n=3)
```

Example 25 (unknown):
```unknown
# just checking the calculations are consistent
sum(rna_nhoods_res[celltype]['Nhood_size_sel']/rna_nhoods_res[celltype]['Nhood_size'] == rna_nhoods_res[celltype]['purity_fine_grain'])
```

Example 26 (python):
```python
# for each celltype
for i,celltype in enumerate(celltypes):
    # entropy score
    res = calculate_entropy(df1 = rna_cell_by_nhood_dfs_sel[celltype], 
                            df2 = atac_cell_by_nhood_dfs[celltype],
                            relative_to_max=False
                                                             )
    # add to nhood result dataframe
    rna_nhoods_res[celltype].loc[res.keys(),'entropy score, selected cells'] = list(res.values())
    # checkign whether there's more nan than expected
    if rna_nhoods_res[celltype]['entropy score, selected cells'].isna().sum()>rna_nhoods_res[celltype]['assigned_fine_grain'].isna().sum():
        print(f'{celltype} has larger nan than expected')

rna_nhoods_res[celltype].head(n=3)
```

Example 27 (unknown):
```unknown
100% (195 of 195) |######################| Elapsed Time: 0:00:00 Time:  0:00:00
100% (300 of 300) |######################| Elapsed Time: 0:00:00 Time:  0:00:00
100% (300 of 300) |######################| Elapsed Time: 0:00:00 Time:  0:00:00
100% (118 of 118) |######################| Elapsed Time: 0:00:00 Time:  0:00:00
100% (83 of 83) |########################| Elapsed Time: 0:00:00 Time:  0:00:00
100% (65 of 65) |########################| Elapsed Time: 0:00:00 Time:  0:00:00
100% (138 of 138) |######################| Elapsed Time: 0:00:00 Time:  0:00:00
100% (65 of 65) |########################| Elapsed Time: 0:00:00 Time:  0:00:00
100% (50 of 50) |########################| Elapsed Time: 0:00:00 Time:  0:00:00
100% (80 of 80) |########################| Elapsed Time: 0:00:00 Time:  0:00:00
100% (161 of 161) |######################| Elapsed Time: 0:00:00 Time:  0:00:00
100% (284 of 284) |######################| Elapsed Time: 0:00:00 Time:  0:00:00
```

Example 28 (python):
```python
# regress out the effect of Nhood_size
for celltype in celltypes:
    df_sel = rna_nhoods_res[celltype].dropna(subset='assigned_fine_grain')
    nhood_sizes = np.array(df_sel['Nhood_size'])
    variances = np.array(df_sel['variance_scVI-latent'])
    entropy_scores = np.array(df_sel['entropy score'])
    fitted_values = model_entropy(y=entropy_scores, x=[nhood_sizes,variances], degree=1)['fitted_values']
    residuals = entropy_scores - fitted_values
    df_out = pd.DataFrame({'nhood':df_sel.index,'fitted':fitted_values,'residual':residuals})
    df_out.set_index('nhood',inplace=True)
    # add to the dataframe
    rna_nhoods_res[celltype]['entropy score, selected cells, fitted'] = df_out['fitted'].reindex(rna_nhoods_res[celltype].index).copy()
    rna_nhoods_res[celltype]['entropy score, selected cells, residual'] = df_out['residual'].reindex(rna_nhoods_res[celltype].index).copy()
```

Example 29 (unknown):
```unknown
utils.save_pkl(rna_nhoods_res,
               '/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/Milo_nhoods_gex/scVI_mid_grain_mod/rna_nhoods_res.pkl'
              )
```

Example 30 (unknown):
```unknown
# read in
rna_nhoods_res = utils.read_pkl('/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/Milo_nhoods_gex/scVI_mid_grain_mod/rna_nhoods_res.pkl')
rna_nhoods_res.keys()
```

Example 31 (unknown):
```unknown
dict_keys(['LymphoidCells', 'LymphaticEndothelialCells', 'EndocardialCells', 'AtrialCardiomyocytes_plusCCS', 'MuralCells', 'Neurons', 'VentricularCardiomyocytes_plusCCS', 'Fibroblasts', 'EpicardialCells', 'MyeloidCells', 'BloodVesselEndothelialCells', 'Glia', 'PericardialCells'])
```

Example 32 (python):
```python
# read in anndata include UMAP embedding
gex_adata_dict = utils.read_pkl(f'{path_to_gex_milo_nhoods}/one-layer_added-cov-region_subsampled-n1000_post_milo_adata_dictionary.pkl')
# atac_adata_dict = utils.read_pkl(f'{path_to_atac_milo_nhoods}/added-cov-region_subsampled-n1000_post_milo_adata_dictionary.pkl')
```

Example 33 (python):
```python
?sc.pl.embedding
```

Example 34 (python):
```python
Signature:
sc.pl.embedding(
    adata: anndata._core.anndata.AnnData,
    basis: str,
    *,
    color: Union[str, Sequence[str], NoneType] = None,
    gene_symbols: Union[str, NoneType] = None,
    use_raw: Union[bool, NoneType] = None,
    sort_order: bool = True,
    edges: bool = False,
    edges_width: float = 0.1,
    edges_color: Union[str, Sequence[float], Sequence[str]] = 'grey',
    neighbors_key: Union[str, NoneType] = None,
    arrows: bool = False,
    arrows_kwds: Union[Mapping[str, Any], NoneType] = None,
    groups: Union[str, NoneType] = None,
    components: Union[str, Sequence[str]] = None,
    dimensions: Union[Tuple[int, int], Sequence[Tuple[int, int]], NoneType] = None,
    layer: Union[str, NoneType] = None,
    projection: Literal['2d', '3d'] = '2d',
    scale_factor: Union[float, NoneType] = None,
    color_map: Union[matplotlib.colors.Colormap, str, NoneType] = None,
    cmap: Union[matplotlib.colors.Colormap, str, NoneType] = None,
    palette: Union[str, Sequence[str], cycler.Cycler, NoneType] = None,
    na_color: Union[str, Tuple[float, ...]] = 'lightgray',
    na_in_legend: bool = True,
    size: Union[float, Sequence[float], NoneType] = None,
    frameon: Union[bool, NoneType] = None,
    legend_fontsize: Union[int, float, Literal['xx-small', 'x-small', 'small', 'medium', 'large', 'x-large', 'xx-large'], NoneType] = None,
    legend_fontweight: Union[int, Literal['light', 'normal', 'medium', 'semibold', 'bold', 'heavy', 'black']] = 'bold',
    legend_loc: str = 'right margin',
    legend_fontoutline: Union[int, NoneType] = None,
    colorbar_loc: Union[str, NoneType] = 'right',
    vmax: Union[str, float, Callable[[Sequence[float]], float], Sequence[Union[str, float, Callable[[Sequence[float]], float]]], NoneType] = None,
    vmin: Union[str, float, Callable[[Sequence[float]], float], Sequence[Union[str, float, Callable[[Sequence[float]], float]]], NoneType] = None,
    vcenter: Union[str, float, Callable[[Sequence[float]], float], Sequence[Union[str, float, Callable[[Sequence[float]], float]]], NoneType] = None,
    norm: Union[matplotlib.colors.Normalize, Sequence[matplotlib.colors.Normalize], NoneType] = None,
    add_outline: Union[bool, NoneType] = False,
    outline_width: Tuple[float, float] = (0.3, 0.05),
    outline_color: Tuple[str, str] = ('black', 'white'),
    ncols: int = 4,
    hspace: float = 0.25,
    wspace: Union[float, NoneType] = None,
    title: Union[str, Sequence[str], NoneType] = None,
    show: Union[bool, NoneType] = None,
    save: Union[bool, str, NoneType] = None,
    ax: Union[matplotlib.axes._axes.Axes, NoneType] = None,
    return_fig: Union[bool, NoneType] = None,
    **kwargs,
) -> Union[matplotlib.figure.Figure, matplotlib.axes._axes.Axes, NoneType]
Docstring:
Scatter plot for user specified embedding basis (e.g. umap, pca, etc)

Parameters
----------
basis : str
    Name of the `obsm` basis to use.
adata : AnnData
    Annotated data matrix.
color : typing.Union[str, typing.Sequence[str], NoneType], optional (default: None)
    Keys for annotations of observations/cells or variables/genes, e.g.,
    `'ann1'` or `['ann1', 'ann2']`.
gene_symbols : typing.Union[str, NoneType], optional (default: None)
    Column name in `.var` DataFrame that stores gene symbols. By default `var_names`
    refer to the index column of the `.var` DataFrame. Setting this option allows
    alternative names to be used.
use_raw : typing.Union[bool, NoneType], optional (default: None)
    Use `.raw` attribute of `adata` for coloring with gene expression. If `None`,
    defaults to `True` if `layer` isn't provided and `adata.raw` is present.
layer : typing.Union[str, NoneType], optional (default: None)
    Name of the AnnData object layer that wants to be plotted. By default
    adata.raw.X is plotted. If `use_raw=False` is set, then `adata.X` is plotted.
    If `layer` is set to a valid layer name, then the layer is plotted. `layer`
    takes precedence over `use_raw`.
edges : bool, optional (default: False)
    Show edges.
edges_width : float, optional (default: 0.1)
    Width of edges.
edges_color : typing.Union[str, typing.Sequence[float], typing.Sequence[str]], optional (default: 'grey')
    Color of edges. See :func:`~networkx.drawing.nx_pylab.draw_networkx_edges`.
neighbors_key : typing.Union[str, NoneType], optional (default: None)
    Where to look for neighbors connectivities.
    If not specified, this looks .obsp['connectivities'] for connectivities
    (default storage place for pp.neighbors).
    If specified, this looks
    .obsp[.uns[neighbors_key]['connectivities_key']] for connectivities.
arrows : bool, optional (default: False)
    Show arrows (deprecated in favour of `scvelo.pl.velocity_embedding`).
arrows_kwds : typing.Union[typing.Mapping[str, typing.Any], NoneType], optional (default: None)
    Passed to :meth:`~matplotlib.axes.Axes.quiver`
sort_order : bool, optional (default: True)
    For continuous annotations used as color parameter, plot data points
    with higher values on top of others.
groups : typing.Union[str, NoneType], optional (default: None)
    Restrict to a few categories in categorical observation annotation.
    The default is not to restrict to any groups.
dimensions : typing.Union[typing.Tuple[int, int], typing.Sequence[typing.Tuple[int, int]], NoneType], optional (default: None)
    0-indexed dimensions of the embedding to plot as integers. E.g. [(0, 1), (1, 2)].
    Unlike `components`, this argument is used in the same way as `colors`, e.g. is
    used to specify a single plot at a time. Will eventually replace the components
    argument.
components : typing.Union[str, typing.Sequence[str]], optional (default: None)
    For instance, `['1,2', '2,3']`. To plot all available components use
    `components='all'`.
projection : typing.Literal['2d', '3d'], optional (default: '2d')
    Projection of plot (default: `'2d'`).
legend_loc : str, optional (default: 'right margin')
    Location of legend, either `'on data'`, `'right margin'` or a valid keyword
    for the `loc` parameter of :class:`~matplotlib.legend.Legend`.
legend_fontsize : typing.Union[int, float, typing.Literal['xx-small', 'x-small', 'small', 'medium', 'large', 'x-large', 'xx-large'], NoneType], optional (default: None)
    Numeric size in pt or string describing the size.
    See :meth:`~matplotlib.text.Text.set_fontsize`.
legend_fontweight : typing.Union[int, typing.Literal['light', 'normal', 'medium', 'semibold', 'bold', 'heavy', 'black']], optional (default: 'bold')
    Legend font weight. A numeric value in range 0-1000 or a string.
    Defaults to `'bold'` if `legend_loc == 'on data'`, otherwise to `'normal'`.
    See :meth:`~matplotlib.text.Text.set_fontweight`.
legend_fontoutline : typing.Union[int, NoneType], optional (default: None)
    Line width of the legend font outline in pt. Draws a white outline using
    the path effect :class:`~matplotlib.patheffects.withStroke`.
colorbar_loc : typing.Union[str, NoneType], optional (default: 'right')
    Where to place the colorbar for continous variables. If `None`, no colorbar
    is added.
size : typing.Union[float, typing.Sequence[float], NoneType], optional (default: None)
    Point size. If `None`, is automatically computed as 120000 / n_cells.
    Can be a sequence containing the size for each cell. The order should be
    the same as in adata.obs.
color_map : typing.Union[matplotlib.colors.Colormap, str, NoneType], optional (default: None)
    Color map to use for continous variables. Can be a name or a
    :class:`~matplotlib.colors.Colormap` instance (e.g. `"magma`", `"viridis"`
    or `mpl.cm.cividis`), see :func:`~matplotlib.cm.get_cmap`.
    If `None`, the value of `mpl.rcParams["image.cmap"]` is used.
    The default `color_map` can be set using :func:`~scanpy.set_figure_params`.
palette : typing.Union[str, typing.Sequence[str], cycler.Cycler, NoneType], optional (default: None)
    Colors to use for plotting categorical annotation groups.
    The palette can be a valid :class:`~matplotlib.colors.ListedColormap` name
    (`'Set2'`, `'tab20'`, …), a :class:`~cycler.Cycler` object, a dict mapping
    categories to colors, or a sequence of colors. Colors must be valid to
    matplotlib. (see :func:`~matplotlib.colors.is_color_like`).
    If `None`, `mpl.rcParams["axes.prop_cycle"]` is used unless the categorical
    variable already has colors stored in `adata.uns["{var}_colors"]`.
    If provided, values of `adata.uns["{var}_colors"]` will be set.
na_color : typing.Union[str, typing.Tuple[float, ...]], optional (default: 'lightgray')
    Color to use for null or masked values. Can be anything matplotlib accepts as a
    color. Used for all points if `color=None`.
na_in_legend : bool, optional (default: True)
    If there are missing values, whether they get an entry in the legend. Currently
    only implemented for categorical legends.
frameon : typing.Union[bool, NoneType], optional (default: None)
    Draw a frame around the scatter plot. Defaults to value set in
    :func:`~scanpy.set_figure_params`, defaults to `True`.
title : typing.Union[str, typing.Sequence[str], NoneType], optional (default: None)
    Provide title for panels either as string or list of strings,
    e.g. `['title1', 'title2', ...]`.

vmin : typing.Union[str, float, typing.Callable[[typing.Sequence[float]], float], typing.Sequence[typing.Union[str, float, typing.Callable[[typing.Sequence[float]], float]]], NoneType], optional (default: None)
    The value representing the lower limit of the color scale. Values smaller than vmin are plotted
    with the same color as vmin. vmin can be a number, a string, a function or `None`. If
    vmin is a string and has the format `pN`, this is interpreted as a vmin=percentile(N).
    For example vmin='p1.5' is interpreted as the 1.5 percentile. If vmin is function, then
    vmin is interpreted as the return value of the function over the list of values to plot.
    For example to set vmin tp the mean of the values to plot, `def my_vmin(values): return
    np.mean(values)` and then set `vmin=my_vmin`. If vmin is None (default) an automatic
    minimum value is used as defined by matplotlib `scatter` function. When making multiple
    plots, vmin can be a list of values, one for each plot. For example `vmin=[0.1, 'p1', None, my_vmin]`
vmax : typing.Union[str, float, typing.Callable[[typing.Sequence[float]], float], typing.Sequence[typing.Union[str, float, typing.Callable[[typing.Sequence[float]], float]]], NoneType], optional (default: None)
    The value representing the upper limit of the color scale. The format is the same as for `vmin`.
vcenter : typing.Union[str, float, typing.Callable[[typing.Sequence[float]], float], typing.Sequence[typing.Union[str, float, typing.Callable[[typing.Sequence[float]], float]]], NoneType], optional (default: None)
    The value representing the center of the color scale. Useful for diverging colormaps.
    The format is the same as for `vmin`.
    Example: sc.pl.umap(adata, color='TREM2', vcenter='p50', cmap='RdBu_r')
add_outline : typing.Union[bool, NoneType], optional (default: False)
    If set to True, this will add a thin border around groups of dots. In some situations
    this can enhance the aesthetics of the resulting image
outline_color : typing.Tuple[str, str], optional (default: ('black', 'white'))
    Tuple with two valid color names used to adjust the add_outline. The first color is the
    border color (default: black), while the second color is a gap color between the
    border color and the scatter dot (default: white).
outline_width : typing.Tuple[float, float], optional (default: (0.3, 0.05))
    Tuple with two width numbers used to adjust the outline. The first value is the width
    of the border color as a fraction of the scatter dot size (default: 0.3). The second value is
    width of the gap color (default: 0.05).
ncols : int, optional (default: 4)
    Number of panels per row.
wspace : typing.Union[float, NoneType], optional (default: None)
    Adjust the width of the space between multiple panels.
hspace : float, optional (default: 0.25)
    Adjust the height of the space between multiple panels.
return_fig : typing.Union[bool, NoneType], optional (default: None)
    Return the matplotlib figure.
kwargs : _empty
    Arguments to pass to :func:`matplotlib.pyplot.scatter`,
    for instance: the maximum and minimum values (e.g. `vmin=-2, vmax=5`).
show : typing.Union[bool, NoneType], optional (default: None)
     Show the plot, do not return axis.
save : typing.Union[bool, str, NoneType], optional (default: None)
    If `True` or a `str`, save the figure.
    A string is appended to the default filename.
    Infer the filetype if ending on {`'.pdf'`, `'.png'`, `'.svg'`}.
ax : typing.Union[matplotlib.axes._axes.Axes, NoneType], optional (default: None)
    A matplotlib axes object. Only works if plotting a single component.

Returns
-------
If `show==False` a :class:`~matplotlib.axes.Axes` or a list of it.
File:      /nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py
Type:      function
```

Example 35 (unknown):
```unknown
len(celltypes)
```

Example 36 (unknown):
```unknown
i=0, row:0, col:0,1
i=1, row:0, col:2,3
i=2, row:1, col:0,1
i=3, row:1, col:2,3
i=4, row:2, col:0,1
i=5, row:2, col:2,3
```

Example 37 (python):
```python
i = 3
if i%2==0:
    col_pos = i%4
else:
    col_pos = (i%4)+1
print(col_pos)
print(col_pos+1)
```

Example 38 (python):
```python
# prepare dictionary to store labels and colors
colors = {}
fig,axs=plt.subplots(nrows=6,ncols=4,figsize=(20,30))
for i,celltype in enumerate(celltypes):
    ad = gex_adata_dict[celltype].uns['nhood_adata']
    ad.obs['nhood'] = [f'{celltype}_nhood_{str(int(x)+1)}' for x in ad.obs.index]
    # add values
    ad.obs[f'assigned_fine_grain'] = list(rna_nhoods_res[celltype]['assigned_fine_grain'].reindex(ad.obs['nhood']))
    ad.obs[f'entropy score'] = list(rna_nhoods_res[celltype]['entropy score, residual'].reindex(ad.obs['nhood']))
    # print(celltype)
    row_pos = i//2
    if i%2==0:
        col_pos_1 = 0
        col_pos_2 = 1
    else:
        col_pos_1 = 2
        col_pos_2 = 3
    sc.pl.embedding(ad,basis='X_milo_graph',size=ad.obs['Nhood_size']*7,
                    color=['entropy score'],
                    cmap='inferno', frameon=False, wspace=0.15, 
                    title=f'{celltype}\nentropy score', show=False,
                    ax=axs[row_pos,col_pos_1]
                   )
    sc.pl.embedding(ad,basis='X_milo_graph',size=ad.obs['Nhood_size']*7,
                    color=['assigned_fine_grain'],
                    cmap='inferno', frameon=False, wspace=0.15, 
                    title=f'{celltype}\nassigned_fine_grain', show=False,
                    ax=axs[row_pos,col_pos_2]
                   )
    # store colors
    colors[celltype]={}
    colors[celltype]['labels'] = list(ad.obs[f'assigned_fine_grain'].cat.categories)
    colors[celltype]['colors'] = ad.uns[f'assigned_fine_grain_colors']
fig.savefig(f"{figdir}/entropyscore_fine-grain-assignment.pdf", bbox_inches="tight",dpi=300)
```

Example 39 (python):
```python
nrows = 3
ncols = 4
fig,axs=plt.subplots(nrows=nrows,ncols=ncols,figsize=(30,12))
for i,celltype in enumerate(celltypes):
    ad = gex_adata_dict[celltype].uns['nhood_adata']
    ad.obs['nhood'] = [f'{celltype}_nhood_{str(int(x)+1)}' for x in ad.obs.index]
    # add fine-grained labels
    ad.obs[celltype] = list(rna_nhoods_res[celltype]['assigned_fine_grain'].reindex(ad.obs['nhood']))
    sc.pl.embedding(ad,basis='X_milo_graph',size=ad.obs['Nhood_size']*7,
                    color=[celltype], # legend_loc='on data', 
                    legend_fontsize=12,
                    cmap='inferno', frameon=False,
                    show=False, ax=axs[i//ncols,i%ncols]
                   )
fig.tight_layout(w_pad=0.2)
```

Example 40 (python):
```python
nrows = 3
ncols = 4
fig,axs=plt.subplots(nrows=nrows,ncols=ncols,figsize=(20,12))
for i,celltype in enumerate(celltypes):
    ad = gex_adata_dict[celltype].uns['nhood_adata']
    ad.obs['nhood'] = [f'{celltype}_nhood_{str(int(x)+1)}' for x in ad.obs.index]
    # add entropy scores
    ad.obs[celltype] = list(rna_nhoods_res[celltype]['entropy score, residual'].reindex(ad.obs['nhood']))
    sc.pl.embedding(ad,basis='X_milo_graph',size=ad.obs['Nhood_size']*7,
                    color=[celltype],
                    cmap='inferno', frameon=False,
                    show=False, ax=axs[i//ncols,i%ncols]
                   )
fig.tight_layout()
```

Example 41 (python):
```python
# Prepare concatenated dataframe
for i,celltype in enumerate(celltypes):
    subset = rna_nhoods_res[celltype].copy()
    subset['mid_grain_mod'] = celltype
    if i==0:
        nhoods_res_concat = subset.copy()
    else:
        nhoods_res_concat = pd.concat([nhoods_res_concat,subset])
nhoods_res_concat['mid_grain_mod']=nhoods_res_concat['mid_grain_mod'].astype('category')
nhoods_res_concat['mid_grain_mod'].cat.reorder_categories(celltypes,inplace=True)
nhoods_res_concat=nhoods_res_concat.dropna(subset='assigned_fine_grain')

# Create a catplot with boxplots for each gene
g = sns.catplot(x='assigned_fine_grain', y='entropy score, selected cells, residual', col='mid_grain_mod', data=nhoods_res_concat, kind='box',
                col_wrap=3, sharey=False, sharex=False, height=6, aspect=0.8, palette="Blues")

# Overlay swarmplot on each Axes
# We'll use the 'hue' parameter to match the subsets and turn off dodge to align the points with the boxes.
for celltype, ax in zip(celltypes, g.axes.flat):
    subset = nhoods_res_concat[nhoods_res_concat['mid_grain_mod'] == celltype]
    sns.swarmplot(x='assigned_fine_grain', y='entropy score, selected cells, residual', data=subset, color='k', size=5, ax=ax, dodge=False)
    
# Update titles to remove " ="
for ax, celltype in zip(g.axes.flat, celltypes):
    ax.set_title(celltype)

# Rotate x-tick labels and remove x-axis labels
for i,ax in enumerate(g.axes.flat):
    ax.set_xticklabels(ax.get_xticklabels(), rotation=45, ha="right", rotation_mode="anchor")  # Rotate x-tick labels
    # ax.set_xticklabels('')
    ax.set_xlabel('')  # Remove x-axis labels
    if i!=0:
        ax.set_ylabel('')

# Adjust the spacing of the plots
g.fig.subplots_adjust(wspace=0.2, hspace=1.2)

# Show the plot
plt.show()
```

Example 42 (unknown):
```unknown
/tmp/ipykernel_60954/2274242884.py:10: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
  nhoods_res_concat['mid_grain_mod'].cat.reorder_categories(celltypes,inplace=True)
```

Example 43 (python):
```python
# Prepare concatenated dataframe
for i,celltype in enumerate(celltypes):
    subset = rna_nhoods_res[celltype].copy()
    subset['mid_grain_mod'] = celltype
    if i==0:
        nhoods_res_concat = subset.copy()
    else:
        nhoods_res_concat = pd.concat([nhoods_res_concat,subset])
nhoods_res_concat['mid_grain_mod']=nhoods_res_concat['mid_grain_mod'].astype('category')
nhoods_res_concat['mid_grain_mod'].cat.reorder_categories(celltypes,inplace=True)
nhoods_res_concat=nhoods_res_concat.dropna(subset='assigned_fine_grain')

# Create a catplot with boxplots for each gene
g = sns.catplot(x='assigned_fine_grain', y='purity_fine_grain', col='mid_grain_mod', data=nhoods_res_concat, kind='box',
                col_wrap=6, sharey=False, sharex=False, height=6, aspect=0.8, palette="Blues")

# Overlay swarmplot on each Axes
# We'll use the 'hue' parameter to match the subsets and turn off dodge to align the points with the boxes.
for celltype, ax in zip(celltypes, g.axes.flat):
    subset = nhoods_res_concat[nhoods_res_concat['mid_grain_mod'] == celltype]
    sns.swarmplot(x='assigned_fine_grain', y='purity_fine_grain', data=subset, color='k', size=5, ax=ax, dodge=False)
    
# Update titles to remove " ="
for ax, celltype in zip(g.axes.flat, celltypes):
    ax.set_title(celltype)

# Rotate x-tick labels and remove x-axis labels
for i,ax in enumerate(g.axes.flat):
    ax.set_xticklabels(ax.get_xticklabels(), rotation=45, ha="right", rotation_mode="anchor")  # Rotate x-tick labels
    # ax.set_xticklabels('')
    ax.set_xlabel('')  # Remove x-axis labels
    if i!=0:
        ax.set_ylabel('')

# Adjust the spacing of the plots
g.fig.subplots_adjust(wspace=0.2, hspace=1)

# Show the plot
plt.show()
```

Example 44 (unknown):
```unknown
/tmp/ipykernel_60954/3286563489.py:10: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
  nhoods_res_concat['mid_grain_mod'].cat.reorder_categories(celltypes,inplace=True)
```

Example 45 (python):
```python
import scipy.stats as stats
import scikit_posthocs as spost
```

Example 46 (unknown):
```unknown
celltype = 'AtrialCardiomyocytes_plusCCS'
assigned = rna_nhoods_res[celltype].copy()
assigned = assigned.dropna(subset='assigned_fine_grain')
set(assigned['assigned_fine_grain'])
```

Example 47 (unknown):
```unknown
{'AtrialCardiomyocytesCycling',
 'AtrialCardiomyocytesLeft',
 'AtrialCardiomyocytesRight',
 'AtrioventricularNodeCardiomyocytes',
 'SinoatrialNodeCardiomyocytes'}
```

Example 48 (python):
```python
celltype = 'AtrialCardiomyocytes_plusCCS'
map_celltypes = {
    'AtrialCardiomyocytesCycling':'AtrialWorkingCM',
     'AtrialCardiomyocytesLeft':'AtrialWorkingCM',
     'AtrialCardiomyocytesRight':'AtrialWorkingCM',
     'AtrioventricularNodeCardiomyocytes':'AtrialCCS',
     'SinoatrialNodeCardiomyocytes':'AtrialCCS'
}

assigned = rna_nhoods_res[celltype].copy()
assigned = assigned.dropna(subset='assigned_fine_grain')
assigned['mapped_celltype'] = assigned['assigned_fine_grain'].copy()
assigned.replace({'mapped_celltype':map_celltypes},inplace=True)

# Grouping data by SampleLabel
grouped_data = assigned.groupby('mapped_celltype')['entropy score, selected cells, residual'].apply(list)

# Kruskal-Wallis Test
stat, p_value = stats.kruskal(*grouped_data)
print("Kruskal-Wallis Test:")
print(f"Statistic: {stat}, P-value: {p_value}")
print("")

# also confirm with mann-whitney-u test (scipy)
stat, p_value = stats.mannwhitneyu(*grouped_data)
print("Mann-Whitney U rank Test:")
print(f"Statistic: {stat}, P-value: {p_value}")
```

Example 49 (unknown):
```unknown
Kruskal-Wallis Test:
Statistic: 10.054975332593926, P-value: 0.0015193705181662676

Mann-Whitney U rank Test:
Statistic: 1791.0, P-value: 0.0015293858581563156
```

Example 50 (unknown):
```unknown
celltype = 'VentricularCardiomyocytes_plusCCS'
assigned = rna_nhoods_res[celltype].copy()
assigned = assigned.dropna(subset='assigned_fine_grain')
set(assigned['assigned_fine_grain'])
```

Example 51 (unknown):
```unknown
{'VentricularCardiomyocytesCycling',
 'VentricularCardiomyocytesLeftCompact',
 'VentricularCardiomyocytesLeftTrabeculated',
 'VentricularCardiomyocytesRightCompact',
 'VentricularCardiomyocytesRightTrabeculated',
 'VentricularConductionSystemDistal',
 'VentricularConductionSystemProximal'}
```

Example 52 (python):
```python
celltype = 'VentricularCardiomyocytes_plusCCS'
map_celltypes = {
    'VentricularCardiomyocytesCycling':'VentricularWorkingCM',
     'VentricularCardiomyocytesLeftCompact':'VentricularWorkingCM',
     'VentricularCardiomyocytesLeftTrabeculated':'VentricularWorkingCM',
     'VentricularCardiomyocytesRightCompact':'VentricularWorkingCM',
     'VentricularCardiomyocytesRightTrabeculated':'VentricularWorkingCM',
     'VentricularConductionSystemDistal':'VentricularCCS',
     'VentricularConductionSystemProximal':'VentricularCCS'
}

assigned = rna_nhoods_res[celltype].copy()
assigned = assigned.dropna(subset='assigned_fine_grain')
assigned['mapped_celltype'] = assigned['assigned_fine_grain'].copy()
assigned.replace({'mapped_celltype':map_celltypes},inplace=True)

# Grouping data by SampleLabel
grouped_data = assigned.groupby('mapped_celltype')['entropy score, selected cells, residual'].apply(list)

# Kruskal-Wallis Test
stat, p_value = stats.kruskal(*grouped_data)
print("Kruskal-Wallis Test:")
print(f"Statistic: {stat}, P-value: {p_value}")
print("")

# also confirm with mann-whitney-u test (scipy)
stat, p_value = stats.mannwhitneyu(*grouped_data)
print("Mann-Whitney U rank Test:")
print(f"Statistic: {stat}, P-value: {p_value}")
```

Example 53 (unknown):
```unknown
Kruskal-Wallis Test:
Statistic: 39.17342310868321, P-value: 3.877830158632655e-10

Mann-Whitney U rank Test:
Statistic: 974.0, P-value: 3.922386003787531e-10
```

Example 54 (unknown):
```unknown
celltype = 'MyeloidCells'
assigned = rna_nhoods_res[celltype].copy()
assigned = assigned.dropna(subset='assigned_fine_grain')
set(assigned['assigned_fine_grain'])
```

Example 55 (unknown):
```unknown
{'DendriticCellsMature',
 'DendriticCellsType1',
 'MacrophagesATF3pos',
 'MacrophagesCX3CR1pos',
 'MacrophagesLYVE1pos',
 'MacrophagesTIMD4pos',
 'MastCells',
 'Megakaryocytes',
 'MonocyteDerivedCells',
 'Monocytes',
 'PlasmacytoidDendriticCells'}
```

Example 56 (python):
```python
celltype = 'MyeloidCells'
map_celltypes = {
    'Monocytes':'Mo,MP',
    'MonocyteDerivedCells':'Mo,MP',
    'MacrophagesCX3CR1pos':'Mo,MP',
    'MacrophagesLYVE1pos':'Mo,MP',
    'MacrophagesTIMD4pos':'Mo,MP',
    'MacrophagesATF3pos':'Mo,MP',
    'DendriticCellsType1':'DC',
    'DendriticCellsMature':'DC',
    'PlasmacytoidDendriticCells':'DC',
     'MastCells':'Mast',
     'Megakaryocytes':'Mega',
}

assigned = rna_nhoods_res[celltype].copy()
assigned = assigned.dropna(subset='assigned_fine_grain')
assigned['mapped_celltype'] = assigned['assigned_fine_grain'].copy()
assigned.replace({'mapped_celltype':map_celltypes},inplace=True)

# Grouping data by SampleLabel
grouped_data = assigned.groupby('mapped_celltype')['entropy score, selected cells, residual'].apply(list)

# Kruskal-Wallis Test
stat, p_value = stats.kruskal(*grouped_data)
print("Kruskal-Wallis Test:")
print(f"Statistic: {stat}, P-value: {p_value}")

# Perform Post Hoc analysis if Kruskal-Wallis test is significant
if p_value < 0.05:
    print("\nPerforming Post Hoc Analysis with Dunn's test...")

    # Post Hoc analysis using scikit-posthocs
    posthoc_results = spost.posthoc_dunn(assigned, 
                                         val_col='entropy score, selected cells, residual', 
                                         group_col='mapped_celltype', 
                                         p_adjust='bonferroni')

    # Print results
    print("Post Hoc Analysis (Dunn's test with Bonferroni correction):")
    display(pd.DataFrame(posthoc_results))
else:
    print("No significant differences found. No need for Post Hoc analysis.")
```

Example 57 (unknown):
```unknown
Kruskal-Wallis Test:
Statistic: 89.22165969298436, P-value: 3.219039117656211e-19

Performing Post Hoc Analysis with Dunn's test...
Post Hoc Analysis (Dunn's test with Bonferroni correction):
```

Example 58 (unknown):
```unknown
['AtrialCardiomyocytes_plusCCS',
 'VentricularCardiomyocytes_plusCCS',
 'Fibroblasts',
 'MuralCells',
 'PericardialCells',
 'EpicardialCells',
 'BloodVesselEndothelialCells',
 'EndocardialCells',
 'Neurons',
 'Glia',
 'MyeloidCells',
 'LymphoidCells']
```

Example 59 (python):
```python
cells_to_plot = ['AtrialCardiomyocytes_plusCCS','VentricularCardiomyocytes_plusCCS','MyeloidCells']

nrows = 1
ncols = 3
fig,axs=plt.subplots(nrows=nrows,ncols=ncols,figsize=(17,5))
for i,celltype in enumerate(cells_to_plot):
    ad = gex_adata_dict[celltype].uns['nhood_adata']
    ad.obs['nhood'] = [f'{celltype}_nhood_{str(int(x)+1)}' for x in ad.obs.index]
    # add entropy scores
    ad.obs[celltype] = list(rna_nhoods_res[celltype]['entropy score, residual'].reindex(ad.obs['nhood']))
    sc.pl.embedding(ad,basis='X_milo_graph',size=ad.obs['Nhood_size']*7,
                    color=[celltype],
                    cmap='inferno', frameon=False,
                    show=False, ax=axs[i]
                   )
fig.tight_layout()
fig.savefig(f"{figdir}/entropyscore_UMAPs_for-main.pdf",bbox_inches="tight",dpi=300)
```

Example 60 (python):
```python
celltype = 'AtrialCardiomyocytes_plusCCS'
relabel = {
    'AtrialCardiomyocytesRight':'Right',
    'AtrialCardiomyocytesLeft':'Left',
    'AtrialCardiomyocytesCycling':'Cycling',
    'SinoatrialNodeCardiomyocytes':'SAN_P',
     'AtrioventricularNodeCardiomyocytes':'AVN_P',
}
map_celltypes = {
    'AtrialCardiomyocytesRight':'WorkingCM',
    'AtrialCardiomyocytesLeft':'WorkingCM',
    'AtrialCardiomyocytesCycling':'WorkingCM',
    'SinoatrialNodeCardiomyocytes':'CCS',
     'AtrioventricularNodeCardiomyocytes':'CCS',
}
map_colors = {
    'WorkingCM':'lightsalmon',
     'CCS':'lightskyblue',
}

assigned = rna_nhoods_res[celltype].copy()
assigned = assigned.dropna(subset='assigned_fine_grain')
assigned['mapped_celltype'] = assigned['assigned_fine_grain'].copy()
assigned.replace({'mapped_celltype':map_celltypes},inplace=True)
assigned['relabeled'] = assigned['assigned_fine_grain'].copy()
assigned.replace({'relabeled':relabel},inplace=True)
assigned['relabeled'] = pd.Categorical(assigned['relabeled'],
                                              categories=relabel.values(),
                                              ordered=True)

# Create a catplot with boxplots
g = sns.catplot(x='relabeled', y='entropy score, selected cells, residual', 
                col=None, data=assigned, kind='box',
                col_wrap=None, sharey=False, sharex=False, height=4, aspect=0.9, 
                palette=[map_colors[x] for x in map_celltypes.values()],
               )
sns.swarmplot(x='relabeled', y='entropy score, selected cells, residual', 
              data=assigned, color='k', size=5,dodge=False)

# Create a list of patches for the legend
patches = [mpatches.Patch(color=color, label=label, linewidth=3) for label, color in map_colors.items()]
# Add legend to the plot
plt.legend(handles=patches, frameon=False)
plt.title('Atrial Cardiomyocytes', pad=50)
plt.xticks(rotation=45, ha="right", rotation_mode="anchor")  # Rotate x-tick labels
plt.xlabel('') # Remove x-axis labels
plt.ylabel('Entropy score')
plt.savefig(f"{figdir}/entropyscore_barplot_{celltype}.pdf", bbox_inches="tight",dpi=300)
plt.show()
```

Example 61 (python):
```python
celltype = 'VentricularCardiomyocytes_plusCCS'
relabel = {
    'VentricularCardiomyocytesRightTrabeculated':'RightTrab',
    'VentricularCardiomyocytesRightCompact':'RightComp',
    'VentricularCardiomyocytesLeftTrabeculated':'LeftTrab',
    'VentricularCardiomyocytesLeftCompact':'LeftComp',
    'VentricularCardiomyocytesCycling':'Cycling',
   'VentricularConductionSystemProximal':'VCSproximal',
     'VentricularConductionSystemDistal':'VCSdistal',
}
map_celltypes = {
    'VentricularCardiomyocytesRightTrabeculated':'WorkingCM',
    'VentricularCardiomyocytesRightCompact':'WorkingCM',
    'VentricularCardiomyocytesLeftTrabeculated':'WorkingCM',
    'VentricularCardiomyocytesLeftCompact':'WorkingCM',
    'VentricularCardiomyocytesCycling':'WorkingCM',
   'VentricularConductionSystemProximal':'CCS',
     'VentricularConductionSystemDistal':'CCS',
}
map_colors = {
    'WorkingCM':'lightsalmon',
     'CCS':'lightskyblue',
}

assigned = rna_nhoods_res[celltype].copy()
assigned = assigned.dropna(subset='assigned_fine_grain')
assigned['mapped_celltype'] = assigned['assigned_fine_grain'].copy()
assigned.replace({'mapped_celltype':map_celltypes},inplace=True)
assigned['relabeled'] = assigned['assigned_fine_grain'].copy()
assigned.replace({'relabeled':relabel},inplace=True)
assigned['relabeled'] = pd.Categorical(assigned['relabeled'],
                                              categories=relabel.values(),
                                              ordered=True)

# Create a catplot with boxplots
g = sns.catplot(x='relabeled', y='entropy score, selected cells, residual', 
                col=None, data=assigned, kind='box',
                col_wrap=None, sharey=False, sharex=False, height=4, aspect=1.1, 
                palette=[map_colors[x] for x in map_celltypes.values()],
               )
sns.swarmplot(x='relabeled', y='entropy score, selected cells, residual', 
              data=assigned, color='k', size=5,dodge=False)

# Create a list of patches for the legend
patches = [mpatches.Patch(color=color, label=label, linewidth=3) for label, color in map_colors.items()]
# Add legend to the plot
plt.legend(handles=patches, frameon=False)
plt.title('Ventricular Cardiomyocytes', pad=50)
plt.xticks(rotation=45, ha="right", rotation_mode="anchor")  # Rotate x-tick labels
plt.xlabel('') # Remove x-axis labels
plt.ylabel('Entropy score')
plt.savefig(f"{figdir}/entropyscore_barplot_{celltype}.pdf", bbox_inches="tight",dpi=300)
plt.show()
```

Example 62 (python):
```python
celltype = 'MyeloidCells'
relabel = {
    'Monocytes':'Mono',
    'MonocyteDerivedCells':'MoDerived',
    'MacrophagesCX3CR1pos':'MP_CX3CR1',
    'MacrophagesLYVE1pos':'MP_LYVE1',
    'MacrophagesTIMD4pos':'MP_TIMD4',
    'MacrophagesATF3pos':'MP_ATF3',
    'DendriticCellsType1':'DC1',
    'DendriticCellsMature':'DCmature',
    'PlasmacytoidDendriticCells':'pDC',
     'MastCells':'Mast',
     'Megakaryocytes':'Mega',
}
map_celltypes = {
    'Monocytes':'Mo,MP',
    'MonocyteDerivedCells':'Mo,MP',
    'MacrophagesCX3CR1pos':'Mo,MP',
    'MacrophagesLYVE1pos':'Mo,MP',
    'MacrophagesTIMD4pos':'Mo,MP',
    'MacrophagesATF3pos':'Mo,MP',
    'DendriticCellsType1':'DC',
    'DendriticCellsMature':'DC',
    'PlasmacytoidDendriticCells':'DC',
     'MastCells':'Mast',
     'Megakaryocytes':'Mega',
}
map_colors = {
    'Mo,MP':'lightsalmon',
     'DC':'lightskyblue',
    'Mast':'limegreen',
    'Mega':'plum',
}

assigned = rna_nhoods_res[celltype].copy()
assigned = assigned.dropna(subset='assigned_fine_grain')
assigned['mapped_celltype'] = assigned['assigned_fine_grain'].copy()
assigned.replace({'mapped_celltype':map_celltypes},inplace=True)
assigned['relabeled'] = assigned['assigned_fine_grain'].copy()
assigned.replace({'relabeled':relabel},inplace=True)
assigned['relabeled'] = pd.Categorical(assigned['relabeled'],
                                              categories=relabel.values(),
                                              ordered=True)

# Create a catplot with boxplots
g = sns.catplot(x='relabeled', y='entropy score, selected cells, residual', 
                col=None, data=assigned, kind='box',
                col_wrap=None, sharey=False, sharex=False, height=4, aspect=1.4, 
                palette=[map_colors[x] for x in map_celltypes.values()],
               )
sns.swarmplot(x='relabeled', y='entropy score, selected cells, residual', 
              data=assigned, color='k', size=5,dodge=False)

# Create a list of patches for the legend
patches = [mpatches.Patch(color=color, label=label, linewidth=3) for label, color in map_colors.items()]
# Add legend to the plot
plt.legend(handles=patches, frameon=False, bbox_to_anchor=(1.25, 1))
plt.title('Myeloid cells', pad=90)
plt.xticks(rotation=45, ha="right", rotation_mode="anchor")  # Rotate x-tick labels
plt.xlabel('') # Remove x-axis labels
plt.ylabel('Entropy score')
plt.savefig(f"{figdir}/entropyscore_barplot_{celltype}.pdf", bbox_inches="tight",dpi=300)
plt.show()
```

---

## Loading packages¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/Visium/Others/plotting_multiple_MP-vessel.html

**Contents:**
- Loading packages¶
- Read in adata¶
- Plotting¶

**Examples:**

Example 1 (python):
```python
import sys
import scanpy as sc
import anndata
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt 
import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 

import seaborn as sns

from cell2location.plt import plot_spatial

import os
```

Example 2 (python):
```python
# import a module given the full path?
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)

spec = importlib.util.spec_from_file_location("module.name", "/home/jovyan/projects/P46_Adult-heart_Visium/notebooks/ver8regions_torepo/_utils.py")
_utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = _utils
spec.loader.exec_module(_utils)
```

Example 3 (python):
```python
sc.settings.set_figure_params(dpi=80,dpi_save=300)
```

Example 4 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk'
```

Example 5 (python):
```python
adata_vis = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/visium-FFPE_foetal_raw.h5ad')
adata_vis
```

Example 6 (unknown):
```unknown
AnnData object with n_obs × n_vars = 19973 × 18085
    obs: 'in_tissue', 'array_row', 'array_col', 'sample', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'mt_frac', 'sangerID', 'Publication', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'status', 'JC_edit'
    var: 'gene_ids', 'feature_types', 'genome', 'SYMBOL', 'mt'
    uns: 'spatial'
    obsm: 'MT', 'means_cell_abundance_w_sf', 'q05_cell_abundance_w_sf', 'q95_cell_abundance_w_sf', 'spatial', 'stds_cell_abundance_w_sf'
```

Example 7 (python):
```python
# add cell2loc output to .obs
# get the cell2loc output
cell2loc_out = adata_vis.obsm['q05_cell_abundance_w_sf'].copy()
## read in cell type name update dictionary
celltype_rename = utils.read_pkl('/nfs/team205/heart/anndata_objects/Foetal/fine_grain_rename_dictionary.pkl')
## update cell type name
cellnames = adata_vis.obsm['q05_cell_abundance_w_sf'].columns
cellnames = [x.split('q05cell_abundance_w_sf_')[1] for x in cellnames]
cellnames = [celltype_rename[x] for x in cellnames]
## add
adata_vis.obs[cellnames] = cell2loc_out
```

Example 8 (unknown):
```unknown
AnnData object with n_obs × n_vars = 19973 × 18085
    obs: 'in_tissue', 'array_row', 'array_col', 'sample', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'mt_frac', 'sangerID', 'Publication', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'status', 'JC_edit', 'AtrialCardiomyocytesLeft', 'AtrialCardiomyocytesRight', 'VentricularCardiomyocytesLeftCompact', 'VentricularCardiomyocytesRightCompact', 'VentricularCardiomyocytesLeftTrabeculated', 'VentricularCardiomyocytesRightTrabeculated', 'SinoatrialNodeCardiomyocytes', 'AtrioventricularNodeCardiomyocytes', 'VentricularConductionSystemProximal', 'VentricularConductionSystemDistal', 'GreatVesselAdventitialFibroblasts', 'CoronaryVesselAdventitialFibroblasts', 'MyocardialInterstitialFibroblasts1', 'MyocardialInterstitialFibroblasts2', 'Myofibroblasts', 'LymphNodeFibroblasticReticularCells', 'ValveInterstitialCells', 'GreatVesselSmoothMuscleCells', 'CoronarySmoothMuscleCells', 'CoronaryPericytes', 'GreatVesselArterialEndothelialCells', 'GreatVesselVenousEndothelialCells', 'CoronaryArterialEndothelialCells', 'CoronaryVenousEndothelialCells', 'CoronaryCapillaryEndothelialCells', 'EndocardialCells', 'EndocardialCushionCells', 'ValveEndothelialCells', 'LymphaticEndothelialCells', 'MesothelialEpicardialCells', 'EpicardiumDerivedCells', 'NeuronPrecursors', 'ChromaffinCells', 'SympatheticNeurons', 'ParaympatheticNeurons', 'SchwannCellPrecursors', 'SchwannCells', 'MonocytesMPOpos', 'Monocytes', 'MonocyteDerivedCells', 'MacrophagesCX3CR1pos', 'MacrophagesTIMD4pos', 'MacrophagesLYVE1pos', 'MacrophagesATF3pos', 'DendriticCellsType1', 'DendriticCellsMature', 'PlasmacytoidDendriticCells', 'MastCells', 'Megakaryocytes', 'TCellsCD4pos', 'TCellsCD8pos', 'TregsCD4pos', 'ProBCells', 'BCells', 'BCellsMS4A1pos', 'NaturalKillerCells', 'InnateLymphoidCells', 'SinoatrialNodePacemakerCells', 'AtrioventricularNodePacemakerCells', 'MyocardialInterstitialFibroblasts', 'SubEpicardialFibroblasts', 'ParasympatheticNeurons'
    var: 'gene_ids', 'feature_types', 'genome', 'SYMBOL', 'mt'
    uns: 'spatial'
    obsm: 'MT', 'means_cell_abundance_w_sf', 'q05_cell_abundance_w_sf', 'q95_cell_abundance_w_sf', 'spatial', 'stds_cell_abundance_w_sf'
```

Example 9 (python):
```python
set(adata_vis.obs['sample'])
```

Example 10 (unknown):
```unknown
{'BHF_F_Hea13295830', 'BHF_F_Hea13295831'}
```

Example 11 (python):
```python
slide = _utils.select_multislide(adata_vis, ['BHF_F_Hea13295830'])
sc.pl.spatial(slide,cmap='RdPu',color='JC_edit', ncols=5, wspace=0,
              size=0,img_key='hires',alpha_img=1,vmin=0, vmax='p99.4',
             title='H&E', legend_loc=None)
```

Example 12 (unknown):
```unknown
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
```

Example 13 (unknown):
```unknown
?plot_spatial
```

Example 14 (python):
```python
Signature: plot_spatial(adata, color, img_key='hires', show_img=True, **kwargs)
Docstring:
Plot spatial abundance of cell types (regulatory programmes) with colour gradient
and interpolation (from Visium anndata).

This method supports only 7 cell types with these colours (in order, which can be changed using reorder_cmap).
'yellow' 'orange' 'blue' 'green' 'purple' 'grey' 'white'

:param adata: adata object with spatial coordinates in adata.obsm['spatial']
:param color: list of adata.obs column names to be plotted
:param kwargs: arguments to plot_spatial_general
:return: matplotlib figure
File:      ~/my-conda-envs/cellpymc/lib/python3.7/site-packages/cell2location/plt/plot_spatial.py
Type:      function
```

Example 15 (python):
```python
# select up to 6 clusters
# https://github.com/BayraktarLab/cell2location/blob/bd2aae833f81d8ab68684aa44f863b36e3af0aed/cell2location/plt/plot_spatial.py#L108

slide = _utils.select_multislide(adata_vis, ['BHF_F_Hea13295830'])

clust_labels = ['MacrophagesLYVE1pos',
                'CoronaryVenousEndothelialCells',
                'CoronarySmoothMuscleCells'
                ]
clust_col = ['' + str(i) for i in clust_labels] # in case column names differ from labels

plt.rcParams['axes.grid'] = False
with mpl.rc_context({'figure.figsize': (15, 15)}):
    fig = plot_spatial(
        adata=slide,
        # labels to show on a plot
        color=clust_col, labels=clust_labels,
        show_img=True, img_alpha=0.5,
        # 'fast' (white background) or 'dark_background'
        style='fast',
        # limit color scale at 99.2% quantile of cell abundance
        max_color_quantile=0.997,
        # size of locations (adjust depending on figure size)
        circle_diameter=3,
        colorbar_position='right',
        colorbar_shape = {'vertical_gaps': 0.5, 'horizontal_gaps': 0.3,
                                    'width': 0.2, 'height': 0.1},
        # crop_x=(650,820),
        # crop_y=(1200,1420),
        reorder_cmap = [3,4,1], # 'yellow' 'orange' 'blue' 'green' 'purple' 'grey' 'white'
    )
    plt.savefig(f"{figdir}/Visium_MP-Vessels_BHF_F_Hea13295830.pdf", bbox_inches="tight",dpi=300)
```

Example 16 (unknown):
```unknown
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
```

Example 17 (python):
```python
# select up to 6 clusters
# https://github.com/BayraktarLab/cell2location/blob/bd2aae833f81d8ab68684aa44f863b36e3af0aed/cell2location/plt/plot_spatial.py#L108

slide = _utils.select_multislide(adata_vis, ['BHF_F_Hea13295830'])

clust_labels = ['MacrophagesLYVE1pos',
                'CoronaryVenousEndothelialCells',
                'CoronarySmoothMuscleCells'
                ]
clust_col = ['' + str(i) for i in clust_labels] # in case column names differ from labels

plt.rcParams['axes.grid'] = False
with mpl.rc_context({'figure.figsize': (12, 10)}):
    fig = plot_spatial(
        adata=slide,
        # labels to show on a plot
        color=clust_col, labels=clust_labels,
        show_img=True, img_alpha=0.5,
        # 'fast' (white background) or 'dark_background'
        style='fast',
        # limit color scale at 99.2% quantile of cell abundance
        max_color_quantile=0.997,
        # size of locations (adjust depending on figure size)
        circle_diameter=30,
        colorbar_position='right',
        colorbar_shape = {'vertical_gaps': 0.5, 'horizontal_gaps': 0.4,
                                    'width': 0.1, 'height': 0.1},
        crop_x=(1500,1680),
        crop_y=(1280,1510),
        reorder_cmap = [3,4,1] # 'yellow' 'orange' 'blue' 'green' 'purple' 'grey' 'white'
    )
    plt.savefig(f"{figdir}/Visium_MP-Vessels_BHF_F_Hea13295830_zoomin1.pdf", bbox_inches="tight",dpi=300)
```

Example 18 (unknown):
```unknown
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
```

Example 19 (python):
```python
# select up to 6 clusters
# https://github.com/BayraktarLab/cell2location/blob/bd2aae833f81d8ab68684aa44f863b36e3af0aed/cell2location/plt/plot_spatial.py#L108

slide = _utils.select_multislide(adata_vis, ['BHF_F_Hea13295830'])

clust_labels = ['MacrophagesLYVE1pos',
                'CoronaryVenousEndothelialCells',
                'CoronarySmoothMuscleCells'
                ]
clust_col = ['' + str(i) for i in clust_labels] # in case column names differ from labels

plt.rcParams['axes.grid'] = False
with mpl.rc_context({'figure.figsize': (12, 10)}):
    fig = plot_spatial(
        adata=slide,
        # labels to show on a plot
        color=clust_col, labels=clust_labels,
        show_img=True, img_alpha=1,
        # 'fast' (white background) or 'dark_background'
        style='fast',
        # limit color scale at 99.2% quantile of cell abundance
        max_color_quantile=0.997,
        # size of locations (adjust depending on figure size)
        circle_diameter=0,
        colorbar_position=None,
        colorbar_shape = {'vertical_gaps': 0.5, 'horizontal_gaps': 0.4,
                                    'width': 0.1, 'height': 0.1},
        crop_x=(1500,1680),
        crop_y=(1280,1510),
        reorder_cmap = [3,4,1] # 'yellow' 'orange' 'blue' 'green' 'purple' 'grey' 'white'
    )
    plt.savefig(f"{figdir}/Visium_MP-Vessels_BHF_F_Hea13295830_zoomin1_HE.pdf", bbox_inches="tight",dpi=300)
```

Example 20 (unknown):
```unknown
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
```

Example 21 (python):
```python
# select up to 6 clusters
# https://github.com/BayraktarLab/cell2location/blob/bd2aae833f81d8ab68684aa44f863b36e3af0aed/cell2location/plt/plot_spatial.py#L108

slide = _utils.select_multislide(adata_vis, ['BHF_F_Hea13295830'])

clust_labels = ['MacrophagesLYVE1pos',
                'CoronaryVenousEndothelialCells',
                'CoronarySmoothMuscleCells'
                ]
clust_col = ['' + str(i) for i in clust_labels] # in case column names differ from labels

plt.rcParams['axes.grid'] = False
with mpl.rc_context({'figure.figsize': (12, 10)}):
    fig = plot_spatial(
        adata=slide,
        # labels to show on a plot
        color=clust_col, labels=clust_labels,
        show_img=True, img_alpha=0.5,
        # 'fast' (white background) or 'dark_background'
        style='fast',
        # limit color scale at 99.2% quantile of cell abundance
        max_color_quantile=0.997,
        # size of locations (adjust depending on figure size)
        circle_diameter=20,
        colorbar_position='right',
        colorbar_shape = {'vertical_gaps': 0.5, 'horizontal_gaps': 0.6,
                                    'width': 0.1, 'height': 0.1},
        crop_x=(90,380),
        crop_y=(1640,1790),
        reorder_cmap = [3,4,1] # 'yellow' 'orange' 'blue' 'green' 'purple' 'grey' 'white'
    )
    plt.savefig(f"{figdir}/Visium_MP-Vessels_BHF_F_Hea13295830_zoomin2.pdf", bbox_inches="tight",dpi=300)
```

Example 22 (unknown):
```unknown
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
```

Example 23 (python):
```python
# select up to 6 clusters
# https://github.com/BayraktarLab/cell2location/blob/bd2aae833f81d8ab68684aa44f863b36e3af0aed/cell2location/plt/plot_spatial.py#L108

slide = _utils.select_multislide(adata_vis, ['BHF_F_Hea13295830'])

clust_labels = ['MacrophagesLYVE1pos',
                'CoronaryVenousEndothelialCells',
                'CoronarySmoothMuscleCells'
                ]
clust_col = ['' + str(i) for i in clust_labels] # in case column names differ from labels

plt.rcParams['axes.grid'] = False
with mpl.rc_context({'figure.figsize': (12, 10)}):
    fig = plot_spatial(
        adata=slide,
        # labels to show on a plot
        color=clust_col, labels=clust_labels,
        show_img=True, img_alpha=1,
        # 'fast' (white background) or 'dark_background'
        style='fast',
        # limit color scale at 99.2% quantile of cell abundance
        max_color_quantile=0.997,
        # size of locations (adjust depending on figure size)
        circle_diameter=0,
        colorbar_position=None,
        colorbar_shape = {'vertical_gaps': 0.5, 'horizontal_gaps': 0.6,
                                    'width': 0.1, 'height': 0.1},
        crop_x=(90,380),
        crop_y=(1640,1790),
        reorder_cmap = [3,4,1] # 'yellow' 'orange' 'blue' 'green' 'purple' 'grey' 'white'
    )
    plt.savefig(f"{figdir}/Visium_MP-Vessels_BHF_F_Hea13295830_zoomin2_HE.pdf", bbox_inches="tight",dpi=300)
```

Example 24 (unknown):
```unknown
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
```

---

## 1. Read in inputs¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig3_SAN_niche/snMultime_no3_create-network_TF-Region-TG_IonChannels.html

**Contents:**
- 1. Read in inputs¶
- Read in and prepare anndata contains gene expression¶
- Read in peak signatures¶
- Read in scenicplus output metadata¶
- 2. DEG: SANCM vs other CMs¶
- 3. DEG: Foetal vs Adult¶
- Read in adult anndata and concatenate with foetal data¶
- Prepare anndata for pseudobulk wilcoxon test¶
- Pseudobulk wilcoxon test¶
- 4. Obtain expressing genes and accessible regions¶
- Genes¶
- Peaks¶
- Prepare scenic metadata¶
- Select interactions¶
- TF-TF network¶
- Save networks¶
- Plot network¶

What this notebook does

**Examples:**

Example 1 (python):
```python
# Parameters
working_dir = "/nfs/team205/heart/Vitro/TF_selection/foetal_SANPcell_vs_workingCM"
path_to_adata_gex = "/nfs/team205/heart/anndata_objects/Foetal/Feb28ObjectRaw_Multiome.h5ad"
path_to_adata_peak_tfidf = "/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/PeakMatrix/Foetal_Peaks_tfidf.h5ad"
path_to_DAR_1 = "/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/MarkerPeaks/SANP-vs-aCM.csv"
path_to_DAR_2 = "/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/MarkerPeaks/aCM-vs-SANP.csv"
path_to_p2g_ArchR = ""
group_key = "fine_grain_aCMs_merged"
group_a = "SinoatrialNodeCardiomyocytes"
group_b = "AtrialCardiomyocytes"
subset_object = True
geneset_list = ["MSigDB_Hallmark_2020", "KEGG_2021_Human", "GO_Biological_Process_2023"]

path_to_eRegulon_signatures_gene = "/nfs/team205/heart/pyscenic/plus/eRegulon_signatures/Cardiomyocytes_Gene_eRegulon_signatures.pkl"
path_to_eRegulon_signatures_region = "/nfs/team205/heart/pyscenic/plus/eRegulon_signatures/Cardiomyocytes_Region_eRegulon_signatures.pkl"

path_to_gene_signatures='/nfs/team205/heart/Vitro/TF_selection/foetal_SANPcell_vs_workingCM/outs/gene_signatures.pkl'
path_to_peak_signatures='/nfs/team205/heart/Vitro/TF_selection/foetal_SANPcell_vs_workingCM/outs/peak_signatures.pkl'

path_to_eRegulon_metadata = "/nfs/team205/heart/pyscenic/plus/eRegulon_metadata/Cardiomyocytes_eRegulon_metadata_filtered.csv" # filtered metadata

###### parameters for TF selection for the population ######
cell_of_intest = 'SinoatrialNodeCardiomyocytes'
groups_for_signatures = ['SinoatrialNodeCardiomyocytes']

# proportion threshold for expressing genes
exp_prop_thresh = 0.1

# proportion threshold for accesible regions
access_prop_thresh = 0.01
```

Example 2 (python):
```python
import scanpy as sc
import pandas as pd
import numpy as np
import anndata

import matplotlib.pyplot as plt
import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 
import seaborn as sns

# import hotspot
```

Example 3 (python):
```python
import session_info
session_info.show()
```

Example 4 (python):
```python
-----
anndata             0.9.2
matplotlib          3.7.2
numpy               1.24.4
pandas              1.5.3
scanpy              1.9.3
seaborn             0.12.2
session_info        1.0.0
-----
```

Example 5 (python):
```python
PIL                         10.0.0
anyio                       NA
arrow                       1.2.3
asttokens                   NA
attr                        23.1.0
attrs                       23.1.0
babel                       2.12.1
backcall                    0.2.0
brotli                      1.0.9
certifi                     2023.11.17
cffi                        1.15.1
charset_normalizer          3.2.0
colorama                    0.4.6
comm                        0.1.4
cycler                      0.10.0
cython_runtime              NA
dateutil                    2.8.2
debugpy                     1.6.8
decorator                   5.1.1
defusedxml                  0.7.1
executing                   1.2.0
fastjsonschema              NA
fqdn                        NA
h5py                        3.9.0
idna                        3.4
igraph                      0.10.6
importlib_metadata          NA
importlib_resources         NA
ipykernel                   6.25.1
isoduration                 NA
jedi                        0.19.0
jinja2                      3.1.2
joblib                      1.3.2
json5                       NA
jsonpointer                 2.0
jsonschema                  4.19.0
jsonschema_specifications   NA
jupyter_events              0.7.0
jupyter_server              2.7.1
jupyterlab_server           2.24.0
kiwisolver                  1.4.5
leidenalg                   0.10.1
llvmlite                    0.40.1
markupsafe                  2.1.3
mpl_toolkits                NA
natsort                     8.4.0
nbformat                    5.9.2
numba                       0.57.1
overrides                   NA
packaging                   23.1
parso                       0.8.3
patsy                       0.5.3
pexpect                     4.8.0
pickleshare                 0.7.5
pkg_resources               NA
plotly                      5.16.1
prometheus_client           NA
prompt_toolkit              3.0.39
psutil                      5.9.5
ptyprocess                  0.7.0
pure_eval                   0.2.2
pydev_ipython               NA
pydevconsole                NA
pydevd                      2.9.5
pydevd_file_utils           NA
pydevd_plugins              NA
pydevd_tracing              NA
pygments                    2.16.1
pyparsing                   3.0.9
pythonjsonlogger            NA
pytz                        2023.3
referencing                 NA
requests                    2.31.0
rfc3339_validator           0.1.4
rfc3986_validator           0.1.1
rpds                        NA
ruamel                      NA
scipy                       1.10.1
send2trash                  NA
six                         1.16.0
sklearn                     1.3.0
sniffio                     1.3.0
socks                       1.7.1
sphinxcontrib               NA
stack_data                  0.6.2
statsmodels                 0.14.0
texttable                   1.6.7
threadpoolctl               3.2.0
tornado                     6.3.3
traitlets                   5.9.0
typing_extensions           NA
uri_template                NA
urllib3                     2.0.4
wcwidth                     0.2.6
webcolors                   1.13
websocket                   1.6.2
yaml                        6.0.1
zipp                        NA
zmq                         25.1.1
zstandard                   0.19.0
```

Example 6 (unknown):
```unknown
-----
IPython             8.12.0
jupyter_client      8.3.0
jupyter_core        4.12.0
jupyterlab          4.0.5
notebook            7.0.2
-----
Python 3.8.17 (default, Jul  5 2023, 21:04:15) [GCC 11.2.0]
Linux-5.4.0-169-generic-x86_64-with-glibc2.17
-----
Session information updated at 2024-03-25 16:32
```

Example 7 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)

spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/regulon_plot.py")
regulon_plot = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = regulon_plot
spec.loader.exec_module(regulon_plot)
```

Example 8 (python):
```python
sc.settings.set_figure_params(dpi=80,dpi_save=300)
```

Example 9 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk'
```

Example 10 (python):
```python
# read in adata
# whole data
adata = sc.read_h5ad(path_to_adata_gex)

# filter and log-normalise
sc.pp.filter_genes(adata, min_cells=3)
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

print(adata.X.data[:10])
adata
```

Example 11 (unknown):
```unknown
[0.41374123 0.22811745 0.22811745 0.41374123 0.22811745 0.41374123
 0.22811745 0.41374123 0.22811745 0.22811745]
```

Example 12 (unknown):
```unknown
AnnData object with n_obs × n_vars = 211145 × 34377
    obs: 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score', 'fine_grain_aCMs_merged', 'fine_grain_aCMs-node_merged'
    var: 'gene_id', 'n_cells'
    uns: 'log1p'
```

Example 13 (unknown):
```unknown
peak_signatures = utils.read_pkl(path_to_peak_signatures)
peak_signatures.keys()
```

Example 14 (unknown):
```unknown
dict_keys(['SinoatrialNodeCardiomyocytes_up', 'SinoatrialNodeCardiomyocytes_down'])
```

Example 15 (python):
```python
sig_peaks = peak_signatures['SinoatrialNodeCardiomyocytes_up']
print(len(sig_peaks))
sig_peaks[:5]
```

Example 16 (unknown):
```unknown
['chr1:55555893_55556393',
 'chr12:115794713_115795213',
 'chr3:113167147_113167647',
 'chr17:65228782_65229282',
 'chr11:116485972_116486472']
```

Example 17 (python):
```python
# scenicplus output
scenic_meta = pd.read_csv(path_to_eRegulon_metadata,index_col=0)
scenic_meta['Region'] = [x.replace('-','_') for x in scenic_meta['Region']]

# select active regulons
scenic_meta = scenic_meta[['_+_' in x for x in scenic_meta['Gene_signature_name']]]
scenic_meta.head()
```

Example 18 (unknown):
```unknown
# select based on R2G_rho
rho_thresh = 0.2
scenic_meta = scenic_meta[scenic_meta['R2G_rho']>rho_thresh]
```

Example 19 (python):
```python
logFC_threshold = 1
group_mean_threshold = 0.1

adata_sub = adata[adata.obs['coarse_grain']=='Cardiomyocytes']
adata_sub = adata_sub[adata_sub.obs['kit_10x']=='Multiome-v1']

res = utils.rank_genes_groups_with_mean(adata_sub,
                                 groupby = 'fine_grain',
                                  groups = ['SinoatrialNodeCardiomyocytes'],
                                  reference = 'rest',
                                  method = 't-test',
                                  pval_cutoff = 0.05,
                                        log2fc_min=logFC_threshold
                                 )
sns.scatterplot(x=res['logfoldchanges'],y=res['group_mean'])
plt.show()
```

Example 20 (python):
```python
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/tools/_rank_genes_groups.py:580: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  adata.uns[key_added] = {}
```

Example 21 (unknown):
```unknown
# filter genes based on the group mean expression
res = res[res['group_mean']>group_mean_threshold]
display(res.head())
degs_sancm_vs_others = list(res['names'])
len(degs_sancm_vs_others)
```

Example 22 (unknown):
```unknown
sig_genes = degs_sancm_vs_others.copy()
```

Example 23 (unknown):
```unknown
'ISL1' in sig_genes
```

Example 24 (python):
```python
# prepare adult anndata
adata_adult = sc.read_h5ad('/nfs/team205/heart/anndata_objects/8regions/RNA_adult-8reg_full_raw_cellstate-annotated.h5ad')

# subset SAN_P_cell
adata_adult = adata_adult[adata_adult.obs['cell_state']=='SAN_P_cell']
adata_adult = adata_adult[adata_adult.obs['kit_10x']=='Multiome-v1']

# log normalise
sc.pp.normalize_total(adata_adult, target_sum=1e4)
sc.pp.log1p(adata_adult)
print(adata_adult.X.data[:10])

# rename
adata_adult.var['gene_id'] =  adata_adult.var_names.copy()
adata_adult.var.set_index('gene_name-new',inplace=True)
adata_adult.var_names = adata_adult.var_names.astype('str')
adata_adult.var_names_make_unique()

adata_adult
```

Example 25 (unknown):
```unknown
[0.911473   0.31626672 0.31626672 0.31626672 0.74952203 0.911473
 0.31626672 0.31626672 0.31626672 0.31626672]
```

Example 26 (python):
```python
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/preprocessing/_normalization.py:170: UserWarning: Received a view of an AnnData. Making a copy.
  view_to_actual(adata)
```

Example 27 (unknown):
```unknown
AnnData object with n_obs × n_vars = 245 × 32732
    obs: 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'n_genes', 'n_genes_by_counts', 'total_counts', 'total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'pct_counts_ribo', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'batch_key', 'leiden_scVI', 'cell_type', 'cell_state_HCAv1', 'cell_state_scNym', 'cell_state_scNym_confidence', 'cell_state', 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'n_counts', '_scvi_batch', '_scvi_labels', 'clus20', 'doublet_cls', 'original_or_new', 'batch', 'scANVI_predictions', 'leiden_scArches'
    var: 'gene_name_scRNA-0-original', 'gene_name_snRNA-1-original', 'gene_name_multiome-2-original', 'gene_id'
    uns: 'age_colors', 'cell_or_nuclei_colors', 'cell_state_colors', 'cell_type_colors', 'donor_colors', 'donor_type_colors', 'facility_colors', 'flushed_colors', 'gender_colors', 'kit_10x_colors', 'leiden', 'modality_colors', 'neighbors', 'original_or_new_colors', 'region_colors', 'region_finest_colors', 'scANVI_predictions_colors', 'umap', 'log1p'
    obsm: 'X_scArches', 'X_umap'
    obsp: 'connectivities', 'distances'
```

Example 28 (python):
```python
# concatenate
## shared features
shared = list(set(adata_sub.var_names).intersection(adata_adult.var_names))

adata_sub.obs['stage'] = 'foetal'
adata_adult.obs['stage'] = 'adult'
adata_adult.obs['week'] = 'adult'
adata_adult.obs['fine_grain'] = 'adult - SAN_P_cell'

adata_conc = adata_sub[:,shared].concatenate(adata_adult[:,shared],
                                             join='inner',
                                             batch_key=None,
                                             index_unique=None
                                            )
```

Example 29 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/anndata/_core/anndata.py:1763: FutureWarning: The AnnData.concatenate method is deprecated in favour of the anndata.concat function. Please use anndata.concat instead.

See the tutorial for concat at: https://anndata.readthedocs.io/en/latest/concatenation.html
  warnings.warn(
```

Example 30 (python):
```python
# combine donor and stage key
adata_conc.obs['donor_new'] = adata_conc.obs['stage'].astype('str')+'_'+adata_conc.obs['donor'].astype('str')
set(adata_conc.obs['donor_new'])
```

Example 31 (unknown):
```unknown
{'adult_A61',
 'adult_AH1',
 'adult_AH2',
 'adult_AV10',
 'adult_AV14',
 'adult_AV3',
 'foetal_C104',
 'foetal_C82',
 'foetal_C85',
 'foetal_C87',
 'foetal_C92',
 'foetal_C98',
 'foetal_Hst33',
 'foetal_Hst36',
 'foetal_Hst39',
 'foetal_Hst40',
 'foetal_Hst41'}
```

Example 32 (python):
```python
# add drimester 
donor_trimester_df = adata_conc.obs[['donor','trimester']].copy()
donor_trimester_df = donor_trimester_df.drop_duplicates()
donor_trimester_df['trimester'] = [x if x==x else 'adult' for x in donor_trimester_df['trimester']]
donor_trimester_df
```

Example 33 (python):
```python
groupby = 'donor_new'
celltypes = ['SinoatrialNodeCardiomyocytes','adult - SAN_P_cell']
n_cell_thresh = 10

# subset anndata
ad = adata_conc[adata_conc.obs['fine_grain'].isin(celltypes)]
# remove group which only has less than a threshold
vc = ad.obs[groupby].value_counts()
grp_to_drop = list(vc.index[vc < 10])
ad = ad[~ad.obs[groupby].isin(grp_to_drop)]
print(f'dropped groups: {grp_to_drop}')

# mean expression per group
exp_mean = utils.grouped_obs_mean(ad, group_key=groupby)

# make anndata
import anndata
adata_pb = anndata.AnnData(
    X=exp_mean.T,
    obs=pd.DataFrame(exp_mean.columns).set_index(0),
    var=pd.DataFrame(exp_mean.index).set_index(0)
)

# add stage key
adata_pb.obs['stage'] = [x.split('_')[0] for x in adata_pb.obs_names]
adata_pb.obs['stage'] = adata_pb.obs['stage'].astype('category')
adata_pb.obs['stage'].cat.reorder_categories(['foetal','adult'],inplace=True)

# add donor and trimester key
adata_pb.obs['donor'] = [x.split('_')[1] for x in adata_pb.obs_names]
obs = adata_pb.obs.reset_index().merge(donor_trimester_df,how='left',on='donor').set_index(0)
obs.index.name = 'index'
adata_pb.obs = obs
adata_pb.obs['trimester'] = adata_pb.obs['trimester'].astype('category')
adata_pb.obs['trimester'].cat.reorder_categories(['firsttrimester','secondtrimester','adult'],inplace=True)


print(adata_pb.obs['stage'].value_counts())
print(adata_pb)
```

Example 34 (unknown):
```unknown
dropped groups: ['foetal_C82', 'foetal_Hst36']
foetal    8
adult     6
Name: stage, dtype: int64
AnnData object with n_obs × n_vars = 14 × 30749
    obs: 'stage', 'donor', 'trimester'
```

Example 35 (python):
```python
/tmp/ipykernel_59443/3068665136.py:27: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
  adata_pb.obs['stage'].cat.reorder_categories(['foetal','adult'],inplace=True)
/tmp/ipykernel_59443/3068665136.py:35: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
  adata_pb.obs['trimester'].cat.reorder_categories(['firsttrimester','secondtrimester','adult'],inplace=True)
```

Example 36 (python):
```python
# rank test
logFC_threshold_FvsA = 1
group_mean_threshold = 0.1

res_foetal_vs_adult = utils.rank_genes_groups_with_mean(adata_pb,
                                 groupby = 'stage',
                                  groups = ['foetal'],
                                  reference = 'adult',
                                  method = 'wilcoxon',
                                  pval_cutoff = 0.1,
                                        log2fc_min=logFC_threshold_FvsA
                                 )

sns.scatterplot(x=res_foetal_vs_adult['logfoldchanges'],y=res_foetal_vs_adult['group_mean'])
plt.show()
```

Example 37 (python):
```python
res_foetal_vs_adult = res_foetal_vs_adult[res_foetal_vs_adult['group_mean']>group_mean_threshold]
degs_foetal_vs_adult = list(res_foetal_vs_adult['names'])
print(len(degs_foetal_vs_adult))
display(res_foetal_vs_adult.head())
```

Example 38 (unknown):
```unknown
res_foetal_vs_adult[res_foetal_vs_adult['names']=='CACNA1D']
```

Example 39 (unknown):
```unknown
len(degs_foetal_vs_adult)
```

Example 40 (python):
```python
# get expression proportions
exp_mat = utils.get_exp_proportion(adata,groupby='fine_grain', use_rep='X')
exp_genes = list(exp_mat.index[exp_mat[cell_of_intest]>exp_prop_thresh])
len(exp_genes)
```

Example 41 (unknown):
```unknown
'ISL1' in exp_genes
```

Example 42 (python):
```python
adata_peak = sc.read_h5ad(path_to_adata_peak_tfidf)
adata_peak
```

Example 43 (unknown):
```unknown
AnnData object with n_obs × n_vars = 167022 × 508040
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels'
    var: 'seqnames', 'start', 'end', 'width', 'strand', 'score', 'replicateScoreQuantile', 'groupScoreQuantile', 'Reproducibility', 'GroupReplicate', 'distToGeneStart', 'nearestGene', 'peakType', 'distToTSS', 'nearestTSS', 'GC', 'idx', 'N'
```

Example 44 (python):
```python
# get accessible proportions
access_mat = utils.get_exp_proportion(adata_peak,groupby='fine_grain', use_rep='X')
```

Example 45 (python):
```python
y = np.array([])
y = np.append(y,1)
y
```

Example 46 (unknown):
```unknown
array([1.])
```

Example 47 (python):
```python
x = np.linspace(0, 1, num=200)
y = np.array([])
for thresh in np.linspace(0, 1, num=200):
    y = np.append(y,sum(access_mat[cell_of_intest]>thresh))
y
```

Example 48 (unknown):
```unknown
array([4.54129e+05, 2.83575e+05, 1.83387e+05, 1.33237e+05, 1.08954e+05,
       8.68990e+04, 7.14970e+04, 6.31020e+04, 5.42520e+04, 4.73650e+04,
       4.19410e+04, 3.84920e+04, 3.46050e+04, 3.13530e+04, 2.92080e+04,
       2.66460e+04, 2.44760e+04, 2.24840e+04, 2.11130e+04, 1.95230e+04,
       1.81140e+04, 1.71810e+04, 1.59230e+04, 1.48690e+04, 1.39060e+04,
       1.32050e+04, 1.24110e+04, 1.16150e+04, 1.11050e+04, 1.04250e+04,
       9.75700e+03, 9.29600e+03, 8.66600e+03, 8.09900e+03, 7.58000e+03,
       7.23600e+03, 6.75900e+03, 6.33300e+03, 6.01400e+03, 5.58700e+03,
       5.19000e+03, 4.79500e+03, 4.52400e+03, 4.19800e+03, 3.91200e+03,
       3.67600e+03, 3.37500e+03, 3.10800e+03, 2.86700e+03, 2.67500e+03,
       2.45100e+03, 2.23600e+03, 2.09100e+03, 1.92000e+03, 1.74200e+03,
       1.63200e+03, 1.45200e+03, 1.32700e+03, 1.19600e+03, 1.08900e+03,
       9.63000e+02, 8.50000e+02, 7.88000e+02, 6.94000e+02, 6.04000e+02,
       5.33000e+02, 4.77000e+02, 4.09000e+02, 3.53000e+02, 3.20000e+02,
       2.65000e+02, 2.26000e+02, 2.02000e+02, 1.74000e+02, 1.28000e+02,
       1.05000e+02, 9.60000e+01, 7.80000e+01, 5.70000e+01, 4.80000e+01,
       3.60000e+01, 2.70000e+01, 2.50000e+01, 1.90000e+01, 1.60000e+01,
       1.50000e+01, 1.30000e+01, 9.00000e+00, 7.00000e+00, 7.00000e+00,
       6.00000e+00, 3.00000e+00, 3.00000e+00, 1.00000e+00, 1.00000e+00,
       1.00000e+00, 1.00000e+00, 1.00000e+00, 1.00000e+00, 1.00000e+00,
       1.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00,
       0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00,
       0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00,
       0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00,
       0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00,
       0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00,
       0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00,
       0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00,
       0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00,
       0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00,
       0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00,
       0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00,
       0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00,
       0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00,
       0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00,
       0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00,
       0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00,
       0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00,
       0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00,
       0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00])
```

Example 49 (python):
```python
plt.rcParams['figure.figsize'] = 10,10
plt.scatter(x,y)
plt.show()
```

Example 50 (unknown):
```unknown
thresh_list = [0,0.01,0.02]
access_mat[cell_of_intest]>access_prop_thresh
```

Example 51 (unknown):
```unknown
chr1:817093_817593          False
chr1:819818_820318          False
chr1:820542_821042          False
chr1:825246_825746          False
chr1:826566_827066           True
                            ...  
chrX:155820072_155820572    False
chrX:155874490_155874990    False
chrX:155880548_155881048     True
chrX:155881064_155881564     True
chrX:155881618_155882118     True
Name: SinoatrialNodeCardiomyocytes, Length: 508040, dtype: bool
```

Example 52 (unknown):
```unknown
access_peaks = list(access_mat.index[access_mat[cell_of_intest]>access_prop_thresh])
len(access_peaks)
```

Example 53 (python):
```python
# add columns to scenicplus metadata
# scenic_meta['Gene_isTF'] = scenic_meta['Gene'].isin(list(set(scenic_meta['TF'])))
scenic_meta['TF_isExp'] = scenic_meta['TF'].isin(exp_genes)
scenic_meta['TF_isSignature'] = (scenic_meta['TF'].isin(sig_genes))&(scenic_meta['TF'].isin(exp_genes))
scenic_meta['Region_isAccess'] = scenic_meta['Region'].isin(access_peaks)
scenic_meta['Region_isSignature'] = (scenic_meta['Region'].isin(sig_peaks))&(scenic_meta['Region'].isin(access_peaks))
scenic_meta.head()
```

Example 54 (python):
```python
ionchannel = list(pd.read_csv('/nfs/team205/kk18/data/geneset/HGNC/HGNC_GID177_Ion-channels.txt',sep='\t')['Approved symbol'].unique())
# gpcr = list(pd.read_csv('/nfs/team205/kk18/data/geneset/HGNC/HGNC_GID139_G-protein-coupled-receptors.txt',sep='\t')['Approved symbol'].unique())
```

Example 55 (unknown):
```unknown
target_genes = list(set(sig_genes).intersection(ionchannel))
target_genes.sort()
target_genes
```

Example 56 (unknown):
```unknown
['CACNA1D',
 'CACNA1G',
 'CACNA2D2',
 'CACNA2D3',
 'CLIC5',
 'GABRB2',
 'GRIA3',
 'GRID2',
 'GRIK1',
 'HCN1',
 'HCN4',
 'KCNJ3',
 'KCNK1',
 'KCNN2',
 'KCNT2',
 'LRRC8C',
 'RYR3',
 'TRPC5',
 'TRPM3']
```

Example 57 (python):
```python
# select interactions

# select interaction which target "target_genes"
scenic_meta_sel = scenic_meta[(scenic_meta['Gene'].isin(target_genes))&\
                               scenic_meta['TF_isExp'] &\
                               scenic_meta['Region_isAccess'] &\
                              (scenic_meta['TF_isSignature']|scenic_meta['Region_isSignature'])
                             ]

# get TFs and target genes in the selected interaction
direct_TFs = list(set(scenic_meta_sel['TF']))
targets = list(set(scenic_meta_sel['Gene']))
# remove TFs already in target genes
TFs_sel = list(set(direct_TFs) - set(targets))

print(len(TFs_sel))
print(f'direct TFs: {direct_TFs}')
print('')

# select higher level interactions 
print('Start looking for higher level TFs')
higher_level_TFs = []
while len(TFs_sel)>0:
    scenic_meta_sel2 = scenic_meta[(scenic_meta['Gene'].isin(TFs_sel))&\
                               scenic_meta['TF_isExp'] &\
                               scenic_meta['Region_isAccess'] &\
                              (scenic_meta['TF_isSignature']|scenic_meta['Region_isSignature'])
                                  ]
    higher_level_TFs = higher_level_TFs + list(set(scenic_meta_sel2['TF']))
    if len(scenic_meta_sel2)>0:
        # concatenate
        scenic_meta_sel = pd.concat([scenic_meta_sel,scenic_meta_sel2])
        scenic_meta_sel = scenic_meta_sel.drop_duplicates()
        # get TFs and target genes in the selected interaction
        TFs_sel = list(set(scenic_meta_sel['TF']))
        targets = list(set(scenic_meta_sel['Gene']))
        # remove TFs already in target genes
        TFs_sel = list(set(TFs_sel) - set(targets))
        print(f'{len(TFs_sel)}: {TFs_sel}')
    else:
        print('no more higher level TFs')
        TFs_sel = []

scenic_meta_sel['direct_TF'] = scenic_meta_sel['TF'].isin(direct_TFs)
higher_level_TFs = list(set(higher_level_TFs)-set(direct_TFs))

# reindex
scenic_meta_sel.index = range(len(scenic_meta_sel))

print("")
print(f'Number of selected interactions: {len(scenic_meta_sel)}')
print(f"Number of TFs: {len(set(scenic_meta_sel['TF']))}")
```

Example 58 (unknown):
```unknown
29
direct TFs: ['RARB', 'FOS', 'MEF2C', 'NFIB', 'PKNOX2', 'MECP2', 'HMGA2', 'MEIS1', 'ZNF138', 'ISL1', 'NFIC', 'TEAD1', 'NR2F1', 'TBX3', 'SMAD9', 'TRPS1', 'TBX5', 'CREM', 'NR3C1', 'NFIA', 'NR2F2', 'TBX18', 'TBX20', 'SMAD3', 'ZNF536', 'POU6F2', 'THRB', 'SP1', 'SHOX2']

Start looking for higher level TFs
19: ['SP4', 'NFIB', 'BHLHE40', 'MECP2', 'MAX', 'ZNF138', 'TEAD1', 'FOXO1', 'CREM', 'GLIS3', 'NR3C1', 'ZNF529', 'JUN', 'ETV6', 'KLF13', 'ZNF41', 'SP1', 'TEAD3', 'MITF']
15: ['MECP2', 'MAX', 'CREM', 'ZNF138', 'ZNF41', 'SP4', 'TEAD1', 'NFIB', 'NR3C1', 'ZNF529', 'BHLHE40', 'JUN', 'KLF13', 'SP1', 'TEAD3']
no more higher level TFs

Number of selected interactions: 357
Number of TFs: 41
```

Example 59 (python):
```python
# get TF-TG dictionary
network = {}
for tf in set(scenic_meta_sel['TF']):
    df_sub = scenic_meta_sel[scenic_meta_sel['TF']==tf]
    network[tf] = list(set(df_sub['Gene']))
print(len(network.keys()))
```

Example 60 (python):
```python
# add metrics
scenic_meta_sel['TF_n_linked_region'] = np.nan
scenic_meta_sel['TF_n_target_genes'] = np.nan
scenic_meta_sel['TF_n_target_sig_regions'] = np.nan

for tf in set(scenic_meta_sel['TF']):
    tf_ind = scenic_meta_sel[scenic_meta_sel['TF']==tf].index
    targets = list(set(scenic_meta_sel.loc[tf_ind,'Gene']))
    scenic_meta_sel.loc[tf_ind,'TF_n_target_genes'] = len(targets)
    regions = list(set(scenic_meta_sel.loc[tf_ind,'Region']))
    regions = list(set(regions).intersection(sig_peaks))
    scenic_meta_sel.loc[tf_ind,'TF_n_target_sig_regions'] = len(regions)
    if tf in list(set(scenic_meta_sel['Gene'])):
        gene_ind = scenic_meta_sel[scenic_meta_sel['Gene']==tf].index
        n_region = len(set(scenic_meta_sel.loc[gene_ind,'Region']))
        scenic_meta_sel.loc[tf_ind,'TF_n_linked_region'] = n_region
    else:
        scenic_meta_sel.loc[tf_ind,'TF_n_linked_region'] = 0
        
scenic_meta_sel.head(3)
```

Example 61 (unknown):
```unknown
scenic_meta_sel_TF_metrices = scenic_meta_sel[['TF','direct_TF','TF_n_target_genes','TF_n_target_sig_regions','TF_n_linked_region']]
# df = df[df['direct_TF']]
scenic_meta_sel_TF_metrices.drop_duplicates(inplace=True)
scenic_meta_sel_TF_metrices.set_index('TF',inplace=True)
scenic_meta_sel_TF_metrices.sort_values('TF_n_target_genes',ascending=False)[:30]
```

Example 62 (python):
```python
/tmp/ipykernel_59443/2002270316.py:3: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  scenic_meta_sel_TF_metrices.drop_duplicates(inplace=True)
```

Example 63 (python):
```python
# select interactions those targets are TFs
scenic_meta_sel_TF = scenic_meta_sel[scenic_meta_sel['Gene'].isin(list(set(scenic_meta_sel['TF'])))]

network_TF = {}
for tf in set(scenic_meta_sel['TF']): # for all the TFs in the network
    if tf in set(scenic_meta_sel_TF['TF']): # whether the TF has TF target
        df_sub = scenic_meta_sel_TF[scenic_meta_sel_TF['TF']==tf]
        network_TF[tf] = list(set(df_sub['Gene']))
    else:
        network_TF[tf] = []
print(len(network_TF.keys()))
```

Example 64 (unknown):
```unknown
# calculate centrality
centrality_type='eigenvector'
centrality_dict = regulon_plot.get_centrality(network_TF,
                   directed=True,
                   centrality_type=centrality_type, # or 'degree','in_dgree','out_degree'
                   out_df=False # if False output dictionary
                  )
centrality_dict
```

Example 65 (unknown):
```unknown
{'RARB': 0.07132633275333206,
 'THRB': 0.35390860130150104,
 'FOS': 1.3629970460658244e-09,
 'TBX18': 0.3101423014476137,
 'GLIS3': 0.04934281480150286,
 'SHOX2': 0.6002172132880007,
 'SP4': 2.5997105523411263e-15,
 'TBX3': 0.15969133415071213,
 'MEF2C': 0.18247366637712445,
 'TBX5': 0.17125320995089946,
 'NFIA': 0.1040669178038443,
 'MEIS1': 0.1132634510375253,
 'NR2F2': 0.17125320995089946,
 'HMGA2': 0.04213053122609575,
 'NR2F1': 0.17125320995089946,
 'PKNOX2': 0.2615518061778349,
 'ZNF536': 0.1257520531317977,
 'POU6F2': 0.10722242062882631,
 'MITF': 0.05352293965092095,
 'NFIB': 2.5997105523411263e-15,
 'SMAD9': 0.031239281230728118,
 'ETV6': 0.18163891467430646,
 'TRPS1': 0.12645377861738516,
 'BHLHE40': 2.5997105523411263e-15,
 'MAX': 2.5997105523411263e-15,
 'ISL1': 0.28192058047477336,
 'TEAD1': 2.5997105523411263e-15,
 'FOXO1': 0.05093147832518298,
 'NFIC': 0.07907813980071299,
 'SMAD3': 0.03953906990035779,
 'TBX20': 0.011392408424827785,
 'ZNF529': 2.5997105523411263e-15,
 'JUN': 2.5997105523411263e-15,
 'KLF13': 2.5997105523411263e-15,
 'ZNF41': 2.5997105523411263e-15,
 'TEAD3': 2.5997105523411263e-15}
```

Example 66 (python):
```python
# add to dataframe and save
scenic_meta_sel_TF_metrices.loc[centrality_dict.keys(),f'centaliry_{centrality_type}'] = list(centrality_dict.values())
scenic_meta_sel_TF_metrices[f'centaliry_{centrality_type}'].fillna(0,inplace=True)
print(len(scenic_meta_sel_TF_metrices))
display(scenic_meta_sel_TF_metrices)

# save
scenic_meta_sel_TF_metrices.to_csv('/nfs/team205/heart/Vitro/TF_selection/foetal_SANPcell_vs_workingCM/TF_selection_df_target-IonChannels.csv')
```

Example 67 (python):
```python
/tmp/ipykernel_59443/1587124067.py:2: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  scenic_meta_sel_TF_metrices.loc[centrality_dict.keys(),f'centaliry_{centrality_type}'] = list(centrality_dict.values())
/tmp/ipykernel_59443/1587124067.py:3: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  scenic_meta_sel_TF_metrices[f'centaliry_{centrality_type}'].fillna(0,inplace=True)
```

Example 68 (unknown):
```unknown
# TF-TG network
utils.save_pkl(network,
               '/nfs/team205/heart/Vitro/TF_selection/foetal_SANPcell_vs_workingCM/TF-TG-network_target-IonChannels.pkl')

# TF-TF network
utils.save_pkl(network_TF,
               '/nfs/team205/heart/Vitro/TF_selection/foetal_SANPcell_vs_workingCM/TF-TF-network_target-IonChannels.pkl')
```

Example 69 (python):
```python
import networkx as nx
```

Example 70 (unknown):
```unknown
# prepare parameters
size_scale = 7000
base_size = 200

node_size_map = {}
for n in network.keys():
    if n in centrality_dict.keys():
        node_size_map[n]=int(centrality_dict[n]*size_scale + base_size)
    else: # means no connection with other TFs
        node_size_map[n]=base_size
        

node_size_map
```

Example 71 (unknown):
```unknown
{'RARB': 699,
 'FOS': 200,
 'SP4': 200,
 'MEF2C': 1477,
 'NFIB': 200,
 'PKNOX2': 2030,
 'BHLHE40': 200,
 'MECP2': 200,
 'MAX': 200,
 'HMGA2': 494,
 'MEIS1': 992,
 'ZNF138': 200,
 'ISL1': 2173,
 'NFIC': 753,
 'TEAD1': 200,
 'FOXO1': 556,
 'NR2F1': 1398,
 'TBX3': 1317,
 'SMAD9': 418,
 'TRPS1': 1085,
 'TBX5': 1398,
 'CREM': 200,
 'GLIS3': 545,
 'NR3C1': 200,
 'ZNF529': 200,
 'JUN': 200,
 'ETV6': 1471,
 'KLF13': 200,
 'NFIA': 928,
 'NR2F2': 1398,
 'TBX18': 2370,
 'TBX20': 279,
 'SMAD3': 476,
 'ZNF536': 1080,
 'POU6F2': 950,
 'ZNF41': 200,
 'THRB': 2677,
 'SP1': 200,
 'TEAD3': 200,
 'SHOX2': 4401,
 'MITF': 574}
```

Example 72 (python):
```python
# Create a directed graph
G = nx.DiGraph()

# Add nodes and edges
for tf, targets in network.items():
    G.add_node(tf, type='TF', size=node_size_map.get(tf, 1))  # Add TF node
    for tg in targets:
        if tg not in G:
            G.add_node(tg, type='TG', size=0)  # Add TG node if not already added
        G.add_edge(tf, tg)  # Add edge

# Define tf_nodes
tf_nodes = [node for node, attr in G.nodes(data=True) if attr['type'] == 'TF']
tg_nodes = [node for node, attr in G.nodes(data=True) if attr['type'] == 'TG']
tg_nodes.sort()

# Identifying direct and indirect TFs
direct_tfs = set()
for tf, targets in network.items():
    for tg in targets:
        if tg in tg_nodes:
            direct_tfs.add(tf)

indirect_tfs = list(set(tf_nodes) - direct_tfs)  # TFs that are not in direct TFs
direct_tfs = list(direct_tfs)
indirect_tfs.sort()
direct_tfs.sort()

# Manually creating circular positions for indirect and direct TFs, and linear positions for TGs
def create_circular_positions(node_list, center_x, center_y, radius):
    angle_step = 2 * np.pi / len(node_list)
    return {node: (center_x + np.cos(i * angle_step) * radius, center_y + np.sin(i * angle_step) * radius) 
            for i, node in enumerate(node_list)}

# Create circular positions for indirect plus direct TFs
all_tfs = direct_tfs+indirect_tfs
all_tfs.sort()
tf_pos = create_circular_positions(all_tfs, 0.5, 0.5, 0.35)

# Linear positions for TGs
tg_x = [1 for _ in tg_nodes]  # All TG nodes have the same x-coordinate
tg_y = np.linspace(0.9, 0.1, len(tg_nodes))  # Evenly spaced y-coordinates
tg_pos = {tg: (x, y) for tg, x, y in zip(tg_nodes, tg_x, tg_y)}

# Combine positions
pos = {**tf_pos, **tg_pos}

# Update node colors: indirect TFs to "lightskyblue"
for node in G.nodes:
    if node in indirect_tfs:
        G.nodes[node]['color'] = 'white'
    else:
        G.nodes[node]['color'] = 'peachpuff' if G.nodes[node]['type'] == 'TF' else 'white'

# node colors
node_colors = [data['color'] for node, data in G.nodes(data=True)]

# node label colors
label_colors = dict((node,'red') if node in degs_foetal_vs_adult else (node,'black') for node in G.nodes())

# Plotting with updated positions
plt.figure(figsize=(8, 8))
nx.draw(G, pos, with_labels=False, node_color=node_colors, node_size=[G.nodes[node]['size'] for node in G.nodes()], 
        edge_color='darkgrey', alpha=0.9, 
        edgecolors="black", linewidths=1, node_shape="o")
# Draw labels with specified colors
for node, color in label_colors.items():
    plt.text(pos[node][0], pos[node][1], s=node, 
             bbox=dict(facecolor='white', alpha=0),
             horizontalalignment='center', verticalalignment='center',
             color=color)

plt.title("SANPC GRN targeting ion channel genes")
plt.savefig(f"{figdir}/SANPC_GRN_IC-genes.pdf", bbox_inches="tight",dpi=300)
plt.show()
```

---

## Post-multiVI, Find neighbors, UMAP/Clustering¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig4_Cardiomyocytes/VentricularCardiomyocytes/multiVI_scFate/no3_post-MultiVI_vCM_Left.html

**Contents:**
- Post-multiVI, Find neighbors, UMAP/Clustering¶
- Assess which MultiVI latent space most correlate with age¶
- Save¶
- Read in (for plotting)¶

**Examples:**

Example 1 (python):
```python
import scanpy as sc
import pandas as pd
import numpy as np
import anndata
import scipy

from scipy.io import mmwrite, mmread
from scipy.sparse import csr_matrix

import sctk
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (unknown):
```unknown
-----
anndata             0.8.0
numpy               1.21.5
pandas              1.3.5
scanpy              1.8.2
scipy               1.7.3
sctk                NA
session_info        1.0.0
-----
```

Example 4 (python):
```python
PIL                 9.1.1
asciitree           NA
backcall            0.2.0
beta_ufunc          NA
binom_ufunc         NA
cached_property     1.5.1
cairo               1.20.0
cffi                1.14.4
cloudpickle         1.6.0
colorama            0.4.4
cycler              0.10.0
cython_runtime      NA
cytoolz             0.11.2
dask                2022.01.0
dateutil            2.8.1
decorator           4.4.2
defusedxml          0.6.0
fasteners           0.17.3
fsspec              2022.01.0
google              NA
h5py                3.1.0
igraph              0.10.2
importlib_metadata  1.7.0
ipykernel           5.4.2
ipython_genutils    0.2.0
ipywidgets          7.6.3
jedi                0.17.2
jinja2              3.1.2
joblib              1.0.0
kiwisolver          1.3.1
leidenalg           0.9.0
llvmlite            0.35.0
louvain             0.7.0
markupsafe          2.1.1
matplotlib          3.5.2
mpl_toolkits        NA
msgpack             1.0.3
natsort             7.1.0
nbinom_ufunc        NA
numba               0.52.0
numcodecs           0.9.1
numexpr             2.7.2
numpy_groupies      0.9.13
packaging           21.3
parso               0.7.1
pexpect             4.8.0
pickleshare         0.7.5
pkg_resources       NA
plotly              5.10.0
prompt_toolkit      3.0.10
psutil              5.9.0
ptyprocess          0.7.0
pyarrow             9.0.0
pygments            2.13.0
pyparsing           2.4.7
pytz                2020.5
scvelo              0.2.4
seaborn             0.11.1
setuptools_scm      NA
simplejson          3.19.1
sinfo               0.3.1
six                 1.14.0
sklearn             1.0.2
sphinxcontrib       NA
statsmodels         0.12.1
storemagic          NA
tables              3.6.1
tblib               1.7.0
texttable           1.6.3
threadpoolctl       2.1.0
tlz                 0.11.2
toolz               0.11.2
tornado             6.1
traitlets           5.0.5
typing_extensions   NA
wcwidth             0.2.5
yaml                5.4.1
zarr                2.11.3
zipp                NA
zmq                 21.0.1
zope                NA
```

Example 5 (unknown):
```unknown
-----
IPython             7.19.0
jupyter_client      6.1.11
jupyter_core        4.7.0
notebook            6.2.0
-----
Python 3.7.9 | packaged by conda-forge | (default, Dec  9 2020, 21:08:20) [GCC 9.3.0]
Linux-4.15.0-112-generic-x86_64-with-debian-bullseye-sid
-----
Session information updated at 2023-08-16 07:45
```

Example 6 (python):
```python
sc.settings.set_figure_params(dpi=120)
```

Example 7 (unknown):
```unknown
!ls /nfs/team205/heart/anndata_objects/Foetal/vCM
```

Example 8 (unknown):
```unknown
MultiVI-model_foetal_vCM_feature-select-001
MultiVI-model_foetal_vCM_onlyMultiome_allregion_feature-select-001
MultiVI-model_foetal_vCM_onlyMultiome_allregion_onlyLeft_feature-select-001
MultiVI-model_foetal_vCM_onlyMultiome_allregion_withMouse_feature-select-001
MultiVI-model_foetal_vCM_onlyMultiome_allregion_woCycling_feature-select-001
MultiVI-model_foetal_vCM_onlyMultiome_allregion_woCycling_withMouse_feature-select-001
MultiVI-model_foetal_vCM_onlyMultiome_feature-select-001
MultiVI-model_foetal_vCM_onlyMultiome_only-whole_feature-select-001
MultiVI-post_foetal_vCM_feature-select-001.h5ad
MultiVI-post_foetal_vCM_onlyMultiome_allregion_feature-select-001.h5ad
MultiVI-post_foetal_vCM_onlyMultiome_allregion_onlyLeft_feature-select-001.h5ad
MultiVI-post_foetal_vCM_onlyMultiome_allregion_withMouse_feature-select-001.h5ad
MultiVI-post_foetal_vCM_onlyMultiome_allregion_woCycling_feature-select-001.h5ad
MultiVI-post_foetal_vCM_onlyMultiome_allregion_woCycling_withMouse_feature-select-001.h5ad
MultiVI-post_foetal_vCM_onlyMultiome_feature-select-001.h5ad
MultiVI-post_foetal_vCM_onlyMultiome_only-whole_feature-select-001.h5ad
MultiVI-pre_mouse-organogenesis_RNA_raw.h5ad
MultiVI-pre_paired_foetal_vCMs_raw.h5ad
MultiVI-pre_scRNA_foetal_vCMs_raw.h5ad
```

Example 9 (python):
```python
adata = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/vCM/MultiVI-post_foetal_vCM_onlyMultiome_allregion_onlyLeft_feature-select-001.h5ad')
adata
```

Example 10 (unknown):
```unknown
AnnData object with n_obs × n_vars = 16451 × 174604
    obs: 'sangerID', 'combinedID', 'donor', 'region', 'age', 'sex', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'week', 'trimester', 'cycling', 'S_score', 'G2M_score', 'phase', 'fine_grain', 'mid_grain', 'coarse_grain', 'batch_key', 'total_counts', 'total_counts_mt', 'total_counts_ribo', 'n_genes_by_counts', 'pct_counts_mt', 'pct_counts_ribo', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'log1p_total_counts_mt', 'log1p_total_counts_ribo', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', '_indices', '_scvi_batch', '_scvi_labels'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id', 'seqnames', 'start', 'end', 'width', 'strand', 'score', 'replicateScoreQuantile', 'groupScoreQuantile', 'Reproducibility', 'GroupReplicate', 'distToGeneStart', 'nearestGene', 'peakType', 'distToTSS', 'nearestTSS', 'GC', 'idx', 'N', 'n_cells'
    uns: '_scvi_manager_uuid', '_scvi_uuid'
    obsm: 'MultiVI_latent', '_scvi_extra_categorical_covs', '_scvi_extra_continuous_covs'
```

Example 11 (python):
```python
adata.X.data[:10]
```

Example 12 (unknown):
```unknown
array([1., 1., 1., 1., 3., 1., 2., 1., 1., 1.], dtype=float32)
```

Example 13 (python):
```python
'''
# log-normalise
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)
'''
```

Example 14 (python):
```python
'\n# log-normalise\nsc.pp.normalize_total(adata, target_sum=1e4)\nsc.pp.log1p(adata)\n'
```

Example 15 (python):
```python
%%time
sc.pp.neighbors(adata, use_rep="MultiVI_latent",n_neighbors=15)
sc.tl.umap(adata, min_dist=0.2, spread=1.0, n_components=3)
sc.tl.leiden(adata, resolution=0.5, key_added='leiden_MultiVI')
```

Example 16 (unknown):
```unknown
CPU times: user 42.1 s, sys: 7.83 s, total: 50 s
Wall time: 41.6 s
```

Example 17 (python):
```python
sc.pl.umap(adata, color=['leiden_MultiVI'], size=10, wspace=0.3, components='all', ncols=3)
```

Example 18 (python):
```python
sc.pl.umap(adata, color=['donor'], size=10, wspace=0.5, components='all', ncols=3)
```

Example 19 (python):
```python
sc.pl.umap(adata, color=['total_counts','pct_counts_mt','pct_counts_ribo',
                                                          'S_score','G2M_score'], 
           size=10, wspace=0.5, components='all', ncols=3, cmap='RdPu')
```

Example 20 (python):
```python
sc.pl.umap(adata, color=['fine_grain'], size=10, wspace=0.5, components='all', ncols=3)
```

Example 21 (python):
```python
for fine in [
    # 'VentricularCardiomyocytesCycling',
    # 'VentricularCardiomyocytesRightTrabeculated', 
    # 'VentricularCardiomyocytesRightCompact', 
 'VentricularCardiomyocytesLeftTrabeculated', 
 'VentricularCardiomyocytesLeftCompact', 
 ]:
    sc.pl.umap(adata, color=['fine_grain'], groups=[fine], size=10, wspace=0.5, components='all', ncols=3)
```

Example 22 (python):
```python
sc.pl.umap(adata, color=['cycling'], size=10, wspace=0.5, components='all', ncols=3)
```

Example 23 (python):
```python
sc.pl.umap(adata, color=['week'], size=10, wspace=0.5, components='all', ncols=3)
```

Example 24 (python):
```python
sc.pl.umap(adata, color=['region'], size=10, wspace=0.5, components='all', ncols=3)
```

Example 25 (python):
```python
adata.obs['week']
```

Example 26 (unknown):
```unknown
barcode
BHF_F_Hea11933669_BHF_F_Hea11596622_AATTAGCGTTAGCATG-1      13W
HCAHeartST13386009_HCAHeartST13303419_ATCCTTAGTGGATTGC-1    20W
BHF_F_Hea13188324_BHF_F_Hea13187626_CGAGGCAAGGCTGTCA-1      20W
BHF_F_Hea13188324_BHF_F_Hea13187626_TAGTAAGCACATACTG-1      20W
BHF_F_Hea13188324_BHF_F_Hea13187626_GTTTAACCACGTAATT-1      20W
                                                           ... 
BHF_F_Hea11933669_BHF_F_Hea11596622_GCCCAAATCACATTGA-1      13W
BHF_F_Hea11933667_BHF_F_Hea11596620_CGCAATTAGGTCTTGG-1      13W
BHF_F_Hea11933667_BHF_F_Hea11596620_GCGCTTGCAGGATGGC-1      13W
BHF_F_Hea11933667_BHF_F_Hea11596620_CTCTTGATCATGCGTG-1      13W
BHF_F_Hea11933668_BHF_F_Hea11596621_AACTAGCTCTTGAGGT-1      13W
Name: week, Length: 16451, dtype: category
Categories (10, object): ['4W', '5W', '7W', '8W', ..., '13W', '14W', '15W', '20W']
```

Example 27 (python):
```python
# get latent space
latent = pd.DataFrame(adata.obsm['MultiVI_latent'])
latent.index = adata.obs_names

# get age (week)
latent['week'] = [int(x.replace('W','')) for x in adata.obs['week']]
# arr = pd.Series([int(x.replace('W','')) for x in adata.obs['week']])

# correlation with week
# res = latent.corrwith(arr, axis=1, method='pearson')
res = latent.corr(method='pearson')['week']
res.sort_values(ascending=False)
```

Example 28 (unknown):
```unknown
week    1.000000
9       0.337589
2       0.233910
1       0.151557
0       0.146214
18      0.123042
10      0.106890
8       0.075663
13     -0.007618
19     -0.033006
6      -0.037545
11     -0.048397
7      -0.091645
15     -0.115572
16     -0.128860
14     -0.142237
12     -0.204365
3      -0.205714
5      -0.224725
4      -0.259518
17     -0.261819
Name: week, dtype: float64
```

Example 29 (unknown):
```unknown
latent.index = [f'cell{x}' for x in range(len(latent))]
latent.to_csv('test.csv')
```

Example 30 (python):
```python
adata.write('/nfs/team205/heart/anndata_objects/Foetal/vCM/MultiVI-post_foetal_vCM_onlyMultiome_allregion_onlyLeft_feature-select-001.h5ad')
```

Example 31 (python):
```python
sc.settings.set_figure_params(dpi=120)
```

Example 32 (python):
```python
# adata = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Vitro/multiVI/MultiVI_Foetal_Adult_CMs_Multiome_feature-select-001.h5ad')
# adata
```

Example 33 (python):
```python
sc.pl.umap(adata, color=['source'], size=5, wspace=0.5, ncols=1, frameon=False)
```

---

## Import modules¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/RNA/scnRNA_Integration-Annotation/Leukocytes/No4_Lymphoid_post-scVI.html

**Contents:**
- Import modules¶
- Function¶
- Read in data¶
- Log-normalise¶
- Clustering, UMAP¶
  - Deciding resolution¶
- Marker gene expressions¶
- DEG analysis¶
- CellTypist prediction¶
- Annotation¶
- Recover row counts and save¶
- Assign non-cycling cell type labels to "T_cycling" cells¶
- 1) Regressing out cell cycle genes¶
- 2) Train LR model with non-cycling cells (only for non-B cells)¶
- 3) Predict cell states¶
- 4) Check marker expressions¶
- 5) Save assigned list¶
- Celltypist, pan-foetal immune¶
- Check matching with previoius annotation¶

gene expression: normalise + logtransform

**Examples:**

Example 1 (python):
```python
import warnings
warnings.simplefilter(action='ignore', category=FutureWarning)

import numpy as np
import numpy.random as random
import pandas as pd
import scanpy as sc
import matplotlib.pyplot as plt
import anndata


import celltypist
from celltypist import models

import sctk
import os
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
-----
anndata             0.8.0
celltypist          0.1.9
matplotlib          3.5.2
numpy               1.21.5
pandas              1.3.5
scanpy              1.8.2
sctk                NA
session_info        1.0.0
-----
```

Example 4 (python):
```python
PIL                 9.1.1
asciitree           NA
backcall            0.2.0
beta_ufunc          NA
binom_ufunc         NA
brotli              NA
cached_property     1.5.1
cairo               1.20.0
certifi             2020.12.05
cffi                1.14.4
chardet             3.0.4
charset_normalizer  2.0.10
cloudpickle         1.6.0
colorama            0.4.4
cycler              0.10.0
cython_runtime      NA
cytoolz             0.11.2
dask                2022.01.0
dateutil            2.8.1
decorator           4.4.2
defusedxml          0.6.0
fasteners           0.17.3
fsspec              2022.01.0
google              NA
h5py                3.1.0
idna                2.10
igraph              0.10.2
importlib_metadata  1.7.0
ipykernel           5.4.2
ipython_genutils    0.2.0
ipywidgets          7.6.3
jedi                0.17.2
jinja2              3.1.2
joblib              1.0.0
kiwisolver          1.3.1
leidenalg           0.9.0
llvmlite            0.35.0
louvain             0.7.0
markupsafe          2.1.1
mpl_toolkits        NA
msgpack             1.0.3
natsort             7.1.0
nbinom_ufunc        NA
numba               0.52.0
numcodecs           0.9.1
numexpr             2.7.2
numpy_groupies      0.9.13
packaging           21.3
parso               0.7.1
pexpect             4.8.0
pickleshare         0.7.5
pkg_resources       NA
plotly              5.10.0
prompt_toolkit      3.0.10
psutil              5.9.0
ptyprocess          0.7.0
pyarrow             9.0.0
pygments            2.13.0
pyparsing           2.4.7
pytz                2020.5
requests            2.27.1
scipy               1.7.3
scvelo              0.2.4
seaborn             0.11.1
setuptools_scm      NA
sinfo               0.3.1
six                 1.14.0
sklearn             1.0.2
socks               1.7.1
sphinxcontrib       NA
statsmodels         0.12.1
storemagic          NA
tables              3.6.1
tblib               1.7.0
texttable           1.6.3
threadpoolctl       2.1.0
tlz                 0.11.2
toolz               0.11.2
tornado             6.1
traitlets           5.0.5
typing_extensions   NA
urllib3             1.26.9
wcwidth             0.2.5
yaml                5.4.1
zarr                2.11.3
zipp                NA
zmq                 21.0.1
zope                NA
```

Example 5 (unknown):
```unknown
-----
IPython             7.19.0
jupyter_client      6.1.11
jupyter_core        4.7.0
notebook            6.2.0
-----
Python 3.7.9 | packaged by conda-forge | (default, Dec  9 2020, 21:08:20) [GCC 9.3.0]
Linux-4.15.0-158-generic-x86_64-with-debian-bullseye-sid
-----
Session information updated at 2023-02-22 12:48
```

Example 6 (python):
```python
sc.settings.set_figure_params(dpi=120, facecolor='white', color_map = 'RdPu',)
```

Example 7 (python):
```python
#>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
# extract rank_genes_groups result 
#>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

import scanpy as sc
import pandas as pd

def rank_genes_groups_to_df(adata, cell):
    
    dic = adata.uns['rank_genes_groups'] 
    
    df = pd.DataFrame({
        'names' : dic['names'][cell],
        'scores' : dic['scores'][cell],
        'pvals' : dic['pvals'][cell],
        'pvals_adj' : dic['pvals_adj'][cell],
        'logfoldchanges' : dic['logfoldchanges'][cell]
    })
    
    return df
```

Example 8 (unknown):
```unknown
compartment = 'Lymphoid'
```

Example 9 (python):
```python
adata = sc.read_h5ad(f'/home/jovyan/mount/gdrive/Colab/data/scVI/subsets/RNA_foetal_{compartment}_post-scVI_16Feb2023.h5ad')
adata
```

Example 10 (unknown):
```unknown
AnnData object with n_obs × n_vars = 21336 × 28811
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', '_scvi_labels', '2Jan_coarse_grain', '2Jan_mid_grain', '2Jan_fine_grain', 'leiden_scVI', 'FACSgate', 'sex', 'week', 'trimester', 'cell_type2'
    var: 'gene_name_scRNA-0', 'gene_id', 'n_cells'
    uns: '_scvi_manager_uuid', '_scvi_uuid'
    obsm: 'X_pca', 'X_scVI', 'X_umap', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 11 (python):
```python
# normalise and logtransform
# adata.raw = adata
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)
```

Example 12 (python):
```python
%%time

sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata, min_dist=0.2, spread=1.0, n_components=3)

resolutions = [0.2,0.3,0.4,0.5,0.6,] # 0.7,0.8,0.9,1.0
for res in resolutions:
    print(res)
    sc.tl.leiden(adata, resolution=res, key_added=f'leiden_scVI_{res}')
    sc.pl.umap(adata, color=[f'leiden_scVI_{res}'], size=5, components='all', wspace=0.5)
```

Example 13 (python):
```python
sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata, min_dist=0.2, spread=1.0, n_components=3)
sc.tl.leiden(adata, resolution=1, key_added='leiden_scVI')
```

Example 14 (python):
```python
sc.pl.umap(adata, color=['leiden_scVI'], size=5, wspace=0.1, legend_loc='on data', legend_fontsize=10,components='all')
```

Example 15 (python):
```python
sc.pl.umap(adata, color=['region','donor','cell_or_nuclei','kit_10x'], size=5, wspace=0.7)
```

Example 16 (python):
```python
sc.pl.umap(adata, color=['week'], size=3, wspace=0.5)
```

Example 17 (python):
```python
sc.pl.umap(adata, color=['scrublet_score','cluster_scrublet_score','doublet_pval','doublet_bh_pval','pct_counts_mt','pct_counts_ribo'], size=2, wspace=0.2, ncols=6)
```

Example 18 (python):
```python
stress_genes=['CD3E','CD4','CD8A','FOXP3','CD79A','CD79B','RAG1','RAG2','MS4A1','IGHD','TRBC1']
stress_genes = [x for x in stress_genes if x in adata.var_names]
sc.pl.umap(adata, color=stress_genes, size=5, wspace=0.2, ncols=6)
```

Example 19 (python):
```python
# Checking glabal markers
markers = {
    'vCM': ['MYH7', 'MYL2', 'FHL2'],
    'aCM': ['NPPA', 'MYL7', 'MYL4'],
    'FB': ['DCN', 'GCN', 'PDGFRA'],
    'EC': ['VWF', 'PECAM1', 'CDH5'],
    'Peri': ['RGS5', 'ABCC9', 'KCNJ8'],
    'SMC': ['MYH11', 'TAGLN', 'ACTA2'],
    'Meso': ['MSLN', 'WT1', 'BNC1'],
    'Neuro': ['PLP1', 'NRXN1', 'NRXN3'],
    'Adipo': ['GPAM', 'FASN', 'LEP'],
    'Myelo': ['CD14', 'C1QA', 'CD68','LYVE1','TIMD4'],
    'Lymph': ['CD8A', 'IL7R', 'CD40LG'],
    'B': ['CD19','SDC1','IGHG1','IGHA1'],
    'Mast': ['KIT', 'CPA3'],
}

for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in adata.var_names]

sc.tl.dendrogram(adata, groupby='leiden_scVI')
sc.pl.dotplot(adata,
              markers, 
              groupby='leiden_scVI',
              dendrogram=True,
              standard_scale="var",
              color_map="Reds",
              # swap_axes=True,
             )
```

Example 20 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: vCM, aCM, FB, etc.
```

Example 21 (python):
```python
# lymphocytes markers

markers={
    'B cell': ['CD19', 'CD79A', 'CD79B', 'IGHM', 'IGHG1', 'IGHA1'],
    'T cell': ['CD3E', 'CD3D', 'TRAC', 'CD4', 'CD8A', 'CD8B'],
    'naive': ['CCR7', 'TCF7', 'LEF1', 'SELL'],
    'act': ['IL2','CD69'],
    'CD4T_reg': ['FOXP3', 'CTLA4', 'IL2RA', 'IKZF2'],
    'cytotoxic': ['PRF1', 'GZMB', 'GZMH', 'NKG7'],
    'trans': ['GZMK', 'TOX'],
    'te': ['TIGIT', 'LAG3', 'PDCD1'],
    'em': ['IL7R', 'RUNX2'],
    'MAIT': ['SLC4A10', 'KLRB1', 'CCR6'],
    'NK': ['KLRF1', 'KLRD1', 'NCAM1', 'FCGR3A'],
    'ILC': ['KIT', 'IL18R1'],
    'cycling': ['MKI67', 'TOP2A', 'UBE2C'],
    'TFs': ['TBX21', 'GATA3', 'RORC', 'BCL6',],
}


for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in adata.var_names]

sc.pl.dotplot(adata,
              markers, 
              groupby='leiden_scVI',
              dendrogram=True,
              standard_scale="var",
              color_map="Reds",
              # swap_axes=True,
             )
```

Example 22 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: B cell, T cell, naive, etc.
```

Example 23 (python):
```python
markers={
     'T cell': ['CD3E', 'CD3D','CD4', 'CD8A', 'CD8B','TRAC','TRBC1','TRDC','TRGC1'],
    'naive': ['TCF7', 'LEF1', 'SELL'],
    'CD4T_reg': ['FOXP3', 'CTLA4', 'IL2RA', 'IKZF2'],
    'cytotoxic': ['PRF1', 'GZMB', 'GZMH', 'NKG7'],
    'NK': ['KLRF1', 'KLRD1', 'NCAM1','FCGR3A','IFNG'],
    'ILC3': ['KIT', 'IL18R1','RORC','IL17A'],
    
    'B': ['CD19','CD20','CD22','CD79A', 'CD79B','MS4A1','IL7R',],
    '':['CCR7','TNFRSF13B',],
    'ProB':['KIT','SPN','CD34'],
    'B1':['CD27','CCR10','MS4A1','SPN'],
    'Plasma': ['CD27','CCR10','CD38','SDC1','JCAIN'],
    'Abs':['IGHM','IGHD', 'IGHG1','IGHG2', 'IGHG3','IGHG4', 'IGHA1','IGHE','IGLC1','IGKC'],
    'Gene editing': ['AICDA', 'RAG1','RAG2'],
    
    
    'Cycling':['MKI67','TOP2A'],

    'TFs': ['TBX21', 'GATA3', 'RORC', 'BCL6',],
}


for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in adata.var_names]

sc.pl.dotplot(adata,
              markers, 
              groupby='leiden_scVI',
              dendrogram=True,
              standard_scale="var",
              color_map="Reds",
              # swap_axes=True,
             )
```

Example 24 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: T cell, naive, CD4T_reg, etc.
```

Example 25 (python):
```python
adata.X.data[:10]
```

Example 26 (python):
```python
sc.tl.rank_genes_groups(adata, 'leiden_scVI', method='t-test')

DEG_dict = {}
for cluster in adata.obs['leiden_scVI'].cat.categories:
    df=rank_genes_groups_to_df(adata, cluster)
    # select genes: significant and up-regulated ones
    df=df[(df['pvals_adj']<0.05)&(df['logfoldchanges']>0)]
    # sort based on the logFC
    df.sort_values('scores',ascending=False,inplace=True)
    DEG_dict[cluster]=list(df[:10]['names'])
    del df

sc.pl.dotplot(adata,
              DEG_dict, 
              groupby='leiden_scVI',
              dendrogram=True,
              standard_scale="var",
              color_map="Reds",
              swap_axes=False,
             )
```

Example 27 (python):
```python
# load model and predict
panfetal_model = models.Model.load('/nfs/team205/kk18/Analysis/celltypist/models/celltypist_model.Pan_Fetal_Human.pkl')
prediction=celltypist.annotate(adata, model = panfetal_model, majority_voting = True)
```

Example 28 (unknown):
```unknown
🔬 Input data has 21336 cells and 28811 genes
🔗 Matching reference genes in the model
🧬 6669 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
👀 Detected a neighborhood graph in the input object, will run over-clustering on the basis of it
⛓️ Over-clustering input data with resolution set to 15
🗳️ Majority voting the predictions
✅ Majority voting done!
```

Example 29 (unknown):
```unknown
celltypist.dotplot(prediction, use_as_reference = 'leiden_scVI', use_as_prediction = 'predicted_labels')
```

Example 30 (python):
```python
sc.pl.umap(adata, color=['leiden_scVI'], size=5, wspace=0.1, legend_loc='on data', components='all', legend_fontsize=7)
```

Example 31 (python):
```python
sc.pl.umap(adata, color=['2Jan_fine_grain'], size=5, wspace=0.1, legend_loc='on data', components='all', legend_fontsize=7)
```

Example 32 (python):
```python
set(adata.obs['2Jan_fine_grain'])
```

Example 33 (unknown):
```unknown
{'B1Cells',
 'BCells',
 'BProCells',
 'Cardiofibromyocytes',
 'CardiofibromyocytesSecretory',
 'CoronaryCapillaryEndothelialCells',
 'CoronaryVesselSmoothMuscleCells',
 'DC1',
 'DC2',
 'DCsPlasmocytoids',
 'EarlyGreatVesselFLRT2',
 'EarlyGreatVesselHOXB4',
 'EndocardialCells',
 'GVAdFibOCA2',
 'GVAdFibOther',
 'GreatVesselArterialEndothelialCells',
 'GreatVesselSmoothMuscleCells',
 'InnateLymphoidCells',
 'MacrophagesLYVE1pos',
 'MacrophagesTIMD4pos',
 'MastCells',
 'MonocytesMPOpos',
 'MyocIntFibROBO2',
 'NaturalKillerCells',
 'SchwannCells',
 'TCellsCD4pos',
 'TCellsCD8pos',
 'Tregs',
 'ValveOther',
 nan}
```

Example 34 (python):
```python
adata.obs['cell_type3']=adata.obs['leiden_scVI'].astype('str').copy()
adata.obs.replace({'cell_type3':{
    '0':'CD4+T', 
    '1':'NK', 
    '2':'CD4+Treg', 
    '3':'B_MS4A1+', # MS4A1, IgD, CellTypist
    '4':'CD8+T', 
    '5':'ILC',  
    '6':'B', 
    '7':'CD4+T',  
    '8':'CD4+T', 
    '9':'CD4+T', 
    '10':'T_cycling',  
    '11':'NK',
    '12':'Bpro', # CD34, RAG genes, CellTypist
    '13':'B_cycling', 
    '14':'CD8+T', 
    '15':'B_MS4A1+',# MS4A1, IgD, CellTypist
    '16':'B_MS4A1+',# MS4A1, IgD, CellTypist
    '17':'B', 
    '18':'CD4+Treg', 
    '19':'B_MS4A1+',# MS4A1, IgD, CellTypist
    '20':'ILC',
    '21':'B_MS4A1+',# MS4A1, IgD, CellTypist
}}, inplace=True)


adata.obs['cell_type3']=adata.obs['cell_type3'].astype('category')
adata.obs['cell_type3'].cat.reorder_categories([
    'CD4+T',
    'CD4+Treg',
    'CD8+T',
    'NK', 
    'ILC',
    'T_cycling', 
    'Bpro',
    'B', 
    'B_MS4A1+',
    'B_cycling',
    # 'Doublet'
], inplace=True)

adata.obs['cell_type3'].value_counts()
```

Example 35 (unknown):
```unknown
CD4+T        5206
B_MS4A1+     3722
NK           2437
CD8+T        2121
CD4+Treg     1940
B            1666
ILC          1639
T_cycling     912
Bpro          850
B_cycling     843
Name: cell_type3, dtype: int64
```

Example 36 (python):
```python
markers={
     'T cell': ['CD3E', 'CD3D','CD4', 'CD8A', 'CD8B','TRAC','TRBC1','TRDC','TRGC1'],
    'naive': ['TCF7', 'LEF1', 'SELL'],
    'CD4T_reg': ['FOXP3', 'CTLA4', 'IL2RA', 'IKZF2'],
    'cytotoxic': ['PRF1', 'GZMB', 'GZMH', 'NKG7'],
    'NK': ['KLRF1', 'KLRD1', 'NCAM1','FCGR3A','IFNG'],
    'ILC3': ['KIT', 'IL18R1','RORC','IL17A'],
    
    'B': ['CD19','CD20','CD22','CD79A', 'CD79B','MS4A1','IL7R',],
    '':['CCR7','TNFRSF13B',],
    'ProB':['KIT','SPN','CD34'],
    'B1':['CD27','CCR10','MS4A1','SPN'],
    'Plasma': ['CD27','CCR10','CD38','SDC1','JCAIN'],
    'Abs':['IGHM','IGHD', 'IGHG1','IGHG2', 'IGHG3','IGHG4', 'IGHA1','IGHE','IGLC1','IGKC'],
    'Gene editing': ['AICDA', 'RAG1','RAG2'],
    
    
    'Cycling':['MKI67','TOP2A'],

    'TFs': ['TBX21', 'GATA3', 'RORC', 'BCL6',],
}


for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in adata.var_names]

sc.pl.dotplot(adata,
              markers, 
              groupby='cell_type3',
              dendrogram=False,
              standard_scale="var",
              color_map="Reds",
              # swap_axes=True,
             )
```

Example 37 (python):
```python
markers={
     'T cell': ['CD3E', 'CD3D','TRAC','TRBC1','TRBV1','TRBV2'],
     'B': ['CD19','CD20','CD22','CD79A', 'CD79B','MS4A1','IL7R',],
   
    
    'Mono':['CCR2','CD14','FCGR3A','CX3CR1','FCGR1A','FCN1','TREM2'],
    'MP':['C1QA','C1QB','CD68','CD163','MERTK','LYVE1','IGF1','TIMD4','CX3CR1','DOCK4','COL1A1','COL1A2','COL3A1'],
    'DC1':['ID2','IRF8','BATF3','CLEC9A'],
    'DC2':['CLEC10A','CD1C','FCER1A',],
    'pDC':['IRF8','IRF4','IL3RA'],
    
   
    'Mast': ['KIT', 'CPA3','HDC','GATA2'],
    'Megakaryocyte':['ITGA2B','MPL','ITGB3',],
    'mesenchymalMP':['COL1A1','COL1A2','COL3A1','C1QA','C1QC','C1QD','VSIG4','COL3A1', 'MDK', 'SPARC', 'ELN'],
    'vCM': ['MYH7', 'MYL2', 'FHL2'],
    'aCM': ['NPPA', 'MYL7', 'MYL4'],
}


for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in adata.var_names]

sc.pl.dotplot(adata,
              markers, 
              groupby='cell_type3',
              dendrogram=False,
              standard_scale="var",
              color_map="Reds",
              # swap_axes=True,
             )
```

Example 38 (python):
```python
sc.settings.set_figure_params(dpi=120,  color_map = 'RdPu',)
sc.pl.umap(adata, color=['cell_type3'], size=5, legend_fontsize=10, frameon=False, components='all', wspace=0.5)
```

Example 39 (python):
```python
sc.tl.rank_genes_groups(adata, 'cell_type3', method='t-test')

DEG_dict = {}
for cluster in adata.obs['cell_type3'].cat.categories:
    df=rank_genes_groups_to_df(adata, cluster)
    # select genes: significant and up-regulated ones
    df=df[(df['pvals_adj']<0.05)&(df['logfoldchanges']>0)]
    # sort based on the logFC
    df.sort_values('scores',ascending=False,inplace=True)
    DEG_dict[cluster]=list(df[:10]['names'])
    del df

sc.pl.dotplot(adata,
              DEG_dict, 
              groupby='cell_type3',
              dendrogram=False,
              standard_scale="var",
              color_map="Reds",
              swap_axes=False,
             )
```

Example 40 (python):
```python
markers={
     'T cell': ['CD3E', 'CD3D','CD4', 'CD8A', 'CD8B'],
    'CD4T_reg': ['FOXP3', 'CTLA4', 'IL2RA', 'IKZF2'],
    'cytotoxic': ['PRF1', 'GZMB','NKG7'],
    'NK': ['KLRF1', 'KLRD1','NCAM1','FCGR3A','IFNG'],
    'ILC': ['KIT', 'IL7R','IL18R1','RORC','IL17A'],   
    'B': ['CD19','CD20','CD22','CD79A', 'CD79B','MS4A1'],
     'ProB':['CD34'],
    'B1':['CD27','CCR10'],
    'Igs':['IGHM','IGHD'],
    'Gene editing': ['AICDA', 'RAG1','RAG2'],
    'Cycling':['MKI67','TOP2A'],
}


for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in adata.var_names]

sc.pl.dotplot(adata[adata.obs['cell_type3']!='Doublet'],
              markers, 
              groupby='cell_type3',
              dendrogram=False,
              standard_scale="var",
              color_map="Reds",
              # swap_axes=True,
             )
```

Example 41 (python):
```python
markers={
    'B': ['CD19','CD20','CD22','CD79A', 'CD79B','MS4A1'],
     'ProB':['CD34'],
    'B1':['CD27','CCR10'],
    'Igs':['IGHM','IGHD'],
    'Gene editing': ['AICDA', 'RAG1','RAG2'],
    'Cycling':['MKI67','TOP2A'],
    'Checking':['JCHAIN','CD38','SDC1'],
}


for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in adata.var_names]

sc.pl.dotplot(adata[adata.obs['cell_type3']!='Doublet'],
              markers, 
              groupby='cell_type3',
              dendrogram=False,
              standard_scale="var",
              color_map="Reds",
              # swap_axes=True,
             )
```

Example 42 (python):
```python
adata.obs['cell_type3'].astype('str') + ' | ' + adata.obs['cell_or_nuclei'].astype('str')
```

Example 43 (unknown):
```unknown
barcode
BHF_F_Hea10402917_ATGTGTGTCGAACGGA-1                                ILC | cell
BHF_F_Hea10402917_CAGGTGCAGGCTATCT-1                                  B | cell
BHF_F_Hea10402917_ATAACGCTCCACGACG-1                              CD4+T | cell
BHF_F_Hea10402917_CACAGTAGTATAGGGC-1                          B_cycling | cell
BHF_F_Hea10402917_GCTGGGTGTGCGGTAA-1                           B_MS4A1+ | cell
                                                                   ...        
HCAHeartST13386010_HCAHeartST13303420_AATTTGCCATAAGCAA-1           NK | nuclei
HCAHeartST13386010_HCAHeartST13303420_GGATTGCGTGGTTCTT-1    B_cycling | nuclei
HCAHeartST13386010_HCAHeartST13303420_TGTGCGGGTGTCACGG-1     CD4+Treg | nuclei
HCAHeartST13386010_HCAHeartST13303420_CTATTCAGTAGGATTT-1          ILC | nuclei
HCAHeartST13386010_HCAHeartST13303420_CTCACACTCGACCTGA-1     CD4+Treg | nuclei
Length: 21336, dtype: object
```

Example 44 (python):
```python
# celltype x cell_nuclei
adata.obs['cell_type3_cllnuclei'] = adata.obs['cell_type3'].astype('str') + ' | ' + adata.obs['cell_or_nuclei'].astype('str')

markers={
    'B': ['CD19','CD20','CD22','CD79A', 'CD79B','MS4A1'],
     'ProB':['CD34'],
    'B1':['CD27','CCR10'],
    'Igs':['IGHM','IGHD'],
    'Gene editing': ['AICDA', 'RAG1','RAG2'],
    'Cycling':['MKI67','TOP2A'],
    'Checking':['JCHAIN','CD38','SDC1'],
}


for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in adata.var_names]

sc.pl.dotplot(adata[(adata.obs['cell_type3']!='Doublet')&(adata.obs['cell_or_nuclei']!='na')&(adata.obs['cell_type3'].isin(['B','B_MS4A1+','B_cycling']))],
              markers, 
              groupby='cell_type3_cllnuclei',
              dendrogram=False,
              standard_scale="var",
              color_map="Reds",
              # swap_axes=True,
             )
```

Example 45 (unknown):
```unknown
... storing 'cell_type3_cllnuclei' as categorical
```

Example 46 (python):
```python
sc.pl.umap(adata, color=['cell_or_nuclei'], groups='cell', size=5, legend_fontsize=10, frameon=False, components='all', wspace=0.5)
```

Example 47 (unknown):
```unknown
... storing 'cell_type3_cllnuclei' as categorical
```

Example 48 (python):
```python
sc.pl.umap(adata, color=['cell_or_nuclei'], groups='nuclei', size=5, legend_fontsize=10, frameon=False, components='all', wspace=0.5)
```

Example 49 (python):
```python
# celltype x cell_nuclei
markers={
    'B': ['CD19','CD20','CD22','CD79A', 'CD79B','MS4A1'],
     'ProB':['CD34'],
    'B1':['CD27','CCR10'],
    'Igs':['IGHM','IGHD'],
    'Gene editing': ['AICDA', 'RAG1','RAG2'],
    'Cycling':['MKI67','TOP2A'],
    'Checking':['JCHAIN','CD38','SDC1'],
}


for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in adata.var_names]

sc.pl.dotplot(adata[(adata.obs['cell_type3']!='Doublet')&(adata.obs['cell_or_nuclei']=='nuclei')],
              markers, 
              groupby='cell_type3',
              dendrogram=False,
              standard_scale="var",
              color_map="Reds",
              # swap_axes=True,
             )
```

Example 50 (python):
```python
adata_raw=sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/Immune/RNA_foetal_raw_immune_16Feb2023.h5ad')
print(adata_raw.shape)

adata_raw=anndata.AnnData(X=adata_raw[adata.obs_names,:].X,
                      obs=adata.obs, # from annotated or subsetted data
                      var=adata_raw.var,
                      obsm=adata.obsm, # from annotated or subsetted data
                      obsp=adata.obsp # from annotated or subsetted data
                               )
print(adata_raw.shape)
```

Example 51 (unknown):
```unknown
(41881, 36601)
(21336, 36601)
```

Example 52 (python):
```python
adata_raw.write('/nfs/team205/heart/anndata_objects/Foetal/Immune/RNA_foetal_raw_Lymphoid_16Feb2023.h5ad')
adata_raw
```

Example 53 (unknown):
```unknown
AnnData object with n_obs × n_vars = 21336 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', '_scvi_labels', '2Jan_coarse_grain', '2Jan_mid_grain', '2Jan_fine_grain', 'leiden_scVI', 'FACSgate', 'sex', 'week', 'trimester', 'cell_type2', 'cell_type3'
    var: 'gene_name_scRNA-0', 'gene_id'
    obsm: 'X_pca', 'X_scVI', 'X_umap', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 54 (python):
```python
sc.pl.umap(adata_raw, color=['cell_type3'], size=5, legend_fontsize=10, frameon=False, components='all', wspace=0.5)
```

Example 55 (python):
```python
# save metadata
adata_raw.obs.to_csv('/nfs/team205/heart/anndata_objects/Foetal/Immune/RNA_foetal_raw_Lymphoid_16Feb2023.obs.csv')
```

Example 56 (python):
```python
# read in saved object
adata_raw = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/Immune/RNA_foetal_raw_Lymphoid_16Feb2023.h5ad')
adata_raw
```

Example 57 (python):
```python
# log-normalise
adata = adata_raw.copy()
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)
```

Example 58 (python):
```python
adata.X.data[:10]
```

Example 59 (unknown):
```unknown
array([0.20843568, 0.38083044, 0.20843568, 0.20843568, 0.20843568,
       0.38083044, 0.38083044, 0.76952875, 0.38083044, 0.38083044],
      dtype=float32)
```

Example 60 (python):
```python
# cell cycle genes
cell_cycle_genes = [x.strip() for x in open('/nfs/team205/kk18/data/geneset/scanpy_cell_cycle_vignette_files/regev_lab_cell_cycle_genes.txt')]

s_genes = cell_cycle_genes[:43]
g2m_genes = cell_cycle_genes[43:]

# select genes which are in the adata
cell_cycle_genes = [x for x in cell_cycle_genes if x in adata.var_names]
s_genes = [x for x in s_genes if x in adata.var_names]
g2m_genes = [x for x in g2m_genes if x in adata.var_names]
```

Example 61 (python):
```python
%%time
adata_reg = adata.copy()

# scaling and scoring
sc.pp.scale(adata_reg)
sc.tl.score_genes_cell_cycle(adata_reg, s_genes=s_genes, g2m_genes=g2m_genes)

# Running a PCA on cell cycle genes reveals, unsurprisingly, cells separate by phase
adata_cc_genes = adata_reg[:, cell_cycle_genes]
sc.tl.pca(adata_cc_genes)
sc.pl.pca_scatter(adata_cc_genes, color=['phase','cell_type3'])
```

Example 62 (unknown):
```unknown
... storing 'phase' as categorical
```

Example 63 (unknown):
```unknown
CPU times: user 13.8 s, sys: 10.1 s, total: 23.9 s
Wall time: 20.1 s
```

Example 64 (python):
```python
%%time

# regress out
sc.pp.regress_out(adata_reg, ['S_score', 'G2M_score'])
sc.pp.scale(adata_reg)

# When running a PCA on only cell cycle genes, cells no longer separate by cell-cycle phase
adata_cc_genes = adata_reg[:, cell_cycle_genes]
sc.tl.pca(adata_cc_genes)
sc.pl.pca_scatter(adata_cc_genes, color=['phase','cell_type3'])
```

Example 65 (unknown):
```unknown
... storing 'phase' as categorical
```

Example 66 (unknown):
```unknown
CPU times: user 7h 15min 49s, sys: 12h 43min 29s, total: 19h 59min 19s
Wall time: 2h 6min 26s
```

Example 67 (python):
```python
# save regressed out object
adata_reg.write('/nfs/team205/heart/anndata_objects/Foetal/Immune/RNA_foetal_raw_Lymphoid_16Feb2023_CCregressed-scaled.h5ad')
adata_reg
```

Example 68 (unknown):
```unknown
AnnData object with n_obs × n_vars = 21336 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', '_scvi_labels', '2Jan_coarse_grain', '2Jan_mid_grain', '2Jan_fine_grain', 'leiden_scVI', 'FACSgate', 'sex', 'week', 'trimester', 'cell_type2', 'cell_type3', 'S_score', 'G2M_score', 'phase'
    var: 'gene_name_scRNA-0', 'gene_id', 'mean', 'std'
    uns: 'log1p'
    obsm: 'X_pca', 'X_scVI', 'X_umap', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 69 (python):
```python
adata_reg = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/Immune/RNA_foetal_raw_Lymphoid_16Feb2023_CCregressed-scaled.h5ad')
adata_reg
```

Example 70 (unknown):
```unknown
AnnData object with n_obs × n_vars = 21336 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', '_scvi_labels', '2Jan_coarse_grain', '2Jan_mid_grain', '2Jan_fine_grain', 'leiden_scVI', 'FACSgate', 'sex', 'week', 'trimester', 'cell_type2', 'cell_type3', 'S_score', 'G2M_score', 'phase'
    var: 'gene_name_scRNA-0', 'gene_id', 'mean', 'std'
    uns: 'log1p'
    obsm: 'X_pca', 'X_scVI', 'X_umap', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 71 (python):
```python
adata_nc = adata_reg[adata_reg.obs['cell_type3'].isin(['CD4+T',
                                                        'CD4+Treg',
                                                        'CD8+T',
                                                        'NK', 
                                                        'ILC',
                                                        ])]
adata_nc.obs['cell_type3'].value_counts()
```

Example 72 (unknown):
```unknown
CD4+T       5206
NK          2437
CD8+T       2121
CD4+Treg    1940
ILC         1639
Name: cell_type3, dtype: int64
```

Example 73 (python):
```python
# downsize for training
adata_nc = sctk.subsample(adata_nc, 
                     fraction=1, 
                     groupby='cell_type3', 
                     min_n=0, max_n=1000, method='random', index_only=False, random_state=0)
adata_nc.obs['cell_type3'].value_counts()
```

Example 74 (unknown):
```unknown
CD4+T       1000
CD4+Treg    1000
CD8+T       1000
NK          1000
ILC         1000
Name: cell_type3, dtype: int64
```

Example 75 (python):
```python
%%time
# train
lr = sctk.LR_train(adata_nc, groupby='cell_type3', use_rep='X',)
```

Example 76 (unknown):
```unknown
CPU times: user 2h 23min 55s, sys: 11.5 s, total: 2h 24min 7s
Wall time: 2h 27min 15s
```

Example 77 (unknown):
```unknown
LogisticRegression(C=0.1, n_jobs=-1, solver='saga', warm_start=True)
```

Example 78 (python):
```python
import joblib

outdir = os.path.dirname('/home/jovyan/projects/P37_Foetal-heart_scnRNA/notebooks/v20230207_withShinhalab/LR/foetal_heart_T-NK-ILC_wo-cycling')
if outdir:
    os.makedirs(outdir, exist_ok=True)
joblib.dump(lr, '/home/jovyan/projects/P37_Foetal-heart_scnRNA/notebooks/v20230207_withShinhalab/LR/foetal_heart_T-NK-ILC_wo-cycling')
```

Example 79 (unknown):
```unknown
['/home/jovyan/projects/P37_Foetal-heart_scnRNA/notebooks/v20230207_withShinhalab/LR/foetal_heart_T-NK-ILC_wo-cycling']
```

Example 80 (python):
```python
adata_cyc = adata_reg[adata_reg.obs['cell_type3']=='T_cycling']
adata_cyc
```

Example 81 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 912 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', '_scvi_labels', '2Jan_coarse_grain', '2Jan_mid_grain', '2Jan_fine_grain', 'leiden_scVI', 'FACSgate', 'sex', 'week', 'trimester', 'cell_type2', 'cell_type3', 'S_score', 'G2M_score', 'phase'
    var: 'gene_name_scRNA-0', 'gene_id', 'mean', 'std'
    uns: 'log1p'
    obsm: 'X_pca', 'X_scVI', 'X_umap', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 82 (python):
```python
result = sctk.LR_predict(adata_cyc, lr, use_rep='X',return_predict=True)
```

Example 83 (unknown):
```unknown
36601 features used for prediction
```

Example 84 (python):
```python
print(result['label'][:5])
result['prob'].head()
```

Example 85 (unknown):
```unknown
['ILC' 'ILC' 'ILC' 'NK' 'NK']
```

Example 86 (python):
```python
adata.obs['LR']=np.nan
adata.obs.loc[result['prob'].index,'LR']=result['label']
adata.obs['LR'].value_counts()
```

Example 87 (unknown):
```unknown
CD4+T       454
CD8+T       176
CD4+Treg    156
ILC          80
NK           46
Name: LR, dtype: int64
```

Example 88 (python):
```python
sc.settings.set_figure_params(dpi=200, facecolor='white', color_map = 'RdPu',)
sc.pl.umap(adata, color=['cell_type3','LR'], legend_fontsize=5)
```

Example 89 (unknown):
```unknown
... storing 'LR' as categorical
```

Example 90 (python):
```python
Tcycling_ids = adata.obs_names[adata.obs['LR'].isna()==False]

adata.obs['cell_type4'] = adata.obs['cell_type3'].copy()
adata.obs['cell_type5'] = adata.obs['cell_type3'].copy()
adata.obs['cell_type4'] = adata.obs['cell_type4'].astype('str')
adata.obs['cell_type5'] = adata.obs['cell_type5'].astype('str')
adata.obs.loc[Tcycling_ids,'cell_type4'] = adata.obs.loc[Tcycling_ids,'LR'].astype('str')+'_cycling'
adata.obs.loc[Tcycling_ids,'cell_type5'] = adata.obs.loc[Tcycling_ids,'LR']
adata.obs['cell_type4'].value_counts()
```

Example 91 (unknown):
```unknown
CD4+T               5206
B_MS4A1+            3722
NK                  2437
CD8+T               2121
CD4+Treg            1940
B                   1666
ILC                 1639
Bpro                 850
B_cycling            843
CD4+T_cycling        454
CD8+T_cycling        176
CD4+Treg_cycling     156
ILC_cycling           80
NK_cycling            46
Name: cell_type4, dtype: int64
```

Example 92 (python):
```python
markers={
     'T cell': ['CD3E', 'CD3D','CD4', 'CD8A', 'CD8B'],
    'CD4T_reg': ['FOXP3', 'CTLA4', 'IL2RA', 'IKZF2','PDCD1','TIGIT'],
    'cytotoxic': ['PRF1', 'GZMB','NKG7'],
    'NK': ['KLRF1', 'KLRD1','NCAM1','FCGR3A','IFNG'],
    'ILC': ['KIT', 'IL7R','IL18R1','RORC','IL17A']
}


for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in adata.var_names]

sc.pl.dotplot(adata,
              markers, 
              groupby='cell_type4',
              dendrogram=False,
              standard_scale="var",
              color_map="Reds",
              # swap_axes=True,
             )
```

Example 93 (python):
```python
markers={
     'T cell': ['CD3E', 'CD3D','CD4', 'CD8A', 'CD8B'],
    'CD4T_reg': ['FOXP3', 'CTLA4', 'IL2RA', 'IKZF2','PDCD1','TIGIT'],
    'cytotoxic': ['PRF1', 'GZMB','NKG7'],
    'NK': ['KLRF1', 'KLRD1','NCAM1','FCGR3A','IFNG'],
    'ILC': ['KIT', 'IL7R','IL18R1','RORC','IL17A']
}


for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in adata.var_names]

sc.pl.dotplot(adata[adata.obs['cell_type5'].isin(['CD4+T','CD4+Treg','CD8+T','ILC','NK'])],
              markers, 
              groupby='cell_type5',
              dendrogram=False,
              standard_scale="var",
              color_map="Reds",
              # swap_axes=True,
             )
```

Example 94 (python):
```python
adata_Tcycling = adata[Tcycling_ids]
adata_Tcycling.shape
```

Example 95 (unknown):
```unknown
(912, 36601)
```

Example 96 (python):
```python
adata_Tcycling.obs['cell_type5'].value_counts()
```

Example 97 (unknown):
```unknown
CD4+T       454
CD8+T       176
CD4+Treg    156
ILC          80
NK           46
Name: cell_type5, dtype: int64
```

Example 98 (python):
```python
FineGrainKey='cell_type5'
for k in list(set(adata_Tcycling.obs[FineGrainKey])):
    k_mod = k.replace('+','')
    np.savetxt('/nfs/team205/heart/anndata_objects/Foetal/Immune/Tcycling/'+k_mod+'.txt', adata_Tcycling[adata_Tcycling.obs[FineGrainKey]==k,:].obs_names, delimiter="\n", fmt="%s")
    print(k)
```

Example 99 (unknown):
```unknown
NK
CD4+T
CD4+Treg
CD8+T
ILC
```

Example 100 (unknown):
```unknown
rclone copy /nfs/team205/heart/anndata_objects/Foetal/Immune/Tcycling \
gdrive:JamesC_Kazumasa/Foetal_Heart/SanjayShinha/toSinhalab/anndata/Immune_16Feb2023/Tcycling
```

Example 101 (python):
```python
# load model and predict
panfetal_model = models.Model.load('/nfs/team205/kk18/Analysis/celltypist/models/celltypist_model.Pan_Fetal_Human.pkl')
sc.pp.neighbors(adata, use_rep="X_scVI")
prediction=celltypist.annotate(adata, model = panfetal_model, majority_voting = True)
```

Example 102 (unknown):
```unknown
🔬 Input data has 21336 cells and 28811 genes
🔗 Matching reference genes in the model
🧬 6669 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
👀 Detected a neighborhood graph in the input object, will run over-clustering on the basis of it
⛓️ Over-clustering input data with resolution set to 15
🗳️ Majority voting the predictions
✅ Majority voting done!
```

Example 103 (unknown):
```unknown
celltypist.dotplot(prediction, use_as_reference = 'cell_type3', use_as_prediction = 'predicted_labels')
```

Example 104 (python):
```python
import seaborn as sns
plt.rcParams['figure.figsize'] = 12,5
df=pd.crosstab(adata.obs['cell_type3'],adata.obs['2Jan_fine_grain'],normalize='columns')
sns.heatmap(df)
```

Example 105 (unknown):
```unknown
<AxesSubplot:xlabel='2Jan_fine_grain', ylabel='cell_type3'>
```

Example 106 (python):
```python
import seaborn as sns
plt.rcParams['figure.figsize'] = 12,5
ad=adata[adata.obs['2Jan_fine_grain']!='nan']
df=pd.crosstab(ad.obs['cell_type3'],ad.obs['2Jan_fine_grain'],normalize='index')
sns.heatmap(df)
```

Example 107 (unknown):
```unknown
<AxesSubplot:xlabel='2Jan_fine_grain', ylabel='cell_type3'>
```

---

## Tutorial on single-cell trajectory alignment using Genes2Genes¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig4_Cardiomyocytes/AtrialCardiomyocytes/g2g_L_v_RaCM_AUC.html

**Contents:**
- Tutorial on single-cell trajectory alignment using Genes2Genes¶
- Make AUC objects¶
- Split into REF and QUERY¶
  - Min max normalize the pseudotime¶
  - Check the number of bins in the optimal binning structure using OptBinning package¶
  - Visualize the interpolation binning structure in terms of the cell type composition¶
  - Run G2G alignment¶
  - The alignment distribution across all genes¶
  - Run gene set overrepresentation analysis over the top k mismatching genes¶
  - Clustering genes using their alignments¶
  - Below is the Levenshtein distance heat map of all genes ordered based on the above clustering structure¶

G2G aims to guide downstream comparative analysis of single-cell reference and query systems along any axis of progression (e.g. differentiation pseudotime, disease/treatment response pseudotime etc.). This notebook describes how we can use G2G framework to infer and analyse gene-level trajectory alignments between a given reference and query dataset.

In this tutorial, we are going to compare trajectories between two treatment groups (PAM and LPS) of mouse bone marrow-derived dendritic cells from Shalek et al (2014). The single cell datasets and their pseudotime estimates were downloaded from https://github.com/shenorrLab/cellAlign (Alpert et al 2018) and packaged into adata objects. There are 2 gene modules: global (core antiviral module) and local (peaked inflammatory module) considered by Alpert et al (2018), and we use the local module as an example comparison.

Load the reference and query anndata objects

At this point we have 4 objects

each has a per cell value (expression or score) for different types of features, and each has the same cells (with psudotemporal ordering in t)

OptBinning estimates 14 optimal number of splits for both reference and query pseudotime distributions. Therefore we choose the same number of interpolation points.

For this dataset, we use the author-given time annotations (1h,2h,4h,6h) as the cell-type annotations.

Next we define a colormap of our choice for these annotations, and call the below function.

This is done by first creating an aligner object, passing and setting all relevant parameters. Next we align all gene pairs. (This step is parallelizing indepedenent gene-alignments to make the process time-efficient, however the computational time for an individual alignment will increase as the number of cells and/or the number of interpolation time points increase.

Now we can check the aggregate (average) alignment across all genes:

We can also visualize this alignment in terms of cell-type composition

We can also visualize an individual gene (e.g. JUNB), displaying its alignment statistics

We can use the alignment similarity percentage statistic of genes to rank genes from highly distant to highly similar

Let us use 30% alignment similarity (=0.3) as a threshold in this case

We first run cluster diagnostics to decide on a distance threshold with a good tradeoff between the number of clusters and the quality of structure. We use levenshtein distance metric

We can inspect structures of distance thresholds that give local optimals of the mean silhouette score.

If we select distance threshold 0.37 which gives a local optimal of 0.3749 silhouette_score with 7 clusters

Below visualizes all alignment paths in each cluster along with its number of genes

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import gseapy as gp

def GO_plot(gene_list, gene_sets=['MSigDB_Hallmark_2020', 'KEGG_2021_Human', 'GO_Biological_Process_2023', 'Reactome_2022'], 
            display_table=False, title_suffix=None, sort_by='Combined Score', n_rank = 10, n_genes=None, FDR=None):
    """
    Generate GO plots and optionally display tables based on enrichment analysis.

    Parameters:
    - gene_list: List of gene names as strings
    - gene_sets: List of gene set names as strings
    - display_table: Boolean flag to display table or not
    - title_suffix: Suffix for the plot title
    - sort_by: Column name to sort the results and plot. Other options in table, e.g. 'n_genes'
    - n_rank: number of terms to include in plot
    """
    
    # Perform enrichment analysis
    enr_res = gp.enrichr(gene_list=gene_list,
                         gene_sets=gene_sets,
                         organism='human',
                         outdir=None)
    
    # Loop through each gene set to process and plot the results
    for geneset in gene_sets:
        print(geneset)
        
        # Filter the results based on the Gene_set
        df = enr_res.results[enr_res.results['Gene_set'] == geneset]
        
        # Calculate the number of genes in the overlap and add as a new column
        df['n_genes'] = [int(x.split('/')[0]) for x in df['Overlap']]
        
        # Sort the dataframe based on the specified column
        df = df.sort_values(sort_by, ascending=False)

        # Only use terms where a threshold number of genes are present
        if n_rank is not None:
            df = df[df.n_genes>=n_genes]

        if FDR is not None:
            df = df[df['Adjusted P-value']<FDR]
        
        # Optionally display the table
        if display_table:
            display(df[:30])
        
        # Generate the bar plot for the top 10 ranked terms
        plt.rcParams['axes.grid'] = False
        plt.rcParams['figure.figsize'] = 4, 3
        plt.barh(width=df[:n_rank][sort_by],
                 y=[x.split(' (')[0] for x in df[:n_rank]['Term']],
                 color='darkred')
        plt.gca().invert_yaxis()
        plt.xlabel(sort_by)
        if title_suffix:
            plt.title(f'{geneset}_{title_suffix}')
        else:
            plt.title(f'{geneset}')
        plt.margins(y=0.02)
        plt.show()
        
    return df
```

Example 2 (python):
```python
import anndata

def split_adata_by_modality(adata, modality_column, modality_values):
    """
    Splits an AnnData object into multiple AnnData objects based on a given modality column.
    
    Parameters:
        adata (anndata.AnnData): The original AnnData object.
        modality_column (str): The name of the column in adata.var that contains the modality information.
        modality_values (list): List of unique modalities that you want to split the adata into.
        
    Returns:
        dict: A dictionary where the keys are the modality names and the values are the new AnnData objects.
    """
    # Initialize an empty dictionary to hold the new AnnData objects
    splitted_adatas = {}
    
    # Loop through each modality value to create a new AnnData object
    for modality in modality_values:
        # Get the indices of the features belonging to the current modality
        feature_idx = adata.var[modality_column] == modality
        
        # Create a new AnnData object with only the features belonging to the current modality
        new_adata = anndata.AnnData(X=adata[:, feature_idx].X,
                                    obs=adata.obs,
                                    var=adata.var.loc[feature_idx],
                                    obsm=adata.obsm,
                                    uns=adata.uns)
        
        # Add the new AnnData object to the dictionary
        splitted_adatas[modality] = new_adata
    
    return splitted_adatas
```

Example 3 (python):
```python
import scanpy as sc
import anndata
import numpy as np
import seaborn as sb
import matplotlib.pyplot as plt
import platform
from optbinning import ContinuousOptimalBinning
import pandas as pd


from genes2genes import Main
from genes2genes import VisualUtils
from genes2genes import ClusterUtils
from genes2genes import TimeSeriesPreprocessor
from genes2genes import PathwayAnalyserV2

print(platform.python_version())
```

Example 4 (python):
```python
import matplotlib
matplotlib.rcParams['pdf.fonttype'] = 42
matplotlib.rcParams['ps.fonttype'] = 42
```

Example 5 (unknown):
```unknown
temporal_ordering='age_days_smoothed_once'
```

Example 6 (python):
```python
paired_adata_with_time_path = "/lustre/scratch126/cellgen/team205/heart/objects/fetal/RNA/aCMs_WholeSample_paired_with_time.h5ad"
```

Example 7 (python):
```python
adata=sc.read(paired_adata_with_time_path)
adata
```

Example 8 (unknown):
```unknown
AnnData object with n_obs × n_vars = 5664 × 544641
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', 'age_days', 'age_days_smoothed_once', 'age_days_normalized', 'age_days_normalized_smoothed_once', 'age_days_smoothed_iteratively', 'age_days_normalized_smoothed_iteratively', 't', 'seg', 'edge', 't_sd', 'milestones'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id'
    uns: 'donor_colors', 'fine_grain_colors', 'graph', 'milestones_colors', 'neighbors', 'ppt', 'pseudotime_list', 'seg_colors', 'umap', 'week_colors'
    obsm: 'X_R', 'X_multiVI', 'X_umap'
    obsp: 'connectivities', 'distances'
```

Example 9 (python):
```python
adata.obs["time"]=adata.obs['t'].copy()
```

Example 10 (python):
```python
adata.X.data[:5]
```

Example 11 (unknown):
```unknown
array([2., 1., 1., 1., 1.], dtype=float32)
```

Example 12 (python):
```python
sc.pl.umap(adata,color=["seg","milestones","t","fine_grain"],ncols=2)
```

Example 13 (python):
```python
modality_column = "modality"
modality_values = ["Gene Expression", "Peaks"]
splitted_adatas = split_adata_by_modality(adata, modality_column, modality_values)

adata_gex = splitted_adatas["Gene Expression"]
print(adata_gex)

adata_atac = splitted_adatas["Peaks"]
print(adata_atac)
```

Example 14 (unknown):
```unknown
AnnData object with n_obs × n_vars = 5664 × 36601
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', 'age_days', 'age_days_smoothed_once', 'age_days_normalized', 'age_days_normalized_smoothed_once', 'age_days_smoothed_iteratively', 'age_days_normalized_smoothed_iteratively', 't', 'seg', 'edge', 't_sd', 'milestones', 'time'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id'
    uns: 'donor_colors', 'fine_grain_colors', 'graph', 'milestones_colors', 'neighbors', 'ppt', 'pseudotime_list', 'seg_colors', 'umap', 'week_colors'
    obsm: 'X_R', 'X_multiVI', 'X_umap'
AnnData object with n_obs × n_vars = 5664 × 508040
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', 'age_days', 'age_days_smoothed_once', 'age_days_normalized', 'age_days_normalized_smoothed_once', 'age_days_smoothed_iteratively', 'age_days_normalized_smoothed_iteratively', 't', 'seg', 'edge', 't_sd', 'milestones', 'time'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id'
    uns: 'donor_colors', 'fine_grain_colors', 'graph', 'milestones_colors', 'neighbors', 'ppt', 'pseudotime_list', 'seg_colors', 'umap', 'week_colors'
    obsm: 'X_R', 'X_multiVI', 'X_umap'
```

Example 15 (unknown):
```unknown
path_to_auc_gene="/nfs/team205/heart/pyscenic/plus/AUCscore/foetal_aCM_n570_eRegulon_AUC_gene.csv"
path_to_auc_region="/nfs/team205/heart/pyscenic/plus/AUCscore/foetal_aCM_n570_eRegulon_AUC_region.csv"
```

Example 16 (python):
```python
# read in regulon scores
auc_gene = pd.read_csv(path_to_auc_gene, index_col=0)
print(auc_gene.shape)
auc_region = pd.read_csv(path_to_auc_region, index_col=0)
print(auc_region.shape)
```

Example 17 (unknown):
```unknown
(17302, 278)
(12964, 278)
```

Example 18 (python):
```python
# make regulon adata, AUC gene
# AUC Gene
# create anndata
adata_aucg = anndata.AnnData(
    X=auc_gene.loc[adata_gex.obs_names],
    obs=adata_gex.obs,
    var=pd.DataFrame(auc_gene.columns).set_index(0),
    obsm=adata_gex.obsm,
    uns= adata_gex.uns
)
# modify varnames
adata_aucg.var_names = [x.split('_(')[0] for x in adata_aucg.var_names]
print(adata_aucg.shape)

# AUC Region
# create anndata
adata_aucr = anndata.AnnData(
    X=auc_region.loc[adata_gex.obs_names],
    obs=adata_gex.obs,
    var=pd.DataFrame(auc_region.columns).set_index(0),
    obsm=adata_gex.obsm
)
# modify varnames
adata_aucr.var_names = [x.split('_(')[0] for x in adata_aucr.var_names]
print(adata_aucr.shape)
```

Example 19 (unknown):
```unknown
(5664, 278)
(5664, 278)
```

Example 20 (python):
```python
# split intlo REF and QUERY
adata_ref=adata_aucg[adata_aucg.obs.fine_grain=="AtrialCardiomyocytesLeft"].copy()
adata_query=adata_aucg[adata_aucg.obs.fine_grain=="AtrialCardiomyocytesRight"].copy()
```

Example 21 (python):
```python
sc.pl.umap(adata_ref,color=["time","fine_grain"],ncols=2)
sc.pl.umap(adata_query,color=["time","fine_grain"],ncols=2)
```

Example 22 (python):
```python
print(adata_ref)
print(adata_query)
```

Example 23 (unknown):
```unknown
AnnData object with n_obs × n_vars = 3041 × 278
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', 'age_days', 'age_days_smoothed_once', 'age_days_normalized', 'age_days_normalized_smoothed_once', 'age_days_smoothed_iteratively', 'age_days_normalized_smoothed_iteratively', 't', 'seg', 'edge', 't_sd', 'milestones', 'time'
    uns: 'donor_colors', 'fine_grain_colors', 'graph', 'milestones_colors', 'neighbors', 'ppt', 'pseudotime_list', 'seg_colors', 'umap', 'week_colors'
    obsm: 'X_R', 'X_multiVI', 'X_umap'
AnnData object with n_obs × n_vars = 2623 × 278
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', 'age_days', 'age_days_smoothed_once', 'age_days_normalized', 'age_days_normalized_smoothed_once', 'age_days_smoothed_iteratively', 'age_days_normalized_smoothed_iteratively', 't', 'seg', 'edge', 't_sd', 'milestones', 'time'
    uns: 'donor_colors', 'fine_grain_colors', 'graph', 'milestones_colors', 'neighbors', 'ppt', 'pseudotime_list', 'seg_colors', 'umap', 'week_colors'
    obsm: 'X_R', 'X_multiVI', 'X_umap'
```

Example 24 (python):
```python
# check the current range
print(min(adata_ref.obs['time']), max(adata_ref.obs['time']))
print(min(adata_query.obs['time']), max(adata_query.obs['time']))

# if it does not follow [0,1] range, run below
adata_ref.obs['time'] = TimeSeriesPreprocessor.Utils.minmax_normalise(np.asarray(adata_ref.obs['time']))
adata_query.obs['time'] = TimeSeriesPreprocessor.Utils.minmax_normalise(np.asarray(adata_query.obs['time']))
```

Example 25 (unknown):
```unknown
0.4222375414474298 8.245970642037001
0.45019867983228623 8.303595486512195
```

Example 26 (python):
```python
# Visualize the pseudotime distributions
sb.kdeplot(adata_ref.obs['time'], fill=True, color='forestgreen') 
sb.kdeplot(adata_query.obs['time'], fill=True, color='midnightblue'); 
plt.xlabel('pseudotime'); plt.legend(); plt.show()
```

Example 27 (unknown):
```unknown
No artists with labels found to put in legend.  Note that artists whose label start with an underscore are ignored when legend() is called with no argument.
```

Example 28 (python):
```python
x = np.asarray(adata_ref.obs.time)
optb = ContinuousOptimalBinning(name='pseudotime', dtype="numerical")
optb.fit(x, x)
print(len(optb.splits))

x = np.asarray(adata_query.obs.time)
optb = ContinuousOptimalBinning(name='pseudotime', dtype="numerical")
optb.fit(x, x)
print(len(optb.splits))
```

Example 29 (python):
```python
col = np.array(sb.color_palette('colorblind'))[range(2)]
col
```

Example 30 (unknown):
```unknown
array([[0.00392157, 0.45098039, 0.69803922],
       [0.87058824, 0.56078431, 0.01960784]])
```

Example 31 (python):
```python
joint_cmap={'AtrialCardiomyocytesLeft':col[0], 'AtrialCardiomyocytesRight':col[1]}
vs = VisualUtils.VisualUtils.get_celltype_composition_across_time(adata_ref, adata_query, n_points=14, 
                                     ANNOTATION_COLNAME='fine_grain', optimal_binning=False)
```

Example 32 (unknown):
```unknown
# trying max n points for optimal binning = 14
====================================================
Optimal equal number of bins for R and Q =  14
```

Example 33 (python):
```python
gene_list = adata_ref.var_names
print(len(gene_list))
```

Example 34 (unknown):
```unknown
gene_list[:10]
```

Example 35 (unknown):
```unknown
Index(['ARID3A_extended_-', 'ARNT2_+', 'ARNTL_+', 'ARNT_+', 'ATF1_+', 'ATF2_+',
       'ATF3_+', 'ATF6_extended_+', 'ATF7_+', 'BCL6_+'],
      dtype='object')
```

Example 36 (python):
```python
%%time

import os
import pickle

# Path for the folder and file
folder_path = '/nfs/team205/jc48/projects/fetal/code/g2g_aligners'
file_path = os.path.join(folder_path, 'AUCgene_aligner.pkl')

# Create folder if it doesn't exist
os.makedirs(folder_path, exist_ok=True)

# Check if the file already exists
if os.path.isfile(file_path):
    # Read the aligner object from the file
    print("this specific aligner already exists, reading it in...")
    with open(file_path, 'rb') as file:
        aligner = pickle.load(file)
else:
    # If the file does not exist, create and configure the aligner
    print("aligner not already made, making it...")
    aligner = Main.RefQueryAligner(adata_ref, adata_query, gene_list, len(vs.optimal_bining_S))
    aligner.WEIGHT_BY_CELL_DENSITY = True
    aligner.WINDOW_SIZE = 0.1
    aligner.state_params = [0.99, 0.1, 0.7]
    aligner.optimal_binning = True
    aligner.opt_binning_S = vs.optimal_bining_S
    aligner.opt_binning_T = vs.optimal_bining_T

    # Align all pairs
    aligner.align_all_pairs()

    # Save the aligner object to the file
    with open(file_path, 'wb') as file:
        pickle.dump(aligner, file)
```

Example 37 (unknown):
```unknown
this specific aligner already exists, reading it in...
CPU times: user 646 ms, sys: 66.9 ms, total: 713 ms
Wall time: 751 ms
```

Example 38 (unknown):
```unknown
aligner.get_aggregate_alignment()
```

Example 39 (unknown):
```unknown
Average Alignment:  IIIDIDIDDIDDMMMMMMMM
```

Example 40 (unknown):
```unknown
vs.visualize_gene_alignment("IDDDMMMMMMMMMIIIDID")#, cmap=joint_cmap)
```

Example 41 (unknown):
```unknown
df = aligner.get_stat_df()
```

Example 42 (unknown):
```unknown
mean matched percentage: 
60.519999999999996 %
```

Example 43 (unknown):
```unknown
VisualUtils.plot_alignmentSim_vs_l2fc(df)
```

Example 44 (unknown):
```unknown
df[(df["alignment_similarity_percentage"]<0.5)&(df["abs_l2fc"]>0)].sort_values('alignment_similarity_percentage', ascending=True)
```

Example 45 (python):
```python
import seaborn as sns
import matplotlib.pyplot as plt

plt.figure(figsize=(10, 10))


#y_axis = 'opt_alignment_cost'
y_axis = 'alignment_similarity_percentage'

sns.scatterplot(data=df, x='l2fc', y=y_axis, edgecolor="none",legend=False, alpha=0.8)

# Add a horizontal line at y=0.5
plt.axhline(0.5, color='gray', linestyle='--', linewidth = 0.5)

# Add text annotations for LaCM and RaCM
plt.text(0.1, 1.05, 'RaCM', transform=plt.gca().transAxes, ha='left', va='top', weight='semibold')
plt.text(0.9, 1.05, 'LaCM', transform=plt.gca().transAxes, ha='right', va='top', weight='semibold')


# Set labels
plt.xlabel("Log2 Fold Change")

if y_axis == "alignment_similarity_percentage":
    plt.ylabel("Alignment Similarity (%)")

if y_axis == "opt_alignment_cost":
    plt.ylabel("Alignment Cost (nits)")

# Save the plot
#plt.savefig("similarity_v_logFC.pdf", format='pdf', bbox_inches='tight')

# Show the plot
plt.show()
```

Example 46 (python):
```python
query_color = 'orange'
ref_color = 'blue'

def plotTimeSeries_cmap(gene, aligner, plot_cells = False, plot_mean_trend= False):
    
        al_obj = aligner.results_map[gene]
        plt.subplots(1,3,figsize=(15,3))
        plt.subplot(1,3,1)
        plotTimeSeriesAlignment_cmap(gene, aligner) 
        plt.subplot(1,3,2)
        max_val = np.max([np.max(np.asarray(aligner.ref_mat[al_obj.gene])), np.max(np.asarray(aligner.query_mat[al_obj.gene]))])
        min_val = np.min([np.min(np.asarray(aligner.ref_mat[al_obj.gene])), np.min(np.asarray(aligner.query_mat[al_obj.gene]))])
        g = sb.scatterplot(x=aligner.query_time, y=np.asarray(aligner.query_mat[al_obj.gene]), alpha=0.7, color = query_color, legend=False,linewidth=0.3, s=20)  
        plt.title('Query')
        plt.ylim([min_val-0.5,max_val+0.5])
        plt.subplot(1,3,3)
        g = sb.scatterplot(x=aligner.ref_time, y=np.asarray(aligner.ref_mat[al_obj.gene]), color = ref_color, alpha=0.7, legend=False,linewidth=0.3,s=20 ) 
        plt.title('Reference')
        plt.ylim([min_val-0.5,max_val+0.5])

def plotTimeSeriesAlignment_cmap(gene, aligner):  
    
        al_obj = aligner.results_map[gene]
        sb.scatterplot(x=al_obj.S.X, y=al_obj.S.Y, color = ref_color ,alpha=0.05, legend=False)#, label='Ref') 
        sb.scatterplot(x=al_obj.T.X, y=al_obj.T.Y, color = query_color ,alpha=0.05, legend=False)#, label ='Query') 
        plot_mean_trends(al_obj)
        plt.title(al_obj.gene)
        plt.xlabel('pseudotime')
        plt.ylabel('Gene expression')
        plt.axis('off')
        
        for i in range(al_obj.matched_region_DE_info.shape[0]):
            S_timebin = int(al_obj.matched_region_DE_info.iloc[i]['ref_bin'])
            T_timebin = int(al_obj.matched_region_DE_info.iloc[i]['query_bin']) 
            x_vals = [al_obj.matched_region_DE_info.iloc[i]['ref_pseudotime'],al_obj.matched_region_DE_info.iloc[i]['query_pseudotime']] 
            y_vals = [al_obj.S.mean_trend[S_timebin ], al_obj.T.mean_trend[T_timebin]] 
            plt.plot(x_vals, y_vals, color='black', linestyle='dashed', linewidth=1.5)

def plot_mean_trends(al_obj):
    # Plotting the mean trends for S and T with specified colors
    al_obj.S.plot_mean_trend(color=ref_color)
    al_obj.T.plot_mean_trend(color=query_color)
```

Example 47 (python):
```python
import matplotlib.pyplot as plt
from contextlib import redirect_stdout
import io


for gene in ['ETS2_-','ETS2_+','ETV6_+','GATA5_+','GATA4_+','THRA_+','PITX2_+','NKX2-5_extended_+']:
    print(gene)
    # Redirect the output of the plot function to a StringIO object
    f = io.StringIO()
    with redirect_stdout(f):
        #VisualUtils.plotTimeSeries(gene, aligner, vs)
        #VisualUtils.plotTimeSeriesAlignment(gene, aligner)
        plotTimeSeriesAlignment_cmap(gene, aligner)

    
    # Now, the plot is created and can be saved
    plt.savefig(f"{gene}_time_series_alignment.png", format='png', bbox_inches='tight')
    plt.show()
```

Example 48 (unknown):
```unknown
NKX2-5_extended_+
```

Example 49 (unknown):
```unknown
lowSimGenes = df[list(df['alignment_similarity_percentage'] < 0.5)]
topSimGenes = df[list(df['alignment_similarity_percentage'] > 0.35)]
```

Example 50 (unknown):
```unknown
# Running experiment to determine the distance threshold with a good trade-off
df = ClusterUtils.run_clustering(aligner, metric='levenshtein', experiment_mode=True)
```

Example 51 (unknown):
```unknown
compute distance matrix
using levenshtein distance metric
```

Example 52 (unknown):
```unknown
74%|██████████████████████████████████████████████████████████████████████████████████████████████████                                   | 73/99 [00:00<00:00, 82.46it/s]
```

Example 53 (unknown):
```unknown
df.sort_values('Distance threshold', ascending=False)
```

Example 54 (unknown):
```unknown
aligner.show_ordered_alignments()
```

Example 55 (unknown):
```unknown
Gene               Alignment
-----------------  ----------------------------
ETS2_+             IIIDIDIDIDIIIIIIIIDDDDDDDDDD
ETS2_-             IIIDIDIDIDIIIIIIIIDDDDDDDDDD
ETV6_+             IIIDIDIDIDIIIIIIIIDDDDDDDDDD
MAF_-              IIIDIDIDIDIIIIIIIIDDDDDDDDDD
PITX2_+            IIIDIDIDIDIIIIIIIIDDDDDDDDDD
E4F1_+             MMMMMMMMMMMMMM
FOXN2_+            MMMMMMMMMMMMMM
IRF7_extended_+    MMMMMMMMMMMMMM
MLX_+              MMMMMMMMMMMMMM
NFYA_+             MMMMMMMMMMMMMM
PHF21A_extended_+  MMMMMMMMMMMMMM
TBP_+              MMMMMMMMMMMMMM
TFAP4_+            MMMMMMMMMMMMMM
TFCP2_+            MMMMMMMMMMMMMM
TFDP2_+            MMMMMMMMMMMMMM
ZBTB20_+           MMMMMMMMMMMMMM
ZNF200_+           MMMMMMMMMMMMMM
ZNF709_+           MMMMMMMMMMMMMM
ZNF526_+           MMWWWWWWIIIIIMMMMMIID
ZNF337_+           MVVVVVVDDDDDDMMMMMMM
PATZ1_+            MMMMMMIDIDMMMMMM
ZNF267_+           MMMVVVDIDDDMMMMMMM
ZNF639_+           MMMMMMIDIDMMMMMM
ZNF394_+           MMMMMIDMMMMMMMM
ZNF676_+           MMMMMIDIDIDMMMMMM
ARNT2_+            MMMMIIDDIDMMMMMMM
ARNT_+             MMMMDDMMMMMMMMII
CEBPZ_+            MMMMDDMMMMMMMMII
CREB3_+            MMMMIIDDMMMMMMMM
FOS_+              MMMMIDMMMMMMMMM
GATA4_+            MMMMIDIDIDMMMMMMM
GATA4_-            MMMMIIDDIDMMMMMMM
GLIS3_-            MMMMIDDMMMMMIMMM
HES1_+             MMMMDDMMMMMMMMVV
HES6_extended_+    MMMMIDIDIDMMMMMMM
HMGA1_+            MMMMDDMMMMMMMMII
IRF3_+             MMMMIDIDMMMMMMMM
KLF11_+            MMMMDDMMMMMMMIIM
KLF2_+             MMMMDDMMMMMMMMVV
PHF20_+            MMMMDIDMMMMMMMMI
SALL2_+            MMMMIDIDMMMMMMMM
SIX4_extended_+    MMMMIIDDMMMMMMMM
TCFL5_+            MMMMIDMMMMMMMMM
ZBTB41_extended_+  MMMMDDMMMMMMMMVV
ZFP90_+            MMMMIIDDIDMMMMMMM
ZNF134_+           MMMMDDMMMMMMMMVV
ZNF135_+           MMMMDDMMMMMMMMVV
ZNF142_+           MMMMIIDDMMMMMMMM
ZNF429_+           MMMMDDMMMMMMMMVV
ZNF552_+           MMMMIIDDMMMMMMMM
ZNF571_+           MMMMIDIDIDMMMMMMM
ZNF610_+           MMMMIDIDMMMMMMMM
ZNF91_+            MMMMDDMMMMMMMMVV
ATF1_+             MMMDDDDMMIIIIMMMMM
ATF3_+             MMMIDDIDIDMMMMMIMM
CREBZF_extended_+  MMMIDIDIDMMMMMMMM
ELF2_+             MMMDIDDMMMMMMMMII
ETV5_+             MMMDDDMMMMMMMMIII
GMEB1_+            MMMIIDDMMMMMMMMM
IRF2_+             MMMDDIDMMMMMMMMVV
JUNB_+             MMMIDMMMMMMMMMM
MECP2_+            MMMDDIDMMMMMMMMII
SRF_+              MMMIDDDMMMMMMMMVV
TEAD2_+            MMMDDDMMMMMMMMVVV
TEAD4_+            MMMIDIDIDIDDMMMMMIID
TFDP1_+            MMMDDDMMMMMMMMVVV
ZBTB11_+           MMMDDDMMMMMMMMIII
ZBTB25_+           MMMDDMMMMMMMMMII
ZBTB5_+            MMMIDIDIDIDIDMMMMMM
ZFP62_+            MMMIIIDDDIDMMMMMMM
ZNF143_+           MMMDDMMMMMMMMMII
ZNF260_+           MMMIDIDIDMMMMMMMM
ZNF646_+           MMMDDDMMMMMMMMVVV
ZNF721_+           MMMDDDMMMMMMMMIII
HAND2_+            MMDMIIDIDIDIDMMMMMM
JUN_+              MMIIDDDDIDIDMMMMMMII
MXD4_+             MMIIIMMDDIDIDDMMMMM
NR3C1_+            MMDDDMMMMMMMMMIII
SMAD3_+            MMDDDDDDMMIIIIMMMIIID
STAT5B_+           MMIIDDMMMMMMMMMM
TBX20_+            MMDDDDIDIDIDMMMMMIIII
ZBTB1_extended_+   MMDDIDMMMMMMMMMVV
ZBTB7B_+           MMDIDIDDMMMMMMIIMM
ZFP14_+            MMIDIDIDIIDDMMMMMMM
ZNF121_+           MMIMMDIDMMMMMMMM
ZNF441_+           MMDDDDMMMIMMMMMIII
ZNF517_+           MMDDDMMMMMMMMMVVV
ZNF566_+           MMDDDDMMIIIIMMMMMM
ZNF641_+           MMIIDIDIDIDIDDMMMMMM
ATF2_+             MDDDDMMMIIIIMMMMMM
BHLHE41_+          MIIIVVVVIIDIIIIDDDDDDDDDDDD
CARF_+             MIIMMMDDMMMMMMMM
CREB1_+            MIIMMDDMMMMMMMMM
NCOA1_+            MIIIMMMDDDMMMMIIIDDD
OVOL2_+            MIIMMDDMMMMMMMMM
SOX4_+             MDDMMIDIDIDIDDMMMMIII
TFEB_+             MIIIIMMMDIDDDDMMMMM
ZNF235_+           MDMMDMMMMMMMMMVV
ZNF454_+           MIIMMMDDMMMMMMMM
ZNF518A_+          DMMMMMMMMMMMMMI
MEIS1_+            DMMMMMMMMMMMIMM
GLIS2_+            IMMMMMDMMMMMMMM
LIN28B_+           IMMMMDIDDMMMMMIMM
RFX2_+             IMMMMDIDIDMMMMMMM
TCF7L1_+           IMMMMDIDMMMMMMMM
ZNF44_+            IMMMMDIDMMMMMMMM
IRF9_+             IMMMDDMMMMMMMIMM
MEF2A_+            IMMMDDDMMMMMMIIMM
NFIC_-             DMMMIIDIDMMMMMMMM
CUX1_+             IMMDIDDDMMMMMMIIMM
FOSL2_+            IMMIIDIDIIDDDDDDMMMMII
NFE2L2_+           IMMDDDMMMMMMMMMII
PLAGL1_extended_+  DMMDIDIDMMMMMMMMVV
PLAGL2_+           IMMDDDDMMMMMMMMVVV
RARA_+             DMMDDMMMMMMMMMIII
RARG_+             IMMDIDDDMMMMMMMMVV
ZBTB10_+           DMMIDIDIDIIIIDDDMMMMM
ZNF117_+           IMMDDMMMMMMMMIMM
ZNF28_+            IMMDDDDMMMMMMMMVVV
ZNF248_+           IDMMMMMMMMMMMMM
ZNF273_+           IDMMMMMMMMMMMMM
ZNF777_+           IDMMMMMMMMMMMMM
ZNF579_+           DDMMMMMMMMMMMMII
STAT4_+            DIVVVVVVDDDDDDMMMMMMM
CREB3L4_+          IIMMMMDDMMMMMMMM
EGR1_+             IIMMMMDDMMMMMMMM
MTF1_+             IIMMMMDDIDMMMMMMM
PEG3_+             DDMMMMIDIDIIMMMMMM
SP1_+              IIMMMMDDMMMMMMMM
STAT2_+            IIMMMMDDMMMMMMMM
ZNF671_+           IIMMMMDDMMMMMMMM
ZNF814_+           IIMMMMDIDDMMMMMMM
E2F4_+             IDMMMDDMMMMMMMMVV
EGR2_+             IDMMMIDIDMMMMMMMM
JDP2_extended_+    DDMMMIIIDIDMMMMMMM
KLF13_+            IIMMMDIDDMMMMMMMM
KLF15_+            IDMMMDDMMMMMMMMII
NFYC_+             IDMMMDDMMMMMMIIMM
NR2C2_+            IDMMMDDMMMMMMMMII
ZFHX2_+            IIMMMDIDIDDMMMMMMM
ZNF281_+           IDMMMDDMMMMMMMMVV
ZNF557_+           IDMMMDDMMMMMMMMVV
ZSCAN18_+          IDMMMDIDMMMMMIMMM
CREB5_+            IDMMIDIDIDDDDMMMMMIII
MEIS2_+            IDMMIIDIDIDDMMMMMMM
MXD1_+             IDMMIDIDIIDDMMMMMMM
NFIB_+             IDMMIIDIDDMMMMMMMM
PBX1_+             DDMMIIDIDIIIDDMMMMMM
ZGPAT_+            DDMMIIDIDIMMMMMMMM
ZNF382_+           IIMMDDMMMMMMMMMM
ZSCAN30_+          IIMMDIDDDDMMMMMMMVV
KLF10_+            DDMIDIDIDMMMMMMMMVV
PRRX1_+            DDMIDIDIDIIMMMMMMMM
ZNF564_+           IDDMMMMMMMMMMMMI
CREB3L2_+          IDDMMMMMMMMMMIMM
ETV1_+             IDDMMMMMMMMMMIMM
ZNF654_+           IDDMMMMMMMMMMIMM
ZBTB8A_+           IDDMMMMMMIMMMMMM
BHLHE40_+          IDDMMMMIIIDDMMMMMM
MAZ_+              IIDMMMMIIDDDMMMMMM
ZNF644_+           IIIMMMMDDWWWIIMMMMM
ZNF780A_+          IIIMMMMDDDMMMMMMM
ZNF783_+           IIIMMMMDDDMMMMMMM
FOSB_+             IIDMMMDMMMMMMMMM
POU2F1_+           IDDMMMIMMMMMMMMM
PRDM10_+           IIDMMMIDDMMMMMMMM
RFX1_+             IDDMMMIIDIDMMMMMMM
RXRG_+             IDDMMMIIDMMMMMMMM
SP4_+              IIIMMMDDDMMMMMMMM
TCF7L2_+           IDDMMMIDIIDMMMMMMM
ZBTB44_+           IDDMMMIIDIIDDMMMMMM
ZNF530_+           IDDMMMIIDMMMMMMMM
ARNTL_+            IDDMMDDDMIIMMMMMMII
ATF7_+             IDDMMIDIDMMMMMIMMM
GMEB2_+            IDDMMDMMMMMMMMMII
KLF15_-            IDDMMIIDIDDDMMMMMMVV
MAX_+              IDDMMDDMMMMMMMMVVV
MBD2_+             IIDMMDDMMMMMMMMMI
NFE2L1_+           IDDMMIIDIDIIDDMMMMMID
NFIA_+             IDDMMIIDMMMMMMMMM
NKX2-5_extended_+  IDDMMDDMMMMMMMMVVV
NPAS2_extended_+   IDDMMDIDMMMMMMMMVV
PRDM6_+            IDDMMIIDIDMMMMMMMM
ZNF23_+            DDDMMIIIDIMMMMMMMM
ZNF25_+            IDDMMDMMMMMMMMMII
ZNF385D_-          IDDMMDIDMMMMMMIIMM
ZNF407_+           IDDMMDDMMMMMMMMIII
ZNF444_+           IIDMMDDDMMMMMMMIIM
ZNF561_+           IDDMMDDMMMMMMMMVVV
ZNF629_+           IDDMMDIDMMMMMMMMII
ZNF35_+            IDDMIDIDIIIDDMMMMMMM
ZNF362_+           DDDMIIDIDIIMMMMMMMM
MYCN_-             IDIDMMMMMMMMMMMM
ELF1_+             DDDDMMMMMIMMMMMIII
TCF3_+             DDDIVVVVVDDDMMMMMMMM
ZNF354C_+          DDDDMMMMMIIIIMMMMM
KLF3_+             IIIDMMMMDDMMMMMMM
RXRB_+             IIIDMMMMDDMMMMMMM
ZNF425_+           IIIDMMMMDDMMMMMMM
ZNF730_+           DDDIVVVVIIDDDDMMMMMMM
ATF6_extended_+    IIIIMMMIIDIDDDDDDMMMM
BCL6_+             IIIDMMMDDMMMMMMMM
KLF16_+            IIIDMMMIDIDDDMMMMMM
NFATC1_+           DDIDMMMIIMMMMMMMM
YY1_+              IDIDMMMIIDIDDMMMMMM
ZBTB7A_+           IDIDMMMIDIIDDMMMMMM
ZNF100_+           IIIDMMMIDDDMMMMMMM
ZNF214_+           IIIDMMMDDMMMMMMMM
ZNF511_+           IDIDMMMIDIIDDMMMMMM
ZNF846_-           IIIDMMMIDDDMMMMMMM
SOX5_extended_+    IDIDMMDDDMMMMMMMIII
ZNF48_+            IDIDMMDDMMMMMMMMVV
ZNF512_+           IDIDMMIIDDMMMMMMMM
ZNF470_+           DDIDMIIDIIIDDMMMMMMM
ZNF711_+           DDDIDMMMMMMMMMMVVV
ZBTB16_+           IDDIDMMMMMMMMMIMM
IRF4_+             DDDDDMMMMMIIIMMMMII
SREBF2_+           DDDDDMMMMIIMMMMMVVV
GTF2IRD1_+         IIIIIMMMIDIIIIIDDDDDDDDDD
MGA_+              IIDIDMMMDMMMMMMMM
PURA_+             DDDDDMMMIIIIIMMMMMM
RELA_+             IIIDIMMMIDDDDMMMMMM
SATB1_+            IIIDIMMMDDDMMMMMMM
VEZF1_+            IIIDIMMMDDDMMMMMMM
ZBTB43_extended_+  IIIDIMMMDDDMMMMMMM
ZNF555_+           DDDIDMMMIIIIDMMMMMM
ZNF770_+           IIDIDMMMIDIDDMMMMMM
FOXJ2_+            DDDIDMMIIIMMMMMMMM
HMGA1_-            IIDIDMMDDMMMMMMIMM
THRB_+             IDDIDMMIMMMMMMMMM
ZNF223_+           IIIDIMMDDDMMMMMMMM
IKZF2_+            DDDDDMIIDIIIIIDDDDMMIIIID
TRPS1_+            IDDIDMDIDMMMMMMMMII
ELK4_+             IDIDIDMMMMMMMMMMM
CEBPD_+            IIIDIDMMMDDMMMMMMM
FOS_-              DDDDDDMMMIIIIMMMMMII
MESP1_extended_+   DDDDDDMMMIIIIIIMMMMM
NR2F1_+            DDDIDDMMMIIIIMMMMMM
TEAD3_+            DDDDDDMMMIIIMMMMMIII
GBX1_extended_+    IIIDIDMMDDMMMMMMMM
MYCN_+             IIIIIIMMDDDDDMMMMMMD
THRA_+             IIIDIIMMDDDDMMMMMMM
ZNF432_+           IIIIIIMMDDDDDDMMMMMM
ZNF43_+            IIIDIDMMDDMMMMMMMM
ZNF503_+           DDDIDDDMMMIIIIIMMMMM
MAF_+              IIIIIIIMMIIDDDDDMMMDDDD
SMAD9_+            IIDIDIDMIIIDDDDMMMMMM
ZNF737_-           IIDIDIDMIDDMMMMMMMM
PRDM16_+           DDIDIDIDMMMMMMMMMVV
E2F5_+             IDIDIDIDMMMMMMMMMM
MAFG_+             IDIDIDIDMMMMMMMMMM
MBD1_+             DDDDDDDIVVVVVIMMMMMMM
ZNF596_+           DDDIDDDDMMMIIIIMMMIIID
ZBED6_extended_+   DDDIDIDIDMMMMMMMMVVV
ZNF202_+           DDDIDIDIDMMMMMMMMVVV
MEF2B_+            DDDDDDDDIVVVVVVIMMMMMM
SREBF1_+           DDIDIDIDIDMMMMMMMMVV
TGIF1_+            DDIDIDIDIDMMMMMMMMVV
MEF2A_-            IDIDIDIDIDMMMMMMMMM
ZSCAN9_+           IDIDIDIDIDMMMMMMMMM
NFIC_+             DDIDIDIDIDMMMMMMMMII
ZNF383_-           IDIDIDIDIDMMMMMMIDMM
ZNF207_+           DDIDIDIDIDDMMMMMMMVVV
HMGA2_+            IDIDIDIDIDDMMMMMMMMI
RUNX1_+            DDIDIDIDIDDMMMMMMMIII
ARID3A_extended_-  IIIDIDIDIDDDMMMMMMMM
MEF2C_+            IIIDIDIDIDDDMMMMMMMM
ZNF148_+           IIIDIDIDIDDDMMMMMMMM
ZNF540_+           IIIDIDIDIDDDMMMMMMMM
ZNF778_+           IIIDIDIDIDDDMMMMMMMM
KLF6_+             IIIDIDIDIDIDDDMMMMMMM
ZNF462_+           IIIDIDIDIDIDDDMMMMMMM
ZNF669_+           IIIDIDIDIDIDDDMMMMMMM
E2F6_+             IIIDIDIDIDIIDDDDMMMMMM
ZNF219_+           IIIDIDIDIDIIDDDDMMMMMM
ZNF549_+           IIIDIDIDIDIIDDDDMMMMMM
SIX5_extended_+    IIIDIDIDIDIIIDDDDDMMMMM
GATA5_+            IIIDIDIDIDIIDDDDDDDMMIIIID
```

Example 56 (unknown):
```unknown
31              ETS2_+
32              ETS2_-
35              ETV6_+
78               MAF_-
119            PITX2_+
            ...       
24              E2F6_+
203           ZNF219_+
242           ZNF549_+
139    SIX5_extended_+
44             GATA5_+
Name: genes, Length: 278, dtype: object
```

Example 57 (unknown):
```unknown
ClusterUtils.run_clustering(aligner, metric='levenshtein', DIST_THRESHOLD=0.5)
```

Example 58 (unknown):
```unknown
compute distance matrix
using levenshtein distance metric
run agglomerative clustering |  0.5
silhouette_score:  0.1830240565803321
```

Example 59 (unknown):
```unknown
ClusterUtils.visualise_clusters(aligner,n_cols = 4, figsize= (10,6))
```

Example 60 (unknown):
```unknown
ClusterUtils.print_cluster_average_alignments(aligner)
```

Example 61 (unknown):
```unknown
cluster:  0 IIIIIMMMIDDDDDDMMMMM 7
cluster:  1 IDMMMIIDDMMMMMMMM 234
cluster:  2 IIIDIDIDIDIIIIIIDDDDDDDMMD 3
cluster:  3 IIDDDDDDMMMIIIMVMMMM 19
cluster:  4 MIIIMMMIIDDDMMMMMDD 1
cluster:  5 DMDDDDMMIIIIIIMMMMMD 1
cluster:  6 IIIDIMIMDDDDMMMMMMM 4
cluster:  7 IIIDDDDDDMMVVMMMVMMM 2
cluster:  8 IIIDIDIDIDIIIIIIIIDDDDDDDDDD 7
```

Example 62 (unknown):
```unknown
# Initialize an empty dictionary
cluster_genes_dict = {}

# Loop through each gene and its corresponding cluster ID
for gene, cluster_id in zip(aligner.gene_list, aligner.cluster_ids):
    # Check if the cluster ID is already in the dictionary
    if cluster_id not in cluster_genes_dict:
        # If not, create a new list for this cluster ID
        cluster_genes_dict[cluster_id] = []
    # Append the gene to the list for this cluster ID
    cluster_genes_dict[cluster_id].append(gene)
```

Example 63 (unknown):
```unknown
clus_of_interest=8
```

Example 64 (python):
```python
cluster_genes_dict[clus_of_interest]
print(len(cluster_genes_dict[clus_of_interest]))
```

Example 65 (unknown):
```unknown
GENE_SUBSET = cluster_genes_dict[clus_of_interest]
aligner.get_aggregate_alignment_for_subset(GENE_SUBSET)
```

Example 66 (unknown):
```unknown
Average Alignment:  IIIDIDIDIDIIIIIIIIDDDDDDDDDD
```

Example 67 (python):
```python
for gene in GENE_SUBSET:
    if len(GENE_SUBSET)<150:
        print(gene, aligner.results_map[gene].alignment_str)
        VisualUtils.plotTimeSeries(gene, aligner, vs)
        plt.show()
```

Example 68 (unknown):
```unknown
BHLHE41_+ MIIIVVVVIIDIIIIDDDDDDDDDDDD
```

Example 69 (unknown):
```unknown
ETS2_+ IIIDIDIDIDIIIIIIIIDDDDDDDDDD
```

Example 70 (unknown):
```unknown
ETS2_- IIIDIDIDIDIIIIIIIIDDDDDDDDDD
```

Example 71 (unknown):
```unknown
ETV6_+ IIIDIDIDIDIIIIIIIIDDDDDDDDDD
```

Example 72 (unknown):
```unknown
GTF2IRD1_+ IIIIIMMMIDIIIIIDDDDDDDDDD
```

Example 73 (unknown):
```unknown
MAF_- IIIDIDIDIDIIIIIIIIDDDDDDDDDD
```

Example 74 (unknown):
```unknown
PITX2_+ IIIDIDIDIDIIIIIIIIDDDDDDDDDD
```

Example 75 (python):
```python
import matplotlib
matplotlib.rcParams['pdf.fonttype'] = 42
matplotlib.rcParams['ps.fonttype'] = 42
```

Example 76 (python):
```python
import matplotlib.pyplot as plt
from contextlib import redirect_stdout
import io


for gene in GENE_SUBSET:
    f = io.StringIO()
    with redirect_stdout(f):
        plotTimeSeries_cmap(gene, aligner)
    plt.savefig(f"./figures/{gene}_time_series_alignment.png", format='png', bbox_inches='tight')
    plt.show()
```

Example 77 (python):
```python
for gene in GENE_SUBSET:
    sc.pl.umap(adata_aucg,color=gene,cmap="RdPu",save=f"_{gene}_AUC.png")
```

Example 78 (unknown):
```unknown
WARNING: saving figure to file figures/umap_BHLHE41_+_AUC.png
```

Example 79 (unknown):
```unknown
WARNING: saving figure to file figures/umap_ETS2_+_AUC.png
```

Example 80 (unknown):
```unknown
WARNING: saving figure to file figures/umap_ETS2_-_AUC.png
```

Example 81 (unknown):
```unknown
WARNING: saving figure to file figures/umap_ETV6_+_AUC.png
```

Example 82 (unknown):
```unknown
WARNING: saving figure to file figures/umap_GTF2IRD1_+_AUC.png
```

Example 83 (unknown):
```unknown
WARNING: saving figure to file figures/umap_MAF_-_AUC.png
```

Example 84 (unknown):
```unknown
WARNING: saving figure to file figures/umap_PITX2_+_AUC.png
```

Example 85 (python):
```python
import seaborn as sns
import matplotlib.pyplot as plt
from scipy.stats import pearsonr

# Assuming plot_data is defined as per your code snippet
plot_data = adata_aucg.obs[["t", "age_days_smoothed_once"]]

# Calculate Pearson's R and the p-value
pearson_r, p_value = pearsonr(plot_data['age_days_smoothed_once'], plot_data['t'])
p_value_text = f"{p_value:.2e}" if p_value > 1.0e-8 else "< 1.0e-8"

# Create scatter plot with specified aesthetics
sns.regplot(data=plot_data, x="age_days_smoothed_once", y="t", 
            scatter_kws={'alpha':0.5, 's':20, 'edgecolor':None},
            line_kws={'color':'black'}, # Sets the regression line color to black
            ci=95)

# Add text annotation for Pearson's R and adjusted p-value on the plot
plt.text(x=min(plot_data['age_days_smoothed_once']), y=max(plot_data['t']),
         s=f"Pearson's R: {pearson_r:.2f}\nP-value: {p_value_text}",
         backgroundcolor='white', ha='left', va='top')  # Adjust text positioning as needed
plt.xlabel("age (days)")
plt.ylabel("pseudotime")
plt.title('Pseudotime correlates with sample age')
plt.tight_layout()
plt.savefig(f"./figures/age_pseudotime_correlation.pdf", format='pdf', bbox_inches='tight')
plt.show()
```

Example 86 (unknown):
```unknown
df = aligner.get_stat_df()
```

Example 87 (unknown):
```unknown
mean matched percentage: 
60.519999999999996 %
```

Example 88 (unknown):
```unknown
df["alignment_similarity_percentage"] *= 100
```

Example 89 (unknown):
```unknown
filtered_df = df[df["alignment_similarity_percentage"] < 25]

# Sort primarily by 'l2fc' and secondarily by 'and_logfc', both in ascending order
sorted_df = filtered_df.sort_values(by=['l2fc'])
sorted_df
```

Example 90 (unknown):
```unknown
LOW_SIM_REGULONS = sorted_df['Gene'].unique().tolist()
```

Example 91 (python):
```python
import seaborn as sns
import matplotlib.pyplot as plt

plt.figure(figsize=(4, 4))

# Assuming 'df' is your DataFrame and 'genes_of_interest' is your list of genes
genes_of_interest = GENE_SUBSET

# Choose the y-axis based on your need
y_axis = 'alignment_similarity_percentage'  # or 'opt_alignment_cost'

# Plotting the scatter plot
sns.scatterplot(data=df, x='l2fc', y=y_axis, edgecolor="none", legend=False, alpha=0.3, color='black')
sns.scatterplot(data=df[df["Gene"].isin(LOW_SIM_REGULONS)], x='l2fc', y=y_axis, edgecolor="none", legend=False, color='red', alpha=0.8)

# Add a horizontal line at y=0.5
plt.axvline(0, color='gray', linestyle='--', linewidth=0.5)

# Add text annotations for LaCM and RaCM
plt.text(0.1, 1.05, 'RaCM', transform=plt.gca().transAxes, ha='left', va='top', weight='semibold')
plt.text(0.9, 1.05, 'LaCM', transform=plt.gca().transAxes, ha='right', va='top', weight='semibold')

# Label points corresponding to specific genes
#for _, row in df.iterrows():
#    if row['Gene'] in genes_of_interest:
#        plt.text(row['l2fc'], row[y_axis], row['Gene'], color='red', style='italic')

# Set labels
plt.xlabel("Log2 Fold Change")

if y_axis == "alignment_similarity_percentage":
    plt.ylabel("Alignment Similarity (%)")
elif y_axis == "opt_alignment_cost":
    plt.ylabel("Alignment Cost (nits)")

# Save the plot
plt.savefig("./figures/similarity_v_logFC_AUC.pdf", format='pdf', bbox_inches='tight')

# Show the plot
plt.show()
```

Example 92 (python):
```python
import seaborn as sns
import matplotlib.pyplot as plt

plt.figure(figsize=(6, 6))

# Assuming 'df' is your DataFrame and 'genes_of_interest' is your list of genes
genes_of_interest = GENE_SUBSET  # Make sure this is defined as your subset of interest

# Choose the y-axis based on your need
y_axis = 'alignment_similarity_percentage'  # or 'opt_alignment_cost'

# Plotting the background scatter plot
sns.scatterplot(data=df[~df["Gene"].isin(LOW_SIM_REGULONS)], x='l2fc', y=y_axis, edgecolor="none", legend=False, alpha=0.3, color='black')
sns.scatterplot(data=df[df["Gene"].isin(LOW_SIM_REGULONS)], x='l2fc', y=y_axis, edgecolor="none", legend=False, color='red', alpha=0.8)

# Instead of plotting red points, we will annotate gene names directly on the plot for LOW_SIM_REGULONS
for _, row in df[df["Gene"].isin(LOW_SIM_REGULONS)].iterrows():
    plt.text(row['l2fc']+0.01, row[y_axis]+1, row['Gene'], color='red', style='italic', alpha=0.8, size='x-small')

# Add a horizontal line at y=0
plt.axvline(0, color='gray', linestyle='--', linewidth=0.5)

# Add text annotations for LaCM and RaCM
plt.text(0.1, 1.05, 'RaCM', transform=plt.gca().transAxes, ha='left', va='top', weight='semibold')
plt.text(0.9, 1.05, 'LaCM', transform=plt.gca().transAxes, ha='right', va='top', weight='semibold')

# Set labels
plt.xlabel("Log2 Fold Change")
if y_axis == "alignment_similarity_percentage":
    plt.ylabel("Alignment Similarity (%)")
elif y_axis == "opt_alignment_cost":
    plt.ylabel("Alignment Cost (nits)")

# Save the plot
plt.savefig("./figures/similarity_v_logFC_AUC.pdf", format='pdf', bbox_inches='tight')

# Show the plot
plt.show()
```

Example 93 (python):
```python
sc.pl.matrixplot(adata_aucg,
             var_names = LOW_SIM_REGULONS,
             groupby="fine_grain",
             colorbar_title="Mean regulon\nactivity score",
                save="_RAS_LvRaCMs.pdf",
                cmap="RdPu",
                vmax=0.2,
                var_group_positions=[(0,1),(2,10)],
             var_group_labels=["RaCM","LaCM"])
```

Example 94 (unknown):
```unknown
WARNING: saving figure to file figures/matrixplot__RAS_LvRaCMs.pdf
```

Example 95 (python):
```python
adata_aucg.obs['t'] = pd.to_numeric(adata_aucg.obs['t'])
adata_aucg = adata_aucg[adata_aucg.obs['t'].sort_values().index]
```

Example 96 (python):
```python
sc.pl.heatmap(adata_aucg,var_names = LOW_SIM_REGULONS,
             groupby="fine_grain",
             standard_scale='var',
             cmap='RdPu',
              var_group_positions=[(0,1),(2,10)],
             var_group_labels=["RaCM","LaCM"],
              save="_RAS_LvRaCMs.pdf",
             )
```

Example 97 (unknown):
```unknown
WARNING: saving figure to file figures/heatmap_RAS_LvRaCMs.pdf
```

Example 98 (python):
```python
import scipy
# Extract the expression data for these genes
expression_data = adata_aucg[:, LOW_SIM_REGULONS].X

# Depending on the AnnData storage, the expression data might need conversion to a dense format
if scipy.sparse.issparse(expression_data):
    expression_data = expression_data.toarray()

# Create a DataFrame with the expression data
df_expression = pd.DataFrame(expression_data, columns=LOW_SIM_REGULONS)

# Add the pseudotime to the DataFrame
df_expression['pseudotime'] = adata_aucg.obs['t'].values

# Optionally, add your 'fine_grain' cluster information if you want to plot clusters in order
df_expression['fine_grain'] = adata_aucg.obs['fine_grain'].values

# Melt the DataFrame for easier plotting with seaborn
df_melted = df_expression.melt(id_vars=['pseudotime', 'fine_grain'], var_name='Gene', value_name='Expression')

# Plotting with seaborn
plt.figure(figsize=(10, 8))
sns.heatmap(df_melted.pivot_table(index='Gene', columns='pseudotime', values='Expression'), cmap='RdPu')

plt.title('Gene Expression and Pseudotime Heatmap')
plt.xlabel('Pseudotime')
plt.ylabel('Gene')
plt.show()
```

Example 99 (python):
```python
import matplotlib.pyplot as plt
from contextlib import redirect_stdout
import io


for gene in LOW_SIM_REGULONS:
    f = io.StringIO()
    with redirect_stdout(f):
        plotTimeSeries_cmap(gene, aligner)
    plt.savefig(f"./figures/{gene}_time_series_alignment.png", format='png', bbox_inches='tight')
    plt.show()
```

Example 100 (unknown):
```unknown
LOW_SIM_REGULONS
```

Example 101 (unknown):
```unknown
['ETS2_+',
 'MAF_+',
 'IKZF2_+',
 'ETV6_+',
 'BHLHE41_+',
 'GTF2IRD1_+',
 'GATA5_+',
 'PITX2_+',
 'SIX5_extended_+',
 'MAF_-',
 'ETS2_-']
```

Example 102 (python):
```python
for gene in LOW_SIM_REGULONS:
    sc.pl.umap(adata_aucg,color=gene,cmap="RdPu",save=f"_{gene}_AUC.png")
```

Example 103 (unknown):
```unknown
WARNING: saving figure to file figures/umap_ETS2_+_AUC.png
```

Example 104 (unknown):
```unknown
WARNING: saving figure to file figures/umap_MAF_+_AUC.png
```

Example 105 (unknown):
```unknown
WARNING: saving figure to file figures/umap_IKZF2_+_AUC.png
```

Example 106 (unknown):
```unknown
WARNING: saving figure to file figures/umap_ETV6_+_AUC.png
```

Example 107 (unknown):
```unknown
WARNING: saving figure to file figures/umap_BHLHE41_+_AUC.png
```

Example 108 (unknown):
```unknown
WARNING: saving figure to file figures/umap_GTF2IRD1_+_AUC.png
```

Example 109 (unknown):
```unknown
WARNING: saving figure to file figures/umap_GATA5_+_AUC.png
```

Example 110 (unknown):
```unknown
WARNING: saving figure to file figures/umap_PITX2_+_AUC.png
```

Example 111 (unknown):
```unknown
WARNING: saving figure to file figures/umap_SIX5_extended_+_AUC.png
```

Example 112 (unknown):
```unknown
WARNING: saving figure to file figures/umap_MAF_-_AUC.png
```

Example 113 (unknown):
```unknown
WARNING: saving figure to file figures/umap_ETS2_-_AUC.png
```

---

## Import modules¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/RNA/scnRNA_Integration-Annotation/Leukocytes/No5_immune-collect-annotations.html

**Contents:**
- Import modules¶
- Read in immune raw data¶
- Read in myeloid and lymphoid objects for the annotation¶
- Add annotation¶
- new_fine_grain¶
- new_mid_grain¶
- Set category¶
- Save¶
- Markers¶

41881 rows × 53 columns

**Examples:**

Example 1 (python):
```python
import anndata
import numpy as np
import pandas as pd
import scanpy as sc

import matplotlib.pyplot as plt
import seaborn as sns

import celltypist
from celltypist import models

import session_info
```

Example 2 (python):
```python
sc.settings.set_figure_params(dpi=120, color_map = 'RdPu',)
```

Example 3 (unknown):
```unknown
session_info.show()
```

Example 4 (python):
```python
-----
anndata             0.8.0
celltypist          0.1.9
matplotlib          3.5.2
numpy               1.21.5
pandas              1.3.5
scanpy              1.8.2
seaborn             0.11.1
session_info        1.0.0
-----
```

Example 5 (python):
```python
PIL                 9.1.1
asciitree           NA
backcall            0.2.0
beta_ufunc          NA
binom_ufunc         NA
brotli              NA
cached_property     1.5.1
cairo               1.20.0
certifi             2020.12.05
cffi                1.14.4
chardet             3.0.4
charset_normalizer  2.0.10
cloudpickle         1.6.0
colorama            0.4.4
cycler              0.10.0
cython_runtime      NA
cytoolz             0.11.2
dask                2022.01.0
dateutil            2.8.1
decorator           4.4.2
defusedxml          0.6.0
fasteners           0.17.3
fsspec              2022.01.0
google              NA
h5py                3.1.0
idna                2.10
igraph              0.10.2
importlib_metadata  1.7.0
ipykernel           5.4.2
ipython_genutils    0.2.0
ipywidgets          7.6.3
jedi                0.17.2
jinja2              3.1.2
joblib              1.0.0
kiwisolver          1.3.1
leidenalg           0.9.0
llvmlite            0.35.0
louvain             0.7.0
markupsafe          2.1.1
mpl_toolkits        NA
msgpack             1.0.3
natsort             7.1.0
nbinom_ufunc        NA
numba               0.52.0
numcodecs           0.9.1
numexpr             2.7.2
packaging           21.3
parso               0.7.1
pexpect             4.8.0
pickleshare         0.7.5
pkg_resources       NA
plotly              5.10.0
prompt_toolkit      3.0.10
psutil              5.9.0
ptyprocess          0.7.0
pyarrow             9.0.0
pygments            2.13.0
pyparsing           2.4.7
pytz                2020.5
requests            2.27.1
scipy               1.7.3
setuptools_scm      NA
sinfo               0.3.1
six                 1.14.0
sklearn             1.0.2
socks               1.7.1
sphinxcontrib       NA
statsmodels         0.12.1
storemagic          NA
tables              3.6.1
tblib               1.7.0
texttable           1.6.3
threadpoolctl       2.1.0
tlz                 0.11.2
toolz               0.11.2
tornado             6.1
traitlets           5.0.5
typing_extensions   NA
urllib3             1.26.9
wcwidth             0.2.5
yaml                5.4.1
zarr                2.11.3
zipp                NA
zmq                 21.0.1
zope                NA
```

Example 6 (unknown):
```unknown
-----
IPython             7.19.0
jupyter_client      6.1.11
jupyter_core        4.7.0
notebook            6.2.0
-----
Python 3.7.9 | packaged by conda-forge | (default, Dec  9 2020, 21:08:20) [GCC 9.3.0]
Linux-4.15.0-158-generic-x86_64-with-debian-bullseye-sid
-----
Session information updated at 2023-02-21 09:51
```

Example 7 (unknown):
```unknown
!ls /nfs/team205/heart/anndata_objects/Foetal/Immune/
```

Example 8 (unknown):
```unknown
C97_index.csv
RNA_foetal_raw_immune_16Feb2023.h5ad
RNA_foetal_raw_Lymphoid_16Feb2023.h5ad
RNA_foetal_raw_Lymphoid_16Feb2023.obs.csv
RNA_foetal_raw_Myeloid_16Feb2023.h5ad
RNA_foetal_raw_Myeloid_16Feb2023.obs.csv
```

Example 9 (python):
```python
adata = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/Immune/RNA_foetal_raw_immune_16Feb2023.h5ad')
adata
```

Example 10 (unknown):
```unknown
AnnData object with n_obs × n_vars = 41881 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', '_scvi_labels', '2Jan_coarse_grain', '2Jan_mid_grain', '2Jan_fine_grain', 'leiden_scVI', 'FACSgate', 'sex', 'week', 'trimester', 'cell_type2'
    var: 'gene_name_scRNA-0', 'gene_id'
    obsm: 'X_pca', 'X_scVI', 'X_umap', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 11 (python):
```python
adata.X.data[:10]
```

Example 12 (unknown):
```unknown
array([1., 2., 1., 1., 1., 2., 2., 5., 2., 2.], dtype=float32)
```

Example 13 (python):
```python
sc.pl.umap(adata, color=['cell_type2'], size=5, legend_fontsize=10, frameon=False, components='all', wspace=0.5)
```

Example 14 (unknown):
```unknown
!ls /nfs/team205/heart/anndata_objects/Foetal/Immune/
```

Example 15 (python):
```python
lymph = pd.read_csv('/nfs/team205/heart/anndata_objects/Foetal/Immune/RNA_foetal_raw_Lymphoid_16Feb2023.obs.csv',index_col=0)
lymph.head()
```

Example 16 (python):
```python
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/IPython/core/interactiveshell.py:3418: DtypeWarning: Columns (37) have mixed types.Specify dtype option on import or set low_memory=False.
  exec(code_obj, self.user_global_ns, self.user_ns)
```

Example 17 (python):
```python
myelo = pd.read_csv('/nfs/team205/heart/anndata_objects/Foetal/Immune/RNA_foetal_raw_Myeloid_16Feb2023.obs.csv',index_col=0)
myelo.head()
```

Example 18 (python):
```python
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/IPython/core/interactiveshell.py:3418: DtypeWarning: Columns (37) have mixed types.Specify dtype option on import or set low_memory=False.
  exec(code_obj, self.user_global_ns, self.user_ns)
```

Example 19 (python):
```python
adata.obs['new_fine_grain'] = adata.obs['cell_type2'].astype('str')
adata.obs.loc[lymph.index,'new_fine_grain'] = lymph['cell_type3'].astype('str')
adata.obs.loc[myelo.index,'new_fine_grain'] = myelo['cell_type3'].astype('str')
adata.obs['new_fine_grain'].value_counts()
```

Example 20 (javascript):
```javascript
MP_LYVE1+            9748
CD4+T                5206
B_MS4A1+             3722
NK                   2437
CD8+T                2121
CD4+Treg             1940
MP_LYVE1+ATF3+       1844
B                    1666
MP_LYVE1+_cycling    1650
MP_CX3CR1+           1648
ILC                  1639
T_cycling             912
Mono                  900
Mast cell             866
Bpro                  850
B_cycling             843
Mono-derived          661
DC1                   609
Plasmacytoid DC       608
Doublet               583
MP_TIMD4+             470
Mono_MPO+             414
Megakaryocyte         360
DC_mature             184
Name: new_fine_grain, dtype: int64
```

Example 21 (python):
```python
sc.pl.umap(adata, color=['new_fine_grain'], size=5, legend_fontsize=7, frameon=False, components='all', wspace=0.6)
```

Example 22 (unknown):
```unknown
... storing 'new_fine_grain' as categorical
```

Example 23 (python):
```python
set(adata.obs['new_fine_grain'])
```

Example 24 (unknown):
```unknown
{'B',
 'B_MS4A1+',
 'B_cycling',
 'Bpro',
 'CD4+T',
 'CD4+Treg',
 'CD8+T',
 'DC1',
 'DC_mature',
 'Doublet',
 'ILC',
 'MP_CX3CR1+',
 'MP_LYVE1+',
 'MP_LYVE1+ATF3+',
 'MP_LYVE1+_cycling',
 'MP_TIMD4+',
 'Mast cell',
 'Megakaryocyte',
 'Mono',
 'Mono-derived',
 'Mono_MPO+',
 'NK',
 'Plasmacytoid DC',
 'T_cycling'}
```

Example 25 (python):
```python
fine2mid = {
    'Mono':'MyeloidCells', 
    'Mono_MPO+':'MyeloidCells', 
    'Mono-derived':'MyeloidCells',
    'MP_CX3CR1+':'MyeloidCells',
    # 'MP_TREM2+':'MyeloidCells', 
    'MP_TIMD4+':'MyeloidCells',
    'MP_LYVE1+':'MyeloidCells',
    # 'MP_LYVE1+JUN+':'MyeloidCells',
    'MP_LYVE1+ATF3+':'MyeloidCells',
    'MP_LYVE1+_cycling':'MyeloidCells',
    'DC1':'MyeloidCells',
    'DC_mature':'MyeloidCells',
    'Plasmacytoid DC':'MyeloidCells',
    'Mast cell':'MyeloidCells',
    'Megakaryocyte':'MyeloidCells',
    'CD4+T':'LymphoidCells',
    'CD4+Treg':'LymphoidCells',
    'CD8+T':'LymphoidCells',
    'NK':'LymphoidCells',
    'ILC':'LymphoidCells',
    'T_cycling':'LymphoidCells',
    'Bpro':'LymphoidCells',
    'B':'LymphoidCells',
    'B_MS4A1+':'LymphoidCells',
    'B_cycling':'LymphoidCells',
    'Doublet':'Doublet',
}

adata.obs['new_mid_grain']=adata.obs['new_fine_grain'].copy()
adata.obs.replace({'new_mid_grain':fine2mid},inplace=True)
pd.crosstab(adata.obs['new_fine_grain'],adata.obs['new_mid_grain'])
```

Example 26 (python):
```python
adata.obs['new_mid_grain']=adata.obs['new_mid_grain'].astype('category')
adata.obs['new_mid_grain'].cat.reorder_categories([
   'MyeloidCells', 
    'LymphoidCells',
    'Doublet',
     ], inplace=True)

adata.obs['new_fine_grain']=adata.obs['new_fine_grain'].astype('category')
adata.obs['new_fine_grain'].cat.reorder_categories(list(fine2mid.keys()), inplace=True)
```

Example 27 (python):
```python
sc.pl.umap(adata, color=['new_fine_grain'], size=5, legend_fontsize=10, frameon=False, components='all', wspace=0.8)
```

Example 28 (python):
```python
sc.pl.umap(adata, color=['new_fine_grain'], groups='Mono-derived',size=5, legend_fontsize=7, frameon=False, components='all', wspace=0.6)
```

Example 29 (python):
```python
sc.pl.umap(adata, color=['new_mid_grain'], size=5, legend_fontsize=10, frameon=False, components='all', wspace=0.3)
```

Example 30 (python):
```python
adata.X.data[:10]
```

Example 31 (unknown):
```unknown
array([1., 2., 1., 1., 1., 2., 2., 5., 2., 2.], dtype=float32)
```

Example 32 (unknown):
```unknown
!ls /nfs/team205/heart/anndata_objects/Foetal/Immune
```

Example 33 (unknown):
```unknown
C97_index.csv
RNA_foetal_raw_immune_16Feb2023.h5ad
RNA_foetal_raw_Lymphoid_16Feb2023.h5ad
RNA_foetal_raw_Lymphoid_16Feb2023.obs.csv
RNA_foetal_raw_Myeloid_16Feb2023.h5ad
RNA_foetal_raw_Myeloid_16Feb2023.obs.csv
```

Example 34 (python):
```python
adata.write('/nfs/team205/heart/anndata_objects/Foetal/Immune/RNA_foetal_raw_immune_16Feb2023.h5ad')
adata
```

Example 35 (unknown):
```unknown
AnnData object with n_obs × n_vars = 41881 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', '_scvi_labels', '2Jan_coarse_grain', '2Jan_mid_grain', '2Jan_fine_grain', 'leiden_scVI', 'FACSgate', 'sex', 'week', 'trimester', 'cell_type2', 'new_fine_grain', 'new_mid_grain'
    var: 'gene_name_scRNA-0', 'gene_id'
    uns: 'cell_type2_colors', 'new_fine_grain_colors', 'new_mid_grain_colors'
    obsm: 'X_pca', 'X_scVI', 'X_umap', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 36 (python):
```python
# save .obs
adata.obs.to_csv('/nfs/team205/heart/anndata_objects/Foetal/Immune/RNA_foetal_raw_immune_16Feb2023.obs.csv')
```

Example 37 (unknown):
```unknown
!ls -l  /nfs/team205/heart/anndata_objects/Foetal/Immune
```

Example 38 (unknown):
```unknown
total 1500548
-rw-r--r-- 1 20383 1403    264965 Feb 16 10:37 C97_index.csv
-rw-r--r-- 1 20383 1403 763332444 Feb 17 13:21 RNA_foetal_raw_immune_16Feb2023.h5ad
-rw-r--r-- 1 20383 1403  23704564 Feb 17 13:58 RNA_foetal_raw_immune_16Feb2023.obs.csv
-rw-r--r-- 1 20383 1403 369182732 Feb 17 12:59 RNA_foetal_raw_Lymphoid_16Feb2023.h5ad
-rw-r--r-- 1 20383 1403  12130193 Feb 17 13:15 RNA_foetal_raw_Lymphoid_16Feb2023.obs.csv
-rw-r--r-- 1 20383 1403 358041637 Feb 16 23:33 RNA_foetal_raw_Myeloid_16Feb2023.h5ad
-rw-r--r-- 1 20383 1403   9891132 Feb 16 23:33 RNA_foetal_raw_Myeloid_16Feb2023.obs.csv
```

Example 39 (unknown):
```unknown
# rclone to gdrive
rclone copy /nfs/team205/heart/anndata_objects/Foetal/Immune/RNA_foetal_raw_immune_16Feb2023.h5ad \
gdrive:JamesC_Kazumasa/Foetal_Heart/SanjayShinha/toSinhalab/anndata/Immune_16Feb2023
    
rclone copy /nfs/team205/heart/anndata_objects/Foetal/Immune/C97_index.csv \
gdrive:JamesC_Kazumasa/Foetal_Heart/SanjayShinha/toSinhalab/anndata/Immune_16Feb2023
```

Example 40 (python):
```python
adata = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/Immune/RNA_foetal_raw_immune_16Feb2023.h5ad')
adata
```

Example 41 (unknown):
```unknown
AnnData object with n_obs × n_vars = 41881 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', '_scvi_labels', '2Jan_coarse_grain', '2Jan_mid_grain', '2Jan_fine_grain', 'leiden_scVI', 'FACSgate', 'sex', 'week', 'trimester', 'cell_type2', 'new_fine_grain', 'new_mid_grain'
    var: 'gene_name_scRNA-0', 'gene_id'
    uns: 'cell_type2_colors', 'new_fine_grain_colors', 'new_mid_grain_colors'
    obsm: 'X_pca', 'X_scVI', 'X_umap', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 42 (python):
```python
adata_lognorm = adata.copy()
sc.pp.normalize_total(adata_lognorm, target_sum=1e4)
sc.pp.log1p(adata_lognorm)
```

Example 43 (python):
```python
markers={
    'Mono':['CCR2','FCN1','MPO'],
    'MP general':['C1QA','C1QB','CD68','MRC1','CD163','LYVE1','FOLR2'],
    'CXCR1+':['CX3CR1','FCGR1A','FCGR3A','C3'],
    'TREM2+':['TREM2','SPP1'],
    'PtdSer-R':['TIMD4','MERTK','AXL','ITGAV'],
    'FOS-JUN':['FOS','JUN'],
    'Collagen':['COL1A1','COL1A2','COL3A1'],
    'DC general':['FLT3','HLA-DPA1','HLA-DPB1','CIITA'],
    'DC1':['ID2','BATF3','CLEC9A'],
    'DC2':['CLEC10A','CD1C'],
    'DC_mature':['CCR7','CD83', 'LY75','LAMP3'],
    'pDC':['IRF8','IRF4','IL3RA'],
      'Mast': ['KIT', 'CPA3','HDC','GATA2'],
    'T cell': ['CD3E', 'CD3D','CD4', 'CD8A', 'CD8B'],
    'CD4T_reg': ['FOXP3', 'CTLA4', 'IL2RA'],
    'cytotoxic': ['PRF1', 'GZMB','NKG7'],
    'NK': ['KLRF1', 'KLRD1','NCAM1',],
    'ILC': ['KIT', 'IL7R','IL18R1','RORC','IL17A'],   
    'B': ['CD19','CD20','CD22','CD79A', 'CD79B'],
     'ProB':['CD34'],
    'B1':['MS4A1','IGHM','IGHD','MZB1'],
    'Gene editing': ['RAG1','RAG2'],
    'Cycling':['MKI67','TOP2A'],
}

for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in adata.var_names]

dp = sc.pl.dotplot(adata_lognorm,
              markers, 
              groupby='new_fine_grain',
              dendrogram=False,
              standard_scale="var",
              color_map="Reds",
              return_fig=True
              # swap_axes=True,
             )
dp.add_totals().show()
```

Example 44 (python):
```python
markers={
     'Mono':['CCR2','FCN1','MPO'],
    'Mono-derived':['CLEC10A','CD1C'],
    'MP':['C1QA','C1QB','CD68','MRC1','CD163','LYVE1','FOLR2'],
    'CX3CR1+':['CX3CR1','FCGR1A','FCGR3A','C3','P2RY12','SALL1','C3','OLFML3','HTRA1','TREM2',],
    'PtdSer-R':['TIMD4','MERTK','AXL','ITGAV'],
    'ATF3':['ATF3'],
    'DC general':['FLT3','HLA-DPA1','HLA-DPB1','CIITA'],
    'DC1':['ID2','BATF3','CLEC9A'],
    'pDC':['IRF8','IRF4','IL3RA'],
    'Mast': ['KIT', 'CPA3','HDC','GATA2'],
  
    
    'T cell': ['CD3E', 'CD3D','CD4', 'CD8A', 'CD8B'],
    'CD4T_reg': ['FOXP3', 'CTLA4', 'IL2RA', 'IKZF2'],
    'cytotoxic': ['PRF1', 'GZMB','NKG7'],
    'NK': ['KLRF1', 'KLRD1','NCAM1','FCGR3A','IFNG'],
    'ILC': ['KIT', 'IL7R','IL18R1','RORC','IL17A'],   
    'B': ['CD19','CD20','CD22','CD79A', 'CD79B','MS4A1'],
     'ProB':['CD34'],
    'B1':['CD27','CCR10'],
    'Igs':['IGHM','IGHD'],
    'Gene editing': ['AICDA', 'RAG1','RAG2'],
    'Cycling':['MKI67','TOP2A'],
}

for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in adata.var_names]

dp = sc.pl.dotplot(adata_lognorm,
              markers, 
              groupby='new_fine_grain',
              dendrogram=False,
              standard_scale="var",
              color_map="Reds",
              return_fig=True
              # swap_axes=True,
             )
dp.add_totals().show()
```

Example 45 (python):
```python
markers={
     'T cell': ['CD3E', 'CD3D','TRAC','TRBC1',],
     'B': ['CD19','CD20','CD22','CD79A', 'CD79B','MS4A1'],
   
    
    'Mono':['CCR2','CD14','FCGR3A','CX3CR1','FCGR1A','FCN1','TREM2'],
    'MP':['C1QA','C1QB','CD68','CD163','MERTK','LYVE1','IGF1','TIMD4','CX3CR1','DOCK4'],
    'pDC':['IRF8','IRF4','IL3RA'],
     'DC_mature':['FLT3', 'CCR7','LAMP3', 'IL15', 'CD83', 'LY75', ],
    'Megakaryocyte':['ITGA2B','MPL','ITGB3',],
    'Mast': ['KIT', 'CPA3','HDC','GATA2'],
}


for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in adata.var_names]

sc.pl.dotplot(adata_lognorm,
              markers, 
              groupby='cell_type2',
              dendrogram=False,
              standard_scale="var",
              color_map="Reds",
              # swap_axes=True,
             )
```

---

## Read in¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/Xenium/no1_per-sample-processing/C194-HEA-0-FFPE-1/no1_QC_filetering.html

**Contents:**
- Read in¶
- Calculate quality control metrics¶
- Filtering out low-count cells¶
- Save¶

**Examples:**

Example 1 (python):
```python
import scanpy as sc
import numpy as np
import pandas as pd
import os

import matplotlib.pyplot as plt
import seaborn as sns
```

Example 2 (unknown):
```unknown
!ls /home/kk837/rfs/rfs-teichlab-nfs-iCNyzSAaucw/jc2226/heart_team_data/xenium/20241113__133901__JK_HEA5K_Hs_heart_vessels
```

Example 3 (unknown):
```unknown
output-XETG00155__0038587__C194-HEA-0-FFPE-1__20241113__134112
output-XETG00155__0038587__Hst45-HEA-0-FFPE-1__20241113__134112
output-XETG00155__0038587__Hst60-HEA-0-FFPE-1__20241113__134112
output-XETG00155__0038592__AH13-HEA-36-FFPE-1__20241113__134112
output-XETG00155__0038592__AH14-HEA-29-FFPE-2__20241113__134112
output-XETG00155__0038592__AH14-HEA-36-FFPE-1__20241113__134112
```

Example 4 (python):
```python
sample_id = 'C194-HEA-0-FFPE-1'
out_dir = f'/home/kk837/rfs/rfs-teichlab-nfs-iCNyzSAaucw/jc2226/heart_team_data/xenium/20241113__133901__JK_HEA5K_Hs_heart_vessels/output-XETG00155__0038587__{sample_id}__20241113__134112'

adata = sc.read_10x_h5(
    filename=os.path.join(out_dir,"cell_feature_matrix.h5")
)
print(adata.X.data[:5])
adata
```

Example 5 (unknown):
```unknown
[5. 1. 1. 2. 2.]
```

Example 6 (unknown):
```unknown
AnnData object with n_obs × n_vars = 146232 × 5001
    var: 'gene_ids', 'feature_types', 'genome'
```

Example 7 (python):
```python
# add obs
df = pd.read_csv(
    os.path.join(out_dir,"cells.csv.gz")
)
df.set_index(adata.obs_names, inplace=True)
adata.obs = df.copy()
adata.obs.head()
```

Example 8 (python):
```python
adata.obsm["spatial"] = adata.obs[["x_centroid", "y_centroid"]].copy().to_numpy()
```

Example 9 (unknown):
```unknown
AnnData object with n_obs × n_vars = 146232 × 5001
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method'
    var: 'gene_ids', 'feature_types', 'genome'
    obsm: 'spatial'
```

Example 10 (python):
```python
sc.pl.spatial(adata,color=['total_counts'],cmap='RdPu',spot_size=20,vmax='p95')
```

Example 11 (python):
```python
/tmp/ipykernel_364135/27570213.py:1: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(adata,color=['total_counts'],cmap='RdPu',spot_size=20,vmax='p95')
```

Example 12 (python):
```python
adata.obs['segmentation_method'].value_counts()
```

Example 13 (unknown):
```unknown
segmentation_method
Segmented by interior stain (18S)                       114599
Segmented by boundary stain (ATP1A1+CD45+E-Cadherin)     30718
Segmented by nucleus expansion of 5.0µm                    915
Name: count, dtype: int64
```

Example 14 (python):
```python
plt.hist(adata.obs.total_counts, bins=1000)
# plt.xlim(0,500)
plt.xscale('log')
plt.show()
```

Example 15 (python):
```python
sc.pp.calculate_qc_metrics(adata, percent_top=(10, 20, 50, 150), inplace=True)
```

Example 16 (python):
```python
# The percentage of control probes and control codewords can be calculated from adata.obs
cprobes = (
    adata.obs["control_probe_counts"].sum() / adata.obs["total_counts"].sum() * 100
)
cwords = (
    adata.obs["control_codeword_counts"].sum() / adata.obs["total_counts"].sum() * 100
)
print(f"Negative DNA probe count % : {cprobes}")
print(f"Negative decoding count % : {cwords}")
```

Example 17 (unknown):
```unknown
Negative DNA probe count % : 0.0008306279191532507
Negative decoding count % : 0.004514935328449433
```

Example 18 (python):
```python
# Next we plot the distribution of total transcripts per cell, unique transcripts per cell, area of segmented cells and the ratio of nuclei area to their cells
fig, axs = plt.subplots(1, 4, figsize=(20, 4))

axs[0].set_title("Total transcripts per cell")
sns.histplot(
    adata.obs["total_counts"],
    kde=False,
    ax=axs[0],
)

axs[1].set_title("Unique transcripts per cell")
sns.histplot(
    adata.obs["n_genes_by_counts"],
    kde=False,
    ax=axs[1],
)


axs[2].set_title("Area of segmented cells")
sns.histplot(
    adata.obs["cell_area"],
    kde=False,
    ax=axs[2],
)

axs[3].set_title("Nucleus ratio")
sns.histplot(
    adata.obs["nucleus_area"] / adata.obs["cell_area"],
    kde=False,
    ax=axs[3],
)
```

Example 19 (unknown):
```unknown
<Axes: title={'center': 'Nucleus ratio'}, ylabel='Count'>
```

Example 20 (python):
```python
plt.hist(adata.obs.n_genes_by_counts, bins=1000)
# plt.xlim(0,500)
plt.xscale('log')
plt.show()
```

Example 21 (python):
```python
sc.pp.filter_cells(adata,min_counts=50)
sc.pp.filter_cells(adata,min_genes=10)
```

Example 22 (python):
```python
adata.write(f'/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium/{sample_id}_5K_filtered_raw.h5ad')
```

Example 23 (unknown):
```unknown
'C194-HEA-0-FFPE-1'
```

Example 24 (unknown):
```unknown
AnnData object with n_obs × n_vars = 144916 × 5001
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts'
    obsm: 'spatial'
```

---

## Read in cellname-change dictionary which also defines cell order¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/ATAC/ATAC/DAR_plots_finegrain-specific.html

**Contents:**
- Read in cellname-change dictionary which also defines cell order¶
- Read in anndata (peak)¶
- Get accessible proportion matrix¶
- Read in DA results¶
- Defining peaks and plot cluster map¶
- Proportion of peak types¶

6539373 rows × 9 columns

**Examples:**

Example 1 (python):
```python
import scanpy as sc
import pandas as pd
import numpy as np
import anndata
import matplotlib.pyplot as plt
import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 
import seaborn as sns
import os
# import snapatac2 as snap
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
-----
anndata             0.9.2
matplotlib          3.7.2
numpy               1.24.4
pandas              1.5.3
scanpy              1.9.3
seaborn             0.12.2
session_info        1.0.0
-----
```

Example 4 (python):
```python
PIL                         10.0.0
anyio                       NA
arrow                       1.2.3
asttokens                   NA
attr                        23.1.0
attrs                       23.1.0
babel                       2.12.1
backcall                    0.2.0
brotli                      1.0.9
certifi                     2023.11.17
cffi                        1.15.1
charset_normalizer          3.2.0
colorama                    0.4.6
comm                        0.1.4
cycler                      0.10.0
cython_runtime              NA
dateutil                    2.8.2
debugpy                     1.6.8
decorator                   5.1.1
defusedxml                  0.7.1
executing                   1.2.0
fastjsonschema              NA
fqdn                        NA
h5py                        3.9.0
idna                        3.4
igraph                      0.10.6
importlib_metadata          NA
importlib_resources         NA
ipykernel                   6.25.1
isoduration                 NA
jedi                        0.19.0
jinja2                      3.1.2
joblib                      1.3.2
json5                       NA
jsonpointer                 2.0
jsonschema                  4.19.0
jsonschema_specifications   NA
jupyter_events              0.7.0
jupyter_server              2.7.1
jupyterlab_server           2.24.0
kiwisolver                  1.4.5
leidenalg                   0.10.1
llvmlite                    0.40.1
markupsafe                  2.1.3
mpl_toolkits                NA
natsort                     8.4.0
nbformat                    5.9.2
numba                       0.57.1
overrides                   NA
packaging                   23.1
parso                       0.8.3
patsy                       0.5.3
pexpect                     4.8.0
pickleshare                 0.7.5
pkg_resources               NA
plotly                      5.16.1
prometheus_client           NA
prompt_toolkit              3.0.39
psutil                      5.9.5
ptyprocess                  0.7.0
pure_eval                   0.2.2
pydev_ipython               NA
pydevconsole                NA
pydevd                      2.9.5
pydevd_file_utils           NA
pydevd_plugins              NA
pydevd_tracing              NA
pygments                    2.16.1
pyparsing                   3.0.9
pythonjsonlogger            NA
pytz                        2023.3
referencing                 NA
requests                    2.31.0
rfc3339_validator           0.1.4
rfc3986_validator           0.1.1
rpds                        NA
ruamel                      NA
scipy                       1.10.1
send2trash                  NA
six                         1.16.0
sklearn                     1.3.0
sniffio                     1.3.0
socks                       1.7.1
sphinxcontrib               NA
stack_data                  0.6.2
statsmodels                 0.14.0
texttable                   1.6.7
threadpoolctl               3.2.0
tornado                     6.3.3
traitlets                   5.9.0
typing_extensions           NA
uri_template                NA
urllib3                     2.0.4
wcwidth                     0.2.6
webcolors                   1.13
websocket                   1.6.2
yaml                        6.0.1
zipp                        NA
zmq                         25.1.1
zstandard                   0.19.0
```

Example 5 (unknown):
```unknown
-----
IPython             8.12.0
jupyter_client      8.3.0
jupyter_core        4.12.0
jupyterlab          4.0.5
notebook            7.0.2
-----
Python 3.8.17 (default, Jul  5 2023, 21:04:15) [GCC 11.2.0]
Linux-4.15.0-220-generic-x86_64-with-glibc2.17
-----
Session information updated at 2024-02-09 11:58
```

Example 6 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 7 (python):
```python
sc.settings.set_figure_params(dpi=80,dpi_save=300)
```

Example 8 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk'
```

Example 9 (unknown):
```unknown
celltype_rename = utils.read_pkl('/nfs/team205/heart/anndata_objects/Foetal/fine_grain_rename_dictionary.pkl')
finegrain_order = list(celltype_rename.keys())
```

Example 10 (unknown):
```unknown
peakmatrix_dir = '/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/PeakMatrix'
```

Example 11 (python):
```python
# peak matrix, not-binarised
adata = sc.read_h5ad(f'{peakmatrix_dir}/Foetal_Peaks_tfidf.h5ad')
```

Example 12 (python):
```python
# rename fine_grain cell types
adata.obs['fine_grain'] = [celltype_rename[x] for x in adata.obs['fine_grain']]
```

Example 13 (unknown):
```unknown
AnnData object with n_obs × n_vars = 167022 × 508040
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels'
    var: 'seqnames', 'start', 'end', 'width', 'strand', 'score', 'replicateScoreQuantile', 'groupScoreQuantile', 'Reproducibility', 'GroupReplicate', 'distToGeneStart', 'nearestGene', 'peakType', 'distToTSS', 'nearestTSS', 'GC', 'idx', 'N'
```

Example 14 (python):
```python
adata.X.data[:10]
```

Example 15 (unknown):
```unknown
array([1.768611 , 1.9229858, 3.635446 , 1.3192275, 1.4049321, 1.8123279,
       1.6739427, 2.9703567, 3.3852873, 1.2617257], dtype=float32)
```

Example 16 (python):
```python
# accessible matrix
acc_mat = utils.get_exp_proportion(adata, groupby='fine_grain', use_rep='X')
```

Example 17 (unknown):
```unknown
dar_res_dir = '/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/MarkerPeaks'
os.listdir(dar_res_dir)
```

Example 18 (unknown):
```unknown
['SANP-vs-aCM.csv',
 'global_FDR001.csv',
 '.ipynb_checkpoints',
 'archive',
 'per-mid-grain.csv',
 'Mesenchymal_fine-grain_all.csv',
 'Cardiomyocytes_fine-grain_all.csv',
 'Leukocytes_fine-grain_all.csv',
 'Endothelium_fine-grain_all.csv',
 'Epicardium_fine-grain_all.csv',
 'Neural_fine-grain_all.csv',
 'global_coarse-grain_FDR005_logFC1.csv',
 'vCM_Left_Trab-vs-Compact.csv',
 'vCM_Left_Compact-vs-Trab.csv',
 'aCM-vs-SANP.csv']
```

Example 19 (python):
```python
DAR_res={}
for file in os.listdir(dar_res_dir):
    if '_fine-grain_all.csv' in file:
        celltype = file.split('_fine-grain_all.csv')[0]
        # read in 
        df = pd.read_csv(f'{dar_res_dir}/{file}')
        # make peak names
        df['names'] = df['seqnames']+':'+df['start'].astype('str')+'_'+df['end'].astype('str')
        # add to the dictionary
        DAR_res[celltype] = df.copy()
DAR_res.keys()
```

Example 20 (unknown):
```unknown
dict_keys(['Mesenchymal', 'Cardiomyocytes', 'Leukocytes', 'Endothelium', 'Epicardium', 'Neural'])
```

Example 21 (unknown):
```unknown
DAR_res['Cardiomyocytes'].head()
```

Example 22 (python):
```python
from scipy.stats import zscore
plt.rcParams['axes.grid'] = False

# set variables
accessible_proportion_thresh = 0.1
pval_thresh = 0.05
logFC_thresh_specific = 1
logFC_thresh_conserved = 0.3 # conserved: peaks which are not significant and the logFC is lower than this threshold 

# for storing results
specific_peaks = {}
conserved_peaks = {}

for celltype in DAR_res.keys():
    print(f'##### {celltype} #####')
    
    res = DAR_res[celltype].copy()
    # rename fine_grain cell types
    res['celltype'] = [celltype_rename[x] for x in res['celltype']]
    
    # select peaks accessible in at least one fine grain cell type
    df = acc_mat[list(set(res['celltype']))]
    peaks_open = list(df.index[df.max(axis=1)>accessible_proportion_thresh])
    res = res[res['names'].isin(peaks_open)]
    
    # select specific peaks    
    res_spec = res[(res['FDR']<pval_thresh)&(res['Log2FC']>logFC_thresh_specific)]
    peaks_spec = list(set(res_spec['names']))
    specific_peaks[celltype] = peaks_spec
    print(f'specific peaks: {len(peaks_spec)}')
    
    # select conserved peaks  
    ## prepare pivot dataframe
    res_pivot_logFC = res.pivot(index='names',columns='celltype',values='Log2FC')
    res_pivot_pval = res.pivot(index='names',columns='celltype',values='FDR')
    ## select
    peaks_cons = list(res_pivot_logFC.index[(res_pivot_logFC.max(axis=1)<logFC_thresh_conserved)&\
                                            (res_pivot_pval.min(axis=1)>pval_thresh)])
    conserved_peaks[celltype] = peaks_cons
    print(f'conserved peaks: {len(peaks_cons)}')
    
    # prepare mean count dataframe
    peak_count_df = adata[:,peaks_spec+peaks_cons].to_df()
    peak_count_df['fine_grain'] = adata.obs['fine_grain']
    peak_count_df = peak_count_df.groupby('fine_grain').mean().T
    peak_count_df = peak_count_df[list(set(res['celltype']))]
    ## scale
    peak_count_df = peak_count_df.apply(zscore,axis=1)
    ## separate
    df_spec = peak_count_df.loc[peaks_spec]
    df_cons = peak_count_df.loc[peaks_cons]
    
    # plot specific peaks
    if len(df_spec)>1:
        sns.clustermap(df_spec,yticklabels=False,figsize=(len(set(res['celltype']))/2.5, 10),
                       cmap='viridis')
        plt.title(f'{celltype} (specific peaks)')
        plt.savefig(f'{figdir}/Peak-heatmap_specific_{celltype}.pdf', bbox_inches="tight",dpi=300)
        plt.show()
    else:
        continue
    
    if len(df_cons)>1:
        # reoirder columns
        l = list(df_cons.columns)
        l.sort()
        df_cons = df_cons[l]
        # plot
        sns.clustermap(df_cons,col_cluster=False,yticklabels=False,figsize=(len(set(res['celltype']))/2.5, 10),
                      cmap='viridis')
        plt.title(f'{celltype} (conserved peaks)')
        plt.show()
    else:
        continue
    
    print('')
```

Example 23 (unknown):
```unknown
##### Mesenchymal #####
specific peaks: 2755
conserved peaks: 427
```

Example 24 (python):
```python
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/seaborn/matrix.py:560: UserWarning: Clustering large matrix with scipy. Installing `fastcluster` may give better performance.
  warnings.warn(msg)
```

Example 25 (unknown):
```unknown
##### Cardiomyocytes #####
specific peaks: 857
conserved peaks: 1492
```

Example 26 (python):
```python
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/seaborn/matrix.py:560: UserWarning: Clustering large matrix with scipy. Installing `fastcluster` may give better performance.
  warnings.warn(msg)
```

Example 27 (python):
```python
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/seaborn/matrix.py:560: UserWarning: Clustering large matrix with scipy. Installing `fastcluster` may give better performance.
  warnings.warn(msg)
```

Example 28 (unknown):
```unknown
##### Leukocytes #####
specific peaks: 9013
conserved peaks: 84
```

Example 29 (python):
```python
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/seaborn/matrix.py:560: UserWarning: Clustering large matrix with scipy. Installing `fastcluster` may give better performance.
  warnings.warn(msg)
```

Example 30 (unknown):
```unknown
##### Endothelium #####
specific peaks: 1535
conserved peaks: 941
```

Example 31 (python):
```python
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/seaborn/matrix.py:560: UserWarning: Clustering large matrix with scipy. Installing `fastcluster` may give better performance.
  warnings.warn(msg)
```

Example 32 (unknown):
```unknown
##### Epicardium #####
specific peaks: 1
conserved peaks: 8794
##### Neural #####
specific peaks: 2069
conserved peaks: 2361
```

Example 33 (python):
```python
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/seaborn/matrix.py:560: UserWarning: Clustering large matrix with scipy. Installing `fastcluster` may give better performance.
  warnings.warn(msg)
```

Example 34 (python):
```python
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/seaborn/matrix.py:560: UserWarning: Clustering large matrix with scipy. Installing `fastcluster` may give better performance.
  warnings.warn(msg)
```

Example 35 (python):
```python
plt.rcParams['figure.figsize'] = 10,3
fig, axs = plt.subplots(nrows=1,ncols=5)

celltypes_to_plot = ['Cardiomyocytes', 'Mesenchymal', 'Endothelium','Leukocytes', 'Neural']
for i,celltype in enumerate(celltypes_to_plot):
    # prepare dataframe
    peak_meta = adata.var.copy()
    peak_meta['type'] = np.nan
    peak_meta.loc[specific_peaks[celltype],'type'] = 'specific'
    peak_meta.loc[conserved_peaks[celltype],'type'] = 'conserved'
    peak_meta.dropna(inplace=True)
    
    peak_meta['peakType'] = peak_meta['peakType'].astype('category')
    peak_meta['peakType'].cat.reorder_categories(['Promoter','Exonic','Intronic','Distal'],
                                            inplace=True)
    
    # plot
    if len(set(peak_meta['type']))==2:
        tmp = pd.crosstab(peak_meta['type'], peak_meta['peakType'], normalize='index')
        # tmp.plot.bar().legend(loc=1)
        tmp.plot.bar(stacked=True,width=0.7,ax=axs[i],legend=False)
        axs[i].grid(False)
        axs[i].set_title(celltype)
        axs[i].set_xlabel('')
        axs[i].set_xticklabels(axs[i].get_xticklabels(), rotation=45, 
                                  ha='right',rotation_mode='anchor')
        if i==0:
            axs[i].set_ylabel('proportion')
        else:
            axs[i].set_ylabel('')
        
        if i==(len(celltypes_to_plot)-1):
            axs[i].legend(bbox_to_anchor=(1.0, 1.0))
fig.tight_layout()
fig.savefig(f'{figdir}/Peak-barplot_conserved-specific_per-celltype.pdf', bbox_inches="tight",dpi=300)
```

Example 36 (unknown):
```unknown
/tmp/ipykernel_26894/2711423203.py:14: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
  peak_meta['peakType'].cat.reorder_categories(['Promoter','Exonic','Intronic','Distal'],
/tmp/ipykernel_26894/2711423203.py:14: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
  peak_meta['peakType'].cat.reorder_categories(['Promoter','Exonic','Intronic','Distal'],
/tmp/ipykernel_26894/2711423203.py:14: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
  peak_meta['peakType'].cat.reorder_categories(['Promoter','Exonic','Intronic','Distal'],
/tmp/ipykernel_26894/2711423203.py:14: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
  peak_meta['peakType'].cat.reorder_categories(['Promoter','Exonic','Intronic','Distal'],
/tmp/ipykernel_26894/2711423203.py:14: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
  peak_meta['peakType'].cat.reorder_categories(['Promoter','Exonic','Intronic','Distal'],
```

---

## Import libraries¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig4_Cardiomyocytes/AtrialCardiomyocytes/CellOracle/CellOracle_no3_summarise-output.html

**Contents:**
- Import libraries¶
- Parameters¶
- Read in Oracle object¶
- Read in celloracle in-silico perturbation output¶
- Annotate grids¶
- Plot heatmap of inner-product score¶

ref: https://morris-lab.github.io/CellOracle.documentation/notebooks/04_Network_analysis/Network_analysis_with_Paul_etal_2015_data.html

First we need to align cells and grids

Next, lets give ID for each grid and make sure we are assigning the calculated inner product score correctly

Next step is to annotate each grid

5664 rows × 3 columns

6777 rows × 6 columns

**Examples:**

Example 1 (python):
```python
import os
import sys

import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import scanpy as sc
import seaborn as sns

import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42
```

Example 2 (python):
```python
import celloracle as co
co.__version__
```

Example 3 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)

spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/regulon_plot.py")
regulon_plot = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = regulon_plot
spec.loader.exec_module(regulon_plot)
```

Example 4 (python):
```python
sc.settings.set_figure_params(dpi=120)
```

Example 5 (python):
```python
%matplotlib inline
```

Example 6 (unknown):
```unknown
!ls /nfs/team205/heart/anndata_objects/Foetal/celloracle
```

Example 7 (unknown):
```unknown
archive_AtrialCM-Left_in-silico_perturbation_May2025.pkl
archive_AtrialCM-Right_in-silico_perturbation_May2025.pkl
AtrialCM_in-silico_perturbation_May2025.pkl
AtrialCM-Right_in-silico_perturbation_May2025.pkl
figures_pm
foetal_AtrialCM.celloracle.oracle
foetal_AtrialCM_links.celloracle.links
```

Example 8 (unknown):
```unknown
# Parameters
# working_dir = "/nfs/team205/heart/Vitro/TF_selection/foetal_vCM_left_compact-vs-trabeculated_downstream/CO_in-silico-perturbation_papermill"
path_to_oracle_object = "/nfs/team205/heart/anndata_objects/Foetal/celloracle/foetal_AtrialCM.celloracle.oracle"
path_to_oracle_links = "/nfs/team205/heart/anndata_objects/Foetal/celloracle/foetal_AtrialCM_links.celloracle.links"
n_grid = 40
min_mass = 0.011
cell_group_name = "AtrialCM"
path_to_res_dict = f"/nfs/team205/heart/anndata_objects/Foetal/celloracle/{cell_group_name}_in-silico_perturbation_May2025.pkl"
```

Example 9 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk/Aug2024'
table_dir = '/nfs/team205/heart/fetal_ms_figures/tables/Aug2024'
```

Example 10 (unknown):
```unknown
# Load
oracle = co.load_hdf5(path_to_oracle_object)
```

Example 11 (python):
```python
%matplotlib inline
# Check clustering data
celltype_col = 'fine_grain_time-binned'
sc.pl.umap(oracle.adata, color=celltype_col)
```

Example 12 (python):
```python
dev_dict = utils.read_pkl(path_to_res_dict)
print(len(dev_dict.keys()))
dev_dict.keys()
```

Example 13 (unknown):
```unknown
dict_keys(['ETS2', 'MAF', 'IKZF2', 'ETV6', 'BHLHE41', 'GTF2IRD1', 'GATA5', 'PITX2', 'SIX5'])
```

Example 14 (unknown):
```unknown
?dev.visualize_development_module_layout_0
```

Example 15 (unknown):
```unknown
Object `dev.visualize_development_module_layout_0` not found.
```

Example 16 (python):
```python
%matplotlib inline
# Hide all INFO (and DEBUG) messages from Matplotlib
import logging
logging.getLogger("matplotlib").setLevel(logging.WARNING)

vm = 0.2
scale_simulation = 5
scale_dev = 40

for tf in dev_dict.keys():
    dev = dev_dict[tf]
    print(f'##### {tf} #####')
    dev.visualize_development_module_layout_0(s=5,
                                              scale_for_simulation=scale_simulation,
                                              s_grid=50,
                                              scale_for_pseudotime=scale_dev,
                                              vm=vm)
    plt.show()
```

Example 17 (unknown):
```unknown
##### ETS2 #####
```

Example 18 (unknown):
```unknown
##### MAF #####
```

Example 19 (unknown):
```unknown
##### IKZF2 #####
```

Example 20 (unknown):
```unknown
##### ETV6 #####
```

Example 21 (unknown):
```unknown
##### BHLHE41 #####
```

Example 22 (unknown):
```unknown
##### GTF2IRD1 #####
```

Example 23 (unknown):
```unknown
##### GATA5 #####
```

Example 24 (unknown):
```unknown
##### PITX2 #####
```

Example 25 (unknown):
```unknown
##### SIX5 #####
```

Example 26 (unknown):
```unknown
# pick one tf for assessing the alignment is correct
tf = 'PITX2'
dev = dev_dict[tf]
```

Example 27 (python):
```python
## let's make sure the obtained coordinates are making sense
# cell embedding
cell_embedding_coords = oracle.adata.obsm['X_umap'].copy()
sns.scatterplot(x=cell_embedding_coords[:,0],
                y=cell_embedding_coords[:,1],
                s=3
               )
plt.title('cell embedding')
plt.show()

# grid embedding
grid_embedding_coords = dev.gridpoints_coordinates[~dev.mass_filter_reference]
sns.scatterplot(x=grid_embedding_coords[:,0],
                y=grid_embedding_coords[:,1],
                s=25
               )
plt.title('grid embedding')
plt.show()
```

Example 28 (python):
```python
# groundtruth plot, same as above
print(f'##### {tf} #####')
dev.plot_inner_product_on_grid(vm=vm)
plt.show()
```

Example 29 (unknown):
```unknown
##### PITX2 #####
```

Example 30 (python):
```python
# align score with coordinates
inner_product_df = dev.inner_product_df.copy()
inner_product_df[['x','y']] = dev.gridpoints_coordinates[~dev.mass_filter_reference]

from matplotlib.colors import TwoSlopeNorm
x = inner_product_df['x']
y = inner_product_df['y']
c = inner_product_df['score']

# set up the norm so that 0 is centered
norm = TwoSlopeNorm(vmin=c.min(),
                    vcenter=0,
                    vmax=c.max())

sns.scatterplot(x=x,y=y,c=c,cmap='PiYG',hue_norm=norm,s=25)
plt.show()
```

Example 31 (python):
```python
celltype_df = oracle.adata.obs[[celltype_col]].copy()
celltype_df[['x','y']] = oracle.adata.obsm['X_umap'].copy()
celltype_df
```

Example 32 (python):
```python
# annotate grids
import pandas as pd
import numpy as np
from sklearn.neighbors import NearestNeighbors

def annotate_grid(cells_df, grid_df, k, celltype_col):
    """
    Annotate each grid point with the nearest cell_type label from cells_df.
    
    Parameters:
    - cells_df: DataFrame with columns ['x', 'y', celltype_col]
    - grid_df:  DataFrame with columns ['x', 'y']
    - k:        Number of nearest neighbors to consider (majority vote if k > 1)
    
    Returns:
    - grid_df with an added 'cell_type' column.
    """
    coords = cells_df[['x', 'y']].values
    nbrs = NearestNeighbors(n_neighbors=k).fit(coords)
    distances, indices = nbrs.kneighbors(grid_df[['x', 'y']].values)
    
    if k == 1:
        grid_df[celltype_col] = cells_df[celltype_col].values[indices.flatten()]
    else:
        # Majority vote over k neighbors
        labels = cells_df[celltype_col].values[indices]  # shape (n_grid, k)
        majority = []
        for row in labels:
            uniq, counts = np.unique(row, return_counts=True)
            majority.append(uniq[np.argmax(counts)])
        grid_df[celltype_col] = majority
    
    return grid_df
```

Example 33 (python):
```python
# to decide k
celltype_df.shape[0]/inner_product_df.shape[0]
```

Example 34 (unknown):
```unknown
7.52191235059761
```

Example 35 (unknown):
```unknown
celltype_labels = annotate_grid(cells_df=celltype_df, 
                                  grid_df=inner_product_df, 
                                  k=7,
                                  celltype_col=celltype_col)[celltype_col]
celltype_labels
```

Example 36 (unknown):
```unknown
0       AtrialCardiomyocytesLeft_t3
1       AtrialCardiomyocytesLeft_t3
2       AtrialCardiomyocytesLeft_t3
3       AtrialCardiomyocytesLeft_t3
4       AtrialCardiomyocytesLeft_t3
                   ...             
748     AtrialCardiomyocytesLeft_t1
749     AtrialCardiomyocytesLeft_t1
750     AtrialCardiomyocytesLeft_t1
751    AtrialCardiomyocytesRight_t2
752    AtrialCardiomyocytesRight_t2
Name: fine_grain_time-binned, Length: 753, dtype: object
```

Example 37 (python):
```python
x = inner_product_df['x']
y = inner_product_df['y']
c = celltype_labels
sns.scatterplot(x=x,y=y,hue=c,s=25)
plt.legend(bbox_to_anchor=(1, 0.5))
plt.show()
```

Example 38 (python):
```python
# update cell type name
df = pd.read_csv('/nfs/team205/heart/anndata_objects/Foetal/finegrain_name_mapping.csv')
celltype_mapping = df.set_index('Full_name')['Short_name_finalised'].to_dict()
celltype_labels_mod = [f'{celltype_mapping[x.split("_")[0]]}_{x.split("_")[1]}' for x in celltype_labels]
celltype_labels_mod[:5]
```

Example 39 (unknown):
```unknown
['aCML_t3', 'aCML_t3', 'aCML_t3', 'aCML_t3', 'aCML_t3']
```

Example 40 (python):
```python
# read in post-g2g-selection csv file and get selected TFs
# log2FC<0 means TFs specific to Right-aCM
# log2FC>0 means TFs specific to Left-aCM
g2g_sel = pd.read_csv('/nfs/team205/kk18/notebooks/Foetal/AtrialCM_celloracle/AtrialCM_post-g2g_selected-recgulons.csv')
display(g2g_sel)

# TFs, only activator
list_of_TFs = [x.split('_')[0] for x in g2g_sel['Gene'] if '_+' in x]
print(len(list_of_TFs))
```

Example 41 (python):
```python
for i,tf in enumerate(dev_dict.keys()):
    dev = dev_dict[tf]
    inner_product_df = dev.inner_product_df.copy()
    inner_product_df['TF'] = tf
    inner_product_df[celltype_col] = celltype_labels_mod
    if i==0:
        inner_product_df_all = inner_product_df.copy()
    else:
        inner_product_df_all = pd.concat([inner_product_df_all,inner_product_df])
inner_product_df_all
```

Example 42 (python):
```python
aggr_df = inner_product_df_all[['score','TF',celltype_col]].groupby(['TF',celltype_col]).sum() # "sum", so this is reflecting the number of grids (cells)
aggr_df = aggr_df.reset_index()
df = aggr_df.pivot(index='TF',columns=celltype_col,values='score')
# reorder based on g2g-log2FC values
df = df.loc[list_of_TFs.rev]

plt.rcParams['figure.figsize'] = 3,3
sns.heatmap(df, center=0, cmap='PiYG')
plt.savefig(f"{figdir}/AtrialCM_CellOracle_heatmap.pdf", bbox_inches="tight",dpi=300)
plt.show()
```

Example 43 (unknown):
```unknown
2025-05-13 17:05:12,591 - INFO - maxp pruned
2025-05-13 17:05:12,601 - INFO - cmap pruned
2025-05-13 17:05:12,603 - INFO - kern dropped
2025-05-13 17:05:12,604 - INFO - post pruned
2025-05-13 17:05:12,604 - INFO - FFTM dropped
2025-05-13 17:05:12,606 - INFO - GPOS pruned
2025-05-13 17:05:12,614 - INFO - GSUB pruned
2025-05-13 17:05:12,622 - INFO - glyf pruned
2025-05-13 17:05:12,625 - INFO - Added gid0 to subset
2025-05-13 17:05:12,627 - INFO - Added first four glyphs to subset
2025-05-13 17:05:12,627 - INFO - Closing glyph list over 'MATH': 44 glyphs before
2025-05-13 17:05:12,628 - INFO - Glyph names: ['.notdef', '.null', 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'K', 'L', 'M', 'P', 'R', 'S', 'T', 'V', 'X', 'Z', 'a', 'b', 'd', 'e', 'f', 'five', 'four', 'g', 'hyphen', 'i', 'm', 'minus', 'n', 'nonmarkingreturn', 'one', 'r', 'six', 'space', 't', 'three', 'two', 'underscore', 'zero']
2025-05-13 17:05:12,629 - INFO - Glyph IDs:   [0, 1, 2, 3, 16, 19, 20, 21, 22, 23, 24, 25, 36, 37, 38, 39, 40, 41, 42, 43, 44, 46, 47, 48, 51, 53, 54, 55, 57, 59, 61, 66, 68, 69, 71, 72, 73, 74, 76, 80, 81, 85, 87, 3228]
2025-05-13 17:05:12,633 - INFO - Closed glyph list over 'MATH': 44 glyphs after
2025-05-13 17:05:12,634 - INFO - Glyph names: ['.notdef', '.null', 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'K', 'L', 'M', 'P', 'R', 'S', 'T', 'V', 'X', 'Z', 'a', 'b', 'd', 'e', 'f', 'five', 'four', 'g', 'hyphen', 'i', 'm', 'minus', 'n', 'nonmarkingreturn', 'one', 'r', 'six', 'space', 't', 'three', 'two', 'underscore', 'zero']
2025-05-13 17:05:12,634 - INFO - Glyph IDs:   [0, 1, 2, 3, 16, 19, 20, 21, 22, 23, 24, 25, 36, 37, 38, 39, 40, 41, 42, 43, 44, 46, 47, 48, 51, 53, 54, 55, 57, 59, 61, 66, 68, 69, 71, 72, 73, 74, 76, 80, 81, 85, 87, 3228]
2025-05-13 17:05:12,635 - INFO - Closing glyph list over 'GSUB': 44 glyphs before
2025-05-13 17:05:12,635 - INFO - Glyph names: ['.notdef', '.null', 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'K', 'L', 'M', 'P', 'R', 'S', 'T', 'V', 'X', 'Z', 'a', 'b', 'd', 'e', 'f', 'five', 'four', 'g', 'hyphen', 'i', 'm', 'minus', 'n', 'nonmarkingreturn', 'one', 'r', 'six', 'space', 't', 'three', 'two', 'underscore', 'zero']
2025-05-13 17:05:12,636 - INFO - Glyph IDs:   [0, 1, 2, 3, 16, 19, 20, 21, 22, 23, 24, 25, 36, 37, 38, 39, 40, 41, 42, 43, 44, 46, 47, 48, 51, 53, 54, 55, 57, 59, 61, 66, 68, 69, 71, 72, 73, 74, 76, 80, 81, 85, 87, 3228]
2025-05-13 17:05:12,640 - INFO - Closed glyph list over 'GSUB': 47 glyphs after
2025-05-13 17:05:12,647 - INFO - Glyph names: ['.notdef', '.null', 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'K', 'L', 'M', 'P', 'R', 'S', 'T', 'V', 'X', 'Z', 'a', 'b', 'd', 'e', 'f', 'fi', 'five', 'four', 'g', 'hyphen', 'i', 'm', 'minus', 'n', 'nonmarkingreturn', 'one', 'r', 'six', 'space', 't', 'three', 'two', 'underscore', 'uniFB00', 'uniFB03', 'zero']
2025-05-13 17:05:12,647 - INFO - Glyph IDs:   [0, 1, 2, 3, 16, 19, 20, 21, 22, 23, 24, 25, 36, 37, 38, 39, 40, 41, 42, 43, 44, 46, 47, 48, 51, 53, 54, 55, 57, 59, 61, 66, 68, 69, 71, 72, 73, 74, 76, 80, 81, 85, 87, 3228, 5038, 5039, 5041]
2025-05-13 17:05:12,648 - INFO - Closing glyph list over 'glyf': 47 glyphs before
2025-05-13 17:05:12,648 - INFO - Glyph names: ['.notdef', '.null', 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'K', 'L', 'M', 'P', 'R', 'S', 'T', 'V', 'X', 'Z', 'a', 'b', 'd', 'e', 'f', 'fi', 'five', 'four', 'g', 'hyphen', 'i', 'm', 'minus', 'n', 'nonmarkingreturn', 'one', 'r', 'six', 'space', 't', 'three', 'two', 'underscore', 'uniFB00', 'uniFB03', 'zero']
2025-05-13 17:05:12,649 - INFO - Glyph IDs:   [0, 1, 2, 3, 16, 19, 20, 21, 22, 23, 24, 25, 36, 37, 38, 39, 40, 41, 42, 43, 44, 46, 47, 48, 51, 53, 54, 55, 57, 59, 61, 66, 68, 69, 71, 72, 73, 74, 76, 80, 81, 85, 87, 3228, 5038, 5039, 5041]
2025-05-13 17:05:12,650 - INFO - Closed glyph list over 'glyf': 47 glyphs after
2025-05-13 17:05:12,650 - INFO - Glyph names: ['.notdef', '.null', 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'K', 'L', 'M', 'P', 'R', 'S', 'T', 'V', 'X', 'Z', 'a', 'b', 'd', 'e', 'f', 'fi', 'five', 'four', 'g', 'hyphen', 'i', 'm', 'minus', 'n', 'nonmarkingreturn', 'one', 'r', 'six', 'space', 't', 'three', 'two', 'underscore', 'uniFB00', 'uniFB03', 'zero']
2025-05-13 17:05:12,651 - INFO - Glyph IDs:   [0, 1, 2, 3, 16, 19, 20, 21, 22, 23, 24, 25, 36, 37, 38, 39, 40, 41, 42, 43, 44, 46, 47, 48, 51, 53, 54, 55, 57, 59, 61, 66, 68, 69, 71, 72, 73, 74, 76, 80, 81, 85, 87, 3228, 5038, 5039, 5041]
2025-05-13 17:05:12,652 - INFO - Retaining 47 glyphs
2025-05-13 17:05:12,659 - INFO - head subsetting not needed
2025-05-13 17:05:12,659 - INFO - hhea subsetting not needed
2025-05-13 17:05:12,660 - INFO - maxp subsetting not needed
2025-05-13 17:05:12,660 - INFO - OS/2 subsetting not needed
2025-05-13 17:05:12,664 - INFO - hmtx subsetted
2025-05-13 17:05:12,667 - INFO - cmap subsetted
2025-05-13 17:05:12,674 - INFO - fpgm subsetting not needed
2025-05-13 17:05:12,675 - INFO - prep subsetting not needed
2025-05-13 17:05:12,675 - INFO - cvt  subsetting not needed
2025-05-13 17:05:12,676 - INFO - loca subsetting not needed
2025-05-13 17:05:12,676 - INFO - post subsetted
2025-05-13 17:05:12,677 - INFO - gasp subsetting not needed
2025-05-13 17:05:12,677 - INFO - MATH subsetted
2025-05-13 17:05:12,680 - INFO - GDEF subsetted
2025-05-13 17:05:12,730 - INFO - GPOS subsetted
2025-05-13 17:05:12,733 - INFO - GSUB subsetted
2025-05-13 17:05:12,738 - INFO - name subsetting not needed
2025-05-13 17:05:12,741 - INFO - glyf subsetted
2025-05-13 17:05:12,742 - INFO - head pruned
2025-05-13 17:05:12,746 - INFO - OS/2 Unicode ranges pruned: [0, 38]
2025-05-13 17:05:12,747 - INFO - OS/2 CodePage ranges pruned: [0]
2025-05-13 17:05:12,748 - INFO - glyf pruned
2025-05-13 17:05:12,749 - INFO - GDEF pruned
2025-05-13 17:05:12,750 - INFO - GPOS pruned
2025-05-13 17:05:12,750 - INFO - GSUB pruned
2025-05-13 17:05:12,768 - INFO - name pruned
```

---

## Functions¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/Visium/Visium_Intagration-Annotation/Visium_integration_no3_annotation.html

**Contents:**
- Functions¶
- Cell types for supporting annotation¶
- Read in anndata¶
- Annotate¶
- Subset and re-integration - Myocardium¶
- Subset and re-integration - CCS, Valve cells¶
- Subset and re-integration - Atrium¶
- Subset and re-integration - Vessels¶
- Subset and re-integration - Node cells¶
- Annotate based on cell2location estimated abundance¶
- Collect all annotations so far¶
- Refine annotations (with using manual annotations)¶
  - Fully dependent on manual annotation¶
  - Partly dependent on manual annotation¶
  - Coronary vessel vs Great vessel¶
  - Myocardium Atrial¶
  - Myocardium Ventricular - Compact¶
  - Myocardium Ventricular - Trabeculated¶
- Add to anndata¶
- Save¶

39223 rows × 7 columns

if "pre-final" is 'tbd -' or 'Uncertain', then use manual annotation

Overwrite "coronary vessel" spots ("pre-final") which were manualy annotated as "Great vessels"

**Examples:**

Example 1 (python):
```python
import sys
import scanpy as sc
import anndata
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt 
import matplotlib as mpl

from matplotlib import rcParams
rcParams['pdf.fonttype'] = 42 # enables correct plotting of text
import seaborn as sns

import os

import bbknn
```

Example 2 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 3 (python):
```python
# This function selects the data for multiple slides from the spatial anndata object.
# Modified `select_slide` in cell2loc for multiple slides

# @param adata: anndata object with multiple spatial experiments
# @param s: list of selected slides
# @param s_col: column in adata.obs listing experiment name for each location

def select_multislide(adata, s, s_col='sample'):

    slides = adata[adata.obs[s_col].isin(s), :]
    s_keys = list(slides.uns['spatial'].keys())
    
    uns_dict = {}
    for ss in s:
        s_spatial=np.array(s_keys)[[ss in k for k in s_keys]][0]
        uns_dict[s_spatial]=slides.uns['spatial'][s_spatial]
    
    slides.uns['spatial'] = uns_dict

    return slides
```

Example 4 (unknown):
```unknown
celltypes_for_annotation = {
    'Atrium':['AtrialCardiomyocytesLeft','AtrialCardiomyocytesRight'],
    'Ventricle':['VentricularCardiomyocytesLeftCompact',
                 'VentricularCardiomyocytesRightCompact',
                 'VentricularCardiomyocytesLeftTrabeculated',
                 'VentricularCardiomyocytesRightTrabeculated'],
    'CCS':['SinoatrialNodeCardiomyocytes',
             'AtrioventricularNodeCardiomyocytes',
             'VentricularConductionSystemProximal',
             'VentricularConductionSystemDistal',],
    'Adventitia':['GreatVesselAdventitialFibroblasts','CoronaryVesselAdventitialFibroblasts'],
    'LN':['LymphNodeFibroblasticReticularCells'],
    'Valve':['ValveInterstitialCells'],
    'Endocardium':['EndocardialCells','EndocardialCushionCells'],
    'Epicardium':['MesothelialEpicardialCells','EpicardiumDerivedCells'],
    'DuctusArteriosus':['DuctusArteriosusSmoothMuscleCells']
}
```

Example 5 (python):
```python
# the anndata has "gene expression" in .X
# cell2location output is in .obsm['q05_cell_abundance_w_sf']
# kNN graph for UMAP/clustering was generated using cell2location output
adata_vis = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/visium_foetal_healthy-all_raw.h5ad')
adata_vis
```

Example 6 (unknown):
```unknown
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/anndata/_core/anndata.py:1830: UserWarning: Variable names are not unique. To make them unique, call `.var_names_make_unique`.
  utils.warn_names_duplicates("var")
```

Example 7 (unknown):
```unknown
AnnData object with n_obs × n_vars = 39223 × 33562
    obs: 'array_row', 'array_col', 'sample', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'mt_frac', 'sangerID', 'Publication', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'status', 'cell2loc-map_group', 'JC_edit', 'sample_for_cell2loc', 'week', 'leiden', 'region_cluster', 'AtrialCardiomyocytesLeft', 'AtrialCardiomyocytesRight', 'VentricularCardiomyocytesLeftCompact', 'VentricularCardiomyocytesRightCompact', 'VentricularCardiomyocytesLeftTrabeculated', 'VentricularCardiomyocytesRightTrabeculated', 'SinoatrialNodeCardiomyocytes', 'AtrioventricularNodeCardiomyocytes', 'VentricularConductionSystemProximal', 'VentricularConductionSystemDistal', 'GreatVesselAdventitialFibroblasts', 'CoronaryVesselAdventitialFibroblasts', 'MyocardialInterstitialFibroblasts1', 'MyocardialInterstitialFibroblasts2', 'Myofibroblasts', 'LymphNodeFibroblasticReticularCells', 'ValveInterstitialCells', 'GreatVesselSmoothMuscleCells', 'CoronarySmoothMuscleCells', 'CoronaryPericytes', 'GreatVesselArterialEndothelialCells', 'GreatVesselVenousEndothelialCells', 'CoronaryArterialEndothelialCells', 'CoronaryVenousEndothelialCells', 'CoronaryCapillaryEndothelialCells', 'EndocardialCells', 'EndocardialCushionCells', 'ValveEndothelialCells', 'LymphaticEndothelialCells', 'MesothelialEpicardialCells', 'EpicardiumDerivedCells', 'NeuronPrecursors', 'ChromaffinCells', 'SympatheticNeurons', 'ParaympatheticNeurons', 'SchwannCellPrecursors', 'SchwannCells', 'MonocytesMPOpos', 'Monocytes', 'MonocyteDerivedCells', 'MacrophagesCX3CR1pos', 'MacrophagesTIMD4pos', 'MacrophagesLYVE1pos', 'MacrophagesATF3pos', 'DendriticCellsType1', 'DendriticCellsMature', 'PlasmacytoidDendriticCells', 'MastCells', 'Megakaryocytes', 'TCellsCD4pos', 'TCellsCD8pos', 'TregsCD4pos', 'ProBCells', 'BCells', 'BCellsMS4A1pos', 'NaturalKillerCells', 'InnateLymphoidCells', 'DuctusArteriosusSmoothMuscleCells', 'AtrialCardiomyocytesLeft_minmax', 'AtrialCardiomyocytesRight_minmax', 'VentricularCardiomyocytesLeftCompact_minmax', 'VentricularCardiomyocytesRightCompact_minmax', 'VentricularCardiomyocytesLeftTrabeculated_minmax', 'VentricularCardiomyocytesRightTrabeculated_minmax', 'SinoatrialNodeCardiomyocytes_minmax', 'AtrioventricularNodeCardiomyocytes_minmax', 'VentricularConductionSystemProximal_minmax', 'VentricularConductionSystemDistal_minmax', 'GreatVesselAdventitialFibroblasts_minmax', 'CoronaryVesselAdventitialFibroblasts_minmax', 'MyocardialInterstitialFibroblasts1_minmax', 'MyocardialInterstitialFibroblasts2_minmax', 'Myofibroblasts_minmax', 'LymphNodeFibroblasticReticularCells_minmax', 'ValveInterstitialCells_minmax', 'GreatVesselSmoothMuscleCells_minmax', 'CoronarySmoothMuscleCells_minmax', 'CoronaryPericytes_minmax', 'GreatVesselArterialEndothelialCells_minmax', 'GreatVesselVenousEndothelialCells_minmax', 'CoronaryArterialEndothelialCells_minmax', 'CoronaryVenousEndothelialCells_minmax', 'CoronaryCapillaryEndothelialCells_minmax', 'EndocardialCells_minmax', 'EndocardialCushionCells_minmax', 'ValveEndothelialCells_minmax', 'LymphaticEndothelialCells_minmax', 'MesothelialEpicardialCells_minmax', 'EpicardiumDerivedCells_minmax', 'NeuronPrecursors_minmax', 'ChromaffinCells_minmax', 'SympatheticNeurons_minmax', 'ParaympatheticNeurons_minmax', 'SchwannCellPrecursors_minmax', 'SchwannCells_minmax', 'MonocytesMPOpos_minmax', 'Monocytes_minmax', 'MonocyteDerivedCells_minmax', 'MacrophagesCX3CR1pos_minmax', 'MacrophagesTIMD4pos_minmax', 'MacrophagesLYVE1pos_minmax', 'MacrophagesATF3pos_minmax', 'DendriticCellsType1_minmax', 'DendriticCellsMature_minmax', 'PlasmacytoidDendriticCells_minmax', 'MastCells_minmax', 'Megakaryocytes_minmax', 'TCellsCD4pos_minmax', 'TCellsCD8pos_minmax', 'TregsCD4pos_minmax', 'ProBCells_minmax', 'BCells_minmax', 'BCellsMS4A1pos_minmax', 'NaturalKillerCells_minmax', 'InnateLymphoidCells_minmax', 'DuctusArteriosusSmoothMuscleCells_minmax', 'annotation', 'annotation_final'
    var: 'gene_ids'
    uns: 'JC_edit_colors', 'annotation_colors', 'annotation_final_colors', 'cell2loc-map_group_colors', 'leiden', 'neighbors', 'region_cluster_colors', 'sample_for_cell2loc_colors', 'umap'
    obsm: 'MT', 'X_umap', 'means_cell_abundance_w_sf', 'q05_cell_abundance_w_sf', 'q05_cell_abundance_w_sf_ndarray', 'q95_cell_abundance_w_sf', 'spatial', 'stds_cell_abundance_w_sf'
    obsp: 'connectivities', 'distances'
```

Example 8 (python):
```python
with mpl.rc_context({'axes.facecolor':  'white',
                     'figure.figsize': [6, 6]}):
    sc.pl.umap(adata_vis, color=['region_cluster'], size=5,
               color_map = 'RdPu', ncols = 3, wspace=0.3,
               legend_fontsize=14, frameon=False, legend_loc='on data')
```

Example 9 (python):
```python
# add cell2loc out to .obs
cellnames = adata_vis.obsm['q05_cell_abundance_w_sf'].columns
cellnames = [x.split('q05cell_abundance_w_sf_')[1] for x in cellnames]
adata_vis.obs[cellnames] = adata_vis.obsm['q05_cell_abundance_w_sf'].copy()

# add normalise cell2loc output: min-max
df = adata_vis.obsm['q05_cell_abundance_w_sf'].copy()
df = (df-df.min())/(df.max()-df.min())
adata_vis.obs[[f'{x}_minmax' for x in cellnames]] = df.copy()
```

Example 10 (python):
```python
for structure,celltypes in celltypes_for_annotation.items():
    print(f'##### {structure} #####')
    with mpl.rc_context({'axes.facecolor':  'white',
                         'figure.figsize': [4, 4]}):
        sc.pl.umap(adata_vis, color=[f'{x}_minmax' for x in celltypes], size=20,
                   color_map = 'RdPu', ncols = 4, wspace=0.3, vmax='p99.98',
                   legend_fontsize=14, frameon=False)
```

Example 11 (unknown):
```unknown
##### Atrium #####
```

Example 12 (unknown):
```unknown
##### Ventricle #####
```

Example 13 (unknown):
```unknown
##### CCS #####
```

Example 14 (unknown):
```unknown
##### Adventitia #####
```

Example 15 (unknown):
```unknown
##### LN #####
```

Example 16 (unknown):
```unknown
##### Valve #####
```

Example 17 (unknown):
```unknown
##### Endocardium #####
```

Example 18 (unknown):
```unknown
##### Epicardium #####
```

Example 19 (unknown):
```unknown
##### DuctusArteriosus #####
```

Example 20 (python):
```python
adata_vis.obs['annotation'] = adata_vis.obs['region_cluster']
adata_vis.obs['annotation'] = adata_vis.obs['annotation'].astype('str')
adata_vis.obs.replace({'annotation': {
    '0': 'Ventricle', 
    '1': 'Ventricle', 
    '2': 'Ventricle', 
    '3': 'Ventricle', 
    '4': 'Ventricle', 
    '5': 'Ventricle', 
    '6': 'Ventricle', 
    '7': 'Ventricle', 
    '8': 'Ventricle', 
    '9': 'Atrium', 
    '10': 'AV ring - Valves', 
    '11': 'Ventricle', 
    '12': 'Vessels', 
    '13': 'Atrium', 
    '14': 'Ventricle', 
    '15': 'Atrium', 
    '16': 'Vessels', 
    '17': 'Vessels', 
    '18': 'Ventricle', 
    '19': 'Ventricle',   
    '20': 'Vessels', 
    '21': 'Vessels', 
    '22': 'Vessels', 
    '23': 'Vessels', 
}}, inplace=True)
```

Example 21 (python):
```python
with mpl.rc_context({'axes.facecolor':  'white',
                     'figure.figsize': [6, 6]}):
    sc.pl.umap(adata_vis, color=['annotation'], size=5,
               color_map = 'RdPu', ncols = 3, wspace=0.3,
               legend_fontsize=14, frameon=False, legend_loc='on data')
```

Example 22 (python):
```python
set(adata_vis.obs['annotation'])
```

Example 23 (unknown):
```unknown
{'AV ring - Valves', 'Atrium', 'Ventricle', 'Vessels'}
```

Example 24 (python):
```python
# save annotations
annot_df = adata_vis.obs[['annotation']].copy()
annot_df.rename(columns={'annotation':'global_annotation'},inplace=True)
annot_df.head()
```

Example 25 (unknown):
```unknown
structures = ['AV ring - Valves','Atrium','Ventricle']
```

Example 26 (python):
```python
# subset
adata_sub = adata_vis[adata_vis.obs['annotation'].isin(structures)]

# # compute KNN using the cell2location output stored in adata.obsm
# bbknn
bbknn.bbknn(adata_sub, batch_key='sample_for_cell2loc', use_rep='q05_cell_abundance_w_sf_ndarray')

# Cluster spots into regions using scanpy
sc.tl.leiden(adata_sub, resolution=2)

# add region as categorical variable
adata_sub.obs["region_cluster"] = adata_sub.obs["leiden"].astype("category")

# compute UMAP using KNN graph based on the cell2location output
sc.tl.umap(adata_sub, min_dist = 0.2, spread = 1)  # n_components=3
```

Example 27 (unknown):
```unknown
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/anndata/compat/_overloaded_dict.py:179: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  ovld.data[key] = neighbors
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/anndata/_core/anndata.py:1830: UserWarning: Variable names are not unique. To make them unique, call `.var_names_make_unique`.
  utils.warn_names_duplicates("var")
```

Example 28 (python):
```python
# show regions in UMAP coordinates
with mpl.rc_context({'axes.facecolor':  'white',
                     'figure.figsize': [6, 6]}):
    sc.pl.umap(adata_sub, color=['week','JC_edit'], size=10,
               color_map = 'RdPu', ncols = 3, wspace=0.2,
               legend_fontsize=14, frameon=False, components='all')
```

Example 29 (python):
```python
with mpl.rc_context({'axes.facecolor':  'white',
                     'figure.figsize': [6, 6]}):
    sc.pl.umap(adata_sub, color=['JC_edit'], groups='AVN',size=10,
               color_map = 'RdPu', ncols = 3, wspace=0.2,legend_loc='on data',
               legend_fontsize=10, frameon=False, components='all')
```

Example 30 (python):
```python
with mpl.rc_context({'axes.facecolor':  'white',
                     'figure.figsize': [6, 6]}):
    sc.pl.umap(adata_sub, color=['region_cluster'], size=10,
               color_map = 'RdPu', ncols = 3, wspace=0.2,legend_loc='on data',
               legend_fontsize=10, frameon=False, components='all')
```

Example 31 (python):
```python
for structure,celltypes in celltypes_for_annotation.items():
    print(f'##### {structure} #####')
    with mpl.rc_context({'axes.facecolor':  'white',
                         'figure.figsize': [4, 4]}):
        sc.pl.umap(adata_sub, color=[f'{x}_minmax' for x in celltypes], size=10,
                   color_map = 'RdPu', ncols = 4, wspace=0.1, vmax='p99.98',
                   legend_fontsize=14, frameon=False)
```

Example 32 (unknown):
```unknown
##### Atrium #####
```

Example 33 (unknown):
```unknown
##### Ventricle #####
```

Example 34 (unknown):
```unknown
##### CCS #####
```

Example 35 (unknown):
```unknown
##### Adventitia #####
```

Example 36 (unknown):
```unknown
##### LN #####
```

Example 37 (unknown):
```unknown
##### Valve #####
```

Example 38 (unknown):
```unknown
##### Endocardium #####
```

Example 39 (unknown):
```unknown
##### Epicardium #####
```

Example 40 (unknown):
```unknown
##### DuctusArteriosus #####
```

Example 41 (python):
```python
with mpl.rc_context({'axes.facecolor':  'white',
                     'figure.figsize': [6, 6]}):
    sc.pl.umap(adata_sub, color=['region_cluster'], groups=['11','14','21'],size=10,
               color_map = 'RdPu', ncols = 3, wspace=0.2,legend_loc='on data',
               legend_fontsize=10, frameon=False, components='all')
```

Example 42 (python):
```python
adata_sub.var_names_make_unique()
sc.pl.dotplot(adata_sub,
              cellnames, 
              groupby='region_cluster',
              dendrogram=False,
              standard_scale="var",
              color_map="Reds",
              expression_cutoff=0.1
             )
```

Example 43 (python):
```python
adata_sub.obs['annotation'] = adata_sub.obs['region_cluster']
adata_sub.obs['annotation'] = adata_sub.obs['annotation'].astype('str')
adata_sub.obs.replace({'annotation': {
    '0': 'Ventricle - Compact - Right', 
    '1': 'tbd - Atrium - Left', 
    '2': 'Ventricle - Compact - Left', 
    '3': 'Ventricle - Compact - Left', 
    '4': 'Ventricle - Trabeculated - Right', 
    '5': 'Ventricle - Compact - Right',  
    '6': 'Ventricle - Trabeculated - Left', 
    '7': 'Ventricle - Compact - Left',  
    '8': 'Ventricle - Compact - Left',  
    '9': 'tbd - CCS, Valves', 
    '10': 'Ventricle - Trabeculated - Left', 
    '11': 'tbd - Ventricle', 
    '12': 'tbd - Atrium - Left', 
    '13': 'tbd - Atrium - Right', 
    '14': 'tbd - Ventricle', 
    '15': 'Endocardium - Ventricular', 
    '16': 'Ventricle - Trabeculated - Left', 
    '17': 'VCS - Distal', 
    '18': 'tbd - CCS, Valves', 
    '19': 'Ventricle - Compact - Left',  
    '20': 'tbd - CCS, Valves', 
    '21': 'tbd - Ventricle', 
    '22': 'Endocardium - Ventricular', 
    '23': 'Adventitia - Coronary', 
    '24': 'Ventricle - Compact - Right', 
}}, inplace=True)
```

Example 44 (python):
```python
with mpl.rc_context({'axes.facecolor':  'white',
                     'figure.figsize': [6, 6]}):
    sc.pl.umap(adata_sub, color=['annotation'], size=5,
               color_map = 'RdPu', ncols = 3, wspace=0.3,
               legend_fontsize=10, frameon=False, legend_loc='on data')
```

Example 45 (python):
```python
adata_sub.var_names_make_unique()
sc.pl.dotplot(adata_sub,
              cellnames, 
              groupby='annotation',
              dendrogram=False,
              standard_scale="var",
              color_map="Reds",
              expression_cutoff=0.1
             )
```

Example 46 (python):
```python
# get annotation
annot_df['myocardium_annotation'] = adata_sub.obs['annotation'].reindex(annot_df.index)
annot_df.head()
```

Example 47 (unknown):
```unknown
structures = ['tbd - CCS, Valves']

# get index
indices = list(annot_df.index[annot_df['myocardium_annotation'].isin(structures)])
```

Example 48 (python):
```python
# subset
adata_sub = adata_vis[indices]

# # compute KNN using the cell2location output stored in adata.obsm
# bbknn
bbknn.bbknn(adata_sub, batch_key='sample_for_cell2loc', use_rep='q05_cell_abundance_w_sf_ndarray')

# Cluster spots into regions using scanpy
sc.tl.leiden(adata_sub, resolution=2)

# add region as categorical variable
adata_sub.obs["region_cluster"] = adata_sub.obs["leiden"].astype("category")

# compute UMAP using KNN graph based on the cell2location output
sc.tl.umap(adata_sub, min_dist = 0.2, spread = 1)  # n_components=3
```

Example 49 (unknown):
```unknown
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/anndata/compat/_overloaded_dict.py:179: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  ovld.data[key] = neighbors
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/anndata/_core/anndata.py:1830: UserWarning: Variable names are not unique. To make them unique, call `.var_names_make_unique`.
  utils.warn_names_duplicates("var")
```

Example 50 (python):
```python
# show regions in UMAP coordinates
with mpl.rc_context({'axes.facecolor':  'white',
                     'figure.figsize': [4, 4]}):
    sc.pl.umap(adata_sub, color=['week','JC_edit'], size=20,
               color_map = 'RdPu', ncols = 3, wspace=0.2,
               legend_fontsize=14, frameon=False, components='all')
```

Example 51 (python):
```python
with mpl.rc_context({'axes.facecolor':  'white',
                     'figure.figsize': [4, 4]}):
    sc.pl.umap(adata_sub, color=['region_cluster'], size=20,
               color_map = 'RdPu', ncols = 3, wspace=0.2,legend_loc='on data',
               legend_fontsize=10, frameon=False, components='all')
```

Example 52 (python):
```python
with mpl.rc_context({'axes.facecolor':  'white',
                     'figure.figsize': [4, 4]}):
    sc.pl.umap(adata_sub, color=['region_cluster'], groups=['11','14','21','23'],size=10,
               color_map = 'RdPu', ncols = 3, wspace=0.2,legend_loc='on data',
               legend_fontsize=10, frameon=False, components='all')
```

Example 53 (python):
```python
for structure,celltypes in celltypes_for_annotation.items():
    print(f'##### {structure} #####')
    with mpl.rc_context({'axes.facecolor':  'white',
                         'figure.figsize': [4, 4]}):
        sc.pl.umap(adata_sub, color=[f'{x}_minmax' for x in celltypes], size=20,
                   color_map = 'RdPu', ncols = 4, wspace=0.3, vmax='p99.98',
                   legend_fontsize=14, frameon=False)
```

Example 54 (unknown):
```unknown
##### Atrium #####
```

Example 55 (unknown):
```unknown
##### Ventricle #####
```

Example 56 (unknown):
```unknown
##### CCS #####
```

Example 57 (unknown):
```unknown
##### Adventitia #####
```

Example 58 (unknown):
```unknown
##### LN #####
```

Example 59 (unknown):
```unknown
##### Valve #####
```

Example 60 (unknown):
```unknown
##### Endocardium #####
```

Example 61 (unknown):
```unknown
##### Epicardium #####
```

Example 62 (unknown):
```unknown
##### DuctusArteriosus #####
```

Example 63 (python):
```python
adata_sub.var_names_make_unique()
sc.pl.dotplot(adata_sub,
              cellnames, 
              groupby='region_cluster',
              dendrogram=False,
              standard_scale="var",
              color_map="Reds",
              expression_cutoff=0.1
             )
```

Example 64 (python):
```python
adata_sub.obs['annotation'] = adata_sub.obs['region_cluster']
adata_sub.obs['annotation'] = adata_sub.obs['annotation'].astype('str')
adata_sub.obs.replace({'annotation': {
    '0': 'VCS - Proximal', 
    '1': 'VCS - Proximal', 
    '2': 'tbd - AVring, Valves', 
    '3': 'tbd - AVring, Valves', 
    '4': 'Ventricle - Compact - Right', 
    '5': 'Ventricle - Trabeculated - Left', 
    '6': 'tbd - AVring, Valves', 
    '7': 'tbd - AVring, Valves',  
    '8': 'Endocardial cusion',  
    '9': 'tbd - Nodes',  
    '10': 'tbd - AVring, Valves', 
    '11': 'VCS - Proximal', 
    '12': 'tbd - AVring, Valves', 
    '13': 'tbd - AVring, Valves', 
    '14': 'tbd - AVring, Valves', 
    '15': 'Adventitia - Coronary', 
    '16': 'Endocardial cusion', 
}}, inplace=True)
```

Example 65 (python):
```python
with mpl.rc_context({'axes.facecolor':  'white',
                     'figure.figsize': [6, 6]}):
    sc.pl.umap(adata_sub, color=['annotation'], size=20,
               color_map = 'RdPu', ncols = 3, wspace=0.3,
               legend_fontsize=10, frameon=False, legend_loc='on data')
```

Example 66 (python):
```python
adata_sub.var_names_make_unique()
sc.pl.dotplot(adata_sub,
              cellnames, 
              groupby='annotation',
              dendrogram=False,
              standard_scale="var",
              color_map="Reds",
              expression_cutoff=0.1
             )
```

Example 67 (python):
```python
# get annotation
annot_df['CCS-Valve_annotation'] = adata_sub.obs['annotation'].reindex(annot_df.index)
annot_df.head()
```

Example 68 (unknown):
```unknown
structures = ['tbd - Atrium - Right','tbd - Atrium - Left']

# get index
indices = list(annot_df.index[annot_df['myocardium_annotation'].isin(structures)])
```

Example 69 (python):
```python
# subset
adata_sub = adata_vis[indices]

# # compute KNN using the cell2location output stored in adata.obsm
# bbknn
bbknn.bbknn(adata_sub, batch_key='sample_for_cell2loc', use_rep='q05_cell_abundance_w_sf_ndarray')

# Cluster spots into regions using scanpy
sc.tl.leiden(adata_sub, resolution=1.5)

# add region as categorical variable
adata_sub.obs["region_cluster"] = adata_sub.obs["leiden"].astype("category")

# compute UMAP using KNN graph based on the cell2location output
sc.tl.umap(adata_sub, min_dist = 0.2, spread = 1)  # n_components=3
```

Example 70 (unknown):
```unknown
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/anndata/compat/_overloaded_dict.py:179: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  ovld.data[key] = neighbors
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/anndata/_core/anndata.py:1830: UserWarning: Variable names are not unique. To make them unique, call `.var_names_make_unique`.
  utils.warn_names_duplicates("var")
```

Example 71 (python):
```python
# show regions in UMAP coordinates
with mpl.rc_context({'axes.facecolor':  'white',
                     'figure.figsize': [3, 3]}):
    sc.pl.umap(adata_sub, color=['week','JC_edit'], size=20,
               color_map = 'RdPu', ncols = 3, wspace=0.3,
               legend_fontsize=14, frameon=False, components='all')
```

Example 72 (python):
```python
# show regions in UMAP coordinates
with mpl.rc_context({'axes.facecolor':  'white',
                     'figure.figsize': [3, 3]}):
    sc.pl.umap(adata_sub, color=['sample'], size=20,
               color_map = 'RdPu', ncols = 3, wspace=0.2,
               legend_fontsize=14, frameon=False, components='all')
```

Example 73 (python):
```python
with mpl.rc_context({'axes.facecolor':  'white',
                     'figure.figsize': [3, 3]}):
    sc.pl.umap(adata_sub, color=['region_cluster'], size=20,
               color_map = 'RdPu', ncols = 3, wspace=0.2,legend_loc='on data',
               legend_fontsize=10, frameon=False, components='all')
```

Example 74 (python):
```python
with mpl.rc_context({'axes.facecolor':  'white',
                     'figure.figsize': [3, 3]}):
    sc.pl.umap(adata_sub, color=['region_cluster'], groups=['5','13'],size=10,
               color_map = 'RdPu', ncols = 3, wspace=0.2,legend_loc='on data',
               legend_fontsize=10, frameon=False, components='all')
```

Example 75 (python):
```python
for structure,celltypes in celltypes_for_annotation.items():
    print(f'##### {structure} #####')
    with mpl.rc_context({'axes.facecolor':  'white',
                         'figure.figsize': [3, 3]}):
        sc.pl.umap(adata_sub, color=[f'{x}_minmax' for x in celltypes], size=20,
                   color_map = 'RdPu', ncols = 4, wspace=0.3, vmax='p99.98',
                   legend_fontsize=14, frameon=False)
```

Example 76 (unknown):
```unknown
##### Atrium #####
```

Example 77 (unknown):
```unknown
##### Ventricle #####
```

Example 78 (unknown):
```unknown
##### CCS #####
```

Example 79 (unknown):
```unknown
##### Adventitia #####
```

Example 80 (unknown):
```unknown
##### LN #####
```

Example 81 (unknown):
```unknown
##### Valve #####
```

Example 82 (unknown):
```unknown
##### Endocardium #####
```

Example 83 (unknown):
```unknown
##### Epicardium #####
```

Example 84 (unknown):
```unknown
##### DuctusArteriosus #####
```

Example 85 (python):
```python
adata_sub.var_names_make_unique()
sc.pl.dotplot(adata_sub,
              cellnames, 
              groupby='region_cluster',
              dendrogram=False,
              standard_scale="var",
              color_map="Reds",
              expression_cutoff=0.1
             )
```

Example 86 (python):
```python
adata_sub.obs['annotation'] = adata_sub.obs['region_cluster']
adata_sub.obs['annotation'] = adata_sub.obs['annotation'].astype('str')
adata_sub.obs.replace({'annotation': {
    '0': 'Atrium - Left', 
    '1': 'Atrium - Right', 
    '2': 'Adventitia - Coronary', 
    '3': 'Atrium - Left', 
    '4': 'Atrium - Right', 
    '5': 'Endocardium - Atrial', 
    '6': 'Atrium - Left', 
    '7': 'Atrium - Left', 
    '8': 'Atrium - Left', 
    '9': 'tbd - Nodes', 
    '10': 'tbd - Nodes', 
    '11': 'Atrium - Left', 
    '12': 'Atrium - Left', 
    '13': 'Endocardium - Atrial', 
    '14': 'Uncertain', 
}}, inplace=True)
```

Example 87 (python):
```python
with mpl.rc_context({'axes.facecolor':  'white',
                     'figure.figsize': [3, 3]}):
    sc.pl.umap(adata_sub, color=['annotation'], size=20,
               color_map = 'RdPu', ncols = 3, wspace=0.3,
               legend_fontsize=10, frameon=False, legend_loc='on data')
```

Example 88 (python):
```python
adata_sub.var_names_make_unique()
sc.pl.dotplot(adata_sub,
              cellnames, 
              groupby='annotation',
              dendrogram=False,
              standard_scale="var",
              color_map="Reds",
              expression_cutoff=0.1
             )
```

Example 89 (python):
```python
# get annotation
annot_df['Atrium_annotation'] = adata_sub.obs['annotation'].reindex(annot_df.index)
annot_df.head()
```

Example 90 (unknown):
```unknown
celltypes_for_annotation = {
    'Atrium':['AtrialCardiomyocytesLeft','AtrialCardiomyocytesRight'],
    'Ventricle':['VentricularCardiomyocytesLeftCompact',
                 'VentricularCardiomyocytesRightCompact',
                 'VentricularCardiomyocytesLeftTrabeculated',
                 'VentricularCardiomyocytesRightTrabeculated'],
    'CCS':['SinoatrialNodeCardiomyocytes',
             'AtrioventricularNodeCardiomyocytes',
             'VentricularConductionSystemProximal',
             'VentricularConductionSystemDistal',],
    'Adventitia':['GreatVesselAdventitialFibroblasts','CoronaryVesselAdventitialFibroblasts'],
    'LN':['LymphNodeFibroblasticReticularCells'],
    'Valve':['ValveInterstitialCells'],
    'Endocardium':['EndocardialCells','EndocardialCushionCells'],
    'Epicardium':['MesothelialEpicardialCells','EpicardiumDerivedCells'],
    'DuctusArteriosus':['DuctusArteriosusSmoothMuscleCells'],
    'Vessels':['GreatVesselSmoothMuscleCells',
 'CoronarySmoothMuscleCells',
 'CoronaryPericytes',
 'GreatVesselArterialEndothelialCells',
 'GreatVesselVenousEndothelialCells',
 'CoronaryArterialEndothelialCells',
 'CoronaryVenousEndothelialCells',
 'CoronaryCapillaryEndothelialCells',]
}
```

Example 91 (unknown):
```unknown
structures = ['Vessels']

# get index
indices = list(annot_df.index[annot_df['global_annotation'].isin(structures)])
```

Example 92 (python):
```python
# subset
adata_sub = adata_vis[indices]

# # compute KNN using the cell2location output stored in adata.obsm
# bbknn
bbknn.bbknn(adata_sub, batch_key='sample_for_cell2loc', use_rep='q05_cell_abundance_w_sf_ndarray')

# Cluster spots into regions using scanpy
sc.tl.leiden(adata_sub, resolution=1.5)

# add region as categorical variable
adata_sub.obs["region_cluster"] = adata_sub.obs["leiden"].astype("category")

# compute UMAP using KNN graph based on the cell2location output
sc.tl.umap(adata_sub, min_dist = 0.2, spread = 1)  # n_components=3
```

Example 93 (unknown):
```unknown
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/anndata/compat/_overloaded_dict.py:179: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  ovld.data[key] = neighbors
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/anndata/_core/anndata.py:1830: UserWarning: Variable names are not unique. To make them unique, call `.var_names_make_unique`.
  utils.warn_names_duplicates("var")
```

Example 94 (python):
```python
# show regions in UMAP coordinates
with mpl.rc_context({'axes.facecolor':  'white',
                     'figure.figsize': [3, 3]}):
    sc.pl.umap(adata_sub, color=['week','JC_edit'], size=10,
               color_map = 'RdPu', ncols = 3, wspace=0.3,
               legend_fontsize=14, frameon=False, components='all')
```

Example 95 (python):
```python
# show regions in UMAP coordinates
with mpl.rc_context({'axes.facecolor':  'white',
                     'figure.figsize': [3, 3]}):
    sc.pl.umap(adata_sub, color=['sample'], size=20,
               color_map = 'RdPu', ncols = 3, wspace=0.2,
               legend_fontsize=14, frameon=False, components='all')
```

Example 96 (python):
```python
with mpl.rc_context({'axes.facecolor':  'white',
                     'figure.figsize': [3, 3]}):
    sc.pl.umap(adata_sub, color=['region_cluster'], size=10,
               color_map = 'RdPu', ncols = 3, wspace=0.2,legend_loc='on data',
               legend_fontsize=10, frameon=False, components='all')
```

Example 97 (python):
```python
with mpl.rc_context({'axes.facecolor':  'white',
                     'figure.figsize': [3, 3]}):
    sc.pl.umap(adata_sub, color=['region_cluster'], groups=['13'],size=10,
               color_map = 'RdPu', ncols = 3, wspace=0.2,legend_loc='on data',
               legend_fontsize=10, frameon=False, components='all')
```

Example 98 (python):
```python
for structure,celltypes in celltypes_for_annotation.items():
    print(f'##### {structure} #####')
    with mpl.rc_context({'axes.facecolor':  'white',
                         'figure.figsize': [3, 3]}):
        sc.pl.umap(adata_sub, color=[f'{x}_minmax' for x in celltypes], size=20,
                   color_map = 'RdPu', ncols = 4, wspace=0.3, vmax='p99.98',
                   legend_fontsize=14, frameon=False)
```

Example 99 (unknown):
```unknown
##### Atrium #####
```

Example 100 (unknown):
```unknown
##### Ventricle #####
```

Example 101 (unknown):
```unknown
##### CCS #####
```

Example 102 (unknown):
```unknown
##### Adventitia #####
```

Example 103 (unknown):
```unknown
##### LN #####
```

Example 104 (unknown):
```unknown
##### Valve #####
```

Example 105 (unknown):
```unknown
##### Endocardium #####
```

Example 106 (unknown):
```unknown
##### Epicardium #####
```

Example 107 (unknown):
```unknown
##### DuctusArteriosus #####
```

Example 108 (unknown):
```unknown
##### Vessels #####
```

Example 109 (python):
```python
adata_sub.var_names_make_unique()
sc.pl.dotplot(adata_sub,
              cellnames, 
              groupby='region_cluster',
              dendrogram=False,
              standard_scale="var",
              color_map="Reds",
              expression_cutoff=0.1
             )
```

Example 110 (python):
```python
adata_sub.obs['annotation'] = adata_sub.obs['region_cluster']
adata_sub.obs['annotation'] = adata_sub.obs['annotation'].astype('str')
adata_sub.obs.replace({'annotation': {
    '0': 'Adventitia - Coronary', 
    '1': 'Great vessel', 
    '2': 'Adventitia - GV', 
    '3': 'tbd - AVring, Valves', 
    '4': 'Great vessel', 
    '5': 'Adventitia - GV', 
    '6': 'Great vessel', 
    '7': 'Great vessel', 
    '8': 'Great vessel', 
    '9': 'Great vessel', 
    '10': 'Adventitia - Coronary', 
    '11': 'Great vessel', 
    '12': 'Adventitia - GV', 
    '13': 'Great vessel', 
    '14': 'Great vessel', 
    '15': 'Atrium - Left', 
    '16': 'Great vessel', 
    '17': 'Coronary vessel', 
    '18': 'Great vessel', 
    '19': 'Coronary vessel', 
    '20': 'tbd - Ventricle', 
}}, inplace=True)
```

Example 111 (python):
```python
with mpl.rc_context({'axes.facecolor':  'white',
                     'figure.figsize': [3, 3]}):
    sc.pl.umap(adata_sub, color=['annotation'], size=20,
               color_map = 'RdPu', ncols = 3, wspace=0.3,
               legend_fontsize=10, frameon=False, legend_loc='on data')
```

Example 112 (python):
```python
adata_sub.var_names_make_unique()
sc.pl.dotplot(adata_sub,
              cellnames, 
              groupby='annotation',
              dendrogram=False,
              standard_scale="var",
              color_map="Reds",
              expression_cutoff=0.1
             )
```

Example 113 (python):
```python
# get annotation
annot_df['Vessels_annotation'] = adata_sub.obs['annotation'].reindex(annot_df.index)
annot_df.head()
```

Example 114 (unknown):
```unknown
structures = ['tbd - Nodes']

# get index
indices = list(annot_df.index[(annot_df['CCS-Valve_annotation'].isin(structures))|(annot_df['Atrium_annotation'].isin(structures))])
indices = list(set(indices))
```

Example 115 (python):
```python
# subset
adata_sub = adata_vis[indices]

# # compute KNN using the cell2location output stored in adata.obsm
# bbknn
bbknn.bbknn(adata_sub, batch_key='donor', use_rep='q05_cell_abundance_w_sf_ndarray')

# Cluster spots into regions using scanpy
sc.tl.leiden(adata_sub, resolution=1)

# add region as categorical variable
adata_sub.obs["region_cluster"] = adata_sub.obs["leiden"].astype("category")

# compute UMAP using KNN graph based on the cell2location output
sc.tl.umap(adata_sub, min_dist = 0.2, spread = 1, n_components=3)  # n_components=3
```

Example 116 (unknown):
```unknown
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/anndata/compat/_overloaded_dict.py:179: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  ovld.data[key] = neighbors
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/anndata/_core/anndata.py:1830: UserWarning: Variable names are not unique. To make them unique, call `.var_names_make_unique`.
  utils.warn_names_duplicates("var")
```

Example 117 (python):
```python
# show regions in UMAP coordinates
with mpl.rc_context({'axes.facecolor':  'white',
                     'figure.figsize': [3, 3]}):
    sc.pl.umap(adata_sub, color=['week','JC_edit'], size=50,
               color_map = 'RdPu', ncols = 3, wspace=0.7,
               legend_fontsize=14, frameon=False, components='all')
```

Example 118 (python):
```python
# show regions in UMAP coordinates
with mpl.rc_context({'axes.facecolor':  'white',
                     'figure.figsize': [3, 3]}):
    sc.pl.umap(adata_sub, color=['sample'], size=50,
               color_map = 'RdPu', ncols = 3, wspace=0.2,
               legend_fontsize=14, frameon=False, components='1,2')
```

Example 119 (python):
```python
with mpl.rc_context({'axes.facecolor':  'white',
                     'figure.figsize': [3, 3]}):
    sc.pl.umap(adata_sub, color=['region_cluster'], size=50,
               color_map = 'RdPu', ncols = 3, wspace=0.2,legend_loc='on data',
               legend_fontsize=10, frameon=False, components='all')
```

Example 120 (python):
```python
with mpl.rc_context({'axes.facecolor':  'white',
                     'figure.figsize': [3, 3]}):
    sc.pl.umap(adata_sub, color=['region_cluster'], groups=['6'],size=10,
               color_map = 'RdPu', ncols = 3, wspace=0.2,legend_loc='on data',
               legend_fontsize=10, frameon=False, components='all')
```

Example 121 (python):
```python
for structure,celltypes in celltypes_for_annotation.items():
    print(f'##### {structure} #####')
    with mpl.rc_context({'axes.facecolor':  'white',
                         'figure.figsize': [3, 3]}):
        sc.pl.umap(adata_sub, color=[f'{x}_minmax' for x in celltypes], size=50,
                   color_map = 'RdPu', ncols = 3, wspace=0.3, vmax='p99.98',
                   legend_fontsize=14, frameon=False, components='all')
```

Example 122 (unknown):
```unknown
##### Atrium #####
```

Example 123 (unknown):
```unknown
##### Ventricle #####
```

Example 124 (unknown):
```unknown
##### CCS #####
```

Example 125 (unknown):
```unknown
##### Adventitia #####
```

Example 126 (unknown):
```unknown
##### LN #####
```

Example 127 (unknown):
```unknown
##### Valve #####
```

Example 128 (unknown):
```unknown
##### Endocardium #####
```

Example 129 (unknown):
```unknown
##### Epicardium #####
```

Example 130 (unknown):
```unknown
##### DuctusArteriosus #####
```

Example 131 (unknown):
```unknown
##### Vessels #####
```

Example 132 (python):
```python
adata_sub.var_names_make_unique()
sc.pl.dotplot(adata_sub,
              cellnames, 
              groupby='region_cluster',
              dendrogram=False,
              standard_scale="var",
              color_map="Reds",
              expression_cutoff=0.1
             )
```

Example 133 (python):
```python
adata_sub.obs['annotation'] = adata_sub.obs['region_cluster']
adata_sub.obs['annotation'] = adata_sub.obs['annotation'].astype('str')
adata_sub.obs.replace({'annotation': {
    '0': 'Atrium - Left', 
    '1': 'Atrium - Left', 
    '2': 'tbd - AVring, Valves', 
    '3': 'tbd - AVring, Valves', 
    '4': 'VCS - Proximal', 
    '5': 'AVnode', 
    '6': 'AVnode', 
    '7': 'SAnode', 
    '8': 'Atrium - Left', 
}}, inplace=True)
```

Example 134 (python):
```python
with mpl.rc_context({'axes.facecolor':  'white',
                     'figure.figsize': [3, 3]}):
    sc.pl.umap(adata_sub, color=['annotation'], size=50,
               color_map = 'RdPu', ncols = 3, wspace=0.3,
               legend_fontsize=10, frameon=False, legend_loc='on data', components='all')
```

Example 135 (python):
```python
adata_sub.var_names_make_unique()
sc.pl.dotplot(adata_sub,
              cellnames, 
              groupby='annotation',
              dendrogram=False,
              standard_scale="var",
              color_map="Reds",
              expression_cutoff=0.1
             )
```

Example 136 (python):
```python
# get annotation
annot_df['Nodes_annotation'] = adata_sub.obs['annotation'].reindex(annot_df.index)
annot_df.head()
```

Example 137 (python):
```python
# prepare column
annot_df['cell2loc-abundance_annotation'] = np.nan
```

Example 138 (python):
```python
# annotate "Ductus arteriosus" based on DuctusArteriosusSMC abuncance
thresh = 5

adata_vis.obs['DuctusArteriosusSmoothMuscleCells'].hist(bins=100)
plt.ylim(0,200)
plt.axvline(x=thresh, color='r', linestyle='--')
plt.show()

obsnames = adata_vis.obs_names[adata_vis.obs['DuctusArteriosusSmoothMuscleCells']>thresh]
annot_df.loc[obsnames,'cell2loc-abundance_annotation'] = 'Ductus arteriosus'

len(obsnames)
```

Example 139 (python):
```python
annotation_final = []
for ind in annot_df.index:
    l = list(annot_df.loc[ind,['myocardium_annotation', 'CCS-Valve_annotation',
                                   'Atrium_annotation', 'Vessels_annotation', 'Nodes_annotation',
                                   'cell2loc-abundance_annotation']])
    l = [x for x in l if x==x]
    if len(l)>0:
        ll = [x for x in l if 'tbd -' not in x]
        if len(ll)==1:
            annotation_final.append(ll[0])
        # elif l[-1]=='Epicardium':
        #     annotation_final.append(l[-1])
        elif l[-1]=='Ductus arteriosus':
            annotation_final.append(l[-1])
        else:
            annotation_final.append(' | '.join(l))
    else:
        annotation_final.append(np.nan)
        
annot_df['pre-final'] = annotation_final.copy()
annot_df.head()
```

Example 140 (unknown):
```unknown
set(annot_df['pre-final'])
```

Example 141 (unknown):
```unknown
{'AVnode',
 'Adventitia - Coronary',
 'Adventitia - GV',
 'Atrium - Left',
 'Atrium - Right',
 'Coronary vessel',
 'Ductus arteriosus',
 'Endocardial cusion',
 'Endocardium - Atrial',
 'Endocardium - Ventricular',
 'Great vessel',
 'SAnode',
 'Uncertain',
 'VCS - Distal',
 'VCS - Proximal',
 'Ventricle - Compact - Left',
 'Ventricle - Compact - Right',
 'Ventricle - Trabeculated - Left',
 'Ventricle - Trabeculated - Right',
 'tbd - AVring, Valves',
 'tbd - CCS, Valves | tbd - AVring, Valves',
 'tbd - CCS, Valves | tbd - Nodes | tbd - AVring, Valves',
 'tbd - Ventricle'}
```

Example 142 (unknown):
```unknown
# rename some of them
annot_df.replace({'pre-final':{
    'tbd - CCS, Valves | tbd - AVring, Valves':'tbd - AVring, Valves',
    'tbd - CCS, Valves | tbd - Nodes | tbd - AVring, Valves':'tbd - AVring, Valves',
    'tbd - Atrium - Left | tbd - Nodes | tbd - AVring, Valves':'tbd - AVring, Valves'
}},inplace=True)
set(annot_df['pre-final'])
```

Example 143 (unknown):
```unknown
{'AVnode',
 'Adventitia - Coronary',
 'Adventitia - GV',
 'Atrium - Left',
 'Atrium - Right',
 'Coronary vessel',
 'Ductus arteriosus',
 'Endocardial cusion',
 'Endocardium - Atrial',
 'Endocardium - Ventricular',
 'Great vessel',
 'SAnode',
 'Uncertain',
 'VCS - Distal',
 'VCS - Proximal',
 'Ventricle - Compact - Left',
 'Ventricle - Compact - Right',
 'Ventricle - Trabeculated - Left',
 'Ventricle - Trabeculated - Right',
 'tbd - AVring, Valves',
 'tbd - Ventricle'}
```

Example 144 (python):
```python
# add manual annotation
annot_df['JC_edit'] = adata_vis.obs['JC_edit'].reindex(annot_df.index)
annot_df.head()
```

Example 145 (python):
```python
# cross table
pd.crosstab(annot_df['pre-final'],annot_df['JC_edit'])
```

Example 146 (unknown):
```unknown
# prepare column
annot_df['final'] = annot_df['pre-final'].copy()
```

Example 147 (unknown):
```unknown
# AVN
obsnames = annot_df.index[annot_df['JC_edit']=='AVN']
annot_df.loc[obsnames,'final'] = 'AVnode'
# LN
obsnames = annot_df.index[annot_df['JC_edit']=='LN']
annot_df.loc[obsnames,'final'] = 'LN'
# Epicardium
obsnames = annot_df.index[annot_df['JC_edit']=='Epicardium']
annot_df.loc[obsnames,'final'] = 'Epicardium'
```

Example 148 (python):
```python
annots = ['AV ring','AV valves','Aortic valve','Valve apparatus','Epicardium','Coronary artery','Great vessels','Connective tissue']
for anno in annots:
    obsnames = annot_df.index[(annot_df['JC_edit']==anno)&\
                              ([('tbd -' in x)|(x=='Uncertain') for x in annot_df['pre-final']])]
                              
    print(f'{anno}, n_spot to modify: {len(obsnames)}')
    if anno=='Connective tissue':
        annot_df.loc[obsnames,'final'] = 'AV ring'
    else:
        annot_df.loc[obsnames,'final'] = anno
```

Example 149 (unknown):
```unknown
AV ring, n_spot to modify: 343
AV valves, n_spot to modify: 421
Aortic valve, n_spot to modify: 229
Valve apparatus, n_spot to modify: 205
Epicardium, n_spot to modify: 128
Coronary artery, n_spot to modify: 4
Great vessels, n_spot to modify: 518
Connective tissue, n_spot to modify: 7
```

Example 150 (python):
```python
obsnames = annot_df.index[(annot_df['pre-final']=='Coronary vessel')&\
                          (annot_df['JC_edit']=='Great vessels')]
                              
print(f'n_spot to modify: {len(obsnames)}')
annot_df.loc[obsnames,'final'] = 'Great vessel'
```

Example 151 (unknown):
```unknown
n_spot to modify: 105
```

Example 152 (python):
```python
obsnames = annot_df.index[(annot_df['JC_edit']=='Myocardium - Atrial')&\
                              ([('tbd -' in x)|(x=='Uncertain') for x in annot_df['pre-final']])]
print(len(obsnames))
ad = adata_vis[obsnames].copy()
sns.scatterplot(ad.obs['AtrialCardiomyocytesRight'],ad.obs['AtrialCardiomyocytesLeft'],
                hue=annot_df.loc[obsnames,'final'])
plt.legend(loc='upper right',bbox_to_anchor=(2.5, 1))
plt.show()

# decide right or left based on cell2loc abundance
df = ad.obs.loc[obsnames,['AtrialCardiomyocytesRight','AtrialCardiomyocytesLeft']]
df['annotation'] = 'Atrium - Left'
obsnames_right = df.index[df['AtrialCardiomyocytesRight']>df['AtrialCardiomyocytesLeft']]
df.loc[obsnames_right,'annotation'] = 'Atrium - Right'

# add annotation
annot_df.loc[obsnames,'final'] = df['annotation'].copy()

sns.scatterplot(ad.obs['AtrialCardiomyocytesRight'],ad.obs['AtrialCardiomyocytesLeft'],
                hue=annot_df.loc[obsnames,'final'])
plt.legend(loc='upper right',bbox_to_anchor=(1.2, 1))
plt.show()
```

Example 153 (python):
```python
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/anndata/_core/anndata.py:1830: UserWarning: Variable names are not unique. To make them unique, call `.var_names_make_unique`.
  utils.warn_names_duplicates("var")
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/seaborn/_decorators.py:43: FutureWarning: Pass the following variables as keyword args: x, y. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.
  FutureWarning
```

Example 154 (python):
```python
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/seaborn/_decorators.py:43: FutureWarning: Pass the following variables as keyword args: x, y. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.
  FutureWarning
```

Example 155 (python):
```python
obsnames = annot_df.index[(annot_df['JC_edit']=='Myocardium - Ventricular - Compact')&\
                              ([('tbd -' in x)|(x=='Uncertain') for x in annot_df['pre-final']])]
print(len(obsnames))
ad = adata_vis[obsnames].copy()
sns.scatterplot(ad.obs['VentricularCardiomyocytesRightCompact'],ad.obs['VentricularCardiomyocytesLeftCompact'],
                hue=annot_df.loc[obsnames,'pre-final'])
plt.legend(loc='upper right',bbox_to_anchor=(2, 1))
plt.show()

# decide right or left based on cell2loc abundance
df = ad.obs.loc[obsnames,['VentricularCardiomyocytesRightCompact','VentricularCardiomyocytesLeftCompact']]
df['annotation'] = 'Ventricle - Compact - Left'
obsnames_right = df.index[df['VentricularCardiomyocytesRightCompact']>df['VentricularCardiomyocytesLeftCompact']]
df.loc[obsnames_right,'annotation'] = 'Ventricle - Compact - Right'

# add annotation
annot_df.loc[obsnames,'final'] = df['annotation'].copy()

sns.scatterplot(ad.obs['VentricularCardiomyocytesRightCompact'],ad.obs['VentricularCardiomyocytesLeftCompact'],
                hue=annot_df.loc[obsnames,'final'])
plt.legend(loc='upper right',bbox_to_anchor=(2, 1))
plt.show()
```

Example 156 (python):
```python
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/anndata/_core/anndata.py:1830: UserWarning: Variable names are not unique. To make them unique, call `.var_names_make_unique`.
  utils.warn_names_duplicates("var")
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/seaborn/_decorators.py:43: FutureWarning: Pass the following variables as keyword args: x, y. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.
  FutureWarning
```

Example 157 (python):
```python
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/seaborn/_decorators.py:43: FutureWarning: Pass the following variables as keyword args: x, y. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.
  FutureWarning
```

Example 158 (python):
```python
obsnames = annot_df.index[(annot_df['JC_edit'].isin(['Myocardium - Papillary Muscle',
                                                     'Myocardium - Ventricular - Trabeculated',
                                                     'Bundle branch - left','Bundle branch - right'
                                                    ]))&\
                              ([('tbd -' in x)|(x=='Uncertain') for x in annot_df['pre-final']])]
print(len(obsnames))
ad = adata_vis[obsnames].copy()
sns.scatterplot(ad.obs['VentricularCardiomyocytesRightTrabeculated'],ad.obs['VentricularCardiomyocytesLeftTrabeculated'],
                hue=annot_df.loc[obsnames,'pre-final'])
plt.legend(loc='upper right',bbox_to_anchor=(2, 1))
plt.show()

# decide right or left based on cell2loc abundance
df = ad.obs.loc[obsnames,['VentricularCardiomyocytesRightTrabeculated','VentricularCardiomyocytesLeftTrabeculated']]
df['annotation'] = 'Ventricle - Trabeculated - Left'
obsnames_right = df.index[df['VentricularCardiomyocytesRightTrabeculated']>df['VentricularCardiomyocytesLeftTrabeculated']]
df.loc[obsnames_right,'annotation'] = 'Ventricle - Trabeculated - Right'

# add annotation
annot_df.loc[obsnames,'final'] = df['annotation'].copy()

sns.scatterplot(ad.obs['VentricularCardiomyocytesRightTrabeculated'],ad.obs['VentricularCardiomyocytesLeftTrabeculated'],
                hue=annot_df.loc[obsnames,'final'])
plt.legend(loc='upper right',bbox_to_anchor=(2, 1))
plt.show()
```

Example 159 (python):
```python
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/anndata/_core/anndata.py:1830: UserWarning: Variable names are not unique. To make them unique, call `.var_names_make_unique`.
  utils.warn_names_duplicates("var")
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/seaborn/_decorators.py:43: FutureWarning: Pass the following variables as keyword args: x, y. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.
  FutureWarning
```

Example 160 (python):
```python
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/seaborn/_decorators.py:43: FutureWarning: Pass the following variables as keyword args: x, y. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.
  FutureWarning
```

Example 161 (unknown):
```unknown
# refine name
annot_df.replace({'final':{
    'Coronary artery':'Coronary vessel',
    'Endocardium - Atrial':'Endocardium',
    'Endocardium - Ventricular':'Endocardium',
    'Epicardium - Atrial':'Epicardium',
    'Epicardium - Ventricular':'Epicardium',
    'Great vessels':'Great vessel',
}},inplace=True)
set(annot_df['final'])
```

Example 162 (unknown):
```unknown
{'AV ring',
 'AV valves',
 'AVnode',
 'Adventitia - Coronary',
 'Adventitia - GV',
 'Aortic valve',
 'Atrium - Left',
 'Atrium - Right',
 'Coronary vessel',
 'Ductus arteriosus',
 'Endocardial cusion',
 'Endocardium',
 'Epicardium',
 'Great vessel',
 'LN',
 'SAnode',
 'VCS - Distal',
 'VCS - Proximal',
 'Valve apparatus',
 'Ventricle - Compact - Left',
 'Ventricle - Compact - Right',
 'Ventricle - Trabeculated - Left',
 'Ventricle - Trabeculated - Right'}
```

Example 163 (python):
```python
# cross table
pd.crosstab(annot_df['final'],annot_df['JC_edit'])
```

Example 164 (python):
```python
adata_vis.obs['annotation_final'] = annot_df['final'].reindex(adata_vis.obs_names)
adata_vis.obs['annotation_final'] = adata_vis.obs['annotation_final'].astype('category')
```

Example 165 (python):
```python
with mpl.rc_context({'axes.facecolor':  'white',
                     'figure.figsize': [6, 6]}):
    sc.pl.umap(adata_vis, color=['annotation_final','JC_edit'], size=5,
               color_map = 'RdPu', ncols = 3, wspace=0.1,
               legend_fontsize=10, frameon=False, legend_loc='on data')
```

Example 166 (python):
```python
annot_df.to_csv('/nfs/team205/heart/anndata_objects/Foetal/visium_foetal_healthy-all_spot-annotation.csv')
adata_vis.write('/nfs/team205/heart/anndata_objects/Foetal/visium_foetal_healthy-all_raw.h5ad')
```

Example 167 (unknown):
```unknown
# rclone to gdrive
rclone copy /nfs/team205/heart/anndata_objects/Foetal/visium_foetal_healthy-all_raw.h5ad \
gdrive:JamesC_Kazumasa/Foetal_Heart/SanjayShinha/toSinhalab/anndata/Visium/
```

---

## Read in¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/Xenium/no1_per-sample-processing/Hst45-HEA-0-FFPE-1/no1_QC_filetering.html

**Contents:**
- Read in¶
- Calculate quality control metrics¶
- Filtering out low-count cells¶
- Save¶

**Examples:**

Example 1 (python):
```python
import scanpy as sc
import numpy as np
import pandas as pd
import os

import matplotlib.pyplot as plt
import seaborn as sns
```

Example 2 (unknown):
```unknown
!ls /home/kk837/rfs/rfs-teichlab-nfs-iCNyzSAaucw/jc2226/heart_team_data/xenium/20241113__133901__JK_HEA5K_Hs_heart_vessels
```

Example 3 (unknown):
```unknown
output-XETG00155__0038587__C194-HEA-0-FFPE-1__20241113__134112
output-XETG00155__0038587__Hst45-HEA-0-FFPE-1__20241113__134112
output-XETG00155__0038587__Hst60-HEA-0-FFPE-1__20241113__134112
output-XETG00155__0038592__AH13-HEA-36-FFPE-1__20241113__134112
output-XETG00155__0038592__AH14-HEA-29-FFPE-2__20241113__134112
output-XETG00155__0038592__AH14-HEA-36-FFPE-1__20241113__134112
```

Example 4 (python):
```python
sample_id = 'Hst45-HEA-0-FFPE-1'
out_dir = f'/home/kk837/rfs/rfs-teichlab-nfs-iCNyzSAaucw/jc2226/heart_team_data/xenium/20241113__133901__JK_HEA5K_Hs_heart_vessels/output-XETG00155__0038587__{sample_id}__20241113__134112'

adata = sc.read_10x_h5(
    filename=os.path.join(out_dir,"cell_feature_matrix.h5")
)
print(adata.X.data[:5])
adata
```

Example 5 (unknown):
```unknown
[1. 1. 1. 1. 4.]
```

Example 6 (unknown):
```unknown
AnnData object with n_obs × n_vars = 219198 × 5001
    var: 'gene_ids', 'feature_types', 'genome'
```

Example 7 (python):
```python
# add obs
df = pd.read_csv(
    os.path.join(out_dir,"cells.csv.gz")
)
df.set_index(adata.obs_names, inplace=True)
adata.obs = df.copy()
adata.obs.head()
```

Example 8 (python):
```python
adata.obsm["spatial"] = adata.obs[["x_centroid", "y_centroid"]].copy().to_numpy()
```

Example 9 (unknown):
```unknown
AnnData object with n_obs × n_vars = 219198 × 5001
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method'
    var: 'gene_ids', 'feature_types', 'genome'
    obsm: 'spatial'
```

Example 10 (python):
```python
sc.pl.spatial(adata,color=['total_counts'],cmap='RdPu',spot_size=20,vmax='p95')
```

Example 11 (python):
```python
/tmp/ipykernel_184408/27570213.py:1: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(adata,color=['total_counts'],cmap='RdPu',spot_size=20,vmax='p95')
```

Example 12 (python):
```python
adata.obs['segmentation_method'].value_counts()
```

Example 13 (unknown):
```unknown
segmentation_method
Segmented by interior stain (18S)                       213482
Segmented by boundary stain (ATP1A1+CD45+E-Cadherin)      2861
Segmented by nucleus expansion of 5.0µm                   2855
Name: count, dtype: int64
```

Example 14 (python):
```python
plt.hist(adata.obs.total_counts, bins=1000)
# plt.xlim(0,500)
plt.xscale('log')
plt.show()
```

Example 15 (python):
```python
sc.pp.calculate_qc_metrics(adata, percent_top=(10, 20, 50, 150), inplace=True)
```

Example 16 (python):
```python
# The percentage of control probes and control codewords can be calculated from adata.obs
cprobes = (
    adata.obs["control_probe_counts"].sum() / adata.obs["total_counts"].sum() * 100
)
cwords = (
    adata.obs["control_codeword_counts"].sum() / adata.obs["total_counts"].sum() * 100
)
print(f"Negative DNA probe count % : {cprobes}")
print(f"Negative decoding count % : {cwords}")
```

Example 17 (unknown):
```unknown
Negative DNA probe count % : 0.0009415129050999326
Negative decoding count % : 0.004741297553449501
```

Example 18 (python):
```python
# Next we plot the distribution of total transcripts per cell, unique transcripts per cell, area of segmented cells and the ratio of nuclei area to their cells
fig, axs = plt.subplots(1, 4, figsize=(20, 4))

axs[0].set_title("Total transcripts per cell")
sns.histplot(
    adata.obs["total_counts"],
    kde=False,
    ax=axs[0],
)

axs[1].set_title("Unique transcripts per cell")
sns.histplot(
    adata.obs["n_genes_by_counts"],
    kde=False,
    ax=axs[1],
)


axs[2].set_title("Area of segmented cells")
sns.histplot(
    adata.obs["cell_area"],
    kde=False,
    ax=axs[2],
)

axs[3].set_title("Nucleus ratio")
sns.histplot(
    adata.obs["nucleus_area"] / adata.obs["cell_area"],
    kde=False,
    ax=axs[3],
)
```

Example 19 (unknown):
```unknown
<Axes: title={'center': 'Nucleus ratio'}, ylabel='Count'>
```

Example 20 (python):
```python
plt.hist(adata.obs.n_genes_by_counts, bins=1000)
# plt.xlim(0,500)
plt.xscale('log')
plt.show()
```

Example 21 (python):
```python
sc.pp.filter_cells(adata,min_counts=50)
sc.pp.filter_cells(adata,min_genes=10)
```

Example 22 (python):
```python
adata.write(f'/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium/{sample_id}_5K_filtered_raw.h5ad')
```

Example 23 (unknown):
```unknown
'Hst45-HEA-0-FFPE-1'
```

Example 24 (unknown):
```unknown
AnnData object with n_obs × n_vars = 217361 × 5001
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts'
    obsm: 'spatial'
```

---

## 

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/RNA/scnRNA_Integration-Annotation/Endothelial/CoronaryCellCyclingscFates.html

**Examples:**

Example 1 (unknown):
```unknown
#Added the keys for age and sex
```

Example 2 (python):
```python
# Import packages, initial settings

import numpy as np
import matplotlib.pyplot as plt
import scanpy as sc
import pandas as pd
import anndata
import scrublet as scr
import seaborn as sns
import six
import sys
sys.modules['sklearn.externals.six'] = six
import scvi

plt.rcParams['pdf.fonttype'] = 42
plt.rcParams['ps.fonttype'] = 42

sc.settings.verbosity = 3  # verbosity: errors (0), warnings (1), info (2), hints (3)

from matplotlib.colors import LinearSegmentedColormap
cmap = LinearSegmentedColormap.from_list(name='gene_cmap', colors=['lightgrey', 'thistle', 'red', 'darkred']) 

sc.settings.set_figure_params(dpi=80, color_map='viridis', vector_friendly=False,  dpi_save=300, facecolor='white')
sc.logging.print_version_and_date()
#sc.logging.print_versions()
```

Example 3 (unknown):
```unknown
Global seed set to 0
```

Example 4 (unknown):
```unknown
Running Scanpy 1.8.1, on 2023-09-19 14:34.
```

Example 5 (python):
```python
adata=sc.read('./Objects/Feb28ObjectRaw.h5ad')
print(adata.shape)
```

Example 6 (unknown):
```unknown
(297473, 36601)
```

Example 7 (python):
```python
FineGrainPalette={
    'AtrialCardiomyocytesLeft':'#eac075',
    'AtrialCardiomyocytesRight':'#4c6b8a',
    'AtrialCardiomyocytesCycling':'#e94bc7',
    'VentricularCardiomyocytesLeftCompact':'#ab1c3c',
    'VentricularCardiomyocytesRightCompact':'#fdba45',
    'VentricularCardiomyocytesLeftTrabeculated':'#2b9200',
    'VentricularCardiomyocytesRightTrabeculated':'#0273a8',
    'VentricularCardiomyocytesCycling':'#e1acff',
    'SinoatrialNodeCardiomyocytes':'#854834',
    'AtrioventricularNodeCardiomyocytes':'#688500',
    'VentricularConductionSystemProximal':'#01ccc8',
    'VentricularConductionSystemDistal':'#b58fab',
    'GreatVesselAdventitialFibroblasts':'#ff7e97',
    'CoronaryVesselAdventitialFibroblasts':'#ff9b6e',
    'MyocardialInterstitialFibroblasts1':'#8a6800',
    'MyocardialInterstitialFibroblasts2':'#206601',
    'Myofibroblasts':'#1bdebf',
    'LymphNodeFibroblasticReticularCells':'#005aa0',
    'ValveInterstitialCells':'#e478ff',
    'GreatVesselSmoothMuscleCells':'#ff665f',
    'CoronarySmoothMuscleCells':'#d2cb18',
    'DuctusArteriosusSmoothMuscleCells':'#83d7b3',
    'CoronaryPericytes':'#9088ff',
    'PericardialCells1':'#c95e00',
    'PericardialCells2':'#01e077',
    'PericardialCells3':'#ff88e4',
    'GreatVesselArterialEndothelialCells':'#d4004b',
    'GreatVesselVenousEndothelialCells':'#8f4300',
    'CoronaryArterialEndothelialCells':'#89da5a',
    'CoronaryVenousEndothelialCells':'#006f86',
    'CoronaryCapillaryEndothelialCells':'#aa7dff',
    'EndocardialCells':'#ff8241',
    'EndocardialCushionCells':'#46603a',
    'ValveEndothelialCells':'#414eb6',
    'LymphaticEndothelialCells':'#8b3f60',
    'MesothelialEpicardialCells':'#0295fe',
    'EpicardiumDerivedCells':'#ff92c4',
    'NeuronPrecursors':'#d30038',
    'ChromaffinCells':'#726800',
    'SympatheticNeurons':'#01894b',
    'ParaympatheticNeurons':'#545191',
    'SchwannCellPrecursors':'#b0d276',
    'SchwannCells':'#b30079',
    'MonocytesMPOpos':'#c50066',
    'Monocytes':'#fab4b6',
    'MonocyteDerivedCells':'#d14008',
    'MacrophagesCX3CR1pos':'#eabe97',
    'MacrophagesTIMD4pos':'#b2a200',
    'MacrophagesLYVE1pos':'#4a6005',
    'MacrophagesATF3pos':'#3db833',
    'DendriticCellsType1':'#007859',
    'DendriticCellsMature':'#01a9db',
    'PlasmacytoidDendriticCells':'#2c54ab',
    'MastCells':'#d0bef2',
    'Megakaryocytes':'#94298f',
    'TCellsCD4pos':'#ff387c',
    'TCellsCD8pos':'#9e3429',
    'TregsCD4pos':'#b37700',
    'ProBCells':'#8dab00',
    'BCells':'#a8d1a1',
    'BCellsMS4A1pos':'#00d2f0',
    'NaturalKillerCells':'#abb4ff',
    'InnateLymphoidCells':'#72497f',
}
sc.pl.umap(adata, color='fine_grain',palette=FineGrainPalette)

MidGrainPalette={
    'AtrialCardiomyocytes':'#af3a4d',
    'VentricularCardiomyocytes':'#b1756c',
    'CardiacConductionSystem':'#d45633',
    'Fibroblasts':'#9b6e4e',
    'MuralCells':'#ffb92f',
    'PericardialCells':'#b4f03b',
    'BloodVesselEndothelialCells':'#72cc57',
    'EndocardialCells':'#436e43',
    'LymphaticEndothelialCells':'#85cca0',
    'EpicardialCells':'#5eabb0',
    'Neurons':'#7bbdff',
    'Glia':'#403c90',
    'MyeloidCells':'#420066',
    'LymphoidCells':'#f553f6',
}
sc.pl.umap(adata, color='mid_grain',palette=MidGrainPalette)

CoarseGrainPalette={
    'Cardiomyocytes':'#bc5950',
    'Mesenchymal':'#af9c4c',
    'Endothelium':'#599e58',
    'Epicardium':'#5eabb0',
    'Neural':'#5e75b1',
    'Leukocytes':'#9a52a0',
}
sc.pl.umap(adata, color='coarse_grain',palette=CoarseGrainPalette)
```

Example 8 (unknown):
```unknown
/home/sinhalab/miniconda3/envs/scFates/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 9 (unknown):
```unknown
/home/sinhalab/miniconda3/envs/scFates/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 10 (unknown):
```unknown
/home/sinhalab/miniconda3/envs/scFates/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 11 (python):
```python
set(adata.obs.fine_grain)
```

Example 12 (unknown):
```unknown
{'AtrialCardiomyocytesCycling',
 'AtrialCardiomyocytesLeft',
 'AtrialCardiomyocytesRight',
 'AtrioventricularNodePacemakerCells',
 'BCells',
 'BCellsMS4A1pos',
 'ChromaffinCells',
 'CoronaryArterialEndothelialCells',
 'CoronaryCapillaryEndothelialCells',
 'CoronaryPericytes',
 'CoronarySmoothMuscleCells',
 'CoronaryVenousEndothelialCells',
 'CoronaryVesselAdventitialFibroblasts',
 'DendriticCellsMature',
 'DendriticCellsType1',
 'DuctusArteriosusSmoothMuscleCells',
 'EndocardialCells',
 'EndocardialCushionCells',
 'EpicardiumDerivedCells',
 'GreatVesselAdventitialFibroblasts',
 'GreatVesselArterialEndothelialCells',
 'GreatVesselSmoothMuscleCells',
 'GreatVesselVenousEndothelialCells',
 'InnateLymphoidCells',
 'LymphNodeFibroblasticReticularCells',
 'LymphaticEndothelialCells',
 'MacrophagesATF3pos',
 'MacrophagesCX3CR1pos',
 'MacrophagesLYVE1pos',
 'MacrophagesTIMD4pos',
 'MastCells',
 'Megakaryocytes',
 'MesothelialEpicardialCells',
 'MonocyteDerivedCells',
 'Monocytes',
 'MonocytesMPOpos',
 'MyocardialInterstitialFibroblasts',
 'Myofibroblasts',
 'NaturalKillerCells',
 'NeuronPrecursors',
 'ParasympatheticNeurons',
 'PericardialCellsFibrous',
 'PericardialCellsIntermediate',
 'PericardialCellsParietal',
 'PlasmacytoidDendriticCells',
 'ProBCells',
 'SchwannCellPrecursors',
 'SchwannCells',
 'SinoatrialNodePacemakerCells',
 'SubEpicardialFibroblasts',
 'SympatheticNeurons',
 'TCellsCD4pos',
 'TCellsCD8pos',
 'TregsCD4pos',
 'ValveEndothelialCells',
 'ValveInterstitialCells',
 'VentricularCardiomyocytesCycling',
 'VentricularCardiomyocytesLeftCompact',
 'VentricularCardiomyocytesLeftTrabeculated',
 'VentricularCardiomyocytesRightCompact',
 'VentricularCardiomyocytesRightTrabeculated',
 'VentricularConductionSystemDistal',
 'VentricularConductionSystemProximal'}
```

Example 13 (python):
```python
adata = adata[adata.obs['fine_grain'].isin(['GreatVesselArterialEndothelialCells','GreatVesselVenousEndothelialCells','CoronaryArterialEndothelialCells','CoronaryCapillaryEndothelialCells','CoronaryVenousEndothelialCells'])].copy()
adata=adata[adata.obs.cycling=='no',:].copy()
```

Example 14 (python):
```python
cell_cycle_genes = ['MCM5','PCNA','TYMS','FEN1','MCM2','MCM4','RRM1','UNG','GINS2','MCM6','CDCA7','DTL','PRIM1','UHRF1','MLF1IP','HELLS','RFC2','RPA2','NASP','RAD51AP1','GMNN','WDR76','SLBP','CCNE2','UBR7','POLD3','MSH2','ATAD2','RAD51','RRM2','CDC45','CDC6','EXO1','TIPIN','DSCC1','BLM','CASP8AP2','USP1','CLSPN','POLA1','CHAF1B','BRIP1','E2F8','HMGB2','CDK1','NUSAP1','UBE2C','BIRC5','TPX2','TOP2A','NDC80','CKS2','NUF2','CKS1B','MKI67','TMPO','CENPF','TACC3','FAM64A','SMC4','CCNB2','CKAP2L','CKAP2','AURKB','BUB1','KIF11','ANP32E','TUBB4B','GTSE1','KIF20B','HJURP','CDCA3','HN1','CDC20','TTK','CDC25C','KIF2C','RANGAP1','NCAPD2','DLGAP5','CDCA2','CDCA8','ECT2','KIF23','HMMR','AURKA','PSRC1','ANLN','LBR','CKAP5','CENPE','CTCF','NEK2','G2E3','GAS2L3','CBX5','CENPA']
s_genes = cell_cycle_genes[:43]
s_genes = [x for x in s_genes if x in adata.var_names]
print('S genes:', s_genes, '\n')
g2m_genes = cell_cycle_genes[43:]
g2m_genes = [x for x in g2m_genes if x in adata.var_names]
print('G2 genes:', g2m_genes, '\n')
cell_cycle_genes = [x for x in cell_cycle_genes if x in adata.var_names]

hb_genes1 = ['HBB','HBD','HBG1','HBG2','HBE1','HBZ','HBM','HBA2','HBA1', 'HBQ1']
hb_genes1 = [x for x in hb_genes1 if x in adata.var_names]
print('Hemoglobin genes set 1:', hb_genes1, '\n')

hb_genes2 = ['AHSP','GYPA','RHAG','GFI1B','KEL','EPB42','ALAS2','TSPO2','GYPB','GYPE','SPTA1','RHCE','KLF1','RHD','SLC4A1','SNX22','SQSTM1','SOD1','FTH1','MAP1LC3B','SLC25A37','TRIM58','HMBS']
hb_genes2 = [x for x in hb_genes2 if x in adata.var_names]
print('Hemoglobin genes set 2:', hb_genes2, '\n')

mito_genes=['MT-ND1','MT-ND2','MT-CO1','MT-CO2','MT-ATP8','MT-ATP6','MT-CO3','MT-ND3','MT-ND4L', 'MT-ND4', 'MT-ND5', 'MT-ND6', 'MT-CYB']
mito_genes = [x for x in mito_genes if x in adata.var_names]
print('Mitochondrial genes:', mito_genes, '\n')

ribo_genes = ['RPLP0','RPLP1','RPLP2','RPL3','RPL3L','RPL4','RPL5','RPL6','RPL7','RPL7A','RPL7L1','RPL8','RPL9','RPL10','RPL10A','RPL10L','RPL11','RPL12','RPL13','RPL13A','RPL14','RPL15','RPL17','RPL18','RPL18A','RPL19','RPL21','RPL22','RPL22L1','RPL23','RPL23A','RPL24','RPL26','RPL26L1','RPL27','RPL27A','RPL28','RPL29','RPL30','RPL31','RPL32','RPL34','RPL35','RPL35A','RPL36','RPL36A','RPL36AL','RPL37','RPL37A','RPL38','RPL39','RPL39L','UBA52','RPL41','RPSA','RPS2','RPS3','RPS3A','RPS4X','RPS4Y1','RPS4Y2','RPS5','RPS6','RPS7','RPS8','RPS9','RPS10','RPS11','RPS12','RPS13','RPS14','RPS15','RPS15A','RPS16','RPS17','RPS18','RPS19','RPS20','RPS21','RPS23','RPS24','RPS25','RPS26','RPS27','RPS27A','RPS27L','RPS28','RPS29','FAU']
ribo_genes = [x for x in ribo_genes if x in adata.var_names]
print('Ribo genes:', ribo_genes, '\n')

stress_genes=['FOS','FOSB','JUN','JUNB','JUND','ZFP36','SOCS3','CEBPD','CEBPB','ATF3','EGR1','HSPA1A','HSPA1B','HSP90AA1','HSP90AB1','HSPA8','HSPB1','HSPE1','IER3','IER2','BTG1','BTG2','DUSP1']
stress_genes = [x for x in stress_genes if x in adata.var_names]
print('Tissue dissociation stress associated genes:', stress_genes, '\n')

FoxProteins=['FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1']
chrY_genes=['SRY', 'RPS4Y1', 'ZFY', 'ZFY-AS1', 'LINC00278', 'PCDH11Y', 'TTTY2B', 'TBL1Y', 'PRKY', 'USP9Y', 'DDX3Y', 'UTY', 'TMSB4Y', 'NLGN4Y', 'TTTY14', 'KDM5D', 'TTTY10', 'EIF1AY', 'PRORY', 'LINC00266-4P']
MembraneProteins1=['AADAT','AAK1','ABCA12','ABCA7','ABCB1','ABCB11','ABCB4','ABCB6','ABCC1','ABCC3','ABCC4','ABCC5','ABCC6','ABCG2','ABHD3','ABI1','ABLIM3','AC004691.2','AC004922.1','AC005324.3','AC005726.1','AC008403.1','AC138696.1','ACIN1','ACKR3','ACLY','ACOT13','ACOT9','ACTN1','ACTR10','ACTR6','ADA','ADAM10','ADAM12','ADAM22','ADAM28','ADAM33','ADAMTS1','ADAMTS2','ADAMTSL4','ADAP1','ADCK5','ADD1','ADD2','ADGRB2','ADGRD1','ADGRG2','ADGRG7','ADGRL2','ADH1A','ADH1B','ADH1C','ADH7','ADK','ADORA1','ADRA1B','ADRM1','ADSS','AFAP1L2','AFDN','AFG1L','AGER','AGRN','AHNAK','AHNAK2','AIFM2','AKAP10','AKAP11','AKAP12','AKR1B10','AKR1B15','AL132780.3','AL139300.1','AL355315.1','AL358113.1','AL645922.1','ALDH3A1','ALK','ALOX15B','ALOXE3','ALPI','ALPP','ALPPL2','ALS2CR12','AMACR','AMDHD1','AMER1','AMER2','AMIGO2','AMOT','AMOTL1','AMOTL2','AMPH','ANK2','ANK3','ANKEF1','ANKFN1','ANKIB1','ANKLE2','ANKRA2','ANKRD13A','ANKRD35','ANKRD42','ANKRD45','ANKS1B','ANO1','ANO10','ANO2','ANO6','ANPEP','ANXA1','ANXA13','ANXA2','ANXA3','AP2M1','APBB1','APBB1IP','APC','APLP1','APMAP','APOC3','APOD','APOLD1','AQP1','AQP10','AQP3','AQP4','AQP5','ARAP1','ARF1','ARHGAP18','ARHGAP19','ARHGAP21','ARHGAP22','ARHGAP23','ARHGAP24','ARHGAP28','ARHGAP29','ARHGAP33','ARHGAP36','ARHGAP40','ARHGAP9','ARHGDIG','ARHGEF1','ARHGEF11','ARHGEF12','ARHGEF15','ARHGEF26','ARHGEF28','ARHGEF40','ARHGEF5','ARID2','ARID4A','ARID5B','ARL13A','ARL15','ARL4C','ARMC12','ARMCX5','ARPC1A','ARPC5','ARRB2','ARRDC4','ARVCF','ASAP1','ASB5','ASGR1','ASGR2','ASH2L','ASIC1','ASIC4','ASPM','ASPSCR1','ATG13','ATG3','ATG7','ATIC','ATOX1','ATP10D','ATP12A','ATP13A3','ATP1B3','ATP23','ATP2B1','ATP2B4','ATP2C2','ATP6V0A2','ATP6V1H','ATP8A2','ATP8B1','ATXN10','ATXN3','ATXN7L3','AVEN','AVPI1','AXIN2','AXL','B2M','B3GALT2','BACE1','BAG3','BAIAP2','BAIAP2L1','BANK1','BASP1','BBOF1','BCAR1','BCAR3','BCL2L12','BCORL1','BCR','BDKRB2','BEND6','BFSP1','BFSP2','BICD2','BIN2','BLNK','BLVRB','BMPR2','BMX','BOC','BOLL','BORCS5','BORCS8-MEF2B','BRK1','BRPF1','BRSK1','BSDC1','BST2','BTK','BTN2A1','BTRC','BUB1B-PAK6','BVES','BZW2','C10orf90','C11orf42','C11orf52','C12orf73','C12orf76','C14orf37','C14orf93','C16orf74','C17orf58','C17orf62','C18orf21','C19orf33','C19orf38','C19orf48','C19orf71','C1orf116','C1orf189','C1orf35','C1orf54','C1orf68','C1orf74','C1QBP','C2orf72','C3orf14','C3orf20','C3orf38','C4orf19','C4orf22','C7orf43','C8orf33','C8orf37','C9orf153','C9orf3','CA14','CA9','CABLES2','CACNA1E','CACNG4','CADM1','CALCRL','CALD1','CALML5','CALN1','CAMK2A','CAMK2B','CAMK2D','CAMK2G','CAP2','CAPS','CASKIN2','CASP4','CAST','CAV3','CAVIN1','CAVIN2','CAVIN3','CAVIN4','CBWD1','CBWD2','CBWD3','CBWD5','CBWD6','CC2D1A','CCBE1','CCDC107','CCDC112','CCDC115','CCDC124','CCDC136','CCDC144NL','CCDC15','CCDC154','CCDC170','CCDC24','CCDC28A','CCDC59','CCDC62','CCDC66','CCDC69','CCDC8','CCDC83','CCDC85C','CCDC93','CCDC97','CCNT2','CCNYL1','CCR3','CCR4','CCT3','CD109','CD163','CD164L2','CD200R1','CD226','CD248','CD274','CD2AP','CD300A','CD300LB','CD33','CD34','CD37','CD38','CD3E','CD3G','CD4','CD44','CD46','CD47','CD48','CD79B','CD81','CD84','CD86','CD8A','CD8B','CD9','CD93','CD99L2','CDAN1','CDC42EP1','CDC42EP3','CDC42EP4','CDC42SE1','CDCA4','CDH1','CDH13','CDH15','CDH17','CDH2','CDH26','CDH3','CDH4','CDH5','CDH6','CDH8','CDIPT','CDK16','CDK5','CDK5RAP2','CDK7','CDRT15','CDV3','CEACAM5','CELSR1','CENPJ','CEP162','CEP295','CEP55','CEP57L1','CEP89','CERCAM','CFAP157','CFAP206','CFAP58','CFAP65','CFB','CFL2','CFLAR','CGN','CGNL1','CHCHD7','CHIC2','CHMP2A','CHRM3','CHRNA1','CHRNA5','CHRNB4','CHRND','CIB1','CIB3','CIP2A','CKAP5','CLCA2','CLCA4','CLCN5','CLCN6','CLDN11','CLDN18','CLDN2','CLDN23','CLDN3','CLDN4','CLDN7','CLDN9','CLEC2A','CLEC4F','CLEC5A','CLECL1','CLIC2','CLIC5','CLTB','CLUL1','CMTM4','CMTM7','CMTR2','CMYA5','CNGA1','CNKSR2','CNNM1','CNNM4','CNP','CNR1','CNTD1','COBL','COBLL1','COG3','COL16A1','COL17A1','COL4A5','COL6A2','COLEC12','COLQ','COMMD4','COMTD1','COPRS','COQ7','CORIN','CORO1B','CORO2A','CORO2B','COX6B2','CPEB3','CPPED1','CPZ','CRB3','CREB5','CRIP1','CRIP2','CRK','CRLF3','CROCC','CRTC1','CRYAB','CRYBG3','CRYL1','CSDE1','CSF1','CSF1R','CSPG4','CST6','CTNNA1','CTNNB1','CTNNBIP1','CTNND1','CTNND2','CTSF','CTSV','CTTN','CTXN1','CUEDC1','CUTA','CX3CL1','CX3CR1','CXADR','CXCR2','CYB5D2','CYBRD1','CYFIP2','CYP20A1','CYP24A1','CYTH2','CYTH4','DAAM1','DAB2','DAG1','DAGLB','DAW1','DBN1','DCAF13','DCAF16','DCBLD2','DCHS1','DCHS2','DCP2','DCT','DCTN6','DCUN1D2','DDAH1','DDB2','DDIAS','DDIT4L','DDR1','DDR2','DDX58','DDX60L','DENND6B','DFFA','DGKG','DGKH','DHDDS','DHPS','DHRS7B','DIAPH1','DIAPH3','DIO2','DIS3L','DLG1','DLG2','DLG5','DLGAP3','DLL3','DMWD','DNAAF1','DNAAF4','DNAH17','DNAJA4','DNAJB13','DNAJB4','DNAJC18','DNAJC5','DNAJC6','DNAJC9','DOC2A','DOCK4','DOK3','DONSON','DPEP1','DPH1','DPPA5','DPYSL2','DQX1','DRD1','DRP2','DSC3','DSEL','DSG2','DSP','DTNA','DUS1L','DUSP15','DXO','DYSF','ECSCR','EDA2R','EDNRA','EDNRB','EEF1AKMT1','EEF2','EFCAB1','EFCAB11','EFCAB2','EFCAB7','EFHC2','EFNB1','EFR3A','EFR3B','EGFR','EHBP1','EHD1','EHD2','EHD3','EHD4','EIF2B2','EIF5','EIF5B','ELFN1','ELMO3','ELN','ELOC','EMP3','ENAH','ENC1','ENG','ENKD1','ENO1','ENO2','ENO3','ENOX1','ENOX2','EPB41','EPB41L1','EPB41L2','EPB41L3','EPB41L4A','EPB41L4B','EPB41L5','EPCAM','EPHA2','EPHA3','EPHB1','EPHB2','EPN1','EPOR','EPS8L2','ERAP1','ERBB2','ERBB3','ERBIN','ERC1','ERCC4','ERMN','ESCO2','ETNK1','ETS2','EVA1A','EVI2B','EXOC1','EXOC3L1','EXOC7','EZR','F11R','F2RL1','F2RL3','FAAP20','FABP2','FABP3','FABP5','FAIM','FAM126A','FAM126B','FAM129A','FAM129B','FAM13A','FAM163B','FAM171A1','FAM193A','FAM205A','FAM210B','FAM222A','FAM45A','FAM47E','FAM47E-STBD1','FAM72A','FAM72B','FAM72C','FAM72D','FANCG','FAS','FASN','FBLIM1','FBLN2','FBLN5','FBLN7','FBP2','FBRSL1','FBXL2','FBXO16','FBXO38','FBXO41','FBXW11','FCAR','FCGBP','FCGR1A','FCGR1B','FCGR2A','FCHSD2','FFAR4','FGD1','FGD5','FGFR2','FGR','FHIT','FHL1','FIBCD1','FIGN','FILIP1','FILIP1L','FKBP2','FLAD1','FLCN','FLNA','FLNB','FLNC','FLOT1','FLOT2','FLRT3','FLT1','FLT4','FLVCR1','FMN2','FMNL2','FMNL3','FNBP1','FOXA2','FOXN3','FP565260.3','FRMD3','FRMD7','FRMD8','FRS2','FSCN1','FTCD','FUT2','FXYD3','FYB1','FYN','FZD1','FZD2','FZD4','FZD6','GAB1','GAB2','GABRA1','GABRA2','GABRA3','GABRA5','GABRG1','GAD1','GADL1','GAGE1','GAGE10','GAGE12C','GAGE12D','GAGE12E','GAGE12F','GAGE12G','GAGE12H','GAGE12J','GAGE13','GAGE2A','GAGE2E','GALNT10','GALR1','GAP43','GAPDH','GAPT','GAPVD1','GAS2','GAS2L1','GAS7','GAS8','GBP4','GCA','GCC1','GCLM','GCSAML-AS1','GFI1B','GFOD1','GFRA3','GHR','GID8','GINM1','GIPC1','GJA1','GJB3','GJB4','GJB6','GJB7','GJC1','GJC2','GLO1','GLP2R','GLRX','GLUL','GMIP','GNAI2','GNAQ','GNAS','GNB1','GNB2','GNB3','GNB4','GNG2','GP9','GPAT3','GPATCH1','GPBP1','GPBP1L1','GPC1','GPC3','GPC4','GPI','GPR1','GPR142','GPR156','GPR157','GPR160','GPR173','GPR20','GPR27','GPR50','GPR63','GPRC5A','GPRIN1','GPRIN2','GPX8','GRAMD1A','GRAMD1B','GRAMD1C','GRAP2','GRB7','GRID2IP','GRIK5','GRK3','GRK5','GRPR','GSDMA','GTF2F1','GTSE1','GUCA1A','GYPA','GYPC','HAPLN3','HARBI1','HAS1','HAUS4','HAUS7','HCAR2','HCAR3','HCFC2','HCK','HCLS1','HCST','HDAC8','HEG1','HELZ','HENMT1','HERC2','HERPUD1','HFE','HIF1A','HIF3A','HINT1','HIP1R','HLA-A','HLA-DPB1','HMGCS1','HMOX1','HOMER3','HOXC5','HOXD4','HPS3','HRASLS5','HRH1','HS6ST3','HSH2D','HSPB1','HSPG2','HTR1B','HTR7','HTRA1','HUNK','HYDIN','HYI','HYLS1','ICAM1','ICAM2','ICOS','ICOSLG','IFFO2','IFIT5','IFNGR1','IFRD1','IFT122','IGFL2','IGSF11','IGSF23','IKZF5','IL10RA','IL12RB1','IL12RB2','IL13','IL13RA2','IL16','IL17RB','IL1R1','IL1RL1','IL21R','IL4R','IL6ST','IL7R','ILDR2','ILK','INPP5A','INSR','INTS12','IP6K2','IPCEF1','IQGAP1','IQGAP2','IREB2','IRF2','IRS1','ISLR','ITGA2B','ITGA3','ITGA4','ITGA7','ITGA8','ITGA9','ITGB1','ITGB2','ITGB3','ITGB4','ITGB5','ITGB6','ITGB7','ITGB8','ITK','ITM2A','ITPKA','ITSN1','ITSN2','IYD','JAG1','JAK2','JAK3','JAM2','JAML','JCAD','JMJD4','JPT2','JUP','KANK1','KANK3','KANSL2','KARS','KATNB1','KCNAB1','KCNB2','KCNH8','KCNJ15','KCNK13','KCNK17','KCNK6','KCNK7','KCNK9','KCNN2','KCNS3','KCNU1','KDF1','KDR','KEL','KIAA0556','KIAA0895','KIAA1324','KIAA1522','KIAA1958','KIF17','KIF19','KIF21A','KIF21B','KIF26A','KIF26B','KISS1R','KIT','KLC1','KLC2','KLF9','KLHDC7B','KLHL41','KLHL6','KLHL7','KLK7','KLRC1','KLRC2','KLRF1','KLRK1','KMT2D','KNDC1','KNSTRN','KNTC1','KPTN','KRT75','KRT77','KRTAP25-1','L1CAM','L3MBTL1','LAIR1','LAMA4','LAMC1','LAMP2','LAMTOR1','LAPTM4A','LASP1','LAT','LAT2','LAX1','LCP1','LCTL','LDB2','LDLRAD3','LENG1','LGALS3','LGALS4','LGALS8','LGR6','LHFPL2','LHX6','LIMA1','LINGO1','LIPJ','LMCD1','LMOD2','LMTK2','LMTK3','LNX1','LNX2','LONRF1','LONRF3','LOXHD1','LPAR1','LPAR3','LPAR4','LPIN1','LPIN2','LRCH2','LRP11','LRP12','LRRC15','LRRC26','LRRC34','LRRC45','LRRC69','LRRC7','LRRC9','LRRFIP2','LRRIQ3','LSMEM2','LSR','LTB4R2','LUZP2','LY6K','LYN','LYPD1','LYST','LZTS1','LZTS2','MACF1','MADD','MAGI1','MAGI3','MAMDC2','MAP1S','MAP2','MAP2K1','MAP2K4','MAP2K7','MAP3K12','MAP3K21','MAP3K6','MAP3K7','MAP4','MAP4K1','MAP4K5','MAP6','MAPK15','MAPK8IP1','MAPT','MARCH10','MARCH7','MARCKS','MARK2','MARVELD2','MAS1L','MAT2B','MBP','MC5R','MCAM','MCEMP1','MCF2L2','MCMBP','MCTS1','MDGA1','ME1','MEF2B','MEIOC','MEPCE','MERTK','MET','METAP1','METAP2','METTL15','METTL25','METTL9','MFAP3L','MFSD1','MFSD2A','MGAT4C','MIB1','MICAL2','MICAL3','MICALL1','MICB','MIEN1','MIGA2','MISP','MKLN1','MKRN3','MLF2','MLKL','MLST8','MMACHC','MMP10','MMP15','MMP24','MMP25','MOAP1','MPHOSPH8','MPHOSPH9','MPI','MPIG6B','MPL','MPP1','MPP5','MPP7','MPZL1','MPZL2','MR1','MRGPRF','MRNIP','MRPL42','MRPL44','MRPL46','MRPS18B','MS4A1','MS4A12','MS4A14','MS4A15','MSN','MSRA','MTCP1','MTFR1L','MTHFR','MTM1','MTPN','MTSS1','MUC1','MUC20','MVD','MYH11','MYH9','MYL4','MYLK','MYO10','MYO1B','MYO1C','MYO1E','MYO1G','MYO3A','MYO5B','MYO9A','MYOF','N4BP2','NAA35','NAB1','NAIF1','NAMPT','NAP1L2','NAPG','NARFL','NCAM1','NCAM2','NCK1','NCKAP1L','NCKAP5','NCKIPSD','NCOA1','NCS1','NDC1','NECTIN2','NECTIN3','NECTIN4','NEDD1','NEDD9','NEO1','NEU1','NEXN','NF2','NFE2L2','NFKBID','NGEF']
MembraneProteins2=['NGFR','NHS','NHSL2','NID2','NINJ2','NKTR','NKX2-8','NLGN1','NLGN3','NLK','NLRX1','NMB','NMBR','NME1','NME2','NMT1','NMT2','NMUR2','NOS1','NOTCH2','NPFFR2','NPL','NPNT','NPR1','NPTN','NPTXR','NR2C1','NRAS','NRCAM','NRP2','NRSN1','NRXN1','NSD1','NSFL1C','NT5E','NTNG1','NTNG2','NTSR1','NUDCD3','NUDT16L1','NUDT9','NUMB','NUP35','NUS1','OAS3','OBSCN','OCA2','OCLN','OCM','OCM2','ODC1','ODR4','OGT','OLFM4','OLIG2','OLR1','OPHN1','OPRK1','OR10P1','OR1N2','OR2C3','OR6B2','OR6B3','OR6C6','OR8S1','OSBPL10','OSBPL6','OSBPL7','OSCAR','OSCP1','OSGEP','OSR2','OTOP2','OTULIN','OXGR1','P3H3','PACSIN2','PACSIN3','PAFAH1B2','PAG1','PAIP1','PAK1','PAK4','PAK6','PALLD','PALM','PALM2','PALM2-AKAP2','PALM3','PALMD','PANX1','PANX2','PAQR8','PARD3','PARD3B','PARD6A','PARD6G','PARP6','PATJ','PAWR','PCBD2','PCDH1','PCDH17','PCDH7','PCDHA1','PCDHA11','PCDHA12','PCDHA4','PCDHA5','PCDHB1','PCDHB11','PCDHB12','PCDHB16','PCDHB4','PCDHB5','PCDHGA1','PCDHGA10','PCDHGA11','PCDHGA12','PCDHGA2','PCDHGA3','PCDHGA4','PCDHGA5','PCDHGA6','PCDHGA7','PCDHGA8','PCDHGA9','PCDHGB1','PCDHGB2','PCDHGB3','PCDHGB4','PCDHGB5','PCDHGB6','PCDHGB7','PCDHGC3','PCDHGC4','PCDHGC5','PCLO','PCMTD1','PDAP1','PDCD7','PDE3A','PDE4A','PDE4D','PDE9A','PDGFC','PDGFRA','PDHX','PDLIM1','PDLIM5','PDPN','PDZD2','PDZD8','PEAR1','PEBP1','PFDN1','PGAP3','PGLYRP2','PGLYRP4','PGRMC2','PHACTR1','PHACTR4','PHF20L1','PHGDH','PHLDB2','PHLDB3','PHOSPHO1','PHPT1','PI3','PI4K2A','PI4KA','PIEZO2','PIGB','PIGN','PIGW','PIGX','PIK3AP1','PIK3CG','PIK3R6','PIP4K2A','PIP4K2C','PIP5K1A','PIRT','PITPNM3','PKD2L2','PKN1','PKN2','PKP1','PKP2','PKP3','PKP4','PLAA','PLAUR','PLCB1','PLCB2','PLCD3','PLCD4','PLEKHA4','PLEKHA6','PLEKHA7','PLEKHG3','PLEKHG4','PLEKHG6','PLEKHH2','PLEKHO2','PLIN4','PLPP1','PLPPR5','PLS3','PLSCR1','PLVAP','PLXNA3','PLXNA4','PNCK','PNISR','PODXL','POLE','POU2F3','PPEF1','PPFIBP1','PPIP5K1','PPL','PPM1A','PPM1F','PPP1CA','PPP1CB','PPP1R12B','PPP1R16A','PPP1R16B','PPP1R18','PPP1R27','PPP1R9A','PPP2R1B','PPP3R1','PPP3R2','PPP4C','PPP6R3','PPTC7','PRAME','PRC1','PRDX6','PRIMA1','PRKCA','PRKCE','PRKCH','PRKCZ','PRKD1','PROM2','PROSER2','PRR15','PRR32','PRR7','PRRC2A','PRRG1','PRRT2','PRRT3','PRRT4','PRTG','PRX','PSCA','PSD','PSD2','PSEN1','PSMC5','PSMC6','PSMD9','PSTPIP2','PTAFR','PTCHD4','PTK2B','PTK6','PTPN12','PTPN21','PTPN22','PTPN4','PTPRB','PTPRG','PTPRK','PTPRM','PTPRR','PTPRS','PTX3','PURG','PVALB','PVR','PXDC1','PXK','PYGL','PYGO2','PYM1','RAB13','RAB23','RAB25','RAB26','RAB27B','RAB3B','RAB3GAP2','RAB43','RAB8A','RABEPK','RAD51C','RAD52','RAI2','RALA','RALGAPA2','RAPGEF2','RAPH1','RAPSN','RASA4','RASA4B','RASAL1','RASAL2','RASAL3','RASD1','RASD2','RASGRP1','RASGRP2','RASGRP4','RASSF2','RASSF3','RASSF4','RBM46','RC3H1','RCCD1','RCSD1','RD3L','RDX','RECK','RELL1','REPS1','RET','RFLNB','RFPL1','RFPL2','RFPL3','RGP1','RGS10','RGS14','RGS19','RGS3','RGS7BP','RGS9BP','RHCE','RHD','RHEX','RHOA','RHOBTB2','RHOJ','RHOXF2','RHOXF2B','RIC8A','RIC8B','RIF1','RILPL1','RIMBP3','RIMBP3B','RIMBP3C','RIMS2','RIOK2','RIOK3','RIPK1','RND1','RNF114','RNF145','RNF146','RNF149','RNF165','RNF166','RNF39','ROBO1','ROBO3','ROBO4','ROM1','RP2','RPE','RPEL1','RPGRIP1L','RPIA','RPL36A','RPL36A-HNRNPH2','RPL36AL','RPSA','RPUSD1','RRAD','RRP12','RRP7A','RSBN1L','RSC1A1','RSPH3','RTL3','RTL4','RTN4R','RTN4RL1','RYR2','S100A13','S100A14','S100A16','S100A2','S100A3','S100A4','S100A6','S100A9','SAMD4A','SAMHD1','SAMSN1','SAPCD1','SASH3','SC5D','SCAMP1','SCAMP4','SCEL','SCHIP1','SCIN','SCN1A','SCN5A','SCN8A','SCN9A','SCNN1D','SCNN1G','SCRIB','SCUBE3','SDC2','SDC4','SDE2','SDHB','SEC16B','SEMA3C','SEMA3D','SEMA4B','SEMA4F','SEMA6C','SEMA6D','SEPT2','SEPT3','SERINC2','SERINC5','SERPINB3','SFSWAP','SGCE','SGK494','SH3BGR','SH3D21','SHANK1','SHANK3','SHARPIN','SHISA6','SHKBP1','SHROOM2','SHTN1','SIGLEC10','SIGLEC14','SIGLEC6','SIGLEC7','SIN3B','SIPA1','SIPA1L1','SIPA1L3','SKAP1','SLAMF6','SLC10A5','SLC12A2','SLC13A3','SLC13A5','SLC14A1','SLC16A1','SLC16A10','SLC16A2','SLC16A3','SLC16A7','SLC16A9','SLC17A5','SLC19A1','SLC19A3','SLC1A5','SLC1A6','SLC22A16','SLC23A3','SLC24A4','SLC26A9','SLC27A1','SLC28A1','SLC2A1','SLC2A11','SLC2A12','SLC2A14','SLC2A2','SLC2A3','SLC30A1','SLC31A2','SLC34A1','SLC34A2','SLC35F5','SLC35G2','SLC36A4','SLC38A5','SLC38A6','SLC38A7','SLC39A10','SLC39A14','SLC39A2','SLC39A6','SLC3A2','SLC40A1','SLC41A3','SLC43A1','SLC43A3','SLC44A2','SLC45A4','SLC46A1','SLC46A3','SLC4A1AP','SLC4A2','SLC4A7','SLC4A8','SLC5A12','SLC5A3','SLC5A6','SLC5A7','SLC6A6','SLC7A14','SLC7A2','SLC7A5','SLC7A6','SLC8A1','SLC9A1','SLC9A2','SLC9A3','SLC9A3R1','SLC9A3R2','SLC9A4','SLC9A5','SLC9B2','SLC9C2','SLCO1A2','SLCO1B3','SLCO4A1','SLCO4C1','SLCO5A1','SLIT1','SLIT3','SLITRK6','SLK','SLX4','SMAGP','SMAP1','SMOC1','SMPD2','SMPX','SNAP23','SNAP25','SNRK','SNTB2','SNTG2','SNX15','SNX32','SNX9','SOCS3','SOCS5','SOD1','SORBS1','SORCS1','SORT1','SOX11','SPAG16','SPANXN4','SPART','SPATA12','SPATC1L','SPERT','SPG11','SPHK1','SPINK13','SPINT1','SPINT2','SPN','SPPL2B','SPPL3','SPRED3','SPRYD3','SPTBN2','SRC','SRCIN1','SRD5A3','SRPK1','SRPRB','SRRM3','SSFA2','SSH1','SSH3','SSTR1','SSTR5','SSX2IP','ST8SIA5','STAB1','STAC','STAC2','STAMBP','STAMBPL1','STARD4','STAT2','STEAP4','STIL','STIP1','STK10','STK16','STK17A','STK17B','STOM','STOML2','STOML3','STX18','STX4','STXBP4','STYK1','SUCLG1','SUCLG2','SUGT1','SUMF2','SUPT6H','SUSD2','SVIL','SWAP70','SYCP2','SYK','SYMPK','SYNC','SYNGR3','SYNPO2L','SYPL2','SYT4','SYTL2','SYVN1','TACC2','TACR2','TACSTD2','TASP1','TAZ','TBC1D10A','TBC1D2','TBC1D24','TBC1D30','TBC1D9B','TBXA2R','TCAF2','TCAF2C','TCHP','TCIM','TDG','TDO2','TDP1','TEC','TEDC2','TEK','TES','TEX14','TEX9','TGFBR1','TGFBR2','TGM1','TGM2','THBS1','THBS2','THEGL','THY1','TIAM1','TICAM1','TIGD7','TIMD4','TJP1','TJP2','TJP3','TLN1','TLR4','TM2D2','TM4SF20','TM4SF4','TM4SF5','TMC5','TMED8','TMEM101','TMEM107','TMEM108','TMEM121B','TMEM139','TMEM141','TMEM164','TMEM176B','TMEM183A','TMEM186','TMEM2','TMEM218','TMEM222','TMEM229A','TMEM245','TMEM251','TMEM254','TMEM258','TMEM266','TMEM268','TMEM50A','TMEM63B','TMPRSS15','TMPRSS2','TMTC2','TNFRSF10D','TNFRSF11A','TNFRSF12A','TNFRSF21','TNFRSF25','TNFRSF8','TNFRSF9','TNFSF13B','TNK1','TNKS1BP1','TNNI2','TOGARAM2','TOLLIP','TOM1','TP53BP2','TPD52L1','TPRN','TRAF7','TRAIP','TRAPPC13','TRAT1','TRDN','TREML1','TRIB1','TRIM16','TRIM16L','TRIM38','TRIM4','TRIM44','TRIM54','TRIM71','TRIML2','TRIP6','TRMT1','TRMT10B','TRMT13','TRMT6','TRO','TRPM1','TRPM4','TRPM7','TRPV2','TRPV3','TSG101','TSGA13','TSHZ3','TSPAN15','TSPAN6','TSPO2','TSR2','TSSK4','TTC17','TTC23','TTC7B','TTLL12','TTLL5','TULP3','TWF2','TXNDC11','TXNIP','TYROBP','TYW3','UAP1','UBA3','UBA52','UBAC1','UBALD1','UBAP1','UBASH3B','UBE2A','UBE2B','UBE2C','UBE2D1','UBE2D2','UBE2D3','UBE2D4','UBE2K','UBE2QL1','UBE2S','UBE3C','UBQLN2','UBR2','UBTD1','UCP3','ULBP1','ULBP2','UNC5A','UNC79','UQCC1','USP14','USP15','USP2','USP20','USP21','USP4','USP53','UTP20','UTRN','VAMP5','VAMP8','VANGL1','VASP','VCAM1','VCPIP1','VIL1','VPS13B','VPS36','VPS50','VPS8','VSIG4','VTCN1','VWA5B1','VWA7','WAS','WASF2','WBP4','WDFY3','WDPCP','WDR1','WDR12','WDR13','WDR53','WDR6','WDR66','WIPF1','WWP1','XIAP','XIRP1','XIRP2','XPC','XPNPEP2','XPO6','XRN1','XYLT1','YES1','YIPF1','YIPF4','YOD1','YTHDC1','ZBED3','ZBTB33','ZBTB8OS','ZDHHC2','ZDHHC20','ZDHHC22','ZDHHC5','ZFP41','ZFP90','ZFYVE28','ZGPAT','ZMYND10','ZNF10','ZNF18','ZNF185','ZNF2','ZNF20','ZNF227','ZNF229','ZNF236','ZNF276','ZNF277','ZNF337','ZNF397','ZNF407','ZNF497','ZNF503','ZNF510','ZNF536','ZNF571','ZNF598','ZNF599','ZNF646','ZNF655','ZNF688','ZNF706','ZNF717','ZNF783','ZNF79','ZNF839','ZSCAN22','ZSCAN29','ZSWIM1','ZSWIM8','ZYX']
MembraneProteins=MembraneProteins1+MembraneProteins2

TFs1=['AC008770.3','AC023509.3','AC092835.1','AC138696.1','ADNP','ADNP2','AEBP1','AEBP2','AHCTF1','AHDC1','AHR','AHRR','AIRE','AKAP8','AKAP8L','AKNA','ALX1','ALX3','ALX4','ANHX','ANKZF1','AR','ARGFX','ARHGAP35','ARID2','ARID3A','ARID3B','ARID3C','ARID5A','ARID5B','ARNT','ARNT2','ARNTL','ARNTL2','ARX','ASCL1','ASCL2','ASCL3','ASCL4','ASCL5','ASH1L','ATF1','ATF2','ATF3','ATF4','ATF5','ATF6','ATF6B','ATF7','ATMIN','ATOH1','ATOH7','ATOH8','BACH1','BACH2','BARHL1','BARHL2','BARX1','BARX2','BATF','BATF2','BATF3','BAZ2A','BAZ2B','BBX','BCL11A','BCL11B','BCL6','BCL6B','BHLHA15','BHLHA9','BHLHE22','BHLHE23','BHLHE40','BHLHE41','BNC1','BNC2','BORCS8-MEF2B','BPTF','BRF2','BSX','C11orf95','CAMTA1','CAMTA2','CARF','CASZ1','CBX2','CC2D1A','CCDC169-SOHLH2','CCDC17','CDC5L','CDX1','CDX2','CDX4','CEBPA','CEBPB','CEBPD','CEBPE','CEBPG','CEBPZ','CENPA','CENPB','CENPBD1','CENPS','CENPT','CENPX','CGGBP1','CHAMP1','CHCHD3','CIC','CLOCK','CPEB1','CPXCR1','CREB1','CREB3','CREB3L1','CREB3L2','CREB3L3','CREB3L4','CREB5','CREBL2','CREBZF','CREM','CRX','CSRNP1','CSRNP2','CSRNP3','CTCF','CTCFL','CUX1','CUX2','CXXC1','CXXC4','CXXC5','DACH1','DACH2','DBP','DBX1','DBX2','DDIT3','DEAF1','DLX1','DLX2','DLX3','DLX4','DLX5','DLX6','DMBX1','DMRT1','DMRT2','DMRT3','DMRTA1','DMRTA2','DMRTB1','DMRTC2','DMTF1','DNMT1','DNTTIP1','DOT1L','DPF1','DPF3','DPRX','DR1','DRAP1','DRGX','DUX1','DUX3','DUX4','DUXA','DZIP1','E2F1','E2F2','E2F3','E2F4','E2F5','E2F6','E2F7','E2F8','E4F1','EBF1','EBF2','EBF3','EBF4','EEA1','EGR1','EGR2','EGR3','EGR4','EHF','ELF1','ELF2','ELF3','ELF4','ELF5','ELK1','ELK3','ELK4','EMX1','EMX2','EN1','EN2','EOMES','EPAS1','ERF','ERG','ESR1','ESR2','ESRRA','ESRRB','ESRRG','ESX1','ETS1','ETS2','ETV1','ETV2','ETV3','ETV3L','ETV4','ETV5','ETV6','ETV7','EVX1','EVX2','FAM170A','FAM200B','FBXL19','FERD3L','FEV','FEZF1','FEZF2','FIGLA','FIZ1','FLI1','FLYWCH1','FOS','FOSB','FOSL1','FOSL2','FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXN4','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1','GABPA','GATA1','GATA2','GATA3','GATA4','GATA5','GATA6','GATAD2A','GATAD2B','GBX1','GBX2','GCM1','GCM2','GFI1','GFI1B','GLI1','GLI2','GLI3','GLI4','GLIS1','GLIS2','GLIS3','GLMP','GLYR1','GMEB1','GMEB2','GPBP1','GPBP1L1','GRHL1','GRHL2','GRHL3','GSC','GSC2','GSX1','GSX2','GTF2B','GTF2I','GTF2IRD1','GTF2IRD2','GTF2IRD2B','GTF3A','GZF1','HAND1','HAND2','HBP1','HDX','HELT','HES1','HES2','HES3','HES4','HES5','HES6','HES7','HESX1','HEY1','HEY2','HEYL','HHEX','HIC1','HIC2','HIF1A','HIF3A','HINFP','HIVEP1','HIVEP2','HIVEP3','HKR1','HLF','HLX','HMBOX1','HMG20A','HMG20B','HMGA1','HMGA2','HMGN3','HMX1','HMX2','HMX3','HNF1A','HNF1B','HNF4A','HNF4G','HOMEZ','HOXA1','HOXA10','HOXA11','HOXA13','HOXA2','HOXA3','HOXA4','HOXA5','HOXA6','HOXA7','HOXA9','HOXB1','HOXB13','HOXB2','HOXB3','HOXB4','HOXB5','HOXB6','HOXB7','HOXB8','HOXB9','HOXC10','HOXC11','HOXC12','HOXC13','HOXC4','HOXC5','HOXC6','HOXC8','HOXC9','HOXD1','HOXD10','HOXD11','HOXD12','HOXD13','HOXD3','HOXD4','HOXD8','HOXD9','HSF1','HSF2','HSF4','HSF5','HSFX1','HSFX2','HSFY1','HSFY2','IKZF1','IKZF2','IKZF3','IKZF4','IKZF5','INSM1','INSM2','IRF1','IRF2','IRF3','IRF4','IRF5','IRF6','IRF7','IRF8','IRF9','IRX1','IRX2','IRX3','IRX4','IRX5','IRX6','ISL1','ISL2','ISX','JAZF1','JDP2','JRK','JRKL','JUN','JUNB','JUND','KAT7','KCMF1','KCNIP3','KDM2A','KDM2B','KDM5B','KIN','KLF1','KLF10','KLF11','KLF12','KLF13','KLF14','KLF15','KLF16','KLF17','KLF2','KLF3','KLF4','KLF5','KLF6','KLF7','KLF8','KLF9','KMT2A','KMT2B','L3MBTL1','L3MBTL3','L3MBTL4','LBX1','LBX2','LCOR','LCORL','LEF1','LEUTX','LHX1','LHX2','LHX3','LHX4','LHX5','LHX6','LHX8','LHX9','LIN28A','LIN28B','LIN54','LMX1A','LMX1B','LTF','LYL1','MAF','MAFA','MAFB','MAFF','MAFG','MAFK','MAX','MAZ','MBD1','MBD2','MBD3','MBD4','MBD6','MBNL2','MECOM','MECP2','MEF2A','MEF2B','MEF2C','MEF2D','MEIS1','MEIS2','MEIS3','MEOX1','MEOX2','MESP1','MESP2','MGA','MITF','MIXL1','MKX','MLX','MLXIP','MLXIPL','MNT','MNX1','MSANTD1','MSANTD3','MSANTD4','MSC','MSGN1','MSX1','MSX2','MTERF1','MTERF2','MTERF3','MTERF4','MTF1','MTF2','MXD1','MXD3','MXD4','MXI1','MYB','MYBL1','MYBL2','MYC','MYCL','MYCN','MYF5','MYF6','MYNN','MYOD1','MYOG','MYPOP','MYRF','MYRFL','MYSM1','MYT1','MYT1L','MZF1','NACC2','NAIF1','NANOG','NANOGNB','NANOGP8','NCOA1','NCOA2','NCOA3','NEUROD1','NEUROD2','NEUROD4','NEUROD6','NEUROG1','NEUROG2','NEUROG3','NFAT5','NFATC1','NFATC2','NFATC3','NFATC4','NFE2','NFE2L1','NFE2L2','NFE2L3','NFE4','NFIA','NFIB','NFIC','NFIL3','NFIX','NFKB1','NFKB2','NFX1','NFXL1','NFYA','NFYB','NFYC','NHLH1','NHLH2','NKRF','NKX1-1','NKX1-2','NKX2-1','NKX2-2','NKX2-3','NKX2-4','NKX2-5','NKX2-6','NKX2-8','NKX3-1','NKX3-2','NKX6-1','NKX6-2','NKX6-3','NME2','NOBOX','NOTO','NPAS1','NPAS2','NPAS3','NPAS4','NR0B1','NR1D1','NR1D2','NR1H2','NR1H3','NR1H4','NR1I2','NR1I3','NR2C1','NR2C2','NR2E1','NR2E3','NR2F1','NR2F2','NR2F6','NR3C1','NR3C2','NR4A1','NR4A2','NR4A3','NR5A1','NR5A2','NR6A1','NRF1','NRL','OLIG1','OLIG2','OLIG3','ONECUT1','ONECUT2','ONECUT3','OSR1','OSR2','OTP','OTX1','OTX2','OVOL1','OVOL2','OVOL3','PA2G4','PATZ1','PAX1','PAX2','PAX3','PAX4','PAX5','PAX6','PAX7','PAX8','PAX9','PBX1','PBX2','PBX3','PBX4','PCGF2','PCGF6','PDX1','PEG3','PGR','PHF1','PHF19 ','PHF20','PHF21A','PHOX2A','PHOX2B','PIN1','PITX1','PITX2','PITX3','PKNOX1','PKNOX2','PLAG1','PLAGL1','PLAGL2','PLSCR1','POGK','POU1F1','POU2AF1','POU2F1','POU2F2','POU2F3','POU3F1','POU3F2','POU3F3','POU3F4','POU4F1','POU4F2','POU4F3','POU5F1','POU5F1B','POU5F2','POU6F1','POU6F2','PPARA','PPARD','PPARG','PRDM1','PRDM10','PRDM12','PRDM13','PRDM14','PRDM15','PRDM16','PRDM2','PRDM4','PRDM5','PRDM6','PRDM8','PRDM9','PREB','PRMT3','PROP1','PROX1','PROX2','PRR12','PRRX1','PRRX2','PTF1A','PURA','PURB','PURG','RAG1','RARA','RARB','RARG','RAX','RAX2','RBAK','RBCK1','RBPJ','RBPJL','RBSN','REL','RELA','RELB','REPIN1','REST','REXO4','RFX1','RFX2','RFX3','RFX4','RFX5','RFX6','RFX7','RFX8','RHOXF1','RHOXF2','RHOXF2B','RLF','RORA','RORB','RORC','RREB1','RUNX1','RUNX2','RUNX3','RXRA','RXRB','RXRG','SAFB','SAFB2','SALL1','SALL2','SALL3','SALL4','SATB1','SATB2','SCMH1','SCML4','SCRT1','SCRT2','SCX','SEBOX','SETBP1','SETDB1','SETDB2','SGSM2','SHOX','SHOX2','SIM1','SIM2','SIX1','SIX2','SIX3','SIX4','SIX5','SIX6','SKI','SKIL','SKOR1','SKOR2','SLC2A4RG','SMAD1','SMAD3','SMAD4','SMAD5','SMAD9','SMYD3','SNAI1','SNAI2','SNAI3','SNAPC2','SNAPC4','SNAPC5','SOHLH1','SOHLH2','SON','SOX1','SOX10','SOX11']
TFs2=['SOX12','SOX13','SOX14','SOX15','SOX17','SOX18','SOX2','SOX21','SOX3','SOX30','SOX4','SOX5','SOX6','SOX7','SOX8','SOX9','SP1','SP100','SP110','SP140','SP140L','SP2','SP3','SP4','SP5','SP6','SP7','SP8','SP9','SPDEF','SPEN','SPI1','SPIB','SPIC','SPZ1','SRCAP','SREBF1','SREBF2','SRF','SRY','ST18','STAT1','STAT2','STAT3','STAT4','STAT5A','STAT5B','STAT6','T','TAL1','TAL2','TBP','TBPL1','TBPL2','TBR1','TBX1','TBX10','TBX15','TBX18','TBX19','TBX2','TBX20','TBX21','TBX22','TBX3','TBX4','TBX5','TBX6','TCF12','TCF15','TCF20','TCF21','TCF23','TCF24','TCF3','TCF4','TCF7','TCF7L1','TCF7L2','TCFL5','TEAD1','TEAD2','TEAD3','TEAD4','TEF','TERB1','TERF1','TERF2','TET1','TET2','TET3','TFAP2A','TFAP2B','TFAP2C','TFAP2D','TFAP2E','TFAP4','TFCP2','TFCP2L1','TFDP1','TFDP2','TFDP3','TFE3','TFEB','TFEC','TGIF1','TGIF2','TGIF2LX','TGIF2LY','THAP1','THAP10','THAP11','THAP12','THAP2','THAP3','THAP4','THAP5','THAP6','THAP7','THAP8','THAP9','THRA','THRB','THYN1','TIGD1','TIGD2','TIGD3','TIGD4','TIGD5','TIGD6','TIGD7','TLX1','TLX2','TLX3','TMF1','TOPORS','TP53','TP63','TP73','TPRX1','TRAFD1','TRERF1','TRPS1','TSC22D1','TSHZ1','TSHZ2','TSHZ3','TTF1','TWIST1','TWIST2','UBP1','UNCX','USF1','USF2','USF3','VAX1','VAX2','VDR','VENTX','VEZF1','VSX1','VSX2','WIZ','WT1','XBP1','XPA','YBX1','YBX2','YBX3','YY1','YY2','ZBED1','ZBED2','ZBED3','ZBED4','ZBED5','ZBED6','ZBED9','ZBTB1','ZBTB10','ZBTB11','ZBTB12','ZBTB14','ZBTB16','ZBTB17','ZBTB18','ZBTB2','ZBTB20','ZBTB21','ZBTB22','ZBTB24','ZBTB25','ZBTB26','ZBTB3','ZBTB32','ZBTB33','ZBTB34','ZBTB37','ZBTB38','ZBTB39','ZBTB4','ZBTB40','ZBTB41','ZBTB42','ZBTB43','ZBTB44','ZBTB45','ZBTB46','ZBTB47','ZBTB48','ZBTB49','ZBTB5','ZBTB6','ZBTB7A','ZBTB7B','ZBTB7C','ZBTB8A','ZBTB8B','ZBTB9','ZC3H8','ZEB1','ZEB2','ZFAT','ZFHX2','ZFHX3','ZFHX4','ZFP1','ZFP14','ZFP2','ZFP28','ZFP3','ZFP30','ZFP37','ZFP41','ZFP42','ZFP57','ZFP62','ZFP64','ZFP69','ZFP69B','ZFP82','ZFP90','ZFP91','ZFP92','ZFPM1','ZFPM2','ZFX','ZFY','ZGLP1','ZGPAT','ZHX1','ZHX2','ZHX3','ZIC1','ZIC2','ZIC3','ZIC4','ZIC5','ZIK1','ZIM2','ZIM3','ZKSCAN1','ZKSCAN2','ZKSCAN3','ZKSCAN4','ZKSCAN5','ZKSCAN7','ZKSCAN8','ZMAT1','ZMAT4','ZNF10','ZNF100','ZNF101','ZNF107','ZNF112','ZNF114','ZNF117','ZNF12','ZNF121','ZNF124','ZNF131','ZNF132','ZNF133','ZNF134','ZNF135','ZNF136','ZNF138','ZNF14','ZNF140','ZNF141','ZNF142','ZNF143','ZNF146','ZNF148','ZNF154','ZNF155','ZNF157','ZNF16','ZNF160','ZNF165','ZNF169','ZNF17','ZNF174','ZNF175','ZNF177','ZNF18','ZNF180','ZNF181','ZNF182','ZNF184','ZNF189','ZNF19','ZNF195','ZNF197','ZNF2','ZNF20','ZNF200','ZNF202','ZNF205','ZNF207','ZNF208','ZNF211','ZNF212','ZNF213','ZNF214','ZNF215','ZNF217','ZNF219','ZNF22','ZNF221','ZNF222','ZNF223','ZNF224','ZNF225','ZNF226','ZNF227','ZNF229','ZNF23','ZNF230','ZNF232','ZNF233','ZNF234','ZNF235','ZNF236','ZNF239','ZNF24','ZNF248','ZNF25','ZNF250','ZNF251','ZNF253','ZNF254','ZNF256','ZNF257','ZNF26','ZNF260','ZNF263','ZNF264','ZNF266','ZNF267','ZNF268','ZNF273','ZNF274','ZNF275','ZNF276','ZNF277','ZNF28','ZNF280A','ZNF280B','ZNF280C','ZNF280D','ZNF281','ZNF282','ZNF283','ZNF284','ZNF285','ZNF286A','ZNF286B','ZNF287','ZNF292','ZNF296','ZNF3','ZNF30','ZNF300','ZNF302','ZNF304','ZNF311','ZNF316','ZNF317','ZNF318','ZNF319','ZNF32','ZNF320','ZNF322','ZNF324','ZNF324B','ZNF326','ZNF329','ZNF331','ZNF333','ZNF334','ZNF335','ZNF337','ZNF33A','ZNF33B','ZNF34','ZNF341','ZNF343','ZNF345','ZNF346','ZNF347','ZNF35','ZNF350','ZNF354A','ZNF354B','ZNF354C','ZNF358','ZNF362','ZNF365','ZNF366','ZNF367','ZNF37A','ZNF382','ZNF383','ZNF384','ZNF385A','ZNF385B','ZNF385C','ZNF385D','ZNF391','ZNF394','ZNF395','ZNF396','ZNF397','ZNF398','ZNF404','ZNF407','ZNF408','ZNF41','ZNF410','ZNF414','ZNF415','ZNF416','ZNF417','ZNF418','ZNF419','ZNF420','ZNF423','ZNF425','ZNF426','ZNF428','ZNF429','ZNF43','ZNF430','ZNF431','ZNF432','ZNF433','ZNF436','ZNF438','ZNF439','ZNF44','ZNF440','ZNF441','ZNF442','ZNF443','ZNF444','ZNF445','ZNF446','ZNF449','ZNF45','ZNF451','ZNF454','ZNF460','ZNF461','ZNF462','ZNF467','ZNF468','ZNF469','ZNF470','ZNF471','ZNF473','ZNF474','ZNF479','ZNF48','ZNF480','ZNF483','ZNF484','ZNF485','ZNF486','ZNF487','ZNF488','ZNF490','ZNF491','ZNF492','ZNF493','ZNF496','ZNF497','ZNF500','ZNF501','ZNF502','ZNF503','ZNF506','ZNF507','ZNF510','ZNF511','ZNF512','ZNF512B','ZNF513','ZNF514','ZNF516','ZNF517','ZNF518A','ZNF518B','ZNF519','ZNF521','ZNF524','ZNF525','ZNF526','ZNF527','ZNF528','ZNF529','ZNF530','ZNF532','ZNF534','ZNF536','ZNF540','ZNF541','ZNF543','ZNF544','ZNF546','ZNF547','ZNF548','ZNF549','ZNF550','ZNF551','ZNF552','ZNF554','ZNF555','ZNF556','ZNF557','ZNF558','ZNF559','ZNF560','ZNF561','ZNF562','ZNF563','ZNF564','ZNF565','ZNF566','ZNF567','ZNF568','ZNF569','ZNF57','ZNF570','ZNF571','ZNF572','ZNF573','ZNF574','ZNF575','ZNF576','ZNF577','ZNF578','ZNF579','ZNF580','ZNF581','ZNF582','ZNF583','ZNF584','ZNF585A','ZNF585B','ZNF586','ZNF587','ZNF587B','ZNF589','ZNF592','ZNF594','ZNF595','ZNF596','ZNF597','ZNF598','ZNF599','ZNF600','ZNF605','ZNF606','ZNF607','ZNF608','ZNF609','ZNF610','ZNF611','ZNF613','ZNF614','ZNF615','ZNF616','ZNF618','ZNF619','ZNF620','ZNF621','ZNF623','ZNF624','ZNF625','ZNF626','ZNF627','ZNF628','ZNF629','ZNF630','ZNF639','ZNF641','ZNF644','ZNF645','ZNF646','ZNF648','ZNF649','ZNF652','ZNF653','ZNF654','ZNF655','ZNF658','ZNF66','ZNF660','ZNF662','ZNF664','ZNF665','ZNF667','ZNF668','ZNF669','ZNF670','ZNF671','ZNF672','ZNF674','ZNF675','ZNF676','ZNF677','ZNF678','ZNF679','ZNF680','ZNF681','ZNF682','ZNF683','ZNF684','ZNF687','ZNF688','ZNF689','ZNF69','ZNF691','ZNF692','ZNF695','ZNF696','ZNF697','ZNF699','ZNF7','ZNF70','ZNF700','ZNF701','ZNF703','ZNF704','ZNF705A','ZNF705B','ZNF705D','ZNF705E','ZNF705G','ZNF706','ZNF707','ZNF708','ZNF709','ZNF71','ZNF710','ZNF711','ZNF713','ZNF714','ZNF716','ZNF717','ZNF718','ZNF721','ZNF724','ZNF726','ZNF727','ZNF728','ZNF729','ZNF730','ZNF732','ZNF735','ZNF736','ZNF737','ZNF74','ZNF740','ZNF746','ZNF747','ZNF749','ZNF750','ZNF75A','ZNF75D','ZNF76','ZNF761','ZNF763','ZNF764','ZNF765','ZNF766','ZNF768','ZNF77','ZNF770','ZNF771','ZNF772','ZNF773','ZNF774','ZNF775','ZNF776','ZNF777','ZNF778','ZNF780A','ZNF780B','ZNF781','ZNF782','ZNF783','ZNF784','ZNF785','ZNF786','ZNF787','ZNF788','ZNF789','ZNF79','ZNF790','ZNF791','ZNF792','ZNF793','ZNF799','ZNF8','ZNF80','ZNF800','ZNF804A','ZNF804B','ZNF805','ZNF808','ZNF81','ZNF813','ZNF814','ZNF816','ZNF821','ZNF823','ZNF827','ZNF829','ZNF83','ZNF830','ZNF831','ZNF835','ZNF836','ZNF837','ZNF84','ZNF841','ZNF843','ZNF844','ZNF845','ZNF846','ZNF85','ZNF850','ZNF852','ZNF853','ZNF860','ZNF865','ZNF878','ZNF879','ZNF880','ZNF883','ZNF888','ZNF891','ZNF90','ZNF91','ZNF92','ZNF93','ZNF98','ZNF99','ZSCAN1','ZSCAN10','ZSCAN12','ZSCAN16','ZSCAN18','ZSCAN2','ZSCAN20','ZSCAN21','ZSCAN22','ZSCAN23','ZSCAN25','ZSCAN26','ZSCAN29','ZSCAN30','ZSCAN31','ZSCAN32','ZSCAN4','ZSCAN5A','ZSCAN5B','ZSCAN5C','ZSCAN9','ZUFSP','ZXDA','ZXDB','ZXDC','ZZZ3']
TFs=TFs1+TFs2

#TFs other, inclusive of the above list, not necessarily DNA binding proteins but affect transcription
TFs1Other=['AATF','ABL1','ABRA','AC008770.3','AC020915.1','AC023509.3','AC092835.1','AC105001.2','AC138696.1','ACOD1','ACTR3B','ACTR3C','ADAM8','ADAMTS17','ADAMTS19','ADAR','ADGRA1','ADGRG3','ADNP','ADNP2','AEBP1','AEBP2','AES','AFAP1 ','AFF1','AFF3','AFF4','AGER','AGT','AHCTF1','AHDC1','AHR','AHRR','AIM2','AIRE','AKAP8','AKAP8L','AKNA','AKT1','ALK','ALKBH2','ALX1','ALX3','ALX4','AMH','ANAPC2','ANHX','ANKRD30A','ANKZF1','ANXA3','ANXA4','AP5Z1','APBB3','APC2','APEX1','APOA4','APOBEC3A','APOBEC3B','APOBEC3G','APTX','AR','ARGFX','ARHGAP35','ARHGEF2','ARHGEF5','ARID1A','ARID1B','ARID2','ARID3A','ARID3B','ARID3C','ARID4A','ARID4B','ARID5A','ARID5B','ARNT','ARNT2','ARNTL','ARNTL2','ARRB1','ARRB2','ARX','ASCC1','ASCL1','ASCL2','ASCL3','ASCL4','ASCL5','ASH1L','ATF1','ATF2','ATF3','ATF4','ATF5','ATF6','ATF6B','ATF7','ATF7IP','ATM','ATMIN','ATOH1','ATOH7','ATOH8','ATR','ATRX','ATXN7','BACH1','BACH2','BAG1','BANF1','BANP','BARHL1','BARHL2','BARX1','BARX2','BATF','BATF2','BATF3','BAZ2A','BAZ2B','BBX','BCL10','BCL11A','BCL11B','BCL3','BCL6','BCL6B','BCLAF1','BCOR','BDP1','BEND4','BEX1','BHLHA15','BHLHA9','BHLHE22','BHLHE23','BHLHE40','BHLHE41','BHMG1','BIN1','BLM','BLZF1','BMI1','BMP2','BMP4','BMP7','BMPR1A','BNC1','BNC2','BOD1L1','BOLA1','BOLA2','BOLA2B','BOLA3','BORCS8-MEF2B','BPNT1','BPTF','BRCA1','BRD4','BRD7','BRD8','BRD9','BRF1','BRF2','BRMS1','BRPF1','BSX','BTAF1','BTBD8','BTG2','BTK','BTRC','BUD31','C11orf95','C14orf39','C17orf49','C1D','C20orf194','C3orf33','CACTIN','CAMK1D','CAMK2A','CAMTA1','CAMTA2','CAPN15','CAPN3','CARD11','CARD14','CARF','CARHSP1','CASP8AP2','CASZ1','CAT','CBFA2T2','CBFA2T3','CBFB','CBL','CBLL1','CBX2','CC2D1A','CC2D1B','CCDC124','CCDC169-SOHLH2','CCDC17','CCDC3','CCDC71','CCDC83','CCNT1','CCNT2','CD36','CD40','CD40LG','CDC5L','CDC6','CDK5RAP3','CDK7','CDK8','CDK9','CDKN1A','CDKN2A','CDX1','CDX2','CDX4','CEBPA','CEBPB','CEBPD','CEBPE','CEBPG','CEBPZ','CELF3','CENPA','CENPB','CENPBD1','CENPS','CENPT','CENPX','CERS2','CERS3','CERS4','CERS5','CERS6','CFLAR','CGGBP1','CHAMP1','CHCHD3','CHD1','CHD2','CHD3','CHD4','CHD6','CHD7','CHD9','CHP1','CHRAC1','CHUK','CHURC1','CIB1','CIC','CIITA','CIR1','CITED1','CITED2','CITED4','CIZ1','CLOCK','CLU','CMKLR1','CNBP','CNOT3','CNOT4','CNOT7','CNOT8','COMMD1','COMMD6','COMMD7','COPS2','COPS3','COPS4','CPEB1','CPSF4','CPSF4L','CPXCR1','CRAMP1','CREB1','CREB3','CREB3L1','CREB3L2','CREB3L3','CREB3L4','CREB5','CREBBP','CREBL2','CREBRF','CREBZF','CREG1','CREM','CRTC1','CRTC2','CRTC3','CRX','CSDC2','CSDE1','CSRNP1','CSRNP2','CSRNP3','CTBP1','CTBP2','CTCF','CTCFL','CTH','CTNNB1','CTNNBIP1','CUL1','CUL2','CUL3','CUL4A','CUL4B','CUL5','CUX1','CUX2','CXXC1','CXXC4','CXXC5','CYLD','CYP1B1','CYTL1','DAB2IP','DACH1','DACH2','DAP','DAXX','DBP','DBX1','DBX2','DCP1A','DDB1','DDB2','DDIT3','DDN','DDR2','DDRGK1','DDX3X','DDX5','DDX58','DEAF1','DEK','DEPDC1','DEPDC1B','DEPDC4','DEPDC5','DEPDC7','DEPTOR','DHX34','DHX36','DHX57','DISP3','DLX1','DLX2','DLX3','DLX4','DLX5','DLX6','DMAP1','DMBX1','DMC1','DMRT1','DMRT2','DMRT3','DMRTA1','DMRTA2','DMRTB1','DMRTC1','DMRTC2','DMTF1','DNA2','DNAJA3','DNAJC1','DNAJC2','DNAJC21','DNMT1','DNMT3A','DNMT3B','DNMT3L','DNTT','DNTTIP1','DOT1L','DPF1','DPF2','DPF3','DPRX','DR1','DRAP1','DRGX','DSP','DTX1','DUS3L','DUSP12','DUX1','DUX3','DUX4','DUXA','DVL1','DVL2','DVL3','DZIP1','DZIP1L','E2F1','E2F2','E2F3','E2F4','E2F5','E2F6','E2F7','E2F8','E4F1','EAF2','EBF1','EBF2','EBF3','EBF4','EDA','EDA2R','EDF1','EDN1','EEA1','EED','EGLN1','EGR1','EGR2','EGR3','EGR4','EHF','EID1','EID2','EIF2AK2','EIF2AK4','EIF3C','EIF3CL','EIF3K','ELF1','ELF2','ELF3','ELF4','ELF5','ELK1','ELK3','ELK4','ELMSAN1','EME1','EMX1','EMX2','EN1','EN2','ENO1','EOMES','EP300','EP400','EPAS1','EPHA5','EPM2AIP1','ERBIN','ERC1','ERCC3','ERCC4','ERF','ERG','ESR1','ESR2','ESRRA','ESRRB','ESRRG','ESX1','ETS1','ETS2','ETV1','ETV2','ETV3','ETV3L','ETV4','ETV5','ETV6','ETV7','EVX1','EVX2','EWSR1','EXO1','EXOC2','EXOG','EZH1','EZH2','FAAP24','FAM153C','FAM170A','FAM171B','FAM200A','FAM200B','FAN1','FANCA','FANCD2','FANK1','FARSA','FARSB','FBN1','FBXL19','FBXO41','FEN1','FER','FERD3L','FEV','FEZF1','FEZF2','FGD1','FHL2','FIGLA','FIZ1','FLI1','FLNA','FLYWCH1','FMNL2','FOS','FOSB','FOSL1','FOSL2','FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L2','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXN4','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1','FUBP1','FUBP3','FUS','FZD1','FZD2','FZD4','FZD6','GABPA','GABPB1','GABPB2','GADD45A','GAS6','GAS7','GATA1','GATA2','GATA3','GATA4','GATA5','GATA6','GATAD1','GATAD2A','GATAD2B','GBX1','GBX2','GCFC2','GCM1','GCM2','GEN1','GFI1','GFI1B','GLA','GLI1','GLI2','GLI3','GLI4','GLIS1','GLIS2','GLIS3','GLMP','GLYR1','GMEB1','GMEB2','GOLGB1','GON4L','GPATCH8','GPBP1','GPBP1L1','GPER1','GPR155','GREM1','GRHL1','GRHL2','GRHL3','GRM6','GSC','GSC2','GSX1','GSX2','GTF2A1','GTF2A1L','GTF2A2','GTF2B','GTF2E2','GTF2F1','GTF2F2','GTF2H1','GTF2H2','GTF2H2C','GTF2H3','GTF2H4','GTF2I','GTF2IRD1','GTF2IRD2','GTF2IRD2B','GTF3A','GTF3C1','GTF3C2','GULP1','GZF1','H1F0','H1FOO','H1FX','HAND1','HAND2','HAVCR2','HBP1','HCFC1','HCK','HDAC1','HDAC10','HDAC2','HDAC3','HDAC4','HDAC5','HDAC6','HDAC7','HDAC8','HDAC9','HDX','HELT','HELZ','HELZ2','HES1','HES2','HES3','HES4','HES5','HES6','HES7','HESX1','HEY1','HEY2','HEYL','HHEX','HIC1','HIC2','HIF1A','HIF3A','HINFP','HIPK2','HIRA','HIST1H1A','HIST1H1B','HIST1H1C','HIST1H1D','HIST1H1E','HIST1H1T','HIVEP1','HIVEP2','HIVEP3','HKR1','HLA-DQB1','HLA-DQB2','HLA-DRB1','HLA-DRB3','HLA-DRB5','HLF','HLTF','HLX','HMBOX1','HMG20A','HMG20B','HMGA1','HMGA2','HMGB1','HMGB2','HMGB3','HMGB4','HMGN1','HMGN3','HMGXB3','HMGXB4','HMOX1','HMX1','HMX2','HMX3','HNF1A','HNF1B','HNF4A','HNF4G','HNRNPA1','HNRNPA3','HNRNPAB','HNRNPCL2','HNRNPD','HNRNPDL','HNRNPK','HNRNPL','HNRNPU','HOMEZ','HOOK2','HOPX','HOXA1','HOXA10','HOXA11','HOXA13','HOXA2','HOXA3','HOXA4','HOXA5','HOXA6','HOXA7','HOXA9','HOXB1','HOXB13','HOXB2','HOXB3','HOXB4','HOXB5','HOXB6','HOXB7','HOXB8','HOXB9','HOXC10','HOXC11','HOXC12','HOXC13','HOXC4','HOXC5','HOXC6','HOXC8','HOXC9','HOXD1','HOXD10','HOXD11','HOXD12','HOXD13','HOXD3','HOXD4','HOXD8','HOXD9','HP1BP3','HR','HSBP1','HSF1','HSF2','HSF4','HSF5','HSFX1','HSFX2','HSFY1','HSFY2','HSPA1A','HSPA1B','ICAM1','ID1','ID2','ID3','ID4','IFI16','IFNB1','IGHM','IKBKB','IKBKG','IKZF1','IKZF2','IKZF3','IKZF4','IKZF5','IL10','IL1B','IL1RAP','IL4','IL5','IL6','INF2','ING4','INS','INSM1','INSM2','IQGAP1','IRAK1','IRAK2','IRAK3','IRF1','IRF2','IRF2BP1','IRF3','IRF4','IRF5','IRF6','IRF7','IRF8','IRF9','IRX1','IRX2','IRX3','IRX4','IRX5','IRX6','ISL1','ISL2','ISX','ITCH','ITGB2','ITGB3BP','IZUMO2','JADE1','JAK2','JARID2','JAZF1','JDP2','JMY','JRK','JRKL','JUN','JUNB','JUND','JUP','KANSL2','KAT2A','KAT2B','KAT5','KAT6A','KAT6B','KAT7','KAT8','KCMF1','KCNIP3','KDM1A','KDM2A','KDM2B','KDM3A','KDM5A','KDM5B','KDM5C','KDM5D','KEAP1','KIAA1549','KIAA1683','KIAA1841','KIAA1958','KIF15','KIN','KIT','KLF1','KLF10','KLF11','KLF12','KLF13','KLF14','KLF15','KLF16','KLF17','KLF2','KLF3','KLF4','KLF5','KLF6','KLF7','KLF8','KLF9','KMT2A','KMT2B','KMT2C','KMT2D','KNTC1','KRAS','KRBOX1','KRT13','KRTAP5-1','KRTAP5-9','KRTAP9-9','L3MBTL1','L3MBTL3','L3MBTL4','LARP1','LARP1B','LARP4','LARP4B','LARP6','LARP7','LBX1','LBX2','LCOR','LCORL','LEF1','LEUTX','LGALS9','LGR4','LHX1','LHX2','LHX3','LHX4','LHX5','LHX6','LHX8','LHX9','LIG1','LIG3','LIN28A','LIN28B','LIN54','LMO2','LMO4','LMX1A','LMX1B','LPP','LRP5','LRP6','LRP8','LRRFIP1','LRRFIP2','LSR','LTF','LTN1','LYAR','LYL1','LZTFL1','LZTR1','LZTS1','MACC1','MACF1','MAD2L2','MAEL','MAF','MAFA','MAFB','MAFF','MAFG','MAFK','MALT1','MAML1','MAML3','MAP2K5','MAP3K10','MAP3K13','MAP3K7','MAPK1','MAPK10','MAPK11','MAPK14','MAPK3','MAPK8','MAPK9','MATR3','MAVS','MAX','MAZ','MB21D1','MBD1','MBD2','MBD3','MBD4','MBD6','MBNL1','MBNL2','MBNL3','MBTPS2','MDFI','MDM2','MDM4','MECOM','MECP2','MED1','MED12','MED13','MED13L','MED14','MED15','MED16','MED17','MED21','MED23','MED24','MED6','MEF2A','MEF2B','MEF2C','MEF2D','MEIS1','MEIS2','MEIS3','MEN1','MEOX1','MEOX2','MESP1','MESP2','MET','MGA','MGMT','MID2','MIER1','MIER2','MIER3','MINK1','MIS18BP1','MITF','MIXL1','MKL1','MKL2','MKRN1','MKRN2','MKRN3','MKX','MLLT1','MLLT10','MLLT11','MLLT3','MLLT6','MLX','MLXIP','MLXIPL','MN1','MNAT1','MNT','MNX1','MORC3','MPG','MRPL28','MRRF','MSANTD1','MSANTD3','MSANTD4','MSC','MSGN1','MSH2','MSH3','MSH6','MSL3','MSRB2','MST1R','MSX1','MSX2','MTA1','MTA2','MTA3','MTDH','MTERF1','MTERF2','MTERF3','MTERF4','MTF1','MTF2','MTPN','MUS81','MXD1','MXD3','MXD4','MXI1','MYB','MYBBP1A','MYBL1','MYBL2','MYC','MYCBP','MYCL','MYCN','MYD88','MYEF2','MYF5','MYF6','MYNN','MYOCD','MYOD1','MYOG','MYPOP','MYRF','MYRFL','MYSM1','MYT1','MYT1L','MZF1','NAB2','NABP2','NACC2','NAIF1','NANOG','NANOGNB','NANOGP8','NCOA1','NCOA2','NCOA3','NCOA4','NCOA5','NCOA6','NCOA7','NCOR1','NCOR2','NDC80','NDN','NDP','NEB','NEIL1','NEUROD1','NEUROD2','NEUROD4','NEUROD6','NEUROG1','NEUROG2','NEUROG3','NFAM1','NFAT5','NFATC1','NFATC2','NFATC3','NFATC4','NFE2','NFE2L1','NFE2L2','NFE2L3','NFE4','NFIA','NFIB','NFIC','NFIL3','NFIX','NFKB1','NFKB2','NFKBIA','NFKBIB','NFKBID','NFKBIL1','NFKBIZ','NFRKB','NFX1','NFXL1','NFYA','NFYB','NFYC','NHLH1','NHLH2','NKRF','NKX1-1','NKX1-2','NKX2-1','NKX2-2','NKX2-3','NKX2-4','NKX2-5','NKX2-6','NKX2-8','NKX3-1','NKX3-2','NKX6-1','NKX6-2','NKX6-3','NLRC3','NLRC4','NLRC5','NLRP12','NLRP2B','NLRP3','NME2','NOBOX','NOC3L','NOC4L','NOCT','NOD1','NOD2','NODAL','NONO','NOP2','NOTCH1','NOTCH2','NOTCH3','NOTO','NPAS1','NPAS2','NPAS3','NPAS4','NPM1','NR0B1','NR0B2','NR1D1','NR1D2','NR1H2','NR1H3','NR1H4','NR1I2','NR1I3','NR2C1','NR2C2','NR2E1','NR2E3','NR2F1','NR2F2','NR2F6','NR3C1','NR3C2','NR4A1','NR4A2','NR4A3','NR5A1','NR5A2','NR6A1','NRBF2','NRF1','NRG1','NRIP1','NRIP3','NRK','NRL','NSD1','NSD2','NTRK1','NUCKS1','NUFIP1','NUP98','NUPL2']
TFs2Other=['OGG1','OLIG1','OLIG2','OLIG3','ONECUT1','ONECUT2','ONECUT3','OPRD1','OSR1','OSR2','OTOP3','OTP','OTULIN','OTX1','OTX2','OVOL1','OVOL2','OVOL3','PA2G4','PABPN1','PARK7','PARP1','PARP10','PARP12','PATZ1','PAWR','PAX1','PAX2','PAX3','PAX4','PAX5','PAX6','PAX7','PAX8','PAX9','PAXBP1','PAXIP1','PBRM1','PBX1','PBX2','PBX3','PBX4','PCBD1','PCBP1','PCBP2','PCBP3','PCGF2','PCGF6','PCNA','PCSK6','PDCD7','PDS5B','PDX1','PEG3','PELI1','PER1','PER2','PER3','PEX14','PFDN1','PGBD1','PGR','PGS1','PHB','PHB2','PHF1','PHF19 ','PHF20','PHF21A','PHF5A','PHOX2A','PHOX2B','PHTF1','PHTF2','PIAS1','PIAS2','PIAS3','PIAS4','PIDD1','PIKFYVE','PIM1','PIN1','PINK1','PITX1','PITX2','PITX3','PKHD1','PKHD1L1','PKNOX1','PKNOX2','PLA2G10','PLA2G1B','PLAG1','PLAGL1','PLAGL2','PLEK','PLEK2','PLEKHA4','PLPP3','PLSCR1','PLXNA1','PLXNA2','PLXNA3','PLXNA4','PLXNB1','PLXNB2','PLXNB3','PLXNC1','PLXND1','PML','PMS1','PNKP','POGK','POGZ','POLA1','POLB','POLE3','POLE4','POLG','POLH','POLI','POLK','POLL','POLM','POLN','POLQ','POLR1B','POLR2A','POLR2B','POLR2E','POLR2J','POLR2K','POLR2L','POLR3A','POLRMT','POT1','POU1F1','POU2AF1','POU2F1','POU2F2','POU2F3','POU3F1','POU3F2','POU3F3','POU3F4','POU4F1','POU4F2','POU4F3','POU5F1','POU5F1B','POU5F2','POU6F1','POU6F2','PPARA','PPARD','PPARG','PPARGC1A','PPARGC1B','PPP1R10','PPP1R13L','PPP1R21','PPP2R3B','PPP2R5B','PPP3CA','PPRC1','PQBP1','PRAM1','PRAP1','PRB3','PRDM1','PRDM10','PRDM11','PRDM12','PRDM13','PRDM14','PRDM15','PRDM16','PRDM2','PRDM4','PRDM5','PRDM6','PRDM7','PRDM8','PRDM9','PRDX3','PREB','PREX1','PREX2','PRH2','PRIM2','PRIMPOL','PRKCB','PRKCH','PRKCI','PRKCQ','PRKCZ','PRKD1','PRKD2','PRKN','PRMT2','PRMT3','PRNP','PROP1','PROX1','PROX2','PRR12','PRR3','PRRX1','PRRX2','PSMA6','PSMC5','PSMD10','PSMD11','PSMD12','PSPC1','PTCH1','PTEN','PTF1A','PTGIS','PTH','PTTG1','PUM3','PURA','PURB','PURG','PYCARD','PYDC1','PYDC2','QRICH1','RAB11FIP4','RAB7B','RAD21','RAD51','RAG1','RAI1','RAPGEF3','RAPGEF4','RAPGEF5','RARA','RARB','RARG','RAX','RAX2','RB1','RBAK','RBBP7','RBCK1','RBL1','RBL2','RBM10','RBM20','RBM22','RBM26','RBM27','RBM5','RBM6','RBPJ','RBPJL','RBSN','RC3H1','RC3H2','RCAN1','RCOR1','RCOR2','RCOR3','RECQL','REL','RELA','RELB','RELL2','RELN','REPIN1','RERE','REST','REV1','REXO4','RFC1','RFX1','RFX2','RFX3','RFX4','RFX5','RFX6','RFX7','RFX8','RFXANK','RFXAP','RGCC','RGS11','RGS6','RGS7','RGS9','RHEBL1','RHOXF1','RHOXF2','RHOXF2B','RING1','RIOK2','RIPK1','RIPK2','RIPK3','RIPK4','RLF','RLIM','RNASE2','RNASEH1','RNF113A','RNF113B','RNF114','RNF125','RNF138','RNF14','RNF166','RNF2','RNF219','RNF25','RNF31','RNF4','RNF41','RORA','RORB','RORC','RPA2','RPA4','RPL36A','RPL7','RPL7A','RPS27A','RPS6KA4','RPS6KA5','RREB1','RRN3','RUNX1','RUNX1T1','RUNX2','RUNX3','RWDD3','RXRA','RXRB','RXRG','RYBP','S100A12','S100A8','S100A9','SAFB','SAFB2','SALL1','SALL2','SALL3','SALL4','SART1','SATB1','SATB2','SAV1','SCAND1','SCAND2P','SCAPER','SCMH1','SCML1','SCML2','SCML4','SCOC','SCRT1','SCRT2','SCX','SEBOX','SEMA4A','SERTAD1','SERTAD2','SETBP1','SETD6','SETDB1','SETDB2','SF3A2','SF3A3','SFPQ','SFRP4','SFRP5','SFTPB','SGK1','SGK3','SGSM2','SHH','SHOX','SHOX2','SHPRH','SIGIRR','SIK1','SIM1','SIM2','SIN3A','SIN3B','SIRT1','SIRT2','SIRT3','SIRT6','SIVA1','SIX1','SIX2','SIX3','SIX4','SIX5','SIX6','SKI','SKIDA1','SKIL','SKOR1','SKOR2','SLC22A4','SLC26A10','SLC26A3','SLC2A4RG','SLC30A9','SLC39A10','SLC4A10','SMAD1','SMAD2','SMAD3','SMAD4','SMAD5','SMAD6','SMAD7','SMAD9','SMARCA1','SMARCA2','SMARCA4','SMARCA5','SMARCAL1','SMARCB1','SMARCC1','SMARCC2','SMARCD1','SMARCD2','SMARCD3','SMARCE1','SMC3','SMO','SMUG1','SMYD1','SMYD3','SNAI1','SNAI2','SNAI3','SNAPC2','SNAPC4','SNAPC5','SNRPC','SNURF','SNW1','SOGA3','SOHLH1','SOHLH2','SON','SORBS2','SOX1','SOX10','SOX11','SOX12','SOX13','SOX14','SOX15','SOX17','SOX18','SOX2','SOX21','SOX3','SOX30','SOX4','SOX5','SOX6','SOX7','SOX8','SOX9','SP1','SP100','SP110','SP140','SP140L','SP2','SP3','SP4','SP5','SP6','SP7','SP8','SP9','SPDEF','SPEN','SPHK1','SPI1','SPIB','SPIC','SPZ1','SRCAP','SREBF1','SREBF2','SRF','SRY','SSB','SSBP2','SSBP3','SSBP4','SSH1','SSH2','SSH3','SSRP1','SSX1','ST18','STAG1','STAT1','STAT2','STAT3','STAT4','STAT5A','STAT5B','STAT6','STK16','STK3','STK36','STRN','STRN3','SUB1','SUFU','SUMO1','SUPT16H','SUPT4H1','SUPT5H','SUPT6H','SUV39H1','SUV39H2','SUZ12','SYK','T','TAB1','TAB2','TAB3','TADA2A','TADA2B','TADA3','TAF1','TAF10','TAF11','TAF12','TAF13','TAF1A','TAF1B','TAF1C','TAF1L','TAF2','TAF3','TAF4','TAF4B','TAF5','TAF5L','TAF6','TAF6L','TAF7','TAF7L','TAF8','TAF9','TAF9B','TAL1','TAL2','TARDBP','TAX1BP1','TBL1Y','TBP','TBPL1','TBPL2','TBR1','TBX1','TBX10','TBX15','TBX18','TBX19','TBX2','TBX20','TBX21','TBX22','TBX3','TBX4','TBX5','TBX6','TCEA2','TCEAL1','TCEAL7','TCEAL8','TCF12','TCF15','TCF19','TCF20','TCF21','TCF23','TCF24','TCF25','TCF3','TCF4','TCF7','TCF7L1','TCF7L2','TCFL5','TDG','TDP1','TDP2','TEAD1','TEAD2','TEAD3','TEAD4','TEF','TERB1','TERF1','TERF2','TERF2IP','TESMIN','TET1','TET2','TET3','TFAM','TFAP2A','TFAP2B','TFAP2C','TFAP2D','TFAP2E','TFAP4','TFCP2','TFCP2L1','TFDP1','TFDP2','TFDP3','TFE3','TFEB','TFEC','TGFB1','TGFB1I1','TGIF1','TGIF2','TGIF2LX','TGIF2LY','TGS1','THAP1','THAP10','THAP11','THAP12','THAP2','THAP3','THAP4','THAP5','THAP6','THAP7','THAP8','THAP9','THRA','THRAP3','THRB','THYN1','TIA1','TICAM1','TIGD1','TIGD2','TIGD3','TIGD4','TIGD5','TIGD6','TIGD7','TIPARP','TIRAP','TLE1','TLE2','TLE3','TLE4','TLE6','TLR2','TLR3','TLR4','TLR9','TLX1','TLX2','TLX3','TMEM235','TMF1','TNF','TNFAIP3','TNFRSF11A','TNFRSF4','TNFSF11','TNFSF18','TNFSF4','TOE1','TONSL','TOP1','TOP2A','TOP2B','TOPORS','TOX','TOX2','TOX3','TOX4','TP53','TP53BP1','TP63','TP73','TPRX1','TRADD','TRAF1','TRAF2','TRAF3','TRAF4','TRAF5','TRAF6','TRAFD1','TRAK2','TRAPPC9','TRERF1','TREX1','TRIB1','TRIB3','TRIM13','TRIM14','TRIM15','TRIM21','TRIM22','TRIM23','TRIM24','TRIM25','TRIM26','TRIM27','TRIM28','TRIM29','TRIM3','TRIM31','TRIM32','TRIM33','TRIM34','TRIM37','TRIM38','TRIM40','TRIM5','TRIM52','TRIM62','TRIM8','TRIP11','TRIT1','TRMT1','TRMT1L','TRPS1','TRRAP','TSC2','TSC22D1','TSC22D2','TSC22D3','TSC22D4','TSHZ1','TSHZ2','TSHZ3','TSN','TSSK4','TTF1','TUB','TULP4','TUT1','TWIST1','TWIST2','TXK','U2AF1','U2AF1L4','UBA52','UBB','UBC','UBE2K','UBE2N','UBE2V1','UBN1','UBP1','UBR4','UBTF','UBTFL1','UBTFL6','UFL1','UHRF1','UHRF2','UNCX','UNG','UNK','UNKL','USF1','USF2','USF3','USP39','USP7','UTF1','VAV1','VAX1','VAX2','VDR','VEGFA','VENTX','VEZF1','VGLL1','VGLL2','VGLL3','VHLL','VPS72','VSX1','VSX2','WDHD1','WFS1','WIZ','WNT1','WNT10B','WNT2','WNT3A','WNT5A','WNT8B','WRN','WT1','WWOX','WWP2','WWTR1','XBP1','XCL1','XPA','YBX1','YBX2','YBX3','YEATS2','YEATS4','YOD1','YY1','YY2','ZBED1','ZBED2','ZBED3','ZBED4','ZBED5','ZBED6','ZBED6CL','ZBED8','ZBED9','ZBP1','ZBTB1','ZBTB10','ZBTB11','ZBTB12','ZBTB14','ZBTB16','ZBTB17','ZBTB18','ZBTB2','ZBTB20','ZBTB21','ZBTB22','ZBTB24','ZBTB25','ZBTB26','ZBTB3','ZBTB32','ZBTB33','ZBTB34','ZBTB37','ZBTB38','ZBTB39','ZBTB4','ZBTB40','ZBTB41','ZBTB42','ZBTB43','ZBTB44','ZBTB45','ZBTB46','ZBTB47','ZBTB48','ZBTB49','ZBTB5','ZBTB6','ZBTB7A','ZBTB7B','ZBTB7C','ZBTB8A','ZBTB8B','ZBTB9','ZC3H10','ZC3H11A','ZC3H12A','ZC3H13','ZC3H14','ZC3H15','ZC3H18','ZC3H3','ZC3H4','ZC3H6','ZC3H7A','ZC3H7B','ZC3H8','ZC3HAV1','ZCCHC11','ZCCHC6','ZDHHC11','ZDHHC19','ZEB1','ZEB2','ZFAND5','ZFAT','ZFHX2','ZFHX3','ZFHX4','ZFP1','ZFP14','ZFP2','ZFP28','ZFP3','ZFP30','ZFP36','ZFP36L1','ZFP36L2','ZFP37','ZFP41','ZFP42','ZFP57','ZFP62','ZFP64','ZFP69','ZFP69B','ZFP82','ZFP90','ZFP91','ZFP92','ZFPL1','ZFPM1','ZFPM2','ZFR','ZFR2','ZFX','ZFY','ZFYVE26','ZGLP1','ZGPAT','ZHX1','ZHX2','ZHX3','ZIC1','ZIC2','ZIC3','ZIC4','ZIC5','ZIK1','ZIM2','ZIM3','ZKSCAN1','ZKSCAN2','ZKSCAN3','ZKSCAN4','ZKSCAN5','ZKSCAN7','ZKSCAN8','ZMAT1','ZMAT2','ZMAT3','ZMAT4','ZMAT5','ZMYM1','ZMYM2','ZMYM3','ZMYM4','ZMYM5','ZMYM6','ZMYND10','ZNF10','ZNF100','ZNF101','ZNF106','ZNF107','ZNF112','ZNF114','ZNF117','ZNF12','ZNF121','ZNF124','ZNF131','ZNF132','ZNF133','ZNF134','ZNF135','ZNF136','ZNF137P','ZNF138','ZNF14','ZNF140','ZNF141','ZNF142','ZNF143','ZNF146','ZNF148','ZNF154','ZNF155','ZNF157','ZNF16','ZNF160','ZNF165','ZNF169','ZNF17','ZNF174','ZNF175','ZNF177','ZNF18','ZNF180','ZNF181','ZNF182','ZNF184','ZNF185','ZNF189','ZNF19','ZNF195','ZNF197','ZNF2','ZNF20','ZNF200','ZNF202','ZNF205','ZNF207','ZNF208','ZNF211','ZNF212','ZNF213','ZNF214','ZNF215','ZNF217','ZNF219','ZNF22','ZNF221','ZNF222','ZNF223','ZNF224','ZNF225','ZNF226','ZNF227','ZNF229','ZNF23','ZNF230','ZNF232','ZNF233','ZNF234','ZNF235','ZNF236','ZNF239','ZNF24','ZNF248','ZNF25','ZNF250','ZNF251','ZNF253','ZNF254','ZNF256','ZNF257','ZNF26','ZNF260','ZNF263','ZNF264','ZNF266','ZNF267','ZNF268','ZNF273','ZNF274','ZNF275','ZNF276','ZNF277','ZNF28','ZNF280A','ZNF280B','ZNF280C','ZNF280D','ZNF281','ZNF282','ZNF283','ZNF284','ZNF285','ZNF286A','ZNF286B','ZNF287','ZNF292','ZNF296','ZNF3','ZNF30','ZNF300','ZNF302','ZNF304','ZNF311','ZNF316','ZNF317','ZNF318','ZNF319','ZNF32','ZNF320','ZNF322','ZNF324','ZNF324B','ZNF326','ZNF329','ZNF330','ZNF331','ZNF333','ZNF334','ZNF335','ZNF337','ZNF33A','ZNF33B','ZNF34','ZNF341','ZNF343','ZNF345','ZNF346','ZNF347','ZNF35','ZNF350','ZNF354A','ZNF354B','ZNF354C','ZNF355P','ZNF358','ZNF362','ZNF365','ZNF366','ZNF367','ZNF37A','ZNF382','ZNF383','ZNF384','ZNF385A','ZNF385B','ZNF385C','ZNF385D','ZNF391','ZNF394','ZNF395','ZNF396','ZNF397','ZNF398','ZNF404','ZNF407','ZNF408','ZNF41','ZNF410','ZNF414','ZNF415','ZNF416','ZNF417','ZNF418','ZNF419','ZNF420','ZNF423','ZNF425','ZNF426','ZNF428','ZNF429','ZNF43','ZNF430','ZNF431','ZNF432','ZNF433','ZNF436','ZNF438','ZNF439','ZNF44','ZNF440','ZNF441','ZNF442','ZNF443','ZNF444','ZNF445','ZNF446','ZNF449','ZNF45','ZNF451','ZNF454','ZNF460','ZNF461','ZNF462','ZNF467','ZNF468','ZNF469','ZNF470','ZNF471','ZNF473','ZNF474','ZNF479','ZNF48','ZNF480','ZNF483','ZNF484','ZNF485','ZNF486','ZNF487','ZNF488','ZNF490','ZNF491','ZNF492','ZNF493','ZNF496','ZNF497','ZNF500','ZNF501','ZNF502','ZNF503','ZNF506','ZNF507','ZNF510','ZNF511','ZNF512','ZNF512B','ZNF513','ZNF514','ZNF516','ZNF517','ZNF518A','ZNF518B','ZNF519']
TFs3Other=['ZNF521','ZNF524','ZNF525','ZNF526','ZNF527','ZNF528','ZNF529','ZNF530','ZNF532','ZNF534','ZNF536','ZNF540','ZNF541','ZNF542P','ZNF543','ZNF544','ZNF546','ZNF547','ZNF548','ZNF549','ZNF550','ZNF551','ZNF552','ZNF554','ZNF555','ZNF556','ZNF557','ZNF558','ZNF559','ZNF560','ZNF561','ZNF562','ZNF563','ZNF564','ZNF565','ZNF566','ZNF567','ZNF568','ZNF569','ZNF57','ZNF570','ZNF571','ZNF572','ZNF573','ZNF574','ZNF575','ZNF576','ZNF577','ZNF578','ZNF579','ZNF580','ZNF581','ZNF582','ZNF583','ZNF584','ZNF585A','ZNF585B','ZNF586','ZNF587','ZNF587B','ZNF589','ZNF592','ZNF593','ZNF594','ZNF595','ZNF596','ZNF597','ZNF598','ZNF599','ZNF600','ZNF605','ZNF606','ZNF607','ZNF608','ZNF609','ZNF610','ZNF611','ZNF613','ZNF614','ZNF615','ZNF616','ZNF618','ZNF619','ZNF620','ZNF621','ZNF622','ZNF623','ZNF624','ZNF625','ZNF626','ZNF627','ZNF628','ZNF629','ZNF630','ZNF638','ZNF639','ZNF641','ZNF644','ZNF645','ZNF646','ZNF648','ZNF649','ZNF652','ZNF653','ZNF654','ZNF655','ZNF658','ZNF66','ZNF660','ZNF662','ZNF664','ZNF665','ZNF667','ZNF668','ZNF669','ZNF670','ZNF671','ZNF672','ZNF674','ZNF675','ZNF676','ZNF677','ZNF678','ZNF679','ZNF680','ZNF681','ZNF682','ZNF683','ZNF684','ZNF687','ZNF688','ZNF689','ZNF69','ZNF691','ZNF692','ZNF695','ZNF696','ZNF697','ZNF699','ZNF7','ZNF70','ZNF700','ZNF701','ZNF703','ZNF704','ZNF705A','ZNF705B','ZNF705D','ZNF705E','ZNF705G','ZNF706','ZNF707','ZNF708','ZNF709','ZNF71','ZNF710','ZNF711','ZNF713','ZNF714','ZNF716','ZNF717','ZNF718','ZNF720','ZNF721','ZNF724','ZNF726','ZNF727','ZNF728','ZNF729','ZNF73','ZNF730','ZNF732','ZNF735','ZNF736','ZNF737','ZNF738','ZNF74','ZNF740','ZNF746','ZNF747','ZNF749','ZNF750','ZNF75A','ZNF75D','ZNF76','ZNF761','ZNF763','ZNF764','ZNF765','ZNF766','ZNF768','ZNF77','ZNF770','ZNF771','ZNF772','ZNF773','ZNF774','ZNF775','ZNF776','ZNF777','ZNF778','ZNF780A','ZNF780B','ZNF781','ZNF782','ZNF783','ZNF784','ZNF785','ZNF786','ZNF787','ZNF788','ZNF789','ZNF79','ZNF790','ZNF791','ZNF792','ZNF793','ZNF799','ZNF8','ZNF80','ZNF800','ZNF804A','ZNF804B','ZNF805','ZNF808','ZNF81','ZNF812P','ZNF813','ZNF814','ZNF816','ZNF821','ZNF823','ZNF827','ZNF829','ZNF83','ZNF830','ZNF831','ZNF835','ZNF836','ZNF837','ZNF839','ZNF84','ZNF841','ZNF843','ZNF844','ZNF845','ZNF846','ZNF85','ZNF850','ZNF852','ZNF853','ZNF860','ZNF862','ZNF865','ZNF878','ZNF879','ZNF880','ZNF883','ZNF888','ZNF891','ZNF90','ZNF91','ZNF92','ZNF93','ZNF98','ZNF99','ZNFX1','ZRANB2','ZRSR1','ZRSR2','ZSCAN1','ZSCAN10','ZSCAN12','ZSCAN16','ZSCAN18','ZSCAN2','ZSCAN20','ZSCAN21','ZSCAN22','ZSCAN23','ZSCAN25','ZSCAN26','ZSCAN29','ZSCAN30','ZSCAN31','ZSCAN32','ZSCAN4','ZSCAN5A','ZSCAN5B','ZSCAN5C','ZSCAN5DP','ZSCAN9','ZUFSP','ZXDA','ZXDB','ZXDC','ZZZ3']
TFs4Other=['ATF2','ATF6B','ATN1','BHLHA15','BHLHB2','BHLHB4 ','BHLHB5','BHLHB9','C13ORF8','C20ORF194','CDX2','CRAMP1L','ELP3','ERVK3-1','ETF1','FLJ12895','FLJ20321','FLJ25476','FLJ35784','FOXM1','FOXO2','FOXR1','FOXR2','GATA1','GRLF1','KDM4C','KDM4E','KIAA0863','KIAA0961','KLRG1','LBXCOR1','LEF1','LHX7','LOC126295','LOC51058','LOC91661','MGC4728','MYOD1','MYOG','MYRF','NEUROD4','NEUROG1','NFAT2','NFAT4','NFE2L1','NR1A2','NR1B1','NR1B2','NR1B3','NR1C1','NR1C2','NR1C3','NR1F3','NR1I1','NR2A1','NR2B3','NR3A1','NR3A2','NR3B3','OTEX','PEPP-2','PHF2','POU2F1','POU2F2','POU4F3','POU5F1','POU6F1','PRDM3','PRKRIP1','PRKRIR','SETD1A','SMAD8','SMYD2','SOX12','SOX14','SOX15','TBX14','TCF1','TCF17','TCF2','TCF9','TEAD1','TPRXL','UBTFL1','WHSC1','ZBTB14','ZBTB18','ZBTB19','ZBTB21','ZBTB27','ZBTB28','ZBTB35','ZBTB49','ZFP161','ZFP28','ZKSCAN10','ZKSCAN11','ZKSCAN12','ZKSCAN13','ZKSCAN14','ZKSCAN16','ZKSCAN19','ZKSCAN21','ZKSCAN24','ZKSCAN6','ZKSCAN8','ZNF10','ZNF112','ZNF187','ZNF193','ZNF204P','ZNF238','ZNF271','ZNF295','ZNF305','ZNF323','ZNF434','ZNF499','ZNF542','ZNF545','ZNF553','ZNF56','ZNF625','ZNF643','ZNF698','ZNF705F','ZNF751','ZNF752','ZNF753','ZNF785','ZNF806','ZNF816','ZNF833P','ZNF906','ZSCAN1','ZSCAN25','ZSCAN26','ZSCAN31','ZSCAN32','ZSCAN5D','ZSCAN9']
TFsOther=TFs1Other+TFs2Other+TFs3Other+TFs4Other
```

Example 15 (unknown):
```unknown
S genes: ['MCM5', 'PCNA', 'TYMS', 'FEN1', 'MCM2', 'MCM4', 'RRM1', 'UNG', 'GINS2', 'MCM6', 'CDCA7', 'DTL', 'PRIM1', 'UHRF1', 'HELLS', 'RFC2', 'RPA2', 'NASP', 'RAD51AP1', 'GMNN', 'WDR76', 'SLBP', 'CCNE2', 'UBR7', 'POLD3', 'MSH2', 'ATAD2', 'RAD51', 'RRM2', 'CDC45', 'CDC6', 'EXO1', 'TIPIN', 'DSCC1', 'BLM', 'CASP8AP2', 'USP1', 'CLSPN', 'POLA1', 'CHAF1B', 'BRIP1', 'E2F8'] 

G2 genes: ['HMGB2', 'CDK1', 'NUSAP1', 'UBE2C', 'BIRC5', 'TPX2', 'TOP2A', 'NDC80', 'CKS2', 'NUF2', 'CKS1B', 'MKI67', 'TMPO', 'CENPF', 'TACC3', 'SMC4', 'CCNB2', 'CKAP2L', 'CKAP2', 'AURKB', 'BUB1', 'KIF11', 'ANP32E', 'TUBB4B', 'GTSE1', 'KIF20B', 'HJURP', 'CDCA3', 'CDC20', 'TTK', 'CDC25C', 'KIF2C', 'RANGAP1', 'NCAPD2', 'DLGAP5', 'CDCA2', 'CDCA8', 'ECT2', 'KIF23', 'HMMR', 'AURKA', 'PSRC1', 'ANLN', 'LBR', 'CKAP5', 'CENPE', 'CTCF', 'NEK2', 'G2E3', 'GAS2L3', 'CBX5', 'CENPA'] 

Hemoglobin genes set 1: ['HBB', 'HBD', 'HBG1', 'HBG2', 'HBE1', 'HBZ', 'HBM', 'HBA2', 'HBA1', 'HBQ1'] 

Hemoglobin genes set 2: ['AHSP', 'GYPA', 'RHAG', 'GFI1B', 'KEL', 'EPB42', 'ALAS2', 'TSPO2', 'GYPB', 'GYPE', 'SPTA1', 'RHCE', 'KLF1', 'RHD', 'SLC4A1', 'SNX22', 'SQSTM1', 'SOD1', 'FTH1', 'MAP1LC3B', 'SLC25A37', 'TRIM58', 'HMBS'] 

Mitochondrial genes: ['MT-ND1', 'MT-ND2', 'MT-CO1', 'MT-CO2', 'MT-ATP8', 'MT-ATP6', 'MT-CO3', 'MT-ND3', 'MT-ND4L', 'MT-ND4', 'MT-ND5', 'MT-ND6', 'MT-CYB'] 

Ribo genes: ['RPLP0', 'RPLP1', 'RPLP2', 'RPL3', 'RPL3L', 'RPL4', 'RPL5', 'RPL6', 'RPL7', 'RPL7A', 'RPL7L1', 'RPL8', 'RPL9', 'RPL10', 'RPL10A', 'RPL10L', 'RPL11', 'RPL12', 'RPL13', 'RPL13A', 'RPL14', 'RPL15', 'RPL17', 'RPL18', 'RPL18A', 'RPL19', 'RPL21', 'RPL22', 'RPL22L1', 'RPL23', 'RPL23A', 'RPL24', 'RPL26', 'RPL26L1', 'RPL27', 'RPL27A', 'RPL28', 'RPL29', 'RPL30', 'RPL31', 'RPL32', 'RPL34', 'RPL35', 'RPL35A', 'RPL36', 'RPL36A', 'RPL36AL', 'RPL37', 'RPL37A', 'RPL38', 'RPL39', 'RPL39L', 'UBA52', 'RPL41', 'RPSA', 'RPS2', 'RPS3', 'RPS3A', 'RPS4X', 'RPS4Y1', 'RPS4Y2', 'RPS5', 'RPS6', 'RPS7', 'RPS8', 'RPS9', 'RPS10', 'RPS11', 'RPS12', 'RPS13', 'RPS14', 'RPS15', 'RPS15A', 'RPS16', 'RPS17', 'RPS18', 'RPS19', 'RPS20', 'RPS21', 'RPS23', 'RPS24', 'RPS25', 'RPS26', 'RPS27', 'RPS27A', 'RPS27L', 'RPS28', 'RPS29', 'FAU'] 

Tissue dissociation stress associated genes: ['FOS', 'FOSB', 'JUN', 'JUNB', 'JUND', 'ZFP36', 'SOCS3', 'CEBPD', 'CEBPB', 'ATF3', 'EGR1', 'HSPA1A', 'HSPA1B', 'HSP90AA1', 'HSP90AB1', 'HSPA8', 'HSPB1', 'HSPE1', 'IER3', 'IER2', 'BTG1', 'BTG2', 'DUSP1']
```

Example 16 (python):
```python
adata=sc.read('./Objects/Feb28ObjectRaw.h5ad')
adata = adata[adata.obs['fine_grain'].isin(['CoronaryArterialEndothelialCells','CoronaryCapillaryEndothelialCells','CoronaryVenousEndothelialCells'])].copy()
print(adata.shape)
```

Example 17 (unknown):
```unknown
(8375, 36601)
```

Example 18 (python):
```python
# preserve raw count
adata.layers["counts"] = adata.X.copy()

# normalise and log-transform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key='batch_key')

print(sum(adata.var.highly_variable))
sc.pl.highly_variable_genes(adata)

# freeze the state in `.raw`
adata.raw = adata

# subset hvgs
adata = adata[:, adata.var.highly_variable]

adata=adata.copy()

sc.tl.score_genes_cell_cycle(adata, s_genes=s_genes, g2m_genes=g2m_genes)
#sc.pp.regress_out(adata, ['S_score','G2M_score'])

#batch correction
scvi.model.SCVI.setup_anndata(adata, 
                              layer="counts", 
                              batch_key="batch_key",
                              categorical_covariate_keys=['phase'],
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo'])

vae = scvi.model.SCVI(adata, n_hidden = 128, n_latent = 50, n_layers = 3, dispersion = 'gene-batch')
vae.train(max_epochs=50)
    
# Convergence plot
x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
plt.plot(x, vae.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
plt.clf()

# add the latent space to adata
adata.obsm["X_scVI"] = vae.get_latent_representation()

sc.tl.score_genes(adata, stress_genes, score_name='stress_score')
sc.tl.score_genes(adata, hb_genes1, score_name='hb1_score')

sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata,min_dist=0.1)
#sc.tl.tsne(adata)
sc.pl.umap(adata, color=['log1p_total_counts','n_genes','n_genes_by_counts','total_counts','pct_counts_mt','pct_counts_ribo'],ncols=5)
sc.pl.umap(adata, color=['scrublet_score','stress_score', 'hb1_score','phase'], ncols=5,)
sc.pl.umap(adata, color=['donor','age','week','trimester','region','phase','cycling','cell_or_nuclei','modality','kit_10x','FACSgate','heart_or_greatvessels','coarse_grain','mid_grain','fine_grain'],wspace=0.8,ncols=5)
```

Example 19 (python):
```python
normalizing counts per cell
    finished (0:00:00)
extracting highly variable genes
    finished (0:00:08)
--> added
    'highly_variable', boolean vector (adata.var)
    'means', float vector (adata.var)
    'dispersions', float vector (adata.var)
    'dispersions_norm', float vector (adata.var)
1332
```

Example 20 (python):
```python
calculating cell cycle phase
computing score 'S_score'
WARNING: genes are not in var_names and ignored: ['MCM5', 'FEN1', 'MCM2', 'MCM4', 'UNG', 'GINS2', 'MCM6', 'CDCA7', 'PRIM1', 'UHRF1', 'RFC2', 'RPA2', 'NASP', 'GMNN', 'SLBP', 'CCNE2', 'UBR7', 'POLD3', 'MSH2', 'RAD51', 'CDC45', 'CDC6', 'EXO1', 'TIPIN', 'DSCC1', 'CASP8AP2', 'USP1', 'POLA1', 'CHAF1B', 'E2F8']
    finished: added
    'S_score', score of gene set (adata.obs).
    329 total control genes are used. (0:00:00)
computing score 'G2M_score'
WARNING: genes are not in var_names and ignored: ['TMPO', 'CKAP2L', 'ANP32E', 'HJURP', 'CDCA3', 'NCAPD2', 'CDCA2', 'PSRC1', 'LBR', 'CKAP5', 'CTCF', 'NEK2', 'G2E3', 'CBX5']
    finished: added
    'G2M_score', score of gene set (adata.obs).
    729 total control genes are used. (0:00:00)
-->     'phase', cell cycle phase (adata.obs)
```

Example 21 (python):
```python
/home/sinhalab/miniconda3/envs/scanpynew/lib/python3.8/site-packages/torchmetrics/utilities/prints.py:36: UserWarning: Torchmetrics v0.9 introduced a new argument class property called `full_state_update` that has
                not been set for this class (ElboMetric). The property determines if `update` by
                default needs access to the full metric state. If this is not the case, significant speedups can be
                achieved and we recommend setting this to `False`.
                We provide an checking function
                `from torchmetrics.utilities import check_forward_no_full_state`
                that can be used to check if the `full_state_update=True` (old and potential slower behaviour,
                default for now) or if `full_state_update=False` can be used safely.
                
  warnings.warn(*args, **kwargs)
GPU available: False, used: False
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
```

Example 22 (unknown):
```unknown
Epoch 1/50:   0%|                                        | 0/50 [00:00<?, ?it/s]
```

Example 23 (python):
```python
/home/sinhalab/miniconda3/envs/scanpynew/lib/python3.8/site-packages/torchmetrics/utilities/prints.py:36: UserWarning: Torchmetrics v0.9 introduced a new argument class property called `full_state_update` that has
                not been set for this class (ResultMetric). The property determines if `update` by
                default needs access to the full metric state. If this is not the case, significant speedups can be
                achieved and we recommend setting this to `False`.
                We provide an checking function
                `from torchmetrics.utilities import check_forward_no_full_state`
                that can be used to check if the `full_state_update=True` (old and potential slower behaviour,
                default for now) or if `full_state_update=False` can be used safely.
                
  warnings.warn(*args, **kwargs)
```

Example 24 (unknown):
```unknown
Epoch 50/50: 100%|███████████| 50/50 [00:30<00:00,  1.66it/s, loss=676, v_num=1]
```

Example 25 (python):
```python
computing score 'stress_score'
WARNING: genes are not in var_names and ignored: ['JUND', 'HSP90AA1', 'HSP90AB1', 'HSPA8', 'HSPB1', 'HSPE1', 'BTG1']
    finished: added
    'stress_score', score of gene set (adata.obs).
    479 total control genes are used. (0:00:00)
computing score 'hb1_score'
WARNING: genes are not in var_names and ignored: ['HBD', 'HBE1', 'HBM', 'HBQ1']
    finished: added
    'hb1_score', score of gene set (adata.obs).
    244 total control genes are used. (0:00:00)
computing neighbors
```

Example 26 (python):
```python
/home/sinhalab/miniconda3/envs/scanpynew/lib/python3.8/site-packages/numba/core/typed_passes.py:313: NumbaPerformanceWarning: 
The keyword argument 'parallel=True' was specified but no transformation for parallel execution was possible.

To find out why, try turning on parallel diagnostics, see https://numba.pydata.org/numba-doc/latest/user/parallel.html#diagnostics for help.

File "../../miniconda3/envs/scanpynew/lib/python3.8/site-packages/umap/rp_tree.py", line 135:
@numba.njit(fastmath=True, nogil=True, parallel=True)
def euclidean_random_projection_split(data, indices, rng_state):
^

  warnings.warn(errors.NumbaPerformanceWarning(msg,
/home/sinhalab/miniconda3/envs/scanpynew/lib/python3.8/site-packages/umap/nndescent.py:91: NumbaPerformanceWarning: 
The keyword argument 'parallel=True' was specified but no transformation for parallel execution was possible.

To find out why, try turning on parallel diagnostics, see https://numba.pydata.org/numba-doc/latest/user/parallel.html#diagnostics for help.

File "../../miniconda3/envs/scanpynew/lib/python3.8/site-packages/umap/utils.py", line 409:
@numba.njit(parallel=True)
def build_candidates(current_graph, n_vertices, n_neighbors, max_candidates, rng_state):
^

  candidate_neighbors = build_candidates(
/home/sinhalab/miniconda3/envs/scanpynew/lib/python3.8/site-packages/numba/core/typed_passes.py:313: NumbaPerformanceWarning: 
The keyword argument 'parallel=True' was specified but no transformation for parallel execution was possible.

To find out why, try turning on parallel diagnostics, see https://numba.pydata.org/numba-doc/latest/user/parallel.html#diagnostics for help.

File "../../miniconda3/envs/scanpynew/lib/python3.8/site-packages/umap/nndescent.py", line 47:
    @numba.njit(parallel=True)
    def nn_descent(
    ^

  warnings.warn(errors.NumbaPerformanceWarning(msg,
```

Example 27 (python):
```python
finished: added to `.uns['neighbors']`
    `.obsp['distances']`, distances for each pair of neighbors
    `.obsp['connectivities']`, weighted adjacency matrix (0:00:03)
computing UMAP
    finished: added
    'X_umap', UMAP coordinates (adata.obsm) (0:00:05)
```

Example 28 (unknown):
```unknown
/home/sinhalab/miniconda3/envs/scanpynew/lib/python3.8/site-packages/anndata/_core/anndata.py:1220: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Removing unused categories will always return a new Categorical object.
  c.reorder_categories(natsorted(c.categories), inplace=True)
... storing 'phase' as categorical
```

Example 29 (unknown):
```unknown
<Figure size 320x320 with 0 Axes>
```

Example 30 (python):
```python
sc.tl.diffmap(adata)
sc.pl.diffmap(adata, color='fine_grain')
sc.pl.diffmap(adata, color='cycling')
```

Example 31 (python):
```python
computing Diffusion Maps using n_comps=15(=n_dcs)
computing transitions
    finished (0:00:00)
    eigenvalues of transition matrix
    [1.         0.99651    0.9899577  0.9877106  0.9849084  0.9792807
     0.97557443 0.9727674  0.9723406  0.97022986 0.9688626  0.9665088
     0.96357495 0.96243817 0.95802355]
    finished: added
    'X_diffmap', diffmap coordinates (adata.obsm)
    'diffmap_evals', eigenvalues of transition matrix (adata.uns) (0:00:00)
```

Example 32 (python):
```python
#adata.write('./CoronaryEndothelialCellsscFates.h5ad')
```

Example 33 (python):
```python
import pandas as pd
import numpy as np
import anndata


import scipy
import matplotlib.pyplot as plt
import scanpy as sc
import scFates as scf
import palantir

plt.rcParams['pdf.fonttype'] = 42
plt.rcParams['ps.fonttype'] = 42
plt.rcParams["font.family"] = "sans-serif"
sc.settings.verbosity = 3  # verbosity: errors (0), warnings (1), info (2), hints (3)

from matplotlib.colors import LinearSegmentedColormap
cmap = LinearSegmentedColormap.from_list(name='gene_cmap', colors=['lightgrey', 'thistle', 'red', 'darkred']) 
import matplotlib.pyplot as plt
import matplotlib.font_manager

sc.settings.set_figure_params(dpi=80, color_map='viridis', vector_friendly=False,  dpi_save=300, facecolor='white')
sc.logging.print_version_and_date()
#sc.logging.print_versions()
```

Example 34 (unknown):
```unknown
findfont: Font family ['Raleway'] not found. Falling back to DejaVu Sans.
```

Example 35 (unknown):
```unknown
Running Scanpy 1.9.3, on 2023-09-20 12:10.
```

Example 36 (python):
```python
sc.tl.diffmap(adata)
sc.pl.diffmap(adata, color='fine_grain')
```

Example 37 (python):
```python
sc.pl.diffmap(adata, color='fine_grain', dimensions= [(0, 1), (1, 2), (0, 2)])
sc.pl.diffmap(adata, color='cycling', dimensions= [(0, 1), (1, 2), (0, 2)])
```

Example 38 (unknown):
```unknown
/home/sinhalab/miniconda3/envs/scFates/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/home/sinhalab/miniconda3/envs/scFates/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/home/sinhalab/miniconda3/envs/scFates/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 39 (unknown):
```unknown
/home/sinhalab/miniconda3/envs/scFates/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/home/sinhalab/miniconda3/envs/scFates/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/home/sinhalab/miniconda3/envs/scFates/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 40 (python):
```python
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D

color_categories=pd.Categorical(adata.obs['fine_grain'])
colors=color_categories.codes
X_diffmap=adata.obsm['X_diffmap']
fig=plt.figure(figsize=(8,6))
ax=fig.add_subplot(111,projection='3d')
ax.scatter(X_diffmap[:,1],X_diffmap[:,2],X_diffmap[:,3], c=colors)
ax.view_init(elev=20, azim=50)

color_categories=pd.Categorical(adata.obs['cycling'])
colors=color_categories.codes
X_diffmap=adata.obsm['X_diffmap']
fig=plt.figure(figsize=(8,6))
ax=fig.add_subplot(111,projection='3d')
ax.scatter(X_diffmap[:,1],X_diffmap[:,2],X_diffmap[:,3], c=colors)
ax.view_init(elev=20, azim=50)
```

Example 41 (python):
```python
adata=sc.read('./CoronaryEndothelialCellsscFates.h5ad')
print(adata.shape)
```

Example 42 (unknown):
```unknown
(8375, 1332)
```

Example 43 (unknown):
```unknown
cell_cycle_genes = ['MCM5','PCNA','TYMS','FEN1','MCM2','MCM4','RRM1','UNG','GINS2','MCM6','CDCA7','DTL','PRIM1','UHRF1','MLF1IP','HELLS','RFC2','RPA2','NASP','RAD51AP1','GMNN','WDR76','SLBP','CCNE2','UBR7','POLD3','MSH2','ATAD2','RAD51','RRM2','CDC45','CDC6','EXO1','TIPIN','DSCC1','BLM','CASP8AP2','USP1','CLSPN','POLA1','CHAF1B','BRIP1','E2F8','HMGB2','CDK1','NUSAP1','UBE2C','BIRC5','TPX2','TOP2A','NDC80','CKS2','NUF2','CKS1B','MKI67','TMPO','CENPF','TACC3','FAM64A','SMC4','CCNB2','CKAP2L','CKAP2','AURKB','BUB1','KIF11','ANP32E','TUBB4B','GTSE1','KIF20B','HJURP','CDCA3','HN1','CDC20','TTK','CDC25C','KIF2C','RANGAP1','NCAPD2','DLGAP5','CDCA2','CDCA8','ECT2','KIF23','HMMR','AURKA','PSRC1','ANLN','LBR','CKAP5','CENPE','CTCF','NEK2','G2E3','GAS2L3','CBX5','CENPA']
```

Example 44 (python):
```python
sc.pl.diffmap(adata, color='fine_grain')
sc.pl.diffmap(adata, color='cycling')
```

Example 45 (unknown):
```unknown
/home/sinhalab/miniconda3/envs/scFates/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 46 (unknown):
```unknown
/home/sinhalab/miniconda3/envs/scFates/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 47 (python):
```python
#adata.obsm['X_umap']=adata.obsm['X_diffmap'] #dont run this
```

Example 48 (python):
```python
projections = pd.DataFrame(adata.obsm["X_scVI"],index=adata.obs_names)
projections
# fate probability mapping of the cells
dm_res = palantir.utils.run_diffusion_maps(projections)
ms_data = palantir.utils.determine_multiscale_space(dm_res,n_eigs=4)
# Generate embedding from the multiscale diffusion space
adata.obsm["X_palantir"]=ms_data.values
sc.pp.neighbors(adata,n_neighbors=30,use_rep="X_palantir")
sc.pl.embedding(adata, basis='X_palantir',color=['fine_grain'], components='all',ncols=3,wspace=0.6,size=10)

# draw ForceAtlas2 embedding using UMAP dim1 and dim2 as initial position
# make obsm with UMAP1 and 2
adata.obsm['X_umap12'] = adata.obsm['X_umap'][:,[0,1]]
sc.tl.draw_graph(adata,init_pos='X_umap12')
# Plot
sc.set_figure_params(dpi=120)
sc.pl.draw_graph(adata,color="fine_grain",color_map="RdBu_r")
```

Example 49 (unknown):
```unknown
Determing nearest neighbor graph...
computing neighbors
```

Example 50 (python):
```python
/home/sinhalab/miniconda3/envs/scFates/lib/python3.8/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm
```

Example 51 (unknown):
```unknown
finished: added to `.uns['neighbors']`
    `.obsp['distances']`, distances for each pair of neighbors
    `.obsp['connectivities']`, weighted adjacency matrix (0:00:10)
computing neighbors
    finished: added to `.uns['neighbors']`
    `.obsp['distances']`, distances for each pair of neighbors
    `.obsp['connectivities']`, weighted adjacency matrix (0:00:01)
```

Example 52 (unknown):
```unknown
/home/sinhalab/miniconda3/envs/scFates/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/home/sinhalab/miniconda3/envs/scFates/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/home/sinhalab/miniconda3/envs/scFates/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 53 (python):
```python
drawing single-cell graph using layout 'fa'
WARNING: Package 'fa2' is not installed, falling back to layout 'fr'.To use the faster and better ForceAtlas2 layout, install package 'fa2' (`pip install fa2`).
    finished: added
    'X_draw_graph_fr', graph_drawing coordinates (adata.obsm) (0:00:11)
```

Example 54 (unknown):
```unknown
/home/sinhalab/miniconda3/envs/scFates/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 55 (python):
```python
scf.tl.curve(adata,Nodes=40,use_rep="X_palantir",ndims_rep=2,epg_lambda=0.0001,epg_mu=0.1)
scf.pl.graph(adata,basis="palantir")
scf.tl.convert_to_soft(adata,0.1,500)
scf.pl.graph(adata,basis="palantir")

scf.tl.root(adata,"NR2F2")
scf.tl.pseudotime(adata,n_jobs=20,n_map=100,seed=42)
scf.pl.trajectory(adata,basis="palantir",arrows=True,arrow_offset=3)
scf.pl.trajectory(adata,basis="palantir",arrows=False,arrow_offset=3)
adata=adata.raw.to_adata()
```

Example 56 (python):
```python
inferring a principal curve --> parameters used 
    40 principal points, mu = 0.1, lambda = 0.0001
    finished (0:00:01) --> added 
    .uns['epg'] dictionnary containing inferred elastic curve generated from elpigraph.
    .obsm['X_R'] hard assignment of cells to principal points.
    .uns['graph']['B'] adjacency matrix of the principal points.
    .uns['graph']['F'], coordinates of principal points in representation space.
```

Example 57 (unknown):
```unknown
/home/sinhalab/miniconda3/envs/scFates/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap', 'norm' will be ignored
  cax = scatter(
```

Example 58 (unknown):
```unknown
Converting R into soft assignment matrix
    finished (0:00:00) --> updated 
    .obsm['X_R'] converted soft assignment of cells to principal points.
    .uns['graph']['F'] coordinates of principal points in representation space.
```

Example 59 (unknown):
```unknown
/home/sinhalab/miniconda3/envs/scFates/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap', 'norm' will be ignored
  cax = scatter(
```

Example 60 (python):
```python
automatic root selection using NR2F2 values
node 0 selected as a root --> added
    .uns['graph']['root'] selected root.
    .uns['graph']['pp_info'] for each PP, its distance vs root and segment assignment.
    .uns['graph']['pp_seg'] segments network information.
projecting cells onto the principal graph
    mappings: 100%|███████████████████████████| 100/100 [00:29<00:00,  3.35it/s]
    finished (0:00:32) --> added
    .obs['edge'] assigned edge.
    .obs['t'] pseudotime value.
    .obs['seg'] segment of the tree assigned.
    .obs['milestones'] milestone assigned.
    .uns['pseudotime_list'] list of cell projection from all mappings.
```

Example 61 (unknown):
```unknown
/home/sinhalab/miniconda3/envs/scFates/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap', 'norm' will be ignored
  cax = scatter(
```

Example 62 (unknown):
```unknown
/home/sinhalab/miniconda3/envs/scFates/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap', 'norm' will be ignored
  cax = scatter(
```

Example 63 (python):
```python
scf.tl.linearity_deviation(adata,start_milestone="0",end_milestone="2",n_jobs=20,plot=True,basis="palantir")
scf.pl.linearity_deviation(adata,start_milestone="0",end_milestone="2")
sc.pl.umap(adata,color=["H19","IGFBP3","PDGFB"])
```

Example 64 (python):
```python
Estimation of deviation from linearity
    cells on the bridge: 100%|█████████████| 5175/5175 [00:10<00:00, 476.72it/s]
```

Example 65 (unknown):
```unknown
/home/sinhalab/miniconda3/envs/scFates/lib/python3.8/site-packages/scFates/tools/linearity_deviation.py:144: RuntimeWarning: invalid value encountered in divide
  return rs.mean(axis=1) / X_all.std(axis=0)
/home/sinhalab/miniconda3/envs/scFates/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 66 (python):
```python
/home/sinhalab/miniconda3/envs/scFates/lib/python3.8/site-packages/scFates/tools/linearity_deviation.py:164: RuntimeWarning: Mean of empty slice
  adata.var[name + "_rss"] = np.nanmean(np.vstack(rss), axis=0)
```

Example 67 (unknown):
```unknown
finished (0:00:16) --> added 
    .var['0->2_rss'], pearson residuals of the linear fit.
    .obs['0->2_lindev_sel'], cell selections used for the test.
```

Example 68 (python):
```python
scf.tl.test_association(adata,n_jobs=20)
scf.tl.test_association(adata,reapply_filters=True,A_cut=.5)
scf.pl.test_association(adata)
scf.tl.fit(adata,n_jobs=20)
```

Example 69 (python):
```python
test features for association with the trajectory
    single mapping : 100%|████████████████| 36601/36601 [15:03<00:00, 40.51it/s]
    found 233 significant features (0:15:03) --> added
    .var['p_val'] values from statistical test.
    .var['fdr'] corrected values from multiple testing.
    .var['st'] proportion of mapping in which feature is significant.
    .var['A'] amplitue of change of tested feature.
    .var['signi'] feature is significantly changing along pseudotime.
    .uns['stat_assoc_list'] list of fitted features on the graph for all mappings.
reapplied filters, 1024 significant features
fit features associated with the trajectory
    single mapping : 100%|██████████████████| 1024/1024 [00:32<00:00, 31.48it/s]
    finished (adata subsetted to keep only fitted features!) (0:00:33) --> added
    .layers['fitted'], fitted features on the trajectory for all mappings.
    .raw, unfiltered data.
```

Example 70 (python):
```python
scf.pl.single_trend(adata,"NR2F2",basis="palantir",color_exp="k")
scf.pl.single_trend(adata,"RGCC",basis="palantir",color_exp="k")
scf.pl.single_trend(adata,"HEY2",basis="palantir",color_exp="k")
scf.pl.single_trend(adata,"TOP2A",basis="palantir",color_exp="k")
scf.pl.single_trend(adata,"MKI67",basis="palantir",color_exp="k")
```

Example 71 (python):
```python
GOI=['UBE2C','NUSAP1','CENPF','TOP2A','MKI67','NR2F2','RGCC','HEY2']
scf.pl.trends(adata,features=GOI,basis="palantir")
```

Example 72 (python):
```python
adata.write('./CoronaryEndotheliumscFatesProcessedForCellCycling.h5ad')
```

Example 73 (python):
```python
/home/sinhalab/miniconda3/envs/scFates/lib/python3.8/site-packages/anndata/_io/specs/methods.py:292: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.
  write_elem(f, k, np.array(elem), dataset_kwargs=dataset_kwargs)
```

Example 74 (python):
```python
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
File ~/miniconda3/envs/scFates/lib/python3.8/site-packages/anndata/_io/utils.py:214, in report_write_key_on_error.<locals>.func_wrapper(elem, key, val, *args, **kwargs)
    213 try:
--> 214     return func(elem, key, val, *args, **kwargs)
    215 except Exception as e:

File ~/miniconda3/envs/scFates/lib/python3.8/site-packages/anndata/_io/specs/registry.py:171, in write_elem(f, k, elem, modifiers, *args, **kwargs)
    167 if (
    168     hasattr(elem, "dtype")
    169     and (dest_type, (t, elem.dtype.kind), modifiers) in _REGISTRY.write
    170 ):
--> 171     _REGISTRY.get_writer(dest_type, (t, elem.dtype.kind), modifiers)(
    172         f, k, elem, *args, **kwargs
    173     )
    174 else:

File ~/miniconda3/envs/scFates/lib/python3.8/site-packages/anndata/_io/specs/registry.py:24, in write_spec.<locals>.decorator.<locals>.wrapper(g, k, *args, **kwargs)
     22 @wraps(func)
     23 def wrapper(g, k, *args, **kwargs):
---> 24     result = func(g, k, *args, **kwargs)
     25     g[k].attrs.setdefault("encoding-type", spec.encoding_type)

File ~/miniconda3/envs/scFates/lib/python3.8/site-packages/anndata/_io/specs/methods.py:346, in write_vlen_string_array(f, k, elem, dataset_kwargs)
    345 str_dtype = h5py.special_dtype(vlen=str)
--> 346 f.create_dataset(k, data=elem.astype(str_dtype), dtype=str_dtype, **dataset_kwargs)

File ~/miniconda3/envs/scFates/lib/python3.8/site-packages/h5py/_hl/group.py:183, in Group.create_dataset(self, name, shape, dtype, data, **kwds)
    181         group = self.require_group(parent_path)
--> 183 dsid = dataset.make_new_dset(group, shape, dtype, data, name, **kwds)
    184 dset = dataset.Dataset(dsid)

File ~/miniconda3/envs/scFates/lib/python3.8/site-packages/h5py/_hl/dataset.py:168, in make_new_dset(parent, shape, dtype, data, name, chunks, compression, shuffle, fletcher32, maxshape, compression_opts, fillvalue, scaleoffset, track_times, external, track_order, dcpl, dapl, efile_prefix, virtual_prefix, allow_unknown_filter, rdcc_nslots, rdcc_nbytes, rdcc_w0)
    167 if (data is not None) and (not isinstance(data, Empty)):
--> 168     dset_id.write(h5s.ALL, h5s.ALL, data)
    170 return dset_id

File h5py/_objects.pyx:54, in h5py._objects.with_phil.wrapper()

File h5py/_objects.pyx:55, in h5py._objects.with_phil.wrapper()

File h5py/h5d.pyx:280, in h5py.h5d.DatasetID.write()

File h5py/_proxy.pyx:145, in h5py._proxy.dset_rw()

File h5py/_conv.pyx:444, in h5py._conv.str2vlen()

File h5py/_conv.pyx:95, in h5py._conv.generic_converter()

File h5py/_conv.pyx:249, in h5py._conv.conv_str2vlen()

TypeError: Can't implicitly convert non-string objects to strings

The above exception was the direct cause of the following exception:

TypeError                                 Traceback (most recent call last)
Cell In[51], line 1
----> 1 adata.write('./CoronaryEndotheliumscFatesProcessedForCellCycling.h5ad')

File ~/miniconda3/envs/scFates/lib/python3.8/site-packages/anndata/_core/anndata.py:1918, in AnnData.write_h5ad(self, filename, compression, compression_opts, force_dense, as_dense)
   1915 if filename is None:
   1916     filename = self.filename
-> 1918 _write_h5ad(
   1919     Path(filename),
   1920     self,
   1921     compression=compression,
   1922     compression_opts=compression_opts,
   1923     force_dense=force_dense,
   1924     as_dense=as_dense,
   1925 )
   1927 if self.isbacked:
   1928     self.file.filename = filename

File ~/miniconda3/envs/scFates/lib/python3.8/site-packages/anndata/_io/h5ad.py:105, in write_h5ad(filepath, adata, force_dense, as_dense, dataset_kwargs, **kwargs)
    103 write_elem(f, "varp", dict(adata.varp), dataset_kwargs=dataset_kwargs)
    104 write_elem(f, "layers", dict(adata.layers), dataset_kwargs=dataset_kwargs)
--> 105 write_elem(f, "uns", dict(adata.uns), dataset_kwargs=dataset_kwargs)

File ~/miniconda3/envs/scFates/lib/python3.8/site-packages/anndata/_io/utils.py:214, in report_write_key_on_error.<locals>.func_wrapper(elem, key, val, *args, **kwargs)
    211 @wraps(func)
    212 def func_wrapper(elem, key, val, *args, **kwargs):
    213     try:
--> 214         return func(elem, key, val, *args, **kwargs)
    215     except Exception as e:
    216         if "Above error raised while writing key" in format(e):

File ~/miniconda3/envs/scFates/lib/python3.8/site-packages/anndata/_io/specs/registry.py:175, in write_elem(f, k, elem, modifiers, *args, **kwargs)
    171     _REGISTRY.get_writer(dest_type, (t, elem.dtype.kind), modifiers)(
    172         f, k, elem, *args, **kwargs
    173     )
    174 else:
--> 175     _REGISTRY.get_writer(dest_type, t, modifiers)(f, k, elem, *args, **kwargs)

File ~/miniconda3/envs/scFates/lib/python3.8/site-packages/anndata/_io/specs/registry.py:24, in write_spec.<locals>.decorator.<locals>.wrapper(g, k, *args, **kwargs)
     22 @wraps(func)
     23 def wrapper(g, k, *args, **kwargs):
---> 24     result = func(g, k, *args, **kwargs)
     25     g[k].attrs.setdefault("encoding-type", spec.encoding_type)
     26     g[k].attrs.setdefault("encoding-version", spec.encoding_version)

File ~/miniconda3/envs/scFates/lib/python3.8/site-packages/anndata/_io/specs/methods.py:281, in write_mapping(f, k, v, dataset_kwargs)
    279 g = f.create_group(k)
    280 for sub_k, sub_v in v.items():
--> 281     write_elem(g, sub_k, sub_v, dataset_kwargs=dataset_kwargs)

File ~/miniconda3/envs/scFates/lib/python3.8/site-packages/anndata/_io/utils.py:214, in report_write_key_on_error.<locals>.func_wrapper(elem, key, val, *args, **kwargs)
    211 @wraps(func)
    212 def func_wrapper(elem, key, val, *args, **kwargs):
    213     try:
--> 214         return func(elem, key, val, *args, **kwargs)
    215     except Exception as e:
    216         if "Above error raised while writing key" in format(e):

File ~/miniconda3/envs/scFates/lib/python3.8/site-packages/anndata/_io/specs/registry.py:175, in write_elem(f, k, elem, modifiers, *args, **kwargs)
    171     _REGISTRY.get_writer(dest_type, (t, elem.dtype.kind), modifiers)(
    172         f, k, elem, *args, **kwargs
    173     )
    174 else:
--> 175     _REGISTRY.get_writer(dest_type, t, modifiers)(f, k, elem, *args, **kwargs)

File ~/miniconda3/envs/scFates/lib/python3.8/site-packages/anndata/_io/specs/registry.py:24, in write_spec.<locals>.decorator.<locals>.wrapper(g, k, *args, **kwargs)
     22 @wraps(func)
     23 def wrapper(g, k, *args, **kwargs):
---> 24     result = func(g, k, *args, **kwargs)
     25     g[k].attrs.setdefault("encoding-type", spec.encoding_type)
     26     g[k].attrs.setdefault("encoding-version", spec.encoding_version)

File ~/miniconda3/envs/scFates/lib/python3.8/site-packages/anndata/_io/specs/methods.py:281, in write_mapping(f, k, v, dataset_kwargs)
    279 g = f.create_group(k)
    280 for sub_k, sub_v in v.items():
--> 281     write_elem(g, sub_k, sub_v, dataset_kwargs=dataset_kwargs)

File ~/miniconda3/envs/scFates/lib/python3.8/site-packages/anndata/_io/utils.py:214, in report_write_key_on_error.<locals>.func_wrapper(elem, key, val, *args, **kwargs)
    211 @wraps(func)
    212 def func_wrapper(elem, key, val, *args, **kwargs):
    213     try:
--> 214         return func(elem, key, val, *args, **kwargs)
    215     except Exception as e:
    216         if "Above error raised while writing key" in format(e):

File ~/miniconda3/envs/scFates/lib/python3.8/site-packages/anndata/_io/specs/registry.py:175, in write_elem(f, k, elem, modifiers, *args, **kwargs)
    171     _REGISTRY.get_writer(dest_type, (t, elem.dtype.kind), modifiers)(
    172         f, k, elem, *args, **kwargs
    173     )
    174 else:
--> 175     _REGISTRY.get_writer(dest_type, t, modifiers)(f, k, elem, *args, **kwargs)

File ~/miniconda3/envs/scFates/lib/python3.8/site-packages/anndata/_io/specs/registry.py:24, in write_spec.<locals>.decorator.<locals>.wrapper(g, k, *args, **kwargs)
     22 @wraps(func)
     23 def wrapper(g, k, *args, **kwargs):
---> 24     result = func(g, k, *args, **kwargs)
     25     g[k].attrs.setdefault("encoding-type", spec.encoding_type)
     26     g[k].attrs.setdefault("encoding-version", spec.encoding_version)

File ~/miniconda3/envs/scFates/lib/python3.8/site-packages/anndata/_io/specs/methods.py:292, in write_list(f, k, elem, dataset_kwargs)
    289 @_REGISTRY.register_write(H5Group, list, IOSpec("array", "0.2.0"))
    290 @_REGISTRY.register_write(ZarrGroup, list, IOSpec("array", "0.2.0"))
    291 def write_list(f, k, elem, dataset_kwargs=MappingProxyType({})):
--> 292     write_elem(f, k, np.array(elem), dataset_kwargs=dataset_kwargs)

File ~/miniconda3/envs/scFates/lib/python3.8/site-packages/anndata/_io/utils.py:220, in report_write_key_on_error.<locals>.func_wrapper(elem, key, val, *args, **kwargs)
    218 else:
    219     parent = _get_parent(elem)
--> 220     raise type(e)(
    221         f"{e}\n\n"
    222         f"Above error raised while writing key {key!r} of {type(elem)} "
    223         f"to {parent}"
    224     ) from e

TypeError: Can't implicitly convert non-string objects to strings

Above error raised while writing key 'Edges' of <class 'h5py._hl.group.Group'> to /
```

Example 75 (python):
```python
bdata=adata.copy()
```

Example 76 (python):
```python
scf.tl.cluster(adata,n_neighbors=20,metric="correlation")
adata.var.cluters.unique()
```

Example 77 (python):
```python
Clustering features using fitted layer
computing PCA
    with n_comps=50
    finished (0:00:01)
computing neighbors
    using 'X_pca' with n_pcs = 50
    finished: added to `.uns['neighbors']`
    `.obsp['distances']`, distances for each pair of neighbors
    `.obsp['connectivities']`, weighted adjacency matrix (0:00:01)
running Leiden clustering
    finished: found 12 clusters and added
    'leiden', the cluster labels (adata.obs, categorical) (0:00:00)
    finished (0:00:03) --> added 
    .var['cluters'] identified modules.
```

Example 78 (unknown):
```unknown
['1', '3', '7', '5', '0', ..., '6', '4', '9', '11', '10']
Length: 12
Categories (12, object): ['0', '1', '2', '3', ..., '8', '9', '10', '11']
```

Example 79 (python):
```python
for c in adata.var["cluters"].unique():
    scf.pl.trends(adata,features=adata.var_names[adata.var.cluters==c],basis="palantir")
```

Example 80 (python):
```python
GOI=['NR2F2','UBE2C','NUSAP1','ASPM','CENPF','TOP2A','MKI67','RGCC','HEY2']
scf.pl.trends(adata,features=GOI,basis="palantir")
```

Example 81 (python):
```python
GOI=['UBE2C','NUSAP1','ASPM','CENPF','TOP2A','MKI67']
scf.pl.trends(adata,features=GOI,basis="palantir")
```

Example 82 (python):
```python
GOI=['NR2F2']
scf.pl.trends(adata,features=GOI,basis="palantir")
```

Example 83 (python):
```python
sc.pl.draw_graph(adata,color="fine_grain",color_map="RdBu_r")
```

Example 84 (unknown):
```unknown
/home/sinhalab/miniconda3/envs/scFates/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 85 (python):
```python
sc.pl.embedding(adata, basis='X_palantir',color=['fine_grain'],ncols=3,wspace=0.6,size=10)
```

Example 86 (unknown):
```unknown
/home/sinhalab/miniconda3/envs/scFates/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

---

## Import modules¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig4_Cardiomyocytes/VentricularCardiomyocytes/multiVI_scFate/no2_run_MultiVI_vCM_Left.html

**Contents:**
- Import modules¶
- Read in data¶
- Setup and training MultiVI (with continuous covariates)¶
  - All region, add 'region' key as a covariate¶

16451 rows × 33 columns

544641 rows × 21 columns

**Examples:**

Example 1 (python):
```python
# this is for the jupyter notebook at JupyterLab
# to make sys.path between the terminal and Jupyter same
import sys

print(sys.path) # especially the path to site-packages are different... so thats why couldn't find correct package...

# sys.path from terminal at scvi0190_env
# obtained by excuting `python -c "import sys; print('\n'.join(sys.path))"`
sys.path = [
    "/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python39.zip",
    "/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9",
    "/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/lib-dynload",
    "/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages"
]

print(sys.path)
```

Example 2 (unknown):
```unknown
['/nfs/team205/kk18/Analysis/multiVI/notebooks', '/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python39.zip', '/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9', '/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/lib-dynload', '', '/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages']
['/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python39.zip', '/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9', '/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/lib-dynload', '/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages']
```

Example 3 (python):
```python
import scvi
import anndata
import numpy as np
import pandas as pd
import scanpy as sc
import matplotlib.pyplot as plt

scvi.settings.seed = 420
```

Example 4 (python):
```python
Global seed set to 0
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/flax/core/frozen_dict.py:169: FutureWarning: jax.tree_util.register_keypaths is deprecated, and will be removed in a future release. Please use `register_pytree_with_keys()` instead.
  jax.tree_util.register_keypaths(
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/pytorch_lightning/utilities/warnings.py:53: LightningDeprecationWarning: pytorch_lightning.utilities.warnings.rank_zero_deprecation has been deprecated in v1.6 and will be removed in v1.8. Use the equivalent function from the pytorch_lightning.utilities.rank_zero module instead.
  new_rank_zero_deprecation(
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/pytorch_lightning/utilities/warnings.py:58: LightningDeprecationWarning: The `pytorch_lightning.loggers.base.rank_zero_experiment` is deprecated in v1.7 and will be removed in v1.9. Please use `pytorch_lightning.loggers.logger.rank_zero_experiment` instead.
  return new_rank_zero_deprecation(*args, **kwargs)
Global seed set to 420
```

Example 5 (python):
```python
import session_info
session_info.show()
```

Example 6 (python):
```python
-----
anndata             0.8.0
matplotlib          3.7.1
numpy               1.23.5
pandas              1.5.3
scanpy              1.9.3
scvi                0.19.0
session_info        1.0.0
-----
```

Example 7 (python):
```python
PIL                 9.4.0
absl                NA
asttokens           NA
attr                22.2.0
backcall            0.2.0
certifi             2022.12.07
cffi                1.15.1
charset_normalizer  3.1.0
chex                0.1.6
comm                0.1.2
contextlib2         NA
cycler              0.10.0
cython_runtime      NA
dateutil            2.8.2
debugpy             1.6.6
decorator           5.1.1
defusedxml          0.7.1
deprecate           0.3.2
docrep              0.3.2
etils               1.1.0
executing           1.2.0
flax                0.6.7
fsspec              2023.3.0
google              NA
h5py                3.8.0
idna                3.4
igraph              0.10.4
importlib_metadata  NA
importlib_resources NA
ipykernel           6.21.3
ipython_genutils    0.2.0
ipywidgets          8.0.4
jax                 0.4.10
jaxlib              0.4.10
jedi                0.18.2
joblib              1.2.0
kiwisolver          1.4.4
leidenalg           0.9.1
llvmlite            0.39.1
ml_collections      NA
ml_dtypes           0.2.0
mpl_toolkits        NA
msgpack             1.0.5
mudata              0.2.1
multipledispatch    0.6.0
natsort             8.3.1
numba               0.56.4
numpyro             0.11.0
nvidia              NA
opt_einsum          v3.3.0
optax               0.1.4
packaging           23.0
parso               0.8.3
pexpect             4.8.0
pickleshare         0.7.5
pkg_resources       NA
platformdirs        3.1.1
prompt_toolkit      3.0.38
psutil              5.9.4
ptyprocess          0.7.0
pure_eval           0.2.2
pydev_ipython       NA
pydevconsole        NA
pydevd              2.9.5
pydevd_file_utils   NA
pydevd_plugins      NA
pydevd_tracing      NA
pygments            2.14.0
pyparsing           3.0.9
pyro                1.8.4
pytorch_lightning   1.7.7
pytz                2022.7.1
requests            2.28.2
rich                NA
scipy               1.10.1
setuptools          65.6.3
six                 1.16.0
sklearn             1.2.2
stack_data          0.6.2
tensorboard         2.12.0
texttable           1.6.7
threadpoolctl       3.1.0
toolz               0.12.0
torch               1.13.1+cu117
torchmetrics        0.11.4
torchvision         0.14.1+cu117
tornado             6.2
tqdm                4.65.0
traitlets           5.9.0
tree                0.1.8
typing_extensions   NA
urllib3             1.26.15
wcwidth             0.2.6
yaml                6.0
zipp                NA
zmq                 25.0.1
zoneinfo            NA
```

Example 8 (unknown):
```unknown
-----
IPython             8.11.0
jupyter_client      8.0.3
jupyter_core        5.2.0
notebook            6.5.3
-----
Python 3.9.16 (main, Mar  8 2023, 14:00:05) [GCC 11.2.0]
Linux-4.15.0-213-generic-x86_64-with-glibc2.27
-----
Session information updated at 2023-08-15 23:34
```

Example 9 (python):
```python
# paired
# only multiome
adata_mvi = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/vCM/MultiVI-pre_paired_foetal_vCMs_raw.h5ad')
adata_mvi
```

Example 10 (unknown):
```unknown
AnnData object with n_obs × n_vars = 35705 × 544641
    obs: 'sangerID', 'combinedID', 'donor', 'region', 'age', 'sex', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'week', 'trimester', 'cycling', 'S_score', 'G2M_score', 'phase', 'fine_grain', 'mid_grain', 'coarse_grain', 'batch_key', 'total_counts', 'total_counts_mt', 'total_counts_ribo', 'n_genes_by_counts', 'pct_counts_mt', 'pct_counts_ribo', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'log1p_total_counts_mt', 'log1p_total_counts_ribo', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id', 'seqnames', 'start', 'end', 'width', 'strand', 'score', 'replicateScoreQuantile', 'groupScoreQuantile', 'Reproducibility', 'GroupReplicate', 'distToGeneStart', 'nearestGene', 'peakType', 'distToTSS', 'nearestTSS', 'GC', 'idx', 'N'
```

Example 11 (python):
```python
adata_mvi.obs['fine_grain'].value_counts()
```

Example 12 (unknown):
```unknown
VentricularCardiomyocytesRightCompact         12008
VentricularCardiomyocytesLeftCompact          11397
VentricularCardiomyocytesLeftTrabeculated      5054
VentricularCardiomyocytesCycling               4977
VentricularCardiomyocytesRightTrabeculated     2269
Name: fine_grain, dtype: int64
```

Example 13 (python):
```python
# remove cycling population
adata_mvi = adata_mvi[adata_mvi.obs['fine_grain'].isin([
    'VentricularCardiomyocytesLeftCompact','VentricularCardiomyocytesLeftTrabeculated'
])]
adata_mvi.obs['fine_grain'].value_counts()
```

Example 14 (unknown):
```unknown
VentricularCardiomyocytesLeftCompact         11397
VentricularCardiomyocytesLeftTrabeculated     5054
Name: fine_grain, dtype: int64
```

Example 15 (python):
```python
adata_mvi.obs
```

Example 16 (python):
```python
adata_mvi.var['modality'].value_counts()
```

Example 17 (unknown):
```unknown
Peaks              508040
Gene Expression     36601
Name: modality, dtype: int64
```

Example 18 (python):
```python
adata_mvi.obs['modality'].value_counts()
```

Example 19 (unknown):
```unknown
Multiome    16451
Name: modality, dtype: int64
```

Example 20 (python):
```python
# MultiVI requires the features to be ordered, 
# such that genes appear before genomic regions. In this case this is already the case, but it’s always good to verify:
adata_mvi = adata_mvi[:, adata_mvi.var["modality"].argsort()].copy()
adata_mvi.var
```

Example 21 (python):
```python
adata_mvi_sub = adata_mvi.copy()
```

Example 22 (python):
```python
# We also filter features to remove those that appear in fewer than 1% of the cells
print(adata_mvi_sub.shape)
sc.pp.filter_genes(adata_mvi_sub, min_cells=int(adata_mvi_sub.shape[0] * 0.01))
print(adata_mvi_sub.shape)
```

Example 23 (unknown):
```unknown
(16451, 544641)
(16451, 174604)
```

Example 24 (python):
```python
adata_mvi_sub.var['modality'].value_counts()
```

Example 25 (unknown):
```unknown
Peaks              161398
Gene Expression     13206
Name: modality, dtype: int64
```

Example 26 (python):
```python
adata_mvi_sub.obs['modality'].value_counts()
```

Example 27 (unknown):
```unknown
Multiome    16451
Name: modality, dtype: int64
```

Example 28 (python):
```python
pd.crosstab(adata_mvi_sub.obs['batch_key'],adata_mvi_sub.obs['region'])
```

Example 29 (python):
```python
scvi.model.MULTIVI.setup_anndata(adata_mvi_sub, batch_key='modality',
                                categorical_covariate_keys=['batch_key','region'], ### adding 'region' ###
                                continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo',
                                                          'S_score','G2M_score'])
```

Example 30 (unknown):
```unknown
No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.)
```

Example 31 (python):
```python
# When creating the object, we need to specify how many of the features are genes, and how many are genomic regions. 
# This is so MultiVI can determine the exact architecture for each modality.
mvi = scvi.model.MULTIVI(
    adata_mvi_sub,
    n_genes=(adata_mvi_sub.var['modality']=='Gene Expression').sum(),
    n_regions=(adata_mvi_sub.var['modality']=='Peaks').sum(),
)
mvi.view_anndata_setup()
```

Example 32 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/torch/nn/init.py:405: UserWarning: Initializing zero-element tensors is a no-op
  warnings.warn("Initializing zero-element tensors is a no-op")
```

Example 33 (unknown):
```unknown
Anndata setup with scvi-tools version 0.19.0.
```

Example 34 (unknown):
```unknown
Setup via `MULTIVI.setup_anndata` with arguments:
```

Example 35 (unknown):
```unknown
{
│   'layer': None,
│   'batch_key': 'modality',
│   'size_factor_key': None,
│   'categorical_covariate_keys': ['batch_key', 'region'],
│   'continuous_covariate_keys': ['total_counts', 'pct_counts_mt', 'pct_counts_ribo', 'S_score', 'G2M_score'],
│   'protein_expression_obsm_key': None,
│   'protein_names_uns_key': None
}
```

Example 36 (unknown):
```unknown
Summary Statistics          
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃     Summary Stat Key     ┃ Value  ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│         n_batch          │   1    │
│         n_cells          │ 16451  │
│ n_extra_categorical_covs │   2    │
│ n_extra_continuous_covs  │   5    │
│         n_labels         │   1    │
│          n_vars          │ 174604 │
└──────────────────────────┴────────┘
```

Example 37 (python):
```python
Data Registry                             
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃      Registry Key      ┃            scvi-tools Location             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│           X            │                  adata.X                   │
│         batch          │          adata.obs['_scvi_batch']          │
│ extra_categorical_covs │ adata.obsm['_scvi_extra_categorical_covs'] │
│ extra_continuous_covs  │ adata.obsm['_scvi_extra_continuous_covs']  │
│         ind_x          │           adata.obs['_indices']            │
│         labels         │         adata.obs['_scvi_labels']          │
└────────────────────────┴────────────────────────────────────────────┘
```

Example 38 (python):
```python
batch State Registry                    
┏━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location    ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['modality'] │  Multiome  │          0          │
└───────────────────────┴────────────┴─────────────────────┘
```

Example 39 (python):
```python
labels State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 40 (python):
```python
batch State Registry                    
┏━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location    ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['modality'] │  Multiome  │          0          │
└───────────────────────┴────────────┴─────────────────────┘
```

Example 41 (python):
```python
extra_categorical_covs State Registry                       
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location     ┃           Categories            ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['batch_key'] │     C82_Nuclei_Multiome-v1      │          0          │
│                        │     C85_Nuclei_Multiome-v1      │          1          │
│                        │     C87_Nuclei_Multiome-v1      │          2          │
│                        │     C92_Nuclei_Multiome-v1      │          3          │
│                        │     C98_Nuclei_Multiome-v1      │          4          │
│                        │     C104_Nuclei_Multiome-v1     │          5          │
│                        │    Hst33_Nuclei_Multiome-v1     │          6          │
│                        │    Hst36_Nuclei_Multiome-v1     │          7          │
│                        │    Hst39_Nuclei_Multiome-v1     │          8          │
│                        │    Hst40_Nuclei_Multiome-v1     │          9          │
│                        │    Hst41_Nuclei_Multiome-v1     │         10          │
│                        │                                 │                     │
│  adata.obs['region']   │              apex               │          0          │
│                        │              atria              │          1          │
│                        │          atrial septum          │          2          │
│                        │          whole sample           │          3          │
│                        │       heart without node        │          4          │
│                        │         left ventricle          │          5          │
│                        │              node               │          6          │
│                        │          outflow tract          │          7          │
│                        │         right ventricle         │          8          │
│                        │ right ventricle and papillaries │          9          │
│                        │                                 │                     │
└────────────────────────┴─────────────────────────────────┴─────────────────────┘
```

Example 42 (python):
```python
extra_continuous_covs State   
            Registry            
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃       Source Location        ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│  adata.obs['total_counts']   │
│  adata.obs['pct_counts_mt']  │
│ adata.obs['pct_counts_ribo'] │
│     adata.obs['S_score']     │
│    adata.obs['G2M_score']    │
└──────────────────────────────┘
```

Example 43 (unknown):
```unknown
mvi.train()
```

Example 44 (unknown):
```unknown
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/pytorch_lightning/trainer/configuration_validator.py:267: LightningDeprecationWarning: The `Callback.on_epoch_end` hook was deprecated in v1.6 and will be removed in v1.8. Please use `Callback.on_<train/validation/test>_epoch_end` instead.
  rank_zero_deprecation(
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
```

Example 45 (unknown):
```unknown
Epoch 269/500:  54%|██████████████████████                   | 269/500 [1:57:00<1:40:28, 26.10s/it, loss=1.42e+04, v_num=1]
Monitored metric reconstruction_loss_validation did not improve in the last 50 records. Best score: 30056.787. Signaling Trainer to stop.
```

Example 46 (unknown):
```unknown
mvi.save("/nfs/team205/heart/anndata_objects/Foetal/vCM/MultiVI-model_foetal_vCM_onlyMultiome_allregion_onlyLeft_feature-select-001")
```

Example 47 (python):
```python
adata_mvi_sub.obsm["MultiVI_latent"] = mvi.get_latent_representation()
```

Example 48 (python):
```python
adata_mvi_sub.write('/nfs/team205/heart/anndata_objects/Foetal/vCM/MultiVI-post_foetal_vCM_onlyMultiome_allregion_onlyLeft_feature-select-001.h5ad')
```

Example 49 (python):
```python
# Convergency plot
mvi.history['elbo_train']
x = np.linspace(1, (len(mvi.history['elbo_train'])), (len(mvi.history['elbo_train'])))
plt.plot(x, mvi.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
```

---

## Multiplexed samples ID¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/RNA/QC/Demultiplex/No4_Assign-donors.html

**Contents:**
- Multiplexed samples ID¶
- Read in SOC result¶
- Assign donors with CMO result (for SOC unassigned nuclei)¶
- Make assignment dataframe and save¶

**Examples:**

Example 1 (python):
```python
import warnings
warnings.simplefilter(action='ignore', category=FutureWarning)

import numpy as np
import pandas as pd
import scanpy as sc
import session_info
import matplotlib.pyplot as plt
import seaborn as sns

import os
```

Example 2 (unknown):
```unknown
session_info.show()
```

Example 3 (python):
```python
-----
matplotlib          3.5.2
numpy               1.21.5
pandas              1.3.5
scanpy              1.8.2
seaborn             0.11.1
session_info        1.0.0
-----
```

Example 4 (python):
```python
PIL                 9.1.1
anndata             0.8.0
asciitree           NA
backcall            0.2.0
beta_ufunc          NA
binom_ufunc         NA
cached_property     1.5.1
cairo               1.20.0
cffi                1.14.4
cloudpickle         1.6.0
colorama            0.4.4
cycler              0.10.0
cython_runtime      NA
cytoolz             0.11.2
dask                2022.01.0
dateutil            2.8.1
decorator           4.4.2
defusedxml          0.6.0
fasteners           0.17.3
fsspec              2022.01.0
google              NA
h5py                3.1.0
igraph              0.10.2
importlib_metadata  1.7.0
ipykernel           5.4.2
ipython_genutils    0.2.0
ipywidgets          7.6.3
jedi                0.17.2
jinja2              3.1.2
joblib              1.0.0
kiwisolver          1.3.1
leidenalg           0.9.0
llvmlite            0.35.0
louvain             0.7.0
markupsafe          2.1.1
mpl_toolkits        NA
msgpack             1.0.3
natsort             7.1.0
nbinom_ufunc        NA
numba               0.52.0
numcodecs           0.9.1
numexpr             2.7.2
packaging           21.3
parso               0.7.1
pexpect             4.8.0
pickleshare         0.7.5
pkg_resources       NA
plotly              5.10.0
prompt_toolkit      3.0.10
psutil              5.9.0
ptyprocess          0.7.0
pyarrow             9.0.0
pygments            2.13.0
pyparsing           2.4.7
pytz                2020.5
scipy               1.7.3
setuptools_scm      NA
sinfo               0.3.1
six                 1.14.0
sklearn             1.0.2
sphinxcontrib       NA
statsmodels         0.12.1
storemagic          NA
tables              3.6.1
tblib               1.7.0
texttable           1.6.3
threadpoolctl       2.1.0
tlz                 0.11.2
toolz               0.11.2
tornado             6.1
traitlets           5.0.5
typing_extensions   NA
wcwidth             0.2.5
yaml                5.4.1
zarr                2.11.3
zipp                NA
zmq                 21.0.1
zope                NA
```

Example 5 (unknown):
```unknown
-----
IPython             7.19.0
jupyter_client      6.1.11
jupyter_core        4.7.0
notebook            6.2.0
-----
Python 3.7.9 | packaged by conda-forge | (default, Dec  9 2020, 21:08:20) [GCC 9.3.0]
Linux-4.15.0-112-generic-x86_64-with-debian-bullseye-sid
-----
Session information updated at 2023-02-08 21:05
```

Example 6 (python):
```python
sc.settings.set_figure_params(dpi=120)
```

Example 7 (unknown):
```unknown
multiplexed_samples = ['7089STDY13216920_BHF_F_Hea13242527','7089STDY13216921_BHF_F_Hea13242528',
                       '7089STDY13216922_BHF_F_Hea13242529','7089STDY13216923_BHF_F_Hea13242530']
```

Example 8 (python):
```python
# read in
soc = pd.read_csv('/nfs/team205/heart/mapped/cellranger-arc200/SOC_out/SOC-clusters_multiome-foetal_20221221.csv',
                 index_col=0)
print(soc.shape)
soc.head()
```

Example 9 (unknown):
```unknown
(74984, 10)
```

Example 10 (unknown):
```unknown
soc['donor_assignment'].value_counts()
```

Example 11 (unknown):
```unknown
Hst39               36323
Hst40               20246
donor_unassigned    13322
Hst36                5093
Name: donor_assignment, dtype: int64
```

Example 12 (python):
```python
soc_assigned_obsnames = list(soc.index[soc['donor_assignment']!='donor_unassigned'])
soc_unassigned_obsnames = list(soc.index[soc['donor_assignment']=='donor_unassigned'])
print(f'SOC assigned: {len(soc_assigned_obsnames)}')
print(f'SOC unassigned: {len(soc_unassigned_obsnames)}')
```

Example 13 (unknown):
```unknown
SOC assigned: 61662
SOC unassigned: 13322
```

Example 14 (python):
```python
# read in
cmo = pd.read_csv('/nfs/team205/heart/mapped/CMO/result_summary/CMO-result_multiome-foetal_20230207.csv',
                 index_col=0)
# extract soc-unassigned nuclei
cmo = cmo.loc[soc_unassigned_obsnames]

print(cmo.shape)
cmo.head()
```

Example 15 (python):
```python
# assign donors which has prop_delta > 2.4282365
# refer to notebook No3
cmo_assigned_obsnames = list(cmo.index[cmo['ratio'] > 2.4282365])
print(f'CMO assigned: {len(cmo_assigned_obsnames)}')

# change assignment to "unassigned" for the nuclei which have "prop_delta" lower than the threshold
cmo.loc[[x for x in cmo.index if x not in cmo_assigned_obsnames],'donor_assignment'] = 'donor_unassigned'
cmo['donor_assignment'].value_counts()
```

Example 16 (unknown):
```unknown
CMO assigned: 1885
```

Example 17 (unknown):
```unknown
donor_unassigned    11437
Hst40                1690
Hst36                 155
Hst39                  40
Name: donor_assignment, dtype: int64
```

Example 18 (unknown):
```unknown
# SOC
col_to_add = ['status','log_prob_singleton','log_prob_doublet', 'donor_assignment']
summary = soc[col_to_add].copy()
summary.columns = [f'SOC | {x}' for x in summary.columns]
summary.head()
```

Example 19 (python):
```python
# make final assignment column by adding CMO result
summary.rename(columns={'SOC | donor_assignment':'SOC-CMO_assignment'},inplace=True)
summary.loc[cmo.index,'SOC-CMO_assignment'] = cmo['donor_assignment'].astype('str')
print(summary['SOC-CMO_assignment'].value_counts())
summary.head()
```

Example 20 (unknown):
```unknown
Hst39               36363
Hst40               21936
donor_unassigned    11437
Hst36                5248
Name: SOC-CMO_assignment, dtype: int64
```

Example 21 (unknown):
```unknown
# save
summary.to_csv('/nfs/team205/heart/mapped/cellranger-arc200/SOC_out/SOC-CMO-assignment_multiome-foetal_20230207.csv')
```

---

## Variables¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/VisiumHD/no7_manual-annotation_ventricularCM.html

**Contents:**
- Variables¶
- Read in data¶
- Remove mid-grain markers from the feature¶
- PCA, Clustering, UMAP¶
- Markers¶
- Subset "uncertain" cells¶
- Subset uncertain cells - 2nd round¶
- Add the annotation to the original adata¶
- Save¶

2366 rows × 7 columns

--> cluster 1 has low total_counts and bin_counts. and no specific marker expression --> will annotated as "unassigned"

--> cluster 1 looks like VCS-distal

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os

import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 

import celltypist
```

Example 2 (python):
```python
# import session_info
# session_info.show()
```

Example 3 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 4 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 5 (python):
```python
adata_dir = '/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/subsets'
sample_id = 'HEA_FOET14880396'
celltype = 'VentricularCardiomyocytes'
```

Example 6 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk/Aug2024'
table_dir = '/nfs/team205/heart/fetal_ms_figures/tables/Aug2024'
```

Example 7 (python):
```python
# read in
adata = sc.read_h5ad(f'{adata_dir}/{sample_id}_{celltype}_b2c_cells_filtered_lognorm.h5ad')
adata
```

Example 8 (unknown):
```unknown
AnnData object with n_obs × n_vars = 61075 × 17994
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'leiden_0.2', 'leiden_0.4', 'leiden_0.6', 'leiden_1', 'leiden_1.2', 'leiden_1.5', 'leiden_2'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1.2', 'leiden_1.2_colors', 'leiden_1.5', 'leiden_1.5_colors', 'leiden_1_colors', 'leiden_2', 'leiden_2_colors', 'log1p', 'neighbors', 'pca', 'spatial', 'umap'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 9 (python):
```python
# read in
midgrain_markers = pd.read_csv('/nfs/team205/kk18/notebooks/Foetal/VisiumHD/re-seq_mpp0p3/sc_midgrain_markers.csv',
                               index_col=0
                              )
# remove the markers of celltype of interest
midgrain_markers = midgrain_markers[midgrain_markers['group']!=celltype]
midgrain_markers
```

Example 10 (unknown):
```unknown
'GATA2' in list(midgrain_markers['names'])
```

Example 11 (python):
```python
# remove
mask = [x not in list(midgrain_markers['names']) for x in adata.var_names]
bdata = adata[:,mask]
bdata
```

Example 12 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 61075 × 16746
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'leiden_0.2', 'leiden_0.4', 'leiden_0.6', 'leiden_1', 'leiden_1.2', 'leiden_1.5', 'leiden_2'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1.2', 'leiden_1.2_colors', 'leiden_1.5', 'leiden_1.5_colors', 'leiden_1_colors', 'leiden_2', 'leiden_2_colors', 'log1p', 'neighbors', 'pca', 'spatial', 'umap'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 13 (python):
```python
# HVGs, using the object which doesn't have midgrain markers
sc.pp.highly_variable_genes(bdata, flavor='seurat', min_mean=0.0125, max_mean=3, min_disp=0.5)
print(f'number of hvgs: {sum(bdata.var.highly_variable)}')
# PCA using the degected HVGs
sc.tl.pca(bdata, use_highly_variable=True)
adata.obsm['X_pca'] = bdata.obsm['X_pca'].copy()
# create neighbour graph
sc.pp.neighbors(adata,n_neighbors=15,use_rep='X_pca')
# visualisation with umap
sc.tl.umap(adata,min_dist=0.5)
sc.pl.embedding(adata,basis='X_umap',color=['bin_count','total_counts','n_genes_by_counts',"conf_score_midmod2fine","celltypist_midmod2fine",],
                wspace=0.2,cmap='RdPu',vmax='p100',ncols=5)
# leiden clustering
resolutions_list = [0.2,0.4,0.6,1,1.2,1.5,2]
for resolution in resolutions_list:
    sc.tl.leiden(adata,resolution=resolution,key_added=f'leiden_{str(resolution)}',n_iterations=2)
sc.pl.embedding(adata,basis='X_umap',
                color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                wspace=0.3)
```

Example 14 (unknown):
```unknown
number of hvgs: 2241
```

Example 15 (python):
```python
'HEY2' in adata.var_names
```

Example 16 (python):
```python
markers = {
    'VCSprox':['TBX3','PRDM16','NIBAN1','PHACTR1','FGF18'], # 'PRDM16','NIBAN1','PHACTR1','FGF18' are the markers compared to VCSdistal
    'VCSdist':['SEMA3A','IRX2','IRX3','OPCML','NTN1','EPHA4','GABRB2'],
    'Trab':['ADAMTSL1','SGCD','CSMD1'],
    'Compact':['TBX20','HEY2'],
    'Cycling':['MKI67','TOP2A']
}

genes = []
for g in markers.values():
    genes = genes + [x for x in g if (x in adata.var_names)&(x not in genes)]
for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in adata.var_names]
len(genes)
```

Example 17 (python):
```python
sc.pl.embedding(adata,basis='X_umap',color=genes, size=10,
               ncols=6, wspace=0.2,cmap='RdPu',vmax='p99.7')
```

Example 18 (unknown):
```unknown
resolution_sel = 1
```

Example 19 (python):
```python
markers_others = {
    'CM': ['TTN', 'TNNT2','MYH6',
        'MYH7', 'MYL2', 'FHL2', # vCN
            'NPPA', 'MYL7', 'MYL4'], #aCM
    'FB': ['DCN', 'GCN', 'PDGFRA','COL1A1','COL1A2'],
    'EC': ['VWF', 'PECAM1', 'CDH5','RGCC', 'FABP5',],
    'Peri': ['RGS5', 'ABCC9', 'KCNJ8'],
    'SMC': ['MYH11', 'TAGLN', 'ACTA2'],
    'Meso': ['MSLN', 'WT1', 'BNC1'],
    'Neuro': ['PLP1', 'NRXN1', 'NRXN3','PRPH', 'NEFL', 'NEFM', 'NEFH', 'STMN2', 'SST', 'SLC10A4', 'SLC18A3', 'TH', 'NPY'],
    'Adipo': ['GPAM', 'FASN', 'LEP'],
    'Myelo': ['CD14', 'C1QA', 'CD68','LYVE1','TIMD4'],
    'Lymph': ['CD8A', 'IL7R', 'CD40LG'],
    'B': ['CD19','SDC1','IGHG1','IGHA1'],
    'Mast': ['KIT', 'CPA3'],
}

for key in markers_others.keys():
    markers_others[key]=[x for x in markers_others[key] if x in adata.var_names]

sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
sc.pl.dotplot(adata,
              markers_others, 
              groupby=f'leiden_{str(resolution_sel)}',
              dendrogram=True,
              standard_scale="var",
              color_map="Reds",
              swap_axes=False,
             )
```

Example 20 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: CM, FB, EC, etc.
```

Example 21 (python):
```python
sc.tl.dendrogram(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.pl.dotplot(adata,
              var_names=markers,
              groupby=f'leiden_{str(resolution_sel)}',
              standard_scale='var',
              dendrogram=True
             )
```

Example 22 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: VCSprox, VCSdist, Trab, etc.
```

Example 23 (python):
```python
# DEG analysis
print(adata.X.data[:5])
adata.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata, dendrogram=True, standard_scale='var', return_fig=True)
dp.add_totals().show()
```

Example 24 (unknown):
```unknown
[1.47180343 1.33566585 1.26948215 1.24032132 1.26104042]
```

Example 25 (python):
```python
# DEG analysis
print(adata.X.data[:5])
adata.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata, dendrogram=True, standard_scale='var', return_fig=True, n_genes=5)
dp.add_totals().show()
```

Example 26 (unknown):
```unknown
[1.47180343 1.33566585 1.26948215 1.24032132 1.26104042]
```

Example 27 (python):
```python
sc.pl.embedding(adata,basis='X_umap',
                color=f'leiden_{str(resolution_sel)}',
                wspace=0.3, legend_loc='on data')
```

Example 28 (python):
```python
sc.pl.embedding(adata,basis='X_umap',
                color=['celltypist_midmod2fine', 'conf_score_midmod2fine'],
                wspace=0.7)
```

Example 29 (python):
```python
# umap per celltype to highlight
group_col = 'celltypist_midmod2fine'
groups_to_plot = list(set(adata.obs[group_col]))

plt.rcParams['axes.grid'] = False
fig,axs=plt.subplots(nrows=2,ncols=4,figsize=(15,7))
# Flatten the axs array for easier iteration in case of multiple rows and columns
axs_flat = axs.flatten()

for i,grp in enumerate(groups_to_plot):
    sc.pl.umap(adata, color=[group_col], groups=grp, 
               size=10, frameon=False,
               show=False, ax=axs_flat[i], legend_loc=None, legend_fontsize=5,
               title=grp
              )
# Turn off any remaining subplots
for j in range(len(groups_to_plot), len(axs_flat)):
    fig.delaxes(axs_flat[j])

# fig.suptitle('Main Title for the Figure')
fig.tight_layout()
```

Example 30 (python):
```python
sc.pl.violin(adata,keys=['total_counts','bin_count'],groupby=f'leiden_{str(resolution_sel)}')
```

Example 31 (unknown):
```unknown
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 32 (python):
```python
# annotate
adata.obs['fine_grain_manual'] = adata.obs[f'leiden_{str(resolution_sel)}'].copy()
adata.obs.replace({'fine_grain_manual':{
    '0':'VentricularCardiomyocytesCompact',
    '1':'unassigned',
    '2':'uncertain',
    '3':'uncertain',
    '4':'uncertain',
    '5':'VentricularConductionSystemProximal',
    '6':'VentricularConductionSystemProximal',
    '7':'VentricularCardiomyocytes_DoubletsWithErythrocytes',
    '8':'VentricularCardiomyocytes_Doublets',
    '9':'uncertain',
}},inplace=True)
adata.obs['fine_grain_manual'].value_counts()
```

Example 33 (unknown):
```unknown
fine_grain_manual
uncertain                                             27579
VentricularCardiomyocytesCompact                      13082
unassigned                                            12767
VentricularConductionSystemProximal                    5902
VentricularCardiomyocytes_DoubletsWithErythrocytes     1723
VentricularCardiomyocytes_Doublets                       22
Name: count, dtype: int64
```

Example 34 (unknown):
```unknown
'0':'VentricularCardiomyocytesCompact',
    '1':'VentricularCardiomyocytesCompact',
    '2':'VentricularCardiomyocytesTrabeculated',
    '3':'VentricularConductionSystem',
```

Example 35 (python):
```python
ad = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/Feb28ObjectRaw_finegrain_updated.h5ad',backed='r')
set(ad.obs['fine_grain'])
```

Example 36 (unknown):
```unknown
{'AtrialCardiomyocytesCycling',
 'AtrialCardiomyocytesLeft',
 'AtrialCardiomyocytesRight',
 'AtrioventricularNodePacemakerCells',
 'BCells',
 'BCellsMS4A1pos',
 'ChromaffinCells',
 'CoronaryArterialEndothelialCells',
 'CoronaryCapillaryEndothelialCells',
 'CoronaryPericytes',
 'CoronarySmoothMuscleCells',
 'CoronaryVenousEndothelialCells',
 'CoronaryVesselAdventitialFibroblasts',
 'DendriticCellsMature',
 'DendriticCellsType1',
 'DuctusArteriosusSmoothMuscleCells',
 'EndocardialCells',
 'EndocardialCushionCells',
 'EpicardiumDerivedCells',
 'GreatVesselAdventitialFibroblasts',
 'GreatVesselArterialEndothelialCells',
 'GreatVesselSmoothMuscleCells',
 'GreatVesselVenousEndothelialCells',
 'InnateLymphoidCells',
 'LymphNodeFibroblasticReticularCells',
 'LymphaticEndothelialCells',
 'MacrophagesATF3pos',
 'MacrophagesCX3CR1pos',
 'MacrophagesLYVE1pos',
 'MacrophagesTIMD4pos',
 'MastCells',
 'Megakaryocytes',
 'MesothelialEpicardialCells',
 'MonocyteDerivedCells',
 'Monocytes',
 'MonocytesMPOpos',
 'MyocardialInterstitialFibroblasts',
 'Myofibroblasts',
 'NaturalKillerCells',
 'NeuronPrecursors',
 'ParasympatheticNeurons',
 'PericardialCellsFibrous',
 'PericardialCellsIntermediate',
 'PericardialCellsParietal',
 'PlasmacytoidDendriticCells',
 'ProBCells',
 'SchwannCellPrecursors',
 'SchwannCells',
 'SinoatrialNodePacemakerCells',
 'SubEpicardialFibroblasts',
 'SympatheticNeurons',
 'TCellsCD4pos',
 'TCellsCD8pos',
 'TregsCD4pos',
 'ValveEndothelialCells',
 'ValveInterstitialCells',
 'VentricularCardiomyocytesCycling',
 'VentricularCardiomyocytesLeftCompact',
 'VentricularCardiomyocytesLeftTrabeculated',
 'VentricularCardiomyocytesRightCompact',
 'VentricularCardiomyocytesRightTrabeculated',
 'VentricularConductionSystemDistal',
 'VentricularConductionSystemProximal'}
```

Example 37 (python):
```python
sc.pl.embedding(adata,basis='X_umap',
                color='fine_grain_manual',
                frameon=False,
                wspace=0.3)
```

Example 38 (python):
```python
sc.tl.dendrogram(adata,groupby='fine_grain_manual')
sc.pl.dotplot(adata,
              var_names=markers,
              groupby='fine_grain_manual',
              standard_scale='var',
              dendrogram=True
             )
```

Example 39 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: VentricularCardiomyocytesCompact, unassigned, uncertain, etc.
var_group_labels: VCSprox, VCSdist, Trab, etc.
```

Example 40 (python):
```python
set(adata.obs['fine_grain_manual'])
```

Example 41 (unknown):
```unknown
{'VentricularCardiomyocytesCompact',
 'VentricularCardiomyocytes_Doublets',
 'VentricularCardiomyocytes_DoubletsWithErythrocytes',
 'VentricularConductionSystemProximal',
 'unassigned',
 'uncertain'}
```

Example 42 (python):
```python
adata_sub = adata[adata.obs['fine_grain_manual'].isin(['uncertain'])]

# drop leiden clustering results
resolutions_list = [0.2,0.4,0.6,1, 1.2,1.5,2]
for resolution in resolutions_list:
    adata_sub.obs.drop(f'leiden_{str(resolution)}',axis=1,inplace=True)
adata_sub
```

Example 43 (unknown):
```unknown
AnnData object with n_obs × n_vars = 27579 × 17994
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'fine_grain_manual'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'hvg', 'leiden_0.2', 'leiden_0.4', 'leiden_0.6', 'leiden_1', 'leiden_1.2', 'leiden_1.5', 'leiden_1.5_colors', 'leiden_1_colors', 'leiden_2', 'leiden_2_colors', 'log1p', 'neighbors', 'pca', 'spatial', 'umap', 'dendrogram_leiden_1', 'rank_genes_groups', 'fine_grain_manual_colors', 'dendrogram_fine_grain_manual'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 44 (python):
```python
# remove mid_grain markers
mask = [x not in list(midgrain_markers['names']) for x in adata_sub.var_names]
bdata = adata_sub[:,mask]
bdata
```

Example 45 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 27579 × 16746
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'fine_grain_manual'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'hvg', 'leiden_0.2', 'leiden_0.4', 'leiden_0.6', 'leiden_1', 'leiden_1.2', 'leiden_1.5', 'leiden_1.5_colors', 'leiden_1_colors', 'leiden_2', 'leiden_2_colors', 'log1p', 'neighbors', 'pca', 'spatial', 'umap', 'dendrogram_leiden_1', 'rank_genes_groups', 'fine_grain_manual_colors', 'dendrogram_fine_grain_manual'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 46 (python):
```python
# HVGs, using the object which doesn't have midgrain markers
sc.pp.highly_variable_genes(bdata, flavor='seurat', min_mean=0.0125, max_mean=3, min_disp=0.5)
print(f'number of hvgs: {sum(bdata.var.highly_variable)}')
# PCA using the degected HVGs
sc.tl.pca(bdata, use_highly_variable=True)
adata_sub.obsm['X_pca'] = bdata.obsm['X_pca'].copy()
# create neighbour graph
sc.pp.neighbors(adata_sub,n_neighbors=15,use_rep='X_pca')
```

Example 47 (unknown):
```unknown
number of hvgs: 2182
```

Example 48 (python):
```python
sc.tl.umap(adata_sub,min_dist=0.1)
```

Example 49 (python):
```python
%%time
# reclustering
# leiden clustering
resolutions_list = [0.2,0.4,0.6,1, 1.2,1.5,2]
for resolution in resolutions_list:
    sc.tl.leiden(adata_sub,resolution=resolution,key_added=f'leiden_{str(resolution)}',n_iterations=2)
sc.pl.embedding(adata_sub,basis='X_umap',
                color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                wspace=0.3)
```

Example 50 (unknown):
```unknown
CPU times: user 17.1 s, sys: 466 ms, total: 17.5 s
Wall time: 18.1 s
```

Example 51 (python):
```python
markers = {
    # 'VCSprox':['TBX3','PRDM16','NIBAN1','PHACTR1','FGF18'], # 'PRDM16','NIBAN1','PHACTR1','FGF18' are the markers compared to VCSdistal
    'VCSdist':['SEMA3A','IRX2','IRX3','OPCML','NTN1','EPHA4','GABRB2'],
    'Trab':['ADAMTSL1','SGCD','CSMD1'],
    'Compact':['TBX20','HEY2'],
    'Cycling':['MKI67','TOP2A']
}

genes = []
for g in markers.values():
    genes = genes + [x for x in g if (x in adata.var_names)&(x not in genes)]
for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in adata.var_names]
len(genes)
```

Example 52 (python):
```python
sc.pl.embedding(adata_sub,basis='X_umap',color=genes, size=10,
               ncols=6, wspace=0.2,cmap='RdPu',vmax='p99.7')
```

Example 53 (python):
```python
adata_sub_resolution_sel = 0.6
```

Example 54 (python):
```python
markers_others = {
    'CM': ['TTN', 'TNNT2','MYH6',
        'MYH7', 'MYL2', 'FHL2', # vCN
            'NPPA', 'MYL7', 'MYL4'], #aCM
    'FB': ['DCN', 'GCN', 'PDGFRA','COL1A1','COL1A2'],
    'EC': ['VWF', 'PECAM1', 'CDH5','RGCC', 'FABP5',],
    'Peri': ['RGS5', 'ABCC9', 'KCNJ8'],
    'SMC': ['MYH11', 'TAGLN', 'ACTA2'],
    'Meso': ['MSLN', 'WT1', 'BNC1'],
    'Neuro': ['PLP1', 'NRXN1', 'NRXN3','PRPH', 'NEFL', 'NEFM', 'NEFH', 'STMN2', 'SST', 'SLC10A4', 'SLC18A3', 'TH', 'NPY'],
    'Adipo': ['GPAM', 'FASN', 'LEP'],
    'Myelo': ['CD14', 'C1QA', 'CD68','LYVE1','TIMD4'],
    'Lymph': ['CD8A', 'IL7R', 'CD40LG'],
    'B': ['CD19','SDC1','IGHG1','IGHA1'],
    'Mast': ['KIT', 'CPA3'],
}

for key in markers_others.keys():
    markers_others[key]=[x for x in markers_others[key] if x in adata_sub.var_names]

sc.tl.dendrogram(adata_sub, groupby=f'leiden_{str(adata_sub_resolution_sel)}')
sc.pl.dotplot(adata_sub,
              markers_others, 
              groupby=f'leiden_{str(adata_sub_resolution_sel)}',
              dendrogram=True,
              standard_scale="var",
              color_map="Reds",
              swap_axes=False,
             )
```

Example 55 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2
var_group_labels: CM, FB, EC, etc.
```

Example 56 (python):
```python
sc.tl.dendrogram(adata_sub,groupby=f'leiden_{str(adata_sub_resolution_sel)}')
sc.pl.dotplot(adata_sub,
              var_names=markers,
              groupby=f'leiden_{str(adata_sub_resolution_sel)}',
              standard_scale='var',
              dendrogram=True
             )
```

Example 57 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2
var_group_labels: VCSdist, Trab, Compact, etc.
```

Example 58 (python):
```python
# DEG analysis
print(adata_sub.X.data[:5])
adata_sub.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata_sub,groupby=f'leiden_{str(adata_sub_resolution_sel)}')
sc.tl.dendrogram(adata_sub, groupby=f'leiden_{str(adata_sub_resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata_sub, dendrogram=True, standard_scale='var', return_fig=True)
dp.add_totals().show()
```

Example 59 (unknown):
```unknown
[2.16681224 1.99582797 1.93329773 1.91105105 1.8337824 ]
```

Example 60 (python):
```python
# DEG analysis
print(adata_sub.X.data[:5])
adata_sub.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata_sub,groupby=f'leiden_{str(adata_sub_resolution_sel)}')
sc.tl.dendrogram(adata_sub, groupby=f'leiden_{str(adata_sub_resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata_sub, dendrogram=True, standard_scale='var', return_fig=True, n_genes=5)
dp.add_totals().show()
```

Example 61 (unknown):
```unknown
[2.16681224 1.99582797 1.93329773 1.91105105 1.8337824 ]
```

Example 62 (python):
```python
sc.pl.embedding(adata_sub,basis='X_umap',
                color=f'leiden_{str(adata_sub_resolution_sel)}',
                wspace=0.3, legend_loc='on data')
```

Example 63 (python):
```python
# annotate
adata_sub.obs['fine_grain_manual'] = adata_sub.obs[f'leiden_{str(adata_sub_resolution_sel)}'].copy()
adata_sub.obs.replace({'fine_grain_manual':{
    '0':'VentricularCardiomyocytesCompact', # cluster 0 has more FB and vasculature signatures compared to cluster 1. Potentially there's some level of contamination. Which also suggests these population are more close to FB/Vasc.
    '1':'VentricularCardiomyocytesCompact',
    '2':'VentricularConductionSystemDistal_VentricularCardiomyocytesTrabeculated',
}},inplace=True)
adata_sub.obs['fine_grain_manual'].value_counts()
```

Example 64 (unknown):
```unknown
fine_grain_manual
VentricularCardiomyocytesCompact                                           19888
VentricularConductionSystemDistal_VentricularCardiomyocytesTrabeculated     7691
Name: count, dtype: int64
```

Example 65 (python):
```python
sc.pl.embedding(adata_sub,basis='X_umap',
                color='fine_grain_manual',
                frameon=False,
                wspace=0.3)
```

Example 66 (python):
```python
sc.tl.dendrogram(adata_sub,groupby=f'leiden_{str(adata_sub_resolution_sel)}')
sc.pl.dotplot(adata_sub,
              var_names=markers,
              groupby='fine_grain_manual',
              standard_scale='var',
              dendrogram=True
             )
```

Example 67 (unknown):
```unknown
WARNING: Dendrogram not added. Dendrogram is added only when the number of categories to plot > 2
```

Example 68 (python):
```python
adata_sub2 = adata_sub[adata_sub.obs['fine_grain_manual'].isin(['VentricularConductionSystemDistal_VentricularCardiomyocytesTrabeculated'])]

# drop leiden clustering results
resolutions_list = [0.2,0.4,0.6,1, 1.2,1.5,2]
for resolution in resolutions_list:
    adata_sub2.obs.drop(f'leiden_{str(resolution)}',axis=1,inplace=True)
adata_sub2
```

Example 69 (unknown):
```unknown
AnnData object with n_obs × n_vars = 7691 × 17994
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'fine_grain_manual'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'hvg', 'leiden_0.2', 'leiden_0.4', 'leiden_0.6', 'leiden_1', 'leiden_1.2', 'leiden_1.5', 'leiden_1.5_colors', 'leiden_1_colors', 'leiden_2', 'leiden_2_colors', 'log1p', 'neighbors', 'pca', 'spatial', 'umap', 'dendrogram_leiden_1', 'rank_genes_groups', 'fine_grain_manual_colors', 'dendrogram_fine_grain_manual', 'leiden_0.2_colors', 'leiden_0.4_colors', 'leiden_0.6_colors', 'leiden_1.2_colors', 'dendrogram_leiden_0.6'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 70 (python):
```python
# remove mid_grain markers
mask = [x not in list(midgrain_markers['names']) for x in adata_sub2.var_names]
bdata = adata_sub2[:,mask]
bdata
```

Example 71 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 7691 × 16746
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'fine_grain_manual'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'hvg', 'leiden_0.2', 'leiden_0.4', 'leiden_0.6', 'leiden_1', 'leiden_1.2', 'leiden_1.5', 'leiden_1.5_colors', 'leiden_1_colors', 'leiden_2', 'leiden_2_colors', 'log1p', 'neighbors', 'pca', 'spatial', 'umap', 'dendrogram_leiden_1', 'rank_genes_groups', 'fine_grain_manual_colors', 'dendrogram_fine_grain_manual', 'leiden_0.2_colors', 'leiden_0.4_colors', 'leiden_0.6_colors', 'leiden_1.2_colors', 'dendrogram_leiden_0.6'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 72 (python):
```python
# HVGs, using the object which doesn't have midgrain markers
sc.pp.highly_variable_genes(bdata, flavor='seurat', min_mean=0.0125, max_mean=3, min_disp=0.5)
print(f'number of hvgs: {sum(bdata.var.highly_variable)}')
# PCA using the degected HVGs
sc.tl.pca(bdata, use_highly_variable=True)
adata_sub2.obsm['X_pca'] = bdata.obsm['X_pca'].copy()
# create neighbour graph
sc.pp.neighbors(adata_sub2,n_neighbors=15,use_rep='X_pca')
```

Example 73 (unknown):
```unknown
number of hvgs: 3227
```

Example 74 (python):
```python
sc.tl.umap(adata_sub2,min_dist=0.1)
```

Example 75 (python):
```python
%%time
# reclustering
# leiden clustering
resolutions_list = [0.2,0.4,0.6,1, 1.2,1.5,2]
for resolution in resolutions_list:
    sc.tl.leiden(adata_sub2,resolution=resolution,key_added=f'leiden_{str(resolution)}',n_iterations=2)
sc.pl.embedding(adata_sub2,basis='X_umap',
                color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                wspace=0.3)
```

Example 76 (unknown):
```unknown
CPU times: user 6.04 s, sys: 148 ms, total: 6.19 s
Wall time: 6.37 s
```

Example 77 (python):
```python
markers = {
    # 'VCSprox':['TBX3','PRDM16','NIBAN1','PHACTR1','FGF18'], # 'PRDM16','NIBAN1','PHACTR1','FGF18' are the markers compared to VCSdistal
    'VCSdist':['SEMA3A','IRX2','IRX3','OPCML','NTN1','EPHA4','GABRB2'],
    # 'Trab':['ADAMTSL1','SGCD','CSMD1'],
    # 'Compact':['TBX20','HEY2'],
    'Cycling':['MKI67','TOP2A']
}

genes = []
for g in markers.values():
    genes = genes + [x for x in g if (x in adata_sub2.var_names)&(x not in genes)]
for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in adata_sub2.var_names]
len(genes)
```

Example 78 (python):
```python
sc.pl.embedding(adata_sub2,basis='X_umap',color=genes, size=10,
               ncols=6, wspace=0.2,cmap='RdPu',vmax='p99.7')
```

Example 79 (python):
```python
adata_sub_resolution_sel = 1
```

Example 80 (python):
```python
markers_others = {
    'CM': ['TTN', 'TNNT2','MYH6',
        'MYH7', 'MYL2', 'FHL2', # vCN
            'NPPA', 'MYL7', 'MYL4'], #aCM
    'FB': ['DCN', 'GCN', 'PDGFRA','COL1A1','COL1A2'],
    'EC': ['VWF', 'PECAM1', 'CDH5','RGCC', 'FABP5',],
    'Peri': ['RGS5', 'ABCC9', 'KCNJ8'],
    'SMC': ['MYH11', 'TAGLN', 'ACTA2'],
    'Meso': ['MSLN', 'WT1', 'BNC1'],
    'Neuro': ['PLP1', 'NRXN1', 'NRXN3','PRPH', 'NEFL', 'NEFM', 'NEFH', 'STMN2', 'SST', 'SLC10A4', 'SLC18A3', 'TH', 'NPY'],
    'Adipo': ['GPAM', 'FASN', 'LEP'],
    'Myelo': ['CD14', 'C1QA', 'CD68','LYVE1','TIMD4'],
    'Lymph': ['CD8A', 'IL7R', 'CD40LG'],
    'B': ['CD19','SDC1','IGHG1','IGHA1'],
    'Mast': ['KIT', 'CPA3'],
}

for key in markers_others.keys():
    markers_others[key]=[x for x in markers_others[key] if x in adata_sub2.var_names]

sc.tl.dendrogram(adata_sub2, groupby=f'leiden_{str(adata_sub_resolution_sel)}')
sc.pl.dotplot(adata_sub2,
              markers_others, 
              groupby=f'leiden_{str(adata_sub_resolution_sel)}',
              dendrogram=True,
              standard_scale="var",
              color_map="Reds",
              swap_axes=False,
             )
```

Example 81 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: CM, FB, EC, etc.
```

Example 82 (python):
```python
sc.tl.dendrogram(adata_sub2,groupby=f'leiden_{str(adata_sub_resolution_sel)}')
sc.pl.dotplot(adata_sub2,
              var_names=markers,
              groupby=f'leiden_{str(adata_sub_resolution_sel)}',
              standard_scale='var',
              dendrogram=True
             )
```

Example 83 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: VCSdist, Cycling
```

Example 84 (python):
```python
# DEG analysis
print(adata_sub2.X.data[:5])
adata_sub2.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata_sub2,groupby=f'leiden_{str(adata_sub_resolution_sel)}')
sc.tl.dendrogram(adata_sub2, groupby=f'leiden_{str(adata_sub_resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata_sub2, dendrogram=True, standard_scale='var', return_fig=True)
dp.add_totals().show()
```

Example 85 (unknown):
```unknown
[2.71216227 1.03967184 1.35760811 1.70411582 1.28981545]
```

Example 86 (python):
```python
# DEG analysis
print(adata_sub2.X.data[:5])
adata_sub2.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata_sub2,groupby=f'leiden_{str(adata_sub_resolution_sel)}')
sc.tl.dendrogram(adata_sub2, groupby=f'leiden_{str(adata_sub_resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata_sub2, dendrogram=True, standard_scale='var', return_fig=True, n_genes=5)
dp.add_totals().show()
```

Example 87 (unknown):
```unknown
[2.71216227 1.03967184 1.35760811 1.70411582 1.28981545]
```

Example 88 (python):
```python
sc.pl.embedding(adata_sub2,basis='X_umap',
                color=f'leiden_{str(adata_sub_resolution_sel)}',
                wspace=0.3, legend_loc='on data')
```

Example 89 (python):
```python
# annotate
adata_sub2.obs['fine_grain_manual'] = adata_sub2.obs[f'leiden_{str(adata_sub_resolution_sel)}'].copy()
adata_sub2.obs.replace({'fine_grain_manual':{
    '0':'VentricularCardiomyocytesTrabeculated',
    '1':'VentricularConductionSystemDistal',
    '2':'VentricularCardiomyocytesTrabeculated',
    '3':'VentricularCardiomyocytesTrabeculated',
    '4':'VentricularCardiomyocytesTrabeculated',
    '5':'VentricularCardiomyocytesTrabeculated',
    '6':'VentricularCardiomyocytesTrabeculated',
    '7':'VentricularCardiomyocytesTrabeculated',
    '8':'VentricularCardiomyocytesTrabeculated',
    '9':'VentricularCardiomyocytesTrabeculated',
}},inplace=True)
adata_sub2.obs['fine_grain_manual'].value_counts()
```

Example 90 (unknown):
```unknown
fine_grain_manual
VentricularCardiomyocytesTrabeculated    6349
VentricularConductionSystemDistal        1342
Name: count, dtype: int64
```

Example 91 (python):
```python
sc.pl.embedding(adata_sub2,basis='X_umap',
                color='fine_grain_manual',
                frameon=False,
                wspace=0.3)
```

Example 92 (python):
```python
sc.tl.dendrogram(adata_sub2,groupby=f'leiden_{str(adata_sub_resolution_sel)}')
sc.pl.dotplot(adata_sub2,
              var_names=markers,
              groupby='fine_grain_manual',
              standard_scale='var',
              dendrogram=True
             )
```

Example 93 (unknown):
```unknown
WARNING: Dendrogram not added. Dendrogram is added only when the number of categories to plot > 2
```

Example 94 (python):
```python
# add the annotation to the original adata
adata.obs['fine_grain_manual'] = adata.obs['fine_grain_manual'].astype('str')
adata.obs.loc[adata_sub.obs_names,'fine_grain_manual'] = adata_sub.obs['fine_grain_manual'].astype('str').copy()
adata.obs.loc[adata_sub2.obs_names,'fine_grain_manual'] = adata_sub2.obs['fine_grain_manual'].astype('str').copy()
adata.obs['fine_grain_manual'].value_counts()
```

Example 95 (unknown):
```unknown
fine_grain_manual
VentricularCardiomyocytesCompact                      32970
unassigned                                            12767
VentricularCardiomyocytesTrabeculated                  6349
VentricularConductionSystemProximal                    5902
VentricularCardiomyocytes_DoubletsWithErythrocytes     1723
VentricularConductionSystemDistal                      1342
VentricularCardiomyocytes_Doublets                       22
Name: count, dtype: int64
```

Example 96 (python):
```python
sc.pl.embedding(adata,basis='X_umap',
                color='fine_grain_manual',
                frameon=False,
                wspace=0.3)
```

Example 97 (unknown):
```unknown
... storing 'fine_grain_manual' as categorical
```

Example 98 (python):
```python
markers = {
    'VCSprox':['TBX3','PRDM16','NIBAN1','PHACTR1','FGF18'], # 'PRDM16','NIBAN1','PHACTR1','FGF18' are the markers compared to VCSdistal
    'VCSdist':['SEMA3A','IRX2','IRX3','OPCML','NTN1','EPHA4','GABRB2'],
    'Trab':['ADAMTSL1','SGCD','CSMD1'],
    'Compact':['TBX20','HEY2'],
    'Cycling':['MKI67','TOP2A']
}

genes = []
for g in markers.values():
    genes = genes + [x for x in g if (x in adata_sub2.var_names)&(x not in genes)]
for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in adata_sub2.var_names]
len(genes)
```

Example 99 (python):
```python
sc.tl.dendrogram(adata,groupby='fine_grain_manual')
sc.pl.dotplot(adata,
              var_names=markers,
              groupby='fine_grain_manual',
              standard_scale='var',
              dendrogram=True
             )
```

Example 100 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: VentricularCardiomyocytesCompact, VentricularCardiomyocytesTrabeculated, VentricularCardiomyocytes_Doublets, etc.
var_group_labels: VCSprox, VCSdist, Trab, etc.
```

Example 101 (python):
```python
# save
adata.write(f'{adata_dir}/{sample_id}_{celltype}_b2c_cells_filtered_lognorm.h5ad')
print(f'{adata_dir}/{sample_id}_{celltype}_b2c_cells_filtered_lognorm.h5ad')
```

Example 102 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/subsets/HEA_FOET14880396_VentricularCardiomyocytes_b2c_cells_filtered_lognorm.h5ad
```

---

## Variables¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/VisiumHD/no7_manual-annotation_MuralCells.html

**Contents:**
- Variables¶
- Read in data¶
- Remove mid-grain markers from the feature¶
- PCA, Clustering, UMAP¶
- Markers¶
- Subset "uncertain" cells¶
- Add the annotation to the original adata¶
- Save¶

2545 rows × 7 columns

--> cluster 5 has low total_counts and bin_counts. and no specific marker expression --> will annotated as "unassigned"

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os

import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 

import celltypist
```

Example 2 (python):
```python
# import session_info
# session_info.show()
```

Example 3 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 4 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 5 (python):
```python
adata_dir = '/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/subsets'
sample_id = 'HEA_FOET14880396'
celltype = 'MuralCells'
```

Example 6 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk/Aug2024'
table_dir = '/nfs/team205/heart/fetal_ms_figures/tables/Aug2024'
```

Example 7 (python):
```python
# read in
adata = sc.read_h5ad(f'{adata_dir}/{sample_id}_{celltype}_b2c_cells_filtered_lognorm.h5ad')
adata
```

Example 8 (unknown):
```unknown
AnnData object with n_obs × n_vars = 10785 × 17693
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'leiden_0.2', 'leiden_0.4', 'leiden_0.6', 'leiden_1', 'leiden_1.2', 'leiden_1.5', 'leiden_2'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1.2', 'leiden_1.2_colors', 'leiden_1.5', 'leiden_1.5_colors', 'leiden_1_colors', 'leiden_2', 'leiden_2_colors', 'log1p', 'neighbors', 'pca', 'spatial', 'umap'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 9 (python):
```python
# read in
midgrain_markers = pd.read_csv('/nfs/team205/kk18/notebooks/Foetal/VisiumHD/re-seq_mpp0p3/sc_midgrain_markers.csv',
                               index_col=0
                              )
# remove the markers of celltype of interest
midgrain_markers = midgrain_markers[midgrain_markers['group']!=celltype]
midgrain_markers
```

Example 10 (unknown):
```unknown
'GATA2' in list(midgrain_markers['names'])
```

Example 11 (python):
```python
# remove
mask = [x not in list(midgrain_markers['names']) for x in adata.var_names]
bdata = adata[:,mask]
bdata
```

Example 12 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 10785 × 16421
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'leiden_0.2', 'leiden_0.4', 'leiden_0.6', 'leiden_1', 'leiden_1.2', 'leiden_1.5', 'leiden_2'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1.2', 'leiden_1.2_colors', 'leiden_1.5', 'leiden_1.5_colors', 'leiden_1_colors', 'leiden_2', 'leiden_2_colors', 'log1p', 'neighbors', 'pca', 'spatial', 'umap'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 13 (python):
```python
# HVGs, using the object which doesn't have midgrain markers
sc.pp.highly_variable_genes(bdata, flavor='seurat', min_mean=0.0125, max_mean=3, min_disp=0.5)
print(f'number of hvgs: {sum(bdata.var.highly_variable)}')
# PCA using the degected HVGs
sc.tl.pca(bdata, use_highly_variable=True)
adata.obsm['X_pca'] = bdata.obsm['X_pca'].copy()
# create neighbour graph
sc.pp.neighbors(adata,n_neighbors=15,use_rep='X_pca')
# visualisation with umap
sc.tl.umap(adata,min_dist=0.5)
sc.pl.embedding(adata,basis='X_umap',color=['bin_count','total_counts','n_genes_by_counts',"conf_score_midmod2fine","celltypist_midmod2fine",],
                wspace=0.2,cmap='RdPu',vmax='p100',ncols=5)
# leiden clustering
resolutions_list = [0.2,0.4,0.6,1,1.2,1.5,2]
for resolution in resolutions_list:
    sc.tl.leiden(adata,resolution=resolution,key_added=f'leiden_{str(resolution)}',n_iterations=2)
sc.pl.embedding(adata,basis='X_umap',
                color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                wspace=0.3)
```

Example 14 (unknown):
```unknown
number of hvgs: 3008
```

Example 15 (python):
```python
'HEY2' in adata.var_names
```

Example 16 (python):
```python
# read in markers identified using single-nucleus data
markers_df = pd.read_csv(f'/nfs/team205/kk18/notebooks/Foetal/VisiumHD/re-seq_mpp0p3/sc_{celltype}_finegrain_markers.csv')

mask1 = markers_df['group_mean']>1
mask2 = markers_df['logfoldchanges']>1
markers_df = markers_df[mask1&mask2]

markers_df
```

Example 17 (python):
```python
markers = {}
n_top = 10
for c in set(markers_df['group']):
    markers_df_sub = markers_df[markers_df['group']==c]
    markers_df_sub.sort_values('logfoldchanges',ascending=False)
    markers[c]=list(markers_df_sub[:n_top]['names'])

# manually add genes
markers['Cycling'] = ['MKI67','TOP2A'] 

genes = []
for g in markers.values():
    genes = genes + [x for x in g if (x in list(adata.var_names))&(x not in genes)]
for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in list(adata.var_names)]
len(genes)
```

Example 18 (python):
```python
sc.pl.embedding(adata,basis='X_umap',color=genes, size=10,
               ncols=6, wspace=0.2,cmap='RdPu',vmax='p99.7')
```

Example 19 (unknown):
```unknown
resolution_sel = 1
```

Example 20 (python):
```python
markers_others = {
    'CM': ['TTN', 'TNNT2','MYH6',
        'MYH7', 'MYL2', 'FHL2', # vCN
            'NPPA', 'MYL7', 'MYL4'], #aCM
    'FB': ['DCN', 'GCN', 'PDGFRA','COL1A1','COL1A2'],
    'EC': ['VWF', 'PECAM1', 'CDH5','RGCC', 'FABP5',],
    'Peri': ['RGS5', 'ABCC9', 'KCNJ8'],
    'SMC': ['MYH11', 'TAGLN', 'ACTA2'],
    'Meso': ['MSLN', 'WT1', 'BNC1'],
    'Neuro': ['PLP1', 'NRXN1', 'NRXN3','PRPH', 'NEFL', 'NEFM', 'NEFH', 'STMN2', 'SST', 'SLC10A4', 'SLC18A3', 'TH', 'NPY'],
    'Adipo': ['GPAM', 'FASN', 'LEP'],
    'Myelo': ['CD14', 'C1QA', 'CD68','LYVE1','TIMD4'],
    'Lymph': ['CD8A', 'IL7R', 'CD40LG'],
    'B': ['CD19','SDC1','IGHG1','IGHA1'],
    'Mast': ['KIT', 'CPA3'],
}

for key in markers_others.keys():
    markers_others[key]=[x for x in markers_others[key] if x in adata.var_names]

sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
sc.pl.dotplot(adata,
              markers_others, 
              groupby=f'leiden_{str(resolution_sel)}',
              dendrogram=True,
              standard_scale="var",
              color_map="Reds",
              swap_axes=False,
             )
```

Example 21 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: CM, FB, EC, etc.
```

Example 22 (python):
```python
sc.tl.dendrogram(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.pl.dotplot(adata,
              var_names=markers,
              groupby=f'leiden_{str(resolution_sel)}',
              standard_scale='var',
              dendrogram=True
             )
```

Example 23 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: CoronaryPericytes, DuctusArteriosusSmoothMuscleCells, GreatVesselSmoothMuscleCells, etc.
```

Example 24 (python):
```python
# DEG analysis
print(adata.X.data[:5])
adata.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata, dendrogram=True, standard_scale='var', return_fig=True)
dp.add_totals().show()
```

Example 25 (unknown):
```unknown
[2.35484271 1.76428034 1.58063847 1.74054927 1.3504131 ]
```

Example 26 (python):
```python
# DEG analysis
print(adata.X.data[:5])
adata.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata, dendrogram=True, standard_scale='var', return_fig=True, n_genes=5)
dp.add_totals().show()
```

Example 27 (unknown):
```unknown
[2.35484271 1.76428034 1.58063847 1.74054927 1.3504131 ]
```

Example 28 (python):
```python
sc.pl.embedding(adata,basis='X_umap',
                color=f'leiden_{str(resolution_sel)}',
                wspace=0.3, legend_loc='on data')
```

Example 29 (python):
```python
sc.pl.embedding(adata,basis='X_umap',
                color=['celltypist_midmod2fine', 'conf_score_midmod2fine'],
                wspace=0.7)
```

Example 30 (python):
```python
# umap per celltype to highlight
group_col = 'celltypist_midmod2fine'
groups_to_plot = list(set(adata.obs[group_col]))

plt.rcParams['axes.grid'] = False
fig,axs=plt.subplots(nrows=2,ncols=4,figsize=(15,7))
# Flatten the axs array for easier iteration in case of multiple rows and columns
axs_flat = axs.flatten()

for i,grp in enumerate(groups_to_plot):
    sc.pl.umap(adata, color=[group_col], groups=grp, 
               size=30, frameon=False,
               show=False, ax=axs_flat[i], legend_loc=None, legend_fontsize=5,
               title=grp
              )
# Turn off any remaining subplots
for j in range(len(groups_to_plot), len(axs_flat)):
    fig.delaxes(axs_flat[j])

# fig.suptitle('Main Title for the Figure')
fig.tight_layout()
```

Example 31 (python):
```python
sc.pl.violin(adata,keys=['total_counts','bin_count'],groupby=f'leiden_{str(resolution_sel)}')
```

Example 32 (unknown):
```unknown
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 33 (unknown):
```unknown
'MuralCells'
```

Example 34 (python):
```python
# annotate
adata.obs['fine_grain_manual'] = adata.obs[f'leiden_{str(resolution_sel)}'].copy()
adata.obs.replace({'fine_grain_manual':{
    '0':'MuralCells',
    '1':'MuralCells',
    '2':'MuralCells', 
    '3':'MuralCells',
    '4':'MuralCells',
    '5':'unassigned',
    '6':'MuralCells',
    '7':'MuralCells',
    '8':'MuralCells',
}},inplace=True)
adata.obs['fine_grain_manual'].value_counts()
```

Example 35 (unknown):
```unknown
fine_grain_manual
MuralCells    9969
unassigned     816
Name: count, dtype: int64
```

Example 36 (python):
```python
ad = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/Feb28ObjectRaw_finegrain_updated.h5ad',backed='r')
ad = ad[ad.obs['mid_grain']==celltype]
set(ad.obs['fine_grain'])
```

Example 37 (unknown):
```unknown
{'CoronaryPericytes',
 'CoronarySmoothMuscleCells',
 'DuctusArteriosusSmoothMuscleCells',
 'GreatVesselSmoothMuscleCells'}
```

Example 38 (python):
```python
sc.pl.embedding(adata,basis='X_umap',
                color='fine_grain_manual',
                frameon=False,
                wspace=0.3)
```

Example 39 (python):
```python
adata_sub = adata[adata.obs['fine_grain_manual'].isin(['MuralCells'])]

# drop leiden clustering results
resolutions_list = [0.2,0.4,0.6,1, 1.2,1.5,2]
for resolution in resolutions_list:
    adata_sub.obs.drop(f'leiden_{str(resolution)}',axis=1,inplace=True)
adata_sub
```

Example 40 (unknown):
```unknown
AnnData object with n_obs × n_vars = 9969 × 17693
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'fine_grain_manual'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.6', 'leiden_1', 'leiden_1.2', 'leiden_1.5', 'leiden_1_colors', 'leiden_2', 'leiden_2_colors', 'log1p', 'neighbors', 'pca', 'spatial', 'umap', 'dendrogram_leiden_1', 'rank_genes_groups', 'fine_grain_manual_colors'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 41 (python):
```python
# remove mid_grain markers
mask = [x not in list(midgrain_markers['names']) for x in adata_sub.var_names]
bdata = adata_sub[:,mask]
bdata
```

Example 42 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 9969 × 16421
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'fine_grain_manual'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.6', 'leiden_1', 'leiden_1.2', 'leiden_1.5', 'leiden_1_colors', 'leiden_2', 'leiden_2_colors', 'log1p', 'neighbors', 'pca', 'spatial', 'umap', 'dendrogram_leiden_1', 'rank_genes_groups', 'fine_grain_manual_colors'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 43 (python):
```python
# HVGs, using the object which doesn't have midgrain markers
sc.pp.highly_variable_genes(bdata, flavor='seurat', min_mean=0.0125, max_mean=3, min_disp=0.5)
print(f'number of hvgs: {sum(bdata.var.highly_variable)}')
# PCA using the degected HVGs
sc.tl.pca(bdata, use_highly_variable=True)
adata_sub.obsm['X_pca'] = bdata.obsm['X_pca'].copy()
# create neighbour graph
sc.pp.neighbors(adata_sub,n_neighbors=15,use_rep='X_pca')
```

Example 44 (unknown):
```unknown
number of hvgs: 2864
```

Example 45 (python):
```python
sc.tl.umap(adata_sub,min_dist=0.1)
```

Example 46 (python):
```python
%%time
# reclustering
# leiden clustering
resolutions_list = [0.2,0.4,0.6,1, 1.2,1.5,2]
for resolution in resolutions_list:
    sc.tl.leiden(adata_sub,resolution=resolution,key_added=f'leiden_{str(resolution)}',n_iterations=2)
sc.pl.embedding(adata_sub,basis='X_umap',
                color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                wspace=0.3)
```

Example 47 (unknown):
```unknown
CPU times: user 4.44 s, sys: 63.9 ms, total: 4.5 s
Wall time: 4.78 s
```

Example 48 (python):
```python
sc.pl.embedding(adata_sub,basis='X_umap',color=genes, size=10,
               ncols=6, wspace=0.2,cmap='RdPu',vmax='p99.7')
```

Example 49 (python):
```python
# umap per celltype to highlight
group_col = 'celltypist_midmod2fine'
groups_to_plot = list(set(adata_sub.obs[group_col]))

plt.rcParams['axes.grid'] = False
fig,axs=plt.subplots(nrows=2,ncols=4,figsize=(15,7))
# Flatten the axs array for easier iteration in case of multiple rows and columns
axs_flat = axs.flatten()

for i,grp in enumerate(groups_to_plot):
    sc.pl.umap(adata_sub, color=[group_col], groups=grp, 
               size=30, frameon=False,
               show=False, ax=axs_flat[i], legend_loc=None, legend_fontsize=5,
               title=grp
              )
# Turn off any remaining subplots
for j in range(len(groups_to_plot), len(axs_flat)):
    fig.delaxes(axs_flat[j])

# fig.suptitle('Main Title for the Figure')
fig.tight_layout()
```

Example 50 (python):
```python
adata_sub_resolution_sel = 1
```

Example 51 (python):
```python
markers_others = {
    'CM': ['TTN', 'TNNT2','MYH6',
        'MYH7', 'MYL2', 'FHL2', # vCN
            'NPPA', 'MYL7', 'MYL4'], #aCM
    'FB': ['DCN', 'GCN', 'PDGFRA','COL1A1','COL1A2'],
    'EC': ['VWF', 'PECAM1', 'CDH5','RGCC', 'FABP5',],
    'Peri': ['RGS5', 'ABCC9', 'KCNJ8'],
    'SMC': ['MYH11', 'TAGLN', 'ACTA2'],
    'Meso': ['MSLN', 'WT1', 'BNC1'],
    'Neuro': ['PLP1', 'NRXN1', 'NRXN3','PRPH', 'NEFL', 'NEFM', 'NEFH', 'STMN2', 'SST', 'SLC10A4', 'SLC18A3', 'TH', 'NPY'],
    'Adipo': ['GPAM', 'FASN', 'LEP'],
    'Myelo': ['CD14', 'C1QA', 'CD68','LYVE1','TIMD4'],
    'Lymph': ['CD8A', 'IL7R', 'CD40LG'],
    'B': ['CD19','SDC1','IGHG1','IGHA1'],
    'Mast': ['KIT', 'CPA3'],
}

for key in markers_others.keys():
    markers_others[key]=[x for x in markers_others[key] if x in adata_sub.var_names]

sc.tl.dendrogram(adata_sub, groupby=f'leiden_{str(adata_sub_resolution_sel)}')
sc.pl.dotplot(adata_sub,
              markers_others, 
              groupby=f'leiden_{str(adata_sub_resolution_sel)}',
              dendrogram=True,
              standard_scale="var",
              color_map="Reds",
              swap_axes=False,
             )
```

Example 52 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: CM, FB, EC, etc.
```

Example 53 (python):
```python
sc.tl.dendrogram(adata_sub,groupby=f'leiden_{str(adata_sub_resolution_sel)}')
sc.pl.dotplot(adata_sub,
              var_names=markers,
              groupby=f'leiden_{str(adata_sub_resolution_sel)}',
              standard_scale='var',
              dendrogram=True
             )
```

Example 54 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: CoronaryPericytes, DuctusArteriosusSmoothMuscleCells, GreatVesselSmoothMuscleCells, etc.
```

Example 55 (python):
```python
# DEG analysis
print(adata_sub.X.data[:5])
adata_sub.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata_sub,groupby=f'leiden_{str(adata_sub_resolution_sel)}')
sc.tl.dendrogram(adata_sub, groupby=f'leiden_{str(adata_sub_resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata_sub, dendrogram=True, standard_scale='var', return_fig=True)
dp.add_totals().show()
```

Example 56 (unknown):
```unknown
[2.35484271 1.76428034 1.58063847 1.74054927 1.3504131 ]
```

Example 57 (python):
```python
# DEG analysis
print(adata_sub.X.data[:5])
adata_sub.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata_sub,groupby=f'leiden_{str(adata_sub_resolution_sel)}')
sc.tl.dendrogram(adata_sub, groupby=f'leiden_{str(adata_sub_resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata_sub, dendrogram=True, standard_scale='var', return_fig=True, n_genes=5)
dp.add_totals().show()
```

Example 58 (unknown):
```unknown
[2.35484271 1.76428034 1.58063847 1.74054927 1.3504131 ]
```

Example 59 (python):
```python
sc.pl.embedding(adata_sub,basis='X_umap',
                color=f'leiden_{str(adata_sub_resolution_sel)}',
                wspace=0.3, legend_loc='on data')
```

Example 60 (unknown):
```unknown
{'CoronaryPericytes',
 'CoronarySmoothMuscleCells',
 'DuctusArteriosusSmoothMuscleCells',
 'GreatVesselSmoothMuscleCells'}
```

Example 61 (unknown):
```unknown
'MuralCells'
```

Example 62 (python):
```python
# annotate
adata_sub.obs['fine_grain_manual'] = adata_sub.obs[f'leiden_{str(adata_sub_resolution_sel)}'].copy()
adata_sub.obs.replace({'fine_grain_manual':{
    '0':'GreatVesselSmoothMuscleCells', 
    '1':'DuctusArteriosusSmoothMuscleCells',
    '2':'GreatVesselSmoothMuscleCells',
    '3':'GreatVesselSmoothMuscleCellsTFPI2pos',
    '4':'GreatVesselSmoothMuscleCells',
    '5':'CoronaryPericytes',
    '6':'CoronarySmoothMuscleCells',
    '7':'DuctusArteriosusSmoothMuscleCells',
    '8':'GreatVesselSmoothMuscleCellsPTGER4pos',
    '9':'GreatVesselSmoothMuscleCells',
}},inplace=True)
adata_sub.obs['fine_grain_manual'].value_counts()
```

Example 63 (unknown):
```unknown
fine_grain_manual
GreatVesselSmoothMuscleCells             4843
DuctusArteriosusSmoothMuscleCells        2145
GreatVesselSmoothMuscleCellsTFPI2pos     1519
CoronaryPericytes                         753
CoronarySmoothMuscleCells                 424
GreatVesselSmoothMuscleCellsPTGER4pos     285
Name: count, dtype: int64
```

Example 64 (python):
```python
sc.pl.embedding(adata_sub,basis='X_umap',
                color='fine_grain_manual',
                frameon=False,
                wspace=0.3)
```

Example 65 (python):
```python
markers['Others']=['PTGER4','TFPI2']

sc.tl.dendrogram(adata_sub,groupby=f'leiden_{str(adata_sub_resolution_sel)}')
sc.pl.dotplot(adata_sub,
              var_names=markers,
              groupby='fine_grain_manual',
              standard_scale='var',
              dendrogram=True
             )
```

Example 66 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: GreatVesselSmoothMuscleCells, DuctusArteriosusSmoothMuscleCells, GreatVesselSmoothMuscleCellsTFPI2pos, etc.
var_group_labels: CoronaryPericytes, DuctusArteriosusSmoothMuscleCells, GreatVesselSmoothMuscleCells, etc.
```

Example 67 (python):
```python
# add the annotation to the original adata
adata.obs['fine_grain_manual'] = adata.obs['fine_grain_manual'].astype('str')
adata.obs.loc[adata_sub.obs_names,'fine_grain_manual'] = adata_sub.obs['fine_grain_manual'].astype('str').copy()
adata.obs['fine_grain_manual'].value_counts()
```

Example 68 (unknown):
```unknown
fine_grain_manual
GreatVesselSmoothMuscleCells             4843
DuctusArteriosusSmoothMuscleCells        2145
GreatVesselSmoothMuscleCellsTFPI2pos     1519
unassigned                                816
CoronaryPericytes                         753
CoronarySmoothMuscleCells                 424
GreatVesselSmoothMuscleCellsPTGER4pos     285
Name: count, dtype: int64
```

Example 69 (python):
```python
sc.pl.embedding(adata,basis='X_umap',
                color='fine_grain_manual',
                frameon=False,
                wspace=0.3)
```

Example 70 (unknown):
```unknown
... storing 'fine_grain_manual' as categorical
```

Example 71 (python):
```python
sc.tl.dendrogram(adata,groupby='fine_grain_manual')
sc.pl.dotplot(adata,
              var_names=markers,
              groupby='fine_grain_manual',
              standard_scale='var',
              dendrogram=True
             )
```

Example 72 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: CoronaryPericytes, CoronarySmoothMuscleCells, DuctusArteriosusSmoothMuscleCells, etc.
var_group_labels: CoronaryPericytes, DuctusArteriosusSmoothMuscleCells, GreatVesselSmoothMuscleCells, etc.
```

Example 73 (python):
```python
# save
adata.write(f'{adata_dir}/{sample_id}_{celltype}_b2c_cells_filtered_lognorm.h5ad')
print(f'{adata_dir}/{sample_id}_{celltype}_b2c_cells_filtered_lognorm.h5ad')
```

Example 74 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/subsets/HEA_FOET14880396_MuralCells_b2c_cells_filtered_lognorm.h5ad
```

---

## Variables¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/Xenium/no2_integration_and_cell-type-annotation/no6_manual-annotation_neurons-glia.html

**Contents:**
- Variables¶
- Read in data¶

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os

import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 

import scanpy.external as sce
```

Example 2 (python):
```python
# import session_info
# session_info.show()
```

Example 3 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/home/kk837/rfs/rfs-teichlab-nfs-iCNyzSAaucw/kk837/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 4 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 5 (python):
```python
adata_dir = '/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium/subsets'
sample_id = 'C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat'
# celltype = 'Neurons'
```

Example 6 (unknown):
```unknown
# figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk/Aug2024'
# table_dir = '/nfs/team205/heart/fetal_ms_figures/tables/Aug2024'
```

Example 7 (unknown):
```unknown
latent_space = 'pca_harmony'
```

Example 8 (python):
```python
# read in
ad1 = sc.read_h5ad(f'{adata_dir}/{sample_id}_5K_Neurons_lognorm.h5ad')
ad2 = sc.read_h5ad(f'{adata_dir}/{sample_id}_5K_Glia_lognorm.h5ad')

adata = ad1.concatenate(ad2, 
            join='inner', 
            batch_key=None, 
            index_unique=None)
adata
```

Example 9 (python):
```python
/tmp/ipykernel_349665/1641403099.py:5: FutureWarning: Use anndata.concat instead of AnnData.concatenate, AnnData.concatenate is deprecated and will be removed in the future. See the tutorial for concat at: https://anndata.readthedocs.io/en/latest/concatenation.html
  adata = ad1.concatenate(ad2,
```

Example 10 (unknown):
```unknown
AnnData object with n_obs × n_vars = 8968 × 4226
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_3.0', 'multi_celltypes_coarse', 'leiden_0.2', 'leiden_0.4', 'leiden_0.6', 'leiden_1', 'leiden_1.2', 'leiden_1.5', 'leiden_2', 'tmp_annotation', 'fine_grain_manual'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells-0', 'highly_variable-0', 'means-0', 'dispersions-0', 'dispersions_norm-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1', 'n_cells-1', 'highly_variable-1', 'means-1', 'dispersions-1', 'dispersions_norm-1'
    obsm: 'X_pca', 'X_pca_harmony', 'X_umap_pca_harmony', 'spatial'
```

Example 11 (python):
```python
adata.X.data[:5]
```

Example 12 (unknown):
```unknown
array([3.4320538, 3.4320538, 3.4320538, 3.4320538, 3.4320538],
      dtype=float32)
```

Example 13 (python):
```python
## Remove mid-grain markers from the feature
# read in markers
midgrain_markers = pd.read_csv('/home/kk837/rfs/rfs-teichlab-nfs-iCNyzSAaucw/kk837/notebooks/Foetal/VisiumHD/re-seq_mpp0p3/sc_midgrain_markers.csv',
                               index_col=0)
# remove the markers of celltype of interest
midgrain_markers = midgrain_markers[midgrain_markers['group'].isin(['Neurons','Glia'])==False]
# remove
mask = [x not in list(midgrain_markers['names']) for x in adata.var_names]
bdata = adata[:,mask]
bdata
```

Example 14 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 8968 × 3793
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_3.0', 'multi_celltypes_coarse', 'leiden_0.2', 'leiden_0.4', 'leiden_0.6', 'leiden_1', 'leiden_1.2', 'leiden_1.5', 'leiden_2', 'tmp_annotation', 'fine_grain_manual'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells-0', 'highly_variable-0', 'means-0', 'dispersions-0', 'dispersions_norm-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1', 'n_cells-1', 'highly_variable-1', 'means-1', 'dispersions-1', 'dispersions_norm-1'
    obsm: 'X_pca', 'X_pca_harmony', 'X_umap_pca_harmony', 'spatial'
```

Example 15 (python):
```python
# remove cell cycle genes
cell_cycle_genes = ['MCM5','PCNA','TYMS','FEN1','MCM2','MCM4','RRM1','UNG','GINS2','MCM6','CDCA7','DTL','PRIM1','UHRF1','MLF1IP','HELLS','RFC2','RPA2','NASP',
                'RAD51AP1','GMNN','WDR76','SLBP','CCNE2','UBR7','POLD3','MSH2','ATAD2','RAD51','RRM2','CDC45','CDC6','EXO1','TIPIN','DSCC1','BLM','CASP8AP2',
                'USP1','CLSPN','POLA1','CHAF1B','BRIP1','E2F8','HMGB2','CDK1','NUSAP1','UBE2C','BIRC5','TPX2','TOP2A','NDC80','CKS2','NUF2','CKS1B','MKI67',
                'TMPO','CENPF','TACC3','FAM64A','SMC4','CCNB2','CKAP2L','CKAP2','AURKB','BUB1','KIF11','ANP32E','TUBB4B','GTSE1','KIF20B','HJURP','CDCA3','HN1',
                'CDC20','TTK','CDC25C','KIF2C','RANGAP1','NCAPD2','DLGAP5','CDCA2','CDCA8','ECT2','KIF23','HMMR','AURKA','PSRC1','ANLN','LBR','CKAP5','CENPE','CTCF',
                'NEK2','G2E3','GAS2L3','CBX5','CENPA']
mask = [x not in cell_cycle_genes for x in bdata.var_names]
bdata = bdata[:,mask]
bdata.shape
```

Example 16 (unknown):
```unknown
(8968, 3740)
```

Example 17 (python):
```python
# HVGs, using the object which doesn't have midgrain markers
sc.pp.highly_variable_genes(bdata, flavor='seurat', min_mean=0.0125, max_mean=3, min_disp=0.5)
print(f'number of hvgs: {sum(bdata.var.highly_variable)}')
# PCA using the degected HVGs
sc.tl.pca(bdata, use_highly_variable=True)
adata.obsm['X_pca'] = bdata.obsm['X_pca'].copy()
# harmony integration
sce.pp.harmony_integrate(adata, 'tissue_block_id', basis='X_pca', adjusted_basis='X_pca_harmony')
# create neighbour graph
latent_space = 'pca_harmony'
sc.pp.neighbors(adata,n_neighbors=15,use_rep=f"X_{latent_space}",key_added=latent_space)
# visualisation with umap
sc.tl.umap(adata,min_dist=0.1,neighbors_key=latent_space)
adata.obsm[f"X_umap_{latent_space}"]=adata.obsm["X_umap"]
del adata.obsm["X_umap"]
```

Example 18 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_highly_variable_genes.py:698: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  adata.uns["hvg"] = {"flavor": flavor}
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_pca/__init__.py:438: FutureWarning: Argument `use_highly_variable` is deprecated, consider using the mask argument. Use_highly_variable=True can be called through mask_var="highly_variable". Use_highly_variable=False can be called through mask_var=None
  warn(msg, FutureWarning)
```

Example 19 (unknown):
```unknown
number of hvgs: 827
```

Example 20 (python):
```python
2025-06-12 17:52:22,562 - harmonypy - INFO - Computing initial centroids with sklearn.KMeans...
2025-06-12 17:52:25,016 - harmonypy - INFO - sklearn.KMeans initialization complete.
2025-06-12 17:52:25,108 - harmonypy - INFO - Iteration 1 of 10
2025-06-12 17:52:26,304 - harmonypy - INFO - Iteration 2 of 10
2025-06-12 17:52:27,502 - harmonypy - INFO - Converged after 2 iterations
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm
```

Example 21 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=['total_counts','n_genes_by_counts','cell_area','tissue_block_id'],
                    wspace=0.2,cmap='RdPu',vmax='p100')
```

Example 22 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=['fine_grain_manual'],
                    wspace=0.2,cmap='RdPu',vmax='p100')
```

Example 23 (python):
```python
group_col = 'fine_grain_manual'
groups_to_plot = adata.obs[group_col].unique()
basis = f"X_umap_{latent_space}"

ncols = 6
if len(groups_to_plot)%ncols==0:
    nrows = len(groups_to_plot)//ncols
else:
    nrows = len(groups_to_plot)//ncols+1

if len(groups_to_plot)<ncols:
    fig_width = len(groups_to_plot)%ncols
else:
    fig_width = ncols
    
fig,axs=plt.subplots(nrows=nrows,ncols=ncols,figsize=(fig_width*5,nrows*5))
# Flatten the axs array for easier iteration in case of multiple rows and columns
axs_flat = axs.flatten()

for i,grp in enumerate(groups_to_plot):
    # ad = adata.copy()
    # update colors
    # get color in adata
    # ind = list(ad.obs[group_col].cat.categories).index(celltype)
    # ad.uns[group_col][ind] = 'darkred'
    
    # title
    middle_ind = round(len(grp)/2)
    # grp_title = f'{grp[:middle_ind]}\n{grp[middle_ind:]}'
    # plot
    sc.pl.embedding(adata, basis=basis,
                  color=group_col, groups=grp,
                  size=20,legend_loc=None,
                  frameon=False,title=grp,
                  show=False, ax=axs_flat[i],
                 )
# Turn off any remaining subplots
for j in range(len(groups_to_plot), len(axs_flat)):
    fig.delaxes(axs_flat[j])

# fig.suptitle('Main Title for the Figure')
fig.tight_layout()
```

Example 24 (python):
```python
# markers for fine-grained cell types
markers = {
    'Neuronal':['NEFL','RBFOX3'],
    "ChromaffinCells": [
        'DLK1',
        'EPAS1',
        'KCNJ6',
        "TH",    # Tyrosine hydroxylase
        "PNMT",  # Phenylethanolamine N-methyltransferase
        "CHGA",  # Chromogranin A
        "CHGB",  # Chromogranin B
        "PHOX2B"
    ],
    "NeuronPrecursors": [
        "NES",    # Nestin
        "PAX6",   # Paired box 6
        "TUBB3",  # Tubulin Beta 3 Class III
        "DCX",     # Doublecortin
        'ASCL1','NELL2','BNC2'
    ],
    "ParasympatheticNeurons": [
        "PHOX2B",
        "CHAT",    # Choline O-Acetyltransferase
        "SLC18A3", # Solute Carrier Family 18 Member A3 (Vesicular Acetylcholine Transporter)
        "GATA3",
        'NEFL','SST'
    ],
    "SympatheticNeurons": [
        "TH",
        "PHOX2B",
        "GATA3",
        "CHGA",
        "CHGB",
        'DBH','VSTM2L'
    ],
    'EntericNeurons':['NOS1', 'VIP', 'CARTPT', 'RET'],
    
     "astrocytes": ["GFAP", "S100B", "ALDH1L1"],
    "oligodendrocytes": ["OLIG2", "MBP", "PDGFRA"],
    "microglia": ["AIF1"],  # IBA1 protein is encoded by the AIF1 gene
    "schwann_cells": ["SOX10", "MPZ", "S100B"],
    "schwann_cell_precursors": [
                                "PLS3",'SLITRK6','IL1RAPL2'
                               ],
    "ependymal cells":[
                    "FOXJ1",  # Forkhead box J1
                    "S100B",  # S100 Calcium Binding Protein B
                    "VIM",    # Vimentin
                    "PROM1",  # Prominin-1 (CD133)
                    "CDH1",   # Cadherin 1 (E-cadherin)
                    "EPCAM"   # Epithelial Cell Adhesion Molecule
    ],
    
    'Cycling':['MKI67','TOP2A']
}
genes = []
for g in markers.values():
    genes = genes + [x for x in g if (x in adata.var_names)&(x not in genes)]
for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in adata.var_names]

sc.tl.dendrogram(adata,groupby='fine_grain_manual')
sc.pl.dotplot(adata,
              var_names=markers,
              groupby='fine_grain_manual',
              standard_scale='var',
              dendrogram=True
             )
```

Example 25 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: ?EpendymalCells, ChromaffinCells, EntericLikeNeurons, etc.
var_group_labels: Neuronal, ChromaffinCells, NeuronPrecursors, etc.
```

---

## Read in DEG results¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig5_Trisomy21/no12_GSEA_finegrain.html

**Contents:**
- Read in DEG results¶
- GSEA analysis¶
- Euploid-enriched nhoods vs Others¶

20860 rows × 10 columns

**Examples:**

Example 1 (python):
```python
import scanpy as sc
import pandas as pd
import numpy as np
import anndata
import scipy
import matplotlib.pyplot as plt
import seaborn as sns
import os
import warnings
warnings.simplefilter("ignore", UserWarning)
warnings.simplefilter(action='ignore', category=pd.errors.PerformanceWarning)
```

Example 2 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 3 (unknown):
```unknown
os.getcwd()
```

Example 4 (unknown):
```unknown
'/nfs/team205/kk18/notebooks/Foetal/Trisomy21/RNA/milo_Mar2025_refine-nhoods'
```

Example 5 (python):
```python
deg_res = pd.read_csv('/nfs/team205/kk18/notebooks/Foetal/Trisomy21/RNA/milo_Mar2025_refine-nhoods/milo_nhood-reassigned_pseudobulk_edgeR_res_tri21-nhoods_vs_euploid-nhoods_finegrain.csv',
                          index_col=0)
print(deg_res.shape)
deg_res.head()
```

Example 6 (python):
```python
import blitzgsea as blitz
```

Example 7 (python):
```python
%%time
geneset_list = ['MSigDB_Hallmark_2020','KEGG_2021_Human','GO_Biological_Process_2023','Reactome_2022'] # 
deg_fdr_thresh = 0.1
deg_p_thresh = None

################################################
count1 = 0
for celltype in deg_res['celltype'].unique():
    print(celltype)
    # subset
    res_sub = deg_res[deg_res['celltype']==celltype]
    # filter or not based on stats of the DEtest
    if deg_fdr_thresh!=None:
        res_sub = res_sub[res_sub['FDR']<deg_fdr_thresh]
    elif deg_p_thresh!=None:
        res_sub = res_sub[res_sub['PValue']<deg_p_thresh]
    else:
        pass
    # GSEA
    if res_sub.shape[0]>=5:
        res_sub = res_sub[['gene_name','logFC']]
        res_sub.columns = [0,1]
        count2 = 0
        for i, geneset in enumerate(geneset_list):
            # use enrichr submodule to retrieve gene set library
            library = blitz.enrichr.get_library(geneset)
            # run enrichment analysis
            if __name__ == "__main__":  # make sure process is main, when run in a script it can cause errors otherwise
                result = blitz.gsea(res_sub, library)
                result = result.reset_index()
            if result.shape[0]>0:
                count2 += 1
                result['geneset'] = geneset
                # concatenate
                if count2==1:
                    result_concat = result.copy()
                else:
                    result_concat = pd.concat([result_concat,result])
            else:
                print(f'gsea, {geneset}: no results')
        if count2==0: # no results at all
            continue
    else:
        print(f'significant genes({list(res_sub["gene_name"])}) is less than 5')
        continue
    
    # concatenate
    count1 += 1
    result_concat['celltype'] = celltype
    if count1==1:
        result_concat_concat = result_concat.copy()
    else:
        result_concat_concat = pd.concat([result_concat_concat,result_concat])
    del result_concat

# save
result_concat_concat.to_csv('/nfs/team205/kk18/notebooks/Foetal/Trisomy21/RNA/milo_Mar2025_refine-nhoods/GSEA_trisomy21-vs-euploid-nhoods_finegrain.csv')
result_concat_concat
```

Example 8 (unknown):
```unknown
AtrialCardiomyocytesRight
AtrialCardiomyocytesLeft
CoronaryCapillaryEndothelialCells
EndocardialCells
SubEpicardialFibroblasts
GreatVesselAdventitialFibroblasts
GreatVesselSmoothMuscleCells
MacrophagesATF3pos
MacrophagesLYVE1pos
MyocardialInterstitialFibroblasts
VentricularCardiomyocytesCycling
VentricularCardiomyocytesLeftCompact
VentricularCardiomyocytesLeftTrabeculated
VentricularCardiomyocytesRightCompact
CPU times: user 21.6 s, sys: 1.09 s, total: 22.6 s
Wall time: 37.6 s
```

---

## Subcluster¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/RNA/scnRNA_Integration-Annotation/DE compendium.html

**Contents:**
- Subcluster¶

**Examples:**

Example 1 (unknown):
```unknown
#Added the keys for age and sex
```

Example 2 (python):
```python
# Import packages, initial settings

import numpy as np
import matplotlib.pyplot as plt
import scanpy as sc
import pandas as pd
import anndata
import scrublet as scr
import seaborn as sns
import magic
import pybiomart
import six
import sys
sys.modules['sklearn.externals.six'] = six
import scanorama
import harmonypy
import bbknn
import gseapy
from gseapy.plot import barplot, dotplot
import scvelo as scv
import scvi

plt.rcParams['pdf.fonttype'] = 42
plt.rcParams['ps.fonttype'] = 42

sc.settings.verbosity = 3  # verbosity: errors (0), warnings (1), info (2), hints (3)

from matplotlib.colors import LinearSegmentedColormap
cmap = LinearSegmentedColormap.from_list(name='gene_cmap', colors=['lightgrey', 'thistle', 'red', 'darkred']) 

sc.settings.set_figure_params(dpi=80, color_map='viridis', vector_friendly=False,  dpi_save=300, facecolor='white')
sc.logging.print_version_and_date()
#sc.logging.print_versions()
```

Example 3 (unknown):
```unknown
Global seed set to 0
```

Example 4 (unknown):
```unknown
Running Scanpy 1.8.1, on 2023-05-16 17:38.
```

Example 5 (python):
```python
adata=sc.read('./Objects/Feb28ObjectRaw.h5ad')
print(adata.shape)
```

Example 6 (unknown):
```unknown
(297473, 36601)
```

Example 7 (python):
```python
cell_cycle_genes = ['MCM5','PCNA','TYMS','FEN1','MCM2','MCM4','RRM1','UNG','GINS2','MCM6','CDCA7','DTL','PRIM1','UHRF1','MLF1IP','HELLS','RFC2','RPA2','NASP','RAD51AP1','GMNN','WDR76','SLBP','CCNE2','UBR7','POLD3','MSH2','ATAD2','RAD51','RRM2','CDC45','CDC6','EXO1','TIPIN','DSCC1','BLM','CASP8AP2','USP1','CLSPN','POLA1','CHAF1B','BRIP1','E2F8','HMGB2','CDK1','NUSAP1','UBE2C','BIRC5','TPX2','TOP2A','NDC80','CKS2','NUF2','CKS1B','MKI67','TMPO','CENPF','TACC3','FAM64A','SMC4','CCNB2','CKAP2L','CKAP2','AURKB','BUB1','KIF11','ANP32E','TUBB4B','GTSE1','KIF20B','HJURP','CDCA3','HN1','CDC20','TTK','CDC25C','KIF2C','RANGAP1','NCAPD2','DLGAP5','CDCA2','CDCA8','ECT2','KIF23','HMMR','AURKA','PSRC1','ANLN','LBR','CKAP5','CENPE','CTCF','NEK2','G2E3','GAS2L3','CBX5','CENPA']
s_genes = cell_cycle_genes[:43]
s_genes = [x for x in s_genes if x in adata.var_names]
print('S genes:', s_genes, '\n')
g2m_genes = cell_cycle_genes[43:]
g2m_genes = [x for x in g2m_genes if x in adata.var_names]
print('G2 genes:', g2m_genes, '\n')
cell_cycle_genes = [x for x in cell_cycle_genes if x in adata.var_names]

hb_genes1 = ['HBB','HBD','HBG1','HBG2','HBE1','HBZ','HBM','HBA2','HBA1', 'HBQ1']
hb_genes1 = [x for x in hb_genes1 if x in adata.var_names]
print('Hemoglobin genes set 1:', hb_genes1, '\n')

hb_genes2 = ['AHSP','GYPA','RHAG','GFI1B','KEL','EPB42','ALAS2','TSPO2','GYPB','GYPE','SPTA1','RHCE','KLF1','RHD','SLC4A1','SNX22','SQSTM1','SOD1','FTH1','MAP1LC3B','SLC25A37','TRIM58','HMBS']
hb_genes2 = [x for x in hb_genes2 if x in adata.var_names]
print('Hemoglobin genes set 2:', hb_genes2, '\n')

mito_genes=['MT-ND1','MT-ND2','MT-CO1','MT-CO2','MT-ATP8','MT-ATP6','MT-CO3','MT-ND3','MT-ND4L', 'MT-ND4', 'MT-ND5', 'MT-ND6', 'MT-CYB']
mito_genes = [x for x in mito_genes if x in adata.var_names]
print('Mitochondrial genes:', mito_genes, '\n')

ribo_genes = ['RPLP0','RPLP1','RPLP2','RPL3','RPL3L','RPL4','RPL5','RPL6','RPL7','RPL7A','RPL7L1','RPL8','RPL9','RPL10','RPL10A','RPL10L','RPL11','RPL12','RPL13','RPL13A','RPL14','RPL15','RPL17','RPL18','RPL18A','RPL19','RPL21','RPL22','RPL22L1','RPL23','RPL23A','RPL24','RPL26','RPL26L1','RPL27','RPL27A','RPL28','RPL29','RPL30','RPL31','RPL32','RPL34','RPL35','RPL35A','RPL36','RPL36A','RPL36AL','RPL37','RPL37A','RPL38','RPL39','RPL39L','UBA52','RPL41','RPSA','RPS2','RPS3','RPS3A','RPS4X','RPS4Y1','RPS4Y2','RPS5','RPS6','RPS7','RPS8','RPS9','RPS10','RPS11','RPS12','RPS13','RPS14','RPS15','RPS15A','RPS16','RPS17','RPS18','RPS19','RPS20','RPS21','RPS23','RPS24','RPS25','RPS26','RPS27','RPS27A','RPS27L','RPS28','RPS29','FAU']
ribo_genes = [x for x in ribo_genes if x in adata.var_names]
print('Ribo genes:', ribo_genes, '\n')

stress_genes=['FOS','FOSB','JUN','JUNB','JUND','ZFP36','SOCS3','CEBPD','CEBPB','ATF3','EGR1','HSPA1A','HSPA1B','HSP90AA1','HSP90AB1','HSPA8','HSPB1','HSPE1','IER3','IER2','BTG1','BTG2','DUSP1']
stress_genes = [x for x in stress_genes if x in adata.var_names]
print('Tissue dissociation stress associated genes:', stress_genes, '\n')

FoxProteins=['FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1']
chrY_genes=['SRY', 'RPS4Y1', 'ZFY', 'ZFY-AS1', 'LINC00278', 'PCDH11Y', 'TTTY2B', 'TBL1Y', 'PRKY', 'USP9Y', 'DDX3Y', 'UTY', 'TMSB4Y', 'NLGN4Y', 'TTTY14', 'KDM5D', 'TTTY10', 'EIF1AY', 'PRORY', 'LINC00266-4P']
MembraneProteins1=['AADAT','AAK1','ABCA12','ABCA7','ABCB1','ABCB11','ABCB4','ABCB6','ABCC1','ABCC3','ABCC4','ABCC5','ABCC6','ABCG2','ABHD3','ABI1','ABLIM3','AC004691.2','AC004922.1','AC005324.3','AC005726.1','AC008403.1','AC138696.1','ACIN1','ACKR3','ACLY','ACOT13','ACOT9','ACTN1','ACTR10','ACTR6','ADA','ADAM10','ADAM12','ADAM22','ADAM28','ADAM33','ADAMTS1','ADAMTS2','ADAMTSL4','ADAP1','ADCK5','ADD1','ADD2','ADGRB2','ADGRD1','ADGRG2','ADGRG7','ADGRL2','ADH1A','ADH1B','ADH1C','ADH7','ADK','ADORA1','ADRA1B','ADRM1','ADSS','AFAP1L2','AFDN','AFG1L','AGER','AGRN','AHNAK','AHNAK2','AIFM2','AKAP10','AKAP11','AKAP12','AKR1B10','AKR1B15','AL132780.3','AL139300.1','AL355315.1','AL358113.1','AL645922.1','ALDH3A1','ALK','ALOX15B','ALOXE3','ALPI','ALPP','ALPPL2','ALS2CR12','AMACR','AMDHD1','AMER1','AMER2','AMIGO2','AMOT','AMOTL1','AMOTL2','AMPH','ANK2','ANK3','ANKEF1','ANKFN1','ANKIB1','ANKLE2','ANKRA2','ANKRD13A','ANKRD35','ANKRD42','ANKRD45','ANKS1B','ANO1','ANO10','ANO2','ANO6','ANPEP','ANXA1','ANXA13','ANXA2','ANXA3','AP2M1','APBB1','APBB1IP','APC','APLP1','APMAP','APOC3','APOD','APOLD1','AQP1','AQP10','AQP3','AQP4','AQP5','ARAP1','ARF1','ARHGAP18','ARHGAP19','ARHGAP21','ARHGAP22','ARHGAP23','ARHGAP24','ARHGAP28','ARHGAP29','ARHGAP33','ARHGAP36','ARHGAP40','ARHGAP9','ARHGDIG','ARHGEF1','ARHGEF11','ARHGEF12','ARHGEF15','ARHGEF26','ARHGEF28','ARHGEF40','ARHGEF5','ARID2','ARID4A','ARID5B','ARL13A','ARL15','ARL4C','ARMC12','ARMCX5','ARPC1A','ARPC5','ARRB2','ARRDC4','ARVCF','ASAP1','ASB5','ASGR1','ASGR2','ASH2L','ASIC1','ASIC4','ASPM','ASPSCR1','ATG13','ATG3','ATG7','ATIC','ATOX1','ATP10D','ATP12A','ATP13A3','ATP1B3','ATP23','ATP2B1','ATP2B4','ATP2C2','ATP6V0A2','ATP6V1H','ATP8A2','ATP8B1','ATXN10','ATXN3','ATXN7L3','AVEN','AVPI1','AXIN2','AXL','B2M','B3GALT2','BACE1','BAG3','BAIAP2','BAIAP2L1','BANK1','BASP1','BBOF1','BCAR1','BCAR3','BCL2L12','BCORL1','BCR','BDKRB2','BEND6','BFSP1','BFSP2','BICD2','BIN2','BLNK','BLVRB','BMPR2','BMX','BOC','BOLL','BORCS5','BORCS8-MEF2B','BRK1','BRPF1','BRSK1','BSDC1','BST2','BTK','BTN2A1','BTRC','BUB1B-PAK6','BVES','BZW2','C10orf90','C11orf42','C11orf52','C12orf73','C12orf76','C14orf37','C14orf93','C16orf74','C17orf58','C17orf62','C18orf21','C19orf33','C19orf38','C19orf48','C19orf71','C1orf116','C1orf189','C1orf35','C1orf54','C1orf68','C1orf74','C1QBP','C2orf72','C3orf14','C3orf20','C3orf38','C4orf19','C4orf22','C7orf43','C8orf33','C8orf37','C9orf153','C9orf3','CA14','CA9','CABLES2','CACNA1E','CACNG4','CADM1','CALCRL','CALD1','CALML5','CALN1','CAMK2A','CAMK2B','CAMK2D','CAMK2G','CAP2','CAPS','CASKIN2','CASP4','CAST','CAV3','CAVIN1','CAVIN2','CAVIN3','CAVIN4','CBWD1','CBWD2','CBWD3','CBWD5','CBWD6','CC2D1A','CCBE1','CCDC107','CCDC112','CCDC115','CCDC124','CCDC136','CCDC144NL','CCDC15','CCDC154','CCDC170','CCDC24','CCDC28A','CCDC59','CCDC62','CCDC66','CCDC69','CCDC8','CCDC83','CCDC85C','CCDC93','CCDC97','CCNT2','CCNYL1','CCR3','CCR4','CCT3','CD109','CD163','CD164L2','CD200R1','CD226','CD248','CD274','CD2AP','CD300A','CD300LB','CD33','CD34','CD37','CD38','CD3E','CD3G','CD4','CD44','CD46','CD47','CD48','CD79B','CD81','CD84','CD86','CD8A','CD8B','CD9','CD93','CD99L2','CDAN1','CDC42EP1','CDC42EP3','CDC42EP4','CDC42SE1','CDCA4','CDH1','CDH13','CDH15','CDH17','CDH2','CDH26','CDH3','CDH4','CDH5','CDH6','CDH8','CDIPT','CDK16','CDK5','CDK5RAP2','CDK7','CDRT15','CDV3','CEACAM5','CELSR1','CENPJ','CEP162','CEP295','CEP55','CEP57L1','CEP89','CERCAM','CFAP157','CFAP206','CFAP58','CFAP65','CFB','CFL2','CFLAR','CGN','CGNL1','CHCHD7','CHIC2','CHMP2A','CHRM3','CHRNA1','CHRNA5','CHRNB4','CHRND','CIB1','CIB3','CIP2A','CKAP5','CLCA2','CLCA4','CLCN5','CLCN6','CLDN11','CLDN18','CLDN2','CLDN23','CLDN3','CLDN4','CLDN7','CLDN9','CLEC2A','CLEC4F','CLEC5A','CLECL1','CLIC2','CLIC5','CLTB','CLUL1','CMTM4','CMTM7','CMTR2','CMYA5','CNGA1','CNKSR2','CNNM1','CNNM4','CNP','CNR1','CNTD1','COBL','COBLL1','COG3','COL16A1','COL17A1','COL4A5','COL6A2','COLEC12','COLQ','COMMD4','COMTD1','COPRS','COQ7','CORIN','CORO1B','CORO2A','CORO2B','COX6B2','CPEB3','CPPED1','CPZ','CRB3','CREB5','CRIP1','CRIP2','CRK','CRLF3','CROCC','CRTC1','CRYAB','CRYBG3','CRYL1','CSDE1','CSF1','CSF1R','CSPG4','CST6','CTNNA1','CTNNB1','CTNNBIP1','CTNND1','CTNND2','CTSF','CTSV','CTTN','CTXN1','CUEDC1','CUTA','CX3CL1','CX3CR1','CXADR','CXCR2','CYB5D2','CYBRD1','CYFIP2','CYP20A1','CYP24A1','CYTH2','CYTH4','DAAM1','DAB2','DAG1','DAGLB','DAW1','DBN1','DCAF13','DCAF16','DCBLD2','DCHS1','DCHS2','DCP2','DCT','DCTN6','DCUN1D2','DDAH1','DDB2','DDIAS','DDIT4L','DDR1','DDR2','DDX58','DDX60L','DENND6B','DFFA','DGKG','DGKH','DHDDS','DHPS','DHRS7B','DIAPH1','DIAPH3','DIO2','DIS3L','DLG1','DLG2','DLG5','DLGAP3','DLL3','DMWD','DNAAF1','DNAAF4','DNAH17','DNAJA4','DNAJB13','DNAJB4','DNAJC18','DNAJC5','DNAJC6','DNAJC9','DOC2A','DOCK4','DOK3','DONSON','DPEP1','DPH1','DPPA5','DPYSL2','DQX1','DRD1','DRP2','DSC3','DSEL','DSG2','DSP','DTNA','DUS1L','DUSP15','DXO','DYSF','ECSCR','EDA2R','EDNRA','EDNRB','EEF1AKMT1','EEF2','EFCAB1','EFCAB11','EFCAB2','EFCAB7','EFHC2','EFNB1','EFR3A','EFR3B','EGFR','EHBP1','EHD1','EHD2','EHD3','EHD4','EIF2B2','EIF5','EIF5B','ELFN1','ELMO3','ELN','ELOC','EMP3','ENAH','ENC1','ENG','ENKD1','ENO1','ENO2','ENO3','ENOX1','ENOX2','EPB41','EPB41L1','EPB41L2','EPB41L3','EPB41L4A','EPB41L4B','EPB41L5','EPCAM','EPHA2','EPHA3','EPHB1','EPHB2','EPN1','EPOR','EPS8L2','ERAP1','ERBB2','ERBB3','ERBIN','ERC1','ERCC4','ERMN','ESCO2','ETNK1','ETS2','EVA1A','EVI2B','EXOC1','EXOC3L1','EXOC7','EZR','F11R','F2RL1','F2RL3','FAAP20','FABP2','FABP3','FABP5','FAIM','FAM126A','FAM126B','FAM129A','FAM129B','FAM13A','FAM163B','FAM171A1','FAM193A','FAM205A','FAM210B','FAM222A','FAM45A','FAM47E','FAM47E-STBD1','FAM72A','FAM72B','FAM72C','FAM72D','FANCG','FAS','FASN','FBLIM1','FBLN2','FBLN5','FBLN7','FBP2','FBRSL1','FBXL2','FBXO16','FBXO38','FBXO41','FBXW11','FCAR','FCGBP','FCGR1A','FCGR1B','FCGR2A','FCHSD2','FFAR4','FGD1','FGD5','FGFR2','FGR','FHIT','FHL1','FIBCD1','FIGN','FILIP1','FILIP1L','FKBP2','FLAD1','FLCN','FLNA','FLNB','FLNC','FLOT1','FLOT2','FLRT3','FLT1','FLT4','FLVCR1','FMN2','FMNL2','FMNL3','FNBP1','FOXA2','FOXN3','FP565260.3','FRMD3','FRMD7','FRMD8','FRS2','FSCN1','FTCD','FUT2','FXYD3','FYB1','FYN','FZD1','FZD2','FZD4','FZD6','GAB1','GAB2','GABRA1','GABRA2','GABRA3','GABRA5','GABRG1','GAD1','GADL1','GAGE1','GAGE10','GAGE12C','GAGE12D','GAGE12E','GAGE12F','GAGE12G','GAGE12H','GAGE12J','GAGE13','GAGE2A','GAGE2E','GALNT10','GALR1','GAP43','GAPDH','GAPT','GAPVD1','GAS2','GAS2L1','GAS7','GAS8','GBP4','GCA','GCC1','GCLM','GCSAML-AS1','GFI1B','GFOD1','GFRA3','GHR','GID8','GINM1','GIPC1','GJA1','GJB3','GJB4','GJB6','GJB7','GJC1','GJC2','GLO1','GLP2R','GLRX','GLUL','GMIP','GNAI2','GNAQ','GNAS','GNB1','GNB2','GNB3','GNB4','GNG2','GP9','GPAT3','GPATCH1','GPBP1','GPBP1L1','GPC1','GPC3','GPC4','GPI','GPR1','GPR142','GPR156','GPR157','GPR160','GPR173','GPR20','GPR27','GPR50','GPR63','GPRC5A','GPRIN1','GPRIN2','GPX8','GRAMD1A','GRAMD1B','GRAMD1C','GRAP2','GRB7','GRID2IP','GRIK5','GRK3','GRK5','GRPR','GSDMA','GTF2F1','GTSE1','GUCA1A','GYPA','GYPC','HAPLN3','HARBI1','HAS1','HAUS4','HAUS7','HCAR2','HCAR3','HCFC2','HCK','HCLS1','HCST','HDAC8','HEG1','HELZ','HENMT1','HERC2','HERPUD1','HFE','HIF1A','HIF3A','HINT1','HIP1R','HLA-A','HLA-DPB1','HMGCS1','HMOX1','HOMER3','HOXC5','HOXD4','HPS3','HRASLS5','HRH1','HS6ST3','HSH2D','HSPB1','HSPG2','HTR1B','HTR7','HTRA1','HUNK','HYDIN','HYI','HYLS1','ICAM1','ICAM2','ICOS','ICOSLG','IFFO2','IFIT5','IFNGR1','IFRD1','IFT122','IGFL2','IGSF11','IGSF23','IKZF5','IL10RA','IL12RB1','IL12RB2','IL13','IL13RA2','IL16','IL17RB','IL1R1','IL1RL1','IL21R','IL4R','IL6ST','IL7R','ILDR2','ILK','INPP5A','INSR','INTS12','IP6K2','IPCEF1','IQGAP1','IQGAP2','IREB2','IRF2','IRS1','ISLR','ITGA2B','ITGA3','ITGA4','ITGA7','ITGA8','ITGA9','ITGB1','ITGB2','ITGB3','ITGB4','ITGB5','ITGB6','ITGB7','ITGB8','ITK','ITM2A','ITPKA','ITSN1','ITSN2','IYD','JAG1','JAK2','JAK3','JAM2','JAML','JCAD','JMJD4','JPT2','JUP','KANK1','KANK3','KANSL2','KARS','KATNB1','KCNAB1','KCNB2','KCNH8','KCNJ15','KCNK13','KCNK17','KCNK6','KCNK7','KCNK9','KCNN2','KCNS3','KCNU1','KDF1','KDR','KEL','KIAA0556','KIAA0895','KIAA1324','KIAA1522','KIAA1958','KIF17','KIF19','KIF21A','KIF21B','KIF26A','KIF26B','KISS1R','KIT','KLC1','KLC2','KLF9','KLHDC7B','KLHL41','KLHL6','KLHL7','KLK7','KLRC1','KLRC2','KLRF1','KLRK1','KMT2D','KNDC1','KNSTRN','KNTC1','KPTN','KRT75','KRT77','KRTAP25-1','L1CAM','L3MBTL1','LAIR1','LAMA4','LAMC1','LAMP2','LAMTOR1','LAPTM4A','LASP1','LAT','LAT2','LAX1','LCP1','LCTL','LDB2','LDLRAD3','LENG1','LGALS3','LGALS4','LGALS8','LGR6','LHFPL2','LHX6','LIMA1','LINGO1','LIPJ','LMCD1','LMOD2','LMTK2','LMTK3','LNX1','LNX2','LONRF1','LONRF3','LOXHD1','LPAR1','LPAR3','LPAR4','LPIN1','LPIN2','LRCH2','LRP11','LRP12','LRRC15','LRRC26','LRRC34','LRRC45','LRRC69','LRRC7','LRRC9','LRRFIP2','LRRIQ3','LSMEM2','LSR','LTB4R2','LUZP2','LY6K','LYN','LYPD1','LYST','LZTS1','LZTS2','MACF1','MADD','MAGI1','MAGI3','MAMDC2','MAP1S','MAP2','MAP2K1','MAP2K4','MAP2K7','MAP3K12','MAP3K21','MAP3K6','MAP3K7','MAP4','MAP4K1','MAP4K5','MAP6','MAPK15','MAPK8IP1','MAPT','MARCH10','MARCH7','MARCKS','MARK2','MARVELD2','MAS1L','MAT2B','MBP','MC5R','MCAM','MCEMP1','MCF2L2','MCMBP','MCTS1','MDGA1','ME1','MEF2B','MEIOC','MEPCE','MERTK','MET','METAP1','METAP2','METTL15','METTL25','METTL9','MFAP3L','MFSD1','MFSD2A','MGAT4C','MIB1','MICAL2','MICAL3','MICALL1','MICB','MIEN1','MIGA2','MISP','MKLN1','MKRN3','MLF2','MLKL','MLST8','MMACHC','MMP10','MMP15','MMP24','MMP25','MOAP1','MPHOSPH8','MPHOSPH9','MPI','MPIG6B','MPL','MPP1','MPP5','MPP7','MPZL1','MPZL2','MR1','MRGPRF','MRNIP','MRPL42','MRPL44','MRPL46','MRPS18B','MS4A1','MS4A12','MS4A14','MS4A15','MSN','MSRA','MTCP1','MTFR1L','MTHFR','MTM1','MTPN','MTSS1','MUC1','MUC20','MVD','MYH11','MYH9','MYL4','MYLK','MYO10','MYO1B','MYO1C','MYO1E','MYO1G','MYO3A','MYO5B','MYO9A','MYOF','N4BP2','NAA35','NAB1','NAIF1','NAMPT','NAP1L2','NAPG','NARFL','NCAM1','NCAM2','NCK1','NCKAP1L','NCKAP5','NCKIPSD','NCOA1','NCS1','NDC1','NECTIN2','NECTIN3','NECTIN4','NEDD1','NEDD9','NEO1','NEU1','NEXN','NF2','NFE2L2','NFKBID','NGEF']
MembraneProteins2=['NGFR','NHS','NHSL2','NID2','NINJ2','NKTR','NKX2-8','NLGN1','NLGN3','NLK','NLRX1','NMB','NMBR','NME1','NME2','NMT1','NMT2','NMUR2','NOS1','NOTCH2','NPFFR2','NPL','NPNT','NPR1','NPTN','NPTXR','NR2C1','NRAS','NRCAM','NRP2','NRSN1','NRXN1','NSD1','NSFL1C','NT5E','NTNG1','NTNG2','NTSR1','NUDCD3','NUDT16L1','NUDT9','NUMB','NUP35','NUS1','OAS3','OBSCN','OCA2','OCLN','OCM','OCM2','ODC1','ODR4','OGT','OLFM4','OLIG2','OLR1','OPHN1','OPRK1','OR10P1','OR1N2','OR2C3','OR6B2','OR6B3','OR6C6','OR8S1','OSBPL10','OSBPL6','OSBPL7','OSCAR','OSCP1','OSGEP','OSR2','OTOP2','OTULIN','OXGR1','P3H3','PACSIN2','PACSIN3','PAFAH1B2','PAG1','PAIP1','PAK1','PAK4','PAK6','PALLD','PALM','PALM2','PALM2-AKAP2','PALM3','PALMD','PANX1','PANX2','PAQR8','PARD3','PARD3B','PARD6A','PARD6G','PARP6','PATJ','PAWR','PCBD2','PCDH1','PCDH17','PCDH7','PCDHA1','PCDHA11','PCDHA12','PCDHA4','PCDHA5','PCDHB1','PCDHB11','PCDHB12','PCDHB16','PCDHB4','PCDHB5','PCDHGA1','PCDHGA10','PCDHGA11','PCDHGA12','PCDHGA2','PCDHGA3','PCDHGA4','PCDHGA5','PCDHGA6','PCDHGA7','PCDHGA8','PCDHGA9','PCDHGB1','PCDHGB2','PCDHGB3','PCDHGB4','PCDHGB5','PCDHGB6','PCDHGB7','PCDHGC3','PCDHGC4','PCDHGC5','PCLO','PCMTD1','PDAP1','PDCD7','PDE3A','PDE4A','PDE4D','PDE9A','PDGFC','PDGFRA','PDHX','PDLIM1','PDLIM5','PDPN','PDZD2','PDZD8','PEAR1','PEBP1','PFDN1','PGAP3','PGLYRP2','PGLYRP4','PGRMC2','PHACTR1','PHACTR4','PHF20L1','PHGDH','PHLDB2','PHLDB3','PHOSPHO1','PHPT1','PI3','PI4K2A','PI4KA','PIEZO2','PIGB','PIGN','PIGW','PIGX','PIK3AP1','PIK3CG','PIK3R6','PIP4K2A','PIP4K2C','PIP5K1A','PIRT','PITPNM3','PKD2L2','PKN1','PKN2','PKP1','PKP2','PKP3','PKP4','PLAA','PLAUR','PLCB1','PLCB2','PLCD3','PLCD4','PLEKHA4','PLEKHA6','PLEKHA7','PLEKHG3','PLEKHG4','PLEKHG6','PLEKHH2','PLEKHO2','PLIN4','PLPP1','PLPPR5','PLS3','PLSCR1','PLVAP','PLXNA3','PLXNA4','PNCK','PNISR','PODXL','POLE','POU2F3','PPEF1','PPFIBP1','PPIP5K1','PPL','PPM1A','PPM1F','PPP1CA','PPP1CB','PPP1R12B','PPP1R16A','PPP1R16B','PPP1R18','PPP1R27','PPP1R9A','PPP2R1B','PPP3R1','PPP3R2','PPP4C','PPP6R3','PPTC7','PRAME','PRC1','PRDX6','PRIMA1','PRKCA','PRKCE','PRKCH','PRKCZ','PRKD1','PROM2','PROSER2','PRR15','PRR32','PRR7','PRRC2A','PRRG1','PRRT2','PRRT3','PRRT4','PRTG','PRX','PSCA','PSD','PSD2','PSEN1','PSMC5','PSMC6','PSMD9','PSTPIP2','PTAFR','PTCHD4','PTK2B','PTK6','PTPN12','PTPN21','PTPN22','PTPN4','PTPRB','PTPRG','PTPRK','PTPRM','PTPRR','PTPRS','PTX3','PURG','PVALB','PVR','PXDC1','PXK','PYGL','PYGO2','PYM1','RAB13','RAB23','RAB25','RAB26','RAB27B','RAB3B','RAB3GAP2','RAB43','RAB8A','RABEPK','RAD51C','RAD52','RAI2','RALA','RALGAPA2','RAPGEF2','RAPH1','RAPSN','RASA4','RASA4B','RASAL1','RASAL2','RASAL3','RASD1','RASD2','RASGRP1','RASGRP2','RASGRP4','RASSF2','RASSF3','RASSF4','RBM46','RC3H1','RCCD1','RCSD1','RD3L','RDX','RECK','RELL1','REPS1','RET','RFLNB','RFPL1','RFPL2','RFPL3','RGP1','RGS10','RGS14','RGS19','RGS3','RGS7BP','RGS9BP','RHCE','RHD','RHEX','RHOA','RHOBTB2','RHOJ','RHOXF2','RHOXF2B','RIC8A','RIC8B','RIF1','RILPL1','RIMBP3','RIMBP3B','RIMBP3C','RIMS2','RIOK2','RIOK3','RIPK1','RND1','RNF114','RNF145','RNF146','RNF149','RNF165','RNF166','RNF39','ROBO1','ROBO3','ROBO4','ROM1','RP2','RPE','RPEL1','RPGRIP1L','RPIA','RPL36A','RPL36A-HNRNPH2','RPL36AL','RPSA','RPUSD1','RRAD','RRP12','RRP7A','RSBN1L','RSC1A1','RSPH3','RTL3','RTL4','RTN4R','RTN4RL1','RYR2','S100A13','S100A14','S100A16','S100A2','S100A3','S100A4','S100A6','S100A9','SAMD4A','SAMHD1','SAMSN1','SAPCD1','SASH3','SC5D','SCAMP1','SCAMP4','SCEL','SCHIP1','SCIN','SCN1A','SCN5A','SCN8A','SCN9A','SCNN1D','SCNN1G','SCRIB','SCUBE3','SDC2','SDC4','SDE2','SDHB','SEC16B','SEMA3C','SEMA3D','SEMA4B','SEMA4F','SEMA6C','SEMA6D','SEPT2','SEPT3','SERINC2','SERINC5','SERPINB3','SFSWAP','SGCE','SGK494','SH3BGR','SH3D21','SHANK1','SHANK3','SHARPIN','SHISA6','SHKBP1','SHROOM2','SHTN1','SIGLEC10','SIGLEC14','SIGLEC6','SIGLEC7','SIN3B','SIPA1','SIPA1L1','SIPA1L3','SKAP1','SLAMF6','SLC10A5','SLC12A2','SLC13A3','SLC13A5','SLC14A1','SLC16A1','SLC16A10','SLC16A2','SLC16A3','SLC16A7','SLC16A9','SLC17A5','SLC19A1','SLC19A3','SLC1A5','SLC1A6','SLC22A16','SLC23A3','SLC24A4','SLC26A9','SLC27A1','SLC28A1','SLC2A1','SLC2A11','SLC2A12','SLC2A14','SLC2A2','SLC2A3','SLC30A1','SLC31A2','SLC34A1','SLC34A2','SLC35F5','SLC35G2','SLC36A4','SLC38A5','SLC38A6','SLC38A7','SLC39A10','SLC39A14','SLC39A2','SLC39A6','SLC3A2','SLC40A1','SLC41A3','SLC43A1','SLC43A3','SLC44A2','SLC45A4','SLC46A1','SLC46A3','SLC4A1AP','SLC4A2','SLC4A7','SLC4A8','SLC5A12','SLC5A3','SLC5A6','SLC5A7','SLC6A6','SLC7A14','SLC7A2','SLC7A5','SLC7A6','SLC8A1','SLC9A1','SLC9A2','SLC9A3','SLC9A3R1','SLC9A3R2','SLC9A4','SLC9A5','SLC9B2','SLC9C2','SLCO1A2','SLCO1B3','SLCO4A1','SLCO4C1','SLCO5A1','SLIT1','SLIT3','SLITRK6','SLK','SLX4','SMAGP','SMAP1','SMOC1','SMPD2','SMPX','SNAP23','SNAP25','SNRK','SNTB2','SNTG2','SNX15','SNX32','SNX9','SOCS3','SOCS5','SOD1','SORBS1','SORCS1','SORT1','SOX11','SPAG16','SPANXN4','SPART','SPATA12','SPATC1L','SPERT','SPG11','SPHK1','SPINK13','SPINT1','SPINT2','SPN','SPPL2B','SPPL3','SPRED3','SPRYD3','SPTBN2','SRC','SRCIN1','SRD5A3','SRPK1','SRPRB','SRRM3','SSFA2','SSH1','SSH3','SSTR1','SSTR5','SSX2IP','ST8SIA5','STAB1','STAC','STAC2','STAMBP','STAMBPL1','STARD4','STAT2','STEAP4','STIL','STIP1','STK10','STK16','STK17A','STK17B','STOM','STOML2','STOML3','STX18','STX4','STXBP4','STYK1','SUCLG1','SUCLG2','SUGT1','SUMF2','SUPT6H','SUSD2','SVIL','SWAP70','SYCP2','SYK','SYMPK','SYNC','SYNGR3','SYNPO2L','SYPL2','SYT4','SYTL2','SYVN1','TACC2','TACR2','TACSTD2','TASP1','TAZ','TBC1D10A','TBC1D2','TBC1D24','TBC1D30','TBC1D9B','TBXA2R','TCAF2','TCAF2C','TCHP','TCIM','TDG','TDO2','TDP1','TEC','TEDC2','TEK','TES','TEX14','TEX9','TGFBR1','TGFBR2','TGM1','TGM2','THBS1','THBS2','THEGL','THY1','TIAM1','TICAM1','TIGD7','TIMD4','TJP1','TJP2','TJP3','TLN1','TLR4','TM2D2','TM4SF20','TM4SF4','TM4SF5','TMC5','TMED8','TMEM101','TMEM107','TMEM108','TMEM121B','TMEM139','TMEM141','TMEM164','TMEM176B','TMEM183A','TMEM186','TMEM2','TMEM218','TMEM222','TMEM229A','TMEM245','TMEM251','TMEM254','TMEM258','TMEM266','TMEM268','TMEM50A','TMEM63B','TMPRSS15','TMPRSS2','TMTC2','TNFRSF10D','TNFRSF11A','TNFRSF12A','TNFRSF21','TNFRSF25','TNFRSF8','TNFRSF9','TNFSF13B','TNK1','TNKS1BP1','TNNI2','TOGARAM2','TOLLIP','TOM1','TP53BP2','TPD52L1','TPRN','TRAF7','TRAIP','TRAPPC13','TRAT1','TRDN','TREML1','TRIB1','TRIM16','TRIM16L','TRIM38','TRIM4','TRIM44','TRIM54','TRIM71','TRIML2','TRIP6','TRMT1','TRMT10B','TRMT13','TRMT6','TRO','TRPM1','TRPM4','TRPM7','TRPV2','TRPV3','TSG101','TSGA13','TSHZ3','TSPAN15','TSPAN6','TSPO2','TSR2','TSSK4','TTC17','TTC23','TTC7B','TTLL12','TTLL5','TULP3','TWF2','TXNDC11','TXNIP','TYROBP','TYW3','UAP1','UBA3','UBA52','UBAC1','UBALD1','UBAP1','UBASH3B','UBE2A','UBE2B','UBE2C','UBE2D1','UBE2D2','UBE2D3','UBE2D4','UBE2K','UBE2QL1','UBE2S','UBE3C','UBQLN2','UBR2','UBTD1','UCP3','ULBP1','ULBP2','UNC5A','UNC79','UQCC1','USP14','USP15','USP2','USP20','USP21','USP4','USP53','UTP20','UTRN','VAMP5','VAMP8','VANGL1','VASP','VCAM1','VCPIP1','VIL1','VPS13B','VPS36','VPS50','VPS8','VSIG4','VTCN1','VWA5B1','VWA7','WAS','WASF2','WBP4','WDFY3','WDPCP','WDR1','WDR12','WDR13','WDR53','WDR6','WDR66','WIPF1','WWP1','XIAP','XIRP1','XIRP2','XPC','XPNPEP2','XPO6','XRN1','XYLT1','YES1','YIPF1','YIPF4','YOD1','YTHDC1','ZBED3','ZBTB33','ZBTB8OS','ZDHHC2','ZDHHC20','ZDHHC22','ZDHHC5','ZFP41','ZFP90','ZFYVE28','ZGPAT','ZMYND10','ZNF10','ZNF18','ZNF185','ZNF2','ZNF20','ZNF227','ZNF229','ZNF236','ZNF276','ZNF277','ZNF337','ZNF397','ZNF407','ZNF497','ZNF503','ZNF510','ZNF536','ZNF571','ZNF598','ZNF599','ZNF646','ZNF655','ZNF688','ZNF706','ZNF717','ZNF783','ZNF79','ZNF839','ZSCAN22','ZSCAN29','ZSWIM1','ZSWIM8','ZYX']
MembraneProteins=MembraneProteins1+MembraneProteins2

TFs1=['AC008770.3','AC023509.3','AC092835.1','AC138696.1','ADNP','ADNP2','AEBP1','AEBP2','AHCTF1','AHDC1','AHR','AHRR','AIRE','AKAP8','AKAP8L','AKNA','ALX1','ALX3','ALX4','ANHX','ANKZF1','AR','ARGFX','ARHGAP35','ARID2','ARID3A','ARID3B','ARID3C','ARID5A','ARID5B','ARNT','ARNT2','ARNTL','ARNTL2','ARX','ASCL1','ASCL2','ASCL3','ASCL4','ASCL5','ASH1L','ATF1','ATF2','ATF3','ATF4','ATF5','ATF6','ATF6B','ATF7','ATMIN','ATOH1','ATOH7','ATOH8','BACH1','BACH2','BARHL1','BARHL2','BARX1','BARX2','BATF','BATF2','BATF3','BAZ2A','BAZ2B','BBX','BCL11A','BCL11B','BCL6','BCL6B','BHLHA15','BHLHA9','BHLHE22','BHLHE23','BHLHE40','BHLHE41','BNC1','BNC2','BORCS8-MEF2B','BPTF','BRF2','BSX','C11orf95','CAMTA1','CAMTA2','CARF','CASZ1','CBX2','CC2D1A','CCDC169-SOHLH2','CCDC17','CDC5L','CDX1','CDX2','CDX4','CEBPA','CEBPB','CEBPD','CEBPE','CEBPG','CEBPZ','CENPA','CENPB','CENPBD1','CENPS','CENPT','CENPX','CGGBP1','CHAMP1','CHCHD3','CIC','CLOCK','CPEB1','CPXCR1','CREB1','CREB3','CREB3L1','CREB3L2','CREB3L3','CREB3L4','CREB5','CREBL2','CREBZF','CREM','CRX','CSRNP1','CSRNP2','CSRNP3','CTCF','CTCFL','CUX1','CUX2','CXXC1','CXXC4','CXXC5','DACH1','DACH2','DBP','DBX1','DBX2','DDIT3','DEAF1','DLX1','DLX2','DLX3','DLX4','DLX5','DLX6','DMBX1','DMRT1','DMRT2','DMRT3','DMRTA1','DMRTA2','DMRTB1','DMRTC2','DMTF1','DNMT1','DNTTIP1','DOT1L','DPF1','DPF3','DPRX','DR1','DRAP1','DRGX','DUX1','DUX3','DUX4','DUXA','DZIP1','E2F1','E2F2','E2F3','E2F4','E2F5','E2F6','E2F7','E2F8','E4F1','EBF1','EBF2','EBF3','EBF4','EEA1','EGR1','EGR2','EGR3','EGR4','EHF','ELF1','ELF2','ELF3','ELF4','ELF5','ELK1','ELK3','ELK4','EMX1','EMX2','EN1','EN2','EOMES','EPAS1','ERF','ERG','ESR1','ESR2','ESRRA','ESRRB','ESRRG','ESX1','ETS1','ETS2','ETV1','ETV2','ETV3','ETV3L','ETV4','ETV5','ETV6','ETV7','EVX1','EVX2','FAM170A','FAM200B','FBXL19','FERD3L','FEV','FEZF1','FEZF2','FIGLA','FIZ1','FLI1','FLYWCH1','FOS','FOSB','FOSL1','FOSL2','FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXN4','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1','GABPA','GATA1','GATA2','GATA3','GATA4','GATA5','GATA6','GATAD2A','GATAD2B','GBX1','GBX2','GCM1','GCM2','GFI1','GFI1B','GLI1','GLI2','GLI3','GLI4','GLIS1','GLIS2','GLIS3','GLMP','GLYR1','GMEB1','GMEB2','GPBP1','GPBP1L1','GRHL1','GRHL2','GRHL3','GSC','GSC2','GSX1','GSX2','GTF2B','GTF2I','GTF2IRD1','GTF2IRD2','GTF2IRD2B','GTF3A','GZF1','HAND1','HAND2','HBP1','HDX','HELT','HES1','HES2','HES3','HES4','HES5','HES6','HES7','HESX1','HEY1','HEY2','HEYL','HHEX','HIC1','HIC2','HIF1A','HIF3A','HINFP','HIVEP1','HIVEP2','HIVEP3','HKR1','HLF','HLX','HMBOX1','HMG20A','HMG20B','HMGA1','HMGA2','HMGN3','HMX1','HMX2','HMX3','HNF1A','HNF1B','HNF4A','HNF4G','HOMEZ','HOXA1','HOXA10','HOXA11','HOXA13','HOXA2','HOXA3','HOXA4','HOXA5','HOXA6','HOXA7','HOXA9','HOXB1','HOXB13','HOXB2','HOXB3','HOXB4','HOXB5','HOXB6','HOXB7','HOXB8','HOXB9','HOXC10','HOXC11','HOXC12','HOXC13','HOXC4','HOXC5','HOXC6','HOXC8','HOXC9','HOXD1','HOXD10','HOXD11','HOXD12','HOXD13','HOXD3','HOXD4','HOXD8','HOXD9','HSF1','HSF2','HSF4','HSF5','HSFX1','HSFX2','HSFY1','HSFY2','IKZF1','IKZF2','IKZF3','IKZF4','IKZF5','INSM1','INSM2','IRF1','IRF2','IRF3','IRF4','IRF5','IRF6','IRF7','IRF8','IRF9','IRX1','IRX2','IRX3','IRX4','IRX5','IRX6','ISL1','ISL2','ISX','JAZF1','JDP2','JRK','JRKL','JUN','JUNB','JUND','KAT7','KCMF1','KCNIP3','KDM2A','KDM2B','KDM5B','KIN','KLF1','KLF10','KLF11','KLF12','KLF13','KLF14','KLF15','KLF16','KLF17','KLF2','KLF3','KLF4','KLF5','KLF6','KLF7','KLF8','KLF9','KMT2A','KMT2B','L3MBTL1','L3MBTL3','L3MBTL4','LBX1','LBX2','LCOR','LCORL','LEF1','LEUTX','LHX1','LHX2','LHX3','LHX4','LHX5','LHX6','LHX8','LHX9','LIN28A','LIN28B','LIN54','LMX1A','LMX1B','LTF','LYL1','MAF','MAFA','MAFB','MAFF','MAFG','MAFK','MAX','MAZ','MBD1','MBD2','MBD3','MBD4','MBD6','MBNL2','MECOM','MECP2','MEF2A','MEF2B','MEF2C','MEF2D','MEIS1','MEIS2','MEIS3','MEOX1','MEOX2','MESP1','MESP2','MGA','MITF','MIXL1','MKX','MLX','MLXIP','MLXIPL','MNT','MNX1','MSANTD1','MSANTD3','MSANTD4','MSC','MSGN1','MSX1','MSX2','MTERF1','MTERF2','MTERF3','MTERF4','MTF1','MTF2','MXD1','MXD3','MXD4','MXI1','MYB','MYBL1','MYBL2','MYC','MYCL','MYCN','MYF5','MYF6','MYNN','MYOD1','MYOG','MYPOP','MYRF','MYRFL','MYSM1','MYT1','MYT1L','MZF1','NACC2','NAIF1','NANOG','NANOGNB','NANOGP8','NCOA1','NCOA2','NCOA3','NEUROD1','NEUROD2','NEUROD4','NEUROD6','NEUROG1','NEUROG2','NEUROG3','NFAT5','NFATC1','NFATC2','NFATC3','NFATC4','NFE2','NFE2L1','NFE2L2','NFE2L3','NFE4','NFIA','NFIB','NFIC','NFIL3','NFIX','NFKB1','NFKB2','NFX1','NFXL1','NFYA','NFYB','NFYC','NHLH1','NHLH2','NKRF','NKX1-1','NKX1-2','NKX2-1','NKX2-2','NKX2-3','NKX2-4','NKX2-5','NKX2-6','NKX2-8','NKX3-1','NKX3-2','NKX6-1','NKX6-2','NKX6-3','NME2','NOBOX','NOTO','NPAS1','NPAS2','NPAS3','NPAS4','NR0B1','NR1D1','NR1D2','NR1H2','NR1H3','NR1H4','NR1I2','NR1I3','NR2C1','NR2C2','NR2E1','NR2E3','NR2F1','NR2F2','NR2F6','NR3C1','NR3C2','NR4A1','NR4A2','NR4A3','NR5A1','NR5A2','NR6A1','NRF1','NRL','OLIG1','OLIG2','OLIG3','ONECUT1','ONECUT2','ONECUT3','OSR1','OSR2','OTP','OTX1','OTX2','OVOL1','OVOL2','OVOL3','PA2G4','PATZ1','PAX1','PAX2','PAX3','PAX4','PAX5','PAX6','PAX7','PAX8','PAX9','PBX1','PBX2','PBX3','PBX4','PCGF2','PCGF6','PDX1','PEG3','PGR','PHF1','PHF19 ','PHF20','PHF21A','PHOX2A','PHOX2B','PIN1','PITX1','PITX2','PITX3','PKNOX1','PKNOX2','PLAG1','PLAGL1','PLAGL2','PLSCR1','POGK','POU1F1','POU2AF1','POU2F1','POU2F2','POU2F3','POU3F1','POU3F2','POU3F3','POU3F4','POU4F1','POU4F2','POU4F3','POU5F1','POU5F1B','POU5F2','POU6F1','POU6F2','PPARA','PPARD','PPARG','PRDM1','PRDM10','PRDM12','PRDM13','PRDM14','PRDM15','PRDM16','PRDM2','PRDM4','PRDM5','PRDM6','PRDM8','PRDM9','PREB','PRMT3','PROP1','PROX1','PROX2','PRR12','PRRX1','PRRX2','PTF1A','PURA','PURB','PURG','RAG1','RARA','RARB','RARG','RAX','RAX2','RBAK','RBCK1','RBPJ','RBPJL','RBSN','REL','RELA','RELB','REPIN1','REST','REXO4','RFX1','RFX2','RFX3','RFX4','RFX5','RFX6','RFX7','RFX8','RHOXF1','RHOXF2','RHOXF2B','RLF','RORA','RORB','RORC','RREB1','RUNX1','RUNX2','RUNX3','RXRA','RXRB','RXRG','SAFB','SAFB2','SALL1','SALL2','SALL3','SALL4','SATB1','SATB2','SCMH1','SCML4','SCRT1','SCRT2','SCX','SEBOX','SETBP1','SETDB1','SETDB2','SGSM2','SHOX','SHOX2','SIM1','SIM2','SIX1','SIX2','SIX3','SIX4','SIX5','SIX6','SKI','SKIL','SKOR1','SKOR2','SLC2A4RG','SMAD1','SMAD3','SMAD4','SMAD5','SMAD9','SMYD3','SNAI1','SNAI2','SNAI3','SNAPC2','SNAPC4','SNAPC5','SOHLH1','SOHLH2','SON','SOX1','SOX10','SOX11']
TFs2=['SOX12','SOX13','SOX14','SOX15','SOX17','SOX18','SOX2','SOX21','SOX3','SOX30','SOX4','SOX5','SOX6','SOX7','SOX8','SOX9','SP1','SP100','SP110','SP140','SP140L','SP2','SP3','SP4','SP5','SP6','SP7','SP8','SP9','SPDEF','SPEN','SPI1','SPIB','SPIC','SPZ1','SRCAP','SREBF1','SREBF2','SRF','SRY','ST18','STAT1','STAT2','STAT3','STAT4','STAT5A','STAT5B','STAT6','T','TAL1','TAL2','TBP','TBPL1','TBPL2','TBR1','TBX1','TBX10','TBX15','TBX18','TBX19','TBX2','TBX20','TBX21','TBX22','TBX3','TBX4','TBX5','TBX6','TCF12','TCF15','TCF20','TCF21','TCF23','TCF24','TCF3','TCF4','TCF7','TCF7L1','TCF7L2','TCFL5','TEAD1','TEAD2','TEAD3','TEAD4','TEF','TERB1','TERF1','TERF2','TET1','TET2','TET3','TFAP2A','TFAP2B','TFAP2C','TFAP2D','TFAP2E','TFAP4','TFCP2','TFCP2L1','TFDP1','TFDP2','TFDP3','TFE3','TFEB','TFEC','TGIF1','TGIF2','TGIF2LX','TGIF2LY','THAP1','THAP10','THAP11','THAP12','THAP2','THAP3','THAP4','THAP5','THAP6','THAP7','THAP8','THAP9','THRA','THRB','THYN1','TIGD1','TIGD2','TIGD3','TIGD4','TIGD5','TIGD6','TIGD7','TLX1','TLX2','TLX3','TMF1','TOPORS','TP53','TP63','TP73','TPRX1','TRAFD1','TRERF1','TRPS1','TSC22D1','TSHZ1','TSHZ2','TSHZ3','TTF1','TWIST1','TWIST2','UBP1','UNCX','USF1','USF2','USF3','VAX1','VAX2','VDR','VENTX','VEZF1','VSX1','VSX2','WIZ','WT1','XBP1','XPA','YBX1','YBX2','YBX3','YY1','YY2','ZBED1','ZBED2','ZBED3','ZBED4','ZBED5','ZBED6','ZBED9','ZBTB1','ZBTB10','ZBTB11','ZBTB12','ZBTB14','ZBTB16','ZBTB17','ZBTB18','ZBTB2','ZBTB20','ZBTB21','ZBTB22','ZBTB24','ZBTB25','ZBTB26','ZBTB3','ZBTB32','ZBTB33','ZBTB34','ZBTB37','ZBTB38','ZBTB39','ZBTB4','ZBTB40','ZBTB41','ZBTB42','ZBTB43','ZBTB44','ZBTB45','ZBTB46','ZBTB47','ZBTB48','ZBTB49','ZBTB5','ZBTB6','ZBTB7A','ZBTB7B','ZBTB7C','ZBTB8A','ZBTB8B','ZBTB9','ZC3H8','ZEB1','ZEB2','ZFAT','ZFHX2','ZFHX3','ZFHX4','ZFP1','ZFP14','ZFP2','ZFP28','ZFP3','ZFP30','ZFP37','ZFP41','ZFP42','ZFP57','ZFP62','ZFP64','ZFP69','ZFP69B','ZFP82','ZFP90','ZFP91','ZFP92','ZFPM1','ZFPM2','ZFX','ZFY','ZGLP1','ZGPAT','ZHX1','ZHX2','ZHX3','ZIC1','ZIC2','ZIC3','ZIC4','ZIC5','ZIK1','ZIM2','ZIM3','ZKSCAN1','ZKSCAN2','ZKSCAN3','ZKSCAN4','ZKSCAN5','ZKSCAN7','ZKSCAN8','ZMAT1','ZMAT4','ZNF10','ZNF100','ZNF101','ZNF107','ZNF112','ZNF114','ZNF117','ZNF12','ZNF121','ZNF124','ZNF131','ZNF132','ZNF133','ZNF134','ZNF135','ZNF136','ZNF138','ZNF14','ZNF140','ZNF141','ZNF142','ZNF143','ZNF146','ZNF148','ZNF154','ZNF155','ZNF157','ZNF16','ZNF160','ZNF165','ZNF169','ZNF17','ZNF174','ZNF175','ZNF177','ZNF18','ZNF180','ZNF181','ZNF182','ZNF184','ZNF189','ZNF19','ZNF195','ZNF197','ZNF2','ZNF20','ZNF200','ZNF202','ZNF205','ZNF207','ZNF208','ZNF211','ZNF212','ZNF213','ZNF214','ZNF215','ZNF217','ZNF219','ZNF22','ZNF221','ZNF222','ZNF223','ZNF224','ZNF225','ZNF226','ZNF227','ZNF229','ZNF23','ZNF230','ZNF232','ZNF233','ZNF234','ZNF235','ZNF236','ZNF239','ZNF24','ZNF248','ZNF25','ZNF250','ZNF251','ZNF253','ZNF254','ZNF256','ZNF257','ZNF26','ZNF260','ZNF263','ZNF264','ZNF266','ZNF267','ZNF268','ZNF273','ZNF274','ZNF275','ZNF276','ZNF277','ZNF28','ZNF280A','ZNF280B','ZNF280C','ZNF280D','ZNF281','ZNF282','ZNF283','ZNF284','ZNF285','ZNF286A','ZNF286B','ZNF287','ZNF292','ZNF296','ZNF3','ZNF30','ZNF300','ZNF302','ZNF304','ZNF311','ZNF316','ZNF317','ZNF318','ZNF319','ZNF32','ZNF320','ZNF322','ZNF324','ZNF324B','ZNF326','ZNF329','ZNF331','ZNF333','ZNF334','ZNF335','ZNF337','ZNF33A','ZNF33B','ZNF34','ZNF341','ZNF343','ZNF345','ZNF346','ZNF347','ZNF35','ZNF350','ZNF354A','ZNF354B','ZNF354C','ZNF358','ZNF362','ZNF365','ZNF366','ZNF367','ZNF37A','ZNF382','ZNF383','ZNF384','ZNF385A','ZNF385B','ZNF385C','ZNF385D','ZNF391','ZNF394','ZNF395','ZNF396','ZNF397','ZNF398','ZNF404','ZNF407','ZNF408','ZNF41','ZNF410','ZNF414','ZNF415','ZNF416','ZNF417','ZNF418','ZNF419','ZNF420','ZNF423','ZNF425','ZNF426','ZNF428','ZNF429','ZNF43','ZNF430','ZNF431','ZNF432','ZNF433','ZNF436','ZNF438','ZNF439','ZNF44','ZNF440','ZNF441','ZNF442','ZNF443','ZNF444','ZNF445','ZNF446','ZNF449','ZNF45','ZNF451','ZNF454','ZNF460','ZNF461','ZNF462','ZNF467','ZNF468','ZNF469','ZNF470','ZNF471','ZNF473','ZNF474','ZNF479','ZNF48','ZNF480','ZNF483','ZNF484','ZNF485','ZNF486','ZNF487','ZNF488','ZNF490','ZNF491','ZNF492','ZNF493','ZNF496','ZNF497','ZNF500','ZNF501','ZNF502','ZNF503','ZNF506','ZNF507','ZNF510','ZNF511','ZNF512','ZNF512B','ZNF513','ZNF514','ZNF516','ZNF517','ZNF518A','ZNF518B','ZNF519','ZNF521','ZNF524','ZNF525','ZNF526','ZNF527','ZNF528','ZNF529','ZNF530','ZNF532','ZNF534','ZNF536','ZNF540','ZNF541','ZNF543','ZNF544','ZNF546','ZNF547','ZNF548','ZNF549','ZNF550','ZNF551','ZNF552','ZNF554','ZNF555','ZNF556','ZNF557','ZNF558','ZNF559','ZNF560','ZNF561','ZNF562','ZNF563','ZNF564','ZNF565','ZNF566','ZNF567','ZNF568','ZNF569','ZNF57','ZNF570','ZNF571','ZNF572','ZNF573','ZNF574','ZNF575','ZNF576','ZNF577','ZNF578','ZNF579','ZNF580','ZNF581','ZNF582','ZNF583','ZNF584','ZNF585A','ZNF585B','ZNF586','ZNF587','ZNF587B','ZNF589','ZNF592','ZNF594','ZNF595','ZNF596','ZNF597','ZNF598','ZNF599','ZNF600','ZNF605','ZNF606','ZNF607','ZNF608','ZNF609','ZNF610','ZNF611','ZNF613','ZNF614','ZNF615','ZNF616','ZNF618','ZNF619','ZNF620','ZNF621','ZNF623','ZNF624','ZNF625','ZNF626','ZNF627','ZNF628','ZNF629','ZNF630','ZNF639','ZNF641','ZNF644','ZNF645','ZNF646','ZNF648','ZNF649','ZNF652','ZNF653','ZNF654','ZNF655','ZNF658','ZNF66','ZNF660','ZNF662','ZNF664','ZNF665','ZNF667','ZNF668','ZNF669','ZNF670','ZNF671','ZNF672','ZNF674','ZNF675','ZNF676','ZNF677','ZNF678','ZNF679','ZNF680','ZNF681','ZNF682','ZNF683','ZNF684','ZNF687','ZNF688','ZNF689','ZNF69','ZNF691','ZNF692','ZNF695','ZNF696','ZNF697','ZNF699','ZNF7','ZNF70','ZNF700','ZNF701','ZNF703','ZNF704','ZNF705A','ZNF705B','ZNF705D','ZNF705E','ZNF705G','ZNF706','ZNF707','ZNF708','ZNF709','ZNF71','ZNF710','ZNF711','ZNF713','ZNF714','ZNF716','ZNF717','ZNF718','ZNF721','ZNF724','ZNF726','ZNF727','ZNF728','ZNF729','ZNF730','ZNF732','ZNF735','ZNF736','ZNF737','ZNF74','ZNF740','ZNF746','ZNF747','ZNF749','ZNF750','ZNF75A','ZNF75D','ZNF76','ZNF761','ZNF763','ZNF764','ZNF765','ZNF766','ZNF768','ZNF77','ZNF770','ZNF771','ZNF772','ZNF773','ZNF774','ZNF775','ZNF776','ZNF777','ZNF778','ZNF780A','ZNF780B','ZNF781','ZNF782','ZNF783','ZNF784','ZNF785','ZNF786','ZNF787','ZNF788','ZNF789','ZNF79','ZNF790','ZNF791','ZNF792','ZNF793','ZNF799','ZNF8','ZNF80','ZNF800','ZNF804A','ZNF804B','ZNF805','ZNF808','ZNF81','ZNF813','ZNF814','ZNF816','ZNF821','ZNF823','ZNF827','ZNF829','ZNF83','ZNF830','ZNF831','ZNF835','ZNF836','ZNF837','ZNF84','ZNF841','ZNF843','ZNF844','ZNF845','ZNF846','ZNF85','ZNF850','ZNF852','ZNF853','ZNF860','ZNF865','ZNF878','ZNF879','ZNF880','ZNF883','ZNF888','ZNF891','ZNF90','ZNF91','ZNF92','ZNF93','ZNF98','ZNF99','ZSCAN1','ZSCAN10','ZSCAN12','ZSCAN16','ZSCAN18','ZSCAN2','ZSCAN20','ZSCAN21','ZSCAN22','ZSCAN23','ZSCAN25','ZSCAN26','ZSCAN29','ZSCAN30','ZSCAN31','ZSCAN32','ZSCAN4','ZSCAN5A','ZSCAN5B','ZSCAN5C','ZSCAN9','ZUFSP','ZXDA','ZXDB','ZXDC','ZZZ3']
TFs=TFs1+TFs2

#TFs other, inclusive of the above list, not necessarily DNA binding proteins but affect transcription
TFs1Other=['AATF','ABL1','ABRA','AC008770.3','AC020915.1','AC023509.3','AC092835.1','AC105001.2','AC138696.1','ACOD1','ACTR3B','ACTR3C','ADAM8','ADAMTS17','ADAMTS19','ADAR','ADGRA1','ADGRG3','ADNP','ADNP2','AEBP1','AEBP2','AES','AFAP1 ','AFF1','AFF3','AFF4','AGER','AGT','AHCTF1','AHDC1','AHR','AHRR','AIM2','AIRE','AKAP8','AKAP8L','AKNA','AKT1','ALK','ALKBH2','ALX1','ALX3','ALX4','AMH','ANAPC2','ANHX','ANKRD30A','ANKZF1','ANXA3','ANXA4','AP5Z1','APBB3','APC2','APEX1','APOA4','APOBEC3A','APOBEC3B','APOBEC3G','APTX','AR','ARGFX','ARHGAP35','ARHGEF2','ARHGEF5','ARID1A','ARID1B','ARID2','ARID3A','ARID3B','ARID3C','ARID4A','ARID4B','ARID5A','ARID5B','ARNT','ARNT2','ARNTL','ARNTL2','ARRB1','ARRB2','ARX','ASCC1','ASCL1','ASCL2','ASCL3','ASCL4','ASCL5','ASH1L','ATF1','ATF2','ATF3','ATF4','ATF5','ATF6','ATF6B','ATF7','ATF7IP','ATM','ATMIN','ATOH1','ATOH7','ATOH8','ATR','ATRX','ATXN7','BACH1','BACH2','BAG1','BANF1','BANP','BARHL1','BARHL2','BARX1','BARX2','BATF','BATF2','BATF3','BAZ2A','BAZ2B','BBX','BCL10','BCL11A','BCL11B','BCL3','BCL6','BCL6B','BCLAF1','BCOR','BDP1','BEND4','BEX1','BHLHA15','BHLHA9','BHLHE22','BHLHE23','BHLHE40','BHLHE41','BHMG1','BIN1','BLM','BLZF1','BMI1','BMP2','BMP4','BMP7','BMPR1A','BNC1','BNC2','BOD1L1','BOLA1','BOLA2','BOLA2B','BOLA3','BORCS8-MEF2B','BPNT1','BPTF','BRCA1','BRD4','BRD7','BRD8','BRD9','BRF1','BRF2','BRMS1','BRPF1','BSX','BTAF1','BTBD8','BTG2','BTK','BTRC','BUD31','C11orf95','C14orf39','C17orf49','C1D','C20orf194','C3orf33','CACTIN','CAMK1D','CAMK2A','CAMTA1','CAMTA2','CAPN15','CAPN3','CARD11','CARD14','CARF','CARHSP1','CASP8AP2','CASZ1','CAT','CBFA2T2','CBFA2T3','CBFB','CBL','CBLL1','CBX2','CC2D1A','CC2D1B','CCDC124','CCDC169-SOHLH2','CCDC17','CCDC3','CCDC71','CCDC83','CCNT1','CCNT2','CD36','CD40','CD40LG','CDC5L','CDC6','CDK5RAP3','CDK7','CDK8','CDK9','CDKN1A','CDKN2A','CDX1','CDX2','CDX4','CEBPA','CEBPB','CEBPD','CEBPE','CEBPG','CEBPZ','CELF3','CENPA','CENPB','CENPBD1','CENPS','CENPT','CENPX','CERS2','CERS3','CERS4','CERS5','CERS6','CFLAR','CGGBP1','CHAMP1','CHCHD3','CHD1','CHD2','CHD3','CHD4','CHD6','CHD7','CHD9','CHP1','CHRAC1','CHUK','CHURC1','CIB1','CIC','CIITA','CIR1','CITED1','CITED2','CITED4','CIZ1','CLOCK','CLU','CMKLR1','CNBP','CNOT3','CNOT4','CNOT7','CNOT8','COMMD1','COMMD6','COMMD7','COPS2','COPS3','COPS4','CPEB1','CPSF4','CPSF4L','CPXCR1','CRAMP1','CREB1','CREB3','CREB3L1','CREB3L2','CREB3L3','CREB3L4','CREB5','CREBBP','CREBL2','CREBRF','CREBZF','CREG1','CREM','CRTC1','CRTC2','CRTC3','CRX','CSDC2','CSDE1','CSRNP1','CSRNP2','CSRNP3','CTBP1','CTBP2','CTCF','CTCFL','CTH','CTNNB1','CTNNBIP1','CUL1','CUL2','CUL3','CUL4A','CUL4B','CUL5','CUX1','CUX2','CXXC1','CXXC4','CXXC5','CYLD','CYP1B1','CYTL1','DAB2IP','DACH1','DACH2','DAP','DAXX','DBP','DBX1','DBX2','DCP1A','DDB1','DDB2','DDIT3','DDN','DDR2','DDRGK1','DDX3X','DDX5','DDX58','DEAF1','DEK','DEPDC1','DEPDC1B','DEPDC4','DEPDC5','DEPDC7','DEPTOR','DHX34','DHX36','DHX57','DISP3','DLX1','DLX2','DLX3','DLX4','DLX5','DLX6','DMAP1','DMBX1','DMC1','DMRT1','DMRT2','DMRT3','DMRTA1','DMRTA2','DMRTB1','DMRTC1','DMRTC2','DMTF1','DNA2','DNAJA3','DNAJC1','DNAJC2','DNAJC21','DNMT1','DNMT3A','DNMT3B','DNMT3L','DNTT','DNTTIP1','DOT1L','DPF1','DPF2','DPF3','DPRX','DR1','DRAP1','DRGX','DSP','DTX1','DUS3L','DUSP12','DUX1','DUX3','DUX4','DUXA','DVL1','DVL2','DVL3','DZIP1','DZIP1L','E2F1','E2F2','E2F3','E2F4','E2F5','E2F6','E2F7','E2F8','E4F1','EAF2','EBF1','EBF2','EBF3','EBF4','EDA','EDA2R','EDF1','EDN1','EEA1','EED','EGLN1','EGR1','EGR2','EGR3','EGR4','EHF','EID1','EID2','EIF2AK2','EIF2AK4','EIF3C','EIF3CL','EIF3K','ELF1','ELF2','ELF3','ELF4','ELF5','ELK1','ELK3','ELK4','ELMSAN1','EME1','EMX1','EMX2','EN1','EN2','ENO1','EOMES','EP300','EP400','EPAS1','EPHA5','EPM2AIP1','ERBIN','ERC1','ERCC3','ERCC4','ERF','ERG','ESR1','ESR2','ESRRA','ESRRB','ESRRG','ESX1','ETS1','ETS2','ETV1','ETV2','ETV3','ETV3L','ETV4','ETV5','ETV6','ETV7','EVX1','EVX2','EWSR1','EXO1','EXOC2','EXOG','EZH1','EZH2','FAAP24','FAM153C','FAM170A','FAM171B','FAM200A','FAM200B','FAN1','FANCA','FANCD2','FANK1','FARSA','FARSB','FBN1','FBXL19','FBXO41','FEN1','FER','FERD3L','FEV','FEZF1','FEZF2','FGD1','FHL2','FIGLA','FIZ1','FLI1','FLNA','FLYWCH1','FMNL2','FOS','FOSB','FOSL1','FOSL2','FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L2','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXN4','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1','FUBP1','FUBP3','FUS','FZD1','FZD2','FZD4','FZD6','GABPA','GABPB1','GABPB2','GADD45A','GAS6','GAS7','GATA1','GATA2','GATA3','GATA4','GATA5','GATA6','GATAD1','GATAD2A','GATAD2B','GBX1','GBX2','GCFC2','GCM1','GCM2','GEN1','GFI1','GFI1B','GLA','GLI1','GLI2','GLI3','GLI4','GLIS1','GLIS2','GLIS3','GLMP','GLYR1','GMEB1','GMEB2','GOLGB1','GON4L','GPATCH8','GPBP1','GPBP1L1','GPER1','GPR155','GREM1','GRHL1','GRHL2','GRHL3','GRM6','GSC','GSC2','GSX1','GSX2','GTF2A1','GTF2A1L','GTF2A2','GTF2B','GTF2E2','GTF2F1','GTF2F2','GTF2H1','GTF2H2','GTF2H2C','GTF2H3','GTF2H4','GTF2I','GTF2IRD1','GTF2IRD2','GTF2IRD2B','GTF3A','GTF3C1','GTF3C2','GULP1','GZF1','H1F0','H1FOO','H1FX','HAND1','HAND2','HAVCR2','HBP1','HCFC1','HCK','HDAC1','HDAC10','HDAC2','HDAC3','HDAC4','HDAC5','HDAC6','HDAC7','HDAC8','HDAC9','HDX','HELT','HELZ','HELZ2','HES1','HES2','HES3','HES4','HES5','HES6','HES7','HESX1','HEY1','HEY2','HEYL','HHEX','HIC1','HIC2','HIF1A','HIF3A','HINFP','HIPK2','HIRA','HIST1H1A','HIST1H1B','HIST1H1C','HIST1H1D','HIST1H1E','HIST1H1T','HIVEP1','HIVEP2','HIVEP3','HKR1','HLA-DQB1','HLA-DQB2','HLA-DRB1','HLA-DRB3','HLA-DRB5','HLF','HLTF','HLX','HMBOX1','HMG20A','HMG20B','HMGA1','HMGA2','HMGB1','HMGB2','HMGB3','HMGB4','HMGN1','HMGN3','HMGXB3','HMGXB4','HMOX1','HMX1','HMX2','HMX3','HNF1A','HNF1B','HNF4A','HNF4G','HNRNPA1','HNRNPA3','HNRNPAB','HNRNPCL2','HNRNPD','HNRNPDL','HNRNPK','HNRNPL','HNRNPU','HOMEZ','HOOK2','HOPX','HOXA1','HOXA10','HOXA11','HOXA13','HOXA2','HOXA3','HOXA4','HOXA5','HOXA6','HOXA7','HOXA9','HOXB1','HOXB13','HOXB2','HOXB3','HOXB4','HOXB5','HOXB6','HOXB7','HOXB8','HOXB9','HOXC10','HOXC11','HOXC12','HOXC13','HOXC4','HOXC5','HOXC6','HOXC8','HOXC9','HOXD1','HOXD10','HOXD11','HOXD12','HOXD13','HOXD3','HOXD4','HOXD8','HOXD9','HP1BP3','HR','HSBP1','HSF1','HSF2','HSF4','HSF5','HSFX1','HSFX2','HSFY1','HSFY2','HSPA1A','HSPA1B','ICAM1','ID1','ID2','ID3','ID4','IFI16','IFNB1','IGHM','IKBKB','IKBKG','IKZF1','IKZF2','IKZF3','IKZF4','IKZF5','IL10','IL1B','IL1RAP','IL4','IL5','IL6','INF2','ING4','INS','INSM1','INSM2','IQGAP1','IRAK1','IRAK2','IRAK3','IRF1','IRF2','IRF2BP1','IRF3','IRF4','IRF5','IRF6','IRF7','IRF8','IRF9','IRX1','IRX2','IRX3','IRX4','IRX5','IRX6','ISL1','ISL2','ISX','ITCH','ITGB2','ITGB3BP','IZUMO2','JADE1','JAK2','JARID2','JAZF1','JDP2','JMY','JRK','JRKL','JUN','JUNB','JUND','JUP','KANSL2','KAT2A','KAT2B','KAT5','KAT6A','KAT6B','KAT7','KAT8','KCMF1','KCNIP3','KDM1A','KDM2A','KDM2B','KDM3A','KDM5A','KDM5B','KDM5C','KDM5D','KEAP1','KIAA1549','KIAA1683','KIAA1841','KIAA1958','KIF15','KIN','KIT','KLF1','KLF10','KLF11','KLF12','KLF13','KLF14','KLF15','KLF16','KLF17','KLF2','KLF3','KLF4','KLF5','KLF6','KLF7','KLF8','KLF9','KMT2A','KMT2B','KMT2C','KMT2D','KNTC1','KRAS','KRBOX1','KRT13','KRTAP5-1','KRTAP5-9','KRTAP9-9','L3MBTL1','L3MBTL3','L3MBTL4','LARP1','LARP1B','LARP4','LARP4B','LARP6','LARP7','LBX1','LBX2','LCOR','LCORL','LEF1','LEUTX','LGALS9','LGR4','LHX1','LHX2','LHX3','LHX4','LHX5','LHX6','LHX8','LHX9','LIG1','LIG3','LIN28A','LIN28B','LIN54','LMO2','LMO4','LMX1A','LMX1B','LPP','LRP5','LRP6','LRP8','LRRFIP1','LRRFIP2','LSR','LTF','LTN1','LYAR','LYL1','LZTFL1','LZTR1','LZTS1','MACC1','MACF1','MAD2L2','MAEL','MAF','MAFA','MAFB','MAFF','MAFG','MAFK','MALT1','MAML1','MAML3','MAP2K5','MAP3K10','MAP3K13','MAP3K7','MAPK1','MAPK10','MAPK11','MAPK14','MAPK3','MAPK8','MAPK9','MATR3','MAVS','MAX','MAZ','MB21D1','MBD1','MBD2','MBD3','MBD4','MBD6','MBNL1','MBNL2','MBNL3','MBTPS2','MDFI','MDM2','MDM4','MECOM','MECP2','MED1','MED12','MED13','MED13L','MED14','MED15','MED16','MED17','MED21','MED23','MED24','MED6','MEF2A','MEF2B','MEF2C','MEF2D','MEIS1','MEIS2','MEIS3','MEN1','MEOX1','MEOX2','MESP1','MESP2','MET','MGA','MGMT','MID2','MIER1','MIER2','MIER3','MINK1','MIS18BP1','MITF','MIXL1','MKL1','MKL2','MKRN1','MKRN2','MKRN3','MKX','MLLT1','MLLT10','MLLT11','MLLT3','MLLT6','MLX','MLXIP','MLXIPL','MN1','MNAT1','MNT','MNX1','MORC3','MPG','MRPL28','MRRF','MSANTD1','MSANTD3','MSANTD4','MSC','MSGN1','MSH2','MSH3','MSH6','MSL3','MSRB2','MST1R','MSX1','MSX2','MTA1','MTA2','MTA3','MTDH','MTERF1','MTERF2','MTERF3','MTERF4','MTF1','MTF2','MTPN','MUS81','MXD1','MXD3','MXD4','MXI1','MYB','MYBBP1A','MYBL1','MYBL2','MYC','MYCBP','MYCL','MYCN','MYD88','MYEF2','MYF5','MYF6','MYNN','MYOCD','MYOD1','MYOG','MYPOP','MYRF','MYRFL','MYSM1','MYT1','MYT1L','MZF1','NAB2','NABP2','NACC2','NAIF1','NANOG','NANOGNB','NANOGP8','NCOA1','NCOA2','NCOA3','NCOA4','NCOA5','NCOA6','NCOA7','NCOR1','NCOR2','NDC80','NDN','NDP','NEB','NEIL1','NEUROD1','NEUROD2','NEUROD4','NEUROD6','NEUROG1','NEUROG2','NEUROG3','NFAM1','NFAT5','NFATC1','NFATC2','NFATC3','NFATC4','NFE2','NFE2L1','NFE2L2','NFE2L3','NFE4','NFIA','NFIB','NFIC','NFIL3','NFIX','NFKB1','NFKB2','NFKBIA','NFKBIB','NFKBID','NFKBIL1','NFKBIZ','NFRKB','NFX1','NFXL1','NFYA','NFYB','NFYC','NHLH1','NHLH2','NKRF','NKX1-1','NKX1-2','NKX2-1','NKX2-2','NKX2-3','NKX2-4','NKX2-5','NKX2-6','NKX2-8','NKX3-1','NKX3-2','NKX6-1','NKX6-2','NKX6-3','NLRC3','NLRC4','NLRC5','NLRP12','NLRP2B','NLRP3','NME2','NOBOX','NOC3L','NOC4L','NOCT','NOD1','NOD2','NODAL','NONO','NOP2','NOTCH1','NOTCH2','NOTCH3','NOTO','NPAS1','NPAS2','NPAS3','NPAS4','NPM1','NR0B1','NR0B2','NR1D1','NR1D2','NR1H2','NR1H3','NR1H4','NR1I2','NR1I3','NR2C1','NR2C2','NR2E1','NR2E3','NR2F1','NR2F2','NR2F6','NR3C1','NR3C2','NR4A1','NR4A2','NR4A3','NR5A1','NR5A2','NR6A1','NRBF2','NRF1','NRG1','NRIP1','NRIP3','NRK','NRL','NSD1','NSD2','NTRK1','NUCKS1','NUFIP1','NUP98','NUPL2']
TFs2Other=['OGG1','OLIG1','OLIG2','OLIG3','ONECUT1','ONECUT2','ONECUT3','OPRD1','OSR1','OSR2','OTOP3','OTP','OTULIN','OTX1','OTX2','OVOL1','OVOL2','OVOL3','PA2G4','PABPN1','PARK7','PARP1','PARP10','PARP12','PATZ1','PAWR','PAX1','PAX2','PAX3','PAX4','PAX5','PAX6','PAX7','PAX8','PAX9','PAXBP1','PAXIP1','PBRM1','PBX1','PBX2','PBX3','PBX4','PCBD1','PCBP1','PCBP2','PCBP3','PCGF2','PCGF6','PCNA','PCSK6','PDCD7','PDS5B','PDX1','PEG3','PELI1','PER1','PER2','PER3','PEX14','PFDN1','PGBD1','PGR','PGS1','PHB','PHB2','PHF1','PHF19 ','PHF20','PHF21A','PHF5A','PHOX2A','PHOX2B','PHTF1','PHTF2','PIAS1','PIAS2','PIAS3','PIAS4','PIDD1','PIKFYVE','PIM1','PIN1','PINK1','PITX1','PITX2','PITX3','PKHD1','PKHD1L1','PKNOX1','PKNOX2','PLA2G10','PLA2G1B','PLAG1','PLAGL1','PLAGL2','PLEK','PLEK2','PLEKHA4','PLPP3','PLSCR1','PLXNA1','PLXNA2','PLXNA3','PLXNA4','PLXNB1','PLXNB2','PLXNB3','PLXNC1','PLXND1','PML','PMS1','PNKP','POGK','POGZ','POLA1','POLB','POLE3','POLE4','POLG','POLH','POLI','POLK','POLL','POLM','POLN','POLQ','POLR1B','POLR2A','POLR2B','POLR2E','POLR2J','POLR2K','POLR2L','POLR3A','POLRMT','POT1','POU1F1','POU2AF1','POU2F1','POU2F2','POU2F3','POU3F1','POU3F2','POU3F3','POU3F4','POU4F1','POU4F2','POU4F3','POU5F1','POU5F1B','POU5F2','POU6F1','POU6F2','PPARA','PPARD','PPARG','PPARGC1A','PPARGC1B','PPP1R10','PPP1R13L','PPP1R21','PPP2R3B','PPP2R5B','PPP3CA','PPRC1','PQBP1','PRAM1','PRAP1','PRB3','PRDM1','PRDM10','PRDM11','PRDM12','PRDM13','PRDM14','PRDM15','PRDM16','PRDM2','PRDM4','PRDM5','PRDM6','PRDM7','PRDM8','PRDM9','PRDX3','PREB','PREX1','PREX2','PRH2','PRIM2','PRIMPOL','PRKCB','PRKCH','PRKCI','PRKCQ','PRKCZ','PRKD1','PRKD2','PRKN','PRMT2','PRMT3','PRNP','PROP1','PROX1','PROX2','PRR12','PRR3','PRRX1','PRRX2','PSMA6','PSMC5','PSMD10','PSMD11','PSMD12','PSPC1','PTCH1','PTEN','PTF1A','PTGIS','PTH','PTTG1','PUM3','PURA','PURB','PURG','PYCARD','PYDC1','PYDC2','QRICH1','RAB11FIP4','RAB7B','RAD21','RAD51','RAG1','RAI1','RAPGEF3','RAPGEF4','RAPGEF5','RARA','RARB','RARG','RAX','RAX2','RB1','RBAK','RBBP7','RBCK1','RBL1','RBL2','RBM10','RBM20','RBM22','RBM26','RBM27','RBM5','RBM6','RBPJ','RBPJL','RBSN','RC3H1','RC3H2','RCAN1','RCOR1','RCOR2','RCOR3','RECQL','REL','RELA','RELB','RELL2','RELN','REPIN1','RERE','REST','REV1','REXO4','RFC1','RFX1','RFX2','RFX3','RFX4','RFX5','RFX6','RFX7','RFX8','RFXANK','RFXAP','RGCC','RGS11','RGS6','RGS7','RGS9','RHEBL1','RHOXF1','RHOXF2','RHOXF2B','RING1','RIOK2','RIPK1','RIPK2','RIPK3','RIPK4','RLF','RLIM','RNASE2','RNASEH1','RNF113A','RNF113B','RNF114','RNF125','RNF138','RNF14','RNF166','RNF2','RNF219','RNF25','RNF31','RNF4','RNF41','RORA','RORB','RORC','RPA2','RPA4','RPL36A','RPL7','RPL7A','RPS27A','RPS6KA4','RPS6KA5','RREB1','RRN3','RUNX1','RUNX1T1','RUNX2','RUNX3','RWDD3','RXRA','RXRB','RXRG','RYBP','S100A12','S100A8','S100A9','SAFB','SAFB2','SALL1','SALL2','SALL3','SALL4','SART1','SATB1','SATB2','SAV1','SCAND1','SCAND2P','SCAPER','SCMH1','SCML1','SCML2','SCML4','SCOC','SCRT1','SCRT2','SCX','SEBOX','SEMA4A','SERTAD1','SERTAD2','SETBP1','SETD6','SETDB1','SETDB2','SF3A2','SF3A3','SFPQ','SFRP4','SFRP5','SFTPB','SGK1','SGK3','SGSM2','SHH','SHOX','SHOX2','SHPRH','SIGIRR','SIK1','SIM1','SIM2','SIN3A','SIN3B','SIRT1','SIRT2','SIRT3','SIRT6','SIVA1','SIX1','SIX2','SIX3','SIX4','SIX5','SIX6','SKI','SKIDA1','SKIL','SKOR1','SKOR2','SLC22A4','SLC26A10','SLC26A3','SLC2A4RG','SLC30A9','SLC39A10','SLC4A10','SMAD1','SMAD2','SMAD3','SMAD4','SMAD5','SMAD6','SMAD7','SMAD9','SMARCA1','SMARCA2','SMARCA4','SMARCA5','SMARCAL1','SMARCB1','SMARCC1','SMARCC2','SMARCD1','SMARCD2','SMARCD3','SMARCE1','SMC3','SMO','SMUG1','SMYD1','SMYD3','SNAI1','SNAI2','SNAI3','SNAPC2','SNAPC4','SNAPC5','SNRPC','SNURF','SNW1','SOGA3','SOHLH1','SOHLH2','SON','SORBS2','SOX1','SOX10','SOX11','SOX12','SOX13','SOX14','SOX15','SOX17','SOX18','SOX2','SOX21','SOX3','SOX30','SOX4','SOX5','SOX6','SOX7','SOX8','SOX9','SP1','SP100','SP110','SP140','SP140L','SP2','SP3','SP4','SP5','SP6','SP7','SP8','SP9','SPDEF','SPEN','SPHK1','SPI1','SPIB','SPIC','SPZ1','SRCAP','SREBF1','SREBF2','SRF','SRY','SSB','SSBP2','SSBP3','SSBP4','SSH1','SSH2','SSH3','SSRP1','SSX1','ST18','STAG1','STAT1','STAT2','STAT3','STAT4','STAT5A','STAT5B','STAT6','STK16','STK3','STK36','STRN','STRN3','SUB1','SUFU','SUMO1','SUPT16H','SUPT4H1','SUPT5H','SUPT6H','SUV39H1','SUV39H2','SUZ12','SYK','T','TAB1','TAB2','TAB3','TADA2A','TADA2B','TADA3','TAF1','TAF10','TAF11','TAF12','TAF13','TAF1A','TAF1B','TAF1C','TAF1L','TAF2','TAF3','TAF4','TAF4B','TAF5','TAF5L','TAF6','TAF6L','TAF7','TAF7L','TAF8','TAF9','TAF9B','TAL1','TAL2','TARDBP','TAX1BP1','TBL1Y','TBP','TBPL1','TBPL2','TBR1','TBX1','TBX10','TBX15','TBX18','TBX19','TBX2','TBX20','TBX21','TBX22','TBX3','TBX4','TBX5','TBX6','TCEA2','TCEAL1','TCEAL7','TCEAL8','TCF12','TCF15','TCF19','TCF20','TCF21','TCF23','TCF24','TCF25','TCF3','TCF4','TCF7','TCF7L1','TCF7L2','TCFL5','TDG','TDP1','TDP2','TEAD1','TEAD2','TEAD3','TEAD4','TEF','TERB1','TERF1','TERF2','TERF2IP','TESMIN','TET1','TET2','TET3','TFAM','TFAP2A','TFAP2B','TFAP2C','TFAP2D','TFAP2E','TFAP4','TFCP2','TFCP2L1','TFDP1','TFDP2','TFDP3','TFE3','TFEB','TFEC','TGFB1','TGFB1I1','TGIF1','TGIF2','TGIF2LX','TGIF2LY','TGS1','THAP1','THAP10','THAP11','THAP12','THAP2','THAP3','THAP4','THAP5','THAP6','THAP7','THAP8','THAP9','THRA','THRAP3','THRB','THYN1','TIA1','TICAM1','TIGD1','TIGD2','TIGD3','TIGD4','TIGD5','TIGD6','TIGD7','TIPARP','TIRAP','TLE1','TLE2','TLE3','TLE4','TLE6','TLR2','TLR3','TLR4','TLR9','TLX1','TLX2','TLX3','TMEM235','TMF1','TNF','TNFAIP3','TNFRSF11A','TNFRSF4','TNFSF11','TNFSF18','TNFSF4','TOE1','TONSL','TOP1','TOP2A','TOP2B','TOPORS','TOX','TOX2','TOX3','TOX4','TP53','TP53BP1','TP63','TP73','TPRX1','TRADD','TRAF1','TRAF2','TRAF3','TRAF4','TRAF5','TRAF6','TRAFD1','TRAK2','TRAPPC9','TRERF1','TREX1','TRIB1','TRIB3','TRIM13','TRIM14','TRIM15','TRIM21','TRIM22','TRIM23','TRIM24','TRIM25','TRIM26','TRIM27','TRIM28','TRIM29','TRIM3','TRIM31','TRIM32','TRIM33','TRIM34','TRIM37','TRIM38','TRIM40','TRIM5','TRIM52','TRIM62','TRIM8','TRIP11','TRIT1','TRMT1','TRMT1L','TRPS1','TRRAP','TSC2','TSC22D1','TSC22D2','TSC22D3','TSC22D4','TSHZ1','TSHZ2','TSHZ3','TSN','TSSK4','TTF1','TUB','TULP4','TUT1','TWIST1','TWIST2','TXK','U2AF1','U2AF1L4','UBA52','UBB','UBC','UBE2K','UBE2N','UBE2V1','UBN1','UBP1','UBR4','UBTF','UBTFL1','UBTFL6','UFL1','UHRF1','UHRF2','UNCX','UNG','UNK','UNKL','USF1','USF2','USF3','USP39','USP7','UTF1','VAV1','VAX1','VAX2','VDR','VEGFA','VENTX','VEZF1','VGLL1','VGLL2','VGLL3','VHLL','VPS72','VSX1','VSX2','WDHD1','WFS1','WIZ','WNT1','WNT10B','WNT2','WNT3A','WNT5A','WNT8B','WRN','WT1','WWOX','WWP2','WWTR1','XBP1','XCL1','XPA','YBX1','YBX2','YBX3','YEATS2','YEATS4','YOD1','YY1','YY2','ZBED1','ZBED2','ZBED3','ZBED4','ZBED5','ZBED6','ZBED6CL','ZBED8','ZBED9','ZBP1','ZBTB1','ZBTB10','ZBTB11','ZBTB12','ZBTB14','ZBTB16','ZBTB17','ZBTB18','ZBTB2','ZBTB20','ZBTB21','ZBTB22','ZBTB24','ZBTB25','ZBTB26','ZBTB3','ZBTB32','ZBTB33','ZBTB34','ZBTB37','ZBTB38','ZBTB39','ZBTB4','ZBTB40','ZBTB41','ZBTB42','ZBTB43','ZBTB44','ZBTB45','ZBTB46','ZBTB47','ZBTB48','ZBTB49','ZBTB5','ZBTB6','ZBTB7A','ZBTB7B','ZBTB7C','ZBTB8A','ZBTB8B','ZBTB9','ZC3H10','ZC3H11A','ZC3H12A','ZC3H13','ZC3H14','ZC3H15','ZC3H18','ZC3H3','ZC3H4','ZC3H6','ZC3H7A','ZC3H7B','ZC3H8','ZC3HAV1','ZCCHC11','ZCCHC6','ZDHHC11','ZDHHC19','ZEB1','ZEB2','ZFAND5','ZFAT','ZFHX2','ZFHX3','ZFHX4','ZFP1','ZFP14','ZFP2','ZFP28','ZFP3','ZFP30','ZFP36','ZFP36L1','ZFP36L2','ZFP37','ZFP41','ZFP42','ZFP57','ZFP62','ZFP64','ZFP69','ZFP69B','ZFP82','ZFP90','ZFP91','ZFP92','ZFPL1','ZFPM1','ZFPM2','ZFR','ZFR2','ZFX','ZFY','ZFYVE26','ZGLP1','ZGPAT','ZHX1','ZHX2','ZHX3','ZIC1','ZIC2','ZIC3','ZIC4','ZIC5','ZIK1','ZIM2','ZIM3','ZKSCAN1','ZKSCAN2','ZKSCAN3','ZKSCAN4','ZKSCAN5','ZKSCAN7','ZKSCAN8','ZMAT1','ZMAT2','ZMAT3','ZMAT4','ZMAT5','ZMYM1','ZMYM2','ZMYM3','ZMYM4','ZMYM5','ZMYM6','ZMYND10','ZNF10','ZNF100','ZNF101','ZNF106','ZNF107','ZNF112','ZNF114','ZNF117','ZNF12','ZNF121','ZNF124','ZNF131','ZNF132','ZNF133','ZNF134','ZNF135','ZNF136','ZNF137P','ZNF138','ZNF14','ZNF140','ZNF141','ZNF142','ZNF143','ZNF146','ZNF148','ZNF154','ZNF155','ZNF157','ZNF16','ZNF160','ZNF165','ZNF169','ZNF17','ZNF174','ZNF175','ZNF177','ZNF18','ZNF180','ZNF181','ZNF182','ZNF184','ZNF185','ZNF189','ZNF19','ZNF195','ZNF197','ZNF2','ZNF20','ZNF200','ZNF202','ZNF205','ZNF207','ZNF208','ZNF211','ZNF212','ZNF213','ZNF214','ZNF215','ZNF217','ZNF219','ZNF22','ZNF221','ZNF222','ZNF223','ZNF224','ZNF225','ZNF226','ZNF227','ZNF229','ZNF23','ZNF230','ZNF232','ZNF233','ZNF234','ZNF235','ZNF236','ZNF239','ZNF24','ZNF248','ZNF25','ZNF250','ZNF251','ZNF253','ZNF254','ZNF256','ZNF257','ZNF26','ZNF260','ZNF263','ZNF264','ZNF266','ZNF267','ZNF268','ZNF273','ZNF274','ZNF275','ZNF276','ZNF277','ZNF28','ZNF280A','ZNF280B','ZNF280C','ZNF280D','ZNF281','ZNF282','ZNF283','ZNF284','ZNF285','ZNF286A','ZNF286B','ZNF287','ZNF292','ZNF296','ZNF3','ZNF30','ZNF300','ZNF302','ZNF304','ZNF311','ZNF316','ZNF317','ZNF318','ZNF319','ZNF32','ZNF320','ZNF322','ZNF324','ZNF324B','ZNF326','ZNF329','ZNF330','ZNF331','ZNF333','ZNF334','ZNF335','ZNF337','ZNF33A','ZNF33B','ZNF34','ZNF341','ZNF343','ZNF345','ZNF346','ZNF347','ZNF35','ZNF350','ZNF354A','ZNF354B','ZNF354C','ZNF355P','ZNF358','ZNF362','ZNF365','ZNF366','ZNF367','ZNF37A','ZNF382','ZNF383','ZNF384','ZNF385A','ZNF385B','ZNF385C','ZNF385D','ZNF391','ZNF394','ZNF395','ZNF396','ZNF397','ZNF398','ZNF404','ZNF407','ZNF408','ZNF41','ZNF410','ZNF414','ZNF415','ZNF416','ZNF417','ZNF418','ZNF419','ZNF420','ZNF423','ZNF425','ZNF426','ZNF428','ZNF429','ZNF43','ZNF430','ZNF431','ZNF432','ZNF433','ZNF436','ZNF438','ZNF439','ZNF44','ZNF440','ZNF441','ZNF442','ZNF443','ZNF444','ZNF445','ZNF446','ZNF449','ZNF45','ZNF451','ZNF454','ZNF460','ZNF461','ZNF462','ZNF467','ZNF468','ZNF469','ZNF470','ZNF471','ZNF473','ZNF474','ZNF479','ZNF48','ZNF480','ZNF483','ZNF484','ZNF485','ZNF486','ZNF487','ZNF488','ZNF490','ZNF491','ZNF492','ZNF493','ZNF496','ZNF497','ZNF500','ZNF501','ZNF502','ZNF503','ZNF506','ZNF507','ZNF510','ZNF511','ZNF512','ZNF512B','ZNF513','ZNF514','ZNF516','ZNF517','ZNF518A','ZNF518B','ZNF519']
TFs3Other=['ZNF521','ZNF524','ZNF525','ZNF526','ZNF527','ZNF528','ZNF529','ZNF530','ZNF532','ZNF534','ZNF536','ZNF540','ZNF541','ZNF542P','ZNF543','ZNF544','ZNF546','ZNF547','ZNF548','ZNF549','ZNF550','ZNF551','ZNF552','ZNF554','ZNF555','ZNF556','ZNF557','ZNF558','ZNF559','ZNF560','ZNF561','ZNF562','ZNF563','ZNF564','ZNF565','ZNF566','ZNF567','ZNF568','ZNF569','ZNF57','ZNF570','ZNF571','ZNF572','ZNF573','ZNF574','ZNF575','ZNF576','ZNF577','ZNF578','ZNF579','ZNF580','ZNF581','ZNF582','ZNF583','ZNF584','ZNF585A','ZNF585B','ZNF586','ZNF587','ZNF587B','ZNF589','ZNF592','ZNF593','ZNF594','ZNF595','ZNF596','ZNF597','ZNF598','ZNF599','ZNF600','ZNF605','ZNF606','ZNF607','ZNF608','ZNF609','ZNF610','ZNF611','ZNF613','ZNF614','ZNF615','ZNF616','ZNF618','ZNF619','ZNF620','ZNF621','ZNF622','ZNF623','ZNF624','ZNF625','ZNF626','ZNF627','ZNF628','ZNF629','ZNF630','ZNF638','ZNF639','ZNF641','ZNF644','ZNF645','ZNF646','ZNF648','ZNF649','ZNF652','ZNF653','ZNF654','ZNF655','ZNF658','ZNF66','ZNF660','ZNF662','ZNF664','ZNF665','ZNF667','ZNF668','ZNF669','ZNF670','ZNF671','ZNF672','ZNF674','ZNF675','ZNF676','ZNF677','ZNF678','ZNF679','ZNF680','ZNF681','ZNF682','ZNF683','ZNF684','ZNF687','ZNF688','ZNF689','ZNF69','ZNF691','ZNF692','ZNF695','ZNF696','ZNF697','ZNF699','ZNF7','ZNF70','ZNF700','ZNF701','ZNF703','ZNF704','ZNF705A','ZNF705B','ZNF705D','ZNF705E','ZNF705G','ZNF706','ZNF707','ZNF708','ZNF709','ZNF71','ZNF710','ZNF711','ZNF713','ZNF714','ZNF716','ZNF717','ZNF718','ZNF720','ZNF721','ZNF724','ZNF726','ZNF727','ZNF728','ZNF729','ZNF73','ZNF730','ZNF732','ZNF735','ZNF736','ZNF737','ZNF738','ZNF74','ZNF740','ZNF746','ZNF747','ZNF749','ZNF750','ZNF75A','ZNF75D','ZNF76','ZNF761','ZNF763','ZNF764','ZNF765','ZNF766','ZNF768','ZNF77','ZNF770','ZNF771','ZNF772','ZNF773','ZNF774','ZNF775','ZNF776','ZNF777','ZNF778','ZNF780A','ZNF780B','ZNF781','ZNF782','ZNF783','ZNF784','ZNF785','ZNF786','ZNF787','ZNF788','ZNF789','ZNF79','ZNF790','ZNF791','ZNF792','ZNF793','ZNF799','ZNF8','ZNF80','ZNF800','ZNF804A','ZNF804B','ZNF805','ZNF808','ZNF81','ZNF812P','ZNF813','ZNF814','ZNF816','ZNF821','ZNF823','ZNF827','ZNF829','ZNF83','ZNF830','ZNF831','ZNF835','ZNF836','ZNF837','ZNF839','ZNF84','ZNF841','ZNF843','ZNF844','ZNF845','ZNF846','ZNF85','ZNF850','ZNF852','ZNF853','ZNF860','ZNF862','ZNF865','ZNF878','ZNF879','ZNF880','ZNF883','ZNF888','ZNF891','ZNF90','ZNF91','ZNF92','ZNF93','ZNF98','ZNF99','ZNFX1','ZRANB2','ZRSR1','ZRSR2','ZSCAN1','ZSCAN10','ZSCAN12','ZSCAN16','ZSCAN18','ZSCAN2','ZSCAN20','ZSCAN21','ZSCAN22','ZSCAN23','ZSCAN25','ZSCAN26','ZSCAN29','ZSCAN30','ZSCAN31','ZSCAN32','ZSCAN4','ZSCAN5A','ZSCAN5B','ZSCAN5C','ZSCAN5DP','ZSCAN9','ZUFSP','ZXDA','ZXDB','ZXDC','ZZZ3']
TFs4Other=['ATF2','ATF6B','ATN1','BHLHA15','BHLHB2','BHLHB4 ','BHLHB5','BHLHB9','C13ORF8','C20ORF194','CDX2','CRAMP1L','ELP3','ERVK3-1','ETF1','FLJ12895','FLJ20321','FLJ25476','FLJ35784','FOXM1','FOXO2','FOXR1','FOXR2','GATA1','GRLF1','KDM4C','KDM4E','KIAA0863','KIAA0961','KLRG1','LBXCOR1','LEF1','LHX7','LOC126295','LOC51058','LOC91661','MGC4728','MYOD1','MYOG','MYRF','NEUROD4','NEUROG1','NFAT2','NFAT4','NFE2L1','NR1A2','NR1B1','NR1B2','NR1B3','NR1C1','NR1C2','NR1C3','NR1F3','NR1I1','NR2A1','NR2B3','NR3A1','NR3A2','NR3B3','OTEX','PEPP-2','PHF2','POU2F1','POU2F2','POU4F3','POU5F1','POU6F1','PRDM3','PRKRIP1','PRKRIR','SETD1A','SMAD8','SMYD2','SOX12','SOX14','SOX15','TBX14','TCF1','TCF17','TCF2','TCF9','TEAD1','TPRXL','UBTFL1','WHSC1','ZBTB14','ZBTB18','ZBTB19','ZBTB21','ZBTB27','ZBTB28','ZBTB35','ZBTB49','ZFP161','ZFP28','ZKSCAN10','ZKSCAN11','ZKSCAN12','ZKSCAN13','ZKSCAN14','ZKSCAN16','ZKSCAN19','ZKSCAN21','ZKSCAN24','ZKSCAN6','ZKSCAN8','ZNF10','ZNF112','ZNF187','ZNF193','ZNF204P','ZNF238','ZNF271','ZNF295','ZNF305','ZNF323','ZNF434','ZNF499','ZNF542','ZNF545','ZNF553','ZNF56','ZNF625','ZNF643','ZNF698','ZNF705F','ZNF751','ZNF752','ZNF753','ZNF785','ZNF806','ZNF816','ZNF833P','ZNF906','ZSCAN1','ZSCAN25','ZSCAN26','ZSCAN31','ZSCAN32','ZSCAN5D','ZSCAN9']
TFsOther=TFs1Other+TFs2Other+TFs3Other+TFs4Other
```

Example 8 (unknown):
```unknown
S genes: ['MCM5', 'PCNA', 'TYMS', 'FEN1', 'MCM2', 'MCM4', 'RRM1', 'UNG', 'GINS2', 'MCM6', 'CDCA7', 'DTL', 'PRIM1', 'UHRF1', 'HELLS', 'RFC2', 'RPA2', 'NASP', 'RAD51AP1', 'GMNN', 'WDR76', 'SLBP', 'CCNE2', 'UBR7', 'POLD3', 'MSH2', 'ATAD2', 'RAD51', 'RRM2', 'CDC45', 'CDC6', 'EXO1', 'TIPIN', 'DSCC1', 'BLM', 'CASP8AP2', 'USP1', 'CLSPN', 'POLA1', 'CHAF1B', 'BRIP1', 'E2F8'] 

G2 genes: ['HMGB2', 'CDK1', 'NUSAP1', 'UBE2C', 'BIRC5', 'TPX2', 'TOP2A', 'NDC80', 'CKS2', 'NUF2', 'CKS1B', 'MKI67', 'TMPO', 'CENPF', 'TACC3', 'SMC4', 'CCNB2', 'CKAP2L', 'CKAP2', 'AURKB', 'BUB1', 'KIF11', 'ANP32E', 'TUBB4B', 'GTSE1', 'KIF20B', 'HJURP', 'CDCA3', 'CDC20', 'TTK', 'CDC25C', 'KIF2C', 'RANGAP1', 'NCAPD2', 'DLGAP5', 'CDCA2', 'CDCA8', 'ECT2', 'KIF23', 'HMMR', 'AURKA', 'PSRC1', 'ANLN', 'LBR', 'CKAP5', 'CENPE', 'CTCF', 'NEK2', 'G2E3', 'GAS2L3', 'CBX5', 'CENPA'] 

Hemoglobin genes set 1: ['HBB', 'HBD', 'HBG1', 'HBG2', 'HBE1', 'HBZ', 'HBM', 'HBA2', 'HBA1', 'HBQ1'] 

Hemoglobin genes set 2: ['AHSP', 'GYPA', 'RHAG', 'GFI1B', 'KEL', 'EPB42', 'ALAS2', 'TSPO2', 'GYPB', 'GYPE', 'SPTA1', 'RHCE', 'KLF1', 'RHD', 'SLC4A1', 'SNX22', 'SQSTM1', 'SOD1', 'FTH1', 'MAP1LC3B', 'SLC25A37', 'TRIM58', 'HMBS'] 

Mitochondrial genes: ['MT-ND1', 'MT-ND2', 'MT-CO1', 'MT-CO2', 'MT-ATP8', 'MT-ATP6', 'MT-CO3', 'MT-ND3', 'MT-ND4L', 'MT-ND4', 'MT-ND5', 'MT-ND6', 'MT-CYB'] 

Ribo genes: ['RPLP0', 'RPLP1', 'RPLP2', 'RPL3', 'RPL3L', 'RPL4', 'RPL5', 'RPL6', 'RPL7', 'RPL7A', 'RPL7L1', 'RPL8', 'RPL9', 'RPL10', 'RPL10A', 'RPL10L', 'RPL11', 'RPL12', 'RPL13', 'RPL13A', 'RPL14', 'RPL15', 'RPL17', 'RPL18', 'RPL18A', 'RPL19', 'RPL21', 'RPL22', 'RPL22L1', 'RPL23', 'RPL23A', 'RPL24', 'RPL26', 'RPL26L1', 'RPL27', 'RPL27A', 'RPL28', 'RPL29', 'RPL30', 'RPL31', 'RPL32', 'RPL34', 'RPL35', 'RPL35A', 'RPL36', 'RPL36A', 'RPL36AL', 'RPL37', 'RPL37A', 'RPL38', 'RPL39', 'RPL39L', 'UBA52', 'RPL41', 'RPSA', 'RPS2', 'RPS3', 'RPS3A', 'RPS4X', 'RPS4Y1', 'RPS4Y2', 'RPS5', 'RPS6', 'RPS7', 'RPS8', 'RPS9', 'RPS10', 'RPS11', 'RPS12', 'RPS13', 'RPS14', 'RPS15', 'RPS15A', 'RPS16', 'RPS17', 'RPS18', 'RPS19', 'RPS20', 'RPS21', 'RPS23', 'RPS24', 'RPS25', 'RPS26', 'RPS27', 'RPS27A', 'RPS27L', 'RPS28', 'RPS29', 'FAU'] 

Tissue dissociation stress associated genes: ['FOS', 'FOSB', 'JUN', 'JUNB', 'JUND', 'ZFP36', 'SOCS3', 'CEBPD', 'CEBPB', 'ATF3', 'EGR1', 'HSPA1A', 'HSPA1B', 'HSP90AA1', 'HSP90AB1', 'HSPA8', 'HSPB1', 'HSPE1', 'IER3', 'IER2', 'BTG1', 'BTG2', 'DUSP1']
```

Example 9 (python):
```python
#Expression: "YlGn" (e.g. DE plots)
#Divergent expression: "PRGn" (e.g. log2FC)
#Age: "afmhot_r"
#Others: "viridis" (e.g. UMAP expression)

#Define color palette
FineGrainPalette={
    'AtrialCardiomyocytesLeft':'#eac075',
    'AtrialCardiomyocytesRight':'#4c6b8a',
    'AtrialCardiomyocytesCycling':'#e94bc7',
    'VentricularCardiomyocytesLeftCompact':'#ab1c3c',
    'VentricularCardiomyocytesRightCompact':'#fdba45',
    'VentricularCardiomyocytesLeftTrabeculated':'#2b9200',
    'VentricularCardiomyocytesRightTrabeculated':'#0273a8',
    'VentricularCardiomyocytesCycling':'#e1acff',
    'SinoatrialNodePacemakerCells':'#854834',
    'AtrioventricularNodePacemakerCells':'#688500',
    'VentricularConductionSystemProximal':'#01ccc8',
    'VentricularConductionSystemDistal':'#b58fab',
    'GreatVesselAdventitialFibroblasts':'#ff7e97',
    'CoronaryVesselAdventitialFibroblasts':'#ff9b6e',
    'MyocardialInterstitialFibroblasts':'#8a6800',
    'SubEpicardialFibroblasts':'#206601',
    'Myofibroblasts':'#1bdebf',
    'LymphNodeFibroblasticReticularCells':'#005aa0',
    'ValveInterstitialCells':'#e478ff',
    'GreatVesselSmoothMuscleCells':'#ff665f',
    'CoronarySmoothMuscleCells':'#d2cb18',
    'DuctusArteriosusSmoothMuscleCells':'#83d7b3',
    'CoronaryPericytes':'#9088ff',
    'PericardialCellsIntermediate':'#c95e00',
    'PericardialCellsFibrous':'#01e077',
    'PericardialCellsParietal':'#ff88e4',
    'GreatVesselArterialEndothelialCells':'#d4004b',
    'GreatVesselVenousEndothelialCells':'#8f4300',
    'CoronaryArterialEndothelialCells':'#89da5a',
    'CoronaryVenousEndothelialCells':'#006f86',
    'CoronaryCapillaryEndothelialCells':'#aa7dff',
    'EndocardialCells':'#ff8241',
    'EndocardialCushionCells':'#46603a',
    'ValveEndothelialCells':'#414eb6',
    'LymphaticEndothelialCells':'#8b3f60',
    'MesothelialEpicardialCells':'#0295fe',
    'EpicardiumDerivedCells':'#ff92c4',
    'NeuronPrecursors':'#d30038',
    'ChromaffinCells':'#726800',
    'SympatheticNeurons':'#01894b',
    'ParaympatheticNeurons':'#545191',
    'SchwannCellPrecursors':'#b0d276',
    'SchwannCells':'#b30079',
    'MonocytesMPOpos':'#c50066',
    'Monocytes':'#fab4b6',
    'MonocyteDerivedCells':'#d14008',
    'MacrophagesCX3CR1pos':'#eabe97',
    'MacrophagesTIMD4pos':'#b2a200',
    'MacrophagesLYVE1pos':'#4a6005',
    'MacrophagesATF3pos':'#3db833',
    'DendriticCellsType1':'#007859',
    'DendriticCellsMature':'#01a9db',
    'PlasmacytoidDendriticCells':'#2c54ab',
    'MastCells':'#d0bef2',
    'Megakaryocytes':'#94298f',
    'TCellsCD4pos':'#ff387c',
    'TCellsCD8pos':'#9e3429',
    'TregsCD4pos':'#b37700',
    'ProBCells':'#8dab00',
    'BCells':'#a8d1a1',
    'BCellsMS4A1pos':'#00d2f0',
    'NaturalKillerCells':'#abb4ff',
    'InnateLymphoidCells':'#72497f',
}
sc.pl.umap(adata, color='fine_grain',palette=FineGrainPalette)

MidGrainPalette={
    'AtrialCardiomyocytes':'#af3a4d',
    'VentricularCardiomyocytes':'#b1756c',
    'CardiacConductionSystem':'#d45633',
    'Fibroblasts':'#9b6e4e',
    'MuralCells':'#ffb92f',
    'PericardialCells':'#b4f03b',
    'BloodVesselEndothelialCells':'#72cc57',
    'EndocardialCells':'#436e43',
    'LymphaticEndothelialCells':'#85cca0',
    'EpicardialCells':'#5eabb0',
    'Neurons':'#7bbdff',
    'Glia':'#403c90',
    'MyeloidCells':'#420066',
    'LymphoidCells':'#f553f6',
}
sc.pl.umap(adata, color='mid_grain',palette=MidGrainPalette)

CoarseGrainPalette={
    'Cardiomyocytes':'#bc5950',
    'Mesenchymal':'#af9c4c',
    'Endothelium':'#599e58',
    'Epicardium':'#5eabb0',
    'Neural':'#5e75b1',
    'Leukocytes':'#9a52a0',
}
sc.pl.umap(adata, color='coarse_grain',palette=CoarseGrainPalette)

RegionGrainPalette={
    'aorta':'#ffa2bc',
    'apex':'#fe0049',
    'atria':'#732d00',
    'atrial septum':'#ffd54c',
    'base':'#837100',
    'ductus arteriosus':'#bdff8b',
    'whole sample':'#1b6c00',
    'heart without node':'#02e6c6',
    'left ventricle':'#01c1f5',
    'node':'#00466d',
    'outflow tract':'#020011',
    'pericardium':'#7a75ff',
    'pulmonary arches and branches':'#30003e',
    'right ventricle':'#ff88e1',
    'right ventricle and papillaries':'#98004f',
}
sc.pl.umap(adata, color='region',palette=RegionGrainPalette)
```

Example 10 (python):
```python
set(adata.obs.fine_grain)
```

Example 11 (unknown):
```unknown
{'AtrialCardiomyocytesCycling',
 'AtrialCardiomyocytesLeft',
 'AtrialCardiomyocytesRight',
 'AtrioventricularNodePacemakerCells',
 'BCells',
 'BCellsMS4A1pos',
 'ChromaffinCells',
 'CoronaryArterialEndothelialCells',
 'CoronaryCapillaryEndothelialCells',
 'CoronaryPericytes',
 'CoronarySmoothMuscleCells',
 'CoronaryVenousEndothelialCells',
 'CoronaryVesselAdventitialFibroblasts',
 'DendriticCellsMature',
 'DendriticCellsType1',
 'DuctusArteriosusSmoothMuscleCells',
 'EndocardialCells',
 'EndocardialCushionCells',
 'EpicardiumDerivedCells',
 'GreatVesselAdventitialFibroblasts',
 'GreatVesselArterialEndothelialCells',
 'GreatVesselSmoothMuscleCells',
 'GreatVesselVenousEndothelialCells',
 'InnateLymphoidCells',
 'LymphNodeFibroblasticReticularCells',
 'LymphaticEndothelialCells',
 'MacrophagesATF3pos',
 'MacrophagesCX3CR1pos',
 'MacrophagesLYVE1pos',
 'MacrophagesTIMD4pos',
 'MastCells',
 'Megakaryocytes',
 'MesothelialEpicardialCells',
 'MonocyteDerivedCells',
 'Monocytes',
 'MonocytesMPOpos',
 'MyocardialInterstitialFibroblasts',
 'Myofibroblasts',
 'NaturalKillerCells',
 'NeuronPrecursors',
 'ParaympatheticNeurons',
 'PericardialCellsFibrous',
 'PericardialCellsIntermediate',
 'PericardialCellsParietal',
 'PlasmacytoidDendriticCells',
 'ProBCells',
 'SchwannCellPrecursors',
 'SchwannCells',
 'SinoatrialNodePacemakerCells',
 'SubEpicardialFibroblasts',
 'SympatheticNeurons',
 'TCellsCD4pos',
 'TCellsCD8pos',
 'TregsCD4pos',
 'ValveEndothelialCells',
 'ValveInterstitialCells',
 'VentricularCardiomyocytesCycling',
 'VentricularCardiomyocytesLeftCompact',
 'VentricularCardiomyocytesLeftTrabeculated',
 'VentricularCardiomyocytesRightCompact',
 'VentricularCardiomyocytesRightTrabeculated',
 'VentricularConductionSystemDistal',
 'VentricularConductionSystemProximal'}
```

Example 12 (python):
```python
set(adata.obs.mid_grain)
```

Example 13 (unknown):
```unknown
{'AtrialCardiomyocytes',
 'BloodVesselEndothelialCells',
 'CardiacConductionSystem',
 'EndocardialCells',
 'EpicardialCells',
 'Fibroblasts',
 'Glia',
 'LymphaticEndothelialCells',
 'LymphoidCells',
 'MuralCells',
 'MyeloidCells',
 'Neurons',
 'PericardialCells',
 'VentricularCardiomyocytes'}
```

Example 14 (python):
```python
set(adata.obs.coarse_grain)
```

Example 15 (unknown):
```unknown
{'Cardiomyocytes',
 'Endothelium',
 'Epicardium',
 'Leukocytes',
 'Mesenchymal',
 'Neural'}
```

Example 16 (python):
```python
#adata = adata[adata.obs['fine_grain'].isin(['CoronaryVesselAdventitialFibroblasts','MyocardialInterstitialFibroblasts1','MyocardialInterstitialFibroblasts2','Myofibroblasts','GreatVesselAdventitialFibroblasts'])].copy()
#adata = adata[adata.obs['fine_grain'].isin(['CoronaryArterialEndothelialCells','CoronaryCapillaryEndothelialCells','CoronaryVenousEndothelialCells','GreatVesselArterialEndothelialCells','GreatVesselVenousEndothelialCells'])].copy()
#adata = adata[adata.obs['fine_grain'].isin(['CoronaryPericytes','CoronarySmoothMuscleCells','DuctusArteriosusSmoothMuscleCells','GreatVesselSmoothMuscleCells'])].copy()
adata = adata[adata.obs['mid_grain'].isin(['PericardialCells'])].copy()
#adata=adata[adata.obs.mid_grain=='MuralCells',:].copy()
```

Example 17 (unknown):
```unknown
AnnData object with n_obs × n_vars = 16811 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score'
    var: 'gene_name_scRNA-0', 'gene_id'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'age_colors', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'cycling_colors', 'donor_colors', 'fine_grain_colors', 'heart_or_greatvessels_colors', 'hvg', 'kit_10x_colors', 'mid_grain_colors', 'modality_colors', 'neighbors', 'phase_colors', 'region_colors', 'sex_colors', 'trimester_colors', 'umap', 'week_colors'
    obsm: 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 18 (python):
```python
# preserve raw count
adata.layers["counts"] = adata.X.copy()

# normalise and log-transform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key='batch_key')
#sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5)

print(sum(adata.var.highly_variable))
sc.pl.highly_variable_genes(adata)

# freeze the state in `.raw`
adata.raw = adata

# subset hvgs
adata = adata[:, adata.var.highly_variable]

adata=adata.copy()

sc.tl.score_genes_cell_cycle(adata, s_genes=s_genes, g2m_genes=g2m_genes)
#sc.pp.regress_out(adata, ['S_score','G2M_score'])

#batch correction
scvi.model.SCVI.setup_anndata(adata, 
                              layer="counts", 
                              batch_key="batch_key",
                              categorical_covariate_keys=['phase'],
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo'])

vae = scvi.model.SCVI(adata, n_hidden = 128, n_latent = 50, n_layers = 3, dispersion = 'gene-batch')
vae.train(max_epochs=50)
    
# Convergence plot
x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
plt.plot(x, vae.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
plt.clf()

# add the latent space to adata
adata.obsm["X_scVI"] = vae.get_latent_representation()

sc.tl.score_genes(adata, stress_genes, score_name='stress_score')
sc.tl.score_genes(adata, hb_genes1, score_name='hb1_score')

sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata,min_dist=0.1)
#sc.tl.tsne(adata)
sc.pl.umap(adata, color=['log1p_total_counts','n_genes','n_genes_by_counts','total_counts','pct_counts_mt','pct_counts_ribo'],ncols=5)
sc.pl.umap(adata, color=['scrublet_score','stress_score', 'hb1_score','phase'], ncols=5,)

sc.pl.umap(adata, color=['donor','age','week','trimester','region','phase','cycling','cell_or_nuclei','modality','kit_10x','FACSgate','heart_or_greatvessels','coarse_grain','mid_grain','fine_grain'],wspace=0.8,ncols=5)
Genes=['TTN','TNNT2','MYOM1','LDB3','ALDH1A2','TNNT1','KRT19','UPK3B','BNC1','COL6A3','FBLN1','PCOLCE','LRRC17','PCDH18','EMCN','ECSCR','PECAM1','CDH5','ESAM', 'LAPTM5','PTPRC','LCP1','CD53','WAS','PHOX2B','NRXN1','STMN2','GRIK2','L1CAM']
sc.pl.umap(adata, color=Genes)
```

Example 19 (python):
```python
extracting highly variable genes
    finished (0:00:01)
--> added
    'highly_variable', boolean vector (adata.var)
    'means', float vector (adata.var)
    'dispersions', float vector (adata.var)
    'dispersions_norm', float vector (adata.var)
3620
```

Example 20 (python):
```python
calculating cell cycle phase
computing score 'S_score'
WARNING: genes are not in var_names and ignored: ['MCM5', 'FEN1', 'MCM2', 'MCM4', 'RRM1', 'UNG', 'GINS2', 'MCM6', 'CDCA7', 'PRIM1', 'UHRF1', 'RFC2', 'RPA2', 'NASP', 'RAD51AP1', 'GMNN', 'WDR76', 'SLBP', 'CCNE2', 'UBR7', 'POLD3', 'MSH2', 'RAD51', 'CDC45', 'CDC6', 'DSCC1', 'CASP8AP2', 'USP1', 'CLSPN', 'CHAF1B', 'E2F8']
    finished: added
    'S_score', score of gene set (adata.obs).
    418 total control genes are used. (0:00:00)
computing score 'G2M_score'
WARNING: genes are not in var_names and ignored: ['CKS1B', 'TMPO', 'ANP32E', 'TUBB4B', 'HJURP', 'CDC25C', 'KIF2C', 'RANGAP1', 'NCAPD2', 'PSRC1', 'LBR', 'CKAP5', 'CTCF', 'NEK2', 'CBX5', 'CENPA']
    finished: added
    'G2M_score', score of gene set (adata.obs).
    492 total control genes are used. (0:00:00)
-->     'phase', cell cycle phase (adata.obs)
```

Example 21 (unknown):
```unknown
GPU available: False, used: False
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
```

Example 22 (unknown):
```unknown
Epoch 50/50: 100%|██████| 50/50 [02:04<00:00,  2.48s/it, loss=1.06e+03, v_num=1]
```

Example 23 (python):
```python
computing score 'stress_score'
WARNING: genes are not in var_names and ignored: ['FOS', 'FOSB', 'JUN', 'JUNB', 'JUND', 'ZFP36', 'CEBPB', 'EGR1', 'HSPA1A', 'HSPA1B', 'HSP90AA1', 'HSP90AB1', 'HSPA8', 'HSPB1', 'HSPE1', 'BTG1', 'DUSP1']
    finished: added
    'stress_score', score of gene set (adata.obs).
    197 total control genes are used. (0:00:00)
computing score 'hb1_score'
WARNING: genes are not in var_names and ignored: ['HBD', 'HBM', 'HBQ1']
    finished: added
    'hb1_score', score of gene set (adata.obs).
    298 total control genes are used. (0:00:00)
computing neighbors
    finished: added to `.uns['neighbors']`
    `.obsp['distances']`, distances for each pair of neighbors
    `.obsp['connectivities']`, weighted adjacency matrix (0:00:03)
computing UMAP
    finished: added
    'X_umap', UMAP coordinates (adata.obsm) (0:00:04)
```

Example 24 (unknown):
```unknown
... storing 'phase' as categorical
```

Example 25 (unknown):
```unknown
<Figure size 320x320 with 0 Axes>
```

Example 26 (python):
```python
adata.layers["counts"] = adata.X.copy()

# normalise and log-transform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)
```

Example 27 (unknown):
```unknown
normalizing counts per cell
    finished (0:00:01)
```

Example 28 (python):
```python
sc.tl.rank_genes_groups(adata, 'mid_grain', method='t-test', key_added = "t-test")
```

Example 29 (python):
```python
ranking genes
    finished: added to `.uns['t-test']`
    'names', sorted np.recarray to be indexed by group ids
    'scores', sorted np.recarray to be indexed by group ids
    'logfoldchanges', sorted np.recarray to be indexed by group ids
    'pvals', sorted np.recarray to be indexed by group ids
    'pvals_adj', sorted np.recarray to be indexed by group ids (0:00:36)
```

Example 30 (python):
```python
set(adata.obs.coarse_grain)
```

Example 31 (unknown):
```unknown
{'Cardiomyocytes',
 'Endothelium',
 'Epicardium',
 'Leukocytes',
 'Mesenchymal',
 'Neural'}
```

Example 32 (python):
```python
set(adata.obs.fine_grain)
```

Example 33 (unknown):
```unknown
{'AtrialCardiomyocytesCycling',
 'AtrialCardiomyocytesLeft',
 'AtrialCardiomyocytesRight',
 'AtrioventricularNodePacemakerCells',
 'BCells',
 'BCellsMS4A1pos',
 'ChromaffinCells',
 'CoronaryArterialEndothelialCells',
 'CoronaryCapillaryEndothelialCells',
 'CoronaryPericytes',
 'CoronarySmoothMuscleCells',
 'CoronaryVenousEndothelialCells',
 'CoronaryVesselAdventitialFibroblasts',
 'DendriticCellsMature',
 'DendriticCellsType1',
 'DuctusArteriosusSmoothMuscleCells',
 'EndocardialCells',
 'EndocardialCushionCells',
 'EpicardiumDerivedCells',
 'GreatVesselAdventitialFibroblasts',
 'GreatVesselArterialEndothelialCells',
 'GreatVesselSmoothMuscleCells',
 'GreatVesselVenousEndothelialCells',
 'InnateLymphoidCells',
 'LymphNodeFibroblasticReticularCells',
 'LymphaticEndothelialCells',
 'MacrophagesATF3pos',
 'MacrophagesCX3CR1pos',
 'MacrophagesLYVE1pos',
 'MacrophagesTIMD4pos',
 'MastCells',
 'Megakaryocytes',
 'MesothelialEpicardialCells',
 'MonocyteDerivedCells',
 'Monocytes',
 'MonocytesMPOpos',
 'MyocardialInterstitialFibroblasts',
 'Myofibroblasts',
 'NaturalKillerCells',
 'NeuronPrecursors',
 'ParaympatheticNeurons',
 'PericardialCellsFibrous',
 'PericardialCellsIntermediate',
 'PericardialCellsParietal',
 'PlasmacytoidDendriticCells',
 'ProBCells',
 'SchwannCellPrecursors',
 'SchwannCells',
 'SinoatrialNodePacemakerCells',
 'SubEpicardialFibroblasts',
 'SympatheticNeurons',
 'TCellsCD4pos',
 'TCellsCD8pos',
 'TregsCD4pos',
 'ValveEndothelialCells',
 'ValveInterstitialCells',
 'VentricularCardiomyocytesCycling',
 'VentricularCardiomyocytesLeftCompact',
 'VentricularCardiomyocytesLeftTrabeculated',
 'VentricularCardiomyocytesRightCompact',
 'VentricularCardiomyocytesRightTrabeculated',
 'VentricularConductionSystemDistal',
 'VentricularConductionSystemProximal'}
```

Example 34 (python):
```python
ClusterOfInterest='BloodVesselEndothelialCells'
FineGrainCategoriesOrder=['AtrialCardiomyocytesLeft','AtrialCardiomyocytesRight','AtrialCardiomyocytesCycling','VentricularCardiomyocytesLeftCompact','VentricularCardiomyocytesRightCompact','VentricularCardiomyocytesLeftTrabeculated','VentricularCardiomyocytesRightTrabeculated','VentricularCardiomyocytesCycling','SinoatrialNodePacemakerCells','AtrioventricularNodePacemakerCells','VentricularConductionSystemProximal','VentricularConductionSystemDistal','GreatVesselAdventitialFibroblasts','CoronaryVesselAdventitialFibroblasts','SubEpicardialFibroblasts','MyocardialInterstitialFibroblasts','Myofibroblasts','LymphNodeFibroblasticReticularCells','ValveInterstitialCells','GreatVesselSmoothMuscleCells','CoronarySmoothMuscleCells','DuctusArteriosusSmoothMuscleCells','CoronaryPericytes','PericardialCellsIntermediate','PericardialCellsFibrous','PericardialCellsParietal','GreatVesselArterialEndothelialCells','GreatVesselVenousEndothelialCells','CoronaryArterialEndothelialCells','CoronaryVenousEndothelialCells','CoronaryCapillaryEndothelialCells','EndocardialCells','EndocardialCushionCells','ValveEndothelialCells','LymphaticEndothelialCells','MesothelialEpicardialCells','EpicardiumDerivedCells','NeuronPrecursors','ChromaffinCells','SympatheticNeurons','ParaympatheticNeurons','SchwannCellPrecursors','SchwannCells','MonocytesMPOpos','Monocytes','MonocyteDerivedCells','MacrophagesCX3CR1pos','MacrophagesTIMD4pos','MacrophagesLYVE1pos','MacrophagesATF3pos','DendriticCellsType1','DendriticCellsMature','PlasmacytoidDendriticCells','MastCells','Megakaryocytes','TCellsCD4pos','TCellsCD8pos','TregsCD4pos','ProBCells','BCells','BCellsMS4A1pos','NaturalKillerCells','InnateLymphoidCells']
DEGenesForCluster = [x[ClusterOfInterest] for x in adata.uns['t-test']['names'][:50]]
sc.pl.dotplot(adata, DEGenesForCluster ,groupby='fine_grain',categories_order=FineGrainCategoriesOrder,standard_scale='var',cmap='YlGn')
```

Example 35 (python):
```python
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
/tmp/ipykernel_248106/4017030311.py in <module>
      1 ClusterOfInterest='BloodVesselEndothelialCells'
      2 FineGrainCategoriesOrder=['AtrialCardiomyocytesLeft','AtrialCardiomyocytesRight','AtrialCardiomyocytesCycling','VentricularCardiomyocytesLeftCompact','VentricularCardiomyocytesRightCompact','VentricularCardiomyocytesLeftTrabeculated','VentricularCardiomyocytesRightTrabeculated','VentricularCardiomyocytesCycling','SinoatrialNodePacemakerCells','AtrioventricularNodePacemakerCells','VentricularConductionSystemProximal','VentricularConductionSystemDistal','GreatVesselAdventitialFibroblasts','CoronaryVesselAdventitialFibroblasts','SubEpicardialFibroblasts','MyocardialInterstitialFibroblasts','Myofibroblasts','LymphNodeFibroblasticReticularCells','ValveInterstitialCells','GreatVesselSmoothMuscleCells','CoronarySmoothMuscleCells','DuctusArteriosusSmoothMuscleCells','CoronaryPericytes','PericardialCellsIntermediate','PericardialCellsFibrous','PericardialCellsParietal','GreatVesselArterialEndothelialCells','GreatVesselVenousEndothelialCells','CoronaryArterialEndothelialCells','CoronaryVenousEndothelialCells','CoronaryCapillaryEndothelialCells','EndocardialCells','EndocardialCushionCells','ValveEndothelialCells','LymphaticEndothelialCells','MesothelialEpicardialCells','EpicardiumDerivedCells','NeuronPrecursors','ChromaffinCells','SympatheticNeurons','ParaympatheticNeurons','SchwannCellPrecursors','SchwannCells','MonocytesMPOpos','Monocytes','MonocyteDerivedCells','MacrophagesCX3CR1pos','MacrophagesTIMD4pos','MacrophagesLYVE1pos','MacrophagesATF3pos','DendriticCellsType1','DendriticCellsMature','PlasmacytoidDendriticCells','MastCells','Megakaryocytes','TCellsCD4pos','TCellsCD8pos','TregsCD4pos','ProBCells','BCells','BCellsMS4A1pos','NaturalKillerCells','InnateLymphoidCells']
----> 3 DEGenesForCluster = [x[ClusterOfInterest] for x in adata.uns['t-test']['names'][:50]]
      4 sc.pl.dotplot(adata, DEGenesForCluster ,groupby='fine_grain',categories_order=FineGrainCategoriesOrder,standard_scale='var',cmap='YlGn')

~/miniconda3/envs/scanpynew/lib/python3.8/site-packages/anndata/compat/_overloaded_dict.py in __getitem__(self, key)
     98             return self.overloaded[key].get()
     99         else:
--> 100             return self.data[key]
    101 
    102     def __setitem__(self, key, value):

KeyError: 't-test'
```

Example 36 (python):
```python
sc.pl.dotplot(adata, ['TNNT2','ACTN2','LDB3','COL6A3','PCOLCE','LRRC17','EGFL7','PECAM1','CALCRL','ALDH1A2','BNC1','UPK3B','PHOX2B','NRXN1','SORCS1','PTPRC','RUNX1','DOCK2'] ,var_group_positions=[(0,2),(3,5),(6,8),(9,11),(12,14),(15,17)], var_group_labels=['Cardiomyocytes','Mesenchymal','Endothelium','Epicardium','Neural','Leukocytes'], var_group_rotation=45,groupby='fine_grain',categories_order=FineGrainCategoriesOrder,standard_scale='var',cmap='YlGn')
```

Example 37 (python):
```python
sc.pl.dotplot(adata, ['BNC1','WT1','TCF21'],groupby='fine_grain',categories_order=FineGrainCategoriesOrder,standard_scale='var',cmap='YlGn')
```

Example 38 (python):
```python
ax = sc.pl.correlation_matrix(adata, 'fine_grain', figsize=(25,20),cmap='PRGn')
```

Example 39 (python):
```python
#Infer the sex of donors; run for multiome and scRNA separately

chrY_genes=['SRY', 'RPS4Y1', 'ZFY', 'ZFY-AS1', 'LINC00278', 'PCDH11Y', 'TTTY2B', 'TBL1Y', 'PRKY', 'USP9Y', 'DDX3Y', 'UTY', 'TMSB4Y', 'NLGN4Y', 'TTTY14', 'KDM5D', 'TTTY10', 'EIF1AY', 'PRORY', 'LINC00266-4P']

# scRNA 
bdata=sc.read('./Objects/Feb28ObjectRaw.h5ad')
adata=bdata[bdata.obs.modality=='scRNA',:].copy()

adata.obs['percent_chrY'] = np.sum(adata[:, chrY_genes].X, axis=1).A1 / np.sum(adata.X, axis=1).A1 * 100
adata.obs["XIST-counts"] = adata.X[:,adata.var_names.str.match('XIST')].toarray()#sns.jointplot('total_counts', 'pct_counts_mt', data=adata.obs, kind="scatter")
sc.pl.scatter(adata, x='XIST-counts', y='percent_chrY', color="donor")
sc.pl.violin(adata, ["XIST-counts", "percent_chrY"], jitter=0., groupby = 'donor', rotation= 90,figsize=(25,20))

#Multiome
adata=bdata[bdata.obs.modality=='scMultiome',:].copy()

adata.obs['percent_chrY'] = np.sum(adata[:, chrY_genes].X, axis=1).A1 / np.sum(adata.X, axis=1).A1 * 100
adata.obs["XIST-counts"] = adata.X[:,adata.var_names.str.match('XIST')].toarray()#sns.jointplot('total_counts', 'pct_counts_mt', data=adata.obs, kind="scatter")
sc.pl.scatter(adata, x='XIST-counts', y='percent_chrY', color="donor")
sc.pl.violin(adata, ["XIST-counts", "percent_chrY"], jitter=0., groupby = 'donor', rotation= 90,figsize=(25,20))
```

Example 40 (unknown):
```unknown
bdata.obs.modality
```

Example 41 (unknown):
```unknown
barcode
BHF_F_Hea10402917_TGGCTGGGTTGTGGAG-1                             scRNA
BHF_F_Hea10402917_ATGTGTGTCGAACGGA-1                             scRNA
BHF_F_Hea10402917_CAGGTGCAGGCTATCT-1                             scRNA
BHF_F_Hea10402917_TACAGTGTCCAGTATG-1                             scRNA
BHF_F_Hea10402917_ACCAGTATCCACTCCA-1                             scRNA
                                                               ...    
HCAHeartST13386010_HCAHeartST13303420_ATGGCTTAGTGAGGTA-1    scMultiome
HCAHeartST13386010_HCAHeartST13303420_GAACCAGCAGCATGAG-1    scMultiome
HCAHeartST13386010_HCAHeartST13303420_GCGCAATGTAAACAAG-1    scMultiome
HCAHeartST13386010_HCAHeartST13303420_CGTAATGGTATTGAGT-1    scMultiome
HCAHeartST13386010_HCAHeartST13303420_GCTGTAAGTCAATACG-1    scMultiome
Name: modality, Length: 297473, dtype: category
Categories (2, object): ['scMultiome', 'scRNA']
```

Example 42 (python):
```python
gene = 'GJA1'
key_1 = 'week'
key_2 = 'fine_grain'

# subset anndata of a gene of interest
ad = adata[:,list(adata.var_names).index(gene)]
# add gene expression to .obs
ad.obs[gene] = ad.to_df()
# groupby two keys
# this is mean expression
df = ad.obs[[key_1,key_2,gene]]
df = df.groupby([key_1,key_2]).mean().unstack(level=0)
df.columns = df.columns.get_level_values(1)
# plot
plt.rcParams['figure.figsize'] = 4.5,18  # width, height
sns.heatmap(df,cmap='viridis')
plt.title(gene)
```

Example 43 (unknown):
```unknown
Trying to set attribute `.obs` of view, copying.
```

Example 44 (unknown):
```unknown
Text(0.5, 1.0, 'GJA1')
```

Example 45 (python):
```python
gene = 'NPPA'
key_1 = 'week'
key_2 = 'fine_grain'

# subset anndata of a gene of interest
ad = adata[:,list(adata.var_names).index(gene)]
# add gene expression to .obs
ad.obs[gene] = ad.to_df()
# groupby two keys
# this is mean expression
df = ad.obs[[key_1,key_2,gene]]
df = df.groupby([key_1,key_2]).mean().unstack(level=0)
df.columns = df.columns.get_level_values(1)
# plot
plt.rcParams['figure.figsize'] = 4.5,18  # width, height
sns.heatmap(df,cmap='viridis')
plt.title(gene)
```

Example 46 (unknown):
```unknown
Trying to set attribute `.obs` of view, copying.
```

Example 47 (unknown):
```unknown
Text(0.5, 1.0, 'NPPA')
```

---

## Read in adata¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig5_Trisomy21/no09_pseudobulk_finegrain.html

**Contents:**
- Read in adata¶
- Annotate nhoods¶
- Pseudobulk: nhood-by-gene adata¶

**Examples:**

Example 1 (python):
```python
import scanpy as sc
import pandas as pd
import numpy as np
import anndata
import scipy
import matplotlib.pyplot as plt
import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 

import seaborn as sns

import warnings
warnings.simplefilter("ignore", UserWarning)
```

Example 2 (python):
```python
# Configure rpy2 to Use the Correct R Installation:
import os
os.environ['R_HOME'] = '/software/cellgen/team205/kk18/envs/generic_env/lib/R'
# check
from rpy2.rinterface_lib import openrlib
print(openrlib.R_HOME)
```

Example 3 (unknown):
```unknown
/software/cellgen/team205/kk18/envs/generic_env/lib/R
```

Example 4 (python):
```python
import milopy.core as milo
import milopy.plot as milopl
import milopy.utils
```

Example 5 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 6 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 7 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk/Aug2024'
table_dir = '/nfs/team205/heart/fetal_ms_figures/tables/Aug2024'
```

Example 8 (python):
```python
# atlas
adata = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/trisomy21/Euploid_T21Hearts_Aug2024_sel_subsampled-per-cellstate-donor_cells-in-sig-nhoods-sfdr0p1.h5ad')
adata
```

Example 9 (python):
```python
AnnData object with n_obs × n_vars = 14846 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score', 'euploid_or_t21', 'donor_type', 'region_finest', 'flushed', 'scrublet_leiden', 'cluster_scrublet_score', 'leiden_scVI_0.1', 'leiden_scVI_0.2', 'leiden_scVI_0.3', 'leiden_scVI_0.5', 'leiden_scVI_0.7', 'leiden_scVI_1', 'celltypist_coarse', 'conf_score_coarse', 'cells_to_remove', 'celltypist_fine', 'conf_score_fine', 'coarse_grain_merged', 'fine_grain_merged', 'nhood_ixs_random', 'nhood_ixs_refined', 'nhood_kth_distance'
    var: 'gene_id', 'gene_name_scRNA-0-0', 'feature_type-1', 'mt-1', 'ribo-1', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1', 'gene_name_GenomicFeatures', 'gene_biotype_GenomicFeatures', 'gene_seq_start_GenomicFeatures', 'gene_seq_end_GenomicFeatures', 'seq_name_GenomicFeatures', 'seq_strand_GenomicFeatures', 'seq_coord_system_GenomicFeatures', 'description_GenomicFeatures', 'gene_id_version_GenomicFeatures', 'canonical_transcript_GenomicFeatures', 'gene_name'
    uns: 'coarse_grain_merged_colors', 'correcting-donor-region_n-layers-2', 'donor_colors', 'euploid_or_t21_colors', 'fine_grain_merged_colors', 'mid_grain_colors', 'nhood_adata', 'nhood_neighbors_key', 'region_colors', 'scVI_latent_correcting-donor-region_n-layers-2-forMilo', 'umap'
    obsm: 'X_umap_correcting-donor-region_n-layers-2', 'latent_gene_encoding', 'nhoods', 'nhoods_reassigned', 'scVI_latent_correcting-donor-region_n-layers-2'
    obsp: 'correcting-donor-region_n-layers-2_connectivities', 'correcting-donor-region_n-layers-2_distances', 'scVI_latent_correcting-donor-region_n-layers-2-forMilo_connectivities', 'scVI_latent_correcting-donor-region_n-layers-2-forMilo_distances'
```

Example 10 (python):
```python
milopy.utils.annotate_nhoods(adata, anno_col='fine_grain_merged')
```

Example 11 (python):
```python
plt.hist(adata.uns['nhood_adata'].obs["nhood_annotation_frac"],bins=100);
plt.xlabel("celltype fraction")
```

Example 12 (unknown):
```unknown
Text(0.5, 0, 'celltype fraction')
```

Example 13 (python):
```python
# We can see that for the majority of neighbourhoods, almost all cells have the same neighbourhood. We can rename neighbourhoods where less than 60% of the cells have the top label as "Mixed"
adata.uns['nhood_adata'].obs["nhood_annotation"] = adata.uns['nhood_adata'].obs["nhood_annotation"].astype('str')
adata.uns['nhood_adata'].obs.loc[adata.uns['nhood_adata'].obs["nhood_annotation_frac"] < 0.6, "nhood_annotation"] = "Mixed"
```

Example 14 (python):
```python
adata.uns['nhood_adata'].obs['deg_group'].value_counts()
```

Example 15 (unknown):
```unknown
deg_group
trisomy21    339
euploid      238
Name: count, dtype: int64
```

Example 16 (python):
```python
%%time
# first take sum
# with raw gene counts
# np.dot of gene-by-cell and cell-by-nhood (boolean) will make gene-by-nhood with summed counts
X = adata.X.todense().T.copy() # gene-by-cell
Y = adata.obsm['nhoods_reassigned'].todense().copy()
Z = np.matmul(X,Y) # gene-by-nhood

# prepare anndata
adata_nhood = anndata.AnnData(
    X=np.array(Z).T,
    obs=adata.uns['nhood_adata'].obs[['index_cell','logFC','PValue','FDR','SpatialFDR','nhood_annotation','deg_group']].copy(),
    var=adata.var.copy()
)
adata_nhood
```

Example 17 (unknown):
```unknown
CPU times: user 18.1 s, sys: 3.45 s, total: 21.6 s
Wall time: 22.6 s
```

Example 18 (unknown):
```unknown
AnnData object with n_obs × n_vars = 577 × 36601
    obs: 'index_cell', 'logFC', 'PValue', 'FDR', 'SpatialFDR', 'nhood_annotation', 'deg_group'
    var: 'gene_id', 'gene_name_scRNA-0-0', 'feature_type-1', 'mt-1', 'ribo-1', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1', 'gene_name_GenomicFeatures', 'gene_biotype_GenomicFeatures', 'gene_seq_start_GenomicFeatures', 'gene_seq_end_GenomicFeatures', 'seq_name_GenomicFeatures', 'seq_strand_GenomicFeatures', 'seq_coord_system_GenomicFeatures', 'description_GenomicFeatures', 'gene_id_version_GenomicFeatures', 'canonical_transcript_GenomicFeatures', 'gene_name'
```

Example 19 (python):
```python
# save
adata_nhood.write('/nfs/team205/heart/anndata_objects/Foetal/trisomy21/Euploid_T21Hearts_Aug2024_sel_subsampled-per-cellstate-donor_cells-in-sig-nhoods-sfdr0p1_pseudobulked-nhood-reassigned-sum_finegrain.h5ad')
```

---

## Import modules¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/RNA/scnRNA_Integration-Annotation/No2_foetal-global_scVI.html

**Contents:**
- Import modules¶
- Read in pre-processed anndata¶
- Setup anndata for scVI¶
- Creating and training a model¶
- Save post-scVI anndata¶

**Examples:**

Example 1 (python):
```python
import anndata
import matplotlib.pyplot as plt
import seaborn as sns

import numpy as np
import numpy.random as random
import pandas as pd
import scanpy as sc
import scvi
```

Example 2 (unknown):
```unknown
Global seed set to 0
```

Example 3 (python):
```python
import session_info
session_info.show()
```

Example 4 (python):
```python
-----
anndata             0.7.8
matplotlib          3.5.0
numpy               1.20.3
pandas              1.3.4
scanpy              1.8.2
scvi                0.14.5
seaborn             0.11.2
session_info        1.0.0
-----
```

Example 5 (python):
```python
PIL                         8.4.0
absl                        NA
attr                        21.2.0
backcall                    0.2.0
beta_ufunc                  NA
binom_ufunc                 NA
cached_property             1.5.2
cffi                        1.15.0
colorama                    0.4.4
cycler                      0.10.0
cython_runtime              NA
dateutil                    2.8.2
debugpy                     1.5.1
decorator                   5.1.0
defusedxml                  0.7.1
deprecate                   0.3.0
docrep                      0.3.2
entrypoints                 0.3
fsspec                      2021.11.0
google                      NA
h5py                        3.6.0
igraph                      0.9.8
importlib_metadata          1.7.0
ipykernel                   6.5.1
ipython_genutils            0.2.0
ipywidgets                  7.6.5
jedi                        0.18.1
joblib                      1.1.0
kiwisolver                  1.3.2
leidenalg                   0.8.8
llvmlite                    0.37.0
matplotlib_inline           NA
mpl_toolkits                NA
natsort                     8.0.0
nbinom_ufunc                NA
numba                       0.54.1
numexpr                     2.7.3
opt_einsum                  v3.3.0
packaging                   21.3
parso                       0.8.2
pexpect                     4.8.0
pickleshare                 0.7.5
pkg_resources               NA
prompt_toolkit              3.0.22
ptyprocess                  0.7.0
pycparser                   2.21
pydev_ipython               NA
pydevconsole                NA
pydevd                      2.6.0
pydevd_concurrency_analyser NA
pydevd_file_utils           NA
pydevd_plugins              NA
pydevd_tracing              NA
pygments                    2.10.0
pyparsing                   3.0.6
pyro                        1.7.0
pytorch_lightning           1.3.8
pytz                        2021.3
rich                        NA
scipy                       1.7.3
setuptools                  58.0.4
setuptools_scm              NA
sinfo                       0.3.4
six                         1.16.0
sklearn                     1.0.1
statsmodels                 0.13.1
storemagic                  NA
tables                      3.6.1
tensorboard                 2.7.0
texttable                   1.6.4
threadpoolctl               3.0.0
torch                       1.10.0+cu102
torchmetrics                0.6.0
tornado                     6.1
tqdm                        4.62.3
traitlets                   5.1.1
typing_extensions           NA
wcwidth                     0.2.5
yaml                        5.4.1
zipp                        NA
zmq                         22.3.0
```

Example 6 (unknown):
```unknown
-----
IPython             7.29.0
jupyter_client      7.1.0
jupyter_core        4.9.1
notebook            6.4.6
-----
Python 3.7.11 (default, Jul 27 2021, 14:32:16) [GCC 7.5.0]
Linux-4.15.0-176-generic-x86_64-with-debian-buster-sid
-----
Session information updated at 2023-02-09 00:59
```

Example 7 (python):
```python
sc.settings.verbosity = 3             # verbosity: errors (0), warnings (1), info (2), hints (3)
sc.settings.set_figure_params(dpi=80, facecolor='white', color_map = 'RdPu',)
```

Example 8 (python):
```python
adata = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/scVI/RNA_foetal_pp.h5ad')
adata
```

Example 9 (unknown):
```unknown
AnnData object with n_obs × n_vars = 372801 × 3343
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key'
    var: 'gene_name_scRNA-0', 'gene_name_multiome-1', 'n_cells', 'highly_variable', 'means', 'dispersions', 'dispersions_norm', 'highly_variable_nbatches', 'highly_variable_intersection'
    uns: 'hvg'
    obsm: 'latent_gene_encoding'
    layers: 'counts'
```

Example 10 (python):
```python
print(adata)
print(adata.obs['batch_key'].value_counts())
print(adata.obs['cell_or_nuclei'].value_counts())
```

Example 11 (unknown):
```unknown
AnnData object with n_obs × n_vars = 372801 × 3343
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key'
    var: 'gene_name_scRNA-0', 'gene_name_multiome-1', 'n_cells', 'highly_variable', 'means', 'dispersions', 'dispersions_norm', 'highly_variable_nbatches', 'highly_variable_intersection'
    uns: 'hvg'
    obsm: 'latent_gene_encoding'
    layers: 'counts'
Hst41_Nuclei_Multiome-v1    129909
Hst33_Nuclei_Multiome-v1     62684
Hst39_Nuclei_Multiome-v1     30591
C99_Cell_5prime-v2           17921
C97_Cell_5prime-v2           17388
C92_Nuclei_Multiome-v1       15379
C94_Cell_5prime-v2           13971
Hst40_Nuclei_Multiome-v1     11559
BRC2251_Cell_3prime-v3        9651
BRC2256_Cell_3prime-v3        8525
BRC2260_Cell_3prime-v3        8408
Hst36_Nuclei_Multiome-v1      7848
BRC2252_Cell_3prime-v3        7366
C86_Cell_5prime-v2            6567
BRC2263_Cell_3prime-v3        5917
BRC2262_Cell_3prime-v3        4749
C87_Nuclei_Multiome-v1        4499
C104_Nuclei_Multiome-v1       3411
C98_Nuclei_Multiome-v1        3039
C85_Nuclei_Multiome-v1        2808
C82_Nuclei_Multiome-v1         611
Name: batch_key, dtype: int64
Nuclei    272338
Cell      100463
Name: cell_or_nuclei, dtype: int64
```

Example 12 (python):
```python
adata.layers['counts'].data[:10]
```

Example 13 (unknown):
```unknown
array([ 5.,  4., 20.,  1.,  1.,  4.,  4.,  3.,  1.,  2.], dtype=float32)
```

Example 14 (python):
```python
scvi.model.SCVI.setup_anndata(adata, 
                              layer="counts", 
                              batch_key="batch_key",
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo'])
```

Example 15 (python):
```python
INFO     Using batches from adata.obs["batch_key"]                                           
INFO     No label_key inputted, assuming all cells have same label                           
INFO     Using data from adata.layers["counts"]                                              
INFO     Successfully registered anndata object containing 372801 cells, 3343 vars, 21       
         batches, 1 labels, and 0 proteins. Also registered 0 extra categorical covariates   
         and 3 extra continuous covariates.                                                  
INFO     Please do not further modify adata until model is trained.
```

Example 16 (python):
```python
scvi.data.view_anndata_setup(adata)
```

Example 17 (unknown):
```unknown
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/scvi_env/lib/python3.7/site-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function view_anndata_setup is deprecated; This method will be removed in 0.15.0. Please avoid building any new dependencies on it.
  warnings.warn(msg, category=FutureWarning)
```

Example 18 (unknown):
```unknown
Anndata setup with scvi-tools version 0.14.5.
```

Example 19 (unknown):
```unknown
Data Summary               
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃             Data             ┃ Count  ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│            Cells             │ 372801 │
│             Vars             │  3343  │
│            Labels            │   1    │
│           Batches            │   21   │
│           Proteins           │   0    │
│ Extra Categorical Covariates │   0    │
│ Extra Continuous Covariates  │   3    │
└──────────────────────────────┴────────┘
```

Example 20 (python):
```python
SCVI Data Registry                   
┏━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃     Data      ┃         scvi-tools Location          ┃
┡━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│       X       │        adata.layers['counts']        │
│ batch_indices │       adata.obs['_scvi_batch']       │
│    labels     │      adata.obs['_scvi_labels']       │
│   cont_covs   │ adata.obsm['_scvi_extra_continuous'] │
└───────────────┴──────────────────────────────────────┘
```

Example 21 (python):
```python
Label Categories                        
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 22 (python):
```python
Batch Categories                              
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location     ┃        Categories        ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['batch_key'] │  BRC2251_Cell_3prime-v3  │          0          │
│                        │  BRC2252_Cell_3prime-v3  │          1          │
│                        │  BRC2256_Cell_3prime-v3  │          2          │
│                        │  BRC2260_Cell_3prime-v3  │          3          │
│                        │  BRC2262_Cell_3prime-v3  │          4          │
│                        │  BRC2263_Cell_3prime-v3  │          5          │
│                        │  C82_Nuclei_Multiome-v1  │          6          │
│                        │  C85_Nuclei_Multiome-v1  │          7          │
│                        │    C86_Cell_5prime-v2    │          8          │
│                        │  C87_Nuclei_Multiome-v1  │          9          │
│                        │  C92_Nuclei_Multiome-v1  │         10          │
│                        │    C94_Cell_5prime-v2    │         11          │
│                        │    C97_Cell_5prime-v2    │         12          │
│                        │  C98_Nuclei_Multiome-v1  │         13          │
│                        │    C99_Cell_5prime-v2    │         14          │
│                        │ C104_Nuclei_Multiome-v1  │         15          │
│                        │ Hst33_Nuclei_Multiome-v1 │         16          │
│                        │ Hst36_Nuclei_Multiome-v1 │         17          │
│                        │ Hst39_Nuclei_Multiome-v1 │         18          │
│                        │ Hst40_Nuclei_Multiome-v1 │         19          │
│                        │ Hst41_Nuclei_Multiome-v1 │         20          │
└────────────────────────┴──────────────────────────┴─────────────────────┘
```

Example 23 (python):
```python
Extra Continuous Variables                  
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃       Source Location        ┃           Range            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│  adata.obs['total_counts']   │       1000 -> 727776       │
│  adata.obs['pct_counts_mt']  │ 0 -> 19.991365432739257812 │
│ adata.obs['pct_counts_ribo'] │ 0 -> 19.974821090698242188 │
└──────────────────────────────┴────────────────────────────┘
```

Example 24 (python):
```python
vae = scvi.model.SCVI(adata, n_hidden = 128, n_latent = 50, n_layers = 3, dispersion = 'gene-batch')
vae
```

Example 25 (unknown):
```unknown
SCVI Model with the following params: 
n_hidden: 128, n_latent: 50, n_layers: 3, dropout_rate: 0.1, dispersion: gene-batch, 
gene_likelihood: zinb, latent_distribution: normal
Training status: Not Trained
```

Example 26 (python):
```python
To print summary of associated AnnData, use: scvi.data.view_anndata_setup(model.adata)
```

Example 27 (unknown):
```unknown
# train
vae.train()
```

Example 28 (unknown):
```unknown
GPU available: True, used: True
TPU available: False, using: 0 TPU cores
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
```

Example 29 (unknown):
```unknown
Epoch 21/21: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 21/21 [09:38<00:00, 27.55s/it, loss=954, v_num=1]
```

Example 30 (unknown):
```unknown
# save scvi model
vae.save("/nfs/team205/heart/anndata_objects/Foetal/scVI/models/RNA_foetal_global/",overwrite=True)
```

Example 31 (python):
```python
# vae = scvi.model.SCVI.load("/nfs/team205/heart/anndata_objects/Foetal/scVI/models/RNA_foetal_global/", adata)
```

Example 32 (python):
```python
# Convergency plot
vae.history['elbo_train']
x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
plt.plot(x, vae.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
```

Example 33 (python):
```python
adata.obsm["X_scVI"] = vae.get_latent_representation()
adata.write('/nfs/team205/heart/anndata_objects/Foetal/scVI/RNA_foetal_scVI.h5ad')
```

---

## Import modules¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/RNA/QC/Foetal_T21-Multiome-RNA_No2_QC-filter.html

**Contents:**
- Import modules¶
- Functions¶
- Read in anndata¶
- Remove doublets¶
- Remove low quality cells¶
- Compute QC stats¶
- Remove cells exceeding the ribo/mito threshold¶
- Calculate RBC score¶
- Remove RBCs¶
- Save post-filtered anndata¶
- Just checking¶

**Examples:**

Example 1 (python):
```python
import numpy as np
import pandas as pd
import scanpy as sc
import matplotlib.pyplot as plt
import scrublet as scr
import session_info
```

Example 2 (python):
```python
sc.settings.verbosity = 3
sc.settings.set_figure_params(dpi = 160, color_map = 'RdPu', dpi_save = 180, vector_friendly = True, format = 'svg')
session_info.show()
```

Example 3 (python):
```python
-----
matplotlib          3.3.4
numpy               1.20.1
pandas              1.1.4
scanpy              1.7.1
scrublet            NA
session_info        1.0.0
-----
```

Example 4 (unknown):
```unknown
PIL                 8.1.2
anndata             0.7.5
anyio               NA
attr                20.3.0
babel               2.9.0
backcall            0.2.0
brotli              NA
cairo               1.20.0
certifi             2020.12.05
cffi                1.14.5
chardet             4.0.0
cloudpickle         1.6.0
colorama            0.4.4
cycler              0.10.0
cython_runtime      NA
cytoolz             0.11.0
dask                2021.03.1
dateutil            2.8.1
decorator           4.4.2
fsspec              0.8.7
get_version         2.1
google              NA
h5py                2.10.0
idna                2.10
igraph              0.8.3
ipykernel           5.5.0
ipython_genutils    0.2.0
jedi                0.18.0
jinja2              2.11.3
joblib              1.0.1
json5               NA
jsonschema          3.2.0
jupyter_server      1.4.1
jupyterlab_server   2.3.0
kiwisolver          1.3.1
legacy_api_wrap     0.0.0
leidenalg           0.8.3
llvmlite            0.34.0
louvain             0.7.0
markupsafe          1.1.1
mpl_toolkits        NA
natsort             7.1.1
nbclassic           NA
nbformat            5.1.2
numba               0.51.2
numexpr             2.7.3
packaging           20.9
parso               0.8.1
pexpect             4.8.0
pickleshare         0.7.5
pkg_resources       NA
prometheus_client   NA
prompt_toolkit      3.0.16
psutil              5.8.0
ptyprocess          0.7.0
pvectorc            NA
pyarrow             0.16.0
pygments            2.8.0
pyparsing           2.4.7
pyrsistent          NA
pytoml              NA
pytz                2021.1
requests            2.25.1
ruamel              NA
scipy               1.6.1
send2trash          NA
setuptools_scm      NA
simplejson          3.17.2
sinfo               0.3.1
six                 1.15.0
sklearn             0.22
sniffio             1.2.0
socks               1.7.1
sphinxcontrib       NA
storemagic          NA
tables              3.6.1
tblib               1.7.0
texttable           1.6.3
tlz                 0.11.0
toolz               0.11.1
tornado             6.1
traitlets           5.0.5
typing_extensions   NA
uritemplate         3.0.1
urllib3             1.26.3
wcwidth             0.2.5
yaml                5.1.2
zmq                 22.0.3
```

Example 5 (unknown):
```unknown
-----
IPython             7.21.0
jupyter_client      6.1.11
jupyter_core        4.7.1
jupyterlab          3.0.9
notebook            6.2.0
-----
Python 3.8.8 | packaged by conda-forge | (default, Feb 20 2021, 16:22:27) [GCC 9.3.0]
Linux-4.15.0-112-generic-x86_64-with-glibc2.10
-----
Session information updated at 2023-03-14 17:33
```

Example 6 (python):
```python
adata = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/QC/T21_multiome_RNA_foetal_prefilter.h5ad')
adata
```

Example 7 (unknown):
```unknown
AnnData object with n_obs × n_vars = 190750 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval'
    var: 'gene_name', 'feature_type', 'ambient_expression-0', 'ambient_expression-1', 'ambient_expression-2', 'ambient_expression-3', 'ambient_expression-4', 'ambient_expression-5', 'ambient_expression-6', 'ambient_expression-7'
    obsm: 'latent_gene_encoding'
```

Example 8 (python):
```python
adata.obs['combinedID'].value_counts()
```

Example 9 (unknown):
```unknown
BHF_F_Hea13188319_BHF_F_Hea13187621    50000
BHF_F_Hea13188321_BHF_F_Hea13187623    38944
BHF_F_Hea13188325_BHF_F_Hea13187627    23862
BHF_F_Hea13188322_BHF_F_Hea13187624    17452
BHF_F_Hea13188323_BHF_F_Hea13187625    17143
BHF_F_Hea13188327_BHF_F_Hea13187629    16377
BHF_F_Hea13188320_BHF_F_Hea13187622    14000
BHF_F_Hea13188326_BHF_F_Hea13187628    12972
Name: combinedID, dtype: int64
```

Example 10 (python):
```python
adata.obs['scrublet_score'].hist(bins=100)
```

Example 11 (unknown):
```unknown
<AxesSubplot:>
```

Example 12 (python):
```python
# set threshold (arbitrary)
dbl_score_thresh=0.15

prop = np.sum(adata.obs['scrublet_score']>=dbl_score_thresh)/len(adata)
print(f'proportion of doublets: {prop}')
```

Example 13 (unknown):
```unknown
proportion of doublets: 0.2766657929226737
```

Example 14 (python):
```python
# Save doublet ids
doublets = adata.obs_names[(adata.obs['scrublet_score']<dbl_score_thresh)==False]
with open('/nfs/team205/heart/anndata_objects/Foetal/QC/T21_Multiome_rm-doublets.txt', 'w') as f:
    for line in doublets:
        f.write(line)
        f.write('\n')

# Remove doublets
adata = adata[adata.obs['scrublet_score']<dbl_score_thresh]
print(adata.shape)
```

Example 15 (unknown):
```unknown
(137976, 36601)
```

Example 16 (python):
```python
sc.pp.filter_cells(adata, min_genes=500, inplace=True)
sc.pp.filter_cells(adata, min_counts=1000, inplace=True)
adata.shape
```

Example 17 (unknown):
```unknown
filtered out 19359 cells that have less than 500 genes expressed
Trying to set attribute `.obs` of view, copying.
filtered out 7603 cells that have less than 1000 counts
```

Example 18 (unknown):
```unknown
(111014, 36601)
```

Example 19 (python):
```python
adata.var['mt'] = adata.var['gene_name'].str.startswith('MT-')  # annotate the group of mitochondrial genes as 'mt'
adata.var['ribo'] = adata.var['gene_name'].str.startswith('RPS','RPL')  # annotate the group of ribosomal genes as 'ribo'
sc.pp.calculate_qc_metrics(adata, qc_vars=['mt','ribo'], percent_top=None, log1p=True, inplace=True)
```

Example 20 (python):
```python
plt.rcParams['figure.figsize'] = [3, 2]

sc.pl.scatter(adata, x='total_counts', y='n_genes')
sc.pl.scatter(adata, x='total_counts', y='pct_counts_mt')
sc.pl.scatter(adata, x='total_counts', y='pct_counts_ribo')
```

Example 21 (python):
```python
plt.rcParams['figure.figsize'] = [2, 2]
adata = adata[adata.obs['pct_counts_mt'] < 5, :]
adata = adata[adata.obs['pct_counts_ribo'] < 5, :]
sc.pl.violin(adata, ['n_genes', 'total_counts', 'log1p_total_counts','pct_counts_mt', 'pct_counts_ribo'],jitter = 0, multi_panel = True)
```

Example 22 (python):
```python
sc.settings.set_figure_params(dpi = 120, color_map = 'RdPu')
plt.rcParams['figure.figsize'] = 10,3
sc.pl.violin(adata,keys=['log1p_total_counts'],groupby='sangerID',rotation=90,jitter=False)
```

Example 23 (unknown):
```unknown
Trying to set attribute `.uns` of view, copying.
```

Example 24 (python):
```python
plt.rcParams['figure.figsize'] = 10,3
sc.pl.violin(adata,keys=['pct_counts_mt'],groupby='sangerID',rotation=90,jitter=False)
```

Example 25 (python):
```python
plt.rcParams['figure.figsize'] = 10,3
sc.pl.violin(adata,keys=['pct_counts_ribo'],groupby='sangerID',rotation=90,jitter=False)
```

Example 26 (python):
```python
sc.settings.set_figure_params(dpi = 120, color_map = 'RdPu')
```

Example 27 (python):
```python
adata.shape
```

Example 28 (unknown):
```unknown
(84159, 36601)
```

Example 29 (python):
```python
adata_c = adata.copy()

adata_c.var.set_index('gene_name',inplace=True)
adata_c.var_names = adata_c.var_names.astype('str')
adata_c.var_names_make_unique()

sc.pp.filter_genes(adata_c, min_cells=3)
sc.pp.normalize_total(adata_c, target_sum=1e4)
sc.pp.log1p(adata_c)
```

Example 30 (unknown):
```unknown
filtered out 4196 genes that are detected in less than 3 cells
normalizing counts per cell
    finished (0:00:00)
```

Example 31 (python):
```python
# scoring hemoglobin genes
# HGNC, Hemoglobin subunits, groupID:940
genes=['HBA1','HBA2','HBB','HBD','HBE1','HBG1','HBG2','HBM','HBQ1','HBZ']
sc.tl.score_genes(adata_c,gene_list=genes,score_name='HB_score')
```

Example 32 (python):
```python
computing score 'HB_score'
    finished: added
    'HB_score', score of gene set (adata.obs).
    400 total control genes are used. (0:00:03)
```

Example 33 (python):
```python
len(adata_c.obs['sangerID'].unique())
```

Example 34 (python):
```python
sc.settings.set_figure_params(dpi = 80, color_map = 'RdPu', dpi_save = 180, vector_friendly = True, format = 'svg')

fig,axs=plt.subplots(nrows=2,ncols=5,figsize=(15,5))

for i,ID in enumerate(adata_c.obs['sangerID'].unique()):
    sample = adata_c[adata_c.obs['sangerID']==ID]
    axs[i//5,i%5].hist(sample.obs['HB_score'],bins=100)
    axs[i//5,i%5].set_title(ID)

fig.tight_layout()
plt.show()
```

Example 35 (python):
```python
# add HB score to original adata
adata.obs.loc[adata_c.obs_names,'HB_score']=adata_c.obs['HB_score']
```

Example 36 (python):
```python
HBscore_thresh=1

adata=adata[adata.obs['HB_score']<HBscore_thresh]
print(adata.shape)
```

Example 37 (unknown):
```unknown
(82277, 36601)
```

Example 38 (python):
```python
adata.obs['combinedID'].value_counts()
```

Example 39 (unknown):
```unknown
BHF_F_Hea13188319_BHF_F_Hea13187621    21451
BHF_F_Hea13188321_BHF_F_Hea13187623    16869
BHF_F_Hea13188325_BHF_F_Hea13187627     9103
BHF_F_Hea13188323_BHF_F_Hea13187625     8115
BHF_F_Hea13188327_BHF_F_Hea13187629     7705
BHF_F_Hea13188322_BHF_F_Hea13187624     6868
BHF_F_Hea13188320_BHF_F_Hea13187622     6450
BHF_F_Hea13188326_BHF_F_Hea13187628     5716
Name: combinedID, dtype: int64
```

Example 40 (python):
```python
# save
adata.write('/nfs/team205/heart/anndata_objects/Foetal/QC/T21_multiome_RNA_foetal.h5ad')
adata
```

Example 41 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 82277 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score'
    var: 'gene_name', 'feature_type', 'ambient_expression-0', 'ambient_expression-1', 'ambient_expression-2', 'ambient_expression-3', 'ambient_expression-4', 'ambient_expression-5', 'ambient_expression-6', 'ambient_expression-7', 'mt', 'ribo', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts'
    uns: 'sangerID_colors'
    obsm: 'latent_gene_encoding'
```

Example 42 (python):
```python
# read in 
adata = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/QC/T21_multiome_RNA_foetal.h5ad')
adata
```

Example 43 (unknown):
```unknown
AnnData object with n_obs × n_vars = 82277 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score'
    var: 'gene_name', 'feature_type', 'ambient_expression-0', 'ambient_expression-1', 'ambient_expression-2', 'ambient_expression-3', 'ambient_expression-4', 'ambient_expression-5', 'ambient_expression-6', 'ambient_expression-7', 'mt', 'ribo', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts'
    uns: 'sangerID_colors'
    obsm: 'latent_gene_encoding'
```

Example 44 (python):
```python
# set batch key and save again
adata.obs['batch_key']=adata.obs['donor'].astype('str')+'_'+adata.obs['cell_or_nuclei'].astype('str')+'_'+adata.obs['kit_10x'].astype('str')

# save
adata.write('/nfs/team205/heart/anndata_objects/Foetal/QC/T21_multiome_RNA_foetal.h5ad')

adata.obs['batch_key'].value_counts()
```

Example 45 (unknown):
```unknown
... storing 'batch_key' as categorical
```

Example 46 (unknown):
```unknown
Hst42_Nuclei_Multiome-v1    59753
Hst44_Nuclei_Multiome-v1    22524
Name: batch_key, dtype: int64
```

Example 47 (unknown):
```unknown
# rclone to gdrive
rclone copy /nfs/team205/heart/anndata_objects/Foetal/QC/T21_multiome_RNA_foetal.h5ad \
gdrive:JamesC_Kazumasa/Foetal_Heart/SanjayShinha/toSinhalab/anndata/
```

Example 48 (python):
```python
adata.obs['combinedID'].value_counts()
```

Example 49 (python):
```python
adata.obs['donor'].value_counts()
```

Example 50 (python):
```python
pd.crosstab(adata.obs['donor'],adata.obs['age'])
```

Example 51 (python):
```python
pd.crosstab(adata.obs['donor'],adata.obs['gender'])
```

Example 52 (python):
```python
adata.obs['SOC | status'].value_counts()
```

Example 53 (python):
```python
adata.obs[adata.obs['multiplexed']=='Yes']['donor_by_library-prep'].value_counts()
```

---

## Variables¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/Xenium/no2_integration_and_cell-type-annotation/no6_manual-annotation_neurons.html

**Contents:**
- Variables¶
- Read in data¶
- 1st round annotation¶
- Subset and reclustering¶
- Transfer annotation to the original adata¶
- Save¶

--> cluster 2 and 6 has lower total counts and smaller cell area, and no specific gene expression --> annotate as "unassigned"

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os

import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 

import scanpy.external as sce
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/session_info/main.py:213: UserWarning: The '__version__' attribute is deprecated and will be removed in MarkupSafe 3.1. Use feature detection, or `importlib.metadata.version("markupsafe")`, instead.
  mod_version = _find_version(mod.__version__)
```

Example 4 (python):
```python
-----
matplotlib          3.10.1
numpy               2.1.3
pandas              2.2.3
scanpy              1.11.1
seaborn             0.13.2
session_info        v1.0.1
-----
```

Example 5 (python):
```python
Cython              3.1.1
PIL                 11.1.0
anndata             0.11.4
asciitree           NA
asttokens           NA
charset_normalizer  3.4.1
cloudpickle         3.1.1
comm                0.2.2
cycler              0.12.1
cython              3.1.1
cython_runtime      NA
dask                2024.11.2
dateutil            2.9.0.post0
debugpy             1.8.13
decorator           5.2.1
exceptiongroup      1.2.2
executing           2.2.0
h5py                3.13.0
igraph              0.11.8
importlib_metadata  NA
ipykernel           6.29.5
jedi                0.19.2
jinja2              3.1.6
joblib              1.4.2
kiwisolver          1.4.8
legacy_api_wrap     NA
leidenalg           0.10.2
llvmlite            0.44.0
markupsafe          3.0.2
mpl_toolkits        NA
msgpack             1.1.0
natsort             8.4.0
numba               0.61.0
numcodecs           0.13.1
packaging           24.2
parso               0.8.4
patsy               1.0.1
platformdirs        4.3.7
prompt_toolkit      3.0.50
psutil              7.0.0
pure_eval           0.2.3
pyarrow             19.0.1
pydev_ipython       NA
pydevconsole        NA
pydevd              3.2.3
pydevd_file_utils   NA
pydevd_plugins      NA
pydevd_tracing      NA
pygments            2.19.1
pyparsing           3.2.3
pytz                2025.2
scipy               1.15.2
six                 1.17.0
sklearn             1.5.2
stack_data          0.6.3
statsmodels         0.14.4
tblib               3.1.0
texttable           1.7.0
threadpoolctl       3.6.0
tlz                 1.0.0
toolz               1.0.0
tornado             6.4.2
traitlets           5.14.3
typing_extensions   NA
wcwidth             0.2.13
yaml                6.0.2
zarr                2.18.3
zipp                NA
zmq                 26.3.0
zoneinfo            NA
```

Example 6 (unknown):
```unknown
-----
IPython             8.34.0
jupyter_client      8.6.3
jupyter_core        5.7.2
-----
Python 3.10.14 | packaged by conda-forge | (main, Mar 20 2024, 12:45:18) [GCC 12.3.0]
Linux-4.18.0-553.53.1.el8_10.x86_64-x86_64-with-glibc2.28
-----
Session information updated at 2025-06-12 17:35
```

Example 7 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/home/kk837/rfs/rfs-teichlab-nfs-iCNyzSAaucw/kk837/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 8 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 9 (python):
```python
adata_dir = '/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium/subsets'
sample_id = 'C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat'
celltype = 'Neurons'
```

Example 10 (unknown):
```unknown
# figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk/Aug2024'
# table_dir = '/nfs/team205/heart/fetal_ms_figures/tables/Aug2024'
```

Example 11 (unknown):
```unknown
latent_space = 'pca_harmony'
```

Example 12 (python):
```python
# read in
adata = sc.read_h5ad(f'{adata_dir}/{sample_id}_5K_{celltype}_lognorm.h5ad')
adata
```

Example 13 (unknown):
```unknown
AnnData object with n_obs × n_vars = 1992 × 4270
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_3.0', 'multi_celltypes_coarse', 'leiden_0.2', 'leiden_0.4', 'leiden_0.6', 'leiden_1'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1', 'n_cells', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'dendrogram_pca_harmony_leiden_1.0', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1_colors', 'log1p', 'multi_celltypes_coarse_colors', 'pca', 'pca_harmony', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.2_colors', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.3_colors', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.4_colors', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.5_colors', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_0.8_colors', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.0_colors', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_1.5_colors', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_2.0_colors', 'pca_harmony_leiden_3.0', 'pca_harmony_leiden_3.0_colors', 'tissue_block_id_colors', 'umap'
    obsm: 'X_pca', 'X_pca_harmony', 'X_umap_pca_harmony', 'spatial'
    varm: 'PCs'
    obsp: 'pca_harmony_connectivities', 'pca_harmony_distances'
```

Example 14 (python):
```python
adata.X.data[:5]
```

Example 15 (unknown):
```unknown
array([3.4320538, 3.4320538, 4.1089087, 3.4320538, 3.4320538],
      dtype=float32)
```

Example 16 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=['total_counts','n_genes_by_counts','cell_area','tissue_block_id'],
                    wspace=0.2,cmap='RdPu',vmax='p100')
```

Example 17 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=['multi_celltypes_coarse',"conf_score_midmod2fine","celltypist_midmod2fine",],
                    wspace=0.2,cmap='RdPu',vmax='p100')
resolutions_list = [0.2,0.4,0.6,1]
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                    color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                    wspace=0.2)
```

Example 18 (python):
```python
%%time
# leiden clustering
resolutions_list = [# 0.2,0.4,0.6,1, # already done
                    1.2,1.5,2]
for resolution in resolutions_list:
    sc.tl.leiden(adata,neighbors_key=latent_space,
                     resolution=resolution,key_added=f'leiden_{str(resolution)}',n_iterations=2)
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                wspace=0.3)
```

Example 19 (unknown):
```unknown
<timed exec>:5: FutureWarning: In the future, the default backend for leiden will be igraph instead of leidenalg.

 To achieve the future defaults please pass: flavor="igraph" and n_iterations=2.  directed must also be False to work with igraph's implementation.
```

Example 20 (unknown):
```unknown
CPU times: user 1.46 s, sys: 62.3 ms, total: 1.52 s
Wall time: 1.52 s
```

Example 21 (python):
```python
# markers for fine-grained cell types
markers = {
    "ChromaffinCells": [
        'DLK1',
        'EPAS1',
        'KCNJ6',
        "TH",    # Tyrosine hydroxylase
        "PNMT",  # Phenylethanolamine N-methyltransferase
        "CHGA",  # Chromogranin A
        "CHGB",  # Chromogranin B
        "PHOX2B"
    ],
    "NeuronPrecursors": [
        "NES",    # Nestin
        "PAX6",   # Paired box 6
        "TUBB3",  # Tubulin Beta 3 Class III
        "DCX",     # Doublecortin
        'ASCL1','NELL2','BNC2'
    ],
    "ParasympatheticNeurons": [
        "PHOX2B",
        "CHAT",    # Choline O-Acetyltransferase
        "SLC18A3", # Solute Carrier Family 18 Member A3 (Vesicular Acetylcholine Transporter)
        "GATA3",
        'NEFL','SST'
    ],
    "SympatheticNeurons": [
        "TH",
        "PHOX2B",
        "GATA3",
        "CHGA",
        "CHGB",
        'DBH','VSTM2L'
    ],
    'Cycling':['MKI67','TOP2A']
}
genes = []
for g in markers.values():
    genes = genes + [x for x in g if (x in adata.var_names)&(x not in genes)]
for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in adata.var_names]
    
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=genes,
               ncols=5, wspace=0.2,cmap='RdPu',vmax='p100')
```

Example 22 (unknown):
```unknown
resolution_sel = 1.5
```

Example 23 (python):
```python
sc.tl.dendrogram(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.pl.dotplot(adata,
              var_names=markers,
              groupby=f'leiden_{str(resolution_sel)}',
              standard_scale='var',
              dendrogram=True
             )
```

Example 24 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: ChromaffinCells, NeuronPrecursors, ParasympatheticNeurons, etc.
```

Example 25 (python):
```python
print(adata.X.data[:5])
adata.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata, dendrogram=True, standard_scale='var', return_fig=True)
dp.add_totals().show()
```

Example 26 (unknown):
```unknown
[3.4320538 3.4320538 4.1089087 3.4320538 3.4320538]
```

Example 27 (python):
```python
sc.pl.violin(adata,keys=['total_counts','cell_area'],groupby=f'leiden_{str(resolution_sel)}')
```

Example 28 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=f'leiden_{str(resolution_sel)}',legend_loc='on data',
                )
```

Example 29 (python):
```python
# annotate
adata.obs['tmp_annotation'] = adata.obs[f'leiden_{str(resolution_sel)}'].copy()
adata.obs.replace({'tmp_annotation':{
    '0':'Neurons',
    '1':'Neurons',
    '2':'unassigned',
    '3':'Neurons',
    '4':'Neurons',
    '5':'Neurons',
    '6':'unassigned',
    '7':'Neurons',
    '8':'Neurons',
    '9':'Neurons',
    '10':'Neurons',
    '11':'Neurons',
    '12':'Neurons',
    '13':'Neurons',
    '14':'Neurons',
    '15':'Neurons',
}},inplace=True)
adata.obs['tmp_annotation'].value_counts()
```

Example 30 (python):
```python
/tmp/ipykernel_350881/479757587.py:3: FutureWarning: The behavior of Series.replace (and DataFrame.replace) with CategoricalDtype is deprecated. In a future version, replace will only be used for cases that preserve the categories. To change the categories, use ser.cat.rename_categories instead.
  adata.obs.replace({'tmp_annotation':{
```

Example 31 (unknown):
```unknown
tmp_annotation
Neurons       1597
unassigned     395
Name: count, dtype: int64
```

Example 32 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color='tmp_annotation'
                )
```

Example 33 (python):
```python
adata_sub = adata[adata.obs['tmp_annotation'].isin(['Neurons'])]
```

Example 34 (python):
```python
# drop leiden clustering results
resolutions_list = [0.2,0.4,0.6,1, 1.2,1.5,2]
for resolution in resolutions_list:
    adata_sub.obs.drop(f'leiden_{str(resolution)}',axis=1,inplace=True)
adata_sub
```

Example 35 (unknown):
```unknown
AnnData object with n_obs × n_vars = 1597 × 4270
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_3.0', 'multi_celltypes_coarse', 'tmp_annotation'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1', 'n_cells', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'dendrogram_pca_harmony_leiden_1.0', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1_colors', 'log1p', 'multi_celltypes_coarse_colors', 'pca', 'pca_harmony', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.2_colors', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.3_colors', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.4_colors', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.5_colors', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_0.8_colors', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.0_colors', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_1.5_colors', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_2.0_colors', 'pca_harmony_leiden_3.0', 'pca_harmony_leiden_3.0_colors', 'tissue_block_id_colors', 'umap', 'leiden_1.2', 'leiden_1.5', 'leiden_2', 'leiden_1.2_colors', 'leiden_1.5_colors', 'leiden_2_colors', 'dendrogram_leiden_1.5', 'rank_genes_groups', 'tmp_annotation_colors'
    obsm: 'X_pca', 'X_pca_harmony', 'X_umap_pca_harmony', 'spatial'
    varm: 'PCs'
    obsp: 'pca_harmony_connectivities', 'pca_harmony_distances'
```

Example 36 (python):
```python
## Remove mid-grain markers from the feature
# read in markers
midgrain_markers = pd.read_csv('/home/kk837/rfs/rfs-teichlab-nfs-iCNyzSAaucw/kk837/notebooks/Foetal/VisiumHD/re-seq_mpp0p3/sc_midgrain_markers.csv',
                               index_col=0)
# remove the markers of celltype of interest
midgrain_markers = midgrain_markers[midgrain_markers['group']!=celltype]
# remove
mask = [x not in list(midgrain_markers['names']) for x in adata_sub.var_names]
bdata = adata_sub[:,mask]
bdata
```

Example 37 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 1597 × 3822
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_3.0', 'multi_celltypes_coarse', 'tmp_annotation'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1', 'n_cells', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'dendrogram_pca_harmony_leiden_1.0', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1_colors', 'log1p', 'multi_celltypes_coarse_colors', 'pca', 'pca_harmony', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.2_colors', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.3_colors', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.4_colors', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.5_colors', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_0.8_colors', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.0_colors', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_1.5_colors', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_2.0_colors', 'pca_harmony_leiden_3.0', 'pca_harmony_leiden_3.0_colors', 'tissue_block_id_colors', 'umap', 'leiden_1.2', 'leiden_1.5', 'leiden_2', 'leiden_1.2_colors', 'leiden_1.5_colors', 'leiden_2_colors', 'dendrogram_leiden_1.5', 'rank_genes_groups', 'tmp_annotation_colors'
    obsm: 'X_pca', 'X_pca_harmony', 'X_umap_pca_harmony', 'spatial'
    varm: 'PCs'
    obsp: 'pca_harmony_connectivities', 'pca_harmony_distances'
```

Example 38 (python):
```python
# remove cell cycle genes
cell_cycle_genes = ['MCM5','PCNA','TYMS','FEN1','MCM2','MCM4','RRM1','UNG','GINS2','MCM6','CDCA7','DTL','PRIM1','UHRF1','MLF1IP','HELLS','RFC2','RPA2','NASP',
                'RAD51AP1','GMNN','WDR76','SLBP','CCNE2','UBR7','POLD3','MSH2','ATAD2','RAD51','RRM2','CDC45','CDC6','EXO1','TIPIN','DSCC1','BLM','CASP8AP2',
                'USP1','CLSPN','POLA1','CHAF1B','BRIP1','E2F8','HMGB2','CDK1','NUSAP1','UBE2C','BIRC5','TPX2','TOP2A','NDC80','CKS2','NUF2','CKS1B','MKI67',
                'TMPO','CENPF','TACC3','FAM64A','SMC4','CCNB2','CKAP2L','CKAP2','AURKB','BUB1','KIF11','ANP32E','TUBB4B','GTSE1','KIF20B','HJURP','CDCA3','HN1',
                'CDC20','TTK','CDC25C','KIF2C','RANGAP1','NCAPD2','DLGAP5','CDCA2','CDCA8','ECT2','KIF23','HMMR','AURKA','PSRC1','ANLN','LBR','CKAP5','CENPE','CTCF',
                'NEK2','G2E3','GAS2L3','CBX5','CENPA']
mask = [x not in cell_cycle_genes for x in bdata.var_names]
bdata = bdata[:,mask]
bdata.shape
```

Example 39 (unknown):
```unknown
(1597, 3769)
```

Example 40 (python):
```python
# HVGs, using the object which doesn't have midgrain markers
sc.pp.highly_variable_genes(bdata, flavor='seurat', min_mean=0.0125, max_mean=3, min_disp=0.5)
print(f'number of hvgs: {sum(bdata.var.highly_variable)}')
# PCA using the degected HVGs
sc.tl.pca(bdata, use_highly_variable=True)
adata_sub.obsm['X_pca'] = bdata.obsm['X_pca'].copy()
# harmony integration
sce.pp.harmony_integrate(adata_sub, 'tissue_block_id', basis='X_pca', adjusted_basis='X_pca_harmony')
# create neighbour graph
latent_space = 'pca_harmony'
sc.pp.neighbors(adata_sub,n_neighbors=15,use_rep=f"X_{latent_space}",key_added=latent_space)
# visualisation with umap
sc.tl.umap(adata_sub,min_dist=0.5,neighbors_key=latent_space)
adata_sub.obsm[f"X_umap_{latent_space}"]=adata_sub.obsm["X_umap"]
del adata_sub.obsm["X_umap"]
```

Example 41 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_highly_variable_genes.py:698: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  adata.uns["hvg"] = {"flavor": flavor}
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_pca/__init__.py:438: FutureWarning: Argument `use_highly_variable` is deprecated, consider using the mask argument. Use_highly_variable=True can be called through mask_var="highly_variable". Use_highly_variable=False can be called through mask_var=None
  warn(msg, FutureWarning)
```

Example 42 (unknown):
```unknown
number of hvgs: 818
```

Example 43 (python):
```python
2025-06-12 17:39:31,715 - harmonypy - INFO - Computing initial centroids with sklearn.KMeans...
2025-06-12 17:39:31,970 - harmonypy - INFO - sklearn.KMeans initialization complete.
2025-06-12 17:39:32,016 - harmonypy - INFO - Iteration 1 of 10
2025-06-12 17:39:32,138 - harmonypy - INFO - Iteration 2 of 10
2025-06-12 17:39:32,245 - harmonypy - INFO - Converged after 2 iterations
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm
```

Example 44 (python):
```python
%%time
# reclustering
# leiden clustering
resolutions_list = [0.2,0.4,0.6,1, 1.2,1.5,2]
for resolution in resolutions_list:
    sc.tl.leiden(adata_sub,neighbors_key=latent_space,
                 resolution=resolution,key_added=f'leiden_{str(resolution)}',n_iterations=2)
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                wspace=0.3)
```

Example 45 (unknown):
```unknown
CPU times: user 2.66 s, sys: 106 ms, total: 2.77 s
Wall time: 2.78 s
```

Example 46 (python):
```python
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color=genes,
               ncols=5, wspace=0.2,cmap='RdPu',vmax='p100')
```

Example 47 (unknown):
```unknown
sub_resolution_sel = 1
```

Example 48 (python):
```python
sc.tl.dendrogram(adata_sub,groupby=f'leiden_{str(sub_resolution_sel)}')
sc.pl.dotplot(adata_sub,
              var_names=markers,
              groupby=f'leiden_{str(sub_resolution_sel)}',
              standard_scale='var',
              dendrogram=True
             )
```

Example 49 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: ChromaffinCells, NeuronPrecursors, ParasympatheticNeurons, etc.
```

Example 50 (python):
```python
# DEG analysis
print(adata_sub.X.data[:5])
adata_sub.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata_sub,groupby=f'leiden_{str(sub_resolution_sel)}')
sc.tl.dendrogram(adata_sub, groupby=f'leiden_{str(sub_resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata_sub, dendrogram=True, standard_scale='var', return_fig=True)
dp.add_totals().show()
```

Example 51 (unknown):
```unknown
[3.4320538 3.4320538 4.1089087 3.4320538 3.4320538]
```

Example 52 (python):
```python
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color=f'leiden_{str(sub_resolution_sel)}',legend_loc='on data',
                wspace=0.3)
```

Example 53 (python):
```python
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color=['celltypist_midmod2fine', 'conf_score_midmod2fine'],
                wspace=0.7)
```

Example 54 (python):
```python
# annotate
adata_sub.obs['fine_grain_manual'] = adata_sub.obs[f'leiden_{str(sub_resolution_sel)}'].copy()
adata_sub.obs.replace({'fine_grain_manual':{
    '0':'ParasympatheticNeurons',
    '1':'NeuronPrecursors',
    '2':'SympatheticNeurons',
    '3':'NeuronPrecursors',
    '4':'ParasympatheticNeurons',
    '5':'ParasympatheticNeurons',
    '6':'ChromaffinCells',
    '7':'EntericLikeNeurons'
}},inplace=True)
adata_sub.obs['fine_grain_manual'].value_counts()
```

Example 55 (python):
```python
/tmp/ipykernel_350881/3142940995.py:3: FutureWarning: The behavior of Series.replace (and DataFrame.replace) with CategoricalDtype is deprecated. In a future version, replace will only be used for cases that preserve the categories. To change the categories, use ser.cat.rename_categories instead.
  adata_sub.obs.replace({'fine_grain_manual':{
```

Example 56 (unknown):
```unknown
fine_grain_manual
ParasympatheticNeurons    657
NeuronPrecursors          513
SympatheticNeurons        276
ChromaffinCells           106
EntericLikeNeurons         45
Name: count, dtype: int64
```

Example 57 (python):
```python
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color=['NOS1', 'VIP', 'CARTPT', 'RET','TBX3'],
               ncols=5, wspace=0.2,cmap='RdPu',vmax='p100')
```

Example 58 (python):
```python
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color='fine_grain_manual',
                wspace=0.3)
```

Example 59 (python):
```python
# markers for fine-grained cell types
markers = {
    "ChromaffinCells": [
        'DLK1',
        'EPAS1',
        'KCNJ6',
        "TH",    # Tyrosine hydroxylase
        "PNMT",  # Phenylethanolamine N-methyltransferase
        "CHGA",  # Chromogranin A
        "CHGB",  # Chromogranin B
        "PHOX2B"
    ],
    "NeuronPrecursors": [
        "NES",    # Nestin
        "PAX6",   # Paired box 6
        "TUBB3",  # Tubulin Beta 3 Class III
        "DCX",     # Doublecortin
        'ASCL1','NELL2','BNC2'
    ],
    "ParasympatheticNeurons": [
        "PHOX2B",
        "CHAT",    # Choline O-Acetyltransferase
        "SLC18A3", # Solute Carrier Family 18 Member A3 (Vesicular Acetylcholine Transporter)
        "GATA3",
        'NEFL','SST'
    ],
    "SympatheticNeurons": [
        "TH",
        "PHOX2B",
        "GATA3",
        "CHGA",
        "CHGB",
        'DBH','VSTM2L'
    ],
    'EntericNeurons':['NOS1', 'VIP', 'CARTPT', 'RET'],
    'Cycling':['MKI67','TOP2A']
}
genes = []
for g in markers.values():
    genes = genes + [x for x in g if (x in adata.var_names)&(x not in genes)]
for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in adata.var_names]

sc.tl.dendrogram(adata_sub,groupby='fine_grain_manual')
sc.pl.dotplot(adata_sub,
              var_names=markers,
              groupby='fine_grain_manual',
              standard_scale='var',
              dendrogram=True
             )
```

Example 60 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: ParasympatheticNeurons, NeuronPrecursors, SympatheticNeurons, etc.
var_group_labels: ChromaffinCells, NeuronPrecursors, ParasympatheticNeurons, etc.
```

Example 61 (python):
```python
adata.obs['fine_grain_manual'] = adata.obs['tmp_annotation'].astype('str').copy()
adata.obs.loc[adata_sub.obs_names,'fine_grain_manual'] = adata_sub.obs['fine_grain_manual']
```

Example 62 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color='fine_grain_manual'
                )
```

Example 63 (python):
```python
for tissue_id in ['C194-HEA-0-FFPE-1', 'Hst45-HEA-0-FFPE-1']:
    slide = adata[adata.obs['tissue_block_id']==tissue_id]
    sc.pl.spatial(slide,color=['fine_grain_manual'],# groups='SinoatrialNodePacemakerCells',
                  spot_size=60,vmax='p100')
```

Example 64 (python):
```python
/tmp/ipykernel_350881/4023824168.py:3: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],# groups='SinoatrialNodePacemakerCells',
```

Example 65 (python):
```python
/tmp/ipykernel_350881/4023824168.py:3: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],# groups='SinoatrialNodePacemakerCells',
```

Example 66 (python):
```python
for tissue_id in ['C194-HEA-0-FFPE-1', 'Hst45-HEA-0-FFPE-1']:
    slide = adata[adata.obs['tissue_block_id']==tissue_id]
    sc.pl.spatial(slide,color=['fine_grain_manual'],groups='ParasympatheticNeurons',
                  spot_size=80,vmax='p100')
```

Example 67 (python):
```python
/tmp/ipykernel_350881/2352066969.py:3: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups='ParasympatheticNeurons',
```

Example 68 (python):
```python
/tmp/ipykernel_350881/2352066969.py:3: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups='ParasympatheticNeurons',
```

Example 69 (python):
```python
for tissue_id in ['C194-HEA-0-FFPE-1', 'Hst45-HEA-0-FFPE-1']:
    slide = adata[adata.obs['tissue_block_id']==tissue_id]
    sc.pl.spatial(slide,color=['fine_grain_manual'],groups='SympatheticNeurons',
                  spot_size=60,vmax='p100')
```

Example 70 (python):
```python
/tmp/ipykernel_350881/1588966400.py:3: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups='SympatheticNeurons',
```

Example 71 (python):
```python
/tmp/ipykernel_350881/1588966400.py:3: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups='SympatheticNeurons',
```

Example 72 (python):
```python
for tissue_id in ['C194-HEA-0-FFPE-1', 'Hst45-HEA-0-FFPE-1']:
    slide = adata[adata.obs['tissue_block_id']==tissue_id]
    sc.pl.spatial(slide,color=['fine_grain_manual'],groups='EntericLikeNeurons',
                  spot_size=80,vmax='p100')
```

Example 73 (python):
```python
/tmp/ipykernel_350881/1078245424.py:3: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups='EntericLikeNeurons',
```

Example 74 (python):
```python
/tmp/ipykernel_350881/1078245424.py:3: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups='EntericLikeNeurons',
```

Example 75 (python):
```python
# save
adata.write(f'{adata_dir}/{sample_id}_5K_{celltype}_lognorm.h5ad')
print(f'{adata_dir}/{sample_id}_5K_{celltype}_lognorm.h5ad')
```

Example 76 (unknown):
```unknown
/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium/subsets/C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat_5K_Neurons_lognorm.h5ad
```

---

## Variables¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/VisiumHD/no7_manual-annotation_atrialCM.html

**Contents:**
- Variables¶
- Read in data¶
- Remove mid-grain markers from the feature¶
- PCA, Clustering, UMAP¶
- Subset SAN cells¶
- Save¶
- Read in for plotting¶

2378 rows × 7 columns

--> cluster 1 has low total_counts and bin_counts. and no specific marker expression --> will annotated as "unassigned"

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os

import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 

import celltypist
```

Example 2 (python):
```python
# import session_info
# session_info.show()
```

Example 3 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 4 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 5 (python):
```python
adata_dir = '/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/subsets'
sample_id = 'HEA_FOET14880396'
celltype = 'AtrialCardiomyocytes'
```

Example 6 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk/Aug2024'
table_dir = '/nfs/team205/heart/fetal_ms_figures/tables/Aug2024'
```

Example 7 (python):
```python
# read in
adata = sc.read_h5ad(f'{adata_dir}/{sample_id}_{celltype}_b2c_cells_filtered_lognorm.h5ad')
adata
```

Example 8 (unknown):
```unknown
AnnData object with n_obs × n_vars = 13904 × 17650
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'leiden_0.2', 'leiden_0.4', 'leiden_0.6', 'leiden_1', 'leiden_1.2', 'leiden_1.5', 'leiden_2', 'fine_grain_manual'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'dendrogram_fine_grain_manual', 'dendrogram_leiden_1', 'dendrogram_leiden_1.5', 'fine_grain_manual_colors', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1.2', 'leiden_1.2_colors', 'leiden_1.5', 'leiden_1.5_colors', 'leiden_1_colors', 'leiden_2', 'leiden_2_colors', 'log1p', 'neighbors', 'pca', 'rank_genes_groups', 'spatial', 'umap'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 9 (python):
```python
# read in
midgrain_markers = pd.read_csv('/nfs/team205/kk18/notebooks/Foetal/VisiumHD/re-seq_mpp0p3/sc_midgrain_markers.csv',
                               index_col=0
                              )
# remove the markers of celltype of interest
midgrain_markers = midgrain_markers[midgrain_markers['group']!=celltype]
midgrain_markers
```

Example 10 (unknown):
```unknown
'GATA2' in list(midgrain_markers['names'])
```

Example 11 (python):
```python
# remove
mask = [x not in list(midgrain_markers['names']) for x in adata.var_names]
bdata = adata[:,mask]
bdata
```

Example 12 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 13904 × 16391
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'leiden_0.2', 'leiden_0.4', 'leiden_0.6', 'leiden_1', 'leiden_1.2', 'leiden_1.5', 'leiden_2', 'fine_grain_manual'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'dendrogram_fine_grain_manual', 'dendrogram_leiden_1', 'dendrogram_leiden_1.5', 'fine_grain_manual_colors', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.6', 'leiden_1', 'leiden_1.2', 'leiden_1.5', 'leiden_2', 'log1p', 'neighbors', 'pca', 'rank_genes_groups', 'spatial', 'umap'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 13 (python):
```python
# HVGs, using the object which doesn't have midgrain markers
sc.pp.highly_variable_genes(bdata, flavor='seurat', min_mean=0.0125, max_mean=3, min_disp=0.5)
print(f'number of hvgs: {sum(bdata.var.highly_variable)}')
# PCA using the degected HVGs
sc.tl.pca(bdata, use_highly_variable=True)
adata.obsm['X_pca'] = bdata.obsm['X_pca'].copy()
# create neighbour graph
sc.pp.neighbors(adata,n_neighbors=15,use_rep='X_pca')
# visualisation with umap
sc.tl.umap(adata,min_dist=0.5)
sc.pl.embedding(adata,basis='X_umap',color=['bin_count','total_counts','n_genes_by_counts',"conf_score_midmod2fine","celltypist_midmod2fine",],
                wspace=0.2,cmap='RdPu',vmax='p100',ncols=5)
# leiden clustering
resolutions_list = [0.2,0.4,0.6,1,1.2,1.5,2]
for resolution in resolutions_list:
    sc.tl.leiden(adata,resolution=resolution,key_added=f'leiden_{str(resolution)}',n_iterations=2)
sc.pl.embedding(adata,basis='X_umap',
                color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                wspace=0.3)
```

Example 14 (unknown):
```unknown
number of hvgs: 3248
```

Example 15 (python):
```python
genes = ['TBX3','TBX18','SHOX2','HCN1','HCN4','CACNA1D', 'GJA1',# SANPC
         'GREB1L','GNG11','NRXN3','RSPO3', # AVNPC
         'PITX2','KCNMB2','PCDH7','RYR3','MBP', # aCM_left
         'NTM','ROR1','KIF26B','PLCL1','ANGPT1', # aCM_right
         'NKX2-5','MKI67','TOP2A'
        ]
genes = [x for x in genes if x in adata.var_names]
sc.pl.embedding(adata,basis='X_umap',color=genes,
               ncols=5, wspace=0.2,cmap='RdPu',vmax='p99.7')
```

Example 16 (unknown):
```unknown
resolution_sel = 1
```

Example 17 (python):
```python
markers = {
    'CM': ['TTN', 'TNNT2','MYH6',
        'MYH7', 'MYL2', 'FHL2', # vCN
            'NPPA', 'MYL7', 'MYL4'], #aCM
    'FB': ['DCN', 'GCN', 'PDGFRA','COL1A1','COL1A2'],
    'EC': ['VWF', 'PECAM1', 'CDH5','RGCC', 'FABP5',],
    'Peri': ['RGS5', 'ABCC9', 'KCNJ8'],
    'SMC': ['MYH11', 'TAGLN', 'ACTA2'],
    'Meso': ['MSLN', 'WT1', 'BNC1'],
    'Neuro': ['PLP1', 'NRXN1', 'NRXN3','PRPH', 'NEFL', 'NEFM', 'NEFH', 'STMN2', 'SST', 'SLC10A4', 'SLC18A3', 'TH', 'NPY'],
    'Adipo': ['GPAM', 'FASN', 'LEP'],
    'Myelo': ['CD14', 'C1QA', 'CD68','LYVE1','TIMD4'],
    'Lymph': ['CD8A', 'IL7R', 'CD40LG'],
    'B': ['CD19','SDC1','IGHG1','IGHA1'],
    'Mast': ['KIT', 'CPA3'],
}

for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in adata.var_names]

sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
sc.pl.dotplot(adata,
              markers, 
              groupby=f'leiden_{str(resolution_sel)}',
              dendrogram=True,
              standard_scale="var",
              color_map="Reds",
              swap_axes=False,
             )
```

Example 18 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: CM, FB, EC, etc.
```

Example 19 (python):
```python
genes = ['TBX3','TBX18','SHOX2','HCN1','HCN4','CACNA1D', 'GJA1',# SANPC
         'GREB1L','GNG11','NRXN3','RSPO3', # AVNPC
         'PITX2','KCNMB2','PCDH7','RYR3','MBP', # aCM_left
         'NTM','ROR1','KIF26B','PLCL1','ANGPT1', # aCM_right
         'MKI67','TOP2A','NKX2-5'
        ]
genes = [x for x in genes if x in adata.var_names]

sc.tl.dendrogram(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.pl.dotplot(adata,
              var_names=genes,
              groupby=f'leiden_{str(resolution_sel)}',
              standard_scale='var',
              dendrogram=True
             )
```

Example 20 (python):
```python
# DEG analysis
print(adata.X.data[:5])
adata.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata, dendrogram=True, standard_scale='var', return_fig=True)
dp.add_totals().show()
```

Example 21 (unknown):
```unknown
[1.10352081 1.83785365 1.6291659  0.70335836 1.24040478]
```

Example 22 (python):
```python
# DEG analysis
print(adata.X.data[:5])
adata.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata, dendrogram=True, standard_scale='var', return_fig=True, n_genes=5)
dp.add_totals().show()
```

Example 23 (unknown):
```unknown
[1.10352081 1.83785365 1.6291659  0.70335836 1.24040478]
```

Example 24 (python):
```python
sc.pl.embedding(adata,basis='X_umap',
                color=f'leiden_{str(resolution_sel)}',
                wspace=0.3, legend_loc='on data')
```

Example 25 (python):
```python
sc.pl.embedding(adata,basis='X_umap',
                color=['celltypist_midmod2fine', 'conf_score_midmod2fine'],
                wspace=0.7)
```

Example 26 (python):
```python
sc.pl.violin(adata,keys=['total_counts','bin_count'],groupby=f'leiden_{str(resolution_sel)}')
```

Example 27 (unknown):
```unknown
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 28 (python):
```python
# annotate
adata.obs['fine_grain_manual'] = adata.obs[f'leiden_{str(resolution_sel)}'].copy()
adata.obs.replace({'fine_grain_manual':{
    '0':'AtrialCardiomyocytesLeft',
    '1':'unassigned',
    '2':'AtrialCardiomyocytesRight',
    '3':'AtrialCardiomyocytesRight',
    '4':'AtrioventricularNodePacemakerCells',
    '5':'SinoatrialNodePacemakerCells',
    '6':'AtrialCardiomyocytesRight',
    '7':'SinoatrialNodePacemakerCells',
    '8':'AtrialCardiomyocytesLeft',
}},inplace=True)
adata.obs['fine_grain_manual'].value_counts()
```

Example 29 (unknown):
```unknown
fine_grain_manual
AtrialCardiomyocytesRight             5060
AtrialCardiomyocytesLeft              3107
unassigned                            2625
AtrioventricularNodePacemakerCells    1602
SinoatrialNodePacemakerCells          1510
Name: count, dtype: int64
```

Example 30 (python):
```python
ad = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/Feb28ObjectRaw_finegrain_updated.h5ad',backed='r')
set(ad.obs['fine_grain'])
```

Example 31 (unknown):
```unknown
{'AtrialCardiomyocytesCycling',
 'AtrialCardiomyocytesLeft',
 'AtrialCardiomyocytesRight',
 'AtrioventricularNodePacemakerCells',
 'BCells',
 'BCellsMS4A1pos',
 'ChromaffinCells',
 'CoronaryArterialEndothelialCells',
 'CoronaryCapillaryEndothelialCells',
 'CoronaryPericytes',
 'CoronarySmoothMuscleCells',
 'CoronaryVenousEndothelialCells',
 'CoronaryVesselAdventitialFibroblasts',
 'DendriticCellsMature',
 'DendriticCellsType1',
 'DuctusArteriosusSmoothMuscleCells',
 'EndocardialCells',
 'EndocardialCushionCells',
 'EpicardiumDerivedCells',
 'GreatVesselAdventitialFibroblasts',
 'GreatVesselArterialEndothelialCells',
 'GreatVesselSmoothMuscleCells',
 'GreatVesselVenousEndothelialCells',
 'InnateLymphoidCells',
 'LymphNodeFibroblasticReticularCells',
 'LymphaticEndothelialCells',
 'MacrophagesATF3pos',
 'MacrophagesCX3CR1pos',
 'MacrophagesLYVE1pos',
 'MacrophagesTIMD4pos',
 'MastCells',
 'Megakaryocytes',
 'MesothelialEpicardialCells',
 'MonocyteDerivedCells',
 'Monocytes',
 'MonocytesMPOpos',
 'MyocardialInterstitialFibroblasts',
 'Myofibroblasts',
 'NaturalKillerCells',
 'NeuronPrecursors',
 'ParasympatheticNeurons',
 'PericardialCellsFibrous',
 'PericardialCellsIntermediate',
 'PericardialCellsParietal',
 'PlasmacytoidDendriticCells',
 'ProBCells',
 'SchwannCellPrecursors',
 'SchwannCells',
 'SinoatrialNodePacemakerCells',
 'SubEpicardialFibroblasts',
 'SympatheticNeurons',
 'TCellsCD4pos',
 'TCellsCD8pos',
 'TregsCD4pos',
 'ValveEndothelialCells',
 'ValveInterstitialCells',
 'VentricularCardiomyocytesCycling',
 'VentricularCardiomyocytesLeftCompact',
 'VentricularCardiomyocytesLeftTrabeculated',
 'VentricularCardiomyocytesRightCompact',
 'VentricularCardiomyocytesRightTrabeculated',
 'VentricularConductionSystemDistal',
 'VentricularConductionSystemProximal'}
```

Example 32 (python):
```python
sc.pl.embedding(adata,basis='X_umap',
                color='fine_grain_manual',
                frameon=False,
                wspace=0.3)
```

Example 33 (python):
```python
genes = ['TBX3','TBX18','SHOX2','HCN1','HCN4','CACNA1D', 'GJA1',# SANPC
         'GREB1L','GNG11','NRXN3','RSPO3', # AVNPC
         'PITX2','KCNMB2','PCDH7','RYR3','MBP', # aCM_left
         'NTM','ROR1','KIF26B','PLCL1','ANGPT1', # aCM_right
         'MKI67','TOP2A','NKX2-5'
        ]
genes = [x for x in genes if x in adata.var_names]

sc.tl.dendrogram(adata,groupby='fine_grain_manual')
sc.pl.dotplot(adata,
              var_names=genes,
              groupby='fine_grain_manual',
              standard_scale='var',
              dendrogram=True
             )
```

Example 34 (python):
```python
sc.pl.embedding(adata,basis='X_umap',
                color=['SHOX2','fine_grain_manual'],
                frameon=False, cmap='RdPu', size=15,
                wspace=0.3, show=False)
plt.savefig(f'{figdir}/VisiumHD_euploid_atrialCM_UMAP.pdf', bbox_inches="tight",dpi=300)
```

Example 35 (unknown):
```unknown
maxp pruned
cmap pruned
kern dropped
post pruned
FFTM dropped
GPOS pruned
GSUB pruned
glyf pruned
Added gid0 to subset
Added first four glyphs to subset
Closing glyph list over 'MATH': 39 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'C', 'H', 'L', 'N', 'O', 'P', 'R', 'S', 'X', 'a', 'c', 'd', 'e', 'f', 'four', 'g', 'h', 'i', 'k', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'one', 'r', 's', 'space', 't', 'three', 'two', 'u', 'underscore', 'v', 'y', 'zero']
Glyph IDs:   [0, 1, 2, 3, 19, 20, 21, 22, 23, 36, 38, 43, 47, 49, 50, 51, 53, 54, 59, 66, 68, 70, 71, 72, 73, 74, 75, 76, 78, 79, 80, 81, 82, 85, 86, 87, 88, 89, 92]
Closed glyph list over 'MATH': 39 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'C', 'H', 'L', 'N', 'O', 'P', 'R', 'S', 'X', 'a', 'c', 'd', 'e', 'f', 'four', 'g', 'h', 'i', 'k', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'one', 'r', 's', 'space', 't', 'three', 'two', 'u', 'underscore', 'v', 'y', 'zero']
Glyph IDs:   [0, 1, 2, 3, 19, 20, 21, 22, 23, 36, 38, 43, 47, 49, 50, 51, 53, 54, 59, 66, 68, 70, 71, 72, 73, 74, 75, 76, 78, 79, 80, 81, 82, 85, 86, 87, 88, 89, 92]
Closing glyph list over 'GSUB': 39 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'C', 'H', 'L', 'N', 'O', 'P', 'R', 'S', 'X', 'a', 'c', 'd', 'e', 'f', 'four', 'g', 'h', 'i', 'k', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'one', 'r', 's', 'space', 't', 'three', 'two', 'u', 'underscore', 'v', 'y', 'zero']
Glyph IDs:   [0, 1, 2, 3, 19, 20, 21, 22, 23, 36, 38, 43, 47, 49, 50, 51, 53, 54, 59, 66, 68, 70, 71, 72, 73, 74, 75, 76, 78, 79, 80, 81, 82, 85, 86, 87, 88, 89, 92]
Closed glyph list over 'GSUB': 44 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'C', 'H', 'L', 'N', 'O', 'P', 'R', 'S', 'X', 'a', 'c', 'd', 'e', 'f', 'fi', 'fl', 'four', 'g', 'h', 'i', 'k', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'one', 'r', 's', 'space', 't', 'three', 'two', 'u', 'underscore', 'uniFB00', 'uniFB03', 'uniFB04', 'v', 'y', 'zero']
Glyph IDs:   [0, 1, 2, 3, 19, 20, 21, 22, 23, 36, 38, 43, 47, 49, 50, 51, 53, 54, 59, 66, 68, 70, 71, 72, 73, 74, 75, 76, 78, 79, 80, 81, 82, 85, 86, 87, 88, 89, 92, 5038, 5039, 5040, 5041, 5042]
Closing glyph list over 'glyf': 44 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'C', 'H', 'L', 'N', 'O', 'P', 'R', 'S', 'X', 'a', 'c', 'd', 'e', 'f', 'fi', 'fl', 'four', 'g', 'h', 'i', 'k', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'one', 'r', 's', 'space', 't', 'three', 'two', 'u', 'underscore', 'uniFB00', 'uniFB03', 'uniFB04', 'v', 'y', 'zero']
Glyph IDs:   [0, 1, 2, 3, 19, 20, 21, 22, 23, 36, 38, 43, 47, 49, 50, 51, 53, 54, 59, 66, 68, 70, 71, 72, 73, 74, 75, 76, 78, 79, 80, 81, 82, 85, 86, 87, 88, 89, 92, 5038, 5039, 5040, 5041, 5042]
Closed glyph list over 'glyf': 44 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'C', 'H', 'L', 'N', 'O', 'P', 'R', 'S', 'X', 'a', 'c', 'd', 'e', 'f', 'fi', 'fl', 'four', 'g', 'h', 'i', 'k', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'one', 'r', 's', 'space', 't', 'three', 'two', 'u', 'underscore', 'uniFB00', 'uniFB03', 'uniFB04', 'v', 'y', 'zero']
Glyph IDs:   [0, 1, 2, 3, 19, 20, 21, 22, 23, 36, 38, 43, 47, 49, 50, 51, 53, 54, 59, 66, 68, 70, 71, 72, 73, 74, 75, 76, 78, 79, 80, 81, 82, 85, 86, 87, 88, 89, 92, 5038, 5039, 5040, 5041, 5042]
Retaining 44 glyphs
head subsetting not needed
hhea subsetting not needed
maxp subsetting not needed
OS/2 subsetting not needed
hmtx subsetted
cmap subsetted
fpgm subsetting not needed
prep subsetting not needed
cvt  subsetting not needed
loca subsetting not needed
post subsetted
gasp subsetting not needed
MATH subsetted
GDEF subsetted
GPOS subsetted
GSUB subsetted
name subsetting not needed
glyf subsetted
head pruned
OS/2 Unicode ranges pruned: [0]
OS/2 CodePage ranges pruned: [0]
glyf pruned
GDEF pruned
GPOS pruned
GSUB pruned
name pruned
```

Example 36 (python):
```python
sc.pl.embedding(adata,basis='X_umap',
                color='fine_grain_manual',groups='SinoatrialNodePacemakerCells',
                frameon=False,
                wspace=0.3)
```

Example 37 (python):
```python
san = adata[adata.obs['fine_grain_manual']=='SinoatrialNodePacemakerCells']

# drop leiden clustering results
resolutions_list = [0.2,0.4,0.6,1, 1.2,1.5,2]
for resolution in resolutions_list:
    san.obs.drop(f'leiden_{str(resolution)}',axis=1,inplace=True)
san
```

Example 38 (unknown):
```unknown
AnnData object with n_obs × n_vars = 1510 × 17650
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'fine_grain_manual'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'dendrogram_fine_grain_manual', 'dendrogram_leiden_1', 'dendrogram_leiden_1.5', 'fine_grain_manual_colors', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_1', 'leiden_1.2', 'leiden_1.5', 'leiden_1_colors', 'leiden_2', 'log1p', 'neighbors', 'pca', 'rank_genes_groups', 'spatial', 'umap'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 39 (unknown):
```unknown
# remove mid_grain markers
mask = [x not in list(midgrain_markers['names']) for x in san.var_names]
bdata = san[:,mask]
bdata
```

Example 40 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 1510 × 16391
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'fine_grain_manual'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'dendrogram_fine_grain_manual', 'dendrogram_leiden_1', 'dendrogram_leiden_1.5', 'fine_grain_manual_colors', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_1', 'leiden_1.2', 'leiden_1.5', 'leiden_1_colors', 'leiden_2', 'log1p', 'neighbors', 'pca', 'rank_genes_groups', 'spatial', 'umap'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 41 (python):
```python
# HVGs, using the object which doesn't have midgrain markers
sc.pp.highly_variable_genes(bdata, flavor='seurat', min_mean=0.0125, max_mean=3, min_disp=0.5)
print(f'number of hvgs: {sum(bdata.var.highly_variable)}')
# PCA using the degected HVGs
sc.tl.pca(bdata, use_highly_variable=True)
san.obsm['X_pca'] = bdata.obsm['X_pca'].copy()
# create neighbour graph
sc.pp.neighbors(san,n_neighbors=15,use_rep='X_pca')
```

Example 42 (unknown):
```unknown
number of hvgs: 3692
```

Example 43 (python):
```python
sc.tl.umap(san,min_dist=0.1)
```

Example 44 (python):
```python
%%time
# reclustering
# leiden clustering
resolutions_list = [0.2,0.4,0.6,1, 1.2,1.5,2]
for resolution in resolutions_list:
    sc.tl.leiden(san,resolution=resolution,key_added=f'leiden_{str(resolution)}',n_iterations=2)
sc.pl.embedding(san,basis='X_umap',
                color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                wspace=0.3)
```

Example 45 (unknown):
```unknown
CPU times: user 1.45 s, sys: 31.3 ms, total: 1.48 s
Wall time: 1.95 s
```

Example 46 (unknown):
```unknown
san_resolution_sel = 0.6
```

Example 47 (python):
```python
genes = ['SHOX2','HCN4','TBX3','TBX18',
         'GJA1', # CX43
         'GJA5', # CX40
         'NKX2-5',
         'NPPA'
        ]
genes = [x for x in genes if x in san.var_names]
sc.pl.embedding(san,basis='X_umap',
                color=[f'leiden_{str(san_resolution_sel)}']+genes,
               ncols=5, wspace=0.15,cmap='RdPu',vmax='p100')
```

Example 48 (unknown):
```unknown
# annotate
san.obs['fine_grain_manual'] = san.obs[f'leiden_{str(san_resolution_sel)}'].copy()
san.obs.replace({'fine_grain_manual':{
    '0':'SinoatrialNodePacemakerCellsHead',
    '1':'SinoatrialNodePacemakerCellsHorn',
    '2':'SinoatrialNodePacemakerCellsTail',
}},inplace=True)
san.obs['fine_grain_manual'].value_counts()
```

Example 49 (unknown):
```unknown
fine_grain_manual
SinoatrialNodePacemakerCellsHead    636
SinoatrialNodePacemakerCellsHorn    501
SinoatrialNodePacemakerCellsTail    373
Name: count, dtype: int64
```

Example 50 (unknown):
```unknown
san.obs['fine_grain_manual'] = san.obs['fine_grain_manual'].astype('category')
san.obs['fine_grain_manual'] = san.obs['fine_grain_manual'].cat.reorder_categories([
    'SinoatrialNodePacemakerCellsHorn',
    'SinoatrialNodePacemakerCellsHead',
    'SinoatrialNodePacemakerCellsTail'
])
```

Example 51 (python):
```python
genes = ['TBX3','TBX18',
         'GJA1', # CX43
         'GJA5', # CX40
         'NKX2-5','NPPA']
genes = [x for x in genes if x in san.var_names]
sc.pl.dotplot(san,
              var_names=genes,
              groupby='fine_grain_manual',
              standard_scale='var',
              dendrogram=False
             )
```

Example 52 (python):
```python
res = utils.rank_genes_groups_with_mean(san,groupby='fine_grain_manual',
                                        log2fc_min=0,method='t-test')
n_rank = 20
markers = {}
for group in res['group'].unique():
    res_sub = res[res['group']==group]
    res_sub = res_sub.sort_values('scores',ascending=False)
    markers[group] = list(res_sub['names'])[:n_rank]

sc.tl.dendrogram(san,groupby='fine_grain_manual',var_names=markers,)
sc.pl.dotplot(san,
              var_names=markers,
              groupby='fine_grain_manual',
              standard_scale='var',
              dendrogram=True
             )
```

Example 53 (python):
```python
# with removing mid_grain markers
# remove mid_grain markers
mask = [x not in list(midgrain_markers['names']) for x in san.var_names]
bdata = san[:,mask]

res = utils.rank_genes_groups_with_mean(bdata,groupby='fine_grain_manual',
                                        log2fc_min=0,method='t-test')
n_rank = 20
markers = {}
for group in res['group'].unique():
    res_sub = res[res['group']==group]
    res_sub = res_sub.sort_values('scores',ascending=False)
    markers[group] = list(res_sub['names'])[:n_rank]

sc.tl.dendrogram(bdata,groupby='fine_grain_manual',var_names=markers,)
sc.pl.dotplot(bdata,
              var_names=markers,
              groupby='fine_grain_manual',
              standard_scale='var',
              dendrogram=True
             )
```

Example 54 (python):
```python
# add the annotation to the original adata
adata.obs['fine_grain_manual'] = adata.obs['fine_grain_manual'].astype('str')
adata.obs.loc[san.obs_names,'fine_grain_manual'] = san.obs['fine_grain_manual'].astype('str').copy()
adata.obs['fine_grain_manual'].value_counts()
```

Example 55 (unknown):
```unknown
fine_grain_manual
AtrialCardiomyocytesRight             5060
AtrialCardiomyocytesLeft              3107
unassigned                            2625
AtrioventricularNodePacemakerCells    1602
SinoatrialNodePacemakerCellsHead       636
SinoatrialNodePacemakerCellsHorn       501
SinoatrialNodePacemakerCellsTail       373
Name: count, dtype: int64
```

Example 56 (python):
```python
# save
adata.write(f'{adata_dir}/{sample_id}_{celltype}_b2c_cells_filtered_lognorm.h5ad')
print(f'{adata_dir}/{sample_id}_{celltype}_b2c_cells_filtered_lognorm.h5ad')
```

Example 57 (unknown):
```unknown
... storing 'fine_grain_manual' as categorical
```

Example 58 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/subsets/HEA_FOET14880396_AtrialCardiomyocytes_b2c_cells_filtered_lognorm.h5ad
```

Example 59 (python):
```python
adata = sc.read_h5ad(f'{adata_dir}/{sample_id}_{celltype}_b2c_cells_filtered_lognorm.h5ad')
adata
```

Example 60 (unknown):
```unknown
AnnData object with n_obs × n_vars = 13904 × 17650
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'leiden_0.2', 'leiden_0.4', 'leiden_0.6', 'leiden_1', 'leiden_1.2', 'leiden_1.5', 'leiden_2', 'fine_grain_manual'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'dendrogram_fine_grain_manual', 'dendrogram_leiden_1', 'dendrogram_leiden_1.5', 'fine_grain_manual_colors', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1.2', 'leiden_1.2_colors', 'leiden_1.5', 'leiden_1.5_colors', 'leiden_1_colors', 'leiden_2', 'leiden_2_colors', 'log1p', 'neighbors', 'pca', 'rank_genes_groups', 'spatial', 'umap'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 61 (python):
```python
set(adata.obs['fine_grain_manual'])
```

Example 62 (unknown):
```unknown
{'AtrialCardiomyocytesLeft',
 'AtrialCardiomyocytesRight',
 'AtrioventricularNodePacemakerCells',
 'SinoatrialNodePacemakerCellsHead',
 'SinoatrialNodePacemakerCellsHorn',
 'SinoatrialNodePacemakerCellsTail',
 'unassigned'}
```

Example 63 (python):
```python
san = adata[adata.obs['fine_grain_manual'].isin(['SinoatrialNodePacemakerCellsHead',
 'SinoatrialNodePacemakerCellsHorn',
 'SinoatrialNodePacemakerCellsTail'])]
san
```

Example 64 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 1510 × 17650
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'leiden_0.2', 'leiden_0.4', 'leiden_0.6', 'leiden_1', 'leiden_1.2', 'leiden_1.5', 'leiden_2', 'fine_grain_manual'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'dendrogram_fine_grain_manual', 'dendrogram_leiden_1', 'dendrogram_leiden_1.5', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_1', 'leiden_1.2', 'leiden_1.5', 'leiden_1_colors', 'leiden_2', 'log1p', 'neighbors', 'pca', 'rank_genes_groups', 'spatial', 'umap'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 65 (python):
```python
# remove mid_grain markers
mask = [x not in list(midgrain_markers['names']) for x in san.var_names]
bdata = san[:,mask]

# HVGs, using the object which doesn't have midgrain markers
sc.pp.highly_variable_genes(bdata, flavor='seurat', min_mean=0.0125, max_mean=3, min_disp=0.5)
print(f'number of hvgs: {sum(bdata.var.highly_variable)}')
# PCA using the degected HVGs
sc.tl.pca(bdata, use_highly_variable=True)
san.obsm['X_pca'] = bdata.obsm['X_pca'].copy()
# create neighbour graph
sc.pp.neighbors(san,n_neighbors=15,use_rep='X_pca')
# umap
sc.tl.umap(san,min_dist=0.1)
```

Example 66 (unknown):
```unknown
number of hvgs: 3692
```

Example 67 (python):
```python
sc.pl.embedding(san,basis='X_umap',
                color=['fine_grain_manual'],
               frameon=False,show=False)
plt.savefig(f'{figdir}/VisiumHD_euploid_SANPC_UMAP.pdf', bbox_inches="tight",dpi=300)
```

Example 68 (unknown):
```unknown
maxp pruned
cmap pruned
kern dropped
post pruned
FFTM dropped
GPOS pruned
GSUB pruned
glyf pruned
Added gid0 to subset
Added first four glyphs to subset
Closing glyph list over 'MATH': 27 glyphs before
Glyph names: ['.notdef', '.null', 'C', 'H', 'N', 'P', 'S', 'T', 'a', 'c', 'd', 'e', 'f', 'g', 'i', 'k', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'r', 's', 'space', 't', 'u', 'underscore']
Glyph IDs:   [0, 1, 2, 3, 38, 43, 49, 51, 54, 55, 66, 68, 70, 71, 72, 73, 74, 76, 78, 79, 80, 81, 82, 85, 86, 87, 88]
Closed glyph list over 'MATH': 27 glyphs after
Glyph names: ['.notdef', '.null', 'C', 'H', 'N', 'P', 'S', 'T', 'a', 'c', 'd', 'e', 'f', 'g', 'i', 'k', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'r', 's', 'space', 't', 'u', 'underscore']
Glyph IDs:   [0, 1, 2, 3, 38, 43, 49, 51, 54, 55, 66, 68, 70, 71, 72, 73, 74, 76, 78, 79, 80, 81, 82, 85, 86, 87, 88]
Closing glyph list over 'GSUB': 27 glyphs before
Glyph names: ['.notdef', '.null', 'C', 'H', 'N', 'P', 'S', 'T', 'a', 'c', 'd', 'e', 'f', 'g', 'i', 'k', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'r', 's', 'space', 't', 'u', 'underscore']
Glyph IDs:   [0, 1, 2, 3, 38, 43, 49, 51, 54, 55, 66, 68, 70, 71, 72, 73, 74, 76, 78, 79, 80, 81, 82, 85, 86, 87, 88]
Closed glyph list over 'GSUB': 32 glyphs after
Glyph names: ['.notdef', '.null', 'C', 'H', 'N', 'P', 'S', 'T', 'a', 'c', 'd', 'e', 'f', 'fi', 'fl', 'g', 'i', 'k', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'r', 's', 'space', 't', 'u', 'underscore', 'uniFB00', 'uniFB03', 'uniFB04']
Glyph IDs:   [0, 1, 2, 3, 38, 43, 49, 51, 54, 55, 66, 68, 70, 71, 72, 73, 74, 76, 78, 79, 80, 81, 82, 85, 86, 87, 88, 5038, 5039, 5040, 5041, 5042]
Closing glyph list over 'glyf': 32 glyphs before
Glyph names: ['.notdef', '.null', 'C', 'H', 'N', 'P', 'S', 'T', 'a', 'c', 'd', 'e', 'f', 'fi', 'fl', 'g', 'i', 'k', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'r', 's', 'space', 't', 'u', 'underscore', 'uniFB00', 'uniFB03', 'uniFB04']
Glyph IDs:   [0, 1, 2, 3, 38, 43, 49, 51, 54, 55, 66, 68, 70, 71, 72, 73, 74, 76, 78, 79, 80, 81, 82, 85, 86, 87, 88, 5038, 5039, 5040, 5041, 5042]
Closed glyph list over 'glyf': 32 glyphs after
Glyph names: ['.notdef', '.null', 'C', 'H', 'N', 'P', 'S', 'T', 'a', 'c', 'd', 'e', 'f', 'fi', 'fl', 'g', 'i', 'k', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'r', 's', 'space', 't', 'u', 'underscore', 'uniFB00', 'uniFB03', 'uniFB04']
Glyph IDs:   [0, 1, 2, 3, 38, 43, 49, 51, 54, 55, 66, 68, 70, 71, 72, 73, 74, 76, 78, 79, 80, 81, 82, 85, 86, 87, 88, 5038, 5039, 5040, 5041, 5042]
Retaining 32 glyphs
head subsetting not needed
hhea subsetting not needed
maxp subsetting not needed
OS/2 subsetting not needed
hmtx subsetted
cmap subsetted
fpgm subsetting not needed
prep subsetting not needed
cvt  subsetting not needed
loca subsetting not needed
post subsetted
gasp subsetting not needed
MATH subsetted
GDEF subsetted
GPOS subsetted
GSUB subsetted
name subsetting not needed
glyf subsetted
head pruned
OS/2 Unicode ranges pruned: [0]
OS/2 CodePage ranges pruned: [0]
glyf pruned
GDEF pruned
GPOS pruned
GSUB pruned
name pruned
```

Example 69 (python):
```python
?sc.pl.dotplot
```

Example 70 (python):
```python
Signature:
sc.pl.dotplot(
    adata: 'AnnData',
    var_names: '_VarNames | Mapping[str, _VarNames]',
    groupby: 'str | Sequence[str]',
    *,
    use_raw: 'bool | None' = None,
    log: 'bool' = False,
    num_categories: 'int' = 7,
    expression_cutoff: 'float' = 0.0,
    mean_only_expressed: 'bool' = False,
    cmap: 'str' = 'Reds',
    dot_max: 'float | None' = None,
    dot_min: 'float | None' = None,
    standard_scale: "Literal['var', 'group'] | None" = None,
    smallest_dot: 'float | None' = 0.0,
    title: 'str | None' = None,
    colorbar_title: 'str | None' = 'Mean expression\nin group',
    size_title: 'str | None' = 'Fraction of cells\nin group (%)',
    figsize: 'tuple[float, float] | None' = None,
    dendrogram: 'bool | str' = False,
    gene_symbols: 'str | None' = None,
    var_group_positions: 'Sequence[tuple[int, int]] | None' = None,
    var_group_labels: 'Sequence[str] | None' = None,
    var_group_rotation: 'float | None' = None,
    layer: 'str | None' = None,
    swap_axes: 'bool | None' = False,
    dot_color_df: 'pd.DataFrame | None' = None,
    show: 'bool | None' = None,
    save: 'str | bool | None' = None,
    ax: '_AxesSubplot | None' = None,
    return_fig: 'bool | None' = False,
    vmin: 'float | None' = None,
    vmax: 'float | None' = None,
    vcenter: 'float | None' = None,
    norm: 'Normalize | None' = None,
    **kwds,
) -> 'DotPlot | dict | None'
Docstring:
Makes a *dot plot* of the expression values of `var_names`.

For each var_name and each `groupby` category a dot is plotted.
Each dot represents two values: mean expression within each category
(visualized by color) and fraction of cells expressing the `var_name` in the
category (visualized by the size of the dot). If `groupby` is not given,
the dotplot assumes that all data belongs to a single category.

.. note::
   A gene is considered expressed if the expression value in the `adata` (or
   `adata.raw`) is above the specified threshold which is zero by default.

An example of dotplot usage is to visualize, for multiple marker genes,
the mean value and the percentage of cells expressing the gene
across  multiple clusters.

This function provides a convenient interface to the :class:`~scanpy.pl.DotPlot`
class. If you need more flexibility, you should use :class:`~scanpy.pl.DotPlot`
directly.

Parameters
----------
adata : 'AnnData'
    Annotated data matrix.
var_names : '_VarNames | Mapping[str, _VarNames]'
    `var_names` should be a valid subset of `adata.var_names`.
    If `var_names` is a mapping, then the key is used as label
    to group the values (see `var_group_labels`). The mapping values
    should be sequences of valid `adata.var_names`. In this
    case either coloring or 'brackets' are used for the grouping
    of var names depending on the plot. When `var_names` is a mapping,
    then the `var_group_labels` and `var_group_positions` are set.
groupby : 'str | Sequence[str]'
    The key of the observation grouping to consider.
use_raw : 'bool | None', optional (default: None)
    Use `raw` attribute of `adata` if present.
log : 'bool', optional (default: False)
    Plot on logarithmic axis.
num_categories : 'int', optional (default: 7)
    Only used if groupby observation is not categorical. This value
    determines the number of groups into which the groupby observation
    should be subdivided.
categories_order
    Order in which to show the categories. Note: add_dendrogram or add_totals
    can change the categories order.
figsize : 'tuple[float, float] | None', optional (default: None)
    Figure size when `multi_panel=True`.
    Otherwise the `rcParam['figure.figsize]` value is used.
    Format is (width, height)
dendrogram : 'bool | str', optional (default: False)
    If True or a valid dendrogram key, a dendrogram based on the hierarchical
    clustering between the `groupby` categories is added.
    The dendrogram information is computed using :func:`scanpy.tl.dendrogram`.
    If `tl.dendrogram` has not been called previously the function is called
    with default parameters.
gene_symbols : 'str | None', optional (default: None)
    Column name in `.var` DataFrame that stores gene symbols.
    By default `var_names` refer to the index column of the `.var` DataFrame.
    Setting this option allows alternative names to be used.
var_group_positions : 'Sequence[tuple[int, int]] | None', optional (default: None)
    Use this parameter to highlight groups of `var_names`.
    This will draw a 'bracket' or a color block between the given start and end
    positions. If the parameter `var_group_labels` is set, the corresponding
    labels are added on top/left. E.g. `var_group_positions=[(4,10)]`
    will add a bracket between the fourth `var_name` and the tenth `var_name`.
    By giving more positions, more brackets/color blocks are drawn.
var_group_labels : 'Sequence[str] | None', optional (default: None)
    Labels for each of the `var_group_positions` that want to be highlighted.
var_group_rotation : 'float | None', optional (default: None)
    Label rotation degrees.
    By default, labels larger than 4 characters are rotated 90 degrees.
layer : 'str | None', optional (default: None)
    Name of the AnnData object layer that wants to be plotted. By default adata.raw.X is plotted.
    If `use_raw=False` is set, then `adata.X` is plotted. If `layer` is set to a valid layer name,
    then the layer is plotted. `layer` takes precedence over `use_raw`.
title : 'str | None', optional (default: None)
    Title for the figure
colorbar_title : 'str | None', optional (default: 'Mean expression\nin group')
    Title for the color bar. New line character (\n) can be used.
cmap : 'str', optional (default: 'Reds')
    String denoting matplotlib color map.
standard_scale : "Literal['var', 'group'] | None", optional (default: None)
    Whether or not to standardize the given dimension between 0 and 1, meaning for
    each variable or group, subtract the minimum and divide each by its maximum.
swap_axes : 'bool | None', optional (default: False)
     By default, the x axis contains `var_names` (e.g. genes) and the y axis
     the `groupby` categories. By setting `swap_axes` then x are the
     `groupby` categories and y the `var_names`.
return_fig : 'bool | None', optional (default: False)
    Returns :class:`DotPlot` object. Useful for fine-tuning
    the plot. Takes precedence over `show=False`.

size_title : 'str | None', optional (default: 'Fraction of cells\nin group (%)')
    Title for the size legend. New line character (\n) can be used.
expression_cutoff : 'float', optional (default: 0.0)
    Expression cutoff that is used for binarizing the gene expression and
    determining the fraction of cells expressing given genes. A gene is
    expressed only if the expression value is greater than this threshold.
mean_only_expressed : 'bool', optional (default: False)
    If True, gene expression is averaged only over the cells
    expressing the given genes.
dot_max : 'float | None', optional (default: None)
    If none, the maximum dot size is set to the maximum fraction value found
    (e.g. 0.6). If given, the value should be a number between 0 and 1.
    All fractions larger than dot_max are clipped to this value.
dot_min : 'float | None', optional (default: None)
    If none, the minimum dot size is set to 0. If given,
    the value should be a number between 0 and 1.
    All fractions smaller than dot_min are clipped to this value.
smallest_dot : 'float | None', optional (default: 0.0)
    If none, the smallest dot has size 0.
    All expression levels with `dot_min` are plotted with this size.
show : 'bool | None', optional (default: None)
     Show the plot, do not return axis.
save : 'str | bool | None', optional (default: None)
    If `True` or a `str`, save the figure.
    A string is appended to the default filename.
    Infer the filetype if ending on {`'.pdf'`, `'.png'`, `'.svg'`}.
ax : '_AxesSubplot | None', optional (default: None)
    A matplotlib axes object. Only works if plotting a single component.
vmin : 'float | None', optional (default: None)
    The value representing the lower limit of the color scale. Values smaller than vmin are plotted
    with the same color as vmin.
vmax : 'float | None', optional (default: None)
    The value representing the upper limit of the color scale. Values larger than vmax are plotted
    with the same color as vmax.
vcenter : 'float | None', optional (default: None)
    The value representing the center of the color scale. Useful for diverging colormaps.
norm : 'Normalize | None', optional (default: None)
    Custom color normalization object from matplotlib. See
    `https://matplotlib.org/stable/tutorials/colors/colormapnorms.html` for details.
kwds : _empty
    Are passed to :func:`matplotlib.pyplot.scatter`.

Returns
-------
If `return_fig` is `True`, returns a :class:`~scanpy.pl.DotPlot` object,
else if `show` is false, return axes dict

See also
--------
:class:`~scanpy.pl.DotPlot`: The DotPlot class can be used to to control
    several visual parameters not available in this function.
:func:`~scanpy.pl.rank_genes_groups_dotplot`: to plot marker genes
    identified using the :func:`~scanpy.tl.rank_genes_groups` function.

Examples
--------

Create a dot plot using the given markers and the PBMC example dataset grouped by
the category 'bulk_labels'.

.. plot::
    :context: close-figs

    import scanpy as sc
    adata = sc.datasets.pbmc68k_reduced()
    markers = ['C1QA', 'PSAP', 'CD79A', 'CD79B', 'CST3', 'LYZ']
    sc.pl.dotplot(adata, markers, groupby='bulk_labels', dendrogram=True)

Using var_names as dict:

.. plot::
    :context: close-figs

    markers = {'T-cell': 'CD3D', 'B-cell': 'CD79A', 'myeloid': 'CST3'}
    sc.pl.dotplot(adata, markers, groupby='bulk_labels', dendrogram=True)

Get DotPlot object for fine tuning

.. plot::
    :context: close-figs

    dp = sc.pl.dotplot(adata, markers, 'bulk_labels', return_fig=True)
    dp.add_totals().style(dot_edge_color='black', dot_edge_lw=0.5).show()

The axes used can be obtained using the get_axes() method

.. code-block:: python

    axes_dict = dp.get_axes()
    print(axes_dict)
File:      /software/cellgen/team205/kk18/envs/generic_env/lib/python3.9/site-packages/scanpy/plotting/_dotplot.py
Type:      function
```

Example 71 (python):
```python
genes = ['TBX3','TBX18',
         'GJA1', # CX43
         'GJA5', # CX40
         'NKX2-5']
genes = [x for x in genes if x in san.var_names]
sc.tl.dendrogram(san,groupby='fine_grain_manual')
sc.pl.dotplot(san,
              var_names=genes,
              groupby='fine_grain_manual',
              standard_scale='var',
              dendrogram=True,show=False
             )
plt.savefig(f'{figdir}/VisiumHD_euploid_SANPC_dotplot-markers.pdf', bbox_inches="tight",dpi=300)
```

Example 72 (unknown):
```unknown
maxp pruned
cmap pruned
kern dropped
post pruned
FFTM dropped
GPOS pruned
GSUB pruned
glyf pruned
Added gid0 to subset
Added first four glyphs to subset
Closing glyph list over 'MATH': 47 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'B', 'C', 'F', 'G', 'H', 'J', 'K', 'M', 'N', 'P', 'S', 'T', 'X', 'a', 'c', 'd', 'e', 'eight', 'f', 'five', 'g', 'hyphen', 'i', 'k', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'one', 'p', 'parenleft', 'parenright', 'percent', 'r', 's', 'seven', 'space', 't', 'three', 'two', 'u', 'x', 'zero']
Glyph IDs:   [0, 1, 2, 3, 8, 11, 12, 16, 19, 20, 21, 22, 24, 26, 27, 36, 37, 38, 41, 42, 43, 45, 46, 48, 49, 51, 54, 55, 59, 68, 70, 71, 72, 73, 74, 76, 78, 79, 80, 81, 82, 83, 85, 86, 87, 88, 91]
Closed glyph list over 'MATH': 53 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'B', 'C', 'F', 'G', 'H', 'J', 'K', 'M', 'N', 'P', 'S', 'T', 'X', 'a', 'c', 'd', 'e', 'eight', 'f', 'five', 'g', 'hyphen', 'i', 'k', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'one', 'p', 'parenleft', 'parenright', 'percent', 'r', 's', 'seven', 'space', 't', 'three', 'two', 'u', 'uni239B', 'uni239C', 'uni239D', 'uni239E', 'uni239F', 'uni23A0', 'x', 'zero']
Glyph IDs:   [0, 1, 2, 3, 8, 11, 12, 16, 19, 20, 21, 22, 24, 26, 27, 36, 37, 38, 41, 42, 43, 45, 46, 48, 49, 51, 54, 55, 59, 68, 70, 71, 72, 73, 74, 76, 78, 79, 80, 81, 82, 83, 85, 86, 87, 88, 91, 3506, 3507, 3508, 3509, 3510, 3511]
Closing glyph list over 'GSUB': 53 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'B', 'C', 'F', 'G', 'H', 'J', 'K', 'M', 'N', 'P', 'S', 'T', 'X', 'a', 'c', 'd', 'e', 'eight', 'f', 'five', 'g', 'hyphen', 'i', 'k', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'one', 'p', 'parenleft', 'parenright', 'percent', 'r', 's', 'seven', 'space', 't', 'three', 'two', 'u', 'uni239B', 'uni239C', 'uni239D', 'uni239E', 'uni239F', 'uni23A0', 'x', 'zero']
Glyph IDs:   [0, 1, 2, 3, 8, 11, 12, 16, 19, 20, 21, 22, 24, 26, 27, 36, 37, 38, 41, 42, 43, 45, 46, 48, 49, 51, 54, 55, 59, 68, 70, 71, 72, 73, 74, 76, 78, 79, 80, 81, 82, 83, 85, 86, 87, 88, 91, 3506, 3507, 3508, 3509, 3510, 3511]
Closed glyph list over 'GSUB': 58 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'B', 'C', 'F', 'G', 'H', 'J', 'K', 'M', 'N', 'P', 'S', 'T', 'X', 'a', 'c', 'd', 'e', 'eight', 'f', 'fi', 'five', 'fl', 'g', 'hyphen', 'i', 'k', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'one', 'p', 'parenleft', 'parenright', 'percent', 'r', 's', 'seven', 'space', 't', 'three', 'two', 'u', 'uni239B', 'uni239C', 'uni239D', 'uni239E', 'uni239F', 'uni23A0', 'uniFB00', 'uniFB03', 'uniFB04', 'x', 'zero']
Glyph IDs:   [0, 1, 2, 3, 8, 11, 12, 16, 19, 20, 21, 22, 24, 26, 27, 36, 37, 38, 41, 42, 43, 45, 46, 48, 49, 51, 54, 55, 59, 68, 70, 71, 72, 73, 74, 76, 78, 79, 80, 81, 82, 83, 85, 86, 87, 88, 91, 3506, 3507, 3508, 3509, 3510, 3511, 5038, 5039, 5040, 5041, 5042]
Closing glyph list over 'glyf': 58 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'B', 'C', 'F', 'G', 'H', 'J', 'K', 'M', 'N', 'P', 'S', 'T', 'X', 'a', 'c', 'd', 'e', 'eight', 'f', 'fi', 'five', 'fl', 'g', 'hyphen', 'i', 'k', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'one', 'p', 'parenleft', 'parenright', 'percent', 'r', 's', 'seven', 'space', 't', 'three', 'two', 'u', 'uni239B', 'uni239C', 'uni239D', 'uni239E', 'uni239F', 'uni23A0', 'uniFB00', 'uniFB03', 'uniFB04', 'x', 'zero']
Glyph IDs:   [0, 1, 2, 3, 8, 11, 12, 16, 19, 20, 21, 22, 24, 26, 27, 36, 37, 38, 41, 42, 43, 45, 46, 48, 49, 51, 54, 55, 59, 68, 70, 71, 72, 73, 74, 76, 78, 79, 80, 81, 82, 83, 85, 86, 87, 88, 91, 3506, 3507, 3508, 3509, 3510, 3511, 5038, 5039, 5040, 5041, 5042]
Closed glyph list over 'glyf': 58 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'B', 'C', 'F', 'G', 'H', 'J', 'K', 'M', 'N', 'P', 'S', 'T', 'X', 'a', 'c', 'd', 'e', 'eight', 'f', 'fi', 'five', 'fl', 'g', 'hyphen', 'i', 'k', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'one', 'p', 'parenleft', 'parenright', 'percent', 'r', 's', 'seven', 'space', 't', 'three', 'two', 'u', 'uni239B', 'uni239C', 'uni239D', 'uni239E', 'uni239F', 'uni23A0', 'uniFB00', 'uniFB03', 'uniFB04', 'x', 'zero']
Glyph IDs:   [0, 1, 2, 3, 8, 11, 12, 16, 19, 20, 21, 22, 24, 26, 27, 36, 37, 38, 41, 42, 43, 45, 46, 48, 49, 51, 54, 55, 59, 68, 70, 71, 72, 73, 74, 76, 78, 79, 80, 81, 82, 83, 85, 86, 87, 88, 91, 3506, 3507, 3508, 3509, 3510, 3511, 5038, 5039, 5040, 5041, 5042]
Retaining 58 glyphs
head subsetting not needed
hhea subsetting not needed
maxp subsetting not needed
OS/2 subsetting not needed
hmtx subsetted
cmap subsetted
fpgm subsetting not needed
prep subsetting not needed
cvt  subsetting not needed
loca subsetting not needed
post subsetted
gasp subsetting not needed
MATH subsetted
GDEF subsetted
GPOS subsetted
GSUB subsetted
name subsetting not needed
glyf subsetted
head pruned
OS/2 Unicode ranges pruned: [0]
OS/2 CodePage ranges pruned: [0]
glyf pruned
GDEF pruned
GPOS pruned
GSUB pruned
name pruned
```

---

## Import modules¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/RNA/QC/Foetal_Multiome-RNA_No2_QC-filter.html

**Contents:**
- Import modules¶
- Functions¶
- Read in anndata¶
- Remove doublets¶
- Remove low quality cells¶
- Compute QC stats¶
- Remove cells exceeding the ribo/mito threshold¶
- Calculate RBC score¶
- Remove RBCs¶
- Remove donor-unassigned nuclei in multiplexed samples¶
- Update metadata for the multiplexed samples¶
- Save post-filtered anndata¶
- Just checking¶

**Examples:**

Example 1 (python):
```python
import numpy as np
import pandas as pd
import scanpy as sc
import matplotlib.pyplot as plt
import scrublet as scr
import session_info
```

Example 2 (python):
```python
sc.settings.verbosity = 3
sc.settings.set_figure_params(dpi = 160, color_map = 'RdPu', dpi_save = 180, vector_friendly = True, format = 'svg')
session_info.show()
```

Example 3 (python):
```python
-----
matplotlib          3.3.4
numpy               1.20.1
pandas              1.1.4
scanpy              1.7.1
scrublet            NA
session_info        1.0.0
-----
```

Example 4 (unknown):
```unknown
PIL                 8.1.2
anndata             0.7.5
anyio               NA
attr                20.3.0
babel               2.9.0
backcall            0.2.0
brotli              NA
cairo               1.20.0
certifi             2020.12.05
cffi                1.14.5
chardet             4.0.0
cloudpickle         1.6.0
colorama            0.4.4
cycler              0.10.0
cython_runtime      NA
cytoolz             0.11.0
dask                2021.03.1
dateutil            2.8.1
decorator           4.4.2
fsspec              0.8.7
get_version         2.1
google              NA
h5py                2.10.0
idna                2.10
igraph              0.8.3
ipykernel           5.5.0
ipython_genutils    0.2.0
jedi                0.18.0
jinja2              2.11.3
joblib              1.0.1
json5               NA
jsonschema          3.2.0
jupyter_server      1.4.1
jupyterlab_server   2.3.0
kiwisolver          1.3.1
legacy_api_wrap     0.0.0
leidenalg           0.8.3
llvmlite            0.34.0
louvain             0.7.0
markupsafe          1.1.1
mpl_toolkits        NA
natsort             7.1.1
nbclassic           NA
nbformat            5.1.2
numba               0.51.2
numexpr             2.7.3
packaging           20.9
parso               0.8.1
pexpect             4.8.0
pickleshare         0.7.5
pkg_resources       NA
prometheus_client   NA
prompt_toolkit      3.0.16
psutil              5.8.0
ptyprocess          0.7.0
pvectorc            NA
pyarrow             0.16.0
pygments            2.8.0
pyparsing           2.4.7
pyrsistent          NA
pytoml              NA
pytz                2021.1
requests            2.25.1
ruamel              NA
scipy               1.6.1
send2trash          NA
setuptools_scm      NA
simplejson          3.17.2
sinfo               0.3.1
six                 1.15.0
sklearn             0.22
sniffio             1.2.0
socks               1.7.1
sphinxcontrib       NA
storemagic          NA
tables              3.6.1
tblib               1.7.0
texttable           1.6.3
tlz                 0.11.0
toolz               0.11.1
tornado             6.1
traitlets           5.0.5
typing_extensions   NA
uritemplate         3.0.1
urllib3             1.26.3
wcwidth             0.2.5
yaml                5.1.2
zmq                 22.0.3
```

Example 5 (unknown):
```unknown
-----
IPython             7.21.0
jupyter_client      6.1.11
jupyter_core        4.7.1
jupyterlab          3.0.9
notebook            6.2.0
-----
Python 3.8.8 | packaged by conda-forge | (default, Feb 20 2021, 16:22:27) [GCC 9.3.0]
Linux-4.15.0-112-generic-x86_64-with-glibc2.10
-----
Session information updated at 2023-02-08 22:44
```

Example 6 (python):
```python
adata = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/QC/multiome_RNA_foetal_prefilter.h5ad')
adata
```

Example 7 (unknown):
```unknown
AnnData object with n_obs × n_vars = 721979 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval'
    var: 'gene_name', 'feature_type', 'ambient_expression-0', 'ambient_expression-1', 'ambient_expression-10', 'ambient_expression-11', 'ambient_expression-12', 'ambient_expression-13', 'ambient_expression-14', 'ambient_expression-15', 'ambient_expression-16', 'ambient_expression-17', 'ambient_expression-18', 'ambient_expression-19', 'ambient_expression-2', 'ambient_expression-20', 'ambient_expression-21', 'ambient_expression-22', 'ambient_expression-23', 'ambient_expression-24', 'ambient_expression-25', 'ambient_expression-26', 'ambient_expression-27', 'ambient_expression-28', 'ambient_expression-29', 'ambient_expression-3', 'ambient_expression-30', 'ambient_expression-31', 'ambient_expression-32', 'ambient_expression-33', 'ambient_expression-34', 'ambient_expression-35', 'ambient_expression-36', 'ambient_expression-37', 'ambient_expression-38', 'ambient_expression-39', 'ambient_expression-4', 'ambient_expression-5', 'ambient_expression-6', 'ambient_expression-7', 'ambient_expression-8', 'ambient_expression-9'
    obsm: 'latent_gene_encoding'
```

Example 8 (python):
```python
adata.obs['combinedID'].value_counts()
```

Example 9 (unknown):
```unknown
BHF_F_Hea13188316_BHF_F_Hea13187618      59866
7089STDY13216924_BHF_F_Hea13242531       40000
BHF_F_Hea11933665_BHF_F_Hea11596618      33997
7089STDY13216925_BHF_F_Hea13242532       30453
BHF_F_Hea13188318_BHF_F_Hea13187620      27856
7089STDY13216920_BHF_F_Hea13242527       27498
BHF_F_Hea13168898_BHF_F_Hea13168514      24781
7089STDY13216927_BHF_F_Hea13242534       24666
7089STDY13216926_BHF_F_Hea13242533       24602
BHF_F_Hea13188324_BHF_F_Hea13187626      24381
7089STDY13216921_BHF_F_Hea13242528       24211
BHF_F_Hea13168896_BHF_F_Hea13168512      22267
HCAHeartST13386009_HCAHeartST13303419    21577
BHF_F_Hea11933666_BHF_F_Hea11596619      21509
BHF_F_Hea13168899_BHF_F_Hea13168515      21418
7089STDY13216922_BHF_F_Hea13242529       21402
BHF_F_Hea13188317_BHF_F_Hea13187619      21327
BHF_F_Hea11933664_BHF_F_Hea11596617      20338
BHF_F_Hea11933667_BHF_F_Hea11596620      19811
BHF_F_Hea13168894_BHF_F_Hea13168510      19258
BHF_F_Hea11933669_BHF_F_Hea11596622      19029
BHF_F_Hea13168897_BHF_F_Hea13168513      18609
HCAHeartST13386010_HCAHeartST13303420    18350
BHF_F_Hea13168895_BHF_F_Hea13168511      18315
BHF_F_Hea11933668_BHF_F_Hea11596621      17494
BHF_F_Hea13168893_BHF_F_Hea13168509      11782
BHF_F_Hea11933670_BHF_F_Hea11596623      11068
BHF_F_Hea11933673_BHF_F_Hea11596626      10769
BHF_F_Hea11933671_BHF_F_Hea11596624      10256
BHF_F_Hea11064672_BHF_F_Hea11031825      10191
BHF_F_Hea13168892_BHF_F_Hea13168508      10023
BHF_F_Hea11933672_BHF_F_Hea11596625       9711
BHF_F_Hea11933674_BHF_F_Hea11596627       6199
BHF_F_Hea13188328_BHF_F_Hea13187630       5340
BHF_F_Hea11064671_BHF_F_Hea11031824       4757
BHF_F_Hea11064670_BHF_F_Hea11031823       2665
BHF_F_Hea13188329_BHF_F_Hea13187631       1878
7089STDY13216923_BHF_F_Hea13242530        1873
BHF_F_Hea13188330_BHF_F_Hea13187632       1750
BHF_F_Hea11933675_BHF_F_Hea11596628        702
Name: combinedID, dtype: int64
```

Example 10 (python):
```python
adata.obs['scrublet_score'].hist(bins=100)
```

Example 11 (unknown):
```unknown
<AxesSubplot:>
```

Example 12 (python):
```python
# set threshold (arbitrary)
dbl_score_thresh=0.15

prop = np.sum(adata.obs['scrublet_score']>=dbl_score_thresh)/len(adata)
print(f'proportion of doublets: {prop}')
```

Example 13 (unknown):
```unknown
proportion of doublets: 0.2872091847546812
```

Example 14 (python):
```python
# Save doublet ids
doublets = adata.obs_names[(adata.obs['scrublet_score']<dbl_score_thresh)==False]
with open('/nfs/team205/heart/anndata_objects/Foetal/QC/Multiome_rm-doublets.txt', 'w') as f:
    for line in doublets:
        f.write(line)
        f.write('\n')

# Remove doublets
adata = adata[adata.obs['scrublet_score']<dbl_score_thresh]
print(adata.shape)
```

Example 15 (unknown):
```unknown
(514620, 36601)
```

Example 16 (python):
```python
sc.pp.filter_cells(adata, min_genes=500, inplace=True)
sc.pp.filter_cells(adata, min_counts=1000, inplace=True)
adata.shape
```

Example 17 (unknown):
```unknown
filtered out 73084 cells that have less than 500 genes expressed
Trying to set attribute `.obs` of view, copying.
filtered out 41920 cells that have less than 1000 counts
```

Example 18 (unknown):
```unknown
(399616, 36601)
```

Example 19 (python):
```python
adata.var['mt'] = adata.var['gene_name'].str.startswith('MT-')  # annotate the group of mitochondrial genes as 'mt'
adata.var['ribo'] = adata.var['gene_name'].str.startswith('RPS','RPL')  # annotate the group of ribosomal genes as 'ribo'
sc.pp.calculate_qc_metrics(adata, qc_vars=['mt','ribo'], percent_top=None, log1p=True, inplace=True)
```

Example 20 (python):
```python
plt.rcParams['figure.figsize'] = [3, 2]

sc.pl.scatter(adata, x='total_counts', y='n_genes')
sc.pl.scatter(adata, x='total_counts', y='pct_counts_mt')
sc.pl.scatter(adata, x='total_counts', y='pct_counts_ribo')
```

Example 21 (python):
```python
plt.rcParams['figure.figsize'] = [2, 2]
adata = adata[adata.obs['pct_counts_mt'] < 5, :]
adata = adata[adata.obs['pct_counts_ribo'] < 5, :]
sc.pl.violin(adata, ['n_genes', 'total_counts', 'log1p_total_counts','pct_counts_mt', 'pct_counts_ribo'],jitter = 0.1, multi_panel = True)
```

Example 22 (python):
```python
sc.settings.set_figure_params(dpi = 120, color_map = 'RdPu')
plt.rcParams['figure.figsize'] = 10,3
sc.pl.violin(adata,keys=['log1p_total_counts'],groupby='sangerID',rotation=90,jitter=False)
```

Example 23 (unknown):
```unknown
Trying to set attribute `.uns` of view, copying.
```

Example 24 (python):
```python
plt.rcParams['figure.figsize'] = 10,3
sc.pl.violin(adata,keys=['pct_counts_mt'],groupby='sangerID',rotation=90,jitter=False)
```

Example 25 (python):
```python
plt.rcParams['figure.figsize'] = 10,3
sc.pl.violin(adata,keys=['pct_counts_ribo'],groupby='sangerID',rotation=90,jitter=False)
```

Example 26 (python):
```python
sc.settings.set_figure_params(dpi = 120, color_map = 'RdPu')
```

Example 27 (python):
```python
adata.shape
```

Example 28 (unknown):
```unknown
(285185, 36601)
```

Example 29 (python):
```python
adata_c = adata.copy()

adata_c.var.set_index('gene_name',inplace=True)
adata_c.var_names = adata_c.var_names.astype('str')
adata_c.var_names_make_unique()

sc.pp.filter_genes(adata_c, min_cells=3)
sc.pp.normalize_total(adata_c, target_sum=1e4)
sc.pp.log1p(adata_c)
```

Example 30 (unknown):
```unknown
filtered out 2031 genes that are detected in less than 3 cells
normalizing counts per cell
    finished (0:00:02)
```

Example 31 (python):
```python
# scoring hemoglobin genes
# HGNC, Hemoglobin subunits, groupID:940
genes=['HBA1','HBA2','HBB','HBD','HBE1','HBG1','HBG2','HBM','HBQ1','HBZ']
sc.tl.score_genes(adata_c,gene_list=genes,score_name='HB_score')
```

Example 32 (python):
```python
computing score 'HB_score'
    finished: added
    'HB_score', score of gene set (adata.obs).
    399 total control genes are used. (0:00:10)
```

Example 33 (python):
```python
len(adata_c.obs['sangerID'].unique())
```

Example 34 (python):
```python
sc.settings.set_figure_params(dpi = 80, color_map = 'RdPu', dpi_save = 180, vector_friendly = True, format = 'svg')

fig,axs=plt.subplots(nrows=8,ncols=5,figsize=(15,15))

for i,ID in enumerate(adata_c.obs['sangerID'].unique()):
    sample = adata_c[adata_c.obs['sangerID']==ID]
    axs[i//5,i%5].hist(sample.obs['HB_score'],bins=100)
    axs[i//5,i%5].set_title(ID)

fig.tight_layout()
plt.show()
```

Example 35 (python):
```python
# add HB score to original adata
adata.obs.loc[adata_c.obs_names,'HB_score']=adata_c.obs['HB_score']
```

Example 36 (python):
```python
HBscore_thresh=1

adata=adata[adata.obs['HB_score']<HBscore_thresh]
print(adata.shape)
```

Example 37 (unknown):
```unknown
(274033, 36601)
```

Example 38 (python):
```python
adata.obs['combinedID'].value_counts()
```

Example 39 (unknown):
```unknown
BHF_F_Hea13188316_BHF_F_Hea13187618      21518
BHF_F_Hea13168898_BHF_F_Hea13168514      13895
7089STDY13216920_BHF_F_Hea13242527       12771
BHF_F_Hea13168899_BHF_F_Hea13168515      12369
BHF_F_Hea11933665_BHF_F_Hea11596618      11830
BHF_F_Hea13168896_BHF_F_Hea13168512      11191
BHF_F_Hea11933664_BHF_F_Hea11596617      11022
7089STDY13216921_BHF_F_Hea13242528       10826
BHF_F_Hea11933666_BHF_F_Hea11596619      10463
BHF_F_Hea11933667_BHF_F_Hea11596620      10340
BHF_F_Hea13168894_BHF_F_Hea13168510       9994
BHF_F_Hea11933669_BHF_F_Hea11596622       9539
BHF_F_Hea11933668_BHF_F_Hea11596621       9490
BHF_F_Hea13168895_BHF_F_Hea13168511       9431
BHF_F_Hea13168897_BHF_F_Hea13168513       9315
7089STDY13216924_BHF_F_Hea13242531        8598
BHF_F_Hea13188318_BHF_F_Hea13187620       7474
BHF_F_Hea13168893_BHF_F_Hea13168509       6747
BHF_F_Hea13188324_BHF_F_Hea13187626       6266
7089STDY13216927_BHF_F_Hea13242534        6061
HCAHeartST13386010_HCAHeartST13303420     6015
BHF_F_Hea13168892_BHF_F_Hea13168508       5651
7089STDY13216922_BHF_F_Hea13242529        5382
BHF_F_Hea13188317_BHF_F_Hea13187619       5210
HCAHeartST13386009_HCAHeartST13303419     4833
BHF_F_Hea11064672_BHF_F_Hea11031825       4499
7089STDY13216926_BHF_F_Hea13242533        4474
BHF_F_Hea11933673_BHF_F_Hea11596626       4169
BHF_F_Hea11933671_BHF_F_Hea11596624       3861
BHF_F_Hea11933672_BHF_F_Hea11596625       3858
BHF_F_Hea11933670_BHF_F_Hea11596623       3491
7089STDY13216925_BHF_F_Hea13242532        3475
BHF_F_Hea11064671_BHF_F_Hea11031824       2808
BHF_F_Hea11933674_BHF_F_Hea11596627       2777
BHF_F_Hea13188328_BHF_F_Hea13187630       1489
BHF_F_Hea13188329_BHF_F_Hea13187631       1085
BHF_F_Hea13188330_BHF_F_Hea13187632        837
BHF_F_Hea11064670_BHF_F_Hea11031823        611
BHF_F_Hea11933675_BHF_F_Hea11596628        262
7089STDY13216923_BHF_F_Hea13242530         106
Name: combinedID, dtype: int64
```

Example 40 (python):
```python
# read in SOC-CMO result
demul = pd.read_csv('/nfs/team205/heart/mapped/cellranger-arc200/SOC_out/SOC-CMO-assignment_multiome-foetal_20230207.csv',
                   index_col=0)
print(demul.shape)
print(demul['SOC-CMO_assignment'].value_counts())
demul.head()
```

Example 41 (unknown):
```unknown
(74984, 4)
Hst39               36363
Hst40               21936
donor_unassigned    11437
Hst36                5248
Name: SOC-CMO_assignment, dtype: int64
```

Example 42 (python):
```python
# just checking
# multiplexed samples which passed through QC-filtering above
multiplex_obsnames = adata.obs_names[adata.obs['combinedID'].isin(['7089STDY13216920_BHF_F_Hea13242527',
                                                                   '7089STDY13216921_BHF_F_Hea13242528',
                                                                   '7089STDY13216922_BHF_F_Hea13242529',
                                                                   '7089STDY13216923_BHF_F_Hea13242530'])]
print(demul.loc[multiplex_obsnames].shape)
print(demul.loc[multiplex_obsnames]['SOC-CMO_assignment'].value_counts())

# make 'donor_by_libraryprep' column
adata.obs['donor_by_library-prep'] = adata.obs['donor'].copy()
# make 'multiplexed' column
adata.obs['multiplexed'] = 'No'
adata.obs.loc[multiplex_obsnames,'multiplexed'] = 'Yes'

# replace 'donor' column for multiplexed samples
adata.obs['donor'] = adata.obs['donor'].astype('str')
adata.obs.loc[multiplex_obsnames,'donor'] = demul.loc[multiplex_obsnames,'SOC-CMO_assignment']

# add some SOC metrices
cols_to_add = ['SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet']
adata.obs[cols_to_add] = demul[cols_to_add].loc[multiplex_obsnames].reindex(adata.obs_names)

# check
print(adata.obs.loc[multiplex_obsnames]['donor'].value_counts())
print(adata.shape)
```

Example 43 (unknown):
```unknown
Trying to set attribute `.obs` of view, copying.
```

Example 44 (unknown):
```unknown
(29085, 4)
Hst39               17519
Hst40                8084
Hst36                1787
donor_unassigned     1695
Name: SOC-CMO_assignment, dtype: int64
Hst39               17519
Hst40                8084
Hst36                1787
donor_unassigned     1695
Name: donor, dtype: int64
(274033, 36601)
```

Example 45 (python):
```python
# remove unassigned nuclei
adata = adata[adata.obs['donor'] != 'donor_unassigned']
print(adata.shape)
```

Example 46 (unknown):
```unknown
(272338, 36601)
```

Example 47 (python):
```python
# to see which column needs to be updated
adata.obs[adata.obs['multiplexed']=='Yes'][['sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender']].head(n=3)
```

Example 48 (python):
```python
adata.obs[adata.obs['multiplexed']=='Yes'][['sangerID','donor','region']].value_counts()
```

Example 49 (unknown):
```unknown
sangerID          donor  region                                                       
7089STDY13216920  Hst39  Aorta (Hst39), Nodes (Hst40)                                     8819
7089STDY13216921  Hst39  Aorta (Hst39), Nodes (Hst40)                                     7402
7089STDY13216920  Hst40  Aorta (Hst39), Nodes (Hst40)                                     3321
7089STDY13216921  Hst40  Aorta (Hst39), Nodes (Hst40)                                     2934
7089STDY13216922  Hst40  Heart (Hst36), Heart-wo-nodes (Hst39), Heart-wo-nodes (Hst40)    1828
                  Hst36  Heart (Hst36), Heart-wo-nodes (Hst39), Heart-wo-nodes (Hst40)    1786
                  Hst39  Heart (Hst36), Heart-wo-nodes (Hst39), Heart-wo-nodes (Hst40)    1298
7089STDY13216923  Hst40  Heart (Hst36), Heart-wo-nodes (Hst39), Heart-wo-nodes (Hst40)       1
                  Hst36  Heart (Hst36), Heart-wo-nodes (Hst39), Heart-wo-nodes (Hst40)       1
dtype: int64
```

Example 50 (python):
```python
# prepare metadata dictionary
age_gender = {
    'Hst36': ['PCW-12w','Female'],
    'Hst39': ['PCW-15w','Female'],
    'Hst40': ['PCW-14w','Male'],
}

regions = {
    '7089STDY13216920_Hst39':'Aorta',
    '7089STDY13216920_Hst40':'Nodes',
    '7089STDY13216921_Hst39':'Aorta',
    '7089STDY13216921_Hst40':'Nodes',
    '7089STDY13216922_Hst36':'Heart',
    '7089STDY13216922_Hst39':'Heart-wo-nodes',
    '7089STDY13216922_Hst40':'Heart-wo-nodes',
    '7089STDY13216923_Hst36':'Heart',
    '7089STDY13216923_Hst39':'Heart-wo-nodes',
    '7089STDY13216923_Hst40':'Heart-wo-nodes',
}
```

Example 51 (python):
```python
# update
SangerIDs = list(set(adata.obs[adata.obs['multiplexed']=='Yes']['sangerID']))
Donors = list(set(adata.obs[adata.obs['multiplexed']=='Yes']['donor']))

for sID in SangerIDs:
    for d in Donors:
        obsnames_to_update = adata.obs_names[(adata.obs['sangerID']==sID)&(adata.obs['donor']==d)]
        print(f'{sID}, {d}: {len(obsnames_to_update)} nuclei')
        # update
        if len(obsnames_to_update)>0:
            adata.obs.loc[obsnames_to_update,'region'] = regions[f'{sID}_{d}']
            adata.obs.loc[obsnames_to_update,'region_finest'] = regions[f'{sID}_{d}']
            adata.obs.loc[obsnames_to_update,'age'] = age_gender[d][0]
            adata.obs.loc[obsnames_to_update,'gender'] = age_gender[d][1]
        else:
            pass
        del obsnames_to_update
```

Example 52 (unknown):
```unknown
7089STDY13216921, Hst36: 0 nuclei
7089STDY13216921, Hst40: 2934 nuclei
7089STDY13216921, Hst39: 7402 nuclei
7089STDY13216922, Hst36: 1786 nuclei
7089STDY13216922, Hst40: 1828 nuclei
7089STDY13216922, Hst39: 1298 nuclei
7089STDY13216923, Hst36: 1 nuclei
7089STDY13216923, Hst40: 1 nuclei
7089STDY13216923, Hst39: 0 nuclei
7089STDY13216920, Hst36: 0 nuclei
7089STDY13216920, Hst40: 3321 nuclei
7089STDY13216920, Hst39: 8819 nuclei
```

Example 53 (python):
```python
# save
adata.write('/nfs/team205/heart/anndata_objects/Foetal/QC/multiome_RNA_foetal.h5ad')
adata
```

Example 54 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/anndata/_core/anndata.py:1207: ImplicitModificationWarning: Initializing view as actual.
  warnings.warn(
Trying to set attribute `.obs` of view, copying.
... storing 'donor' as categorical
Trying to set attribute `.obs` of view, copying.
... storing 'multiplexed' as categorical
Trying to set attribute `.obs` of view, copying.
... storing 'SOC | status' as categorical
```

Example 55 (unknown):
```unknown
AnnData object with n_obs × n_vars = 272338 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet'
    var: 'gene_name', 'feature_type', 'ambient_expression-0', 'ambient_expression-1', 'ambient_expression-10', 'ambient_expression-11', 'ambient_expression-12', 'ambient_expression-13', 'ambient_expression-14', 'ambient_expression-15', 'ambient_expression-16', 'ambient_expression-17', 'ambient_expression-18', 'ambient_expression-19', 'ambient_expression-2', 'ambient_expression-20', 'ambient_expression-21', 'ambient_expression-22', 'ambient_expression-23', 'ambient_expression-24', 'ambient_expression-25', 'ambient_expression-26', 'ambient_expression-27', 'ambient_expression-28', 'ambient_expression-29', 'ambient_expression-3', 'ambient_expression-30', 'ambient_expression-31', 'ambient_expression-32', 'ambient_expression-33', 'ambient_expression-34', 'ambient_expression-35', 'ambient_expression-36', 'ambient_expression-37', 'ambient_expression-38', 'ambient_expression-39', 'ambient_expression-4', 'ambient_expression-5', 'ambient_expression-6', 'ambient_expression-7', 'ambient_expression-8', 'ambient_expression-9', 'mt', 'ribo', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts'
    uns: 'sangerID_colors'
    obsm: 'latent_gene_encoding'
```

Example 56 (python):
```python
adata.obs['combinedID'].value_counts()
```

Example 57 (unknown):
```unknown
BHF_F_Hea13188316_BHF_F_Hea13187618      21518
BHF_F_Hea13168898_BHF_F_Hea13168514      13895
BHF_F_Hea13168899_BHF_F_Hea13168515      12369
7089STDY13216920_BHF_F_Hea13242527       12140
BHF_F_Hea11933665_BHF_F_Hea11596618      11830
BHF_F_Hea13168896_BHF_F_Hea13168512      11191
BHF_F_Hea11933664_BHF_F_Hea11596617      11022
BHF_F_Hea11933666_BHF_F_Hea11596619      10463
BHF_F_Hea11933667_BHF_F_Hea11596620      10340
7089STDY13216921_BHF_F_Hea13242528       10336
BHF_F_Hea13168894_BHF_F_Hea13168510       9994
BHF_F_Hea11933669_BHF_F_Hea11596622       9539
BHF_F_Hea11933668_BHF_F_Hea11596621       9490
BHF_F_Hea13168895_BHF_F_Hea13168511       9431
BHF_F_Hea13168897_BHF_F_Hea13168513       9315
7089STDY13216924_BHF_F_Hea13242531        8598
BHF_F_Hea13188318_BHF_F_Hea13187620       7474
BHF_F_Hea13168893_BHF_F_Hea13168509       6747
BHF_F_Hea13188324_BHF_F_Hea13187626       6266
7089STDY13216927_BHF_F_Hea13242534        6061
HCAHeartST13386010_HCAHeartST13303420     6015
BHF_F_Hea13168892_BHF_F_Hea13168508       5651
BHF_F_Hea13188317_BHF_F_Hea13187619       5210
7089STDY13216922_BHF_F_Hea13242529        4912
HCAHeartST13386009_HCAHeartST13303419     4833
BHF_F_Hea11064672_BHF_F_Hea11031825       4499
7089STDY13216926_BHF_F_Hea13242533        4474
BHF_F_Hea11933673_BHF_F_Hea11596626       4169
BHF_F_Hea11933671_BHF_F_Hea11596624       3861
BHF_F_Hea11933672_BHF_F_Hea11596625       3858
BHF_F_Hea11933670_BHF_F_Hea11596623       3491
7089STDY13216925_BHF_F_Hea13242532        3475
BHF_F_Hea11064671_BHF_F_Hea11031824       2808
BHF_F_Hea11933674_BHF_F_Hea11596627       2777
BHF_F_Hea13188328_BHF_F_Hea13187630       1489
BHF_F_Hea13188329_BHF_F_Hea13187631       1085
BHF_F_Hea13188330_BHF_F_Hea13187632        837
BHF_F_Hea11064670_BHF_F_Hea11031823        611
BHF_F_Hea11933675_BHF_F_Hea11596628        262
7089STDY13216923_BHF_F_Hea13242530           2
Name: combinedID, dtype: int64
```

Example 58 (python):
```python
adata.obs['donor'].value_counts()
```

Example 59 (unknown):
```unknown
Hst41    129909
Hst33     62684
Hst39     30591
C92       15379
Hst40     11559
Hst36      7848
C87        4499
C104       3411
C98        3039
C85        2808
C82         611
Name: donor, dtype: int64
```

Example 60 (python):
```python
pd.crosstab(adata.obs['donor'],adata.obs['age'])
```

Example 61 (python):
```python
pd.crosstab(adata.obs['donor'],adata.obs['gender'])
```

Example 62 (python):
```python
adata.obs['SOC | status'].value_counts()
```

Example 63 (javascript):
```javascript
nan           244948
singlet        27068
doublet          246
unassigned        76
Name: SOC | status, dtype: int64
```

Example 64 (python):
```python
adata.obs[adata.obs['multiplexed']=='Yes']['donor_by_library-prep'].value_counts()
```

Example 65 (unknown):
```unknown
Hst39, Hst40           22476
Hst36, Hst39, Hst40     4914
Hst41                      0
Hst40                      0
Hst39                      0
Hst36                      0
Hst33                      0
C104                       0
C98                        0
C92                        0
C87                        0
C85                        0
C82                        0
Name: donor_by_library-prep, dtype: int64
```

---

## Function¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig4_Cardiomyocytes/VentricularCardiomyocytes/no1_compact-vs-trab_gene-signatures.html

**Contents:**
- Function¶
- Read in adata¶
- DEG analysis: compact-vs-trab at the end stage¶
- Read in associated genes¶
- Select associated genes which were highest in the group_a or group_b¶
- Associated genes¶
- Select specific and common ones¶
- Save the genes¶
- Scoring and plot¶

i.e. separating into UP or DOWN regulated genes

**Examples:**

Example 1 (python):
```python
# Parameters
working_dir = "/nfs/team205/heart/Vitro/TF_selection"
path_to_adata_gex_dir = "/nfs/team205/heart/anndata_objects/Vitro/post_multiVI_full-features"
group_key = "t_binned"
group_a = "t3"
group_b = "t1"
time_key = "t"
subset_object = False
geneset_list = ["MSigDB_Hallmark_2020", "KEGG_2021_Human", "GO_Biological_Process_2023"]

path_to_eRegulon_signatures_gene = "/nfs/team205/heart/pyscenic/plus/eRegulon_signatures/vCM_onlyLeft_Gene_eRegulon_signatures.pkl"
path_to_eRegulon_signatures_region = "/nfs/team205/heart/pyscenic/plus/eRegulon_signatures/vCM_onlyLeft_Region_eRegulon_signatures.pkl"
```

Example 2 (python):
```python
import scanpy as sc
import pandas as pd
import numpy as np
import anndata

import matplotlib.pyplot as plt
import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 
import seaborn as sns
```

Example 3 (python):
```python
import session_info
session_info.show()
```

Example 4 (python):
```python
-----
anndata             0.9.2
matplotlib          3.7.2
numpy               1.24.4
pandas              1.5.3
scanpy              1.9.3
seaborn             0.12.2
session_info        1.0.0
-----
```

Example 5 (python):
```python
PIL                         10.0.0
anyio                       NA
arrow                       1.2.3
asttokens                   NA
attr                        23.1.0
attrs                       23.1.0
babel                       2.12.1
backcall                    0.2.0
brotli                      1.0.9
certifi                     2023.11.17
cffi                        1.15.1
charset_normalizer          3.2.0
colorama                    0.4.6
comm                        0.1.4
cycler                      0.10.0
cython_runtime              NA
dateutil                    2.8.2
debugpy                     1.6.8
decorator                   5.1.1
defusedxml                  0.7.1
executing                   1.2.0
fastjsonschema              NA
fqdn                        NA
h5py                        3.9.0
idna                        3.4
igraph                      0.10.6
importlib_metadata          NA
importlib_resources         NA
ipykernel                   6.25.1
isoduration                 NA
jedi                        0.19.0
jinja2                      3.1.2
joblib                      1.3.2
json5                       NA
jsonpointer                 2.0
jsonschema                  4.19.0
jsonschema_specifications   NA
jupyter_events              0.7.0
jupyter_server              2.7.1
jupyterlab_server           2.24.0
kiwisolver                  1.4.5
leidenalg                   0.10.1
llvmlite                    0.40.1
markupsafe                  2.1.3
mpl_toolkits                NA
natsort                     8.4.0
nbformat                    5.9.2
numba                       0.57.1
overrides                   NA
packaging                   23.1
parso                       0.8.3
patsy                       0.5.3
pexpect                     4.8.0
pickleshare                 0.7.5
pkg_resources               NA
plotly                      5.16.1
prometheus_client           NA
prompt_toolkit              3.0.39
psutil                      5.9.5
ptyprocess                  0.7.0
pure_eval                   0.2.2
pydev_ipython               NA
pydevconsole                NA
pydevd                      2.9.5
pydevd_file_utils           NA
pydevd_plugins              NA
pydevd_tracing              NA
pygments                    2.16.1
pyparsing                   3.0.9
pythonjsonlogger            NA
pytz                        2023.3
referencing                 NA
requests                    2.31.0
rfc3339_validator           0.1.4
rfc3986_validator           0.1.1
rpds                        NA
ruamel                      NA
scipy                       1.10.1
send2trash                  NA
six                         1.16.0
sklearn                     1.3.0
sniffio                     1.3.0
socks                       1.7.1
sphinxcontrib               NA
stack_data                  0.6.2
statsmodels                 0.14.0
texttable                   1.6.7
threadpoolctl               3.2.0
tornado                     6.3.3
traitlets                   5.9.0
typing_extensions           NA
uri_template                NA
urllib3                     2.0.4
wcwidth                     0.2.6
webcolors                   1.13
websocket                   1.6.2
yaml                        6.0.1
zipp                        NA
zmq                         25.1.1
zstandard                   0.19.0
```

Example 6 (unknown):
```unknown
-----
IPython             8.12.0
jupyter_client      8.3.0
jupyter_core        4.12.0
jupyterlab          4.0.5
notebook            7.0.2
-----
Python 3.8.17 (default, Jul  5 2023, 21:04:15) [GCC 11.2.0]
Linux-4.15.0-220-generic-x86_64-with-glibc2.17
-----
Session information updated at 2024-02-13 16:26
```

Example 7 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 8 (python):
```python
sc.settings.set_figure_params(dpi=80,dpi_save=300)
```

Example 9 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk'
```

Example 10 (python):
```python
path_to_adata_gex_dir
```

Example 11 (unknown):
```unknown
'/nfs/team205/heart/anndata_objects/Vitro/post_multiVI_full-features'
```

Example 12 (python):
```python
adata_compact = sc.read_h5ad(f'{path_to_adata_gex_dir}/Foetal_vCM_Left-compact_rna.h5ad')
# filter and log normalise
# filter
sc.pp.filter_genes(adata_compact, min_cells=3)

# log normalise
sc.pp.normalize_total(adata_compact, target_sum=1e4)
sc.pp.log1p(adata_compact)
print(adata_compact.X.data[:10])

adata_trab = sc.read_h5ad(f'{path_to_adata_gex_dir}/Foetal_vCM_Left-trabeculated_rna.h5ad')
# filter and log normalise
# filter
sc.pp.filter_genes(adata_trab, min_cells=3)

# log normalise
sc.pp.normalize_total(adata_trab, target_sum=1e4)
sc.pp.log1p(adata_trab)
print(adata_trab.X.data[:10])
```

Example 13 (unknown):
```unknown
[1.0619993 1.5653387 1.8985853 1.5653387 2.3476288 1.8985853 1.5653387
 1.0619993 1.0619993 1.0619993]
[1.0934392 1.0934392 1.0934392 1.603227  1.0934392 1.0934392 1.0934392
 1.603227  1.0934392 1.603227 ]
```

Example 14 (python):
```python
sc.pl.draw_graph(adata_compact, color=[group_key, time_key], size=5, wspace=0.3, ncols=3)
```

Example 15 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 16 (python):
```python
sc.pl.draw_graph(adata_trab, color=[group_key, time_key], size=5, wspace=0.3, ncols=3)
```

Example 17 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 18 (python):
```python
# whole data
adata_all = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/vCM/post_scFate/foetal_vCM_onlyLeft.h5ad')

# subset gene expression data
adata_all = adata_all[:,adata_all.var['modality']=='Gene Expression']

# filter and log-normalise
sc.pp.filter_genes(adata_all, min_cells=3)
sc.pp.normalize_total(adata_all, target_sum=1e4)
sc.pp.log1p(adata_all)

print(adata_all.X.data[:10])
adata_all
```

Example 19 (python):
```python
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/preprocessing/_simple.py:251: ImplicitModificationWarning: Trying to modify attribute `.var` of view, initializing view as actual.
  adata.var['n_cells'] = number
```

Example 20 (unknown):
```unknown
[1.0619993 1.5653387 1.8985853 1.5653387 2.3476288 1.8985853 1.5653387
 1.0619993 1.0619993 1.0619993]
```

Example 21 (unknown):
```unknown
AnnData object with n_obs × n_vars = 16451 × 26187
    obs: 'sangerID', 'combinedID', 'donor', 'region', 'age', 'sex', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'week', 'trimester', 'cycling', 'S_score', 'G2M_score', 'phase', 'fine_grain', 'mid_grain', 'coarse_grain', 'batch_key', 'total_counts', 'total_counts_mt', 'total_counts_ribo', 'n_genes_by_counts', 'pct_counts_mt', 'pct_counts_ribo', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'log1p_total_counts_mt', 'log1p_total_counts_ribo', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', '_indices', '_scvi_batch', '_scvi_labels', 'leiden_MultiVI', 't', 'seg', 'edge', 'milestones', 'age_days', 'week_cont'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id', 'seqnames', 'start', 'end', 'width', 'strand', 'score', 'replicateScoreQuantile', 'groupScoreQuantile', 'Reproducibility', 'GroupReplicate', 'distToGeneStart', 'nearestGene', 'peakType', 'distToTSS', 'nearestTSS', 'GC', 'idx', 'N', 'n_cells'
    uns: '_scvi_manager_uuid', '_scvi_uuid', 'cycling_colors', 'donor_colors', 'draw_graph', 'fine_grain_colors', 'graph', 'leiden', 'leiden_MultiVI_colors', 'milestones_colors', 'neighbors', 'ppt', 'pseudotime_list', 'region_colors', 'umap', 'week_colors', 'log1p'
    obsm: 'MultiVI_latent', 'X_R', 'X_draw_graph_fa', 'X_palantir', 'X_umap', 'X_umap13', '_scvi_extra_categorical_covs', '_scvi_extra_continuous_covs'
    obsp: 'connectivities', 'distances'
```

Example 22 (python):
```python
adata_all.obs.fine_grain.value_counts()
```

Example 23 (unknown):
```unknown
VentricularCardiomyocytesLeftCompact         11397
VentricularCardiomyocytesLeftTrabeculated     5054
Name: fine_grain, dtype: int64
```

Example 24 (unknown):
```unknown
logFC_threshold = 0.5
```

Example 25 (python):
```python
# make new label for the testing: combining fine_grain and t_binned
## for each compact and trabeculated
adata_compact.obs['fine_grain_time'] = adata_compact.obs['fine_grain'].astype('str')+'_'+adata_compact.obs['t_binned'].astype('str')
adata_trab.obs['fine_grain_time'] = adata_trab.obs['fine_grain'].astype('str')+'_'+adata_trab.obs['t_binned'].astype('str')

## add to the whold anndata
adata_all.obs['fine_grain_time'] = np.nan
adata_all.obs.loc[adata_compact.obs_names,'fine_grain_time'] = adata_compact.obs['fine_grain_time'].copy()
adata_all.obs.loc[adata_trab.obs_names,'fine_grain_time'] = adata_trab.obs['fine_grain_time'].copy()

# checking all cells has a label
all(adata_all.obs['fine_grain_time'].isna()==False)
```

Example 26 (python):
```python
# compact, up
compact_deg_up = utils.rank_genes_groups_with_mean(adata_all,
                                  groupby='fine_grain_time',
                                  groups=['VentricularCardiomyocytesLeftCompact_t3'],
                                  reference='VentricularCardiomyocytesLeftTrabeculated_t3',
                                  method='t-test',
                                  log2fc_min=logFC_threshold ## thresholdng for UP
                                 )
compact_deg_up = compact_deg_up.sort_values('scores',ascending=False)
compact_deg_up[:20]
```

Example 27 (python):
```python
# compact, down
compact_deg_down = utils.rank_genes_groups_with_mean(adata_all,
                                  groupby='fine_grain_time',
                                  groups=['VentricularCardiomyocytesLeftCompact_t3'],
                                  reference='VentricularCardiomyocytesLeftTrabeculated_t3',
                                  method='t-test',
                                  log2fc_max = -logFC_threshold ## thresholdng for DOWN
                                 )
compact_deg_down = compact_deg_down.sort_values('scores',ascending=True)
compact_deg_down[:20]
```

Example 28 (python):
```python
# trabeculated, up
trab_deg_up = utils.rank_genes_groups_with_mean(adata_all,
                                  groupby='fine_grain_time',
                                  groups=['VentricularCardiomyocytesLeftTrabeculated_t3'],
                                  reference='VentricularCardiomyocytesLeftCompact_t3',
                                  method='t-test',
                                  log2fc_min=logFC_threshold ## thresholdng for UP
                                 )
trab_deg_up = trab_deg_up.sort_values('scores',ascending=False)
trab_deg_up[:20]
```

Example 29 (python):
```python
# trabeculated, down
trab_deg_down = utils.rank_genes_groups_with_mean(adata_all,
                                  groupby='fine_grain_time',
                                  groups=['VentricularCardiomyocytesLeftTrabeculated_t3'],
                                  reference='VentricularCardiomyocytesLeftCompact_t3',
                                  method='t-test',
                                  log2fc_max = -logFC_threshold ## thresholdng for DOWN
                                 )
trab_deg_down = trab_deg_down.sort_values('scores',ascending=True)
trab_deg_down[:20]
```

Example 30 (python):
```python
fit_res_compact = pd.read_csv(f'{working_dir}/foetal_vCM_left-compact/outs/Fitting_{time_key}_Gene.csv',index_col=0)
fit_res_trab = pd.read_csv(f'{working_dir}/foetal_vCM_left-trabeculated/outs/Fitting_{time_key}_Gene.csv',index_col=0)
```

Example 31 (unknown):
```unknown
fit_res_compact.loc['SLC27A1']
```

Example 32 (unknown):
```unknown
p_val                       0.0
A                      0.350825
fdr                         0.0
st                            0
signi                     False
activator_repressor     SLC27A1
Name: SLC27A1, dtype: object
```

Example 33 (unknown):
```unknown
fit_res_trab.loc['SLC27A1']
```

Example 34 (unknown):
```unknown
p_val                   0.00005
A                      0.967977
fdr                         1.0
st                            0
signi                     False
activator_repressor     SLC27A1
Name: SLC27A1, dtype: object
```

Example 35 (python):
```python
# compact
group_mean_compact = utils.grouped_obs_mean(adata_compact,group_key=group_key)

# select upregulated (a>b) or downregulated (b>a) genes
group_a_genes_compact = list(group_mean_compact.index[group_mean_compact[group_a] > group_mean_compact[group_b]])
group_b_genes_compact = list(group_mean_compact.index[group_mean_compact[group_b] > group_mean_compact[group_a]])

group_mean_compact.head()
```

Example 36 (unknown):
```unknown
group_mean_compact.loc['SLC27A1']
```

Example 37 (unknown):
```unknown
t1    0.054584
t2    0.130589
t3    0.216747
Name: SLC27A1, dtype: float64
```

Example 38 (python):
```python
# trabeculated
group_mean_trab = utils.grouped_obs_mean(adata_trab,group_key=group_key)

# select upregulated (a>b) or downregulated (b>a) genes
group_a_genes_trab = list(group_mean_trab.index[group_mean_trab[group_a] > group_mean_trab[group_b]])
group_b_genes_trab = list(group_mean_trab.index[group_mean_trab[group_b] > group_mean_trab[group_a]])

group_mean_trab.head()
```

Example 39 (unknown):
```unknown
group_mean_trab.loc['SLC27A1']
```

Example 40 (unknown):
```unknown
t1    0.025651
t2    0.154269
t3    0.204383
Name: SLC27A1, dtype: float64
```

Example 41 (unknown):
```unknown
fit_fdr_threshold = 0.05
fit_A_threshold = 0.5
```

Example 42 (python):
```python
# compact
fit_res_sel_compact = fit_res_compact[(fit_res_compact['fdr']<fit_fdr_threshold)&\
                                      (fit_res_compact['A']>fit_A_threshold)
                                     ]
# group a, upregulated
shared = list(set(fit_res_sel_compact.index).intersection(group_a_genes_compact))
fit_res_sel_up2a_compact = fit_res_sel_compact.loc[shared]
fit_res_sel_up2a_compact = fit_res_sel_up2a_compact.sort_values('A',ascending=False)
print(f'up towards {group_a}, number of fitted genes: {len(fit_res_sel_up2a_compact)}')
# group b, downregulated
shared = list(set(fit_res_sel_compact.index).intersection(group_b_genes_compact))
fit_res_sel_up2b_compact = fit_res_sel_compact.loc[shared]
fit_res_sel_up2b_compact = fit_res_sel_up2b_compact.sort_values('A',ascending=False)
print(f'up towards {group_b}, number of fitted genes: {len(fit_res_sel_up2b_compact)}')
```

Example 43 (unknown):
```unknown
up towards t3, number of fitted genes: 1073
up towards t1, number of fitted genes: 2176
```

Example 44 (python):
```python
# trabeculated
fit_res_sel_trab = fit_res_trab[(fit_res_trab['fdr']<fit_fdr_threshold)&\
                                (fit_res_trab['A']>fit_A_threshold)
                                     ]
# group a, upregulated
shared = list(set(fit_res_sel_trab.index).intersection(group_a_genes_trab))
fit_res_sel_up2a_trab = fit_res_sel_trab.loc[shared]
fit_res_sel_up2a_trab = fit_res_sel_up2a_trab.sort_values('A',ascending=False)
print(f'up towards {group_a}, number of fitted genes: {len(fit_res_sel_up2a_trab)}')
# group b, downregulated
shared = list(set(fit_res_sel_trab.index).intersection(group_b_genes_trab))
fit_res_sel_up2b_trab = fit_res_sel_trab.loc[shared]
fit_res_sel_up2b_trab = fit_res_sel_up2b_trab.sort_values('A',ascending=False)
print(f'up towards {group_b}, number of fitted genes: {len(fit_res_sel_up2b_trab)}')
```

Example 45 (unknown):
```unknown
up towards t3, number of fitted genes: 893
up towards t1, number of fitted genes: 1972
```

Example 46 (unknown):
```unknown
fit_res_sel_up2a_compact.loc['PRDM16']
```

Example 47 (unknown):
```unknown
p_val                       0.0
A                      1.618132
fdr                         0.0
st                            1
signi                      True
activator_repressor      PRDM16
Name: PRDM16, dtype: object
```

Example 48 (unknown):
```unknown
fit_res_sel_up2a_trab.loc['PRDM16']
```

Example 49 (unknown):
```unknown
p_val                      0.0
A                      1.63836
fdr                        0.0
st                           1
signi                     True
activator_repressor     PRDM16
Name: PRDM16, dtype: object
```

Example 50 (python):
```python
# up toward group a, up-regulated
compact_spec_up2a_genes = list(set(fit_res_sel_up2a_compact.index).intersection(list(compact_deg_up['names'])))
trab_spec_up2a_genes = list(set(fit_res_sel_up2a_trab.index).intersection(list(trab_deg_up['names'])))
common_up2a_genes = list(set(fit_res_sel_up2a_compact.index).intersection(fit_res_sel_up2a_trab.index))
common_up2a_genes = list(set(common_up2a_genes)-set(compact_deg_up['names'])-set(trab_deg_up['names'])) ### removing DEGs
print(f'up toward {group_a}')
print(len(compact_spec_up2a_genes))
print(len(trab_spec_up2a_genes))
print(len(common_up2a_genes))

# up toward group b, down-regulated
compact_spec_up2b_genes = list(set(fit_res_sel_up2b_compact.index).intersection(list(compact_deg_down['names'])))
trab_spec_up2b_genes = list(set(fit_res_sel_up2b_trab.index).intersection(list(trab_deg_down['names'])))
common_up2b_genes = list(set(fit_res_sel_up2b_compact.index).intersection(fit_res_sel_up2b_trab.index))
common_up2b_genes = list(set(common_up2b_genes)-set(compact_deg_down['names'])-set(trab_deg_down['names'])) ### removing DEGs
print(f'up toward {group_b}')
print(len(compact_spec_up2b_genes))
print(len(trab_spec_up2b_genes))
print(len(common_up2b_genes))
```

Example 51 (unknown):
```unknown
up toward t3
130
107
336
up toward t1
185
384
738
```

Example 52 (python):
```python
# venn diagram for upregulated genes
# just checking there is no overlap
from matplotlib_venn import venn3
venn3((set(compact_spec_up2a_genes),set(trab_spec_up2a_genes),set(common_up2a_genes)),
      set_labels=('compact', 'trab', 'common')
     )
```

Example 53 (python):
```python
<matplotlib_venn._common.VennDiagram at 0x14cfcef75a60>
```

Example 54 (python):
```python
# venn diagram for downregulated genes
# just checking there is no overlap
from matplotlib_venn import venn3
venn3((set(compact_spec_up2b_genes),set(trab_spec_up2b_genes),set(common_up2b_genes)),
      set_labels=('compact', 'trab', 'common')
     )
```

Example 55 (python):
```python
<matplotlib_venn._common.VennDiagram at 0x14cfcf01eeb0>
```

Example 56 (python):
```python
gene_signatures = {}
# up, towards group_a
gene_signatures['compact_specific_up'] = compact_spec_up2a_genes
gene_signatures['trab_specific_up'] = trab_spec_up2a_genes
gene_signatures['common_up'] = common_up2a_genes

# down, towards group_b
gene_signatures['compact_specific_down'] = compact_spec_up2b_genes
gene_signatures['trab_specific_down'] = trab_spec_up2b_genes
gene_signatures['common_down'] = common_up2b_genes

print(gene_signatures.keys())

# save
utils.save_pkl(gene_signatures,
           file = f'{working_dir}/foetal_vCM_left_compact-vs-trabeculated_downstream/gene_signatures.pkl'
          )
```

Example 57 (unknown):
```unknown
dict_keys(['compact_specific_up', 'trab_specific_up', 'common_up', 'compact_specific_down', 'trab_specific_down', 'common_down'])
```

Example 58 (unknown):
```unknown
f'{working_dir}/foetal_vCM_left_compact-vs-trabeculated_downstream/gene_signatures.pkl'
```

Example 59 (unknown):
```unknown
'/nfs/team205/heart/Vitro/TF_selection/foetal_vCM_left_compact-vs-trabeculated_downstream/gene_signatures.pkl'
```

Example 60 (python):
```python
adata_all.X.data[:10]
```

Example 61 (unknown):
```unknown
array([1.0619993, 1.5653387, 1.8985853, 1.5653387, 2.3476288, 1.8985853,
       1.5653387, 1.0619993, 1.0619993, 1.0619993], dtype=float32)
```

Example 62 (python):
```python
# upregulated
sc.tl.score_genes(adata_all,compact_spec_up2a_genes,score_name=f'compact ({len(compact_spec_up2a_genes)}g)')
sc.tl.score_genes(adata_all,trab_spec_up2a_genes,score_name=f'trabeculated ({len(trab_spec_up2a_genes)}g)')
sc.tl.score_genes(adata_all,common_up2a_genes,score_name=f'common ({len(common_up2a_genes)}g)')

sc.pl.draw_graph(adata_all, color=['fine_grain',
                                   f'compact ({len(compact_spec_up2a_genes)}g)',
                                   f'trabeculated ({len(trab_spec_up2a_genes)}g)',
                                   f'common ({len(common_up2a_genes)}g)',
                                  ], 
                 frameon=False, legend_loc='on data',legend_fontsize=0,
                 size=7, wspace=0.1, ncols=4, cmap='YlOrRd',show=False)
plt.savefig(f"{figdir}/vCMs-left_force-directed_gene-signatures-upregulated.pdf", bbox_inches="tight",dpi=300)
```

Example 63 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 64 (python):
```python
# downregulated
sc.tl.score_genes(adata_all,compact_spec_up2b_genes,score_name=f'compact ({len(compact_spec_up2b_genes)}g)')
sc.tl.score_genes(adata_all,trab_spec_up2b_genes,score_name=f'trabeculated ({len(trab_spec_up2b_genes)}g)')
sc.tl.score_genes(adata_all,common_up2b_genes,score_name=f'common ({len(common_up2b_genes)}g)')

sc.pl.draw_graph(adata_all, color=['fine_grain',
                                   f'compact ({len(compact_spec_up2b_genes)}g)',
                                   f'trabeculated ({len(trab_spec_up2b_genes)}g)',
                                   f'common ({len(common_up2b_genes)}g)',
                                  ], 
                 frameon=False, legend_loc='on data',legend_fontsize=0,
                 size=7, wspace=0.1, ncols=4, cmap='YlOrRd',show=False)
plt.savefig(f"{figdir}/vCMs-left_force-directed_gene-signatures-downregulated.pdf", bbox_inches="tight",dpi=300)
```

Example 65 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

---

## Aims¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig4_Cardiomyocytes/AtrialCardiomyocytes/Visium_mapping/LvR_aCM_immature_CM_UMAP.html

**Contents:**
- Aims¶
- Read in data with all features¶
- ¶
- Split into GEX and ATAC objects¶
- Make AUC objects¶
- Find features which are Left or Right specific¶
- GEX¶
  - Functional interpretation of Left and Right CM sigantures¶
- ATAC¶
- AUCgene¶
- AUCregion¶
  - Plot features which are specific to Left or Right aCMs¶
- Find features which increase over time¶
  - Functional interpretation of genes which increase over time across all aCMs¶
- Find features which are associated with time and are increasing over time¶
- Fit expression of features which change (increase or decrease) over time¶
- GEX¶
- ATAC¶
- AUCgene¶
- AUCregion¶

We want to demonstrate:

LvR: features which are different at all time (DE testing)

L: features which are associated and increase over time L: features which are associated and decrease over time

R: features which are associated and increase over time R: features which are associated and decrease over time

We start by definign a stricter set of L and R specific genes

**Examples:**

Example 1 (unknown):
```unknown
jctk_dir="/lustre/scratch126/cellgen/team205/jc48/jupyter/jctk"
```

Example 2 (python):
```python
import importlib.util
import sys
import types
import inspect

# ensure the `jctk_dir` path is already specified

def import_and_list_functions(module_name, file_path):
    spec = importlib.util.spec_from_file_location(module_name, file_path)
    module = importlib.util.module_from_spec(spec)
    sys.modules[module_name] = module
    spec.loader.exec_module(module)
    
    function_names = []
    for name, obj in inspect.getmembers(module):
        if isinstance(obj, types.FunctionType):
            function_names.append(name)
    
    print(f"Imported from {module_name}: {function_names}")
    
    return module  # Return the module object

# Import and list functions, then assign to a variable
plot = import_and_list_functions("plot", f"{jctk_dir}/jctk/plot.py")
utils = import_and_list_functions("utils", f"{jctk_dir}/jctk/utils.py")
api = import_and_list_functions("api", f"{jctk_dir}/jctk/api.py")
```

Example 3 (python):
```python
Imported from plot: ['GO_plot', 'highlight_all_categories', 'highlight_categories', 'normalized_stacked_bar_chart']
Imported from utils: ['check_update', 'compute_median_count', 'downsample_adata_proportionately', 'downsample_adata_randomly', 'lognorm_to_counts_with_progress', 'rank_genes_vs_nearest', 'read_gsheet', 'remove_small_batches', 'rename_adata_var_index', 'rules_based_annotation', 'shapiro', 'test_normality']
Imported from api: ['add_genomic_coordinates', 'getClinVargenes', 'getHGNCgroup', 'getHGNCgroup_dict']
```

Example 4 (python):
```python
import anndata

def split_adata_by_modality(adata, modality_column, modality_values):
    """
    Splits an AnnData object into multiple AnnData objects based on a given modality column.
    
    Parameters:
        adata (anndata.AnnData): The original AnnData object.
        modality_column (str): The name of the column in adata.var that contains the modality information.
        modality_values (list): List of unique modalities that you want to split the adata into.
        
    Returns:
        dict: A dictionary where the keys are the modality names and the values are the new AnnData objects.
    """
    # Initialize an empty dictionary to hold the new AnnData objects
    splitted_adatas = {}
    
    # Loop through each modality value to create a new AnnData object
    for modality in modality_values:
        # Get the indices of the features belonging to the current modality
        feature_idx = adata.var[modality_column] == modality
        
        # Create a new AnnData object with only the features belonging to the current modality
        new_adata = anndata.AnnData(X=adata[:, feature_idx].X,
                                    obs=adata.obs,
                                    var=adata.var.loc[feature_idx],
                                    obsm=adata.obsm,
                                    uns=adata.uns)
        
        # Add the new AnnData object to the dictionary
        splitted_adatas[modality] = new_adata
    
    return splitted_adatas
```

Example 5 (python):
```python
path_to_auc_gene="/nfs/team205/heart/pyscenic/plus/AUCscore/foetal_aCM_n570_eRegulon_AUC_gene.csv"
path_to_auc_region="/nfs/team205/heart/pyscenic/plus/AUCscore/foetal_aCM_n570_eRegulon_AUC_region.csv"

paired_adata_with_time_path = "/lustre/scratch126/cellgen/team205/heart/objects/fetal/RNA/aCMs_WholeSample_paired_with_time.h5ad"

temporal_ordering='age_days_smoothed_once'
```

Example 6 (python):
```python
import scanpy as sc
import pandas as pd
```

Example 7 (python):
```python
adata=sc.read(paired_adata_with_time_path)
adata
```

Example 8 (unknown):
```unknown
AnnData object with n_obs × n_vars = 5664 × 544641
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', 'age_days', 'age_days_smoothed_once', 'age_days_normalized', 'age_days_normalized_smoothed_once', 'age_days_smoothed_iteratively', 'age_days_normalized_smoothed_iteratively', 't', 'seg', 'edge', 't_sd', 'milestones'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id'
    uns: 'donor_colors', 'fine_grain_colors', 'graph', 'milestones_colors', 'neighbors', 'ppt', 'pseudotime_list', 'seg_colors', 'umap', 'week_colors'
    obsm: 'X_R', 'X_multiVI', 'X_umap'
    obsp: 'connectivities', 'distances'
```

Example 9 (python):
```python
adata.X.data[:5]
```

Example 10 (unknown):
```unknown
array([2., 1., 1., 1., 1.], dtype=float32)
```

Example 11 (python):
```python
sc.pl.umap(adata,color=["seg","milestones","t",temporal_ordering,"fine_grain"],ncols=2)
```

Example 12 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 13 (python):
```python
sc.pl.umap(adata,color=["fine_grain"],ncols=1)
sc.pl.umap(adata,color=[temporal_ordering,"t"],ncols=1,title="sample age")

sc.pl.umap(adata,color=["fine_grain","age_days","t"],ncols=1, legend_loc=None)
```

Example 14 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 15 (unknown):
```unknown
WARNING: The title list is shorter than the number of panels. Using 'color' value instead for some plots.
```

Example 16 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 17 (python):
```python
adata.shape
```

Example 18 (unknown):
```unknown
(5664, 544641)
```

Example 19 (python):
```python
modality_column = "modality"
modality_values = ["Gene Expression", "Peaks"]
splitted_adatas = split_adata_by_modality(adata, modality_column, modality_values)

adata_gex = splitted_adatas["Gene Expression"]
print(adata_gex)

adata_atac = splitted_adatas["Peaks"]
print(adata_atac)
```

Example 20 (unknown):
```unknown
AnnData object with n_obs × n_vars = 5664 × 36601
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', 'age_days', 'age_days_smoothed_once', 'age_days_normalized', 'age_days_normalized_smoothed_once', 'age_days_smoothed_iteratively', 'age_days_normalized_smoothed_iteratively', 't', 'seg', 'edge', 't_sd', 'milestones'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id'
    uns: 'donor_colors', 'fine_grain_colors', 'graph', 'milestones_colors', 'neighbors', 'ppt', 'pseudotime_list', 'seg_colors', 'umap', 'week_colors'
    obsm: 'X_R', 'X_multiVI', 'X_umap'
AnnData object with n_obs × n_vars = 5664 × 508040
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', 'age_days', 'age_days_smoothed_once', 'age_days_normalized', 'age_days_normalized_smoothed_once', 'age_days_smoothed_iteratively', 'age_days_normalized_smoothed_iteratively', 't', 'seg', 'edge', 't_sd', 'milestones'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id'
    uns: 'donor_colors', 'fine_grain_colors', 'graph', 'milestones_colors', 'neighbors', 'ppt', 'pseudotime_list', 'seg_colors', 'umap', 'week_colors'
    obsm: 'X_R', 'X_multiVI', 'X_umap'
```

Example 21 (python):
```python
adata_gex.X.data[:10]
```

Example 22 (unknown):
```unknown
array([2., 1., 1., 1., 1., 1., 8., 2., 1., 1.], dtype=float32)
```

Example 23 (python):
```python
adata_atac.X.data[:10]
```

Example 24 (unknown):
```unknown
array([2., 1., 1., 1., 1., 2., 2., 1., 1., 2.], dtype=float32)
```

Example 25 (python):
```python
adata_gex.layers["counts"]=adata_gex.X.copy()
adata_atac.layers["counts"]=adata_atac.X.copy()

sc.pp.normalize_total(adata_gex, target_sum=1e4)
sc.pp.normalize_total(adata_atac, target_sum=1e4)

sc.pp.log1p(adata_gex)
sc.pp.log1p(adata_atac)
```

Example 26 (python):
```python
WARNING: adata.X seems to be already log-transformed.
```

Example 27 (python):
```python
adata_gex.X.data[:10]
```

Example 28 (unknown):
```unknown
array([1.559231 , 1.0569503, 1.0569503, 1.0569503, 1.0569503, 1.0569503,
       2.7738788, 1.559231 , 1.0569503, 1.0569503], dtype=float32)
```

Example 29 (python):
```python
adata_atac.X.data[:10]
```

Example 30 (unknown):
```unknown
array([0.6963374 , 0.40759304, 0.40759304, 0.40759304, 0.40759304,
       0.6963374 , 0.6963374 , 0.40759304, 0.40759304, 0.6963374 ],
      dtype=float32)
```

Example 31 (python):
```python
# read in regulon scores
auc_gene = pd.read_csv(path_to_auc_gene, index_col=0)
print(auc_gene.shape)
auc_region = pd.read_csv(path_to_auc_region, index_col=0)
print(auc_region.shape)
```

Example 32 (unknown):
```unknown
(17302, 278)
(12964, 278)
```

Example 33 (python):
```python
# make regulon adata, AUC gene
# AUC Gene
# create anndata
adata_aucg = anndata.AnnData(
    X=auc_gene.loc[adata_gex.obs_names],
    obs=adata_gex.obs,
    var=pd.DataFrame(auc_gene.columns).set_index(0),
    obsm=adata_gex.obsm,
    uns= adata_gex.uns
)
# modify varnames
adata_aucg.var_names = [x.split('_(')[0] for x in adata_aucg.var_names]
print(adata_aucg.shape)

# AUC Region
# create anndata
adata_aucr = anndata.AnnData(
    X=auc_region.loc[adata_gex.obs_names],
    obs=adata_gex.obs,
    var=pd.DataFrame(auc_region.columns).set_index(0),
    obsm=adata_gex.obsm
)
# modify varnames
adata_aucr.var_names = [x.split('_(')[0] for x in adata_aucr.var_names]
print(adata_aucr.shape)
```

Example 34 (unknown):
```unknown
(5664, 278)
(5664, 278)
```

Example 35 (unknown):
```unknown
LEFT_or_RIGHT_features={}
```

Example 36 (unknown):
```unknown
log2fc_min=0.5
pval_cutoff=0.1
```

Example 37 (python):
```python
sc.tl.dendrogram(adata_gex,groupby="fine_grain",use_rep="X_multiVI")
sc.tl.rank_genes_groups(adata_gex,groupby="fine_grain")
sc.pl.rank_genes_groups_dotplot(adata_gex,groupby="fine_grain",n_genes=20)

df=sc.get.rank_genes_groups_df(adata,group="AtrialCardiomyocytesLeft",log2fc_min=log2fc_min)
df=df[df.pvals_adj<pval_cutoff]
LEFT_or_RIGHT_features["LEFT_genes"]=df.names.tolist()

df=sc.get.rank_genes_groups_df(adata,group="AtrialCardiomyocytesRight",log2fc_min=log2fc_min)
df=df[df.pvals_adj<pval_cutoff]
LEFT_or_RIGHT_features["RIGHT_genes"]=df.names.tolist()
```

Example 38 (python):
```python
WARNING: Default of the method has been changed to 't-test' from 't-test_overestim_var'
WARNING: Dendrogram not added. Dendrogram is added only when the number of categories to plot > 2
```

Example 39 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_dotplot.py:749: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap', 'norm' will be ignored
  dot_ax.scatter(x, y, **kwds)
```

Example 40 (unknown):
```unknown
plot.GO_plot(gene_list=LEFT_or_RIGHT_features["RIGHT_genes"],display_table=True,
             gene_sets=['GO_Biological_Process_2021'],
             title_suffix=" AtrialCardiomyocytesRight",
            n_rank=10)
```

Example 41 (unknown):
```unknown
GO_Biological_Process_2021
```

Example 42 (unknown):
```unknown
plot.GO_plot(gene_list=LEFT_or_RIGHT_features["LEFT_genes"],display_table=True,gene_sets=['GO_Biological_Process_2023'],title_suffix=" AtrialCardiomyocytesLeft")
```

Example 43 (unknown):
```unknown
GO_Biological_Process_2023
```

Example 44 (python):
```python
sc.tl.dendrogram(adata_atac,groupby="fine_grain",use_rep="X_multiVI")
sc.tl.rank_genes_groups(adata_atac,groupby="fine_grain")
sc.pl.rank_genes_groups_dotplot(adata_atac,groupby="fine_grain",n_genes=20)

df=sc.get.rank_genes_groups_df(adata,group="AtrialCardiomyocytesLeft",log2fc_min=log2fc_min)
df=df[df.pvals_adj<pval_cutoff]
print(f"adding {len(df.names.tolist())} features for LEFT...")
LEFT_or_RIGHT_features["LEFT_peaks"]=df.names.tolist()

df=sc.get.rank_genes_groups_df(adata,group="AtrialCardiomyocytesRight",log2fc_min=log2fc_min)
df=df[df.pvals_adj<pval_cutoff]
print(f"adding {len(df.names.tolist())} features for RIGHT...")
LEFT_or_RIGHT_features["RIGHT_peaks"]=df.names.tolist()
```

Example 45 (python):
```python
WARNING: Default of the method has been changed to 't-test' from 't-test_overestim_var'
WARNING: Dendrogram not added. Dendrogram is added only when the number of categories to plot > 2
```

Example 46 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_dotplot.py:749: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap', 'norm' will be ignored
  dot_ax.scatter(x, y, **kwds)
```

Example 47 (unknown):
```unknown
adding 5316 features for LEFT...
adding 3871 features for RIGHT...
```

Example 48 (python):
```python
# Instead of doing scanpy DE testing, we use the results of ArchR for this step
df=pd.read_csv("./DAR_testing/L-vs-R_aCM.csv")
df=df[(df.Log2FC>log2fc_min)&(df.FDR<pval_cutoff)]
df["names"]=df["seqnames"].astype(str)+":"+df["start"].astype(str)+"_"+df["end"].astype(str)
print(f"adding {len(df.names.tolist())} features for LEFT...")
LEFT_or_RIGHT_features["LEFT_peaks"]=df.names.tolist()

df=pd.read_csv("./DAR_testing/R-vs-L_aCM.csv")
df=df[(df.Log2FC>log2fc_min)&(df.FDR<pval_cutoff)]
df["names"]=df["seqnames"].astype(str)+":"+df["start"].astype(str)+"_"+df["end"].astype(str)
print(f"adding {len(df.names.tolist())} features for RIGHT...")
LEFT_or_RIGHT_features["RIGHT_peaks"]=df.names.tolist()
```

Example 49 (unknown):
```unknown
adding 132 features for LEFT...
adding 57 features for RIGHT...
```

Example 50 (python):
```python
sc.tl.dendrogram(adata_aucg,groupby="fine_grain",use_rep="X_multiVI")
sc.tl.rank_genes_groups(adata_aucg,groupby="fine_grain")
sc.pl.rank_genes_groups_dotplot(adata_aucg,groupby="fine_grain",n_genes=20)

df=sc.get.rank_genes_groups_df(adata_aucg,group="AtrialCardiomyocytesLeft",log2fc_min=log2fc_min)
df=df[df.pvals_adj<pval_cutoff]
print(f"adding {len(df.names.tolist())} features for LEFT...")
LEFT_or_RIGHT_features["LEFT_AUCgene"]=df.names.tolist()

df=sc.get.rank_genes_groups_df(adata_aucg,group="AtrialCardiomyocytesRight",log2fc_min=log2fc_min)
df=df[df.pvals_adj<pval_cutoff]
print(f"adding {len(df.names.tolist())} features for RIGHT...")
LEFT_or_RIGHT_features["RIGHT_AUCgene"]=df.names.tolist()
```

Example 51 (python):
```python
WARNING: Default of the method has been changed to 't-test' from 't-test_overestim_var'
WARNING: Dendrogram not added. Dendrogram is added only when the number of categories to plot > 2
```

Example 52 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_dotplot.py:749: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap', 'norm' will be ignored
  dot_ax.scatter(x, y, **kwds)
```

Example 53 (unknown):
```unknown
adding 11 features for LEFT...
adding 4 features for RIGHT...
```

Example 54 (python):
```python
sc.tl.dendrogram(adata_aucr,groupby="fine_grain",use_rep="X_multiVI")
sc.tl.rank_genes_groups(adata_aucr,groupby="fine_grain")
sc.pl.rank_genes_groups_dotplot(adata_aucr,groupby="fine_grain",n_genes=20)

df=sc.get.rank_genes_groups_df(adata_aucr,group="AtrialCardiomyocytesLeft",log2fc_min=log2fc_min)
df=df[df.pvals_adj<pval_cutoff]
print(f"adding {len(df.names.tolist())} features for LEFT...")
LEFT_or_RIGHT_features["LEFT_AUCregion"]=df.names.tolist()

df=sc.get.rank_genes_groups_df(adata_aucr,group="AtrialCardiomyocytesRight",log2fc_min=log2fc_min)
df=df[df.pvals_adj<pval_cutoff]
print(f"adding {len(df.names.tolist())} features for RIGHT...")
LEFT_or_RIGHT_features["RIGHT_AUCregion"]=df.names.tolist()
```

Example 55 (python):
```python
WARNING: Default of the method has been changed to 't-test' from 't-test_overestim_var'
WARNING: Dendrogram not added. Dendrogram is added only when the number of categories to plot > 2
```

Example 56 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_dotplot.py:749: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap', 'norm' will be ignored
  dot_ax.scatter(x, y, **kwds)
```

Example 57 (unknown):
```unknown
adding 15 features for LEFT...
adding 0 features for RIGHT...
```

Example 58 (python):
```python
import scanpy as sc  # Import scanpy, assuming it's used for gene scoring and plotting

def plot_sided_features(adata, feature_type, associated_increasing_features_dict, cmap="RdPu", sides=["LEFT", "RIGHT", "COMMON","NON_SIDE_SPECIFIC"]):
    """
    Plots sided features for a given AnnData object and feature type.
    
    Args:
    adata (AnnData): An AnnData object containing gene expression data.
    feature_type (str): The type of feature to plot.
    associated_increasing_features_dict (dict): A dictionary containing associated increasing features.
    """
    # Copy the adata object to avoid modifying the original data
    plot_adata = adata.copy()

    # Initialize an empty list to store score names
    scores = []

    # Iterate over the sides and calculate scores for each
    for side in sides:
        # Retrieve the gene list from the dictionary
        gene_list = associated_increasing_features_dict[f"{feature_type}_{side}"]

        if gene_list:
            # Construct a unique name for the score
            score_name = f"{feature_type}_{side}_({len(gene_list)} features)"
            # Add the score name to the list
            scores.append(score_name)
            # Calculate the score using scanpy's score_genes function
            sc.tl.score_genes(plot_adata, gene_list=gene_list, score_name=score_name)

    # Plot the UMAP with the scores
    if len(scores)>0:
        sc.pl.umap(plot_adata, color=scores, ncols=3, wspace=0.8, cmap=cmap)
```

Example 59 (unknown):
```unknown
# New dictionary to store converted keys
LEFT_or_RIGHT_features_converted = {}

# Iterate through each item in LEFT_or_RIGHT_features
for key, value in LEFT_or_RIGHT_features.items():
    # Split the key on the underscore
    parts = key.split('_')
    
    # Rearrange the parts to match the desired format
    # New key format: 'genes_LEFT', 'genes_RIGHT', etc.
    new_key = f"{parts[1]}_{parts[0]}"

    # Add to the converted dictionary
    LEFT_or_RIGHT_features_converted[new_key] = value
LEFT_or_RIGHT_features_converted.keys()
```

Example 60 (unknown):
```unknown
dict_keys(['genes_LEFT', 'genes_RIGHT', 'peaks_LEFT', 'peaks_RIGHT', 'AUCgene_LEFT', 'AUCgene_RIGHT', 'AUCregion_LEFT', 'AUCregion_RIGHT'])
```

Example 61 (python):
```python
plot_sided_features(adata_gex,
                    sides=["LEFT","RIGHT"],
                    feature_type="genes",
                    associated_increasing_features_dict=LEFT_or_RIGHT_features_converted)

plot_sided_features(adata_atac,
                    sides=["LEFT","RIGHT"],
                    feature_type="peaks",
                    associated_increasing_features_dict=LEFT_or_RIGHT_features_converted)

plot_sided_features(adata_aucg,
                    sides=["LEFT","RIGHT"],
                    feature_type="AUCgene",
                    associated_increasing_features_dict=LEFT_or_RIGHT_features_converted)

plot_sided_features(adata_aucr,
                    sides=["LEFT","RIGHT"],
                    feature_type="AUCregion",
                    associated_increasing_features_dict=LEFT_or_RIGHT_features_converted)
```

Example 62 (python):
```python
print(LEFT_or_RIGHT_features_converted["AUCgene_LEFT"])
print(LEFT_or_RIGHT_features_converted["AUCregion_LEFT"])
print(LEFT_or_RIGHT_features_converted["AUCgene_RIGHT"])
```

Example 63 (unknown):
```unknown
['PITX2_+', 'ETS2_-', 'MAF_-', 'GATA5_+', 'SIX5_extended_+', 'ZNF337_+', 'PEG3_+', 'NFATC1_+', 'ZNF383_-', 'ARID3A_extended_-', 'MEF2A_-']
['GTF2IRD1_+', 'PITX2_+', 'IKZF2_+', 'NFE2L1_+', 'NFE2L2_+', 'TCF7L2_+', 'MGA_+', 'FOXJ2_+', 'MAF_-', 'STAT5B_+', 'MESP1_extended_+', 'ETS2_-', 'STAT4_+', 'ZNF337_+', 'ZNF470_+']
['ETS2_+', 'MAF_+', 'MYCN_+', 'ZNF526_+']
```

Example 64 (python):
```python
import pandas as pd
import numpy as np
from typing import Any

def bin_pseudotime(adata: Any, pseudotime_column: str, n_bins: int) -> Any:
    """
    Bin cells in an AnnData object into n_bins based on a specified pseudotime column.

    Parameters:
    adata (AnnData): The AnnData object containing single-cell data.
    pseudotime_column (str): The name of the column in adata.obs that contains pseudotime values.
    n_bins (int): The number of bins to divide the pseudotime values into.

    Returns:
    AnnData: The AnnData object with an additional column in adata.obs indicating the bin each cell belongs to.
    """
    # Calculate the percentiles that will be used as bin edges
    percentiles = [i*(100/n_bins) for i in range(1, n_bins)]
    bin_edges = np.percentile(adata.obs[pseudotime_column], percentiles)
    
    # Bin values into discrete intervals using pandas.cut
    adata.obs[f'{pseudotime_column}_bins'] = pd.cut(
        adata.obs[pseudotime_column],
        bins=np.insert(bin_edges, [0, len(bin_edges)], [adata.obs[pseudotime_column].min(), adata.obs[pseudotime_column].max()]),  # Include the extreme values of the pseudotime range
        include_lowest=True,
        right=True,
        labels=[str(i) for i in range(n_bins)]
    )
    
    # Return the modified AnnData object
    return adata
```

Example 65 (unknown):
```unknown
n_bins=2 # number of temporal bins
```

Example 66 (python):
```python
adata_gex = bin_pseudotime(adata_gex, 't', n_bins)
adata_atac = bin_pseudotime(adata_atac, 't', n_bins)

adata_gex = bin_pseudotime(adata_gex, temporal_ordering, n_bins)
adata_atac = bin_pseudotime(adata_atac, temporal_ordering, n_bins)

adata_aucg = bin_pseudotime(adata_aucg, 't', n_bins)
adata_aucr = bin_pseudotime(adata_aucg, 't', n_bins)

adata_aucg = bin_pseudotime(adata_aucr, temporal_ordering, n_bins)
adata_aucr = bin_pseudotime(adata_aucr, temporal_ordering, n_bins)

sc.pl.umap(adata_gex,color=["seg","milestones","t",temporal_ordering,"fine_grain","t_bins",f"{temporal_ordering}_bins"],ncols=2,wspace=0.8)
```

Example 67 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 68 (unknown):
```unknown
max_group=n_bins-1
max_group=str(max_group)
max_group
```

Example 69 (python):
```python
adata_gex.obs[f"{temporal_ordering}_bins"].value_counts()
```

Example 70 (unknown):
```unknown
0    2832
1    2832
Name: age_days_smoothed_once_bins, dtype: int64
```

Example 71 (python):
```python
increasing_features_dict={}

sc.tl.rank_genes_groups(adata_gex,groupby=f"{temporal_ordering}_bins",groups=[max_group],reference="0")
sc.pl.rank_genes_groups_dotplot(adata_gex,groupby=f"{temporal_ordering}_bins",n_genes=40,dendrogram=False)

df=sc.get.rank_genes_groups_df(adata_gex,group=max_group,log2fc_min=log2fc_min)
df=df[df.pvals_adj<pval_cutoff]
increasing_features_dict["INCREASING_genes"]=df.names.tolist()




sc.tl.rank_genes_groups(adata_atac,groupby=f"{temporal_ordering}_bins",groups=[max_group],reference="0")
sc.pl.rank_genes_groups_dotplot(adata_atac,groupby=f"{temporal_ordering}_bins",n_genes=40,dendrogram=False)

df=sc.get.rank_genes_groups_df(adata_atac,group=max_group,log2fc_min=log2fc_min)
#df=df[df.pvals_adj<pval_cutoff]
df=df[df.pvals<pval_cutoff] #pvals
increasing_features_dict["INCREASING_peaks"]=df.names.tolist()




sc.tl.rank_genes_groups(adata_aucg,groupby=f"{temporal_ordering}_bins",groups=[max_group],reference="0")
sc.pl.rank_genes_groups_dotplot(adata_aucg,groupby=f"{temporal_ordering}_bins",n_genes=40,dendrogram=False)

df=sc.get.rank_genes_groups_df(adata_aucg,group=max_group,log2fc_min=log2fc_min)
#df=df[df.pvals_adj<pval_cutoff]
df=df[df.pvals<pval_cutoff] #pvals
increasing_features_dict["INCREASING_AUCgene"]=df.names.tolist()




sc.tl.rank_genes_groups(adata_aucr,groupby=f"{temporal_ordering}_bins",groups=[max_group],reference="0")
sc.pl.rank_genes_groups_dotplot(adata_aucr,groupby=f"{temporal_ordering}_bins",n_genes=40,dendrogram=False)

df=sc.get.rank_genes_groups_df(adata_aucr,group=max_group,log2fc_min=log2fc_min)
#df=df[df.pvals_adj<pval_cutoff]
df=df[df.pvals<pval_cutoff] #pvals
increasing_features_dict["INCREASING_AUCregion"]=df.names.tolist()
```

Example 72 (python):
```python
WARNING: Default of the method has been changed to 't-test' from 't-test_overestim_var'
```

Example 73 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_dotplot.py:749: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap', 'norm' will be ignored
  dot_ax.scatter(x, y, **kwds)
```

Example 74 (python):
```python
WARNING: Default of the method has been changed to 't-test' from 't-test_overestim_var'
```

Example 75 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_dotplot.py:749: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap', 'norm' will be ignored
  dot_ax.scatter(x, y, **kwds)
```

Example 76 (python):
```python
WARNING: Default of the method has been changed to 't-test' from 't-test_overestim_var'
```

Example 77 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_dotplot.py:749: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap', 'norm' will be ignored
  dot_ax.scatter(x, y, **kwds)
```

Example 78 (python):
```python
WARNING: Default of the method has been changed to 't-test' from 't-test_overestim_var'
```

Example 79 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_dotplot.py:749: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap', 'norm' will be ignored
  dot_ax.scatter(x, y, **kwds)
```

Example 80 (unknown):
```unknown
plot.GO_plot(gene_list=increasing_features_dict["INCREASING_genes"],display_table=True,
             gene_sets=['MSigDB_Hallmark_2020'],
             title_suffix=" AtrialCardiomyocytesCommon increasing over time")
```

Example 81 (unknown):
```unknown
MSigDB_Hallmark_2020
```

Example 82 (unknown):
```unknown
increasing_features_dict
```

Example 83 (python):
```python
from tqdm import tqdm
import pandas as pd

sort_col = "A"
cutoff_col = 'fdr'
cutoff_value = 1e-1

associated_increasing_features_dict = {}

for feature in tqdm(["genes", "peaks", "AUCgene", "AUCregion"]):
    print("\n######### {} #########\n".format(feature))
    
    associated_increasing_NONsidespecific_features = []

    for side in ["LEFT", "RIGHT"]:
        # Load and sort the data
        df = pd.read_csv(f"./fitted_features/aCMs_WholeSample_scfatespseudotime_{feature}_{side}.csv", index_col=0)
        df = df.sort_values(sort_col, ascending=False).drop(columns="activator_repressor")
        print(f"Total number of {feature}_{side} before filtering: {len(df)}")
        
        # Apply cutoff and filter positive 'A' values
        associated_features = df[(df[cutoff_col] < cutoff_value) & (df["A"] > 0.5)].index.tolist()
        print(f"Number of {feature}_{side} after cutoff and 'A' filtering: {len(associated_features)}")
        
        # Filter for increasing features
        increasing_features = increasing_features_dict[f"INCREASING_{feature}"]
        print(f"Number of increasing {feature}: {len(increasing_features)}")
        
        # Find associated increasing features
        associated_increasing_features = [f for f in associated_features if f in increasing_features]
        print(f"Number of increasing {feature}_{side} after associating: {len(associated_increasing_features)}")
        
        # Filter for side-specific features
        side_specific_features = LEFT_or_RIGHT_features[f"{side}_{feature}"]
        print(f"Number of {side}-specific {feature}: {len(side_specific_features)}")
        
        # Find associated increasing side-specific features
        associated_increasing_sidespecific_features = [f for f in associated_increasing_features if f in side_specific_features]
        print(f"Number of associated increasing {side}-specific {feature}: {len(associated_increasing_sidespecific_features)}")
        
        # Find non-side-specific features
        non_side_specific_features = [f for f in associated_increasing_features if f not in side_specific_features]
        print(f"Number of associated increasing non-{side}-specific {feature}: {len(non_side_specific_features)}")
        
        # Combine non-side-specific features for later processing
        associated_increasing_NONsidespecific_features.extend(non_side_specific_features)
        
        # Store side-specific features
        associated_increasing_features_dict[f"{feature}_{side}"] = associated_increasing_sidespecific_features

    # Deduplicate the list of non-side-specific features
    non_side_specific_set = set(associated_increasing_NONsidespecific_features)
    print(f"Total number of associated increasing non-side-specific {feature}: {len(non_side_specific_set)}")

    # Store these NonSideSpecific features
    associated_increasing_features_dict[f"{feature}_NON_SIDE_SPECIFIC"] = list(non_side_specific_set)

    # Identify common features (present in both LEFT and RIGHT)
    common_features = non_side_specific_set.intersection(
        associated_increasing_features_dict[f"{feature}_LEFT"],
        associated_increasing_features_dict[f"{feature}_RIGHT"]
    )
    associated_increasing_features_dict[f"{feature}_COMMON"] = list(common_features)
    print(f"Total number of common non-side-specific {feature}: {len(common_features)}")

# Calculate and display counts for each category
counts_dict = {key: len(value) for key, value in associated_increasing_features_dict.items()}
print("\nCounts per category:")
for key, value in counts_dict.items():
    print(f"{key}: {value}")
```

Example 84 (unknown):
```unknown
0%|                                                                                                                                                                                                                     | 0/4 [00:00<?, ?it/s]
```

Example 85 (unknown):
```unknown
######### genes #########

Total number of genes_LEFT before filtering: 21922
Number of genes_LEFT after cutoff and 'A' filtering: 3626
Number of increasing genes: 1178
Number of increasing genes_LEFT after associating: 577
Number of LEFT-specific genes: 1440
```

Example 86 (unknown):
```unknown
25%|███████████████████████████████████████████████████▎                                                                                                                                                         | 1/4 [00:00<00:01,  2.77it/s]
```

Example 87 (unknown):
```unknown
Number of associated increasing LEFT-specific genes: 98
Number of associated increasing non-LEFT-specific genes: 479
Total number of genes_RIGHT before filtering: 21024
Number of genes_RIGHT after cutoff and 'A' filtering: 3439
Number of increasing genes: 1178
Number of increasing genes_RIGHT after associating: 485
Number of RIGHT-specific genes: 936
Number of associated increasing RIGHT-specific genes: 83
Number of associated increasing non-RIGHT-specific genes: 402
Total number of associated increasing non-side-specific genes: 569
Total number of common non-side-specific genes: 0

######### peaks #########

Total number of peaks_LEFT before filtering: 167061
Number of peaks_LEFT after cutoff and 'A' filtering: 4835
Number of increasing peaks: 5856
Number of increasing peaks_LEFT after associating: 113
Number of LEFT-specific peaks: 132
Number of associated increasing LEFT-specific peaks: 1
Number of associated increasing non-LEFT-specific peaks: 112
Total number of peaks_RIGHT before filtering: 166063
Number of peaks_RIGHT after cutoff and 'A' filtering: 7171
Number of increasing peaks: 5856
```

Example 88 (unknown):
```unknown
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:02<00:00,  1.90it/s]
```

Example 89 (unknown):
```unknown
Number of increasing peaks_RIGHT after associating: 166
Number of RIGHT-specific peaks: 57
Number of associated increasing RIGHT-specific peaks: 0
Number of associated increasing non-RIGHT-specific peaks: 166
Total number of associated increasing non-side-specific peaks: 262
Total number of common non-side-specific peaks: 0

######### AUCgene #########

Total number of AUCgene_LEFT before filtering: 278
Number of AUCgene_LEFT after cutoff and 'A' filtering: 83
Number of increasing AUCgene: 4
Number of increasing AUCgene_LEFT after associating: 4
Number of LEFT-specific AUCgene: 11
Number of associated increasing LEFT-specific AUCgene: 0
Number of associated increasing non-LEFT-specific AUCgene: 4
Total number of AUCgene_RIGHT before filtering: 278
Number of AUCgene_RIGHT after cutoff and 'A' filtering: 85
Number of increasing AUCgene: 4
Number of increasing AUCgene_RIGHT after associating: 4
Number of RIGHT-specific AUCgene: 4
Number of associated increasing RIGHT-specific AUCgene: 0
Number of associated increasing non-RIGHT-specific AUCgene: 4
Total number of associated increasing non-side-specific AUCgene: 4
Total number of common non-side-specific AUCgene: 0

######### AUCregion #########

Total number of AUCregion_LEFT before filtering: 278
Number of AUCregion_LEFT after cutoff and 'A' filtering: 75
Number of increasing AUCregion: 4
Number of increasing AUCregion_LEFT after associating: 3
Number of LEFT-specific AUCregion: 15
Number of associated increasing LEFT-specific AUCregion: 0
Number of associated increasing non-LEFT-specific AUCregion: 3
Total number of AUCregion_RIGHT before filtering: 278
Number of AUCregion_RIGHT after cutoff and 'A' filtering: 72
Number of increasing AUCregion: 4
Number of increasing AUCregion_RIGHT after associating: 3
Number of RIGHT-specific AUCregion: 0
Number of associated increasing RIGHT-specific AUCregion: 0
Number of associated increasing non-RIGHT-specific AUCregion: 3
Total number of associated increasing non-side-specific AUCregion: 4
Total number of common non-side-specific AUCregion: 0

Counts per category:
genes_LEFT: 98
genes_RIGHT: 83
genes_NON_SIDE_SPECIFIC: 569
genes_COMMON: 0
peaks_LEFT: 1
peaks_RIGHT: 0
peaks_NON_SIDE_SPECIFIC: 262
peaks_COMMON: 0
AUCgene_LEFT: 0
AUCgene_RIGHT: 0
AUCgene_NON_SIDE_SPECIFIC: 4
AUCgene_COMMON: 0
AUCregion_LEFT: 0
AUCregion_RIGHT: 0
AUCregion_NON_SIDE_SPECIFIC: 4
AUCregion_COMMON: 0
```

Example 90 (python):
```python
plot_sided_features(adata_gex,
                    feature_type="genes",
                    associated_increasing_features_dict=associated_increasing_features_dict)

plot_sided_features(adata_atac,
                    feature_type="peaks",
                    associated_increasing_features_dict=associated_increasing_features_dict)

plot_sided_features(adata_aucg,
                    feature_type="AUCgene",
                    associated_increasing_features_dict=associated_increasing_features_dict)

plot_sided_features(adata_aucr,
                    feature_type="AUCregion",
                    associated_increasing_features_dict=associated_increasing_features_dict)
```

Example 91 (unknown):
```unknown
LEFT_or_RIGHT_features={}
```

Example 92 (unknown):
```unknown
log2fc_min=2
pval_cutoff=0.05
```

Example 93 (python):
```python
sc.tl.dendrogram(adata_gex,groupby="fine_grain",use_rep="X_multiVI")
sc.tl.rank_genes_groups(adata_gex,groupby="fine_grain")
sc.pl.rank_genes_groups_dotplot(adata_gex,groupby="fine_grain",n_genes=20)

df=sc.get.rank_genes_groups_df(adata,group="AtrialCardiomyocytesLeft",log2fc_min=log2fc_min)
df=df[df.pvals_adj<pval_cutoff]
LEFT_or_RIGHT_features["LEFT_genes"]=df.names.tolist()

df=sc.get.rank_genes_groups_df(adata,group="AtrialCardiomyocytesRight",log2fc_min=log2fc_min)
df=df[df.pvals_adj<pval_cutoff]
LEFT_or_RIGHT_features["RIGHT_genes"]=df.names.tolist()
```

Example 94 (python):
```python
WARNING: Default of the method has been changed to 't-test' from 't-test_overestim_var'
WARNING: Dendrogram not added. Dendrogram is added only when the number of categories to plot > 2
```

Example 95 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_dotplot.py:749: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap', 'norm' will be ignored
  dot_ax.scatter(x, y, **kwds)
```

Example 96 (python):
```python
sc.tl.dendrogram(adata_atac,groupby="fine_grain",use_rep="X_multiVI")
sc.tl.rank_genes_groups(adata_atac,groupby="fine_grain")
sc.pl.rank_genes_groups_dotplot(adata_atac,groupby="fine_grain",n_genes=20)

df=sc.get.rank_genes_groups_df(adata,group="AtrialCardiomyocytesLeft",log2fc_min=log2fc_min)
df=df[df.pvals_adj<pval_cutoff]
print(f"adding {len(df.names.tolist())} features for LEFT...")
LEFT_or_RIGHT_features["LEFT_peaks"]=df.names.tolist()

df=sc.get.rank_genes_groups_df(adata,group="AtrialCardiomyocytesRight",log2fc_min=log2fc_min)
df=df[df.pvals_adj<pval_cutoff]
print(f"adding {len(df.names.tolist())} features for RIGHT...")
LEFT_or_RIGHT_features["RIGHT_peaks"]=df.names.tolist()
```

Example 97 (python):
```python
WARNING: Default of the method has been changed to 't-test' from 't-test_overestim_var'
WARNING: Dendrogram not added. Dendrogram is added only when the number of categories to plot > 2
```

Example 98 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_dotplot.py:749: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap', 'norm' will be ignored
  dot_ax.scatter(x, y, **kwds)
```

Example 99 (unknown):
```unknown
adding 1100 features for LEFT...
adding 414 features for RIGHT...
```

Example 100 (python):
```python
# Instead of doing scanpy DE testing, we use the results of ArchR for this step
df=pd.read_csv("./DAR_testing/L-vs-R_aCM.csv")
df=df[(df.Log2FC>log2fc_min)&(df.FDR<pval_cutoff)]
df["names"]=df["seqnames"].astype(str)+":"+df["start"].astype(str)+"_"+df["end"].astype(str)
print(f"adding {len(df.names.tolist())} features for LEFT...")
LEFT_or_RIGHT_features["LEFT_peaks"]=df.names.tolist()

df=pd.read_csv("./DAR_testing/R-vs-L_aCM.csv")
df=df[(df.Log2FC>log2fc_min)&(df.FDR<pval_cutoff)]
df["names"]=df["seqnames"].astype(str)+":"+df["start"].astype(str)+"_"+df["end"].astype(str)
print(f"adding {len(df.names.tolist())} features for RIGHT...")
LEFT_or_RIGHT_features["RIGHT_peaks"]=df.names.tolist()
```

Example 101 (unknown):
```unknown
adding 71 features for LEFT...
adding 19 features for RIGHT...
```

Example 102 (python):
```python
sc.tl.dendrogram(adata_aucg,groupby="fine_grain",use_rep="X_multiVI")
sc.tl.rank_genes_groups(adata_aucg,groupby="fine_grain")
sc.pl.rank_genes_groups_dotplot(adata_aucg,groupby="fine_grain",n_genes=20)

df=sc.get.rank_genes_groups_df(adata_aucg,group="AtrialCardiomyocytesLeft",log2fc_min=log2fc_min)
df=df[df.pvals_adj<pval_cutoff]
print(f"adding {len(df.names.tolist())} features for LEFT...")
LEFT_or_RIGHT_features["LEFT_AUCgene"]=df.names.tolist()

df=sc.get.rank_genes_groups_df(adata_aucg,group="AtrialCardiomyocytesRight",log2fc_min=log2fc_min)
df=df[df.pvals_adj<pval_cutoff]
print(f"adding {len(df.names.tolist())} features for RIGHT...")
LEFT_or_RIGHT_features["RIGHT_AUCgene"]=df.names.tolist()
```

Example 103 (python):
```python
WARNING: Default of the method has been changed to 't-test' from 't-test_overestim_var'
WARNING: Dendrogram not added. Dendrogram is added only when the number of categories to plot > 2
```

Example 104 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_dotplot.py:749: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap', 'norm' will be ignored
  dot_ax.scatter(x, y, **kwds)
```

Example 105 (unknown):
```unknown
adding 0 features for LEFT...
adding 0 features for RIGHT...
```

Example 106 (python):
```python
sc.tl.dendrogram(adata_aucr,groupby="fine_grain",use_rep="X_multiVI")
sc.tl.rank_genes_groups(adata_aucr,groupby="fine_grain")
sc.pl.rank_genes_groups_dotplot(adata_aucr,groupby="fine_grain",n_genes=20)

df=sc.get.rank_genes_groups_df(adata_aucr,group="AtrialCardiomyocytesLeft",log2fc_min=log2fc_min)
df=df[df.pvals_adj<pval_cutoff]
print(f"adding {len(df.names.tolist())} features for LEFT...")
LEFT_or_RIGHT_features["LEFT_AUCregion"]=df.names.tolist()

df=sc.get.rank_genes_groups_df(adata_aucr,group="AtrialCardiomyocytesRight",log2fc_min=log2fc_min)
df=df[df.pvals_adj<pval_cutoff]
print(f"adding {len(df.names.tolist())} features for RIGHT...")
LEFT_or_RIGHT_features["RIGHT_AUCregion"]=df.names.tolist()
```

Example 107 (python):
```python
WARNING: Default of the method has been changed to 't-test' from 't-test_overestim_var'
WARNING: Dendrogram not added. Dendrogram is added only when the number of categories to plot > 2
```

Example 108 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_dotplot.py:749: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap', 'norm' will be ignored
  dot_ax.scatter(x, y, **kwds)
```

Example 109 (unknown):
```unknown
adding 0 features for LEFT...
adding 0 features for RIGHT...
```

Example 110 (python):
```python
from tqdm import tqdm
import pandas as pd

sort_col="A"
cutff_col = 'fdr'
cutoff_value = 1e-1

associated_features_dict = {}

for feature in tqdm(["genes", "peaks", "AUCgene", "AUCregion"]):
    print("\n")
    print(f"######### {feature} #########")
    print("\n")
    
    associated_increasing_NONsidespecific_features = []

    for side in ["LEFT", "RIGHT"]:
        # Read and sort the CSV file
        df = pd.read_csv(f"./fitted_features/aCMs_WholeSample_scfatespseudotime_{feature}_{side}.csv", index_col=0).sort_values(sort_col,ascending=False)
        df = df.drop(columns="activator_repressor")
        print(f"Total number of {feature}_{side} before filtering: {len(df)}")
        
        associated_features = df[(df[cutff_col] < cutoff_value) & (df["A"] > 1)].index.tolist()
        print(f"Number of {feature}_{side} after cutoff and 'A' filtering: {len(associated_features)}")
        
        # Filter for side-specific features
        side_specific_features = LEFT_or_RIGHT_features[f"{side}_{feature}"]
        print(f"Number of {side}-specific {feature}: {len(side_specific_features)}")
        
        associated_sidespecific_features = [f for f in associated_features if f in side_specific_features]
        print(f"Number of associated {side}-specific {feature}: {len(associated_sidespecific_features)}")
        
        associated_features_dict[f"{feature}_{side}"] = associated_sidespecific_features

    # Read in COMMON features which are associated with pseudotime
    df = pd.read_csv(f"./fitted_features/aCMs_WholeSample_scfatespseudotime_{feature}_COMMON.csv", index_col=0).sort_values(sort_col,ascending=False)
    df = df.drop(columns="activator_repressor")
    print(f"Total number of {feature}_COMMON before filtering: {len(df)}")
    
    associated_features = df[(df[cutff_col] < cutoff_value) & (df["A"] > 1)].index.tolist()
    print(f"Number of {feature}_COMMON after cutoff and 'A' filtering: {len(associated_features)}")

    # Filter out any features which are known to be side-specific
    LEFT_side_specific_features = LEFT_or_RIGHT_features[f"LEFT_{feature}"]
    RIGHT_side_specific_features = LEFT_or_RIGHT_features[f"RIGHT_{feature}"]
    LEFT_or_RIGHT_side_specific_features=LEFT_side_specific_features+RIGHT_side_specific_features
    print(f"Number of (either) side-specific {feature}: {len(LEFT_or_RIGHT_side_specific_features)}")

    associated_non_sidespecific_features = [f for f in associated_features if f not in LEFT_or_RIGHT_side_specific_features]
    print(f"Number of associated non-side-specific {feature}: {len(associated_non_sidespecific_features)}")

    associated_features_dict[f"{feature}_COMMON"]=associated_non_sidespecific_features

# Calculate the counts for each category
counts_dict = {key: len(value) for key, value in associated_features_dict.items()}
print("\nCounts per category:")
for key, value in counts_dict.items():
    print(f"{key}: {value}")
```

Example 111 (unknown):
```unknown
0%|                                                                                                                                                                                                                     | 0/4 [00:00<?, ?it/s]
```

Example 112 (unknown):
```unknown
######### genes #########


Total number of genes_LEFT before filtering: 21922
Number of genes_LEFT after cutoff and 'A' filtering: 2519
Number of LEFT-specific genes: 209
Number of associated LEFT-specific genes: 28
Total number of genes_RIGHT before filtering: 21024
Number of genes_RIGHT after cutoff and 'A' filtering: 2584
Number of RIGHT-specific genes: 129
Number of associated RIGHT-specific genes: 29
Total number of genes_COMMON before filtering: 23953
```

Example 113 (unknown):
```unknown
25%|███████████████████████████████████████████████████▎                                                                                                                                                         | 1/4 [00:00<00:00,  5.31it/s]
```

Example 114 (unknown):
```unknown
Number of genes_COMMON after cutoff and 'A' filtering: 1081
Number of (either) side-specific genes: 338
Number of associated non-side-specific genes: 1048


######### peaks #########


Total number of peaks_LEFT before filtering: 167061
Number of peaks_LEFT after cutoff and 'A' filtering: 1122
Number of LEFT-specific peaks: 71
Number of associated LEFT-specific peaks: 0
Total number of peaks_RIGHT before filtering: 166063
Number of peaks_RIGHT after cutoff and 'A' filtering: 3933
Number of RIGHT-specific peaks: 19
Number of associated RIGHT-specific peaks: 0
```

Example 115 (unknown):
```unknown
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:01<00:00,  3.84it/s]
```

Example 116 (unknown):
```unknown
Total number of peaks_COMMON before filtering: 165795
Number of peaks_COMMON after cutoff and 'A' filtering: 0
Number of (either) side-specific peaks: 90
Number of associated non-side-specific peaks: 0


######### AUCgene #########


Total number of AUCgene_LEFT before filtering: 278
Number of AUCgene_LEFT after cutoff and 'A' filtering: 62
Number of LEFT-specific AUCgene: 0
Number of associated LEFT-specific AUCgene: 0
Total number of AUCgene_RIGHT before filtering: 278
Number of AUCgene_RIGHT after cutoff and 'A' filtering: 69
Number of RIGHT-specific AUCgene: 0
Number of associated RIGHT-specific AUCgene: 0
Total number of AUCgene_COMMON before filtering: 278
Number of AUCgene_COMMON after cutoff and 'A' filtering: 52
Number of (either) side-specific AUCgene: 0
Number of associated non-side-specific AUCgene: 52


######### AUCregion #########


Total number of AUCregion_LEFT before filtering: 278
Number of AUCregion_LEFT after cutoff and 'A' filtering: 50
Number of LEFT-specific AUCregion: 0
Number of associated LEFT-specific AUCregion: 0
Total number of AUCregion_RIGHT before filtering: 278
Number of AUCregion_RIGHT after cutoff and 'A' filtering: 58
Number of RIGHT-specific AUCregion: 0
Number of associated RIGHT-specific AUCregion: 0
Total number of AUCregion_COMMON before filtering: 278
Number of AUCregion_COMMON after cutoff and 'A' filtering: 56
Number of (either) side-specific AUCregion: 0
Number of associated non-side-specific AUCregion: 56

Counts per category:
genes_LEFT: 28
genes_RIGHT: 29
genes_COMMON: 1048
peaks_LEFT: 0
peaks_RIGHT: 0
peaks_COMMON: 0
AUCgene_LEFT: 0
AUCgene_RIGHT: 0
AUCgene_COMMON: 52
AUCregion_LEFT: 0
AUCregion_RIGHT: 0
AUCregion_COMMON: 56
```

Example 117 (python):
```python
# similar to before but we do not filter using the increasing features dictionary
from tqdm import tqdm
import pandas as pd

sort_col = "A"
cutoff_col = 'fdr'
cutoff_value = 1e-1

associated_increasing_features_dict = {}

for feature in tqdm(["genes", "peaks", "AUCgene", "AUCregion"]):
    print("\n######### {} #########\n".format(feature))
    
    associated_increasing_NONsidespecific_features = []

    for side in ["LEFT", "RIGHT"]:
        # Load and sort the data
        df = pd.read_csv(f"./fitted_features/aCMs_WholeSample_scfatespseudotime_{feature}_{side}.csv", index_col=0)
        df = df.sort_values(sort_col, ascending=False).drop(columns="activator_repressor")
        print(f"Total number of {feature}_{side} before filtering: {len(df)}")
        
        # Apply cutoff and filter positive 'A' values
        associated_features = df[(df[cutoff_col] < cutoff_value) & (df["A"] > 0.5)].index.tolist()
        print(f"Number of {feature}_{side} after cutoff and 'A' filtering: {len(associated_features)}")
        
        # Filter for increasing features
        #increasing_features = increasing_features_dict[f"INCREASING_{feature}"]
        #print(f"Number of increasing {feature}: {len(increasing_features)}")
        
        # Find associated increasing features
        #associated_increasing_features = [f for f in associated_features if f in increasing_features]
        #print(f"Number of increasing {feature}_{side} after associating: {len(associated_increasing_features)}")

        associated_increasing_features=associated_features
        
        # Filter for side-specific features
        side_specific_features = LEFT_or_RIGHT_features[f"{side}_{feature}"]
        print(f"Number of {side}-specific {feature}: {len(side_specific_features)}")
        
        # Find associated increasing side-specific features
        associated_increasing_sidespecific_features = [f for f in associated_increasing_features if f in side_specific_features]
        print(f"Number of associated increasing {side}-specific {feature}: {len(associated_increasing_sidespecific_features)}")
        
        # Find non-side-specific features
        non_side_specific_features = [f for f in associated_increasing_features if f not in side_specific_features]
        print(f"Number of associated increasing non-{side}-specific {feature}: {len(non_side_specific_features)}")
        
        # Combine non-side-specific features for later processing
        associated_increasing_NONsidespecific_features.extend(non_side_specific_features)
        
        # Store side-specific features
        associated_increasing_features_dict[f"{feature}_{side}"] = associated_increasing_sidespecific_features

    # Deduplicate the list of non-side-specific features
    non_side_specific_set = set(associated_increasing_NONsidespecific_features)
    print(f"Total number of associated increasing non-side-specific {feature}: {len(non_side_specific_set)}")

    # Store these NonSideSpecific features
    associated_increasing_features_dict[f"{feature}_NON_SIDE_SPECIFIC"] = list(non_side_specific_set)

    # Identify common features (present in both LEFT and RIGHT)
    common_features = non_side_specific_set.intersection(
        associated_increasing_features_dict[f"{feature}_LEFT"],
        associated_increasing_features_dict[f"{feature}_RIGHT"]
    )
    associated_increasing_features_dict[f"{feature}_COMMON"] = list(common_features)
    print(f"Total number of common non-side-specific {feature}: {len(common_features)}")

# Calculate and display counts for each category
counts_dict = {key: len(value) for key, value in associated_increasing_features_dict.items()}
print("\nCounts per category:")
for key, value in counts_dict.items():
    print(f"{key}: {value}")
```

Example 118 (unknown):
```unknown
0%|                                                                                                                                                                                                                     | 0/4 [00:00<?, ?it/s]
```

Example 119 (unknown):
```unknown
######### genes #########

Total number of genes_LEFT before filtering: 21922
Number of genes_LEFT after cutoff and 'A' filtering: 3626
Number of LEFT-specific genes: 209
Number of associated increasing LEFT-specific genes: 50
Number of associated increasing non-LEFT-specific genes: 3576
Total number of genes_RIGHT before filtering: 21024
Number of genes_RIGHT after cutoff and 'A' filtering: 3439
Number of RIGHT-specific genes: 129
Number of associated increasing RIGHT-specific genes: 42
Number of associated increasing non-RIGHT-specific genes: 3397
Total number of associated increasing non-side-specific genes: 4858
Total number of common non-side-specific genes: 0

######### peaks #########

Total number of peaks_LEFT before filtering: 167061
Number of peaks_LEFT after cutoff and 'A' filtering: 4835
Number of LEFT-specific peaks: 71
Number of associated increasing LEFT-specific peaks: 0
Number of associated increasing non-LEFT-specific peaks: 4835
```

Example 120 (unknown):
```unknown
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:00<00:00, 11.19it/s]
```

Example 121 (unknown):
```unknown
Total number of peaks_RIGHT before filtering: 166063
Number of peaks_RIGHT after cutoff and 'A' filtering: 7171
Number of RIGHT-specific peaks: 19
Number of associated increasing RIGHT-specific peaks: 0
Number of associated increasing non-RIGHT-specific peaks: 7171
Total number of associated increasing non-side-specific peaks: 11469
Total number of common non-side-specific peaks: 0

######### AUCgene #########

Total number of AUCgene_LEFT before filtering: 278
Number of AUCgene_LEFT after cutoff and 'A' filtering: 83
Number of LEFT-specific AUCgene: 0
Number of associated increasing LEFT-specific AUCgene: 0
Number of associated increasing non-LEFT-specific AUCgene: 83
Total number of AUCgene_RIGHT before filtering: 278
Number of AUCgene_RIGHT after cutoff and 'A' filtering: 85
Number of RIGHT-specific AUCgene: 0
Number of associated increasing RIGHT-specific AUCgene: 0
Number of associated increasing non-RIGHT-specific AUCgene: 85
Total number of associated increasing non-side-specific AUCgene: 104
Total number of common non-side-specific AUCgene: 0

######### AUCregion #########

Total number of AUCregion_LEFT before filtering: 278
Number of AUCregion_LEFT after cutoff and 'A' filtering: 75
Number of LEFT-specific AUCregion: 0
Number of associated increasing LEFT-specific AUCregion: 0
Number of associated increasing non-LEFT-specific AUCregion: 75
Total number of AUCregion_RIGHT before filtering: 278
Number of AUCregion_RIGHT after cutoff and 'A' filtering: 72
Number of RIGHT-specific AUCregion: 0
Number of associated increasing RIGHT-specific AUCregion: 0
Number of associated increasing non-RIGHT-specific AUCregion: 72
Total number of associated increasing non-side-specific AUCregion: 99
Total number of common non-side-specific AUCregion: 0

Counts per category:
genes_LEFT: 50
genes_RIGHT: 42
genes_NON_SIDE_SPECIFIC: 4858
genes_COMMON: 0
peaks_LEFT: 0
peaks_RIGHT: 0
peaks_NON_SIDE_SPECIFIC: 11469
peaks_COMMON: 0
AUCgene_LEFT: 0
AUCgene_RIGHT: 0
AUCgene_NON_SIDE_SPECIFIC: 104
AUCgene_COMMON: 0
AUCregion_LEFT: 0
AUCregion_RIGHT: 0
AUCregion_NON_SIDE_SPECIFIC: 99
AUCregion_COMMON: 0
```

Example 122 (python):
```python
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import cross_val_score
from matplotlib import pyplot as plt
from matplotlib import cm
import numpy as np


def plot_fitted_expression(adata, gene_list, title, figsize, fname, show_colorbar=True):
    """
    Plots fitted gene expression over pseudotime for a list of genes.

    Parameters:
    adata (AnnData): Annotated data matrix of shape n_obs × n_vars. Rows correspond to cells and columns to genes.
    gene_list (list): List of genes to be plotted.
    title (str): Title of the plot.
    figsize (tuple): Figure size in the format (width, height).
    show_colorbar (bool): If True, display the color bar. Defaults to True.

    The function plots each gene's fitted expression as a function of pseudotime.
    The genes are ordered based on the pseudotime of their maximal expression.
    """
    pseudotime = adata.obs["t"].values  # Convert to numpy array
    pseudotime_reshaped = pseudotime.reshape(-1, 1)
    pseudotime_plot = np.linspace(min(pseudotime), max(pseudotime), 500)
    pseudotime_plot_reshaped = pseudotime_plot.reshape(-1, 1)
    gene_info = {}

    for gene in tqdm(gene_list, desc="Fitting expression..."):
        gene_expression = adata[:, gene].X.toarray()  # Adjust indexing based on your Anndata structure
        degrees = [1, 2, 3, 4, 5]
        best_score = -np.inf
        best_degree = 0

        for degree in degrees:
            model = make_pipeline(PolynomialFeatures(degree), LinearRegression())
            score = cross_val_score(model, pseudotime_reshaped, gene_expression, cv=5).mean()
            if score > best_score:
                best_score = score
                best_degree = degree

        best_model = make_pipeline(PolynomialFeatures(best_degree), LinearRegression())
        best_model.fit(pseudotime_reshaped, gene_expression)
        fitted_expression_plot = best_model.predict(pseudotime_plot_reshaped)
        max_expression_idx = np.argmax(fitted_expression_plot)
        max_expression_time = pseudotime_plot[max_expression_idx]
        gene_info[gene] = {'model': best_model, 'max_time': max_expression_time}

    sorted_genes = sorted(gene_list, key=lambda gene: gene_info[gene]['max_time'], reverse=True)

    fig, ax = plt.subplots(figsize=figsize)
    cmap = cm.RdPu

    for i, gene in enumerate(sorted_genes):
        best_model = gene_info[gene]['model']
        fitted_expression_plot = best_model.predict(pseudotime_plot_reshaped)
        norm = plt.Normalize(fitted_expression_plot.min(), fitted_expression_plot.max())
        sc = plt.scatter(pseudotime_plot, np.ones_like(pseudotime_plot) * (i + 1), 
                         c=fitted_expression_plot, cmap=cmap, norm=norm, marker='s')

    if show_colorbar:
        plt.colorbar(sc, label='Fitted Gene Expression')

    # Remove the box around the plot
    ax.spines['top'].set_visible(False)
    ax.spines['right'].set_visible(False)
    ax.spines['left'].set_visible(False)

    # Hide x-axis ticks and labels
    ax.set_xticks([])  # This removes the x-axis ticks
    ax.set_xlabel('')  # This removes the x-axis label

    # Adjust x-axis position
    ax.set_ylim(0.5, len(sorted_genes) + 0.5)

    plt.xlabel('Pseudotime')
    plt.ylabel('Gene')
    plt.yticks(np.arange(1, len(sorted_genes) + 1), sorted_genes, fontsize='xx-small', fontstyle='italic')
    plt.title(title)
    plt.savefig(fname=fname, bbox_inches='tight')
    plt.show()
```

Example 123 (python):
```python
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import cross_val_score
from matplotlib import pyplot as plt
from matplotlib import cm
import numpy as np


def plot_fitted_expression(adata, gene_list, title, figsize, fname, show_colorbar=True):
    """
    Plots fitted gene expression over pseudotime for a list of genes.

    Parameters:
    adata (AnnData): Annotated data matrix of shape n_obs × n_vars. Rows correspond to cells and columns to genes.
    gene_list (list): List of genes to be plotted.
    title (str): Title of the plot.
    figsize (tuple): Figure size in the format (width, height).
    show_colorbar (bool): If True, display the color bar. Defaults to True.

    The function plots each gene's fitted expression as a function of pseudotime.
    The genes are ordered based on the pseudotime of their maximal expression.
    """
    pseudotime = adata.obs["t"].values  # Convert to numpy array

    # Normalize pseudotime to [0, 1] range
    pseudotime_min = min(pseudotime)
    pseudotime_max = max(pseudotime)
    pseudotime_normalized = (pseudotime - pseudotime_min) / (pseudotime_max - pseudotime_min)

    # Reshape for model fitting
    pseudotime_reshaped = pseudotime_normalized.reshape(-1, 1)
    
    # Create equally spaced values in [0, 1] for plotting
    pseudotime_plot = np.linspace(0, 1, 500)
    pseudotime_plot_reshaped = pseudotime_plot.reshape(-1, 1)
    
    gene_info = {}

    for gene in tqdm(gene_list, desc="Fitting expression..."):
        gene_expression = adata[:, gene].X.toarray()  # Adjust indexing based on your Anndata structure
        degrees = [1, 2, 3, 4, 5]
        best_score = -np.inf
        best_degree = 0

        for degree in degrees:
            model = make_pipeline(PolynomialFeatures(degree), LinearRegression())
            score = cross_val_score(model, pseudotime_reshaped, gene_expression, cv=5).mean()
            if score > best_score:
                best_score = score
                best_degree = degree

        best_model = make_pipeline(PolynomialFeatures(best_degree), LinearRegression())
        best_model.fit(pseudotime_reshaped, gene_expression)
        fitted_expression_plot = best_model.predict(pseudotime_plot_reshaped)
        max_expression_idx = np.argmax(fitted_expression_plot)
        max_expression_time = pseudotime_plot[max_expression_idx]
        gene_info[gene] = {'model': best_model, 'max_time': max_expression_time}

    sorted_genes = sorted(gene_list, key=lambda gene: gene_info[gene]['max_time'], reverse=True)

    fig, ax = plt.subplots(figsize=figsize)
    cmap = cm.RdPu

    for i, gene in enumerate(sorted_genes):
        best_model = gene_info[gene]['model']
        fitted_expression_plot = best_model.predict(pseudotime_plot_reshaped)
        norm = plt.Normalize(fitted_expression_plot.min(), fitted_expression_plot.max())
        sc = plt.scatter(pseudotime_plot, np.ones_like(pseudotime_plot) * (i + 1), 
                         c=fitted_expression_plot, cmap=cmap, norm=norm, marker='s')

    if show_colorbar:
        plt.colorbar(sc, label='Fitted Gene Expression')

    # Remove the box around the plot
    ax.spines['top'].set_visible(False)
    ax.spines['right'].set_visible(False)
    ax.spines['left'].set_visible(False)

    # If you wish, hide x-axis ticks and labels by uncommenting
    #ax.set_xticks([])  # This removes the x-axis ticks
    #ax.set_xlabel('')  # This removes the x-axis label
    
    ax.set_xlim(0, 1)
    ax.set_ylim(0.5, len(sorted_genes) + 0.5)

    plt.xlabel('Pseudotime')
    plt.ylabel('Gene')
    plt.yticks(np.arange(1, len(sorted_genes) + 1), sorted_genes, fontsize='xx-small', fontstyle='italic')
    plt.title(title)
    plt.savefig(fname=fname, bbox_inches='tight')
    plt.show()
```

Example 124 (unknown):
```unknown
associated_increasing_features_dict.keys()
```

Example 125 (unknown):
```unknown
dict_keys(['genes_LEFT', 'genes_RIGHT', 'genes_NON_SIDE_SPECIFIC', 'genes_COMMON', 'peaks_LEFT', 'peaks_RIGHT', 'peaks_NON_SIDE_SPECIFIC', 'peaks_COMMON', 'AUCgene_LEFT', 'AUCgene_RIGHT', 'AUCgene_NON_SIDE_SPECIFIC', 'AUCgene_COMMON', 'AUCregion_LEFT', 'AUCregion_RIGHT', 'AUCregion_NON_SIDE_SPECIFIC', 'AUCregion_COMMON'])
```

Example 126 (python):
```python
figsize=(2,4.5)
top_n_features=28
file_extension=".png"
file_extension=".svg"

plot_fitted_expression(adata_gex[adata_gex.obs.fine_grain=="AtrialCardiomyocytesLeft"], gene_list=associated_features_dict["genes_LEFT"][:top_n_features], title="LEFT atrial cardiomyocyte maturation", figsize=figsize, show_colorbar=False,fname=f"genes_LEFT{file_extension}")
plot_fitted_expression(adata_gex[adata_gex.obs.fine_grain=="AtrialCardiomyocytesRight"], gene_list=associated_features_dict["genes_RIGHT"][:top_n_features], title="RIGHT atrial cardiomyocyte maturation", figsize=figsize, show_colorbar=False,fname=f"genes_RIGHT{file_extension}")

#top_n_features=80
#figsize=(2,10)

plot_fitted_expression(adata_gex, gene_list=associated_features_dict["genes_COMMON"][:top_n_features], title="COMMON atrial cardiomyocyte maturation", figsize=figsize, show_colorbar=False,fname=f"genes_COMMON{file_extension}")
```

Example 127 (unknown):
```unknown
Fitting expression...: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:02<00:00, 10.86it/s]
```

Example 128 (unknown):
```unknown
Fitting expression...: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:01<00:00, 18.62it/s]
```

Example 129 (unknown):
```unknown
Fitting expression...: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:01<00:00, 15.92it/s]
```

Example 130 (python):
```python
gene_list=associated_features_dict["genes_LEFT"]
adata_subset=adata_gex[adata_gex.obs.fine_grain=="AtrialCardiomyocytesLeft"].copy()
distance_threshold=4.5
```

Example 131 (python):
```python
import seaborn as sns
import pandas as pd
import numpy as np
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import PolynomialFeatures, MinMaxScaler
from sklearn.linear_model import LinearRegression
from scipy.spatial.distance import pdist
from scipy.cluster.hierarchy import linkage, fcluster
from tqdm import tqdm
import matplotlib.pyplot as plt

def create_clustermap(adata, gene_list, n_bins, figsize, desired_n_clusters):
    """
    Creates a clustermap of fitted gene expression across pseudotime bins for a list of genes.

    Parameters:
    adata (AnnData): Annotated data matrix.
    gene_list (list): List of genes to be included.
    n_bins (int): Number of bins to divide pseudotime into.
    figsize (tuple): Figure size in the format (width, height).
    """
    # Normalize and reshape pseudotime
    pseudotime = adata.obs["t"].values
    pseudotime_min = min(pseudotime)
    pseudotime_max = max(pseudotime)
    pseudotime_normalized = (pseudotime - pseudotime_min) / (pseudotime_max - pseudotime_min)
    pseudotime_reshaped = pseudotime_normalized.reshape(-1, 1)

    # Create pseudotime bins
    bins = np.linspace(0, 1, n_bins + 1)
    pseudotime_binned = np.digitize(pseudotime_normalized, bins) - 1

    # DataFrame to store fitted expressions
    expression_data = pd.DataFrame(index=gene_list, columns=range(n_bins))

    print(f"Creating clustermap for {len(gene_list)} features across {n_bins} bins")

    for gene in tqdm(gene_list, desc="Fitting expression..."):
        gene_expression = adata[:, gene].X.toarray()

        # Check if gene_expression is empty or almost empty
        if gene_expression.size == 0 or np.count_nonzero(gene_expression) < 5:
            print(f"Insufficient data for gene: {gene}")
            continue

        # Fit the model
        model = make_pipeline(PolynomialFeatures(2), LinearRegression())
        model.fit(pseudotime_reshaped, gene_expression)

        # Predict expression for each bin
        for i in range(n_bins):
            bin_range = np.linspace(bins[i], bins[i + 1], 10).reshape(-1, 1)
            predicted_expression = model.predict(bin_range).mean()
            expression_data.loc[gene, i] = predicted_expression  # Direct assignment
    
    expression_data = expression_data.astype(float).fillna(0)

    # Perform min-max normalization
    scaler = MinMaxScaler()
    expression_data_normalized = pd.DataFrame(scaler.fit_transform(expression_data.T).T, index=gene_list, columns=expression_data.columns) # row-wise (per gene) min-max normalisation


    # Compute the distance matrix for the rows (genes) and then the linkage matrix
    distance_matrix = pdist(expression_data_normalized)
    row_linkage = linkage(distance_matrix, method='average')

    # Define clusters using the row linkage
    clusters = fcluster(row_linkage, desired_n_clusters, criterion='maxclust')

    # Define clusters using a distance threshold
    clusters = fcluster(row_linkage, distance_threshold, criterion='distance')

    # Generate a color palette with seaborn and map cluster labels to colors
    cluster_colors = sns.color_palette("tab20", n_colors=len(np.unique(clusters)))
    row_colors = pd.Series(clusters, index=gene_list).map(dict(zip(np.unique(clusters), cluster_colors)))

    # Create clustermap with row colors
    g = sns.clustermap(expression_data_normalized, row_linkage=row_linkage, figsize=figsize, cmap='RdPu', yticklabels=True, xticklabels=True, col_cluster=False, row_colors=row_colors)
    g.cax.set_visible(False)
    
    plt.setp(g.ax_heatmap.get_yticklabels(), fontsize='xx-small', fontstyle='italic')
    plt.show()

    return row_linkage, clusters

row_linkage, clusters = create_clustermap(adata=adata_subset, gene_list=gene_list, n_bins=500, figsize=(4,30), desired_n_clusters=3)




from collections import defaultdict

def create_cluster_dict(clusters, gene_list):
    """
    Creates a dictionary mapping cluster names to lists of genes, adjusting for zero-based indexing.

    Parameters:
    clusters (array): Array of cluster assignments for each gene.
    gene_list (list): List of genes corresponding to the clusters.

    Returns:
    dict: Dictionary with cluster names as keys and lists of genes as values.
    """
    cluster_dict = defaultdict(list)
    for gene, cluster in zip(gene_list, clusters):
        #print(f"Gene: {gene}, Cluster: {cluster}")  # Diagnostic print
        cluster_name=f"cluster_{cluster - 1}"
        cluster_dict[cluster_name].append(gene)

    return dict(cluster_dict)

# Example usage
cluster_dict = create_cluster_dict(clusters, gene_list=gene_list)

# Check the clusters and the number of genes in each
for cluster, genes in cluster_dict.items():
    print(f"{cluster}: {len(genes)} genes")

cluster_dict.keys()
scores=[]
for cluster_name,cluster_genes in cluster_dict.items():
    score_name=f"{cluster_name}_({len(cluster_genes)} genes)"
    scores.append(score_name)
    sc.tl.score_genes(adata_gex,gene_list=cluster_genes,score_name=f"{score_name}")

sc.pl.umap(adata_gex, color=scores, ncols=4, wspace=0.8, cmap="RdPu")








import numpy as np
from sklearn.metrics import mean_squared_error
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model import LinearRegression
from sklearn.pipeline import make_pipeline

optimal_orders = {}

for score in scores:
    # Prepare the data
    tmp_df = adata_gex.obs[["t", score]]
    X = tmp_df["t"].values.reshape(-1, 1)
    y = tmp_df[score].values

    # Split the data into training and testing sets
    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

    best_order = 1
    best_mse = float('inf')

    # Test different polynomial orders
    for order in range(1, 6):
        model = make_pipeline(PolynomialFeatures(order), LinearRegression())
        model.fit(X_train, y_train)
        predictions = model.predict(X_test)
        mse = mean_squared_error(y_test, predictions)

        if mse < best_mse:
            best_mse = mse
            best_order = order

    optimal_orders[score] = best_order






import matplotlib.pyplot as plt
import seaborn as sns

# Number of scores (subplots)
num_scores = len(scores)

# Create a figure with a row of subplots
fig, axes = plt.subplots(1, num_scores, figsize=(5 * num_scores, 5), sharey=True)

# Track the global min and max for the y-axis
global_min, global_max = float('inf'), float('-inf')

# Plot each score
for i, score in enumerate(scores):
    # Prepare the data
    tmp_df = adata_gex.obs[["t", score, "fine_grain"]]
    
    # Plot for each fine_grain category
    fine_grain_categories = tmp_df["fine_grain"].unique()
    for category in fine_grain_categories:
        category_data = tmp_df[tmp_df["fine_grain"] == category]
        sns.regplot(ax=axes[i], data=category_data, x="t", y=score, order=optimal_orders[score], scatter=False, label=category)
    
    # Set the title and add legend
    axes[i].set_title(score)
    axes[i].legend()

    # Update global min and max
    current_min, current_max = tmp_df[score].min(), tmp_df[score].max()
    global_min, global_max = min(global_min, current_min), max(global_max, current_max)

# Set the same y-axis limits for all subplots
for ax in axes:
    ax.set_ylim(global_min, global_max)

# Adjust layout
plt.tight_layout()
plt.show()
```

Example 132 (unknown):
```unknown
Creating clustermap for 28 features across 500 bins
```

Example 133 (unknown):
```unknown
Fitting expression...: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:02<00:00, 10.55it/s]
```

Example 134 (unknown):
```unknown
cluster_3: 7 genes
cluster_2: 6 genes
cluster_4: 2 genes
cluster_6: 1 genes
cluster_0: 8 genes
cluster_5: 3 genes
cluster_1: 1 genes
```

Example 135 (unknown):
```unknown
cluster_dict["cluster_3"]
```

Example 136 (unknown):
```unknown
['MBP', 'KCNMB2', 'GAN', 'SLC8A3', 'A2M', 'KCNMB2-AS1', 'KSR1']
```

Example 137 (unknown):
```unknown
gene_set_A=cluster_dict["cluster_3"]+cluster_dict["cluster_4"]+cluster_dict["cluster_6"]
gene_set_A
```

Example 138 (unknown):
```unknown
['MBP',
 'KCNMB2',
 'GAN',
 'SLC8A3',
 'A2M',
 'KCNMB2-AS1',
 'KSR1',
 'PANCR',
 'PITX2',
 'DCBLD2']
```

Example 139 (python):
```python
gene_list=associated_features_dict["genes_RIGHT"]
adata_subset=adata_gex[adata_gex.obs.fine_grain=="AtrialCardiomyocytesRight"].copy()
distance_threshold=4.5
```

Example 140 (python):
```python
import seaborn as sns
import pandas as pd
import numpy as np
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import PolynomialFeatures, MinMaxScaler
from sklearn.linear_model import LinearRegression
from scipy.spatial.distance import pdist
from scipy.cluster.hierarchy import linkage, fcluster
from tqdm import tqdm
import matplotlib.pyplot as plt

def create_clustermap(adata, gene_list, n_bins, figsize, desired_n_clusters):
    """
    Creates a clustermap of fitted gene expression across pseudotime bins for a list of genes.

    Parameters:
    adata (AnnData): Annotated data matrix.
    gene_list (list): List of genes to be included.
    n_bins (int): Number of bins to divide pseudotime into.
    figsize (tuple): Figure size in the format (width, height).
    """
    # Normalize and reshape pseudotime
    pseudotime = adata.obs["t"].values
    pseudotime_min = min(pseudotime)
    pseudotime_max = max(pseudotime)
    pseudotime_normalized = (pseudotime - pseudotime_min) / (pseudotime_max - pseudotime_min)
    pseudotime_reshaped = pseudotime_normalized.reshape(-1, 1)

    # Create pseudotime bins
    bins = np.linspace(0, 1, n_bins + 1)
    pseudotime_binned = np.digitize(pseudotime_normalized, bins) - 1

    # DataFrame to store fitted expressions
    expression_data = pd.DataFrame(index=gene_list, columns=range(n_bins))

    print(f"Creating clustermap for {len(gene_list)} features across {n_bins} bins")

    for gene in tqdm(gene_list, desc="Fitting expression..."):
        gene_expression = adata[:, gene].X.toarray()

        # Check if gene_expression is empty or almost empty
        if gene_expression.size == 0 or np.count_nonzero(gene_expression) < 5:
            print(f"Insufficient data for gene: {gene}")
            continue

        # Fit the model
        model = make_pipeline(PolynomialFeatures(2), LinearRegression())
        model.fit(pseudotime_reshaped, gene_expression)

        # Predict expression for each bin
        for i in range(n_bins):
            bin_range = np.linspace(bins[i], bins[i + 1], 10).reshape(-1, 1)
            predicted_expression = model.predict(bin_range).mean()
            expression_data.loc[gene, i] = predicted_expression  # Direct assignment
    
    expression_data = expression_data.astype(float).fillna(0)

    # Perform min-max normalization
    scaler = MinMaxScaler()
    expression_data_normalized = pd.DataFrame(scaler.fit_transform(expression_data.T).T, index=gene_list, columns=expression_data.columns) # row-wise (per gene) min-max normalisation


    # Compute the distance matrix for the rows (genes) and then the linkage matrix
    distance_matrix = pdist(expression_data_normalized)
    row_linkage = linkage(distance_matrix, method='average')

    # Define clusters using the row linkage
    clusters = fcluster(row_linkage, desired_n_clusters, criterion='maxclust')

    # Define clusters using a distance threshold
    clusters = fcluster(row_linkage, distance_threshold, criterion='distance')

    # Generate a color palette with seaborn and map cluster labels to colors
    cluster_colors = sns.color_palette("tab20", n_colors=len(np.unique(clusters)))
    row_colors = pd.Series(clusters, index=gene_list).map(dict(zip(np.unique(clusters), cluster_colors)))

    # Create clustermap with row colors
    g = sns.clustermap(expression_data_normalized, row_linkage=row_linkage, figsize=figsize, cmap='RdPu', yticklabels=True, xticklabels=True, col_cluster=False, row_colors=row_colors)
    g.cax.set_visible(False)
    
    plt.setp(g.ax_heatmap.get_yticklabels(), fontsize='xx-small', fontstyle='italic')
    plt.show()

    return row_linkage, clusters

row_linkage, clusters = create_clustermap(adata=adata_subset, gene_list=gene_list, n_bins=500, figsize=(4,30), desired_n_clusters=3)




from collections import defaultdict

def create_cluster_dict(clusters, gene_list):
    """
    Creates a dictionary mapping cluster names to lists of genes, adjusting for zero-based indexing.

    Parameters:
    clusters (array): Array of cluster assignments for each gene.
    gene_list (list): List of genes corresponding to the clusters.

    Returns:
    dict: Dictionary with cluster names as keys and lists of genes as values.
    """
    cluster_dict = defaultdict(list)
    for gene, cluster in zip(gene_list, clusters):
        #print(f"Gene: {gene}, Cluster: {cluster}")  # Diagnostic print
        cluster_name=f"cluster_{cluster - 1}"
        cluster_dict[cluster_name].append(gene)

    return dict(cluster_dict)

# Example usage
cluster_dict = create_cluster_dict(clusters, gene_list=gene_list)

# Check the clusters and the number of genes in each
for cluster, genes in cluster_dict.items():
    print(f"{cluster}: {len(genes)} genes")

cluster_dict.keys()
scores=[]
for cluster_name,cluster_genes in cluster_dict.items():
    score_name=f"{cluster_name}_({len(cluster_genes)} genes)"
    scores.append(score_name)
    sc.tl.score_genes(adata_gex,gene_list=cluster_genes,score_name=f"{score_name}")

sc.pl.umap(adata_gex, color=scores, ncols=4, wspace=0.8, cmap="RdPu")








import numpy as np
from sklearn.metrics import mean_squared_error
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model import LinearRegression
from sklearn.pipeline import make_pipeline

optimal_orders = {}

for score in scores:
    # Prepare the data
    tmp_df = adata_gex.obs[["t", score]]
    X = tmp_df["t"].values.reshape(-1, 1)
    y = tmp_df[score].values

    # Split the data into training and testing sets
    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

    best_order = 1
    best_mse = float('inf')

    # Test different polynomial orders
    for order in range(1, 6):
        model = make_pipeline(PolynomialFeatures(order), LinearRegression())
        model.fit(X_train, y_train)
        predictions = model.predict(X_test)
        mse = mean_squared_error(y_test, predictions)

        if mse < best_mse:
            best_mse = mse
            best_order = order

    optimal_orders[score] = best_order






import matplotlib.pyplot as plt
import seaborn as sns

# Number of scores (subplots)
num_scores = len(scores)

# Create a figure with a row of subplots
fig, axes = plt.subplots(1, num_scores, figsize=(5 * num_scores, 5), sharey=True)

# Track the global min and max for the y-axis
global_min, global_max = float('inf'), float('-inf')

# Plot each score
for i, score in enumerate(scores):
    # Prepare the data
    tmp_df = adata_gex.obs[["t", score, "fine_grain"]]
    
    # Plot for each fine_grain category
    fine_grain_categories = tmp_df["fine_grain"].unique()
    for category in fine_grain_categories:
        category_data = tmp_df[tmp_df["fine_grain"] == category]
        sns.regplot(ax=axes[i], data=category_data, x="t", y=score, order=optimal_orders[score], scatter=False, label=category)
    
    # Set the title and add legend
    axes[i].set_title(score)
    axes[i].legend()

    # Update global min and max
    current_min, current_max = tmp_df[score].min(), tmp_df[score].max()
    global_min, global_max = min(global_min, current_min), max(global_max, current_max)

# Set the same y-axis limits for all subplots
for ax in axes:
    ax.set_ylim(global_min, global_max)

# Adjust layout
plt.tight_layout()
plt.show()
```

Example 141 (unknown):
```unknown
Creating clustermap for 29 features across 500 bins
```

Example 142 (unknown):
```unknown
Fitting expression...: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 29/29 [00:02<00:00, 10.71it/s]
```

Example 143 (unknown):
```unknown
cluster_1: 8 genes
cluster_4: 5 genes
cluster_2: 4 genes
cluster_3: 10 genes
cluster_0: 2 genes
```

Example 144 (unknown):
```unknown
gene_set_B=cluster_dict["cluster_2"]+cluster_dict["cluster_1"]
gene_set_B
```

Example 145 (unknown):
```unknown
['DAB1',
 'UST',
 'ANGPT1',
 'GNAL',
 'KIF26B',
 'KCNQ5',
 'HEY1',
 'HSPA6',
 'RFTN1',
 'MAF',
 'OTOGL',
 'PDE1A']
```

Example 146 (python):
```python
sc.pl.umap(adata_gex, color=["milestones","seg"], ncols=4, wspace=0.8)
```

Example 147 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 148 (python):
```python
import pandas as pd

# Assuming adata_gex is your AnnData object containing scRNAseq data
# and it has 't' and 'milestones' in adata_gex.obs

# Filter the rows where 'milestones' is '60'
filtered_data = adata_gex.obs[adata_gex.obs['milestones'] == '60']

# Find the maximum 't' value in this filtered group
bifurcation_point = filtered_data['t'].median()

# Output the bifurcation point
print("Bifurcation Point:", bifurcation_point)
```

Example 149 (unknown):
```unknown
Bifurcation Point: 1.9802904538320618
```

Example 150 (python):
```python
import numpy as np
import pandas as pd

# Assuming adata_gex is your AnnData object containing scRNAseq data
# and bifurcation_point is already defined as shown previously

# Define the number of bins you want
X = 4  # Adjust this value as needed

# Get the minimum 't' value from adata_gex
min_t_value = adata_gex.obs['t'].min()

# Create bins between the minimum 't' value and the bifurcation point
bins = np.linspace(min_t_value, bifurcation_point, X + 1)  # X+1 because we need X bins

# Assign each cell to a pseudotime bin or "post_bifurcation"
adata_gex.obs['pseudotime_bin'] = pd.cut(adata_gex.obs['t'], bins=bins, include_lowest=True, 
                                         labels=[f'pseudotime_bin_{i}' for i in range(1, X + 1)])
adata_gex.obs['pseudotime_bin'] = adata_gex.obs['pseudotime_bin'].cat.add_categories('post_bifurcation')
adata_gex.obs.loc[adata_gex.obs['t'] > bifurcation_point, 'pseudotime_bin'] = 'post_bifurcation'

# You can now work with the 'pseudotime_bin' column in adata_gex.obs for further analysis
sc.pl.umap(adata_gex, color=["pseudotime_bin"], ncols=1, wspace=0.8, size=50)
```

Example 151 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 152 (python):
```python
adata_gex.obs.fine_grain.value_counts()
```

Example 153 (unknown):
```unknown
AtrialCardiomyocytesLeft     3041
AtrialCardiomyocytesRight    2623
Name: fine_grain, dtype: int64
```

Example 154 (python):
```python
import matplotlib
matplotlib.rcParams['pdf.fonttype'] = 42
matplotlib.rcParams['ps.fonttype'] = 42
```

Example 155 (python):
```python
# Step 1: Copy values from 'fine_grain' to the new column
adata_gex.obs['AtrialCardiomyocytesImmature_annotation'] = adata_gex.obs['fine_grain']

# Add new category to the column
new_category = "AtrialCardiomyocytesImmature"
if new_category not in adata_gex.obs['AtrialCardiomyocytesImmature_annotation'].cat.categories:
    adata_gex.obs['AtrialCardiomyocytesImmature_annotation'] = adata_gex.obs['AtrialCardiomyocytesImmature_annotation'].cat.add_categories(new_category)

# Step 2: Modify values for cells not equal to "post_bifurcation" in 'pseudotime_bin'
adata_gex.obs.loc[adata_gex.obs['pseudotime_bin'] != 'post_bifurcation', 'AtrialCardiomyocytesImmature_annotation'] = new_category

# Verify the value counts
print(adata_gex.obs['AtrialCardiomyocytesImmature_annotation'].value_counts())
print("saving annotation for use with c2l...")
adata_gex.obs['AtrialCardiomyocytesImmature_annotation'].to_csv("AtrialCardiomyocytesImmature_annotation.csv")
sc.pl.umap(adata_gex,color="AtrialCardiomyocytesImmature_annotation",save="_AtrialCardiomyocytesImmature_annotation.pdf")
```

Example 156 (unknown):
```unknown
AtrialCardiomyocytesLeft        2828
AtrialCardiomyocytesRight       2509
AtrialCardiomyocytesImmature     327
Name: AtrialCardiomyocytesImmature_annotation, dtype: int64
saving annotation for use with c2l...
WARNING: saving figure to file figures/umap_AtrialCardiomyocytesImmature_annotation.pdf
```

Example 157 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 158 (python):
```python
# Add updated label - we have 2 young and 2 old populations

immature_barcodes = adata_gex.obs[adata_gex.obs['AtrialCardiomyocytesImmature_annotation']=="AtrialCardiomyocytesImmature"].index

# Create a dictionary to hold the updates
updates = {}

for barcode in immature_barcodes:
    if barcode in adata_gex.obs['fine_grain'].index:
        fine_grain_annotation = adata_gex.obs['fine_grain'][barcode]
        if fine_grain_annotation == 'AtrialCardiomyocytesLeft':
            updates[barcode] = 'AtrialCardiomyocytesLeft_immature'
        elif fine_grain_annotation == 'AtrialCardiomyocytesRight':
            updates[barcode] = 'AtrialCardiomyocytesRight_immature'

adata_gex.obs['AtrialCardiomyocytesImmature_annotation_v2'] = adata_gex.obs['AtrialCardiomyocytesImmature_annotation']

# First, add the new categories to the 'AtrialCardiomyocytesImmature_annotation_v2' column
new_categories = ['AtrialCardiomyocytesLeft_immature', 'AtrialCardiomyocytesRight_immature']
adata_gex.obs['AtrialCardiomyocytesImmature_annotation_v2'] = adata_gex.obs['AtrialCardiomyocytesImmature_annotation_v2'].cat.add_categories(new_categories)

# Then, you can apply the updates from the dictionary as before
for barcode, new_annotation in updates.items():
    adata_gex.obs.at[barcode, 'AtrialCardiomyocytesImmature_annotation_v2'] = new_annotation
    
# Remove unused categories from 'AtrialCardiomyocytesImmature_annotation_v2' column
adata_gex.obs['AtrialCardiomyocytesImmature_annotation_v2'] = adata_gex.obs['AtrialCardiomyocytesImmature_annotation_v2'].cat.remove_unused_categories()

# Check the categories after removal of unused ones
adata_gex.obs['AtrialCardiomyocytesImmature_annotation_v2'].value_counts()
```

Example 159 (unknown):
```unknown
AtrialCardiomyocytesLeft              2828
AtrialCardiomyocytesRight             2509
AtrialCardiomyocytesLeft_immature      213
AtrialCardiomyocytesRight_immature     114
Name: AtrialCardiomyocytesImmature_annotation_v2, dtype: int64
```

Example 160 (python):
```python
sc.pl.umap(adata_gex,color="AtrialCardiomyocytesImmature_annotation_v2",save="_AtrialCardiomyocytesImmature_annotation_v2.pdf")
```

Example 161 (unknown):
```unknown
WARNING: saving figure to file figures/umap_AtrialCardiomyocytesImmature_annotation_v2.pdf
```

Example 162 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 163 (python):
```python
adata_gex.X=adata_gex.layers["counts"].copy()
sc.pp.normalize_total(adata_gex, target_sum=1e4)
sc.pp.log1p(adata_gex)
```

Example 164 (python):
```python
WARNING: adata.X seems to be already log-transformed.
```

Example 165 (python):
```python
import matplotlib.pyplot as plt
import seaborn as sns
import pandas as pd
import scipy.stats
from scipy.sparse import issparse
from tqdm import tqdm
import scanpy as sc
import numpy as np

# Function to calculate the correlation matrix and scatter plot data
def calculate_correlation(bin_cells, gene_set_A, gene_set_B, gene_indices):
    combined_gene_set = gene_set_A + gene_set_B
    # Initialize an empty DataFrame for the correlation matrix
    correlation_matrix = pd.DataFrame(index=combined_gene_set, columns=combined_gene_set)
    scatter_data = []

    # Calculate correlations for each pair of genes
    for gene1 in tqdm(combined_gene_set):
        for gene2 in combined_gene_set:
            # Extract expression values for gene1 and gene2
            x_values = bin_cells.X[:, gene_indices[gene1]].toarray().flatten() if issparse(bin_cells.X) else bin_cells.X[:, gene_indices[gene1]].flatten()
            y_values = bin_cells.X[:, gene_indices[gene2]].toarray().flatten() if issparse(bin_cells.X) else bin_cells.X[:, gene_indices[gene2]].flatten()

            # Calculate Pearson correlation
            correlation, _ = scipy.stats.pearsonr(x_values, y_values)

            # Store the correlation in the matrix
            correlation_matrix.at[gene1, gene2] = correlation

    # Calculate mean correlation for each gene with gene_set_A and gene_set_B
    for gene in combined_gene_set:
        mean_corr_with_A = correlation_matrix.loc[gene, gene_set_A].mean()
        mean_corr_with_B = correlation_matrix.loc[gene, gene_set_B].mean()
        gene_set_label = 'Left aCM early module' if gene in gene_set_A else 'Right aCM early module'

        scatter_data.append({'gene': gene, 'mean_corr_with_A': mean_corr_with_A, 'mean_corr_with_B': mean_corr_with_B, 'gene_set': gene_set_label})

    scatter_df = pd.DataFrame(scatter_data)

    return scatter_df

# Function to add labels to the genes closest to the bottom right and top left of each scatter plot
def label_extreme_genes(scatter_df, ax):
    # Define bottom right and top left points
    bottom_right_point = (1, -1)
    top_left_point = (-1, 1)

    # Function to calculate Euclidean distance
    def euclidean_distance(point1, point2):
        return np.sqrt((point1[0] - point2[0])**2 + (point1[1] - point2[1])**2)

    # Find the gene closest to the bottom right
    scatter_df['distance_to_bottom_right'] = scatter_df.apply(lambda row: euclidean_distance((row['mean_corr_with_A'], row['mean_corr_with_B']), bottom_right_point), axis=1)
    bottom_right_gene = scatter_df.loc[scatter_df['distance_to_bottom_right'].idxmin()]

    # Find the gene closest to the top left
    scatter_df['distance_to_top_left'] = scatter_df.apply(lambda row: euclidean_distance((row['mean_corr_with_A'], row['mean_corr_with_B']), top_left_point), axis=1)
    top_left_gene = scatter_df.loc[scatter_df['distance_to_top_left'].idxmin()]

    # Add labels to the plot
    ax.text(bottom_right_gene['mean_corr_with_A'], bottom_right_gene['mean_corr_with_B'], bottom_right_gene['gene'], color='blue', ha='right')
    ax.text(top_left_gene['mean_corr_with_A'], top_left_gene['mean_corr_with_B'], top_left_gene['gene'], color='red', ha='left')

# Assuming gene_set_A and gene_set_B are lists of gene names
# Assuming adata_gex is your AnnData object containing scRNAseq data
gene_indices = {gene: adata_gex.var.index.get_loc(gene) for gene in set(gene_set_A + gene_set_B)}

# Get all unique categories from 'pseudotime_bin' and move 'post_bifurcation' to the end
bin_categories = [cat for cat in adata_gex.obs['pseudotime_bin'].cat.categories if cat != 'post_bifurcation']
bin_categories.append('post_bifurcation')

# Number of bins to create subplots for
num_bins = len(adata_gex.obs['pseudotime_bin'].cat.categories)

# Create a figure with two rows of subplots: one for UMAP and one for scatter plots
fig, axes = plt.subplots(2, num_bins, figsize=(6 * num_bins, 12))

# Loop over each pseudotime bin to create UMAP and scatter plots
for i, bin_label in enumerate(adata_gex.obs['pseudotime_bin'].cat.categories):
    # Skip the post_bifurcation bin if needed
    #if bin_label == 'post_bifurcation':
    #    continue

    # Filter cells in the current bin for scatter plot
    bin_cells = adata_gex[adata_gex.obs['pseudotime_bin'] == bin_label]

    # Calculate scatter plot data
    scatter_df = calculate_correlation(bin_cells, gene_set_A, gene_set_B, gene_indices)

    # Create UMAP plot in the first row
    ax_umap = axes[0, i]
    sc.pl.umap(adata_gex, color="pseudotime_bin", groups=[bin_label], ax=ax_umap, show=False, legend_loc=None, title="", size=50)

    # Create specific scatter plot in the second row
    ax_scatter = axes[1, i]
    sns.scatterplot(data=scatter_df, x='mean_corr_with_A', y='mean_corr_with_B', hue='gene_set', ax=ax_scatter)
    ax_scatter.set_title(f'{bin_label}')
    ax_scatter.axhline(0, color='grey', linestyle='--')
    ax_scatter.axvline(0, color='grey', linestyle='--')
    ax_scatter.set_xlabel('Correlation with Left aCM early module')
    ax_scatter.set_ylabel('Correlation with Right aCM early module')
    ax_scatter.set_xlim(-1, 1)
    ax_scatter.set_ylim(-1, 1)
    ax_scatter.legend(title='Gene Module')

    # Label the extreme genes
    label_extreme_genes(scatter_df, ax_scatter)

plt.tight_layout()
plt.savefig("./figures/bifurcation_analysis.pdf",bbox_inches="tight")
plt.show()
```

Example 166 (unknown):
```unknown
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 22/22 [00:01<00:00, 17.03it/s]
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
  0%|                                                                                                                                              | 0/22 [00:00<?, ?it/s]/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scipy/stats/_stats_py.py:4781: ConstantInputWarning: An input array is constant; the correlation coefficient is not defined.
  warnings.warn(stats.ConstantInputWarning(msg))
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 22/22 [00:01<00:00, 19.90it/s]
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
  0%|                                                                                                                                              | 0/22 [00:00<?, ?it/s]/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scipy/stats/_stats_py.py:4781: ConstantInputWarning: An input array is constant; the correlation coefficient is not defined.
  warnings.warn(stats.ConstantInputWarning(msg))
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 22/22 [00:01<00:00, 20.41it/s]
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
  0%|                                                                                                                                              | 0/22 [00:00<?, ?it/s]/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scipy/stats/_stats_py.py:4781: ConstantInputWarning: An input array is constant; the correlation coefficient is not defined.
  warnings.warn(stats.ConstantInputWarning(msg))
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 22/22 [00:01<00:00, 21.44it/s]
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 22/22 [00:58<00:00,  2.67s/it]
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 167 (python):
```python
import matplotlib.pyplot as plt
import seaborn as sns
import pandas as pd
import scipy.stats
from scipy.sparse import issparse
from tqdm import tqdm
import scanpy as sc
import numpy as np

# Function to calculate the correlation matrix and scatter plot data
def calculate_correlation(bin_cells, gene_set_A, gene_set_B, gene_indices):
    combined_gene_set = gene_set_A + gene_set_B
    # Initialize an empty DataFrame for the correlation matrix
    correlation_matrix = pd.DataFrame(index=combined_gene_set, columns=combined_gene_set)
    scatter_data = []

    # Calculate correlations for each pair of genes
    for gene1 in tqdm(combined_gene_set):
        for gene2 in combined_gene_set:
            # Extract expression values for gene1 and gene2
            x_values = bin_cells.X[:, gene_indices[gene1]].toarray().flatten() if issparse(bin_cells.X) else bin_cells.X[:, gene_indices[gene1]].flatten()
            y_values = bin_cells.X[:, gene_indices[gene2]].toarray().flatten() if issparse(bin_cells.X) else bin_cells.X[:, gene_indices[gene2]].flatten()

            # Calculate Pearson correlation
            correlation, _ = scipy.stats.pearsonr(x_values, y_values)

            # Store the correlation in the matrix
            correlation_matrix.at[gene1, gene2] = correlation

    # Calculate mean correlation for each gene with gene_set_A and gene_set_B
    for gene in combined_gene_set:
        mean_corr_with_A = correlation_matrix.loc[gene, gene_set_A].mean()
        mean_corr_with_B = correlation_matrix.loc[gene, gene_set_B].mean()
        gene_set_label = 'Left aCM early module' if gene in gene_set_A else 'Right aCM early module'

        scatter_data.append({'gene': gene, 'mean_corr_with_A': mean_corr_with_A, 'mean_corr_with_B': mean_corr_with_B, 'gene_set': gene_set_label})

    scatter_df = pd.DataFrame(scatter_data)

    return scatter_df

# Function to add labels to the genes closest to the bottom right and top left of each scatter plot
def label_extreme_genes(scatter_df, ax):
    # Define bottom right and top left points
    bottom_right_point = (1, -1)
    top_left_point = (-1, 1)

    # Function to calculate Euclidean distance
    def euclidean_distance(point1, point2):
        return np.sqrt((point1[0] - point2[0])**2 + (point1[1] - point2[1])**2)

    # Find the gene closest to the bottom right
    scatter_df['distance_to_bottom_right'] = scatter_df.apply(lambda row: euclidean_distance((row['mean_corr_with_A'], row['mean_corr_with_B']), bottom_right_point), axis=1)
    bottom_right_gene = scatter_df.loc[scatter_df['distance_to_bottom_right'].idxmin()]

    # Find the gene closest to the top left
    scatter_df['distance_to_top_left'] = scatter_df.apply(lambda row: euclidean_distance((row['mean_corr_with_A'], row['mean_corr_with_B']), top_left_point), axis=1)
    top_left_gene = scatter_df.loc[scatter_df['distance_to_top_left'].idxmin()]

    # Add labels to the plot
    ax.text(bottom_right_gene['mean_corr_with_A'], bottom_right_gene['mean_corr_with_B'], bottom_right_gene['gene'], color='blue', ha='right')
    ax.text(top_left_gene['mean_corr_with_A'], top_left_gene['mean_corr_with_B'], top_left_gene['gene'], color='red', ha='left')

# Assuming gene_set_A and gene_set_B are lists of gene names
# Assuming adata_gex is your AnnData object containing scRNAseq data
gene_indices = {gene: adata_gex.var.index.get_loc(gene) for gene in set(gene_set_A + gene_set_B)}

# Get all unique categories from 'pseudotime_bin' and move 'post_bifurcation' to the end
bin_categories = [cat for cat in adata_gex.obs['pseudotime_bin'].cat.categories if cat != 'post_bifurcation']
bin_categories.append('post_bifurcation')

# Number of bins to create subplots for
num_bins = len(adata_gex.obs['pseudotime_bin'].cat.categories)

# Create a figure with two rows of subplots: one for UMAP and one for scatter plots
fig, axes = plt.subplots(2, num_bins, figsize=(6 * num_bins, 12))

# Loop over each pseudotime bin to create UMAP and scatter plots
for i, bin_label in enumerate(adata_gex.obs['pseudotime_bin'].cat.categories):
    # Skip the post_bifurcation bin if needed
    #if bin_label == 'post_bifurcation':
    #    continue

    # Filter cells in the current bin for scatter plot
    bin_cells = adata_gex[adata_gex.obs['pseudotime_bin'] == bin_label]

    # Calculate scatter plot data
    scatter_df = calculate_correlation(bin_cells, gene_set_A, gene_set_B, gene_indices)

    # Create UMAP plot in the first row
    ax_umap = axes[0, i]
    sc.pl.umap(adata_gex, color="pseudotime_bin", groups=[bin_label], ax=ax_umap, show=False, legend_loc=None, title="", size=50)

    # Create specific scatter plot in the second row
    ax_scatter = axes[1, i]
    sns.scatterplot(data=scatter_df, x='mean_corr_with_A', y='mean_corr_with_B', hue='gene_set', ax=ax_scatter)
    ax_scatter.set_title(f'{bin_label}')
    ax_scatter.axhline(0, color='grey', linestyle='--')
    ax_scatter.axvline(0, color='grey', linestyle='--')
    ax_scatter.set_xlabel('Correlation with Left aCM early module')
    ax_scatter.set_ylabel('Correlation with Right aCM early module')
    ax_scatter.set_xlim(-1, 1)
    ax_scatter.set_ylim(-1, 1)
    ax_scatter.legend(title='Gene Module')

    # Label the extreme genes
    label_extreme_genes(scatter_df, ax_scatter)

plt.tight_layout()
plt.savefig("./figures/bifurcation_analysis.pdf",bbox_inches="tight")
plt.show()
```

Example 168 (unknown):
```unknown
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 22/22 [00:01<00:00, 19.79it/s]
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
  0%|                                                                                                                                              | 0/22 [00:00<?, ?it/s]/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scipy/stats/_stats_py.py:4781: ConstantInputWarning: An input array is constant; the correlation coefficient is not defined.
  warnings.warn(stats.ConstantInputWarning(msg))
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 22/22 [00:01<00:00, 20.46it/s]
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
  0%|                                                                                                                                              | 0/22 [00:00<?, ?it/s]/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scipy/stats/_stats_py.py:4781: ConstantInputWarning: An input array is constant; the correlation coefficient is not defined.
  warnings.warn(stats.ConstantInputWarning(msg))
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 22/22 [00:01<00:00, 21.47it/s]
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
  0%|                                                                                                                                              | 0/22 [00:00<?, ?it/s]/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scipy/stats/_stats_py.py:4781: ConstantInputWarning: An input array is constant; the correlation coefficient is not defined.
  warnings.warn(stats.ConstantInputWarning(msg))
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 22/22 [00:01<00:00, 21.43it/s]
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 22/22 [00:58<00:00,  2.67s/it]
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 169 (python):
```python
# Create a figure with two rows of subplots: one for UMAP and one for scatter plots with gene names
fig, axes = plt.subplots(2, num_bins, figsize=(6 * num_bins, 12))

# Loop over each pseudotime bin to create UMAP and scatter plots
for i, bin_label in enumerate(adata_gex.obs['pseudotime_bin'].cat.categories):
    # Filter cells in the current bin for scatter plot
    bin_cells = adata_gex[adata_gex.obs['pseudotime_bin'] == bin_label]

    # Calculate scatter plot data
    scatter_df = calculate_correlation(bin_cells, gene_set_A, gene_set_B, gene_indices)

    # Create UMAP plot in the first row
    ax_umap = axes[0, i]
    sc.pl.umap(adata_gex, color="pseudotime_bin", groups=[bin_label], ax=ax_umap, show=False, legend_loc=None, title=bin_label, size=50)

    # Create scatter plot in the second row with gene names
    ax_scatter = axes[1, i]
    # Instead of using sns.scatterplot, we loop through the scatter_df to place text
    for _, row in scatter_df.iterrows():
        ax_scatter.text(row['mean_corr_with_A'], row['mean_corr_with_B'], row['gene'], fontsize=9)
    
    ax_scatter.set_title(f'{bin_label}')
    ax_scatter.axhline(0, color='grey', linestyle='--')
    ax_scatter.axvline(0, color='grey', linestyle='--')
    ax_scatter.set_xlabel('Correlation with Left aCM early module')
    ax_scatter.set_ylabel('Correlation with Right aCM early module')
    ax_scatter.set_xlim(-0.5, 0.5)
    ax_scatter.set_ylim(-0.5, 0.5)

plt.tight_layout()
#plt.savefig("./figures/bifurcation_analysis.pdf",bbox_inches="tight")
plt.show()
```

Example 170 (unknown):
```unknown
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 22/22 [00:01<00:00, 19.95it/s]
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
  0%|                                                                                                                                              | 0/22 [00:00<?, ?it/s]/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scipy/stats/_stats_py.py:4781: ConstantInputWarning: An input array is constant; the correlation coefficient is not defined.
  warnings.warn(stats.ConstantInputWarning(msg))
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 22/22 [00:01<00:00, 20.54it/s]
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
  0%|                                                                                                                                              | 0/22 [00:00<?, ?it/s]/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scipy/stats/_stats_py.py:4781: ConstantInputWarning: An input array is constant; the correlation coefficient is not defined.
  warnings.warn(stats.ConstantInputWarning(msg))
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 22/22 [00:01<00:00, 20.61it/s]
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
  0%|                                                                                                                                              | 0/22 [00:00<?, ?it/s]/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scipy/stats/_stats_py.py:4781: ConstantInputWarning: An input array is constant; the correlation coefficient is not defined.
  warnings.warn(stats.ConstantInputWarning(msg))
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 22/22 [00:01<00:00, 21.67it/s]
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 22/22 [00:58<00:00,  2.68s/it]
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
posx and posy should be finite values
posx and posy should be finite values
posx and posy should be finite values
posx and posy should be finite values
posx and posy should be finite values
posx and posy should be finite values
```

Example 171 (python):
```python
import matplotlib.pyplot as plt
import seaborn as sns
import pandas as pd
import scipy.stats
from scipy.sparse import issparse
from tqdm import tqdm
import scanpy as sc
import numpy as np

# Assuming the rest of your code remains the same...

# Create a figure with two rows of subplots: one for UMAP and one for scatter plots with gene names
fig, axes = plt.subplots(2, num_bins, figsize=(6 * num_bins, 12))

scatter_plot_axis_limit = 0.45

# Loop over each pseudotime bin to create UMAP and scatter plots
for i, bin_label in enumerate(adata_gex.obs['pseudotime_bin'].cat.categories):
    # Filter cells in the current bin for scatter plot
    bin_cells = adata_gex[adata_gex.obs['pseudotime_bin'] == bin_label]

    # Calculate scatter plot data
    scatter_df = calculate_correlation(bin_cells, gene_set_A, gene_set_B, gene_indices)

    # Create UMAP plot in the first row
    ax_umap = axes[0, i]
    sc.pl.umap(adata_gex, color="pseudotime_bin", groups=[bin_label], ax=ax_umap, show=False, legend_loc=None, title=bin_label, size=50)

    # Create scatter plot in the second row with gene names, colored by gene set
    ax_scatter = axes[1, i]
    for _, row in scatter_df.iterrows():
        # Determine the color based on the gene set
        color = 'blue' if row['gene'] in gene_set_A else 'orange'
        ax_scatter.text(row['mean_corr_with_A'], row['mean_corr_with_B'], row['gene'], fontsize=9, color=color, style='italic')
    
    ax_scatter.set_title(f'{bin_label}')
    ax_scatter.axhline(0, color='grey', linestyle='--')
    ax_scatter.axvline(0, color='grey', linestyle='--')
    ax_scatter.set_xlabel('Correlation with Left aCM early module')
    ax_scatter.set_ylabel('Correlation with Right aCM early module')
    ax_scatter.set_xlim(-scatter_plot_axis_limit, scatter_plot_axis_limit)
    ax_scatter.set_ylim(-scatter_plot_axis_limit, scatter_plot_axis_limit)

plt.tight_layout()
plt.savefig("./figures/bifurcation_analysis_text_names.pdf",bbox_inches="tight")
plt.show()
```

Example 172 (unknown):
```unknown
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 22/22 [00:01<00:00, 19.19it/s]
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
  0%|                                                                                                                                              | 0/22 [00:00<?, ?it/s]/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scipy/stats/_stats_py.py:4781: ConstantInputWarning: An input array is constant; the correlation coefficient is not defined.
  warnings.warn(stats.ConstantInputWarning(msg))
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 22/22 [00:01<00:00, 20.37it/s]
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
  0%|                                                                                                                                              | 0/22 [00:00<?, ?it/s]/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scipy/stats/_stats_py.py:4781: ConstantInputWarning: An input array is constant; the correlation coefficient is not defined.
  warnings.warn(stats.ConstantInputWarning(msg))
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 22/22 [00:01<00:00, 21.41it/s]
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
  0%|                                                                                                                                              | 0/22 [00:00<?, ?it/s]/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scipy/stats/_stats_py.py:4781: ConstantInputWarning: An input array is constant; the correlation coefficient is not defined.
  warnings.warn(stats.ConstantInputWarning(msg))
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 22/22 [00:01<00:00, 21.60it/s]
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 22/22 [00:55<00:00,  2.52s/it]
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
posx and posy should be finite values
posx and posy should be finite values
posx and posy should be finite values
posx and posy should be finite values
posx and posy should be finite values
posx and posy should be finite values
posx and posy should be finite values
posx and posy should be finite values
posx and posy should be finite values
posx and posy should be finite values
posx and posy should be finite values
posx and posy should be finite values
posx and posy should be finite values
posx and posy should be finite values
posx and posy should be finite values
```

Example 173 (python):
```python
import matplotlib.pyplot as plt
import seaborn as sns
import pandas as pd
import scipy.stats
from scipy.sparse import issparse
from tqdm import tqdm
import scanpy as sc
import numpy as np

# Assuming the rest of your code remains the same...

# Create a figure with two rows of subplots: one for UMAP and one for scatter plots with gene names
fig, axes = plt.subplots(2, num_bins, figsize=(6 * num_bins, 12))

scatter_plot_axis_limit = 0.3

# Loop over each pseudotime bin to create UMAP and scatter plots
for i, bin_label in enumerate(adata_gex.obs['pseudotime_bin'].cat.categories):
    # Filter cells in the current bin for scatter plot
    bin_cells = adata_gex[adata_gex.obs['pseudotime_bin'] == bin_label]

    # Calculate scatter plot data
    scatter_df = calculate_correlation(bin_cells, gene_set_A, gene_set_B, gene_indices)

    # Create UMAP plot in the first row
    ax_umap = axes[0, i]
    sc.pl.umap(adata_gex, color="pseudotime_bin", groups=[bin_label], ax=ax_umap, show=False, legend_loc=None, title=bin_label, size=50)

    # Create scatter plot in the second row with gene names, colored by gene set
    ax_scatter = axes[1, i]
    for _, row in scatter_df.iterrows():
        # Determine the color based on the gene set
        color = 'blue' if row['gene'] in gene_set_A else 'orange'
        ax_scatter.text(row['mean_corr_with_A'], row['mean_corr_with_B'], row['gene'], fontsize=9, color=color, style='italic')
    
    ax_scatter.set_title(f'{bin_label}')
    ax_scatter.axhline(0, color='grey', linestyle='--')
    ax_scatter.axvline(0, color='grey', linestyle='--')
    ax_scatter.set_xlabel('Correlation with Left aCM early module')
    ax_scatter.set_ylabel('Correlation with Right aCM early module')
    ax_scatter.set_xlim(-scatter_plot_axis_limit, scatter_plot_axis_limit)
    ax_scatter.set_ylim(-scatter_plot_axis_limit, scatter_plot_axis_limit)

plt.tight_layout()
#plt.savefig("./figures/bifurcation_analysis_text_names.pdf",bbox_inches="tight")
plt.show()
```

Example 174 (unknown):
```unknown
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 22/22 [00:01<00:00, 19.87it/s]
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
  0%|                                                                                                                                              | 0/22 [00:00<?, ?it/s]/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scipy/stats/_stats_py.py:4781: ConstantInputWarning: An input array is constant; the correlation coefficient is not defined.
  warnings.warn(stats.ConstantInputWarning(msg))
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 22/22 [00:01<00:00, 20.46it/s]
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
  0%|                                                                                                                                              | 0/22 [00:00<?, ?it/s]/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scipy/stats/_stats_py.py:4781: ConstantInputWarning: An input array is constant; the correlation coefficient is not defined.
  warnings.warn(stats.ConstantInputWarning(msg))
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 22/22 [00:01<00:00, 21.44it/s]
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
  0%|                                                                                                                                              | 0/22 [00:00<?, ?it/s]/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scipy/stats/_stats_py.py:4781: ConstantInputWarning: An input array is constant; the correlation coefficient is not defined.
  warnings.warn(stats.ConstantInputWarning(msg))
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 22/22 [00:01<00:00, 21.70it/s]
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 22/22 [00:55<00:00,  2.51s/it]
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
posx and posy should be finite values
posx and posy should be finite values
posx and posy should be finite values
posx and posy should be finite values
posx and posy should be finite values
posx and posy should be finite values
```

Example 175 (unknown):
```unknown
early_modules_dict={}
early_modules_dict["LEFT"]=gene_set_A
early_modules_dict["RIGHT"]=gene_set_B
```

Example 176 (unknown):
```unknown
['MBP',
 'KCNMB2',
 'GAN',
 'SLC8A3',
 'A2M',
 'KCNMB2-AS1',
 'KSR1',
 'PANCR',
 'PITX2',
 'DCBLD2']
```

Example 177 (unknown):
```unknown
['DAB1',
 'UST',
 'ANGPT1',
 'GNAL',
 'KIF26B',
 'KCNQ5',
 'HEY1',
 'HSPA6',
 'RFTN1',
 'MAF',
 'OTOGL',
 'PDE1A']
```

Example 178 (python):
```python
import pickle

key="aCM_n570" # or (aCM_n1140 or aCM_full)
key="aCM_n1140"
key="aCM_full"
target_type="Gene"
# File path
file_path = f"/nfs/team205/heart/pyscenic/plus/eRegulon_signatures/{key}_{target_type}_eRegulon_signatures.pkl"

# Open the file and load the dictionary
with open(file_path, 'rb') as file:
    TF_TG_dict = pickle.load(file)
```

Example 179 (unknown):
```unknown
results_list=[]
```

Example 180 (python):
```python
side="LEFT"
associated_features_dict.keys()
total_transcriptome_genes=36000
genes_of_interest=early_modules_dict[f"{side}"]

import pandas as pd
from scipy.stats import fisher_exact


# Initialize a list to store the results
results = []

for regulon, regulon_genes in TF_TG_dict.items():
    # Contingency table
    overlap = len(set(regulon_genes) & set(genes_of_interest))
    only_genes_of_interest = len(genes_of_interest) - overlap
    only_regulon_genes = len(regulon_genes) - overlap
    neither = total_transcriptome_genes - (overlap + only_genes_of_interest + only_regulon_genes)
    table = [[overlap, only_genes_of_interest], [only_regulon_genes, neither]]

    # Perform Fisher's Exact Test
    odds_ratio, p_value = fisher_exact(table)
    
    # Append the results
    results.append({"Regulon": regulon, "Odds Ratio": odds_ratio, "pval": p_value})

# Convert the list of dictionaries to a DataFrame
results_df = pd.DataFrame(results)

# Sort the DataFrame by p-value
results_df.sort_values(by="pval", inplace=True)

# Reset index after sorting
results_df.reset_index(drop=True, inplace=True)

# Display the DataFrame
results_df=results_df[results_df.pval<0.01]
results_df["side"]=side
results_df=results_df.reset_index(names='rank')
results_list.append(results_df)
results_df
```

Example 181 (python):
```python
side="RIGHT"
associated_features_dict.keys()
total_transcriptome_genes=36000
genes_of_interest=early_modules_dict[f"{side}"]

import pandas as pd
from scipy.stats import fisher_exact


# Initialize a list to store the results
results = []

for regulon, regulon_genes in TF_TG_dict.items():
    # Contingency table
    overlap = len(set(regulon_genes) & set(genes_of_interest))
    only_genes_of_interest = len(genes_of_interest) - overlap
    only_regulon_genes = len(regulon_genes) - overlap
    neither = total_transcriptome_genes - (overlap + only_genes_of_interest + only_regulon_genes)
    table = [[overlap, only_genes_of_interest], [only_regulon_genes, neither]]

    # Perform Fisher's Exact Test
    odds_ratio, p_value = fisher_exact(table)
    
    # Append the results
    results.append({"Regulon": regulon, "Odds Ratio": odds_ratio, "pval": p_value})

# Convert the list of dictionaries to a DataFrame
results_df = pd.DataFrame(results)

# Sort the DataFrame by p-value
results_df.sort_values(by="pval", inplace=True)

# Reset index after sorting
results_df.reset_index(drop=True, inplace=True)

# Display the DataFrame
results_df=results_df[results_df.pval<0.01]
results_df["side"]=side
results_df=results_df.reset_index(names='rank')
results_list.append(results_df)
results_df
```

Example 182 (python):
```python
results_concat=pd.concat(results_list)
print(results_concat.shape)
results_concat["TF"]=results_concat.Regulon.str.split("_",expand=True)[0]
results_concat
```

Example 183 (python):
```python
chdgene_table=pd.read_csv("chdgene_table.csv")
results_concat=results_concat.merge(chdgene_table, left_on="TF",right_on="Gene", how='left')
results_concat
```

Example 184 (python):
```python
import matplotlib
matplotlib.rcParams['pdf.fonttype'] = 42
matplotlib.rcParams['ps.fonttype'] = 42
```

Example 185 (python):
```python
# Transforming p-values to -log10(p)
results_concat['-log10(pval)'] = -np.log10(results_concat['pval'])

# Create a pivot table for the heatmap
heatmap_data = results_concat.pivot("TF", "side", "-log10(pval)")
heatmap_data = heatmap_data.fillna(value=0)
heatmap_data = heatmap_data.sort_values(by=["LEFT","RIGHT"], ascending=False)

# Initialize with empty strings
annotations = pd.DataFrame('', index=heatmap_data.index, columns=heatmap_data.columns)

# Loop through the original dataframe to populate the annotations
for _, row in results_concat.iterrows():
    tf = row['TF']
    side = row['side']
    chd_classification = row['CHD classification']
    if pd.notna(chd_classification):
        if "ASD" in chd_classification:
            annotations.loc[tf, side] = '**'  # Two asterisks if "ASD" in CHD classification
        else:
            annotations.loc[tf, side] = '*'   # One asterisk otherwise

# Create the heatmap
plt.figure(figsize=(10, 10))
sns.heatmap(heatmap_data, annot=annotations, fmt="s", cmap="RdPu", square=True, cbar=True)
plt.title("Enrichment of early gene modules for regulons\n '*': associated with any CHD, '**' associated with ASD\ncolor scale = -log10(p-value)")
plt.xlabel("Side")
plt.ylabel("Regulon (Transcription Factor)")
plt.savefig("CHD_enrichment.pdf", format='pdf', bbox_inches='tight')
plt.show()
```

Example 186 (unknown):
```unknown
/tmp/ipykernel_4090211/3468513932.py:5: FutureWarning: In a future version of pandas all arguments of DataFrame.pivot will be keyword-only.
  heatmap_data = results_concat.pivot("TF", "side", "-log10(pval)")
```

Example 187 (unknown):
```unknown
[reg for reg in list(TF_TG_dict.keys()) if "PITX" in reg]
```

Example 188 (unknown):
```unknown
['PITX2_+_(130g)']
```

Example 189 (unknown):
```unknown
plot.GO_plot(gene_list=TF_TG_dict["PITX2_+_(130g)"],display_table=True,gene_sets=['GO_Biological_Process_2023'])
```

Example 190 (unknown):
```unknown
GO_Biological_Process_2023
```

Example 191 (python):
```python
sc.pl.dotplot(adata_gex, groupby="AtrialCardiomyocytesImmature_annotation",var_names=TF_TG_dict["PITX2_+_(130g)"])
```

Example 192 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_dotplot.py:749: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap', 'norm' will be ignored
  dot_ax.scatter(x, y, **kwds)
```

---

## 

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/RNA/scnRNA_Integration-Annotation/Annotations/EndotheliumAnnotation.html

**Examples:**

Example 1 (unknown):
```unknown
#Endothelium Annotation
```

Example 2 (python):
```python
# Import packages, initial settings

import numpy as np
import matplotlib.pyplot as plt
import scanpy as sc
import pandas as pd
import anndata
import scrublet as scr
import seaborn as sns
import magic
import pybiomart
import six
import sys
sys.modules['sklearn.externals.six'] = six
import scanorama
import harmonypy
import bbknn
import gseapy
from gseapy.plot import barplot, dotplot
import scvelo as scv
import scvi

plt.rcParams['pdf.fonttype'] = 42
plt.rcParams['ps.fonttype'] = 42

sc.settings.verbosity = 3  # verbosity: errors (0), warnings (1), info (2), hints (3)

from matplotlib.colors import LinearSegmentedColormap
cmap = LinearSegmentedColormap.from_list(name='gene_cmap', colors=['lightgrey', 'thistle', 'red', 'darkred']) 

sc.settings.set_figure_params(dpi=80, color_map='viridis', vector_friendly=False,  dpi_save=300, facecolor='white')
sc.logging.print_version_and_date()
#sc.logging.print_versions()
```

Example 3 (unknown):
```unknown
Global seed set to 0
```

Example 4 (unknown):
```unknown
Running Scanpy 1.8.1, on 2022-12-20 11:15.
```

Example 5 (python):
```python
adata=sc.read('./Objects/20221130RNA_foetal_post-scVI_raw_ExtracardiacRemoved.h5ad')
print(adata.shape)
```

Example 6 (unknown):
```unknown
(306287, 36601)
```

Example 7 (python):
```python
cell_cycle_genes = ['MCM5','PCNA','TYMS','FEN1','MCM2','MCM4','RRM1','UNG','GINS2','MCM6','CDCA7','DTL','PRIM1','UHRF1','MLF1IP','HELLS','RFC2','RPA2','NASP','RAD51AP1','GMNN','WDR76','SLBP','CCNE2','UBR7','POLD3','MSH2','ATAD2','RAD51','RRM2','CDC45','CDC6','EXO1','TIPIN','DSCC1','BLM','CASP8AP2','USP1','CLSPN','POLA1','CHAF1B','BRIP1','E2F8','HMGB2','CDK1','NUSAP1','UBE2C','BIRC5','TPX2','TOP2A','NDC80','CKS2','NUF2','CKS1B','MKI67','TMPO','CENPF','TACC3','FAM64A','SMC4','CCNB2','CKAP2L','CKAP2','AURKB','BUB1','KIF11','ANP32E','TUBB4B','GTSE1','KIF20B','HJURP','CDCA3','HN1','CDC20','TTK','CDC25C','KIF2C','RANGAP1','NCAPD2','DLGAP5','CDCA2','CDCA8','ECT2','KIF23','HMMR','AURKA','PSRC1','ANLN','LBR','CKAP5','CENPE','CTCF','NEK2','G2E3','GAS2L3','CBX5','CENPA']
s_genes = cell_cycle_genes[:43]
s_genes = [x for x in s_genes if x in adata.var_names]
print('S genes:', s_genes, '\n')
g2m_genes = cell_cycle_genes[43:]
g2m_genes = [x for x in g2m_genes if x in adata.var_names]
print('G2 genes:', g2m_genes, '\n')
cell_cycle_genes = [x for x in cell_cycle_genes if x in adata.var_names]

hb_genes1 = ['HBB','HBD','HBG1','HBG2','HBE1','HBZ','HBM','HBA2','HBA1', 'HBQ1']
hb_genes1 = [x for x in hb_genes1 if x in adata.var_names]
print('Hemoglobin genes set 1:', hb_genes1, '\n')

hb_genes2 = ['AHSP','GYPA','RHAG','GFI1B','KEL','EPB42','ALAS2','TSPO2','GYPB','GYPE','SPTA1','RHCE','KLF1','RHD','SLC4A1','SNX22','SQSTM1','SOD1','FTH1','MAP1LC3B','SLC25A37','TRIM58','HMBS']
hb_genes2 = [x for x in hb_genes2 if x in adata.var_names]
print('Hemoglobin genes set 2:', hb_genes2, '\n')

mito_genes=['MT-ND1','MT-ND2','MT-CO1','MT-CO2','MT-ATP8','MT-ATP6','MT-CO3','MT-ND3','MT-ND4L', 'MT-ND4', 'MT-ND5', 'MT-ND6', 'MT-CYB']
mito_genes = [x for x in mito_genes if x in adata.var_names]
print('Mitochondrial genes:', mito_genes, '\n')

ribo_genes = ['RPLP0','RPLP1','RPLP2','RPL3','RPL3L','RPL4','RPL5','RPL6','RPL7','RPL7A','RPL7L1','RPL8','RPL9','RPL10','RPL10A','RPL10L','RPL11','RPL12','RPL13','RPL13A','RPL14','RPL15','RPL17','RPL18','RPL18A','RPL19','RPL21','RPL22','RPL22L1','RPL23','RPL23A','RPL24','RPL26','RPL26L1','RPL27','RPL27A','RPL28','RPL29','RPL30','RPL31','RPL32','RPL34','RPL35','RPL35A','RPL36','RPL36A','RPL36AL','RPL37','RPL37A','RPL38','RPL39','RPL39L','UBA52','RPL41','RPSA','RPS2','RPS3','RPS3A','RPS4X','RPS4Y1','RPS4Y2','RPS5','RPS6','RPS7','RPS8','RPS9','RPS10','RPS11','RPS12','RPS13','RPS14','RPS15','RPS15A','RPS16','RPS17','RPS18','RPS19','RPS20','RPS21','RPS23','RPS24','RPS25','RPS26','RPS27','RPS27A','RPS27L','RPS28','RPS29','FAU']
ribo_genes = [x for x in ribo_genes if x in adata.var_names]
print('Ribo genes:', ribo_genes, '\n')

stress_genes=['FOS','FOSB','JUN','JUNB','JUND','ZFP36','SOCS3','CEBPD','CEBPB','ATF3','EGR1','HSPA1A','HSPA1B','HSP90AA1','HSP90AB1','HSPA8','HSPB1','HSPE1','IER3','IER2','BTG1','BTG2','DUSP1']
stress_genes = [x for x in stress_genes if x in adata.var_names]
print('Tissue dissociation stress associated genes:', stress_genes, '\n')

FoxProteins=['FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1']
chrY_genes=['SRY', 'RPS4Y1', 'ZFY', 'ZFY-AS1', 'LINC00278', 'PCDH11Y', 'TTTY2B', 'TBL1Y', 'PRKY', 'USP9Y', 'DDX3Y', 'UTY', 'TMSB4Y', 'NLGN4Y', 'TTTY14', 'KDM5D', 'TTTY10', 'EIF1AY', 'PRORY', 'LINC00266-4P']
MembraneProteins1=['AADAT','AAK1','ABCA12','ABCA7','ABCB1','ABCB11','ABCB4','ABCB6','ABCC1','ABCC3','ABCC4','ABCC5','ABCC6','ABCG2','ABHD3','ABI1','ABLIM3','AC004691.2','AC004922.1','AC005324.3','AC005726.1','AC008403.1','AC138696.1','ACIN1','ACKR3','ACLY','ACOT13','ACOT9','ACTN1','ACTR10','ACTR6','ADA','ADAM10','ADAM12','ADAM22','ADAM28','ADAM33','ADAMTS1','ADAMTS2','ADAMTSL4','ADAP1','ADCK5','ADD1','ADD2','ADGRB2','ADGRD1','ADGRG2','ADGRG7','ADGRL2','ADH1A','ADH1B','ADH1C','ADH7','ADK','ADORA1','ADRA1B','ADRM1','ADSS','AFAP1L2','AFDN','AFG1L','AGER','AGRN','AHNAK','AHNAK2','AIFM2','AKAP10','AKAP11','AKAP12','AKR1B10','AKR1B15','AL132780.3','AL139300.1','AL355315.1','AL358113.1','AL645922.1','ALDH3A1','ALK','ALOX15B','ALOXE3','ALPI','ALPP','ALPPL2','ALS2CR12','AMACR','AMDHD1','AMER1','AMER2','AMIGO2','AMOT','AMOTL1','AMOTL2','AMPH','ANK2','ANK3','ANKEF1','ANKFN1','ANKIB1','ANKLE2','ANKRA2','ANKRD13A','ANKRD35','ANKRD42','ANKRD45','ANKS1B','ANO1','ANO10','ANO2','ANO6','ANPEP','ANXA1','ANXA13','ANXA2','ANXA3','AP2M1','APBB1','APBB1IP','APC','APLP1','APMAP','APOC3','APOD','APOLD1','AQP1','AQP10','AQP3','AQP4','AQP5','ARAP1','ARF1','ARHGAP18','ARHGAP19','ARHGAP21','ARHGAP22','ARHGAP23','ARHGAP24','ARHGAP28','ARHGAP29','ARHGAP33','ARHGAP36','ARHGAP40','ARHGAP9','ARHGDIG','ARHGEF1','ARHGEF11','ARHGEF12','ARHGEF15','ARHGEF26','ARHGEF28','ARHGEF40','ARHGEF5','ARID2','ARID4A','ARID5B','ARL13A','ARL15','ARL4C','ARMC12','ARMCX5','ARPC1A','ARPC5','ARRB2','ARRDC4','ARVCF','ASAP1','ASB5','ASGR1','ASGR2','ASH2L','ASIC1','ASIC4','ASPM','ASPSCR1','ATG13','ATG3','ATG7','ATIC','ATOX1','ATP10D','ATP12A','ATP13A3','ATP1B3','ATP23','ATP2B1','ATP2B4','ATP2C2','ATP6V0A2','ATP6V1H','ATP8A2','ATP8B1','ATXN10','ATXN3','ATXN7L3','AVEN','AVPI1','AXIN2','AXL','B2M','B3GALT2','BACE1','BAG3','BAIAP2','BAIAP2L1','BANK1','BASP1','BBOF1','BCAR1','BCAR3','BCL2L12','BCORL1','BCR','BDKRB2','BEND6','BFSP1','BFSP2','BICD2','BIN2','BLNK','BLVRB','BMPR2','BMX','BOC','BOLL','BORCS5','BORCS8-MEF2B','BRK1','BRPF1','BRSK1','BSDC1','BST2','BTK','BTN2A1','BTRC','BUB1B-PAK6','BVES','BZW2','C10orf90','C11orf42','C11orf52','C12orf73','C12orf76','C14orf37','C14orf93','C16orf74','C17orf58','C17orf62','C18orf21','C19orf33','C19orf38','C19orf48','C19orf71','C1orf116','C1orf189','C1orf35','C1orf54','C1orf68','C1orf74','C1QBP','C2orf72','C3orf14','C3orf20','C3orf38','C4orf19','C4orf22','C7orf43','C8orf33','C8orf37','C9orf153','C9orf3','CA14','CA9','CABLES2','CACNA1E','CACNG4','CADM1','CALCRL','CALD1','CALML5','CALN1','CAMK2A','CAMK2B','CAMK2D','CAMK2G','CAP2','CAPS','CASKIN2','CASP4','CAST','CAV3','CAVIN1','CAVIN2','CAVIN3','CAVIN4','CBWD1','CBWD2','CBWD3','CBWD5','CBWD6','CC2D1A','CCBE1','CCDC107','CCDC112','CCDC115','CCDC124','CCDC136','CCDC144NL','CCDC15','CCDC154','CCDC170','CCDC24','CCDC28A','CCDC59','CCDC62','CCDC66','CCDC69','CCDC8','CCDC83','CCDC85C','CCDC93','CCDC97','CCNT2','CCNYL1','CCR3','CCR4','CCT3','CD109','CD163','CD164L2','CD200R1','CD226','CD248','CD274','CD2AP','CD300A','CD300LB','CD33','CD34','CD37','CD38','CD3E','CD3G','CD4','CD44','CD46','CD47','CD48','CD79B','CD81','CD84','CD86','CD8A','CD8B','CD9','CD93','CD99L2','CDAN1','CDC42EP1','CDC42EP3','CDC42EP4','CDC42SE1','CDCA4','CDH1','CDH13','CDH15','CDH17','CDH2','CDH26','CDH3','CDH4','CDH5','CDH6','CDH8','CDIPT','CDK16','CDK5','CDK5RAP2','CDK7','CDRT15','CDV3','CEACAM5','CELSR1','CENPJ','CEP162','CEP295','CEP55','CEP57L1','CEP89','CERCAM','CFAP157','CFAP206','CFAP58','CFAP65','CFB','CFL2','CFLAR','CGN','CGNL1','CHCHD7','CHIC2','CHMP2A','CHRM3','CHRNA1','CHRNA5','CHRNB4','CHRND','CIB1','CIB3','CIP2A','CKAP5','CLCA2','CLCA4','CLCN5','CLCN6','CLDN11','CLDN18','CLDN2','CLDN23','CLDN3','CLDN4','CLDN7','CLDN9','CLEC2A','CLEC4F','CLEC5A','CLECL1','CLIC2','CLIC5','CLTB','CLUL1','CMTM4','CMTM7','CMTR2','CMYA5','CNGA1','CNKSR2','CNNM1','CNNM4','CNP','CNR1','CNTD1','COBL','COBLL1','COG3','COL16A1','COL17A1','COL4A5','COL6A2','COLEC12','COLQ','COMMD4','COMTD1','COPRS','COQ7','CORIN','CORO1B','CORO2A','CORO2B','COX6B2','CPEB3','CPPED1','CPZ','CRB3','CREB5','CRIP1','CRIP2','CRK','CRLF3','CROCC','CRTC1','CRYAB','CRYBG3','CRYL1','CSDE1','CSF1','CSF1R','CSPG4','CST6','CTNNA1','CTNNB1','CTNNBIP1','CTNND1','CTNND2','CTSF','CTSV','CTTN','CTXN1','CUEDC1','CUTA','CX3CL1','CX3CR1','CXADR','CXCR2','CYB5D2','CYBRD1','CYFIP2','CYP20A1','CYP24A1','CYTH2','CYTH4','DAAM1','DAB2','DAG1','DAGLB','DAW1','DBN1','DCAF13','DCAF16','DCBLD2','DCHS1','DCHS2','DCP2','DCT','DCTN6','DCUN1D2','DDAH1','DDB2','DDIAS','DDIT4L','DDR1','DDR2','DDX58','DDX60L','DENND6B','DFFA','DGKG','DGKH','DHDDS','DHPS','DHRS7B','DIAPH1','DIAPH3','DIO2','DIS3L','DLG1','DLG2','DLG5','DLGAP3','DLL3','DMWD','DNAAF1','DNAAF4','DNAH17','DNAJA4','DNAJB13','DNAJB4','DNAJC18','DNAJC5','DNAJC6','DNAJC9','DOC2A','DOCK4','DOK3','DONSON','DPEP1','DPH1','DPPA5','DPYSL2','DQX1','DRD1','DRP2','DSC3','DSEL','DSG2','DSP','DTNA','DUS1L','DUSP15','DXO','DYSF','ECSCR','EDA2R','EDNRA','EDNRB','EEF1AKMT1','EEF2','EFCAB1','EFCAB11','EFCAB2','EFCAB7','EFHC2','EFNB1','EFR3A','EFR3B','EGFR','EHBP1','EHD1','EHD2','EHD3','EHD4','EIF2B2','EIF5','EIF5B','ELFN1','ELMO3','ELN','ELOC','EMP3','ENAH','ENC1','ENG','ENKD1','ENO1','ENO2','ENO3','ENOX1','ENOX2','EPB41','EPB41L1','EPB41L2','EPB41L3','EPB41L4A','EPB41L4B','EPB41L5','EPCAM','EPHA2','EPHA3','EPHB1','EPHB2','EPN1','EPOR','EPS8L2','ERAP1','ERBB2','ERBB3','ERBIN','ERC1','ERCC4','ERMN','ESCO2','ETNK1','ETS2','EVA1A','EVI2B','EXOC1','EXOC3L1','EXOC7','EZR','F11R','F2RL1','F2RL3','FAAP20','FABP2','FABP3','FABP5','FAIM','FAM126A','FAM126B','FAM129A','FAM129B','FAM13A','FAM163B','FAM171A1','FAM193A','FAM205A','FAM210B','FAM222A','FAM45A','FAM47E','FAM47E-STBD1','FAM72A','FAM72B','FAM72C','FAM72D','FANCG','FAS','FASN','FBLIM1','FBLN2','FBLN5','FBLN7','FBP2','FBRSL1','FBXL2','FBXO16','FBXO38','FBXO41','FBXW11','FCAR','FCGBP','FCGR1A','FCGR1B','FCGR2A','FCHSD2','FFAR4','FGD1','FGD5','FGFR2','FGR','FHIT','FHL1','FIBCD1','FIGN','FILIP1','FILIP1L','FKBP2','FLAD1','FLCN','FLNA','FLNB','FLNC','FLOT1','FLOT2','FLRT3','FLT1','FLT4','FLVCR1','FMN2','FMNL2','FMNL3','FNBP1','FOXA2','FOXN3','FP565260.3','FRMD3','FRMD7','FRMD8','FRS2','FSCN1','FTCD','FUT2','FXYD3','FYB1','FYN','FZD1','FZD2','FZD4','FZD6','GAB1','GAB2','GABRA1','GABRA2','GABRA3','GABRA5','GABRG1','GAD1','GADL1','GAGE1','GAGE10','GAGE12C','GAGE12D','GAGE12E','GAGE12F','GAGE12G','GAGE12H','GAGE12J','GAGE13','GAGE2A','GAGE2E','GALNT10','GALR1','GAP43','GAPDH','GAPT','GAPVD1','GAS2','GAS2L1','GAS7','GAS8','GBP4','GCA','GCC1','GCLM','GCSAML-AS1','GFI1B','GFOD1','GFRA3','GHR','GID8','GINM1','GIPC1','GJA1','GJB3','GJB4','GJB6','GJB7','GJC1','GJC2','GLO1','GLP2R','GLRX','GLUL','GMIP','GNAI2','GNAQ','GNAS','GNB1','GNB2','GNB3','GNB4','GNG2','GP9','GPAT3','GPATCH1','GPBP1','GPBP1L1','GPC1','GPC3','GPC4','GPI','GPR1','GPR142','GPR156','GPR157','GPR160','GPR173','GPR20','GPR27','GPR50','GPR63','GPRC5A','GPRIN1','GPRIN2','GPX8','GRAMD1A','GRAMD1B','GRAMD1C','GRAP2','GRB7','GRID2IP','GRIK5','GRK3','GRK5','GRPR','GSDMA','GTF2F1','GTSE1','GUCA1A','GYPA','GYPC','HAPLN3','HARBI1','HAS1','HAUS4','HAUS7','HCAR2','HCAR3','HCFC2','HCK','HCLS1','HCST','HDAC8','HEG1','HELZ','HENMT1','HERC2','HERPUD1','HFE','HIF1A','HIF3A','HINT1','HIP1R','HLA-A','HLA-DPB1','HMGCS1','HMOX1','HOMER3','HOXC5','HOXD4','HPS3','HRASLS5','HRH1','HS6ST3','HSH2D','HSPB1','HSPG2','HTR1B','HTR7','HTRA1','HUNK','HYDIN','HYI','HYLS1','ICAM1','ICAM2','ICOS','ICOSLG','IFFO2','IFIT5','IFNGR1','IFRD1','IFT122','IGFL2','IGSF11','IGSF23','IKZF5','IL10RA','IL12RB1','IL12RB2','IL13','IL13RA2','IL16','IL17RB','IL1R1','IL1RL1','IL21R','IL4R','IL6ST','IL7R','ILDR2','ILK','INPP5A','INSR','INTS12','IP6K2','IPCEF1','IQGAP1','IQGAP2','IREB2','IRF2','IRS1','ISLR','ITGA2B','ITGA3','ITGA4','ITGA7','ITGA8','ITGA9','ITGB1','ITGB2','ITGB3','ITGB4','ITGB5','ITGB6','ITGB7','ITGB8','ITK','ITM2A','ITPKA','ITSN1','ITSN2','IYD','JAG1','JAK2','JAK3','JAM2','JAML','JCAD','JMJD4','JPT2','JUP','KANK1','KANK3','KANSL2','KARS','KATNB1','KCNAB1','KCNB2','KCNH8','KCNJ15','KCNK13','KCNK17','KCNK6','KCNK7','KCNK9','KCNN2','KCNS3','KCNU1','KDF1','KDR','KEL','KIAA0556','KIAA0895','KIAA1324','KIAA1522','KIAA1958','KIF17','KIF19','KIF21A','KIF21B','KIF26A','KIF26B','KISS1R','KIT','KLC1','KLC2','KLF9','KLHDC7B','KLHL41','KLHL6','KLHL7','KLK7','KLRC1','KLRC2','KLRF1','KLRK1','KMT2D','KNDC1','KNSTRN','KNTC1','KPTN','KRT75','KRT77','KRTAP25-1','L1CAM','L3MBTL1','LAIR1','LAMA4','LAMC1','LAMP2','LAMTOR1','LAPTM4A','LASP1','LAT','LAT2','LAX1','LCP1','LCTL','LDB2','LDLRAD3','LENG1','LGALS3','LGALS4','LGALS8','LGR6','LHFPL2','LHX6','LIMA1','LINGO1','LIPJ','LMCD1','LMOD2','LMTK2','LMTK3','LNX1','LNX2','LONRF1','LONRF3','LOXHD1','LPAR1','LPAR3','LPAR4','LPIN1','LPIN2','LRCH2','LRP11','LRP12','LRRC15','LRRC26','LRRC34','LRRC45','LRRC69','LRRC7','LRRC9','LRRFIP2','LRRIQ3','LSMEM2','LSR','LTB4R2','LUZP2','LY6K','LYN','LYPD1','LYST','LZTS1','LZTS2','MACF1','MADD','MAGI1','MAGI3','MAMDC2','MAP1S','MAP2','MAP2K1','MAP2K4','MAP2K7','MAP3K12','MAP3K21','MAP3K6','MAP3K7','MAP4','MAP4K1','MAP4K5','MAP6','MAPK15','MAPK8IP1','MAPT','MARCH10','MARCH7','MARCKS','MARK2','MARVELD2','MAS1L','MAT2B','MBP','MC5R','MCAM','MCEMP1','MCF2L2','MCMBP','MCTS1','MDGA1','ME1','MEF2B','MEIOC','MEPCE','MERTK','MET','METAP1','METAP2','METTL15','METTL25','METTL9','MFAP3L','MFSD1','MFSD2A','MGAT4C','MIB1','MICAL2','MICAL3','MICALL1','MICB','MIEN1','MIGA2','MISP','MKLN1','MKRN3','MLF2','MLKL','MLST8','MMACHC','MMP10','MMP15','MMP24','MMP25','MOAP1','MPHOSPH8','MPHOSPH9','MPI','MPIG6B','MPL','MPP1','MPP5','MPP7','MPZL1','MPZL2','MR1','MRGPRF','MRNIP','MRPL42','MRPL44','MRPL46','MRPS18B','MS4A1','MS4A12','MS4A14','MS4A15','MSN','MSRA','MTCP1','MTFR1L','MTHFR','MTM1','MTPN','MTSS1','MUC1','MUC20','MVD','MYH11','MYH9','MYL4','MYLK','MYO10','MYO1B','MYO1C','MYO1E','MYO1G','MYO3A','MYO5B','MYO9A','MYOF','N4BP2','NAA35','NAB1','NAIF1','NAMPT','NAP1L2','NAPG','NARFL','NCAM1','NCAM2','NCK1','NCKAP1L','NCKAP5','NCKIPSD','NCOA1','NCS1','NDC1','NECTIN2','NECTIN3','NECTIN4','NEDD1','NEDD9','NEO1','NEU1','NEXN','NF2','NFE2L2','NFKBID','NGEF']
MembraneProteins2=['NGFR','NHS','NHSL2','NID2','NINJ2','NKTR','NKX2-8','NLGN1','NLGN3','NLK','NLRX1','NMB','NMBR','NME1','NME2','NMT1','NMT2','NMUR2','NOS1','NOTCH2','NPFFR2','NPL','NPNT','NPR1','NPTN','NPTXR','NR2C1','NRAS','NRCAM','NRP2','NRSN1','NRXN1','NSD1','NSFL1C','NT5E','NTNG1','NTNG2','NTSR1','NUDCD3','NUDT16L1','NUDT9','NUMB','NUP35','NUS1','OAS3','OBSCN','OCA2','OCLN','OCM','OCM2','ODC1','ODR4','OGT','OLFM4','OLIG2','OLR1','OPHN1','OPRK1','OR10P1','OR1N2','OR2C3','OR6B2','OR6B3','OR6C6','OR8S1','OSBPL10','OSBPL6','OSBPL7','OSCAR','OSCP1','OSGEP','OSR2','OTOP2','OTULIN','OXGR1','P3H3','PACSIN2','PACSIN3','PAFAH1B2','PAG1','PAIP1','PAK1','PAK4','PAK6','PALLD','PALM','PALM2','PALM2-AKAP2','PALM3','PALMD','PANX1','PANX2','PAQR8','PARD3','PARD3B','PARD6A','PARD6G','PARP6','PATJ','PAWR','PCBD2','PCDH1','PCDH17','PCDH7','PCDHA1','PCDHA11','PCDHA12','PCDHA4','PCDHA5','PCDHB1','PCDHB11','PCDHB12','PCDHB16','PCDHB4','PCDHB5','PCDHGA1','PCDHGA10','PCDHGA11','PCDHGA12','PCDHGA2','PCDHGA3','PCDHGA4','PCDHGA5','PCDHGA6','PCDHGA7','PCDHGA8','PCDHGA9','PCDHGB1','PCDHGB2','PCDHGB3','PCDHGB4','PCDHGB5','PCDHGB6','PCDHGB7','PCDHGC3','PCDHGC4','PCDHGC5','PCLO','PCMTD1','PDAP1','PDCD7','PDE3A','PDE4A','PDE4D','PDE9A','PDGFC','PDGFRA','PDHX','PDLIM1','PDLIM5','PDPN','PDZD2','PDZD8','PEAR1','PEBP1','PFDN1','PGAP3','PGLYRP2','PGLYRP4','PGRMC2','PHACTR1','PHACTR4','PHF20L1','PHGDH','PHLDB2','PHLDB3','PHOSPHO1','PHPT1','PI3','PI4K2A','PI4KA','PIEZO2','PIGB','PIGN','PIGW','PIGX','PIK3AP1','PIK3CG','PIK3R6','PIP4K2A','PIP4K2C','PIP5K1A','PIRT','PITPNM3','PKD2L2','PKN1','PKN2','PKP1','PKP2','PKP3','PKP4','PLAA','PLAUR','PLCB1','PLCB2','PLCD3','PLCD4','PLEKHA4','PLEKHA6','PLEKHA7','PLEKHG3','PLEKHG4','PLEKHG6','PLEKHH2','PLEKHO2','PLIN4','PLPP1','PLPPR5','PLS3','PLSCR1','PLVAP','PLXNA3','PLXNA4','PNCK','PNISR','PODXL','POLE','POU2F3','PPEF1','PPFIBP1','PPIP5K1','PPL','PPM1A','PPM1F','PPP1CA','PPP1CB','PPP1R12B','PPP1R16A','PPP1R16B','PPP1R18','PPP1R27','PPP1R9A','PPP2R1B','PPP3R1','PPP3R2','PPP4C','PPP6R3','PPTC7','PRAME','PRC1','PRDX6','PRIMA1','PRKCA','PRKCE','PRKCH','PRKCZ','PRKD1','PROM2','PROSER2','PRR15','PRR32','PRR7','PRRC2A','PRRG1','PRRT2','PRRT3','PRRT4','PRTG','PRX','PSCA','PSD','PSD2','PSEN1','PSMC5','PSMC6','PSMD9','PSTPIP2','PTAFR','PTCHD4','PTK2B','PTK6','PTPN12','PTPN21','PTPN22','PTPN4','PTPRB','PTPRG','PTPRK','PTPRM','PTPRR','PTPRS','PTX3','PURG','PVALB','PVR','PXDC1','PXK','PYGL','PYGO2','PYM1','RAB13','RAB23','RAB25','RAB26','RAB27B','RAB3B','RAB3GAP2','RAB43','RAB8A','RABEPK','RAD51C','RAD52','RAI2','RALA','RALGAPA2','RAPGEF2','RAPH1','RAPSN','RASA4','RASA4B','RASAL1','RASAL2','RASAL3','RASD1','RASD2','RASGRP1','RASGRP2','RASGRP4','RASSF2','RASSF3','RASSF4','RBM46','RC3H1','RCCD1','RCSD1','RD3L','RDX','RECK','RELL1','REPS1','RET','RFLNB','RFPL1','RFPL2','RFPL3','RGP1','RGS10','RGS14','RGS19','RGS3','RGS7BP','RGS9BP','RHCE','RHD','RHEX','RHOA','RHOBTB2','RHOJ','RHOXF2','RHOXF2B','RIC8A','RIC8B','RIF1','RILPL1','RIMBP3','RIMBP3B','RIMBP3C','RIMS2','RIOK2','RIOK3','RIPK1','RND1','RNF114','RNF145','RNF146','RNF149','RNF165','RNF166','RNF39','ROBO1','ROBO3','ROBO4','ROM1','RP2','RPE','RPEL1','RPGRIP1L','RPIA','RPL36A','RPL36A-HNRNPH2','RPL36AL','RPSA','RPUSD1','RRAD','RRP12','RRP7A','RSBN1L','RSC1A1','RSPH3','RTL3','RTL4','RTN4R','RTN4RL1','RYR2','S100A13','S100A14','S100A16','S100A2','S100A3','S100A4','S100A6','S100A9','SAMD4A','SAMHD1','SAMSN1','SAPCD1','SASH3','SC5D','SCAMP1','SCAMP4','SCEL','SCHIP1','SCIN','SCN1A','SCN5A','SCN8A','SCN9A','SCNN1D','SCNN1G','SCRIB','SCUBE3','SDC2','SDC4','SDE2','SDHB','SEC16B','SEMA3C','SEMA3D','SEMA4B','SEMA4F','SEMA6C','SEMA6D','SEPT2','SEPT3','SERINC2','SERINC5','SERPINB3','SFSWAP','SGCE','SGK494','SH3BGR','SH3D21','SHANK1','SHANK3','SHARPIN','SHISA6','SHKBP1','SHROOM2','SHTN1','SIGLEC10','SIGLEC14','SIGLEC6','SIGLEC7','SIN3B','SIPA1','SIPA1L1','SIPA1L3','SKAP1','SLAMF6','SLC10A5','SLC12A2','SLC13A3','SLC13A5','SLC14A1','SLC16A1','SLC16A10','SLC16A2','SLC16A3','SLC16A7','SLC16A9','SLC17A5','SLC19A1','SLC19A3','SLC1A5','SLC1A6','SLC22A16','SLC23A3','SLC24A4','SLC26A9','SLC27A1','SLC28A1','SLC2A1','SLC2A11','SLC2A12','SLC2A14','SLC2A2','SLC2A3','SLC30A1','SLC31A2','SLC34A1','SLC34A2','SLC35F5','SLC35G2','SLC36A4','SLC38A5','SLC38A6','SLC38A7','SLC39A10','SLC39A14','SLC39A2','SLC39A6','SLC3A2','SLC40A1','SLC41A3','SLC43A1','SLC43A3','SLC44A2','SLC45A4','SLC46A1','SLC46A3','SLC4A1AP','SLC4A2','SLC4A7','SLC4A8','SLC5A12','SLC5A3','SLC5A6','SLC5A7','SLC6A6','SLC7A14','SLC7A2','SLC7A5','SLC7A6','SLC8A1','SLC9A1','SLC9A2','SLC9A3','SLC9A3R1','SLC9A3R2','SLC9A4','SLC9A5','SLC9B2','SLC9C2','SLCO1A2','SLCO1B3','SLCO4A1','SLCO4C1','SLCO5A1','SLIT1','SLIT3','SLITRK6','SLK','SLX4','SMAGP','SMAP1','SMOC1','SMPD2','SMPX','SNAP23','SNAP25','SNRK','SNTB2','SNTG2','SNX15','SNX32','SNX9','SOCS3','SOCS5','SOD1','SORBS1','SORCS1','SORT1','SOX11','SPAG16','SPANXN4','SPART','SPATA12','SPATC1L','SPERT','SPG11','SPHK1','SPINK13','SPINT1','SPINT2','SPN','SPPL2B','SPPL3','SPRED3','SPRYD3','SPTBN2','SRC','SRCIN1','SRD5A3','SRPK1','SRPRB','SRRM3','SSFA2','SSH1','SSH3','SSTR1','SSTR5','SSX2IP','ST8SIA5','STAB1','STAC','STAC2','STAMBP','STAMBPL1','STARD4','STAT2','STEAP4','STIL','STIP1','STK10','STK16','STK17A','STK17B','STOM','STOML2','STOML3','STX18','STX4','STXBP4','STYK1','SUCLG1','SUCLG2','SUGT1','SUMF2','SUPT6H','SUSD2','SVIL','SWAP70','SYCP2','SYK','SYMPK','SYNC','SYNGR3','SYNPO2L','SYPL2','SYT4','SYTL2','SYVN1','TACC2','TACR2','TACSTD2','TASP1','TAZ','TBC1D10A','TBC1D2','TBC1D24','TBC1D30','TBC1D9B','TBXA2R','TCAF2','TCAF2C','TCHP','TCIM','TDG','TDO2','TDP1','TEC','TEDC2','TEK','TES','TEX14','TEX9','TGFBR1','TGFBR2','TGM1','TGM2','THBS1','THBS2','THEGL','THY1','TIAM1','TICAM1','TIGD7','TIMD4','TJP1','TJP2','TJP3','TLN1','TLR4','TM2D2','TM4SF20','TM4SF4','TM4SF5','TMC5','TMED8','TMEM101','TMEM107','TMEM108','TMEM121B','TMEM139','TMEM141','TMEM164','TMEM176B','TMEM183A','TMEM186','TMEM2','TMEM218','TMEM222','TMEM229A','TMEM245','TMEM251','TMEM254','TMEM258','TMEM266','TMEM268','TMEM50A','TMEM63B','TMPRSS15','TMPRSS2','TMTC2','TNFRSF10D','TNFRSF11A','TNFRSF12A','TNFRSF21','TNFRSF25','TNFRSF8','TNFRSF9','TNFSF13B','TNK1','TNKS1BP1','TNNI2','TOGARAM2','TOLLIP','TOM1','TP53BP2','TPD52L1','TPRN','TRAF7','TRAIP','TRAPPC13','TRAT1','TRDN','TREML1','TRIB1','TRIM16','TRIM16L','TRIM38','TRIM4','TRIM44','TRIM54','TRIM71','TRIML2','TRIP6','TRMT1','TRMT10B','TRMT13','TRMT6','TRO','TRPM1','TRPM4','TRPM7','TRPV2','TRPV3','TSG101','TSGA13','TSHZ3','TSPAN15','TSPAN6','TSPO2','TSR2','TSSK4','TTC17','TTC23','TTC7B','TTLL12','TTLL5','TULP3','TWF2','TXNDC11','TXNIP','TYROBP','TYW3','UAP1','UBA3','UBA52','UBAC1','UBALD1','UBAP1','UBASH3B','UBE2A','UBE2B','UBE2C','UBE2D1','UBE2D2','UBE2D3','UBE2D4','UBE2K','UBE2QL1','UBE2S','UBE3C','UBQLN2','UBR2','UBTD1','UCP3','ULBP1','ULBP2','UNC5A','UNC79','UQCC1','USP14','USP15','USP2','USP20','USP21','USP4','USP53','UTP20','UTRN','VAMP5','VAMP8','VANGL1','VASP','VCAM1','VCPIP1','VIL1','VPS13B','VPS36','VPS50','VPS8','VSIG4','VTCN1','VWA5B1','VWA7','WAS','WASF2','WBP4','WDFY3','WDPCP','WDR1','WDR12','WDR13','WDR53','WDR6','WDR66','WIPF1','WWP1','XIAP','XIRP1','XIRP2','XPC','XPNPEP2','XPO6','XRN1','XYLT1','YES1','YIPF1','YIPF4','YOD1','YTHDC1','ZBED3','ZBTB33','ZBTB8OS','ZDHHC2','ZDHHC20','ZDHHC22','ZDHHC5','ZFP41','ZFP90','ZFYVE28','ZGPAT','ZMYND10','ZNF10','ZNF18','ZNF185','ZNF2','ZNF20','ZNF227','ZNF229','ZNF236','ZNF276','ZNF277','ZNF337','ZNF397','ZNF407','ZNF497','ZNF503','ZNF510','ZNF536','ZNF571','ZNF598','ZNF599','ZNF646','ZNF655','ZNF688','ZNF706','ZNF717','ZNF783','ZNF79','ZNF839','ZSCAN22','ZSCAN29','ZSWIM1','ZSWIM8','ZYX']
MembraneProteins=MembraneProteins1+MembraneProteins2

TFs1=['AC008770.3','AC023509.3','AC092835.1','AC138696.1','ADNP','ADNP2','AEBP1','AEBP2','AHCTF1','AHDC1','AHR','AHRR','AIRE','AKAP8','AKAP8L','AKNA','ALX1','ALX3','ALX4','ANHX','ANKZF1','AR','ARGFX','ARHGAP35','ARID2','ARID3A','ARID3B','ARID3C','ARID5A','ARID5B','ARNT','ARNT2','ARNTL','ARNTL2','ARX','ASCL1','ASCL2','ASCL3','ASCL4','ASCL5','ASH1L','ATF1','ATF2','ATF3','ATF4','ATF5','ATF6','ATF6B','ATF7','ATMIN','ATOH1','ATOH7','ATOH8','BACH1','BACH2','BARHL1','BARHL2','BARX1','BARX2','BATF','BATF2','BATF3','BAZ2A','BAZ2B','BBX','BCL11A','BCL11B','BCL6','BCL6B','BHLHA15','BHLHA9','BHLHE22','BHLHE23','BHLHE40','BHLHE41','BNC1','BNC2','BORCS8-MEF2B','BPTF','BRF2','BSX','C11orf95','CAMTA1','CAMTA2','CARF','CASZ1','CBX2','CC2D1A','CCDC169-SOHLH2','CCDC17','CDC5L','CDX1','CDX2','CDX4','CEBPA','CEBPB','CEBPD','CEBPE','CEBPG','CEBPZ','CENPA','CENPB','CENPBD1','CENPS','CENPT','CENPX','CGGBP1','CHAMP1','CHCHD3','CIC','CLOCK','CPEB1','CPXCR1','CREB1','CREB3','CREB3L1','CREB3L2','CREB3L3','CREB3L4','CREB5','CREBL2','CREBZF','CREM','CRX','CSRNP1','CSRNP2','CSRNP3','CTCF','CTCFL','CUX1','CUX2','CXXC1','CXXC4','CXXC5','DACH1','DACH2','DBP','DBX1','DBX2','DDIT3','DEAF1','DLX1','DLX2','DLX3','DLX4','DLX5','DLX6','DMBX1','DMRT1','DMRT2','DMRT3','DMRTA1','DMRTA2','DMRTB1','DMRTC2','DMTF1','DNMT1','DNTTIP1','DOT1L','DPF1','DPF3','DPRX','DR1','DRAP1','DRGX','DUX1','DUX3','DUX4','DUXA','DZIP1','E2F1','E2F2','E2F3','E2F4','E2F5','E2F6','E2F7','E2F8','E4F1','EBF1','EBF2','EBF3','EBF4','EEA1','EGR1','EGR2','EGR3','EGR4','EHF','ELF1','ELF2','ELF3','ELF4','ELF5','ELK1','ELK3','ELK4','EMX1','EMX2','EN1','EN2','EOMES','EPAS1','ERF','ERG','ESR1','ESR2','ESRRA','ESRRB','ESRRG','ESX1','ETS1','ETS2','ETV1','ETV2','ETV3','ETV3L','ETV4','ETV5','ETV6','ETV7','EVX1','EVX2','FAM170A','FAM200B','FBXL19','FERD3L','FEV','FEZF1','FEZF2','FIGLA','FIZ1','FLI1','FLYWCH1','FOS','FOSB','FOSL1','FOSL2','FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXN4','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1','GABPA','GATA1','GATA2','GATA3','GATA4','GATA5','GATA6','GATAD2A','GATAD2B','GBX1','GBX2','GCM1','GCM2','GFI1','GFI1B','GLI1','GLI2','GLI3','GLI4','GLIS1','GLIS2','GLIS3','GLMP','GLYR1','GMEB1','GMEB2','GPBP1','GPBP1L1','GRHL1','GRHL2','GRHL3','GSC','GSC2','GSX1','GSX2','GTF2B','GTF2I','GTF2IRD1','GTF2IRD2','GTF2IRD2B','GTF3A','GZF1','HAND1','HAND2','HBP1','HDX','HELT','HES1','HES2','HES3','HES4','HES5','HES6','HES7','HESX1','HEY1','HEY2','HEYL','HHEX','HIC1','HIC2','HIF1A','HIF3A','HINFP','HIVEP1','HIVEP2','HIVEP3','HKR1','HLF','HLX','HMBOX1','HMG20A','HMG20B','HMGA1','HMGA2','HMGN3','HMX1','HMX2','HMX3','HNF1A','HNF1B','HNF4A','HNF4G','HOMEZ','HOXA1','HOXA10','HOXA11','HOXA13','HOXA2','HOXA3','HOXA4','HOXA5','HOXA6','HOXA7','HOXA9','HOXB1','HOXB13','HOXB2','HOXB3','HOXB4','HOXB5','HOXB6','HOXB7','HOXB8','HOXB9','HOXC10','HOXC11','HOXC12','HOXC13','HOXC4','HOXC5','HOXC6','HOXC8','HOXC9','HOXD1','HOXD10','HOXD11','HOXD12','HOXD13','HOXD3','HOXD4','HOXD8','HOXD9','HSF1','HSF2','HSF4','HSF5','HSFX1','HSFX2','HSFY1','HSFY2','IKZF1','IKZF2','IKZF3','IKZF4','IKZF5','INSM1','INSM2','IRF1','IRF2','IRF3','IRF4','IRF5','IRF6','IRF7','IRF8','IRF9','IRX1','IRX2','IRX3','IRX4','IRX5','IRX6','ISL1','ISL2','ISX','JAZF1','JDP2','JRK','JRKL','JUN','JUNB','JUND','KAT7','KCMF1','KCNIP3','KDM2A','KDM2B','KDM5B','KIN','KLF1','KLF10','KLF11','KLF12','KLF13','KLF14','KLF15','KLF16','KLF17','KLF2','KLF3','KLF4','KLF5','KLF6','KLF7','KLF8','KLF9','KMT2A','KMT2B','L3MBTL1','L3MBTL3','L3MBTL4','LBX1','LBX2','LCOR','LCORL','LEF1','LEUTX','LHX1','LHX2','LHX3','LHX4','LHX5','LHX6','LHX8','LHX9','LIN28A','LIN28B','LIN54','LMX1A','LMX1B','LTF','LYL1','MAF','MAFA','MAFB','MAFF','MAFG','MAFK','MAX','MAZ','MBD1','MBD2','MBD3','MBD4','MBD6','MBNL2','MECOM','MECP2','MEF2A','MEF2B','MEF2C','MEF2D','MEIS1','MEIS2','MEIS3','MEOX1','MEOX2','MESP1','MESP2','MGA','MITF','MIXL1','MKX','MLX','MLXIP','MLXIPL','MNT','MNX1','MSANTD1','MSANTD3','MSANTD4','MSC','MSGN1','MSX1','MSX2','MTERF1','MTERF2','MTERF3','MTERF4','MTF1','MTF2','MXD1','MXD3','MXD4','MXI1','MYB','MYBL1','MYBL2','MYC','MYCL','MYCN','MYF5','MYF6','MYNN','MYOD1','MYOG','MYPOP','MYRF','MYRFL','MYSM1','MYT1','MYT1L','MZF1','NACC2','NAIF1','NANOG','NANOGNB','NANOGP8','NCOA1','NCOA2','NCOA3','NEUROD1','NEUROD2','NEUROD4','NEUROD6','NEUROG1','NEUROG2','NEUROG3','NFAT5','NFATC1','NFATC2','NFATC3','NFATC4','NFE2','NFE2L1','NFE2L2','NFE2L3','NFE4','NFIA','NFIB','NFIC','NFIL3','NFIX','NFKB1','NFKB2','NFX1','NFXL1','NFYA','NFYB','NFYC','NHLH1','NHLH2','NKRF','NKX1-1','NKX1-2','NKX2-1','NKX2-2','NKX2-3','NKX2-4','NKX2-5','NKX2-6','NKX2-8','NKX3-1','NKX3-2','NKX6-1','NKX6-2','NKX6-3','NME2','NOBOX','NOTO','NPAS1','NPAS2','NPAS3','NPAS4','NR0B1','NR1D1','NR1D2','NR1H2','NR1H3','NR1H4','NR1I2','NR1I3','NR2C1','NR2C2','NR2E1','NR2E3','NR2F1','NR2F2','NR2F6','NR3C1','NR3C2','NR4A1','NR4A2','NR4A3','NR5A1','NR5A2','NR6A1','NRF1','NRL','OLIG1','OLIG2','OLIG3','ONECUT1','ONECUT2','ONECUT3','OSR1','OSR2','OTP','OTX1','OTX2','OVOL1','OVOL2','OVOL3','PA2G4','PATZ1','PAX1','PAX2','PAX3','PAX4','PAX5','PAX6','PAX7','PAX8','PAX9','PBX1','PBX2','PBX3','PBX4','PCGF2','PCGF6','PDX1','PEG3','PGR','PHF1','PHF19 ','PHF20','PHF21A','PHOX2A','PHOX2B','PIN1','PITX1','PITX2','PITX3','PKNOX1','PKNOX2','PLAG1','PLAGL1','PLAGL2','PLSCR1','POGK','POU1F1','POU2AF1','POU2F1','POU2F2','POU2F3','POU3F1','POU3F2','POU3F3','POU3F4','POU4F1','POU4F2','POU4F3','POU5F1','POU5F1B','POU5F2','POU6F1','POU6F2','PPARA','PPARD','PPARG','PRDM1','PRDM10','PRDM12','PRDM13','PRDM14','PRDM15','PRDM16','PRDM2','PRDM4','PRDM5','PRDM6','PRDM8','PRDM9','PREB','PRMT3','PROP1','PROX1','PROX2','PRR12','PRRX1','PRRX2','PTF1A','PURA','PURB','PURG','RAG1','RARA','RARB','RARG','RAX','RAX2','RBAK','RBCK1','RBPJ','RBPJL','RBSN','REL','RELA','RELB','REPIN1','REST','REXO4','RFX1','RFX2','RFX3','RFX4','RFX5','RFX6','RFX7','RFX8','RHOXF1','RHOXF2','RHOXF2B','RLF','RORA','RORB','RORC','RREB1','RUNX1','RUNX2','RUNX3','RXRA','RXRB','RXRG','SAFB','SAFB2','SALL1','SALL2','SALL3','SALL4','SATB1','SATB2','SCMH1','SCML4','SCRT1','SCRT2','SCX','SEBOX','SETBP1','SETDB1','SETDB2','SGSM2','SHOX','SHOX2','SIM1','SIM2','SIX1','SIX2','SIX3','SIX4','SIX5','SIX6','SKI','SKIL','SKOR1','SKOR2','SLC2A4RG','SMAD1','SMAD3','SMAD4','SMAD5','SMAD9','SMYD3','SNAI1','SNAI2','SNAI3','SNAPC2','SNAPC4','SNAPC5','SOHLH1','SOHLH2','SON','SOX1','SOX10','SOX11']
TFs2=['SOX12','SOX13','SOX14','SOX15','SOX17','SOX18','SOX2','SOX21','SOX3','SOX30','SOX4','SOX5','SOX6','SOX7','SOX8','SOX9','SP1','SP100','SP110','SP140','SP140L','SP2','SP3','SP4','SP5','SP6','SP7','SP8','SP9','SPDEF','SPEN','SPI1','SPIB','SPIC','SPZ1','SRCAP','SREBF1','SREBF2','SRF','SRY','ST18','STAT1','STAT2','STAT3','STAT4','STAT5A','STAT5B','STAT6','T','TAL1','TAL2','TBP','TBPL1','TBPL2','TBR1','TBX1','TBX10','TBX15','TBX18','TBX19','TBX2','TBX20','TBX21','TBX22','TBX3','TBX4','TBX5','TBX6','TCF12','TCF15','TCF20','TCF21','TCF23','TCF24','TCF3','TCF4','TCF7','TCF7L1','TCF7L2','TCFL5','TEAD1','TEAD2','TEAD3','TEAD4','TEF','TERB1','TERF1','TERF2','TET1','TET2','TET3','TFAP2A','TFAP2B','TFAP2C','TFAP2D','TFAP2E','TFAP4','TFCP2','TFCP2L1','TFDP1','TFDP2','TFDP3','TFE3','TFEB','TFEC','TGIF1','TGIF2','TGIF2LX','TGIF2LY','THAP1','THAP10','THAP11','THAP12','THAP2','THAP3','THAP4','THAP5','THAP6','THAP7','THAP8','THAP9','THRA','THRB','THYN1','TIGD1','TIGD2','TIGD3','TIGD4','TIGD5','TIGD6','TIGD7','TLX1','TLX2','TLX3','TMF1','TOPORS','TP53','TP63','TP73','TPRX1','TRAFD1','TRERF1','TRPS1','TSC22D1','TSHZ1','TSHZ2','TSHZ3','TTF1','TWIST1','TWIST2','UBP1','UNCX','USF1','USF2','USF3','VAX1','VAX2','VDR','VENTX','VEZF1','VSX1','VSX2','WIZ','WT1','XBP1','XPA','YBX1','YBX2','YBX3','YY1','YY2','ZBED1','ZBED2','ZBED3','ZBED4','ZBED5','ZBED6','ZBED9','ZBTB1','ZBTB10','ZBTB11','ZBTB12','ZBTB14','ZBTB16','ZBTB17','ZBTB18','ZBTB2','ZBTB20','ZBTB21','ZBTB22','ZBTB24','ZBTB25','ZBTB26','ZBTB3','ZBTB32','ZBTB33','ZBTB34','ZBTB37','ZBTB38','ZBTB39','ZBTB4','ZBTB40','ZBTB41','ZBTB42','ZBTB43','ZBTB44','ZBTB45','ZBTB46','ZBTB47','ZBTB48','ZBTB49','ZBTB5','ZBTB6','ZBTB7A','ZBTB7B','ZBTB7C','ZBTB8A','ZBTB8B','ZBTB9','ZC3H8','ZEB1','ZEB2','ZFAT','ZFHX2','ZFHX3','ZFHX4','ZFP1','ZFP14','ZFP2','ZFP28','ZFP3','ZFP30','ZFP37','ZFP41','ZFP42','ZFP57','ZFP62','ZFP64','ZFP69','ZFP69B','ZFP82','ZFP90','ZFP91','ZFP92','ZFPM1','ZFPM2','ZFX','ZFY','ZGLP1','ZGPAT','ZHX1','ZHX2','ZHX3','ZIC1','ZIC2','ZIC3','ZIC4','ZIC5','ZIK1','ZIM2','ZIM3','ZKSCAN1','ZKSCAN2','ZKSCAN3','ZKSCAN4','ZKSCAN5','ZKSCAN7','ZKSCAN8','ZMAT1','ZMAT4','ZNF10','ZNF100','ZNF101','ZNF107','ZNF112','ZNF114','ZNF117','ZNF12','ZNF121','ZNF124','ZNF131','ZNF132','ZNF133','ZNF134','ZNF135','ZNF136','ZNF138','ZNF14','ZNF140','ZNF141','ZNF142','ZNF143','ZNF146','ZNF148','ZNF154','ZNF155','ZNF157','ZNF16','ZNF160','ZNF165','ZNF169','ZNF17','ZNF174','ZNF175','ZNF177','ZNF18','ZNF180','ZNF181','ZNF182','ZNF184','ZNF189','ZNF19','ZNF195','ZNF197','ZNF2','ZNF20','ZNF200','ZNF202','ZNF205','ZNF207','ZNF208','ZNF211','ZNF212','ZNF213','ZNF214','ZNF215','ZNF217','ZNF219','ZNF22','ZNF221','ZNF222','ZNF223','ZNF224','ZNF225','ZNF226','ZNF227','ZNF229','ZNF23','ZNF230','ZNF232','ZNF233','ZNF234','ZNF235','ZNF236','ZNF239','ZNF24','ZNF248','ZNF25','ZNF250','ZNF251','ZNF253','ZNF254','ZNF256','ZNF257','ZNF26','ZNF260','ZNF263','ZNF264','ZNF266','ZNF267','ZNF268','ZNF273','ZNF274','ZNF275','ZNF276','ZNF277','ZNF28','ZNF280A','ZNF280B','ZNF280C','ZNF280D','ZNF281','ZNF282','ZNF283','ZNF284','ZNF285','ZNF286A','ZNF286B','ZNF287','ZNF292','ZNF296','ZNF3','ZNF30','ZNF300','ZNF302','ZNF304','ZNF311','ZNF316','ZNF317','ZNF318','ZNF319','ZNF32','ZNF320','ZNF322','ZNF324','ZNF324B','ZNF326','ZNF329','ZNF331','ZNF333','ZNF334','ZNF335','ZNF337','ZNF33A','ZNF33B','ZNF34','ZNF341','ZNF343','ZNF345','ZNF346','ZNF347','ZNF35','ZNF350','ZNF354A','ZNF354B','ZNF354C','ZNF358','ZNF362','ZNF365','ZNF366','ZNF367','ZNF37A','ZNF382','ZNF383','ZNF384','ZNF385A','ZNF385B','ZNF385C','ZNF385D','ZNF391','ZNF394','ZNF395','ZNF396','ZNF397','ZNF398','ZNF404','ZNF407','ZNF408','ZNF41','ZNF410','ZNF414','ZNF415','ZNF416','ZNF417','ZNF418','ZNF419','ZNF420','ZNF423','ZNF425','ZNF426','ZNF428','ZNF429','ZNF43','ZNF430','ZNF431','ZNF432','ZNF433','ZNF436','ZNF438','ZNF439','ZNF44','ZNF440','ZNF441','ZNF442','ZNF443','ZNF444','ZNF445','ZNF446','ZNF449','ZNF45','ZNF451','ZNF454','ZNF460','ZNF461','ZNF462','ZNF467','ZNF468','ZNF469','ZNF470','ZNF471','ZNF473','ZNF474','ZNF479','ZNF48','ZNF480','ZNF483','ZNF484','ZNF485','ZNF486','ZNF487','ZNF488','ZNF490','ZNF491','ZNF492','ZNF493','ZNF496','ZNF497','ZNF500','ZNF501','ZNF502','ZNF503','ZNF506','ZNF507','ZNF510','ZNF511','ZNF512','ZNF512B','ZNF513','ZNF514','ZNF516','ZNF517','ZNF518A','ZNF518B','ZNF519','ZNF521','ZNF524','ZNF525','ZNF526','ZNF527','ZNF528','ZNF529','ZNF530','ZNF532','ZNF534','ZNF536','ZNF540','ZNF541','ZNF543','ZNF544','ZNF546','ZNF547','ZNF548','ZNF549','ZNF550','ZNF551','ZNF552','ZNF554','ZNF555','ZNF556','ZNF557','ZNF558','ZNF559','ZNF560','ZNF561','ZNF562','ZNF563','ZNF564','ZNF565','ZNF566','ZNF567','ZNF568','ZNF569','ZNF57','ZNF570','ZNF571','ZNF572','ZNF573','ZNF574','ZNF575','ZNF576','ZNF577','ZNF578','ZNF579','ZNF580','ZNF581','ZNF582','ZNF583','ZNF584','ZNF585A','ZNF585B','ZNF586','ZNF587','ZNF587B','ZNF589','ZNF592','ZNF594','ZNF595','ZNF596','ZNF597','ZNF598','ZNF599','ZNF600','ZNF605','ZNF606','ZNF607','ZNF608','ZNF609','ZNF610','ZNF611','ZNF613','ZNF614','ZNF615','ZNF616','ZNF618','ZNF619','ZNF620','ZNF621','ZNF623','ZNF624','ZNF625','ZNF626','ZNF627','ZNF628','ZNF629','ZNF630','ZNF639','ZNF641','ZNF644','ZNF645','ZNF646','ZNF648','ZNF649','ZNF652','ZNF653','ZNF654','ZNF655','ZNF658','ZNF66','ZNF660','ZNF662','ZNF664','ZNF665','ZNF667','ZNF668','ZNF669','ZNF670','ZNF671','ZNF672','ZNF674','ZNF675','ZNF676','ZNF677','ZNF678','ZNF679','ZNF680','ZNF681','ZNF682','ZNF683','ZNF684','ZNF687','ZNF688','ZNF689','ZNF69','ZNF691','ZNF692','ZNF695','ZNF696','ZNF697','ZNF699','ZNF7','ZNF70','ZNF700','ZNF701','ZNF703','ZNF704','ZNF705A','ZNF705B','ZNF705D','ZNF705E','ZNF705G','ZNF706','ZNF707','ZNF708','ZNF709','ZNF71','ZNF710','ZNF711','ZNF713','ZNF714','ZNF716','ZNF717','ZNF718','ZNF721','ZNF724','ZNF726','ZNF727','ZNF728','ZNF729','ZNF730','ZNF732','ZNF735','ZNF736','ZNF737','ZNF74','ZNF740','ZNF746','ZNF747','ZNF749','ZNF750','ZNF75A','ZNF75D','ZNF76','ZNF761','ZNF763','ZNF764','ZNF765','ZNF766','ZNF768','ZNF77','ZNF770','ZNF771','ZNF772','ZNF773','ZNF774','ZNF775','ZNF776','ZNF777','ZNF778','ZNF780A','ZNF780B','ZNF781','ZNF782','ZNF783','ZNF784','ZNF785','ZNF786','ZNF787','ZNF788','ZNF789','ZNF79','ZNF790','ZNF791','ZNF792','ZNF793','ZNF799','ZNF8','ZNF80','ZNF800','ZNF804A','ZNF804B','ZNF805','ZNF808','ZNF81','ZNF813','ZNF814','ZNF816','ZNF821','ZNF823','ZNF827','ZNF829','ZNF83','ZNF830','ZNF831','ZNF835','ZNF836','ZNF837','ZNF84','ZNF841','ZNF843','ZNF844','ZNF845','ZNF846','ZNF85','ZNF850','ZNF852','ZNF853','ZNF860','ZNF865','ZNF878','ZNF879','ZNF880','ZNF883','ZNF888','ZNF891','ZNF90','ZNF91','ZNF92','ZNF93','ZNF98','ZNF99','ZSCAN1','ZSCAN10','ZSCAN12','ZSCAN16','ZSCAN18','ZSCAN2','ZSCAN20','ZSCAN21','ZSCAN22','ZSCAN23','ZSCAN25','ZSCAN26','ZSCAN29','ZSCAN30','ZSCAN31','ZSCAN32','ZSCAN4','ZSCAN5A','ZSCAN5B','ZSCAN5C','ZSCAN9','ZUFSP','ZXDA','ZXDB','ZXDC','ZZZ3']
TFs=TFs1+TFs2

#TFs other, inclusive of the above list, not necessarily DNA binding proteins but affect transcription
TFs1Other=['AATF','ABL1','ABRA','AC008770.3','AC020915.1','AC023509.3','AC092835.1','AC105001.2','AC138696.1','ACOD1','ACTR3B','ACTR3C','ADAM8','ADAMTS17','ADAMTS19','ADAR','ADGRA1','ADGRG3','ADNP','ADNP2','AEBP1','AEBP2','AES','AFAP1 ','AFF1','AFF3','AFF4','AGER','AGT','AHCTF1','AHDC1','AHR','AHRR','AIM2','AIRE','AKAP8','AKAP8L','AKNA','AKT1','ALK','ALKBH2','ALX1','ALX3','ALX4','AMH','ANAPC2','ANHX','ANKRD30A','ANKZF1','ANXA3','ANXA4','AP5Z1','APBB3','APC2','APEX1','APOA4','APOBEC3A','APOBEC3B','APOBEC3G','APTX','AR','ARGFX','ARHGAP35','ARHGEF2','ARHGEF5','ARID1A','ARID1B','ARID2','ARID3A','ARID3B','ARID3C','ARID4A','ARID4B','ARID5A','ARID5B','ARNT','ARNT2','ARNTL','ARNTL2','ARRB1','ARRB2','ARX','ASCC1','ASCL1','ASCL2','ASCL3','ASCL4','ASCL5','ASH1L','ATF1','ATF2','ATF3','ATF4','ATF5','ATF6','ATF6B','ATF7','ATF7IP','ATM','ATMIN','ATOH1','ATOH7','ATOH8','ATR','ATRX','ATXN7','BACH1','BACH2','BAG1','BANF1','BANP','BARHL1','BARHL2','BARX1','BARX2','BATF','BATF2','BATF3','BAZ2A','BAZ2B','BBX','BCL10','BCL11A','BCL11B','BCL3','BCL6','BCL6B','BCLAF1','BCOR','BDP1','BEND4','BEX1','BHLHA15','BHLHA9','BHLHE22','BHLHE23','BHLHE40','BHLHE41','BHMG1','BIN1','BLM','BLZF1','BMI1','BMP2','BMP4','BMP7','BMPR1A','BNC1','BNC2','BOD1L1','BOLA1','BOLA2','BOLA2B','BOLA3','BORCS8-MEF2B','BPNT1','BPTF','BRCA1','BRD4','BRD7','BRD8','BRD9','BRF1','BRF2','BRMS1','BRPF1','BSX','BTAF1','BTBD8','BTG2','BTK','BTRC','BUD31','C11orf95','C14orf39','C17orf49','C1D','C20orf194','C3orf33','CACTIN','CAMK1D','CAMK2A','CAMTA1','CAMTA2','CAPN15','CAPN3','CARD11','CARD14','CARF','CARHSP1','CASP8AP2','CASZ1','CAT','CBFA2T2','CBFA2T3','CBFB','CBL','CBLL1','CBX2','CC2D1A','CC2D1B','CCDC124','CCDC169-SOHLH2','CCDC17','CCDC3','CCDC71','CCDC83','CCNT1','CCNT2','CD36','CD40','CD40LG','CDC5L','CDC6','CDK5RAP3','CDK7','CDK8','CDK9','CDKN1A','CDKN2A','CDX1','CDX2','CDX4','CEBPA','CEBPB','CEBPD','CEBPE','CEBPG','CEBPZ','CELF3','CENPA','CENPB','CENPBD1','CENPS','CENPT','CENPX','CERS2','CERS3','CERS4','CERS5','CERS6','CFLAR','CGGBP1','CHAMP1','CHCHD3','CHD1','CHD2','CHD3','CHD4','CHD6','CHD7','CHD9','CHP1','CHRAC1','CHUK','CHURC1','CIB1','CIC','CIITA','CIR1','CITED1','CITED2','CITED4','CIZ1','CLOCK','CLU','CMKLR1','CNBP','CNOT3','CNOT4','CNOT7','CNOT8','COMMD1','COMMD6','COMMD7','COPS2','COPS3','COPS4','CPEB1','CPSF4','CPSF4L','CPXCR1','CRAMP1','CREB1','CREB3','CREB3L1','CREB3L2','CREB3L3','CREB3L4','CREB5','CREBBP','CREBL2','CREBRF','CREBZF','CREG1','CREM','CRTC1','CRTC2','CRTC3','CRX','CSDC2','CSDE1','CSRNP1','CSRNP2','CSRNP3','CTBP1','CTBP2','CTCF','CTCFL','CTH','CTNNB1','CTNNBIP1','CUL1','CUL2','CUL3','CUL4A','CUL4B','CUL5','CUX1','CUX2','CXXC1','CXXC4','CXXC5','CYLD','CYP1B1','CYTL1','DAB2IP','DACH1','DACH2','DAP','DAXX','DBP','DBX1','DBX2','DCP1A','DDB1','DDB2','DDIT3','DDN','DDR2','DDRGK1','DDX3X','DDX5','DDX58','DEAF1','DEK','DEPDC1','DEPDC1B','DEPDC4','DEPDC5','DEPDC7','DEPTOR','DHX34','DHX36','DHX57','DISP3','DLX1','DLX2','DLX3','DLX4','DLX5','DLX6','DMAP1','DMBX1','DMC1','DMRT1','DMRT2','DMRT3','DMRTA1','DMRTA2','DMRTB1','DMRTC1','DMRTC2','DMTF1','DNA2','DNAJA3','DNAJC1','DNAJC2','DNAJC21','DNMT1','DNMT3A','DNMT3B','DNMT3L','DNTT','DNTTIP1','DOT1L','DPF1','DPF2','DPF3','DPRX','DR1','DRAP1','DRGX','DSP','DTX1','DUS3L','DUSP12','DUX1','DUX3','DUX4','DUXA','DVL1','DVL2','DVL3','DZIP1','DZIP1L','E2F1','E2F2','E2F3','E2F4','E2F5','E2F6','E2F7','E2F8','E4F1','EAF2','EBF1','EBF2','EBF3','EBF4','EDA','EDA2R','EDF1','EDN1','EEA1','EED','EGLN1','EGR1','EGR2','EGR3','EGR4','EHF','EID1','EID2','EIF2AK2','EIF2AK4','EIF3C','EIF3CL','EIF3K','ELF1','ELF2','ELF3','ELF4','ELF5','ELK1','ELK3','ELK4','ELMSAN1','EME1','EMX1','EMX2','EN1','EN2','ENO1','EOMES','EP300','EP400','EPAS1','EPHA5','EPM2AIP1','ERBIN','ERC1','ERCC3','ERCC4','ERF','ERG','ESR1','ESR2','ESRRA','ESRRB','ESRRG','ESX1','ETS1','ETS2','ETV1','ETV2','ETV3','ETV3L','ETV4','ETV5','ETV6','ETV7','EVX1','EVX2','EWSR1','EXO1','EXOC2','EXOG','EZH1','EZH2','FAAP24','FAM153C','FAM170A','FAM171B','FAM200A','FAM200B','FAN1','FANCA','FANCD2','FANK1','FARSA','FARSB','FBN1','FBXL19','FBXO41','FEN1','FER','FERD3L','FEV','FEZF1','FEZF2','FGD1','FHL2','FIGLA','FIZ1','FLI1','FLNA','FLYWCH1','FMNL2','FOS','FOSB','FOSL1','FOSL2','FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L2','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXN4','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1','FUBP1','FUBP3','FUS','FZD1','FZD2','FZD4','FZD6','GABPA','GABPB1','GABPB2','GADD45A','GAS6','GAS7','GATA1','GATA2','GATA3','GATA4','GATA5','GATA6','GATAD1','GATAD2A','GATAD2B','GBX1','GBX2','GCFC2','GCM1','GCM2','GEN1','GFI1','GFI1B','GLA','GLI1','GLI2','GLI3','GLI4','GLIS1','GLIS2','GLIS3','GLMP','GLYR1','GMEB1','GMEB2','GOLGB1','GON4L','GPATCH8','GPBP1','GPBP1L1','GPER1','GPR155','GREM1','GRHL1','GRHL2','GRHL3','GRM6','GSC','GSC2','GSX1','GSX2','GTF2A1','GTF2A1L','GTF2A2','GTF2B','GTF2E2','GTF2F1','GTF2F2','GTF2H1','GTF2H2','GTF2H2C','GTF2H3','GTF2H4','GTF2I','GTF2IRD1','GTF2IRD2','GTF2IRD2B','GTF3A','GTF3C1','GTF3C2','GULP1','GZF1','H1F0','H1FOO','H1FX','HAND1','HAND2','HAVCR2','HBP1','HCFC1','HCK','HDAC1','HDAC10','HDAC2','HDAC3','HDAC4','HDAC5','HDAC6','HDAC7','HDAC8','HDAC9','HDX','HELT','HELZ','HELZ2','HES1','HES2','HES3','HES4','HES5','HES6','HES7','HESX1','HEY1','HEY2','HEYL','HHEX','HIC1','HIC2','HIF1A','HIF3A','HINFP','HIPK2','HIRA','HIST1H1A','HIST1H1B','HIST1H1C','HIST1H1D','HIST1H1E','HIST1H1T','HIVEP1','HIVEP2','HIVEP3','HKR1','HLA-DQB1','HLA-DQB2','HLA-DRB1','HLA-DRB3','HLA-DRB5','HLF','HLTF','HLX','HMBOX1','HMG20A','HMG20B','HMGA1','HMGA2','HMGB1','HMGB2','HMGB3','HMGB4','HMGN1','HMGN3','HMGXB3','HMGXB4','HMOX1','HMX1','HMX2','HMX3','HNF1A','HNF1B','HNF4A','HNF4G','HNRNPA1','HNRNPA3','HNRNPAB','HNRNPCL2','HNRNPD','HNRNPDL','HNRNPK','HNRNPL','HNRNPU','HOMEZ','HOOK2','HOPX','HOXA1','HOXA10','HOXA11','HOXA13','HOXA2','HOXA3','HOXA4','HOXA5','HOXA6','HOXA7','HOXA9','HOXB1','HOXB13','HOXB2','HOXB3','HOXB4','HOXB5','HOXB6','HOXB7','HOXB8','HOXB9','HOXC10','HOXC11','HOXC12','HOXC13','HOXC4','HOXC5','HOXC6','HOXC8','HOXC9','HOXD1','HOXD10','HOXD11','HOXD12','HOXD13','HOXD3','HOXD4','HOXD8','HOXD9','HP1BP3','HR','HSBP1','HSF1','HSF2','HSF4','HSF5','HSFX1','HSFX2','HSFY1','HSFY2','HSPA1A','HSPA1B','ICAM1','ID1','ID2','ID3','ID4','IFI16','IFNB1','IGHM','IKBKB','IKBKG','IKZF1','IKZF2','IKZF3','IKZF4','IKZF5','IL10','IL1B','IL1RAP','IL4','IL5','IL6','INF2','ING4','INS','INSM1','INSM2','IQGAP1','IRAK1','IRAK2','IRAK3','IRF1','IRF2','IRF2BP1','IRF3','IRF4','IRF5','IRF6','IRF7','IRF8','IRF9','IRX1','IRX2','IRX3','IRX4','IRX5','IRX6','ISL1','ISL2','ISX','ITCH','ITGB2','ITGB3BP','IZUMO2','JADE1','JAK2','JARID2','JAZF1','JDP2','JMY','JRK','JRKL','JUN','JUNB','JUND','JUP','KANSL2','KAT2A','KAT2B','KAT5','KAT6A','KAT6B','KAT7','KAT8','KCMF1','KCNIP3','KDM1A','KDM2A','KDM2B','KDM3A','KDM5A','KDM5B','KDM5C','KDM5D','KEAP1','KIAA1549','KIAA1683','KIAA1841','KIAA1958','KIF15','KIN','KIT','KLF1','KLF10','KLF11','KLF12','KLF13','KLF14','KLF15','KLF16','KLF17','KLF2','KLF3','KLF4','KLF5','KLF6','KLF7','KLF8','KLF9','KMT2A','KMT2B','KMT2C','KMT2D','KNTC1','KRAS','KRBOX1','KRT13','KRTAP5-1','KRTAP5-9','KRTAP9-9','L3MBTL1','L3MBTL3','L3MBTL4','LARP1','LARP1B','LARP4','LARP4B','LARP6','LARP7','LBX1','LBX2','LCOR','LCORL','LEF1','LEUTX','LGALS9','LGR4','LHX1','LHX2','LHX3','LHX4','LHX5','LHX6','LHX8','LHX9','LIG1','LIG3','LIN28A','LIN28B','LIN54','LMO2','LMO4','LMX1A','LMX1B','LPP','LRP5','LRP6','LRP8','LRRFIP1','LRRFIP2','LSR','LTF','LTN1','LYAR','LYL1','LZTFL1','LZTR1','LZTS1','MACC1','MACF1','MAD2L2','MAEL','MAF','MAFA','MAFB','MAFF','MAFG','MAFK','MALT1','MAML1','MAML3','MAP2K5','MAP3K10','MAP3K13','MAP3K7','MAPK1','MAPK10','MAPK11','MAPK14','MAPK3','MAPK8','MAPK9','MATR3','MAVS','MAX','MAZ','MB21D1','MBD1','MBD2','MBD3','MBD4','MBD6','MBNL1','MBNL2','MBNL3','MBTPS2','MDFI','MDM2','MDM4','MECOM','MECP2','MED1','MED12','MED13','MED13L','MED14','MED15','MED16','MED17','MED21','MED23','MED24','MED6','MEF2A','MEF2B','MEF2C','MEF2D','MEIS1','MEIS2','MEIS3','MEN1','MEOX1','MEOX2','MESP1','MESP2','MET','MGA','MGMT','MID2','MIER1','MIER2','MIER3','MINK1','MIS18BP1','MITF','MIXL1','MKL1','MKL2','MKRN1','MKRN2','MKRN3','MKX','MLLT1','MLLT10','MLLT11','MLLT3','MLLT6','MLX','MLXIP','MLXIPL','MN1','MNAT1','MNT','MNX1','MORC3','MPG','MRPL28','MRRF','MSANTD1','MSANTD3','MSANTD4','MSC','MSGN1','MSH2','MSH3','MSH6','MSL3','MSRB2','MST1R','MSX1','MSX2','MTA1','MTA2','MTA3','MTDH','MTERF1','MTERF2','MTERF3','MTERF4','MTF1','MTF2','MTPN','MUS81','MXD1','MXD3','MXD4','MXI1','MYB','MYBBP1A','MYBL1','MYBL2','MYC','MYCBP','MYCL','MYCN','MYD88','MYEF2','MYF5','MYF6','MYNN','MYOCD','MYOD1','MYOG','MYPOP','MYRF','MYRFL','MYSM1','MYT1','MYT1L','MZF1','NAB2','NABP2','NACC2','NAIF1','NANOG','NANOGNB','NANOGP8','NCOA1','NCOA2','NCOA3','NCOA4','NCOA5','NCOA6','NCOA7','NCOR1','NCOR2','NDC80','NDN','NDP','NEB','NEIL1','NEUROD1','NEUROD2','NEUROD4','NEUROD6','NEUROG1','NEUROG2','NEUROG3','NFAM1','NFAT5','NFATC1','NFATC2','NFATC3','NFATC4','NFE2','NFE2L1','NFE2L2','NFE2L3','NFE4','NFIA','NFIB','NFIC','NFIL3','NFIX','NFKB1','NFKB2','NFKBIA','NFKBIB','NFKBID','NFKBIL1','NFKBIZ','NFRKB','NFX1','NFXL1','NFYA','NFYB','NFYC','NHLH1','NHLH2','NKRF','NKX1-1','NKX1-2','NKX2-1','NKX2-2','NKX2-3','NKX2-4','NKX2-5','NKX2-6','NKX2-8','NKX3-1','NKX3-2','NKX6-1','NKX6-2','NKX6-3','NLRC3','NLRC4','NLRC5','NLRP12','NLRP2B','NLRP3','NME2','NOBOX','NOC3L','NOC4L','NOCT','NOD1','NOD2','NODAL','NONO','NOP2','NOTCH1','NOTCH2','NOTCH3','NOTO','NPAS1','NPAS2','NPAS3','NPAS4','NPM1','NR0B1','NR0B2','NR1D1','NR1D2','NR1H2','NR1H3','NR1H4','NR1I2','NR1I3','NR2C1','NR2C2','NR2E1','NR2E3','NR2F1','NR2F2','NR2F6','NR3C1','NR3C2','NR4A1','NR4A2','NR4A3','NR5A1','NR5A2','NR6A1','NRBF2','NRF1','NRG1','NRIP1','NRIP3','NRK','NRL','NSD1','NSD2','NTRK1','NUCKS1','NUFIP1','NUP98','NUPL2']
TFs2Other=['OGG1','OLIG1','OLIG2','OLIG3','ONECUT1','ONECUT2','ONECUT3','OPRD1','OSR1','OSR2','OTOP3','OTP','OTULIN','OTX1','OTX2','OVOL1','OVOL2','OVOL3','PA2G4','PABPN1','PARK7','PARP1','PARP10','PARP12','PATZ1','PAWR','PAX1','PAX2','PAX3','PAX4','PAX5','PAX6','PAX7','PAX8','PAX9','PAXBP1','PAXIP1','PBRM1','PBX1','PBX2','PBX3','PBX4','PCBD1','PCBP1','PCBP2','PCBP3','PCGF2','PCGF6','PCNA','PCSK6','PDCD7','PDS5B','PDX1','PEG3','PELI1','PER1','PER2','PER3','PEX14','PFDN1','PGBD1','PGR','PGS1','PHB','PHB2','PHF1','PHF19 ','PHF20','PHF21A','PHF5A','PHOX2A','PHOX2B','PHTF1','PHTF2','PIAS1','PIAS2','PIAS3','PIAS4','PIDD1','PIKFYVE','PIM1','PIN1','PINK1','PITX1','PITX2','PITX3','PKHD1','PKHD1L1','PKNOX1','PKNOX2','PLA2G10','PLA2G1B','PLAG1','PLAGL1','PLAGL2','PLEK','PLEK2','PLEKHA4','PLPP3','PLSCR1','PLXNA1','PLXNA2','PLXNA3','PLXNA4','PLXNB1','PLXNB2','PLXNB3','PLXNC1','PLXND1','PML','PMS1','PNKP','POGK','POGZ','POLA1','POLB','POLE3','POLE4','POLG','POLH','POLI','POLK','POLL','POLM','POLN','POLQ','POLR1B','POLR2A','POLR2B','POLR2E','POLR2J','POLR2K','POLR2L','POLR3A','POLRMT','POT1','POU1F1','POU2AF1','POU2F1','POU2F2','POU2F3','POU3F1','POU3F2','POU3F3','POU3F4','POU4F1','POU4F2','POU4F3','POU5F1','POU5F1B','POU5F2','POU6F1','POU6F2','PPARA','PPARD','PPARG','PPARGC1A','PPARGC1B','PPP1R10','PPP1R13L','PPP1R21','PPP2R3B','PPP2R5B','PPP3CA','PPRC1','PQBP1','PRAM1','PRAP1','PRB3','PRDM1','PRDM10','PRDM11','PRDM12','PRDM13','PRDM14','PRDM15','PRDM16','PRDM2','PRDM4','PRDM5','PRDM6','PRDM7','PRDM8','PRDM9','PRDX3','PREB','PREX1','PREX2','PRH2','PRIM2','PRIMPOL','PRKCB','PRKCH','PRKCI','PRKCQ','PRKCZ','PRKD1','PRKD2','PRKN','PRMT2','PRMT3','PRNP','PROP1','PROX1','PROX2','PRR12','PRR3','PRRX1','PRRX2','PSMA6','PSMC5','PSMD10','PSMD11','PSMD12','PSPC1','PTCH1','PTEN','PTF1A','PTGIS','PTH','PTTG1','PUM3','PURA','PURB','PURG','PYCARD','PYDC1','PYDC2','QRICH1','RAB11FIP4','RAB7B','RAD21','RAD51','RAG1','RAI1','RAPGEF3','RAPGEF4','RAPGEF5','RARA','RARB','RARG','RAX','RAX2','RB1','RBAK','RBBP7','RBCK1','RBL1','RBL2','RBM10','RBM20','RBM22','RBM26','RBM27','RBM5','RBM6','RBPJ','RBPJL','RBSN','RC3H1','RC3H2','RCAN1','RCOR1','RCOR2','RCOR3','RECQL','REL','RELA','RELB','RELL2','RELN','REPIN1','RERE','REST','REV1','REXO4','RFC1','RFX1','RFX2','RFX3','RFX4','RFX5','RFX6','RFX7','RFX8','RFXANK','RFXAP','RGCC','RGS11','RGS6','RGS7','RGS9','RHEBL1','RHOXF1','RHOXF2','RHOXF2B','RING1','RIOK2','RIPK1','RIPK2','RIPK3','RIPK4','RLF','RLIM','RNASE2','RNASEH1','RNF113A','RNF113B','RNF114','RNF125','RNF138','RNF14','RNF166','RNF2','RNF219','RNF25','RNF31','RNF4','RNF41','RORA','RORB','RORC','RPA2','RPA4','RPL36A','RPL7','RPL7A','RPS27A','RPS6KA4','RPS6KA5','RREB1','RRN3','RUNX1','RUNX1T1','RUNX2','RUNX3','RWDD3','RXRA','RXRB','RXRG','RYBP','S100A12','S100A8','S100A9','SAFB','SAFB2','SALL1','SALL2','SALL3','SALL4','SART1','SATB1','SATB2','SAV1','SCAND1','SCAND2P','SCAPER','SCMH1','SCML1','SCML2','SCML4','SCOC','SCRT1','SCRT2','SCX','SEBOX','SEMA4A','SERTAD1','SERTAD2','SETBP1','SETD6','SETDB1','SETDB2','SF3A2','SF3A3','SFPQ','SFRP4','SFRP5','SFTPB','SGK1','SGK3','SGSM2','SHH','SHOX','SHOX2','SHPRH','SIGIRR','SIK1','SIM1','SIM2','SIN3A','SIN3B','SIRT1','SIRT2','SIRT3','SIRT6','SIVA1','SIX1','SIX2','SIX3','SIX4','SIX5','SIX6','SKI','SKIDA1','SKIL','SKOR1','SKOR2','SLC22A4','SLC26A10','SLC26A3','SLC2A4RG','SLC30A9','SLC39A10','SLC4A10','SMAD1','SMAD2','SMAD3','SMAD4','SMAD5','SMAD6','SMAD7','SMAD9','SMARCA1','SMARCA2','SMARCA4','SMARCA5','SMARCAL1','SMARCB1','SMARCC1','SMARCC2','SMARCD1','SMARCD2','SMARCD3','SMARCE1','SMC3','SMO','SMUG1','SMYD1','SMYD3','SNAI1','SNAI2','SNAI3','SNAPC2','SNAPC4','SNAPC5','SNRPC','SNURF','SNW1','SOGA3','SOHLH1','SOHLH2','SON','SORBS2','SOX1','SOX10','SOX11','SOX12','SOX13','SOX14','SOX15','SOX17','SOX18','SOX2','SOX21','SOX3','SOX30','SOX4','SOX5','SOX6','SOX7','SOX8','SOX9','SP1','SP100','SP110','SP140','SP140L','SP2','SP3','SP4','SP5','SP6','SP7','SP8','SP9','SPDEF','SPEN','SPHK1','SPI1','SPIB','SPIC','SPZ1','SRCAP','SREBF1','SREBF2','SRF','SRY','SSB','SSBP2','SSBP3','SSBP4','SSH1','SSH2','SSH3','SSRP1','SSX1','ST18','STAG1','STAT1','STAT2','STAT3','STAT4','STAT5A','STAT5B','STAT6','STK16','STK3','STK36','STRN','STRN3','SUB1','SUFU','SUMO1','SUPT16H','SUPT4H1','SUPT5H','SUPT6H','SUV39H1','SUV39H2','SUZ12','SYK','T','TAB1','TAB2','TAB3','TADA2A','TADA2B','TADA3','TAF1','TAF10','TAF11','TAF12','TAF13','TAF1A','TAF1B','TAF1C','TAF1L','TAF2','TAF3','TAF4','TAF4B','TAF5','TAF5L','TAF6','TAF6L','TAF7','TAF7L','TAF8','TAF9','TAF9B','TAL1','TAL2','TARDBP','TAX1BP1','TBL1Y','TBP','TBPL1','TBPL2','TBR1','TBX1','TBX10','TBX15','TBX18','TBX19','TBX2','TBX20','TBX21','TBX22','TBX3','TBX4','TBX5','TBX6','TCEA2','TCEAL1','TCEAL7','TCEAL8','TCF12','TCF15','TCF19','TCF20','TCF21','TCF23','TCF24','TCF25','TCF3','TCF4','TCF7','TCF7L1','TCF7L2','TCFL5','TDG','TDP1','TDP2','TEAD1','TEAD2','TEAD3','TEAD4','TEF','TERB1','TERF1','TERF2','TERF2IP','TESMIN','TET1','TET2','TET3','TFAM','TFAP2A','TFAP2B','TFAP2C','TFAP2D','TFAP2E','TFAP4','TFCP2','TFCP2L1','TFDP1','TFDP2','TFDP3','TFE3','TFEB','TFEC','TGFB1','TGFB1I1','TGIF1','TGIF2','TGIF2LX','TGIF2LY','TGS1','THAP1','THAP10','THAP11','THAP12','THAP2','THAP3','THAP4','THAP5','THAP6','THAP7','THAP8','THAP9','THRA','THRAP3','THRB','THYN1','TIA1','TICAM1','TIGD1','TIGD2','TIGD3','TIGD4','TIGD5','TIGD6','TIGD7','TIPARP','TIRAP','TLE1','TLE2','TLE3','TLE4','TLE6','TLR2','TLR3','TLR4','TLR9','TLX1','TLX2','TLX3','TMEM235','TMF1','TNF','TNFAIP3','TNFRSF11A','TNFRSF4','TNFSF11','TNFSF18','TNFSF4','TOE1','TONSL','TOP1','TOP2A','TOP2B','TOPORS','TOX','TOX2','TOX3','TOX4','TP53','TP53BP1','TP63','TP73','TPRX1','TRADD','TRAF1','TRAF2','TRAF3','TRAF4','TRAF5','TRAF6','TRAFD1','TRAK2','TRAPPC9','TRERF1','TREX1','TRIB1','TRIB3','TRIM13','TRIM14','TRIM15','TRIM21','TRIM22','TRIM23','TRIM24','TRIM25','TRIM26','TRIM27','TRIM28','TRIM29','TRIM3','TRIM31','TRIM32','TRIM33','TRIM34','TRIM37','TRIM38','TRIM40','TRIM5','TRIM52','TRIM62','TRIM8','TRIP11','TRIT1','TRMT1','TRMT1L','TRPS1','TRRAP','TSC2','TSC22D1','TSC22D2','TSC22D3','TSC22D4','TSHZ1','TSHZ2','TSHZ3','TSN','TSSK4','TTF1','TUB','TULP4','TUT1','TWIST1','TWIST2','TXK','U2AF1','U2AF1L4','UBA52','UBB','UBC','UBE2K','UBE2N','UBE2V1','UBN1','UBP1','UBR4','UBTF','UBTFL1','UBTFL6','UFL1','UHRF1','UHRF2','UNCX','UNG','UNK','UNKL','USF1','USF2','USF3','USP39','USP7','UTF1','VAV1','VAX1','VAX2','VDR','VEGFA','VENTX','VEZF1','VGLL1','VGLL2','VGLL3','VHLL','VPS72','VSX1','VSX2','WDHD1','WFS1','WIZ','WNT1','WNT10B','WNT2','WNT3A','WNT5A','WNT8B','WRN','WT1','WWOX','WWP2','WWTR1','XBP1','XCL1','XPA','YBX1','YBX2','YBX3','YEATS2','YEATS4','YOD1','YY1','YY2','ZBED1','ZBED2','ZBED3','ZBED4','ZBED5','ZBED6','ZBED6CL','ZBED8','ZBED9','ZBP1','ZBTB1','ZBTB10','ZBTB11','ZBTB12','ZBTB14','ZBTB16','ZBTB17','ZBTB18','ZBTB2','ZBTB20','ZBTB21','ZBTB22','ZBTB24','ZBTB25','ZBTB26','ZBTB3','ZBTB32','ZBTB33','ZBTB34','ZBTB37','ZBTB38','ZBTB39','ZBTB4','ZBTB40','ZBTB41','ZBTB42','ZBTB43','ZBTB44','ZBTB45','ZBTB46','ZBTB47','ZBTB48','ZBTB49','ZBTB5','ZBTB6','ZBTB7A','ZBTB7B','ZBTB7C','ZBTB8A','ZBTB8B','ZBTB9','ZC3H10','ZC3H11A','ZC3H12A','ZC3H13','ZC3H14','ZC3H15','ZC3H18','ZC3H3','ZC3H4','ZC3H6','ZC3H7A','ZC3H7B','ZC3H8','ZC3HAV1','ZCCHC11','ZCCHC6','ZDHHC11','ZDHHC19','ZEB1','ZEB2','ZFAND5','ZFAT','ZFHX2','ZFHX3','ZFHX4','ZFP1','ZFP14','ZFP2','ZFP28','ZFP3','ZFP30','ZFP36','ZFP36L1','ZFP36L2','ZFP37','ZFP41','ZFP42','ZFP57','ZFP62','ZFP64','ZFP69','ZFP69B','ZFP82','ZFP90','ZFP91','ZFP92','ZFPL1','ZFPM1','ZFPM2','ZFR','ZFR2','ZFX','ZFY','ZFYVE26','ZGLP1','ZGPAT','ZHX1','ZHX2','ZHX3','ZIC1','ZIC2','ZIC3','ZIC4','ZIC5','ZIK1','ZIM2','ZIM3','ZKSCAN1','ZKSCAN2','ZKSCAN3','ZKSCAN4','ZKSCAN5','ZKSCAN7','ZKSCAN8','ZMAT1','ZMAT2','ZMAT3','ZMAT4','ZMAT5','ZMYM1','ZMYM2','ZMYM3','ZMYM4','ZMYM5','ZMYM6','ZMYND10','ZNF10','ZNF100','ZNF101','ZNF106','ZNF107','ZNF112','ZNF114','ZNF117','ZNF12','ZNF121','ZNF124','ZNF131','ZNF132','ZNF133','ZNF134','ZNF135','ZNF136','ZNF137P','ZNF138','ZNF14','ZNF140','ZNF141','ZNF142','ZNF143','ZNF146','ZNF148','ZNF154','ZNF155','ZNF157','ZNF16','ZNF160','ZNF165','ZNF169','ZNF17','ZNF174','ZNF175','ZNF177','ZNF18','ZNF180','ZNF181','ZNF182','ZNF184','ZNF185','ZNF189','ZNF19','ZNF195','ZNF197','ZNF2','ZNF20','ZNF200','ZNF202','ZNF205','ZNF207','ZNF208','ZNF211','ZNF212','ZNF213','ZNF214','ZNF215','ZNF217','ZNF219','ZNF22','ZNF221','ZNF222','ZNF223','ZNF224','ZNF225','ZNF226','ZNF227','ZNF229','ZNF23','ZNF230','ZNF232','ZNF233','ZNF234','ZNF235','ZNF236','ZNF239','ZNF24','ZNF248','ZNF25','ZNF250','ZNF251','ZNF253','ZNF254','ZNF256','ZNF257','ZNF26','ZNF260','ZNF263','ZNF264','ZNF266','ZNF267','ZNF268','ZNF273','ZNF274','ZNF275','ZNF276','ZNF277','ZNF28','ZNF280A','ZNF280B','ZNF280C','ZNF280D','ZNF281','ZNF282','ZNF283','ZNF284','ZNF285','ZNF286A','ZNF286B','ZNF287','ZNF292','ZNF296','ZNF3','ZNF30','ZNF300','ZNF302','ZNF304','ZNF311','ZNF316','ZNF317','ZNF318','ZNF319','ZNF32','ZNF320','ZNF322','ZNF324','ZNF324B','ZNF326','ZNF329','ZNF330','ZNF331','ZNF333','ZNF334','ZNF335','ZNF337','ZNF33A','ZNF33B','ZNF34','ZNF341','ZNF343','ZNF345','ZNF346','ZNF347','ZNF35','ZNF350','ZNF354A','ZNF354B','ZNF354C','ZNF355P','ZNF358','ZNF362','ZNF365','ZNF366','ZNF367','ZNF37A','ZNF382','ZNF383','ZNF384','ZNF385A','ZNF385B','ZNF385C','ZNF385D','ZNF391','ZNF394','ZNF395','ZNF396','ZNF397','ZNF398','ZNF404','ZNF407','ZNF408','ZNF41','ZNF410','ZNF414','ZNF415','ZNF416','ZNF417','ZNF418','ZNF419','ZNF420','ZNF423','ZNF425','ZNF426','ZNF428','ZNF429','ZNF43','ZNF430','ZNF431','ZNF432','ZNF433','ZNF436','ZNF438','ZNF439','ZNF44','ZNF440','ZNF441','ZNF442','ZNF443','ZNF444','ZNF445','ZNF446','ZNF449','ZNF45','ZNF451','ZNF454','ZNF460','ZNF461','ZNF462','ZNF467','ZNF468','ZNF469','ZNF470','ZNF471','ZNF473','ZNF474','ZNF479','ZNF48','ZNF480','ZNF483','ZNF484','ZNF485','ZNF486','ZNF487','ZNF488','ZNF490','ZNF491','ZNF492','ZNF493','ZNF496','ZNF497','ZNF500','ZNF501','ZNF502','ZNF503','ZNF506','ZNF507','ZNF510','ZNF511','ZNF512','ZNF512B','ZNF513','ZNF514','ZNF516','ZNF517','ZNF518A','ZNF518B','ZNF519']
TFs3Other=['ZNF521','ZNF524','ZNF525','ZNF526','ZNF527','ZNF528','ZNF529','ZNF530','ZNF532','ZNF534','ZNF536','ZNF540','ZNF541','ZNF542P','ZNF543','ZNF544','ZNF546','ZNF547','ZNF548','ZNF549','ZNF550','ZNF551','ZNF552','ZNF554','ZNF555','ZNF556','ZNF557','ZNF558','ZNF559','ZNF560','ZNF561','ZNF562','ZNF563','ZNF564','ZNF565','ZNF566','ZNF567','ZNF568','ZNF569','ZNF57','ZNF570','ZNF571','ZNF572','ZNF573','ZNF574','ZNF575','ZNF576','ZNF577','ZNF578','ZNF579','ZNF580','ZNF581','ZNF582','ZNF583','ZNF584','ZNF585A','ZNF585B','ZNF586','ZNF587','ZNF587B','ZNF589','ZNF592','ZNF593','ZNF594','ZNF595','ZNF596','ZNF597','ZNF598','ZNF599','ZNF600','ZNF605','ZNF606','ZNF607','ZNF608','ZNF609','ZNF610','ZNF611','ZNF613','ZNF614','ZNF615','ZNF616','ZNF618','ZNF619','ZNF620','ZNF621','ZNF622','ZNF623','ZNF624','ZNF625','ZNF626','ZNF627','ZNF628','ZNF629','ZNF630','ZNF638','ZNF639','ZNF641','ZNF644','ZNF645','ZNF646','ZNF648','ZNF649','ZNF652','ZNF653','ZNF654','ZNF655','ZNF658','ZNF66','ZNF660','ZNF662','ZNF664','ZNF665','ZNF667','ZNF668','ZNF669','ZNF670','ZNF671','ZNF672','ZNF674','ZNF675','ZNF676','ZNF677','ZNF678','ZNF679','ZNF680','ZNF681','ZNF682','ZNF683','ZNF684','ZNF687','ZNF688','ZNF689','ZNF69','ZNF691','ZNF692','ZNF695','ZNF696','ZNF697','ZNF699','ZNF7','ZNF70','ZNF700','ZNF701','ZNF703','ZNF704','ZNF705A','ZNF705B','ZNF705D','ZNF705E','ZNF705G','ZNF706','ZNF707','ZNF708','ZNF709','ZNF71','ZNF710','ZNF711','ZNF713','ZNF714','ZNF716','ZNF717','ZNF718','ZNF720','ZNF721','ZNF724','ZNF726','ZNF727','ZNF728','ZNF729','ZNF73','ZNF730','ZNF732','ZNF735','ZNF736','ZNF737','ZNF738','ZNF74','ZNF740','ZNF746','ZNF747','ZNF749','ZNF750','ZNF75A','ZNF75D','ZNF76','ZNF761','ZNF763','ZNF764','ZNF765','ZNF766','ZNF768','ZNF77','ZNF770','ZNF771','ZNF772','ZNF773','ZNF774','ZNF775','ZNF776','ZNF777','ZNF778','ZNF780A','ZNF780B','ZNF781','ZNF782','ZNF783','ZNF784','ZNF785','ZNF786','ZNF787','ZNF788','ZNF789','ZNF79','ZNF790','ZNF791','ZNF792','ZNF793','ZNF799','ZNF8','ZNF80','ZNF800','ZNF804A','ZNF804B','ZNF805','ZNF808','ZNF81','ZNF812P','ZNF813','ZNF814','ZNF816','ZNF821','ZNF823','ZNF827','ZNF829','ZNF83','ZNF830','ZNF831','ZNF835','ZNF836','ZNF837','ZNF839','ZNF84','ZNF841','ZNF843','ZNF844','ZNF845','ZNF846','ZNF85','ZNF850','ZNF852','ZNF853','ZNF860','ZNF862','ZNF865','ZNF878','ZNF879','ZNF880','ZNF883','ZNF888','ZNF891','ZNF90','ZNF91','ZNF92','ZNF93','ZNF98','ZNF99','ZNFX1','ZRANB2','ZRSR1','ZRSR2','ZSCAN1','ZSCAN10','ZSCAN12','ZSCAN16','ZSCAN18','ZSCAN2','ZSCAN20','ZSCAN21','ZSCAN22','ZSCAN23','ZSCAN25','ZSCAN26','ZSCAN29','ZSCAN30','ZSCAN31','ZSCAN32','ZSCAN4','ZSCAN5A','ZSCAN5B','ZSCAN5C','ZSCAN5DP','ZSCAN9','ZUFSP','ZXDA','ZXDB','ZXDC','ZZZ3']
TFs4Other=['ATF2','ATF6B','ATN1','BHLHA15','BHLHB2','BHLHB4 ','BHLHB5','BHLHB9','C13ORF8','C20ORF194','CDX2','CRAMP1L','ELP3','ERVK3-1','ETF1','FLJ12895','FLJ20321','FLJ25476','FLJ35784','FOXM1','FOXO2','FOXR1','FOXR2','GATA1','GRLF1','KDM4C','KDM4E','KIAA0863','KIAA0961','KLRG1','LBXCOR1','LEF1','LHX7','LOC126295','LOC51058','LOC91661','MGC4728','MYOD1','MYOG','MYRF','NEUROD4','NEUROG1','NFAT2','NFAT4','NFE2L1','NR1A2','NR1B1','NR1B2','NR1B3','NR1C1','NR1C2','NR1C3','NR1F3','NR1I1','NR2A1','NR2B3','NR3A1','NR3A2','NR3B3','OTEX','PEPP-2','PHF2','POU2F1','POU2F2','POU4F3','POU5F1','POU6F1','PRDM3','PRKRIP1','PRKRIR','SETD1A','SMAD8','SMYD2','SOX12','SOX14','SOX15','TBX14','TCF1','TCF17','TCF2','TCF9','TEAD1','TPRXL','UBTFL1','WHSC1','ZBTB14','ZBTB18','ZBTB19','ZBTB21','ZBTB27','ZBTB28','ZBTB35','ZBTB49','ZFP161','ZFP28','ZKSCAN10','ZKSCAN11','ZKSCAN12','ZKSCAN13','ZKSCAN14','ZKSCAN16','ZKSCAN19','ZKSCAN21','ZKSCAN24','ZKSCAN6','ZKSCAN8','ZNF10','ZNF112','ZNF187','ZNF193','ZNF204P','ZNF238','ZNF271','ZNF295','ZNF305','ZNF323','ZNF434','ZNF499','ZNF542','ZNF545','ZNF553','ZNF56','ZNF625','ZNF643','ZNF698','ZNF705F','ZNF751','ZNF752','ZNF753','ZNF785','ZNF806','ZNF816','ZNF833P','ZNF906','ZSCAN1','ZSCAN25','ZSCAN26','ZSCAN31','ZSCAN32','ZSCAN5D','ZSCAN9']
TFsOther=TFs1Other+TFs2Other+TFs3Other+TFs4Other
```

Example 8 (unknown):
```unknown
S genes: ['MCM5', 'PCNA', 'TYMS', 'FEN1', 'MCM2', 'MCM4', 'RRM1', 'UNG', 'GINS2', 'MCM6', 'CDCA7', 'DTL', 'PRIM1', 'UHRF1', 'HELLS', 'RFC2', 'RPA2', 'NASP', 'RAD51AP1', 'GMNN', 'WDR76', 'SLBP', 'CCNE2', 'UBR7', 'POLD3', 'MSH2', 'ATAD2', 'RAD51', 'RRM2', 'CDC45', 'CDC6', 'EXO1', 'TIPIN', 'DSCC1', 'BLM', 'CASP8AP2', 'USP1', 'CLSPN', 'POLA1', 'CHAF1B', 'BRIP1', 'E2F8'] 

G2 genes: ['HMGB2', 'CDK1', 'NUSAP1', 'UBE2C', 'BIRC5', 'TPX2', 'TOP2A', 'NDC80', 'CKS2', 'NUF2', 'CKS1B', 'MKI67', 'TMPO', 'CENPF', 'TACC3', 'SMC4', 'CCNB2', 'CKAP2L', 'CKAP2', 'AURKB', 'BUB1', 'KIF11', 'ANP32E', 'TUBB4B', 'GTSE1', 'KIF20B', 'HJURP', 'CDCA3', 'CDC20', 'TTK', 'CDC25C', 'KIF2C', 'RANGAP1', 'NCAPD2', 'DLGAP5', 'CDCA2', 'CDCA8', 'ECT2', 'KIF23', 'HMMR', 'AURKA', 'PSRC1', 'ANLN', 'LBR', 'CKAP5', 'CENPE', 'CTCF', 'NEK2', 'G2E3', 'GAS2L3', 'CBX5', 'CENPA'] 

Hemoglobin genes set 1: ['HBB', 'HBD', 'HBG1', 'HBG2', 'HBE1', 'HBZ', 'HBM', 'HBA2', 'HBA1', 'HBQ1'] 

Hemoglobin genes set 2: ['AHSP', 'GYPA', 'RHAG', 'GFI1B', 'KEL', 'EPB42', 'ALAS2', 'TSPO2', 'GYPB', 'GYPE', 'SPTA1', 'RHCE', 'KLF1', 'RHD', 'SLC4A1', 'SNX22', 'SQSTM1', 'SOD1', 'FTH1', 'MAP1LC3B', 'SLC25A37', 'TRIM58', 'HMBS'] 

Mitochondrial genes: ['MT-ND1', 'MT-ND2', 'MT-CO1', 'MT-CO2', 'MT-ATP8', 'MT-ATP6', 'MT-CO3', 'MT-ND3', 'MT-ND4L', 'MT-ND4', 'MT-ND5', 'MT-ND6', 'MT-CYB'] 

Ribo genes: ['RPLP0', 'RPLP1', 'RPLP2', 'RPL3', 'RPL3L', 'RPL4', 'RPL5', 'RPL6', 'RPL7', 'RPL7A', 'RPL7L1', 'RPL8', 'RPL9', 'RPL10', 'RPL10A', 'RPL10L', 'RPL11', 'RPL12', 'RPL13', 'RPL13A', 'RPL14', 'RPL15', 'RPL17', 'RPL18', 'RPL18A', 'RPL19', 'RPL21', 'RPL22', 'RPL22L1', 'RPL23', 'RPL23A', 'RPL24', 'RPL26', 'RPL26L1', 'RPL27', 'RPL27A', 'RPL28', 'RPL29', 'RPL30', 'RPL31', 'RPL32', 'RPL34', 'RPL35', 'RPL35A', 'RPL36', 'RPL36A', 'RPL36AL', 'RPL37', 'RPL37A', 'RPL38', 'RPL39', 'RPL39L', 'UBA52', 'RPL41', 'RPSA', 'RPS2', 'RPS3', 'RPS3A', 'RPS4X', 'RPS4Y1', 'RPS4Y2', 'RPS5', 'RPS6', 'RPS7', 'RPS8', 'RPS9', 'RPS10', 'RPS11', 'RPS12', 'RPS13', 'RPS14', 'RPS15', 'RPS15A', 'RPS16', 'RPS17', 'RPS18', 'RPS19', 'RPS20', 'RPS21', 'RPS23', 'RPS24', 'RPS25', 'RPS26', 'RPS27', 'RPS27A', 'RPS27L', 'RPS28', 'RPS29', 'FAU'] 

Tissue dissociation stress associated genes: ['FOS', 'FOSB', 'JUN', 'JUNB', 'JUND', 'ZFP36', 'SOCS3', 'CEBPD', 'CEBPB', 'ATF3', 'EGR1', 'HSPA1A', 'HSPA1B', 'HSP90AA1', 'HSP90AB1', 'HSPA8', 'HSPB1', 'HSPE1', 'IER3', 'IER2', 'BTG1', 'BTG2', 'DUSP1']
```

Example 9 (python):
```python
set(adata.obs.coarse_grain)
```

Example 10 (unknown):
```unknown
{'Cardiomyocytes',
 'Endothelium',
 'Epicardium',
 'Immune',
 'Mesenchymal',
 'Neural'}
```

Example 11 (python):
```python
adata=adata[adata.obs.coarse_grain=='Endothelium',:].copy()
adata
```

Example 12 (unknown):
```unknown
AnnData object with n_obs × n_vars = 21882 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'batch_key', '_scvi_batch', '_scvi_labels', 'previous_coarse_grain', 'previous_mid_grain', 'previous_fine_grain', 'leiden_scVI', 'FACSgate', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', 'hb2_score', 'leiden', 'coarse_grain', 'sex'
    var: 'gene_name_scRNA-0', 'gene_id'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'dendrogram_leiden', 'donor_colors', 'gender_colors', 'hvg', 'kit_10x_colors', 'leiden', 'leiden_colors', 'modality_colors', 'neighbors', 'phase_colors', 'rank_genes_groups', 'region_colors', 'sex_colors', 'umap'
    obsm: 'X_pca', 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 13 (python):
```python
# preserve raw count
adata.layers["counts"] = adata.X.copy()

# normalise and log-transform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key='batch_key')

print(sum(adata.var.highly_variable))
sc.pl.highly_variable_genes(adata)

# freeze the state in `.raw`
adata.raw = adata
```

Example 14 (python):
```python
normalizing counts per cell
    finished (0:00:00)
extracting highly variable genes
    finished (0:00:09)
--> added
    'highly_variable', boolean vector (adata.var)
    'means', float vector (adata.var)
    'dispersions', float vector (adata.var)
    'dispersions_norm', float vector (adata.var)
2625
```

Example 15 (python):
```python
# subset hvgs
adata = adata[:, adata.var.highly_variable]

adata=adata.copy()

sc.tl.score_genes_cell_cycle(adata, s_genes=s_genes, g2m_genes=g2m_genes)
#sc.pp.regress_out(adata, ['S_score','G2M_score'])

#batch correction
scvi.model.SCVI.setup_anndata(adata, 
                              layer="counts", 
                              batch_key="batch_key",
                              categorical_covariate_keys=['phase'],
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo'])

vae = scvi.model.SCVI(adata, n_hidden = 128, n_latent = 50, n_layers = 3, dispersion = 'gene-batch')
vae.train(max_epochs=50)
    
# Convergence plot
x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
plt.plot(x, vae.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
plt.clf()
```

Example 16 (python):
```python
calculating cell cycle phase
computing score 'S_score'
WARNING: genes are not in var_names and ignored: ['FEN1', 'UNG', 'GINS2', 'MCM6', 'CDCA7', 'PRIM1', 'RFC2', 'RPA2', 'NASP', 'GMNN', 'SLBP', 'CCNE2', 'POLD3', 'MSH2', 'RAD51', 'CDC45', 'CDC6', 'EXO1', 'TIPIN', 'DSCC1', 'CASP8AP2', 'USP1', 'POLA1', 'CHAF1B', 'E2F8']
    finished: added
    'S_score', score of gene set (adata.obs).
    498 total control genes are used. (0:00:00)
computing score 'G2M_score'
WARNING: genes are not in var_names and ignored: ['TMPO', 'ANP32E', 'CDC25C', 'CDCA2', 'LBR', 'CKAP5', 'CTCF', 'G2E3', 'CBX5']
    finished: added
    'G2M_score', score of gene set (adata.obs).
    779 total control genes are used. (0:00:00)
-->     'phase', cell cycle phase (adata.obs)
```

Example 17 (unknown):
```unknown
GPU available: False, used: False
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
```

Example 18 (unknown):
```unknown
Epoch 50/50: 100%|███████| 50/50 [01:54<00:00,  2.29s/it, loss=1.2e+03, v_num=1]
```

Example 19 (unknown):
```unknown
<Figure size 320x320 with 0 Axes>
```

Example 20 (python):
```python
# add the latent space to adata
adata.obsm["X_scVI"] = vae.get_latent_representation()

sc.tl.score_genes(adata, stress_genes, score_name='stress_score')
sc.tl.score_genes(adata, hb_genes1, score_name='hb1_score')
sc.tl.score_genes(adata, hb_genes2, score_name='hb2_score')
sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata,min_dist=0.1)
#sc.tl.tsne(adata)
```

Example 21 (python):
```python
computing score 'stress_score'
WARNING: genes are not in var_names and ignored: ['JUND', 'CEBPB', 'HSP90AA1', 'HSP90AB1', 'HSPA8', 'HSPB1', 'HSPE1', 'BTG1']
    finished: added
    'stress_score', score of gene set (adata.obs).
    342 total control genes are used. (0:00:00)
computing score 'hb1_score'
WARNING: genes are not in var_names and ignored: ['HBD', 'HBM', 'HBQ1']
    finished: added
    'hb1_score', score of gene set (adata.obs).
    299 total control genes are used. (0:00:00)
computing score 'hb2_score'
WARNING: genes are not in var_names and ignored: ['GYPA', 'RHAG', 'GFI1B', 'KEL', 'EPB42', 'ALAS2', 'TSPO2', 'GYPB', 'GYPE', 'SPTA1', 'RHCE', 'KLF1', 'RHD', 'SLC4A1', 'SNX22', 'SQSTM1', 'SOD1', 'FTH1', 'MAP1LC3B', 'SLC25A37', 'TRIM58', 'HMBS']
    finished: added
    'hb2_score', score of gene set (adata.obs).
    50 total control genes are used. (0:00:00)
computing neighbors
    finished: added to `.uns['neighbors']`
    `.obsp['distances']`, distances for each pair of neighbors
    `.obsp['connectivities']`, weighted adjacency matrix (0:00:03)
computing UMAP
    finished: added
    'X_umap', UMAP coordinates (adata.obsm) (0:00:06)
```

Example 22 (python):
```python
sc.pl.umap(adata, color=['log1p_total_counts','n_genes','n_genes_by_counts','total_counts','pct_counts_mt','pct_counts_ribo'],ncols=5)
sc.pl.umap(adata, color=['scrublet_score','stress_score', 'hb1_score','hb2_score','phase'], ncols=5,)
sc.pl.umap(adata, color=['donor','sex','age','region','phase','cell_or_nuclei','modality','kit_10x','FACSgate'],wspace=0.8,ncols=5)
```

Example 23 (unknown):
```unknown
... storing 'phase' as categorical
```

Example 24 (python):
```python
Genes=['TTN','TNNT2','MYOM1','LDB3','ALDH1A2','TNNT1','KRT19','UPK3B','COL6A3','FBLN1','PCOLCE','LRRC17','EMCN','ECSCR','PECAM1','CDH5', 'LAPTM5','PTPRC','LCP1','CD53','PHOX2B','NRXN1','STMN2','GRIK2']
sc.pl.umap(adata, color=Genes)
```

Example 25 (python):
```python
sc.pl.umap(adata, color='previous_fine_grain')
```

Example 26 (python):
```python
#this block is to see where the cells over a certain threshold with TNNT2 expression lies in the UMAP
#enter what gene to set threshold for
#when running this box for the first time, comment the starting with 'del'
#then uncomment it for the other ones --- !!!

CellName='Cardio_toremove'
gene='TTN'
exp_thresh=3
gene_pos=adata.var_names.get_loc(gene)
adata_sel = adata[adata.X[:,gene_pos]>exp_thresh].copy()

l=adata_sel.obs_names
set(l).intersection(adata.obs_names)
print(len(l))

#del adata.obs[CellName]
adata.obs.loc[set(l).intersection(adata.obs_names),CellName]='Yes'
sc.settings.set_figure_params(dpi=200, facecolor='white', color_map = 'RdPu',)
sc.pl.umap(adata, color=[CellName], groups='Yes',size=10, wspace=0.45, legend_fontsize=5, legend_loc='on data',frameon=False)

print(adata)
adata= adata[adata.X[:,gene_pos]<exp_thresh].copy()
print(adata)
```

Example 27 (unknown):
```unknown
... storing 'Cardio_toremove' as categorical
```

Example 28 (unknown):
```unknown
AnnData object with n_obs × n_vars = 21882 × 2625
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'batch_key', '_scvi_batch', '_scvi_labels', 'previous_coarse_grain', 'previous_mid_grain', 'previous_fine_grain', 'leiden_scVI', 'FACSgate', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', 'hb2_score', 'leiden', 'coarse_grain', 'sex', 'Cardio_toremove'
    var: 'gene_name_scRNA-0', 'gene_id', 'highly_variable', 'means', 'dispersions', 'dispersions_norm', 'highly_variable_nbatches', 'highly_variable_intersection'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'dendrogram_leiden', 'donor_colors', 'gender_colors', 'hvg', 'kit_10x_colors', 'leiden', 'leiden_colors', 'modality_colors', 'neighbors', 'phase_colors', 'rank_genes_groups', 'region_colors', 'sex_colors', 'umap', 'log1p', 'age_colors', 'previous_fine_grain_colors', 'Cardio_toremove_colors'
    obsm: 'X_pca', 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    layers: 'counts'
    obsp: 'connectivities', 'distances'
AnnData object with n_obs × n_vars = 21273 × 2625
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'batch_key', '_scvi_batch', '_scvi_labels', 'previous_coarse_grain', 'previous_mid_grain', 'previous_fine_grain', 'leiden_scVI', 'FACSgate', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', 'hb2_score', 'leiden', 'coarse_grain', 'sex', 'Cardio_toremove'
    var: 'gene_name_scRNA-0', 'gene_id', 'highly_variable', 'means', 'dispersions', 'dispersions_norm', 'highly_variable_nbatches', 'highly_variable_intersection'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'dendrogram_leiden', 'donor_colors', 'gender_colors', 'hvg', 'kit_10x_colors', 'leiden', 'leiden_colors', 'modality_colors', 'neighbors', 'phase_colors', 'rank_genes_groups', 'region_colors', 'sex_colors', 'umap', 'log1p', 'age_colors', 'previous_fine_grain_colors', 'Cardio_toremove_colors'
    obsm: 'X_pca', 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    layers: 'counts'
    obsp: 'connectivities', 'distances'
```

Example 29 (python):
```python
#this block is to see where the cells over a certain threshold with TNNT2 expression lies in the UMAP
#enter what gene to set threshold for
#when running this box for the first time, comment the starting with 'del'
#then uncomment it for the other ones --- !!!

CellName='Mesenchymal_toremove'
gene='COL6A3'
exp_thresh=1.5
gene_pos=adata.var_names.get_loc(gene)
adata_sel = adata[adata.X[:,gene_pos]>exp_thresh].copy()

l=adata_sel.obs_names
set(l).intersection(adata.obs_names)
print(len(l))

#del adata.obs[CellName]
adata.obs.loc[set(l).intersection(adata.obs_names),CellName]='Yes'
sc.settings.set_figure_params(dpi=200, facecolor='white', color_map = 'RdPu',)
sc.pl.umap(adata, color=[CellName], groups='Yes',size=10, wspace=0.45, legend_fontsize=5, legend_loc='on data',frameon=False)

print(adata)
adata= adata[adata.X[:,gene_pos]<exp_thresh].copy()
print(adata)
```

Example 30 (unknown):
```unknown
... storing 'Mesenchymal_toremove' as categorical
```

Example 31 (unknown):
```unknown
AnnData object with n_obs × n_vars = 21273 × 2625
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'batch_key', '_scvi_batch', '_scvi_labels', 'previous_coarse_grain', 'previous_mid_grain', 'previous_fine_grain', 'leiden_scVI', 'FACSgate', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', 'hb2_score', 'leiden', 'coarse_grain', 'sex', 'Cardio_toremove', 'Mesenchymal_toremove'
    var: 'gene_name_scRNA-0', 'gene_id', 'highly_variable', 'means', 'dispersions', 'dispersions_norm', 'highly_variable_nbatches', 'highly_variable_intersection'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'dendrogram_leiden', 'donor_colors', 'gender_colors', 'hvg', 'kit_10x_colors', 'leiden', 'leiden_colors', 'modality_colors', 'neighbors', 'phase_colors', 'rank_genes_groups', 'region_colors', 'sex_colors', 'umap', 'log1p', 'age_colors', 'previous_fine_grain_colors', 'Cardio_toremove_colors', 'Mesenchymal_toremove_colors'
    obsm: 'X_pca', 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    layers: 'counts'
    obsp: 'connectivities', 'distances'
AnnData object with n_obs × n_vars = 20129 × 2625
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'batch_key', '_scvi_batch', '_scvi_labels', 'previous_coarse_grain', 'previous_mid_grain', 'previous_fine_grain', 'leiden_scVI', 'FACSgate', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', 'hb2_score', 'leiden', 'coarse_grain', 'sex', 'Cardio_toremove', 'Mesenchymal_toremove'
    var: 'gene_name_scRNA-0', 'gene_id', 'highly_variable', 'means', 'dispersions', 'dispersions_norm', 'highly_variable_nbatches', 'highly_variable_intersection'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'dendrogram_leiden', 'donor_colors', 'gender_colors', 'hvg', 'kit_10x_colors', 'leiden', 'leiden_colors', 'modality_colors', 'neighbors', 'phase_colors', 'rank_genes_groups', 'region_colors', 'sex_colors', 'umap', 'log1p', 'age_colors', 'previous_fine_grain_colors', 'Cardio_toremove_colors', 'Mesenchymal_toremove_colors'
    obsm: 'X_pca', 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    layers: 'counts'
    obsp: 'connectivities', 'distances'
```

Example 32 (python):
```python
#Re-run analyis with cardio-mesenchymal removed

adata_raw=sc.read('./Objects/20221130RNA_foetal_post-scVI_raw_ExtracardiacRemoved.h5ad')

print(adata_raw)
# constructing new anndata with maintaining .obs, .obsm, .obsp, .uns
# overwriting adata_raw
adata_raw=anndata.AnnData(X=adata_raw[adata.obs_names,:].X,
                      obs=adata.obs, # from annotated or subsetted data
                      var=adata_raw.var,
                      obsm=adata.obsm, # from annotated or subsetted data
                      obsp=adata.obsp, # from annotated or subsetted data
                    uns=adata.uns # from annotated or subsetted data
                               )

adata=adata_raw.copy()
print(adata)
```

Example 33 (unknown):
```unknown
AnnData object with n_obs × n_vars = 306287 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'batch_key', '_scvi_batch', '_scvi_labels', 'previous_coarse_grain', 'previous_mid_grain', 'previous_fine_grain', 'leiden_scVI', 'FACSgate', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', 'hb2_score', 'leiden', 'coarse_grain', 'sex'
    var: 'gene_name_scRNA-0', 'gene_id'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'age_colors', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'dendrogram_leiden', 'donor_colors', 'gender_colors', 'hvg', 'kit_10x_colors', 'leiden', 'leiden_colors', 'modality_colors', 'neighbors', 'phase_colors', 'rank_genes_groups', 'region_colors', 'sex_colors', 'umap'
    obsm: 'X_pca', 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
AnnData object with n_obs × n_vars = 20129 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'batch_key', '_scvi_batch', '_scvi_labels', 'previous_coarse_grain', 'previous_mid_grain', 'previous_fine_grain', 'leiden_scVI', 'FACSgate', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', 'hb2_score', 'leiden', 'coarse_grain', 'sex', 'Cardio_toremove', 'Mesenchymal_toremove'
    var: 'gene_name_scRNA-0', 'gene_id'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'dendrogram_leiden', 'donor_colors', 'gender_colors', 'hvg', 'kit_10x_colors', 'leiden', 'leiden_colors', 'modality_colors', 'neighbors', 'phase_colors', 'rank_genes_groups', 'region_colors', 'sex_colors', 'umap', 'log1p', 'age_colors', 'previous_fine_grain_colors', 'Cardio_toremove_colors', 'Mesenchymal_toremove_colors'
    obsm: 'X_pca', 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 34 (python):
```python
# preserve raw count
adata.layers["counts"] = adata.X.copy()

# normalise and log-transform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key='batch_key')

print(sum(adata.var.highly_variable))
sc.pl.highly_variable_genes(adata)

# freeze the state in `.raw`
adata.raw = adata

# subset hvgs
adata = adata[:, adata.var.highly_variable]

adata=adata.copy()

sc.tl.score_genes_cell_cycle(adata, s_genes=s_genes, g2m_genes=g2m_genes)
#sc.pp.regress_out(adata, ['S_score','G2M_score'])

#batch correction
scvi.model.SCVI.setup_anndata(adata, 
                              layer="counts", 
                              batch_key="batch_key",
                              categorical_covariate_keys=['phase'],
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo'])

vae = scvi.model.SCVI(adata, n_hidden = 128, n_latent = 50, n_layers = 3, dispersion = 'gene-batch')
vae.train(max_epochs=50)
    
# Convergence plot
x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
plt.plot(x, vae.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
plt.clf()

# add the latent space to adata
adata.obsm["X_scVI"] = vae.get_latent_representation()

sc.tl.score_genes(adata, stress_genes, score_name='stress_score')
sc.tl.score_genes(adata, hb_genes1, score_name='hb1_score')
sc.tl.score_genes(adata, hb_genes2, score_name='hb2_score')
sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata,min_dist=0.1)
#sc.tl.tsne(adata)

sc.pl.umap(adata, color=['log1p_total_counts','n_genes','n_genes_by_counts','total_counts','pct_counts_mt','pct_counts_ribo'],ncols=5)
sc.pl.umap(adata, color=['scrublet_score','stress_score', 'hb1_score','hb2_score','phase'], ncols=5,)
sc.pl.umap(adata, color=['donor','sex','age','region','phase','cell_or_nuclei','modality','kit_10x','FACSgate'],wspace=0.8,ncols=5)

Genes=['TTN','TNNT2','MYOM1','LDB3','ALDH1A2','TNNT1','KRT19','UPK3B','COL6A3','FBLN1','PCOLCE','LRRC17','EMCN','ECSCR','PECAM1','CDH5', 'LAPTM5','PTPRC','LCP1','CD53','PHOX2B','NRXN1','STMN2','GRIK2']
sc.pl.umap(adata, color=Genes)
```

Example 35 (python):
```python
normalizing counts per cell
    finished (0:00:00)
WARNING: adata.X seems to be already log-transformed.
extracting highly variable genes
    finished (0:00:09)
--> added
    'highly_variable', boolean vector (adata.var)
    'means', float vector (adata.var)
    'dispersions', float vector (adata.var)
    'dispersions_norm', float vector (adata.var)
2493
```

Example 36 (python):
```python
calculating cell cycle phase
computing score 'S_score'
WARNING: genes are not in var_names and ignored: ['FEN1', 'UNG', 'GINS2', 'MCM6', 'CDCA7', 'PRIM1', 'RFC2', 'RPA2', 'NASP', 'GMNN', 'SLBP', 'CCNE2', 'POLD3', 'MSH2', 'RAD51', 'CDC45', 'CDC6', 'EXO1', 'TIPIN', 'DSCC1', 'CASP8AP2', 'USP1', 'POLA1', 'CHAF1B', 'E2F8']
    finished: added
    'S_score', score of gene set (adata.obs).
    497 total control genes are used. (0:00:00)
computing score 'G2M_score'
WARNING: genes are not in var_names and ignored: ['TMPO', 'ANP32E', 'CDC25C', 'CDCA2', 'LBR', 'CKAP5', 'CTCF', 'G2E3', 'CBX5']
    finished: added
    'G2M_score', score of gene set (adata.obs).
    779 total control genes are used. (0:00:00)
-->     'phase', cell cycle phase (adata.obs)
```

Example 37 (unknown):
```unknown
GPU available: False, used: False
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
```

Example 38 (unknown):
```unknown
Epoch 50/50: 100%|██████| 50/50 [01:47<00:00,  2.14s/it, loss=1.13e+03, v_num=1]
```

Example 39 (python):
```python
computing score 'stress_score'
WARNING: genes are not in var_names and ignored: ['JUND', 'CEBPB', 'HSP90AA1', 'HSP90AB1', 'HSPA8', 'HSPB1', 'HSPE1', 'BTG1']
    finished: added
    'stress_score', score of gene set (adata.obs).
    345 total control genes are used. (0:00:00)
computing score 'hb1_score'
WARNING: genes are not in var_names and ignored: ['HBD', 'HBQ1']
    finished: added
    'hb1_score', score of gene set (adata.obs).
    344 total control genes are used. (0:00:00)
computing score 'hb2_score'
WARNING: genes are not in var_names and ignored: ['GYPA', 'RHAG', 'GFI1B', 'KEL', 'EPB42', 'ALAS2', 'TSPO2', 'GYPB', 'GYPE', 'SPTA1', 'RHCE', 'KLF1', 'RHD', 'SLC4A1', 'SNX22', 'SQSTM1', 'SOD1', 'FTH1', 'MAP1LC3B', 'SLC25A37', 'TRIM58', 'HMBS']
    finished: added
    'hb2_score', score of gene set (adata.obs).
    50 total control genes are used. (0:00:00)
computing neighbors
    finished: added to `.uns['neighbors']`
    `.obsp['distances']`, distances for each pair of neighbors
    `.obsp['connectivities']`, weighted adjacency matrix (0:00:01)
computing UMAP
    finished: added
    'X_umap', UMAP coordinates (adata.obsm) (0:00:05)
```

Example 40 (unknown):
```unknown
... storing 'phase' as categorical
```

Example 41 (unknown):
```unknown
<Figure size 800x800 with 0 Axes>
```

Example 42 (python):
```python
sc.pl.umap(adata, color='previous_fine_grain')
```

Example 43 (python):
```python
sc.pl.umap(adata, color='region')
```

Example 44 (python):
```python
TipGenes=['ADM',' ANKRD37',' C1QTNF6',' CLDN5',' COL4A1',' COL4A2',' COTL1',' DLL4',' EDNRB',' FSCN1',' GPIHBP1',' HSPG2',' IGFBP3',' INHBB',' JUP',' KCNE3',' KCNJ8',' LAMA4',' LAMB1',' LXN',' MARCKSL1',' MCAM',' MEST',' N4BP3',' NID2',' NOTCH4',' PLOD1',' PLXND1',' PMEPA1',' PTN',' RAMP3',' RBP1',' RGCC',' RHOC',' TRP53ILL',' UNC5B',' PRND',' LYH6']
sc.settings.set_figure_params(dpi=80, color_map='viridis', vector_friendly=False,  dpi_save=300, facecolor='white')
sc.tl.score_genes(adata, TipGenes, score_name='TipScore')
sc.pl.umap(adata, color='TipScore')
```

Example 45 (python):
```python
computing score 'TipScore'
WARNING: genes are not in var_names and ignored: [' ANKRD37', ' C1QTNF6', ' CLDN5', ' COL4A1', ' COL4A2', ' COTL1', ' DLL4', ' EDNRB', ' FSCN1', ' GPIHBP1', ' HSPG2', ' IGFBP3', ' INHBB', ' JUP', ' KCNE3', ' KCNJ8', ' LAMA4', ' LAMB1', ' LXN', ' MARCKSL1', ' MCAM', ' MEST', ' N4BP3', ' NID2', ' NOTCH4', ' PLOD1', ' PLXND1', ' PMEPA1', ' PTN', ' RAMP3', ' RBP1', ' RGCC', ' RHOC', ' TRP53ILL', ' UNC5B', ' PRND', ' LYH6']
    finished: added
    'TipScore', score of gene set (adata.obs).
    49 total control genes are used. (0:00:00)
```

Example 46 (python):
```python
sc.tl.leiden(adata, resolution=0.25, key_added='leiden')
sc.pl.umap(adata, color='leiden',legend_loc='on data')
```

Example 47 (python):
```python
running Leiden clustering
    finished: found 9 clusters and added
    'leiden', the cluster labels (adata.obs, categorical) (0:00:00)
```

Example 48 (python):
```python
#adata.write('./Objects/Cardiomyocytes.h5ad')
##adata=sc.read('./Objects/Cardiomyocytes.h5ad')
```

Example 49 (python):
```python
sc.tl.rank_genes_groups(adata, groupby='leiden', method='t-test')
sc.tl.dendrogram(adata, groupby='leiden')
sc.pl.dendrogram(adata,groupby='leiden')
sc.pl.rank_genes_groups_matrixplot(adata, groupby='leiden', standard_scale='var')
```

Example 50 (python):
```python
ranking genes
    finished: added to `.uns['rank_genes_groups']`
    'names', sorted np.recarray to be indexed by group ids
    'scores', sorted np.recarray to be indexed by group ids
    'logfoldchanges', sorted np.recarray to be indexed by group ids
    'pvals', sorted np.recarray to be indexed by group ids
    'pvals_adj', sorted np.recarray to be indexed by group ids (0:00:02)
    using 'X_pca' with n_pcs = 50
Storing dendrogram info using `.uns['dendrogram_leiden']`
```

Example 51 (python):
```python
#Dive into great and coronary vessel endothelial cells
adata=adata.copy()
ClustersToStay=['2','5','0','8','7']
adata=adata[[x for x in adata.obs_names if adata.obs.leiden[x] in ClustersToStay],:].copy()
adata

adata_raw=sc.read('./Objects/20221130RNA_foetal_post-scVI_raw_ExtracardiacRemoved.h5ad')

print(adata_raw)
# constructing new anndata with maintaining .obs, .obsm, .obsp, .uns
# overwriting adata_raw
adata_raw=anndata.AnnData(X=adata_raw[adata.obs_names,:].X,
                      obs=adata.obs, # from annotated or subsetted data
                      var=adata_raw.var,
                      obsm=adata.obsm, # from annotated or subsetted data
                      obsp=adata.obsp, # from annotated or subsetted data
                    uns=adata.uns # from annotated or subsetted data
                               )

adata=adata_raw.copy()
print(adata)

# preserve raw count
adata.layers["counts"] = adata.X.copy()

# normalise and log-transform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key='batch_key')

print(sum(adata.var.highly_variable))
sc.pl.highly_variable_genes(adata)

# freeze the state in `.raw`
adata.raw = adata

# subset hvgs
adata = adata[:, adata.var.highly_variable]

adata=adata.copy()

sc.tl.score_genes_cell_cycle(adata, s_genes=s_genes, g2m_genes=g2m_genes)
#sc.pp.regress_out(adata, ['S_score','G2M_score'])

#batch correction
scvi.model.SCVI.setup_anndata(adata, 
                              layer="counts", 
                              batch_key="batch_key",
                              categorical_covariate_keys=['phase'],
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo'])

vae = scvi.model.SCVI(adata, n_hidden = 128, n_latent = 50, n_layers = 3, dispersion = 'gene-batch')
vae.train(max_epochs=50)
    
# Convergence plot
x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
plt.plot(x, vae.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
plt.clf()

# add the latent space to adata
adata.obsm["X_scVI"] = vae.get_latent_representation()

sc.tl.score_genes(adata, stress_genes, score_name='stress_score')
sc.tl.score_genes(adata, hb_genes1, score_name='hb1_score')
#sc.tl.score_genes(adata, hb_genes2, score_name='hb2_score')
sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata,min_dist=0.1)
#sc.tl.tsne(adata)

sc.pl.umap(adata, color=['log1p_total_counts','n_genes','n_genes_by_counts','total_counts','pct_counts_mt','pct_counts_ribo'],ncols=5)
sc.pl.umap(adata, color=['scrublet_score','stress_score', 'hb1_score','phase'], ncols=5,)
sc.pl.umap(adata, color=['donor','sex','age','region','phase','cell_or_nuclei','modality','kit_10x','FACSgate'],wspace=0.8,ncols=5)

Genes=['TTN','TNNT2','MYOM1','LDB3','ALDH1A2','TNNT1','KRT19','UPK3B','COL6A3','FBLN1','PCOLCE','LRRC17','EMCN','ECSCR','PECAM1','CDH5', 'LAPTM5','PTPRC','LCP1','CD53','PHOX2B','NRXN1','STMN2','GRIK2']
sc.pl.umap(adata, color=Genes)
```

Example 52 (python):
```python
computing neighbors
    finished: added to `.uns['neighbors']`
    `.obsp['distances']`, distances for each pair of neighbors
    `.obsp['connectivities']`, weighted adjacency matrix (0:00:01)
computing UMAP
    finished: added
    'X_umap', UMAP coordinates (adata.obsm) (0:00:02)
```

Example 53 (unknown):
```unknown
... storing 'phase' as categorical
```

Example 54 (python):
```python
sc.pl.umap(adata, color=['region'])
```

Example 55 (python):
```python
TipGenes=['ADM',' ANKRD37',' C1QTNF6',' CLDN5',' COL4A1',' COL4A2',' COTL1',' DLL4',' EDNRB',' FSCN1',' GPIHBP1',' HSPG2',' IGFBP3',' INHBB',' JUP',' KCNE3',' KCNJ8',' LAMA4',' LAMB1',' LXN',' MARCKSL1',' MCAM',' MEST',' N4BP3',' NID2',' NOTCH4',' PLOD1',' PLXND1',' PMEPA1',' PTN',' RAMP3',' RBP1',' RGCC',' RHOC',' TRP53ILL',' UNC5B',' PRND',' LYH6']
sc.settings.set_figure_params(dpi=80, color_map='viridis', vector_friendly=False,  dpi_save=300, facecolor='white')
sc.tl.score_genes(adata, TipGenes, score_name='TipScore')
sc.pl.umap(adata, color='TipScore')
```

Example 56 (python):
```python
computing score 'TipScore'
WARNING: genes are not in var_names and ignored: [' ANKRD37', ' C1QTNF6', ' CLDN5', ' COL4A1', ' COL4A2', ' COTL1', ' DLL4', ' EDNRB', ' FSCN1', ' GPIHBP1', ' HSPG2', ' IGFBP3', ' INHBB', ' JUP', ' KCNE3', ' KCNJ8', ' LAMA4', ' LAMB1', ' LXN', ' MARCKSL1', ' MCAM', ' MEST', ' N4BP3', ' NID2', ' NOTCH4', ' PLOD1', ' PLXND1', ' PMEPA1', ' PTN', ' RAMP3', ' RBP1', ' RGCC', ' RHOC', ' TRP53ILL', ' UNC5B', ' PRND', ' LYH6']
    finished: added
    'TipScore', score of gene set (adata.obs).
    49 total control genes are used. (0:00:00)
```

Example 57 (python):
```python
sc.tl.leiden(adata, resolution=0.45, key_added='leiden')
sc.pl.umap(adata, color='leiden',legend_loc='on data')
sc.tl.rank_genes_groups(adata, groupby='leiden', method='t-test')
sc.tl.dendrogram(adata, groupby='leiden')
sc.pl.dendrogram(adata,groupby='leiden')
sc.pl.rank_genes_groups_matrixplot(adata, groupby='leiden', standard_scale='var')
```

Example 58 (python):
```python
running Leiden clustering
    finished: found 10 clusters and added
    'leiden', the cluster labels (adata.obs, categorical) (0:00:00)
```

Example 59 (python):
```python
ranking genes
    finished: added to `.uns['rank_genes_groups']`
    'names', sorted np.recarray to be indexed by group ids
    'scores', sorted np.recarray to be indexed by group ids
    'logfoldchanges', sorted np.recarray to be indexed by group ids
    'pvals', sorted np.recarray to be indexed by group ids
    'pvals_adj', sorted np.recarray to be indexed by group ids (0:00:01)
    using 'X_pca' with n_pcs = 50
Storing dendrogram info using `.uns['dendrogram_leiden']`
```

Example 60 (python):
```python
adata.obs['fine_grain']= '100'

GreatVesselVenousEndothelialCells=['9']
GreatVesselArterialEndothelialCells=['0','6']
CoronaryCapillaryEndothelialCells=['8','2','5','1','4']
CoronaryVenousEndothelialCells=['7']
CoronaryArterialEndothelialCells=['3']

adata.obs['fine_grain'][[x for x in adata.obs_names if adata.obs.leiden[x] in GreatVesselVenousEndothelialCells]]='GreatVesselVenousEndothelialCells'
adata.obs['fine_grain'][[x for x in adata.obs_names if adata.obs.leiden[x] in GreatVesselArterialEndothelialCells]]='GreatVesselArterialEndothelialCells'
adata.obs['fine_grain'][[x for x in adata.obs_names if adata.obs.leiden[x] in CoronaryCapillaryEndothelialCells]]='CoronaryCapillaryEndothelialCells'
adata.obs['fine_grain'][[x for x in adata.obs_names if adata.obs.leiden[x] in CoronaryVenousEndothelialCells]]='CoronaryVenousEndothelialCells'
adata.obs['fine_grain'][[x for x in adata.obs_names if adata.obs.leiden[x] in CoronaryArterialEndothelialCells]]='CoronaryArterialEndothelialCells'

sc.pl.umap(adata, color='fine_grain')
```

Example 61 (unknown):
```unknown
... storing 'fine_grain' as categorical
```

Example 62 (python):
```python
#adata.write('./Objects/Annotated/EndotheliumCoronaryAndGreatVessels.h5ad')
##adata=sc.read('./Objects/Cardiomyocytes.h5ad')
```

Example 63 (python):
```python
adata.obs['fine_grain']= '100'

LymphaticEndothelialCells=['3']
EndocardialCells=['1']
ValveEndothelialCells=['6']
EndocardialCushionCells=['4']

adata.obs['fine_grain'][[x for x in adata.obs_names if adata.obs.leiden[x] in LymphaticEndothelialCells]]='LymphaticEndothelialCells'
adata.obs['fine_grain'][[x for x in adata.obs_names if adata.obs.leiden[x] in EndocardialCells]]='EndocardialCells'
adata.obs['fine_grain'][[x for x in adata.obs_names if adata.obs.leiden[x] in ValveEndothelialCells]]='ValveEndothelialCells'
adata.obs['fine_grain'][[x for x in adata.obs_names if adata.obs.leiden[x] in EndocardialCushionCells]]='EndocardialCushionCells'

sc.pl.umap(adata, color='fine_grain')
```

Example 64 (unknown):
```unknown
... storing 'fine_grain' as categorical
```

Example 65 (python):
```python
bdata=sc.read('./Objects/Annotated/EndotheliumCoronaryAndGreatVessels.h5ad')
set(bdata.obs.fine_grain)
```

Example 66 (unknown):
```unknown
{'CoronaryArterialEndothelialCells',
 'CoronaryCapillaryEndothelialCells',
 'CoronaryVenousEndothelialCells',
 'GreatVesselArterialEndothelialCells',
 'GreatVesselVenousEndothelialCells'}
```

Example 67 (python):
```python
set(adata.obs.fine_grain)
```

Example 68 (unknown):
```unknown
{'100',
 'EndocardialCells',
 'EndocardialCushionCells',
 'LymphaticEndothelialCells',
 'ValveEndothelialCells'}
```

Example 69 (python):
```python
adata.obs['fine_grain']= '0'

for k in range(1, len(adata)+1):
    if adata.obs_names[k-1] in CoronaryArterialEndothelialCells:
        adata.obs.fine_grain[k-1]='CoronaryArterialEndothelialCells'   
    if adata.obs_names[k-1] in CoronaryCapillaryEndothelialCells:
        adata.obs.fine_grain[k-1]='CoronaryCapillaryEndothelialCells' 
    if adata.obs_names[k-1] in CoronaryVenousEndothelialCells:
        adata.obs.fine_grain[k-1]='CoronaryVenousEndothelialCells' 
    if adata.obs_names[k-1] in GreatVesselArterialEndothelialCells:
        adata.obs.fine_grain[k-1]='GreatVesselArterialEndothelialCells' 
    if adata.obs_names[k-1] in GreatVesselVenousEndothelialCells:
        adata.obs.fine_grain[k-1]='GreatVesselVenousEndothelialCells'   
    if adata.obs_names[k-1] in EndocardialCells:
        adata.obs.fine_grain[k-1]='EndocardialCells' 
    if adata.obs_names[k-1] in EndocardialCushionCells:
        adata.obs.fine_grain[k-1]='EndocardialCushionCells' 
    if adata.obs_names[k-1] in LymphaticEndothelialCells:
        adata.obs.fine_grain[k-1]='LymphaticEndothelialCells' 
    if adata.obs_names[k-1] in ValveEndothelialCells:
        adata.obs.fine_grain[k-1]='ValveEndothelialCells'         

sc.pl.umap(adata, color='fine_grain')
```

Example 70 (unknown):
```unknown
... storing 'fine_grain' as categorical
```

Example 71 (python):
```python
adata.write('./Objects/Annotated/Endothelium.h5ad')
##adata=sc.read('./Objects/Cardiomyocytes.h5ad')
```

Example 72 (python):
```python
adata=sc.read('./Objects/Annotated/Endothelium.h5ad')
```

Example 73 (python):
```python
set(adata.obs.fine_grain)
```

Example 74 (unknown):
```unknown
{'CoronaryArterialEndothelialCells',
 'CoronaryCapillaryEndothelialCells',
 'CoronaryVenousEndothelialCells',
 'EndocardialCells',
 'EndocardialCushionCells',
 'GreatVesselArterialEndothelialCells',
 'GreatVesselVenousEndothelialCells',
 'LymphaticEndothelialCells',
 'ValveEndothelialCells'}
```

Example 75 (python):
```python
sc.pl.umap(adata, color='BMP4')
```

Example 76 (python):
```python
bdata=adata[adata.obs.fine_grain=='ValveEndothelialCells',:].copy()
bdatanames=bdata.obs_names

np.savetxt('ValveEndothelialCells.txt', bdatanames, delimiter="\n", fmt="%s")
bdata
```

Example 77 (unknown):
```unknown
AnnData object with n_obs × n_vars = 1042 × 2493
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'batch_key', '_scvi_batch', '_scvi_labels', 'previous_coarse_grain', 'previous_mid_grain', 'previous_fine_grain', 'leiden_scVI', 'FACSgate', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', 'hb2_score', 'leiden', 'coarse_grain', 'sex', 'Cardio_toremove', 'Mesenchymal_toremove', 'TipScore', 'fine_grain'
    var: 'gene_name_scRNA-0', 'gene_id', 'highly_variable', 'means', 'dispersions', 'dispersions_norm', 'highly_variable_nbatches', 'highly_variable_intersection'
    uns: 'Cardio_toremove_colors', 'FACSgate_colors', 'Mesenchymal_toremove_colors', '_scvi_manager_uuid', '_scvi_uuid', 'age_colors', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'dendrogram_leiden', 'donor_colors', 'fine_grain_colors', 'gender_colors', 'hvg', 'kit_10x_colors', 'leiden', 'leiden_colors', 'modality_colors', 'neighbors', 'phase_colors', 'previous_fine_grain_colors', 'rank_genes_groups', 'region_colors', 'sex_colors', 'umap'
    obsm: 'X_pca', 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    layers: 'counts'
    obsp: 'connectivities', 'distances'
```

---

## Variables¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/VisiumHD/no7_manual-annotation_BloodVesselEC.html

**Contents:**
- Variables¶
- Read in data¶
- Remove mid-grain markers from the feature¶
- PCA, Clustering, UMAP¶
- Markers¶
- Save¶

2512 rows × 7 columns

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os

import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 

import celltypist
```

Example 2 (python):
```python
# import session_info
# session_info.show()
```

Example 3 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 4 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 5 (python):
```python
adata_dir = '/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/subsets'
sample_id = 'HEA_FOET14880396'
celltype = 'BloodVesselEndothelialCells'
```

Example 6 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk/Aug2024'
table_dir = '/nfs/team205/heart/fetal_ms_figures/tables/Aug2024'
```

Example 7 (python):
```python
# read in
adata = sc.read_h5ad(f'{adata_dir}/{sample_id}_{celltype}_b2c_cells_filtered_lognorm.h5ad')
adata
```

Example 8 (unknown):
```unknown
AnnData object with n_obs × n_vars = 1765 × 15377
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'leiden_0.2', 'leiden_0.4', 'leiden_0.6', 'leiden_1', 'leiden_1.2', 'leiden_1.5', 'leiden_2'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1.2', 'leiden_1.2_colors', 'leiden_1.5', 'leiden_1.5_colors', 'leiden_1_colors', 'leiden_2', 'leiden_2_colors', 'log1p', 'neighbors', 'pca', 'spatial', 'umap'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 9 (python):
```python
# read in
midgrain_markers = pd.read_csv('/nfs/team205/kk18/notebooks/Foetal/VisiumHD/re-seq_mpp0p3/sc_midgrain_markers.csv',
                               index_col=0
                              )
# remove the markers of celltype of interest
midgrain_markers = midgrain_markers[midgrain_markers['group']!=celltype]
midgrain_markers
```

Example 10 (unknown):
```unknown
'GATA2' in list(midgrain_markers['names'])
```

Example 11 (python):
```python
# remove
mask = [x not in list(midgrain_markers['names']) for x in adata.var_names]
bdata = adata[:,mask]
bdata
```

Example 12 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 1765 × 14107
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'leiden_0.2', 'leiden_0.4', 'leiden_0.6', 'leiden_1', 'leiden_1.2', 'leiden_1.5', 'leiden_2'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1.2', 'leiden_1.2_colors', 'leiden_1.5', 'leiden_1.5_colors', 'leiden_1_colors', 'leiden_2', 'leiden_2_colors', 'log1p', 'neighbors', 'pca', 'spatial', 'umap'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 13 (python):
```python
# HVGs, using the object which doesn't have midgrain markers
sc.pp.highly_variable_genes(bdata, flavor='seurat', min_mean=0.0125, max_mean=3, min_disp=0.5)
print(f'number of hvgs: {sum(bdata.var.highly_variable)}')
# PCA using the degected HVGs
sc.tl.pca(bdata, use_highly_variable=True)
adata.obsm['X_pca'] = bdata.obsm['X_pca'].copy()
# create neighbour graph
sc.pp.neighbors(adata,n_neighbors=15,use_rep='X_pca')
# visualisation with umap
sc.tl.umap(adata,min_dist=0.5)
sc.pl.embedding(adata,basis='X_umap',color=['bin_count','total_counts','n_genes_by_counts',"conf_score_midmod2fine","celltypist_midmod2fine",],
                wspace=0.2,cmap='RdPu',vmax='p100',ncols=5)
# leiden clustering
resolutions_list = [0.2,0.4,0.6,1,1.2,1.5,2]
for resolution in resolutions_list:
    sc.tl.leiden(adata,resolution=resolution,key_added=f'leiden_{str(resolution)}',n_iterations=2)
sc.pl.embedding(adata,basis='X_umap',
                color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                wspace=0.3)
```

Example 14 (unknown):
```unknown
number of hvgs: 3441
```

Example 15 (python):
```python
'HEY2' in adata.var_names
```

Example 16 (python):
```python
# read in markers identified using single-nucleus data
markers_df = pd.read_csv(f'/nfs/team205/kk18/notebooks/Foetal/VisiumHD/re-seq_mpp0p3/sc_{celltype}_finegrain_markers.csv')

mask1 = markers_df['group_mean']>1
mask2 = markers_df['logfoldchanges']>1
markers_df = markers_df[mask1&mask2]

markers_df
```

Example 17 (python):
```python
markers = {}
n_top = 10
for c in set(markers_df['group']):
    markers_df_sub = markers_df[markers_df['group']==c]
    markers_df_sub.sort_values('logfoldchanges',ascending=False)
    markers[c]=list(markers_df_sub[:n_top]['names'])

# manually add genes
markers['Cycling'] = ['MKI67','TOP2A'] 

genes = []
for g in markers.values():
    genes = genes + [x for x in g if (x in list(adata.var_names))&(x not in genes)]
for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in list(adata.var_names)]
len(genes)
```

Example 18 (python):
```python
sc.pl.embedding(adata,basis='X_umap',color=genes, size=10,
               ncols=6, wspace=0.2,cmap='RdPu',vmax='p99.7')
```

Example 19 (unknown):
```unknown
resolution_sel = 1
```

Example 20 (python):
```python
markers_others = {
    'CM': ['TTN', 'TNNT2','MYH6',
        'MYH7', 'MYL2', 'FHL2', # vCN
            'NPPA', 'MYL7', 'MYL4'], #aCM
    'FB': ['DCN', 'GCN', 'PDGFRA','COL1A1','COL1A2'],
    'EC': ['VWF', 'PECAM1', 'CDH5','RGCC', 'FABP5',],
    'Peri': ['RGS5', 'ABCC9', 'KCNJ8'],
    'SMC': ['MYH11', 'TAGLN', 'ACTA2'],
    'Meso': ['MSLN', 'WT1', 'BNC1'],
    'Neuro': ['PLP1', 'NRXN1', 'NRXN3','PRPH', 'NEFL', 'NEFM', 'NEFH', 'STMN2', 'SST', 'SLC10A4', 'SLC18A3', 'TH', 'NPY'],
    'Adipo': ['GPAM', 'FASN', 'LEP'],
    'Myelo': ['CD14', 'C1QA', 'CD68','LYVE1','TIMD4'],
    'Lymph': ['CD8A', 'IL7R', 'CD40LG'],
    'B': ['CD19','SDC1','IGHG1','IGHA1'],
    'Mast': ['KIT', 'CPA3'],
}

for key in markers_others.keys():
    markers_others[key]=[x for x in markers_others[key] if x in adata.var_names]

sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
sc.pl.dotplot(adata,
              markers_others, 
              groupby=f'leiden_{str(resolution_sel)}',
              dendrogram=True,
              standard_scale="var",
              color_map="Reds",
              swap_axes=False,
             )
```

Example 21 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: CM, FB, EC, etc.
```

Example 22 (python):
```python
sc.tl.dendrogram(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.pl.dotplot(adata,
              var_names=markers,
              groupby=f'leiden_{str(resolution_sel)}',
              standard_scale='var',
              dendrogram=True
             )
```

Example 23 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: CoronaryVenousEndothelialCells, GreatVesselArterialEndothelialCells, CoronaryArterialEndothelialCells, etc.
```

Example 24 (python):
```python
# DEG analysis
print(adata.X.data[:5])
adata.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata, dendrogram=True, standard_scale='var', return_fig=True)
dp.add_totals().show()
```

Example 25 (unknown):
```unknown
[1.27693184 1.26688208 1.1734066  2.12904094 1.26688208]
```

Example 26 (python):
```python
# DEG analysis
print(adata.X.data[:5])
adata.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata, dendrogram=True, standard_scale='var', return_fig=True, n_genes=5)
dp.add_totals().show()
```

Example 27 (unknown):
```unknown
[1.27693184 1.26688208 1.1734066  2.12904094 1.26688208]
```

Example 28 (python):
```python
sc.pl.embedding(adata,basis='X_umap',
                color=f'leiden_{str(resolution_sel)}',
                wspace=0.3, legend_loc='on data')
```

Example 29 (python):
```python
sc.pl.embedding(adata,basis='X_umap',
                color=['celltypist_midmod2fine', 'conf_score_midmod2fine'],
                wspace=0.7)
```

Example 30 (python):
```python
# umap per celltype to highlight
group_col = 'celltypist_midmod2fine'
groups_to_plot = list(set(adata.obs[group_col]))

plt.rcParams['axes.grid'] = False
fig,axs=plt.subplots(nrows=2,ncols=4,figsize=(15,7))
# Flatten the axs array for easier iteration in case of multiple rows and columns
axs_flat = axs.flatten()

for i,grp in enumerate(groups_to_plot):
    sc.pl.umap(adata, color=[group_col], groups=grp, 
               size=30, frameon=False,
               show=False, ax=axs_flat[i], legend_loc=None, legend_fontsize=5,
               title=grp
              )
# Turn off any remaining subplots
for j in range(len(groups_to_plot), len(axs_flat)):
    fig.delaxes(axs_flat[j])

# fig.suptitle('Main Title for the Figure')
fig.tight_layout()
```

Example 31 (python):
```python
sc.pl.violin(adata,keys=['total_counts','bin_count'],groupby=f'leiden_{str(resolution_sel)}')
```

Example 32 (unknown):
```unknown
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 33 (python):
```python
# annotate
adata.obs['fine_grain_manual'] = adata.obs[f'leiden_{str(resolution_sel)}'].copy()
adata.obs.replace({'fine_grain_manual':{
    '0':'GreatVesselVenousEndothelialCells',
    '1':'CoronaryCapillaryEndothelialCells',
    '2':'CoronaryCapillaryEndothelialCells', # contamination from CMs
    '3':'CoronaryCapillaryEndothelialCells',
    '4':'GreatVesselArterialEndothelialCells',
    '5':'CoronaryArterialEndothelialCells',
    '6':'GreatVesselArterialEndothelialCells',
    '7':'CoronaryVenousEndothelialCells',
    '8':'CoronaryArterialEndothelialCells',
    '9':'CoronaryCapillaryEndothelialCells', # contamination from erythrocytes
}},inplace=True)
adata.obs['fine_grain_manual'].value_counts()
```

Example 34 (unknown):
```unknown
fine_grain_manual
CoronaryCapillaryEndothelialCells      848
GreatVesselVenousEndothelialCells      313
GreatVesselArterialEndothelialCells    308
CoronaryArterialEndothelialCells       219
CoronaryVenousEndothelialCells          77
Name: count, dtype: int64
```

Example 35 (python):
```python
ad = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/Feb28ObjectRaw_finegrain_updated.h5ad',backed='r')
ad = ad[ad.obs['mid_grain']==celltype]
set(ad.obs['fine_grain'])
```

Example 36 (unknown):
```unknown
{'CoronaryArterialEndothelialCells',
 'CoronaryCapillaryEndothelialCells',
 'CoronaryVenousEndothelialCells',
 'GreatVesselArterialEndothelialCells',
 'GreatVesselVenousEndothelialCells'}
```

Example 37 (python):
```python
sc.pl.embedding(adata,basis='X_umap',
                color='fine_grain_manual',
                frameon=False,
                wspace=0.3)
```

Example 38 (python):
```python
sc.tl.dendrogram(adata,groupby='fine_grain_manual')
sc.pl.dotplot(adata,
              var_names=markers,
              groupby='fine_grain_manual',
              standard_scale='var',
              dendrogram=True
             )
```

Example 39 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: GreatVesselVenousEndothelialCells, CoronaryCapillaryEndothelialCells, GreatVesselArterialEndothelialCells, etc.
var_group_labels: CoronaryVenousEndothelialCells, GreatVesselArterialEndothelialCells, CoronaryArterialEndothelialCells, etc.
```

Example 40 (python):
```python
# save
adata.write(f'{adata_dir}/{sample_id}_{celltype}_b2c_cells_filtered_lognorm.h5ad')
print(f'{adata_dir}/{sample_id}_{celltype}_b2c_cells_filtered_lognorm.h5ad')
```

Example 41 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/subsets/HEA_FOET14880396_BloodVesselEndothelialCells_b2c_cells_filtered_lognorm.h5ad
```

---

## Loading packages¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/Visium/Others/plotting_multiple_LRaCMs.html

**Contents:**
- Loading packages¶
- Function¶
- Read in adata¶
- Plotting¶

**Examples:**

Example 1 (python):
```python
# this is for the jupyter notebook at JupyterLab
# to make sys.path between the terminal and Jupyter same
import sys

print(sys.path) # especially the path to site-packages are different... so thats why couldn't find correct package...

# remove weird paths...
sys.path = sys.path[:4]+[sys.path[6]]
print(sys.path)
```

Example 2 (unknown):
```unknown
['/nfs/team205/kk18/notebooks/P48_Foetal-heart_Visium/notebooks/notebooks_for_manuscript', '/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python39.zip', '/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python3.9', '/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python3.9/lib-dynload', '', '/nfs/users/nfs_k/kk18/.local/lib/python3.9/site-packages', '/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python3.9/site-packages']
['/nfs/team205/kk18/notebooks/P48_Foetal-heart_Visium/notebooks/notebooks_for_manuscript', '/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python39.zip', '/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python3.9', '/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python3.9/lib-dynload', '/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python3.9/site-packages']
```

Example 3 (python):
```python
import sys
import scanpy as sc
import anndata
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt 
import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 

import seaborn as sns

from cell2location.plt import plot_spatial

import os
```

Example 4 (python):
```python
/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python3.9/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm
Global seed set to 0
/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python3.9/site-packages/umap/distances.py:1063: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python3.9/site-packages/umap/distances.py:1071: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python3.9/site-packages/umap/distances.py:1086: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python3.9/site-packages/umap/umap_.py:660: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
```

Example 5 (python):
```python
import session_info
session_info.show()
```

Example 6 (python):
```python
-----
anndata             0.9.1
cell2location       0.1.3
matplotlib          3.7.1
numpy               1.24.3
pandas              2.0.1
scanpy              1.9.3
seaborn             0.12.2
session_info        1.0.0
-----
```

Example 7 (python):
```python
PIL                 9.5.0
absl                NA
asttokens           NA
attr                23.1.0
backcall            0.2.0
certifi             2023.05.07
cffi                1.15.1
charset_normalizer  3.1.0
chex                0.1.7
comm                0.1.3
contextlib2         NA
cycler              0.10.0
cython_runtime      NA
dateutil            2.8.2
debugpy             1.6.7
decorator           5.1.1
defusedxml          0.7.1
docrep              0.3.2
etils               1.2.0
executing           1.2.0
flax                0.6.10
fsspec              2023.5.0
google              NA
h5py                3.8.0
idna                3.4
igraph              0.10.4
importlib_metadata  NA
importlib_resources NA
ipykernel           6.23.0
ipython_genutils    0.2.0
jax                 0.4.9
jaxlib              0.4.9
jedi                0.18.2
joblib              1.2.0
kiwisolver          1.4.4
leidenalg           0.9.1
lightning_fabric    1.9.5
lightning_utilities 0.8.0
llvmlite            0.40.0
ml_collections      NA
ml_dtypes           0.1.0
mpl_toolkits        NA
mpmath              1.3.0
msgpack             1.0.5
mudata              0.2.2
multipledispatch    0.6.0
natsort             8.3.1
numba               0.57.0
numpyro             0.11.0
nvfuser             NA
opt_einsum          v3.3.0
optax               0.1.5
packaging           23.1
parso               0.8.3
patsy               0.5.3
pexpect             4.8.0
pickleshare         0.7.5
pkg_resources       NA
platformdirs        3.5.0
prompt_toolkit      3.0.38
psutil              5.9.5
ptyprocess          0.7.0
pure_eval           0.2.2
pycparser           2.21
pydev_ipython       NA
pydevconsole        NA
pydevd              2.9.5
pydevd_file_utils   NA
pydevd_plugins      NA
pydevd_tracing      NA
pygments            2.15.1
pynndescent         0.5.10
pyparsing           3.0.9
pyro                1.8.4
pytorch_lightning   1.9.5
pytz                2023.3
requests            2.30.0
rich                NA
scipy               1.10.1
scvi                0.20.3
setuptools          66.0.0
six                 1.16.0
sklearn             1.2.2
stack_data          0.6.2
statsmodels         0.14.0
sympy               1.12
texttable           1.6.7
threadpoolctl       3.1.0
toolz               0.12.0
torch               2.0.1+cu117
torchmetrics        0.11.4
tornado             6.3.1
tqdm                4.65.0
traitlets           5.9.0
tree                0.1.8
typing_extensions   NA
umap                0.5.3
urllib3             2.0.2
wcwidth             0.2.6
yaml                6.0
zipp                NA
zmq                 25.0.2
zoneinfo            NA
```

Example 8 (unknown):
```unknown
-----
IPython             8.13.2
jupyter_client      8.2.0
jupyter_core        5.3.0
notebook            6.5.4
-----
Python 3.9.16 (main, Mar  8 2023, 14:00:05) [GCC 11.2.0]
Linux-5.4.0-169-generic-x86_64-with-glibc2.27
-----
Session information updated at 2024-02-12 20:45
```

Example 9 (python):
```python
sc.settings.set_figure_params(dpi=80,dpi_save=300)
```

Example 10 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots'
```

Example 11 (python):
```python
# This function selects the data for multiple slides from the spatial anndata object.
# Modified `select_slide` in cell2loc for multiple slides

# @param adata: anndata object with multiple spatial experiments
# @param s: list of selected slides
# @param s_col: column in adata.obs listing experiment name for each location

def select_multislide(adata, s, s_col='sample'):

    slides = adata[adata.obs[s_col].isin(s), :]
    s_keys = list(slides.uns['spatial'].keys())
    
    uns_dict = {}
    for ss in s:
        s_spatial=np.array(s_keys)[[ss in k for k in s_keys]][0]
        uns_dict[s_spatial]=slides.uns['spatial'][s_spatial]
    
    slides.uns['spatial'] = uns_dict

    return slides
```

Example 12 (python):
```python
adata_vis = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/visium-FFPE_foetal_raw.h5ad')

# add abundance to .obs
df = adata_vis.obsm['q05_cell_abundance_w_sf'].copy()
## rename columns
df.columns = [x.replace('q05cell_abundance_w_sf_','') for x in df.columns]
adata_vis.obs[df.columns]=df.copy()

# set symboßl as var_names
adata_vis.var.set_index('SYMBOL',inplace=True)

adata_vis
```

Example 13 (unknown):
```unknown
AnnData object with n_obs × n_vars = 19973 × 18085
    obs: 'in_tissue', 'array_row', 'array_col', 'sample', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'mt_frac', 'sangerID', 'Publication', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'status', 'JC_edit', 'AtrialCardiomyocytesLeft', 'AtrialCardiomyocytesRight', 'VentricularCardiomyocytesLeftCompact', 'VentricularCardiomyocytesRightCompact', 'VentricularCardiomyocytesLeftTrabeculated', 'VentricularCardiomyocytesRightTrabeculated', 'SinoatrialNodeCardiomyocytes', 'AtrioventricularNodeCardiomyocytes', 'VentricularConductionSystemProximal', 'VentricularConductionSystemDistal', 'GreatVesselAdventitialFibroblasts', 'CoronaryVesselAdventitialFibroblasts', 'MyocardialInterstitialFibroblasts1', 'MyocardialInterstitialFibroblasts2', 'Myofibroblasts', 'LymphNodeFibroblasticReticularCells', 'ValveInterstitialCells', 'GreatVesselSmoothMuscleCells', 'CoronarySmoothMuscleCells', 'CoronaryPericytes', 'GreatVesselArterialEndothelialCells', 'GreatVesselVenousEndothelialCells', 'CoronaryArterialEndothelialCells', 'CoronaryVenousEndothelialCells', 'CoronaryCapillaryEndothelialCells', 'EndocardialCells', 'EndocardialCushionCells', 'ValveEndothelialCells', 'LymphaticEndothelialCells', 'MesothelialEpicardialCells', 'EpicardiumDerivedCells', 'NeuronPrecursors', 'ChromaffinCells', 'SympatheticNeurons', 'ParaympatheticNeurons', 'SchwannCellPrecursors', 'SchwannCells', 'MonocytesMPOpos', 'Monocytes', 'MonocyteDerivedCells', 'MacrophagesCX3CR1pos', 'MacrophagesTIMD4pos', 'MacrophagesLYVE1pos', 'MacrophagesATF3pos', 'DendriticCellsType1', 'DendriticCellsMature', 'PlasmacytoidDendriticCells', 'MastCells', 'Megakaryocytes', 'TCellsCD4pos', 'TCellsCD8pos', 'TregsCD4pos', 'ProBCells', 'BCells', 'BCellsMS4A1pos', 'NaturalKillerCells', 'InnateLymphoidCells'
    var: 'gene_ids', 'feature_types', 'genome', 'mt'
    uns: 'spatial'
    obsm: 'MT', 'means_cell_abundance_w_sf', 'q05_cell_abundance_w_sf', 'q95_cell_abundance_w_sf', 'spatial', 'stds_cell_abundance_w_sf'
```

Example 14 (python):
```python
# select up to 6 clusters
# https://github.com/BayraktarLab/cell2location/blob/bd2aae833f81d8ab68684aa44f863b36e3af0aed/cell2location/plt/plot_spatial.py#L108

slide_id = 'BHF_F_Hea13295830'

slide = select_multislide(adata_vis, [slide_id])

clust_labels = ['AtrialCardiomyocytesLeft',
                'AtrialCardiomyocytesRight'
                ]
clust_col = ['' + str(i) for i in clust_labels] # in case column names differ from labels

plt.rcParams['axes.grid'] = False
with mpl.rc_context({'figure.figsize': (15, 15)}):
    fig = plot_spatial(
        adata=slide,
        # labels to show on a plot
        color=clust_col, labels=clust_labels,
        show_img=True, img_alpha=0.5,
        # 'fast' (white background) or 'dark_background'
        style='fast',
        # limit color scale at 99.2% quantile of cell abundance
        max_color_quantile=0.997,
        # size of locations (adjust depending on figure size)
        circle_diameter=3,
        colorbar_position='bottom',
        colorbar_shape = {'vertical_gaps': 0.1, 'horizontal_gaps': 5,
                                    'width': 1, 'height': 0.02},
        # crop_x=(650,820),
        # crop_y=(1200,1420),
        reorder_cmap = [1,4] # 'yellow' 'orange' 'blue' 'green' 'purple' 'grey' 'white'
    )
    plt.savefig(f"{figdir}/visium_cell2loc_LRaCMs_{slide_id}.pdf", bbox_inches="tight",dpi=300)
```

Example 15 (unknown):
```unknown
/tmp/ipykernel_42878/2805736760.py:18: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  slides.uns['spatial'] = uns_dict
```

Example 16 (unknown):
```unknown
f"{figdir}/visium_cell2loc_LRaCMs_{slide_id}.pdf"
```

Example 17 (unknown):
```unknown
'/nfs/team205/heart/fetal_ms_figures/raw_plots/visium_cell2loc_LRaCMs_BHF_F_Hea13295830.pdf'
```

---

## Inputs¶

**URL:** http://127.0.0.1:9020/notebooks_html/Supplementary_materials/Macrophages/Mo-MP_no6_MP-LYVE1_fitting-features.html

**Contents:**
- Inputs¶
- Read in trajectory object¶
- Show trajectories¶
- Subset populations¶
- Extract gene features and log-normalise¶
- Add normalised peak counts¶
- Read in scenicplus output and create anndata¶
- Differential test¶
- Save anndata¶
- Fitting¶
- Regulon - auc gene¶
- Regulon - auc region¶
- Gene¶
- Region¶
- Save result of differential and fitting result¶

1960 rows × 6 columns

1960 rows × 6 columns

884527 rows × 6 columns

**Examples:**

Example 1 (python):
```python
import scanpy as sc
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import anndata
import scipy
import os

import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
-----
anndata             0.9.2
matplotlib          3.7.2
numpy               1.24.4
pandas              1.5.3
scanpy              1.9.3
scipy               1.10.1
session_info        1.0.0
-----
```

Example 4 (python):
```python
PIL                         10.0.0
anyio                       NA
arrow                       1.2.3
asttokens                   NA
attr                        23.1.0
attrs                       23.1.0
babel                       2.12.1
backcall                    0.2.0
brotli                      1.0.9
certifi                     2023.11.17
cffi                        1.15.1
charset_normalizer          3.2.0
colorama                    0.4.6
comm                        0.1.4
cycler                      0.10.0
cython_runtime              NA
dateutil                    2.8.2
debugpy                     1.6.8
decorator                   5.1.1
defusedxml                  0.7.1
executing                   1.2.0
fastjsonschema              NA
fqdn                        NA
h5py                        3.9.0
idna                        3.4
igraph                      0.10.6
importlib_metadata          NA
importlib_resources         NA
ipykernel                   6.25.1
isoduration                 NA
jedi                        0.19.0
jinja2                      3.1.2
joblib                      1.3.2
json5                       NA
jsonpointer                 2.0
jsonschema                  4.19.0
jsonschema_specifications   NA
jupyter_events              0.7.0
jupyter_server              2.7.1
jupyterlab_server           2.24.0
kiwisolver                  1.4.5
leidenalg                   0.10.1
llvmlite                    0.40.1
markupsafe                  2.1.3
mpl_toolkits                NA
natsort                     8.4.0
nbformat                    5.9.2
numba                       0.57.1
overrides                   NA
packaging                   23.1
parso                       0.8.3
pexpect                     4.8.0
pickleshare                 0.7.5
pkg_resources               NA
plotly                      5.16.1
prometheus_client           NA
prompt_toolkit              3.0.39
psutil                      5.9.5
ptyprocess                  0.7.0
pure_eval                   0.2.2
pydev_ipython               NA
pydevconsole                NA
pydevd                      2.9.5
pydevd_file_utils           NA
pydevd_plugins              NA
pydevd_tracing              NA
pygments                    2.16.1
pyparsing                   3.0.9
pythonjsonlogger            NA
pytz                        2023.3
referencing                 NA
requests                    2.31.0
rfc3339_validator           0.1.4
rfc3986_validator           0.1.1
rpds                        NA
ruamel                      NA
send2trash                  NA
six                         1.16.0
sklearn                     1.3.0
sniffio                     1.3.0
socks                       1.7.1
sphinxcontrib               NA
stack_data                  0.6.2
texttable                   1.6.7
threadpoolctl               3.2.0
tornado                     6.3.3
traitlets                   5.9.0
typing_extensions           NA
uri_template                NA
urllib3                     2.0.4
wcwidth                     0.2.6
webcolors                   1.13
websocket                   1.6.2
yaml                        6.0.1
zipp                        NA
zmq                         25.1.1
zstandard                   0.19.0
```

Example 5 (unknown):
```unknown
-----
IPython             8.12.0
jupyter_client      8.3.0
jupyter_core        4.12.0
jupyterlab          4.0.5
notebook            7.0.2
-----
Python 3.8.17 (default, Jul  5 2023, 21:04:15) [GCC 11.2.0]
Linux-5.4.0-169-generic-x86_64-with-glibc2.17
-----
Session information updated at 2024-02-06 16:16
```

Example 6 (python):
```python
sc.settings.set_figure_params(dpi=120)
```

Example 7 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 8 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk'
```

Example 9 (python):
```python
adata_path = '/nfs/team205/heart/anndata_objects/Foetal/Immune/MoMP_Aug2023/MultiVI-post_foetal_Mono-MP_withContCov-CCCov_13Mar2023_feature-select-0003.h5ad'

# methods to subset populations
key_of_groups = 'fine_grain_mod'
# groups = ['VentricularCardiomyocytesLeftCompact','VentricularCardiomyocytesLeftTrabeculated']
```

Example 10 (python):
```python
adata = sc.read_h5ad(adata_path)

# rename obs_names
adata.obs_names = [x.replace('_expression','').replace('_paired','') for x in adata.obs_names]

adata
```

Example 11 (python):
```python
AnnData object with n_obs × n_vars = 7674 × 142283
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', '_indices', '_scvi_batch', '_scvi_labels', 'fine_grain_withcycling', 't', 'seg', 'edge', 't_sd', 'milestones', 'fine_grain_mod'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id', 'n_cells'
    uns: '_scvi_manager_uuid', '_scvi_uuid', 'adata_auc_gene', 'adata_auc_region', 'adata_gex', 'adata_gex_lognorm', 'adata_peak_tfidf', 'draw_graph', 'fine_grain_colors', 'fine_grain_mod_colors', 'fine_grain_withcycling_colors', 'graph', 'milestones_colors', 'neighbors', 'ppt', 'pseudotime_list', 'seg_colors', 'umap', 'week_colors'
    obsm: 'MultiVI_latent', 'X_R', 'X_draw_graph_fa', 'X_palantir', 'X_umap', 'X_umap13', '_scvi_extra_categorical_covs', '_scvi_extra_continuous_covs'
    obsp: 'connectivities', 'distances'
```

Example 12 (python):
```python
adata.obs[key_of_groups].value_counts()
```

Example 13 (unknown):
```unknown
MacrophagesLYVE1pos     4757
MacrophagesATF3pos       898
MacrophagesCX3CR1pos     804
Monocytes                496
MacrophagesYSderived     207
MonocytesMPOpos          202
MonocyteDerivedCells     199
MacrophagesTIMD4pos      111
Name: fine_grain_mod, dtype: int64
```

Example 14 (python):
```python
sc.pl.draw_graph(adata,color=['t',key_of_groups],color_map="viridis", wspace=0.2)
```

Example 15 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 16 (python):
```python
set(adata.obs['seg'])
```

Example 17 (unknown):
```unknown
{'1', '2', '3', '4', '5', '6', '7', '8', '9'}
```

Example 18 (python):
```python
from PIL import Image
image = Image.open('./MoMP_trajectory.png')

plt.figure(figsize=(5, 5))
plt.imshow(image)
plt.axis('off')
plt.grid(False)
plt.show()
```

Example 19 (python):
```python
plt.rcParams['figure.figsize'] = 1.5,4
segments_to_select = ['3', '4', '5', '6', '7', '8', '9']
adata = adata[adata.obs['seg'].isin(segments_to_select)]
sc.pl.draw_graph(adata,color=['t',key_of_groups],color_map="viridis", wspace=0.5)
```

Example 20 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 21 (python):
```python
# scale t
# min-max scaling
adata.obs['t_scaled'] = (adata.obs['t']-adata.obs['t'].min())/(adata.obs['t'].max()-adata.obs['t'].min())

plt.rcParams['figure.figsize'] = 1.7,3.5
sc.pl.draw_graph(adata,color=['t_scaled'],color_map="viridis",frameon=False,
                 title='MP maturation\n(pseudotime)', show=False
                )
plt.savefig(f"{figdir}/Force-directed-graph_MP_pseudotime.pdf", bbox_inches="tight",dpi=300)
```

Example 22 (python):
```python
/tmp/ipykernel_203412/3235555261.py:3: ImplicitModificationWarning: Trying to modify attribute `.obs` of view, initializing view as actual.
  adata.obs['t_scaled'] = (adata.obs['t']-adata.obs['t'].min())/(adata.obs['t'].max()-adata.obs['t'].min())
```

Example 23 (python):
```python
adata.obs['week']
```

Example 24 (unknown):
```unknown
BHF_F_Hea11933667_BHF_F_Hea11596620_TCATGCCTCGTTATCT-1    13W
BHF_F_Hea11933667_BHF_F_Hea11596620_TCTAATCTCCTCACTA-1    13W
7089STDY13216927_BHF_F_Hea13242534_GTTTCTAGTCCTAGTT-1     12W
BHF_F_Hea11933671_BHF_F_Hea11596624_GCTAGCCAGCACTTGG-1     5W
BHF_F_Hea11933668_BHF_F_Hea11596621_GTTTCTAGTTAGCTGA-1    13W
                                                         ... 
BHF_F_Hea11192328_AAGGTTCTCAGAGGTG-1                       8W
BHF_F_Hea11192328_AAGACCTCACACGCTG-1                       8W
BHF_F_Hea11192328_CATATGGCAGCTCGCA-1                       8W
BHF_F_Hea11192328_CAACCAACACGCGAAA-1                       8W
BHF_F_Hea11192328_GGATGTTGTGTTTGTG-1                       8W
Name: week, Length: 3886, dtype: category
Categories (7, object): ['5W', '7W', '8W', '9W', '10W', '12W', '13W']
```

Example 25 (python):
```python
# convert to continuous week
adata.obs['week_int'] = [int(x.replace('W','')) for x in adata.obs['week']]

plt.rcParams['figure.figsize'] = 1.7,3.5
sc.pl.draw_graph(adata,color=['week_int'],color_map="viridis",frameon=False,
                 title='PCW', show=False
                )
plt.savefig(f"{figdir}/Force-directed-graph_MP_PCW.pdf", bbox_inches="tight",dpi=300)
```

Example 26 (python):
```python
# extract gene features
adata_gex = adata.copy()
gene_vars = adata_gex.var_names[adata_gex.var['modality']=='Gene Expression']
adata_gex = adata_gex[:,gene_vars]

# log normalise 
print(adata_gex.X.data[:10])
sc.pp.normalize_total(adata_gex, target_sum=1e4)
sc.pp.log1p(adata_gex)
print(adata_gex.X.data[:10])

# store to .uns
adata.uns['adata_gex_lognorm']=adata_gex.copy()
adata.uns['adata_gex_lognorm']
```

Example 27 (unknown):
```unknown
[1. 7. 1. 4. 1. 4. 1. 1. 9. 5.]
```

Example 28 (python):
```python
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/preprocessing/_normalization.py:170: UserWarning: Received a view of an AnnData. Making a copy.
  view_to_actual(adata)
```

Example 29 (unknown):
```unknown
[1.0841573 2.6877544 1.0841573 2.1779046 1.0841573 2.1779046 1.0841573
 1.0841573 2.9238348 2.3781319]
```

Example 30 (python):
```python
/tmp/ipykernel_150291/1223271433.py:13: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  adata.uns['adata_gex_lognorm']=adata_gex.copy()
```

Example 31 (python):
```python
AnnData object with n_obs × n_vars = 3886 × 15922
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', '_indices', '_scvi_batch', '_scvi_labels', 'fine_grain_withcycling', 't', 'seg', 'edge', 't_sd', 'milestones', 'fine_grain_mod'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id', 'n_cells'
    uns: '_scvi_manager_uuid', '_scvi_uuid', 'adata_auc_gene', 'adata_auc_region', 'adata_gex', 'adata_gex_lognorm', 'adata_peak_tfidf', 'draw_graph', 'fine_grain_colors', 'fine_grain_mod_colors', 'fine_grain_withcycling_colors', 'graph', 'milestones_colors', 'neighbors', 'ppt', 'pseudotime_list', 'seg_colors', 'umap', 'week_colors', 'log1p'
    obsm: 'MultiVI_latent', 'X_R', 'X_draw_graph_fa', 'X_palantir', 'X_umap', 'X_umap13', '_scvi_extra_categorical_covs', '_scvi_extra_continuous_covs'
    obsp: 'connectivities', 'distances'
```

Example 32 (python):
```python
adata_peak = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/PeakMatrix/Foetal_Peaks_tfidf.h5ad')

shared_cell_ids = list(set(adata.obs_names).intersection(adata_peak.obs_names))

# subset cells and regions
adata.uns['adata_peak_tfidf']=adata_peak[shared_cell_ids,adata.var_names[adata.var['modality']=='Peaks']]
adata_peak = adata.uns['adata_peak_tfidf'].copy()
adata_peak
```

Example 33 (unknown):
```unknown
AnnData object with n_obs × n_vars = 1162 × 126361
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels'
    var: 'seqnames', 'start', 'end', 'width', 'strand', 'score', 'replicateScoreQuantile', 'groupScoreQuantile', 'Reproducibility', 'GroupReplicate', 'distToGeneStart', 'nearestGene', 'peakType', 'distToTSS', 'nearestTSS', 'GC', 'idx', 'N'
```

Example 34 (python):
```python
# add milestones and segments
columns_to_add = ['t', 'seg', 'edge', 't_sd', 'milestones']
adata.uns['adata_peak_tfidf'].obs[columns_to_add] = adata.obs.loc[adata.uns['adata_peak_tfidf'].obs_names, columns_to_add]
adata_peak = adata.uns['adata_peak_tfidf'].copy()
adata_peak
```

Example 35 (python):
```python
/tmp/ipykernel_150291/281467255.py:3: ImplicitModificationWarning: Trying to modify attribute `.obs` of view, initializing view as actual.
  adata.uns['adata_peak_tfidf'].obs[columns_to_add] = adata.obs.loc[adata.uns['adata_peak_tfidf'].obs_names, columns_to_add]
```

Example 36 (unknown):
```unknown
AnnData object with n_obs × n_vars = 1162 × 126361
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 't', 'seg', 'edge', 't_sd', 'milestones'
    var: 'seqnames', 'start', 'end', 'width', 'strand', 'score', 'replicateScoreQuantile', 'groupScoreQuantile', 'Reproducibility', 'GroupReplicate', 'distToGeneStart', 'nearestGene', 'peakType', 'distToTSS', 'nearestTSS', 'GC', 'idx', 'N'
```

Example 37 (python):
```python
# read in regulon score
auc_gene = pd.read_csv('/nfs/team205/heart/pyscenic/plus/AUCscore/MoMP_eRegulon_AUC_gene.csv',index_col=0)
print(auc_gene.shape)
auc_region = pd.read_csv('/nfs/team205/heart/pyscenic/plus/AUCscore/MoMP_eRegulon_AUC_region.csv',index_col=0)
print(auc_region.shape)

# select shared barcodes (ie only multiome data, only monocyte/macrophage)
shared_cell_ids = list(set(auc_gene.index).intersection(auc_region.index).intersection(adata.obs_names))
print(len(shared_cell_ids))
```

Example 38 (unknown):
```unknown
(1899, 280)
(1899, 280)
1162
```

Example 39 (python):
```python
# gene
# select cells in anndata (monocyte and macrophage)
auc_gene = auc_gene.loc[shared_cell_ids]

# create anndata
adata_aucg = anndata.AnnData(
    X=auc_gene,
    obs=adata[shared_cell_ids].obs,
    var=pd.DataFrame(auc_gene.columns).set_index(0),
    obsm=adata[shared_cell_ids].obsm
    # obsp=adata.obsp.copy(),
    # uns=adata.uns.copy(),
)
# modify varnames
adata_aucg.var_names = [x.split('_(')[0] for x in adata_aucg.var_names]

# store to .uns
adata.uns['adata_auc_gene']=adata_aucg.copy()
adata.uns['adata_auc_gene']
```

Example 40 (unknown):
```unknown
AnnData object with n_obs × n_vars = 1162 × 280
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', '_indices', '_scvi_batch', '_scvi_labels', 'fine_grain_withcycling', 't', 'seg', 'edge', 't_sd', 'milestones', 'fine_grain_mod'
    obsm: 'MultiVI_latent', 'X_R', 'X_draw_graph_fa', 'X_palantir', 'X_umap', 'X_umap13', '_scvi_extra_categorical_covs', '_scvi_extra_continuous_covs'
```

Example 41 (python):
```python
# region
# select cells in anndata (monocyte and macrophage)
auc_region = auc_region.loc[shared_cell_ids]

# create anndata
adata_aucr = anndata.AnnData(
    X=auc_region,
    obs=adata[shared_cell_ids].obs,
    var=pd.DataFrame(auc_region.columns).set_index(0),
    obsm=adata[shared_cell_ids].obsm
    # obsp=adata.obsp.copy(),
    # uns=adata.uns.copy(),
)
# modify varnames
adata_aucr.var_names = [x.split('_(')[0] for x in adata_aucr.var_names]

# store to .uns
adata.uns['adata_auc_region']=adata_aucr.copy()
adata.uns['adata_auc_region']
```

Example 42 (unknown):
```unknown
AnnData object with n_obs × n_vars = 1162 × 280
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', '_indices', '_scvi_batch', '_scvi_labels', 'fine_grain_withcycling', 't', 'seg', 'edge', 't_sd', 'milestones', 'fine_grain_mod'
    obsm: 'MultiVI_latent', 'X_R', 'X_draw_graph_fa', 'X_palantir', 'X_umap', 'X_umap13', '_scvi_extra_categorical_covs', '_scvi_extra_continuous_covs'
```

Example 43 (unknown):
```unknown
root_milestone = '29'
```

Example 44 (python):
```python
# prepare a dictionary to put result
diff_dict = {}
diff_dict['auc_gene']=np.nan
diff_dict['auc_region']=np.nan
diff_dict['gene']=np.nan
diff_dict['region']=np.nan
diff_dict
```

Example 45 (unknown):
```unknown
{'auc_gene': nan, 'auc_region': nan, 'gene': nan, 'region': nan}
```

Example 46 (python):
```python
# Differential test
# genes
sc.tl.rank_genes_groups(adata_gex,groupby='milestones',reference=root_milestone,method='t-test')
## Get result and store in .uns
DE_df=sc.get.rank_genes_groups_df(adata_gex, group=None)
diff_dict['gene']=DE_df.copy()

# regions
sc.tl.rank_genes_groups(adata_peak,groupby='milestones',reference=root_milestone,method='t-test')
## Get result and store in .uns
DE_df=sc.get.rank_genes_groups_df(adata_peak, group=None)
diff_dict['region']=DE_df.copy()

# auc-gene
sc.tl.rank_genes_groups(adata_aucg,groupby='milestones',reference=root_milestone,method='t-test')
## Get result and store in .uns
DE_df=sc.get.rank_genes_groups_df(adata_aucg, group=None)
diff_dict['auc_gene']=DE_df.copy()

# auc-region
sc.tl.rank_genes_groups(adata_aucr,groupby='milestones',reference=root_milestone,method='t-test')
## Get result and store in .uns
DE_df=sc.get.rank_genes_groups_df(adata_aucr, group=None)
diff_dict['auc_region']=DE_df.copy()
```

Example 47 (unknown):
```unknown
diff_dict['auc_gene']
```

Example 48 (unknown):
```unknown
diff_dict['auc_region']
```

Example 49 (unknown):
```unknown
diff_dict['region']
```

Example 50 (unknown):
```unknown
segments_to_select.sort()
outdir = f'/nfs/team205/heart/anndata_objects/Foetal/Immune/MoMP_Aug2023/seg_{"-".join(segments_to_select)}'
outdir
```

Example 51 (unknown):
```unknown
'/nfs/team205/heart/anndata_objects/Foetal/Immune/MoMP_Aug2023/seg_3-4-5-6-7-8-9'
```

Example 52 (python):
```python
# save anndata
if os.path.exists(outdir):
    print('directory already exists')
else:
    os.makedirs(outdir)
adata.write(f'{outdir}/adata.h5ad')
adata
```

Example 53 (python):
```python
AnnData object with n_obs × n_vars = 3886 × 142283
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', '_indices', '_scvi_batch', '_scvi_labels', 'fine_grain_withcycling', 't', 'seg', 'edge', 't_sd', 'milestones', 'fine_grain_mod'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id', 'n_cells'
    uns: '_scvi_manager_uuid', '_scvi_uuid', 'adata_auc_gene', 'adata_auc_region', 'adata_gex', 'adata_gex_lognorm', 'adata_peak_tfidf', 'draw_graph', 'fine_grain_colors', 'fine_grain_mod_colors', 'fine_grain_withcycling_colors', 'graph', 'milestones_colors', 'neighbors', 'ppt', 'pseudotime_list', 'seg_colors', 'umap', 'week_colors'
    obsm: 'MultiVI_latent', 'X_R', 'X_draw_graph_fa', 'X_palantir', 'X_umap', 'X_umap13', '_scvi_extra_categorical_covs', '_scvi_extra_continuous_covs'
    obsp: 'connectivities', 'distances'
```

Example 54 (python):
```python
# read in anndata
adata = sc.read_h5ad(f'{outdir}/adata.h5ad')
adata
```

Example 55 (python):
```python
import scFates as scf
```

Example 56 (python):
```python
# prepare a dictionary to put result
fit_dict = {}
fit_dict['auc_gene']=np.nan
fit_dict['auc_region']=np.nan
fit_dict['gene']=np.nan
fit_dict['region']=np.nan
fit_dict
```

Example 57 (unknown):
```unknown
{'auc_gene': nan, 'auc_region': nan, 'gene': nan, 'region': nan}
```

Example 58 (python):
```python
%%time
# fit
scf.tl.test_association(adata.uns['adata_auc_gene'],n_jobs=20)

# extract results and store to the dictionary
df = adata.uns['adata_auc_gene'].var.copy()
df['activator_repressor'] = [x.split('_')[-1] for x in df.index]
df = df.sort_values('A',ascending=False)
fit_dict['auc_gene'] = df.copy()
```

Example 59 (unknown):
```unknown
test features for association with the trajectory
    single mapping :   0%|          | 0/280 [00:00<?, ?it/s]
```

Example 60 (unknown):
```unknown
Unable to determine R library path: Command '('/software/isg/private/R-4.1.2/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/software/isg/private/R-4.1.2/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/software/isg/private/R-4.1.2/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
```

Example 61 (unknown):
```unknown
single mapping :   1%|          | 3/280 [01:01<1:34:08, 20.39s/it]
```

Example 62 (unknown):
```unknown
Unable to determine R library path: Command '('/software/isg/private/R-4.1.2/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/software/isg/private/R-4.1.2/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/software/isg/private/R-4.1.2/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/software/isg/private/R-4.1.2/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
```

Example 63 (unknown):
```unknown
single mapping :   2%|▎         | 7/280 [02:18<1:30:18, 19.85s/it]
```

Example 64 (unknown):
```unknown
Unable to determine R library path: Command '('/software/isg/private/R-4.1.2/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/software/isg/private/R-4.1.2/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/software/isg/private/R-4.1.2/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/software/isg/private/R-4.1.2/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/software/isg/private/R-4.1.2/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/software/isg/private/R-4.1.2/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/software/isg/private/R-4.1.2/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/software/isg/private/R-4.1.2/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/software/isg/private/R-4.1.2/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/software/isg/private/R-4.1.2/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/software/isg/private/R-4.1.2/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/software/isg/private/R-4.1.2/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/joblib/externals/loky/process_executor.py:752: UserWarning: A worker stopped while some jobs were given to the executor. This can be caused by a too short worker timeout or by a memory leak.
  warnings.warn(
```

Example 65 (unknown):
```unknown
single mapping :  69%|██████▉   | 194/280 [06:17<02:47,  1.94s/it]
```

Example 66 (unknown):
```unknown
Unable to determine R library path: Command '('/software/isg/private/R-4.1.2/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
```

Example 67 (python):
```python
single mapping : 100%|██████████| 280/280 [06:45<00:00,  1.45s/it]
    found 0 significant features (0:06:45) --> added
    .var['p_val'] values from statistical test.
    .var['fdr'] corrected values from multiple testing.
    .var['st'] proportion of mapping in which feature is significant.
    .var['A'] amplitue of change of tested feature.
    .var['signi'] feature is significantly changing along pseudotime.
    .uns['stat_assoc_list'] list of fitted features on the graph for all mappings.
CPU times: user 1.89 s, sys: 2.75 s, total: 4.63 s
Wall time: 6min 45s
```

Example 68 (python):
```python
%%time
# fit
scf.tl.test_association(adata.uns['adata_auc_region'],n_jobs=20)

# extract results and store to the dictionary
df = adata.uns['adata_auc_region'].var.copy()
df['activator_repressor'] = [x.split('_')[-1] for x in df.index]
df = df.sort_values('A',ascending=False)
fit_dict['auc_region'] = df.copy()
```

Example 69 (python):
```python
test features for association with the trajectory
    single mapping : 100%|██████████| 280/280 [01:41<00:00,  2.76it/s]
    found 0 significant features (0:01:41) --> added
    .var['p_val'] values from statistical test.
    .var['fdr'] corrected values from multiple testing.
    .var['st'] proportion of mapping in which feature is significant.
    .var['A'] amplitue of change of tested feature.
    .var['signi'] feature is significantly changing along pseudotime.
    .uns['stat_assoc_list'] list of fitted features on the graph for all mappings.
CPU times: user 1.87 s, sys: 620 ms, total: 2.49 s
Wall time: 1min 41s
```

Example 70 (python):
```python
%%time
# fit
scf.tl.test_association(adata.uns['adata_gex_lognorm'],n_jobs=20)

# extract results and store to the dictionary
df = adata.uns['adata_gex_lognorm'].var.copy()
df = df.sort_values('A',ascending=False)
fit_dict['gene'] = df.copy()
```

Example 71 (python):
```python
test features for association with the trajectory
    single mapping : 100%|██████████| 15922/15922 [1:56:39<00:00,  2.27it/s]
    found 1045 significant features (1:56:39) --> added
    .var['p_val'] values from statistical test.
    .var['fdr'] corrected values from multiple testing.
    .var['st'] proportion of mapping in which feature is significant.
    .var['A'] amplitue of change of tested feature.
    .var['signi'] feature is significantly changing along pseudotime.
    .uns['stat_assoc_list'] list of fitted features on the graph for all mappings.
CPU times: user 1min 31s, sys: 28.4 s, total: 2min
Wall time: 1h 56min 45s
```

Example 72 (python):
```python
%%time
# fit
scf.tl.test_association(adata.uns['adata_peak_tfidf'],n_jobs=20)

# extract results and store to the dictionary
df = adata.uns['adata_peak_tfidf'].var.copy()
df = df.sort_values('A',ascending=False)
fit_dict['region'] = df.copy()
```

Example 73 (unknown):
```unknown
test features for association with the trajectory
    single mapping :   0%|          | 0/126361 [00:00<?, ?it/s]
```

Example 74 (unknown):
```unknown
Unable to determine R library path: Command '('/software/isg/private/R-4.1.2/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/software/isg/private/R-4.1.2/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/software/isg/private/R-4.1.2/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
```

Example 75 (unknown):
```unknown
single mapping :   0%|          | 3/126361 [00:48<571:45:08, 16.29s/it]
```

Example 76 (unknown):
```unknown
Unable to determine R library path: Command '('/software/isg/private/R-4.1.2/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/software/isg/private/R-4.1.2/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/software/isg/private/R-4.1.2/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/software/isg/private/R-4.1.2/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/software/isg/private/R-4.1.2/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/software/isg/private/R-4.1.2/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/software/isg/private/R-4.1.2/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/software/isg/private/R-4.1.2/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/software/isg/private/R-4.1.2/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
```

Example 77 (unknown):
```unknown
single mapping :   0%|          | 127/126361 [03:54<64:45:58,  1.85s/it]
```

Example 78 (unknown):
```unknown
Unable to determine R library path: Command '('/software/isg/private/R-4.1.2/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/software/isg/private/R-4.1.2/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/software/isg/private/R-4.1.2/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/software/isg/private/R-4.1.2/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/software/isg/private/R-4.1.2/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/software/isg/private/R-4.1.2/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/software/isg/private/R-4.1.2/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/software/isg/private/R-4.1.2/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
```

Example 79 (python):
```python
single mapping : 100%|██████████| 126361/126361 [5:08:20<00:00,  6.83it/s]  
    found 700 significant features (5:08:20) --> added
    .var['p_val'] values from statistical test.
    .var['fdr'] corrected values from multiple testing.
    .var['st'] proportion of mapping in which feature is significant.
    .var['A'] amplitue of change of tested feature.
    .var['signi'] feature is significantly changing along pseudotime.
    .uns['stat_assoc_list'] list of fitted features on the graph for all mappings.
CPU times: user 6min 36s, sys: 1min 2s, total: 7min 38s
Wall time: 5h 8min 33s
```

Example 80 (unknown):
```unknown
utils.save_pkl(diff_dict, 
         file=f'{outdir}/differential-test_result.pkl')
```

Example 81 (unknown):
```unknown
utils.save_pkl(fit_dict, 
         file=f'{outdir}/fitting_result.pkl')
```

---

## Import module¶

**URL:** http://127.0.0.1:9020/notebooks_html/Supplementary_materials/Epigenetic_stability/no1b_peakVI_per-midgrain.html

**Contents:**
- Import module¶
- Read in data¶
- Subset anndata¶
- Modify mid grain¶
- Run peakVI¶

**Examples:**

Example 1 (python):
```python
import scvi
import anndata
import scipy
import numpy as np
import pandas as pd
import scanpy as sc
import matplotlib.pyplot as plt

scvi.settings.seed = 420
```

Example 2 (python):
```python
Global seed set to 0
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/flax/core/frozen_dict.py:169: FutureWarning: jax.tree_util.register_keypaths is deprecated, and will be removed in a future release. Please use `register_pytree_with_keys()` instead.
  jax.tree_util.register_keypaths(
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/pytorch_lightning/utilities/warnings.py:53: LightningDeprecationWarning: pytorch_lightning.utilities.warnings.rank_zero_deprecation has been deprecated in v1.6 and will be removed in v1.8. Use the equivalent function from the pytorch_lightning.utilities.rank_zero module instead.
  new_rank_zero_deprecation(
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/pytorch_lightning/utilities/warnings.py:58: LightningDeprecationWarning: The `pytorch_lightning.loggers.base.rank_zero_experiment` is deprecated in v1.7 and will be removed in v1.9. Please use `pytorch_lightning.loggers.logger.rank_zero_experiment` instead.
  return new_rank_zero_deprecation(*args, **kwargs)
Global seed set to 420
```

Example 3 (python):
```python
import gc
import torch
```

Example 4 (python):
```python
import session_info
session_info.show()
```

Example 5 (python):
```python
-----
anndata             0.8.0
matplotlib          3.7.1
numpy               1.23.5
pandas              1.5.3
scanpy              1.9.3
scipy               1.10.1
scvi                0.19.0
session_info        1.0.0
torch               1.13.1+cu117
-----
```

Example 6 (python):
```python
PIL                 9.4.0
absl                NA
asttokens           NA
attr                22.2.0
backcall            0.2.0
certifi             2022.12.07
cffi                1.15.1
charset_normalizer  3.1.0
chex                0.1.6
comm                0.1.2
contextlib2         NA
cycler              0.10.0
cython_runtime      NA
dateutil            2.8.2
debugpy             1.6.6
decorator           5.1.1
defusedxml          0.7.1
deprecate           0.3.2
docrep              0.3.2
etils               1.1.0
executing           1.2.0
flax                0.6.7
fsspec              2023.3.0
google              NA
h5py                3.8.0
idna                3.4
igraph              0.10.4
importlib_metadata  NA
importlib_resources NA
ipykernel           6.21.3
ipython_genutils    0.2.0
ipywidgets          8.0.4
jax                 0.4.10
jaxlib              0.4.10
jedi                0.18.2
joblib              1.2.0
kiwisolver          1.4.4
leidenalg           0.9.1
llvmlite            0.39.1
ml_collections      NA
ml_dtypes           0.2.0
mpl_toolkits        NA
msgpack             1.0.5
mudata              0.2.1
multipledispatch    0.6.0
natsort             8.3.1
numba               0.56.4
numpyro             0.11.0
nvidia              NA
opt_einsum          v3.3.0
optax               0.1.4
packaging           23.0
parso               0.8.3
pexpect             4.8.0
pickleshare         0.7.5
pkg_resources       NA
platformdirs        3.1.1
prompt_toolkit      3.0.38
psutil              5.9.4
ptyprocess          0.7.0
pure_eval           0.2.2
pydev_ipython       NA
pydevconsole        NA
pydevd              2.9.5
pydevd_file_utils   NA
pydevd_plugins      NA
pydevd_tracing      NA
pygments            2.14.0
pyparsing           3.0.9
pyro                1.8.4
pytorch_lightning   1.7.7
pytz                2022.7.1
requests            2.28.2
rich                NA
setuptools          65.6.3
six                 1.16.0
sklearn             1.2.2
stack_data          0.6.2
tensorboard         2.12.0
texttable           1.6.7
threadpoolctl       3.1.0
toolz               0.12.0
torchmetrics        0.11.4
torchvision         0.14.1+cu117
tornado             6.2
tqdm                4.65.0
traitlets           5.9.0
tree                0.1.8
typing_extensions   NA
urllib3             1.26.15
wcwidth             0.2.6
yaml                6.0
zipp                NA
zmq                 25.0.1
zoneinfo            NA
```

Example 7 (unknown):
```unknown
-----
IPython             8.11.0
jupyter_client      8.0.3
jupyter_core        5.2.0
notebook            6.5.3
-----
Python 3.9.16 (main, Mar  8 2023, 14:00:05) [GCC 11.2.0]
Linux-4.15.0-214-generic-x86_64-with-glibc2.27
-----
Session information updated at 2023-12-29 23:02
```

Example 8 (python):
```python
adata = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/PeakMatrix/Foetal_Peaks.h5ad')
adata
```

Example 9 (unknown):
```unknown
AnnData object with n_obs × n_vars = 167022 × 508040
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels'
    var: 'seqnames', 'start', 'end', 'width', 'strand', 'score', 'replicateScoreQuantile', 'groupScoreQuantile', 'Reproducibility', 'GroupReplicate', 'distToGeneStart', 'nearestGene', 'peakType', 'distToTSS', 'nearestTSS', 'GC', 'idx', 'N'
```

Example 10 (python):
```python
adata.X.data[:10]
```

Example 11 (unknown):
```unknown
array([2., 2., 2., 2., 1., 1., 2., 1., 2., 2.], dtype=float32)
```

Example 12 (python):
```python
set(adata.obs.batch_key)
```

Example 13 (unknown):
```unknown
{'C104_Nuclei_Multiome-v1',
 'C82_Nuclei_Multiome-v1',
 'C85_Nuclei_Multiome-v1',
 'C87_Nuclei_Multiome-v1',
 'C92_Nuclei_Multiome-v1',
 'C98_Nuclei_Multiome-v1',
 'Hst33_Nuclei_Multiome-v1',
 'Hst36_Nuclei_Multiome-v1',
 'Hst39_Nuclei_Multiome-v1',
 'Hst40_Nuclei_Multiome-v1',
 'Hst41_Nuclei_Multiome-v1'}
```

Example 14 (python):
```python
# read in selected nuclei id
df = pd.read_csv('/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/GEX_ATAC_comparison/subsampled_nuclei_n1000_29Dec2023.txt',
                 header=None)
nuclei_sel = pd.Index(df[0])
print(len(nuclei_sel))
nuclei_sel[:5]
```

Example 15 (unknown):
```unknown
Index(['7089STDY13216921_BHF_F_Hea13242528_TGGGCCTAGATGGACA-1',
       '7089STDY13216921_BHF_F_Hea13242528_AAAGGCTCAGCTACGT-1',
       '7089STDY13216921_BHF_F_Hea13242528_CATCATAAGCGATACT-1',
       '7089STDY13216921_BHF_F_Hea13242528_CAGGCTATCACCAATA-1',
       '7089STDY13216921_BHF_F_Hea13242528_CTAATGTCACTTCATC-1'],
      dtype='object', name=0)
```

Example 16 (python):
```python
# subset
adata = adata[nuclei_sel]
adata
```

Example 17 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 44136 × 508040
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels'
    var: 'seqnames', 'start', 'end', 'width', 'strand', 'score', 'replicateScoreQuantile', 'groupScoreQuantile', 'Reproducibility', 'GroupReplicate', 'distToGeneStart', 'nearestGene', 'peakType', 'distToTSS', 'nearestTSS', 'GC', 'idx', 'N'
```

Example 18 (python):
```python
# get indices of CCS cells
a_ccs_indices = adata.obs_names[adata.obs['fine_grain'].isin(['SinoatrialNodeCardiomyocytes','AtrioventricularNodeCardiomyocytes'])]
v_ccs_indices = adata.obs_names[adata.obs['fine_grain'].isin(['VentricularConductionSystemProximal','VentricularConductionSystemDistal'])]
# modify
adata.obs['mid_grain_mod'] = adata.obs['mid_grain'].astype('str').copy()
adata.obs.replace({'mid_grain_mod':{
    'AtrialCardiomyocytes':'AtrialCardiomyocytes_plusCCS',
    'VentricularCardiomyocytes':'VentricularCardiomyocytes_plusCCS'
}},inplace=True)
adata.obs.loc[a_ccs_indices,'mid_grain_mod'] = 'AtrialCardiomyocytes_plusCCS'
adata.obs.loc[v_ccs_indices,'mid_grain_mod'] = 'VentricularCardiomyocytes_plusCCS'
adata.obs['mid_grain_mod'].value_counts()
```

Example 19 (python):
```python
/tmp/ipykernel_4463/1930117738.py:5: ImplicitModificationWarning: Trying to modify attribute `.obs` of view, initializing view as actual.
  adata.obs['mid_grain_mod'] = adata.obs['mid_grain'].astype('str').copy()
```

Example 20 (unknown):
```unknown
LymphoidCells                        7006
Fibroblasts                          6969
VentricularCardiomyocytes_plusCCS    6531
AtrialCardiomyocytes_plusCCS         4381
MyeloidCells                         4321
MuralCells                           3103
BloodVesselEndothelialCells          3028
PericardialCells                     2187
Glia                                 1759
EndocardialCells                     1410
EpicardialCells                      1340
Neurons                              1101
LymphaticEndothelialCells            1000
Name: mid_grain_mod, dtype: int64
```

Example 21 (python):
```python
# adding 'region' as covariate
outdir = '/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/PeakMatrix/mid_grain_mod'

for celltype in set(adata.obs['mid_grain_mod']):
    print(f'###### {celltype} ######')
    
    # subset
    adata_sub = adata[adata.obs['mid_grain_mod']==celltype]
    print(adata_sub.shape)
    adata_sub = adata_sub.copy()
    
    # train
    scvi.model.PEAKVI.setup_anndata(adata_sub, 
                                    categorical_covariate_keys=['batch_key','region'] # added 'region'
                                   )
    pvi = scvi.model.PEAKVI(adata_sub)
    pvi.view_anndata_setup(adata_sub)
    pvi.train()
    
    # add to anndata
    latent = pvi.get_latent_representation()
    adata_sub.obsm["X_PeakVI"] = latent
    
    # save
    adata_sub.write(f'{outdir}/Foetal_{celltype}_Peaks_post-peakVI_added-cov-region_subsampled-n1000.h5ad')
    pvi.save(f'{outdir}/Foetal_{celltype}_added-cov-region_subsampled-n1000_peakVI-model', overwrite=True)
    
    # Convergency plot
    pvi.history['elbo_train']
    x = np.linspace(1, (len(pvi.history['elbo_train'])), (len(pvi.history['elbo_train'])))
    plt.plot(x, pvi.history['elbo_train'], label="train")
    plt.xlabel('epoch')
    plt.ylabel('elbo_train')
    plt.title(celltype)
    plt.show()
    
    # free up GPU
    del(pvi)
    gc.collect()
    torch.cuda.empty_cache()
```

Example 22 (unknown):
```unknown
###### AtrialCardiomyocytes_plusCCS ######
(4381, 508040)
```

Example 23 (unknown):
```unknown
No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.)
```

Example 24 (unknown):
```unknown
Anndata setup with scvi-tools version 0.19.0.
```

Example 25 (unknown):
```unknown
Setup via `PEAKVI.setup_anndata` with arguments:
```

Example 26 (unknown):
```unknown
{
│   'batch_key': None,
│   'labels_key': None,
│   'categorical_covariate_keys': ['batch_key', 'region'],
│   'continuous_covariate_keys': None,
│   'layer': None
}
```

Example 27 (unknown):
```unknown
Summary Statistics          
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃     Summary Stat Key     ┃ Value  ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│         n_batch          │   1    │
│         n_cells          │  4381  │
│ n_extra_categorical_covs │   2    │
│ n_extra_continuous_covs  │   0    │
│         n_labels         │   1    │
│          n_vars          │ 508040 │
└──────────────────────────┴────────┘
```

Example 28 (python):
```python
Data Registry                             
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃      Registry Key      ┃            scvi-tools Location             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│           X            │                  adata.X                   │
│         batch          │          adata.obs['_scvi_batch']          │
│ extra_categorical_covs │ adata.obsm['_scvi_extra_categorical_covs'] │
│         labels         │         adata.obs['_scvi_labels']          │
└────────────────────────┴────────────────────────────────────────────┘
```

Example 29 (python):
```python
batch State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃     Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_batch'] │     0      │          0          │
└──────────────────────────┴────────────┴─────────────────────┘
```

Example 30 (python):
```python
labels State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 31 (python):
```python
extra_categorical_covs State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location     ┃          Categories           ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['batch_key'] │    C82_Nuclei_Multiome-v1     │          0          │
│                        │    C85_Nuclei_Multiome-v1     │          1          │
│                        │    C87_Nuclei_Multiome-v1     │          2          │
│                        │    C92_Nuclei_Multiome-v1     │          3          │
│                        │    C98_Nuclei_Multiome-v1     │          4          │
│                        │    C104_Nuclei_Multiome-v1    │          5          │
│                        │   Hst33_Nuclei_Multiome-v1    │          6          │
│                        │   Hst36_Nuclei_Multiome-v1    │          7          │
│                        │   Hst39_Nuclei_Multiome-v1    │          8          │
│                        │   Hst40_Nuclei_Multiome-v1    │          9          │
│                        │   Hst41_Nuclei_Multiome-v1    │         10          │
│                        │                               │                     │
│  adata.obs['region']   │             aorta             │          0          │
│                        │             atria             │          1          │
│                        │         atrial septum         │          2          │
│                        │      heart without node       │          3          │
│                        │             node              │          4          │
│                        │         outflow tract         │          5          │
│                        │ pulmonary arches and branches │          6          │
│                        │         whole sample          │          7          │
│                        │                               │                     │
└────────────────────────┴───────────────────────────────┴─────────────────────┘
```

Example 32 (unknown):
```unknown
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/pytorch_lightning/trainer/configuration_validator.py:267: LightningDeprecationWarning: The `Callback.on_epoch_end` hook was deprecated in v1.6 and will be removed in v1.8. Please use `Callback.on_<train/validation/test>_epoch_end` instead.
  rank_zero_deprecation(
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
```

Example 33 (unknown):
```unknown
Epoch 85/500:  17%|███████████████████████▊                                                                                                                    | 85/500 [20:37<1:40:42, 14.56s/it, loss=4.09e+08, v_num=1]
Monitored metric reconstruction_loss_validation did not improve in the last 50 records. Best score: 33620.641. Signaling Trainer to stop.
```

Example 34 (unknown):
```unknown
###### BloodVesselEndothelialCells ######
(3028, 508040)
```

Example 35 (unknown):
```unknown
Anndata setup with scvi-tools version 0.19.0.
```

Example 36 (unknown):
```unknown
Setup via `PEAKVI.setup_anndata` with arguments:
```

Example 37 (unknown):
```unknown
{
│   'batch_key': None,
│   'labels_key': None,
│   'categorical_covariate_keys': ['batch_key', 'region'],
│   'continuous_covariate_keys': None,
│   'layer': None
}
```

Example 38 (unknown):
```unknown
Summary Statistics          
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃     Summary Stat Key     ┃ Value  ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│         n_batch          │   1    │
│         n_cells          │  3028  │
│ n_extra_categorical_covs │   2    │
│ n_extra_continuous_covs  │   0    │
│         n_labels         │   1    │
│          n_vars          │ 508040 │
└──────────────────────────┴────────┘
```

Example 39 (python):
```python
Data Registry                             
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃      Registry Key      ┃            scvi-tools Location             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│           X            │                  adata.X                   │
│         batch          │          adata.obs['_scvi_batch']          │
│ extra_categorical_covs │ adata.obsm['_scvi_extra_categorical_covs'] │
│         labels         │         adata.obs['_scvi_labels']          │
└────────────────────────┴────────────────────────────────────────────┘
```

Example 40 (python):
```python
batch State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃     Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_batch'] │     0      │          0          │
└──────────────────────────┴────────────┴─────────────────────┘
```

Example 41 (python):
```python
labels State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 42 (python):
```python
extra_categorical_covs State Registry                       
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location     ┃           Categories            ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['batch_key'] │     C82_Nuclei_Multiome-v1      │          0          │
│                        │     C85_Nuclei_Multiome-v1      │          1          │
│                        │     C87_Nuclei_Multiome-v1      │          2          │
│                        │     C92_Nuclei_Multiome-v1      │          3          │
│                        │     C98_Nuclei_Multiome-v1      │          4          │
│                        │     C104_Nuclei_Multiome-v1     │          5          │
│                        │    Hst33_Nuclei_Multiome-v1     │          6          │
│                        │    Hst36_Nuclei_Multiome-v1     │          7          │
│                        │    Hst39_Nuclei_Multiome-v1     │          8          │
│                        │    Hst40_Nuclei_Multiome-v1     │          9          │
│                        │    Hst41_Nuclei_Multiome-v1     │         10          │
│                        │                                 │                     │
│  adata.obs['region']   │              aorta              │          0          │
│                        │              apex               │          1          │
│                        │              atria              │          2          │
│                        │          atrial septum          │          3          │
│                        │        ductus arteriosus        │          4          │
│                        │       heart without node        │          5          │
│                        │         left ventricle          │          6          │
│                        │              node               │          7          │
│                        │          outflow tract          │          8          │
│                        │           pericardium           │          9          │
│                        │  pulmonary arches and branches  │         10          │
│                        │         right ventricle         │         11          │
│                        │ right ventricle and papillaries │         12          │
│                        │          whole sample           │         13          │
│                        │                                 │                     │
└────────────────────────┴─────────────────────────────────┴─────────────────────┘
```

Example 43 (unknown):
```unknown
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/pytorch_lightning/trainer/configuration_validator.py:267: LightningDeprecationWarning: The `Callback.on_epoch_end` hook was deprecated in v1.6 and will be removed in v1.8. Please use `Callback.on_<train/validation/test>_epoch_end` instead.
  rank_zero_deprecation(
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
```

Example 44 (unknown):
```unknown
Epoch 87/500:  17%|████████████████████████▎                                                                                                                   | 87/500 [14:18<1:07:56,  9.87s/it, loss=3.18e+08, v_num=1]
Monitored metric reconstruction_loss_validation did not improve in the last 50 records. Best score: 27406.602. Signaling Trainer to stop.
```

Example 45 (unknown):
```unknown
###### Glia ######
(1759, 508040)
```

Example 46 (python):
```python
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/scvi/data/_utils.py:168: UserWarning: Category 5 in adata.obs['batch_key'] has fewer than 3 cells. Models may not train properly.
  warnings.warn(
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/scvi/data/_utils.py:168: UserWarning: Category 7 in adata.obs['region'] has fewer than 3 cells. Models may not train properly.
  warnings.warn(
```

Example 47 (unknown):
```unknown
Anndata setup with scvi-tools version 0.19.0.
```

Example 48 (unknown):
```unknown
Setup via `PEAKVI.setup_anndata` with arguments:
```

Example 49 (unknown):
```unknown
{
│   'batch_key': None,
│   'labels_key': None,
│   'categorical_covariate_keys': ['batch_key', 'region'],
│   'continuous_covariate_keys': None,
│   'layer': None
}
```

Example 50 (unknown):
```unknown
Summary Statistics          
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃     Summary Stat Key     ┃ Value  ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│         n_batch          │   1    │
│         n_cells          │  1759  │
│ n_extra_categorical_covs │   2    │
│ n_extra_continuous_covs  │   0    │
│         n_labels         │   1    │
│          n_vars          │ 508040 │
└──────────────────────────┴────────┘
```

Example 51 (python):
```python
Data Registry                             
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃      Registry Key      ┃            scvi-tools Location             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│           X            │                  adata.X                   │
│         batch          │          adata.obs['_scvi_batch']          │
│ extra_categorical_covs │ adata.obsm['_scvi_extra_categorical_covs'] │
│         labels         │         adata.obs['_scvi_labels']          │
└────────────────────────┴────────────────────────────────────────────┘
```

Example 52 (python):
```python
batch State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃     Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_batch'] │     0      │          0          │
└──────────────────────────┴────────────┴─────────────────────┘
```

Example 53 (python):
```python
labels State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 54 (python):
```python
extra_categorical_covs State Registry                       
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location     ┃           Categories            ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['batch_key'] │     C82_Nuclei_Multiome-v1      │          0          │
│                        │     C85_Nuclei_Multiome-v1      │          1          │
│                        │     C87_Nuclei_Multiome-v1      │          2          │
│                        │     C92_Nuclei_Multiome-v1      │          3          │
│                        │     C98_Nuclei_Multiome-v1      │          4          │
│                        │     C104_Nuclei_Multiome-v1     │          5          │
│                        │    Hst33_Nuclei_Multiome-v1     │          6          │
│                        │    Hst36_Nuclei_Multiome-v1     │          7          │
│                        │    Hst39_Nuclei_Multiome-v1     │          8          │
│                        │    Hst40_Nuclei_Multiome-v1     │          9          │
│                        │    Hst41_Nuclei_Multiome-v1     │         10          │
│                        │                                 │                     │
│  adata.obs['region']   │              aorta              │          0          │
│                        │              atria              │          1          │
│                        │          atrial septum          │          2          │
│                        │        ductus arteriosus        │          3          │
│                        │       heart without node        │          4          │
│                        │         left ventricle          │          5          │
│                        │              node               │          6          │
│                        │          outflow tract          │          7          │
│                        │           pericardium           │          8          │
│                        │  pulmonary arches and branches  │          9          │
│                        │         right ventricle         │         10          │
│                        │ right ventricle and papillaries │         11          │
│                        │          whole sample           │         12          │
│                        │                                 │                     │
└────────────────────────┴─────────────────────────────────┴─────────────────────┘
```

Example 55 (unknown):
```unknown
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/pytorch_lightning/trainer/configuration_validator.py:267: LightningDeprecationWarning: The `Callback.on_epoch_end` hook was deprecated in v1.6 and will be removed in v1.8. Please use `Callback.on_<train/validation/test>_epoch_end` instead.
  rank_zero_deprecation(
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
```

Example 56 (unknown):
```unknown
Epoch 103/500:  21%|████████████████████████████▊                                                                                                               | 103/500 [10:00<38:33,  5.83s/it, loss=3.11e+08, v_num=1]
Monitored metric reconstruction_loss_validation did not improve in the last 50 records. Best score: 27903.340. Signaling Trainer to stop.
```

Example 57 (unknown):
```unknown
###### Neurons ######
(1101, 508040)
```

Example 58 (python):
```python
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/scvi/data/_utils.py:168: UserWarning: Category 5 in adata.obs['batch_key'] has fewer than 3 cells. Models may not train properly.
  warnings.warn(
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/scvi/data/_utils.py:168: UserWarning: Category 9 in adata.obs['region'] has fewer than 3 cells. Models may not train properly.
  warnings.warn(
```

Example 59 (unknown):
```unknown
Anndata setup with scvi-tools version 0.19.0.
```

Example 60 (unknown):
```unknown
Setup via `PEAKVI.setup_anndata` with arguments:
```

Example 61 (unknown):
```unknown
{
│   'batch_key': None,
│   'labels_key': None,
│   'categorical_covariate_keys': ['batch_key', 'region'],
│   'continuous_covariate_keys': None,
│   'layer': None
}
```

Example 62 (unknown):
```unknown
Summary Statistics          
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃     Summary Stat Key     ┃ Value  ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│         n_batch          │   1    │
│         n_cells          │  1101  │
│ n_extra_categorical_covs │   2    │
│ n_extra_continuous_covs  │   0    │
│         n_labels         │   1    │
│          n_vars          │ 508040 │
└──────────────────────────┴────────┘
```

Example 63 (python):
```python
Data Registry                             
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃      Registry Key      ┃            scvi-tools Location             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│           X            │                  adata.X                   │
│         batch          │          adata.obs['_scvi_batch']          │
│ extra_categorical_covs │ adata.obsm['_scvi_extra_categorical_covs'] │
│         labels         │         adata.obs['_scvi_labels']          │
└────────────────────────┴────────────────────────────────────────────┘
```

Example 64 (python):
```python
batch State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃     Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_batch'] │     0      │          0          │
└──────────────────────────┴────────────┴─────────────────────┘
```

Example 65 (python):
```python
labels State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 66 (python):
```python
extra_categorical_covs State Registry                       
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location     ┃           Categories            ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['batch_key'] │     C82_Nuclei_Multiome-v1      │          0          │
│                        │     C85_Nuclei_Multiome-v1      │          1          │
│                        │     C87_Nuclei_Multiome-v1      │          2          │
│                        │     C92_Nuclei_Multiome-v1      │          3          │
│                        │     C98_Nuclei_Multiome-v1      │          4          │
│                        │     C104_Nuclei_Multiome-v1     │          5          │
│                        │    Hst33_Nuclei_Multiome-v1     │          6          │
│                        │    Hst36_Nuclei_Multiome-v1     │          7          │
│                        │    Hst39_Nuclei_Multiome-v1     │          8          │
│                        │    Hst40_Nuclei_Multiome-v1     │          9          │
│                        │    Hst41_Nuclei_Multiome-v1     │         10          │
│                        │                                 │                     │
│  adata.obs['region']   │              aorta              │          0          │
│                        │              apex               │          1          │
│                        │              atria              │          2          │
│                        │          atrial septum          │          3          │
│                        │        ductus arteriosus        │          4          │
│                        │       heart without node        │          5          │
│                        │              node               │          6          │
│                        │           pericardium           │          7          │
│                        │  pulmonary arches and branches  │          8          │
│                        │ right ventricle and papillaries │          9          │
│                        │          whole sample           │         10          │
│                        │                                 │                     │
└────────────────────────┴─────────────────────────────────┴─────────────────────┘
```

Example 67 (unknown):
```unknown
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/pytorch_lightning/trainer/configuration_validator.py:267: LightningDeprecationWarning: The `Callback.on_epoch_end` hook was deprecated in v1.6 and will be removed in v1.8. Please use `Callback.on_<train/validation/test>_epoch_end` instead.
  rank_zero_deprecation(
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py:1892: PossibleUserWarning: The number of training batches (8) is smaller than the logging interval Trainer(log_every_n_steps=10). Set a lower value for log_every_n_steps if you want to see logs for the training epoch.
  rank_zero_warn(
```

Example 68 (unknown):
```unknown
Epoch 101/500:  20%|████████████████████████████▎                                                                                                               | 101/500 [06:13<24:34,  3.69s/it, loss=4.52e+08, v_num=1]
Monitored metric reconstruction_loss_validation did not improve in the last 50 records. Best score: 42978.691. Signaling Trainer to stop.
```

Example 69 (unknown):
```unknown
###### EndocardialCells ######
(1410, 508040)
```

Example 70 (unknown):
```unknown
Anndata setup with scvi-tools version 0.19.0.
```

Example 71 (unknown):
```unknown
Setup via `PEAKVI.setup_anndata` with arguments:
```

Example 72 (unknown):
```unknown
{
│   'batch_key': None,
│   'labels_key': None,
│   'categorical_covariate_keys': ['batch_key', 'region'],
│   'continuous_covariate_keys': None,
│   'layer': None
}
```

Example 73 (unknown):
```unknown
Summary Statistics          
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃     Summary Stat Key     ┃ Value  ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│         n_batch          │   1    │
│         n_cells          │  1410  │
│ n_extra_categorical_covs │   2    │
│ n_extra_continuous_covs  │   0    │
│         n_labels         │   1    │
│          n_vars          │ 508040 │
└──────────────────────────┴────────┘
```

Example 74 (python):
```python
Data Registry                             
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃      Registry Key      ┃            scvi-tools Location             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│           X            │                  adata.X                   │
│         batch          │          adata.obs['_scvi_batch']          │
│ extra_categorical_covs │ adata.obsm['_scvi_extra_categorical_covs'] │
│         labels         │         adata.obs['_scvi_labels']          │
└────────────────────────┴────────────────────────────────────────────┘
```

Example 75 (python):
```python
batch State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃     Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_batch'] │     0      │          0          │
└──────────────────────────┴────────────┴─────────────────────┘
```

Example 76 (python):
```python
labels State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 77 (python):
```python
extra_categorical_covs State Registry                       
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location     ┃           Categories            ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['batch_key'] │     C82_Nuclei_Multiome-v1      │          0          │
│                        │     C85_Nuclei_Multiome-v1      │          1          │
│                        │     C87_Nuclei_Multiome-v1      │          2          │
│                        │     C92_Nuclei_Multiome-v1      │          3          │
│                        │     C98_Nuclei_Multiome-v1      │          4          │
│                        │     C104_Nuclei_Multiome-v1     │          5          │
│                        │    Hst33_Nuclei_Multiome-v1     │          6          │
│                        │    Hst36_Nuclei_Multiome-v1     │          7          │
│                        │    Hst39_Nuclei_Multiome-v1     │          8          │
│                        │    Hst40_Nuclei_Multiome-v1     │          9          │
│                        │    Hst41_Nuclei_Multiome-v1     │         10          │
│                        │                                 │                     │
│  adata.obs['region']   │              aorta              │          0          │
│                        │              apex               │          1          │
│                        │              atria              │          2          │
│                        │          atrial septum          │          3          │
│                        │       heart without node        │          4          │
│                        │         left ventricle          │          5          │
│                        │              node               │          6          │
│                        │          outflow tract          │          7          │
│                        │         right ventricle         │          8          │
│                        │ right ventricle and papillaries │          9          │
│                        │          whole sample           │         10          │
│                        │                                 │                     │
└────────────────────────┴─────────────────────────────────┴─────────────────────┘
```

Example 78 (unknown):
```unknown
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/pytorch_lightning/trainer/configuration_validator.py:267: LightningDeprecationWarning: The `Callback.on_epoch_end` hook was deprecated in v1.6 and will be removed in v1.8. Please use `Callback.on_<train/validation/test>_epoch_end` instead.
  rank_zero_deprecation(
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
```

Example 79 (unknown):
```unknown
Epoch 106/500:  21%|█████████████████████████████▉                                                                                                               | 106/500 [08:17<30:49,  4.69s/it, loss=3.7e+08, v_num=1]
Monitored metric reconstruction_loss_validation did not improve in the last 50 records. Best score: 34130.070. Signaling Trainer to stop.
```

Example 80 (unknown):
```unknown
###### VentricularCardiomyocytes_plusCCS ######
(6531, 508040)
```

Example 81 (unknown):
```unknown
Anndata setup with scvi-tools version 0.19.0.
```

Example 82 (unknown):
```unknown
Setup via `PEAKVI.setup_anndata` with arguments:
```

Example 83 (unknown):
```unknown
{
│   'batch_key': None,
│   'labels_key': None,
│   'categorical_covariate_keys': ['batch_key', 'region'],
│   'continuous_covariate_keys': None,
│   'layer': None
}
```

Example 84 (unknown):
```unknown
Summary Statistics          
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃     Summary Stat Key     ┃ Value  ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│         n_batch          │   1    │
│         n_cells          │  6531  │
│ n_extra_categorical_covs │   2    │
│ n_extra_continuous_covs  │   0    │
│         n_labels         │   1    │
│          n_vars          │ 508040 │
└──────────────────────────┴────────┘
```

Example 85 (python):
```python
Data Registry                             
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃      Registry Key      ┃            scvi-tools Location             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│           X            │                  adata.X                   │
│         batch          │          adata.obs['_scvi_batch']          │
│ extra_categorical_covs │ adata.obsm['_scvi_extra_categorical_covs'] │
│         labels         │         adata.obs['_scvi_labels']          │
└────────────────────────┴────────────────────────────────────────────┘
```

Example 86 (python):
```python
batch State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃     Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_batch'] │     0      │          0          │
└──────────────────────────┴────────────┴─────────────────────┘
```

Example 87 (python):
```python
labels State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 88 (python):
```python
extra_categorical_covs State Registry                       
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location     ┃           Categories            ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['batch_key'] │     C82_Nuclei_Multiome-v1      │          0          │
│                        │     C85_Nuclei_Multiome-v1      │          1          │
│                        │     C87_Nuclei_Multiome-v1      │          2          │
│                        │     C92_Nuclei_Multiome-v1      │          3          │
│                        │     C98_Nuclei_Multiome-v1      │          4          │
│                        │     C104_Nuclei_Multiome-v1     │          5          │
│                        │    Hst33_Nuclei_Multiome-v1     │          6          │
│                        │    Hst36_Nuclei_Multiome-v1     │          7          │
│                        │    Hst39_Nuclei_Multiome-v1     │          8          │
│                        │    Hst40_Nuclei_Multiome-v1     │          9          │
│                        │    Hst41_Nuclei_Multiome-v1     │         10          │
│                        │                                 │                     │
│  adata.obs['region']   │              apex               │          0          │
│                        │              atria              │          1          │
│                        │          atrial septum          │          2          │
│                        │       heart without node        │          3          │
│                        │         left ventricle          │          4          │
│                        │              node               │          5          │
│                        │          outflow tract          │          6          │
│                        │         right ventricle         │          7          │
│                        │ right ventricle and papillaries │          8          │
│                        │          whole sample           │          9          │
│                        │                                 │                     │
└────────────────────────┴─────────────────────────────────┴─────────────────────┘
```

Example 89 (unknown):
```unknown
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/pytorch_lightning/trainer/configuration_validator.py:267: LightningDeprecationWarning: The `Callback.on_epoch_end` hook was deprecated in v1.6 and will be removed in v1.8. Please use `Callback.on_<train/validation/test>_epoch_end` instead.
  rank_zero_deprecation(
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
```

Example 90 (unknown):
```unknown
Epoch 88/500:  18%|████████████████████████▋                                                                                                                   | 88/500 [32:04<2:30:11, 21.87s/it, loss=4.62e+08, v_num=1]
Monitored metric reconstruction_loss_validation did not improve in the last 50 records. Best score: 34277.566. Signaling Trainer to stop.
```

Example 91 (unknown):
```unknown
###### PericardialCells ######
(2187, 508040)
```

Example 92 (python):
```python
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/scvi/data/_utils.py:168: UserWarning: Category 3 in adata.obs['batch_key'] has fewer than 3 cells. Models may not train properly.
  warnings.warn(
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/scvi/data/_utils.py:168: UserWarning: Category 0 in adata.obs['region'] has fewer than 3 cells. Models may not train properly.
  warnings.warn(
```

Example 93 (unknown):
```unknown
Anndata setup with scvi-tools version 0.19.0.
```

Example 94 (unknown):
```unknown
Setup via `PEAKVI.setup_anndata` with arguments:
```

Example 95 (unknown):
```unknown
{
│   'batch_key': None,
│   'labels_key': None,
│   'categorical_covariate_keys': ['batch_key', 'region'],
│   'continuous_covariate_keys': None,
│   'layer': None
}
```

Example 96 (unknown):
```unknown
Summary Statistics          
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃     Summary Stat Key     ┃ Value  ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│         n_batch          │   1    │
│         n_cells          │  2187  │
│ n_extra_categorical_covs │   2    │
│ n_extra_continuous_covs  │   0    │
│         n_labels         │   1    │
│          n_vars          │ 508040 │
└──────────────────────────┴────────┘
```

Example 97 (python):
```python
Data Registry                             
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃      Registry Key      ┃            scvi-tools Location             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│           X            │                  adata.X                   │
│         batch          │          adata.obs['_scvi_batch']          │
│ extra_categorical_covs │ adata.obsm['_scvi_extra_categorical_covs'] │
│         labels         │         adata.obs['_scvi_labels']          │
└────────────────────────┴────────────────────────────────────────────┘
```

Example 98 (python):
```python
batch State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃     Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_batch'] │     0      │          0          │
└──────────────────────────┴────────────┴─────────────────────┘
```

Example 99 (python):
```python
labels State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 100 (python):
```python
extra_categorical_covs State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location     ┃          Categories           ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['batch_key'] │    C87_Nuclei_Multiome-v1     │          0          │
│                        │    C92_Nuclei_Multiome-v1     │          1          │
│                        │    C98_Nuclei_Multiome-v1     │          2          │
│                        │    C104_Nuclei_Multiome-v1    │          3          │
│                        │   Hst39_Nuclei_Multiome-v1    │          4          │
│                        │   Hst40_Nuclei_Multiome-v1    │          5          │
│                        │   Hst41_Nuclei_Multiome-v1    │          6          │
│                        │                               │                     │
│  adata.obs['region']   │             aorta             │          0          │
│                        │         atrial septum         │          1          │
│                        │       ductus arteriosus       │          2          │
│                        │      heart without node       │          3          │
│                        │         outflow tract         │          4          │
│                        │          pericardium          │          5          │
│                        │ pulmonary arches and branches │          6          │
│                        │         whole sample          │          7          │
│                        │                               │                     │
└────────────────────────┴───────────────────────────────┴─────────────────────┘
```

Example 101 (unknown):
```unknown
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/pytorch_lightning/trainer/configuration_validator.py:267: LightningDeprecationWarning: The `Callback.on_epoch_end` hook was deprecated in v1.6 and will be removed in v1.8. Please use `Callback.on_<train/validation/test>_epoch_end` instead.
  rank_zero_deprecation(
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
```

Example 102 (unknown):
```unknown
Epoch 82/500:  16%|███████████████████████▎                                                                                                                      | 82/500 [09:55<50:37,  7.27s/it, loss=3.54e+08, v_num=1]
Monitored metric reconstruction_loss_validation did not improve in the last 50 records. Best score: 30433.793. Signaling Trainer to stop.
```

Example 103 (unknown):
```unknown
###### LymphoidCells ######
(7006, 508040)
```

Example 104 (python):
```python
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/scvi/data/_utils.py:168: UserWarning: Category 4 in adata.obs['batch_key'] has fewer than 3 cells. Models may not train properly.
  warnings.warn(
```

Example 105 (unknown):
```unknown
Anndata setup with scvi-tools version 0.19.0.
```

Example 106 (unknown):
```unknown
Setup via `PEAKVI.setup_anndata` with arguments:
```

Example 107 (unknown):
```unknown
{
│   'batch_key': None,
│   'labels_key': None,
│   'categorical_covariate_keys': ['batch_key', 'region'],
│   'continuous_covariate_keys': None,
│   'layer': None
}
```

Example 108 (unknown):
```unknown
Summary Statistics          
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃     Summary Stat Key     ┃ Value  ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│         n_batch          │   1    │
│         n_cells          │  7006  │
│ n_extra_categorical_covs │   2    │
│ n_extra_continuous_covs  │   0    │
│         n_labels         │   1    │
│          n_vars          │ 508040 │
└──────────────────────────┴────────┘
```

Example 109 (python):
```python
Data Registry                             
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃      Registry Key      ┃            scvi-tools Location             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│           X            │                  adata.X                   │
│         batch          │          adata.obs['_scvi_batch']          │
│ extra_categorical_covs │ adata.obsm['_scvi_extra_categorical_covs'] │
│         labels         │         adata.obs['_scvi_labels']          │
└────────────────────────┴────────────────────────────────────────────┘
```

Example 110 (python):
```python
batch State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃     Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_batch'] │     0      │          0          │
└──────────────────────────┴────────────┴─────────────────────┘
```

Example 111 (python):
```python
labels State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 112 (python):
```python
extra_categorical_covs State Registry                       
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location     ┃           Categories            ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['batch_key'] │     C82_Nuclei_Multiome-v1      │          0          │
│                        │     C85_Nuclei_Multiome-v1      │          1          │
│                        │     C87_Nuclei_Multiome-v1      │          2          │
│                        │     C92_Nuclei_Multiome-v1      │          3          │
│                        │     C98_Nuclei_Multiome-v1      │          4          │
│                        │    Hst33_Nuclei_Multiome-v1     │          5          │
│                        │    Hst36_Nuclei_Multiome-v1     │          6          │
│                        │    Hst39_Nuclei_Multiome-v1     │          7          │
│                        │    Hst40_Nuclei_Multiome-v1     │          8          │
│                        │    Hst41_Nuclei_Multiome-v1     │          9          │
│                        │                                 │                     │
│  adata.obs['region']   │              aorta              │          0          │
│                        │              atria              │          1          │
│                        │          atrial septum          │          2          │
│                        │        ductus arteriosus        │          3          │
│                        │       heart without node        │          4          │
│                        │         left ventricle          │          5          │
│                        │              node               │          6          │
│                        │           pericardium           │          7          │
│                        │  pulmonary arches and branches  │          8          │
│                        │         right ventricle         │          9          │
│                        │ right ventricle and papillaries │         10          │
│                        │          whole sample           │         11          │
│                        │                                 │                     │
└────────────────────────┴─────────────────────────────────┴─────────────────────┘
```

Example 113 (unknown):
```unknown
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/pytorch_lightning/trainer/configuration_validator.py:267: LightningDeprecationWarning: The `Callback.on_epoch_end` hook was deprecated in v1.6 and will be removed in v1.8. Please use `Callback.on_<train/validation/test>_epoch_end` instead.
  rank_zero_deprecation(
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
```

Example 114 (unknown):
```unknown
Epoch 81/500:  16%|██████████████████████▋                                                                                                                     | 81/500 [31:31<2:43:04, 23.35s/it, loss=2.75e+08, v_num=1]
Monitored metric reconstruction_loss_validation did not improve in the last 50 records. Best score: 21196.828. Signaling Trainer to stop.
```

Example 115 (unknown):
```unknown
###### EpicardialCells ######
(1340, 508040)
```

Example 116 (python):
```python
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/scvi/data/_utils.py:168: UserWarning: Category 12 in adata.obs['region'] has fewer than 3 cells. Models may not train properly.
  warnings.warn(
```

Example 117 (unknown):
```unknown
Anndata setup with scvi-tools version 0.19.0.
```

Example 118 (unknown):
```unknown
Setup via `PEAKVI.setup_anndata` with arguments:
```

Example 119 (unknown):
```unknown
{
│   'batch_key': None,
│   'labels_key': None,
│   'categorical_covariate_keys': ['batch_key', 'region'],
│   'continuous_covariate_keys': None,
│   'layer': None
}
```

Example 120 (unknown):
```unknown
Summary Statistics          
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃     Summary Stat Key     ┃ Value  ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│         n_batch          │   1    │
│         n_cells          │  1340  │
│ n_extra_categorical_covs │   2    │
│ n_extra_continuous_covs  │   0    │
│         n_labels         │   1    │
│          n_vars          │ 508040 │
└──────────────────────────┴────────┘
```

Example 121 (python):
```python
Data Registry                             
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃      Registry Key      ┃            scvi-tools Location             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│           X            │                  adata.X                   │
│         batch          │          adata.obs['_scvi_batch']          │
│ extra_categorical_covs │ adata.obsm['_scvi_extra_categorical_covs'] │
│         labels         │         adata.obs['_scvi_labels']          │
└────────────────────────┴────────────────────────────────────────────┘
```

Example 122 (python):
```python
batch State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃     Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_batch'] │     0      │          0          │
└──────────────────────────┴────────────┴─────────────────────┘
```

Example 123 (python):
```python
labels State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 124 (python):
```python
extra_categorical_covs State Registry                       
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location     ┃           Categories            ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['batch_key'] │     C82_Nuclei_Multiome-v1      │          0          │
│                        │     C85_Nuclei_Multiome-v1      │          1          │
│                        │     C87_Nuclei_Multiome-v1      │          2          │
│                        │     C92_Nuclei_Multiome-v1      │          3          │
│                        │     C98_Nuclei_Multiome-v1      │          4          │
│                        │     C104_Nuclei_Multiome-v1     │          5          │
│                        │    Hst33_Nuclei_Multiome-v1     │          6          │
│                        │    Hst36_Nuclei_Multiome-v1     │          7          │
│                        │    Hst39_Nuclei_Multiome-v1     │          8          │
│                        │    Hst40_Nuclei_Multiome-v1     │          9          │
│                        │    Hst41_Nuclei_Multiome-v1     │         10          │
│                        │                                 │                     │
│  adata.obs['region']   │              aorta              │          0          │
│                        │              apex               │          1          │
│                        │              atria              │          2          │
│                        │          atrial septum          │          3          │
│                        │        ductus arteriosus        │          4          │
│                        │       heart without node        │          5          │
│                        │         left ventricle          │          6          │
│                        │              node               │          7          │
│                        │          outflow tract          │          8          │
│                        │           pericardium           │          9          │
│                        │  pulmonary arches and branches  │         10          │
│                        │         right ventricle         │         11          │
│                        │ right ventricle and papillaries │         12          │
│                        │          whole sample           │         13          │
│                        │                                 │                     │
└────────────────────────┴─────────────────────────────────┴─────────────────────┘
```

Example 125 (unknown):
```unknown
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/pytorch_lightning/trainer/configuration_validator.py:267: LightningDeprecationWarning: The `Callback.on_epoch_end` hook was deprecated in v1.6 and will be removed in v1.8. Please use `Callback.on_<train/validation/test>_epoch_end` instead.
  rank_zero_deprecation(
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
```

Example 126 (unknown):
```unknown
Epoch 113/500:  23%|███████████████████████████████▋                                                                                                            | 113/500 [08:31<29:13,  4.53s/it, loss=4.23e+08, v_num=1]
Monitored metric reconstruction_loss_validation did not improve in the last 50 records. Best score: 37883.770. Signaling Trainer to stop.
```

Example 127 (unknown):
```unknown
###### MuralCells ######
(3103, 508040)
```

Example 128 (python):
```python
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/scvi/data/_utils.py:168: UserWarning: Category 1 in adata.obs['region'] has fewer than 3 cells. Models may not train properly.
  warnings.warn(
```

Example 129 (unknown):
```unknown
Anndata setup with scvi-tools version 0.19.0.
```

Example 130 (unknown):
```unknown
Setup via `PEAKVI.setup_anndata` with arguments:
```

Example 131 (unknown):
```unknown
{
│   'batch_key': None,
│   'labels_key': None,
│   'categorical_covariate_keys': ['batch_key', 'region'],
│   'continuous_covariate_keys': None,
│   'layer': None
}
```

Example 132 (unknown):
```unknown
Summary Statistics          
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃     Summary Stat Key     ┃ Value  ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│         n_batch          │   1    │
│         n_cells          │  3103  │
│ n_extra_categorical_covs │   2    │
│ n_extra_continuous_covs  │   0    │
│         n_labels         │   1    │
│          n_vars          │ 508040 │
└──────────────────────────┴────────┘
```

Example 133 (python):
```python
Data Registry                             
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃      Registry Key      ┃            scvi-tools Location             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│           X            │                  adata.X                   │
│         batch          │          adata.obs['_scvi_batch']          │
│ extra_categorical_covs │ adata.obsm['_scvi_extra_categorical_covs'] │
│         labels         │         adata.obs['_scvi_labels']          │
└────────────────────────┴────────────────────────────────────────────┘
```

Example 134 (python):
```python
batch State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃     Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_batch'] │     0      │          0          │
└──────────────────────────┴────────────┴─────────────────────┘
```

Example 135 (python):
```python
labels State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 136 (python):
```python
extra_categorical_covs State Registry                       
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location     ┃           Categories            ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['batch_key'] │     C82_Nuclei_Multiome-v1      │          0          │
│                        │     C85_Nuclei_Multiome-v1      │          1          │
│                        │     C87_Nuclei_Multiome-v1      │          2          │
│                        │     C92_Nuclei_Multiome-v1      │          3          │
│                        │     C98_Nuclei_Multiome-v1      │          4          │
│                        │     C104_Nuclei_Multiome-v1     │          5          │
│                        │    Hst33_Nuclei_Multiome-v1     │          6          │
│                        │    Hst36_Nuclei_Multiome-v1     │          7          │
│                        │    Hst39_Nuclei_Multiome-v1     │          8          │
│                        │    Hst40_Nuclei_Multiome-v1     │          9          │
│                        │    Hst41_Nuclei_Multiome-v1     │         10          │
│                        │                                 │                     │
│  adata.obs['region']   │              aorta              │          0          │
│                        │              apex               │          1          │
│                        │              atria              │          2          │
│                        │          atrial septum          │          3          │
│                        │        ductus arteriosus        │          4          │
│                        │       heart without node        │          5          │
│                        │         left ventricle          │          6          │
│                        │              node               │          7          │
│                        │          outflow tract          │          8          │
│                        │           pericardium           │          9          │
│                        │  pulmonary arches and branches  │         10          │
│                        │         right ventricle         │         11          │
│                        │ right ventricle and papillaries │         12          │
│                        │          whole sample           │         13          │
│                        │                                 │                     │
└────────────────────────┴─────────────────────────────────┴─────────────────────┘
```

Example 137 (unknown):
```unknown
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/pytorch_lightning/trainer/configuration_validator.py:267: LightningDeprecationWarning: The `Callback.on_epoch_end` hook was deprecated in v1.6 and will be removed in v1.8. Please use `Callback.on_<train/validation/test>_epoch_end` instead.
  rank_zero_deprecation(
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
```

Example 138 (unknown):
```unknown
Epoch 90/500:  18%|█████████████████████████▏                                                                                                                  | 90/500 [15:26<1:10:22, 10.30s/it, loss=3.15e+08, v_num=1]
Monitored metric reconstruction_loss_validation did not improve in the last 50 records. Best score: 27358.064. Signaling Trainer to stop.
```

Example 139 (unknown):
```unknown
###### Fibroblasts ######
(6969, 508040)
```

Example 140 (unknown):
```unknown
Anndata setup with scvi-tools version 0.19.0.
```

Example 141 (unknown):
```unknown
Setup via `PEAKVI.setup_anndata` with arguments:
```

Example 142 (unknown):
```unknown
{
│   'batch_key': None,
│   'labels_key': None,
│   'categorical_covariate_keys': ['batch_key', 'region'],
│   'continuous_covariate_keys': None,
│   'layer': None
}
```

Example 143 (unknown):
```unknown
Summary Statistics          
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃     Summary Stat Key     ┃ Value  ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│         n_batch          │   1    │
│         n_cells          │  6969  │
│ n_extra_categorical_covs │   2    │
│ n_extra_continuous_covs  │   0    │
│         n_labels         │   1    │
│          n_vars          │ 508040 │
└──────────────────────────┴────────┘
```

Example 144 (python):
```python
Data Registry                             
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃      Registry Key      ┃            scvi-tools Location             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│           X            │                  adata.X                   │
│         batch          │          adata.obs['_scvi_batch']          │
│ extra_categorical_covs │ adata.obsm['_scvi_extra_categorical_covs'] │
│         labels         │         adata.obs['_scvi_labels']          │
└────────────────────────┴────────────────────────────────────────────┘
```

Example 145 (python):
```python
batch State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃     Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_batch'] │     0      │          0          │
└──────────────────────────┴────────────┴─────────────────────┘
```

Example 146 (python):
```python
labels State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 147 (python):
```python
extra_categorical_covs State Registry                       
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location     ┃           Categories            ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['batch_key'] │     C82_Nuclei_Multiome-v1      │          0          │
│                        │     C85_Nuclei_Multiome-v1      │          1          │
│                        │     C87_Nuclei_Multiome-v1      │          2          │
│                        │     C92_Nuclei_Multiome-v1      │          3          │
│                        │     C98_Nuclei_Multiome-v1      │          4          │
│                        │     C104_Nuclei_Multiome-v1     │          5          │
│                        │    Hst33_Nuclei_Multiome-v1     │          6          │
│                        │    Hst36_Nuclei_Multiome-v1     │          7          │
│                        │    Hst39_Nuclei_Multiome-v1     │          8          │
│                        │    Hst40_Nuclei_Multiome-v1     │          9          │
│                        │    Hst41_Nuclei_Multiome-v1     │         10          │
│                        │                                 │                     │
│  adata.obs['region']   │              aorta              │          0          │
│                        │              apex               │          1          │
│                        │              atria              │          2          │
│                        │          atrial septum          │          3          │
│                        │        ductus arteriosus        │          4          │
│                        │       heart without node        │          5          │
│                        │         left ventricle          │          6          │
│                        │              node               │          7          │
│                        │          outflow tract          │          8          │
│                        │           pericardium           │          9          │
│                        │  pulmonary arches and branches  │         10          │
│                        │         right ventricle         │         11          │
│                        │ right ventricle and papillaries │         12          │
│                        │          whole sample           │         13          │
│                        │                                 │                     │
└────────────────────────┴─────────────────────────────────┴─────────────────────┘
```

Example 148 (unknown):
```unknown
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/pytorch_lightning/trainer/configuration_validator.py:267: LightningDeprecationWarning: The `Callback.on_epoch_end` hook was deprecated in v1.6 and will be removed in v1.8. Please use `Callback.on_<train/validation/test>_epoch_end` instead.
  rank_zero_deprecation(
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
```

Example 149 (unknown):
```unknown
Epoch 73/500:  15%|████████████████████▍                                                                                                                       | 73/500 [28:09<2:44:43, 23.15s/it, loss=4.02e+08, v_num=1]
Monitored metric reconstruction_loss_validation did not improve in the last 50 records. Best score: 29998.793. Signaling Trainer to stop.
```

Example 150 (unknown):
```unknown
###### MyeloidCells ######
(4321, 508040)
```

Example 151 (unknown):
```unknown
Anndata setup with scvi-tools version 0.19.0.
```

Example 152 (unknown):
```unknown
Setup via `PEAKVI.setup_anndata` with arguments:
```

Example 153 (unknown):
```unknown
{
│   'batch_key': None,
│   'labels_key': None,
│   'categorical_covariate_keys': ['batch_key', 'region'],
│   'continuous_covariate_keys': None,
│   'layer': None
}
```

Example 154 (unknown):
```unknown
Summary Statistics          
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃     Summary Stat Key     ┃ Value  ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│         n_batch          │   1    │
│         n_cells          │  4321  │
│ n_extra_categorical_covs │   2    │
│ n_extra_continuous_covs  │   0    │
│         n_labels         │   1    │
│          n_vars          │ 508040 │
└──────────────────────────┴────────┘
```

Example 155 (python):
```python
Data Registry                             
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃      Registry Key      ┃            scvi-tools Location             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│           X            │                  adata.X                   │
│         batch          │          adata.obs['_scvi_batch']          │
│ extra_categorical_covs │ adata.obsm['_scvi_extra_categorical_covs'] │
│         labels         │         adata.obs['_scvi_labels']          │
└────────────────────────┴────────────────────────────────────────────┘
```

Example 156 (python):
```python
batch State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃     Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_batch'] │     0      │          0          │
└──────────────────────────┴────────────┴─────────────────────┘
```

Example 157 (python):
```python
labels State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 158 (python):
```python
extra_categorical_covs State Registry                       
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location     ┃           Categories            ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['batch_key'] │     C82_Nuclei_Multiome-v1      │          0          │
│                        │     C85_Nuclei_Multiome-v1      │          1          │
│                        │     C87_Nuclei_Multiome-v1      │          2          │
│                        │     C92_Nuclei_Multiome-v1      │          3          │
│                        │     C98_Nuclei_Multiome-v1      │          4          │
│                        │     C104_Nuclei_Multiome-v1     │          5          │
│                        │    Hst33_Nuclei_Multiome-v1     │          6          │
│                        │    Hst36_Nuclei_Multiome-v1     │          7          │
│                        │    Hst39_Nuclei_Multiome-v1     │          8          │
│                        │    Hst40_Nuclei_Multiome-v1     │          9          │
│                        │    Hst41_Nuclei_Multiome-v1     │         10          │
│                        │                                 │                     │
│  adata.obs['region']   │              aorta              │          0          │
│                        │              apex               │          1          │
│                        │              atria              │          2          │
│                        │          atrial septum          │          3          │
│                        │        ductus arteriosus        │          4          │
│                        │       heart without node        │          5          │
│                        │         left ventricle          │          6          │
│                        │              node               │          7          │
│                        │          outflow tract          │          8          │
│                        │           pericardium           │          9          │
│                        │  pulmonary arches and branches  │         10          │
│                        │         right ventricle         │         11          │
│                        │ right ventricle and papillaries │         12          │
│                        │          whole sample           │         13          │
│                        │                                 │                     │
└────────────────────────┴─────────────────────────────────┴─────────────────────┘
```

Example 159 (unknown):
```unknown
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/pytorch_lightning/trainer/configuration_validator.py:267: LightningDeprecationWarning: The `Callback.on_epoch_end` hook was deprecated in v1.6 and will be removed in v1.8. Please use `Callback.on_<train/validation/test>_epoch_end` instead.
  rank_zero_deprecation(
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
```

Example 160 (unknown):
```unknown
Epoch 84/500:  17%|███████████████████████▌                                                                                                                    | 84/500 [20:24<1:41:05, 14.58s/it, loss=3.46e+08, v_num=1]
Monitored metric reconstruction_loss_validation did not improve in the last 50 records. Best score: 29149.953. Signaling Trainer to stop.
```

Example 161 (unknown):
```unknown
###### LymphaticEndothelialCells ######
(1000, 508040)
```

Example 162 (python):
```python
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/scvi/data/_utils.py:168: UserWarning: Category 5 in adata.obs['batch_key'] has fewer than 3 cells. Models may not train properly.
  warnings.warn(
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/scvi/data/_utils.py:168: UserWarning: Category 1 in adata.obs['region'] has fewer than 3 cells. Models may not train properly.
  warnings.warn(
```

Example 163 (unknown):
```unknown
Anndata setup with scvi-tools version 0.19.0.
```

Example 164 (unknown):
```unknown
Setup via `PEAKVI.setup_anndata` with arguments:
```

Example 165 (unknown):
```unknown
{
│   'batch_key': None,
│   'labels_key': None,
│   'categorical_covariate_keys': ['batch_key', 'region'],
│   'continuous_covariate_keys': None,
│   'layer': None
}
```

Example 166 (unknown):
```unknown
Summary Statistics          
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃     Summary Stat Key     ┃ Value  ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│         n_batch          │   1    │
│         n_cells          │  1000  │
│ n_extra_categorical_covs │   2    │
│ n_extra_continuous_covs  │   0    │
│         n_labels         │   1    │
│          n_vars          │ 508040 │
└──────────────────────────┴────────┘
```

Example 167 (python):
```python
Data Registry                             
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃      Registry Key      ┃            scvi-tools Location             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│           X            │                  adata.X                   │
│         batch          │          adata.obs['_scvi_batch']          │
│ extra_categorical_covs │ adata.obsm['_scvi_extra_categorical_covs'] │
│         labels         │         adata.obs['_scvi_labels']          │
└────────────────────────┴────────────────────────────────────────────┘
```

Example 168 (python):
```python
batch State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃     Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_batch'] │     0      │          0          │
└──────────────────────────┴────────────┴─────────────────────┘
```

Example 169 (python):
```python
labels State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 170 (python):
```python
extra_categorical_covs State Registry                       
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location     ┃           Categories            ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['batch_key'] │     C82_Nuclei_Multiome-v1      │          0          │
│                        │     C85_Nuclei_Multiome-v1      │          1          │
│                        │     C87_Nuclei_Multiome-v1      │          2          │
│                        │     C92_Nuclei_Multiome-v1      │          3          │
│                        │     C98_Nuclei_Multiome-v1      │          4          │
│                        │     C104_Nuclei_Multiome-v1     │          5          │
│                        │    Hst33_Nuclei_Multiome-v1     │          6          │
│                        │    Hst36_Nuclei_Multiome-v1     │          7          │
│                        │    Hst39_Nuclei_Multiome-v1     │          8          │
│                        │    Hst40_Nuclei_Multiome-v1     │          9          │
│                        │    Hst41_Nuclei_Multiome-v1     │         10          │
│                        │                                 │                     │
│  adata.obs['region']   │              aorta              │          0          │
│                        │              apex               │          1          │
│                        │              atria              │          2          │
│                        │          atrial septum          │          3          │
│                        │        ductus arteriosus        │          4          │
│                        │       heart without node        │          5          │
│                        │         left ventricle          │          6          │
│                        │              node               │          7          │
│                        │          outflow tract          │          8          │
│                        │           pericardium           │          9          │
│                        │  pulmonary arches and branches  │         10          │
│                        │         right ventricle         │         11          │
│                        │ right ventricle and papillaries │         12          │
│                        │          whole sample           │         13          │
│                        │                                 │                     │
└────────────────────────┴─────────────────────────────────┴─────────────────────┘
```

Example 171 (unknown):
```unknown
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/pytorch_lightning/trainer/configuration_validator.py:267: LightningDeprecationWarning: The `Callback.on_epoch_end` hook was deprecated in v1.6 and will be removed in v1.8. Please use `Callback.on_<train/validation/test>_epoch_end` instead.
  rank_zero_deprecation(
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py:1892: PossibleUserWarning: The number of training batches (8) is smaller than the logging interval Trainer(log_every_n_steps=10). Set a lower value for log_every_n_steps if you want to see logs for the training epoch.
  rank_zero_warn(
```

Example 172 (unknown):
```unknown
Epoch 105/500:  21%|█████████████████████████████▍                                                                                                              | 105/500 [06:14<23:29,  3.57s/it, loss=3.15e+08, v_num=1]
Monitored metric reconstruction_loss_validation did not improve in the last 50 records. Best score: 30158.045. Signaling Trainer to stop.
```

Example 173 (python):
```python
# NOT adding 'region' as covariate
outdir = '/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/PeakMatrix/mid_grain_mod'

for celltype in set(adata.obs['mid_grain_mod']):
    print(f'###### {celltype} ######')
    
    # subset
    adata_sub = adata[adata.obs['mid_grain_mod']==celltype]
    print(adata_sub.shape)
    adata_sub = adata_sub.copy()
    
    # train
    scvi.model.PEAKVI.setup_anndata(adata_sub, 
                                    categorical_covariate_keys=['batch_key']
                                   )
    pvi = scvi.model.PEAKVI(adata_sub)
    pvi.view_anndata_setup(adata_sub)
    pvi.train()
    
    # add to anndata
    latent = pvi.get_latent_representation()
    adata_sub.obsm["X_PeakVI"] = latent
    
    # save
    adata_sub.write(f'{outdir}/Foetal_{celltype}_Peaks_post-peakVI_subsampled-n1000.h5ad')
    pvi.save(f'{outdir}/Foetal_{celltype}_subsampled-n1000_peakVI-model', overwrite=True)
    
    # Convergency plot
    pvi.history['elbo_train']
    x = np.linspace(1, (len(pvi.history['elbo_train'])), (len(pvi.history['elbo_train'])))
    plt.plot(x, pvi.history['elbo_train'], label="train")
    plt.xlabel('epoch')
    plt.ylabel('elbo_train')
    plt.title(celltype)
    plt.show()
    
    # free up GPU
    del(pvi)
    gc.collect()
    torch.cuda.empty_cache()
```

Example 174 (unknown):
```unknown
###### AtrialCardiomyocytes_plusCCS ######
(4381, 508040)
```

Example 175 (unknown):
```unknown
Anndata setup with scvi-tools version 0.19.0.
```

Example 176 (unknown):
```unknown
Setup via `PEAKVI.setup_anndata` with arguments:
```

Example 177 (unknown):
```unknown
{
│   'batch_key': None,
│   'labels_key': None,
│   'categorical_covariate_keys': ['batch_key'],
│   'continuous_covariate_keys': None,
│   'layer': None
}
```

Example 178 (unknown):
```unknown
Summary Statistics          
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃     Summary Stat Key     ┃ Value  ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│         n_batch          │   1    │
│         n_cells          │  4381  │
│ n_extra_categorical_covs │   1    │
│ n_extra_continuous_covs  │   0    │
│         n_labels         │   1    │
│          n_vars          │ 508040 │
└──────────────────────────┴────────┘
```

Example 179 (python):
```python
Data Registry                             
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃      Registry Key      ┃            scvi-tools Location             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│           X            │                  adata.X                   │
│         batch          │          adata.obs['_scvi_batch']          │
│ extra_categorical_covs │ adata.obsm['_scvi_extra_categorical_covs'] │
│         labels         │         adata.obs['_scvi_labels']          │
└────────────────────────┴────────────────────────────────────────────┘
```

Example 180 (python):
```python
batch State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃     Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_batch'] │     0      │          0          │
└──────────────────────────┴────────────┴─────────────────────┘
```

Example 181 (python):
```python
labels State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 182 (python):
```python
extra_categorical_covs State Registry                   
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location     ┃        Categories        ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['batch_key'] │  C82_Nuclei_Multiome-v1  │          0          │
│                        │  C85_Nuclei_Multiome-v1  │          1          │
│                        │  C87_Nuclei_Multiome-v1  │          2          │
│                        │  C92_Nuclei_Multiome-v1  │          3          │
│                        │  C98_Nuclei_Multiome-v1  │          4          │
│                        │ C104_Nuclei_Multiome-v1  │          5          │
│                        │ Hst33_Nuclei_Multiome-v1 │          6          │
│                        │ Hst36_Nuclei_Multiome-v1 │          7          │
│                        │ Hst39_Nuclei_Multiome-v1 │          8          │
│                        │ Hst40_Nuclei_Multiome-v1 │          9          │
│                        │ Hst41_Nuclei_Multiome-v1 │         10          │
│                        │                          │                     │
└────────────────────────┴──────────────────────────┴─────────────────────┘
```

Example 183 (unknown):
```unknown
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/pytorch_lightning/trainer/configuration_validator.py:267: LightningDeprecationWarning: The `Callback.on_epoch_end` hook was deprecated in v1.6 and will be removed in v1.8. Please use `Callback.on_<train/validation/test>_epoch_end` instead.
  rank_zero_deprecation(
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
```

Example 184 (unknown):
```unknown
Epoch 85/500:  17%|███████████████████████▉                                                                                                                     | 85/500 [20:32<1:40:16, 14.50s/it, loss=4.1e+08, v_num=1]
Monitored metric reconstruction_loss_validation did not improve in the last 50 records. Best score: 33508.570. Signaling Trainer to stop.
```

Example 185 (unknown):
```unknown
###### BloodVesselEndothelialCells ######
(3028, 508040)
```

Example 186 (unknown):
```unknown
Anndata setup with scvi-tools version 0.19.0.
```

Example 187 (unknown):
```unknown
Setup via `PEAKVI.setup_anndata` with arguments:
```

Example 188 (unknown):
```unknown
{
│   'batch_key': None,
│   'labels_key': None,
│   'categorical_covariate_keys': ['batch_key'],
│   'continuous_covariate_keys': None,
│   'layer': None
}
```

Example 189 (unknown):
```unknown
Summary Statistics          
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃     Summary Stat Key     ┃ Value  ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│         n_batch          │   1    │
│         n_cells          │  3028  │
│ n_extra_categorical_covs │   1    │
│ n_extra_continuous_covs  │   0    │
│         n_labels         │   1    │
│          n_vars          │ 508040 │
└──────────────────────────┴────────┘
```

Example 190 (python):
```python
Data Registry                             
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃      Registry Key      ┃            scvi-tools Location             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│           X            │                  adata.X                   │
│         batch          │          adata.obs['_scvi_batch']          │
│ extra_categorical_covs │ adata.obsm['_scvi_extra_categorical_covs'] │
│         labels         │         adata.obs['_scvi_labels']          │
└────────────────────────┴────────────────────────────────────────────┘
```

Example 191 (python):
```python
batch State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃     Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_batch'] │     0      │          0          │
└──────────────────────────┴────────────┴─────────────────────┘
```

Example 192 (python):
```python
labels State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 193 (python):
```python
extra_categorical_covs State Registry                   
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location     ┃        Categories        ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['batch_key'] │  C82_Nuclei_Multiome-v1  │          0          │
│                        │  C85_Nuclei_Multiome-v1  │          1          │
│                        │  C87_Nuclei_Multiome-v1  │          2          │
│                        │  C92_Nuclei_Multiome-v1  │          3          │
│                        │  C98_Nuclei_Multiome-v1  │          4          │
│                        │ C104_Nuclei_Multiome-v1  │          5          │
│                        │ Hst33_Nuclei_Multiome-v1 │          6          │
│                        │ Hst36_Nuclei_Multiome-v1 │          7          │
│                        │ Hst39_Nuclei_Multiome-v1 │          8          │
│                        │ Hst40_Nuclei_Multiome-v1 │          9          │
│                        │ Hst41_Nuclei_Multiome-v1 │         10          │
│                        │                          │                     │
└────────────────────────┴──────────────────────────┴─────────────────────┘
```

Example 194 (unknown):
```unknown
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/pytorch_lightning/trainer/configuration_validator.py:267: LightningDeprecationWarning: The `Callback.on_epoch_end` hook was deprecated in v1.6 and will be removed in v1.8. Please use `Callback.on_<train/validation/test>_epoch_end` instead.
  rank_zero_deprecation(
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
```

Example 195 (unknown):
```unknown
Epoch 87/500:  17%|████████████████████████▎                                                                                                                   | 87/500 [14:41<1:09:46, 10.14s/it, loss=3.24e+08, v_num=1]
Monitored metric reconstruction_loss_validation did not improve in the last 50 records. Best score: 27426.242. Signaling Trainer to stop.
```

Example 196 (unknown):
```unknown
###### Glia ######
(1759, 508040)
```

Example 197 (python):
```python
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/scvi/data/_utils.py:168: UserWarning: Category 5 in adata.obs['batch_key'] has fewer than 3 cells. Models may not train properly.
  warnings.warn(
```

Example 198 (unknown):
```unknown
Anndata setup with scvi-tools version 0.19.0.
```

Example 199 (unknown):
```unknown
Setup via `PEAKVI.setup_anndata` with arguments:
```

Example 200 (unknown):
```unknown
{
│   'batch_key': None,
│   'labels_key': None,
│   'categorical_covariate_keys': ['batch_key'],
│   'continuous_covariate_keys': None,
│   'layer': None
}
```

Example 201 (unknown):
```unknown
Summary Statistics          
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃     Summary Stat Key     ┃ Value  ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│         n_batch          │   1    │
│         n_cells          │  1759  │
│ n_extra_categorical_covs │   1    │
│ n_extra_continuous_covs  │   0    │
│         n_labels         │   1    │
│          n_vars          │ 508040 │
└──────────────────────────┴────────┘
```

Example 202 (python):
```python
Data Registry                             
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃      Registry Key      ┃            scvi-tools Location             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│           X            │                  adata.X                   │
│         batch          │          adata.obs['_scvi_batch']          │
│ extra_categorical_covs │ adata.obsm['_scvi_extra_categorical_covs'] │
│         labels         │         adata.obs['_scvi_labels']          │
└────────────────────────┴────────────────────────────────────────────┘
```

Example 203 (python):
```python
batch State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃     Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_batch'] │     0      │          0          │
└──────────────────────────┴────────────┴─────────────────────┘
```

Example 204 (python):
```python
labels State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 205 (python):
```python
extra_categorical_covs State Registry                   
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location     ┃        Categories        ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['batch_key'] │  C82_Nuclei_Multiome-v1  │          0          │
│                        │  C85_Nuclei_Multiome-v1  │          1          │
│                        │  C87_Nuclei_Multiome-v1  │          2          │
│                        │  C92_Nuclei_Multiome-v1  │          3          │
│                        │  C98_Nuclei_Multiome-v1  │          4          │
│                        │ C104_Nuclei_Multiome-v1  │          5          │
│                        │ Hst33_Nuclei_Multiome-v1 │          6          │
│                        │ Hst36_Nuclei_Multiome-v1 │          7          │
│                        │ Hst39_Nuclei_Multiome-v1 │          8          │
│                        │ Hst40_Nuclei_Multiome-v1 │          9          │
│                        │ Hst41_Nuclei_Multiome-v1 │         10          │
│                        │                          │                     │
└────────────────────────┴──────────────────────────┴─────────────────────┘
```

Example 206 (unknown):
```unknown
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/pytorch_lightning/trainer/configuration_validator.py:267: LightningDeprecationWarning: The `Callback.on_epoch_end` hook was deprecated in v1.6 and will be removed in v1.8. Please use `Callback.on_<train/validation/test>_epoch_end` instead.
  rank_zero_deprecation(
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
```

Example 207 (unknown):
```unknown
Epoch 100/500:  20%|████████████████████████████                                                                                                                | 100/500 [09:48<39:14,  5.89s/it, loss=3.08e+08, v_num=1]
Monitored metric reconstruction_loss_validation did not improve in the last 50 records. Best score: 27881.668. Signaling Trainer to stop.
```

Example 208 (unknown):
```unknown
###### Neurons ######
(1101, 508040)
```

Example 209 (python):
```python
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/scvi/data/_utils.py:168: UserWarning: Category 5 in adata.obs['batch_key'] has fewer than 3 cells. Models may not train properly.
  warnings.warn(
```

Example 210 (unknown):
```unknown
Anndata setup with scvi-tools version 0.19.0.
```

Example 211 (unknown):
```unknown
Setup via `PEAKVI.setup_anndata` with arguments:
```

Example 212 (unknown):
```unknown
{
│   'batch_key': None,
│   'labels_key': None,
│   'categorical_covariate_keys': ['batch_key'],
│   'continuous_covariate_keys': None,
│   'layer': None
}
```

Example 213 (unknown):
```unknown
Summary Statistics          
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃     Summary Stat Key     ┃ Value  ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│         n_batch          │   1    │
│         n_cells          │  1101  │
│ n_extra_categorical_covs │   1    │
│ n_extra_continuous_covs  │   0    │
│         n_labels         │   1    │
│          n_vars          │ 508040 │
└──────────────────────────┴────────┘
```

Example 214 (python):
```python
Data Registry                             
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃      Registry Key      ┃            scvi-tools Location             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│           X            │                  adata.X                   │
│         batch          │          adata.obs['_scvi_batch']          │
│ extra_categorical_covs │ adata.obsm['_scvi_extra_categorical_covs'] │
│         labels         │         adata.obs['_scvi_labels']          │
└────────────────────────┴────────────────────────────────────────────┘
```

Example 215 (python):
```python
batch State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃     Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_batch'] │     0      │          0          │
└──────────────────────────┴────────────┴─────────────────────┘
```

Example 216 (python):
```python
labels State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 217 (python):
```python
extra_categorical_covs State Registry                   
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location     ┃        Categories        ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['batch_key'] │  C82_Nuclei_Multiome-v1  │          0          │
│                        │  C85_Nuclei_Multiome-v1  │          1          │
│                        │  C87_Nuclei_Multiome-v1  │          2          │
│                        │  C92_Nuclei_Multiome-v1  │          3          │
│                        │  C98_Nuclei_Multiome-v1  │          4          │
│                        │ C104_Nuclei_Multiome-v1  │          5          │
│                        │ Hst33_Nuclei_Multiome-v1 │          6          │
│                        │ Hst36_Nuclei_Multiome-v1 │          7          │
│                        │ Hst39_Nuclei_Multiome-v1 │          8          │
│                        │ Hst40_Nuclei_Multiome-v1 │          9          │
│                        │ Hst41_Nuclei_Multiome-v1 │         10          │
│                        │                          │                     │
└────────────────────────┴──────────────────────────┴─────────────────────┘
```

Example 218 (unknown):
```unknown
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/pytorch_lightning/trainer/configuration_validator.py:267: LightningDeprecationWarning: The `Callback.on_epoch_end` hook was deprecated in v1.6 and will be removed in v1.8. Please use `Callback.on_<train/validation/test>_epoch_end` instead.
  rank_zero_deprecation(
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py:1892: PossibleUserWarning: The number of training batches (8) is smaller than the logging interval Trainer(log_every_n_steps=10). Set a lower value for log_every_n_steps if you want to see logs for the training epoch.
  rank_zero_warn(
```

Example 219 (unknown):
```unknown
Epoch 110/500:  22%|██████████████████████████████▊                                                                                                             | 110/500 [06:52<24:23,  3.75s/it, loss=4.47e+08, v_num=1]
Monitored metric reconstruction_loss_validation did not improve in the last 50 records. Best score: 42944.938. Signaling Trainer to stop.
```

Example 220 (unknown):
```unknown
###### EndocardialCells ######
(1410, 508040)
```

Example 221 (unknown):
```unknown
Anndata setup with scvi-tools version 0.19.0.
```

Example 222 (unknown):
```unknown
Setup via `PEAKVI.setup_anndata` with arguments:
```

Example 223 (unknown):
```unknown
{
│   'batch_key': None,
│   'labels_key': None,
│   'categorical_covariate_keys': ['batch_key'],
│   'continuous_covariate_keys': None,
│   'layer': None
}
```

Example 224 (unknown):
```unknown
Summary Statistics          
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃     Summary Stat Key     ┃ Value  ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│         n_batch          │   1    │
│         n_cells          │  1410  │
│ n_extra_categorical_covs │   1    │
│ n_extra_continuous_covs  │   0    │
│         n_labels         │   1    │
│          n_vars          │ 508040 │
└──────────────────────────┴────────┘
```

Example 225 (python):
```python
Data Registry                             
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃      Registry Key      ┃            scvi-tools Location             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│           X            │                  adata.X                   │
│         batch          │          adata.obs['_scvi_batch']          │
│ extra_categorical_covs │ adata.obsm['_scvi_extra_categorical_covs'] │
│         labels         │         adata.obs['_scvi_labels']          │
└────────────────────────┴────────────────────────────────────────────┘
```

Example 226 (python):
```python
batch State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃     Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_batch'] │     0      │          0          │
└──────────────────────────┴────────────┴─────────────────────┘
```

Example 227 (python):
```python
labels State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 228 (python):
```python
extra_categorical_covs State Registry                   
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location     ┃        Categories        ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['batch_key'] │  C82_Nuclei_Multiome-v1  │          0          │
│                        │  C85_Nuclei_Multiome-v1  │          1          │
│                        │  C87_Nuclei_Multiome-v1  │          2          │
│                        │  C92_Nuclei_Multiome-v1  │          3          │
│                        │  C98_Nuclei_Multiome-v1  │          4          │
│                        │ C104_Nuclei_Multiome-v1  │          5          │
│                        │ Hst33_Nuclei_Multiome-v1 │          6          │
│                        │ Hst36_Nuclei_Multiome-v1 │          7          │
│                        │ Hst39_Nuclei_Multiome-v1 │          8          │
│                        │ Hst40_Nuclei_Multiome-v1 │          9          │
│                        │ Hst41_Nuclei_Multiome-v1 │         10          │
│                        │                          │                     │
└────────────────────────┴──────────────────────────┴─────────────────────┘
```

Example 229 (unknown):
```unknown
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/pytorch_lightning/trainer/configuration_validator.py:267: LightningDeprecationWarning: The `Callback.on_epoch_end` hook was deprecated in v1.6 and will be removed in v1.8. Please use `Callback.on_<train/validation/test>_epoch_end` instead.
  rank_zero_deprecation(
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
```

Example 230 (unknown):
```unknown
Epoch 107/500:  21%|█████████████████████████████▉                                                                                                              | 107/500 [08:28<31:07,  4.75s/it, loss=3.81e+08, v_num=1]
Monitored metric reconstruction_loss_validation did not improve in the last 50 records. Best score: 34209.277. Signaling Trainer to stop.
```

Example 231 (unknown):
```unknown
###### VentricularCardiomyocytes_plusCCS ######
(6531, 508040)
```

Example 232 (unknown):
```unknown
Anndata setup with scvi-tools version 0.19.0.
```

Example 233 (unknown):
```unknown
Setup via `PEAKVI.setup_anndata` with arguments:
```

Example 234 (unknown):
```unknown
{
│   'batch_key': None,
│   'labels_key': None,
│   'categorical_covariate_keys': ['batch_key'],
│   'continuous_covariate_keys': None,
│   'layer': None
}
```

Example 235 (unknown):
```unknown
Summary Statistics          
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃     Summary Stat Key     ┃ Value  ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│         n_batch          │   1    │
│         n_cells          │  6531  │
│ n_extra_categorical_covs │   1    │
│ n_extra_continuous_covs  │   0    │
│         n_labels         │   1    │
│          n_vars          │ 508040 │
└──────────────────────────┴────────┘
```

Example 236 (python):
```python
Data Registry                             
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃      Registry Key      ┃            scvi-tools Location             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│           X            │                  adata.X                   │
│         batch          │          adata.obs['_scvi_batch']          │
│ extra_categorical_covs │ adata.obsm['_scvi_extra_categorical_covs'] │
│         labels         │         adata.obs['_scvi_labels']          │
└────────────────────────┴────────────────────────────────────────────┘
```

Example 237 (python):
```python
batch State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃     Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_batch'] │     0      │          0          │
└──────────────────────────┴────────────┴─────────────────────┘
```

Example 238 (python):
```python
labels State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 239 (python):
```python
extra_categorical_covs State Registry                   
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location     ┃        Categories        ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['batch_key'] │  C82_Nuclei_Multiome-v1  │          0          │
│                        │  C85_Nuclei_Multiome-v1  │          1          │
│                        │  C87_Nuclei_Multiome-v1  │          2          │
│                        │  C92_Nuclei_Multiome-v1  │          3          │
│                        │  C98_Nuclei_Multiome-v1  │          4          │
│                        │ C104_Nuclei_Multiome-v1  │          5          │
│                        │ Hst33_Nuclei_Multiome-v1 │          6          │
│                        │ Hst36_Nuclei_Multiome-v1 │          7          │
│                        │ Hst39_Nuclei_Multiome-v1 │          8          │
│                        │ Hst40_Nuclei_Multiome-v1 │          9          │
│                        │ Hst41_Nuclei_Multiome-v1 │         10          │
│                        │                          │                     │
└────────────────────────┴──────────────────────────┴─────────────────────┘
```

Example 240 (unknown):
```unknown
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/pytorch_lightning/trainer/configuration_validator.py:267: LightningDeprecationWarning: The `Callback.on_epoch_end` hook was deprecated in v1.6 and will be removed in v1.8. Please use `Callback.on_<train/validation/test>_epoch_end` instead.
  rank_zero_deprecation(
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
```

Example 241 (unknown):
```unknown
Epoch 82/500:  16%|██████████████████████▉                                                                                                                     | 82/500 [30:02<2:33:06, 21.98s/it, loss=4.66e+08, v_num=1]
Monitored metric reconstruction_loss_validation did not improve in the last 50 records. Best score: 34396.367. Signaling Trainer to stop.
```

Example 242 (unknown):
```unknown
###### PericardialCells ######
(2187, 508040)
```

Example 243 (python):
```python
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/scvi/data/_utils.py:168: UserWarning: Category 3 in adata.obs['batch_key'] has fewer than 3 cells. Models may not train properly.
  warnings.warn(
```

Example 244 (unknown):
```unknown
Anndata setup with scvi-tools version 0.19.0.
```

Example 245 (unknown):
```unknown
Setup via `PEAKVI.setup_anndata` with arguments:
```

Example 246 (unknown):
```unknown
{
│   'batch_key': None,
│   'labels_key': None,
│   'categorical_covariate_keys': ['batch_key'],
│   'continuous_covariate_keys': None,
│   'layer': None
}
```

Example 247 (unknown):
```unknown
Summary Statistics          
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃     Summary Stat Key     ┃ Value  ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│         n_batch          │   1    │
│         n_cells          │  2187  │
│ n_extra_categorical_covs │   1    │
│ n_extra_continuous_covs  │   0    │
│         n_labels         │   1    │
│          n_vars          │ 508040 │
└──────────────────────────┴────────┘
```

Example 248 (python):
```python
Data Registry                             
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃      Registry Key      ┃            scvi-tools Location             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│           X            │                  adata.X                   │
│         batch          │          adata.obs['_scvi_batch']          │
│ extra_categorical_covs │ adata.obsm['_scvi_extra_categorical_covs'] │
│         labels         │         adata.obs['_scvi_labels']          │
└────────────────────────┴────────────────────────────────────────────┘
```

Example 249 (python):
```python
batch State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃     Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_batch'] │     0      │          0          │
└──────────────────────────┴────────────┴─────────────────────┘
```

Example 250 (python):
```python
labels State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 251 (python):
```python
extra_categorical_covs State Registry                   
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location     ┃        Categories        ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['batch_key'] │  C87_Nuclei_Multiome-v1  │          0          │
│                        │  C92_Nuclei_Multiome-v1  │          1          │
│                        │  C98_Nuclei_Multiome-v1  │          2          │
│                        │ C104_Nuclei_Multiome-v1  │          3          │
│                        │ Hst39_Nuclei_Multiome-v1 │          4          │
│                        │ Hst40_Nuclei_Multiome-v1 │          5          │
│                        │ Hst41_Nuclei_Multiome-v1 │          6          │
│                        │                          │                     │
└────────────────────────┴──────────────────────────┴─────────────────────┘
```

Example 252 (unknown):
```unknown
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/pytorch_lightning/trainer/configuration_validator.py:267: LightningDeprecationWarning: The `Callback.on_epoch_end` hook was deprecated in v1.6 and will be removed in v1.8. Please use `Callback.on_<train/validation/test>_epoch_end` instead.
  rank_zero_deprecation(
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
```

Example 253 (unknown):
```unknown
Epoch 89/500:  18%|█████████████████████████▎                                                                                                                    | 89/500 [10:57<50:34,  7.38s/it, loss=3.53e+08, v_num=1]
Monitored metric reconstruction_loss_validation did not improve in the last 50 records. Best score: 30579.912. Signaling Trainer to stop.
```

Example 254 (unknown):
```unknown
###### LymphoidCells ######
(7006, 508040)
```

Example 255 (python):
```python
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/scvi/data/_utils.py:168: UserWarning: Category 4 in adata.obs['batch_key'] has fewer than 3 cells. Models may not train properly.
  warnings.warn(
```

Example 256 (unknown):
```unknown
Anndata setup with scvi-tools version 0.19.0.
```

Example 257 (unknown):
```unknown
Setup via `PEAKVI.setup_anndata` with arguments:
```

Example 258 (unknown):
```unknown
{
│   'batch_key': None,
│   'labels_key': None,
│   'categorical_covariate_keys': ['batch_key'],
│   'continuous_covariate_keys': None,
│   'layer': None
}
```

Example 259 (unknown):
```unknown
Summary Statistics          
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃     Summary Stat Key     ┃ Value  ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│         n_batch          │   1    │
│         n_cells          │  7006  │
│ n_extra_categorical_covs │   1    │
│ n_extra_continuous_covs  │   0    │
│         n_labels         │   1    │
│          n_vars          │ 508040 │
└──────────────────────────┴────────┘
```

Example 260 (python):
```python
Data Registry                             
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃      Registry Key      ┃            scvi-tools Location             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│           X            │                  adata.X                   │
│         batch          │          adata.obs['_scvi_batch']          │
│ extra_categorical_covs │ adata.obsm['_scvi_extra_categorical_covs'] │
│         labels         │         adata.obs['_scvi_labels']          │
└────────────────────────┴────────────────────────────────────────────┘
```

Example 261 (python):
```python
batch State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃     Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_batch'] │     0      │          0          │
└──────────────────────────┴────────────┴─────────────────────┘
```

Example 262 (python):
```python
labels State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 263 (python):
```python
extra_categorical_covs State Registry                   
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location     ┃        Categories        ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['batch_key'] │  C82_Nuclei_Multiome-v1  │          0          │
│                        │  C85_Nuclei_Multiome-v1  │          1          │
│                        │  C87_Nuclei_Multiome-v1  │          2          │
│                        │  C92_Nuclei_Multiome-v1  │          3          │
│                        │  C98_Nuclei_Multiome-v1  │          4          │
│                        │ Hst33_Nuclei_Multiome-v1 │          5          │
│                        │ Hst36_Nuclei_Multiome-v1 │          6          │
│                        │ Hst39_Nuclei_Multiome-v1 │          7          │
│                        │ Hst40_Nuclei_Multiome-v1 │          8          │
│                        │ Hst41_Nuclei_Multiome-v1 │          9          │
│                        │                          │                     │
└────────────────────────┴──────────────────────────┴─────────────────────┘
```

Example 264 (unknown):
```unknown
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/pytorch_lightning/trainer/configuration_validator.py:267: LightningDeprecationWarning: The `Callback.on_epoch_end` hook was deprecated in v1.6 and will be removed in v1.8. Please use `Callback.on_<train/validation/test>_epoch_end` instead.
  rank_zero_deprecation(
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
```

Example 265 (unknown):
```unknown
Epoch 82/500:  16%|██████████████████████▉                                                                                                                     | 82/500 [32:31<2:45:47, 23.80s/it, loss=2.72e+08, v_num=1]
Monitored metric reconstruction_loss_validation did not improve in the last 50 records. Best score: 21162.615. Signaling Trainer to stop.
```

Example 266 (unknown):
```unknown
###### EpicardialCells ######
(1340, 508040)
```

Example 267 (unknown):
```unknown
Anndata setup with scvi-tools version 0.19.0.
```

Example 268 (unknown):
```unknown
Setup via `PEAKVI.setup_anndata` with arguments:
```

Example 269 (unknown):
```unknown
{
│   'batch_key': None,
│   'labels_key': None,
│   'categorical_covariate_keys': ['batch_key'],
│   'continuous_covariate_keys': None,
│   'layer': None
}
```

Example 270 (unknown):
```unknown
Summary Statistics          
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃     Summary Stat Key     ┃ Value  ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│         n_batch          │   1    │
│         n_cells          │  1340  │
│ n_extra_categorical_covs │   1    │
│ n_extra_continuous_covs  │   0    │
│         n_labels         │   1    │
│          n_vars          │ 508040 │
└──────────────────────────┴────────┘
```

Example 271 (python):
```python
Data Registry                             
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃      Registry Key      ┃            scvi-tools Location             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│           X            │                  adata.X                   │
│         batch          │          adata.obs['_scvi_batch']          │
│ extra_categorical_covs │ adata.obsm['_scvi_extra_categorical_covs'] │
│         labels         │         adata.obs['_scvi_labels']          │
└────────────────────────┴────────────────────────────────────────────┘
```

Example 272 (python):
```python
batch State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃     Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_batch'] │     0      │          0          │
└──────────────────────────┴────────────┴─────────────────────┘
```

Example 273 (python):
```python
labels State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 274 (python):
```python
extra_categorical_covs State Registry                   
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location     ┃        Categories        ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['batch_key'] │  C82_Nuclei_Multiome-v1  │          0          │
│                        │  C85_Nuclei_Multiome-v1  │          1          │
│                        │  C87_Nuclei_Multiome-v1  │          2          │
│                        │  C92_Nuclei_Multiome-v1  │          3          │
│                        │  C98_Nuclei_Multiome-v1  │          4          │
│                        │ C104_Nuclei_Multiome-v1  │          5          │
│                        │ Hst33_Nuclei_Multiome-v1 │          6          │
│                        │ Hst36_Nuclei_Multiome-v1 │          7          │
│                        │ Hst39_Nuclei_Multiome-v1 │          8          │
│                        │ Hst40_Nuclei_Multiome-v1 │          9          │
│                        │ Hst41_Nuclei_Multiome-v1 │         10          │
│                        │                          │                     │
└────────────────────────┴──────────────────────────┴─────────────────────┘
```

Example 275 (unknown):
```unknown
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/pytorch_lightning/trainer/configuration_validator.py:267: LightningDeprecationWarning: The `Callback.on_epoch_end` hook was deprecated in v1.6 and will be removed in v1.8. Please use `Callback.on_<train/validation/test>_epoch_end` instead.
  rank_zero_deprecation(
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
```

Example 276 (unknown):
```unknown
Epoch 100/500:  20%|████████████████████████████                                                                                                                | 100/500 [07:39<30:37,  4.59s/it, loss=4.42e+08, v_num=1]
Monitored metric reconstruction_loss_validation did not improve in the last 50 records. Best score: 37964.527. Signaling Trainer to stop.
```

Example 277 (unknown):
```unknown
###### MuralCells ######
(3103, 508040)
```

Example 278 (unknown):
```unknown
Anndata setup with scvi-tools version 0.19.0.
```

Example 279 (unknown):
```unknown
Setup via `PEAKVI.setup_anndata` with arguments:
```

Example 280 (unknown):
```unknown
{
│   'batch_key': None,
│   'labels_key': None,
│   'categorical_covariate_keys': ['batch_key'],
│   'continuous_covariate_keys': None,
│   'layer': None
}
```

Example 281 (unknown):
```unknown
Summary Statistics          
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃     Summary Stat Key     ┃ Value  ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│         n_batch          │   1    │
│         n_cells          │  3103  │
│ n_extra_categorical_covs │   1    │
│ n_extra_continuous_covs  │   0    │
│         n_labels         │   1    │
│          n_vars          │ 508040 │
└──────────────────────────┴────────┘
```

Example 282 (python):
```python
Data Registry                             
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃      Registry Key      ┃            scvi-tools Location             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│           X            │                  adata.X                   │
│         batch          │          adata.obs['_scvi_batch']          │
│ extra_categorical_covs │ adata.obsm['_scvi_extra_categorical_covs'] │
│         labels         │         adata.obs['_scvi_labels']          │
└────────────────────────┴────────────────────────────────────────────┘
```

Example 283 (python):
```python
batch State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃     Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_batch'] │     0      │          0          │
└──────────────────────────┴────────────┴─────────────────────┘
```

Example 284 (python):
```python
labels State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 285 (python):
```python
extra_categorical_covs State Registry                   
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location     ┃        Categories        ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['batch_key'] │  C82_Nuclei_Multiome-v1  │          0          │
│                        │  C85_Nuclei_Multiome-v1  │          1          │
│                        │  C87_Nuclei_Multiome-v1  │          2          │
│                        │  C92_Nuclei_Multiome-v1  │          3          │
│                        │  C98_Nuclei_Multiome-v1  │          4          │
│                        │ C104_Nuclei_Multiome-v1  │          5          │
│                        │ Hst33_Nuclei_Multiome-v1 │          6          │
│                        │ Hst36_Nuclei_Multiome-v1 │          7          │
│                        │ Hst39_Nuclei_Multiome-v1 │          8          │
│                        │ Hst40_Nuclei_Multiome-v1 │          9          │
│                        │ Hst41_Nuclei_Multiome-v1 │         10          │
│                        │                          │                     │
└────────────────────────┴──────────────────────────┴─────────────────────┘
```

Example 286 (unknown):
```unknown
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/pytorch_lightning/trainer/configuration_validator.py:267: LightningDeprecationWarning: The `Callback.on_epoch_end` hook was deprecated in v1.6 and will be removed in v1.8. Please use `Callback.on_<train/validation/test>_epoch_end` instead.
  rank_zero_deprecation(
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
```

Example 287 (unknown):
```unknown
Epoch 89/500:  18%|████████████████████████▉                                                                                                                   | 89/500 [15:13<1:10:20, 10.27s/it, loss=3.14e+08, v_num=1]
Monitored metric reconstruction_loss_validation did not improve in the last 50 records. Best score: 27392.287. Signaling Trainer to stop.
```

Example 288 (unknown):
```unknown
###### Fibroblasts ######
(6969, 508040)
```

Example 289 (unknown):
```unknown
Anndata setup with scvi-tools version 0.19.0.
```

Example 290 (unknown):
```unknown
Setup via `PEAKVI.setup_anndata` with arguments:
```

Example 291 (unknown):
```unknown
{
│   'batch_key': None,
│   'labels_key': None,
│   'categorical_covariate_keys': ['batch_key'],
│   'continuous_covariate_keys': None,
│   'layer': None
}
```

Example 292 (unknown):
```unknown
Summary Statistics          
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃     Summary Stat Key     ┃ Value  ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│         n_batch          │   1    │
│         n_cells          │  6969  │
│ n_extra_categorical_covs │   1    │
│ n_extra_continuous_covs  │   0    │
│         n_labels         │   1    │
│          n_vars          │ 508040 │
└──────────────────────────┴────────┘
```

Example 293 (python):
```python
Data Registry                             
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃      Registry Key      ┃            scvi-tools Location             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│           X            │                  adata.X                   │
│         batch          │          adata.obs['_scvi_batch']          │
│ extra_categorical_covs │ adata.obsm['_scvi_extra_categorical_covs'] │
│         labels         │         adata.obs['_scvi_labels']          │
└────────────────────────┴────────────────────────────────────────────┘
```

Example 294 (python):
```python
batch State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃     Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_batch'] │     0      │          0          │
└──────────────────────────┴────────────┴─────────────────────┘
```

Example 295 (python):
```python
labels State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 296 (python):
```python
extra_categorical_covs State Registry                   
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location     ┃        Categories        ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['batch_key'] │  C82_Nuclei_Multiome-v1  │          0          │
│                        │  C85_Nuclei_Multiome-v1  │          1          │
│                        │  C87_Nuclei_Multiome-v1  │          2          │
│                        │  C92_Nuclei_Multiome-v1  │          3          │
│                        │  C98_Nuclei_Multiome-v1  │          4          │
│                        │ C104_Nuclei_Multiome-v1  │          5          │
│                        │ Hst33_Nuclei_Multiome-v1 │          6          │
│                        │ Hst36_Nuclei_Multiome-v1 │          7          │
│                        │ Hst39_Nuclei_Multiome-v1 │          8          │
│                        │ Hst40_Nuclei_Multiome-v1 │          9          │
│                        │ Hst41_Nuclei_Multiome-v1 │         10          │
│                        │                          │                     │
└────────────────────────┴──────────────────────────┴─────────────────────┘
```

Example 297 (unknown):
```unknown
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/pytorch_lightning/trainer/configuration_validator.py:267: LightningDeprecationWarning: The `Callback.on_epoch_end` hook was deprecated in v1.6 and will be removed in v1.8. Please use `Callback.on_<train/validation/test>_epoch_end` instead.
  rank_zero_deprecation(
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
```

Example 298 (unknown):
```unknown
Epoch 79/500:  16%|██████████████████████                                                                                                                      | 79/500 [30:43<2:43:41, 23.33s/it, loss=4.04e+08, v_num=1]
Monitored metric reconstruction_loss_validation did not improve in the last 50 records. Best score: 30068.029. Signaling Trainer to stop.
```

Example 299 (unknown):
```unknown
###### MyeloidCells ######
(4321, 508040)
```

Example 300 (unknown):
```unknown
Anndata setup with scvi-tools version 0.19.0.
```

Example 301 (unknown):
```unknown
Setup via `PEAKVI.setup_anndata` with arguments:
```

Example 302 (unknown):
```unknown
{
│   'batch_key': None,
│   'labels_key': None,
│   'categorical_covariate_keys': ['batch_key'],
│   'continuous_covariate_keys': None,
│   'layer': None
}
```

Example 303 (unknown):
```unknown
Summary Statistics          
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃     Summary Stat Key     ┃ Value  ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│         n_batch          │   1    │
│         n_cells          │  4321  │
│ n_extra_categorical_covs │   1    │
│ n_extra_continuous_covs  │   0    │
│         n_labels         │   1    │
│          n_vars          │ 508040 │
└──────────────────────────┴────────┘
```

Example 304 (python):
```python
Data Registry                             
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃      Registry Key      ┃            scvi-tools Location             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│           X            │                  adata.X                   │
│         batch          │          adata.obs['_scvi_batch']          │
│ extra_categorical_covs │ adata.obsm['_scvi_extra_categorical_covs'] │
│         labels         │         adata.obs['_scvi_labels']          │
└────────────────────────┴────────────────────────────────────────────┘
```

Example 305 (python):
```python
batch State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃     Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_batch'] │     0      │          0          │
└──────────────────────────┴────────────┴─────────────────────┘
```

Example 306 (python):
```python
labels State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 307 (python):
```python
extra_categorical_covs State Registry                   
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location     ┃        Categories        ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['batch_key'] │  C82_Nuclei_Multiome-v1  │          0          │
│                        │  C85_Nuclei_Multiome-v1  │          1          │
│                        │  C87_Nuclei_Multiome-v1  │          2          │
│                        │  C92_Nuclei_Multiome-v1  │          3          │
│                        │  C98_Nuclei_Multiome-v1  │          4          │
│                        │ C104_Nuclei_Multiome-v1  │          5          │
│                        │ Hst33_Nuclei_Multiome-v1 │          6          │
│                        │ Hst36_Nuclei_Multiome-v1 │          7          │
│                        │ Hst39_Nuclei_Multiome-v1 │          8          │
│                        │ Hst40_Nuclei_Multiome-v1 │          9          │
│                        │ Hst41_Nuclei_Multiome-v1 │         10          │
│                        │                          │                     │
└────────────────────────┴──────────────────────────┴─────────────────────┘
```

Example 308 (unknown):
```unknown
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/pytorch_lightning/trainer/configuration_validator.py:267: LightningDeprecationWarning: The `Callback.on_epoch_end` hook was deprecated in v1.6 and will be removed in v1.8. Please use `Callback.on_<train/validation/test>_epoch_end` instead.
  rank_zero_deprecation(
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
```

Example 309 (unknown):
```unknown
Epoch 85/500:  17%|███████████████████████▊                                                                                                                    | 85/500 [20:33<1:40:20, 14.51s/it, loss=3.54e+08, v_num=1]
Monitored metric reconstruction_loss_validation did not improve in the last 50 records. Best score: 29083.998. Signaling Trainer to stop.
```

Example 310 (unknown):
```unknown
###### LymphaticEndothelialCells ######
(1000, 508040)
```

Example 311 (python):
```python
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/scvi/data/_utils.py:168: UserWarning: Category 5 in adata.obs['batch_key'] has fewer than 3 cells. Models may not train properly.
  warnings.warn(
```

Example 312 (unknown):
```unknown
Anndata setup with scvi-tools version 0.19.0.
```

Example 313 (unknown):
```unknown
Setup via `PEAKVI.setup_anndata` with arguments:
```

Example 314 (unknown):
```unknown
{
│   'batch_key': None,
│   'labels_key': None,
│   'categorical_covariate_keys': ['batch_key'],
│   'continuous_covariate_keys': None,
│   'layer': None
}
```

Example 315 (unknown):
```unknown
Summary Statistics          
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃     Summary Stat Key     ┃ Value  ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│         n_batch          │   1    │
│         n_cells          │  1000  │
│ n_extra_categorical_covs │   1    │
│ n_extra_continuous_covs  │   0    │
│         n_labels         │   1    │
│          n_vars          │ 508040 │
└──────────────────────────┴────────┘
```

Example 316 (python):
```python
Data Registry                             
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃      Registry Key      ┃            scvi-tools Location             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│           X            │                  adata.X                   │
│         batch          │          adata.obs['_scvi_batch']          │
│ extra_categorical_covs │ adata.obsm['_scvi_extra_categorical_covs'] │
│         labels         │         adata.obs['_scvi_labels']          │
└────────────────────────┴────────────────────────────────────────────┘
```

Example 317 (python):
```python
batch State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃     Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_batch'] │     0      │          0          │
└──────────────────────────┴────────────┴─────────────────────┘
```

Example 318 (python):
```python
labels State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 319 (python):
```python
extra_categorical_covs State Registry                   
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location     ┃        Categories        ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['batch_key'] │  C82_Nuclei_Multiome-v1  │          0          │
│                        │  C85_Nuclei_Multiome-v1  │          1          │
│                        │  C87_Nuclei_Multiome-v1  │          2          │
│                        │  C92_Nuclei_Multiome-v1  │          3          │
│                        │  C98_Nuclei_Multiome-v1  │          4          │
│                        │ C104_Nuclei_Multiome-v1  │          5          │
│                        │ Hst33_Nuclei_Multiome-v1 │          6          │
│                        │ Hst36_Nuclei_Multiome-v1 │          7          │
│                        │ Hst39_Nuclei_Multiome-v1 │          8          │
│                        │ Hst40_Nuclei_Multiome-v1 │          9          │
│                        │ Hst41_Nuclei_Multiome-v1 │         10          │
│                        │                          │                     │
└────────────────────────┴──────────────────────────┴─────────────────────┘
```

Example 320 (unknown):
```unknown
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/pytorch_lightning/trainer/configuration_validator.py:267: LightningDeprecationWarning: The `Callback.on_epoch_end` hook was deprecated in v1.6 and will be removed in v1.8. Please use `Callback.on_<train/validation/test>_epoch_end` instead.
  rank_zero_deprecation(
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py:1892: PossibleUserWarning: The number of training batches (8) is smaller than the logging interval Trainer(log_every_n_steps=10). Set a lower value for log_every_n_steps if you want to see logs for the training epoch.
  rank_zero_warn(
```

Example 321 (unknown):
```unknown
Epoch 100/500:  20%|████████████████████████████                                                                                                                | 100/500 [05:55<23:41,  3.55s/it, loss=3.19e+08, v_num=1]
Monitored metric reconstruction_loss_validation did not improve in the last 50 records. Best score: 30174.766. Signaling Trainer to stop.
```

---

## 

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/RNA/scnRNA_Integration-Annotation/Annotations/ImmuneAnnotationKazumasa.html

**Examples:**

Example 1 (python):
```python
# Import packages, initial settings

import numpy as np
import matplotlib.pyplot as plt
import scanpy as sc
import pandas as pd
import anndata
import scrublet as scr
import seaborn as sns
import magic
import pybiomart
import six
import sys
sys.modules['sklearn.externals.six'] = six
import scanorama
import harmonypy
import bbknn
import gseapy
from gseapy.plot import barplot, dotplot
import scvelo as scv
import scvi

plt.rcParams['pdf.fonttype'] = 42
plt.rcParams['ps.fonttype'] = 42

sc.settings.verbosity = 3  # verbosity: errors (0), warnings (1), info (2), hints (3)

from matplotlib.colors import LinearSegmentedColormap
cmap = LinearSegmentedColormap.from_list(name='gene_cmap', colors=['lightgrey', 'thistle', 'red', 'darkred']) 

sc.settings.set_figure_params(dpi=80, color_map='viridis', vector_friendly=False,  dpi_save=300, facecolor='white')
sc.logging.print_version_and_date()
#sc.logging.print_versions()
```

Example 2 (unknown):
```unknown
Global seed set to 0
```

Example 3 (unknown):
```unknown
Running Scanpy 1.8.1, on 2022-12-21 19:40.
```

Example 4 (python):
```python
adata=sc.read('./Objects/Annotated/Mesenchymal.h5ad')
print(adata.shape)

#sc.pp.normalize_total(adata, target_sum=1e4)
#sc.pp.log1p(adata)
```

Example 5 (unknown):
```unknown
(129898, 2479)
```

Example 6 (python):
```python
set(adata.obs.new_fine_grain)
```

Example 7 (unknown):
```unknown
{'B',
 'B1',
 'B_cycling',
 'Bpro',
 'CD4+T',
 'CD4+Treg',
 'CD8+T',
 'DC1',
 'DC2',
 'DC_mature',
 'Doublet',
 'ILC',
 'MP_CX3CR1+',
 'MP_LYVE1+',
 'MP_LYVE1+Collagen+',
 'MP_LYVE1+JUN+',
 'MP_LYVE1+_cycling',
 'MP_TIMD4+',
 'MP_TREM2+',
 'Mast cell',
 'Mono',
 'Mono_MPO+',
 'NK',
 'Plasmacytoid DC',
 'T-NK_cycling'}
```

Example 8 (python):
```python
#ClustersToStay=['T-NK_cycling','NK','CD8+T','CD4+Treg','CD4+T']
#adata=adata[[x for x in adata.obs_names if adata.obs.new_fine_grain[x] in ClustersToStay],:].copy()
#sc.pl.umap(adata, color='new_fine_grain')


adata=adata[adata.obs.new_fine_grain=='T-NK_cycling',:].copy()
#adata
```

Example 9 (python):
```python
sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata,min_dist=0.3)
#sc.tl.tsne(adata)

sc.pl.umap(adata, color=['log1p_total_counts','n_genes','n_genes_by_counts','total_counts','pct_counts_mt','pct_counts_ribo'],ncols=5)
sc.pl.umap(adata, color=['scrublet_score','stress_score', 'hb1_score','phase'], ncols=5,)
sc.pl.umap(adata, color=['donor','sex','age','region','phase','cell_or_nuclei','modality','kit_10x','FACSgate'],wspace=0.8,ncols=5)

Genes=['TTN','TNNT2','MYOM1','LDB3','ALDH1A2','TNNT1','KRT19','UPK3B','COL6A3','FBLN1','PCOLCE','LRRC17','EMCN','ECSCR','PECAM1','CDH5', 'LAPTM5','PTPRC','LCP1','CD53','PHOX2B','NRXN1','STMN2','GRIK2']
sc.pl.umap(adata, color=Genes)
```

Example 10 (python):
```python
computing neighbors
    finished: added to `.uns['neighbors']`
    `.obsp['distances']`, distances for each pair of neighbors
    `.obsp['connectivities']`, weighted adjacency matrix (0:00:00)
computing UMAP
    finished: added
    'X_umap', UMAP coordinates (adata.obsm) (0:00:01)
```

Example 11 (python):
```python
#sc.pl.umap(adata, color='new_fine_grain')
#sc.pl.umap(adata, color=['CD8Anew_fine_grain')
sc.pl.umap(adata, color=['NKG7','TGFBR3','GZMA'])
sc.pl.umap(adata, color=['CSGALNACT1','ANK3'])
sc.pl.umap(adata, color=['NELL2'])
sc.pl.umap(adata, color=['BCL2','RTKN2','GBP2'])
```

Example 12 (python):
```python
bdata=adata[adata.obs.new_fine_grain=='T-NK_cycling',:].copy()
sc.pl.umap(bdata, color=['CD7','NCAM1','FOXP3'])
```

Example 13 (python):
```python
sc.tl.leiden(adata, resolution=0.4, key_added='leiden')
sc.pl.umap(adata, color='leiden',legend_loc='on data')
```

Example 14 (python):
```python
running Leiden clustering
    finished: found 8 clusters and added
    'leiden', the cluster labels (adata.obs, categorical) (0:00:00)
```

Example 15 (python):
```python
sc.pl.umap(adata, color='leiden',legend_loc='on data')
```

Example 16 (python):
```python
ClustersToStay=['5']

bdata=adata[[x for x in adata.obs_names if adata.obs.leiden[x] in ClustersToStay],:].copy()

bdatanames=bdata.obs_names

print(len(bdatanames))
np.savetxt('BAdd.txt', bdatanames, delimiter="\n", fmt="%s")
```

Example 17 (unknown):
```unknown
AnnData object with n_obs × n_vars = 2138 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'batch_key', '_scvi_batch', '_scvi_labels', 'previous_coarse_grain', 'previous_mid_grain', 'previous_fine_grain', 'leiden_scVI', 'FACSgate', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', 'hb2_score', 'leiden', 'coarse_grain', 'sex', 'age_binned', 'leiden_scVI_0.2', 'leiden_scVI_0.3', 'leiden_scVI_0.4', 'leiden_scVI_0.5', 'leiden_scVI_0.6', 'cell_type2', 'new_fine_grain', 'new_mid_grain'
    var: 'gene_name_scRNA-0', 'gene_id'
    uns: 'cell_type2_colors', 'fine_grain_colors', 'mid_grain_colors', 'new_fine_grain_colors', 'new_mid_grain_colors', 'log1p', 'neighbors', 'umap', 'phase_colors', 'donor_colors', 'sex_colors', 'age_colors', 'region_colors', 'cell_or_nuclei_colors', 'modality_colors', 'kit_10x_colors', 'FACSgate_colors', 'leiden', 'leiden_colors', 'rank_genes_groups', 'dendrogram_leiden'
    obsm: 'X_pca', 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 18 (python):
```python
sc.tl.rank_genes_groups(adata, groupby='leiden', method='t-test')
sc.tl.dendrogram(adata, groupby='leiden')
#sc.pl.dendrogram(adata,groupby='leiden')
sc.pl.rank_genes_groups_matrixplot(adata, groupby='leiden', standard_scale='var')
```

Example 19 (python):
```python
ranking genes
    finished: added to `.uns['rank_genes_groups']`
    'names', sorted np.recarray to be indexed by group ids
    'scores', sorted np.recarray to be indexed by group ids
    'logfoldchanges', sorted np.recarray to be indexed by group ids
    'pvals', sorted np.recarray to be indexed by group ids
    'pvals_adj', sorted np.recarray to be indexed by group ids (0:00:00)
    using 'X_pca' with n_pcs = 50
Storing dendrogram info using `.uns['dendrogram_leiden']`
```

Example 20 (python):
```python
cdata=sc.read('./Objects/Annotated/Immune.h5ad')
sc.pl.dotplot(cdata, ['CD74','VPREB3','CD74','CD79B'] ,groupby='new_fine_grain',standard_scale='var',cmap='PRGn')
```

Example 21 (python):
```python
ClusterOfInterest='CD4+Treg'
#sc.tl.rank_genes_groups(adata, 'new_fine_grain', method='t-test', key_added = "t-test")
DEGenesForCluster = [x[ClusterOfInterest] for x in adata.uns['t-test']['names'][:50]]
sc.pl.dotplot(adata, DEGenesForCluster ,groupby='new_fine_grain',standard_scale='var',cmap='PRGn')
#sc.pl.umap(adata, color=DEGenesForCluster)
```

Example 22 (python):
```python
Genes=['TTN','TNNT2','MYOM1','LDB3','ALDH1A2','TNNT1','KRT19','UPK3B','COL6A3','FBLN1','PCOLCE','LRRC17','EMCN','ECSCR','PECAM1','CDH5', 'LAPTM5','PTPRC','LCP1','CD53','PHOX2B','NRXN1','STMN2','GRIK2']
Genes=['PTPRC']
sc.pl.umap(adata, color=Genes)
```

Example 23 (python):
```python
sc.pl.umap(adata, color=['new_fine_grain'],groups=['B','B1','B_cycling','Bpro'])
sc.pl.umap(adata, color=['new_fine_grain'],groups=['CD4+T','
sc.pl.umap(adata, color=['new_fine_grain'],groups=['ILC'])
sc.pl.umap(adata, color=['new_fine_grain'],groups=['MP_CX3CR1+','MP_LYVE1+','MP_LYVE1+Collagen+','MP_LYVE1+JUN+','MP_LYVE1+_cycling','MP_TIMD4+','MP_TREM2+'])
sc.pl.umap(adata, color=['new_fine_grain'],groups=['Mono','Mono_MPO+'])
sc.pl.umap(adata, color=['new_fine_grain'],groups=['Mast cell'])
sc.pl.umap(adata, color=['new_fine_grain'],groups=['DC1','DC2','DC_mature','Plasmacytoid DC'])CD4+Treg','CD8+T','T-NK_cycling','NK'])
```

Example 24 (python):
```python
sc.pl.umap(adata, color=['new_fine_grain'],groups=['DC2','Plasmacytoid DC'])
```

Example 25 (python):
```python
sc.pl.umap(adata, color=['phase'])
```

Example 26 (python):
```python
sc.pl.umap(adata, color=['new_fine_grain'],groups=['MP_CX3CR1+','MP_LYVE1+','MP_LYVE1+Collagen+','MP_LYVE1+JUN+','MP_LYVE1+_cycling','MP_TIMD4+','MP_TREM2+'])
sc.pl.umap(adata, color=['new_fine_grain'],groups=['MP_LYVE1+Collagen+'])
```

Example 27 (python):
```python
sc.tl.rank_genes_groups(adata, groupby='new_fine_grain', method='t-test')
sc.tl.dendrogram(adata, groupby='new_fine_grain')
#sc.pl.dendrogram(adata,groupby='leiden')
sc.pl.rank_genes_groups_matrixplot(adata, groupby='new_fine_grain', standard_scale='var')
```

Example 28 (python):
```python
ranking genes
    finished: added to `.uns['rank_genes_groups']`
    'names', sorted np.recarray to be indexed by group ids
    'scores', sorted np.recarray to be indexed by group ids
    'logfoldchanges', sorted np.recarray to be indexed by group ids
    'pvals', sorted np.recarray to be indexed by group ids
    'pvals_adj', sorted np.recarray to be indexed by group ids (0:00:08)
    using 'X_pca' with n_pcs = 50
Storing dendrogram info using `.uns['dendrogram_new_fine_grain']`
```

Example 29 (python):
```python
sc.pl.dotplot(adata, ['MS4A1','PCDH9','ERG','RORC','CCL5','CD8A','FOXP3','ANK3','GATA2','CUX2','LAMP3','CLEC10A','CLNK','LYVE1','STX11','NLRP3','NR4A3','TIMD4','TREM2','CX3CR1','MPO','PLBD1'] ,groupby='new_fine_grain',standard_scale='var',cmap='PRGn')
```

---

## Functions¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/Visium/Visium_Intagration-Annotation/Visium_integration_no1_bbknn.html

**Contents:**
- Functions¶
- Read in visium anndata¶
- Clustering using cell2location output¶
- with batch correction¶
- Save¶

**Examples:**

Example 1 (python):
```python
import sys
import scanpy as sc
import anndata
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt 
import matplotlib as mpl

from matplotlib import rcParams
rcParams['pdf.fonttype'] = 42 # enables correct plotting of text
import seaborn as sns

import bbknn

import milopy.core as milo
import milopy.plot as milopl

import os
```

Example 2 (python):
```python
sc.settings.set_figure_params(dpi=120)
```

Example 3 (python):
```python
# This function selects the data for multiple slides from the spatial anndata object.
# Modified `select_slide` in cell2loc for multiple slides

# @param adata: anndata object with multiple spatial experiments
# @param s: list of selected slides
# @param s_col: column in adata.obs listing experiment name for each location

def select_multislide(adata, s, s_col='sample'):

    slides = adata[adata.obs[s_col].isin(s), :]
    s_keys = list(slides.uns['spatial'].keys())
    
    uns_dict = {}
    for ss in s:
        s_spatial=np.array(s_keys)[[ss in k for k in s_keys]][0]
        uns_dict[s_spatial]=slides.uns['spatial'][s_spatial]
    
    slides.uns['spatial'] = uns_dict

    return slides
```

Example 4 (unknown):
```unknown
!ls /nfs/team205/heart/anndata_objects/Foetal
```

Example 5 (python):
```python
14Feb_fromSemih		      RNA_foetal_post-scVI_raw.h5ad
archive			      RNA_foetal_raw.h5ad
Feb28ObjectRaw.h5ad	      scArches
Feb28ObjectRaw_Multiome.h5ad  scVI
Feb28ObjectRaw_Multiome.rds   trisomy21
Feb28ObjectRaw.obs.csv	      visium-FFPE_foetal_raw.h5ad
Feb28ObjectRaw_original.h5ad  visium_foetal_healthy-all_raw.h5ad
Feb28ObjectRaw_re-scVI.h5ad   visium_foetal_raw.h5ad
He_Cell_foetal-lung	      visium-OCT_foetal_acrossage_raw.h5ad
Immune			      visium-OCT_foetal_DuctusA_raw.h5ad
metadata		      visium-OCT_foetal_raw.h5ad
multiome_ATAC		      visium-OCT-LV_foetal_raw.h5ad
QC
```

Example 6 (python):
```python
adata_oct = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/visium-OCT_foetal_raw.h5ad')

adata_da = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/visium-OCT_foetal_DuctusA_raw.h5ad')
adata_da.obs['sample_for_cell2loc'] = adata_da.obs['sample'].copy()

# adata_lv = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/visium-OCT-LV_foetal_raw.h5ad')

adata_ffpe = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/visium-FFPE_foetal_raw.h5ad')
adata_ffpe.obs['sample_for_cell2loc'] = adata_ffpe.obs['sample'].copy()
adata_ffpe.obs['cell2loc-map_group'] = 'FFPE_16PCW'
```

Example 7 (python):
```python
adata_vis = adata_oct.concatenate(
        [adata_da,adata_ffpe], # adata_lv,
        batch_key=None,
        # uns_merge="unique",
        index_unique=None,
        join="outer")
adata_vis
```

Example 8 (python):
```python
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/anndata/_core/anndata.py:1785: FutureWarning: X.dtype being converted to np.float32 from float64. In the next version of anndata (0.9) conversion will not be automatic. Pass dtype explicitly to avoid this warning. Pass `AnnData(X, dtype=X.dtype, ...)` to get the future behavour.
  [AnnData(sparse.csr_matrix(a.shape), obs=a.obs) for a in all_adatas],
```

Example 9 (unknown):
```unknown
AnnData object with n_obs × n_vars = 39563 × 33562
    obs: 'in_tissue', 'array_row', 'array_col', 'sample', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'mt_frac', 'sangerID', 'Publication', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'status', 'cell2loc-map_group', 'JC_edit', 'sample_for_cell2loc'
    var: 'gene_ids-0', 'feature_types-0', 'genome-0', 'SYMBOL-0', 'mt-0', 'gene_ids-1', 'feature_types-1', 'genome-1', 'SYMBOL-1', 'mt-1', 'gene_ids-2', 'feature_types-2', 'genome-2', 'SYMBOL-2', 'mt-2'
    obsm: 'MT', 'means_cell_abundance_w_sf', 'q05_cell_abundance_w_sf', 'q95_cell_abundance_w_sf', 'spatial', 'stds_cell_abundance_w_sf'
```

Example 10 (python):
```python
pd.crosstab(adata_vis.obs['sample_for_cell2loc'],adata_vis.obs['cell2loc-map_group'])
```

Example 11 (python):
```python
pd.crosstab(adata_vis.obs['cell2loc-map_group'],adata_vis.obs['JC_edit'],)
```

Example 12 (python):
```python
# make complete SYMBOL column
na_index = adata_vis.var_names[adata_vis.var['SYMBOL-0'].isna()]
adata_vis.var['SYMBOL'] = adata_vis.var['SYMBOL-0'].copy()
adata_vis.var.loc[na_index,'SYMBOL'] = adata_vis.var.loc[na_index,'SYMBOL-2']
adata_vis.var = adata_vis.var[['SYMBOL']]
sum(adata_vis.var['SYMBOL'].isna())
```

Example 13 (python):
```python
# fillna
# "DuctusArteriosusSmoothMuscleCells"
adata_vis.obsm['q05_cell_abundance_w_sf'] = adata_vis.obsm['q05_cell_abundance_w_sf'].fillna(0)
```

Example 14 (python):
```python
# set symboßl as var_names
adata_vis.var['gene_ids'] = adata_vis.var_names.copy()
adata_vis.var.set_index('SYMBOL',inplace=True)

adata_vis
```

Example 15 (unknown):
```unknown
AnnData object with n_obs × n_vars = 39563 × 33562
    obs: 'in_tissue', 'array_row', 'array_col', 'sample', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'mt_frac', 'sangerID', 'Publication', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'status', 'cell2loc-map_group', 'JC_edit', 'sample_for_cell2loc'
    var: 'gene_ids'
    obsm: 'MT', 'means_cell_abundance_w_sf', 'q05_cell_abundance_w_sf', 'q95_cell_abundance_w_sf', 'spatial', 'stds_cell_abundance_w_sf'
```

Example 16 (python):
```python
# remove Cavity
# filter out spots: remove 'Cavity'
adata_vis = adata_vis[adata_vis.obs['JC_edit'].isin(['Cavity'])==False]
```

Example 17 (python):
```python
# set 'week'
adata_vis.obs['week'] = adata_vis.obs['cell2loc-map_group'].astype('str').copy()
adata_vis.obs.replace({'week':{
    '13PCW':'13W', 
    '20PCW':'20W',
    '4PCW':'4W',
    '5PCW':'5W',
    '7PCW':'7W',
    '7PCW_DA':'7W',
    'FFPE_16PCW':'16W'
}},inplace=True)
```

Example 18 (unknown):
```unknown
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/ipykernel_launcher.py:2: ImplicitModificationWarning: Trying to modify attribute `.obs` of view, initializing view as actual.
  
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/anndata/_core/anndata.py:1830: UserWarning: Variable names are not unique. To make them unique, call `.var_names_make_unique`.
  utils.warn_names_duplicates("var")
```

Example 19 (python):
```python
adata_vis.var.head()
```

Example 20 (python):
```python
len(set(adata_vis.obs['sample_for_cell2loc']))
```

Example 21 (python):
```python
# compute KNN using the cell2location output stored in adata.obsm
# input of use_rep should be numpy.ndarray
adata_vis.obsm['q05_cell_abundance_w_sf_ndarray'] = np.array(adata_vis.obsm['q05_cell_abundance_w_sf'])
# bbknn
bbknn.bbknn(adata_vis, batch_key='sample_for_cell2loc', use_rep='q05_cell_abundance_w_sf_ndarray')

# Cluster spots into regions using scanpy
sc.tl.leiden(adata_vis, resolution=1.5)

# add region as categorical variable
adata_vis.obs["region_cluster"] = adata_vis.obs["leiden"].astype("category")
```

Example 22 (python):
```python
'''
# harmony
import scanpy.external as sce
sce.pp.harmony_integrate(adata_vis, 'sample_for_cell2loc', basis='q05_cell_abundance_w_sf_ndarray', adjusted_basis='X_cell2loc_harmony')

# compute KNN using the cell2location output stored in adata.obsm
sc.pp.neighbors(adata_vis, use_rep='X_cell2loc_harmony',
                n_neighbors = 10)

# Cluster spots into regions using scanpy
sc.tl.leiden(adata_vis, resolution=0.5)

# add region as categorical variable
adata_vis.obs["region_cluster"] = adata_vis.obs["leiden"].astype("category")
'''
```

Example 23 (python):
```python
# compute UMAP using KNN graph based on the cell2location output
sc.tl.umap(adata_vis, min_dist = 0.2, spread = 1)  # n_components=3
```

Example 24 (python):
```python
# show regions in UMAP coordinates
with mpl.rc_context({'axes.facecolor':  'white',
                     'figure.figsize': [6, 6]}):
    sc.pl.umap(adata_vis, color=['region_cluster','sample_for_cell2loc'], size=5,
               color_map = 'RdPu', ncols = 3, wspace=0.3,
               legend_fontsize=14, frameon=False, components='all')
```

Example 25 (python):
```python
with mpl.rc_context({'axes.facecolor':  'white',
                     'figure.figsize': [4, 6]}):
    sc.pl.umap(adata_vis, color=['JC_edit','cell2loc-map_group'], size=5,
               color_map = 'RdPu', ncols = 1, wspace=0.7,components='all',
               legend_fontsize=14, frameon=False)
```

Example 26 (python):
```python
with mpl.rc_context({'axes.facecolor':  'white',
                     'figure.figsize': [6, 6]}):
    sc.pl.umap(adata_vis, color=['JC_edit'], groups='AVN', size=5,
               color_map = 'RdPu', ncols = 1, wspace=0.7,
               legend_fontsize=14, frameon=False)
```

Example 27 (python):
```python
adata_vis.obs.drop('in_tissue',axis=1,inplace=True)
adata_vis.obs['array_row'] = adata_vis.obs['array_row'].astype('int')
adata_vis.obs['array_col'] = adata_vis.obs['array_col'].astype('int')
adata_vis.write('/nfs/team205/heart/anndata_objects/Foetal/visium_foetal_healthy-all_raw.h5ad')
```

---

## Variables¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/Xenium/no2_integration_and_cell-type-annotation/no6_manual-annotation_MuralCells.html

**Contents:**
- Variables¶
- Read in data¶
- Markers¶
- 1st round annotation¶
- Subset and reclustering¶
- Transfer annotation to the original adata¶
- Save¶

--> cluster 1 has lower total counts and smaller cell area, and no specific gene expression --> annotate as "unassigned"

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os

import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 

import scanpy.external as sce
```

Example 2 (python):
```python
# import session_info
# session_info.show()
```

Example 3 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/home/kk837/rfs/rfs-teichlab-nfs-iCNyzSAaucw/kk837/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 4 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 5 (python):
```python
adata_dir = '/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium/subsets'
sample_id = 'C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat'
celltype = 'MuralCells'
```

Example 6 (unknown):
```unknown
# figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk/Aug2024'
# table_dir = '/nfs/team205/heart/fetal_ms_figures/tables/Aug2024'
```

Example 7 (unknown):
```unknown
latent_space = 'pca_harmony'
```

Example 8 (python):
```python
# read in
adata = sc.read_h5ad(f'{adata_dir}/{sample_id}_5K_{celltype}_lognorm.h5ad')
adata
```

Example 9 (unknown):
```unknown
AnnData object with n_obs × n_vars = 10986 × 4771
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_3.0', 'multi_celltypes_coarse', 'leiden_0.2', 'leiden_0.4', 'leiden_0.6', 'leiden_1'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1', 'n_cells', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'dendrogram_pca_harmony_leiden_1.0', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1_colors', 'log1p', 'multi_celltypes_coarse_colors', 'pca', 'pca_harmony', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.2_colors', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.3_colors', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.4_colors', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.5_colors', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_0.8_colors', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.0_colors', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_1.5_colors', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_2.0_colors', 'pca_harmony_leiden_3.0', 'pca_harmony_leiden_3.0_colors', 'tissue_block_id_colors', 'umap'
    obsm: 'X_pca', 'X_pca_harmony', 'X_umap_pca_harmony', 'spatial'
    varm: 'PCs'
    obsp: 'pca_harmony_connectivities', 'pca_harmony_distances'
```

Example 10 (python):
```python
adata.X.data[:5]
```

Example 11 (unknown):
```unknown
array([2.999922, 2.999922, 2.999922, 2.999922, 2.999922], dtype=float32)
```

Example 12 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=['total_counts','n_genes_by_counts','cell_area','tissue_block_id'],
                    wspace=0.2,cmap='RdPu',vmax='p100')
```

Example 13 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=['multi_celltypes_coarse',"conf_score_midmod2fine","celltypist_midmod2fine",],
                    wspace=0.2,cmap='RdPu',vmax='p100')
resolutions_list = [0.2,0.4,0.6,1]
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                    color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                    wspace=0.2)
```

Example 14 (python):
```python
%%time
# leiden clustering
resolutions_list = [# 0.2,0.4,0.6,1, # already done
                    1.2,1.5,2]
for resolution in resolutions_list:
    sc.tl.leiden(adata,neighbors_key=latent_space,
                     resolution=resolution,key_added=f'leiden_{str(resolution)}',n_iterations=2)
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                wspace=0.3)
```

Example 15 (unknown):
```unknown
<timed exec>:5: FutureWarning: In the future, the default backend for leiden will be igraph instead of leidenalg.

 To achieve the future defaults please pass: flavor="igraph" and n_iterations=2.  directed must also be False to work with igraph's implementation.
```

Example 16 (unknown):
```unknown
CPU times: user 7.11 s, sys: 112 ms, total: 7.23 s
Wall time: 7.25 s
```

Example 17 (python):
```python
# read in markers identified using single-nucleus data
markers_df = pd.read_csv(f'/home/kk837/rfs/rfs-teichlab-nfs-iCNyzSAaucw/kk837/notebooks/Foetal/VisiumHD/re-seq_mpp0p3/sc_{celltype}_finegrain_markers.csv')

mask1 = markers_df['group_mean']>1
mask2 = markers_df['logfoldchanges']>1
markers_df = markers_df[mask1&mask2]

markers_df
```

Example 18 (python):
```python
markers = {}
n_top = 15
for c in set(markers_df['group']):
    markers_df_sub = markers_df[markers_df['group']==c]
    markers_df_sub.sort_values('logfoldchanges',ascending=False)
    markers[c]=list(markers_df_sub[:n_top]['names'])

# manually add genes
markers['Cycling'] = ['MKI67','TOP2A'] 

genes = []
for g in markers.values():
    genes = genes + [x for x in g if (x in list(adata.var_names))&(x not in genes)]
for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in list(adata.var_names)]
len(genes)
```

Example 19 (python):
```python
sc.pl.embedding(adata,basis=f"X_umap_{latent_space}",color=genes, size=10,
               ncols=6, wspace=0.2,cmap='RdPu',vmax='p100')
```

Example 20 (unknown):
```unknown
resolution_sel = 0.4
```

Example 21 (python):
```python
markers_others = {
    'CM': ['TTN', 'TNNT2','MYH6',
        'MYH7', 'MYL2', 'FHL2', # vCN
            'NPPA', 'MYL7', 'MYL4'], #aCM
    'FB': ['DCN', 'GCN', 'PDGFRA','COL1A1','COL1A2'],
    'EC': ['VWF', 'PECAM1', 'CDH5','RGCC', 'FABP5',],
    'Peri': ['RGS5', 'ABCC9', 'KCNJ8'],
    'SMC': ['MYH11', 'TAGLN', 'ACTA2'],
    'Meso': ['MSLN', 'WT1', 'BNC1'],
    'Neuro': ['PLP1', 'NRXN1', 'NRXN3','PRPH', 'NEFL', 'NEFM', 'NEFH', 'STMN2', 'SST', 'SLC10A4', 'SLC18A3', 'TH', 'NPY'],
    'Adipo': ['GPAM', 'FASN', 'LEP'],
    'Myelo': ['CD14', 'C1QA', 'CD68','LYVE1','TIMD4'],
    'Lymph': ['CD8A', 'IL7R', 'CD40LG'],
    'B': ['CD19','SDC1','IGHG1','IGHA1'],
    'Mast': ['KIT', 'CPA3'],
}

for key in markers_others.keys():
    markers_others[key]=[x for x in markers_others[key] if x in adata.var_names]

sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
sc.pl.dotplot(adata,
              markers_others, 
              groupby=f'leiden_{str(resolution_sel)}',
              dendrogram=True,
              standard_scale="var",
              color_map="Reds",
              swap_axes=False,
             )
```

Example 22 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: CM, FB, EC, etc.
```

Example 23 (python):
```python
sc.tl.dendrogram(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.pl.dotplot(adata,
              var_names=markers,
              groupby=f'leiden_{str(resolution_sel)}',
              standard_scale='var',
              dendrogram=True
             )
```

Example 24 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: CoronaryPericytes, DuctusArteriosusSmoothMuscleCells, CoronarySmoothMuscleCells, etc.
```

Example 25 (python):
```python
print(adata.X.data[:5])
adata.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata, dendrogram=True, standard_scale='var', return_fig=True)
dp.add_totals().show()
```

Example 26 (unknown):
```unknown
[2.999922 2.999922 2.999922 2.999922 2.999922]
```

Example 27 (python):
```python
sc.pl.violin(adata,keys=['total_counts','cell_area'],groupby=f'leiden_{str(resolution_sel)}')
```

Example 28 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=f'leiden_{str(resolution_sel)}',legend_loc='on data',
                )
```

Example 29 (unknown):
```unknown
'MuralCells'
```

Example 30 (python):
```python
# annotate
adata.obs['tmp_annotation'] = adata.obs[f'leiden_{str(resolution_sel)}'].copy()
adata.obs.replace({'tmp_annotation':{
    '0':'MuralCells',
    '1':'unassigned',
    '2':'MuralCells',
    '3':'MuralCells',
    '4':'MuralCells',
}},inplace=True)
adata.obs['tmp_annotation'].value_counts()
```

Example 31 (python):
```python
/tmp/ipykernel_158712/842149592.py:3: FutureWarning: The behavior of Series.replace (and DataFrame.replace) with CategoricalDtype is deprecated. In a future version, replace will only be used for cases that preserve the categories. To change the categories, use ser.cat.rename_categories instead.
  adata.obs.replace({'tmp_annotation':{
```

Example 32 (unknown):
```unknown
tmp_annotation
MuralCells    7486
unassigned    3500
Name: count, dtype: int64
```

Example 33 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color='tmp_annotation'
                )
```

Example 34 (python):
```python
adata_sub = adata[adata.obs['tmp_annotation'].isin(['MuralCells'])]
```

Example 35 (python):
```python
# drop leiden clustering results
resolutions_list = [0.2,0.4,0.6,1, 1.2,1.5,2]
for resolution in resolutions_list:
    adata_sub.obs.drop(f'leiden_{str(resolution)}',axis=1,inplace=True)
adata_sub
```

Example 36 (unknown):
```unknown
AnnData object with n_obs × n_vars = 7486 × 4771
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_3.0', 'multi_celltypes_coarse', 'tmp_annotation'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1', 'n_cells', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'dendrogram_pca_harmony_leiden_1.0', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1_colors', 'log1p', 'multi_celltypes_coarse_colors', 'pca', 'pca_harmony', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.2_colors', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.3_colors', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.4_colors', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.5_colors', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_0.8_colors', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.0_colors', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_1.5_colors', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_2.0_colors', 'pca_harmony_leiden_3.0', 'pca_harmony_leiden_3.0_colors', 'tissue_block_id_colors', 'umap', 'leiden_1.2', 'leiden_1.5', 'leiden_2', 'leiden_1.2_colors', 'leiden_1.5_colors', 'leiden_2_colors', 'dendrogram_leiden_0.4', 'rank_genes_groups', 'tmp_annotation_colors'
    obsm: 'X_pca', 'X_pca_harmony', 'X_umap_pca_harmony', 'spatial'
    varm: 'PCs'
    obsp: 'pca_harmony_connectivities', 'pca_harmony_distances'
```

Example 37 (python):
```python
## Remove mid-grain markers from the feature
# read in markers
midgrain_markers = pd.read_csv('/home/kk837/rfs/rfs-teichlab-nfs-iCNyzSAaucw/kk837/notebooks/Foetal/VisiumHD/re-seq_mpp0p3/sc_midgrain_markers.csv',
                               index_col=0)
# remove the markers of celltype of interest
midgrain_markers = midgrain_markers[midgrain_markers['group']!=celltype]
# remove
mask = [x not in list(midgrain_markers['names']) for x in adata_sub.var_names]
bdata = adata_sub[:,mask]
bdata
```

Example 38 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 7486 × 4298
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_3.0', 'multi_celltypes_coarse', 'tmp_annotation'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1', 'n_cells', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'dendrogram_pca_harmony_leiden_1.0', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1_colors', 'log1p', 'multi_celltypes_coarse_colors', 'pca', 'pca_harmony', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.2_colors', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.3_colors', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.4_colors', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.5_colors', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_0.8_colors', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.0_colors', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_1.5_colors', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_2.0_colors', 'pca_harmony_leiden_3.0', 'pca_harmony_leiden_3.0_colors', 'tissue_block_id_colors', 'umap', 'leiden_1.2', 'leiden_1.5', 'leiden_2', 'leiden_1.2_colors', 'leiden_1.5_colors', 'leiden_2_colors', 'dendrogram_leiden_0.4', 'rank_genes_groups', 'tmp_annotation_colors'
    obsm: 'X_pca', 'X_pca_harmony', 'X_umap_pca_harmony', 'spatial'
    varm: 'PCs'
    obsp: 'pca_harmony_connectivities', 'pca_harmony_distances'
```

Example 39 (python):
```python
'''
# remove cell cycle genes
cell_cycle_genes = ['MCM5','PCNA','TYMS','FEN1','MCM2','MCM4','RRM1','UNG','GINS2','MCM6','CDCA7','DTL','PRIM1','UHRF1','MLF1IP','HELLS','RFC2','RPA2','NASP',
                'RAD51AP1','GMNN','WDR76','SLBP','CCNE2','UBR7','POLD3','MSH2','ATAD2','RAD51','RRM2','CDC45','CDC6','EXO1','TIPIN','DSCC1','BLM','CASP8AP2',
                'USP1','CLSPN','POLA1','CHAF1B','BRIP1','E2F8','HMGB2','CDK1','NUSAP1','UBE2C','BIRC5','TPX2','TOP2A','NDC80','CKS2','NUF2','CKS1B','MKI67',
                'TMPO','CENPF','TACC3','FAM64A','SMC4','CCNB2','CKAP2L','CKAP2','AURKB','BUB1','KIF11','ANP32E','TUBB4B','GTSE1','KIF20B','HJURP','CDCA3','HN1',
                'CDC20','TTK','CDC25C','KIF2C','RANGAP1','NCAPD2','DLGAP5','CDCA2','CDCA8','ECT2','KIF23','HMMR','AURKA','PSRC1','ANLN','LBR','CKAP5','CENPE','CTCF',
                'NEK2','G2E3','GAS2L3','CBX5','CENPA']
mask = [x not in cell_cycle_genes for x in bdata.var_names]
bdata = bdata[:,mask]
bdata.shape
'''
```

Example 40 (python):
```python
# HVGs, using the object which doesn't have midgrain markers
sc.pp.highly_variable_genes(bdata, flavor='seurat', min_mean=0.0125, max_mean=3, min_disp=0.5)
print(f'number of hvgs: {sum(bdata.var.highly_variable)}')
# PCA using the degected HVGs
sc.tl.pca(bdata, use_highly_variable=True)
adata_sub.obsm['X_pca'] = bdata.obsm['X_pca'].copy()
# harmony integration
sce.pp.harmony_integrate(adata_sub, 'tissue_block_id', basis='X_pca', adjusted_basis='X_pca_harmony')
# create neighbour graph
latent_space = 'pca_harmony'
sc.pp.neighbors(adata_sub,n_neighbors=15,use_rep=f"X_{latent_space}",key_added=latent_space)
# visualisation with umap
sc.tl.umap(adata_sub,min_dist=0.5,neighbors_key=latent_space)
adata_sub.obsm[f"X_umap_{latent_space}"]=adata_sub.obsm["X_umap"]
del adata_sub.obsm["X_umap"]
```

Example 41 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_highly_variable_genes.py:698: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  adata.uns["hvg"] = {"flavor": flavor}
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_pca/__init__.py:438: FutureWarning: Argument `use_highly_variable` is deprecated, consider using the mask argument. Use_highly_variable=True can be called through mask_var="highly_variable". Use_highly_variable=False can be called through mask_var=None
  warn(msg, FutureWarning)
```

Example 42 (unknown):
```unknown
number of hvgs: 821
```

Example 43 (python):
```python
2025-06-12 16:43:00,616 - harmonypy - INFO - Computing initial centroids with sklearn.KMeans...
2025-06-12 16:43:02,801 - harmonypy - INFO - sklearn.KMeans initialization complete.
2025-06-12 16:43:02,917 - harmonypy - INFO - Iteration 1 of 10
2025-06-12 16:43:03,945 - harmonypy - INFO - Iteration 2 of 10
2025-06-12 16:43:04,940 - harmonypy - INFO - Converged after 2 iterations
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm
```

Example 44 (python):
```python
%%time
# reclustering
# leiden clustering
resolutions_list = [0.2,0.4,0.6,1, 1.2,1.5,2]
for resolution in resolutions_list:
    sc.tl.leiden(adata_sub,neighbors_key=latent_space,
                 resolution=resolution,key_added=f'leiden_{str(resolution)}',n_iterations=2)
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                wspace=0.3)
```

Example 45 (unknown):
```unknown
CPU times: user 10.7 s, sys: 129 ms, total: 10.8 s
Wall time: 10.9 s
```

Example 46 (python):
```python
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",size=10,
                color=genes,
               ncols=6, wspace=0.2,cmap='RdPu',vmax='p100')
```

Example 47 (python):
```python
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color=["conf_score_midmod2fine","celltypist_midmod2fine",],
                    wspace=0.2,cmap='RdPu',vmax='p100')
```

Example 48 (python):
```python
# umap per celltype to highlight
group_col = 'celltypist_midmod2fine'
groups_to_plot = list(set(adata_sub.obs[group_col]))

plt.rcParams['axes.grid'] = False
fig,axs=plt.subplots(nrows=2,ncols=4,figsize=(15,8))
# Flatten the axs array for easier iteration in case of multiple rows and columns
axs_flat = axs.flatten()

for i,grp in enumerate(groups_to_plot):
    sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                    color=[group_col], groups=grp, 
               size=10, frameon=False,
               show=False, ax=axs_flat[i], legend_loc=None, legend_fontsize=5,
               title=grp
              )
# Turn off any remaining subplots
for j in range(len(groups_to_plot), len(axs_flat)):
    fig.delaxes(axs_flat[j])

# fig.suptitle('Main Title for the Figure')
fig.tight_layout()
```

Example 49 (unknown):
```unknown
sub_resolution_sel = 0.6
```

Example 50 (python):
```python
markers_others = {
    'CM': ['TTN', 'TNNT2','MYH6',
        'MYH7', 'MYL2', 'FHL2', # vCN
            'NPPA', 'MYL7', 'MYL4'], #aCM
    'FB': ['DCN', 'GCN', 'PDGFRA','COL1A1','COL1A2'],
    'EC': ['VWF', 'PECAM1', 'CDH5','RGCC', 'FABP5',],
    'Peri': ['RGS5', 'ABCC9', 'KCNJ8'],
    'SMC': ['MYH11', 'TAGLN', 'ACTA2'],
    'Meso': ['MSLN', 'WT1', 'BNC1'],
    'Neuro': ['PLP1', 'NRXN1', 'NRXN3','PRPH', 'NEFL', 'NEFM', 'NEFH', 'STMN2', 'SST', 'SLC10A4', 'SLC18A3', 'TH', 'NPY'],
    'Adipo': ['GPAM', 'FASN', 'LEP'],
    'Myelo': ['CD14', 'C1QA', 'CD68','LYVE1','TIMD4'],
    'Lymph': ['CD8A', 'IL7R', 'CD40LG'],
    'B': ['CD19','SDC1','IGHG1','IGHA1'],
    'Mast': ['KIT', 'CPA3'],
}

for key in markers_others.keys():
    markers_others[key]=[x for x in markers_others[key] if x in adata.var_names]

sc.tl.dendrogram(adata_sub, groupby=f'leiden_{str(sub_resolution_sel)}')
sc.pl.dotplot(adata_sub,
              markers_others, 
              groupby=f'leiden_{str(sub_resolution_sel)}',
              dendrogram=True,
              standard_scale="var",
              color_map="Reds",
              swap_axes=False,
             )
```

Example 51 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: CM, FB, EC, etc.
```

Example 52 (python):
```python
sc.tl.dendrogram(adata_sub,groupby=f'leiden_{str(sub_resolution_sel)}')
sc.pl.dotplot(adata_sub,
              var_names=markers,
              groupby=f'leiden_{str(sub_resolution_sel)}',
              standard_scale='var',
              dendrogram=True
             )
```

Example 53 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: CoronaryPericytes, DuctusArteriosusSmoothMuscleCells, CoronarySmoothMuscleCells, etc.
```

Example 54 (python):
```python
# DEG analysis
print(adata_sub.X.data[:5])
adata_sub.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata_sub,groupby=f'leiden_{str(sub_resolution_sel)}')
sc.tl.dendrogram(adata_sub, groupby=f'leiden_{str(sub_resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata_sub, dendrogram=True, standard_scale='var', return_fig=True)
dp.add_totals().show()
```

Example 55 (unknown):
```unknown
[2.4700055 3.11994   3.11994   3.7907598 2.4700055]
```

Example 56 (python):
```python
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color=f'leiden_{str(sub_resolution_sel)}',legend_loc='on data',
                wspace=0.3)
```

Example 57 (python):
```python
# annotate
adata_sub.obs['fine_grain_manual'] = adata_sub.obs[f'leiden_{str(sub_resolution_sel)}'].copy()
adata_sub.obs.replace({'fine_grain_manual':{
    '0':'GreatVesselSmoothMuscleCells',
    '1':'CoronaryPericytes',
    '2':'GreatVesselSmoothMuscleCellsCycling',
    '3':'CoronarySmoothMuscleCells',
    '4':'DuctusArteriosusSmoothMuscleCells',
}},inplace=True)
adata_sub.obs['fine_grain_manual'].value_counts()
```

Example 58 (python):
```python
/tmp/ipykernel_158712/3157763043.py:3: FutureWarning: The behavior of Series.replace (and DataFrame.replace) with CategoricalDtype is deprecated. In a future version, replace will only be used for cases that preserve the categories. To change the categories, use ser.cat.rename_categories instead.
  adata_sub.obs.replace({'fine_grain_manual':{
```

Example 59 (unknown):
```unknown
fine_grain_manual
GreatVesselSmoothMuscleCells           2358
CoronaryPericytes                      1990
GreatVesselSmoothMuscleCellsCycling    1312
CoronarySmoothMuscleCells              1026
DuctusArteriosusSmoothMuscleCells       800
Name: count, dtype: int64
```

Example 60 (python):
```python
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color='fine_grain_manual',
                wspace=0.3)
```

Example 61 (python):
```python
sc.tl.dendrogram(adata_sub,groupby='fine_grain_manual')
sc.pl.dotplot(adata_sub,
              var_names=markers,
              groupby='fine_grain_manual',
              standard_scale='var',
              dendrogram=True
             )
```

Example 62 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: GreatVesselSmoothMuscleCells, CoronaryPericytes, GreatVesselSmoothMuscleCellsCycling, etc.
var_group_labels: CoronaryPericytes, DuctusArteriosusSmoothMuscleCells, CoronarySmoothMuscleCells, etc.
```

Example 63 (python):
```python
adata.obs['fine_grain_manual'] = adata.obs['tmp_annotation'].astype('str').copy()
adata.obs.loc[adata_sub.obs_names,'fine_grain_manual'] = adata_sub.obs['fine_grain_manual']
```

Example 64 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color='fine_grain_manual'
                )
```

Example 65 (python):
```python
sc.tl.dendrogram(adata,groupby='fine_grain_manual')
sc.pl.dotplot(adata,
              var_names=markers,
              groupby='fine_grain_manual',
              standard_scale='var',
              dendrogram=True
             )
```

Example 66 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: CoronaryPericytes, CoronarySmoothMuscleCells, DuctusArteriosusSmoothMuscleCells, etc.
var_group_labels: CoronaryPericytes, DuctusArteriosusSmoothMuscleCells, CoronarySmoothMuscleCells, etc.
```

Example 67 (python):
```python
for ct in set(adata.obs['fine_grain_manual']):
    for tissue_id in ['C194-HEA-0-FFPE-1', 'Hst45-HEA-0-FFPE-1']:
        slide = adata[adata.obs['tissue_block_id']==tissue_id]
        sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
                      spot_size=60,vmax='p100')
```

Example 68 (python):
```python
/tmp/ipykernel_158712/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 69 (python):
```python
/tmp/ipykernel_158712/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 70 (python):
```python
/tmp/ipykernel_158712/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 71 (python):
```python
/tmp/ipykernel_158712/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 72 (python):
```python
/tmp/ipykernel_158712/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 73 (python):
```python
/tmp/ipykernel_158712/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 74 (python):
```python
/tmp/ipykernel_158712/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 75 (python):
```python
/tmp/ipykernel_158712/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 76 (python):
```python
/tmp/ipykernel_158712/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 77 (python):
```python
/tmp/ipykernel_158712/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 78 (python):
```python
/tmp/ipykernel_158712/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 79 (python):
```python
/tmp/ipykernel_158712/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 80 (python):
```python
# save
adata.write(f'{adata_dir}/{sample_id}_5K_{celltype}_lognorm.h5ad')
print(f'{adata_dir}/{sample_id}_5K_{celltype}_lognorm.h5ad')
```

Example 81 (unknown):
```unknown
/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium/subsets/C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat_5K_MuralCells_lognorm.h5ad
```

---

## Creat Arrow files¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/ATAC/ATAC_preprocess/ArchR_no1_createArrowFiles.html

**Contents:**
- Creat Arrow files¶

**Examples:**

Example 1 (unknown):
```unknown
library(ArchR)
library(tidyverse)
```

Example 2 (python):
```python
# Prepare input files
metadata = read.csv('/nfs/team205/heart/anndata_objects/Foetal/metadata/HeartTeamSamples_Mappeddata_20230208.csv')
metadata = metadata %>% filter(Publication=='Foetal')
metadata = metadata %>% filter(status=='Healthy')
metadata = metadata %>% filter(modality=='Multiome-ATAC')
nrow(metadata)

inputFiles = paste0(metadata %>% pull('Mapping_farm'),'/atac_fragments.tsv.gz')
names(inputFiles) = metadata %>% pull('CombinedID_Multiome')
inputFiles
```

Example 3 (unknown):
```unknown
# before starting a project we must set the ArchRGenome and default threads for parallelization.
# Setting default genome to Hg38.
addArchRGenome("hg38")
```

Example 4 (unknown):
```unknown
Setting default genome to Hg38.
```

Example 5 (unknown):
```unknown
# Setting default number of Parallel threads to 16
addArchRThreads(threads = 5)
```

Example 6 (unknown):
```unknown
Setting default number of Parallel threads to 5.
```

Example 7 (unknown):
```unknown
# ArrowFiles <- createArrowFiles(
#   inputFiles = inputFiles,
#   sampleNames = names(inputFiles),
#   minTSS = 4, #Dont set this too high because you can always increase later
#   minFrags = 1000, 
#   addTileMat = TRUE,
#   addGeneScoreMat = TRUE
# )
```

Example 8 (unknown):
```unknown
Using GeneAnnotation set by addArchRGenome(Hg38)!

Using GeneAnnotation set by addArchRGenome(Hg38)!

ArchR logging to : ArchRLogs/ArchR-createArrows-e075303579c-Date-2023-03-08_Time-01-11-16.log
If there is an issue, please report to github with logFile!

Cleaning Temporary Files

2023-03-08 01:11:17 : Batch Execution w/ safelapply!, 0 mins elapsed.
```

Example 9 (unknown):
```unknown
done_files = list.files('/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR',pattern = '.arrow')
inputFiles_forrerun = inputFiles[setdiff(names(inputFiles),unlist(strsplit(done_files,'.arrow')))]
inputFiles_forrerun
```

Example 10 (unknown):
```unknown
ArrowFiles <- createArrowFiles(
  inputFiles = inputFiles_forrerun,
  sampleNames = names(inputFiles_forrerun),
  minTSS = 4, #Dont set this too high because you can always increase later
  minFrags = 1000, 
  addTileMat = TRUE,
  addGeneScoreMat = TRUE
)
```

Example 11 (unknown):
```unknown
Using GeneAnnotation set by addArchRGenome(Hg38)!

Using GeneAnnotation set by addArchRGenome(Hg38)!

ArchR logging to : ArchRLogs/ArchR-createArrows-35a75076e3b-Date-2023-03-08_Time-18-21-49.log
If there is an issue, please report to github with logFile!

Cleaning Temporary Files

2023-03-08 18:21:50 : Batch Execution w/ safelapply!, 0 mins elapsed.

ArchR logging successful to : ArchRLogs/ArchR-createArrows-35a75076e3b-Date-2023-03-08_Time-18-21-49.log
```

Example 12 (unknown):
```unknown
?createArrowFiles
```

---

## Read in ArchR project¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/ATAC/ATAC_preprocess/ArchR_no5-2_Project-to-Matrix.html

**Contents:**
- Read in ArchR project¶
- Peak matrix¶
- Save as matrix¶
- GeneScoreMatrix¶
- Save as matrix¶

Ref: https://www.archrproject.com/bookdown/creating-an-archrproject-1.html

**Examples:**

Example 1 (unknown):
```unknown
library(ArchR)
library(tidyverse)
library(BSgenome.Hsapiens.UCSC.hg38)
library(SingleCellExperiment)
```

Example 2 (python):
```python
/ |
                                                 /    \
            .                                  /      |.
            \\\                              /        |.
              \\\                          /           `|.
                \\\                      /              |.
                  \                    /                |\
                  \\#####\           /                  ||
                ==###########>      /                   ||
                 \\##==......\    /                     ||
            ______ =       =|__ /__                     ||      \\\
        ,--' ,----`-,__ ___/'  --,-`-===================##========>
       \               '        ##_______ _____ ,--,__,=##,__   ///
        ,    __==    ___,-,__,--'#'  ==='      `-'    | ##,-/
        -,____,---'       \\####\\________________,--\\_##,/
           ___      .______        ______  __    __  .______      
          /   \     |   _  \      /      ||  |  |  | |   _  \     
         /  ^  \    |  |_)  |    |  ,----'|  |__|  | |  |_)  |    
        /  /_\  \   |      /     |  |     |   __   | |      /     
       /  _____  \  |  |\  \\___ |  `----.|  |  |  | |  |\  \\___.
      /__/     \__\ | _| `._____| \______||__|  |__| | _| `._____|
    

ArchR : Version 1.0.2
For more information see our website : www.ArchRProject.com
If you encounter a bug please report : https://github.com/GreenleafLab/ArchR/issues

Loading Required Packages...

	Loading Package : grid v3.6.1

	Loading Package : gridExtra v2.3

	Loading Package : gtools v3.9.4

	Loading Package : gtable v0.3.3

	Loading Package : ggplot2 v3.4.2

	Loading Package : magrittr v2.0.3

	Loading Package : plyr v1.8.8

	Loading Package : stringr v1.5.0

	Loading Package : data.table v1.14.8

	Loading Package : matrixStats v1.0.0

	Loading Package : S4Vectors v0.24.4

	Loading Package : GenomicRanges v1.38.0

	Loading Package : BiocGenerics v0.32.0

	Loading Package : Matrix v1.5.4.1

	Loading Package : Rcpp v1.0.10

	Loading Package : SummarizedExperiment v1.16.1

	Loading Package : rhdf5 v2.30.1

── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
✔ dplyr     1.1.2     ✔ readr     2.1.4
✔ forcats   1.0.0     ✔ tibble    3.2.1
✔ lubridate 1.9.2     ✔ tidyr     1.3.0
✔ purrr     1.0.1     
── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
✖ lubridate::%within%()    masks IRanges::%within%()
✖ dplyr::arrange()         masks plyr::arrange()
✖ dplyr::between()         masks data.table::between()
✖ dplyr::collapse()        masks IRanges::collapse()
✖ dplyr::combine()         masks Biobase::combine(), BiocGenerics::combine(), gridExtra::combine()
✖ purrr::compact()         masks plyr::compact()
✖ dplyr::count()           masks matrixStats::count(), plyr::count()
✖ dplyr::desc()            masks IRanges::desc(), plyr::desc()
✖ tidyr::expand()          masks Matrix::expand(), S4Vectors::expand()
✖ tidyr::extract()         masks magrittr::extract()
✖ dplyr::failwith()        masks plyr::failwith()
✖ dplyr::filter()          masks stats::filter()
✖ dplyr::first()           masks S4Vectors::first(), data.table::first()
✖ lubridate::hour()        masks data.table::hour()
✖ dplyr::id()              masks plyr::id()
✖ lubridate::isoweek()     masks data.table::isoweek()
✖ dplyr::lag()             masks stats::lag()
✖ dplyr::last()            masks data.table::last()
✖ lubridate::mday()        masks data.table::mday()
✖ lubridate::minute()      masks data.table::minute()
✖ lubridate::month()       masks data.table::month()
✖ dplyr::mutate()          masks plyr::mutate()
✖ tidyr::pack()            masks Matrix::pack()
✖ BiocGenerics::Position() masks ggplot2::Position(), base::Position()
✖ lubridate::quarter()     masks data.table::quarter()
✖ purrr::reduce()          masks GenomicRanges::reduce(), IRanges::reduce()
✖ dplyr::rename()          masks S4Vectors::rename(), plyr::rename()
✖ lubridate::second()      masks S4Vectors::second(), data.table::second()
✖ lubridate::second<-()    masks S4Vectors::second<-()
✖ purrr::set_names()       masks magrittr::set_names()
✖ purrr::simplify()        masks DelayedArray::simplify()
✖ dplyr::slice()           masks IRanges::slice()
✖ dplyr::summarise()       masks plyr::summarise()
✖ dplyr::summarize()       masks plyr::summarize()
✖ purrr::transpose()       masks data.table::transpose()
✖ tidyr::unpack()          masks Matrix::unpack()
✖ lubridate::wday()        masks data.table::wday()
✖ lubridate::week()        masks data.table::week()
✖ lubridate::yday()        masks data.table::yday()
✖ lubridate::year()        masks data.table::year()
ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors
Loading required package: BSgenome

Loading required package: Biostrings

Loading required package: XVector


Attaching package: ‘XVector’


The following object is masked from ‘package:purrr’:

    compact


The following object is masked from ‘package:plyr’:

    compact



Attaching package: ‘Biostrings’


The following object is masked from ‘package:base’:

    strsplit


Loading required package: rtracklayer
```

Example 3 (unknown):
```unknown
# before starting a project we must set the ArchRGenome and default threads for parallelization.
# Setting default genome to Hg38.
addArchRGenome("hg38")
```

Example 4 (unknown):
```unknown
Setting default genome to Hg38.
```

Example 5 (unknown):
```unknown
# Setting default number of Parallel threads to 16
addArchRThreads(threads = 1)
```

Example 6 (unknown):
```unknown
Setting default number of Parallel threads to 1.
```

Example 7 (unknown):
```unknown
archr_project_path = '/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output'
proj = loadArchRProject(path = archr_project_path, showLogo = FALSE)
proj
```

Example 8 (unknown):
```unknown
Successfully loaded ArchRProject!


           ___      .______        ______  __    __  .______      
          /   \     |   _  \      /      ||  |  |  | |   _  \     
         /  ^  \    |  |_)  |    |  ,----'|  |__|  | |  |_)  |    
        /  /_\  \   |      /     |  |     |   __   | |      /     
       /  _____  \  |  |\  \\___ |  `----.|  |  |  | |  |\  \\___.
      /__/     \__\ | _| `._____| \______||__|  |__| | _| `._____|
```

Example 9 (unknown):
```unknown
class: ArchRProject 
outputDirectory: /nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output 
samples(37): BHF_F_Hea11064670_BHF_F_Hea11031823
  BHF_F_Hea11064671_BHF_F_Hea11031824 ...
  HCAHeartST13386009_HCAHeartST13303419
  HCAHeartST13386010_HCAHeartST13303420
sampleColData names(1): ArrowFiles
cellColData names(63): Sample TSSEnrichment ... Gex_MitoRatio
  Gex_RiboRatio
numberOfCells(1): 167022
medianTSS(1): 11.614
medianFrags(1): 10338.5
```

Example 10 (unknown):
```unknown
PeakMatrix = getMatrixFromProject(
             ArchRProj = proj,
              useMatrix = "PeakMatrix",
              useSeqnames = NULL,
              verbose = TRUE,
              binarize = FALSE,
              threads = getArchRThreads(),
              logFile = createLogFile("getMatrixFromProject")
             )
```

Example 11 (unknown):
```unknown
ArchR logging to : ArchRLogs/ArchR-getMatrixFromProject-23502ed73aec-Date-2023-03-12_Time-19-51-39.log
If there is an issue, please report to github with logFile!

2023-03-12 20:13:14 : Organizing colData, 21.576 mins elapsed.

2023-03-12 20:13:16 : Organizing rowData, 21.621 mins elapsed.

2023-03-12 20:13:16 : Organizing rowRanges, 21.622 mins elapsed.

2023-03-12 20:13:16 : Organizing Assays (1 of 1), 21.624 mins elapsed.

2023-03-12 20:23:20 : Constructing SummarizedExperiment, 31.689 mins elapsed.

2023-03-12 20:29:42 : Finished Matrix Creation, 38.046 mins elapsed.
```

Example 12 (unknown):
```unknown
# get peak names as ranges
PeakSet = getPeakSet(proj)
df = data.frame(PeakSet)
ranges = paste0(df[['seqnames']],':',df[['start']],'_',df[['end']])
head(ranges)

# add rawnames of PeakMatrix
rownames(PeakMatrix) = ranges
PeakMatrix

# add rawnames of PeakMatrix binarised
# rownames(PeakMatrix_bi) = ranges
# PeakMatrix_bi
```

Example 13 (python):
```python
class: RangedSummarizedExperiment 
dim: 508040 167022 
metadata(0):
assays(1): PeakMatrix
rownames(508040): chr1:817093_817593 chr1:819818_820318 ...
  chrX:155881064_155881564 chrX:155881618_155882118
rowData names(1): idx
colnames(167022): 7089STDY13216921_BHF_F_Hea13242528#GCAATAGAGTTATGTG-1
  7089STDY13216921_BHF_F_Hea13242528#TGGGCCTAGATGGACA-1 ...
  BHF_F_Hea11933675_BHF_F_Hea11596628#GTACTAATCGTTTCCA-1
  7089STDY13216923_BHF_F_Hea13242530#GCTGGTTCAAATTCGT-1
colData names(63): BlacklistRatio Gex_MitoRatio ... ReadsInPeaks FRIP
```

Example 14 (unknown):
```unknown
# save as cell-by-peak
writeMM(obj = assay(PeakMatrix), file="/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/PeakMatrix/matrix.mtx")
```

Example 15 (unknown):
```unknown
write.table(rownames(PeakMatrix), file="/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/PeakMatrix/features.tsv",
           col.names = FALSE, row.names = FALSE, quote = FALSE, sep='\t')
```

Example 16 (unknown):
```unknown
# row.names = TRUE to put the features in the 2nd column (for scanpy.read_10x_mtx later we use)
write.table(colnames(PeakMatrix), file="/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/PeakMatrix/barcodes.tsv",
           col.names = FALSE, row.names = TRUE, quote = FALSE, sep='\t')
```

Example 17 (unknown):
```unknown
colnames(PeakMatrix)
```

Example 18 (unknown):
```unknown
# at farm
gzip -v /nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/PeakMatrix/matrix.mtx
# gzip -v /nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/PeakMatrix/matrix_binarised.mtx
gzip -v /nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/PeakMatrix/features.tsv
gzip -v /nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/PeakMatrix/barcodes.tsv
```

Example 19 (unknown):
```unknown
GeneScoreMatrix = getMatrixFromProject(
             ArchRProj = proj,
              useMatrix = "GeneScoreMatrix",
              useSeqnames = NULL,
              verbose = TRUE,
              binarize = FALSE,
              threads = getArchRThreads(),
              logFile = createLogFile("getMatrixFromProject")
             )
```

Example 20 (unknown):
```unknown
ArchR logging to : ArchRLogs/ArchR-getMatrixFromProject-2d21e6a568ca7-Date-2023-11-05_Time-22-34-07.log
If there is an issue, please report to github with logFile!

2023-11-05 23:01:34 : Organizing colData, 27.446 mins elapsed.

2023-11-05 23:01:38 : Organizing rowData, 27.515 mins elapsed.

2023-11-05 23:01:38 : Organizing rowRanges, 27.515 mins elapsed.

2023-11-05 23:01:38 : Organizing Assays (1 of 1), 27.516 mins elapsed.

2023-11-05 23:15:40 : Constructing SummarizedExperiment, 41.545 mins elapsed.

2023-11-05 23:15:45 : Finished Matrix Creation, 41.624 mins elapsed.
```

Example 21 (unknown):
```unknown
GeneScoreMatrix
```

Example 22 (python):
```python
class: SummarizedExperiment 
dim: 24919 167022 
metadata(0):
assays(1): GeneScoreMatrix
rownames: NULL
rowData names(6): seqnames start ... name idx
colnames(167022): 7089STDY13216921_BHF_F_Hea13242528#GCAATAGAGTTATGTG-1
  7089STDY13216921_BHF_F_Hea13242528#TGGGCCTAGATGGACA-1 ...
  BHF_F_Hea11933675_BHF_F_Hea11596628#GTACTAATCGTTTCCA-1
  7089STDY13216923_BHF_F_Hea13242530#GCTGGTTCAAATTCGT-1
colData names(63): BlacklistRatio Gex_MitoRatio ... ReadsInPeaks FRIP
```

Example 23 (unknown):
```unknown
length(colnames(GeneScoreMatrix))
head(colnames(GeneScoreMatrix))
```

Example 24 (unknown):
```unknown
# add rawnames of GeneScoreMatrix
rownames(GeneScoreMatrix) = rowData(GeneScoreMatrix)$name
GeneScoreMatrix
```

Example 25 (python):
```python
class: SummarizedExperiment 
dim: 24919 167022 
metadata(0):
assays(1): GeneScoreMatrix
rownames(24919): OR4F5 LOC729737 ... F8A2 TMLHE
rowData names(6): seqnames start ... name idx
colnames(167022): 7089STDY13216921_BHF_F_Hea13242528#GCAATAGAGTTATGTG-1
  7089STDY13216921_BHF_F_Hea13242528#TGGGCCTAGATGGACA-1 ...
  BHF_F_Hea11933675_BHF_F_Hea11596628#GTACTAATCGTTTCCA-1
  7089STDY13216923_BHF_F_Hea13242530#GCTGGTTCAAATTCGT-1
colData names(63): BlacklistRatio Gex_MitoRatio ... ReadsInPeaks FRIP
```

Example 26 (unknown):
```unknown
# save
writeMM(obj = assay(GeneScoreMatrix), file="/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/GeneScoreMatrix/matrix.mtx")
```

Example 27 (unknown):
```unknown
head(rownames(GeneScoreMatrix))
head(colnames(GeneScoreMatrix))
```

Example 28 (unknown):
```unknown
write.table(rownames(GeneScoreMatrix), file="/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/GeneScoreMatrix/features.tsv",
           col.names = FALSE, row.names = FALSE, quote = FALSE, sep='\t')

# row.names = TRUE to put the features in the 2nd column (for scanpy.read_10x_mtx later we use)
write.table(colnames(GeneScoreMatrix), file="/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/GeneScoreMatrix/barcodes.tsv",
           col.names = FALSE, row.names = TRUE, quote = FALSE, sep='\t')
```

Example 29 (unknown):
```unknown
%%bash
gzip -v /nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/GeneScoreMatrix/matrix.mtx
gzip -v /nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/GeneScoreMatrix/features.tsv
gzip -v /nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/GeneScoreMatrix/barcodes.tsv
```

Example 30 (unknown):
```unknown
Error in parse(text = x, srcfile = src): <text>:1:1: unexpected input
1: %bash
    ^
Traceback:
```

---

## 

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/RNA/scnRNA_Integration-Annotation/Annotations/EpicardiumAnnotation.html

**Examples:**

Example 1 (unknown):
```unknown
#Epicardial annotation
```

Example 2 (python):
```python
# Import packages, initial settings

import numpy as np
import matplotlib.pyplot as plt
import scanpy as sc
import pandas as pd
import anndata
import scrublet as scr
import seaborn as sns
import magic
import pybiomart
import six
import sys
sys.modules['sklearn.externals.six'] = six
import scanorama
import harmonypy
import bbknn
import gseapy
from gseapy.plot import barplot, dotplot
import scvelo as scv
import scvi

plt.rcParams['pdf.fonttype'] = 42
plt.rcParams['ps.fonttype'] = 42

sc.settings.verbosity = 3  # verbosity: errors (0), warnings (1), info (2), hints (3)

from matplotlib.colors import LinearSegmentedColormap
cmap = LinearSegmentedColormap.from_list(name='gene_cmap', colors=['lightgrey', 'thistle', 'red', 'darkred']) 

sc.settings.set_figure_params(dpi=80, color_map='viridis', vector_friendly=False,  dpi_save=300, facecolor='white')
sc.logging.print_version_and_date()
#sc.logging.print_versions()
```

Example 3 (unknown):
```unknown
Global seed set to 0
```

Example 4 (unknown):
```unknown
Running Scanpy 1.8.1, on 2022-12-20 11:13.
```

Example 5 (python):
```python
adata=sc.read('./Objects/20221130RNA_foetal_post-scVI_raw_ExtracardiacRemoved.h5ad')
print(adata.shape)
```

Example 6 (unknown):
```unknown
(306287, 36601)
```

Example 7 (python):
```python
cell_cycle_genes = ['MCM5','PCNA','TYMS','FEN1','MCM2','MCM4','RRM1','UNG','GINS2','MCM6','CDCA7','DTL','PRIM1','UHRF1','MLF1IP','HELLS','RFC2','RPA2','NASP','RAD51AP1','GMNN','WDR76','SLBP','CCNE2','UBR7','POLD3','MSH2','ATAD2','RAD51','RRM2','CDC45','CDC6','EXO1','TIPIN','DSCC1','BLM','CASP8AP2','USP1','CLSPN','POLA1','CHAF1B','BRIP1','E2F8','HMGB2','CDK1','NUSAP1','UBE2C','BIRC5','TPX2','TOP2A','NDC80','CKS2','NUF2','CKS1B','MKI67','TMPO','CENPF','TACC3','FAM64A','SMC4','CCNB2','CKAP2L','CKAP2','AURKB','BUB1','KIF11','ANP32E','TUBB4B','GTSE1','KIF20B','HJURP','CDCA3','HN1','CDC20','TTK','CDC25C','KIF2C','RANGAP1','NCAPD2','DLGAP5','CDCA2','CDCA8','ECT2','KIF23','HMMR','AURKA','PSRC1','ANLN','LBR','CKAP5','CENPE','CTCF','NEK2','G2E3','GAS2L3','CBX5','CENPA']
s_genes = cell_cycle_genes[:43]
s_genes = [x for x in s_genes if x in adata.var_names]
print('S genes:', s_genes, '\n')
g2m_genes = cell_cycle_genes[43:]
g2m_genes = [x for x in g2m_genes if x in adata.var_names]
print('G2 genes:', g2m_genes, '\n')
cell_cycle_genes = [x for x in cell_cycle_genes if x in adata.var_names]

hb_genes1 = ['HBB','HBD','HBG1','HBG2','HBE1','HBZ','HBM','HBA2','HBA1', 'HBQ1']
hb_genes1 = [x for x in hb_genes1 if x in adata.var_names]
print('Hemoglobin genes set 1:', hb_genes1, '\n')

hb_genes2 = ['AHSP','GYPA','RHAG','GFI1B','KEL','EPB42','ALAS2','TSPO2','GYPB','GYPE','SPTA1','RHCE','KLF1','RHD','SLC4A1','SNX22','SQSTM1','SOD1','FTH1','MAP1LC3B','SLC25A37','TRIM58','HMBS']
hb_genes2 = [x for x in hb_genes2 if x in adata.var_names]
print('Hemoglobin genes set 2:', hb_genes2, '\n')

mito_genes=['MT-ND1','MT-ND2','MT-CO1','MT-CO2','MT-ATP8','MT-ATP6','MT-CO3','MT-ND3','MT-ND4L', 'MT-ND4', 'MT-ND5', 'MT-ND6', 'MT-CYB']
mito_genes = [x for x in mito_genes if x in adata.var_names]
print('Mitochondrial genes:', mito_genes, '\n')

ribo_genes = ['RPLP0','RPLP1','RPLP2','RPL3','RPL3L','RPL4','RPL5','RPL6','RPL7','RPL7A','RPL7L1','RPL8','RPL9','RPL10','RPL10A','RPL10L','RPL11','RPL12','RPL13','RPL13A','RPL14','RPL15','RPL17','RPL18','RPL18A','RPL19','RPL21','RPL22','RPL22L1','RPL23','RPL23A','RPL24','RPL26','RPL26L1','RPL27','RPL27A','RPL28','RPL29','RPL30','RPL31','RPL32','RPL34','RPL35','RPL35A','RPL36','RPL36A','RPL36AL','RPL37','RPL37A','RPL38','RPL39','RPL39L','UBA52','RPL41','RPSA','RPS2','RPS3','RPS3A','RPS4X','RPS4Y1','RPS4Y2','RPS5','RPS6','RPS7','RPS8','RPS9','RPS10','RPS11','RPS12','RPS13','RPS14','RPS15','RPS15A','RPS16','RPS17','RPS18','RPS19','RPS20','RPS21','RPS23','RPS24','RPS25','RPS26','RPS27','RPS27A','RPS27L','RPS28','RPS29','FAU']
ribo_genes = [x for x in ribo_genes if x in adata.var_names]
print('Ribo genes:', ribo_genes, '\n')

stress_genes=['FOS','FOSB','JUN','JUNB','JUND','ZFP36','SOCS3','CEBPD','CEBPB','ATF3','EGR1','HSPA1A','HSPA1B','HSP90AA1','HSP90AB1','HSPA8','HSPB1','HSPE1','IER3','IER2','BTG1','BTG2','DUSP1']
stress_genes = [x for x in stress_genes if x in adata.var_names]
print('Tissue dissociation stress associated genes:', stress_genes, '\n')

FoxProteins=['FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1']
chrY_genes=['SRY', 'RPS4Y1', 'ZFY', 'ZFY-AS1', 'LINC00278', 'PCDH11Y', 'TTTY2B', 'TBL1Y', 'PRKY', 'USP9Y', 'DDX3Y', 'UTY', 'TMSB4Y', 'NLGN4Y', 'TTTY14', 'KDM5D', 'TTTY10', 'EIF1AY', 'PRORY', 'LINC00266-4P']
MembraneProteins1=['AADAT','AAK1','ABCA12','ABCA7','ABCB1','ABCB11','ABCB4','ABCB6','ABCC1','ABCC3','ABCC4','ABCC5','ABCC6','ABCG2','ABHD3','ABI1','ABLIM3','AC004691.2','AC004922.1','AC005324.3','AC005726.1','AC008403.1','AC138696.1','ACIN1','ACKR3','ACLY','ACOT13','ACOT9','ACTN1','ACTR10','ACTR6','ADA','ADAM10','ADAM12','ADAM22','ADAM28','ADAM33','ADAMTS1','ADAMTS2','ADAMTSL4','ADAP1','ADCK5','ADD1','ADD2','ADGRB2','ADGRD1','ADGRG2','ADGRG7','ADGRL2','ADH1A','ADH1B','ADH1C','ADH7','ADK','ADORA1','ADRA1B','ADRM1','ADSS','AFAP1L2','AFDN','AFG1L','AGER','AGRN','AHNAK','AHNAK2','AIFM2','AKAP10','AKAP11','AKAP12','AKR1B10','AKR1B15','AL132780.3','AL139300.1','AL355315.1','AL358113.1','AL645922.1','ALDH3A1','ALK','ALOX15B','ALOXE3','ALPI','ALPP','ALPPL2','ALS2CR12','AMACR','AMDHD1','AMER1','AMER2','AMIGO2','AMOT','AMOTL1','AMOTL2','AMPH','ANK2','ANK3','ANKEF1','ANKFN1','ANKIB1','ANKLE2','ANKRA2','ANKRD13A','ANKRD35','ANKRD42','ANKRD45','ANKS1B','ANO1','ANO10','ANO2','ANO6','ANPEP','ANXA1','ANXA13','ANXA2','ANXA3','AP2M1','APBB1','APBB1IP','APC','APLP1','APMAP','APOC3','APOD','APOLD1','AQP1','AQP10','AQP3','AQP4','AQP5','ARAP1','ARF1','ARHGAP18','ARHGAP19','ARHGAP21','ARHGAP22','ARHGAP23','ARHGAP24','ARHGAP28','ARHGAP29','ARHGAP33','ARHGAP36','ARHGAP40','ARHGAP9','ARHGDIG','ARHGEF1','ARHGEF11','ARHGEF12','ARHGEF15','ARHGEF26','ARHGEF28','ARHGEF40','ARHGEF5','ARID2','ARID4A','ARID5B','ARL13A','ARL15','ARL4C','ARMC12','ARMCX5','ARPC1A','ARPC5','ARRB2','ARRDC4','ARVCF','ASAP1','ASB5','ASGR1','ASGR2','ASH2L','ASIC1','ASIC4','ASPM','ASPSCR1','ATG13','ATG3','ATG7','ATIC','ATOX1','ATP10D','ATP12A','ATP13A3','ATP1B3','ATP23','ATP2B1','ATP2B4','ATP2C2','ATP6V0A2','ATP6V1H','ATP8A2','ATP8B1','ATXN10','ATXN3','ATXN7L3','AVEN','AVPI1','AXIN2','AXL','B2M','B3GALT2','BACE1','BAG3','BAIAP2','BAIAP2L1','BANK1','BASP1','BBOF1','BCAR1','BCAR3','BCL2L12','BCORL1','BCR','BDKRB2','BEND6','BFSP1','BFSP2','BICD2','BIN2','BLNK','BLVRB','BMPR2','BMX','BOC','BOLL','BORCS5','BORCS8-MEF2B','BRK1','BRPF1','BRSK1','BSDC1','BST2','BTK','BTN2A1','BTRC','BUB1B-PAK6','BVES','BZW2','C10orf90','C11orf42','C11orf52','C12orf73','C12orf76','C14orf37','C14orf93','C16orf74','C17orf58','C17orf62','C18orf21','C19orf33','C19orf38','C19orf48','C19orf71','C1orf116','C1orf189','C1orf35','C1orf54','C1orf68','C1orf74','C1QBP','C2orf72','C3orf14','C3orf20','C3orf38','C4orf19','C4orf22','C7orf43','C8orf33','C8orf37','C9orf153','C9orf3','CA14','CA9','CABLES2','CACNA1E','CACNG4','CADM1','CALCRL','CALD1','CALML5','CALN1','CAMK2A','CAMK2B','CAMK2D','CAMK2G','CAP2','CAPS','CASKIN2','CASP4','CAST','CAV3','CAVIN1','CAVIN2','CAVIN3','CAVIN4','CBWD1','CBWD2','CBWD3','CBWD5','CBWD6','CC2D1A','CCBE1','CCDC107','CCDC112','CCDC115','CCDC124','CCDC136','CCDC144NL','CCDC15','CCDC154','CCDC170','CCDC24','CCDC28A','CCDC59','CCDC62','CCDC66','CCDC69','CCDC8','CCDC83','CCDC85C','CCDC93','CCDC97','CCNT2','CCNYL1','CCR3','CCR4','CCT3','CD109','CD163','CD164L2','CD200R1','CD226','CD248','CD274','CD2AP','CD300A','CD300LB','CD33','CD34','CD37','CD38','CD3E','CD3G','CD4','CD44','CD46','CD47','CD48','CD79B','CD81','CD84','CD86','CD8A','CD8B','CD9','CD93','CD99L2','CDAN1','CDC42EP1','CDC42EP3','CDC42EP4','CDC42SE1','CDCA4','CDH1','CDH13','CDH15','CDH17','CDH2','CDH26','CDH3','CDH4','CDH5','CDH6','CDH8','CDIPT','CDK16','CDK5','CDK5RAP2','CDK7','CDRT15','CDV3','CEACAM5','CELSR1','CENPJ','CEP162','CEP295','CEP55','CEP57L1','CEP89','CERCAM','CFAP157','CFAP206','CFAP58','CFAP65','CFB','CFL2','CFLAR','CGN','CGNL1','CHCHD7','CHIC2','CHMP2A','CHRM3','CHRNA1','CHRNA5','CHRNB4','CHRND','CIB1','CIB3','CIP2A','CKAP5','CLCA2','CLCA4','CLCN5','CLCN6','CLDN11','CLDN18','CLDN2','CLDN23','CLDN3','CLDN4','CLDN7','CLDN9','CLEC2A','CLEC4F','CLEC5A','CLECL1','CLIC2','CLIC5','CLTB','CLUL1','CMTM4','CMTM7','CMTR2','CMYA5','CNGA1','CNKSR2','CNNM1','CNNM4','CNP','CNR1','CNTD1','COBL','COBLL1','COG3','COL16A1','COL17A1','COL4A5','COL6A2','COLEC12','COLQ','COMMD4','COMTD1','COPRS','COQ7','CORIN','CORO1B','CORO2A','CORO2B','COX6B2','CPEB3','CPPED1','CPZ','CRB3','CREB5','CRIP1','CRIP2','CRK','CRLF3','CROCC','CRTC1','CRYAB','CRYBG3','CRYL1','CSDE1','CSF1','CSF1R','CSPG4','CST6','CTNNA1','CTNNB1','CTNNBIP1','CTNND1','CTNND2','CTSF','CTSV','CTTN','CTXN1','CUEDC1','CUTA','CX3CL1','CX3CR1','CXADR','CXCR2','CYB5D2','CYBRD1','CYFIP2','CYP20A1','CYP24A1','CYTH2','CYTH4','DAAM1','DAB2','DAG1','DAGLB','DAW1','DBN1','DCAF13','DCAF16','DCBLD2','DCHS1','DCHS2','DCP2','DCT','DCTN6','DCUN1D2','DDAH1','DDB2','DDIAS','DDIT4L','DDR1','DDR2','DDX58','DDX60L','DENND6B','DFFA','DGKG','DGKH','DHDDS','DHPS','DHRS7B','DIAPH1','DIAPH3','DIO2','DIS3L','DLG1','DLG2','DLG5','DLGAP3','DLL3','DMWD','DNAAF1','DNAAF4','DNAH17','DNAJA4','DNAJB13','DNAJB4','DNAJC18','DNAJC5','DNAJC6','DNAJC9','DOC2A','DOCK4','DOK3','DONSON','DPEP1','DPH1','DPPA5','DPYSL2','DQX1','DRD1','DRP2','DSC3','DSEL','DSG2','DSP','DTNA','DUS1L','DUSP15','DXO','DYSF','ECSCR','EDA2R','EDNRA','EDNRB','EEF1AKMT1','EEF2','EFCAB1','EFCAB11','EFCAB2','EFCAB7','EFHC2','EFNB1','EFR3A','EFR3B','EGFR','EHBP1','EHD1','EHD2','EHD3','EHD4','EIF2B2','EIF5','EIF5B','ELFN1','ELMO3','ELN','ELOC','EMP3','ENAH','ENC1','ENG','ENKD1','ENO1','ENO2','ENO3','ENOX1','ENOX2','EPB41','EPB41L1','EPB41L2','EPB41L3','EPB41L4A','EPB41L4B','EPB41L5','EPCAM','EPHA2','EPHA3','EPHB1','EPHB2','EPN1','EPOR','EPS8L2','ERAP1','ERBB2','ERBB3','ERBIN','ERC1','ERCC4','ERMN','ESCO2','ETNK1','ETS2','EVA1A','EVI2B','EXOC1','EXOC3L1','EXOC7','EZR','F11R','F2RL1','F2RL3','FAAP20','FABP2','FABP3','FABP5','FAIM','FAM126A','FAM126B','FAM129A','FAM129B','FAM13A','FAM163B','FAM171A1','FAM193A','FAM205A','FAM210B','FAM222A','FAM45A','FAM47E','FAM47E-STBD1','FAM72A','FAM72B','FAM72C','FAM72D','FANCG','FAS','FASN','FBLIM1','FBLN2','FBLN5','FBLN7','FBP2','FBRSL1','FBXL2','FBXO16','FBXO38','FBXO41','FBXW11','FCAR','FCGBP','FCGR1A','FCGR1B','FCGR2A','FCHSD2','FFAR4','FGD1','FGD5','FGFR2','FGR','FHIT','FHL1','FIBCD1','FIGN','FILIP1','FILIP1L','FKBP2','FLAD1','FLCN','FLNA','FLNB','FLNC','FLOT1','FLOT2','FLRT3','FLT1','FLT4','FLVCR1','FMN2','FMNL2','FMNL3','FNBP1','FOXA2','FOXN3','FP565260.3','FRMD3','FRMD7','FRMD8','FRS2','FSCN1','FTCD','FUT2','FXYD3','FYB1','FYN','FZD1','FZD2','FZD4','FZD6','GAB1','GAB2','GABRA1','GABRA2','GABRA3','GABRA5','GABRG1','GAD1','GADL1','GAGE1','GAGE10','GAGE12C','GAGE12D','GAGE12E','GAGE12F','GAGE12G','GAGE12H','GAGE12J','GAGE13','GAGE2A','GAGE2E','GALNT10','GALR1','GAP43','GAPDH','GAPT','GAPVD1','GAS2','GAS2L1','GAS7','GAS8','GBP4','GCA','GCC1','GCLM','GCSAML-AS1','GFI1B','GFOD1','GFRA3','GHR','GID8','GINM1','GIPC1','GJA1','GJB3','GJB4','GJB6','GJB7','GJC1','GJC2','GLO1','GLP2R','GLRX','GLUL','GMIP','GNAI2','GNAQ','GNAS','GNB1','GNB2','GNB3','GNB4','GNG2','GP9','GPAT3','GPATCH1','GPBP1','GPBP1L1','GPC1','GPC3','GPC4','GPI','GPR1','GPR142','GPR156','GPR157','GPR160','GPR173','GPR20','GPR27','GPR50','GPR63','GPRC5A','GPRIN1','GPRIN2','GPX8','GRAMD1A','GRAMD1B','GRAMD1C','GRAP2','GRB7','GRID2IP','GRIK5','GRK3','GRK5','GRPR','GSDMA','GTF2F1','GTSE1','GUCA1A','GYPA','GYPC','HAPLN3','HARBI1','HAS1','HAUS4','HAUS7','HCAR2','HCAR3','HCFC2','HCK','HCLS1','HCST','HDAC8','HEG1','HELZ','HENMT1','HERC2','HERPUD1','HFE','HIF1A','HIF3A','HINT1','HIP1R','HLA-A','HLA-DPB1','HMGCS1','HMOX1','HOMER3','HOXC5','HOXD4','HPS3','HRASLS5','HRH1','HS6ST3','HSH2D','HSPB1','HSPG2','HTR1B','HTR7','HTRA1','HUNK','HYDIN','HYI','HYLS1','ICAM1','ICAM2','ICOS','ICOSLG','IFFO2','IFIT5','IFNGR1','IFRD1','IFT122','IGFL2','IGSF11','IGSF23','IKZF5','IL10RA','IL12RB1','IL12RB2','IL13','IL13RA2','IL16','IL17RB','IL1R1','IL1RL1','IL21R','IL4R','IL6ST','IL7R','ILDR2','ILK','INPP5A','INSR','INTS12','IP6K2','IPCEF1','IQGAP1','IQGAP2','IREB2','IRF2','IRS1','ISLR','ITGA2B','ITGA3','ITGA4','ITGA7','ITGA8','ITGA9','ITGB1','ITGB2','ITGB3','ITGB4','ITGB5','ITGB6','ITGB7','ITGB8','ITK','ITM2A','ITPKA','ITSN1','ITSN2','IYD','JAG1','JAK2','JAK3','JAM2','JAML','JCAD','JMJD4','JPT2','JUP','KANK1','KANK3','KANSL2','KARS','KATNB1','KCNAB1','KCNB2','KCNH8','KCNJ15','KCNK13','KCNK17','KCNK6','KCNK7','KCNK9','KCNN2','KCNS3','KCNU1','KDF1','KDR','KEL','KIAA0556','KIAA0895','KIAA1324','KIAA1522','KIAA1958','KIF17','KIF19','KIF21A','KIF21B','KIF26A','KIF26B','KISS1R','KIT','KLC1','KLC2','KLF9','KLHDC7B','KLHL41','KLHL6','KLHL7','KLK7','KLRC1','KLRC2','KLRF1','KLRK1','KMT2D','KNDC1','KNSTRN','KNTC1','KPTN','KRT75','KRT77','KRTAP25-1','L1CAM','L3MBTL1','LAIR1','LAMA4','LAMC1','LAMP2','LAMTOR1','LAPTM4A','LASP1','LAT','LAT2','LAX1','LCP1','LCTL','LDB2','LDLRAD3','LENG1','LGALS3','LGALS4','LGALS8','LGR6','LHFPL2','LHX6','LIMA1','LINGO1','LIPJ','LMCD1','LMOD2','LMTK2','LMTK3','LNX1','LNX2','LONRF1','LONRF3','LOXHD1','LPAR1','LPAR3','LPAR4','LPIN1','LPIN2','LRCH2','LRP11','LRP12','LRRC15','LRRC26','LRRC34','LRRC45','LRRC69','LRRC7','LRRC9','LRRFIP2','LRRIQ3','LSMEM2','LSR','LTB4R2','LUZP2','LY6K','LYN','LYPD1','LYST','LZTS1','LZTS2','MACF1','MADD','MAGI1','MAGI3','MAMDC2','MAP1S','MAP2','MAP2K1','MAP2K4','MAP2K7','MAP3K12','MAP3K21','MAP3K6','MAP3K7','MAP4','MAP4K1','MAP4K5','MAP6','MAPK15','MAPK8IP1','MAPT','MARCH10','MARCH7','MARCKS','MARK2','MARVELD2','MAS1L','MAT2B','MBP','MC5R','MCAM','MCEMP1','MCF2L2','MCMBP','MCTS1','MDGA1','ME1','MEF2B','MEIOC','MEPCE','MERTK','MET','METAP1','METAP2','METTL15','METTL25','METTL9','MFAP3L','MFSD1','MFSD2A','MGAT4C','MIB1','MICAL2','MICAL3','MICALL1','MICB','MIEN1','MIGA2','MISP','MKLN1','MKRN3','MLF2','MLKL','MLST8','MMACHC','MMP10','MMP15','MMP24','MMP25','MOAP1','MPHOSPH8','MPHOSPH9','MPI','MPIG6B','MPL','MPP1','MPP5','MPP7','MPZL1','MPZL2','MR1','MRGPRF','MRNIP','MRPL42','MRPL44','MRPL46','MRPS18B','MS4A1','MS4A12','MS4A14','MS4A15','MSN','MSRA','MTCP1','MTFR1L','MTHFR','MTM1','MTPN','MTSS1','MUC1','MUC20','MVD','MYH11','MYH9','MYL4','MYLK','MYO10','MYO1B','MYO1C','MYO1E','MYO1G','MYO3A','MYO5B','MYO9A','MYOF','N4BP2','NAA35','NAB1','NAIF1','NAMPT','NAP1L2','NAPG','NARFL','NCAM1','NCAM2','NCK1','NCKAP1L','NCKAP5','NCKIPSD','NCOA1','NCS1','NDC1','NECTIN2','NECTIN3','NECTIN4','NEDD1','NEDD9','NEO1','NEU1','NEXN','NF2','NFE2L2','NFKBID','NGEF']
MembraneProteins2=['NGFR','NHS','NHSL2','NID2','NINJ2','NKTR','NKX2-8','NLGN1','NLGN3','NLK','NLRX1','NMB','NMBR','NME1','NME2','NMT1','NMT2','NMUR2','NOS1','NOTCH2','NPFFR2','NPL','NPNT','NPR1','NPTN','NPTXR','NR2C1','NRAS','NRCAM','NRP2','NRSN1','NRXN1','NSD1','NSFL1C','NT5E','NTNG1','NTNG2','NTSR1','NUDCD3','NUDT16L1','NUDT9','NUMB','NUP35','NUS1','OAS3','OBSCN','OCA2','OCLN','OCM','OCM2','ODC1','ODR4','OGT','OLFM4','OLIG2','OLR1','OPHN1','OPRK1','OR10P1','OR1N2','OR2C3','OR6B2','OR6B3','OR6C6','OR8S1','OSBPL10','OSBPL6','OSBPL7','OSCAR','OSCP1','OSGEP','OSR2','OTOP2','OTULIN','OXGR1','P3H3','PACSIN2','PACSIN3','PAFAH1B2','PAG1','PAIP1','PAK1','PAK4','PAK6','PALLD','PALM','PALM2','PALM2-AKAP2','PALM3','PALMD','PANX1','PANX2','PAQR8','PARD3','PARD3B','PARD6A','PARD6G','PARP6','PATJ','PAWR','PCBD2','PCDH1','PCDH17','PCDH7','PCDHA1','PCDHA11','PCDHA12','PCDHA4','PCDHA5','PCDHB1','PCDHB11','PCDHB12','PCDHB16','PCDHB4','PCDHB5','PCDHGA1','PCDHGA10','PCDHGA11','PCDHGA12','PCDHGA2','PCDHGA3','PCDHGA4','PCDHGA5','PCDHGA6','PCDHGA7','PCDHGA8','PCDHGA9','PCDHGB1','PCDHGB2','PCDHGB3','PCDHGB4','PCDHGB5','PCDHGB6','PCDHGB7','PCDHGC3','PCDHGC4','PCDHGC5','PCLO','PCMTD1','PDAP1','PDCD7','PDE3A','PDE4A','PDE4D','PDE9A','PDGFC','PDGFRA','PDHX','PDLIM1','PDLIM5','PDPN','PDZD2','PDZD8','PEAR1','PEBP1','PFDN1','PGAP3','PGLYRP2','PGLYRP4','PGRMC2','PHACTR1','PHACTR4','PHF20L1','PHGDH','PHLDB2','PHLDB3','PHOSPHO1','PHPT1','PI3','PI4K2A','PI4KA','PIEZO2','PIGB','PIGN','PIGW','PIGX','PIK3AP1','PIK3CG','PIK3R6','PIP4K2A','PIP4K2C','PIP5K1A','PIRT','PITPNM3','PKD2L2','PKN1','PKN2','PKP1','PKP2','PKP3','PKP4','PLAA','PLAUR','PLCB1','PLCB2','PLCD3','PLCD4','PLEKHA4','PLEKHA6','PLEKHA7','PLEKHG3','PLEKHG4','PLEKHG6','PLEKHH2','PLEKHO2','PLIN4','PLPP1','PLPPR5','PLS3','PLSCR1','PLVAP','PLXNA3','PLXNA4','PNCK','PNISR','PODXL','POLE','POU2F3','PPEF1','PPFIBP1','PPIP5K1','PPL','PPM1A','PPM1F','PPP1CA','PPP1CB','PPP1R12B','PPP1R16A','PPP1R16B','PPP1R18','PPP1R27','PPP1R9A','PPP2R1B','PPP3R1','PPP3R2','PPP4C','PPP6R3','PPTC7','PRAME','PRC1','PRDX6','PRIMA1','PRKCA','PRKCE','PRKCH','PRKCZ','PRKD1','PROM2','PROSER2','PRR15','PRR32','PRR7','PRRC2A','PRRG1','PRRT2','PRRT3','PRRT4','PRTG','PRX','PSCA','PSD','PSD2','PSEN1','PSMC5','PSMC6','PSMD9','PSTPIP2','PTAFR','PTCHD4','PTK2B','PTK6','PTPN12','PTPN21','PTPN22','PTPN4','PTPRB','PTPRG','PTPRK','PTPRM','PTPRR','PTPRS','PTX3','PURG','PVALB','PVR','PXDC1','PXK','PYGL','PYGO2','PYM1','RAB13','RAB23','RAB25','RAB26','RAB27B','RAB3B','RAB3GAP2','RAB43','RAB8A','RABEPK','RAD51C','RAD52','RAI2','RALA','RALGAPA2','RAPGEF2','RAPH1','RAPSN','RASA4','RASA4B','RASAL1','RASAL2','RASAL3','RASD1','RASD2','RASGRP1','RASGRP2','RASGRP4','RASSF2','RASSF3','RASSF4','RBM46','RC3H1','RCCD1','RCSD1','RD3L','RDX','RECK','RELL1','REPS1','RET','RFLNB','RFPL1','RFPL2','RFPL3','RGP1','RGS10','RGS14','RGS19','RGS3','RGS7BP','RGS9BP','RHCE','RHD','RHEX','RHOA','RHOBTB2','RHOJ','RHOXF2','RHOXF2B','RIC8A','RIC8B','RIF1','RILPL1','RIMBP3','RIMBP3B','RIMBP3C','RIMS2','RIOK2','RIOK3','RIPK1','RND1','RNF114','RNF145','RNF146','RNF149','RNF165','RNF166','RNF39','ROBO1','ROBO3','ROBO4','ROM1','RP2','RPE','RPEL1','RPGRIP1L','RPIA','RPL36A','RPL36A-HNRNPH2','RPL36AL','RPSA','RPUSD1','RRAD','RRP12','RRP7A','RSBN1L','RSC1A1','RSPH3','RTL3','RTL4','RTN4R','RTN4RL1','RYR2','S100A13','S100A14','S100A16','S100A2','S100A3','S100A4','S100A6','S100A9','SAMD4A','SAMHD1','SAMSN1','SAPCD1','SASH3','SC5D','SCAMP1','SCAMP4','SCEL','SCHIP1','SCIN','SCN1A','SCN5A','SCN8A','SCN9A','SCNN1D','SCNN1G','SCRIB','SCUBE3','SDC2','SDC4','SDE2','SDHB','SEC16B','SEMA3C','SEMA3D','SEMA4B','SEMA4F','SEMA6C','SEMA6D','SEPT2','SEPT3','SERINC2','SERINC5','SERPINB3','SFSWAP','SGCE','SGK494','SH3BGR','SH3D21','SHANK1','SHANK3','SHARPIN','SHISA6','SHKBP1','SHROOM2','SHTN1','SIGLEC10','SIGLEC14','SIGLEC6','SIGLEC7','SIN3B','SIPA1','SIPA1L1','SIPA1L3','SKAP1','SLAMF6','SLC10A5','SLC12A2','SLC13A3','SLC13A5','SLC14A1','SLC16A1','SLC16A10','SLC16A2','SLC16A3','SLC16A7','SLC16A9','SLC17A5','SLC19A1','SLC19A3','SLC1A5','SLC1A6','SLC22A16','SLC23A3','SLC24A4','SLC26A9','SLC27A1','SLC28A1','SLC2A1','SLC2A11','SLC2A12','SLC2A14','SLC2A2','SLC2A3','SLC30A1','SLC31A2','SLC34A1','SLC34A2','SLC35F5','SLC35G2','SLC36A4','SLC38A5','SLC38A6','SLC38A7','SLC39A10','SLC39A14','SLC39A2','SLC39A6','SLC3A2','SLC40A1','SLC41A3','SLC43A1','SLC43A3','SLC44A2','SLC45A4','SLC46A1','SLC46A3','SLC4A1AP','SLC4A2','SLC4A7','SLC4A8','SLC5A12','SLC5A3','SLC5A6','SLC5A7','SLC6A6','SLC7A14','SLC7A2','SLC7A5','SLC7A6','SLC8A1','SLC9A1','SLC9A2','SLC9A3','SLC9A3R1','SLC9A3R2','SLC9A4','SLC9A5','SLC9B2','SLC9C2','SLCO1A2','SLCO1B3','SLCO4A1','SLCO4C1','SLCO5A1','SLIT1','SLIT3','SLITRK6','SLK','SLX4','SMAGP','SMAP1','SMOC1','SMPD2','SMPX','SNAP23','SNAP25','SNRK','SNTB2','SNTG2','SNX15','SNX32','SNX9','SOCS3','SOCS5','SOD1','SORBS1','SORCS1','SORT1','SOX11','SPAG16','SPANXN4','SPART','SPATA12','SPATC1L','SPERT','SPG11','SPHK1','SPINK13','SPINT1','SPINT2','SPN','SPPL2B','SPPL3','SPRED3','SPRYD3','SPTBN2','SRC','SRCIN1','SRD5A3','SRPK1','SRPRB','SRRM3','SSFA2','SSH1','SSH3','SSTR1','SSTR5','SSX2IP','ST8SIA5','STAB1','STAC','STAC2','STAMBP','STAMBPL1','STARD4','STAT2','STEAP4','STIL','STIP1','STK10','STK16','STK17A','STK17B','STOM','STOML2','STOML3','STX18','STX4','STXBP4','STYK1','SUCLG1','SUCLG2','SUGT1','SUMF2','SUPT6H','SUSD2','SVIL','SWAP70','SYCP2','SYK','SYMPK','SYNC','SYNGR3','SYNPO2L','SYPL2','SYT4','SYTL2','SYVN1','TACC2','TACR2','TACSTD2','TASP1','TAZ','TBC1D10A','TBC1D2','TBC1D24','TBC1D30','TBC1D9B','TBXA2R','TCAF2','TCAF2C','TCHP','TCIM','TDG','TDO2','TDP1','TEC','TEDC2','TEK','TES','TEX14','TEX9','TGFBR1','TGFBR2','TGM1','TGM2','THBS1','THBS2','THEGL','THY1','TIAM1','TICAM1','TIGD7','TIMD4','TJP1','TJP2','TJP3','TLN1','TLR4','TM2D2','TM4SF20','TM4SF4','TM4SF5','TMC5','TMED8','TMEM101','TMEM107','TMEM108','TMEM121B','TMEM139','TMEM141','TMEM164','TMEM176B','TMEM183A','TMEM186','TMEM2','TMEM218','TMEM222','TMEM229A','TMEM245','TMEM251','TMEM254','TMEM258','TMEM266','TMEM268','TMEM50A','TMEM63B','TMPRSS15','TMPRSS2','TMTC2','TNFRSF10D','TNFRSF11A','TNFRSF12A','TNFRSF21','TNFRSF25','TNFRSF8','TNFRSF9','TNFSF13B','TNK1','TNKS1BP1','TNNI2','TOGARAM2','TOLLIP','TOM1','TP53BP2','TPD52L1','TPRN','TRAF7','TRAIP','TRAPPC13','TRAT1','TRDN','TREML1','TRIB1','TRIM16','TRIM16L','TRIM38','TRIM4','TRIM44','TRIM54','TRIM71','TRIML2','TRIP6','TRMT1','TRMT10B','TRMT13','TRMT6','TRO','TRPM1','TRPM4','TRPM7','TRPV2','TRPV3','TSG101','TSGA13','TSHZ3','TSPAN15','TSPAN6','TSPO2','TSR2','TSSK4','TTC17','TTC23','TTC7B','TTLL12','TTLL5','TULP3','TWF2','TXNDC11','TXNIP','TYROBP','TYW3','UAP1','UBA3','UBA52','UBAC1','UBALD1','UBAP1','UBASH3B','UBE2A','UBE2B','UBE2C','UBE2D1','UBE2D2','UBE2D3','UBE2D4','UBE2K','UBE2QL1','UBE2S','UBE3C','UBQLN2','UBR2','UBTD1','UCP3','ULBP1','ULBP2','UNC5A','UNC79','UQCC1','USP14','USP15','USP2','USP20','USP21','USP4','USP53','UTP20','UTRN','VAMP5','VAMP8','VANGL1','VASP','VCAM1','VCPIP1','VIL1','VPS13B','VPS36','VPS50','VPS8','VSIG4','VTCN1','VWA5B1','VWA7','WAS','WASF2','WBP4','WDFY3','WDPCP','WDR1','WDR12','WDR13','WDR53','WDR6','WDR66','WIPF1','WWP1','XIAP','XIRP1','XIRP2','XPC','XPNPEP2','XPO6','XRN1','XYLT1','YES1','YIPF1','YIPF4','YOD1','YTHDC1','ZBED3','ZBTB33','ZBTB8OS','ZDHHC2','ZDHHC20','ZDHHC22','ZDHHC5','ZFP41','ZFP90','ZFYVE28','ZGPAT','ZMYND10','ZNF10','ZNF18','ZNF185','ZNF2','ZNF20','ZNF227','ZNF229','ZNF236','ZNF276','ZNF277','ZNF337','ZNF397','ZNF407','ZNF497','ZNF503','ZNF510','ZNF536','ZNF571','ZNF598','ZNF599','ZNF646','ZNF655','ZNF688','ZNF706','ZNF717','ZNF783','ZNF79','ZNF839','ZSCAN22','ZSCAN29','ZSWIM1','ZSWIM8','ZYX']
MembraneProteins=MembraneProteins1+MembraneProteins2

TFs1=['AC008770.3','AC023509.3','AC092835.1','AC138696.1','ADNP','ADNP2','AEBP1','AEBP2','AHCTF1','AHDC1','AHR','AHRR','AIRE','AKAP8','AKAP8L','AKNA','ALX1','ALX3','ALX4','ANHX','ANKZF1','AR','ARGFX','ARHGAP35','ARID2','ARID3A','ARID3B','ARID3C','ARID5A','ARID5B','ARNT','ARNT2','ARNTL','ARNTL2','ARX','ASCL1','ASCL2','ASCL3','ASCL4','ASCL5','ASH1L','ATF1','ATF2','ATF3','ATF4','ATF5','ATF6','ATF6B','ATF7','ATMIN','ATOH1','ATOH7','ATOH8','BACH1','BACH2','BARHL1','BARHL2','BARX1','BARX2','BATF','BATF2','BATF3','BAZ2A','BAZ2B','BBX','BCL11A','BCL11B','BCL6','BCL6B','BHLHA15','BHLHA9','BHLHE22','BHLHE23','BHLHE40','BHLHE41','BNC1','BNC2','BORCS8-MEF2B','BPTF','BRF2','BSX','C11orf95','CAMTA1','CAMTA2','CARF','CASZ1','CBX2','CC2D1A','CCDC169-SOHLH2','CCDC17','CDC5L','CDX1','CDX2','CDX4','CEBPA','CEBPB','CEBPD','CEBPE','CEBPG','CEBPZ','CENPA','CENPB','CENPBD1','CENPS','CENPT','CENPX','CGGBP1','CHAMP1','CHCHD3','CIC','CLOCK','CPEB1','CPXCR1','CREB1','CREB3','CREB3L1','CREB3L2','CREB3L3','CREB3L4','CREB5','CREBL2','CREBZF','CREM','CRX','CSRNP1','CSRNP2','CSRNP3','CTCF','CTCFL','CUX1','CUX2','CXXC1','CXXC4','CXXC5','DACH1','DACH2','DBP','DBX1','DBX2','DDIT3','DEAF1','DLX1','DLX2','DLX3','DLX4','DLX5','DLX6','DMBX1','DMRT1','DMRT2','DMRT3','DMRTA1','DMRTA2','DMRTB1','DMRTC2','DMTF1','DNMT1','DNTTIP1','DOT1L','DPF1','DPF3','DPRX','DR1','DRAP1','DRGX','DUX1','DUX3','DUX4','DUXA','DZIP1','E2F1','E2F2','E2F3','E2F4','E2F5','E2F6','E2F7','E2F8','E4F1','EBF1','EBF2','EBF3','EBF4','EEA1','EGR1','EGR2','EGR3','EGR4','EHF','ELF1','ELF2','ELF3','ELF4','ELF5','ELK1','ELK3','ELK4','EMX1','EMX2','EN1','EN2','EOMES','EPAS1','ERF','ERG','ESR1','ESR2','ESRRA','ESRRB','ESRRG','ESX1','ETS1','ETS2','ETV1','ETV2','ETV3','ETV3L','ETV4','ETV5','ETV6','ETV7','EVX1','EVX2','FAM170A','FAM200B','FBXL19','FERD3L','FEV','FEZF1','FEZF2','FIGLA','FIZ1','FLI1','FLYWCH1','FOS','FOSB','FOSL1','FOSL2','FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXN4','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1','GABPA','GATA1','GATA2','GATA3','GATA4','GATA5','GATA6','GATAD2A','GATAD2B','GBX1','GBX2','GCM1','GCM2','GFI1','GFI1B','GLI1','GLI2','GLI3','GLI4','GLIS1','GLIS2','GLIS3','GLMP','GLYR1','GMEB1','GMEB2','GPBP1','GPBP1L1','GRHL1','GRHL2','GRHL3','GSC','GSC2','GSX1','GSX2','GTF2B','GTF2I','GTF2IRD1','GTF2IRD2','GTF2IRD2B','GTF3A','GZF1','HAND1','HAND2','HBP1','HDX','HELT','HES1','HES2','HES3','HES4','HES5','HES6','HES7','HESX1','HEY1','HEY2','HEYL','HHEX','HIC1','HIC2','HIF1A','HIF3A','HINFP','HIVEP1','HIVEP2','HIVEP3','HKR1','HLF','HLX','HMBOX1','HMG20A','HMG20B','HMGA1','HMGA2','HMGN3','HMX1','HMX2','HMX3','HNF1A','HNF1B','HNF4A','HNF4G','HOMEZ','HOXA1','HOXA10','HOXA11','HOXA13','HOXA2','HOXA3','HOXA4','HOXA5','HOXA6','HOXA7','HOXA9','HOXB1','HOXB13','HOXB2','HOXB3','HOXB4','HOXB5','HOXB6','HOXB7','HOXB8','HOXB9','HOXC10','HOXC11','HOXC12','HOXC13','HOXC4','HOXC5','HOXC6','HOXC8','HOXC9','HOXD1','HOXD10','HOXD11','HOXD12','HOXD13','HOXD3','HOXD4','HOXD8','HOXD9','HSF1','HSF2','HSF4','HSF5','HSFX1','HSFX2','HSFY1','HSFY2','IKZF1','IKZF2','IKZF3','IKZF4','IKZF5','INSM1','INSM2','IRF1','IRF2','IRF3','IRF4','IRF5','IRF6','IRF7','IRF8','IRF9','IRX1','IRX2','IRX3','IRX4','IRX5','IRX6','ISL1','ISL2','ISX','JAZF1','JDP2','JRK','JRKL','JUN','JUNB','JUND','KAT7','KCMF1','KCNIP3','KDM2A','KDM2B','KDM5B','KIN','KLF1','KLF10','KLF11','KLF12','KLF13','KLF14','KLF15','KLF16','KLF17','KLF2','KLF3','KLF4','KLF5','KLF6','KLF7','KLF8','KLF9','KMT2A','KMT2B','L3MBTL1','L3MBTL3','L3MBTL4','LBX1','LBX2','LCOR','LCORL','LEF1','LEUTX','LHX1','LHX2','LHX3','LHX4','LHX5','LHX6','LHX8','LHX9','LIN28A','LIN28B','LIN54','LMX1A','LMX1B','LTF','LYL1','MAF','MAFA','MAFB','MAFF','MAFG','MAFK','MAX','MAZ','MBD1','MBD2','MBD3','MBD4','MBD6','MBNL2','MECOM','MECP2','MEF2A','MEF2B','MEF2C','MEF2D','MEIS1','MEIS2','MEIS3','MEOX1','MEOX2','MESP1','MESP2','MGA','MITF','MIXL1','MKX','MLX','MLXIP','MLXIPL','MNT','MNX1','MSANTD1','MSANTD3','MSANTD4','MSC','MSGN1','MSX1','MSX2','MTERF1','MTERF2','MTERF3','MTERF4','MTF1','MTF2','MXD1','MXD3','MXD4','MXI1','MYB','MYBL1','MYBL2','MYC','MYCL','MYCN','MYF5','MYF6','MYNN','MYOD1','MYOG','MYPOP','MYRF','MYRFL','MYSM1','MYT1','MYT1L','MZF1','NACC2','NAIF1','NANOG','NANOGNB','NANOGP8','NCOA1','NCOA2','NCOA3','NEUROD1','NEUROD2','NEUROD4','NEUROD6','NEUROG1','NEUROG2','NEUROG3','NFAT5','NFATC1','NFATC2','NFATC3','NFATC4','NFE2','NFE2L1','NFE2L2','NFE2L3','NFE4','NFIA','NFIB','NFIC','NFIL3','NFIX','NFKB1','NFKB2','NFX1','NFXL1','NFYA','NFYB','NFYC','NHLH1','NHLH2','NKRF','NKX1-1','NKX1-2','NKX2-1','NKX2-2','NKX2-3','NKX2-4','NKX2-5','NKX2-6','NKX2-8','NKX3-1','NKX3-2','NKX6-1','NKX6-2','NKX6-3','NME2','NOBOX','NOTO','NPAS1','NPAS2','NPAS3','NPAS4','NR0B1','NR1D1','NR1D2','NR1H2','NR1H3','NR1H4','NR1I2','NR1I3','NR2C1','NR2C2','NR2E1','NR2E3','NR2F1','NR2F2','NR2F6','NR3C1','NR3C2','NR4A1','NR4A2','NR4A3','NR5A1','NR5A2','NR6A1','NRF1','NRL','OLIG1','OLIG2','OLIG3','ONECUT1','ONECUT2','ONECUT3','OSR1','OSR2','OTP','OTX1','OTX2','OVOL1','OVOL2','OVOL3','PA2G4','PATZ1','PAX1','PAX2','PAX3','PAX4','PAX5','PAX6','PAX7','PAX8','PAX9','PBX1','PBX2','PBX3','PBX4','PCGF2','PCGF6','PDX1','PEG3','PGR','PHF1','PHF19 ','PHF20','PHF21A','PHOX2A','PHOX2B','PIN1','PITX1','PITX2','PITX3','PKNOX1','PKNOX2','PLAG1','PLAGL1','PLAGL2','PLSCR1','POGK','POU1F1','POU2AF1','POU2F1','POU2F2','POU2F3','POU3F1','POU3F2','POU3F3','POU3F4','POU4F1','POU4F2','POU4F3','POU5F1','POU5F1B','POU5F2','POU6F1','POU6F2','PPARA','PPARD','PPARG','PRDM1','PRDM10','PRDM12','PRDM13','PRDM14','PRDM15','PRDM16','PRDM2','PRDM4','PRDM5','PRDM6','PRDM8','PRDM9','PREB','PRMT3','PROP1','PROX1','PROX2','PRR12','PRRX1','PRRX2','PTF1A','PURA','PURB','PURG','RAG1','RARA','RARB','RARG','RAX','RAX2','RBAK','RBCK1','RBPJ','RBPJL','RBSN','REL','RELA','RELB','REPIN1','REST','REXO4','RFX1','RFX2','RFX3','RFX4','RFX5','RFX6','RFX7','RFX8','RHOXF1','RHOXF2','RHOXF2B','RLF','RORA','RORB','RORC','RREB1','RUNX1','RUNX2','RUNX3','RXRA','RXRB','RXRG','SAFB','SAFB2','SALL1','SALL2','SALL3','SALL4','SATB1','SATB2','SCMH1','SCML4','SCRT1','SCRT2','SCX','SEBOX','SETBP1','SETDB1','SETDB2','SGSM2','SHOX','SHOX2','SIM1','SIM2','SIX1','SIX2','SIX3','SIX4','SIX5','SIX6','SKI','SKIL','SKOR1','SKOR2','SLC2A4RG','SMAD1','SMAD3','SMAD4','SMAD5','SMAD9','SMYD3','SNAI1','SNAI2','SNAI3','SNAPC2','SNAPC4','SNAPC5','SOHLH1','SOHLH2','SON','SOX1','SOX10','SOX11']
TFs2=['SOX12','SOX13','SOX14','SOX15','SOX17','SOX18','SOX2','SOX21','SOX3','SOX30','SOX4','SOX5','SOX6','SOX7','SOX8','SOX9','SP1','SP100','SP110','SP140','SP140L','SP2','SP3','SP4','SP5','SP6','SP7','SP8','SP9','SPDEF','SPEN','SPI1','SPIB','SPIC','SPZ1','SRCAP','SREBF1','SREBF2','SRF','SRY','ST18','STAT1','STAT2','STAT3','STAT4','STAT5A','STAT5B','STAT6','T','TAL1','TAL2','TBP','TBPL1','TBPL2','TBR1','TBX1','TBX10','TBX15','TBX18','TBX19','TBX2','TBX20','TBX21','TBX22','TBX3','TBX4','TBX5','TBX6','TCF12','TCF15','TCF20','TCF21','TCF23','TCF24','TCF3','TCF4','TCF7','TCF7L1','TCF7L2','TCFL5','TEAD1','TEAD2','TEAD3','TEAD4','TEF','TERB1','TERF1','TERF2','TET1','TET2','TET3','TFAP2A','TFAP2B','TFAP2C','TFAP2D','TFAP2E','TFAP4','TFCP2','TFCP2L1','TFDP1','TFDP2','TFDP3','TFE3','TFEB','TFEC','TGIF1','TGIF2','TGIF2LX','TGIF2LY','THAP1','THAP10','THAP11','THAP12','THAP2','THAP3','THAP4','THAP5','THAP6','THAP7','THAP8','THAP9','THRA','THRB','THYN1','TIGD1','TIGD2','TIGD3','TIGD4','TIGD5','TIGD6','TIGD7','TLX1','TLX2','TLX3','TMF1','TOPORS','TP53','TP63','TP73','TPRX1','TRAFD1','TRERF1','TRPS1','TSC22D1','TSHZ1','TSHZ2','TSHZ3','TTF1','TWIST1','TWIST2','UBP1','UNCX','USF1','USF2','USF3','VAX1','VAX2','VDR','VENTX','VEZF1','VSX1','VSX2','WIZ','WT1','XBP1','XPA','YBX1','YBX2','YBX3','YY1','YY2','ZBED1','ZBED2','ZBED3','ZBED4','ZBED5','ZBED6','ZBED9','ZBTB1','ZBTB10','ZBTB11','ZBTB12','ZBTB14','ZBTB16','ZBTB17','ZBTB18','ZBTB2','ZBTB20','ZBTB21','ZBTB22','ZBTB24','ZBTB25','ZBTB26','ZBTB3','ZBTB32','ZBTB33','ZBTB34','ZBTB37','ZBTB38','ZBTB39','ZBTB4','ZBTB40','ZBTB41','ZBTB42','ZBTB43','ZBTB44','ZBTB45','ZBTB46','ZBTB47','ZBTB48','ZBTB49','ZBTB5','ZBTB6','ZBTB7A','ZBTB7B','ZBTB7C','ZBTB8A','ZBTB8B','ZBTB9','ZC3H8','ZEB1','ZEB2','ZFAT','ZFHX2','ZFHX3','ZFHX4','ZFP1','ZFP14','ZFP2','ZFP28','ZFP3','ZFP30','ZFP37','ZFP41','ZFP42','ZFP57','ZFP62','ZFP64','ZFP69','ZFP69B','ZFP82','ZFP90','ZFP91','ZFP92','ZFPM1','ZFPM2','ZFX','ZFY','ZGLP1','ZGPAT','ZHX1','ZHX2','ZHX3','ZIC1','ZIC2','ZIC3','ZIC4','ZIC5','ZIK1','ZIM2','ZIM3','ZKSCAN1','ZKSCAN2','ZKSCAN3','ZKSCAN4','ZKSCAN5','ZKSCAN7','ZKSCAN8','ZMAT1','ZMAT4','ZNF10','ZNF100','ZNF101','ZNF107','ZNF112','ZNF114','ZNF117','ZNF12','ZNF121','ZNF124','ZNF131','ZNF132','ZNF133','ZNF134','ZNF135','ZNF136','ZNF138','ZNF14','ZNF140','ZNF141','ZNF142','ZNF143','ZNF146','ZNF148','ZNF154','ZNF155','ZNF157','ZNF16','ZNF160','ZNF165','ZNF169','ZNF17','ZNF174','ZNF175','ZNF177','ZNF18','ZNF180','ZNF181','ZNF182','ZNF184','ZNF189','ZNF19','ZNF195','ZNF197','ZNF2','ZNF20','ZNF200','ZNF202','ZNF205','ZNF207','ZNF208','ZNF211','ZNF212','ZNF213','ZNF214','ZNF215','ZNF217','ZNF219','ZNF22','ZNF221','ZNF222','ZNF223','ZNF224','ZNF225','ZNF226','ZNF227','ZNF229','ZNF23','ZNF230','ZNF232','ZNF233','ZNF234','ZNF235','ZNF236','ZNF239','ZNF24','ZNF248','ZNF25','ZNF250','ZNF251','ZNF253','ZNF254','ZNF256','ZNF257','ZNF26','ZNF260','ZNF263','ZNF264','ZNF266','ZNF267','ZNF268','ZNF273','ZNF274','ZNF275','ZNF276','ZNF277','ZNF28','ZNF280A','ZNF280B','ZNF280C','ZNF280D','ZNF281','ZNF282','ZNF283','ZNF284','ZNF285','ZNF286A','ZNF286B','ZNF287','ZNF292','ZNF296','ZNF3','ZNF30','ZNF300','ZNF302','ZNF304','ZNF311','ZNF316','ZNF317','ZNF318','ZNF319','ZNF32','ZNF320','ZNF322','ZNF324','ZNF324B','ZNF326','ZNF329','ZNF331','ZNF333','ZNF334','ZNF335','ZNF337','ZNF33A','ZNF33B','ZNF34','ZNF341','ZNF343','ZNF345','ZNF346','ZNF347','ZNF35','ZNF350','ZNF354A','ZNF354B','ZNF354C','ZNF358','ZNF362','ZNF365','ZNF366','ZNF367','ZNF37A','ZNF382','ZNF383','ZNF384','ZNF385A','ZNF385B','ZNF385C','ZNF385D','ZNF391','ZNF394','ZNF395','ZNF396','ZNF397','ZNF398','ZNF404','ZNF407','ZNF408','ZNF41','ZNF410','ZNF414','ZNF415','ZNF416','ZNF417','ZNF418','ZNF419','ZNF420','ZNF423','ZNF425','ZNF426','ZNF428','ZNF429','ZNF43','ZNF430','ZNF431','ZNF432','ZNF433','ZNF436','ZNF438','ZNF439','ZNF44','ZNF440','ZNF441','ZNF442','ZNF443','ZNF444','ZNF445','ZNF446','ZNF449','ZNF45','ZNF451','ZNF454','ZNF460','ZNF461','ZNF462','ZNF467','ZNF468','ZNF469','ZNF470','ZNF471','ZNF473','ZNF474','ZNF479','ZNF48','ZNF480','ZNF483','ZNF484','ZNF485','ZNF486','ZNF487','ZNF488','ZNF490','ZNF491','ZNF492','ZNF493','ZNF496','ZNF497','ZNF500','ZNF501','ZNF502','ZNF503','ZNF506','ZNF507','ZNF510','ZNF511','ZNF512','ZNF512B','ZNF513','ZNF514','ZNF516','ZNF517','ZNF518A','ZNF518B','ZNF519','ZNF521','ZNF524','ZNF525','ZNF526','ZNF527','ZNF528','ZNF529','ZNF530','ZNF532','ZNF534','ZNF536','ZNF540','ZNF541','ZNF543','ZNF544','ZNF546','ZNF547','ZNF548','ZNF549','ZNF550','ZNF551','ZNF552','ZNF554','ZNF555','ZNF556','ZNF557','ZNF558','ZNF559','ZNF560','ZNF561','ZNF562','ZNF563','ZNF564','ZNF565','ZNF566','ZNF567','ZNF568','ZNF569','ZNF57','ZNF570','ZNF571','ZNF572','ZNF573','ZNF574','ZNF575','ZNF576','ZNF577','ZNF578','ZNF579','ZNF580','ZNF581','ZNF582','ZNF583','ZNF584','ZNF585A','ZNF585B','ZNF586','ZNF587','ZNF587B','ZNF589','ZNF592','ZNF594','ZNF595','ZNF596','ZNF597','ZNF598','ZNF599','ZNF600','ZNF605','ZNF606','ZNF607','ZNF608','ZNF609','ZNF610','ZNF611','ZNF613','ZNF614','ZNF615','ZNF616','ZNF618','ZNF619','ZNF620','ZNF621','ZNF623','ZNF624','ZNF625','ZNF626','ZNF627','ZNF628','ZNF629','ZNF630','ZNF639','ZNF641','ZNF644','ZNF645','ZNF646','ZNF648','ZNF649','ZNF652','ZNF653','ZNF654','ZNF655','ZNF658','ZNF66','ZNF660','ZNF662','ZNF664','ZNF665','ZNF667','ZNF668','ZNF669','ZNF670','ZNF671','ZNF672','ZNF674','ZNF675','ZNF676','ZNF677','ZNF678','ZNF679','ZNF680','ZNF681','ZNF682','ZNF683','ZNF684','ZNF687','ZNF688','ZNF689','ZNF69','ZNF691','ZNF692','ZNF695','ZNF696','ZNF697','ZNF699','ZNF7','ZNF70','ZNF700','ZNF701','ZNF703','ZNF704','ZNF705A','ZNF705B','ZNF705D','ZNF705E','ZNF705G','ZNF706','ZNF707','ZNF708','ZNF709','ZNF71','ZNF710','ZNF711','ZNF713','ZNF714','ZNF716','ZNF717','ZNF718','ZNF721','ZNF724','ZNF726','ZNF727','ZNF728','ZNF729','ZNF730','ZNF732','ZNF735','ZNF736','ZNF737','ZNF74','ZNF740','ZNF746','ZNF747','ZNF749','ZNF750','ZNF75A','ZNF75D','ZNF76','ZNF761','ZNF763','ZNF764','ZNF765','ZNF766','ZNF768','ZNF77','ZNF770','ZNF771','ZNF772','ZNF773','ZNF774','ZNF775','ZNF776','ZNF777','ZNF778','ZNF780A','ZNF780B','ZNF781','ZNF782','ZNF783','ZNF784','ZNF785','ZNF786','ZNF787','ZNF788','ZNF789','ZNF79','ZNF790','ZNF791','ZNF792','ZNF793','ZNF799','ZNF8','ZNF80','ZNF800','ZNF804A','ZNF804B','ZNF805','ZNF808','ZNF81','ZNF813','ZNF814','ZNF816','ZNF821','ZNF823','ZNF827','ZNF829','ZNF83','ZNF830','ZNF831','ZNF835','ZNF836','ZNF837','ZNF84','ZNF841','ZNF843','ZNF844','ZNF845','ZNF846','ZNF85','ZNF850','ZNF852','ZNF853','ZNF860','ZNF865','ZNF878','ZNF879','ZNF880','ZNF883','ZNF888','ZNF891','ZNF90','ZNF91','ZNF92','ZNF93','ZNF98','ZNF99','ZSCAN1','ZSCAN10','ZSCAN12','ZSCAN16','ZSCAN18','ZSCAN2','ZSCAN20','ZSCAN21','ZSCAN22','ZSCAN23','ZSCAN25','ZSCAN26','ZSCAN29','ZSCAN30','ZSCAN31','ZSCAN32','ZSCAN4','ZSCAN5A','ZSCAN5B','ZSCAN5C','ZSCAN9','ZUFSP','ZXDA','ZXDB','ZXDC','ZZZ3']
TFs=TFs1+TFs2

#TFs other, inclusive of the above list, not necessarily DNA binding proteins but affect transcription
TFs1Other=['AATF','ABL1','ABRA','AC008770.3','AC020915.1','AC023509.3','AC092835.1','AC105001.2','AC138696.1','ACOD1','ACTR3B','ACTR3C','ADAM8','ADAMTS17','ADAMTS19','ADAR','ADGRA1','ADGRG3','ADNP','ADNP2','AEBP1','AEBP2','AES','AFAP1 ','AFF1','AFF3','AFF4','AGER','AGT','AHCTF1','AHDC1','AHR','AHRR','AIM2','AIRE','AKAP8','AKAP8L','AKNA','AKT1','ALK','ALKBH2','ALX1','ALX3','ALX4','AMH','ANAPC2','ANHX','ANKRD30A','ANKZF1','ANXA3','ANXA4','AP5Z1','APBB3','APC2','APEX1','APOA4','APOBEC3A','APOBEC3B','APOBEC3G','APTX','AR','ARGFX','ARHGAP35','ARHGEF2','ARHGEF5','ARID1A','ARID1B','ARID2','ARID3A','ARID3B','ARID3C','ARID4A','ARID4B','ARID5A','ARID5B','ARNT','ARNT2','ARNTL','ARNTL2','ARRB1','ARRB2','ARX','ASCC1','ASCL1','ASCL2','ASCL3','ASCL4','ASCL5','ASH1L','ATF1','ATF2','ATF3','ATF4','ATF5','ATF6','ATF6B','ATF7','ATF7IP','ATM','ATMIN','ATOH1','ATOH7','ATOH8','ATR','ATRX','ATXN7','BACH1','BACH2','BAG1','BANF1','BANP','BARHL1','BARHL2','BARX1','BARX2','BATF','BATF2','BATF3','BAZ2A','BAZ2B','BBX','BCL10','BCL11A','BCL11B','BCL3','BCL6','BCL6B','BCLAF1','BCOR','BDP1','BEND4','BEX1','BHLHA15','BHLHA9','BHLHE22','BHLHE23','BHLHE40','BHLHE41','BHMG1','BIN1','BLM','BLZF1','BMI1','BMP2','BMP4','BMP7','BMPR1A','BNC1','BNC2','BOD1L1','BOLA1','BOLA2','BOLA2B','BOLA3','BORCS8-MEF2B','BPNT1','BPTF','BRCA1','BRD4','BRD7','BRD8','BRD9','BRF1','BRF2','BRMS1','BRPF1','BSX','BTAF1','BTBD8','BTG2','BTK','BTRC','BUD31','C11orf95','C14orf39','C17orf49','C1D','C20orf194','C3orf33','CACTIN','CAMK1D','CAMK2A','CAMTA1','CAMTA2','CAPN15','CAPN3','CARD11','CARD14','CARF','CARHSP1','CASP8AP2','CASZ1','CAT','CBFA2T2','CBFA2T3','CBFB','CBL','CBLL1','CBX2','CC2D1A','CC2D1B','CCDC124','CCDC169-SOHLH2','CCDC17','CCDC3','CCDC71','CCDC83','CCNT1','CCNT2','CD36','CD40','CD40LG','CDC5L','CDC6','CDK5RAP3','CDK7','CDK8','CDK9','CDKN1A','CDKN2A','CDX1','CDX2','CDX4','CEBPA','CEBPB','CEBPD','CEBPE','CEBPG','CEBPZ','CELF3','CENPA','CENPB','CENPBD1','CENPS','CENPT','CENPX','CERS2','CERS3','CERS4','CERS5','CERS6','CFLAR','CGGBP1','CHAMP1','CHCHD3','CHD1','CHD2','CHD3','CHD4','CHD6','CHD7','CHD9','CHP1','CHRAC1','CHUK','CHURC1','CIB1','CIC','CIITA','CIR1','CITED1','CITED2','CITED4','CIZ1','CLOCK','CLU','CMKLR1','CNBP','CNOT3','CNOT4','CNOT7','CNOT8','COMMD1','COMMD6','COMMD7','COPS2','COPS3','COPS4','CPEB1','CPSF4','CPSF4L','CPXCR1','CRAMP1','CREB1','CREB3','CREB3L1','CREB3L2','CREB3L3','CREB3L4','CREB5','CREBBP','CREBL2','CREBRF','CREBZF','CREG1','CREM','CRTC1','CRTC2','CRTC3','CRX','CSDC2','CSDE1','CSRNP1','CSRNP2','CSRNP3','CTBP1','CTBP2','CTCF','CTCFL','CTH','CTNNB1','CTNNBIP1','CUL1','CUL2','CUL3','CUL4A','CUL4B','CUL5','CUX1','CUX2','CXXC1','CXXC4','CXXC5','CYLD','CYP1B1','CYTL1','DAB2IP','DACH1','DACH2','DAP','DAXX','DBP','DBX1','DBX2','DCP1A','DDB1','DDB2','DDIT3','DDN','DDR2','DDRGK1','DDX3X','DDX5','DDX58','DEAF1','DEK','DEPDC1','DEPDC1B','DEPDC4','DEPDC5','DEPDC7','DEPTOR','DHX34','DHX36','DHX57','DISP3','DLX1','DLX2','DLX3','DLX4','DLX5','DLX6','DMAP1','DMBX1','DMC1','DMRT1','DMRT2','DMRT3','DMRTA1','DMRTA2','DMRTB1','DMRTC1','DMRTC2','DMTF1','DNA2','DNAJA3','DNAJC1','DNAJC2','DNAJC21','DNMT1','DNMT3A','DNMT3B','DNMT3L','DNTT','DNTTIP1','DOT1L','DPF1','DPF2','DPF3','DPRX','DR1','DRAP1','DRGX','DSP','DTX1','DUS3L','DUSP12','DUX1','DUX3','DUX4','DUXA','DVL1','DVL2','DVL3','DZIP1','DZIP1L','E2F1','E2F2','E2F3','E2F4','E2F5','E2F6','E2F7','E2F8','E4F1','EAF2','EBF1','EBF2','EBF3','EBF4','EDA','EDA2R','EDF1','EDN1','EEA1','EED','EGLN1','EGR1','EGR2','EGR3','EGR4','EHF','EID1','EID2','EIF2AK2','EIF2AK4','EIF3C','EIF3CL','EIF3K','ELF1','ELF2','ELF3','ELF4','ELF5','ELK1','ELK3','ELK4','ELMSAN1','EME1','EMX1','EMX2','EN1','EN2','ENO1','EOMES','EP300','EP400','EPAS1','EPHA5','EPM2AIP1','ERBIN','ERC1','ERCC3','ERCC4','ERF','ERG','ESR1','ESR2','ESRRA','ESRRB','ESRRG','ESX1','ETS1','ETS2','ETV1','ETV2','ETV3','ETV3L','ETV4','ETV5','ETV6','ETV7','EVX1','EVX2','EWSR1','EXO1','EXOC2','EXOG','EZH1','EZH2','FAAP24','FAM153C','FAM170A','FAM171B','FAM200A','FAM200B','FAN1','FANCA','FANCD2','FANK1','FARSA','FARSB','FBN1','FBXL19','FBXO41','FEN1','FER','FERD3L','FEV','FEZF1','FEZF2','FGD1','FHL2','FIGLA','FIZ1','FLI1','FLNA','FLYWCH1','FMNL2','FOS','FOSB','FOSL1','FOSL2','FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L2','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXN4','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1','FUBP1','FUBP3','FUS','FZD1','FZD2','FZD4','FZD6','GABPA','GABPB1','GABPB2','GADD45A','GAS6','GAS7','GATA1','GATA2','GATA3','GATA4','GATA5','GATA6','GATAD1','GATAD2A','GATAD2B','GBX1','GBX2','GCFC2','GCM1','GCM2','GEN1','GFI1','GFI1B','GLA','GLI1','GLI2','GLI3','GLI4','GLIS1','GLIS2','GLIS3','GLMP','GLYR1','GMEB1','GMEB2','GOLGB1','GON4L','GPATCH8','GPBP1','GPBP1L1','GPER1','GPR155','GREM1','GRHL1','GRHL2','GRHL3','GRM6','GSC','GSC2','GSX1','GSX2','GTF2A1','GTF2A1L','GTF2A2','GTF2B','GTF2E2','GTF2F1','GTF2F2','GTF2H1','GTF2H2','GTF2H2C','GTF2H3','GTF2H4','GTF2I','GTF2IRD1','GTF2IRD2','GTF2IRD2B','GTF3A','GTF3C1','GTF3C2','GULP1','GZF1','H1F0','H1FOO','H1FX','HAND1','HAND2','HAVCR2','HBP1','HCFC1','HCK','HDAC1','HDAC10','HDAC2','HDAC3','HDAC4','HDAC5','HDAC6','HDAC7','HDAC8','HDAC9','HDX','HELT','HELZ','HELZ2','HES1','HES2','HES3','HES4','HES5','HES6','HES7','HESX1','HEY1','HEY2','HEYL','HHEX','HIC1','HIC2','HIF1A','HIF3A','HINFP','HIPK2','HIRA','HIST1H1A','HIST1H1B','HIST1H1C','HIST1H1D','HIST1H1E','HIST1H1T','HIVEP1','HIVEP2','HIVEP3','HKR1','HLA-DQB1','HLA-DQB2','HLA-DRB1','HLA-DRB3','HLA-DRB5','HLF','HLTF','HLX','HMBOX1','HMG20A','HMG20B','HMGA1','HMGA2','HMGB1','HMGB2','HMGB3','HMGB4','HMGN1','HMGN3','HMGXB3','HMGXB4','HMOX1','HMX1','HMX2','HMX3','HNF1A','HNF1B','HNF4A','HNF4G','HNRNPA1','HNRNPA3','HNRNPAB','HNRNPCL2','HNRNPD','HNRNPDL','HNRNPK','HNRNPL','HNRNPU','HOMEZ','HOOK2','HOPX','HOXA1','HOXA10','HOXA11','HOXA13','HOXA2','HOXA3','HOXA4','HOXA5','HOXA6','HOXA7','HOXA9','HOXB1','HOXB13','HOXB2','HOXB3','HOXB4','HOXB5','HOXB6','HOXB7','HOXB8','HOXB9','HOXC10','HOXC11','HOXC12','HOXC13','HOXC4','HOXC5','HOXC6','HOXC8','HOXC9','HOXD1','HOXD10','HOXD11','HOXD12','HOXD13','HOXD3','HOXD4','HOXD8','HOXD9','HP1BP3','HR','HSBP1','HSF1','HSF2','HSF4','HSF5','HSFX1','HSFX2','HSFY1','HSFY2','HSPA1A','HSPA1B','ICAM1','ID1','ID2','ID3','ID4','IFI16','IFNB1','IGHM','IKBKB','IKBKG','IKZF1','IKZF2','IKZF3','IKZF4','IKZF5','IL10','IL1B','IL1RAP','IL4','IL5','IL6','INF2','ING4','INS','INSM1','INSM2','IQGAP1','IRAK1','IRAK2','IRAK3','IRF1','IRF2','IRF2BP1','IRF3','IRF4','IRF5','IRF6','IRF7','IRF8','IRF9','IRX1','IRX2','IRX3','IRX4','IRX5','IRX6','ISL1','ISL2','ISX','ITCH','ITGB2','ITGB3BP','IZUMO2','JADE1','JAK2','JARID2','JAZF1','JDP2','JMY','JRK','JRKL','JUN','JUNB','JUND','JUP','KANSL2','KAT2A','KAT2B','KAT5','KAT6A','KAT6B','KAT7','KAT8','KCMF1','KCNIP3','KDM1A','KDM2A','KDM2B','KDM3A','KDM5A','KDM5B','KDM5C','KDM5D','KEAP1','KIAA1549','KIAA1683','KIAA1841','KIAA1958','KIF15','KIN','KIT','KLF1','KLF10','KLF11','KLF12','KLF13','KLF14','KLF15','KLF16','KLF17','KLF2','KLF3','KLF4','KLF5','KLF6','KLF7','KLF8','KLF9','KMT2A','KMT2B','KMT2C','KMT2D','KNTC1','KRAS','KRBOX1','KRT13','KRTAP5-1','KRTAP5-9','KRTAP9-9','L3MBTL1','L3MBTL3','L3MBTL4','LARP1','LARP1B','LARP4','LARP4B','LARP6','LARP7','LBX1','LBX2','LCOR','LCORL','LEF1','LEUTX','LGALS9','LGR4','LHX1','LHX2','LHX3','LHX4','LHX5','LHX6','LHX8','LHX9','LIG1','LIG3','LIN28A','LIN28B','LIN54','LMO2','LMO4','LMX1A','LMX1B','LPP','LRP5','LRP6','LRP8','LRRFIP1','LRRFIP2','LSR','LTF','LTN1','LYAR','LYL1','LZTFL1','LZTR1','LZTS1','MACC1','MACF1','MAD2L2','MAEL','MAF','MAFA','MAFB','MAFF','MAFG','MAFK','MALT1','MAML1','MAML3','MAP2K5','MAP3K10','MAP3K13','MAP3K7','MAPK1','MAPK10','MAPK11','MAPK14','MAPK3','MAPK8','MAPK9','MATR3','MAVS','MAX','MAZ','MB21D1','MBD1','MBD2','MBD3','MBD4','MBD6','MBNL1','MBNL2','MBNL3','MBTPS2','MDFI','MDM2','MDM4','MECOM','MECP2','MED1','MED12','MED13','MED13L','MED14','MED15','MED16','MED17','MED21','MED23','MED24','MED6','MEF2A','MEF2B','MEF2C','MEF2D','MEIS1','MEIS2','MEIS3','MEN1','MEOX1','MEOX2','MESP1','MESP2','MET','MGA','MGMT','MID2','MIER1','MIER2','MIER3','MINK1','MIS18BP1','MITF','MIXL1','MKL1','MKL2','MKRN1','MKRN2','MKRN3','MKX','MLLT1','MLLT10','MLLT11','MLLT3','MLLT6','MLX','MLXIP','MLXIPL','MN1','MNAT1','MNT','MNX1','MORC3','MPG','MRPL28','MRRF','MSANTD1','MSANTD3','MSANTD4','MSC','MSGN1','MSH2','MSH3','MSH6','MSL3','MSRB2','MST1R','MSX1','MSX2','MTA1','MTA2','MTA3','MTDH','MTERF1','MTERF2','MTERF3','MTERF4','MTF1','MTF2','MTPN','MUS81','MXD1','MXD3','MXD4','MXI1','MYB','MYBBP1A','MYBL1','MYBL2','MYC','MYCBP','MYCL','MYCN','MYD88','MYEF2','MYF5','MYF6','MYNN','MYOCD','MYOD1','MYOG','MYPOP','MYRF','MYRFL','MYSM1','MYT1','MYT1L','MZF1','NAB2','NABP2','NACC2','NAIF1','NANOG','NANOGNB','NANOGP8','NCOA1','NCOA2','NCOA3','NCOA4','NCOA5','NCOA6','NCOA7','NCOR1','NCOR2','NDC80','NDN','NDP','NEB','NEIL1','NEUROD1','NEUROD2','NEUROD4','NEUROD6','NEUROG1','NEUROG2','NEUROG3','NFAM1','NFAT5','NFATC1','NFATC2','NFATC3','NFATC4','NFE2','NFE2L1','NFE2L2','NFE2L3','NFE4','NFIA','NFIB','NFIC','NFIL3','NFIX','NFKB1','NFKB2','NFKBIA','NFKBIB','NFKBID','NFKBIL1','NFKBIZ','NFRKB','NFX1','NFXL1','NFYA','NFYB','NFYC','NHLH1','NHLH2','NKRF','NKX1-1','NKX1-2','NKX2-1','NKX2-2','NKX2-3','NKX2-4','NKX2-5','NKX2-6','NKX2-8','NKX3-1','NKX3-2','NKX6-1','NKX6-2','NKX6-3','NLRC3','NLRC4','NLRC5','NLRP12','NLRP2B','NLRP3','NME2','NOBOX','NOC3L','NOC4L','NOCT','NOD1','NOD2','NODAL','NONO','NOP2','NOTCH1','NOTCH2','NOTCH3','NOTO','NPAS1','NPAS2','NPAS3','NPAS4','NPM1','NR0B1','NR0B2','NR1D1','NR1D2','NR1H2','NR1H3','NR1H4','NR1I2','NR1I3','NR2C1','NR2C2','NR2E1','NR2E3','NR2F1','NR2F2','NR2F6','NR3C1','NR3C2','NR4A1','NR4A2','NR4A3','NR5A1','NR5A2','NR6A1','NRBF2','NRF1','NRG1','NRIP1','NRIP3','NRK','NRL','NSD1','NSD2','NTRK1','NUCKS1','NUFIP1','NUP98','NUPL2']
TFs2Other=['OGG1','OLIG1','OLIG2','OLIG3','ONECUT1','ONECUT2','ONECUT3','OPRD1','OSR1','OSR2','OTOP3','OTP','OTULIN','OTX1','OTX2','OVOL1','OVOL2','OVOL3','PA2G4','PABPN1','PARK7','PARP1','PARP10','PARP12','PATZ1','PAWR','PAX1','PAX2','PAX3','PAX4','PAX5','PAX6','PAX7','PAX8','PAX9','PAXBP1','PAXIP1','PBRM1','PBX1','PBX2','PBX3','PBX4','PCBD1','PCBP1','PCBP2','PCBP3','PCGF2','PCGF6','PCNA','PCSK6','PDCD7','PDS5B','PDX1','PEG3','PELI1','PER1','PER2','PER3','PEX14','PFDN1','PGBD1','PGR','PGS1','PHB','PHB2','PHF1','PHF19 ','PHF20','PHF21A','PHF5A','PHOX2A','PHOX2B','PHTF1','PHTF2','PIAS1','PIAS2','PIAS3','PIAS4','PIDD1','PIKFYVE','PIM1','PIN1','PINK1','PITX1','PITX2','PITX3','PKHD1','PKHD1L1','PKNOX1','PKNOX2','PLA2G10','PLA2G1B','PLAG1','PLAGL1','PLAGL2','PLEK','PLEK2','PLEKHA4','PLPP3','PLSCR1','PLXNA1','PLXNA2','PLXNA3','PLXNA4','PLXNB1','PLXNB2','PLXNB3','PLXNC1','PLXND1','PML','PMS1','PNKP','POGK','POGZ','POLA1','POLB','POLE3','POLE4','POLG','POLH','POLI','POLK','POLL','POLM','POLN','POLQ','POLR1B','POLR2A','POLR2B','POLR2E','POLR2J','POLR2K','POLR2L','POLR3A','POLRMT','POT1','POU1F1','POU2AF1','POU2F1','POU2F2','POU2F3','POU3F1','POU3F2','POU3F3','POU3F4','POU4F1','POU4F2','POU4F3','POU5F1','POU5F1B','POU5F2','POU6F1','POU6F2','PPARA','PPARD','PPARG','PPARGC1A','PPARGC1B','PPP1R10','PPP1R13L','PPP1R21','PPP2R3B','PPP2R5B','PPP3CA','PPRC1','PQBP1','PRAM1','PRAP1','PRB3','PRDM1','PRDM10','PRDM11','PRDM12','PRDM13','PRDM14','PRDM15','PRDM16','PRDM2','PRDM4','PRDM5','PRDM6','PRDM7','PRDM8','PRDM9','PRDX3','PREB','PREX1','PREX2','PRH2','PRIM2','PRIMPOL','PRKCB','PRKCH','PRKCI','PRKCQ','PRKCZ','PRKD1','PRKD2','PRKN','PRMT2','PRMT3','PRNP','PROP1','PROX1','PROX2','PRR12','PRR3','PRRX1','PRRX2','PSMA6','PSMC5','PSMD10','PSMD11','PSMD12','PSPC1','PTCH1','PTEN','PTF1A','PTGIS','PTH','PTTG1','PUM3','PURA','PURB','PURG','PYCARD','PYDC1','PYDC2','QRICH1','RAB11FIP4','RAB7B','RAD21','RAD51','RAG1','RAI1','RAPGEF3','RAPGEF4','RAPGEF5','RARA','RARB','RARG','RAX','RAX2','RB1','RBAK','RBBP7','RBCK1','RBL1','RBL2','RBM10','RBM20','RBM22','RBM26','RBM27','RBM5','RBM6','RBPJ','RBPJL','RBSN','RC3H1','RC3H2','RCAN1','RCOR1','RCOR2','RCOR3','RECQL','REL','RELA','RELB','RELL2','RELN','REPIN1','RERE','REST','REV1','REXO4','RFC1','RFX1','RFX2','RFX3','RFX4','RFX5','RFX6','RFX7','RFX8','RFXANK','RFXAP','RGCC','RGS11','RGS6','RGS7','RGS9','RHEBL1','RHOXF1','RHOXF2','RHOXF2B','RING1','RIOK2','RIPK1','RIPK2','RIPK3','RIPK4','RLF','RLIM','RNASE2','RNASEH1','RNF113A','RNF113B','RNF114','RNF125','RNF138','RNF14','RNF166','RNF2','RNF219','RNF25','RNF31','RNF4','RNF41','RORA','RORB','RORC','RPA2','RPA4','RPL36A','RPL7','RPL7A','RPS27A','RPS6KA4','RPS6KA5','RREB1','RRN3','RUNX1','RUNX1T1','RUNX2','RUNX3','RWDD3','RXRA','RXRB','RXRG','RYBP','S100A12','S100A8','S100A9','SAFB','SAFB2','SALL1','SALL2','SALL3','SALL4','SART1','SATB1','SATB2','SAV1','SCAND1','SCAND2P','SCAPER','SCMH1','SCML1','SCML2','SCML4','SCOC','SCRT1','SCRT2','SCX','SEBOX','SEMA4A','SERTAD1','SERTAD2','SETBP1','SETD6','SETDB1','SETDB2','SF3A2','SF3A3','SFPQ','SFRP4','SFRP5','SFTPB','SGK1','SGK3','SGSM2','SHH','SHOX','SHOX2','SHPRH','SIGIRR','SIK1','SIM1','SIM2','SIN3A','SIN3B','SIRT1','SIRT2','SIRT3','SIRT6','SIVA1','SIX1','SIX2','SIX3','SIX4','SIX5','SIX6','SKI','SKIDA1','SKIL','SKOR1','SKOR2','SLC22A4','SLC26A10','SLC26A3','SLC2A4RG','SLC30A9','SLC39A10','SLC4A10','SMAD1','SMAD2','SMAD3','SMAD4','SMAD5','SMAD6','SMAD7','SMAD9','SMARCA1','SMARCA2','SMARCA4','SMARCA5','SMARCAL1','SMARCB1','SMARCC1','SMARCC2','SMARCD1','SMARCD2','SMARCD3','SMARCE1','SMC3','SMO','SMUG1','SMYD1','SMYD3','SNAI1','SNAI2','SNAI3','SNAPC2','SNAPC4','SNAPC5','SNRPC','SNURF','SNW1','SOGA3','SOHLH1','SOHLH2','SON','SORBS2','SOX1','SOX10','SOX11','SOX12','SOX13','SOX14','SOX15','SOX17','SOX18','SOX2','SOX21','SOX3','SOX30','SOX4','SOX5','SOX6','SOX7','SOX8','SOX9','SP1','SP100','SP110','SP140','SP140L','SP2','SP3','SP4','SP5','SP6','SP7','SP8','SP9','SPDEF','SPEN','SPHK1','SPI1','SPIB','SPIC','SPZ1','SRCAP','SREBF1','SREBF2','SRF','SRY','SSB','SSBP2','SSBP3','SSBP4','SSH1','SSH2','SSH3','SSRP1','SSX1','ST18','STAG1','STAT1','STAT2','STAT3','STAT4','STAT5A','STAT5B','STAT6','STK16','STK3','STK36','STRN','STRN3','SUB1','SUFU','SUMO1','SUPT16H','SUPT4H1','SUPT5H','SUPT6H','SUV39H1','SUV39H2','SUZ12','SYK','T','TAB1','TAB2','TAB3','TADA2A','TADA2B','TADA3','TAF1','TAF10','TAF11','TAF12','TAF13','TAF1A','TAF1B','TAF1C','TAF1L','TAF2','TAF3','TAF4','TAF4B','TAF5','TAF5L','TAF6','TAF6L','TAF7','TAF7L','TAF8','TAF9','TAF9B','TAL1','TAL2','TARDBP','TAX1BP1','TBL1Y','TBP','TBPL1','TBPL2','TBR1','TBX1','TBX10','TBX15','TBX18','TBX19','TBX2','TBX20','TBX21','TBX22','TBX3','TBX4','TBX5','TBX6','TCEA2','TCEAL1','TCEAL7','TCEAL8','TCF12','TCF15','TCF19','TCF20','TCF21','TCF23','TCF24','TCF25','TCF3','TCF4','TCF7','TCF7L1','TCF7L2','TCFL5','TDG','TDP1','TDP2','TEAD1','TEAD2','TEAD3','TEAD4','TEF','TERB1','TERF1','TERF2','TERF2IP','TESMIN','TET1','TET2','TET3','TFAM','TFAP2A','TFAP2B','TFAP2C','TFAP2D','TFAP2E','TFAP4','TFCP2','TFCP2L1','TFDP1','TFDP2','TFDP3','TFE3','TFEB','TFEC','TGFB1','TGFB1I1','TGIF1','TGIF2','TGIF2LX','TGIF2LY','TGS1','THAP1','THAP10','THAP11','THAP12','THAP2','THAP3','THAP4','THAP5','THAP6','THAP7','THAP8','THAP9','THRA','THRAP3','THRB','THYN1','TIA1','TICAM1','TIGD1','TIGD2','TIGD3','TIGD4','TIGD5','TIGD6','TIGD7','TIPARP','TIRAP','TLE1','TLE2','TLE3','TLE4','TLE6','TLR2','TLR3','TLR4','TLR9','TLX1','TLX2','TLX3','TMEM235','TMF1','TNF','TNFAIP3','TNFRSF11A','TNFRSF4','TNFSF11','TNFSF18','TNFSF4','TOE1','TONSL','TOP1','TOP2A','TOP2B','TOPORS','TOX','TOX2','TOX3','TOX4','TP53','TP53BP1','TP63','TP73','TPRX1','TRADD','TRAF1','TRAF2','TRAF3','TRAF4','TRAF5','TRAF6','TRAFD1','TRAK2','TRAPPC9','TRERF1','TREX1','TRIB1','TRIB3','TRIM13','TRIM14','TRIM15','TRIM21','TRIM22','TRIM23','TRIM24','TRIM25','TRIM26','TRIM27','TRIM28','TRIM29','TRIM3','TRIM31','TRIM32','TRIM33','TRIM34','TRIM37','TRIM38','TRIM40','TRIM5','TRIM52','TRIM62','TRIM8','TRIP11','TRIT1','TRMT1','TRMT1L','TRPS1','TRRAP','TSC2','TSC22D1','TSC22D2','TSC22D3','TSC22D4','TSHZ1','TSHZ2','TSHZ3','TSN','TSSK4','TTF1','TUB','TULP4','TUT1','TWIST1','TWIST2','TXK','U2AF1','U2AF1L4','UBA52','UBB','UBC','UBE2K','UBE2N','UBE2V1','UBN1','UBP1','UBR4','UBTF','UBTFL1','UBTFL6','UFL1','UHRF1','UHRF2','UNCX','UNG','UNK','UNKL','USF1','USF2','USF3','USP39','USP7','UTF1','VAV1','VAX1','VAX2','VDR','VEGFA','VENTX','VEZF1','VGLL1','VGLL2','VGLL3','VHLL','VPS72','VSX1','VSX2','WDHD1','WFS1','WIZ','WNT1','WNT10B','WNT2','WNT3A','WNT5A','WNT8B','WRN','WT1','WWOX','WWP2','WWTR1','XBP1','XCL1','XPA','YBX1','YBX2','YBX3','YEATS2','YEATS4','YOD1','YY1','YY2','ZBED1','ZBED2','ZBED3','ZBED4','ZBED5','ZBED6','ZBED6CL','ZBED8','ZBED9','ZBP1','ZBTB1','ZBTB10','ZBTB11','ZBTB12','ZBTB14','ZBTB16','ZBTB17','ZBTB18','ZBTB2','ZBTB20','ZBTB21','ZBTB22','ZBTB24','ZBTB25','ZBTB26','ZBTB3','ZBTB32','ZBTB33','ZBTB34','ZBTB37','ZBTB38','ZBTB39','ZBTB4','ZBTB40','ZBTB41','ZBTB42','ZBTB43','ZBTB44','ZBTB45','ZBTB46','ZBTB47','ZBTB48','ZBTB49','ZBTB5','ZBTB6','ZBTB7A','ZBTB7B','ZBTB7C','ZBTB8A','ZBTB8B','ZBTB9','ZC3H10','ZC3H11A','ZC3H12A','ZC3H13','ZC3H14','ZC3H15','ZC3H18','ZC3H3','ZC3H4','ZC3H6','ZC3H7A','ZC3H7B','ZC3H8','ZC3HAV1','ZCCHC11','ZCCHC6','ZDHHC11','ZDHHC19','ZEB1','ZEB2','ZFAND5','ZFAT','ZFHX2','ZFHX3','ZFHX4','ZFP1','ZFP14','ZFP2','ZFP28','ZFP3','ZFP30','ZFP36','ZFP36L1','ZFP36L2','ZFP37','ZFP41','ZFP42','ZFP57','ZFP62','ZFP64','ZFP69','ZFP69B','ZFP82','ZFP90','ZFP91','ZFP92','ZFPL1','ZFPM1','ZFPM2','ZFR','ZFR2','ZFX','ZFY','ZFYVE26','ZGLP1','ZGPAT','ZHX1','ZHX2','ZHX3','ZIC1','ZIC2','ZIC3','ZIC4','ZIC5','ZIK1','ZIM2','ZIM3','ZKSCAN1','ZKSCAN2','ZKSCAN3','ZKSCAN4','ZKSCAN5','ZKSCAN7','ZKSCAN8','ZMAT1','ZMAT2','ZMAT3','ZMAT4','ZMAT5','ZMYM1','ZMYM2','ZMYM3','ZMYM4','ZMYM5','ZMYM6','ZMYND10','ZNF10','ZNF100','ZNF101','ZNF106','ZNF107','ZNF112','ZNF114','ZNF117','ZNF12','ZNF121','ZNF124','ZNF131','ZNF132','ZNF133','ZNF134','ZNF135','ZNF136','ZNF137P','ZNF138','ZNF14','ZNF140','ZNF141','ZNF142','ZNF143','ZNF146','ZNF148','ZNF154','ZNF155','ZNF157','ZNF16','ZNF160','ZNF165','ZNF169','ZNF17','ZNF174','ZNF175','ZNF177','ZNF18','ZNF180','ZNF181','ZNF182','ZNF184','ZNF185','ZNF189','ZNF19','ZNF195','ZNF197','ZNF2','ZNF20','ZNF200','ZNF202','ZNF205','ZNF207','ZNF208','ZNF211','ZNF212','ZNF213','ZNF214','ZNF215','ZNF217','ZNF219','ZNF22','ZNF221','ZNF222','ZNF223','ZNF224','ZNF225','ZNF226','ZNF227','ZNF229','ZNF23','ZNF230','ZNF232','ZNF233','ZNF234','ZNF235','ZNF236','ZNF239','ZNF24','ZNF248','ZNF25','ZNF250','ZNF251','ZNF253','ZNF254','ZNF256','ZNF257','ZNF26','ZNF260','ZNF263','ZNF264','ZNF266','ZNF267','ZNF268','ZNF273','ZNF274','ZNF275','ZNF276','ZNF277','ZNF28','ZNF280A','ZNF280B','ZNF280C','ZNF280D','ZNF281','ZNF282','ZNF283','ZNF284','ZNF285','ZNF286A','ZNF286B','ZNF287','ZNF292','ZNF296','ZNF3','ZNF30','ZNF300','ZNF302','ZNF304','ZNF311','ZNF316','ZNF317','ZNF318','ZNF319','ZNF32','ZNF320','ZNF322','ZNF324','ZNF324B','ZNF326','ZNF329','ZNF330','ZNF331','ZNF333','ZNF334','ZNF335','ZNF337','ZNF33A','ZNF33B','ZNF34','ZNF341','ZNF343','ZNF345','ZNF346','ZNF347','ZNF35','ZNF350','ZNF354A','ZNF354B','ZNF354C','ZNF355P','ZNF358','ZNF362','ZNF365','ZNF366','ZNF367','ZNF37A','ZNF382','ZNF383','ZNF384','ZNF385A','ZNF385B','ZNF385C','ZNF385D','ZNF391','ZNF394','ZNF395','ZNF396','ZNF397','ZNF398','ZNF404','ZNF407','ZNF408','ZNF41','ZNF410','ZNF414','ZNF415','ZNF416','ZNF417','ZNF418','ZNF419','ZNF420','ZNF423','ZNF425','ZNF426','ZNF428','ZNF429','ZNF43','ZNF430','ZNF431','ZNF432','ZNF433','ZNF436','ZNF438','ZNF439','ZNF44','ZNF440','ZNF441','ZNF442','ZNF443','ZNF444','ZNF445','ZNF446','ZNF449','ZNF45','ZNF451','ZNF454','ZNF460','ZNF461','ZNF462','ZNF467','ZNF468','ZNF469','ZNF470','ZNF471','ZNF473','ZNF474','ZNF479','ZNF48','ZNF480','ZNF483','ZNF484','ZNF485','ZNF486','ZNF487','ZNF488','ZNF490','ZNF491','ZNF492','ZNF493','ZNF496','ZNF497','ZNF500','ZNF501','ZNF502','ZNF503','ZNF506','ZNF507','ZNF510','ZNF511','ZNF512','ZNF512B','ZNF513','ZNF514','ZNF516','ZNF517','ZNF518A','ZNF518B','ZNF519']
TFs3Other=['ZNF521','ZNF524','ZNF525','ZNF526','ZNF527','ZNF528','ZNF529','ZNF530','ZNF532','ZNF534','ZNF536','ZNF540','ZNF541','ZNF542P','ZNF543','ZNF544','ZNF546','ZNF547','ZNF548','ZNF549','ZNF550','ZNF551','ZNF552','ZNF554','ZNF555','ZNF556','ZNF557','ZNF558','ZNF559','ZNF560','ZNF561','ZNF562','ZNF563','ZNF564','ZNF565','ZNF566','ZNF567','ZNF568','ZNF569','ZNF57','ZNF570','ZNF571','ZNF572','ZNF573','ZNF574','ZNF575','ZNF576','ZNF577','ZNF578','ZNF579','ZNF580','ZNF581','ZNF582','ZNF583','ZNF584','ZNF585A','ZNF585B','ZNF586','ZNF587','ZNF587B','ZNF589','ZNF592','ZNF593','ZNF594','ZNF595','ZNF596','ZNF597','ZNF598','ZNF599','ZNF600','ZNF605','ZNF606','ZNF607','ZNF608','ZNF609','ZNF610','ZNF611','ZNF613','ZNF614','ZNF615','ZNF616','ZNF618','ZNF619','ZNF620','ZNF621','ZNF622','ZNF623','ZNF624','ZNF625','ZNF626','ZNF627','ZNF628','ZNF629','ZNF630','ZNF638','ZNF639','ZNF641','ZNF644','ZNF645','ZNF646','ZNF648','ZNF649','ZNF652','ZNF653','ZNF654','ZNF655','ZNF658','ZNF66','ZNF660','ZNF662','ZNF664','ZNF665','ZNF667','ZNF668','ZNF669','ZNF670','ZNF671','ZNF672','ZNF674','ZNF675','ZNF676','ZNF677','ZNF678','ZNF679','ZNF680','ZNF681','ZNF682','ZNF683','ZNF684','ZNF687','ZNF688','ZNF689','ZNF69','ZNF691','ZNF692','ZNF695','ZNF696','ZNF697','ZNF699','ZNF7','ZNF70','ZNF700','ZNF701','ZNF703','ZNF704','ZNF705A','ZNF705B','ZNF705D','ZNF705E','ZNF705G','ZNF706','ZNF707','ZNF708','ZNF709','ZNF71','ZNF710','ZNF711','ZNF713','ZNF714','ZNF716','ZNF717','ZNF718','ZNF720','ZNF721','ZNF724','ZNF726','ZNF727','ZNF728','ZNF729','ZNF73','ZNF730','ZNF732','ZNF735','ZNF736','ZNF737','ZNF738','ZNF74','ZNF740','ZNF746','ZNF747','ZNF749','ZNF750','ZNF75A','ZNF75D','ZNF76','ZNF761','ZNF763','ZNF764','ZNF765','ZNF766','ZNF768','ZNF77','ZNF770','ZNF771','ZNF772','ZNF773','ZNF774','ZNF775','ZNF776','ZNF777','ZNF778','ZNF780A','ZNF780B','ZNF781','ZNF782','ZNF783','ZNF784','ZNF785','ZNF786','ZNF787','ZNF788','ZNF789','ZNF79','ZNF790','ZNF791','ZNF792','ZNF793','ZNF799','ZNF8','ZNF80','ZNF800','ZNF804A','ZNF804B','ZNF805','ZNF808','ZNF81','ZNF812P','ZNF813','ZNF814','ZNF816','ZNF821','ZNF823','ZNF827','ZNF829','ZNF83','ZNF830','ZNF831','ZNF835','ZNF836','ZNF837','ZNF839','ZNF84','ZNF841','ZNF843','ZNF844','ZNF845','ZNF846','ZNF85','ZNF850','ZNF852','ZNF853','ZNF860','ZNF862','ZNF865','ZNF878','ZNF879','ZNF880','ZNF883','ZNF888','ZNF891','ZNF90','ZNF91','ZNF92','ZNF93','ZNF98','ZNF99','ZNFX1','ZRANB2','ZRSR1','ZRSR2','ZSCAN1','ZSCAN10','ZSCAN12','ZSCAN16','ZSCAN18','ZSCAN2','ZSCAN20','ZSCAN21','ZSCAN22','ZSCAN23','ZSCAN25','ZSCAN26','ZSCAN29','ZSCAN30','ZSCAN31','ZSCAN32','ZSCAN4','ZSCAN5A','ZSCAN5B','ZSCAN5C','ZSCAN5DP','ZSCAN9','ZUFSP','ZXDA','ZXDB','ZXDC','ZZZ3']
TFs4Other=['ATF2','ATF6B','ATN1','BHLHA15','BHLHB2','BHLHB4 ','BHLHB5','BHLHB9','C13ORF8','C20ORF194','CDX2','CRAMP1L','ELP3','ERVK3-1','ETF1','FLJ12895','FLJ20321','FLJ25476','FLJ35784','FOXM1','FOXO2','FOXR1','FOXR2','GATA1','GRLF1','KDM4C','KDM4E','KIAA0863','KIAA0961','KLRG1','LBXCOR1','LEF1','LHX7','LOC126295','LOC51058','LOC91661','MGC4728','MYOD1','MYOG','MYRF','NEUROD4','NEUROG1','NFAT2','NFAT4','NFE2L1','NR1A2','NR1B1','NR1B2','NR1B3','NR1C1','NR1C2','NR1C3','NR1F3','NR1I1','NR2A1','NR2B3','NR3A1','NR3A2','NR3B3','OTEX','PEPP-2','PHF2','POU2F1','POU2F2','POU4F3','POU5F1','POU6F1','PRDM3','PRKRIP1','PRKRIR','SETD1A','SMAD8','SMYD2','SOX12','SOX14','SOX15','TBX14','TCF1','TCF17','TCF2','TCF9','TEAD1','TPRXL','UBTFL1','WHSC1','ZBTB14','ZBTB18','ZBTB19','ZBTB21','ZBTB27','ZBTB28','ZBTB35','ZBTB49','ZFP161','ZFP28','ZKSCAN10','ZKSCAN11','ZKSCAN12','ZKSCAN13','ZKSCAN14','ZKSCAN16','ZKSCAN19','ZKSCAN21','ZKSCAN24','ZKSCAN6','ZKSCAN8','ZNF10','ZNF112','ZNF187','ZNF193','ZNF204P','ZNF238','ZNF271','ZNF295','ZNF305','ZNF323','ZNF434','ZNF499','ZNF542','ZNF545','ZNF553','ZNF56','ZNF625','ZNF643','ZNF698','ZNF705F','ZNF751','ZNF752','ZNF753','ZNF785','ZNF806','ZNF816','ZNF833P','ZNF906','ZSCAN1','ZSCAN25','ZSCAN26','ZSCAN31','ZSCAN32','ZSCAN5D','ZSCAN9']
TFsOther=TFs1Other+TFs2Other+TFs3Other+TFs4Other
```

Example 8 (unknown):
```unknown
S genes: ['MCM5', 'PCNA', 'TYMS', 'FEN1', 'MCM2', 'MCM4', 'RRM1', 'UNG', 'GINS2', 'MCM6', 'CDCA7', 'DTL', 'PRIM1', 'UHRF1', 'HELLS', 'RFC2', 'RPA2', 'NASP', 'RAD51AP1', 'GMNN', 'WDR76', 'SLBP', 'CCNE2', 'UBR7', 'POLD3', 'MSH2', 'ATAD2', 'RAD51', 'RRM2', 'CDC45', 'CDC6', 'EXO1', 'TIPIN', 'DSCC1', 'BLM', 'CASP8AP2', 'USP1', 'CLSPN', 'POLA1', 'CHAF1B', 'BRIP1', 'E2F8'] 

G2 genes: ['HMGB2', 'CDK1', 'NUSAP1', 'UBE2C', 'BIRC5', 'TPX2', 'TOP2A', 'NDC80', 'CKS2', 'NUF2', 'CKS1B', 'MKI67', 'TMPO', 'CENPF', 'TACC3', 'SMC4', 'CCNB2', 'CKAP2L', 'CKAP2', 'AURKB', 'BUB1', 'KIF11', 'ANP32E', 'TUBB4B', 'GTSE1', 'KIF20B', 'HJURP', 'CDCA3', 'CDC20', 'TTK', 'CDC25C', 'KIF2C', 'RANGAP1', 'NCAPD2', 'DLGAP5', 'CDCA2', 'CDCA8', 'ECT2', 'KIF23', 'HMMR', 'AURKA', 'PSRC1', 'ANLN', 'LBR', 'CKAP5', 'CENPE', 'CTCF', 'NEK2', 'G2E3', 'GAS2L3', 'CBX5', 'CENPA'] 

Hemoglobin genes set 1: ['HBB', 'HBD', 'HBG1', 'HBG2', 'HBE1', 'HBZ', 'HBM', 'HBA2', 'HBA1', 'HBQ1'] 

Hemoglobin genes set 2: ['AHSP', 'GYPA', 'RHAG', 'GFI1B', 'KEL', 'EPB42', 'ALAS2', 'TSPO2', 'GYPB', 'GYPE', 'SPTA1', 'RHCE', 'KLF1', 'RHD', 'SLC4A1', 'SNX22', 'SQSTM1', 'SOD1', 'FTH1', 'MAP1LC3B', 'SLC25A37', 'TRIM58', 'HMBS'] 

Mitochondrial genes: ['MT-ND1', 'MT-ND2', 'MT-CO1', 'MT-CO2', 'MT-ATP8', 'MT-ATP6', 'MT-CO3', 'MT-ND3', 'MT-ND4L', 'MT-ND4', 'MT-ND5', 'MT-ND6', 'MT-CYB'] 

Ribo genes: ['RPLP0', 'RPLP1', 'RPLP2', 'RPL3', 'RPL3L', 'RPL4', 'RPL5', 'RPL6', 'RPL7', 'RPL7A', 'RPL7L1', 'RPL8', 'RPL9', 'RPL10', 'RPL10A', 'RPL10L', 'RPL11', 'RPL12', 'RPL13', 'RPL13A', 'RPL14', 'RPL15', 'RPL17', 'RPL18', 'RPL18A', 'RPL19', 'RPL21', 'RPL22', 'RPL22L1', 'RPL23', 'RPL23A', 'RPL24', 'RPL26', 'RPL26L1', 'RPL27', 'RPL27A', 'RPL28', 'RPL29', 'RPL30', 'RPL31', 'RPL32', 'RPL34', 'RPL35', 'RPL35A', 'RPL36', 'RPL36A', 'RPL36AL', 'RPL37', 'RPL37A', 'RPL38', 'RPL39', 'RPL39L', 'UBA52', 'RPL41', 'RPSA', 'RPS2', 'RPS3', 'RPS3A', 'RPS4X', 'RPS4Y1', 'RPS4Y2', 'RPS5', 'RPS6', 'RPS7', 'RPS8', 'RPS9', 'RPS10', 'RPS11', 'RPS12', 'RPS13', 'RPS14', 'RPS15', 'RPS15A', 'RPS16', 'RPS17', 'RPS18', 'RPS19', 'RPS20', 'RPS21', 'RPS23', 'RPS24', 'RPS25', 'RPS26', 'RPS27', 'RPS27A', 'RPS27L', 'RPS28', 'RPS29', 'FAU'] 

Tissue dissociation stress associated genes: ['FOS', 'FOSB', 'JUN', 'JUNB', 'JUND', 'ZFP36', 'SOCS3', 'CEBPD', 'CEBPB', 'ATF3', 'EGR1', 'HSPA1A', 'HSPA1B', 'HSP90AA1', 'HSP90AB1', 'HSPA8', 'HSPB1', 'HSPE1', 'IER3', 'IER2', 'BTG1', 'BTG2', 'DUSP1']
```

Example 9 (python):
```python
set(adata.obs.coarse_grain)
```

Example 10 (unknown):
```unknown
{'Cardiomyocytes',
 'Endothelium',
 'Epicardium',
 'Immune',
 'Mesenchymal',
 'Neural'}
```

Example 11 (python):
```python
adata=adata[adata.obs.coarse_grain=='Epicardium',:].copy()
adata
```

Example 12 (unknown):
```unknown
AnnData object with n_obs × n_vars = 3879 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'batch_key', '_scvi_batch', '_scvi_labels', 'previous_coarse_grain', 'previous_mid_grain', 'previous_fine_grain', 'leiden_scVI', 'FACSgate', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', 'hb2_score', 'leiden', 'coarse_grain', 'sex'
    var: 'gene_name_scRNA-0', 'gene_id'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'dendrogram_leiden', 'donor_colors', 'gender_colors', 'hvg', 'kit_10x_colors', 'leiden', 'leiden_colors', 'modality_colors', 'neighbors', 'phase_colors', 'rank_genes_groups', 'region_colors', 'sex_colors', 'umap'
    obsm: 'X_pca', 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 13 (python):
```python
# preserve raw count
adata.layers["counts"] = adata.X.copy()

# normalise and log-transform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key='batch_key')

print(sum(adata.var.highly_variable))
sc.pl.highly_variable_genes(adata)

# freeze the state in `.raw`
adata.raw = adata
```

Example 14 (python):
```python
normalizing counts per cell
    finished (0:00:00)
extracting highly variable genes
    finished (0:00:06)
--> added
    'highly_variable', boolean vector (adata.var)
    'means', float vector (adata.var)
    'dispersions', float vector (adata.var)
    'dispersions_norm', float vector (adata.var)
1166
```

Example 15 (python):
```python
# subset hvgs
adata = adata[:, adata.var.highly_variable]

adata=adata.copy()

sc.tl.score_genes_cell_cycle(adata, s_genes=s_genes, g2m_genes=g2m_genes)
#sc.pp.regress_out(adata, ['S_score','G2M_score'])

#batch correction
scvi.model.SCVI.setup_anndata(adata, 
                              layer="counts", 
                              batch_key="batch_key",
                              categorical_covariate_keys=['phase'],
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo'])

vae = scvi.model.SCVI(adata, n_hidden = 128, n_latent = 50, n_layers = 3, dispersion = 'gene-batch')
vae.train(max_epochs=50)
    
# Convergence plot
x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
plt.plot(x, vae.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
plt.clf()
```

Example 16 (python):
```python
calculating cell cycle phase
computing score 'S_score'
WARNING: genes are not in var_names and ignored: ['MCM5', 'PCNA', 'FEN1', 'MCM2', 'MCM4', 'UNG', 'GINS2', 'MCM6', 'CDCA7', 'DTL', 'PRIM1', 'UHRF1', 'HELLS', 'RFC2', 'RPA2', 'NASP', 'GMNN', 'WDR76', 'SLBP', 'CCNE2', 'UBR7', 'POLD3', 'MSH2', 'RAD51', 'CDC45', 'CDC6', 'EXO1', 'TIPIN', 'DSCC1', 'BLM', 'CASP8AP2', 'USP1', 'CLSPN', 'POLA1', 'CHAF1B', 'E2F8']
    finished: added
    'S_score', score of gene set (adata.obs).
    164 total control genes are used. (0:00:00)
computing score 'G2M_score'
WARNING: genes are not in var_names and ignored: ['CKS1B', 'TMPO', 'CKAP2L', 'ANP32E', 'HJURP', 'CDC25C', 'KIF2C', 'RANGAP1', 'CDCA2', 'PSRC1', 'LBR', 'CKAP5', 'CTCF', 'NEK2', 'G2E3', 'GAS2L3', 'CBX5']
    finished: added
    'G2M_score', score of gene set (adata.obs).
    519 total control genes are used. (0:00:00)
-->     'phase', cell cycle phase (adata.obs)
```

Example 17 (unknown):
```unknown
GPU available: False, used: False
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
```

Example 18 (unknown):
```unknown
Epoch 50/50: 100%|███████████| 50/50 [00:15<00:00,  3.21it/s, loss=930, v_num=1]
```

Example 19 (unknown):
```unknown
<Figure size 320x320 with 0 Axes>
```

Example 20 (python):
```python
# add the latent space to adata
adata.obsm["X_scVI"] = vae.get_latent_representation()

sc.tl.score_genes(adata, stress_genes, score_name='stress_score')
sc.tl.score_genes(adata, hb_genes1, score_name='hb1_score')
sc.tl.score_genes(adata, hb_genes2, score_name='hb2_score')
sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata,min_dist=0.3)
#sc.tl.tsne(adata)
```

Example 21 (python):
```python
computing score 'stress_score'
WARNING: genes are not in var_names and ignored: ['JUND', 'CEBPB', 'HSP90AA1', 'HSP90AB1', 'HSPA8', 'HSPB1', 'HSPE1', 'BTG1']
    finished: added
    'stress_score', score of gene set (adata.obs).
    221 total control genes are used. (0:00:00)
computing score 'hb1_score'
WARNING: genes are not in var_names and ignored: ['HBB', 'HBD', 'HBE1', 'HBZ', 'HBM', 'HBQ1']
    finished: added
    'hb1_score', score of gene set (adata.obs).
    143 total control genes are used. (0:00:00)
computing score 'hb2_score'
WARNING: genes are not in var_names and ignored: ['AHSP', 'GYPA', 'RHAG', 'GFI1B', 'KEL', 'EPB42', 'ALAS2', 'TSPO2', 'GYPB', 'GYPE', 'SPTA1', 'RHCE', 'KLF1', 'RHD', 'SLC4A1', 'SNX22', 'SOD1', 'FTH1', 'MAP1LC3B', 'SLC25A37', 'TRIM58', 'HMBS']
    finished: added
    'hb2_score', score of gene set (adata.obs).
    48 total control genes are used. (0:00:00)
computing neighbors
    finished: added to `.uns['neighbors']`
    `.obsp['distances']`, distances for each pair of neighbors
    `.obsp['connectivities']`, weighted adjacency matrix (0:00:01)
computing UMAP
    finished: added
    'X_umap', UMAP coordinates (adata.obsm) (0:00:02)
```

Example 22 (python):
```python
sc.pl.umap(adata, color=['log1p_total_counts','n_genes','n_genes_by_counts','total_counts','pct_counts_mt','pct_counts_ribo'],ncols=5)
sc.pl.umap(adata, color=['scrublet_score','stress_score', 'hb1_score','hb2_score','phase'], ncols=5,)
sc.pl.umap(adata, color=['donor','sex','age','region','phase','cell_or_nuclei','modality','kit_10x','FACSgate'],wspace=0.8,ncols=5)
```

Example 23 (python):
```python
Genes=['TTN','TNNT2','MYOM1','LDB3','ALDH1A2','TNNT1','KRT19','UPK3B','COL6A3','FBLN1','PCOLCE','LRRC17','EMCN','ECSCR','PECAM1','CDH5', 'LAPTM5','PTPRC','LCP1','CD53','PHOX2B','NRXN1','STMN2','GRIK2']
sc.pl.umap(adata, color=Genes)
```

Example 24 (python):
```python
sc.pl.umap(adata, color='previous_fine_grain')
```

Example 25 (python):
```python
sc.pl.umap(adata, color=['TCF21','POSTN','TBX18'])
```

Example 26 (python):
```python
sc.tl.leiden(adata, resolution=0.05, key_added='leiden')
sc.pl.umap(adata, color='leiden',legend_loc='on data')
```

Example 27 (python):
```python
running Leiden clustering
    finished: found 2 clusters and added
    'leiden', the cluster labels (adata.obs, categorical) (0:00:00)
```

Example 28 (python):
```python
sc.tl.rank_genes_groups(adata, groupby='leiden', method='t-test')
sc.tl.dendrogram(adata, groupby='leiden')
sc.pl.dendrogram(adata,groupby='leiden')
sc.pl.rank_genes_groups_matrixplot(adata, groupby='leiden', standard_scale='var')
```

Example 29 (python):
```python
ranking genes
    finished: added to `.uns['rank_genes_groups']`
    'names', sorted np.recarray to be indexed by group ids
    'scores', sorted np.recarray to be indexed by group ids
    'logfoldchanges', sorted np.recarray to be indexed by group ids
    'pvals', sorted np.recarray to be indexed by group ids
    'pvals_adj', sorted np.recarray to be indexed by group ids (0:00:00)
    using 'X_pca' with n_pcs = 50
Storing dendrogram info using `.uns['dendrogram_leiden']`
```

Example 30 (unknown):
```unknown
WARNING: Dendrogram not added. Dendrogram is added only when the number of categories to plot > 2
```

Example 31 (python):
```python
set(adata.obs.previous_fine_grain)
```

Example 32 (unknown):
```unknown
{'Cardiofibromyocytes',
 'CoronaryEndothelialCellsArterial',
 'CoronaryVesselAdventitialFibroblasts',
 'EarlyGreatVesselCells',
 'EndocardialCells',
 'FibroblasticEpicardialCells',
 'GreatVesselAdventitialFibroblasts',
 'GreatVesselSmoothMuscleCells',
 'MesothelialEpicardialCells',
 'MyocardialInterstitialFibroblasts',
 'Myofibroblasts',
 'ValveInterstitialCells',
 nan}
```

Example 33 (python):
```python
adata.obs['fine_grain']= 'na'

adata.obs['fine_grain'][adata.obs['leiden']=='0']='MesothelialEpicardialCells'
adata.obs['fine_grain'][adata.obs['leiden']=='1']='FibroblasticEpicardialCells'

sc.pl.umap(adata, color='fine_grain')
```

Example 34 (unknown):
```unknown
... storing 'fine_grain' as categorical
```

Example 35 (python):
```python
adata.write('./Objects/Annotated/Epicardium.h5ad')
##adata=sc.read('./Objects/Annotated/Epicardium.h5ad')
```

Example 36 (python):
```python
adata=sc.read('./Objects/Annotated/Epicardium.h5ad')
```

Example 37 (python):
```python
set(adata.obs.fine_grain)
```

Example 38 (unknown):
```unknown
{'FibroblasticEpicardialCells', 'MesothelialEpicardialCells'}
```

Example 39 (python):
```python
bdata=adata[adata.obs.fine_grain=='MesothelialEpicardialCells',:].copy()
bdatanames=bdata.obs_names

np.savetxt('MesothelialEpicardialCells.txt', bdatanames, delimiter="\n", fmt="%s")
```

---

## Import modules¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig4_Cardiomyocytes/VentricularCardiomyocytes/TransmuralAxis/tm_axis_gene.html

**Contents:**
- Import modules¶
- Functions¶
- Read in adata¶
- Add axis¶
- Differential test¶
- Slect genes to plot¶
- Pathway analysis¶
- Plot genes¶

**Examples:**

Example 1 (python):
```python
import os
import numpy as np
import scanpy as sc
import anndata
import pandas as pd
import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 
import matplotlib.pyplot as plt
import seaborn as sns

import warnings
warnings.simplefilter(action='ignore', category=FutureWarning)
warnings.simplefilter("ignore", UserWarning)
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
-----
anndata             0.9.2
matplotlib          3.7.2
numpy               1.24.4
pandas              1.5.3
scanpy              1.9.3
seaborn             0.12.2
session_info        1.0.0
-----
```

Example 4 (python):
```python
PIL                         10.0.0
anyio                       NA
arrow                       1.2.3
asttokens                   NA
attr                        23.1.0
attrs                       23.1.0
babel                       2.12.1
backcall                    0.2.0
brotli                      1.0.9
certifi                     2023.11.17
cffi                        1.15.1
charset_normalizer          3.2.0
colorama                    0.4.6
comm                        0.1.4
cycler                      0.10.0
cython_runtime              NA
dateutil                    2.8.2
debugpy                     1.6.8
decorator                   5.1.1
defusedxml                  0.7.1
executing                   1.2.0
fastjsonschema              NA
fqdn                        NA
h5py                        3.9.0
idna                        3.4
igraph                      0.10.6
importlib_metadata          NA
importlib_resources         NA
ipykernel                   6.25.1
isoduration                 NA
jedi                        0.19.0
jinja2                      3.1.2
joblib                      1.3.2
json5                       NA
jsonpointer                 2.0
jsonschema                  4.19.0
jsonschema_specifications   NA
jupyter_events              0.7.0
jupyter_server              2.7.1
jupyterlab_server           2.24.0
kiwisolver                  1.4.5
leidenalg                   0.10.1
llvmlite                    0.40.1
markupsafe                  2.1.3
mpl_toolkits                NA
natsort                     8.4.0
nbformat                    5.9.2
numba                       0.57.1
overrides                   NA
packaging                   23.1
parso                       0.8.3
patsy                       0.5.3
pexpect                     4.8.0
pickleshare                 0.7.5
pkg_resources               NA
plotly                      5.16.1
prometheus_client           NA
prompt_toolkit              3.0.39
psutil                      5.9.5
ptyprocess                  0.7.0
pure_eval                   0.2.2
pydev_ipython               NA
pydevconsole                NA
pydevd                      2.9.5
pydevd_file_utils           NA
pydevd_plugins              NA
pydevd_tracing              NA
pygments                    2.16.1
pyparsing                   3.0.9
pythonjsonlogger            NA
pytz                        2023.3
referencing                 NA
requests                    2.31.0
rfc3339_validator           0.1.4
rfc3986_validator           0.1.1
rpds                        NA
ruamel                      NA
scipy                       1.10.1
send2trash                  NA
six                         1.16.0
sklearn                     1.3.0
sniffio                     1.3.0
socks                       1.7.1
sphinxcontrib               NA
stack_data                  0.6.2
statsmodels                 0.14.0
texttable                   1.6.7
threadpoolctl               3.2.0
tornado                     6.3.3
traitlets                   5.9.0
typing_extensions           NA
uri_template                NA
urllib3                     2.0.4
wcwidth                     0.2.6
webcolors                   1.13
websocket                   1.6.2
yaml                        6.0.1
zipp                        NA
zmq                         25.1.1
zstandard                   0.19.0
```

Example 5 (unknown):
```unknown
-----
IPython             8.12.0
jupyter_client      8.3.0
jupyter_core        4.12.0
jupyterlab          4.0.5
notebook            7.0.2
-----
Python 3.8.17 (default, Jul  5 2023, 21:04:15) [GCC 11.2.0]
Linux-5.4.0-171-generic-x86_64-with-glibc2.17
-----
Session information updated at 2024-02-22 22:02
```

Example 6 (python):
```python
import importlib.util
import sys

spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)

spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/tissueaxis.py")
tissueaxis = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = tissueaxis
spec.loader.exec_module(tissueaxis)
```

Example 7 (python):
```python
sc.settings.set_figure_params(dpi=80,dpi_save=300)
```

Example 8 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk'
```

Example 9 (python):
```python
# This function selects the data for multiple slides from the spatial anndata object.
# Modified `select_slide` in cell2loc for multiple slides

# @param adata: anndata object with multiple spatial experiments
# @param s: list of selected slides
# @param s_col: column in adata.obs listing experiment name for each location

def select_multislide(adata, s, s_col='sample'):

    slides = adata[adata.obs[s_col].isin(s), :]
    s_keys = list(slides.uns['spatial'].keys())
    
    uns_dict = {}
    for ss in s:
        s_spatial=np.array(s_keys)[[ss in k for k in s_keys]][0]
        uns_dict[s_spatial]=slides.uns['spatial'][s_spatial]
    
    slides.uns['spatial'] = uns_dict

    return slides
```

Example 10 (python):
```python
adata_oct_healthy = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/visium-OCT_foetal_raw.h5ad')
print(adata_oct_healthy.shape)
adata_oct_healthy_LV = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/visium-OCT-LV_foetal_raw.h5ad')
print(adata_oct_healthy_LV.shape)
```

Example 11 (unknown):
```unknown
(14261, 33538)
(6767, 33538)
```

Example 12 (python):
```python
# concatenate oct adata
adata_oct = adata_oct_healthy.concatenate(
    [adata_oct_healthy_LV],
    batch_key=None,
    index_unique=None,
    join="inner"
)
```

Example 13 (python):
```python
# log-normalise
sc.pp.normalize_total(adata_oct, target_sum=1e4)
sc.pp.log1p(adata_oct)

# sc.pp.normalize_total(adata_ffpe, target_sum=1e4)
# sc.pp.log1p(adata_ffpe)

# reset var_names
adata_oct.var.set_index('SYMBOL',inplace=True)
# adata_ffpe.var.set_index('SYMBOL',inplace=True)
```

Example 14 (unknown):
```unknown
!ls /nfs/team205/heart/visium_axis/foetal/transmural_LV
```

Example 15 (unknown):
```unknown
BHF_F_Hea13295830.csv	HCAHeartST10317186.csv	HCAHeartST13162330.csv
BHF_F_Hea13295831.csv	HCAHeartST11605165.csv	HCAHeartST13162331.csv
HCAHeartST10317184.csv	HCAHeartST11605167.csv	HCAHeartST13162336.csv
HCAHeartST10317185.csv	HCAHeartST11605168.csv
```

Example 16 (python):
```python
# add
oct_samples = list(set(adata_oct.obs['sample']))

adata_oct.obs['transmural_axis'] = np.nan

axis_files = os.listdir('/nfs/team205/heart/visium_axis/foetal/transmural_LV')
for file in axis_files:
    # read in
    axis = pd.read_csv(f'/nfs/team205/heart/visium_axis/foetal/transmural_LV/{file}',index_col=0)
    # add to anndata
    if file.split('.csv')[0] in oct_samples:
        adata_oct.obs.loc[axis.index,'transmural_axis'] = axis.copy()

# select spots with axis
adata_oct = adata_oct[adata_oct.obs['transmural_axis'].isna()==False]
```

Example 17 (python):
```python
# make bin
adata_oct.obs['transmural_axis_bin'] = pd.cut(adata_oct.obs['transmural_axis'],bins=15)
```

Example 18 (python):
```python
# annotate bin, OCT
adata_oct.obs['tm_bin_annotations'] = adata_oct.obs['transmural_axis_bin'].astype('str').copy()
adata_oct.obs.replace({'tm_bin_annotations':{
    '(-1.002, -0.867]':'outer',
    '(-0.867, -0.733]':'outer',
    '(-0.733, -0.6]':'outer',
    '(-0.6, -0.467]':'outer',
    '(-0.467, -0.333]':'outer',
    '(-0.333, -0.2]':'middle',
    '(-0.2, -0.0667]':'middle',
    '(-0.0667, 0.0667]':'middle',
    '(0.0667, 0.2]':'middle',
    '(0.2, 0.333]':'middle',
    '(0.333, 0.467]':'inner',
    '(0.467, 0.6]':'inner',
    '(0.6, 0.733]':'inner',
    '(0.733, 0.867]':'inner',
    '(0.867, 1.0]':'inner',
}}, inplace=True)

set(adata_oct.obs['tm_bin_annotations'])
```

Example 19 (unknown):
```unknown
{'inner', 'middle', 'outer'}
```

Example 20 (python):
```python
# mean value per group
group_mean_val = utils.grouped_obs_mean(adata_oct, group_key='tm_bin_annotations')
# DE test
sc.tl.rank_genes_groups(adata_oct, 'tm_bin_annotations', method='t-test')
res = sc.get.rank_genes_groups_df(adata_oct, group=None, pval_cutoff=0.05)
# add mean gene expression
df = group_mean_val.reset_index().melt(id_vars=['SYMBOL'])
df.rename(columns={'SYMBOL':'names','variable':'group','value':'mean_gene_exp'},
          inplace=True)
diff_res = res.merge(df, how='left', on=['group','names'])
diff_res.head()
```

Example 21 (python):
```python
sns.scatterplot(data=diff_res, x='logfoldchanges',y='mean_gene_exp')
```

Example 22 (unknown):
```unknown
<Axes: xlabel='logfoldchanges', ylabel='mean_gene_exp'>
```

Example 23 (python):
```python
sns.scatterplot(data=diff_res, x='scores',y='mean_gene_exp')
```

Example 24 (unknown):
```unknown
<Axes: xlabel='scores', ylabel='mean_gene_exp'>
```

Example 25 (python):
```python
groups = ['outer','middle','inner']
logfc_thresh = 0.1
mean_exp_thresh = 0.05

gene_lists = {}
for grp in groups:
    gene_lists[f'{grp}_up'] = np.nan
    gene_lists[f'{grp}_down'] = np.nan
```

Example 26 (unknown):
```unknown
display(diff_res[diff_res['names']=='NRG1'])
```

Example 27 (unknown):
```unknown
['outer', 'middle', 'inner']
```

Example 28 (python):
```python
for grp in groups:
    # up genes
    genes = diff_res[(diff_res['group']==grp)&\
                (diff_res['logfoldchanges'] > logfc_thresh)&\
                (diff_res['mean_gene_exp'] > mean_exp_thresh)
               ]['names']
    gene_lists[f'{grp}_up'] = list(genes)
    print(f"{grp}_up: {len(gene_lists[f'{grp}_up'])}")

    # down genes
    genes = diff_res[(diff_res['group']==grp)&\
                (diff_res['logfoldchanges'] < -logfc_thresh)
                # (res['mean_gene_exp'] > mean_exp_thresh)
               ]['names']
    gene_lists[f'{grp}_down'] = list(genes)
    print(f"{grp}_down: {len(gene_lists[f'{grp}_down'])}")
```

Example 29 (unknown):
```unknown
outer_up: 837
outer_down: 2790
middle_up: 151
middle_down: 373
inner_up: 2447
inner_down: 1187
```

Example 30 (unknown):
```unknown
gene_lists.keys()
```

Example 31 (unknown):
```unknown
dict_keys(['outer_up', 'outer_down', 'middle_up', 'middle_down', 'inner_up', 'inner_down'])
```

Example 32 (python):
```python
import gseapy as gp
pd.set_option('display.max_rows', None)

geneset_list = ['MSigDB_Hallmark_2020','KEGG_2021_Human','GO_Biological_Process_2023']
key_to_sort = 'n_genes'
n_rank = 10

enr_res = {}
for key in gene_lists.keys():
    if ('up' in key) & (len(gene_lists[key])>0):
        print(f'##### {key} #####')

        enr_res[key] = gp.enrichr(gene_list=gene_lists[key],
                         gene_sets=geneset_list,
                         organism='human', # don't forget to set organism to the one you desired! e.g. Yeast
                         outdir=None, # don't write to disk
                        )

        for geneset in geneset_list:
            print(geneset)
            df = enr_res[key].results[enr_res[key].results['Gene_set'] == geneset]
            df = df[df['Adjusted P-value'] < 0.01]
            df['n_genes'] = [int(x.split('/')[0]) for x in df['Overlap']]
            df = df.sort_values(key_to_sort,ascending=False)
            # df = df.sort_values('Combined Score',ascending=False)
            display(df)

            # plot
            df = df[df['Adjusted P-value'] < 0.01]
            plt.rcParams['axes.grid'] = False
            plt.rcParams['figure.figsize'] = 4,3
            plt.barh(width=df[:n_rank][key_to_sort],
                     y=[x.split(' (')[0] for x in df[:n_rank]['Term']],
                     color='darkred')
            plt.gca().invert_yaxis()
            plt.xlabel(key_to_sort)
            plt.title(f'{key}, {geneset}')
            plt.margins(y=0.03)
            plt.savefig(f'{figdir}/tm-axis_pathways_{key}_{geneset}.pdf', bbox_inches="tight",dpi=300)
            plt.show()

        print('')
```

Example 33 (unknown):
```unknown
##### outer_up #####
MSigDB_Hallmark_2020
```

Example 34 (unknown):
```unknown
KEGG_2021_Human
```

Example 35 (unknown):
```unknown
GO_Biological_Process_2023
```

Example 36 (unknown):
```unknown
##### middle_up #####
MSigDB_Hallmark_2020
```

Example 37 (unknown):
```unknown
KEGG_2021_Human
```

Example 38 (unknown):
```unknown
GO_Biological_Process_2023
```

Example 39 (unknown):
```unknown
##### inner_up #####
MSigDB_Hallmark_2020
```

Example 40 (unknown):
```unknown
KEGG_2021_Human
```

Example 41 (unknown):
```unknown
GO_Biological_Process_2023
```

Example 42 (python):
```python
key = 'inner_up'
hight = 10

tissueaxis.plot_axis(adata_oct,
              axis_col='transmural_axis',
              n_bin=15,
              feature_type='gene', # 'gene' or 'celltype'
              features_to_plot=gene_lists[key],
              features_to_remove=None,
              threshold_express_prop=None, # at quantile=0.9
              threshold_cell_abundance=None, # at quantile=0.9
              correlation=False,
              min_max_scale=True,
              row_cluster=True,
              figsize=(4,hight),
              cbar_pos=(-0.2, 0.5, 0.05, 0.18),
              cmap='viridis',
          xticklabels=False,
          yticklabels=False,
              title=None,
              return_feature = False,
                     save=f'{figdir}/tm-axis_OCT_genes_{key}.pdf',
    save_dpi=300,
             )
```

Example 43 (python):
```python
key = 'outer_up'
hight = 10 * len(gene_lists['outer_up'])/len(gene_lists['inner_up'])

tissueaxis.plot_axis(adata_oct,
              axis_col='transmural_axis',
              n_bin=15,
              feature_type='gene', # 'gene' or 'celltype'
              features_to_plot=gene_lists[key],
              features_to_remove=None,
              threshold_express_prop=None, # at quantile=0.9
              threshold_cell_abundance=None, # at quantile=0.9
              correlation=False,
              min_max_scale=True,
              row_cluster=True,
              figsize=(4,hight),
              cbar_pos=(-0.2, 0.5, 0.05, 0.18),
              cmap='viridis',
          xticklabels=False,
          yticklabels=False,
              title=None,
              return_feature = False,
                     save=f'{figdir}/tm-axis_OCT_genes_{key}.pdf',
    save_dpi=300,
             )
```

Example 44 (python):
```python
tissueaxis.plot_axis(adata_oct,
              axis_col='transmural_axis',
              n_bin=15,
              feature_type='gene', # 'gene' or 'celltype'
              features_to_plot=gene_lists[f'inner_up']+gene_lists[f'outer_up'],
              features_to_remove=None,
              threshold_express_prop=None, # at quantile=0.9
              threshold_cell_abundance=None, # at quantile=0.9
              correlation=False,
              min_max_scale=True,
              row_cluster=True,
              figsize=(4,10),
              cbar_pos=(-0.2, 0.5, 0.05, 0.18),
              cmap='viridis',
          xticklabels=False,
          yticklabels=False,
              title=None,
              return_feature = False,
                     save=f'{figdir}/tm-axis_OCT_genes.pdf',
    save_dpi=300,
             )
```

---

## Variables¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/Xenium/no2_integration_and_cell-type-annotation/no6_manual-annotation_EndocardialCells.html

**Contents:**
- Variables¶
- Read in data¶
- Markers¶
- 1st round annotation¶
- Subset and reclustering¶
- Transfer annotation to the original adata¶
- Save¶

--> cluster 0 has lower total counts and smaller cell area, and no specific gene expression --> annotate as "unassigned"

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os

import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 

import scanpy.external as sce
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/session_info/main.py:213: UserWarning: The '__version__' attribute is deprecated and will be removed in MarkupSafe 3.1. Use feature detection, or `importlib.metadata.version("markupsafe")`, instead.
  mod_version = _find_version(mod.__version__)
```

Example 4 (python):
```python
-----
matplotlib          3.10.1
numpy               2.1.3
pandas              2.2.3
scanpy              1.11.1
seaborn             0.13.2
session_info        v1.0.1
-----
```

Example 5 (python):
```python
Cython              3.1.1
PIL                 11.1.0
anndata             0.11.4
asciitree           NA
asttokens           NA
charset_normalizer  3.4.1
cloudpickle         3.1.1
comm                0.2.2
cycler              0.12.1
cython              3.1.1
cython_runtime      NA
dask                2024.11.2
dateutil            2.9.0.post0
debugpy             1.8.13
decorator           5.2.1
exceptiongroup      1.2.2
executing           2.2.0
h5py                3.13.0
igraph              0.11.8
importlib_metadata  NA
ipykernel           6.29.5
jedi                0.19.2
jinja2              3.1.6
joblib              1.4.2
kiwisolver          1.4.8
legacy_api_wrap     NA
leidenalg           0.10.2
llvmlite            0.44.0
markupsafe          3.0.2
mpl_toolkits        NA
msgpack             1.1.0
natsort             8.4.0
numba               0.61.0
numcodecs           0.13.1
packaging           24.2
parso               0.8.4
patsy               1.0.1
platformdirs        4.3.7
prompt_toolkit      3.0.50
psutil              7.0.0
pure_eval           0.2.3
pyarrow             19.0.1
pydev_ipython       NA
pydevconsole        NA
pydevd              3.2.3
pydevd_file_utils   NA
pydevd_plugins      NA
pydevd_tracing      NA
pygments            2.19.1
pyparsing           3.2.3
pytz                2025.2
scipy               1.15.2
six                 1.17.0
sklearn             1.5.2
stack_data          0.6.3
statsmodels         0.14.4
tblib               3.1.0
texttable           1.7.0
threadpoolctl       3.6.0
tlz                 1.0.0
toolz               1.0.0
tornado             6.4.2
traitlets           5.14.3
typing_extensions   NA
wcwidth             0.2.13
yaml                6.0.2
zarr                2.18.3
zipp                NA
zmq                 26.3.0
zoneinfo            NA
```

Example 6 (unknown):
```unknown
-----
IPython             8.34.0
jupyter_client      8.6.3
jupyter_core        5.7.2
-----
Python 3.10.14 | packaged by conda-forge | (main, Mar 20 2024, 12:45:18) [GCC 12.3.0]
Linux-4.18.0-553.53.1.el8_10.x86_64-x86_64-with-glibc2.28
-----
Session information updated at 2025-06-09 22:25
```

Example 7 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/home/kk837/rfs/rfs-teichlab-nfs-iCNyzSAaucw/kk837/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 8 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 9 (python):
```python
adata_dir = '/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium/subsets'
sample_id = 'C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat'
celltype = 'EndocardialCells'
```

Example 10 (unknown):
```unknown
# figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk/Aug2024'
# table_dir = '/nfs/team205/heart/fetal_ms_figures/tables/Aug2024'
```

Example 11 (unknown):
```unknown
latent_space = 'pca_harmony'
```

Example 12 (python):
```python
# read in
adata = sc.read_h5ad(f'{adata_dir}/{sample_id}_5K_{celltype}_lognorm.h5ad')
adata
```

Example 13 (unknown):
```unknown
AnnData object with n_obs × n_vars = 5202 × 4492
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_3.0', 'multi_celltypes_coarse', 'leiden_0.2', 'leiden_0.4', 'leiden_0.6', 'leiden_1'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1', 'n_cells', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'dendrogram_pca_harmony_leiden_1.0', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1_colors', 'log1p', 'multi_celltypes_coarse_colors', 'pca', 'pca_harmony', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.2_colors', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.3_colors', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.4_colors', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.5_colors', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_0.8_colors', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.0_colors', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_1.5_colors', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_2.0_colors', 'pca_harmony_leiden_3.0', 'pca_harmony_leiden_3.0_colors', 'tissue_block_id_colors', 'umap'
    obsm: 'X_pca', 'X_pca_harmony', 'X_umap_pca_harmony', 'spatial'
    varm: 'PCs'
    obsp: 'pca_harmony_connectivities', 'pca_harmony_distances'
```

Example 14 (python):
```python
adata.X.data[:5]
```

Example 15 (unknown):
```unknown
array([3.6159909, 2.569763 , 2.569763 , 3.2238812, 2.569763 ],
      dtype=float32)
```

Example 16 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=['total_counts','n_genes_by_counts','cell_area','tissue_block_id'],
                    wspace=0.2,cmap='RdPu',vmax='p100')
```

Example 17 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=['multi_celltypes_coarse',"conf_score_midmod2fine","celltypist_midmod2fine",],
                    wspace=0.2,cmap='RdPu',vmax='p100')
resolutions_list = [0.2,0.4,0.6,1]
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                    color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                    wspace=0.2)
```

Example 18 (python):
```python
%%time
# leiden clustering
resolutions_list = [# 0.2,0.4,0.6,1, # already done
                    1.2,1.5,2]
for resolution in resolutions_list:
    sc.tl.leiden(adata,neighbors_key=latent_space,
                     resolution=resolution,key_added=f'leiden_{str(resolution)}',n_iterations=2)
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                wspace=0.3)
```

Example 19 (unknown):
```unknown
<timed exec>:5: FutureWarning: In the future, the default backend for leiden will be igraph instead of leidenalg.

 To achieve the future defaults please pass: flavor="igraph" and n_iterations=2.  directed must also be False to work with igraph's implementation.
```

Example 20 (unknown):
```unknown
CPU times: user 3.72 s, sys: 86.5 ms, total: 3.81 s
Wall time: 3.82 s
```

Example 21 (python):
```python
# read in markers identified using single-nucleus data
markers_df = pd.read_csv(f'/home/kk837/rfs/rfs-teichlab-nfs-iCNyzSAaucw/kk837/notebooks/Foetal/VisiumHD/re-seq_mpp0p3/sc_{celltype}_finegrain_markers.csv')

mask1 = markers_df['group_mean']>1
mask2 = markers_df['logfoldchanges']>1
markers_df = markers_df[mask1&mask2]

markers_df
```

Example 22 (python):
```python
markers = {}
n_top = 20
for c in set(markers_df['group']):
    markers_df_sub = markers_df[markers_df['group']==c]
    markers_df_sub.sort_values('logfoldchanges',ascending=False)
    markers[c]=list(markers_df_sub[:n_top]['names'])

# manually add genes
markers['Cycling'] = ['MKI67','TOP2A'] 

genes = []
for g in markers.values():
    genes = genes + [x for x in g if (x in list(adata.var_names))&(x not in genes)]
for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in list(adata.var_names)]
len(genes)
```

Example 23 (python):
```python
sc.pl.embedding(adata,basis=f"X_umap_{latent_space}",color=genes, size=20,
               ncols=6, wspace=0.2,cmap='RdPu',vmax='p100')
```

Example 24 (python):
```python
sc.pl.embedding(adata,basis=f"X_umap_{latent_space}",
                color=['celltypist_midmod2fine', 'conf_score_midmod2fine'],
                wspace=0.7)
```

Example 25 (python):
```python
# umap per celltype to highlight
group_col = 'celltypist_midmod2fine'
groups_to_plot = list(set(adata.obs[group_col]))

plt.rcParams['axes.grid'] = False
fig,axs=plt.subplots(nrows=2,ncols=4,figsize=(15,7))
# Flatten the axs array for easier iteration in case of multiple rows and columns
axs_flat = axs.flatten()

for i,grp in enumerate(groups_to_plot):
    sc.pl.embedding(adata,basis=f"X_umap_{latent_space}",
                    color=[group_col], groups=grp, 
               size=30, frameon=False,
               show=False, ax=axs_flat[i], legend_loc=None, legend_fontsize=5,
               title=grp
              )
# Turn off any remaining subplots
for j in range(len(groups_to_plot), len(axs_flat)):
    fig.delaxes(axs_flat[j])

# fig.suptitle('Main Title for the Figure')
fig.tight_layout()
```

Example 26 (unknown):
```unknown
resolution_sel = 0.6
```

Example 27 (python):
```python
markers_others = {
    'CM': ['TTN', 'TNNT2','MYH6',
        'MYH7', 'MYL2', 'FHL2', # vCN
            'NPPA', 'MYL7', 'MYL4'], #aCM
    'FB': ['DCN', 'GCN', 'PDGFRA','COL1A1','COL1A2'],
    'EC': ['VWF', 'PECAM1', 'CDH5','RGCC', 'FABP5',],
    'Peri': ['RGS5', 'ABCC9', 'KCNJ8'],
    'SMC': ['MYH11', 'TAGLN', 'ACTA2'],
    'Meso': ['MSLN', 'WT1', 'BNC1'],
    'Neuro': ['PLP1', 'NRXN1', 'NRXN3','PRPH', 'NEFL', 'NEFM', 'NEFH', 'STMN2', 'SST', 'SLC10A4', 'SLC18A3', 'TH', 'NPY'],
    'Adipo': ['GPAM', 'FASN', 'LEP'],
    'Myelo': ['CD14', 'C1QA', 'CD68','LYVE1','TIMD4'],
    'Lymph': ['CD8A', 'IL7R', 'CD40LG'],
    'B': ['CD19','SDC1','IGHG1','IGHA1'],
    'Mast': ['KIT', 'CPA3'],
}

for key in markers_others.keys():
    markers_others[key]=[x for x in markers_others[key] if x in adata.var_names]

sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
sc.pl.dotplot(adata,
              markers_others, 
              groupby=f'leiden_{str(resolution_sel)}',
              dendrogram=True,
              standard_scale="var",
              color_map="Reds",
              swap_axes=False,
             )
```

Example 28 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: CM, FB, EC, etc.
```

Example 29 (python):
```python
sc.tl.dendrogram(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.pl.dotplot(adata,
              var_names=markers,
              groupby=f'leiden_{str(resolution_sel)}',
              standard_scale='var',
              dendrogram=True
             )
```

Example 30 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: ValveEndothelialCells, EndocardialCushionCells, EndocardialCells, etc.
```

Example 31 (python):
```python
print(adata.X.data[:5])
adata.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata, dendrogram=True, standard_scale='var', return_fig=True)
dp.add_totals().show()
```

Example 32 (unknown):
```unknown
[3.6159909 2.569763  2.569763  3.2238812 2.569763 ]
```

Example 33 (python):
```python
sc.pl.violin(adata,keys=['total_counts','cell_area'],groupby=f'leiden_{str(resolution_sel)}')
```

Example 34 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=f'leiden_{str(resolution_sel)}',legend_loc='on data',
                )
```

Example 35 (unknown):
```unknown
'EndocardialCells'
```

Example 36 (python):
```python
# annotate
adata.obs['tmp_annotation'] = adata.obs[f'leiden_{str(resolution_sel)}'].copy()
adata.obs.replace({'tmp_annotation':{
    '0':'unassigned',
    '1':'EndocardialCells',
    '2':'EndocardialCells',
    '3':'EndocardialCells',
    '4':'EndocardialCells',
    '5':'Erythrocytes',
}},inplace=True)
adata.obs['tmp_annotation'].value_counts()
```

Example 37 (python):
```python
/tmp/ipykernel_69127/744100964.py:3: FutureWarning: The behavior of Series.replace (and DataFrame.replace) with CategoricalDtype is deprecated. In a future version, replace will only be used for cases that preserve the categories. To change the categories, use ser.cat.rename_categories instead.
  adata.obs.replace({'tmp_annotation':{
```

Example 38 (unknown):
```unknown
tmp_annotation
EndocardialCells    3569
unassigned          1614
Erythrocytes          19
Name: count, dtype: int64
```

Example 39 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color='tmp_annotation'
                )
```

Example 40 (python):
```python
adata_sub = adata[adata.obs['tmp_annotation'].isin(['EndocardialCells'])]
```

Example 41 (python):
```python
# drop leiden clustering results
resolutions_list = [0.2,0.4,0.6,1, 1.2,1.5,2]
for resolution in resolutions_list:
    adata_sub.obs.drop(f'leiden_{str(resolution)}',axis=1,inplace=True)
adata_sub
```

Example 42 (unknown):
```unknown
AnnData object with n_obs × n_vars = 3569 × 4492
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_3.0', 'multi_celltypes_coarse', 'tmp_annotation'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1', 'n_cells', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'dendrogram_pca_harmony_leiden_1.0', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1_colors', 'log1p', 'multi_celltypes_coarse_colors', 'pca', 'pca_harmony', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.2_colors', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.3_colors', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.4_colors', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.5_colors', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_0.8_colors', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.0_colors', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_1.5_colors', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_2.0_colors', 'pca_harmony_leiden_3.0', 'pca_harmony_leiden_3.0_colors', 'tissue_block_id_colors', 'umap', 'leiden_1.2', 'leiden_1.5', 'leiden_2', 'leiden_1.2_colors', 'leiden_1.5_colors', 'leiden_2_colors', 'dendrogram_leiden_0.6', 'rank_genes_groups', 'tmp_annotation_colors'
    obsm: 'X_pca', 'X_pca_harmony', 'X_umap_pca_harmony', 'spatial'
    varm: 'PCs'
    obsp: 'pca_harmony_connectivities', 'pca_harmony_distances'
```

Example 43 (python):
```python
## Remove mid-grain markers from the feature
# read in markers
midgrain_markers = pd.read_csv('/home/kk837/rfs/rfs-teichlab-nfs-iCNyzSAaucw/kk837/notebooks/Foetal/VisiumHD/re-seq_mpp0p3/sc_midgrain_markers.csv',
                               index_col=0)
# remove the markers of celltype of interest
midgrain_markers = midgrain_markers[midgrain_markers['group']!=celltype]
# remove
mask = [x not in list(midgrain_markers['names']) for x in adata_sub.var_names]
bdata = adata_sub[:,mask]
bdata
```

Example 44 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 3569 × 4027
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_3.0', 'multi_celltypes_coarse', 'tmp_annotation'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1', 'n_cells', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'dendrogram_pca_harmony_leiden_1.0', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1_colors', 'log1p', 'multi_celltypes_coarse_colors', 'pca', 'pca_harmony', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.2_colors', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.3_colors', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.4_colors', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.5_colors', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_0.8_colors', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.0_colors', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_1.5_colors', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_2.0_colors', 'pca_harmony_leiden_3.0', 'pca_harmony_leiden_3.0_colors', 'tissue_block_id_colors', 'umap', 'leiden_1.2', 'leiden_1.5', 'leiden_2', 'leiden_1.2_colors', 'leiden_1.5_colors', 'leiden_2_colors', 'dendrogram_leiden_0.6', 'rank_genes_groups', 'tmp_annotation_colors'
    obsm: 'X_pca', 'X_pca_harmony', 'X_umap_pca_harmony', 'spatial'
    varm: 'PCs'
    obsp: 'pca_harmony_connectivities', 'pca_harmony_distances'
```

Example 45 (python):
```python
'''
# remove cell cycle genes
cell_cycle_genes = ['MCM5','PCNA','TYMS','FEN1','MCM2','MCM4','RRM1','UNG','GINS2','MCM6','CDCA7','DTL','PRIM1','UHRF1','MLF1IP','HELLS','RFC2','RPA2','NASP',
                'RAD51AP1','GMNN','WDR76','SLBP','CCNE2','UBR7','POLD3','MSH2','ATAD2','RAD51','RRM2','CDC45','CDC6','EXO1','TIPIN','DSCC1','BLM','CASP8AP2',
                'USP1','CLSPN','POLA1','CHAF1B','BRIP1','E2F8','HMGB2','CDK1','NUSAP1','UBE2C','BIRC5','TPX2','TOP2A','NDC80','CKS2','NUF2','CKS1B','MKI67',
                'TMPO','CENPF','TACC3','FAM64A','SMC4','CCNB2','CKAP2L','CKAP2','AURKB','BUB1','KIF11','ANP32E','TUBB4B','GTSE1','KIF20B','HJURP','CDCA3','HN1',
                'CDC20','TTK','CDC25C','KIF2C','RANGAP1','NCAPD2','DLGAP5','CDCA2','CDCA8','ECT2','KIF23','HMMR','AURKA','PSRC1','ANLN','LBR','CKAP5','CENPE','CTCF',
                'NEK2','G2E3','GAS2L3','CBX5','CENPA']
mask = [x not in cell_cycle_genes for x in bdata.var_names]
bdata = bdata[:,mask]
bdata.shape
'''
```

Example 46 (python):
```python
# HVGs, using the object which doesn't have midgrain markers
sc.pp.highly_variable_genes(bdata, flavor='seurat', min_mean=0.0125, max_mean=3, min_disp=0.5)
print(f'number of hvgs: {sum(bdata.var.highly_variable)}')
# PCA using the degected HVGs
sc.tl.pca(bdata, use_highly_variable=True)
adata_sub.obsm['X_pca'] = bdata.obsm['X_pca'].copy()
# harmony integration
sce.pp.harmony_integrate(adata_sub, 'tissue_block_id', basis='X_pca', adjusted_basis='X_pca_harmony')
# create neighbour graph
latent_space = 'pca_harmony'
sc.pp.neighbors(adata_sub,n_neighbors=15,use_rep=f"X_{latent_space}",key_added=latent_space)
# visualisation with umap
sc.tl.umap(adata_sub,min_dist=0.5,neighbors_key=latent_space)
adata_sub.obsm[f"X_umap_{latent_space}"]=adata_sub.obsm["X_umap"]
del adata_sub.obsm["X_umap"]
```

Example 47 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_highly_variable_genes.py:698: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  adata.uns["hvg"] = {"flavor": flavor}
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_pca/__init__.py:438: FutureWarning: Argument `use_highly_variable` is deprecated, consider using the mask argument. Use_highly_variable=True can be called through mask_var="highly_variable". Use_highly_variable=False can be called through mask_var=None
  warn(msg, FutureWarning)
```

Example 48 (unknown):
```unknown
number of hvgs: 733
```

Example 49 (python):
```python
2025-06-09 22:29:47,867 - harmonypy - INFO - Computing initial centroids with sklearn.KMeans...
2025-06-09 22:29:49,453 - harmonypy - INFO - sklearn.KMeans initialization complete.
2025-06-09 22:29:49,467 - harmonypy - INFO - Iteration 1 of 10
2025-06-09 22:29:49,847 - harmonypy - INFO - Iteration 2 of 10
2025-06-09 22:29:50,229 - harmonypy - INFO - Converged after 2 iterations
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm
```

Example 50 (python):
```python
%%time
# reclustering
# leiden clustering
resolutions_list = [0.2,0.4,0.6,1, 1.2,1.5,2]
for resolution in resolutions_list:
    sc.tl.leiden(adata_sub,neighbors_key=latent_space,
                 resolution=resolution,key_added=f'leiden_{str(resolution)}',n_iterations=2)
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                wspace=0.3)
```

Example 51 (unknown):
```unknown
CPU times: user 5.2 s, sys: 110 ms, total: 5.31 s
Wall time: 5.33 s
```

Example 52 (python):
```python
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",size=10,
                color=genes,
               ncols=6, wspace=0.2,cmap='RdPu',vmax='p100')
```

Example 53 (python):
```python
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color=["conf_score_midmod2fine","celltypist_midmod2fine",],
                    wspace=0.2,cmap='RdPu',vmax='p100')
```

Example 54 (python):
```python
# umap per celltype to highlight
group_col = 'celltypist_midmod2fine'
groups_to_plot = list(set(adata_sub.obs[group_col]))

plt.rcParams['axes.grid'] = False
fig,axs=plt.subplots(nrows=2,ncols=4,figsize=(15,8))
# Flatten the axs array for easier iteration in case of multiple rows and columns
axs_flat = axs.flatten()

for i,grp in enumerate(groups_to_plot):
    sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                    color=[group_col], groups=grp, 
               size=20, frameon=False,
               show=False, ax=axs_flat[i], legend_loc=None, legend_fontsize=5,
               title=grp
              )
# Turn off any remaining subplots
for j in range(len(groups_to_plot), len(axs_flat)):
    fig.delaxes(axs_flat[j])

# fig.suptitle('Main Title for the Figure')
fig.tight_layout()
```

Example 55 (unknown):
```unknown
sub_resolution_sel = 1.5
```

Example 56 (python):
```python
markers_others = {
    'CM': ['TTN', 'TNNT2','MYH6',
        'MYH7', 'MYL2', 'FHL2', # vCN
            'NPPA', 'MYL7', 'MYL4'], #aCM
    'FB': ['DCN', 'GCN', 'PDGFRA','COL1A1','COL1A2'],
    'EC': ['VWF', 'PECAM1', 'CDH5','RGCC', 'FABP5',],
    'Peri': ['RGS5', 'ABCC9', 'KCNJ8'],
    'SMC': ['MYH11', 'TAGLN', 'ACTA2'],
    'Meso': ['MSLN', 'WT1', 'BNC1'],
    'Neuro': ['PLP1', 'NRXN1', 'NRXN3','PRPH', 'NEFL', 'NEFM', 'NEFH', 'STMN2', 'SST', 'SLC10A4', 'SLC18A3', 'TH', 'NPY'],
    'Adipo': ['GPAM', 'FASN', 'LEP'],
    'Myelo': ['CD14', 'C1QA', 'CD68','LYVE1','TIMD4'],
    'Lymph': ['CD8A', 'IL7R', 'CD40LG'],
    'B': ['CD19','SDC1','IGHG1','IGHA1'],
    'Mast': ['KIT', 'CPA3'],
}

for key in markers_others.keys():
    markers_others[key]=[x for x in markers_others[key] if x in adata.var_names]

sc.tl.dendrogram(adata_sub, groupby=f'leiden_{str(sub_resolution_sel)}')
sc.pl.dotplot(adata_sub,
              markers_others, 
              groupby=f'leiden_{str(sub_resolution_sel)}',
              dendrogram=True,
              standard_scale="var",
              color_map="Reds",
              swap_axes=False,
             )
```

Example 57 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: CM, FB, EC, etc.
```

Example 58 (python):
```python
sc.tl.dendrogram(adata_sub,groupby=f'leiden_{str(sub_resolution_sel)}')
sc.pl.dotplot(adata_sub,
              var_names=markers,
              groupby=f'leiden_{str(sub_resolution_sel)}',
              standard_scale='var',
              dendrogram=True
             )
```

Example 59 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: ValveEndothelialCells, EndocardialCushionCells, EndocardialCells, etc.
```

Example 60 (python):
```python
# DEG analysis
print(adata_sub.X.data[:5])
adata_sub.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata_sub,groupby=f'leiden_{str(sub_resolution_sel)}')
sc.tl.dendrogram(adata_sub, groupby=f'leiden_{str(sub_resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata_sub, dendrogram=True, standard_scale='var', return_fig=True)
dp.add_totals().show()
```

Example 61 (unknown):
```unknown
[3.6159909 2.569763  2.569763  3.2238812 2.569763 ]
```

Example 62 (python):
```python
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color=f'leiden_{str(sub_resolution_sel)}',legend_loc='on data',
                wspace=0.3)
```

Example 63 (python):
```python
# annotate
adata_sub.obs['fine_grain_manual'] = adata_sub.obs[f'leiden_{str(sub_resolution_sel)}'].copy()
adata_sub.obs.replace({'fine_grain_manual':{
    '0':'EndocardialCells_Doublets',
    '1':'ValveEndothelialCells',
    '2':'EndocardialCellsCycling',
    '3':'EndocardialCells',
    '4':'EndocardialCells',
    '5':'EndocardialCells',
    '6':'EndocardialCushionCells',
    '7':'EndocardialCushionCells',
    '8':'EndocardialCells',
    '9':'ValveEndothelialCells',
    '10':'EndocardialCells',
    '11':'EndocardialCells',
    '12':'EndocardialCells',
    '13':'EndocardialCells',
    '14':'ValveEndothelialCells',
    '15':'EndocardialCells',
}},inplace=True)
adata_sub.obs['fine_grain_manual'].value_counts()
```

Example 64 (python):
```python
/tmp/ipykernel_69127/3524787546.py:3: FutureWarning: The behavior of Series.replace (and DataFrame.replace) with CategoricalDtype is deprecated. In a future version, replace will only be used for cases that preserve the categories. To change the categories, use ser.cat.rename_categories instead.
  adata_sub.obs.replace({'fine_grain_manual':{
```

Example 65 (unknown):
```unknown
fine_grain_manual
EndocardialCells             1670
ValveEndothelialCells         635
EndocardialCells_Doublets     474
EndocardialCushionCells       432
EndocardialCellsCycling       358
Name: count, dtype: int64
```

Example 66 (python):
```python
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color='fine_grain_manual',
                wspace=0.3)
```

Example 67 (python):
```python
sc.tl.dendrogram(adata_sub,groupby='fine_grain_manual')
sc.pl.dotplot(adata_sub,
              var_names=markers,
              groupby='fine_grain_manual',
              standard_scale='var',
              dendrogram=True
             )
```

Example 68 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: EndocardialCells_Doublets, ValveEndothelialCells, EndocardialCellsCycling, etc.
var_group_labels: ValveEndothelialCells, EndocardialCushionCells, EndocardialCells, etc.
```

Example 69 (python):
```python
adata.obs['fine_grain_manual'] = adata.obs['tmp_annotation'].astype('str').copy()
adata.obs.loc[adata_sub.obs_names,'fine_grain_manual'] = adata_sub.obs['fine_grain_manual']
```

Example 70 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color='fine_grain_manual'
                )
```

Example 71 (python):
```python
sc.tl.dendrogram(adata,groupby='fine_grain_manual')
sc.pl.dotplot(adata,
              var_names=markers,
              groupby='fine_grain_manual',
              standard_scale='var',
              dendrogram=True
             )
```

Example 72 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: EndocardialCells, EndocardialCellsCycling, EndocardialCells_Doublets, etc.
var_group_labels: ValveEndothelialCells, EndocardialCushionCells, EndocardialCells, etc.
```

Example 73 (python):
```python
for ct in set(adata.obs['fine_grain_manual']):
    for tissue_id in ['C194-HEA-0-FFPE-1', 'Hst45-HEA-0-FFPE-1']:
        slide = adata[adata.obs['tissue_block_id']==tissue_id]
        sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
                      spot_size=60,vmax='p100')
```

Example 74 (python):
```python
/tmp/ipykernel_69127/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 75 (python):
```python
/tmp/ipykernel_69127/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 76 (python):
```python
/tmp/ipykernel_69127/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 77 (python):
```python
/tmp/ipykernel_69127/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 78 (python):
```python
/tmp/ipykernel_69127/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 79 (python):
```python
/tmp/ipykernel_69127/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 80 (python):
```python
/tmp/ipykernel_69127/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 81 (python):
```python
/tmp/ipykernel_69127/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 82 (python):
```python
/tmp/ipykernel_69127/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 83 (python):
```python
/tmp/ipykernel_69127/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 84 (python):
```python
/tmp/ipykernel_69127/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 85 (python):
```python
/tmp/ipykernel_69127/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 86 (python):
```python
/tmp/ipykernel_69127/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 87 (python):
```python
/tmp/ipykernel_69127/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 88 (python):
```python
# save
adata.write(f'{adata_dir}/{sample_id}_5K_{celltype}_lognorm.h5ad')
print(f'{adata_dir}/{sample_id}_5K_{celltype}_lognorm.h5ad')
```

Example 89 (unknown):
```unknown
/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium/subsets/C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat_5K_EndocardialCells_lognorm.h5ad
```

---

## Variables¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/VisiumHD/no7_manual-annotation_myeloid.html

**Contents:**
- Variables¶
- Read in data¶
- Remove mid-grain markers from the feature¶
- PCA, Clustering, UMAP¶
- Subset and reclustering¶
- Transfer annotation to the original adata¶
- Save¶

--> cluster 3 and 8 are MacrophagesCX3CR1pos (cluster 4 is likely to be monocytes)

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os

import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 

import celltypist
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
-----
celltypist          1.6.3
matplotlib          3.9.2
numpy               2.0.2
pandas              2.2.3
scanpy              1.10.3
seaborn             0.13.2
session_info        1.0.0
-----
```

Example 4 (python):
```python
PIL                 11.0.0
anndata             0.10.9
asttokens           NA
certifi             2024.08.30
charset_normalizer  3.4.0
comm                0.2.2
cycler              0.12.1
cython_runtime      NA
dateutil            2.9.0.post0
debugpy             1.8.9
decorator           5.1.1
exceptiongroup      1.2.2
executing           2.1.0
google              NA
h5py                3.12.1
idna                3.10
igraph              0.10.2
importlib_metadata  NA
importlib_resources NA
ipykernel           6.29.5
jedi                0.19.2
joblib              1.4.2
kiwisolver          1.4.7
legacy_api_wrap     NA
leidenalg           0.9.0
llvmlite            0.43.0
mpl_toolkits        NA
natsort             8.4.0
numba               0.60.0
packaging           24.2
parso               0.8.4
patsy               1.0.1
pexpect             4.9.0
platformdirs        4.3.6
prompt_toolkit      3.0.48
psutil              6.1.0
ptyprocess          0.7.0
pure_eval           0.2.3
pydev_ipython       NA
pydevconsole        NA
pydevd              3.2.3
pydevd_file_utils   NA
pydevd_plugins      NA
pydevd_tracing      NA
pygments            2.18.0
pyparsing           3.2.0
pytz                2024.2
requests            2.32.3
scipy               1.13.1
six                 1.16.0
sklearn             1.5.2
stack_data          0.6.3
statsmodels         0.14.4
texttable           1.6.7
threadpoolctl       3.5.0
tornado             6.4.2
traitlets           5.14.3
typing_extensions   NA
urllib3             2.2.3
wcwidth             0.2.13
yaml                6.0.2
zipp                NA
zmq                 26.2.0
zoneinfo            NA
```

Example 5 (unknown):
```unknown
-----
IPython             8.18.1
jupyter_client      8.6.3
jupyter_core        5.7.2
-----
Python 3.9.20 | packaged by conda-forge | (main, Sep 30 2024, 17:49:10) [GCC 13.3.0]
Linux-5.15.0-126-generic-x86_64-with-glibc2.35
-----
Session information updated at 2024-12-08 11:26
```

Example 6 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 7 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 8 (python):
```python
adata_dir = '/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/subsets'
sample_id = 'HEA_FOET14880396'
celltype = 'MyeloidCells'
```

Example 9 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk/Aug2024'
table_dir = '/nfs/team205/heart/fetal_ms_figures/tables/Aug2024'
```

Example 10 (python):
```python
# read in
adata = sc.read_h5ad(f'{adata_dir}/{sample_id}_{celltype}_b2c_cells_filtered_lognorm.h5ad')
adata
```

Example 11 (unknown):
```unknown
AnnData object with n_obs × n_vars = 526 × 13502
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'leiden_0.2', 'leiden_0.4', 'leiden_0.6', 'leiden_1', 'leiden_1.2', 'leiden_1.5', 'leiden_2', 'fine_grain_manual', 'leiden_0.5'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'dendrogram_fine_grain_manual', 'dendrogram_leiden_0.4', 'dendrogram_leiden_0.6', 'dendrogram_leiden_1', 'dendrogram_leiden_1.2', 'dendrogram_leiden_1.5', 'fine_grain_manual_colors', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.5', 'leiden_0.5_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1.2', 'leiden_1.2_colors', 'leiden_1.5', 'leiden_1.5_colors', 'leiden_1_colors', 'leiden_2', 'leiden_2_colors', 'log1p', 'neighbors', 'pca', 'rank_genes_groups', 'spatial', 'umap'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 12 (python):
```python
# read in
midgrain_markers = pd.read_csv('/nfs/team205/kk18/notebooks/Foetal/VisiumHD/re-seq_mpp0p3/sc_midgrain_markers.csv',
                               index_col=0
                              )
# remove the markers of celltype of interest
midgrain_markers = midgrain_markers[midgrain_markers['group']!=celltype]
midgrain_markers.head()
```

Example 13 (unknown):
```unknown
'GATA2' in list(midgrain_markers['names'])
```

Example 14 (python):
```python
# remove
mask = [x not in list(midgrain_markers['names']) for x in adata.var_names]
bdata = adata[:,mask]
bdata
```

Example 15 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 526 × 12303
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'leiden_0.2', 'leiden_0.4', 'leiden_0.6', 'leiden_1', 'leiden_1.2', 'leiden_1.5', 'leiden_2', 'fine_grain_manual', 'leiden_0.5'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'dendrogram_fine_grain_manual', 'dendrogram_leiden_0.4', 'dendrogram_leiden_0.6', 'dendrogram_leiden_1', 'dendrogram_leiden_1.2', 'dendrogram_leiden_1.5', 'fine_grain_manual_colors', 'hvg', 'leiden_0.2', 'leiden_0.4', 'leiden_0.5', 'leiden_0.5_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1.2', 'leiden_1.2_colors', 'leiden_1.5', 'leiden_1.5_colors', 'leiden_1_colors', 'leiden_2', 'leiden_2_colors', 'log1p', 'neighbors', 'pca', 'rank_genes_groups', 'spatial', 'umap'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 16 (python):
```python
# remove cell cycle genes
cell_cycle_genes = ['MCM5','PCNA','TYMS','FEN1','MCM2','MCM4','RRM1','UNG','GINS2','MCM6','CDCA7','DTL','PRIM1','UHRF1','MLF1IP','HELLS','RFC2','RPA2','NASP',
                'RAD51AP1','GMNN','WDR76','SLBP','CCNE2','UBR7','POLD3','MSH2','ATAD2','RAD51','RRM2','CDC45','CDC6','EXO1','TIPIN','DSCC1','BLM','CASP8AP2',
                'USP1','CLSPN','POLA1','CHAF1B','BRIP1','E2F8','HMGB2','CDK1','NUSAP1','UBE2C','BIRC5','TPX2','TOP2A','NDC80','CKS2','NUF2','CKS1B','MKI67',
                'TMPO','CENPF','TACC3','FAM64A','SMC4','CCNB2','CKAP2L','CKAP2','AURKB','BUB1','KIF11','ANP32E','TUBB4B','GTSE1','KIF20B','HJURP','CDCA3','HN1',
                'CDC20','TTK','CDC25C','KIF2C','RANGAP1','NCAPD2','DLGAP5','CDCA2','CDCA8','ECT2','KIF23','HMMR','AURKA','PSRC1','ANLN','LBR','CKAP5','CENPE','CTCF',
                'NEK2','G2E3','GAS2L3','CBX5','CENPA']
mask = [x not in cell_cycle_genes for x in bdata.var_names]
bdata = bdata[:,mask]
bdata.shape
```

Example 17 (unknown):
```unknown
(526, 12209)
```

Example 18 (unknown):
```unknown
'MYH11' in list(bdata.var_names)
```

Example 19 (python):
```python
# HVGs, using the object which doesn't have midgrain markers
sc.pp.highly_variable_genes(bdata, flavor='seurat', min_mean=0.0125, max_mean=3, min_disp=0.5)
print(f'number of hvgs: {sum(bdata.var.highly_variable)}')
# PCA using the degected HVGs
sc.tl.pca(bdata, use_highly_variable=True)
adata.obsm['X_pca'] = bdata.obsm['X_pca'].copy()
# create neighbour graph
sc.pp.neighbors(adata,n_neighbors=15,use_rep='X_pca')
# visualisation with umap
sc.tl.umap(adata,min_dist=0.5)
sc.pl.embedding(adata,basis='X_umap',color=['bin_count','total_counts','n_genes_by_counts',"conf_score_midmod2fine","celltypist_midmod2fine",],
                wspace=0.2,cmap='RdPu',vmax='p100',ncols=5)
# leiden clustering
resolutions_list = [0.2,0.4,0.5,0.6,1,1.2,1.5,2]
for resolution in resolutions_list:
    sc.tl.leiden(adata,resolution=resolution,key_added=f'leiden_{str(resolution)}',n_iterations=2)
sc.pl.embedding(adata,basis='X_umap',
                color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                wspace=0.3)
```

Example 20 (unknown):
```unknown
number of hvgs: 2881
```

Example 21 (python):
```python
/software/cellgen/team205/kk18/envs/generic_env/lib/python3.9/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm
```

Example 22 (python):
```python
markers={
    'Mono':['CCR2','FCN1','VCAN','MPO'],
    'MP':['C1QA','C1QB','CD68','MRC1','CD163','MERTK'],
    'CX3CR1+':['CX3CR1','C3','TREM2','P2RY12','CD200R1'],
    'LYVE1+':['LYVE1','FOLR2'],
     'pDC':['IRF8','IRF4','IL3RA'],
     'DC_mature':['FLT3', 'CCR7','LAMP3', 'IL15', 'CD83', 'LY75', ],
    'Megakaryocyte':['ITGA2B','MPL','ITGB3',],
    'Mast': ['KIT', 'CPA3','HDC','GATA2'],
    'Cycling':['MKI67','TOP2A'],
}
genes = []
for g in markers.values():
    genes = genes + [x for x in g if (x in adata.var_names)&(x not in genes)]
sc.pl.embedding(adata,basis='X_umap',color=genes,
               ncols=5, wspace=0.2,cmap='RdPu',vmax='p99.7')
```

Example 23 (unknown):
```unknown
resolution_sel = 0.6
```

Example 24 (python):
```python
markers = {
    'CM': ['TTN', 'TNNT2','MYH6',
        'MYH7', 'MYL2', 'FHL2', # vCN
            'NPPA', 'MYL7', 'MYL4'], #aCM
    'FB': ['DCN', 'GCN', 'PDGFRA','COL1A1','COL1A2'],
    'EC': ['VWF', 'PECAM1', 'CDH5','RGCC', 'FABP5',],
    'Peri': ['RGS5', 'ABCC9', 'KCNJ8'],
    'SMC': ['MYH11', 'TAGLN', 'ACTA2'],
    'Meso': ['MSLN', 'WT1', 'BNC1'],
    'Neuro': ['PLP1', 'NRXN1', 'NRXN3','PRPH', 'NEFL', 'NEFM', 'NEFH', 'STMN2', 'SST', 'SLC10A4', 'SLC18A3', 'TH', 'NPY'],
    'Adipo': ['GPAM', 'FASN', 'LEP'],
    'Myelo': ['CD14', 'C1QA', 'CD68','LYVE1','TIMD4'],
    'Lymph': ['CD8A', 'IL7R', 'CD40LG'],
    'B': ['CD19','SDC1','IGHG1','IGHA1'],
    'Mast': ['KIT', 'CPA3'],
}

for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in adata.var_names]

sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
sc.pl.dotplot(adata,
              markers, 
              groupby=f'leiden_{str(resolution_sel)}',
              dendrogram=True,
              standard_scale="var",
              color_map="Reds",
              swap_axes=False,
             )
```

Example 25 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: CM, FB, EC, etc.
```

Example 26 (python):
```python
markers={
    'Mono':['CCR2','FCN1','VCAN','MPO'],
    'MP':['C1QA','C1QB','CD68','MRC1','CD163','MERTK'],
    'CX3CR1+':['CX3CR1','C3','TREM2','P2RY12','CD200R1'],
    'LYVE1+':['LYVE1','FOLR2'],
     'pDC':['IRF8','IRF4','IL3RA'],
     'DC_mature':['FLT3', 'CCR7','LAMP3', 'IL15', 'CD83', 'LY75', ],
    'Megakaryocyte':['ITGA2B','MPL','ITGB3',],
    'Mast': ['KIT', 'CPA3','HDC','GATA2'],
    'Cycling':['MKI67','TOP2A'],
}
genes = []
markers_sel = {}
for k,g in markers.items():
    markers_sel[k] = [x for x in g if x in adata.var_names]
    genes = genes + [x for x in g if (x in adata.var_names)&(x not in genes)]
    
sc.tl.dendrogram(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.pl.dotplot(adata,
              var_names=markers_sel,
              groupby=f'leiden_{str(resolution_sel)}',
              standard_scale='var',
              dendrogram=True
             )
```

Example 27 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: Mono, MP, CX3CR1+, etc.
```

Example 28 (python):
```python
# DEG analysis
print(adata.X.data[:5])
adata.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata, dendrogram=True, standard_scale='var', return_fig=True)
dp.add_totals().show()
```

Example 29 (unknown):
```unknown
[1.41297585 1.03895088 1.42284238 1.5805253  1.88093143]
```

Example 30 (python):
```python
# DEG analysis
print(adata.X.data[:5])
adata.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata, dendrogram=True, standard_scale='var', return_fig=True, n_genes=5)
dp.add_totals().show()
```

Example 31 (unknown):
```unknown
[1.41297585 1.03895088 1.42284238 1.5805253  1.88093143]
```

Example 32 (python):
```python
sc.pl.embedding(adata,basis='X_umap',
                color=f'leiden_{str(resolution_sel)}',
                wspace=0.3, legend_loc='on data')
```

Example 33 (python):
```python
sc.pl.embedding(adata,basis='X_umap',
                color=['celltypist_midmod2fine', 'conf_score_midmod2fine'],
                wspace=0.7)
```

Example 34 (python):
```python
sc.pl.violin(adata,keys=['total_counts','bin_count'],groupby=f'leiden_{str(resolution_sel)}')
```

Example 35 (unknown):
```unknown
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 36 (python):
```python
# annotate
adata.obs['fine_grain_manual'] = adata.obs[f'leiden_{str(resolution_sel)}'].copy()
adata.obs.replace({'fine_grain_manual':{
    '0':'Macrophages',
    '1':'Macrophages',
    '2':'Monocytes',
    '3':'Erythrocytes',
    '4':'Megakaryocytes',
}},inplace=True)
adata.obs['fine_grain_manual'].value_counts()
```

Example 37 (unknown):
```unknown
fine_grain_manual
Macrophages       440
Monocytes          36
Erythrocytes       26
Megakaryocytes     24
Name: count, dtype: int64
```

Example 38 (python):
```python
ad = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/Feb28ObjectRaw_finegrain_updated.h5ad',backed='r')
set(ad.obs['fine_grain'])
```

Example 39 (unknown):
```unknown
{'AtrialCardiomyocytesCycling',
 'AtrialCardiomyocytesLeft',
 'AtrialCardiomyocytesRight',
 'AtrioventricularNodePacemakerCells',
 'BCells',
 'BCellsMS4A1pos',
 'ChromaffinCells',
 'CoronaryArterialEndothelialCells',
 'CoronaryCapillaryEndothelialCells',
 'CoronaryPericytes',
 'CoronarySmoothMuscleCells',
 'CoronaryVenousEndothelialCells',
 'CoronaryVesselAdventitialFibroblasts',
 'DendriticCellsMature',
 'DendriticCellsType1',
 'DuctusArteriosusSmoothMuscleCells',
 'EndocardialCells',
 'EndocardialCushionCells',
 'EpicardiumDerivedCells',
 'GreatVesselAdventitialFibroblasts',
 'GreatVesselArterialEndothelialCells',
 'GreatVesselSmoothMuscleCells',
 'GreatVesselVenousEndothelialCells',
 'InnateLymphoidCells',
 'LymphNodeFibroblasticReticularCells',
 'LymphaticEndothelialCells',
 'MacrophagesATF3pos',
 'MacrophagesCX3CR1pos',
 'MacrophagesLYVE1pos',
 'MacrophagesTIMD4pos',
 'MastCells',
 'Megakaryocytes',
 'MesothelialEpicardialCells',
 'MonocyteDerivedCells',
 'Monocytes',
 'MonocytesMPOpos',
 'MyocardialInterstitialFibroblasts',
 'Myofibroblasts',
 'NaturalKillerCells',
 'NeuronPrecursors',
 'ParasympatheticNeurons',
 'PericardialCellsFibrous',
 'PericardialCellsIntermediate',
 'PericardialCellsParietal',
 'PlasmacytoidDendriticCells',
 'ProBCells',
 'SchwannCellPrecursors',
 'SchwannCells',
 'SinoatrialNodePacemakerCells',
 'SubEpicardialFibroblasts',
 'SympatheticNeurons',
 'TCellsCD4pos',
 'TCellsCD8pos',
 'TregsCD4pos',
 'ValveEndothelialCells',
 'ValveInterstitialCells',
 'VentricularCardiomyocytesCycling',
 'VentricularCardiomyocytesLeftCompact',
 'VentricularCardiomyocytesLeftTrabeculated',
 'VentricularCardiomyocytesRightCompact',
 'VentricularCardiomyocytesRightTrabeculated',
 'VentricularConductionSystemDistal',
 'VentricularConductionSystemProximal'}
```

Example 40 (python):
```python
sc.pl.embedding(adata,basis='X_umap',
                color='fine_grain_manual',
                frameon=False,
                wspace=0.3)
```

Example 41 (python):
```python
markers={
    'Mono':['CCR2','FCN1','VCAN','MPO'],
    'MP':['C1QA','C1QB','CD68','MRC1','CD163','MERTK'],
    'CX3CR1+':['CX3CR1','C3','TREM2','P2RY12','CD200R1'],
    'LYVE1+':['LYVE1','FOLR2'],
     'pDC':['IRF8','IRF4','IL3RA'],
     'DC_mature':['FLT3', 'CCR7','LAMP3', 'IL15', 'CD83', 'LY75', ],
    'Megakaryocyte':['ITGA2B','MPL','ITGB3',],
    'Mast': ['KIT', 'CPA3','HDC','GATA2'],
    'Cycling':['MKI67','TOP2A'],
}
genes = []
markers_sel = {}
for k,g in markers.items():
    markers_sel[k] = [x for x in g if x in adata.var_names]
    genes = genes + [x for x in g if (x in adata.var_names)&(x not in genes)]

sc.tl.dendrogram(adata,groupby='fine_grain_manual')
sc.pl.dotplot(adata,
              var_names=markers_sel,
              groupby='fine_grain_manual',
              standard_scale='var',
              dendrogram=True
             )
```

Example 42 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: Macrophages, Monocytes, Erythrocytes, etc.
var_group_labels: Mono, MP, CX3CR1+, etc.
```

Example 43 (python):
```python
adata_sub = adata[adata.obs['fine_grain_manual'].isin(['Monocytes','Macrophages'])]
adata_sub.obs['fine_grain_manual'].value_counts()
```

Example 44 (unknown):
```unknown
fine_grain_manual
Macrophages    440
Monocytes       36
Name: count, dtype: int64
```

Example 45 (python):
```python
# drop leiden clustering results
resolutions_list = [0.2,0.4,0.6,1, 1.2,1.5,2]
for resolution in resolutions_list:
    adata_sub.obs.drop(f'leiden_{str(resolution)}',axis=1,inplace=True)
adata_sub
```

Example 46 (unknown):
```unknown
AnnData object with n_obs × n_vars = 476 × 13502
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'fine_grain_manual', 'leiden_0.5'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'dendrogram_fine_grain_manual', 'dendrogram_leiden_0.4', 'dendrogram_leiden_0.6', 'dendrogram_leiden_1', 'dendrogram_leiden_1.2', 'dendrogram_leiden_1.5', 'hvg', 'leiden_0.2', 'leiden_0.4', 'leiden_0.5', 'leiden_0.5_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1.2', 'leiden_1.2_colors', 'leiden_1.5', 'leiden_1.5_colors', 'leiden_1_colors', 'leiden_2', 'leiden_2_colors', 'log1p', 'neighbors', 'pca', 'rank_genes_groups', 'spatial', 'umap'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 47 (python):
```python
## Remove mid-grain markers from the feature
# read in markers
midgrain_markers = pd.read_csv('/nfs/team205/kk18/notebooks/Foetal/VisiumHD/re-seq_mpp0p3/sc_midgrain_markers.csv',
                               index_col=0)
# remove the markers of celltype of interest
midgrain_markers = midgrain_markers[midgrain_markers['group']!=celltype]
# remove
mask = [x not in list(midgrain_markers['names']) for x in adata_sub.var_names]
bdata = adata_sub[:,mask]
bdata
```

Example 48 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 476 × 12303
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'fine_grain_manual', 'leiden_0.5'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'dendrogram_fine_grain_manual', 'dendrogram_leiden_0.4', 'dendrogram_leiden_0.6', 'dendrogram_leiden_1', 'dendrogram_leiden_1.2', 'dendrogram_leiden_1.5', 'hvg', 'leiden_0.2', 'leiden_0.4', 'leiden_0.5', 'leiden_0.5_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1.2', 'leiden_1.2_colors', 'leiden_1.5', 'leiden_1.5_colors', 'leiden_1_colors', 'leiden_2', 'leiden_2_colors', 'log1p', 'neighbors', 'pca', 'rank_genes_groups', 'spatial', 'umap'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 49 (python):
```python
# remove cell cycle genes
cell_cycle_genes = ['MCM5','PCNA','TYMS','FEN1','MCM2','MCM4','RRM1','UNG','GINS2','MCM6','CDCA7','DTL','PRIM1','UHRF1','MLF1IP','HELLS','RFC2','RPA2','NASP',
                'RAD51AP1','GMNN','WDR76','SLBP','CCNE2','UBR7','POLD3','MSH2','ATAD2','RAD51','RRM2','CDC45','CDC6','EXO1','TIPIN','DSCC1','BLM','CASP8AP2',
                'USP1','CLSPN','POLA1','CHAF1B','BRIP1','E2F8','HMGB2','CDK1','NUSAP1','UBE2C','BIRC5','TPX2','TOP2A','NDC80','CKS2','NUF2','CKS1B','MKI67',
                'TMPO','CENPF','TACC3','FAM64A','SMC4','CCNB2','CKAP2L','CKAP2','AURKB','BUB1','KIF11','ANP32E','TUBB4B','GTSE1','KIF20B','HJURP','CDCA3','HN1',
                'CDC20','TTK','CDC25C','KIF2C','RANGAP1','NCAPD2','DLGAP5','CDCA2','CDCA8','ECT2','KIF23','HMMR','AURKA','PSRC1','ANLN','LBR','CKAP5','CENPE','CTCF',
                'NEK2','G2E3','GAS2L3','CBX5','CENPA']
mask = [x not in cell_cycle_genes for x in bdata.var_names]
bdata = bdata[:,mask]
bdata.shape
```

Example 50 (unknown):
```unknown
(476, 12209)
```

Example 51 (python):
```python
# HVGs, using the object which doesn't have midgrain markers
sc.pp.highly_variable_genes(bdata, flavor='seurat', min_mean=0.0125, max_mean=3, min_disp=0.5)
print(f'number of hvgs: {sum(bdata.var.highly_variable)}')
# PCA using the degected HVGs
sc.tl.pca(bdata, use_highly_variable=True)
adata_sub.obsm['X_pca'] = bdata.obsm['X_pca'].copy()
# create neighbour graph
sc.pp.neighbors(adata_sub,n_neighbors=15,use_rep='X_pca')
# visualisation with umap
sc.tl.umap(adata_sub,min_dist=0.5)
sc.pl.embedding(adata_sub,basis='X_umap',color=['bin_count','total_counts','n_genes_by_counts',"conf_score_midmod2fine","celltypist_midmod2fine",],
                wspace=0.2,cmap='RdPu',vmax='p100',ncols=5)
# leiden clustering
resolutions_list = [0.2,0.4,0.5,0.6,1,1.2,1.5,2]
for resolution in resolutions_list:
    sc.tl.leiden(adata_sub,resolution=resolution,key_added=f'leiden_{str(resolution)}',n_iterations=2)
sc.pl.embedding(adata_sub,basis='X_umap',
                color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                wspace=0.3)
```

Example 52 (unknown):
```unknown
number of hvgs: 2992
```

Example 53 (python):
```python
markers={
    'Mono':['CCR2','FCN1','VCAN','MPO'],
    'MP':['C1QA','C1QB','CD68','MRC1','CD163','MERTK'],
    'CX3CR1+':['CX3CR1','C3','TREM2','P2RY12','CD200R1'],
    'LYVE1+':['LYVE1','FOLR2'],
     'pDC':['IRF8','IRF4','IL3RA'],
     'DC_mature':['FLT3', 'CCR7','LAMP3', 'IL15', 'CD83', 'LY75', ],
    'Megakaryocyte':['ITGA2B','MPL','ITGB3',],
    'Mast': ['KIT', 'CPA3','HDC','GATA2'],
    'Cycling':['MKI67','TOP2A'],
}
genes = []
for g in markers.values():
    genes = genes + [x for x in g if (x in adata_sub.var_names)&(x not in genes)]
sc.pl.embedding(adata_sub,basis='X_umap',color=genes,
               ncols=5, wspace=0.2,cmap='RdPu',vmax='p99.7')
```

Example 54 (unknown):
```unknown
sub_resolution_sel = 1
```

Example 55 (python):
```python
sc.tl.dendrogram(adata_sub,groupby=f'leiden_{str(sub_resolution_sel)}')
sc.pl.dotplot(adata_sub,
              var_names=markers_sel,
              groupby=f'leiden_{str(sub_resolution_sel)}',
              standard_scale='var',
              dendrogram=True
             )
```

Example 56 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: Mono, MP, CX3CR1+, etc.
```

Example 57 (python):
```python
# DEG analysis
print(adata_sub.X.data[:5])
adata_sub.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata_sub,groupby=f'leiden_{str(sub_resolution_sel)}')
sc.tl.dendrogram(adata_sub, groupby=f'leiden_{str(sub_resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata_sub, dendrogram=True, standard_scale='var', return_fig=True)
dp.add_totals().show()
```

Example 58 (unknown):
```unknown
[1.41297585 1.03895088 1.42284238 1.5805253  1.88093143]
```

Example 59 (python):
```python
sc.pl.embedding(adata_sub,basis='X_umap',
                color=f'leiden_{str(sub_resolution_sel)}',
                wspace=0.3, legend_loc='on data')
```

Example 60 (python):
```python
# calculate CX3CR1/LYVE1 expression ratio
# to annotate MacrophagesCX3CR1pos
df = adata_sub.to_df()[['CX3CR1','LYVE1']].copy()
df[f'leiden_{str(sub_resolution_sel)}'] = adata_sub.obs[f'leiden_{str(sub_resolution_sel)}']
df = df.groupby(f'leiden_{str(sub_resolution_sel)}').mean()
df['CX3CR1-to-LYVE1'] = df['CX3CR1']/df['LYVE1']
df.sort_values('CX3CR1-to-LYVE1',ascending=False)
```

Example 61 (python):
```python
# annotate
adata_sub.obs['fine_grain_manual'] = adata_sub.obs[f'leiden_{str(sub_resolution_sel)}'].copy()
adata_sub.obs.replace({'fine_grain_manual':{
    '0':'MacrophagesLYVE1pos',
    '1':'MacrophagesLYVE1pos',
    '2':'MacrophagesLYVE1pos',
    '3':'MacrophagesCX3CR1pos',
    '4':'Monocytes',
    '5':'MacrophagesLYVE1pos',
    '6':'MonocyteDerivedCells',
     '7':'MacrophagesLYVE1pos',
     '8':'MacrophagesCX3CR1pos',
    '9':'MacrophagesLYVE1pos',
}},inplace=True)
adata_sub.obs['fine_grain_manual'].value_counts()
```

Example 62 (unknown):
```unknown
fine_grain_manual
MacrophagesLYVE1pos     299
MacrophagesCX3CR1pos     90
Monocytes                47
MonocyteDerivedCells     40
Name: count, dtype: int64
```

Example 63 (python):
```python
ad = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/Feb28ObjectRaw_finegrain_updated.h5ad',backed='r')
set(ad.obs['fine_grain'])
```

Example 64 (unknown):
```unknown
{'AtrialCardiomyocytesCycling',
 'AtrialCardiomyocytesLeft',
 'AtrialCardiomyocytesRight',
 'AtrioventricularNodePacemakerCells',
 'BCells',
 'BCellsMS4A1pos',
 'ChromaffinCells',
 'CoronaryArterialEndothelialCells',
 'CoronaryCapillaryEndothelialCells',
 'CoronaryPericytes',
 'CoronarySmoothMuscleCells',
 'CoronaryVenousEndothelialCells',
 'CoronaryVesselAdventitialFibroblasts',
 'DendriticCellsMature',
 'DendriticCellsType1',
 'DuctusArteriosusSmoothMuscleCells',
 'EndocardialCells',
 'EndocardialCushionCells',
 'EpicardiumDerivedCells',
 'GreatVesselAdventitialFibroblasts',
 'GreatVesselArterialEndothelialCells',
 'GreatVesselSmoothMuscleCells',
 'GreatVesselVenousEndothelialCells',
 'InnateLymphoidCells',
 'LymphNodeFibroblasticReticularCells',
 'LymphaticEndothelialCells',
 'MacrophagesATF3pos',
 'MacrophagesCX3CR1pos',
 'MacrophagesLYVE1pos',
 'MacrophagesTIMD4pos',
 'MastCells',
 'Megakaryocytes',
 'MesothelialEpicardialCells',
 'MonocyteDerivedCells',
 'Monocytes',
 'MonocytesMPOpos',
 'MyocardialInterstitialFibroblasts',
 'Myofibroblasts',
 'NaturalKillerCells',
 'NeuronPrecursors',
 'ParasympatheticNeurons',
 'PericardialCellsFibrous',
 'PericardialCellsIntermediate',
 'PericardialCellsParietal',
 'PlasmacytoidDendriticCells',
 'ProBCells',
 'SchwannCellPrecursors',
 'SchwannCells',
 'SinoatrialNodePacemakerCells',
 'SubEpicardialFibroblasts',
 'SympatheticNeurons',
 'TCellsCD4pos',
 'TCellsCD8pos',
 'TregsCD4pos',
 'ValveEndothelialCells',
 'ValveInterstitialCells',
 'VentricularCardiomyocytesCycling',
 'VentricularCardiomyocytesLeftCompact',
 'VentricularCardiomyocytesLeftTrabeculated',
 'VentricularCardiomyocytesRightCompact',
 'VentricularCardiomyocytesRightTrabeculated',
 'VentricularConductionSystemDistal',
 'VentricularConductionSystemProximal'}
```

Example 65 (python):
```python
sc.pl.embedding(adata_sub, basis=f"X_umap",
                color='fine_grain_manual',
                wspace=0.3)
```

Example 66 (python):
```python
sc.tl.dendrogram(adata_sub,groupby='fine_grain_manual')
sc.pl.dotplot(adata_sub,
              var_names=markers_sel,
              groupby='fine_grain_manual',
              standard_scale='var',
              dendrogram=True
             )
```

Example 67 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: MacrophagesLYVE1pos, MacrophagesCX3CR1pos, Monocytes, etc.
var_group_labels: Mono, MP, CX3CR1+, etc.
```

Example 68 (python):
```python
adata.obs['fine_grain_manual'] = adata.obs['fine_grain_manual'].astype('str')
adata.obs.loc[adata_sub.obs_names,'fine_grain_manual'] = adata_sub.obs['fine_grain_manual'].astype('str').copy()
```

Example 69 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap",
                color='fine_grain_manual'
                )
```

Example 70 (unknown):
```unknown
... storing 'fine_grain_manual' as categorical
```

Example 71 (python):
```python
# save
adata.write(f'{adata_dir}/{sample_id}_{celltype}_b2c_cells_filtered_lognorm.h5ad')
print(f'{adata_dir}/{sample_id}_{celltype}_b2c_cells_filtered_lognorm.h5ad')
```

---

## Function¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig3_SAN_niche/CX3CL1_dotplot.html

**Contents:**
- Function¶
- Read in adata¶
- Check expression of the candidate markers¶

**Examples:**

Example 1 (python):
```python
import scanpy as sc
import pandas as pd
import numpy as np
import anndata

import matplotlib.pyplot as plt
import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42
import seaborn as sns
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
-----
anndata             0.9.2
matplotlib          3.7.2
numpy               1.24.4
pandas              1.5.3
scanpy              1.9.3
seaborn             0.12.2
session_info        1.0.0
-----
```

Example 4 (python):
```python
PIL                         10.0.0
anyio                       NA
arrow                       1.2.3
asttokens                   NA
attr                        23.1.0
attrs                       23.1.0
babel                       2.12.1
backcall                    0.2.0
brotli                      1.0.9
certifi                     2023.11.17
cffi                        1.15.1
charset_normalizer          3.2.0
colorama                    0.4.6
comm                        0.1.4
cycler                      0.10.0
cython_runtime              NA
dateutil                    2.8.2
debugpy                     1.6.8
decorator                   5.1.1
defusedxml                  0.7.1
executing                   1.2.0
fastjsonschema              NA
fqdn                        NA
h5py                        3.9.0
idna                        3.4
igraph                      0.10.6
importlib_metadata          NA
importlib_resources         NA
ipykernel                   6.25.1
isoduration                 NA
jedi                        0.19.0
jinja2                      3.1.2
joblib                      1.3.2
json5                       NA
jsonpointer                 2.0
jsonschema                  4.19.0
jsonschema_specifications   NA
jupyter_events              0.7.0
jupyter_server              2.7.1
jupyterlab_server           2.24.0
kiwisolver                  1.4.5
leidenalg                   0.10.1
llvmlite                    0.40.1
markupsafe                  2.1.3
mpl_toolkits                NA
natsort                     8.4.0
nbformat                    5.9.2
numba                       0.57.1
overrides                   NA
packaging                   23.1
parso                       0.8.3
patsy                       0.5.3
pexpect                     4.8.0
pickleshare                 0.7.5
pkg_resources               NA
plotly                      5.16.1
prometheus_client           NA
prompt_toolkit              3.0.39
psutil                      5.9.5
ptyprocess                  0.7.0
pure_eval                   0.2.2
pydev_ipython               NA
pydevconsole                NA
pydevd                      2.9.5
pydevd_file_utils           NA
pydevd_plugins              NA
pydevd_tracing              NA
pygments                    2.16.1
pyparsing                   3.0.9
pythonjsonlogger            NA
pytz                        2023.3
referencing                 NA
requests                    2.31.0
rfc3339_validator           0.1.4
rfc3986_validator           0.1.1
rpds                        NA
ruamel                      NA
scipy                       1.10.1
send2trash                  NA
six                         1.16.0
sklearn                     1.3.0
sniffio                     1.3.0
socks                       1.7.1
sphinxcontrib               NA
stack_data                  0.6.2
statsmodels                 0.14.0
texttable                   1.6.7
threadpoolctl               3.2.0
tornado                     6.3.3
traitlets                   5.9.0
typing_extensions           NA
uri_template                NA
urllib3                     2.0.4
wcwidth                     0.2.6
webcolors                   1.13
websocket                   1.6.2
yaml                        6.0.1
zipp                        NA
zmq                         25.1.1
zstandard                   0.19.0
```

Example 5 (unknown):
```unknown
-----
IPython             8.12.0
jupyter_client      8.3.0
jupyter_core        4.12.0
jupyterlab          4.0.5
notebook            7.0.2
-----
Python 3.8.17 (default, Jul  5 2023, 21:04:15) [GCC 11.2.0]
Linux-5.4.0-169-generic-x86_64-with-glibc2.17
-----
Session information updated at 2024-02-12 09:08
```

Example 6 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 7 (python):
```python
sc.settings.set_figure_params(dpi=120)
```

Example 8 (python):
```python
# Parameters
path_to_adata_gex = "/nfs/team205/heart/anndata_objects/Foetal/Feb28ObjectRaw.h5ad"
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk'
```

Example 9 (python):
```python
adata = sc.read_h5ad(path_to_adata_gex)
# filter and log normalise
# filter
sc.pp.filter_genes(adata, min_cells=3)

# log normalise
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)
print(adata.X.data[:10])
```

Example 10 (unknown):
```unknown
[0.09802545 0.09802545 0.41538525 0.41538525 0.6009918  0.34498122
 0.8496726  0.34498122 0.2692425  0.09802545]
```

Example 11 (python):
```python
adata.obs['cell_or_nuclei'].value_counts()
```

Example 12 (unknown):
```unknown
nuclei    211145
cell       86328
Name: cell_or_nuclei, dtype: int64
```

Example 13 (python):
```python
markers = ['CX3CL1']

sc.pl.dotplot(adata, 
             markers, 
              groupby='fine_grain',
              dendrogram=False,
              standard_scale="var",
              color_map="Reds",
              swap_axes=True,
              show=False
             )
plt.savefig(f'{figdir}/CX3CL1_expression_dotplot.pdf',bbox_inches="tight",dpi=300)
plt.show()
```

Example 14 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_dotplot.py:749: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap', 'norm' will be ignored
  dot_ax.scatter(x, y, **kwds)
```

Example 15 (python):
```python
# only multiome
# just checking whether the result is similar
markers = ['CX3CL1']

sc.pl.dotplot(adata[adata.obs['kit_10x']=='Multiome-v1'], 
             markers, 
              groupby='fine_grain',
              dendrogram=False,
              standard_scale="var",
              color_map="Reds",
              swap_axes=True,
              # return_fig=True
             )
```

Example 16 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_dotplot.py:749: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap', 'norm' will be ignored
  dot_ax.scatter(x, y, **kwds)
```

---

## Import modules¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/RNA/QC/Foetal_Multiome-RNA_No1_make-anndata_run-scrublet.html

**Contents:**
- Import modules¶
- Functions¶
- Create anndata object¶
- Run scrublet¶
- Save¶

721979 rows × 21 columns

**Examples:**

Example 1 (python):
```python
import numpy as np
import pandas as pd
import scanpy as sc
import matplotlib.pyplot as plt
import scrublet as scr
import session_info
```

Example 2 (python):
```python
sc.settings.verbosity = 3
sc.settings.set_figure_params(dpi = 160, color_map = 'RdPu', dpi_save = 180, vector_friendly = True, format = 'svg')
session_info.show()
```

Example 3 (python):
```python
-----
matplotlib          3.3.4
numpy               1.20.1
pandas              1.1.4
scanpy              1.7.1
scipy               1.6.1
scrublet            NA
session_info        1.0.0
statsmodels         0.12.2
tables              3.6.1
-----
```

Example 4 (unknown):
```unknown
PIL                 8.1.2
anndata             0.7.5
anyio               NA
attr                20.3.0
babel               2.9.0
backcall            0.2.0
brotli              NA
cairo               1.20.0
certifi             2020.12.05
cffi                1.14.5
chardet             4.0.0
cloudpickle         1.6.0
colorama            0.4.4
cycler              0.10.0
cython_runtime      NA
cytoolz             0.11.0
dask                2021.03.1
dateutil            2.8.1
decorator           4.4.2
fsspec              0.8.7
get_version         2.1
google              NA
h5py                2.10.0
idna                2.10
igraph              0.8.3
ipykernel           5.5.0
ipython_genutils    0.2.0
jedi                0.18.0
jinja2              2.11.3
joblib              1.0.1
json5               NA
jsonschema          3.2.0
jupyter_server      1.4.1
jupyterlab_server   2.3.0
kiwisolver          1.3.1
legacy_api_wrap     0.0.0
leidenalg           0.8.3
llvmlite            0.34.0
louvain             0.7.0
markupsafe          1.1.1
mpl_toolkits        NA
natsort             7.1.1
nbclassic           NA
nbformat            5.1.2
numba               0.51.2
numexpr             2.7.3
packaging           20.9
parso               0.8.1
pexpect             4.8.0
pickleshare         0.7.5
pkg_resources       NA
prometheus_client   NA
prompt_toolkit      3.0.16
psutil              5.8.0
ptyprocess          0.7.0
pvectorc            NA
pyarrow             0.16.0
pygments            2.8.0
pyparsing           2.4.7
pyrsistent          NA
pytoml              NA
pytz                2021.1
requests            2.25.1
ruamel              NA
send2trash          NA
setuptools_scm      NA
simplejson          3.17.2
sinfo               0.3.1
six                 1.15.0
sklearn             0.22
sniffio             1.2.0
socks               1.7.1
sphinxcontrib       NA
storemagic          NA
tblib               1.7.0
texttable           1.6.3
tlz                 0.11.0
toolz               0.11.1
tornado             6.1
traitlets           5.0.5
typing_extensions   NA
uritemplate         3.0.1
urllib3             1.26.3
wcwidth             0.2.5
yaml                5.1.2
zmq                 22.0.3
```

Example 5 (unknown):
```unknown
-----
IPython             7.21.0
jupyter_client      6.1.11
jupyter_core        4.7.1
jupyterlab          3.0.9
notebook            6.2.0
-----
Python 3.8.8 | packaged by conda-forge | (default, Feb 20 2021, 16:22:27) [GCC 9.3.0]
Linux-4.15.0-112-generic-x86_64-with-glibc2.10
-----
Session information updated at 2023-02-08 21:30
```

Example 6 (python):
```python
"""Functions for downstream work with outputs of remove-background."""

# to read cellbender outputs<br>
# https://github.com/broadinstitute/CellBender/issues/57

import tables
import numpy as np
import scipy.sparse as sp
from typing import Dict


def dict_from_h5(file: str) -> Dict[str, np.ndarray]:
    """Read in everything from an h5 file and put into a dictionary."""
    d = {}
    with tables.open_file(file) as f:
        # read in everything
        for array in f.walk_nodes("/", "Array"):
            d[array.name] = array.read()
    return d


def anndata_from_h5(file: str,
                    analyzed_barcodes_only: bool = True) -> 'anndata.AnnData':
    """Load an output h5 file into an AnnData object for downstream work.

    Args:
        file: The h5 file
        analyzed_barcodes_only: False to load all barcodes, so that the size of
            the AnnData object will match the size of the input raw count matrix.
            True to load a limited set of barcodes: only those analyzed by the
            algorithm. This allows relevant latent variables to be loaded
            properly into adata.obs and adata.obsm, rather than adata.uns.

    Returns:
        adata: The anndata object, populated with inferred latent variables
            and metadata.

    """

    try:
        import anndata
    except ImportError:
        raise ImportError('The anndata package must be installed to use the '
                          'function anndata_from_h5()')

    d = dict_from_h5(file)
    X = sp.csc_matrix((d.pop('data'), d.pop('indices'), d.pop('indptr')),
                      shape=d.pop('shape')).transpose().tocsr()

    if analyzed_barcodes_only:
        if 'barcodes_analyzed_inds' in d.keys():
            X = X[d['barcodes_analyzed_inds'], :]
            d['barcodes'] = d['barcodes'][d['barcodes_analyzed_inds']]
        elif 'barcode_indices_for_latents' in d.keys():
            X = X[d['barcode_indices_for_latents'], :]
            d['barcodes'] = d['barcodes'][d['barcode_indices_for_latents']]
        else:
            print('Warning: analyzed_barcodes_only=True, but the key '
                  '"barcodes_analyzed_inds" or "barcode_indices_for_latents" '
                  'is missing from the h5 file. '
                  'Will output all barcodes, and proceed as if '
                  'analyzed_barcodes_only=False')

    # Construct the count matrix.
    adata = anndata.AnnData(X=X,
                            obs={'barcode': d.pop('barcodes').astype(str)},
                            var={'gene_name': (d.pop('gene_names') if 'gene_names' in d.keys()
                                               else d.pop('name')).astype(str)})
    adata.obs.set_index('barcode', inplace=True)
    adata.var.set_index('gene_name', inplace=True)

    # Add other information to the adata object in the appropriate slot.
    for key, value in d.items():
        try:
            value = np.asarray(value)
            if len(value.shape) == 0:
                adata.uns[key] = value
            elif value.shape[0] == X.shape[0]:
                if (len(value.shape) < 2) or (value.shape[1] < 2):
                    adata.obs[key] = value
                else:
                    adata.obsm[key] = value
            elif value.shape[0] == X.shape[1]:
                if value.dtype.name.startswith('bytes'):
                    adata.var[key] = value.astype(str)
                else:
                    adata.var[key] = value
            else:
                adata.uns[key] = value
        except Exception:
            print('Unable to load data into AnnData: ', key, value, type(value))

    if analyzed_barcodes_only:
        for col in adata.obs.columns[adata.obs.columns.str.startswith('barcodes_analyzed')
                                     | adata.obs.columns.str.startswith('barcode_indices')]:
            try:
                del adata.obs[col]
            except Exception:
                pass

    return adata
```

Example 7 (python):
```python
# from https://github.com/Teichlab/mapcloud/blob/master/scripts/starsolo/postprocess.py

from statsmodels.stats.multitest import multipletests
# from emptydrops.matrix import CountMatrix
# from emptydrops import find_nonambient_barcodes
import scrublet as scr
import scanpy as sc
import pandas as pd
import numpy as np
import scipy

#some functions that Ni uses in scanpy scripts to run scrublet
#which in turn are inspired by my original notebook on the matter
#(extracted from scanpy_scripts 0.2.10 to get around scanpy version incompatibility)
def test_outlier(x, upper_mad_only=True):
	med = np.median(x)
	if upper_mad_only:
		mad = np.median(x[x>med] - med) * 1.4826
	else:
		mad = np.median(np.abs(x - med)) * 1.4826
	pvals = 1 - scipy.stats.norm.cdf(x, loc=med, scale=mad)
	bh_pvals = multipletests(pvals, method='fdr_bh')[1]
	return pvals, bh_pvals

def run_scrublet(adata, resolution_function=None):
	old_verbosity = sc.settings.verbosity
	sc.settings.verbosity = 1
	if resolution_function is None:
		resolution_function = lambda x: np.maximum(np.maximum(np.log10(x)-1, 0)**2, 0.1)
	scrub = scr.Scrublet(adata.X)
	#this has the potential to brick for poor quality data
	#if so, abort it and everything downstream
	try:
		ds, pd = scrub.scrub_doublets(verbose=False)
	except:
		return
	adata.obs['scrublet_score'] = ds

	adata_copy = adata.copy()
	sc.pp.filter_genes(adata_copy, min_cells=3)
	sc.pp.normalize_total(adata_copy, target_sum=1e4)
	sc.pp.log1p(adata_copy)
	sc.pp.highly_variable_genes(adata_copy, min_mean=0.0125, max_mean=3, min_disp=0.5, subset=True)
	sc.pp.scale(adata_copy, zero_center=False)
	sc.pp.pca(adata_copy, svd_solver='arpack', zero_center=False)
	sc.pp.neighbors(adata_copy, n_pcs=30)
	sc.tl.umap(adata_copy)
	sc.tl.leiden(adata_copy, resolution=1)
	for clst in np.unique(adata_copy.obs['leiden']):
		clst_size = sum(adata_copy.obs['leiden'] == clst)
		sc.tl.leiden(adata_copy, restrict_to=('leiden', [clst]), resolution=resolution_function(clst_size), key_added='leiden_R')
		adata_copy.obs['leiden'] = adata_copy.obs['leiden_R']
	clst_meds = []
	for clst in np.unique(adata_copy.obs['leiden']):
		k = adata_copy.obs['leiden'] == clst
		clst_med = np.median(adata_copy.obs.loc[k, 'scrublet_score'])
		adata_copy.obs.loc[k, 'cluster_scrublet_score'] = clst_med
		clst_meds.append(clst_med)
	clst_meds = np.array(clst_meds)
	pvals, bh_pvals = test_outlier(clst_meds)
	for i, clst in enumerate(np.unique(adata_copy.obs['leiden'])):
		k = adata_copy.obs['leiden'] == clst
		adata_copy.obs.loc[k, 'pval'] = pvals[i]
		adata_copy.obs.loc[k, 'bh_pval'] = bh_pvals[i]
	sc.settings.verbosity = old_verbosity
	#need to also export the clustering, for soupx purposes
	adata.obs['scrublet_leiden'] = adata_copy.obs['leiden']
	adata.obs['scrublet_score'] = adata_copy.obs['scrublet_score']
	adata.obs['cluster_scrublet_score'] = adata_copy.obs['cluster_scrublet_score']
	adata.obs['doublet_pval'] = adata_copy.obs['pval']
	adata.obs['doublet_bh_pval'] = adata_copy.obs['bh_pval']
	del adata_copy
```

Example 8 (python):
```python
# read in metadata
metadata = pd.read_csv('/nfs/team205/heart/anndata_objects/Foetal/metadata/HeartTeamSamples_Mappeddata_20230208.csv', sep = ',', index_col = None)
metadata = metadata[metadata['Publication']=='Foetal']

# select healthy data
metadata = metadata[metadata['status']=='Healthy']


print(metadata['modality'].value_counts())
```

Example 9 (unknown):
```unknown
Multiome-RNA     40
Multiome-ATAC    40
Visium           23
scRNA            20
Visium-FFPE       2
Name: modality, dtype: int64
```

Example 10 (python):
```python
# select modality
metadata = metadata[metadata['modality']=='Multiome-RNA']
```

Example 11 (python):
```python
########### editted: put id to var_names ###########

# read in
adatas=[]
for i in range(len(metadata)):
    print(metadata.iloc[i]['combinedID'])
    
    path=metadata.iloc[i]['CellBender_out']
    adatas.append(anndata_from_h5(path + '/' + path.split('/')[-1] + '_cellbender_out_filtered.h5', analyzed_barcodes_only=False))
    del path
    
    # replace var_names with ensembleID
    adatas[i].var.reset_index(inplace=True)
    if 'id' in adatas[i].var.columns:
        adatas[i].var.set_index('id',inplace=True)
    else:
        adatas[i].var.set_index('genes',inplace=True)
    
    # modify barcodes
    adatas[i].obs.index = metadata.iloc[i]['combinedID'] + '_' + adatas[i].obs.index 
    
    # add metadata
    for col in ['sangerID','combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age',
                'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x','flushed']:
        adatas[i].obs[col] = metadata.iloc[i][col]


# concatenate
adata = adatas[0].concatenate(adatas[1:], index_unique = None, batch_key=None)
adata.shape
```

Example 12 (unknown):
```unknown
BHF_F_Hea11064670_BHF_F_Hea11031823
BHF_F_Hea11064671_BHF_F_Hea11031824
BHF_F_Hea11064672_BHF_F_Hea11031825
BHF_F_Hea11933664_BHF_F_Hea11596617
BHF_F_Hea11933665_BHF_F_Hea11596618
BHF_F_Hea11933666_BHF_F_Hea11596619
BHF_F_Hea11933667_BHF_F_Hea11596620
BHF_F_Hea11933668_BHF_F_Hea11596621
BHF_F_Hea11933669_BHF_F_Hea11596622
BHF_F_Hea11933670_BHF_F_Hea11596623
BHF_F_Hea11933671_BHF_F_Hea11596624
BHF_F_Hea11933672_BHF_F_Hea11596625
BHF_F_Hea11933673_BHF_F_Hea11596626
BHF_F_Hea11933674_BHF_F_Hea11596627
BHF_F_Hea11933675_BHF_F_Hea11596628
BHF_F_Hea13188316_BHF_F_Hea13187618
BHF_F_Hea13188317_BHF_F_Hea13187619
BHF_F_Hea13188318_BHF_F_Hea13187620
BHF_F_Hea13188324_BHF_F_Hea13187626
BHF_F_Hea13188328_BHF_F_Hea13187630
BHF_F_Hea13188329_BHF_F_Hea13187631
BHF_F_Hea13188330_BHF_F_Hea13187632
BHF_F_Hea13168892_BHF_F_Hea13168508
BHF_F_Hea13168893_BHF_F_Hea13168509
BHF_F_Hea13168894_BHF_F_Hea13168510
BHF_F_Hea13168895_BHF_F_Hea13168511
BHF_F_Hea13168896_BHF_F_Hea13168512
BHF_F_Hea13168897_BHF_F_Hea13168513
BHF_F_Hea13168898_BHF_F_Hea13168514
BHF_F_Hea13168899_BHF_F_Hea13168515
7089STDY13216920_BHF_F_Hea13242527
7089STDY13216921_BHF_F_Hea13242528
7089STDY13216922_BHF_F_Hea13242529
7089STDY13216923_BHF_F_Hea13242530
7089STDY13216924_BHF_F_Hea13242531
7089STDY13216925_BHF_F_Hea13242532
7089STDY13216926_BHF_F_Hea13242533
7089STDY13216927_BHF_F_Hea13242534
HCAHeartST13386009_HCAHeartST13303419
HCAHeartST13386010_HCAHeartST13303420
```

Example 13 (unknown):
```unknown
(721979, 36601)
```

Example 14 (python):
```python
for i in range(len(adatas)):
    print(adatas[i].var.columns)
```

Example 15 (unknown):
```unknown
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
```

Example 16 (python):
```python
adata.obs['combinedID'].value_counts()
```

Example 17 (unknown):
```unknown
BHF_F_Hea13188316_BHF_F_Hea13187618      59866
7089STDY13216924_BHF_F_Hea13242531       40000
BHF_F_Hea11933665_BHF_F_Hea11596618      33997
7089STDY13216925_BHF_F_Hea13242532       30453
BHF_F_Hea13188318_BHF_F_Hea13187620      27856
7089STDY13216920_BHF_F_Hea13242527       27498
BHF_F_Hea13168898_BHF_F_Hea13168514      24781
7089STDY13216927_BHF_F_Hea13242534       24666
7089STDY13216926_BHF_F_Hea13242533       24602
BHF_F_Hea13188324_BHF_F_Hea13187626      24381
7089STDY13216921_BHF_F_Hea13242528       24211
BHF_F_Hea13168896_BHF_F_Hea13168512      22267
HCAHeartST13386009_HCAHeartST13303419    21577
BHF_F_Hea11933666_BHF_F_Hea11596619      21509
BHF_F_Hea13168899_BHF_F_Hea13168515      21418
7089STDY13216922_BHF_F_Hea13242529       21402
BHF_F_Hea13188317_BHF_F_Hea13187619      21327
BHF_F_Hea11933664_BHF_F_Hea11596617      20338
BHF_F_Hea11933667_BHF_F_Hea11596620      19811
BHF_F_Hea13168894_BHF_F_Hea13168510      19258
BHF_F_Hea11933669_BHF_F_Hea11596622      19029
BHF_F_Hea13168897_BHF_F_Hea13168513      18609
HCAHeartST13386010_HCAHeartST13303420    18350
BHF_F_Hea13168895_BHF_F_Hea13168511      18315
BHF_F_Hea11933668_BHF_F_Hea11596621      17494
BHF_F_Hea13168893_BHF_F_Hea13168509      11782
BHF_F_Hea11933670_BHF_F_Hea11596623      11068
BHF_F_Hea11933673_BHF_F_Hea11596626      10769
BHF_F_Hea11933671_BHF_F_Hea11596624      10256
BHF_F_Hea11064672_BHF_F_Hea11031825      10191
BHF_F_Hea13168892_BHF_F_Hea13168508      10023
BHF_F_Hea11933672_BHF_F_Hea11596625       9711
BHF_F_Hea11933674_BHF_F_Hea11596627       6199
BHF_F_Hea13188328_BHF_F_Hea13187630       5340
BHF_F_Hea11064671_BHF_F_Hea11031824       4757
BHF_F_Hea11064670_BHF_F_Hea11031823       2665
BHF_F_Hea13188329_BHF_F_Hea13187631       1878
7089STDY13216923_BHF_F_Hea13242530        1873
BHF_F_Hea13188330_BHF_F_Hea13187632       1750
BHF_F_Hea11933675_BHF_F_Hea11596628        702
Name: combinedID, dtype: int64
```

Example 18 (python):
```python
%%time

# per rxn
for i,ID in enumerate(adata.obs['combinedID'].unique()):
    print(ID)
    
    ad = adata[adata.obs['combinedID'] == ID].copy()
    run_scrublet(ad)
    if i==0:
        meta = ad.obs
    else:
        meta = pd.concat([meta, ad.obs])
    del ad
```

Example 19 (unknown):
```unknown
BHF_F_Hea11064670_BHF_F_Hea11031823
```

Example 20 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 21 (unknown):
```unknown
BHF_F_Hea11064671_BHF_F_Hea11031824
BHF_F_Hea11064672_BHF_F_Hea11031825
```

Example 22 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 23 (unknown):
```unknown
BHF_F_Hea11933664_BHF_F_Hea11596617
```

Example 24 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 25 (unknown):
```unknown
BHF_F_Hea11933665_BHF_F_Hea11596618
```

Example 26 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 27 (unknown):
```unknown
BHF_F_Hea11933666_BHF_F_Hea11596619
```

Example 28 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 29 (unknown):
```unknown
BHF_F_Hea11933667_BHF_F_Hea11596620
```

Example 30 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 31 (unknown):
```unknown
BHF_F_Hea11933668_BHF_F_Hea11596621
```

Example 32 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 33 (unknown):
```unknown
BHF_F_Hea11933669_BHF_F_Hea11596622
```

Example 34 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 35 (unknown):
```unknown
BHF_F_Hea11933670_BHF_F_Hea11596623
```

Example 36 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 37 (unknown):
```unknown
BHF_F_Hea11933671_BHF_F_Hea11596624
```

Example 38 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 39 (unknown):
```unknown
BHF_F_Hea11933672_BHF_F_Hea11596625
```

Example 40 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 41 (unknown):
```unknown
BHF_F_Hea11933673_BHF_F_Hea11596626
```

Example 42 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 43 (unknown):
```unknown
BHF_F_Hea11933674_BHF_F_Hea11596627
```

Example 44 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 45 (unknown):
```unknown
BHF_F_Hea11933675_BHF_F_Hea11596628
BHF_F_Hea13188316_BHF_F_Hea13187618
```

Example 46 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 47 (unknown):
```unknown
BHF_F_Hea13188317_BHF_F_Hea13187619
```

Example 48 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 49 (unknown):
```unknown
BHF_F_Hea13188318_BHF_F_Hea13187620
```

Example 50 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 51 (unknown):
```unknown
BHF_F_Hea13188324_BHF_F_Hea13187626
```

Example 52 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 53 (unknown):
```unknown
BHF_F_Hea13188328_BHF_F_Hea13187630
```

Example 54 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 55 (unknown):
```unknown
BHF_F_Hea13188329_BHF_F_Hea13187631
```

Example 56 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 57 (unknown):
```unknown
BHF_F_Hea13188330_BHF_F_Hea13187632
```

Example 58 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 59 (unknown):
```unknown
BHF_F_Hea13168892_BHF_F_Hea13168508
```

Example 60 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 61 (unknown):
```unknown
BHF_F_Hea13168893_BHF_F_Hea13168509
```

Example 62 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 63 (unknown):
```unknown
BHF_F_Hea13168894_BHF_F_Hea13168510
```

Example 64 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 65 (unknown):
```unknown
BHF_F_Hea13168895_BHF_F_Hea13168511
```

Example 66 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 67 (unknown):
```unknown
BHF_F_Hea13168896_BHF_F_Hea13168512
```

Example 68 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 69 (unknown):
```unknown
BHF_F_Hea13168897_BHF_F_Hea13168513
```

Example 70 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 71 (unknown):
```unknown
BHF_F_Hea13168898_BHF_F_Hea13168514
```

Example 72 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 73 (unknown):
```unknown
BHF_F_Hea13168899_BHF_F_Hea13168515
```

Example 74 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 75 (unknown):
```unknown
7089STDY13216920_BHF_F_Hea13242527
```

Example 76 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 77 (unknown):
```unknown
7089STDY13216921_BHF_F_Hea13242528
```

Example 78 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 79 (unknown):
```unknown
7089STDY13216922_BHF_F_Hea13242529
```

Example 80 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 81 (unknown):
```unknown
7089STDY13216923_BHF_F_Hea13242530
7089STDY13216924_BHF_F_Hea13242531
7089STDY13216925_BHF_F_Hea13242532
```

Example 82 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 83 (unknown):
```unknown
7089STDY13216926_BHF_F_Hea13242533
```

Example 84 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 85 (unknown):
```unknown
7089STDY13216927_BHF_F_Hea13242534
```

Example 86 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 87 (unknown):
```unknown
HCAHeartST13386009_HCAHeartST13303419
```

Example 88 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 89 (unknown):
```unknown
HCAHeartST13386010_HCAHeartST13303420
```

Example 90 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 91 (unknown):
```unknown
CPU times: user 2h 18min 29s, sys: 1h 16min 7s, total: 3h 34min 36s
Wall time: 59min 16s
```

Example 92 (python):
```python
meta_scrub = meta[['scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval']].copy()
meta_scrub.shape
```

Example 93 (unknown):
```unknown
(721979, 5)
```

Example 94 (python):
```python
if meta_scrub.reindex(adata.obs.index).index.equals(adata.obs.index):
    adata.obs = pd.concat([adata.obs, meta_scrub.reindex(adata.obs.index)], axis=1)
else:
    raise Exception('Different barcodes in meta and adata')
```

Example 95 (python):
```python
adata.write('/nfs/team205/heart/anndata_objects/Foetal/QC/multiome_RNA_foetal_prefilter.h5ad')
```

Example 96 (unknown):
```unknown
... storing 'sangerID' as categorical
... storing 'combinedID' as categorical
... storing 'donor' as categorical
... storing 'donor_type' as categorical
... storing 'region' as categorical
... storing 'region_finest' as categorical
... storing 'age' as categorical
... storing 'gender' as categorical
... storing 'facility' as categorical
... storing 'cell_or_nuclei' as categorical
... storing 'modality' as categorical
... storing 'kit_10x' as categorical
... storing 'flushed' as categorical
... storing 'scrublet_leiden' as categorical
... storing 'gene_name' as categorical
... storing 'feature_type' as categorical
```

Example 97 (python):
```python
sc.pl.violin(adata,
             ['scrublet_score', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval'],jitter = 0.2, multi_panel = True)
```

Example 98 (python):
```python
adata.obs['donor'].value_counts()
```

Example 99 (unknown):
```unknown
Hst41                  319810
Hst33                  132178
Hst39                   64602
Hst39, Hst40            51709
C92                     41804
Hst40                   30453
Hst36                   24666
Hst36, Hst39, Hst40     23275
C87                     10191
C104                     8968
C98                      6901
C85                      4757
C82                      2665
Name: donor, dtype: int64
```

---

## Read in data¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/VisiumHD/no1_bin2cell_HE-nucleus-segmentation.html

**Contents:**
- Read in data¶
- Pre-processing¶
- H&E segmentation¶

The required inputs are the 2um bin output from the quantification pipeline, along with the high resolution H&E image that was used with it. Download both and extract them into the same directory as the notebook before proceeding.

Let's slightly filter the object - require the genes to show up in three spots, and require the spots to have any information at all (the data is extremely sparse at this stage).

Over the course of the demo, bin2cell will perform a segmentation of both the H&E image and a gene expression representation of the data. When performing segmentation, the resolution of the input images is controlled via the mpp parameter. This stands for microns per pixel and translates to how many micrometers are captured in each pixel of the input. For example, if using the array coordinates (present as .obs["array_row"] and .obs["array_col"]) as an image, each of the pixels would have 2 micrometers in it, so the mpp of that particular representation is 2.

In local testing, using an mpp of 0.5 has worked well with both GEX and H&E segmentation.

Since we're already generating a custom resolution H&E image, b2c.scaled_he_image() stores it within the object so it can be used for visualisation. The function crops the image to an area around the actual spatial grid present in the object, and the new coordinates are captured in .obsm["spatial_cropped"]. The new image can be used for plotting by providing basis="spatial_cropped" and img_key="0.5_mpp" to sc.pl.spatial(). For segmentation purposes, the image needs to be saved to the drive, and the function does so to a user-specified save_path.

Visium HD suffers from variable bin sizing. When printing the chips, the 2um bins can have about 10% variability in their width/height. Inspecting the total counts per spot reveals a characteristic striped appearance, with some rows/columns capturing visibly fewer transcripts than others.

To overcome this, b2c.destripe() identifies a user-specified quantile (by default 0.99) of total counts for each row, then divides the counts of the spots in that row by that value. This procedure is then repeated for the columns. .obs[adjusted_counts_key] (by default "n_counts_adjusted") is obtained by multiplying the resulting per-spot factor by the global quantile of count totals, and the count matrix is by default rescaled to match it.

Prior to the adjustment, the count total has a visible striped appearance. The effect is diminished after the correction.

We will begin by performing H&E segmentation with StarDist's H&E model. This is likely to be the main segmentation used in analysis due to consistently picking out nuclei, getting reliable information on the location of cells in the data. It is recommended to lower the prob_thresh, making the model more lenient with regard to what it calls as nuclei - the default setting is quite stringent, while we want to seed a good number of putative cells in the object.

The segmentation results are turned to a sparse matrix and efficiently stored in an .npz file. If desired, they can be read via scipy.sparse.load_npz(), and the sparse matrix's dimensions will match those of the input image.

However, it is important to ensure that the gene expression bin grid is well aligned with the H&E image, or the mismatch will have bins loading segmentation results for sections of the tissue they are quite distant from. Visualising the total expression across parts of the tissue, similar to the plot made to show the "striped" effect earlier, is likely to be helpful in assessing the situation. In the event of a misalignment, the alignment can be corrected via 10X's loupe browser.

We can now load the resulting cell calls into the object. For any given bin, its corresponding pixel in the segmented image will be located and any potential label transferred over. It's important to inform b2c.insert_labels() of whether the segmented image was based on array (GEX visualisation of the grid) or spatial (rescaled H&E image) coordinates, which exact spatial representation matches the image, and what mpp was used.

Let's visualise the nuclear H&E segmentation on our selected region. The nuclei are identified successfully.

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import os

import bin2cell as b2c

#create directory for stardist input/output files
os.makedirs("stardist", exist_ok=True)
```

Example 2 (unknown):
```unknown
os.chdir('/nfs/team205/kk18/notebooks/Foetal/VisiumHD/re-seq_mpp0p3')
os.getcwd()
```

Example 3 (unknown):
```unknown
'/nfs/team205/kk18/notebooks/Foetal/VisiumHD/re-seq_mpp0p3'
```

Example 4 (unknown):
```unknown
!ls /lustre/scratch126/cellgen/team205/sharedData/ny1/24-11-26-visiumhd/HEA_FOET14880396/outs
```

Example 5 (unknown):
```unknown
binned_outputs	  metrics_summary.csv  probe_set.csv  web_summary.html
feature_slice.h5  molecule_info.h5     spatial
```

Example 6 (unknown):
```unknown
path = "/lustre/scratch126/cellgen/team205/sharedData/ny1/24-11-26-visiumhd/HEA_FOET14880396/outs/binned_outputs/square_002um"
spaceranger_image_path = '/lustre/scratch126/cellgen/team205/sharedData/ny1/24-11-26-visiumhd/HEA_FOET14880396/outs/spatial'
source_image_path = "/lustre/scratch126/cellgen/team205/kk18/VisiumHD/source_images/C194_HEA_0_FFPE_1_STAN_1EAF2_s3_2024_04_23_13_40_00.tiff"
```

Example 7 (python):
```python
adata = b2c.read_visium(path, 
                        spaceranger_image_path=spaceranger_image_path, 
                        source_image_path = source_image_path
                       )
adata.var_names_make_unique()
adata
```

Example 8 (unknown):
```unknown
anndata.py (1758): Variable names are not unique. To make them unique, call `.var_names_make_unique`.
anndata.py (1758): Variable names are not unique. To make them unique, call `.var_names_make_unique`.
```

Example 9 (unknown):
```unknown
AnnData object with n_obs × n_vars = 5032713 × 18085
    obs: 'in_tissue', 'array_row', 'array_col'
    var: 'gene_ids', 'feature_types', 'genome'
    uns: 'spatial'
    obsm: 'spatial'
```

Example 10 (python):
```python
sc.pp.filter_genes(adata, min_cells=3)
sc.pp.filter_cells(adata, min_counts=1)
adata
```

Example 11 (unknown):
```unknown
AnnData object with n_obs × n_vars = 5001851 × 18051
    obs: 'in_tissue', 'array_row', 'array_col', 'n_counts'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells'
    uns: 'spatial'
    obsm: 'spatial'
```

Example 12 (python):
```python
mpp = 0.3
b2c.scaled_he_image(adata, mpp=mpp, save_path="stardist/he.tiff")
```

Example 13 (unknown):
```unknown
Cropped spatial coordinates key: spatial_cropped_150_buffer
Image key: 0.3_mpp_150_buffer
```

Example 14 (python):
```python
b2c.destripe(adata,
             quantile=0.99, 
    counts_key='n_counts',
    factor_key='destripe_factor',
    adjusted_counts_key='n_counts_adjusted',
    adjust_counts=True)
```

Example 15 (unknown):
```unknown
_construct.py (149): Series.__getitem__ treating keys as positions is deprecated. In a future version, integer keys will always be treated as labels (consistent with DataFrame behavior). To access a value by position, use `ser.iloc[pos]`
```

Example 16 (python):
```python
#define a mask to easily pull out this region of the object in the future
mask = ((adata.obs['array_row'] >= 1450) & 
        (adata.obs['array_row'] <= 1550) & 
        (adata.obs['array_col'] >= 1400) & 
        (adata.obs['array_col'] <= 1600)
       )

bdata = adata[mask]
sc.pl.spatial(bdata, color=[None, "n_counts", "n_counts_adjusted"], color_map="OrRd",
              img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer")
```

Example 17 (unknown):
```unknown
anndata.py (1146): Trying to modify attribute `.var` of view, initializing view as actual.
anndata.py (1146): Trying to modify attribute `.var` of view, initializing view as actual.
```

Example 18 (python):
```python
sc.pl.spatial(adata,img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer")
```

Example 19 (unknown):
```unknown
KeyboardInterrupt
```

Example 20 (python):
```python
adata.shape
```

Example 21 (unknown):
```unknown
(5001851, 18051)
```

Example 22 (python):
```python
prob_thresh_dict = {
    '0p01':0.01,
     '0p03':0.03,
    '0p05':0.05,
    '0p1':0.1,
}

for key,prob_thresh in prob_thresh_dict.items():
    print(f'### {key} ###')
    b2c.stardist(image_path="stardist/he.tiff", 
                 labels_npz_path=f"stardist/he_{key}.npz", 
                 stardist_model="2D_versatile_he", 
                 prob_thresh=prob_thresh
                )
    print('')
```

Example 23 (unknown):
```unknown
### 0p01 ###
```

Example 24 (unknown):
```unknown
2024-12-05 10:30:10.228900: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2024-12-05 10:30:10.908609: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2024-12-05 10:30:11.205982: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:477] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1733394611.683311 2850957 cuda_dnn.cc:8310] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
E0000 00:00:1733394611.753294 2850957 cuda_blas.cc:1418] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-12-05 10:30:12.674898: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
```

Example 25 (unknown):
```unknown
# default setting
b2c.stardist(image_path="stardist/he.tiff", 
                 labels_npz_path=f"stardist/he_default.npz", 
                 stardist_model="2D_versatile_he", 
                 # prob_thresh=prob_thresh
                )
```

Example 26 (python):
```python
prob_thresh_dict = {
    '0p01':0.01,
     '0p03':0.03,
    '0p05':0.05,
    '0p1':0.1,
    'default':'default'
}

for key,prob_thresh in prob_thresh_dict.items():
    print(f'### {key} ###')
    b2c.insert_labels(adata, 
                  labels_npz_path=f"stardist/he_{key}.npz", 
                  basis="spatial", 
                  spatial_key="spatial_cropped",
                  mpp=mpp, 
                  labels_key=f"labels_he_{key}"
                 )

adata
```

Example 27 (python):
```python
mask = ((adata.obs['array_row'] >= 1050) & 
        (adata.obs['array_row'] <= 1100) & 
        (adata.obs['array_col'] >= 1100) & 
        (adata.obs['array_col'] <= 1150)
       )

bdata = adata[mask]

for key in prob_thresh_dict.keys():
    #0 means unassigned
    bdata = bdata[bdata.obs[f'labels_he_{key}']>0]
    bdata.obs[f'labels_he_{key}'] = bdata.obs[f'labels_he_{key}'].astype(str)
    print(len(set(bdata.obs[f'labels_he_{key}'])))
    sc.pl.spatial(bdata, color=[None,f'labels_he_{key}'], 
                  img_key="0.3_mpp", basis="spatial_cropped", legend_loc=None,
                  ncols=2
                 )
```

---

## 1. Read in inputs¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig3_SAN_niche/snMultime_no3_create-network_TF-Region-TG_SANP-signatures.html

**Contents:**
- 1. Read in inputs¶
- Read in and prepare anndata contains gene expression¶
- Read in peak signatures¶
- Read in scenicplus output metadata¶
- 2. DEG: SANCM vs other CMs¶
- Prepare anndata for pseudobulk wilcoxon test¶
- Pseudobulk wilcoxon test¶
- 4. Obtain expressing genes and accessible regions¶
- Genes¶
- Peaks¶
- Prepare scenic metadata¶
- Select interactions¶
- TF-TF network¶
- Calculate centrality¶
- Save networks¶
- Plot network¶

What this notebook does

**Examples:**

Example 1 (python):
```python
# Parameters
working_dir = "/nfs/team205/heart/Vitro/TF_selection/foetal_SANPcell_vs_workingCM"
path_to_adata_gex = "/nfs/team205/heart/anndata_objects/Foetal/Feb28ObjectRaw_Multiome.h5ad"
path_to_adata_peak_tfidf = "/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/PeakMatrix/Foetal_Peaks_tfidf.h5ad"
path_to_DAR_1 = "/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/MarkerPeaks/SANP-vs-aCM.csv"
path_to_DAR_2 = "/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/MarkerPeaks/aCM-vs-SANP.csv"
path_to_p2g_ArchR = ""
group_key = "fine_grain_aCMs_merged"
group_a = "SinoatrialNodeCardiomyocytes"
group_b = "AtrialCardiomyocytes"
subset_object = True
geneset_list = ["MSigDB_Hallmark_2020", "KEGG_2021_Human", "GO_Biological_Process_2023"]

path_to_eRegulon_signatures_gene = "/nfs/team205/heart/pyscenic/plus/eRegulon_signatures/Cardiomyocytes_Gene_eRegulon_signatures.pkl"
path_to_eRegulon_signatures_region = "/nfs/team205/heart/pyscenic/plus/eRegulon_signatures/Cardiomyocytes_Region_eRegulon_signatures.pkl"

path_to_gene_signatures='/nfs/team205/heart/Vitro/TF_selection/foetal_SANPcell_vs_workingCM/outs/gene_signatures.pkl'
path_to_peak_signatures='/nfs/team205/heart/Vitro/TF_selection/foetal_SANPcell_vs_workingCM/outs/peak_signatures.pkl'

path_to_eRegulon_metadata = "/nfs/team205/heart/pyscenic/plus/eRegulon_metadata/Cardiomyocytes_eRegulon_metadata_filtered.csv" # filtered metadata

###### parameters for TF selection for the population ######
cell_of_intest = 'SinoatrialNodeCardiomyocytes'
groups_for_signatures = ['SinoatrialNodeCardiomyocytes']

# proportion threshold for expressing genes
exp_prop_thresh = 0.1

# proportion threshold for accesible regions
access_prop_thresh = 0.01
```

Example 2 (python):
```python
import scanpy as sc
import pandas as pd
import numpy as np
import anndata

import matplotlib.pyplot as plt
import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 
import seaborn as sns

import os
```

Example 3 (python):
```python
import session_info
session_info.show()
```

Example 4 (python):
```python
-----
anndata             0.9.2
matplotlib          3.7.2
numpy               1.24.4
pandas              1.5.3
scanpy              1.9.3
seaborn             0.12.2
session_info        1.0.0
-----
```

Example 5 (python):
```python
PIL                         10.0.0
anyio                       NA
arrow                       1.2.3
asttokens                   NA
attr                        23.1.0
attrs                       23.1.0
babel                       2.12.1
backcall                    0.2.0
brotli                      1.0.9
certifi                     2023.11.17
cffi                        1.15.1
charset_normalizer          3.2.0
colorama                    0.4.6
comm                        0.1.4
cycler                      0.10.0
cython_runtime              NA
dateutil                    2.8.2
debugpy                     1.6.8
decorator                   5.1.1
defusedxml                  0.7.1
executing                   1.2.0
fastjsonschema              NA
fqdn                        NA
h5py                        3.9.0
idna                        3.4
igraph                      0.10.6
importlib_metadata          NA
importlib_resources         NA
ipykernel                   6.25.1
isoduration                 NA
jedi                        0.19.0
jinja2                      3.1.2
joblib                      1.3.2
json5                       NA
jsonpointer                 2.0
jsonschema                  4.19.0
jsonschema_specifications   NA
jupyter_events              0.7.0
jupyter_server              2.7.1
jupyterlab_server           2.24.0
kiwisolver                  1.4.5
leidenalg                   0.10.1
llvmlite                    0.40.1
markupsafe                  2.1.3
mpl_toolkits                NA
natsort                     8.4.0
nbformat                    5.9.2
numba                       0.57.1
overrides                   NA
packaging                   23.1
parso                       0.8.3
patsy                       0.5.3
pexpect                     4.8.0
pickleshare                 0.7.5
pkg_resources               NA
plotly                      5.16.1
prometheus_client           NA
prompt_toolkit              3.0.39
psutil                      5.9.5
ptyprocess                  0.7.0
pure_eval                   0.2.2
pydev_ipython               NA
pydevconsole                NA
pydevd                      2.9.5
pydevd_file_utils           NA
pydevd_plugins              NA
pydevd_tracing              NA
pygments                    2.16.1
pyparsing                   3.0.9
pythonjsonlogger            NA
pytz                        2023.3
referencing                 NA
requests                    2.31.0
rfc3339_validator           0.1.4
rfc3986_validator           0.1.1
rpds                        NA
ruamel                      NA
scipy                       1.10.1
send2trash                  NA
six                         1.16.0
sklearn                     1.3.0
sniffio                     1.3.0
socks                       1.7.1
sphinxcontrib               NA
stack_data                  0.6.2
statsmodels                 0.14.0
texttable                   1.6.7
threadpoolctl               3.2.0
tornado                     6.3.3
traitlets                   5.9.0
typing_extensions           NA
uri_template                NA
urllib3                     2.0.4
wcwidth                     0.2.6
webcolors                   1.13
websocket                   1.6.2
yaml                        6.0.1
zipp                        NA
zmq                         25.1.1
zstandard                   0.19.0
```

Example 6 (unknown):
```unknown
-----
IPython             8.12.0
jupyter_client      8.3.0
jupyter_core        4.12.0
jupyterlab          4.0.5
notebook            7.0.2
-----
Python 3.8.17 (default, Jul  5 2023, 21:04:15) [GCC 11.2.0]
Linux-5.4.0-169-generic-x86_64-with-glibc2.17
-----
Session information updated at 2024-03-25 16:10
```

Example 7 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)

spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/regulon_plot.py")
regulon_plot = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = regulon_plot
spec.loader.exec_module(regulon_plot)
```

Example 8 (python):
```python
sc.settings.set_figure_params(dpi=80,dpi_save=300)
```

Example 9 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk'
table_dir = '/nfs/team205/heart/fetal_ms_figures/tables'
```

Example 10 (python):
```python
# read in adata
# whole data
adata = sc.read_h5ad(path_to_adata_gex)

# filter and log-normalise
sc.pp.filter_genes(adata, min_cells=3)
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

print(adata.X.data[:10])
adata
```

Example 11 (unknown):
```unknown
[0.41374123 0.22811745 0.22811745 0.41374123 0.22811745 0.41374123
 0.22811745 0.41374123 0.22811745 0.22811745]
```

Example 12 (unknown):
```unknown
AnnData object with n_obs × n_vars = 211145 × 34377
    obs: 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score', 'fine_grain_aCMs_merged', 'fine_grain_aCMs-node_merged'
    var: 'gene_id', 'n_cells'
    uns: 'log1p'
```

Example 13 (unknown):
```unknown
peak_signatures = utils.read_pkl(path_to_peak_signatures)
peak_signatures.keys()
```

Example 14 (unknown):
```unknown
dict_keys(['SinoatrialNodeCardiomyocytes_up', 'SinoatrialNodeCardiomyocytes_down'])
```

Example 15 (python):
```python
sig_peaks = peak_signatures['SinoatrialNodeCardiomyocytes_up']
print(len(sig_peaks))
sig_peaks[:5]
```

Example 16 (unknown):
```unknown
['chr1:55555893_55556393',
 'chr12:115794713_115795213',
 'chr3:113167147_113167647',
 'chr17:65228782_65229282',
 'chr11:116485972_116486472']
```

Example 17 (python):
```python
# scenicplus output
scenic_meta = pd.read_csv(path_to_eRegulon_metadata,index_col=0)
scenic_meta['Region'] = [x.replace('-','_') for x in scenic_meta['Region']]

# select active regulons
scenic_meta = scenic_meta[['_+_' in x for x in scenic_meta['Gene_signature_name']]]
scenic_meta.head()
```

Example 18 (unknown):
```unknown
# select based on R2G_rho
rho_thresh = 0.2
scenic_meta = scenic_meta[scenic_meta['R2G_rho']>rho_thresh]
```

Example 19 (python):
```python
logFC_threshold = 1
group_mean_threshold = 0.1

adata_sub = adata[adata.obs['coarse_grain']=='Cardiomyocytes']
adata_sub = adata_sub[adata_sub.obs['kit_10x']=='Multiome-v1']

res = utils.rank_genes_groups_with_mean(adata_sub,
                                 groupby = 'fine_grain',
                                  groups = ['SinoatrialNodeCardiomyocytes'],
                                  reference = 'rest',
                                  method = 't-test',
                                  pval_cutoff = 0.05,
                                        log2fc_min=logFC_threshold
                                 )
sns.scatterplot(x=res['logfoldchanges'],y=res['group_mean'])
plt.show()
```

Example 20 (python):
```python
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/tools/_rank_genes_groups.py:580: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  adata.uns[key_added] = {}
```

Example 21 (unknown):
```unknown
# filter genes based on the group mean expression
res = res[res['group_mean']>group_mean_threshold]
display(res.head())
degs_sancm_vs_others = list(res['names'])
len(degs_sancm_vs_others)
```

Example 22 (unknown):
```unknown
sig_genes = degs_sancm_vs_others.copy()
```

Example 23 (unknown):
```unknown
'ISL1' in sig_genes
```

Example 24 (python):
```python
# prepare adult anndata
adata_adult = sc.read_h5ad('/nfs/team205/heart/anndata_objects/8regions/RNA_adult-8reg_full_raw_cellstate-annotated.h5ad')

# subset SAN_P_cell
adata_adult = adata_adult[adata_adult.obs['cell_state']=='SAN_P_cell']
adata_adult = adata_adult[adata_adult.obs['kit_10x']=='Multiome-v1']

# log normalise
sc.pp.normalize_total(adata_adult, target_sum=1e4)
sc.pp.log1p(adata_adult)
print(adata_adult.X.data[:10])

# rename
adata_adult.var['gene_id'] =  adata_adult.var_names.copy()
adata_adult.var.set_index('gene_name-new',inplace=True)
adata_adult.var_names = adata_adult.var_names.astype('str')
adata_adult.var_names_make_unique()

adata_adult
```

Example 25 (unknown):
```unknown
[0.911473   0.31626672 0.31626672 0.31626672 0.74952203 0.911473
 0.31626672 0.31626672 0.31626672 0.31626672]
```

Example 26 (python):
```python
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/preprocessing/_normalization.py:170: UserWarning: Received a view of an AnnData. Making a copy.
  view_to_actual(adata)
```

Example 27 (unknown):
```unknown
AnnData object with n_obs × n_vars = 245 × 32732
    obs: 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'n_genes', 'n_genes_by_counts', 'total_counts', 'total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'pct_counts_ribo', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'batch_key', 'leiden_scVI', 'cell_type', 'cell_state_HCAv1', 'cell_state_scNym', 'cell_state_scNym_confidence', 'cell_state', 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'n_counts', '_scvi_batch', '_scvi_labels', 'clus20', 'doublet_cls', 'original_or_new', 'batch', 'scANVI_predictions', 'leiden_scArches'
    var: 'gene_name_scRNA-0-original', 'gene_name_snRNA-1-original', 'gene_name_multiome-2-original', 'gene_id'
    uns: 'age_colors', 'cell_or_nuclei_colors', 'cell_state_colors', 'cell_type_colors', 'donor_colors', 'donor_type_colors', 'facility_colors', 'flushed_colors', 'gender_colors', 'kit_10x_colors', 'leiden', 'modality_colors', 'neighbors', 'original_or_new_colors', 'region_colors', 'region_finest_colors', 'scANVI_predictions_colors', 'umap', 'log1p'
    obsm: 'X_scArches', 'X_umap'
    obsp: 'connectivities', 'distances'
```

Example 28 (python):
```python
# concatenate
## shared features
shared = list(set(adata_sub.var_names).intersection(adata_adult.var_names))

adata_sub.obs['stage'] = 'foetal'
adata_adult.obs['stage'] = 'adult'
adata_adult.obs['week'] = 'adult'
adata_adult.obs['fine_grain'] = 'adult - SAN_P_cell'

adata_conc = adata_sub[:,shared].concatenate(adata_adult[:,shared],
                                             join='inner',
                                             batch_key=None,
                                             index_unique=None
                                            )
```

Example 29 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/anndata/_core/anndata.py:1763: FutureWarning: The AnnData.concatenate method is deprecated in favour of the anndata.concat function. Please use anndata.concat instead.

See the tutorial for concat at: https://anndata.readthedocs.io/en/latest/concatenation.html
  warnings.warn(
```

Example 30 (python):
```python
# combine donor and stage key
adata_conc.obs['donor_new'] = adata_conc.obs['stage'].astype('str')+'_'+adata_conc.obs['donor'].astype('str')
set(adata_conc.obs['donor_new'])
```

Example 31 (unknown):
```unknown
{'adult_A61',
 'adult_AH1',
 'adult_AH2',
 'adult_AV10',
 'adult_AV14',
 'adult_AV3',
 'foetal_C104',
 'foetal_C82',
 'foetal_C85',
 'foetal_C87',
 'foetal_C92',
 'foetal_C98',
 'foetal_Hst33',
 'foetal_Hst36',
 'foetal_Hst39',
 'foetal_Hst40',
 'foetal_Hst41'}
```

Example 32 (python):
```python
# add drimester 
donor_trimester_df = adata_conc.obs[['donor','trimester']].copy()
donor_trimester_df = donor_trimester_df.drop_duplicates()
donor_trimester_df['trimester'] = [x if x==x else 'adult' for x in donor_trimester_df['trimester']]
donor_trimester_df
```

Example 33 (python):
```python
groupby = 'donor_new'
celltypes = ['SinoatrialNodeCardiomyocytes','adult - SAN_P_cell']
n_cell_thresh = 10

# subset anndata
ad = adata_conc[adata_conc.obs['fine_grain'].isin(celltypes)]
# remove group which only has less than a threshold
vc = ad.obs[groupby].value_counts()
grp_to_drop = list(vc.index[vc < 10])
ad = ad[~ad.obs[groupby].isin(grp_to_drop)]
print(f'dropped groups: {grp_to_drop}')

# mean expression per group
exp_mean = utils.grouped_obs_mean(ad, group_key=groupby)

# make anndata
import anndata
adata_pb = anndata.AnnData(
    X=exp_mean.T,
    obs=pd.DataFrame(exp_mean.columns).set_index(0),
    var=pd.DataFrame(exp_mean.index).set_index(0)
)

# add stage key
adata_pb.obs['stage'] = [x.split('_')[0] for x in adata_pb.obs_names]
adata_pb.obs['stage'] = adata_pb.obs['stage'].astype('category')
adata_pb.obs['stage'].cat.reorder_categories(['foetal','adult'],inplace=True)

# add donor and trimester key
adata_pb.obs['donor'] = [x.split('_')[1] for x in adata_pb.obs_names]
obs = adata_pb.obs.reset_index().merge(donor_trimester_df,how='left',on='donor').set_index(0)
obs.index.name = 'index'
adata_pb.obs = obs
adata_pb.obs['trimester'] = adata_pb.obs['trimester'].astype('category')
adata_pb.obs['trimester'].cat.reorder_categories(['firsttrimester','secondtrimester','adult'],inplace=True)


print(adata_pb.obs['stage'].value_counts())
print(adata_pb)
```

Example 34 (unknown):
```unknown
dropped groups: ['foetal_C82', 'foetal_Hst36']
foetal    8
adult     6
Name: stage, dtype: int64
AnnData object with n_obs × n_vars = 14 × 30749
    obs: 'stage', 'donor', 'trimester'
```

Example 35 (python):
```python
/tmp/ipykernel_257905/3068665136.py:27: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
  adata_pb.obs['stage'].cat.reorder_categories(['foetal','adult'],inplace=True)
/tmp/ipykernel_257905/3068665136.py:35: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
  adata_pb.obs['trimester'].cat.reorder_categories(['firsttrimester','secondtrimester','adult'],inplace=True)
```

Example 36 (python):
```python
# rank test
logFC_threshold_FvsA = 1
group_mean_threshold = 0.1

res_foetal_vs_adult = utils.rank_genes_groups_with_mean(adata_pb,
                                 groupby = 'stage',
                                  groups = ['foetal'],
                                  reference = 'adult',
                                  method = 'wilcoxon',
                                  pval_cutoff = 0.1,
                                        log2fc_min=logFC_threshold_FvsA
                                 )

sns.scatterplot(x=res_foetal_vs_adult['logfoldchanges'],y=res_foetal_vs_adult['group_mean'])
plt.show()
```

Example 37 (python):
```python
res_foetal_vs_adult = res_foetal_vs_adult[res_foetal_vs_adult['group_mean']>group_mean_threshold]
degs_foetal_vs_adult = list(res_foetal_vs_adult['names'])
print(len(degs_foetal_vs_adult))
display(res_foetal_vs_adult.head())
```

Example 38 (unknown):
```unknown
res_foetal_vs_adult[res_foetal_vs_adult['names']=='CACNA1D']
```

Example 39 (unknown):
```unknown
len(degs_foetal_vs_adult)
```

Example 40 (python):
```python
# get expression proportions
exp_mat = utils.get_exp_proportion(adata,groupby='fine_grain', use_rep='X')
exp_genes = list(exp_mat.index[exp_mat[cell_of_intest]>exp_prop_thresh])
len(exp_genes)
```

Example 41 (unknown):
```unknown
'ISL1' in exp_genes
```

Example 42 (python):
```python
adata_peak = sc.read_h5ad(path_to_adata_peak_tfidf)
adata_peak
```

Example 43 (unknown):
```unknown
AnnData object with n_obs × n_vars = 167022 × 508040
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels'
    var: 'seqnames', 'start', 'end', 'width', 'strand', 'score', 'replicateScoreQuantile', 'groupScoreQuantile', 'Reproducibility', 'GroupReplicate', 'distToGeneStart', 'nearestGene', 'peakType', 'distToTSS', 'nearestTSS', 'GC', 'idx', 'N'
```

Example 44 (python):
```python
# get accessible proportions
access_mat = utils.get_exp_proportion(adata_peak,groupby='fine_grain', use_rep='X')
```

Example 45 (unknown):
```unknown
access_peaks = list(access_mat.index[access_mat[cell_of_intest]>access_prop_thresh])
len(access_peaks)
```

Example 46 (python):
```python
# add columns to scenicplus metadata
# scenic_meta['Gene_isTF'] = scenic_meta['Gene'].isin(list(set(scenic_meta['TF'])))
scenic_meta['TF_isExp'] = scenic_meta['TF'].isin(exp_genes)
scenic_meta['TF_isSignature'] = (scenic_meta['TF'].isin(sig_genes))&(scenic_meta['TF'].isin(exp_genes))
scenic_meta['Region_isAccess'] = scenic_meta['Region'].isin(access_peaks)
scenic_meta['Region_isSignature'] = (scenic_meta['Region'].isin(sig_peaks))&(scenic_meta['Region'].isin(access_peaks))
scenic_meta.head()
```

Example 47 (python):
```python
# save the modified metadata
scenic_meta.to_csv(os.path.join(table_dir,'csvs/scenicplus_eRegulon_metadata_CMs_SANPC-highlighted.csv'))
```

Example 48 (python):
```python
target_genes = sig_genes.copy() # all the signature genes
# removing TFs from the list
```

Example 49 (python):
```python
# select interactions

# select interaction which target "target_genes"
scenic_meta_sel = scenic_meta[(scenic_meta['Gene'].isin(target_genes))&\
                               scenic_meta['TF_isExp'] &\
                               scenic_meta['Region_isAccess'] &\
                              (scenic_meta['TF_isSignature']|scenic_meta['Region_isSignature'])
                             ]

# get TFs and target genes in the selected interaction
direct_TFs = list(set(scenic_meta_sel['TF']))
targets = list(set(scenic_meta_sel['Gene']))
# remove TFs already in target genes
TFs_sel = list(set(direct_TFs) - set(targets))

print(len(TFs_sel))
print(f'direct TFs: {direct_TFs}')
print('')

# select higher level interactions 
print('Start looking for higher level TFs')
higher_level_TFs = []
while len(TFs_sel)>0:
    scenic_meta_sel2 = scenic_meta[(scenic_meta['Gene'].isin(TFs_sel))&\
                               scenic_meta['TF_isExp'] &\
                               scenic_meta['Region_isAccess'] &\
                              (scenic_meta['TF_isSignature']|scenic_meta['Region_isSignature'])
                                  ]
    higher_level_TFs = higher_level_TFs + list(set(scenic_meta_sel2['TF']))
    if len(scenic_meta_sel2)>0:
        # concatenate
        scenic_meta_sel = pd.concat([scenic_meta_sel,scenic_meta_sel2])
        scenic_meta_sel = scenic_meta_sel.drop_duplicates()
        # get TFs and target genes in the selected interaction
        TFs_sel = list(set(scenic_meta_sel['TF']))
        targets = list(set(scenic_meta_sel['Gene']))
        # remove TFs already in target genes
        TFs_sel = list(set(TFs_sel) - set(targets))
        print(f'{len(TFs_sel)}: {TFs_sel}')
    else:
        print('no more higher level TFs')
        TFs_sel = []

scenic_meta_sel['direct_TF'] = scenic_meta_sel['TF'].isin(direct_TFs)
higher_level_TFs = list(set(higher_level_TFs)-set(direct_TFs))

# reindex
scenic_meta_sel.index = range(len(scenic_meta_sel))

print("")
print(f'Number of selected interactions: {len(scenic_meta_sel)}')
print(f"Number of TFs: {len(set(scenic_meta_sel['TF']))}")
```

Example 50 (unknown):
```unknown
103
direct TFs: ['YY1', 'MITF', 'NFIC', 'TBX5', 'TCF4', 'ZNF519', 'NCOA3', 'SP4', 'TBX18', 'MBNL2', 'RFX3', 'BACH2', 'MECP2', 'NR3C1', 'USF2', 'SP3', 'NR2F1', 'NR4A3', 'ELF2', 'GLIS3', 'BHLHE40', 'MEF2A', 'ESRRG', 'POU6F2', 'KAT7', 'GTF2I', 'ZNF536', 'RARA', 'PBX1', 'ATF1', 'TCF12', 'TBX3', 'MEF2D', 'E2F3', 'CTCF', 'PKNOX2', 'ZNF33B', 'ZNF529', 'ZNF260', 'JUND', 'ZNF148', 'ZNF83', 'ZBTB20', 'MAX', 'TFCP2', 'GABPA', 'POU2F1', 'TFDP1', 'TGIF2', 'ZNF644', 'CREB3L2', 'NFIA', 'THRB', 'SHOX2', 'ZNF100', 'GTF2IRD1', 'JUN', 'FOS', 'TBX20', 'IRF2', 'PHF21A', 'CREB1', 'ELF1', 'SATB1', 'TCF7L1', 'ZNF382', 'MEF2C', 'ZNF248', 'NFIB', 'SMAD9', 'HMGA2', 'CLOCK', 'ZNF138', 'GATA4', 'CREM', 'ZNF41', 'NR3C2', 'SMAD4', 'KLF3', 'PPARA', 'ZNF704', 'CREB5', 'SMAD3', 'GATA6', 'MEIS2', 'PKNOX1', 'NFE2L2', 'BNC2', 'NR4A1', 'FOXO1', 'LCORL', 'ISL1', 'PURA', 'KLF12', 'TRPS1', 'NCOA1', 'TCF7L2', 'MEIS1', 'NR6A1', 'SMAD5', 'VEZF1', 'E2F8', 'ESRRB', 'PBX3', 'CUX1', 'ZNF680', 'RFX2', 'MAZ', 'ZNF283', 'SP1', 'HAND2', 'SREBF2', 'ZNF337', 'KLF6', 'NR2F2', 'RXRA', 'TEAD1', 'ZNF611', 'ETV6', 'ZNF141', 'KLF13', 'TEAD3', 'RARB']

Start looking for higher level TFs
79: ['YY1', 'TCF4', 'ZNF519', 'NCOA3', 'SP4', 'RFX3', 'BACH2', 'MECP2', 'NR3C1', 'USF2', 'SP3', 'NR4A3', 'ELF2', 'BHLHE40', 'MEF2A', 'KAT7', 'RARA', 'ATF1', 'ZNF654', 'TCF12', 'MEF2D', 'E2F3', 'CTCF', 'ZNF33B', 'ZNF529', 'ZNF260', 'JUND', 'ZNF148', 'MAX', 'TFCP2', 'TFDP1', 'POU2F1', 'TGIF2', 'ZNF644', 'ZNF100', 'GTF2IRD1', 'JUN', 'IRF2', 'PHF21A', 'CREB1', 'ELF1', 'SATB1', 'ZNF382', 'ZNF248', 'NFIB', 'CLOCK', 'ZNF138', 'CREM', 'ZNF41', 'NR3C2', 'SMAD4', 'KLF3', 'PPARA', 'ZNF704', 'GATA6', 'PKNOX1', 'NFE2L2', 'LCORL', 'PURA', 'NCOA1', 'TCF7L2', 'NR6A1', 'SMAD5', 'VEZF1', 'E2F8', 'ESRRB', 'PBX3', 'ZNF680', 'RFX2', 'ZNF283', 'MAZ', 'SP1', 'SREBF2', 'ZNF337', 'RXRA', 'TEAD1', 'ZNF141', 'KLF13', 'TEAD3']
no more higher level TFs

Number of selected interactions: 3221
Number of TFs: 124
```

Example 51 (python):
```python
# get TF-TG dictionary
network = {}
for tf in set(scenic_meta_sel['TF']):
    df_sub = scenic_meta_sel[scenic_meta_sel['TF']==tf]
    network[tf] = list(set(df_sub['Gene']))
print(len(network.keys()))
```

Example 52 (unknown):
```unknown
# signature genes which are not included in the interaction
genes_not_incl = set(sig_genes) - set(scenic_meta_sel['Gene'])
genes_not_incl
```

Example 53 (unknown):
```unknown
{'AC004540.1',
 'AC004594.1',
 'AC004784.1',
 'AC005009.2',
 'AC005699.1',
 'AC006148.1',
 'AC006230.1',
 'AC007319.1',
 'AC007364.1',
 'AC008124.1',
 'AC008591.1',
 'AC009264.1',
 'AC010201.3',
 'AC010478.1',
 'AC012078.2',
 'AC012404.1',
 'AC012409.2',
 'AC012459.1',
 'AC015631.1',
 'AC015923.1',
 'AC020687.1',
 'AC020909.2',
 'AC025159.1',
 'AC040168.1',
 'AC053527.2',
 'AC068587.4',
 'AC069228.1',
 'AC073050.1',
 'AC073389.2',
 'AC079142.1',
 'AC090572.2',
 'AC090578.1',
 'AC092353.1',
 'AC092353.2',
 'AC092691.1',
 'AC092691.3',
 'AC100803.4',
 'AC104211.1',
 'AC104596.1',
 'AC112487.1',
 'AC117464.1',
 'AC119674.1',
 'AC139718.2',
 'AC245297.3',
 'ADAM22',
 'ADAMTS3',
 'ADARB2',
 'ADORA2A-AS1',
 'AF064860.1',
 'AGBL3',
 'AKT3',
 'AL022068.1',
 'AL050309.1',
 'AL078600.1',
 'AL136456.1',
 'AL139294.1',
 'AL513166.1',
 'AL590006.1',
 'AL596087.2',
 'ANKAR',
 'AP001347.1',
 'AP005262.2',
 'AR',
 'ARHGEF7',
 'ATF7IP2',
 'AZIN1-AS1',
 'BX284613.2',
 'C2CD6',
 'C3orf70',
 'C7',
 'CALN1',
 'CASC15',
 'CFAP221',
 'CNBD1',
 'CNTNAP3B',
 'COL6A3',
 'COL6A6',
 'CRYM',
 'DACH2',
 'DBN1',
 'DCN',
 'DGKB',
 'DOCK3',
 'DPH6-DT',
 'ENTPD1-AS1',
 'ERMARD',
 'ERVMER61-1',
 'FAM66C',
 'FAM66D',
 'FBLN1',
 'FN1',
 'FRG1-DT',
 'FSIP2',
 'FSIP2-AS1',
 'FUT8',
 'GDNF-AS1',
 'GEM',
 'GPR37',
 'GRB14',
 'GRIK1',
 'GRM5',
 'GRXCR2',
 'GSTA4',
 'GSTO1',
 'HBZ',
 'HOTAIRM1',
 'HS6ST3',
 'HULC',
 'INTS6L',
 'KHDRBS2',
 'KIAA0319',
 'KLHL4',
 'LINC00648',
 'LINC00886',
 'LINC00937',
 'LINC00970',
 'LINC01060',
 'LINC01117',
 'LINC01122',
 'LINC01170',
 'LINC01184',
 'LINC01237',
 'LINC01266',
 'LINC01355',
 'LINC01619',
 'LINC01880',
 'LINC02147',
 'LINC02197',
 'LINC02208',
 'LINC02268',
 'LINC02798',
 'LRCH2',
 'LTBP4',
 'LUM',
 'MAGI2-AS3',
 'MCPH1-AS1',
 'MDGA2',
 'MEF2C-AS1',
 'MIPOL1',
 'MIR100HG',
 'MIR4300HG',
 'MMP16',
 'MRPS30-DT',
 'MTCP1',
 'MYCBP2-AS1',
 'MYLK-AS1',
 'NCAM2',
 'NLGN4X',
 'NLGN4Y',
 'NME4',
 'NOMO1',
 'NOVA1',
 'NPTN',
 'NR2F1-AS1',
 'NR2F2-AS1',
 'OGN',
 'PCLO',
 'PLSCR1',
 'POSTN',
 'PRKD1',
 'PRKG1-AS1',
 'PTCHD1-AS',
 'PTK2B',
 'PTPRE',
 'PURG',
 'RAB31',
 'RBMS3',
 'RBMS3-AS3',
 'RBP1',
 'RCAN1',
 'RFX3-AS1',
 'RGS7BP',
 'RIPK2',
 'RMDN2',
 'RPS6KA5',
 'RPS6KA6',
 'SERPINI1',
 'SESTD1',
 'SH3BGRL',
 'SLC25A21',
 'SLC2A3',
 'SLC38A6',
 'SLC7A2',
 'SPARCL1',
 'SRGAP2B',
 'ST6GALNAC5',
 'ST8SIA4',
 'STS',
 'SYNPR',
 'TAPT1-AS1',
 'TBX5-AS1',
 'TENM3-AS1',
 'TFPI',
 'TMEM232',
 'TMEM72-AS1',
 'TRPC5',
 'TSTD3',
 'TTLL7',
 'UBA6-AS1',
 'UGT2B4',
 'UNC79',
 'UPRT',
 'USP2-AS1',
 'UTY',
 'VEGFC',
 'VLDLR-AS1',
 'VSIG10',
 'Z85996.3',
 'ZDHHC15',
 'ZNF25',
 'ZNF483',
 'ZRANB2-AS2'}
```

Example 54 (python):
```python
# add metrics
scenic_meta_sel['TF_n_linked_region'] = np.nan
scenic_meta_sel['TF_n_target_genes'] = np.nan
scenic_meta_sel['TF_n_target_sig_regions'] = np.nan

for tf in set(scenic_meta_sel['TF']):
    tf_ind = scenic_meta_sel[scenic_meta_sel['TF']==tf].index
    targets = list(set(scenic_meta_sel.loc[tf_ind,'Gene']))
    scenic_meta_sel.loc[tf_ind,'TF_n_target_genes'] = len(targets)
    regions = list(set(scenic_meta_sel.loc[tf_ind,'Region']))
    regions = list(set(regions).intersection(sig_peaks))
    scenic_meta_sel.loc[tf_ind,'TF_n_target_sig_regions'] = len(regions)
    if tf in list(set(scenic_meta_sel['Gene'])):
        gene_ind = scenic_meta_sel[scenic_meta_sel['Gene']==tf].index
        n_region = len(set(scenic_meta_sel.loc[gene_ind,'Region']))
        scenic_meta_sel.loc[tf_ind,'TF_n_linked_region'] = n_region
    else:
        scenic_meta_sel.loc[tf_ind,'TF_n_linked_region'] = 0
        
scenic_meta_sel.head(3)
```

Example 55 (unknown):
```unknown
scenic_meta_sel_TF_metrices = scenic_meta_sel[['TF','direct_TF','TF_n_target_genes','TF_n_target_sig_regions','TF_n_linked_region']]
# df = df[df['direct_TF']]
scenic_meta_sel_TF_metrices.drop_duplicates(inplace=True)
scenic_meta_sel_TF_metrices.set_index('TF',inplace=True)
scenic_meta_sel_TF_metrices.sort_values('TF_n_target_genes',ascending=False)[:30]
```

Example 56 (python):
```python
/tmp/ipykernel_257905/2002270316.py:3: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  scenic_meta_sel_TF_metrices.drop_duplicates(inplace=True)
```

Example 57 (python):
```python
# select interactions those targets are TFs
scenic_meta_sel_TF = scenic_meta_sel[scenic_meta_sel['Gene'].isin(list(set(scenic_meta_sel['TF'])))]

network_TF = {}
for tf in set(scenic_meta_sel['TF']): # for all the TFs in the network
    if tf in set(scenic_meta_sel_TF['TF']): # whether the TF has TF target
        df_sub = scenic_meta_sel_TF[scenic_meta_sel_TF['TF']==tf]
        network_TF[tf] = list(set(df_sub['Gene']))
    # else:
    #    network_TF[tf] = []
print(len(network_TF.keys()))
```

Example 58 (unknown):
```unknown
centrality_type='eigenvector'
centrality_dict = regulon_plot.get_centrality(network_TF,
                   directed=True,
                   centrality_type=centrality_type, # or 'degree','in_dgree','out_degree'
                   out_df=False # if False output dictionary
                  )
centrality_dict
```

Example 59 (unknown):
```unknown
{'MITF': 0.049283207312519395,
 'THRB': 0.3258743088735697,
 'TBX5': 0.1576876663350341,
 'MBNL2': 0.07520027692231576,
 'NR2F1': 0.1576876663350341,
 'SMAD3': 0.036407046988683106,
 'ZNF536': 0.11579080941535508,
 'ETV6': 0.16725068440370758,
 'TCF7L1': 0.11413507214995011,
 'PKNOX2': 0.24083340658969826,
 'SHOX2': 0.5526719860295682,
 'CUX1': 0.036407046988683106,
 'FOXO1': 0.04689702436756745,
 'NR2F2': 0.1576876663350341,
 'HAND2': 0.07281409397736383,
 'MEF2C': 0.16801931261233685,
 'TCF4': 2.393778722528883e-15,
 'MEIS2': 0.016832573884945252,
 'SP4': 2.393778722528883e-15,
 'TBX3': 0.14704164566243016,
 'TBX18': 0.28557488505513406,
 'CREB3L2': 0.1940947133237148,
 'NFIC': 0.07281409397736383,
 'GLIS3': 0.04543420423291107,
 'TBX20': 0.010489977378886729,
 'CREB5': 0.08572932912523773,
 'BHLHE40': 2.393778722528883e-15,
 'MEF2A': 2.393778722528883e-15,
 'PBX1': 0.02212373949323264,
 'ESRRG': 0.03394955944879274,
 'POU6F2': 0.09872897151884444,
 'ISL1': 0.2595887016636425,
 'ZNF654': 2.393778722528883e-15,
 'GATA4': 0.04481257611127607,
 'BNC2': 0.21245384340382825,
 'TRPS1': 0.11643694886151107,
 'ZNF529': 2.393778722528883e-15,
 'ZNF148': 2.393778722528883e-15,
 'MAX': 2.393778722528883e-15,
 'NR4A1': 0.12005192785758158,
 'NFIA': 0.0958234267017418,
 'SMAD9': 0.02876471253689032,
 'ZBTB20': 0.12144208013029432,
 'JUN': 2.393778722528883e-15,
 'FOS': 1.255029458877223e-09,
 'KLF6': 1.255029458877223e-09,
 'PHF21A': 2.393778722528883e-15,
 'GTF2I': 0.03879322993363506,
 'MEIS1': 0.10429147156003983,
 'HMGA2': 0.03879322993363506,
 'GABPA': 0.03879322993363506,
 'RARB': 0.06567633367767987,
 'NFIB': 2.393778722528883e-15,
 'ZNF41': 2.393778722528883e-15,
 'GATA6': 2.393778722528883e-15,
 'KLF12': 4.787557445057766e-14,
 'TCF7L2': 2.393778722528883e-15,
 'ZNF611': 0.024078988891304342,
 'PBX3': 2.393778722528883e-15,
 'ZNF83': 4.787557445057766e-14,
 'TEAD1': 2.393778722528883e-15,
 'KLF13': 2.393778722528883e-15,
 'TEAD3': 2.393778722528883e-15}
```

Example 60 (python):
```python
# add to dataframe and save
scenic_meta_sel_TF_metrices.loc[centrality_dict.keys(),f'centaliry_{centrality_type}'] = list(centrality_dict.values())
scenic_meta_sel_TF_metrices[f'centaliry_{centrality_type}'].fillna(0,inplace=True)
print(len(scenic_meta_sel_TF_metrices))
display(scenic_meta_sel_TF_metrices)

# save
scenic_meta_sel_TF_metrices.to_csv('/nfs/team205/heart/Vitro/TF_selection/foetal_SANPcell_vs_workingCM/TF_selection_df_target-SANPC-signatures.csv')
```

Example 61 (python):
```python
/tmp/ipykernel_257905/3865649271.py:2: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  scenic_meta_sel_TF_metrices.loc[centrality_dict.keys(),f'centaliry_{centrality_type}'] = list(centrality_dict.values())
/tmp/ipykernel_257905/3865649271.py:3: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  scenic_meta_sel_TF_metrices[f'centaliry_{centrality_type}'].fillna(0,inplace=True)
```

Example 62 (unknown):
```unknown
# TF-TG network
utils.save_pkl(network,
               '/nfs/team205/heart/Vitro/TF_selection/foetal_SANPcell_vs_workingCM/TF-TG-network_target-SANPC-signatures.pkl')

# TF-TF network
utils.save_pkl(network_TF,
               '/nfs/team205/heart/Vitro/TF_selection/foetal_SANPcell_vs_workingCM/TF-TF-network_target-SANPC-signatures.pkl')
```

Example 63 (python):
```python
import networkx as nx
```

Example 64 (unknown):
```unknown
# prepare parameters
size_scale = 7000
base_size = 200

node_size_map = {}
for n in network.keys():
    if n in centrality_dict.keys():
        node_size_map[n]=int(centrality_dict[n]*size_scale + base_size)
    else: # means no connection with other TFs
        node_size_map[n]=base_size
        

node_size_map
```

Example 65 (unknown):
```unknown
{'YY1': 200,
 'MITF': 544,
 'NFIC': 709,
 'TBX5': 1303,
 'TCF4': 200,
 'ZNF519': 200,
 'NCOA3': 200,
 'SP4': 200,
 'TBX18': 2199,
 'MBNL2': 726,
 'RFX3': 200,
 'BACH2': 200,
 'MECP2': 200,
 'NR3C1': 200,
 'USF2': 200,
 'SP3': 200,
 'NR2F1': 1303,
 'NR4A3': 200,
 'ELF2': 200,
 'GLIS3': 518,
 'BHLHE40': 200,
 'MEF2A': 200,
 'ESRRG': 437,
 'POU6F2': 891,
 'KAT7': 200,
 'GTF2I': 471,
 'ZNF536': 1010,
 'RARA': 200,
 'PBX1': 354,
 'ATF1': 200,
 'ZNF654': 200,
 'TCF12': 200,
 'TBX3': 1229,
 'MEF2D': 200,
 'E2F3': 200,
 'CTCF': 200,
 'PKNOX2': 1885,
 'ZNF33B': 200,
 'ZNF529': 200,
 'ZNF260': 200,
 'JUND': 200,
 'ZNF148': 200,
 'ZNF83': 200,
 'ZBTB20': 1050,
 'MAX': 200,
 'TFCP2': 200,
 'GABPA': 471,
 'POU2F1': 200,
 'TFDP1': 200,
 'TGIF2': 200,
 'ZNF644': 200,
 'CREB3L2': 1558,
 'NFIA': 870,
 'THRB': 2481,
 'SHOX2': 4068,
 'ZNF100': 200,
 'GTF2IRD1': 200,
 'JUN': 200,
 'FOS': 200,
 'TBX20': 273,
 'IRF2': 200,
 'PHF21A': 200,
 'CREB1': 200,
 'ELF1': 200,
 'SATB1': 200,
 'TCF7L1': 998,
 'ZNF382': 200,
 'MEF2C': 1376,
 'ZNF248': 200,
 'NFIB': 200,
 'SMAD9': 401,
 'HMGA2': 471,
 'CLOCK': 200,
 'ZNF138': 200,
 'GATA4': 513,
 'CREM': 200,
 'ZNF41': 200,
 'NR3C2': 200,
 'SMAD4': 200,
 'KLF3': 200,
 'PPARA': 200,
 'ZNF704': 200,
 'CREB5': 800,
 'SMAD3': 454,
 'GATA6': 200,
 'MEIS2': 317,
 'PKNOX1': 200,
 'NFE2L2': 200,
 'BNC2': 1687,
 'NR4A1': 1040,
 'FOXO1': 528,
 'LCORL': 200,
 'ISL1': 2017,
 'PURA': 200,
 'KLF12': 200,
 'TRPS1': 1015,
 'NCOA1': 200,
 'TCF7L2': 200,
 'MEIS1': 930,
 'NR6A1': 200,
 'SMAD5': 200,
 'VEZF1': 200,
 'E2F8': 200,
 'ESRRB': 200,
 'PBX3': 200,
 'CUX1': 454,
 'ZNF680': 200,
 'RFX2': 200,
 'MAZ': 200,
 'ZNF283': 200,
 'SP1': 200,
 'HAND2': 709,
 'SREBF2': 200,
 'ZNF337': 200,
 'KLF6': 200,
 'NR2F2': 1303,
 'RXRA': 200,
 'TEAD1': 200,
 'ZNF611': 368,
 'ETV6': 1370,
 'ZNF141': 200,
 'KLF13': 200,
 'TEAD3': 200,
 'RARB': 659}
```

Example 66 (unknown):
```unknown
regulon_plot.draw_networkx(
    network=network_TF,
    nodes_color_base='white',
    nodes_color_map=None, # list
    nodes_size_base=400,
    nodes_size_map=node_size_map,
     spring_layout_k=1.8,
    spring_layout_scale=1,
    figsize =(7,7),
    title='SANPC TF network',
    random_seed=1,
    numpy_random_seed=100,
    pos_seed=5,
    savefig=f"{figdir}/SANPC_TF-network.pdf",
    save_dpi=300
)
```

---

## Import modules¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig5_Trisomy21/no04_euploid-trisomy21_scVI.html

**Contents:**
- Import modules¶
- Read in anndata¶
- Run scVI, categorical covariates to correct: donor+region¶
- Run scVI, categorical covariates to correct: only region¶
- Run scVI, categorical covariates to correct: only donor¶

**Examples:**

Example 1 (python):
```python
# this is for the jupyter notebook at JupyterLab
# to make sys.path between the terminal and Jupyter same
import sys

print(sys.path) # especially the path to site-packages are different... so thats why couldn't find correct package...

# sys.path from terminal at scvi0190_env
# obtained by excuting `python -c "import sys; print('\n'.join(sys.path))"`
sys.path = [
    "/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python39.zip",
    "/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9",
    "/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/lib-dynload",
    "/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages"
]

print(sys.path)
```

Example 2 (unknown):
```unknown
['/nfs/team205/kk18/notebooks/Foetal/Trisomy21/RNA', '/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python39.zip', '/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9', '/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/lib-dynload', '', '/nfs/users/nfs_k/kk18/.local/lib/python3.9/site-packages', '/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages']
['/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python39.zip', '/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9', '/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/lib-dynload', '/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages']
```

Example 3 (python):
```python
# import warnings
# warnings.simplefilter("ignore", UserWarning)

import anndata
import matplotlib.pyplot as plt
import seaborn as sns

import numpy as np
import numpy.random as random
import pandas as pd
import scanpy as sc

import scvi
```

Example 4 (python):
```python
Global seed set to 0
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/flax/core/frozen_dict.py:169: FutureWarning: jax.tree_util.register_keypaths is deprecated, and will be removed in a future release. Please use `register_pytree_with_keys()` instead.
  jax.tree_util.register_keypaths(
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/pytorch_lightning/utilities/warnings.py:53: LightningDeprecationWarning: pytorch_lightning.utilities.warnings.rank_zero_deprecation has been deprecated in v1.6 and will be removed in v1.8. Use the equivalent function from the pytorch_lightning.utilities.rank_zero module instead.
  new_rank_zero_deprecation(
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/pytorch_lightning/utilities/warnings.py:58: LightningDeprecationWarning: The `pytorch_lightning.loggers.base.rank_zero_experiment` is deprecated in v1.7 and will be removed in v1.9. Please use `pytorch_lightning.loggers.logger.rank_zero_experiment` instead.
  return new_rank_zero_deprecation(*args, **kwargs)
```

Example 5 (python):
```python
import gc
import torch
```

Example 6 (python):
```python
import session_info
session_info.show()
```

Example 7 (python):
```python
-----
anndata             0.8.0
matplotlib          3.7.1
numpy               1.23.5
pandas              1.5.3
scanpy              1.9.3
scvi                0.19.0
seaborn             0.12.2
session_info        1.0.0
torch               1.13.1+cu117
-----
```

Example 8 (python):
```python
PIL                 9.4.0
absl                NA
asttokens           NA
attr                22.2.0
backcall            0.2.0
certifi             2022.12.07
cffi                1.15.1
charset_normalizer  3.1.0
chex                0.1.6
comm                0.1.2
contextlib2         NA
cycler              0.10.0
cython_runtime      NA
dateutil            2.8.2
debugpy             1.6.6
decorator           5.1.1
defusedxml          0.7.1
deprecate           0.3.2
docrep              0.3.2
etils               1.1.0
executing           1.2.0
flax                0.6.7
fsspec              2023.3.0
google              NA
h5py                3.8.0
idna                3.4
igraph              0.10.4
importlib_metadata  NA
importlib_resources NA
ipykernel           6.21.3
ipython_genutils    0.2.0
ipywidgets          8.0.4
jax                 0.4.10
jaxlib              0.4.10
jedi                0.18.2
joblib              1.2.0
kiwisolver          1.4.4
leidenalg           0.9.1
llvmlite            0.39.1
ml_collections      NA
ml_dtypes           0.2.0
mpl_toolkits        NA
msgpack             1.0.5
mudata              0.2.1
multipledispatch    0.6.0
natsort             8.3.1
numba               0.56.4
numpyro             0.11.0
nvidia              NA
opt_einsum          v3.3.0
optax               0.1.4
packaging           23.0
parso               0.8.3
pexpect             4.8.0
pickleshare         0.7.5
pkg_resources       NA
platformdirs        3.1.1
prompt_toolkit      3.0.38
psutil              5.9.4
ptyprocess          0.7.0
pure_eval           0.2.2
pydev_ipython       NA
pydevconsole        NA
pydevd              2.9.5
pydevd_file_utils   NA
pydevd_plugins      NA
pydevd_tracing      NA
pygments            2.14.0
pyparsing           3.0.9
pyro                1.8.4
pytorch_lightning   1.7.7
pytz                2022.7.1
requests            2.28.2
rich                NA
scipy               1.10.1
setuptools          65.6.3
six                 1.16.0
sklearn             1.2.2
stack_data          0.6.2
statsmodels         0.13.5
tensorboard         2.12.0
texttable           1.6.7
threadpoolctl       3.1.0
toolz               0.12.0
torchmetrics        0.11.4
torchvision         0.14.1+cu117
tornado             6.2
tqdm                4.65.0
traitlets           5.9.0
tree                0.1.8
typing_extensions   NA
urllib3             1.26.15
wcwidth             0.2.6
yaml                6.0
zipp                NA
zmq                 25.0.1
zoneinfo            NA
```

Example 9 (unknown):
```unknown
-----
IPython             8.11.0
jupyter_client      8.0.3
jupyter_core        5.2.0
notebook            6.5.3
-----
Python 3.9.16 (main, Mar  8 2023, 14:00:05) [GCC 11.2.0]
Linux-5.15.0-117-generic-x86_64-with-glibc2.35
-----
Session information updated at 2024-08-15 15:09
```

Example 10 (python):
```python
sc.settings.verbosity = 3             # verbosity: errors (0), warnings (1), info (2), hints (3)
sc.settings.set_figure_params(dpi=80, facecolor='white', color_map = 'RdPu',)
```

Example 11 (python):
```python
# import own function
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 12 (python):
```python
import os
os.getcwd()
```

Example 13 (unknown):
```unknown
'/nfs/team205/kk18/notebooks/Foetal/Trisomy21/RNA'
```

Example 14 (python):
```python
adata = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/trisomy21/Euploid_T21Hearts_Aug2024_sel_subsampled-per-cellstate-donor.h5ad')
adata
```

Example 15 (unknown):
```unknown
AnnData object with n_obs × n_vars = 83477 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score', 'euploid_or_t21', 'donor_type', 'region_finest', 'flushed', 'scrublet_leiden', 'cluster_scrublet_score', 'leiden_scVI_0.1', 'leiden_scVI_0.2', 'leiden_scVI_0.3', 'leiden_scVI_0.5', 'leiden_scVI_0.7', 'leiden_scVI_1', 'celltypist_coarse', 'conf_score_coarse', 'cells_to_remove', 'celltypist_fine', 'conf_score_fine', 'coarse_grain_merged', 'fine_grain_merged'
    var: 'gene_id', 'gene_name_scRNA-0-0', 'feature_type-1', 'ambient_expression-0-1', 'ambient_expression-1-1', 'ambient_expression-10-1', 'ambient_expression-11-1', 'ambient_expression-12-1', 'ambient_expression-13-1', 'ambient_expression-14-1', 'ambient_expression-2-1', 'ambient_expression-3-1', 'ambient_expression-4-1', 'ambient_expression-5-1', 'ambient_expression-6-1', 'ambient_expression-7-1', 'ambient_expression-8-1', 'ambient_expression-9-1', 'mt-1', 'ribo-1', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1'
    obsm: 'X_scVI', 'X_umap', 'latent_gene_encoding'
```

Example 16 (python):
```python
# Setup anndata for scVI
bdata = adata.copy()
scvi.model.SCVI.setup_anndata(bdata, 
                              # layer="counts", 
                              batch_key="donor", # we want to correct for this 
                              categorical_covariate_keys=['region'],  # we want to correct for this
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo']) 

# with three n_layer conditions
n_layer_list = [1,2,3]

for n_layers in n_layer_list:
    print(f'##### {n_layers} #####')
    # setup
    vae = scvi.model.SCVI(bdata, n_hidden = 128, n_latent = 50, n_layers = n_layers, dispersion = 'gene-batch')
    vae.view_anndata_setup(bdata)
    # train
    vae.train(max_epochs=400)
    # save trained model
    vae.save(f"/nfs/team205/heart/anndata_objects/Foetal/scVI/models/Euploid_T21Hearts_Aug2024_sel_subsampled-per-cellstate-donor_correcting-donor-region_n-layers-{str(n_layers)}/",
             overwrite=True)
    # save latent space
    latent = sc.AnnData(vae.get_latent_representation(),
                    obs=bdata.obs.copy())
    # drop all the obs columns, for simplicity
    latent.obs = latent.obs.drop(columns=latent.obs.columns, axis=1)
    latent.write(f'/nfs/team205/heart/anndata_objects/Foetal/scVI/latent_variables/Euploid_T21Hearts_Aug2024_sel_subsampled-per-cellstate-donor_correcting-donor-region_n-layers-{str(n_layers)}.h5ad')
    # Convergency plot
    vae.history['elbo_train']
    x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
    plt.plot(x, vae.history['elbo_train'], label="train")
    plt.xlabel('epoch')
    plt.ylabel('elbo_train')
    plt.show()
    
    del vae
    # free up GPU
    gc.collect()
    torch.cuda.empty_cache()
    print('')
```

Example 17 (unknown):
```unknown
No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.)
```

Example 18 (python):
```python
# Setup anndata for scVI
bdata = adata.copy()
scvi.model.SCVI.setup_anndata(bdata, 
                              # layer="counts", 
                              batch_key="region", # we want to correct for this 
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo'])

# with three n_layer conditions
n_layer_list = [1,2,3]

for n_layers in n_layer_list:
    print(f'##### {n_layers} #####')
    # setup
    vae = scvi.model.SCVI(bdata, n_hidden = 128, n_latent = 50, n_layers = n_layers, dispersion = 'gene-batch')
    vae.view_anndata_setup(bdata)
    # train
    vae.train(max_epochs=400)
    # save trained model
    vae.save(f"/nfs/team205/heart/anndata_objects/Foetal/scVI/models/Euploid_T21Hearts_Aug2024_sel_subsampled-per-cellstate-donor_correcting-region_n-layers-{str(n_layers)}/",
             overwrite=True)
    # save latent space
    latent = sc.AnnData(vae.get_latent_representation(),
                    obs=bdata.obs.copy())
    # drop all the obs columns, for simplicity
    latent.obs = latent.obs.drop(columns=latent.obs.columns, axis=1)
    latent.write(f'/nfs/team205/heart/anndata_objects/Foetal/scVI/latent_variables/Euploid_T21Hearts_Aug2024_sel_subsampled-per-cellstate-donor_correcting-region_n-layers-{str(n_layers)}.h5ad')
    # Convergency plot
    vae.history['elbo_train']
    x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
    plt.plot(x, vae.history['elbo_train'], label="train")
    plt.xlabel('epoch')
    plt.ylabel('elbo_train')
    plt.show()
    
    del vae
    # free up GPU
    gc.collect()
    torch.cuda.empty_cache()
    print('')
```

Example 19 (python):
```python
# Setup anndata for scVI
bdata = adata.copy()
scvi.model.SCVI.setup_anndata(bdata, 
                              # layer="counts", 
                              batch_key="donor", # we want to correct for this 
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo'])

# with three n_layer conditions
n_layer_list = [1,2,3]

for n_layers in n_layer_list:
    print(f'##### {n_layers} #####')
    # setup
    vae = scvi.model.SCVI(bdata, n_hidden = 128, n_latent = 50, n_layers = n_layers, dispersion = 'gene-batch')
    vae.view_anndata_setup(bdata)
    # train
    vae.train(max_epochs=400)
    # save trained model
    vae.save(f"/nfs/team205/heart/anndata_objects/Foetal/scVI/models/Euploid_T21Hearts_Aug2024_sel_subsampled-per-cellstate-donor_correcting-donor_n-layers-{str(n_layers)}/",
             overwrite=True)
    # save latent space
    latent = sc.AnnData(vae.get_latent_representation(),
                    obs=bdata.obs.copy())
    # drop all the obs columns, for simplicity
    latent.obs = latent.obs.drop(columns=latent.obs.columns, axis=1)
    latent.write(f'/nfs/team205/heart/anndata_objects/Foetal/scVI/latent_variables/Euploid_T21Hearts_Aug2024_sel_subsampled-per-cellstate-donor_correcting-donor_n-layers-{str(n_layers)}.h5ad')
    # Convergency plot
    vae.history['elbo_train']
    x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
    plt.plot(x, vae.history['elbo_train'], label="train")
    plt.xlabel('epoch')
    plt.ylabel('elbo_train')
    plt.show()
    
    del vae
    # free up GPU
    gc.collect()
    torch.cuda.empty_cache()
    print('')
```

---

## Variables¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig3_SAN_niche/VisiumHD_CCSaxis_no1_annotate-landmark.html

**Contents:**
- Variables¶
- Read in data¶
- Plot¶
- Annotate¶
- Save¶

bin2cell output which has celltypist output as well

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os

import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 

import celltypist
```

Example 2 (python):
```python
# import session_info
# session_info.show()
```

Example 3 (python):
```python
sc.settings.set_figure_params(dpi=120)
```

Example 4 (python):
```python
adata_dir = '/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects'
sample_id = 'HEA_FOET14880396'
```

Example 5 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk/Aug2024'
table_dir = '/nfs/team205/heart/fetal_ms_figures/tables/Aug2024'
```

Example 6 (python):
```python
# read in 
adata = sc.read_h5ad(f'{adata_dir}/{sample_id}_b2c_cells_filtered_lognorm.h5ad')
adata
```

Example 7 (unknown):
```unknown
AnnData object with n_obs × n_vars = 129803 × 18047
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'fine_grain_4Jan2025', 'HistologicalAnnotation'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'HistologicalAnnotation_colors', 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'fine_grain_4Jan2025_colors', 'hvg', 'log1p', 'neighbors', 'pca', 'spatial', 'umap'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 8 (python):
```python
sc.pl.embedding(adata,basis='X_umap',color=["celltypist_mid_fin"],wspace=0.2,cmap='RdPu')
```

Example 9 (python):
```python
sc.pl.embedding(adata,basis='X_umap',color=['fine_grain_4Jan2025'],wspace=0.2,cmap='RdPu')
```

Example 10 (python):
```python
cells = [
    'SinoatrialNodePacemakerCellsHorn',
    'SinoatrialNodePacemakerCellsHead',
    'SinoatrialNodePacemakerCellsTail',
    # 'AtrioventricularNodePacemakerCells',
    # 'VentricularConductionSystem'
    
    # 'MacrophagesCX3CR1pos',
    # 'ParasympatheticNeurons'
]

celltype_key = 'fine_grain_4Jan2025'
adata.uns[f"{celltype_key}_colors"][list(adata.obs[celltype_key].cat.categories).index('SinoatrialNodePacemakerCellsHorn')]='tab:orange'
adata.uns[f"{celltype_key}_colors"][list(adata.obs[celltype_key].cat.categories).index('SinoatrialNodePacemakerCellsHead')]='tab:blue'
adata.uns[f"{celltype_key}_colors"][list(adata.obs[celltype_key].cat.categories).index('SinoatrialNodePacemakerCellsTail',)]='tab:green'
# adata.uns[f"{celltype_key}_colors"][list(adata.obs[celltype_key].cat.categories).index('AtrioventricularNodePacemakerCells')]='orange'
# adata.uns[f"{celltype_key}_colors"][list(adata.obs[celltype_key].cat.categories).index('VentricularConductionSystem')]='mediumpurple'

sc.pl.spatial(adata, color=celltype_key, groups=cells,
              img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer",
              size=10,legend_fontsize=30,spot_size=10,
                  frameon=False,) # save=f'_{c}_b2c',
```

Example 11 (python):
```python
cells = [
     'SinoatrialNodePacemakerCellsHorn',
    'SinoatrialNodePacemakerCellsHead',
    'SinoatrialNodePacemakerCellsTail',
    # 'AtrioventricularNodePacemakerCells',
    # 'VentricularConductionSystem'
]

mask = ((adata.obs['array_row'] >= 1700) & 
        (adata.obs['array_row'] <= 2300) & 
        (adata.obs['array_col'] >= 600) & 
        (adata.obs['array_col'] <= 1200)
       )
bdata = adata[mask]
n_celltype = len(bdata.obs[celltype_key].cat.categories)
bdata.uns[f'{celltype_key}_colors'] = list(np.repeat('#ffff00',n_celltype))

# update colors
for c in cells:
    # get color in adata
    ind = list(adata.obs[celltype_key].cat.categories).index(c)
    color = adata.uns[f'{celltype_key}_colors'][ind]
    # update color in bdata
    ind = list(bdata.obs[celltype_key].cat.categories).index(c)
    bdata.uns[f'{celltype_key}_colors'][ind] = color

sc.pl.spatial(bdata, color=celltype_key, groups=cells,
              img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer",
              size=10,legend_fontsize=30,spot_size=10,
                  frameon=False,) # save=f'_{c}_b2c',
```

Example 12 (python):
```python
# start
cells = [
     'SinoatrialNodePacemakerCellsHorn',
    #'SinoatrialNodePacemakerCellsHead',
    #'SinoatrialNodePacemakerCellsTail',
    # 'AtrioventricularNodePacemakerCells',
    # 'VentricularConductionSystem'
]

mask = ((adata.obs['array_row'] >= 2250) & 
        (adata.obs['array_row'] <= 2300) & 
        (adata.obs['array_col'] >= 700) & 
        (adata.obs['array_col'] <= 820)
       )
bdata = adata[mask]
n_celltype = len(bdata.obs[celltype_key].cat.categories)
bdata.uns[f'{celltype_key}_colors'] = list(np.repeat('#ffff00',n_celltype))

# update colors
for c in cells:
    # get color in adata
    ind = list(adata.obs[celltype_key].cat.categories).index(c)
    color = adata.uns[f'{celltype_key}_colors'][ind]
    # update color in bdata
    ind = list(bdata.obs[celltype_key].cat.categories).index(c)
    bdata.uns[f'{celltype_key}_colors'][ind] = color

sc.pl.spatial(bdata, color=celltype_key, groups=cells,
              img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer",
              size=10,legend_fontsize=30,spot_size=10,
                  frameon=False,) # save=f'_{c}_b2c',

# assign
start_cells = bdata.obs_names[bdata.obs[celltype_key]=='SinoatrialNodePacemakerCellsHorn']
print(len(start_cells))
```

Example 13 (python):
```python
# end
cells = [
    #  'SinoatrialNodePacemakerCellsHorn',
    # 'SinoatrialNodePacemakerCellsHead',
    'SinoatrialNodePacemakerCellsTail',
    # 'AtrioventricularNodePacemakerCells',
    # 'VentricularConductionSystem'
]

mask = ((adata.obs['array_row'] >= 1800) & 
        (adata.obs['array_row'] <= 2200) & 
        (adata.obs['array_col'] >= 800) & 
        (adata.obs['array_col'] <= 870)
       )
bdata = adata[mask]
n_celltype = len(bdata.obs[celltype_key].cat.categories)
bdata.uns[f'{celltype_key}_colors'] = list(np.repeat('#ffff00',n_celltype))

# update colors
for c in cells:
    # get color in adata
    ind = list(adata.obs[celltype_key].cat.categories).index(c)
    color = adata.uns[f'{celltype_key}_colors'][ind]
    # update color in bdata
    ind = list(bdata.obs[celltype_key].cat.categories).index(c)
    bdata.uns[f'{celltype_key}_colors'][ind] = color

sc.pl.spatial(bdata, color=celltype_key, groups=cells,
              img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer",
              size=10,legend_fontsize=30,spot_size=10,
                  frameon=False,) # save=f'_{c}_b2c',

# assign
end_cells = bdata.obs_names[bdata.obs[celltype_key]=='SinoatrialNodePacemakerCellsTail']
print(len(end_cells))
```

Example 14 (python):
```python
# plot landmarks
adata.obs['SANaxis_landmarks'] = 'nan'
adata.obs.loc[start_cells,'SANaxis_landmarks'] = 'start'
adata.obs.loc[end_cells,'SANaxis_landmarks'] = 'end'

mask = ((adata.obs['array_row'] >= 1700) & 
        (adata.obs['array_row'] <= 2300) & 
        (adata.obs['array_col'] >= 600) & 
        (adata.obs['array_col'] <= 1200)
       )
bdata = adata[mask]

sc.pl.spatial(bdata, color='SANaxis_landmarks', groups=['start','end'],
              img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer",
              size=10,legend_fontsize=30, spot_size=10,
                  frameon=False,) # save=f'_{c}_b2c',
```

Example 15 (unknown):
```unknown
... storing 'SANaxis_landmarks' as categorical
```

Example 16 (python):
```python
adata.write(f'{adata_dir}/{sample_id}_b2c_cells_filtered_lognorm.h5ad')
adata
```

Example 17 (unknown):
```unknown
... storing 'SANaxis_landmarks' as categorical
```

Example 18 (unknown):
```unknown
AnnData object with n_obs × n_vars = 129803 × 18047
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'fine_grain_4Jan2025', 'HistologicalAnnotation', 'SANaxis_landmarks'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'HistologicalAnnotation_colors', 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'fine_grain_4Jan2025_colors', 'hvg', 'log1p', 'neighbors', 'pca', 'spatial', 'umap'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

---

## Read in¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/Xenium/no2_integration_and_cell-type-annotation/no03-2_harmony_clustering.html

**Contents:**
- Read in¶
- Clustering¶
- Save¶

**Examples:**

Example 1 (python):
```python
import scanpy as sc
import numpy as np
import pandas as pd
import os
# import rapids_singlecell as rsc
from tqdm import tqdm

import matplotlib.pyplot as plt
import seaborn as sns
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
sc.settings.set_figure_params(dpi=120)
```

Example 4 (python):
```python
objects_dir = '/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium'
path_adata = f'{objects_dir}/C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat_5K_filtered_raw.h5ad'
adata = sc.read_h5ad(path_adata)
adata
```

Example 5 (unknown):
```unknown
latent_space = 'pca_harmony'
```

Example 6 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=['total_counts','tissue_block_id'], wspace=0.3, cmap='RdPu')
```

Example 7 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=['celltypist_mid_fin'], wspace=0.2, cmap='RdPu')
```

Example 8 (python):
```python
%%time
latent_space='pca_harmony'
leiden_keys=[]
for res in tqdm([0.2,0.3,0.4,0.5,0.8,1.0,1.5,2.0,3.0]):
    leiden_key = f"leiden_{res}"
    key_added=f"{latent_space}_{leiden_key}"
    sc.tl.leiden(adata,neighbors_key=latent_space,resolution=res,key_added=key_added)
    leiden_keys.append(key_added)
```

Example 9 (python):
```python
sc.pl.embedding(adata,basis=f"X_umap_{latent_space}",
                color=leiden_keys,wspace=0.3,ncols=5)
```

Example 10 (python):
```python
adata.write(path_adata)
adata
```

---

## Import modules¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig2_Overview_of_cellular_niches_and_TissueTypist/tissuetypist_assess_Xenium5000.html

**Contents:**
- Import modules¶
- Read in qeury adata¶
- Read in reference adata¶
- Harmonise cell type names¶
- Calculate cell type proportions per niche¶
  - Query¶
  - Reference¶
- Subset shared cell types¶

**Examples:**

Example 1 (python):
```python
import numpy as np
import pandas as pd
import scanpy as sc
import anndata
import os
import matplotlib.pyplot as plt
import seaborn as sns
import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 

sc.settings.verbosity = 1  # verbosity: errors (0), warnings (1), info (2), hints (3)
```

Example 2 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 3 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk/Aug2024'
table_dir = '/nfs/team205/heart/fetal_ms_figures/tables/Aug2024'
```

Example 4 (python):
```python
# read in 
celltype_col = 'fine_grain_4Jan2025'
path_to_adata = '/lustre/scratch126/cellgen/team205/kk18/xenium/objects/foetal/C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat_5K_filtered_raw.h5ad'
adata = sc.read_h5ad(path_to_adata)
print(adata.X.data[:5])
adata
```

Example 5 (unknown):
```unknown
[5. 1. 1. 2. 2.]
```

Example 6 (unknown):
```unknown
AnnData object with n_obs × n_vars = 362277 × 5001
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'fine_grain_8Dec2024', 'multi_celltypes_coarse', 'cluster_cellcharter', 'fine_grain_4Jan2025', 'sliding_window_assignment', 'globalX', 'globalY', 'predicted_labels_sliding_window_assignment', 'conf_score_sliding_window_assignment', 'window_col', 'window_row', 'tt_prediction', 'tt_prediction_mod'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1'
    uns: 'celltypist_mid_fin_colors', 'fine_grain_4Jan2025_colors', 'fine_grain_8Dec2024_colors', 'multi_celltypes_coarse_colors', 'pca', 'pca_harmony', 'spatial_neighbors', 'tissue_block_id_colors', 'umap'
    obsm: 'X_cellcharter', 'X_pca', 'X_pca_harmony', 'X_umap_pca', 'X_umap_pca_harmony', 'spatial'
    varm: 'PCs'
    obsp: 'pca_harmony_connectivities', 'pca_harmony_distances', 'spatial_connectivities', 'spatial_distances'
```

Example 7 (python):
```python
# update cell type name for plotting
df = pd.read_csv('/nfs/team205/heart/anndata_objects/Foetal/finegrain_name_mapping.csv')
celltype_mapping = df.set_index('Full_name')['Short_name_finalised'].to_dict()
adata.obs.replace({celltype_col:celltype_mapping},inplace=True)
```

Example 8 (python):
```python
/tmp/ipykernel_3792320/1210424873.py:4: FutureWarning: The behavior of Series.replace (and DataFrame.replace) with CategoricalDtype is deprecated. In a future version, replace will only be used for cases that preserve the categories. To change the categories, use ser.cat.rename_categories instead.
  adata.obs.replace({celltype_col:celltype_mapping},inplace=True)
```

Example 9 (python):
```python
adata_ref = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/visium_foetal_healthy-all_cell2loc-adata.h5ad')
adata_ref
```

Example 10 (unknown):
```unknown
AnnData object with n_obs × n_vars = 39223 × 58
    obs: 'array_row', 'array_col', 'sample', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'mt_frac', 'sangerID', 'Publication', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'status', 'cell2loc-map_group', 'JC_edit', 'sample_for_cell2loc', 'week', 'leiden', 'annotation_final', 'annotation_final_mod'
    uns: 'JC_edit_colors', 'annotation_colors', 'annotation_final_colors', 'cell2loc-map_group_colors', 'leiden', 'neighbors', 'region_cluster_colors', 'sample_for_cell2loc_colors', 'umap'
    obsm: 'MT', 'X_umap', 'means_cell_abundance_w_sf', 'q05_cell_abundance_w_sf', 'q05_cell_abundance_w_sf_ndarray', 'q95_cell_abundance_w_sf', 'spatial', 'stds_cell_abundance_w_sf'
    obsp: 'connectivities', 'distances'
```

Example 11 (python):
```python
# update cell type name
df = pd.read_csv('/nfs/team205/heart/anndata_objects/Foetal/finegrain_name_mapping.csv')
celltype_mapping = df.set_index('Full_name')['Short_name_finalised'].to_dict()
adata_ref.var_names = [celltype_mapping[x] for x in adata_ref.var_names]
```

Example 12 (python):
```python
celltype_mapping_2 = {
    'BMS4':'B',
    'CorAdvFB':'FB',
    'GVAdvFB':'FB',
    'MonoMPO':'Mono',
    'MyoFB':'FB',
    'MyoFb':'FB',
     'ProB':'B',
    'SubEpiFb':'FB',
    'vCMCL':'vCM',
    'vCMCR':'vCM',
    'vCMTL':'vCM',
    'vCMTR':'vCM',
    'FBCOL2A1':'FB',
    'GVSMCP':'GVSMC',
    'GVSMCT':'GVSMC',
    'SANPCHd':'SANPC',
     'SANPCHn':'SANPC',
     'SANPCT':'SANPC',
    'CorCapECN':'CorCapEC',
    'DCMat':'DC',
 'DCT1':'DC',
}
adata.obs[celltype_col] = [celltype_mapping_2[x] if x in celltype_mapping_2.keys() else x for x in adata.obs[celltype_col]]
adata_ref.var_names = [celltype_mapping_2[x] if x in celltype_mapping_2.keys() else x for x in adata_ref.var_names]
```

Example 13 (python):
```python
set(adata_ref.var_names) - set(adata.obs[celltype_col])
```

Example 14 (unknown):
```unknown
{'LNFRC',
 'MPATF3',
 'MPTIMD4',
 'MoDC',
 'NK',
 'Treg',
 'VCSProx',
 'ValveIC',
 'pDC'}
```

Example 15 (python):
```python
set(adata.obs[celltype_col]) - set(adata_ref.var_names)
```

Example 16 (unknown):
```unknown
{'?EpendymalCells',
 '?EpithelialCells',
 '?SkeletalMuscle',
 '?ThyroidCells',
 'AtrialCardiomyocytes_DoubletWithEC',
 'AtrialCardiomyocytes_DoubletWithFB',
 'CorCapECCyc',
 'CoronaryCapillaryEndothelialCells_DoubletsWithEryth',
 'ELNeuro',
 'EndocardCyc',
 'EndocardialCells_Doublets',
 'Eryth',
 'ErythCyc',
 'FBCyc',
 'GVSMCCyc',
 'MyeloidCells_Doublets',
 'PericCyc',
 'PericFib',
 'PericInt',
 'SOX2_LAMB3_pos',
 'SchwCyc',
 'SchwNeuro',
 'ThyroidEpithelialCells',
 'aCMCyc',
 'doublets',
 'unassigned',
 'unclassified',
 'vCMCyc'}
```

Example 17 (python):
```python
ctab = pd.crosstab(adata.obs[celltype_col],adata.obs['tt_prediction'])
prop_df = (ctab/ctab.sum(axis=0))
prop_df.sum(axis=0)
```

Example 18 (unknown):
```unknown
tt_prediction
AV ring                      1.0
AVnode                       1.0
Adventitia - GV              1.0
Atrium                       1.0
Coronary vessel              1.0
Ductus arteriosus            1.0
Endocardial cusion           1.0
Endocardium - Atrial         1.0
Endocardium - Ventricular    1.0
Epicardium                   1.0
Great vessel                 1.0
SAnode                       1.0
VCS - Distal                 1.0
VCS - Proximal               1.0
Valves                       1.0
Ventricle - Compact          1.0
Ventricle - Trabeculated     1.0
dtype: float64
```

Example 19 (unknown):
```unknown
prop_df.head()
```

Example 20 (python):
```python
df = adata_ref.to_df()
df['annotation_final_mod'] = adata_ref.obs['annotation_final_mod'].copy()
df = df.groupby('annotation_final_mod').sum()
df = df.groupby(df.columns, axis=1).sum()

prop_df_ref = (df.T/df.sum(axis=1))
prop_df_ref.sum(axis=0)
```

Example 21 (unknown):
```unknown
/tmp/ipykernel_3792320/4260203183.py:3: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning.
  df = df.groupby('annotation_final_mod').sum()
/tmp/ipykernel_3792320/4260203183.py:4: FutureWarning: DataFrame.groupby with axis=1 is deprecated. Do `frame.T.groupby(...)` without axis instead.
  df = df.groupby(df.columns, axis=1).sum()
```

Example 22 (unknown):
```unknown
annotation_final_mod
Atrium                       1.0
Ventricle - Compact          1.0
Ventricle - Trabeculated     1.0
SAnode                       1.0
AVnode                       1.0
VCS - Proximal               1.0
VCS - Distal                 1.0
Great vessel                 1.0
Coronary vessel              1.0
Ductus arteriosus            1.0
Adventitia - Coronary        1.0
Adventitia - GV              1.0
Epicardium                   1.0
Endocardium - Atrial         1.0
Endocardium - Ventricular    1.0
Endocardial cusion           1.0
AV ring                      1.0
Valves                       1.0
LN                           1.0
dtype: float64
```

Example 23 (unknown):
```unknown
prop_df_ref.head()
```

Example 24 (python):
```python
# shared cell types
shared = [x for x in prop_df.index if x in prop_df_ref.index]
print(len(shared))
prop_df = prop_df.loc[shared]
prop_df_ref = prop_df_ref.loc[shared]
```

Example 25 (python):
```python
# shared niches
shared = [x for x in prop_df.columns if x in prop_df_ref.columns]
print(len(shared))
prop_df = prop_df[shared]
prop_df_ref = prop_df_ref[shared]
```

Example 26 (python):
```python
print(prop_df.shape)
print(prop_df_ref.shape)
```

Example 27 (unknown):
```unknown
(38, 17)
(38, 17)
```

Example 28 (python):
```python
%%time
corr, pval = utils.df_pearson_corr_with_p_vectorized(prop_df, prop_df_ref, p_adjust='fdr_bh')

# draw the initial clustermap (for ordering only)
cg = sns.clustermap(
    corr,
    cmap='vlag',
    center=0,
    metric='euclidean',
    method='average',
    figsize=(6, 6)
)

# 4) extract the leaf order and reindex both matrices
order = cg.dendrogram_row.reordered_ind
labels = list(corr.index[order])

corr_ord = corr.loc[labels, labels]
pval_ord = pval.loc[labels, labels]

# 5) replot as a heatmap and annotate significance
fig, ax = plt.subplots(figsize=(6, 6))
sns.heatmap(
    corr_ord,
    cmap='vlag',
    center=0,
    square=True,
    cbar_kws={'label': 'Pearson r'},
    ax=ax
)

# add asterisks for p < 0.01
n = corr_ord.shape[0]
for i in range(n):
    for j in range(n):
        if pval_ord.iat[i, j] < 0.01:
            ax.text(j + 0.5, i + 0.5, '*',
                    ha='center', va='center',
                    fontsize=10)

ax.set_xticklabels(labels, rotation=90)
ax.set_xlabel('Reference')
ax.set_yticklabels(labels, rotation=0)
ax.set_ylabel('Xenium-5K, TissueTypist prediction')
plt.tight_layout()
plt.savefig(f'{figdir}/TissueTypist_celltype-prop-corr_Xenium5K.pdf', bbox_inches="tight",dpi=300)
plt.show()
```

Example 29 (unknown):
```unknown
CPU times: user 1.23 s, sys: 69 ms, total: 1.3 s
Wall time: 1.37 s
```

---

## Variables¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig3_SAN_niche/VisiumHD_CCSaxis_no5_HD_to_single-cell.html

**Contents:**
- Variables¶
- Read in data¶
- Subset CCS cells¶
- Select gene features¶
- Single-cell data: generate latent space and perform clustering¶
- CellTypist model training¶
- CellTypist - Predict¶
- Manually annotate cycling cells and transitional cells¶
- Clustering-based vs CellTypist-based, which is better?¶
- Annotate¶
- Save¶

bin2cell output which has celltypist output as well

1510 rows × 14912 columns

Gene features to use: - all genes - HVGs - axis genes: with or without midgrain cell type markers

--> CellTypist, All gene is slightly better than clustering --> will go with CellTypist

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os

import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42
```

Example 2 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/tissueaxis.py")
tissueaxis = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = tissueaxis
spec.loader.exec_module(tissueaxis)

spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 3 (python):
```python
import scanpy.external as sce
```

Example 4 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 5 (python):
```python
adata_dir = '/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects'
sample_id = 'HEA_FOET14880396'
```

Example 6 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk/Aug2024'
table_dir = '/nfs/team205/heart/fetal_ms_figures/tables/Aug2024'
```

Example 7 (unknown):
```unknown
celltype_key = 'fine_grain_8Dec2024'
axis_name = 'SAN_axis'
```

Example 8 (python):
```python
# read in HD data
adata = sc.read_h5ad(f'{adata_dir}/{sample_id}_b2c_cells_filtered_lognorm.h5ad')
adata
```

Example 9 (unknown):
```unknown
AnnData object with n_obs × n_vars = 129803 × 18047
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'fine_grain_8Dec2024', 'AtrialCCS_landmarks', 'SANaxis_landmarks', 'SAN_axis', 'SAN_axis_bin_annot'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'fine_grain_8Dec2024_colors', 'hvg', 'log1p', 'neighbors', 'pca', 'spatial', 'umap'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 10 (python):
```python
# read in single-cell data
adata_sc = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/Feb28ObjectRaw_finegrain_updated.h5ad')

# subset multiome data
adata_sc = adata_sc[adata_sc.obs['kit_10x']=='Multiome-v1']
adata_sc
```

Example 11 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 211145 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score'
    var: 'gene_name_scRNA-0', 'gene_id'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'age_colors', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'cycling_colors', 'donor_colors', 'fine_grain_colors', 'heart_or_greatvessels_colors', 'hvg', 'kit_10x_colors', 'modality_colors', 'neighbors', 'phase_colors', 'region_colors', 'sex_colors', 'trimester_colors', 'umap', 'week_colors'
    obsm: 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 12 (python):
```python
# subset
cells = [
     'SinoatrialNodePacemakerCellsHead',
 'SinoatrialNodePacemakerCellsHorn',
 'SinoatrialNodePacemakerCellsTail',
]

adata = adata[adata.obs[celltype_key].isin(cells)]
adata.obs[celltype_key].value_counts()
```

Example 13 (unknown):
```unknown
fine_grain_8Dec2024
SinoatrialNodePacemakerCellsHead    636
SinoatrialNodePacemakerCellsHorn    501
SinoatrialNodePacemakerCellsTail    373
Name: count, dtype: int64
```

Example 14 (python):
```python
adata.X.data[:5]
```

Example 15 (unknown):
```unknown
array([1.31041406, 1.56012627, 1.65582528, 1.31041406, 1.65582528])
```

Example 16 (python):
```python
cells = [
     'SinoatrialNodePacemakerCells',
]

adata_sc = adata_sc[adata_sc.obs['fine_grain'].isin(cells)]
adata_sc.obs['fine_grain'].value_counts()
```

Example 17 (unknown):
```unknown
fine_grain
SinoatrialNodePacemakerCells    833
Name: count, dtype: int64
```

Example 18 (python):
```python
print(adata_sc.X.data[:5])
sc.pp.normalize_total(adata_sc, target_sum=1e4)
sc.pp.log1p(adata_sc)
print(adata_sc.X.data[:5])
```

Example 19 (unknown):
```unknown
[3. 1. 1. 4. 1.]
```

Example 20 (python):
```python
/software/cellgen/team205/kk18/envs/generic_env/lib/python3.9/site-packages/scanpy/preprocessing/_normalization.py:207: UserWarning: Received a view of an AnnData. Making a copy.
  view_to_actual(adata)
```

Example 21 (unknown):
```unknown
[2.1097739 1.2283119 1.2283119 2.3666706 1.2283119]
```

Example 22 (python):
```python
# filter genes
sc.pp.filter_genes(adata, min_cells=3)
print(adata.shape)
sc.pp.filter_genes(adata_sc, min_cells=3)
print(adata_sc.shape)
```

Example 23 (python):
```python
/software/cellgen/team205/kk18/envs/generic_env/lib/python3.9/site-packages/scanpy/preprocessing/_simple.py:283: ImplicitModificationWarning: Trying to modify attribute `.var` of view, initializing view as actual.
  adata.var["n_cells"] = number
```

Example 24 (unknown):
```unknown
(1510, 14912)
(833, 19057)
```

Example 25 (python):
```python
adata.to_df()
```

Example 26 (python):
```python
# prepare input data
# Assume `cell_by_gene_matrix` is your gene expression DataFrame
# and `axis_values` is a pandas Series with axis values
cell_by_gene_matrix = adata.to_df().copy()
axis_values = adata.obs[axis_name].copy()
```

Example 27 (python):
```python
%%time
import numpy as np
import pandas as pd
from sklearn.preprocessing import PolynomialFeatures
from sklearn.metrics import r2_score
import statsmodels.api as sm
from tqdm import tqdm

# Define polynomial degree range
polynomial_degrees = [1, 2, 3, 4]

# Dictionary to store results for each gene
gene_results = {}

# Iterate through each gene
for gene in tqdm(cell_by_gene_matrix.columns):
    # Store the best fit info for the gene
    best_r2 = -np.inf
    best_degree = None
    best_pvalue = None
    
    # Extract gene expression as dependent variable
    y = cell_by_gene_matrix[gene].values.reshape(-1, 1)
    
    # Iterate through polynomial degrees
    for degree in polynomial_degrees:
        # Create polynomial features
        poly = PolynomialFeatures(degree)
        X_poly = poly.fit_transform(axis_values.values.reshape(-1, 1))
        
        # Fit the model using statsmodels for p-values
        model = sm.OLS(y, X_poly).fit()
        
        # Calculate R^2 score and p-values
        r2 = model.rsquared
        pvalues = model.pvalues  # p-values for each term in the polynomial
        
        # Get the minimum p-value as an indicator of significance
        min_pvalue = np.min(pvalues)
        
        # Update best fit if current model is better
        if r2 > best_r2:
            best_r2 = r2
            best_degree = degree
            best_pvalue = min_pvalue
            
    # Store the best result for the gene
    gene_results[gene] = {'best_r2': best_r2, 'best_degree': best_degree, 'best_pvalue': best_pvalue}

# Convert results to a DataFrame
results_df = pd.DataFrame.from_dict(gene_results, orient='index')
```

Example 28 (unknown):
```unknown
100%|██████████| 14912/14912 [01:27<00:00, 171.15it/s]
```

Example 29 (unknown):
```unknown
CPU times: user 1min 8s, sys: 81.1 ms, total: 1min 8s
Wall time: 1min 28s
```

Example 30 (python):
```python
# Filter genes with significant association (e.g., R^2 > 0.3 and p-value < 0.05)
significant_df = results_df[(results_df['best_r2'] > 0.1) & (results_df['best_pvalue'] < 0.05)]
significant_genes = list(significant_df.index)
print(len(significant_genes))
significant_df.index
```

Example 31 (unknown):
```unknown
Index(['NPPA', 'NPPB', 'DHRS3', 'HSPB7', 'FABP3', 'SLC2A1', 'CNN3', 'GJA5',
       'PRRX1', 'CD34',
       ...
       'MMP15', 'TOP2A', 'COL1A1', 'CKM', 'VSTM2L', 'TGM2', 'MB', 'LGALS1',
       'SMS', 'ATP1B4'],
      dtype='object', length=102)
```

Example 32 (python):
```python
# remove mid-grained cell type markers
# read in
midgrain_markers = pd.read_csv('/nfs/team205/kk18/notebooks/Foetal/VisiumHD/re-seq_mpp0p3/sc_midgrain_markers.csv',
                               index_col=0
                              )
# remove the markers of celltype of interest
midgrain_markers = midgrain_markers[midgrain_markers['group'].isin(['AtrialCardiomyocytes','VentricularCardiomyocytes'])==False]

significant_genes_without_midgrain_markers = [x for x in significant_genes if x not in list(midgrain_markers['names'])]
print(len(significant_genes_without_midgrain_markers))
```

Example 33 (python):
```python
tissueaxis.plot_axis(adata,
              axis_col=axis_name,
              binned_col=f'{axis_name}_bin_annot',
              feature_type='gene', # 'gene' or 'celltype'
              features_to_plot=significant_genes,
              features_to_remove=None,
              threshold_express_prop=0.1, # at quantile=0.9
              threshold_cell_abundance=None, # at quantile=0.9
              correlation=False,
              min_max_scale=True,
              row_cluster=True,
              figsize=(4,18),
              cbar_pos=(-0.2, 0.5, 0.05, 0.18),
              cmap='viridis',
          xticklabels=True,
          yticklabels=True,
              title=None,
              return_feature = False,
                     # save=f'{figdir}/tm-axis_OCT_genes_{key}.pdf',
    save_dpi=300,
             )
```

Example 34 (unknown):
```unknown
/nfs/team205/kk18/function/python/tissueaxis.py:57: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning.
  df_gene_exp_prop = df_gene_exp_bi.groupby(f'{axis_col}_binned').mean()
/nfs/team205/kk18/function/python/tissueaxis.py:134: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning.
  df_plot = df_plot.set_index('index').groupby(['variable',f'{axis_col}_binned']).mean().reset_index()
```

Example 35 (python):
```python
tissueaxis.plot_axis(adata,
              axis_col=axis_name,
              binned_col=f'{axis_name}_bin_annot',
              feature_type='gene', # 'gene' or 'celltype'
              features_to_plot=significant_genes_without_midgrain_markers,
              features_to_remove=None,
              threshold_express_prop=0.1, # at quantile=0.9
              threshold_cell_abundance=None, # at quantile=0.9
              correlation=False,
              min_max_scale=True,
              row_cluster=True,
              figsize=(4,12),
              cbar_pos=(-0.2, 0.5, 0.05, 0.18),
              cmap='viridis',
          xticklabels=True,
          yticklabels=True,
              title=None,
              return_feature = False,
                     # save=f'{figdir}/tm-axis_OCT_genes_{key}.pdf',
    save_dpi=300,
             )
```

Example 36 (unknown):
```unknown
/nfs/team205/kk18/function/python/tissueaxis.py:57: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning.
  df_gene_exp_prop = df_gene_exp_bi.groupby(f'{axis_col}_binned').mean()
/nfs/team205/kk18/function/python/tissueaxis.py:134: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning.
  df_plot = df_plot.set_index('index').groupby(['variable',f'{axis_col}_binned']).mean().reset_index()
```

Example 37 (unknown):
```unknown
os.getcwd()
```

Example 38 (unknown):
```unknown
'/nfs/team205/kk18/notebooks/Foetal/VisiumHD/re-seq_mpp0p3'
```

Example 39 (python):
```python
# HVGs, using the object which doesn't have midgrain markers
sc.pp.highly_variable_genes(adata_sc, flavor='seurat', min_mean=0.0125, max_mean=3, min_disp=0.5)
print(f'number of hvgs: {sum(adata_sc.var.highly_variable)}')
# PCA using the degected HVGs
sc.tl.pca(adata_sc, use_highly_variable=True)
# create neighbour graph
sce.pp.harmony_integrate(adata_sc, 'batch_key')
sc.pp.neighbors(adata_sc, use_rep='X_pca_harmony',n_neighbors=15)
sc.tl.umap(adata_sc, min_dist=0.5, spread=1.0)
```

Example 40 (unknown):
```unknown
number of hvgs: 4721
```

Example 41 (unknown):
```unknown
2024-12-12 16:30:53,521 - harmonypy - INFO - Computing initial centroids with sklearn.KMeans...
Computing initial centroids with sklearn.KMeans...
2024-12-12 16:30:53,610 - harmonypy - INFO - sklearn.KMeans initialization complete.
sklearn.KMeans initialization complete.
2024-12-12 16:30:53,613 - harmonypy - INFO - Iteration 1 of 10
Iteration 1 of 10
2024-12-12 16:30:53,706 - harmonypy - INFO - Iteration 2 of 10
Iteration 2 of 10
2024-12-12 16:30:53,799 - harmonypy - INFO - Iteration 3 of 10
Iteration 3 of 10
2024-12-12 16:30:53,887 - harmonypy - INFO - Iteration 4 of 10
Iteration 4 of 10
2024-12-12 16:30:53,971 - harmonypy - INFO - Iteration 5 of 10
Iteration 5 of 10
2024-12-12 16:30:54,029 - harmonypy - INFO - Iteration 6 of 10
Iteration 6 of 10
2024-12-12 16:30:54,081 - harmonypy - INFO - Iteration 7 of 10
Iteration 7 of 10
2024-12-12 16:30:54,172 - harmonypy - INFO - Iteration 8 of 10
Iteration 8 of 10
2024-12-12 16:30:54,325 - harmonypy - INFO - Iteration 9 of 10
Iteration 9 of 10
2024-12-12 16:30:54,429 - harmonypy - INFO - Iteration 10 of 10
Iteration 10 of 10
2024-12-12 16:30:54,529 - harmonypy - INFO - Stopped before convergence
Stopped before convergence
```

Example 42 (python):
```python
sc.pl.embedding(adata_sc, basis='umap',
                    color=['kit_10x','donor','fine_grain'],
               size=20, wspace=0.6, frameon=False,legend_fontsize=14)
```

Example 43 (python):
```python
# leiden clustering
resolutions_list = [0.2,0.4,0.6,1,1.2,1.5,2]
for resolution in resolutions_list:
    sc.tl.leiden(adata_sc,resolution=resolution,key_added=f'leiden_{str(resolution)}',n_iterations=2)
sc.pl.embedding(adata_sc,basis='X_umap',
                color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                wspace=0.3)
```

Example 44 (python):
```python
genes = ['SHOX2','HCN4','TBX3','TBX18',
         'GJA1', # CX43
         'GJA5', # CX40
         'NKX2-5',
         'NPPA',
         'TOP2A','MKI67'
        ]

sc.pl.embedding(adata_sc, basis='umap',
                        color=genes,
                   size=30, wspace=0.2, frameon=False,cmap='RdPu')
```

Example 45 (unknown):
```unknown
leiden_res_sel = 1
```

Example 46 (python):
```python
import celltypist
```

Example 47 (python):
```python
sc.pp.highly_variable_genes(adata, flavor='seurat', min_mean=0.0125, max_mean=3, min_disp=0.5)
print(f'number of hvgs: {sum(adata.var.highly_variable)}')
hvgs = adata.var_names[adata.var.highly_variable]
hvgs[:5]
```

Example 48 (unknown):
```unknown
number of hvgs: 3600
```

Example 49 (unknown):
```unknown
Index(['HES4', 'ISG15', 'C1orf159', 'ACAP3', 'INTS11'], dtype='object')
```

Example 50 (python):
```python
features = {}
features['all'] = list(adata.var_names)
features['hvg'] = hvgs
features['axis_gene'] = significant_genes
features['axis_gene_wo_midmarkers'] = significant_genes_without_midgrain_markers
```

Example 51 (python):
```python
for key,value in features.items():
    print(key)
    model = celltypist.train(adata[:,value],  # selecting features
                         labels = celltype_key,
                         n_jobs = 10,
                         feature_selection = False, # since manually selected
                         check_expression = False
                        )
    # Save the model
    model.write(f'/nfs/team205/kk18/notebooks/Foetal/VisiumHD/re-seq_mpp0p3/celltypist/model_HD-SANPC_{key}.pkl')
```

Example 52 (unknown):
```unknown
🍳 Preparing data before training
```

Example 53 (unknown):
```unknown
🔬 Input data has 1510 cells and 14912 genes
⚖️ Scaling input data
🏋️ Training data using logistic regression
✅ Model training done!
🍳 Preparing data before training
```

Example 54 (unknown):
```unknown
🔬 Input data has 1510 cells and 3600 genes
⚖️ Scaling input data
🏋️ Training data using logistic regression
✅ Model training done!
🍳 Preparing data before training
```

Example 55 (unknown):
```unknown
🔬 Input data has 1510 cells and 102 genes
⚖️ Scaling input data
🏋️ Training data using logistic regression
✅ Model training done!
🍳 Preparing data before training
```

Example 56 (unknown):
```unknown
axis_gene_wo_midmarkers
```

Example 57 (unknown):
```unknown
🔬 Input data has 1510 cells and 55 genes
⚖️ Scaling input data
🏋️ Training data using logistic regression
✅ Model training done!
```

Example 58 (unknown):
```unknown
os.getcwd()
```

Example 59 (unknown):
```unknown
'/nfs/team205/kk18/notebooks/Foetal/VisiumHD/re-seq_mpp0p3'
```

Example 60 (python):
```python
adata_sc.X.data[:5]
```

Example 61 (unknown):
```unknown
array([2.1097739, 1.2283119, 1.2283119, 2.3666706, 1.2283119],
      dtype=float32)
```

Example 62 (python):
```python
for key in features.keys():
    print(key)
    predictions = celltypist.annotate(adata_sc, 
                                      model = f'/nfs/team205/kk18/notebooks/Foetal/VisiumHD/re-seq_mpp0p3/celltypist/model_HD-SANPC_{key}.pkl', 
                                      majority_voting = False)
    # plot
    celltypist.dotplot(predictions, use_as_reference = f'leiden_{str(leiden_res_sel)}', use_as_prediction = 'predicted_labels', vmax=1, show=False)
    # add to anndata
    adata_sc = predictions.to_adata(insert_labels = True, insert_conf = True)
    adata_sc.obs.rename(columns={
        'predicted_labels':f'celltypist_prediction_{key}',
        'conf_score':f'celltypist_conf_score_{key}'
    },inplace=True)
```

Example 63 (unknown):
```unknown
⚠️ Warning: invalid expression matrix, expect ALL genes and log1p normalized expression to 10000 counts per cell. The prediction result may not be accurate
🔬 Input data has 833 cells and 19057 genes
🔗 Matching reference genes in the model
```

Example 64 (unknown):
```unknown
🧬 13037 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
⚠️ Warning: invalid expression matrix, expect ALL genes and log1p normalized expression to 10000 counts per cell. The prediction result may not be accurate
🔬 Input data has 833 cells and 19057 genes
🔗 Matching reference genes in the model
```

Example 65 (unknown):
```unknown
🧬 3177 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
⚠️ Warning: invalid expression matrix, expect ALL genes and log1p normalized expression to 10000 counts per cell. The prediction result may not be accurate
🔬 Input data has 833 cells and 19057 genes
🔗 Matching reference genes in the model
```

Example 66 (unknown):
```unknown
🧬 101 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
⚠️ Warning: invalid expression matrix, expect ALL genes and log1p normalized expression to 10000 counts per cell. The prediction result may not be accurate
🔬 Input data has 833 cells and 19057 genes
🔗 Matching reference genes in the model
🧬 54 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
```

Example 67 (unknown):
```unknown
axis_gene_wo_midmarkers
```

Example 68 (python):
```python
for key in features.keys():
    # plot on umap
    sc.pl.embedding(adata_sc, basis='umap',
                        color=[f'celltypist_conf_score_{key}',f'celltypist_prediction_{key}'],
                   size=30, wspace=0.4, frameon=False,legend_fontsize=14)
```

Example 69 (python):
```python
genes = ['SHOX2',
    'TBX3','TBX18',
         'GJA1', # CX43
         'GJA5', # CX40
         'NKX2-5','NPPA']
genes = [x for x in genes if x in adata_sc.var_names]
sc.pl.dotplot(adata_sc,
              var_names=genes,
              groupby=f'leiden_{str(leiden_res_sel)}',
              standard_scale='var',
              dendrogram=True
             )
```

Example 70 (python):
```python
# DEG analysis
print(adata_sc.X.data[:5])
adata_sc.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata_sc,groupby=f'leiden_{str(leiden_res_sel)}')
sc.tl.dendrogram(adata_sc, groupby=f'leiden_{str(leiden_res_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata_sc, dendrogram=True, standard_scale='var', return_fig=True)
dp.add_totals().show()
```

Example 71 (unknown):
```unknown
[2.1097739 1.2283119 1.2283119 2.3666706 1.2283119]
```

Example 72 (unknown):
```unknown
leiden_res_sel
```

Example 73 (python):
```python
sc.pl.embedding(adata_sc, basis='umap',
                        color=[f'leiden_{str(leiden_res_sel)}'],legend_loc='on data',
                   size=30, wspace=0.4, frameon=False,legend_fontsize=14)
```

Example 74 (python):
```python
# annotate
adata_sc.obs['SANPC_subpopulations'] = adata_sc.obs[f'leiden_{str(leiden_res_sel)}'].copy()
adata_sc.obs.replace({'SANPC_subpopulations':{
    '0':'SinoatrialNodePacemakerCellsHead',
    '1':'SinoatrialNodePacemakerCellsTail',
    '2':'SinoatrialNodeTransitionalCells',
    '3':'SinoatrialNodePacemakerCellsTail',
    '4':'SinoatrialNodeTransitionalCells',
    '5':'SinoatrialNodePacemakerCellsHead',
    '6':'SinoatrialNodePacemakerCellsHorn',
    '7':'SinoatrialNodeTransitionalCells',
    '8':'SinoatrialNodePacemakerCellsTail',
    '9':'SinoatrialNodePacemakerCellsHead',
    '10':'SinoatrialNodePacemakerCellsCycling',
    '11':'SinoatrialNodePacemakerCellsHead'
}},inplace=True)
adata_sc.obs['SANPC_subpopulations'].value_counts()
```

Example 75 (unknown):
```unknown
SANPC_subpopulations
SinoatrialNodePacemakerCellsHead       289
SinoatrialNodePacemakerCellsTail       226
SinoatrialNodeTransitionalCells        209
SinoatrialNodePacemakerCellsHorn        65
SinoatrialNodePacemakerCellsCycling     44
Name: count, dtype: int64
```

Example 76 (python):
```python
sc.pl.embedding(adata_sc, basis='umap',
                        color=['SANPC_subpopulations'],
                   size=30, wspace=0.4, frameon=False,legend_fontsize=14)
```

Example 77 (python):
```python
for key in features.keys():
    print(key)
    predictions = celltypist.annotate(adata_sc, 
                                      model = f'/nfs/team205/kk18/notebooks/Foetal/VisiumHD/re-seq_mpp0p3/celltypist/model_HD-SANPC_{key}.pkl', 
                                      majority_voting = False)
    # plot
    celltypist.dotplot(predictions, use_as_reference = 'SANPC_subpopulations', use_as_prediction = 'predicted_labels', vmax=1, show=False)
```

Example 78 (unknown):
```unknown
⚠️ Warning: invalid expression matrix, expect ALL genes and log1p normalized expression to 10000 counts per cell. The prediction result may not be accurate
🔬 Input data has 833 cells and 19057 genes
🔗 Matching reference genes in the model
```

Example 79 (unknown):
```unknown
🧬 13037 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
⚠️ Warning: invalid expression matrix, expect ALL genes and log1p normalized expression to 10000 counts per cell. The prediction result may not be accurate
🔬 Input data has 833 cells and 19057 genes
🔗 Matching reference genes in the model
```

Example 80 (unknown):
```unknown
🧬 3177 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
⚠️ Warning: invalid expression matrix, expect ALL genes and log1p normalized expression to 10000 counts per cell. The prediction result may not be accurate
🔬 Input data has 833 cells and 19057 genes
🔗 Matching reference genes in the model
🧬 101 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
```

Example 81 (unknown):
```unknown
axis_gene
axis_gene_wo_midmarkers
```

Example 82 (unknown):
```unknown
⚠️ Warning: invalid expression matrix, expect ALL genes and log1p normalized expression to 10000 counts per cell. The prediction result may not be accurate
🔬 Input data has 833 cells and 19057 genes
🔗 Matching reference genes in the model
🧬 54 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
```

Example 83 (python):
```python
# remove cycling clusters
adata_sc_sel = adata_sc[adata_sc.obs['SANPC_subpopulations'].isin(['SinoatrialNodePacemakerCellsCycling'])==False]
adata_sc_sel.obs['SANPC_subpopulations'].value_counts()
```

Example 84 (unknown):
```unknown
SANPC_subpopulations
SinoatrialNodePacemakerCellsHead    289
SinoatrialNodePacemakerCellsTail    226
SinoatrialNodeTransitionalCells     209
SinoatrialNodePacemakerCellsHorn     65
Name: count, dtype: int64
```

Example 85 (python):
```python
# markers for assessing
genes = ['SHOX2',
    'TBX3','TBX18',
         'GJA1', # CX43
         'GJA5', # CX40
         'NKX2-5',# 'NPPA'
        ]
genes = [x for x in genes if x in adata_sc.var_names]
```

Example 86 (python):
```python
# clustering based annotation
sc.pl.dotplot(adata_sc_sel,
                  var_names=genes,
                  groupby='SANPC_subpopulations',
                  standard_scale='var',
                  dendrogram=True,
                  title=key
                 )
```

Example 87 (python):
```python
WARNING: dendrogram data not found (using key=dendrogram_SANPC_subpopulations). Running `sc.tl.dendrogram` with default parameters. For fine tuning it is recommended to run `sc.tl.dendrogram` independently.
```

Example 88 (python):
```python
# celltypist based annotation
conf_score_thresh = 0.6

transitional_obsnames = list(adata_sc_sel.obs_names[adata_sc_sel.obs['SANPC_subpopulations']=='SinoatrialNodeTransitionalCells'])
for key in features.keys():
    ad = adata_sc_sel.copy()
    ad.obs['tmp_annotation'] = ad.obs[f'celltypist_prediction_{key}'].astype('str').copy()
    ad.obs.loc[transitional_obsnames, 'tmp_annotation'] = 'SinoatrialNodeTransitionalCells'
    # remove low confidence cells
    ad = ad[ad.obs[f'celltypist_conf_score_{key}']>conf_score_thresh]
    print(key)
    print(ad.obs['tmp_annotation'].value_counts())
    sc.pl.dotplot(ad,
                  var_names=genes,
                  groupby='tmp_annotation',
                  standard_scale='var',
                  dendrogram=True,
                 )
```

Example 89 (unknown):
```unknown
all
tmp_annotation
SinoatrialNodePacemakerCellsHead    405
SinoatrialNodeTransitionalCells     183
SinoatrialNodePacemakerCellsTail     93
SinoatrialNodePacemakerCellsHorn     52
Name: count, dtype: int64
```

Example 90 (unknown):
```unknown
... storing 'tmp_annotation' as categorical
```

Example 91 (unknown):
```unknown
... storing 'tmp_annotation' as categorical
```

Example 92 (unknown):
```unknown
hvg
tmp_annotation
SinoatrialNodePacemakerCellsHead    296
SinoatrialNodeTransitionalCells     130
SinoatrialNodePacemakerCellsTail    117
SinoatrialNodePacemakerCellsHorn     23
Name: count, dtype: int64
```

Example 93 (unknown):
```unknown
... storing 'tmp_annotation' as categorical
```

Example 94 (unknown):
```unknown
axis_gene
tmp_annotation
SinoatrialNodePacemakerCellsHead    302
SinoatrialNodePacemakerCellsHorn    100
SinoatrialNodeTransitionalCells      85
SinoatrialNodePacemakerCellsTail     29
Name: count, dtype: int64
```

Example 95 (unknown):
```unknown
... storing 'tmp_annotation' as categorical
```

Example 96 (unknown):
```unknown
axis_gene_wo_midmarkers
tmp_annotation
SinoatrialNodePacemakerCellsHead    144
SinoatrialNodePacemakerCellsTail    115
SinoatrialNodeTransitionalCells      95
SinoatrialNodePacemakerCellsHorn     80
Name: count, dtype: int64
```

Example 97 (unknown):
```unknown
conf_score_thresh = 0.6
key = 'all'
```

Example 98 (python):
```python
transitional_obsnames = list(adata_sc.obs_names[adata_sc.obs['SANPC_subpopulations']=='SinoatrialNodeTransitionalCells'])
cyclingcell_obsnames = list(adata_sc.obs_names[adata_sc.obs['SANPC_subpopulations']=='SinoatrialNodePacemakerCellsCycling'])
low_conf_obsnames = list(adata_sc.obs_names[adata_sc.obs[f'celltypist_conf_score_{key}']<conf_score_thresh])

adata_sc.obs['SANPC_subpopulations'] = adata_sc.obs[f'celltypist_prediction_{key}'].astype('str').copy()
adata_sc.obs.loc[low_conf_obsnames,'SANPC_subpopulations'] = 'SinoatrialNodePacemakerCells_celltypist-low-confidence'
adata_sc.obs.loc[transitional_obsnames,'SANPC_subpopulations'] = 'SinoatrialNodeTransitionalCells'
adata_sc.obs.loc[cyclingcell_obsnames,'SANPC_subpopulations'] = 'SinoatrialNodePacemakerCellsCycling'
adata_sc.obs['SANPC_subpopulations'].value_counts()
```

Example 99 (unknown):
```unknown
SANPC_subpopulations
SinoatrialNodePacemakerCellsHead                          405
SinoatrialNodeTransitionalCells                           209
SinoatrialNodePacemakerCellsTail                           93
SinoatrialNodePacemakerCellsHorn                           52
SinoatrialNodePacemakerCellsCycling                        44
SinoatrialNodePacemakerCells_celltypist-low-confidence     30
Name: count, dtype: int64
```

Example 100 (python):
```python
sc.pl.embedding(adata_sc, basis='umap',
                        color=['SANPC_subpopulations'],
                   size=30, wspace=0.4, frameon=False,legend_fontsize=14)
```

Example 101 (unknown):
```unknown
... storing 'SANPC_subpopulations' as categorical
```

Example 102 (python):
```python
# markers for assessing
genes = ['SHOX2',
    'TBX3','TBX18',
         'GJA1', # CX43
         'GJA5', # CX40
         'NKX2-5',# 'NPPA'
         'MKI67','TOP2A'
        ]
genes = [x for x in genes if x in adata_sc.var_names]

sc.pl.dotplot(adata_sc,
                  var_names=genes,
                  groupby='SANPC_subpopulations',
                  standard_scale='var',
                  dendrogram=True,
                  title=key
                 )
```

Example 103 (python):
```python
# without cycling and transitional cells
genes = ['SHOX2',
    'TBX3','TBX18',
         'GJA1', # CX43
         'GJA5', # CX40
         'NKX2-5',# 'NPPA'
        ]
genes = [x for x in genes if x in adata_sc.var_names]

sc.pl.dotplot(adata_sc[adata_sc.obs['SANPC_subpopulations'].isin(['SinoatrialNodePacemakerCellsCycling','SinoatrialNodeTransitionalCells','SinoatrialNodePacemakerCells_celltypist-low-confidence'])==False],
                  var_names=genes,
                  groupby='SANPC_subpopulations',
                  standard_scale='var',
                  dendrogram=False,
                  title=key
                 )
```

Example 104 (python):
```python
# read in original raw data again
adata_sc_all = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/Feb28ObjectRaw_finegrain_updated.h5ad')
```

Example 105 (python):
```python
# subset multiome and sanpc data
adata_sc_sanpc_multiome = adata_sc_all[adata_sc_all.obs['kit_10x']=='Multiome-v1']
adata_sc_sanpc_multiome = adata_sc_sanpc_multiome[adata_sc_sanpc_multiome.obs['fine_grain']=='SinoatrialNodePacemakerCells']
adata_sc_sanpc_multiome
```

Example 106 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 833 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score'
    var: 'gene_name_scRNA-0', 'gene_id'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'age_colors', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'cycling_colors', 'donor_colors', 'fine_grain_colors', 'heart_or_greatvessels_colors', 'hvg', 'kit_10x_colors', 'modality_colors', 'neighbors', 'phase_colors', 'region_colors', 'sex_colors', 'trimester_colors', 'umap', 'week_colors'
    obsm: 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 107 (python):
```python
# copy the labels
adata_sc_sanpc_multiome.obs['SANPC_subpopulations'] = adata_sc.obs['SANPC_subpopulations'].reindex(adata_sc_sanpc_multiome.obs_names)
adata_sc_sanpc_multiome.obs['SANPC_subpopulations'].value_counts()
```

Example 108 (unknown):
```unknown
SANPC_subpopulations
SinoatrialNodePacemakerCellsHead                          405
SinoatrialNodeTransitionalCells                           209
SinoatrialNodePacemakerCellsTail                           93
SinoatrialNodePacemakerCellsHorn                           52
SinoatrialNodePacemakerCellsCycling                        44
SinoatrialNodePacemakerCells_celltypist-low-confidence     30
Name: count, dtype: int64
```

Example 109 (python):
```python
adata_sc_sanpc_multiome.write('/nfs/team205/heart/anndata_objects/Foetal/Feb28ObjectRaw_finegrain_updated_SANPC.h5ad')
print(adata_sc_sanpc_multiome.X.data[:5])
```

Example 110 (unknown):
```unknown
[3. 1. 1. 4. 1.]
```

Example 111 (python):
```python
adata_sc_sanpc_multiome
```

Example 112 (unknown):
```unknown
AnnData object with n_obs × n_vars = 833 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score', 'SANPC_subpopulations'
    var: 'gene_name_scRNA-0', 'gene_id'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'age_colors', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'cycling_colors', 'donor_colors', 'fine_grain_colors', 'heart_or_greatvessels_colors', 'hvg', 'kit_10x_colors', 'modality_colors', 'neighbors', 'phase_colors', 'region_colors', 'sex_colors', 'trimester_colors', 'umap', 'week_colors'
    obsm: 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

---

## Variables¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/VisiumHD/no7_manual-annotation_EndocardialCells.html

**Contents:**
- Variables¶
- Read in data¶
- Remove mid-grain markers from the feature¶
- PCA, Clustering, UMAP¶
- Markers¶
- Subset "uncertain" cells¶
- Add the annotation to the original adata¶
- Save¶

2385 rows × 7 columns

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os

import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 

import celltypist
```

Example 2 (python):
```python
# import session_info
# session_info.show()
```

Example 3 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 4 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 5 (python):
```python
adata_dir = '/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/subsets'
sample_id = 'HEA_FOET14880396'
celltype = 'EndocardialCells'
```

Example 6 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk/Aug2024'
table_dir = '/nfs/team205/heart/fetal_ms_figures/tables/Aug2024'
```

Example 7 (python):
```python
# read in
adata = sc.read_h5ad(f'{adata_dir}/{sample_id}_{celltype}_b2c_cells_filtered_lognorm.h5ad')
adata
```

Example 8 (unknown):
```unknown
AnnData object with n_obs × n_vars = 1526 × 14451
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'leiden_0.2', 'leiden_0.4', 'leiden_0.6', 'leiden_1', 'leiden_1.2', 'leiden_1.5', 'leiden_2'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1.2', 'leiden_1.2_colors', 'leiden_1.5', 'leiden_1.5_colors', 'leiden_1_colors', 'leiden_2', 'leiden_2_colors', 'log1p', 'neighbors', 'pca', 'spatial', 'umap'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 9 (python):
```python
# read in
midgrain_markers = pd.read_csv('/nfs/team205/kk18/notebooks/Foetal/VisiumHD/re-seq_mpp0p3/sc_midgrain_markers.csv',
                               index_col=0
                              )
# remove the markers of celltype of interest
midgrain_markers = midgrain_markers[midgrain_markers['group']!=celltype]
midgrain_markers
```

Example 10 (python):
```python
# remove
mask = [x not in list(midgrain_markers['names']) for x in adata.var_names]
bdata = adata[:,mask]
bdata
```

Example 11 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 1526 × 13211
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'leiden_0.2', 'leiden_0.4', 'leiden_0.6', 'leiden_1', 'leiden_1.2', 'leiden_1.5', 'leiden_2'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1.2', 'leiden_1.2_colors', 'leiden_1.5', 'leiden_1.5_colors', 'leiden_1_colors', 'leiden_2', 'leiden_2_colors', 'log1p', 'neighbors', 'pca', 'spatial', 'umap'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 12 (python):
```python
# HVGs, using the object which doesn't have midgrain markers
sc.pp.highly_variable_genes(bdata, flavor='seurat', min_mean=0.0125, max_mean=3, min_disp=0.5)
print(f'number of hvgs: {sum(bdata.var.highly_variable)}')
# PCA using the degected HVGs
sc.tl.pca(bdata, use_highly_variable=True)
adata.obsm['X_pca'] = bdata.obsm['X_pca'].copy()
# create neighbour graph
sc.pp.neighbors(adata,n_neighbors=15,use_rep='X_pca')
# visualisation with umap
sc.tl.umap(adata,min_dist=0.5)
sc.pl.embedding(adata,basis='X_umap',color=['bin_count','total_counts','n_genes_by_counts',"conf_score_midmod2fine","celltypist_midmod2fine",],
                wspace=0.2,cmap='RdPu',vmax='p100',ncols=5)
# leiden clustering
resolutions_list = [0.2,0.4,0.6,1,1.2,1.5,2]
for resolution in resolutions_list:
    sc.tl.leiden(adata,resolution=resolution,key_added=f'leiden_{str(resolution)}',n_iterations=2)
sc.pl.embedding(adata,basis='X_umap',
                color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                wspace=0.3)
```

Example 13 (unknown):
```unknown
number of hvgs: 3336
```

Example 14 (python):
```python
# read in markers identified using single-nucleus data
markers_df = pd.read_csv(f'/nfs/team205/kk18/notebooks/Foetal/VisiumHD/re-seq_mpp0p3/sc_{celltype}_finegrain_markers.csv')

mask1 = markers_df['group_mean']>1
mask2 = markers_df['logfoldchanges']>1
markers_df = markers_df[mask1&mask2]

markers_df
```

Example 15 (python):
```python
markers = {}
n_top = 10
for c in set(markers_df['group']):
    markers_df_sub = markers_df[markers_df['group']==c]
    markers_df_sub.sort_values('logfoldchanges',ascending=False)
    markers[c]=list(markers_df_sub[:n_top]['names'])

# manually add genes
markers['Cycling'] = ['MKI67','TOP2A'] 

genes = []
for g in markers.values():
    genes = genes + [x for x in g if (x in list(adata.var_names))&(x not in genes)]
for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in list(adata.var_names)]
len(genes)
```

Example 16 (python):
```python
sc.pl.embedding(adata,basis='X_umap',color=genes, size=20,
               ncols=6, wspace=0.2,cmap='RdPu',vmax='p99.7')
```

Example 17 (python):
```python
sc.pl.embedding(adata,basis='X_umap',
                color=['celltypist_midmod2fine', 'conf_score_midmod2fine'],
                wspace=0.7)
```

Example 18 (python):
```python
# umap per celltype to highlight
group_col = 'celltypist_midmod2fine'
groups_to_plot = list(set(adata.obs[group_col]))

plt.rcParams['axes.grid'] = False
fig,axs=plt.subplots(nrows=2,ncols=4,figsize=(15,7))
# Flatten the axs array for easier iteration in case of multiple rows and columns
axs_flat = axs.flatten()

for i,grp in enumerate(groups_to_plot):
    sc.pl.umap(adata, color=[group_col], groups=grp, 
               size=30, frameon=False,
               show=False, ax=axs_flat[i], legend_loc=None, legend_fontsize=5,
               title=grp
              )
# Turn off any remaining subplots
for j in range(len(groups_to_plot), len(axs_flat)):
    fig.delaxes(axs_flat[j])

# fig.suptitle('Main Title for the Figure')
fig.tight_layout()
```

Example 19 (unknown):
```unknown
resolution_sel = 1
```

Example 20 (python):
```python
markers_others = {
    'CM': ['TTN', 'TNNT2','MYH6',
        'MYH7', 'MYL2', 'FHL2', # vCN
            'NPPA', 'MYL7', 'MYL4'], #aCM
    'FB': ['DCN', 'GCN', 'PDGFRA','COL1A1','COL1A2'],
    'EC': ['VWF', 'PECAM1', 'CDH5','RGCC', 'FABP5',],
    'Peri': ['RGS5', 'ABCC9', 'KCNJ8'],
    'SMC': ['MYH11', 'TAGLN', 'ACTA2'],
    'Meso': ['MSLN', 'WT1', 'BNC1'],
    'Neuro': ['PLP1', 'NRXN1', 'NRXN3','PRPH', 'NEFL', 'NEFM', 'NEFH', 'STMN2', 'SST', 'SLC10A4', 'SLC18A3', 'TH', 'NPY'],
    'Adipo': ['GPAM', 'FASN', 'LEP'],
    'Myelo': ['CD14', 'C1QA', 'CD68','LYVE1','TIMD4'],
    'Lymph': ['CD8A', 'IL7R', 'CD40LG'],
    'B': ['CD19','SDC1','IGHG1','IGHA1'],
    'Mast': ['KIT', 'CPA3'],
}

for key in markers_others.keys():
    markers_others[key]=[x for x in markers_others[key] if x in adata.var_names]

sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
sc.pl.dotplot(adata,
              markers_others, 
              groupby=f'leiden_{str(resolution_sel)}',
              dendrogram=True,
              standard_scale="var",
              color_map="Reds",
              swap_axes=False,
             )
```

Example 21 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: CM, FB, EC, etc.
```

Example 22 (python):
```python
sc.tl.dendrogram(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.pl.dotplot(adata,
              var_names=markers,
              groupby=f'leiden_{str(resolution_sel)}',
              standard_scale='var',
              dendrogram=True
             )
```

Example 23 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: EndocardialCushionCells, ValveEndothelialCells, EndocardialCells, etc.
```

Example 24 (python):
```python
# DEG analysis
print(adata.X.data[:5])
adata.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata, dendrogram=True, standard_scale='var', return_fig=True)
dp.add_totals().show()
```

Example 25 (unknown):
```unknown
[1.84898172 2.52192218 2.53797117 1.50684599 2.05779654]
```

Example 26 (python):
```python
# DEG analysis
print(adata.X.data[:5])
adata.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata, dendrogram=True, standard_scale='var', return_fig=True, n_genes=5)
dp.add_totals().show()
```

Example 27 (unknown):
```unknown
[1.84898172 2.52192218 2.53797117 1.50684599 2.05779654]
```

Example 28 (python):
```python
sc.pl.embedding(adata,basis='X_umap',
                color=f'leiden_{str(resolution_sel)}',
                wspace=0.3, legend_loc='on data')
```

Example 29 (python):
```python
sc.pl.violin(adata,keys=['total_counts','bin_count'],groupby=f'leiden_{str(resolution_sel)}')
```

Example 30 (unknown):
```unknown
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 31 (python):
```python
# annotate
adata.obs['fine_grain_manual'] = adata.obs[f'leiden_{str(resolution_sel)}'].copy()
adata.obs.replace({'fine_grain_manual':{
    '0':'EndocardialCells',
    '1':'ValveEndothelialCells',
    '2':'uncertain',
    '3':'ValveEndothelialCells',
    '4':'EndocardialCushionCells', # contamination from valveECs
    '5':'EndocardialCushionCells',
    '6':'Erythrocytes',
}},inplace=True)
adata.obs['fine_grain_manual'].value_counts()
```

Example 32 (unknown):
```unknown
fine_grain_manual
ValveEndothelialCells      585
EndocardialCells           349
uncertain                  297
EndocardialCushionCells    284
Erythrocytes                11
Name: count, dtype: int64
```

Example 33 (python):
```python
ad = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/Feb28ObjectRaw_finegrain_updated.h5ad',backed='r')
ad = ad[ad.obs['mid_grain']==celltype]
set(ad.obs['fine_grain'])
```

Example 34 (unknown):
```unknown
{'EndocardialCells', 'EndocardialCushionCells', 'ValveEndothelialCells'}
```

Example 35 (python):
```python
sc.pl.embedding(adata,basis='X_umap',
                color='fine_grain_manual',
                frameon=False,
                wspace=0.3)
```

Example 36 (python):
```python
adata_sub = adata[adata.obs['fine_grain_manual'].isin(['uncertain'])]

# drop leiden clustering results
resolutions_list = [0.2,0.4,0.6,1, 1.2,1.5,2]
for resolution in resolutions_list:
    adata_sub.obs.drop(f'leiden_{str(resolution)}',axis=1,inplace=True)
adata_sub
```

Example 37 (unknown):
```unknown
AnnData object with n_obs × n_vars = 297 × 14451
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'fine_grain_manual'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'hvg', 'leiden_0.2', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_1', 'leiden_1.2', 'leiden_1.5', 'leiden_1.5_colors', 'leiden_1_colors', 'leiden_2', 'log1p', 'neighbors', 'pca', 'spatial', 'umap', 'dendrogram_leiden_1', 'rank_genes_groups', 'dendrogram_leiden_2', 'dendrogram_leiden_1.5', 'fine_grain_manual_colors'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 38 (python):
```python
# remove mid_grain markers
mask = [x not in list(midgrain_markers['names']) for x in adata_sub.var_names]
bdata = adata_sub[:,mask]
bdata
```

Example 39 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 297 × 13211
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'fine_grain_manual'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'hvg', 'leiden_0.2', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_1', 'leiden_1.2', 'leiden_1.5', 'leiden_1.5_colors', 'leiden_1_colors', 'leiden_2', 'log1p', 'neighbors', 'pca', 'spatial', 'umap', 'dendrogram_leiden_1', 'rank_genes_groups', 'dendrogram_leiden_2', 'dendrogram_leiden_1.5', 'fine_grain_manual_colors'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 40 (python):
```python
# HVGs, using the object which doesn't have midgrain markers
sc.pp.highly_variable_genes(bdata, flavor='seurat', min_mean=0.0125, max_mean=3, min_disp=0.5)
print(f'number of hvgs: {sum(bdata.var.highly_variable)}')
# PCA using the degected HVGs
sc.tl.pca(bdata, use_highly_variable=True)
adata_sub.obsm['X_pca'] = bdata.obsm['X_pca'].copy()
# create neighbour graph
sc.pp.neighbors(adata_sub,n_neighbors=15,use_rep='X_pca')
```

Example 41 (unknown):
```unknown
number of hvgs: 3218
```

Example 42 (python):
```python
sc.tl.umap(adata_sub,min_dist=0.1)
```

Example 43 (python):
```python
%%time
# reclustering
# leiden clustering
resolutions_list = [0.2,0.4,0.6,1, 1.2,1.5,2]
for resolution in resolutions_list:
    sc.tl.leiden(adata_sub,resolution=resolution,key_added=f'leiden_{str(resolution)}',n_iterations=2)
sc.pl.embedding(adata_sub,basis='X_umap',
                color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                wspace=0.3)
```

Example 44 (unknown):
```unknown
CPU times: user 1.44 s, sys: 39.8 ms, total: 1.48 s
Wall time: 1.53 s
```

Example 45 (python):
```python
sc.pl.embedding(adata_sub,basis='X_umap',color=genes, size=10,
               ncols=6, wspace=0.2,cmap='RdPu',vmax='p99.7')
```

Example 46 (python):
```python
# umap per celltype to highlight
group_col = 'celltypist_midmod2fine'
groups_to_plot = list(set(adata_sub.obs[group_col]))

plt.rcParams['axes.grid'] = False
fig,axs=plt.subplots(nrows=2,ncols=4,figsize=(15,7))
# Flatten the axs array for easier iteration in case of multiple rows and columns
axs_flat = axs.flatten()

for i,grp in enumerate(groups_to_plot):
    sc.pl.umap(adata_sub, color=[group_col], groups=grp, 
               size=30, frameon=False,
               show=False, ax=axs_flat[i], legend_loc=None, legend_fontsize=5,
               title=grp
              )
# Turn off any remaining subplots
for j in range(len(groups_to_plot), len(axs_flat)):
    fig.delaxes(axs_flat[j])

# fig.suptitle('Main Title for the Figure')
fig.tight_layout()
```

Example 47 (python):
```python
adata_sub_resolution_sel = 0.6
```

Example 48 (python):
```python
markers_sub = {k:v for k,v in markers.items() if k in ['EndocardialCushionCells','EndocardialCells']}

sc.tl.dendrogram(adata_sub,groupby=f'leiden_{str(adata_sub_resolution_sel)}')
sc.pl.dotplot(adata_sub,
              var_names=markers_sub,
              groupby=f'leiden_{str(adata_sub_resolution_sel)}',
              standard_scale='var',
              dendrogram=True
             )
```

Example 49 (unknown):
```unknown
WARNING: Dendrogram not added. Dendrogram is added only when the number of categories to plot > 2
```

Example 50 (python):
```python
sc.pl.embedding(adata_sub,basis='X_umap',
                color=f'leiden_{str(adata_sub_resolution_sel)}',
                wspace=0.3, legend_loc='on data')
```

Example 51 (python):
```python
# annotate
adata_sub.obs['fine_grain_manual'] = adata_sub.obs[f'leiden_{str(adata_sub_resolution_sel)}'].copy()
adata_sub.obs.replace({'fine_grain_manual':{
    '0':'EndocardialCushionCells', 
    '1':'EndocardialCells',
}},inplace=True)
adata_sub.obs['fine_grain_manual'].value_counts()
```

Example 52 (unknown):
```unknown
fine_grain_manual
EndocardialCushionCells    247
EndocardialCells            50
Name: count, dtype: int64
```

Example 53 (python):
```python
sc.pl.embedding(adata_sub,basis='X_umap',
                color='fine_grain_manual',
                frameon=False,
                wspace=0.3)
```

Example 54 (python):
```python
markers_sub = {k:v for k,v in markers.items() if k in ['EndocardialCushionCells','EndocardialCells']}

sc.tl.dendrogram(adata_sub,groupby=f'leiden_{str(adata_sub_resolution_sel)}')
sc.pl.dotplot(adata_sub,
              var_names=markers_sub,
              groupby='fine_grain_manual',
              standard_scale='var',
              dendrogram=True
             )
```

Example 55 (unknown):
```unknown
WARNING: Dendrogram not added. Dendrogram is added only when the number of categories to plot > 2
```

Example 56 (python):
```python
# add the annotation to the original adata
adata.obs['fine_grain_manual'] = adata.obs['fine_grain_manual'].astype('str')
adata.obs.loc[adata_sub.obs_names,'fine_grain_manual'] = adata_sub.obs['fine_grain_manual'].astype('str').copy()
adata.obs['fine_grain_manual'].value_counts()
```

Example 57 (unknown):
```unknown
fine_grain_manual
ValveEndothelialCells      585
EndocardialCushionCells    531
EndocardialCells           399
Erythrocytes                11
Name: count, dtype: int64
```

Example 58 (python):
```python
sc.pl.embedding(adata,basis='X_umap',
                color='fine_grain_manual',
                frameon=False,
                wspace=0.3)
```

Example 59 (unknown):
```unknown
... storing 'fine_grain_manual' as categorical
```

Example 60 (python):
```python
sc.tl.dendrogram(adata,groupby='fine_grain_manual')
sc.pl.dotplot(adata,
              var_names=markers,
              groupby='fine_grain_manual',
              standard_scale='var',
              dendrogram=True
             )
```

Example 61 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: EndocardialCells, EndocardialCushionCells, Erythrocytes, etc.
var_group_labels: EndocardialCushionCells, ValveEndothelialCells, EndocardialCells, etc.
```

Example 62 (python):
```python
# save
adata.write(f'{adata_dir}/{sample_id}_{celltype}_b2c_cells_filtered_lognorm.h5ad')
print(f'{adata_dir}/{sample_id}_{celltype}_b2c_cells_filtered_lognorm.h5ad')
```

Example 63 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/subsets/HEA_FOET14880396_EndocardialCells_b2c_cells_filtered_lognorm.h5ad
```

---

## 

**URL:** http://127.0.0.1:9020/notebooks_html/Fig4_Cardiomyocytes/AtrialCardiomyocytes/Visium_mapping/Atrial_Proportions_BoxPlot_ratio.html

4100 rows × 8 columns

4100 rows × 9 columns

4100 rows × 15 columns

4100 rows × 15 columns

4100 rows × 15 columns

2523 rows × 15 columns

16400 rows × 3 columns

16400 rows × 3 columns

**Examples:**

Example 1 (python):
```python
import pandas as pd
import plotly.express as px
```

Example 2 (python):
```python
import matplotlib
matplotlib.rcParams['pdf.fonttype'] = 42
matplotlib.rcParams['ps.fonttype'] = 42
```

Example 3 (python):
```python
OCT = pd.read_csv('AtrialProportionsOCT.csv')
OCT['modality']="OCT"
FFPE = pd.read_csv('AtrialProportionsFFPE.csv')
FFPE['modality']="FFPE"
df=pd.concat([OCT,FFPE])
df
```

Example 4 (python):
```python
def age_to_days(age_str):
    # Split the string into parts
    age_type, age_time = age_str.split('-')

    # Initialize weeks and days
    weeks = 0
    days = 0

    # Extract weeks and days from the age_time string
    if 'w' in age_time:
        week_day_parts = age_time.split('w')
        weeks = int(week_day_parts[0])  # Convert weeks to integer
        if len(week_day_parts) > 1 and 'd' in week_day_parts[1]:
            days = int(week_day_parts[1].replace('d', ''))  # Convert days to integer

    # Convert gestational age (GA) to post conception weeks (PCW)
    if age_type == 'GA':
        weeks -= 2

    # Convert weeks and days to total days
    total_days = weeks * 7 + days
    return total_days

# Apply the conversion to the DataFrame
df['days_post_conception'] = df['age'].apply(age_to_days)
df
```

Example 5 (unknown):
```unknown
df.days_post_conception = df.days_post_conception.astype('category')
df.days_post_conception = df.days_post_conception.astype(int)
```

Example 6 (python):
```python
# Adding a new column 'post_conception_weeks' based on the 'days post conception' column
def days_to_weeks_days(days):
    weeks = days // 7
    days_remaining = days % 7
    return f"{weeks}w{days_remaining}d"

df['post_conception_weeks'] = df['days_post_conception'].apply(days_to_weeks_days)

# Displaying the first few rows of the DataFrame to verify the new column
df.head()
```

Example 7 (unknown):
```unknown
# Verifying the changes by inspecting the value counts again
df['post_conception_weeks'].value_counts()
```

Example 8 (unknown):
```unknown
16w0d    2045
7w4d     1210
13w0d     682
5w4d      163
Name: post_conception_weeks, dtype: int64
```

Example 9 (python):
```python
# Correcting the order of the categories for the 'post_conception_weeks' column
correct_order_categories = ['16w0d', '13w0d','7w4d','5w4d']
correct_order_categories = ['5w4d', '7w4d','13w0d','16w0d']
df['post_conception_weeks'] = pd.Categorical(values=df['post_conception_weeks'], categories=correct_order_categories, ordered=True)

# Verifying the changes by inspecting the value counts again
df['post_conception_weeks'].dtypes
```

Example 10 (unknown):
```unknown
CategoricalDtype(categories=['5w4d', '7w4d', '13w0d', '16w0d'], ordered=True)
```

Example 11 (unknown):
```unknown
# Assuming 'data' is your pandas DataFrame
# Calculate the total for Left and Right cells separately
df['AtrialCardiomyocytesLeft_immature'] + df['AtrialCardiomyocytesLeft']
```

Example 12 (unknown):
```unknown
0        2.222819
1        4.122041
2        9.167294
3        1.971105
4        2.066072
          ...    
2040     3.813929
2041     1.510159
2042    16.247391
2043     3.719476
2044    21.171020
Length: 4100, dtype: float64
```

Example 13 (unknown):
```unknown
# Assuming 'data' is your pandas DataFrame
# Calculate the total for Left and Right cells separately
total_left = df['AtrialCardiomyocytesLeft_immature'] + df['AtrialCardiomyocytesLeft']
total_right = df['AtrialCardiomyocytesRight_immature'] + df['AtrialCardiomyocytesRight']

# Calculate the proportions for each cell type relative to their side
df['AtrialCardiomyocytesLeft_immature_prop'] = df['AtrialCardiomyocytesLeft_immature'] / total_left
df['AtrialCardiomyocytesLeft_prop'] = df['AtrialCardiomyocytesLeft'] / total_left

df['AtrialCardiomyocytesRight_immature_prop'] = df['AtrialCardiomyocytesRight_immature'] / total_right
df['AtrialCardiomyocytesRight_prop'] = df['AtrialCardiomyocytesRight'] / total_right

# Check the new columns in the DataFrame
df[['AtrialCardiomyocytesLeft_immature_prop', 'AtrialCardiomyocytesLeft_prop', 'AtrialCardiomyocytesRight_immature_prop', 'AtrialCardiomyocytesRight_prop']]
df
```

Example 14 (unknown):
```unknown
df.AtrialCardiomyocytesLeft_immature.hist(bins=50)
```

Example 15 (python):
```python
import seaborn as sns
import matplotlib.pyplot as plt
```

Example 16 (python):
```python
import pandas as pd

# Specify the columns to check for the condition
columns_to_check = [
    'AtrialCardiomyocytesLeft_immature',
    'AtrialCardiomyocytesLeft',
    'AtrialCardiomyocytesRight',
    'AtrialCardiomyocytesRight_immature'
]

# Add 'filter_out' column based on the condition
df['filter_out'] = df[columns_to_check].apply(lambda x: (x < 0.01).any(), axis=1)
df
```

Example 17 (unknown):
```unknown
df[df['filter_out']==False]
```

Example 18 (unknown):
```unknown
# Assuming 'data' is your DataFrame and it already includes an 'age' column
df_long = df.melt(id_vars='post_conception_weeks', var_name='cell_type', value_name='proportion', 
                    value_vars=['AtrialCardiomyocytesLeft_immature_prop', 'AtrialCardiomyocytesLeft_prop', 
                                'AtrialCardiomyocytesRight_prop', 'AtrialCardiomyocytesRight_immature_prop'])
df_long
```

Example 19 (unknown):
```unknown
#df_long.replace(to_replace='AtrialCardiomyocytesLeft_immature_prop', value='AtrialCardiomyocytesLeft_immature', inplace=True)
df_long.replace(to_replace='AtrialCardiomyocytesLeft_prop', value='AtrialCardiomyocytesLeft_mature_prop', inplace=True)
#df_long.replace(to_replace='AtrialCardiomyocytesRight_immature_prop', value='AtrialCardiomyocytesRight_immature', inplace=True)
df_long.replace(to_replace='AtrialCardiomyocytesRight_prop', value='AtrialCardiomyocytesRight_mature_prop', inplace=True)
df_long
```

Example 20 (python):
```python
# Define the desired order of 'cell_type' labels
cell_type_order = [
    'AtrialCardiomyocytesLeft_immature_prop', 
    'AtrialCardiomyocytesLeft_mature_prop', 
    'AtrialCardiomyocytesRight_immature_prop', 
    'AtrialCardiomyocytesRight_mature_prop'
]

# Convert the 'cell_type' column to a categorical type with the specified order
df_long['cell_type'] = pd.Categorical(df_long['cell_type'], categories=cell_type_order, ordered=True)

# and the legend will be in the second row.
fig, (ax_main, ax_legend) = plt.subplots(2, 1, figsize=(10, 8), 
                                         gridspec_kw={'height_ratios': [4, 1], 'hspace': 0.05})

# The main strip plot without specifying legend=False
sns.stripplot(data=df_long, x='post_conception_weeks', y='proportion', hue='cell_type', dodge=True, ax=ax_main)

# Hide the legend created by seaborn in the main plot
ax_main.get_legend().remove()

# Extract handles and labels for the legend after creating the plot
handles, labels = ax_main.get_legend_handles_labels()

# Creating the legend in the subplot with 2 columns
ax_legend.legend(handles, labels, loc='center', ncol=2, frameon=False)
ax_legend.axis('off')  # Hide the axes of the legend subplot

plt.show()
```

Example 21 (python):
```python
# Creating the figure and axes for the plot and the legend
fig, (ax_main, ax_legend) = plt.subplots(2, 1, figsize=(10, 8), 
                                         gridspec_kw={'height_ratios': [4, 1], 'hspace': 0.05})

# Plotting the box plot first
sns.boxplot(data=df_long, x='post_conception_weeks', y='proportion', hue='cell_type', ax=ax_main, 
            fliersize=0,  # Optionally hide the outliers in the boxplot to avoid overlap with the stripplot points
            dodge=True, saturation = 1)

# Adding the strip plot on top
sns.stripplot(data=df_long, x='post_conception_weeks', y='proportion', hue='cell_type', dodge=True, ax=ax_main,
              alpha=0.3)

# Hide the legend created by seaborn in the main plot to avoid duplication
ax_main.get_legend().remove()

# Extract handles and labels for the legend after creating the plot
handles, labels = ax_main.get_legend_handles_labels()

# Creating the legend in the subplot with 2 columns
ax_legend.legend(handles[:len(cell_type_order)], labels[:len(cell_type_order)], loc='center', ncol=2, frameon=False)
ax_legend.axis('off')  # Hide the axes of the legend subplot
plt.tight_layout()
plt.savefig("./figures/aCMs_immature_proportion_boxplots.pdf",bbox_inches='tight')
plt.show()
```

Example 22 (python):
```python
/tmp/ipykernel_167508/380920426.py:23: UserWarning: This figure includes Axes that are not compatible with tight_layout, so results might be incorrect.
  plt.tight_layout()
```

---

## Import modules¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/RNA/scnRNA_Integration-Annotation/No3_global_UMAP-clustering.html

**Contents:**
- Import modules¶
- Read in data¶
- Clustering, UMAP, check modalities, QCs¶
- Proportions¶
- Reconstruct raw counts¶
- Save¶

**Examples:**

Example 1 (python):
```python
import numpy as np
import numpy.random as random
import pandas as pd
import scanpy as sc
import anndata
import matplotlib.pyplot as plt
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
-----
anndata             0.7.5
matplotlib          3.3.4
numpy               1.20.1
pandas              1.1.4
scanpy              1.7.1
session_info        1.0.0
-----
```

Example 4 (unknown):
```unknown
PIL                 8.1.2
anyio               NA
attr                20.3.0
babel               2.9.0
backcall            0.2.0
brotli              NA
cairo               1.20.0
certifi             2020.12.05
cffi                1.14.5
chardet             4.0.0
cloudpickle         1.6.0
colorama            0.4.4
cycler              0.10.0
cython_runtime      NA
cytoolz             0.11.0
dask                2021.03.1
dateutil            2.8.1
decorator           4.4.2
fsspec              0.8.7
get_version         2.1
google              NA
h5py                2.10.0
idna                2.10
igraph              0.8.3
ipykernel           5.5.0
ipython_genutils    0.2.0
jedi                0.18.0
jinja2              2.11.3
joblib              1.0.1
json5               NA
jsonschema          3.2.0
jupyter_server      1.4.1
jupyterlab_server   2.3.0
kiwisolver          1.3.1
legacy_api_wrap     0.0.0
leidenalg           0.8.3
llvmlite            0.34.0
louvain             0.7.0
markupsafe          1.1.1
mpl_toolkits        NA
natsort             7.1.1
nbclassic           NA
nbformat            5.1.2
numba               0.51.2
numexpr             2.7.3
packaging           20.9
parso               0.8.1
pexpect             4.8.0
pickleshare         0.7.5
pkg_resources       NA
prometheus_client   NA
prompt_toolkit      3.0.16
psutil              5.8.0
ptyprocess          0.7.0
pvectorc            NA
pyarrow             0.16.0
pygments            2.8.0
pyparsing           2.4.7
pyrsistent          NA
pytoml              NA
pytz                2021.1
requests            2.25.1
ruamel              NA
scipy               1.6.1
send2trash          NA
setuptools_scm      NA
simplejson          3.17.2
sinfo               0.3.1
six                 1.15.0
sklearn             0.22
sniffio             1.2.0
socks               1.7.1
sphinxcontrib       NA
storemagic          NA
tables              3.6.1
tblib               1.7.0
texttable           1.6.3
tlz                 0.11.0
toolz               0.11.1
tornado             6.1
traitlets           5.0.5
typing_extensions   NA
uritemplate         3.0.1
urllib3             1.26.3
wcwidth             0.2.5
yaml                5.1.2
zmq                 22.0.3
```

Example 5 (unknown):
```unknown
-----
IPython             7.21.0
jupyter_client      6.1.11
jupyter_core        4.7.1
jupyterlab          3.0.9
notebook            6.2.0
-----
Python 3.8.8 | packaged by conda-forge | (default, Feb 20 2021, 16:22:27) [GCC 9.3.0]
Linux-4.15.0-112-generic-x86_64-with-glibc2.10
-----
Session information updated at 2023-02-16 10:29
```

Example 6 (python):
```python
sc.settings.verbosity = 3             # verbosity: errors (0), warnings (1), info (2), hints (3)
sc.settings.set_figure_params(dpi=200, color_map = 'RdPu',)
```

Example 7 (python):
```python
adata = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/scVI/RNA_foetal_scVI.h5ad')
adata
```

Example 8 (unknown):
```unknown
AnnData object with n_obs × n_vars = 372801 × 3343
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', '_scvi_labels'
    var: 'gene_name_scRNA-0', 'gene_name_multiome-1', 'n_cells', 'highly_variable', 'means', 'dispersions', 'dispersions_norm', 'highly_variable_nbatches', 'highly_variable_intersection'
    uns: '_scvi', 'hvg'
    obsm: 'X_scVI', '_scvi_extra_continuous', 'latent_gene_encoding'
    layers: 'counts'
```

Example 9 (unknown):
```unknown
genename_column='gene_name_multiome-1'
```

Example 10 (python):
```python
# add previous annotation
prev=sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/2JanCell2locObjectRaw.h5ad',backed='r')

adata.obs['2Jan_coarse_grain'] = prev.obs['coarse_grain'].reindex(adata.obs_names)
adata.obs['2Jan_mid_grain'] = prev.obs['mid_grain'].reindex(adata.obs_names)
adata.obs['2Jan_fine_grain'] = prev.obs['fine_grain'].reindex(adata.obs_names)
```

Example 11 (python):
```python
pd.crosstab(adata.obs['donor'],adata.obs['age'])
```

Example 12 (python):
```python
%%time
sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata, min_dist=0.2)
sc.tl.leiden(adata, resolution=1, key_added='leiden_scVI',)
```

Example 13 (python):
```python
computing neighbors
    finished: added to `.uns['neighbors']`
    `.obsp['distances']`, distances for each pair of neighbors
    `.obsp['connectivities']`, weighted adjacency matrix (0:01:10)
computing UMAP
    finished: added
    'X_umap', UMAP coordinates (adata.obsm) (0:07:45)
running Leiden clustering
    finished: found 48 clusters and added
    'leiden_scVI', the cluster labels (adata.obs, categorical) (0:06:27)
```

Example 14 (unknown):
```unknown
CPU times: user 35min 35s, sys: 10min 35s, total: 46min 11s
Wall time: 15min 23s
```

Example 15 (python):
```python
sc.settings.set_figure_params(dpi=200, facecolor='white', color_map = 'RdPu',)
sc.pl.umap(adata, color=['leiden_scVI','2Jan_mid_grain'], size=2, wspace=0.1, legend_loc='on data', legend_fontsize=5)
```

Example 16 (python):
```python
sc.settings.set_figure_params(dpi=200, facecolor='white', color_map = 'RdPu',)
sc.pl.umap(adata, color=['cell_or_nuclei','kit_10x','donor'], size=2, wspace=0.1, legend_loc='on data', legend_fontsize=5)
```

Example 17 (python):
```python
sc.pl.umap(adata, color=['2Jan_fine_grain'], size=2, wspace=0.1, legend_loc='on data', legend_fontsize=3)
```

Example 18 (python):
```python
sc.pl.umap(adata, color=['age'], size=1, legend_fontsize=7)
```

Example 19 (python):
```python
sc.pl.umap(adata, color=['region'], size=1, legend_fontsize=7)
```

Example 20 (python):
```python
sc.pl.umap(adata, color=['multiplexed'], groups='Yes', size=1, legend_fontsize=7)
```

Example 21 (python):
```python
sc.pl.umap(adata, color=['SOC | status'], groups='doublet', size=1, legend_fontsize=7)
```

Example 22 (python):
```python
sc.pl.umap(adata, color=['region'], groups=['Pericardium','Aorta','Left ventricle','Pulmonary artery + main bronchus'], size=2, legend_loc='on data', legend_fontsize=5)
```

Example 23 (python):
```python
sc.pl.umap(adata, color=['age'], groups=['PCW-12w','PCW-13w','PCW-14w','PCW-15w','PCW-20w'], size=2, legend_loc='on data', legend_fontsize=5, ncols=5)
```

Example 24 (python):
```python
sc.pl.umap(adata, color=['donor','cell_or_nuclei','kit_10x','facility'], size=0.5, wspace=0.6)
```

Example 25 (python):
```python
sc.settings.set_figure_params(dpi=200, color_map = 'RdPu',)
sc.pl.umap(adata, color=['n_genes', 'n_genes_by_counts',
       'total_counts','pct_counts_mt','pct_counts_ribo'], size=2, wspace=0.2, ncols=5)
```

Example 26 (python):
```python
sc.pl.umap(adata, color=['scrublet_score','cluster_scrublet_score','doublet_pval','doublet_bh_pval'], size=2, wspace=0.2, ncols=5)
```

Example 27 (python):
```python
plt.rcParams['figure.figsize'] = 5,5
tmp = pd.crosstab(adata.obs['kit_10x'], adata.obs['age'], normalize='index')
# tmp.plot.bar().legend(loc=1)
ax = tmp.plot.bar(stacked=True,width=0.7)
ax.grid(False)
ax.set_ylabel('proportion')
ax.legend(bbox_to_anchor=(1.0, 1.0))
ax.plot()
```

Example 28 (python):
```python
adata_raw=sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/RNA_foetal_raw.h5ad')
print(adata_raw)

adata_raw=anndata.AnnData(X=adata_raw[adata.obs_names,:].X,
                      obs=adata.obs, # from annotated data
                      var=adata_raw.var,
                      obsm=adata.obsm, # from annotated data
                      obsp=adata.obsp # from annotated data
                               )
print(adata_raw)
```

Example 29 (unknown):
```unknown
AnnData object with n_obs × n_vars = 372801 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet'
    var: 'gene_name_scRNA-0', 'gene_name_multiome-1'
    obsm: 'latent_gene_encoding'
AnnData object with n_obs × n_vars = 372801 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', '_scvi_labels', '2Jan_coarse_grain', '2Jan_mid_grain', '2Jan_fine_grain', 'leiden_scVI'
    var: 'gene_name_scRNA-0', 'gene_name_multiome-1'
    obsm: 'X_scVI', '_scvi_extra_continuous', 'latent_gene_encoding', 'X_umap'
    obsp: 'distances', 'connectivities'
```

Example 30 (python):
```python
### add CD45 sorting key ###
cd45 = pd.read_csv('/nfs/team205/heart/anndata_objects/Foetal/metadata/5prime_FACS.csv')
cd45.rename(columns={'FACS gate':'FACSgate'},inplace=True)
print(cd45)

adata_raw.obs = adata_raw.obs.reset_index().merge(cd45,how='left',on='sangerID').set_index('barcode')
adata_raw.obs[['FACSgate','sangerID']].value_counts()
```

Example 31 (unknown):
```unknown
sangerID FACSgate
0  BHF_F_Hea10402917  CD45pos
1  BHF_F_Hea10402918  CD45neg
2  BHF_F_Hea11192323  CD45pos
3  BHF_F_Hea11192324  CD45neg
4  BHF_F_Hea11192325  CD45pos
5  BHF_F_Hea11192326  CD45neg
6  BHF_F_Hea11192327  CD45pos
7  BHF_F_Hea11192328  CD45neg
```

Example 32 (unknown):
```unknown
FACSgate  sangerID         
CD45neg   BHF_F_Hea11192328    12056
          BHF_F_Hea11192324    10553
          BHF_F_Hea11192326     9876
CD45pos   BHF_F_Hea11192325     7512
          BHF_F_Hea11192327     5865
CD45neg   BHF_F_Hea10402918     4340
CD45pos   BHF_F_Hea11192323     3418
          BHF_F_Hea10402917     2227
dtype: int64
```

Example 33 (python):
```python
adata_raw.obs['FACSgate'].unique()
```

Example 34 (unknown):
```unknown
array(['CD45pos', 'CD45neg', nan], dtype=object)
```

Example 35 (python):
```python
adata_raw.write('/nfs/team205/heart/anndata_objects/Foetal/RNA_foetal_post-scVI_raw.h5ad')
adata_raw
```

Example 36 (unknown):
```unknown
... storing 'sangerID' as categorical
... storing 'FACSgate' as categorical
```

Example 37 (unknown):
```unknown
AnnData object with n_obs × n_vars = 372801 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', '_scvi_labels', '2Jan_coarse_grain', '2Jan_mid_grain', '2Jan_fine_grain', 'leiden_scVI', 'FACSgate'
    var: 'gene_name_scRNA-0', 'gene_name_multiome-1'
    obsm: 'X_scVI', '_scvi_extra_continuous', 'latent_gene_encoding', 'X_umap'
    obsp: 'distances', 'connectivities'
```

Example 38 (python):
```python
adata_raw=sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/RNA_foetal_post-scVI_raw.h5ad')
adata_raw
```

Example 39 (unknown):
```unknown
AnnData object with n_obs × n_vars = 372801 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', '_scvi_labels', '2Jan_coarse_grain', '2Jan_mid_grain', '2Jan_fine_grain', 'leiden_scVI', 'FACSgate'
    var: 'gene_name_scRNA-0', 'gene_name_multiome-1'
    obsm: 'X_scVI', 'X_umap', '_scvi_extra_continuous', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 40 (python):
```python
adata_raw[adata_raw.obs['2Jan_coarse_grain']=='Leukocytes'].obs['age'].value_counts()
```

Example 41 (unknown):
```unknown
PCW-20w     21354
GA-10w4d     5353
GA-9w4d      3025
11w3d        1987
PCW-13w      1728
GA-12w5d     1627
9w6d          890
9w3d          604
8w5d          520
GA-7w4d       482
8w2d          398
9w1d          376
GA-9w         359
GA-11w2d      101
GA-11w3d       62
GA-6w3d        40
Name: age, dtype: int64
```

Example 42 (python):
```python
adata_raw.obs['2Jan_coarse_grain'].value_counts()
```

Example 43 (unknown):
```unknown
Mesenchymal       128528
Cardiomyocytes     96591
Leukocytes         38906
Endothelium        20122
Neural              5998
Epicardium          3879
Name: 2Jan_coarse_grain, dtype: int64
```

Example 44 (python):
```python
set(adata_raw.obs['cell_or_nuclei'])
```

Example 45 (unknown):
```unknown
{'Cell', 'Nuclei'}
```

Example 46 (python):
```python
import sys
sys.getsizeof(adata_raw)
```

Example 47 (unknown):
```unknown
# rclone to gdrive
rclone copy /nfs/team205/heart/anndata_objects/Foetal/RNA_foetal_post-scVI_raw.h5ad \
gdrive:JamesC_Kazumasa/Foetal_Heart/SanjayShinha/toSinhalab/anndata/
```

Example 48 (python):
```python
adata_raw.obs['region'].value_counts()
```

Example 49 (python):
```python
sc.pl.umap(adata_raw, color=['region'], groups=['Pericardium','Aorta','Left ventricle','Pulmonary artery  + main bronchus'], size=2,legend_fontsize=7)
```

Example 50 (python):
```python
sc.pl.umap(adata_raw, color=['region'], groups=['Pericardium'], size=2,legend_fontsize=7)
```

Example 51 (python):
```python
sc.pl.umap(adata_raw[adata_raw.obs['region']!='Pulmonary artery  + main bronchus'], 
           color=['previous_mid_grain'], size=2,legend_fontsize=7)
```

Example 52 (python):
```python
adata_raw[adata_raw.obs['age']=='PCW-20w']
```

Example 53 (python):
```python
ad = adata_raw[adata_raw.obs['age']=='PCW-20w']

plt.rcParams['figure.figsize'] = 5,5
tmp = pd.crosstab(ad.obs['previous_mid_grain'], ad.obs['region'], normalize='index')
# tmp.plot.bar().legend(loc=1)
ax = tmp.plot.bar(stacked=True,width=0.7)
ax.grid(False)
ax.set_ylabel('proportion')
ax.legend(bbox_to_anchor=(1.0, 1.0))
ax.plot()
```

Example 54 (python):
```python
# filter and lognormalised
sc.pp.filter_genes(adata_raw, min_cells=3)
sc.pp.normalize_total(adata_raw, target_sum=1e4)
sc.pp.log1p(adata_raw)
```

Example 55 (python):
```python
# set gene name as var_names 
adata_raw.var['gene_id']=adata_raw.var_names.copy()
adata_raw.var.set_index('gene_name_multiome-1',inplace=True)
adata_raw.var_names=adata_raw.var_names.astype('str')
adata_raw.var_names_make_unique()
```

Example 56 (python):
```python
sc.pl.umap(adata_raw, 
           color=['FOXP3','CD4','CD8A','CD3E'], size=2,legend_fontsize=7)
```

Example 57 (python):
```python
for reg in set(adata_raw.obs['region']):
    print(reg)
    sc.pl.umap(adata_raw[adata_raw.obs['region']==reg], 
           color=['FOXP3','CD4','CD8A','CD3E'], size=2,legend_fontsize=7)
```

Example 58 (python):
```python
set(adata_raw.obs['region'])
```

Example 59 (python):
```python
pd.crosstab(ad.obs['previous_mid_grain'], ad.obs['region'], normalize='index')
```

---

## 

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/RNA/scnRNA_Integration-Annotation/Annotations/NeuralAnnotation.html

**Examples:**

Example 1 (unknown):
```unknown
#Neural Annotation
```

Example 2 (python):
```python
# Import packages, initial settings

import numpy as np
import matplotlib.pyplot as plt
import scanpy as sc
import pandas as pd
import anndata
import scrublet as scr
import seaborn as sns
import magic
import pybiomart
import six
import sys
sys.modules['sklearn.externals.six'] = six
import scanorama
import harmonypy
import bbknn
import gseapy
from gseapy.plot import barplot, dotplot
import scvelo as scv
import scvi

plt.rcParams['pdf.fonttype'] = 42
plt.rcParams['ps.fonttype'] = 42

sc.settings.verbosity = 3  # verbosity: errors (0), warnings (1), info (2), hints (3)

from matplotlib.colors import LinearSegmentedColormap
cmap = LinearSegmentedColormap.from_list(name='gene_cmap', colors=['lightgrey', 'thistle', 'red', 'darkred']) 

sc.settings.set_figure_params(dpi=80, color_map='viridis', vector_friendly=False,  dpi_save=300, facecolor='white')
sc.logging.print_version_and_date()
#sc.logging.print_versions()
```

Example 3 (unknown):
```unknown
Global seed set to 0
```

Example 4 (unknown):
```unknown
Running Scanpy 1.8.1, on 2022-12-02 16:35.
```

Example 5 (python):
```python
adata=sc.read('./Objects/20221130RNA_foetal_post-scVI_raw_ExtracardiacRemoved.h5ad')
print(adata.shape)
```

Example 6 (unknown):
```unknown
(306287, 36601)
```

Example 7 (python):
```python
cell_cycle_genes = ['MCM5','PCNA','TYMS','FEN1','MCM2','MCM4','RRM1','UNG','GINS2','MCM6','CDCA7','DTL','PRIM1','UHRF1','MLF1IP','HELLS','RFC2','RPA2','NASP','RAD51AP1','GMNN','WDR76','SLBP','CCNE2','UBR7','POLD3','MSH2','ATAD2','RAD51','RRM2','CDC45','CDC6','EXO1','TIPIN','DSCC1','BLM','CASP8AP2','USP1','CLSPN','POLA1','CHAF1B','BRIP1','E2F8','HMGB2','CDK1','NUSAP1','UBE2C','BIRC5','TPX2','TOP2A','NDC80','CKS2','NUF2','CKS1B','MKI67','TMPO','CENPF','TACC3','FAM64A','SMC4','CCNB2','CKAP2L','CKAP2','AURKB','BUB1','KIF11','ANP32E','TUBB4B','GTSE1','KIF20B','HJURP','CDCA3','HN1','CDC20','TTK','CDC25C','KIF2C','RANGAP1','NCAPD2','DLGAP5','CDCA2','CDCA8','ECT2','KIF23','HMMR','AURKA','PSRC1','ANLN','LBR','CKAP5','CENPE','CTCF','NEK2','G2E3','GAS2L3','CBX5','CENPA']
s_genes = cell_cycle_genes[:43]
s_genes = [x for x in s_genes if x in adata.var_names]
print('S genes:', s_genes, '\n')
g2m_genes = cell_cycle_genes[43:]
g2m_genes = [x for x in g2m_genes if x in adata.var_names]
print('G2 genes:', g2m_genes, '\n')
cell_cycle_genes = [x for x in cell_cycle_genes if x in adata.var_names]

hb_genes1 = ['HBB','HBD','HBG1','HBG2','HBE1','HBZ','HBM','HBA2','HBA1', 'HBQ1']
hb_genes1 = [x for x in hb_genes1 if x in adata.var_names]
print('Hemoglobin genes set 1:', hb_genes1, '\n')

hb_genes2 = ['AHSP','GYPA','RHAG','GFI1B','KEL','EPB42','ALAS2','TSPO2','GYPB','GYPE','SPTA1','RHCE','KLF1','RHD','SLC4A1','SNX22','SQSTM1','SOD1','FTH1','MAP1LC3B','SLC25A37','TRIM58','HMBS']
hb_genes2 = [x for x in hb_genes2 if x in adata.var_names]
print('Hemoglobin genes set 2:', hb_genes2, '\n')

mito_genes=['MT-ND1','MT-ND2','MT-CO1','MT-CO2','MT-ATP8','MT-ATP6','MT-CO3','MT-ND3','MT-ND4L', 'MT-ND4', 'MT-ND5', 'MT-ND6', 'MT-CYB']
mito_genes = [x for x in mito_genes if x in adata.var_names]
print('Mitochondrial genes:', mito_genes, '\n')

ribo_genes = ['RPLP0','RPLP1','RPLP2','RPL3','RPL3L','RPL4','RPL5','RPL6','RPL7','RPL7A','RPL7L1','RPL8','RPL9','RPL10','RPL10A','RPL10L','RPL11','RPL12','RPL13','RPL13A','RPL14','RPL15','RPL17','RPL18','RPL18A','RPL19','RPL21','RPL22','RPL22L1','RPL23','RPL23A','RPL24','RPL26','RPL26L1','RPL27','RPL27A','RPL28','RPL29','RPL30','RPL31','RPL32','RPL34','RPL35','RPL35A','RPL36','RPL36A','RPL36AL','RPL37','RPL37A','RPL38','RPL39','RPL39L','UBA52','RPL41','RPSA','RPS2','RPS3','RPS3A','RPS4X','RPS4Y1','RPS4Y2','RPS5','RPS6','RPS7','RPS8','RPS9','RPS10','RPS11','RPS12','RPS13','RPS14','RPS15','RPS15A','RPS16','RPS17','RPS18','RPS19','RPS20','RPS21','RPS23','RPS24','RPS25','RPS26','RPS27','RPS27A','RPS27L','RPS28','RPS29','FAU']
ribo_genes = [x for x in ribo_genes if x in adata.var_names]
print('Ribo genes:', ribo_genes, '\n')

stress_genes=['FOS','FOSB','JUN','JUNB','JUND','ZFP36','SOCS3','CEBPD','CEBPB','ATF3','EGR1','HSPA1A','HSPA1B','HSP90AA1','HSP90AB1','HSPA8','HSPB1','HSPE1','IER3','IER2','BTG1','BTG2','DUSP1']
stress_genes = [x for x in stress_genes if x in adata.var_names]
print('Tissue dissociation stress associated genes:', stress_genes, '\n')

FoxProteins=['FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1']
chrY_genes=['SRY', 'RPS4Y1', 'ZFY', 'ZFY-AS1', 'LINC00278', 'PCDH11Y', 'TTTY2B', 'TBL1Y', 'PRKY', 'USP9Y', 'DDX3Y', 'UTY', 'TMSB4Y', 'NLGN4Y', 'TTTY14', 'KDM5D', 'TTTY10', 'EIF1AY', 'PRORY', 'LINC00266-4P']
MembraneProteins1=['AADAT','AAK1','ABCA12','ABCA7','ABCB1','ABCB11','ABCB4','ABCB6','ABCC1','ABCC3','ABCC4','ABCC5','ABCC6','ABCG2','ABHD3','ABI1','ABLIM3','AC004691.2','AC004922.1','AC005324.3','AC005726.1','AC008403.1','AC138696.1','ACIN1','ACKR3','ACLY','ACOT13','ACOT9','ACTN1','ACTR10','ACTR6','ADA','ADAM10','ADAM12','ADAM22','ADAM28','ADAM33','ADAMTS1','ADAMTS2','ADAMTSL4','ADAP1','ADCK5','ADD1','ADD2','ADGRB2','ADGRD1','ADGRG2','ADGRG7','ADGRL2','ADH1A','ADH1B','ADH1C','ADH7','ADK','ADORA1','ADRA1B','ADRM1','ADSS','AFAP1L2','AFDN','AFG1L','AGER','AGRN','AHNAK','AHNAK2','AIFM2','AKAP10','AKAP11','AKAP12','AKR1B10','AKR1B15','AL132780.3','AL139300.1','AL355315.1','AL358113.1','AL645922.1','ALDH3A1','ALK','ALOX15B','ALOXE3','ALPI','ALPP','ALPPL2','ALS2CR12','AMACR','AMDHD1','AMER1','AMER2','AMIGO2','AMOT','AMOTL1','AMOTL2','AMPH','ANK2','ANK3','ANKEF1','ANKFN1','ANKIB1','ANKLE2','ANKRA2','ANKRD13A','ANKRD35','ANKRD42','ANKRD45','ANKS1B','ANO1','ANO10','ANO2','ANO6','ANPEP','ANXA1','ANXA13','ANXA2','ANXA3','AP2M1','APBB1','APBB1IP','APC','APLP1','APMAP','APOC3','APOD','APOLD1','AQP1','AQP10','AQP3','AQP4','AQP5','ARAP1','ARF1','ARHGAP18','ARHGAP19','ARHGAP21','ARHGAP22','ARHGAP23','ARHGAP24','ARHGAP28','ARHGAP29','ARHGAP33','ARHGAP36','ARHGAP40','ARHGAP9','ARHGDIG','ARHGEF1','ARHGEF11','ARHGEF12','ARHGEF15','ARHGEF26','ARHGEF28','ARHGEF40','ARHGEF5','ARID2','ARID4A','ARID5B','ARL13A','ARL15','ARL4C','ARMC12','ARMCX5','ARPC1A','ARPC5','ARRB2','ARRDC4','ARVCF','ASAP1','ASB5','ASGR1','ASGR2','ASH2L','ASIC1','ASIC4','ASPM','ASPSCR1','ATG13','ATG3','ATG7','ATIC','ATOX1','ATP10D','ATP12A','ATP13A3','ATP1B3','ATP23','ATP2B1','ATP2B4','ATP2C2','ATP6V0A2','ATP6V1H','ATP8A2','ATP8B1','ATXN10','ATXN3','ATXN7L3','AVEN','AVPI1','AXIN2','AXL','B2M','B3GALT2','BACE1','BAG3','BAIAP2','BAIAP2L1','BANK1','BASP1','BBOF1','BCAR1','BCAR3','BCL2L12','BCORL1','BCR','BDKRB2','BEND6','BFSP1','BFSP2','BICD2','BIN2','BLNK','BLVRB','BMPR2','BMX','BOC','BOLL','BORCS5','BORCS8-MEF2B','BRK1','BRPF1','BRSK1','BSDC1','BST2','BTK','BTN2A1','BTRC','BUB1B-PAK6','BVES','BZW2','C10orf90','C11orf42','C11orf52','C12orf73','C12orf76','C14orf37','C14orf93','C16orf74','C17orf58','C17orf62','C18orf21','C19orf33','C19orf38','C19orf48','C19orf71','C1orf116','C1orf189','C1orf35','C1orf54','C1orf68','C1orf74','C1QBP','C2orf72','C3orf14','C3orf20','C3orf38','C4orf19','C4orf22','C7orf43','C8orf33','C8orf37','C9orf153','C9orf3','CA14','CA9','CABLES2','CACNA1E','CACNG4','CADM1','CALCRL','CALD1','CALML5','CALN1','CAMK2A','CAMK2B','CAMK2D','CAMK2G','CAP2','CAPS','CASKIN2','CASP4','CAST','CAV3','CAVIN1','CAVIN2','CAVIN3','CAVIN4','CBWD1','CBWD2','CBWD3','CBWD5','CBWD6','CC2D1A','CCBE1','CCDC107','CCDC112','CCDC115','CCDC124','CCDC136','CCDC144NL','CCDC15','CCDC154','CCDC170','CCDC24','CCDC28A','CCDC59','CCDC62','CCDC66','CCDC69','CCDC8','CCDC83','CCDC85C','CCDC93','CCDC97','CCNT2','CCNYL1','CCR3','CCR4','CCT3','CD109','CD163','CD164L2','CD200R1','CD226','CD248','CD274','CD2AP','CD300A','CD300LB','CD33','CD34','CD37','CD38','CD3E','CD3G','CD4','CD44','CD46','CD47','CD48','CD79B','CD81','CD84','CD86','CD8A','CD8B','CD9','CD93','CD99L2','CDAN1','CDC42EP1','CDC42EP3','CDC42EP4','CDC42SE1','CDCA4','CDH1','CDH13','CDH15','CDH17','CDH2','CDH26','CDH3','CDH4','CDH5','CDH6','CDH8','CDIPT','CDK16','CDK5','CDK5RAP2','CDK7','CDRT15','CDV3','CEACAM5','CELSR1','CENPJ','CEP162','CEP295','CEP55','CEP57L1','CEP89','CERCAM','CFAP157','CFAP206','CFAP58','CFAP65','CFB','CFL2','CFLAR','CGN','CGNL1','CHCHD7','CHIC2','CHMP2A','CHRM3','CHRNA1','CHRNA5','CHRNB4','CHRND','CIB1','CIB3','CIP2A','CKAP5','CLCA2','CLCA4','CLCN5','CLCN6','CLDN11','CLDN18','CLDN2','CLDN23','CLDN3','CLDN4','CLDN7','CLDN9','CLEC2A','CLEC4F','CLEC5A','CLECL1','CLIC2','CLIC5','CLTB','CLUL1','CMTM4','CMTM7','CMTR2','CMYA5','CNGA1','CNKSR2','CNNM1','CNNM4','CNP','CNR1','CNTD1','COBL','COBLL1','COG3','COL16A1','COL17A1','COL4A5','COL6A2','COLEC12','COLQ','COMMD4','COMTD1','COPRS','COQ7','CORIN','CORO1B','CORO2A','CORO2B','COX6B2','CPEB3','CPPED1','CPZ','CRB3','CREB5','CRIP1','CRIP2','CRK','CRLF3','CROCC','CRTC1','CRYAB','CRYBG3','CRYL1','CSDE1','CSF1','CSF1R','CSPG4','CST6','CTNNA1','CTNNB1','CTNNBIP1','CTNND1','CTNND2','CTSF','CTSV','CTTN','CTXN1','CUEDC1','CUTA','CX3CL1','CX3CR1','CXADR','CXCR2','CYB5D2','CYBRD1','CYFIP2','CYP20A1','CYP24A1','CYTH2','CYTH4','DAAM1','DAB2','DAG1','DAGLB','DAW1','DBN1','DCAF13','DCAF16','DCBLD2','DCHS1','DCHS2','DCP2','DCT','DCTN6','DCUN1D2','DDAH1','DDB2','DDIAS','DDIT4L','DDR1','DDR2','DDX58','DDX60L','DENND6B','DFFA','DGKG','DGKH','DHDDS','DHPS','DHRS7B','DIAPH1','DIAPH3','DIO2','DIS3L','DLG1','DLG2','DLG5','DLGAP3','DLL3','DMWD','DNAAF1','DNAAF4','DNAH17','DNAJA4','DNAJB13','DNAJB4','DNAJC18','DNAJC5','DNAJC6','DNAJC9','DOC2A','DOCK4','DOK3','DONSON','DPEP1','DPH1','DPPA5','DPYSL2','DQX1','DRD1','DRP2','DSC3','DSEL','DSG2','DSP','DTNA','DUS1L','DUSP15','DXO','DYSF','ECSCR','EDA2R','EDNRA','EDNRB','EEF1AKMT1','EEF2','EFCAB1','EFCAB11','EFCAB2','EFCAB7','EFHC2','EFNB1','EFR3A','EFR3B','EGFR','EHBP1','EHD1','EHD2','EHD3','EHD4','EIF2B2','EIF5','EIF5B','ELFN1','ELMO3','ELN','ELOC','EMP3','ENAH','ENC1','ENG','ENKD1','ENO1','ENO2','ENO3','ENOX1','ENOX2','EPB41','EPB41L1','EPB41L2','EPB41L3','EPB41L4A','EPB41L4B','EPB41L5','EPCAM','EPHA2','EPHA3','EPHB1','EPHB2','EPN1','EPOR','EPS8L2','ERAP1','ERBB2','ERBB3','ERBIN','ERC1','ERCC4','ERMN','ESCO2','ETNK1','ETS2','EVA1A','EVI2B','EXOC1','EXOC3L1','EXOC7','EZR','F11R','F2RL1','F2RL3','FAAP20','FABP2','FABP3','FABP5','FAIM','FAM126A','FAM126B','FAM129A','FAM129B','FAM13A','FAM163B','FAM171A1','FAM193A','FAM205A','FAM210B','FAM222A','FAM45A','FAM47E','FAM47E-STBD1','FAM72A','FAM72B','FAM72C','FAM72D','FANCG','FAS','FASN','FBLIM1','FBLN2','FBLN5','FBLN7','FBP2','FBRSL1','FBXL2','FBXO16','FBXO38','FBXO41','FBXW11','FCAR','FCGBP','FCGR1A','FCGR1B','FCGR2A','FCHSD2','FFAR4','FGD1','FGD5','FGFR2','FGR','FHIT','FHL1','FIBCD1','FIGN','FILIP1','FILIP1L','FKBP2','FLAD1','FLCN','FLNA','FLNB','FLNC','FLOT1','FLOT2','FLRT3','FLT1','FLT4','FLVCR1','FMN2','FMNL2','FMNL3','FNBP1','FOXA2','FOXN3','FP565260.3','FRMD3','FRMD7','FRMD8','FRS2','FSCN1','FTCD','FUT2','FXYD3','FYB1','FYN','FZD1','FZD2','FZD4','FZD6','GAB1','GAB2','GABRA1','GABRA2','GABRA3','GABRA5','GABRG1','GAD1','GADL1','GAGE1','GAGE10','GAGE12C','GAGE12D','GAGE12E','GAGE12F','GAGE12G','GAGE12H','GAGE12J','GAGE13','GAGE2A','GAGE2E','GALNT10','GALR1','GAP43','GAPDH','GAPT','GAPVD1','GAS2','GAS2L1','GAS7','GAS8','GBP4','GCA','GCC1','GCLM','GCSAML-AS1','GFI1B','GFOD1','GFRA3','GHR','GID8','GINM1','GIPC1','GJA1','GJB3','GJB4','GJB6','GJB7','GJC1','GJC2','GLO1','GLP2R','GLRX','GLUL','GMIP','GNAI2','GNAQ','GNAS','GNB1','GNB2','GNB3','GNB4','GNG2','GP9','GPAT3','GPATCH1','GPBP1','GPBP1L1','GPC1','GPC3','GPC4','GPI','GPR1','GPR142','GPR156','GPR157','GPR160','GPR173','GPR20','GPR27','GPR50','GPR63','GPRC5A','GPRIN1','GPRIN2','GPX8','GRAMD1A','GRAMD1B','GRAMD1C','GRAP2','GRB7','GRID2IP','GRIK5','GRK3','GRK5','GRPR','GSDMA','GTF2F1','GTSE1','GUCA1A','GYPA','GYPC','HAPLN3','HARBI1','HAS1','HAUS4','HAUS7','HCAR2','HCAR3','HCFC2','HCK','HCLS1','HCST','HDAC8','HEG1','HELZ','HENMT1','HERC2','HERPUD1','HFE','HIF1A','HIF3A','HINT1','HIP1R','HLA-A','HLA-DPB1','HMGCS1','HMOX1','HOMER3','HOXC5','HOXD4','HPS3','HRASLS5','HRH1','HS6ST3','HSH2D','HSPB1','HSPG2','HTR1B','HTR7','HTRA1','HUNK','HYDIN','HYI','HYLS1','ICAM1','ICAM2','ICOS','ICOSLG','IFFO2','IFIT5','IFNGR1','IFRD1','IFT122','IGFL2','IGSF11','IGSF23','IKZF5','IL10RA','IL12RB1','IL12RB2','IL13','IL13RA2','IL16','IL17RB','IL1R1','IL1RL1','IL21R','IL4R','IL6ST','IL7R','ILDR2','ILK','INPP5A','INSR','INTS12','IP6K2','IPCEF1','IQGAP1','IQGAP2','IREB2','IRF2','IRS1','ISLR','ITGA2B','ITGA3','ITGA4','ITGA7','ITGA8','ITGA9','ITGB1','ITGB2','ITGB3','ITGB4','ITGB5','ITGB6','ITGB7','ITGB8','ITK','ITM2A','ITPKA','ITSN1','ITSN2','IYD','JAG1','JAK2','JAK3','JAM2','JAML','JCAD','JMJD4','JPT2','JUP','KANK1','KANK3','KANSL2','KARS','KATNB1','KCNAB1','KCNB2','KCNH8','KCNJ15','KCNK13','KCNK17','KCNK6','KCNK7','KCNK9','KCNN2','KCNS3','KCNU1','KDF1','KDR','KEL','KIAA0556','KIAA0895','KIAA1324','KIAA1522','KIAA1958','KIF17','KIF19','KIF21A','KIF21B','KIF26A','KIF26B','KISS1R','KIT','KLC1','KLC2','KLF9','KLHDC7B','KLHL41','KLHL6','KLHL7','KLK7','KLRC1','KLRC2','KLRF1','KLRK1','KMT2D','KNDC1','KNSTRN','KNTC1','KPTN','KRT75','KRT77','KRTAP25-1','L1CAM','L3MBTL1','LAIR1','LAMA4','LAMC1','LAMP2','LAMTOR1','LAPTM4A','LASP1','LAT','LAT2','LAX1','LCP1','LCTL','LDB2','LDLRAD3','LENG1','LGALS3','LGALS4','LGALS8','LGR6','LHFPL2','LHX6','LIMA1','LINGO1','LIPJ','LMCD1','LMOD2','LMTK2','LMTK3','LNX1','LNX2','LONRF1','LONRF3','LOXHD1','LPAR1','LPAR3','LPAR4','LPIN1','LPIN2','LRCH2','LRP11','LRP12','LRRC15','LRRC26','LRRC34','LRRC45','LRRC69','LRRC7','LRRC9','LRRFIP2','LRRIQ3','LSMEM2','LSR','LTB4R2','LUZP2','LY6K','LYN','LYPD1','LYST','LZTS1','LZTS2','MACF1','MADD','MAGI1','MAGI3','MAMDC2','MAP1S','MAP2','MAP2K1','MAP2K4','MAP2K7','MAP3K12','MAP3K21','MAP3K6','MAP3K7','MAP4','MAP4K1','MAP4K5','MAP6','MAPK15','MAPK8IP1','MAPT','MARCH10','MARCH7','MARCKS','MARK2','MARVELD2','MAS1L','MAT2B','MBP','MC5R','MCAM','MCEMP1','MCF2L2','MCMBP','MCTS1','MDGA1','ME1','MEF2B','MEIOC','MEPCE','MERTK','MET','METAP1','METAP2','METTL15','METTL25','METTL9','MFAP3L','MFSD1','MFSD2A','MGAT4C','MIB1','MICAL2','MICAL3','MICALL1','MICB','MIEN1','MIGA2','MISP','MKLN1','MKRN3','MLF2','MLKL','MLST8','MMACHC','MMP10','MMP15','MMP24','MMP25','MOAP1','MPHOSPH8','MPHOSPH9','MPI','MPIG6B','MPL','MPP1','MPP5','MPP7','MPZL1','MPZL2','MR1','MRGPRF','MRNIP','MRPL42','MRPL44','MRPL46','MRPS18B','MS4A1','MS4A12','MS4A14','MS4A15','MSN','MSRA','MTCP1','MTFR1L','MTHFR','MTM1','MTPN','MTSS1','MUC1','MUC20','MVD','MYH11','MYH9','MYL4','MYLK','MYO10','MYO1B','MYO1C','MYO1E','MYO1G','MYO3A','MYO5B','MYO9A','MYOF','N4BP2','NAA35','NAB1','NAIF1','NAMPT','NAP1L2','NAPG','NARFL','NCAM1','NCAM2','NCK1','NCKAP1L','NCKAP5','NCKIPSD','NCOA1','NCS1','NDC1','NECTIN2','NECTIN3','NECTIN4','NEDD1','NEDD9','NEO1','NEU1','NEXN','NF2','NFE2L2','NFKBID','NGEF']
MembraneProteins2=['NGFR','NHS','NHSL2','NID2','NINJ2','NKTR','NKX2-8','NLGN1','NLGN3','NLK','NLRX1','NMB','NMBR','NME1','NME2','NMT1','NMT2','NMUR2','NOS1','NOTCH2','NPFFR2','NPL','NPNT','NPR1','NPTN','NPTXR','NR2C1','NRAS','NRCAM','NRP2','NRSN1','NRXN1','NSD1','NSFL1C','NT5E','NTNG1','NTNG2','NTSR1','NUDCD3','NUDT16L1','NUDT9','NUMB','NUP35','NUS1','OAS3','OBSCN','OCA2','OCLN','OCM','OCM2','ODC1','ODR4','OGT','OLFM4','OLIG2','OLR1','OPHN1','OPRK1','OR10P1','OR1N2','OR2C3','OR6B2','OR6B3','OR6C6','OR8S1','OSBPL10','OSBPL6','OSBPL7','OSCAR','OSCP1','OSGEP','OSR2','OTOP2','OTULIN','OXGR1','P3H3','PACSIN2','PACSIN3','PAFAH1B2','PAG1','PAIP1','PAK1','PAK4','PAK6','PALLD','PALM','PALM2','PALM2-AKAP2','PALM3','PALMD','PANX1','PANX2','PAQR8','PARD3','PARD3B','PARD6A','PARD6G','PARP6','PATJ','PAWR','PCBD2','PCDH1','PCDH17','PCDH7','PCDHA1','PCDHA11','PCDHA12','PCDHA4','PCDHA5','PCDHB1','PCDHB11','PCDHB12','PCDHB16','PCDHB4','PCDHB5','PCDHGA1','PCDHGA10','PCDHGA11','PCDHGA12','PCDHGA2','PCDHGA3','PCDHGA4','PCDHGA5','PCDHGA6','PCDHGA7','PCDHGA8','PCDHGA9','PCDHGB1','PCDHGB2','PCDHGB3','PCDHGB4','PCDHGB5','PCDHGB6','PCDHGB7','PCDHGC3','PCDHGC4','PCDHGC5','PCLO','PCMTD1','PDAP1','PDCD7','PDE3A','PDE4A','PDE4D','PDE9A','PDGFC','PDGFRA','PDHX','PDLIM1','PDLIM5','PDPN','PDZD2','PDZD8','PEAR1','PEBP1','PFDN1','PGAP3','PGLYRP2','PGLYRP4','PGRMC2','PHACTR1','PHACTR4','PHF20L1','PHGDH','PHLDB2','PHLDB3','PHOSPHO1','PHPT1','PI3','PI4K2A','PI4KA','PIEZO2','PIGB','PIGN','PIGW','PIGX','PIK3AP1','PIK3CG','PIK3R6','PIP4K2A','PIP4K2C','PIP5K1A','PIRT','PITPNM3','PKD2L2','PKN1','PKN2','PKP1','PKP2','PKP3','PKP4','PLAA','PLAUR','PLCB1','PLCB2','PLCD3','PLCD4','PLEKHA4','PLEKHA6','PLEKHA7','PLEKHG3','PLEKHG4','PLEKHG6','PLEKHH2','PLEKHO2','PLIN4','PLPP1','PLPPR5','PLS3','PLSCR1','PLVAP','PLXNA3','PLXNA4','PNCK','PNISR','PODXL','POLE','POU2F3','PPEF1','PPFIBP1','PPIP5K1','PPL','PPM1A','PPM1F','PPP1CA','PPP1CB','PPP1R12B','PPP1R16A','PPP1R16B','PPP1R18','PPP1R27','PPP1R9A','PPP2R1B','PPP3R1','PPP3R2','PPP4C','PPP6R3','PPTC7','PRAME','PRC1','PRDX6','PRIMA1','PRKCA','PRKCE','PRKCH','PRKCZ','PRKD1','PROM2','PROSER2','PRR15','PRR32','PRR7','PRRC2A','PRRG1','PRRT2','PRRT3','PRRT4','PRTG','PRX','PSCA','PSD','PSD2','PSEN1','PSMC5','PSMC6','PSMD9','PSTPIP2','PTAFR','PTCHD4','PTK2B','PTK6','PTPN12','PTPN21','PTPN22','PTPN4','PTPRB','PTPRG','PTPRK','PTPRM','PTPRR','PTPRS','PTX3','PURG','PVALB','PVR','PXDC1','PXK','PYGL','PYGO2','PYM1','RAB13','RAB23','RAB25','RAB26','RAB27B','RAB3B','RAB3GAP2','RAB43','RAB8A','RABEPK','RAD51C','RAD52','RAI2','RALA','RALGAPA2','RAPGEF2','RAPH1','RAPSN','RASA4','RASA4B','RASAL1','RASAL2','RASAL3','RASD1','RASD2','RASGRP1','RASGRP2','RASGRP4','RASSF2','RASSF3','RASSF4','RBM46','RC3H1','RCCD1','RCSD1','RD3L','RDX','RECK','RELL1','REPS1','RET','RFLNB','RFPL1','RFPL2','RFPL3','RGP1','RGS10','RGS14','RGS19','RGS3','RGS7BP','RGS9BP','RHCE','RHD','RHEX','RHOA','RHOBTB2','RHOJ','RHOXF2','RHOXF2B','RIC8A','RIC8B','RIF1','RILPL1','RIMBP3','RIMBP3B','RIMBP3C','RIMS2','RIOK2','RIOK3','RIPK1','RND1','RNF114','RNF145','RNF146','RNF149','RNF165','RNF166','RNF39','ROBO1','ROBO3','ROBO4','ROM1','RP2','RPE','RPEL1','RPGRIP1L','RPIA','RPL36A','RPL36A-HNRNPH2','RPL36AL','RPSA','RPUSD1','RRAD','RRP12','RRP7A','RSBN1L','RSC1A1','RSPH3','RTL3','RTL4','RTN4R','RTN4RL1','RYR2','S100A13','S100A14','S100A16','S100A2','S100A3','S100A4','S100A6','S100A9','SAMD4A','SAMHD1','SAMSN1','SAPCD1','SASH3','SC5D','SCAMP1','SCAMP4','SCEL','SCHIP1','SCIN','SCN1A','SCN5A','SCN8A','SCN9A','SCNN1D','SCNN1G','SCRIB','SCUBE3','SDC2','SDC4','SDE2','SDHB','SEC16B','SEMA3C','SEMA3D','SEMA4B','SEMA4F','SEMA6C','SEMA6D','SEPT2','SEPT3','SERINC2','SERINC5','SERPINB3','SFSWAP','SGCE','SGK494','SH3BGR','SH3D21','SHANK1','SHANK3','SHARPIN','SHISA6','SHKBP1','SHROOM2','SHTN1','SIGLEC10','SIGLEC14','SIGLEC6','SIGLEC7','SIN3B','SIPA1','SIPA1L1','SIPA1L3','SKAP1','SLAMF6','SLC10A5','SLC12A2','SLC13A3','SLC13A5','SLC14A1','SLC16A1','SLC16A10','SLC16A2','SLC16A3','SLC16A7','SLC16A9','SLC17A5','SLC19A1','SLC19A3','SLC1A5','SLC1A6','SLC22A16','SLC23A3','SLC24A4','SLC26A9','SLC27A1','SLC28A1','SLC2A1','SLC2A11','SLC2A12','SLC2A14','SLC2A2','SLC2A3','SLC30A1','SLC31A2','SLC34A1','SLC34A2','SLC35F5','SLC35G2','SLC36A4','SLC38A5','SLC38A6','SLC38A7','SLC39A10','SLC39A14','SLC39A2','SLC39A6','SLC3A2','SLC40A1','SLC41A3','SLC43A1','SLC43A3','SLC44A2','SLC45A4','SLC46A1','SLC46A3','SLC4A1AP','SLC4A2','SLC4A7','SLC4A8','SLC5A12','SLC5A3','SLC5A6','SLC5A7','SLC6A6','SLC7A14','SLC7A2','SLC7A5','SLC7A6','SLC8A1','SLC9A1','SLC9A2','SLC9A3','SLC9A3R1','SLC9A3R2','SLC9A4','SLC9A5','SLC9B2','SLC9C2','SLCO1A2','SLCO1B3','SLCO4A1','SLCO4C1','SLCO5A1','SLIT1','SLIT3','SLITRK6','SLK','SLX4','SMAGP','SMAP1','SMOC1','SMPD2','SMPX','SNAP23','SNAP25','SNRK','SNTB2','SNTG2','SNX15','SNX32','SNX9','SOCS3','SOCS5','SOD1','SORBS1','SORCS1','SORT1','SOX11','SPAG16','SPANXN4','SPART','SPATA12','SPATC1L','SPERT','SPG11','SPHK1','SPINK13','SPINT1','SPINT2','SPN','SPPL2B','SPPL3','SPRED3','SPRYD3','SPTBN2','SRC','SRCIN1','SRD5A3','SRPK1','SRPRB','SRRM3','SSFA2','SSH1','SSH3','SSTR1','SSTR5','SSX2IP','ST8SIA5','STAB1','STAC','STAC2','STAMBP','STAMBPL1','STARD4','STAT2','STEAP4','STIL','STIP1','STK10','STK16','STK17A','STK17B','STOM','STOML2','STOML3','STX18','STX4','STXBP4','STYK1','SUCLG1','SUCLG2','SUGT1','SUMF2','SUPT6H','SUSD2','SVIL','SWAP70','SYCP2','SYK','SYMPK','SYNC','SYNGR3','SYNPO2L','SYPL2','SYT4','SYTL2','SYVN1','TACC2','TACR2','TACSTD2','TASP1','TAZ','TBC1D10A','TBC1D2','TBC1D24','TBC1D30','TBC1D9B','TBXA2R','TCAF2','TCAF2C','TCHP','TCIM','TDG','TDO2','TDP1','TEC','TEDC2','TEK','TES','TEX14','TEX9','TGFBR1','TGFBR2','TGM1','TGM2','THBS1','THBS2','THEGL','THY1','TIAM1','TICAM1','TIGD7','TIMD4','TJP1','TJP2','TJP3','TLN1','TLR4','TM2D2','TM4SF20','TM4SF4','TM4SF5','TMC5','TMED8','TMEM101','TMEM107','TMEM108','TMEM121B','TMEM139','TMEM141','TMEM164','TMEM176B','TMEM183A','TMEM186','TMEM2','TMEM218','TMEM222','TMEM229A','TMEM245','TMEM251','TMEM254','TMEM258','TMEM266','TMEM268','TMEM50A','TMEM63B','TMPRSS15','TMPRSS2','TMTC2','TNFRSF10D','TNFRSF11A','TNFRSF12A','TNFRSF21','TNFRSF25','TNFRSF8','TNFRSF9','TNFSF13B','TNK1','TNKS1BP1','TNNI2','TOGARAM2','TOLLIP','TOM1','TP53BP2','TPD52L1','TPRN','TRAF7','TRAIP','TRAPPC13','TRAT1','TRDN','TREML1','TRIB1','TRIM16','TRIM16L','TRIM38','TRIM4','TRIM44','TRIM54','TRIM71','TRIML2','TRIP6','TRMT1','TRMT10B','TRMT13','TRMT6','TRO','TRPM1','TRPM4','TRPM7','TRPV2','TRPV3','TSG101','TSGA13','TSHZ3','TSPAN15','TSPAN6','TSPO2','TSR2','TSSK4','TTC17','TTC23','TTC7B','TTLL12','TTLL5','TULP3','TWF2','TXNDC11','TXNIP','TYROBP','TYW3','UAP1','UBA3','UBA52','UBAC1','UBALD1','UBAP1','UBASH3B','UBE2A','UBE2B','UBE2C','UBE2D1','UBE2D2','UBE2D3','UBE2D4','UBE2K','UBE2QL1','UBE2S','UBE3C','UBQLN2','UBR2','UBTD1','UCP3','ULBP1','ULBP2','UNC5A','UNC79','UQCC1','USP14','USP15','USP2','USP20','USP21','USP4','USP53','UTP20','UTRN','VAMP5','VAMP8','VANGL1','VASP','VCAM1','VCPIP1','VIL1','VPS13B','VPS36','VPS50','VPS8','VSIG4','VTCN1','VWA5B1','VWA7','WAS','WASF2','WBP4','WDFY3','WDPCP','WDR1','WDR12','WDR13','WDR53','WDR6','WDR66','WIPF1','WWP1','XIAP','XIRP1','XIRP2','XPC','XPNPEP2','XPO6','XRN1','XYLT1','YES1','YIPF1','YIPF4','YOD1','YTHDC1','ZBED3','ZBTB33','ZBTB8OS','ZDHHC2','ZDHHC20','ZDHHC22','ZDHHC5','ZFP41','ZFP90','ZFYVE28','ZGPAT','ZMYND10','ZNF10','ZNF18','ZNF185','ZNF2','ZNF20','ZNF227','ZNF229','ZNF236','ZNF276','ZNF277','ZNF337','ZNF397','ZNF407','ZNF497','ZNF503','ZNF510','ZNF536','ZNF571','ZNF598','ZNF599','ZNF646','ZNF655','ZNF688','ZNF706','ZNF717','ZNF783','ZNF79','ZNF839','ZSCAN22','ZSCAN29','ZSWIM1','ZSWIM8','ZYX']
MembraneProteins=MembraneProteins1+MembraneProteins2

TFs1=['AC008770.3','AC023509.3','AC092835.1','AC138696.1','ADNP','ADNP2','AEBP1','AEBP2','AHCTF1','AHDC1','AHR','AHRR','AIRE','AKAP8','AKAP8L','AKNA','ALX1','ALX3','ALX4','ANHX','ANKZF1','AR','ARGFX','ARHGAP35','ARID2','ARID3A','ARID3B','ARID3C','ARID5A','ARID5B','ARNT','ARNT2','ARNTL','ARNTL2','ARX','ASCL1','ASCL2','ASCL3','ASCL4','ASCL5','ASH1L','ATF1','ATF2','ATF3','ATF4','ATF5','ATF6','ATF6B','ATF7','ATMIN','ATOH1','ATOH7','ATOH8','BACH1','BACH2','BARHL1','BARHL2','BARX1','BARX2','BATF','BATF2','BATF3','BAZ2A','BAZ2B','BBX','BCL11A','BCL11B','BCL6','BCL6B','BHLHA15','BHLHA9','BHLHE22','BHLHE23','BHLHE40','BHLHE41','BNC1','BNC2','BORCS8-MEF2B','BPTF','BRF2','BSX','C11orf95','CAMTA1','CAMTA2','CARF','CASZ1','CBX2','CC2D1A','CCDC169-SOHLH2','CCDC17','CDC5L','CDX1','CDX2','CDX4','CEBPA','CEBPB','CEBPD','CEBPE','CEBPG','CEBPZ','CENPA','CENPB','CENPBD1','CENPS','CENPT','CENPX','CGGBP1','CHAMP1','CHCHD3','CIC','CLOCK','CPEB1','CPXCR1','CREB1','CREB3','CREB3L1','CREB3L2','CREB3L3','CREB3L4','CREB5','CREBL2','CREBZF','CREM','CRX','CSRNP1','CSRNP2','CSRNP3','CTCF','CTCFL','CUX1','CUX2','CXXC1','CXXC4','CXXC5','DACH1','DACH2','DBP','DBX1','DBX2','DDIT3','DEAF1','DLX1','DLX2','DLX3','DLX4','DLX5','DLX6','DMBX1','DMRT1','DMRT2','DMRT3','DMRTA1','DMRTA2','DMRTB1','DMRTC2','DMTF1','DNMT1','DNTTIP1','DOT1L','DPF1','DPF3','DPRX','DR1','DRAP1','DRGX','DUX1','DUX3','DUX4','DUXA','DZIP1','E2F1','E2F2','E2F3','E2F4','E2F5','E2F6','E2F7','E2F8','E4F1','EBF1','EBF2','EBF3','EBF4','EEA1','EGR1','EGR2','EGR3','EGR4','EHF','ELF1','ELF2','ELF3','ELF4','ELF5','ELK1','ELK3','ELK4','EMX1','EMX2','EN1','EN2','EOMES','EPAS1','ERF','ERG','ESR1','ESR2','ESRRA','ESRRB','ESRRG','ESX1','ETS1','ETS2','ETV1','ETV2','ETV3','ETV3L','ETV4','ETV5','ETV6','ETV7','EVX1','EVX2','FAM170A','FAM200B','FBXL19','FERD3L','FEV','FEZF1','FEZF2','FIGLA','FIZ1','FLI1','FLYWCH1','FOS','FOSB','FOSL1','FOSL2','FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXN4','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1','GABPA','GATA1','GATA2','GATA3','GATA4','GATA5','GATA6','GATAD2A','GATAD2B','GBX1','GBX2','GCM1','GCM2','GFI1','GFI1B','GLI1','GLI2','GLI3','GLI4','GLIS1','GLIS2','GLIS3','GLMP','GLYR1','GMEB1','GMEB2','GPBP1','GPBP1L1','GRHL1','GRHL2','GRHL3','GSC','GSC2','GSX1','GSX2','GTF2B','GTF2I','GTF2IRD1','GTF2IRD2','GTF2IRD2B','GTF3A','GZF1','HAND1','HAND2','HBP1','HDX','HELT','HES1','HES2','HES3','HES4','HES5','HES6','HES7','HESX1','HEY1','HEY2','HEYL','HHEX','HIC1','HIC2','HIF1A','HIF3A','HINFP','HIVEP1','HIVEP2','HIVEP3','HKR1','HLF','HLX','HMBOX1','HMG20A','HMG20B','HMGA1','HMGA2','HMGN3','HMX1','HMX2','HMX3','HNF1A','HNF1B','HNF4A','HNF4G','HOMEZ','HOXA1','HOXA10','HOXA11','HOXA13','HOXA2','HOXA3','HOXA4','HOXA5','HOXA6','HOXA7','HOXA9','HOXB1','HOXB13','HOXB2','HOXB3','HOXB4','HOXB5','HOXB6','HOXB7','HOXB8','HOXB9','HOXC10','HOXC11','HOXC12','HOXC13','HOXC4','HOXC5','HOXC6','HOXC8','HOXC9','HOXD1','HOXD10','HOXD11','HOXD12','HOXD13','HOXD3','HOXD4','HOXD8','HOXD9','HSF1','HSF2','HSF4','HSF5','HSFX1','HSFX2','HSFY1','HSFY2','IKZF1','IKZF2','IKZF3','IKZF4','IKZF5','INSM1','INSM2','IRF1','IRF2','IRF3','IRF4','IRF5','IRF6','IRF7','IRF8','IRF9','IRX1','IRX2','IRX3','IRX4','IRX5','IRX6','ISL1','ISL2','ISX','JAZF1','JDP2','JRK','JRKL','JUN','JUNB','JUND','KAT7','KCMF1','KCNIP3','KDM2A','KDM2B','KDM5B','KIN','KLF1','KLF10','KLF11','KLF12','KLF13','KLF14','KLF15','KLF16','KLF17','KLF2','KLF3','KLF4','KLF5','KLF6','KLF7','KLF8','KLF9','KMT2A','KMT2B','L3MBTL1','L3MBTL3','L3MBTL4','LBX1','LBX2','LCOR','LCORL','LEF1','LEUTX','LHX1','LHX2','LHX3','LHX4','LHX5','LHX6','LHX8','LHX9','LIN28A','LIN28B','LIN54','LMX1A','LMX1B','LTF','LYL1','MAF','MAFA','MAFB','MAFF','MAFG','MAFK','MAX','MAZ','MBD1','MBD2','MBD3','MBD4','MBD6','MBNL2','MECOM','MECP2','MEF2A','MEF2B','MEF2C','MEF2D','MEIS1','MEIS2','MEIS3','MEOX1','MEOX2','MESP1','MESP2','MGA','MITF','MIXL1','MKX','MLX','MLXIP','MLXIPL','MNT','MNX1','MSANTD1','MSANTD3','MSANTD4','MSC','MSGN1','MSX1','MSX2','MTERF1','MTERF2','MTERF3','MTERF4','MTF1','MTF2','MXD1','MXD3','MXD4','MXI1','MYB','MYBL1','MYBL2','MYC','MYCL','MYCN','MYF5','MYF6','MYNN','MYOD1','MYOG','MYPOP','MYRF','MYRFL','MYSM1','MYT1','MYT1L','MZF1','NACC2','NAIF1','NANOG','NANOGNB','NANOGP8','NCOA1','NCOA2','NCOA3','NEUROD1','NEUROD2','NEUROD4','NEUROD6','NEUROG1','NEUROG2','NEUROG3','NFAT5','NFATC1','NFATC2','NFATC3','NFATC4','NFE2','NFE2L1','NFE2L2','NFE2L3','NFE4','NFIA','NFIB','NFIC','NFIL3','NFIX','NFKB1','NFKB2','NFX1','NFXL1','NFYA','NFYB','NFYC','NHLH1','NHLH2','NKRF','NKX1-1','NKX1-2','NKX2-1','NKX2-2','NKX2-3','NKX2-4','NKX2-5','NKX2-6','NKX2-8','NKX3-1','NKX3-2','NKX6-1','NKX6-2','NKX6-3','NME2','NOBOX','NOTO','NPAS1','NPAS2','NPAS3','NPAS4','NR0B1','NR1D1','NR1D2','NR1H2','NR1H3','NR1H4','NR1I2','NR1I3','NR2C1','NR2C2','NR2E1','NR2E3','NR2F1','NR2F2','NR2F6','NR3C1','NR3C2','NR4A1','NR4A2','NR4A3','NR5A1','NR5A2','NR6A1','NRF1','NRL','OLIG1','OLIG2','OLIG3','ONECUT1','ONECUT2','ONECUT3','OSR1','OSR2','OTP','OTX1','OTX2','OVOL1','OVOL2','OVOL3','PA2G4','PATZ1','PAX1','PAX2','PAX3','PAX4','PAX5','PAX6','PAX7','PAX8','PAX9','PBX1','PBX2','PBX3','PBX4','PCGF2','PCGF6','PDX1','PEG3','PGR','PHF1','PHF19 ','PHF20','PHF21A','PHOX2A','PHOX2B','PIN1','PITX1','PITX2','PITX3','PKNOX1','PKNOX2','PLAG1','PLAGL1','PLAGL2','PLSCR1','POGK','POU1F1','POU2AF1','POU2F1','POU2F2','POU2F3','POU3F1','POU3F2','POU3F3','POU3F4','POU4F1','POU4F2','POU4F3','POU5F1','POU5F1B','POU5F2','POU6F1','POU6F2','PPARA','PPARD','PPARG','PRDM1','PRDM10','PRDM12','PRDM13','PRDM14','PRDM15','PRDM16','PRDM2','PRDM4','PRDM5','PRDM6','PRDM8','PRDM9','PREB','PRMT3','PROP1','PROX1','PROX2','PRR12','PRRX1','PRRX2','PTF1A','PURA','PURB','PURG','RAG1','RARA','RARB','RARG','RAX','RAX2','RBAK','RBCK1','RBPJ','RBPJL','RBSN','REL','RELA','RELB','REPIN1','REST','REXO4','RFX1','RFX2','RFX3','RFX4','RFX5','RFX6','RFX7','RFX8','RHOXF1','RHOXF2','RHOXF2B','RLF','RORA','RORB','RORC','RREB1','RUNX1','RUNX2','RUNX3','RXRA','RXRB','RXRG','SAFB','SAFB2','SALL1','SALL2','SALL3','SALL4','SATB1','SATB2','SCMH1','SCML4','SCRT1','SCRT2','SCX','SEBOX','SETBP1','SETDB1','SETDB2','SGSM2','SHOX','SHOX2','SIM1','SIM2','SIX1','SIX2','SIX3','SIX4','SIX5','SIX6','SKI','SKIL','SKOR1','SKOR2','SLC2A4RG','SMAD1','SMAD3','SMAD4','SMAD5','SMAD9','SMYD3','SNAI1','SNAI2','SNAI3','SNAPC2','SNAPC4','SNAPC5','SOHLH1','SOHLH2','SON','SOX1','SOX10','SOX11']
TFs2=['SOX12','SOX13','SOX14','SOX15','SOX17','SOX18','SOX2','SOX21','SOX3','SOX30','SOX4','SOX5','SOX6','SOX7','SOX8','SOX9','SP1','SP100','SP110','SP140','SP140L','SP2','SP3','SP4','SP5','SP6','SP7','SP8','SP9','SPDEF','SPEN','SPI1','SPIB','SPIC','SPZ1','SRCAP','SREBF1','SREBF2','SRF','SRY','ST18','STAT1','STAT2','STAT3','STAT4','STAT5A','STAT5B','STAT6','T','TAL1','TAL2','TBP','TBPL1','TBPL2','TBR1','TBX1','TBX10','TBX15','TBX18','TBX19','TBX2','TBX20','TBX21','TBX22','TBX3','TBX4','TBX5','TBX6','TCF12','TCF15','TCF20','TCF21','TCF23','TCF24','TCF3','TCF4','TCF7','TCF7L1','TCF7L2','TCFL5','TEAD1','TEAD2','TEAD3','TEAD4','TEF','TERB1','TERF1','TERF2','TET1','TET2','TET3','TFAP2A','TFAP2B','TFAP2C','TFAP2D','TFAP2E','TFAP4','TFCP2','TFCP2L1','TFDP1','TFDP2','TFDP3','TFE3','TFEB','TFEC','TGIF1','TGIF2','TGIF2LX','TGIF2LY','THAP1','THAP10','THAP11','THAP12','THAP2','THAP3','THAP4','THAP5','THAP6','THAP7','THAP8','THAP9','THRA','THRB','THYN1','TIGD1','TIGD2','TIGD3','TIGD4','TIGD5','TIGD6','TIGD7','TLX1','TLX2','TLX3','TMF1','TOPORS','TP53','TP63','TP73','TPRX1','TRAFD1','TRERF1','TRPS1','TSC22D1','TSHZ1','TSHZ2','TSHZ3','TTF1','TWIST1','TWIST2','UBP1','UNCX','USF1','USF2','USF3','VAX1','VAX2','VDR','VENTX','VEZF1','VSX1','VSX2','WIZ','WT1','XBP1','XPA','YBX1','YBX2','YBX3','YY1','YY2','ZBED1','ZBED2','ZBED3','ZBED4','ZBED5','ZBED6','ZBED9','ZBTB1','ZBTB10','ZBTB11','ZBTB12','ZBTB14','ZBTB16','ZBTB17','ZBTB18','ZBTB2','ZBTB20','ZBTB21','ZBTB22','ZBTB24','ZBTB25','ZBTB26','ZBTB3','ZBTB32','ZBTB33','ZBTB34','ZBTB37','ZBTB38','ZBTB39','ZBTB4','ZBTB40','ZBTB41','ZBTB42','ZBTB43','ZBTB44','ZBTB45','ZBTB46','ZBTB47','ZBTB48','ZBTB49','ZBTB5','ZBTB6','ZBTB7A','ZBTB7B','ZBTB7C','ZBTB8A','ZBTB8B','ZBTB9','ZC3H8','ZEB1','ZEB2','ZFAT','ZFHX2','ZFHX3','ZFHX4','ZFP1','ZFP14','ZFP2','ZFP28','ZFP3','ZFP30','ZFP37','ZFP41','ZFP42','ZFP57','ZFP62','ZFP64','ZFP69','ZFP69B','ZFP82','ZFP90','ZFP91','ZFP92','ZFPM1','ZFPM2','ZFX','ZFY','ZGLP1','ZGPAT','ZHX1','ZHX2','ZHX3','ZIC1','ZIC2','ZIC3','ZIC4','ZIC5','ZIK1','ZIM2','ZIM3','ZKSCAN1','ZKSCAN2','ZKSCAN3','ZKSCAN4','ZKSCAN5','ZKSCAN7','ZKSCAN8','ZMAT1','ZMAT4','ZNF10','ZNF100','ZNF101','ZNF107','ZNF112','ZNF114','ZNF117','ZNF12','ZNF121','ZNF124','ZNF131','ZNF132','ZNF133','ZNF134','ZNF135','ZNF136','ZNF138','ZNF14','ZNF140','ZNF141','ZNF142','ZNF143','ZNF146','ZNF148','ZNF154','ZNF155','ZNF157','ZNF16','ZNF160','ZNF165','ZNF169','ZNF17','ZNF174','ZNF175','ZNF177','ZNF18','ZNF180','ZNF181','ZNF182','ZNF184','ZNF189','ZNF19','ZNF195','ZNF197','ZNF2','ZNF20','ZNF200','ZNF202','ZNF205','ZNF207','ZNF208','ZNF211','ZNF212','ZNF213','ZNF214','ZNF215','ZNF217','ZNF219','ZNF22','ZNF221','ZNF222','ZNF223','ZNF224','ZNF225','ZNF226','ZNF227','ZNF229','ZNF23','ZNF230','ZNF232','ZNF233','ZNF234','ZNF235','ZNF236','ZNF239','ZNF24','ZNF248','ZNF25','ZNF250','ZNF251','ZNF253','ZNF254','ZNF256','ZNF257','ZNF26','ZNF260','ZNF263','ZNF264','ZNF266','ZNF267','ZNF268','ZNF273','ZNF274','ZNF275','ZNF276','ZNF277','ZNF28','ZNF280A','ZNF280B','ZNF280C','ZNF280D','ZNF281','ZNF282','ZNF283','ZNF284','ZNF285','ZNF286A','ZNF286B','ZNF287','ZNF292','ZNF296','ZNF3','ZNF30','ZNF300','ZNF302','ZNF304','ZNF311','ZNF316','ZNF317','ZNF318','ZNF319','ZNF32','ZNF320','ZNF322','ZNF324','ZNF324B','ZNF326','ZNF329','ZNF331','ZNF333','ZNF334','ZNF335','ZNF337','ZNF33A','ZNF33B','ZNF34','ZNF341','ZNF343','ZNF345','ZNF346','ZNF347','ZNF35','ZNF350','ZNF354A','ZNF354B','ZNF354C','ZNF358','ZNF362','ZNF365','ZNF366','ZNF367','ZNF37A','ZNF382','ZNF383','ZNF384','ZNF385A','ZNF385B','ZNF385C','ZNF385D','ZNF391','ZNF394','ZNF395','ZNF396','ZNF397','ZNF398','ZNF404','ZNF407','ZNF408','ZNF41','ZNF410','ZNF414','ZNF415','ZNF416','ZNF417','ZNF418','ZNF419','ZNF420','ZNF423','ZNF425','ZNF426','ZNF428','ZNF429','ZNF43','ZNF430','ZNF431','ZNF432','ZNF433','ZNF436','ZNF438','ZNF439','ZNF44','ZNF440','ZNF441','ZNF442','ZNF443','ZNF444','ZNF445','ZNF446','ZNF449','ZNF45','ZNF451','ZNF454','ZNF460','ZNF461','ZNF462','ZNF467','ZNF468','ZNF469','ZNF470','ZNF471','ZNF473','ZNF474','ZNF479','ZNF48','ZNF480','ZNF483','ZNF484','ZNF485','ZNF486','ZNF487','ZNF488','ZNF490','ZNF491','ZNF492','ZNF493','ZNF496','ZNF497','ZNF500','ZNF501','ZNF502','ZNF503','ZNF506','ZNF507','ZNF510','ZNF511','ZNF512','ZNF512B','ZNF513','ZNF514','ZNF516','ZNF517','ZNF518A','ZNF518B','ZNF519','ZNF521','ZNF524','ZNF525','ZNF526','ZNF527','ZNF528','ZNF529','ZNF530','ZNF532','ZNF534','ZNF536','ZNF540','ZNF541','ZNF543','ZNF544','ZNF546','ZNF547','ZNF548','ZNF549','ZNF550','ZNF551','ZNF552','ZNF554','ZNF555','ZNF556','ZNF557','ZNF558','ZNF559','ZNF560','ZNF561','ZNF562','ZNF563','ZNF564','ZNF565','ZNF566','ZNF567','ZNF568','ZNF569','ZNF57','ZNF570','ZNF571','ZNF572','ZNF573','ZNF574','ZNF575','ZNF576','ZNF577','ZNF578','ZNF579','ZNF580','ZNF581','ZNF582','ZNF583','ZNF584','ZNF585A','ZNF585B','ZNF586','ZNF587','ZNF587B','ZNF589','ZNF592','ZNF594','ZNF595','ZNF596','ZNF597','ZNF598','ZNF599','ZNF600','ZNF605','ZNF606','ZNF607','ZNF608','ZNF609','ZNF610','ZNF611','ZNF613','ZNF614','ZNF615','ZNF616','ZNF618','ZNF619','ZNF620','ZNF621','ZNF623','ZNF624','ZNF625','ZNF626','ZNF627','ZNF628','ZNF629','ZNF630','ZNF639','ZNF641','ZNF644','ZNF645','ZNF646','ZNF648','ZNF649','ZNF652','ZNF653','ZNF654','ZNF655','ZNF658','ZNF66','ZNF660','ZNF662','ZNF664','ZNF665','ZNF667','ZNF668','ZNF669','ZNF670','ZNF671','ZNF672','ZNF674','ZNF675','ZNF676','ZNF677','ZNF678','ZNF679','ZNF680','ZNF681','ZNF682','ZNF683','ZNF684','ZNF687','ZNF688','ZNF689','ZNF69','ZNF691','ZNF692','ZNF695','ZNF696','ZNF697','ZNF699','ZNF7','ZNF70','ZNF700','ZNF701','ZNF703','ZNF704','ZNF705A','ZNF705B','ZNF705D','ZNF705E','ZNF705G','ZNF706','ZNF707','ZNF708','ZNF709','ZNF71','ZNF710','ZNF711','ZNF713','ZNF714','ZNF716','ZNF717','ZNF718','ZNF721','ZNF724','ZNF726','ZNF727','ZNF728','ZNF729','ZNF730','ZNF732','ZNF735','ZNF736','ZNF737','ZNF74','ZNF740','ZNF746','ZNF747','ZNF749','ZNF750','ZNF75A','ZNF75D','ZNF76','ZNF761','ZNF763','ZNF764','ZNF765','ZNF766','ZNF768','ZNF77','ZNF770','ZNF771','ZNF772','ZNF773','ZNF774','ZNF775','ZNF776','ZNF777','ZNF778','ZNF780A','ZNF780B','ZNF781','ZNF782','ZNF783','ZNF784','ZNF785','ZNF786','ZNF787','ZNF788','ZNF789','ZNF79','ZNF790','ZNF791','ZNF792','ZNF793','ZNF799','ZNF8','ZNF80','ZNF800','ZNF804A','ZNF804B','ZNF805','ZNF808','ZNF81','ZNF813','ZNF814','ZNF816','ZNF821','ZNF823','ZNF827','ZNF829','ZNF83','ZNF830','ZNF831','ZNF835','ZNF836','ZNF837','ZNF84','ZNF841','ZNF843','ZNF844','ZNF845','ZNF846','ZNF85','ZNF850','ZNF852','ZNF853','ZNF860','ZNF865','ZNF878','ZNF879','ZNF880','ZNF883','ZNF888','ZNF891','ZNF90','ZNF91','ZNF92','ZNF93','ZNF98','ZNF99','ZSCAN1','ZSCAN10','ZSCAN12','ZSCAN16','ZSCAN18','ZSCAN2','ZSCAN20','ZSCAN21','ZSCAN22','ZSCAN23','ZSCAN25','ZSCAN26','ZSCAN29','ZSCAN30','ZSCAN31','ZSCAN32','ZSCAN4','ZSCAN5A','ZSCAN5B','ZSCAN5C','ZSCAN9','ZUFSP','ZXDA','ZXDB','ZXDC','ZZZ3']
TFs=TFs1+TFs2

#TFs other, inclusive of the above list, not necessarily DNA binding proteins but affect transcription
TFs1Other=['AATF','ABL1','ABRA','AC008770.3','AC020915.1','AC023509.3','AC092835.1','AC105001.2','AC138696.1','ACOD1','ACTR3B','ACTR3C','ADAM8','ADAMTS17','ADAMTS19','ADAR','ADGRA1','ADGRG3','ADNP','ADNP2','AEBP1','AEBP2','AES','AFAP1 ','AFF1','AFF3','AFF4','AGER','AGT','AHCTF1','AHDC1','AHR','AHRR','AIM2','AIRE','AKAP8','AKAP8L','AKNA','AKT1','ALK','ALKBH2','ALX1','ALX3','ALX4','AMH','ANAPC2','ANHX','ANKRD30A','ANKZF1','ANXA3','ANXA4','AP5Z1','APBB3','APC2','APEX1','APOA4','APOBEC3A','APOBEC3B','APOBEC3G','APTX','AR','ARGFX','ARHGAP35','ARHGEF2','ARHGEF5','ARID1A','ARID1B','ARID2','ARID3A','ARID3B','ARID3C','ARID4A','ARID4B','ARID5A','ARID5B','ARNT','ARNT2','ARNTL','ARNTL2','ARRB1','ARRB2','ARX','ASCC1','ASCL1','ASCL2','ASCL3','ASCL4','ASCL5','ASH1L','ATF1','ATF2','ATF3','ATF4','ATF5','ATF6','ATF6B','ATF7','ATF7IP','ATM','ATMIN','ATOH1','ATOH7','ATOH8','ATR','ATRX','ATXN7','BACH1','BACH2','BAG1','BANF1','BANP','BARHL1','BARHL2','BARX1','BARX2','BATF','BATF2','BATF3','BAZ2A','BAZ2B','BBX','BCL10','BCL11A','BCL11B','BCL3','BCL6','BCL6B','BCLAF1','BCOR','BDP1','BEND4','BEX1','BHLHA15','BHLHA9','BHLHE22','BHLHE23','BHLHE40','BHLHE41','BHMG1','BIN1','BLM','BLZF1','BMI1','BMP2','BMP4','BMP7','BMPR1A','BNC1','BNC2','BOD1L1','BOLA1','BOLA2','BOLA2B','BOLA3','BORCS8-MEF2B','BPNT1','BPTF','BRCA1','BRD4','BRD7','BRD8','BRD9','BRF1','BRF2','BRMS1','BRPF1','BSX','BTAF1','BTBD8','BTG2','BTK','BTRC','BUD31','C11orf95','C14orf39','C17orf49','C1D','C20orf194','C3orf33','CACTIN','CAMK1D','CAMK2A','CAMTA1','CAMTA2','CAPN15','CAPN3','CARD11','CARD14','CARF','CARHSP1','CASP8AP2','CASZ1','CAT','CBFA2T2','CBFA2T3','CBFB','CBL','CBLL1','CBX2','CC2D1A','CC2D1B','CCDC124','CCDC169-SOHLH2','CCDC17','CCDC3','CCDC71','CCDC83','CCNT1','CCNT2','CD36','CD40','CD40LG','CDC5L','CDC6','CDK5RAP3','CDK7','CDK8','CDK9','CDKN1A','CDKN2A','CDX1','CDX2','CDX4','CEBPA','CEBPB','CEBPD','CEBPE','CEBPG','CEBPZ','CELF3','CENPA','CENPB','CENPBD1','CENPS','CENPT','CENPX','CERS2','CERS3','CERS4','CERS5','CERS6','CFLAR','CGGBP1','CHAMP1','CHCHD3','CHD1','CHD2','CHD3','CHD4','CHD6','CHD7','CHD9','CHP1','CHRAC1','CHUK','CHURC1','CIB1','CIC','CIITA','CIR1','CITED1','CITED2','CITED4','CIZ1','CLOCK','CLU','CMKLR1','CNBP','CNOT3','CNOT4','CNOT7','CNOT8','COMMD1','COMMD6','COMMD7','COPS2','COPS3','COPS4','CPEB1','CPSF4','CPSF4L','CPXCR1','CRAMP1','CREB1','CREB3','CREB3L1','CREB3L2','CREB3L3','CREB3L4','CREB5','CREBBP','CREBL2','CREBRF','CREBZF','CREG1','CREM','CRTC1','CRTC2','CRTC3','CRX','CSDC2','CSDE1','CSRNP1','CSRNP2','CSRNP3','CTBP1','CTBP2','CTCF','CTCFL','CTH','CTNNB1','CTNNBIP1','CUL1','CUL2','CUL3','CUL4A','CUL4B','CUL5','CUX1','CUX2','CXXC1','CXXC4','CXXC5','CYLD','CYP1B1','CYTL1','DAB2IP','DACH1','DACH2','DAP','DAXX','DBP','DBX1','DBX2','DCP1A','DDB1','DDB2','DDIT3','DDN','DDR2','DDRGK1','DDX3X','DDX5','DDX58','DEAF1','DEK','DEPDC1','DEPDC1B','DEPDC4','DEPDC5','DEPDC7','DEPTOR','DHX34','DHX36','DHX57','DISP3','DLX1','DLX2','DLX3','DLX4','DLX5','DLX6','DMAP1','DMBX1','DMC1','DMRT1','DMRT2','DMRT3','DMRTA1','DMRTA2','DMRTB1','DMRTC1','DMRTC2','DMTF1','DNA2','DNAJA3','DNAJC1','DNAJC2','DNAJC21','DNMT1','DNMT3A','DNMT3B','DNMT3L','DNTT','DNTTIP1','DOT1L','DPF1','DPF2','DPF3','DPRX','DR1','DRAP1','DRGX','DSP','DTX1','DUS3L','DUSP12','DUX1','DUX3','DUX4','DUXA','DVL1','DVL2','DVL3','DZIP1','DZIP1L','E2F1','E2F2','E2F3','E2F4','E2F5','E2F6','E2F7','E2F8','E4F1','EAF2','EBF1','EBF2','EBF3','EBF4','EDA','EDA2R','EDF1','EDN1','EEA1','EED','EGLN1','EGR1','EGR2','EGR3','EGR4','EHF','EID1','EID2','EIF2AK2','EIF2AK4','EIF3C','EIF3CL','EIF3K','ELF1','ELF2','ELF3','ELF4','ELF5','ELK1','ELK3','ELK4','ELMSAN1','EME1','EMX1','EMX2','EN1','EN2','ENO1','EOMES','EP300','EP400','EPAS1','EPHA5','EPM2AIP1','ERBIN','ERC1','ERCC3','ERCC4','ERF','ERG','ESR1','ESR2','ESRRA','ESRRB','ESRRG','ESX1','ETS1','ETS2','ETV1','ETV2','ETV3','ETV3L','ETV4','ETV5','ETV6','ETV7','EVX1','EVX2','EWSR1','EXO1','EXOC2','EXOG','EZH1','EZH2','FAAP24','FAM153C','FAM170A','FAM171B','FAM200A','FAM200B','FAN1','FANCA','FANCD2','FANK1','FARSA','FARSB','FBN1','FBXL19','FBXO41','FEN1','FER','FERD3L','FEV','FEZF1','FEZF2','FGD1','FHL2','FIGLA','FIZ1','FLI1','FLNA','FLYWCH1','FMNL2','FOS','FOSB','FOSL1','FOSL2','FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L2','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXN4','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1','FUBP1','FUBP3','FUS','FZD1','FZD2','FZD4','FZD6','GABPA','GABPB1','GABPB2','GADD45A','GAS6','GAS7','GATA1','GATA2','GATA3','GATA4','GATA5','GATA6','GATAD1','GATAD2A','GATAD2B','GBX1','GBX2','GCFC2','GCM1','GCM2','GEN1','GFI1','GFI1B','GLA','GLI1','GLI2','GLI3','GLI4','GLIS1','GLIS2','GLIS3','GLMP','GLYR1','GMEB1','GMEB2','GOLGB1','GON4L','GPATCH8','GPBP1','GPBP1L1','GPER1','GPR155','GREM1','GRHL1','GRHL2','GRHL3','GRM6','GSC','GSC2','GSX1','GSX2','GTF2A1','GTF2A1L','GTF2A2','GTF2B','GTF2E2','GTF2F1','GTF2F2','GTF2H1','GTF2H2','GTF2H2C','GTF2H3','GTF2H4','GTF2I','GTF2IRD1','GTF2IRD2','GTF2IRD2B','GTF3A','GTF3C1','GTF3C2','GULP1','GZF1','H1F0','H1FOO','H1FX','HAND1','HAND2','HAVCR2','HBP1','HCFC1','HCK','HDAC1','HDAC10','HDAC2','HDAC3','HDAC4','HDAC5','HDAC6','HDAC7','HDAC8','HDAC9','HDX','HELT','HELZ','HELZ2','HES1','HES2','HES3','HES4','HES5','HES6','HES7','HESX1','HEY1','HEY2','HEYL','HHEX','HIC1','HIC2','HIF1A','HIF3A','HINFP','HIPK2','HIRA','HIST1H1A','HIST1H1B','HIST1H1C','HIST1H1D','HIST1H1E','HIST1H1T','HIVEP1','HIVEP2','HIVEP3','HKR1','HLA-DQB1','HLA-DQB2','HLA-DRB1','HLA-DRB3','HLA-DRB5','HLF','HLTF','HLX','HMBOX1','HMG20A','HMG20B','HMGA1','HMGA2','HMGB1','HMGB2','HMGB3','HMGB4','HMGN1','HMGN3','HMGXB3','HMGXB4','HMOX1','HMX1','HMX2','HMX3','HNF1A','HNF1B','HNF4A','HNF4G','HNRNPA1','HNRNPA3','HNRNPAB','HNRNPCL2','HNRNPD','HNRNPDL','HNRNPK','HNRNPL','HNRNPU','HOMEZ','HOOK2','HOPX','HOXA1','HOXA10','HOXA11','HOXA13','HOXA2','HOXA3','HOXA4','HOXA5','HOXA6','HOXA7','HOXA9','HOXB1','HOXB13','HOXB2','HOXB3','HOXB4','HOXB5','HOXB6','HOXB7','HOXB8','HOXB9','HOXC10','HOXC11','HOXC12','HOXC13','HOXC4','HOXC5','HOXC6','HOXC8','HOXC9','HOXD1','HOXD10','HOXD11','HOXD12','HOXD13','HOXD3','HOXD4','HOXD8','HOXD9','HP1BP3','HR','HSBP1','HSF1','HSF2','HSF4','HSF5','HSFX1','HSFX2','HSFY1','HSFY2','HSPA1A','HSPA1B','ICAM1','ID1','ID2','ID3','ID4','IFI16','IFNB1','IGHM','IKBKB','IKBKG','IKZF1','IKZF2','IKZF3','IKZF4','IKZF5','IL10','IL1B','IL1RAP','IL4','IL5','IL6','INF2','ING4','INS','INSM1','INSM2','IQGAP1','IRAK1','IRAK2','IRAK3','IRF1','IRF2','IRF2BP1','IRF3','IRF4','IRF5','IRF6','IRF7','IRF8','IRF9','IRX1','IRX2','IRX3','IRX4','IRX5','IRX6','ISL1','ISL2','ISX','ITCH','ITGB2','ITGB3BP','IZUMO2','JADE1','JAK2','JARID2','JAZF1','JDP2','JMY','JRK','JRKL','JUN','JUNB','JUND','JUP','KANSL2','KAT2A','KAT2B','KAT5','KAT6A','KAT6B','KAT7','KAT8','KCMF1','KCNIP3','KDM1A','KDM2A','KDM2B','KDM3A','KDM5A','KDM5B','KDM5C','KDM5D','KEAP1','KIAA1549','KIAA1683','KIAA1841','KIAA1958','KIF15','KIN','KIT','KLF1','KLF10','KLF11','KLF12','KLF13','KLF14','KLF15','KLF16','KLF17','KLF2','KLF3','KLF4','KLF5','KLF6','KLF7','KLF8','KLF9','KMT2A','KMT2B','KMT2C','KMT2D','KNTC1','KRAS','KRBOX1','KRT13','KRTAP5-1','KRTAP5-9','KRTAP9-9','L3MBTL1','L3MBTL3','L3MBTL4','LARP1','LARP1B','LARP4','LARP4B','LARP6','LARP7','LBX1','LBX2','LCOR','LCORL','LEF1','LEUTX','LGALS9','LGR4','LHX1','LHX2','LHX3','LHX4','LHX5','LHX6','LHX8','LHX9','LIG1','LIG3','LIN28A','LIN28B','LIN54','LMO2','LMO4','LMX1A','LMX1B','LPP','LRP5','LRP6','LRP8','LRRFIP1','LRRFIP2','LSR','LTF','LTN1','LYAR','LYL1','LZTFL1','LZTR1','LZTS1','MACC1','MACF1','MAD2L2','MAEL','MAF','MAFA','MAFB','MAFF','MAFG','MAFK','MALT1','MAML1','MAML3','MAP2K5','MAP3K10','MAP3K13','MAP3K7','MAPK1','MAPK10','MAPK11','MAPK14','MAPK3','MAPK8','MAPK9','MATR3','MAVS','MAX','MAZ','MB21D1','MBD1','MBD2','MBD3','MBD4','MBD6','MBNL1','MBNL2','MBNL3','MBTPS2','MDFI','MDM2','MDM4','MECOM','MECP2','MED1','MED12','MED13','MED13L','MED14','MED15','MED16','MED17','MED21','MED23','MED24','MED6','MEF2A','MEF2B','MEF2C','MEF2D','MEIS1','MEIS2','MEIS3','MEN1','MEOX1','MEOX2','MESP1','MESP2','MET','MGA','MGMT','MID2','MIER1','MIER2','MIER3','MINK1','MIS18BP1','MITF','MIXL1','MKL1','MKL2','MKRN1','MKRN2','MKRN3','MKX','MLLT1','MLLT10','MLLT11','MLLT3','MLLT6','MLX','MLXIP','MLXIPL','MN1','MNAT1','MNT','MNX1','MORC3','MPG','MRPL28','MRRF','MSANTD1','MSANTD3','MSANTD4','MSC','MSGN1','MSH2','MSH3','MSH6','MSL3','MSRB2','MST1R','MSX1','MSX2','MTA1','MTA2','MTA3','MTDH','MTERF1','MTERF2','MTERF3','MTERF4','MTF1','MTF2','MTPN','MUS81','MXD1','MXD3','MXD4','MXI1','MYB','MYBBP1A','MYBL1','MYBL2','MYC','MYCBP','MYCL','MYCN','MYD88','MYEF2','MYF5','MYF6','MYNN','MYOCD','MYOD1','MYOG','MYPOP','MYRF','MYRFL','MYSM1','MYT1','MYT1L','MZF1','NAB2','NABP2','NACC2','NAIF1','NANOG','NANOGNB','NANOGP8','NCOA1','NCOA2','NCOA3','NCOA4','NCOA5','NCOA6','NCOA7','NCOR1','NCOR2','NDC80','NDN','NDP','NEB','NEIL1','NEUROD1','NEUROD2','NEUROD4','NEUROD6','NEUROG1','NEUROG2','NEUROG3','NFAM1','NFAT5','NFATC1','NFATC2','NFATC3','NFATC4','NFE2','NFE2L1','NFE2L2','NFE2L3','NFE4','NFIA','NFIB','NFIC','NFIL3','NFIX','NFKB1','NFKB2','NFKBIA','NFKBIB','NFKBID','NFKBIL1','NFKBIZ','NFRKB','NFX1','NFXL1','NFYA','NFYB','NFYC','NHLH1','NHLH2','NKRF','NKX1-1','NKX1-2','NKX2-1','NKX2-2','NKX2-3','NKX2-4','NKX2-5','NKX2-6','NKX2-8','NKX3-1','NKX3-2','NKX6-1','NKX6-2','NKX6-3','NLRC3','NLRC4','NLRC5','NLRP12','NLRP2B','NLRP3','NME2','NOBOX','NOC3L','NOC4L','NOCT','NOD1','NOD2','NODAL','NONO','NOP2','NOTCH1','NOTCH2','NOTCH3','NOTO','NPAS1','NPAS2','NPAS3','NPAS4','NPM1','NR0B1','NR0B2','NR1D1','NR1D2','NR1H2','NR1H3','NR1H4','NR1I2','NR1I3','NR2C1','NR2C2','NR2E1','NR2E3','NR2F1','NR2F2','NR2F6','NR3C1','NR3C2','NR4A1','NR4A2','NR4A3','NR5A1','NR5A2','NR6A1','NRBF2','NRF1','NRG1','NRIP1','NRIP3','NRK','NRL','NSD1','NSD2','NTRK1','NUCKS1','NUFIP1','NUP98','NUPL2']
TFs2Other=['OGG1','OLIG1','OLIG2','OLIG3','ONECUT1','ONECUT2','ONECUT3','OPRD1','OSR1','OSR2','OTOP3','OTP','OTULIN','OTX1','OTX2','OVOL1','OVOL2','OVOL3','PA2G4','PABPN1','PARK7','PARP1','PARP10','PARP12','PATZ1','PAWR','PAX1','PAX2','PAX3','PAX4','PAX5','PAX6','PAX7','PAX8','PAX9','PAXBP1','PAXIP1','PBRM1','PBX1','PBX2','PBX3','PBX4','PCBD1','PCBP1','PCBP2','PCBP3','PCGF2','PCGF6','PCNA','PCSK6','PDCD7','PDS5B','PDX1','PEG3','PELI1','PER1','PER2','PER3','PEX14','PFDN1','PGBD1','PGR','PGS1','PHB','PHB2','PHF1','PHF19 ','PHF20','PHF21A','PHF5A','PHOX2A','PHOX2B','PHTF1','PHTF2','PIAS1','PIAS2','PIAS3','PIAS4','PIDD1','PIKFYVE','PIM1','PIN1','PINK1','PITX1','PITX2','PITX3','PKHD1','PKHD1L1','PKNOX1','PKNOX2','PLA2G10','PLA2G1B','PLAG1','PLAGL1','PLAGL2','PLEK','PLEK2','PLEKHA4','PLPP3','PLSCR1','PLXNA1','PLXNA2','PLXNA3','PLXNA4','PLXNB1','PLXNB2','PLXNB3','PLXNC1','PLXND1','PML','PMS1','PNKP','POGK','POGZ','POLA1','POLB','POLE3','POLE4','POLG','POLH','POLI','POLK','POLL','POLM','POLN','POLQ','POLR1B','POLR2A','POLR2B','POLR2E','POLR2J','POLR2K','POLR2L','POLR3A','POLRMT','POT1','POU1F1','POU2AF1','POU2F1','POU2F2','POU2F3','POU3F1','POU3F2','POU3F3','POU3F4','POU4F1','POU4F2','POU4F3','POU5F1','POU5F1B','POU5F2','POU6F1','POU6F2','PPARA','PPARD','PPARG','PPARGC1A','PPARGC1B','PPP1R10','PPP1R13L','PPP1R21','PPP2R3B','PPP2R5B','PPP3CA','PPRC1','PQBP1','PRAM1','PRAP1','PRB3','PRDM1','PRDM10','PRDM11','PRDM12','PRDM13','PRDM14','PRDM15','PRDM16','PRDM2','PRDM4','PRDM5','PRDM6','PRDM7','PRDM8','PRDM9','PRDX3','PREB','PREX1','PREX2','PRH2','PRIM2','PRIMPOL','PRKCB','PRKCH','PRKCI','PRKCQ','PRKCZ','PRKD1','PRKD2','PRKN','PRMT2','PRMT3','PRNP','PROP1','PROX1','PROX2','PRR12','PRR3','PRRX1','PRRX2','PSMA6','PSMC5','PSMD10','PSMD11','PSMD12','PSPC1','PTCH1','PTEN','PTF1A','PTGIS','PTH','PTTG1','PUM3','PURA','PURB','PURG','PYCARD','PYDC1','PYDC2','QRICH1','RAB11FIP4','RAB7B','RAD21','RAD51','RAG1','RAI1','RAPGEF3','RAPGEF4','RAPGEF5','RARA','RARB','RARG','RAX','RAX2','RB1','RBAK','RBBP7','RBCK1','RBL1','RBL2','RBM10','RBM20','RBM22','RBM26','RBM27','RBM5','RBM6','RBPJ','RBPJL','RBSN','RC3H1','RC3H2','RCAN1','RCOR1','RCOR2','RCOR3','RECQL','REL','RELA','RELB','RELL2','RELN','REPIN1','RERE','REST','REV1','REXO4','RFC1','RFX1','RFX2','RFX3','RFX4','RFX5','RFX6','RFX7','RFX8','RFXANK','RFXAP','RGCC','RGS11','RGS6','RGS7','RGS9','RHEBL1','RHOXF1','RHOXF2','RHOXF2B','RING1','RIOK2','RIPK1','RIPK2','RIPK3','RIPK4','RLF','RLIM','RNASE2','RNASEH1','RNF113A','RNF113B','RNF114','RNF125','RNF138','RNF14','RNF166','RNF2','RNF219','RNF25','RNF31','RNF4','RNF41','RORA','RORB','RORC','RPA2','RPA4','RPL36A','RPL7','RPL7A','RPS27A','RPS6KA4','RPS6KA5','RREB1','RRN3','RUNX1','RUNX1T1','RUNX2','RUNX3','RWDD3','RXRA','RXRB','RXRG','RYBP','S100A12','S100A8','S100A9','SAFB','SAFB2','SALL1','SALL2','SALL3','SALL4','SART1','SATB1','SATB2','SAV1','SCAND1','SCAND2P','SCAPER','SCMH1','SCML1','SCML2','SCML4','SCOC','SCRT1','SCRT2','SCX','SEBOX','SEMA4A','SERTAD1','SERTAD2','SETBP1','SETD6','SETDB1','SETDB2','SF3A2','SF3A3','SFPQ','SFRP4','SFRP5','SFTPB','SGK1','SGK3','SGSM2','SHH','SHOX','SHOX2','SHPRH','SIGIRR','SIK1','SIM1','SIM2','SIN3A','SIN3B','SIRT1','SIRT2','SIRT3','SIRT6','SIVA1','SIX1','SIX2','SIX3','SIX4','SIX5','SIX6','SKI','SKIDA1','SKIL','SKOR1','SKOR2','SLC22A4','SLC26A10','SLC26A3','SLC2A4RG','SLC30A9','SLC39A10','SLC4A10','SMAD1','SMAD2','SMAD3','SMAD4','SMAD5','SMAD6','SMAD7','SMAD9','SMARCA1','SMARCA2','SMARCA4','SMARCA5','SMARCAL1','SMARCB1','SMARCC1','SMARCC2','SMARCD1','SMARCD2','SMARCD3','SMARCE1','SMC3','SMO','SMUG1','SMYD1','SMYD3','SNAI1','SNAI2','SNAI3','SNAPC2','SNAPC4','SNAPC5','SNRPC','SNURF','SNW1','SOGA3','SOHLH1','SOHLH2','SON','SORBS2','SOX1','SOX10','SOX11','SOX12','SOX13','SOX14','SOX15','SOX17','SOX18','SOX2','SOX21','SOX3','SOX30','SOX4','SOX5','SOX6','SOX7','SOX8','SOX9','SP1','SP100','SP110','SP140','SP140L','SP2','SP3','SP4','SP5','SP6','SP7','SP8','SP9','SPDEF','SPEN','SPHK1','SPI1','SPIB','SPIC','SPZ1','SRCAP','SREBF1','SREBF2','SRF','SRY','SSB','SSBP2','SSBP3','SSBP4','SSH1','SSH2','SSH3','SSRP1','SSX1','ST18','STAG1','STAT1','STAT2','STAT3','STAT4','STAT5A','STAT5B','STAT6','STK16','STK3','STK36','STRN','STRN3','SUB1','SUFU','SUMO1','SUPT16H','SUPT4H1','SUPT5H','SUPT6H','SUV39H1','SUV39H2','SUZ12','SYK','T','TAB1','TAB2','TAB3','TADA2A','TADA2B','TADA3','TAF1','TAF10','TAF11','TAF12','TAF13','TAF1A','TAF1B','TAF1C','TAF1L','TAF2','TAF3','TAF4','TAF4B','TAF5','TAF5L','TAF6','TAF6L','TAF7','TAF7L','TAF8','TAF9','TAF9B','TAL1','TAL2','TARDBP','TAX1BP1','TBL1Y','TBP','TBPL1','TBPL2','TBR1','TBX1','TBX10','TBX15','TBX18','TBX19','TBX2','TBX20','TBX21','TBX22','TBX3','TBX4','TBX5','TBX6','TCEA2','TCEAL1','TCEAL7','TCEAL8','TCF12','TCF15','TCF19','TCF20','TCF21','TCF23','TCF24','TCF25','TCF3','TCF4','TCF7','TCF7L1','TCF7L2','TCFL5','TDG','TDP1','TDP2','TEAD1','TEAD2','TEAD3','TEAD4','TEF','TERB1','TERF1','TERF2','TERF2IP','TESMIN','TET1','TET2','TET3','TFAM','TFAP2A','TFAP2B','TFAP2C','TFAP2D','TFAP2E','TFAP4','TFCP2','TFCP2L1','TFDP1','TFDP2','TFDP3','TFE3','TFEB','TFEC','TGFB1','TGFB1I1','TGIF1','TGIF2','TGIF2LX','TGIF2LY','TGS1','THAP1','THAP10','THAP11','THAP12','THAP2','THAP3','THAP4','THAP5','THAP6','THAP7','THAP8','THAP9','THRA','THRAP3','THRB','THYN1','TIA1','TICAM1','TIGD1','TIGD2','TIGD3','TIGD4','TIGD5','TIGD6','TIGD7','TIPARP','TIRAP','TLE1','TLE2','TLE3','TLE4','TLE6','TLR2','TLR3','TLR4','TLR9','TLX1','TLX2','TLX3','TMEM235','TMF1','TNF','TNFAIP3','TNFRSF11A','TNFRSF4','TNFSF11','TNFSF18','TNFSF4','TOE1','TONSL','TOP1','TOP2A','TOP2B','TOPORS','TOX','TOX2','TOX3','TOX4','TP53','TP53BP1','TP63','TP73','TPRX1','TRADD','TRAF1','TRAF2','TRAF3','TRAF4','TRAF5','TRAF6','TRAFD1','TRAK2','TRAPPC9','TRERF1','TREX1','TRIB1','TRIB3','TRIM13','TRIM14','TRIM15','TRIM21','TRIM22','TRIM23','TRIM24','TRIM25','TRIM26','TRIM27','TRIM28','TRIM29','TRIM3','TRIM31','TRIM32','TRIM33','TRIM34','TRIM37','TRIM38','TRIM40','TRIM5','TRIM52','TRIM62','TRIM8','TRIP11','TRIT1','TRMT1','TRMT1L','TRPS1','TRRAP','TSC2','TSC22D1','TSC22D2','TSC22D3','TSC22D4','TSHZ1','TSHZ2','TSHZ3','TSN','TSSK4','TTF1','TUB','TULP4','TUT1','TWIST1','TWIST2','TXK','U2AF1','U2AF1L4','UBA52','UBB','UBC','UBE2K','UBE2N','UBE2V1','UBN1','UBP1','UBR4','UBTF','UBTFL1','UBTFL6','UFL1','UHRF1','UHRF2','UNCX','UNG','UNK','UNKL','USF1','USF2','USF3','USP39','USP7','UTF1','VAV1','VAX1','VAX2','VDR','VEGFA','VENTX','VEZF1','VGLL1','VGLL2','VGLL3','VHLL','VPS72','VSX1','VSX2','WDHD1','WFS1','WIZ','WNT1','WNT10B','WNT2','WNT3A','WNT5A','WNT8B','WRN','WT1','WWOX','WWP2','WWTR1','XBP1','XCL1','XPA','YBX1','YBX2','YBX3','YEATS2','YEATS4','YOD1','YY1','YY2','ZBED1','ZBED2','ZBED3','ZBED4','ZBED5','ZBED6','ZBED6CL','ZBED8','ZBED9','ZBP1','ZBTB1','ZBTB10','ZBTB11','ZBTB12','ZBTB14','ZBTB16','ZBTB17','ZBTB18','ZBTB2','ZBTB20','ZBTB21','ZBTB22','ZBTB24','ZBTB25','ZBTB26','ZBTB3','ZBTB32','ZBTB33','ZBTB34','ZBTB37','ZBTB38','ZBTB39','ZBTB4','ZBTB40','ZBTB41','ZBTB42','ZBTB43','ZBTB44','ZBTB45','ZBTB46','ZBTB47','ZBTB48','ZBTB49','ZBTB5','ZBTB6','ZBTB7A','ZBTB7B','ZBTB7C','ZBTB8A','ZBTB8B','ZBTB9','ZC3H10','ZC3H11A','ZC3H12A','ZC3H13','ZC3H14','ZC3H15','ZC3H18','ZC3H3','ZC3H4','ZC3H6','ZC3H7A','ZC3H7B','ZC3H8','ZC3HAV1','ZCCHC11','ZCCHC6','ZDHHC11','ZDHHC19','ZEB1','ZEB2','ZFAND5','ZFAT','ZFHX2','ZFHX3','ZFHX4','ZFP1','ZFP14','ZFP2','ZFP28','ZFP3','ZFP30','ZFP36','ZFP36L1','ZFP36L2','ZFP37','ZFP41','ZFP42','ZFP57','ZFP62','ZFP64','ZFP69','ZFP69B','ZFP82','ZFP90','ZFP91','ZFP92','ZFPL1','ZFPM1','ZFPM2','ZFR','ZFR2','ZFX','ZFY','ZFYVE26','ZGLP1','ZGPAT','ZHX1','ZHX2','ZHX3','ZIC1','ZIC2','ZIC3','ZIC4','ZIC5','ZIK1','ZIM2','ZIM3','ZKSCAN1','ZKSCAN2','ZKSCAN3','ZKSCAN4','ZKSCAN5','ZKSCAN7','ZKSCAN8','ZMAT1','ZMAT2','ZMAT3','ZMAT4','ZMAT5','ZMYM1','ZMYM2','ZMYM3','ZMYM4','ZMYM5','ZMYM6','ZMYND10','ZNF10','ZNF100','ZNF101','ZNF106','ZNF107','ZNF112','ZNF114','ZNF117','ZNF12','ZNF121','ZNF124','ZNF131','ZNF132','ZNF133','ZNF134','ZNF135','ZNF136','ZNF137P','ZNF138','ZNF14','ZNF140','ZNF141','ZNF142','ZNF143','ZNF146','ZNF148','ZNF154','ZNF155','ZNF157','ZNF16','ZNF160','ZNF165','ZNF169','ZNF17','ZNF174','ZNF175','ZNF177','ZNF18','ZNF180','ZNF181','ZNF182','ZNF184','ZNF185','ZNF189','ZNF19','ZNF195','ZNF197','ZNF2','ZNF20','ZNF200','ZNF202','ZNF205','ZNF207','ZNF208','ZNF211','ZNF212','ZNF213','ZNF214','ZNF215','ZNF217','ZNF219','ZNF22','ZNF221','ZNF222','ZNF223','ZNF224','ZNF225','ZNF226','ZNF227','ZNF229','ZNF23','ZNF230','ZNF232','ZNF233','ZNF234','ZNF235','ZNF236','ZNF239','ZNF24','ZNF248','ZNF25','ZNF250','ZNF251','ZNF253','ZNF254','ZNF256','ZNF257','ZNF26','ZNF260','ZNF263','ZNF264','ZNF266','ZNF267','ZNF268','ZNF273','ZNF274','ZNF275','ZNF276','ZNF277','ZNF28','ZNF280A','ZNF280B','ZNF280C','ZNF280D','ZNF281','ZNF282','ZNF283','ZNF284','ZNF285','ZNF286A','ZNF286B','ZNF287','ZNF292','ZNF296','ZNF3','ZNF30','ZNF300','ZNF302','ZNF304','ZNF311','ZNF316','ZNF317','ZNF318','ZNF319','ZNF32','ZNF320','ZNF322','ZNF324','ZNF324B','ZNF326','ZNF329','ZNF330','ZNF331','ZNF333','ZNF334','ZNF335','ZNF337','ZNF33A','ZNF33B','ZNF34','ZNF341','ZNF343','ZNF345','ZNF346','ZNF347','ZNF35','ZNF350','ZNF354A','ZNF354B','ZNF354C','ZNF355P','ZNF358','ZNF362','ZNF365','ZNF366','ZNF367','ZNF37A','ZNF382','ZNF383','ZNF384','ZNF385A','ZNF385B','ZNF385C','ZNF385D','ZNF391','ZNF394','ZNF395','ZNF396','ZNF397','ZNF398','ZNF404','ZNF407','ZNF408','ZNF41','ZNF410','ZNF414','ZNF415','ZNF416','ZNF417','ZNF418','ZNF419','ZNF420','ZNF423','ZNF425','ZNF426','ZNF428','ZNF429','ZNF43','ZNF430','ZNF431','ZNF432','ZNF433','ZNF436','ZNF438','ZNF439','ZNF44','ZNF440','ZNF441','ZNF442','ZNF443','ZNF444','ZNF445','ZNF446','ZNF449','ZNF45','ZNF451','ZNF454','ZNF460','ZNF461','ZNF462','ZNF467','ZNF468','ZNF469','ZNF470','ZNF471','ZNF473','ZNF474','ZNF479','ZNF48','ZNF480','ZNF483','ZNF484','ZNF485','ZNF486','ZNF487','ZNF488','ZNF490','ZNF491','ZNF492','ZNF493','ZNF496','ZNF497','ZNF500','ZNF501','ZNF502','ZNF503','ZNF506','ZNF507','ZNF510','ZNF511','ZNF512','ZNF512B','ZNF513','ZNF514','ZNF516','ZNF517','ZNF518A','ZNF518B','ZNF519']
TFs3Other=['ZNF521','ZNF524','ZNF525','ZNF526','ZNF527','ZNF528','ZNF529','ZNF530','ZNF532','ZNF534','ZNF536','ZNF540','ZNF541','ZNF542P','ZNF543','ZNF544','ZNF546','ZNF547','ZNF548','ZNF549','ZNF550','ZNF551','ZNF552','ZNF554','ZNF555','ZNF556','ZNF557','ZNF558','ZNF559','ZNF560','ZNF561','ZNF562','ZNF563','ZNF564','ZNF565','ZNF566','ZNF567','ZNF568','ZNF569','ZNF57','ZNF570','ZNF571','ZNF572','ZNF573','ZNF574','ZNF575','ZNF576','ZNF577','ZNF578','ZNF579','ZNF580','ZNF581','ZNF582','ZNF583','ZNF584','ZNF585A','ZNF585B','ZNF586','ZNF587','ZNF587B','ZNF589','ZNF592','ZNF593','ZNF594','ZNF595','ZNF596','ZNF597','ZNF598','ZNF599','ZNF600','ZNF605','ZNF606','ZNF607','ZNF608','ZNF609','ZNF610','ZNF611','ZNF613','ZNF614','ZNF615','ZNF616','ZNF618','ZNF619','ZNF620','ZNF621','ZNF622','ZNF623','ZNF624','ZNF625','ZNF626','ZNF627','ZNF628','ZNF629','ZNF630','ZNF638','ZNF639','ZNF641','ZNF644','ZNF645','ZNF646','ZNF648','ZNF649','ZNF652','ZNF653','ZNF654','ZNF655','ZNF658','ZNF66','ZNF660','ZNF662','ZNF664','ZNF665','ZNF667','ZNF668','ZNF669','ZNF670','ZNF671','ZNF672','ZNF674','ZNF675','ZNF676','ZNF677','ZNF678','ZNF679','ZNF680','ZNF681','ZNF682','ZNF683','ZNF684','ZNF687','ZNF688','ZNF689','ZNF69','ZNF691','ZNF692','ZNF695','ZNF696','ZNF697','ZNF699','ZNF7','ZNF70','ZNF700','ZNF701','ZNF703','ZNF704','ZNF705A','ZNF705B','ZNF705D','ZNF705E','ZNF705G','ZNF706','ZNF707','ZNF708','ZNF709','ZNF71','ZNF710','ZNF711','ZNF713','ZNF714','ZNF716','ZNF717','ZNF718','ZNF720','ZNF721','ZNF724','ZNF726','ZNF727','ZNF728','ZNF729','ZNF73','ZNF730','ZNF732','ZNF735','ZNF736','ZNF737','ZNF738','ZNF74','ZNF740','ZNF746','ZNF747','ZNF749','ZNF750','ZNF75A','ZNF75D','ZNF76','ZNF761','ZNF763','ZNF764','ZNF765','ZNF766','ZNF768','ZNF77','ZNF770','ZNF771','ZNF772','ZNF773','ZNF774','ZNF775','ZNF776','ZNF777','ZNF778','ZNF780A','ZNF780B','ZNF781','ZNF782','ZNF783','ZNF784','ZNF785','ZNF786','ZNF787','ZNF788','ZNF789','ZNF79','ZNF790','ZNF791','ZNF792','ZNF793','ZNF799','ZNF8','ZNF80','ZNF800','ZNF804A','ZNF804B','ZNF805','ZNF808','ZNF81','ZNF812P','ZNF813','ZNF814','ZNF816','ZNF821','ZNF823','ZNF827','ZNF829','ZNF83','ZNF830','ZNF831','ZNF835','ZNF836','ZNF837','ZNF839','ZNF84','ZNF841','ZNF843','ZNF844','ZNF845','ZNF846','ZNF85','ZNF850','ZNF852','ZNF853','ZNF860','ZNF862','ZNF865','ZNF878','ZNF879','ZNF880','ZNF883','ZNF888','ZNF891','ZNF90','ZNF91','ZNF92','ZNF93','ZNF98','ZNF99','ZNFX1','ZRANB2','ZRSR1','ZRSR2','ZSCAN1','ZSCAN10','ZSCAN12','ZSCAN16','ZSCAN18','ZSCAN2','ZSCAN20','ZSCAN21','ZSCAN22','ZSCAN23','ZSCAN25','ZSCAN26','ZSCAN29','ZSCAN30','ZSCAN31','ZSCAN32','ZSCAN4','ZSCAN5A','ZSCAN5B','ZSCAN5C','ZSCAN5DP','ZSCAN9','ZUFSP','ZXDA','ZXDB','ZXDC','ZZZ3']
TFs4Other=['ATF2','ATF6B','ATN1','BHLHA15','BHLHB2','BHLHB4 ','BHLHB5','BHLHB9','C13ORF8','C20ORF194','CDX2','CRAMP1L','ELP3','ERVK3-1','ETF1','FLJ12895','FLJ20321','FLJ25476','FLJ35784','FOXM1','FOXO2','FOXR1','FOXR2','GATA1','GRLF1','KDM4C','KDM4E','KIAA0863','KIAA0961','KLRG1','LBXCOR1','LEF1','LHX7','LOC126295','LOC51058','LOC91661','MGC4728','MYOD1','MYOG','MYRF','NEUROD4','NEUROG1','NFAT2','NFAT4','NFE2L1','NR1A2','NR1B1','NR1B2','NR1B3','NR1C1','NR1C2','NR1C3','NR1F3','NR1I1','NR2A1','NR2B3','NR3A1','NR3A2','NR3B3','OTEX','PEPP-2','PHF2','POU2F1','POU2F2','POU4F3','POU5F1','POU6F1','PRDM3','PRKRIP1','PRKRIR','SETD1A','SMAD8','SMYD2','SOX12','SOX14','SOX15','TBX14','TCF1','TCF17','TCF2','TCF9','TEAD1','TPRXL','UBTFL1','WHSC1','ZBTB14','ZBTB18','ZBTB19','ZBTB21','ZBTB27','ZBTB28','ZBTB35','ZBTB49','ZFP161','ZFP28','ZKSCAN10','ZKSCAN11','ZKSCAN12','ZKSCAN13','ZKSCAN14','ZKSCAN16','ZKSCAN19','ZKSCAN21','ZKSCAN24','ZKSCAN6','ZKSCAN8','ZNF10','ZNF112','ZNF187','ZNF193','ZNF204P','ZNF238','ZNF271','ZNF295','ZNF305','ZNF323','ZNF434','ZNF499','ZNF542','ZNF545','ZNF553','ZNF56','ZNF625','ZNF643','ZNF698','ZNF705F','ZNF751','ZNF752','ZNF753','ZNF785','ZNF806','ZNF816','ZNF833P','ZNF906','ZSCAN1','ZSCAN25','ZSCAN26','ZSCAN31','ZSCAN32','ZSCAN5D','ZSCAN9']
TFsOther=TFs1Other+TFs2Other+TFs3Other+TFs4Other
```

Example 8 (unknown):
```unknown
S genes: ['MCM5', 'PCNA', 'TYMS', 'FEN1', 'MCM2', 'MCM4', 'RRM1', 'UNG', 'GINS2', 'MCM6', 'CDCA7', 'DTL', 'PRIM1', 'UHRF1', 'HELLS', 'RFC2', 'RPA2', 'NASP', 'RAD51AP1', 'GMNN', 'WDR76', 'SLBP', 'CCNE2', 'UBR7', 'POLD3', 'MSH2', 'ATAD2', 'RAD51', 'RRM2', 'CDC45', 'CDC6', 'EXO1', 'TIPIN', 'DSCC1', 'BLM', 'CASP8AP2', 'USP1', 'CLSPN', 'POLA1', 'CHAF1B', 'BRIP1', 'E2F8'] 

G2 genes: ['HMGB2', 'CDK1', 'NUSAP1', 'UBE2C', 'BIRC5', 'TPX2', 'TOP2A', 'NDC80', 'CKS2', 'NUF2', 'CKS1B', 'MKI67', 'TMPO', 'CENPF', 'TACC3', 'SMC4', 'CCNB2', 'CKAP2L', 'CKAP2', 'AURKB', 'BUB1', 'KIF11', 'ANP32E', 'TUBB4B', 'GTSE1', 'KIF20B', 'HJURP', 'CDCA3', 'CDC20', 'TTK', 'CDC25C', 'KIF2C', 'RANGAP1', 'NCAPD2', 'DLGAP5', 'CDCA2', 'CDCA8', 'ECT2', 'KIF23', 'HMMR', 'AURKA', 'PSRC1', 'ANLN', 'LBR', 'CKAP5', 'CENPE', 'CTCF', 'NEK2', 'G2E3', 'GAS2L3', 'CBX5', 'CENPA'] 

Hemoglobin genes set 1: ['HBB', 'HBD', 'HBG1', 'HBG2', 'HBE1', 'HBZ', 'HBM', 'HBA2', 'HBA1', 'HBQ1'] 

Hemoglobin genes set 2: ['AHSP', 'GYPA', 'RHAG', 'GFI1B', 'KEL', 'EPB42', 'ALAS2', 'TSPO2', 'GYPB', 'GYPE', 'SPTA1', 'RHCE', 'KLF1', 'RHD', 'SLC4A1', 'SNX22', 'SQSTM1', 'SOD1', 'FTH1', 'MAP1LC3B', 'SLC25A37', 'TRIM58', 'HMBS'] 

Mitochondrial genes: ['MT-ND1', 'MT-ND2', 'MT-CO1', 'MT-CO2', 'MT-ATP8', 'MT-ATP6', 'MT-CO3', 'MT-ND3', 'MT-ND4L', 'MT-ND4', 'MT-ND5', 'MT-ND6', 'MT-CYB'] 

Ribo genes: ['RPLP0', 'RPLP1', 'RPLP2', 'RPL3', 'RPL3L', 'RPL4', 'RPL5', 'RPL6', 'RPL7', 'RPL7A', 'RPL7L1', 'RPL8', 'RPL9', 'RPL10', 'RPL10A', 'RPL10L', 'RPL11', 'RPL12', 'RPL13', 'RPL13A', 'RPL14', 'RPL15', 'RPL17', 'RPL18', 'RPL18A', 'RPL19', 'RPL21', 'RPL22', 'RPL22L1', 'RPL23', 'RPL23A', 'RPL24', 'RPL26', 'RPL26L1', 'RPL27', 'RPL27A', 'RPL28', 'RPL29', 'RPL30', 'RPL31', 'RPL32', 'RPL34', 'RPL35', 'RPL35A', 'RPL36', 'RPL36A', 'RPL36AL', 'RPL37', 'RPL37A', 'RPL38', 'RPL39', 'RPL39L', 'UBA52', 'RPL41', 'RPSA', 'RPS2', 'RPS3', 'RPS3A', 'RPS4X', 'RPS4Y1', 'RPS4Y2', 'RPS5', 'RPS6', 'RPS7', 'RPS8', 'RPS9', 'RPS10', 'RPS11', 'RPS12', 'RPS13', 'RPS14', 'RPS15', 'RPS15A', 'RPS16', 'RPS17', 'RPS18', 'RPS19', 'RPS20', 'RPS21', 'RPS23', 'RPS24', 'RPS25', 'RPS26', 'RPS27', 'RPS27A', 'RPS27L', 'RPS28', 'RPS29', 'FAU'] 

Tissue dissociation stress associated genes: ['FOS', 'FOSB', 'JUN', 'JUNB', 'JUND', 'ZFP36', 'SOCS3', 'CEBPD', 'CEBPB', 'ATF3', 'EGR1', 'HSPA1A', 'HSPA1B', 'HSP90AA1', 'HSP90AB1', 'HSPA8', 'HSPB1', 'HSPE1', 'IER3', 'IER2', 'BTG1', 'BTG2', 'DUSP1']
```

Example 9 (python):
```python
set(adata.obs.coarse_grain)
```

Example 10 (unknown):
```unknown
{'Cardiomyocytes',
 'Endothelium',
 'Epicardium',
 'Immune',
 'Mesenchymal',
 'Neural'}
```

Example 11 (python):
```python
adata=adata[adata.obs.coarse_grain=='Neural',:].copy()
adata
```

Example 12 (unknown):
```unknown
AnnData object with n_obs × n_vars = 6002 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'batch_key', '_scvi_batch', '_scvi_labels', 'previous_coarse_grain', 'previous_mid_grain', 'previous_fine_grain', 'leiden_scVI', 'FACSgate', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', 'hb2_score', 'leiden', 'coarse_grain', 'sex'
    var: 'gene_name_scRNA-0', 'gene_id'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'dendrogram_leiden', 'donor_colors', 'gender_colors', 'hvg', 'kit_10x_colors', 'leiden', 'leiden_colors', 'modality_colors', 'neighbors', 'phase_colors', 'rank_genes_groups', 'region_colors', 'sex_colors', 'umap'
    obsm: 'X_pca', 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 13 (python):
```python
# preserve raw count
adata.layers["counts"] = adata.X.copy()

# normalise and log-transform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key='batch_key')

print(sum(adata.var.highly_variable))
sc.pl.highly_variable_genes(adata)

# freeze the state in `.raw`
adata.raw = adata
```

Example 14 (python):
```python
normalizing counts per cell
    finished (0:00:00)
extracting highly variable genes
    finished (0:00:07)
--> added
    'highly_variable', boolean vector (adata.var)
    'means', float vector (adata.var)
    'dispersions', float vector (adata.var)
    'dispersions_norm', float vector (adata.var)
1571
```

Example 15 (python):
```python
# subset hvgs
adata = adata[:, adata.var.highly_variable]

adata=adata.copy()

sc.tl.score_genes_cell_cycle(adata, s_genes=s_genes, g2m_genes=g2m_genes)
#sc.pp.regress_out(adata, ['S_score','G2M_score'])

#batch correction
scvi.model.SCVI.setup_anndata(adata, 
                              layer="counts", 
                              batch_key="batch_key",
                              categorical_covariate_keys=['phase'],
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo'])

vae = scvi.model.SCVI(adata, n_hidden = 128, n_latent = 50, n_layers = 3, dispersion = 'gene-batch')
vae.train(max_epochs=50)
    
# Convergence plot
x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
plt.plot(x, vae.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
plt.clf()
```

Example 16 (python):
```python
calculating cell cycle phase
computing score 'S_score'
WARNING: genes are not in var_names and ignored: ['PCNA', 'FEN1', 'MCM2', 'RRM1', 'UNG', 'GINS2', 'MCM6', 'PRIM1', 'RFC2', 'RPA2', 'NASP', 'GMNN', 'SLBP', 'CCNE2', 'UBR7', 'POLD3', 'MSH2', 'RAD51', 'CDC45', 'CDC6', 'EXO1', 'TIPIN', 'DSCC1', 'BLM', 'CASP8AP2', 'USP1', 'POLA1', 'CHAF1B', 'E2F8']
    finished: added
    'S_score', score of gene set (adata.obs).
    369 total control genes are used. (0:00:00)
computing score 'G2M_score'
WARNING: genes are not in var_names and ignored: ['TMPO', 'ANP32E', 'CDC25C', 'RANGAP1', 'NCAPD2', 'PSRC1', 'LBR', 'CKAP5', 'CTCF', 'G2E3', 'GAS2L3', 'CBX5']
    finished: added
    'G2M_score', score of gene set (adata.obs).
    724 total control genes are used. (0:00:00)
-->     'phase', cell cycle phase (adata.obs)
```

Example 17 (unknown):
```unknown
GPU available: False, used: False
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
```

Example 18 (unknown):
```unknown
Epoch 50/50: 100%|███████████| 50/50 [00:24<00:00,  2.04it/s, loss=797, v_num=1]
```

Example 19 (unknown):
```unknown
<Figure size 320x320 with 0 Axes>
```

Example 20 (python):
```python
# add the latent space to adata
adata.obsm["X_scVI"] = vae.get_latent_representation()
sc.tl.score_genes(adata, stress_genes, score_name='stress_score')
sc.tl.score_genes(adata, hb_genes1, score_name='hb1_score')
#sc.tl.score_genes(adata, hb_genes2, score_name='hb2_score')

sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata,min_dist=0.1)
#sc.tl.tsne(adata)
```

Example 21 (python):
```python
computing score 'stress_score'
WARNING: genes are not in var_names and ignored: ['FOSB', 'JUN', 'JUND', 'SOCS3', 'CEBPB', 'ATF3', 'HSP90AA1', 'HSP90AB1', 'HSPA8', 'HSPB1', 'HSPE1', 'BTG1']
    finished: added
    'stress_score', score of gene set (adata.obs).
    390 total control genes are used. (0:00:00)
computing score 'hb1_score'
WARNING: genes are not in var_names and ignored: ['HBD', 'HBM', 'HBQ1']
    finished: added
    'hb1_score', score of gene set (adata.obs).
    345 total control genes are used. (0:00:00)
computing neighbors
    finished: added to `.uns['neighbors']`
    `.obsp['distances']`, distances for each pair of neighbors
    `.obsp['connectivities']`, weighted adjacency matrix (0:00:01)
computing UMAP
    finished: added
    'X_umap', UMAP coordinates (adata.obsm) (0:00:04)
```

Example 22 (python):
```python
sc.pl.umap(adata, color=['log1p_total_counts','n_genes','n_genes_by_counts','total_counts','pct_counts_mt','pct_counts_ribo'],ncols=5)
sc.pl.umap(adata, color=['scrublet_score','stress_score', 'hb1_score','hb2_score','phase'], ncols=5,)
sc.pl.umap(adata, color=['donor','sex','age','region','phase','cell_or_nuclei','modality','kit_10x','FACSgate'],wspace=0.8,ncols=5)
```

Example 23 (unknown):
```unknown
... storing 'phase' as categorical
```

Example 24 (python):
```python
Genes=['TTN','TNNT2','MYOM1','LDB3','ALDH1A2','TNNT1','KRT19','UPK3B','COL6A3','FBLN1','PCOLCE','LRRC17','EMCN','ECSCR','PECAM1','CDH5', 'LAPTM5','PTPRC','LCP1','CD53','PHOX2B','NRXN1','STMN2','GRIK2']
sc.pl.umap(adata, color=Genes)
```

Example 25 (python):
```python
sc.pl.umap(adata, color='previous_fine_grain')
```

Example 26 (python):
```python
#this block is to see where the cells over a certain threshold with TNNT2 expression lies in the UMAP
#enter what gene to set threshold for
#when running this box for the first time, comment the starting with 'del'
#then uncomment it for the other ones --- !!!

CellName='Cardio_toremove'
gene='TTN'
exp_thresh=2
gene_pos=adata.var_names.get_loc(gene)
adata_sel = adata[adata.X[:,gene_pos]>exp_thresh].copy()

l=adata_sel.obs_names
set(l).intersection(adata.obs_names)
print(len(l))

#del adata.obs[CellName]
adata.obs.loc[set(l).intersection(adata.obs_names),CellName]='Yes'
sc.settings.set_figure_params(dpi=200, facecolor='white', color_map = 'RdPu',)
sc.pl.umap(adata, color=[CellName], groups='Yes',size=10, wspace=0.45, legend_fontsize=5, legend_loc='on data',frameon=False)

print(adata)
adata= adata[adata.X[:,gene_pos]<exp_thresh].copy()
print(adata)
```

Example 27 (unknown):
```unknown
... storing 'Cardio_toremove' as categorical
```

Example 28 (unknown):
```unknown
AnnData object with n_obs × n_vars = 6002 × 1571
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'batch_key', '_scvi_batch', '_scvi_labels', 'previous_coarse_grain', 'previous_mid_grain', 'previous_fine_grain', 'leiden_scVI', 'FACSgate', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', 'hb2_score', 'leiden', 'coarse_grain', 'sex', 'Cardio_toremove'
    var: 'gene_name_scRNA-0', 'gene_id', 'highly_variable', 'means', 'dispersions', 'dispersions_norm', 'highly_variable_nbatches', 'highly_variable_intersection'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'dendrogram_leiden', 'donor_colors', 'gender_colors', 'hvg', 'kit_10x_colors', 'leiden', 'leiden_colors', 'modality_colors', 'neighbors', 'phase_colors', 'rank_genes_groups', 'region_colors', 'sex_colors', 'umap', 'log1p', 'age_colors', 'previous_fine_grain_colors', 'Cardio_toremove_colors'
    obsm: 'X_pca', 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    layers: 'counts'
    obsp: 'connectivities', 'distances'
AnnData object with n_obs × n_vars = 5832 × 1571
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'batch_key', '_scvi_batch', '_scvi_labels', 'previous_coarse_grain', 'previous_mid_grain', 'previous_fine_grain', 'leiden_scVI', 'FACSgate', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', 'hb2_score', 'leiden', 'coarse_grain', 'sex', 'Cardio_toremove'
    var: 'gene_name_scRNA-0', 'gene_id', 'highly_variable', 'means', 'dispersions', 'dispersions_norm', 'highly_variable_nbatches', 'highly_variable_intersection'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'dendrogram_leiden', 'donor_colors', 'gender_colors', 'hvg', 'kit_10x_colors', 'leiden', 'leiden_colors', 'modality_colors', 'neighbors', 'phase_colors', 'rank_genes_groups', 'region_colors', 'sex_colors', 'umap', 'log1p', 'age_colors', 'previous_fine_grain_colors', 'Cardio_toremove_colors'
    obsm: 'X_pca', 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    layers: 'counts'
    obsp: 'connectivities', 'distances'
```

Example 29 (python):
```python
#this block is to see where the cells over a certain threshold with TNNT2 expression lies in the UMAP
#enter what gene to set threshold for
#when running this box for the first time, comment the starting with 'del'
#then uncomment it for the other ones --- !!!

CellName='Mesenchymal_toremove'
gene='COL6A3'
exp_thresh=1.5
gene_pos=adata.var_names.get_loc(gene)
adata_sel = adata[adata.X[:,gene_pos]>exp_thresh].copy()

l=adata_sel.obs_names
set(l).intersection(adata.obs_names)
print(len(l))

#del adata.obs[CellName]
adata.obs.loc[set(l).intersection(adata.obs_names),CellName]='Yes'
sc.settings.set_figure_params(dpi=200, facecolor='white', color_map = 'RdPu',)
sc.pl.umap(adata, color=[CellName], groups='Yes',size=10, wspace=0.45, legend_fontsize=5, legend_loc='on data',frameon=False)

print(adata)
adata= adata[adata.X[:,gene_pos]<exp_thresh].copy()
print(adata)
```

Example 30 (unknown):
```unknown
... storing 'Mesenchymal_toremove' as categorical
```

Example 31 (unknown):
```unknown
AnnData object with n_obs × n_vars = 5832 × 1571
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'batch_key', '_scvi_batch', '_scvi_labels', 'previous_coarse_grain', 'previous_mid_grain', 'previous_fine_grain', 'leiden_scVI', 'FACSgate', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', 'hb2_score', 'leiden', 'coarse_grain', 'sex', 'Cardio_toremove', 'Mesenchymal_toremove'
    var: 'gene_name_scRNA-0', 'gene_id', 'highly_variable', 'means', 'dispersions', 'dispersions_norm', 'highly_variable_nbatches', 'highly_variable_intersection'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'dendrogram_leiden', 'donor_colors', 'gender_colors', 'hvg', 'kit_10x_colors', 'leiden', 'leiden_colors', 'modality_colors', 'neighbors', 'phase_colors', 'rank_genes_groups', 'region_colors', 'sex_colors', 'umap', 'log1p', 'age_colors', 'previous_fine_grain_colors', 'Cardio_toremove_colors', 'Mesenchymal_toremove_colors'
    obsm: 'X_pca', 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    layers: 'counts'
    obsp: 'connectivities', 'distances'
AnnData object with n_obs × n_vars = 5670 × 1571
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'batch_key', '_scvi_batch', '_scvi_labels', 'previous_coarse_grain', 'previous_mid_grain', 'previous_fine_grain', 'leiden_scVI', 'FACSgate', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', 'hb2_score', 'leiden', 'coarse_grain', 'sex', 'Cardio_toremove', 'Mesenchymal_toremove'
    var: 'gene_name_scRNA-0', 'gene_id', 'highly_variable', 'means', 'dispersions', 'dispersions_norm', 'highly_variable_nbatches', 'highly_variable_intersection'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'dendrogram_leiden', 'donor_colors', 'gender_colors', 'hvg', 'kit_10x_colors', 'leiden', 'leiden_colors', 'modality_colors', 'neighbors', 'phase_colors', 'rank_genes_groups', 'region_colors', 'sex_colors', 'umap', 'log1p', 'age_colors', 'previous_fine_grain_colors', 'Cardio_toremove_colors', 'Mesenchymal_toremove_colors'
    obsm: 'X_pca', 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    layers: 'counts'
    obsp: 'connectivities', 'distances'
```

Example 32 (python):
```python
#Re-run analyis with cardio-mesenchymal removed

adata_raw=sc.read('./Objects/20221130RNA_foetal_post-scVI_raw_ExtracardiacRemoved.h5ad')

print(adata_raw)
# constructing new anndata with maintaining .obs, .obsm, .obsp, .uns
# overwriting adata_raw
adata_raw=anndata.AnnData(X=adata_raw[adata.obs_names,:].X,
                      obs=adata.obs, # from annotated or subsetted data
                      var=adata_raw.var,
                      obsm=adata.obsm, # from annotated or subsetted data
                      obsp=adata.obsp, # from annotated or subsetted data
                    uns=adata.uns # from annotated or subsetted data
                               )

adata=adata_raw.copy()
print(adata)
```

Example 33 (unknown):
```unknown
AnnData object with n_obs × n_vars = 306287 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'batch_key', '_scvi_batch', '_scvi_labels', 'previous_coarse_grain', 'previous_mid_grain', 'previous_fine_grain', 'leiden_scVI', 'FACSgate', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', 'hb2_score', 'leiden', 'coarse_grain', 'sex'
    var: 'gene_name_scRNA-0', 'gene_id'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'age_colors', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'dendrogram_leiden', 'donor_colors', 'gender_colors', 'hvg', 'kit_10x_colors', 'leiden', 'leiden_colors', 'modality_colors', 'neighbors', 'phase_colors', 'rank_genes_groups', 'region_colors', 'sex_colors', 'umap'
    obsm: 'X_pca', 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
AnnData object with n_obs × n_vars = 5670 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'batch_key', '_scvi_batch', '_scvi_labels', 'previous_coarse_grain', 'previous_mid_grain', 'previous_fine_grain', 'leiden_scVI', 'FACSgate', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', 'hb2_score', 'leiden', 'coarse_grain', 'sex', 'Cardio_toremove', 'Mesenchymal_toremove'
    var: 'gene_name_scRNA-0', 'gene_id'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'dendrogram_leiden', 'donor_colors', 'gender_colors', 'hvg', 'kit_10x_colors', 'leiden', 'leiden_colors', 'modality_colors', 'neighbors', 'phase_colors', 'rank_genes_groups', 'region_colors', 'sex_colors', 'umap', 'log1p', 'age_colors', 'previous_fine_grain_colors', 'Cardio_toremove_colors', 'Mesenchymal_toremove_colors'
    obsm: 'X_pca', 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 34 (python):
```python
# preserve raw count
adata.layers["counts"] = adata.X.copy()

# normalise and log-transform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key='batch_key')

print(sum(adata.var.highly_variable))
sc.pl.highly_variable_genes(adata)

# freeze the state in `.raw`
adata.raw = adata

# subset hvgs
adata = adata[:, adata.var.highly_variable]

adata=adata.copy()

sc.tl.score_genes_cell_cycle(adata, s_genes=s_genes, g2m_genes=g2m_genes)
#sc.pp.regress_out(adata, ['S_score','G2M_score'])

#batch correction
scvi.model.SCVI.setup_anndata(adata, 
                              layer="counts", 
                              batch_key="batch_key",
                              categorical_covariate_keys=['phase'],
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo'])

vae = scvi.model.SCVI(adata, n_hidden = 128, n_latent = 50, n_layers = 3, dispersion = 'gene-batch')
vae.train(max_epochs=50)
    
# Convergence plot
x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
plt.plot(x, vae.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
plt.clf()

# add the latent space to adata
adata.obsm["X_scVI"] = vae.get_latent_representation()

sc.tl.score_genes(adata, stress_genes, score_name='stress_score')
sc.tl.score_genes(adata, hb_genes1, score_name='hb1_score')
sc.tl.score_genes(adata, hb_genes2, score_name='hb2_score')
sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata,min_dist=0.1)
#sc.tl.tsne(adata)

sc.pl.umap(adata, color=['log1p_total_counts','n_genes','n_genes_by_counts','total_counts','pct_counts_mt','pct_counts_ribo'],ncols=5)
sc.pl.umap(adata, color=['scrublet_score','stress_score', 'hb1_score','hb2_score','phase'], ncols=5,)
sc.pl.umap(adata, color=['donor','sex','age','region','phase','cell_or_nuclei','modality','kit_10x','FACSgate'],wspace=0.8,ncols=5)

Genes=['TTN','TNNT2','MYOM1','LDB3','ALDH1A2','TNNT1','KRT19','UPK3B','COL6A3','FBLN1','PCOLCE','LRRC17','EMCN','ECSCR','PECAM1','CDH5', 'LAPTM5','PTPRC','LCP1','CD53','PHOX2B','NRXN1','STMN2','GRIK2']
sc.pl.umap(adata, color=Genes)
```

Example 35 (python):
```python
normalizing counts per cell
    finished (0:00:00)
WARNING: adata.X seems to be already log-transformed.
extracting highly variable genes
    finished (0:00:07)
--> added
    'highly_variable', boolean vector (adata.var)
    'means', float vector (adata.var)
    'dispersions', float vector (adata.var)
    'dispersions_norm', float vector (adata.var)
1518
```

Example 36 (python):
```python
calculating cell cycle phase
computing score 'S_score'
WARNING: genes are not in var_names and ignored: ['PCNA', 'FEN1', 'MCM2', 'MCM4', 'RRM1', 'UNG', 'GINS2', 'MCM6', 'PRIM1', 'RFC2', 'RPA2', 'NASP', 'GMNN', 'SLBP', 'CCNE2', 'UBR7', 'POLD3', 'MSH2', 'RAD51', 'CDC45', 'CDC6', 'EXO1', 'TIPIN', 'DSCC1', 'BLM', 'CASP8AP2', 'USP1', 'POLA1', 'CHAF1B', 'E2F8']
    finished: added
    'S_score', score of gene set (adata.obs).
    410 total control genes are used. (0:00:00)
computing score 'G2M_score'
WARNING: genes are not in var_names and ignored: ['TMPO', 'HJURP', 'CDC25C', 'RANGAP1', 'NCAPD2', 'PSRC1', 'LBR', 'CKAP5', 'CTCF', 'G2E3', 'GAS2L3', 'CBX5']
    finished: added
    'G2M_score', score of gene set (adata.obs).
    772 total control genes are used. (0:00:00)
-->     'phase', cell cycle phase (adata.obs)
```

Example 37 (unknown):
```unknown
GPU available: False, used: False
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
```

Example 38 (unknown):
```unknown
Epoch 50/50: 100%|███████████| 50/50 [00:22<00:00,  2.21it/s, loss=778, v_num=1]
```

Example 39 (python):
```python
computing score 'stress_score'
WARNING: genes are not in var_names and ignored: ['FOSB', 'JUN', 'JUND', 'SOCS3', 'ATF3', 'HSP90AA1', 'HSP90AB1', 'HSPA8', 'HSPB1', 'HSPE1', 'IER3', 'BTG1']
    finished: added
    'stress_score', score of gene set (adata.obs).
    491 total control genes are used. (0:00:00)
computing score 'hb1_score'
WARNING: genes are not in var_names and ignored: ['HBD', 'HBM', 'HBQ1']
    finished: added
    'hb1_score', score of gene set (adata.obs).
    293 total control genes are used. (0:00:00)
computing score 'hb2_score'
WARNING: genes are not in var_names and ignored: ['AHSP', 'GYPA', 'RHAG', 'GFI1B', 'KEL', 'EPB42', 'ALAS2', 'TSPO2', 'GYPB', 'GYPE', 'SPTA1', 'RHCE', 'KLF1', 'RHD', 'SLC4A1', 'SNX22', 'SQSTM1', 'SOD1', 'FTH1', 'MAP1LC3B', 'TRIM58', 'HMBS']
    finished: added
    'hb2_score', score of gene set (adata.obs).
    49 total control genes are used. (0:00:00)
computing neighbors
    finished: added to `.uns['neighbors']`
    `.obsp['distances']`, distances for each pair of neighbors
    `.obsp['connectivities']`, weighted adjacency matrix (0:00:00)
computing UMAP
    finished: added
    'X_umap', UMAP coordinates (adata.obsm) (0:00:03)
```

Example 40 (unknown):
```unknown
... storing 'phase' as categorical
```

Example 41 (unknown):
```unknown
<Figure size 800x800 with 0 Axes>
```

Example 42 (python):
```python
sc.pl.umap(adata, color='previous_fine_grain')
```

Example 43 (python):
```python
sc.pl.umap(adata, color='region')
```

Example 44 (python):
```python
sc.tl.leiden(adata, resolution=0.15, key_added='leiden')
sc.pl.umap(adata, color='leiden',legend_loc='on data')
```

Example 45 (python):
```python
running Leiden clustering
    finished: found 5 clusters and added
    'leiden', the cluster labels (adata.obs, categorical) (0:00:00)
```

Example 46 (python):
```python
Genes1=['IL1RAPL2','PHOX2B','TBX3','PCSK2','PRKG1']
Genes2=['NR2F2','RELN','GASK1B','ITIH5','LAMA2','MBP','COL14A1','MATN2','ITGB8','GJA1']
sc.pl.umap(adata, color=Genes1)
sc.pl.umap(adata, color=Genes2)
```

Example 47 (python):
```python
sc.settings.set_figure_params(dpi=80, color_map='viridis', vector_friendly=False,  dpi_save=300, facecolor='white')
ClusterOfInterest='1'
#sc.tl.rank_genes_groups(adata, 'leiden', method='t-test', key_added = "wilcoxon")
DEGenesForCluster = [x[ClusterOfInterest] for x in adata.uns['t-test']['names'][:100]]
sc.pl.umap(adata, color=DEGenesForCluster)
```

Example 48 (python):
```python
sc.pl.umap(adata, color='PLLP')
```

Example 49 (python):
```python
ClusterOfInterest='0'
#sc.tl.rank_genes_groups(adata, 'leiden', method='t-test', key_added = "wilcoxon")
DEGenesForCluster = [x[ClusterOfInterest] for x in adata.uns['t-test']['names'][:100]]
sc.pl.umap(adata, color=DEGenesForCluster)
```

Example 50 (python):
```python
sc.tl.rank_genes_groups(adata, 'leiden', groups=['0'], reference='1', method='wilcoxon')
sc.pl.rank_genes_groups(adata, groups=['0'], n_genes=20)
```

Example 51 (python):
```python
ranking genes
    finished: added to `.uns['rank_genes_groups']`
    'names', sorted np.recarray to be indexed by group ids
    'scores', sorted np.recarray to be indexed by group ids
    'logfoldchanges', sorted np.recarray to be indexed by group ids
    'pvals', sorted np.recarray to be indexed by group ids
    'pvals_adj', sorted np.recarray to be indexed by group ids (0:00:02)
```

Example 52 (python):
```python
#adata.write('./Objects/Neural.h5ad')
##adata=sc.read('./Objects/Neural.h5ad')
```

Example 53 (python):
```python
sc.tl.rank_genes_groups(adata, groupby='leiden', method='t-test')
sc.tl.dendrogram(adata, groupby='leiden')
#sc.pl.dendrogram(adata,groupby='leiden')
sc.pl.rank_genes_groups_matrixplot(adata, groupby='leiden', standard_scale='var')
```

Example 54 (python):
```python
ranking genes
    finished: added to `.uns['rank_genes_groups']`
    'names', sorted np.recarray to be indexed by group ids
    'scores', sorted np.recarray to be indexed by group ids
    'logfoldchanges', sorted np.recarray to be indexed by group ids
    'pvals', sorted np.recarray to be indexed by group ids
    'pvals_adj', sorted np.recarray to be indexed by group ids (0:00:00)
    using 'X_pca' with n_pcs = 50
Storing dendrogram info using `.uns['dendrogram_leiden']`
```

---

## Read in¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/ATAC/ATAC_preprocess/anndata_PeakMatrix_to_anndata.html

**Contents:**
- Read in¶
- Make anndata¶
- Add cell type annotation¶
- Refine metadata¶

ref: is peak matrix extracted with getMatrixFromProject normalized? NO https://github.com/GreenleafLab/ArchR/discussions/943

508040 rows × 0 columns

167022 rows × 0 columns

167022 rows × 0 columns

**Examples:**

Example 1 (python):
```python
import scanpy as sc
import pandas as pd
import numpy as np
import anndata

from scipy.io import mmwrite, mmread
from scipy.sparse import csr_matrix
```

Example 2 (unknown):
```unknown
!ls -l /nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/PeakMatrix
```

Example 3 (unknown):
```unknown
total 3673116
-rw-r--r-- 1 20383 1000624    1492397 Mar 12 20:40 barcodes.tsv.gz
-rw-r--r-- 1 20383 1000624    4082604 Mar 12 20:40 features.tsv.gz
-rw-r--r-- 1 20383 1000624 3755691976 Mar 12 20:40 matrix.mtx.gz
```

Example 4 (unknown):
```unknown
peakmatrix_dir = '/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/PeakMatrix'
```

Example 5 (unknown):
```unknown
# peak matrix, not-binarised
mat = mmread(f'{peakmatrix_dir}/matrix.mtx.gz')
mat
```

Example 6 (python):
```python
<508040x167022 sparse matrix of type '<class 'numpy.int64'>'
	with 1173033588 stored elements in COOrdinate format>
```

Example 7 (python):
```python
# peak names
features = pd.read_csv(f'{peakmatrix_dir}/features.tsv.gz', header=None, sep='\t')[[0]]
features.set_index(0,inplace=True)
features
```

Example 8 (python):
```python
# cell barcodes
cells = pd.read_csv(f'{peakmatrix_dir}/barcodes.tsv.gz', header=None, sep='\t')[[1]]
cells.set_index(1,inplace=True)
cells
```

Example 9 (python):
```python
adata = anndata.AnnData(
    X=mat.T,
    obs=cells,
    var=features
)

# rename obs_names
adata.obs_names = [x.replace('#','_') for x in adata.obs_names]

# convert sparce matrix
adata.X = adata.X.tocsr()
adata.X
```

Example 10 (python):
```python
<167022x508040 sparse matrix of type '<class 'numpy.float32'>'
	with 1173033588 stored elements in Compressed Sparse Row format>
```

Example 11 (unknown):
```unknown
AnnData object with n_obs × n_vars = 167022 × 508040
```

Example 12 (python):
```python
adata.X.data[:10]
```

Example 13 (unknown):
```unknown
array([2., 2., 2., 2., 1., 1., 2., 1., 2., 2.], dtype=float32)
```

Example 14 (python):
```python
np.max(adata.X.data)
```

Example 15 (python):
```python
obs = pd.read_csv('/nfs/team205/heart/anndata_objects/Foetal/Feb28ObjectRaw.obs.csv',index_col=0)

# add cell type labels and other metadata
metadata_to_add = ['sangerID', 'combinedID', 'donor', 
       'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 
       'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts',
       'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts',
       'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt',
       'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo',
       'pct_counts_ribo', 'HB_score', 'multiplexed',
        'batch_key','region','FACSgate','sex','week','trimester',
                     'fine_grain','mid_grain','coarse_grain',
                     'heart_or_greatvessels','cycling','S_score','G2M_score','phase','stress_score','hb1_score']
adata.obs[metadata_to_add] = obs[metadata_to_add].reindex(adata.obs_names).copy()
adata
```

Example 16 (unknown):
```unknown
AnnData object with n_obs × n_vars = 167022 × 508040
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score'
```

Example 17 (python):
```python
adata.var.rename_axis('Peaks', axis=0, inplace=True)
```

Example 18 (python):
```python
# save
adata.write(f'{peakmatrix_dir}/Foetal_Peaks.h5ad')
```

Example 19 (unknown):
```unknown
... storing 'sangerID' as categorical
... storing 'combinedID' as categorical
... storing 'donor' as categorical
... storing 'facility' as categorical
... storing 'cell_or_nuclei' as categorical
... storing 'modality' as categorical
... storing 'kit_10x' as categorical
... storing 'multiplexed' as categorical
... storing 'batch_key' as categorical
... storing 'region' as categorical
... storing 'FACSgate' as categorical
... storing 'sex' as categorical
... storing 'week' as categorical
... storing 'trimester' as categorical
... storing 'fine_grain' as categorical
... storing 'mid_grain' as categorical
... storing 'coarse_grain' as categorical
... storing 'heart_or_greatvessels' as categorical
... storing 'cycling' as categorical
... storing 'phase' as categorical
```

Example 20 (python):
```python
adata = sc.read_h5ad(f'{peakmatrix_dir}/Foetal_Peaks.h5ad')
adata
```

Example 21 (unknown):
```unknown
AnnData object with n_obs × n_vars = 167022 × 508040
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score'
```

Example 22 (python):
```python
# drop some obs columns
metadata_to_drop = [
       'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts',
       'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts',
       'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt',
       'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo',
       'pct_counts_ribo', 'HB_score', 'cycling','S_score','G2M_score','phase','stress_score','hb1_score']

adata.obs.drop(metadata_to_drop,axis=1,inplace=True)
```

Example 23 (python):
```python
# add peak metadata
meta = pd.read_csv('/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/PeakMatrix/peak_metadata.tsv',
           sep='\t')
meta.set_index('range',inplace=True)

# add to anndata
adata.var[meta.columns] = meta.reindex(adata.var_names)
adata.var.head()
```

Example 24 (python):
```python
# save
adata.write(f'{peakmatrix_dir}/Foetal_Peaks.h5ad')
```

Example 25 (unknown):
```unknown
... storing 'seqnames' as categorical
... storing 'strand' as categorical
... storing 'GroupReplicate' as categorical
... storing 'nearestGene' as categorical
... storing 'peakType' as categorical
... storing 'nearestTSS' as categorical
```

Example 26 (python):
```python
adata.X.data[:30]
```

Example 27 (unknown):
```unknown
array([2., 2., 2., 2., 1., 1., 2., 1., 2., 2., 2., 2., 3., 1., 2., 2., 3.,
       1., 2., 2., 2., 2., 2., 2., 1., 1., 2., 4., 2., 2.], dtype=float32)
```

Example 28 (unknown):
```unknown
f'{peakmatrix_dir}/Foetal_Peaks.h5ad'
```

Example 29 (unknown):
```unknown
'/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/PeakMatrix/Foetal_Peaks.h5ad'
```

Example 30 (unknown):
```unknown
# rclone to gdrive
rclone copy /nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/PeakMatrix/Foetal_Peaks.h5ad \
gdrive:JamesC_Kazumasa/Foetal_Heart/SanjayShinha/toSinhalab/anndata/Peak_foetal_global_raw.h5ad
```

Example 31 (python):
```python
adata.obs['cell_type3'].value_counts()
```

---

## Read in adata¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig5_Trisomy21/no11_deg_finegrain.html

**Contents:**
- Read in adata¶
- DEG test for each celltype, logFC_direction: euploid vs reference¶

42412 rows × 7 columns

**Examples:**

Example 1 (python):
```python
import scanpy as sc
import pandas as pd
import numpy as np
import anndata
import scipy
import matplotlib.pyplot as plt
import seaborn as sns

import warnings
warnings.simplefilter("ignore", UserWarning)
warnings.simplefilter(action='ignore', category=pd.errors.PerformanceWarning)
```

Example 2 (python):
```python
# Configure rpy2 to Use the Correct R Installation:
import os
os.environ['R_HOME'] = '/software/cellgen/team205/kk18/envs/generic_env/lib/R'
# check
from rpy2.rinterface_lib import openrlib
print(openrlib.R_HOME)
```

Example 3 (unknown):
```unknown
/software/cellgen/team205/kk18/envs/generic_env/lib/R
```

Example 4 (python):
```python
import rpy2.rinterface_lib.callbacks
import anndata2ri
import logging

from rpy2.robjects import pandas2ri
from rpy2.robjects import r

sc.settings.verbosity = 0
rpy2.rinterface_lib.callbacks.logger.setLevel(logging.ERROR)

pandas2ri.activate()
anndata2ri.activate()

%load_ext rpy2.ipython
```

Example 5 (unknown):
```unknown
/tmp/ipykernel_41476/3191876078.py:12: DeprecationWarning: The global conversion available with activate() is deprecated and will be removed in the next major release. Use a local converter.
  anndata2ri.activate()
```

Example 6 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 7 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 8 (unknown):
```unknown
%%R
library(edgeR)
library(tidyverse)
```

Example 9 (unknown):
```unknown
── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
✔ dplyr     1.1.4     ✔ readr     2.1.5
✔ forcats   1.0.0     ✔ stringr   1.5.1
✔ ggplot2   3.5.1     ✔ tibble    3.2.1
✔ lubridate 1.9.4     ✔ tidyr     1.3.1
✔ purrr     1.0.2     
── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter() masks stats::filter()
✖ dplyr::lag()    masks stats::lag()
ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors
```

Example 10 (unknown):
```unknown
Loading required package: limma
```

Example 11 (python):
```python
# pseudobulked per nhood
# nhood-by-gene
adata = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/trisomy21/Euploid_T21Hearts_Aug2024_sel_subsampled-per-cellstate-donor_cells-in-sig-nhoods-sfdr0p1_pseudobulked-nhood-reassigned-sum_finegrain.h5ad')
adata
```

Example 12 (unknown):
```unknown
AnnData object with n_obs × n_vars = 577 × 36601
    obs: 'index_cell', 'logFC', 'PValue', 'FDR', 'SpatialFDR', 'nhood_annotation', 'deg_group'
    var: 'gene_id', 'gene_name_scRNA-0-0', 'feature_type-1', 'mt-1', 'ribo-1', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1', 'gene_name_GenomicFeatures', 'gene_biotype_GenomicFeatures', 'gene_seq_start_GenomicFeatures', 'gene_seq_end_GenomicFeatures', 'seq_name_GenomicFeatures', 'seq_strand_GenomicFeatures', 'seq_coord_system_GenomicFeatures', 'description_GenomicFeatures', 'gene_id_version_GenomicFeatures', 'canonical_transcript_GenomicFeatures', 'gene_name'
```

Example 13 (python):
```python
celltype_list = [x for x in adata.obs['nhood_annotation'].unique() if x!='Mixed']
celltype_list
```

Example 14 (unknown):
```unknown
['AtrialCardiomyocytesRight',
 'AtrialCardiomyocytesCycling',
 'VentricularConductionSystemProximal',
 'AtrialCardiomyocytesLeft',
 'AtrioventricularNodePacemakerCells',
 'CoronaryCapillaryEndothelialCells',
 'CoronaryPericytes',
 'CoronaryVesselAdventitialFibroblasts',
 'DuctusArteriosusSmoothMuscleCells',
 'EndocardialCells',
 'EndocardialCushionCells',
 'SubEpicardialFibroblasts',
 'GreatVesselAdventitialFibroblasts',
 'GreatVesselArterialEndothelialCells',
 'GreatVesselSmoothMuscleCells',
 'MacrophagesATF3pos',
 'MacrophagesLYVE1pos',
 'Megakaryocytes',
 'MesothelialEpicardialCells',
 'MyocardialInterstitialFibroblasts',
 'ValveInterstitialCells',
 'Myofibroblasts',
 'ParasympatheticNeurons',
 'VentricularCardiomyocytesCycling',
 'VentricularCardiomyocytesRightTrabeculated',
 'VentricularCardiomyocytesLeftCompact',
 'VentricularCardiomyocytesLeftTrabeculated',
 'VentricularCardiomyocytesRightCompact']
```

Example 15 (python):
```python
# select celltypes which has n_sample>=3
ctab = pd.crosstab(adata.obs['nhood_annotation'],adata.obs['deg_group'])[['euploid','trisomy21']]
celltypes_sel = list(ctab.index[(ctab>=3).sum(axis=1)==2])
celltypes_sel = [x for x in celltypes_sel if x!='Mixed']
adata_sub = adata[adata.obs['nhood_annotation'].isin(celltypes_sel)]
# subset groups
adata_sub = adata_sub[adata_sub.obs['deg_group'].isin(['euploid','trisomy21'])]
pd.crosstab(adata_sub.obs['nhood_annotation'],adata_sub.obs['deg_group'])
```

Example 16 (python):
```python
%%R -i adata_sub
class(adata_sub)
```

Example 17 (unknown):
```unknown
[1] "SingleCellExperiment"
attr(,"package")
[1] "SingleCellExperiment"
```

Example 18 (python):
```python
%%R
# https://github.com/compbiomed/singleCellTK/blob/40527afd2e58e14451aae655c0d8fae4a7a0f54b/R/subsetSCE.R#L140
subsetSCECols <- function(inSCE, index = NULL, bool = NULL, colData = NULL) {

  if(is.null(index) & is.null(bool) & is.null(colData)) {
    stop("At least one of 'index', 'bool', or 'colData' must be supplied.")
  }
  final.ix <- rep(TRUE, ncol(inSCE))

  # Parse index containing integers
  if(!is.null(index)) {
    if(min(index) < 1 | max(index) > ncol(inSCE)) {
      stop("'index' must contain integers between 1 and the number of columns ",
           "in 'inSCE': ", ncol(inSCE))
    }
    final.ix[!seq(ncol(inSCE)) %in% index] <- FALSE
  }

  # Parse Boolean vector
  if(!is.null(bool)) {
    if(length(bool) != ncol(inSCE) | !is.logical(bool)) {
      stop("'bool' must be a logical vector the same length as the number of ",
           "colmns in 'inSCE': ", ncol(inSCE))
    }
    final.ix[!bool] <- FALSE
  }

  # Parse expressions for colData variables
  if(!is.null(colData)) {
    for(i in seq_along(colData)) {
      temp <- eval(parse(text = as.character(colData[i])),
                   envir = as.data.frame(colData(inSCE)))
      if(length(temp) != ncol(inSCE) | !is.logical(temp)) {
        stop("The expression ", colData[i], " did not produce a boolean ",
             "vector the same length as the number of columns in 'inSCE'. ",
             "Please ensure that the spelling of the variable you are ",
             "trying to use matches one of the column names in ",
             "'colData(inSCE)' and that your expression is valid.")
      }
      final.ix[!temp] <- FALSE
    }
  }

  inSCE <- inSCE[,final.ix]
  return(inSCE)
  }

# https://www.sc-best-practices.org/conditions/differential_gene_expression.html
fit_model <- function(adata_){
    # create an edgeR object with counts and grouping factor
    y <- DGEList(assay(adata_, "X"), group = colData(adata_)$deg_group)
    # filter out genes with low counts
    print("Dimensions before subsetting:")
    print(dim(y))
    print("")
    keep <- filterByExpr(y)
    y <- y[keep, , keep.lib.sizes=FALSE]
    print("Dimensions after subsetting:")
    print(dim(y))
    print("")
    # normalize
    y <- calcNormFactors(y)
    # create a vector that is a concatenation of condition and cell type that we will later use with contrasts
    group <- colData(adata_)$deg_group
    # sex <- colData(adata_)$sex
    # replicate <- colData(adata_)$donor
    # create a design matrix: here we have multiple donors so also consider that in the design matrix
    design <- model.matrix(~ 0 + group)
    # estimate dispersion
    y <- estimateDisp(y, design = design)
    # fit the model
    fit <- glmQLFit(y, design)
    return(list("fit"=fit, "design"=design, "y"=y))
}
```

Example 19 (python):
```python
%%time
%%R -o tt_all 
celltype_list <- unique(colData(adata_sub)$nhood_annotation)
for(celltype in celltype_list){
    print(celltype)
    # subset
    sce <- subsetSCECols(adata_sub, colData = "nhood_annotation == celltype")
    # fit model
    outs <-fit_model(sce)
    # get output
    fit <- outs$fit
    y <- outs$y
    # deg
    myContrast <- makeContrasts('grouptrisomy21-groupeuploid', levels = y$design)
    qlf <- glmQLFTest(fit, contrast=myContrast)
    # get all of the DE genes and calculate Benjamini-Hochberg adjusted FDR
    tt <- topTags(qlf, n = Inf)
    tt <- tt$table
    tt <- rownames_to_column(tt, var = "gene_name")
    tt$celltype <- celltype
    # rbind
    if(celltype==celltype_list[1]){
        tt_all = tt
    }else{
        tt_all = rbind(tt_all,tt)
    }
    print('')
}
```

Example 20 (unknown):
```unknown
[1] "AtrialCardiomyocytesRight"
[1] "Dimensions before subsetting:"
[1] 36601    70
[1] ""
[1] "Dimensions after subsetting:"
[1] 3761   70
[1] ""
[1] ""
[1] "AtrialCardiomyocytesLeft"
[1] "Dimensions before subsetting:"
[1] 36601    36
[1] ""
[1] "Dimensions after subsetting:"
[1] 3740   36
[1] ""
[1] ""
[1] "CoronaryCapillaryEndothelialCells"
[1] "Dimensions before subsetting:"
[1] 36601    11
[1] ""
[1] "Dimensions after subsetting:"
[1] 2684   11
[1] ""
[1] ""
[1] "EndocardialCells"
[1] "Dimensions before subsetting:"
[1] 36601    12
[1] ""
[1] "Dimensions after subsetting:"
[1] 5171   12
[1] ""
[1] ""
[1] "SubEpicardialFibroblasts"
[1] "Dimensions before subsetting:"
[1] 36601    15
[1] ""
[1] "Dimensions after subsetting:"
[1] 2628   15
[1] ""
[1] ""
[1] "GreatVesselAdventitialFibroblasts"
[1] "Dimensions before subsetting:"
[1] 36601    16
[1] ""
[1] "Dimensions after subsetting:"
[1] 3758   16
[1] ""
[1] ""
[1] "GreatVesselSmoothMuscleCells"
[1] "Dimensions before subsetting:"
[1] 36601    22
[1] ""
[1] "Dimensions after subsetting:"
[1] 2607   22
[1] ""
[1] ""
[1] "MacrophagesATF3pos"
[1] "Dimensions before subsetting:"
[1] 36601    12
[1] ""
[1] "Dimensions after subsetting:"
[1] 1570   12
[1] ""
[1] ""
[1] "MacrophagesLYVE1pos"
[1] "Dimensions before subsetting:"
[1] 36601    44
[1] ""
[1] "Dimensions after subsetting:"
[1] 2509   44
[1] ""
[1] ""
[1] "MyocardialInterstitialFibroblasts"
[1] "Dimensions before subsetting:"
[1] 36601    43
[1] ""
[1] "Dimensions after subsetting:"
[1] 2267   43
[1] ""
[1] ""
[1] "VentricularCardiomyocytesCycling"
[1] "Dimensions before subsetting:"
[1] 36601    44
[1] ""
[1] "Dimensions after subsetting:"
[1] 3294   44
[1] ""
[1] ""
[1] "VentricularCardiomyocytesLeftCompact"
[1] "Dimensions before subsetting:"
[1] 36601    46
[1] ""
[1] "Dimensions after subsetting:"
[1] 2269   46
[1] ""
[1] ""
[1] "VentricularCardiomyocytesLeftTrabeculated"
[1] "Dimensions before subsetting:"
[1] 36601     8
[1] ""
[1] "Dimensions after subsetting:"
[1] 2930    8
[1] ""
[1] ""
[1] "VentricularCardiomyocytesRightCompact"
[1] "Dimensions before subsetting:"
[1] 36601    20
[1] ""
[1] "Dimensions after subsetting:"
[1] 3224   20
[1] ""
[1] ""
CPU times: user 16.5 s, sys: 244 ms, total: 16.7 s
Wall time: 17.9 s
```

Example 21 (unknown):
```unknown
# save
tt_all.to_csv('./milo_nhood-reassigned_pseudobulk_edgeR_res_tri21-nhoods_vs_euploid-nhoods_finegrain.csv')
```

---

## Make custom annotation for cell2location¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig4_Cardiomyocytes/AtrialCardiomyocytes/Visium_mapping/make_aCMimmature_annotation_for_c2l.html

**Contents:**
- Make custom annotation for cell2location¶

A custom annotation which is the same as fine_grain except for the addition of "AtrialCardiomycoytesImmature"

The idea is to inspect whhether this signature is seen in the eary visium samples.

5664 rows × 1 columns

**Examples:**

Example 1 (python):
```python
import pandas as pd
import os
import scanpy as sc
```

Example 2 (python):
```python
# read in custom annotation (only applied to aCMs of course)
annotation_df=pd.read_csv("AtrialCardiomyocytesImmature_annotation.csv",index_col=0)
annotation_df
```

Example 3 (python):
```python
# read in the main object
adata=sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/Feb28ObjectRaw.h5ad')
adata
```

Example 4 (unknown):
```unknown
AnnData object with n_obs × n_vars = 297473 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score'
    var: 'gene_name_scRNA-0', 'gene_id'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'age_colors', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'cycling_colors', 'donor_colors', 'fine_grain_colors', 'heart_or_greatvessels_colors', 'hvg', 'kit_10x_colors', 'mid_grain_colors', 'modality_colors', 'neighbors', 'phase_colors', 'region_colors', 'sex_colors', 'trimester_colors', 'umap', 'week_colors'
    obsm: 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 5 (python):
```python
sc.pl.umap(adata,color="fine_grain")
```

Example 6 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 7 (python):
```python
# Add updated label - we have 2 young and 2 old populations

immature_barcodes = annotation_df[annotation_df['AtrialCardiomyocytesImmature_annotation']=="AtrialCardiomyocytesImmature"].index

# Create a dictionary to hold the updates
updates = {}

for barcode in immature_barcodes:
    if barcode in adata.obs['fine_grain'].index:
        fine_grain_annotation = adata.obs['fine_grain'][barcode]
        if fine_grain_annotation == 'AtrialCardiomyocytesLeft':
            updates[barcode] = 'AtrialCardiomyocytesLeft_immature'
        elif fine_grain_annotation == 'AtrialCardiomyocytesRight':
            updates[barcode] = 'AtrialCardiomyocytesRight_immature'

annotation_df['AtrialCardiomyocytesImmature_annotation_v2'] = annotation_df['AtrialCardiomyocytesImmature_annotation']

# Apply the updates from the dictionary
for barcode, new_annotation in updates.items():
    annotation_df.loc[barcode, 'AtrialCardiomyocytesImmature_annotation_v2'] = new_annotation
```

Example 8 (unknown):
```unknown
annotation_df[['AtrialCardiomyocytesImmature_annotation','AtrialCardiomyocytesImmature_annotation_v2']].value_counts()
```

Example 9 (unknown):
```unknown
AtrialCardiomyocytesImmature_annotation  AtrialCardiomyocytesImmature_annotation_v2
AtrialCardiomyocytesLeft                 AtrialCardiomyocytesLeft                      2828
AtrialCardiomyocytesRight                AtrialCardiomyocytesRight                     2509
AtrialCardiomyocytesImmature             AtrialCardiomyocytesLeft_immature              213
                                         AtrialCardiomyocytesRight_immature             114
dtype: int64
```

Example 10 (unknown):
```unknown
annotation_df['AtrialCardiomyocytesImmature_annotation']
```

Example 11 (unknown):
```unknown
barcode
BHF_F_Hea11933667_BHF_F_Hea11596620_CTTCTCAAGCGGTTAT-1       AtrialCardiomyocytesRight
BHF_F_Hea11933674_BHF_F_Hea11596627_GATTAGCTCAGCACGC-1        AtrialCardiomyocytesLeft
BHF_F_Hea11933674_BHF_F_Hea11596627_CATGCAAGTTAGGTGC-1        AtrialCardiomyocytesLeft
BHF_F_Hea11933668_BHF_F_Hea11596621_CCTTACTCAACACCTA-1       AtrialCardiomyocytesRight
BHF_F_Hea11933666_BHF_F_Hea11596619_AGCTTCCTCGTTATAG-1        AtrialCardiomyocytesLeft
                                                                      ...             
BHF_F_Hea11064672_BHF_F_Hea11031825_GCACTTACAGCCAGAA-1        AtrialCardiomyocytesLeft
BHF_F_Hea11933669_BHF_F_Hea11596622_GCTAACAGTTAGAGGG-1       AtrialCardiomyocytesRight
BHF_F_Hea11933673_BHF_F_Hea11596626_GCCTATTGTGTGTCCC-1    AtrialCardiomyocytesImmature
BHF_F_Hea11064671_BHF_F_Hea11031824_CGTCAATAGCTAAGTC-1        AtrialCardiomyocytesLeft
BHF_F_Hea11064671_BHF_F_Hea11031824_CTTACTAGTTGTTGCT-1        AtrialCardiomyocytesLeft
Name: AtrialCardiomyocytesImmature_annotation, Length: 5664, dtype: object
```

Example 12 (python):
```python
# Step 1: Initialize 'fine_grain_modified' with values from 'fine_grain' and convert to string
adata.obs['fine_grain_modified'] = adata.obs['fine_grain'].astype(str)

# Step 2: Update 'fine_grain_modified' for barcodes present in 'annotation_df'
for barcode in annotation_df.index:
    if barcode in adata.obs.index:
        new_value = annotation_df.at[barcode, 'AtrialCardiomyocytesImmature_annotation_v2']
        adata.obs.at[barcode, 'fine_grain_modified'] = new_value

# Optional: Convert 'fine_grain_modified' back to categorical
adata.obs['fine_grain_modified'] = adata.obs['fine_grain_modified'].astype('category')

sc.pl.umap(adata,color="fine_grain_modified")
```

Example 13 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 14 (python):
```python
adata.obs.fine_grain_modified.to_csv("fine_grain_modified.csv")
```

Example 15 (python):
```python
bdata = adata[adata.obs]
```

---

## Loading packages¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig4_Cardiomyocytes/AtrialCardiomyocytes/Visium_mapping/c2l_pt1_reference.html

**Contents:**
- Loading packages¶
- Functions¶
- Loading scRNA-seq reference data¶
- Make sure .var_names are ENSEMBL IDs¶
- Make reference signatures¶

297473 rows × 1 columns

**Examples:**

Example 1 (unknown):
```unknown
run = "20Mar2024_fetal_OCT"
```

Example 2 (unknown):
```unknown
celltype_key = 'fine_grain_modified'
```

Example 3 (python):
```python
# create paths and names to results folders for reference regression and cell2location models
import os
results_folder = os.path.join(os.getcwd(),"analysis/Visium/cell2location_results")
ref_run_name = f'{results_folder}/reference_signatures/{run}'
run_name = f'{results_folder}/cell2location_map/{run}'

os.makedirs(ref_run_name, exist_ok=True)
os.makedirs(run_name, exist_ok=True)
```

Example 4 (python):
```python
import sys
import scanpy as sc
import anndata
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt 
import matplotlib as mpl

import cell2location
import scvi

from matplotlib import rcParams
rcParams['pdf.fonttype'] = 42 # enables correct plotting of text
import seaborn as sns
```

Example 5 (python):
```python
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm
Global seed set to 0
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/umap/distances.py:1063: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/umap/distances.py:1071: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/umap/distances.py:1086: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/umap/umap_.py:660: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
```

Example 6 (python):
```python
import warnings
# Suppress UserWarnings
warnings.filterwarnings('ignore', category=UserWarning)
```

Example 7 (python):
```python
import session_info
session_info.show()
```

Example 8 (python):
```python
-----
anndata             0.8.0
cell2location       0.1.3
matplotlib          3.7.2
numpy               1.24.4
pandas              1.5.3
scanpy              1.9.3
scvi                0.20.3
seaborn             0.12.2
session_info        1.0.0
-----
```

Example 9 (unknown):
```unknown
PIL                         10.0.0
absl                        NA
anyio                       NA
arrow                       1.2.3
asttokens                   NA
attr                        23.1.0
attrs                       23.1.0
babel                       2.12.1
backcall                    0.2.0
brotli                      1.0.9
certifi                     2023.07.22
cffi                        1.15.1
charset_normalizer          3.2.0
chex                        0.1.7
comm                        0.1.3
contextlib2                 NA
cycler                      0.10.0
cython_runtime              NA
dateutil                    2.8.2
debugpy                     1.6.7
decorator                   5.1.1
defusedxml                  0.7.1
docrep                      0.3.2
etils                       1.4.0
executing                   1.2.0
fastjsonschema              NA
flax                        0.7.0
fqdn                        NA
fsspec                      2023.6.0
graphviz                    0.20.1
h5py                        3.9.0
idna                        3.4
igraph                      0.10.6
importlib_resources         NA
ipykernel                   6.25.0
isoduration                 NA
jax                         0.4.14
jaxlib                      0.4.14
jedi                        0.18.2
jinja2                      3.1.2
joblib                      1.3.1
json5                       NA
jsonpointer                 2.0
jsonschema                  4.18.4
jsonschema_specifications   NA
jupyter_events              0.6.3
jupyter_server              2.7.0
jupyterlab_server           2.24.0
kiwisolver                  1.4.4
leidenalg                   0.10.1
lightning_fabric            1.9.5
lightning_utilities         0.9.0
llvmlite                    0.40.1
markupsafe                  2.1.3
ml_collections              NA
ml_dtypes                   0.2.0
mpl_toolkits                NA
mpmath                      1.3.0
msgpack                     1.0.5
mudata                      0.2.3
multipledispatch            0.6.0
natsort                     8.4.0
nbformat                    5.9.1
numba                       0.57.1
numpyro                     0.12.1
nvfuser                     NA
opt_einsum                  v3.3.0
optax                       0.1.7
overrides                   NA
packaging                   23.1
parso                       0.8.3
patsy                       0.5.3
pexpect                     4.8.0
pickleshare                 0.7.5
pkg_resources               NA
platformdirs                3.9.1
plotly                      5.16.1
prometheus_client           NA
prompt_toolkit              3.0.39
psutil                      5.9.5
ptyprocess                  0.7.0
pure_eval                   0.2.2
pyarrow                     12.0.1
pycparser                   2.21
pydev_ipython               NA
pydevconsole                NA
pydevd                      2.9.5
pydevd_file_utils           NA
pydevd_plugins              NA
pydevd_tracing              NA
pygments                    2.15.1
pynndescent                 0.5.10
pyparsing                   3.0.9
pyro                        1.8.6
pythonjsonlogger            NA
pytorch_lightning           1.9.5
pytz                        2023.3
referencing                 NA
regex                       2.5.129
requests                    2.31.0
rfc3339_validator           0.1.4
rfc3986_validator           0.1.1
rich                        NA
rpds                        NA
scipy                       1.11.1
send2trash                  NA
setuptools                  68.0.0
six                         1.16.0
sklearn                     1.3.0
sniffio                     1.3.0
socks                       1.7.1
sparse                      0.14.0
stack_data                  0.6.2
statsmodels                 0.14.0
sympy                       1.12
texttable                   1.6.7
threadpoolctl               3.2.0
toolz                       0.12.0
torch                       2.0.1+cu117
torchmetrics                1.0.1
tornado                     6.3.2
tqdm                        4.65.0
traitlets                   5.9.0
tree                        0.1.8
typing_extensions           NA
umap                        0.5.3
uri_template                NA
urllib3                     1.26.16
wcwidth                     0.2.6
webcolors                   1.13
websocket                   1.6.1
yaml                        6.0
zmq                         25.1.0
```

Example 10 (unknown):
```unknown
-----
IPython             8.14.0
jupyter_client      8.3.0
jupyter_core        5.3.1
jupyterlab          4.0.3
notebook            7.0.0
-----
Python 3.11.4 | packaged by conda-forge | (main, Jun 10 2023, 18:08:17) [GCC 12.2.0]
Linux-5.15.0-88-generic-x86_64-with-glibc2.35
-----
Session information updated at 2023-11-21 11:02
```

Example 11 (python):
```python
# Importing necessary libraries
from cell2location.utils.filtering import filter_genes
from cell2location.models import RegressionModel

# Function to make c2l refernce
def c2l_make_reference(adata, subset_value,max_epochs=250):
    print(f'##### {subset_value} #####')
    print(adata.obs[celltype_key].value_counts())

    print(f'filtering features...')
    selected = filter_genes(adata, cell_count_cutoff=5, cell_percentage_cutoff2=0.03, nonz_mean_cutoff=1.12)
    adata = adata[:, selected].copy()

    print(f'removing cell types which occur very seldom...')
    counts = adata.obs[celltype_key].value_counts()
    adata = adata[adata.obs[celltype_key].isin(list(counts.keys()[counts > 2]))]

    print(f'setting up model...')
    RegressionModel.setup_anndata(adata=adata, batch_key='batch_key', labels_key=celltype_key)
    mod = RegressionModel(adata)
    mod.view_anndata_setup()
    
    print(f'training model for {max_epochs} epochs...')
    mod.train(max_epochs=max_epochs, batch_size=2500, train_size=1, lr=0.002, use_gpu=True)
    
    print(f'view history...')
    mod.plot_history(20)
    
    print(f'extract posteriors...')
    adata = mod.export_posterior(adata, sample_kwargs={'num_samples': 1000, 'batch_size': 2500, 'use_gpu': True})
    print(f'saving model to {ref_run_name}/{subset_value}...')
    mod.save(f"{ref_run_name}/{subset_value}/", overwrite=True)
    print(f'saving reference to {ref_run_name}/{subset_value}...')
    adata.write(f"{ref_run_name}/{subset_value}/sc.h5ad")
    print(f'plotting QC...')
    mod.plot_QC()
    
    if 'means_per_cluster_mu_fg' in adata.varm.keys():
        inf_aver = adata.varm['means_per_cluster_mu_fg'][[f'means_per_cluster_mu_fg_{i}' for i in adata.uns['mod']['factor_names']]].copy()
    else:
        inf_aver = adata.var[[f'means_per_cluster_mu_fg_{i}' for i in adata.uns['mod']['factor_names']]].copy()
    inf_aver.columns = adata.uns['mod']['factor_names']
    inf_aver.to_csv(f"{ref_run_name}/{subset_value}/inf_aver.csv")
    
    del adata, mod, inf_aver, selected, counts
```

Example 12 (python):
```python
# Read scRNA data
scRNA_path='/nfs/team205/heart/anndata_objects/Foetal/Feb28ObjectRaw.h5ad'
adata = sc.read(scRNA_path)
adata
```

Example 13 (unknown):
```unknown
AnnData object with n_obs × n_vars = 297473 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score'
    var: 'gene_name_scRNA-0', 'gene_id'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'age_colors', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'cycling_colors', 'donor_colors', 'fine_grain_colors', 'heart_or_greatvessels_colors', 'hvg', 'kit_10x_colors', 'mid_grain_colors', 'modality_colors', 'neighbors', 'phase_colors', 'region_colors', 'sex_colors', 'trimester_colors', 'umap', 'week_colors'
    obsm: 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 14 (python):
```python
adata.X.data[:10]
```

Example 15 (unknown):
```unknown
array([ 1.,  1.,  5.,  5.,  8.,  4., 13.,  4.,  3.,  1.], dtype=float32)
```

Example 16 (python):
```python
adata.layers["counts"]=adata.X.copy() # counts to .X
```

Example 17 (python):
```python
adata.X.data[:10]
```

Example 18 (unknown):
```unknown
array([ 1.,  1.,  5.,  5.,  8.,  4., 13.,  4.,  3.,  1.], dtype=float32)
```

Example 19 (python):
```python
# Import custom annotation
fine_grain_modified=pd.read_csv("fine_grain_modified.csv", index_col=0)
fine_grain_modified
```

Example 20 (python):
```python
adata.obs=adata.obs.merge(fine_grain_modified,left_index=True, right_index=True)
adata.obs.head(3)
```

Example 21 (python):
```python
set(adata.obs[celltype_key])
```

Example 22 (unknown):
```unknown
{'AtrialCardiomyocytesCycling',
 'AtrialCardiomyocytesImmature',
 'AtrialCardiomyocytesLeft',
 'AtrialCardiomyocytesRight',
 'AtrioventricularNodeCardiomyocytes',
 'BCells',
 'BCellsMS4A1pos',
 'ChromaffinCells',
 'CoronaryArterialEndothelialCells',
 'CoronaryCapillaryEndothelialCells',
 'CoronaryPericytes',
 'CoronarySmoothMuscleCells',
 'CoronaryVenousEndothelialCells',
 'CoronaryVesselAdventitialFibroblasts',
 'DendriticCellsMature',
 'DendriticCellsType1',
 'DuctusArteriosusSmoothMuscleCells',
 'EndocardialCells',
 'EndocardialCushionCells',
 'EpicardiumDerivedCells',
 'GreatVesselAdventitialFibroblasts',
 'GreatVesselArterialEndothelialCells',
 'GreatVesselSmoothMuscleCells',
 'GreatVesselVenousEndothelialCells',
 'InnateLymphoidCells',
 'LymphNodeFibroblasticReticularCells',
 'LymphaticEndothelialCells',
 'MacrophagesATF3pos',
 'MacrophagesCX3CR1pos',
 'MacrophagesLYVE1pos',
 'MacrophagesTIMD4pos',
 'MastCells',
 'Megakaryocytes',
 'MesothelialEpicardialCells',
 'MonocyteDerivedCells',
 'Monocytes',
 'MonocytesMPOpos',
 'MyocardialInterstitialFibroblasts1',
 'MyocardialInterstitialFibroblasts2',
 'Myofibroblasts',
 'NaturalKillerCells',
 'NeuronPrecursors',
 'ParaympatheticNeurons',
 'PericardialCells1',
 'PericardialCells2',
 'PericardialCells3',
 'PlasmacytoidDendriticCells',
 'ProBCells',
 'SchwannCellPrecursors',
 'SchwannCells',
 'SinoatrialNodeCardiomyocytes',
 'SympatheticNeurons',
 'TCellsCD4pos',
 'TCellsCD8pos',
 'TregsCD4pos',
 'ValveEndothelialCells',
 'ValveInterstitialCells',
 'VentricularCardiomyocytesCycling',
 'VentricularCardiomyocytesLeftCompact',
 'VentricularCardiomyocytesLeftTrabeculated',
 'VentricularCardiomyocytesRightCompact',
 'VentricularCardiomyocytesRightTrabeculated',
 'VentricularConductionSystemDistal',
 'VentricularConductionSystemProximal'}
```

Example 23 (python):
```python
adata.var.head(3)
```

Example 24 (python):
```python
adata.var=adata.var.set_index("gene_id")
```

Example 25 (python):
```python
adata.var.head(3)
```

Example 26 (python):
```python
adata.obs['batch_key'].value_counts()
```

Example 27 (unknown):
```unknown
Hst41_Nuclei_Multiome-v1    104878
Hst33_Nuclei_Multiome-v1     38234
Hst39_Nuclei_Multiome-v1     25386
C99_Cell_5prime-v2           17547
C92_Nuclei_Multiome-v1       13770
C94_Cell_5prime-v2           12985
Hst40_Nuclei_Multiome-v1      9708
C97_Cell_5prime-v2            9702
BRC2251_Cell_3prime-v3        9121
BRC2256_Cell_3prime-v3        7948
Hst36_Nuclei_Multiome-v1      7268
BRC2252_Cell_3prime-v3        6854
C86_Cell_5prime-v2            6376
BRC2260_Cell_3prime-v3        6004
BRC2263_Cell_3prime-v3        5662
BRC2262_Cell_3prime-v3        4129
C87_Nuclei_Multiome-v1        3907
C98_Nuclei_Multiome-v1        2660
C104_Nuclei_Multiome-v1       2490
C85_Nuclei_Multiome-v1        2436
C82_Nuclei_Multiome-v1         408
Name: batch_key, dtype: int64
```

Example 28 (python):
```python
set(adata.obs['batch_key'])
```

Example 29 (unknown):
```unknown
{'BRC2251_Cell_3prime-v3',
 'BRC2252_Cell_3prime-v3',
 'BRC2256_Cell_3prime-v3',
 'BRC2260_Cell_3prime-v3',
 'BRC2262_Cell_3prime-v3',
 'BRC2263_Cell_3prime-v3',
 'C104_Nuclei_Multiome-v1',
 'C82_Nuclei_Multiome-v1',
 'C85_Nuclei_Multiome-v1',
 'C86_Cell_5prime-v2',
 'C87_Nuclei_Multiome-v1',
 'C92_Nuclei_Multiome-v1',
 'C94_Cell_5prime-v2',
 'C97_Cell_5prime-v2',
 'C98_Nuclei_Multiome-v1',
 'C99_Cell_5prime-v2',
 'Hst33_Nuclei_Multiome-v1',
 'Hst36_Nuclei_Multiome-v1',
 'Hst39_Nuclei_Multiome-v1',
 'Hst40_Nuclei_Multiome-v1',
 'Hst41_Nuclei_Multiome-v1'}
```

Example 30 (unknown):
```unknown
subset=False
```

Example 31 (python):
```python
%%time
if subset:
    subset_values=adata.obs[sc_subset_key].unique().tolist()
    for value in subset_values:
        adata_ref_subset = adata[adata.obs[sc_subset_key].isin([value])].copy()
        c2l_make_reference(adata_ref_subset, value)
else:
    adata_ref = adata.copy()
    c2l_make_reference(adata_ref, "all_data")
```

Example 32 (unknown):
```unknown
##### LA #####
AtrialCardiomyocytesLeft                  19443
MP_LYVE1+                                  6955
Pericyte_1                                 5845
EC_capillary                               5764
SAN_FB                                     4311
                                          ...  
valve_FB_MV2                                  5
VentricularCardiomyocytesLeftStressed         2
valve_FB_CS                                   1
VentricularCardiomyocytesRight                1
VentricularCardiomyocytesRightStressed        1
Name: cell_state, Length: 70, dtype: int64
filtering features...
```

Example 33 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/pandas/core/arraylike.py:402: RuntimeWarning: divide by zero encountered in log10
  result = getattr(ufunc, method)(*inputs, **kwargs)
```

Example 34 (unknown):
```unknown
removing cell types which occur very seldom...
setting up model...
```

Example 35 (unknown):
```unknown
No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.)
```

Example 36 (unknown):
```unknown
Anndata setup with scvi-tools version 0.20.3.
```

Example 37 (unknown):
```unknown
Setup via `RegressionModel.setup_anndata` with arguments:
```

Example 38 (unknown):
```unknown
{
│   'layer': None,
│   'batch_key': 'batch_key',
│   'labels_key': 'cell_state',
│   'categorical_covariate_keys': None,
│   'continuous_covariate_keys': None
}
```

Example 39 (unknown):
```unknown
Summary Statistics         
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┓
┃     Summary Stat Key     ┃ Value ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━┩
│         n_batch          │  30   │
│         n_cells          │ 89744 │
│ n_extra_categorical_covs │   0   │
│ n_extra_continuous_covs  │   0   │
│         n_labels         │  66   │
│          n_vars          │ 13571 │
└──────────────────────────┴───────┘
```

Example 40 (python):
```python
Data Registry                
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Registry Key ┃    scvi-tools Location    ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│      X       │          adata.X          │
│    batch     │ adata.obs['_scvi_batch']  │
│    ind_x     │   adata.obs['_indices']   │
│    labels    │ adata.obs['_scvi_labels'] │
└──────────────┴───────────────────────────┘
```

Example 41 (python):
```python
batch State Registry                           
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location     ┃       Categories        ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['batch_key'] │ A61_Nuclei_Multiome-v1  │          0          │
│                        │ AH5_Nuclei_Multiome-v1  │          1          │
│                        │ AV1_Nuclei_Multiome-v1  │          2          │
│                        │ AV2_Nuclei_Multiome-v1  │          3          │
│                        │ AV10_Nuclei_Multiome-v1 │          4          │
│                        │ AV11_Nuclei_Multiome-v1 │          5          │
│                        │ AV13_Nuclei_Multiome-v1 │          6          │
│                        │ AV14_Nuclei_Multiome-v1 │          7          │
│                        │   D1_Nuclei_3prime-v2   │          8          │
│                        │   D2_Nuclei_3prime-v2   │          9          │
│                        │    D3_Cell_3prime-v2    │         10          │
│                        │   D3_Nuclei_3prime-v2   │         11          │
│                        │    D4_Cell_3prime-v2    │         12          │
│                        │   D4_Nuclei_3prime-v2   │         13          │
│                        │    D5_Cell_3prime-v2    │         14          │
│                        │   D5_Nuclei_3prime-v2   │         15          │
│                        │    D6_Cell_3prime-v2    │         16          │
│                        │    D6_Cell_3prime-v3    │         17          │
│                        │   D6_Nuclei_3prime-v2   │         18          │
│                        │    D7_Cell_3prime-v2    │         19          │
│                        │   D7_Nuclei_3prime-v2   │         20          │
│                        │  D8_Nuclei_Multiome-v1  │         21          │
│                        │   D11_Cell_3prime-v3    │         22          │
│                        │  D11_Nuclei_3prime-v3   │         23          │
│                        │   H2_Nuclei_3prime-v3   │         24          │
│                        │   H3_Nuclei_3prime-v3   │         25          │
│                        │   H4_Nuclei_3prime-v3   │         26          │
│                        │   H5_Nuclei_3prime-v3   │         27          │
│                        │   H6_Nuclei_3prime-v3   │         28          │
│                        │   H7_Nuclei_3prime-v3   │         29          │
└────────────────────────┴─────────────────────────┴─────────────────────┘
```

Example 42 (python):
```python
labels State Registry                                
┏━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃     Source Location     ┃            Categories             ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['cell_state'] │            Adipocytes             │          0          │
│                         │       AppendageNeuralCells        │          1          │
│                         │       Artifact mesothelial        │          2          │
│                         │          Atrial pericyte          │          3          │
│                         │     AtrialCardiomyocytesLeft      │          4          │
│                         │     AtrialCardiomyocytesRight     │          5          │
│                         │      AtrialConductionSystem       │          6          │
│                         │                 B                 │          7          │
│                         │             B_plasma              │          8          │
│                         │               CD4+T               │          9          │
│                         │            CD4+T_naive            │         10          │
│                         │             CD4+T_reg             │         11          │
│                         │               CD8+T               │         12          │
│                         │     CD55+ FGFR1+ mesothelial      │         13          │
│                         │        CRIM1+ mesothelial         │         14          │
│                         │           Coronary vSMC           │         15          │
│                         │     CoronaryArteryAdipocytes      │         16          │
│                         │                DC                 │         17          │
│                         │            EC_arterial            │         18          │
│                         │           EC_capillary            │         19          │
│                         │       EC_capillary_IGFBP5+        │         20          │
│                         │        EC_capillary_VEGFA+        │         21          │
│                         │     EC_capillary_VEGFA+NOVA1+     │         22          │
│                         │        EC_capillary_immune        │         23          │
│                         │          EC_endocardial           │         24          │
│                         │             EC_venous             │         25          │
│                         │         EClikeNeuralCells         │         26          │
│                         │         Great vessel vSMC         │         27          │
│                         │     LymphaticEndothelialCells     │         28          │
│                         │             MP_CXCL8+             │         29          │
│                         │             MP_FABP5+             │         30          │
│                         │             MP_LYVE1+             │         31          │
│                         │          MP_LYVE1+AGBL4+          │         32          │
│                         │            MP_cycling             │         33          │
│                         │             MastCells             │         34          │
│                         │        Microvascular SMC_1        │         35          │
│                         │        Microvascular SMC_2        │         36          │
│                         │               Mo/MP               │         37          │
│                         │               Mono                │         38          │
│                         │             NK_CD16hi             │         39          │
│                         │             NK_CD56hi             │         40          │
│                         │            NeuralCells            │         41          │
│                         │         NodalNeuralCells          │         42          │
│                         │          PacemakerCells           │         43          │
│                         │            Pericyte_1             │         44          │
│                         │            Pericyte_2             │         45          │
│                         │            Pericyte_3             │         46          │
│                         │        Proliferating vSMC         │         47          │
│                         │      PulmonaryVeinAdipocytes      │         48          │
│                         │              SAN_FB               │         49          │
│                         │     SOD2+ TIMP1+ mesothelial      │         50          │
│                         │   SinoatrialNodeCardiomyocytes    │         51          │
│                         │   VentricularCardiomyocytesLeft   │         52          │
│                         │ VentricularConductionSystemDistal │         53          │
│                         │         VesselNeuralCells         │         54          │
│                         │               aFB1                │         55          │
│                         │               aFB4                │         56          │
│                         │             doublets              │         57          │
│                         │               vFB1                │         58          │
│                         │          vFB2_activated           │         59          │
│                         │              vSMC_1               │         60          │
│                         │              vSMC_2               │         61          │
│                         │            valve_FB_AV            │         62          │
│                         │           valve_FB_MV1            │         63          │
│                         │           valve_FB_MV2            │         64          │
│                         │            vascular_FB            │         65          │
└─────────────────────────┴───────────────────────────────────┴─────────────────────┘
```

Example 43 (unknown):
```unknown
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
```

Example 44 (unknown):
```unknown
training model for 250 epochs...
```

Example 45 (unknown):
```unknown
You are using a CUDA device ('NVIDIA A100-SXM4-80GB') that has Tensor Cores. To properly utilize them, you should set `torch.set_float32_matmul_precision('medium' | 'high')` which will trade-off precision for performance. For more details, read https://pytorch.org/docs/stable/generated/torch.set_float32_matmul_precision.html#torch.set_float32_matmul_precision
```

Example 46 (python):
```python
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
File <timed exec>:5

Cell In[17], line 24, in c2l_make_reference(adata, subset_value, max_epochs)
     21 mod.view_anndata_setup()
     23 print(f'training model for {max_epochs} epochs...')
---> 24 mod.train(max_epochs=max_epochs, batch_size=2500, train_size=1, lr=0.002, use_gpu=True)
     26 print(f'view history...')
     27 mod.plot_history(20)

File /lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/cell2location/models/reference/_reference_model.py:157, in RegressionModel.train(self, max_epochs, batch_size, train_size, lr, **kwargs)
    154 kwargs["train_size"] = train_size
    155 kwargs["lr"] = lr
--> 157 super().train(**kwargs)

File /lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/scvi/model/base/_pyromixin.py:172, in PyroSviTrainMixin.train(self, max_epochs, use_gpu, train_size, validation_size, batch_size, early_stopping, lr, training_plan, plan_kwargs, **trainer_kwargs)
    162 trainer_kwargs["callbacks"].append(PyroJitGuideWarmup())
    164 runner = self._train_runner_cls(
    165     self,
    166     training_plan=training_plan,
   (...)
    170     **trainer_kwargs,
    171 )
--> 172 return runner()

File /lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/scvi/train/_trainrunner.py:82, in TrainRunner.__call__(self)
     79 if hasattr(self.data_splitter, "n_val"):
     80     self.training_plan.n_obs_validation = self.data_splitter.n_val
---> 82 self.trainer.fit(self.training_plan, self.data_splitter)
     83 self._update_history()
     85 # data splitter only gets these attrs after fit

File /lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/scvi/train/_trainer.py:193, in Trainer.fit(self, *args, **kwargs)
    187 if isinstance(args[0], PyroTrainingPlan):
    188     warnings.filterwarnings(
    189         action="ignore",
    190         category=UserWarning,
    191         message="`LightningModule.configure_optimizers` returned `None`",
    192     )
--> 193 super().fit(*args, **kwargs)

File /lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/pytorch_lightning/trainer/trainer.py:608, in Trainer.fit(self, model, train_dataloaders, val_dataloaders, datamodule, ckpt_path)
    606 model = self._maybe_unwrap_optimized(model)
    607 self.strategy._lightning_module = model
--> 608 call._call_and_handle_interrupt(
    609     self, self._fit_impl, model, train_dataloaders, val_dataloaders, datamodule, ckpt_path
    610 )

File /lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/pytorch_lightning/trainer/call.py:38, in _call_and_handle_interrupt(trainer, trainer_fn, *args, **kwargs)
     36         return trainer.strategy.launcher.launch(trainer_fn, *args, trainer=trainer, **kwargs)
     37     else:
---> 38         return trainer_fn(*args, **kwargs)
     40 except _TunerExitException:
     41     trainer._call_teardown_hook()

File /lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/pytorch_lightning/trainer/trainer.py:650, in Trainer._fit_impl(self, model, train_dataloaders, val_dataloaders, datamodule, ckpt_path)
    643 ckpt_path = ckpt_path or self.resume_from_checkpoint
    644 self._ckpt_path = self._checkpoint_connector._set_ckpt_path(
    645     self.state.fn,
    646     ckpt_path,  # type: ignore[arg-type]
    647     model_provided=True,
    648     model_connected=self.lightning_module is not None,
    649 )
--> 650 self._run(model, ckpt_path=self.ckpt_path)
    652 assert self.state.stopped
    653 self.training = False

File /lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/pytorch_lightning/trainer/trainer.py:1093, in Trainer._run(self, model, ckpt_path)
   1090 self._logger_connector.reset_metrics()
   1092 # strategy will configure model and move it to the device
-> 1093 self.strategy.setup(self)
   1095 # hook
   1096 if self.state.fn == TrainerFn.FITTING:

File /lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/pytorch_lightning/strategies/single_device.py:73, in SingleDeviceStrategy.setup(self, trainer)
     72 def setup(self, trainer: pl.Trainer) -> None:
---> 73     self.model_to_device()
     74     super().setup(trainer)

File /lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/pytorch_lightning/strategies/single_device.py:70, in SingleDeviceStrategy.model_to_device(self)
     68 def model_to_device(self) -> None:
     69     assert self.model is not None, "self.model must be set before self.model.to()"
---> 70     self.model.to(self.root_device)

File /lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/lightning_fabric/utilities/device_dtype_mixin.py:54, in _DeviceDtypeModuleMixin.to(self, *args, **kwargs)
     52 device, dtype = torch._C._nn._parse_to(*args, **kwargs)[:2]
     53 self.__update_properties(device=device, dtype=dtype)
---> 54 return super().to(*args, **kwargs)

File /lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/torch/nn/modules/module.py:1145, in Module.to(self, *args, **kwargs)
   1141         return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None,
   1142                     non_blocking, memory_format=convert_to_format)
   1143     return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
-> 1145 return self._apply(convert)

File /lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/torch/nn/modules/module.py:797, in Module._apply(self, fn)
    795 def _apply(self, fn):
    796     for module in self.children():
--> 797         module._apply(fn)
    799     def compute_should_use_set_data(tensor, tensor_applied):
    800         if torch._has_compatible_shallow_copy_type(tensor, tensor_applied):
    801             # If the new tensor has compatible tensor type as the existing tensor,
    802             # the current behavior is to change the tensor in-place using `.data =`,
   (...)
    807             # global flag to let the user control whether they want the future
    808             # behavior of overwriting the existing tensor or not.

File /lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/torch/nn/modules/module.py:797, in Module._apply(self, fn)
    795 def _apply(self, fn):
    796     for module in self.children():
--> 797         module._apply(fn)
    799     def compute_should_use_set_data(tensor, tensor_applied):
    800         if torch._has_compatible_shallow_copy_type(tensor, tensor_applied):
    801             # If the new tensor has compatible tensor type as the existing tensor,
    802             # the current behavior is to change the tensor in-place using `.data =`,
   (...)
    807             # global flag to let the user control whether they want the future
    808             # behavior of overwriting the existing tensor or not.

File /lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/torch/nn/modules/module.py:844, in Module._apply(self, fn)
    842 for key, buf in self._buffers.items():
    843     if buf is not None:
--> 844         self._buffers[key] = fn(buf)
    846 return self

File /lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/torch/nn/modules/module.py:1143, in Module.to.<locals>.convert(t)
   1140 if convert_to_format is not None and t.dim() in (4, 5):
   1141     return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None,
   1142                 non_blocking, memory_format=convert_to_format)
-> 1143 return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)

RuntimeError: CUDA error: out of memory
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.
```

---

## Functions¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig4_Cardiomyocytes/AtrialCardiomyocytes/Visium_mapping/c2l_pt3_plot_FFPE.html

**Contents:**
- Functions¶
- Read in metadata¶
- Data, parameters¶

2045 rows × 5 columns

2045 rows × 8 columns

**Examples:**

Example 1 (unknown):
```unknown
run='20Mar2024_fetal_FFPE'
```

Example 2 (python):
```python
import sys
import scanpy as sc
import anndata
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt 
import matplotlib as mpl

import cell2location
import scvi

from matplotlib import rcParams
rcParams['pdf.fonttype'] = 42 # enables correct plotting of text
import seaborn as sns
```

Example 3 (python):
```python
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm
Global seed set to 0
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/umap/distances.py:1063: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/umap/distances.py:1071: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/umap/distances.py:1086: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/umap/umap_.py:660: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
```

Example 4 (python):
```python
import gc
import torch
```

Example 5 (python):
```python
def read_and_qc(sample_name, path):
    r""" This function reads the data for one 10X spatial experiment into the anndata object.
    It also calculates QC metrics. Modify this function if required by your workflow.

    :param sample_name: Name of the sample
    :param path: path to data
    """

    adata = sc.read_visium(path + str(sample_name),
                           count_file='filtered_feature_bc_matrix.h5', load_images=True)
    adata.var_names_make_unique(join='-')
    
    adata.obs['sample'] = sample_name
    
    # rename genes to ENSEMBL
    adata.var['SYMBOL'] = adata.var_names
    adata.var_names = adata.var['gene_ids']
    adata.var_names.name = None

    # Calculate QC metrics
    sc.pp.calculate_qc_metrics(adata, inplace=True)
    adata.var['mt'] = [gene.startswith('MT-') for gene in adata.var['SYMBOL']]
    adata.obs['mt_frac'] = adata[:, adata.var['mt'].tolist()].X.sum(1).A.squeeze()/adata.obs['total_counts']

    # add sample name to obs names
    adata.obs["sample"] = [str(i) for i in adata.obs['sample']]
    adata.obs_names = adata.obs["sample"] \
                          + '_' + adata.obs_names
    adata.obs.index.name = 'spot_id'

    return adata

def select_slide(adata, s, s_col='sample'):
    r""" This function selects the data for one slide from the spatial anndata object.

    :param adata: Anndata object with multiple spatial experiments
    :param s: name of selected experiment
    :param s_col: column in adata.obs listing experiment name for each location
    """

    slide = adata[adata.obs[s_col].isin([s]), :]
    s_keys = list(slide.uns['spatial'].keys())
    s_spatial = np.array(s_keys)[[s in k for k in s_keys]][0]

    slide.uns['spatial'] = {s_spatial: slide.uns['spatial'][s_spatial]}

    return slide
```

Example 6 (python):
```python
meta=pd.read_csv('/nfs/team205/heart/anndata_objects/Foetal/metadata/HeartTeamSamples_Mappeddata_20230208_for-cell2loc.csv',
                index_col=0)
meta
```

Example 7 (python):
```python
pd.crosstab(meta['donor'],meta['cell2loc-map_group'])
```

Example 8 (unknown):
```unknown
# select slides
# Healthy FFPE
meta = meta[meta['cell2loc-map_group'].isin(['FFPE_16PCW'])]

sampleIDs = list(meta['sangerID'])
len(sampleIDs)
```

Example 9 (python):
```python
import os
results_folder = os.path.join(os.getcwd(),"analysis/Visium/cell2location_results")

ref_run_name = f'{results_folder}/reference_signatures/{run}'
run_name = f'{results_folder}/cell2location_map/{run}'
```

Example 10 (unknown):
```unknown
visium_path='/nfs/team205/heart/mapped/spaceranger200/'

celltype_key='fine_grain_modified'

vis_subset_key='cell2loc-map_group'
```

Example 11 (python):
```python
%%time
subset_values = os.listdir(os.path.join(results_folder, "cell2location_map", run))
print(f"Found these subsets {subset_values}")

for value in subset_values:
    print(f"###### {value} ######")
    adata_sub = sc.read(f"{run_name}/{value}/sp.h5ad")
    print("visium read in")
    adata_sub.obs[adata_sub.uns['mod']['factor_names']] = adata_sub.obsm['q05_cell_abundance_w_sf']
    IDs = meta[meta[vis_subset_key] == value].sangerID.tolist()
    print(f"{value}: {IDs}")

    for sample in IDs:
        try:
            slide = select_slide(adata_sub, sample)
        except IndexError as e:
            print(f"Sample {sample} not found in adata: {e}")
            continue  # Skip this sample if not found

        slide.obs[slide.uns['mod']['factor_names']] = slide.obsm['q05_cell_abundance_w_sf']
        list_of_cell_types = slide.uns['mod']['factor_names'].tolist()
        print("plotting HE image...")
        sc.pl.spatial(slide, color=None, size=0, img_key='hires', title=f"HE:{sample}", 
                      save=f"{run}__{value}__{sample}__HE.png", show=False)

        # Plotting abundances for all relevant cell states
        if list_of_cell_types:
            print("plotting abundances for all cell states...")
            sc.pl.spatial(slide, color=list_of_cell_types, cmap="RdPu", ncols=10, size=1.3, 
                          img_key='hires', vmin=0, vmax='p99.2', alpha_img=0.3, 
                          save=f"{run}__{value}__{sample}__all_cell_states_abundances.png", show=False)
```

Example 12 (unknown):
```unknown
Found these subsets ['5PCW', '20PCW', '13PCW', '4PCW', '7PCW', 'FFPE_16PCW']
###### 5PCW ######
visium read in
5PCW: []
###### 20PCW ######
visium read in
20PCW: []
###### 13PCW ######
visium read in
13PCW: []
###### 4PCW ######
visium read in
4PCW: []
###### 7PCW ######
visium read in
7PCW: []
###### FFPE_16PCW ######
visium read in
FFPE_16PCW: ['BHF_F_Hea13295830', 'BHF_F_Hea13295831']
```

Example 13 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
```

Example 14 (unknown):
```unknown
plotting HE image...
```

Example 15 (python):
```python
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
File <timed exec>:22

File /lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:1003, in spatial(adata, basis, img, img_key, library_id, crop_coord, alpha_img, bw, size, scale_factor, spot_size, na_color, show, return_fig, save, **kwargs)
   1000     cmap_img = None
   1001 circle_radius = size * scale_factor * spot_size * 0.5
-> 1003 axs = embedding(
   1004     adata,
   1005     basis=basis,
   1006     scale_factor=scale_factor,
   1007     size=circle_radius,
   1008     na_color=na_color,
   1009     show=False,
   1010     save=False,
   1011     **kwargs,
   1012 )
   1013 if not isinstance(axs, list):
   1014     axs = [axs]

File /lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392, in embedding(adata, basis, color, gene_symbols, use_raw, sort_order, edges, edges_width, edges_color, neighbors_key, arrows, arrows_kwds, groups, components, dimensions, layer, projection, scale_factor, color_map, cmap, palette, na_color, na_in_legend, size, frameon, legend_fontsize, legend_fontweight, legend_loc, legend_fontoutline, colorbar_loc, vmax, vmin, vcenter, norm, add_outline, outline_width, outline_color, ncols, hspace, wspace, title, show, save, ax, return_fig, **kwargs)
    389         # if user did not set alpha, set alpha to 0.7
    390         kwargs['alpha'] = 0.7 if alpha is None else alpha
--> 392     cax = scatter(
    393         coords[:, 0],
    394         coords[:, 1],
    395         marker=".",
    396         c=color_vector,
    397         rasterized=settings._vector_friendly,
    398         norm=normalize,
    399         **kwargs,
    400     )
    402 # remove y and x ticks
    403 ax.set_yticks([])

File /lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/scanpy/plotting/_utils.py:1108, in circles(x, y, s, ax, marker, c, vmin, vmax, scale_factor, **kwargs)
   1105 # You can set `facecolor` with an array for each patch,
   1106 # while you can only set `facecolors` with a value for all.
   1107 if scale_factor != 1.0:
-> 1108     x = x * scale_factor
   1109     y = y * scale_factor
   1110 zipped = np.broadcast(x, y, s)

TypeError: can't multiply sequence by non-int of type 'float'
```

Example 16 (python):
```python
%%time

adata_dict={}

subset_values = os.listdir(os.path.join(results_folder, "cell2location_map", run))
print(f"Found these subsets {subset_values}")

for value in subset_values:
    print(f"###### {value} ######")
    adata_sub = sc.read(f"{run_name}/{value}/sp.h5ad")
    print("visium read in")
    adata_sub.obs[adata_sub.uns['mod']['factor_names']] = adata_sub.obsm['q05_cell_abundance_w_sf']
    IDs = meta[meta[vis_subset_key] == value].sangerID.tolist()
    print(f"{value}: {IDs}")

    for sample in IDs:
        try:
            slide = select_slide(adata_sub, sample)
        except IndexError as e:
            print(f"Sample {sample} not found in adata: {e}")
            continue  # Skip this sample if not found

        slide.obs[slide.uns['mod']['factor_names']] = slide.obsm['q05_cell_abundance_w_sf']
        adata_dict[sample]=slide
```

Example 17 (unknown):
```unknown
Found these subsets ['5PCW', '20PCW', '13PCW', '4PCW', '7PCW', 'FFPE_16PCW']
###### 5PCW ######
visium read in
5PCW: []
###### 20PCW ######
visium read in
20PCW: []
###### 13PCW ######
visium read in
13PCW: []
###### 4PCW ######
visium read in
4PCW: []
###### 7PCW ######
visium read in
7PCW: []
###### FFPE_16PCW ######
visium read in
FFPE_16PCW: ['BHF_F_Hea13295830', 'BHF_F_Hea13295831']
```

Example 18 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
```

Example 19 (unknown):
```unknown
CPU times: user 2.11 s, sys: 8.17 s, total: 10.3 s
Wall time: 15.2 s
```

Example 20 (python):
```python
adata_dict.keys()
```

Example 21 (unknown):
```unknown
dict_keys(['BHF_F_Hea13295830', 'BHF_F_Hea13295831'])
```

Example 22 (python):
```python
from tqdm import tqdm
atrial_data_dict={}
for sample,tmp_adata in tqdm(adata_dict.items()):
    print(f"sample: {sample},  age is {tmp_adata.obs.age.unique().tolist()[0]}")
    tmp_df=tmp_adata[tmp_adata.obs.JC_edit=="Myocardium - Atrial"].obs[['AtrialCardiomyocytesImmature', 'AtrialCardiomyocytesLeft', 'AtrialCardiomyocytesRight']]

    if tmp_df.shape[0]>0:
        tmp_df["sample"]=sample
        tmp_df["age"]=tmp_adata.obs.age.unique().tolist()[0]
        atrial_data_dict[sample]=tmp_df
    else:
        print(f"no atrial myocardium spots annotated for sample: {sample}")
```

Example 23 (python):
```python
0%|                                                                                                                                                                                                                   | 0/2 [00:00<?, ?it/s]/tmp/ipykernel_1570535/2563514209.py:8: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  tmp_df["sample"]=sample
/tmp/ipykernel_1570535/2563514209.py:9: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  tmp_df["age"]=tmp_adata.obs.age.unique().tolist()[0]
/tmp/ipykernel_1570535/2563514209.py:8: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  tmp_df["sample"]=sample
/tmp/ipykernel_1570535/2563514209.py:9: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  tmp_df["age"]=tmp_adata.obs.age.unique().tolist()[0]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 89.72it/s]
```

Example 24 (unknown):
```unknown
sample: BHF_F_Hea13295830,  age is PCW-16w
sample: BHF_F_Hea13295831,  age is PCW-16w
```

Example 25 (python):
```python
data=pd.concat(list(atrial_data_dict.values()))
data
```

Example 26 (unknown):
```unknown
# Assuming 'data' is your pandas DataFrame
# Calculate the total for each row
total = data['AtrialCardiomyocytesImmature'] + data['AtrialCardiomyocytesRight'] + data['AtrialCardiomyocytesLeft']

# Calculate the proportions for each column
data['AtrialCardiomyocytesImmature_prop'] = data['AtrialCardiomyocytesImmature'] / total
data['AtrialCardiomyocytesRight_prop'] = data['AtrialCardiomyocytesRight'] / total
data['AtrialCardiomyocytesLeft_prop'] = data['AtrialCardiomyocytesLeft'] / total

# The new DataFrame 'data' now includes the proportion columns
data
```

Example 27 (python):
```python
import plotly.express as px
fig = px.scatter_ternary(data,
                         a="AtrialCardiomyocytesImmature_prop", b="AtrialCardiomyocytesLeft_prop", c="AtrialCardiomyocytesRight_prop",
                        color="age",
                        opacity=0.8)
fig.show()
```

Example 28 (unknown):
```unknown
data.to_csv("AtrialProportionsFFPE.csv")
```

---

## Functions¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/Visium/Visium_Intagration-Annotation/Visium_integration_no2_plots-UMAP-celltypes.html

**Contents:**
- Functions¶
- Read in anndata¶
- Basic plots¶
- Plot cell abundances¶
- Plot clusters on visium sections¶
  - FFPE¶
  - OCT¶
- DotPlot¶
- DA test¶

**Examples:**

Example 1 (python):
```python
import sys
import scanpy as sc
import anndata
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt 
import matplotlib as mpl

from matplotlib import rcParams
rcParams['pdf.fonttype'] = 42 # enables correct plotting of text
import seaborn as sns

import os
```

Example 2 (python):
```python
sc.settings.set_figure_params(dpi=120)
```

Example 3 (python):
```python
# This function selects the data for multiple slides from the spatial anndata object.
# Modified `select_slide` in cell2loc for multiple slides

# @param adata: anndata object with multiple spatial experiments
# @param s: list of selected slides
# @param s_col: column in adata.obs listing experiment name for each location

def select_multislide(adata, s, s_col='sample'):

    slides = adata[adata.obs[s_col].isin(s), :]
    s_keys = list(slides.uns['spatial'].keys())
    
    uns_dict = {}
    for ss in s:
        s_spatial=np.array(s_keys)[[ss in k for k in s_keys]][0]
        uns_dict[s_spatial]=slides.uns['spatial'][s_spatial]
    
    slides.uns['spatial'] = uns_dict

    return slides
```

Example 4 (python):
```python
# the anndata has "gene expression" in .X
# cell2location output is in .obsm['q05_cell_abundance_w_sf']
# kNN graph for UMAP/clustering was generated using cell2location output
adata_vis = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/visium_foetal_healthy-all_raw.h5ad')
adata_vis
```

Example 5 (unknown):
```unknown
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/anndata/_core/anndata.py:1830: UserWarning: Variable names are not unique. To make them unique, call `.var_names_make_unique`.
  utils.warn_names_duplicates("var")
```

Example 6 (unknown):
```unknown
AnnData object with n_obs × n_vars = 39223 × 33562
    obs: 'array_row', 'array_col', 'sample', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'mt_frac', 'sangerID', 'Publication', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'status', 'cell2loc-map_group', 'JC_edit', 'sample_for_cell2loc', 'week', 'leiden', 'region_cluster'
    var: 'gene_ids'
    uns: 'JC_edit_colors', 'cell2loc-map_group_colors', 'leiden', 'neighbors', 'region_cluster_colors', 'sample_for_cell2loc_colors', 'umap'
    obsm: 'MT', 'X_umap', 'means_cell_abundance_w_sf', 'q05_cell_abundance_w_sf', 'q05_cell_abundance_w_sf_ndarray', 'q95_cell_abundance_w_sf', 'spatial', 'stds_cell_abundance_w_sf'
    obsp: 'connectivities', 'distances'
```

Example 7 (python):
```python
# log-normalise gene expression data
sc.pp.filter_genes(adata_vis, min_cells=3)
sc.pp.normalize_total(adata_vis, target_sum=1e4)
sc.pp.log1p(adata_vis)
```

Example 8 (python):
```python
# just checking
pd.crosstab(adata_vis.obs['sample_for_cell2loc'],adata_vis.obs['cell2loc-map_group'])
```

Example 9 (python):
```python
# just checking
pd.crosstab(adata_vis.obs['JC_edit'],adata_vis.obs['cell2loc-map_group'])
```

Example 10 (python):
```python
with mpl.rc_context({'axes.facecolor':  'white',
                     'figure.figsize': [6, 6]}):
    sc.pl.umap(adata_vis, color=['region_cluster','week','sample'], size=5,
               color_map = 'RdPu', ncols = 3, wspace=0.3,
               legend_fontsize=14, frameon=False)
```

Example 11 (python):
```python
with mpl.rc_context({'axes.facecolor':  'white',
                     'figure.figsize': [6, 6]}):
    sc.pl.umap(adata_vis, color=['region_cluster'], groups=['14','6','4'],size=5,
               color_map = 'RdPu', ncols = 3, wspace=0.3,
               legend_fontsize=14, frameon=False)
```

Example 12 (python):
```python
with mpl.rc_context({'axes.facecolor':  'white',
                     'figure.figsize': [6, 6]}):
    sc.pl.umap(adata_vis, color=['total_counts','n_genes_by_counts'], size=5,
               color_map = 'RdPu', ncols = 3, wspace=0.3,
               legend_fontsize=14, frameon=False)
```

Example 13 (python):
```python
# manually annotated structures, plotting separately
with mpl.rc_context({'axes.facecolor':  'white',
                     'figure.figsize': [6, 6]}):
    
    fig, axs = plt.subplots(nrows=2, ncols=3, figsize=(20,10))
    dotsize = 10
    
    sc.pl.umap(adata_vis, color=['JC_edit'], groups=['Myocardium - Atrial',
                                                 'Myocardium - Papillary Muscle',
                                                 'Myocardium - Ventricular - Compact',
                                                 'Myocardium - Ventricular - Trabeculated'],
               size=dotsize, title='Myocardium', legend_fontsize=10, frameon=False,
              show=False, ax=axs[0,0])

    sc.pl.umap(adata_vis, color=['JC_edit'], groups=['Coronary artery','Great vessels',],
               size=dotsize, title='Vessels',legend_fontsize=10, frameon=False,
              show=False, ax=axs[0,1])
    
    sc.pl.umap(adata_vis, color=['JC_edit'], groups=['AV ring','AV valves','Aortic valve','Valve apparatus'],
               size=dotsize, title='Valves + alpha', legend_fontsize=10, frameon=False,
              show=False, ax=axs[0,2])

    sc.pl.umap(adata_vis, color=['JC_edit'], groups=['AVN','Bundle branch - left','Bundle branch - right'],
               size=dotsize, title='CCSs', legend_fontsize=10, frameon=False,
              show=False, ax=axs[1,0])

    sc.pl.umap(adata_vis, color=['JC_edit'], groups=['Epicardium'],
               size=dotsize, title='Epicardium', legend_fontsize=10, frameon=False,
              show=False, ax=axs[1,1])
    
    sc.pl.umap(adata_vis, color=['JC_edit'], groups=['LN'],
               size=dotsize, title='LN', legend_fontsize=10, frameon=False,
              show=False, ax=axs[1,2])
    plt.tight_layout()
```

Example 14 (python):
```python
# prepare cell2location anndata
cellnames = adata_vis.obsm['q05_cell_abundance_w_sf'].columns
cellnames = [x.split('q05cell_abundance_w_sf_')[1] for x in cellnames]
adata_c2l = anndata.AnnData(
    X=adata_vis.obsm['q05_cell_abundance_w_sf_ndarray'].copy(),
    obs=adata_vis.obs.copy(),
    var=pd.DataFrame(cellnames).set_index(0),
    obsm=adata_vis.obsm.copy(),
    obsp=adata_vis.obsp.copy(),
    uns=adata_vis.uns.copy()
)
adata_c2l
```

Example 15 (python):
```python
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/ipykernel_launcher.py:10: FutureWarning: X.dtype being converted to np.float32 from float64. In the next version of anndata (0.9) conversion will not be automatic. Pass dtype explicitly to avoid this warning. Pass `AnnData(X, dtype=X.dtype, ...)` to get the future behavour.
  # Remove the CWD from sys.path while we load stuff.
```

Example 16 (unknown):
```unknown
AnnData object with n_obs × n_vars = 39223 × 58
    obs: 'array_row', 'array_col', 'sample', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'mt_frac', 'sangerID', 'Publication', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'status', 'cell2loc-map_group', 'JC_edit', 'sample_for_cell2loc', 'week', 'leiden', 'region_cluster'
    uns: 'JC_edit_colors', 'cell2loc-map_group_colors', 'leiden', 'neighbors', 'region_cluster_colors', 'sample_for_cell2loc_colors', 'umap', 'log1p', 'week_colors'
    obsm: 'MT', 'X_umap', 'means_cell_abundance_w_sf', 'q05_cell_abundance_w_sf', 'q05_cell_abundance_w_sf_ndarray', 'q95_cell_abundance_w_sf', 'spatial', 'stds_cell_abundance_w_sf'
    obsp: 'connectivities', 'distances'
```

Example 17 (python):
```python
with mpl.rc_context({'axes.facecolor':  'white',
                     'figure.figsize': [6, 6]}):
    sc.pl.umap(adata_c2l, color=cellnames, size=20,
               color_map = 'RdPu', ncols = 4, wspace=0.1, vmax='p99.98',
               legend_fontsize=14, frameon=False)
```

Example 18 (python):
```python
# need to read in each object to plot visium section...(as far as i know)

adata_oct = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/visium-OCT_foetal_raw.h5ad')

adata_da = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/visium-OCT_foetal_DuctusA_raw.h5ad')
adata_da.obs['sample_for_cell2loc'] = adata_da.obs['sample'].copy()

adata_ffpe = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/visium-FFPE_foetal_raw.h5ad')
adata_ffpe.obs['sample_for_cell2loc'] = adata_ffpe.obs['sample'].copy()
adata_ffpe.obs['cell2loc-map_group'] = 'FFPE_16PCW'
adata_ffpe = adata_ffpe[adata_ffpe.obs['JC_edit'].isin(['Cavity'])==False]
```

Example 19 (python):
```python
# set color
## for clusters
palette1={}
for i,annot in enumerate(adata_vis.obs['region_cluster'].cat.categories):
    palette1[annot]=adata_vis.uns['region_cluster_colors'][i]

## for JC_edit
colormap='tab20'
palette2={}
for i,annot in enumerate(adata_vis.obs['JC_edit'].cat.categories):
    palette2[annot]=plt.get_cmap(colormap).colors[i]
```

Example 20 (python):
```python
adata_ffpe.obs['region_cluster'] = adata_vis.obs['region_cluster'].reindex(adata_ffpe.obs_names)
slides_to_plot = list(set(adata_ffpe.obs['sample']))

for s in slides_to_plot:
    slide = select_multislide(adata_ffpe, [s])
    with mpl.rc_context({'figure.figsize': [4.5, 4],
                        'axes.titlesize':20,}):
        fig, axs = plt.subplots(nrows=1, ncols=2, figsize=(15,5))

        sc.pl.spatial(slide,
                      color=['region_cluster'], 
                      size=1.2,img_key='hires',alpha_img=0.3, palette=palette1,
                      show=False, ax=axs[0],title = f'region_cluster, {s}'
                     )
        sc.pl.spatial(slide,
                      color=['JC_edit'], 
                      size=1.2,img_key='hires',alpha_img=0.3, palette=palette2,
                      show=False, ax=axs[1], title = f'JC_edit, {s}'
                     )
        plt.tight_layout()
```

Example 21 (python):
```python
slide = select_multislide(adata_ffpe, ['BHF_F_Hea13295831'])

with mpl.rc_context({'figure.figsize': [4.5, 4],
                    'axes.titlesize':20,}):

    fig, axs = plt.subplots(nrows=4, ncols=6, figsize=(20,15))
    for i,g in enumerate(adata_vis.obs['region_cluster'].cat.categories):
        sc.pl.spatial(slide,
                      color=['region_cluster'], groups=g,
                      size=1.2,img_key='hires',alpha_img=0.1, palette=palette1,
                      show=False, ax=axs[i//6,i%6]
                     )
    plt.tight_layout()
```

Example 22 (python):
```python
adata_oct.obs['region_cluster'] = adata_vis.obs['region_cluster'].reindex(adata_oct.obs_names)
slides_to_plot = list(set(adata_oct.obs['sample']))
slides_to_plot.sort()

for s in slides_to_plot:
    slide = select_multislide(adata_oct, [s])
    with mpl.rc_context({'figure.figsize': [4.5, 4],
                        'axes.titlesize':20,}):

        fig, axs = plt.subplots(nrows=1, ncols=2, figsize=(15,5))

        sc.pl.spatial(slide,
                      color=['region_cluster'], 
                      size=1.2,img_key='hires',alpha_img=0.3, palette=palette1,
                      show=False, ax=axs[0], title = f'region_cluster, {s}'
                     )
        sc.pl.spatial(slide,
                      color=['JC_edit'], 
                      size=1.2,img_key='hires',alpha_img=0.3, palette=palette2,
                      show=False, ax=axs[1], title = f'JC_edit, {s}'
                     )
        plt.tight_layout()
```

Example 23 (python):
```python
slide = select_multislide(adata_oct, ['HCAHeartST11605165'])

with mpl.rc_context({'figure.figsize': [4.5, 4],
                    'axes.titlesize':20,}):

    fig, axs = plt.subplots(nrows=4, ncols=6, figsize=(20,15))
    for i,g in enumerate(adata_vis.obs['region_cluster'].cat.categories):
        sc.pl.spatial(slide,
                      color=['region_cluster'], groups=g,
                      size=1.2,img_key='hires',alpha_img=0.1, palette=palette1,
                      show=False, ax=axs[i//6,i%6]
                     )
    plt.tight_layout()
```

Example 24 (python):
```python
sc.pl.dotplot(adata_c2l,
              adata_c2l.var_names, 
              groupby='JC_edit',
              dendrogram=True,
              standard_scale="var",
              color_map="Reds",
              expression_cutoff=0.1
             )
```

Example 25 (python):
```python
WARNING: dendrogram data not found (using key=dendrogram_JC_edit). Running `sc.tl.dendrogram` with default parameters. For fine tuning it is recommended to run `sc.tl.dendrogram` independently.
WARNING: You’re trying to run this on 58 dimensions of `.X`, if you really want this, set `use_rep='X'`.
         Falling back to preprocessing with `sc.pp.pca` and default params.
```

Example 26 (python):
```python
sc.pl.dotplot(adata_c2l,
              adata_c2l.var_names, 
              groupby='region_cluster',
              dendrogram=True,
              standard_scale="var",
              color_map="Reds",
              expression_cutoff=0.1
             )
```

Example 27 (python):
```python
WARNING: dendrogram data not found (using key=dendrogram_region_cluster). Running `sc.tl.dendrogram` with default parameters. For fine tuning it is recommended to run `sc.tl.dendrogram` independently.
```

Example 28 (python):
```python
sc.tl.rank_genes_groups(adata_c2l,
                       groupby='region_cluster')
sc.pl.rank_genes_groups_dotplot(adata_c2l,
                               standard_scale='var',
                               dendrogram=False)
```

Example 29 (python):
```python
WARNING: Default of the method has been changed to 't-test' from 't-test_overestim_var'
```

Example 30 (python):
```python
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/scanpy/tools/_rank_genes_groups.py:394: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead.  To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, 'names'] = self.var_names[global_indices]
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/scanpy/tools/_rank_genes_groups.py:396: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead.  To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, 'scores'] = scores[global_indices]
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/scanpy/tools/_rank_genes_groups.py:399: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead.  To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, 'pvals'] = pvals[global_indices]
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/scanpy/tools/_rank_genes_groups.py:409: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead.  To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, 'pvals_adj'] = pvals_adj[global_indices]
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/scanpy/tools/_rank_genes_groups.py:421: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead.  To get a de-fragmented frame, use `newframe = frame.copy()`
  foldchanges[global_indices]
```

---

## Import required modules¶

**URL:** http://127.0.0.1:9020/notebooks_html/Supplementary_materials/Macrophages/CellTypist_model_training_YS-MP.html

**Contents:**
- Import required modules¶
- Prepare functions¶
- Read in anndata¶
- Subset macrophages¶
- Merge non-pre, non-microglia populations¶
- Get informative genes¶
- HVGs¶
- Training CellTypist model¶
- Select cells to train¶
- Train¶

**Examples:**

Example 1 (python):
```python
import warnings
warnings.simplefilter(action='ignore', category=FutureWarning)

import scanpy as sc
import anndata
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import matplotlib as mpl
import seaborn as sns
import os 
from mpl_toolkits.axes_grid1 import make_axes_locatable

import celltypist
from celltypist import models

import scipy
import scanpy.external as sce
from fastprogress.fastprogress import progress_bar
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
-----
anndata             0.9.2
celltypist          1.6.0
matplotlib          3.5.2
mpl_toolkits        NA
numpy               1.22.3
pandas              1.5.3
scanpy              1.8.2
scipy               1.9.1
seaborn             0.12.2
session_info        1.0.0
-----
```

Example 4 (python):
```python
PIL                 10.0.0
asttokens           NA
backcall            0.2.0
beta_ufunc          NA
binom_ufunc         NA
certifi             2023.07.22
cffi                1.15.1
charset_normalizer  3.2.0
colorama            0.4.6
comm                0.1.4
cpuinfo             NA
cycler              0.10.0
cython_runtime      NA
dateutil            2.8.2
debugpy             1.6.7.post1
decorator           5.1.1
executing           1.2.0
h5py                3.9.0
hypergeom_ufunc     NA
idna                3.4
igraph              0.10.6
importlib_metadata  NA
ipykernel           6.25.1
jedi                0.19.0
joblib              1.3.2
kiwisolver          1.4.5
leidenalg           0.10.1
llvmlite            0.40.1
natsort             8.4.0
nbinom_ufunc        NA
ncf_ufunc           NA
numba               0.57.1
numexpr             2.8.5
packaging           23.1
parso               0.8.3
patsy               0.5.3
pexpect             4.8.0
pickleshare         0.7.5
pkg_resources       NA
platformdirs        3.10.0
plotly              5.16.1
prompt_toolkit      3.0.39
psutil              5.9.5
ptyprocess          0.7.0
pure_eval           0.2.2
pydev_ipython       NA
pydevconsole        NA
pydevd              2.9.5
pydevd_file_utils   NA
pydevd_plugins      NA
pydevd_tracing      NA
pygments            2.16.1
pyparsing           3.0.9
pytz                2023.3
requests            2.31.0
sinfo               0.3.4
six                 1.16.0
sklearn             1.3.0
sphinxcontrib       NA
stack_data          0.6.2
statsmodels         0.14.0
tables              3.8.0
texttable           1.6.7
threadpoolctl       3.2.0
tornado             6.3.3
traitlets           5.9.0
typing_extensions   NA
urllib3             2.0.4
wcwidth             0.2.6
zipp                NA
zmq                 25.1.1
```

Example 5 (unknown):
```unknown
-----
IPython             8.12.2
jupyter_client      8.3.0
jupyter_core        5.3.1
-----
Python 3.8.17 (default, Jul  5 2023, 21:04:15) [GCC 11.2.0]
Linux-4.15.0-213-generic-x86_64-with-glibc2.17
-----
Session information updated at 2023-11-18 10:54
```

Example 6 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 7 (python):
```python
sc.settings.verbosity = 1          # verbosity: errors (0), warnings (1), info (2), hints (3)
sc.settings.set_figure_params(dpi=80)
```

Example 8 (python):
```python
adata = sc.read_h5ad('/lustre/scratch126/cellgen/team205/kk18/datasets/YS/ys_portal_object_20221208.h5ad')
adata
```

Example 9 (unknown):
```unknown
AnnData object with n_obs × n_vars = 169494 × 37318
    obs: 'component', 'stage', 'sex', 'sort.ids', 'fetal.ids', 'orig.dataset', 'sequencing.type', 'lanes', 'LVL1', 'LVL2', 'LVL3'
    obsm: 'X_umap'
```

Example 10 (python):
```python
sc.pl.umap(adata, color='LVL3')
```

Example 11 (python):
```python
adata.X.data[:5]
```

Example 12 (unknown):
```unknown
array([0.94222367, 0.94222367, 0.94222367, 0.94222367, 0.94222367],
      dtype=float32)
```

Example 13 (python):
```python
set(adata.obs['LVL3'])
```

Example 14 (unknown):
```unknown
{'CMP',
 'DC2_CYCLING',
 'DEF_HSPC_1',
 'DEF_HSPC_2',
 'EARLY_ERYTHROID',
 'ELP',
 'ENDODERM',
 'ENDOTHELIUM_AEC',
 'ENDOTHELIUM_PROLIF_AEC',
 'ENDOTHELIUM_PROLIF_SINUSOIDAL_EC',
 'ENDOTHELIUM_SINUSOIDAL',
 'ENDOTHELIUM_VWF',
 'EOSINOPHIL_BASOPHIL',
 'EO_BASO_MAST_PRECURSOR',
 'ERYTHROID',
 'FIBROBLAST',
 'HE',
 'HSPC_1',
 'HSPC_2',
 'ILC_PRECURSOR',
 'IMMATURE_ENDOTHELIUM',
 'LMPP',
 'LYMPHOID B LIN',
 'MAC DC2',
 'MACROPHAGE_IRON_RECYCLING',
 'MACROPHAGE_KUPFFER_LIKE',
 'MACROPHAGE_LYVE1_HIGH',
 'MACROPHAGE_MICROGLIA',
 'MACROPHAGE_PRE_MAC',
 'MACROPHAGE_PROLIFERATING',
 'MAST_CELL',
 'MEMP',
 'MESOTHELIUM',
 'MID_ERYTHROID',
 'MK',
 'MONO MAC DC2',
 'MONO MAC PRE DC2',
 'MONOCYTE_0',
 'MONOCYTE_1',
 'MONOCYTE_MACROPHAGE',
 'MOP',
 'NEUTROPHIL_PRECURSOR',
 'NK',
 'PDC PRECURSOR',
 'PRE DC2',
 'PRIM_HSPC_1',
 'PRIM_HSPC_2',
 'PROMONOCYTE',
 'SMOOTH_MUSCLE'}
```

Example 15 (python):
```python
set(adata.obs['fetal.ids'])
```

Example 16 (unknown):
```unknown
{'F138',
 'F32',
 'F35',
 'F37',
 'F61',
 'F79',
 'F80',
 'Wang_CS10',
 'Wang_CS11',
 'mikola_1'}
```

Example 17 (python):
```python
cells_to_select = ['MACROPHAGE_IRON_RECYCLING',
 'MACROPHAGE_KUPFFER_LIKE',
 'MACROPHAGE_LYVE1_HIGH',
 'MACROPHAGE_MICROGLIA',
 'MACROPHAGE_PRE_MAC']
# excluding proliferating cells

adata = adata[adata.obs['LVL3'].isin(cells_to_select)]
adata.obs['LVL3'].value_counts()
```

Example 18 (unknown):
```unknown
MACROPHAGE_IRON_RECYCLING    22946
MACROPHAGE_LYVE1_HIGH         8385
MACROPHAGE_KUPFFER_LIKE       7724
MACROPHAGE_MICROGLIA           613
MACROPHAGE_PRE_MAC              29
Name: LVL3, dtype: int64
```

Example 19 (python):
```python
# filter genes
sc.pp.filter_genes(adata, min_cells=3)
```

Example 20 (python):
```python
sc.pl.umap(adata, color='LVL3')
```

Example 21 (python):
```python
for celltype in cells_to_select:
    sc.pl.umap(adata, color='LVL3', groups=celltype, size=50)
```

Example 22 (python):
```python
sc.pl.umap(adata, color='LVL3', groups='MACROPHAGE_PRE_MAC', size=50)
```

Example 23 (python):
```python
sc.pl.dotplot(adata,var_names=['LYVE1','CX3CR1','TREM2'],groupby='LVL3')
```

Example 24 (python):
```python
adata.obs['LVL3_mod'] = adata.obs['LVL3'].copy()
adata.obs.replace({'LVL3_mod':{
    'MACROPHAGE_IRON_RECYCLING':'MACROPHAGE',
 'MACROPHAGE_KUPFFER_LIKE':'MACROPHAGE',
 'MACROPHAGE_LYVE1_HIGH':'MACROPHAGE',
 # 'MACROPHAGE_MICROGLIA':'MACROPHAGE_MICROGLIA',
 # 'MACROPHAGE_PRE_MAC':'MACROPHAGE_PRE_MAC'
}}, inplace=True)
adata.obs['LVL3_mod'].value_counts()
```

Example 25 (unknown):
```unknown
MACROPHAGE              39055
MACROPHAGE_MICROGLIA      613
MACROPHAGE_PRE_MAC         29
Name: LVL3_mod, dtype: int64
```

Example 26 (unknown):
```unknown
AnnData object with n_obs × n_vars = 39697 × 28532
    obs: 'component', 'stage', 'sex', 'sort.ids', 'fetal.ids', 'orig.dataset', 'sequencing.type', 'lanes', 'LVL1', 'LVL2', 'LVL3', 'LVL3_mod'
    var: 'n_cells'
    uns: 'LVL3_colors'
    obsm: 'X_umap'
```

Example 27 (unknown):
```unknown
groupby = 'LVL3_mod'
max_n = 30
n_iteration = 100
n_detected = 30
batch_key = 'fetal.ids'
```

Example 28 (python):
```python
# iteration
res = {}
for i in progress_bar(range(n_iteration)):
    # subsample
    adata_sub = utils.sctk_subsample(adata, fraction=1, groupby=groupby, max_n=max_n, random_state=i)
    # remove donors those cell number is 1
    vals = adata_sub.obs[batch_key].value_counts()
    samples_to_remove = list(vals.index[vals==1])
    adata_sub = adata_sub[adata_sub.obs[batch_key].isin(samples_to_remove)==False]
    # hvg detection
    sc.pp.highly_variable_genes(adata_sub, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key=batch_key)
    # print(f'n_hvg: {sum(adata_sub.var.highly_variable)}')
    # store data
    res[i] = adata_sub.var.highly_variable
    del adata_sub

# plot
l = []
for i in range(n_iteration):
    l.append(len(res_df.index[res_df.sum(axis=1)>i]))
plt.scatter(x=range(n_iteration), y=l)
```

Example 29 (unknown):
```unknown
final seleted hvgs: 3089
```

Example 30 (python):
```python
# selection
res_df = pd.DataFrame(res)
hvgs = list(res_df.index[res_df.sum(axis=1)>=n_detected])
print(f'final seleted hvgs: {len(hvgs)}')
```

Example 31 (unknown):
```unknown
final seleted hvgs: 868
```

Example 32 (python):
```python
from geosketch import gs
```

Example 33 (unknown):
```unknown
Signature: gs(X, N, **kwargs)
Docstring:
Geometric sketching.
Wrapper around `gs_gap()`, see parameter documentation in that
function.
File:      ~/my-conda-envs/vitro/lib/python3.8/site-packages/geosketch/sketch.py
Type:      function
```

Example 34 (python):
```python
groups_to_subset = ['MACROPHAGE','MACROPHAGE_MICROGLIA']
max_n = 30
batch_key = 'fetal.ids'

adata.obs['select'] = True
adata.obs['geosketch_index'] = False
for grp in groups_to_subset:
    print(f'##### {grp} #####')
    # subset
    adata_sub = adata[adata.obs['LVL3_mod']==grp]
    # remove samples those cell number is 1
    vals = adata_sub.obs[batch_key].value_counts()
    samples_to_remove = list(vals.index[vals==1])
    obsnames_to_remove = adata_sub.obs_names[adata_sub.obs[batch_key].isin(samples_to_remove)]
    adata_sub = adata_sub[adata_sub.obs[batch_key].isin(samples_to_remove)==False]
    # hvg detection
    sc.pp.highly_variable_genes(adata_sub, min_mean=0.0125, max_mean=3, min_disp=0.5, 
                                batch_key=batch_key, subset=True)
    print(f'n_hvg: {adata_sub.shape[1]}')
    print(f'n_cell: {adata_sub.shape[0]}')
    # scale, pca
    print('pca, batch correction')
    sc.pp.scale(adata_sub, max_value=10)
    sc.tl.pca(adata_sub, svd_solver='arpack')
    # harmony for batch correction
    sce.pp.harmony_integrate(adata_sub, key=batch_key, max_iter_harmony=30)
    # geosketch
    print('geosketching')
    sketch_index = gs(adata_sub.obsm['X_pca_harmony'], max_n, replace=False)
    # store to adata
    obsnames = adata_sub.obs_names[sketch_index]
    obsnames_others = adata_sub.obs_names[[x for x in range(len(adata_sub.obs_names)) if x not in sketch_index]]
    adata.obs.loc[obsnames_to_remove,'select'] = False # removed cells before hvg detection
    adata.obs.loc[obsnames_others,'select'] = False
    adata.obs.loc[obsnames,'geosketch_index'] = True
    print('')
```

Example 35 (unknown):
```unknown
##### MACROPHAGE #####
n_hvg: 1280
n_cell: 39055
pca, batch correction
```

Example 36 (unknown):
```unknown
2023-11-18 11:56:57,958 - harmonypy - INFO - Computing initial centroids with sklearn.KMeans...
Computing initial centroids with sklearn.KMeans...
2023-11-18 11:57:31,447 - harmonypy - INFO - sklearn.KMeans initialization complete.
sklearn.KMeans initialization complete.
2023-11-18 11:57:31,865 - harmonypy - INFO - Iteration 1 of 30
Iteration 1 of 30
2023-11-18 11:58:02,145 - harmonypy - INFO - Iteration 2 of 30
Iteration 2 of 30
2023-11-18 11:58:32,740 - harmonypy - INFO - Converged after 2 iterations
Converged after 2 iterations
```

Example 37 (unknown):
```unknown
geosketching

##### MACROPHAGE_MICROGLIA #####
n_hvg: 904
n_cell: 612
pca, batch correction
```

Example 38 (unknown):
```unknown
2023-11-18 11:58:43,743 - harmonypy - INFO - Computing initial centroids with sklearn.KMeans...
Computing initial centroids with sklearn.KMeans...
2023-11-18 11:58:44,653 - harmonypy - INFO - sklearn.KMeans initialization complete.
sklearn.KMeans initialization complete.
2023-11-18 11:58:44,721 - harmonypy - INFO - Iteration 1 of 30
Iteration 1 of 30
2023-11-18 11:58:45,055 - harmonypy - INFO - Iteration 2 of 30
Iteration 2 of 30
2023-11-18 11:58:45,368 - harmonypy - INFO - Iteration 3 of 30
Iteration 3 of 30
2023-11-18 11:58:45,566 - harmonypy - INFO - Iteration 4 of 30
Iteration 4 of 30
2023-11-18 11:58:45,761 - harmonypy - INFO - Iteration 5 of 30
Iteration 5 of 30
2023-11-18 11:58:45,954 - harmonypy - INFO - Iteration 6 of 30
Iteration 6 of 30
2023-11-18 11:58:46,057 - harmonypy - INFO - Iteration 7 of 30
Iteration 7 of 30
2023-11-18 11:58:46,326 - harmonypy - INFO - Iteration 8 of 30
Iteration 8 of 30
2023-11-18 11:58:46,356 - harmonypy - INFO - Iteration 9 of 30
Iteration 9 of 30
2023-11-18 11:58:46,526 - harmonypy - INFO - Iteration 10 of 30
Iteration 10 of 30
2023-11-18 11:58:46,645 - harmonypy - INFO - Iteration 11 of 30
Iteration 11 of 30
2023-11-18 11:58:46,759 - harmonypy - INFO - Iteration 12 of 30
Iteration 12 of 30
2023-11-18 11:58:46,929 - harmonypy - INFO - Iteration 13 of 30
Iteration 13 of 30
2023-11-18 11:58:47,050 - harmonypy - INFO - Iteration 14 of 30
Iteration 14 of 30
2023-11-18 11:58:47,159 - harmonypy - INFO - Iteration 15 of 30
Iteration 15 of 30
2023-11-18 11:58:47,330 - harmonypy - INFO - Iteration 16 of 30
Iteration 16 of 30
2023-11-18 11:58:47,437 - harmonypy - INFO - Iteration 17 of 30
Iteration 17 of 30
2023-11-18 11:58:47,544 - harmonypy - INFO - Converged after 17 iterations
Converged after 17 iterations
```

Example 39 (unknown):
```unknown
geosketching
```

Example 40 (python):
```python
pd.crosstab(adata.obs['select'],adata.obs['geosketch_index'])
```

Example 41 (python):
```python
# select
adata_sel = adata[adata.obs['select']]
adata_sel.obs['LVL3_mod'].value_counts()
```

Example 42 (unknown):
```unknown
MACROPHAGE              30
MACROPHAGE_MICROGLIA    30
MACROPHAGE_PRE_MAC      29
Name: LVL3_mod, dtype: int64
```

Example 43 (python):
```python
'''
# subset
max_n = 30
adata_sub = utils.sctk_subsample(adata, fraction=1, groupby=groupby, max_n=max_n, random_state=0)
adata_sub.obs[groupby].value_counts()
'''
```

Example 44 (python):
```python
%%time
model = celltypist.train(adata_sel[:,hvgs],  # selecting defined hvgs
                         labels = groupby,
                         n_jobs = 10,
                         feature_selection = False, # since manually selected
                         check_expression = False
                        )

# Save the model
model.write('/nfs/team205/heart/anndata_objects/Foetal/celltypist/model_YS_portal_object_20221208_onlyMP_hvg_18Nov2023.pkl')
```

Example 45 (unknown):
```unknown
🍳 Preparing data before training
🔬 Input data has 89 cells and 868 genes
⚖️ Scaling input data
🏋️ Training data using logistic regression
✅ Model training done!
```

Example 46 (unknown):
```unknown
CPU times: user 23.7 ms, sys: 8.92 ms, total: 32.6 ms
Wall time: 63.1 ms
```

Example 47 (python):
```python
%%time
# train with all genes
model = celltypist.train(adata_sel,  # all genes
                         labels = groupby,
                         n_jobs = 10,
                         feature_selection = False, # since manually selected
                         check_expression = False
                        )

# Save the model
model.write('/nfs/team205/heart/anndata_objects/Foetal/celltypist/model_YS_portal_object_20221208_onlyMP_allgenes_18Nov2023.pkl')
```

Example 48 (unknown):
```unknown
🍳 Preparing data before training
✂️ 11196 non-expressed genes are filtered out
🔬 Input data has 89 cells and 17336 genes
⚖️ Scaling input data
🏋️ Training data using logistic regression
✅ Model training done!
```

Example 49 (unknown):
```unknown
CPU times: user 55.9 ms, sys: 11.7 ms, total: 67.6 ms
Wall time: 697 ms
```

---

## 

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/RNA/scnRNA_Integration-Annotation/Endothelial/Figure2EndothelialCells.html

36601 rows × 5 columns

36601 rows × 5 columns

**Examples:**

Example 1 (unknown):
```unknown
#Added the keys for age and sex
```

Example 2 (python):
```python
# Import packages, initial settings

import numpy as np
import matplotlib.pyplot as plt
import scanpy as sc
import pandas as pd
import anndata
import scrublet as scr
import seaborn as sns
import six
import sys
sys.modules['sklearn.externals.six'] = six
import scvi

plt.rcParams['pdf.fonttype'] = 42
plt.rcParams['ps.fonttype'] = 42

sc.settings.verbosity = 3  # verbosity: errors (0), warnings (1), info (2), hints (3)

from matplotlib.colors import LinearSegmentedColormap
cmap = LinearSegmentedColormap.from_list(name='gene_cmap', colors=['lightgrey', 'thistle', 'red', 'darkred']) 

sc.settings.set_figure_params(dpi=80, color_map='viridis', vector_friendly=False,  dpi_save=300, facecolor='white')
sc.logging.print_version_and_date()
#sc.logging.print_versions()
```

Example 3 (python):
```python
---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
Cell In[1], line 8
      6 import pandas as pd
      7 import anndata
----> 8 import scrublet as scr
      9 import seaborn as sns
     10 import six

ModuleNotFoundError: No module named 'scrublet'
```

Example 4 (python):
```python
adata=sc.read('./Objects/Feb28ObjectRaw.h5ad')
print(adata.shape)
```

Example 5 (python):
```python
FineGrainPalette={
    'AtrialCardiomyocytesLeft':'#eac075',
    'AtrialCardiomyocytesRight':'#4c6b8a',
    'AtrialCardiomyocytesCycling':'#e94bc7',
    'VentricularCardiomyocytesLeftCompact':'#ab1c3c',
    'VentricularCardiomyocytesRightCompact':'#fdba45',
    'VentricularCardiomyocytesLeftTrabeculated':'#2b9200',
    'VentricularCardiomyocytesRightTrabeculated':'#0273a8',
    'VentricularCardiomyocytesCycling':'#e1acff',
    'SinoatrialNodeCardiomyocytes':'#854834',
    'AtrioventricularNodeCardiomyocytes':'#688500',
    'VentricularConductionSystemProximal':'#01ccc8',
    'VentricularConductionSystemDistal':'#b58fab',
    'GreatVesselAdventitialFibroblasts':'#ff7e97',
    'CoronaryVesselAdventitialFibroblasts':'#ff9b6e',
    'MyocardialInterstitialFibroblasts1':'#8a6800',
    'MyocardialInterstitialFibroblasts2':'#206601',
    'Myofibroblasts':'#1bdebf',
    'LymphNodeFibroblasticReticularCells':'#005aa0',
    'ValveInterstitialCells':'#e478ff',
    'GreatVesselSmoothMuscleCells':'#ff665f',
    'CoronarySmoothMuscleCells':'#d2cb18',
    'DuctusArteriosusSmoothMuscleCells':'#83d7b3',
    'CoronaryPericytes':'#9088ff',
    'PericardialCells1':'#c95e00',
    'PericardialCells2':'#01e077',
    'PericardialCells3':'#ff88e4',
    'GreatVesselArterialEndothelialCells':'#d4004b',
    'GreatVesselVenousEndothelialCells':'#8f4300',
    'CoronaryArterialEndothelialCells':'#89da5a',
    'CoronaryVenousEndothelialCells':'#006f86',
    'CoronaryCapillaryEndothelialCells':'#aa7dff',
    'EndocardialCells':'#ff8241',
    'EndocardialCushionCells':'#46603a',
    'ValveEndothelialCells':'#414eb6',
    'LymphaticEndothelialCells':'#8b3f60',
    'MesothelialEpicardialCells':'#0295fe',
    'EpicardiumDerivedCells':'#ff92c4',
    'NeuronPrecursors':'#d30038',
    'ChromaffinCells':'#726800',
    'SympatheticNeurons':'#01894b',
    'ParaympatheticNeurons':'#545191',
    'SchwannCellPrecursors':'#b0d276',
    'SchwannCells':'#b30079',
    'MonocytesMPOpos':'#c50066',
    'Monocytes':'#fab4b6',
    'MonocyteDerivedCells':'#d14008',
    'MacrophagesCX3CR1pos':'#eabe97',
    'MacrophagesTIMD4pos':'#b2a200',
    'MacrophagesLYVE1pos':'#4a6005',
    'MacrophagesATF3pos':'#3db833',
    'DendriticCellsType1':'#007859',
    'DendriticCellsMature':'#01a9db',
    'PlasmacytoidDendriticCells':'#2c54ab',
    'MastCells':'#d0bef2',
    'Megakaryocytes':'#94298f',
    'TCellsCD4pos':'#ff387c',
    'TCellsCD8pos':'#9e3429',
    'TregsCD4pos':'#b37700',
    'ProBCells':'#8dab00',
    'BCells':'#a8d1a1',
    'BCellsMS4A1pos':'#00d2f0',
    'NaturalKillerCells':'#abb4ff',
    'InnateLymphoidCells':'#72497f',
}
sc.pl.umap(adata, color='fine_grain',palette=FineGrainPalette)

MidGrainPalette={
    'AtrialCardiomyocytes':'#af3a4d',
    'VentricularCardiomyocytes':'#b1756c',
    'CardiacConductionSystem':'#d45633',
    'Fibroblasts':'#9b6e4e',
    'MuralCells':'#ffb92f',
    'PericardialCells':'#b4f03b',
    'BloodVesselEndothelialCells':'#72cc57',
    'EndocardialCells':'#436e43',
    'LymphaticEndothelialCells':'#85cca0',
    'EpicardialCells':'#5eabb0',
    'Neurons':'#7bbdff',
    'Glia':'#403c90',
    'MyeloidCells':'#420066',
    'LymphoidCells':'#f553f6',
}
sc.pl.umap(adata, color='mid_grain',palette=MidGrainPalette)

CoarseGrainPalette={
    'Cardiomyocytes':'#bc5950',
    'Mesenchymal':'#af9c4c',
    'Endothelium':'#599e58',
    'Epicardium':'#5eabb0',
    'Neural':'#5e75b1',
    'Leukocytes':'#9a52a0',
}
sc.pl.umap(adata, color='coarse_grain',palette=CoarseGrainPalette)
```

Example 6 (python):
```python
set(adata.obs.fine_grain)
```

Example 7 (python):
```python
adata = adata[adata.obs['fine_grain'].isin(['GreatVesselArterialEndothelialCells','GreatVesselVenousEndothelialCells','CoronaryArterialEndothelialCells','CoronaryCapillaryEndothelialCells','CoronaryVenousEndothelialCells'])].copy()
adata=adata[adata.obs.cycling=='no',:].copy()
```

Example 8 (python):
```python
cell_cycle_genes = ['MCM5','PCNA','TYMS','FEN1','MCM2','MCM4','RRM1','UNG','GINS2','MCM6','CDCA7','DTL','PRIM1','UHRF1','MLF1IP','HELLS','RFC2','RPA2','NASP','RAD51AP1','GMNN','WDR76','SLBP','CCNE2','UBR7','POLD3','MSH2','ATAD2','RAD51','RRM2','CDC45','CDC6','EXO1','TIPIN','DSCC1','BLM','CASP8AP2','USP1','CLSPN','POLA1','CHAF1B','BRIP1','E2F8','HMGB2','CDK1','NUSAP1','UBE2C','BIRC5','TPX2','TOP2A','NDC80','CKS2','NUF2','CKS1B','MKI67','TMPO','CENPF','TACC3','FAM64A','SMC4','CCNB2','CKAP2L','CKAP2','AURKB','BUB1','KIF11','ANP32E','TUBB4B','GTSE1','KIF20B','HJURP','CDCA3','HN1','CDC20','TTK','CDC25C','KIF2C','RANGAP1','NCAPD2','DLGAP5','CDCA2','CDCA8','ECT2','KIF23','HMMR','AURKA','PSRC1','ANLN','LBR','CKAP5','CENPE','CTCF','NEK2','G2E3','GAS2L3','CBX5','CENPA']
s_genes = cell_cycle_genes[:43]
s_genes = [x for x in s_genes if x in adata.var_names]
print('S genes:', s_genes, '\n')
g2m_genes = cell_cycle_genes[43:]
g2m_genes = [x for x in g2m_genes if x in adata.var_names]
print('G2 genes:', g2m_genes, '\n')
cell_cycle_genes = [x for x in cell_cycle_genes if x in adata.var_names]

hb_genes1 = ['HBB','HBD','HBG1','HBG2','HBE1','HBZ','HBM','HBA2','HBA1', 'HBQ1']
hb_genes1 = [x for x in hb_genes1 if x in adata.var_names]
print('Hemoglobin genes set 1:', hb_genes1, '\n')

hb_genes2 = ['AHSP','GYPA','RHAG','GFI1B','KEL','EPB42','ALAS2','TSPO2','GYPB','GYPE','SPTA1','RHCE','KLF1','RHD','SLC4A1','SNX22','SQSTM1','SOD1','FTH1','MAP1LC3B','SLC25A37','TRIM58','HMBS']
hb_genes2 = [x for x in hb_genes2 if x in adata.var_names]
print('Hemoglobin genes set 2:', hb_genes2, '\n')

mito_genes=['MT-ND1','MT-ND2','MT-CO1','MT-CO2','MT-ATP8','MT-ATP6','MT-CO3','MT-ND3','MT-ND4L', 'MT-ND4', 'MT-ND5', 'MT-ND6', 'MT-CYB']
mito_genes = [x for x in mito_genes if x in adata.var_names]
print('Mitochondrial genes:', mito_genes, '\n')

ribo_genes = ['RPLP0','RPLP1','RPLP2','RPL3','RPL3L','RPL4','RPL5','RPL6','RPL7','RPL7A','RPL7L1','RPL8','RPL9','RPL10','RPL10A','RPL10L','RPL11','RPL12','RPL13','RPL13A','RPL14','RPL15','RPL17','RPL18','RPL18A','RPL19','RPL21','RPL22','RPL22L1','RPL23','RPL23A','RPL24','RPL26','RPL26L1','RPL27','RPL27A','RPL28','RPL29','RPL30','RPL31','RPL32','RPL34','RPL35','RPL35A','RPL36','RPL36A','RPL36AL','RPL37','RPL37A','RPL38','RPL39','RPL39L','UBA52','RPL41','RPSA','RPS2','RPS3','RPS3A','RPS4X','RPS4Y1','RPS4Y2','RPS5','RPS6','RPS7','RPS8','RPS9','RPS10','RPS11','RPS12','RPS13','RPS14','RPS15','RPS15A','RPS16','RPS17','RPS18','RPS19','RPS20','RPS21','RPS23','RPS24','RPS25','RPS26','RPS27','RPS27A','RPS27L','RPS28','RPS29','FAU']
ribo_genes = [x for x in ribo_genes if x in adata.var_names]
print('Ribo genes:', ribo_genes, '\n')

stress_genes=['FOS','FOSB','JUN','JUNB','JUND','ZFP36','SOCS3','CEBPD','CEBPB','ATF3','EGR1','HSPA1A','HSPA1B','HSP90AA1','HSP90AB1','HSPA8','HSPB1','HSPE1','IER3','IER2','BTG1','BTG2','DUSP1']
stress_genes = [x for x in stress_genes if x in adata.var_names]
print('Tissue dissociation stress associated genes:', stress_genes, '\n')

FoxProteins=['FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1']
chrY_genes=['SRY', 'RPS4Y1', 'ZFY', 'ZFY-AS1', 'LINC00278', 'PCDH11Y', 'TTTY2B', 'TBL1Y', 'PRKY', 'USP9Y', 'DDX3Y', 'UTY', 'TMSB4Y', 'NLGN4Y', 'TTTY14', 'KDM5D', 'TTTY10', 'EIF1AY', 'PRORY', 'LINC00266-4P']
MembraneProteins1=['AADAT','AAK1','ABCA12','ABCA7','ABCB1','ABCB11','ABCB4','ABCB6','ABCC1','ABCC3','ABCC4','ABCC5','ABCC6','ABCG2','ABHD3','ABI1','ABLIM3','AC004691.2','AC004922.1','AC005324.3','AC005726.1','AC008403.1','AC138696.1','ACIN1','ACKR3','ACLY','ACOT13','ACOT9','ACTN1','ACTR10','ACTR6','ADA','ADAM10','ADAM12','ADAM22','ADAM28','ADAM33','ADAMTS1','ADAMTS2','ADAMTSL4','ADAP1','ADCK5','ADD1','ADD2','ADGRB2','ADGRD1','ADGRG2','ADGRG7','ADGRL2','ADH1A','ADH1B','ADH1C','ADH7','ADK','ADORA1','ADRA1B','ADRM1','ADSS','AFAP1L2','AFDN','AFG1L','AGER','AGRN','AHNAK','AHNAK2','AIFM2','AKAP10','AKAP11','AKAP12','AKR1B10','AKR1B15','AL132780.3','AL139300.1','AL355315.1','AL358113.1','AL645922.1','ALDH3A1','ALK','ALOX15B','ALOXE3','ALPI','ALPP','ALPPL2','ALS2CR12','AMACR','AMDHD1','AMER1','AMER2','AMIGO2','AMOT','AMOTL1','AMOTL2','AMPH','ANK2','ANK3','ANKEF1','ANKFN1','ANKIB1','ANKLE2','ANKRA2','ANKRD13A','ANKRD35','ANKRD42','ANKRD45','ANKS1B','ANO1','ANO10','ANO2','ANO6','ANPEP','ANXA1','ANXA13','ANXA2','ANXA3','AP2M1','APBB1','APBB1IP','APC','APLP1','APMAP','APOC3','APOD','APOLD1','AQP1','AQP10','AQP3','AQP4','AQP5','ARAP1','ARF1','ARHGAP18','ARHGAP19','ARHGAP21','ARHGAP22','ARHGAP23','ARHGAP24','ARHGAP28','ARHGAP29','ARHGAP33','ARHGAP36','ARHGAP40','ARHGAP9','ARHGDIG','ARHGEF1','ARHGEF11','ARHGEF12','ARHGEF15','ARHGEF26','ARHGEF28','ARHGEF40','ARHGEF5','ARID2','ARID4A','ARID5B','ARL13A','ARL15','ARL4C','ARMC12','ARMCX5','ARPC1A','ARPC5','ARRB2','ARRDC4','ARVCF','ASAP1','ASB5','ASGR1','ASGR2','ASH2L','ASIC1','ASIC4','ASPM','ASPSCR1','ATG13','ATG3','ATG7','ATIC','ATOX1','ATP10D','ATP12A','ATP13A3','ATP1B3','ATP23','ATP2B1','ATP2B4','ATP2C2','ATP6V0A2','ATP6V1H','ATP8A2','ATP8B1','ATXN10','ATXN3','ATXN7L3','AVEN','AVPI1','AXIN2','AXL','B2M','B3GALT2','BACE1','BAG3','BAIAP2','BAIAP2L1','BANK1','BASP1','BBOF1','BCAR1','BCAR3','BCL2L12','BCORL1','BCR','BDKRB2','BEND6','BFSP1','BFSP2','BICD2','BIN2','BLNK','BLVRB','BMPR2','BMX','BOC','BOLL','BORCS5','BORCS8-MEF2B','BRK1','BRPF1','BRSK1','BSDC1','BST2','BTK','BTN2A1','BTRC','BUB1B-PAK6','BVES','BZW2','C10orf90','C11orf42','C11orf52','C12orf73','C12orf76','C14orf37','C14orf93','C16orf74','C17orf58','C17orf62','C18orf21','C19orf33','C19orf38','C19orf48','C19orf71','C1orf116','C1orf189','C1orf35','C1orf54','C1orf68','C1orf74','C1QBP','C2orf72','C3orf14','C3orf20','C3orf38','C4orf19','C4orf22','C7orf43','C8orf33','C8orf37','C9orf153','C9orf3','CA14','CA9','CABLES2','CACNA1E','CACNG4','CADM1','CALCRL','CALD1','CALML5','CALN1','CAMK2A','CAMK2B','CAMK2D','CAMK2G','CAP2','CAPS','CASKIN2','CASP4','CAST','CAV3','CAVIN1','CAVIN2','CAVIN3','CAVIN4','CBWD1','CBWD2','CBWD3','CBWD5','CBWD6','CC2D1A','CCBE1','CCDC107','CCDC112','CCDC115','CCDC124','CCDC136','CCDC144NL','CCDC15','CCDC154','CCDC170','CCDC24','CCDC28A','CCDC59','CCDC62','CCDC66','CCDC69','CCDC8','CCDC83','CCDC85C','CCDC93','CCDC97','CCNT2','CCNYL1','CCR3','CCR4','CCT3','CD109','CD163','CD164L2','CD200R1','CD226','CD248','CD274','CD2AP','CD300A','CD300LB','CD33','CD34','CD37','CD38','CD3E','CD3G','CD4','CD44','CD46','CD47','CD48','CD79B','CD81','CD84','CD86','CD8A','CD8B','CD9','CD93','CD99L2','CDAN1','CDC42EP1','CDC42EP3','CDC42EP4','CDC42SE1','CDCA4','CDH1','CDH13','CDH15','CDH17','CDH2','CDH26','CDH3','CDH4','CDH5','CDH6','CDH8','CDIPT','CDK16','CDK5','CDK5RAP2','CDK7','CDRT15','CDV3','CEACAM5','CELSR1','CENPJ','CEP162','CEP295','CEP55','CEP57L1','CEP89','CERCAM','CFAP157','CFAP206','CFAP58','CFAP65','CFB','CFL2','CFLAR','CGN','CGNL1','CHCHD7','CHIC2','CHMP2A','CHRM3','CHRNA1','CHRNA5','CHRNB4','CHRND','CIB1','CIB3','CIP2A','CKAP5','CLCA2','CLCA4','CLCN5','CLCN6','CLDN11','CLDN18','CLDN2','CLDN23','CLDN3','CLDN4','CLDN7','CLDN9','CLEC2A','CLEC4F','CLEC5A','CLECL1','CLIC2','CLIC5','CLTB','CLUL1','CMTM4','CMTM7','CMTR2','CMYA5','CNGA1','CNKSR2','CNNM1','CNNM4','CNP','CNR1','CNTD1','COBL','COBLL1','COG3','COL16A1','COL17A1','COL4A5','COL6A2','COLEC12','COLQ','COMMD4','COMTD1','COPRS','COQ7','CORIN','CORO1B','CORO2A','CORO2B','COX6B2','CPEB3','CPPED1','CPZ','CRB3','CREB5','CRIP1','CRIP2','CRK','CRLF3','CROCC','CRTC1','CRYAB','CRYBG3','CRYL1','CSDE1','CSF1','CSF1R','CSPG4','CST6','CTNNA1','CTNNB1','CTNNBIP1','CTNND1','CTNND2','CTSF','CTSV','CTTN','CTXN1','CUEDC1','CUTA','CX3CL1','CX3CR1','CXADR','CXCR2','CYB5D2','CYBRD1','CYFIP2','CYP20A1','CYP24A1','CYTH2','CYTH4','DAAM1','DAB2','DAG1','DAGLB','DAW1','DBN1','DCAF13','DCAF16','DCBLD2','DCHS1','DCHS2','DCP2','DCT','DCTN6','DCUN1D2','DDAH1','DDB2','DDIAS','DDIT4L','DDR1','DDR2','DDX58','DDX60L','DENND6B','DFFA','DGKG','DGKH','DHDDS','DHPS','DHRS7B','DIAPH1','DIAPH3','DIO2','DIS3L','DLG1','DLG2','DLG5','DLGAP3','DLL3','DMWD','DNAAF1','DNAAF4','DNAH17','DNAJA4','DNAJB13','DNAJB4','DNAJC18','DNAJC5','DNAJC6','DNAJC9','DOC2A','DOCK4','DOK3','DONSON','DPEP1','DPH1','DPPA5','DPYSL2','DQX1','DRD1','DRP2','DSC3','DSEL','DSG2','DSP','DTNA','DUS1L','DUSP15','DXO','DYSF','ECSCR','EDA2R','EDNRA','EDNRB','EEF1AKMT1','EEF2','EFCAB1','EFCAB11','EFCAB2','EFCAB7','EFHC2','EFNB1','EFR3A','EFR3B','EGFR','EHBP1','EHD1','EHD2','EHD3','EHD4','EIF2B2','EIF5','EIF5B','ELFN1','ELMO3','ELN','ELOC','EMP3','ENAH','ENC1','ENG','ENKD1','ENO1','ENO2','ENO3','ENOX1','ENOX2','EPB41','EPB41L1','EPB41L2','EPB41L3','EPB41L4A','EPB41L4B','EPB41L5','EPCAM','EPHA2','EPHA3','EPHB1','EPHB2','EPN1','EPOR','EPS8L2','ERAP1','ERBB2','ERBB3','ERBIN','ERC1','ERCC4','ERMN','ESCO2','ETNK1','ETS2','EVA1A','EVI2B','EXOC1','EXOC3L1','EXOC7','EZR','F11R','F2RL1','F2RL3','FAAP20','FABP2','FABP3','FABP5','FAIM','FAM126A','FAM126B','FAM129A','FAM129B','FAM13A','FAM163B','FAM171A1','FAM193A','FAM205A','FAM210B','FAM222A','FAM45A','FAM47E','FAM47E-STBD1','FAM72A','FAM72B','FAM72C','FAM72D','FANCG','FAS','FASN','FBLIM1','FBLN2','FBLN5','FBLN7','FBP2','FBRSL1','FBXL2','FBXO16','FBXO38','FBXO41','FBXW11','FCAR','FCGBP','FCGR1A','FCGR1B','FCGR2A','FCHSD2','FFAR4','FGD1','FGD5','FGFR2','FGR','FHIT','FHL1','FIBCD1','FIGN','FILIP1','FILIP1L','FKBP2','FLAD1','FLCN','FLNA','FLNB','FLNC','FLOT1','FLOT2','FLRT3','FLT1','FLT4','FLVCR1','FMN2','FMNL2','FMNL3','FNBP1','FOXA2','FOXN3','FP565260.3','FRMD3','FRMD7','FRMD8','FRS2','FSCN1','FTCD','FUT2','FXYD3','FYB1','FYN','FZD1','FZD2','FZD4','FZD6','GAB1','GAB2','GABRA1','GABRA2','GABRA3','GABRA5','GABRG1','GAD1','GADL1','GAGE1','GAGE10','GAGE12C','GAGE12D','GAGE12E','GAGE12F','GAGE12G','GAGE12H','GAGE12J','GAGE13','GAGE2A','GAGE2E','GALNT10','GALR1','GAP43','GAPDH','GAPT','GAPVD1','GAS2','GAS2L1','GAS7','GAS8','GBP4','GCA','GCC1','GCLM','GCSAML-AS1','GFI1B','GFOD1','GFRA3','GHR','GID8','GINM1','GIPC1','GJA1','GJB3','GJB4','GJB6','GJB7','GJC1','GJC2','GLO1','GLP2R','GLRX','GLUL','GMIP','GNAI2','GNAQ','GNAS','GNB1','GNB2','GNB3','GNB4','GNG2','GP9','GPAT3','GPATCH1','GPBP1','GPBP1L1','GPC1','GPC3','GPC4','GPI','GPR1','GPR142','GPR156','GPR157','GPR160','GPR173','GPR20','GPR27','GPR50','GPR63','GPRC5A','GPRIN1','GPRIN2','GPX8','GRAMD1A','GRAMD1B','GRAMD1C','GRAP2','GRB7','GRID2IP','GRIK5','GRK3','GRK5','GRPR','GSDMA','GTF2F1','GTSE1','GUCA1A','GYPA','GYPC','HAPLN3','HARBI1','HAS1','HAUS4','HAUS7','HCAR2','HCAR3','HCFC2','HCK','HCLS1','HCST','HDAC8','HEG1','HELZ','HENMT1','HERC2','HERPUD1','HFE','HIF1A','HIF3A','HINT1','HIP1R','HLA-A','HLA-DPB1','HMGCS1','HMOX1','HOMER3','HOXC5','HOXD4','HPS3','HRASLS5','HRH1','HS6ST3','HSH2D','HSPB1','HSPG2','HTR1B','HTR7','HTRA1','HUNK','HYDIN','HYI','HYLS1','ICAM1','ICAM2','ICOS','ICOSLG','IFFO2','IFIT5','IFNGR1','IFRD1','IFT122','IGFL2','IGSF11','IGSF23','IKZF5','IL10RA','IL12RB1','IL12RB2','IL13','IL13RA2','IL16','IL17RB','IL1R1','IL1RL1','IL21R','IL4R','IL6ST','IL7R','ILDR2','ILK','INPP5A','INSR','INTS12','IP6K2','IPCEF1','IQGAP1','IQGAP2','IREB2','IRF2','IRS1','ISLR','ITGA2B','ITGA3','ITGA4','ITGA7','ITGA8','ITGA9','ITGB1','ITGB2','ITGB3','ITGB4','ITGB5','ITGB6','ITGB7','ITGB8','ITK','ITM2A','ITPKA','ITSN1','ITSN2','IYD','JAG1','JAK2','JAK3','JAM2','JAML','JCAD','JMJD4','JPT2','JUP','KANK1','KANK3','KANSL2','KARS','KATNB1','KCNAB1','KCNB2','KCNH8','KCNJ15','KCNK13','KCNK17','KCNK6','KCNK7','KCNK9','KCNN2','KCNS3','KCNU1','KDF1','KDR','KEL','KIAA0556','KIAA0895','KIAA1324','KIAA1522','KIAA1958','KIF17','KIF19','KIF21A','KIF21B','KIF26A','KIF26B','KISS1R','KIT','KLC1','KLC2','KLF9','KLHDC7B','KLHL41','KLHL6','KLHL7','KLK7','KLRC1','KLRC2','KLRF1','KLRK1','KMT2D','KNDC1','KNSTRN','KNTC1','KPTN','KRT75','KRT77','KRTAP25-1','L1CAM','L3MBTL1','LAIR1','LAMA4','LAMC1','LAMP2','LAMTOR1','LAPTM4A','LASP1','LAT','LAT2','LAX1','LCP1','LCTL','LDB2','LDLRAD3','LENG1','LGALS3','LGALS4','LGALS8','LGR6','LHFPL2','LHX6','LIMA1','LINGO1','LIPJ','LMCD1','LMOD2','LMTK2','LMTK3','LNX1','LNX2','LONRF1','LONRF3','LOXHD1','LPAR1','LPAR3','LPAR4','LPIN1','LPIN2','LRCH2','LRP11','LRP12','LRRC15','LRRC26','LRRC34','LRRC45','LRRC69','LRRC7','LRRC9','LRRFIP2','LRRIQ3','LSMEM2','LSR','LTB4R2','LUZP2','LY6K','LYN','LYPD1','LYST','LZTS1','LZTS2','MACF1','MADD','MAGI1','MAGI3','MAMDC2','MAP1S','MAP2','MAP2K1','MAP2K4','MAP2K7','MAP3K12','MAP3K21','MAP3K6','MAP3K7','MAP4','MAP4K1','MAP4K5','MAP6','MAPK15','MAPK8IP1','MAPT','MARCH10','MARCH7','MARCKS','MARK2','MARVELD2','MAS1L','MAT2B','MBP','MC5R','MCAM','MCEMP1','MCF2L2','MCMBP','MCTS1','MDGA1','ME1','MEF2B','MEIOC','MEPCE','MERTK','MET','METAP1','METAP2','METTL15','METTL25','METTL9','MFAP3L','MFSD1','MFSD2A','MGAT4C','MIB1','MICAL2','MICAL3','MICALL1','MICB','MIEN1','MIGA2','MISP','MKLN1','MKRN3','MLF2','MLKL','MLST8','MMACHC','MMP10','MMP15','MMP24','MMP25','MOAP1','MPHOSPH8','MPHOSPH9','MPI','MPIG6B','MPL','MPP1','MPP5','MPP7','MPZL1','MPZL2','MR1','MRGPRF','MRNIP','MRPL42','MRPL44','MRPL46','MRPS18B','MS4A1','MS4A12','MS4A14','MS4A15','MSN','MSRA','MTCP1','MTFR1L','MTHFR','MTM1','MTPN','MTSS1','MUC1','MUC20','MVD','MYH11','MYH9','MYL4','MYLK','MYO10','MYO1B','MYO1C','MYO1E','MYO1G','MYO3A','MYO5B','MYO9A','MYOF','N4BP2','NAA35','NAB1','NAIF1','NAMPT','NAP1L2','NAPG','NARFL','NCAM1','NCAM2','NCK1','NCKAP1L','NCKAP5','NCKIPSD','NCOA1','NCS1','NDC1','NECTIN2','NECTIN3','NECTIN4','NEDD1','NEDD9','NEO1','NEU1','NEXN','NF2','NFE2L2','NFKBID','NGEF']
MembraneProteins2=['NGFR','NHS','NHSL2','NID2','NINJ2','NKTR','NKX2-8','NLGN1','NLGN3','NLK','NLRX1','NMB','NMBR','NME1','NME2','NMT1','NMT2','NMUR2','NOS1','NOTCH2','NPFFR2','NPL','NPNT','NPR1','NPTN','NPTXR','NR2C1','NRAS','NRCAM','NRP2','NRSN1','NRXN1','NSD1','NSFL1C','NT5E','NTNG1','NTNG2','NTSR1','NUDCD3','NUDT16L1','NUDT9','NUMB','NUP35','NUS1','OAS3','OBSCN','OCA2','OCLN','OCM','OCM2','ODC1','ODR4','OGT','OLFM4','OLIG2','OLR1','OPHN1','OPRK1','OR10P1','OR1N2','OR2C3','OR6B2','OR6B3','OR6C6','OR8S1','OSBPL10','OSBPL6','OSBPL7','OSCAR','OSCP1','OSGEP','OSR2','OTOP2','OTULIN','OXGR1','P3H3','PACSIN2','PACSIN3','PAFAH1B2','PAG1','PAIP1','PAK1','PAK4','PAK6','PALLD','PALM','PALM2','PALM2-AKAP2','PALM3','PALMD','PANX1','PANX2','PAQR8','PARD3','PARD3B','PARD6A','PARD6G','PARP6','PATJ','PAWR','PCBD2','PCDH1','PCDH17','PCDH7','PCDHA1','PCDHA11','PCDHA12','PCDHA4','PCDHA5','PCDHB1','PCDHB11','PCDHB12','PCDHB16','PCDHB4','PCDHB5','PCDHGA1','PCDHGA10','PCDHGA11','PCDHGA12','PCDHGA2','PCDHGA3','PCDHGA4','PCDHGA5','PCDHGA6','PCDHGA7','PCDHGA8','PCDHGA9','PCDHGB1','PCDHGB2','PCDHGB3','PCDHGB4','PCDHGB5','PCDHGB6','PCDHGB7','PCDHGC3','PCDHGC4','PCDHGC5','PCLO','PCMTD1','PDAP1','PDCD7','PDE3A','PDE4A','PDE4D','PDE9A','PDGFC','PDGFRA','PDHX','PDLIM1','PDLIM5','PDPN','PDZD2','PDZD8','PEAR1','PEBP1','PFDN1','PGAP3','PGLYRP2','PGLYRP4','PGRMC2','PHACTR1','PHACTR4','PHF20L1','PHGDH','PHLDB2','PHLDB3','PHOSPHO1','PHPT1','PI3','PI4K2A','PI4KA','PIEZO2','PIGB','PIGN','PIGW','PIGX','PIK3AP1','PIK3CG','PIK3R6','PIP4K2A','PIP4K2C','PIP5K1A','PIRT','PITPNM3','PKD2L2','PKN1','PKN2','PKP1','PKP2','PKP3','PKP4','PLAA','PLAUR','PLCB1','PLCB2','PLCD3','PLCD4','PLEKHA4','PLEKHA6','PLEKHA7','PLEKHG3','PLEKHG4','PLEKHG6','PLEKHH2','PLEKHO2','PLIN4','PLPP1','PLPPR5','PLS3','PLSCR1','PLVAP','PLXNA3','PLXNA4','PNCK','PNISR','PODXL','POLE','POU2F3','PPEF1','PPFIBP1','PPIP5K1','PPL','PPM1A','PPM1F','PPP1CA','PPP1CB','PPP1R12B','PPP1R16A','PPP1R16B','PPP1R18','PPP1R27','PPP1R9A','PPP2R1B','PPP3R1','PPP3R2','PPP4C','PPP6R3','PPTC7','PRAME','PRC1','PRDX6','PRIMA1','PRKCA','PRKCE','PRKCH','PRKCZ','PRKD1','PROM2','PROSER2','PRR15','PRR32','PRR7','PRRC2A','PRRG1','PRRT2','PRRT3','PRRT4','PRTG','PRX','PSCA','PSD','PSD2','PSEN1','PSMC5','PSMC6','PSMD9','PSTPIP2','PTAFR','PTCHD4','PTK2B','PTK6','PTPN12','PTPN21','PTPN22','PTPN4','PTPRB','PTPRG','PTPRK','PTPRM','PTPRR','PTPRS','PTX3','PURG','PVALB','PVR','PXDC1','PXK','PYGL','PYGO2','PYM1','RAB13','RAB23','RAB25','RAB26','RAB27B','RAB3B','RAB3GAP2','RAB43','RAB8A','RABEPK','RAD51C','RAD52','RAI2','RALA','RALGAPA2','RAPGEF2','RAPH1','RAPSN','RASA4','RASA4B','RASAL1','RASAL2','RASAL3','RASD1','RASD2','RASGRP1','RASGRP2','RASGRP4','RASSF2','RASSF3','RASSF4','RBM46','RC3H1','RCCD1','RCSD1','RD3L','RDX','RECK','RELL1','REPS1','RET','RFLNB','RFPL1','RFPL2','RFPL3','RGP1','RGS10','RGS14','RGS19','RGS3','RGS7BP','RGS9BP','RHCE','RHD','RHEX','RHOA','RHOBTB2','RHOJ','RHOXF2','RHOXF2B','RIC8A','RIC8B','RIF1','RILPL1','RIMBP3','RIMBP3B','RIMBP3C','RIMS2','RIOK2','RIOK3','RIPK1','RND1','RNF114','RNF145','RNF146','RNF149','RNF165','RNF166','RNF39','ROBO1','ROBO3','ROBO4','ROM1','RP2','RPE','RPEL1','RPGRIP1L','RPIA','RPL36A','RPL36A-HNRNPH2','RPL36AL','RPSA','RPUSD1','RRAD','RRP12','RRP7A','RSBN1L','RSC1A1','RSPH3','RTL3','RTL4','RTN4R','RTN4RL1','RYR2','S100A13','S100A14','S100A16','S100A2','S100A3','S100A4','S100A6','S100A9','SAMD4A','SAMHD1','SAMSN1','SAPCD1','SASH3','SC5D','SCAMP1','SCAMP4','SCEL','SCHIP1','SCIN','SCN1A','SCN5A','SCN8A','SCN9A','SCNN1D','SCNN1G','SCRIB','SCUBE3','SDC2','SDC4','SDE2','SDHB','SEC16B','SEMA3C','SEMA3D','SEMA4B','SEMA4F','SEMA6C','SEMA6D','SEPT2','SEPT3','SERINC2','SERINC5','SERPINB3','SFSWAP','SGCE','SGK494','SH3BGR','SH3D21','SHANK1','SHANK3','SHARPIN','SHISA6','SHKBP1','SHROOM2','SHTN1','SIGLEC10','SIGLEC14','SIGLEC6','SIGLEC7','SIN3B','SIPA1','SIPA1L1','SIPA1L3','SKAP1','SLAMF6','SLC10A5','SLC12A2','SLC13A3','SLC13A5','SLC14A1','SLC16A1','SLC16A10','SLC16A2','SLC16A3','SLC16A7','SLC16A9','SLC17A5','SLC19A1','SLC19A3','SLC1A5','SLC1A6','SLC22A16','SLC23A3','SLC24A4','SLC26A9','SLC27A1','SLC28A1','SLC2A1','SLC2A11','SLC2A12','SLC2A14','SLC2A2','SLC2A3','SLC30A1','SLC31A2','SLC34A1','SLC34A2','SLC35F5','SLC35G2','SLC36A4','SLC38A5','SLC38A6','SLC38A7','SLC39A10','SLC39A14','SLC39A2','SLC39A6','SLC3A2','SLC40A1','SLC41A3','SLC43A1','SLC43A3','SLC44A2','SLC45A4','SLC46A1','SLC46A3','SLC4A1AP','SLC4A2','SLC4A7','SLC4A8','SLC5A12','SLC5A3','SLC5A6','SLC5A7','SLC6A6','SLC7A14','SLC7A2','SLC7A5','SLC7A6','SLC8A1','SLC9A1','SLC9A2','SLC9A3','SLC9A3R1','SLC9A3R2','SLC9A4','SLC9A5','SLC9B2','SLC9C2','SLCO1A2','SLCO1B3','SLCO4A1','SLCO4C1','SLCO5A1','SLIT1','SLIT3','SLITRK6','SLK','SLX4','SMAGP','SMAP1','SMOC1','SMPD2','SMPX','SNAP23','SNAP25','SNRK','SNTB2','SNTG2','SNX15','SNX32','SNX9','SOCS3','SOCS5','SOD1','SORBS1','SORCS1','SORT1','SOX11','SPAG16','SPANXN4','SPART','SPATA12','SPATC1L','SPERT','SPG11','SPHK1','SPINK13','SPINT1','SPINT2','SPN','SPPL2B','SPPL3','SPRED3','SPRYD3','SPTBN2','SRC','SRCIN1','SRD5A3','SRPK1','SRPRB','SRRM3','SSFA2','SSH1','SSH3','SSTR1','SSTR5','SSX2IP','ST8SIA5','STAB1','STAC','STAC2','STAMBP','STAMBPL1','STARD4','STAT2','STEAP4','STIL','STIP1','STK10','STK16','STK17A','STK17B','STOM','STOML2','STOML3','STX18','STX4','STXBP4','STYK1','SUCLG1','SUCLG2','SUGT1','SUMF2','SUPT6H','SUSD2','SVIL','SWAP70','SYCP2','SYK','SYMPK','SYNC','SYNGR3','SYNPO2L','SYPL2','SYT4','SYTL2','SYVN1','TACC2','TACR2','TACSTD2','TASP1','TAZ','TBC1D10A','TBC1D2','TBC1D24','TBC1D30','TBC1D9B','TBXA2R','TCAF2','TCAF2C','TCHP','TCIM','TDG','TDO2','TDP1','TEC','TEDC2','TEK','TES','TEX14','TEX9','TGFBR1','TGFBR2','TGM1','TGM2','THBS1','THBS2','THEGL','THY1','TIAM1','TICAM1','TIGD7','TIMD4','TJP1','TJP2','TJP3','TLN1','TLR4','TM2D2','TM4SF20','TM4SF4','TM4SF5','TMC5','TMED8','TMEM101','TMEM107','TMEM108','TMEM121B','TMEM139','TMEM141','TMEM164','TMEM176B','TMEM183A','TMEM186','TMEM2','TMEM218','TMEM222','TMEM229A','TMEM245','TMEM251','TMEM254','TMEM258','TMEM266','TMEM268','TMEM50A','TMEM63B','TMPRSS15','TMPRSS2','TMTC2','TNFRSF10D','TNFRSF11A','TNFRSF12A','TNFRSF21','TNFRSF25','TNFRSF8','TNFRSF9','TNFSF13B','TNK1','TNKS1BP1','TNNI2','TOGARAM2','TOLLIP','TOM1','TP53BP2','TPD52L1','TPRN','TRAF7','TRAIP','TRAPPC13','TRAT1','TRDN','TREML1','TRIB1','TRIM16','TRIM16L','TRIM38','TRIM4','TRIM44','TRIM54','TRIM71','TRIML2','TRIP6','TRMT1','TRMT10B','TRMT13','TRMT6','TRO','TRPM1','TRPM4','TRPM7','TRPV2','TRPV3','TSG101','TSGA13','TSHZ3','TSPAN15','TSPAN6','TSPO2','TSR2','TSSK4','TTC17','TTC23','TTC7B','TTLL12','TTLL5','TULP3','TWF2','TXNDC11','TXNIP','TYROBP','TYW3','UAP1','UBA3','UBA52','UBAC1','UBALD1','UBAP1','UBASH3B','UBE2A','UBE2B','UBE2C','UBE2D1','UBE2D2','UBE2D3','UBE2D4','UBE2K','UBE2QL1','UBE2S','UBE3C','UBQLN2','UBR2','UBTD1','UCP3','ULBP1','ULBP2','UNC5A','UNC79','UQCC1','USP14','USP15','USP2','USP20','USP21','USP4','USP53','UTP20','UTRN','VAMP5','VAMP8','VANGL1','VASP','VCAM1','VCPIP1','VIL1','VPS13B','VPS36','VPS50','VPS8','VSIG4','VTCN1','VWA5B1','VWA7','WAS','WASF2','WBP4','WDFY3','WDPCP','WDR1','WDR12','WDR13','WDR53','WDR6','WDR66','WIPF1','WWP1','XIAP','XIRP1','XIRP2','XPC','XPNPEP2','XPO6','XRN1','XYLT1','YES1','YIPF1','YIPF4','YOD1','YTHDC1','ZBED3','ZBTB33','ZBTB8OS','ZDHHC2','ZDHHC20','ZDHHC22','ZDHHC5','ZFP41','ZFP90','ZFYVE28','ZGPAT','ZMYND10','ZNF10','ZNF18','ZNF185','ZNF2','ZNF20','ZNF227','ZNF229','ZNF236','ZNF276','ZNF277','ZNF337','ZNF397','ZNF407','ZNF497','ZNF503','ZNF510','ZNF536','ZNF571','ZNF598','ZNF599','ZNF646','ZNF655','ZNF688','ZNF706','ZNF717','ZNF783','ZNF79','ZNF839','ZSCAN22','ZSCAN29','ZSWIM1','ZSWIM8','ZYX']
MembraneProteins=MembraneProteins1+MembraneProteins2

TFs1=['AC008770.3','AC023509.3','AC092835.1','AC138696.1','ADNP','ADNP2','AEBP1','AEBP2','AHCTF1','AHDC1','AHR','AHRR','AIRE','AKAP8','AKAP8L','AKNA','ALX1','ALX3','ALX4','ANHX','ANKZF1','AR','ARGFX','ARHGAP35','ARID2','ARID3A','ARID3B','ARID3C','ARID5A','ARID5B','ARNT','ARNT2','ARNTL','ARNTL2','ARX','ASCL1','ASCL2','ASCL3','ASCL4','ASCL5','ASH1L','ATF1','ATF2','ATF3','ATF4','ATF5','ATF6','ATF6B','ATF7','ATMIN','ATOH1','ATOH7','ATOH8','BACH1','BACH2','BARHL1','BARHL2','BARX1','BARX2','BATF','BATF2','BATF3','BAZ2A','BAZ2B','BBX','BCL11A','BCL11B','BCL6','BCL6B','BHLHA15','BHLHA9','BHLHE22','BHLHE23','BHLHE40','BHLHE41','BNC1','BNC2','BORCS8-MEF2B','BPTF','BRF2','BSX','C11orf95','CAMTA1','CAMTA2','CARF','CASZ1','CBX2','CC2D1A','CCDC169-SOHLH2','CCDC17','CDC5L','CDX1','CDX2','CDX4','CEBPA','CEBPB','CEBPD','CEBPE','CEBPG','CEBPZ','CENPA','CENPB','CENPBD1','CENPS','CENPT','CENPX','CGGBP1','CHAMP1','CHCHD3','CIC','CLOCK','CPEB1','CPXCR1','CREB1','CREB3','CREB3L1','CREB3L2','CREB3L3','CREB3L4','CREB5','CREBL2','CREBZF','CREM','CRX','CSRNP1','CSRNP2','CSRNP3','CTCF','CTCFL','CUX1','CUX2','CXXC1','CXXC4','CXXC5','DACH1','DACH2','DBP','DBX1','DBX2','DDIT3','DEAF1','DLX1','DLX2','DLX3','DLX4','DLX5','DLX6','DMBX1','DMRT1','DMRT2','DMRT3','DMRTA1','DMRTA2','DMRTB1','DMRTC2','DMTF1','DNMT1','DNTTIP1','DOT1L','DPF1','DPF3','DPRX','DR1','DRAP1','DRGX','DUX1','DUX3','DUX4','DUXA','DZIP1','E2F1','E2F2','E2F3','E2F4','E2F5','E2F6','E2F7','E2F8','E4F1','EBF1','EBF2','EBF3','EBF4','EEA1','EGR1','EGR2','EGR3','EGR4','EHF','ELF1','ELF2','ELF3','ELF4','ELF5','ELK1','ELK3','ELK4','EMX1','EMX2','EN1','EN2','EOMES','EPAS1','ERF','ERG','ESR1','ESR2','ESRRA','ESRRB','ESRRG','ESX1','ETS1','ETS2','ETV1','ETV2','ETV3','ETV3L','ETV4','ETV5','ETV6','ETV7','EVX1','EVX2','FAM170A','FAM200B','FBXL19','FERD3L','FEV','FEZF1','FEZF2','FIGLA','FIZ1','FLI1','FLYWCH1','FOS','FOSB','FOSL1','FOSL2','FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXN4','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1','GABPA','GATA1','GATA2','GATA3','GATA4','GATA5','GATA6','GATAD2A','GATAD2B','GBX1','GBX2','GCM1','GCM2','GFI1','GFI1B','GLI1','GLI2','GLI3','GLI4','GLIS1','GLIS2','GLIS3','GLMP','GLYR1','GMEB1','GMEB2','GPBP1','GPBP1L1','GRHL1','GRHL2','GRHL3','GSC','GSC2','GSX1','GSX2','GTF2B','GTF2I','GTF2IRD1','GTF2IRD2','GTF2IRD2B','GTF3A','GZF1','HAND1','HAND2','HBP1','HDX','HELT','HES1','HES2','HES3','HES4','HES5','HES6','HES7','HESX1','HEY1','HEY2','HEYL','HHEX','HIC1','HIC2','HIF1A','HIF3A','HINFP','HIVEP1','HIVEP2','HIVEP3','HKR1','HLF','HLX','HMBOX1','HMG20A','HMG20B','HMGA1','HMGA2','HMGN3','HMX1','HMX2','HMX3','HNF1A','HNF1B','HNF4A','HNF4G','HOMEZ','HOXA1','HOXA10','HOXA11','HOXA13','HOXA2','HOXA3','HOXA4','HOXA5','HOXA6','HOXA7','HOXA9','HOXB1','HOXB13','HOXB2','HOXB3','HOXB4','HOXB5','HOXB6','HOXB7','HOXB8','HOXB9','HOXC10','HOXC11','HOXC12','HOXC13','HOXC4','HOXC5','HOXC6','HOXC8','HOXC9','HOXD1','HOXD10','HOXD11','HOXD12','HOXD13','HOXD3','HOXD4','HOXD8','HOXD9','HSF1','HSF2','HSF4','HSF5','HSFX1','HSFX2','HSFY1','HSFY2','IKZF1','IKZF2','IKZF3','IKZF4','IKZF5','INSM1','INSM2','IRF1','IRF2','IRF3','IRF4','IRF5','IRF6','IRF7','IRF8','IRF9','IRX1','IRX2','IRX3','IRX4','IRX5','IRX6','ISL1','ISL2','ISX','JAZF1','JDP2','JRK','JRKL','JUN','JUNB','JUND','KAT7','KCMF1','KCNIP3','KDM2A','KDM2B','KDM5B','KIN','KLF1','KLF10','KLF11','KLF12','KLF13','KLF14','KLF15','KLF16','KLF17','KLF2','KLF3','KLF4','KLF5','KLF6','KLF7','KLF8','KLF9','KMT2A','KMT2B','L3MBTL1','L3MBTL3','L3MBTL4','LBX1','LBX2','LCOR','LCORL','LEF1','LEUTX','LHX1','LHX2','LHX3','LHX4','LHX5','LHX6','LHX8','LHX9','LIN28A','LIN28B','LIN54','LMX1A','LMX1B','LTF','LYL1','MAF','MAFA','MAFB','MAFF','MAFG','MAFK','MAX','MAZ','MBD1','MBD2','MBD3','MBD4','MBD6','MBNL2','MECOM','MECP2','MEF2A','MEF2B','MEF2C','MEF2D','MEIS1','MEIS2','MEIS3','MEOX1','MEOX2','MESP1','MESP2','MGA','MITF','MIXL1','MKX','MLX','MLXIP','MLXIPL','MNT','MNX1','MSANTD1','MSANTD3','MSANTD4','MSC','MSGN1','MSX1','MSX2','MTERF1','MTERF2','MTERF3','MTERF4','MTF1','MTF2','MXD1','MXD3','MXD4','MXI1','MYB','MYBL1','MYBL2','MYC','MYCL','MYCN','MYF5','MYF6','MYNN','MYOD1','MYOG','MYPOP','MYRF','MYRFL','MYSM1','MYT1','MYT1L','MZF1','NACC2','NAIF1','NANOG','NANOGNB','NANOGP8','NCOA1','NCOA2','NCOA3','NEUROD1','NEUROD2','NEUROD4','NEUROD6','NEUROG1','NEUROG2','NEUROG3','NFAT5','NFATC1','NFATC2','NFATC3','NFATC4','NFE2','NFE2L1','NFE2L2','NFE2L3','NFE4','NFIA','NFIB','NFIC','NFIL3','NFIX','NFKB1','NFKB2','NFX1','NFXL1','NFYA','NFYB','NFYC','NHLH1','NHLH2','NKRF','NKX1-1','NKX1-2','NKX2-1','NKX2-2','NKX2-3','NKX2-4','NKX2-5','NKX2-6','NKX2-8','NKX3-1','NKX3-2','NKX6-1','NKX6-2','NKX6-3','NME2','NOBOX','NOTO','NPAS1','NPAS2','NPAS3','NPAS4','NR0B1','NR1D1','NR1D2','NR1H2','NR1H3','NR1H4','NR1I2','NR1I3','NR2C1','NR2C2','NR2E1','NR2E3','NR2F1','NR2F2','NR2F6','NR3C1','NR3C2','NR4A1','NR4A2','NR4A3','NR5A1','NR5A2','NR6A1','NRF1','NRL','OLIG1','OLIG2','OLIG3','ONECUT1','ONECUT2','ONECUT3','OSR1','OSR2','OTP','OTX1','OTX2','OVOL1','OVOL2','OVOL3','PA2G4','PATZ1','PAX1','PAX2','PAX3','PAX4','PAX5','PAX6','PAX7','PAX8','PAX9','PBX1','PBX2','PBX3','PBX4','PCGF2','PCGF6','PDX1','PEG3','PGR','PHF1','PHF19 ','PHF20','PHF21A','PHOX2A','PHOX2B','PIN1','PITX1','PITX2','PITX3','PKNOX1','PKNOX2','PLAG1','PLAGL1','PLAGL2','PLSCR1','POGK','POU1F1','POU2AF1','POU2F1','POU2F2','POU2F3','POU3F1','POU3F2','POU3F3','POU3F4','POU4F1','POU4F2','POU4F3','POU5F1','POU5F1B','POU5F2','POU6F1','POU6F2','PPARA','PPARD','PPARG','PRDM1','PRDM10','PRDM12','PRDM13','PRDM14','PRDM15','PRDM16','PRDM2','PRDM4','PRDM5','PRDM6','PRDM8','PRDM9','PREB','PRMT3','PROP1','PROX1','PROX2','PRR12','PRRX1','PRRX2','PTF1A','PURA','PURB','PURG','RAG1','RARA','RARB','RARG','RAX','RAX2','RBAK','RBCK1','RBPJ','RBPJL','RBSN','REL','RELA','RELB','REPIN1','REST','REXO4','RFX1','RFX2','RFX3','RFX4','RFX5','RFX6','RFX7','RFX8','RHOXF1','RHOXF2','RHOXF2B','RLF','RORA','RORB','RORC','RREB1','RUNX1','RUNX2','RUNX3','RXRA','RXRB','RXRG','SAFB','SAFB2','SALL1','SALL2','SALL3','SALL4','SATB1','SATB2','SCMH1','SCML4','SCRT1','SCRT2','SCX','SEBOX','SETBP1','SETDB1','SETDB2','SGSM2','SHOX','SHOX2','SIM1','SIM2','SIX1','SIX2','SIX3','SIX4','SIX5','SIX6','SKI','SKIL','SKOR1','SKOR2','SLC2A4RG','SMAD1','SMAD3','SMAD4','SMAD5','SMAD9','SMYD3','SNAI1','SNAI2','SNAI3','SNAPC2','SNAPC4','SNAPC5','SOHLH1','SOHLH2','SON','SOX1','SOX10','SOX11']
TFs2=['SOX12','SOX13','SOX14','SOX15','SOX17','SOX18','SOX2','SOX21','SOX3','SOX30','SOX4','SOX5','SOX6','SOX7','SOX8','SOX9','SP1','SP100','SP110','SP140','SP140L','SP2','SP3','SP4','SP5','SP6','SP7','SP8','SP9','SPDEF','SPEN','SPI1','SPIB','SPIC','SPZ1','SRCAP','SREBF1','SREBF2','SRF','SRY','ST18','STAT1','STAT2','STAT3','STAT4','STAT5A','STAT5B','STAT6','T','TAL1','TAL2','TBP','TBPL1','TBPL2','TBR1','TBX1','TBX10','TBX15','TBX18','TBX19','TBX2','TBX20','TBX21','TBX22','TBX3','TBX4','TBX5','TBX6','TCF12','TCF15','TCF20','TCF21','TCF23','TCF24','TCF3','TCF4','TCF7','TCF7L1','TCF7L2','TCFL5','TEAD1','TEAD2','TEAD3','TEAD4','TEF','TERB1','TERF1','TERF2','TET1','TET2','TET3','TFAP2A','TFAP2B','TFAP2C','TFAP2D','TFAP2E','TFAP4','TFCP2','TFCP2L1','TFDP1','TFDP2','TFDP3','TFE3','TFEB','TFEC','TGIF1','TGIF2','TGIF2LX','TGIF2LY','THAP1','THAP10','THAP11','THAP12','THAP2','THAP3','THAP4','THAP5','THAP6','THAP7','THAP8','THAP9','THRA','THRB','THYN1','TIGD1','TIGD2','TIGD3','TIGD4','TIGD5','TIGD6','TIGD7','TLX1','TLX2','TLX3','TMF1','TOPORS','TP53','TP63','TP73','TPRX1','TRAFD1','TRERF1','TRPS1','TSC22D1','TSHZ1','TSHZ2','TSHZ3','TTF1','TWIST1','TWIST2','UBP1','UNCX','USF1','USF2','USF3','VAX1','VAX2','VDR','VENTX','VEZF1','VSX1','VSX2','WIZ','WT1','XBP1','XPA','YBX1','YBX2','YBX3','YY1','YY2','ZBED1','ZBED2','ZBED3','ZBED4','ZBED5','ZBED6','ZBED9','ZBTB1','ZBTB10','ZBTB11','ZBTB12','ZBTB14','ZBTB16','ZBTB17','ZBTB18','ZBTB2','ZBTB20','ZBTB21','ZBTB22','ZBTB24','ZBTB25','ZBTB26','ZBTB3','ZBTB32','ZBTB33','ZBTB34','ZBTB37','ZBTB38','ZBTB39','ZBTB4','ZBTB40','ZBTB41','ZBTB42','ZBTB43','ZBTB44','ZBTB45','ZBTB46','ZBTB47','ZBTB48','ZBTB49','ZBTB5','ZBTB6','ZBTB7A','ZBTB7B','ZBTB7C','ZBTB8A','ZBTB8B','ZBTB9','ZC3H8','ZEB1','ZEB2','ZFAT','ZFHX2','ZFHX3','ZFHX4','ZFP1','ZFP14','ZFP2','ZFP28','ZFP3','ZFP30','ZFP37','ZFP41','ZFP42','ZFP57','ZFP62','ZFP64','ZFP69','ZFP69B','ZFP82','ZFP90','ZFP91','ZFP92','ZFPM1','ZFPM2','ZFX','ZFY','ZGLP1','ZGPAT','ZHX1','ZHX2','ZHX3','ZIC1','ZIC2','ZIC3','ZIC4','ZIC5','ZIK1','ZIM2','ZIM3','ZKSCAN1','ZKSCAN2','ZKSCAN3','ZKSCAN4','ZKSCAN5','ZKSCAN7','ZKSCAN8','ZMAT1','ZMAT4','ZNF10','ZNF100','ZNF101','ZNF107','ZNF112','ZNF114','ZNF117','ZNF12','ZNF121','ZNF124','ZNF131','ZNF132','ZNF133','ZNF134','ZNF135','ZNF136','ZNF138','ZNF14','ZNF140','ZNF141','ZNF142','ZNF143','ZNF146','ZNF148','ZNF154','ZNF155','ZNF157','ZNF16','ZNF160','ZNF165','ZNF169','ZNF17','ZNF174','ZNF175','ZNF177','ZNF18','ZNF180','ZNF181','ZNF182','ZNF184','ZNF189','ZNF19','ZNF195','ZNF197','ZNF2','ZNF20','ZNF200','ZNF202','ZNF205','ZNF207','ZNF208','ZNF211','ZNF212','ZNF213','ZNF214','ZNF215','ZNF217','ZNF219','ZNF22','ZNF221','ZNF222','ZNF223','ZNF224','ZNF225','ZNF226','ZNF227','ZNF229','ZNF23','ZNF230','ZNF232','ZNF233','ZNF234','ZNF235','ZNF236','ZNF239','ZNF24','ZNF248','ZNF25','ZNF250','ZNF251','ZNF253','ZNF254','ZNF256','ZNF257','ZNF26','ZNF260','ZNF263','ZNF264','ZNF266','ZNF267','ZNF268','ZNF273','ZNF274','ZNF275','ZNF276','ZNF277','ZNF28','ZNF280A','ZNF280B','ZNF280C','ZNF280D','ZNF281','ZNF282','ZNF283','ZNF284','ZNF285','ZNF286A','ZNF286B','ZNF287','ZNF292','ZNF296','ZNF3','ZNF30','ZNF300','ZNF302','ZNF304','ZNF311','ZNF316','ZNF317','ZNF318','ZNF319','ZNF32','ZNF320','ZNF322','ZNF324','ZNF324B','ZNF326','ZNF329','ZNF331','ZNF333','ZNF334','ZNF335','ZNF337','ZNF33A','ZNF33B','ZNF34','ZNF341','ZNF343','ZNF345','ZNF346','ZNF347','ZNF35','ZNF350','ZNF354A','ZNF354B','ZNF354C','ZNF358','ZNF362','ZNF365','ZNF366','ZNF367','ZNF37A','ZNF382','ZNF383','ZNF384','ZNF385A','ZNF385B','ZNF385C','ZNF385D','ZNF391','ZNF394','ZNF395','ZNF396','ZNF397','ZNF398','ZNF404','ZNF407','ZNF408','ZNF41','ZNF410','ZNF414','ZNF415','ZNF416','ZNF417','ZNF418','ZNF419','ZNF420','ZNF423','ZNF425','ZNF426','ZNF428','ZNF429','ZNF43','ZNF430','ZNF431','ZNF432','ZNF433','ZNF436','ZNF438','ZNF439','ZNF44','ZNF440','ZNF441','ZNF442','ZNF443','ZNF444','ZNF445','ZNF446','ZNF449','ZNF45','ZNF451','ZNF454','ZNF460','ZNF461','ZNF462','ZNF467','ZNF468','ZNF469','ZNF470','ZNF471','ZNF473','ZNF474','ZNF479','ZNF48','ZNF480','ZNF483','ZNF484','ZNF485','ZNF486','ZNF487','ZNF488','ZNF490','ZNF491','ZNF492','ZNF493','ZNF496','ZNF497','ZNF500','ZNF501','ZNF502','ZNF503','ZNF506','ZNF507','ZNF510','ZNF511','ZNF512','ZNF512B','ZNF513','ZNF514','ZNF516','ZNF517','ZNF518A','ZNF518B','ZNF519','ZNF521','ZNF524','ZNF525','ZNF526','ZNF527','ZNF528','ZNF529','ZNF530','ZNF532','ZNF534','ZNF536','ZNF540','ZNF541','ZNF543','ZNF544','ZNF546','ZNF547','ZNF548','ZNF549','ZNF550','ZNF551','ZNF552','ZNF554','ZNF555','ZNF556','ZNF557','ZNF558','ZNF559','ZNF560','ZNF561','ZNF562','ZNF563','ZNF564','ZNF565','ZNF566','ZNF567','ZNF568','ZNF569','ZNF57','ZNF570','ZNF571','ZNF572','ZNF573','ZNF574','ZNF575','ZNF576','ZNF577','ZNF578','ZNF579','ZNF580','ZNF581','ZNF582','ZNF583','ZNF584','ZNF585A','ZNF585B','ZNF586','ZNF587','ZNF587B','ZNF589','ZNF592','ZNF594','ZNF595','ZNF596','ZNF597','ZNF598','ZNF599','ZNF600','ZNF605','ZNF606','ZNF607','ZNF608','ZNF609','ZNF610','ZNF611','ZNF613','ZNF614','ZNF615','ZNF616','ZNF618','ZNF619','ZNF620','ZNF621','ZNF623','ZNF624','ZNF625','ZNF626','ZNF627','ZNF628','ZNF629','ZNF630','ZNF639','ZNF641','ZNF644','ZNF645','ZNF646','ZNF648','ZNF649','ZNF652','ZNF653','ZNF654','ZNF655','ZNF658','ZNF66','ZNF660','ZNF662','ZNF664','ZNF665','ZNF667','ZNF668','ZNF669','ZNF670','ZNF671','ZNF672','ZNF674','ZNF675','ZNF676','ZNF677','ZNF678','ZNF679','ZNF680','ZNF681','ZNF682','ZNF683','ZNF684','ZNF687','ZNF688','ZNF689','ZNF69','ZNF691','ZNF692','ZNF695','ZNF696','ZNF697','ZNF699','ZNF7','ZNF70','ZNF700','ZNF701','ZNF703','ZNF704','ZNF705A','ZNF705B','ZNF705D','ZNF705E','ZNF705G','ZNF706','ZNF707','ZNF708','ZNF709','ZNF71','ZNF710','ZNF711','ZNF713','ZNF714','ZNF716','ZNF717','ZNF718','ZNF721','ZNF724','ZNF726','ZNF727','ZNF728','ZNF729','ZNF730','ZNF732','ZNF735','ZNF736','ZNF737','ZNF74','ZNF740','ZNF746','ZNF747','ZNF749','ZNF750','ZNF75A','ZNF75D','ZNF76','ZNF761','ZNF763','ZNF764','ZNF765','ZNF766','ZNF768','ZNF77','ZNF770','ZNF771','ZNF772','ZNF773','ZNF774','ZNF775','ZNF776','ZNF777','ZNF778','ZNF780A','ZNF780B','ZNF781','ZNF782','ZNF783','ZNF784','ZNF785','ZNF786','ZNF787','ZNF788','ZNF789','ZNF79','ZNF790','ZNF791','ZNF792','ZNF793','ZNF799','ZNF8','ZNF80','ZNF800','ZNF804A','ZNF804B','ZNF805','ZNF808','ZNF81','ZNF813','ZNF814','ZNF816','ZNF821','ZNF823','ZNF827','ZNF829','ZNF83','ZNF830','ZNF831','ZNF835','ZNF836','ZNF837','ZNF84','ZNF841','ZNF843','ZNF844','ZNF845','ZNF846','ZNF85','ZNF850','ZNF852','ZNF853','ZNF860','ZNF865','ZNF878','ZNF879','ZNF880','ZNF883','ZNF888','ZNF891','ZNF90','ZNF91','ZNF92','ZNF93','ZNF98','ZNF99','ZSCAN1','ZSCAN10','ZSCAN12','ZSCAN16','ZSCAN18','ZSCAN2','ZSCAN20','ZSCAN21','ZSCAN22','ZSCAN23','ZSCAN25','ZSCAN26','ZSCAN29','ZSCAN30','ZSCAN31','ZSCAN32','ZSCAN4','ZSCAN5A','ZSCAN5B','ZSCAN5C','ZSCAN9','ZUFSP','ZXDA','ZXDB','ZXDC','ZZZ3']
TFs=TFs1+TFs2

#TFs other, inclusive of the above list, not necessarily DNA binding proteins but affect transcription
TFs1Other=['AATF','ABL1','ABRA','AC008770.3','AC020915.1','AC023509.3','AC092835.1','AC105001.2','AC138696.1','ACOD1','ACTR3B','ACTR3C','ADAM8','ADAMTS17','ADAMTS19','ADAR','ADGRA1','ADGRG3','ADNP','ADNP2','AEBP1','AEBP2','AES','AFAP1 ','AFF1','AFF3','AFF4','AGER','AGT','AHCTF1','AHDC1','AHR','AHRR','AIM2','AIRE','AKAP8','AKAP8L','AKNA','AKT1','ALK','ALKBH2','ALX1','ALX3','ALX4','AMH','ANAPC2','ANHX','ANKRD30A','ANKZF1','ANXA3','ANXA4','AP5Z1','APBB3','APC2','APEX1','APOA4','APOBEC3A','APOBEC3B','APOBEC3G','APTX','AR','ARGFX','ARHGAP35','ARHGEF2','ARHGEF5','ARID1A','ARID1B','ARID2','ARID3A','ARID3B','ARID3C','ARID4A','ARID4B','ARID5A','ARID5B','ARNT','ARNT2','ARNTL','ARNTL2','ARRB1','ARRB2','ARX','ASCC1','ASCL1','ASCL2','ASCL3','ASCL4','ASCL5','ASH1L','ATF1','ATF2','ATF3','ATF4','ATF5','ATF6','ATF6B','ATF7','ATF7IP','ATM','ATMIN','ATOH1','ATOH7','ATOH8','ATR','ATRX','ATXN7','BACH1','BACH2','BAG1','BANF1','BANP','BARHL1','BARHL2','BARX1','BARX2','BATF','BATF2','BATF3','BAZ2A','BAZ2B','BBX','BCL10','BCL11A','BCL11B','BCL3','BCL6','BCL6B','BCLAF1','BCOR','BDP1','BEND4','BEX1','BHLHA15','BHLHA9','BHLHE22','BHLHE23','BHLHE40','BHLHE41','BHMG1','BIN1','BLM','BLZF1','BMI1','BMP2','BMP4','BMP7','BMPR1A','BNC1','BNC2','BOD1L1','BOLA1','BOLA2','BOLA2B','BOLA3','BORCS8-MEF2B','BPNT1','BPTF','BRCA1','BRD4','BRD7','BRD8','BRD9','BRF1','BRF2','BRMS1','BRPF1','BSX','BTAF1','BTBD8','BTG2','BTK','BTRC','BUD31','C11orf95','C14orf39','C17orf49','C1D','C20orf194','C3orf33','CACTIN','CAMK1D','CAMK2A','CAMTA1','CAMTA2','CAPN15','CAPN3','CARD11','CARD14','CARF','CARHSP1','CASP8AP2','CASZ1','CAT','CBFA2T2','CBFA2T3','CBFB','CBL','CBLL1','CBX2','CC2D1A','CC2D1B','CCDC124','CCDC169-SOHLH2','CCDC17','CCDC3','CCDC71','CCDC83','CCNT1','CCNT2','CD36','CD40','CD40LG','CDC5L','CDC6','CDK5RAP3','CDK7','CDK8','CDK9','CDKN1A','CDKN2A','CDX1','CDX2','CDX4','CEBPA','CEBPB','CEBPD','CEBPE','CEBPG','CEBPZ','CELF3','CENPA','CENPB','CENPBD1','CENPS','CENPT','CENPX','CERS2','CERS3','CERS4','CERS5','CERS6','CFLAR','CGGBP1','CHAMP1','CHCHD3','CHD1','CHD2','CHD3','CHD4','CHD6','CHD7','CHD9','CHP1','CHRAC1','CHUK','CHURC1','CIB1','CIC','CIITA','CIR1','CITED1','CITED2','CITED4','CIZ1','CLOCK','CLU','CMKLR1','CNBP','CNOT3','CNOT4','CNOT7','CNOT8','COMMD1','COMMD6','COMMD7','COPS2','COPS3','COPS4','CPEB1','CPSF4','CPSF4L','CPXCR1','CRAMP1','CREB1','CREB3','CREB3L1','CREB3L2','CREB3L3','CREB3L4','CREB5','CREBBP','CREBL2','CREBRF','CREBZF','CREG1','CREM','CRTC1','CRTC2','CRTC3','CRX','CSDC2','CSDE1','CSRNP1','CSRNP2','CSRNP3','CTBP1','CTBP2','CTCF','CTCFL','CTH','CTNNB1','CTNNBIP1','CUL1','CUL2','CUL3','CUL4A','CUL4B','CUL5','CUX1','CUX2','CXXC1','CXXC4','CXXC5','CYLD','CYP1B1','CYTL1','DAB2IP','DACH1','DACH2','DAP','DAXX','DBP','DBX1','DBX2','DCP1A','DDB1','DDB2','DDIT3','DDN','DDR2','DDRGK1','DDX3X','DDX5','DDX58','DEAF1','DEK','DEPDC1','DEPDC1B','DEPDC4','DEPDC5','DEPDC7','DEPTOR','DHX34','DHX36','DHX57','DISP3','DLX1','DLX2','DLX3','DLX4','DLX5','DLX6','DMAP1','DMBX1','DMC1','DMRT1','DMRT2','DMRT3','DMRTA1','DMRTA2','DMRTB1','DMRTC1','DMRTC2','DMTF1','DNA2','DNAJA3','DNAJC1','DNAJC2','DNAJC21','DNMT1','DNMT3A','DNMT3B','DNMT3L','DNTT','DNTTIP1','DOT1L','DPF1','DPF2','DPF3','DPRX','DR1','DRAP1','DRGX','DSP','DTX1','DUS3L','DUSP12','DUX1','DUX3','DUX4','DUXA','DVL1','DVL2','DVL3','DZIP1','DZIP1L','E2F1','E2F2','E2F3','E2F4','E2F5','E2F6','E2F7','E2F8','E4F1','EAF2','EBF1','EBF2','EBF3','EBF4','EDA','EDA2R','EDF1','EDN1','EEA1','EED','EGLN1','EGR1','EGR2','EGR3','EGR4','EHF','EID1','EID2','EIF2AK2','EIF2AK4','EIF3C','EIF3CL','EIF3K','ELF1','ELF2','ELF3','ELF4','ELF5','ELK1','ELK3','ELK4','ELMSAN1','EME1','EMX1','EMX2','EN1','EN2','ENO1','EOMES','EP300','EP400','EPAS1','EPHA5','EPM2AIP1','ERBIN','ERC1','ERCC3','ERCC4','ERF','ERG','ESR1','ESR2','ESRRA','ESRRB','ESRRG','ESX1','ETS1','ETS2','ETV1','ETV2','ETV3','ETV3L','ETV4','ETV5','ETV6','ETV7','EVX1','EVX2','EWSR1','EXO1','EXOC2','EXOG','EZH1','EZH2','FAAP24','FAM153C','FAM170A','FAM171B','FAM200A','FAM200B','FAN1','FANCA','FANCD2','FANK1','FARSA','FARSB','FBN1','FBXL19','FBXO41','FEN1','FER','FERD3L','FEV','FEZF1','FEZF2','FGD1','FHL2','FIGLA','FIZ1','FLI1','FLNA','FLYWCH1','FMNL2','FOS','FOSB','FOSL1','FOSL2','FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L2','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXN4','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1','FUBP1','FUBP3','FUS','FZD1','FZD2','FZD4','FZD6','GABPA','GABPB1','GABPB2','GADD45A','GAS6','GAS7','GATA1','GATA2','GATA3','GATA4','GATA5','GATA6','GATAD1','GATAD2A','GATAD2B','GBX1','GBX2','GCFC2','GCM1','GCM2','GEN1','GFI1','GFI1B','GLA','GLI1','GLI2','GLI3','GLI4','GLIS1','GLIS2','GLIS3','GLMP','GLYR1','GMEB1','GMEB2','GOLGB1','GON4L','GPATCH8','GPBP1','GPBP1L1','GPER1','GPR155','GREM1','GRHL1','GRHL2','GRHL3','GRM6','GSC','GSC2','GSX1','GSX2','GTF2A1','GTF2A1L','GTF2A2','GTF2B','GTF2E2','GTF2F1','GTF2F2','GTF2H1','GTF2H2','GTF2H2C','GTF2H3','GTF2H4','GTF2I','GTF2IRD1','GTF2IRD2','GTF2IRD2B','GTF3A','GTF3C1','GTF3C2','GULP1','GZF1','H1F0','H1FOO','H1FX','HAND1','HAND2','HAVCR2','HBP1','HCFC1','HCK','HDAC1','HDAC10','HDAC2','HDAC3','HDAC4','HDAC5','HDAC6','HDAC7','HDAC8','HDAC9','HDX','HELT','HELZ','HELZ2','HES1','HES2','HES3','HES4','HES5','HES6','HES7','HESX1','HEY1','HEY2','HEYL','HHEX','HIC1','HIC2','HIF1A','HIF3A','HINFP','HIPK2','HIRA','HIST1H1A','HIST1H1B','HIST1H1C','HIST1H1D','HIST1H1E','HIST1H1T','HIVEP1','HIVEP2','HIVEP3','HKR1','HLA-DQB1','HLA-DQB2','HLA-DRB1','HLA-DRB3','HLA-DRB5','HLF','HLTF','HLX','HMBOX1','HMG20A','HMG20B','HMGA1','HMGA2','HMGB1','HMGB2','HMGB3','HMGB4','HMGN1','HMGN3','HMGXB3','HMGXB4','HMOX1','HMX1','HMX2','HMX3','HNF1A','HNF1B','HNF4A','HNF4G','HNRNPA1','HNRNPA3','HNRNPAB','HNRNPCL2','HNRNPD','HNRNPDL','HNRNPK','HNRNPL','HNRNPU','HOMEZ','HOOK2','HOPX','HOXA1','HOXA10','HOXA11','HOXA13','HOXA2','HOXA3','HOXA4','HOXA5','HOXA6','HOXA7','HOXA9','HOXB1','HOXB13','HOXB2','HOXB3','HOXB4','HOXB5','HOXB6','HOXB7','HOXB8','HOXB9','HOXC10','HOXC11','HOXC12','HOXC13','HOXC4','HOXC5','HOXC6','HOXC8','HOXC9','HOXD1','HOXD10','HOXD11','HOXD12','HOXD13','HOXD3','HOXD4','HOXD8','HOXD9','HP1BP3','HR','HSBP1','HSF1','HSF2','HSF4','HSF5','HSFX1','HSFX2','HSFY1','HSFY2','HSPA1A','HSPA1B','ICAM1','ID1','ID2','ID3','ID4','IFI16','IFNB1','IGHM','IKBKB','IKBKG','IKZF1','IKZF2','IKZF3','IKZF4','IKZF5','IL10','IL1B','IL1RAP','IL4','IL5','IL6','INF2','ING4','INS','INSM1','INSM2','IQGAP1','IRAK1','IRAK2','IRAK3','IRF1','IRF2','IRF2BP1','IRF3','IRF4','IRF5','IRF6','IRF7','IRF8','IRF9','IRX1','IRX2','IRX3','IRX4','IRX5','IRX6','ISL1','ISL2','ISX','ITCH','ITGB2','ITGB3BP','IZUMO2','JADE1','JAK2','JARID2','JAZF1','JDP2','JMY','JRK','JRKL','JUN','JUNB','JUND','JUP','KANSL2','KAT2A','KAT2B','KAT5','KAT6A','KAT6B','KAT7','KAT8','KCMF1','KCNIP3','KDM1A','KDM2A','KDM2B','KDM3A','KDM5A','KDM5B','KDM5C','KDM5D','KEAP1','KIAA1549','KIAA1683','KIAA1841','KIAA1958','KIF15','KIN','KIT','KLF1','KLF10','KLF11','KLF12','KLF13','KLF14','KLF15','KLF16','KLF17','KLF2','KLF3','KLF4','KLF5','KLF6','KLF7','KLF8','KLF9','KMT2A','KMT2B','KMT2C','KMT2D','KNTC1','KRAS','KRBOX1','KRT13','KRTAP5-1','KRTAP5-9','KRTAP9-9','L3MBTL1','L3MBTL3','L3MBTL4','LARP1','LARP1B','LARP4','LARP4B','LARP6','LARP7','LBX1','LBX2','LCOR','LCORL','LEF1','LEUTX','LGALS9','LGR4','LHX1','LHX2','LHX3','LHX4','LHX5','LHX6','LHX8','LHX9','LIG1','LIG3','LIN28A','LIN28B','LIN54','LMO2','LMO4','LMX1A','LMX1B','LPP','LRP5','LRP6','LRP8','LRRFIP1','LRRFIP2','LSR','LTF','LTN1','LYAR','LYL1','LZTFL1','LZTR1','LZTS1','MACC1','MACF1','MAD2L2','MAEL','MAF','MAFA','MAFB','MAFF','MAFG','MAFK','MALT1','MAML1','MAML3','MAP2K5','MAP3K10','MAP3K13','MAP3K7','MAPK1','MAPK10','MAPK11','MAPK14','MAPK3','MAPK8','MAPK9','MATR3','MAVS','MAX','MAZ','MB21D1','MBD1','MBD2','MBD3','MBD4','MBD6','MBNL1','MBNL2','MBNL3','MBTPS2','MDFI','MDM2','MDM4','MECOM','MECP2','MED1','MED12','MED13','MED13L','MED14','MED15','MED16','MED17','MED21','MED23','MED24','MED6','MEF2A','MEF2B','MEF2C','MEF2D','MEIS1','MEIS2','MEIS3','MEN1','MEOX1','MEOX2','MESP1','MESP2','MET','MGA','MGMT','MID2','MIER1','MIER2','MIER3','MINK1','MIS18BP1','MITF','MIXL1','MKL1','MKL2','MKRN1','MKRN2','MKRN3','MKX','MLLT1','MLLT10','MLLT11','MLLT3','MLLT6','MLX','MLXIP','MLXIPL','MN1','MNAT1','MNT','MNX1','MORC3','MPG','MRPL28','MRRF','MSANTD1','MSANTD3','MSANTD4','MSC','MSGN1','MSH2','MSH3','MSH6','MSL3','MSRB2','MST1R','MSX1','MSX2','MTA1','MTA2','MTA3','MTDH','MTERF1','MTERF2','MTERF3','MTERF4','MTF1','MTF2','MTPN','MUS81','MXD1','MXD3','MXD4','MXI1','MYB','MYBBP1A','MYBL1','MYBL2','MYC','MYCBP','MYCL','MYCN','MYD88','MYEF2','MYF5','MYF6','MYNN','MYOCD','MYOD1','MYOG','MYPOP','MYRF','MYRFL','MYSM1','MYT1','MYT1L','MZF1','NAB2','NABP2','NACC2','NAIF1','NANOG','NANOGNB','NANOGP8','NCOA1','NCOA2','NCOA3','NCOA4','NCOA5','NCOA6','NCOA7','NCOR1','NCOR2','NDC80','NDN','NDP','NEB','NEIL1','NEUROD1','NEUROD2','NEUROD4','NEUROD6','NEUROG1','NEUROG2','NEUROG3','NFAM1','NFAT5','NFATC1','NFATC2','NFATC3','NFATC4','NFE2','NFE2L1','NFE2L2','NFE2L3','NFE4','NFIA','NFIB','NFIC','NFIL3','NFIX','NFKB1','NFKB2','NFKBIA','NFKBIB','NFKBID','NFKBIL1','NFKBIZ','NFRKB','NFX1','NFXL1','NFYA','NFYB','NFYC','NHLH1','NHLH2','NKRF','NKX1-1','NKX1-2','NKX2-1','NKX2-2','NKX2-3','NKX2-4','NKX2-5','NKX2-6','NKX2-8','NKX3-1','NKX3-2','NKX6-1','NKX6-2','NKX6-3','NLRC3','NLRC4','NLRC5','NLRP12','NLRP2B','NLRP3','NME2','NOBOX','NOC3L','NOC4L','NOCT','NOD1','NOD2','NODAL','NONO','NOP2','NOTCH1','NOTCH2','NOTCH3','NOTO','NPAS1','NPAS2','NPAS3','NPAS4','NPM1','NR0B1','NR0B2','NR1D1','NR1D2','NR1H2','NR1H3','NR1H4','NR1I2','NR1I3','NR2C1','NR2C2','NR2E1','NR2E3','NR2F1','NR2F2','NR2F6','NR3C1','NR3C2','NR4A1','NR4A2','NR4A3','NR5A1','NR5A2','NR6A1','NRBF2','NRF1','NRG1','NRIP1','NRIP3','NRK','NRL','NSD1','NSD2','NTRK1','NUCKS1','NUFIP1','NUP98','NUPL2']
TFs2Other=['OGG1','OLIG1','OLIG2','OLIG3','ONECUT1','ONECUT2','ONECUT3','OPRD1','OSR1','OSR2','OTOP3','OTP','OTULIN','OTX1','OTX2','OVOL1','OVOL2','OVOL3','PA2G4','PABPN1','PARK7','PARP1','PARP10','PARP12','PATZ1','PAWR','PAX1','PAX2','PAX3','PAX4','PAX5','PAX6','PAX7','PAX8','PAX9','PAXBP1','PAXIP1','PBRM1','PBX1','PBX2','PBX3','PBX4','PCBD1','PCBP1','PCBP2','PCBP3','PCGF2','PCGF6','PCNA','PCSK6','PDCD7','PDS5B','PDX1','PEG3','PELI1','PER1','PER2','PER3','PEX14','PFDN1','PGBD1','PGR','PGS1','PHB','PHB2','PHF1','PHF19 ','PHF20','PHF21A','PHF5A','PHOX2A','PHOX2B','PHTF1','PHTF2','PIAS1','PIAS2','PIAS3','PIAS4','PIDD1','PIKFYVE','PIM1','PIN1','PINK1','PITX1','PITX2','PITX3','PKHD1','PKHD1L1','PKNOX1','PKNOX2','PLA2G10','PLA2G1B','PLAG1','PLAGL1','PLAGL2','PLEK','PLEK2','PLEKHA4','PLPP3','PLSCR1','PLXNA1','PLXNA2','PLXNA3','PLXNA4','PLXNB1','PLXNB2','PLXNB3','PLXNC1','PLXND1','PML','PMS1','PNKP','POGK','POGZ','POLA1','POLB','POLE3','POLE4','POLG','POLH','POLI','POLK','POLL','POLM','POLN','POLQ','POLR1B','POLR2A','POLR2B','POLR2E','POLR2J','POLR2K','POLR2L','POLR3A','POLRMT','POT1','POU1F1','POU2AF1','POU2F1','POU2F2','POU2F3','POU3F1','POU3F2','POU3F3','POU3F4','POU4F1','POU4F2','POU4F3','POU5F1','POU5F1B','POU5F2','POU6F1','POU6F2','PPARA','PPARD','PPARG','PPARGC1A','PPARGC1B','PPP1R10','PPP1R13L','PPP1R21','PPP2R3B','PPP2R5B','PPP3CA','PPRC1','PQBP1','PRAM1','PRAP1','PRB3','PRDM1','PRDM10','PRDM11','PRDM12','PRDM13','PRDM14','PRDM15','PRDM16','PRDM2','PRDM4','PRDM5','PRDM6','PRDM7','PRDM8','PRDM9','PRDX3','PREB','PREX1','PREX2','PRH2','PRIM2','PRIMPOL','PRKCB','PRKCH','PRKCI','PRKCQ','PRKCZ','PRKD1','PRKD2','PRKN','PRMT2','PRMT3','PRNP','PROP1','PROX1','PROX2','PRR12','PRR3','PRRX1','PRRX2','PSMA6','PSMC5','PSMD10','PSMD11','PSMD12','PSPC1','PTCH1','PTEN','PTF1A','PTGIS','PTH','PTTG1','PUM3','PURA','PURB','PURG','PYCARD','PYDC1','PYDC2','QRICH1','RAB11FIP4','RAB7B','RAD21','RAD51','RAG1','RAI1','RAPGEF3','RAPGEF4','RAPGEF5','RARA','RARB','RARG','RAX','RAX2','RB1','RBAK','RBBP7','RBCK1','RBL1','RBL2','RBM10','RBM20','RBM22','RBM26','RBM27','RBM5','RBM6','RBPJ','RBPJL','RBSN','RC3H1','RC3H2','RCAN1','RCOR1','RCOR2','RCOR3','RECQL','REL','RELA','RELB','RELL2','RELN','REPIN1','RERE','REST','REV1','REXO4','RFC1','RFX1','RFX2','RFX3','RFX4','RFX5','RFX6','RFX7','RFX8','RFXANK','RFXAP','RGCC','RGS11','RGS6','RGS7','RGS9','RHEBL1','RHOXF1','RHOXF2','RHOXF2B','RING1','RIOK2','RIPK1','RIPK2','RIPK3','RIPK4','RLF','RLIM','RNASE2','RNASEH1','RNF113A','RNF113B','RNF114','RNF125','RNF138','RNF14','RNF166','RNF2','RNF219','RNF25','RNF31','RNF4','RNF41','RORA','RORB','RORC','RPA2','RPA4','RPL36A','RPL7','RPL7A','RPS27A','RPS6KA4','RPS6KA5','RREB1','RRN3','RUNX1','RUNX1T1','RUNX2','RUNX3','RWDD3','RXRA','RXRB','RXRG','RYBP','S100A12','S100A8','S100A9','SAFB','SAFB2','SALL1','SALL2','SALL3','SALL4','SART1','SATB1','SATB2','SAV1','SCAND1','SCAND2P','SCAPER','SCMH1','SCML1','SCML2','SCML4','SCOC','SCRT1','SCRT2','SCX','SEBOX','SEMA4A','SERTAD1','SERTAD2','SETBP1','SETD6','SETDB1','SETDB2','SF3A2','SF3A3','SFPQ','SFRP4','SFRP5','SFTPB','SGK1','SGK3','SGSM2','SHH','SHOX','SHOX2','SHPRH','SIGIRR','SIK1','SIM1','SIM2','SIN3A','SIN3B','SIRT1','SIRT2','SIRT3','SIRT6','SIVA1','SIX1','SIX2','SIX3','SIX4','SIX5','SIX6','SKI','SKIDA1','SKIL','SKOR1','SKOR2','SLC22A4','SLC26A10','SLC26A3','SLC2A4RG','SLC30A9','SLC39A10','SLC4A10','SMAD1','SMAD2','SMAD3','SMAD4','SMAD5','SMAD6','SMAD7','SMAD9','SMARCA1','SMARCA2','SMARCA4','SMARCA5','SMARCAL1','SMARCB1','SMARCC1','SMARCC2','SMARCD1','SMARCD2','SMARCD3','SMARCE1','SMC3','SMO','SMUG1','SMYD1','SMYD3','SNAI1','SNAI2','SNAI3','SNAPC2','SNAPC4','SNAPC5','SNRPC','SNURF','SNW1','SOGA3','SOHLH1','SOHLH2','SON','SORBS2','SOX1','SOX10','SOX11','SOX12','SOX13','SOX14','SOX15','SOX17','SOX18','SOX2','SOX21','SOX3','SOX30','SOX4','SOX5','SOX6','SOX7','SOX8','SOX9','SP1','SP100','SP110','SP140','SP140L','SP2','SP3','SP4','SP5','SP6','SP7','SP8','SP9','SPDEF','SPEN','SPHK1','SPI1','SPIB','SPIC','SPZ1','SRCAP','SREBF1','SREBF2','SRF','SRY','SSB','SSBP2','SSBP3','SSBP4','SSH1','SSH2','SSH3','SSRP1','SSX1','ST18','STAG1','STAT1','STAT2','STAT3','STAT4','STAT5A','STAT5B','STAT6','STK16','STK3','STK36','STRN','STRN3','SUB1','SUFU','SUMO1','SUPT16H','SUPT4H1','SUPT5H','SUPT6H','SUV39H1','SUV39H2','SUZ12','SYK','T','TAB1','TAB2','TAB3','TADA2A','TADA2B','TADA3','TAF1','TAF10','TAF11','TAF12','TAF13','TAF1A','TAF1B','TAF1C','TAF1L','TAF2','TAF3','TAF4','TAF4B','TAF5','TAF5L','TAF6','TAF6L','TAF7','TAF7L','TAF8','TAF9','TAF9B','TAL1','TAL2','TARDBP','TAX1BP1','TBL1Y','TBP','TBPL1','TBPL2','TBR1','TBX1','TBX10','TBX15','TBX18','TBX19','TBX2','TBX20','TBX21','TBX22','TBX3','TBX4','TBX5','TBX6','TCEA2','TCEAL1','TCEAL7','TCEAL8','TCF12','TCF15','TCF19','TCF20','TCF21','TCF23','TCF24','TCF25','TCF3','TCF4','TCF7','TCF7L1','TCF7L2','TCFL5','TDG','TDP1','TDP2','TEAD1','TEAD2','TEAD3','TEAD4','TEF','TERB1','TERF1','TERF2','TERF2IP','TESMIN','TET1','TET2','TET3','TFAM','TFAP2A','TFAP2B','TFAP2C','TFAP2D','TFAP2E','TFAP4','TFCP2','TFCP2L1','TFDP1','TFDP2','TFDP3','TFE3','TFEB','TFEC','TGFB1','TGFB1I1','TGIF1','TGIF2','TGIF2LX','TGIF2LY','TGS1','THAP1','THAP10','THAP11','THAP12','THAP2','THAP3','THAP4','THAP5','THAP6','THAP7','THAP8','THAP9','THRA','THRAP3','THRB','THYN1','TIA1','TICAM1','TIGD1','TIGD2','TIGD3','TIGD4','TIGD5','TIGD6','TIGD7','TIPARP','TIRAP','TLE1','TLE2','TLE3','TLE4','TLE6','TLR2','TLR3','TLR4','TLR9','TLX1','TLX2','TLX3','TMEM235','TMF1','TNF','TNFAIP3','TNFRSF11A','TNFRSF4','TNFSF11','TNFSF18','TNFSF4','TOE1','TONSL','TOP1','TOP2A','TOP2B','TOPORS','TOX','TOX2','TOX3','TOX4','TP53','TP53BP1','TP63','TP73','TPRX1','TRADD','TRAF1','TRAF2','TRAF3','TRAF4','TRAF5','TRAF6','TRAFD1','TRAK2','TRAPPC9','TRERF1','TREX1','TRIB1','TRIB3','TRIM13','TRIM14','TRIM15','TRIM21','TRIM22','TRIM23','TRIM24','TRIM25','TRIM26','TRIM27','TRIM28','TRIM29','TRIM3','TRIM31','TRIM32','TRIM33','TRIM34','TRIM37','TRIM38','TRIM40','TRIM5','TRIM52','TRIM62','TRIM8','TRIP11','TRIT1','TRMT1','TRMT1L','TRPS1','TRRAP','TSC2','TSC22D1','TSC22D2','TSC22D3','TSC22D4','TSHZ1','TSHZ2','TSHZ3','TSN','TSSK4','TTF1','TUB','TULP4','TUT1','TWIST1','TWIST2','TXK','U2AF1','U2AF1L4','UBA52','UBB','UBC','UBE2K','UBE2N','UBE2V1','UBN1','UBP1','UBR4','UBTF','UBTFL1','UBTFL6','UFL1','UHRF1','UHRF2','UNCX','UNG','UNK','UNKL','USF1','USF2','USF3','USP39','USP7','UTF1','VAV1','VAX1','VAX2','VDR','VEGFA','VENTX','VEZF1','VGLL1','VGLL2','VGLL3','VHLL','VPS72','VSX1','VSX2','WDHD1','WFS1','WIZ','WNT1','WNT10B','WNT2','WNT3A','WNT5A','WNT8B','WRN','WT1','WWOX','WWP2','WWTR1','XBP1','XCL1','XPA','YBX1','YBX2','YBX3','YEATS2','YEATS4','YOD1','YY1','YY2','ZBED1','ZBED2','ZBED3','ZBED4','ZBED5','ZBED6','ZBED6CL','ZBED8','ZBED9','ZBP1','ZBTB1','ZBTB10','ZBTB11','ZBTB12','ZBTB14','ZBTB16','ZBTB17','ZBTB18','ZBTB2','ZBTB20','ZBTB21','ZBTB22','ZBTB24','ZBTB25','ZBTB26','ZBTB3','ZBTB32','ZBTB33','ZBTB34','ZBTB37','ZBTB38','ZBTB39','ZBTB4','ZBTB40','ZBTB41','ZBTB42','ZBTB43','ZBTB44','ZBTB45','ZBTB46','ZBTB47','ZBTB48','ZBTB49','ZBTB5','ZBTB6','ZBTB7A','ZBTB7B','ZBTB7C','ZBTB8A','ZBTB8B','ZBTB9','ZC3H10','ZC3H11A','ZC3H12A','ZC3H13','ZC3H14','ZC3H15','ZC3H18','ZC3H3','ZC3H4','ZC3H6','ZC3H7A','ZC3H7B','ZC3H8','ZC3HAV1','ZCCHC11','ZCCHC6','ZDHHC11','ZDHHC19','ZEB1','ZEB2','ZFAND5','ZFAT','ZFHX2','ZFHX3','ZFHX4','ZFP1','ZFP14','ZFP2','ZFP28','ZFP3','ZFP30','ZFP36','ZFP36L1','ZFP36L2','ZFP37','ZFP41','ZFP42','ZFP57','ZFP62','ZFP64','ZFP69','ZFP69B','ZFP82','ZFP90','ZFP91','ZFP92','ZFPL1','ZFPM1','ZFPM2','ZFR','ZFR2','ZFX','ZFY','ZFYVE26','ZGLP1','ZGPAT','ZHX1','ZHX2','ZHX3','ZIC1','ZIC2','ZIC3','ZIC4','ZIC5','ZIK1','ZIM2','ZIM3','ZKSCAN1','ZKSCAN2','ZKSCAN3','ZKSCAN4','ZKSCAN5','ZKSCAN7','ZKSCAN8','ZMAT1','ZMAT2','ZMAT3','ZMAT4','ZMAT5','ZMYM1','ZMYM2','ZMYM3','ZMYM4','ZMYM5','ZMYM6','ZMYND10','ZNF10','ZNF100','ZNF101','ZNF106','ZNF107','ZNF112','ZNF114','ZNF117','ZNF12','ZNF121','ZNF124','ZNF131','ZNF132','ZNF133','ZNF134','ZNF135','ZNF136','ZNF137P','ZNF138','ZNF14','ZNF140','ZNF141','ZNF142','ZNF143','ZNF146','ZNF148','ZNF154','ZNF155','ZNF157','ZNF16','ZNF160','ZNF165','ZNF169','ZNF17','ZNF174','ZNF175','ZNF177','ZNF18','ZNF180','ZNF181','ZNF182','ZNF184','ZNF185','ZNF189','ZNF19','ZNF195','ZNF197','ZNF2','ZNF20','ZNF200','ZNF202','ZNF205','ZNF207','ZNF208','ZNF211','ZNF212','ZNF213','ZNF214','ZNF215','ZNF217','ZNF219','ZNF22','ZNF221','ZNF222','ZNF223','ZNF224','ZNF225','ZNF226','ZNF227','ZNF229','ZNF23','ZNF230','ZNF232','ZNF233','ZNF234','ZNF235','ZNF236','ZNF239','ZNF24','ZNF248','ZNF25','ZNF250','ZNF251','ZNF253','ZNF254','ZNF256','ZNF257','ZNF26','ZNF260','ZNF263','ZNF264','ZNF266','ZNF267','ZNF268','ZNF273','ZNF274','ZNF275','ZNF276','ZNF277','ZNF28','ZNF280A','ZNF280B','ZNF280C','ZNF280D','ZNF281','ZNF282','ZNF283','ZNF284','ZNF285','ZNF286A','ZNF286B','ZNF287','ZNF292','ZNF296','ZNF3','ZNF30','ZNF300','ZNF302','ZNF304','ZNF311','ZNF316','ZNF317','ZNF318','ZNF319','ZNF32','ZNF320','ZNF322','ZNF324','ZNF324B','ZNF326','ZNF329','ZNF330','ZNF331','ZNF333','ZNF334','ZNF335','ZNF337','ZNF33A','ZNF33B','ZNF34','ZNF341','ZNF343','ZNF345','ZNF346','ZNF347','ZNF35','ZNF350','ZNF354A','ZNF354B','ZNF354C','ZNF355P','ZNF358','ZNF362','ZNF365','ZNF366','ZNF367','ZNF37A','ZNF382','ZNF383','ZNF384','ZNF385A','ZNF385B','ZNF385C','ZNF385D','ZNF391','ZNF394','ZNF395','ZNF396','ZNF397','ZNF398','ZNF404','ZNF407','ZNF408','ZNF41','ZNF410','ZNF414','ZNF415','ZNF416','ZNF417','ZNF418','ZNF419','ZNF420','ZNF423','ZNF425','ZNF426','ZNF428','ZNF429','ZNF43','ZNF430','ZNF431','ZNF432','ZNF433','ZNF436','ZNF438','ZNF439','ZNF44','ZNF440','ZNF441','ZNF442','ZNF443','ZNF444','ZNF445','ZNF446','ZNF449','ZNF45','ZNF451','ZNF454','ZNF460','ZNF461','ZNF462','ZNF467','ZNF468','ZNF469','ZNF470','ZNF471','ZNF473','ZNF474','ZNF479','ZNF48','ZNF480','ZNF483','ZNF484','ZNF485','ZNF486','ZNF487','ZNF488','ZNF490','ZNF491','ZNF492','ZNF493','ZNF496','ZNF497','ZNF500','ZNF501','ZNF502','ZNF503','ZNF506','ZNF507','ZNF510','ZNF511','ZNF512','ZNF512B','ZNF513','ZNF514','ZNF516','ZNF517','ZNF518A','ZNF518B','ZNF519']
TFs3Other=['ZNF521','ZNF524','ZNF525','ZNF526','ZNF527','ZNF528','ZNF529','ZNF530','ZNF532','ZNF534','ZNF536','ZNF540','ZNF541','ZNF542P','ZNF543','ZNF544','ZNF546','ZNF547','ZNF548','ZNF549','ZNF550','ZNF551','ZNF552','ZNF554','ZNF555','ZNF556','ZNF557','ZNF558','ZNF559','ZNF560','ZNF561','ZNF562','ZNF563','ZNF564','ZNF565','ZNF566','ZNF567','ZNF568','ZNF569','ZNF57','ZNF570','ZNF571','ZNF572','ZNF573','ZNF574','ZNF575','ZNF576','ZNF577','ZNF578','ZNF579','ZNF580','ZNF581','ZNF582','ZNF583','ZNF584','ZNF585A','ZNF585B','ZNF586','ZNF587','ZNF587B','ZNF589','ZNF592','ZNF593','ZNF594','ZNF595','ZNF596','ZNF597','ZNF598','ZNF599','ZNF600','ZNF605','ZNF606','ZNF607','ZNF608','ZNF609','ZNF610','ZNF611','ZNF613','ZNF614','ZNF615','ZNF616','ZNF618','ZNF619','ZNF620','ZNF621','ZNF622','ZNF623','ZNF624','ZNF625','ZNF626','ZNF627','ZNF628','ZNF629','ZNF630','ZNF638','ZNF639','ZNF641','ZNF644','ZNF645','ZNF646','ZNF648','ZNF649','ZNF652','ZNF653','ZNF654','ZNF655','ZNF658','ZNF66','ZNF660','ZNF662','ZNF664','ZNF665','ZNF667','ZNF668','ZNF669','ZNF670','ZNF671','ZNF672','ZNF674','ZNF675','ZNF676','ZNF677','ZNF678','ZNF679','ZNF680','ZNF681','ZNF682','ZNF683','ZNF684','ZNF687','ZNF688','ZNF689','ZNF69','ZNF691','ZNF692','ZNF695','ZNF696','ZNF697','ZNF699','ZNF7','ZNF70','ZNF700','ZNF701','ZNF703','ZNF704','ZNF705A','ZNF705B','ZNF705D','ZNF705E','ZNF705G','ZNF706','ZNF707','ZNF708','ZNF709','ZNF71','ZNF710','ZNF711','ZNF713','ZNF714','ZNF716','ZNF717','ZNF718','ZNF720','ZNF721','ZNF724','ZNF726','ZNF727','ZNF728','ZNF729','ZNF73','ZNF730','ZNF732','ZNF735','ZNF736','ZNF737','ZNF738','ZNF74','ZNF740','ZNF746','ZNF747','ZNF749','ZNF750','ZNF75A','ZNF75D','ZNF76','ZNF761','ZNF763','ZNF764','ZNF765','ZNF766','ZNF768','ZNF77','ZNF770','ZNF771','ZNF772','ZNF773','ZNF774','ZNF775','ZNF776','ZNF777','ZNF778','ZNF780A','ZNF780B','ZNF781','ZNF782','ZNF783','ZNF784','ZNF785','ZNF786','ZNF787','ZNF788','ZNF789','ZNF79','ZNF790','ZNF791','ZNF792','ZNF793','ZNF799','ZNF8','ZNF80','ZNF800','ZNF804A','ZNF804B','ZNF805','ZNF808','ZNF81','ZNF812P','ZNF813','ZNF814','ZNF816','ZNF821','ZNF823','ZNF827','ZNF829','ZNF83','ZNF830','ZNF831','ZNF835','ZNF836','ZNF837','ZNF839','ZNF84','ZNF841','ZNF843','ZNF844','ZNF845','ZNF846','ZNF85','ZNF850','ZNF852','ZNF853','ZNF860','ZNF862','ZNF865','ZNF878','ZNF879','ZNF880','ZNF883','ZNF888','ZNF891','ZNF90','ZNF91','ZNF92','ZNF93','ZNF98','ZNF99','ZNFX1','ZRANB2','ZRSR1','ZRSR2','ZSCAN1','ZSCAN10','ZSCAN12','ZSCAN16','ZSCAN18','ZSCAN2','ZSCAN20','ZSCAN21','ZSCAN22','ZSCAN23','ZSCAN25','ZSCAN26','ZSCAN29','ZSCAN30','ZSCAN31','ZSCAN32','ZSCAN4','ZSCAN5A','ZSCAN5B','ZSCAN5C','ZSCAN5DP','ZSCAN9','ZUFSP','ZXDA','ZXDB','ZXDC','ZZZ3']
TFs4Other=['ATF2','ATF6B','ATN1','BHLHA15','BHLHB2','BHLHB4 ','BHLHB5','BHLHB9','C13ORF8','C20ORF194','CDX2','CRAMP1L','ELP3','ERVK3-1','ETF1','FLJ12895','FLJ20321','FLJ25476','FLJ35784','FOXM1','FOXO2','FOXR1','FOXR2','GATA1','GRLF1','KDM4C','KDM4E','KIAA0863','KIAA0961','KLRG1','LBXCOR1','LEF1','LHX7','LOC126295','LOC51058','LOC91661','MGC4728','MYOD1','MYOG','MYRF','NEUROD4','NEUROG1','NFAT2','NFAT4','NFE2L1','NR1A2','NR1B1','NR1B2','NR1B3','NR1C1','NR1C2','NR1C3','NR1F3','NR1I1','NR2A1','NR2B3','NR3A1','NR3A2','NR3B3','OTEX','PEPP-2','PHF2','POU2F1','POU2F2','POU4F3','POU5F1','POU6F1','PRDM3','PRKRIP1','PRKRIR','SETD1A','SMAD8','SMYD2','SOX12','SOX14','SOX15','TBX14','TCF1','TCF17','TCF2','TCF9','TEAD1','TPRXL','UBTFL1','WHSC1','ZBTB14','ZBTB18','ZBTB19','ZBTB21','ZBTB27','ZBTB28','ZBTB35','ZBTB49','ZFP161','ZFP28','ZKSCAN10','ZKSCAN11','ZKSCAN12','ZKSCAN13','ZKSCAN14','ZKSCAN16','ZKSCAN19','ZKSCAN21','ZKSCAN24','ZKSCAN6','ZKSCAN8','ZNF10','ZNF112','ZNF187','ZNF193','ZNF204P','ZNF238','ZNF271','ZNF295','ZNF305','ZNF323','ZNF434','ZNF499','ZNF542','ZNF545','ZNF553','ZNF56','ZNF625','ZNF643','ZNF698','ZNF705F','ZNF751','ZNF752','ZNF753','ZNF785','ZNF806','ZNF816','ZNF833P','ZNF906','ZSCAN1','ZSCAN25','ZSCAN26','ZSCAN31','ZSCAN32','ZSCAN5D','ZSCAN9']
TFsOther=TFs1Other+TFs2Other+TFs3Other+TFs4Other
```

Example 9 (unknown):
```unknown
S genes: ['MCM5', 'PCNA', 'TYMS', 'FEN1', 'MCM2', 'MCM4', 'RRM1', 'UNG', 'GINS2', 'MCM6', 'CDCA7', 'DTL', 'PRIM1', 'UHRF1', 'HELLS', 'RFC2', 'RPA2', 'NASP', 'RAD51AP1', 'GMNN', 'WDR76', 'SLBP', 'CCNE2', 'UBR7', 'POLD3', 'MSH2', 'ATAD2', 'RAD51', 'RRM2', 'CDC45', 'CDC6', 'EXO1', 'TIPIN', 'DSCC1', 'BLM', 'CASP8AP2', 'USP1', 'CLSPN', 'POLA1', 'CHAF1B', 'BRIP1', 'E2F8'] 

G2 genes: ['HMGB2', 'CDK1', 'NUSAP1', 'UBE2C', 'BIRC5', 'TPX2', 'TOP2A', 'NDC80', 'CKS2', 'NUF2', 'CKS1B', 'MKI67', 'TMPO', 'CENPF', 'TACC3', 'SMC4', 'CCNB2', 'CKAP2L', 'CKAP2', 'AURKB', 'BUB1', 'KIF11', 'ANP32E', 'TUBB4B', 'GTSE1', 'KIF20B', 'HJURP', 'CDCA3', 'CDC20', 'TTK', 'CDC25C', 'KIF2C', 'RANGAP1', 'NCAPD2', 'DLGAP5', 'CDCA2', 'CDCA8', 'ECT2', 'KIF23', 'HMMR', 'AURKA', 'PSRC1', 'ANLN', 'LBR', 'CKAP5', 'CENPE', 'CTCF', 'NEK2', 'G2E3', 'GAS2L3', 'CBX5', 'CENPA'] 

Hemoglobin genes set 1: ['HBB', 'HBD', 'HBG1', 'HBG2', 'HBE1', 'HBZ', 'HBM', 'HBA2', 'HBA1', 'HBQ1'] 

Hemoglobin genes set 2: ['AHSP', 'GYPA', 'RHAG', 'GFI1B', 'KEL', 'EPB42', 'ALAS2', 'TSPO2', 'GYPB', 'GYPE', 'SPTA1', 'RHCE', 'KLF1', 'RHD', 'SLC4A1', 'SNX22', 'SQSTM1', 'SOD1', 'FTH1', 'MAP1LC3B', 'SLC25A37', 'TRIM58', 'HMBS'] 

Mitochondrial genes: ['MT-ND1', 'MT-ND2', 'MT-CO1', 'MT-CO2', 'MT-ATP8', 'MT-ATP6', 'MT-CO3', 'MT-ND3', 'MT-ND4L', 'MT-ND4', 'MT-ND5', 'MT-ND6', 'MT-CYB'] 

Ribo genes: ['RPLP0', 'RPLP1', 'RPLP2', 'RPL3', 'RPL3L', 'RPL4', 'RPL5', 'RPL6', 'RPL7', 'RPL7A', 'RPL7L1', 'RPL8', 'RPL9', 'RPL10', 'RPL10A', 'RPL10L', 'RPL11', 'RPL12', 'RPL13', 'RPL13A', 'RPL14', 'RPL15', 'RPL17', 'RPL18', 'RPL18A', 'RPL19', 'RPL21', 'RPL22', 'RPL22L1', 'RPL23', 'RPL23A', 'RPL24', 'RPL26', 'RPL26L1', 'RPL27', 'RPL27A', 'RPL28', 'RPL29', 'RPL30', 'RPL31', 'RPL32', 'RPL34', 'RPL35', 'RPL35A', 'RPL36', 'RPL36A', 'RPL36AL', 'RPL37', 'RPL37A', 'RPL38', 'RPL39', 'RPL39L', 'UBA52', 'RPL41', 'RPSA', 'RPS2', 'RPS3', 'RPS3A', 'RPS4X', 'RPS4Y1', 'RPS4Y2', 'RPS5', 'RPS6', 'RPS7', 'RPS8', 'RPS9', 'RPS10', 'RPS11', 'RPS12', 'RPS13', 'RPS14', 'RPS15', 'RPS15A', 'RPS16', 'RPS17', 'RPS18', 'RPS19', 'RPS20', 'RPS21', 'RPS23', 'RPS24', 'RPS25', 'RPS26', 'RPS27', 'RPS27A', 'RPS27L', 'RPS28', 'RPS29', 'FAU'] 

Tissue dissociation stress associated genes: ['FOS', 'FOSB', 'JUN', 'JUNB', 'JUND', 'ZFP36', 'SOCS3', 'CEBPD', 'CEBPB', 'ATF3', 'EGR1', 'HSPA1A', 'HSPA1B', 'HSP90AA1', 'HSP90AB1', 'HSPA8', 'HSPB1', 'HSPE1', 'IER3', 'IER2', 'BTG1', 'BTG2', 'DUSP1']
```

Example 10 (python):
```python
# preserve raw count
adata.layers["counts"] = adata.X.copy()

# normalise and log-transform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key='batch_key')

print(sum(adata.var.highly_variable))
sc.pl.highly_variable_genes(adata)

# freeze the state in `.raw`
adata.raw = adata

# subset hvgs
adata = adata[:, adata.var.highly_variable]

adata=adata.copy()

sc.tl.score_genes_cell_cycle(adata, s_genes=s_genes, g2m_genes=g2m_genes)
#sc.pp.regress_out(adata, ['S_score','G2M_score'])

#batch correction
scvi.model.SCVI.setup_anndata(adata, 
                              layer="counts", 
                              batch_key="batch_key",
                              categorical_covariate_keys=['phase'],
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo'])

vae = scvi.model.SCVI(adata, n_hidden = 128, n_latent = 50, n_layers = 3, dispersion = 'gene-batch')
vae.train(max_epochs=50)
    
# Convergence plot
x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
plt.plot(x, vae.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
plt.clf()

# add the latent space to adata
adata.obsm["X_scVI"] = vae.get_latent_representation()

sc.tl.score_genes(adata, stress_genes, score_name='stress_score')
sc.tl.score_genes(adata, hb_genes1, score_name='hb1_score')

sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata,min_dist=0.1)
#sc.tl.tsne(adata)
sc.pl.umap(adata, color=['log1p_total_counts','n_genes','n_genes_by_counts','total_counts','pct_counts_mt','pct_counts_ribo'],ncols=5)
sc.pl.umap(adata, color=['scrublet_score','stress_score', 'hb1_score','phase'], ncols=5,)
sc.pl.umap(adata, color=['donor','age','week','trimester','region','phase','cycling','cell_or_nuclei','modality','kit_10x','FACSgate','heart_or_greatvessels','coarse_grain','mid_grain','fine_grain'],wspace=0.8,ncols=5)
```

Example 11 (python):
```python
normalizing counts per cell
    finished (0:00:00)
extracting highly variable genes
    finished (0:00:08)
--> added
    'highly_variable', boolean vector (adata.var)
    'means', float vector (adata.var)
    'dispersions', float vector (adata.var)
    'dispersions_norm', float vector (adata.var)
1634
```

Example 12 (python):
```python
calculating cell cycle phase
computing score 'S_score'
WARNING: genes are not in var_names and ignored: ['MCM5', 'FEN1', 'MCM2', 'MCM4', 'UNG', 'GINS2', 'MCM6', 'CDCA7', 'PRIM1', 'UHRF1', 'RFC2', 'RPA2', 'NASP', 'GMNN', 'SLBP', 'CCNE2', 'UBR7', 'POLD3', 'MSH2', 'CDC45', 'CDC6', 'EXO1', 'DSCC1', 'CASP8AP2', 'USP1', 'POLA1', 'CHAF1B', 'E2F8']
    finished: added
    'S_score', score of gene set (adata.obs).
    411 total control genes are used. (0:00:00)
computing score 'G2M_score'
WARNING: genes are not in var_names and ignored: ['TMPO', 'CKAP2L', 'ANP32E', 'CDCA3', 'RANGAP1', 'NCAPD2', 'CDCA2', 'PSRC1', 'LBR', 'CKAP5', 'CTCF', 'NEK2', 'G2E3', 'CBX5']
    finished: added
    'G2M_score', score of gene set (adata.obs).
    732 total control genes are used. (0:00:00)
-->     'phase', cell cycle phase (adata.obs)
```

Example 13 (python):
```python
/home/sinhalab/miniconda3/envs/scanpynew/lib/python3.8/site-packages/torchmetrics/utilities/prints.py:36: UserWarning: Torchmetrics v0.9 introduced a new argument class property called `full_state_update` that has
                not been set for this class (ElboMetric). The property determines if `update` by
                default needs access to the full metric state. If this is not the case, significant speedups can be
                achieved and we recommend setting this to `False`.
                We provide an checking function
                `from torchmetrics.utilities import check_forward_no_full_state`
                that can be used to check if the `full_state_update=True` (old and potential slower behaviour,
                default for now) or if `full_state_update=False` can be used safely.
                
  warnings.warn(*args, **kwargs)
GPU available: False, used: False
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
```

Example 14 (unknown):
```unknown
Epoch 1/50:   0%|                                        | 0/50 [00:00<?, ?it/s]
```

Example 15 (python):
```python
/home/sinhalab/miniconda3/envs/scanpynew/lib/python3.8/site-packages/torchmetrics/utilities/prints.py:36: UserWarning: Torchmetrics v0.9 introduced a new argument class property called `full_state_update` that has
                not been set for this class (ResultMetric). The property determines if `update` by
                default needs access to the full metric state. If this is not the case, significant speedups can be
                achieved and we recommend setting this to `False`.
                We provide an checking function
                `from torchmetrics.utilities import check_forward_no_full_state`
                that can be used to check if the `full_state_update=True` (old and potential slower behaviour,
                default for now) or if `full_state_update=False` can be used safely.
                
  warnings.warn(*args, **kwargs)
```

Example 16 (unknown):
```unknown
Epoch 50/50: 100%|███████████| 50/50 [00:51<00:00,  1.03s/it, loss=806, v_num=1]
```

Example 17 (python):
```python
computing score 'stress_score'
WARNING: genes are not in var_names and ignored: ['JUND', 'CEBPB', 'HSP90AA1', 'HSP90AB1', 'HSPA8', 'HSPB1', 'HSPE1', 'BTG1']
    finished: added
    'stress_score', score of gene set (adata.obs).
    389 total control genes are used. (0:00:00)
computing score 'hb1_score'
WARNING: genes are not in var_names and ignored: ['HBD', 'HBM', 'HBQ1']
    finished: added
    'hb1_score', score of gene set (adata.obs).
    294 total control genes are used. (0:00:00)
computing neighbors
```

Example 18 (python):
```python
/home/sinhalab/miniconda3/envs/scanpynew/lib/python3.8/site-packages/numba/core/typed_passes.py:313: NumbaPerformanceWarning: 
The keyword argument 'parallel=True' was specified but no transformation for parallel execution was possible.

To find out why, try turning on parallel diagnostics, see https://numba.pydata.org/numba-doc/latest/user/parallel.html#diagnostics for help.

File "../../miniconda3/envs/scanpynew/lib/python3.8/site-packages/umap/rp_tree.py", line 135:
@numba.njit(fastmath=True, nogil=True, parallel=True)
def euclidean_random_projection_split(data, indices, rng_state):
^

  warnings.warn(errors.NumbaPerformanceWarning(msg,
/home/sinhalab/miniconda3/envs/scanpynew/lib/python3.8/site-packages/umap/nndescent.py:91: NumbaPerformanceWarning: 
The keyword argument 'parallel=True' was specified but no transformation for parallel execution was possible.

To find out why, try turning on parallel diagnostics, see https://numba.pydata.org/numba-doc/latest/user/parallel.html#diagnostics for help.

File "../../miniconda3/envs/scanpynew/lib/python3.8/site-packages/umap/utils.py", line 409:
@numba.njit(parallel=True)
def build_candidates(current_graph, n_vertices, n_neighbors, max_candidates, rng_state):
^

  candidate_neighbors = build_candidates(
/home/sinhalab/miniconda3/envs/scanpynew/lib/python3.8/site-packages/numba/core/typed_passes.py:313: NumbaPerformanceWarning: 
The keyword argument 'parallel=True' was specified but no transformation for parallel execution was possible.

To find out why, try turning on parallel diagnostics, see https://numba.pydata.org/numba-doc/latest/user/parallel.html#diagnostics for help.

File "../../miniconda3/envs/scanpynew/lib/python3.8/site-packages/umap/nndescent.py", line 47:
    @numba.njit(parallel=True)
    def nn_descent(
    ^

  warnings.warn(errors.NumbaPerformanceWarning(msg,
```

Example 19 (python):
```python
finished: added to `.uns['neighbors']`
    `.obsp['distances']`, distances for each pair of neighbors
    `.obsp['connectivities']`, weighted adjacency matrix (0:00:03)
computing UMAP
    finished: added
    'X_umap', UMAP coordinates (adata.obsm) (0:00:03)
```

Example 20 (unknown):
```unknown
/home/sinhalab/miniconda3/envs/scanpynew/lib/python3.8/site-packages/anndata/_core/anndata.py:1220: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Removing unused categories will always return a new Categorical object.
  c.reorder_categories(natsorted(c.categories), inplace=True)
... storing 'phase' as categorical
```

Example 21 (unknown):
```unknown
<Figure size 320x320 with 0 Axes>
```

Example 22 (python):
```python
sc.tl.diffmap(adata)
sc.pl.diffmap(adata, color='fine_grain')
```

Example 23 (python):
```python
computing Diffusion Maps using n_comps=15(=n_dcs)
computing transitions
    finished (0:00:00)
    eigenvalues of transition matrix
    [1.         0.9950726  0.99062014 0.9852053  0.9790444  0.97708017
     0.97520936 0.9719767  0.96848094 0.9680828  0.9639776  0.96136785
     0.95495147 0.9540259  0.9515269 ]
    finished: added
    'X_diffmap', diffmap coordinates (adata.obsm)
    'diffmap_evals', eigenvalues of transition matrix (adata.uns) (0:00:00)
```

Example 24 (python):
```python
adata.write('./EndothelialCellsForscFatesDiffMapCellCyclingCellsIN.h5ad')
```

Example 25 (python):
```python
sc.pl.umap(adata, color=['fine_grain'], ncols=5,)
```

Example 26 (python):
```python
import pandas as pd
import numpy as np
import anndata


import scipy
import matplotlib.pyplot as plt
import scanpy as sc
import scFates as scf
import palantir

plt.rcParams['pdf.fonttype'] = 42
plt.rcParams['ps.fonttype'] = 42
plt.rcParams["font.family"] = "sans-serif"
sc.settings.verbosity = 3  # verbosity: errors (0), warnings (1), info (2), hints (3)

from matplotlib.colors import LinearSegmentedColormap
cmap = LinearSegmentedColormap.from_list(name='gene_cmap', colors=['lightgrey', 'thistle', 'red', 'darkred']) 
import matplotlib.pyplot as plt
import matplotlib.font_manager

sc.settings.set_figure_params(dpi=80, color_map='viridis', vector_friendly=False,  dpi_save=300, facecolor='white')
sc.logging.print_version_and_date()
#sc.logging.print_versions()
```

Example 27 (unknown):
```unknown
findfont: Font family ['Raleway'] not found. Falling back to DejaVu Sans.
```

Example 28 (unknown):
```unknown
Running Scanpy 1.9.3, on 2023-09-19 11:55.
```

Example 29 (python):
```python
adata=sc.read('./EndothelialCellsForscFates.h5ad')
print(adata.shape)
```

Example 30 (unknown):
```unknown
(8359, 1450)
```

Example 31 (unknown):
```unknown
cell_cycle_genes = ['MCM5','PCNA','TYMS','FEN1','MCM2','MCM4','RRM1','UNG','GINS2','MCM6','CDCA7','DTL','PRIM1','UHRF1','MLF1IP','HELLS','RFC2','RPA2','NASP','RAD51AP1','GMNN','WDR76','SLBP','CCNE2','UBR7','POLD3','MSH2','ATAD2','RAD51','RRM2','CDC45','CDC6','EXO1','TIPIN','DSCC1','BLM','CASP8AP2','USP1','CLSPN','POLA1','CHAF1B','BRIP1','E2F8','HMGB2','CDK1','NUSAP1','UBE2C','BIRC5','TPX2','TOP2A','NDC80','CKS2','NUF2','CKS1B','MKI67','TMPO','CENPF','TACC3','FAM64A','SMC4','CCNB2','CKAP2L','CKAP2','AURKB','BUB1','KIF11','ANP32E','TUBB4B','GTSE1','KIF20B','HJURP','CDCA3','HN1','CDC20','TTK','CDC25C','KIF2C','RANGAP1','NCAPD2','DLGAP5','CDCA2','CDCA8','ECT2','KIF23','HMMR','AURKA','PSRC1','ANLN','LBR','CKAP5','CENPE','CTCF','NEK2','G2E3','GAS2L3','CBX5','CENPA']
```

Example 32 (python):
```python
projections = pd.DataFrame(adata.obsm["X_scVI"],index=adata.obs_names)
projections
# fate probability mapping of the cells
dm_res = palantir.utils.run_diffusion_maps(projections)
ms_data = palantir.utils.determine_multiscale_space(dm_res,n_eigs=4)
# Generate embedding from the multiscale diffusion space
adata.obsm["X_palantir"]=ms_data.values
sc.pp.neighbors(adata,n_neighbors=30,use_rep="X_palantir")
sc.pl.embedding(adata, basis='X_palantir',color=['fine_grain'], components='all',ncols=3,wspace=0.6,size=10)

# draw ForceAtlas2 embedding using UMAP dim1 and dim2 as initial position
# make obsm with UMAP1 and 2
adata.obsm['X_umap12'] = adata.obsm['X_umap'][:,[0,1]]
sc.tl.draw_graph(adata,init_pos='X_umap12')
# Plot
sc.set_figure_params(dpi=120)
sc.pl.draw_graph(adata,color="fine_grain",color_map="RdBu_r")
```

Example 33 (unknown):
```unknown
Determing nearest neighbor graph...
computing neighbors
```

Example 34 (python):
```python
/home/sinhalab/miniconda3/envs/scFates/lib/python3.8/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm
```

Example 35 (unknown):
```unknown
finished: added to `.uns['neighbors']`
    `.obsp['distances']`, distances for each pair of neighbors
    `.obsp['connectivities']`, weighted adjacency matrix (0:00:10)
computing neighbors
    finished: added to `.uns['neighbors']`
    `.obsp['distances']`, distances for each pair of neighbors
    `.obsp['connectivities']`, weighted adjacency matrix (0:00:01)
```

Example 36 (unknown):
```unknown
/home/sinhalab/miniconda3/envs/scFates/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/home/sinhalab/miniconda3/envs/scFates/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/home/sinhalab/miniconda3/envs/scFates/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 37 (python):
```python
drawing single-cell graph using layout 'fa'
WARNING: Package 'fa2' is not installed, falling back to layout 'fr'.To use the faster and better ForceAtlas2 layout, install package 'fa2' (`pip install fa2`).
    finished: added
    'X_draw_graph_fr', graph_drawing coordinates (adata.obsm) (0:00:10)
```

Example 38 (unknown):
```unknown
/home/sinhalab/miniconda3/envs/scFates/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 39 (unknown):
```unknown
AnnData object with n_obs × n_vars = 8359 × 1450
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score'
    var: 'gene_name_scRNA-0', 'gene_id', 'highly_variable', 'means', 'dispersions', 'dispersions_norm', 'highly_variable_nbatches', 'highly_variable_intersection'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'age_colors', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'cycling_colors', 'donor_colors', 'fine_grain_colors', 'heart_or_greatvessels_colors', 'hvg', 'kit_10x_colors', 'mid_grain_colors', 'modality_colors', 'neighbors', 'phase_colors', 'region_colors', 'sex_colors', 'trimester_colors', 'umap', 'week_colors', 'draw_graph'
    obsm: 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding', 'X_palantir', 'X_umap12', 'X_draw_graph_fr'
    layers: 'counts'
    obsp: 'connectivities', 'distances'
```

Example 40 (python):
```python
scf.tl.curve(adata,Nodes=30,use_rep="X_umap",ndims_rep=2,epg_lambda=0.0005,epg_mu=10)
scf.pl.graph(adata,basis="umap")
scf.tl.convert_to_soft(adata,1,1000)
scf.pl.graph(adata,basis="umap")
scf.tl.root(adata,"NR2F2")
scf.tl.pseudotime(adata,n_jobs=20,n_map=100,seed=42)
scf.pl.trajectory(adata,basis="umap",arrows=True,arrow_offset=3)
scf.pl.trajectory(adata,basis="umap",arrows=False,arrow_offset=3)
adata=adata.raw.to_adata()
```

Example 41 (python):
```python
inferring a principal curve --> parameters used 
    30 principal points, mu = 10, lambda = 0.0005
    finished (0:00:01) --> added 
    .uns['epg'] dictionnary containing inferred elastic curve generated from elpigraph.
    .obsm['X_R'] hard assignment of cells to principal points.
    .uns['graph']['B'] adjacency matrix of the principal points.
    .uns['graph']['F'], coordinates of principal points in representation space.
```

Example 42 (unknown):
```unknown
/home/sinhalab/miniconda3/envs/scFates/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap', 'norm' will be ignored
  cax = scatter(
```

Example 43 (unknown):
```unknown
Converting R into soft assignment matrix
    finished (0:00:00) --> updated 
    .obsm['X_R'] converted soft assignment of cells to principal points.
    .uns['graph']['F'] coordinates of principal points in representation space.
```

Example 44 (unknown):
```unknown
/home/sinhalab/miniconda3/envs/scFates/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap', 'norm' will be ignored
  cax = scatter(
```

Example 45 (python):
```python
automatic root selection using NR2F2 values
node 0 selected as a root --> added
    .uns['graph']['root'] selected root.
    .uns['graph']['pp_info'] for each PP, its distance vs root and segment assignment.
    .uns['graph']['pp_seg'] segments network information.
projecting cells onto the principal graph
    mappings: 100%|███████████████████████████| 100/100 [00:30<00:00,  3.31it/s]
    finished (0:00:32) --> added
    .obs['edge'] assigned edge.
    .obs['t'] pseudotime value.
    .obs['seg'] segment of the tree assigned.
    .obs['milestones'] milestone assigned.
    .uns['pseudotime_list'] list of cell projection from all mappings.
```

Example 46 (unknown):
```unknown
/home/sinhalab/miniconda3/envs/scFates/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap', 'norm' will be ignored
  cax = scatter(
```

Example 47 (unknown):
```unknown
/home/sinhalab/miniconda3/envs/scFates/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap', 'norm' will be ignored
  cax = scatter(
```

Example 48 (python):
```python
scf.tl.linearity_deviation(adata,start_milestone="0",end_milestone="2",n_jobs=20,plot=True,basis="umap")
scf.pl.linearity_deviation(adata,start_milestone="0",end_milestone="2")
sc.pl.umap(adata,color=["H19","IGFBP3","PDGFB"])
```

Example 49 (python):
```python
Estimation of deviation from linearity
    cells on the bridge: 100%|█████████████| 5357/5357 [00:11<00:00, 469.64it/s]
```

Example 50 (unknown):
```unknown
/home/sinhalab/miniconda3/envs/scFates/lib/python3.8/site-packages/scFates/tools/linearity_deviation.py:144: RuntimeWarning: invalid value encountered in divide
  return rs.mean(axis=1) / X_all.std(axis=0)
/home/sinhalab/miniconda3/envs/scFates/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 51 (python):
```python
/home/sinhalab/miniconda3/envs/scFates/lib/python3.8/site-packages/scFates/tools/linearity_deviation.py:164: RuntimeWarning: Mean of empty slice
  adata.var[name + "_rss"] = np.nanmean(np.vstack(rss), axis=0)
```

Example 52 (unknown):
```unknown
finished (0:00:17) --> added 
    .var['0->2_rss'], pearson residuals of the linear fit.
    .obs['0->2_lindev_sel'], cell selections used for the test.
```

Example 53 (python):
```python
scf.tl.test_association(adata,n_jobs=20)
scf.tl.test_association(adata,reapply_filters=True,A_cut=.5)
scf.pl.test_association(adata)
scf.tl.fit(adata,n_jobs=20)
```

Example 54 (unknown):
```unknown
test features for association with the trajectory
    single mapping :  10%|█▋               | 3665/36601 [01:19<11:55, 46.06it/s]
```

Example 55 (python):
```python
---------------------------------------------------------------------------
KeyboardInterrupt                         Traceback (most recent call last)
Cell In[9], line 1
----> 1 scf.tl.test_association(adata,n_jobs=20)
      2 scf.tl.test_association(adata,reapply_filters=True,A_cut=.5)
      3 scf.pl.test_association(adata)

File ~/miniconda3/envs/scFates/lib/python3.8/site-packages/scFates/tools/test_association.py:162, in test_association(adata, n_map, n_jobs, spline_df, fdr_cut, A_cut, st_cut, reapply_filters, plot, copy, layer)
    159     stat["fdr"] = multipletests(stat.p_val, method="bonferroni")[1]
    160     return stat
--> 162 stat_assoc_l = ProgressParallel(
    163     n_jobs=1 if n_map == 1 else n_jobs,
    164     total=n_map,
    165     file=sys.stdout,
    166     use_tqdm=n_map > 1,
    167     desc="    multi mapping ",
    168 )(delayed(test_assoc_map)(m) for m in range(n_map))
    170 adata = apply_filters(adata, stat_assoc_l, fdr_cut, A_cut, st_cut)
    172 logg.info(
    173     "    found " + str(sum(adata.var["signi"])) + " significant features",
    174     time=True,
    175     end=" " if settings.verbosity > 2 else "\n",
    176 )

File ~/miniconda3/envs/scFates/lib/python3.8/site-packages/scFates/tools/utils.py:46, in ProgressParallel.__call__(self, *args, **kwargs)
     39 def __call__(self, *args, **kwargs):
     40     with tqdm(
     41         disable=not self._use_tqdm,
     42         total=self._total,
     43         desc=self._desc,
     44         file=self._file,
     45     ) as self._pbar:
---> 46         return Parallel.__call__(self, *args, **kwargs)

File ~/miniconda3/envs/scFates/lib/python3.8/site-packages/joblib/parallel.py:1085, in Parallel.__call__(self, iterable)
   1076 try:
   1077     # Only set self._iterating to True if at least a batch
   1078     # was dispatched. In particular this covers the edge
   (...)
   1082     # was very quick and its callback already dispatched all the
   1083     # remaining jobs.
   1084     self._iterating = False
-> 1085     if self.dispatch_one_batch(iterator):
   1086         self._iterating = self._original_iterator is not None
   1088     while self.dispatch_one_batch(iterator):

File ~/miniconda3/envs/scFates/lib/python3.8/site-packages/joblib/parallel.py:901, in Parallel.dispatch_one_batch(self, iterator)
    899     return False
    900 else:
--> 901     self._dispatch(tasks)
    902     return True

File ~/miniconda3/envs/scFates/lib/python3.8/site-packages/joblib/parallel.py:819, in Parallel._dispatch(self, batch)
    817 with self._lock:
    818     job_idx = len(self._jobs)
--> 819     job = self._backend.apply_async(batch, callback=cb)
    820     # A job can complete so quickly than its callback is
    821     # called before we get here, causing self._jobs to
    822     # grow. To ensure correct results ordering, .insert is
    823     # used (rather than .append) in the following line
    824     self._jobs.insert(job_idx, job)

File ~/miniconda3/envs/scFates/lib/python3.8/site-packages/joblib/_parallel_backends.py:208, in SequentialBackend.apply_async(self, func, callback)
    206 def apply_async(self, func, callback=None):
    207     """Schedule a func to be run"""
--> 208     result = ImmediateResult(func)
    209     if callback:
    210         callback(result)

File ~/miniconda3/envs/scFates/lib/python3.8/site-packages/joblib/_parallel_backends.py:597, in ImmediateResult.__init__(self, batch)
    594 def __init__(self, batch):
    595     # Don't delay the application, to avoid keeping the input
    596     # arguments in memory
--> 597     self.results = batch()

File ~/miniconda3/envs/scFates/lib/python3.8/site-packages/joblib/parallel.py:288, in BatchedCalls.__call__(self)
    284 def __call__(self):
    285     # Set the default nested backend to self._backend but do not set the
    286     # change the default number of processes to -1
    287     with parallel_backend(self._backend, n_jobs=self._n_jobs):
--> 288         return [func(*args, **kwargs)
    289                 for func, args, kwargs in self.items]

File ~/miniconda3/envs/scFates/lib/python3.8/site-packages/joblib/parallel.py:288, in <listcomp>(.0)
    284 def __call__(self):
    285     # Set the default nested backend to self._backend but do not set the
    286     # change the default number of processes to -1
    287     with parallel_backend(self._backend, n_jobs=self._n_jobs):
--> 288         return [func(*args, **kwargs)
    289                 for func, args, kwargs in self.items]

File ~/miniconda3/envs/scFates/lib/python3.8/site-packages/scFates/tools/test_association.py:151, in test_association.<locals>.test_assoc_map(m)
    148     df = adata.uns["pseudotime_list"][str(m)]
    149 data = list(zip([df] * len(Xgenes), Xgenes))
--> 151 stat = ProgressParallel(
    152     n_jobs=n_jobs if n_map == 1 else 1,
    153     total=len(data),
    154     file=sys.stdout,
    155     use_tqdm=n_map == 1,
    156     desc="    single mapping ",
    157 )(delayed(test_assoc)(data[d], spline_df) for d in range(len(data)))
    158 stat = pd.DataFrame(stat, index=adata.var_names, columns=["p_val", "A"])
    159 stat["fdr"] = multipletests(stat.p_val, method="bonferroni")[1]

File ~/miniconda3/envs/scFates/lib/python3.8/site-packages/scFates/tools/utils.py:46, in ProgressParallel.__call__(self, *args, **kwargs)
     39 def __call__(self, *args, **kwargs):
     40     with tqdm(
     41         disable=not self._use_tqdm,
     42         total=self._total,
     43         desc=self._desc,
     44         file=self._file,
     45     ) as self._pbar:
---> 46         return Parallel.__call__(self, *args, **kwargs)

File ~/miniconda3/envs/scFates/lib/python3.8/site-packages/joblib/parallel.py:1098, in Parallel.__call__(self, iterable)
   1095     self._iterating = False
   1097 with self._backend.retrieval_context():
-> 1098     self.retrieve()
   1099 # Make sure that we get a last message telling us we are done
   1100 elapsed_time = time.time() - self._start_time

File ~/miniconda3/envs/scFates/lib/python3.8/site-packages/joblib/parallel.py:975, in Parallel.retrieve(self)
    973 try:
    974     if getattr(self._backend, 'supports_timeout', False):
--> 975         self._output.extend(job.get(timeout=self.timeout))
    976     else:
    977         self._output.extend(job.get())

File ~/miniconda3/envs/scFates/lib/python3.8/site-packages/joblib/_parallel_backends.py:567, in LokyBackend.wrap_future_result(future, timeout)
    564 """Wrapper for Future.result to implement the same behaviour as
    565 AsyncResults.get from multiprocessing."""
    566 try:
--> 567     return future.result(timeout=timeout)
    568 except CfTimeoutError as e:
    569     raise TimeoutError from e

File ~/miniconda3/envs/scFates/lib/python3.8/concurrent/futures/_base.py:439, in Future.result(self, timeout)
    436 elif self._state == FINISHED:
    437     return self.__get_result()
--> 439 self._condition.wait(timeout)
    441 if self._state in [CANCELLED, CANCELLED_AND_NOTIFIED]:
    442     raise CancelledError()

File ~/miniconda3/envs/scFates/lib/python3.8/threading.py:302, in Condition.wait(self, timeout)
    300 try:    # restore state no matter what (e.g., KeyboardInterrupt)
    301     if timeout is None:
--> 302         waiter.acquire()
    303         gotit = True
    304     else:

KeyboardInterrupt:
```

Example 56 (python):
```python
scf.pl.single_trend(adata,"TCF4",basis="umap",color_exp="k")
scf.pl.single_trend(adata,"NR2F2",basis="umap",color_exp="k")
scf.pl.single_trend(adata,"APLN",basis="umap",color_exp="k")
scf.pl.single_trend(adata,"HEY2",basis="umap",color_exp="k")
```

Example 57 (python):
```python
scf.tl.cluster(adata,n_neighbors=20,metric="correlation")
adata.var.cluters.unique()
```

Example 58 (python):
```python
Clustering features using fitted layer
computing PCA
    with n_comps=50
    finished (0:00:01)
computing neighbors
    using 'X_pca' with n_pcs = 50
    finished: added to `.uns['neighbors']`
    `.obsp['distances']`, distances for each pair of neighbors
    `.obsp['connectivities']`, weighted adjacency matrix (0:00:01)
running Leiden clustering
    finished: found 11 clusters and added
    'leiden', the cluster labels (adata.obs, categorical) (0:00:00)
    finished (0:00:02) --> added 
    .var['cluters'] identified modules.
```

Example 59 (unknown):
```unknown
['4', '9', '2', '8', '10', ..., '0', '6', '1', '3', '7']
Length: 11
Categories (11, object): ['0', '1', '2', '3', ..., '7', '8', '9', '10']
```

Example 60 (python):
```python
for c in adata.var["cluters"].unique():
    scf.pl.trends(adata,features=adata.var_names[adata.var.cluters==c],basis="umap")
```

Example 61 (python):
```python
scf.pl.trends(adata,features=['NR2F2', 'RGCC','TCF4','HEY2','COL4A1','COL4A2','HEY2'],basis="umap")
```

Example 62 (python):
```python
set(adata.obs.fine_grain)
```

Example 63 (unknown):
```unknown
{'CoronaryArterialEndothelialCells',
 'CoronaryCapillaryEndothelialCells',
 'CoronaryVenousEndothelialCells',
 'GreatVesselArterialEndothelialCells',
 'GreatVesselVenousEndothelialCells'}
```

Example 64 (unknown):
```unknown
AnnData object with n_obs × n_vars = 8359 × 1052
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score', 't', 'seg', 'edge', 't_sd', 'milestones', '0->2_lindev_sel'
    var: 'gene_name_scRNA-0', 'gene_id', 'highly_variable', 'means', 'dispersions', 'dispersions_norm', 'highly_variable_nbatches', 'highly_variable_intersection', '0->2_rss', 'p_val', 'A', 'fdr', 'st', 'signi', 'cluters'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'age_colors', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'cycling_colors', 'donor_colors', 'fine_grain_colors', 'heart_or_greatvessels_colors', 'hvg', 'kit_10x_colors', 'mid_grain_colors', 'modality_colors', 'neighbors', 'phase_colors', 'region_colors', 'sex_colors', 'trimester_colors', 'umap', 'week_colors', 'draw_graph', 'graph', 'epg', 'pseudotime_list', 'milestones_colors', 'seg_colors', '0->2_lindev_sel_colors', 'stat_assoc_list'
    obsm: 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding', 'X_palantir', 'X_umap12', 'X_draw_graph_fr', 'X_R'
    layers: 'fitted'
    obsp: 'connectivities', 'distances'
```

Example 65 (python):
```python
b=adata.uns['stat_assoc_list']
b
#b=list(
#np.savetxt('significantlist.txt', b, delimiter="\n", fmt="%s")
```

Example 66 (python):
```python
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
Cell In[19], line 1
----> 1 b=adata.uns['stat_assoc_list']
      2 b
      3 #b=list(
      4 #np.savetxt('significantlist.txt', b, delimiter="\n", fmt="%s")

File ~/miniconda3/envs/scFates/lib/python3.8/site-packages/anndata/compat/_overloaded_dict.py:100, in OverloadedDict.__getitem__(self, key)
     98     return self.overloaded[key].get()
     99 else:
--> 100     return self.data[key]

KeyError: 'stat_assoc_list'
```

Example 67 (python):
```python
b=adata.uns['stat_assoc_list']
b=list(b.values())[0]
print(b)
b.to_csv('stat_assoc_list.csv')
```

Example 68 (unknown):
```unknown
{'0':                          p_val         A  fdr  st  signi
 gene_name_multiome-1                                    
 MIR1302-2HG           1.000000  0.000000  1.0   0  False
 FAM138A               1.000000  0.000000  1.0   0  False
 OR4F5                 1.000000  0.000000  1.0   0  False
 AL627309.1            0.143696  0.005061  1.0   0  False
 AL627309.3            0.802828  0.000241  1.0   0  False
 ...                        ...       ...  ...  ..    ...
 AC141272.1            1.000000  0.000000  1.0   0  False
 AC023491.2            1.000000  0.000000  1.0   0  False
 AC007325.1            1.000000  0.000000  1.0   0  False
 AC007325.4            0.948555  0.001124  1.0   0  False
 AC007325.2            0.002238  0.007391  1.0   0  False
 
 [36601 rows x 5 columns]}
```

Example 69 (python):
```python
b=adata.uns['stat_assoc_list']
b=list(b.values())[0]
b
```

Example 70 (unknown):
```unknown
b['signi']=False
b['signi'][[x for x in b.signi if x in Genes]]=True
b['signi'][[x for x in b.signi if x=='FAM138A']]=True
b
```

Example 71 (python):
```python
/tmp/ipykernel_86192/4246633740.py:2: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  b['signi'][[x for x in b.signi if x in Genes]]=True
/tmp/ipykernel_86192/4246633740.py:3: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  b['signi'][[x for x in b.signi if x=='FAM138A']]=True
```

Example 72 (python):
```python
Genes=['COL4A1','COL4A2','H19','COL15A1','ODGFB','ADGRF5','IL32','RGCC','ETS1','KDR','FABP5','ADAMTS9','MYO1B','A2M','ADGRL4','NR2F2','NTS','IL1RL1','ROBO1','RASSF9','KCNIP4','APLNR','FLRT2','TSHZ2','MYRIP','PDE3A','EFNA5','BMP6','SLIT3','SERPINE2','GJA5','SRGN','JAG1','TMEM100','EGFL8','PCSK5','CXCL12','IL33','TSPO','MEOX2','CCL14','APLNR','PRCP','LDB2','FLRT2','COL3A1','CPE','PCDH7','BMPER','HMCN1','CLU','PLXNA4','MEIS2','FBLN2','NRG3','CPXM2','ELN','SULF1','MMP16','FBLN5','DKK2','LTBP4','BGN','LTC4S','MMP2','HEY2','DKK','KCRD12','SULT1E1']
SignificantList=['HES4','VWA1','PRDM16-DT','PRDM16','RERE','CTNNBIP1','RBP7','DHRS3','FBLIM1','MFAP2','ECE1','HSPG2','ID3','CLIC4','STMN1','IFI6','MARCKSL1','GJA4','MACF1','YBX1','RPS8','PIK3R3','EFCAB14','PLPP3','JUN','NFIA','ROR1','JAK1','LEPR','PDE4B','GNG12','NEGR1','SLC44A5','ST6GALNAC3','GIPC2','ADGRL4','ADGRL2','DDAH1','CCN1','TGFBR3','EVI5','FNBP1L','ARHGAP29','CNN3','PALMD','COL11A1','VAV3','FAM102B','RHOC','TSPAN2','CD58','TXNIP','GJA5','C1orf54','S100A11','S100A6','S100A16','ADAM15','EFNA1','FDPS','NES','IFI16','CADM3','ACKR1','DUSP23','TAGLN2','LINC01133','RGS5','PBX1','RCSD1','DNM3','RASAL2','QSOX1','LAMC1','ARPC5','C1orf21','IVNS1ABP','HMCN1','PTGS2','KCNT2','CFH','MIR181A1HG','NR5A2','NAV1','BTG2','PIK3C2B','CD55','CD34','PLXNA2','PTPN14','CAPN2','H3F3A','ITPKB','FAM89A','EGLN1','DISC1','SIPA1L2','LYST','CHRM3','PXDN','RPS7','SOX11','RNF144A','ID2','ASAP2','RHOB','GDF7','ITSN2','PPP1CB','LBH','LTBP1','RASGRP3','CRIM1','ATL2','SRSF7','SLC8A1','EML4','ZFP36L2','PRKCE','EPAS1','CALM2','SPTBN1','RPS27A','CCDC88A','CFAP36','CCDC85A','DYSF','EXOC6B','TMSB10','MAP4K4','IL1R1','SH3RF3','CLASP1','GYPC','NCKAP5','MGAT5','CXCR4','MBD5','ARL5A','FMNL2','PKP4','FIGN','COBLL1','NOSTRIN','ITGA6','ITPRID2','AC007319.1','CALCRL','TFPI','GULP1','COL3A1','COL5A2','HIBCH','GLS','MYO1B','CAVIN2','HECW2','PLCL1','BMPR2','RAPH1','PARD3B','NRP2','EEF1B2','FN1','IGFBP2','IGFBP5','EPHA4','SERPINE2','PTMA','NPPC','DGKD','AGAP1','ACKR3','LRRFIP1','ITPR1','LMCD1','VGLL4','RPL32','FBLN2','RFTN1','THRB','SLC4A7','RBMS3','TGFBR2','RPSA','MYRIP','CTNNB1','SNRK','CLEC3B','LIMD1','MAP4','SEMA3F','GNAI2','HYAL2','RPL29','SEMA3G','ARHGEF3','FHIT','PTPRG','ADAMTS9','MAGI1','FRMD4B','FOXP1','CNTN3','ROBO2','ROBO1','MTRNR2L12','CRYBG3','FILIP1L','ABI3BP','NFKBIZ','ALCAM','CBLB','CD47','PHLDB2','SLC9C1','CCDC80','ZBTB20','LSAMP','AC092691.1','FSTL1','KALRN','HEG1','MGLL','TRH','SLCO2A1','PLSCR4','TM4SF18','TM4SF1','WWTR1','TSC22D2','MBNL1','MECOM','SKIL','PLD1','FNDC3B','NLGN1','NAALADL2','YEATS2','ST6GAL1','HES1','ATP13A3','CYTL1','LDB2','LAP3','KCNIP4','STIM2','PCDH7','KLHL5','RPL9','ATP8A1','OCIAD2','KIT','KDR','IGFBP7','ADGRL3','SULT1B1','SULT1E1','AFF1','SPARCL1','MMRN1','CCSER1','UNC5C','EMCN','PPP3CA','DKK2','COL25A1','CFI','CAMK2D','USP53','FAT4','MGST2','MAML3','INPP4B','SMAD1','SLC10A7','NR3C2','LRBA','RPS3A','GUCY1A1','GRIA2','GASK1B','FNIP2','RAPGEF2','CPE','TLL1','PALLD','SH3RF1','GPM6A','VEGFC','STOX2','PDLIM3','SORBS2','SEMA5A','TRIO','FBXL7','MYO10','MTMR12','EGFLAM','LIFR','GHR','ITGA1','ITGA2','ARL15','ESM1','PLPP1','MAP3K1','PLK2','RAB3C','PDE4D','ADAMTS6','MAST4','MAP1B','ARHGEF28','RPS23','VCAN','MEF2C','MCTP1','LNPEP','ST8SIA4','EFNA5','NREP','EPB41L4A','SEMA6A','LOX','SNCAIP','SLC12A2','FBN2','HINT1','SKP1','SPOCK1','EGR1','ECSCR','PCDH12','SPRY4','ARHGAP26','DPYSL3','ABLIM3','AFAP1L1','CSNK1A1','RPS14','SYNPO','SPARC','GALNT10','SGCD','ADAM19','EBF1','SLIT3','RANBP17','FGF18','NEURL1B','DUSP1','COL23A1','LTC4S','BTNL9','FOXC1','GMDS','SERPINB9','SERPINB6','BMP6','TXNDC5','NEDD9','EDN1','GFOD1','KIF13A','RNF144B','CASC15','CARMIL1','HLA-A','HLA-E','TUBB','IER3','HLA-B','CLIC1','EGFL8','NOTCH4','RPS10','RPL10A','PI16','CLIC5','ADGRF5','LRRC1','DST','EYS','ADGRB3','EEF1A1','AL357507.1','COL12A1','FILIP1','MYO6','PHIP','HMGN3','TENT5A','SNX3','AMD1','REV3L','FYN','MARCKS','MAN1A1','GJA1','NKAIN2','HEY2','NCOA7','RSPO3','AL356124.1','ARHGAP18','CCN2','RPS12','SGK1','PDE7B','ADGRG6','UTRN','SAMD5','AL033504.1','SASH1','PLEKHG1','MTHFD1L','AKAP12','SYNE1','MYCT1','TIAM2','ARID1B','SNX9','SYNJ2','IGF2R','RPS6KA2','AFDN','ACTB','FSCN1','NDUFA4','THSD7A','ARL4A','MEOX2','TSPAN13','AHR','RAPGEF5','AC004540.1','SKAP2','CREB5','AQP1','BMPER','ELMO1','RALA','INHBA','RAMP3','IGFBP3','GRB10','AUTS2','ELN','HSPB1','MAGI2','GNAI1','CD36','SEMA3C','CACNA2D1','CDK14','CDK6','GNG11','COL1A2','PPP1R9A','ARPC1B','SERPINE1','RELN','LAMB1','IMMP2L','DOCK4','MDFIC','CAV2','CAV1','MET','CAPZA2','CADPS2','PODXL','PLXNA4','CALD1','PTN','GIMAP7','GIMAP4','GIMAP1','ESYT2','PRAG1','SOX7','GATA4','DLC1','TUSC3','MTUS1','CSGALNACT1','LOXL2','PPP2R2A','DPYSL2','CLU','HMBOX1','SARAF','LEPROTL1','RBPMS','NRG1','RAB11FIP1','TCIM','SFRP1','PCMTD1','SOX17','XKR4','LYN','TOX','CHD7','CLVS1','PREX2','SULF1','KCNB2','RPL7','HEY1','FABP5','FABP4','MMP16','SLC26A7','RUNX1T1','RPL30','RNF19A','PABPC1','AZIN1','CTHRC1','ZFPM2','OXR1','EXT1','SAMD12','SNTB1','HAS2','MTSS1','PCAT1','PVT1','ASAP1','NDRG1','PTK2','KANK1','IL33','AL583785.1','FREM1','BNC2','ADAMTSL1','RPS6','MLLT3','TEK','B4GALT1','IL11RA','TPM2','CNTNAP3B','PGM5','TRPM3','ALDH1A1','PCSK5','GNA14','TLE1','DAPK1','OMD','AOPEP','COL15A1','SEC61B','KLF4','CTNNAL1','PALM2-AKAP2','RGS3','GSN','STOM','PTGS1','DENND1A','HSPA5','SH2D3C','ENG','SPTAN1','FNBP1','RAPGEF1','COL5A1','EGFL7','DIPK1B','PTGDS','NPDC1','KLF6','NET1','SFMBT2','CELF2','CAMK1D','FRMD4A','FAM107B','VIM','ST8SIA6','MRC1','KIAA1217','MPP7','SVIL','JCAD','MAP3K8','NRP1','PARD3','CCNY','CXCL12','DEPP1','PRKG1','BICC1','ARID5B','JMJD1C','SRGN','PPA1','PALD1','UNC5B','CDH23','VSIR','PSAP','DDIT4','MCU','OIT3','VCL','LRMDA','RPS24','PLAC9','PRXL2A','NRG3','MMRN2','EXOC6','MYOF','PDLIM1','SORBS1','ENTPD1','ENTPD1-AS1','SFRP5','CRTAC1','SH3PXD2A','ADD3','PDCD4','ATRNL1','GRK5','PLEKHA1','HTRA1','CPXM2','CHST15','EBF3','IFITM2','IFITM3','CD151','CTSD','TNNT3','H19','IGF2','HBG2','CAVIN3','SBF2','GALNT18','DKK3','MICAL2','SOX6','RPS13','LGR4','CD59','LMO2','LDLRAD3','LRRC4C','HSD17B12','TSPAN18','TP53I11','MDK','PTPRJ','APLNR','CTNND1','FTH1','NEAT1','MALAT1','LTBP3','CFL1','EFEMP2','CCDC85B','GSTP1','CCND1','ANO1','FCHSD2','SERPINH1','LRRC32','AAMDC','NDUFC2','GAB2','PRCP','DLG2','TMEM135','SESN3','MAML2','PDGFD','SIK2','NCAM1','CADM1','FXYD6','MPZL2','MCAM','THY1','MIR100HG','ESAM','ROBO4','PKNOX2','ETS1','FLI1','CACNA1C','ANO2','VWF','CD9','GAPDH','PTMS','A2M','YBX3','LRP6','EMP1','MGP','PDE3A','LDHB','SOX5','BCAT1','ITPR2','STK38L','PPFIBP1','TMTC1','FGD4','PRICKLE1','DHH','TUBA1B','TUBA1A','FMNL3','LIMA1','ACVRL1','NR4A1','ITGA5','RPS26','MYL6','NACA','LRIG3','MSRB3','IRAK3','PTPRB','PHLDA1','NAV3','TMTC2','RASSF9','NTS','KITLG','BTG1','NUDT4','ELK3','HSP90B1','NUAK1','ALDH2','RPL6','NOS1','PXN','UBC','RPL21','FLT1','MEDAG','POSTN','LHFPL6','FOXO1','RGCC','DGKH','ENOX1','TSC22D1','LRCH1','ITM2B','LPAR6','FNDC3A','DLEU2','PCDH17','AL159156.1','DACH1','KCTD12','EDNRB','POU4F1','ABCC4','DOCK9','FGF14','EFNB2','ARGLU1','FAM155A','COL4A1','COL4A2','ARHGEF7','ATP11A','MCF2L','GAS6','RNASE1','DAD1','STXBP6','NPAS3','CLEC14A','GNG2','NID2','GPR137C','SAMD4A','FBXO34','PELI2','PRKCH','SYNE2','FUT8','GPHN','ACTN1','RGS6','NUMB','LTBP2','PGF','FOS','VASH1','FLRT2','CALM1','RPS6KA5','CCDC88C','FBLN5','MEG3','MEG8','JAG2','CRIP2','CRIP1','SNHG14','FAM189A1','TJP1','MEIS2','THBS1','SRP14','DLL4','SERF2','B2M','SEMA6D','FBN1','CGNL1','MYO1E','RORA','TPM1','HERC1','SMAD6','GLCE','UACA','PEAK1','RPS17','PDE8A','AKAP13','HAPLN3','SLCO3A1','LINC01197','AC012409.2','NR2F2-AS1','NR2F2','MEF2A','CHSY1','RPS2','NPW','SLC9A3R2','IL32','RPS15A','ACSM3','NETO2','CHD9','MMP2','LPCAT2','CDH11','CDH5','NFAT5','GLG1','GABARAPL2','ADAMTS18','PLCG2','CDH13','COTL1','GSE1','CYBA','TUBB3','RFLNB','NXN','ABR','HIC1','ATP2A3','PFN1','RNASEK','TMEM88','RPL26','PIK3R6','NTN1','UBB','SNHG29','SPECC1','RPL23A','SSH2','MYO1D','MMP28','CCL14','RPL23','RPL19','IGFBP4','EIF1','JUP','RAMP2','MEOX1','GRN','NPEPPS','COL1A1','TMEM100','DGKE','CA4','MRC2','TANC2','ACE','ICAM2','PECAM1','PITPNC1','ABCA8','H3F3B','SEC14L1','SOCS3','ACTG1','TBCD','COLEC12','YES1','MYL12A','MYL12B','DLGAP1','EPB41L3','ARHGAP28','PTPRM','LDLRAD4','GATA6','ZNF521','SETBP1','RPL17','TCF4','RNF152','BCL2','CBLN2','NFATC1','GPX4','CIRBP','ABHD17A','GADD45B','TNFAIP8L1','INSR','CD320','KANK3','CDC37','DOCK6','JUNB','NFIX','LYL1','IER2','AC020916.1','KLF2','CPAMD8','PLVAP','BST2','RPL18A','ACTN4','GMFG','ZFP36','PLD3','LTBP4','PCAT19','BCAM','FOSB','SELENOW','CARD8','EMP3','FTL','RPL13A','RPS11','RRAS','MYADM','TCF15','PRND','RASSF2','PLCB1','PLCB4','JAG1','MACROD2','DSTN','RALGAPA2','CD93','CST3','ID1','PROCR','MYL9','TGM2','PPP1R16B','TOX2','SULF2','PREX1','PTGIS','TSHZ2','PMEPA1','GNAS','ATP5F1E','COL9A3','SOX18','MIR99AHG','JAM2','ATP5PF','APP','CYYR1','LINC01695','EVA1C','ITSN1','ETS2','COL18A1','CLDN5','TBX1','COMT','MIF','ADORA2A','SMTN','YWHAH','TIMP3','MYH9','LGALS1','PDGFB','RPL3','TSPO','SHANK3','CD99','MXRA5','FRMPD4','TMSB4X','BMX','NHS','SAT1','IL1RAPL1','DMD','TSPAN7','DIPK2B','TIMP1','RPS4X','XIST','FTX','ITM2A','DIAPH2','NRK','PLS3','DANT2','KLHL13','APLN','FHL1','FGF13','BGN','RPL10','MT-ND1','MT-ND2','MT-CO1','MT-CO2','MT-ATP8','MT-ATP6','MT-CO3','MT-ND3','MT-ND4','MT-ND5','MT-CYB']
Genes=list(set(Genes)&set(SignificantList))
g1=scf.pl.trends(adata,
                 highlight_features=Genes,
                 figsize=(4,20),
                 n_features=30,
                 annot='fine_grain',
                 root_milestone="0",
                 milestones=["2"],
                 #branch="Ery",
                 plot_emb=False,ordering="max",return_genes=True)
```

Example 73 (python):
```python
set(adata.obs.fine_grain)
```

Example 74 (unknown):
```unknown
{'CoronaryArterialEndothelialCells',
 'CoronaryCapillaryEndothelialCells',
 'CoronaryVenousEndothelialCells',
 'GreatVesselArterialEndothelialCells',
 'GreatVesselVenousEndothelialCells'}
```

Example 75 (python):
```python
ClusterOfInterest='CoronaryCapillaryEndothelialCells'
#sc.tl.rank_genes_groups(adata, 'fine_grain', method='t-test', key_added = "t-test")
DEGenesForCluster = [x[ClusterOfInterest] for x in adata.uns['t-test']['names'][:50]]
sc.pl.dotplot(adata, DEGenesForCluster ,groupby='fine_grain',standard_scale='var',cmap='PRGn')
#sc.pl.umap(adata, color=DEGenesForCluster)
```

Example 76 (unknown):
```unknown
/home/sinhalab/miniconda3/envs/scFates/lib/python3.8/site-packages/scanpy/plotting/_dotplot.py:749: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap', 'norm' will be ignored
  dot_ax.scatter(x, y, **kwds)
```

Example 77 (unknown):
```unknown
AllGenes=['COL4A1','COL4A2','H19','COL15A1','ODGFB','ADGRF5','IL32','RGCC','ETS1','KDR','FABP5','ADAMTS9','MYO1B','A2M','ADGRL4','NR2F2','NTS','IL1RL1','ROBO1','RASSF9','KCNIP4','APLNR','FLRT2','TSHZ2','MYRIP','PDE3A','EFNA5','BMP6','SLIT3','SERPINE2','GJA5','SRGN','JAG1','TMEM100','EGFL8','PCSK5','CXCL12','IL33','TSPO','MEOX2','CCL14','APLNR','PRCP','LDB2','FLRT2','COL3A1','CPE','PCDH7','BMPER','HMCN1','CLU','PLXNA4','MEIS2','FBLN2','NRG3','CPXM2','ELN','SULF1','MMP16','FBLN5','DKK2','LTBP4','BGN','LTC4S','MMP2','HEY2','DKK','KCRD12','SULT1E1']
GeneList=['COL4A1','COL4A2','H19','COL15A1','ADGRF5','IL32','RGCC','ETS1','KDR','FABP5','ADAMTS9','MYO1B','A2M','ADGRL4','NR2F2','NTS','ROBO1','RASSF9','KCNIP4','APLNR','FLRT2','TSHZ2','MYRIP','PDE3A','EFNA5','BMP6','SLIT3','SERPINE2','GJA5','SRGN','JAG1','TMEM100','EGFL8','PCSK5','CXCL12','IL33','TSPO','MEOX2','CCL14','APLNR','PRCP','LDB2','FLRT2','COL3A1','CPE','PCDH7','BMPER','HMCN1','CLU','PLXNA4','MEIS2','FBLN2','NRG3','CPXM2','ELN','SULF1','MMP16','FBLN5','DKK2','LTBP4','BGN','LTC4S','MMP2','HEY2','SULT1E1']
```

Example 78 (python):
```python
metadata=[]
for k in range(0,len(adata)):
    a=adata.obs_names[k]+' , '+str(adata.obs.t[k])+' , '+adata.obs.fine_grain[k]
    metadata.append(a)
    
b=pd.DataFrame(metadata)
b.to_csv('EndothelialTime.csv')
```

Example 79 (python):
```python
print(adata)
adata=adata[:,[x for x in adata.var_names if x in AllGenes]].copy()
print(adata)
```

Example 80 (unknown):
```unknown
AnnData object with n_obs × n_vars = 8359 × 1450
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score'
    var: 'gene_name_scRNA-0', 'gene_id', 'highly_variable', 'means', 'dispersions', 'dispersions_norm', 'highly_variable_nbatches', 'highly_variable_intersection'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'age_colors', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'cycling_colors', 'donor_colors', 'fine_grain_colors', 'heart_or_greatvessels_colors', 'hvg', 'kit_10x_colors', 'mid_grain_colors', 'modality_colors', 'neighbors', 'phase_colors', 'region_colors', 'sex_colors', 'trimester_colors', 'umap', 'week_colors'
    obsm: 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    layers: 'counts'
    obsp: 'connectivities', 'distances'
AnnData object with n_obs × n_vars = 8359 × 56
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score'
    var: 'gene_name_scRNA-0', 'gene_id', 'highly_variable', 'means', 'dispersions', 'dispersions_norm', 'highly_variable_nbatches', 'highly_variable_intersection'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'age_colors', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'cycling_colors', 'donor_colors', 'fine_grain_colors', 'heart_or_greatvessels_colors', 'hvg', 'kit_10x_colors', 'mid_grain_colors', 'modality_colors', 'neighbors', 'phase_colors', 'region_colors', 'sex_colors', 'trimester_colors', 'umap', 'week_colors'
    obsm: 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    layers: 'counts'
    obsp: 'connectivities', 'distances'
```

Example 81 (python):
```python
adata.write('./EndothelialCellsForscFatesVince.h5ad')
```

---

## Import modules¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/RNA/QC/Foetal_scRNA_include-doublets.html

**Contents:**
- Import modules¶
- Functions¶
- Read in anndata¶
- Remove low quality cells¶
- Compute QC stats¶
- ~Remove cells exceeding the ribo/mito threshold~¶
- Calculate RBC score¶
- Remove RBCs¶
- Save¶

**Examples:**

Example 1 (python):
```python
import numpy as np
import pandas as pd
import scanpy as sc
import matplotlib.pyplot as plt
import scrublet as scr
import session_info
```

Example 2 (python):
```python
sc.settings.verbosity = 3
sc.settings.set_figure_params(dpi = 120, color_map = 'RdPu', dpi_save = 180, vector_friendly = True, format = 'svg')
session_info.show()
```

Example 3 (python):
```python
-----
matplotlib          3.3.4
numpy               1.20.1
pandas              1.1.4
scanpy              1.7.1
scrublet            NA
session_info        1.0.0
-----
```

Example 4 (unknown):
```unknown
PIL                 8.1.2
anndata             0.7.5
anyio               NA
attr                20.3.0
babel               2.9.0
backcall            0.2.0
brotli              NA
cairo               1.20.0
certifi             2020.12.05
cffi                1.14.5
chardet             4.0.0
cloudpickle         1.6.0
colorama            0.4.4
cycler              0.10.0
cython_runtime      NA
cytoolz             0.11.0
dask                2021.03.1
dateutil            2.8.1
decorator           4.4.2
fsspec              0.8.7
get_version         2.1
google              NA
h5py                2.10.0
idna                2.10
igraph              0.8.3
ipykernel           5.5.0
ipython_genutils    0.2.0
jedi                0.18.0
jinja2              2.11.3
joblib              1.0.1
json5               NA
jsonschema          3.2.0
jupyter_server      1.4.1
jupyterlab_server   2.3.0
kiwisolver          1.3.1
legacy_api_wrap     0.0.0
leidenalg           0.8.3
llvmlite            0.34.0
louvain             0.7.0
markupsafe          1.1.1
mpl_toolkits        NA
natsort             7.1.1
nbclassic           NA
nbformat            5.1.2
numba               0.51.2
numexpr             2.7.3
packaging           20.9
parso               0.8.1
pexpect             4.8.0
pickleshare         0.7.5
pkg_resources       NA
prometheus_client   NA
prompt_toolkit      3.0.16
psutil              5.8.0
ptyprocess          0.7.0
pvectorc            NA
pyarrow             0.16.0
pygments            2.8.0
pyparsing           2.4.7
pyrsistent          NA
pytoml              NA
pytz                2021.1
requests            2.25.1
ruamel              NA
scipy               1.6.1
send2trash          NA
setuptools_scm      NA
simplejson          3.17.2
sinfo               0.3.1
six                 1.15.0
sklearn             0.22
sniffio             1.2.0
socks               1.7.1
sphinxcontrib       NA
storemagic          NA
tables              3.6.1
tblib               1.7.0
texttable           1.6.3
tlz                 0.11.0
toolz               0.11.1
tornado             6.1
traitlets           5.0.5
typing_extensions   NA
uritemplate         3.0.1
urllib3             1.26.3
wcwidth             0.2.5
yaml                5.1.2
zmq                 22.0.3
```

Example 5 (unknown):
```unknown
-----
IPython             7.21.0
jupyter_client      6.1.11
jupyter_core        4.7.1
jupyterlab          3.0.9
notebook            6.2.0
-----
Python 3.8.8 | packaged by conda-forge | (default, Feb 20 2021, 16:22:27) [GCC 9.3.0]
Linux-4.15.0-213-generic-x86_64-with-glibc2.10
-----
Session information updated at 2023-12-12 15:07
```

Example 6 (python):
```python
adata = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/QC/scRNA_foetal_prefilter.h5ad')
adata
```

Example 7 (unknown):
```unknown
AnnData object with n_obs × n_vars = 205457 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval'
    var: 'gene_name', 'feature_type', 'ambient_expression-0', 'ambient_expression-1', 'ambient_expression-10', 'ambient_expression-11', 'ambient_expression-12', 'ambient_expression-13', 'ambient_expression-14', 'ambient_expression-15', 'ambient_expression-16', 'ambient_expression-17', 'ambient_expression-18', 'ambient_expression-19', 'ambient_expression-2', 'ambient_expression-3', 'ambient_expression-4', 'ambient_expression-5', 'ambient_expression-6', 'ambient_expression-7', 'ambient_expression-8', 'ambient_expression-9'
    obsm: 'latent_gene_encoding'
```

Example 8 (python):
```python
adata.obs['sangerID'].value_counts()
```

Example 9 (unknown):
```unknown
Sinha_F2             24275
Sinha_E2             24227
BHF_F_Hea11192328    21714
Sinha_E8             17219
BHF_F_Hea11192324    16688
BHF_F_Hea11192326    14117
BHF_F_Hea11192325    10676
Sinha_F4             10652
BHF_F_Hea11192327     9130
BHF_F_Hea10402918     8010
Sinha_F7              6973
Sinha_D8              6714
Sinha_A1              6640
Sinha_B1              5988
BHF_F_Hea11192323     4893
Sinha_C8              4587
BHF_F_Hea10402917     3752
Sinha_G4              3389
Sinha_F8              3203
Sinha_G7              2610
Name: sangerID, dtype: int64
```

Example 10 (python):
```python
sc.pp.filter_cells(adata, min_genes=500, inplace=True)
sc.pp.filter_cells(adata, min_counts=1000, inplace=True)
print(adata.shape)
```

Example 11 (unknown):
```unknown
filtered out 62633 cells that have less than 500 genes expressed
filtered out 2190 cells that have less than 1000 counts
```

Example 12 (unknown):
```unknown
(140634, 36601)
```

Example 13 (python):
```python
adata.obs['sangerID'].value_counts()
```

Example 14 (unknown):
```unknown
BHF_F_Hea11192328    15321
BHF_F_Hea11192326    13251
BHF_F_Hea11192324    13229
Sinha_E2             11525
Sinha_E8             10036
BHF_F_Hea11192325     9421
Sinha_F2              9259
Sinha_F4              9215
BHF_F_Hea11192327     7387
Sinha_D8              6107
BHF_F_Hea10402918     5734
Sinha_F7              5547
Sinha_A1              4671
BHF_F_Hea11192323     4000
Sinha_C8              3981
Sinha_B1              3606
Sinha_G4              2604
BHF_F_Hea10402917     2517
Sinha_G7              1779
Sinha_F8              1444
Name: sangerID, dtype: int64
```

Example 15 (python):
```python
adata.var['mt'] = adata.var['gene_name'].str.startswith('MT-')  # annotate the group of mitochondrial genes as 'mt'
adata.var['ribo'] = adata.var['gene_name'].str.startswith('RPS','RPL')  # annotate the group of ribosomal genes as 'mt'
sc.pp.calculate_qc_metrics(adata, qc_vars=['mt','ribo'], percent_top=None, log1p=True, inplace=True)
```

Example 16 (python):
```python
plt.rcParams['figure.figsize'] = [3, 2]

print('### Sanger ###')
fig,axs=plt.subplots(nrows=1,ncols=3,figsize=(10,3))
sc.pl.scatter(adata[adata.obs['facility']=='Sanger'], x='total_counts', y='n_genes', show=False, ax=axs[0])
sc.pl.scatter(adata[adata.obs['facility']=='Sanger'], x='total_counts', y='pct_counts_mt', show=False, ax=axs[1])
sc.pl.scatter(adata[adata.obs['facility']=='Sanger'], x='total_counts', y='pct_counts_ribo', show=False, ax=axs[2])

fig.tight_layout()
plt.show()

plt.rcParams['figure.figsize'] = [3, 2]

print('### CSCI ###')
fig,axs=plt.subplots(nrows=1,ncols=3,figsize=(10,3))
sc.pl.scatter(adata[adata.obs['facility']=='CSCI'], x='total_counts', y='n_genes', show=False, ax=axs[0])
sc.pl.scatter(adata[adata.obs['facility']=='CSCI'], x='total_counts', y='pct_counts_mt', show=False, ax=axs[1])
sc.pl.scatter(adata[adata.obs['facility']=='CSCI'], x='total_counts', y='pct_counts_ribo', show=False, ax=axs[2])

fig.tight_layout()
plt.show()
```

Example 17 (unknown):
```unknown
### Sanger ###
```

Example 18 (unknown):
```unknown
### CSCI ###
```

Example 19 (python):
```python
'''
plt.rcParams['figure.figsize'] = [2, 2]
adata = adata[adata.obs['pct_counts_mt'] < 20, :]
adata = adata[adata.obs['pct_counts_ribo'] < 20, :]
sc.pl.violin(adata, ['n_genes', 'total_counts','log1p_total_counts', 'pct_counts_mt', 'pct_counts_ribo'],jitter = 0.2, multi_panel = True)
'''
```

Example 20 (python):
```python
'''
sc.settings.set_figure_params(dpi = 120, color_map = 'RdPu')
plt.rcParams['figure.figsize'] = [8, 2]
sc.pl.violin(adata,keys=['log1p_total_counts'],groupby='sangerID',rotation=90,jitter=False)
'''
```

Example 21 (python):
```python
# print(adata.shape)
# adata.obs['sangerID'].value_counts()
```

Example 22 (python):
```python
adata_c = adata.copy()

adata_c.var.set_index('gene_name',inplace=True)
adata_c.var_names = adata_c.var_names.astype('str')
adata_c.var_names_make_unique()

sc.pp.filter_genes(adata_c, min_cells=3)
sc.pp.normalize_total(adata_c, target_sum=1e4)
sc.pp.log1p(adata_c)
```

Example 23 (unknown):
```unknown
filtered out 5009 genes that are detected in less than 3 cells
normalizing counts per cell
    finished (0:00:01)
```

Example 24 (python):
```python
# scoring hemoglobin genes
# HGNC, Hemoglobin subunits, groupID:940
genes=['HBA1','HBA2','HBB','HBD','HBE1','HBG1','HBG2','HBM','HBQ1','HBZ']
sc.tl.score_genes(adata_c,gene_list=genes,score_name='HB_score')
```

Example 25 (python):
```python
computing score 'HB_score'
    finished: added
    'HB_score', score of gene set (adata.obs).
    250 total control genes are used. (0:00:05)
```

Example 26 (python):
```python
len(adata_c.obs['sangerID'].unique())
```

Example 27 (python):
```python
sc.settings.set_figure_params(dpi = 80, color_map = 'RdPu', dpi_save = 180, vector_friendly = True, format = 'svg')

fig,axs=plt.subplots(nrows=4,ncols=5,figsize=(15,7))

for i,ID in enumerate(adata_c.obs['sangerID'].unique()):
    sample = adata_c[adata_c.obs['sangerID']==ID]
    axs[i//5,i%5].hist(sample.obs['HB_score'],bins=100)
    axs[i//5,i%5].set_title(ID)

fig.tight_layout()
plt.show()
```

Example 28 (python):
```python
# add HB score to original adata
adata.obs.loc[adata_c.obs_names,'HB_score']=adata_c.obs['HB_score']
```

Example 29 (python):
```python
HBscore_thresh=2

adata=adata[adata.obs['HB_score']<HBscore_thresh]
print(adata.shape)
```

Example 30 (unknown):
```unknown
(121458, 36601)
```

Example 31 (python):
```python
# save
adata.write('/nfs/team205/heart/anndata_objects/Foetal/QC/scRNA_foetal_include-doublets_13Dec2023.h5ad')
adata
```

Example 32 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 121458 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score'
    var: 'gene_name', 'feature_type', 'ambient_expression-0', 'ambient_expression-1', 'ambient_expression-10', 'ambient_expression-11', 'ambient_expression-12', 'ambient_expression-13', 'ambient_expression-14', 'ambient_expression-15', 'ambient_expression-16', 'ambient_expression-17', 'ambient_expression-18', 'ambient_expression-19', 'ambient_expression-2', 'ambient_expression-3', 'ambient_expression-4', 'ambient_expression-5', 'ambient_expression-6', 'ambient_expression-7', 'ambient_expression-8', 'ambient_expression-9', 'mt', 'ribo', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts'
    obsm: 'latent_gene_encoding'
```

Example 33 (python):
```python
adata.obs['sangerID'].value_counts()
```

Example 34 (unknown):
```unknown
BHF_F_Hea11192328    15280
BHF_F_Hea11192326    13229
BHF_F_Hea11192324    13207
BHF_F_Hea11192325     9418
Sinha_E8              8789
Sinha_F4              8655
BHF_F_Hea11192327     7381
BHF_F_Hea10402918     5675
Sinha_D8              5431
Sinha_F7              5157
Sinha_F2              4874
Sinha_E2              4377
BHF_F_Hea11192323     3998
Sinha_C8              3387
Sinha_A1              2921
BHF_F_Hea10402917     2510
Sinha_B1              2330
Sinha_G4              2250
Sinha_G7              1494
Sinha_F8              1095
Name: sangerID, dtype: int64
```

---

## Read in adata¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig2_Overview_of_cellular_niches_and_TissueTypist/tissuetypist_Xenium-5K_TissueTypist-prediction_plotting.html

**Contents:**
- Read in adata¶
- Log-normalisation¶
- Plot cell types¶
- UMAP¶
- TissueTypist niches¶

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os

import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42
```

Example 2 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 3 (unknown):
```unknown
os.getcwd()
```

Example 4 (unknown):
```unknown
'/rfs/project/rfs-iCNyzSAaucw/kk837/notebooks/Foetal/Xenium/5K/C194_Hst45/CamU'
```

Example 5 (unknown):
```unknown
figdir = '/rfs/project/rfs-iCNyzSAaucw/kk837/notebooks/Foetal/fetal_ms_figures'
# table_dir = '/nfs/team205/heart/fetal_ms_figures/tables/Aug2024'
```

Example 6 (unknown):
```unknown
celltype_key = 'fine_grain_12Jun2025'
```

Example 7 (python):
```python
objects_dir = '/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium'
path_adata = f'{objects_dir}/C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat_5K_filtered_raw.h5ad'
adata = sc.read_h5ad(path_adata)
adata
```

Example 8 (unknown):
```unknown
AnnData object with n_obs × n_vars = 362277 × 5001
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_3.0', 'multi_celltypes_coarse', 'fine_grain_12Jun2025', 'sliding_window_assignment', 'window_col', 'window_row', 'globalX', 'globalY', 'tt_prediction'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_mid_fin_colors', 'dendrogram_pca_harmony_leiden_1.0', 'fine_grain_12Jun2025_colors', 'multi_celltypes_coarse_colors', 'pca', 'pca_harmony', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.2_colors', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.3_colors', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.4_colors', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.5_colors', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_0.8_colors', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.0_colors', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_1.5_colors', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_2.0_colors', 'pca_harmony_leiden_3.0', 'pca_harmony_leiden_3.0_colors', 'tissue_block_id_colors', 'umap'
    obsm: 'X_pca', 'X_pca_harmony', 'X_umap_pca_harmony', 'spatial'
    varm: 'PCs'
    obsp: 'pca_harmony_connectivities', 'pca_harmony_distances'
```

Example 9 (unknown):
```unknown
os.getcwd()
```

Example 10 (unknown):
```unknown
'/rfs/project/rfs-iCNyzSAaucw/kk837/notebooks/Foetal/Xenium/5K/C194_Hst45/CamU'
```

Example 11 (python):
```python
adata.X.data[:5]
```

Example 12 (unknown):
```unknown
array([5., 1., 1., 2., 2.], dtype=float32)
```

Example 13 (python):
```python
# celltype name mapping for plotting
df = pd.read_csv('/rfs/project/rfs-iCNyzSAaucw/kk837/notebooks/Foetal/finegrain_name_mapping.csv')
celltype_mapping = df.set_index('Full_name')['Short_name_finalised'].to_dict()
adata.obs.replace({celltype_key:celltype_mapping},inplace=True)
```

Example 14 (python):
```python
/tmp/ipykernel_968450/407166641.py:4: FutureWarning: The behavior of Series.replace (and DataFrame.replace) with CategoricalDtype is deprecated. In a future version, replace will only be used for cases that preserve the categories. To change the categories, use ser.cat.rename_categories instead.
  adata.obs.replace({celltype_key:celltype_mapping},inplace=True)
```

Example 15 (python):
```python
# log-normalise the count
sc.pp.normalize_total(adata,target_sum=1e4)
sc.pp.log1p(adata)
adata.X.data[:5]
```

Example 16 (unknown):
```unknown
array([4.2093534, 2.6576407, 2.6576407, 3.3151019, 3.3151019],
      dtype=float32)
```

Example 17 (python):
```python
set(adata.obs[celltype_key])
```

Example 18 (unknown):
```unknown
{'?EpendymalCells',
 '?EpithelialCells',
 '?SkeletalMuscle',
 '?ThyroidCells',
 'AVNPC',
 'AtrialCardiomyocytes_DoubletWithEC',
 'AtrialCardiomyocytes_DoubletWithFB',
 'B',
 'ChromC',
 'CorArtEC',
 'CorCapEC',
 'CorCapECCyc',
 'CorCapECN',
 'CorPeri',
 'CorSMC',
 'CorVenEC',
 'CoronaryCapillaryEndothelialCells_DoubletsWithEryth',
 'DASMC',
 'DC',
 'ELNeuro',
 'EPDC',
 'EndoCush',
 'Endocard',
 'EndocardCyc',
 'EndocardialCells_Doublets',
 'Eryth',
 'ErythCyc',
 'FB',
 'FBCOL2A1',
 'FBCyc',
 'GVArtEC',
 'GVSMC',
 'GVSMCCyc',
 'GVVenEC',
 'ILC',
 'LymphEC',
 'MPLYVE1',
 'MacCX3',
 'MastC',
 'MegaK',
 'MesEpiC',
 'Mono',
 'MyeloidCells_Doublets',
 'NeuroPre',
 'ParaN',
 'PericCyc',
 'PericFib',
 'PericInt',
 'SANPC',
 'SOX2_LAMB3_pos',
 'SchwC',
 'SchwCyc',
 'SchwP',
 'SchwannCells_DoubletsWithNeurons',
 'SympN',
 'TCD4',
 'TCD8',
 'ThyroidEpithelialCells',
 'VCSDist',
 'ValveEC',
 'aCMCyc',
 'aCML',
 'aCMR',
 'doublets',
 'unassigned',
 'unclassified',
 'vCM',
 'vCMCyc'}
```

Example 19 (python):
```python
tissue_id = 'C194-HEA-0-FFPE-1'
slide = adata[adata.obs['tissue_block_id']==tissue_id]

sc.pl.spatial(slide,color=[celltype_key],
              spot_size=15,vmax='p100',
              crop_coord=(4600,5600,4250,5250)
             )
```

Example 20 (python):
```python
/tmp/ipykernel_3227243/3010556783.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=[celltype_col],
```

Example 21 (python):
```python
tissue_id = 'C194-HEA-0-FFPE-1'
slide = adata[adata.obs['tissue_block_id']==tissue_id]

celltypes_list = ['AVNPC', 'CorCapEC', 'CorPeri', 'DASMC', 'EPDC', 'EndoCush', 'Endocard',
       'Eryth', 'FB', 'GVSMC', 'GVSMCT', 'GVVenEC', 'LymphEC', 'MPLYVE1',
       'MacCX3', 'MesEpiC', 'MoDC', 'ParaN', 'PericFib', 'SANPC', 'SchwC',
       'SchwNeuro', 'ValveEC', 'ValveIC', 'aCML', 'aCMR']
color_list = ['#ffff00',
 '#1ce6ff',
 '#ff34ff',
 '#a30059',
 '#ffdbe5',
 '#63ffac',
 '#b79762',
 '#004d43',
 '#8fb0ff',
 '#997d87',
 '#5a0007',
 'red',
 '#4a3b53',
 '#ff2f80',
 '#ba0900',
 '#6b7900',
 '#00c2a0',
 'blue',
 '#b903aa',
 (0.5, 0.0, 0.5, 0.7),
 '#7b4f4b',
 '#013349',
 '#00846f',
 '#372101',
 '#ffb500',
 '#c2ffed',
 '#cc0744',
 '#c0b9b2']

for celltype,color in zip(celltypes_list,color_list):
    # update colors
    if celltype in set(slide.obs[celltype_key]):
        slide.uns[f"{celltype_key}_colors"][list(slide.obs[celltype_key].cat.categories).index(celltype)]=color

celltypes_to_plot = ['SANPC',
                     'CorCapEC',
                     'CorPeri',
                     'GVVenEC',
                     'MesEpiC',
                     'FB',
                     'aCMR',
                     'AVNPC',
                     'ParaN',
                     'SchwP',
                    ]

sc.pl.spatial(slide,color=[celltype_key],groups=celltypes_to_plot,
              spot_size=15,vmax='p100',
              crop_coord=(4600,5600,4250,5250),
              show=False
             )
plt.savefig(f'{figdir}/Xenium5K_euploid_SAN_for-Fig1.pdf',bbox_inches="tight",dpi=300)
```

Example 22 (python):
```python
/tmp/ipykernel_3227243/2717981830.py:54: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=[celltype_key],groups=celltypes_to_plot,
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/plotting/_utils.py:406: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  adata.uns[color_key] = _palette
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/plotting/_tools/scatterplots.py:1148: UserWarning: *c* argument looks like a single numeric RGB or RGBA sequence, which should be avoided as value-mapping will have precedence in case its length matches with *x* & *y*.  Please use the *color* keyword-argument or provide a 2D array with a single row if you intend to specify the same RGB or RGBA value for all points.
  ax.scatter([], [], c=palette[label], label=label)
```

Example 23 (python):
```python
ADATA_PATH = "/rfs/project/rfs-iCNyzSAaucw/kk837/data_objects/Foetal/RNA/Feb28ObjectRaw_finegrain_updated.h5ad"
ad = sc.read_h5ad(ADATA_PATH, backed='r')

# Step 1: Create mapping (already done)
cat = "mid_grain"
mid2colours = dict(zip(ad.obs[cat].cat.categories, ad.uns[f"{cat}_colors"]))

# Step 2: Extract the labels and colors in the correct order
mid_grains = list(mid2colours.keys())
palette = list(mid2colours.values())
num_mid_grains = len(mid_grains)

# Step 3: Plot
plt.figure(figsize=(num_mid_grains * 0.4, 2))  # wide figure
sns.palplot(palette)
plt.xticks(np.arange(num_mid_grains), mid_grains, rotation=90)
plt.title("Mid Grain to Color Mapping")
plt.tight_layout()
plt.show()
```

Example 24 (python):
```python
/tmp/ipykernel_794362/455462603.py:18: UserWarning: Tight layout not applied. The bottom and top margins cannot be made large enough to accommodate all Axes decorations.
  plt.tight_layout()
```

Example 25 (unknown):
```unknown
<Figure size 448x160 with 0 Axes>
```

Example 26 (python):
```python
adata.obs['celltypist_mid_fin'] = pd.Categorical(
    adata.obs['celltypist_mid_fin'],
    categories=ad.obs[cat].cat.categories,
    ordered=True
)
```

Example 27 (python):
```python
latent_space = 'pca_harmony'
sc.pl.embedding(adata,
                    basis=f"X_umap_{latent_space}",
                    color='celltypist_mid_fin',palette=palette,
                show=False
                   )
plt.savefig(f'{figdir}/Xenium5K_UMAP-midgrain_for-Fig1.pdf',bbox_inches="tight",dpi=300)
```

Example 28 (python):
```python
tissue_id = 'C194-HEA-0-FFPE-1'
slide = adata[adata.obs['tissue_block_id']==tissue_id]

sc.pl.spatial(slide,color=['celltypist_mid_fin'],
              spot_size=15,vmax='p100',
              # crop_coord=(4600,5600,4250,5250),
              show=False
             )
# plt.savefig(f'{figdir}/Xenium5K_euploid_SAN_for-Fig1.pdf',bbox_inches="tight",dpi=300)
```

Example 29 (python):
```python
/tmp/ipykernel_794362/1888614849.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['celltypist_mid_fin'],
```

Example 30 (unknown):
```unknown
[<Axes: title={'center': 'celltypist_mid_fin'}, xlabel='spatial1', ylabel='spatial2'>]
```

Example 31 (unknown):
```unknown
AnnotationPalette={ 
    'Atrium':'#eac075',
        'Ventricle - Compact':'#f1acfa',
     'Ventricle - Trabeculated':'#f09951',
        'SAnode':'#8A2BE2',
        'AVnode':'#062b4a',
    'VCS - Proximal':'#FFFF00',
        'VCS - Distal':'#7FFFD4',
        'Epicardium':'#01ccc8',
        'Endocardium - Atrial':'#DC143C',
        'Endocardium - Ventricular':'#f53527',
        'Endocardial cusion':'#ff7e97',
        'Great vessel':'#ff9b6e',
    'Ductus arteriosus':'#8a6800',
        'Coronary vessel':'#206601',
        'Adventitia - Coronary':'#CAFF70',
     'Adventitia - GV':'#005aa0',
        'AV ring':'#e478ff',
    'Valves':'#3300FF',
    'LN':'#9088ff',
}
```

Example 32 (python):
```python
tissue_id = 'C194-HEA-0-FFPE-1'
slide = adata[adata.obs['tissue_block_id']==tissue_id]

sc.pl.spatial(slide,color=['tt_prediction'],
              spot_size=15,vmax='p100',
              # crop_coord=(4600,5600,4250,5250),
              show=False,
              palette=AnnotationPalette
             )
plt.savefig(f'{figdir}/TissueTypist_Xenium5K_whole-heart.pdf',bbox_inches="tight",dpi=300)
```

Example 33 (python):
```python
/tmp/ipykernel_968450/2063101229.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['tt_prediction'],
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/plotting/_utils.py:487: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  adata.uns[value_to_plot + "_colors"] = colors_list
```

Example 34 (python):
```python
sc.pl.spatial(slide,color=['tt_prediction'],
              spot_size=15,vmax='p100',
              crop_coord=(5500,6800,4250,5500),
              show=False,
              palette=AnnotationPalette
             )
plt.savefig(f'{figdir}/TissueTypist_Xenium5K_Atrium-AVN.pdf',bbox_inches="tight",dpi=300)
```

Example 35 (python):
```python
/tmp/ipykernel_968450/2129560865.py:1: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['tt_prediction'],
```

Example 36 (python):
```python
sc.pl.spatial(slide,color=['tt_prediction'],
              spot_size=15,vmax='p100',
              crop_coord=(8000,9500,2500,4000),
              show=False,
              palette=AnnotationPalette
             )
plt.savefig(f'{figdir}/TissueTypist_Xenium5K_Ventricle.pdf',bbox_inches="tight",dpi=300)
```

Example 37 (python):
```python
/tmp/ipykernel_968450/2311424067.py:1: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['tt_prediction'],
```

Example 38 (python):
```python
sc.pl.spatial(slide,color=['tt_prediction'],
              spot_size=15,vmax='p100',
              crop_coord=(3500,5000,1800,3300),
              show=False,
              palette=AnnotationPalette
             )
plt.savefig(f'{figdir}/TissueTypist_Xenium5K_GV.pdf',bbox_inches="tight",dpi=300)
```

Example 39 (python):
```python
/tmp/ipykernel_968450/3875201264.py:1: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['tt_prediction'],
```

---

## Import modules¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig5_Trisomy21/no03-2_fraction-of-compactCM.html

**Contents:**
- Import modules¶
- Read in anndata¶
- Calculate and compare fraction¶
- Left vCMs¶
- Right vCMs¶

• Calculating fraction using NOT subsampled adata

**Examples:**

Example 1 (python):
```python
import warnings
warnings.simplefilter("ignore", UserWarning)

import anndata
import matplotlib.pyplot as plt
import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 
import seaborn as sns

import numpy as np
import numpy.random as random
import pandas as pd
import scanpy as sc
```

Example 2 (python):
```python
sc.settings.verbosity = 3             # verbosity: errors (0), warnings (1), info (2), hints (3)
sc.settings.set_figure_params(dpi=80, facecolor='white', color_map = 'RdPu',)
```

Example 3 (python):
```python
# import own function
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 4 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk/Aug2024'
table_dir = '/nfs/team205/heart/fetal_ms_figures/tables/Aug2024'
```

Example 5 (python):
```python
import os
os.getcwd()
```

Example 6 (unknown):
```unknown
'/nfs/team205/kk18/notebooks/Foetal/Trisomy21/RNA'
```

Example 7 (python):
```python
adata = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/trisomy21/Euploid_T21Hearts_Aug2024_sel.h5ad')
adata
```

Example 8 (unknown):
```unknown
AnnData object with n_obs × n_vars = 147494 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score', 'euploid_or_t21', 'donor_type', 'region_finest', 'flushed', 'scrublet_leiden', 'cluster_scrublet_score', 'leiden_scVI_0.1', 'leiden_scVI_0.2', 'leiden_scVI_0.3', 'leiden_scVI_0.5', 'leiden_scVI_0.7', 'leiden_scVI_1', 'celltypist_coarse', 'conf_score_coarse', 'cells_to_remove', 'celltypist_fine', 'conf_score_fine', 'coarse_grain_merged', 'fine_grain_merged'
    var: 'gene_id', 'gene_name_scRNA-0-0', 'feature_type-1', 'ambient_expression-0-1', 'ambient_expression-1-1', 'ambient_expression-10-1', 'ambient_expression-11-1', 'ambient_expression-12-1', 'ambient_expression-13-1', 'ambient_expression-14-1', 'ambient_expression-2-1', 'ambient_expression-3-1', 'ambient_expression-4-1', 'ambient_expression-5-1', 'ambient_expression-6-1', 'ambient_expression-7-1', 'ambient_expression-8-1', 'ambient_expression-9-1', 'mt-1', 'ribo-1', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1'
    obsm: 'X_scVI', 'X_umap', 'latent_gene_encoding'
```

Example 9 (python):
```python
from scipy.stats import ranksums
```

Example 10 (python):
```python
cells_to_include = ['VentricularCardiomyocytesLeftCompact','VentricularCardiomyocytesLeftTrabeculated']
group_col = 'fine_grain_merged'

ad = adata[adata.obs[group_col].isin(cells_to_include)]
ad.obs['condition_donor'] = ad.obs['euploid_or_t21'].astype('str')+'_'+ad.obs['donor'].astype('str')

# sangerID proportion
crosstab=pd.crosstab(ad.obs['condition_donor'],ad.obs[group_col])
# Normalize each row to sum to 1 (i.e., to get fractions)
crosstab_normalized = crosstab.div(crosstab.sum(axis=1), axis=0)

# Plotting the stacked bar chart
ax = crosstab_normalized.plot(kind='bar',stacked=True,figsize=(4, 4),legend=False)

# Adding labels and title
plt.rcParams['axes.grid'] = False
plt.ylabel('Fraction')
plt.title('Normalized Stacked Bar Chart')
plt.legend(bbox_to_anchor=(0.7, -1, 0.5, 0.5))
plt.xlabel(None)
# Show the plot
plt.show()
```

Example 11 (python):
```python
cells_to_include = ['VentricularCardiomyocytesLeftCompact','VentricularCardiomyocytesLeftTrabeculated']
group_col = 'fine_grain_merged'
split_col = 'euploid_or_t21'
celltype = 'VentricularCardiomyocytesLeftCompact'
ncols = 6


ad = adata[adata.obs[group_col].isin(cells_to_include)]
ad.obs['condition_donor'] = ad.obs['euploid_or_t21'].astype('str')+'_'+ad.obs['donor'].astype('str')

# sangerID proportion
crosstab=pd.crosstab(ad.obs['condition_donor'],ad.obs[group_col])
# Normalize each row to sum to 1 (i.e., to get fractions)
crosstab_normalized = crosstab.div(crosstab.sum(axis=1), axis=0)

# prepare data for plotting
data = crosstab_normalized.reset_index().melt(id_vars=['condition_donor'])
data['euploid_or_t21'] = [x.split('_')[0] for x in data['condition_donor']]
data = data[data[group_col]==celltype]
palette = ['gray','lightgray']

# Create a catplot with boxplots for each gene
g = sns.catplot(x=split_col, y='value', col='fine_grain_merged', data=data, kind='box', palette=palette,
                col_wrap=ncols, sharey=False, sharex=False, height=3, aspect=0.6)

# Overlay swarmplot on each Axes
# We'll use the 'hue' parameter to match the subsets and turn off dodge to align the points with the boxes.
for celltype, ax in zip(data['fine_grain_merged'].unique(), g.axes.flat):
    subset = data[data['fine_grain_merged'] == celltype]
    sns.swarmplot(x=split_col, y='value', data=subset, color='k', size=5, ax=ax, dodge=False)

# Update titles to remove "gene ="
for ax, celltype in zip(g.axes.flat, data['fine_grain_merged'].unique()):
    middle_ind = round(len(celltype)/2)
    celltype_title = f'{celltype[:middle_ind]}\n{celltype[middle_ind:]}'
    ax.set_title(celltype_title)

# Rotate x-tick labels and remove x-axis labels
for i,ax in enumerate(g.axes.flat):
    # ax.set_xticklabels(ax.get_xticklabels(), rotation=45, ha="right", rotation_mode="anchor")  # Rotate x-tick labels
    ax.set_xticklabels('')
    ax.set_xlabel('')  # Remove x-axis labels
    if i!=0:
        ax.set_ylabel('')

# Adjust the spacing of the plots
g.fig.subplots_adjust(wspace=0.4, hspace=0.4)

# Manually create a legend for the groups.
# Since there are no legend handles for the swarmplot, we create them manually.
from matplotlib.patches import Patch
legend_patches = [Patch(facecolor=palette[0], edgecolor='k', label='euploid'),
                  Patch(facecolor=palette[1], edgecolor='k', label='tri21'),
                  ]

# Add a single legend for the entire figure with our custom patches
g.fig.legend(handles=legend_patches, labels=['euploid', 'tri21'], title='', loc='center', bbox_to_anchor=(0.3, 0.8))

# Show the plot
plt.savefig(f'{figdir}/Tri21_BoxPlot_vCMLeft-fraction-compact-trab.pdf',bbox_inches="tight",dpi=300)
plt.show()

# statistical test
ranksums(data[data['euploid_or_t21']=='euploid']['value'],
         data[data['euploid_or_t21']=='tri21']['value']
        )
```

Example 12 (python):
```python
/tmp/ipykernel_3787554/1312438397.py:24: FutureWarning: 

Passing `palette` without assigning `hue` is deprecated and will be removed in v0.14.0. Assign the `x` variable to `hue` and set `legend=False` for the same effect.

  g = sns.catplot(x=split_col, y='value', col='fine_grain_merged', data=data, kind='box', palette=palette,
```

Example 13 (unknown):
```unknown
RanksumsResult(statistic=2.1213203435596424, pvalue=0.03389485352468927)
```

Example 14 (python):
```python
cells_to_include = ['VentricularCardiomyocytesRightCompact','VentricularCardiomyocytesRightTrabeculated']
group_col = 'fine_grain_merged'

ad = adata[adata.obs[group_col].isin(cells_to_include)]
ad.obs['condition_donor'] = ad.obs['euploid_or_t21'].astype('str')+'_'+ad.obs['donor'].astype('str')

# sangerID proportion
crosstab=pd.crosstab(ad.obs['condition_donor'],ad.obs[group_col])
# Normalize each row to sum to 1 (i.e., to get fractions)
crosstab_normalized = crosstab.div(crosstab.sum(axis=1), axis=0)

# Plotting the stacked bar chart
ax = crosstab_normalized.plot(kind='bar',stacked=True,figsize=(4, 4),legend=False)

# Adding labels and title
plt.rcParams['axes.grid'] = False
plt.ylabel('Fraction')
plt.title('Normalized Stacked Bar Chart')
plt.legend(bbox_to_anchor=(0.7, -1, 0.5, 0.5))
plt.xlabel(None)
# Show the plot
plt.show()
```

Example 15 (python):
```python
cells_to_include = ['VentricularCardiomyocytesRightCompact','VentricularCardiomyocytesRightTrabeculated']
group_col = 'fine_grain_merged'
split_col = 'euploid_or_t21'
celltype = 'VentricularCardiomyocytesRightCompact'


ad = adata[adata.obs[group_col].isin(cells_to_include)]
ad.obs['condition_donor'] = ad.obs['euploid_or_t21'].astype('str')+'_'+ad.obs['donor'].astype('str')

# sangerID proportion
crosstab=pd.crosstab(ad.obs['condition_donor'],ad.obs[group_col])
# Normalize each row to sum to 1 (i.e., to get fractions)
crosstab_normalized = crosstab.div(crosstab.sum(axis=1), axis=0)

# prepare data for plotting
data = crosstab_normalized.reset_index().melt(id_vars=['condition_donor'])
data['euploid_or_t21'] = [x.split('_')[0] for x in data['condition_donor']]
data = data[data[group_col]==celltype]
palette = ['gray','lightgray']

# Create a catplot with boxplots for each gene
g = sns.catplot(x=split_col, y='value', col='fine_grain_merged', data=data, kind='box', palette=palette,
                col_wrap=ncols, sharey=False, sharex=False, height=3, aspect=0.6)

# Overlay swarmplot on each Axes
# We'll use the 'hue' parameter to match the subsets and turn off dodge to align the points with the boxes.
for celltype, ax in zip(data['fine_grain_merged'].unique(), g.axes.flat):
    subset = data[data['fine_grain_merged'] == celltype]
    sns.swarmplot(x=split_col, y='value', data=subset, color='k', size=5, ax=ax, dodge=False)

# Update titles to remove "gene ="
for ax, celltype in zip(g.axes.flat, data['fine_grain_merged'].unique()):
    middle_ind = round(len(celltype)/2)
    celltype_title = f'{celltype[:middle_ind]}\n{celltype[middle_ind:]}'
    ax.set_title(celltype_title)

# Rotate x-tick labels and remove x-axis labels
for i,ax in enumerate(g.axes.flat):
    # ax.set_xticklabels(ax.get_xticklabels(), rotation=45, ha="right", rotation_mode="anchor")  # Rotate x-tick labels
    ax.set_xticklabels('')
    ax.set_xlabel('')  # Remove x-axis labels
    if i!=0:
        ax.set_ylabel('')

# Adjust the spacing of the plots
g.fig.subplots_adjust(wspace=0.4, hspace=0.4)

# Manually create a legend for the groups.
# Since there are no legend handles for the swarmplot, we create them manually.
from matplotlib.patches import Patch
legend_patches = [Patch(facecolor=palette[0], edgecolor='k', label='euploid'),
                  Patch(facecolor=palette[1], edgecolor='k', label='tri21'),
                  ]

# Add a single legend for the entire figure with our custom patches
g.fig.legend(handles=legend_patches, labels=['euploid', 'tri21'], title='', loc='center', bbox_to_anchor=(0.3, 0.8))

# Show the plot
plt.savefig(f'{figdir}/Tri21_BoxPlot_vCMRight-fraction-compact-trab.pdf',bbox_inches="tight",dpi=300)
plt.show()

# statistical test
ranksums(data[data['euploid_or_t21']=='euploid']['value'],
         data[data['euploid_or_t21']=='tri21']['value']
        )
```

Example 16 (python):
```python
/tmp/ipykernel_3787554/2915419533.py:22: FutureWarning: 

Passing `palette` without assigning `hue` is deprecated and will be removed in v0.14.0. Assign the `x` variable to `hue` and set `legend=False` for the same effect.

  g = sns.catplot(x=split_col, y='value', col='fine_grain_merged', data=data, kind='box', palette=palette,
```

Example 17 (unknown):
```unknown
RanksumsResult(statistic=2.1213203435596424, pvalue=0.03389485352468927)
```

---

## Submid job¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig3_SAN_niche/cellphonedb-VisiumHD_no2_run.html

**Contents:**
- Submid job¶
- Check error files¶
- Check output files¶

Job <97409> is submitted to queue .

**Examples:**

Example 1 (unknown):
```unknown
!cat /nfs/team205/heart/cellphonedb/run_script/run_cellphone_v5_foetal_visiumhd_HEA_FOET14880396_statistical_SAnode.py
```

Example 2 (python):
```python
import pandas as pd
import sys
import os
from cellphonedb.src.core.methods import cpdb_statistical_analysis_method

# Input files
cpdb_file_path = '/nfs/team205/heart/cellphonedb/db/v5.0.0/cellphonedb.zip'
meta_file_path = '/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/cellphonedb/HEA_FOET14880396/inputs/meta_SAnode.txt'
counts_file_path = '/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/cellphonedb/HEA_FOET14880396/inputs/log_norm_counts_SAnode.h5ad'
out_path = '/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/cellphonedb/HEA_FOET14880396/SAnode'
if not os.path.exists(out_path):
    os.makedirs(out_path)
min_percent = sys.argv[1]

print(f'##### All cells, min %:{min_percent} #####')

# run
cpdb_results = cpdb_statistical_analysis_method.call(
    cpdb_file_path = cpdb_file_path,                 # mandatory: CellphoneDB database zip file.
    meta_file_path = meta_file_path,                 # mandatory: tsv file defining barcodes to cell label.
    counts_file_path = counts_file_path,             # mandatory: normalized count matrix - a path to the counts file, or an in-memory AnnData object
    counts_data = 'hgnc_symbol',                     # defines the gene annotation in counts matrix.
    active_tfs_file_path = None,           # optional: defines cell types and their active TFs.
    microenvs_file_path = None,       # optional (default: None): defines cells per microenvironment.
    score_interactions = True,                       # optional: whether to score interactions or not. 
    iterations = 1000,                               # denotes the number of shufflings performed in the analysis.
    threshold = float(min_percent),                                 # defines the min % of cells expressing a gene for this to be employed in the analysis.
    threads = 5,                                     # number of threads to use in the analysis.
    debug_seed = 42,                                 # debug randome seed. To disable >=0.
    result_precision = 3,                            # Sets the rounding for the mean values in significan_means.
    pvalue = 0.1,                                   # P-value threshold to employ for significance.
    subsampling = False,                             # To enable subsampling the data (geometri sketching).
    subsampling_log = False,                         # (mandatory) enable subsampling log1p for non log-transformed data inputs.
    subsampling_num_pc = 100,                        # Number of componets to subsample via geometric skectching (dafault: 100).
    subsampling_num_cells = 1000,                    # Number of cells to subsample (integer) (default: 1/3 of the dataset).
    separator = '|',                                 # Sets the string to employ to separate cells in the results dataframes "cellA|CellB".
    debug = False,                                   # Saves all intermediate tables employed during the analysis in pkl format.
    output_path = out_path,                          # Path to save results.
    output_suffix = None                             # Replaces the timestamp in the output files by a user defined string in the  (default: None).
    )
```

Example 3 (unknown):
```unknown
module load cellgen/conda
conda activate cpdb
cd /nfs/team205/heart/cellphonedb

MIN_PERCENT=0.05
bsub \
-G team205 \
-q normal -M20000 -R"select[mem>20000] rusage[mem=20000] span[hosts=1]" \
-n 4 \
-o ./log/foetal_visiumhd_HEA_FOET14880396_output.%J \
-e ./log/foetal_visiumhd_HEA_FOET14880396_errorfile.%J \
python ./run_script/run_cellphone_v5_foetal_visiumhd_HEA_FOET14880396_statistical_SAnode.py $MIN_PERCENT
```

Example 4 (unknown):
```unknown
%%bash
FILE_path=/nfs/team205/heart/cellphonedb/log/foetal_visiumhd_HEA_FOET14880396_errorfile.97409
echo ${FILE_path}
cat ${FILE_path}
```

Example 5 (unknown):
```unknown
/nfs/team205/heart/cellphonedb/log/foetal_visiumhd_HEA_FOET14880396_errorfile.97409
100%|██████████| 1000/1000 [00:14<00:00, 71.33it/s]
100%|██████████| 11/11 [00:00<00:00, 278.56it/s]
100%|██████████| 11/11 [00:00<00:00, 806.77it/s]
100%|██████████| 121/121 [00:29<00:00,  4.05it/s]
```

Example 6 (unknown):
```unknown
%%bash
PATH_out=/nfs/team205/heart/cellphonedb/log/foetal_visiumhd_HEA_FOET14880396_output.97409
cat ${PATH_out}
```

Example 7 (python):
```python
##### All cells, min %:0.05 #####
Reading user files...
The following user files were loaded successfully:
/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/cellphonedb/HEA_FOET14880396/inputs/log_norm_counts_SAnode.h5ad
/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/cellphonedb/HEA_FOET14880396/inputs/meta_SAnode.txt
[ ][CORE][04/05/25-10:49:38][INFO] [Cluster Statistical Analysis] Threshold:0.05 Iterations:1000 Debug-seed:42 Threads:5 Precision:3
[ ][CORE][04/05/25-10:49:38][WARNING] Debug random seed enabled. Set to 42
[ ][CORE][04/05/25-10:49:39][INFO] Running Real Analysis
[ ][CORE][04/05/25-10:49:39][INFO] Running Statistical Analysis
[ ][CORE][04/05/25-10:49:54][INFO] Building Pvalues result
[ ][CORE][04/05/25-10:49:54][INFO] Building results
[ ][CORE][04/05/25-10:49:54][INFO] Scoring interactions: Filtering genes per cell type..
[ ][CORE][04/05/25-10:49:54][INFO] Scoring interactions: Calculating mean expression of each gene per group/cell type..
[ ][CORE][04/05/25-10:49:54][INFO] Scoring interactions: Calculating scores for all interactions and cell types..
Saved deconvoluted to /lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/cellphonedb/HEA_FOET14880396/SAnode/statistical_analysis_deconvoluted_05_04_2025_105025.txt
Saved deconvoluted_percents to /lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/cellphonedb/HEA_FOET14880396/SAnode/statistical_analysis_deconvoluted_percents_05_04_2025_105025.txt
Saved means to /lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/cellphonedb/HEA_FOET14880396/SAnode/statistical_analysis_means_05_04_2025_105025.txt
Saved pvalues to /lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/cellphonedb/HEA_FOET14880396/SAnode/statistical_analysis_pvalues_05_04_2025_105025.txt
Saved significant_means to /lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/cellphonedb/HEA_FOET14880396/SAnode/statistical_analysis_significant_means_05_04_2025_105025.txt
Saved interaction_scores to /lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/cellphonedb/HEA_FOET14880396/SAnode/statistical_analysis_interaction_scores_05_04_2025_105025.txt

------------------------------------------------------------
Sender: LSF System <lsfadmin@node-14-11>
Subject: Job 97409: <python ./run_script/run_cellphone_v5_foetal_visiumhd_HEA_FOET14880396_statistical_SAnode.py 0.05> in cluster <farm22> Done

Job <python ./run_script/run_cellphone_v5_foetal_visiumhd_HEA_FOET14880396_statistical_SAnode.py 0.05> was submitted from host <farm22-head1> by user <kk18> in cluster <farm22> at Sun May  4 10:47:34 2025
Job was executed on host(s) <4*node-14-11>, in queue <normal>, as user <kk18> in cluster <farm22> at Sun May  4 10:47:36 2025
</nfs/users/nfs_k/kk18> was used as the home directory.
</nfs/team205/heart/cellphonedb> was used as the working directory.
Started at Sun May  4 10:47:36 2025
Terminated at Sun May  4 10:50:39 2025
Results reported at Sun May  4 10:50:39 2025

Your job looked like:

------------------------------------------------------------
# LSBATCH: User input
python ./run_script/run_cellphone_v5_foetal_visiumhd_HEA_FOET14880396_statistical_SAnode.py 0.05
------------------------------------------------------------

Successfully completed.

Resource usage summary:

    CPU time :                                   179.27 sec.
    Max Memory :                                 2015 MB
    Average Memory :                             513.89 MB
    Total Requested Memory :                     20000.00 MB
    Delta Memory :                               17985.00 MB
    Max Swap :                                   -
    Max Processes :                              8
    Max Threads :                                13
    Run time :                                   166 sec.
    Turnaround time :                            185 sec.

The output (if any) is above this job summary.



PS:

Read file <./log/foetal_visiumhd_HEA_FOET14880396_errorfile.97409> for stderr output of this job.
```

---

## Subcluster¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/RNA/scnRNA_Integration-Annotation/Mesenchymal/MyocardialFibroblasts.html

**Contents:**
- Subcluster¶
- Investigate if there are adventitial specific genes¶

**Examples:**

Example 1 (unknown):
```unknown
#Added the keys for age and sex
```

Example 2 (python):
```python
# Import packages, initial settings

import numpy as np
import matplotlib.pyplot as plt
import scanpy as sc
import pandas as pd
import anndata
import scrublet as scr
import seaborn as sns
import magic
import pybiomart
import six
import sys
sys.modules['sklearn.externals.six'] = six
import scanorama
import harmonypy
import bbknn
import gseapy
from gseapy.plot import barplot, dotplot
import scvelo as scv
import scvi

plt.rcParams['pdf.fonttype'] = 42
plt.rcParams['ps.fonttype'] = 42

sc.settings.verbosity = 3  # verbosity: errors (0), warnings (1), info (2), hints (3)

from matplotlib.colors import LinearSegmentedColormap
cmap = LinearSegmentedColormap.from_list(name='gene_cmap', colors=['lightgrey', 'thistle', 'red', 'darkred']) 

sc.settings.set_figure_params(dpi=80, color_map='viridis', vector_friendly=False,  dpi_save=300, facecolor='white')
sc.logging.print_version_and_date()
#sc.logging.print_versions()
```

Example 3 (unknown):
```unknown
Global seed set to 0
```

Example 4 (unknown):
```unknown
Running Scanpy 1.8.1, on 2023-05-16 17:19.
```

Example 5 (python):
```python
adata=sc.read('./Objects/Feb28ObjectRaw.h5ad')
print(adata.shape)
```

Example 6 (unknown):
```unknown
(297473, 36601)
```

Example 7 (python):
```python
cell_cycle_genes = ['MCM5','PCNA','TYMS','FEN1','MCM2','MCM4','RRM1','UNG','GINS2','MCM6','CDCA7','DTL','PRIM1','UHRF1','MLF1IP','HELLS','RFC2','RPA2','NASP','RAD51AP1','GMNN','WDR76','SLBP','CCNE2','UBR7','POLD3','MSH2','ATAD2','RAD51','RRM2','CDC45','CDC6','EXO1','TIPIN','DSCC1','BLM','CASP8AP2','USP1','CLSPN','POLA1','CHAF1B','BRIP1','E2F8','HMGB2','CDK1','NUSAP1','UBE2C','BIRC5','TPX2','TOP2A','NDC80','CKS2','NUF2','CKS1B','MKI67','TMPO','CENPF','TACC3','FAM64A','SMC4','CCNB2','CKAP2L','CKAP2','AURKB','BUB1','KIF11','ANP32E','TUBB4B','GTSE1','KIF20B','HJURP','CDCA3','HN1','CDC20','TTK','CDC25C','KIF2C','RANGAP1','NCAPD2','DLGAP5','CDCA2','CDCA8','ECT2','KIF23','HMMR','AURKA','PSRC1','ANLN','LBR','CKAP5','CENPE','CTCF','NEK2','G2E3','GAS2L3','CBX5','CENPA']
s_genes = cell_cycle_genes[:43]
s_genes = [x for x in s_genes if x in adata.var_names]
print('S genes:', s_genes, '\n')
g2m_genes = cell_cycle_genes[43:]
g2m_genes = [x for x in g2m_genes if x in adata.var_names]
print('G2 genes:', g2m_genes, '\n')
cell_cycle_genes = [x for x in cell_cycle_genes if x in adata.var_names]

hb_genes1 = ['HBB','HBD','HBG1','HBG2','HBE1','HBZ','HBM','HBA2','HBA1', 'HBQ1']
hb_genes1 = [x for x in hb_genes1 if x in adata.var_names]
print('Hemoglobin genes set 1:', hb_genes1, '\n')

hb_genes2 = ['AHSP','GYPA','RHAG','GFI1B','KEL','EPB42','ALAS2','TSPO2','GYPB','GYPE','SPTA1','RHCE','KLF1','RHD','SLC4A1','SNX22','SQSTM1','SOD1','FTH1','MAP1LC3B','SLC25A37','TRIM58','HMBS']
hb_genes2 = [x for x in hb_genes2 if x in adata.var_names]
print('Hemoglobin genes set 2:', hb_genes2, '\n')

mito_genes=['MT-ND1','MT-ND2','MT-CO1','MT-CO2','MT-ATP8','MT-ATP6','MT-CO3','MT-ND3','MT-ND4L', 'MT-ND4', 'MT-ND5', 'MT-ND6', 'MT-CYB']
mito_genes = [x for x in mito_genes if x in adata.var_names]
print('Mitochondrial genes:', mito_genes, '\n')

ribo_genes = ['RPLP0','RPLP1','RPLP2','RPL3','RPL3L','RPL4','RPL5','RPL6','RPL7','RPL7A','RPL7L1','RPL8','RPL9','RPL10','RPL10A','RPL10L','RPL11','RPL12','RPL13','RPL13A','RPL14','RPL15','RPL17','RPL18','RPL18A','RPL19','RPL21','RPL22','RPL22L1','RPL23','RPL23A','RPL24','RPL26','RPL26L1','RPL27','RPL27A','RPL28','RPL29','RPL30','RPL31','RPL32','RPL34','RPL35','RPL35A','RPL36','RPL36A','RPL36AL','RPL37','RPL37A','RPL38','RPL39','RPL39L','UBA52','RPL41','RPSA','RPS2','RPS3','RPS3A','RPS4X','RPS4Y1','RPS4Y2','RPS5','RPS6','RPS7','RPS8','RPS9','RPS10','RPS11','RPS12','RPS13','RPS14','RPS15','RPS15A','RPS16','RPS17','RPS18','RPS19','RPS20','RPS21','RPS23','RPS24','RPS25','RPS26','RPS27','RPS27A','RPS27L','RPS28','RPS29','FAU']
ribo_genes = [x for x in ribo_genes if x in adata.var_names]
print('Ribo genes:', ribo_genes, '\n')

stress_genes=['FOS','FOSB','JUN','JUNB','JUND','ZFP36','SOCS3','CEBPD','CEBPB','ATF3','EGR1','HSPA1A','HSPA1B','HSP90AA1','HSP90AB1','HSPA8','HSPB1','HSPE1','IER3','IER2','BTG1','BTG2','DUSP1']
stress_genes = [x for x in stress_genes if x in adata.var_names]
print('Tissue dissociation stress associated genes:', stress_genes, '\n')

FoxProteins=['FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1']
chrY_genes=['SRY', 'RPS4Y1', 'ZFY', 'ZFY-AS1', 'LINC00278', 'PCDH11Y', 'TTTY2B', 'TBL1Y', 'PRKY', 'USP9Y', 'DDX3Y', 'UTY', 'TMSB4Y', 'NLGN4Y', 'TTTY14', 'KDM5D', 'TTTY10', 'EIF1AY', 'PRORY', 'LINC00266-4P']
MembraneProteins1=['AADAT','AAK1','ABCA12','ABCA7','ABCB1','ABCB11','ABCB4','ABCB6','ABCC1','ABCC3','ABCC4','ABCC5','ABCC6','ABCG2','ABHD3','ABI1','ABLIM3','AC004691.2','AC004922.1','AC005324.3','AC005726.1','AC008403.1','AC138696.1','ACIN1','ACKR3','ACLY','ACOT13','ACOT9','ACTN1','ACTR10','ACTR6','ADA','ADAM10','ADAM12','ADAM22','ADAM28','ADAM33','ADAMTS1','ADAMTS2','ADAMTSL4','ADAP1','ADCK5','ADD1','ADD2','ADGRB2','ADGRD1','ADGRG2','ADGRG7','ADGRL2','ADH1A','ADH1B','ADH1C','ADH7','ADK','ADORA1','ADRA1B','ADRM1','ADSS','AFAP1L2','AFDN','AFG1L','AGER','AGRN','AHNAK','AHNAK2','AIFM2','AKAP10','AKAP11','AKAP12','AKR1B10','AKR1B15','AL132780.3','AL139300.1','AL355315.1','AL358113.1','AL645922.1','ALDH3A1','ALK','ALOX15B','ALOXE3','ALPI','ALPP','ALPPL2','ALS2CR12','AMACR','AMDHD1','AMER1','AMER2','AMIGO2','AMOT','AMOTL1','AMOTL2','AMPH','ANK2','ANK3','ANKEF1','ANKFN1','ANKIB1','ANKLE2','ANKRA2','ANKRD13A','ANKRD35','ANKRD42','ANKRD45','ANKS1B','ANO1','ANO10','ANO2','ANO6','ANPEP','ANXA1','ANXA13','ANXA2','ANXA3','AP2M1','APBB1','APBB1IP','APC','APLP1','APMAP','APOC3','APOD','APOLD1','AQP1','AQP10','AQP3','AQP4','AQP5','ARAP1','ARF1','ARHGAP18','ARHGAP19','ARHGAP21','ARHGAP22','ARHGAP23','ARHGAP24','ARHGAP28','ARHGAP29','ARHGAP33','ARHGAP36','ARHGAP40','ARHGAP9','ARHGDIG','ARHGEF1','ARHGEF11','ARHGEF12','ARHGEF15','ARHGEF26','ARHGEF28','ARHGEF40','ARHGEF5','ARID2','ARID4A','ARID5B','ARL13A','ARL15','ARL4C','ARMC12','ARMCX5','ARPC1A','ARPC5','ARRB2','ARRDC4','ARVCF','ASAP1','ASB5','ASGR1','ASGR2','ASH2L','ASIC1','ASIC4','ASPM','ASPSCR1','ATG13','ATG3','ATG7','ATIC','ATOX1','ATP10D','ATP12A','ATP13A3','ATP1B3','ATP23','ATP2B1','ATP2B4','ATP2C2','ATP6V0A2','ATP6V1H','ATP8A2','ATP8B1','ATXN10','ATXN3','ATXN7L3','AVEN','AVPI1','AXIN2','AXL','B2M','B3GALT2','BACE1','BAG3','BAIAP2','BAIAP2L1','BANK1','BASP1','BBOF1','BCAR1','BCAR3','BCL2L12','BCORL1','BCR','BDKRB2','BEND6','BFSP1','BFSP2','BICD2','BIN2','BLNK','BLVRB','BMPR2','BMX','BOC','BOLL','BORCS5','BORCS8-MEF2B','BRK1','BRPF1','BRSK1','BSDC1','BST2','BTK','BTN2A1','BTRC','BUB1B-PAK6','BVES','BZW2','C10orf90','C11orf42','C11orf52','C12orf73','C12orf76','C14orf37','C14orf93','C16orf74','C17orf58','C17orf62','C18orf21','C19orf33','C19orf38','C19orf48','C19orf71','C1orf116','C1orf189','C1orf35','C1orf54','C1orf68','C1orf74','C1QBP','C2orf72','C3orf14','C3orf20','C3orf38','C4orf19','C4orf22','C7orf43','C8orf33','C8orf37','C9orf153','C9orf3','CA14','CA9','CABLES2','CACNA1E','CACNG4','CADM1','CALCRL','CALD1','CALML5','CALN1','CAMK2A','CAMK2B','CAMK2D','CAMK2G','CAP2','CAPS','CASKIN2','CASP4','CAST','CAV3','CAVIN1','CAVIN2','CAVIN3','CAVIN4','CBWD1','CBWD2','CBWD3','CBWD5','CBWD6','CC2D1A','CCBE1','CCDC107','CCDC112','CCDC115','CCDC124','CCDC136','CCDC144NL','CCDC15','CCDC154','CCDC170','CCDC24','CCDC28A','CCDC59','CCDC62','CCDC66','CCDC69','CCDC8','CCDC83','CCDC85C','CCDC93','CCDC97','CCNT2','CCNYL1','CCR3','CCR4','CCT3','CD109','CD163','CD164L2','CD200R1','CD226','CD248','CD274','CD2AP','CD300A','CD300LB','CD33','CD34','CD37','CD38','CD3E','CD3G','CD4','CD44','CD46','CD47','CD48','CD79B','CD81','CD84','CD86','CD8A','CD8B','CD9','CD93','CD99L2','CDAN1','CDC42EP1','CDC42EP3','CDC42EP4','CDC42SE1','CDCA4','CDH1','CDH13','CDH15','CDH17','CDH2','CDH26','CDH3','CDH4','CDH5','CDH6','CDH8','CDIPT','CDK16','CDK5','CDK5RAP2','CDK7','CDRT15','CDV3','CEACAM5','CELSR1','CENPJ','CEP162','CEP295','CEP55','CEP57L1','CEP89','CERCAM','CFAP157','CFAP206','CFAP58','CFAP65','CFB','CFL2','CFLAR','CGN','CGNL1','CHCHD7','CHIC2','CHMP2A','CHRM3','CHRNA1','CHRNA5','CHRNB4','CHRND','CIB1','CIB3','CIP2A','CKAP5','CLCA2','CLCA4','CLCN5','CLCN6','CLDN11','CLDN18','CLDN2','CLDN23','CLDN3','CLDN4','CLDN7','CLDN9','CLEC2A','CLEC4F','CLEC5A','CLECL1','CLIC2','CLIC5','CLTB','CLUL1','CMTM4','CMTM7','CMTR2','CMYA5','CNGA1','CNKSR2','CNNM1','CNNM4','CNP','CNR1','CNTD1','COBL','COBLL1','COG3','COL16A1','COL17A1','COL4A5','COL6A2','COLEC12','COLQ','COMMD4','COMTD1','COPRS','COQ7','CORIN','CORO1B','CORO2A','CORO2B','COX6B2','CPEB3','CPPED1','CPZ','CRB3','CREB5','CRIP1','CRIP2','CRK','CRLF3','CROCC','CRTC1','CRYAB','CRYBG3','CRYL1','CSDE1','CSF1','CSF1R','CSPG4','CST6','CTNNA1','CTNNB1','CTNNBIP1','CTNND1','CTNND2','CTSF','CTSV','CTTN','CTXN1','CUEDC1','CUTA','CX3CL1','CX3CR1','CXADR','CXCR2','CYB5D2','CYBRD1','CYFIP2','CYP20A1','CYP24A1','CYTH2','CYTH4','DAAM1','DAB2','DAG1','DAGLB','DAW1','DBN1','DCAF13','DCAF16','DCBLD2','DCHS1','DCHS2','DCP2','DCT','DCTN6','DCUN1D2','DDAH1','DDB2','DDIAS','DDIT4L','DDR1','DDR2','DDX58','DDX60L','DENND6B','DFFA','DGKG','DGKH','DHDDS','DHPS','DHRS7B','DIAPH1','DIAPH3','DIO2','DIS3L','DLG1','DLG2','DLG5','DLGAP3','DLL3','DMWD','DNAAF1','DNAAF4','DNAH17','DNAJA4','DNAJB13','DNAJB4','DNAJC18','DNAJC5','DNAJC6','DNAJC9','DOC2A','DOCK4','DOK3','DONSON','DPEP1','DPH1','DPPA5','DPYSL2','DQX1','DRD1','DRP2','DSC3','DSEL','DSG2','DSP','DTNA','DUS1L','DUSP15','DXO','DYSF','ECSCR','EDA2R','EDNRA','EDNRB','EEF1AKMT1','EEF2','EFCAB1','EFCAB11','EFCAB2','EFCAB7','EFHC2','EFNB1','EFR3A','EFR3B','EGFR','EHBP1','EHD1','EHD2','EHD3','EHD4','EIF2B2','EIF5','EIF5B','ELFN1','ELMO3','ELN','ELOC','EMP3','ENAH','ENC1','ENG','ENKD1','ENO1','ENO2','ENO3','ENOX1','ENOX2','EPB41','EPB41L1','EPB41L2','EPB41L3','EPB41L4A','EPB41L4B','EPB41L5','EPCAM','EPHA2','EPHA3','EPHB1','EPHB2','EPN1','EPOR','EPS8L2','ERAP1','ERBB2','ERBB3','ERBIN','ERC1','ERCC4','ERMN','ESCO2','ETNK1','ETS2','EVA1A','EVI2B','EXOC1','EXOC3L1','EXOC7','EZR','F11R','F2RL1','F2RL3','FAAP20','FABP2','FABP3','FABP5','FAIM','FAM126A','FAM126B','FAM129A','FAM129B','FAM13A','FAM163B','FAM171A1','FAM193A','FAM205A','FAM210B','FAM222A','FAM45A','FAM47E','FAM47E-STBD1','FAM72A','FAM72B','FAM72C','FAM72D','FANCG','FAS','FASN','FBLIM1','FBLN2','FBLN5','FBLN7','FBP2','FBRSL1','FBXL2','FBXO16','FBXO38','FBXO41','FBXW11','FCAR','FCGBP','FCGR1A','FCGR1B','FCGR2A','FCHSD2','FFAR4','FGD1','FGD5','FGFR2','FGR','FHIT','FHL1','FIBCD1','FIGN','FILIP1','FILIP1L','FKBP2','FLAD1','FLCN','FLNA','FLNB','FLNC','FLOT1','FLOT2','FLRT3','FLT1','FLT4','FLVCR1','FMN2','FMNL2','FMNL3','FNBP1','FOXA2','FOXN3','FP565260.3','FRMD3','FRMD7','FRMD8','FRS2','FSCN1','FTCD','FUT2','FXYD3','FYB1','FYN','FZD1','FZD2','FZD4','FZD6','GAB1','GAB2','GABRA1','GABRA2','GABRA3','GABRA5','GABRG1','GAD1','GADL1','GAGE1','GAGE10','GAGE12C','GAGE12D','GAGE12E','GAGE12F','GAGE12G','GAGE12H','GAGE12J','GAGE13','GAGE2A','GAGE2E','GALNT10','GALR1','GAP43','GAPDH','GAPT','GAPVD1','GAS2','GAS2L1','GAS7','GAS8','GBP4','GCA','GCC1','GCLM','GCSAML-AS1','GFI1B','GFOD1','GFRA3','GHR','GID8','GINM1','GIPC1','GJA1','GJB3','GJB4','GJB6','GJB7','GJC1','GJC2','GLO1','GLP2R','GLRX','GLUL','GMIP','GNAI2','GNAQ','GNAS','GNB1','GNB2','GNB3','GNB4','GNG2','GP9','GPAT3','GPATCH1','GPBP1','GPBP1L1','GPC1','GPC3','GPC4','GPI','GPR1','GPR142','GPR156','GPR157','GPR160','GPR173','GPR20','GPR27','GPR50','GPR63','GPRC5A','GPRIN1','GPRIN2','GPX8','GRAMD1A','GRAMD1B','GRAMD1C','GRAP2','GRB7','GRID2IP','GRIK5','GRK3','GRK5','GRPR','GSDMA','GTF2F1','GTSE1','GUCA1A','GYPA','GYPC','HAPLN3','HARBI1','HAS1','HAUS4','HAUS7','HCAR2','HCAR3','HCFC2','HCK','HCLS1','HCST','HDAC8','HEG1','HELZ','HENMT1','HERC2','HERPUD1','HFE','HIF1A','HIF3A','HINT1','HIP1R','HLA-A','HLA-DPB1','HMGCS1','HMOX1','HOMER3','HOXC5','HOXD4','HPS3','HRASLS5','HRH1','HS6ST3','HSH2D','HSPB1','HSPG2','HTR1B','HTR7','HTRA1','HUNK','HYDIN','HYI','HYLS1','ICAM1','ICAM2','ICOS','ICOSLG','IFFO2','IFIT5','IFNGR1','IFRD1','IFT122','IGFL2','IGSF11','IGSF23','IKZF5','IL10RA','IL12RB1','IL12RB2','IL13','IL13RA2','IL16','IL17RB','IL1R1','IL1RL1','IL21R','IL4R','IL6ST','IL7R','ILDR2','ILK','INPP5A','INSR','INTS12','IP6K2','IPCEF1','IQGAP1','IQGAP2','IREB2','IRF2','IRS1','ISLR','ITGA2B','ITGA3','ITGA4','ITGA7','ITGA8','ITGA9','ITGB1','ITGB2','ITGB3','ITGB4','ITGB5','ITGB6','ITGB7','ITGB8','ITK','ITM2A','ITPKA','ITSN1','ITSN2','IYD','JAG1','JAK2','JAK3','JAM2','JAML','JCAD','JMJD4','JPT2','JUP','KANK1','KANK3','KANSL2','KARS','KATNB1','KCNAB1','KCNB2','KCNH8','KCNJ15','KCNK13','KCNK17','KCNK6','KCNK7','KCNK9','KCNN2','KCNS3','KCNU1','KDF1','KDR','KEL','KIAA0556','KIAA0895','KIAA1324','KIAA1522','KIAA1958','KIF17','KIF19','KIF21A','KIF21B','KIF26A','KIF26B','KISS1R','KIT','KLC1','KLC2','KLF9','KLHDC7B','KLHL41','KLHL6','KLHL7','KLK7','KLRC1','KLRC2','KLRF1','KLRK1','KMT2D','KNDC1','KNSTRN','KNTC1','KPTN','KRT75','KRT77','KRTAP25-1','L1CAM','L3MBTL1','LAIR1','LAMA4','LAMC1','LAMP2','LAMTOR1','LAPTM4A','LASP1','LAT','LAT2','LAX1','LCP1','LCTL','LDB2','LDLRAD3','LENG1','LGALS3','LGALS4','LGALS8','LGR6','LHFPL2','LHX6','LIMA1','LINGO1','LIPJ','LMCD1','LMOD2','LMTK2','LMTK3','LNX1','LNX2','LONRF1','LONRF3','LOXHD1','LPAR1','LPAR3','LPAR4','LPIN1','LPIN2','LRCH2','LRP11','LRP12','LRRC15','LRRC26','LRRC34','LRRC45','LRRC69','LRRC7','LRRC9','LRRFIP2','LRRIQ3','LSMEM2','LSR','LTB4R2','LUZP2','LY6K','LYN','LYPD1','LYST','LZTS1','LZTS2','MACF1','MADD','MAGI1','MAGI3','MAMDC2','MAP1S','MAP2','MAP2K1','MAP2K4','MAP2K7','MAP3K12','MAP3K21','MAP3K6','MAP3K7','MAP4','MAP4K1','MAP4K5','MAP6','MAPK15','MAPK8IP1','MAPT','MARCH10','MARCH7','MARCKS','MARK2','MARVELD2','MAS1L','MAT2B','MBP','MC5R','MCAM','MCEMP1','MCF2L2','MCMBP','MCTS1','MDGA1','ME1','MEF2B','MEIOC','MEPCE','MERTK','MET','METAP1','METAP2','METTL15','METTL25','METTL9','MFAP3L','MFSD1','MFSD2A','MGAT4C','MIB1','MICAL2','MICAL3','MICALL1','MICB','MIEN1','MIGA2','MISP','MKLN1','MKRN3','MLF2','MLKL','MLST8','MMACHC','MMP10','MMP15','MMP24','MMP25','MOAP1','MPHOSPH8','MPHOSPH9','MPI','MPIG6B','MPL','MPP1','MPP5','MPP7','MPZL1','MPZL2','MR1','MRGPRF','MRNIP','MRPL42','MRPL44','MRPL46','MRPS18B','MS4A1','MS4A12','MS4A14','MS4A15','MSN','MSRA','MTCP1','MTFR1L','MTHFR','MTM1','MTPN','MTSS1','MUC1','MUC20','MVD','MYH11','MYH9','MYL4','MYLK','MYO10','MYO1B','MYO1C','MYO1E','MYO1G','MYO3A','MYO5B','MYO9A','MYOF','N4BP2','NAA35','NAB1','NAIF1','NAMPT','NAP1L2','NAPG','NARFL','NCAM1','NCAM2','NCK1','NCKAP1L','NCKAP5','NCKIPSD','NCOA1','NCS1','NDC1','NECTIN2','NECTIN3','NECTIN4','NEDD1','NEDD9','NEO1','NEU1','NEXN','NF2','NFE2L2','NFKBID','NGEF']
MembraneProteins2=['NGFR','NHS','NHSL2','NID2','NINJ2','NKTR','NKX2-8','NLGN1','NLGN3','NLK','NLRX1','NMB','NMBR','NME1','NME2','NMT1','NMT2','NMUR2','NOS1','NOTCH2','NPFFR2','NPL','NPNT','NPR1','NPTN','NPTXR','NR2C1','NRAS','NRCAM','NRP2','NRSN1','NRXN1','NSD1','NSFL1C','NT5E','NTNG1','NTNG2','NTSR1','NUDCD3','NUDT16L1','NUDT9','NUMB','NUP35','NUS1','OAS3','OBSCN','OCA2','OCLN','OCM','OCM2','ODC1','ODR4','OGT','OLFM4','OLIG2','OLR1','OPHN1','OPRK1','OR10P1','OR1N2','OR2C3','OR6B2','OR6B3','OR6C6','OR8S1','OSBPL10','OSBPL6','OSBPL7','OSCAR','OSCP1','OSGEP','OSR2','OTOP2','OTULIN','OXGR1','P3H3','PACSIN2','PACSIN3','PAFAH1B2','PAG1','PAIP1','PAK1','PAK4','PAK6','PALLD','PALM','PALM2','PALM2-AKAP2','PALM3','PALMD','PANX1','PANX2','PAQR8','PARD3','PARD3B','PARD6A','PARD6G','PARP6','PATJ','PAWR','PCBD2','PCDH1','PCDH17','PCDH7','PCDHA1','PCDHA11','PCDHA12','PCDHA4','PCDHA5','PCDHB1','PCDHB11','PCDHB12','PCDHB16','PCDHB4','PCDHB5','PCDHGA1','PCDHGA10','PCDHGA11','PCDHGA12','PCDHGA2','PCDHGA3','PCDHGA4','PCDHGA5','PCDHGA6','PCDHGA7','PCDHGA8','PCDHGA9','PCDHGB1','PCDHGB2','PCDHGB3','PCDHGB4','PCDHGB5','PCDHGB6','PCDHGB7','PCDHGC3','PCDHGC4','PCDHGC5','PCLO','PCMTD1','PDAP1','PDCD7','PDE3A','PDE4A','PDE4D','PDE9A','PDGFC','PDGFRA','PDHX','PDLIM1','PDLIM5','PDPN','PDZD2','PDZD8','PEAR1','PEBP1','PFDN1','PGAP3','PGLYRP2','PGLYRP4','PGRMC2','PHACTR1','PHACTR4','PHF20L1','PHGDH','PHLDB2','PHLDB3','PHOSPHO1','PHPT1','PI3','PI4K2A','PI4KA','PIEZO2','PIGB','PIGN','PIGW','PIGX','PIK3AP1','PIK3CG','PIK3R6','PIP4K2A','PIP4K2C','PIP5K1A','PIRT','PITPNM3','PKD2L2','PKN1','PKN2','PKP1','PKP2','PKP3','PKP4','PLAA','PLAUR','PLCB1','PLCB2','PLCD3','PLCD4','PLEKHA4','PLEKHA6','PLEKHA7','PLEKHG3','PLEKHG4','PLEKHG6','PLEKHH2','PLEKHO2','PLIN4','PLPP1','PLPPR5','PLS3','PLSCR1','PLVAP','PLXNA3','PLXNA4','PNCK','PNISR','PODXL','POLE','POU2F3','PPEF1','PPFIBP1','PPIP5K1','PPL','PPM1A','PPM1F','PPP1CA','PPP1CB','PPP1R12B','PPP1R16A','PPP1R16B','PPP1R18','PPP1R27','PPP1R9A','PPP2R1B','PPP3R1','PPP3R2','PPP4C','PPP6R3','PPTC7','PRAME','PRC1','PRDX6','PRIMA1','PRKCA','PRKCE','PRKCH','PRKCZ','PRKD1','PROM2','PROSER2','PRR15','PRR32','PRR7','PRRC2A','PRRG1','PRRT2','PRRT3','PRRT4','PRTG','PRX','PSCA','PSD','PSD2','PSEN1','PSMC5','PSMC6','PSMD9','PSTPIP2','PTAFR','PTCHD4','PTK2B','PTK6','PTPN12','PTPN21','PTPN22','PTPN4','PTPRB','PTPRG','PTPRK','PTPRM','PTPRR','PTPRS','PTX3','PURG','PVALB','PVR','PXDC1','PXK','PYGL','PYGO2','PYM1','RAB13','RAB23','RAB25','RAB26','RAB27B','RAB3B','RAB3GAP2','RAB43','RAB8A','RABEPK','RAD51C','RAD52','RAI2','RALA','RALGAPA2','RAPGEF2','RAPH1','RAPSN','RASA4','RASA4B','RASAL1','RASAL2','RASAL3','RASD1','RASD2','RASGRP1','RASGRP2','RASGRP4','RASSF2','RASSF3','RASSF4','RBM46','RC3H1','RCCD1','RCSD1','RD3L','RDX','RECK','RELL1','REPS1','RET','RFLNB','RFPL1','RFPL2','RFPL3','RGP1','RGS10','RGS14','RGS19','RGS3','RGS7BP','RGS9BP','RHCE','RHD','RHEX','RHOA','RHOBTB2','RHOJ','RHOXF2','RHOXF2B','RIC8A','RIC8B','RIF1','RILPL1','RIMBP3','RIMBP3B','RIMBP3C','RIMS2','RIOK2','RIOK3','RIPK1','RND1','RNF114','RNF145','RNF146','RNF149','RNF165','RNF166','RNF39','ROBO1','ROBO3','ROBO4','ROM1','RP2','RPE','RPEL1','RPGRIP1L','RPIA','RPL36A','RPL36A-HNRNPH2','RPL36AL','RPSA','RPUSD1','RRAD','RRP12','RRP7A','RSBN1L','RSC1A1','RSPH3','RTL3','RTL4','RTN4R','RTN4RL1','RYR2','S100A13','S100A14','S100A16','S100A2','S100A3','S100A4','S100A6','S100A9','SAMD4A','SAMHD1','SAMSN1','SAPCD1','SASH3','SC5D','SCAMP1','SCAMP4','SCEL','SCHIP1','SCIN','SCN1A','SCN5A','SCN8A','SCN9A','SCNN1D','SCNN1G','SCRIB','SCUBE3','SDC2','SDC4','SDE2','SDHB','SEC16B','SEMA3C','SEMA3D','SEMA4B','SEMA4F','SEMA6C','SEMA6D','SEPT2','SEPT3','SERINC2','SERINC5','SERPINB3','SFSWAP','SGCE','SGK494','SH3BGR','SH3D21','SHANK1','SHANK3','SHARPIN','SHISA6','SHKBP1','SHROOM2','SHTN1','SIGLEC10','SIGLEC14','SIGLEC6','SIGLEC7','SIN3B','SIPA1','SIPA1L1','SIPA1L3','SKAP1','SLAMF6','SLC10A5','SLC12A2','SLC13A3','SLC13A5','SLC14A1','SLC16A1','SLC16A10','SLC16A2','SLC16A3','SLC16A7','SLC16A9','SLC17A5','SLC19A1','SLC19A3','SLC1A5','SLC1A6','SLC22A16','SLC23A3','SLC24A4','SLC26A9','SLC27A1','SLC28A1','SLC2A1','SLC2A11','SLC2A12','SLC2A14','SLC2A2','SLC2A3','SLC30A1','SLC31A2','SLC34A1','SLC34A2','SLC35F5','SLC35G2','SLC36A4','SLC38A5','SLC38A6','SLC38A7','SLC39A10','SLC39A14','SLC39A2','SLC39A6','SLC3A2','SLC40A1','SLC41A3','SLC43A1','SLC43A3','SLC44A2','SLC45A4','SLC46A1','SLC46A3','SLC4A1AP','SLC4A2','SLC4A7','SLC4A8','SLC5A12','SLC5A3','SLC5A6','SLC5A7','SLC6A6','SLC7A14','SLC7A2','SLC7A5','SLC7A6','SLC8A1','SLC9A1','SLC9A2','SLC9A3','SLC9A3R1','SLC9A3R2','SLC9A4','SLC9A5','SLC9B2','SLC9C2','SLCO1A2','SLCO1B3','SLCO4A1','SLCO4C1','SLCO5A1','SLIT1','SLIT3','SLITRK6','SLK','SLX4','SMAGP','SMAP1','SMOC1','SMPD2','SMPX','SNAP23','SNAP25','SNRK','SNTB2','SNTG2','SNX15','SNX32','SNX9','SOCS3','SOCS5','SOD1','SORBS1','SORCS1','SORT1','SOX11','SPAG16','SPANXN4','SPART','SPATA12','SPATC1L','SPERT','SPG11','SPHK1','SPINK13','SPINT1','SPINT2','SPN','SPPL2B','SPPL3','SPRED3','SPRYD3','SPTBN2','SRC','SRCIN1','SRD5A3','SRPK1','SRPRB','SRRM3','SSFA2','SSH1','SSH3','SSTR1','SSTR5','SSX2IP','ST8SIA5','STAB1','STAC','STAC2','STAMBP','STAMBPL1','STARD4','STAT2','STEAP4','STIL','STIP1','STK10','STK16','STK17A','STK17B','STOM','STOML2','STOML3','STX18','STX4','STXBP4','STYK1','SUCLG1','SUCLG2','SUGT1','SUMF2','SUPT6H','SUSD2','SVIL','SWAP70','SYCP2','SYK','SYMPK','SYNC','SYNGR3','SYNPO2L','SYPL2','SYT4','SYTL2','SYVN1','TACC2','TACR2','TACSTD2','TASP1','TAZ','TBC1D10A','TBC1D2','TBC1D24','TBC1D30','TBC1D9B','TBXA2R','TCAF2','TCAF2C','TCHP','TCIM','TDG','TDO2','TDP1','TEC','TEDC2','TEK','TES','TEX14','TEX9','TGFBR1','TGFBR2','TGM1','TGM2','THBS1','THBS2','THEGL','THY1','TIAM1','TICAM1','TIGD7','TIMD4','TJP1','TJP2','TJP3','TLN1','TLR4','TM2D2','TM4SF20','TM4SF4','TM4SF5','TMC5','TMED8','TMEM101','TMEM107','TMEM108','TMEM121B','TMEM139','TMEM141','TMEM164','TMEM176B','TMEM183A','TMEM186','TMEM2','TMEM218','TMEM222','TMEM229A','TMEM245','TMEM251','TMEM254','TMEM258','TMEM266','TMEM268','TMEM50A','TMEM63B','TMPRSS15','TMPRSS2','TMTC2','TNFRSF10D','TNFRSF11A','TNFRSF12A','TNFRSF21','TNFRSF25','TNFRSF8','TNFRSF9','TNFSF13B','TNK1','TNKS1BP1','TNNI2','TOGARAM2','TOLLIP','TOM1','TP53BP2','TPD52L1','TPRN','TRAF7','TRAIP','TRAPPC13','TRAT1','TRDN','TREML1','TRIB1','TRIM16','TRIM16L','TRIM38','TRIM4','TRIM44','TRIM54','TRIM71','TRIML2','TRIP6','TRMT1','TRMT10B','TRMT13','TRMT6','TRO','TRPM1','TRPM4','TRPM7','TRPV2','TRPV3','TSG101','TSGA13','TSHZ3','TSPAN15','TSPAN6','TSPO2','TSR2','TSSK4','TTC17','TTC23','TTC7B','TTLL12','TTLL5','TULP3','TWF2','TXNDC11','TXNIP','TYROBP','TYW3','UAP1','UBA3','UBA52','UBAC1','UBALD1','UBAP1','UBASH3B','UBE2A','UBE2B','UBE2C','UBE2D1','UBE2D2','UBE2D3','UBE2D4','UBE2K','UBE2QL1','UBE2S','UBE3C','UBQLN2','UBR2','UBTD1','UCP3','ULBP1','ULBP2','UNC5A','UNC79','UQCC1','USP14','USP15','USP2','USP20','USP21','USP4','USP53','UTP20','UTRN','VAMP5','VAMP8','VANGL1','VASP','VCAM1','VCPIP1','VIL1','VPS13B','VPS36','VPS50','VPS8','VSIG4','VTCN1','VWA5B1','VWA7','WAS','WASF2','WBP4','WDFY3','WDPCP','WDR1','WDR12','WDR13','WDR53','WDR6','WDR66','WIPF1','WWP1','XIAP','XIRP1','XIRP2','XPC','XPNPEP2','XPO6','XRN1','XYLT1','YES1','YIPF1','YIPF4','YOD1','YTHDC1','ZBED3','ZBTB33','ZBTB8OS','ZDHHC2','ZDHHC20','ZDHHC22','ZDHHC5','ZFP41','ZFP90','ZFYVE28','ZGPAT','ZMYND10','ZNF10','ZNF18','ZNF185','ZNF2','ZNF20','ZNF227','ZNF229','ZNF236','ZNF276','ZNF277','ZNF337','ZNF397','ZNF407','ZNF497','ZNF503','ZNF510','ZNF536','ZNF571','ZNF598','ZNF599','ZNF646','ZNF655','ZNF688','ZNF706','ZNF717','ZNF783','ZNF79','ZNF839','ZSCAN22','ZSCAN29','ZSWIM1','ZSWIM8','ZYX']
MembraneProteins=MembraneProteins1+MembraneProteins2

TFs1=['AC008770.3','AC023509.3','AC092835.1','AC138696.1','ADNP','ADNP2','AEBP1','AEBP2','AHCTF1','AHDC1','AHR','AHRR','AIRE','AKAP8','AKAP8L','AKNA','ALX1','ALX3','ALX4','ANHX','ANKZF1','AR','ARGFX','ARHGAP35','ARID2','ARID3A','ARID3B','ARID3C','ARID5A','ARID5B','ARNT','ARNT2','ARNTL','ARNTL2','ARX','ASCL1','ASCL2','ASCL3','ASCL4','ASCL5','ASH1L','ATF1','ATF2','ATF3','ATF4','ATF5','ATF6','ATF6B','ATF7','ATMIN','ATOH1','ATOH7','ATOH8','BACH1','BACH2','BARHL1','BARHL2','BARX1','BARX2','BATF','BATF2','BATF3','BAZ2A','BAZ2B','BBX','BCL11A','BCL11B','BCL6','BCL6B','BHLHA15','BHLHA9','BHLHE22','BHLHE23','BHLHE40','BHLHE41','BNC1','BNC2','BORCS8-MEF2B','BPTF','BRF2','BSX','C11orf95','CAMTA1','CAMTA2','CARF','CASZ1','CBX2','CC2D1A','CCDC169-SOHLH2','CCDC17','CDC5L','CDX1','CDX2','CDX4','CEBPA','CEBPB','CEBPD','CEBPE','CEBPG','CEBPZ','CENPA','CENPB','CENPBD1','CENPS','CENPT','CENPX','CGGBP1','CHAMP1','CHCHD3','CIC','CLOCK','CPEB1','CPXCR1','CREB1','CREB3','CREB3L1','CREB3L2','CREB3L3','CREB3L4','CREB5','CREBL2','CREBZF','CREM','CRX','CSRNP1','CSRNP2','CSRNP3','CTCF','CTCFL','CUX1','CUX2','CXXC1','CXXC4','CXXC5','DACH1','DACH2','DBP','DBX1','DBX2','DDIT3','DEAF1','DLX1','DLX2','DLX3','DLX4','DLX5','DLX6','DMBX1','DMRT1','DMRT2','DMRT3','DMRTA1','DMRTA2','DMRTB1','DMRTC2','DMTF1','DNMT1','DNTTIP1','DOT1L','DPF1','DPF3','DPRX','DR1','DRAP1','DRGX','DUX1','DUX3','DUX4','DUXA','DZIP1','E2F1','E2F2','E2F3','E2F4','E2F5','E2F6','E2F7','E2F8','E4F1','EBF1','EBF2','EBF3','EBF4','EEA1','EGR1','EGR2','EGR3','EGR4','EHF','ELF1','ELF2','ELF3','ELF4','ELF5','ELK1','ELK3','ELK4','EMX1','EMX2','EN1','EN2','EOMES','EPAS1','ERF','ERG','ESR1','ESR2','ESRRA','ESRRB','ESRRG','ESX1','ETS1','ETS2','ETV1','ETV2','ETV3','ETV3L','ETV4','ETV5','ETV6','ETV7','EVX1','EVX2','FAM170A','FAM200B','FBXL19','FERD3L','FEV','FEZF1','FEZF2','FIGLA','FIZ1','FLI1','FLYWCH1','FOS','FOSB','FOSL1','FOSL2','FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXN4','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1','GABPA','GATA1','GATA2','GATA3','GATA4','GATA5','GATA6','GATAD2A','GATAD2B','GBX1','GBX2','GCM1','GCM2','GFI1','GFI1B','GLI1','GLI2','GLI3','GLI4','GLIS1','GLIS2','GLIS3','GLMP','GLYR1','GMEB1','GMEB2','GPBP1','GPBP1L1','GRHL1','GRHL2','GRHL3','GSC','GSC2','GSX1','GSX2','GTF2B','GTF2I','GTF2IRD1','GTF2IRD2','GTF2IRD2B','GTF3A','GZF1','HAND1','HAND2','HBP1','HDX','HELT','HES1','HES2','HES3','HES4','HES5','HES6','HES7','HESX1','HEY1','HEY2','HEYL','HHEX','HIC1','HIC2','HIF1A','HIF3A','HINFP','HIVEP1','HIVEP2','HIVEP3','HKR1','HLF','HLX','HMBOX1','HMG20A','HMG20B','HMGA1','HMGA2','HMGN3','HMX1','HMX2','HMX3','HNF1A','HNF1B','HNF4A','HNF4G','HOMEZ','HOXA1','HOXA10','HOXA11','HOXA13','HOXA2','HOXA3','HOXA4','HOXA5','HOXA6','HOXA7','HOXA9','HOXB1','HOXB13','HOXB2','HOXB3','HOXB4','HOXB5','HOXB6','HOXB7','HOXB8','HOXB9','HOXC10','HOXC11','HOXC12','HOXC13','HOXC4','HOXC5','HOXC6','HOXC8','HOXC9','HOXD1','HOXD10','HOXD11','HOXD12','HOXD13','HOXD3','HOXD4','HOXD8','HOXD9','HSF1','HSF2','HSF4','HSF5','HSFX1','HSFX2','HSFY1','HSFY2','IKZF1','IKZF2','IKZF3','IKZF4','IKZF5','INSM1','INSM2','IRF1','IRF2','IRF3','IRF4','IRF5','IRF6','IRF7','IRF8','IRF9','IRX1','IRX2','IRX3','IRX4','IRX5','IRX6','ISL1','ISL2','ISX','JAZF1','JDP2','JRK','JRKL','JUN','JUNB','JUND','KAT7','KCMF1','KCNIP3','KDM2A','KDM2B','KDM5B','KIN','KLF1','KLF10','KLF11','KLF12','KLF13','KLF14','KLF15','KLF16','KLF17','KLF2','KLF3','KLF4','KLF5','KLF6','KLF7','KLF8','KLF9','KMT2A','KMT2B','L3MBTL1','L3MBTL3','L3MBTL4','LBX1','LBX2','LCOR','LCORL','LEF1','LEUTX','LHX1','LHX2','LHX3','LHX4','LHX5','LHX6','LHX8','LHX9','LIN28A','LIN28B','LIN54','LMX1A','LMX1B','LTF','LYL1','MAF','MAFA','MAFB','MAFF','MAFG','MAFK','MAX','MAZ','MBD1','MBD2','MBD3','MBD4','MBD6','MBNL2','MECOM','MECP2','MEF2A','MEF2B','MEF2C','MEF2D','MEIS1','MEIS2','MEIS3','MEOX1','MEOX2','MESP1','MESP2','MGA','MITF','MIXL1','MKX','MLX','MLXIP','MLXIPL','MNT','MNX1','MSANTD1','MSANTD3','MSANTD4','MSC','MSGN1','MSX1','MSX2','MTERF1','MTERF2','MTERF3','MTERF4','MTF1','MTF2','MXD1','MXD3','MXD4','MXI1','MYB','MYBL1','MYBL2','MYC','MYCL','MYCN','MYF5','MYF6','MYNN','MYOD1','MYOG','MYPOP','MYRF','MYRFL','MYSM1','MYT1','MYT1L','MZF1','NACC2','NAIF1','NANOG','NANOGNB','NANOGP8','NCOA1','NCOA2','NCOA3','NEUROD1','NEUROD2','NEUROD4','NEUROD6','NEUROG1','NEUROG2','NEUROG3','NFAT5','NFATC1','NFATC2','NFATC3','NFATC4','NFE2','NFE2L1','NFE2L2','NFE2L3','NFE4','NFIA','NFIB','NFIC','NFIL3','NFIX','NFKB1','NFKB2','NFX1','NFXL1','NFYA','NFYB','NFYC','NHLH1','NHLH2','NKRF','NKX1-1','NKX1-2','NKX2-1','NKX2-2','NKX2-3','NKX2-4','NKX2-5','NKX2-6','NKX2-8','NKX3-1','NKX3-2','NKX6-1','NKX6-2','NKX6-3','NME2','NOBOX','NOTO','NPAS1','NPAS2','NPAS3','NPAS4','NR0B1','NR1D1','NR1D2','NR1H2','NR1H3','NR1H4','NR1I2','NR1I3','NR2C1','NR2C2','NR2E1','NR2E3','NR2F1','NR2F2','NR2F6','NR3C1','NR3C2','NR4A1','NR4A2','NR4A3','NR5A1','NR5A2','NR6A1','NRF1','NRL','OLIG1','OLIG2','OLIG3','ONECUT1','ONECUT2','ONECUT3','OSR1','OSR2','OTP','OTX1','OTX2','OVOL1','OVOL2','OVOL3','PA2G4','PATZ1','PAX1','PAX2','PAX3','PAX4','PAX5','PAX6','PAX7','PAX8','PAX9','PBX1','PBX2','PBX3','PBX4','PCGF2','PCGF6','PDX1','PEG3','PGR','PHF1','PHF19 ','PHF20','PHF21A','PHOX2A','PHOX2B','PIN1','PITX1','PITX2','PITX3','PKNOX1','PKNOX2','PLAG1','PLAGL1','PLAGL2','PLSCR1','POGK','POU1F1','POU2AF1','POU2F1','POU2F2','POU2F3','POU3F1','POU3F2','POU3F3','POU3F4','POU4F1','POU4F2','POU4F3','POU5F1','POU5F1B','POU5F2','POU6F1','POU6F2','PPARA','PPARD','PPARG','PRDM1','PRDM10','PRDM12','PRDM13','PRDM14','PRDM15','PRDM16','PRDM2','PRDM4','PRDM5','PRDM6','PRDM8','PRDM9','PREB','PRMT3','PROP1','PROX1','PROX2','PRR12','PRRX1','PRRX2','PTF1A','PURA','PURB','PURG','RAG1','RARA','RARB','RARG','RAX','RAX2','RBAK','RBCK1','RBPJ','RBPJL','RBSN','REL','RELA','RELB','REPIN1','REST','REXO4','RFX1','RFX2','RFX3','RFX4','RFX5','RFX6','RFX7','RFX8','RHOXF1','RHOXF2','RHOXF2B','RLF','RORA','RORB','RORC','RREB1','RUNX1','RUNX2','RUNX3','RXRA','RXRB','RXRG','SAFB','SAFB2','SALL1','SALL2','SALL3','SALL4','SATB1','SATB2','SCMH1','SCML4','SCRT1','SCRT2','SCX','SEBOX','SETBP1','SETDB1','SETDB2','SGSM2','SHOX','SHOX2','SIM1','SIM2','SIX1','SIX2','SIX3','SIX4','SIX5','SIX6','SKI','SKIL','SKOR1','SKOR2','SLC2A4RG','SMAD1','SMAD3','SMAD4','SMAD5','SMAD9','SMYD3','SNAI1','SNAI2','SNAI3','SNAPC2','SNAPC4','SNAPC5','SOHLH1','SOHLH2','SON','SOX1','SOX10','SOX11']
TFs2=['SOX12','SOX13','SOX14','SOX15','SOX17','SOX18','SOX2','SOX21','SOX3','SOX30','SOX4','SOX5','SOX6','SOX7','SOX8','SOX9','SP1','SP100','SP110','SP140','SP140L','SP2','SP3','SP4','SP5','SP6','SP7','SP8','SP9','SPDEF','SPEN','SPI1','SPIB','SPIC','SPZ1','SRCAP','SREBF1','SREBF2','SRF','SRY','ST18','STAT1','STAT2','STAT3','STAT4','STAT5A','STAT5B','STAT6','T','TAL1','TAL2','TBP','TBPL1','TBPL2','TBR1','TBX1','TBX10','TBX15','TBX18','TBX19','TBX2','TBX20','TBX21','TBX22','TBX3','TBX4','TBX5','TBX6','TCF12','TCF15','TCF20','TCF21','TCF23','TCF24','TCF3','TCF4','TCF7','TCF7L1','TCF7L2','TCFL5','TEAD1','TEAD2','TEAD3','TEAD4','TEF','TERB1','TERF1','TERF2','TET1','TET2','TET3','TFAP2A','TFAP2B','TFAP2C','TFAP2D','TFAP2E','TFAP4','TFCP2','TFCP2L1','TFDP1','TFDP2','TFDP3','TFE3','TFEB','TFEC','TGIF1','TGIF2','TGIF2LX','TGIF2LY','THAP1','THAP10','THAP11','THAP12','THAP2','THAP3','THAP4','THAP5','THAP6','THAP7','THAP8','THAP9','THRA','THRB','THYN1','TIGD1','TIGD2','TIGD3','TIGD4','TIGD5','TIGD6','TIGD7','TLX1','TLX2','TLX3','TMF1','TOPORS','TP53','TP63','TP73','TPRX1','TRAFD1','TRERF1','TRPS1','TSC22D1','TSHZ1','TSHZ2','TSHZ3','TTF1','TWIST1','TWIST2','UBP1','UNCX','USF1','USF2','USF3','VAX1','VAX2','VDR','VENTX','VEZF1','VSX1','VSX2','WIZ','WT1','XBP1','XPA','YBX1','YBX2','YBX3','YY1','YY2','ZBED1','ZBED2','ZBED3','ZBED4','ZBED5','ZBED6','ZBED9','ZBTB1','ZBTB10','ZBTB11','ZBTB12','ZBTB14','ZBTB16','ZBTB17','ZBTB18','ZBTB2','ZBTB20','ZBTB21','ZBTB22','ZBTB24','ZBTB25','ZBTB26','ZBTB3','ZBTB32','ZBTB33','ZBTB34','ZBTB37','ZBTB38','ZBTB39','ZBTB4','ZBTB40','ZBTB41','ZBTB42','ZBTB43','ZBTB44','ZBTB45','ZBTB46','ZBTB47','ZBTB48','ZBTB49','ZBTB5','ZBTB6','ZBTB7A','ZBTB7B','ZBTB7C','ZBTB8A','ZBTB8B','ZBTB9','ZC3H8','ZEB1','ZEB2','ZFAT','ZFHX2','ZFHX3','ZFHX4','ZFP1','ZFP14','ZFP2','ZFP28','ZFP3','ZFP30','ZFP37','ZFP41','ZFP42','ZFP57','ZFP62','ZFP64','ZFP69','ZFP69B','ZFP82','ZFP90','ZFP91','ZFP92','ZFPM1','ZFPM2','ZFX','ZFY','ZGLP1','ZGPAT','ZHX1','ZHX2','ZHX3','ZIC1','ZIC2','ZIC3','ZIC4','ZIC5','ZIK1','ZIM2','ZIM3','ZKSCAN1','ZKSCAN2','ZKSCAN3','ZKSCAN4','ZKSCAN5','ZKSCAN7','ZKSCAN8','ZMAT1','ZMAT4','ZNF10','ZNF100','ZNF101','ZNF107','ZNF112','ZNF114','ZNF117','ZNF12','ZNF121','ZNF124','ZNF131','ZNF132','ZNF133','ZNF134','ZNF135','ZNF136','ZNF138','ZNF14','ZNF140','ZNF141','ZNF142','ZNF143','ZNF146','ZNF148','ZNF154','ZNF155','ZNF157','ZNF16','ZNF160','ZNF165','ZNF169','ZNF17','ZNF174','ZNF175','ZNF177','ZNF18','ZNF180','ZNF181','ZNF182','ZNF184','ZNF189','ZNF19','ZNF195','ZNF197','ZNF2','ZNF20','ZNF200','ZNF202','ZNF205','ZNF207','ZNF208','ZNF211','ZNF212','ZNF213','ZNF214','ZNF215','ZNF217','ZNF219','ZNF22','ZNF221','ZNF222','ZNF223','ZNF224','ZNF225','ZNF226','ZNF227','ZNF229','ZNF23','ZNF230','ZNF232','ZNF233','ZNF234','ZNF235','ZNF236','ZNF239','ZNF24','ZNF248','ZNF25','ZNF250','ZNF251','ZNF253','ZNF254','ZNF256','ZNF257','ZNF26','ZNF260','ZNF263','ZNF264','ZNF266','ZNF267','ZNF268','ZNF273','ZNF274','ZNF275','ZNF276','ZNF277','ZNF28','ZNF280A','ZNF280B','ZNF280C','ZNF280D','ZNF281','ZNF282','ZNF283','ZNF284','ZNF285','ZNF286A','ZNF286B','ZNF287','ZNF292','ZNF296','ZNF3','ZNF30','ZNF300','ZNF302','ZNF304','ZNF311','ZNF316','ZNF317','ZNF318','ZNF319','ZNF32','ZNF320','ZNF322','ZNF324','ZNF324B','ZNF326','ZNF329','ZNF331','ZNF333','ZNF334','ZNF335','ZNF337','ZNF33A','ZNF33B','ZNF34','ZNF341','ZNF343','ZNF345','ZNF346','ZNF347','ZNF35','ZNF350','ZNF354A','ZNF354B','ZNF354C','ZNF358','ZNF362','ZNF365','ZNF366','ZNF367','ZNF37A','ZNF382','ZNF383','ZNF384','ZNF385A','ZNF385B','ZNF385C','ZNF385D','ZNF391','ZNF394','ZNF395','ZNF396','ZNF397','ZNF398','ZNF404','ZNF407','ZNF408','ZNF41','ZNF410','ZNF414','ZNF415','ZNF416','ZNF417','ZNF418','ZNF419','ZNF420','ZNF423','ZNF425','ZNF426','ZNF428','ZNF429','ZNF43','ZNF430','ZNF431','ZNF432','ZNF433','ZNF436','ZNF438','ZNF439','ZNF44','ZNF440','ZNF441','ZNF442','ZNF443','ZNF444','ZNF445','ZNF446','ZNF449','ZNF45','ZNF451','ZNF454','ZNF460','ZNF461','ZNF462','ZNF467','ZNF468','ZNF469','ZNF470','ZNF471','ZNF473','ZNF474','ZNF479','ZNF48','ZNF480','ZNF483','ZNF484','ZNF485','ZNF486','ZNF487','ZNF488','ZNF490','ZNF491','ZNF492','ZNF493','ZNF496','ZNF497','ZNF500','ZNF501','ZNF502','ZNF503','ZNF506','ZNF507','ZNF510','ZNF511','ZNF512','ZNF512B','ZNF513','ZNF514','ZNF516','ZNF517','ZNF518A','ZNF518B','ZNF519','ZNF521','ZNF524','ZNF525','ZNF526','ZNF527','ZNF528','ZNF529','ZNF530','ZNF532','ZNF534','ZNF536','ZNF540','ZNF541','ZNF543','ZNF544','ZNF546','ZNF547','ZNF548','ZNF549','ZNF550','ZNF551','ZNF552','ZNF554','ZNF555','ZNF556','ZNF557','ZNF558','ZNF559','ZNF560','ZNF561','ZNF562','ZNF563','ZNF564','ZNF565','ZNF566','ZNF567','ZNF568','ZNF569','ZNF57','ZNF570','ZNF571','ZNF572','ZNF573','ZNF574','ZNF575','ZNF576','ZNF577','ZNF578','ZNF579','ZNF580','ZNF581','ZNF582','ZNF583','ZNF584','ZNF585A','ZNF585B','ZNF586','ZNF587','ZNF587B','ZNF589','ZNF592','ZNF594','ZNF595','ZNF596','ZNF597','ZNF598','ZNF599','ZNF600','ZNF605','ZNF606','ZNF607','ZNF608','ZNF609','ZNF610','ZNF611','ZNF613','ZNF614','ZNF615','ZNF616','ZNF618','ZNF619','ZNF620','ZNF621','ZNF623','ZNF624','ZNF625','ZNF626','ZNF627','ZNF628','ZNF629','ZNF630','ZNF639','ZNF641','ZNF644','ZNF645','ZNF646','ZNF648','ZNF649','ZNF652','ZNF653','ZNF654','ZNF655','ZNF658','ZNF66','ZNF660','ZNF662','ZNF664','ZNF665','ZNF667','ZNF668','ZNF669','ZNF670','ZNF671','ZNF672','ZNF674','ZNF675','ZNF676','ZNF677','ZNF678','ZNF679','ZNF680','ZNF681','ZNF682','ZNF683','ZNF684','ZNF687','ZNF688','ZNF689','ZNF69','ZNF691','ZNF692','ZNF695','ZNF696','ZNF697','ZNF699','ZNF7','ZNF70','ZNF700','ZNF701','ZNF703','ZNF704','ZNF705A','ZNF705B','ZNF705D','ZNF705E','ZNF705G','ZNF706','ZNF707','ZNF708','ZNF709','ZNF71','ZNF710','ZNF711','ZNF713','ZNF714','ZNF716','ZNF717','ZNF718','ZNF721','ZNF724','ZNF726','ZNF727','ZNF728','ZNF729','ZNF730','ZNF732','ZNF735','ZNF736','ZNF737','ZNF74','ZNF740','ZNF746','ZNF747','ZNF749','ZNF750','ZNF75A','ZNF75D','ZNF76','ZNF761','ZNF763','ZNF764','ZNF765','ZNF766','ZNF768','ZNF77','ZNF770','ZNF771','ZNF772','ZNF773','ZNF774','ZNF775','ZNF776','ZNF777','ZNF778','ZNF780A','ZNF780B','ZNF781','ZNF782','ZNF783','ZNF784','ZNF785','ZNF786','ZNF787','ZNF788','ZNF789','ZNF79','ZNF790','ZNF791','ZNF792','ZNF793','ZNF799','ZNF8','ZNF80','ZNF800','ZNF804A','ZNF804B','ZNF805','ZNF808','ZNF81','ZNF813','ZNF814','ZNF816','ZNF821','ZNF823','ZNF827','ZNF829','ZNF83','ZNF830','ZNF831','ZNF835','ZNF836','ZNF837','ZNF84','ZNF841','ZNF843','ZNF844','ZNF845','ZNF846','ZNF85','ZNF850','ZNF852','ZNF853','ZNF860','ZNF865','ZNF878','ZNF879','ZNF880','ZNF883','ZNF888','ZNF891','ZNF90','ZNF91','ZNF92','ZNF93','ZNF98','ZNF99','ZSCAN1','ZSCAN10','ZSCAN12','ZSCAN16','ZSCAN18','ZSCAN2','ZSCAN20','ZSCAN21','ZSCAN22','ZSCAN23','ZSCAN25','ZSCAN26','ZSCAN29','ZSCAN30','ZSCAN31','ZSCAN32','ZSCAN4','ZSCAN5A','ZSCAN5B','ZSCAN5C','ZSCAN9','ZUFSP','ZXDA','ZXDB','ZXDC','ZZZ3']
TFs=TFs1+TFs2

#TFs other, inclusive of the above list, not necessarily DNA binding proteins but affect transcription
TFs1Other=['AATF','ABL1','ABRA','AC008770.3','AC020915.1','AC023509.3','AC092835.1','AC105001.2','AC138696.1','ACOD1','ACTR3B','ACTR3C','ADAM8','ADAMTS17','ADAMTS19','ADAR','ADGRA1','ADGRG3','ADNP','ADNP2','AEBP1','AEBP2','AES','AFAP1 ','AFF1','AFF3','AFF4','AGER','AGT','AHCTF1','AHDC1','AHR','AHRR','AIM2','AIRE','AKAP8','AKAP8L','AKNA','AKT1','ALK','ALKBH2','ALX1','ALX3','ALX4','AMH','ANAPC2','ANHX','ANKRD30A','ANKZF1','ANXA3','ANXA4','AP5Z1','APBB3','APC2','APEX1','APOA4','APOBEC3A','APOBEC3B','APOBEC3G','APTX','AR','ARGFX','ARHGAP35','ARHGEF2','ARHGEF5','ARID1A','ARID1B','ARID2','ARID3A','ARID3B','ARID3C','ARID4A','ARID4B','ARID5A','ARID5B','ARNT','ARNT2','ARNTL','ARNTL2','ARRB1','ARRB2','ARX','ASCC1','ASCL1','ASCL2','ASCL3','ASCL4','ASCL5','ASH1L','ATF1','ATF2','ATF3','ATF4','ATF5','ATF6','ATF6B','ATF7','ATF7IP','ATM','ATMIN','ATOH1','ATOH7','ATOH8','ATR','ATRX','ATXN7','BACH1','BACH2','BAG1','BANF1','BANP','BARHL1','BARHL2','BARX1','BARX2','BATF','BATF2','BATF3','BAZ2A','BAZ2B','BBX','BCL10','BCL11A','BCL11B','BCL3','BCL6','BCL6B','BCLAF1','BCOR','BDP1','BEND4','BEX1','BHLHA15','BHLHA9','BHLHE22','BHLHE23','BHLHE40','BHLHE41','BHMG1','BIN1','BLM','BLZF1','BMI1','BMP2','BMP4','BMP7','BMPR1A','BNC1','BNC2','BOD1L1','BOLA1','BOLA2','BOLA2B','BOLA3','BORCS8-MEF2B','BPNT1','BPTF','BRCA1','BRD4','BRD7','BRD8','BRD9','BRF1','BRF2','BRMS1','BRPF1','BSX','BTAF1','BTBD8','BTG2','BTK','BTRC','BUD31','C11orf95','C14orf39','C17orf49','C1D','C20orf194','C3orf33','CACTIN','CAMK1D','CAMK2A','CAMTA1','CAMTA2','CAPN15','CAPN3','CARD11','CARD14','CARF','CARHSP1','CASP8AP2','CASZ1','CAT','CBFA2T2','CBFA2T3','CBFB','CBL','CBLL1','CBX2','CC2D1A','CC2D1B','CCDC124','CCDC169-SOHLH2','CCDC17','CCDC3','CCDC71','CCDC83','CCNT1','CCNT2','CD36','CD40','CD40LG','CDC5L','CDC6','CDK5RAP3','CDK7','CDK8','CDK9','CDKN1A','CDKN2A','CDX1','CDX2','CDX4','CEBPA','CEBPB','CEBPD','CEBPE','CEBPG','CEBPZ','CELF3','CENPA','CENPB','CENPBD1','CENPS','CENPT','CENPX','CERS2','CERS3','CERS4','CERS5','CERS6','CFLAR','CGGBP1','CHAMP1','CHCHD3','CHD1','CHD2','CHD3','CHD4','CHD6','CHD7','CHD9','CHP1','CHRAC1','CHUK','CHURC1','CIB1','CIC','CIITA','CIR1','CITED1','CITED2','CITED4','CIZ1','CLOCK','CLU','CMKLR1','CNBP','CNOT3','CNOT4','CNOT7','CNOT8','COMMD1','COMMD6','COMMD7','COPS2','COPS3','COPS4','CPEB1','CPSF4','CPSF4L','CPXCR1','CRAMP1','CREB1','CREB3','CREB3L1','CREB3L2','CREB3L3','CREB3L4','CREB5','CREBBP','CREBL2','CREBRF','CREBZF','CREG1','CREM','CRTC1','CRTC2','CRTC3','CRX','CSDC2','CSDE1','CSRNP1','CSRNP2','CSRNP3','CTBP1','CTBP2','CTCF','CTCFL','CTH','CTNNB1','CTNNBIP1','CUL1','CUL2','CUL3','CUL4A','CUL4B','CUL5','CUX1','CUX2','CXXC1','CXXC4','CXXC5','CYLD','CYP1B1','CYTL1','DAB2IP','DACH1','DACH2','DAP','DAXX','DBP','DBX1','DBX2','DCP1A','DDB1','DDB2','DDIT3','DDN','DDR2','DDRGK1','DDX3X','DDX5','DDX58','DEAF1','DEK','DEPDC1','DEPDC1B','DEPDC4','DEPDC5','DEPDC7','DEPTOR','DHX34','DHX36','DHX57','DISP3','DLX1','DLX2','DLX3','DLX4','DLX5','DLX6','DMAP1','DMBX1','DMC1','DMRT1','DMRT2','DMRT3','DMRTA1','DMRTA2','DMRTB1','DMRTC1','DMRTC2','DMTF1','DNA2','DNAJA3','DNAJC1','DNAJC2','DNAJC21','DNMT1','DNMT3A','DNMT3B','DNMT3L','DNTT','DNTTIP1','DOT1L','DPF1','DPF2','DPF3','DPRX','DR1','DRAP1','DRGX','DSP','DTX1','DUS3L','DUSP12','DUX1','DUX3','DUX4','DUXA','DVL1','DVL2','DVL3','DZIP1','DZIP1L','E2F1','E2F2','E2F3','E2F4','E2F5','E2F6','E2F7','E2F8','E4F1','EAF2','EBF1','EBF2','EBF3','EBF4','EDA','EDA2R','EDF1','EDN1','EEA1','EED','EGLN1','EGR1','EGR2','EGR3','EGR4','EHF','EID1','EID2','EIF2AK2','EIF2AK4','EIF3C','EIF3CL','EIF3K','ELF1','ELF2','ELF3','ELF4','ELF5','ELK1','ELK3','ELK4','ELMSAN1','EME1','EMX1','EMX2','EN1','EN2','ENO1','EOMES','EP300','EP400','EPAS1','EPHA5','EPM2AIP1','ERBIN','ERC1','ERCC3','ERCC4','ERF','ERG','ESR1','ESR2','ESRRA','ESRRB','ESRRG','ESX1','ETS1','ETS2','ETV1','ETV2','ETV3','ETV3L','ETV4','ETV5','ETV6','ETV7','EVX1','EVX2','EWSR1','EXO1','EXOC2','EXOG','EZH1','EZH2','FAAP24','FAM153C','FAM170A','FAM171B','FAM200A','FAM200B','FAN1','FANCA','FANCD2','FANK1','FARSA','FARSB','FBN1','FBXL19','FBXO41','FEN1','FER','FERD3L','FEV','FEZF1','FEZF2','FGD1','FHL2','FIGLA','FIZ1','FLI1','FLNA','FLYWCH1','FMNL2','FOS','FOSB','FOSL1','FOSL2','FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L2','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXN4','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1','FUBP1','FUBP3','FUS','FZD1','FZD2','FZD4','FZD6','GABPA','GABPB1','GABPB2','GADD45A','GAS6','GAS7','GATA1','GATA2','GATA3','GATA4','GATA5','GATA6','GATAD1','GATAD2A','GATAD2B','GBX1','GBX2','GCFC2','GCM1','GCM2','GEN1','GFI1','GFI1B','GLA','GLI1','GLI2','GLI3','GLI4','GLIS1','GLIS2','GLIS3','GLMP','GLYR1','GMEB1','GMEB2','GOLGB1','GON4L','GPATCH8','GPBP1','GPBP1L1','GPER1','GPR155','GREM1','GRHL1','GRHL2','GRHL3','GRM6','GSC','GSC2','GSX1','GSX2','GTF2A1','GTF2A1L','GTF2A2','GTF2B','GTF2E2','GTF2F1','GTF2F2','GTF2H1','GTF2H2','GTF2H2C','GTF2H3','GTF2H4','GTF2I','GTF2IRD1','GTF2IRD2','GTF2IRD2B','GTF3A','GTF3C1','GTF3C2','GULP1','GZF1','H1F0','H1FOO','H1FX','HAND1','HAND2','HAVCR2','HBP1','HCFC1','HCK','HDAC1','HDAC10','HDAC2','HDAC3','HDAC4','HDAC5','HDAC6','HDAC7','HDAC8','HDAC9','HDX','HELT','HELZ','HELZ2','HES1','HES2','HES3','HES4','HES5','HES6','HES7','HESX1','HEY1','HEY2','HEYL','HHEX','HIC1','HIC2','HIF1A','HIF3A','HINFP','HIPK2','HIRA','HIST1H1A','HIST1H1B','HIST1H1C','HIST1H1D','HIST1H1E','HIST1H1T','HIVEP1','HIVEP2','HIVEP3','HKR1','HLA-DQB1','HLA-DQB2','HLA-DRB1','HLA-DRB3','HLA-DRB5','HLF','HLTF','HLX','HMBOX1','HMG20A','HMG20B','HMGA1','HMGA2','HMGB1','HMGB2','HMGB3','HMGB4','HMGN1','HMGN3','HMGXB3','HMGXB4','HMOX1','HMX1','HMX2','HMX3','HNF1A','HNF1B','HNF4A','HNF4G','HNRNPA1','HNRNPA3','HNRNPAB','HNRNPCL2','HNRNPD','HNRNPDL','HNRNPK','HNRNPL','HNRNPU','HOMEZ','HOOK2','HOPX','HOXA1','HOXA10','HOXA11','HOXA13','HOXA2','HOXA3','HOXA4','HOXA5','HOXA6','HOXA7','HOXA9','HOXB1','HOXB13','HOXB2','HOXB3','HOXB4','HOXB5','HOXB6','HOXB7','HOXB8','HOXB9','HOXC10','HOXC11','HOXC12','HOXC13','HOXC4','HOXC5','HOXC6','HOXC8','HOXC9','HOXD1','HOXD10','HOXD11','HOXD12','HOXD13','HOXD3','HOXD4','HOXD8','HOXD9','HP1BP3','HR','HSBP1','HSF1','HSF2','HSF4','HSF5','HSFX1','HSFX2','HSFY1','HSFY2','HSPA1A','HSPA1B','ICAM1','ID1','ID2','ID3','ID4','IFI16','IFNB1','IGHM','IKBKB','IKBKG','IKZF1','IKZF2','IKZF3','IKZF4','IKZF5','IL10','IL1B','IL1RAP','IL4','IL5','IL6','INF2','ING4','INS','INSM1','INSM2','IQGAP1','IRAK1','IRAK2','IRAK3','IRF1','IRF2','IRF2BP1','IRF3','IRF4','IRF5','IRF6','IRF7','IRF8','IRF9','IRX1','IRX2','IRX3','IRX4','IRX5','IRX6','ISL1','ISL2','ISX','ITCH','ITGB2','ITGB3BP','IZUMO2','JADE1','JAK2','JARID2','JAZF1','JDP2','JMY','JRK','JRKL','JUN','JUNB','JUND','JUP','KANSL2','KAT2A','KAT2B','KAT5','KAT6A','KAT6B','KAT7','KAT8','KCMF1','KCNIP3','KDM1A','KDM2A','KDM2B','KDM3A','KDM5A','KDM5B','KDM5C','KDM5D','KEAP1','KIAA1549','KIAA1683','KIAA1841','KIAA1958','KIF15','KIN','KIT','KLF1','KLF10','KLF11','KLF12','KLF13','KLF14','KLF15','KLF16','KLF17','KLF2','KLF3','KLF4','KLF5','KLF6','KLF7','KLF8','KLF9','KMT2A','KMT2B','KMT2C','KMT2D','KNTC1','KRAS','KRBOX1','KRT13','KRTAP5-1','KRTAP5-9','KRTAP9-9','L3MBTL1','L3MBTL3','L3MBTL4','LARP1','LARP1B','LARP4','LARP4B','LARP6','LARP7','LBX1','LBX2','LCOR','LCORL','LEF1','LEUTX','LGALS9','LGR4','LHX1','LHX2','LHX3','LHX4','LHX5','LHX6','LHX8','LHX9','LIG1','LIG3','LIN28A','LIN28B','LIN54','LMO2','LMO4','LMX1A','LMX1B','LPP','LRP5','LRP6','LRP8','LRRFIP1','LRRFIP2','LSR','LTF','LTN1','LYAR','LYL1','LZTFL1','LZTR1','LZTS1','MACC1','MACF1','MAD2L2','MAEL','MAF','MAFA','MAFB','MAFF','MAFG','MAFK','MALT1','MAML1','MAML3','MAP2K5','MAP3K10','MAP3K13','MAP3K7','MAPK1','MAPK10','MAPK11','MAPK14','MAPK3','MAPK8','MAPK9','MATR3','MAVS','MAX','MAZ','MB21D1','MBD1','MBD2','MBD3','MBD4','MBD6','MBNL1','MBNL2','MBNL3','MBTPS2','MDFI','MDM2','MDM4','MECOM','MECP2','MED1','MED12','MED13','MED13L','MED14','MED15','MED16','MED17','MED21','MED23','MED24','MED6','MEF2A','MEF2B','MEF2C','MEF2D','MEIS1','MEIS2','MEIS3','MEN1','MEOX1','MEOX2','MESP1','MESP2','MET','MGA','MGMT','MID2','MIER1','MIER2','MIER3','MINK1','MIS18BP1','MITF','MIXL1','MKL1','MKL2','MKRN1','MKRN2','MKRN3','MKX','MLLT1','MLLT10','MLLT11','MLLT3','MLLT6','MLX','MLXIP','MLXIPL','MN1','MNAT1','MNT','MNX1','MORC3','MPG','MRPL28','MRRF','MSANTD1','MSANTD3','MSANTD4','MSC','MSGN1','MSH2','MSH3','MSH6','MSL3','MSRB2','MST1R','MSX1','MSX2','MTA1','MTA2','MTA3','MTDH','MTERF1','MTERF2','MTERF3','MTERF4','MTF1','MTF2','MTPN','MUS81','MXD1','MXD3','MXD4','MXI1','MYB','MYBBP1A','MYBL1','MYBL2','MYC','MYCBP','MYCL','MYCN','MYD88','MYEF2','MYF5','MYF6','MYNN','MYOCD','MYOD1','MYOG','MYPOP','MYRF','MYRFL','MYSM1','MYT1','MYT1L','MZF1','NAB2','NABP2','NACC2','NAIF1','NANOG','NANOGNB','NANOGP8','NCOA1','NCOA2','NCOA3','NCOA4','NCOA5','NCOA6','NCOA7','NCOR1','NCOR2','NDC80','NDN','NDP','NEB','NEIL1','NEUROD1','NEUROD2','NEUROD4','NEUROD6','NEUROG1','NEUROG2','NEUROG3','NFAM1','NFAT5','NFATC1','NFATC2','NFATC3','NFATC4','NFE2','NFE2L1','NFE2L2','NFE2L3','NFE4','NFIA','NFIB','NFIC','NFIL3','NFIX','NFKB1','NFKB2','NFKBIA','NFKBIB','NFKBID','NFKBIL1','NFKBIZ','NFRKB','NFX1','NFXL1','NFYA','NFYB','NFYC','NHLH1','NHLH2','NKRF','NKX1-1','NKX1-2','NKX2-1','NKX2-2','NKX2-3','NKX2-4','NKX2-5','NKX2-6','NKX2-8','NKX3-1','NKX3-2','NKX6-1','NKX6-2','NKX6-3','NLRC3','NLRC4','NLRC5','NLRP12','NLRP2B','NLRP3','NME2','NOBOX','NOC3L','NOC4L','NOCT','NOD1','NOD2','NODAL','NONO','NOP2','NOTCH1','NOTCH2','NOTCH3','NOTO','NPAS1','NPAS2','NPAS3','NPAS4','NPM1','NR0B1','NR0B2','NR1D1','NR1D2','NR1H2','NR1H3','NR1H4','NR1I2','NR1I3','NR2C1','NR2C2','NR2E1','NR2E3','NR2F1','NR2F2','NR2F6','NR3C1','NR3C2','NR4A1','NR4A2','NR4A3','NR5A1','NR5A2','NR6A1','NRBF2','NRF1','NRG1','NRIP1','NRIP3','NRK','NRL','NSD1','NSD2','NTRK1','NUCKS1','NUFIP1','NUP98','NUPL2']
TFs2Other=['OGG1','OLIG1','OLIG2','OLIG3','ONECUT1','ONECUT2','ONECUT3','OPRD1','OSR1','OSR2','OTOP3','OTP','OTULIN','OTX1','OTX2','OVOL1','OVOL2','OVOL3','PA2G4','PABPN1','PARK7','PARP1','PARP10','PARP12','PATZ1','PAWR','PAX1','PAX2','PAX3','PAX4','PAX5','PAX6','PAX7','PAX8','PAX9','PAXBP1','PAXIP1','PBRM1','PBX1','PBX2','PBX3','PBX4','PCBD1','PCBP1','PCBP2','PCBP3','PCGF2','PCGF6','PCNA','PCSK6','PDCD7','PDS5B','PDX1','PEG3','PELI1','PER1','PER2','PER3','PEX14','PFDN1','PGBD1','PGR','PGS1','PHB','PHB2','PHF1','PHF19 ','PHF20','PHF21A','PHF5A','PHOX2A','PHOX2B','PHTF1','PHTF2','PIAS1','PIAS2','PIAS3','PIAS4','PIDD1','PIKFYVE','PIM1','PIN1','PINK1','PITX1','PITX2','PITX3','PKHD1','PKHD1L1','PKNOX1','PKNOX2','PLA2G10','PLA2G1B','PLAG1','PLAGL1','PLAGL2','PLEK','PLEK2','PLEKHA4','PLPP3','PLSCR1','PLXNA1','PLXNA2','PLXNA3','PLXNA4','PLXNB1','PLXNB2','PLXNB3','PLXNC1','PLXND1','PML','PMS1','PNKP','POGK','POGZ','POLA1','POLB','POLE3','POLE4','POLG','POLH','POLI','POLK','POLL','POLM','POLN','POLQ','POLR1B','POLR2A','POLR2B','POLR2E','POLR2J','POLR2K','POLR2L','POLR3A','POLRMT','POT1','POU1F1','POU2AF1','POU2F1','POU2F2','POU2F3','POU3F1','POU3F2','POU3F3','POU3F4','POU4F1','POU4F2','POU4F3','POU5F1','POU5F1B','POU5F2','POU6F1','POU6F2','PPARA','PPARD','PPARG','PPARGC1A','PPARGC1B','PPP1R10','PPP1R13L','PPP1R21','PPP2R3B','PPP2R5B','PPP3CA','PPRC1','PQBP1','PRAM1','PRAP1','PRB3','PRDM1','PRDM10','PRDM11','PRDM12','PRDM13','PRDM14','PRDM15','PRDM16','PRDM2','PRDM4','PRDM5','PRDM6','PRDM7','PRDM8','PRDM9','PRDX3','PREB','PREX1','PREX2','PRH2','PRIM2','PRIMPOL','PRKCB','PRKCH','PRKCI','PRKCQ','PRKCZ','PRKD1','PRKD2','PRKN','PRMT2','PRMT3','PRNP','PROP1','PROX1','PROX2','PRR12','PRR3','PRRX1','PRRX2','PSMA6','PSMC5','PSMD10','PSMD11','PSMD12','PSPC1','PTCH1','PTEN','PTF1A','PTGIS','PTH','PTTG1','PUM3','PURA','PURB','PURG','PYCARD','PYDC1','PYDC2','QRICH1','RAB11FIP4','RAB7B','RAD21','RAD51','RAG1','RAI1','RAPGEF3','RAPGEF4','RAPGEF5','RARA','RARB','RARG','RAX','RAX2','RB1','RBAK','RBBP7','RBCK1','RBL1','RBL2','RBM10','RBM20','RBM22','RBM26','RBM27','RBM5','RBM6','RBPJ','RBPJL','RBSN','RC3H1','RC3H2','RCAN1','RCOR1','RCOR2','RCOR3','RECQL','REL','RELA','RELB','RELL2','RELN','REPIN1','RERE','REST','REV1','REXO4','RFC1','RFX1','RFX2','RFX3','RFX4','RFX5','RFX6','RFX7','RFX8','RFXANK','RFXAP','RGCC','RGS11','RGS6','RGS7','RGS9','RHEBL1','RHOXF1','RHOXF2','RHOXF2B','RING1','RIOK2','RIPK1','RIPK2','RIPK3','RIPK4','RLF','RLIM','RNASE2','RNASEH1','RNF113A','RNF113B','RNF114','RNF125','RNF138','RNF14','RNF166','RNF2','RNF219','RNF25','RNF31','RNF4','RNF41','RORA','RORB','RORC','RPA2','RPA4','RPL36A','RPL7','RPL7A','RPS27A','RPS6KA4','RPS6KA5','RREB1','RRN3','RUNX1','RUNX1T1','RUNX2','RUNX3','RWDD3','RXRA','RXRB','RXRG','RYBP','S100A12','S100A8','S100A9','SAFB','SAFB2','SALL1','SALL2','SALL3','SALL4','SART1','SATB1','SATB2','SAV1','SCAND1','SCAND2P','SCAPER','SCMH1','SCML1','SCML2','SCML4','SCOC','SCRT1','SCRT2','SCX','SEBOX','SEMA4A','SERTAD1','SERTAD2','SETBP1','SETD6','SETDB1','SETDB2','SF3A2','SF3A3','SFPQ','SFRP4','SFRP5','SFTPB','SGK1','SGK3','SGSM2','SHH','SHOX','SHOX2','SHPRH','SIGIRR','SIK1','SIM1','SIM2','SIN3A','SIN3B','SIRT1','SIRT2','SIRT3','SIRT6','SIVA1','SIX1','SIX2','SIX3','SIX4','SIX5','SIX6','SKI','SKIDA1','SKIL','SKOR1','SKOR2','SLC22A4','SLC26A10','SLC26A3','SLC2A4RG','SLC30A9','SLC39A10','SLC4A10','SMAD1','SMAD2','SMAD3','SMAD4','SMAD5','SMAD6','SMAD7','SMAD9','SMARCA1','SMARCA2','SMARCA4','SMARCA5','SMARCAL1','SMARCB1','SMARCC1','SMARCC2','SMARCD1','SMARCD2','SMARCD3','SMARCE1','SMC3','SMO','SMUG1','SMYD1','SMYD3','SNAI1','SNAI2','SNAI3','SNAPC2','SNAPC4','SNAPC5','SNRPC','SNURF','SNW1','SOGA3','SOHLH1','SOHLH2','SON','SORBS2','SOX1','SOX10','SOX11','SOX12','SOX13','SOX14','SOX15','SOX17','SOX18','SOX2','SOX21','SOX3','SOX30','SOX4','SOX5','SOX6','SOX7','SOX8','SOX9','SP1','SP100','SP110','SP140','SP140L','SP2','SP3','SP4','SP5','SP6','SP7','SP8','SP9','SPDEF','SPEN','SPHK1','SPI1','SPIB','SPIC','SPZ1','SRCAP','SREBF1','SREBF2','SRF','SRY','SSB','SSBP2','SSBP3','SSBP4','SSH1','SSH2','SSH3','SSRP1','SSX1','ST18','STAG1','STAT1','STAT2','STAT3','STAT4','STAT5A','STAT5B','STAT6','STK16','STK3','STK36','STRN','STRN3','SUB1','SUFU','SUMO1','SUPT16H','SUPT4H1','SUPT5H','SUPT6H','SUV39H1','SUV39H2','SUZ12','SYK','T','TAB1','TAB2','TAB3','TADA2A','TADA2B','TADA3','TAF1','TAF10','TAF11','TAF12','TAF13','TAF1A','TAF1B','TAF1C','TAF1L','TAF2','TAF3','TAF4','TAF4B','TAF5','TAF5L','TAF6','TAF6L','TAF7','TAF7L','TAF8','TAF9','TAF9B','TAL1','TAL2','TARDBP','TAX1BP1','TBL1Y','TBP','TBPL1','TBPL2','TBR1','TBX1','TBX10','TBX15','TBX18','TBX19','TBX2','TBX20','TBX21','TBX22','TBX3','TBX4','TBX5','TBX6','TCEA2','TCEAL1','TCEAL7','TCEAL8','TCF12','TCF15','TCF19','TCF20','TCF21','TCF23','TCF24','TCF25','TCF3','TCF4','TCF7','TCF7L1','TCF7L2','TCFL5','TDG','TDP1','TDP2','TEAD1','TEAD2','TEAD3','TEAD4','TEF','TERB1','TERF1','TERF2','TERF2IP','TESMIN','TET1','TET2','TET3','TFAM','TFAP2A','TFAP2B','TFAP2C','TFAP2D','TFAP2E','TFAP4','TFCP2','TFCP2L1','TFDP1','TFDP2','TFDP3','TFE3','TFEB','TFEC','TGFB1','TGFB1I1','TGIF1','TGIF2','TGIF2LX','TGIF2LY','TGS1','THAP1','THAP10','THAP11','THAP12','THAP2','THAP3','THAP4','THAP5','THAP6','THAP7','THAP8','THAP9','THRA','THRAP3','THRB','THYN1','TIA1','TICAM1','TIGD1','TIGD2','TIGD3','TIGD4','TIGD5','TIGD6','TIGD7','TIPARP','TIRAP','TLE1','TLE2','TLE3','TLE4','TLE6','TLR2','TLR3','TLR4','TLR9','TLX1','TLX2','TLX3','TMEM235','TMF1','TNF','TNFAIP3','TNFRSF11A','TNFRSF4','TNFSF11','TNFSF18','TNFSF4','TOE1','TONSL','TOP1','TOP2A','TOP2B','TOPORS','TOX','TOX2','TOX3','TOX4','TP53','TP53BP1','TP63','TP73','TPRX1','TRADD','TRAF1','TRAF2','TRAF3','TRAF4','TRAF5','TRAF6','TRAFD1','TRAK2','TRAPPC9','TRERF1','TREX1','TRIB1','TRIB3','TRIM13','TRIM14','TRIM15','TRIM21','TRIM22','TRIM23','TRIM24','TRIM25','TRIM26','TRIM27','TRIM28','TRIM29','TRIM3','TRIM31','TRIM32','TRIM33','TRIM34','TRIM37','TRIM38','TRIM40','TRIM5','TRIM52','TRIM62','TRIM8','TRIP11','TRIT1','TRMT1','TRMT1L','TRPS1','TRRAP','TSC2','TSC22D1','TSC22D2','TSC22D3','TSC22D4','TSHZ1','TSHZ2','TSHZ3','TSN','TSSK4','TTF1','TUB','TULP4','TUT1','TWIST1','TWIST2','TXK','U2AF1','U2AF1L4','UBA52','UBB','UBC','UBE2K','UBE2N','UBE2V1','UBN1','UBP1','UBR4','UBTF','UBTFL1','UBTFL6','UFL1','UHRF1','UHRF2','UNCX','UNG','UNK','UNKL','USF1','USF2','USF3','USP39','USP7','UTF1','VAV1','VAX1','VAX2','VDR','VEGFA','VENTX','VEZF1','VGLL1','VGLL2','VGLL3','VHLL','VPS72','VSX1','VSX2','WDHD1','WFS1','WIZ','WNT1','WNT10B','WNT2','WNT3A','WNT5A','WNT8B','WRN','WT1','WWOX','WWP2','WWTR1','XBP1','XCL1','XPA','YBX1','YBX2','YBX3','YEATS2','YEATS4','YOD1','YY1','YY2','ZBED1','ZBED2','ZBED3','ZBED4','ZBED5','ZBED6','ZBED6CL','ZBED8','ZBED9','ZBP1','ZBTB1','ZBTB10','ZBTB11','ZBTB12','ZBTB14','ZBTB16','ZBTB17','ZBTB18','ZBTB2','ZBTB20','ZBTB21','ZBTB22','ZBTB24','ZBTB25','ZBTB26','ZBTB3','ZBTB32','ZBTB33','ZBTB34','ZBTB37','ZBTB38','ZBTB39','ZBTB4','ZBTB40','ZBTB41','ZBTB42','ZBTB43','ZBTB44','ZBTB45','ZBTB46','ZBTB47','ZBTB48','ZBTB49','ZBTB5','ZBTB6','ZBTB7A','ZBTB7B','ZBTB7C','ZBTB8A','ZBTB8B','ZBTB9','ZC3H10','ZC3H11A','ZC3H12A','ZC3H13','ZC3H14','ZC3H15','ZC3H18','ZC3H3','ZC3H4','ZC3H6','ZC3H7A','ZC3H7B','ZC3H8','ZC3HAV1','ZCCHC11','ZCCHC6','ZDHHC11','ZDHHC19','ZEB1','ZEB2','ZFAND5','ZFAT','ZFHX2','ZFHX3','ZFHX4','ZFP1','ZFP14','ZFP2','ZFP28','ZFP3','ZFP30','ZFP36','ZFP36L1','ZFP36L2','ZFP37','ZFP41','ZFP42','ZFP57','ZFP62','ZFP64','ZFP69','ZFP69B','ZFP82','ZFP90','ZFP91','ZFP92','ZFPL1','ZFPM1','ZFPM2','ZFR','ZFR2','ZFX','ZFY','ZFYVE26','ZGLP1','ZGPAT','ZHX1','ZHX2','ZHX3','ZIC1','ZIC2','ZIC3','ZIC4','ZIC5','ZIK1','ZIM2','ZIM3','ZKSCAN1','ZKSCAN2','ZKSCAN3','ZKSCAN4','ZKSCAN5','ZKSCAN7','ZKSCAN8','ZMAT1','ZMAT2','ZMAT3','ZMAT4','ZMAT5','ZMYM1','ZMYM2','ZMYM3','ZMYM4','ZMYM5','ZMYM6','ZMYND10','ZNF10','ZNF100','ZNF101','ZNF106','ZNF107','ZNF112','ZNF114','ZNF117','ZNF12','ZNF121','ZNF124','ZNF131','ZNF132','ZNF133','ZNF134','ZNF135','ZNF136','ZNF137P','ZNF138','ZNF14','ZNF140','ZNF141','ZNF142','ZNF143','ZNF146','ZNF148','ZNF154','ZNF155','ZNF157','ZNF16','ZNF160','ZNF165','ZNF169','ZNF17','ZNF174','ZNF175','ZNF177','ZNF18','ZNF180','ZNF181','ZNF182','ZNF184','ZNF185','ZNF189','ZNF19','ZNF195','ZNF197','ZNF2','ZNF20','ZNF200','ZNF202','ZNF205','ZNF207','ZNF208','ZNF211','ZNF212','ZNF213','ZNF214','ZNF215','ZNF217','ZNF219','ZNF22','ZNF221','ZNF222','ZNF223','ZNF224','ZNF225','ZNF226','ZNF227','ZNF229','ZNF23','ZNF230','ZNF232','ZNF233','ZNF234','ZNF235','ZNF236','ZNF239','ZNF24','ZNF248','ZNF25','ZNF250','ZNF251','ZNF253','ZNF254','ZNF256','ZNF257','ZNF26','ZNF260','ZNF263','ZNF264','ZNF266','ZNF267','ZNF268','ZNF273','ZNF274','ZNF275','ZNF276','ZNF277','ZNF28','ZNF280A','ZNF280B','ZNF280C','ZNF280D','ZNF281','ZNF282','ZNF283','ZNF284','ZNF285','ZNF286A','ZNF286B','ZNF287','ZNF292','ZNF296','ZNF3','ZNF30','ZNF300','ZNF302','ZNF304','ZNF311','ZNF316','ZNF317','ZNF318','ZNF319','ZNF32','ZNF320','ZNF322','ZNF324','ZNF324B','ZNF326','ZNF329','ZNF330','ZNF331','ZNF333','ZNF334','ZNF335','ZNF337','ZNF33A','ZNF33B','ZNF34','ZNF341','ZNF343','ZNF345','ZNF346','ZNF347','ZNF35','ZNF350','ZNF354A','ZNF354B','ZNF354C','ZNF355P','ZNF358','ZNF362','ZNF365','ZNF366','ZNF367','ZNF37A','ZNF382','ZNF383','ZNF384','ZNF385A','ZNF385B','ZNF385C','ZNF385D','ZNF391','ZNF394','ZNF395','ZNF396','ZNF397','ZNF398','ZNF404','ZNF407','ZNF408','ZNF41','ZNF410','ZNF414','ZNF415','ZNF416','ZNF417','ZNF418','ZNF419','ZNF420','ZNF423','ZNF425','ZNF426','ZNF428','ZNF429','ZNF43','ZNF430','ZNF431','ZNF432','ZNF433','ZNF436','ZNF438','ZNF439','ZNF44','ZNF440','ZNF441','ZNF442','ZNF443','ZNF444','ZNF445','ZNF446','ZNF449','ZNF45','ZNF451','ZNF454','ZNF460','ZNF461','ZNF462','ZNF467','ZNF468','ZNF469','ZNF470','ZNF471','ZNF473','ZNF474','ZNF479','ZNF48','ZNF480','ZNF483','ZNF484','ZNF485','ZNF486','ZNF487','ZNF488','ZNF490','ZNF491','ZNF492','ZNF493','ZNF496','ZNF497','ZNF500','ZNF501','ZNF502','ZNF503','ZNF506','ZNF507','ZNF510','ZNF511','ZNF512','ZNF512B','ZNF513','ZNF514','ZNF516','ZNF517','ZNF518A','ZNF518B','ZNF519']
TFs3Other=['ZNF521','ZNF524','ZNF525','ZNF526','ZNF527','ZNF528','ZNF529','ZNF530','ZNF532','ZNF534','ZNF536','ZNF540','ZNF541','ZNF542P','ZNF543','ZNF544','ZNF546','ZNF547','ZNF548','ZNF549','ZNF550','ZNF551','ZNF552','ZNF554','ZNF555','ZNF556','ZNF557','ZNF558','ZNF559','ZNF560','ZNF561','ZNF562','ZNF563','ZNF564','ZNF565','ZNF566','ZNF567','ZNF568','ZNF569','ZNF57','ZNF570','ZNF571','ZNF572','ZNF573','ZNF574','ZNF575','ZNF576','ZNF577','ZNF578','ZNF579','ZNF580','ZNF581','ZNF582','ZNF583','ZNF584','ZNF585A','ZNF585B','ZNF586','ZNF587','ZNF587B','ZNF589','ZNF592','ZNF593','ZNF594','ZNF595','ZNF596','ZNF597','ZNF598','ZNF599','ZNF600','ZNF605','ZNF606','ZNF607','ZNF608','ZNF609','ZNF610','ZNF611','ZNF613','ZNF614','ZNF615','ZNF616','ZNF618','ZNF619','ZNF620','ZNF621','ZNF622','ZNF623','ZNF624','ZNF625','ZNF626','ZNF627','ZNF628','ZNF629','ZNF630','ZNF638','ZNF639','ZNF641','ZNF644','ZNF645','ZNF646','ZNF648','ZNF649','ZNF652','ZNF653','ZNF654','ZNF655','ZNF658','ZNF66','ZNF660','ZNF662','ZNF664','ZNF665','ZNF667','ZNF668','ZNF669','ZNF670','ZNF671','ZNF672','ZNF674','ZNF675','ZNF676','ZNF677','ZNF678','ZNF679','ZNF680','ZNF681','ZNF682','ZNF683','ZNF684','ZNF687','ZNF688','ZNF689','ZNF69','ZNF691','ZNF692','ZNF695','ZNF696','ZNF697','ZNF699','ZNF7','ZNF70','ZNF700','ZNF701','ZNF703','ZNF704','ZNF705A','ZNF705B','ZNF705D','ZNF705E','ZNF705G','ZNF706','ZNF707','ZNF708','ZNF709','ZNF71','ZNF710','ZNF711','ZNF713','ZNF714','ZNF716','ZNF717','ZNF718','ZNF720','ZNF721','ZNF724','ZNF726','ZNF727','ZNF728','ZNF729','ZNF73','ZNF730','ZNF732','ZNF735','ZNF736','ZNF737','ZNF738','ZNF74','ZNF740','ZNF746','ZNF747','ZNF749','ZNF750','ZNF75A','ZNF75D','ZNF76','ZNF761','ZNF763','ZNF764','ZNF765','ZNF766','ZNF768','ZNF77','ZNF770','ZNF771','ZNF772','ZNF773','ZNF774','ZNF775','ZNF776','ZNF777','ZNF778','ZNF780A','ZNF780B','ZNF781','ZNF782','ZNF783','ZNF784','ZNF785','ZNF786','ZNF787','ZNF788','ZNF789','ZNF79','ZNF790','ZNF791','ZNF792','ZNF793','ZNF799','ZNF8','ZNF80','ZNF800','ZNF804A','ZNF804B','ZNF805','ZNF808','ZNF81','ZNF812P','ZNF813','ZNF814','ZNF816','ZNF821','ZNF823','ZNF827','ZNF829','ZNF83','ZNF830','ZNF831','ZNF835','ZNF836','ZNF837','ZNF839','ZNF84','ZNF841','ZNF843','ZNF844','ZNF845','ZNF846','ZNF85','ZNF850','ZNF852','ZNF853','ZNF860','ZNF862','ZNF865','ZNF878','ZNF879','ZNF880','ZNF883','ZNF888','ZNF891','ZNF90','ZNF91','ZNF92','ZNF93','ZNF98','ZNF99','ZNFX1','ZRANB2','ZRSR1','ZRSR2','ZSCAN1','ZSCAN10','ZSCAN12','ZSCAN16','ZSCAN18','ZSCAN2','ZSCAN20','ZSCAN21','ZSCAN22','ZSCAN23','ZSCAN25','ZSCAN26','ZSCAN29','ZSCAN30','ZSCAN31','ZSCAN32','ZSCAN4','ZSCAN5A','ZSCAN5B','ZSCAN5C','ZSCAN5DP','ZSCAN9','ZUFSP','ZXDA','ZXDB','ZXDC','ZZZ3']
TFs4Other=['ATF2','ATF6B','ATN1','BHLHA15','BHLHB2','BHLHB4 ','BHLHB5','BHLHB9','C13ORF8','C20ORF194','CDX2','CRAMP1L','ELP3','ERVK3-1','ETF1','FLJ12895','FLJ20321','FLJ25476','FLJ35784','FOXM1','FOXO2','FOXR1','FOXR2','GATA1','GRLF1','KDM4C','KDM4E','KIAA0863','KIAA0961','KLRG1','LBXCOR1','LEF1','LHX7','LOC126295','LOC51058','LOC91661','MGC4728','MYOD1','MYOG','MYRF','NEUROD4','NEUROG1','NFAT2','NFAT4','NFE2L1','NR1A2','NR1B1','NR1B2','NR1B3','NR1C1','NR1C2','NR1C3','NR1F3','NR1I1','NR2A1','NR2B3','NR3A1','NR3A2','NR3B3','OTEX','PEPP-2','PHF2','POU2F1','POU2F2','POU4F3','POU5F1','POU6F1','PRDM3','PRKRIP1','PRKRIR','SETD1A','SMAD8','SMYD2','SOX12','SOX14','SOX15','TBX14','TCF1','TCF17','TCF2','TCF9','TEAD1','TPRXL','UBTFL1','WHSC1','ZBTB14','ZBTB18','ZBTB19','ZBTB21','ZBTB27','ZBTB28','ZBTB35','ZBTB49','ZFP161','ZFP28','ZKSCAN10','ZKSCAN11','ZKSCAN12','ZKSCAN13','ZKSCAN14','ZKSCAN16','ZKSCAN19','ZKSCAN21','ZKSCAN24','ZKSCAN6','ZKSCAN8','ZNF10','ZNF112','ZNF187','ZNF193','ZNF204P','ZNF238','ZNF271','ZNF295','ZNF305','ZNF323','ZNF434','ZNF499','ZNF542','ZNF545','ZNF553','ZNF56','ZNF625','ZNF643','ZNF698','ZNF705F','ZNF751','ZNF752','ZNF753','ZNF785','ZNF806','ZNF816','ZNF833P','ZNF906','ZSCAN1','ZSCAN25','ZSCAN26','ZSCAN31','ZSCAN32','ZSCAN5D','ZSCAN9']
TFsOther=TFs1Other+TFs2Other+TFs3Other+TFs4Other
```

Example 8 (unknown):
```unknown
S genes: ['MCM5', 'PCNA', 'TYMS', 'FEN1', 'MCM2', 'MCM4', 'RRM1', 'UNG', 'GINS2', 'MCM6', 'CDCA7', 'DTL', 'PRIM1', 'UHRF1', 'HELLS', 'RFC2', 'RPA2', 'NASP', 'RAD51AP1', 'GMNN', 'WDR76', 'SLBP', 'CCNE2', 'UBR7', 'POLD3', 'MSH2', 'ATAD2', 'RAD51', 'RRM2', 'CDC45', 'CDC6', 'EXO1', 'TIPIN', 'DSCC1', 'BLM', 'CASP8AP2', 'USP1', 'CLSPN', 'POLA1', 'CHAF1B', 'BRIP1', 'E2F8'] 

G2 genes: ['HMGB2', 'CDK1', 'NUSAP1', 'UBE2C', 'BIRC5', 'TPX2', 'TOP2A', 'NDC80', 'CKS2', 'NUF2', 'CKS1B', 'MKI67', 'TMPO', 'CENPF', 'TACC3', 'SMC4', 'CCNB2', 'CKAP2L', 'CKAP2', 'AURKB', 'BUB1', 'KIF11', 'ANP32E', 'TUBB4B', 'GTSE1', 'KIF20B', 'HJURP', 'CDCA3', 'CDC20', 'TTK', 'CDC25C', 'KIF2C', 'RANGAP1', 'NCAPD2', 'DLGAP5', 'CDCA2', 'CDCA8', 'ECT2', 'KIF23', 'HMMR', 'AURKA', 'PSRC1', 'ANLN', 'LBR', 'CKAP5', 'CENPE', 'CTCF', 'NEK2', 'G2E3', 'GAS2L3', 'CBX5', 'CENPA'] 

Hemoglobin genes set 1: ['HBB', 'HBD', 'HBG1', 'HBG2', 'HBE1', 'HBZ', 'HBM', 'HBA2', 'HBA1', 'HBQ1'] 

Hemoglobin genes set 2: ['AHSP', 'GYPA', 'RHAG', 'GFI1B', 'KEL', 'EPB42', 'ALAS2', 'TSPO2', 'GYPB', 'GYPE', 'SPTA1', 'RHCE', 'KLF1', 'RHD', 'SLC4A1', 'SNX22', 'SQSTM1', 'SOD1', 'FTH1', 'MAP1LC3B', 'SLC25A37', 'TRIM58', 'HMBS'] 

Mitochondrial genes: ['MT-ND1', 'MT-ND2', 'MT-CO1', 'MT-CO2', 'MT-ATP8', 'MT-ATP6', 'MT-CO3', 'MT-ND3', 'MT-ND4L', 'MT-ND4', 'MT-ND5', 'MT-ND6', 'MT-CYB'] 

Ribo genes: ['RPLP0', 'RPLP1', 'RPLP2', 'RPL3', 'RPL3L', 'RPL4', 'RPL5', 'RPL6', 'RPL7', 'RPL7A', 'RPL7L1', 'RPL8', 'RPL9', 'RPL10', 'RPL10A', 'RPL10L', 'RPL11', 'RPL12', 'RPL13', 'RPL13A', 'RPL14', 'RPL15', 'RPL17', 'RPL18', 'RPL18A', 'RPL19', 'RPL21', 'RPL22', 'RPL22L1', 'RPL23', 'RPL23A', 'RPL24', 'RPL26', 'RPL26L1', 'RPL27', 'RPL27A', 'RPL28', 'RPL29', 'RPL30', 'RPL31', 'RPL32', 'RPL34', 'RPL35', 'RPL35A', 'RPL36', 'RPL36A', 'RPL36AL', 'RPL37', 'RPL37A', 'RPL38', 'RPL39', 'RPL39L', 'UBA52', 'RPL41', 'RPSA', 'RPS2', 'RPS3', 'RPS3A', 'RPS4X', 'RPS4Y1', 'RPS4Y2', 'RPS5', 'RPS6', 'RPS7', 'RPS8', 'RPS9', 'RPS10', 'RPS11', 'RPS12', 'RPS13', 'RPS14', 'RPS15', 'RPS15A', 'RPS16', 'RPS17', 'RPS18', 'RPS19', 'RPS20', 'RPS21', 'RPS23', 'RPS24', 'RPS25', 'RPS26', 'RPS27', 'RPS27A', 'RPS27L', 'RPS28', 'RPS29', 'FAU'] 

Tissue dissociation stress associated genes: ['FOS', 'FOSB', 'JUN', 'JUNB', 'JUND', 'ZFP36', 'SOCS3', 'CEBPD', 'CEBPB', 'ATF3', 'EGR1', 'HSPA1A', 'HSPA1B', 'HSP90AA1', 'HSP90AB1', 'HSPA8', 'HSPB1', 'HSPE1', 'IER3', 'IER2', 'BTG1', 'BTG2', 'DUSP1']
```

Example 9 (python):
```python
#Expression: "YlGn" (e.g. DE plots)
#Divergent expression: "PRGn" (e.g. log2FC)
#Age: "afmhot_r"
#Others: "viridis" (e.g. UMAP expression)

#Define color palette
FineGrainPalette={
    'AtrialCardiomyocytesLeft':'#eac075',
    'AtrialCardiomyocytesRight':'#4c6b8a',
    'AtrialCardiomyocytesCycling':'#e94bc7',
    'VentricularCardiomyocytesLeftCompact':'#ab1c3c',
    'VentricularCardiomyocytesRightCompact':'#fdba45',
    'VentricularCardiomyocytesLeftTrabeculated':'#2b9200',
    'VentricularCardiomyocytesRightTrabeculated':'#0273a8',
    'VentricularCardiomyocytesCycling':'#e1acff',
    'SinoatrialNodePacemakerCells':'#854834',
    'AtrioventricularNodePacemakerCells':'#688500',
    'VentricularConductionSystemProximal':'#01ccc8',
    'VentricularConductionSystemDistal':'#b58fab',
    'GreatVesselAdventitialFibroblasts':'#ff7e97',
    'CoronaryVesselAdventitialFibroblasts':'#ff9b6e',
    'MyocardialInterstitialFibroblasts':'#8a6800',
    'SubEpicardialFibroblasts':'#206601',
    'Myofibroblasts':'#1bdebf',
    'LymphNodeFibroblasticReticularCells':'#005aa0',
    'ValveInterstitialCells':'#e478ff',
    'GreatVesselSmoothMuscleCells':'#ff665f',
    'CoronarySmoothMuscleCells':'#d2cb18',
    'DuctusArteriosusSmoothMuscleCells':'#83d7b3',
    'CoronaryPericytes':'#9088ff',
    'PericardialCellsIntermediate':'#c95e00',
    'PericardialCellsFibrous':'#01e077',
    'PericardialCellsParietal':'#ff88e4',
    'GreatVesselArterialEndothelialCells':'#d4004b',
    'GreatVesselVenousEndothelialCells':'#8f4300',
    'CoronaryArterialEndothelialCells':'#89da5a',
    'CoronaryVenousEndothelialCells':'#006f86',
    'CoronaryCapillaryEndothelialCells':'#aa7dff',
    'EndocardialCells':'#ff8241',
    'EndocardialCushionCells':'#46603a',
    'ValveEndothelialCells':'#414eb6',
    'LymphaticEndothelialCells':'#8b3f60',
    'MesothelialEpicardialCells':'#0295fe',
    'EpicardiumDerivedCells':'#ff92c4',
    'NeuronPrecursors':'#d30038',
    'ChromaffinCells':'#726800',
    'SympatheticNeurons':'#01894b',
    'ParaympatheticNeurons':'#545191',
    'SchwannCellPrecursors':'#b0d276',
    'SchwannCells':'#b30079',
    'MonocytesMPOpos':'#c50066',
    'Monocytes':'#fab4b6',
    'MonocyteDerivedCells':'#d14008',
    'MacrophagesCX3CR1pos':'#eabe97',
    'MacrophagesTIMD4pos':'#b2a200',
    'MacrophagesLYVE1pos':'#4a6005',
    'MacrophagesATF3pos':'#3db833',
    'DendriticCellsType1':'#007859',
    'DendriticCellsMature':'#01a9db',
    'PlasmacytoidDendriticCells':'#2c54ab',
    'MastCells':'#d0bef2',
    'Megakaryocytes':'#94298f',
    'TCellsCD4pos':'#ff387c',
    'TCellsCD8pos':'#9e3429',
    'TregsCD4pos':'#b37700',
    'ProBCells':'#8dab00',
    'BCells':'#a8d1a1',
    'BCellsMS4A1pos':'#00d2f0',
    'NaturalKillerCells':'#abb4ff',
    'InnateLymphoidCells':'#72497f',
}
sc.pl.umap(adata, color='fine_grain',palette=FineGrainPalette)

MidGrainPalette={
    'AtrialCardiomyocytes':'#af3a4d',
    'VentricularCardiomyocytes':'#b1756c',
    'CardiacConductionSystem':'#d45633',
    'Fibroblasts':'#9b6e4e',
    'MuralCells':'#ffb92f',
    'PericardialCells':'#b4f03b',
    'BloodVesselEndothelialCells':'#72cc57',
    'EndocardialCells':'#436e43',
    'LymphaticEndothelialCells':'#85cca0',
    'EpicardialCells':'#5eabb0',
    'Neurons':'#7bbdff',
    'Glia':'#403c90',
    'MyeloidCells':'#420066',
    'LymphoidCells':'#f553f6',
}
sc.pl.umap(adata, color='mid_grain',palette=MidGrainPalette)

CoarseGrainPalette={
    'Cardiomyocytes':'#bc5950',
    'Mesenchymal':'#af9c4c',
    'Endothelium':'#599e58',
    'Epicardium':'#5eabb0',
    'Neural':'#5e75b1',
    'Leukocytes':'#9a52a0',
}
sc.pl.umap(adata, color='coarse_grain',palette=CoarseGrainPalette)

RegionGrainPalette={
    'aorta':'#ffa2bc',
    'apex':'#fe0049',
    'atria':'#732d00',
    'atrial septum':'#ffd54c',
    'base':'#837100',
    'ductus arteriosus':'#bdff8b',
    'whole sample':'#1b6c00',
    'heart without node':'#02e6c6',
    'left ventricle':'#01c1f5',
    'node':'#00466d',
    'outflow tract':'#020011',
    'pericardium':'#7a75ff',
    'pulmonary arches and branches':'#30003e',
    'right ventricle':'#ff88e1',
    'right ventricle and papillaries':'#98004f',
}
sc.pl.umap(adata, color='region',palette=RegionGrainPalette)
```

Example 10 (python):
```python
set(adata.obs.fine_grain)
```

Example 11 (unknown):
```unknown
{'AtrialCardiomyocytesCycling',
 'AtrialCardiomyocytesLeft',
 'AtrialCardiomyocytesRight',
 'AtrioventricularNodePacemakerCells',
 'BCells',
 'BCellsMS4A1pos',
 'ChromaffinCells',
 'CoronaryArterialEndothelialCells',
 'CoronaryCapillaryEndothelialCells',
 'CoronaryPericytes',
 'CoronarySmoothMuscleCells',
 'CoronaryVenousEndothelialCells',
 'CoronaryVesselAdventitialFibroblasts',
 'DendriticCellsMature',
 'DendriticCellsType1',
 'DuctusArteriosusSmoothMuscleCells',
 'EndocardialCells',
 'EndocardialCushionCells',
 'EpicardiumDerivedCells',
 'GreatVesselAdventitialFibroblasts',
 'GreatVesselArterialEndothelialCells',
 'GreatVesselSmoothMuscleCells',
 'GreatVesselVenousEndothelialCells',
 'InnateLymphoidCells',
 'LymphNodeFibroblasticReticularCells',
 'LymphaticEndothelialCells',
 'MacrophagesATF3pos',
 'MacrophagesCX3CR1pos',
 'MacrophagesLYVE1pos',
 'MacrophagesTIMD4pos',
 'MastCells',
 'Megakaryocytes',
 'MesothelialEpicardialCells',
 'MonocyteDerivedCells',
 'Monocytes',
 'MonocytesMPOpos',
 'MyocardialInterstitialFibroblasts',
 'Myofibroblasts',
 'NaturalKillerCells',
 'NeuronPrecursors',
 'ParaympatheticNeurons',
 'PericardialCellsFibrous',
 'PericardialCellsIntermediate',
 'PericardialCellsParietal',
 'PlasmacytoidDendriticCells',
 'ProBCells',
 'SchwannCellPrecursors',
 'SchwannCells',
 'SinoatrialNodePacemakerCells',
 'SubEpicardialFibroblasts',
 'SympatheticNeurons',
 'TCellsCD4pos',
 'TCellsCD8pos',
 'TregsCD4pos',
 'ValveEndothelialCells',
 'ValveInterstitialCells',
 'VentricularCardiomyocytesCycling',
 'VentricularCardiomyocytesLeftCompact',
 'VentricularCardiomyocytesLeftTrabeculated',
 'VentricularCardiomyocytesRightCompact',
 'VentricularCardiomyocytesRightTrabeculated',
 'VentricularConductionSystemDistal',
 'VentricularConductionSystemProximal'}
```

Example 12 (python):
```python
set(adata.obs.mid_grain)
```

Example 13 (unknown):
```unknown
{'AtrialCardiomyocytes',
 'BloodVesselEndothelialCells',
 'CardiacConductionSystem',
 'EndocardialCells',
 'EpicardialCells',
 'Fibroblasts',
 'Glia',
 'LymphaticEndothelialCells',
 'LymphoidCells',
 'MuralCells',
 'MyeloidCells',
 'Neurons',
 'PericardialCells',
 'VentricularCardiomyocytes'}
```

Example 14 (python):
```python
set(adata.obs.coarse_grain)
```

Example 15 (unknown):
```unknown
{'Cardiomyocytes',
 'Endothelium',
 'Epicardium',
 'Leukocytes',
 'Mesenchymal',
 'Neural'}
```

Example 16 (python):
```python
#adata = adata[adata.obs['fine_grain'].isin(['CoronaryVesselAdventitialFibroblasts','MyocardialInterstitialFibroblasts1','MyocardialInterstitialFibroblasts2','Myofibroblasts','GreatVesselAdventitialFibroblasts'])].copy()
#adata = adata[adata.obs['fine_grain'].isin(['CoronaryArterialEndothelialCells','CoronaryCapillaryEndothelialCells','CoronaryVenousEndothelialCells','GreatVesselArterialEndothelialCells','GreatVesselVenousEndothelialCells'])].copy()
adata = adata[adata.obs['fine_grain'].isin(['Myofibroblasts','CoronaryVesselAdventitialFibroblasts','SubEpicardialFibroblasts','MyocardialInterstitialFibroblasts'])].copy()
#adata = adata[adata.obs['mid_grain'].isin(['PericardialCells'])].copy()
#adata=adata[adata.obs.mid_grain=='MuralCells',:].copy()
```

Example 17 (unknown):
```unknown
AnnData object with n_obs × n_vars = 32645 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score'
    var: 'gene_name_scRNA-0', 'gene_id'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'age_colors', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'cycling_colors', 'donor_colors', 'fine_grain_colors', 'heart_or_greatvessels_colors', 'hvg', 'kit_10x_colors', 'mid_grain_colors', 'modality_colors', 'neighbors', 'phase_colors', 'region_colors', 'sex_colors', 'trimester_colors', 'umap', 'week_colors'
    obsm: 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 18 (python):
```python
set(adata.obs.fine_grain)
```

Example 19 (unknown):
```unknown
{'CoronaryVesselAdventitialFibroblasts',
 'MyocardialInterstitialFibroblasts',
 'Myofibroblasts',
 'SubEpicardialFibroblasts'}
```

Example 20 (python):
```python
# preserve raw count
adata.layers["counts"] = adata.X.copy()

# normalise and log-transform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key='batch_key')
#sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5)

print(sum(adata.var.highly_variable))
sc.pl.highly_variable_genes(adata)

# freeze the state in `.raw`
adata.raw = adata

# subset hvgs
adata = adata[:, adata.var.highly_variable]

adata=adata.copy()

sc.tl.score_genes_cell_cycle(adata, s_genes=s_genes, g2m_genes=g2m_genes)
#sc.pp.regress_out(adata, ['S_score','G2M_score'])

#batch correction
scvi.model.SCVI.setup_anndata(adata, 
                              layer="counts", 
                              batch_key="batch_key",
                              categorical_covariate_keys=['phase'],
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo'])

vae = scvi.model.SCVI(adata, n_hidden = 128, n_latent = 50, n_layers = 3, dispersion = 'gene-batch')
vae.train(max_epochs=50)
    
# Convergence plot
x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
plt.plot(x, vae.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
plt.clf()

# add the latent space to adata
adata.obsm["X_scVI"] = vae.get_latent_representation()

sc.tl.score_genes(adata, stress_genes, score_name='stress_score')
sc.tl.score_genes(adata, hb_genes1, score_name='hb1_score')

sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata,min_dist=0.1)
#sc.tl.tsne(adata)
sc.pl.umap(adata, color=['log1p_total_counts','n_genes','n_genes_by_counts','total_counts','pct_counts_mt','pct_counts_ribo'],ncols=5)
sc.pl.umap(adata, color=['scrublet_score','stress_score', 'hb1_score','phase'], ncols=5,)

sc.pl.umap(adata, color=['donor','age','week','trimester','region','phase','cycling','cell_or_nuclei','modality','kit_10x','FACSgate','heart_or_greatvessels','coarse_grain','mid_grain','fine_grain'],wspace=0.8,ncols=5)
Genes=['TTN','TNNT2','MYOM1','LDB3','ALDH1A2','TNNT1','KRT19','UPK3B','BNC1','COL6A3','FBLN1','PCOLCE','LRRC17','PCDH18','EMCN','ECSCR','PECAM1','CDH5','ESAM', 'LAPTM5','PTPRC','LCP1','CD53','WAS','PHOX2B','NRXN1','STMN2','GRIK2','L1CAM']
sc.pl.umap(adata, color=Genes)
```

Example 21 (python):
```python
normalizing counts per cell
    finished (0:00:00)
extracting highly variable genes
    finished (0:00:10)
--> added
    'highly_variable', boolean vector (adata.var)
    'means', float vector (adata.var)
    'dispersions', float vector (adata.var)
    'dispersions_norm', float vector (adata.var)
1790
```

Example 22 (python):
```python
calculating cell cycle phase
computing score 'S_score'
WARNING: genes are not in var_names and ignored: ['FEN1', 'MCM2', 'MCM4', 'UNG', 'GINS2', 'MCM6', 'CDCA7', 'PRIM1', 'RFC2', 'RPA2', 'NASP', 'GMNN', 'WDR76', 'SLBP', 'CCNE2', 'UBR7', 'POLD3', 'MSH2', 'RAD51', 'CDC45', 'CDC6', 'EXO1', 'TIPIN', 'DSCC1', 'BLM', 'CASP8AP2', 'USP1', 'POLA1', 'CHAF1B', 'E2F8']
    finished: added
    'S_score', score of gene set (adata.obs).
    370 total control genes are used. (0:00:00)
computing score 'G2M_score'
WARNING: genes are not in var_names and ignored: ['ANP32E', 'RANGAP1', 'LBR', 'CTCF', 'G2E3', 'CBX5']
    finished: added
    'G2M_score', score of gene set (adata.obs).
    735 total control genes are used. (0:00:00)
-->     'phase', cell cycle phase (adata.obs)
```

Example 23 (unknown):
```unknown
GPU available: False, used: False
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
```

Example 24 (unknown):
```unknown
Epoch 50/50: 100%|███████████| 50/50 [02:21<00:00,  2.83s/it, loss=766, v_num=1]
```

Example 25 (python):
```python
computing score 'stress_score'
WARNING: genes are not in var_names and ignored: ['JUND', 'CEBPB', 'HSP90AA1', 'HSP90AB1', 'HSPA8', 'HSPB1', 'HSPE1', 'BTG1']
    finished: added
    'stress_score', score of gene set (adata.obs).
    489 total control genes are used. (0:00:00)
computing score 'hb1_score'
WARNING: genes are not in var_names and ignored: ['HBD', 'HBE1', 'HBZ', 'HBM', 'HBQ1']
    finished: added
    'hb1_score', score of gene set (adata.obs).
    198 total control genes are used. (0:00:00)
computing neighbors
    finished: added to `.uns['neighbors']`
    `.obsp['distances']`, distances for each pair of neighbors
    `.obsp['connectivities']`, weighted adjacency matrix (0:00:04)
computing UMAP
    finished: added
    'X_umap', UMAP coordinates (adata.obsm) (0:00:09)
```

Example 26 (unknown):
```unknown
... storing 'phase' as categorical
```

Example 27 (unknown):
```unknown
<Figure size 320x320 with 0 Axes>
```

Example 28 (python):
```python
sc.pl.umap(adata, color='region')
```

Example 29 (python):
```python
sc.pl.umap(adata, color='fine_grain')
```

Example 30 (python):
```python
sc.tl.leiden(adata, resolution=0.2, key_added='leiden')
sc.pl.umap(adata, color='leiden',legend_loc='on data')
```

Example 31 (python):
```python
running Leiden clustering
    finished: found 4 clusters and added
    'leiden', the cluster labels (adata.obs, categorical) (0:00:04)
```

Example 32 (python):
```python
set(adata.obs.fine_grain)
```

Example 33 (unknown):
```unknown
{'CoronaryVesselAdventitialFibroblasts',
 'MyocardialInterstitialFibroblasts',
 'Myofibroblasts',
 'SubEpicardialFibroblasts'}
```

Example 34 (python):
```python
FineGrainPalette2={
    'CoronaryVesselAdventitialFibroblasts':'#ff9b6e',
    'MyocardialInterstitialFibroblasts':'#8a6800',
    'SubEpicardialFibroblasts':'#206601',
    'Myofibroblasts':'#1bdebf',
    'cycling':'#005aa0',
}

adata.obs['fine_grain2']= 'na'
adata.obs['fine_grain2'][[x for x in adata.obs_names if adata.obs.fine_grain[x]=='CoronaryVesselAdventitialFibroblasts']]='CoronaryVesselAdventitialFibroblasts'
adata.obs['fine_grain2'][[x for x in adata.obs_names if adata.obs.fine_grain[x]=='MyocardialInterstitialFibroblasts']]='MyocardialInterstitialFibroblasts'
adata.obs['fine_grain2'][[x for x in adata.obs_names if adata.obs.fine_grain[x]=='SubEpicardialFibroblasts']]='SubEpicardialFibroblasts'
adata.obs['fine_grain2'][[x for x in adata.obs_names if adata.obs.fine_grain[x]=='Myofibroblasts']]='Myofibroblasts'
adata.obs['fine_grain2'][[x for x in adata.obs_names if adata.obs.leiden[x] in ['3']]]='cycling'
sc.pl.umap(adata, color='fine_grain2',palette=FineGrainPalette2)
```

Example 35 (unknown):
```unknown
... storing 'fine_grain2' as categorical
```

Example 36 (python):
```python
sc.pl.umap(adata, color='region',groups=['left ventricle','right ventricle','right ventricle and papillaries'])
sc.pl.umap(adata, color='region',groups=['base','atrial septum','atria'])
sc.pl.umap(adata, color='region',groups=['node','atria'])
```

Example 37 (python):
```python
ClusterOfInterest='Myofibroblasts'
#sc.tl.rank_genes_groups(adata, 'fine_grain', method='t-test', key_added = "t-test")
DEGenesForCluster = [x[ClusterOfInterest] for x in adata.uns['t-test']['names'][:50]]
sc.pl.dotplot(adata, DEGenesForCluster ,groupby='fine_grain',standard_scale='var',cmap='PRGn')
#sc.pl.umap(adata, color=DEGenesForCluster)
```

Example 38 (python):
```python
FineGrainCategoriesOrder=['CoronaryVesselAdventitialFibroblasts','MyocardialInterstitialFibroblasts1','MyocardialInterstitialFibroblasts2','Myofibroblasts']
Genes=['DCN','PDGFRA','TCF21','MOXD1','FNDC1','FBLN5','DES','CEMIP','SCN7A','COL15A1','ROBO2','ROBO2','CD34','COL25A1','MYRIP','NRG1','SLIT3','MIR100HG','FLRT2','BNC2','CASC15','TSHZ2','PRRX1','SFRP1','BRINP3','EPHA3','IGFBP7','SEMA5A','APOE','ITGA1','HES4','NOTCH3','PLA2G5','APOE','MCAM']
sc.pl.dotplot(adata , var_group_positions=[(3,7),(8,15),(16,25),(26,34)], var_group_labels=['coronary adventitial','myocardial fibroblasts 1','myocardial fibroblasts 2','myofibroblasts'], var_group_rotation=90,groupby='fine_grain',categories_order=FineGrainCategoriesOrder,standard_scale='var',cmap='YlGn')
```

Example 39 (python):
```python
Genes=['DCN','PDGFRA','TCF21','MOXD1','FNDC1','FBLN5','DES','CEMIP','SLIT3','MIR100HG','FLRT2','BNC2','CASC15','TSHZ2','PRRX1','SFRP1','BRINP3','EPHA3','SCN7A','COL15A1','ROBO2','CD34','COL25A1','MYRIP','NRG1','IGFBP7','SEMA5A','APOE','ITGA1','HES4','NOTCH3','PLA2G5','MCAM']
print(set([x for x in Genes if Genes.count(x) > 1]))
FineGrainCategoriesOrder=['CoronaryVesselAdventitialFibroblasts','SubEpicardialFibroblasts','MyocardialInterstitialFibroblasts','Myofibroblasts']
sc.pl.dotplot(adata, Genes ,var_group_positions=[(0,7),(8,17),(18,24),(25,33)], var_group_labels=['coronary adventitial','subepicardial','myocardial interstitial','myofibroblasts'], var_group_rotation=0,groupby='fine_grain',categories_order=FineGrainCategoriesOrder,standard_scale='var',cmap='YlGn')
```

Example 40 (python):
```python
bdata=adata[adata.obs.fine_grain2=='Myofibroblasts',:].copy()
sc.tl.umap(bdata,min_dist=0.1)
```

Example 41 (python):
```python
computing UMAP
    finished: added
    'X_umap', UMAP coordinates (adata.obsm) (0:00:02)
```

Example 42 (python):
```python
sc.pl.umap(bdata, color=['MYH11','PLA2G5','ACTA2','PLA2G5','PDGFRA','PDGFRB','HES4','TAGLN'])
```

Example 43 (python):
```python
adata=sc.read('./Objects/Feb28ObjectRaw.h5ad')
print(adata.shape)

adata = adata[adata.obs['fine_grain'].isin(['Myofibroblasts','CoronaryVesselAdventitialFibroblasts','MyocardialInterstitialFibroblasts1','MyocardialInterstitialFibroblasts2','GreatVesselAdventitialFibroblasts'])].copy()


# preserve raw count
adata.layers["counts"] = adata.X.copy()

# normalise and log-transform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key='batch_key')
#sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5)

print(sum(adata.var.highly_variable))
sc.pl.highly_variable_genes(adata)

# freeze the state in `.raw`
adata.raw = adata

# subset hvgs
adata = adata[:, adata.var.highly_variable]

adata=adata.copy()

sc.tl.score_genes_cell_cycle(adata, s_genes=s_genes, g2m_genes=g2m_genes)
#sc.pp.regress_out(adata, ['S_score','G2M_score'])

#batch correction
scvi.model.SCVI.setup_anndata(adata, 
                              layer="counts", 
                              batch_key="batch_key",
                              categorical_covariate_keys=['phase'],
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo'])

vae = scvi.model.SCVI(adata, n_hidden = 128, n_latent = 50, n_layers = 3, dispersion = 'gene-batch')
vae.train(max_epochs=50)
    
# Convergence plot
x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
plt.plot(x, vae.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
plt.clf()

# add the latent space to adata
adata.obsm["X_scVI"] = vae.get_latent_representation()

sc.tl.score_genes(adata, stress_genes, score_name='stress_score')
sc.tl.score_genes(adata, hb_genes1, score_name='hb1_score')

sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata,min_dist=0.1)
#sc.tl.tsne(adata)
sc.pl.umap(adata, color=['log1p_total_counts','n_genes','n_genes_by_counts','total_counts','pct_counts_mt','pct_counts_ribo'],ncols=5)
sc.pl.umap(adata, color=['scrublet_score','stress_score', 'hb1_score','phase'], ncols=5,)

sc.pl.umap(adata, color=['donor','age','week','trimester','region','phase','cycling','cell_or_nuclei','modality','kit_10x','FACSgate','heart_or_greatvessels','coarse_grain','mid_grain','fine_grain'],wspace=0.8,ncols=5)
Genes=['TTN','TNNT2','MYOM1','LDB3','ALDH1A2','TNNT1','KRT19','UPK3B','BNC1','COL6A3','FBLN1','PCOLCE','LRRC17','PCDH18','EMCN','ECSCR','PECAM1','CDH5','ESAM', 'LAPTM5','PTPRC','LCP1','CD53','WAS','PHOX2B','NRXN1','STMN2','GRIK2','L1CAM']
sc.pl.umap(adata, color=Genes)
```

Example 44 (python):
```python
(297473, 36601)
normalizing counts per cell
    finished (0:00:00)
extracting highly variable genes
    finished (0:00:14)
--> added
    'highly_variable', boolean vector (adata.var)
    'means', float vector (adata.var)
    'dispersions', float vector (adata.var)
    'dispersions_norm', float vector (adata.var)
2154
```

Example 45 (python):
```python
calculating cell cycle phase
computing score 'S_score'
WARNING: genes are not in var_names and ignored: ['FEN1', 'MCM2', 'MCM4', 'RRM1', 'UNG', 'GINS2', 'MCM6', 'CDCA7', 'PRIM1', 'RFC2', 'RPA2', 'NASP', 'GMNN', 'SLBP', 'CCNE2', 'UBR7', 'POLD3', 'MSH2', 'RAD51', 'CDC45', 'CDC6', 'EXO1', 'TIPIN', 'DSCC1', 'BLM', 'CASP8AP2', 'USP1', 'POLA1', 'CHAF1B', 'E2F8']
    finished: added
    'S_score', score of gene set (adata.obs).
    373 total control genes are used. (0:00:01)
computing score 'G2M_score'
WARNING: genes are not in var_names and ignored: ['CKS1B', 'ANP32E', 'RANGAP1', 'LBR', 'CTCF', 'G2E3', 'CBX5']
    finished: added
    'G2M_score', score of gene set (adata.obs).
    650 total control genes are used. (0:00:00)
-->     'phase', cell cycle phase (adata.obs)
```

Example 46 (unknown):
```unknown
GPU available: False, used: False
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
```

Example 47 (unknown):
```unknown
Epoch 50/50: 100%|███████████| 50/50 [08:03<00:00,  9.67s/it, loss=883, v_num=1]
```

Example 48 (python):
```python
computing score 'stress_score'
WARNING: genes are not in var_names and ignored: ['JUND', 'CEBPB', 'HSP90AA1', 'HSP90AB1', 'HSPA8', 'HSPB1', 'HSPE1', 'BTG1']
    finished: added
    'stress_score', score of gene set (adata.obs).
    290 total control genes are used. (0:00:00)
computing score 'hb1_score'
WARNING: genes are not in var_names and ignored: ['HBD', 'HBZ', 'HBM', 'HBQ1']
    finished: added
    'hb1_score', score of gene set (adata.obs).
    248 total control genes are used. (0:00:00)
computing neighbors
    finished: added to `.uns['neighbors']`
    `.obsp['distances']`, distances for each pair of neighbors
    `.obsp['connectivities']`, weighted adjacency matrix (0:00:07)
computing UMAP
    finished: added
    'X_umap', UMAP coordinates (adata.obsm) (0:00:28)
```

Example 49 (unknown):
```unknown
... storing 'phase' as categorical
```

Example 50 (unknown):
```unknown
<Figure size 320x320 with 0 Axes>
```

Example 51 (python):
```python
adata.obs['adventitial']= 'na'
adata.obs['adventitial'][[x for x in adata.obs_names if adata.obs.fine_grain[x] in ['Myofibroblasts','MyocardialInterstitialFibroblasts1','MyocardialInterstitialFibroblasts2']]]='no'
adata.obs['adventitial'][[x for x in adata.obs_names if adata.obs.fine_grain[x] in ['CoronaryVesselAdventitialFibroblasts','GreatVesselAdventitialFibroblasts']]]='yes'
sc.pl.umap(adata, color='adventitial')
```

Example 52 (unknown):
```unknown
... storing 'adventitial' as categorical
```

Example 53 (python):
```python
sc.tl.rank_genes_groups(adata, 'adventitial', method='t-test', key_added = "t-test")

ClusterOfInterest='yes'
DEGenesForCluster = [x[ClusterOfInterest] for x in adata.uns['t-test']['names'][:50]]
sc.pl.dotplot(adata, DEGenesForCluster ,groupby='fine_grain',standard_scale='var',cmap='PRGn')

ClusterOfInterest='no'
DEGenesForCluster = [x[ClusterOfInterest] for x in adata.uns['t-test']['names'][:50]]
sc.pl.dotplot(adata, DEGenesForCluster ,groupby='fine_grain',standard_scale='var',cmap='PRGn')
```

Example 54 (python):
```python
ranking genes
    finished: added to `.uns['t-test']`
    'names', sorted np.recarray to be indexed by group ids
    'scores', sorted np.recarray to be indexed by group ids
    'logfoldchanges', sorted np.recarray to be indexed by group ids
    'pvals', sorted np.recarray to be indexed by group ids
    'pvals_adj', sorted np.recarray to be indexed by group ids (0:00:03)
```

Example 55 (python):
```python
FineGrainCategoriesOrder=['GreatVesselAdventitialFibroblasts','CoronaryVesselAdventitialFibroblasts','MyocardialInterstitialFibroblasts1','MyocardialInterstitialFibroblasts2','Myofibroblasts']
sc.pl.dotplot(adata, ['SLIT3','SERPINF1','FBLN5','KAZN','GAS1','ZFHX4','CRABP2','SOD3'] ,categories_order=FineGrainCategoriesOrder,groupby='fine_grain',standard_scale='var',cmap='YlGn')
```

Example 56 (python):
```python
sc.pl.dotplot(adata, ['DCN','PDGFRA','TCF21','MOXD1','FNDC1','FBLN5','DES','CEMIP','SCN7A','COL15A1','ROBO2','ROBO2','CD34','COL25A1','MYRIP','NRG1','SLIT3','MIR100HG','FLRT2','BNC2','CASC15','TSHZ2','PRRX1','SFRP1','BRINP3','EPHA3','IGFBP7','SEMA5A','APOE','ITGA1','HES4','NOTCH3','PLA2G5','APOE','MCAM'] ,categories_order=FineGrainCategoriesOrder,groupby='fine_grain',standard_scale='var',cmap='YlGn')
```

---

## Import modules¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/RNA/scnRNA_Integration-Annotation/No1_global_prepare-for-scVI.html

**Contents:**
- Import modules¶
- Read in data¶
- Concatenate data¶
- Pre-process¶
- Set batch_key¶
- hvg identification¶
- Save¶

**Examples:**

Example 1 (python):
```python
import numpy as np
import numpy.random as random
import pandas as pd
import scanpy as sc
import seaborn as sns
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
-----
numpy               1.20.1
pandas              1.1.4
scanpy              1.7.1
seaborn             0.11.1
session_info        1.0.0
-----
```

Example 4 (python):
```python
PIL                 8.1.2
anndata             0.7.5
anyio               NA
attr                20.3.0
babel               2.9.0
backcall            0.2.0
brotli              NA
cairo               1.20.0
certifi             2020.12.05
cffi                1.14.5
chardet             4.0.0
cloudpickle         1.6.0
colorama            0.4.4
cycler              0.10.0
cython_runtime      NA
cytoolz             0.11.0
dask                2021.03.1
dateutil            2.8.1
decorator           4.4.2
fsspec              0.8.7
get_version         2.1
google              NA
h5py                2.10.0
idna                2.10
igraph              0.8.3
ipykernel           5.5.0
ipython_genutils    0.2.0
ipywidgets          7.6.3
jedi                0.18.0
jinja2              2.11.3
joblib              1.0.1
json5               NA
jsonschema          3.2.0
jupyter_server      1.4.1
jupyterlab_server   2.3.0
kiwisolver          1.3.1
legacy_api_wrap     0.0.0
leidenalg           0.8.3
llvmlite            0.34.0
louvain             0.7.0
markupsafe          1.1.1
matplotlib          3.3.4
mpl_toolkits        NA
natsort             7.1.1
nbclassic           NA
nbformat            5.1.2
numba               0.51.2
numexpr             2.7.3
packaging           20.9
parso               0.8.1
pexpect             4.8.0
pickleshare         0.7.5
pkg_resources       NA
prometheus_client   NA
prompt_toolkit      3.0.16
psutil              5.8.0
ptyprocess          0.7.0
pvectorc            NA
pyarrow             0.16.0
pygments            2.8.0
pyparsing           2.4.7
pyrsistent          NA
pytoml              NA
pytz                2021.1
requests            2.25.1
ruamel              NA
scipy               1.6.1
send2trash          NA
setuptools_scm      NA
simplejson          3.17.2
sinfo               0.3.1
six                 1.15.0
sklearn             0.22
sniffio             1.2.0
socks               1.7.1
sphinxcontrib       NA
statsmodels         0.12.2
storemagic          NA
tables              3.6.1
tblib               1.7.0
texttable           1.6.3
tlz                 0.11.0
toolz               0.11.1
tornado             6.1
traitlets           5.0.5
typing_extensions   NA
uritemplate         3.0.1
urllib3             1.26.3
wcwidth             0.2.5
yaml                5.1.2
zmq                 22.0.3
```

Example 5 (unknown):
```unknown
-----
IPython             7.21.0
jupyter_client      6.1.11
jupyter_core        4.7.1
jupyterlab          3.0.9
notebook            6.2.0
-----
Python 3.8.8 | packaged by conda-forge | (default, Feb 20 2021, 16:22:27) [GCC 9.3.0]
Linux-4.15.0-112-generic-x86_64-with-glibc2.10
-----
Session information updated at 2023-02-08 22:51
```

Example 6 (python):
```python
sc.settings.verbosity = 3             # verbosity: errors (0), warnings (1), info (2), hints (3)
sc.settings.set_figure_params(dpi=80, facecolor='white', color_map = 'RdPu',)
```

Example 7 (python):
```python
cell = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/QC/scRNA_foetal.h5ad')

# tidy up .var
cell.var=cell.var[['gene_name']]
cell.var.rename(columns={'gene_name':'gene_name_scRNA'},inplace=True)

# add some columns
cell.obs['donor_by_library-prep'] = cell.obs['donor'].copy()
cell.obs['multiplexed'] = 'No'

cell
```

Example 8 (unknown):
```unknown
AnnData object with n_obs × n_vars = 100463 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed'
    var: 'gene_name_scRNA'
    uns: 'sangerID_colors'
    obsm: 'latent_gene_encoding'
```

Example 9 (python):
```python
multi = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/QC/multiome_RNA_foetal.h5ad')

# tidy up .var
multi.var=multi.var[['gene_name']]
multi.var.rename(columns={'gene_name':'gene_name_multiome'},inplace=True)

multi
```

Example 10 (unknown):
```unknown
AnnData object with n_obs × n_vars = 272338 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet'
    var: 'gene_name_multiome'
    uns: 'sangerID_colors'
    obsm: 'latent_gene_encoding'
```

Example 11 (python):
```python
adata = cell.concatenate(multi, join='inner', batch_key=None, index_unique=None)
adata
```

Example 12 (unknown):
```unknown
AnnData object with n_obs × n_vars = 372801 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet'
    var: 'gene_name_scRNA-0', 'gene_name_multiome-1'
    obsm: 'latent_gene_encoding'
```

Example 13 (python):
```python
adata.obs['donor'].value_counts()
```

Example 14 (unknown):
```unknown
Hst41      129909
Hst33       62684
Hst39       30591
C99         17921
C97         17388
C92         15379
C94         13971
Hst40       11559
BRC2251      9651
BRC2256      8525
BRC2260      8408
Hst36        7848
BRC2252      7366
C86          6567
BRC2263      5917
BRC2262      4749
C87          4499
C104         3411
C98          3039
C85          2808
C82           611
Name: donor, dtype: int64
```

Example 15 (python):
```python
pd.crosstab(adata.obs['facility'],adata.obs['kit_10x'])
```

Example 16 (python):
```python
# save
adata.write('/nfs/team205/heart/anndata_objects/Foetal/RNA_foetal_raw.h5ad')
```

Example 17 (unknown):
```unknown
... storing 'sangerID' as categorical
... storing 'combinedID' as categorical
... storing 'donor' as categorical
... storing 'region' as categorical
... storing 'region_finest' as categorical
... storing 'age' as categorical
... storing 'gender' as categorical
... storing 'facility' as categorical
... storing 'cell_or_nuclei' as categorical
... storing 'modality' as categorical
... storing 'kit_10x' as categorical
... storing 'scrublet_leiden' as categorical
... storing 'donor_by_library-prep' as categorical
... storing 'multiplexed' as categorical
... storing 'SOC | status' as categorical
```

Example 18 (python):
```python
adata.X.data[:10]
```

Example 19 (unknown):
```unknown
array([ 1.,  1.,  5.,  5.,  8.,  4., 13.,  4.,  3.,  1.], dtype=float32)
```

Example 20 (python):
```python
# preserve raw count
adata.layers["counts"] = adata.X.copy()

# filter genes
sc.pp.filter_genes(adata, min_cells=3)

# normalise and log-transform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

adata.shape
```

Example 21 (unknown):
```unknown
filtered out 1675 genes that are detected in less than 3 cells
normalizing counts per cell
    finished (0:00:03)
```

Example 22 (unknown):
```unknown
(372801, 34926)
```

Example 23 (python):
```python
# combining Donor, Cell_or_Nuclei, and Kit_10X
adata.obs['batch_key']=adata.obs['donor'].astype('str')+'_'+adata.obs['cell_or_nuclei'].astype('str')+'_'+adata.obs['kit_10x'].astype('str')
adata.obs['batch_key'].value_counts()
```

Example 24 (unknown):
```unknown
Hst41_Nuclei_Multiome-v1    129909
Hst33_Nuclei_Multiome-v1     62684
Hst39_Nuclei_Multiome-v1     30591
C99_Cell_5prime-v2           17921
C97_Cell_5prime-v2           17388
C92_Nuclei_Multiome-v1       15379
C94_Cell_5prime-v2           13971
Hst40_Nuclei_Multiome-v1     11559
BRC2251_Cell_3prime-v3        9651
BRC2256_Cell_3prime-v3        8525
BRC2260_Cell_3prime-v3        8408
Hst36_Nuclei_Multiome-v1      7848
BRC2252_Cell_3prime-v3        7366
C86_Cell_5prime-v2            6567
BRC2263_Cell_3prime-v3        5917
BRC2262_Cell_3prime-v3        4749
C87_Nuclei_Multiome-v1        4499
C104_Nuclei_Multiome-v1       3411
C98_Nuclei_Multiome-v1        3039
C85_Nuclei_Multiome-v1        2808
C82_Nuclei_Multiome-v1         611
Name: batch_key, dtype: int64
```

Example 25 (python):
```python
sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key='batch_key')

print(sum(adata.var.highly_variable))
sc.pl.highly_variable_genes(adata)
```

Example 26 (python):
```python
extracting highly variable genes
... storing 'batch_key' as categorical
    finished (0:00:49)
--> added
    'highly_variable', boolean vector (adata.var)
    'means', float vector (adata.var)
    'dispersions', float vector (adata.var)
    'dispersions_norm', float vector (adata.var)
```

Example 27 (python):
```python
# freeze the state in `.raw`
adata.raw = adata
```

Example 28 (python):
```python
# subset hvgs
adata = adata[:, adata.var.highly_variable]
```

Example 29 (python):
```python
adata.write('/nfs/team205/heart/anndata_objects/Foetal/scVI/RNA_foetal_pp.h5ad') # batch_key: Donor, Cell_or_Nuclei, and Kit_10X
adata
```

Example 30 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 372801 × 3343
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key'
    var: 'gene_name_scRNA-0', 'gene_name_multiome-1', 'n_cells', 'highly_variable', 'means', 'dispersions', 'dispersions_norm', 'highly_variable_nbatches', 'highly_variable_intersection'
    uns: 'log1p', 'hvg'
    obsm: 'latent_gene_encoding'
    layers: 'counts'
```

---

## Functions¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/Visium/Visium_Intagration-Annotation/Visium_integration_no4_plots-each-annotation.html

**Contents:**
- Functions¶
- Read in anndata¶
- Modify annotation¶
- Plot on visium sections¶

**Examples:**

Example 1 (python):
```python
import sys
import scanpy as sc
import anndata
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt 
import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 
import seaborn as sns

import os
```

Example 2 (python):
```python
sc.settings.set_figure_params(dpi=80,dpi_save=300)
```

Example 3 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk'
```

Example 4 (python):
```python
# This function selects the data for multiple slides from the spatial anndata object.
# Modified `select_slide` in cell2loc for multiple slides

# @param adata: anndata object with multiple spatial experiments
# @param s: list of selected slides
# @param s_col: column in adata.obs listing experiment name for each location

def select_multislide(adata, s, s_col='sample'):

    slides = adata[adata.obs[s_col].isin(s), :]
    s_keys = list(slides.uns['spatial'].keys())
    
    uns_dict = {}
    for ss in s:
        s_spatial=np.array(s_keys)[[ss in k for k in s_keys]][0]
        uns_dict[s_spatial]=slides.uns['spatial'][s_spatial]
    
    slides.uns['spatial'] = uns_dict

    return slides
```

Example 5 (python):
```python
annot = pd.read_csv('/nfs/team205/heart/anndata_objects/Foetal/visium_foetal_healthy-all_spot-annotation.csv')
annot.set_index('spot_id',inplace=True)
set(annot['final'])
```

Example 6 (unknown):
```unknown
{'AV ring',
 'AV valves',
 'AVnode',
 'Adventitia - Coronary',
 'Adventitia - GV',
 'Aortic valve',
 'Atrium - Left',
 'Atrium - Right',
 'Coronary vessel',
 'Ductus arteriosus',
 'Endocardial cusion',
 'Endocardium - Atrial',
 'Endocardium - Ventricular',
 'Epicardium',
 'Great vessel',
 'LN',
 'SAnode',
 'VCS - Distal',
 'VCS - Proximal',
 'Valve apparatus',
 'Ventricle - Compact - Left',
 'Ventricle - Compact - Right',
 'Ventricle - Trabeculated - Left',
 'Ventricle - Trabeculated - Right'}
```

Example 7 (python):
```python
# the anndata has "gene expression" in .X
# cell2location output is in .obsm['q05_cell_abundance_w_sf']
# kNN graph for UMAP/clustering was generated using cell2location output
adata_vis = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/visium_foetal_healthy-all_raw.h5ad')

# modify annotations
annotation_edit_dict = {
   'AV valves':'Valves',
    'Aortic valve':'Valves',
    'Valve apparatus':'Valves',
    'Atrium - Left':'Atrium',
     'Atrium - Right':'Atrium',
    'Ventricle - Compact - Left':'Ventricle - Compact',
    'Ventricle - Compact - Right':'Ventricle - Compact',
    'Ventricle - Trabeculated - Left':'Ventricle - Trabeculated',
    'Ventricle - Trabeculated - Right':'Ventricle - Trabeculated',
}
adata_vis.obs['annotation_final_mod']=adata_vis.obs['annotation_final'].astype('str').copy()
adata_vis.obs.replace({'annotation_final_mod':annotation_edit_dict}, inplace=True)

# set palette and categorie order
AnnotationPalette={ 
    'Atrium':'#eac075',
        'Ventricle - Compact':'#f1acfa',
     'Ventricle - Trabeculated':'#f09951',
        'SAnode':'#8A2BE2',
        'AVnode':'#062b4a',
    'VCS - Proximal':'#FFFF00',
        'VCS - Distal':'#7FFFD4',
        'Epicardium':'#01ccc8',
        'Endocardium - Atrial':'#DC143C',
        'Endocardium - Ventricular':'#f53527',
        'Endocardial cusion':'#ff7e97',
        'Great vessel':'#ff9b6e',
    'Ductus arteriosus':'#8a6800',
        'Coronary vessel':'#206601',
        'Adventitia - Coronary':'#CAFF70',
     'Adventitia - GV':'#005aa0',
        'AV ring':'#e478ff',
    'Valves':'#3300FF',
    'LN':'#9088ff',
}
adata_vis.obs['annotation_final_mod'] = adata_vis.obs['annotation_final_mod'].astype('category')
adata_vis.obs['annotation_final_mod'] = adata_vis.obs['annotation_final_mod'].cat.reorder_categories(AnnotationPalette.keys())

adata_vis
```

Example 8 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/anndata/_core/anndata.py:1840: UserWarning: Variable names are not unique. To make them unique, call `.var_names_make_unique`.
  utils.warn_names_duplicates("var")
```

Example 9 (unknown):
```unknown
AnnData object with n_obs × n_vars = 39223 × 33562
    obs: 'array_row', 'array_col', 'sample', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'mt_frac', 'sangerID', 'Publication', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'status', 'cell2loc-map_group', 'JC_edit', 'sample_for_cell2loc', 'week', 'leiden', 'region_cluster', 'AtrialCardiomyocytesLeft', 'AtrialCardiomyocytesRight', 'VentricularCardiomyocytesLeftCompact', 'VentricularCardiomyocytesRightCompact', 'VentricularCardiomyocytesLeftTrabeculated', 'VentricularCardiomyocytesRightTrabeculated', 'SinoatrialNodeCardiomyocytes', 'AtrioventricularNodeCardiomyocytes', 'VentricularConductionSystemProximal', 'VentricularConductionSystemDistal', 'GreatVesselAdventitialFibroblasts', 'CoronaryVesselAdventitialFibroblasts', 'MyocardialInterstitialFibroblasts1', 'MyocardialInterstitialFibroblasts2', 'Myofibroblasts', 'LymphNodeFibroblasticReticularCells', 'ValveInterstitialCells', 'GreatVesselSmoothMuscleCells', 'CoronarySmoothMuscleCells', 'CoronaryPericytes', 'GreatVesselArterialEndothelialCells', 'GreatVesselVenousEndothelialCells', 'CoronaryArterialEndothelialCells', 'CoronaryVenousEndothelialCells', 'CoronaryCapillaryEndothelialCells', 'EndocardialCells', 'EndocardialCushionCells', 'ValveEndothelialCells', 'LymphaticEndothelialCells', 'MesothelialEpicardialCells', 'EpicardiumDerivedCells', 'NeuronPrecursors', 'ChromaffinCells', 'SympatheticNeurons', 'ParaympatheticNeurons', 'SchwannCellPrecursors', 'SchwannCells', 'MonocytesMPOpos', 'Monocytes', 'MonocyteDerivedCells', 'MacrophagesCX3CR1pos', 'MacrophagesTIMD4pos', 'MacrophagesLYVE1pos', 'MacrophagesATF3pos', 'DendriticCellsType1', 'DendriticCellsMature', 'PlasmacytoidDendriticCells', 'MastCells', 'Megakaryocytes', 'TCellsCD4pos', 'TCellsCD8pos', 'TregsCD4pos', 'ProBCells', 'BCells', 'BCellsMS4A1pos', 'NaturalKillerCells', 'InnateLymphoidCells', 'DuctusArteriosusSmoothMuscleCells', 'AtrialCardiomyocytesLeft_minmax', 'AtrialCardiomyocytesRight_minmax', 'VentricularCardiomyocytesLeftCompact_minmax', 'VentricularCardiomyocytesRightCompact_minmax', 'VentricularCardiomyocytesLeftTrabeculated_minmax', 'VentricularCardiomyocytesRightTrabeculated_minmax', 'SinoatrialNodeCardiomyocytes_minmax', 'AtrioventricularNodeCardiomyocytes_minmax', 'VentricularConductionSystemProximal_minmax', 'VentricularConductionSystemDistal_minmax', 'GreatVesselAdventitialFibroblasts_minmax', 'CoronaryVesselAdventitialFibroblasts_minmax', 'MyocardialInterstitialFibroblasts1_minmax', 'MyocardialInterstitialFibroblasts2_minmax', 'Myofibroblasts_minmax', 'LymphNodeFibroblasticReticularCells_minmax', 'ValveInterstitialCells_minmax', 'GreatVesselSmoothMuscleCells_minmax', 'CoronarySmoothMuscleCells_minmax', 'CoronaryPericytes_minmax', 'GreatVesselArterialEndothelialCells_minmax', 'GreatVesselVenousEndothelialCells_minmax', 'CoronaryArterialEndothelialCells_minmax', 'CoronaryVenousEndothelialCells_minmax', 'CoronaryCapillaryEndothelialCells_minmax', 'EndocardialCells_minmax', 'EndocardialCushionCells_minmax', 'ValveEndothelialCells_minmax', 'LymphaticEndothelialCells_minmax', 'MesothelialEpicardialCells_minmax', 'EpicardiumDerivedCells_minmax', 'NeuronPrecursors_minmax', 'ChromaffinCells_minmax', 'SympatheticNeurons_minmax', 'ParaympatheticNeurons_minmax', 'SchwannCellPrecursors_minmax', 'SchwannCells_minmax', 'MonocytesMPOpos_minmax', 'Monocytes_minmax', 'MonocyteDerivedCells_minmax', 'MacrophagesCX3CR1pos_minmax', 'MacrophagesTIMD4pos_minmax', 'MacrophagesLYVE1pos_minmax', 'MacrophagesATF3pos_minmax', 'DendriticCellsType1_minmax', 'DendriticCellsMature_minmax', 'PlasmacytoidDendriticCells_minmax', 'MastCells_minmax', 'Megakaryocytes_minmax', 'TCellsCD4pos_minmax', 'TCellsCD8pos_minmax', 'TregsCD4pos_minmax', 'ProBCells_minmax', 'BCells_minmax', 'BCellsMS4A1pos_minmax', 'NaturalKillerCells_minmax', 'InnateLymphoidCells_minmax', 'DuctusArteriosusSmoothMuscleCells_minmax', 'annotation', 'annotation_final', 'annotation_final_coarse', 'annotation_final_mod'
    var: 'gene_ids'
    uns: 'JC_edit_colors', 'annotation_colors', 'annotation_final_colors', 'cell2loc-map_group_colors', 'leiden', 'neighbors', 'region_cluster_colors', 'sample_for_cell2loc_colors', 'umap'
    obsm: 'MT', 'X_umap', 'means_cell_abundance_w_sf', 'q05_cell_abundance_w_sf', 'q05_cell_abundance_w_sf_ndarray', 'q95_cell_abundance_w_sf', 'spatial', 'stds_cell_abundance_w_sf'
    obsp: 'connectivities', 'distances'
```

Example 10 (python):
```python
with mpl.rc_context({'axes.facecolor':  'white',
                     'figure.figsize': [6, 6]}):
    sc.pl.umap(adata_vis, color=['annotation_final_mod'], size=5,
               color_map = 'RdPu', ncols = 3, wspace=0.3,
               legend_fontsize=14, frameon=False, palette=AnnotationPalette,
               show=False
              )
    plt.savefig(f"{figdir}/UMAP_Visium_annotations.pdf", bbox_inches="tight",dpi=300)
```

Example 11 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 12 (python):
```python
# need to read in each object to plot visium section...(as far as i know)
adata_ffpe = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/visium-FFPE_foetal_raw.h5ad')
adata_ffpe.obs['sample_for_cell2loc'] = adata_ffpe.obs['sample'].copy()
adata_ffpe.obs['cell2loc-map_group'] = 'FFPE_16PCW'
adata_ffpe = adata_ffpe[adata_ffpe.obs['JC_edit'].isin(['Cavity'])==False]

# add annotation
adata_ffpe.obs['annotation_final_mod'] = adata_vis.obs['annotation_final_mod'].reindex(adata_ffpe.obs_names)
```

Example 13 (python):
```python
/tmp/ipykernel_58671/3263657043.py:8: ImplicitModificationWarning: Trying to modify attribute `.obs` of view, initializing view as actual.
  adata_ffpe.obs['annotation_final_mod'] = adata_vis.obs['annotation_final_mod'].reindex(adata_ffpe.obs_names)
```

Example 14 (python):
```python
slide_id = 'BHF_F_Hea13295831'
slide = select_multislide(adata_ffpe, [slide_id])

with mpl.rc_context({'figure.figsize': [4.5, 4],
                    'axes.titlesize':14,}):

    sc.pl.spatial(slide,
                  color=['annotation_final_mod'], 
                  size=0,img_key='hires',alpha_img=1,title='',legend_loc=None,
                  show=False
                                   )
    plt.savefig(f"{figdir}/visium_HE_{slide_id}.pdf", bbox_inches="tight",dpi=300)
    
    sc.pl.spatial(slide,
                  color=['annotation_final_mod'], 
                  size=1.2,img_key='hires',alpha_img=0.3, palette=AnnotationPalette,
                  show=False
                 )
    plt.savefig(f"{figdir}/visium_annotation_{slide_id}.pdf", bbox_inches="tight",dpi=300)
```

Example 15 (unknown):
```unknown
/tmp/ipykernel_58671/2805736760.py:18: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  slides.uns['spatial'] = uns_dict
```

Example 16 (python):
```python
slide_id = 'BHF_F_Hea13295831'
slide = select_multislide(adata_ffpe, [slide_id])

with mpl.rc_context({'figure.figsize': [4.5, 4],
                    'axes.titlesize':14,}):

    sc.pl.spatial(slide,
                  color=['annotation_final_mod'], 
                  size=0,img_key='hires',alpha_img=1,title='',legend_loc=None,
                  crop_coord = [8000,15500,7000,14000],show=False
                 )
    plt.savefig(f"{figdir}/visium_cropped_HE_{slide_id}.pdf", bbox_inches="tight",dpi=300)
    
    sc.pl.spatial(slide,
                  color=['annotation_final_mod'], 
                  size=1.2,img_key='hires',alpha_img=0.3, palette=AnnotationPalette,
                  crop_coord = [8000,15500,7000,14000],show=False
                 )
    plt.savefig(f"{figdir}/visium_cropped_annotation_{slide_id}.pdf", bbox_inches="tight",dpi=300)
```

Example 17 (unknown):
```unknown
/tmp/ipykernel_58671/2805736760.py:18: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  slides.uns['spatial'] = uns_dict
```

---

## Read in result files¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig2_Overview_of_cellular_niches_and_TissueTypist/tissuetypist_crossvalidation_no2_summary.html

**Contents:**
- Read in result files¶
- Edge feature: distance to edge¶
  - Stats¶
  - Accuracy¶
  - F1 score¶

**Examples:**

Example 1 (python):
```python
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os

import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42
```

Example 2 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk/Aug2024'
table_dir = '/nfs/team205/heart/fetal_ms_figures/tables/Aug2024'
```

Example 3 (unknown):
```unknown
dir_to_results = '/nfs/team205/kk18/notebooks/P48_Foetal-heart_Visium/notebooks/TissueTypist/tissuetypist_package/cross_validation_results/7Apr2025_without-CoronaryAdv'
```

Example 4 (unknown):
```unknown
os.listdir(dir_to_results)
```

Example 5 (unknown):
```unknown
['is_edge_f1_score_class_per_fold.csv',
 'distance_to_edge_f1_score_class_per_fold.csv',
 'distance_to_edge_results_per_fold.csv',
 'is_edge_results_per_fold.csv']
```

Example 6 (python):
```python
edge_feature = 'distance_to_edge'

accuracy_df = pd.read_csv(f'{dir_to_results}/{edge_feature}_results_per_fold.csv',index_col=0)
display(accuracy_df.head())
f1score_df = pd.read_csv(f'{dir_to_results}/{edge_feature}_f1_score_class_per_fold.csv',index_col=0)
display(f1score_df.head())
```

Example 7 (unknown):
```unknown
f1score_df[(f1score_df['weight_to_neighbours']==1)&(f1score_df['weight_to_edge']==0)]
```

Example 8 (python):
```python
import pandas as pd
import numpy as np
from scipy.stats import kruskal, mannwhitneyu
import statsmodels.stats.multitest as smm

def two_factor_test(df,
                    factor_A_col,
                    factor_B_col,
                    score
                   ):
    ### 0. prepare levels for the two factors
    levels_A = list(set(df[factor_A_col]))
    levels_A.sort()
    levels_B = list(set(df[factor_B_col]))
    levels_B.sort()
    
    ### 1. Test the main effect of A using a Kruskal–Wallis test
    # First, group the data by factor A (pooling across B)
    grouped_A = [group[score].values for name, group in df.groupby(factor_A_col)]
    kw_A, p_kw_A = kruskal(*grouped_A)
    print(f"Kruskal-Wallis test for the main effect of {factor_A_col}:")
    print(f"Statistic: {kw_A:.4f}, p-value: {p_kw_A:.4f}")
    
    ### 2. Test the effect of B within each level of A
    for a in levels_A:
        print(f"\nTesting the effect of {factor_B_col} within {factor_A_col} = {a}")
        subset = df[df[factor_A_col] == a]

        # Use Kruskal-Wallis test for factor B within this subset.
        grouped_B = [group[score].values for name, group in subset.groupby(factor_B_col)]
        kw_B, p_kw_B = kruskal(*grouped_B)
        print(f"Kruskal-Wallis test for '{factor_B_col}' within '{factor_A_col}' = {a}: Statistic: {kw_B:.4f}, p-value: {p_kw_B:.4f}")

        # Perform pairwise Mann–Whitney U tests between levels of B
        # (only if you wish to run post-hoc pairwise comparisons)
        condition_keys = sorted(subset[factor_B_col].unique())
        comparisons = []
        p_values = []

        for i in range(len(condition_keys)):
            for j in range(i + 1, len(condition_keys)):
                group1 = subset[subset[factor_B_col] == condition_keys[i]][score]
                group2 = subset[subset[factor_B_col] == condition_keys[j]][score]
                stat, p = mannwhitneyu(group1, group2, alternative='two-sided')
                comparisons.append(f"{condition_keys[i]} vs {condition_keys[j]}")
                p_values.append(p)

        # Apply fdr_bh correction for multiple comparisons
        reject, pvals_corrected, _, _ = smm.multipletests(p_values, method='fdr_bh')
        print(f"Post-hoc pairwise comparisons for '{factor_B_col}' within '{factor_A_col}' = {str(a)}:")
        for comp, p_val, rej in zip(comparisons, pvals_corrected, reject):
            print(f"Comparison {comp}: Corrected p = {p_val:.4f}, Significant = {rej}")
            
    ### 3. return the outputs
    return kw_A, p_kw_A
```

Example 9 (unknown):
```unknown
# accuracy_df
two_factor_test(accuracy_df,
                    factor_A_col='weight_to_neighbours',
                    factor_B_col='weight_to_edge',
                    score='accuracy'
                   )
```

Example 10 (unknown):
```unknown
Kruskal-Wallis test for the main effect of weight_to_neighbours:
Statistic: 46.6325, p-value: 0.0000

Testing the effect of weight_to_edge within weight_to_neighbours = 0.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.0: Statistic: 1.4626, p-value: 0.4813
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.0:
Comparison 0 vs 1: Corrected p = 0.5296, Significant = False
Comparison 0 vs 5: Corrected p = 0.5296, Significant = False
Comparison 1 vs 5: Corrected p = 0.5296, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 0.1
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.1: Statistic: 1.2600, p-value: 0.5326
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.1:
Comparison 0 vs 1: Corrected p = 0.8413, Significant = False
Comparison 0 vs 5: Corrected p = 0.6310, Significant = False
Comparison 1 vs 5: Corrected p = 0.6310, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 0.3
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.3: Statistic: 1.8200, p-value: 0.4025
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.3:
Comparison 0 vs 1: Corrected p = 0.5476, Significant = False
Comparison 0 vs 5: Corrected p = 0.5476, Significant = False
Comparison 1 vs 5: Corrected p = 0.5476, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 1.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 1.0: Statistic: 2.9706, p-value: 0.2264
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 1.0:
Comparison 0 vs 1: Corrected p = 0.8335, Significant = False
Comparison 0 vs 5: Corrected p = 0.3333, Significant = False
Comparison 1 vs 5: Corrected p = 0.3333, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 3.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 3.0: Statistic: 2.5800, p-value: 0.2753
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 3.0:
Comparison 0 vs 1: Corrected p = 0.8413, Significant = False
Comparison 0 vs 5: Corrected p = 0.3333, Significant = False
Comparison 1 vs 5: Corrected p = 0.3333, Significant = False
```

Example 11 (python):
```python
(np.float64(46.63245124819787), np.float64(1.8188129472739346e-09))
```

Example 12 (unknown):
```unknown
# accuracy_df
two_factor_test(accuracy_df,
                    factor_A_col='weight_to_neighbours',
                    factor_B_col='weight_to_edge',
                    score='balanced_accuracy'
                   )
```

Example 13 (unknown):
```unknown
Kruskal-Wallis test for the main effect of weight_to_neighbours:
Statistic: 20.0716, p-value: 0.0005

Testing the effect of weight_to_edge within weight_to_neighbours = 0.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.0: Statistic: 1.1400, p-value: 0.5655
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.0:
Comparison 0 vs 1: Corrected p = 0.5476, Significant = False
Comparison 0 vs 5: Corrected p = 0.5476, Significant = False
Comparison 1 vs 5: Corrected p = 0.5476, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 0.1
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.1: Statistic: 0.6200, p-value: 0.7334
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.1:
Comparison 0 vs 1: Corrected p = 0.8413, Significant = False
Comparison 0 vs 5: Corrected p = 0.8413, Significant = False
Comparison 1 vs 5: Corrected p = 0.8413, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 0.3
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.3: Statistic: 0.7400, p-value: 0.6907
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.3:
Comparison 0 vs 1: Corrected p = 0.8413, Significant = False
Comparison 0 vs 5: Corrected p = 0.8413, Significant = False
Comparison 1 vs 5: Corrected p = 0.8413, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 1.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 1.0: Statistic: 1.4600, p-value: 0.4819
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 1.0:
Comparison 0 vs 1: Corrected p = 0.6905, Significant = False
Comparison 0 vs 5: Corrected p = 0.6905, Significant = False
Comparison 1 vs 5: Corrected p = 0.6905, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 3.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 3.0: Statistic: 0.3800, p-value: 0.8270
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 3.0:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False
```

Example 14 (python):
```python
(np.float64(20.07157894736841), np.float64(0.000483409762777016))
```

Example 15 (python):
```python
# f1 score
y_value_list = [x for x in f1score_df.columns if ('f1_class_' in x)&('Adventitia - Coronary' not in x)]

tissues_significant = []
for tissue in y_value_list:
    print(f'########## {tissue} ##########')
    df = f1score_df[[tissue,'weight_to_neighbours','weight_to_edge']]
    df.columns = ['f1_score','weight_to_neighbours','weight_to_edge']
    kw_A, p_kw_A = two_factor_test(df,
                    factor_A_col='weight_to_neighbours',
                    factor_B_col='weight_to_edge',
                    score='f1_score'
                   )
    if p_kw_A<0.05:
        tissues_significant.append(tissue)
    print(f'')
    print(f'')
```

Example 16 (unknown):
```unknown
########## f1_class_Atrium ##########
Kruskal-Wallis test for the main effect of weight_to_neighbours:
Statistic: 20.5078, p-value: 0.0004

Testing the effect of weight_to_edge within weight_to_neighbours = 0.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.0: Statistic: 0.0050, p-value: 0.9975
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.0:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 0.1
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.1: Statistic: 0.0201, p-value: 0.9900
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.1:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 0.3
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.3: Statistic: 0.0050, p-value: 0.9975
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.3:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 1.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 1.0: Statistic: 0.0351, p-value: 0.9826
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 1.0:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 3.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 3.0: Statistic: 0.1425, p-value: 0.9312
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 3.0:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False


########## f1_class_Ventricle - Compact ##########
Kruskal-Wallis test for the main effect of weight_to_neighbours:
Statistic: 42.4393, p-value: 0.0000

Testing the effect of weight_to_edge within weight_to_neighbours = 0.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.0: Statistic: 2.2200, p-value: 0.3296
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.0:
Comparison 0 vs 1: Corrected p = 0.5476, Significant = False
Comparison 0 vs 5: Corrected p = 0.5476, Significant = False
Comparison 1 vs 5: Corrected p = 0.5476, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 0.1
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.1: Statistic: 1.5200, p-value: 0.4677
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.1:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 0.6310, Significant = False
Comparison 1 vs 5: Corrected p = 0.6310, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 0.3
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.3: Statistic: 2.4800, p-value: 0.2894
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.3:
Comparison 0 vs 1: Corrected p = 0.6905, Significant = False
Comparison 0 vs 5: Corrected p = 0.4524, Significant = False
Comparison 1 vs 5: Corrected p = 0.6310, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 1.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 1.0: Statistic: 4.0923, p-value: 0.1292
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 1.0:
Comparison 0 vs 1: Corrected p = 0.6004, Significant = False
Comparison 0 vs 5: Corrected p = 0.2262, Significant = False
Comparison 1 vs 5: Corrected p = 0.2262, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 3.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 3.0: Statistic: 3.4400, p-value: 0.1791
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 3.0:
Comparison 0 vs 1: Corrected p = 0.4206, Significant = False
Comparison 0 vs 5: Corrected p = 0.3333, Significant = False
Comparison 1 vs 5: Corrected p = 0.3333, Significant = False


########## f1_class_Ventricle - Trabeculated ##########
Kruskal-Wallis test for the main effect of weight_to_neighbours:
Statistic: 32.1279, p-value: 0.0000

Testing the effect of weight_to_edge within weight_to_neighbours = 0.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.0: Statistic: 0.1400, p-value: 0.9324
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.0:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 0.1
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.1: Statistic: 0.3757, p-value: 0.8288
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.1:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 0.3
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.3: Statistic: 0.0150, p-value: 0.9925
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.3:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 1.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 1.0: Statistic: 0.5000, p-value: 0.7788
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 1.0:
Comparison 0 vs 1: Corrected p = 0.6905, Significant = False
Comparison 0 vs 5: Corrected p = 0.6905, Significant = False
Comparison 1 vs 5: Corrected p = 0.6905, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 3.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 3.0: Statistic: 0.4200, p-value: 0.8106
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 3.0:
Comparison 0 vs 1: Corrected p = 0.8413, Significant = False
Comparison 0 vs 5: Corrected p = 0.8413, Significant = False
Comparison 1 vs 5: Corrected p = 0.8413, Significant = False


########## f1_class_SAnode ##########
Kruskal-Wallis test for the main effect of weight_to_neighbours:
Statistic: 4.7938, p-value: 0.3091

Testing the effect of weight_to_edge within weight_to_neighbours = 0.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.0: Statistic: 0.0713, p-value: 0.9650
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.0:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 0.1
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.1: Statistic: 0.0771, p-value: 0.9622
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.1:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 0.3
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.3: Statistic: 0.0000, p-value: 1.0000
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.3:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 1.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 1.0: Statistic: 0.1465, p-value: 0.9294
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 1.0:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 3.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 3.0: Statistic: 0.0000, p-value: 1.0000
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 3.0:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False


########## f1_class_AVnode ##########
Kruskal-Wallis test for the main effect of weight_to_neighbours:
Statistic: 9.8756, p-value: 0.0426

Testing the effect of weight_to_edge within weight_to_neighbours = 0.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.0: Statistic: 0.0615, p-value: 0.9697
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.0:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 0.1
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.1: Statistic: 0.0358, p-value: 0.9822
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.1:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 0.3
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.3: Statistic: 0.0615, p-value: 0.9697
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.3:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 1.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 1.0: Statistic: 0.2201, p-value: 0.8958
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 1.0:
Comparison 0 vs 1: Corrected p = 0.9161, Significant = False
Comparison 0 vs 5: Corrected p = 0.9161, Significant = False
Comparison 1 vs 5: Corrected p = 0.9161, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 3.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 3.0: Statistic: 0.7538, p-value: 0.6860
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 3.0:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 0.7887, Significant = False
Comparison 1 vs 5: Corrected p = 0.7887, Significant = False


########## f1_class_VCS - Proximal ##########
Kruskal-Wallis test for the main effect of weight_to_neighbours:
Statistic: 7.7439, p-value: 0.1014

Testing the effect of weight_to_edge within weight_to_neighbours = 0.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.0: Statistic: 0.3757, p-value: 0.8288
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.0:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 0.1
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.1: Statistic: 0.1553, p-value: 0.9253
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.1:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 0.3
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.3: Statistic: 0.1400, p-value: 0.9324
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.3:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 1.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 1.0: Statistic: 0.0202, p-value: 0.9899
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 1.0:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 3.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 3.0: Statistic: 1.9086, p-value: 0.3851
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 3.0:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 0.4274, Significant = False
Comparison 1 vs 5: Corrected p = 0.7925, Significant = False


########## f1_class_VCS - Distal ##########
Kruskal-Wallis test for the main effect of weight_to_neighbours:
Statistic: 22.5262, p-value: 0.0002

Testing the effect of weight_to_edge within weight_to_neighbours = 0.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.0: Statistic: 0.0351, p-value: 0.9826
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.0:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 0.1
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.1: Statistic: 0.3657, p-value: 0.8329
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.1:
Comparison 0 vs 1: Corrected p = 0.8413, Significant = False
Comparison 0 vs 5: Corrected p = 0.8413, Significant = False
Comparison 1 vs 5: Corrected p = 0.8413, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 0.3
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.3: Statistic: 0.3670, p-value: 0.8324
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.3:
Comparison 0 vs 1: Corrected p = 0.8335, Significant = False
Comparison 0 vs 5: Corrected p = 0.8335, Significant = False
Comparison 1 vs 5: Corrected p = 0.8335, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 1.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 1.0: Statistic: 0.0200, p-value: 0.9900
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 1.0:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 3.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 3.0: Statistic: 0.0353, p-value: 0.9825
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 3.0:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False


########## f1_class_Great vessel ##########
Kruskal-Wallis test for the main effect of weight_to_neighbours:
Statistic: 36.7687, p-value: 0.0000

Testing the effect of weight_to_edge within weight_to_neighbours = 0.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.0: Statistic: 1.2200, p-value: 0.5434
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.0:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 0.6310, Significant = False
Comparison 1 vs 5: Corrected p = 0.6310, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 0.1
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.1: Statistic: 0.4558, p-value: 0.7962
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.1:
Comparison 0 vs 1: Corrected p = 0.9166, Significant = False
Comparison 0 vs 5: Corrected p = 0.9166, Significant = False
Comparison 1 vs 5: Corrected p = 0.9166, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 0.3
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.3: Statistic: 0.5159, p-value: 0.7726
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.3:
Comparison 0 vs 1: Corrected p = 0.8413, Significant = False
Comparison 0 vs 5: Corrected p = 0.8413, Significant = False
Comparison 1 vs 5: Corrected p = 0.8413, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 1.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 1.0: Statistic: 0.0651, p-value: 0.9680
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 1.0:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 3.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 3.0: Statistic: 0.1352, p-value: 0.9346
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 3.0:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False


########## f1_class_Coronary vessel ##########
Kruskal-Wallis test for the main effect of weight_to_neighbours:
Statistic: 17.0662, p-value: 0.0019

Testing the effect of weight_to_edge within weight_to_neighbours = 0.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.0: Statistic: 0.0000, p-value: 1.0000
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.0:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 0.1
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.1: Statistic: 0.0636, p-value: 0.9687
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.1:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 0.3
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.3: Statistic: 0.0000, p-value: 1.0000
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.3:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 1.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 1.0: Statistic: 0.0193, p-value: 0.9904
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 1.0:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 3.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 3.0: Statistic: 0.1615, p-value: 0.9224
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 3.0:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False


########## f1_class_Ductus arteriosus ##########
Kruskal-Wallis test for the main effect of weight_to_neighbours:
Statistic: 1.1268, p-value: 0.8900

Testing the effect of weight_to_edge within weight_to_neighbours = 0.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.0: Statistic: 0.1083, p-value: 0.9473
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.0:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 0.1
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.1: Statistic: 0.0153, p-value: 0.9924
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.1:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 0.3
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.3: Statistic: 0.0155, p-value: 0.9923
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.3:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 1.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 1.0: Statistic: 0.5538, p-value: 0.7581
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 1.0:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 0.8972, Significant = False
Comparison 1 vs 5: Corrected p = 0.8972, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 3.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 3.0: Statistic: 0.2000, p-value: 0.9048
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 3.0:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False


########## f1_class_Adventitia - GV ##########
Kruskal-Wallis test for the main effect of weight_to_neighbours:
Statistic: 40.6570, p-value: 0.0000

Testing the effect of weight_to_edge within weight_to_neighbours = 0.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.0: Statistic: 0.0452, p-value: 0.9776
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.0:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 0.1
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.1: Statistic: 0.1261, p-value: 0.9389
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.1:
Comparison 0 vs 1: Corrected p = 0.9166, Significant = False
Comparison 0 vs 5: Corrected p = 0.9166, Significant = False
Comparison 1 vs 5: Corrected p = 0.9166, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 0.3
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.3: Statistic: 0.1065, p-value: 0.9481
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.3:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 1.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 1.0: Statistic: 0.0809, p-value: 0.9604
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 1.0:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 3.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 3.0: Statistic: 0.5458, p-value: 0.7612
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 3.0:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 0.9006, Significant = False
Comparison 1 vs 5: Corrected p = 0.9006, Significant = False


########## f1_class_Epicardium ##########
Kruskal-Wallis test for the main effect of weight_to_neighbours:
Statistic: 12.9879, p-value: 0.0113

Testing the effect of weight_to_edge within weight_to_neighbours = 0.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.0: Statistic: 10.8200, p-value: 0.0045
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.0:
Comparison 0 vs 1: Corrected p = 0.0952, Significant = False
Comparison 0 vs 5: Corrected p = 0.0119, Significant = True
Comparison 1 vs 5: Corrected p = 0.0119, Significant = True

Testing the effect of weight_to_edge within weight_to_neighbours = 0.1
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.1: Statistic: 11.1800, p-value: 0.0037
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.1:
Comparison 0 vs 1: Corrected p = 0.0556, Significant = False
Comparison 0 vs 5: Corrected p = 0.0119, Significant = True
Comparison 1 vs 5: Corrected p = 0.0119, Significant = True

Testing the effect of weight_to_edge within weight_to_neighbours = 0.3
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.3: Statistic: 9.2600, p-value: 0.0098
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.3:
Comparison 0 vs 1: Corrected p = 0.0833, Significant = False
Comparison 0 vs 5: Corrected p = 0.0238, Significant = True
Comparison 1 vs 5: Corrected p = 0.0952, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 1.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 1.0: Statistic: 8.0600, p-value: 0.0178
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 1.0:
Comparison 0 vs 1: Corrected p = 0.6905, Significant = False
Comparison 0 vs 5: Corrected p = 0.0238, Significant = True
Comparison 1 vs 5: Corrected p = 0.0238, Significant = True

Testing the effect of weight_to_edge within weight_to_neighbours = 3.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 3.0: Statistic: 7.8791, p-value: 0.0195
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 3.0:
Comparison 0 vs 1: Corrected p = 0.3457, Significant = False
Comparison 0 vs 5: Corrected p = 0.0476, Significant = True
Comparison 1 vs 5: Corrected p = 0.0476, Significant = True


########## f1_class_Endocardium - Atrial ##########
Kruskal-Wallis test for the main effect of weight_to_neighbours:
Statistic: 19.0450, p-value: 0.0008

Testing the effect of weight_to_edge within weight_to_neighbours = 0.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.0: Statistic: 0.0955, p-value: 0.9534
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.0:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 0.1
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.1: Statistic: 0.4230, p-value: 0.8094
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.1:
Comparison 0 vs 1: Corrected p = 0.8325, Significant = False
Comparison 0 vs 5: Corrected p = 0.8325, Significant = False
Comparison 1 vs 5: Corrected p = 0.8325, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 0.3
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.3: Statistic: 0.0151, p-value: 0.9925
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.3:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 1.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 1.0: Statistic: 1.0437, p-value: 0.5934
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 1.0:
Comparison 0 vs 1: Corrected p = 0.8335, Significant = False
Comparison 0 vs 5: Corrected p = 0.8214, Significant = False
Comparison 1 vs 5: Corrected p = 0.8214, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 3.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 3.0: Statistic: 0.0459, p-value: 0.9773
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 3.0:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False


########## f1_class_Endocardium - Ventricular ##########
Kruskal-Wallis test for the main effect of weight_to_neighbours:
Statistic: 34.8658, p-value: 0.0000

Testing the effect of weight_to_edge within weight_to_neighbours = 0.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.0: Statistic: 0.1252, p-value: 0.9393
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.0:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 0.1
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.1: Statistic: 0.1800, p-value: 0.9139
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.1:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 0.3
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.3: Statistic: 0.1800, p-value: 0.9139
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.3:
Comparison 0 vs 1: Corrected p = 0.8413, Significant = False
Comparison 0 vs 5: Corrected p = 0.8413, Significant = False
Comparison 1 vs 5: Corrected p = 0.8413, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 1.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 1.0: Statistic: 0.1553, p-value: 0.9253
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 1.0:
Comparison 0 vs 1: Corrected p = 0.9166, Significant = False
Comparison 0 vs 5: Corrected p = 0.9166, Significant = False
Comparison 1 vs 5: Corrected p = 0.9166, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 3.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 3.0: Statistic: 0.0200, p-value: 0.9900
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 3.0:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False


########## f1_class_Endocardial cusion ##########
Kruskal-Wallis test for the main effect of weight_to_neighbours:
Statistic: 18.1907, p-value: 0.0011

Testing the effect of weight_to_edge within weight_to_neighbours = 0.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.0: Statistic: 0.0356, p-value: 0.9823
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.0:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 0.1
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.1: Statistic: 0.1069, p-value: 0.9479
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.1:
Comparison 0 vs 1: Corrected p = 0.9161, Significant = False
Comparison 0 vs 5: Corrected p = 0.9161, Significant = False
Comparison 1 vs 5: Corrected p = 0.9161, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 0.3
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.3: Statistic: 1.0101, p-value: 0.6035
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.3:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 0.6781, Significant = False
Comparison 1 vs 5: Corrected p = 0.6781, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 1.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 1.0: Statistic: 0.1377, p-value: 0.9335
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 1.0:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 3.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 3.0: Statistic: 0.0619, p-value: 0.9695
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 3.0:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False


########## f1_class_AV ring ##########
Kruskal-Wallis test for the main effect of weight_to_neighbours:
Statistic: 13.0272, p-value: 0.0111

Testing the effect of weight_to_edge within weight_to_neighbours = 0.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.0: Statistic: 0.1806, p-value: 0.9136
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.0:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 0.1
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.1: Statistic: 0.2454, p-value: 0.8845
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.1:
Comparison 0 vs 1: Corrected p = 0.9166, Significant = False
Comparison 0 vs 5: Corrected p = 0.9166, Significant = False
Comparison 1 vs 5: Corrected p = 0.9166, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 0.3
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.3: Statistic: 0.5829, p-value: 0.7472
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.3:
Comparison 0 vs 1: Corrected p = 0.9966, Significant = False
Comparison 0 vs 5: Corrected p = 0.9966, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 1.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 1.0: Statistic: 0.1362, p-value: 0.9342
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 1.0:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 3.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 3.0: Statistic: 0.4079, p-value: 0.8155
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 3.0:
Comparison 0 vs 1: Corrected p = 0.7533, Significant = False
Comparison 0 vs 5: Corrected p = 0.7533, Significant = False
Comparison 1 vs 5: Corrected p = 0.7533, Significant = False


########## f1_class_Valves ##########
Kruskal-Wallis test for the main effect of weight_to_neighbours:
Statistic: 33.6727, p-value: 0.0000

Testing the effect of weight_to_edge within weight_to_neighbours = 0.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.0: Statistic: 1.4576, p-value: 0.4825
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.0:
Comparison 0 vs 1: Corrected p = 0.6950, Significant = False
Comparison 0 vs 5: Corrected p = 0.6950, Significant = False
Comparison 1 vs 5: Corrected p = 0.8413, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 0.1
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.1: Statistic: 0.2865, p-value: 0.8665
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.1:
Comparison 0 vs 1: Corrected p = 0.9166, Significant = False
Comparison 0 vs 5: Corrected p = 0.9166, Significant = False
Comparison 1 vs 5: Corrected p = 0.9166, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 0.3
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 0.3: Statistic: 0.0955, p-value: 0.9534
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 0.3:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 1.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 1.0: Statistic: 0.1069, p-value: 0.9479
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 1.0:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False

Testing the effect of weight_to_edge within weight_to_neighbours = 3.0
Kruskal-Wallis test for 'weight_to_edge' within 'weight_to_neighbours' = 3.0: Statistic: 0.2435, p-value: 0.8854
Post-hoc pairwise comparisons for 'weight_to_edge' within 'weight_to_neighbours' = 3.0:
Comparison 0 vs 1: Corrected p = 1.0000, Significant = False
Comparison 0 vs 5: Corrected p = 1.0000, Significant = False
Comparison 1 vs 5: Corrected p = 1.0000, Significant = False
```

Example 17 (unknown):
```unknown
tissues_significant
```

Example 18 (unknown):
```unknown
['f1_class_Atrium',
 'f1_class_Ventricle - Compact',
 'f1_class_Ventricle - Trabeculated',
 'f1_class_AVnode',
 'f1_class_VCS - Distal',
 'f1_class_Great vessel',
 'f1_class_Coronary vessel',
 'f1_class_Adventitia - GV',
 'f1_class_Epicardium',
 'f1_class_Endocardium - Atrial',
 'f1_class_Endocardium - Ventricular',
 'f1_class_Endocardial cusion',
 'f1_class_AV ring',
 'f1_class_Valves']
```

Example 19 (python):
```python
for y_value in ['accuracy','balanced_accuracy']:
    plt.figure(figsize=(3, 3))
    sns.boxplot(data=accuracy_df,x='weight_to_neighbours',y=y_value,hue='weight_to_edge',palette='Set2') # ''
    plt.xticks(rotation=90)
    plt.legend(loc='upper right',title='edge_weight',bbox_to_anchor=(1.6, 1))
    plt.title('')
    plt.savefig(f'{figdir}/TissueTypist_cross-validation_overall-{y_value}.pdf', bbox_inches="tight",dpi=300)
    plt.show()
```

Example 20 (unknown):
```unknown
y_value_list = [x for x in f1score_df.columns if ('f1_class_' in x)&('Adventitia - Coronary' not in x)]
len(y_value_list)
```

Example 21 (python):
```python
plt.rcParams['axes.grid'] = False
fig,axs=plt.subplots(nrows=3,ncols=6,figsize=(20,8))
# Turn the (3×6) array into a flat list of 18 Axes
axs = axs.flatten()
for i,y_value in enumerate(y_value_list):
    sns.boxplot(data=f1score_df,x='weight_to_neighbours',y=y_value,
                hue='weight_to_edge',palette='Set2',
                ax=axs[i])
    axs[i].set_title(y_value.replace('f1_class_',''))
    axs[i].set_xlabel('neighbour_weight')
    axs[i].set_ylabel('f1 score')
    axs[i].tick_params(axis='x', rotation=90)
    if i==(len(y_value_list)-1):
        axs[i].legend(loc='lower right',title='edge_weight',bbox_to_anchor=(0, -0.9))
    else:
        axs[i].get_legend().remove()
# fig.suptitle('Main Title for the Figure')
# fig.tight_layout(w_pad=1)
fig.subplots_adjust(
    left=0.1, right=0.9,      # margins from figure edge
    bottom=0.1, top=0.9,
    wspace=0.4,               # horizontal space between cols
    hspace=0.6                # vertical space between rows
)
plt.savefig(f'{figdir}/TissueTypist_cross-validation_each-label-f1score.pdf', bbox_inches="tight",dpi=300)
plt.show()
```

---

## Import modules¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/RNA/QC/Foetal_scRNA_No1_make-anndata_run-scrublet.html

**Contents:**
- Import modules¶
- Functions¶
- Create anndata object¶
- Run scrublet¶
- Save¶

205457 rows × 21 columns

**Examples:**

Example 1 (python):
```python
import numpy as np
import pandas as pd
import scanpy as sc
import matplotlib.pyplot as plt
import scrublet as scr
import session_info
```

Example 2 (python):
```python
sc.settings.verbosity = 3
sc.settings.set_figure_params(dpi = 120, color_map = 'RdPu', dpi_save = 180, vector_friendly = True, format = 'svg')
session_info.show()
```

Example 3 (python):
```python
-----
matplotlib          3.3.4
numpy               1.20.1
pandas              1.1.4
scanpy              1.7.1
scrublet            NA
session_info        1.0.0
-----
```

Example 4 (unknown):
```unknown
PIL                 8.1.2
anndata             0.7.5
anyio               NA
attr                20.3.0
babel               2.9.0
backcall            0.2.0
brotli              NA
cairo               1.20.0
certifi             2020.12.05
cffi                1.14.5
chardet             4.0.0
cloudpickle         1.6.0
colorama            0.4.4
cycler              0.10.0
cython_runtime      NA
cytoolz             0.11.0
dask                2021.03.1
dateutil            2.8.1
decorator           4.4.2
fsspec              0.8.7
get_version         2.1
google              NA
h5py                2.10.0
idna                2.10
igraph              0.8.3
ipykernel           5.5.0
ipython_genutils    0.2.0
jedi                0.18.0
jinja2              2.11.3
joblib              1.0.1
json5               NA
jsonschema          3.2.0
jupyter_server      1.4.1
jupyterlab_server   2.3.0
kiwisolver          1.3.1
legacy_api_wrap     0.0.0
leidenalg           0.8.3
llvmlite            0.34.0
louvain             0.7.0
markupsafe          1.1.1
mpl_toolkits        NA
natsort             7.1.1
nbclassic           NA
nbformat            5.1.2
numba               0.51.2
numexpr             2.7.3
packaging           20.9
parso               0.8.1
pexpect             4.8.0
pickleshare         0.7.5
pkg_resources       NA
prometheus_client   NA
prompt_toolkit      3.0.16
psutil              5.8.0
ptyprocess          0.7.0
pvectorc            NA
pyarrow             0.16.0
pygments            2.8.0
pyparsing           2.4.7
pyrsistent          NA
pytoml              NA
pytz                2021.1
requests            2.25.1
ruamel              NA
scipy               1.6.1
send2trash          NA
setuptools_scm      NA
simplejson          3.17.2
sinfo               0.3.1
six                 1.15.0
sklearn             0.22
sniffio             1.2.0
socks               1.7.1
sphinxcontrib       NA
storemagic          NA
tables              3.6.1
tblib               1.7.0
texttable           1.6.3
tlz                 0.11.0
toolz               0.11.1
tornado             6.1
traitlets           5.0.5
typing_extensions   NA
uritemplate         3.0.1
urllib3             1.26.3
wcwidth             0.2.5
yaml                5.1.2
zmq                 22.0.3
```

Example 5 (unknown):
```unknown
-----
IPython             7.21.0
jupyter_client      6.1.11
jupyter_core        4.7.1
jupyterlab          3.0.9
notebook            6.2.0
-----
Python 3.8.8 | packaged by conda-forge | (default, Feb 20 2021, 16:22:27) [GCC 9.3.0]
Linux-4.15.0-158-generic-x86_64-with-glibc2.10
-----
Session information updated at 2022-12-23 09:18
```

Example 6 (python):
```python
"""Functions for downstream work with outputs of remove-background."""

# to read cellbender outputs<br>
# https://github.com/broadinstitute/CellBender/issues/57

import tables
import numpy as np
import scipy.sparse as sp
from typing import Dict


def dict_from_h5(file: str) -> Dict[str, np.ndarray]:
    """Read in everything from an h5 file and put into a dictionary."""
    d = {}
    with tables.open_file(file) as f:
        # read in everything
        for array in f.walk_nodes("/", "Array"):
            d[array.name] = array.read()
    return d


def anndata_from_h5(file: str,
                    analyzed_barcodes_only: bool = True) -> 'anndata.AnnData':
    """Load an output h5 file into an AnnData object for downstream work.

    Args:
        file: The h5 file
        analyzed_barcodes_only: False to load all barcodes, so that the size of
            the AnnData object will match the size of the input raw count matrix.
            True to load a limited set of barcodes: only those analyzed by the
            algorithm. This allows relevant latent variables to be loaded
            properly into adata.obs and adata.obsm, rather than adata.uns.

    Returns:
        adata: The anndata object, populated with inferred latent variables
            and metadata.

    """

    try:
        import anndata
    except ImportError:
        raise ImportError('The anndata package must be installed to use the '
                          'function anndata_from_h5()')

    d = dict_from_h5(file)
    X = sp.csc_matrix((d.pop('data'), d.pop('indices'), d.pop('indptr')),
                      shape=d.pop('shape')).transpose().tocsr()

    if analyzed_barcodes_only:
        if 'barcodes_analyzed_inds' in d.keys():
            X = X[d['barcodes_analyzed_inds'], :]
            d['barcodes'] = d['barcodes'][d['barcodes_analyzed_inds']]
        elif 'barcode_indices_for_latents' in d.keys():
            X = X[d['barcode_indices_for_latents'], :]
            d['barcodes'] = d['barcodes'][d['barcode_indices_for_latents']]
        else:
            print('Warning: analyzed_barcodes_only=True, but the key '
                  '"barcodes_analyzed_inds" or "barcode_indices_for_latents" '
                  'is missing from the h5 file. '
                  'Will output all barcodes, and proceed as if '
                  'analyzed_barcodes_only=False')

    # Construct the count matrix.
    adata = anndata.AnnData(X=X,
                            obs={'barcode': d.pop('barcodes').astype(str)},
                            var={'gene_name': (d.pop('gene_names') if 'gene_names' in d.keys()
                                               else d.pop('name')).astype(str)})
    adata.obs.set_index('barcode', inplace=True)
    adata.var.set_index('gene_name', inplace=True)

    # Add other information to the adata object in the appropriate slot.
    for key, value in d.items():
        try:
            value = np.asarray(value)
            if len(value.shape) == 0:
                adata.uns[key] = value
            elif value.shape[0] == X.shape[0]:
                if (len(value.shape) < 2) or (value.shape[1] < 2):
                    adata.obs[key] = value
                else:
                    adata.obsm[key] = value
            elif value.shape[0] == X.shape[1]:
                if value.dtype.name.startswith('bytes'):
                    adata.var[key] = value.astype(str)
                else:
                    adata.var[key] = value
            else:
                adata.uns[key] = value
        except Exception:
            print('Unable to load data into AnnData: ', key, value, type(value))

    if analyzed_barcodes_only:
        for col in adata.obs.columns[adata.obs.columns.str.startswith('barcodes_analyzed')
                                     | adata.obs.columns.str.startswith('barcode_indices')]:
            try:
                del adata.obs[col]
            except Exception:
                pass

    return adata
```

Example 7 (python):
```python
# from https://github.com/Teichlab/mapcloud/blob/master/scripts/starsolo/postprocess.py

from statsmodels.stats.multitest import multipletests
# from emptydrops.matrix import CountMatrix
# from emptydrops import find_nonambient_barcodes
import scrublet as scr
import scanpy as sc
import pandas as pd
import numpy as np
import scipy

#some functions that Ni uses in scanpy scripts to run scrublet
#which in turn are inspired by my original notebook on the matter
#(extracted from scanpy_scripts 0.2.10 to get around scanpy version incompatibility)
def test_outlier(x, upper_mad_only=True):
	med = np.median(x)
	if upper_mad_only:
		mad = np.median(x[x>med] - med) * 1.4826
	else:
		mad = np.median(np.abs(x - med)) * 1.4826
	pvals = 1 - scipy.stats.norm.cdf(x, loc=med, scale=mad)
	bh_pvals = multipletests(pvals, method='fdr_bh')[1]
	return pvals, bh_pvals

def run_scrublet(adata, resolution_function=None):
	old_verbosity = sc.settings.verbosity
	sc.settings.verbosity = 1
	if resolution_function is None:
		resolution_function = lambda x: np.maximum(np.maximum(np.log10(x)-1, 0)**2, 0.1)
	scrub = scr.Scrublet(adata.X)
	#this has the potential to brick for poor quality data
	#if so, abort it and everything downstream
	try:
		ds, pd = scrub.scrub_doublets(verbose=False)
	except:
		return
	adata.obs['scrublet_score'] = ds

	adata_copy = adata.copy()
	sc.pp.filter_genes(adata_copy, min_cells=3)
	sc.pp.normalize_total(adata_copy, target_sum=1e4)
	sc.pp.log1p(adata_copy)
	sc.pp.highly_variable_genes(adata_copy, min_mean=0.0125, max_mean=3, min_disp=0.5, subset=True)
	sc.pp.scale(adata_copy, zero_center=False)
	sc.pp.pca(adata_copy, svd_solver='arpack', zero_center=False)
	sc.pp.neighbors(adata_copy, n_pcs=30)
	sc.tl.umap(adata_copy)
	sc.tl.leiden(adata_copy, resolution=1)
	for clst in np.unique(adata_copy.obs['leiden']):
		clst_size = sum(adata_copy.obs['leiden'] == clst)
		sc.tl.leiden(adata_copy, restrict_to=('leiden', [clst]), resolution=resolution_function(clst_size), key_added='leiden_R')
		adata_copy.obs['leiden'] = adata_copy.obs['leiden_R']
	clst_meds = []
	for clst in np.unique(adata_copy.obs['leiden']):
		k = adata_copy.obs['leiden'] == clst
		clst_med = np.median(adata_copy.obs.loc[k, 'scrublet_score'])
		adata_copy.obs.loc[k, 'cluster_scrublet_score'] = clst_med
		clst_meds.append(clst_med)
	clst_meds = np.array(clst_meds)
	pvals, bh_pvals = test_outlier(clst_meds)
	for i, clst in enumerate(np.unique(adata_copy.obs['leiden'])):
		k = adata_copy.obs['leiden'] == clst
		adata_copy.obs.loc[k, 'pval'] = pvals[i]
		adata_copy.obs.loc[k, 'bh_pval'] = bh_pvals[i]
	sc.settings.verbosity = old_verbosity
	#need to also export the clustering, for soupx purposes
	adata.obs['scrublet_leiden'] = adata_copy.obs['leiden']
	adata.obs['scrublet_score'] = adata_copy.obs['scrublet_score']
	adata.obs['cluster_scrublet_score'] = adata_copy.obs['cluster_scrublet_score']
	adata.obs['doublet_pval'] = adata_copy.obs['pval']
	adata.obs['doublet_bh_pval'] = adata_copy.obs['bh_pval']
	del adata_copy
```

Example 8 (python):
```python
# read in metadata
metadata = pd.read_csv('/nfs/team205/heart/anndata_objects/Foetal/metadata/HeartTeamSamples_Mappeddata_20221223.csv', sep = ',', index_col = None)
metadata = metadata[metadata['Publication']=='Foetal']

# select healthy data
metadata = metadata[metadata['status']=='Healthy']


print(metadata['modality'].value_counts())
```

Example 9 (unknown):
```unknown
Multiome-ATAC    38
Multiome-RNA     38
Visium           23
scRNA            20
Name: modality, dtype: int64
```

Example 10 (python):
```python
metadata['region'].value_counts()
```

Example 11 (unknown):
```unknown
Heart                                                            59
Apex                                                              8
Left ventricle                                                    7
Base                                                              6
Outflow Tract                                                     5
Pulmonary arches and branches                                     4
Aorta (Hst39), Nodes (Hst40)                                      4
Aorta                                                             4
Pericardium                                                       4
Heart-wo-nodes                                                    4
Heart (Hst36), Heart-wo-nodes (Hst39), Heart-wo-nodes (Hst40)     4
Ductus Arterious                                                  4
Atrial septum                                                     2
Right ventricle + Papillary                                       2
Nodes                                                             2
Name: region, dtype: int64
```

Example 12 (python):
```python
# select modality
metadata = metadata[metadata['modality']=='scRNA']
```

Example 13 (python):
```python
pd.crosstab(metadata['kit_10x'], metadata['facility'])
```

Example 14 (python):
```python
########### editted: put id to var_names ###########

# read in
adatas=[]
for i in range(len(metadata)):
    print(metadata.iloc[i]['sangerID'])
    
    path=metadata.iloc[i]['CellBender_out']
    adatas.append(anndata_from_h5(path + '/' + path.split('/')[-1] + '_cellbender_out_filtered.h5', analyzed_barcodes_only=False))
    del path
    
    # replace var_names with ensembleID
    adatas[i].var.reset_index(inplace=True)
    if 'id' in adatas[i].var.columns:
        adatas[i].var.set_index('id',inplace=True)
    else:
        adatas[i].var.set_index('genes',inplace=True)
    
    # modify barcodes
    adatas[i].obs.index = metadata.iloc[i]['sangerID'] + '_' + adatas[i].obs.index 
    
    # add metadata
    for col in ['sangerID','combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age',
                'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x','flushed']:
        adatas[i].obs[col] = metadata.iloc[i][col]


# concatenate
adata = adatas[0].concatenate(adatas[1:], index_unique = None, batch_key=None)
adata.shape
```

Example 15 (unknown):
```unknown
BHF_F_Hea10402917
BHF_F_Hea10402918
BHF_F_Hea11192323
BHF_F_Hea11192324
BHF_F_Hea11192325
BHF_F_Hea11192326
BHF_F_Hea11192327
BHF_F_Hea11192328
Sinha_A1
Sinha_B1
Sinha_C8
Sinha_D8
Sinha_E2
Sinha_E8
Sinha_F2
Sinha_F4
Sinha_F7
Sinha_F8
Sinha_G4
Sinha_G7
```

Example 16 (unknown):
```unknown
(205457, 36601)
```

Example 17 (python):
```python
for i in range(len(adatas)):
    print(adatas[i].var.columns)
```

Example 18 (unknown):
```unknown
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
```

Example 19 (python):
```python
adata.obs['sangerID'].value_counts()
```

Example 20 (unknown):
```unknown
Sinha_F2             24275
Sinha_E2             24227
BHF_F_Hea11192328    21714
Sinha_E8             17219
BHF_F_Hea11192324    16688
BHF_F_Hea11192326    14117
BHF_F_Hea11192325    10676
Sinha_F4             10652
BHF_F_Hea11192327     9130
BHF_F_Hea10402918     8010
Sinha_F7              6973
Sinha_D8              6714
Sinha_A1              6640
Sinha_B1              5988
BHF_F_Hea11192323     4893
Sinha_C8              4587
BHF_F_Hea10402917     3752
Sinha_G4              3389
Sinha_F8              3203
Sinha_G7              2610
Name: sangerID, dtype: int64
```

Example 21 (python):
```python
adata.obs['sangerID'].unique()
```

Example 22 (unknown):
```unknown
array(['BHF_F_Hea10402917', 'BHF_F_Hea10402918', 'BHF_F_Hea11192323',
       'BHF_F_Hea11192324', 'BHF_F_Hea11192325', 'BHF_F_Hea11192326',
       'BHF_F_Hea11192327', 'BHF_F_Hea11192328', 'Sinha_A1', 'Sinha_B1',
       'Sinha_C8', 'Sinha_D8', 'Sinha_E2', 'Sinha_E8', 'Sinha_F2',
       'Sinha_F4', 'Sinha_F7', 'Sinha_F8', 'Sinha_G4', 'Sinha_G7'],
      dtype=object)
```

Example 23 (python):
```python
%%time

# per rxn
for i,ID in enumerate(adata.obs['sangerID'].unique()):
    print(ID)
    
    ad = adata[adata.obs['sangerID'] == ID].copy()
    run_scrublet(ad)
    if i==0:
        meta = ad.obs
    else:
        meta = pd.concat([meta, ad.obs])
    del ad
```

Example 24 (unknown):
```unknown
BHF_F_Hea10402917
```

Example 25 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 26 (unknown):
```unknown
BHF_F_Hea10402918
```

Example 27 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 28 (unknown):
```unknown
BHF_F_Hea11192323
```

Example 29 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 30 (unknown):
```unknown
BHF_F_Hea11192324
```

Example 31 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 32 (unknown):
```unknown
BHF_F_Hea11192325
```

Example 33 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 34 (unknown):
```unknown
BHF_F_Hea11192326
```

Example 35 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 36 (unknown):
```unknown
BHF_F_Hea11192327
```

Example 37 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 38 (unknown):
```unknown
BHF_F_Hea11192328
```

Example 39 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 40 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 41 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 42 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 43 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 44 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 45 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 46 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 47 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 48 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 49 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 50 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 51 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 52 (unknown):
```unknown
CPU times: user 57min 17s, sys: 50min 50s, total: 1h 48min 7s
Wall time: 24min 42s
```

Example 53 (python):
```python
meta_scrub = meta[['scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval']].copy()
meta_scrub.shape
```

Example 54 (unknown):
```unknown
(205457, 5)
```

Example 55 (python):
```python
if meta_scrub.reindex(adata.obs.index).index.equals(adata.obs.index):
    adata.obs = pd.concat([adata.obs, meta_scrub.reindex(adata.obs.index)], axis=1)
else:
    raise Exception('Different barcodes in meta and adata')
```

Example 56 (python):
```python
adata.write('/nfs/team205/heart/anndata_objects/Foetal/QC/scRNA_foetal_prefilter.h5ad')
```

Example 57 (unknown):
```unknown
... storing 'sangerID' as categorical
... storing 'combinedID' as categorical
... storing 'donor' as categorical
... storing 'donor_type' as categorical
... storing 'region' as categorical
... storing 'region_finest' as categorical
... storing 'age' as categorical
... storing 'gender' as categorical
... storing 'facility' as categorical
... storing 'cell_or_nuclei' as categorical
... storing 'modality' as categorical
... storing 'kit_10x' as categorical
... storing 'flushed' as categorical
... storing 'scrublet_leiden' as categorical
... storing 'gene_name' as categorical
... storing 'feature_type' as categorical
```

Example 58 (python):
```python
sc.pl.violin(adata[adata.obs['facility']=='Sanger'],
             ['scrublet_score', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval'],jitter = 0.2, multi_panel = True)

sc.pl.violin(adata[adata.obs['facility']=='CSCI'],
             ['scrublet_score', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval'],jitter = 0.2, multi_panel = True)
```

Example 59 (python):
```python
adata.obs['sangerID'].value_counts()
```

Example 60 (unknown):
```unknown
Sinha_F2             24275
Sinha_E2             24227
BHF_F_Hea11192328    21714
Sinha_E8             17219
BHF_F_Hea11192324    16688
BHF_F_Hea11192326    14117
BHF_F_Hea11192325    10676
Sinha_F4             10652
BHF_F_Hea11192327     9130
BHF_F_Hea10402918     8010
Sinha_F7              6973
Sinha_D8              6714
Sinha_A1              6640
Sinha_B1              5988
BHF_F_Hea11192323     4893
Sinha_C8              4587
BHF_F_Hea10402917     3752
Sinha_G4              3389
Sinha_F8              3203
Sinha_G7              2610
Name: sangerID, dtype: int64
```

---

## Read modules¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig4_Cardiomyocytes/VentricularCardiomyocytes/TransmuralAxis/tm_axis_calculation_OrganAxis.html

**Contents:**
- Read modules¶
- Function¶
- List of samples¶
- Core axis calculation per slide¶

**Examples:**

Example 1 (unknown):
```unknown
TissueTag: OrganAxis
```

Example 2 (python):
```python
import os
import numpy as np 
# from bokeh.plotting import show, output_notebook
import pandas as pd
import socket
# os.chdir(path_to_tissue_tag)
# import tissue_tag as tt
import matplotlib.pyplot as plt
import scanpy as sc
# output_notebook()
# host = '5016'
```

Example 3 (python):
```python
# kk18
# based on a cell2location function
def select_multislide(adata, s, s_col='sample'):

    slides = adata[adata.obs[s_col].isin(s), :]
    s_keys = list(slides.uns['spatial'].keys())
    
    uns_dict = {}
    for ss in s:
        s_spatial=np.array(s_keys)[[ss in k for k in s_keys]][0]
        uns_dict[s_spatial]=slides.uns['spatial'][s_spatial]
    
    slides.uns['spatial'] = uns_dict

    return slides

# ny1, TissueTag:OrganAxis
def axis_2p_norm(
    Dist2ClusterAll,
    structure_list,
    weights = [1,1], 
):
    import warnings
    warnings.filterwarnings("ignore")
    # CMA calculations 
    fa = weights[0]
    fb = weights[0]
    axis = np.array([( fa*int(a) - fb*int(b) ) / ( fa*int(a) + fb*int(b) ) for a,b in zip(Dist2ClusterAll[structure_list[0]], Dist2ClusterAll[structure_list[1]])])
    return axis

# ny1, TissueTag:OrganAxis
def dist2cluster(df, annotation, ppm, KNN=4, calc_dist=True, logscale=False):
    """
    Calculates the minimal Euclidean distance of each point in space to an anatomical structure.

    Parameters
    ----------
    df : pandas.DataFrame
        Dataframe containing spatial data.
    annotation : str
        The column in df that contains the structure annotation.
    ppm : float
        the coordinates of df in pixel scale
    KNN : int, optional
        Determines the number of closest points used to calculate the mean distance. Default is 4.
    calc_dist : bool, optional
        If true, calculates the distance. Default is True.
    logscale : bool, optional
        If true, applies a log10 transformation to the distance. Default is False.
    

    Returns
    -------
    dict
        Dictionary with the median distance for each category in the annotation column.
    """
    pd.options.mode.chained_assignment = None  # default='warn'
    from scipy.spatial import distance_matrix, distance
    print('calculating distance matrix')
    a = np.vstack([df['x'],df['y']]).astype(int)
    distM = distance.cdist(a.T,a.T, metric='euclidean')

    Dist2ClusterAll = {}
    categories = np.unique(df[annotation])

    for idx, c in enumerate(categories):
        indextmp = df[annotation] == c
        if len(np.where(indextmp)[0]) > KNN:
            print(c)
            Dist2ClusterAll[c] = np.median(np.sort(distM[indextmp, :], axis=0)[range(KNN), :], axis=0)

    for c in categories: 
        if c != 'unassigned':
            if calc_dist:
                if logscale:
                    df["L2_dist_log10_"+annotation+'_'+c] = np.log10(Dist2ClusterAll[c]/ppm)
                else:
                    df["L2_dist_"+annotation+'_'+c] = Dist2ClusterAll[c]/ppm
            df[annotation] = categories[np.argmin(np.array(list(Dist2ClusterAll.values())), axis=0)]
    
    return Dist2ClusterAll
```

Example 4 (unknown):
```unknown
# selected visium slides which have transmural LV
# and annotated "tm_axis" (inmost or outmost) and "tm_axis_region-select" (LV)
path_to_annotation = '/nfs/team205/heart/visium_annotations/foetal/transmural'
samples = [x.split('_tm_axis.csv')[0] for x in os.listdir(path_to_annotation) if '_tm_axis.csv' in x]
samples.sort()
samples
```

Example 5 (unknown):
```unknown
['BHF_F_Hea13295830',
 'BHF_F_Hea13295831',
 'HCAHeartST10317184',
 'HCAHeartST10317185',
 'HCAHeartST10317186',
 'HCAHeartST11605165',
 'HCAHeartST11605167',
 'HCAHeartST11605168',
 'HCAHeartST13162330',
 'HCAHeartST13162331',
 'HCAHeartST13162336']
```

Example 6 (python):
```python
# sample metadata
meta=pd.read_csv('/nfs/team205/heart/anndata_objects/Foetal/metadata/HeartTeamSamples_Mappeddata_20230208.csv')
meta=meta[['Publication','sangerID', 'combinedID',
       'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender',
       'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'status']]
meta=meta[meta['modality'].isin(['Visium','Visium-FFPE'])]

# select slides for foetal publication
meta=meta[meta['Publication'].isin(['Foetal'])]

# samples for tm_axis
meta.set_index('sangerID',inplace=True)
meta = meta.loc[samples]
meta.reset_index(inplace=True)

# group
meta['group'] = meta['modality']+'_'+meta['status']+'_'+meta['region']
meta
```

Example 7 (unknown):
```unknown
# prepare dictionary
sample_dict = {}
for grp in set(meta['group']):
    sample_dict[grp] = list(meta[meta['group']==grp]['sangerID'])
sample_dict
```

Example 8 (unknown):
```unknown
{'Visium_Healthy_Left ventricle': ['HCAHeartST13162330', 'HCAHeartST13162331'],
 'Visium_Healthy_Heart': ['HCAHeartST10317184',
  'HCAHeartST10317185',
  'HCAHeartST10317186',
  'HCAHeartST11605165',
  'HCAHeartST11605167',
  'HCAHeartST11605168'],
 'Visium_Trisomy21_Heart': ['HCAHeartST13162336'],
 'Visium-FFPE_Healthy_Heart': ['BHF_F_Hea13295830', 'BHF_F_Hea13295831']}
```

Example 9 (unknown):
```unknown
!ls /nfs/team205/heart/anndata_objects/Foetal
```

Example 10 (python):
```python
14Feb_fromSemih		      RNA_foetal_post-scVI_raw.h5ad
archive			      RNA_foetal_raw.h5ad
Feb28ObjectRaw.h5ad	      scArches
Feb28ObjectRaw_Multiome.h5ad  scVI
Feb28ObjectRaw_Multiome.rds   TipAndStalkAndGVSMCsPartitionedForCell2Loc.h5ad
Feb28ObjectRaw.obs.csv	      trisomy21
Feb28ObjectRaw_original.h5ad  visium-FFPE_foetal_raw.h5ad
Feb28ObjectRaw_re-scVI.h5ad   visium_foetal_healthy-all_raw.h5ad
He_Cell_foetal-lung	      visium_foetal_healthy-all_spot-annotation.csv
Hill2022.h5ad		      visium_foetal_raw.h5ad
Hill2022_QCapplied.h5ad       visium-OCT_foetal_acrossage_raw.h5ad
Immune			      visium-OCT_foetal_DuctusA_raw.h5ad
metadata		      visium-OCT_foetal_raw.h5ad
multiome_ATAC		      visium-OCT-LV_foetal_raw.h5ad
multiVI			      visium-OCT-T21_foetal_raw.h5ad
QC
```

Example 11 (python):
```python
sc.set_figure_params(figsize=[5,5],dpi=100)
```

Example 12 (python):
```python
for grp in sample_dict.keys():
    print(f'##### {grp} #####')
    
    # read in visium adata object
    if grp=='Visium-FFPE_Healthy_Heart':
        adata_vis = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/visium-FFPE_foetal_raw.h5ad')
    elif grp=='Visium_Healthy_Heart':
        adata_vis = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/visium-OCT_foetal_raw.h5ad')
    elif grp=='Visium_Healthy_Left ventricle':
        adata_vis = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/visium-OCT-LV_foetal_raw.h5ad')
    elif grp=='Visium_Trisomy21_Heart':
        adata_vis = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/visium-OCT-T21_foetal_raw.h5ad')    
    adata_vis.obs['array_row'] = adata_vis.obs['array_row'].astype('int')
    adata_vis.obs['array_col'] = adata_vis.obs['array_col'].astype('int')
    adata_vis.obsm['spatial'] = adata_vis.obsm['spatial'].astype('int')

    for slide_id in sample_dict[grp]:
        print(slide_id) 
        # select one slide
        slide = select_multislide(adata_vis, [slide_id], s_col='sample')

        # read in annotation file and add
        ## annotation to select
        annot_select_region = pd.read_csv(f'{path_to_annotation}/{slide_id}_tm_axis_select-region.csv',index_col=0)
        annot_select_region.index = slide_id+'_'+annot_select_region.index
        annot_select_region.fillna('none',inplace=True)
        slide.obs['tm_axis_select-region'] = annot_select_region['tm_axis_select-region'].reindex(slide.obs_names)
        ## annotation, transmural
        annot_tm = pd.read_csv(f'{path_to_annotation}/{slide_id}_tm_axis.csv',index_col=0)
        annot_tm.index = slide_id+'_'+annot_tm.index
        annot_tm.fillna('none',inplace=True)
        slide.obs['tm_annotation'] = annot_tm['tm_axis'].reindex(slide.obs_names)
        ## plot
        sc.pl.spatial(slide,color=['tm_axis_select-region','tm_annotation'],ncols=2,wspace=0.2)

        # prepare dataframe for distance calculation
        obs_df = slide.obs.copy()
        obs_df.rename(columns={'array_row':'y','array_col':'x'},inplace=True)

        regions_to_select = [x for x in set(obs_df['tm_axis_select-region']) if x!='none']
        # if the slide has one heart 
        if len(regions_to_select)==1: 
            # select region
            obs_df_sel = obs_df[obs_df['tm_axis_select-region']==regions_to_select[0]]
    
            # calculate distance
            print('calculating distances')
            dist2clusterall = dist2cluster(obs_df_sel,annotation='tm_annotation',KNN=1,ppm=1)
            slide.obs.loc[obs_df_sel.index,obs_df_sel.columns] = obs_df_sel.copy()
    
            # calculate core axis
            arr = axis_2p_norm(
                dist2clusterall,
                ['outmost','inmost'],
                weights = [1,1], 
            )
            slide.obs.loc[obs_df_sel.index,'transmural_axis'] = arr
        
        # if the slide has multiple heart
        elif len(regions_to_select)>1:
            for region in regions_to_select:
                # select region
                obs_df_sel = obs_df[obs_df['tm_axis_select-region']==region]
        
                # calculate distance
                print('calculating distances')
                dist2clusterall = dist2cluster(obs_df_sel,annotation='tm_annotation',KNN=1,ppm=1)
                slide.obs.loc[obs_df_sel.index,obs_df_sel.columns] = obs_df_sel.copy()
        
                # calculate core axis
                arr = axis_2p_norm(
                    dist2clusterall,
                    ['outmost','inmost'],
                    weights = [1,1], 
                )
                slide.obs.loc[obs_df_sel.index,'transmural_axis'] = arr

        # plot
        sc.pl.spatial(slide,color=['transmural_axis'],cmap='gist_rainbow',ncols=2,wspace=0.25)
        # save axis
        slide.obs[['transmural_axis']].to_csv(f'/nfs/team205/heart/visium_axis/foetal/transmural_LV/{slide_id}.csv')
```

Example 13 (unknown):
```unknown
##### Visium_Healthy_Left ventricle #####
HCAHeartST13162330
```

Example 14 (unknown):
```unknown
calculating distances
calculating distance matrix
inmost
none
outmost
```

Example 15 (unknown):
```unknown
HCAHeartST13162331
```

Example 16 (unknown):
```unknown
calculating distances
calculating distance matrix
inmost
none
outmost
```

Example 17 (unknown):
```unknown
##### Visium_Healthy_Heart #####
HCAHeartST10317184
```

Example 18 (unknown):
```unknown
calculating distances
calculating distance matrix
inmost
none
outmost
```

Example 19 (unknown):
```unknown
HCAHeartST10317185
```

Example 20 (unknown):
```unknown
calculating distances
calculating distance matrix
inmost
none
outmost
```

Example 21 (unknown):
```unknown
HCAHeartST10317186
```

Example 22 (unknown):
```unknown
calculating distances
calculating distance matrix
inmost
none
outmost
```

Example 23 (unknown):
```unknown
HCAHeartST11605165
```

Example 24 (unknown):
```unknown
calculating distances
calculating distance matrix
inmost
none
outmost
```

Example 25 (unknown):
```unknown
HCAHeartST11605167
```

Example 26 (unknown):
```unknown
calculating distances
calculating distance matrix
inmost
none
outmost
calculating distances
calculating distance matrix
inmost
none
outmost
```

Example 27 (unknown):
```unknown
HCAHeartST11605168
```

Example 28 (unknown):
```unknown
calculating distances
calculating distance matrix
inmost
none
outmost
```

Example 29 (unknown):
```unknown
##### Visium_Trisomy21_Heart #####
HCAHeartST13162336
```

Example 30 (unknown):
```unknown
calculating distances
calculating distance matrix
inmost
none
outmost
```

Example 31 (unknown):
```unknown
##### Visium-FFPE_Healthy_Heart #####
BHF_F_Hea13295830
```

Example 32 (unknown):
```unknown
calculating distances
calculating distance matrix
inmost
none
outmost
```

Example 33 (unknown):
```unknown
BHF_F_Hea13295831
```

Example 34 (unknown):
```unknown
calculating distances
calculating distance matrix
inmost
none
outmost
```

Example 35 (unknown):
```unknown
!ls /nfs/team205/heart/visium_axis/foetal/transmural_LV
```

Example 36 (unknown):
```unknown
BHF_F_Hea13295830.csv	HCAHeartST10317186.csv	HCAHeartST13162330.csv
BHF_F_Hea13295831.csv	HCAHeartST11605165.csv	HCAHeartST13162331.csv
HCAHeartST10317184.csv	HCAHeartST11605167.csv	HCAHeartST13162336.csv
HCAHeartST10317185.csv	HCAHeartST11605168.csv
```

---

## Import required modules¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/Visium/Others/Parasymp-vs-Symp_aggr.html

**Contents:**
- Import required modules¶
- Function¶
- Read in anndata¶
- Merge categories: tissue annotation and week¶
- Preapare cell2location output anndata¶
- Select regions and cells to plot¶
- Get spots which includes PSNS or SNS and mean abundance¶
- How about in single-cell data¶

**Examples:**

Example 1 (python):
```python
import warnings
warnings.simplefilter(action='ignore', category=FutureWarning)

import scanpy as sc
import anndata
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 

import seaborn as sns
import os 
from mpl_toolkits.axes_grid1 import make_axes_locatable

import scipy
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
-----
anndata             0.9.2
matplotlib          3.7.2
mpl_toolkits        NA
numpy               1.24.4
pandas              1.5.3
scanpy              1.9.3
scipy               1.10.1
seaborn             0.12.2
session_info        1.0.0
-----
```

Example 4 (python):
```python
PIL                         10.0.0
anyio                       NA
arrow                       1.2.3
asttokens                   NA
attr                        23.1.0
attrs                       23.1.0
babel                       2.12.1
backcall                    0.2.0
brotli                      1.0.9
certifi                     2023.11.17
cffi                        1.15.1
charset_normalizer          3.2.0
colorama                    0.4.6
comm                        0.1.4
cycler                      0.10.0
cython_runtime              NA
dateutil                    2.8.2
debugpy                     1.6.8
decorator                   5.1.1
defusedxml                  0.7.1
executing                   1.2.0
fastjsonschema              NA
fqdn                        NA
h5py                        3.9.0
idna                        3.4
igraph                      0.10.6
importlib_metadata          NA
importlib_resources         NA
ipykernel                   6.25.1
isoduration                 NA
jedi                        0.19.0
jinja2                      3.1.2
joblib                      1.3.2
json5                       NA
jsonpointer                 2.0
jsonschema                  4.19.0
jsonschema_specifications   NA
jupyter_events              0.7.0
jupyter_server              2.7.1
jupyterlab_server           2.24.0
kiwisolver                  1.4.5
leidenalg                   0.10.1
llvmlite                    0.40.1
markupsafe                  2.1.3
natsort                     8.4.0
nbformat                    5.9.2
numba                       0.57.1
overrides                   NA
packaging                   23.1
parso                       0.8.3
patsy                       0.5.3
pexpect                     4.8.0
pickleshare                 0.7.5
pkg_resources               NA
plotly                      5.16.1
prometheus_client           NA
prompt_toolkit              3.0.39
psutil                      5.9.5
ptyprocess                  0.7.0
pure_eval                   0.2.2
pydev_ipython               NA
pydevconsole                NA
pydevd                      2.9.5
pydevd_file_utils           NA
pydevd_plugins              NA
pydevd_tracing              NA
pygments                    2.16.1
pyparsing                   3.0.9
pythonjsonlogger            NA
pytz                        2023.3
referencing                 NA
requests                    2.31.0
rfc3339_validator           0.1.4
rfc3986_validator           0.1.1
rpds                        NA
ruamel                      NA
send2trash                  NA
six                         1.16.0
sklearn                     1.3.0
sniffio                     1.3.0
socks                       1.7.1
sphinxcontrib               NA
stack_data                  0.6.2
statsmodels                 0.14.0
texttable                   1.6.7
threadpoolctl               3.2.0
tornado                     6.3.3
traitlets                   5.9.0
typing_extensions           NA
uri_template                NA
urllib3                     2.0.4
wcwidth                     0.2.6
webcolors                   1.13
websocket                   1.6.2
yaml                        6.0.1
zipp                        NA
zmq                         25.1.1
zstandard                   0.19.0
```

Example 5 (unknown):
```unknown
-----
IPython             8.12.0
jupyter_client      8.3.0
jupyter_core        4.12.0
jupyterlab          4.0.5
notebook            7.0.2
-----
Python 3.8.17 (default, Jul  5 2023, 21:04:15) [GCC 11.2.0]
Linux-5.4.0-171-generic-x86_64-with-glibc2.17
-----
Session information updated at 2024-03-13 13:18
```

Example 6 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 7 (python):
```python
sc.settings.verbosity = 1          # verbosity: errors (0), warnings (1), info (2), hints (3)
sc.settings.set_figure_params(dpi=80,dpi_save=300)
```

Example 8 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk'
```

Example 9 (python):
```python
import matplotlib.colors as mcolors
from matplotlib.cm import ScalarMappable
from matplotlib.lines import Line2D

def plot_cell2loc_out(
    adata_c2l,
    celltype,
    regions_sel,
    xaxis,
    yaxis,
    abundance_thresh = 0.05,
    mean_abundance_max = 0.3,
    count_log_max=7,
    nspots_for_legend = [0, 50, 100],
    savefig = None
):
    # select spots which have the celltype of interest
    # define based on an abundance threshold
    mask = adata_c2l.to_df()[celltype]>abundance_thresh
    ad = adata_c2l[mask]
    print(f'n_spot which contains {celltype}: {ad.shape[0]}')

    # preparing dataframe
    plot_df = ad.to_df()[[celltype]]
    plot_df[[yaxis,xaxis]] = ad.obs[[yaxis,xaxis]].copy()
    plot_df['count'] = 1
    plot_df = plot_df.groupby([yaxis,xaxis]).sum().loc[regions_sel]
    plot_df['mean'] = plot_df[celltype]/plot_df['count']
    plot_df['count_log'] = np.log1p(plot_df['count'])

    # plotting
    plt.rcParams['axes.grid'] = False
    plt.rcParams['figure.figsize'] = 1.2,4
    norm = mcolors.Normalize(vmin=0, vmax=mean_abundance_max)
    sns.scatterplot(data=plot_df, x=xaxis,y=yaxis,
                    hue='mean', hue_norm=norm,
                    size='count_log', size_norm=(0,count_log_max),sizes=(0, 200),
                   palette='Reds',edgecolor='black')

    # Remove the "mean" legend
    plt.legend([],[], frameon=False)

    # Custom legend for sizes - example
    # Define a few representative sizes and their corresponding original count values
    # sizes = nspots_for_legend
    log_sizes = np.log1p(nspots_for_legend)  # Calculate log sizes for positioning in the plot (optional if you want exact matches)
    # Calculate markersizes based on your specific transformation and plotting parameters
    # This step involves converting log_sizes into the scale used in your plot
    markersizes = np.interp(log_sizes, [0, count_log_max], [0, 200])  # Adjust the range [0, 200] as per your plot
    # Create custom legend handles for sizes
    size_handles = [Line2D([0], [0], marker='o', color='w', markerfacecolor='gray', 
                       markersize=np.sqrt(markersize), label=f'{size}') for size, markersize in zip(nspots_for_legend, markersizes)]

    # Add the custom size legend to the plot
    plt.legend(handles=size_handles, title="n_spot", bbox_to_anchor=(2.5, 1), loc=2, borderaxespad=0.)

    # Show the colorbar with the normalized hue scale
    cbar = plt.colorbar(ScalarMappable(norm=norm, cmap="Reds"))
    cbar.set_label('Mean abundance', rotation=270, labelpad=15)
    plt.xticks(rotation=90)
    plt.margins(x=0.3,y=0.07)
    plt.title(celltype)
    if savefig!=None:
        plt.savefig(savefig, bbox_inches="tight",dpi=300)
    plt.show()
```

Example 10 (python):
```python
adata_vis = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/visium_foetal_healthy-all_raw.h5ad')
adata_vis.var_names_make_unique()

# log-transform
sc.pp.normalize_total(adata_vis, target_sum=1e4)
sc.pp.log1p(adata_vis)

adata_vis
```

Example 11 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/anndata/_core/anndata.py:1840: UserWarning: Variable names are not unique. To make them unique, call `.var_names_make_unique`.
  utils.warn_names_duplicates("var")
```

Example 12 (unknown):
```unknown
AnnData object with n_obs × n_vars = 39223 × 33562
    obs: 'array_row', 'array_col', 'sample', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'mt_frac', 'sangerID', 'Publication', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'status', 'cell2loc-map_group', 'JC_edit', 'sample_for_cell2loc', 'week', 'leiden', 'region_cluster', 'AtrialCardiomyocytesLeft', 'AtrialCardiomyocytesRight', 'VentricularCardiomyocytesLeftCompact', 'VentricularCardiomyocytesRightCompact', 'VentricularCardiomyocytesLeftTrabeculated', 'VentricularCardiomyocytesRightTrabeculated', 'SinoatrialNodeCardiomyocytes', 'AtrioventricularNodeCardiomyocytes', 'VentricularConductionSystemProximal', 'VentricularConductionSystemDistal', 'GreatVesselAdventitialFibroblasts', 'CoronaryVesselAdventitialFibroblasts', 'MyocardialInterstitialFibroblasts1', 'MyocardialInterstitialFibroblasts2', 'Myofibroblasts', 'LymphNodeFibroblasticReticularCells', 'ValveInterstitialCells', 'GreatVesselSmoothMuscleCells', 'CoronarySmoothMuscleCells', 'CoronaryPericytes', 'GreatVesselArterialEndothelialCells', 'GreatVesselVenousEndothelialCells', 'CoronaryArterialEndothelialCells', 'CoronaryVenousEndothelialCells', 'CoronaryCapillaryEndothelialCells', 'EndocardialCells', 'EndocardialCushionCells', 'ValveEndothelialCells', 'LymphaticEndothelialCells', 'MesothelialEpicardialCells', 'EpicardiumDerivedCells', 'NeuronPrecursors', 'ChromaffinCells', 'SympatheticNeurons', 'ParaympatheticNeurons', 'SchwannCellPrecursors', 'SchwannCells', 'MonocytesMPOpos', 'Monocytes', 'MonocyteDerivedCells', 'MacrophagesCX3CR1pos', 'MacrophagesTIMD4pos', 'MacrophagesLYVE1pos', 'MacrophagesATF3pos', 'DendriticCellsType1', 'DendriticCellsMature', 'PlasmacytoidDendriticCells', 'MastCells', 'Megakaryocytes', 'TCellsCD4pos', 'TCellsCD8pos', 'TregsCD4pos', 'ProBCells', 'BCells', 'BCellsMS4A1pos', 'NaturalKillerCells', 'InnateLymphoidCells', 'DuctusArteriosusSmoothMuscleCells', 'AtrialCardiomyocytesLeft_minmax', 'AtrialCardiomyocytesRight_minmax', 'VentricularCardiomyocytesLeftCompact_minmax', 'VentricularCardiomyocytesRightCompact_minmax', 'VentricularCardiomyocytesLeftTrabeculated_minmax', 'VentricularCardiomyocytesRightTrabeculated_minmax', 'SinoatrialNodeCardiomyocytes_minmax', 'AtrioventricularNodeCardiomyocytes_minmax', 'VentricularConductionSystemProximal_minmax', 'VentricularConductionSystemDistal_minmax', 'GreatVesselAdventitialFibroblasts_minmax', 'CoronaryVesselAdventitialFibroblasts_minmax', 'MyocardialInterstitialFibroblasts1_minmax', 'MyocardialInterstitialFibroblasts2_minmax', 'Myofibroblasts_minmax', 'LymphNodeFibroblasticReticularCells_minmax', 'ValveInterstitialCells_minmax', 'GreatVesselSmoothMuscleCells_minmax', 'CoronarySmoothMuscleCells_minmax', 'CoronaryPericytes_minmax', 'GreatVesselArterialEndothelialCells_minmax', 'GreatVesselVenousEndothelialCells_minmax', 'CoronaryArterialEndothelialCells_minmax', 'CoronaryVenousEndothelialCells_minmax', 'CoronaryCapillaryEndothelialCells_minmax', 'EndocardialCells_minmax', 'EndocardialCushionCells_minmax', 'ValveEndothelialCells_minmax', 'LymphaticEndothelialCells_minmax', 'MesothelialEpicardialCells_minmax', 'EpicardiumDerivedCells_minmax', 'NeuronPrecursors_minmax', 'ChromaffinCells_minmax', 'SympatheticNeurons_minmax', 'ParaympatheticNeurons_minmax', 'SchwannCellPrecursors_minmax', 'SchwannCells_minmax', 'MonocytesMPOpos_minmax', 'Monocytes_minmax', 'MonocyteDerivedCells_minmax', 'MacrophagesCX3CR1pos_minmax', 'MacrophagesTIMD4pos_minmax', 'MacrophagesLYVE1pos_minmax', 'MacrophagesATF3pos_minmax', 'DendriticCellsType1_minmax', 'DendriticCellsMature_minmax', 'PlasmacytoidDendriticCells_minmax', 'MastCells_minmax', 'Megakaryocytes_minmax', 'TCellsCD4pos_minmax', 'TCellsCD8pos_minmax', 'TregsCD4pos_minmax', 'ProBCells_minmax', 'BCells_minmax', 'BCellsMS4A1pos_minmax', 'NaturalKillerCells_minmax', 'InnateLymphoidCells_minmax', 'DuctusArteriosusSmoothMuscleCells_minmax', 'annotation', 'annotation_final', 'annotation_final_coarse'
    var: 'gene_ids'
    uns: 'JC_edit_colors', 'annotation_colors', 'annotation_final_colors', 'cell2loc-map_group_colors', 'leiden', 'neighbors', 'region_cluster_colors', 'sample_for_cell2loc_colors', 'umap', 'log1p'
    obsm: 'MT', 'X_umap', 'means_cell_abundance_w_sf', 'q05_cell_abundance_w_sf', 'q05_cell_abundance_w_sf_ndarray', 'q95_cell_abundance_w_sf', 'spatial', 'stds_cell_abundance_w_sf'
    obsp: 'connectivities', 'distances'
```

Example 13 (python):
```python
# modify annotations for matching with factor analysis
annotation_edit_dict = {
   'AV valves':'Valves',
    'Aortic valve':'Valves',
    'Valve apparatus':'Valves',
    'Atrium - Left':'Atrium',
     'Atrium - Right':'Atrium',
    'Ventricle - Compact - Left':'Ventricle - Compact',
    'Ventricle - Compact - Right':'Ventricle - Compact',
    'Ventricle - Trabeculated - Left':'Ventricle - Trabeculated',
    'Ventricle - Trabeculated - Right':'Ventricle - Trabeculated',
}
adata_vis.obs['annotation_final_mod']=adata_vis.obs['annotation_final'].astype('str').copy()
adata_vis.obs.replace({'annotation_final_mod':annotation_edit_dict}, 
                      inplace=True)
adata_vis.obs['annotation_final_mod'] = adata_vis.obs['annotation_final_mod'].astype('category')

#reorder annotations
annotations_order = [
   'Atrium',
        'Ventricle - Compact',
     'Ventricle - Trabeculated',
        'SAnode',
        'AVnode',
    'VCS - Proximal',
        'VCS - Distal',
        'Great vessel',
        'Coronary vessel',
    'Ductus arteriosus',
        'Adventitia - Coronary',
     'Adventitia - GV',
    'Epicardium',
        'Endocardium - Atrial', 
    'Endocardium - Ventricular',
        'Endocardial cusion',
        'AV ring',
    'Valves',
    'LN',
            ]
adata_vis.obs['annotation_final_mod'].cat.reorder_categories(annotations_order, inplace=True)
```

Example 14 (python):
```python
pd.crosstab(adata_vis.obs['week'],adata_vis.obs['modality'])
```

Example 15 (python):
```python
pd.crosstab(adata_vis.obs['annotation_final_mod'],adata_vis.obs['modality'])
```

Example 16 (python):
```python
set(adata_vis.obs['week'])
```

Example 17 (unknown):
```unknown
{'13W', '16W', '4W', '5W', '7W'}
```

Example 18 (python):
```python
tissue_merge = {
    # 'AV ring',
    'AVnode':'Node',
    'Adventitia - Coronary':'Coronary tissues',
    'Adventitia - GV':'GV tissues',
    # 'Atrium',
    'Coronary vessel':'Coronary tissues',
    # 'Ductus arteriosus',
     # 'Endocardial cusion',
     'Endocardium - Atrial': 'Endocardium',
     'Endocardium - Ventricular': 'Endocardium',
     # 'Epicardium',
    'Great vessel':'GV tissues',
     # 'LN',
     'SAnode':'Node',
     'VCS - Distal':'VCS',
     'VCS - Proximal':'VCS',
     # 'Valves',
     'Ventricle - Compact': 'Ventricle',
     'Ventricle - Trabeculated': 'Ventricle'
}

week_merge = {
    '4W': '4-5W',
    '5W': '4-5W',
    '7W': '7W',
    '13W': '13-16W',
    '16W': '13-16W', 
}

# merge
adata_vis.obs['annotation_merged'] = adata_vis.obs['annotation_final_mod'].astype('str').copy()
adata_vis.obs.replace({'annotation_merged':tissue_merge},inplace=True)
adata_vis.obs['week_merged'] = adata_vis.obs['week'].astype('str').copy()
adata_vis.obs.replace({'week_merged':week_merge},inplace=True)
adata_vis.obs['week_merged'] = adata_vis.obs['week_merged'].astype('category')
adata_vis.obs['week_merged'].cat.reorder_categories(['4-5W','7W','13-16W'],inplace=True)
```

Example 19 (python):
```python
# get the cell2loc output
cell2loc_out = adata_vis.obsm['q05_cell_abundance_w_sf'].copy()
## read in cell type name update dictionary
celltype_rename = utils.read_pkl('/nfs/team205/heart/anndata_objects/Foetal/fine_grain_rename_dictionary.pkl')
## update cell type name
cellnames = adata_vis.obsm['q05_cell_abundance_w_sf'].columns
cellnames = [x.split('q05cell_abundance_w_sf_')[1] for x in cellnames]
cellnames = [celltype_rename[x] for x in cellnames]

# select obs columns
obs_columns = ['array_row', 'array_col', 'sample', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes',
               'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'mt_frac', 'sangerID', 'Publication', 'combinedID', 'donor', 
               'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei',
               'modality', 'kit_10x', 'flushed', 'status', 'cell2loc-map_group', 'JC_edit', 'sample_for_cell2loc', 'week', 'leiden', 'annotation_final','annotation_final_mod',
              'annotation_merged','week_merged'
              ]

# make anndata
adata_c2l = anndata.AnnData(
    X=adata_vis.obsm['q05_cell_abundance_w_sf_ndarray'].copy(),
    obs=adata_vis.obs[obs_columns],
    var=pd.DataFrame(cellnames).set_index(0),
    obsm=adata_vis.obsm.copy(),
    obsp=adata_vis.obsp.copy(),
    uns=adata_vis.uns.copy()
)
adata_c2l
```

Example 20 (unknown):
```unknown
AnnData object with n_obs × n_vars = 39223 × 58
    obs: 'array_row', 'array_col', 'sample', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'mt_frac', 'sangerID', 'Publication', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'status', 'cell2loc-map_group', 'JC_edit', 'sample_for_cell2loc', 'week', 'leiden', 'annotation_final', 'annotation_final_mod', 'annotation_merged', 'week_merged'
    uns: 'JC_edit_colors', 'annotation_colors', 'annotation_final_colors', 'cell2loc-map_group_colors', 'leiden', 'neighbors', 'region_cluster_colors', 'sample_for_cell2loc_colors', 'umap', 'log1p'
    obsm: 'MT', 'X_umap', 'means_cell_abundance_w_sf', 'q05_cell_abundance_w_sf', 'q05_cell_abundance_w_sf_ndarray', 'q95_cell_abundance_w_sf', 'spatial', 'stds_cell_abundance_w_sf'
    obsp: 'connectivities', 'distances'
```

Example 21 (python):
```python
# select tissues
df = pd.crosstab(adata_c2l.obs['annotation_merged'],adata_c2l.obs['week_merged'])
regions_sel = list(df.index[(df>0).sum(axis=1)==3])
regions_sel
```

Example 22 (unknown):
```unknown
['Atrium',
 'Coronary tissues',
 'Endocardial cusion',
 'Endocardium',
 'Epicardium',
 'GV tissues',
 'Node',
 'VCS',
 'Ventricle']
```

Example 23 (python):
```python
adata_c2l.to_df()['ParasympatheticNeurons'].hist(bins=1000)
plt.yscale('log')
plt.xlim(0,0.5)
```

Example 24 (python):
```python
adata_c2l.to_df()['SympatheticNeurons'].hist(bins=1000)
plt.yscale('log')
plt.xlim(0,0.5)
```

Example 25 (python):
```python
celltype = 'ParasympatheticNeurons'
plot_cell2loc_out(
    adata_c2l,
    celltype=celltype,
    regions_sel=regions_sel,
    xaxis='week_merged',
    yaxis='annotation_merged',
    abundance_thresh = 0.05,
    mean_abundance_max = 0.3,
    count_log_max=7,
    nspots_for_legend = [0, 10, 100],
    savefig = f"{figdir}/visium_dotplot_{celltype}.pdf"
)
```

Example 26 (unknown):
```unknown
n_spot which contains ParasympatheticNeurons: 2850
```

Example 27 (python):
```python
/tmp/ipykernel_152303/533952583.py:58: MatplotlibDeprecationWarning: Unable to determine Axes to steal space for Colorbar. Using gca(), but will raise in the future. Either provide the *cax* argument to use as the Axes for the Colorbar, provide the *ax* argument to steal space from it, or add *mappable* to an Axes.
  cbar = plt.colorbar(ScalarMappable(norm=norm, cmap="Reds"))
```

Example 28 (python):
```python
celltype = 'SympatheticNeurons'
plot_cell2loc_out(
    adata_c2l,
    celltype=celltype,
    regions_sel=regions_sel,
    xaxis='week_merged',
    yaxis='annotation_merged',
    abundance_thresh = 0.05,
    mean_abundance_max = 0.3,
    count_log_max=7,
    nspots_for_legend = [0, 10, 100],
    savefig = f"{figdir}/visium_dotplot_{celltype}.pdf"
)
```

Example 29 (unknown):
```unknown
n_spot which contains SympatheticNeurons: 532
```

Example 30 (python):
```python
/tmp/ipykernel_152303/533952583.py:58: MatplotlibDeprecationWarning: Unable to determine Axes to steal space for Colorbar. Using gca(), but will raise in the future. Either provide the *cax* argument to use as the Axes for the Colorbar, provide the *ax* argument to steal space from it, or add *mappable* to an Axes.
  cbar = plt.colorbar(ScalarMappable(norm=norm, cmap="Reds"))
```

Example 31 (python):
```python
# read in sigle-cell data
adata = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/Feb28ObjectRaw_finegrain_updated.h5ad')

# subset neural compartment
adata = adata[adata.obs['coarse_grain']=='Neural']

# filtering genes
sc.pp.filter_genes(adata, min_cells=3)

# log-transform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

adata
```

Example 32 (python):
```python
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/preprocessing/_simple.py:251: ImplicitModificationWarning: Trying to modify attribute `.var` of view, initializing view as actual.
  adata.var['n_cells'] = number
```

Example 33 (unknown):
```unknown
AnnData object with n_obs × n_vars = 6149 × 26660
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score'
    var: 'gene_name_scRNA-0', 'gene_id', 'n_cells'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'age_colors', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'cycling_colors', 'donor_colors', 'fine_grain_colors', 'heart_or_greatvessels_colors', 'hvg', 'kit_10x_colors', 'modality_colors', 'neighbors', 'phase_colors', 'region_colors', 'sex_colors', 'trimester_colors', 'umap', 'week_colors', 'log1p'
    obsm: 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 34 (python):
```python
set(adata.obs.region)
```

Example 35 (unknown):
```unknown
{'aorta',
 'apex',
 'atria',
 'atrial septum',
 'base',
 'ductus arteriosus',
 'heart without node',
 'left ventricle',
 'node',
 'outflow tract',
 'pericardium',
 'pulmonary arches and branches',
 'right ventricle',
 'right ventricle and papillaries',
 'whole sample'}
```

Example 36 (python):
```python
# merge regions
region_merge = {
    # 'aorta',
 # 'apex',
 'atria': 'atrium',
 'atrial septum': 'atrium',
 # 'base',
 # 'ductus arteriosus',
 # 'heart without node',
 # 'left ventricle',
 # 'node',
 # 'outflow tract',
 # 'pericardium',
 # 'pulmonary arches and branches',
 'right ventricle': 'right ventricle',
 'right ventricle and papillaries': 'right ventricle',
 # 'whole sample'
}

# merge
adata.obs['region_merged'] = adata.obs['region'].astype('str').copy()
adata.obs.replace({'region_merged':region_merge},inplace=True)
```

Example 37 (python):
```python
# subset
# nuclei
adata_sub = adata[adata.obs['cell_or_nuclei']=='nuclei']
# non whole heart data
adata_sub = adata_sub[adata_sub.obs['region']!='whole sample']
adata_sub = adata_sub[adata_sub.obs['region']!='heart without node']
display(pd.crosstab(adata_sub.obs['fine_grain'],adata_sub.obs['week']))
display(pd.crosstab(adata_sub.obs['fine_grain'],adata_sub.obs['region']))
```

Example 38 (python):
```python
# remove 4W since it's tiny number of nuclei
adata_sub = adata_sub[adata_sub.obs['week']!='4W']
display(pd.crosstab(adata_sub.obs['fine_grain'],adata_sub.obs['week']))
```

Example 39 (python):
```python
# select by celltype
ad = adata_sub[adata_sub.obs['fine_grain'].isin(['ParasympatheticNeurons','SympatheticNeurons'])]
# ad = adata_sub.copy()
print(ad.shape)

# preparing dataframe
plot_df = ad.obs[['region_merged','fine_grain']].copy()
plot_df['count'] = 1
plot_df = plot_df.groupby(['region_merged','fine_grain']).sum()
# add missing regions as count = 0
missing_regions = set(adata_sub.obs.region_merged) - set(ad.obs.region_merged)
for reg in missing_regions:
    plot_df.loc[(reg,'SympatheticNeurons'),'count']=0
    plot_df.loc[(reg,'ParasympatheticNeurons'),'count']=0
# plot_df['mean'] = plot_df[celltype]/plot_df['count']
plot_df['count_log'] = np.log1p(plot_df['count'])

# plotting
plt.rcParams['axes.grid'] = False
plt.rcParams['figure.figsize'] = 1,3
vmax = 50
norm = mcolors.Normalize(vmin=0, vmax=vmax)
sns.scatterplot(data=plot_df,x='fine_grain',y='region_merged',edgecolor='black',
                hue='count', size='count', palette='Blues',size_norm=(0,vmax),sizes=(0, 200),hue_norm=norm,
                legend=False
               )
# Show the colorbar with the normalized hue scale
plt.colorbar(ScalarMappable(norm=norm, cmap="Blues"))
# plt.legend(loc=None, bbox_to_anchor=(4, 0.5))
plt.xticks(rotation=90)
plt.margins(x=0.6,y=0.1)
plt.title('Nuclei number')
plt.savefig(f"{figdir}/snRNAseq_dotplot_PSNS-SNS.pdf", bbox_inches="tight",dpi=300)
plt.show()
```

Example 40 (unknown):
```unknown
(181, 26660)
```

Example 41 (python):
```python
/tmp/ipykernel_152303/2641053010.py:28: MatplotlibDeprecationWarning: Unable to determine Axes to steal space for Colorbar. Using gca(), but will raise in the future. Either provide the *cax* argument to use as the Axes for the Colorbar, provide the *ax* argument to steal space from it, or add *mappable* to an Axes.
  plt.colorbar(ScalarMappable(norm=norm, cmap="Blues"))
```

---

## Read in¶

**URL:** http://127.0.0.1:9020/notebooks_html/Supplementary_materials/Macrophages/Mo-MP_no4_Milo_age-enrichment.html

**Contents:**
- Read in¶
- Define neighbourhoods¶
- Differential abundance testing with GLM¶

322 rows × 12 columns

**Examples:**

Example 1 (python):
```python
import scanpy as sc
import pandas as pd
import numpy as np
import anndata
import scipy
import matplotlib.pyplot as plt
import seaborn as sns

import milopy.core as milo
import milopy.plot as milopl
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
-----
anndata             0.8.0
matplotlib          3.5.2
milopy              0.1.1
numpy               1.21.5
pandas              1.3.5
scanpy              1.8.2
scipy               1.7.3
seaborn             0.11.1
session_info        1.0.0
-----
```

Example 4 (python):
```python
PIL                 9.1.1
asciitree           NA
backcall            0.2.0
beta_ufunc          NA
binom_ufunc         NA
cached_property     1.5.1
cairo               1.20.0
cffi                1.14.4
cloudpickle         1.6.0
colorama            0.4.4
cycler              0.10.0
cython_runtime      NA
cytoolz             0.11.2
dask                2022.01.0
dateutil            2.8.1
decorator           4.4.2
defusedxml          0.6.0
fasteners           0.17.3
fsspec              2022.01.0
google              NA
h5py                3.1.0
igraph              0.10.2
importlib_metadata  1.7.0
ipykernel           5.4.2
ipython_genutils    0.2.0
ipywidgets          7.6.3
jedi                0.17.2
jinja2              3.1.2
joblib              1.0.0
kiwisolver          1.3.1
leidenalg           0.9.0
llvmlite            0.35.0
louvain             0.7.0
markupsafe          2.1.1
mpl_toolkits        NA
msgpack             1.0.3
natsort             7.1.0
nbinom_ufunc        NA
numba               0.52.0
numcodecs           0.9.1
numexpr             2.7.2
packaging           21.3
parso               0.7.1
pexpect             4.8.0
pickleshare         0.7.5
pkg_resources       NA
plotly              5.10.0
prompt_toolkit      3.0.10
psutil              5.9.0
ptyprocess          0.7.0
pyarrow             9.0.0
pygments            2.13.0
pyparsing           2.4.7
pytz                2020.5
rpy2                3.4.2
setuptools_scm      NA
simplejson          3.19.1
sinfo               0.3.1
six                 1.14.0
sklearn             1.0.2
sphinxcontrib       NA
statsmodels         0.12.1
storemagic          NA
tables              3.6.1
tblib               1.7.0
texttable           1.6.3
threadpoolctl       2.1.0
tlz                 0.11.2
toolz               0.11.2
tornado             6.1
traitlets           5.0.5
typing_extensions   NA
tzlocal             NA
wcwidth             0.2.5
yaml                5.4.1
zarr                2.11.3
zipp                NA
zmq                 21.0.1
zope                NA
```

Example 5 (unknown):
```unknown
-----
IPython             7.19.0
jupyter_client      6.1.11
jupyter_core        4.7.0
notebook            6.2.0
-----
Python 3.7.9 | packaged by conda-forge | (default, Dec  9 2020, 21:08:20) [GCC 9.3.0]
Linux-4.15.0-213-generic-x86_64-with-debian-bullseye-sid
-----
Session information updated at 2023-11-18 20:24
```

Example 6 (python):
```python
## r2py setup
import anndata2ri
import rpy2.rinterface_lib.callbacks
import logging
rpy2.rinterface_lib.callbacks.logger.setLevel(logging.ERROR)

anndata2ri.activate()
```

Example 7 (python):
```python
sc.settings.set_figure_params(dpi=200)
```

Example 8 (python):
```python
adata = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/Immune/MoMP_Aug2023/MultiVI-post_foetal_Mono-MP_withContCov-CCCov_13Mar2023_feature-select-0003.h5ad')

# modify obs names
adata.obs_names = [x.replace('_paired','').replace('_expression','') for x in adata.obs_names]

# subset only macrophages
adata = adata[adata.obs['fine_grain_mod'].isin(['MacrophagesATF3pos',
                                                         'MacrophagesCX3CR1pos',
                                                         'MacrophagesLYVE1pos',
                                                         'MacrophagesTIMD4pos',
                                                         'MacrophagesYSderived',])]
adata
```

Example 9 (python):
```python
View of AnnData object with n_obs × n_vars = 6777 × 142283
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', '_indices', '_scvi_batch', '_scvi_labels', 'fine_grain_withcycling', 't', 'seg', 'edge', 't_sd', 'milestones', 'fine_grain_mod'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id', 'n_cells'
    uns: '_scvi_manager_uuid', '_scvi_uuid', 'adata_auc_gene', 'adata_auc_region', 'adata_gex', 'adata_gex_lognorm', 'adata_peak_tfidf', 'draw_graph', 'fine_grain_colors', 'fine_grain_mod_colors', 'fine_grain_withcycling_colors', 'graph', 'milestones_colors', 'neighbors', 'ppt', 'pseudotime_list', 'seg_colors', 'umap', 'week_colors'
    obsm: 'MultiVI_latent', 'X_R', 'X_draw_graph_fa', 'X_palantir', 'X_umap', 'X_umap13', '_scvi_extra_categorical_covs', '_scvi_extra_continuous_covs'
    obsp: 'connectivities', 'distances'
```

Example 10 (python):
```python
adata.X.data[:10]
```

Example 11 (unknown):
```unknown
array([1., 1., 1., 2., 1., 1., 1., 2., 3., 1.], dtype=float32)
```

Example 12 (python):
```python
sc.pl.draw_graph(adata,color='fine_grain_mod', frameon=False)
```

Example 13 (python):
```python
sc.pl.draw_graph(adata,color='kit_10x', frameon=False)
```

Example 14 (unknown):
```unknown
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
```

Example 15 (python):
```python
%%time
## Recompute KNN graph
sc.pp.neighbors(adata, use_rep = "MultiVI_latent", n_neighbors = 50, key_added="multivi")

## Construct neighbourhoods
milo.make_nhoods(adata, neighbors_key="multivi", prop=0.05)

## Count cells in neighbourhoods
milo.count_nhoods(adata, sample_col="sangerID")
```

Example 16 (unknown):
```unknown
CPU times: user 3.43 s, sys: 7.29 s, total: 10.7 s
Wall time: 2.3 s
```

Example 17 (unknown):
```unknown
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/anndata/_core/anndata.py:121: ImplicitModificationWarning: Transforming to str index.
  warnings.warn("Transforming to str index.", ImplicitModificationWarning)
```

Example 18 (python):
```python
sns.set_context("talk")
plt.rcParams["figure.figsize"] = [8,6]
plt.hist(np.array(adata.obsm['nhoods'].sum(0)).ravel(), bins=50);
plt.xlabel('# cells in neighbourhood'); plt.ylabel('# neighbourhoods');
# plt.savefig(figdir+'nhood_size_hist.pdf',bbox_inches='tight');
```

Example 19 (python):
```python
adata.uns["nhood_adata"]
```

Example 20 (unknown):
```unknown
AnnData object with n_obs × n_vars = 322 × 21
    obs: 'index_cell', 'kth_distance'
    uns: 'sample_col'
```

Example 21 (python):
```python
adata.obs["week_continuous"] = [int(x.split('W')[0]) for x in adata.obs["week"]] #rather than adata.obs["week"].cat.codes
pd.crosstab(adata.obs['week_continuous'],adata.obs['week'])
```

Example 22 (python):
```python
milo.DA_nhoods(adata, design="~ week_continuous")
```

Example 23 (python):
```python
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/rpy2/robjects/vectors.py:980: UserWarning: R object inheriting from "POSIXct" but without attribute "tzone".
  warnings.warn('R object inheriting from "POSIXct" but without '
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/milopy/core.py:285: FutureWarning: In a future version of pandas all arguments of concat except for the argument 'objs' will be keyword-only
  nhood_adata.obs = pd.concat([nhood_adata.obs, res], 1)
```

Example 24 (python):
```python
import milopy.utils
milopy.utils.build_nhood_graph(adata,basis='X_draw_graph_fa')
```

Example 25 (python):
```python
plt.rcParams["figure.figsize"] = [5,5]
milopl.plot_nhood_graph(adata, 
                        alpha=0.1, ## SpatialFDR level (1%) 
                        min_size=3, ## Size of smallest dot
                        title = 'DA log-Fold Change\n(FDR < 0.1)'
                       )
```

Example 26 (python):
```python
milopy.utils.annotate_nhoods(adata, anno_col='fine_grain_mod')
```

Example 27 (python):
```python
da_results = adata.uns['nhood_adata'].obs.copy()
# drop nan
da_results.dropna(axis=0,inplace=True)
# reindex
da_results.index = list(range(len(da_results)))
# add 'Nhood' column
da_results['Nhood'] = da_results.index.astype('int')+1 
da_results['Nhood'] = da_results['Nhood'].astype('str')

da_results
```

Example 28 (python):
```python
plt.hist(adata.uns['nhood_adata'].obs["nhood_annotation_frac"]);
plt.xlabel("celltype fraction")
```

Example 29 (unknown):
```unknown
Text(0.5, 0, 'celltype fraction')
```

Example 30 (unknown):
```unknown
%load_ext rpy2.ipython
```

Example 31 (unknown):
```unknown
The rpy2.ipython extension is already loaded. To reload it, use:
  %reload_ext rpy2.ipython
```

Example 32 (javascript):
```javascript
%%R
library(miloR)
library(tidyverse)
library(ggbeeswarm)

# modifying color scale
plotDAbeeswarm_mod <- function(da.res, group.by=NULL, alpha=0.1, subset.nhoods=NULL){
  if (!is.null(group.by)) {
    if (!group.by %in% colnames(da.res)) {
      stop(group.by, " is not a column in da.res. Have you forgot to run annotateNhoods(x, da.res, ", group.by,")?")
    }
    if (is.numeric(da.res[,group.by])) {
      # stop(group.by, " is a numeric variable. Please bin to use for grouping.")
    }
    da.res <- mutate(da.res, group_by = da.res[,group.by])
  } else {
    da.res <- mutate(da.res, group_by = "g1")
  }

  if (!is.factor(da.res[,"group_by"])) {
    message("Converting group_by to factor...")
    da.res <- mutate(da.res, group_by = factor(group_by, levels=unique(group_by)))
    # anno_vec <- factor(anno_vec, levels=unique(anno_vec))
  }

  if (!is.null(subset.nhoods)) {
    da.res <- da.res[subset.nhoods,]
  }
  
  # Get position with ggbeeswarm
  beeswarm_pos <- ggplot_build(
    da.res %>%
      mutate(is_signif = ifelse(SpatialFDR < alpha, 1, 0)) %>%
      arrange(group_by) %>%
      ggplot(aes(group_by, logFC)) +
      geom_quasirandom()
  )
  
  pos_x <- beeswarm_pos$data[[1]]$x
  pos_y <- beeswarm_pos$data[[1]]$y
  
  n_groups <- unique(da.res$group_by) %>% length()
  
  da.res %>%
    mutate(is_signif = ifelse(SpatialFDR < alpha, 1, 0)) %>%
    mutate(logFC_color = ifelse(is_signif==1, logFC, NA)) %>%
    arrange(group_by) %>%
    mutate(Nhood=factor(Nhood, levels=unique(Nhood))) %>%
    mutate(pos_x = pos_x, pos_y=pos_y) %>%
    ggplot(aes(pos_x, pos_y, color=logFC_color)) +
    scale_color_gradient2(low = "blue",high = "red") +
    guides(color="none") +
    xlab(group.by) + ylab("Log Fold Change") +
    scale_x_continuous(
      breaks = seq(1,n_groups),
      labels = setNames(levels(da.res$group_by), seq(1,n_groups))
      ) +
    geom_point() +
    coord_flip() +
    theme_bw(base_size=22) +
    theme(strip.text.y =  element_text(angle=0))

}
```

Example 33 (unknown):
```unknown
%%R
head(da_results)
```

Example 34 (unknown):
```unknown
index_cell kth_distance
1 BHF_F_Hea11933670_BHF_F_Hea11596623_GATTGATGTTGAGCCG-1     1.456107
2 BHF_F_Hea11933668_BHF_F_Hea11596621_GCCAGGTTCACCGGTA-1     2.753402
3  7089STDY13216927_BHF_F_Hea13242534_CACCTGTTCCCGAAGC-1     1.462260
4 BHF_F_Hea11933667_BHF_F_Hea11596620_GTTGGAGCAACAGGAT-1     1.732683
5 BHF_F_Hea11933668_BHF_F_Hea11596621_TCGCGAGGTGTTTCAC-1     1.725373
6 BHF_F_Hea11933667_BHF_F_Hea11596620_GTGAGGAGTTGTTCAC-1     1.519573
       logFC   logCPM         F       PValue          FDR   SpatialFDR
1 -0.4854753 13.40691  6.165901 1.305019e-02 7.372215e-02 7.966068e-02
2  1.1289741 13.87388 35.805067 2.304259e-09 2.473238e-07 4.691488e-07
3  0.3866579 13.73443  4.126973 4.224791e-02 1.581840e-01 1.639379e-01
4  0.7250017 13.79357 17.435932 3.012828e-05 6.929505e-04 9.498297e-04
5  0.2479639 13.22380  1.731931 1.882138e-01 3.987162e-01 4.042951e-01
6  0.3353341 13.20775  2.624008 1.053098e-01 2.969872e-01 3.030582e-01
  Nhood_size    nhood_annotation nhood_annotation_frac Nhood
1         65 MacrophagesLYVE1pos             0.9076923     1
2         77 MacrophagesLYVE1pos             0.6103896     2
3         73 MacrophagesLYVE1pos             0.9726027     3
4         89 MacrophagesLYVE1pos             0.6516854     4
5         76 MacrophagesLYVE1pos             0.9473684     5
6         75 MacrophagesLYVE1pos             0.8800000     6
```

Example 35 (unknown):
```unknown
%%R -i da_results -w 600 -h 170
# da_results = da_results %>% filter(nhood_annotation %in% c(Monocytes))
plotDAbeeswarm_mod(da_results, group.by = "nhood_annotation")
```

---

## Import modules¶

**URL:** http://127.0.0.1:9020/notebooks_html/Supplementary_materials/Macrophages/CellTypist_predict.html

**Contents:**
- Import modules¶
- Read in adata¶
- Subset macrophages¶
- CellTypist prediction¶
- Plot probability on umap¶
- Annotate YS-derived macrophage population¶
- Plot microglia marker genes¶

**Examples:**

Example 1 (python):
```python
import os
import numpy as np
import scanpy as sc
import anndata
import pandas as pd
import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42
import matplotlib.pyplot as plt
import seaborn as sns

import celltypist
from celltypist import models

import warnings
warnings.simplefilter(action='ignore', category=FutureWarning)
warnings.simplefilter("ignore", UserWarning)
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
-----
anndata             0.9.2
celltypist          1.6.2
matplotlib          3.7.2
numpy               1.24.4
pandas              1.5.3
scanpy              1.9.3
seaborn             0.12.2
session_info        1.0.0
-----
```

Example 4 (python):
```python
PIL                         10.0.0
anyio                       NA
arrow                       1.2.3
asttokens                   NA
attr                        23.1.0
attrs                       23.1.0
babel                       2.12.1
backcall                    0.2.0
brotli                      1.0.9
certifi                     2023.11.17
cffi                        1.15.1
charset_normalizer          3.2.0
colorama                    0.4.6
comm                        0.1.4
cycler                      0.10.0
cython_runtime              NA
dateutil                    2.8.2
debugpy                     1.6.8
decorator                   5.1.1
defusedxml                  0.7.1
executing                   1.2.0
fastjsonschema              NA
fqdn                        NA
h5py                        3.9.0
idna                        3.4
igraph                      0.10.6
importlib_metadata          NA
importlib_resources         NA
ipykernel                   6.25.1
isoduration                 NA
jedi                        0.19.0
jinja2                      3.1.2
joblib                      1.3.2
json5                       NA
jsonpointer                 2.0
jsonschema                  4.19.0
jsonschema_specifications   NA
jupyter_events              0.7.0
jupyter_server              2.7.1
jupyterlab_server           2.24.0
kiwisolver                  1.4.5
leidenalg                   0.10.1
llvmlite                    0.40.1
markupsafe                  2.1.3
matplotlib_inline           0.1.6
mpl_toolkits                NA
natsort                     8.4.0
nbformat                    5.9.2
numba                       0.57.1
overrides                   NA
packaging                   23.1
parso                       0.8.3
patsy                       0.5.3
pexpect                     4.8.0
pickleshare                 0.7.5
pkg_resources               NA
plotly                      5.16.1
prometheus_client           NA
prompt_toolkit              3.0.39
psutil                      5.9.5
ptyprocess                  0.7.0
pure_eval                   0.2.2
pydev_ipython               NA
pydevconsole                NA
pydevd                      2.9.5
pydevd_file_utils           NA
pydevd_plugins              NA
pydevd_tracing              NA
pygments                    2.16.1
pyparsing                   3.0.9
pythonjsonlogger            NA
pytz                        2023.3
referencing                 NA
requests                    2.31.0
rfc3339_validator           0.1.4
rfc3986_validator           0.1.1
rpds                        NA
ruamel                      NA
scipy                       1.10.1
send2trash                  NA
six                         1.16.0
sklearn                     1.3.0
sniffio                     1.3.0
socks                       1.7.1
sphinxcontrib               NA
stack_data                  0.6.2
statsmodels                 0.14.0
texttable                   1.6.7
threadpoolctl               3.2.0
tornado                     6.3.3
traitlets                   5.9.0
typing_extensions           NA
uri_template                NA
urllib3                     2.0.4
wcwidth                     0.2.6
webcolors                   1.13
websocket                   1.6.2
yaml                        6.0.1
zipp                        NA
zmq                         25.1.1
zstandard                   0.19.0
```

Example 5 (unknown):
```unknown
-----
IPython             8.12.0
jupyter_client      8.3.0
jupyter_core        4.12.0
jupyterlab          4.0.5
notebook            7.0.2
-----
Python 3.8.17 (default, Jul  5 2023, 21:04:15) [GCC 11.2.0]
Linux-5.4.0-169-generic-x86_64-with-glibc2.17
-----
Session information updated at 2024-02-05 12:40
```

Example 6 (python):
```python
import importlib.util
import sys

spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)

spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/tissueaxis.py")
tissueaxis = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = tissueaxis
spec.loader.exec_module(tissueaxis)
```

Example 7 (python):
```python
sc.settings.set_figure_params(dpi=80,dpi_save=300)
```

Example 8 (python):
```python
%matplotlib inline
```

Example 9 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk'
```

Example 10 (python):
```python
# read in
adata_dir = '/nfs/team205/heart/anndata_objects/Foetal/Immune/MoMP_Aug2023'
adata = sc.read_h5ad(f'{adata_dir}/MultiVI-post_foetal_Mono-MP_withContCov-CCCov_13Mar2023_feature-select-0003.h5ad')

# get gene expression data
adata_gex = adata.uns['adata_gex_lognorm'].copy()
print(adata_gex.X.data[:5])
adata_gex
```

Example 11 (unknown):
```unknown
[0.9766816 0.9766816 1.7862254 2.7663608 0.9766816]
```

Example 12 (python):
```python
AnnData object with n_obs × n_vars = 7674 × 15922
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', '_indices', '_scvi_batch', '_scvi_labels', 'fine_grain_withcycling', 't', 'seg', 'edge', 't_sd', 'milestones'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id', 'n_cells'
    uns: '_scvi_manager_uuid', '_scvi_uuid', 'adata_auc_gene', 'adata_auc_region', 'adata_gex', 'adata_gex_lognorm', 'draw_graph', 'fine_grain_colors', 'fine_grain_withcycling_colors', 'graph', 'log1p', 'milestones_colors', 'neighbors', 'ppt', 'pseudotime_list', 'seg_colors', 'umap', 'week_colors'
    obsm: 'MultiVI_latent', 'X_R', 'X_draw_graph_fa', 'X_palantir', 'X_umap', 'X_umap13', '_scvi_extra_categorical_covs', '_scvi_extra_continuous_covs'
    obsp: 'connectivities', 'distances'
```

Example 13 (python):
```python
adata_gex.var.head()
```

Example 14 (python):
```python
sc.pl.draw_graph(adata_gex,color=['seg','milestones'],cmap='RdPu',size=30)
```

Example 15 (python):
```python
cells_to_select = ['MacrophagesATF3pos',
 'MacrophagesCX3CR1pos',
 'MacrophagesLYVE1pos',
 'MacrophagesTIMD4pos']

adata_gex = adata_gex[adata_gex.obs['fine_grain'].isin(cells_to_select)]
adata_gex.obs['fine_grain'].value_counts()
```

Example 16 (unknown):
```unknown
MacrophagesLYVE1pos     4939
MacrophagesATF3pos       898
MacrophagesCX3CR1pos     826
MacrophagesTIMD4pos      112
Name: fine_grain, dtype: int64
```

Example 17 (python):
```python
set(adata_gex.obs['kit_10x'])
```

Example 18 (unknown):
```unknown
{'5prime-v2', 'Multiome-v1'}
```

Example 19 (python):
```python
adata_gex.obs['milestones'].value_counts()
```

Example 20 (unknown):
```unknown
47    2762
5     1105
28     872
11     608
32     518
20     340
44     339
29     205
Name: milestones, dtype: int64
```

Example 21 (python):
```python
sc.pl.draw_graph(adata_gex,color=['seg','milestones'],cmap='RdPu',size=30)
```

Example 22 (python):
```python
adata_gex.X.data[:10]
```

Example 23 (unknown):
```unknown
array([1.7789584, 1.7789584, 1.7789584, 2.3839223, 1.7789584, 1.7789584,
       1.7789584, 2.3839223, 2.7581758, 1.7789584], dtype=float32)
```

Example 24 (python):
```python
predictions = celltypist.annotate(adata_gex, 
                                  model = '/nfs/team205/heart/anndata_objects/Foetal/celltypist/model_YS_portal_object_20221208_onlyMP_hvg_18Nov2023.pkl', 
                                  majority_voting = False)

# plot
celltypist.dotplot(predictions, use_as_reference = 'milestones', use_as_prediction = 'predicted_labels', vmax=1, show=False)
plt.savefig(f'{figdir}/MoMP_celltypist-prediction_milestones.pdf', bbox_inches="tight",dpi=300)

celltypist.dotplot(predictions, use_as_reference = 'seg', use_as_prediction = 'predicted_labels', vmax=1)
```

Example 25 (unknown):
```unknown
🔬 Input data has 6749 cells and 15922 genes
🔗 Matching reference genes in the model
🧬 828 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
maxp pruned
cmap pruned
kern dropped
post pruned
FFTM dropped
GPOS pruned
GSUB pruned
glyf pruned
Added gid0 to subset
Added first four glyphs to subset
Closing glyph list over 'GSUB': 46 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'C', 'E', 'F', 'G', 'H', 'I', 'L', 'M', 'O', 'P', 'R', 'T', 'a', 'b', 'c', 'e', 'eight', 'f', 'five', 'four', 'i', 'l', 'n', 'nine', 'nonmarkingreturn', 'o', 'one', 'p', 'parenleft', 'parenright', 'percent', 'period', 'r', 's', 'seven', 'six', 'space', 't', 'three', 'two', 'underscore', 'y', 'zero']
Glyph IDs:   [0, 1, 2, 3, 8, 11, 12, 17, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 36, 38, 40, 41, 42, 43, 44, 47, 48, 50, 51, 53, 55, 66, 68, 69, 70, 72, 73, 76, 79, 81, 82, 83, 85, 86, 87, 92]
Closed glyph list over 'GSUB': 51 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'C', 'E', 'F', 'G', 'H', 'I', 'L', 'M', 'O', 'P', 'R', 'T', 'a', 'b', 'c', 'e', 'eight', 'f', 'fi', 'five', 'fl', 'four', 'i', 'l', 'n', 'nine', 'nonmarkingreturn', 'o', 'one', 'p', 'parenleft', 'parenright', 'percent', 'period', 'r', 's', 'seven', 'six', 'space', 't', 'three', 'two', 'underscore', 'uniFB00', 'uniFB03', 'uniFB04', 'y', 'zero']
Glyph IDs:   [0, 1, 2, 3, 8, 11, 12, 17, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 36, 38, 40, 41, 42, 43, 44, 47, 48, 50, 51, 53, 55, 66, 68, 69, 70, 72, 73, 76, 79, 81, 82, 83, 85, 86, 87, 92, 5038, 5039, 5040, 5041, 5042]
Closing glyph list over 'MATH': 51 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'C', 'E', 'F', 'G', 'H', 'I', 'L', 'M', 'O', 'P', 'R', 'T', 'a', 'b', 'c', 'e', 'eight', 'f', 'fi', 'five', 'fl', 'four', 'i', 'l', 'n', 'nine', 'nonmarkingreturn', 'o', 'one', 'p', 'parenleft', 'parenright', 'percent', 'period', 'r', 's', 'seven', 'six', 'space', 't', 'three', 'two', 'underscore', 'uniFB00', 'uniFB03', 'uniFB04', 'y', 'zero']
Glyph IDs:   [0, 1, 2, 3, 8, 11, 12, 17, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 36, 38, 40, 41, 42, 43, 44, 47, 48, 50, 51, 53, 55, 66, 68, 69, 70, 72, 73, 76, 79, 81, 82, 83, 85, 86, 87, 92, 5038, 5039, 5040, 5041, 5042]
Closed glyph list over 'MATH': 57 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'C', 'E', 'F', 'G', 'H', 'I', 'L', 'M', 'O', 'P', 'R', 'T', 'a', 'b', 'c', 'e', 'eight', 'f', 'fi', 'five', 'fl', 'four', 'i', 'l', 'n', 'nine', 'nonmarkingreturn', 'o', 'one', 'p', 'parenleft', 'parenright', 'percent', 'period', 'r', 's', 'seven', 'six', 'space', 't', 'three', 'two', 'underscore', 'uni239B', 'uni239C', 'uni239D', 'uni239E', 'uni239F', 'uni23A0', 'uniFB00', 'uniFB03', 'uniFB04', 'y', 'zero']
Glyph IDs:   [0, 1, 2, 3, 8, 11, 12, 17, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 36, 38, 40, 41, 42, 43, 44, 47, 48, 50, 51, 53, 55, 66, 68, 69, 70, 72, 73, 76, 79, 81, 82, 83, 85, 86, 87, 92, 3506, 3507, 3508, 3509, 3510, 3511, 5038, 5039, 5040, 5041, 5042]
Closing glyph list over 'glyf': 57 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'C', 'E', 'F', 'G', 'H', 'I', 'L', 'M', 'O', 'P', 'R', 'T', 'a', 'b', 'c', 'e', 'eight', 'f', 'fi', 'five', 'fl', 'four', 'i', 'l', 'n', 'nine', 'nonmarkingreturn', 'o', 'one', 'p', 'parenleft', 'parenright', 'percent', 'period', 'r', 's', 'seven', 'six', 'space', 't', 'three', 'two', 'underscore', 'uni239B', 'uni239C', 'uni239D', 'uni239E', 'uni239F', 'uni23A0', 'uniFB00', 'uniFB03', 'uniFB04', 'y', 'zero']
Glyph IDs:   [0, 1, 2, 3, 8, 11, 12, 17, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 36, 38, 40, 41, 42, 43, 44, 47, 48, 50, 51, 53, 55, 66, 68, 69, 70, 72, 73, 76, 79, 81, 82, 83, 85, 86, 87, 92, 3506, 3507, 3508, 3509, 3510, 3511, 5038, 5039, 5040, 5041, 5042]
Closed glyph list over 'glyf': 57 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'C', 'E', 'F', 'G', 'H', 'I', 'L', 'M', 'O', 'P', 'R', 'T', 'a', 'b', 'c', 'e', 'eight', 'f', 'fi', 'five', 'fl', 'four', 'i', 'l', 'n', 'nine', 'nonmarkingreturn', 'o', 'one', 'p', 'parenleft', 'parenright', 'percent', 'period', 'r', 's', 'seven', 'six', 'space', 't', 'three', 'two', 'underscore', 'uni239B', 'uni239C', 'uni239D', 'uni239E', 'uni239F', 'uni23A0', 'uniFB00', 'uniFB03', 'uniFB04', 'y', 'zero']
Glyph IDs:   [0, 1, 2, 3, 8, 11, 12, 17, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 36, 38, 40, 41, 42, 43, 44, 47, 48, 50, 51, 53, 55, 66, 68, 69, 70, 72, 73, 76, 79, 81, 82, 83, 85, 86, 87, 92, 3506, 3507, 3508, 3509, 3510, 3511, 5038, 5039, 5040, 5041, 5042]
Retaining 57 glyphs
head subsetting not needed
hhea subsetting not needed
maxp subsetting not needed
OS/2 subsetting not needed
hmtx subsetted
cmap subsetted
fpgm subsetting not needed
prep subsetting not needed
cvt  subsetting not needed
loca subsetting not needed
post subsetted
gasp subsetting not needed
MATH subsetted
GDEF subsetted
GPOS subsetted
GSUB subsetted
name subsetting not needed
glyf subsetted
head pruned
OS/2 Unicode ranges pruned: [0]
glyf pruned
GDEF pruned
GPOS pruned
GSUB pruned
name pruned
```

Example 26 (python):
```python
# add probability to adata
df = predictions.probability_matrix.copy()
adata_gex.obs[df.columns] = df.copy()
```

Example 27 (python):
```python
sc.pl.draw_graph(adata_gex,
                 color=['MACROPHAGE_PRE_MAC', 'MACROPHAGE_MICROGLIA', 'MACROPHAGE', ],
                 cmap='RdPu',size=30, show=False, frameon=False)
plt.savefig(f'{figdir}/Force-directed-graph_MoMP_YS_Mac_probability.pdf', bbox_inches="tight",dpi=300)
```

Example 28 (unknown):
```unknown
maxp pruned
cmap pruned
kern dropped
post pruned
FFTM dropped
GPOS pruned
GSUB pruned
glyf pruned
Added gid0 to subset
Added first four glyphs to subset
Closing glyph list over 'GSUB': 22 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'C', 'E', 'G', 'H', 'I', 'L', 'M', 'O', 'P', 'R', 'eight', 'four', 'nonmarkingreturn', 'period', 'six', 'space', 'two', 'underscore', 'zero']
Glyph IDs:   [0, 1, 2, 3, 17, 19, 21, 23, 25, 27, 36, 38, 40, 42, 43, 44, 47, 48, 50, 51, 53, 66]
Closed glyph list over 'GSUB': 22 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'C', 'E', 'G', 'H', 'I', 'L', 'M', 'O', 'P', 'R', 'eight', 'four', 'nonmarkingreturn', 'period', 'six', 'space', 'two', 'underscore', 'zero']
Glyph IDs:   [0, 1, 2, 3, 17, 19, 21, 23, 25, 27, 36, 38, 40, 42, 43, 44, 47, 48, 50, 51, 53, 66]
Closing glyph list over 'MATH': 22 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'C', 'E', 'G', 'H', 'I', 'L', 'M', 'O', 'P', 'R', 'eight', 'four', 'nonmarkingreturn', 'period', 'six', 'space', 'two', 'underscore', 'zero']
Glyph IDs:   [0, 1, 2, 3, 17, 19, 21, 23, 25, 27, 36, 38, 40, 42, 43, 44, 47, 48, 50, 51, 53, 66]
Closed glyph list over 'MATH': 22 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'C', 'E', 'G', 'H', 'I', 'L', 'M', 'O', 'P', 'R', 'eight', 'four', 'nonmarkingreturn', 'period', 'six', 'space', 'two', 'underscore', 'zero']
Glyph IDs:   [0, 1, 2, 3, 17, 19, 21, 23, 25, 27, 36, 38, 40, 42, 43, 44, 47, 48, 50, 51, 53, 66]
Closing glyph list over 'glyf': 22 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'C', 'E', 'G', 'H', 'I', 'L', 'M', 'O', 'P', 'R', 'eight', 'four', 'nonmarkingreturn', 'period', 'six', 'space', 'two', 'underscore', 'zero']
Glyph IDs:   [0, 1, 2, 3, 17, 19, 21, 23, 25, 27, 36, 38, 40, 42, 43, 44, 47, 48, 50, 51, 53, 66]
Closed glyph list over 'glyf': 22 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'C', 'E', 'G', 'H', 'I', 'L', 'M', 'O', 'P', 'R', 'eight', 'four', 'nonmarkingreturn', 'period', 'six', 'space', 'two', 'underscore', 'zero']
Glyph IDs:   [0, 1, 2, 3, 17, 19, 21, 23, 25, 27, 36, 38, 40, 42, 43, 44, 47, 48, 50, 51, 53, 66]
Retaining 22 glyphs
head subsetting not needed
hhea subsetting not needed
maxp subsetting not needed
OS/2 subsetting not needed
hmtx subsetted
cmap subsetted
fpgm subsetting not needed
prep subsetting not needed
cvt  subsetting not needed
loca subsetting not needed
post subsetted
gasp subsetting not needed
MATH subsetted
GDEF subsetted
GPOS subsetted
GSUB subsetted
name subsetting not needed
glyf subsetted
head pruned
OS/2 Unicode ranges pruned: [0]
glyf pruned
GDEF pruned
GPOS pruned
GSUB pruned
name pruned
```

Example 29 (python):
```python
pd.crosstab(adata.obs['fine_grain'],adata.obs['seg'])
```

Example 30 (python):
```python
adata.obs['fine_grain_mod'] = adata.obs['fine_grain'].astype('str').copy()
obsnames = adata.obs_names[adata.obs['milestones']=='29']
adata.obs.loc[obsnames,'fine_grain_mod'] = 'MacrophagesYSderived'
adata.obs['fine_grain_mod'].value_counts()
```

Example 31 (unknown):
```unknown
MacrophagesLYVE1pos     4757
MacrophagesATF3pos       898
MacrophagesCX3CR1pos     804
Monocytes                496
MacrophagesYSderived     207
MonocytesMPOpos          202
MonocyteDerivedCells     199
MacrophagesTIMD4pos      111
Name: fine_grain_mod, dtype: int64
```

Example 32 (python):
```python
sc.pl.draw_graph(adata,color=['fine_grain_mod'],cmap='RdPu',size=20,
                 frameon=False, show=False
                )
plt.savefig(f'{figdir}/Force-directed-graph_MoMP_celltype_annotation.pdf', bbox_inches="tight",dpi=300)
```

Example 33 (unknown):
```unknown
... storing 'fine_grain_mod' as categorical
maxp pruned
cmap pruned
kern dropped
post pruned
FFTM dropped
GPOS pruned
GSUB pruned
glyf pruned
Added gid0 to subset
Added first four glyphs to subset
Closing glyph list over 'GSUB': 42 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'C', 'D', 'E', 'F', 'I', 'L', 'M', 'O', 'P', 'R', 'S', 'T', 'V', 'X', 'Y', 'a', 'c', 'd', 'e', 'f', 'four', 'g', 'h', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'one', 'p', 'r', 's', 'space', 't', 'three', 'underscore', 'v', 'y']
Glyph IDs:   [0, 1, 2, 3, 20, 22, 23, 36, 38, 39, 40, 41, 44, 47, 48, 50, 51, 53, 54, 55, 57, 59, 60, 66, 68, 70, 71, 72, 73, 74, 75, 76, 79, 80, 81, 82, 83, 85, 86, 87, 89, 92]
Closed glyph list over 'GSUB': 47 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'C', 'D', 'E', 'F', 'I', 'L', 'M', 'O', 'P', 'R', 'S', 'T', 'V', 'X', 'Y', 'a', 'c', 'd', 'e', 'f', 'fi', 'fl', 'four', 'g', 'h', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'one', 'p', 'r', 's', 'space', 't', 'three', 'underscore', 'uniFB00', 'uniFB03', 'uniFB04', 'v', 'y']
Glyph IDs:   [0, 1, 2, 3, 20, 22, 23, 36, 38, 39, 40, 41, 44, 47, 48, 50, 51, 53, 54, 55, 57, 59, 60, 66, 68, 70, 71, 72, 73, 74, 75, 76, 79, 80, 81, 82, 83, 85, 86, 87, 89, 92, 5038, 5039, 5040, 5041, 5042]
Closing glyph list over 'MATH': 47 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'C', 'D', 'E', 'F', 'I', 'L', 'M', 'O', 'P', 'R', 'S', 'T', 'V', 'X', 'Y', 'a', 'c', 'd', 'e', 'f', 'fi', 'fl', 'four', 'g', 'h', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'one', 'p', 'r', 's', 'space', 't', 'three', 'underscore', 'uniFB00', 'uniFB03', 'uniFB04', 'v', 'y']
Glyph IDs:   [0, 1, 2, 3, 20, 22, 23, 36, 38, 39, 40, 41, 44, 47, 48, 50, 51, 53, 54, 55, 57, 59, 60, 66, 68, 70, 71, 72, 73, 74, 75, 76, 79, 80, 81, 82, 83, 85, 86, 87, 89, 92, 5038, 5039, 5040, 5041, 5042]
Closed glyph list over 'MATH': 47 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'C', 'D', 'E', 'F', 'I', 'L', 'M', 'O', 'P', 'R', 'S', 'T', 'V', 'X', 'Y', 'a', 'c', 'd', 'e', 'f', 'fi', 'fl', 'four', 'g', 'h', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'one', 'p', 'r', 's', 'space', 't', 'three', 'underscore', 'uniFB00', 'uniFB03', 'uniFB04', 'v', 'y']
Glyph IDs:   [0, 1, 2, 3, 20, 22, 23, 36, 38, 39, 40, 41, 44, 47, 48, 50, 51, 53, 54, 55, 57, 59, 60, 66, 68, 70, 71, 72, 73, 74, 75, 76, 79, 80, 81, 82, 83, 85, 86, 87, 89, 92, 5038, 5039, 5040, 5041, 5042]
Closing glyph list over 'glyf': 47 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'C', 'D', 'E', 'F', 'I', 'L', 'M', 'O', 'P', 'R', 'S', 'T', 'V', 'X', 'Y', 'a', 'c', 'd', 'e', 'f', 'fi', 'fl', 'four', 'g', 'h', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'one', 'p', 'r', 's', 'space', 't', 'three', 'underscore', 'uniFB00', 'uniFB03', 'uniFB04', 'v', 'y']
Glyph IDs:   [0, 1, 2, 3, 20, 22, 23, 36, 38, 39, 40, 41, 44, 47, 48, 50, 51, 53, 54, 55, 57, 59, 60, 66, 68, 70, 71, 72, 73, 74, 75, 76, 79, 80, 81, 82, 83, 85, 86, 87, 89, 92, 5038, 5039, 5040, 5041, 5042]
Closed glyph list over 'glyf': 47 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'C', 'D', 'E', 'F', 'I', 'L', 'M', 'O', 'P', 'R', 'S', 'T', 'V', 'X', 'Y', 'a', 'c', 'd', 'e', 'f', 'fi', 'fl', 'four', 'g', 'h', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'one', 'p', 'r', 's', 'space', 't', 'three', 'underscore', 'uniFB00', 'uniFB03', 'uniFB04', 'v', 'y']
Glyph IDs:   [0, 1, 2, 3, 20, 22, 23, 36, 38, 39, 40, 41, 44, 47, 48, 50, 51, 53, 54, 55, 57, 59, 60, 66, 68, 70, 71, 72, 73, 74, 75, 76, 79, 80, 81, 82, 83, 85, 86, 87, 89, 92, 5038, 5039, 5040, 5041, 5042]
Retaining 47 glyphs
head subsetting not needed
hhea subsetting not needed
maxp subsetting not needed
OS/2 subsetting not needed
hmtx subsetted
cmap subsetted
fpgm subsetting not needed
prep subsetting not needed
cvt  subsetting not needed
loca subsetting not needed
post subsetted
gasp subsetting not needed
MATH subsetted
GDEF subsetted
GPOS subsetted
GSUB subsetted
name subsetting not needed
glyf subsetted
head pruned
OS/2 Unicode ranges pruned: [0]
glyf pruned
GDEF pruned
GPOS pruned
GSUB pruned
name pruned
```

Example 34 (python):
```python
# save
adata.write(f'{outdir}/MultiVI-post_foetal_Mono-MP_withContCov-CCCov_13Mar2023_feature-select-0003.h5ad')
```

Example 35 (unknown):
```unknown
f'{outdir}/MultiVI-post_foetal_Mono-MP_withContCov-CCCov_13Mar2023_feature-select-0003.h5ad'
```

Example 36 (unknown):
```unknown
'/nfs/team205/heart/anndata_objects/Foetal/Immune/MoMP_Aug2023/MultiVI-post_foetal_Mono-MP_withContCov-CCCov_13Mar2023_feature-select-0003.h5ad'
```

Example 37 (python):
```python
adata_gex.obs['fine_grain_mod'] = adata.obs['fine_grain_mod'].reindex(adata_gex.obs_names)

predictions = celltypist.annotate(adata_gex, 
                                  model = '/nfs/team205/heart/anndata_objects/Foetal/celltypist/model_YS_portal_object_20221208_onlyMP_hvg_18Nov2023.pkl', 
                                  majority_voting = False
                                 )

# plot
celltypist.dotplot(predictions, use_as_reference = 'fine_grain_mod', use_as_prediction = 'predicted_labels', vmax=1, show=False)
plt.savefig(f'{figdir}/MoMP_celltypist-prediction_celltypes.pdf', bbox_inches="tight",dpi=300)
```

Example 38 (unknown):
```unknown
🔬 Input data has 6749 cells and 15922 genes
🔗 Matching reference genes in the model
🧬 828 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
maxp pruned
cmap pruned
kern dropped
post pruned
FFTM dropped
GPOS pruned
GSUB pruned
glyf pruned
Added gid0 to subset
Added first four glyphs to subset
Closing glyph list over 'GSUB': 53 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'L', 'M', 'O', 'P', 'R', 'S', 'T', 'V', 'X', 'Y', 'a', 'b', 'c', 'd', 'e', 'eight', 'f', 'five', 'four', 'g', 'h', 'i', 'l', 'n', 'nonmarkingreturn', 'o', 'one', 'p', 'parenleft', 'parenright', 'percent', 'period', 'r', 's', 'six', 'space', 't', 'three', 'two', 'underscore', 'v', 'y', 'zero']
Glyph IDs:   [0, 1, 2, 3, 8, 11, 12, 17, 19, 20, 21, 22, 23, 24, 25, 27, 36, 38, 39, 40, 41, 42, 43, 44, 47, 48, 50, 51, 53, 54, 55, 57, 59, 60, 66, 68, 69, 70, 71, 72, 73, 74, 75, 76, 79, 81, 82, 83, 85, 86, 87, 89, 92]
Closed glyph list over 'GSUB': 58 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'L', 'M', 'O', 'P', 'R', 'S', 'T', 'V', 'X', 'Y', 'a', 'b', 'c', 'd', 'e', 'eight', 'f', 'fi', 'five', 'fl', 'four', 'g', 'h', 'i', 'l', 'n', 'nonmarkingreturn', 'o', 'one', 'p', 'parenleft', 'parenright', 'percent', 'period', 'r', 's', 'six', 'space', 't', 'three', 'two', 'underscore', 'uniFB00', 'uniFB03', 'uniFB04', 'v', 'y', 'zero']
Glyph IDs:   [0, 1, 2, 3, 8, 11, 12, 17, 19, 20, 21, 22, 23, 24, 25, 27, 36, 38, 39, 40, 41, 42, 43, 44, 47, 48, 50, 51, 53, 54, 55, 57, 59, 60, 66, 68, 69, 70, 71, 72, 73, 74, 75, 76, 79, 81, 82, 83, 85, 86, 87, 89, 92, 5038, 5039, 5040, 5041, 5042]
Closing glyph list over 'MATH': 58 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'L', 'M', 'O', 'P', 'R', 'S', 'T', 'V', 'X', 'Y', 'a', 'b', 'c', 'd', 'e', 'eight', 'f', 'fi', 'five', 'fl', 'four', 'g', 'h', 'i', 'l', 'n', 'nonmarkingreturn', 'o', 'one', 'p', 'parenleft', 'parenright', 'percent', 'period', 'r', 's', 'six', 'space', 't', 'three', 'two', 'underscore', 'uniFB00', 'uniFB03', 'uniFB04', 'v', 'y', 'zero']
Glyph IDs:   [0, 1, 2, 3, 8, 11, 12, 17, 19, 20, 21, 22, 23, 24, 25, 27, 36, 38, 39, 40, 41, 42, 43, 44, 47, 48, 50, 51, 53, 54, 55, 57, 59, 60, 66, 68, 69, 70, 71, 72, 73, 74, 75, 76, 79, 81, 82, 83, 85, 86, 87, 89, 92, 5038, 5039, 5040, 5041, 5042]
Closed glyph list over 'MATH': 64 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'L', 'M', 'O', 'P', 'R', 'S', 'T', 'V', 'X', 'Y', 'a', 'b', 'c', 'd', 'e', 'eight', 'f', 'fi', 'five', 'fl', 'four', 'g', 'h', 'i', 'l', 'n', 'nonmarkingreturn', 'o', 'one', 'p', 'parenleft', 'parenright', 'percent', 'period', 'r', 's', 'six', 'space', 't', 'three', 'two', 'underscore', 'uni239B', 'uni239C', 'uni239D', 'uni239E', 'uni239F', 'uni23A0', 'uniFB00', 'uniFB03', 'uniFB04', 'v', 'y', 'zero']
Glyph IDs:   [0, 1, 2, 3, 8, 11, 12, 17, 19, 20, 21, 22, 23, 24, 25, 27, 36, 38, 39, 40, 41, 42, 43, 44, 47, 48, 50, 51, 53, 54, 55, 57, 59, 60, 66, 68, 69, 70, 71, 72, 73, 74, 75, 76, 79, 81, 82, 83, 85, 86, 87, 89, 92, 3506, 3507, 3508, 3509, 3510, 3511, 5038, 5039, 5040, 5041, 5042]
Closing glyph list over 'glyf': 64 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'L', 'M', 'O', 'P', 'R', 'S', 'T', 'V', 'X', 'Y', 'a', 'b', 'c', 'd', 'e', 'eight', 'f', 'fi', 'five', 'fl', 'four', 'g', 'h', 'i', 'l', 'n', 'nonmarkingreturn', 'o', 'one', 'p', 'parenleft', 'parenright', 'percent', 'period', 'r', 's', 'six', 'space', 't', 'three', 'two', 'underscore', 'uni239B', 'uni239C', 'uni239D', 'uni239E', 'uni239F', 'uni23A0', 'uniFB00', 'uniFB03', 'uniFB04', 'v', 'y', 'zero']
Glyph IDs:   [0, 1, 2, 3, 8, 11, 12, 17, 19, 20, 21, 22, 23, 24, 25, 27, 36, 38, 39, 40, 41, 42, 43, 44, 47, 48, 50, 51, 53, 54, 55, 57, 59, 60, 66, 68, 69, 70, 71, 72, 73, 74, 75, 76, 79, 81, 82, 83, 85, 86, 87, 89, 92, 3506, 3507, 3508, 3509, 3510, 3511, 5038, 5039, 5040, 5041, 5042]
Closed glyph list over 'glyf': 64 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'L', 'M', 'O', 'P', 'R', 'S', 'T', 'V', 'X', 'Y', 'a', 'b', 'c', 'd', 'e', 'eight', 'f', 'fi', 'five', 'fl', 'four', 'g', 'h', 'i', 'l', 'n', 'nonmarkingreturn', 'o', 'one', 'p', 'parenleft', 'parenright', 'percent', 'period', 'r', 's', 'six', 'space', 't', 'three', 'two', 'underscore', 'uni239B', 'uni239C', 'uni239D', 'uni239E', 'uni239F', 'uni23A0', 'uniFB00', 'uniFB03', 'uniFB04', 'v', 'y', 'zero']
Glyph IDs:   [0, 1, 2, 3, 8, 11, 12, 17, 19, 20, 21, 22, 23, 24, 25, 27, 36, 38, 39, 40, 41, 42, 43, 44, 47, 48, 50, 51, 53, 54, 55, 57, 59, 60, 66, 68, 69, 70, 71, 72, 73, 74, 75, 76, 79, 81, 82, 83, 85, 86, 87, 89, 92, 3506, 3507, 3508, 3509, 3510, 3511, 5038, 5039, 5040, 5041, 5042]
Retaining 64 glyphs
head subsetting not needed
hhea subsetting not needed
maxp subsetting not needed
OS/2 subsetting not needed
hmtx subsetted
cmap subsetted
fpgm subsetting not needed
prep subsetting not needed
cvt  subsetting not needed
loca subsetting not needed
post subsetted
gasp subsetting not needed
MATH subsetted
GDEF subsetted
GPOS subsetted
GSUB subsetted
name subsetting not needed
glyf subsetted
head pruned
OS/2 Unicode ranges pruned: [0]
glyf pruned
GDEF pruned
GPOS pruned
GSUB pruned
name pruned
```

Example 39 (python):
```python
# get gene expression data
ad = adata.uns['adata_gex_lognorm'].copy()
ad.obs['fine_grain_mod'] = adata.obs['fine_grain_mod'].reindex(ad.obs_names)
print(ad.X.data[:5])
ad
```

Example 40 (unknown):
```unknown
[0.9766816 0.9766816 1.7862254 2.7663608 0.9766816]
```

Example 41 (python):
```python
AnnData object with n_obs × n_vars = 7674 × 15922
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', '_indices', '_scvi_batch', '_scvi_labels', 'fine_grain_withcycling', 't', 'seg', 'edge', 't_sd', 'milestones', 'fine_grain_mod'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id', 'n_cells'
    uns: '_scvi_manager_uuid', '_scvi_uuid', 'adata_auc_gene', 'adata_auc_region', 'adata_gex', 'adata_gex_lognorm', 'draw_graph', 'fine_grain_colors', 'fine_grain_withcycling_colors', 'graph', 'log1p', 'milestones_colors', 'neighbors', 'ppt', 'pseudotime_list', 'seg_colors', 'umap', 'week_colors'
    obsm: 'MultiVI_latent', 'X_R', 'X_draw_graph_fa', 'X_palantir', 'X_umap', 'X_umap13', '_scvi_extra_categorical_covs', '_scvi_extra_continuous_covs'
    obsp: 'connectivities', 'distances'
```

Example 42 (python):
```python
genes = ['CXCR1','TREM2','P2RY12','C3','TGFB1','IL1B','TLR2','TLR4','P2RX7','RAGE','CD47','ADGRG1']
genes = [x for x in genes if x in ad.var_names]

sc.pl.dotplot(ad,
              var_names=genes,
              groupby='fine_grain_mod',
              standard_scale = 'var'
             )
```

Example 43 (python):
```python
genes = ['CX3CR1','TREM2','P2RY12','ADGRG1','C3','TGFB1','IL1B']
genes = [x for x in genes if x in ad.var_names]

sc.pl.dotplot(ad,
              var_names=genes,
              groupby='fine_grain_mod',
              standard_scale = 'var',
              show=False
             )
plt.savefig(f'{figdir}/MoMP_microglia-markers_dotplot.pdf', bbox_inches="tight",dpi=300)
```

Example 44 (unknown):
```unknown
maxp pruned
cmap pruned
kern dropped
post pruned
FFTM dropped
GPOS pruned
GSUB pruned
glyf pruned
Added gid0 to subset
Added first four glyphs to subset
Closing glyph list over 'GSUB': 53 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'I', 'L', 'M', 'O', 'P', 'R', 'S', 'T', 'V', 'X', 'Y', 'a', 'c', 'd', 'e', 'f', 'five', 'four', 'g', 'h', 'i', 'l', 'n', 'nine', 'nonmarkingreturn', 'o', 'one', 'p', 'parenleft', 'parenright', 'percent', 'period', 'r', 's', 'seven', 'space', 't', 'three', 'two', 'u', 'v', 'x', 'y', 'zero']
Glyph IDs:   [0, 1, 2, 3, 8, 11, 12, 17, 19, 20, 21, 22, 23, 24, 26, 28, 36, 37, 38, 39, 40, 41, 42, 44, 47, 48, 50, 51, 53, 54, 55, 57, 59, 60, 68, 70, 71, 72, 73, 74, 75, 76, 79, 81, 82, 83, 85, 86, 87, 88, 89, 91, 92]
Closed glyph list over 'GSUB': 58 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'I', 'L', 'M', 'O', 'P', 'R', 'S', 'T', 'V', 'X', 'Y', 'a', 'c', 'd', 'e', 'f', 'fi', 'five', 'fl', 'four', 'g', 'h', 'i', 'l', 'n', 'nine', 'nonmarkingreturn', 'o', 'one', 'p', 'parenleft', 'parenright', 'percent', 'period', 'r', 's', 'seven', 'space', 't', 'three', 'two', 'u', 'uniFB00', 'uniFB03', 'uniFB04', 'v', 'x', 'y', 'zero']
Glyph IDs:   [0, 1, 2, 3, 8, 11, 12, 17, 19, 20, 21, 22, 23, 24, 26, 28, 36, 37, 38, 39, 40, 41, 42, 44, 47, 48, 50, 51, 53, 54, 55, 57, 59, 60, 68, 70, 71, 72, 73, 74, 75, 76, 79, 81, 82, 83, 85, 86, 87, 88, 89, 91, 92, 5038, 5039, 5040, 5041, 5042]
Closing glyph list over 'MATH': 58 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'I', 'L', 'M', 'O', 'P', 'R', 'S', 'T', 'V', 'X', 'Y', 'a', 'c', 'd', 'e', 'f', 'fi', 'five', 'fl', 'four', 'g', 'h', 'i', 'l', 'n', 'nine', 'nonmarkingreturn', 'o', 'one', 'p', 'parenleft', 'parenright', 'percent', 'period', 'r', 's', 'seven', 'space', 't', 'three', 'two', 'u', 'uniFB00', 'uniFB03', 'uniFB04', 'v', 'x', 'y', 'zero']
Glyph IDs:   [0, 1, 2, 3, 8, 11, 12, 17, 19, 20, 21, 22, 23, 24, 26, 28, 36, 37, 38, 39, 40, 41, 42, 44, 47, 48, 50, 51, 53, 54, 55, 57, 59, 60, 68, 70, 71, 72, 73, 74, 75, 76, 79, 81, 82, 83, 85, 86, 87, 88, 89, 91, 92, 5038, 5039, 5040, 5041, 5042]
Closed glyph list over 'MATH': 64 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'I', 'L', 'M', 'O', 'P', 'R', 'S', 'T', 'V', 'X', 'Y', 'a', 'c', 'd', 'e', 'f', 'fi', 'five', 'fl', 'four', 'g', 'h', 'i', 'l', 'n', 'nine', 'nonmarkingreturn', 'o', 'one', 'p', 'parenleft', 'parenright', 'percent', 'period', 'r', 's', 'seven', 'space', 't', 'three', 'two', 'u', 'uni239B', 'uni239C', 'uni239D', 'uni239E', 'uni239F', 'uni23A0', 'uniFB00', 'uniFB03', 'uniFB04', 'v', 'x', 'y', 'zero']
Glyph IDs:   [0, 1, 2, 3, 8, 11, 12, 17, 19, 20, 21, 22, 23, 24, 26, 28, 36, 37, 38, 39, 40, 41, 42, 44, 47, 48, 50, 51, 53, 54, 55, 57, 59, 60, 68, 70, 71, 72, 73, 74, 75, 76, 79, 81, 82, 83, 85, 86, 87, 88, 89, 91, 92, 3506, 3507, 3508, 3509, 3510, 3511, 5038, 5039, 5040, 5041, 5042]
Closing glyph list over 'glyf': 64 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'I', 'L', 'M', 'O', 'P', 'R', 'S', 'T', 'V', 'X', 'Y', 'a', 'c', 'd', 'e', 'f', 'fi', 'five', 'fl', 'four', 'g', 'h', 'i', 'l', 'n', 'nine', 'nonmarkingreturn', 'o', 'one', 'p', 'parenleft', 'parenright', 'percent', 'period', 'r', 's', 'seven', 'space', 't', 'three', 'two', 'u', 'uni239B', 'uni239C', 'uni239D', 'uni239E', 'uni239F', 'uni23A0', 'uniFB00', 'uniFB03', 'uniFB04', 'v', 'x', 'y', 'zero']
Glyph IDs:   [0, 1, 2, 3, 8, 11, 12, 17, 19, 20, 21, 22, 23, 24, 26, 28, 36, 37, 38, 39, 40, 41, 42, 44, 47, 48, 50, 51, 53, 54, 55, 57, 59, 60, 68, 70, 71, 72, 73, 74, 75, 76, 79, 81, 82, 83, 85, 86, 87, 88, 89, 91, 92, 3506, 3507, 3508, 3509, 3510, 3511, 5038, 5039, 5040, 5041, 5042]
Closed glyph list over 'glyf': 64 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'I', 'L', 'M', 'O', 'P', 'R', 'S', 'T', 'V', 'X', 'Y', 'a', 'c', 'd', 'e', 'f', 'fi', 'five', 'fl', 'four', 'g', 'h', 'i', 'l', 'n', 'nine', 'nonmarkingreturn', 'o', 'one', 'p', 'parenleft', 'parenright', 'percent', 'period', 'r', 's', 'seven', 'space', 't', 'three', 'two', 'u', 'uni239B', 'uni239C', 'uni239D', 'uni239E', 'uni239F', 'uni23A0', 'uniFB00', 'uniFB03', 'uniFB04', 'v', 'x', 'y', 'zero']
Glyph IDs:   [0, 1, 2, 3, 8, 11, 12, 17, 19, 20, 21, 22, 23, 24, 26, 28, 36, 37, 38, 39, 40, 41, 42, 44, 47, 48, 50, 51, 53, 54, 55, 57, 59, 60, 68, 70, 71, 72, 73, 74, 75, 76, 79, 81, 82, 83, 85, 86, 87, 88, 89, 91, 92, 3506, 3507, 3508, 3509, 3510, 3511, 5038, 5039, 5040, 5041, 5042]
Retaining 64 glyphs
head subsetting not needed
hhea subsetting not needed
maxp subsetting not needed
OS/2 subsetting not needed
hmtx subsetted
cmap subsetted
fpgm subsetting not needed
prep subsetting not needed
cvt  subsetting not needed
loca subsetting not needed
post subsetted
gasp subsetting not needed
MATH subsetted
GDEF subsetted
GPOS subsetted
GSUB subsetted
name subsetting not needed
glyf subsetted
head pruned
OS/2 Unicode ranges pruned: [0]
glyf pruned
GDEF pruned
GPOS pruned
GSUB pruned
name pruned
```

---

## Function¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig3_SAN_niche/snMultime_no2_region-signatures.html

**Contents:**
- Function¶
- Read in DAR result¶
- Save the peaks¶

**Examples:**

Example 1 (python):
```python
# Parameters
working_dir = "/nfs/team205/heart/Vitro/TF_selection/foetal_SANPcell_vs_workingCM"
path_to_adata_gex = "/nfs/team205/heart/anndata_objects/Foetal/Feb28ObjectRaw_Multiome.h5ad"
path_to_adata_peak_tfidf = "/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/PeakMatrix/Foetal_Peaks_tfidf.h5ad"
path_to_DAR_1 = "/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/MarkerPeaks/SANP-vs-aCM.csv"
path_to_DAR_2 = "/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/MarkerPeaks/aCM-vs-SANP.csv"
path_to_p2g_ArchR = ""
group_key = "fine_grain_aCMs_merged"
group_a = "SinoatrialNodeCardiomyocytes"
group_b = "AtrialCardiomyocytes"
subset_object = True
geneset_list = ["MSigDB_Hallmark_2020", "KEGG_2021_Human", "GO_Biological_Process_2023"]

path_to_eRegulon_signatures_gene = "/nfs/team205/heart/pyscenic/plus/eRegulon_signatures/Cardiomyocytes_Gene_eRegulon_signatures.pkl"
path_to_eRegulon_signatures_region = "/nfs/team205/heart/pyscenic/plus/eRegulon_signatures/Cardiomyocytes_Region_eRegulon_signatures.pkl"
```

Example 2 (python):
```python
!ls /nfs/team205/heart/pyscenic/plus/eRegulon_metadata
```

Example 3 (python):
```python
aCM_full_eRegulon_metadata.csv
aCM_full_eRegulon_metadata_filtered.csv
aCM_n1140_eRegulon_metadata.csv
aCM_n1140_eRegulon_metadata_filtered.csv
aCM_n570_eRegulon_metadata.csv
aCM_n570_eRegulon_metadata_filtered.csv
Cardiomyocytes_eRegulon_metadata.csv
Cardiomyocytes_eRegulon_metadata_filtered.csv
Endothelium_eRegulon_metadata.csv
Endothelium_eRegulon_metadata_filtered.csv
Epicardium_eRegulon_metadata.csv
Epicardium_eRegulon_metadata_filtered.csv
Foetal_Adult_CMs_eRegulon_metadata.csv
Foetal_Adult_CMs_eRegulon_metadata_filtered.csv
Global_eRegulon_metadata.csv
Global_eRegulon_metadata_filtered.csv
Leukocytes_eRegulon_metadata.csv
Leukocytes_eRegulon_metadata_filtered.csv
Mesenchymal_eRegulon_metadata.csv
Mesenchymal_eRegulon_metadata_filtered.csv
MoMP_eRegulon_metadata.csv
Neural_eRegulon_metadata.csv
Neural_eRegulon_metadata_filtered.csv
vCM_onlyLeft_eRegulon_metadata.csv
vCM_onlyLeft_eRegulon_metadata_filtered.csv
```

Example 4 (python):
```python
import scanpy as sc
import pandas as pd
import numpy as np
import anndata

import matplotlib.pyplot as plt
import seaborn as sns
```

Example 5 (python):
```python
import session_info
session_info.show()
```

Example 6 (python):
```python
-----
anndata             0.9.2
matplotlib          3.5.2
numpy               1.22.3
pandas              1.5.3
scanpy              1.8.2
seaborn             0.12.2
session_info        1.0.0
-----
```

Example 7 (python):
```python
PIL                 10.0.0
asttokens           NA
backcall            0.2.0
beta_ufunc          NA
binom_ufunc         NA
cffi                1.15.1
colorama            0.4.6
comm                0.1.4
cpuinfo             NA
cycler              0.10.0
cython_runtime      NA
dateutil            2.8.2
debugpy             1.6.7.post1
decorator           5.1.1
executing           1.2.0
h5py                3.9.0
hypergeom_ufunc     NA
igraph              0.10.6
importlib_metadata  NA
ipykernel           6.25.1
jedi                0.19.0
joblib              1.3.2
kiwisolver          1.4.5
leidenalg           0.10.1
llvmlite            0.40.1
mpl_toolkits        NA
natsort             8.4.0
nbinom_ufunc        NA
ncf_ufunc           NA
numba               0.57.1
numexpr             2.8.5
packaging           23.1
parso               0.8.3
patsy               0.5.3
pexpect             4.8.0
pickleshare         0.7.5
pkg_resources       NA
platformdirs        3.10.0
plotly              5.16.1
prompt_toolkit      3.0.39
psutil              5.9.5
ptyprocess          0.7.0
pure_eval           0.2.2
pydev_ipython       NA
pydevconsole        NA
pydevd              2.9.5
pydevd_file_utils   NA
pydevd_plugins      NA
pydevd_tracing      NA
pygments            2.16.1
pyparsing           3.0.9
pytz                2023.3
scipy               1.9.1
sinfo               0.3.4
six                 1.16.0
sklearn             1.3.0
sphinxcontrib       NA
stack_data          0.6.2
statsmodels         0.14.0
tables              3.8.0
texttable           1.6.7
threadpoolctl       3.2.0
tornado             6.3.3
traitlets           5.9.0
typing_extensions   NA
wcwidth             0.2.6
zipp                NA
zmq                 25.1.1
```

Example 8 (unknown):
```unknown
-----
IPython             8.12.2
jupyter_client      8.3.0
jupyter_core        5.3.1
-----
Python 3.8.17 (default, Jul  5 2023, 21:04:15) [GCC 11.2.0]
Linux-4.15.0-212-generic-x86_64-with-glibc2.17
-----
Session information updated at 2023-11-05 12:00
```

Example 9 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 10 (python):
```python
sc.settings.set_figure_params(dpi=120)
```

Example 11 (python):
```python
logFC_threshold = 0
FDR_threshold = 0.1

# SANP
dar = pd.read_csv(path_to_DAR_1)
dar_up = dar[(dar['Log2FC']> logFC_threshold)&(dar['FDR']<FDR_threshold)]
dar_down = dar[(dar['Log2FC']< -logFC_threshold)&(dar['FDR']<FDR_threshold)]
print(f'up: {len(dar_up)} regions')
print(f'down: {len(dar_down)} regions')
```

Example 12 (unknown):
```unknown
up: 2197 regions
down: 1183 regions
```

Example 13 (python):
```python
peak_signatures = {}
# up, towards group_a
peak_signatures[f'{group_a}_up'] = list(dar_up['seqnames'].astype('str')+':'+dar_up['start'].astype('str')+'_'+dar_up['end'].astype('str'))
peak_signatures[f'{group_a}_down'] = list(dar_down['seqnames'].astype('str')+':'+dar_down['start'].astype('str')+'_'+dar_down['end'].astype('str'))

print(peak_signatures.keys())

# save
utils.save_pkl(peak_signatures,
           file = f'{working_dir}/outs/peak_signatures.pkl'
          )
```

Example 14 (unknown):
```unknown
dict_keys(['SinoatrialNodeCardiomyocytes_up', 'SinoatrialNodeCardiomyocytes_down'])
```

Example 15 (unknown):
```unknown
f'{working_dir}/outs/peak_signatures.pkl'
```

Example 16 (unknown):
```unknown
'/nfs/team205/heart/Vitro/TF_selection/foetal_SANPcell_vs_workingCM/outs/peak_signatures.pkl'
```

---

## Variables¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/Xenium/no2_integration_and_cell-type-annotation/no6_manual-annotation_atrialCM.html

**Contents:**
- Variables¶
- Read in data¶
- Annotate low quality cells as "unclassified"¶
- Subset and reclustering¶
- Transfer annotation to the original adata¶
- Save¶

--> cluster 4,5,8,10 has lower total counts and smaller cell area --> annotate as "unclassified"

--> CXCL12 expression in SAN-PC?? --> cluster 0,10,15,12,17 are likely to be "doublets" (high expression of other cell type markers, such as POSTN, KDR)

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os

import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 

import scanpy.external as sce
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/session_info/main.py:213: UserWarning: The '__version__' attribute is deprecated and will be removed in MarkupSafe 3.1. Use feature detection, or `importlib.metadata.version("markupsafe")`, instead.
  mod_version = _find_version(mod.__version__)
```

Example 4 (python):
```python
-----
matplotlib          3.10.1
numpy               2.1.3
pandas              2.2.3
scanpy              1.11.1
seaborn             0.13.2
session_info        v1.0.1
-----
```

Example 5 (python):
```python
Cython              3.1.1
PIL                 11.1.0
anndata             0.11.4
asciitree           NA
asttokens           NA
charset_normalizer  3.4.1
cloudpickle         3.1.1
comm                0.2.2
cycler              0.12.1
cython              3.1.1
cython_runtime      NA
dask                2024.11.2
dateutil            2.9.0.post0
debugpy             1.8.13
decorator           5.2.1
exceptiongroup      1.2.2
executing           2.2.0
h5py                3.13.0
igraph              0.11.8
importlib_metadata  NA
ipykernel           6.29.5
jedi                0.19.2
jinja2              3.1.6
joblib              1.4.2
kiwisolver          1.4.8
legacy_api_wrap     NA
leidenalg           0.10.2
llvmlite            0.44.0
markupsafe          3.0.2
mpl_toolkits        NA
msgpack             1.1.0
natsort             8.4.0
numba               0.61.0
numcodecs           0.13.1
packaging           24.2
parso               0.8.4
patsy               1.0.1
platformdirs        4.3.7
prompt_toolkit      3.0.50
psutil              7.0.0
pure_eval           0.2.3
pyarrow             19.0.1
pydev_ipython       NA
pydevconsole        NA
pydevd              3.2.3
pydevd_file_utils   NA
pydevd_plugins      NA
pydevd_tracing      NA
pygments            2.19.1
pyparsing           3.2.3
pytz                2025.2
scipy               1.15.2
six                 1.17.0
sklearn             1.5.2
stack_data          0.6.3
statsmodels         0.14.4
tblib               3.1.0
texttable           1.7.0
threadpoolctl       3.6.0
tlz                 1.0.0
toolz               1.0.0
tornado             6.4.2
traitlets           5.14.3
typing_extensions   NA
wcwidth             0.2.13
yaml                6.0.2
zarr                2.18.3
zipp                NA
zmq                 26.3.0
zoneinfo            NA
```

Example 6 (unknown):
```unknown
-----
IPython             8.34.0
jupyter_client      8.6.3
jupyter_core        5.7.2
-----
Python 3.10.14 | packaged by conda-forge | (main, Mar 20 2024, 12:45:18) [GCC 12.3.0]
Linux-4.18.0-553.53.1.el8_10.x86_64-x86_64-with-glibc2.28
-----
Session information updated at 2025-06-09 20:35
```

Example 7 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/home/kk837/rfs/rfs-teichlab-nfs-iCNyzSAaucw/kk837/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 8 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 9 (python):
```python
adata_dir = '/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium/subsets'
sample_id = 'C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat'
celltype = 'AtrialCardiomyocytes'
```

Example 10 (unknown):
```unknown
# figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk/Aug2024'
# table_dir = '/nfs/team205/heart/fetal_ms_figures/tables/Aug2024'
```

Example 11 (unknown):
```unknown
latent_space = 'pca_harmony'
```

Example 12 (python):
```python
# read in
adata = sc.read_h5ad(f'{adata_dir}/{sample_id}_5K_{celltype}_lognorm.h5ad')
adata
```

Example 13 (unknown):
```unknown
AnnData object with n_obs × n_vars = 47076 × 5000
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_3.0', 'multi_celltypes_coarse', 'leiden_0.2', 'leiden_0.4', 'leiden_0.6', 'leiden_1'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1', 'n_cells', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'dendrogram_pca_harmony_leiden_1.0', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1_colors', 'log1p', 'multi_celltypes_coarse_colors', 'pca', 'pca_harmony', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.2_colors', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.3_colors', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.4_colors', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.5_colors', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_0.8_colors', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.0_colors', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_1.5_colors', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_2.0_colors', 'pca_harmony_leiden_3.0', 'pca_harmony_leiden_3.0_colors', 'tissue_block_id_colors', 'umap'
    obsm: 'X_pca', 'X_pca_harmony', 'X_umap_pca_harmony', 'spatial'
    varm: 'PCs'
    obsp: 'pca_harmony_connectivities', 'pca_harmony_distances'
```

Example 14 (python):
```python
adata.X.data[:5]
```

Example 15 (unknown):
```unknown
array([3.5825472, 3.5825472, 3.5825472, 3.5825472, 3.5825472],
      dtype=float32)
```

Example 16 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=['total_counts','n_genes_by_counts','cell_area','tissue_block_id'],
                    wspace=0.2,cmap='RdPu',vmax='p100')
```

Example 17 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=['multi_celltypes_coarse',"conf_score_midmod2fine","celltypist_midmod2fine",],
                    wspace=0.2,cmap='RdPu',vmax='p100')
resolutions_list = [0.2,0.4,0.6,1]
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                    color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                    wspace=0.2)
```

Example 18 (python):
```python
%%time
# leiden clustering
resolutions_list = [# 0.2,0.4,0.6,1, # already done
                    1.2,1.5,2]
for resolution in resolutions_list:
    sc.tl.leiden(adata,neighbors_key=latent_space,
                     resolution=resolution,key_added=f'leiden_{str(resolution)}',n_iterations=2)
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                wspace=0.3)
```

Example 19 (unknown):
```unknown
<timed exec>:5: FutureWarning: In the future, the default backend for leiden will be igraph instead of leidenalg.

 To achieve the future defaults please pass: flavor="igraph" and n_iterations=2.  directed must also be False to work with igraph's implementation.
```

Example 20 (unknown):
```unknown
CPU times: user 50.2 s, sys: 400 ms, total: 50.6 s
Wall time: 50.7 s
```

Example 21 (python):
```python
# check marker expressions
genes = ['TBX3','TBX18','SHOX2','HCN1','HCN4','CACNA1D', 'GJA1','VSNL1',# SANPC
         'GREB1L','GNG11','NRXN3','RSPO3', # AVNPC
         'PITX2','KCNMB2','PCDH7','RYR3','MBP', # aCM_left
         'NTM','ROR1','KIF26B','PLCL1','ANGPT1', # aCM_right
         'NKX2-5','MKI67','TOP2A'
        ]
genes = [x for x in genes if x in adata.var_names]
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=genes,
               ncols=5, wspace=0.2,cmap='RdPu',vmax='p100')
```

Example 22 (unknown):
```unknown
resolution_sel = 1.5
```

Example 23 (python):
```python
sc.pl.violin(adata,keys=['total_counts','cell_area'],groupby=f'leiden_{str(resolution_sel)}')
```

Example 24 (python):
```python
# annotate
adata.obs['tmp_annotation'] = adata.obs[f'leiden_{str(resolution_sel)}'].copy()
adata.obs.replace({'tmp_annotation':{
    '0':'AtrialCardiomyocytes',
    '1':'AtrialCardiomyocytes',
    '2':'AtrialCardiomyocytes',
    '3':'AtrialCardiomyocytes',
    '4':'unclassified',
    '5':'unclassified',
    '6':'AtrialCardiomyocytes',
    '7':'AtrialCardiomyocytes',
    '8':'unclassified',
    '9':'AtrialCardiomyocytes',
    '10':'unclassified',
    '11':'AtrialCardiomyocytes',
    '12':'AtrialCardiomyocytes',
    '13':'AtrialCardiomyocytes',
    '14':'AtrialCardiomyocytes',
    '15':'AtrialCardiomyocytes',
}},inplace=True)
adata.obs['tmp_annotation'].value_counts()
```

Example 25 (python):
```python
/tmp/ipykernel_60220/1768720550.py:3: FutureWarning: The behavior of Series.replace (and DataFrame.replace) with CategoricalDtype is deprecated. In a future version, replace will only be used for cases that preserve the categories. To change the categories, use ser.cat.rename_categories instead.
  adata.obs.replace({'tmp_annotation':{
```

Example 26 (unknown):
```unknown
tmp_annotation
AtrialCardiomyocytes    33947
unclassified            13129
Name: count, dtype: int64
```

Example 27 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color='tmp_annotation',
               ncols=5, wspace=0.2)
```

Example 28 (python):
```python
adata_sub = adata[adata.obs['tmp_annotation']=='AtrialCardiomyocytes']
```

Example 29 (python):
```python
# drop leiden clustering results
resolutions_list = [0.2,0.4,0.6,1, 1.2,1.5,2]
for resolution in resolutions_list:
    adata_sub.obs.drop(f'leiden_{str(resolution)}',axis=1,inplace=True)
adata_sub
```

Example 30 (unknown):
```unknown
AnnData object with n_obs × n_vars = 33947 × 5000
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_3.0', 'multi_celltypes_coarse', 'tmp_annotation'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1', 'n_cells', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'dendrogram_pca_harmony_leiden_1.0', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1_colors', 'log1p', 'multi_celltypes_coarse_colors', 'pca', 'pca_harmony', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.2_colors', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.3_colors', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.4_colors', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.5_colors', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_0.8_colors', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.0_colors', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_1.5_colors', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_2.0_colors', 'pca_harmony_leiden_3.0', 'pca_harmony_leiden_3.0_colors', 'tissue_block_id_colors', 'umap', 'leiden_1.2', 'leiden_1.5', 'leiden_2', 'leiden_1.2_colors', 'leiden_1.5_colors', 'leiden_2_colors', 'tmp_annotation_colors'
    obsm: 'X_pca', 'X_pca_harmony', 'X_umap_pca_harmony', 'spatial'
    varm: 'PCs'
    obsp: 'pca_harmony_connectivities', 'pca_harmony_distances'
```

Example 31 (python):
```python
# HVGs
# sc.pp.highly_variable_genes(adata_sub, flavor='seurat', min_mean=0.0125, max_mean=3, min_disp=0.5)
# print(f'number of hvgs: {sum(adata_sub.var.highly_variable)}')
# PCA
sc.tl.pca(adata_sub, use_highly_variable=False)
# harmony integration
sce.pp.harmony_integrate(adata_sub, 'tissue_block_id', basis='X_pca', adjusted_basis='X_pca_harmony')
# create neighbour graph
latent_space = 'pca_harmony'
sc.pp.neighbors(adata_sub,n_neighbors=15,use_rep=f"X_{latent_space}",key_added=latent_space)
# visualisation with umap
sc.tl.umap(adata_sub,min_dist=0.5,neighbors_key=latent_space)
adata_sub.obsm[f"X_umap_{latent_space}"]=adata_sub.obsm["X_umap"]
del adata_sub.obsm["X_umap"]
```

Example 32 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_pca/__init__.py:438: FutureWarning: Argument `use_highly_variable` is deprecated, consider using the mask argument. Use_highly_variable=True can be called through mask_var="highly_variable". Use_highly_variable=False can be called through mask_var=None
  warn(msg, FutureWarning)
2025-06-09 20:36:42,658 - harmonypy - INFO - Computing initial centroids with sklearn.KMeans...
2025-06-09 20:36:46,050 - harmonypy - INFO - sklearn.KMeans initialization complete.
2025-06-09 20:36:46,145 - harmonypy - INFO - Iteration 1 of 10
2025-06-09 20:36:51,846 - harmonypy - INFO - Iteration 2 of 10
2025-06-09 20:36:57,428 - harmonypy - INFO - Converged after 2 iterations
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm
```

Example 33 (python):
```python
%%time
# reclustering
# leiden clustering
resolutions_list = [0.2,0.4,0.6,1, 1.2,1.5,2]
for resolution in resolutions_list:
    sc.tl.leiden(adata_sub,neighbors_key=latent_space,
                 resolution=resolution,key_added=f'leiden_{str(resolution)}',n_iterations=2)
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                wspace=0.3)
```

Example 34 (unknown):
```unknown
CPU times: user 55.8 s, sys: 561 ms, total: 56.4 s
Wall time: 56.5 s
```

Example 35 (python):
```python
genes = ['TBX3','TBX18','SHOX2','HCN1','HCN4','CACNA1D', 'GJA1','VSNL1',# SANPC
         'GREB1L','GNG11','NRXN3','RSPO3', # AVNPC
         'PITX2','KCNMB2','PCDH7','RYR3','MBP', # aCM_left
         'NTM','ROR1','KIF26B','PLCL1','ANGPT1', # aCM_right
         'NKX2-5','MKI67','TOP2A'
        ]
genes = [x for x in genes if x in adata_sub.var_names]
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color=genes,
               ncols=5, wspace=0.2,cmap='RdPu',vmax='p100')
```

Example 36 (unknown):
```unknown
sub_resolution_sel = 2
```

Example 37 (python):
```python
genes = ['TTN', 'TNNT2','MYH6', # generic CM marker
        'MYH7', 'MYL2', 'FHL2', # vCN
            'NPPA', 'MYL7', 'MYL4', #aCM
    'TBX3','TBX18','SHOX2','HCN1','HCN4','CACNA1D', 'GJA1','VSNL1',# SANPC
         'GREB1L','GNG11','NRXN3','RSPO3', # AVNPC
         'PITX2','KCNMB2','PCDH7','RYR3','MBP', # aCM_left
         'NTM','ROR1','KIF26B','PLCL1','ANGPT1', # aCM_right
         'MKI67','TOP2A','NKX2-5'
        ]
genes = [x for x in genes if x in adata.var_names]

sc.tl.dendrogram(adata_sub,groupby=f'leiden_{str(sub_resolution_sel)}')
sc.pl.dotplot(adata_sub,
              var_names=genes,
              groupby=f'leiden_{str(sub_resolution_sel)}',
              standard_scale='var',
              dendrogram=True
             )
```

Example 38 (python):
```python
# DEG analysis
print(adata_sub.X.data[:5])
adata_sub.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata_sub,groupby=f'leiden_{str(sub_resolution_sel)}')
sc.tl.dendrogram(adata_sub, groupby=f'leiden_{str(sub_resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata_sub, dendrogram=True, standard_scale='var', return_fig=True)
dp.add_totals().show()
```

Example 39 (unknown):
```unknown
[2.4920871 2.4920871 3.1429856 2.4920871 4.3644395]
```

Example 40 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:458: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "names"] = self.var_names[global_indices]
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:460: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "scores"] = scores[global_indices]
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:463: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "pvals"] = pvals[global_indices]
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:473: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "pvals_adj"] = pvals_adj[global_indices]
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:484: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "logfoldchanges"] = np.log2(
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:458: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "names"] = self.var_names[global_indices]
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:460: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "scores"] = scores[global_indices]
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:463: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "pvals"] = pvals[global_indices]
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:473: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "pvals_adj"] = pvals_adj[global_indices]
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:484: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "logfoldchanges"] = np.log2(
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:458: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "names"] = self.var_names[global_indices]
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:460: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "scores"] = scores[global_indices]
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:463: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "pvals"] = pvals[global_indices]
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:473: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "pvals_adj"] = pvals_adj[global_indices]
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:484: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "logfoldchanges"] = np.log2(
```

Example 41 (python):
```python
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color=f'leiden_{str(sub_resolution_sel)}',legend_loc='on data',
                wspace=0.3)
```

Example 42 (python):
```python
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color=f'multi_celltypes_coarse',legend_loc='on data',
                wspace=0.3)
```

Example 43 (python):
```python
# annotate
adata_sub.obs['fine_grain_manual'] = adata_sub.obs[f'leiden_{str(sub_resolution_sel)}'].copy()
adata_sub.obs.replace({'fine_grain_manual':{
    '0':'AtrialCardiomyocytes_DoubletWithFB',
    '1':'AtrialCardiomyocytesRight',
    '2':'AtrialCardiomyocytesLeft',
    '3':'AtrialCardiomyocytesRight',
    '4':'AtrioventricularNodePacemakerCells',
    '5':'AtrialCardiomyocytesRight',
    '6':'AtrialCardiomyocytesLeft',
    '7':'AtrialCardiomyocytesLeft',
    '8':'AtrialCardiomyocytesCycling',
    '9':'AtrialCardiomyocytesCycling',
    '10':'AtrialCardiomyocytes_DoubletWithEC',
    '11':'AtrialCardiomyocytesCycling',
    '12':'AtrialCardiomyocytes_DoubletWithFB',
    '13':'AtrialCardiomyocytesCycling',
    '14':'AtrialCardiomyocytesRight',
    '15':'AtrialCardiomyocytes_DoubletWithFB',
    '16':'AtrialCardiomyocytesCycling',
    '17':'AtrialCardiomyocytes_DoubletWithEC',
    '18':'AtrialCardiomyocytesRight',
    '19':'SinoatrialNodePacemakerCells',
    '20':'AtrialCardiomyocytesRight',
    '21':'AtrialCardiomyocytesLeft',
    '22':'AtrialCardiomyocytesLeft',
}},inplace=True)
adata_sub.obs['fine_grain_manual'].value_counts()
```

Example 44 (python):
```python
/tmp/ipykernel_60220/1879809394.py:3: FutureWarning: The behavior of Series.replace (and DataFrame.replace) with CategoricalDtype is deprecated. In a future version, replace will only be used for cases that preserve the categories. To change the categories, use ser.cat.rename_categories instead.
  adata_sub.obs.replace({'fine_grain_manual':{
```

Example 45 (unknown):
```unknown
fine_grain_manual
AtrialCardiomyocytesRight             9633
AtrialCardiomyocytesLeft              7206
AtrialCardiomyocytesCycling           7112
AtrialCardiomyocytes_DoubletWithFB    5201
AtrialCardiomyocytes_DoubletWithEC    2232
AtrioventricularNodePacemakerCells    2085
SinoatrialNodePacemakerCells           478
Name: count, dtype: int64
```

Example 46 (python):
```python
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color='fine_grain_manual',
                wspace=0.3)
```

Example 47 (python):
```python
adata.obs['fine_grain_manual'] = 'unclassified'
adata.obs.loc[adata_sub.obs_names,'fine_grain_manual'] = adata_sub.obs['fine_grain_manual']
```

Example 48 (python):
```python
for tissue_id in ['C194-HEA-0-FFPE-1', 'Hst45-HEA-0-FFPE-1']:
    slide = adata[adata.obs['tissue_block_id']==tissue_id]
    sc.pl.spatial(slide,color=['fine_grain_manual'],# groups='SinoatrialNodePacemakerCells',
                  spot_size=30,vmax='p100')
```

Example 49 (python):
```python
/tmp/ipykernel_60220/681652047.py:3: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],# groups='SinoatrialNodePacemakerCells',
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/anndata/_core/anndata.py:1138: ImplicitModificationWarning: Trying to modify attribute `.obs` of view, initializing view as actual.
  df[key] = c
```

Example 50 (python):
```python
/tmp/ipykernel_60220/681652047.py:3: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],# groups='SinoatrialNodePacemakerCells',
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/anndata/_core/anndata.py:1138: ImplicitModificationWarning: Trying to modify attribute `.obs` of view, initializing view as actual.
  df[key] = c
```

Example 51 (python):
```python
for tissue_id in ['C194-HEA-0-FFPE-1', 'Hst45-HEA-0-FFPE-1']:
    slide = adata[adata.obs['tissue_block_id']==tissue_id]
    sc.pl.spatial(slide,color=['fine_grain_manual'],groups='SinoatrialNodePacemakerCells',
                  spot_size=60,vmax='p100')
```

Example 52 (python):
```python
/tmp/ipykernel_60220/1370592205.py:3: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups='SinoatrialNodePacemakerCells',
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/anndata/_core/anndata.py:1138: ImplicitModificationWarning: Trying to modify attribute `.obs` of view, initializing view as actual.
  df[key] = c
```

Example 53 (python):
```python
/tmp/ipykernel_60220/1370592205.py:3: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups='SinoatrialNodePacemakerCells',
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/anndata/_core/anndata.py:1138: ImplicitModificationWarning: Trying to modify attribute `.obs` of view, initializing view as actual.
  df[key] = c
```

Example 54 (python):
```python
for tissue_id in ['C194-HEA-0-FFPE-1', 'Hst45-HEA-0-FFPE-1']:
    slide = adata[adata.obs['tissue_block_id']==tissue_id]
    sc.pl.spatial(slide,color=['fine_grain_manual'],
                  groups=['AtrialCardiomyocytesRight'],
                  spot_size=60,vmax='p100')
```

Example 55 (python):
```python
/tmp/ipykernel_60220/2612643899.py:3: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/anndata/_core/anndata.py:1138: ImplicitModificationWarning: Trying to modify attribute `.obs` of view, initializing view as actual.
  df[key] = c
```

Example 56 (python):
```python
/tmp/ipykernel_60220/2612643899.py:3: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/anndata/_core/anndata.py:1138: ImplicitModificationWarning: Trying to modify attribute `.obs` of view, initializing view as actual.
  df[key] = c
```

Example 57 (python):
```python
for tissue_id in ['C194-HEA-0-FFPE-1', 'Hst45-HEA-0-FFPE-1']:
    slide = adata[adata.obs['tissue_block_id']==tissue_id]
    sc.pl.spatial(slide,color=['fine_grain_manual'],
                  groups=['AtrialCardiomyocytesLeft'],
                  spot_size=60,vmax='p100')
```

Example 58 (python):
```python
/tmp/ipykernel_60220/3050971786.py:3: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/anndata/_core/anndata.py:1138: ImplicitModificationWarning: Trying to modify attribute `.obs` of view, initializing view as actual.
  df[key] = c
```

Example 59 (python):
```python
/tmp/ipykernel_60220/3050971786.py:3: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/anndata/_core/anndata.py:1138: ImplicitModificationWarning: Trying to modify attribute `.obs` of view, initializing view as actual.
  df[key] = c
```

Example 60 (python):
```python
for tissue_id in ['C194-HEA-0-FFPE-1', 'Hst45-HEA-0-FFPE-1']:
    slide = adata[adata.obs['tissue_block_id']==tissue_id]
    sc.pl.spatial(slide,color=['CXCL12'],cmap='RdPu',
                  spot_size=60,vmax='p100')
```

Example 61 (python):
```python
/tmp/ipykernel_60220/2026526932.py:3: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['CXCL12'],cmap='RdPu',
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/anndata/_core/anndata.py:1138: ImplicitModificationWarning: Trying to modify attribute `.obs` of view, initializing view as actual.
  df[key] = c
```

Example 62 (python):
```python
/tmp/ipykernel_60220/2026526932.py:3: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['CXCL12'],cmap='RdPu',
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/anndata/_core/anndata.py:1138: ImplicitModificationWarning: Trying to modify attribute `.obs` of view, initializing view as actual.
  df[key] = c
```

Example 63 (python):
```python
# save
adata.write(f'{adata_dir}/{sample_id}_5K_{celltype}_lognorm.h5ad')
print(f'{adata_dir}/{sample_id}_5K_{celltype}_lognorm.h5ad')
```

Example 64 (unknown):
```unknown
/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium/subsets/C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat_5K_AtrialCardiomyocytes_lognorm.h5ad
```

---

## Read in adata¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig5_Trisomy21/no07_milo.html

**Contents:**
- Read in adata¶
- Milo¶
- scVI - n_layers:2¶
- Plot¶

6693 rows × 8 columns

--> logFC<0 is euploid

577 rows × 11 columns

**Examples:**

Example 1 (python):
```python
import scanpy as sc
import pandas as pd
import numpy as np
import anndata
import scipy
import matplotlib.pyplot as plt
import seaborn as sns
import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 

import warnings
warnings.simplefilter("ignore", UserWarning)
```

Example 2 (python):
```python
'''
# Configure rpy2 to Use the Correct R Installation:
import os
os.environ['R_HOME'] = '/nfs/team205/kk18/miniconda3/envs/vitro/lib/R'
# check
from rpy2.rinterface_lib import openrlib
print(openrlib.R_HOME)
'''
```

Example 3 (python):
```python
# import milopy.core as milo
# import milopy.plot as milopl
# import milopy.utils
```

Example 4 (python):
```python
# import session_info
# session_info.show()
```

Example 5 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 6 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 7 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk/Aug2024'
table_dir = '/nfs/team205/heart/fetal_ms_figures/tables/Aug2024'
```

Example 8 (python):
```python
# atlas
adata = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/trisomy21/Euploid_T21Hearts_Aug2024_sel_subsampled-per-cellstate-donor.h5ad')
adata
```

Example 9 (python):
```python
AnnData object with n_obs × n_vars = 83477 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score', 'euploid_or_t21', 'donor_type', 'region_finest', 'flushed', 'scrublet_leiden', 'cluster_scrublet_score', 'leiden_scVI_0.1', 'leiden_scVI_0.2', 'leiden_scVI_0.3', 'leiden_scVI_0.5', 'leiden_scVI_0.7', 'leiden_scVI_1', 'celltypist_coarse', 'conf_score_coarse', 'cells_to_remove', 'celltypist_fine', 'conf_score_fine', 'coarse_grain_merged', 'fine_grain_merged', 'nhood_ixs_random', 'nhood_ixs_refined', 'nhood_kth_distance'
    var: 'gene_id', 'gene_name_scRNA-0-0', 'feature_type-1', 'mt-1', 'ribo-1', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1', 'gene_name_GenomicFeatures', 'gene_biotype_GenomicFeatures', 'gene_seq_start_GenomicFeatures', 'gene_seq_end_GenomicFeatures', 'seq_name_GenomicFeatures', 'seq_strand_GenomicFeatures', 'seq_coord_system_GenomicFeatures', 'description_GenomicFeatures', 'gene_id_version_GenomicFeatures', 'canonical_transcript_GenomicFeatures', 'gene_name'
    uns: 'coarse_grain_merged_colors', 'correcting-donor-region_n-layers-2', 'donor_colors', 'euploid_or_t21_colors', 'fine_grain_merged_colors', 'mid_grain_colors', 'nhood_adata', 'nhood_neighbors_key', 'region_colors', 'scVI_latent_correcting-donor-region_n-layers-2-forMilo', 'umap'
    obsm: 'X_umap_correcting-donor-region_n-layers-2', 'latent_gene_encoding', 'nhoods', 'scVI_latent_correcting-donor-region_n-layers-2'
    obsp: 'correcting-donor-region_n-layers-2_connectivities', 'correcting-donor-region_n-layers-2_distances', 'scVI_latent_correcting-donor-region_n-layers-2-forMilo_connectivities', 'scVI_latent_correcting-donor-region_n-layers-2-forMilo_distances'
```

Example 10 (unknown):
```unknown
basis = 'umap_correcting-donor-region_n-layers-2'
```

Example 11 (python):
```python
pd.crosstab(adata.obs['donor'],adata.obs['sangerID']).T
```

Example 12 (python):
```python
sc.pl.embedding(adata, basis=basis,
                    color=['coarse_grain_merged','euploid_or_t21'],
               size=5, wspace=0.5, ncols=6, components='all',cmap='RdPu',frameon=False,legend_fontsize=14)
```

Example 13 (python):
```python
# umap per celltype to highlight
group_col = 'region'
groups_to_plot = ['Heart',
                  'Great vessels', 
                 'Left atria',
                 'Left ventricle',
                 'Pericardial sac + RA + LA + IAS',
                 'Right ventricle',
                  # 'Pericardium',
                 ]

plt.rcParams['axes.grid'] = False
fig,axs=plt.subplots(nrows=2,ncols=4,figsize=(15,7))
# Flatten the axs array for easier iteration in case of multiple rows and columns
axs_flat = axs.flatten()

for i,grp in enumerate(groups_to_plot):
    sc.pl.embedding(adata, basis=basis,
                    color=[group_col], groups=grp, 
               size=10, frameon=False,
               show=False, ax=axs_flat[i], legend_loc=None, legend_fontsize=5,
               title=grp
              )
# Turn off any remaining subplots
for j in range(len(groups_to_plot), len(axs_flat)):
    fig.delaxes(axs_flat[j])

# fig.suptitle('Main Title for the Figure')
fig.tight_layout()
```

Example 14 (python):
```python
sc.pl.embedding(adata[adata.obs['euploid_or_t21']=='tri21'], 
                basis=basis,
                    color=['conf_score_fine'], 
               size=10, wspace=0.4, ncols=6, components='all',frameon=False,legend_fontsize=30, title='tri21, celltypist confidence score')
```

Example 15 (python):
```python
adata.obs['euploid_or_t21'].cat.categories
```

Example 16 (unknown):
```unknown
Index(['euploid', 'tri21'], dtype='object')
```

Example 17 (python):
```python
%%time
scVI_latent_rep = 'scVI_latent_correcting-donor-region_n-layers-2'
# compute the k-nearest-neighbor graph that is used in both clustering and umap algorithms
sc.pp.neighbors(adata, use_rep=scVI_latent_rep, n_neighbors=20, key_added=f"{scVI_latent_rep}-forMilo")
# Construct neighbourhoods
milo.make_nhoods(adata, neighbors_key=f"{scVI_latent_rep}-forMilo", prop=0.1)
# Count cells in neighbourhoods
# Milo leverages the variation in cell numbers between replicates for the same experimental condition to test for differential abundance. 
milo.count_nhoods(adata, sample_col="sangerID")
# Test for differential abundance between conditions
milo.DA_nhoods(adata, design="~ euploid_or_t21")
## Check results
milo_results = adata.uns["nhood_adata"].obs
milo_results
```

Example 18 (unknown):
```unknown
CPU times: user 24.7 s, sys: 423 ms, total: 25.1 s
Wall time: 26.2 s
```

Example 19 (unknown):
```unknown
milo_results['SpatialFDR'].hist(bins=100)
```

Example 20 (python):
```python
nhood_size = np.array(adata.obsm["nhoods"].sum(0)).ravel()
plt.hist(nhood_size, bins=100);
```

Example 21 (python):
```python
sc.pl.embedding(adata, basis=basis,
                    color=['coarse_grain_merged'],
               size=5, wspace=0.5, ncols=6, components='all',cmap='RdPu',frameon=False,legend_fontsize=14)
```

Example 22 (python):
```python
%%time
# Visualize results on UMAP embedding
milopy.utils.build_nhood_graph(adata,basis=f'X_{basis}')
plt.rcParams["figure.figsize"] = [12,10]
milopl.plot_nhood_graph(adata, 
                        alpha=0.1, ## SpatialFDR level
                        min_size=2, ## Size of smallest dot
                        show=False
                       )
plt.savefig(f'{figdir}/Tri21_RNA_Milo_UMAP.pdf', bbox_inches="tight",dpi=300)
```

Example 23 (unknown):
```unknown
CPU times: user 792 ms, sys: 8.5 ms, total: 801 ms
Wall time: 1.22 s
```

Example 24 (python):
```python
milopy.utils.annotate_nhoods(adata, anno_col='fine_grain_merged')
```

Example 25 (python):
```python
adata_copy = adata.copy()
milopy.utils.annotate_nhoods(adata_copy, anno_col='euploid_or_t21')
df = adata_copy.uns['nhood_adata'].obs.copy()
df[df['SpatialFDR']<0.1]
```

Example 26 (python):
```python
plt.hist(adata.uns['nhood_adata'].obs["nhood_annotation_frac"],bins=100);
plt.xlabel("celltype fraction")
```

Example 27 (unknown):
```unknown
Text(0.5, 0, 'celltype fraction')
```

Example 28 (python):
```python
# We can see that for the majority of neighbourhoods, almost all cells have the same neighbourhood. We can rename neighbourhoods where less than 60% of the cells have the top label as "Mixed"
adata.uns['nhood_adata'].obs.loc[adata.uns['nhood_adata'].obs["nhood_annotation_frac"] < 0.6, "nhood_annotation"] = "Mixed"
```

Example 29 (python):
```python
# save
# modify nhood names
adata.uns['nhood_adata'].obs_names = 'nhood'+adata.uns['nhood_adata'].obs_names
adata.write('/nfs/team205/heart/anndata_objects/Foetal/trisomy21/Euploid_T21Hearts_Aug2024_sel_subsampled-per-cellstate-donor.h5ad')
adata.uns['nhood_adata'].obs.head()
```

Example 30 (python):
```python
adata = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/trisomy21/Euploid_T21Hearts_Aug2024_sel_subsampled-per-cellstate-donor.h5ad')
adata.uns['nhood_adata'].obs.head()
```

Example 31 (python):
```python
# from CamU
adata = sc.read_h5ad('/rfs/project/rfs-iCNyzSAaucw/kk837/data_objects/Foetal/trisomy21/Euploid_T21Hearts_Aug2024_sel_subsampled-per-cellstate-donor.h5ad')
adata.uns['nhood_adata'].obs.head()
```

Example 32 (unknown):
```unknown
figdir = '/rfs/project/rfs-iCNyzSAaucw/kk837/notebooks/Foetal/fetal_ms_figures'
```

Example 33 (python):
```python
# map cell type name to short versions
df = pd.read_csv('/rfs/project/rfs-iCNyzSAaucw/kk837/notebooks/Foetal/finegrain_name_mapping.csv').set_index('Full_name')
mapping = df['Short_name_finalised'].to_dict()
adata.uns['nhood_adata'].obs.replace({'nhood_annotation':mapping},inplace=True)
```

Example 34 (python):
```python
# drop cell types which have lower than certain number of significant nhoods
sig_nhoods_thresh = 3

ad = adata.uns['nhood_adata'].copy()

data = ad.obs[['SpatialFDR','nhood_annotation','logFC']]
data = data[data['nhood_annotation']!='Mixed']
# drop cell types which has lower than 3 nhood
val_counts = data['nhood_annotation'].value_counts()
data = data[data['nhood_annotation'].isin(list(val_counts.index[val_counts>=3]))]
# highlight significant nhoods
mask_sig = data['SpatialFDR'] < 0.1
mask_eup = data['logFC'] < 0
mask_t21 = data['logFC'] > 0
nhoods_eup = list(data.index[mask_sig&mask_eup])
nhoods_t21 = list(data.index[mask_sig&mask_t21])
data['nhood_type'] = 'mixture'
data.loc[nhoods_eup,'nhood_type'] = 'euploid'
data.loc[nhoods_t21,'nhood_type'] = 'tri21'

# select cell types based on the number of significant nhoods
data['significant'] = data['SpatialFDR'] < 0.1 # to count number of significant nhood
sig_counts = data[['nhood_annotation','significant']].groupby('nhood_annotation').sum()
mask = sig_counts['significant'] >= sig_nhoods_thresh
cells_sel = sig_counts.index[mask]
data = data[data['nhood_annotation'].isin(cells_sel)]

# reorder cell types based on the number of significant nhoods
data['nhood_annotation'] = data['nhood_annotation'].astype('category')
data['nhood_annotation'] = data['nhood_annotation'].cat.reorder_categories(
    list(data.groupby('nhood_annotation').sum().sort_values('significant',ascending=False).index)
)

# Set the size of the plot
plt.figure(figsize=(3, 6))

# Create the violin plot
sns.violinplot(data=data, x='logFC', y='nhood_annotation', inner='box', scale='width', color='lightgray')

# Add jittered points without dodge to align them on the same y-axis location
sns.stripplot(data=data, x='logFC', y='nhood_annotation', jitter=True, size=5,
              hue='nhood_type', palette={'euploid': 'blue', 
                                         'mixture': 'gray',
                                         'tri21': 'red'
                                        }, 
              marker='o', alpha=0.7)

# Remove the legend created by hue
plt.legend([],[], frameon=False)
# Display the plot
# plt.title(comp)
plt.savefig(f'{figdir}/Tri21_RNA_Milo_beeswarm_sort_by_n-significant_celltype-selected.pdf', bbox_inches="tight",dpi=300)
plt.show()
```

Example 35 (python):
```python
/tmp/ipykernel_293470/1713106557.py:31: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning.
  list(data.groupby('nhood_annotation').sum().sort_values('significant',ascending=False).index)
/tmp/ipykernel_293470/1713106557.py:38: FutureWarning: 

The `scale` parameter has been renamed and will be removed in v0.15.0. Pass `density_norm='width'` for the same effect.
  sns.violinplot(data=data, x='logFC', y='nhood_annotation', inner='box', scale='width', color='lightgray')
```

Example 36 (python):
```python
# all the cell types

ad = adata.uns['nhood_adata'].copy()

data = ad.obs[['SpatialFDR','nhood_annotation','logFC']]
data = data[data['nhood_annotation']!='Mixed']
# drop cell types which has lower than 3 nhood
val_counts = data['nhood_annotation'].value_counts()
data = data[data['nhood_annotation'].isin(list(val_counts.index[val_counts>=3]))]
# highlight significant nhoods
mask_sig = data['SpatialFDR'] < 0.1
mask_eup = data['logFC'] < 0
mask_t21 = data['logFC'] > 0
nhoods_eup = list(data.index[mask_sig&mask_eup])
nhoods_t21 = list(data.index[mask_sig&mask_t21])
data['nhood_type'] = 'mixture'
data.loc[nhoods_eup,'nhood_type'] = 'euploid'
data.loc[nhoods_t21,'nhood_type'] = 'tri21'

# reorder based on number of significant nhoods
# reorder cell types
data['nhood_annotation'] = data['nhood_annotation'].astype('category')
data['significant'] = data['SpatialFDR'] < 0.1 # to count number of significant nhood
data['nhood_annotation'] = data['nhood_annotation'].cat.reorder_categories(
    list(data.groupby('nhood_annotation').sum().sort_values('significant',ascending=False).index)
)

# Set the size of the plot
plt.figure(figsize=(3, 10))

# Create the violin plot
sns.violinplot(data=data, x='logFC', y='nhood_annotation', inner='box', scale='width', color='lightgray')

# Add jittered points without dodge to align them on the same y-axis location
sns.stripplot(data=data, x='logFC', y='nhood_annotation', jitter=True, size=7,
              hue='nhood_type', palette={'euploid': 'blue', 
                                         'mixture': 'gray',
                                         'tri21': 'red'
                                        }, 
              marker='o', alpha=0.7)

# Remove the legend created by hue
plt.legend([],[], frameon=False)
# Display the plot
# plt.title(comp)
# plt.savefig(f'{figdir}/Tri21_RNA_Milo_beeswarm_sort_by_n-significant.pdf', bbox_inches="tight",dpi=300)
plt.show()
```

Example 37 (python):
```python
/tmp/ipykernel_293470/2917940356.py:25: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning.
  list(data.groupby('nhood_annotation').sum().sort_values('significant',ascending=False).index)
/tmp/ipykernel_293470/2917940356.py:32: FutureWarning: 

The `scale` parameter has been renamed and will be removed in v0.15.0. Pass `density_norm='width'` for the same effect.
  sns.violinplot(data=data, x='logFC', y='nhood_annotation', inner='box', scale='width', color='lightgray')
```

---

## Import required modules¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig2_Overview_of_cellular_niches_and_TissueTypist/tissuetypist_crossvalidation_no1_run.html

**Contents:**
- Import required modules¶
- Variables¶
- Read in pre-processed dataframe¶
- Cross validation¶

5 rows × 5221 columns

**Examples:**

Example 1 (python):
```python
import warnings
warnings.simplefilter(action='ignore', category=FutureWarning)
warnings.filterwarnings('ignore')

import scanpy as sc
import anndata
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import matplotlib as mpl
import seaborn as sns
import os
```

Example 2 (python):
```python
import tissuetypist as tt
```

Example 3 (python):
```python
import importlib
importlib.reload(tt)
```

Example 4 (python):
```python
<module 'tissuetypist' from '/nfs/team205/kk18/github/TissueTypist/tissuetypist/__init__.py'>
```

Example 5 (python):
```python
sc.settings.verbosity = 1          # verbosity: errors (0), warnings (1), info (2), hints (3)
sc.settings.set_figure_params(dpi=80)
```

Example 6 (unknown):
```unknown
dir_to_results = '/nfs/team205/kk18/notebooks/P48_Foetal-heart_Visium/notebooks/TissueTypist/tissuetypist_package/cross_validation_results/7Apr2025_without-CoronaryAdv'
# make the directory if not exist
if not os.path.exists(dir_to_results):
    os.makedirs(dir_to_results)
```

Example 7 (python):
```python
data = pd.read_pickle('/nfs/team205/kk18/notebooks/P48_Foetal-heart_Visium/notebooks/TissueTypist/tissuetypist_package/preprocessed_df/visiumsd_oct.pkl')
print(data.shape)
data.head()
```

Example 8 (unknown):
```unknown
(19590, 5221)
```

Example 9 (python):
```python
# remove coronary adventitia spots
data = data[data['tissue']!='Adventitia - Coronary']
print(data.shape)
```

Example 10 (unknown):
```unknown
(19179, 5221)
```

Example 11 (unknown):
```unknown
# Training data: weight to the seighbour spot data
weight2neighbour_list = [0,0.1,0.3,1,3]
weight2isedge_list = [0,1,5]
edge_feature_list = ['distance_to_edge','is_edge']
```

Example 12 (python):
```python
# prepare features
own_features = [x for x in data.columns if '_own' in x]
neighbour_features = [x for x in data.columns if '_neighbour-max' in x]
print(len(own_features))
print(own_features[:5])
print(len(neighbour_features))
print(neighbour_features[:5])
```

Example 13 (unknown):
```unknown
2607
['ITGA2_own', 'ZBTB42_own', 'MATN2_own', 'GLT8D2_own', 'CFH_own']
2607
['ITGA2_neighbour-max', 'ZBTB42_neighbour-max', 'MATN2_neighbour-max', 'GLT8D2_neighbour-max', 'CFH_neighbour-max']
```

Example 14 (python):
```python
for edge_feature in edge_feature_list:
    all_fold_results = []
    f1_score_class_results = []
    for neighbour_weight in weight2neighbour_list:
        for edge_weight in weight2isedge_list:
            print(f"Weight to neighbour spots: {neighbour_weight}, Weight to {edge_feature}: {edge_weight}")
            df_folds, f1_score_class_df = tt.cross_validation(data,
                                                              own_features,
                                                              neighbour_features,
                                                              edge_feature,
                                                              neighbour_weight,
                                                              edge_weight,
                                                              tissue_col='tissue',
                                                             )
            all_fold_results.append(df_folds)
            f1_score_class_results.append(f1_score_class_df)

    # Concatenate all factors into one tidy DataFrame
    results_per_fold = pd.concat(all_fold_results, ignore_index=True)
    f1_score_class_per_fold = pd.concat(f1_score_class_results, ignore_index=True)
    # save
    results_per_fold.to_csv(f'{dir_to_results}/{edge_feature}_results_per_fold.csv')
    f1_score_class_per_fold.to_csv(f'{dir_to_results}/{edge_feature}_f1_score_class_per_fold.csv')
```

Example 15 (unknown):
```unknown
Weight to neighbour spots: 0, Weight to distance_to_edge: 0
```

Example 16 (unknown):
```unknown
Weight to neighbour spots: 0, Weight to distance_to_edge: 1
```

Example 17 (unknown):
```unknown
Weight to neighbour spots: 0, Weight to distance_to_edge: 5
```

Example 18 (unknown):
```unknown
Weight to neighbour spots: 0.1, Weight to distance_to_edge: 0
```

Example 19 (unknown):
```unknown
Weight to neighbour spots: 0.1, Weight to distance_to_edge: 1
```

Example 20 (unknown):
```unknown
Weight to neighbour spots: 0.1, Weight to distance_to_edge: 5
```

Example 21 (unknown):
```unknown
Weight to neighbour spots: 0.3, Weight to distance_to_edge: 0
```

Example 22 (unknown):
```unknown
Weight to neighbour spots: 0.3, Weight to distance_to_edge: 1
```

Example 23 (unknown):
```unknown
Weight to neighbour spots: 0.3, Weight to distance_to_edge: 5
```

Example 24 (unknown):
```unknown
Weight to neighbour spots: 1, Weight to distance_to_edge: 0
```

Example 25 (unknown):
```unknown
Weight to neighbour spots: 1, Weight to distance_to_edge: 1
```

Example 26 (unknown):
```unknown
Weight to neighbour spots: 1, Weight to distance_to_edge: 5
```

Example 27 (unknown):
```unknown
Weight to neighbour spots: 3, Weight to distance_to_edge: 0
```

Example 28 (unknown):
```unknown
Weight to neighbour spots: 3, Weight to distance_to_edge: 1
```

Example 29 (unknown):
```unknown
Weight to neighbour spots: 3, Weight to distance_to_edge: 5
```

Example 30 (unknown):
```unknown
Weight to neighbour spots: 0, Weight to is_edge: 0
```

Example 31 (unknown):
```unknown
Weight to neighbour spots: 0, Weight to is_edge: 1
```

Example 32 (unknown):
```unknown
Weight to neighbour spots: 0, Weight to is_edge: 5
```

Example 33 (unknown):
```unknown
Weight to neighbour spots: 0.1, Weight to is_edge: 0
```

Example 34 (unknown):
```unknown
Weight to neighbour spots: 0.1, Weight to is_edge: 1
```

Example 35 (unknown):
```unknown
Weight to neighbour spots: 0.1, Weight to is_edge: 5
```

Example 36 (unknown):
```unknown
Weight to neighbour spots: 0.3, Weight to is_edge: 0
```

Example 37 (unknown):
```unknown
Weight to neighbour spots: 0.3, Weight to is_edge: 1
```

Example 38 (unknown):
```unknown
Weight to neighbour spots: 0.3, Weight to is_edge: 5
```

Example 39 (unknown):
```unknown
Weight to neighbour spots: 1, Weight to is_edge: 0
```

Example 40 (unknown):
```unknown
Weight to neighbour spots: 1, Weight to is_edge: 1
```

Example 41 (unknown):
```unknown
Weight to neighbour spots: 1, Weight to is_edge: 5
```

Example 42 (unknown):
```unknown
Weight to neighbour spots: 3, Weight to is_edge: 0
```

Example 43 (unknown):
```unknown
Weight to neighbour spots: 3, Weight to is_edge: 1
```

Example 44 (unknown):
```unknown
Weight to neighbour spots: 3, Weight to is_edge: 5
```

---

## Import modules¶

**URL:** http://127.0.0.1:9020/notebooks_html/Supplementary_materials/Macrophages/Mo-MP_no2_MultiVI_run-GPU_Mo-MP.html

**Contents:**
- Import modules¶
- Read in data¶
- Concatenate anndata¶
- Setup and training MultiVI (with continuous covariates)¶

1899 rows × 40 columns

7674 rows × 1 columns

544641 rows × 3 columns

MultiVI requires the main batch annotation to correspond to the modality of the samples. Other batch annotation, such as in the case of multiple RNA-only batches, can be specified using categorical_covariate_keys.

**Examples:**

Example 1 (python):
```python
import scvi
import anndata
import numpy as np
import pandas as pd
import scanpy as sc
import matplotlib.pyplot as plt

scvi.settings.seed = 420
```

Example 2 (python):
```python
Global seed set to 0
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/flax/core/frozen_dict.py:169: FutureWarning: jax.tree_util.register_keypaths is deprecated, and will be removed in a future release. Please use `register_pytree_with_keys()` instead.
  jax.tree_util.register_keypaths(
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/pytorch_lightning/utilities/warnings.py:53: LightningDeprecationWarning: pytorch_lightning.utilities.warnings.rank_zero_deprecation has been deprecated in v1.6 and will be removed in v1.8. Use the equivalent function from the pytorch_lightning.utilities.rank_zero module instead.
  new_rank_zero_deprecation(
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/pytorch_lightning/utilities/warnings.py:58: LightningDeprecationWarning: The `pytorch_lightning.loggers.base.rank_zero_experiment` is deprecated in v1.7 and will be removed in v1.9. Please use `pytorch_lightning.loggers.logger.rank_zero_experiment` instead.
  return new_rank_zero_deprecation(*args, **kwargs)
Global seed set to 420
```

Example 3 (python):
```python
import session_info
session_info.show()
```

Example 4 (python):
```python
-----
anndata             0.8.0
matplotlib          3.7.1
numpy               1.23.5
pandas              1.5.3
scanpy              1.9.3
scvi                0.19.0
session_info        1.0.0
-----
```

Example 5 (python):
```python
PIL                 9.4.0
absl                NA
asttokens           NA
attr                22.2.0
backcall            0.2.0
certifi             2022.12.07
cffi                1.15.1
charset_normalizer  3.1.0
chex                0.1.6
comm                0.1.2
contextlib2         NA
cycler              0.10.0
cython_runtime      NA
dateutil            2.8.2
debugpy             1.6.6
decorator           5.1.1
defusedxml          0.7.1
deprecate           0.3.2
docrep              0.3.2
etils               1.1.0
executing           1.2.0
flax                0.6.7
fsspec              2023.3.0
google              NA
h5py                3.8.0
idna                3.4
igraph              0.10.4
importlib_metadata  NA
importlib_resources NA
ipykernel           6.21.3
ipython_genutils    0.2.0
ipywidgets          8.0.4
jax                 0.4.6
jaxlib              0.4.6
jedi                0.18.2
joblib              1.2.0
kiwisolver          1.4.4
leidenalg           0.9.1
llvmlite            0.39.1
ml_collections      NA
mpl_toolkits        NA
msgpack             1.0.5
mudata              0.2.1
multipledispatch    0.6.0
natsort             8.3.1
numba               0.56.4
numpyro             0.11.0
nvidia              NA
opt_einsum          v3.3.0
optax               0.1.4
packaging           23.0
parso               0.8.3
pexpect             4.8.0
pickleshare         0.7.5
pkg_resources       NA
platformdirs        3.1.1
prompt_toolkit      3.0.38
psutil              5.9.4
ptyprocess          0.7.0
pure_eval           0.2.2
pydev_ipython       NA
pydevconsole        NA
pydevd              2.9.5
pydevd_file_utils   NA
pydevd_plugins      NA
pydevd_tracing      NA
pygments            2.14.0
pyparsing           3.0.9
pyro                1.8.4
pytorch_lightning   1.7.7
pytz                2022.7.1
requests            2.28.2
rich                NA
scipy               1.10.1
setuptools          65.6.3
six                 1.16.0
sklearn             1.2.2
stack_data          0.6.2
tensorboard         2.12.0
texttable           1.6.7
threadpoolctl       3.1.0
toolz               0.12.0
torch               1.13.1+cu117
torchmetrics        0.11.4
torchvision         0.14.1+cu117
tornado             6.2
tqdm                4.65.0
traitlets           5.9.0
tree                0.1.8
typing_extensions   NA
urllib3             1.26.15
wcwidth             0.2.6
yaml                6.0
zipp                NA
zmq                 25.0.1
zoneinfo            NA
```

Example 6 (unknown):
```unknown
-----
IPython             8.11.0
jupyter_client      8.0.3
jupyter_core        5.2.0
notebook            6.5.3
-----
Python 3.9.16 (main, Mar  8 2023, 14:00:05) [GCC 11.2.0]
Linux-4.15.0-200-generic-x86_64-with-glibc2.27
-----
Session information updated at 2023-03-13 21:45
```

Example 7 (python):
```python
# paired
adata_paired = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/Immune/MultiVI-pre_paired_foetal_MoMP_raw.h5ad')
adata_paired
```

Example 8 (unknown):
```unknown
AnnData object with n_obs × n_vars = 1899 × 544641
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id'
```

Example 9 (python):
```python
adata_paired.obs
```

Example 10 (python):
```python
sum(adata_paired.var['modality']=='Gene Expression')
```

Example 11 (python):
```python
# scrna
adata_scrna = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/Immune/MultiVI-pre_scRNA_foetal_MoMP_raw.h5ad')
adata_scrna
```

Example 12 (unknown):
```unknown
AnnData object with n_obs × n_vars = 5775 × 36601
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id'
```

Example 13 (python):
```python
adata_mvi = scvi.data.organize_multiome_anndatas(adata_paired, adata_scrna)
adata_mvi
```

Example 14 (python):
```python
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1785: FutureWarning: X.dtype being converted to np.float32 from float64. In the next version of anndata (0.9) conversion will not be automatic. Pass dtype explicitly to avoid this warning. Pass `AnnData(X, dtype=X.dtype, ...)` to get the future behavour.
  [AnnData(sparse.csr_matrix(a.shape), obs=a.obs) for a in all_adatas],
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1785: FutureWarning: X.dtype being converted to np.float32 from float64. In the next version of anndata (0.9) conversion will not be automatic. Pass dtype explicitly to avoid this warning. Pass `AnnData(X, dtype=X.dtype, ...)` to get the future behavour.
  [AnnData(sparse.csr_matrix(a.shape), obs=a.obs) for a in all_adatas],
```

Example 15 (unknown):
```unknown
AnnData object with n_obs × n_vars = 7674 × 544641
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id'
```

Example 16 (python):
```python
adata_mvi.obs[['modality']]
```

Example 17 (python):
```python
# MultiVI requires the features to be ordered, 
# such that genes appear before genomic regions. In this case this is already the case, but it’s always good to verify:
adata_mvi = adata_mvi[:, adata_mvi.var["modality"].argsort()].copy()
adata_mvi.var
```

Example 18 (python):
```python
# We also filter features to remove those that appear in fewer than 1% of the cells
# modified to 0.3% to retain around 100000 features
print(adata_mvi.shape)
sc.pp.filter_genes(adata_mvi, min_cells=int(adata_mvi.shape[0] * 0.003))
print(adata_mvi.shape)
```

Example 19 (unknown):
```unknown
(7674, 544641)
(7674, 142283)
```

Example 20 (python):
```python
adata_mvi.var['modality'].value_counts()
```

Example 21 (unknown):
```unknown
Peaks              126361
Gene Expression     15922
Name: modality, dtype: int64
```

Example 22 (python):
```python
adata_mvi.obs['modality'].value_counts()
```

Example 23 (unknown):
```unknown
expression    5775
paired        1899
Name: modality, dtype: int64
```

Example 24 (python):
```python
adata_mvi.obs['batch_key'].value_counts()
```

Example 25 (unknown):
```unknown
C99_Cell_5prime-v2          4129
C94_Cell_5prime-v2          1080
Hst33_Nuclei_Multiome-v1     838
C86_Cell_5prime-v2           566
C92_Nuclei_Multiome-v1       417
C98_Nuclei_Multiome-v1       319
Hst36_Nuclei_Multiome-v1     120
C85_Nuclei_Multiome-v1        81
C87_Nuclei_Multiome-v1        74
C82_Nuclei_Multiome-v1        50
Name: batch_key, dtype: int64
```

Example 26 (python):
```python
scvi.model.MULTIVI.setup_anndata(adata_mvi, batch_key='modality',
                                categorical_covariate_keys=['batch_key'],
                                continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo',
                                                          'S_score','G2M_score'])
```

Example 27 (unknown):
```unknown
No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.)
```

Example 28 (python):
```python
# When creating the object, we need to specify how many of the features are genes, and how many are genomic regions. 
# This is so MultiVI can determine the exact architecture for each modality.
mvi = scvi.model.MULTIVI(
    adata_mvi,
    n_genes=(adata_mvi.var['modality']=='Gene Expression').sum(),
    n_regions=(adata_mvi.var['modality']=='Peaks').sum(),
)
mvi.view_anndata_setup()
```

Example 29 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/torch/nn/init.py:405: UserWarning: Initializing zero-element tensors is a no-op
  warnings.warn("Initializing zero-element tensors is a no-op")
```

Example 30 (unknown):
```unknown
Anndata setup with scvi-tools version 0.19.0.
```

Example 31 (unknown):
```unknown
Setup via `MULTIVI.setup_anndata` with arguments:
```

Example 32 (unknown):
```unknown
{
│   'layer': None,
│   'batch_key': 'modality',
│   'size_factor_key': None,
│   'categorical_covariate_keys': ['batch_key'],
│   'continuous_covariate_keys': ['total_counts', 'pct_counts_mt', 'pct_counts_ribo', 'S_score', 'G2M_score'],
│   'protein_expression_obsm_key': None,
│   'protein_names_uns_key': None
}
```

Example 33 (unknown):
```unknown
Summary Statistics          
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃     Summary Stat Key     ┃ Value  ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│         n_batch          │   2    │
│         n_cells          │  7674  │
│ n_extra_categorical_covs │   1    │
│ n_extra_continuous_covs  │   5    │
│         n_labels         │   1    │
│          n_vars          │ 142283 │
└──────────────────────────┴────────┘
```

Example 34 (python):
```python
Data Registry                             
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃      Registry Key      ┃            scvi-tools Location             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│           X            │                  adata.X                   │
│         batch          │          adata.obs['_scvi_batch']          │
│ extra_categorical_covs │ adata.obsm['_scvi_extra_categorical_covs'] │
│ extra_continuous_covs  │ adata.obsm['_scvi_extra_continuous_covs']  │
│         ind_x          │           adata.obs['_indices']            │
│         labels         │         adata.obs['_scvi_labels']          │
└────────────────────────┴────────────────────────────────────────────┘
```

Example 35 (python):
```python
batch State Registry                    
┏━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location    ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['modality'] │ expression │          0          │
│                       │   paired   │          1          │
└───────────────────────┴────────────┴─────────────────────┘
```

Example 36 (python):
```python
labels State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 37 (python):
```python
batch State Registry                    
┏━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location    ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['modality'] │ expression │          0          │
│                       │   paired   │          1          │
└───────────────────────┴────────────┴─────────────────────┘
```

Example 38 (python):
```python
extra_categorical_covs State Registry                   
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location     ┃        Categories        ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['batch_key'] │  C82_Nuclei_Multiome-v1  │          0          │
│                        │  C85_Nuclei_Multiome-v1  │          1          │
│                        │    C86_Cell_5prime-v2    │          2          │
│                        │  C87_Nuclei_Multiome-v1  │          3          │
│                        │  C92_Nuclei_Multiome-v1  │          4          │
│                        │    C94_Cell_5prime-v2    │          5          │
│                        │  C98_Nuclei_Multiome-v1  │          6          │
│                        │    C99_Cell_5prime-v2    │          7          │
│                        │ Hst33_Nuclei_Multiome-v1 │          8          │
│                        │ Hst36_Nuclei_Multiome-v1 │          9          │
│                        │                          │                     │
└────────────────────────┴──────────────────────────┴─────────────────────┘
```

Example 39 (python):
```python
extra_continuous_covs State   
            Registry            
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃       Source Location        ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│  adata.obs['total_counts']   │
│  adata.obs['pct_counts_mt']  │
│ adata.obs['pct_counts_ribo'] │
│     adata.obs['S_score']     │
│    adata.obs['G2M_score']    │
└──────────────────────────────┘
```

Example 40 (unknown):
```unknown
mvi.train()
```

Example 41 (unknown):
```unknown
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/pytorch_lightning/trainer/configuration_validator.py:267: LightningDeprecationWarning: The `Callback.on_epoch_end` hook was deprecated in v1.6 and will be removed in v1.8. Please use `Callback.on_<train/validation/test>_epoch_end` instead.
  rank_zero_deprecation(
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
```

Example 42 (unknown):
```unknown
Epoch 404/500:  81%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████                              | 404/500 [52:59<12:35,  7.87s/it, loss=6.21e+03, v_num=1]
Monitored metric reconstruction_loss_validation did not improve in the last 50 records. Best score: 12617.060. Signaling Trainer to stop.
```

Example 43 (unknown):
```unknown
mvi.save("/nfs/team205/heart/anndata_objects/Foetal/Immune/MultiVI-model_foetal_Mono-MP_withContCov-CCCov_13Mar2023")
```

Example 44 (python):
```python
adata_mvi.obsm["MultiVI_latent"] = mvi.get_latent_representation()
```

Example 45 (python):
```python
adata_mvi.write('/nfs/team205/heart/anndata_objects/Foetal/Immune/MultiVI-post_foetal_Mono-MP_withContCov-CCCov_13Mar2023.h5ad')
```

Example 46 (python):
```python
# Convergency plot
mvi.history['elbo_train']
x = np.linspace(1, (len(mvi.history['elbo_train'])), (len(mvi.history['elbo_train'])))
plt.plot(x, mvi.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
```

---

## 

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/RNA/scnRNA_Integration-Annotation/Annotations/MesenchymalAnnotation.html

**Examples:**

Example 1 (python):
```python
# Import packages, initial settings

import numpy as np
import matplotlib.pyplot as plt
import scanpy as sc
import pandas as pd
import anndata
import scrublet as scr
import seaborn as sns
import magic
import pybiomart
import six
import sys
sys.modules['sklearn.externals.six'] = six
import scanorama
import harmonypy
import bbknn
import gseapy
from gseapy.plot import barplot, dotplot
import scvelo as scv
import scvi

plt.rcParams['pdf.fonttype'] = 42
plt.rcParams['ps.fonttype'] = 42

sc.settings.verbosity = 3  # verbosity: errors (0), warnings (1), info (2), hints (3)

from matplotlib.colors import LinearSegmentedColormap
cmap = LinearSegmentedColormap.from_list(name='gene_cmap', colors=['lightgrey', 'thistle', 'red', 'darkred']) 

sc.settings.set_figure_params(dpi=80, color_map='viridis', vector_friendly=False,  dpi_save=300, facecolor='white')
sc.logging.print_version_and_date()
#sc.logging.print_versions()
```

Example 2 (python):
```python
adata=sc.read('./Objects/20221130RNA_foetal_post-scVI_raw_ExtracardiacRemoved.h5ad')
print(adata.shape)
```

Example 3 (python):
```python
cell_cycle_genes = ['MCM5','PCNA','TYMS','FEN1','MCM2','MCM4','RRM1','UNG','GINS2','MCM6','CDCA7','DTL','PRIM1','UHRF1','MLF1IP','HELLS','RFC2','RPA2','NASP','RAD51AP1','GMNN','WDR76','SLBP','CCNE2','UBR7','POLD3','MSH2','ATAD2','RAD51','RRM2','CDC45','CDC6','EXO1','TIPIN','DSCC1','BLM','CASP8AP2','USP1','CLSPN','POLA1','CHAF1B','BRIP1','E2F8','HMGB2','CDK1','NUSAP1','UBE2C','BIRC5','TPX2','TOP2A','NDC80','CKS2','NUF2','CKS1B','MKI67','TMPO','CENPF','TACC3','FAM64A','SMC4','CCNB2','CKAP2L','CKAP2','AURKB','BUB1','KIF11','ANP32E','TUBB4B','GTSE1','KIF20B','HJURP','CDCA3','HN1','CDC20','TTK','CDC25C','KIF2C','RANGAP1','NCAPD2','DLGAP5','CDCA2','CDCA8','ECT2','KIF23','HMMR','AURKA','PSRC1','ANLN','LBR','CKAP5','CENPE','CTCF','NEK2','G2E3','GAS2L3','CBX5','CENPA']
s_genes = cell_cycle_genes[:43]
s_genes = [x for x in s_genes if x in adata.var_names]
print('S genes:', s_genes, '\n')
g2m_genes = cell_cycle_genes[43:]
g2m_genes = [x for x in g2m_genes if x in adata.var_names]
print('G2 genes:', g2m_genes, '\n')
cell_cycle_genes = [x for x in cell_cycle_genes if x in adata.var_names]

hb_genes1 = ['HBB','HBD','HBG1','HBG2','HBE1','HBZ','HBM','HBA2','HBA1', 'HBQ1']
hb_genes1 = [x for x in hb_genes1 if x in adata.var_names]
print('Hemoglobin genes set 1:', hb_genes1, '\n')

hb_genes2 = ['AHSP','GYPA','RHAG','GFI1B','KEL','EPB42','ALAS2','TSPO2','GYPB','GYPE','SPTA1','RHCE','KLF1','RHD','SLC4A1','SNX22','SQSTM1','SOD1','FTH1','MAP1LC3B','SLC25A37','TRIM58','HMBS']
hb_genes2 = [x for x in hb_genes2 if x in adata.var_names]
print('Hemoglobin genes set 2:', hb_genes2, '\n')

mito_genes=['MT-ND1','MT-ND2','MT-CO1','MT-CO2','MT-ATP8','MT-ATP6','MT-CO3','MT-ND3','MT-ND4L', 'MT-ND4', 'MT-ND5', 'MT-ND6', 'MT-CYB']
mito_genes = [x for x in mito_genes if x in adata.var_names]
print('Mitochondrial genes:', mito_genes, '\n')

ribo_genes = ['RPLP0','RPLP1','RPLP2','RPL3','RPL3L','RPL4','RPL5','RPL6','RPL7','RPL7A','RPL7L1','RPL8','RPL9','RPL10','RPL10A','RPL10L','RPL11','RPL12','RPL13','RPL13A','RPL14','RPL15','RPL17','RPL18','RPL18A','RPL19','RPL21','RPL22','RPL22L1','RPL23','RPL23A','RPL24','RPL26','RPL26L1','RPL27','RPL27A','RPL28','RPL29','RPL30','RPL31','RPL32','RPL34','RPL35','RPL35A','RPL36','RPL36A','RPL36AL','RPL37','RPL37A','RPL38','RPL39','RPL39L','UBA52','RPL41','RPSA','RPS2','RPS3','RPS3A','RPS4X','RPS4Y1','RPS4Y2','RPS5','RPS6','RPS7','RPS8','RPS9','RPS10','RPS11','RPS12','RPS13','RPS14','RPS15','RPS15A','RPS16','RPS17','RPS18','RPS19','RPS20','RPS21','RPS23','RPS24','RPS25','RPS26','RPS27','RPS27A','RPS27L','RPS28','RPS29','FAU']
ribo_genes = [x for x in ribo_genes if x in adata.var_names]
print('Ribo genes:', ribo_genes, '\n')

stress_genes=['FOS','FOSB','JUN','JUNB','JUND','ZFP36','SOCS3','CEBPD','CEBPB','ATF3','EGR1','HSPA1A','HSPA1B','HSP90AA1','HSP90AB1','HSPA8','HSPB1','HSPE1','IER3','IER2','BTG1','BTG2','DUSP1']
stress_genes = [x for x in stress_genes if x in adata.var_names]
print('Tissue dissociation stress associated genes:', stress_genes, '\n')

FoxProteins=['FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1']
chrY_genes=['SRY', 'RPS4Y1', 'ZFY', 'ZFY-AS1', 'LINC00278', 'PCDH11Y', 'TTTY2B', 'TBL1Y', 'PRKY', 'USP9Y', 'DDX3Y', 'UTY', 'TMSB4Y', 'NLGN4Y', 'TTTY14', 'KDM5D', 'TTTY10', 'EIF1AY', 'PRORY', 'LINC00266-4P']
MembraneProteins1=['AADAT','AAK1','ABCA12','ABCA7','ABCB1','ABCB11','ABCB4','ABCB6','ABCC1','ABCC3','ABCC4','ABCC5','ABCC6','ABCG2','ABHD3','ABI1','ABLIM3','AC004691.2','AC004922.1','AC005324.3','AC005726.1','AC008403.1','AC138696.1','ACIN1','ACKR3','ACLY','ACOT13','ACOT9','ACTN1','ACTR10','ACTR6','ADA','ADAM10','ADAM12','ADAM22','ADAM28','ADAM33','ADAMTS1','ADAMTS2','ADAMTSL4','ADAP1','ADCK5','ADD1','ADD2','ADGRB2','ADGRD1','ADGRG2','ADGRG7','ADGRL2','ADH1A','ADH1B','ADH1C','ADH7','ADK','ADORA1','ADRA1B','ADRM1','ADSS','AFAP1L2','AFDN','AFG1L','AGER','AGRN','AHNAK','AHNAK2','AIFM2','AKAP10','AKAP11','AKAP12','AKR1B10','AKR1B15','AL132780.3','AL139300.1','AL355315.1','AL358113.1','AL645922.1','ALDH3A1','ALK','ALOX15B','ALOXE3','ALPI','ALPP','ALPPL2','ALS2CR12','AMACR','AMDHD1','AMER1','AMER2','AMIGO2','AMOT','AMOTL1','AMOTL2','AMPH','ANK2','ANK3','ANKEF1','ANKFN1','ANKIB1','ANKLE2','ANKRA2','ANKRD13A','ANKRD35','ANKRD42','ANKRD45','ANKS1B','ANO1','ANO10','ANO2','ANO6','ANPEP','ANXA1','ANXA13','ANXA2','ANXA3','AP2M1','APBB1','APBB1IP','APC','APLP1','APMAP','APOC3','APOD','APOLD1','AQP1','AQP10','AQP3','AQP4','AQP5','ARAP1','ARF1','ARHGAP18','ARHGAP19','ARHGAP21','ARHGAP22','ARHGAP23','ARHGAP24','ARHGAP28','ARHGAP29','ARHGAP33','ARHGAP36','ARHGAP40','ARHGAP9','ARHGDIG','ARHGEF1','ARHGEF11','ARHGEF12','ARHGEF15','ARHGEF26','ARHGEF28','ARHGEF40','ARHGEF5','ARID2','ARID4A','ARID5B','ARL13A','ARL15','ARL4C','ARMC12','ARMCX5','ARPC1A','ARPC5','ARRB2','ARRDC4','ARVCF','ASAP1','ASB5','ASGR1','ASGR2','ASH2L','ASIC1','ASIC4','ASPM','ASPSCR1','ATG13','ATG3','ATG7','ATIC','ATOX1','ATP10D','ATP12A','ATP13A3','ATP1B3','ATP23','ATP2B1','ATP2B4','ATP2C2','ATP6V0A2','ATP6V1H','ATP8A2','ATP8B1','ATXN10','ATXN3','ATXN7L3','AVEN','AVPI1','AXIN2','AXL','B2M','B3GALT2','BACE1','BAG3','BAIAP2','BAIAP2L1','BANK1','BASP1','BBOF1','BCAR1','BCAR3','BCL2L12','BCORL1','BCR','BDKRB2','BEND6','BFSP1','BFSP2','BICD2','BIN2','BLNK','BLVRB','BMPR2','BMX','BOC','BOLL','BORCS5','BORCS8-MEF2B','BRK1','BRPF1','BRSK1','BSDC1','BST2','BTK','BTN2A1','BTRC','BUB1B-PAK6','BVES','BZW2','C10orf90','C11orf42','C11orf52','C12orf73','C12orf76','C14orf37','C14orf93','C16orf74','C17orf58','C17orf62','C18orf21','C19orf33','C19orf38','C19orf48','C19orf71','C1orf116','C1orf189','C1orf35','C1orf54','C1orf68','C1orf74','C1QBP','C2orf72','C3orf14','C3orf20','C3orf38','C4orf19','C4orf22','C7orf43','C8orf33','C8orf37','C9orf153','C9orf3','CA14','CA9','CABLES2','CACNA1E','CACNG4','CADM1','CALCRL','CALD1','CALML5','CALN1','CAMK2A','CAMK2B','CAMK2D','CAMK2G','CAP2','CAPS','CASKIN2','CASP4','CAST','CAV3','CAVIN1','CAVIN2','CAVIN3','CAVIN4','CBWD1','CBWD2','CBWD3','CBWD5','CBWD6','CC2D1A','CCBE1','CCDC107','CCDC112','CCDC115','CCDC124','CCDC136','CCDC144NL','CCDC15','CCDC154','CCDC170','CCDC24','CCDC28A','CCDC59','CCDC62','CCDC66','CCDC69','CCDC8','CCDC83','CCDC85C','CCDC93','CCDC97','CCNT2','CCNYL1','CCR3','CCR4','CCT3','CD109','CD163','CD164L2','CD200R1','CD226','CD248','CD274','CD2AP','CD300A','CD300LB','CD33','CD34','CD37','CD38','CD3E','CD3G','CD4','CD44','CD46','CD47','CD48','CD79B','CD81','CD84','CD86','CD8A','CD8B','CD9','CD93','CD99L2','CDAN1','CDC42EP1','CDC42EP3','CDC42EP4','CDC42SE1','CDCA4','CDH1','CDH13','CDH15','CDH17','CDH2','CDH26','CDH3','CDH4','CDH5','CDH6','CDH8','CDIPT','CDK16','CDK5','CDK5RAP2','CDK7','CDRT15','CDV3','CEACAM5','CELSR1','CENPJ','CEP162','CEP295','CEP55','CEP57L1','CEP89','CERCAM','CFAP157','CFAP206','CFAP58','CFAP65','CFB','CFL2','CFLAR','CGN','CGNL1','CHCHD7','CHIC2','CHMP2A','CHRM3','CHRNA1','CHRNA5','CHRNB4','CHRND','CIB1','CIB3','CIP2A','CKAP5','CLCA2','CLCA4','CLCN5','CLCN6','CLDN11','CLDN18','CLDN2','CLDN23','CLDN3','CLDN4','CLDN7','CLDN9','CLEC2A','CLEC4F','CLEC5A','CLECL1','CLIC2','CLIC5','CLTB','CLUL1','CMTM4','CMTM7','CMTR2','CMYA5','CNGA1','CNKSR2','CNNM1','CNNM4','CNP','CNR1','CNTD1','COBL','COBLL1','COG3','COL16A1','COL17A1','COL4A5','COL6A2','COLEC12','COLQ','COMMD4','COMTD1','COPRS','COQ7','CORIN','CORO1B','CORO2A','CORO2B','COX6B2','CPEB3','CPPED1','CPZ','CRB3','CREB5','CRIP1','CRIP2','CRK','CRLF3','CROCC','CRTC1','CRYAB','CRYBG3','CRYL1','CSDE1','CSF1','CSF1R','CSPG4','CST6','CTNNA1','CTNNB1','CTNNBIP1','CTNND1','CTNND2','CTSF','CTSV','CTTN','CTXN1','CUEDC1','CUTA','CX3CL1','CX3CR1','CXADR','CXCR2','CYB5D2','CYBRD1','CYFIP2','CYP20A1','CYP24A1','CYTH2','CYTH4','DAAM1','DAB2','DAG1','DAGLB','DAW1','DBN1','DCAF13','DCAF16','DCBLD2','DCHS1','DCHS2','DCP2','DCT','DCTN6','DCUN1D2','DDAH1','DDB2','DDIAS','DDIT4L','DDR1','DDR2','DDX58','DDX60L','DENND6B','DFFA','DGKG','DGKH','DHDDS','DHPS','DHRS7B','DIAPH1','DIAPH3','DIO2','DIS3L','DLG1','DLG2','DLG5','DLGAP3','DLL3','DMWD','DNAAF1','DNAAF4','DNAH17','DNAJA4','DNAJB13','DNAJB4','DNAJC18','DNAJC5','DNAJC6','DNAJC9','DOC2A','DOCK4','DOK3','DONSON','DPEP1','DPH1','DPPA5','DPYSL2','DQX1','DRD1','DRP2','DSC3','DSEL','DSG2','DSP','DTNA','DUS1L','DUSP15','DXO','DYSF','ECSCR','EDA2R','EDNRA','EDNRB','EEF1AKMT1','EEF2','EFCAB1','EFCAB11','EFCAB2','EFCAB7','EFHC2','EFNB1','EFR3A','EFR3B','EGFR','EHBP1','EHD1','EHD2','EHD3','EHD4','EIF2B2','EIF5','EIF5B','ELFN1','ELMO3','ELN','ELOC','EMP3','ENAH','ENC1','ENG','ENKD1','ENO1','ENO2','ENO3','ENOX1','ENOX2','EPB41','EPB41L1','EPB41L2','EPB41L3','EPB41L4A','EPB41L4B','EPB41L5','EPCAM','EPHA2','EPHA3','EPHB1','EPHB2','EPN1','EPOR','EPS8L2','ERAP1','ERBB2','ERBB3','ERBIN','ERC1','ERCC4','ERMN','ESCO2','ETNK1','ETS2','EVA1A','EVI2B','EXOC1','EXOC3L1','EXOC7','EZR','F11R','F2RL1','F2RL3','FAAP20','FABP2','FABP3','FABP5','FAIM','FAM126A','FAM126B','FAM129A','FAM129B','FAM13A','FAM163B','FAM171A1','FAM193A','FAM205A','FAM210B','FAM222A','FAM45A','FAM47E','FAM47E-STBD1','FAM72A','FAM72B','FAM72C','FAM72D','FANCG','FAS','FASN','FBLIM1','FBLN2','FBLN5','FBLN7','FBP2','FBRSL1','FBXL2','FBXO16','FBXO38','FBXO41','FBXW11','FCAR','FCGBP','FCGR1A','FCGR1B','FCGR2A','FCHSD2','FFAR4','FGD1','FGD5','FGFR2','FGR','FHIT','FHL1','FIBCD1','FIGN','FILIP1','FILIP1L','FKBP2','FLAD1','FLCN','FLNA','FLNB','FLNC','FLOT1','FLOT2','FLRT3','FLT1','FLT4','FLVCR1','FMN2','FMNL2','FMNL3','FNBP1','FOXA2','FOXN3','FP565260.3','FRMD3','FRMD7','FRMD8','FRS2','FSCN1','FTCD','FUT2','FXYD3','FYB1','FYN','FZD1','FZD2','FZD4','FZD6','GAB1','GAB2','GABRA1','GABRA2','GABRA3','GABRA5','GABRG1','GAD1','GADL1','GAGE1','GAGE10','GAGE12C','GAGE12D','GAGE12E','GAGE12F','GAGE12G','GAGE12H','GAGE12J','GAGE13','GAGE2A','GAGE2E','GALNT10','GALR1','GAP43','GAPDH','GAPT','GAPVD1','GAS2','GAS2L1','GAS7','GAS8','GBP4','GCA','GCC1','GCLM','GCSAML-AS1','GFI1B','GFOD1','GFRA3','GHR','GID8','GINM1','GIPC1','GJA1','GJB3','GJB4','GJB6','GJB7','GJC1','GJC2','GLO1','GLP2R','GLRX','GLUL','GMIP','GNAI2','GNAQ','GNAS','GNB1','GNB2','GNB3','GNB4','GNG2','GP9','GPAT3','GPATCH1','GPBP1','GPBP1L1','GPC1','GPC3','GPC4','GPI','GPR1','GPR142','GPR156','GPR157','GPR160','GPR173','GPR20','GPR27','GPR50','GPR63','GPRC5A','GPRIN1','GPRIN2','GPX8','GRAMD1A','GRAMD1B','GRAMD1C','GRAP2','GRB7','GRID2IP','GRIK5','GRK3','GRK5','GRPR','GSDMA','GTF2F1','GTSE1','GUCA1A','GYPA','GYPC','HAPLN3','HARBI1','HAS1','HAUS4','HAUS7','HCAR2','HCAR3','HCFC2','HCK','HCLS1','HCST','HDAC8','HEG1','HELZ','HENMT1','HERC2','HERPUD1','HFE','HIF1A','HIF3A','HINT1','HIP1R','HLA-A','HLA-DPB1','HMGCS1','HMOX1','HOMER3','HOXC5','HOXD4','HPS3','HRASLS5','HRH1','HS6ST3','HSH2D','HSPB1','HSPG2','HTR1B','HTR7','HTRA1','HUNK','HYDIN','HYI','HYLS1','ICAM1','ICAM2','ICOS','ICOSLG','IFFO2','IFIT5','IFNGR1','IFRD1','IFT122','IGFL2','IGSF11','IGSF23','IKZF5','IL10RA','IL12RB1','IL12RB2','IL13','IL13RA2','IL16','IL17RB','IL1R1','IL1RL1','IL21R','IL4R','IL6ST','IL7R','ILDR2','ILK','INPP5A','INSR','INTS12','IP6K2','IPCEF1','IQGAP1','IQGAP2','IREB2','IRF2','IRS1','ISLR','ITGA2B','ITGA3','ITGA4','ITGA7','ITGA8','ITGA9','ITGB1','ITGB2','ITGB3','ITGB4','ITGB5','ITGB6','ITGB7','ITGB8','ITK','ITM2A','ITPKA','ITSN1','ITSN2','IYD','JAG1','JAK2','JAK3','JAM2','JAML','JCAD','JMJD4','JPT2','JUP','KANK1','KANK3','KANSL2','KARS','KATNB1','KCNAB1','KCNB2','KCNH8','KCNJ15','KCNK13','KCNK17','KCNK6','KCNK7','KCNK9','KCNN2','KCNS3','KCNU1','KDF1','KDR','KEL','KIAA0556','KIAA0895','KIAA1324','KIAA1522','KIAA1958','KIF17','KIF19','KIF21A','KIF21B','KIF26A','KIF26B','KISS1R','KIT','KLC1','KLC2','KLF9','KLHDC7B','KLHL41','KLHL6','KLHL7','KLK7','KLRC1','KLRC2','KLRF1','KLRK1','KMT2D','KNDC1','KNSTRN','KNTC1','KPTN','KRT75','KRT77','KRTAP25-1','L1CAM','L3MBTL1','LAIR1','LAMA4','LAMC1','LAMP2','LAMTOR1','LAPTM4A','LASP1','LAT','LAT2','LAX1','LCP1','LCTL','LDB2','LDLRAD3','LENG1','LGALS3','LGALS4','LGALS8','LGR6','LHFPL2','LHX6','LIMA1','LINGO1','LIPJ','LMCD1','LMOD2','LMTK2','LMTK3','LNX1','LNX2','LONRF1','LONRF3','LOXHD1','LPAR1','LPAR3','LPAR4','LPIN1','LPIN2','LRCH2','LRP11','LRP12','LRRC15','LRRC26','LRRC34','LRRC45','LRRC69','LRRC7','LRRC9','LRRFIP2','LRRIQ3','LSMEM2','LSR','LTB4R2','LUZP2','LY6K','LYN','LYPD1','LYST','LZTS1','LZTS2','MACF1','MADD','MAGI1','MAGI3','MAMDC2','MAP1S','MAP2','MAP2K1','MAP2K4','MAP2K7','MAP3K12','MAP3K21','MAP3K6','MAP3K7','MAP4','MAP4K1','MAP4K5','MAP6','MAPK15','MAPK8IP1','MAPT','MARCH10','MARCH7','MARCKS','MARK2','MARVELD2','MAS1L','MAT2B','MBP','MC5R','MCAM','MCEMP1','MCF2L2','MCMBP','MCTS1','MDGA1','ME1','MEF2B','MEIOC','MEPCE','MERTK','MET','METAP1','METAP2','METTL15','METTL25','METTL9','MFAP3L','MFSD1','MFSD2A','MGAT4C','MIB1','MICAL2','MICAL3','MICALL1','MICB','MIEN1','MIGA2','MISP','MKLN1','MKRN3','MLF2','MLKL','MLST8','MMACHC','MMP10','MMP15','MMP24','MMP25','MOAP1','MPHOSPH8','MPHOSPH9','MPI','MPIG6B','MPL','MPP1','MPP5','MPP7','MPZL1','MPZL2','MR1','MRGPRF','MRNIP','MRPL42','MRPL44','MRPL46','MRPS18B','MS4A1','MS4A12','MS4A14','MS4A15','MSN','MSRA','MTCP1','MTFR1L','MTHFR','MTM1','MTPN','MTSS1','MUC1','MUC20','MVD','MYH11','MYH9','MYL4','MYLK','MYO10','MYO1B','MYO1C','MYO1E','MYO1G','MYO3A','MYO5B','MYO9A','MYOF','N4BP2','NAA35','NAB1','NAIF1','NAMPT','NAP1L2','NAPG','NARFL','NCAM1','NCAM2','NCK1','NCKAP1L','NCKAP5','NCKIPSD','NCOA1','NCS1','NDC1','NECTIN2','NECTIN3','NECTIN4','NEDD1','NEDD9','NEO1','NEU1','NEXN','NF2','NFE2L2','NFKBID','NGEF']
MembraneProteins2=['NGFR','NHS','NHSL2','NID2','NINJ2','NKTR','NKX2-8','NLGN1','NLGN3','NLK','NLRX1','NMB','NMBR','NME1','NME2','NMT1','NMT2','NMUR2','NOS1','NOTCH2','NPFFR2','NPL','NPNT','NPR1','NPTN','NPTXR','NR2C1','NRAS','NRCAM','NRP2','NRSN1','NRXN1','NSD1','NSFL1C','NT5E','NTNG1','NTNG2','NTSR1','NUDCD3','NUDT16L1','NUDT9','NUMB','NUP35','NUS1','OAS3','OBSCN','OCA2','OCLN','OCM','OCM2','ODC1','ODR4','OGT','OLFM4','OLIG2','OLR1','OPHN1','OPRK1','OR10P1','OR1N2','OR2C3','OR6B2','OR6B3','OR6C6','OR8S1','OSBPL10','OSBPL6','OSBPL7','OSCAR','OSCP1','OSGEP','OSR2','OTOP2','OTULIN','OXGR1','P3H3','PACSIN2','PACSIN3','PAFAH1B2','PAG1','PAIP1','PAK1','PAK4','PAK6','PALLD','PALM','PALM2','PALM2-AKAP2','PALM3','PALMD','PANX1','PANX2','PAQR8','PARD3','PARD3B','PARD6A','PARD6G','PARP6','PATJ','PAWR','PCBD2','PCDH1','PCDH17','PCDH7','PCDHA1','PCDHA11','PCDHA12','PCDHA4','PCDHA5','PCDHB1','PCDHB11','PCDHB12','PCDHB16','PCDHB4','PCDHB5','PCDHGA1','PCDHGA10','PCDHGA11','PCDHGA12','PCDHGA2','PCDHGA3','PCDHGA4','PCDHGA5','PCDHGA6','PCDHGA7','PCDHGA8','PCDHGA9','PCDHGB1','PCDHGB2','PCDHGB3','PCDHGB4','PCDHGB5','PCDHGB6','PCDHGB7','PCDHGC3','PCDHGC4','PCDHGC5','PCLO','PCMTD1','PDAP1','PDCD7','PDE3A','PDE4A','PDE4D','PDE9A','PDGFC','PDGFRA','PDHX','PDLIM1','PDLIM5','PDPN','PDZD2','PDZD8','PEAR1','PEBP1','PFDN1','PGAP3','PGLYRP2','PGLYRP4','PGRMC2','PHACTR1','PHACTR4','PHF20L1','PHGDH','PHLDB2','PHLDB3','PHOSPHO1','PHPT1','PI3','PI4K2A','PI4KA','PIEZO2','PIGB','PIGN','PIGW','PIGX','PIK3AP1','PIK3CG','PIK3R6','PIP4K2A','PIP4K2C','PIP5K1A','PIRT','PITPNM3','PKD2L2','PKN1','PKN2','PKP1','PKP2','PKP3','PKP4','PLAA','PLAUR','PLCB1','PLCB2','PLCD3','PLCD4','PLEKHA4','PLEKHA6','PLEKHA7','PLEKHG3','PLEKHG4','PLEKHG6','PLEKHH2','PLEKHO2','PLIN4','PLPP1','PLPPR5','PLS3','PLSCR1','PLVAP','PLXNA3','PLXNA4','PNCK','PNISR','PODXL','POLE','POU2F3','PPEF1','PPFIBP1','PPIP5K1','PPL','PPM1A','PPM1F','PPP1CA','PPP1CB','PPP1R12B','PPP1R16A','PPP1R16B','PPP1R18','PPP1R27','PPP1R9A','PPP2R1B','PPP3R1','PPP3R2','PPP4C','PPP6R3','PPTC7','PRAME','PRC1','PRDX6','PRIMA1','PRKCA','PRKCE','PRKCH','PRKCZ','PRKD1','PROM2','PROSER2','PRR15','PRR32','PRR7','PRRC2A','PRRG1','PRRT2','PRRT3','PRRT4','PRTG','PRX','PSCA','PSD','PSD2','PSEN1','PSMC5','PSMC6','PSMD9','PSTPIP2','PTAFR','PTCHD4','PTK2B','PTK6','PTPN12','PTPN21','PTPN22','PTPN4','PTPRB','PTPRG','PTPRK','PTPRM','PTPRR','PTPRS','PTX3','PURG','PVALB','PVR','PXDC1','PXK','PYGL','PYGO2','PYM1','RAB13','RAB23','RAB25','RAB26','RAB27B','RAB3B','RAB3GAP2','RAB43','RAB8A','RABEPK','RAD51C','RAD52','RAI2','RALA','RALGAPA2','RAPGEF2','RAPH1','RAPSN','RASA4','RASA4B','RASAL1','RASAL2','RASAL3','RASD1','RASD2','RASGRP1','RASGRP2','RASGRP4','RASSF2','RASSF3','RASSF4','RBM46','RC3H1','RCCD1','RCSD1','RD3L','RDX','RECK','RELL1','REPS1','RET','RFLNB','RFPL1','RFPL2','RFPL3','RGP1','RGS10','RGS14','RGS19','RGS3','RGS7BP','RGS9BP','RHCE','RHD','RHEX','RHOA','RHOBTB2','RHOJ','RHOXF2','RHOXF2B','RIC8A','RIC8B','RIF1','RILPL1','RIMBP3','RIMBP3B','RIMBP3C','RIMS2','RIOK2','RIOK3','RIPK1','RND1','RNF114','RNF145','RNF146','RNF149','RNF165','RNF166','RNF39','ROBO1','ROBO3','ROBO4','ROM1','RP2','RPE','RPEL1','RPGRIP1L','RPIA','RPL36A','RPL36A-HNRNPH2','RPL36AL','RPSA','RPUSD1','RRAD','RRP12','RRP7A','RSBN1L','RSC1A1','RSPH3','RTL3','RTL4','RTN4R','RTN4RL1','RYR2','S100A13','S100A14','S100A16','S100A2','S100A3','S100A4','S100A6','S100A9','SAMD4A','SAMHD1','SAMSN1','SAPCD1','SASH3','SC5D','SCAMP1','SCAMP4','SCEL','SCHIP1','SCIN','SCN1A','SCN5A','SCN8A','SCN9A','SCNN1D','SCNN1G','SCRIB','SCUBE3','SDC2','SDC4','SDE2','SDHB','SEC16B','SEMA3C','SEMA3D','SEMA4B','SEMA4F','SEMA6C','SEMA6D','SEPT2','SEPT3','SERINC2','SERINC5','SERPINB3','SFSWAP','SGCE','SGK494','SH3BGR','SH3D21','SHANK1','SHANK3','SHARPIN','SHISA6','SHKBP1','SHROOM2','SHTN1','SIGLEC10','SIGLEC14','SIGLEC6','SIGLEC7','SIN3B','SIPA1','SIPA1L1','SIPA1L3','SKAP1','SLAMF6','SLC10A5','SLC12A2','SLC13A3','SLC13A5','SLC14A1','SLC16A1','SLC16A10','SLC16A2','SLC16A3','SLC16A7','SLC16A9','SLC17A5','SLC19A1','SLC19A3','SLC1A5','SLC1A6','SLC22A16','SLC23A3','SLC24A4','SLC26A9','SLC27A1','SLC28A1','SLC2A1','SLC2A11','SLC2A12','SLC2A14','SLC2A2','SLC2A3','SLC30A1','SLC31A2','SLC34A1','SLC34A2','SLC35F5','SLC35G2','SLC36A4','SLC38A5','SLC38A6','SLC38A7','SLC39A10','SLC39A14','SLC39A2','SLC39A6','SLC3A2','SLC40A1','SLC41A3','SLC43A1','SLC43A3','SLC44A2','SLC45A4','SLC46A1','SLC46A3','SLC4A1AP','SLC4A2','SLC4A7','SLC4A8','SLC5A12','SLC5A3','SLC5A6','SLC5A7','SLC6A6','SLC7A14','SLC7A2','SLC7A5','SLC7A6','SLC8A1','SLC9A1','SLC9A2','SLC9A3','SLC9A3R1','SLC9A3R2','SLC9A4','SLC9A5','SLC9B2','SLC9C2','SLCO1A2','SLCO1B3','SLCO4A1','SLCO4C1','SLCO5A1','SLIT1','SLIT3','SLITRK6','SLK','SLX4','SMAGP','SMAP1','SMOC1','SMPD2','SMPX','SNAP23','SNAP25','SNRK','SNTB2','SNTG2','SNX15','SNX32','SNX9','SOCS3','SOCS5','SOD1','SORBS1','SORCS1','SORT1','SOX11','SPAG16','SPANXN4','SPART','SPATA12','SPATC1L','SPERT','SPG11','SPHK1','SPINK13','SPINT1','SPINT2','SPN','SPPL2B','SPPL3','SPRED3','SPRYD3','SPTBN2','SRC','SRCIN1','SRD5A3','SRPK1','SRPRB','SRRM3','SSFA2','SSH1','SSH3','SSTR1','SSTR5','SSX2IP','ST8SIA5','STAB1','STAC','STAC2','STAMBP','STAMBPL1','STARD4','STAT2','STEAP4','STIL','STIP1','STK10','STK16','STK17A','STK17B','STOM','STOML2','STOML3','STX18','STX4','STXBP4','STYK1','SUCLG1','SUCLG2','SUGT1','SUMF2','SUPT6H','SUSD2','SVIL','SWAP70','SYCP2','SYK','SYMPK','SYNC','SYNGR3','SYNPO2L','SYPL2','SYT4','SYTL2','SYVN1','TACC2','TACR2','TACSTD2','TASP1','TAZ','TBC1D10A','TBC1D2','TBC1D24','TBC1D30','TBC1D9B','TBXA2R','TCAF2','TCAF2C','TCHP','TCIM','TDG','TDO2','TDP1','TEC','TEDC2','TEK','TES','TEX14','TEX9','TGFBR1','TGFBR2','TGM1','TGM2','THBS1','THBS2','THEGL','THY1','TIAM1','TICAM1','TIGD7','TIMD4','TJP1','TJP2','TJP3','TLN1','TLR4','TM2D2','TM4SF20','TM4SF4','TM4SF5','TMC5','TMED8','TMEM101','TMEM107','TMEM108','TMEM121B','TMEM139','TMEM141','TMEM164','TMEM176B','TMEM183A','TMEM186','TMEM2','TMEM218','TMEM222','TMEM229A','TMEM245','TMEM251','TMEM254','TMEM258','TMEM266','TMEM268','TMEM50A','TMEM63B','TMPRSS15','TMPRSS2','TMTC2','TNFRSF10D','TNFRSF11A','TNFRSF12A','TNFRSF21','TNFRSF25','TNFRSF8','TNFRSF9','TNFSF13B','TNK1','TNKS1BP1','TNNI2','TOGARAM2','TOLLIP','TOM1','TP53BP2','TPD52L1','TPRN','TRAF7','TRAIP','TRAPPC13','TRAT1','TRDN','TREML1','TRIB1','TRIM16','TRIM16L','TRIM38','TRIM4','TRIM44','TRIM54','TRIM71','TRIML2','TRIP6','TRMT1','TRMT10B','TRMT13','TRMT6','TRO','TRPM1','TRPM4','TRPM7','TRPV2','TRPV3','TSG101','TSGA13','TSHZ3','TSPAN15','TSPAN6','TSPO2','TSR2','TSSK4','TTC17','TTC23','TTC7B','TTLL12','TTLL5','TULP3','TWF2','TXNDC11','TXNIP','TYROBP','TYW3','UAP1','UBA3','UBA52','UBAC1','UBALD1','UBAP1','UBASH3B','UBE2A','UBE2B','UBE2C','UBE2D1','UBE2D2','UBE2D3','UBE2D4','UBE2K','UBE2QL1','UBE2S','UBE3C','UBQLN2','UBR2','UBTD1','UCP3','ULBP1','ULBP2','UNC5A','UNC79','UQCC1','USP14','USP15','USP2','USP20','USP21','USP4','USP53','UTP20','UTRN','VAMP5','VAMP8','VANGL1','VASP','VCAM1','VCPIP1','VIL1','VPS13B','VPS36','VPS50','VPS8','VSIG4','VTCN1','VWA5B1','VWA7','WAS','WASF2','WBP4','WDFY3','WDPCP','WDR1','WDR12','WDR13','WDR53','WDR6','WDR66','WIPF1','WWP1','XIAP','XIRP1','XIRP2','XPC','XPNPEP2','XPO6','XRN1','XYLT1','YES1','YIPF1','YIPF4','YOD1','YTHDC1','ZBED3','ZBTB33','ZBTB8OS','ZDHHC2','ZDHHC20','ZDHHC22','ZDHHC5','ZFP41','ZFP90','ZFYVE28','ZGPAT','ZMYND10','ZNF10','ZNF18','ZNF185','ZNF2','ZNF20','ZNF227','ZNF229','ZNF236','ZNF276','ZNF277','ZNF337','ZNF397','ZNF407','ZNF497','ZNF503','ZNF510','ZNF536','ZNF571','ZNF598','ZNF599','ZNF646','ZNF655','ZNF688','ZNF706','ZNF717','ZNF783','ZNF79','ZNF839','ZSCAN22','ZSCAN29','ZSWIM1','ZSWIM8','ZYX']
MembraneProteins=MembraneProteins1+MembraneProteins2

TFs1=['AC008770.3','AC023509.3','AC092835.1','AC138696.1','ADNP','ADNP2','AEBP1','AEBP2','AHCTF1','AHDC1','AHR','AHRR','AIRE','AKAP8','AKAP8L','AKNA','ALX1','ALX3','ALX4','ANHX','ANKZF1','AR','ARGFX','ARHGAP35','ARID2','ARID3A','ARID3B','ARID3C','ARID5A','ARID5B','ARNT','ARNT2','ARNTL','ARNTL2','ARX','ASCL1','ASCL2','ASCL3','ASCL4','ASCL5','ASH1L','ATF1','ATF2','ATF3','ATF4','ATF5','ATF6','ATF6B','ATF7','ATMIN','ATOH1','ATOH7','ATOH8','BACH1','BACH2','BARHL1','BARHL2','BARX1','BARX2','BATF','BATF2','BATF3','BAZ2A','BAZ2B','BBX','BCL11A','BCL11B','BCL6','BCL6B','BHLHA15','BHLHA9','BHLHE22','BHLHE23','BHLHE40','BHLHE41','BNC1','BNC2','BORCS8-MEF2B','BPTF','BRF2','BSX','C11orf95','CAMTA1','CAMTA2','CARF','CASZ1','CBX2','CC2D1A','CCDC169-SOHLH2','CCDC17','CDC5L','CDX1','CDX2','CDX4','CEBPA','CEBPB','CEBPD','CEBPE','CEBPG','CEBPZ','CENPA','CENPB','CENPBD1','CENPS','CENPT','CENPX','CGGBP1','CHAMP1','CHCHD3','CIC','CLOCK','CPEB1','CPXCR1','CREB1','CREB3','CREB3L1','CREB3L2','CREB3L3','CREB3L4','CREB5','CREBL2','CREBZF','CREM','CRX','CSRNP1','CSRNP2','CSRNP3','CTCF','CTCFL','CUX1','CUX2','CXXC1','CXXC4','CXXC5','DACH1','DACH2','DBP','DBX1','DBX2','DDIT3','DEAF1','DLX1','DLX2','DLX3','DLX4','DLX5','DLX6','DMBX1','DMRT1','DMRT2','DMRT3','DMRTA1','DMRTA2','DMRTB1','DMRTC2','DMTF1','DNMT1','DNTTIP1','DOT1L','DPF1','DPF3','DPRX','DR1','DRAP1','DRGX','DUX1','DUX3','DUX4','DUXA','DZIP1','E2F1','E2F2','E2F3','E2F4','E2F5','E2F6','E2F7','E2F8','E4F1','EBF1','EBF2','EBF3','EBF4','EEA1','EGR1','EGR2','EGR3','EGR4','EHF','ELF1','ELF2','ELF3','ELF4','ELF5','ELK1','ELK3','ELK4','EMX1','EMX2','EN1','EN2','EOMES','EPAS1','ERF','ERG','ESR1','ESR2','ESRRA','ESRRB','ESRRG','ESX1','ETS1','ETS2','ETV1','ETV2','ETV3','ETV3L','ETV4','ETV5','ETV6','ETV7','EVX1','EVX2','FAM170A','FAM200B','FBXL19','FERD3L','FEV','FEZF1','FEZF2','FIGLA','FIZ1','FLI1','FLYWCH1','FOS','FOSB','FOSL1','FOSL2','FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXN4','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1','GABPA','GATA1','GATA2','GATA3','GATA4','GATA5','GATA6','GATAD2A','GATAD2B','GBX1','GBX2','GCM1','GCM2','GFI1','GFI1B','GLI1','GLI2','GLI3','GLI4','GLIS1','GLIS2','GLIS3','GLMP','GLYR1','GMEB1','GMEB2','GPBP1','GPBP1L1','GRHL1','GRHL2','GRHL3','GSC','GSC2','GSX1','GSX2','GTF2B','GTF2I','GTF2IRD1','GTF2IRD2','GTF2IRD2B','GTF3A','GZF1','HAND1','HAND2','HBP1','HDX','HELT','HES1','HES2','HES3','HES4','HES5','HES6','HES7','HESX1','HEY1','HEY2','HEYL','HHEX','HIC1','HIC2','HIF1A','HIF3A','HINFP','HIVEP1','HIVEP2','HIVEP3','HKR1','HLF','HLX','HMBOX1','HMG20A','HMG20B','HMGA1','HMGA2','HMGN3','HMX1','HMX2','HMX3','HNF1A','HNF1B','HNF4A','HNF4G','HOMEZ','HOXA1','HOXA10','HOXA11','HOXA13','HOXA2','HOXA3','HOXA4','HOXA5','HOXA6','HOXA7','HOXA9','HOXB1','HOXB13','HOXB2','HOXB3','HOXB4','HOXB5','HOXB6','HOXB7','HOXB8','HOXB9','HOXC10','HOXC11','HOXC12','HOXC13','HOXC4','HOXC5','HOXC6','HOXC8','HOXC9','HOXD1','HOXD10','HOXD11','HOXD12','HOXD13','HOXD3','HOXD4','HOXD8','HOXD9','HSF1','HSF2','HSF4','HSF5','HSFX1','HSFX2','HSFY1','HSFY2','IKZF1','IKZF2','IKZF3','IKZF4','IKZF5','INSM1','INSM2','IRF1','IRF2','IRF3','IRF4','IRF5','IRF6','IRF7','IRF8','IRF9','IRX1','IRX2','IRX3','IRX4','IRX5','IRX6','ISL1','ISL2','ISX','JAZF1','JDP2','JRK','JRKL','JUN','JUNB','JUND','KAT7','KCMF1','KCNIP3','KDM2A','KDM2B','KDM5B','KIN','KLF1','KLF10','KLF11','KLF12','KLF13','KLF14','KLF15','KLF16','KLF17','KLF2','KLF3','KLF4','KLF5','KLF6','KLF7','KLF8','KLF9','KMT2A','KMT2B','L3MBTL1','L3MBTL3','L3MBTL4','LBX1','LBX2','LCOR','LCORL','LEF1','LEUTX','LHX1','LHX2','LHX3','LHX4','LHX5','LHX6','LHX8','LHX9','LIN28A','LIN28B','LIN54','LMX1A','LMX1B','LTF','LYL1','MAF','MAFA','MAFB','MAFF','MAFG','MAFK','MAX','MAZ','MBD1','MBD2','MBD3','MBD4','MBD6','MBNL2','MECOM','MECP2','MEF2A','MEF2B','MEF2C','MEF2D','MEIS1','MEIS2','MEIS3','MEOX1','MEOX2','MESP1','MESP2','MGA','MITF','MIXL1','MKX','MLX','MLXIP','MLXIPL','MNT','MNX1','MSANTD1','MSANTD3','MSANTD4','MSC','MSGN1','MSX1','MSX2','MTERF1','MTERF2','MTERF3','MTERF4','MTF1','MTF2','MXD1','MXD3','MXD4','MXI1','MYB','MYBL1','MYBL2','MYC','MYCL','MYCN','MYF5','MYF6','MYNN','MYOD1','MYOG','MYPOP','MYRF','MYRFL','MYSM1','MYT1','MYT1L','MZF1','NACC2','NAIF1','NANOG','NANOGNB','NANOGP8','NCOA1','NCOA2','NCOA3','NEUROD1','NEUROD2','NEUROD4','NEUROD6','NEUROG1','NEUROG2','NEUROG3','NFAT5','NFATC1','NFATC2','NFATC3','NFATC4','NFE2','NFE2L1','NFE2L2','NFE2L3','NFE4','NFIA','NFIB','NFIC','NFIL3','NFIX','NFKB1','NFKB2','NFX1','NFXL1','NFYA','NFYB','NFYC','NHLH1','NHLH2','NKRF','NKX1-1','NKX1-2','NKX2-1','NKX2-2','NKX2-3','NKX2-4','NKX2-5','NKX2-6','NKX2-8','NKX3-1','NKX3-2','NKX6-1','NKX6-2','NKX6-3','NME2','NOBOX','NOTO','NPAS1','NPAS2','NPAS3','NPAS4','NR0B1','NR1D1','NR1D2','NR1H2','NR1H3','NR1H4','NR1I2','NR1I3','NR2C1','NR2C2','NR2E1','NR2E3','NR2F1','NR2F2','NR2F6','NR3C1','NR3C2','NR4A1','NR4A2','NR4A3','NR5A1','NR5A2','NR6A1','NRF1','NRL','OLIG1','OLIG2','OLIG3','ONECUT1','ONECUT2','ONECUT3','OSR1','OSR2','OTP','OTX1','OTX2','OVOL1','OVOL2','OVOL3','PA2G4','PATZ1','PAX1','PAX2','PAX3','PAX4','PAX5','PAX6','PAX7','PAX8','PAX9','PBX1','PBX2','PBX3','PBX4','PCGF2','PCGF6','PDX1','PEG3','PGR','PHF1','PHF19 ','PHF20','PHF21A','PHOX2A','PHOX2B','PIN1','PITX1','PITX2','PITX3','PKNOX1','PKNOX2','PLAG1','PLAGL1','PLAGL2','PLSCR1','POGK','POU1F1','POU2AF1','POU2F1','POU2F2','POU2F3','POU3F1','POU3F2','POU3F3','POU3F4','POU4F1','POU4F2','POU4F3','POU5F1','POU5F1B','POU5F2','POU6F1','POU6F2','PPARA','PPARD','PPARG','PRDM1','PRDM10','PRDM12','PRDM13','PRDM14','PRDM15','PRDM16','PRDM2','PRDM4','PRDM5','PRDM6','PRDM8','PRDM9','PREB','PRMT3','PROP1','PROX1','PROX2','PRR12','PRRX1','PRRX2','PTF1A','PURA','PURB','PURG','RAG1','RARA','RARB','RARG','RAX','RAX2','RBAK','RBCK1','RBPJ','RBPJL','RBSN','REL','RELA','RELB','REPIN1','REST','REXO4','RFX1','RFX2','RFX3','RFX4','RFX5','RFX6','RFX7','RFX8','RHOXF1','RHOXF2','RHOXF2B','RLF','RORA','RORB','RORC','RREB1','RUNX1','RUNX2','RUNX3','RXRA','RXRB','RXRG','SAFB','SAFB2','SALL1','SALL2','SALL3','SALL4','SATB1','SATB2','SCMH1','SCML4','SCRT1','SCRT2','SCX','SEBOX','SETBP1','SETDB1','SETDB2','SGSM2','SHOX','SHOX2','SIM1','SIM2','SIX1','SIX2','SIX3','SIX4','SIX5','SIX6','SKI','SKIL','SKOR1','SKOR2','SLC2A4RG','SMAD1','SMAD3','SMAD4','SMAD5','SMAD9','SMYD3','SNAI1','SNAI2','SNAI3','SNAPC2','SNAPC4','SNAPC5','SOHLH1','SOHLH2','SON','SOX1','SOX10','SOX11']
TFs2=['SOX12','SOX13','SOX14','SOX15','SOX17','SOX18','SOX2','SOX21','SOX3','SOX30','SOX4','SOX5','SOX6','SOX7','SOX8','SOX9','SP1','SP100','SP110','SP140','SP140L','SP2','SP3','SP4','SP5','SP6','SP7','SP8','SP9','SPDEF','SPEN','SPI1','SPIB','SPIC','SPZ1','SRCAP','SREBF1','SREBF2','SRF','SRY','ST18','STAT1','STAT2','STAT3','STAT4','STAT5A','STAT5B','STAT6','T','TAL1','TAL2','TBP','TBPL1','TBPL2','TBR1','TBX1','TBX10','TBX15','TBX18','TBX19','TBX2','TBX20','TBX21','TBX22','TBX3','TBX4','TBX5','TBX6','TCF12','TCF15','TCF20','TCF21','TCF23','TCF24','TCF3','TCF4','TCF7','TCF7L1','TCF7L2','TCFL5','TEAD1','TEAD2','TEAD3','TEAD4','TEF','TERB1','TERF1','TERF2','TET1','TET2','TET3','TFAP2A','TFAP2B','TFAP2C','TFAP2D','TFAP2E','TFAP4','TFCP2','TFCP2L1','TFDP1','TFDP2','TFDP3','TFE3','TFEB','TFEC','TGIF1','TGIF2','TGIF2LX','TGIF2LY','THAP1','THAP10','THAP11','THAP12','THAP2','THAP3','THAP4','THAP5','THAP6','THAP7','THAP8','THAP9','THRA','THRB','THYN1','TIGD1','TIGD2','TIGD3','TIGD4','TIGD5','TIGD6','TIGD7','TLX1','TLX2','TLX3','TMF1','TOPORS','TP53','TP63','TP73','TPRX1','TRAFD1','TRERF1','TRPS1','TSC22D1','TSHZ1','TSHZ2','TSHZ3','TTF1','TWIST1','TWIST2','UBP1','UNCX','USF1','USF2','USF3','VAX1','VAX2','VDR','VENTX','VEZF1','VSX1','VSX2','WIZ','WT1','XBP1','XPA','YBX1','YBX2','YBX3','YY1','YY2','ZBED1','ZBED2','ZBED3','ZBED4','ZBED5','ZBED6','ZBED9','ZBTB1','ZBTB10','ZBTB11','ZBTB12','ZBTB14','ZBTB16','ZBTB17','ZBTB18','ZBTB2','ZBTB20','ZBTB21','ZBTB22','ZBTB24','ZBTB25','ZBTB26','ZBTB3','ZBTB32','ZBTB33','ZBTB34','ZBTB37','ZBTB38','ZBTB39','ZBTB4','ZBTB40','ZBTB41','ZBTB42','ZBTB43','ZBTB44','ZBTB45','ZBTB46','ZBTB47','ZBTB48','ZBTB49','ZBTB5','ZBTB6','ZBTB7A','ZBTB7B','ZBTB7C','ZBTB8A','ZBTB8B','ZBTB9','ZC3H8','ZEB1','ZEB2','ZFAT','ZFHX2','ZFHX3','ZFHX4','ZFP1','ZFP14','ZFP2','ZFP28','ZFP3','ZFP30','ZFP37','ZFP41','ZFP42','ZFP57','ZFP62','ZFP64','ZFP69','ZFP69B','ZFP82','ZFP90','ZFP91','ZFP92','ZFPM1','ZFPM2','ZFX','ZFY','ZGLP1','ZGPAT','ZHX1','ZHX2','ZHX3','ZIC1','ZIC2','ZIC3','ZIC4','ZIC5','ZIK1','ZIM2','ZIM3','ZKSCAN1','ZKSCAN2','ZKSCAN3','ZKSCAN4','ZKSCAN5','ZKSCAN7','ZKSCAN8','ZMAT1','ZMAT4','ZNF10','ZNF100','ZNF101','ZNF107','ZNF112','ZNF114','ZNF117','ZNF12','ZNF121','ZNF124','ZNF131','ZNF132','ZNF133','ZNF134','ZNF135','ZNF136','ZNF138','ZNF14','ZNF140','ZNF141','ZNF142','ZNF143','ZNF146','ZNF148','ZNF154','ZNF155','ZNF157','ZNF16','ZNF160','ZNF165','ZNF169','ZNF17','ZNF174','ZNF175','ZNF177','ZNF18','ZNF180','ZNF181','ZNF182','ZNF184','ZNF189','ZNF19','ZNF195','ZNF197','ZNF2','ZNF20','ZNF200','ZNF202','ZNF205','ZNF207','ZNF208','ZNF211','ZNF212','ZNF213','ZNF214','ZNF215','ZNF217','ZNF219','ZNF22','ZNF221','ZNF222','ZNF223','ZNF224','ZNF225','ZNF226','ZNF227','ZNF229','ZNF23','ZNF230','ZNF232','ZNF233','ZNF234','ZNF235','ZNF236','ZNF239','ZNF24','ZNF248','ZNF25','ZNF250','ZNF251','ZNF253','ZNF254','ZNF256','ZNF257','ZNF26','ZNF260','ZNF263','ZNF264','ZNF266','ZNF267','ZNF268','ZNF273','ZNF274','ZNF275','ZNF276','ZNF277','ZNF28','ZNF280A','ZNF280B','ZNF280C','ZNF280D','ZNF281','ZNF282','ZNF283','ZNF284','ZNF285','ZNF286A','ZNF286B','ZNF287','ZNF292','ZNF296','ZNF3','ZNF30','ZNF300','ZNF302','ZNF304','ZNF311','ZNF316','ZNF317','ZNF318','ZNF319','ZNF32','ZNF320','ZNF322','ZNF324','ZNF324B','ZNF326','ZNF329','ZNF331','ZNF333','ZNF334','ZNF335','ZNF337','ZNF33A','ZNF33B','ZNF34','ZNF341','ZNF343','ZNF345','ZNF346','ZNF347','ZNF35','ZNF350','ZNF354A','ZNF354B','ZNF354C','ZNF358','ZNF362','ZNF365','ZNF366','ZNF367','ZNF37A','ZNF382','ZNF383','ZNF384','ZNF385A','ZNF385B','ZNF385C','ZNF385D','ZNF391','ZNF394','ZNF395','ZNF396','ZNF397','ZNF398','ZNF404','ZNF407','ZNF408','ZNF41','ZNF410','ZNF414','ZNF415','ZNF416','ZNF417','ZNF418','ZNF419','ZNF420','ZNF423','ZNF425','ZNF426','ZNF428','ZNF429','ZNF43','ZNF430','ZNF431','ZNF432','ZNF433','ZNF436','ZNF438','ZNF439','ZNF44','ZNF440','ZNF441','ZNF442','ZNF443','ZNF444','ZNF445','ZNF446','ZNF449','ZNF45','ZNF451','ZNF454','ZNF460','ZNF461','ZNF462','ZNF467','ZNF468','ZNF469','ZNF470','ZNF471','ZNF473','ZNF474','ZNF479','ZNF48','ZNF480','ZNF483','ZNF484','ZNF485','ZNF486','ZNF487','ZNF488','ZNF490','ZNF491','ZNF492','ZNF493','ZNF496','ZNF497','ZNF500','ZNF501','ZNF502','ZNF503','ZNF506','ZNF507','ZNF510','ZNF511','ZNF512','ZNF512B','ZNF513','ZNF514','ZNF516','ZNF517','ZNF518A','ZNF518B','ZNF519','ZNF521','ZNF524','ZNF525','ZNF526','ZNF527','ZNF528','ZNF529','ZNF530','ZNF532','ZNF534','ZNF536','ZNF540','ZNF541','ZNF543','ZNF544','ZNF546','ZNF547','ZNF548','ZNF549','ZNF550','ZNF551','ZNF552','ZNF554','ZNF555','ZNF556','ZNF557','ZNF558','ZNF559','ZNF560','ZNF561','ZNF562','ZNF563','ZNF564','ZNF565','ZNF566','ZNF567','ZNF568','ZNF569','ZNF57','ZNF570','ZNF571','ZNF572','ZNF573','ZNF574','ZNF575','ZNF576','ZNF577','ZNF578','ZNF579','ZNF580','ZNF581','ZNF582','ZNF583','ZNF584','ZNF585A','ZNF585B','ZNF586','ZNF587','ZNF587B','ZNF589','ZNF592','ZNF594','ZNF595','ZNF596','ZNF597','ZNF598','ZNF599','ZNF600','ZNF605','ZNF606','ZNF607','ZNF608','ZNF609','ZNF610','ZNF611','ZNF613','ZNF614','ZNF615','ZNF616','ZNF618','ZNF619','ZNF620','ZNF621','ZNF623','ZNF624','ZNF625','ZNF626','ZNF627','ZNF628','ZNF629','ZNF630','ZNF639','ZNF641','ZNF644','ZNF645','ZNF646','ZNF648','ZNF649','ZNF652','ZNF653','ZNF654','ZNF655','ZNF658','ZNF66','ZNF660','ZNF662','ZNF664','ZNF665','ZNF667','ZNF668','ZNF669','ZNF670','ZNF671','ZNF672','ZNF674','ZNF675','ZNF676','ZNF677','ZNF678','ZNF679','ZNF680','ZNF681','ZNF682','ZNF683','ZNF684','ZNF687','ZNF688','ZNF689','ZNF69','ZNF691','ZNF692','ZNF695','ZNF696','ZNF697','ZNF699','ZNF7','ZNF70','ZNF700','ZNF701','ZNF703','ZNF704','ZNF705A','ZNF705B','ZNF705D','ZNF705E','ZNF705G','ZNF706','ZNF707','ZNF708','ZNF709','ZNF71','ZNF710','ZNF711','ZNF713','ZNF714','ZNF716','ZNF717','ZNF718','ZNF721','ZNF724','ZNF726','ZNF727','ZNF728','ZNF729','ZNF730','ZNF732','ZNF735','ZNF736','ZNF737','ZNF74','ZNF740','ZNF746','ZNF747','ZNF749','ZNF750','ZNF75A','ZNF75D','ZNF76','ZNF761','ZNF763','ZNF764','ZNF765','ZNF766','ZNF768','ZNF77','ZNF770','ZNF771','ZNF772','ZNF773','ZNF774','ZNF775','ZNF776','ZNF777','ZNF778','ZNF780A','ZNF780B','ZNF781','ZNF782','ZNF783','ZNF784','ZNF785','ZNF786','ZNF787','ZNF788','ZNF789','ZNF79','ZNF790','ZNF791','ZNF792','ZNF793','ZNF799','ZNF8','ZNF80','ZNF800','ZNF804A','ZNF804B','ZNF805','ZNF808','ZNF81','ZNF813','ZNF814','ZNF816','ZNF821','ZNF823','ZNF827','ZNF829','ZNF83','ZNF830','ZNF831','ZNF835','ZNF836','ZNF837','ZNF84','ZNF841','ZNF843','ZNF844','ZNF845','ZNF846','ZNF85','ZNF850','ZNF852','ZNF853','ZNF860','ZNF865','ZNF878','ZNF879','ZNF880','ZNF883','ZNF888','ZNF891','ZNF90','ZNF91','ZNF92','ZNF93','ZNF98','ZNF99','ZSCAN1','ZSCAN10','ZSCAN12','ZSCAN16','ZSCAN18','ZSCAN2','ZSCAN20','ZSCAN21','ZSCAN22','ZSCAN23','ZSCAN25','ZSCAN26','ZSCAN29','ZSCAN30','ZSCAN31','ZSCAN32','ZSCAN4','ZSCAN5A','ZSCAN5B','ZSCAN5C','ZSCAN9','ZUFSP','ZXDA','ZXDB','ZXDC','ZZZ3']
TFs=TFs1+TFs2

#TFs other, inclusive of the above list, not necessarily DNA binding proteins but affect transcription
TFs1Other=['AATF','ABL1','ABRA','AC008770.3','AC020915.1','AC023509.3','AC092835.1','AC105001.2','AC138696.1','ACOD1','ACTR3B','ACTR3C','ADAM8','ADAMTS17','ADAMTS19','ADAR','ADGRA1','ADGRG3','ADNP','ADNP2','AEBP1','AEBP2','AES','AFAP1 ','AFF1','AFF3','AFF4','AGER','AGT','AHCTF1','AHDC1','AHR','AHRR','AIM2','AIRE','AKAP8','AKAP8L','AKNA','AKT1','ALK','ALKBH2','ALX1','ALX3','ALX4','AMH','ANAPC2','ANHX','ANKRD30A','ANKZF1','ANXA3','ANXA4','AP5Z1','APBB3','APC2','APEX1','APOA4','APOBEC3A','APOBEC3B','APOBEC3G','APTX','AR','ARGFX','ARHGAP35','ARHGEF2','ARHGEF5','ARID1A','ARID1B','ARID2','ARID3A','ARID3B','ARID3C','ARID4A','ARID4B','ARID5A','ARID5B','ARNT','ARNT2','ARNTL','ARNTL2','ARRB1','ARRB2','ARX','ASCC1','ASCL1','ASCL2','ASCL3','ASCL4','ASCL5','ASH1L','ATF1','ATF2','ATF3','ATF4','ATF5','ATF6','ATF6B','ATF7','ATF7IP','ATM','ATMIN','ATOH1','ATOH7','ATOH8','ATR','ATRX','ATXN7','BACH1','BACH2','BAG1','BANF1','BANP','BARHL1','BARHL2','BARX1','BARX2','BATF','BATF2','BATF3','BAZ2A','BAZ2B','BBX','BCL10','BCL11A','BCL11B','BCL3','BCL6','BCL6B','BCLAF1','BCOR','BDP1','BEND4','BEX1','BHLHA15','BHLHA9','BHLHE22','BHLHE23','BHLHE40','BHLHE41','BHMG1','BIN1','BLM','BLZF1','BMI1','BMP2','BMP4','BMP7','BMPR1A','BNC1','BNC2','BOD1L1','BOLA1','BOLA2','BOLA2B','BOLA3','BORCS8-MEF2B','BPNT1','BPTF','BRCA1','BRD4','BRD7','BRD8','BRD9','BRF1','BRF2','BRMS1','BRPF1','BSX','BTAF1','BTBD8','BTG2','BTK','BTRC','BUD31','C11orf95','C14orf39','C17orf49','C1D','C20orf194','C3orf33','CACTIN','CAMK1D','CAMK2A','CAMTA1','CAMTA2','CAPN15','CAPN3','CARD11','CARD14','CARF','CARHSP1','CASP8AP2','CASZ1','CAT','CBFA2T2','CBFA2T3','CBFB','CBL','CBLL1','CBX2','CC2D1A','CC2D1B','CCDC124','CCDC169-SOHLH2','CCDC17','CCDC3','CCDC71','CCDC83','CCNT1','CCNT2','CD36','CD40','CD40LG','CDC5L','CDC6','CDK5RAP3','CDK7','CDK8','CDK9','CDKN1A','CDKN2A','CDX1','CDX2','CDX4','CEBPA','CEBPB','CEBPD','CEBPE','CEBPG','CEBPZ','CELF3','CENPA','CENPB','CENPBD1','CENPS','CENPT','CENPX','CERS2','CERS3','CERS4','CERS5','CERS6','CFLAR','CGGBP1','CHAMP1','CHCHD3','CHD1','CHD2','CHD3','CHD4','CHD6','CHD7','CHD9','CHP1','CHRAC1','CHUK','CHURC1','CIB1','CIC','CIITA','CIR1','CITED1','CITED2','CITED4','CIZ1','CLOCK','CLU','CMKLR1','CNBP','CNOT3','CNOT4','CNOT7','CNOT8','COMMD1','COMMD6','COMMD7','COPS2','COPS3','COPS4','CPEB1','CPSF4','CPSF4L','CPXCR1','CRAMP1','CREB1','CREB3','CREB3L1','CREB3L2','CREB3L3','CREB3L4','CREB5','CREBBP','CREBL2','CREBRF','CREBZF','CREG1','CREM','CRTC1','CRTC2','CRTC3','CRX','CSDC2','CSDE1','CSRNP1','CSRNP2','CSRNP3','CTBP1','CTBP2','CTCF','CTCFL','CTH','CTNNB1','CTNNBIP1','CUL1','CUL2','CUL3','CUL4A','CUL4B','CUL5','CUX1','CUX2','CXXC1','CXXC4','CXXC5','CYLD','CYP1B1','CYTL1','DAB2IP','DACH1','DACH2','DAP','DAXX','DBP','DBX1','DBX2','DCP1A','DDB1','DDB2','DDIT3','DDN','DDR2','DDRGK1','DDX3X','DDX5','DDX58','DEAF1','DEK','DEPDC1','DEPDC1B','DEPDC4','DEPDC5','DEPDC7','DEPTOR','DHX34','DHX36','DHX57','DISP3','DLX1','DLX2','DLX3','DLX4','DLX5','DLX6','DMAP1','DMBX1','DMC1','DMRT1','DMRT2','DMRT3','DMRTA1','DMRTA2','DMRTB1','DMRTC1','DMRTC2','DMTF1','DNA2','DNAJA3','DNAJC1','DNAJC2','DNAJC21','DNMT1','DNMT3A','DNMT3B','DNMT3L','DNTT','DNTTIP1','DOT1L','DPF1','DPF2','DPF3','DPRX','DR1','DRAP1','DRGX','DSP','DTX1','DUS3L','DUSP12','DUX1','DUX3','DUX4','DUXA','DVL1','DVL2','DVL3','DZIP1','DZIP1L','E2F1','E2F2','E2F3','E2F4','E2F5','E2F6','E2F7','E2F8','E4F1','EAF2','EBF1','EBF2','EBF3','EBF4','EDA','EDA2R','EDF1','EDN1','EEA1','EED','EGLN1','EGR1','EGR2','EGR3','EGR4','EHF','EID1','EID2','EIF2AK2','EIF2AK4','EIF3C','EIF3CL','EIF3K','ELF1','ELF2','ELF3','ELF4','ELF5','ELK1','ELK3','ELK4','ELMSAN1','EME1','EMX1','EMX2','EN1','EN2','ENO1','EOMES','EP300','EP400','EPAS1','EPHA5','EPM2AIP1','ERBIN','ERC1','ERCC3','ERCC4','ERF','ERG','ESR1','ESR2','ESRRA','ESRRB','ESRRG','ESX1','ETS1','ETS2','ETV1','ETV2','ETV3','ETV3L','ETV4','ETV5','ETV6','ETV7','EVX1','EVX2','EWSR1','EXO1','EXOC2','EXOG','EZH1','EZH2','FAAP24','FAM153C','FAM170A','FAM171B','FAM200A','FAM200B','FAN1','FANCA','FANCD2','FANK1','FARSA','FARSB','FBN1','FBXL19','FBXO41','FEN1','FER','FERD3L','FEV','FEZF1','FEZF2','FGD1','FHL2','FIGLA','FIZ1','FLI1','FLNA','FLYWCH1','FMNL2','FOS','FOSB','FOSL1','FOSL2','FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L2','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXN4','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1','FUBP1','FUBP3','FUS','FZD1','FZD2','FZD4','FZD6','GABPA','GABPB1','GABPB2','GADD45A','GAS6','GAS7','GATA1','GATA2','GATA3','GATA4','GATA5','GATA6','GATAD1','GATAD2A','GATAD2B','GBX1','GBX2','GCFC2','GCM1','GCM2','GEN1','GFI1','GFI1B','GLA','GLI1','GLI2','GLI3','GLI4','GLIS1','GLIS2','GLIS3','GLMP','GLYR1','GMEB1','GMEB2','GOLGB1','GON4L','GPATCH8','GPBP1','GPBP1L1','GPER1','GPR155','GREM1','GRHL1','GRHL2','GRHL3','GRM6','GSC','GSC2','GSX1','GSX2','GTF2A1','GTF2A1L','GTF2A2','GTF2B','GTF2E2','GTF2F1','GTF2F2','GTF2H1','GTF2H2','GTF2H2C','GTF2H3','GTF2H4','GTF2I','GTF2IRD1','GTF2IRD2','GTF2IRD2B','GTF3A','GTF3C1','GTF3C2','GULP1','GZF1','H1F0','H1FOO','H1FX','HAND1','HAND2','HAVCR2','HBP1','HCFC1','HCK','HDAC1','HDAC10','HDAC2','HDAC3','HDAC4','HDAC5','HDAC6','HDAC7','HDAC8','HDAC9','HDX','HELT','HELZ','HELZ2','HES1','HES2','HES3','HES4','HES5','HES6','HES7','HESX1','HEY1','HEY2','HEYL','HHEX','HIC1','HIC2','HIF1A','HIF3A','HINFP','HIPK2','HIRA','HIST1H1A','HIST1H1B','HIST1H1C','HIST1H1D','HIST1H1E','HIST1H1T','HIVEP1','HIVEP2','HIVEP3','HKR1','HLA-DQB1','HLA-DQB2','HLA-DRB1','HLA-DRB3','HLA-DRB5','HLF','HLTF','HLX','HMBOX1','HMG20A','HMG20B','HMGA1','HMGA2','HMGB1','HMGB2','HMGB3','HMGB4','HMGN1','HMGN3','HMGXB3','HMGXB4','HMOX1','HMX1','HMX2','HMX3','HNF1A','HNF1B','HNF4A','HNF4G','HNRNPA1','HNRNPA3','HNRNPAB','HNRNPCL2','HNRNPD','HNRNPDL','HNRNPK','HNRNPL','HNRNPU','HOMEZ','HOOK2','HOPX','HOXA1','HOXA10','HOXA11','HOXA13','HOXA2','HOXA3','HOXA4','HOXA5','HOXA6','HOXA7','HOXA9','HOXB1','HOXB13','HOXB2','HOXB3','HOXB4','HOXB5','HOXB6','HOXB7','HOXB8','HOXB9','HOXC10','HOXC11','HOXC12','HOXC13','HOXC4','HOXC5','HOXC6','HOXC8','HOXC9','HOXD1','HOXD10','HOXD11','HOXD12','HOXD13','HOXD3','HOXD4','HOXD8','HOXD9','HP1BP3','HR','HSBP1','HSF1','HSF2','HSF4','HSF5','HSFX1','HSFX2','HSFY1','HSFY2','HSPA1A','HSPA1B','ICAM1','ID1','ID2','ID3','ID4','IFI16','IFNB1','IGHM','IKBKB','IKBKG','IKZF1','IKZF2','IKZF3','IKZF4','IKZF5','IL10','IL1B','IL1RAP','IL4','IL5','IL6','INF2','ING4','INS','INSM1','INSM2','IQGAP1','IRAK1','IRAK2','IRAK3','IRF1','IRF2','IRF2BP1','IRF3','IRF4','IRF5','IRF6','IRF7','IRF8','IRF9','IRX1','IRX2','IRX3','IRX4','IRX5','IRX6','ISL1','ISL2','ISX','ITCH','ITGB2','ITGB3BP','IZUMO2','JADE1','JAK2','JARID2','JAZF1','JDP2','JMY','JRK','JRKL','JUN','JUNB','JUND','JUP','KANSL2','KAT2A','KAT2B','KAT5','KAT6A','KAT6B','KAT7','KAT8','KCMF1','KCNIP3','KDM1A','KDM2A','KDM2B','KDM3A','KDM5A','KDM5B','KDM5C','KDM5D','KEAP1','KIAA1549','KIAA1683','KIAA1841','KIAA1958','KIF15','KIN','KIT','KLF1','KLF10','KLF11','KLF12','KLF13','KLF14','KLF15','KLF16','KLF17','KLF2','KLF3','KLF4','KLF5','KLF6','KLF7','KLF8','KLF9','KMT2A','KMT2B','KMT2C','KMT2D','KNTC1','KRAS','KRBOX1','KRT13','KRTAP5-1','KRTAP5-9','KRTAP9-9','L3MBTL1','L3MBTL3','L3MBTL4','LARP1','LARP1B','LARP4','LARP4B','LARP6','LARP7','LBX1','LBX2','LCOR','LCORL','LEF1','LEUTX','LGALS9','LGR4','LHX1','LHX2','LHX3','LHX4','LHX5','LHX6','LHX8','LHX9','LIG1','LIG3','LIN28A','LIN28B','LIN54','LMO2','LMO4','LMX1A','LMX1B','LPP','LRP5','LRP6','LRP8','LRRFIP1','LRRFIP2','LSR','LTF','LTN1','LYAR','LYL1','LZTFL1','LZTR1','LZTS1','MACC1','MACF1','MAD2L2','MAEL','MAF','MAFA','MAFB','MAFF','MAFG','MAFK','MALT1','MAML1','MAML3','MAP2K5','MAP3K10','MAP3K13','MAP3K7','MAPK1','MAPK10','MAPK11','MAPK14','MAPK3','MAPK8','MAPK9','MATR3','MAVS','MAX','MAZ','MB21D1','MBD1','MBD2','MBD3','MBD4','MBD6','MBNL1','MBNL2','MBNL3','MBTPS2','MDFI','MDM2','MDM4','MECOM','MECP2','MED1','MED12','MED13','MED13L','MED14','MED15','MED16','MED17','MED21','MED23','MED24','MED6','MEF2A','MEF2B','MEF2C','MEF2D','MEIS1','MEIS2','MEIS3','MEN1','MEOX1','MEOX2','MESP1','MESP2','MET','MGA','MGMT','MID2','MIER1','MIER2','MIER3','MINK1','MIS18BP1','MITF','MIXL1','MKL1','MKL2','MKRN1','MKRN2','MKRN3','MKX','MLLT1','MLLT10','MLLT11','MLLT3','MLLT6','MLX','MLXIP','MLXIPL','MN1','MNAT1','MNT','MNX1','MORC3','MPG','MRPL28','MRRF','MSANTD1','MSANTD3','MSANTD4','MSC','MSGN1','MSH2','MSH3','MSH6','MSL3','MSRB2','MST1R','MSX1','MSX2','MTA1','MTA2','MTA3','MTDH','MTERF1','MTERF2','MTERF3','MTERF4','MTF1','MTF2','MTPN','MUS81','MXD1','MXD3','MXD4','MXI1','MYB','MYBBP1A','MYBL1','MYBL2','MYC','MYCBP','MYCL','MYCN','MYD88','MYEF2','MYF5','MYF6','MYNN','MYOCD','MYOD1','MYOG','MYPOP','MYRF','MYRFL','MYSM1','MYT1','MYT1L','MZF1','NAB2','NABP2','NACC2','NAIF1','NANOG','NANOGNB','NANOGP8','NCOA1','NCOA2','NCOA3','NCOA4','NCOA5','NCOA6','NCOA7','NCOR1','NCOR2','NDC80','NDN','NDP','NEB','NEIL1','NEUROD1','NEUROD2','NEUROD4','NEUROD6','NEUROG1','NEUROG2','NEUROG3','NFAM1','NFAT5','NFATC1','NFATC2','NFATC3','NFATC4','NFE2','NFE2L1','NFE2L2','NFE2L3','NFE4','NFIA','NFIB','NFIC','NFIL3','NFIX','NFKB1','NFKB2','NFKBIA','NFKBIB','NFKBID','NFKBIL1','NFKBIZ','NFRKB','NFX1','NFXL1','NFYA','NFYB','NFYC','NHLH1','NHLH2','NKRF','NKX1-1','NKX1-2','NKX2-1','NKX2-2','NKX2-3','NKX2-4','NKX2-5','NKX2-6','NKX2-8','NKX3-1','NKX3-2','NKX6-1','NKX6-2','NKX6-3','NLRC3','NLRC4','NLRC5','NLRP12','NLRP2B','NLRP3','NME2','NOBOX','NOC3L','NOC4L','NOCT','NOD1','NOD2','NODAL','NONO','NOP2','NOTCH1','NOTCH2','NOTCH3','NOTO','NPAS1','NPAS2','NPAS3','NPAS4','NPM1','NR0B1','NR0B2','NR1D1','NR1D2','NR1H2','NR1H3','NR1H4','NR1I2','NR1I3','NR2C1','NR2C2','NR2E1','NR2E3','NR2F1','NR2F2','NR2F6','NR3C1','NR3C2','NR4A1','NR4A2','NR4A3','NR5A1','NR5A2','NR6A1','NRBF2','NRF1','NRG1','NRIP1','NRIP3','NRK','NRL','NSD1','NSD2','NTRK1','NUCKS1','NUFIP1','NUP98','NUPL2']
TFs2Other=['OGG1','OLIG1','OLIG2','OLIG3','ONECUT1','ONECUT2','ONECUT3','OPRD1','OSR1','OSR2','OTOP3','OTP','OTULIN','OTX1','OTX2','OVOL1','OVOL2','OVOL3','PA2G4','PABPN1','PARK7','PARP1','PARP10','PARP12','PATZ1','PAWR','PAX1','PAX2','PAX3','PAX4','PAX5','PAX6','PAX7','PAX8','PAX9','PAXBP1','PAXIP1','PBRM1','PBX1','PBX2','PBX3','PBX4','PCBD1','PCBP1','PCBP2','PCBP3','PCGF2','PCGF6','PCNA','PCSK6','PDCD7','PDS5B','PDX1','PEG3','PELI1','PER1','PER2','PER3','PEX14','PFDN1','PGBD1','PGR','PGS1','PHB','PHB2','PHF1','PHF19 ','PHF20','PHF21A','PHF5A','PHOX2A','PHOX2B','PHTF1','PHTF2','PIAS1','PIAS2','PIAS3','PIAS4','PIDD1','PIKFYVE','PIM1','PIN1','PINK1','PITX1','PITX2','PITX3','PKHD1','PKHD1L1','PKNOX1','PKNOX2','PLA2G10','PLA2G1B','PLAG1','PLAGL1','PLAGL2','PLEK','PLEK2','PLEKHA4','PLPP3','PLSCR1','PLXNA1','PLXNA2','PLXNA3','PLXNA4','PLXNB1','PLXNB2','PLXNB3','PLXNC1','PLXND1','PML','PMS1','PNKP','POGK','POGZ','POLA1','POLB','POLE3','POLE4','POLG','POLH','POLI','POLK','POLL','POLM','POLN','POLQ','POLR1B','POLR2A','POLR2B','POLR2E','POLR2J','POLR2K','POLR2L','POLR3A','POLRMT','POT1','POU1F1','POU2AF1','POU2F1','POU2F2','POU2F3','POU3F1','POU3F2','POU3F3','POU3F4','POU4F1','POU4F2','POU4F3','POU5F1','POU5F1B','POU5F2','POU6F1','POU6F2','PPARA','PPARD','PPARG','PPARGC1A','PPARGC1B','PPP1R10','PPP1R13L','PPP1R21','PPP2R3B','PPP2R5B','PPP3CA','PPRC1','PQBP1','PRAM1','PRAP1','PRB3','PRDM1','PRDM10','PRDM11','PRDM12','PRDM13','PRDM14','PRDM15','PRDM16','PRDM2','PRDM4','PRDM5','PRDM6','PRDM7','PRDM8','PRDM9','PRDX3','PREB','PREX1','PREX2','PRH2','PRIM2','PRIMPOL','PRKCB','PRKCH','PRKCI','PRKCQ','PRKCZ','PRKD1','PRKD2','PRKN','PRMT2','PRMT3','PRNP','PROP1','PROX1','PROX2','PRR12','PRR3','PRRX1','PRRX2','PSMA6','PSMC5','PSMD10','PSMD11','PSMD12','PSPC1','PTCH1','PTEN','PTF1A','PTGIS','PTH','PTTG1','PUM3','PURA','PURB','PURG','PYCARD','PYDC1','PYDC2','QRICH1','RAB11FIP4','RAB7B','RAD21','RAD51','RAG1','RAI1','RAPGEF3','RAPGEF4','RAPGEF5','RARA','RARB','RARG','RAX','RAX2','RB1','RBAK','RBBP7','RBCK1','RBL1','RBL2','RBM10','RBM20','RBM22','RBM26','RBM27','RBM5','RBM6','RBPJ','RBPJL','RBSN','RC3H1','RC3H2','RCAN1','RCOR1','RCOR2','RCOR3','RECQL','REL','RELA','RELB','RELL2','RELN','REPIN1','RERE','REST','REV1','REXO4','RFC1','RFX1','RFX2','RFX3','RFX4','RFX5','RFX6','RFX7','RFX8','RFXANK','RFXAP','RGCC','RGS11','RGS6','RGS7','RGS9','RHEBL1','RHOXF1','RHOXF2','RHOXF2B','RING1','RIOK2','RIPK1','RIPK2','RIPK3','RIPK4','RLF','RLIM','RNASE2','RNASEH1','RNF113A','RNF113B','RNF114','RNF125','RNF138','RNF14','RNF166','RNF2','RNF219','RNF25','RNF31','RNF4','RNF41','RORA','RORB','RORC','RPA2','RPA4','RPL36A','RPL7','RPL7A','RPS27A','RPS6KA4','RPS6KA5','RREB1','RRN3','RUNX1','RUNX1T1','RUNX2','RUNX3','RWDD3','RXRA','RXRB','RXRG','RYBP','S100A12','S100A8','S100A9','SAFB','SAFB2','SALL1','SALL2','SALL3','SALL4','SART1','SATB1','SATB2','SAV1','SCAND1','SCAND2P','SCAPER','SCMH1','SCML1','SCML2','SCML4','SCOC','SCRT1','SCRT2','SCX','SEBOX','SEMA4A','SERTAD1','SERTAD2','SETBP1','SETD6','SETDB1','SETDB2','SF3A2','SF3A3','SFPQ','SFRP4','SFRP5','SFTPB','SGK1','SGK3','SGSM2','SHH','SHOX','SHOX2','SHPRH','SIGIRR','SIK1','SIM1','SIM2','SIN3A','SIN3B','SIRT1','SIRT2','SIRT3','SIRT6','SIVA1','SIX1','SIX2','SIX3','SIX4','SIX5','SIX6','SKI','SKIDA1','SKIL','SKOR1','SKOR2','SLC22A4','SLC26A10','SLC26A3','SLC2A4RG','SLC30A9','SLC39A10','SLC4A10','SMAD1','SMAD2','SMAD3','SMAD4','SMAD5','SMAD6','SMAD7','SMAD9','SMARCA1','SMARCA2','SMARCA4','SMARCA5','SMARCAL1','SMARCB1','SMARCC1','SMARCC2','SMARCD1','SMARCD2','SMARCD3','SMARCE1','SMC3','SMO','SMUG1','SMYD1','SMYD3','SNAI1','SNAI2','SNAI3','SNAPC2','SNAPC4','SNAPC5','SNRPC','SNURF','SNW1','SOGA3','SOHLH1','SOHLH2','SON','SORBS2','SOX1','SOX10','SOX11','SOX12','SOX13','SOX14','SOX15','SOX17','SOX18','SOX2','SOX21','SOX3','SOX30','SOX4','SOX5','SOX6','SOX7','SOX8','SOX9','SP1','SP100','SP110','SP140','SP140L','SP2','SP3','SP4','SP5','SP6','SP7','SP8','SP9','SPDEF','SPEN','SPHK1','SPI1','SPIB','SPIC','SPZ1','SRCAP','SREBF1','SREBF2','SRF','SRY','SSB','SSBP2','SSBP3','SSBP4','SSH1','SSH2','SSH3','SSRP1','SSX1','ST18','STAG1','STAT1','STAT2','STAT3','STAT4','STAT5A','STAT5B','STAT6','STK16','STK3','STK36','STRN','STRN3','SUB1','SUFU','SUMO1','SUPT16H','SUPT4H1','SUPT5H','SUPT6H','SUV39H1','SUV39H2','SUZ12','SYK','T','TAB1','TAB2','TAB3','TADA2A','TADA2B','TADA3','TAF1','TAF10','TAF11','TAF12','TAF13','TAF1A','TAF1B','TAF1C','TAF1L','TAF2','TAF3','TAF4','TAF4B','TAF5','TAF5L','TAF6','TAF6L','TAF7','TAF7L','TAF8','TAF9','TAF9B','TAL1','TAL2','TARDBP','TAX1BP1','TBL1Y','TBP','TBPL1','TBPL2','TBR1','TBX1','TBX10','TBX15','TBX18','TBX19','TBX2','TBX20','TBX21','TBX22','TBX3','TBX4','TBX5','TBX6','TCEA2','TCEAL1','TCEAL7','TCEAL8','TCF12','TCF15','TCF19','TCF20','TCF21','TCF23','TCF24','TCF25','TCF3','TCF4','TCF7','TCF7L1','TCF7L2','TCFL5','TDG','TDP1','TDP2','TEAD1','TEAD2','TEAD3','TEAD4','TEF','TERB1','TERF1','TERF2','TERF2IP','TESMIN','TET1','TET2','TET3','TFAM','TFAP2A','TFAP2B','TFAP2C','TFAP2D','TFAP2E','TFAP4','TFCP2','TFCP2L1','TFDP1','TFDP2','TFDP3','TFE3','TFEB','TFEC','TGFB1','TGFB1I1','TGIF1','TGIF2','TGIF2LX','TGIF2LY','TGS1','THAP1','THAP10','THAP11','THAP12','THAP2','THAP3','THAP4','THAP5','THAP6','THAP7','THAP8','THAP9','THRA','THRAP3','THRB','THYN1','TIA1','TICAM1','TIGD1','TIGD2','TIGD3','TIGD4','TIGD5','TIGD6','TIGD7','TIPARP','TIRAP','TLE1','TLE2','TLE3','TLE4','TLE6','TLR2','TLR3','TLR4','TLR9','TLX1','TLX2','TLX3','TMEM235','TMF1','TNF','TNFAIP3','TNFRSF11A','TNFRSF4','TNFSF11','TNFSF18','TNFSF4','TOE1','TONSL','TOP1','TOP2A','TOP2B','TOPORS','TOX','TOX2','TOX3','TOX4','TP53','TP53BP1','TP63','TP73','TPRX1','TRADD','TRAF1','TRAF2','TRAF3','TRAF4','TRAF5','TRAF6','TRAFD1','TRAK2','TRAPPC9','TRERF1','TREX1','TRIB1','TRIB3','TRIM13','TRIM14','TRIM15','TRIM21','TRIM22','TRIM23','TRIM24','TRIM25','TRIM26','TRIM27','TRIM28','TRIM29','TRIM3','TRIM31','TRIM32','TRIM33','TRIM34','TRIM37','TRIM38','TRIM40','TRIM5','TRIM52','TRIM62','TRIM8','TRIP11','TRIT1','TRMT1','TRMT1L','TRPS1','TRRAP','TSC2','TSC22D1','TSC22D2','TSC22D3','TSC22D4','TSHZ1','TSHZ2','TSHZ3','TSN','TSSK4','TTF1','TUB','TULP4','TUT1','TWIST1','TWIST2','TXK','U2AF1','U2AF1L4','UBA52','UBB','UBC','UBE2K','UBE2N','UBE2V1','UBN1','UBP1','UBR4','UBTF','UBTFL1','UBTFL6','UFL1','UHRF1','UHRF2','UNCX','UNG','UNK','UNKL','USF1','USF2','USF3','USP39','USP7','UTF1','VAV1','VAX1','VAX2','VDR','VEGFA','VENTX','VEZF1','VGLL1','VGLL2','VGLL3','VHLL','VPS72','VSX1','VSX2','WDHD1','WFS1','WIZ','WNT1','WNT10B','WNT2','WNT3A','WNT5A','WNT8B','WRN','WT1','WWOX','WWP2','WWTR1','XBP1','XCL1','XPA','YBX1','YBX2','YBX3','YEATS2','YEATS4','YOD1','YY1','YY2','ZBED1','ZBED2','ZBED3','ZBED4','ZBED5','ZBED6','ZBED6CL','ZBED8','ZBED9','ZBP1','ZBTB1','ZBTB10','ZBTB11','ZBTB12','ZBTB14','ZBTB16','ZBTB17','ZBTB18','ZBTB2','ZBTB20','ZBTB21','ZBTB22','ZBTB24','ZBTB25','ZBTB26','ZBTB3','ZBTB32','ZBTB33','ZBTB34','ZBTB37','ZBTB38','ZBTB39','ZBTB4','ZBTB40','ZBTB41','ZBTB42','ZBTB43','ZBTB44','ZBTB45','ZBTB46','ZBTB47','ZBTB48','ZBTB49','ZBTB5','ZBTB6','ZBTB7A','ZBTB7B','ZBTB7C','ZBTB8A','ZBTB8B','ZBTB9','ZC3H10','ZC3H11A','ZC3H12A','ZC3H13','ZC3H14','ZC3H15','ZC3H18','ZC3H3','ZC3H4','ZC3H6','ZC3H7A','ZC3H7B','ZC3H8','ZC3HAV1','ZCCHC11','ZCCHC6','ZDHHC11','ZDHHC19','ZEB1','ZEB2','ZFAND5','ZFAT','ZFHX2','ZFHX3','ZFHX4','ZFP1','ZFP14','ZFP2','ZFP28','ZFP3','ZFP30','ZFP36','ZFP36L1','ZFP36L2','ZFP37','ZFP41','ZFP42','ZFP57','ZFP62','ZFP64','ZFP69','ZFP69B','ZFP82','ZFP90','ZFP91','ZFP92','ZFPL1','ZFPM1','ZFPM2','ZFR','ZFR2','ZFX','ZFY','ZFYVE26','ZGLP1','ZGPAT','ZHX1','ZHX2','ZHX3','ZIC1','ZIC2','ZIC3','ZIC4','ZIC5','ZIK1','ZIM2','ZIM3','ZKSCAN1','ZKSCAN2','ZKSCAN3','ZKSCAN4','ZKSCAN5','ZKSCAN7','ZKSCAN8','ZMAT1','ZMAT2','ZMAT3','ZMAT4','ZMAT5','ZMYM1','ZMYM2','ZMYM3','ZMYM4','ZMYM5','ZMYM6','ZMYND10','ZNF10','ZNF100','ZNF101','ZNF106','ZNF107','ZNF112','ZNF114','ZNF117','ZNF12','ZNF121','ZNF124','ZNF131','ZNF132','ZNF133','ZNF134','ZNF135','ZNF136','ZNF137P','ZNF138','ZNF14','ZNF140','ZNF141','ZNF142','ZNF143','ZNF146','ZNF148','ZNF154','ZNF155','ZNF157','ZNF16','ZNF160','ZNF165','ZNF169','ZNF17','ZNF174','ZNF175','ZNF177','ZNF18','ZNF180','ZNF181','ZNF182','ZNF184','ZNF185','ZNF189','ZNF19','ZNF195','ZNF197','ZNF2','ZNF20','ZNF200','ZNF202','ZNF205','ZNF207','ZNF208','ZNF211','ZNF212','ZNF213','ZNF214','ZNF215','ZNF217','ZNF219','ZNF22','ZNF221','ZNF222','ZNF223','ZNF224','ZNF225','ZNF226','ZNF227','ZNF229','ZNF23','ZNF230','ZNF232','ZNF233','ZNF234','ZNF235','ZNF236','ZNF239','ZNF24','ZNF248','ZNF25','ZNF250','ZNF251','ZNF253','ZNF254','ZNF256','ZNF257','ZNF26','ZNF260','ZNF263','ZNF264','ZNF266','ZNF267','ZNF268','ZNF273','ZNF274','ZNF275','ZNF276','ZNF277','ZNF28','ZNF280A','ZNF280B','ZNF280C','ZNF280D','ZNF281','ZNF282','ZNF283','ZNF284','ZNF285','ZNF286A','ZNF286B','ZNF287','ZNF292','ZNF296','ZNF3','ZNF30','ZNF300','ZNF302','ZNF304','ZNF311','ZNF316','ZNF317','ZNF318','ZNF319','ZNF32','ZNF320','ZNF322','ZNF324','ZNF324B','ZNF326','ZNF329','ZNF330','ZNF331','ZNF333','ZNF334','ZNF335','ZNF337','ZNF33A','ZNF33B','ZNF34','ZNF341','ZNF343','ZNF345','ZNF346','ZNF347','ZNF35','ZNF350','ZNF354A','ZNF354B','ZNF354C','ZNF355P','ZNF358','ZNF362','ZNF365','ZNF366','ZNF367','ZNF37A','ZNF382','ZNF383','ZNF384','ZNF385A','ZNF385B','ZNF385C','ZNF385D','ZNF391','ZNF394','ZNF395','ZNF396','ZNF397','ZNF398','ZNF404','ZNF407','ZNF408','ZNF41','ZNF410','ZNF414','ZNF415','ZNF416','ZNF417','ZNF418','ZNF419','ZNF420','ZNF423','ZNF425','ZNF426','ZNF428','ZNF429','ZNF43','ZNF430','ZNF431','ZNF432','ZNF433','ZNF436','ZNF438','ZNF439','ZNF44','ZNF440','ZNF441','ZNF442','ZNF443','ZNF444','ZNF445','ZNF446','ZNF449','ZNF45','ZNF451','ZNF454','ZNF460','ZNF461','ZNF462','ZNF467','ZNF468','ZNF469','ZNF470','ZNF471','ZNF473','ZNF474','ZNF479','ZNF48','ZNF480','ZNF483','ZNF484','ZNF485','ZNF486','ZNF487','ZNF488','ZNF490','ZNF491','ZNF492','ZNF493','ZNF496','ZNF497','ZNF500','ZNF501','ZNF502','ZNF503','ZNF506','ZNF507','ZNF510','ZNF511','ZNF512','ZNF512B','ZNF513','ZNF514','ZNF516','ZNF517','ZNF518A','ZNF518B','ZNF519']
TFs3Other=['ZNF521','ZNF524','ZNF525','ZNF526','ZNF527','ZNF528','ZNF529','ZNF530','ZNF532','ZNF534','ZNF536','ZNF540','ZNF541','ZNF542P','ZNF543','ZNF544','ZNF546','ZNF547','ZNF548','ZNF549','ZNF550','ZNF551','ZNF552','ZNF554','ZNF555','ZNF556','ZNF557','ZNF558','ZNF559','ZNF560','ZNF561','ZNF562','ZNF563','ZNF564','ZNF565','ZNF566','ZNF567','ZNF568','ZNF569','ZNF57','ZNF570','ZNF571','ZNF572','ZNF573','ZNF574','ZNF575','ZNF576','ZNF577','ZNF578','ZNF579','ZNF580','ZNF581','ZNF582','ZNF583','ZNF584','ZNF585A','ZNF585B','ZNF586','ZNF587','ZNF587B','ZNF589','ZNF592','ZNF593','ZNF594','ZNF595','ZNF596','ZNF597','ZNF598','ZNF599','ZNF600','ZNF605','ZNF606','ZNF607','ZNF608','ZNF609','ZNF610','ZNF611','ZNF613','ZNF614','ZNF615','ZNF616','ZNF618','ZNF619','ZNF620','ZNF621','ZNF622','ZNF623','ZNF624','ZNF625','ZNF626','ZNF627','ZNF628','ZNF629','ZNF630','ZNF638','ZNF639','ZNF641','ZNF644','ZNF645','ZNF646','ZNF648','ZNF649','ZNF652','ZNF653','ZNF654','ZNF655','ZNF658','ZNF66','ZNF660','ZNF662','ZNF664','ZNF665','ZNF667','ZNF668','ZNF669','ZNF670','ZNF671','ZNF672','ZNF674','ZNF675','ZNF676','ZNF677','ZNF678','ZNF679','ZNF680','ZNF681','ZNF682','ZNF683','ZNF684','ZNF687','ZNF688','ZNF689','ZNF69','ZNF691','ZNF692','ZNF695','ZNF696','ZNF697','ZNF699','ZNF7','ZNF70','ZNF700','ZNF701','ZNF703','ZNF704','ZNF705A','ZNF705B','ZNF705D','ZNF705E','ZNF705G','ZNF706','ZNF707','ZNF708','ZNF709','ZNF71','ZNF710','ZNF711','ZNF713','ZNF714','ZNF716','ZNF717','ZNF718','ZNF720','ZNF721','ZNF724','ZNF726','ZNF727','ZNF728','ZNF729','ZNF73','ZNF730','ZNF732','ZNF735','ZNF736','ZNF737','ZNF738','ZNF74','ZNF740','ZNF746','ZNF747','ZNF749','ZNF750','ZNF75A','ZNF75D','ZNF76','ZNF761','ZNF763','ZNF764','ZNF765','ZNF766','ZNF768','ZNF77','ZNF770','ZNF771','ZNF772','ZNF773','ZNF774','ZNF775','ZNF776','ZNF777','ZNF778','ZNF780A','ZNF780B','ZNF781','ZNF782','ZNF783','ZNF784','ZNF785','ZNF786','ZNF787','ZNF788','ZNF789','ZNF79','ZNF790','ZNF791','ZNF792','ZNF793','ZNF799','ZNF8','ZNF80','ZNF800','ZNF804A','ZNF804B','ZNF805','ZNF808','ZNF81','ZNF812P','ZNF813','ZNF814','ZNF816','ZNF821','ZNF823','ZNF827','ZNF829','ZNF83','ZNF830','ZNF831','ZNF835','ZNF836','ZNF837','ZNF839','ZNF84','ZNF841','ZNF843','ZNF844','ZNF845','ZNF846','ZNF85','ZNF850','ZNF852','ZNF853','ZNF860','ZNF862','ZNF865','ZNF878','ZNF879','ZNF880','ZNF883','ZNF888','ZNF891','ZNF90','ZNF91','ZNF92','ZNF93','ZNF98','ZNF99','ZNFX1','ZRANB2','ZRSR1','ZRSR2','ZSCAN1','ZSCAN10','ZSCAN12','ZSCAN16','ZSCAN18','ZSCAN2','ZSCAN20','ZSCAN21','ZSCAN22','ZSCAN23','ZSCAN25','ZSCAN26','ZSCAN29','ZSCAN30','ZSCAN31','ZSCAN32','ZSCAN4','ZSCAN5A','ZSCAN5B','ZSCAN5C','ZSCAN5DP','ZSCAN9','ZUFSP','ZXDA','ZXDB','ZXDC','ZZZ3']
TFs4Other=['ATF2','ATF6B','ATN1','BHLHA15','BHLHB2','BHLHB4 ','BHLHB5','BHLHB9','C13ORF8','C20ORF194','CDX2','CRAMP1L','ELP3','ERVK3-1','ETF1','FLJ12895','FLJ20321','FLJ25476','FLJ35784','FOXM1','FOXO2','FOXR1','FOXR2','GATA1','GRLF1','KDM4C','KDM4E','KIAA0863','KIAA0961','KLRG1','LBXCOR1','LEF1','LHX7','LOC126295','LOC51058','LOC91661','MGC4728','MYOD1','MYOG','MYRF','NEUROD4','NEUROG1','NFAT2','NFAT4','NFE2L1','NR1A2','NR1B1','NR1B2','NR1B3','NR1C1','NR1C2','NR1C3','NR1F3','NR1I1','NR2A1','NR2B3','NR3A1','NR3A2','NR3B3','OTEX','PEPP-2','PHF2','POU2F1','POU2F2','POU4F3','POU5F1','POU6F1','PRDM3','PRKRIP1','PRKRIR','SETD1A','SMAD8','SMYD2','SOX12','SOX14','SOX15','TBX14','TCF1','TCF17','TCF2','TCF9','TEAD1','TPRXL','UBTFL1','WHSC1','ZBTB14','ZBTB18','ZBTB19','ZBTB21','ZBTB27','ZBTB28','ZBTB35','ZBTB49','ZFP161','ZFP28','ZKSCAN10','ZKSCAN11','ZKSCAN12','ZKSCAN13','ZKSCAN14','ZKSCAN16','ZKSCAN19','ZKSCAN21','ZKSCAN24','ZKSCAN6','ZKSCAN8','ZNF10','ZNF112','ZNF187','ZNF193','ZNF204P','ZNF238','ZNF271','ZNF295','ZNF305','ZNF323','ZNF434','ZNF499','ZNF542','ZNF545','ZNF553','ZNF56','ZNF625','ZNF643','ZNF698','ZNF705F','ZNF751','ZNF752','ZNF753','ZNF785','ZNF806','ZNF816','ZNF833P','ZNF906','ZSCAN1','ZSCAN25','ZSCAN26','ZSCAN31','ZSCAN32','ZSCAN5D','ZSCAN9']
TFsOther=TFs1Other+TFs2Other+TFs3Other+TFs4Other
```

Example 4 (python):
```python
# Remove the obvious pericardium and valve cells

sc.pl.umap(adata, color='leiden',groups='8',legend_loc='on data')
PericardialCells = adata.obs.index[adata.obs['leiden'] == '8']

sc.tl.leiden(adata, resolution=0.5, key_added='leiden')
sc.pl.umap(adata, color='leiden',groups='13',legend_loc='on data')

print(len(adata))
adata=adata[adata.obs.leiden!='13',:].copy()
print(len(adata))

adata=adata[[x for x in adata.obs_names if x not in PericardialCells],:].copy()
print(len(adata))
```

Example 5 (python):
```python
set(adata.obs.coarse_grain)
```

Example 6 (python):
```python
adata=adata[adata.obs.coarse_grain=='Mesenchymal',:].copy()
adata
```

Example 7 (python):
```python
# preserve raw count
adata.layers["counts"] = adata.X.copy()

# normalise and log-transform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key='batch_key')

print(sum(adata.var.highly_variable))
sc.pl.highly_variable_genes(adata)

# freeze the state in `.raw`
adata.raw = adata
```

Example 8 (python):
```python
# subset hvgs
adata = adata[:, adata.var.highly_variable]

adata=adata.copy()

sc.tl.score_genes_cell_cycle(adata, s_genes=s_genes, g2m_genes=g2m_genes)
#sc.pp.regress_out(adata, ['S_score','G2M_score'])

#batch correction
scvi.model.SCVI.setup_anndata(adata, 
                              layer="counts", 
                              batch_key="batch_key",
                              categorical_covariate_keys=['phase'],
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo'])

vae = scvi.model.SCVI(adata, n_hidden = 128, n_latent = 50, n_layers = 3, dispersion = 'gene-batch')
vae.train(max_epochs=50)
    
# Convergence plot
x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
plt.plot(x, vae.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
plt.clf()

# add the latent space to adata
adata.obsm["X_scVI"] = vae.get_latent_representation()

sc.tl.score_genes(adata, stress_genes, score_name='stress_score')
sc.tl.score_genes(adata, hb_genes1, score_name='hb1_score')
#sc.tl.score_genes(adata, hb_genes2, score_name='hb2_score')
sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata,min_dist=0.1)
#sc.tl.tsne(adata)

sc.pl.umap(adata, color=['log1p_total_counts','n_genes','n_genes_by_counts','total_counts','pct_counts_mt','pct_counts_ribo'],ncols=5)
sc.pl.umap(adata, color=['scrublet_score','stress_score', 'hb1_score','phase'], ncols=5,)
sc.pl.umap(adata, color=['donor','sex','age','region','phase','cell_or_nuclei','modality','kit_10x','FACSgate'],wspace=0.8,ncols=5)
Genes=['TTN','TNNT2','MYOM1','LDB3','ALDH1A2','TNNT1','KRT19','UPK3B','COL6A3','FBLN1','PCOLCE','LRRC17','EMCN','ECSCR','PECAM1','CDH5', 'LAPTM5','PTPRC','LCP1','CD53','PHOX2B','NRXN1','STMN2','GRIK2']
sc.pl.umap(adata, color=Genes)
```

Example 9 (python):
```python
#this block is to see where the cells over a certain threshold with TNNT2 expression lies in the UMAP
#enter what gene to set threshold for
#when running this box for the first time, comment the starting with 'del'
#then uncomment it for the other ones --- !!!

CellName='Cardio_toremove'
gene='TNNT2'
exp_thresh=2.2
gene_pos=adata.var_names.get_loc(gene)
adata_sel = adata[adata.X[:,gene_pos]>exp_thresh].copy()

l=adata_sel.obs_names
set(l).intersection(adata.obs_names)
print(len(l))

#del adata.obs[CellName]
adata.obs.loc[set(l).intersection(adata.obs_names),CellName]='Yes'
sc.settings.set_figure_params(dpi=200, facecolor='white', color_map = 'RdPu',)
sc.pl.umap(adata, color=[CellName], groups='Yes',size=10, wspace=0.45, legend_fontsize=5, legend_loc='on data',frameon=False)

print(adata)
adata= adata[adata.X[:,gene_pos]<exp_thresh].copy()
print(adata)
```

Example 10 (python):
```python
# Look for previous fine_grains
ClustersToStay=['GreatVesselSmoothMuscleCells','MyocardialInterstitialFibroblasts','Myofibroblasts','ValveInterstitialCells','CoronaryPericytes','CoronarySmoothMuscleCells','CoronaryVesselAdventitialFibroblasts','EarlyGreatVesselCells','GreatVesselAdventitialFibroblasts']
bdata=adata[[x for x in adata.obs_names if adata.obs.previous_fine_grain[x] in ClustersToStay],:].copy()
sc.pl.umap(bdata, color='previous_fine_grain')
```

Example 11 (python):
```python
sc.tl.leiden(adata, resolution=0.1, key_added='leiden')
sc.pl.umap(adata, color='leiden',legend_loc='on data')
```

Example 12 (python):
```python
sc.pl.umap(adata, color=['donor','sex','age','region','phase','cell_or_nuclei','modality','kit_10x','FACSgate'],wspace=0.8,ncols=5)
```

Example 13 (python):
```python
sc.tl.leiden(adata, resolution=0.075, key_added='leiden')
sc.pl.umap(adata, color='leiden',legend_loc='on data')
```

Example 14 (python):
```python
sc.pl.umap(adata, color=['FOXC1','MYH11','PDGFRB','PDGFRA','DCN','TCF21','CLU','GDF10'],legend_loc='on data')
```

Example 15 (python):
```python
sc.tl.rank_genes_groups(adata, groupby='leiden', method='t-test')
sc.tl.dendrogram(adata, groupby='leiden')
#sc.pl.dendrogram(adata,groupby='leiden')
sc.pl.rank_genes_groups_matrixplot(adata, groupby='leiden', standard_scale='var')
```

Example 16 (python):
```python
#adata.write('./Objects/MesenchymalWithoutPericardiumAndValves.h5ad')
#adata=sc.read('./Objects/MesenchymalWithoutPericardiumAndValves.h5ad')
```

Example 17 (python):
```python
#Re-run individual mesenchymal compartments
#Great vessel smooth muscle cells

adata=sc.read('./Objects/MesenchymalWithoutPericardiumAndValves.h5ad')
adata=adata[adata.obs.leiden=='2',:].copy()
print(len(adata))
adata_raw=sc.read('./Objects/20221130RNA_foetal_post-scVI_raw_ExtracardiacRemoved.h5ad')

print(adata_raw)
# constructing new anndata with maintaining .obs, .obsm, .obsp, .uns
# overwriting adata_raw
adata_raw=anndata.AnnData(X=adata_raw[adata.obs_names,:].X,
                      obs=adata.obs, # from annotated or subsetted data
                      var=adata_raw.var,
                      obsm=adata.obsm, # from annotated or subsetted data
                      obsp=adata.obsp, # from annotated or subsetted data
                    uns=adata.uns # from annotated or subsetted data
                               )

adata=adata_raw.copy()
print(adata)

# preserve raw count
adata.layers["counts"] = adata.X.copy()

# normalise and log-transform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key='batch_key')

print(sum(adata.var.highly_variable))
sc.pl.highly_variable_genes(adata)

# freeze the state in `.raw`
adata.raw = adata

# subset hvgs
adata = adata[:, adata.var.highly_variable]

adata=adata.copy()

sc.tl.score_genes_cell_cycle(adata, s_genes=s_genes, g2m_genes=g2m_genes)
#sc.pp.regress_out(adata, ['S_score','G2M_score'])

#batch correction
scvi.model.SCVI.setup_anndata(adata, 
                              layer="counts", 
                              batch_key="batch_key",
                              categorical_covariate_keys=['phase'],
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo'])

vae = scvi.model.SCVI(adata, n_hidden = 128, n_latent = 50, n_layers = 3, dispersion = 'gene-batch')
vae.train(max_epochs=50)
    
# Convergence plot
x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
plt.plot(x, vae.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
plt.clf()

# add the latent space to adata
adata.obsm["X_scVI"] = vae.get_latent_representation()

sc.tl.score_genes(adata, stress_genes, score_name='stress_score')
sc.tl.score_genes(adata, hb_genes1, score_name='hb1_score')

#sc.tl.score_genes(adata, hb_genes2, score_name='hb2_score')
sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata,min_dist=0.1)
#sc.tl.tsne(adata)

sc.pl.umap(adata, color=['log1p_total_counts','n_genes','n_genes_by_counts','total_counts','pct_counts_mt','pct_counts_ribo'],ncols=5)
sc.pl.umap(adata, color=['scrublet_score','stress_score', 'hb1_score','phase'], ncols=5,)
sc.pl.umap(adata, color=['donor','sex','age','region','phase','cell_or_nuclei','modality','kit_10x','FACSgate'],wspace=0.8,ncols=5)

Genes=['TTN','TNNT2','MYOM1','LDB3','ALDH1A2','TNNT1','KRT19','UPK3B','COL6A3','FBLN1','PCOLCE','LRRC17','EMCN','ECSCR','PECAM1','CDH5', 'LAPTM5','PTPRC','LCP1','CD53','PHOX2B','NRXN1','STMN2','GRIK2']
sc.pl.umap(adata, color=Genes)
```

Example 18 (python):
```python
sc.tl.leiden(adata, resolution=0.69, key_added='leiden')
sc.pl.umap(adata, color='leiden',groups='8',legend_loc='on data')
sc.pl.umap(adata, color=['RGS5','ROBO2','DCLK2','PDE1C','IL17B'])
```

Example 19 (python):
```python
sc.tl.rank_genes_groups(adata, groupby='leiden', method='t-test')
sc.tl.dendrogram(adata, groupby='leiden')
#sc.pl.dendrogram(adata,groupby='leiden')
sc.pl.rank_genes_groups_matrixplot(adata, groupby='leiden', standard_scale='var')
```

Example 20 (python):
```python
sc.pl.umap(adata, color='leiden',legend_loc='on data')
```

Example 21 (python):
```python
#Re-run individual mesenchymal compartments
#Early great vessel cells

adata=sc.read('./Objects/MesenchymalWithoutPericardiumAndValves.h5ad')
adata=adata[adata.obs.leiden=='3',:].copy()
print(len(adata))
adata_raw=sc.read('./Objects/20221130RNA_foetal_post-scVI_raw_ExtracardiacRemoved.h5ad')

print(adata_raw)
# constructing new anndata with maintaining .obs, .obsm, .obsp, .uns
# overwriting adata_raw
adata_raw=anndata.AnnData(X=adata_raw[adata.obs_names,:].X,
                      obs=adata.obs, # from annotated or subsetted data
                      var=adata_raw.var,
                      obsm=adata.obsm, # from annotated or subsetted data
                      obsp=adata.obsp, # from annotated or subsetted data
                    uns=adata.uns # from annotated or subsetted data
                               )

adata=adata_raw.copy()
print(adata)

# preserve raw count
adata.layers["counts"] = adata.X.copy()

# normalise and log-transform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key='batch_key')

print(sum(adata.var.highly_variable))
sc.pl.highly_variable_genes(adata)

# freeze the state in `.raw`
adata.raw = adata

# subset hvgs
adata = adata[:, adata.var.highly_variable]

adata=adata.copy()

sc.tl.score_genes_cell_cycle(adata, s_genes=s_genes, g2m_genes=g2m_genes)
#sc.pp.regress_out(adata, ['S_score','G2M_score'])

#batch correction
scvi.model.SCVI.setup_anndata(adata, 
                              layer="counts", 
                              batch_key="batch_key",
                              categorical_covariate_keys=['phase'],
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo'])

vae = scvi.model.SCVI(adata, n_hidden = 128, n_latent = 50, n_layers = 3, dispersion = 'gene-batch')
vae.train(max_epochs=50)
    
# Convergence plot
x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
plt.plot(x, vae.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
plt.clf()

# add the latent space to adata
adata.obsm["X_scVI"] = vae.get_latent_representation()

sc.tl.score_genes(adata, stress_genes, score_name='stress_score')
sc.tl.score_genes(adata, hb_genes1, score_name='hb1_score')
#sc.tl.score_genes(adata, hb_genes2, score_name='hb2_score')

sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata,min_dist=0.1)
#sc.tl.tsne(adata)

sc.pl.umap(adata, color=['log1p_total_counts','n_genes','n_genes_by_counts','total_counts','pct_counts_mt','pct_counts_ribo'],ncols=5)
sc.pl.umap(adata, color=['scrublet_score','stress_score', 'hb1_score','phase'], ncols=5,)
sc.pl.umap(adata, color=['donor','sex','age','region','phase','cell_or_nuclei','modality','kit_10x','FACSgate'],wspace=0.8,ncols=5)

Genes=['TTN','TNNT2','MYOM1','LDB3','ALDH1A2','TNNT1','KRT19','UPK3B','COL6A3','FBLN1','PCOLCE','LRRC17','EMCN','ECSCR','PECAM1','CDH5', 'LAPTM5','PTPRC','LCP1','CD53','PHOX2B','NRXN1','STMN2','GRIK2']
sc.pl.umap(adata, color=Genes)
```

Example 22 (python):
```python
sc.pl.umap(adata, color='previous_fine_grain')
```

Example 23 (python):
```python
# Look for previous fine_grains
ClustersToStay=['NA','GreatVesselSmoothMuscleCells','MyocardialInterstitialFibroblasts','Myofibroblasts','ValveInterstitialCells','CoronaryPericytes','CoronarySmoothMuscleCells','CoronaryVesselAdventitialFibroblasts','EarlyGreatVesselCells','GreatVesselAdventitialFibroblasts']
bdata=adata[[x for x in adata.obs_names if adata.obs.previous_fine_grain[x] in ClustersToStay],:].copy()
sc.pl.umap(bdata, color='previous_fine_grain')
```

Example 24 (python):
```python
sc.tl.leiden(adata, resolution=0.2, key_added='leiden')
sc.pl.umap(adata, color='leiden',legend_loc='on data')

sc.tl.rank_genes_groups(adata, groupby='leiden', method='t-test')
sc.tl.dendrogram(adata, groupby='leiden')
#sc.pl.dendrogram(adata,groupby='leiden')
sc.pl.rank_genes_groups_matrixplot(adata, groupby='leiden', standard_scale='var')
```

Example 25 (python):
```python
#Re-run individual mesenchymal compartments
#Great vessel adventitial fibroblasts

adata=sc.read('./Objects/MesenchymalWithoutPericardiumAndValves.h5ad')
adata=adata[adata.obs.leiden=='0',:].copy()
print(len(adata))
adata_raw=sc.read('./Objects/20221130RNA_foetal_post-scVI_raw_ExtracardiacRemoved.h5ad')

print(adata_raw)
# constructing new anndata with maintaining .obs, .obsm, .obsp, .uns
# overwriting adata_raw
adata_raw=anndata.AnnData(X=adata_raw[adata.obs_names,:].X,
                      obs=adata.obs, # from annotated or subsetted data
                      var=adata_raw.var,
                      obsm=adata.obsm, # from annotated or subsetted data
                      obsp=adata.obsp, # from annotated or subsetted data
                    uns=adata.uns # from annotated or subsetted data
                               )

adata=adata_raw.copy()
print(adata)

# preserve raw count
adata.layers["counts"] = adata.X.copy()

# normalise and log-transform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key='batch_key')

print(sum(adata.var.highly_variable))
sc.pl.highly_variable_genes(adata)

# freeze the state in `.raw`
adata.raw = adata

# subset hvgs
adata = adata[:, adata.var.highly_variable]

adata=adata.copy()

sc.tl.score_genes_cell_cycle(adata, s_genes=s_genes, g2m_genes=g2m_genes)
#sc.pp.regress_out(adata, ['S_score','G2M_score'])

#batch correction
scvi.model.SCVI.setup_anndata(adata, 
                              layer="counts", 
                              batch_key="batch_key",
                              categorical_covariate_keys=['phase'],
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo'])

vae = scvi.model.SCVI(adata, n_hidden = 128, n_latent = 50, n_layers = 3, dispersion = 'gene-batch')
vae.train(max_epochs=50)
    
# Convergence plot
x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
plt.plot(x, vae.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
plt.clf()

# add the latent space to adata
adata.obsm["X_scVI"] = vae.get_latent_representation()

sc.tl.score_genes(adata, stress_genes, score_name='stress_score')
sc.tl.score_genes(adata, hb_genes1, score_name='hb1_score')

#sc.tl.score_genes(adata, hb_genes2, score_name='hb2_score')
sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata,min_dist=0.1)
#sc.tl.tsne(adata)

sc.pl.umap(adata, color=['log1p_total_counts','n_genes','n_genes_by_counts','total_counts','pct_counts_mt','pct_counts_ribo'],ncols=5)
sc.pl.umap(adata, color=['scrublet_score','stress_score', 'hb1_score','phase'], ncols=5,)
sc.pl.umap(adata, color=['donor','sex','age','region','phase','cell_or_nuclei','modality','kit_10x','FACSgate'],wspace=0.8,ncols=5)

Genes=['TTN','TNNT2','MYOM1','LDB3','ALDH1A2','TNNT1','KRT19','UPK3B','COL6A3','FBLN1','PCOLCE','LRRC17','EMCN','ECSCR','PECAM1','CDH5', 'LAPTM5','PTPRC','LCP1','CD53','PHOX2B','NRXN1','STMN2','GRIK2']
sc.pl.umap(adata, color=Genes)
```

Example 26 (python):
```python
sc.tl.leiden(adata, resolution=0.2, key_added='leiden')
sc.pl.umap(adata, color='leiden',legend_loc='on data')

sc.tl.rank_genes_groups(adata, groupby='leiden', method='t-test')
sc.tl.dendrogram(adata, groupby='leiden')
#sc.pl.dendrogram(adata,groupby='leiden')
sc.pl.rank_genes_groups_matrixplot(adata,groupby='leiden', standard_scale='var')

sc.pl.umap(adata, color=['SRPX','GDF10'],legend_loc='on data')
```

Example 27 (python):
```python
#Re-run individual mesenchymal compartments
#Myocardial fibroblasts and coronary mural cells

adata=sc.read('./Objects/MesenchymalWithoutPericardiumAndValves.h5ad')
adata=adata[adata.obs.leiden=='1',:].copy()
print(len(adata))
adata_raw=sc.read('./Objects/20221130RNA_foetal_post-scVI_raw_ExtracardiacRemoved.h5ad')

print(adata_raw)
# constructing new anndata with maintaining .obs, .obsm, .obsp, .uns
# overwriting adata_raw
adata_raw=anndata.AnnData(X=adata_raw[adata.obs_names,:].X,
                      obs=adata.obs, # from annotated or subsetted data
                      var=adata_raw.var,
                      obsm=adata.obsm, # from annotated or subsetted data
                      obsp=adata.obsp, # from annotated or subsetted data
                    uns=adata.uns # from annotated or subsetted data
                               )

adata=adata_raw.copy()
print(adata)

# preserve raw count
adata.layers["counts"] = adata.X.copy()

# normalise and log-transform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key='batch_key')

print(sum(adata.var.highly_variable))
sc.pl.highly_variable_genes(adata)

# freeze the state in `.raw`
adata.raw = adata

# subset hvgs
adata = adata[:, adata.var.highly_variable]

adata=adata.copy()

sc.tl.score_genes_cell_cycle(adata, s_genes=s_genes, g2m_genes=g2m_genes)
#sc.pp.regress_out(adata, ['S_score','G2M_score'])

#batch correction
scvi.model.SCVI.setup_anndata(adata, 
                              layer="counts", 
                              batch_key="batch_key",
                              categorical_covariate_keys=['phase'],
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo'])

vae = scvi.model.SCVI(adata, n_hidden = 128, n_latent = 50, n_layers = 3, dispersion = 'gene-batch')
vae.train(max_epochs=50)
    
# Convergence plot
x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
plt.plot(x, vae.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
plt.clf()

# add the latent space to adata
adata.obsm["X_scVI"] = vae.get_latent_representation()

sc.tl.score_genes(adata, stress_genes, score_name='stress_score')
sc.tl.score_genes(adata, hb_genes1, score_name='hb1_score')
sc.tl.score_genes(adata, hb_genes2, score_name='hb2_score')
sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata,min_dist=0.1)
#sc.tl.tsne(adata)

sc.pl.umap(adata, color=['log1p_total_counts','n_genes','n_genes_by_counts','total_counts','pct_counts_mt','pct_counts_ribo'],ncols=5)
sc.pl.umap(adata, color=['scrublet_score','stress_score', 'hb1_score','hb2_score','phase'], ncols=5,)
sc.pl.umap(adata, color=['donor','sex','age','region','phase','cell_or_nuclei','modality','kit_10x','FACSgate'],wspace=0.8,ncols=5)

Genes=['TTN','TNNT2','MYOM1','LDB3','ALDH1A2','TNNT1','KRT19','UPK3B','COL6A3','FBLN1','PCOLCE','LRRC17','EMCN','ECSCR','PECAM1','CDH5', 'LAPTM5','PTPRC','LCP1','CD53','PHOX2B','NRXN1','STMN2','GRIK2']
sc.pl.umap(adata, color=Genes)
```

Example 28 (python):
```python
# Look for previous fine_grains
ClustersToStay=['NA','GreatVesselSmoothMuscleCells','MyocardialInterstitialFibroblasts','Myofibroblasts','ValveInterstitialCells','CoronaryPericytes','CoronarySmoothMuscleCells','CoronaryVesselAdventitialFibroblasts','EarlyGreatVesselCells','GreatVesselAdventitialFibroblasts']
bdata=adata[[x for x in adata.obs_names if adata.obs.previous_fine_grain[x] in ClustersToStay],:].copy()
sc.pl.umap(bdata, color='previous_fine_grain')
```

Example 29 (python):
```python
sc.tl.leiden(adata, resolution=2, key_added='leiden')
sc.pl.umap(adata, color='leiden',groups='38',legend_loc='on data')
```

Example 30 (python):
```python
#Remove the TNNT2 cluster
print(adata)
adata=adata[adata.obs.leiden!='38',:].copy()
print(adata)
```

Example 31 (python):
```python
sc.tl.leiden(adata, resolution=0.1, key_added='leiden')
sc.pl.umap(adata, color='leiden',legend_loc='on data')
```

Example 32 (python):
```python
adata=adata[adata.obs.leiden=='1',:].copy()
#adata.write('./CoronaryMuralCells.h5ad')
#adata=sc.read('./CoronaryMuralCells.h5ad')
```

Example 33 (python):
```python
#Re-run individual mesenchymal compartments
#Remaining myocardial fibroblasts

adata=sc.read('./Objects/MesenchymalWithoutPericardiumAndValves.h5ad')
adata=adata[adata.obs.leiden=='1',:].copy()

CoronaryPericytes=[x.strip() for x in open('./Index/Mesenchymal/CoronaryPericytes.txt')]
CoronarySmoothMuscleCells=[x.strip() for x in open('./Index/Mesenchymal/CoronarySmoothMuscleCells.txt')]
RBCsRemovedFromMyocardialFibroblasts=[x.strip() for x in open('./RBCsRemovedFromMyocardialFibroblasts.txt')]

ShouldNotBeIn=RBCsRemovedFromMyocardialFibroblasts+CoronarySmoothMuscleCells+CoronaryPericytes
print(len(ShouldNotBeIn))
adata=adata[[x for x in adata.obs_names if x not in ShouldNotBeIn],:].copy()


print(len(adata))
adata_raw=sc.read('./Objects/20221130RNA_foetal_post-scVI_raw_ExtracardiacRemoved.h5ad')

print(adata_raw)
# constructing new anndata with maintaining .obs, .obsm, .obsp, .uns
# overwriting adata_raw
adata_raw=anndata.AnnData(X=adata_raw[adata.obs_names,:].X,
                      obs=adata.obs, # from annotated or subsetted data
                      var=adata_raw.var,
                      obsm=adata.obsm, # from annotated or subsetted data
                      obsp=adata.obsp, # from annotated or subsetted data
                    uns=adata.uns # from annotated or subsetted data
                               )

adata=adata_raw.copy()
print(adata)

# preserve raw count
adata.layers["counts"] = adata.X.copy()

# normalise and log-transform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key='batch_key')

print(sum(adata.var.highly_variable))
sc.pl.highly_variable_genes(adata)

# freeze the state in `.raw`
adata.raw = adata

# subset hvgs
adata = adata[:, adata.var.highly_variable]

adata=adata.copy()

sc.tl.score_genes_cell_cycle(adata, s_genes=s_genes, g2m_genes=g2m_genes)
#sc.pp.regress_out(adata, ['S_score','G2M_score'])

#batch correction
scvi.model.SCVI.setup_anndata(adata, 
                              layer="counts", 
                              batch_key="batch_key",
                              categorical_covariate_keys=['phase'],
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo'])

vae = scvi.model.SCVI(adata, n_hidden = 128, n_latent = 50, n_layers = 3, dispersion = 'gene-batch')
vae.train(max_epochs=50)
    
# Convergence plot
x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
plt.plot(x, vae.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
plt.clf()

# add the latent space to adata
adata.obsm["X_scVI"] = vae.get_latent_representation()

sc.tl.score_genes(adata, stress_genes, score_name='stress_score')
sc.tl.score_genes(adata, hb_genes1, score_name='hb1_score')
#sc.tl.score_genes(adata, hb_genes2, score_name='hb2_score')

sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata,min_dist=0.1)
#sc.tl.tsne(adata)

sc.pl.umap(adata, color=['log1p_total_counts','n_genes','n_genes_by_counts','total_counts','pct_counts_mt','pct_counts_ribo'],ncols=5)
sc.pl.umap(adata, color=['scrublet_score','stress_score', 'hb1_score','phase'], ncols=5,)
sc.pl.umap(adata, color=['donor','sex','age','region','phase','cell_or_nuclei','modality','kit_10x','FACSgate'],wspace=0.8,ncols=5)

Genes=['TTN','TNNT2','MYOM1','LDB3','ALDH1A2','TNNT1','KRT19','UPK3B','COL6A3','FBLN1','PCOLCE','LRRC17','EMCN','ECSCR','PECAM1','CDH5', 'LAPTM5','PTPRC','LCP1','CD53','PHOX2B','NRXN1','STMN2','GRIK2']
sc.pl.umap(adata, color=Genes)
```

Example 34 (python):
```python
sc.pl.umap(adata, color=['FBLN5','CD34','SCN7A','DCN','TCF21','CLU','HAPLN1','ACTA2','PDGFRB','PDGFRA'],legend_loc='on data')
sc.pl.umap(adata, color=['CLU','PDLIM3','NCAM1','HAPLN1','COL9A2','BMPER','SCX','SRPX'],legend_loc='on data')
```

Example 35 (python):
```python
sc.tl.leiden(adata, resolution=0.5, key_added='leiden')
sc.pl.umap(adata, color='leiden',legend_loc='on data')
```

Example 36 (python):
```python
sc.tl.rank_genes_groups(adata, groupby='leiden', method='t-test')
sc.tl.dendrogram(adata, groupby='leiden')
#sc.pl.dendrogram(adata,groupby='leiden')
sc.pl.rank_genes_groups_matrixplot(adata, groupby='leiden', standard_scale='var')
```

Example 37 (python):
```python
bdata=adata[adata.obs.leiden=='1',:].copy()
bdatanames=bdata.obs_names

np.savetxt('Myofibroblasts.txt', bdatanames, delimiter="\n", fmt="%s")
```

Example 38 (python):
```python
print(len(adata))
adata=adata[adata.obs.leiden!='1',:].copy()
print(len(adata))
```

Example 39 (python):
```python
print(len(adata))
adata=adata[adata.obs.leiden!='2',:].copy()
print(len(adata))
```

Example 40 (python):
```python
bdatanames=adata.obs_names
np.savetxt('MyocardialInterstitialFibroblasts.txt', bdatanames, delimiter="\n", fmt="%s")
```

Example 41 (python):
```python
sc.tl.leiden(adata, resolution=0.2, key_added='leiden')
sc.pl.umap(adata, color='leiden',legend_loc='on data')
```

Example 42 (python):
```python
sc.tl.rank_genes_groups(adata, groupby='leiden', method='t-test')
sc.tl.dendrogram(adata, groupby='leiden')
#sc.pl.dendrogram(adata,groupby='leiden')
sc.pl.rank_genes_groups_matrixplot(adata, groupby='leiden', standard_scale='var')
```

Example 43 (python):
```python
sc.pl.umap(adata, color=['SLIT3','PIEZO2','TSHZ2','FLRT2','PRRX1','MAPK10'],legend_loc='on data')
sc.pl.umap(adata, color=['SCN7A','ROBO2','TLL2'],legend_loc='on data')
sc.pl.umap(adata, color=['region'],groups=['atrial septum','left ventricle','right ventricle + papillary'])
```

Example 44 (python):
```python
#Combine all barcodes

adata=sc.read('./Objects/20221130RNA_foetal_post-scVI_raw_ExtracardiacRemoved.h5ad')
print(adata.shape)

adata=adata[adata.obs.coarse_grain=='Mesenchymal',:].copy()
print(adata.shape)

CoronaryPericytes=[x.strip() for x in open('./Index/Mesenchymal/CoronaryPericytes.txt')]
CoronarySmoothMuscleCells=[x.strip() for x in open('./Index/Mesenchymal/CoronarySmoothMuscleCells.txt')]
CoronaryVesselAdventitialFibroblasts=[x.strip() for x in open('./Index/Mesenchymal/CoronaryVesselAdventitialFibroblasts.txt')]
DuctusArteriosusSmoothMuscleCells=[x.strip() for x in open('./Index/Mesenchymal/DuctusArteriosusSmoothMuscleCells.txt')]
EarlyGreatVesselCells=[x.strip() for x in open('./Index/Mesenchymal/EarlyGreatVesselCells.txt')]
GreatVesselAdventitialFibroblasts=[x.strip() for x in open('./Index/Mesenchymal/GreatVesselAdventitialFibroblasts.txt')]
GreatVesselSmoothMuscleCells=[x.strip() for x in open('./Index/Mesenchymal/GreatVesselSmoothMuscleCells.txt')]
MyocardialInterstitialFibroblasts=[x.strip() for x in open('./Index/Mesenchymal/MyocardialInterstitialFibroblasts.txt')]
Myofibroblasts=[x.strip() for x in open('./Index/Mesenchymal/Myofibroblasts.txt')]
PericardialCells=[x.strip() for x in open('./Index/Mesenchymal/PericardialCells.txt')]
ValveInterstitialCells=[x.strip() for x in open('./Index/Mesenchymal/ValveInterstitialCells.txt')]

MesenchymalIndex=CoronaryPericytes+CoronarySmoothMuscleCells+CoronaryVesselAdventitialFibroblasts+DuctusArteriosusSmoothMuscleCells+EarlyGreatVesselCells+GreatVesselAdventitialFibroblasts+GreatVesselSmoothMuscleCells+MyocardialInterstitialFibroblasts+Myofibroblasts+PericardialCells+ValveInterstitialCells


adata.obs['fine_grain']= 'na'

for k in range(1, len(adata)+1):
    if adata.obs_names[k-1] in CoronaryPericytes:
        adata.obs.fine_grain[k-1]='CoronaryPericytes'   
    if adata.obs_names[k-1] in CoronarySmoothMuscleCells:
        adata.obs.fine_grain[k-1]='CoronarySmoothMuscleCells'  
    if adata.obs_names[k-1] in CoronaryVesselAdventitialFibroblasts:
        adata.obs.fine_grain[k-1]='CoronaryVesselAdventitialFibroblasts'   
    if adata.obs_names[k-1] in DuctusArteriosusSmoothMuscleCells:
        adata.obs.fine_grain[k-1]='DuctusArteriosusSmoothMuscleCells'  
    if adata.obs_names[k-1] in EarlyGreatVesselCells:
        adata.obs.fine_grain[k-1]='EarlyGreatVesselCells'   
    if adata.obs_names[k-1] in GreatVesselSmoothMuscleCells:
        adata.obs.fine_grain[k-1]='GreatVesselSmoothMuscleCells'   
    if adata.obs_names[k-1] in MyocardialInterstitialFibroblasts:
        adata.obs.fine_grain[k-1]='MyocardialInterstitialFibroblasts'   
    if adata.obs_names[k-1] in Myofibroblasts:
        adata.obs.fine_grain[k-1]='Myofibroblasts'  
    if adata.obs_names[k-1] in PericardialCells:
        adata.obs.fine_grain[k-1]='PericardialCells'  
    if adata.obs_names[k-1] in ValveInterstitialCells:
        adata.obs.fine_grain[k-1]='ValveInterstitialCells'  

for k in range(1, len(adata)+1):        
    if adata.obs_names[k-1] in GreatVesselAdventitialFibroblasts:
        adata.obs.fine_grain[k-1]='GreatVesselAdventitialFibroblasts'  

sc.pl.umap(adata, color=['fine_grain'])

print(adata.shape)
adata=adata[adata.obs.fine_grain!='na',:].copy()
print(adata.shape)
```

Example 45 (python):
```python
# preserve raw count
adata.layers["counts"] = adata.X.copy()

# normalise and log-transform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key='batch_key')

print(sum(adata.var.highly_variable))
sc.pl.highly_variable_genes(adata)

# freeze the state in `.raw`
adata.raw = adata

# subset hvgs
adata = adata[:, adata.var.highly_variable]

adata=adata.copy()

sc.tl.score_genes_cell_cycle(adata, s_genes=s_genes, g2m_genes=g2m_genes)
#sc.pp.regress_out(adata, ['S_score','G2M_score'])

#batch correction
scvi.model.SCVI.setup_anndata(adata, 
                              layer="counts", 
                              batch_key="batch_key",
                              categorical_covariate_keys=['phase'],
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo'])

vae = scvi.model.SCVI(adata, n_hidden = 128, n_latent = 50, n_layers = 3, dispersion = 'gene-batch')
vae.train(max_epochs=20)
    
# Convergence plot
x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
plt.plot(x, vae.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
plt.clf()

# add the latent space to adata
adata.obsm["X_scVI"] = vae.get_latent_representation()

sc.tl.score_genes(adata, stress_genes, score_name='stress_score')
sc.tl.score_genes(adata, hb_genes1, score_name='hb1_score')
#sc.tl.score_genes(adata, hb_genes2, score_name='hb2_score')
sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata,min_dist=0.1)
#sc.tl.tsne(adata)

sc.pl.umap(adata, color=['log1p_total_counts','n_genes','n_genes_by_counts','total_counts','pct_counts_mt','pct_counts_ribo'],ncols=5)
sc.pl.umap(adata, color=['scrublet_score','stress_score', 'hb1_score','phase'], ncols=5,)
sc.pl.umap(adata, color=['donor','sex','age','region','phase','cell_or_nuclei','modality','kit_10x','FACSgate'],wspace=0.8,ncols=5)
Genes=['TTN','TNNT2','MYOM1','LDB3','ALDH1A2','TNNT1','KRT19','UPK3B','COL6A3','FBLN1','PCOLCE','LRRC17','EMCN','ECSCR','PECAM1','CDH5', 'LAPTM5','PTPRC','LCP1','CD53','PHOX2B','NRXN1','STMN2','GRIK2']
sc.pl.umap(adata, color=Genes)
```

Example 46 (python):
```python
sc.pl.umap(adata, color='fine_grain')
```

Example 47 (python):
```python
sc.pl.umap(adata, color='age')
```

Example 48 (python):
```python
sc.pl.umap(adata, color='region')
```

---

## Import modules¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig5_Trisomy21/no02_t21_post-scVI_cleanup_and_celltypist.html

**Contents:**
- Import modules¶
- Read in data¶
- Filter and log-normalise¶
- CellTypist prediction¶
- Select cells to remove¶
- Remove from the raw data¶
- recalculate UMAP, etc¶
- Modify obs¶
- CellTypist prediction for fine-grained labels (run per coarse-grained)¶

"Pericardium" region is not represented in "Heart" and other region samples. In addition, "Pericardium" sample is coming from one donor "Pericardium" sample will be in the dataset but will be removed at downstream analysis (e.g. Milo)

**Examples:**

Example 1 (python):
```python
import numpy as np
import numpy.random as random
import pandas as pd
import scanpy as sc
import matplotlib.pyplot as plt
import seaborn as sns
import celltypist
```

Example 2 (python):
```python
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm
```

Example 3 (python):
```python
import session_info
session_info.show()
```

Example 4 (python):
```python
-----
celltypist          1.6.2
matplotlib          3.7.2
numpy               1.22.4
pandas              1.5.3
scanpy              1.9.3
seaborn             0.12.2
session_info        1.0.0
-----
```

Example 5 (python):
```python
PIL                         10.0.0
anndata                     0.9.2
anyio                       NA
arrow                       1.2.3
asciitree                   NA
asttokens                   NA
attr                        23.1.0
attrs                       23.1.0
babel                       2.12.1
backcall                    0.2.0
brotli                      1.0.9
certifi                     2023.11.17
cffi                        1.15.1
charset_normalizer          3.2.0
cloudpickle                 3.0.0
colorama                    0.4.6
comm                        0.1.4
cycler                      0.10.0
cython_runtime              NA
dask                        2023.5.0
dateutil                    2.8.2
debugpy                     1.6.8
decorator                   5.1.1
defusedxml                  0.7.1
executing                   1.2.0
fasteners                   0.19
fastjsonschema              NA
fqdn                        NA
h5py                        3.9.0
idna                        3.4
igraph                      0.10.6
importlib_metadata          NA
importlib_resources         NA
ipykernel                   6.25.1
isoduration                 NA
jedi                        0.19.0
jinja2                      3.1.2
joblib                      1.3.2
json5                       NA
jsonpointer                 2.0
jsonschema                  4.19.0
jsonschema_specifications   NA
jupyter_events              0.7.0
jupyter_server              2.7.1
jupyterlab_server           2.24.0
kiwisolver                  1.4.5
leidenalg                   0.10.1
llvmlite                    0.38.1
markupsafe                  2.1.3
mpl_toolkits                NA
natsort                     8.4.0
nbformat                    5.9.2
numba                       0.55.2
numcodecs                   0.12.1
opt_einsum                  v3.3.0
overrides                   NA
packaging                   23.1
parso                       0.8.3
patsy                       0.5.3
pexpect                     4.8.0
pickleshare                 0.7.5
pkg_resources               NA
plotly                      5.16.1
prometheus_client           NA
prompt_toolkit              3.0.39
psutil                      5.9.5
ptyprocess                  0.7.0
pure_eval                   0.2.2
pyarrow                     15.0.2
pydev_ipython               NA
pydevconsole                NA
pydevd                      2.9.5
pydevd_file_utils           NA
pydevd_plugins              NA
pydevd_tracing              NA
pygments                    2.16.1
pyparsing                   3.0.9
pythonjsonlogger            NA
pytz                        2023.3
referencing                 NA
requests                    2.31.0
rfc3339_validator           0.1.4
rfc3986_validator           0.1.1
rpds                        NA
ruamel                      NA
scipy                       1.10.1
send2trash                  NA
six                         1.16.0
sklearn                     1.3.0
sniffio                     1.3.0
socks                       1.7.1
sphinxcontrib               NA
stack_data                  0.6.2
statsmodels                 0.14.0
texttable                   1.6.7
threadpoolctl               3.2.0
tlz                         0.12.1
toolz                       0.12.1
torch                       1.13.1+cu117
tornado                     6.3.3
tqdm                        4.66.1
traitlets                   5.9.0
typing_extensions           NA
uri_template                NA
urllib3                     2.0.4
wcwidth                     0.2.6
webcolors                   1.13
websocket                   1.6.2
yaml                        6.0.1
zarr                        2.16.1
zipp                        NA
zmq                         25.1.1
zstandard                   0.19.0
```

Example 6 (unknown):
```unknown
-----
IPython             8.12.0
jupyter_client      8.3.0
jupyter_core        4.12.0
jupyterlab          4.0.5
notebook            7.0.2
-----
Python 3.8.17 (default, Jul  5 2023, 21:04:15) [GCC 11.2.0]
Linux-5.15.0-113-generic-x86_64-with-glibc2.17
-----
Session information updated at 2024-08-01 08:21
```

Example 7 (python):
```python
# import own function
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 8 (python):
```python
sc.settings.verbosity = 3             # verbosity: errors (0), warnings (1), info (2), hints (3)
sc.settings.set_figure_params(dpi=120)
```

Example 9 (python):
```python
adata = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/trisomy21/T21Hearts_July2024.h5ad')
adata
```

Example 10 (unknown):
```unknown
AnnData object with n_obs × n_vars = 121452 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'batch_key', 'leiden_scVI_0.1', 'leiden_scVI_0.2', 'leiden_scVI_0.3', 'leiden_scVI_0.5', 'leiden_scVI_0.7', 'leiden_scVI_1'
    var: 'gene_name', 'feature_type', 'ambient_expression-0', 'ambient_expression-1', 'ambient_expression-10', 'ambient_expression-11', 'ambient_expression-12', 'ambient_expression-13', 'ambient_expression-14', 'ambient_expression-2', 'ambient_expression-3', 'ambient_expression-4', 'ambient_expression-5', 'ambient_expression-6', 'ambient_expression-7', 'ambient_expression-8', 'ambient_expression-9', 'mt', 'ribo', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts'
    uns: 'age_colors', 'donor_colors', 'leiden', 'leiden_scVI_0.1_colors', 'leiden_scVI_0.2_colors', 'leiden_scVI_0.3_colors', 'leiden_scVI_0.5_colors', 'leiden_scVI_0.7_colors', 'leiden_scVI_1_colors', 'neighbors', 'region_colors', 'sangerID_colors', 'umap'
    obsm: 'X_scVI', 'X_umap', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 11 (python):
```python
adata.var['gene_id'] = adata.var_names.copy()
adata.var.set_index('gene_name',inplace=True)
adata.var_names = adata.var_names.astype('str')
adata.var_names_make_unique()
```

Example 12 (python):
```python
sc.pp.filter_genes(adata, min_cells=3)
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)
```

Example 13 (unknown):
```unknown
filtered out 3535 genes that are detected in less than 3 cells
normalizing counts per cell
    finished (0:00:01)
```

Example 14 (python):
```python
print(adata.shape)
adata.X.data[:5]
```

Example 15 (unknown):
```unknown
(121452, 33066)
```

Example 16 (unknown):
```unknown
array([0.0534004 , 0.10409318, 0.19836526, 0.0534004 , 0.15233971],
      dtype=float32)
```

Example 17 (python):
```python
# coarse-grained
predictions = celltypist.annotate(adata, 
                                  model = '/nfs/team205/kk18/notebooks/Foetal/VisiumHD/mpp0p3/celltypist_models/from_foetal_portal/foetal_heart_coarse.pkl', 
                                  majority_voting = False)

# add to anndata
adata = predictions.to_adata(insert_labels = True, insert_conf = True)
adata.obs.rename(columns={'predicted_labels':'celltypist_coarse',
                              'conf_score':'conf_score_coarse'
                             },inplace=True)
adata.obs.head()
```

Example 18 (unknown):
```unknown
🔬 Input data has 121452 cells and 33066 genes
🔗 Matching reference genes in the model
🧬 29480 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
```

Example 19 (python):
```python
sc.pl.umap(adata, color=['celltypist_coarse','conf_score_coarse','leiden_scVI_1'], size=1, legend_fontsize=6, wspace=0.2)
```

Example 20 (python):
```python
# umap per celltype to highlight
group_col = 'region'
groups_to_plot = ['Heart',
                  'Great vessels', 
                 'Left atria',
                 'Left ventricle',
                 'Pericardial sac + RA + LA + IAS',
                 'Right ventricle',
                  'Pericardium',
                 ]

plt.rcParams['axes.grid'] = False
fig,axs=plt.subplots(nrows=2,ncols=4,figsize=(15,7))
# Flatten the axs array for easier iteration in case of multiple rows and columns
axs_flat = axs.flatten()

for i,grp in enumerate(groups_to_plot):
    sc.pl.umap(adata, color=[group_col], groups=grp, 
               size=10, frameon=False,
               show=False, ax=axs_flat[i], legend_loc=None, legend_fontsize=5,
               title=grp
              )
# Turn off any remaining subplots
for j in range(len(groups_to_plot), len(axs_flat)):
    fig.delaxes(axs_flat[j])

# fig.suptitle('Main Title for the Figure')
fig.tight_layout()
```

Example 21 (python):
```python
# mean condidence score
group_col = 'leiden_scVI_1'
conf_score_col = 'conf_score_coarse'
df = adata.obs[[group_col,conf_score_col]].groupby(group_col).mean()
df.sort_values(conf_score_col,ascending=False)
```

Example 22 (python):
```python
# cell type marker expression
markers = {
    'vCM': ['MYH7', 'MYL2', 'FHL2'],
    'aCM': ['NPPA', 'MYL7', 'MYL4'],
    'FB': ['DCN', 'GCN', 'PDGFRA'],
    'EC': ['VWF', 'PECAM1', 'CDH5'],
    'Peri': ['RGS5', 'ABCC9', 'KCNJ8'],
    'SMC': ['MYH11', 'TAGLN', 'ACTA2'],
    'Meso': ['MSLN', 'WT1', 'BNC1'],
    'Neuro': ['PLP1', 'NRXN1', 'NRXN3'],
    'Adipo': ['GPAM', 'FASN', 'LEP'],
    'Myelo': ['CD14', 'C1QA', 'CD68','LYVE1','TIMD4'],
    'Lymph': ['CD8A', 'IL7R', 'CD40LG'],
    'B': ['CD19','SDC1','IGHG1','IGHA1'],
    'Mast': ['KIT', 'CPA3'],
}

for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in adata.var_names]

sc.tl.dendrogram(adata, groupby=group_col)
sc.pl.dotplot(adata,
              markers, 
              groupby=group_col,
              dendrogram=True,
              standard_scale="var",
              color_map="Reds",
              # swap_axes=True,
             )
```

Example 23 (unknown):
```unknown
using 'X_pca' with n_pcs = 50
Storing dendrogram info using `.uns['dendrogram_leiden_scVI_1']`
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: vCM, aCM, FB, etc.
```

Example 24 (python):
```python
# sangerID proportion
crosstab=pd.crosstab(adata.obs[group_col],adata.obs["region"])
# Normalize each row to sum to 1 (i.e., to get fractions)
crosstab_normalized = crosstab.div(crosstab.sum(axis=1), axis=0)

# Plotting the stacked bar chart
ax = crosstab_normalized.plot(kind='bar',stacked=True,figsize=(10, 4),legend=False)

# Adding labels and title
plt.ylabel('Fraction')
plt.title('Normalized Stacked Bar Chart')
plt.legend(bbox_to_anchor=(0.95, 0.5, 0.5, 0.5))

# Show the plot
plt.show()
```

Example 25 (python):
```python
# donor proportion
crosstab=pd.crosstab(adata.obs[group_col],adata.obs["donor"])
# Normalize each row to sum to 1 (i.e., to get fractions)
crosstab_normalized = crosstab.div(crosstab.sum(axis=1), axis=0)

# Plotting the stacked bar chart
ax = crosstab_normalized.plot(kind='bar',stacked=True,figsize=(10, 4),legend=False)

# Adding labels and title
plt.ylabel('Fraction')
plt.title('Normalized Stacked Bar Chart')
plt.legend(bbox_to_anchor=(0.8, 0., 0.5, 0.5))

# Show the plot
plt.show()
```

Example 26 (python):
```python
# sangerID proportion
crosstab=pd.crosstab(adata.obs[group_col],adata.obs["sangerID"])
# Normalize each row to sum to 1 (i.e., to get fractions)
crosstab_normalized = crosstab.div(crosstab.sum(axis=1), axis=0)

# Plotting the stacked bar chart
ax = crosstab_normalized.plot(kind='bar',stacked=True,figsize=(10, 4),legend=False)

# Adding labels and title
plt.ylabel('Fraction')
plt.title('Normalized Stacked Bar Chart')
plt.legend(bbox_to_anchor=(0.8, 0.5, 0.5, 0.5))

# Show the plot
plt.show()
```

Example 27 (python):
```python
sc.settings.set_figure_params(dpi=120)
sc.pl.umap(adata, color=['leiden_scVI_1'], size=1, legend_fontsize=6, wspace=0.2, legend_loc='on data')
```

Example 28 (python):
```python
conf_score_thresh = 0.6

# identify single-sample clusters
crosstab=pd.crosstab(adata.obs[group_col],adata.obs["sangerID"])
crosstab_normalized = crosstab.div(crosstab.sum(axis=1), axis=0)
res = crosstab_normalized.max(axis=1)
single_sample_clusters = list(res.index[res>0.9])
print(f'single_sample_clusters: {single_sample_clusters}')

# identify pericardium clusters
crosstab=pd.crosstab(adata.obs[group_col],adata.obs["region"])
crosstab_normalized = crosstab.div(crosstab.sum(axis=1), axis=0)
res = crosstab_normalized['Pericardium']
pericardium_clusters = list(res.index[res>0.9])
print(f'pericardium_clusters: {pericardium_clusters}')

single_butnot_peri_clusters = list(set(single_sample_clusters)-set(pericardium_clusters))
print(f'single_butnot_peri_clusters: {single_butnot_peri_clusters}')

mask1 = adata.obs[group_col].isin(['4','19'])
mask2 = adata.obs[conf_score_col] < conf_score_thresh
mask3 = adata.obs[group_col].isin(single_butnot_peri_clusters)
adata.obs['cells_to_remove'] = 'No'
adata.obs.loc[adata.obs_names[mask1|mask2|mask3],'cells_to_remove'] = 'Yes'
sc.pl.umap(adata, color=['cells_to_remove'], size=1, legend_fontsize=6, wspace=0.2)
```

Example 29 (unknown):
```unknown
... storing 'cells_to_remove' as categorical
```

Example 30 (unknown):
```unknown
single_sample_clusters: ['4', '12', '14', '17', '19', '21', '27', '30', '36']
pericardium_clusters: ['12', '14', '17', '19', '21', '27', '30']
single_butnot_peri_clusters: ['36', '4']
```

Example 31 (python):
```python
adata_raw = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/trisomy21/T21Hearts_July2024.h5ad')
adata_raw
```

Example 32 (unknown):
```unknown
AnnData object with n_obs × n_vars = 121452 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'batch_key', 'leiden_scVI_0.1', 'leiden_scVI_0.2', 'leiden_scVI_0.3', 'leiden_scVI_0.5', 'leiden_scVI_0.7', 'leiden_scVI_1'
    var: 'gene_name', 'feature_type', 'ambient_expression-0', 'ambient_expression-1', 'ambient_expression-10', 'ambient_expression-11', 'ambient_expression-12', 'ambient_expression-13', 'ambient_expression-14', 'ambient_expression-2', 'ambient_expression-3', 'ambient_expression-4', 'ambient_expression-5', 'ambient_expression-6', 'ambient_expression-7', 'ambient_expression-8', 'ambient_expression-9', 'mt', 'ribo', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts'
    uns: 'age_colors', 'donor_colors', 'leiden', 'leiden_scVI_0.1_colors', 'leiden_scVI_0.2_colors', 'leiden_scVI_0.3_colors', 'leiden_scVI_0.5_colors', 'leiden_scVI_0.7_colors', 'leiden_scVI_1_colors', 'neighbors', 'region_colors', 'sangerID_colors', 'umap'
    obsm: 'X_scVI', 'X_umap', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 33 (python):
```python
adata_raw.var['gene_id'] = adata_raw.var_names.copy()
adata_raw.var.set_index('gene_name',inplace=True)
adata_raw.var_names = adata_raw.var_names.astype('str')
adata_raw.var_names_make_unique()
```

Example 34 (python):
```python
# add prediction results, etc
adata_raw.obs[['celltypist_coarse','conf_score_coarse','cells_to_remove']] = adata[adata_raw.obs_names].obs[['celltypist_coarse','conf_score_coarse','cells_to_remove']].copy()
```

Example 35 (python):
```python
# remove
adata_raw = adata_raw[adata_raw.obs['cells_to_remove']=='No']
sc.pl.umap(adata_raw, color=['cells_to_remove'], size=1, legend_fontsize=6, wspace=0.2)
```

Example 36 (python):
```python
# save
adata_raw.write('/nfs/team205/heart/anndata_objects/Foetal/trisomy21/T21Hearts_cleaned_July2024.h5ad')
```

Example 37 (python):
```python
%%time
sc.pp.neighbors(adata_raw, use_rep="X_scVI")
sc.tl.umap(adata_raw, min_dist=0.2)
```

Example 38 (python):
```python
computing neighbors
    finished: added to `.uns['neighbors']`
    `.obsp['distances']`, distances for each pair of neighbors
    `.obsp['connectivities']`, weighted adjacency matrix (0:00:37)
computing UMAP
    finished: added
    'X_umap', UMAP coordinates (adata.obsm) (0:02:11)
CPU times: user 2min 46s, sys: 733 ms, total: 2min 47s
Wall time: 2min 49s
```

Example 39 (python):
```python
sc.pl.umap(adata_raw, color=['celltypist_coarse'], frameon=False)
```

Example 40 (python):
```python
sc.pl.umap(adata_raw, color=['donor'], frameon=False)
```

Example 41 (python):
```python
# save
adata_raw.write('/nfs/team205/heart/anndata_objects/Foetal/trisomy21/T21Hearts_cleaned_July2024.h5ad')
```

Example 42 (python):
```python
# age
adata_raw.obs.replace({'age':{
    'PCW-11w':'11W0D',
    'PCW-13w':'13W0D',
    'PCW-14w':'14W0D',
}},inplace=True)

adata_raw.obs['week'] = [x.split('0D')[0] for x in adata_raw.obs['age']]
adata_raw.obs['trimester'] = adata_raw.obs['age'].astype('str').copy()
adata_raw.obs.replace({'trimester':{
    '11W0D':'firsttrimester',
    '13W0D':'secondtrimester',
    '14W0D':'secondtrimester',
}},inplace=True)
```

Example 43 (python):
```python
adata_raw.obs.drop(['gender'],axis=1,inplace=True)
```

Example 44 (python):
```python
# sex
adata_raw.obs['sex'] = adata_raw.obs['donor'].astype('str').copy()
adata_raw.obs.replace({'sex':{
    'Hst32':'female',
    'Hst42':'male',
    'Hst44':'male',
    'Hst48':'male'
}},inplace=True)
```

Example 45 (python):
```python
# save
adata_raw.write('/nfs/team205/heart/anndata_objects/Foetal/trisomy21/T21Hearts_cleaned_July2024.h5ad')
adata_raw
```

Example 46 (unknown):
```unknown
... storing 'week' as categorical
... storing 'trimester' as categorical
... storing 'sex' as categorical
```

Example 47 (unknown):
```unknown
AnnData object with n_obs × n_vars = 110693 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'batch_key', 'leiden_scVI_0.1', 'leiden_scVI_0.2', 'leiden_scVI_0.3', 'leiden_scVI_0.5', 'leiden_scVI_0.7', 'leiden_scVI_1', 'celltypist_coarse', 'conf_score_coarse', 'cells_to_remove', 'week', 'trimester', 'sex'
    var: 'feature_type', 'ambient_expression-0', 'ambient_expression-1', 'ambient_expression-10', 'ambient_expression-11', 'ambient_expression-12', 'ambient_expression-13', 'ambient_expression-14', 'ambient_expression-2', 'ambient_expression-3', 'ambient_expression-4', 'ambient_expression-5', 'ambient_expression-6', 'ambient_expression-7', 'ambient_expression-8', 'ambient_expression-9', 'mt', 'ribo', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'gene_id'
    uns: 'age_colors', 'donor_colors', 'leiden', 'leiden_scVI_0.1_colors', 'leiden_scVI_0.2_colors', 'leiden_scVI_0.3_colors', 'leiden_scVI_0.5_colors', 'leiden_scVI_0.7_colors', 'leiden_scVI_1_colors', 'neighbors', 'region_colors', 'sangerID_colors', 'umap', 'cells_to_remove_colors', 'celltypist_coarse_colors'
    obsm: 'X_scVI', 'X_umap', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 48 (python):
```python
adata_raw = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/trisomy21/T21Hearts_cleaned_July2024.h5ad')
adata_raw
```

Example 49 (unknown):
```unknown
AnnData object with n_obs × n_vars = 110693 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'batch_key', 'leiden_scVI_0.1', 'leiden_scVI_0.2', 'leiden_scVI_0.3', 'leiden_scVI_0.5', 'leiden_scVI_0.7', 'leiden_scVI_1', 'celltypist_coarse', 'conf_score_coarse', 'cells_to_remove', 'week', 'trimester', 'sex', 'celltypist_fine', 'conf_score_fine'
    var: 'feature_type', 'ambient_expression-0', 'ambient_expression-1', 'ambient_expression-10', 'ambient_expression-11', 'ambient_expression-12', 'ambient_expression-13', 'ambient_expression-14', 'ambient_expression-2', 'ambient_expression-3', 'ambient_expression-4', 'ambient_expression-5', 'ambient_expression-6', 'ambient_expression-7', 'ambient_expression-8', 'ambient_expression-9', 'mt', 'ribo', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'gene_id'
    uns: 'age_colors', 'cells_to_remove_colors', 'celltypist_coarse_colors', 'donor_colors', 'leiden', 'leiden_scVI_0.1_colors', 'leiden_scVI_0.2_colors', 'leiden_scVI_0.3_colors', 'leiden_scVI_0.5_colors', 'leiden_scVI_0.7_colors', 'leiden_scVI_1_colors', 'neighbors', 'region_colors', 'sangerID_colors', 'umap'
    obsm: 'X_scVI', 'X_umap', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 50 (python):
```python
adata = adata_raw.copy()
sc.pp.filter_genes(adata, min_cells=3)
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)
adata.X.data[:5]
```

Example 51 (unknown):
```unknown
filtered out 3663 genes that are detected in less than 3 cells
normalizing counts per cell
    finished (0:00:00)
```

Example 52 (unknown):
```unknown
array([0.0534004 , 0.10409318, 0.19836526, 0.0534004 , 0.15233971],
      dtype=float32)
```

Example 53 (python):
```python
adata.obs['celltypist_coarse'].value_counts()
```

Example 54 (unknown):
```unknown
Mesenchymal       49009
Cardiomyocytes    42792
Endothelium        8089
Leukocytes         7262
Epicardium         2071
Neural             1470
Name: celltypist_coarse, dtype: int64
```

Example 55 (python):
```python
# predict per coarse_grained cell type
for celltype in adata.obs['celltypist_coarse'].unique():
    print(celltype)
    # subset adata
    adata_sub = adata[adata.obs['celltypist_coarse']==celltype]
    # drop all the obs columns
    adata_sub.obs.drop(columns=adata_sub.obs.columns,inplace=True)
    # predict
    predictions = celltypist.annotate(adata_sub, 
                                      model = f'/nfs/team205/kk18/notebooks/Foetal/Trisomy21/celltypist_models/coarse2fine_{celltype}.pkl', 
                                      majority_voting = False)

    # add to subsetted anndata
    adata_sub = predictions.to_adata(insert_labels = True, insert_conf = True)
    adata_sub.obs.rename(columns={'predicted_labels':'celltypist_fine',
                                  'conf_score':'conf_score_fine'
                                 },inplace=True)
    # add to the original anndata
    adata.obs.loc[adata_sub.obs_names,'celltypist_fine'] = list(adata_sub.obs['celltypist_fine'])
    adata.obs.loc[adata_sub.obs_names,'conf_score_fine'] = list(adata_sub.obs['conf_score_fine'])
```

Example 56 (unknown):
```unknown
Mesenchymal
```

Example 57 (unknown):
```unknown
🔬 Input data has 49009 cells and 32938 genes
🔗 Matching reference genes in the model
🧬 32502 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
🔬 Input data has 8089 cells and 32938 genes
🔗 Matching reference genes in the model
```

Example 58 (unknown):
```unknown
Endothelium
```

Example 59 (unknown):
```unknown
🧬 31696 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
🔬 Input data has 2071 cells and 32938 genes
🔗 Matching reference genes in the model
```

Example 60 (unknown):
```unknown
🧬 30529 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
```

Example 61 (unknown):
```unknown
Cardiomyocytes
```

Example 62 (unknown):
```unknown
🔬 Input data has 42792 cells and 32938 genes
🔗 Matching reference genes in the model
🧬 31533 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
🔬 Input data has 1470 cells and 32938 genes
🔗 Matching reference genes in the model
```

Example 63 (unknown):
```unknown
🧬 29361 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
🔬 Input data has 7262 cells and 32938 genes
🔗 Matching reference genes in the model
```

Example 64 (unknown):
```unknown
🧬 31793 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
```

Example 65 (python):
```python
# add prediction results
adata_raw.obs[['celltypist_fine','conf_score_fine']] = adata[adata_raw.obs_names].obs[['celltypist_fine','conf_score_fine']].copy()
```

Example 66 (python):
```python
sc.pl.umap(adata_raw, color=['conf_score_fine','celltypist_fine'], frameon=False)
```

Example 67 (python):
```python
sc.pl.umap(adata_raw, color=['celltypist_coarse'], frameon=False)
```

Example 68 (python):
```python
# save
adata_raw.write('/nfs/team205/heart/anndata_objects/Foetal/trisomy21/T21Hearts_cleaned_July2024.h5ad')
adata_raw
```

Example 69 (unknown):
```unknown
AnnData object with n_obs × n_vars = 110693 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'batch_key', 'leiden_scVI_0.1', 'leiden_scVI_0.2', 'leiden_scVI_0.3', 'leiden_scVI_0.5', 'leiden_scVI_0.7', 'leiden_scVI_1', 'celltypist_coarse', 'conf_score_coarse', 'cells_to_remove', 'week', 'trimester', 'sex', 'celltypist_fine', 'conf_score_fine'
    var: 'feature_type', 'ambient_expression-0', 'ambient_expression-1', 'ambient_expression-10', 'ambient_expression-11', 'ambient_expression-12', 'ambient_expression-13', 'ambient_expression-14', 'ambient_expression-2', 'ambient_expression-3', 'ambient_expression-4', 'ambient_expression-5', 'ambient_expression-6', 'ambient_expression-7', 'ambient_expression-8', 'ambient_expression-9', 'mt', 'ribo', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'gene_id'
    uns: 'age_colors', 'cells_to_remove_colors', 'celltypist_coarse_colors', 'donor_colors', 'leiden', 'leiden_scVI_0.1_colors', 'leiden_scVI_0.2_colors', 'leiden_scVI_0.3_colors', 'leiden_scVI_0.5_colors', 'leiden_scVI_0.7_colors', 'leiden_scVI_1_colors', 'neighbors', 'region_colors', 'sangerID_colors', 'umap', 'celltypist_fine_colors'
    obsm: 'X_scVI', 'X_umap', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 70 (python):
```python
# save .obs for later use
adata_raw.obs.to_csv('/nfs/team205/heart/anndata_objects/Foetal/trisomy21/T21Hearts_cleaned_July2024.obs.csv')
```

---

## Variables¶

**URL:** http://127.0.0.1:9020/notebooks_html/Supplementary_materials/Macrophages/Mo-MP_no1_MultiVI_prep-CPU.html

**Contents:**
- Variables¶
- Read in¶
- Prepare paired anndata¶
- Prepare RNA data¶

36601 rows × 3 columns

**Examples:**

Example 1 (python):
```python
import scanpy as sc
import pandas as pd
import numpy as np
import anndata
import scipy

from scipy.io import mmwrite, mmread
from scipy.sparse import csr_matrix

import sctk
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (unknown):
```unknown
-----
anndata             0.8.0
numpy               1.21.5
pandas              1.3.5
scanpy              1.8.2
scipy               1.7.3
sctk                NA
session_info        1.0.0
-----
```

Example 4 (python):
```python
PIL                 9.1.1
asciitree           NA
backcall            0.2.0
beta_ufunc          NA
binom_ufunc         NA
cached_property     1.5.1
cairo               1.20.0
cffi                1.14.4
cloudpickle         1.6.0
colorama            0.4.4
cycler              0.10.0
cython_runtime      NA
cytoolz             0.11.2
dask                2022.01.0
dateutil            2.8.1
decorator           4.4.2
defusedxml          0.6.0
fasteners           0.17.3
fsspec              2022.01.0
google              NA
h5py                3.1.0
igraph              0.10.2
importlib_metadata  1.7.0
ipykernel           5.4.2
ipython_genutils    0.2.0
ipywidgets          7.6.3
jedi                0.17.2
jinja2              3.1.2
joblib              1.0.0
kiwisolver          1.3.1
leidenalg           0.9.0
llvmlite            0.35.0
louvain             0.7.0
markupsafe          2.1.1
matplotlib          3.5.2
mpl_toolkits        NA
msgpack             1.0.3
natsort             7.1.0
nbinom_ufunc        NA
numba               0.52.0
numcodecs           0.9.1
numexpr             2.7.2
numpy_groupies      0.9.13
packaging           21.3
parso               0.7.1
pexpect             4.8.0
pickleshare         0.7.5
pkg_resources       NA
plotly              5.10.0
prompt_toolkit      3.0.10
psutil              5.9.0
ptyprocess          0.7.0
pyarrow             9.0.0
pygments            2.13.0
pyparsing           2.4.7
pytz                2020.5
scvelo              0.2.4
seaborn             0.11.1
setuptools_scm      NA
sinfo               0.3.1
six                 1.14.0
sklearn             1.0.2
sphinxcontrib       NA
statsmodels         0.12.1
storemagic          NA
tables              3.6.1
tblib               1.7.0
texttable           1.6.3
threadpoolctl       2.1.0
tlz                 0.11.2
toolz               0.11.2
tornado             6.1
traitlets           5.0.5
typing_extensions   NA
wcwidth             0.2.5
yaml                5.4.1
zarr                2.11.3
zipp                NA
zmq                 21.0.1
zope                NA
```

Example 5 (unknown):
```unknown
-----
IPython             7.19.0
jupyter_client      6.1.11
jupyter_core        4.7.0
notebook            6.2.0
-----
Python 3.7.9 | packaged by conda-forge | (default, Dec  9 2020, 21:08:20) [GCC 9.3.0]
Linux-4.15.0-158-generic-x86_64-with-debian-bullseye-sid
-----
Session information updated at 2023-03-15 10:59
```

Example 6 (python):
```python
sc.settings.set_figure_params(dpi=120)
```

Example 7 (unknown):
```unknown
cells_to_analyse = ['MonocytesMPOpos', 
                        'Monocytes', 
                        'MonocyteDerivedCells',
                        'MacrophagesCX3CR1pos', 
                        'MacrophagesTIMD4pos', 
                        'MacrophagesLYVE1pos',
                        'MacrophagesATF3pos'
                   ]

celltype_key = 'fine_grain'

region_to_analyse = ['whole sample']
```

Example 8 (unknown):
```unknown
!ls /nfs/team205/heart/anndata_objects/Foetal
```

Example 9 (python):
```python
14Feb_fromSemih		      metadata
archive			      multiome_ATAC
Feb28ObjectRaw.h5ad	      QC
Feb28ObjectRaw_Multiome.h5ad  RNA_foetal_post-scVI_raw.h5ad
Feb28ObjectRaw_Multiome.rds   RNA_foetal_raw.h5ad
Feb28ObjectRaw.obs.csv	      scVI
Feb28ObjectRaw_original.h5ad  visium_foetal_raw.h5ad
Immune
```

Example 10 (python):
```python
# rna
adata_rna = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/Feb28ObjectRaw.h5ad')
print(adata_rna.obs[celltype_key].value_counts())

# subset cells
adata_rna = adata_rna[adata_rna.obs[celltype_key].isin(cells_to_analyse)]

# subset regions
adata_rna = adata_rna[adata_rna.obs['region'].isin(region_to_analyse)]

# add modality to .var
adata_rna.var['modality']='Gene Expression'

adata_rna
```

Example 11 (unknown):
```unknown
GreatVesselAdventitialFibroblasts        61593
GreatVesselSmoothMuscleCells             22992
VentricularCardiomyocytesRightCompact    16215
VentricularCardiomyocytesLeftCompact     15591
MyocardialInterstitialFibroblasts1       13880
                                         ...  
MacrophagesTIMD4pos                        470
MonocytesMPOpos                            414
Megakaryocytes                             360
ChromaffinCells                            228
DendriticCellsMature                       184
Name: fine_grain, Length: 63, dtype: int64
```

Example 12 (unknown):
```unknown
AnnData object with n_obs × n_vars = 7905 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score'
    var: 'gene_name_scRNA-0', 'gene_id', 'modality'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'age_colors', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'cycling_colors', 'donor_colors', 'fine_grain_colors', 'heart_or_greatvessels_colors', 'hvg', 'kit_10x_colors', 'modality_colors', 'neighbors', 'phase_colors', 'region_colors', 'sex_colors', 'trimester_colors', 'umap', 'week_colors'
    obsm: 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 13 (python):
```python
pd.crosstab(adata_rna.obs[celltype_key],adata_rna.obs['region'])
```

Example 14 (python):
```python
pd.crosstab(adata_rna.obs[celltype_key],adata_rna.obs['kit_10x'])
```

Example 15 (python):
```python
pd.crosstab(adata_rna.obs[celltype_key],adata_rna.obs['cycling'])
```

Example 16 (python):
```python
adata_rna.obs['kit_10x'].value_counts()
```

Example 17 (unknown):
```unknown
5prime-v2      5775
Multiome-v1    2130
Name: kit_10x, dtype: int64
```

Example 18 (python):
```python
# atac
# ie only Multiome data
adata_atac = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/PeakMatrix/Foetal_Peaks.h5ad')

# subset cells
adata_atac = adata_atac[adata_atac.obs[celltype_key].isin(cells_to_analyse)]
print(adata_atac.obs[celltype_key].value_counts())

# subset regions
adata_atac = adata_atac[adata_atac.obs['region'].isin(region_to_analyse)]

# add modality to .var
adata_atac.var['modality']='Peaks'

adata_atac
```

Example 19 (unknown):
```unknown
MacrophagesLYVE1pos     4157
MacrophagesATF3pos       611
MacrophagesCX3CR1pos     551
MonocyteDerivedCells     386
MacrophagesTIMD4pos      303
Monocytes                117
MonocytesMPOpos           34
Name: fine_grain, dtype: int64
```

Example 20 (unknown):
```unknown
AnnData object with n_obs × n_vars = 1899 × 508040
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score'
    var: 'modality'
```

Example 21 (python):
```python
pd.crosstab(adata_atac.obs[celltype_key],adata_atac.obs['region'])
```

Example 22 (python):
```python
# shared barcodes, ie Multiome data (not all, since MultiomeRNA has a bit more data than MultiomeATAC)
barcodes_inter=list(set(adata_rna.obs_names).intersection(adata_atac.obs_names))
len(barcodes_inter)

# take shared cells
multiome_rna=adata_rna[barcodes_inter,:]
multiome_atac=adata_atac[barcodes_inter,:]

# concatenate
adata_paired=anndata.AnnData(
    X=scipy.sparse.hstack([multiome_rna.X,multiome_atac.X]).tocsr(), # concatenate sparse matrix and convert to Compressed Sparse Row format
    obs=multiome_rna.obs[['sangerID', 'combinedID', 'donor', 
                       'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 
                       'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts',
                       'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts',
                       'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt',
                       'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo',
                       'pct_counts_ribo', 'HB_score', 'multiplexed',
                        'batch_key','region','FACSgate','sex','week','trimester',
                     'fine_grain','mid_grain','coarse_grain',
                     'heart_or_greatvessels','cycling','S_score','G2M_score','phase','stress_score','hb1_score']],
    var=pd.concat([multiome_rna.var[['modality','gene_name_scRNA-0','gene_id']], \
                   multiome_atac.var[['modality']]
                  ],axis=0)
)
adata_paired
```

Example 23 (unknown):
```unknown
AnnData object with n_obs × n_vars = 1899 × 544641
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id'
```

Example 24 (python):
```python
# modify modality columns in .obs
adata_paired.obs['modality'] = 'Multiome'
adata_paired.obs['modality'].value_counts()
```

Example 25 (unknown):
```unknown
Multiome    1899
Name: modality, dtype: int64
```

Example 26 (python):
```python
# save
adata_paired.write('/nfs/team205/heart/anndata_objects/Foetal/Immune/MultiVI-pre_paired_foetal_MoMP_raw.h5ad')
```

Example 27 (python):
```python
adata_paired.obs.head()
```

Example 28 (python):
```python
# select single-cell data (3p and 5p)
adata_scrna = adata_rna[adata_rna.obs['kit_10x'].isin(['3prime-v3','5prime-v2'])]
adata_scrna.obs['kit_10x'].value_counts()
```

Example 29 (unknown):
```unknown
5prime-v2    5775
Name: kit_10x, dtype: int64
```

Example 30 (python):
```python
adata_scrna.X.data[:10]
```

Example 31 (unknown):
```unknown
array([1., 5., 1., 2., 1., 3., 1., 1., 2., 1.], dtype=float32)
```

Example 32 (python):
```python
# recreate anndata
adata_scrna=anndata.AnnData(
    X=adata_scrna.X, # concatenate sparse matrix and convert to Compressed Sparse Row format
    obs=adata_scrna.obs[['sangerID', 'combinedID', 'donor', 
                       'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 
                       'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts',
                       'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts',
                       'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt',
                       'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo',
                       'pct_counts_ribo', 'HB_score', 'multiplexed',
                        'batch_key','region','FACSgate','sex','week','trimester',
                     'fine_grain','mid_grain','coarse_grain',
                     'heart_or_greatvessels','cycling','S_score','G2M_score','phase','stress_score','hb1_score']],
    var=adata_scrna.var[['modality','gene_name_scRNA-0','gene_id']]
)
adata_scrna
```

Example 33 (unknown):
```unknown
AnnData object with n_obs × n_vars = 5775 × 36601
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id'
```

Example 34 (python):
```python
# modify modality columns in .obs
adata_scrna.obs['modality'] = 'scRNA'
adata_scrna.obs['modality'].value_counts()
```

Example 35 (unknown):
```unknown
scRNA    5775
Name: modality, dtype: int64
```

Example 36 (python):
```python
adata_scrna.var
```

Example 37 (python):
```python
# save
adata_scrna.write('/nfs/team205/heart/anndata_objects/Foetal/Immune/MultiVI-pre_scRNA_foetal_MoMP_raw.h5ad')
```

---

## Loading packages¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig4_Cardiomyocytes/AtrialCardiomyocytes/Visium_mapping/c2l_pt1_reference_FFPE.html

**Contents:**
- Loading packages¶
- Functions¶
- Loading scRNA-seq reference data¶
- Make sure .var_names are ENSEMBL IDs¶
- Reduce to only those features present in the FFPE probeset¶
- Inspect batch key¶
- Make reference signatures¶

297473 rows × 1 columns

**Examples:**

Example 1 (unknown):
```unknown
run = "20Mar2024_fetal_FFPE"
```

Example 2 (unknown):
```unknown
celltype_key = 'fine_grain_modified'
```

Example 3 (python):
```python
# create paths and names to results folders for reference regression and cell2location models
import os
results_folder = os.path.join(os.getcwd(),"analysis/Visium/cell2location_results")
ref_run_name = f'{results_folder}/reference_signatures/{run}'
run_name = f'{results_folder}/cell2location_map/{run}'

os.makedirs(ref_run_name, exist_ok=True)
os.makedirs(run_name, exist_ok=True)
```

Example 4 (python):
```python
import sys
import scanpy as sc
import anndata
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt 
import matplotlib as mpl

import cell2location
import scvi

from matplotlib import rcParams
rcParams['pdf.fonttype'] = 42 # enables correct plotting of text
import seaborn as sns
```

Example 5 (python):
```python
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm
Global seed set to 0
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/umap/distances.py:1063: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/umap/distances.py:1071: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/umap/distances.py:1086: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/umap/umap_.py:660: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
```

Example 6 (python):
```python
import warnings
# Suppress UserWarnings
warnings.filterwarnings('ignore', category=UserWarning)
```

Example 7 (python):
```python
import session_info
session_info.show()
```

Example 8 (python):
```python
-----
anndata             0.8.0
cell2location       0.1.3
matplotlib          3.7.2
numpy               1.24.4
pandas              1.5.3
scanpy              1.9.3
scvi                0.20.3
seaborn             0.12.2
session_info        1.0.0
-----
```

Example 9 (unknown):
```unknown
PIL                         10.0.0
absl                        NA
anyio                       NA
arrow                       1.2.3
asttokens                   NA
attr                        23.1.0
attrs                       23.1.0
babel                       2.12.1
backcall                    0.2.0
brotli                      1.0.9
certifi                     2023.07.22
cffi                        1.15.1
charset_normalizer          3.2.0
chex                        0.1.7
comm                        0.1.3
contextlib2                 NA
cycler                      0.10.0
cython_runtime              NA
dateutil                    2.8.2
debugpy                     1.6.7
decorator                   5.1.1
defusedxml                  0.7.1
docrep                      0.3.2
etils                       1.4.0
executing                   1.2.0
fastjsonschema              NA
flax                        0.7.0
fqdn                        NA
fsspec                      2023.6.0
graphviz                    0.20.1
h5py                        3.9.0
idna                        3.4
igraph                      0.10.6
importlib_resources         NA
ipykernel                   6.25.0
isoduration                 NA
jax                         0.4.14
jaxlib                      0.4.14
jedi                        0.18.2
jinja2                      3.1.2
joblib                      1.3.1
json5                       NA
jsonpointer                 2.0
jsonschema                  4.18.4
jsonschema_specifications   NA
jupyter_events              0.6.3
jupyter_server              2.7.0
jupyterlab_server           2.24.0
kiwisolver                  1.4.4
leidenalg                   0.10.1
lightning_fabric            1.9.5
lightning_utilities         0.9.0
llvmlite                    0.40.1
markupsafe                  2.1.3
ml_collections              NA
ml_dtypes                   0.2.0
mpl_toolkits                NA
mpmath                      1.3.0
msgpack                     1.0.5
mudata                      0.2.3
multipledispatch            0.6.0
natsort                     8.4.0
nbformat                    5.9.1
numba                       0.57.1
numpyro                     0.12.1
nvfuser                     NA
opt_einsum                  v3.3.0
optax                       0.1.7
overrides                   NA
packaging                   23.1
parso                       0.8.3
patsy                       0.5.3
pexpect                     4.8.0
pickleshare                 0.7.5
pkg_resources               NA
platformdirs                3.9.1
plotly                      5.16.1
prometheus_client           NA
prompt_toolkit              3.0.39
psutil                      5.9.5
ptyprocess                  0.7.0
pure_eval                   0.2.2
pyarrow                     12.0.1
pycparser                   2.21
pydev_ipython               NA
pydevconsole                NA
pydevd                      2.9.5
pydevd_file_utils           NA
pydevd_plugins              NA
pydevd_tracing              NA
pygments                    2.15.1
pynndescent                 0.5.10
pyparsing                   3.0.9
pyro                        1.8.6
pythonjsonlogger            NA
pytorch_lightning           1.9.5
pytz                        2023.3
referencing                 NA
regex                       2.5.129
requests                    2.31.0
rfc3339_validator           0.1.4
rfc3986_validator           0.1.1
rich                        NA
rpds                        NA
scipy                       1.11.1
send2trash                  NA
setuptools                  68.0.0
six                         1.16.0
sklearn                     1.3.0
sniffio                     1.3.0
socks                       1.7.1
sparse                      0.14.0
stack_data                  0.6.2
statsmodels                 0.14.0
sympy                       1.12
texttable                   1.6.7
threadpoolctl               3.2.0
toolz                       0.12.0
torch                       2.0.1+cu117
torchmetrics                1.0.1
tornado                     6.3.2
tqdm                        4.65.0
traitlets                   5.9.0
tree                        0.1.8
typing_extensions           NA
umap                        0.5.3
uri_template                NA
urllib3                     1.26.16
wcwidth                     0.2.6
webcolors                   1.13
websocket                   1.6.1
yaml                        6.0
zmq                         25.1.0
```

Example 10 (unknown):
```unknown
-----
IPython             8.14.0
jupyter_client      8.3.0
jupyter_core        5.3.1
jupyterlab          4.0.3
notebook            7.0.0
-----
Python 3.11.4 | packaged by conda-forge | (main, Jun 10 2023, 18:08:17) [GCC 12.2.0]
Linux-5.4.0-167-generic-x86_64-with-glibc2.27
-----
Session information updated at 2023-12-15 15:04
```

Example 11 (python):
```python
# Importing necessary libraries
from cell2location.utils.filtering import filter_genes
from cell2location.models import RegressionModel

# Function to make c2l refernce
def c2l_make_reference(adata, subset_value,max_epochs=250):
    print(f'##### {subset_value} #####')
    print(adata.obs[celltype_key].value_counts())

    print(f'filtering features...')
    selected = filter_genes(adata, cell_count_cutoff=5, cell_percentage_cutoff2=0.03, nonz_mean_cutoff=1.12)
    adata = adata[:, selected].copy()

    print(f'removing cell types which occur very seldom...')
    counts = adata.obs[celltype_key].value_counts()
    adata = adata[adata.obs[celltype_key].isin(list(counts.keys()[counts > 2]))]

    print(f'setting up model...')
    RegressionModel.setup_anndata(adata=adata, batch_key='batch_key', labels_key=celltype_key)
    mod = RegressionModel(adata)
    mod.view_anndata_setup()
    
    print(f'training model for {max_epochs} epochs...')
    mod.train(max_epochs=max_epochs, batch_size=2500, train_size=1, lr=0.002, use_gpu=True)
    
    print(f'view history...')
    mod.plot_history(20)
    
    print(f'extract posteriors...')
    adata = mod.export_posterior(adata, sample_kwargs={'num_samples': 1000, 'batch_size': 2500, 'use_gpu': True})
    print(f'saving model to {ref_run_name}/{subset_value}...')
    mod.save(f"{ref_run_name}/{subset_value}/", overwrite=True)
    print(f'saving reference to {ref_run_name}/{subset_value}...')
    adata.write(f"{ref_run_name}/{subset_value}/sc.h5ad")
    print(f'plotting QC...')
    mod.plot_QC()
    
    if 'means_per_cluster_mu_fg' in adata.varm.keys():
        inf_aver = adata.varm['means_per_cluster_mu_fg'][[f'means_per_cluster_mu_fg_{i}' for i in adata.uns['mod']['factor_names']]].copy()
    else:
        inf_aver = adata.var[[f'means_per_cluster_mu_fg_{i}' for i in adata.uns['mod']['factor_names']]].copy()
    inf_aver.columns = adata.uns['mod']['factor_names']
    inf_aver.to_csv(f"{ref_run_name}/{subset_value}/inf_aver.csv")
    
    del adata, mod, inf_aver, selected, counts
```

Example 12 (python):
```python
# Read scRNA data
scRNA_path='/nfs/team205/heart/anndata_objects/Foetal/Feb28ObjectRaw.h5ad'
adata = sc.read(scRNA_path)
adata
```

Example 13 (unknown):
```unknown
AnnData object with n_obs × n_vars = 297473 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score'
    var: 'gene_name_scRNA-0', 'gene_id'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'age_colors', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'cycling_colors', 'donor_colors', 'fine_grain_colors', 'heart_or_greatvessels_colors', 'hvg', 'kit_10x_colors', 'mid_grain_colors', 'modality_colors', 'neighbors', 'phase_colors', 'region_colors', 'sex_colors', 'trimester_colors', 'umap', 'week_colors'
    obsm: 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 14 (python):
```python
adata.X.data[:10]
```

Example 15 (unknown):
```unknown
array([ 1.,  1.,  5.,  5.,  8.,  4., 13.,  4.,  3.,  1.], dtype=float32)
```

Example 16 (python):
```python
adata.layers["counts"]=adata.X.copy() # counts to .X
```

Example 17 (python):
```python
adata.X.data[:10]
```

Example 18 (unknown):
```unknown
array([ 1.,  1.,  5.,  5.,  8.,  4., 13.,  4.,  3.,  1.], dtype=float32)
```

Example 19 (python):
```python
# Import custom annotation
fine_grain_modified=pd.read_csv("fine_grain_modified.csv", index_col=0)
fine_grain_modified
```

Example 20 (python):
```python
adata.obs=adata.obs.merge(fine_grain_modified,left_index=True, right_index=True)
adata.obs.head(3)
```

Example 21 (python):
```python
set(adata.obs[celltype_key])
```

Example 22 (unknown):
```unknown
{'AtrialCardiomyocytesCycling',
 'AtrialCardiomyocytesImmature',
 'AtrialCardiomyocytesLeft',
 'AtrialCardiomyocytesRight',
 'AtrioventricularNodeCardiomyocytes',
 'BCells',
 'BCellsMS4A1pos',
 'ChromaffinCells',
 'CoronaryArterialEndothelialCells',
 'CoronaryCapillaryEndothelialCells',
 'CoronaryPericytes',
 'CoronarySmoothMuscleCells',
 'CoronaryVenousEndothelialCells',
 'CoronaryVesselAdventitialFibroblasts',
 'DendriticCellsMature',
 'DendriticCellsType1',
 'DuctusArteriosusSmoothMuscleCells',
 'EndocardialCells',
 'EndocardialCushionCells',
 'EpicardiumDerivedCells',
 'GreatVesselAdventitialFibroblasts',
 'GreatVesselArterialEndothelialCells',
 'GreatVesselSmoothMuscleCells',
 'GreatVesselVenousEndothelialCells',
 'InnateLymphoidCells',
 'LymphNodeFibroblasticReticularCells',
 'LymphaticEndothelialCells',
 'MacrophagesATF3pos',
 'MacrophagesCX3CR1pos',
 'MacrophagesLYVE1pos',
 'MacrophagesTIMD4pos',
 'MastCells',
 'Megakaryocytes',
 'MesothelialEpicardialCells',
 'MonocyteDerivedCells',
 'Monocytes',
 'MonocytesMPOpos',
 'MyocardialInterstitialFibroblasts1',
 'MyocardialInterstitialFibroblasts2',
 'Myofibroblasts',
 'NaturalKillerCells',
 'NeuronPrecursors',
 'ParaympatheticNeurons',
 'PericardialCells1',
 'PericardialCells2',
 'PericardialCells3',
 'PlasmacytoidDendriticCells',
 'ProBCells',
 'SchwannCellPrecursors',
 'SchwannCells',
 'SinoatrialNodeCardiomyocytes',
 'SympatheticNeurons',
 'TCellsCD4pos',
 'TCellsCD8pos',
 'TregsCD4pos',
 'ValveEndothelialCells',
 'ValveInterstitialCells',
 'VentricularCardiomyocytesCycling',
 'VentricularCardiomyocytesLeftCompact',
 'VentricularCardiomyocytesLeftTrabeculated',
 'VentricularCardiomyocytesRightCompact',
 'VentricularCardiomyocytesRightTrabeculated',
 'VentricularConductionSystemDistal',
 'VentricularConductionSystemProximal'}
```

Example 23 (python):
```python
adata.var.head(3)
```

Example 24 (python):
```python
adata.var=adata.var.set_index("gene_id")
```

Example 25 (python):
```python
adata.var.head(3)
```

Example 26 (python):
```python
v2_probes=pd.read_csv('FFPE_v2_probe_set.csv',skiprows=[0,1,2,3,4])
v2_probes['gene_id_clean'] = v2_probes['gene_id'].apply(lambda x: x.split('_')[-1] if '_' in x else x)
display(v2_probes.head(3))
v2_probe_genes = list(v2_probes[['DEPRECATED_' not in x for x in v2_probes['gene_id']]]['gene_id'])
print(len(v2_probe_genes))
print(v2_probe_genes[:5])
```

Example 27 (unknown):
```unknown
54018
['ENSG00000000003', 'ENSG00000000003', 'ENSG00000000003', 'ENSG00000000005', 'ENSG00000000005']
```

Example 28 (python):
```python
# actually do the subsetting, this is on the shared ensemble IDs
print(f"How many ensembl IDs are shared between the scRNAseq and FFPE Visium? {len(set(adata.var_names).intersection(v2_probe_genes))}")
print(adata.shape)
adata = adata[:,list(set(adata.var_names).intersection(v2_probe_genes))]
print(adata.shape)
```

Example 29 (unknown):
```unknown
How many ensembl IDs are shared between the scRNAseq and FFPE Visium? 18536
(297473, 36601)
(297473, 18536)
```

Example 30 (python):
```python
adata.obs['batch_key'].value_counts()
```

Example 31 (unknown):
```unknown
Hst41_Nuclei_Multiome-v1    104878
Hst33_Nuclei_Multiome-v1     38234
Hst39_Nuclei_Multiome-v1     25386
C99_Cell_5prime-v2           17547
C92_Nuclei_Multiome-v1       13770
C94_Cell_5prime-v2           12985
Hst40_Nuclei_Multiome-v1      9708
C97_Cell_5prime-v2            9702
BRC2251_Cell_3prime-v3        9121
BRC2256_Cell_3prime-v3        7948
Hst36_Nuclei_Multiome-v1      7268
BRC2252_Cell_3prime-v3        6854
C86_Cell_5prime-v2            6376
BRC2260_Cell_3prime-v3        6004
BRC2263_Cell_3prime-v3        5662
BRC2262_Cell_3prime-v3        4129
C87_Nuclei_Multiome-v1        3907
C98_Nuclei_Multiome-v1        2660
C104_Nuclei_Multiome-v1       2490
C85_Nuclei_Multiome-v1        2436
C82_Nuclei_Multiome-v1         408
Name: batch_key, dtype: int64
```

Example 32 (python):
```python
set(adata.obs['batch_key'])
```

Example 33 (unknown):
```unknown
{'BRC2251_Cell_3prime-v3',
 'BRC2252_Cell_3prime-v3',
 'BRC2256_Cell_3prime-v3',
 'BRC2260_Cell_3prime-v3',
 'BRC2262_Cell_3prime-v3',
 'BRC2263_Cell_3prime-v3',
 'C104_Nuclei_Multiome-v1',
 'C82_Nuclei_Multiome-v1',
 'C85_Nuclei_Multiome-v1',
 'C86_Cell_5prime-v2',
 'C87_Nuclei_Multiome-v1',
 'C92_Nuclei_Multiome-v1',
 'C94_Cell_5prime-v2',
 'C97_Cell_5prime-v2',
 'C98_Nuclei_Multiome-v1',
 'C99_Cell_5prime-v2',
 'Hst33_Nuclei_Multiome-v1',
 'Hst36_Nuclei_Multiome-v1',
 'Hst39_Nuclei_Multiome-v1',
 'Hst40_Nuclei_Multiome-v1',
 'Hst41_Nuclei_Multiome-v1'}
```

Example 34 (unknown):
```unknown
subset=False
```

Example 35 (python):
```python
%%time
if subset:
    subset_values=adata.obs[sc_subset_key].unique().tolist()
    for value in subset_values:
        adata_ref_subset = adata[adata.obs[sc_subset_key].isin([value])].copy()
        c2l_make_reference(adata_ref_subset, value)
else:
    adata_ref = adata.copy()
    c2l_make_reference(adata_ref, "all_data")
```

Example 36 (unknown):
```unknown
##### LA #####
AtrialCardiomyocytesLeft                  19443
MP_LYVE1+                                  6955
Pericyte_1                                 5845
EC_capillary                               5764
SAN_FB                                     4311
                                          ...  
valve_FB_MV2                                  5
VentricularCardiomyocytesLeftStressed         2
valve_FB_CS                                   1
VentricularCardiomyocytesRight                1
VentricularCardiomyocytesRightStressed        1
Name: cell_state, Length: 70, dtype: int64
filtering features...
```

Example 37 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/pandas/core/arraylike.py:402: RuntimeWarning: divide by zero encountered in log10
  result = getattr(ufunc, method)(*inputs, **kwargs)
```

Example 38 (unknown):
```unknown
removing cell types which occur very seldom...
setting up model...
```

Example 39 (unknown):
```unknown
No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.)
```

Example 40 (unknown):
```unknown
Anndata setup with scvi-tools version 0.20.3.
```

Example 41 (unknown):
```unknown
Setup via `RegressionModel.setup_anndata` with arguments:
```

Example 42 (unknown):
```unknown
{
│   'layer': None,
│   'batch_key': 'batch_key',
│   'labels_key': 'cell_state',
│   'categorical_covariate_keys': None,
│   'continuous_covariate_keys': None
}
```

Example 43 (unknown):
```unknown
Summary Statistics         
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┓
┃     Summary Stat Key     ┃ Value ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━┩
│         n_batch          │  30   │
│         n_cells          │ 89744 │
│ n_extra_categorical_covs │   0   │
│ n_extra_continuous_covs  │   0   │
│         n_labels         │  66   │
│          n_vars          │ 13571 │
└──────────────────────────┴───────┘
```

Example 44 (python):
```python
Data Registry                
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Registry Key ┃    scvi-tools Location    ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│      X       │          adata.X          │
│    batch     │ adata.obs['_scvi_batch']  │
│    ind_x     │   adata.obs['_indices']   │
│    labels    │ adata.obs['_scvi_labels'] │
└──────────────┴───────────────────────────┘
```

Example 45 (python):
```python
batch State Registry                           
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location     ┃       Categories        ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['batch_key'] │ A61_Nuclei_Multiome-v1  │          0          │
│                        │ AH5_Nuclei_Multiome-v1  │          1          │
│                        │ AV1_Nuclei_Multiome-v1  │          2          │
│                        │ AV2_Nuclei_Multiome-v1  │          3          │
│                        │ AV10_Nuclei_Multiome-v1 │          4          │
│                        │ AV11_Nuclei_Multiome-v1 │          5          │
│                        │ AV13_Nuclei_Multiome-v1 │          6          │
│                        │ AV14_Nuclei_Multiome-v1 │          7          │
│                        │   D1_Nuclei_3prime-v2   │          8          │
│                        │   D2_Nuclei_3prime-v2   │          9          │
│                        │    D3_Cell_3prime-v2    │         10          │
│                        │   D3_Nuclei_3prime-v2   │         11          │
│                        │    D4_Cell_3prime-v2    │         12          │
│                        │   D4_Nuclei_3prime-v2   │         13          │
│                        │    D5_Cell_3prime-v2    │         14          │
│                        │   D5_Nuclei_3prime-v2   │         15          │
│                        │    D6_Cell_3prime-v2    │         16          │
│                        │    D6_Cell_3prime-v3    │         17          │
│                        │   D6_Nuclei_3prime-v2   │         18          │
│                        │    D7_Cell_3prime-v2    │         19          │
│                        │   D7_Nuclei_3prime-v2   │         20          │
│                        │  D8_Nuclei_Multiome-v1  │         21          │
│                        │   D11_Cell_3prime-v3    │         22          │
│                        │  D11_Nuclei_3prime-v3   │         23          │
│                        │   H2_Nuclei_3prime-v3   │         24          │
│                        │   H3_Nuclei_3prime-v3   │         25          │
│                        │   H4_Nuclei_3prime-v3   │         26          │
│                        │   H5_Nuclei_3prime-v3   │         27          │
│                        │   H6_Nuclei_3prime-v3   │         28          │
│                        │   H7_Nuclei_3prime-v3   │         29          │
└────────────────────────┴─────────────────────────┴─────────────────────┘
```

Example 46 (python):
```python
labels State Registry                                
┏━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃     Source Location     ┃            Categories             ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['cell_state'] │            Adipocytes             │          0          │
│                         │       AppendageNeuralCells        │          1          │
│                         │       Artifact mesothelial        │          2          │
│                         │          Atrial pericyte          │          3          │
│                         │     AtrialCardiomyocytesLeft      │          4          │
│                         │     AtrialCardiomyocytesRight     │          5          │
│                         │      AtrialConductionSystem       │          6          │
│                         │                 B                 │          7          │
│                         │             B_plasma              │          8          │
│                         │               CD4+T               │          9          │
│                         │            CD4+T_naive            │         10          │
│                         │             CD4+T_reg             │         11          │
│                         │               CD8+T               │         12          │
│                         │     CD55+ FGFR1+ mesothelial      │         13          │
│                         │        CRIM1+ mesothelial         │         14          │
│                         │           Coronary vSMC           │         15          │
│                         │     CoronaryArteryAdipocytes      │         16          │
│                         │                DC                 │         17          │
│                         │            EC_arterial            │         18          │
│                         │           EC_capillary            │         19          │
│                         │       EC_capillary_IGFBP5+        │         20          │
│                         │        EC_capillary_VEGFA+        │         21          │
│                         │     EC_capillary_VEGFA+NOVA1+     │         22          │
│                         │        EC_capillary_immune        │         23          │
│                         │          EC_endocardial           │         24          │
│                         │             EC_venous             │         25          │
│                         │         EClikeNeuralCells         │         26          │
│                         │         Great vessel vSMC         │         27          │
│                         │     LymphaticEndothelialCells     │         28          │
│                         │             MP_CXCL8+             │         29          │
│                         │             MP_FABP5+             │         30          │
│                         │             MP_LYVE1+             │         31          │
│                         │          MP_LYVE1+AGBL4+          │         32          │
│                         │            MP_cycling             │         33          │
│                         │             MastCells             │         34          │
│                         │        Microvascular SMC_1        │         35          │
│                         │        Microvascular SMC_2        │         36          │
│                         │               Mo/MP               │         37          │
│                         │               Mono                │         38          │
│                         │             NK_CD16hi             │         39          │
│                         │             NK_CD56hi             │         40          │
│                         │            NeuralCells            │         41          │
│                         │         NodalNeuralCells          │         42          │
│                         │          PacemakerCells           │         43          │
│                         │            Pericyte_1             │         44          │
│                         │            Pericyte_2             │         45          │
│                         │            Pericyte_3             │         46          │
│                         │        Proliferating vSMC         │         47          │
│                         │      PulmonaryVeinAdipocytes      │         48          │
│                         │              SAN_FB               │         49          │
│                         │     SOD2+ TIMP1+ mesothelial      │         50          │
│                         │   SinoatrialNodeCardiomyocytes    │         51          │
│                         │   VentricularCardiomyocytesLeft   │         52          │
│                         │ VentricularConductionSystemDistal │         53          │
│                         │         VesselNeuralCells         │         54          │
│                         │               aFB1                │         55          │
│                         │               aFB4                │         56          │
│                         │             doublets              │         57          │
│                         │               vFB1                │         58          │
│                         │          vFB2_activated           │         59          │
│                         │              vSMC_1               │         60          │
│                         │              vSMC_2               │         61          │
│                         │            valve_FB_AV            │         62          │
│                         │           valve_FB_MV1            │         63          │
│                         │           valve_FB_MV2            │         64          │
│                         │            vascular_FB            │         65          │
└─────────────────────────┴───────────────────────────────────┴─────────────────────┘
```

Example 47 (unknown):
```unknown
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
```

Example 48 (unknown):
```unknown
training model for 250 epochs...
```

Example 49 (unknown):
```unknown
You are using a CUDA device ('NVIDIA A100-SXM4-80GB') that has Tensor Cores. To properly utilize them, you should set `torch.set_float32_matmul_precision('medium' | 'high')` which will trade-off precision for performance. For more details, read https://pytorch.org/docs/stable/generated/torch.set_float32_matmul_precision.html#torch.set_float32_matmul_precision
```

Example 50 (python):
```python
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
File <timed exec>:5

Cell In[17], line 24, in c2l_make_reference(adata, subset_value, max_epochs)
     21 mod.view_anndata_setup()
     23 print(f'training model for {max_epochs} epochs...')
---> 24 mod.train(max_epochs=max_epochs, batch_size=2500, train_size=1, lr=0.002, use_gpu=True)
     26 print(f'view history...')
     27 mod.plot_history(20)

File /lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/cell2location/models/reference/_reference_model.py:157, in RegressionModel.train(self, max_epochs, batch_size, train_size, lr, **kwargs)
    154 kwargs["train_size"] = train_size
    155 kwargs["lr"] = lr
--> 157 super().train(**kwargs)

File /lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/scvi/model/base/_pyromixin.py:172, in PyroSviTrainMixin.train(self, max_epochs, use_gpu, train_size, validation_size, batch_size, early_stopping, lr, training_plan, plan_kwargs, **trainer_kwargs)
    162 trainer_kwargs["callbacks"].append(PyroJitGuideWarmup())
    164 runner = self._train_runner_cls(
    165     self,
    166     training_plan=training_plan,
   (...)
    170     **trainer_kwargs,
    171 )
--> 172 return runner()

File /lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/scvi/train/_trainrunner.py:82, in TrainRunner.__call__(self)
     79 if hasattr(self.data_splitter, "n_val"):
     80     self.training_plan.n_obs_validation = self.data_splitter.n_val
---> 82 self.trainer.fit(self.training_plan, self.data_splitter)
     83 self._update_history()
     85 # data splitter only gets these attrs after fit

File /lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/scvi/train/_trainer.py:193, in Trainer.fit(self, *args, **kwargs)
    187 if isinstance(args[0], PyroTrainingPlan):
    188     warnings.filterwarnings(
    189         action="ignore",
    190         category=UserWarning,
    191         message="`LightningModule.configure_optimizers` returned `None`",
    192     )
--> 193 super().fit(*args, **kwargs)

File /lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/pytorch_lightning/trainer/trainer.py:608, in Trainer.fit(self, model, train_dataloaders, val_dataloaders, datamodule, ckpt_path)
    606 model = self._maybe_unwrap_optimized(model)
    607 self.strategy._lightning_module = model
--> 608 call._call_and_handle_interrupt(
    609     self, self._fit_impl, model, train_dataloaders, val_dataloaders, datamodule, ckpt_path
    610 )

File /lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/pytorch_lightning/trainer/call.py:38, in _call_and_handle_interrupt(trainer, trainer_fn, *args, **kwargs)
     36         return trainer.strategy.launcher.launch(trainer_fn, *args, trainer=trainer, **kwargs)
     37     else:
---> 38         return trainer_fn(*args, **kwargs)
     40 except _TunerExitException:
     41     trainer._call_teardown_hook()

File /lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/pytorch_lightning/trainer/trainer.py:650, in Trainer._fit_impl(self, model, train_dataloaders, val_dataloaders, datamodule, ckpt_path)
    643 ckpt_path = ckpt_path or self.resume_from_checkpoint
    644 self._ckpt_path = self._checkpoint_connector._set_ckpt_path(
    645     self.state.fn,
    646     ckpt_path,  # type: ignore[arg-type]
    647     model_provided=True,
    648     model_connected=self.lightning_module is not None,
    649 )
--> 650 self._run(model, ckpt_path=self.ckpt_path)
    652 assert self.state.stopped
    653 self.training = False

File /lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/pytorch_lightning/trainer/trainer.py:1093, in Trainer._run(self, model, ckpt_path)
   1090 self._logger_connector.reset_metrics()
   1092 # strategy will configure model and move it to the device
-> 1093 self.strategy.setup(self)
   1095 # hook
   1096 if self.state.fn == TrainerFn.FITTING:

File /lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/pytorch_lightning/strategies/single_device.py:73, in SingleDeviceStrategy.setup(self, trainer)
     72 def setup(self, trainer: pl.Trainer) -> None:
---> 73     self.model_to_device()
     74     super().setup(trainer)

File /lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/pytorch_lightning/strategies/single_device.py:70, in SingleDeviceStrategy.model_to_device(self)
     68 def model_to_device(self) -> None:
     69     assert self.model is not None, "self.model must be set before self.model.to()"
---> 70     self.model.to(self.root_device)

File /lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/lightning_fabric/utilities/device_dtype_mixin.py:54, in _DeviceDtypeModuleMixin.to(self, *args, **kwargs)
     52 device, dtype = torch._C._nn._parse_to(*args, **kwargs)[:2]
     53 self.__update_properties(device=device, dtype=dtype)
---> 54 return super().to(*args, **kwargs)

File /lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/torch/nn/modules/module.py:1145, in Module.to(self, *args, **kwargs)
   1141         return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None,
   1142                     non_blocking, memory_format=convert_to_format)
   1143     return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
-> 1145 return self._apply(convert)

File /lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/torch/nn/modules/module.py:797, in Module._apply(self, fn)
    795 def _apply(self, fn):
    796     for module in self.children():
--> 797         module._apply(fn)
    799     def compute_should_use_set_data(tensor, tensor_applied):
    800         if torch._has_compatible_shallow_copy_type(tensor, tensor_applied):
    801             # If the new tensor has compatible tensor type as the existing tensor,
    802             # the current behavior is to change the tensor in-place using `.data =`,
   (...)
    807             # global flag to let the user control whether they want the future
    808             # behavior of overwriting the existing tensor or not.

File /lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/torch/nn/modules/module.py:797, in Module._apply(self, fn)
    795 def _apply(self, fn):
    796     for module in self.children():
--> 797         module._apply(fn)
    799     def compute_should_use_set_data(tensor, tensor_applied):
    800         if torch._has_compatible_shallow_copy_type(tensor, tensor_applied):
    801             # If the new tensor has compatible tensor type as the existing tensor,
    802             # the current behavior is to change the tensor in-place using `.data =`,
   (...)
    807             # global flag to let the user control whether they want the future
    808             # behavior of overwriting the existing tensor or not.

File /lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/torch/nn/modules/module.py:844, in Module._apply(self, fn)
    842 for key, buf in self._buffers.items():
    843     if buf is not None:
--> 844         self._buffers[key] = fn(buf)
    846 return self

File /lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/torch/nn/modules/module.py:1143, in Module.to.<locals>.convert(t)
   1140 if convert_to_format is not None and t.dim() in (4, 5):
   1141     return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None,
   1142                 non_blocking, memory_format=convert_to_format)
-> 1143 return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)

RuntimeError: CUDA error: out of memory
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.
```

---

## Variables¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/VisiumHD/no5_celltypist_prediction.html

**Contents:**
- Variables¶
- Read in data¶
- Normalise the counts¶
- UMAP¶
- CellTypist prediction: coarse-grain¶
- CellTypist prediction: mid-grain¶
- celltypist prediction: fine-grain¶
- Save¶

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os

import celltypist
```

Example 2 (python):
```python
# import session_info
# session_info.show()
```

Example 3 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 4 (python):
```python
adata_dir = '/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects'
sample_id = 'HEA_FOET14880396'
```

Example 5 (unknown):
```unknown
os.getcwd()
```

Example 6 (unknown):
```unknown
'/nfs/team205/kk18/notebooks/Foetal/VisiumHD/re-seq_mpp0p3'
```

Example 7 (python):
```python
adata = sc.read_h5ad(f'{adata_dir}/{sample_id}_b2c_cells_filtered.h5ad')
adata
```

Example 8 (unknown):
```unknown
AnnData object with n_obs × n_vars = 129803 × 18047
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts'
    uns: 'spatial'
    obsm: 'spatial', 'spatial_cropped_150_buffer'
```

Example 9 (python):
```python
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)
```

Example 10 (python):
```python
adata.X.data[:5]
```

Example 11 (unknown):
```unknown
array([1.27187559, 1.95641568, 1.75023687, 1.96760567, 2.7810601 ])
```

Example 12 (python):
```python
# sc.pp.regress_out(adata, ["total_counts", "pct_counts_mt"])
```

Example 13 (python):
```python
# HVGs
sc.pp.highly_variable_genes(adata, flavor='seurat', min_mean=0.0125, max_mean=3, min_disp=0.5)
sc.pl.highly_variable_genes(adata)
print(f'number of hvgs: {sum(adata.var.highly_variable)}')
# PCA
sc.tl.pca(adata, use_highly_variable=True)
# create neighbour graph
sc.pp.neighbors(adata,n_neighbors=15,use_rep='X_pca')
# visualisation with umap
sc.tl.umap(adata,min_dist=0.2)
sc.pl.embedding(adata,basis='X_umap',color=['bin_count','total_counts','n_genes_by_counts',"pct_counts_mt"],wspace=0.2,cmap='RdPu',vmax='p100')
```

Example 14 (unknown):
```unknown
number of hvgs: 3066
```

Example 15 (python):
```python
# coarse-grained
predictions = celltypist.annotate(adata, 
                                  model = '/nfs/team205/kk18/notebooks/Foetal/VisiumHD/mpp0p3/celltypist_models/from_foetal_portal/foetal_heart_coarse.pkl', 
                                  majority_voting = False)

# add to anndata
adata = predictions.to_adata(insert_labels = True, insert_conf = True)
adata.obs.rename(columns={'predicted_labels':'celltypist_coarse',
                              'conf_score':'conf_score_coarse'
                             },inplace=True)
adata.obs.head()
```

Example 16 (unknown):
```unknown
🔬 Input data has 129803 cells and 18047 genes
🔗 Matching reference genes in the model
🧬 17059 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
```

Example 17 (python):
```python
sc.pl.embedding(adata,
                    basis=f"X_umap",
                    color=['celltypist_coarse','conf_score_coarse'],
                   ncols=4,wspace=0.5
                   )
```

Example 18 (python):
```python
# celltypist confidence
plt.figure(figsize=(6, 2))
sns.histplot(adata.obs[f'conf_score_coarse'], kde=False, bins=100)
# plt.yscale('log')
plt.title('coarse_grained')
plt.show()
```

Example 19 (python):
```python
# check
conf_thresh = 0.5 # starting from relaxed threshold
pd.crosstab(adata.obs['celltypist_coarse'],adata.obs[f'conf_score_coarse']<conf_thresh)
```

Example 20 (python):
```python
# finalised coarse-grained labels
conf_thresh = 0.5
adata.obs['celltypist_coarse_fin'] = adata.obs['celltypist_coarse'].astype('str').copy()
low_quality_cells = adata.obs_names[adata.obs[f'conf_score_coarse']<conf_thresh]
adata.obs.loc[low_quality_cells,'celltypist_coarse_fin'] = 'unassigned'
adata.obs['celltypist_coarse_fin'].value_counts()
```

Example 21 (unknown):
```unknown
celltypist_coarse_fin
Cardiomyocytes    76088
Mesenchymal       36370
unassigned        10482
Endothelium        4380
Neural             1254
Leukocytes          626
Epicardium          603
Name: count, dtype: int64
```

Example 22 (python):
```python
sc.pl.embedding(adata,
                    basis=f"X_umap",
                    color=['celltypist_coarse_fin'],
                   ncols=4,wspace=0.5
                   )
```

Example 23 (unknown):
```unknown
... storing 'celltypist_coarse_fin' as categorical
```

Example 24 (python):
```python
sc.pl.embedding(adata,
                    basis=f"X_umap",
                    color=['celltypist_coarse_fin'],groups='unassigned',
                   ncols=4,wspace=0.5
                   )
```

Example 25 (python):
```python
adata.obs['celltypist_coarse2midmod'] = np.nan
adata.obs['conf_score_coarse2midmod'] = np.nan

for celltype in adata.obs['celltypist_coarse_fin'].unique():
    if celltype not in ['unassigned', 'Epicardium']: # 'Epicardium' is also 'Epicardium' in mid-grained
        print(celltype)
        # subset
        adata_sub = adata[adata.obs['celltypist_coarse_fin']==celltype]
        # predict
        predictions = celltypist.annotate(adata_sub, 
                                      model = f'/nfs/team205/kk18/notebooks/Foetal/VisiumHD/mpp0p3/celltypist_models/coarse2midmod_{celltype}.pkl', 
                                      majority_voting = False)
        # add to anndata
        adata_sub = predictions.to_adata(insert_labels = True, insert_conf = True)
        adata.obs.loc[adata_sub.obs_names,'celltypist_coarse2midmod'] = adata_sub.obs['predicted_labels']
        adata.obs.loc[adata_sub.obs_names,'conf_score_coarse2midmod'] = adata_sub.obs['conf_score']
        print('')
```

Example 26 (unknown):
```unknown
🔬 Input data has 626 cells and 18047 genes
🔗 Matching reference genes in the model
🧬 16956 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
```

Example 27 (unknown):
```unknown
Mesenchymal
```

Example 28 (unknown):
```unknown
🔬 Input data has 36370 cells and 18047 genes
🔗 Matching reference genes in the model
🧬 17166 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
```

Example 29 (unknown):
```unknown
Endothelium
```

Example 30 (unknown):
```unknown
🔬 Input data has 4380 cells and 18047 genes
🔗 Matching reference genes in the model
🧬 16989 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
```

Example 31 (unknown):
```unknown
🔬 Input data has 1254 cells and 18047 genes
🔗 Matching reference genes in the model
🧬 16779 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
```

Example 32 (unknown):
```unknown
Cardiomyocytes
```

Example 33 (unknown):
```unknown
🔬 Input data has 76088 cells and 18047 genes
🔗 Matching reference genes in the model
🧬 16534 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
```

Example 34 (python):
```python
sc.pl.embedding(adata,
                    basis=f"X_umap",
                    color=['celltypist_coarse2midmod','conf_score_coarse2midmod'],
                   ncols=4,wspace=0.5
                   )
```

Example 35 (unknown):
```unknown
... storing 'celltypist_coarse2midmod' as categorical
```

Example 36 (python):
```python
# check
conf_thresh = 0.8
pd.crosstab(adata.obs['celltypist_coarse2midmod'],adata.obs[f'conf_score_coarse2midmod']<conf_thresh)
```

Example 37 (python):
```python
# finalised mid-grained labels
conf_thresh = 0.8
adata.obs['celltypist_mid_fin'] = adata.obs['celltypist_coarse_fin'].astype('str').copy()
adata.obs.replace({'celltypist_mid_fin':{'Epicardium':'EpicardialCells'}},inplace=True)
passed = adata.obs_names[adata.obs['conf_score_coarse2midmod']>=conf_thresh]
low_quality_cells = adata.obs_names[adata.obs[f'conf_score_coarse2midmod']<conf_thresh]

adata.obs.loc[passed,'celltypist_mid_fin'] = adata.obs.loc[passed,'celltypist_coarse2midmod'].copy()
adata.obs.loc[low_quality_cells,'celltypist_mid_fin'] = 'unassigned'
adata.obs['celltypist_mid_fin'].value_counts()
```

Example 38 (unknown):
```unknown
celltypist_mid_fin
VentricularCardiomyocytes      61075
Fibroblasts                    20318
unassigned                     13951
AtrialCardiomyocytes           13904
MuralCells                     10785
PericardialCells                3061
BloodVesselEndothelialCells     1765
EndocardialCells                1526
LymphaticEndothelialCells       1025
EpicardialCells                  603
Neurons                          589
Glia                             588
MyeloidCells                     526
LymphoidCells                     87
Name: count, dtype: int64
```

Example 39 (python):
```python
sc.pl.embedding(adata,
                    basis=f"X_umap",
                    color=['celltypist_mid_fin'],
                   ncols=4,wspace=0.5
                   )
```

Example 40 (unknown):
```unknown
... storing 'celltypist_mid_fin' as categorical
```

Example 41 (python):
```python
adata.obs['celltypist_midmod2fine'] = np.nan
adata.obs['conf_score_midmod2fine'] = np.nan

for celltype in adata.obs['celltypist_mid_fin'].unique():
    if celltype not in ['unassigned', 'LymphaticEndothelialCells']: # 'LymphaticEndothelialCells' is also 'LymphaticEndothelialCells' in fine-grained
        print(celltype)
        # subset
        adata_sub = adata[adata.obs['celltypist_mid_fin']==celltype]
        # predict
        predictions = celltypist.annotate(adata_sub, 
                                      model = f'/nfs/team205/kk18/notebooks/Foetal/VisiumHD/mpp0p3/celltypist_models/midmod2fine_{celltype}.pkl', 
                                      majority_voting = False)
        # add to anndata
        adata_sub = predictions.to_adata(insert_labels = True, insert_conf = True)
        adata.obs.loc[adata_sub.obs_names,'celltypist_midmod2fine'] = adata_sub.obs['predicted_labels']
        adata.obs.loc[adata_sub.obs_names,'conf_score_midmod2fine'] = adata_sub.obs['conf_score']
        print('')
```

Example 42 (unknown):
```unknown
LymphoidCells
```

Example 43 (unknown):
```unknown
🔬 Input data has 87 cells and 18047 genes
🔗 Matching reference genes in the model
🧬 17008 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
```

Example 44 (unknown):
```unknown
MyeloidCells
```

Example 45 (unknown):
```unknown
🔬 Input data has 526 cells and 18047 genes
🔗 Matching reference genes in the model
🧬 17087 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
```

Example 46 (unknown):
```unknown
Fibroblasts
```

Example 47 (unknown):
```unknown
🔬 Input data has 20318 cells and 18047 genes
🔗 Matching reference genes in the model
🧬 17022 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
```

Example 48 (unknown):
```unknown
PericardialCells
```

Example 49 (unknown):
```unknown
🔬 Input data has 3061 cells and 18047 genes
🔗 Matching reference genes in the model
🧬 16563 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
```

Example 50 (unknown):
```unknown
BloodVesselEndothelialCells
```

Example 51 (unknown):
```unknown
🔬 Input data has 1765 cells and 18047 genes
🔗 Matching reference genes in the model
🧬 16931 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
```

Example 52 (unknown):
```unknown
🔬 Input data has 10785 cells and 18047 genes
🔗 Matching reference genes in the model
🧬 16683 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
```

Example 53 (unknown):
```unknown
🔬 Input data has 589 cells and 18047 genes
🔗 Matching reference genes in the model
🧬 16254 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
```

Example 54 (unknown):
```unknown
🔬 Input data has 588 cells and 18047 genes
🔗 Matching reference genes in the model
🧬 16388 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
```

Example 55 (unknown):
```unknown
EpicardialCells
```

Example 56 (unknown):
```unknown
🔬 Input data has 603 cells and 18047 genes
🔗 Matching reference genes in the model
🧬 16982 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
```

Example 57 (unknown):
```unknown
AtrialCardiomyocytes
```

Example 58 (unknown):
```unknown
🔬 Input data has 13904 cells and 18047 genes
🔗 Matching reference genes in the model
🧬 16481 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
```

Example 59 (unknown):
```unknown
EndocardialCells
```

Example 60 (unknown):
```unknown
🔬 Input data has 1526 cells and 18047 genes
🔗 Matching reference genes in the model
🧬 16303 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
```

Example 61 (unknown):
```unknown
VentricularCardiomyocytes
```

Example 62 (unknown):
```unknown
🔬 Input data has 61075 cells and 18047 genes
🔗 Matching reference genes in the model
🧬 16613 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
```

Example 63 (python):
```python
sc.pl.embedding(adata,
                    basis=f"X_umap",
                    color=['celltypist_midmod2fine','conf_score_midmod2fine'],
                   ncols=1,wspace=0.5
                   )
```

Example 64 (unknown):
```unknown
... storing 'celltypist_midmod2fine' as categorical
```

Example 65 (python):
```python
adata.write(f'{adata_dir}/{sample_id}_b2c_cells_filtered_lognorm.h5ad')
adata
```

Example 66 (unknown):
```unknown
AnnData object with n_obs × n_vars = 129803 × 18047
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'spatial', 'log1p', 'hvg', 'pca', 'neighbors', 'umap', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_coarse2midmod_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors'
    obsm: 'spatial', 'spatial_cropped_150_buffer', 'X_pca', 'X_umap'
    varm: 'PCs'
    obsp: 'distances', 'connectivities'
```

Example 67 (python):
```python
print(f'{adata_dir}/{sample_id}_b2c_cells_filtered_lognorm.h5ad')
```

Example 68 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/HEA_FOET14880396_b2c_cells_filtered_lognorm.h5ad
```

---

## Target: SANPC signatures¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig3_SAN_niche/snMultime_no4_prepare_supp_table.html

**Contents:**
- Target: SANPC signatures¶
- Target: Ion channels¶
- Target: Axon guidances¶

**Examples:**

Example 1 (python):
```python
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import os
```

Example 2 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 3 (unknown):
```unknown
table_dir = '/nfs/team205/heart/fetal_ms_figures/tables'
```

Example 4 (python):
```python
scenic_meta_path = '/nfs/team205/heart/fetal_ms_figures/tables/csvs/scenicplus_eRegulon_metadata_CMs_SANPC-highlighted.csv'
```

Example 5 (python):
```python
TF_TG_network_path = '/nfs/team205/heart/Vitro/TF_selection/foetal_SANPcell_vs_workingCM/TF-TG-network_target-SANPC-signatures.pkl'
TF_TF_network_path = '/nfs/team205/heart/Vitro/TF_selection/foetal_SANPcell_vs_workingCM/TF-TF-network_target-SANPC-signatures.pkl'
TF_info_path = '/nfs/team205/heart/Vitro/TF_selection/foetal_SANPcell_vs_workingCM/TF_selection_df_target-SANPC-signatures.csv'

# TF_TG_network
network = utils.read_pkl(TF_TG_network_path)
print(len(network.keys()))

# TF info
TF_info = pd.read_csv(TF_info_path)
print(TF_info.shape)
display(TF_info.head())

# grn table
grn = pd.DataFrame({'TF':network.keys(),'Targets':network.values()})
grn = grn.merge(TF_info,how='left',on='TF')
grn = grn[['TF','Targets','centaliry_eigenvector']]
print(grn.shape)
display(grn.head())

# save the daraframe
grn.to_csv(os.path.join(table_dir,'csvs/TF_TG_network_SANPC_target-SANPC-signatures.csv'))
```

Example 6 (unknown):
```unknown
124
(124, 6)
```

Example 7 (python):
```python
TF_TG_network_path = '/nfs/team205/heart/Vitro/TF_selection/foetal_SANPcell_vs_workingCM/TF-TG-network_target-IonChannels.pkl'
TF_TF_network_path = '/nfs/team205/heart/Vitro/TF_selection/foetal_SANPcell_vs_workingCM/TF-TF-network_target-IonChannels.pkl'
TF_info_path = '/nfs/team205/heart/Vitro/TF_selection/foetal_SANPcell_vs_workingCM/TF_selection_df_target-IonChannels.csv'

# TF_TG_network
network = utils.read_pkl(TF_TG_network_path)
print(len(network.keys()))

# TF info
TF_info = pd.read_csv(TF_info_path)
print(TF_info.shape)
display(TF_info.head())

# grn table
grn = pd.DataFrame({'TF':network.keys(),'Targets':network.values()})
grn = grn.merge(TF_info,how='left',on='TF')
grn = grn[['TF','Targets','centaliry_eigenvector']]
print(grn.shape)
display(grn.head())

# save the daraframe
grn.to_csv(os.path.join(table_dir,'csvs/TF_TG_network_SANPC_target-IonChannels.csv'))
```

Example 8 (python):
```python
TF_TG_network_path = '/nfs/team205/heart/Vitro/TF_selection/foetal_SANPcell_vs_workingCM/TF-TG-network_target-AxonGuidances.pkl'
TF_TF_network_path = '/nfs/team205/heart/Vitro/TF_selection/foetal_SANPcell_vs_workingCM/TF-TF-network_target-AxonGuidances.pkl'
TF_info_path = '/nfs/team205/heart/Vitro/TF_selection/foetal_SANPcell_vs_workingCM/TF_selection_df_target-AxonGuidances.csv'

# TF_TG_network
network = utils.read_pkl(TF_TG_network_path)
print(len(network.keys()))

# TF info
TF_info = pd.read_csv(TF_info_path)
print(TF_info.shape)
display(TF_info.head())

# grn table
grn = pd.DataFrame({'TF':network.keys(),'Targets':network.values()})
grn = grn.merge(TF_info,how='left',on='TF')
grn = grn[['TF','Targets','centaliry_eigenvector']]
print(grn.shape)
display(grn.head())

# save the daraframe
grn.to_csv(os.path.join(table_dir,'csvs/TF_TG_network_SANPC_target-AxonGuidances.csv'))
```

---

## Functions¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/Visium/cell2location/wholeheart_with_DA/no2_mapping_OCT_DA.html

**Contents:**
  - Loading packages¶
- Functions¶
- Read in metadata¶
- Data, parameters¶
- Loading visium data¶
- Add metadata and annotation¶
- Remove spots based on manual annotation¶
- Filter out low quality spots¶
- Remove mitochondrial genes¶
- Run cell2location mapping¶

**Examples:**

Example 1 (python):
```python
import sys
import scanpy as sc
import anndata
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt 
import matplotlib as mpl

import cell2location
import scvi

from matplotlib import rcParams
rcParams['pdf.fonttype'] = 42 # enables correct plotting of text
import seaborn as sns
```

Example 2 (unknown):
```unknown
Global seed set to 0
```

Example 3 (python):
```python
# import gc
# import torch
```

Example 4 (python):
```python
def read_and_qc(sample_name, path):
    r""" This function reads the data for one 10X spatial experiment into the anndata object.
    It also calculates QC metrics. Modify this function if required by your workflow.

    :param sample_name: Name of the sample
    :param path: path to data
    """

    adata = sc.read_visium(path + str(sample_name),
                           count_file='filtered_feature_bc_matrix.h5', load_images=True)
    adata.var_names_make_unique(join='-')
    
    adata.obs['sample'] = sample_name
    
    # rename genes to ENSEMBL
    adata.var['SYMBOL'] = adata.var_names
    adata.var_names = adata.var['gene_ids']
    adata.var_names.name = None

    # Calculate QC metrics
    sc.pp.calculate_qc_metrics(adata, inplace=True)
    adata.var['mt'] = [gene.startswith('MT-') for gene in adata.var['SYMBOL']]
    adata.obs['mt_frac'] = adata[:, adata.var['mt'].tolist()].X.sum(1).A.squeeze()/adata.obs['total_counts']

    # add sample name to obs names
    adata.obs["sample"] = [str(i) for i in adata.obs['sample']]
    adata.obs_names = adata.obs["sample"] \
                          + '_' + adata.obs_names
    adata.obs.index.name = 'spot_id'

    return adata

def select_slide(adata, s, s_col='sample'):
    r""" This function selects the data for one slide from the spatial anndata object.

    :param adata: Anndata object with multiple spatial experiments
    :param s: name of selected experiment
    :param s_col: column in adata.obs listing experiment name for each location
    """

    slide = adata[adata.obs[s_col].isin([s]), :]
    s_keys = list(slide.uns['spatial'].keys())
    s_spatial = np.array(s_keys)[[s in k for k in s_keys]][0]

    slide.uns['spatial'] = {s_spatial: slide.uns['spatial'][s_spatial]}

    return slide
```

Example 5 (python):
```python
meta=pd.read_csv('/nfs/team205/heart/anndata_objects/Foetal/metadata/HeartTeamSamples_Mappeddata_20230208.csv',
                index_col=0)
meta=meta[['Publication','sangerID', 'combinedID',
       'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender',
       'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'status']]

# select slides for foetal publication
meta=meta[meta['Publication'].isin(['Foetal'])]
# select only OCT slides
meta=meta[meta['modality'].isin(['Visium'])]
# select only Healthy slides
meta=meta[meta['status'].isin(['Healthy'])]
```

Example 6 (unknown):
```unknown
# select slides which contain DuctusArterious
sampleIDs = ['HCAHeartST10298118','HCAHeartST10298119','HCAHeartST10298121','HCAHeartST10317187','HCAHeartST10317188']
meta = meta[meta['sangerID'].isin(sampleIDs)]
meta
```

Example 7 (unknown):
```unknown
results_folder = '/nfs/team205/heart/cell2location/foetal'

# create paths and names to results folders for reference regression and cell2location models
ref_run_name = f'{results_folder}/reference_signatures/ver28Feb2023/whole_woPeriCell'
run_name = f'{results_folder}/cell2location_map/ver28Feb2023'
```

Example 8 (unknown):
```unknown
visium_path='/nfs/team205/heart/mapped/spaceranger110/'

celltype_key='fine_grain'
```

Example 9 (python):
```python
# Read the data of the interested region into anndata objects
slides = []
for i in sampleIDs:
    slides.append(read_and_qc(i, path=visium_path))

# Combine anndata objects together
adata_vis = slides[0].concatenate(
    slides[1:],
    batch_key="sample",
    uns_merge="unique",
    batch_categories=sampleIDs,
    index_unique=None
)
```

Example 10 (unknown):
```unknown
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
```

Example 11 (python):
```python
print(adata_vis.shape)
adata_vis.obs['sample'].value_counts()
```

Example 12 (unknown):
```unknown
(5552, 33538)
```

Example 13 (unknown):
```unknown
HCAHeartST10298121    1180
HCAHeartST10298119    1176
HCAHeartST10298118    1106
HCAHeartST10317188    1083
HCAHeartST10317187    1007
Name: sample, dtype: int64
```

Example 14 (python):
```python
# PLOT QC FOR EACH SAMPLE
fig, axs = plt.subplots(len(slides), 4, figsize=(15, 4*len(slides)-4))
for i, s in enumerate(adata_vis.obs['sample'].unique()):
    #fig.suptitle('Covariates for filtering')

    slide = select_slide(adata_vis, s)
    sns.distplot(slide.obs['total_counts'],
                 kde=False, ax = axs[i, 0])
    axs[i, 0].set_xlim(0, adata_vis.obs['total_counts'].max())
    axs[i, 0].set_xlabel(f'total_counts | {s}')

    sns.distplot(slide.obs['total_counts']\
                 [slide.obs['total_counts']<2000],
                 kde=False, bins=40, ax = axs[i, 1])
    axs[i, 1].set_xlim(0, 2000)
    axs[i, 1].set_xlabel(f'total_counts | {s}')

    sns.distplot(slide.obs['n_genes_by_counts'],
                 kde=False, bins=60, ax = axs[i, 2])
    axs[i, 2].set_xlim(0, adata_vis.obs['n_genes_by_counts'].max())
    axs[i, 2].set_xlabel(f'n_genes_by_counts | {s}')

    sns.distplot(slide.obs['n_genes_by_counts']\
                 [slide.obs['n_genes_by_counts']<1000],
                 kde=False, bins=60, ax = axs[i, 3])
    axs[i, 3].set_xlim(0, 1000)
    axs[i, 3].set_xlabel(f'n_genes_by_counts | {s}')

plt.tight_layout()
```

Example 15 (python):
```python
Trying to set attribute `._uns` of view, copying.
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/seaborn/distributions.py:2619: FutureWarning: `distplot` is a deprecated function and will be removed in a future version. Please adapt your code to use either `displot` (a figure-level function with similar flexibility) or `histplot` (an axes-level function for histograms).
Trying to set attribute `._uns` of view, copying.
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/seaborn/distributions.py:2657: RuntimeWarning: Mean of empty slice.
Trying to set attribute `._uns` of view, copying.
Trying to set attribute `._uns` of view, copying.
Trying to set attribute `._uns` of view, copying.
```

Example 16 (python):
```python
# metadata
adata_vis.obs = adata_vis.obs.reset_index().merge(meta, how='left', left_on='sample', right_on='sangerID').set_index('spot_id')
adata_vis
```

Example 17 (unknown):
```unknown
AnnData object with n_obs × n_vars = 5552 × 33538
    obs: 'in_tissue', 'array_row', 'array_col', 'sample', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'mt_frac', 'Publication', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'status'
    var: 'gene_ids', 'feature_types', 'genome', 'SYMBOL', 'mt', 'n_cells_by_counts-HCAHeartST10298118', 'mean_counts-HCAHeartST10298118', 'log1p_mean_counts-HCAHeartST10298118', 'pct_dropout_by_counts-HCAHeartST10298118', 'total_counts-HCAHeartST10298118', 'log1p_total_counts-HCAHeartST10298118', 'n_cells_by_counts-HCAHeartST10298119', 'mean_counts-HCAHeartST10298119', 'log1p_mean_counts-HCAHeartST10298119', 'pct_dropout_by_counts-HCAHeartST10298119', 'total_counts-HCAHeartST10298119', 'log1p_total_counts-HCAHeartST10298119', 'n_cells_by_counts-HCAHeartST10298121', 'mean_counts-HCAHeartST10298121', 'log1p_mean_counts-HCAHeartST10298121', 'pct_dropout_by_counts-HCAHeartST10298121', 'total_counts-HCAHeartST10298121', 'log1p_total_counts-HCAHeartST10298121', 'n_cells_by_counts-HCAHeartST10317187', 'mean_counts-HCAHeartST10317187', 'log1p_mean_counts-HCAHeartST10317187', 'pct_dropout_by_counts-HCAHeartST10317187', 'total_counts-HCAHeartST10317187', 'log1p_total_counts-HCAHeartST10317187', 'n_cells_by_counts-HCAHeartST10317188', 'mean_counts-HCAHeartST10317188', 'log1p_mean_counts-HCAHeartST10317188', 'pct_dropout_by_counts-HCAHeartST10317188', 'total_counts-HCAHeartST10317188', 'log1p_total_counts-HCAHeartST10317188'
    uns: 'spatial'
    obsm: 'spatial'
```

Example 18 (python):
```python
adata_vis.obs.head()
```

Example 19 (python):
```python
# annotation
path_to_annot='/nfs/team205/heart/visium_annotations/foetal'
category_name='JC_edit'

# read in annotation csv file and concatenate
for i, ID in enumerate(sampleIDs):
    if f'{ID}_JC.csv' in os.listdir(path_to_annot):
    
        print(ID)
        annot_df = pd.read_csv(f'{path_to_annot}/{ID}_JC.csv')
        annot_df['spot_id']=ID+'_'+annot_df['Barcode']

        if i==0:
            annot_df_all=annot_df.copy()
        else:
            annot_df_all=pd.concat([annot_df_all,annot_df])
    else:
        print(f'{ID} is not annotated')

# add annotations to adata_vis
adata_vis.obs=pd.concat([adata_vis.obs,
                     annot_df_all.set_index('spot_id')[[category_name]].reindex(adata_vis.obs.index)],
                     axis=1)
# replace nan to 'none'
adata_vis.obs[category_name].fillna('none', inplace=True)

# minor modification
annotation_edit_dict = {
     'AV Ring':'AV ring',
    'Aorta':'Great vessels',
    'Coronary Artery':'Coronary artery',
    'Mitral Valve':'AV valves',
    'Mitral valve':'AV valves',
    'Myocardium - Ventricular - Trabecular':'Myocardium - Ventricular - Trabeculated',
    'Myocardim - Ventricular - Trabeculated':'Myocardium - Ventricular - Trabeculated',
    'Myocardim - Ventricular - Compact':'Myocardium - Ventricular - Compact',
    'unknown':'Unannotated',
    'Unknown':'Unannotated',
}
adata_vis.obs.replace({category_name:annotation_edit_dict}, inplace=True)
set(adata_vis.obs[category_name])
```

Example 20 (unknown):
```unknown
HCAHeartST10298118
HCAHeartST10298119
HCAHeartST10298121
HCAHeartST10317187
HCAHeartST10317188
```

Example 21 (unknown):
```unknown
{'Cavity',
 'Epicardium',
 'Great vessels',
 'Myocardium - Atrial',
 'Myocardium - Ventricular - Compact',
 'Myocardium - Ventricular - Trabeculated',
 'none'}
```

Example 22 (python):
```python
# remove 'none' and 'no_tissue'
adata_vis = adata_vis[adata_vis.obs[category_name].isin(['none','Out_of_tissue'])==False]

# remove 'Cavity'
adata_vis = adata_vis[adata_vis.obs[category_name].isin(['Cavity'])==False]

# remove 'Unannotated'
adata_vis = adata_vis[adata_vis.obs[category_name].isin(['Unannotated'])==False]
```

Example 23 (python):
```python
# order annotations
annotations_order = ['Myocardium - Atrial',
                     'Myocardium - Ventricular - Compact',
                     'Myocardium - Ventricular - Trabeculated',
                    # 'Myocardium - Papillary Muscle',
                    'Epicardium',
                     # 'AV valves',
                     # 'Aortic valve',
                    # 'Valve apparatus',
                   # 'AV ring',
                    # 'Connective tissue',
                     'Great vessels',
                     # 'Coronary artery',
                     # 'Nerve',
                     ]
adata_vis.obs[category_name] = adata_vis.obs[category_name].astype('category')
adata_vis.obs[category_name].cat.reorder_categories(annotations_order,inplace=True)
```

Example 24 (unknown):
```unknown
Trying to set attribute `.obs` of view, copying.
/tmp/ipykernel_26774/3723571678.py:17: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
```

Example 25 (python):
```python
pd.crosstab(adata_vis.obs['age'],adata_vis.obs[category_name])
```

Example 26 (python):
```python
# filter out low quality spots
sc.pp.filter_cells(adata_vis, min_counts = 500)
sc.pp.filter_cells(adata_vis, min_genes = 300)

# Spot number per slide
adata_vis.obs['sample'].value_counts()
```

Example 27 (unknown):
```unknown
HCAHeartST10298121    1151
HCAHeartST10298119    1139
HCAHeartST10298118    1082
HCAHeartST10317188    1070
HCAHeartST10317187     991
Name: sample, dtype: int64
```

Example 28 (python):
```python
# find mitochondria-encoded (MT) genes
adata_vis.var['MT_gene'] = [gene.startswith('MT-') for gene in adata_vis.var['SYMBOL']]
print(sum(adata_vis.var['MT_gene']))
    
# remove MT genes for spatial mapping (keeping their counts in the object)
adata_vis.obsm['MT'] = adata_vis[:, adata_vis.var['MT_gene'].values].X.toarray()
adata_vis = adata_vis[:, ~adata_vis.var['MT_gene'].values]
print(sum(adata_vis.var['MT_gene']))
```

Example 29 (python):
```python
# convert adata_vis.var_names to symbol
adata_vis.var.set_index('SYMBOL',inplace=True)
adata_vis.var.head()
```

Example 30 (python):
```python
%%time

# read in reference signatures
inf_aver = pd.read_csv(f"{ref_run_name}/inf_aver.csv", index_col=0)

# find shared genes and subset both anndata and reference signatures
intersect = np.intersect1d(adata_vis.var_names, inf_aver.index)
adata_vis = adata_vis[:, intersect].copy()
inf_aver = inf_aver.loc[intersect, :].copy()
print(f'shape of inf_aver: {inf_aver.shape}')
print('reference cell types')
print(inf_aver.columns)

# subset adata
adata_vis=adata_vis.copy()
# prepare anndata for cell2location model
scvi.data.setup_anndata(adata=adata_vis, batch_key="sample")
scvi.data.view_anndata_setup(adata_vis)
```

Example 31 (python):
```python
shape of inf_aver: (16306, 58)
reference cell types
Index(['AtrialCardiomyocytesLeft', 'AtrialCardiomyocytesRight',
       'VentricularCardiomyocytesLeftCompact',
       'VentricularCardiomyocytesRightCompact',
       'VentricularCardiomyocytesLeftTrabeculated',
       'VentricularCardiomyocytesRightTrabeculated',
       'SinoatrialNodeCardiomyocytes', 'AtrioventricularNodeCardiomyocytes',
       'VentricularConductionSystemProximal',
       'VentricularConductionSystemDistal',
       'GreatVesselAdventitialFibroblasts',
       'CoronaryVesselAdventitialFibroblasts',
       'MyocardialInterstitialFibroblasts1',
       'MyocardialInterstitialFibroblasts2', 'Myofibroblasts',
       'LymphNodeFibroblasticReticularCells', 'ValveInterstitialCells',
       'GreatVesselSmoothMuscleCells', 'CoronarySmoothMuscleCells',
       'DuctusArteriosusSmoothMuscleCells', 'CoronaryPericytes',
       'GreatVesselArterialEndothelialCells',
       'GreatVesselVenousEndothelialCells', 'CoronaryArterialEndothelialCells',
       'CoronaryVenousEndothelialCells', 'CoronaryCapillaryEndothelialCells',
       'EndocardialCells', 'EndocardialCushionCells', 'ValveEndothelialCells',
       'LymphaticEndothelialCells', 'MesothelialEpicardialCells',
       'EpicardiumDerivedCells', 'NeuronPrecursors', 'ChromaffinCells',
       'SympatheticNeurons', 'ParaympatheticNeurons', 'SchwannCellPrecursors',
       'SchwannCells', 'MonocytesMPOpos', 'Monocytes', 'MonocyteDerivedCells',
       'MacrophagesCX3CR1pos', 'MacrophagesTIMD4pos', 'MacrophagesLYVE1pos',
       'MacrophagesATF3pos', 'DendriticCellsType1', 'DendriticCellsMature',
       'PlasmacytoidDendriticCells', 'MastCells', 'Megakaryocytes',
       'TCellsCD4pos', 'TCellsCD8pos', 'TregsCD4pos', 'ProBCells', 'BCells',
       'BCellsMS4A1pos', 'NaturalKillerCells', 'InnateLymphoidCells'],
      dtype='object')
INFO     Using batches from adata.obs["sample"]                                              
INFO     No label_key inputted, assuming all cells have same label                           
INFO     Using data from adata.X
```

Example 32 (unknown):
```unknown
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function setup_anndata is deprecated; Please use the model-specific setup_anndata methods instead. The global method will be removed in version 0.15.0.
```

Example 33 (python):
```python
INFO     Successfully registered anndata object containing 5433 cells, 16306 vars, 5 batches,
         1 labels, and 0 proteins. Also registered 0 extra categorical covariates and 0 extra
         continuous covariates.                                                              
INFO     Please do not further modify adata until model is trained.
```

Example 34 (unknown):
```unknown
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function view_anndata_setup is deprecated; This method will be removed in 0.15.0. Please avoid building any new dependencies on it.
```

Example 35 (unknown):
```unknown
Anndata setup with scvi-tools version 0.14.4.
```

Example 36 (unknown):
```unknown
Data Summary              
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┓
┃             Data             ┃ Count ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━┩
│            Cells             │ 5433  │
│             Vars             │ 16306 │
│            Labels            │   1   │
│           Batches            │   5   │
│           Proteins           │   0   │
│ Extra Categorical Covariates │   0   │
│ Extra Continuous Covariates  │   0   │
└──────────────────────────────┴───────┘
```

Example 37 (python):
```python
SCVI Data Registry              
┏━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃     Data      ┃    scvi-tools Location    ┃
┡━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│       X       │          adata.X          │
│ batch_indices │ adata.obs['_scvi_batch']  │
│    labels     │ adata.obs['_scvi_labels'] │
└───────────────┴───────────────────────────┘
```

Example 38 (python):
```python
Label Categories                        
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 39 (python):
```python
Batch Categories                         
┏━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃   Source Location   ┃     Categories     ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['sample'] │ HCAHeartST10298118 │          0          │
│                     │ HCAHeartST10298119 │          1          │
│                     │ HCAHeartST10298121 │          2          │
│                     │ HCAHeartST10317187 │          3          │
│                     │ HCAHeartST10317188 │          4          │
└─────────────────────┴────────────────────┴─────────────────────┘
```

Example 40 (unknown):
```unknown
CPU times: user 1.15 s, sys: 283 ms, total: 1.43 s
Wall time: 1.43 s
```

Example 41 (python):
```python
# create and train the model
mod = cell2location.models.Cell2location(
    adata_vis, cell_state_df=inf_aver, 
    # the expected average cell abundance: tissue-dependent 
    # hyper-prior which can be estimated from paired histology:
    N_cells_per_location=20,
    # hyperparameter controlling normalisation of
    # within-experiment variation in RNA detection (using default here):
    detection_alpha=20
) 
adata_vis.var.head()
```

Example 42 (python):
```python
mod.train(max_epochs=20000, 
          # train using full data (batch_size=None)
          batch_size=None, 
          # use all data points in training because 
          # we need to estimate cell abundance at all locations
          train_size=1,
          use_gpu=True)

# plot ELBO loss history during training, removing first 100 epochs from the plot
mod.plot_history(1000)
plt.legend(labels=['full data training']);

# In this section, we export the estimated cell abundance (summary of the posterior distribution).
adata_vis = mod.export_posterior(
    adata_vis, sample_kwargs={'num_samples': 1000, 'batch_size': mod.adata.n_obs, 'use_gpu': True}
)

# Save model
mod.save(f"{run_name}/DuctusArteriosus", overwrite=True)

# Save anndata object with results
adata_vis.write(f"{run_name}/DuctusArteriosus/sp.h5ad")

# Examine reconstruction accuracy to assess if there are any issues with mapping
# the plot should be roughly diagonal, strong deviations will signal problems
mod.plot_QC()
```

Example 43 (unknown):
```unknown
GPU available: True, used: True
TPU available: False, using: 0 TPU cores
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/pytorch_lightning/trainer/configuration_validator.py:99: UserWarning: you passed in a val_dataloader but have no validation_step. Skipping val loop
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
```

Example 44 (unknown):
```unknown
Epoch 20000/20000: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20000/20000 [44:49<00:00,  7.44it/s, v_num=1, elbo_train=5.14e+7]
Sampling local variables, batch: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:36<00:00, 36.09s/it]
Sampling global variables, sample: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 999/999 [00:34<00:00, 29.37it/s]
```

Example 45 (unknown):
```unknown
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'sample' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'Publication' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'sangerID' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'combinedID' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'donor' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'donor_type' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'region' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'region_finest' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'age' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'gender' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'facility' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'cell_or_nuclei' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'modality' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'kit_10x' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'flushed' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'status' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'feature_types' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'genome' as categorical
```

---

## Import modules¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig2_Overview_of_cellular_niches_and_TissueTypist/tissuetypist_assess_VisiumHD.html

**Contents:**
- Import modules¶
- Read in qeury adata¶
- Read in reference adata¶
- Harmonise cell type names¶
- Calculate cell type proportions per niche¶
  - Query¶
  - Reference¶
- Subset shared cell types¶

**Examples:**

Example 1 (python):
```python
import numpy as np
import pandas as pd
import scanpy as sc
import anndata
import os
import matplotlib.pyplot as plt
import seaborn as sns
import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 

sc.settings.verbosity = 1  # verbosity: errors (0), warnings (1), info (2), hints (3)
```

Example 2 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/rfs/project/rfs-iCNyzSAaucw/kk837/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 3 (unknown):
```unknown
figdir = '/home/kk837/rfs/rfs-teichlab-nfs-iCNyzSAaucw/kk837/notebooks/Foetal/fetal_ms_figures'
```

Example 4 (unknown):
```unknown
coord_columns = ('window_col', 'window_row')
list(coord_columns)
```

Example 5 (unknown):
```unknown
['window_col', 'window_row']
```

Example 6 (python):
```python
# read in 
sample_id = 'HEA_FOET14880396'
celltype_col = 'fine_grain_4Jan2025'
adata = sc.read_h5ad(f'/rfs/project/rfs-iCNyzSAaucw/kk837/data_objects/Foetal/VisiumHD/{sample_id}_b2c_cells_filtered_lognorm.h5ad')
adata
```

Example 7 (unknown):
```unknown
AnnData object with n_obs × n_vars = 129803 × 18047
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'fine_grain_4Jan2025', 'HistologicalAnnotation', 'SANaxis_landmarks', 'SAN_axis', 'SAN_axis_bin_annot', 'tt_prediction'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'HistologicalAnnotation_colors', 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'fine_grain_4Jan2025_colors', 'hvg', 'log1p', 'neighbors', 'pca', 'spatial', 'tt_prediction_colors', 'umap'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 8 (python):
```python
adata.X.data[:5]
```

Example 9 (unknown):
```unknown
array([1.27187559, 1.95641568, 1.75023687, 1.96760567, 2.7810601 ])
```

Example 10 (python):
```python
# update cell type name for plotting
df = pd.read_csv('/rfs/project/rfs-iCNyzSAaucw/kk837/data_objects/Foetal/finegrain_name_mapping.csv')
celltype_mapping = df.set_index('Full_name')['Short_name_finalised'].to_dict()
adata.obs.replace({celltype_col:{
    'SinoatrialNodePacemakerCellsHorn':'SinusHornPacemakerCells'
}},inplace=True)
adata.obs.replace({celltype_col:celltype_mapping},inplace=True)
```

Example 11 (python):
```python
/tmp/ipykernel_240910/4045961491.py:4: FutureWarning: The behavior of Series.replace (and DataFrame.replace) with CategoricalDtype is deprecated. In a future version, replace will only be used for cases that preserve the categories. To change the categories, use ser.cat.rename_categories instead.
  adata.obs.replace({celltype_col:{
/tmp/ipykernel_240910/4045961491.py:7: FutureWarning: The behavior of Series.replace (and DataFrame.replace) with CategoricalDtype is deprecated. In a future version, replace will only be used for cases that preserve the categories. To change the categories, use ser.cat.rename_categories instead.
  adata.obs.replace({celltype_col:celltype_mapping},inplace=True)
```

Example 12 (python):
```python
adata_ref = sc.read_h5ad('/rfs/project/rfs-iCNyzSAaucw/kk837/data_objects/Foetal/Visium/visium_foetal_healthy-all_cell2loc-adata.h5ad')
adata_ref
```

Example 13 (unknown):
```unknown
AnnData object with n_obs × n_vars = 39223 × 58
    obs: 'array_row', 'array_col', 'sample', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'mt_frac', 'sangerID', 'Publication', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'status', 'cell2loc-map_group', 'JC_edit', 'sample_for_cell2loc', 'week', 'leiden', 'annotation_final', 'annotation_final_mod'
    uns: 'JC_edit_colors', 'annotation_colors', 'annotation_final_colors', 'cell2loc-map_group_colors', 'leiden', 'neighbors', 'region_cluster_colors', 'sample_for_cell2loc_colors', 'umap'
    obsm: 'MT', 'X_umap', 'means_cell_abundance_w_sf', 'q05_cell_abundance_w_sf', 'q05_cell_abundance_w_sf_ndarray', 'q95_cell_abundance_w_sf', 'spatial', 'stds_cell_abundance_w_sf'
    obsp: 'connectivities', 'distances'
```

Example 14 (python):
```python
# update cell type name
df = pd.read_csv('/rfs/project/rfs-iCNyzSAaucw/kk837/data_objects/Foetal/finegrain_name_mapping.csv')
celltype_mapping = df.set_index('Full_name')['Short_name_finalised'].to_dict()
adata_ref.var_names = [celltype_mapping[x] for x in adata_ref.var_names]
```

Example 15 (python):
```python
celltype_mapping_2 = {
    'BMS4':'B',
    'CorAdvFB':'FB',
    'GVAdvFB':'FB',
    'MonoMPO':'Mono',
    'MyoFB':'FB',
    'MyoFb':'FB',
     'ProB':'B',
    'SubEpiFb':'FB',
    'vCMCL':'vCMC',
    'vCMCR':'vCMC',
    'vCMTL':'vCMT',
    'vCMTR':'vCMT',
    'FBCOL2A1':'FB',
    'GVSMCP':'GVSMC',
    'GVSMCT':'GVSMC',
    'SANPCHd':'SANPC',
     'SHPC':'SANPC',
     'SANPCTl':'SANPC',
}
adata.obs[celltype_col] = [celltype_mapping_2[x] if x in celltype_mapping_2.keys() else x for x in adata.obs[celltype_col]]
adata_ref.var_names = [celltype_mapping_2[x] if x in celltype_mapping_2.keys() else x for x in adata_ref.var_names]
```

Example 16 (python):
```python
set(adata_ref.var_names) - set(adata.obs[celltype_col])
```

Example 17 (unknown):
```unknown
{'DCMat',
 'DCT1',
 'LNFRC',
 'MPATF3',
 'MPTIMD4',
 'MastC',
 'NK',
 'TCD4',
 'TCD8',
 'Treg',
 'pDC'}
```

Example 18 (python):
```python
set(adata.obs[celltype_col]) - set(adata_ref.var_names)
```

Example 19 (unknown):
```unknown
{'Eryth',
 'Fibroblasts_DoubletsWithCMs',
 'PericFib',
 'PericInt',
 'PericardialCells_DoubletsWithCMs',
 'SchwNeuro',
 'VentricularCardiomyocytes_Doublets',
 'VentricularCardiomyocytes_DoubletsWithErythrocytes',
 'unassigned'}
```

Example 20 (python):
```python
ctab = pd.crosstab(adata.obs[celltype_col],adata.obs['tt_prediction'])
prop_df = (ctab/ctab.sum(axis=0))
prop_df.sum(axis=0)
```

Example 21 (unknown):
```unknown
tt_prediction
AV ring                      1.0
AVnode                       1.0
Adventitia - GV              1.0
Atrium                       1.0
Coronary vessel              1.0
Ductus arteriosus            1.0
Endocardial cusion           1.0
Endocardium - Atrial         1.0
Endocardium - Ventricular    1.0
Epicardium                   1.0
Great vessel                 1.0
SAnode                       1.0
VCS - Distal                 1.0
VCS - Proximal               1.0
Valves                       1.0
Ventricle - Compact          1.0
Ventricle - Trabeculated     1.0
dtype: float64
```

Example 22 (unknown):
```unknown
prop_df.head()
```

Example 23 (python):
```python
df = adata_ref.to_df()
df['annotation_final_mod'] = adata_ref.obs['annotation_final_mod'].copy()
df = df.groupby('annotation_final_mod').sum()
df = df.groupby(df.columns, axis=1).sum()

prop_df_ref = (df.T/df.sum(axis=1))
prop_df_ref.sum(axis=0)
```

Example 24 (unknown):
```unknown
/tmp/ipykernel_240910/4260203183.py:3: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning.
  df = df.groupby('annotation_final_mod').sum()
/tmp/ipykernel_240910/4260203183.py:4: FutureWarning: DataFrame.groupby with axis=1 is deprecated. Do `frame.T.groupby(...)` without axis instead.
  df = df.groupby(df.columns, axis=1).sum()
```

Example 25 (unknown):
```unknown
annotation_final_mod
Atrium                       1.0
Ventricle - Compact          1.0
Ventricle - Trabeculated     1.0
SAnode                       1.0
AVnode                       1.0
VCS - Proximal               1.0
VCS - Distal                 1.0
Great vessel                 1.0
Coronary vessel              1.0
Ductus arteriosus            1.0
Adventitia - Coronary        1.0
Adventitia - GV              1.0
Epicardium                   1.0
Endocardium - Atrial         1.0
Endocardium - Ventricular    1.0
Endocardial cusion           1.0
AV ring                      1.0
Valves                       1.0
LN                           1.0
dtype: float64
```

Example 26 (unknown):
```unknown
prop_df_ref.head()
```

Example 27 (python):
```python
# shared cell types
shared = [x for x in prop_df.index if x in prop_df_ref.index]
print(len(shared))
prop_df = prop_df.loc[shared]
prop_df_ref = prop_df_ref.loc[shared]
```

Example 28 (python):
```python
# shared niches
shared = [x for x in prop_df.columns if x in prop_df_ref.columns]
print(len(shared))
prop_df = prop_df[shared]
prop_df_ref = prop_df_ref[shared]
```

Example 29 (python):
```python
print(prop_df.shape)
print(prop_df_ref.shape)
```

Example 30 (unknown):
```unknown
(38, 17)
(38, 17)
```

Example 31 (python):
```python
%%time
corr, pval = utils.df_pearson_corr_with_p_vectorized(prop_df, prop_df_ref, p_adjust='fdr_bh')

# draw the initial clustermap (for ordering only)
cg = sns.clustermap(
    corr,
    cmap='vlag',
    center=0,
    metric='euclidean',
    method='average',
    figsize=(6, 6)
)

# 4) extract the leaf order and reindex both matrices
order = cg.dendrogram_row.reordered_ind
labels = list(corr.index[order])

corr_ord = corr.loc[labels, labels]
pval_ord = pval.loc[labels, labels]

# 5) replot as a heatmap and annotate significance
fig, ax = plt.subplots(figsize=(6, 6))
sns.heatmap(
    corr_ord,
    cmap='vlag',
    center=0,
    square=True,
    cbar_kws={'label': 'Pearson r'},
    ax=ax
)

# add asterisks for p < 0.01
n = corr_ord.shape[0]
for i in range(n):
    for j in range(n):
        if pval_ord.iat[i, j] < 0.01:
            ax.text(j + 0.5, i + 0.5, '*',
                    ha='center', va='center',
                    fontsize=10)

ax.set_xticklabels(labels, rotation=90)
ax.set_xlabel('Reference')
ax.set_yticklabels(labels, rotation=0)
ax.set_ylabel('Visium HD, TissueTypist prediction')
plt.tight_layout()
plt.savefig(f'{figdir}/TissueTypist_celltype-prop-corr_VisiumHD.pdf', bbox_inches="tight",dpi=300)
plt.show()
```

Example 32 (unknown):
```unknown
CPU times: user 932 ms, sys: 236 ms, total: 1.17 s
Wall time: 7.21 s
```

---

## Import required modules¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/VisiumHD/VisiumHD-8um_QC-filtering.html

**Contents:**
- Import required modules¶
- Read in 8um bin data¶
- QC and preprocessing¶
- Filtering¶
- Save¶

311232 rows × 14 columns

**Examples:**

Example 1 (python):
```python
import warnings
warnings.simplefilter(action='ignore', category=FutureWarning)
warnings.filterwarnings('ignore')

import scanpy as sc
import anndata
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import matplotlib as mpl
import seaborn as sns
import os
```

Example 2 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/rfs/project/rfs-iCNyzSAaucw/kk837/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 3 (python):
```python
sc.settings.verbosity = 1          # verbosity: errors (0), warnings (1), info (2), hints (3)
sc.settings.set_figure_params(dpi=80)
```

Example 4 (unknown):
```unknown
!ls /rfs/project/rfs-iCNyzSAaucw/jc2226/heart_team_data/mapped/spaceranger301
```

Example 5 (unknown):
```unknown
HEA_AD14880300	HEA_AD14880301	HEA_FOET14880396  HEA_FOET14880397
```

Example 6 (python):
```python
# read in object which has window assignment
sample_id = 'HEA_FOET14880396'
path_adata = f"/rfs/project/rfs-iCNyzSAaucw/jc2226/heart_team_data/mapped/spaceranger301/{sample_id}/outs/binned_outputs/square_008um"
adata = sc.read_10x_h5(f'{path_adata}/filtered_feature_bc_matrix.h5')
adata.var_names_make_unique()
adata
```

Example 7 (unknown):
```unknown
AnnData object with n_obs × n_vars = 320061 × 18085
    var: 'gene_ids', 'feature_types', 'genome'
```

Example 8 (python):
```python
adata.obs.head()
```

Example 9 (python):
```python
adata.var.head()
```

Example 10 (python):
```python
adata.var["mt"] = adata.var_names.str.startswith("MT-")
sc.pp.calculate_qc_metrics(adata, qc_vars=["mt"], inplace=True)
```

Example 11 (python):
```python
fig, axs = plt.subplots(1, 4, figsize=(15, 3))
sns.histplot(adata.obs["total_counts"], kde=False, ax=axs[0])
sns.histplot(
    adata.obs["total_counts"][adata.obs["total_counts"] < 1000],
    kde=False,
    bins=40,
    ax=axs[1],
)
sns.histplot(adata.obs["n_genes_by_counts"], kde=False, bins=60, ax=axs[2])
sns.histplot(
    adata.obs["n_genes_by_counts"][adata.obs["n_genes_by_counts"] < 1000],
    kde=False,
    bins=60,
    ax=axs[3],
)
fig.suptitle('8um')
fig.tight_layout()
```

Example 12 (python):
```python
fig, axs = plt.subplots(1, 2, figsize=(10, 3))
sns.histplot(
    adata.obs["total_counts"][adata.obs["total_counts"] < 8000],
    kde=False,
    bins=60,
    ax=axs[0],
)
sns.histplot(
    adata.obs["n_genes_by_counts"][adata.obs["n_genes_by_counts"] < 3500],
    kde=False,
    bins=60,
    ax=axs[1],
)
fig.suptitle('8um')
fig.tight_layout()
```

Example 13 (python):
```python
# mito
sns.histplot(adata.obs['pct_counts_mt'], kde=False)
plt.title('8um')
plt.show()
```

Example 14 (python):
```python
# filtering out poor quality cells/bins
sc.pp.filter_cells(adata, min_genes=100)
print(f'shape of 8um: {adata.shape}')
```

Example 15 (unknown):
```unknown
shape of 8um: (311232, 18085)
```

Example 16 (python):
```python
# prepare coordinates
adata.obs['binned_8_row'] = [int(x.split('_')[2]) for x in adata.obs_names]
adata.obs['binned_8_col'] = [int(x.split('_')[3].replace('-1','')) for x in adata.obs_names]
sns.scatterplot(x='binned_8_col',y='binned_8_row',data=adata.obs)
plt.show()
adata.obs
```

Example 17 (python):
```python
adata.write(f'/rfs/project/rfs-iCNyzSAaucw/kk837/data_objects/Foetal/VisiumHD/{sample_id}_8um-bins_filtered.h5ad')
```

Example 18 (python):
```python
print(f'/rfs/project/rfs-iCNyzSAaucw/kk837/data_objects/Foetal/VisiumHD/{sample_id}_8um-bins_filtered.h5ad')
```

Example 19 (unknown):
```unknown
/rfs/project/rfs-iCNyzSAaucw/kk837/data_objects/Foetal/VisiumHD/HEA_FOET14880396_8um-bins_filtered.h5ad
```

---

## Tutorial on single-cell trajectory alignment using Genes2Genes¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig4_Cardiomyocytes/AtrialCardiomyocytes/g2g_L_v_RaCM_pseudotime_associated_genes.html

**Contents:**
- Tutorial on single-cell trajectory alignment using Genes2Genes¶
  - Min max normalize the pseudotime¶
  - Check the number of bins in the optimal binning structure using OptBinning package¶
  - Visualize the interpolation binning structure in terms of the cell type composition¶
  - Run G2G alignment¶
  - The alignment distribution across all genes¶
  - Run gene set overrepresentation analysis over the top k mismatching genes¶
- COMMON | HIGH SIMILARITY | INCREASING¶
- COMMON | HIGH SIMILARITY | DECREASING¶
- COMMON | HIGH SIMILALRITY | DECREASING¶
- LEFT | LOW SIMILALRITY | INCREASING¶
- Look for enrichment of CHD genes in any of these groups¶
  - Clustering genes using their alignments¶
  - Below is the Levenshtein distance heat map of all genes ordered based on the above clustering structure¶

G2G aims to guide downstream comparative analysis of single-cell reference and query systems along any axis of progression (e.g. differentiation pseudotime, disease/treatment response pseudotime etc.). This notebook describes how we can use G2G framework to infer and analyse gene-level trajectory alignments between a given reference and query dataset.

In this tutorial, we are going to compare trajectories between two treatment groups (PAM and LPS) of mouse bone marrow-derived dendritic cells from Shalek et al (2014). The single cell datasets and their pseudotime estimates were downloaded from https://github.com/shenorrLab/cellAlign (Alpert et al 2018) and packaged into adata objects. There are 2 gene modules: global (core antiviral module) and local (peaked inflammatory module) considered by Alpert et al (2018), and we use the local module as an example comparison.

Load the reference and query anndata objects

OptBinning estimates 14 optimal number of splits for both reference and query pseudotime distributions. Therefore we choose the same number of interpolation points.

For this dataset, we use the author-given time annotations (1h,2h,4h,6h) as the cell-type annotations.

Next we define a colormap of our choice for these annotations, and call the below function.

This is done by first creating an aligner object, passing and setting all relevant parameters. Next we align all gene pairs. (This step is parallelizing indepedenent gene-alignments to make the process time-efficient, however the computational time for an individual alignment will increase as the number of cells and/or the number of interpolation time points increase.

Now we can check the aggregate (average) alignment across all genes:

We can also visualize this alignment in terms of cell-type composition

We can also visualize an individual gene (e.g. JUNB), displaying its alignment statistics

To check only the cell plots of a gene alignment (e.g. SERTAD2)

The below attributes and functions can be used to examine any gene-alignment object

We can use the alignment similarity percentage statistic of genes to rank genes from highly distant to highly similar

1546 rows × 6 columns

1546 rows × 8 columns

1546 rows × 8 columns

Let us use 30% alignment similarity (=0.3) as a threshold in this case

1546 rows × 9 columns

We first run cluster diagnostics to decide on a distance threshold with a good tradeoff between the number of clusters and the quality of structure. We use levenshtein distance metric

We can inspect structures of distance thresholds that give local optimals of the mean silhouette score.

If we select distance threshold 0.37 which gives a local optimal of 0.3749 silhouette_score with 7 clusters

Below visualizes all alignment paths in each cluster along with its number of genes

1546 rows × 5 columns

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import gseapy as gp

def GO_plot(gene_list, gene_sets=['MSigDB_Hallmark_2020', 'KEGG_2021_Human', 'GO_Biological_Process_2023', 'Reactome_2022'], 
            display_table=False, title_suffix=None, sort_by='Combined Score', n_rank = 10, n_genes=None, FDR=None):
    """
    Generate GO plots and optionally display tables based on enrichment analysis.

    Parameters:
    - gene_list: List of gene names as strings
    - gene_sets: List of gene set names as strings
    - display_table: Boolean flag to display table or not
    - title_suffix: Suffix for the plot title
    - sort_by: Column name to sort the results and plot. Other options in table, e.g. 'n_genes'
    - n_rank: number of terms to include in plot
    """
    
    # Perform enrichment analysis
    enr_res = gp.enrichr(gene_list=gene_list,
                         gene_sets=gene_sets,
                         organism='human',
                         outdir=None)
    
    # Loop through each gene set to process and plot the results
    for geneset in gene_sets:
        print(geneset)
        
        # Filter the results based on the Gene_set
        df = enr_res.results[enr_res.results['Gene_set'] == geneset]
        
        # Calculate the number of genes in the overlap and add as a new column
        df['n_genes'] = [int(x.split('/')[0]) for x in df['Overlap']]
        
        # Sort the dataframe based on the specified column
        df = df.sort_values(sort_by, ascending=False)

        # Only use terms where a threshold number of genes are present
        if n_rank is not None:
            df = df[df.n_genes>=n_genes]

        if FDR is not None:
            df = df[df['Adjusted P-value']<FDR]
        
        # Optionally display the table
        if display_table:
            display(df[:30])
        
        # Generate the bar plot for the top 10 ranked terms
        plt.rcParams['axes.grid'] = False
        plt.rcParams['figure.figsize'] = 4, 3
        plt.barh(width=df[:n_rank][sort_by],
                 y=[x.split(' (')[0] for x in df[:n_rank]['Term']],
                 color='darkred')
        plt.gca().invert_yaxis()
        plt.xlabel(sort_by)
        if title_suffix:
            plt.title(f'{geneset}_{title_suffix}')
        else:
            plt.title(f'{geneset}')
        plt.margins(y=0.02)
        plt.show()
        
    return df
```

Example 2 (python):
```python
import anndata

def split_adata_by_modality(adata, modality_column, modality_values):
    """
    Splits an AnnData object into multiple AnnData objects based on a given modality column.
    
    Parameters:
        adata (anndata.AnnData): The original AnnData object.
        modality_column (str): The name of the column in adata.var that contains the modality information.
        modality_values (list): List of unique modalities that you want to split the adata into.
        
    Returns:
        dict: A dictionary where the keys are the modality names and the values are the new AnnData objects.
    """
    # Initialize an empty dictionary to hold the new AnnData objects
    splitted_adatas = {}
    
    # Loop through each modality value to create a new AnnData object
    for modality in modality_values:
        # Get the indices of the features belonging to the current modality
        feature_idx = adata.var[modality_column] == modality
        
        # Create a new AnnData object with only the features belonging to the current modality
        new_adata = anndata.AnnData(X=adata[:, feature_idx].X,
                                    obs=adata.obs,
                                    var=adata.var.loc[feature_idx],
                                    obsm=adata.obsm,
                                    uns=adata.uns)
        
        # Add the new AnnData object to the dictionary
        splitted_adatas[modality] = new_adata
    
    return splitted_adatas
```

Example 3 (python):
```python
import scanpy as sc
import anndata
import numpy as np
import seaborn as sb
import matplotlib.pyplot as plt
import platform
from optbinning import ContinuousOptimalBinning
import pandas as pd


from genes2genes import Main
from genes2genes import VisualUtils
from genes2genes import ClusterUtils
from genes2genes import TimeSeriesPreprocessor
from genes2genes import PathwayAnalyserV2

print(platform.python_version())
```

Example 4 (python):
```python
import matplotlib
matplotlib.rcParams['pdf.fonttype'] = 42
matplotlib.rcParams['ps.fonttype'] = 42
```

Example 5 (unknown):
```unknown
temporal_ordering='age_days_smoothed_once'
```

Example 6 (python):
```python
paired_adata_with_time_path = "/lustre/scratch126/cellgen/team205/heart/objects/fetal/RNA/aCMs_WholeSample_paired_with_time.h5ad"
```

Example 7 (python):
```python
adata=sc.read(paired_adata_with_time_path)
adata
```

Example 8 (unknown):
```unknown
AnnData object with n_obs × n_vars = 5664 × 544641
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', 'age_days', 'age_days_smoothed_once', 'age_days_normalized', 'age_days_normalized_smoothed_once', 'age_days_smoothed_iteratively', 'age_days_normalized_smoothed_iteratively', 't', 'seg', 'edge', 't_sd', 'milestones'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id'
    uns: 'donor_colors', 'fine_grain_colors', 'graph', 'milestones_colors', 'neighbors', 'ppt', 'pseudotime_list', 'seg_colors', 'umap', 'week_colors'
    obsm: 'X_R', 'X_multiVI', 'X_umap'
    obsp: 'connectivities', 'distances'
```

Example 9 (python):
```python
adata.obs["time"]=adata.obs['t'].copy()
```

Example 10 (python):
```python
adata.X.data[:5]
```

Example 11 (unknown):
```unknown
array([2., 1., 1., 1., 1.], dtype=float32)
```

Example 12 (python):
```python
sc.pl.umap(adata,color=["seg","milestones","t","fine_grain"],ncols=2)
```

Example 13 (python):
```python
modality_column = "modality"
modality_values = ["Gene Expression", "Peaks"]
splitted_adatas = split_adata_by_modality(adata, modality_column, modality_values)

adata_gex = splitted_adatas["Gene Expression"]
print(adata_gex)

adata_atac = splitted_adatas["Peaks"]
print(adata_atac)
```

Example 14 (unknown):
```unknown
AnnData object with n_obs × n_vars = 5664 × 36601
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', 'age_days', 'age_days_smoothed_once', 'age_days_normalized', 'age_days_normalized_smoothed_once', 'age_days_smoothed_iteratively', 'age_days_normalized_smoothed_iteratively', 't', 'seg', 'edge', 't_sd', 'milestones', 'time'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id'
    uns: 'donor_colors', 'fine_grain_colors', 'graph', 'milestones_colors', 'neighbors', 'ppt', 'pseudotime_list', 'seg_colors', 'umap', 'week_colors'
    obsm: 'X_R', 'X_multiVI', 'X_umap'
AnnData object with n_obs × n_vars = 5664 × 508040
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', 'age_days', 'age_days_smoothed_once', 'age_days_normalized', 'age_days_normalized_smoothed_once', 'age_days_smoothed_iteratively', 'age_days_normalized_smoothed_iteratively', 't', 'seg', 'edge', 't_sd', 'milestones', 'time'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id'
    uns: 'donor_colors', 'fine_grain_colors', 'graph', 'milestones_colors', 'neighbors', 'ppt', 'pseudotime_list', 'seg_colors', 'umap', 'week_colors'
    obsm: 'X_R', 'X_multiVI', 'X_umap'
```

Example 15 (python):
```python
sc.pl.umap(adata_gex,color='VASH2',cmap="RdPu")
```

Example 16 (python):
```python
# split intlo REF and QUERY
adata_ref=adata_gex[adata_gex.obs.fine_grain=="AtrialCardiomyocytesLeft"].copy()
adata_query=adata_gex[adata_gex.obs.fine_grain=="AtrialCardiomyocytesRight"].copy()
```

Example 17 (python):
```python
sc.pl.umap(adata_ref,color=["time","fine_grain"],ncols=2)
sc.pl.umap(adata_query,color=["time","fine_grain"],ncols=2)
```

Example 18 (python):
```python
sc.pp.normalize_total(adata_ref, target_sum=1e4)
sc.pp.log1p(adata_ref)

sc.pp.normalize_total(adata_query, target_sum=1e4)
sc.pp.log1p(adata_query)
```

Example 19 (python):
```python
print(adata_ref)
print(adata_query)
```

Example 20 (unknown):
```unknown
AnnData object with n_obs × n_vars = 3041 × 36601
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', 'age_days', 'age_days_smoothed_once', 'age_days_normalized', 'age_days_normalized_smoothed_once', 'age_days_smoothed_iteratively', 'age_days_normalized_smoothed_iteratively', 't', 'seg', 'edge', 't_sd', 'milestones', 'time'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id'
    uns: 'donor_colors', 'fine_grain_colors', 'graph', 'milestones_colors', 'neighbors', 'ppt', 'pseudotime_list', 'seg_colors', 'umap', 'week_colors', 'log1p'
    obsm: 'X_R', 'X_multiVI', 'X_umap'
AnnData object with n_obs × n_vars = 2623 × 36601
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', 'age_days', 'age_days_smoothed_once', 'age_days_normalized', 'age_days_normalized_smoothed_once', 'age_days_smoothed_iteratively', 'age_days_normalized_smoothed_iteratively', 't', 'seg', 'edge', 't_sd', 'milestones', 'time'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id'
    uns: 'donor_colors', 'fine_grain_colors', 'graph', 'milestones_colors', 'neighbors', 'ppt', 'pseudotime_list', 'seg_colors', 'umap', 'week_colors', 'log1p'
    obsm: 'X_R', 'X_multiVI', 'X_umap'
```

Example 21 (python):
```python
# check the current range
print(min(adata_ref.obs['time']), max(adata_ref.obs['time']))
print(min(adata_query.obs['time']), max(adata_query.obs['time']))

# if it does not follow [0,1] range, run below
adata_ref.obs['time'] = TimeSeriesPreprocessor.Utils.minmax_normalise(np.asarray(adata_ref.obs['time']))
adata_query.obs['time'] = TimeSeriesPreprocessor.Utils.minmax_normalise(np.asarray(adata_query.obs['time']))
```

Example 22 (unknown):
```unknown
0.4222375414474298 8.245970642037001
0.45019867983228623 8.303595486512195
```

Example 23 (python):
```python
# Visualize the pseudotime distributions
sb.kdeplot(adata_ref.obs['time'], fill=True, color='forestgreen') 
sb.kdeplot(adata_query.obs['time'], fill=True, color='midnightblue'); 
plt.xlabel('pseudotime'); plt.legend(); plt.show()
```

Example 24 (unknown):
```unknown
No artists with labels found to put in legend.  Note that artists whose label start with an underscore are ignored when legend() is called with no argument.
```

Example 25 (python):
```python
x = np.asarray(adata_ref.obs.time)
optb = ContinuousOptimalBinning(name='pseudotime', dtype="numerical")
optb.fit(x, x)
print(len(optb.splits))

x = np.asarray(adata_query.obs.time)
optb = ContinuousOptimalBinning(name='pseudotime', dtype="numerical")
optb.fit(x, x)
print(len(optb.splits))
```

Example 26 (python):
```python
col = np.array(sb.color_palette('colorblind'))[range(2)]
col
```

Example 27 (unknown):
```unknown
array([[0.00392157, 0.45098039, 0.69803922],
       [0.87058824, 0.56078431, 0.01960784]])
```

Example 28 (python):
```python
joint_cmap={'AtrialCardiomyocytesLeft':col[0], 'AtrialCardiomyocytesRight':col[1]}
vs = VisualUtils.VisualUtils.get_celltype_composition_across_time(adata_ref, adata_query, n_points=14, 
                                     ANNOTATION_COLNAME='fine_grain', optimal_binning=False)
```

Example 29 (unknown):
```unknown
# trying max n points for optimal binning = 14
====================================================
Optimal equal number of bins for R and Q =  14
```

Example 30 (python):
```python
# reads in the gene lists defined in the other notebook

import csv
import os

# Directory containing the CSV files
input_folder = "gene_lists"

# Initialize an empty dictionary to store the results
gene_lists = {}

# Iterate over each file in the directory
for filename in os.listdir(input_folder):
    if filename.endswith(".csv"):
        key = filename[:-4]  # Remove the '.csv' extension to get the key
        with open(os.path.join(input_folder, filename), 'r') as file:
            reader = csv.reader(file)
            # Convert each row into an item of the list (assuming one item per row)
            gene_list = [row[0] for row in reader]
            gene_lists[key] = gene_list

for k, v in gene_lists.items():
    print(f"{k}: {len(v)}")
```

Example 31 (unknown):
```unknown
INCREASING_genes: 1178
LEFT_DECREASING_AUCgene: 4
LEFT_INCREASING_peaks: 0
DECREASING_AUCregion: 35
RIGHT_INCREASING_genes: 61
LEFT_INCREASING_AUCgene: 0
LEFT_DECREASING_genes: 97
LEFT_genes: 277
RIGHT_AUCregion: 0
LEFT_INCREASING_genes: 72
COMMON_AUCgene: 44
RIGHT_DECREASING_peaks: 0
LEFT_INCREASING_AUCregion: 0
LEFT_peaks: 0
COMMON_genes: 1334
RIGHT_DECREASING_AUCgene: 1
RIGHT_DECREASING_genes: 57
LEFT_AUCgene: 5
LEFT_DECREASING_peaks: 0
DECREASING_genes: 5174
RIGHT_AUCgene: 2
LEFT_AUCregion: 2
RIGHT_INCREASING_AUCregion: 0
INCREASING_AUCregion: 4
DECREASING_AUCgene: 35
COMMON_peaks: 40
RIGHT_DECREASING_AUCregion: 0
INCREASING_peaks: 1571
DECREASING_peaks: 39998
RIGHT_peaks: 1
RIGHT_genes: 206
COMMON_AUCregion: 24
RIGHT_INCREASING_AUCgene: 0
LEFT_DECREASING_AUCregion: 0
INCREASING_AUCgene: 4
RIGHT_INCREASING_peaks: 0
```

Example 32 (unknown):
```unknown
# Assuming LEFT_genes, RIGHT_genes, and COMMON_genes are already defined as lists
LEFT_genes_set = set(gene_lists['LEFT_genes'])
RIGHT_genes_set = set(gene_lists['RIGHT_genes'])
COMMON_genes_set = set(gene_lists['COMMON_genes'])

# Create the union of these sets
gene_list_set = LEFT_genes_set | RIGHT_genes_set | COMMON_genes_set
```

Example 33 (python):
```python
gene_list = list(gene_list_set)
print(len(gene_list))
```

Example 34 (python):
```python
# Gets a a list of all human TFs
#TFs = pd.read_csv("human_TF_list.csv", index_col=0, header=1)
#TFs = TFs.Name.unique().tolist()
#gene_list = [gene for gene in TFs if gene in adata_ref.var_names]
```

Example 35 (unknown):
```unknown
gene_list[:10]
```

Example 36 (unknown):
```unknown
['TEAD1',
 'KHDRBS2',
 'MYPN',
 'CTNND1',
 'AL008730.1',
 'HSPA6',
 'NPTN',
 'EXOC6B',
 'FREM2',
 'PPP4C']
```

Example 37 (python):
```python
%%time

import os
import pickle

# Path for the folder and file
folder_path = '/nfs/team205/jc48/projects/fetal/code/g2g_aligners'
file_path = os.path.join(folder_path, 'COMMON_LEFT_RIGHT_aligner.pkl')

# Create folder if it doesn't exist
os.makedirs(folder_path, exist_ok=True)

# Check if the file already exists
if os.path.isfile(file_path):
    # Read the aligner object from the file
    print("this specific aligner already exists, reading it in...")
    with open(file_path, 'rb') as file:
        aligner = pickle.load(file)
else:
    # If the file does not exist, create and configure the aligner
    print("aligner not already made, making it...")
    aligner = Main.RefQueryAligner(adata_ref, adata_query, gene_list, len(vs.optimal_bining_S))
    aligner.WEIGHT_BY_CELL_DENSITY = True
    aligner.WINDOW_SIZE = 0.1
    aligner.state_params = [0.99, 0.1, 0.7]
    aligner.optimal_binning = True
    aligner.opt_binning_S = vs.optimal_bining_S
    aligner.opt_binning_T = vs.optimal_bining_T

    # Align all pairs
    aligner.align_all_pairs()

    # Save the aligner object to the file
    with open(file_path, 'wb') as file:
        pickle.dump(aligner, file)
```

Example 38 (unknown):
```unknown
this specific aligner already exists, reading it in...
CPU times: user 2.83 s, sys: 706 ms, total: 3.54 s
Wall time: 3.54 s
```

Example 39 (unknown):
```unknown
aligner.get_aggregate_alignment()
```

Example 40 (unknown):
```unknown
Average Alignment:  IIIDMMDIDDMMMMMMMM
```

Example 41 (unknown):
```unknown
vs.visualize_gene_alignment("IDDDMMMMMMMMMIIIDID")#, cmap=joint_cmap)
```

Example 42 (unknown):
```unknown
VisualUtils.show_gene_alignment('PITX2', aligner, vs)#, joint_cmap)
```

Example 43 (unknown):
```unknown
IIIDIDIDIDIIIIIIIIDDDDDDDDDD
Optimal alignment cost: 72.073 nits
Alignment similarity percentage: 0.0 %
```

Example 44 (unknown):
```unknown
df = aligner.get_stat_df()
```

Example 45 (unknown):
```unknown
mean matched percentage: 
64.46 %
```

Example 46 (unknown):
```unknown
VisualUtils.plot_alignmentSim_vs_l2fc(df)
```

Example 47 (unknown):
```unknown
gene_list = df[(df["alignment_similarity_percentage"]<0.45)&(df["abs_l2fc"]>0)].sort_values('alignment_similarity_percentage', ascending=True).Gene.tolist()
gene_list[:10]
```

Example 48 (unknown):
```unknown
['PITX2',
 'VASH2',
 'PANCR',
 'NAV3',
 'NTM',
 'PCDH7',
 'ROR1',
 'DCBLD2',
 'KCNMB2-AS1',
 'ZNF385B']
```

Example 49 (unknown):
```unknown
# adjust 'alignment_similarity_percentage' so it is actuall a perecntage
df['alignment_similarity_percentage'] *= 100
df
```

Example 50 (python):
```python
LEFT_genes_set = set(gene_lists['LEFT_genes'])
RIGHT_genes_set = set(gene_lists['RIGHT_genes'])
COMMON_genes_set = set(gene_lists['COMMON_genes'])

def classify_trajectory(gene):
    if gene in LEFT_genes_set:
        return "LaCM"
    elif gene in RIGHT_genes_set:
        return "RaCM"
    else:
        return "Common"

# Add the 'trajectory' column
df['trajectory'] = df['Gene'].apply(classify_trajectory)





INCREASING_genes_set = set(gene_lists['INCREASING_genes'])
DECREASING_genes_set = set(gene_lists['DECREASING_genes'])

def classify_trend(gene):
    if gene in INCREASING_genes_set:
        return "Increasing"
    elif gene in RIGHT_genes_set:
        return "Decreasing"
    else:
        return "Unchanging"

# Add the 'trajectory' column
df['trend'] = df['Gene'].apply(classify_trend)

df
```

Example 51 (unknown):
```unknown
df[(df["opt_alignment_cost"]<50)&(df["abs_l2fc"]>4)].sort_values('alignment_similarity_percentage', ascending=True).head(20)
```

Example 52 (unknown):
```unknown
genes_of_interest = ["GATA4","PITX2","NTM"]
#genes_of_interest = ["ATP5F1B","NDUFA11","NDUFB11","NDUFAB1","ATP5MC2"]
```

Example 53 (python):
```python
import seaborn as sns
import matplotlib.pyplot as plt

plt.figure(figsize=(6, 6))

markers = {
    "Increasing":"^",
    "Decreasing":"v",
    "Unchanging":"o"
}

#y_axis = 'opt_alignment_cost'
y_axis = 'alignment_similarity_percentage'

sns.scatterplot(data=df, x='l2fc', y=y_axis, hue='trajectory', edgecolor="none",
                #style = 'trend', markers = markers,
                legend=False, alpha=0.8)

# Add a horizontal line at y=0.5
plt.axhline(50, color='gray', linestyle='--', linewidth = 0.5)

# Add text annotations for LaCM and RaCM
plt.text(0.1, 1.05, 'RaCM', transform=plt.gca().transAxes, ha='left', va='top', weight='semibold')
plt.text(0.9, 1.05, 'LaCM', transform=plt.gca().transAxes, ha='right', va='top', weight='semibold')

# Label specific genes
for gene in genes_of_interest:
    gene_data = df[df['Gene'] == gene]
    for _, row in gene_data.iterrows():
        plt.text(row['l2fc'], row[y_axis], gene, 
                 horizontalalignment='right', size='small', color='black', weight='regular', style='italic')

# Set labels
plt.xlabel("Log2 Fold Change")

if y_axis == "alignment_similarity_percentage":
    plt.ylabel("Alignment Similarity (%)")

if y_axis == "opt_alignment_cost":
    plt.ylabel("Alignment Cost (nits)")

# Save the plot
plt.savefig("similarity_v_logFC.pdf", format='pdf', bbox_inches='tight')

# Show the plot
plt.show()
```

Example 54 (python):
```python
import seaborn as sns
import matplotlib.pyplot as plt
import pandas as pd

# Assuming 'df' is your DataFrame and 'genes_of_interest' is defined

plt.figure(figsize=(6, 6))

markers = {
    "Increasing": "^",
    "Decreasing": "v",
    "Unchanging": "o"
}

y_axis = 'alignment_similarity_percentage'  # or 'opt_alignment_cost' based on your need

# Plotting the main scatterplot
sns.scatterplot(data=df, x='l2fc', y=y_axis,
                #hue='trajectory',
                edgecolor="none",
                #style='trend', markers=markers,
                legend=False, alpha=0.8)

# Add a horizontal line at y=0.5
plt.axhline(50, color='gray', linestyle='--', linewidth=0.5)

# Add text annotations for LaCM and RaCM
plt.text(0.1, 1.05, 'RaCM', transform=plt.gca().transAxes, ha='left', va='top', weight='semibold')
plt.text(0.9, 1.05, 'LaCM', transform=plt.gca().transAxes, ha='right', va='top', weight='semibold')

# Label specific genes
for gene in genes_of_interest:
    gene_data = df[df['Gene'] == gene]
    for _, row in gene_data.iterrows():
        plt.text(row['l2fc'], row[y_axis], gene, 
                 horizontalalignment='right', size='medium', color='red', weight='regular', style='italic')

# Set labels
plt.xlabel("Log2 Fold Change")
if y_axis == "alignment_similarity_percentage":
    plt.ylabel("Alignment Similarity (%)")
elif y_axis == "opt_alignment_cost":
    plt.ylabel("Alignment Cost (nits)")

# Draw rectangles to indicate boundaries of zoom-in plots
from matplotlib.patches import Rectangle

# Adjusted rectangle for blue boundary (zoom-in area)
plt.gca().add_patch(Rectangle((0, 0), 5.5, 25, linewidth=1, linestyle='--', edgecolor='blue', facecolor='none', alpha=0.6))
# Adjusted rectangle for orange boundary (zoom-in area)
plt.gca().add_patch(Rectangle((-3, 0), 3, 25, linewidth=1, linestyle='--', edgecolor='orange', facecolor='none', alpha=0.6))
plt.savefig("./figures/similarity_v_DE.pdf", bbox_inches='tight')
# Show the main plot
plt.show()
```

Example 55 (python):
```python
import seaborn as sns
import matplotlib.pyplot as plt
import pandas as pd

plt.figure(figsize=(5, 2.5))


# Filter the DataFrame for the first scatterplot
df_filtered_2 = df[(df['l2fc'] < 0) & (df['l2fc'] > -6) &
                   (df['alignment_similarity_percentage'] < 25) & 
                   (df['alignment_similarity_percentage'] >= 0)]

specific_genes = ['MIR100HG', 'PLCL1', 'DGKH']
y_adjustment = 2  # The y-axis adjustment

# Sort the DataFrame based on 'l2fc' to ensure consistent adjustment application
df_filtered_2_sorted = df_filtered_2.sort_values(by='l2fc')

# Apply y-axis adjustments for specific genes
for i, gene_name in enumerate(specific_genes):
    # Find the index of the gene in the DataFrame
    gene_idx = df_filtered_2_sorted[df_filtered_2_sorted['Gene'] == gene_name].index
    if len(gene_idx) > 0:
        # Adjust the 'alignment_similarity_percentage' by 0.05 units times the position in the specific_genes list
        df_filtered_2_sorted.loc[gene_idx, 'alignment_similarity_percentage'] += y_adjustment * (i + 1)



# Now, plot each point. Since the DataFrame is already adjusted, no need for further manipulation
for _, row in df_filtered_2_sorted.iterrows():
    plt.text(row['l2fc'], row['alignment_similarity_percentage'], row['Gene'],
             fontdict={'horizontalalignment': 'center', 'verticalalignment': 'center', 'size': 9, 'weight': 'regular', 'color': 'orange'},
             bbox={'boxstyle': "round4", 'fc': 'white', 'ec': 'orange'})

plt.xlabel("Log2 Fold Change")
plt.ylabel("Alignment Similarity (%)")
plt.xlim(-3, 0)
plt.ylim(0, max(df_filtered_2_sorted['alignment_similarity_percentage'].max() + 5, 25))
plt.savefig("./figures/low_similarity_RaCMs.pdf", bbox_inches='tight')
plt.show()
```

Example 56 (unknown):
```unknown
df_filtered_2_sorted
```

Example 57 (python):
```python
import seaborn as sns
import matplotlib.pyplot as plt
import pandas as pd

plt.figure(figsize=(5, 2.5))

# Filter the DataFrame for the first scatterplot
df_filtered_1 = df[(df['l2fc'] > 0) & (df['l2fc'] < 6) &
                   (df['alignment_similarity_percentage'] < 25) & 
                   (df['alignment_similarity_percentage'] >= 0)]

specific_genes = ['PITX2', 'PANCR', 'PCDH7']
y_adjustment = 5  # The y-axis adjustment

# Sort the DataFrame based on 'l2fc' to ensure consistent adjustment application
df_filtered_1_sorted = df_filtered_1.sort_values(by='l2fc')

# Apply y-axis adjustments for specific genes
for i, gene_name in enumerate(specific_genes):
    # Find the index of the gene in the DataFrame
    gene_idx = df_filtered_1_sorted[df_filtered_1_sorted['Gene'] == gene_name].index
    if len(gene_idx) > 0:
        # Adjust the 'alignment_similarity_percentage' by 0.05 units times the position in the specific_genes list
        df_filtered_1_sorted.loc[gene_idx, 'alignment_similarity_percentage'] += y_adjustment * (i + 1)



# Now, plot each point. Since the DataFrame is already adjusted, no need for further manipulation
for _, row in df_filtered_1_sorted.iterrows():
    plt.text(row['l2fc'], row['alignment_similarity_percentage'], row['Gene'],
             fontdict={'horizontalalignment': 'center', 'verticalalignment': 'center', 'size': 9, 'weight': 'regular', 'color': 'blue'},
             bbox={'boxstyle': "round4", 'fc': 'white', 'ec': 'blue'})

plt.xlabel("Log2 Fold Change")
plt.ylabel("Alignment Similarity (%)")
plt.xlim(0, 5.5)
plt.ylim(0, max(df_filtered_1_sorted['alignment_similarity_percentage'].max() + 5, 25))
plt.savefig("./figures/low_similarity_LaCMs.pdf", bbox_inches='tight')
plt.show()
```

Example 58 (unknown):
```unknown
df_filtered_1_sorted
```

Example 59 (python):
```python
query_color = 'orange'
ref_color = 'blue'

def plotTimeSeries_cmap(gene, aligner, plot_cells = False, plot_mean_trend= False):
    
        al_obj = aligner.results_map[gene]
        plt.subplots(1,3,figsize=(15,3))
        plt.subplot(1,3,1)
        plotTimeSeriesAlignment(gene, aligner) 
        plt.subplot(1,3,2)
        max_val = np.max([np.max(np.asarray(aligner.ref_mat[al_obj.gene])), np.max(np.asarray(aligner.query_mat[al_obj.gene]))])
        min_val = np.min([np.min(np.asarray(aligner.ref_mat[al_obj.gene])), np.min(np.asarray(aligner.query_mat[al_obj.gene]))])
        g = sb.scatterplot(x=aligner.query_time, y=np.asarray(aligner.query_mat[al_obj.gene]), alpha=0.7, color = query_color, legend=False,linewidth=0.3, s=20)  
        plt.title('Query')
        plt.ylim([min_val-0.5,max_val+0.5])
        plt.subplot(1,3,3)
        g = sb.scatterplot(x=aligner.ref_time, y=np.asarray(aligner.ref_mat[al_obj.gene]), color = ref_color, alpha=0.7, legend=False,linewidth=0.3,s=20 ) 
        plt.title('Reference')
        plt.ylim([min_val-0.5,max_val+0.5])

def plotTimeSeriesAlignment_cmap(gene, aligner):  
    
        al_obj = aligner.results_map[gene]
        sb.scatterplot(x=al_obj.S.X, y=al_obj.S.Y, color = ref_color ,alpha=0.05, legend=False)#, label='Ref') 
        sb.scatterplot(x=al_obj.T.X, y=al_obj.T.Y, color = query_color ,alpha=0.05, legend=False)#, label ='Query') 
        plot_mean_trends(al_obj)
        plt.title(al_obj.gene)
        plt.xlabel('pseudotime')
        plt.ylabel('Gene expression')
        plt.axis('off')
        
        for i in range(al_obj.matched_region_DE_info.shape[0]):
            S_timebin = int(al_obj.matched_region_DE_info.iloc[i]['ref_bin'])
            T_timebin = int(al_obj.matched_region_DE_info.iloc[i]['query_bin']) 
            x_vals = [al_obj.matched_region_DE_info.iloc[i]['ref_pseudotime'],al_obj.matched_region_DE_info.iloc[i]['query_pseudotime']] 
            y_vals = [al_obj.S.mean_trend[S_timebin ], al_obj.T.mean_trend[T_timebin]] 
            plt.plot(x_vals, y_vals, color='black', linestyle='dashed', linewidth=1.5)

def plot_mean_trends(al_obj):
    # Plotting the mean trends for S and T with specified colors
    al_obj.S.plot_mean_trend(color=ref_color)
    al_obj.T.plot_mean_trend(color=query_color)
```

Example 60 (python):
```python
import matplotlib.pyplot as plt
from contextlib import redirect_stdout
import io


for gene in genes_of_interest:
    print(gene)
    # Redirect the output of the plot function to a StringIO object
    f = io.StringIO()
    with redirect_stdout(f):
        #VisualUtils.plotTimeSeries(gene, aligner, vs)
        #VisualUtils.plotTimeSeriesAlignment(gene, aligner)
        plotTimeSeriesAlignment_cmap(gene, aligner)

    
    # Now, the plot is created and can be saved
    plt.savefig(f"{gene}_time_series_alignment.png", format='png', bbox_inches='tight')
    plt.show()
```

Example 61 (unknown):
```unknown
df.loc[df['Gene']=='MB',:]
```

Example 62 (unknown):
```unknown
lowSimGenes = df[list(df['alignment_similarity_percentage'] < 30)]
topSimGenes = df[list(df['alignment_similarity_percentage'] > 30)]
```

Example 63 (unknown):
```unknown
highly_similar_common_genes = topSimGenes[topSimGenes['trajectory']=="Common"]['Gene'].tolist()
INCREASING_highly_similar_common_genes = [gene for gene in highly_similar_common_genes if gene in gene_lists['INCREASING_genes']]
len(INCREASING_highly_similar_common_genes)
```

Example 64 (unknown):
```unknown
gsea_results=GO_plot(gene_list = INCREASING_highly_similar_common_genes, n_genes=5, display_table=True,
        #gene_sets=['MSigDB_Hallmark_2020','GO_Biological_Process_2023','KEGG_2021_Human'],
        gene_sets=['MSigDB_Hallmark_2020'],
                    FDR=0.1)
```

Example 65 (unknown):
```unknown
MSigDB_Hallmark_2020
```

Example 66 (python):
```python
gsea_results["pct_genes_in_set"] = round((gsea_results["n_genes"] / len(INCREASING_highly_similar_common_genes)) * 100, 2)
gsea_results["neglog10"] = np.negative(np.log10(gsea_results['Adjusted P-value']))
gsea_results = gsea_results[gsea_results['Adjusted P-value']<0.05]
gsea_results
```

Example 67 (python):
```python
import matplotlib.pyplot as plt
import seaborn as sns
import pandas as pd

# Assuming gsea_results is your DataFrame

# Sort the DataFrame in decreasing order of neglog10 values
gsea_results_sorted = gsea_results.sort_values(by="neglog10", ascending=True)

plt.figure(figsize=(2, 3))

# Iterate over each row in the sorted DataFrame and plot individually
for _, row in gsea_results_sorted.iterrows():
    edge_color = 'red' if row["neglog10"] > 2 else 'black'
    plt.scatter(x=row["neglog10"], y=row["Term"], edgecolor=edge_color, color='white', lw=1)

plt.title("COMMON | HIGH SIMILARITY | INCREASING")
plt.ylabel("MSigDB Term")
plt.xlabel("-log(FDR)")

# Save the figure
plt.savefig(fname="./figures/COMMON___HIGH_SIMILARITY___INCREASING.pdf", format='pdf', bbox_inches='tight')

# Show the plot
plt.show()
```

Example 68 (unknown):
```unknown
highly_similar_common_genes = topSimGenes[topSimGenes['trajectory']=="Common"]['Gene'].tolist()
DECREASING_highly_similar_common_genes = [gene for gene in highly_similar_common_genes if gene in gene_lists['DECREASING_genes']]
len(DECREASING_highly_similar_common_genes)
```

Example 69 (unknown):
```unknown
gsea_results=GO_plot(gene_list = DECREASING_highly_similar_common_genes, n_genes=5, display_table=True,
        #gene_sets=['MSigDB_Hallmark_2020','GO_Biological_Process_2023','KEGG_2021_Human'],
        gene_sets=['MSigDB_Hallmark_2020'],
                    FDR=0.1)
```

Example 70 (unknown):
```unknown
MSigDB_Hallmark_2020
```

Example 71 (python):
```python
gsea_results["pct_genes_in_set"] = round((gsea_results["n_genes"] / len(DECREASING_highly_similar_common_genes)) * 100, 2)
gsea_results["neglog10"] = np.negative(np.log10(gsea_results['Adjusted P-value']))
gsea_results = gsea_results[gsea_results['Adjusted P-value']<0.05]
gsea_results
```

Example 72 (python):
```python
import matplotlib.pyplot as plt
import seaborn as sns
import pandas as pd

# Assuming gsea_results is your DataFrame

# Sort the DataFrame in decreasing order of neglog10 values
gsea_results_sorted = gsea_results.sort_values(by="neglog10", ascending=True)

plt.figure(figsize=(2, 3))

# Iterate over each row in the sorted DataFrame and plot individually
for _, row in gsea_results_sorted.iterrows():
    edge_color = 'red' if row["neglog10"] > 2 else 'black'
    plt.scatter(x=row["neglog10"], y=row["Term"], edgecolor=edge_color, color='white', lw=1)

plt.title("COMMON | HIGH SIMILARITY | DECREASING")
plt.ylabel("MSigDB Term")
plt.xlabel("-log(FDR)")

# Save the figure
plt.savefig(fname="./figures/COMMON___HIGH_SIMILARITY___DECREASING.pdf", format='pdf', bbox_inches='tight')

# Show the plot
plt.show()
```

Example 73 (unknown):
```unknown
terms = ["Fatty Acid Metabolism","Myogenesis"]

term_genes_dict = {}

for term in terms:
    genes_series = gsea_results[gsea_results["Term"] == term]['Genes'].str.split(";")
    term_associated_genes_list = genes_series.sum()
    term_genes_dict[term] = term_associated_genes_list
term_genes_dict
```

Example 74 (unknown):
```unknown
{'Fatty Acid Metabolism': ['HADHB', 'AUH', 'MLYCD', 'XIST', 'ACO2', 'PPARA'],
 'Myogenesis': ['CAMK2B',
  'DTNA',
  'COL4A2',
  'MB',
  'CDH13',
  'PDE4DIP',
  'SORBS1',
  'LARGE1',
  'PPFIA4',
  'SGCG']}
```

Example 75 (unknown):
```unknown
GENE_SUBSET = term_genes_dict["Myogenesis"]
aligner.get_aggregate_alignment_for_subset(GENE_SUBSET)
```

Example 76 (unknown):
```unknown
Average Alignment:  IIIDDMMMDMMMMMMMM
```

Example 77 (python):
```python
import matplotlib.pyplot as plt
from contextlib import redirect_stdout
import io


for gene in term_genes_dict["Myogenesis"]:
    print(gene)
    # Redirect the output of the plot function to a StringIO object
    f = io.StringIO()
    with redirect_stdout(f):
        #VisualUtils.plotTimeSeries(gene, aligner, vs)
        #VisualUtils.plotTimeSeriesAlignment(gene, aligner)
        plotTimeSeriesAlignment_cmap(gene, aligner)

    
    # Now, the plot is created and can be saved
    plt.savefig(f"{gene}_time_series_alignment.png", format='png', bbox_inches='tight')
    plt.show()
```

Example 78 (unknown):
```unknown
highly_similar_common_genes = topSimGenes[topSimGenes['trajectory']=="Common"]['Gene'].tolist()
DECREASING_highly_similar_common_genes = [gene for gene in highly_similar_common_genes if gene in gene_lists['DECREASING_genes']]
len(DECREASING_highly_similar_common_genes)
```

Example 79 (unknown):
```unknown
gsea_results=GO_plot(gene_list = DECREASING_highly_similar_common_genes, n_genes=5, display_table=False,
        #gene_sets=['MSigDB_Hallmark_2020','GO_Biological_Process_2023','KEGG_2021_Human'])
        gene_sets=['MSigDB_Hallmark_2020'],
                     FDR=0.1)
```

Example 80 (unknown):
```unknown
MSigDB_Hallmark_2020
```

Example 81 (python):
```python
gsea_results["pct_genes_in_set"] = round((gsea_results["n_genes"] / len(DECREASING_highly_similar_common_genes)) * 100, 2)
gsea_results["neglog10"] = np.negative(np.log10(gsea_results['Adjusted P-value']))
gsea_results = gsea_results[gsea_results['Adjusted P-value']<0.2]
gsea_results
```

Example 82 (python):
```python
plt.figure(figsize=(2,3))
sb.scatterplot(data = gsea_results, x = "neglog10", y = "Term", hue = "neglog10", legend = False, palette = "viridis", size = "pct_genes_in_set", edgecolor='black')
plt.title("COMMON | HIGH SIMILARITY | DECREASING")
plt.ylabel("MSigDB Term")
plt.xlabel("-log(adjusted p)")
plt.savefig(fname="./figures/COMMON___HIGH SIMILARITY___DECREASING.pdf", format='pdf', bbox_inches='tight')
plt.show()
```

Example 83 (unknown):
```unknown
lowly_similar_left_genes = lowSimGenes[lowSimGenes['trajectory']=="RaCM"]['Gene'].tolist()
INCREASING_lowly_similar_left_genes = [gene for gene in lowly_similar_left_genes if gene in gene_lists['INCREASING_genes']]
len(INCREASING_lowly_similar_left_genes)
```

Example 84 (unknown):
```unknown
gsea_results=GO_plot(gene_list = INCREASING_lowly_similar_left_genes, n_genes=1, display_table=True,
        gene_sets=['MSigDB_Hallmark_2020','GO_Biological_Process_2023','KEGG_2021_Human'])
        #gene_sets=['MSigDB_Hallmark_2020'],
                     #FDR=0.1)
```

Example 85 (unknown):
```unknown
MSigDB_Hallmark_2020
```

Example 86 (unknown):
```unknown
GO_Biological_Process_2023
```

Example 87 (unknown):
```unknown
KEGG_2021_Human
```

Example 88 (python):
```python
gsea_results["pct_genes_in_set"] = round((gsea_results["n_genes"] / len(INCREASING_highly_similar_common_genes)) * 100, 2)
gsea_results["neglog10"] = np.negative(np.log10(gsea_results['Adjusted P-value']))
gsea_results = gsea_results[gsea_results['Adjusted P-value']<0.2]
gsea_results
```

Example 89 (python):
```python
plt.figure(figsize=(2,3))
sb.scatterplot(data = gsea_results, x = "neglog10", y = "Term", hue = "neglog10", legend = False, palette = "viridis", size = "pct_genes_in_set", edgecolor='black')
plt.title("COMMON | HIGH SIMILARITY | DECREASING")
plt.ylabel("MSigDB Term")
plt.xlabel("-log(adjusted p)")
plt.savefig(fname="./figures/COMMON___HIGH SIMILARITY___DECREASING.pdf", format='pdf', bbox_inches='tight')
plt.show()
```

Example 90 (python):
```python
import matplotlib.pyplot as plt
import seaborn as sb

# Assuming gsea_results is your DataFrame and it's already loaded.

# Create the figure.
plt.figure(figsize=(2, 3))

# Unique terms for plotting on y-axis.
terms = gsea_results['Term'].unique()

# Mapping terms to y-values.
term_to_y = {term: i for i, term in enumerate(terms)}

# Plot each point individually.
for _, row in gsea_results.iterrows():
    edge_color = 'red' if row['neglog10'] > 2 else 'black'
    plt.scatter(x=row['neglog10'], y=term_to_y[row['Term']], s=row['pct_genes_in_set']*10,  # Adjust size as needed
                color=sb.color_palette("viridis", as_cmap=True)(row['neglog10']), edgecolor=edge_color)

# Set the y-ticks to the term names.
plt.yticks(ticks=list(term_to_y.values()), labels=list(term_to_y.keys()))

# Set title and labels.
plt.title("COMMON | HIGH SIMILARITY | DECREASING")
plt.ylabel("MSigDB Term")
plt.xlabel("-log(adjusted p)")

# Remove the legend created by seaborn
plt.legend([],[], frameon=False)

# Save and show the plot.
plt.savefig(fname="./figures/COMMON___HIGH SIMILARITY___DECREASING.pdf", format='pdf', bbox_inches='tight')
plt.show()
```

Example 91 (unknown):
```unknown
terms = ["Fatty Acid Metabolism","Myogenesis", "Oxidative Phosphorylation"]

term_genes_dict = {}

for term in terms:
    genes_series = gsea_results[gsea_results["Term"] == term]['Genes'].str.split(";")
    term_associated_genes_list = genes_series.sum()
    term_genes_dict[term] = term_associated_genes_list
term_genes_dict
```

Example 92 (unknown):
```unknown
{'Fatty Acid Metabolism': ['HSP90AA1',
  'FH',
  'MDH2',
  'ECH1',
  'RAP1GDS1',
  'ENO3',
  'PRDX6',
  'YWHAH'],
 'Myogenesis': ['EIF4A2',
  'MEF2C',
  'TNNC1',
  'TPM2',
  'MYL3',
  'PGAM2',
  'HSPB2',
  'TNNI1',
  'ENO3'],
 'Oxidative Phosphorylation': ['FH',
  'SLC25A3',
  'NDUFB8',
  'NDUFB7',
  'COX4I1',
  'NDUFB3',
  'ATP5MC3',
  'UQCR11',
  'UQCR10',
  'ETFB',
  'MRPL34',
  'COX6A1',
  'COX5A',
  'PRDX3',
  'LDHB',
  'ATP5F1B',
  'UQCRFS1',
  'POLR2F',
  'CYC1',
  'ATP5MF',
  'COX8A',
  'TIMM8B',
  'ATP5PF',
  'ATP5PD',
  'NDUFA6',
  'GPX4',
  'ATP5PB',
  'MDH2',
  'ECH1',
  'IDH2',
  'ATP5F1D',
  'ATP5F1E',
  'OXA1L',
  'NDUFS6',
  'ATP5PO',
  'CYCS',
  'SLC25A5',
  'ATP6V0C',
  'SLC25A11',
  'SLC25A6']}
```

Example 93 (unknown):
```unknown
GENE_SUBSET = term_genes_dict["Oxidative Phosphorylation"]
aligner.get_aggregate_alignment_for_subset(GENE_SUBSET)
```

Example 94 (unknown):
```unknown
Average Alignment:  IIDMMMMIDDMMMMMMM
```

Example 95 (python):
```python
import matplotlib.pyplot as plt
from contextlib import redirect_stdout
import io


for gene in term_genes_dict["Oxidative Phosphorylation"]:
    print(gene)
    # Redirect the output of the plot function to a StringIO object
    f = io.StringIO()
    with redirect_stdout(f):
        #VisualUtils.plotTimeSeries(gene, aligner, vs)
        #VisualUtils.plotTimeSeriesAlignment(gene, aligner)
        plotTimeSeriesAlignment_cmap(gene, aligner)

    
    # Now, the plot is created and can be saved
    plt.savefig(f"{gene}_time_series_alignment.png", format='png', bbox_inches='tight')
    plt.show()
```

Example 96 (python):
```python
chd_genes = pd.read_excel("chdgene_table.xlsx", header=1, index_col=0)
chd_genes
```

Example 97 (unknown):
```unknown
ASD_genes = chd_genes.loc[chd_genes["CHD classification"].str.contains("ASD"),:].index.tolist()
ASD_genes
```

Example 98 (unknown):
```unknown
['ABL1',
 'ACTC1',
 'ACVR1',
 'ACVR2B',
 'ANKRD11',
 'ARID1A',
 'ARID1B',
 'B3GAT3',
 'BCOR',
 'BMPR2',
 'BRAF',
 'CDK13',
 'CHD4',
 'CHD7',
 'CHST14',
 'CITED2',
 'CREBBP',
 'DNAH11',
 'DNAH5',
 'EFTUD2',
 'ELN',
 'EVC',
 'EVC2',
 'FGFR2',
 'FLNA',
 'FOXC1',
 'FOXP1',
 'GATA4',
 'GATA5',
 'GATA6',
 'GLI3',
 'GPC3',
 'HAAO',
 'HDAC8',
 'HNRNPK',
 'HRAS',
 'KANSL1',
 'KAT6A',
 'KAT6B',
 'KDM2B',
 'KDM6A',
 'KMT2A',
 'KMT2D',
 'KRAS',
 'KYNU',
 'LTBP2',
 'LZTR1',
 'MAP2K1',
 'MAP2K2',
 'MED12',
 'MEIS2',
 'MYBPC3',
 'MYH6',
 'MYH7',
 'NF1',
 'NIPBL',
 'NKX2-5',
 'NODAL',
 'NONO',
 'NOTCH2',
 'NPHP3',
 'NR2F2',
 'NSD1',
 'PBX1',
 'PIGV',
 'PITX2',
 'PTPN11',
 'RAB23',
 'RAF1',
 'RERE',
 'RIT1',
 'SALL1',
 'SETD5',
 'SHOC2',
 'SMAD2',
 'SMAD4',
 'SMARCA4',
 'SMARCB1',
 'SMARCE1',
 'SMC1A',
 'SMC3',
 'SON',
 1,
 'STRA6',
 'TAB2',
 'TBX20',
 'TBX5',
 'TGFBR1',
 'TGFBR2',
 'THOC6',
 'TLL1',
 'TMEM260',
 'TRAF7',
 'TXNL4A',
 'UBR1',
 'WASHC5',
 'ZEB2',
 'ZIC3']
```

Example 99 (unknown):
```unknown
all_CHD_genes = chd_genes.index.tolist()
```

Example 100 (unknown):
```unknown
df['CHD_associated'] = False
df.loc[df['Gene'].isin(all_CHD_genes),'CHD_associated'] = True
df
```

Example 101 (python):
```python
import seaborn as sns
import matplotlib.pyplot as plt

plt.figure(figsize=(6, 6))

markers = {
    "Increasing":"^",
    "Decreasing":"v",
    "Unchanging":"o"
}

#y_axis = 'opt_alignment_cost'
y_axis = 'alignment_similarity_percentage'

sns.scatterplot(data=df, x='l2fc', y=y_axis, hue='CHD_associated', edgecolor="none",
                style = 'trend', markers = markers,
                legend=False, alpha=0.8)

# Add a horizontal line at y=0.5
plt.axhline(0.5, color='gray', linestyle='--', linewidth = 0.5)

# Add text annotations for LaCM and RaCM
plt.text(0.1, 1.05, 'RaCM', transform=plt.gca().transAxes, ha='left', va='top', weight='semibold')
plt.text(0.9, 1.05, 'LaCM', transform=plt.gca().transAxes, ha='right', va='top', weight='semibold')

# Label specific genes
for gene in genes_of_interest:
    gene_data = df[df['Gene'] == gene]
    for _, row in gene_data.iterrows():
        plt.text(row['l2fc'], row[y_axis], gene, 
                 horizontalalignment='right', size='small', color='black', weight='regular', style='italic')

# Set labels
plt.xlabel("Log2 Fold Change")

if y_axis == "alignment_similarity_percentage":
    plt.ylabel("Alignment Similarity (%)")

if y_axis == "opt_alignment_cost":
    plt.ylabel("Alignment Cost (nits)")

# Save the plot
#plt.savefig("similarity_v_logFC.pdf", format='pdf', bbox_inches='tight')

# Show the plot
plt.show()
```

Example 102 (python):
```python
#sns.stripplot(data = df, x = 'CHD_associated', y = 'alignment_similarity_percentage', hue = 'CHD_associated', legend=False, jitter=0.05, alpha = 0.1)
sns.violinplot(data = df, x = 'trajectory', y = 'alignment_similarity_percentage', hue = 'CHD_associated', legend=False, alpha = 0.2, bw_adjust=2)
```

Example 103 (unknown):
```unknown
<Axes: xlabel='trajectory', ylabel='alignment_similarity_percentage'>
```

Example 104 (python):
```python
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
from scipy import stats

# Your DataFrame manipulation and t-test code here
df['alignment_similarity_percentage'] = pd.to_numeric(df['alignment_similarity_percentage'], errors='coerce')

# Define the trajectories to be tested
trajectories = ['Common', 'LaCM', 'RaCM']
p_values = {}

# Perform t-tests and store p-values in a dictionary with trajectories as keys
for trajectory in trajectories:
    group1 = df[(df['trajectory'] == trajectory) & (df['CHD_associated'] == True)]['alignment_similarity_percentage'].dropna()
    group2 = df[(df['trajectory'] == trajectory) & (df['CHD_associated'] == False)]['alignment_similarity_percentage'].dropna()
    t_stat, p_value = stats.ttest_ind(group1, group2)
    p_values[trajectory] = p_value

# Plotting the violin plot
plt.figure(figsize=(10,6))
#ax = sns.violinplot(data=df, x='trajectory', y='alignment_similarity_percentage', hue='CHD_associated', alpha=0.2, bw_adjust=2, order=trajectories)
ax = sns.boxplot(data=df, x='trajectory', y='alignment_similarity_percentage', hue='CHD_associated', order=trajectories)

# Remove top and right spines
ax.spines['top'].set_visible(False)
ax.spines['right'].set_visible(False)

# Function to add significance text
def add_significance_text(x_pos, y, text, color='k'):
    plt.text(x_pos, y, text, ha='center', va='bottom', color=color)

# Adding the statistical annotations
y_max = df['alignment_similarity_percentage'].max()
h = 2  # Offset for the annotation

# Annotate each group with its corresponding p-value
for i, trajectory in enumerate(trajectories):
    p_value = p_values[trajectory]
    p_text = f"p={p_value:.2e}" if p_value < 0.001 else f"p={p_value:.3f}"
    add_significance_text(i, y_max + h, p_text)


plt.show()
```

Example 105 (unknown):
```unknown
GO_plot(gene_list = topDEgenes[topDEgenes['trajectory']=="LaCM"]['Gene'].tolist(), n_genes=3, display_table=True)
```

Example 106 (unknown):
```unknown
MSigDB_Hallmark_2020
```

Example 107 (unknown):
```unknown
KEGG_2021_Human
```

Example 108 (unknown):
```unknown
GO_Biological_Process_2023
```

Example 109 (unknown):
```unknown
Reactome_2022
```

Example 110 (unknown):
```unknown
# Running experiment to determine the distance threshold with a good trade-off
df = ClusterUtils.run_clustering(aligner, metric='levenshtein', experiment_mode=True)
```

Example 111 (unknown):
```unknown
compute distance matrix
using levenshtein distance metric
```

Example 112 (unknown):
```unknown
78%|██████████████████████████████████████████████████████████████████████████████████▍                       | 77/99 [00:22<00:06,  3.39it/s]
```

Example 113 (unknown):
```unknown
df.sort_values('Distance threshold', ascending=False)
```

Example 114 (unknown):
```unknown
aligner.show_ordered_alignments()
```

Example 115 (unknown):
```unknown
Gene          Alignment
------------  ----------------------------
PITX2         IIIDIDIDIDIIIIIIIIDDDDDDDDDD
DCBLD2        IIIDIDIDIDIIIIIIIIDDDDDDDDDD
PANCR         IIIDIDIDIDIIIIIIIIDDDDDDDDDD
PCDH7         IIIDIDIDIDIIIIIIIIDDDDDDDDDD
NAV3          IIIDIDIDIDIIIIIIIIDDDDDDDDDD
VASH2         IIIDIDIDIDIIIIIIIIDDDDDDDDDD
NTM           IIIDIDIDIDIIIIIIIIDDDDDDDDDD
ROR1          IIIDIDIDIDIIIIIIIIDDDDDDDDDD
GLCCI1        MMMMMMMMMMMMMM
MORF4L1       MMMMMMMMMMMMMM
PJA2          MMMMMMMMMMMMMM
RO60          MMMMMMMMMMMMMM
NAP1L1        MMMMMMMMMMMMMM
PPP4C         MMMMMMMMMMMMMM
CWC27         MMMMMMMMMMMMMM
MAPKAPK2      MMMMMMMMMMMMMM
MARCH6        MMMMMMMMMMMMMM
AL008730.1    MMMMMMMMMMMVVVDDD
TRIM24        MMMMMMMMMMMMMM
CITED2        MMMMMMMMMMMMMM
CT69          MMMMMMMMMMMMMM
ERBIN         MMMMMMMMMMMMMM
NIBAN1        MMMMMMMMMMMMMM
GATA4         MMMMMMMMMMMMMM
LDHA          MMMMMMMMMMMMMM
GAB1          MMMMMMMMMMMMMM
IDI2-AS1      MMMMMMMMMMMMMM
EXT1          MMMMMMMMMMMMMM
CCDC144A      MMMMMMMMMMMMMM
RPS5          MMMMMMMMMMMMMM
WDR37         MMMMMMMMMMMMMM
YWHAE         MMMMMMMMMMMMMM
FOXJ3         MMMMMMMMMMMMMM
AFF4          MMMMMMMMMMMMMM
PITPNM2       MMMMMMMMMMMMMM
SEPTIN7       MMMMMMMMMMMMMM
CNN1          MMMMMMMMMMMMMM
LIMS2         MMMMMMMMMMMMMM
ANKRD12       MMMMMMMMMMMMMM
ZNF106        MMMMMMMMMMMMMM
LINC01608     MMMMMMMVVVVVVVDDDDDDD
UBE4B         MMMMMMMMMMMMMM
USP24         MMMMMMMMMMMMMM
CSNK1A1       MMMMMMMMMMMMMM
LINC01913     MMMMMMMMMVVVVVDDDDD
AC002310.1    MMMMMMMMVVVVVVDDDDDD
XRN1          MMMMMMMMMMMMMM
ZDHHC21       MMMMMMMMMMMMMM
CD81          MMMMMMMMMMMMMM
FABP2         MMMMMMMMMWWWWWIIIII
UBR5          MMMMMMMMMMMMMM
DMAC1         MMMMMMMMMMMMMM
KANK2         MMMMMMMMMMMMMM
ZNF780B       MMMMMMMMMMMMMM
TPT1-AS1      MMMMMMMMMMMMMM
XRCC5         MMMMMMMMMMMMMM
FSTL1         MMMMMMMMMMMMMM
SRP9          MMMMMMMMMMMMMM
SNHG32        MMMMMMMMMMMMMM
LEPROT        MMMMMMMMMMMMMM
MGAT4C        MMMMMMMMMMMMMM
ABCC5         MMMMMMMMMMMMMM
NNT-AS1       MMMMMMMMMMMMMM
ACKR3         MMMMMMMMMWWWIIIMM
BMP10         MMMMWWWWWDDMMIIDIIIIII
LGALSL-DT     MMMMMMMMMIDMMMM
ARHGEF26-AS1  MMMMMMMMMIIDDMMM
TOP1MT        MMMMMMMMMIDMMMM
AC046158.1    MMMMMMMMMIDMMMM
PRX           MMMMMMMMMIDMMMM
HSF2          MMVVVVVVDDDDDDMMMMMM
MTRNR2L1      MMMMMMMMIDMMMMM
KCNK1         MMMMMMMMIIMMMDIDD
MARK2         MMVVVVVVDDDDDDMMMMMM
INTS5         MMMMMMMMIDMMMMM
PRDM5         MMVVVVVDDDIDDDMMMMMM
CALD1         MVVVVVVDDDDDDMMMMMMM
KIAA1324L     MMVVVVVDDDDDMMMMMMM
AL009177.1    MMMMWWWIIIMMMMMMM
ESRRB         MMMVVVVDDDDMMMMMMM
ST3GAL1       MMVVVVVDDDDDDDMMMMIIID
SP100         MMMMMMMIDMMMMMM
ALG10B        MMMMMMMIIDDMMMMM
AL137003.2    MMVVVVVDDDDDMMMMMMM
PKIG          MMWWWWWIIIIIMMMMMMM
AL160191.1    MMMMMMMIDIDMMMMM
STT3B         MMVVVVDIDDDDMMMMMMM
AKR1B1        MVVVVVDIDIDDDDDMMMMMM
YWHAH         MMVVVVDDDIDDMMMMMMM
HBG1          MMVVVVDDDDMMMMMMMM
FH            MMMMMMIDMMMMMMM
PPA1          MVVVVVDIDIDDDDDMMMMMM
TARBP1        MMVVVVDDDDMMMMMMMM
PSMB4         MMMMMIDIDMMMMMMM
HMGN2         MMMMMIDIDMMMMMMM
ETFB          MMMMMIDIDMMMMMMM
FILIP1        MMMMMIDMMMMMMMM
DHX32         MMMMMIDMMMMMMMM
SLC25A11      MMMMMIDIDMMMMMMM
PTTG1IP       MMMMMIDMMMMMMMM
ZNHIT1        MMMMMIDIDMMMMMMM
PEBP1         MMMMMIDIDMMMMMMM
NDUFB8        MMMMMIDIDMMMMMMM
AOPEP         MMMMMIDMMMMMMMM
CDC42         MMMMMIIDDMMMMMMM
MZT2B         MMMMMIDMMMMMMMM
ZFP36L1       MMMMMIIDDMMMMMMM
H2AFV         MMMMMIDIDMMMMMMM
VPS8          MMMMMIDIDIDMMMMMM
GPAT3         MMMMMIDIDMMMMMMM
EIF4A2        MMMMMIIDDMMMMMMM
GTF2IRD1      MMMMMIDIDMMMMMMM
EIF1          MMMMMIDIDMMMMMMM
HIGD2A        MMMMMIDIDMMMMMMM
CCSER2        MMMMMIDMMMMMMMM
POMP          MMMMMIDIDMMMMMMM
ATP5PO        MMMMMIDIDMMMMMMM
DUSP16        MMVVVDDDMMMMMMMMM
RACK1         MMMMMIIDDMMMMMMM
UGGT2         MMMMMIDMMMMMMMM
CASZ1         MMMMMIDMMMMMMMM
ANKH          MMMMMIDMMMMMMMM
ADI1          MMMMMIDMMMMMMMM
NDUFB2        MMMMMIIDDMMMMMMM
DYNLT1        MMMMMIDMMMMMMMM
OAZ1          MMMMMIIDDIDMMMMMM
SYNE1         MMMMMIDIDMMMMMMM
MT-ND2        MMMMMIDMMMMMMMM
TUT4          MMMMMIDMMMMMMMM
MARC2         MMMMMIDMMMMMMMM
PSPC1         MMMMMIDIDMMMMMMM
PRDX5         MMMMMIDIDMMMMMMM
HSPA6         MMMMIDMMMMMMMMM
KDM4B         MMMMIDMMMMMMMMM
KCNN2         MMMMDDDMMMMMMMIII
ATP5MC1       MMMMIDIDIDIDMMMMMM
KDELR1        MMMMDDMMMMMMMMVV
HBZ           MMMMIIIIIMMMMMWWWWW
COP1          MMMMIIDDMMMMMMMM
ZNF91         MMMMDDMMMMMMMMVV
RPS14         MMMMIDMMMMMMMMM
ST7           MMMMIDIDIDMMMMMMM
AL136171.2    MMMMIDMMMMMMMMM
BBX           MMMMIDMMMMMMMMM
BSG           MMMMIIDDMMMMMMMM
YWHAQ         MMMMIDMMMMMMMMM
USH1C         MMMMDDDMMMMMMMIII
MTUS2         MMMMIDMMMMMMMMM
ALS2          MMMMIDIDMMMMMMMM
MYO18A        MMMMIDMMMMMMMMM
HSPD1         MMMMIIDDIDMMMMMMM
COX4I1        MMMMIIDDIDMMMMMMM
NRP1          MMMMIIDIDDMMMMMMM
HSF1          MMMMIDIDIDMMMMMMM
PPP2R2D       MMMMIDMMMMMMMMM
EIF3K         MMMMIIDDIDIDMMMMMM
EIF4G2        MMMMDDMMMMMMMMVV
PARD3B        MMMMIDMMMMMMMMM
RAN           MMMMDDMMMMMMMMVV
SUN1          MMMMDDMMMMMMMMVV
ATP5F1D       MMMMIDIDIDMMMMMMM
EIF3E         MMMMDDMMMMMMMMVV
PPARA         MMMMIDIDMMMMMMMM
PDZRN3        MMMMIDIDMMMMMMMM
C17orf49      MMMMDDMMMMMMMMVV
FNIP1         MMMMDDMMMMMMMMVV
BMP5          MMMMIDIDMMMMMMMM
NDUFB4        MMMMIDIDIDMMMMMMM
FXR1          MMMMIDIDMMMMMMMM
TRMT9B        MMMMIDIDMMMMMMMM
PSMB2         MMMMIIDIDDMMMMMMM
NUCKS1        MMMMIDMMMMMMMMM
CUX1          MMMMIDIDMMMMMMMM
PRDX3         MMMMDDMMMMMMMMVV
ATP5F1C       MMMMIDIDIDMMMMMMM
SELENOW       MMMMDDMMMMMMMMVV
RPS27A        MMMMDDMMMMMMMMII
NEAT1         MMMMDDMMMMMMIIMM
COPA          MMMMIIDDMMMMMMMM
TOM1L2        MMMMIDIDMMMMMMMM
SPAG7         MMMMIIDDIDIDMMMMMM
ECH1          MMMMIDMMMMMMMMM
CISD1         MMMMIDIDIDMMMMMMM
GPC6          MMMMIDIDIDMMMMMMM
TNK2          MMMMIDIDMMMMMMMM
HINT1         MMMMDMMMMMMMMMI
EIF4H         MMMMDDMMMMMMMMVV
EIF4A1        MMMMIDMMMMMMMMM
FAM171A1      MMMMIDIDMMMMMMMM
ZNF638        MMMMIIDDMMMMMMMM
STARD4-AS1    MMMMIDIDMMMMMMMM
PDHX          MMMMIDIDIDMMMMMMM
MICAL3        MMMMIIDDMMMMMMMM
NR4A3         MMMMIDMMMMMMMMM
AC007376.2    MMMMDIDIDDMMMMMMII
RPL15         MMMMIDMMMMMMMMM
PLEKHA2       MMMMIIDDMMMMMMMM
PRKD1         MMMMIDMMMMMMMMM
UQCRFS1       MMMMIIDDIDIDMMMMMM
NR3C1         MMMMDDMMMMMMMMVV
LDHB          MMMMIDIDIDIDMMMMMM
AP2M1         MMMMIDIDIDMMMMMMM
COX6A1        MMMMDDMMMMMMMMVV
PARK7         MMMMIIDDIDMMMMMMM
SSBP3         MMMMIDIDMMMMMMMM
CYC1          MMMMDDDMMMMMMMVVV
PHB           MMMMDDMMMMMMMMVV
SLMAP         MMMMDDMMMMMMMMVV
MAPRE1        MMMMDIDMMMMMMIMM
HBA1          MMMMIDIDMMMMMMMM
SORBS1        MMMMDMMMMMMMIMM
PSMB6         MMMMIDIDIDMMMMMMM
ATP5F1A       MMMMIIIDDDIDMMMMMM
MFSD4B        MMMMIDIDMMMMMMMM
PHF14         MMMMIDMMMMMMMMM
SMAP1         MMMMIIDDIDMMMMMMM
ECI1          MMMMIIDDIDMMMMMMM
PFN1          MMMMDDMMMMMMMMVV
NFE2L1        MMMMIIDDIDMMMMMMM
EIF1AX        MMMMIIDDMMMMMMMM
CYCS          MMMMDDMMMMMMMMII
AFAP1L1       MMMMIDIDMMMMMMMM
FOXK1         MMMMIDIDMMMMMMMM
SSPN          MMMMIIDDMMMMMMMM
RBMS1         MMMMDDMMMMMMMMVV
A4GALT        MMMMIIDDMMMMMMMM
IQSEC1        MMMMIDMMMMMMMMM
CLEC2D        MMMMDMMMMMMMIMM
VPS53         MMMMIDIDMMMMMMMM
MT-ND4L       MMMMDIDMMMMMMIMM
MT-ND3        MMMMDIDIDMMMMMMMI
CALM2         MMMMDDMMMMMMMMVV
RPL17         MMMMDDMMMMMMMMII
PPIA          MMMMDDMMMMMMMMII
PXDNL         MMMMDDMMMMMMMMII
PHF3          MMMMIDMMMMMMMMM
TBX5          MMMMDDMMMMMMMMVV
DACH1         MMMMIDMMMMMMMMM
TRAK1         MMMMIDIDIDMMMMMMM
LEMD3         MMMMDDMMMMMMMMVV
C7orf50       MMMMIDIDIDMMMMMMM
DNM2          MMMMIIDDMMMMMMMM
PSMD8         MMMMIDMMMMMMMMM
ACOX3         MMMMIDIDMMMMMMMM
DECR1         MMMMIIDDIDMMMMMMM
SET           MMMMIDMMMMMMMMM
GEM           MMMMDMMMMMMMIMM
MAP4K4        MMMMIDIDIDMMMMMMM
CSRP2         MMMMDDMMMMMMMMVV
AL139317.5    MMMMIIDDMMMMMMMM
CCNI          MMMMIDIDIDMMMMMMM
KCTD1         MMMMIDIDIDIDMMMMMM
NME4          MMMMIDIDMMMMMMMM
DDHD1         MMMMDDMMMMMMMMVV
CPSF6         MMMMIDIDMMMMMMMM
MTR           MMMMIIDDMMMMMMMM
TNRC18        MMMMIDMMMMMMMMM
ADCY5         MMMMIDIDMMMMMMMM
MRPL34        MMMMIDMMMMMMMMM
RAMP1         MMMMDDMMMMMMMMII
TMEM161B-AS1  MMMMIIDDMMMMMMMM
DARS          MMMDDMMMMMMMMMVV
DYNC2H1       MMMIIIDDDMMMMMMMM
CLCN6         MMMIDIDMMMMMMMMM
ANXA3         MMMDDIDIDMMMMMMMVV
PGRMC1        MMMIDIDIDIDMMMMMMM
VCP           MMMDDIDMMMMMMMMVV
EPN2          MMMDDMMMMMMMMMII
LRPPRC        MMMIDIDIDMMMMMMMM
ERH           MMMDDDMMMMMMMMVVV
ANKRD18A      MMMIIDDIDMMMMMMMM
BMPR1A        MMMIDIDIDMMMMMMMM
RTRAF         MMMDDMMMMMMMMMVV
ATG2B         MMMIIIDDDMMMMMMMM
ETNK1         MMMIIDDIDMMMMMMMM
AHNAK         MMMDDMMMMMMMMMVV
PRDX2         MMMDDDMMMMMMMMVVV
GPX4          MMMDDDMMMMMMMMVVV
RANBP17       MMMIDIDIDIDMMMMMMM
ACTN1         MMMIDIDMMMMMMMMM
COLQ          MMMIDIDIDMMMMMMMM
RBM8A         MMMDDDMMMMMMMMVVV
EPS15L1       MMMDDMMMMMMMMMVV
XIST          MMMIDIDIDIDMMMMMMM
LMO7          MMMIDIDMMMMMMMMM
CAP2          MMMIDIDIDMMMMMMMM
VCL           MMMIDIDIDMMMMMMMM
CADPS         MMMDDDMIIIMMMMMMM
SKP1          MMMDDDMMMMMMMMVVV
PKM           MMMIDIDIDIDMMMMMMM
HK2           MMMIDDIDMIMMMMMMM
GADD45G       MMMIIDDIDIDMMMMMMM
SCAF11        MMMIIDDMMMMMMMMM
RASD1         MMMIIDDIDMMMMMMMM
ELOVL5        MMMIIDDIDIDMMMMMMM
SLC25A3       MMMDDDMMMMMMMMVVV
GOLGA4        MMMIIDDIDIDMMMMMMM
PRKAR1A       MMMDMMMMMMMMIMM
HSPA5         MMMIDIDIDMMMMMMMM
FRAS1         MMMDDDMMMMMMMMVVV
PPP1CB        MMMIDMMMMMMMMMM
SNRPG         MMMIDIDIDIDMMMMMMM
NCL           MMMIDIDMMMMMMMMM
DANCR         MMMIDIDIDMMMMMMMM
C9orf72       MMMDDDIDMMMMMMMVVV
UBE2O         MMMDIDMMMMMMMIMM
PTPRD         MMMIDIDIDMMMMMMMM
AC087286.2    MMMIIDDIDIDMMMMMMM
H3F3A         MMMDDDMMMMMMMMIII
H3F3B         MMMDDDMMMMMMMMVVV
FAM110B       MMMDIDIDMMMMMMIMM
ARL17B        MMMIDDMMMMMMMIMM
GNAQ          MMMIDIDIDMMMMMMMM
BTG1          MMMIDIDIDIDMMMMMMM
DYRK2         MMMDDMMMMMMMMMVV
DNAJA1        MMMIIDDIDIDMMMMMMM
RBM24         MMMDDMMIIMMMMMMM
LINC01278     MMMDDMMMMMMMMMVV
OGA           MMMDDDMMMMMMMMVVV
TCEAL9        MMMIDIDIDMMMMMMMM
RPLP2         MMMIDIDMMMMMMMMM
PTPN21        MMMDDDMMMMMMMMVVV
UQCRC1        MMMDDDMMMMMMMMVVV
CNOT11        MMMDDDMMMMMMMMVVV
PPIB          MMMDDDMMMMMMMMVVV
NAV2-AS2      MMMIDIDIDIDMMMMMMM
WLS           MMMDDMMIIMMMMMMM
PSD3          MMMDMMMMMMMMIMM
JPH2          MMMDDDMIIIMMMMMMM
PTMS          MMMDDDMMMMMMMMVVV
NOP10         MMMIIDDIDIDMMMMMMM
ASAH1         MMMIDIDIDMMMMMMMM
DCUN1D2       MMMDMMMMMMMMIMM
NIN           MMMIDIDIDIDMMMMMMM
TMEM135       MMMIIDDIDMMMMMMMM
PSMD1         MMMDDDMMMMMMMMVVV
COL2A1        MMMDDDMMMMMMMMVVV
FBH1          MMMIIDDIDMMMMMMMM
CFL1          MMMDDDMMMMMMMMVVV
PSMB1         MMMDDDMMMMMMMMVVV
RPL18         MMMIIDDIDIDMMMMMMM
CNBP          MMMDDMMMMMMMMMVV
RAB20         MMMIDIDIDIDMMMMMMM
SPCS1         MMMIDIDIDIDMMMMMMM
PLEKHJ1       MMMDIDDDMMMMMMMVVV
LINC01505     MMMDDDMMMMMMMMVVV
TAGLN         MMMIIDDIDMMMMMMMM
HSPE1         MMMIDDIDMMMMMMMMI
LMBRD2        MMMDDDMMMMMMMMVVV
SMARCE1       MMMIIDDIDIDMMMMMMM
IRF2BP2       MMMIDIDMMMMMMMMM
CERS6         MMMIIDDMMMMMMMMM
DDT           MMMDDDMMMMMMMMVVV
PHACTR3       MMMIIMMMMMMMMDDM
SMTNL2        MMMDDDMMMMMMMMVVV
SLC25A4       MMMDIDDMMMMMMMMII
PSMD14        MMMDDMMMMMMMMMVV
LMAN2         MMMIDIDIDIDMMMMMMM
FAM162A       MMMIIDDMMMMMMMMM
FHIT          MMMDDMMMMMMMMMVV
DENND4A       MMMIDIDIDIDMMMMMMM
IGF1R         MMMDDDMMIIIMMMMMM
PSMB5         MMMDDDMMMMMMMMVVV
SUCO          MMMIDDMMMMMMMIMM
HNRNPK        MMMDDMMMMMMMMMVV
CKAP4         MMMDDDMMMMMMMMVVV
MDH1          MMMDDDMMMMMMMMIII
SLC8A1-AS1    MMMIDIDIDMMMMMMMM
NR2F2         MMMDDIDMMMMMMMMVV
MPP7          MMMIDIDIDMMMMMMMM
SLC25A27      MMMIIDDIDMMMMMMMM
MPDZ          MMMDDMMMMMMMMMVV
SPIRE1        MMMIDIDIDMMMMMMMM
ANKRD1        MMMIDDMMMMMMMMMI
ATP5MC3       MMMDDDMMMMMMMMVVV
SPPL3         MMMIIDDIDMMMMMMMM
ST6GALNAC3    MMIDDDDMMMMMMMMVVV
NES           MMDDDMMMMMMMMMVVV
VPS37B        MMIIMMDIDIDDMMMMMM
AKAP6         MMIIIMMDDDMMMMMMM
PPP1R1A       MMIIIDDIDIDIDDMMMMMM
AFF2          MMDDDIDMMIDIIIMMMMM
KCNIP4        MMIDDDIDIDMMMMMMMVV
STK38         MMIDIDIDIDMMMMMMMM
GPRC5B        MMDDDDIDMMMIIIIMMMM
VDAC1         MMDDDMMIIIMMMMMMM
CLDN6         MMIIMMMMMMDDMMMM
MDGA2         MMIIIDIDIDIDDDMMMMMM
NPM1          MMIMMIDDIDMMMMMMM
NPAS2         MMIDIDIDIDMMMMMMMM
IQGAP1        MMIDDMMMMMMMMIMM
PLXNC1        MMIIIIIDIDIDIIDDDDDMMMD
ACTN4         MMIDDDIDIDMMMMMMMVV
BRINP3        MMIIMIDIDIDIIIDDDDMMMD
USP6NL        MMIDIDIDIDMMMMMMMM
CAMKMT        MMDDMMMMMMMMMMVV
NPPB          MMIMMIDDMMMMMMMM
MT-ND4        MMIIDDIDIDIDMMMMMMM
PCDH9         MMDDDMMMMMIIIMMMM
FGF12         MMIDDDDMMMMMMMMVVV
NEDD9         MMDDDIDIDDMMMMMVVVVVD
PSAP          MMDDDMMMMMMMMMVVV
ARHGAP10      MMDDDDMMMMIIIIMMMM
GAN           MMIIDDIDIDIDMMMMMMM
DTNA          MMIIDDIDIDIDMMMMMMM
SYNDIG1       MMIDDDDIDMMMMMMVVVVD
JARID2        MMIIDDIDMMMMMMMMM
HNRNPA1       MMIDDDDMMMMMMMMVVV
NCKAP5        MMDDMMMMMMMMMMVV
MMP23B        MMIIDMMMMDMMMMMM
TSTD3         MMIDIDIDIIDDMMMMMMM
MYH7          MMDIDIDMMMMMIMMMM
ABTB2         MMIDIDIDIDIDMMMMMMM
NUAK1         MMIIIIDIDIDIDDDDMMMMM
EIF3F         MMIMMDMMMMMMMMM
ARHGAP35      MMDDDMMMMMMMMMVVV
CREB3L2       MMDDMMMMMMMMMMVV
VDAC2         MMDDDMMMMMMMMMVVV
RTN4          MMIDDDIDMMMMMMMMVV
GNG7          MMIDDDIDMMMMMMMMVV
RAC1          MMDDDMMMMMMMMMVVV
AKAP13        MMIDDMMIDIDIIDMMMMM
FDFT1         MMIIDIDIDIDDMMMMMMM
STAT4         MMIIDDIDMMMMMMMMM
MLLT3         MMIMMDIDMMMMMMMM
PDE8B         MMIMMDIDIDMMMMMMM
SMG1          MMIIMMDDMMMMMMMM
TUSC3         MMIMMDMMMMMMMMM
FGF13         MMIIIMDDDMMMMMMMM
PWRN1         MMDMDDMMMMMMMMVVV
RIC8B         MMIIDDIDMMMMMMMMM
SHOX2         MMIDDMMMMMMMMMMI
ALDOC         MMIIDDIDIDIDMMMMMMM
TENM2         MMDDMMMMMMMMMMVV
KLHL24        MMIDIDIDIDMMMMMMMM
CRTC3         MMIIMMDDMMMMMMMM
MSMO1         MMIIIIMDDDDMMMMMMM
ARFGEF2       MMDDIDMMMMMMMMMVV
AC092640.1    MMIDIDIDMMMMMMMMM
RPL41         MMIDDDMMMMMMMMMII
PLCB4         MMIDIDIDIDIDMMMMMMID
BCAR3         MMIIDDIDIDIDMMMMMMM
RPL36A        MMIDIDIDMMMMMMMMM
PGK1          MMIDDDIDMMMMMMMMVV
RNGTT         MMDDIDMMMMMMMMMVV
SQSTM1        MIMMMMDMMMMMMMM
RPL10A        MIIMMMDDMMMMMMMM
CNTN4         MIIDDIDIDMMMMMMMMM
PIP5K1B       MDDDMMMMMMMMMMVVV
PDK1          MDMMMMMMMMMMIMM
MCRIP1        MIIMMMMDDMMMMMMM
LSAMP         MDMMDMMMMMMMMMVV
NFASC         MDIVVVVVDDDDDMMMMMMM
KSR1          MIIIIDIDIDIDDDDMMMMMM
A2M           MIIIDIDIDIIDDDDMMMMMM
EIF2AK3       MIMMDDIDMMMMMMIMM
ITSN2         MDDDDDMIIIIDIIMMMMMM
ADAM23        MDDDDDMMIIIIIMMMMMM
RPL26         MIIIMMMDDDMMMMMMM
NR2F1         MIIIIDDDIDIDDMMMMMMM
ITM2C         MIIIMMDDDMMMMMMMM
UQCRH         MIIMMMMDDMMMMMMM
ANO5          MIIIDIDIDIIDDDDMMMMMM
PRKG1         MIDDDDMMIIIMMMMMMM
PIM1          MIMMMMDMMMMMMMM
RPL28         MIMMMDIDMMMMMMMM
TUBE1         MIDIDIDIDIDMMMMMMMM
TBCA          MIMMMMDIDMMMMMMM
CCND2         MIMMMMDMMMMMMMM
AFG3L2        MDDDMMMMMMMMMMVVV
NDUFB5        MIMMMMDIDMMMMMMM
NOP53         MIMMMDMMMMMMMMM
PLEKHG4B      MDDMDIDMMMMMMMMVVV
RPS29         MIIMMMDDMMMMMMMM
CCT2          MIIMMMMDDMMMMMMM
MYO18B        MIMMMDIDMMMMMMMM
TPP2          MIIIIMMDDDDMMMMMMM
DOCK9         MIIIMMMDDDMMMMMMM
AC006460.2    MDMMMIDIMMMMMMMM
MAP4          MIIMMMDDMMMMMMMM
ARL6IP5       MIIMMMDIDDMMMMMMM
HMBOX1        MDDDDDMMIIIIIMMMMMM
RPL14         MIIMMDDMMMMMMMMM
ATF3          MIIDDIDMMMMMMMMMM
SLC47A1       MDMMMMMVVVDDMMMMM
TET2          MDDDMMMMMMMMMMVVV
NDUFC1        MIIMMMDIDDMMMMMMM
NRG3          MIIIIDIDIDDDDMMMMMMM
RPS19         MIIMMMDIDDMMMMMMM
METTL26       MIMMDIDIDIDMMMMMMM
PCBP1         MIIMMMDDMMMMMMMM
RELN          MDDDMMIIIMMMMMMMM
H2AFY         MIIIIDIDIIDDDDDMMMMMM
MAF           MIIIIIIDIDIDDDDDDMMMMM
GRIA1         MIIIIIMMMMMMMMDDDDD
CARMIL1       MIDDIDMMMMMMMMIMM
KCNMB2-AS1    MIIIIDIDIDIIIIIIIDDDDDDDDDD
GABRB2        MIMMDIDIDMMMMMMMM
HEY1          MIIIIMMMMMMMMMWWWW
AC092353.2    MIIDDDIDIDMMMMMMIMM
CCNL1         MDMMIDIDIMMMMMMMM
UQCC2         MDDMMMMMMMMMMMVV
NEDD8         MDMMMMMIMMMMMMM
EEF1D         MIIMMMMDDMMMMMMM
CCND1         MIDDDMMMMMMMMMMVV
PNPLA7        MIDDDIDMMMMMMMMMII
DKK3          MIIIDIDIDIDDDMMMMMMM
RPL31         MIIMMMMDDMMMMMMM
FGFR1         MDMMMIMMMMMMMMM
TXNIP         MDMMMMMMMMMMIMM
TRPM3         MIIIIIDIDIDIDDDDDMMMMM
UBA52         MIIMMMDIDDMMMMMMM
RPE           MIIIIDIDIDIDDDDMMMMMM
AFF1          MDMMMMMIMMMMMMM
PNKD          MIIMMMMDDMMMMMMM
SIK2          MDDMMIDIDIIMMMMMMM
AL356737.2    MIIIIIMMDDDDDMMMMMM
GNAL          MIIIDIDIDIDIIDDDDMMMMM
MCM6          MIIMMMMMMDDMMMMM
BVES          MDMDIDDMMIIIMMMMMM
CDH12         MIIDDIDIDIDDDMMMMMMVV
RPL27         MIIIMMMDDDMMMMMMM
EEF1G         MIIMMMDIDIDDMMMMMM
SNU13         MIIIMMDDDMMMMMMMM
WDR62         DMMMMMMMMMMVVVVDDD
Z94721.1      DMMMMMMVVVVVVVIDWWWWWW
AL022097.1    DMMMMMMVVVVVVVIDWWWWWW
SLC5A12       DMMMMMMMMMMMMIID
TRPC4         DMMMMMMMMMMMMIDI
BMP2          IMMMMMMMMWWWIIMMM
PTGDS         DMMMMMMMMMIMMMM
SEMA5A        IMMVVVVVVDDDDDDDMMMMM
DLGAP1        IMWWWWWWWIIIIIIIIIIIIDDDDDD
ST3GAL6       IMMVVVVVDDDDDDMMMMMM
RFX2          DMMMMVVVDDMMMMMMM
MYCN          IMMMMMMMIDWWWWIIIMM
SPRY4-AS1     IMMVVVVDDDDDMMMMMMM
WASHC2A       IMMVVVVDIDDDDDMMMMMM
MAP3K14       IMMVVVVDDDDDMMMMMMM
MTRNR2L12     IMMMMMDDMMMMMIMM
HECTD4        IMMMMMDIDMMMMMMM
CFLAR         IMMMMMDMMMMMMMM
USP22         IMMMMMDIDMMMMMMM
CTNND1        IMMMMMDMMMMMMMM
CEP95         IMMMMMDMMMMMMMM
PIP4K2A       IMMMMMDMMMMMMMM
GRB10         IMMMMMDMMMMMMMM
MRPS24        IMMMMMIDDMMMMMMM
PPP6R3        IMMMMMDMMMMMMMM
ATP5PB        IMMMMMDIDMMMMMMM
UTY           DMMMMIIIVVVVVVVDDDDDDDDD
PSMC5         IMMMMDIDMMMMMMMM
PSMA7         IMMMMDMMMMMMMMM
SNX24         IMMMMDMMMMMMMMM
C1orf122      IMMMMDMMMMMMMMM
MLEC          IMMMMDMMMMMMMMM
SH3PXD2A      IMMMMDMMMMMMMMM
EIF5A         IMMMMDMMMMMMMMM
TMED9         IMMMMDMMMMMMMMM
PGM1          IMMMMDIDMMMMMMMM
SH3BGR        IMMMMDIDMMMMMMMM
PPM1K         DMMMMIDIDIIDMMMMMM
SYNPO         IMMMMDIDMMMMMMMM
EML4          IMMMMDIDMMMMMMMM
MAZ           IMMMMDMMMMMMMMM
PRKAR2B       IMMMMDIDIDMMMMMMM
CPNE5         IMMMMDIDMMMMMMMM
MYO1E         IMMMMDIDMMMMMMMM
PRMT1         DMMMMIIDIDMMMMMMM
HBG2          IMMMMDMMMMMMMMM
RILPL1        IMMMMDIDMMMMMMMM
ADCY6         IMMMMDIDMMMMMMMM
ELOC          IMMMMDMMMMMMMMM
PDZD2         IMMMMDIDMMMMMMMM
PDLIM3        IMMMMDIDMMMMMMMM
HSP90AB1      IMMMMDIDMMMMMMMM
MEIS2         IMMMMDIDIDMMMMMMM
PMVK          IMMMMDMMMMMMMMM
DIP2C         IMMMMDIDMMMMMMMM
TRPM7         IMMMMDIDMMMMMMMM
SCN5A         IMMMMDIDMMMMMMMM
BZW2          DMMMMIIDIDMMMMMMM
TRIR          IMMMMIDIDDMMMMMMM
MT-ATP8       IMMMMDDMMMMMMIMM
GJA1          DMMMMIIDMMMMMMMM
MARK3         IMMMMDIDMMMMMMMM
LINC00881     IMMMMDDMMMMMMIMM
RPL21         IMMMMDIDIDMMMMMMM
OXA1L         IMMMMDMMMMMMMMM
UBB           DMMMMIIDIDMMMMMMM
MPC1          IMMMMDIDMMMMMMMM
GOT1          IMMMIDIDIDDMMMMMMM
AGPAT5        DMMMIDIDIMMMMMMMM
UBC           IMMMDDDMMMMMMMMII
KIF9-AS1      IMMMDIDIDMMMMMMMM
PDE3A         DMMMIDIDMMMMMMIMM
TIAM1         IMMMDIDMMMMMMMMM
MAST4         DMMMIIDIDMMMMMMMM
PLXNA2        IMMMIDIDIDIDDMMMMMM
TCEAL4        IMMMDDDDMMMMMMMVVV
MPP3          IMMMDDDMMMMMMMMII
FBXL7         IMMMDIDIDMMMMMMMM
MLLT11        IMMMIDIDIDDMMMMMMM
HBA2          IMMMDIDMMMMMMMMM
FSTL4         IMMMIDIDIDDMMMMMMM
MEF2C         DMMMIDIMMMMMMMMM
FAM78A        IMMMIDIDIDIDDMMMMMM
CDCA7L        IMMMDIDIDMMMMMMMM
PGAM1         IMMMDDDMMMMMMMMVV
CRYBG3        IMMMDDDMMMMMMMMVV
AUH           IMMMDIDIDMMMMMMMM
SLC2A13       IMMMDDDMMMMMMMMII
DPF3          IMMMDDDMMMMMMMIIID
AC093849.2    IMMMDIDIDIDMMMMMMM
PFKP          IMMMDIDIDMMMMMMMM
STMN1         IMMMDDDMMMMMMMMVV
MARCH3        DMMMDDMMMMMMMMIII
HSP90AA1      IMMMDIDIDIDMMMMMMM
ARHGEF12      DMMMIIDIDMMMMMMMM
PPIC          IMMMDDDMMMMMMMMVV
MAGED2        IMMDDDDMMMMMMMMVVV
GSTP1         IMMDDDDMMMMMMMMVVV
LRRC52-AS1    IMMIDIDIDIDDMMMMMMM
MDK           IMMDIDDDMMMMMMMMVV
SPINT2        IMMIDIDIDIDDMMMMMMM
GPATCH8       IMMDDDMMMMMMMMMVV
ADGRD1        IMMDDDDMMMMMMMMVVV
AL049828.1    DMMIDIDIDIMMMMMMMM
DPYSL3        IMMDIDIDDMMMMMMIMM
CNN3          IMMDDDMMMMMMMMMVV
LIFR          IMMDIDDMMMMMMMIMM
LINC02268     IMMDIDDDMMMMMMMMVV
IL1RAPL1      DMMDDMMIIIMMMMMMM
ATP2B4        DMMDIDIDMMMMMMMMVV
ZBTB16        IMMDDDDMMMMMMMMVVV
FERMT2        IMMDIDIDMMMMMMMMM
TNNI3K        IMMDDDDMMMMMMMMIII
HSPB7         DMMIMMIIDDMMMMMMM
CBX5          IMMDDDDMMMMMMMMVVV
ACO2          IMMDDDMMIIMMMMMMM
ZBTB20        IMDDMMMMMMMMMIMM
PLXDC2        IMIIMMIDDDDMMMMMMM
NEK10         IMIIMMDDDMMMMMMMM
CSRNP3        DDMMMMMMMMMMMMVV
PSMA2         DDMMMMMMMMMMMMVV
AC098617.1    DDMMMMMMMMMMMMVV
RLF           DDMMMMMMMMMMMMVV
HADHB         DDMMMMMMMMMMMMVV
CDCA8         DDMMMMMMMMMVVVVVDDD
NUP35         IIMMMMMMMMMMMMWW
PARM1         DDMMMMMMMMMMMMVV
ZCCHC8        DDMMMMMMMMMMMMVV
MFN2          DDMMMMMMMMMMMMVV
TTN-AS1       DDMMMMMMMMMMMMVV
TBX18         DDMMMMMMMMMMVVVVDD
GNG11         IDMMMMMMMMMMMMM
MLLT6         IDMMMMMMMMMMMMM
AL603840.1    IDMMMMMMMMMMMMM
RPL22L1       IDMMMMMMMMMMMMM
PROX1         IDMMMMMMMMMMMMM
VEGFA         IDMMMMMMMMMMMMM
NORAD         IDMMMMMMMMMMMMM
PRDX6         IDMMMMMMMMMMMMM
PSMC3         IDMMMMMMMMMMMMM
SGCG          IDMMMMMMMMMMMMM
SPECC1L       IDMMMMMMMMMMMMM
KDM2A         IDMMMMMMMMMMMMM
CAMTA1        IDMMMMMMMMMMMMM
KIAA1217      IDMMMMMMMMMMMMM
DYRK3         IDMMMMMMMMMMMMM
PNRC1         IDMMMMMMMMMMMMM
UQCRC2        IDMMMMMMMMMMMMM
FIGN          IDMMMMMMMMMMMMM
MAPRE2        IDMMMMMMMMMMMMM
MSI2          IDMMMMMMMMMMMMM
LGALS8        IDMMMMMMMMMMMMM
PPFIA4        IDMMMMMMMMMMMMM
C1orf198      IDMMMMMMMMMMMMM
DNAH14        IDMMMMMMMMMMMMM
OIP5-AS1      IDMMMMMMMMMMMMM
ESYT2         IDMMMMMMMMMMMMM
SLC6A6        DDMMMMMMMMMMMMII
LMOD1         DDMMMMMVVVVDDMMMMM
FEN1          DDMMMMMVVVVIDWWWWWWIIII
RTKN2         DDMMMMMVVVVDDMMMMM
EPAS1         DIVVVVVVDDDDDDMMMMMMM
SORCS2        IDMMVVVVDIDDDDMMMMMM
ALKBH7        DDMMMMMIIMMMMMMM
MT-CO1        IIMMMMMDDMMMMMMM
RPL3          IIMMMMMDDMMMMMMM
BTBD9         DDMMMMMIIMMMMMMM
MYO9B         DDMMMMMIIMMMMMMM
AC138305.1    IIMMMMMDDMMMMMMM
TPM2          IIMMMMMDDMMMMMMM
LNPEP         IIMMMMDDMMMMMMMM
ADAMTS6       IIMMMMDIDDMMMMMMM
PPARGC1A      IIMMMMDDIDMMMMMMM
ARL15         IIMMMMDDMMMMMMMM
PPP1CC        IIMMMMDDMMMMMMMM
CRPPA         IIMMMMDDMMMMMMMM
RHEB          IIMMMMDDMMMMMMMM
GABARAP       IDMMMMIDIDMMMMMMM
SRSF2         IIMMMMDDMMMMMMMM
RAP1GAP2      IIMMMMDDMMMMMMMM
FRMD3         IIMMMMDDMMMMMMMM
MT-ND5        IIMMMMDDMMMMMMMM
NR4A1         IIMMMMDDMMMMMMMM
BABAM2        IIMMMMDDMMMMMMMM
MPRIP         IIMMMMDDMMMMMMMM
SOX5          IDMMMMDMMMMMMIMM
TBL1XR1       IIMMMMDDMMMMMMMM
UBE3A         IIMMMMDDMMMMMMMM
USP9Y         DDMMMMDMMMMMMIIIID
FAM53B        IIMMMMDDMMMMMMMM
CCN1          IIMMMMDDMMMMMMMM
SLC25A5       IDMMMMIDIDMMMMMMM
ST6GAL1       IIMMMMDDMMMMMMMM
SAP30         IIMMMMDDMMMMMMMM
OGDH          IIMMMMDDMMMMMMMM
DHX15         IIMMMMDDMMMMMMMM
NDUFS1        IIMMMMDDMMMMMMMM
HSPB8         IIMMMMDDMMMMMMMM
NFYC          IIMMMMDDMMMMMMMM
PIK3R1        IDMMMMDDMMMMMMMVV
SERBP1        IIMMMMDDMMMMMMMM
STRN3         IIMMMMDDMMMMMMMM
RPL6          IIMMMMDDMMMMMMMM
PDIA3         IIMMMMDDMMMMMMMM
PAFAH1B1      IIMMMMDDMMMMMMMM
RPL19         IIMMMMDIDDMMMMMMM
VPS35L        IIMMMMDDMMMMMMMM
WNK2          IIMMMMDDMMMMMMMM
HIPK3         IIMMMMDDMMMMMMMM
MIA2          IIMMMMDDMMMMMMMM
HIVEP2        IIMMMMDDMMMMMMMM
COX7A2        IDMMMMIIDDMMMMMMM
JAG1          IIMMMMDDMMMMMMMM
HIF3A         IIMMMMDIDDMMMMMMM
HHATL         IIMMMMDDMMMMMMMM
PDHA1         IDMMMMIDIDMMMMMMM
SUMO2         IDMMMMIIDIDDMMMMMM
FNBP4         IIMMMMDDMMMMMMMM
IDH2          IDMMMMDDMMMMMMMVV
MT-ND1        IIMMMMDDMMMMMMMM
HSPB1         IIMMMMDDMMMMMMMM
HSPA8         IIMMMMDDIDMMMMMMM
SRP14         IDMMMMIDMMMMMMMM
XPO1          IIMMMMDDMMMMMMMM
RTL8C         IDMMMMIIDDMMMMMMM
CHN1          IIMMMMDDMMMMMMMM
TRRAP         IIMMMMDDMMMMMMMM
HMGN3         IDMMMIDMMMMMMMMM
SYNPO2L       IDMMMIIDIDDMMMMMMM
TMBIM6        DDMMMIDIDIIMMMMMMM
BEX3          IDMMMDDMMMMMMMMVV
PAGR1         IDMMMDDMMMMMMMMVV
STARD3NL      IIMMMDIDIDDMMMMMMM
MYL9          IDMMMDDMMMMMMMMVV
YWHAB         IDMMMIIDIDDMMMMMMM
PLOD2         IDMMMDIDMMMMMMIMM
TECRL         IIMMMIDIDIDDMMMMMMD
HDAC2         IDMMMDDMMMMMMMMVV
APOBEC2       DDMMMIIIDIDMMMMMMM
NDUFA13       IDMMMDDMMMMMMMMII
PCMTD1        IIMMMDDMMMMMMMMM
LINC01954     IDMMMDMMMMMMMIMM
TBX20         IIMMMDDDMMMMMMIMM
SNHG6         IIMMMDDIDIDMMMMMMM
DSTN          IIMMMDDMMMMMMMMM
COL4A2        IIMMMDDDMMMMMMIMM
OST4          IIMMMDDMMMMMMMMM
GFOD1         IIMMMDIDDMMMMMMMM
ENO3          IDMMMIIDIDIDDMMMMMM
SSBP2         DDMMMDMMMMMMMMVVV
TCEAL8        IDMMMDDMMMMMMMMVV
TPI1          IDMMMDIDDMMMMMMMII
SLC25A6       IDMMMDDMMMMMMMMVV
TUBB          IDMMMDDMMMMMMMMVV
MYL6          IDMMMDDMMMMMMMMII
KLF12         IDMMMIDMMMMMMMMM
PPM1B         IIMMMDDMMMMMMMMM
VWDE          IIMMMDIDDMMMMMMMM
KDM4C         DDMMMIDIIMMMMMMMM
COL18A1       IDMMMDDMMMMMMMMVV
EID1          IDMMMDMMMMMMMMMI
ANKRD11       IIMMMDIDDMMMMMMMM
DIP2B         IDMMMIDIDMMMMMMMM
CORIN         IIMMMIDIDDDMMMMMMM
TSC22D2       IDMMMIDIDMMMMMMMM
HDAC9         IIMMMDIDDMMMMMMMM
MTHFR         IIMMMIDIDIDDDMMMMMM
FBXL17        DDMMMIDIDIIMMMMMMM
RNF149        IIMMMDIDDMMMMMMMM
ADAMTS9       IIMMMDIDIDDMMMMMMM
CFL2          IDMMMDDMMMMMMMMVV
TRPS1         IDMMMDDMMMMMMMMVV
DSG2          IIMMMDDDDMMMIIMMMM
RPSA          IDMMMIDMMMMMMMMM
PA2G4         IDMMMDDMMMMMMMMVV
MCRS1         IIMMMDIDIDDMMMMMMM
KRT8          IDMMMDDMMMMMMMMVV
RORA          IIMMMIDIDIDDDMMMMMM
PTOV1         IDMMMIDIDIDMMMMMMM
CACNA1D       IIMMMDIDIDDMMMMMMM
PHC2          IIMMMIDIDDDMMMMMMM
FKBP3         IDMMMIDIDIDMMMMMMM
AC040174.1    IDMMMIDIDIDMMMMMMM
PACS1         IDMMMIDIDMMMMMMMM
EPB41L4A      IDMMMDDMMMMMMMMVV
MGST3         IDMMMIIDIDDMMMMMMM
EEF1A2        IDMMMIDIDMMMMMMMM
ADAMTS1       IIMMMDDDDMMMMMMMVV
DAPK3         IDMMMIDMMMMMMMMM
ALPK3         IDMMMDIDMMMMMMIMM
ARRDC3-AS1    IIMMMDIDIDDMMMMMMM
PPP1R14C      IIMMMDDMMMMMMMMM
COX7A2L       IDMMMIDIDIIDDMMMMMM
ATP5ME        IDMMMIDMMMMMMMMM
HCN1          DDMMMIIIDIDMMMMMMM
PCGF5         IIMMMDDMMMMMMMMM
CKMT2         IIMMMDIDIDDMMMMMMM
STAT3         DDMMMIDIIMMMMMMMM
LINC02248     DDMMIDIDIIIIDDDMMMMIID
RYR3          DDMMIDIDIIIIDDMMMMMM
CD164         IIMMDIDDMMMMMMMMM
USP28         DDMMIDIMMMMMMMIMM
TUFM          DDMMDMMMMMMMMMVVV
TNNI3         IIMMIIDIDDDDMMMMMMM
DST           DDMMDDMMIIIIMMMMMM
AC004540.1    IIMMIDIDIIIDDDDDMMMMM
BRWD1         IIMMDDIDIDMMMMMMMM
PPP2R3A       IDMMDDMMMMMMMMMVV
KCNH7         DDMMIDIDIIMMMMMMMM
RASGRP3       DDMMIIMMMMMMMMMM
KCNQ5         IIMMIMMIIDDDDMMMMMD
MYH9          IDMMIDIDIDDDMMMMMMII
LRIG1         IIMMDIDIDDMMMMMMMM
COL21A1       IIMMIDIDIDDDMMMMMMM
CAMK2B        IDMMIDIDIDMMMMMMMM
ASPH          DDMMIDIIMMMMMMMMM
PLCL1         IIMMIDIDIIIIIIIIDDDDDDDDDD
ZMAT4         IDMMDDIDMMMMMMMMVV
TBX3          IIMMIDIDIDDDMMMMMMM
ARID5B        IIMMDDDDMMMMMMMMVV
TSHZ2         IDMMIDIDIIIDDDMMMMMID
KIAA0232      IIMMIDIDIDDDMMMMMMM
KALRN         IIMMDIDIDDDMMMMMIMM
RPS28         IDMMDDMMMMMMMMMII
LARGE1        DDMMIDIDMMMMMMIIMM
SQLE          IDMMIDIDIIDDMMMMMMM
DNAJB6        IDMMIDIDMMMMMMMMM
PANK3         IIMMDIDIDDMMMMMMMM
UGP2          IDMMIMMIDDMMMMMMM
NPPA          DDMMIIIIDDMMMMMMMM
ADAMTS19      DDMMDMMMMMMMMMVVV
AP005329.2    IIMMDDMMMMMMMMMM
SUCLG1        IIMMIDIDIDDDMMMMMMM
ATP11A        DDMMIIDIDIMMMMMMMM
SRSF10        DDMMIDIIMMMMMMMMM
DRAP1         DDMMDMMMMMMMMMVVV
ITGA9         IIMMDDDDDMMMIIIMMMM
CPEB4         IIMMDIDDMMMMMMMMM
PURG          IDMMIDIMMMMMMMDMM
MBNL2         IDMMDDMMMMMMMMMVV
GALNT17       DDMMIDIDMMMMMMMMII
MAP3K20       DDMDDMMMIIIIMMMMMM
CNTN5         IDMIIDIDIIIIDDDDDMMMMM
SLIT3         IIMIIDIDIDDDDMMMMMMM
FAM189A1      IDMIIMMDDMMMMMMMM
PDE7A         IDMDDMMIIMMMMMMMM
KIDINS220     IIMIIDIDIDDDDMMMMMMM
UBR2          DDDMMMMMMMMMMMVVV
TUBA1A        DDDMMMMMMMMMMMVVV
SDK1          DDDMMMMMMMMMMMVVV
DLGAP4        DDDMMMMMMMMMMMVVV
HMGB2         DDDMMMMMMMMMMMVVV
P4HA1         DDDMMMMMMMMMMMVVV
YPEL2         DDDMMMMMMMMMMMVVV
FAM13C        DDDMMMMMMMMMVVVVVDD
RASSF3        DDDMMMMMMMMMMMVVV
NTRK2         IDDMMMMMMMMMVVVVDDD
TPT1          IDDMMMMMMMMMMMMI
MTUS1         IDDMMMMMMMMMMMMI
CSNK2B        IDDMMMMMMMMMMMMI
TIMM8B        IDDMMMMMMMMMMMMI
MARCKS        DDDMMMMMMMMMMMIII
DCLK1         IIIMMMMMMWWWWWIIMMM
LRRFIP1       IDDMMMMMMMMMMIMM
LDLRAD4       IDDMMMMMMMMMMIMM
GULP1         IDDMMMMMMMMMMIMM
CARMN         IDDMMMMMMMMMMIMM
BRAF          IDDMMMMMMMMMMIMM
AGO2          IDDMMMMMMMMMMIMM
KMT2E         IDDMMMMMMMMMMIMM
ANK3          IDDMMMMMMMMMMIMM
VMP1          IDDMMMMMMMMMMIMM
ABHD18        IDDMMMMMMMMMMIMM
LINC00310     DDDMMMMVVVVDMMMMMM
ADAMTS7       IDDMMMMMMMMIMMMM
SLC5A3        IDDMMMMMIMMMMMMM
LRRC7         DDDMMMMMIIIMMMMMM
PHLPP1        DDDMMMMMIIIMMMMMM
RPS15         IIIMMMMDDDMMMMMMM
STOX2         IIIMWWWIDIDMMMMMMMM
MRFAP1        IIIMMMMDDDMMMMMMM
NXN           IIIMMMMDDDMMMMMMM
CTNNAL1       IIIMMMMDDDMMMMMMM
NDUFA12       IIDMMMMDIDMMMMMMM
CD99          IIIMMMMDDDMMMMMMM
FRY           IIDMMMMDMMMMMMMM
TBC1D8        IIIMMMMDDDMMMMMMM
FTH1          IIDMMMMDIDMMMMMMM
RPS23         IIIMMMMDDDMMMMMMM
EEF1B2        IIIMMMMDDDMMMMMMM
RPS4X         IIIMMMMDDDMMMMMMM
TTC28         IIIMMMMDDDMMMMMMM
MYOM2         IIIMMMMDDDMMMMMMM
RPL37         IIIMMMMDDDMMMMMMM
ATP5PF        IIDMMMMIDDMMMMMMM
EDF1          IIDMMMMDIDMMMMMMM
CHCHD2        IIDMMMMDIDMMMMMMM
TESC          IIIMMMMDDDMMMMMMM
RPL32         IIIMMMMDDDMMMMMMM
RPS6          IIIMMMMDDDMMMMMMM
RPLP0         IIIMMMMDDIDDMMMMMM
PLXNA4        IIIMMMMIDIDDDDMMMMM
RPS9          IIIMMMMDDDMMMMMMM
RPL24         IIIMMMMDDDMMMMMMM
RPS3          IIIMMMMDDDMMMMMMM
NDUFB9        IIDMMMMIDIDDMMMMMM
RPL7          IIDMMMMIDDMMMMMMM
SYNPR         DDDMMMMIIIMMMMMMM
FAU           IIIMMMMDDDMMMMMMM
MAN1A1        IIIMMMMDDDMMMMMMM
RPS7          IIIMMMMDDDMMMMMMM
UQCR10        IIDMMMMDIDMMMMMMM
MT-ATP6       IIIMMMMDDDMMMMMMM
PREP          IIIMMMMDDDMMMMMMM
RPL13A        IIIMMMMDDDMMMMMMM
COL4A6        IIDMMMMDMMMMMMMM
RPL18A        IIIMMMMDDDMMMMMMM
UNC5C         IDDMMMMIMMMMMMMM
RPL5          IIIMMMMDDDMMMMMMM
ATP1B1        IIDMMMMDIDMMMMMMM
PFDN5         IIIMMMMDDDMMMMMMM
RPL22         IIDMMMMDMMMMMMMM
ATP5MG        IIDMMMMDIDMMMMMMM
CHCHD10       IIDMMMMIDDMMMMMMM
CALM3         IIDMMMMIDDMMMMMMM
SOD1          IIIMMMMDDDMMMMMMM
RPL9          IIIMMMMDDDMMMMMMM
DGKD          IIIMMMMDDDMMMMMMM
C19orf53      IIDMMMMDIDMMMMMMM
NDUFB7        IIDMMMMIDDMMMMMMM
NSD3          IIIMMMDDDMMMMMMMM
MBOAT2        IIIMMMDDDMMMMMMMM
KDELR2        IIIMMMDDDMMMMMMMM
SUB1          IIDMMMIDIDDMMMMMMM
SAFB2         IIIMMMDDDMMMMMMMM
DGLUCY        IIIMMMDDDMMMMMMMM
NUTM2A-AS1    IIDMMMDMMMMMMMMM
C5orf24       IDDMMMDMMMMMMMMVV
OPTN          IIIMMMDDDMMMMMMMM
UBE2E2        IIIMMMDDDMMMMMMMM
PPP2R5C       IDDMMMIIDMMMMMMMM
MB            IIIMMMDDDDDMMMMMMII
TEAD1         IDDMMMIDMMMMMMMIID
RPS2          IIDMMMDMMMMMMMMM
RPL36         IIIMMMDDDMMMMMMMM
MAGED1        IDDMMMIIDIDMMMMMMM
CCN2          IIIMMMDDIDDMMMMMMM
GRAMD4        IIDMMMDMMMMMMMMM
RCAN1         IIIMMMDDDMMMMMMMM
ACOT11        IIIMMMDDDMMMMMMMM
HSPA1B        IIIMMMDDDMMMMMMMM
PALM2-AKAP2   IIDMMMDMMMMMMMMM
PTPRM         IIIMMMDDDMMMMMMMM
LYPLAL1       DDDMMMIIIMMMMMMMM
RGL1          IDDMMMDMMMMMMMMVV
HIPK2         IIIMMMDDDMMMMMMMM
HNRNPC        IDDMMMIDIDIMMMMMMM
RPS15A        IIDMMMDIDMMMMMMMM
YBX1          IIDMMMIDIDDMMMMMMM
COX6C         IDDMMMIIDIDMMMMMMM
COPS9         IIDMMMDIDMMMMMMMM
PTMA          IIDMMMDIDMMMMMMMM
INPP4B        IIIMMMDIDDDMMMMMMM
NDUFV2        IIDMMMIDDMMMMMMMM
RPL23A        IIIMMMDDDMMMMMMMM
ZADH2         IIIMMMDDIDDMMMMMMM
SLK           IDDMMMIMMMMMMMMM
HSPA1A        IIIMMMDDDMMMMMMMM
SKI           IIIMMMDDDMMMMMMMM
POLR2F        IIIMMMDDDMMMMMMMM
HSP90B1       IIIMMMDDDMMMMMMMM
SEPTIN11      IDDMMMIIDMMMMMMMM
SNX3          IIDMMMIDIDDMMMMMMM
RPL7A         IIDMMMDMMMMMMMMM
TSPAN9        IDDMMMIMMMMMMMMM
TNKS          IIIMMMDDDMMMMMMMM
ATP1A1        IIIMMMDDDMMMMMMMM
SNRPC         IIIMMMDIDDDMMMMMMM
TCAP          IDDMMMIIDIDMMMMMMM
OGT           IIIMMMDDDMMMMMMMM
NDUFA6        IDDMMMIDIDIMMMMMMM
BTF3          IIDMMMIIDDDMMMMMMM
NME2          IIDMMMDMMMMMMMMM
N4BP2L2       IIIMMMDDDMMMMMMMM
AC119673.2    IIDMMMDDDMMMMMMMVV
ELOB          IIDMMMDMMMMMMMMM
CIRBP         IDDMMMIIDIIDDMMMMMM
CORO1C        IIIMMMDDDMMMMMMMM
VSNL1         IDDMMMIIDIDMMMMMMM
PPDPF         IDDMMMIIDIDMMMMMMM
FEM1C         IDDMMMIIDMMMMMMMM
MACF1         IDDMMMIMMMMMMMMM
TMSB10        IIIMMMDDDMMMMMMMM
ENO1          IDDMMMIIDIDMMMMMMM
LRRTM3        IDDMMMIIDIIDDMMMMMM
NDUFB3        IIDMMMIDIDDMMMMMMM
ARF1          IDDMMMIDIDIMMMMMMM
HNRNPA3       IIIMMMDDDMMMMMMMM
ENAH          IIIMMMDDDMMMMMMMM
FAM13B        IIIMMMDDDMMMMMMMM
PAG1          IIIMMMIIDIDDDMMMMIDDD
SIVA1         IIDMMMDMMMMMMMMM
KRAS          IIIMMMDDIDDMMMMMMM
RRAD          IIDMMMDMMMMMMMMM
ZNF521        IIDMMMDMMMMMMMMM
LCLAT1        IDDMMIDIMMMMMMMMM
GREB1L        IDDMMIDIDIMMMMMMMM
LMO4          IDDMMDDMMMMMMMMVVV
DHRS3         IIIMMDIDIDDDMMMMMMM
PABPC1        IDDMMDMMMMMMMMMVV
SNRPD2        IDDMMDMMMMMMMMMVV
HECW2         IIIMMDDDDDDMMIIIMMMM
SMAD7         IDDMMDMMMMMMMMMII
TGFBR3        IDDMMIIDIDMMMMMMMM
GUK1          IIIMMDIDIDDDMMMMMMM
ARMCX3        IIIMMDDIDDMMMMMMMM
PITPNC1       IIIMMDDDDDMMMMMMMVV
ATP6V0C       IDDMMIMMMMMMMMMM
CD63          DDDMMIDIDIIIIDMMMMMM
KCNJ3         IDDMMIDIDIMMMMMMMM
TSPAN18       IIIMMDIDDDMMMMMMMM
ZFHX3         IIDMMIDIDDMMMMMMMM
SLC4A4        IDDMMIDIDIMMMMMMMM
ZNF608        IDDMMIDIDIIIDDMMMMMM
MBTPS1        IDDMMIDIMMMMMMMMM
BAMBI         IDDMMIIDMMMMMMMMM
AP1S2         IIIMMDIDIDDDMMMMMMM
CYP2J2        DDDMMIIIMMMMMMMMM
SRL           IDDMMIDIDIMMMMMMMM
AC083870.1    IIIMMIIDDDDDMMMMMMM
CADPS2        IDDMMIIDIDMMMMMMMM
SPATS2L       IDDMMIDMMMMMMMIMM
GRIP1         IDDMMIDIDIMMMMMMMM
NONO          IDDMMDDMMMMMMMMVVV
RBFOX1        IIIMMDDDDDDMMMMMMVVV
ZFAND5        IDDMMIDIMMMMMMMMM
RGS6          IDDMMDIDMMMMMMMMII
YWHAZ         IDDMMIMMMMMMMMMM
MAP1B         IDDMMDDMMIIIMMMMMM
PLN           IIIMMDDIDDMMMMMMMM
HMGA2         IDDMMIIDMMMMMMMMM
NDRG4         IIIMMDIDIDDDMMMMMMM
TMEM71        DDDMMDDMMMIIIMMMMII
INSR          IDDMMDDMMMMMMMMVVV
SYNPO2        IIDMMIDIDIDDMMMMMMM
NFIA          IDDMMIDMMMMMMMIMM
RABGEF1       IIIMMDDDMMMMMMMMM
EPHA4         IIIMMDDDDMMIMMMMMM
BHLHE40       IDDMMIDIDIMMMMMMMM
NDUFA11       IDDMMIDIDIIDMMMMMMM
PPP2R5A       IIIMMDDDMMMMMMMMM
GJC1          IDDMMDMMMMMMMMMVV
SPHKAP        DDDMIIDIDIIIDMMMMMMM
GLI3          DDDMIIIDIMMMMMMMMM
CDH11         IDDMIDIDIIDMMMMMMMM
SFRP1         IIDMIIDIDIIDDDDMMMMMM
SOBP          IDDMIIDIDIIDDMMMMMMM
C1orf21       IDDMIIDIDIIDDMMMMMMM
SMAD6         IDDMIIDIDIIDDMMMMMMM
CHRM2         DDDMIIDIDIIIDMMMMMMM
FAT3          IDDMIIIMMIDDDMMMMMM
SLC2A3        IIDMIDIDIDDMMMMMMMM
FAM124A       DDDMIDMMMMMMMMMVVV
TACC2         DDDMDDMMMIIIIIMMMMM
RFC3          DDIDMMMMMMMMMMMVV
EXOC6B        DDIDMMMMMMMMMMMVV
GLCE          DDIDMMMMMMMMMMMVV
PTAR1         DDIDMMMMMMMMMMMVV
NPTN          DDIDMMMMMMMMMMMVV
RB1CC1        DDIDMMMMMMMMMMMVV
VIM           DDIDMMMMMMMMMMMVV
RPS25         IDIDMMMMMMMMMMMM
TLE5          IDIDMMMMMMMMMMMM
HEG1          IDIDMMMMMMMMMMMM
ZFPM2         IDIDMMMMMMMMMMMM
SNHG29        IDIDMMMMMMMMMMMM
MIR99AHG      IDIDMMMMMMMMMMMM
NFIB          IDIDMMMMMMMMMMMM
PLAAT3        IDIDMMMMMMMMMMMM
AC092691.1    IDIDMMMMMMMWWWWWIIIII
ARL6IP4       IDIDMMMMMMMMMMMM
CDKN1A        IDIDMMMMMMMMMMMM
MXRA7         IDIDMMMMMMMMMMMM
ATP1B2        IDIDMMMMMMMMMMMM
ESRRG         IDIDMMMMMMMMMMMM
TENM4         IDIDMMMMMMMMMMMM
RRAGD         IDIDMMMMMMMMMMMM
PAPOLA        IDIDMMMMMMMMMMMM
ZEB2          IDIDMMMMMMMMMMMM
RPS12         IDIDMMMMMMMMMMMM
RAI1          IDIDMMMMMMMMMMMM
RIPK4         IIIDMMMMMMMMMMDDM
PDE1A         IIIDMMMMMMWWWWIIMMM
LRRC4C        IDIDMMMMMMMMIIDIIDDD
FITM1         DDDDMMMMMIIIIMMMMM
ERC2          DDDDMMMMMIIIIMMMMM
SLC8A3        IDIDMMMMIDMMMMMMID
BNIP3         DDDDMMMMIIIIMMMMMM
TOMM7         IIIDMMMMDDMMMMMMM
RPL8          IIIDMMMMDDMMMMMMM
ST3GAL4       IIIIMMMMDDDDMMMMMM
MDH2          DDDDMMMMIIIIMMMMMM
NR3C2         IIIDMMMMDDMMMMMMM
CYTH3         DDDDMMMMIIIIMMMMMM
AC087457.1    DDIDMMMMIIMMMMMMM
TNNI1         IIIDMMMMDDMMMMMMM
TOMM20        IIIDMMMMDDMMMMMMM
ALPK2         DDIDMMMMIIMMMMMMM
RALGPS2       DDDDMMMMIIIIMMMMMM
LRMDA         DDDDMMMMIIIIMMMMMM
FBXO32        IIIIMMMMDDDDMMMMMM
ADGRL2        DDDDMMMMIIIIMMMMMM
NDUFA4        IIIDMMMMDDMMMMMMM
COX7C         IIIDMMMMDDMMMMMMM
KDM6A         IIIDMMMDDMMMMMMMM
CPNE3         IIIDMMMDDMMMMMMMM
RPS20         IIIDMMMDDMMMMMMMM
MT-CYB        IIIIMMMDDDDMMMMMMM
BMP7          IIIIMMMDDDDMMMMMMM
SAMD12        IIIIMMMDDDDMMMMMMM
ATP5PD        IDIDMMMIIDDMMMMMMM
DAB2IP        IIIIMMMDDDDMMMMMMM
MEF2A         IIIDMMMDDMMMMMMMM
JUND          IIIDMMMDDMMMMMMMM
COX5A         IDIDMMMIDIIDDMMMMMM
SRGAP1        IIIIMMMDDDDMMMMMMM
MAGI1         IIIDMMMDIDDMMMMMMM
NDUFB1        IIIDMMMIDDDMMMMMMM
FBRSL1        IIIDMMMDDMMMMMMMM
RPL35         IIIDMMMDDMMMMMMMM
RPL35A        IIIIMMMDDDDMMMMMMM
RPS27         IIIDMMMDIDDMMMMMMM
BACH2         IIIDMMMDDMMMMMMMM
PHPT1         IIIDMMMDIDDMMMMMMM
ITGB1         IIIDMMMDDMMMMMMMM
RPL37A        IIIIMMMDDDDMMMMMMM
SLC1A3        DDDDMMMIIIIMMMMMMM
SMPX          IIIIMMMDDDDMMMMMMM
EIF3H         DDDDMMMIIIIMMMMMMM
TRDN-AS1      IIIDMMMDDMMMMMMMM
AZIN1         IIIDMMMDDMMMMMMMM
GJA5          IIIIMMMDDIDDDMMMMMM
PLCXD3        DDDDMMMIIIMMMMMIMM
HK1           IIIDMMMIDDDMMMMMMM
RPL36AL       IIIDMMMDDMMMMMMMM
FIS1          IIIDMMMDIDDMMMMMMM
EEF1A1        IIIIMMMDDDDMMMMMMM
PPP1R3C       IIIDMMMDDMMMMMMMM
COPE          IIIDMMMDIDDMMMMMMM
RBM6          IIIIMMMDDDDMMMMMMM
EFNA5         IIIIMMMDDDDMMMMMMM
ATP5F1B       IIIDMMMIDIDDDMMMMMM
NDUFB10       IDIDMMMIIDDMMMMMMM
NDUFAF8       IIIDMMMDDMMMMMMMM
FLNC          IIIDMMMDIDDMMMMMMM
RPL10         IIIDMMMDIDDMMMMMMM
SORT1         IIIIMMMDDDDMMMMMMM
COX8A         IIIDMMMIDIDDDMMMMMM
RPS3A         IIIIMMMDDDDMMMMMMM
RPS17         IIIDMMMDDMMMMMMMM
OBSCN         IIIIMMMDDDDMMMMMMM
RPS16         IIIDMMMDDMMMMMMMM
COX7B         IDIDMMMIIDDMMMMMMM
RPL13         IIIDMMMDDMMMMMMMM
NDUFS6        IDIDMMMIDIDMMMMMMM
COX6B1        IDIDMMMIIDDMMMMMMM
MT-CO2        IIIIMMMDDDDMMMMMMM
DUOX2         IIIIMMMDDIDDDMMMMMM
RPS21         IIIIMMMDDDDMMMMMMM
RPL30         IIIIMMMDDDDMMMMMMM
CSRP3         IIIIMMMDDDDMMMMMMM
ATXN1         IIIIMMMDDDDMMMMMMM
APLP2         IIIDMMMDDIDMMMMMMM
ABCB4         IIIIMMMDDDDMMMMMMM
GAPDH         IIIDMMMIDDDMMMMMMM
PTK2          IDIDMMIDMMMMMMMMM
SLC2A1        DDDDMMIIIIMMMMMMMM
RFTN1         IDIDMMIIDIIIIIDDDDDMMMD
CMSS1         IDIDMMIDIDMMMMMMMM
NRP2          DDIDMMIIMMMMMMMMM
LDB3          IDIDMMIIDIDDMMMMMMM
HAND2-AS1     IIIIMMDDDDMMMMMMMM
RABAC1        IDIDMMIIDIDDMMMMMMM
BARD1         IIIDMMDIDDMMMMMMMM
CGNL1         IDIDMMDDMMMMMMMMVV
PPP2R1A       IIIIMMDIDDDDMMMMMMM
IRF4          IIIIMMDDDDDDMMMMMMVV
STARD9        IDIDMMDDMMMMMMMMVV
DNAJC15       IDIDMMIDMMMMMMMMM
RSRC1         IDIDMMIIDDMMMMMMMM
FOXO3         IDIDMMDDMMMMMMMMVV
RCSD1         IDIDMMIDMMMMMMMMM
FGF1          IIIIMMDDDDMMMMMMMM
SGCD          DDDDMMIIMMMMMMMMVV
PRICKLE1      IDIDMMIDIDDMMMMMMIID
CTIF          IIIIMMDDDDMMMMMMMM
KRT10         IIIIMMDIDDDDMMMMMMM
PLD1          IDIDMMIIDDMMMMMMMM
SUPT3H        IDIDMMIIDDMMMMMMMM
STAG2         IIIIMMDDDDMMMMMMMM
PGM5          IDIDMMDDMMMMMMMMVV
TANC2         IDIDMMDDMMMMMMMMII
CDH4          IIIIMMIDIDIIDDDDDDMMMM
SELENOT       IIIDMMIDIDDDMMMMMMM
HNRNPA0       IDIDMMDDMMMMMMMMVV
PTPRK         IDIDMMIDIDMMMMMMMM
MYL12A        IIIDMMDDMMMMMMMMM
NEXN          DDDDMMIIMMMMMMMMVV
CLIP4         IDIDMMIIDIDDMMMMMMM
CHST11        IDIDMIIDIIDDDDDMMMMIIID
NREP          IIIDMDDIDMMMMMMMMM
TSC22D1       DDIDMDIDMMMMMMMMVVV
MYL3          DDDDMIIDIIMMMMMMMMI
TRABD2B       IDIDMDDDMIIIMMMMMMM
TMEM178A      DDIDMIIDIIDMMMMMMMM
FREM2         DDIDMIIMMMMMMMMMM
FBXO40        DDIDMIDIDMMMMMMMMVV
CDK14         IDIDMDIDMMMMMMMIMM
WSB1          DDIDMIDIDIIMMMMMMMM
MID1          IIIDMIDIDDDMMMMMMMM
RBPMS2        DDDDMIIIIMMMMMMMMM
AC008591.1    DDIDMDMMMMMMMMMVVV
DES           DDIDMDDMIIIIMMMMMMM
XYLT1         DDDDMIIIIIDMMMMMMMM
CAST          DDIDMDMMMMMMMMMVVV
TMEM163       DDDDMIIDMMMMMMMMIII
PRDM6         IDIDMIIDIIDDDMMMMMMM
ENOX1         IDIDMIMIIDDDMMMMMMM
ERO1A         DDDIDMMMMMMMMMMVVV
STK38L        DDDIDMMMMMMMMMMVVV
CREB5         DDDIDMMMMMMMMMMVVV
RPS10         DDDIDMMMMMMMMMMVVV
NKX2-5        DDDIDMMMMMMMMMMVVV
FOXN3         DDDIDMMMMMMMMMMVVV
ELF1          DDDIDMMMMMMMMMMVVV
MPPED2        DDDIDMMMMMMMMMMVVV
MLYCD         DDDIDMMMMMMMMMMVVV
NBEA          DDDIDMMMMMMMMMMVVV
CAMK1D        DDDIDMMMMMMMMMMIII
MYO16         IIDIDMMMMMMMMMMIDD
CASQ2         DDDIDMMMMMMMMMMIII
PCBP2         IDDIDMMMMMMMMMIMM
RGS3          IDDIDMMMMMMMMMIMM
NEURL1B       DDDIDMMMMVVVVDMMMMM
RPL39         DDDDDMMVVVVIMMMMMMM
ATP5IF1       IIIIDWWWWWWIIIMMMMMMM
TPX2          IIIDIMMMMMDDDMMMMM
TUBA1B        DDDDIVVVVVDDMMMMMMMM
FMNL2         IDDIDMMMMMIMMMMMM
ULK4          DDDDDMMMMIIIIIMMMMM
PPP2R2B       DDDIDMMMMIIIMMMMMM
MYPN          DDDDDMMMMIIIIIMMMMM
MYH7B         IDDIDMMMIMMMMMMMM
RPL4          IIIDIMMMDDDMMMMMMM
EGFR          DDDDDMMMIIIIIMMMMMM
BNC2          DDDDDMMMIIIIIIIIIIIDWWWWW
KIF26B        IIIIIMMMIIDDDDMMMIDDDD
ATP13A3       DDDIDMMMIIIMMMMMMM
PGAM2         DDDDDMMMIIIIIMMMMMM
PRNP          IIIDIMMMDDDMMMMMMM
DAPK2         IIIDIMMMDDMMMMMMMD
RPS18         IIIDIMMMDDDMMMMMMM
SERF2         IIIDIMMMDDDMMMMMMM
SGIP1         IIIIIMMMDDDDDMMMMMM
RPLP1         IIIDIMMMDDDMMMMMMM
RPL23         IIIDIMMMDDDMMMMMMM
DAB1          IIIIIMMMIIIDDDDDDDMMMD
BRI3          IIDIDMMMIDDMMMMMMM
CMC2          DDDDDMMMIIIIIMMMMMM
SLC27A6       IIIDIMMMDDDMMMMMMM
RPS26         IIIDIMMMDDDMMMMMMM
MICOS10       IIIDIMMMDDDMMMMMMM
MIF           IIDIDMMMDMMMMMMMM
NDUFS5        IIDIDMMMIDDMMMMMMM
PRKCE         IIIIIMMMDDDDMMMMMMD
TNNC1         IIIDIMMMIDDDDMMMMMM
HSPB2         IIDIDMMMIDDMMMMMMM
ATP5MPL       IIIDIMMMDDDMMMMMMM
GOLIM4        DDDIDMMMIIIMMMMMMM
NDUFB11       DDDDDMMMIIIIIMMMMMM
MBP           IIIIIMMMDDDDDMMMIIIDDD
COX5B         IIIDIMMMDDDMMMMMMM
RPS8          IIIDIMMMDDDMMMMMMM
GAS5          IIIDIMMDDDMMMMMMMM
ADGRL3        IDDIDMMDMMMMMMMMVV
NDUFA1        DDDDDMMIIIIIMMMMMMM
MT-CO3        IIIIIMMDDDDDMMMMMMM
FHL1          IIIIIMMDDDDDMMMMMMM
AC011900.1    DDDDDMMIIIIIMMMMMMM
ATP5F1E       IIDIDMMIDIDDMMMMMMM
PLCL2         DDDIDMMIIIMMMMMMMM
JAK1          IIIDIMMDDDMMMMMMMM
MBNL1         IIIDIMMDDDMMMMMMMM
PEAK1         IIIIIMMDDDDDMMMMMMM
BEX1          DDDDDMMIIIIIMMMMMMM
UQCRQ         IIIIIMMDDDDDMMMMMMM
RPS13         IIDIDMMDMMMMMMMMM
EEF2          IIDIDMMDMMMMMMMMM
UBE2E3        IIDIDMMDMMMMMMMMM
RAB21         IIIIIMMDDDDDMMMMMMM
CDC42EP3      IIDIDMMDMMMMMMMMM
NR2F1-AS1     IIIDIMMDDDMMMMMMMM
KHDRBS2       IDDIDMMDMMMMMMMMVV
DSP           DDDDDMMIIIIIMMMMMMM
PDGFC         DDDIDMMIIIIIDDMMMMMM
RPL12         IIIDIMMDDDMMMMMMMM
DNAH11        DDDDDMMIIIMMMMMMMVV
ATP5MF        IIDIDMMIIDDDMMMMMMM
FNDC3B        IIIDIMMDDDMMMMMMMM
GNG5          IIIDIMMDDDMMMMMMMM
MAGI2         DDDIDMMIIDIIMMMMMMM
RNASEK        IIDIDMMIDIDDMMMMMMM
RPL29         IIIDIMMDDDMMMMMMMM
FREM1         IIIDIMMDDDMMMMMMMM
MYOZ2         IIIIIMMDDDDDMMMMMMM
CKM           DDDIDMIDIIIIDMMMMMMM
HSDL1         DDDIDMIIIMMMMMMMMM
JUN           IIIIIMDDDDDMMMMMMMM
NMNAT2        IIIIIMDDDDDMMMMMMMM
TANC1         DDDDDMIIDIIIIIIDDMMMMM
FOXP2         DDIDIDMMMMMMMMMMVV
LAPTM4A       DDIDIDMMMMMMMMMMVV
CDH13         DDIDIDMMMMMMMMMMVV
PRKN          DDIDIDMMMMMMMMMMVV
FBXO31        DDIDIDMMMMMMMMMMVV
SFRP5         IDIDIDMMMMMMMMMMM
DSC3          IDIDIDMMMMMMMMMMM
KANK1         IDIDIDMMMMMMMMMMM
TNS1          IDIDIDMMMMMMMMMMM
SPATA5        IDIDIDMMMMMMMMMMM
FZD1          IDIDIDMMMMMMMMMMM
GDF10         IIIDIDMMMMMMMMMMDD
AC008050.1    DDIDIDMMMMMMMMMMII
E2F8          DDIDIDMMMMMMMMIIMM
RAP1GDS1      DDIDIDMMMMIIMMMMMM
LMOD2         DDDIDDMMMMIIIIMMMMM
FBXO42        DDDIDDMMMMIIIIMMMMM
MED12L        DDDIDDMMMMIIIIMMMMM
B2M           IIDIDIVVVVDDDDDDMMMMMM
ELL2          DDDIDDMMMIIIIMMMMMM
RREB1         DDDIDDMMMIIIIMMMMMM
UQCR11        DDDIDDMMMIIIIMMMMMM
RPL11         DDDIDDMMMIIIIMMMMMM
AC018464.1    IIIDIDMMDDMMMMMMMM
DLC1          IIIDIDMMDDMMMMMMMM
IGDCC4        IIIIIIMMDDDDDDMMMMMM
COX7A1        IIIDIIMMDDDDMMMMMMM
CACNA1C       DDDDDDMMIIIMMMMMMVVV
DIAPH3        IIIIIIMMDDDDDDMMMMMM
LRRC49        IIIDIDMMDDMMMMMMMM
PYGB          DDIDIDMMIIMMMMMMMM
COX6A2        IIIDIDMMIDDDMMMMMMM
COL4A1        IIIIIIMMDDDDDDMMMMMM
MIR924HG      DDDDDDMMIIIIIIMMMMMM
RASGEF1B      IIIDIIMMDDDDMMMMMMM
ADAMTS8       IIIIIIMMDDDDDDMMMMMM
NDUFAB1       IIIIIIMMDDDDDDMMMMMM
LRRFIP2       DDDIDDMMIIIIMMMMMMM
FAT1          IIIIIIMMDDDDDDMMMMMM
FABP3         IIIDIDMMIDDDMMMMMMM
PDE4DIP       IDIDIDMMIDMMMMMMMM
FAM189A2      IIIIIIMMDDDDDDMMMMMM
FARP1         IIIIIIMMDDDDDDMMMMMM
PRSS35        IIIDIDMMDDMMMMMMMM
TCF7L1        IIIDIDMMDDMMMMMMMM
LAMA4         IIIDIDMMDDMMMMMMMM
ID2           IIIDIDMMDDMMMMMMMM
CPVL          DDDIDDMIIIIMMMMMMMM
EIF1B         DDDIDDMIIIIMMMMMMMM
HIF1A-AS3     DDDIDIDMMMMMMMMMVVV
PRKAA2        DDDIDIDMMMMMMMMMVVV
DACH2         DDDIDIDMMMMMMMMMVVV
CYP7B1        DDDIDIDMMMMMMMMMVVV
GPRIN3        DDDIDIDMMMMMMMMMVVV
MRPL41        DDDIDIDMMMMMMMMMVVV
RNF216        DDDIDIDMMMMMMMMMVVV
AC073050.1    IIIIIIIMMMMMMMWWWWWDD
INTS6L        DDDIDIDMMMMMMMMMVVV
MITF          DDDIDIDMMMMMMMMMVVV
CASC15        IIDIDIDMMMMMMMMDMM
TXLNB         IDDIDIDMMMMMMMMIMM
PLPP7         IDDIDIDMMMMMMMMIMM
DLG2          DDDIDDDMMVVVIIMMMMMM
COL13A1       DDDIDDDMMMMIIIIIMMMM
RCAN2         DDDIDIDMMMIIIMMMMMM
ABRA          IIIIIDIMMDDDDDMMMMMM
ANGPT1        IIIDIDIMMIIIDDDDDMMMMD
ARHGAP24      IIIIIDIMMDDDDDMMMMMM
ARHGAP17      DDDIDIDMMIIIMMMMMMM
CRIP2         DDDIDDDMMIIIIIMMMMMM
MRPS6         DDDIDDDMMIIIIIMMMMMM
ITPR2         IIIIIDIMMDDDDDMMMMMM
LMTK2         DDDIDDDMIIIIIMMMMMMM
AC009878.2    DDDIDDDMIIIIIIDMMMMMM
ITGA6         IIIDIDIMDDDMMMMMMMM
NEDD4L        DDDIDDDMIIIIIIDMMMMMM
EPB41L3       IIIIIDIMDDDDDMMMMMMM
THSD4         DDDIDDDMIIIIIIIDDMMMMM
AC147651.1    DDDIDDDMIIIIIMMMMMMM
NACA          DDIDIDIDMMMMMMMMMVV
NRXN1         DDIDIDIDMMMMMMMMMVV
NEGR1         IIIDIDIDMMMMMMMMMWW
TTC7B         DDIDIDIDMMMMMMMMMVV
OSBPL2        DDIDIDIDMMMMMMMMMVV
LTBP1         DDIDIDIDMMMMMMMMMVV
CAMK2G        IIIDIDIDMMMMMMMMMWW
KANSL1        DDIDIDIDMMMMMMMMMVV
CPEB3         IDIDIDIDMMMMMMMMMM
B4GALNT3      IDIDIDIDMMMMMMMMMM
OXCT1         IDIDIDIDMMMMMMMMMM
ZMIZ1         IDIDIDIDMMMMMMMMMM
RPL27A        DDIDIDIDMMMMMMMMMII
RPS24         DDIDIDIDMMMMMMMMMII
APBA1         DDIDIDIDMMMMMMMMMII
TGFB2         IIIDIDIDMMMMMMMMDDM
VDAC3         DDDDDDDIVVVVVIMMMMMMM
RPS11         DDDDDDDIVVVVVIMMMMMMM
ADARB2        DDIDIDIDMMMMMIIMMMM
CLSTN2        IIIDIDIDMMMMDDMMMMM
PRDX1         DDDIDDDDMMIIIIMMMMIIID
GRK5          DDDIDIDDMMIIIIMMMMMM
MYLK3         IIIDIDIDMMDDMMMMMMM
ATP5MC2       DDDIDIDDMMIIIIMMMMMM
CXCL12        DDDIDIDDMMIIIIMMMMMM
PLCB1         IIIIIDIIMDDDDDDMMMMMID
ROR2          DDDIDIDDMIIIIMMMMMMM
MMP15         DDDIDIDIDMMMMMMMMVVV
BANF1         DDDIDIDIDMMMMMMMMVVV
PAPPA         IIDIDIDIDMMMMMWWWWWIIII
UQCRB         DDDIDIDIDMMMMMMMMIII
SLC24A3       IIDIDIDIDMMMMMMMMDM
FYN           IDDIDIDIDMMMMMMMIMM
PPP1R3B       DDIDIDIDIDMMMMMMMMVV
CDK18         DDIDIDIDIDMMMMMMMMVV
ADAMTS9-AS2   DDIDIDIDIDMMMMMMMMVV
WDR60         DDIDIDIDIDMMMMMMMMVV
IDH3A         IDIDIDIDIDMMMMMMMMM
NID1          IDIDIDIDIDMMMMMMMMM
GPX3          IDIDIDIDIDMMMMMMMMM
VASH1         IDIDIDIDIDMMMMMMMMM
AEBP2         IDIDIDIDIDMMMMMMMMM
BMT2          IDIDIDIDIDMMMMMMMMM
VCAN          IDIDIDIDIDMMMMMMMMM
DYRK1A        IDIDIDIDIDMMMMMMMMM
RPL38         IDIDIDIDIDMMMMMMMMM
RBMS3         DDIDIDIDIDMMMMMMMMII
EMC10         DDIDIDIDIDMMMMMMMMII
SH3D19        DDIDIDIDIDMMMMMMMMII
EPS8          DDIDIDIDIDMMMMMMMIIID
FNIP2         DDIDIDIDIDMMMMMMIIMM
RICTOR        DDIDIDIDIDMMMMMMIIMM
MIR100HG      IIIIIIIIIIMMIIDDDDDDDDDDDD
KCNMB2        IIIIIIIIIIMMIIDDDDDDDDDDDD
ZNF385B       IIIIIIIIIIMMIIDDDDDDDDDDDD
FAM155A       DDIDIDIDIDDMMMMMMMVVV
PATJ          IDIDIDIDIDDMMMMMMIMM
L3MBTL4       IDIDIDIDIDDDMMMMMMMVV
CAV1          IIIDIDIDIDDDMMMMMMMM
CAV2          IIIDIDIDIDDDMMMMMMMM
SIK3          IIIDIDIDIDDDMMMMMMMM
AC020909.2    IIIDIDIDIDDDMMMMMMMM
NAMPT         IIIDIDIDIDDDMMMMMMMM
TMEM108       IIIDIDIDIDDDMMMMMMMM
WNK1          IIIDIDIDIDDDMMMMMMMM
ANK2          IIIDIDIDIDDDMMMMMMMM
PDLIM1        IIIDIDIDIDDDMMMMMMMM
HOMER1        IIIDIDIDIDDDMMMMMMMM
AC109466.1    IIIDIDIDIDDDMMMMMMMM
AL078590.2    IIIDIDIDIDDDMMMMMMMM
SETBP1        IIIDIDIDIDDDMMMMMMMM
NAV2          IIIDIDIDIDDDMMMMMMMM
CRYAB         IIIDIDIDIDDDMMMMMMMM
RAPGEF2       IIIDIDIDIDDDMMMMMMMM
ATAD1         IIIDIDIDIDDDMMMMMMMM
PCMTD2        IIIDIDIDIDDDMMMMMMMM
INPP5A        IIIDIDIDIDDDMMMMMMMM
PRELID2       IIIDIDIDIDDDMMMMMMMM
PICALM        IIIDIDIDIDDDMMMMMMMM
ACTN2         IIIDIDIDIDDDMMMMMMMM
CTNNA1        IIIDIDIDIDDDMMMMMMMM
DDR2          IDIDIDIDIDDDMMMMMIIMM
CMYA5         IDIDIDIDIDDDDMMMMMMIII
Z85996.3      IIIDIDIDIDIDDDMMMMMMM
SGSM1         IIIDIDIDIDIDDDMMMMMMM
CPNE4         IIIDIDIDIDIDDDMMMMMMM
KANSL1L       IIIDIDIDIDIDDDMMMMMMM
LINC01184     IIIDIDIDIDIDDDMMMMMMM
MCEMP1        IIIDIDIDIDIDDDMMMMMMM
ERVMER61-1    IIIDIDIDIDIDDDMMMMMMM
THSD7A        IIIDIDIDIDIDDDMMMMMMM
COL1A2        IIIDIDIDIDIDDDMMMMMMM
NTN4          IIIDIDIDIDIDDDMMMMMMM
ADAM19        IIIDIDIDIDIDDDMMMMMMM
RNF150        IIIDIDIDIDIDDDMMMMMMM
LINC01374     IIIDIDIDIDIDDDMMMMMMM
CCSER1        IIIDIDIDIDIDDDMMMMMMM
RPL34         IIIDIDIDIDIDDDMMMMMMM
MLLT10        IIIDIDIDIDIDDDMMMMMMM
RAPGEF4       IIIDIDIDIDIDDDMMMMMMM
TENM3         IIIDIDIDIDIDDDMMMMMMM
LIMCH1        IIIDIDIDIDIIDDDDMMMMMM
WDFY3         IIIDIDIDIDIIDDDDMMMMMM
CTNNB1        IIIDIDIDIDIIDDDDMMMMMM
CKB           IIIDIDIDIDIIDDDDMMMMMM
SH3KBP1       IIIDIDIDIDIIDDDDMMMMMM
EPHA7         IIIDIDIDIDIIDDDDMMMMMM
SLCO3A1       IIIDIDIDIDIIDDDDMMMMMM
RASAL2        IIIDIDIDIDIIDDDDMMMMMM
DUOXA2        IIIDIDIDIDIIDDDDMMMMMM
OTOGL         IIIDIDIDIDIIDDDDMMMMMM
MYO1D         IIIDIDIDIDIIDDDDMMMMMM
HTR4          IIIDIDIDIDIIDDDDMMMMMM
DPYD          IIIDIDIDIDIIDDDDMMMMMM
KIF13A        IIIDIDIDIDIIDDDDMMMMMM
PRRX1         IIIDIDIDIDIIDDDDMMMMMM
CACNB2        IIIDIDIDIDIIDDDDMMMMMID
INHBA         IIIDIDIDIDIIDDDDMMMIIIDDD
UST           IIIDIDIDIDIIIDDDDMMMMMD
DOK6          IIIDIDIDIDIIIDDDDDMMMMM
RABGAP1L      IIIDIDIDIDIIDDDDDDMMMIIID
DGKH          IIIDIDIDIDIIDDDDDDMMMIIID
PDE1C         IIIDIDIDIDIIIIDDDDDDMMMID
```

Example 116 (unknown):
```unknown
102        PITX2
524       DCBLD2
839        PANCR
1089       PCDH7
1254        NAV3
          ...   
573          UST
9           DOK6
539     RABGAP1L
1267        DGKH
808        PDE1C
Name: genes, Length: 1546, dtype: object
```

Example 117 (unknown):
```unknown
ClusterUtils.run_clustering(aligner, metric='levenshtein', DIST_THRESHOLD=0.4)
```

Example 118 (unknown):
```unknown
compute distance matrix
using levenshtein distance metric
run agglomerative clustering |  0.4
silhouette_score:  0.1241659753131663
```

Example 119 (unknown):
```unknown
ClusterUtils.visualise_clusters(aligner,n_cols = 4, figsize= (10,6))
```

Example 120 (unknown):
```unknown
ClusterUtils.print_cluster_average_alignments(aligner)
```

Example 121 (unknown):
```unknown
cluster:  0 IIDDMMMMMMMMMVVVDDD 7
cluster:  1 IIIIMMMMIDDDDMMMMMD 6
cluster:  2 IIIDMMMDDMMMMMMMM 1091
cluster:  3 IIIDIDIDIDIIIIIDDDDDDMMMD 15
cluster:  4 IIDDDMMMMMMMIMMMVD 15
cluster:  5 MIIIMMMMMMMMMMWWW 3
cluster:  6 IIDDDDDMMMIIIMMMMMM 89
cluster:  7 IIDMMMMMMVVVVVDDMWWWW 3
cluster:  8 MIIMVVVDDDDDMMMMMMM 24
cluster:  9 IIIDDDMWWWIIIMMMMMMM 4
cluster:  10 IIIDDDDDDMVMVMVMMMMM 5
cluster:  11 IIDDMMMMMVVDDMMMMM 6
cluster:  12 IDDDDDMWWIIIIIIIIIIIIDDDDDD 1
cluster:  13 IIIDIDIDIDIIIIIIIIDDDDDDDDDD 13
cluster:  14 MMMMMMMMMWWWIIIMM 7
cluster:  15 MMMMWWWWWDMMMIDIIIIII 1
cluster:  16 IIIDIDIDMWMMMMWWWWWIIII 1
cluster:  17 MIMMMMMMMMMMMWM 2
cluster:  18 MMIIIDDDDDMMMMMVMVVD 2
cluster:  19 MMMMMMMMVVVVVVDDDDDD 4
cluster:  20 IIMMIDDDDDMMMIMMMVM 7
cluster:  21 MMMMMMMMVIIMMMDDD 1
cluster:  22 IDMMMIDDMMMMMMMMV 237
cluster:  23 DDDDDMMMIIIIIIIIIIDDDMWW 1
cluster:  24 IDDMIIMVMIDDDMMMMMM 1
```

Example 122 (unknown):
```unknown
# Initialize an empty dictionary
cluster_genes_dict = {}

# Loop through each gene and its corresponding cluster ID
for gene, cluster_id in zip(aligner.gene_list, aligner.cluster_ids):
    # Check if the cluster ID is already in the dictionary
    if cluster_id not in cluster_genes_dict:
        # If not, create a new list for this cluster ID
        cluster_genes_dict[cluster_id] = []
    # Append the gene to the list for this cluster ID
    cluster_genes_dict[cluster_id].append(gene)
```

Example 123 (unknown):
```unknown
clus_of_interest=13
```

Example 124 (python):
```python
cluster_genes_dict[clus_of_interest]
print(len(cluster_genes_dict[clus_of_interest]))
```

Example 125 (unknown):
```unknown
GENE_SUBSET = cluster_genes_dict[clus_of_interest]
aligner.get_aggregate_alignment_for_subset(GENE_SUBSET)
```

Example 126 (unknown):
```unknown
Average Alignment:  IIIDIDIDIDIIIIIIIIDDDDDDDDDD
```

Example 127 (python):
```python
for gene in GENE_SUBSET:
    if len(GENE_SUBSET)<150:
        print(gene, aligner.results_map[gene].alignment_str)
        VisualUtils.plotTimeSeries(gene, aligner, vs)
        plt.show()
```

Example 128 (unknown):
```unknown
PITX2 IIIDIDIDIDIIIIIIIIDDDDDDDDDD
```

Example 129 (unknown):
```unknown
DCBLD2 IIIDIDIDIDIIIIIIIIDDDDDDDDDD
```

Example 130 (unknown):
```unknown
PLCL1 IIMMIDIDIIIIIIIIDDDDDDDDDD
```

Example 131 (unknown):
```unknown
PANCR IIIDIDIDIDIIIIIIIIDDDDDDDDDD
```

Example 132 (unknown):
```unknown
MIR100HG IIIIIIIIIIMMIIDDDDDDDDDDDD
```

Example 133 (unknown):
```unknown
KCNMB2 IIIIIIIIIIMMIIDDDDDDDDDDDD
```

Example 134 (unknown):
```unknown
KCNMB2-AS1 MIIIIDIDIDIIIIIIIDDDDDDDDDD
```

Example 135 (unknown):
```unknown
PCDH7 IIIDIDIDIDIIIIIIIIDDDDDDDDDD
```

Example 136 (unknown):
```unknown
ZNF385B IIIIIIIIIIMMIIDDDDDDDDDDDD
```

Example 137 (unknown):
```unknown
NAV3 IIIDIDIDIDIIIIIIIIDDDDDDDDDD
```

Example 138 (unknown):
```unknown
VASH2 IIIDIDIDIDIIIIIIIIDDDDDDDDDD
```

Example 139 (unknown):
```unknown
NTM IIIDIDIDIDIIIIIIIIDDDDDDDDDD
```

Example 140 (unknown):
```unknown
ROR1 IIIDIDIDIDIIIIIIIIDDDDDDDDDD
```

Example 141 (python):
```python
import pandas as pd

gene_list = aligner.gene_list
cluster_ids = aligner.cluster_ids

# Convert them into a pandas DataFrame
cluster_df = pd.DataFrame({
    'gene': gene_list,
    'cluster_id': cluster_ids
})


LEFT_genes_set = set(gene_lists['LEFT_genes'])
RIGHT_genes_set = set(gene_lists['RIGHT_genes'])
COMMON_genes_set = set(gene_lists['COMMON_genes'])

# Add columns to cluster_df for LEFT, RIGHT, and COMMON
cluster_df['LEFT'] = cluster_df['gene'].apply(lambda x: x in LEFT_genes_set)
cluster_df['RIGHT'] = cluster_df['gene'].apply(lambda x: x in RIGHT_genes_set)
cluster_df['COMMON'] = cluster_df['gene'].apply(lambda x: x in COMMON_genes_set)

cluster_df
```

Example 142 (unknown):
```unknown
list_for_testing = cluster_df[(cluster_df['cluster_id']==clus_of_interest)&(cluster_df['LEFT']==True)].gene.tolist()
GO_plot(gene_list=list_for_testing, display_table=True, n_genes=n_genes)
```

Example 143 (unknown):
```unknown
MSigDB_Hallmark_2020
```

Example 144 (unknown):
```unknown
KEGG_2021_Human
```

Example 145 (unknown):
```unknown
GO_Biological_Process_2023
```

Example 146 (unknown):
```unknown
Reactome_2022
```

Example 147 (unknown):
```unknown
list_for_testing = cluster_df[(cluster_df['cluster_id']==clus_of_interest)&(cluster_df['RIGHT']==True)].gene.tolist()
GO_plot(gene_list=list_for_testing, display_table=True, n_genes=n_genes)
```

Example 148 (unknown):
```unknown
MSigDB_Hallmark_2020
```

Example 149 (unknown):
```unknown
KEGG_2021_Human
```

Example 150 (unknown):
```unknown
GO_Biological_Process_2023
```

Example 151 (unknown):
```unknown
Reactome_2022
```

Example 152 (unknown):
```unknown
list_for_testing = cluster_df[(cluster_df['cluster_id']==clus_of_interest)].gene.tolist()
GO_plot(gene_list=list_for_testing, display_table=True, n_genes=n_genes)
```

Example 153 (unknown):
```unknown
MSigDB_Hallmark_2020
```

Example 154 (unknown):
```unknown
KEGG_2021_Human
```

Example 155 (unknown):
```unknown
GO_Biological_Process_2023
```

Example 156 (unknown):
```unknown
Reactome_2022
```

---

## Read in adata¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/Xenium/no2_integration_and_cell-type-annotation/no04_mark_multi_celltypes_coarse.html

**Contents:**
- Read in adata¶
- Plot leiden clusters¶
- Mark ?suspicious clusters in the center¶
- Save¶

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os

import celltypist
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
-----
celltypist          1.6.3
matplotlib          3.10.1
numpy               2.1.3
pandas              2.2.3
scanpy              1.11.1
seaborn             0.13.2
session_info        v1.0.1
-----
```

Example 4 (python):
```python
PIL                 11.1.0
anndata             0.11.4
asciitree           NA
asttokens           NA
certifi             2025.01.31
charset_normalizer  3.4.1
cloudpickle         3.1.1
comm                0.2.2
cycler              0.12.1
cython_runtime      NA
dask                2024.11.2
dateutil            2.9.0.post0
debugpy             1.8.13
decorator           5.2.1
exceptiongroup      1.2.2
executing           2.2.0
h5py                3.13.0
idna                3.10
igraph              0.11.8
importlib_metadata  NA
ipykernel           6.29.5
jedi                0.19.2
jinja2              3.1.6
joblib              1.4.2
kiwisolver          1.4.8
legacy_api_wrap     NA
leidenalg           0.10.2
llvmlite            0.44.0
markupsafe          3.0.2
mpl_toolkits        NA
msgpack             1.1.0
natsort             8.4.0
numba               0.61.0
numcodecs           0.13.1
packaging           24.2
parso               0.8.4
patsy               1.0.1
platformdirs        4.3.7
prompt_toolkit      3.0.50
psutil              7.0.0
pure_eval           0.2.3
pyarrow             19.0.1
pydev_ipython       NA
pydevconsole        NA
pydevd              3.2.3
pydevd_file_utils   NA
pydevd_plugins      NA
pydevd_tracing      NA
pygments            2.19.1
pyparsing           3.2.3
pytz                2025.2
requests            2.32.3
scipy               1.15.2
six                 1.17.0
sklearn             1.5.2
stack_data          0.6.3
statsmodels         0.14.4
tblib               3.1.0
texttable           1.7.0
threadpoolctl       3.6.0
tlz                 1.0.0
toolz               1.0.0
tornado             6.4.2
traitlets           5.14.3
typing_extensions   NA
urllib3             2.3.0
wcwidth             0.2.13
yaml                6.0.2
zarr                2.18.3
zipp                NA
zmq                 26.3.0
zoneinfo            NA
```

Example 5 (unknown):
```unknown
-----
IPython             8.34.0
jupyter_client      8.6.3
jupyter_core        5.7.2
-----
Python 3.10.14 | packaged by conda-forge | (main, Mar 20 2024, 12:45:18) [GCC 12.3.0]
Linux-4.18.0-553.45.1.el8_10.x86_64-x86_64-with-glibc2.28
-----
Session information updated at 2025-05-30 00:11
```

Example 6 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 7 (python):
```python
objects_dir = '/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium'
path_adata = f'{objects_dir}/C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat_5K_filtered_raw.h5ad'
adata = sc.read_h5ad(path_adata)
adata
```

Example 8 (unknown):
```unknown
AnnData object with n_obs × n_vars = 362277 × 5001
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_3.0'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1'
    uns: 'celltypist_mid_fin_colors', 'pca', 'pca_harmony', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.2_colors', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.3_colors', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.4_colors', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.5_colors', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_0.8_colors', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.0_colors', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_1.5_colors', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_2.0_colors', 'pca_harmony_leiden_3.0', 'pca_harmony_leiden_3.0_colors', 'tissue_block_id_colors', 'umap'
    obsm: 'X_pca', 'X_pca_harmony', 'X_umap_pca_harmony', 'spatial'
    varm: 'PCs'
    obsp: 'pca_harmony_connectivities', 'pca_harmony_distances'
```

Example 9 (unknown):
```unknown
os.getcwd()
```

Example 10 (unknown):
```unknown
'/rfs/project/rfs-iCNyzSAaucw/kk837/notebooks/Foetal/Xenium/5K/C194_Hst45/CamU'
```

Example 11 (python):
```python
adata.X.data[:5]
```

Example 12 (unknown):
```unknown
array([5., 1., 1., 2., 2.], dtype=float32)
```

Example 13 (unknown):
```unknown
latent_space = 'pca_harmony'
leiden_keys=[]
for res in [0.2,0.3,0.4,0.5,0.8,1.0,1.5,2.0,3.0]:
    leiden_key = f"leiden_{res}"
    key_added=f"{latent_space}_{leiden_key}"
    leiden_keys.append(key_added)
```

Example 14 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                    color=leiden_keys,
                   ncols=4
                   )
```

Example 15 (python):
```python
# plot celltypist prediction, for reference
sc.pl.embedding(adata,
                    basis=f"X_umap_{latent_space}",
                    color=['celltypist_coarse2midmod','conf_score_coarse2midmod'],
                   ncols=4,wspace=0.7
                   )
```

Example 16 (python):
```python
sc.pl.embedding(adata,
                    basis=f"X_umap_{latent_space}",
                    color=['celltypist_coarse','conf_score_coarse'],
                   ncols=4,wspace=0.5
                   )
```

Example 17 (unknown):
```unknown
leiden_key_sel = 'pca_harmony_leiden_1.0'
```

Example 18 (python):
```python
sc.pl.embedding(adata,
                    basis=f"X_umap_{latent_space}",
                    color=leiden_key_sel,
                   ncols=4
                   )
```

Example 19 (python):
```python
markers = {
    'CM': ['TTN', 'TNNT2','MYH6',
        'MYH7', 'MYL2', 'FHL2', # vCN
            'NPPA', 'MYL7', 'MYL4'], #aCM
    'FB': ['DCN', 'GCN', 'PDGFRA','COL1A1','COL1A2'],
    'EC': ['VWF', 'PECAM1', 'CDH5','RGCC', 'FABP5',],
    'Peri': ['RGS5', 'ABCC9', 'KCNJ8'],
    'SMC': ['MYH11', 'TAGLN', 'ACTA2'],
    'Meso': ['MSLN', 'WT1', 'BNC1'],
    'Neuro': ['PLP1', 'NRXN1', 'NRXN3','PRPH', 'NEFL', 'NEFM', 'NEFH', 'STMN2', 'SST', 'SLC10A4', 'SLC18A3', 'TH', 'NPY'],
    'Adipo': ['GPAM', 'FASN', 'LEP'],
    'Myelo': ['CD14', 'C1QA', 'CD68','LYVE1','TIMD4'],
    'Lymph': ['CD8A', 'IL7R', 'CD40LG'],
    'B': ['CD19','SDC1','IGHG1','IGHA1'],
    'Mast': ['KIT', 'CPA3'],
}

for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in adata.var_names]

sc.tl.dendrogram(adata, groupby=leiden_key_sel)
sc.pl.dotplot(adata,
              markers, 
              groupby=leiden_key_sel,
              dendrogram=True,
              standard_scale="var",
              color_map="Reds",
              swap_axes=False,
             )
```

Example 20 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: CM, FB, EC, etc.
```

Example 21 (python):
```python
# mark potentially multiple cell type cells
clusters_to_mark = ['5','10']
sc.pl.embedding(adata,
                    basis=f"X_umap_{latent_space}",
                    color=leiden_key_sel,
                groups=clusters_to_mark,
                   ncols=4
                   )
```

Example 22 (python):
```python
adata.obs['multi_celltypes_coarse'] = 'No'
obsnames = adata.obs_names[adata.obs[leiden_key_sel].isin(clusters_to_mark)]
adata.obs.loc[obsnames,'multi_celltypes_coarse']='Yes'
sc.pl.embedding(adata,
                    basis=f"X_umap_{latent_space}",
                    color='multi_celltypes_coarse',
                
                   )
```

Example 23 (unknown):
```unknown
... storing 'multi_celltypes_coarse' as categorical
```

Example 24 (python):
```python
adata.obs_names
```

Example 25 (unknown):
```unknown
Index(['C194-HEA-0-FFPE-1_aaaafiob-1', 'C194-HEA-0-FFPE-1_aaaajopp-1',
       'C194-HEA-0-FFPE-1_aaaanald-1', 'C194-HEA-0-FFPE-1_aaabdjka-1',
       'C194-HEA-0-FFPE-1_aaabglmg-1', 'C194-HEA-0-FFPE-1_aaachofo-1',
       'C194-HEA-0-FFPE-1_aaadibmf-1', 'C194-HEA-0-FFPE-1_aaaenklb-1',
       'C194-HEA-0-FFPE-1_aaafclmo-1', 'C194-HEA-0-FFPE-1_aaafjldl-1',
       ...
       'Hst45-HEA-0-FFPE-1_oigbdbao-1', 'Hst45-HEA-0-FFPE-1_oigbenna-1',
       'Hst45-HEA-0-FFPE-1_oigbmoed-1', 'Hst45-HEA-0-FFPE-1_oigcbndd-1',
       'Hst45-HEA-0-FFPE-1_oigcinom-1', 'Hst45-HEA-0-FFPE-1_oigcomgh-1',
       'Hst45-HEA-0-FFPE-1_oigeappc-1', 'Hst45-HEA-0-FFPE-1_oigeiifh-1',
       'Hst45-HEA-0-FFPE-1_oigeklem-1', 'Hst45-HEA-0-FFPE-1_oigenchi-1'],
      dtype='object', length=362277)
```

Example 26 (python):
```python
adata.write(path_adata)
path_adata
```

Example 27 (unknown):
```unknown
'/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium/C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat_5K_filtered_raw.h5ad'
```

---

## Import modules¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig4_Cardiomyocytes/VentricularCardiomyocytes/TransmuralAxis/tm-axis_cell2loc_compact-trab_transition.html

**Contents:**
- Import modules¶
- Functions¶
- Read in adata¶
- Add axis¶
- Plot transmural axis¶
- Concatenate OCT and FFPE object¶
- Plot cells along the axis and time¶
- Cell type transition along transmural axis¶
- Plot compact and trabeculated CM for each age¶

**Examples:**

Example 1 (python):
```python
import os
import numpy as np
import scanpy as sc
import anndata
import pandas as pd
import matplotlib as mpl
import matplotlib.pyplot as plt
import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 
import seaborn as sns

import warnings
warnings.simplefilter(action='ignore', category=FutureWarning)
warnings.simplefilter("ignore", UserWarning)
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
-----
anndata             0.9.2
matplotlib          3.7.2
numpy               1.24.4
pandas              1.5.3
scanpy              1.9.3
seaborn             0.12.2
session_info        1.0.0
-----
```

Example 4 (python):
```python
PIL                         10.0.0
anyio                       NA
arrow                       1.2.3
asttokens                   NA
attr                        23.1.0
attrs                       23.1.0
babel                       2.12.1
backcall                    0.2.0
brotli                      1.0.9
certifi                     2023.11.17
cffi                        1.15.1
charset_normalizer          3.2.0
colorama                    0.4.6
comm                        0.1.4
cycler                      0.10.0
cython_runtime              NA
dateutil                    2.8.2
debugpy                     1.6.8
decorator                   5.1.1
defusedxml                  0.7.1
executing                   1.2.0
fastjsonschema              NA
fqdn                        NA
h5py                        3.9.0
idna                        3.4
igraph                      0.10.6
importlib_metadata          NA
importlib_resources         NA
ipykernel                   6.25.1
isoduration                 NA
jedi                        0.19.0
jinja2                      3.1.2
joblib                      1.3.2
json5                       NA
jsonpointer                 2.0
jsonschema                  4.19.0
jsonschema_specifications   NA
jupyter_events              0.7.0
jupyter_server              2.7.1
jupyterlab_server           2.24.0
kiwisolver                  1.4.5
leidenalg                   0.10.1
llvmlite                    0.40.1
markupsafe                  2.1.3
mpl_toolkits                NA
natsort                     8.4.0
nbformat                    5.9.2
numba                       0.57.1
overrides                   NA
packaging                   23.1
parso                       0.8.3
patsy                       0.5.3
pexpect                     4.8.0
pickleshare                 0.7.5
pkg_resources               NA
plotly                      5.16.1
prometheus_client           NA
prompt_toolkit              3.0.39
psutil                      5.9.5
ptyprocess                  0.7.0
pure_eval                   0.2.2
pydev_ipython               NA
pydevconsole                NA
pydevd                      2.9.5
pydevd_file_utils           NA
pydevd_plugins              NA
pydevd_tracing              NA
pygments                    2.16.1
pyparsing                   3.0.9
pythonjsonlogger            NA
pytz                        2023.3
referencing                 NA
requests                    2.31.0
rfc3339_validator           0.1.4
rfc3986_validator           0.1.1
rpds                        NA
ruamel                      NA
scipy                       1.10.1
send2trash                  NA
six                         1.16.0
sklearn                     1.3.0
sniffio                     1.3.0
socks                       1.7.1
sphinxcontrib               NA
stack_data                  0.6.2
statsmodels                 0.14.0
texttable                   1.6.7
threadpoolctl               3.2.0
tornado                     6.3.3
traitlets                   5.9.0
typing_extensions           NA
uri_template                NA
urllib3                     2.0.4
wcwidth                     0.2.6
webcolors                   1.13
websocket                   1.6.2
yaml                        6.0.1
zipp                        NA
zmq                         25.1.1
zstandard                   0.19.0
```

Example 5 (unknown):
```unknown
-----
IPython             8.12.0
jupyter_client      8.3.0
jupyter_core        4.12.0
jupyterlab          4.0.5
notebook            7.0.2
-----
Python 3.8.17 (default, Jul  5 2023, 21:04:15) [GCC 11.2.0]
Linux-5.4.0-169-generic-x86_64-with-glibc2.17
-----
Session information updated at 2024-03-20 22:41
```

Example 6 (python):
```python
import importlib.util
import sys

spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)

spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/tissueaxis.py")
tissueaxis = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = tissueaxis
spec.loader.exec_module(tissueaxis)
```

Example 7 (python):
```python
sc.settings.set_figure_params(dpi=80,dpi_save=300)
```

Example 8 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk'
```

Example 9 (python):
```python
# This function selects the data for multiple slides from the spatial anndata object.
# Modified `select_slide` in cell2loc for multiple slides

# @param adata: anndata object with multiple spatial experiments
# @param s: list of selected slides
# @param s_col: column in adata.obs listing experiment name for each location

def select_multislide(adata, s, s_col='sample'):

    slides = adata[adata.obs[s_col].isin(s), :]
    s_keys = list(slides.uns['spatial'].keys())
    
    uns_dict = {}
    for ss in s:
        s_spatial=np.array(s_keys)[[ss in k for k in s_keys]][0]
        uns_dict[s_spatial]=slides.uns['spatial'][s_spatial]
    
    slides.uns['spatial'] = uns_dict

    return slides
```

Example 10 (python):
```python
adata_ffpe = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/visium-FFPE_foetal_raw.h5ad')
print(adata_ffpe.shape)
adata_oct_healthy = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/visium-OCT_foetal_raw.h5ad')
print(adata_oct_healthy.shape)
adata_oct_healthy_LV = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/visium-OCT-LV_foetal_raw.h5ad')
print(adata_oct_healthy_LV.shape)
adata_oct_t21 = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/visium-OCT-T21_foetal_raw.h5ad')
print(adata_oct_t21.shape)
```

Example 11 (unknown):
```unknown
(19973, 18085)
(14261, 33538)
(6767, 33538)
(11238, 33538)
```

Example 12 (python):
```python
# concatenate oct adata
adata_oct_healthy.obs['status'] = 'Healthy'
adata_oct_healthy_LV.obs['status'] = 'Healthy'
adata_oct_t21.obs['status'] = 'Trisomy21'

adata_oct = adata_oct_healthy.concatenate(
    [adata_oct_healthy_LV,adata_oct_t21],
    batch_key=None,
    index_unique=None,
    join="inner"
)

adata_oct.obs['status'].value_counts()
```

Example 13 (unknown):
```unknown
Healthy      21028
Trisomy21    11238
Name: status, dtype: int64
```

Example 14 (python):
```python
# log-normalise
sc.pp.normalize_total(adata_oct, target_sum=1e4)
sc.pp.log1p(adata_oct)

sc.pp.normalize_total(adata_ffpe, target_sum=1e4)
sc.pp.log1p(adata_ffpe)

# reset var_names
adata_oct.var.set_index('SYMBOL',inplace=True)
adata_ffpe.var.set_index('SYMBOL',inplace=True)
```

Example 15 (python):
```python
# get all celltypes
# only the LV cell types will be appeared since concatenated with LV object
all_celltypes = adata_oct.obsm['q05_cell_abundance_w_sf'].columns
all_celltypes = [x.replace('q05cell_abundance_w_sf_','') for x in all_celltypes]
print(len(all_celltypes))
print(all_celltypes[:5])
```

Example 16 (unknown):
```unknown
41
['VentricularCardiomyocytesLeftCompact', 'VentricularCardiomyocytesLeftTrabeculated', 'VentricularConductionSystemProximal', 'VentricularConductionSystemDistal', 'CoronaryVesselAdventitialFibroblasts']
```

Example 17 (python):
```python
# subset cell types in ffpe
adata_ffpe.obsm['q05_cell_abundance_w_sf'] = adata_ffpe.obsm['q05_cell_abundance_w_sf'][[f'q05cell_abundance_w_sf_{x}' for x in all_celltypes]]

# modify cell type names
celltype_rename = utils.read_pkl('/nfs/team205/heart/anndata_objects/Foetal/fine_grain_rename_dictionary.pkl')
adata_ffpe.obsm['q05_cell_abundance_w_sf'].columns = [x.replace(x.split('q05cell_abundance_w_sf_')[1],celltype_rename[x.split('q05cell_abundance_w_sf_')[1]]) \
                                                      for x in adata_ffpe.obsm['q05_cell_abundance_w_sf'].columns]
adata_oct.obsm['q05_cell_abundance_w_sf'].columns = [x.replace(x.split('q05cell_abundance_w_sf_')[1],celltype_rename[x.split('q05cell_abundance_w_sf_')[1]]) \
                                                      for x in adata_oct.obsm['q05_cell_abundance_w_sf'].columns]
```

Example 18 (python):
```python
# add
ffpe_samples = list(set(adata_ffpe.obs['sample']))
oct_samples = list(set(adata_oct.obs['sample']))

adata_oct.obs['transmural_axis'] = np.nan
adata_ffpe.obs['transmural_axis'] = np.nan

axis_files = os.listdir('/nfs/team205/heart/visium_axis/foetal/transmural_LV')
for file in axis_files:
    # read in
    axis = pd.read_csv(f'/nfs/team205/heart/visium_axis/foetal/transmural_LV/{file}',index_col=0)
    # add to anndata
    if file.split('.csv')[0] in oct_samples:
        adata_oct.obs.loc[axis.index,'transmural_axis'] = axis.copy()
    elif file.split('.csv')[0] in ffpe_samples:
        adata_ffpe.obs.loc[axis.index,'transmural_axis'] = axis.copy()

# select spots with axis
# adata_oct = adata_oct[adata_oct.obs['transmural_axis'].isna()==False]
# adata_ffpe = adata_ffpe[adata_ffpe.obs['transmural_axis'].isna()==False]
```

Example 19 (python):
```python
pd.crosstab(adata_oct.obs['sample'],adata_oct.obs['transmural_axis'].isna()==False)
```

Example 20 (python):
```python
pd.crosstab(adata_ffpe.obs['sample'],adata_ffpe.obs['transmural_axis'].isna()==False)
```

Example 21 (python):
```python
pd.crosstab(adata_oct.obs['sample'],adata_oct.obs['cell2loc-map_group'])
```

Example 22 (unknown):
```unknown
slide_id = 'BHF_F_Hea13295831'
```

Example 23 (python):
```python
slide = select_multislide(adata_ffpe, [slide_id])
sc.pl.spatial(slide,color=['transmural_axis'],cmap='gist_rainbow',show=False)
plt.savefig(f"{figdir}/visium_tm-axis_{slide_id}.pdf", bbox_inches="tight",dpi=300)
```

Example 24 (python):
```python
# select spots with axis
adata_oct = adata_oct[adata_oct.obs['transmural_axis'].isna()==False]
adata_ffpe = adata_ffpe[adata_ffpe.obs['transmural_axis'].isna()==False]
# for later plotting
adata_oct_healthy = adata_oct_healthy[list(set(adata_oct_healthy.obs_names).intersection(adata_oct.obs_names))]
adata_oct_healthy_LV = adata_oct_healthy_LV[list(set(adata_oct_healthy_LV.obs_names).intersection(adata_oct.obs_names))]
```

Example 25 (python):
```python
# concatenate OCT and FFPE objects
adata_ffpe.obs['cell2loc-map_group'] = '16PCW_FFPE'

adata_all = adata_oct.concatenate(
    adata_ffpe,
    batch_key=None,
    index_unique=None,
    join="inner"
)
```

Example 26 (python):
```python
pd.crosstab(adata_all.obs['sample'],adata_all.obs['cell2loc-map_group'])
```

Example 27 (python):
```python
features_to_plot = 'VentricularCardiomyocytesLeftCompact'

tissueaxis.plot_axis_split(adata_all,
                           feature_to_plot=features_to_plot,
                           feature_type='celltype',
                           splitby='cell2loc-map_group',
                           groups_order=[ '5PCW', '7PCW','13PCW', '16PCW_FFPE','20PCW'],
                           figsize=(5, 3.5),
                           cbar_pos=(0, 0.7, 0.05, 0.18),
                           save=f'{figdir}/tm-axis_{features_to_plot}_per-week.pdf'
                          )
```

Example 28 (python):
```python
features_to_plot = 'VentricularCardiomyocytesLeftTrabeculated'

tissueaxis.plot_axis_split(adata_all,
                           feature_to_plot=features_to_plot,
                           feature_type='celltype',
                           splitby='cell2loc-map_group',
                           groups_order=[ '5PCW', '7PCW','13PCW', '16PCW_FFPE','20PCW'],
                           figsize=(5, 3.5),
                           cbar_pos=(0, 0.7, 0.05, 0.18),
                           save=f'{figdir}/tm-axis_{features_to_plot}_per-week.pdf'
                          )
```

Example 29 (python):
```python
features_to_plot = 'CoronaryCapillaryEndothelialCells'

tissueaxis.plot_axis_split(adata_all,
                           feature_to_plot=features_to_plot,
                           feature_type='celltype',
                           splitby='cell2loc-map_group',
                           groups_order=[ '5PCW', '7PCW','13PCW', '16PCW_FFPE','20PCW'],
                           figsize=(5, 3.5),
                           cbar_pos=(0, 0.7, 0.05, 0.18),
                           save=f'{figdir}/tm-axis_{features_to_plot}_per-week.pdf'
                          )
```

Example 30 (python):
```python
# only euploid
tissueaxis.plot_axis(adata_all[adata_all.obs['status']=='Healthy'],
                     axis_col='transmural_axis',
                     n_bin=15,
                     feature_type='celltype',
                     correlation=False,
                     threshold_cell_abundance=0.1,
                     min_max_scale=True,
                     figsize=(3, 7),
                     save=f'{figdir}/tm-axis_OCT-FFPE_celltypes_thresh-0p1.pdf'
                    )
```

Example 31 (python):
```python
# only euploid
tissueaxis.plot_axis(adata_all[adata_all.obs['status']=='Healthy'],
                     axis_col='transmural_axis',
                     n_bin=15,
                     feature_type='celltype',
                     correlation=False,
                     threshold_cell_abundance=0.4,
                     min_max_scale=True,
                     figsize=(3, 4)
                     # save=f'{figdir}/tm-axis_OCT_celltypes.pdf'
                    )
```

Example 32 (python):
```python
# OCT, cell type transition
# only euploid
tissueaxis.plot_axis(adata_oct[adata_oct.obs['status']=='Healthy'],
                     axis_col='transmural_axis',
                     n_bin=15,
                     feature_type='celltype',
                     correlation=False,
                     threshold_cell_abundance=0.1,
                     # save=f'{figdir}/tm-axis_OCT_celltypes.pdf'
                    )
```

Example 33 (python):
```python
# FFPE, cell type transition
# only euploid
tissueaxis.plot_axis(adata_ffpe[adata_ffpe.obs['status']=='Healthy'],
                     axis_col='transmural_axis',
                     n_bin=15,
                     feature_type='celltype',
                     correlation=False,
                     threshold_cell_abundance=0.03,
                    )
```

Example 34 (python):
```python
pd.crosstab(adata_oct_healthy.obs['sample'],adata_oct_healthy.obs['cell2loc-map_group'])
```

Example 35 (python):
```python
pd.crosstab(adata_oct_healthy_LV.obs['sample'],adata_oct_healthy_LV.obs['cell2loc-map_group'])
```

Example 36 (python):
```python
slide_id = 'HCAHeartST11605167'
slide = select_multislide(adata_oct_healthy, [slide_id])
cell2loc_out = slide.obsm['q05_cell_abundance_w_sf'].copy()
cell2loc_out.columns = [x.replace('q05cell_abundance_w_sf_','') for x in cell2loc_out.columns]
slide.obs[cell2loc_out.columns] = cell2loc_out.copy()
sc.pl.spatial(slide,color=['VentricularCardiomyocytesLeftCompact','VentricularCardiomyocytesLeftTrabeculated'],show=False)
```

Example 37 (unknown):
```unknown
[<Axes: title={'center': 'VentricularCardiomyocytesLeftCompact'}, xlabel='spatial1', ylabel='spatial2'>,
 <Axes: title={'center': 'VentricularCardiomyocytesLeftTrabeculated'}, xlabel='spatial1', ylabel='spatial2'>]
```

Example 38 (python):
```python
slide_id = 'HCAHeartST11605168'
slide = select_multislide(adata_oct_healthy, [slide_id])
cell2loc_out = slide.obsm['q05_cell_abundance_w_sf'].copy()
cell2loc_out.columns = [x.replace('q05cell_abundance_w_sf_','') for x in cell2loc_out.columns]
slide.obs[cell2loc_out.columns] = cell2loc_out.copy()
sc.pl.spatial(slide,color=['VentricularCardiomyocytesLeftCompact','VentricularCardiomyocytesLeftTrabeculated'],show=False)
```

Example 39 (unknown):
```unknown
[<Axes: title={'center': 'VentricularCardiomyocytesLeftCompact'}, xlabel='spatial1', ylabel='spatial2'>,
 <Axes: title={'center': 'VentricularCardiomyocytesLeftTrabeculated'}, xlabel='spatial1', ylabel='spatial2'>]
```

Example 40 (python):
```python
slide_id = 'HCAHeartST11605165'
slide = select_multislide(adata_oct_healthy, [slide_id])
cell2loc_out = slide.obsm['q05_cell_abundance_w_sf'].copy()
cell2loc_out.columns = [x.replace('q05cell_abundance_w_sf_','') for x in cell2loc_out.columns]
slide.obs[cell2loc_out.columns] = cell2loc_out.copy()
sc.pl.spatial(slide,color=['VentricularCardiomyocytesLeftCompact','VentricularCardiomyocytesLeftTrabeculated'],show=False)
```

Example 41 (unknown):
```unknown
[<Axes: title={'center': 'VentricularCardiomyocytesLeftCompact'}, xlabel='spatial1', ylabel='spatial2'>,
 <Axes: title={'center': 'VentricularCardiomyocytesLeftTrabeculated'}, xlabel='spatial1', ylabel='spatial2'>]
```

Example 42 (python):
```python
slide_id = 'HCAHeartST10317185'
slide = select_multislide(adata_oct_healthy, [slide_id])
cell2loc_out = slide.obsm['q05_cell_abundance_w_sf'].copy()
cell2loc_out.columns = [x.replace('q05cell_abundance_w_sf_','') for x in cell2loc_out.columns]
slide.obs[cell2loc_out.columns] = cell2loc_out.copy()
sc.pl.spatial(slide,color=['VentricularCardiomyocytesLeftCompact','VentricularCardiomyocytesLeftTrabeculated'],show=False)
```

Example 43 (unknown):
```unknown
[<Axes: title={'center': 'VentricularCardiomyocytesLeftCompact'}, xlabel='spatial1', ylabel='spatial2'>,
 <Axes: title={'center': 'VentricularCardiomyocytesLeftTrabeculated'}, xlabel='spatial1', ylabel='spatial2'>]
```

Example 44 (python):
```python
slide_id = 'HCAHeartST13162330'
slide = select_multislide(adata_oct_healthy_LV, [slide_id])
cell2loc_out = slide.obsm['q05_cell_abundance_w_sf'].copy()
cell2loc_out.columns = [x.replace('q05cell_abundance_w_sf_','') for x in cell2loc_out.columns]
slide.obs[cell2loc_out.columns] = cell2loc_out.copy()
sc.pl.spatial(slide,color=['VentricularCardiomyocytesLeftCompact','VentricularCardiomyocytesLeftTrabeculated'],show=False)
```

Example 45 (unknown):
```unknown
[<Axes: title={'center': 'VentricularCardiomyocytesLeftCompact'}, xlabel='spatial1', ylabel='spatial2'>,
 <Axes: title={'center': 'VentricularCardiomyocytesLeftTrabeculated'}, xlabel='spatial1', ylabel='spatial2'>]
```

Example 46 (python):
```python
slide_id = 'HCAHeartST13162331'
slide = select_multislide(adata_oct_healthy_LV, [slide_id])
cell2loc_out = slide.obsm['q05_cell_abundance_w_sf'].copy()
cell2loc_out.columns = [x.replace('q05cell_abundance_w_sf_','') for x in cell2loc_out.columns]
slide.obs[cell2loc_out.columns] = cell2loc_out.copy()
sc.pl.spatial(slide,color=['VentricularCardiomyocytesLeftCompact','VentricularCardiomyocytesLeftTrabeculated'],show=False)
```

Example 47 (unknown):
```unknown
[<Axes: title={'center': 'VentricularCardiomyocytesLeftCompact'}, xlabel='spatial1', ylabel='spatial2'>,
 <Axes: title={'center': 'VentricularCardiomyocytesLeftTrabeculated'}, xlabel='spatial1', ylabel='spatial2'>]
```

Example 48 (python):
```python
slide_id = 'BHF_F_Hea13295831'
slide = select_multislide(adata_ffpe, [slide_id])
cell2loc_out = slide.obsm['q05_cell_abundance_w_sf'].copy()
cell2loc_out.columns = [x.replace('q05cell_abundance_w_sf_','') for x in cell2loc_out.columns]
slide.obs[cell2loc_out.columns] = cell2loc_out.copy()
sc.pl.spatial(slide,color=['VentricularCardiomyocytesLeftCompact','VentricularCardiomyocytesLeftTrabeculated'],show=False)
```

Example 49 (unknown):
```unknown
[<Axes: title={'center': 'VentricularCardiomyocytesLeftCompact'}, xlabel='spatial1', ylabel='spatial2'>,
 <Axes: title={'center': 'VentricularCardiomyocytesLeftTrabeculated'}, xlabel='spatial1', ylabel='spatial2'>]
```

Example 50 (unknown):
```unknown
[<Axes: title={'center': 'VentricularCardiomyocytesLeftCompact'}, xlabel='spatial1', ylabel='spatial2'>,
 <Axes: title={'center': 'VentricularCardiomyocytesLeftTrabeculated'}, xlabel='spatial1', ylabel='spatial2'>]
```

Example 51 (unknown):
```unknown
cell2loc_out.columns
```

Example 52 (unknown):
```unknown
Index(['VentricularCardiomyocytesLeftCompact',
       'VentricularCardiomyocytesLeftTrabeculated',
       'VentricularConductionSystemProximal',
       'VentricularConductionSystemDistal',
       'CoronaryVesselAdventitialFibroblasts',
       'MyocardialInterstitialFibroblasts', 'SubEpicardialFibroblasts',
       'Myofibroblasts', 'LymphNodeFibroblasticReticularCells',
       'CoronarySmoothMuscleCells', 'CoronaryPericytes',
       'CoronaryArterialEndothelialCells', 'CoronaryVenousEndothelialCells',
       'CoronaryCapillaryEndothelialCells', 'EndocardialCells',
       'LymphaticEndothelialCells', 'MesothelialEpicardialCells',
       'EpicardiumDerivedCells', 'NeuronPrecursors', 'ChromaffinCells',
       'SympatheticNeurons', 'ParasympatheticNeurons', 'SchwannCellPrecursors',
       'SchwannCells', 'MonocytesMPOpos', 'Monocytes', 'MonocyteDerivedCells',
       'MacrophagesCX3CR1pos', 'MacrophagesTIMD4pos', 'MacrophagesLYVE1pos',
       'MacrophagesATF3pos', 'DendriticCellsType1', 'DendriticCellsMature',
       'PlasmacytoidDendriticCells', 'MastCells', 'TCellsCD4pos',
       'TCellsCD8pos', 'TregsCD4pos', 'BCellsMS4A1pos', 'NaturalKillerCells',
       'InnateLymphoidCells'],
      dtype='object')
```

---

## Loading packages¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/Visium/cell2location/wholeheart_with_DA/no1_reference-signatures_woPericardial.html

**Contents:**
- Loading packages¶
- Data, parameters¶
- Loading scRNA-seq reference data¶
- Remove cells¶
- Add Atria Ventricle key
- Estimation of reference cell type signatures (NB regression)¶

17277 rows × 58 columns

**Examples:**

Example 1 (python):
```python
import sys
import scanpy as sc
import anndata
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt 
import matplotlib as mpl

import cell2location
import scvi

from matplotlib import rcParams
rcParams['pdf.fonttype'] = 42 # enables correct plotting of text
import seaborn as sns
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
-----
anndata             0.7.8
cell2location       NA
matplotlib          3.5.0
numpy               1.20.3
pandas              1.3.4
scanpy              1.9.3
scvi                0.14.4
seaborn             0.11.2
session_info        1.0.0
-----
```

Example 4 (python):
```python
PIL                         8.4.0
absl                        NA
adjustText                  NA
arviz                       0.10.0
attr                        21.2.0
backcall                    0.2.0
beta_ufunc                  NA
binom_ufunc                 NA
cffi                        1.15.0
cftime                      1.5.1.1
colorama                    0.4.4
cycler                      0.10.0
cython_runtime              NA
dateutil                    2.8.2
debugpy                     1.5.1
decorator                   5.1.0
defusedxml                  0.7.1
deprecate                   0.3.0
descartes                   NA
dill                        0.3.4
docrep                      0.3.2
entrypoints                 0.3
fastprogress                0.2.7
fontTools                   4.28.1
fsspec                      2021.11.0
google                      NA
h5py                        3.6.0
igraph                      0.9.8
ipykernel                   6.5.1
ipython_genutils            0.2.0
ipywidgets                  7.6.5
jedi                        0.18.1
joblib                      1.1.0
kiwisolver                  1.3.2
lazylinker_ext              NA
leidenalg                   0.8.8
llvmlite                    0.37.0
matplotlib_inline           NA
mizani                      0.7.3
mpl_toolkits                NA
natsort                     8.0.0
nbinom_ufunc                NA
netCDF4                     1.5.8
numba                       0.54.1
numexpr                     2.7.3
opt_einsum                  v3.3.0
packaging                   21.3
palettable                  3.3.0
parso                       0.8.2
patsy                       0.5.2
pexpect                     4.8.0
pickleshare                 0.7.5
pkg_resources               NA
plotnine                    0.8.0
prompt_toolkit              3.0.22
ptyprocess                  0.7.0
pycparser                   2.21
pydev_ipython               NA
pydevconsole                NA
pydevd                      2.6.0
pydevd_concurrency_analyser NA
pydevd_file_utils           NA
pydevd_plugins              NA
pydevd_tracing              NA
pyexpat                     NA
pygments                    2.10.0
pymc3                       3.10.0
pynndescent                 0.5.5
pyparsing                   3.0.6
pyro                        1.7.0
pytorch_lightning           1.3.8
pytz                        2021.3
rich                        NA
scipy                       1.7.2
setuptools                  58.0.4
setuptools_scm              NA
six                         1.16.0
sklearn                     1.0.1
statsmodels                 0.13.1
storemagic                  NA
tensorboard                 2.7.0
texttable                   1.6.4
theano                      1.0.11
threadpoolctl               3.0.0
torch                       1.10.0+cu102
torchmetrics                0.6.0
tornado                     6.1
tqdm                        4.62.3
traitlets                   5.1.1
typing_extensions           NA
umap                        0.5.2
wcwidth                     0.2.5
xarray                      0.20.1
yaml                        5.4.1
zmq                         22.3.0
```

Example 5 (unknown):
```unknown
-----
IPython             7.29.0
jupyter_client      7.0.6
jupyter_core        4.9.1
notebook            6.4.6
-----
Python 3.9.7 (default, Sep 16 2021, 13:09:58) [GCC 7.5.0]
Linux-4.15.0-202-generic-x86_64-with-glibc2.27
-----
Session information updated at 2023-03-22 07:40
```

Example 6 (unknown):
```unknown
results_folder = '/nfs/team205/heart/cell2location/foetal'

# create paths and names to results folders for reference regression and cell2location models
ref_run_name = f'{results_folder}/reference_signatures/ver28Feb2023'
run_name = f'{results_folder}/cell2location_map/ver28Feb2023'
```

Example 7 (unknown):
```unknown
scRNA_path='/nfs/team205/heart/anndata_objects/Foetal/Feb28ObjectRaw.h5ad'
celltype_key='fine_grain'
```

Example 8 (python):
```python
# Read scRNA data
adata = sc.read(scRNA_path)
adata
```

Example 9 (unknown):
```unknown
AnnData object with n_obs × n_vars = 297473 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score'
    var: 'gene_name_scRNA-0', 'gene_id'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'age_colors', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'cycling_colors', 'donor_colors', 'fine_grain_colors', 'heart_or_greatvessels_colors', 'hvg', 'kit_10x_colors', 'mid_grain_colors', 'modality_colors', 'neighbors', 'phase_colors', 'region_colors', 'sex_colors', 'trimester_colors', 'umap', 'week_colors'
    obsm: 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 10 (python):
```python
adata.X.data[:10]
```

Example 11 (unknown):
```unknown
array([ 1.,  1.,  5.,  5.,  8.,  4., 13.,  4.,  3.,  1.], dtype=float32)
```

Example 12 (python):
```python
set(adata.obs[celltype_key])
```

Example 13 (unknown):
```unknown
{'AtrialCardiomyocytesCycling',
 'AtrialCardiomyocytesLeft',
 'AtrialCardiomyocytesRight',
 'AtrioventricularNodeCardiomyocytes',
 'BCells',
 'BCellsMS4A1pos',
 'ChromaffinCells',
 'CoronaryArterialEndothelialCells',
 'CoronaryCapillaryEndothelialCells',
 'CoronaryPericytes',
 'CoronarySmoothMuscleCells',
 'CoronaryVenousEndothelialCells',
 'CoronaryVesselAdventitialFibroblasts',
 'DendriticCellsMature',
 'DendriticCellsType1',
 'DuctusArteriosusSmoothMuscleCells',
 'EndocardialCells',
 'EndocardialCushionCells',
 'EpicardiumDerivedCells',
 'GreatVesselAdventitialFibroblasts',
 'GreatVesselArterialEndothelialCells',
 'GreatVesselSmoothMuscleCells',
 'GreatVesselVenousEndothelialCells',
 'InnateLymphoidCells',
 'LymphNodeFibroblasticReticularCells',
 'LymphaticEndothelialCells',
 'MacrophagesATF3pos',
 'MacrophagesCX3CR1pos',
 'MacrophagesLYVE1pos',
 'MacrophagesTIMD4pos',
 'MastCells',
 'Megakaryocytes',
 'MesothelialEpicardialCells',
 'MonocyteDerivedCells',
 'Monocytes',
 'MonocytesMPOpos',
 'MyocardialInterstitialFibroblasts1',
 'MyocardialInterstitialFibroblasts2',
 'Myofibroblasts',
 'NaturalKillerCells',
 'NeuronPrecursors',
 'ParaympatheticNeurons',
 'PericardialCells1',
 'PericardialCells2',
 'PericardialCells3',
 'PlasmacytoidDendriticCells',
 'ProBCells',
 'SchwannCellPrecursors',
 'SchwannCells',
 'SinoatrialNodeCardiomyocytes',
 'SympatheticNeurons',
 'TCellsCD4pos',
 'TCellsCD8pos',
 'TregsCD4pos',
 'ValveEndothelialCells',
 'ValveInterstitialCells',
 'VentricularCardiomyocytesCycling',
 'VentricularCardiomyocytesLeftCompact',
 'VentricularCardiomyocytesLeftTrabeculated',
 'VentricularCardiomyocytesRightCompact',
 'VentricularCardiomyocytesRightTrabeculated',
 'VentricularConductionSystemDistal',
 'VentricularConductionSystemProximal'}
```

Example 14 (python):
```python
# Make sure .var_names are ENSEMBL IDs
adata.var.head()
```

Example 15 (python):
```python
adata.obs['batch_key'].value_counts()
```

Example 16 (unknown):
```unknown
Hst41_Nuclei_Multiome-v1    104878
Hst33_Nuclei_Multiome-v1     38234
Hst39_Nuclei_Multiome-v1     25386
C99_Cell_5prime-v2           17547
C92_Nuclei_Multiome-v1       13770
C94_Cell_5prime-v2           12985
Hst40_Nuclei_Multiome-v1      9708
C97_Cell_5prime-v2            9702
BRC2251_Cell_3prime-v3        9121
BRC2256_Cell_3prime-v3        7948
Hst36_Nuclei_Multiome-v1      7268
BRC2252_Cell_3prime-v3        6854
C86_Cell_5prime-v2            6376
BRC2260_Cell_3prime-v3        6004
BRC2263_Cell_3prime-v3        5662
BRC2262_Cell_3prime-v3        4129
C87_Nuclei_Multiome-v1        3907
C98_Nuclei_Multiome-v1        2660
C104_Nuclei_Multiome-v1       2490
C85_Nuclei_Multiome-v1        2436
C82_Nuclei_Multiome-v1         408
Name: batch_key, dtype: int64
```

Example 17 (python):
```python
set(adata.obs['batch_key'])
```

Example 18 (unknown):
```unknown
{'BRC2251_Cell_3prime-v3',
 'BRC2252_Cell_3prime-v3',
 'BRC2256_Cell_3prime-v3',
 'BRC2260_Cell_3prime-v3',
 'BRC2262_Cell_3prime-v3',
 'BRC2263_Cell_3prime-v3',
 'C104_Nuclei_Multiome-v1',
 'C82_Nuclei_Multiome-v1',
 'C85_Nuclei_Multiome-v1',
 'C86_Cell_5prime-v2',
 'C87_Nuclei_Multiome-v1',
 'C92_Nuclei_Multiome-v1',
 'C94_Cell_5prime-v2',
 'C97_Cell_5prime-v2',
 'C98_Nuclei_Multiome-v1',
 'C99_Cell_5prime-v2',
 'Hst33_Nuclei_Multiome-v1',
 'Hst36_Nuclei_Multiome-v1',
 'Hst39_Nuclei_Multiome-v1',
 'Hst40_Nuclei_Multiome-v1',
 'Hst41_Nuclei_Multiome-v1'}
```

Example 19 (python):
```python
# remove pericardial cells
adata = adata[adata.obs['fine_grain'].isin(['PericardialCells1','PericardialCells2','PericardialCells3'])==False]

# remove cycling cells
adata = adata[adata.obs['cycling']=='no']

adata.shape
```

Example 20 (unknown):
```unknown
(186909, 36601)
```

Example 21 (python):
```python
'''
atrial_or_ventricular = {
    # 'AtrialCardiomyocytesCycling':'atrial',
 'AtrialCardiomyocytesLeft':'atrial',
 'AtrialCardiomyocytesRight':'atrial',
 'AtrioventricularNodeCardiomyocytes':'either',
 'BCells':'either',
 'BCellsMS4A1pos':'either',
 'ChromaffinCells':'either',
 'CoronaryArterialEndothelialCells':'either',
 'CoronaryCapillaryEndothelialCells':'either',
 'CoronaryPericytes':'either',
 'CoronarySmoothMuscleCells':'either',
 'CoronaryVenousEndothelialCells':'either',
 'CoronaryVesselAdventitialFibroblasts':'either',
 'DendriticCellsMature':'either',
 'DendriticCellsType1':'either',
 'DuctusArteriosusSmoothMuscleCells':'atrial',
 'EndocardialCells':'either',
 'EndocardialCushionCells':'either',
 'EpicardiumDerivedCells':'either',
 'GreatVesselAdventitialFibroblasts':'atrial',
 'GreatVesselArterialEndothelialCells':'atrial',
 'GreatVesselSmoothMuscleCells':'atrial',
 'GreatVesselVenousEndothelialCells':'atrial',
 'InnateLymphoidCells':'either',
 'LymphNodeFibroblasticReticularCells':'either',
 'LymphaticEndothelialCells':'either',
 'MacrophagesATF3pos':'either',
 'MacrophagesCX3CR1pos':'either',
 'MacrophagesLYVE1pos':'either',
 'MacrophagesTIMD4pos':'either',
 'MastCells':'either',
 'Megakaryocytes':'either',
 'MesothelialEpicardialCells':'either',
 'MonocyteDerivedCells':'either',
 'Monocytes':'either',
 'MonocytesMPOpos':'either',
 'MyocardialInterstitialFibroblasts1':'either',
 'MyocardialInterstitialFibroblasts2':'either',
 'Myofibroblasts':'either',
 'NaturalKillerCells':'either',
 'NeuronPrecursors':'either',
 'ParaympatheticNeurons':'either',
 # 'PericardialCells1':'either',
 # 'PericardialCells2':'either',
 # 'PericardialCells3':'either',
 'PlasmacytoidDendriticCells':'either',
 'ProBCells':'either',
 'SchwannCellPrecursors':'either',
 'SchwannCells':'either',
 'SinoatrialNodeCardiomyocytes':'atrial',
 'SympatheticNeurons':'either',
 'TCellsCD4pos':'either',
 'TCellsCD8pos':'either',
 'TregsCD4pos':'either',
 'ValveEndothelialCells':'either',
 'ValveInterstitialCells':'either',
 # 'VentricularCardiomyocytesCycling':'ventricular',
 'VentricularCardiomyocytesLeftCompact':'ventricular',
 'VentricularCardiomyocytesLeftTrabeculated':'ventricular',
 'VentricularCardiomyocytesRightCompact':'ventricular',
 'VentricularCardiomyocytesRightTrabeculated':'ventricular',
 'VentricularConductionSystemDistal':'ventricular',
 'VentricularConductionSystemProximal':'ventricular'}

if set(adata.obs['fine_grain']) == set(atrial_or_ventricular.keys()):
    adata.obs['atrialorventricular']=adata.obs['fine_grain'].astype('str').copy()
    adata.obs.replace({'atrialorventricular':atrial_or_ventricular},inplace=True)
else:
    print('ERROR: the cell type names are not matching')

pd.set_option("max_rows", 100)
pd.crosstab(adata.obs['fine_grain'],adata.obs['atrialorventricular'])
'''
```

Example 22 (unknown):
```unknown
regions = ['whole_woPeriCell'] #,'atrial','ventricular'
```

Example 23 (python):
```python
%%time

from cell2location.utils.filtering import filter_genes
from cell2location.models import RegressionModel

for reg in regions:
    print(f'##### {reg} #####')
    
    '''
    # subset adata
    if reg == 'whole_woPeriCell':
        adata_ref = adata.copy()
    else:
        adata_ref = adata[adata.obs['atrialorventricular'].isin(['either',reg])]
    '''
    
    adata_ref = adata.copy()
    
    # show cell types
    adata_ref.obs['fine_grain'].value_counts()
    
    # before we estimate the reference cell type signature we recommend to perform very permissive genes selection
    # in this 2D histogram orange rectangle lays over excluded genes.
    # In this case, the downloaded dataset was already filtered using this method,
    # hence no density under the orange rectangle
    selected = filter_genes(adata_ref, cell_count_cutoff=5, cell_percentage_cutoff2=0.03, nonz_mean_cutoff=1.12)
    # filter the object
    adata_ref = adata_ref[:, selected].copy()

    # remove cell_states, which have only 2 cells or less
    counts = adata_ref.obs[celltype_key].value_counts()
    adata_ref = adata_ref[adata_ref.obs[celltype_key].isin(list(counts.keys()[counts>2]))]

    # prepare anndata for the regression model
    adata_ref=adata_ref.copy()
    scvi.data.setup_anndata(adata=adata_ref, 
                            # 10X reaction / sample / batch
                            batch_key='batch_key', 
                            # cell type, covariate used for constructing signatures
                            labels_key=celltype_key, 
                            # multiplicative technical effects (platform, 3' vs 5', donor effect)
                            # categorical_covariate_keys=['Method']
                           )
    scvi.data.view_anndata_setup(adata_ref)

    # create and train the regression model
    mod = RegressionModel(adata_ref) 

    # Use all data for training (validation not implemented yet, train_size=1)
    mod.train(max_epochs=1000, batch_size=2500, train_size=1, lr=0.002, use_gpu=True)

    # plot ELBO loss history during training, removing first 20 epochs from the plot
    mod.plot_history(20)

    # In this section, we export the estimated cell abundance (summary of the posterior distribution).
    adata_ref = mod.export_posterior(
        adata_ref, sample_kwargs={'num_samples': 1000, 'batch_size': 2500, 'use_gpu': True}
    )

    # Save model
    mod.save(f"{ref_run_name}/{reg}/", overwrite=True)

    # Save anndata object with results
    # adata_file = f"{ref_run_name}/{reg}/sc.h5ad"
    adata_ref.write(f"{ref_run_name}/{reg}/sc.h5ad")

    mod.plot_QC()

    # export estimated expression in each cluster
    if 'means_per_cluster_mu_fg' in adata_ref.varm.keys():
        inf_aver = adata_ref.varm['means_per_cluster_mu_fg'][[f'means_per_cluster_mu_fg_{i}' 
                                        for i in adata_ref.uns['mod']['factor_names']]].copy()
    else:
        inf_aver = adata_ref.var[[f'means_per_cluster_mu_fg_{i}' 
                                        for i in adata_ref.uns['mod']['factor_names']]].copy()
    inf_aver.columns = adata_ref.uns['mod']['factor_names']
    inf_aver.iloc[0:5, 0:5]

    # save inf_aver
    inf_aver.to_csv(f"{ref_run_name}/{reg}/inf_aver.csv")
    
    del adata_ref,mod,inf_aver,selected,counts
```

Example 24 (unknown):
```unknown
##### whole_woPeriCell #####
```

Example 25 (python):
```python
INFO     Using batches from adata.obs["batch_key"]                                           
INFO     Using labels from adata.obs["fine_grain"]                                           
INFO     Using data from adata.X
```

Example 26 (unknown):
```unknown
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function setup_anndata is deprecated; Please use the model-specific setup_anndata methods instead. The global method will be removed in version 0.15.0.
```

Example 27 (python):
```python
INFO     Successfully registered anndata object containing 186909 cells, 17277 vars, 21      
         batches, 58 labels, and 0 proteins. Also registered 0 extra categorical covariates  
         and 0 extra continuous covariates.                                                  
INFO     Please do not further modify adata until model is trained.
```

Example 28 (unknown):
```unknown
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function view_anndata_setup is deprecated; This method will be removed in 0.15.0. Please avoid building any new dependencies on it.
```

Example 29 (unknown):
```unknown
Anndata setup with scvi-tools version 0.14.4.
```

Example 30 (unknown):
```unknown
Data Summary               
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃             Data             ┃ Count  ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│            Cells             │ 186909 │
│             Vars             │ 17277  │
│            Labels            │   58   │
│           Batches            │   21   │
│           Proteins           │   0    │
│ Extra Categorical Covariates │   0    │
│ Extra Continuous Covariates  │   0    │
└──────────────────────────────┴────────┘
```

Example 31 (python):
```python
SCVI Data Registry              
┏━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃     Data      ┃    scvi-tools Location    ┃
┡━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│       X       │          adata.X          │
│ batch_indices │ adata.obs['_scvi_batch']  │
│    labels     │ adata.obs['_scvi_labels'] │
└───────────────┴───────────────────────────┘
```

Example 32 (python):
```python
Label Categories                                       
┏━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┓
┃     Source Location     ┃                 Categories                 ┃ scvi-tools Encodin ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['fine_grain'] │          AtrialCardiomyocytesLeft          │         0          │
│                         │         AtrialCardiomyocytesRight          │         1          │
│                         │    VentricularCardiomyocytesLeftCompact    │         2          │
│                         │   VentricularCardiomyocytesRightCompact    │         3          │
│                         │ VentricularCardiomyocytesLeftTrabeculated  │         4          │
│                         │ VentricularCardiomyocytesRightTrabeculated │         5          │
│                         │        SinoatrialNodeCardiomyocytes        │         6          │
│                         │     AtrioventricularNodeCardiomyocytes     │         7          │
│                         │    VentricularConductionSystemProximal     │         8          │
│                         │     VentricularConductionSystemDistal      │         9          │
│                         │     GreatVesselAdventitialFibroblasts      │         10         │
│                         │    CoronaryVesselAdventitialFibroblasts    │         11         │
│                         │     MyocardialInterstitialFibroblasts1     │         12         │
│                         │     MyocardialInterstitialFibroblasts2     │         13         │
│                         │               Myofibroblasts               │         14         │
│                         │    LymphNodeFibroblasticReticularCells     │         15         │
│                         │           ValveInterstitialCells           │         16         │
│                         │        GreatVesselSmoothMuscleCells        │         17         │
│                         │         CoronarySmoothMuscleCells          │         18         │
│                         │     DuctusArteriosusSmoothMuscleCells      │         19         │
│                         │             CoronaryPericytes              │         20         │
│                         │    GreatVesselArterialEndothelialCells     │         21         │
│                         │     GreatVesselVenousEndothelialCells      │         22         │
│                         │      CoronaryArterialEndothelialCells      │         23         │
│                         │       CoronaryVenousEndothelialCells       │         24         │
│                         │     CoronaryCapillaryEndothelialCells      │         25         │
│                         │              EndocardialCells              │         26         │
│                         │          EndocardialCushionCells           │         27         │
│                         │           ValveEndothelialCells            │         28         │
│                         │         LymphaticEndothelialCells          │         29         │
│                         │         MesothelialEpicardialCells         │         30         │
│                         │           EpicardiumDerivedCells           │         31         │
│                         │              NeuronPrecursors              │         32         │
│                         │              ChromaffinCells               │         33         │
│                         │             SympatheticNeurons             │         34         │
│                         │           ParaympatheticNeurons            │         35         │
│                         │           SchwannCellPrecursors            │         36         │
│                         │                SchwannCells                │         37         │
│                         │              MonocytesMPOpos               │         38         │
│                         │                 Monocytes                  │         39         │
│                         │            MonocyteDerivedCells            │         40         │
│                         │            MacrophagesCX3CR1pos            │         41         │
│                         │            MacrophagesTIMD4pos             │         42         │
│                         │            MacrophagesLYVE1pos             │         43         │
│                         │             MacrophagesATF3pos             │         44         │
│                         │            DendriticCellsType1             │         45         │
│                         │            DendriticCellsMature            │         46         │
│                         │         PlasmacytoidDendriticCells         │         47         │
│                         │                 MastCells                  │         48         │
│                         │               Megakaryocytes               │         49         │
│                         │                TCellsCD4pos                │         50         │
│                         │                TCellsCD8pos                │         51         │
│                         │                TregsCD4pos                 │         52         │
│                         │                 ProBCells                  │         53         │
│                         │                   BCells                   │         54         │
│                         │               BCellsMS4A1pos               │         55         │
│                         │             NaturalKillerCells             │         56         │
│                         │            InnateLymphoidCells             │         57         │
└─────────────────────────┴────────────────────────────────────────────┴────────────────────┘
```

Example 33 (python):
```python
Batch Categories                              
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location     ┃        Categories        ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['batch_key'] │  BRC2251_Cell_3prime-v3  │          0          │
│                        │  BRC2252_Cell_3prime-v3  │          1          │
│                        │  BRC2256_Cell_3prime-v3  │          2          │
│                        │  BRC2260_Cell_3prime-v3  │          3          │
│                        │  BRC2262_Cell_3prime-v3  │          4          │
│                        │  BRC2263_Cell_3prime-v3  │          5          │
│                        │  C82_Nuclei_Multiome-v1  │          6          │
│                        │  C85_Nuclei_Multiome-v1  │          7          │
│                        │    C86_Cell_5prime-v2    │          8          │
│                        │  C87_Nuclei_Multiome-v1  │          9          │
│                        │  C92_Nuclei_Multiome-v1  │         10          │
│                        │    C94_Cell_5prime-v2    │         11          │
│                        │    C97_Cell_5prime-v2    │         12          │
│                        │  C98_Nuclei_Multiome-v1  │         13          │
│                        │    C99_Cell_5prime-v2    │         14          │
│                        │ C104_Nuclei_Multiome-v1  │         15          │
│                        │ Hst33_Nuclei_Multiome-v1 │         16          │
│                        │ Hst36_Nuclei_Multiome-v1 │         17          │
│                        │ Hst39_Nuclei_Multiome-v1 │         18          │
│                        │ Hst40_Nuclei_Multiome-v1 │         19          │
│                        │ Hst41_Nuclei_Multiome-v1 │         20          │
└────────────────────────┴──────────────────────────┴─────────────────────┘
```

Example 34 (unknown):
```unknown
GPU available: True, used: True
TPU available: False, using: 0 TPU cores
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/pytorch_lightning/trainer/configuration_validator.py:99: UserWarning: you passed in a val_dataloader but have no validation_step. Skipping val loop
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
```

Example 35 (unknown):
```unknown
Epoch 1000/1000: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1000/1000 [6:10:43<00:00, 22.24s/it, v_num=1, elbo_train=1.19e+9]
Sampling local variables, batch:   0%|                                                                                                                                                                                                  | 0/75 [00:02<?, ?it/s]
Sampling global variables, sample: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 999/999 [00:17<00:00, 58.03it/s]
```

Example 36 (unknown):
```unknown
CPU times: user 4h 39min, sys: 2h 58min 41s, total: 7h 37min 42s
Wall time: 6h 13min 10s
```

Example 37 (python):
```python
pd.read_csv(f"{ref_run_name}/{reg}/inf_aver.csv", index_col=0)
```

---

## Function¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig3_SAN_niche/snMultime_no1_FoetalSANCM_genes.html

**Contents:**
- Function¶
- Read in adata¶
- Subset anndata¶
- DEG: SANCM vs other CMs¶
- DEG: Foetal vs Adult¶
- Read in adult anndata and concatenate with foetal data¶
- Prepare anndata for pseudobulk wilcoxon test¶
- Pseudobulk wilcoxon test¶
- Obtain shared DEGs¶
- Pathway enrichment¶
- Plotting¶
- Plot axon guidance genes¶
- Glutermatergic signaling genes¶
- Axon guidance receptor expressions¶

34377 rows × 2 columns

**Examples:**

Example 1 (python):
```python
# Parameters
working_dir = "/nfs/team205/heart/Vitro/TF_selection/foetal_SANPcell_vs_workingCM"
path_to_adata_gex = "/nfs/team205/heart/anndata_objects/Foetal/Feb28ObjectRaw_Multiome.h5ad"
path_to_DAR_1 = "/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/MarkerPeaks/SANP-vs-aCM.csv"
path_to_DAR_2 = "/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/MarkerPeaks/aCM-vs-SANP.csv"
path_to_p2g_ArchR = ""
group_key = "fine_grain_aCMs_merged"
group_a = "SinoatrialNodePacemakerCells"
group_b = "AtrialCardiomyocytes"
subset_object = True
geneset_list = ["MSigDB_Hallmark_2020", "KEGG_2021_Human", "GO_Biological_Process_2023"]

path_to_eRegulon_signatures_gene = "/nfs/team205/heart/pyscenic/plus/eRegulon_signatures/Cardiomyocytes_Gene_eRegulon_signatures.pkl"
path_to_eRegulon_signatures_region = "/nfs/team205/heart/pyscenic/plus/eRegulon_signatures/Cardiomyocytes_Region_eRegulon_signatures.pkl"
```

Example 2 (python):
```python
import scanpy as sc
import pandas as pd
import numpy as np
import anndata

import matplotlib.pyplot as plt
import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42
import seaborn as sns

import warnings
warnings.simplefilter("ignore", UserWarning)
```

Example 3 (python):
```python
import session_info
session_info.show()
```

Example 4 (python):
```python
-----
anndata             0.9.2
matplotlib          3.7.2
numpy               1.24.4
pandas              1.5.3
scanpy              1.9.3
seaborn             0.12.2
session_info        1.0.0
-----
```

Example 5 (python):
```python
PIL                         10.0.0
anyio                       NA
arrow                       1.2.3
asttokens                   NA
attr                        23.1.0
attrs                       23.1.0
babel                       2.12.1
backcall                    0.2.0
brotli                      1.0.9
certifi                     2023.11.17
cffi                        1.15.1
charset_normalizer          3.2.0
colorama                    0.4.6
comm                        0.1.4
cycler                      0.10.0
cython_runtime              NA
dateutil                    2.8.2
debugpy                     1.6.8
decorator                   5.1.1
defusedxml                  0.7.1
executing                   1.2.0
fastjsonschema              NA
fqdn                        NA
h5py                        3.9.0
idna                        3.4
igraph                      0.10.6
importlib_metadata          NA
importlib_resources         NA
ipykernel                   6.25.1
isoduration                 NA
jedi                        0.19.0
jinja2                      3.1.2
joblib                      1.3.2
json5                       NA
jsonpointer                 2.0
jsonschema                  4.19.0
jsonschema_specifications   NA
jupyter_events              0.7.0
jupyter_server              2.7.1
jupyterlab_server           2.24.0
kiwisolver                  1.4.5
leidenalg                   0.10.1
llvmlite                    0.40.1
markupsafe                  2.1.3
mpl_toolkits                NA
natsort                     8.4.0
nbformat                    5.9.2
numba                       0.57.1
overrides                   NA
packaging                   23.1
parso                       0.8.3
patsy                       0.5.3
pexpect                     4.8.0
pickleshare                 0.7.5
pkg_resources               NA
plotly                      5.16.1
prometheus_client           NA
prompt_toolkit              3.0.39
psutil                      5.9.5
ptyprocess                  0.7.0
pure_eval                   0.2.2
pydev_ipython               NA
pydevconsole                NA
pydevd                      2.9.5
pydevd_file_utils           NA
pydevd_plugins              NA
pydevd_tracing              NA
pygments                    2.16.1
pyparsing                   3.0.9
pythonjsonlogger            NA
pytz                        2023.3
referencing                 NA
requests                    2.31.0
rfc3339_validator           0.1.4
rfc3986_validator           0.1.1
rpds                        NA
ruamel                      NA
scipy                       1.10.1
send2trash                  NA
six                         1.16.0
sklearn                     1.3.0
sniffio                     1.3.0
socks                       1.7.1
sphinxcontrib               NA
stack_data                  0.6.2
statsmodels                 0.14.0
texttable                   1.6.7
threadpoolctl               3.2.0
tornado                     6.3.3
traitlets                   5.9.0
typing_extensions           NA
uri_template                NA
urllib3                     2.0.4
wcwidth                     0.2.6
webcolors                   1.13
websocket                   1.6.2
yaml                        6.0.1
zipp                        NA
zmq                         25.1.1
zstandard                   0.19.0
```

Example 6 (unknown):
```unknown
-----
IPython             8.12.0
jupyter_client      8.3.0
jupyter_core        4.12.0
jupyterlab          4.0.5
notebook            7.0.2
-----
Python 3.8.17 (default, Jul  5 2023, 21:04:15) [GCC 11.2.0]
Linux-4.15.0-220-generic-x86_64-with-glibc2.17
-----
Session information updated at 2024-02-08 18:20
```

Example 7 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 8 (python):
```python
sc.settings.set_figure_params(dpi=80,dpi_save=300)
```

Example 9 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk'
```

Example 10 (python):
```python
import httplib2 as http
import json
import pandas as pd

def getHGNCgroup(GID, sortby='name', print_info=True):
    # http request
    target = f'https://www.genenames.org/cgi-bin/genegroup/download?id={GID}&type=branch&format=json'
    method = 'GET'
    h = http.Http()
    response, content = h.request(target,method)

    # parses the JSON
    if response['status'] == '200':
        data = json.loads(content)
    else:
        print('Error detected: ' + response['status'])
        return None
    
    # makes lists for symbol, name, group name
    symbol = []
    name = []
    group_name = []
    hgnc = []

    # extracts these from the JSON
    for i in range(len(data)):
        symbol.append((data[i]['approvedSymbol']))
        name.append((data[i]['approvedName']))
        group_name.append((data[i]['groupName']))
        hgnc.append((data[i]['hgncID']))

    # creates a data frame from the extracted information
    group_info = {
        'symbol':symbol,
        'name':name,
        'group_name':group_name,
        'hgnc':hgnc
    }
    group_info = pd.DataFrame.from_dict(group_info).sort_values(sortby,ascending=True)

    # print all unique group names if print_info is True
    if print_info:
        unique_group_names = set(group_name)
        print(f'HGNC Group {GID}, contains:')
        for grp_name in unique_group_names:
            print(grp_name)

    return group_info
```

Example 11 (python):
```python
adata = sc.read_h5ad(path_to_adata_gex)

# rename fine-grained cell types
celltype_rename = utils.read_pkl('/nfs/team205/heart/anndata_objects/Foetal/fine_grain_rename_dictionary.pkl')
adata.obs.replace({'fine_grain':celltype_rename},inplace=True)

# filter and log normalise
# filter
sc.pp.filter_genes(adata, min_cells=3)

# log normalise
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)
print(adata.X.data[:10])
```

Example 12 (unknown):
```unknown
[0.41374126 0.22811745 0.22811745 0.41374126 0.22811745 0.41374126
 0.22811745 0.41374126 0.22811745 0.22811745]
```

Example 13 (python):
```python
adata.obs['kit_10x'].value_counts()
```

Example 14 (unknown):
```unknown
Multiome-v1    211145
Name: kit_10x, dtype: int64
```

Example 15 (python):
```python
adata_sub = adata[adata.obs['coarse_grain']=='Cardiomyocytes']
adata_sub
```

Example 16 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 62104 × 34377
    obs: 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score', 'fine_grain_aCMs_merged', 'fine_grain_aCMs-node_merged'
    var: 'gene_id', 'n_cells'
    uns: 'log1p'
```

Example 17 (python):
```python
adata_sub.var
```

Example 18 (python):
```python
adata_sub.obs.fine_grain.unique()
```

Example 19 (unknown):
```unknown
['VentricularCardiomyocytesLeftTrabeculated', 'AtrioventricularNodePacemakerCells', 'AtrialCardiomyocytesLeft', 'AtrialCardiomyocytesCycling', 'VentricularCardiomyocytesCycling', ..., 'AtrialCardiomyocytesRight', 'VentricularCardiomyocytesRightCompact', 'SinoatrialNodePacemakerCells', 'VentricularCardiomyocytesRightTrabeculated', 'VentricularConductionSystemDistal']
Length: 12
Categories (12, object): ['AtrialCardiomyocytesLeft', 'AtrialCardiomyocytesRight', 'AtrialCardiomyocytesCycling', 'VentricularCardiomyocytesLeftCompact', ..., 'SinoatrialNodePacemakerCells', 'AtrioventricularNodePacemakerCells', 'VentricularConductionSystemProximal', 'VentricularConductionSystemDistal']
```

Example 20 (unknown):
```unknown
logFC_threshold = 1
group_mean_threshold = 0.1
```

Example 21 (python):
```python
res = utils.rank_genes_groups_with_mean(adata_sub,
                                 groupby = 'fine_grain',
                                  groups = ['SinoatrialNodePacemakerCells'],
                                  reference = 'rest',
                                  method = 't-test',
                                  pval_cutoff = 0.05,
                                        log2fc_min=logFC_threshold
                                 )
sns.scatterplot(x=res['logfoldchanges'],y=res['group_mean'])
plt.show()
```

Example 22 (python):
```python
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/tools/_rank_genes_groups.py:580: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  adata.uns[key_added] = {}
```

Example 23 (unknown):
```unknown
# filter genes based on the group mean expression
res = res[res['group_mean']>group_mean_threshold]
display(res.head())
degs_sancm_vs_others = list(res['names'])
len(degs_sancm_vs_others)
```

Example 24 (python):
```python
# prepare adult anndata
adata_adult = sc.read_h5ad('/nfs/team205/heart/anndata_objects/8regions/RNA_adult-8reg_full_raw_cellstate-annotated.h5ad')

# subset SAN_P_cell
adata_adult = adata_adult[adata_adult.obs['cell_state']=='SAN_P_cell']
adata_adult = adata_adult[adata_adult.obs['kit_10x']=='Multiome-v1']

# log normalise
sc.pp.normalize_total(adata_adult, target_sum=1e4)
sc.pp.log1p(adata_adult)
print(adata_adult.X.data[:10])

# rename
adata_adult.var['gene_id'] =  adata_adult.var_names.copy()
adata_adult.var.set_index('gene_name-new',inplace=True)
adata_adult.var_names = adata_adult.var_names.astype('str')
adata_adult.var_names_make_unique()

adata_adult
```

Example 25 (unknown):
```unknown
[0.911473   0.31626672 0.31626672 0.31626672 0.74952203 0.911473
 0.31626672 0.31626672 0.31626672 0.31626672]
```

Example 26 (python):
```python
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/preprocessing/_normalization.py:170: UserWarning: Received a view of an AnnData. Making a copy.
  view_to_actual(adata)
```

Example 27 (unknown):
```unknown
AnnData object with n_obs × n_vars = 245 × 32732
    obs: 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'n_genes', 'n_genes_by_counts', 'total_counts', 'total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'pct_counts_ribo', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'batch_key', 'leiden_scVI', 'cell_type', 'cell_state_HCAv1', 'cell_state_scNym', 'cell_state_scNym_confidence', 'cell_state', 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'n_counts', '_scvi_batch', '_scvi_labels', 'clus20', 'doublet_cls', 'original_or_new', 'batch', 'scANVI_predictions', 'leiden_scArches'
    var: 'gene_name_scRNA-0-original', 'gene_name_snRNA-1-original', 'gene_name_multiome-2-original', 'gene_id'
    uns: 'age_colors', 'cell_or_nuclei_colors', 'cell_state_colors', 'cell_type_colors', 'donor_colors', 'donor_type_colors', 'facility_colors', 'flushed_colors', 'gender_colors', 'kit_10x_colors', 'leiden', 'modality_colors', 'neighbors', 'original_or_new_colors', 'region_colors', 'region_finest_colors', 'scANVI_predictions_colors', 'umap', 'log1p'
    obsm: 'X_scArches', 'X_umap'
    obsp: 'connectivities', 'distances'
```

Example 28 (python):
```python
# concatenate
## shared features
shared = list(set(adata_sub.var_names).intersection(adata_adult.var_names))

adata_sub.obs['stage'] = 'foetal'
adata_adult.obs['stage'] = 'adult'
adata_adult.obs['week'] = 'adult'
adata_adult.obs['fine_grain'] = 'adult - SAN_P_cell'

adata_conc = adata_sub[:,shared].concatenate(adata_adult[:,shared],
                                             join='inner',
                                             batch_key=None,
                                             index_unique=None
                                            )
```

Example 29 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/anndata/_core/anndata.py:1763: FutureWarning: The AnnData.concatenate method is deprecated in favour of the anndata.concat function. Please use anndata.concat instead.

See the tutorial for concat at: https://anndata.readthedocs.io/en/latest/concatenation.html
  warnings.warn(
```

Example 30 (python):
```python
# combine donor and stage key
adata_conc.obs['donor_new'] = adata_conc.obs['stage'].astype('str')+'_'+adata_conc.obs['donor'].astype('str')
set(adata_conc.obs['donor_new'])
```

Example 31 (unknown):
```unknown
{'adult_A61',
 'adult_AH1',
 'adult_AH2',
 'adult_AV10',
 'adult_AV14',
 'adult_AV3',
 'foetal_C104',
 'foetal_C82',
 'foetal_C85',
 'foetal_C87',
 'foetal_C92',
 'foetal_C98',
 'foetal_Hst33',
 'foetal_Hst36',
 'foetal_Hst39',
 'foetal_Hst40',
 'foetal_Hst41'}
```

Example 32 (python):
```python
# add drimester 
donor_trimester_df = adata_conc.obs[['donor','trimester']].copy()
donor_trimester_df = donor_trimester_df.drop_duplicates()
donor_trimester_df['trimester'] = [x if x==x else 'adult' for x in donor_trimester_df['trimester']]
donor_trimester_df
```

Example 33 (python):
```python
groupby = 'donor_new'
celltypes = ['SinoatrialNodePacemakerCells','adult - SAN_P_cell']
n_cell_thresh = 10

# subset anndata
ad = adata_conc[adata_conc.obs['fine_grain'].isin(celltypes)]
# remove group which only has less than a threshold
vc = ad.obs[groupby].value_counts()
grp_to_drop = list(vc.index[vc < 10])
ad = ad[~ad.obs[groupby].isin(grp_to_drop)]
print(f'dropped groups: {grp_to_drop}')

# mean expression per group
exp_mean = utils.grouped_obs_mean(ad, group_key=groupby)

# make anndata
import anndata
adata_pb = anndata.AnnData(
    X=exp_mean.T,
    obs=pd.DataFrame(exp_mean.columns).set_index(0),
    var=pd.DataFrame(exp_mean.index).set_index(0)
)

# add stage key
adata_pb.obs['stage'] = [x.split('_')[0] for x in adata_pb.obs_names]
adata_pb.obs['stage'] = adata_pb.obs['stage'].astype('category')
adata_pb.obs['stage'].cat.reorder_categories(['foetal','adult'],inplace=True)

# add donor and trimester key
adata_pb.obs['donor'] = [x.split('_')[1] for x in adata_pb.obs_names]
obs = adata_pb.obs.reset_index().merge(donor_trimester_df,how='left',on='donor').set_index(0)
obs.index.name = 'index'
adata_pb.obs = obs
adata_pb.obs['trimester'] = adata_pb.obs['trimester'].astype('category')
adata_pb.obs['trimester'].cat.reorder_categories(['firsttrimester','secondtrimester','adult'],inplace=True)


print(adata_pb.obs['stage'].value_counts())
print(adata_pb)
```

Example 34 (unknown):
```unknown
dropped groups: ['foetal_C82', 'foetal_Hst36']
foetal    8
adult     6
Name: stage, dtype: int64
AnnData object with n_obs × n_vars = 14 × 30749
    obs: 'stage', 'donor', 'trimester'
```

Example 35 (python):
```python
/tmp/ipykernel_30942/3488307275.py:27: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
  adata_pb.obs['stage'].cat.reorder_categories(['foetal','adult'],inplace=True)
/tmp/ipykernel_30942/3488307275.py:35: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
  adata_pb.obs['trimester'].cat.reorder_categories(['firsttrimester','secondtrimester','adult'],inplace=True)
```

Example 36 (python):
```python
# rank test
logFC_threshold_FvsA = 1
group_mean_threshold = 0.1

res_foetal_vs_adult = utils.rank_genes_groups_with_mean(adata_pb,
                                 groupby = 'stage',
                                  groups = ['foetal'],
                                  reference = 'adult',
                                  method = 'wilcoxon',
                                  pval_cutoff = 0.1,
                                        log2fc_min=logFC_threshold_FvsA
                                 )

sns.scatterplot(x=res_foetal_vs_adult['logfoldchanges'],y=res_foetal_vs_adult['group_mean'])
plt.show()
```

Example 37 (python):
```python
res_foetal_vs_adult = res_foetal_vs_adult[res_foetal_vs_adult['group_mean']>group_mean_threshold]
degs_foetal_vs_adult = list(res_foetal_vs_adult['names'])
print(len(degs_foetal_vs_adult))
display(res_foetal_vs_adult.head())
```

Example 38 (unknown):
```unknown
res_foetal_vs_adult[res_foetal_vs_adult['names']=='CXCL12']
```

Example 39 (python):
```python
from matplotlib_venn import venn2
venn2(subsets=(set(degs_sancm_vs_others),set(degs_foetal_vs_adult)),
      set_labels=('SAN_P vs Others (foetal)', 'Foetal vs Adult')
     )
```

Example 40 (python):
```python
<matplotlib_venn._common.VennDiagram at 0x153fd0827cd0>
```

Example 41 (unknown):
```unknown
degs_common = list(set(degs_sancm_vs_others).intersection(degs_foetal_vs_adult))
len(degs_common)
```

Example 42 (python):
```python
import gseapy as gp
```

Example 43 (unknown):
```unknown
# Re-enrichmant analysis for plotting
enr_sancm_vs_othercm = gp.enrichr(gene_list=degs_sancm_vs_others,
                 gene_sets=geneset_list,
                 organism='human', # don't forget to set organism to the one you desired! e.g. Yeast
                 outdir=None, # don't write to disk
                )

enr_common = gp.enrichr(gene_list=degs_common,
                 gene_sets=geneset_list,
                 organism='human', # don't forget to set organism to the one you desired! e.g. Yeast
                 outdir=None, # don't write to disk
                )
```

Example 44 (python):
```python
# foetal, SANP vs other CM
geneset = 'GO_Biological_Process_2023'

df = enr_sancm_vs_othercm.results[enr_sancm_vs_othercm.results['Gene_set'] == geneset]
df = df[df['Adjusted P-value'] < 0.1]
df = df.sort_values('Combined Score',ascending=False)

# plot
n_rank = 10
plt.rcParams['axes.grid'] = False
plt.rcParams['figure.figsize'] = 4,2.5
plt.barh(width=df[:n_rank]['Combined Score'],
         y=[x.split(' (')[0] for x in df[:n_rank]['Term']],
         color='darkred')
plt.gca().invert_yaxis()
plt.xlabel('Combined Score')
plt.title(f'SAN pacemaker cell vs other CMs\n{geneset}')
plt.margins(y=0.03)
plt.savefig(f'{figdir}/Pathway-enr_SANP_sancm_vs_othercm_genes.pdf',bbox_inches="tight",dpi=300)
plt.show()
```

Example 45 (python):
```python
# foetal vs adult, SANP
geneset = 'GO_Biological_Process_2023'

df = enr_common.results[enr_common.results['Gene_set'] == geneset]
df = df[df['Adjusted P-value'] < 0.1]
df = df.sort_values('Combined Score',ascending=False)

# plot
n_rank = 10
plt.rcParams['axes.grid'] = False
plt.rcParams['figure.figsize'] = 4,2.5
plt.barh(width=df[:n_rank]['Combined Score'],
         y=[x.split(' (')[0] for x in df[:n_rank]['Term']],
         color='darkred')
plt.gca().invert_yaxis()
plt.xlabel('Combined Score')
plt.title(f'Foetal vs Adult SAN pacemaker cell\n{geneset}')
plt.margins(y=0.03)
plt.savefig(f'{figdir}/Pathway-enr_SANP_common_genes.pdf',bbox_inches="tight",dpi=300)
plt.show()
```

Example 46 (python):
```python
# read in manually collected axon guidance genes
axon_genes = list(pd.read_csv('./Axon_guidance.csv')['Gene'])

# genes which is DEGs for both foetal-sancm-vs-others and foetal-vs-adult
genes_sel1 = [x for x in axon_genes if x in degs_common]
genes_sel1
```

Example 47 (unknown):
```unknown
['SEMA3A', 'SLIT2', 'SLIT3', 'EFNB2', 'CXCL12', 'FGF10', 'VEGFC', 'NELL1']
```

Example 48 (python):
```python
# plotting the genes
genes = genes_sel1.copy()

group_col = 'trimester'

data = adata_pb.to_df()[genes]
data[group_col] = adata_pb.obs[group_col]

# Melt the DataFrame to get a long-form DataFrame suitable for facetting
melted_data = data.melt(id_vars=group_col, value_vars=genes, var_name='gene', value_name='mean expression')

# Create a catplot with boxplots for each gene
g = sns.catplot(x=group_col, y='mean expression', col='gene', data=melted_data, kind='box',
                col_wrap=4, sharey=False, sharex=False, height=2, aspect=0.8)

# Overlay swarmplot on each Axes
# We'll use the 'hue' parameter to match the subsets and turn off dodge to align the points with the boxes.
for gene, ax in zip(melted_data['gene'].unique(), g.axes.flat):
    subset = melted_data[melted_data['gene'] == gene]
    sns.swarmplot(x=group_col, y='mean expression', data=subset, color='k', size=5, ax=ax, dodge=False)
    
# Update titles to remove "gene ="
for ax, gene in zip(g.axes.flat, genes):
    ax.set_title(gene)

# Rotate x-tick labels and remove x-axis labels
for i,ax in enumerate(g.axes.flat):
    # ax.set_xticklabels(ax.get_xticklabels(), rotation=45, ha="right", rotation_mode="anchor")  # Rotate x-tick labels
    ax.set_xticklabels('')
    ax.set_xlabel('')  # Remove x-axis labels
    if i!=0:
        ax.set_ylabel('')

# Adjust the spacing of the plots
g.fig.subplots_adjust(wspace=0.4, hspace=0.4)

# Manually create a legend for the groups.
# Since there are no legend handles for the swarmplot, we create them manually.
from matplotlib.patches import Patch
legend_patches = [Patch(facecolor='#0343DF', edgecolor='k', label='firsttrimester'),
                  Patch(facecolor='#FFA500', edgecolor='k', label='secondtrimester'),
                  Patch(facecolor='#008000', edgecolor='k', label='adult')]

# Add a single legend for the entire figure with our custom patches
g.fig.legend(handles=legend_patches, labels=['firsttrimester', 'secondtrimester', 'adult'], title='', loc='center', bbox_to_anchor=(1.05, 0.8))

# Show the plot
plt.savefig(f'{figdir}/Boxplot_SANPC_axon-guidance-genes.pdf',bbox_inches="tight",dpi=300)
plt.show()
```

Example 49 (python):
```python
genes = ['GLS','GRID2','GRIA3','SLC1A3','SLC17A5','SLC38A1','SNAP25','STX16','STX17'] #'CHRM2',
genes.sort()
# genes = genes + ['GRIA1','GRIK3','GLUL','SLC1A2','SLC38A5','SLC38A9','NGF']

group_col = 'trimester'

data = adata_pb.to_df()[genes]
data[group_col] = adata_pb.obs[group_col]

# Melt the DataFrame to get a long-form DataFrame suitable for facetting
melted_data = data.melt(id_vars=group_col, value_vars=genes, var_name='gene', value_name='mean expression')

# Create a catplot with boxplots for each gene
g = sns.catplot(x=group_col, y='mean expression', col='gene', data=melted_data, kind='box',
                col_wrap=9, sharey=False, sharex=False, height=3, aspect=0.5)

# Overlay swarmplot on each Axes
# We'll use the 'hue' parameter to match the subsets and turn off dodge to align the points with the boxes.
for gene, ax in zip(melted_data['gene'].unique(), g.axes.flat):
    subset = melted_data[melted_data['gene'] == gene]
    sns.swarmplot(x=group_col, y='mean expression', data=subset, color='k', size=7, ax=ax, dodge=False)
    
# Update titles to remove "gene ="
for ax, gene in zip(g.axes.flat, genes):
    ax.set_title(gene)

# Rotate x-tick labels and remove x-axis labels
for i,ax in enumerate(g.axes.flat):
    # ax.set_xticklabels(ax.get_xticklabels(), rotation=45, ha="right", rotation_mode="anchor")  # Rotate x-tick labels
    ax.set_xticklabels('')
    ax.set_xlabel('')  # Remove x-axis labels
    if i!=0:
        ax.set_ylabel('')

# Adjust the spacing of the plots
g.fig.subplots_adjust(wspace=0.5, hspace=0.2)

# Manually create a legend for the groups.
# Since there are no legend handles for the swarmplot, we create them manually.
from matplotlib.patches import Patch
legend_patches = [Patch(facecolor='#0343DF', edgecolor='k', label='firsttrimester'),
                  Patch(facecolor='#FFA500', edgecolor='k', label='secondtrimester'),
                  Patch(facecolor='#008000', edgecolor='k', label='adult')]

# Add a single legend for the entire figure with our custom patches
g.fig.legend(handles=legend_patches, labels=['firsttrimester', 'secondtrimester', 'adult'], title='', loc='upper right', bbox_to_anchor=(0.15, 0.15))

# Show the plot
plt.savefig(f'{figdir}/Boxplot_SANPC_glutermatergic-genes.pdf',bbox_inches="tight",dpi=300)
plt.show()
```

Example 50 (unknown):
```unknown
LR = {
    'SEMA':['PLXNA1', 'PLXNA2', 'PLXNA3', 'PLXNA4','PLXND1'],
    'SLIT':['ROBO1','ROBO2','ROBO3','ROBO4'],
    'CXCL12':['CXCR4'],
    'NELL':['CNTNAP4'],
}
```

Example 51 (python):
```python
# at SAnode
cells = [
    'SinoatrialNodePacemakerCells',
 'ParasympatheticNeurons',
 'MacrophagesCX3CR1pos',
]

sc.pl.dotplot(adata[adata.obs['fine_grain'].isin(cells)],
              LR, 
              groupby='fine_grain',
              dendrogram=False,
              standard_scale="var",
              color_map="Reds",
              swap_axes=False,
              show=False
             )
plt.savefig(f'{figdir}/Dotplot_SAN_Axon-guidance-receptors.pdf',bbox_inches="tight",dpi=300)
```

---

## Import module¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/ATAC/ATAC/run_peakVI.html

**Contents:**
- Import module¶
- Read in data¶
- Setup anndata for scVI¶

**Examples:**

Example 1 (python):
```python
import scvi
import anndata
import scipy
import numpy as np
import pandas as pd
import scanpy as sc
import matplotlib.pyplot as plt

scvi.settings.seed = 420
```

Example 2 (python):
```python
Global seed set to 0
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/flax/core/frozen_dict.py:169: FutureWarning: jax.tree_util.register_keypaths is deprecated, and will be removed in a future release. Please use `register_pytree_with_keys()` instead.
  jax.tree_util.register_keypaths(
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/pytorch_lightning/utilities/warnings.py:53: LightningDeprecationWarning: pytorch_lightning.utilities.warnings.rank_zero_deprecation has been deprecated in v1.6 and will be removed in v1.8. Use the equivalent function from the pytorch_lightning.utilities.rank_zero module instead.
  new_rank_zero_deprecation(
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/pytorch_lightning/utilities/warnings.py:58: LightningDeprecationWarning: The `pytorch_lightning.loggers.base.rank_zero_experiment` is deprecated in v1.7 and will be removed in v1.9. Please use `pytorch_lightning.loggers.logger.rank_zero_experiment` instead.
  return new_rank_zero_deprecation(*args, **kwargs)
Global seed set to 420
```

Example 3 (python):
```python
adata = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/PeakMatrix/Foetal_Peaks.h5ad')
adata
```

Example 4 (unknown):
```unknown
AnnData object with n_obs × n_vars = 167022 × 508040
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score'
```

Example 5 (python):
```python
adata.X.data[:10]
```

Example 6 (unknown):
```unknown
array([2., 2., 2., 2., 1., 1., 2., 1., 2., 2.], dtype=float32)
```

Example 7 (python):
```python
scvi.model.PEAKVI.setup_anndata(adata, batch_key="donor")
```

Example 8 (unknown):
```unknown
No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.)
```

Example 9 (python):
```python
pvi = scvi.model.PEAKVI(adata)
pvi.train()
```

Example 10 (unknown):
```unknown
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/pytorch_lightning/trainer/configuration_validator.py:267: LightningDeprecationWarning: The `Callback.on_epoch_end` hook was deprecated in v1.6 and will be removed in v1.8. Please use `Callback.on_<train/validation/test>_epoch_end` instead.
  rank_zero_deprecation(
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
```

Example 11 (unknown):
```unknown
Epoch 18/500:   3%|█████▏                                                                                                                                                  | 17/500 [2:22:27<67:38:33, 504.17s/it, loss=4.17e+08, v_num=1]
```

Example 12 (python):
```python
latent = pvi.get_latent_representation()
adata.obsm["X_PeakVI"] = latent

print(latent.shape)
```

Example 13 (python):
```python
adata.write('/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/PeakMatrix/Foetal_Peaks_post-peakVI.h5ad')
```

Example 14 (unknown):
```unknown
pvi.save('/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/PeakMatrix/Foetal_peakVI-model', overwrite=True)
```

Example 15 (python):
```python
# Convergency plot
pvi.history['elbo_train']
x = np.linspace(1, (len(pvi.history['elbo_train'])), (len(pvi.history['elbo_train'])))
plt.plot(x, pvi.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
```

---

## Import modules¶

**URL:** http://127.0.0.1:9020/notebooks_html/Supplementary_materials/Epigenetic_stability/no1a_scVI_per-midgrain.html

**Contents:**
- Import modules¶
- Parameters¶
- Read in anndata¶
- Subset anndata¶
- Modify mid grained cell type labels: merge CCS and other CMs¶
- Run per condition¶

**Examples:**

Example 1 (python):
```python
# import warnings
# warnings.simplefilter("ignore", UserWarning)

import anndata
import matplotlib.pyplot as plt
import seaborn as sns

import numpy as np
import numpy.random as random
import pandas as pd
import scanpy as sc

import scvi
```

Example 2 (python):
```python
Global seed set to 0
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/flax/core/frozen_dict.py:169: FutureWarning: jax.tree_util.register_keypaths is deprecated, and will be removed in a future release. Please use `register_pytree_with_keys()` instead.
  jax.tree_util.register_keypaths(
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/pytorch_lightning/utilities/warnings.py:53: LightningDeprecationWarning: pytorch_lightning.utilities.warnings.rank_zero_deprecation has been deprecated in v1.6 and will be removed in v1.8. Use the equivalent function from the pytorch_lightning.utilities.rank_zero module instead.
  new_rank_zero_deprecation(
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/pytorch_lightning/utilities/warnings.py:58: LightningDeprecationWarning: The `pytorch_lightning.loggers.base.rank_zero_experiment` is deprecated in v1.7 and will be removed in v1.9. Please use `pytorch_lightning.loggers.logger.rank_zero_experiment` instead.
  return new_rank_zero_deprecation(*args, **kwargs)
```

Example 3 (python):
```python
import gc
import torch
```

Example 4 (python):
```python
import session_info
session_info.show()
```

Example 5 (python):
```python
-----
anndata             0.8.0
matplotlib          3.7.1
numpy               1.23.5
pandas              1.5.3
scanpy              1.9.3
scvi                0.19.0
seaborn             0.12.2
session_info        1.0.0
torch               1.13.1+cu117
-----
```

Example 6 (python):
```python
PIL                 9.4.0
absl                NA
asttokens           NA
attr                22.2.0
backcall            0.2.0
certifi             2022.12.07
cffi                1.15.1
charset_normalizer  3.1.0
chex                0.1.6
comm                0.1.2
contextlib2         NA
cycler              0.10.0
cython_runtime      NA
dateutil            2.8.2
debugpy             1.6.6
decorator           5.1.1
defusedxml          0.7.1
deprecate           0.3.2
docrep              0.3.2
etils               1.1.0
executing           1.2.0
flax                0.6.7
fsspec              2023.3.0
google              NA
h5py                3.8.0
idna                3.4
igraph              0.10.4
importlib_metadata  NA
importlib_resources NA
ipykernel           6.21.3
ipython_genutils    0.2.0
ipywidgets          8.0.4
jax                 0.4.10
jaxlib              0.4.10
jedi                0.18.2
joblib              1.2.0
kiwisolver          1.4.4
leidenalg           0.9.1
llvmlite            0.39.1
ml_collections      NA
ml_dtypes           0.2.0
mpl_toolkits        NA
msgpack             1.0.5
mudata              0.2.1
multipledispatch    0.6.0
natsort             8.3.1
numba               0.56.4
numpyro             0.11.0
nvidia              NA
opt_einsum          v3.3.0
optax               0.1.4
packaging           23.0
parso               0.8.3
pexpect             4.8.0
pickleshare         0.7.5
pkg_resources       NA
platformdirs        3.1.1
prompt_toolkit      3.0.38
psutil              5.9.4
ptyprocess          0.7.0
pure_eval           0.2.2
pydev_ipython       NA
pydevconsole        NA
pydevd              2.9.5
pydevd_file_utils   NA
pydevd_plugins      NA
pydevd_tracing      NA
pygments            2.14.0
pyparsing           3.0.9
pyro                1.8.4
pytorch_lightning   1.7.7
pytz                2022.7.1
requests            2.28.2
rich                NA
scipy               1.10.1
setuptools          65.6.3
six                 1.16.0
sklearn             1.2.2
stack_data          0.6.2
statsmodels         0.13.5
tensorboard         2.12.0
texttable           1.6.7
threadpoolctl       3.1.0
toolz               0.12.0
torchmetrics        0.11.4
torchvision         0.14.1+cu117
tornado             6.2
tqdm                4.65.0
traitlets           5.9.0
tree                0.1.8
typing_extensions   NA
urllib3             1.26.15
wcwidth             0.2.6
yaml                6.0
zipp                NA
zmq                 25.0.1
zoneinfo            NA
```

Example 7 (unknown):
```unknown
-----
IPython             8.11.0
jupyter_client      8.0.3
jupyter_core        5.2.0
notebook            6.5.3
-----
Python 3.9.16 (main, Mar  8 2023, 14:00:05) [GCC 11.2.0]
Linux-4.15.0-214-generic-x86_64-with-glibc2.27
-----
Session information updated at 2023-12-29 22:41
```

Example 8 (python):
```python
sc.settings.verbosity = 3             # verbosity: errors (0), warnings (1), info (2), hints (3)
sc.settings.set_figure_params(dpi=80, facecolor='white', color_map = 'RdPu',)
```

Example 9 (python):
```python
path_to_adata_gex = '/nfs/team205/heart/anndata_objects/Foetal/Feb28ObjectRaw_Multiome.h5ad'
celltype_col = 'mid_grain_mod'
model_architecture_conditions = ['one-layer',
              # 'two-layer',
              # 'three-layer'
                                ]
```

Example 10 (python):
```python
adata = sc.read_h5ad(path_to_adata_gex)
adata
```

Example 11 (unknown):
```unknown
AnnData object with n_obs × n_vars = 211145 × 36601
    obs: 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score', 'fine_grain_aCMs_merged', 'fine_grain_aCMs-node_merged'
    var: 'gene_id'
```

Example 12 (python):
```python
adata.X.data[:10]
```

Example 13 (unknown):
```unknown
array([2., 1., 1., 2., 1., 2., 1., 2., 1., 1.], dtype=float32)
```

Example 14 (python):
```python
# read in selected nuclei id
df = pd.read_csv('/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/GEX_ATAC_comparison/subsampled_nuclei_n1000_29Dec2023.txt',
                 header=None)
nuclei_sel = pd.Index(df[0])
print(len(nuclei_sel))
nuclei_sel[:5]
```

Example 15 (unknown):
```unknown
Index(['7089STDY13216921_BHF_F_Hea13242528_TGGGCCTAGATGGACA-1',
       '7089STDY13216921_BHF_F_Hea13242528_AAAGGCTCAGCTACGT-1',
       '7089STDY13216921_BHF_F_Hea13242528_CATCATAAGCGATACT-1',
       '7089STDY13216921_BHF_F_Hea13242528_CAGGCTATCACCAATA-1',
       '7089STDY13216921_BHF_F_Hea13242528_CTAATGTCACTTCATC-1'],
      dtype='object', name=0)
```

Example 16 (python):
```python
# subset
adata = adata[nuclei_sel]
adata
```

Example 17 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 44136 × 36601
    obs: 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score', 'fine_grain_aCMs_merged', 'fine_grain_aCMs-node_merged'
    var: 'gene_id'
```

Example 18 (python):
```python
# get indices of CCS cells
a_ccs_indices = adata.obs_names[adata.obs['fine_grain'].isin(['SinoatrialNodeCardiomyocytes','AtrioventricularNodeCardiomyocytes'])]
v_ccs_indices = adata.obs_names[adata.obs['fine_grain'].isin(['VentricularConductionSystemProximal','VentricularConductionSystemDistal'])]

# modify: gex data
adata.obs['mid_grain_mod'] = adata.obs['mid_grain'].astype('str').copy()
adata.obs.replace({'mid_grain_mod':{
    'AtrialCardiomyocytes':'AtrialCardiomyocytes_plusCCS',
    'VentricularCardiomyocytes':'VentricularCardiomyocytes_plusCCS'
}},inplace=True)
adata.obs.loc[a_ccs_indices,'mid_grain_mod'] = 'AtrialCardiomyocytes_plusCCS'
adata.obs.loc[v_ccs_indices,'mid_grain_mod'] = 'VentricularCardiomyocytes_plusCCS'
adata.obs['mid_grain_mod'].value_counts()
```

Example 19 (python):
```python
/tmp/ipykernel_65247/1262357881.py:6: ImplicitModificationWarning: Trying to modify attribute `.obs` of view, initializing view as actual.
  adata.obs['mid_grain_mod'] = adata.obs['mid_grain'].astype('str').copy()
```

Example 20 (unknown):
```unknown
LymphoidCells                        7006
Fibroblasts                          6969
VentricularCardiomyocytes_plusCCS    6531
AtrialCardiomyocytes_plusCCS         4381
MyeloidCells                         4321
MuralCells                           3103
BloodVesselEndothelialCells          3028
PericardialCells                     2187
Glia                                 1759
EndocardialCells                     1410
EpicardialCells                      1340
Neurons                              1101
LymphaticEndothelialCells            1000
Name: mid_grain_mod, dtype: int64
```

Example 21 (python):
```python
%%time
for cond in model_architecture_conditions:
    for celltype in set(adata.obs[celltype_col]):
        print(f'##############################')
        print(f'##### {cond}: {celltype} #####')
        print(f'##############################')

        # subset
        adata_sub = adata[adata.obs[celltype_col]==celltype]
        # filter genes
        sc.pp.filter_genes(adata_sub, min_cells=3)

        # setup anndata for scVI
        adata_sub = adata_sub.copy()
        scvi.model.SCVI.setup_anndata(adata_sub, 
                                  # layer="counts", 
                                  categorical_covariate_keys=['batch_key','region'], # added 'region'
                                  continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo',
                                                             'S_score', 'G2M_score'
                                                            ])
        # creating a model
        if cond == 'three-layer':
            vae = scvi.model.SCVI(adata_sub, n_hidden = 128, n_latent = 50, n_layers = 3, dispersion = 'gene-batch')
        elif cond == 'two-layer':
            vae = scvi.model.SCVI(adata_sub, n_hidden=128, n_latent=50, n_layers=2, dispersion = 'gene-batch')
        elif cond == 'one-layer':
            vae = scvi.model.SCVI(adata_sub, n_hidden=128, n_latent=50, n_layers=1, dispersion = 'gene-batch')
        print(vae)
        vae.view_anndata_setup(adata_sub)
        # train
        vae.train()

        # save scvi model
        vae.save(f"/nfs/team205/heart/anndata_objects/Foetal/scVI/models/subsets/Foetal_multiome_{celltype}_{cond}_added-cov-region_subsampled-n1000/",overwrite=True)

        # save scVI lagent space
        adata_sub.obsm["X_scVI"] = vae.get_latent_representation()
        adata_sub.write(f'/nfs/team205/heart/anndata_objects/Foetal/scVI/subsets/Foetal_multiome_{celltype}_{cond}_added-cov-region_subsampled-n1000.h5ad')

        # Convergency plot
        vae.history['elbo_train']
        x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
        plt.plot(x, vae.history['elbo_train'], label="train")
        plt.xlabel('epoch')
        plt.ylabel('elbo_train')
        plt.title(f'{cond}: {celltype}')
        plt.show()

        del adata_sub, vae
        
        # free up GPU
        gc.collect()
        torch.cuda.empty_cache()
        print('')
```

Example 22 (unknown):
```unknown
##############################
##### one-layer: LymphaticEndothelialCells #####
##############################
filtered out 15921 genes that are detected in less than 3 cells
```

Example 23 (python):
```python
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/scanpy/preprocessing/_simple.py:251: ImplicitModificationWarning: Trying to modify attribute `.var` of view, initializing view as actual.
  adata.var['n_cells'] = number
No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.)
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/scvi/data/_utils.py:168: UserWarning: Category 5 in adata.obs['batch_key'] has fewer than 3 cells. Models may not train properly.
  warnings.warn(
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/scvi/data/_utils.py:168: UserWarning: Category 1 in adata.obs['region'] has fewer than 3 cells. Models may not train properly.
  warnings.warn(
```

Example 24 (unknown):
```unknown
SCVI Model with the following params: 
n_hidden: 128, n_latent: 50, n_layers: 1, dropout_rate: 0.1, dispersion: gene-batch, gene_likelihood: zinb, 
latent_distribution: normal
Training status: Not Trained
```

Example 25 (unknown):
```unknown
Anndata setup with scvi-tools version 0.19.0.
```

Example 26 (unknown):
```unknown
Setup via `SCVI.setup_anndata` with arguments:
```

Example 27 (unknown):
```unknown
{
│   'layer': None,
│   'batch_key': None,
│   'labels_key': None,
│   'size_factor_key': None,
│   'categorical_covariate_keys': ['batch_key', 'region'],
│   'continuous_covariate_keys': ['total_counts', 'pct_counts_mt', 'pct_counts_ribo', 'S_score', 'G2M_score']
}
```

Example 28 (unknown):
```unknown
Summary Statistics         
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┓
┃     Summary Stat Key     ┃ Value ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━┩
│         n_batch          │   1   │
│         n_cells          │ 1000  │
│ n_extra_categorical_covs │   2   │
│ n_extra_continuous_covs  │   5   │
│         n_labels         │   1   │
│          n_vars          │ 20680 │
└──────────────────────────┴───────┘
```

Example 29 (python):
```python
Data Registry                             
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃      Registry Key      ┃            scvi-tools Location             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│           X            │                  adata.X                   │
│         batch          │          adata.obs['_scvi_batch']          │
│ extra_categorical_covs │ adata.obsm['_scvi_extra_categorical_covs'] │
│ extra_continuous_covs  │ adata.obsm['_scvi_extra_continuous_covs']  │
│         labels         │         adata.obs['_scvi_labels']          │
└────────────────────────┴────────────────────────────────────────────┘
```

Example 30 (python):
```python
batch State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃     Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_batch'] │     0      │          0          │
└──────────────────────────┴────────────┴─────────────────────┘
```

Example 31 (python):
```python
labels State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 32 (python):
```python
extra_categorical_covs State Registry                       
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location     ┃           Categories            ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['batch_key'] │     C82_Nuclei_Multiome-v1      │          0          │
│                        │     C85_Nuclei_Multiome-v1      │          1          │
│                        │     C87_Nuclei_Multiome-v1      │          2          │
│                        │     C92_Nuclei_Multiome-v1      │          3          │
│                        │     C98_Nuclei_Multiome-v1      │          4          │
│                        │     C104_Nuclei_Multiome-v1     │          5          │
│                        │    Hst33_Nuclei_Multiome-v1     │          6          │
│                        │    Hst36_Nuclei_Multiome-v1     │          7          │
│                        │    Hst39_Nuclei_Multiome-v1     │          8          │
│                        │    Hst40_Nuclei_Multiome-v1     │          9          │
│                        │    Hst41_Nuclei_Multiome-v1     │         10          │
│                        │                                 │                     │
│  adata.obs['region']   │              aorta              │          0          │
│                        │              apex               │          1          │
│                        │              atria              │          2          │
│                        │          atrial septum          │          3          │
│                        │        ductus arteriosus        │          4          │
│                        │          whole sample           │          5          │
│                        │       heart without node        │          6          │
│                        │         left ventricle          │          7          │
│                        │              node               │          8          │
│                        │          outflow tract          │          9          │
│                        │           pericardium           │         10          │
│                        │  pulmonary arches and branches  │         11          │
│                        │         right ventricle         │         12          │
│                        │ right ventricle and papillaries │         13          │
│                        │                                 │                     │
└────────────────────────┴─────────────────────────────────┴─────────────────────┘
```

Example 33 (python):
```python
extra_continuous_covs State   
            Registry            
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃       Source Location        ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│  adata.obs['total_counts']   │
│  adata.obs['pct_counts_mt']  │
│ adata.obs['pct_counts_ribo'] │
│     adata.obs['S_score']     │
│    adata.obs['G2M_score']    │
└──────────────────────────────┘
```

Example 34 (unknown):
```unknown
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py:1892: PossibleUserWarning: The number of training batches (8) is smaller than the logging interval Trainer(log_every_n_steps=10). Set a lower value for log_every_n_steps if you want to see logs for the training epoch.
  rank_zero_warn(
```

Example 35 (unknown):
```unknown
Epoch 400/400: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 400/400 [01:58<00:00,  5.16it/s, loss=7.65e+03, v_num=1]
```

Example 36 (unknown):
```unknown
`Trainer.fit` stopped: `max_epochs=400` reached.
```

Example 37 (unknown):
```unknown
Epoch 400/400: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 400/400 [01:58<00:00,  3.37it/s, loss=7.65e+03, v_num=1]
```

Example 38 (unknown):
```unknown
##############################
##### one-layer: LymphoidCells #####
##############################
filtered out 12160 genes that are detected in less than 3 cells
```

Example 39 (python):
```python
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/scanpy/preprocessing/_simple.py:251: ImplicitModificationWarning: Trying to modify attribute `.var` of view, initializing view as actual.
  adata.var['n_cells'] = number
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/scvi/data/_utils.py:168: UserWarning: Category 4 in adata.obs['batch_key'] has fewer than 3 cells. Models may not train properly.
  warnings.warn(
```

Example 40 (unknown):
```unknown
SCVI Model with the following params: 
n_hidden: 128, n_latent: 50, n_layers: 1, dropout_rate: 0.1, dispersion: gene-batch, gene_likelihood: zinb, 
latent_distribution: normal
Training status: Not Trained
```

Example 41 (unknown):
```unknown
Anndata setup with scvi-tools version 0.19.0.
```

Example 42 (unknown):
```unknown
Setup via `SCVI.setup_anndata` with arguments:
```

Example 43 (unknown):
```unknown
{
│   'layer': None,
│   'batch_key': None,
│   'labels_key': None,
│   'size_factor_key': None,
│   'categorical_covariate_keys': ['batch_key', 'region'],
│   'continuous_covariate_keys': ['total_counts', 'pct_counts_mt', 'pct_counts_ribo', 'S_score', 'G2M_score']
}
```

Example 44 (unknown):
```unknown
Summary Statistics         
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┓
┃     Summary Stat Key     ┃ Value ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━┩
│         n_batch          │   1   │
│         n_cells          │ 7006  │
│ n_extra_categorical_covs │   2   │
│ n_extra_continuous_covs  │   5   │
│         n_labels         │   1   │
│          n_vars          │ 24441 │
└──────────────────────────┴───────┘
```

Example 45 (python):
```python
Data Registry                             
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃      Registry Key      ┃            scvi-tools Location             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│           X            │                  adata.X                   │
│         batch          │          adata.obs['_scvi_batch']          │
│ extra_categorical_covs │ adata.obsm['_scvi_extra_categorical_covs'] │
│ extra_continuous_covs  │ adata.obsm['_scvi_extra_continuous_covs']  │
│         labels         │         adata.obs['_scvi_labels']          │
└────────────────────────┴────────────────────────────────────────────┘
```

Example 46 (python):
```python
batch State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃     Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_batch'] │     0      │          0          │
└──────────────────────────┴────────────┴─────────────────────┘
```

Example 47 (python):
```python
labels State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 48 (python):
```python
extra_categorical_covs State Registry                       
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location     ┃           Categories            ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['batch_key'] │     C82_Nuclei_Multiome-v1      │          0          │
│                        │     C85_Nuclei_Multiome-v1      │          1          │
│                        │     C87_Nuclei_Multiome-v1      │          2          │
│                        │     C92_Nuclei_Multiome-v1      │          3          │
│                        │     C98_Nuclei_Multiome-v1      │          4          │
│                        │    Hst33_Nuclei_Multiome-v1     │          5          │
│                        │    Hst36_Nuclei_Multiome-v1     │          6          │
│                        │    Hst39_Nuclei_Multiome-v1     │          7          │
│                        │    Hst40_Nuclei_Multiome-v1     │          8          │
│                        │    Hst41_Nuclei_Multiome-v1     │          9          │
│                        │                                 │                     │
│  adata.obs['region']   │              aorta              │          0          │
│                        │              atria              │          1          │
│                        │          atrial septum          │          2          │
│                        │        ductus arteriosus        │          3          │
│                        │          whole sample           │          4          │
│                        │       heart without node        │          5          │
│                        │         left ventricle          │          6          │
│                        │              node               │          7          │
│                        │           pericardium           │          8          │
│                        │  pulmonary arches and branches  │          9          │
│                        │         right ventricle         │         10          │
│                        │ right ventricle and papillaries │         11          │
│                        │                                 │                     │
└────────────────────────┴─────────────────────────────────┴─────────────────────┘
```

Example 49 (python):
```python
extra_continuous_covs State   
            Registry            
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃       Source Location        ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│  adata.obs['total_counts']   │
│  adata.obs['pct_counts_mt']  │
│ adata.obs['pct_counts_ribo'] │
│     adata.obs['S_score']     │
│    adata.obs['G2M_score']    │
└──────────────────────────────┘
```

Example 50 (unknown):
```unknown
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
```

Example 51 (unknown):
```unknown
Epoch 400/400: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 400/400 [09:14<00:00,  1.38s/it, loss=4.76e+03, v_num=1]
```

Example 52 (unknown):
```unknown
`Trainer.fit` stopped: `max_epochs=400` reached.
```

Example 53 (unknown):
```unknown
Epoch 400/400: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 400/400 [09:14<00:00,  1.39s/it, loss=4.76e+03, v_num=1]
```

Example 54 (unknown):
```unknown
##############################
##### one-layer: MuralCells #####
##############################
filtered out 13545 genes that are detected in less than 3 cells
```

Example 55 (python):
```python
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/scanpy/preprocessing/_simple.py:251: ImplicitModificationWarning: Trying to modify attribute `.var` of view, initializing view as actual.
  adata.var['n_cells'] = number
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/scvi/data/_utils.py:168: UserWarning: Category 1 in adata.obs['region'] has fewer than 3 cells. Models may not train properly.
  warnings.warn(
```

Example 56 (unknown):
```unknown
SCVI Model with the following params: 
n_hidden: 128, n_latent: 50, n_layers: 1, dropout_rate: 0.1, dispersion: gene-batch, gene_likelihood: zinb, 
latent_distribution: normal
Training status: Not Trained
```

Example 57 (unknown):
```unknown
Anndata setup with scvi-tools version 0.19.0.
```

Example 58 (unknown):
```unknown
Setup via `SCVI.setup_anndata` with arguments:
```

Example 59 (unknown):
```unknown
{
│   'layer': None,
│   'batch_key': None,
│   'labels_key': None,
│   'size_factor_key': None,
│   'categorical_covariate_keys': ['batch_key', 'region'],
│   'continuous_covariate_keys': ['total_counts', 'pct_counts_mt', 'pct_counts_ribo', 'S_score', 'G2M_score']
}
```

Example 60 (unknown):
```unknown
Summary Statistics         
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┓
┃     Summary Stat Key     ┃ Value ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━┩
│         n_batch          │   1   │
│         n_cells          │ 3103  │
│ n_extra_categorical_covs │   2   │
│ n_extra_continuous_covs  │   5   │
│         n_labels         │   1   │
│          n_vars          │ 23056 │
└──────────────────────────┴───────┘
```

Example 61 (python):
```python
Data Registry                             
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃      Registry Key      ┃            scvi-tools Location             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│           X            │                  adata.X                   │
│         batch          │          adata.obs['_scvi_batch']          │
│ extra_categorical_covs │ adata.obsm['_scvi_extra_categorical_covs'] │
│ extra_continuous_covs  │ adata.obsm['_scvi_extra_continuous_covs']  │
│         labels         │         adata.obs['_scvi_labels']          │
└────────────────────────┴────────────────────────────────────────────┘
```

Example 62 (python):
```python
batch State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃     Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_batch'] │     0      │          0          │
└──────────────────────────┴────────────┴─────────────────────┘
```

Example 63 (python):
```python
labels State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 64 (python):
```python
extra_categorical_covs State Registry                       
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location     ┃           Categories            ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['batch_key'] │     C82_Nuclei_Multiome-v1      │          0          │
│                        │     C85_Nuclei_Multiome-v1      │          1          │
│                        │     C87_Nuclei_Multiome-v1      │          2          │
│                        │     C92_Nuclei_Multiome-v1      │          3          │
│                        │     C98_Nuclei_Multiome-v1      │          4          │
│                        │     C104_Nuclei_Multiome-v1     │          5          │
│                        │    Hst33_Nuclei_Multiome-v1     │          6          │
│                        │    Hst36_Nuclei_Multiome-v1     │          7          │
│                        │    Hst39_Nuclei_Multiome-v1     │          8          │
│                        │    Hst40_Nuclei_Multiome-v1     │          9          │
│                        │    Hst41_Nuclei_Multiome-v1     │         10          │
│                        │                                 │                     │
│  adata.obs['region']   │              aorta              │          0          │
│                        │              apex               │          1          │
│                        │              atria              │          2          │
│                        │          atrial septum          │          3          │
│                        │        ductus arteriosus        │          4          │
│                        │          whole sample           │          5          │
│                        │       heart without node        │          6          │
│                        │         left ventricle          │          7          │
│                        │              node               │          8          │
│                        │          outflow tract          │          9          │
│                        │           pericardium           │         10          │
│                        │  pulmonary arches and branches  │         11          │
│                        │         right ventricle         │         12          │
│                        │ right ventricle and papillaries │         13          │
│                        │                                 │                     │
└────────────────────────┴─────────────────────────────────┴─────────────────────┘
```

Example 65 (python):
```python
extra_continuous_covs State   
            Registry            
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃       Source Location        ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│  adata.obs['total_counts']   │
│  adata.obs['pct_counts_mt']  │
│ adata.obs['pct_counts_ribo'] │
│     adata.obs['S_score']     │
│    adata.obs['G2M_score']    │
└──────────────────────────────┘
```

Example 66 (unknown):
```unknown
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
```

Example 67 (unknown):
```unknown
Epoch 400/400: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 400/400 [04:05<00:00,  1.62it/s, loss=4.98e+03, v_num=1]
```

Example 68 (unknown):
```unknown
`Trainer.fit` stopped: `max_epochs=400` reached.
```

Example 69 (unknown):
```unknown
Epoch 400/400: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 400/400 [04:05<00:00,  1.63it/s, loss=4.98e+03, v_num=1]
```

Example 70 (unknown):
```unknown
##############################
##### one-layer: VentricularCardiomyocytes_plusCCS #####
##############################
filtered out 12175 genes that are detected in less than 3 cells
```

Example 71 (python):
```python
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/scanpy/preprocessing/_simple.py:251: ImplicitModificationWarning: Trying to modify attribute `.var` of view, initializing view as actual.
  adata.var['n_cells'] = number
```

Example 72 (unknown):
```unknown
SCVI Model with the following params: 
n_hidden: 128, n_latent: 50, n_layers: 1, dropout_rate: 0.1, dispersion: gene-batch, gene_likelihood: zinb, 
latent_distribution: normal
Training status: Not Trained
```

Example 73 (unknown):
```unknown
Anndata setup with scvi-tools version 0.19.0.
```

Example 74 (unknown):
```unknown
Setup via `SCVI.setup_anndata` with arguments:
```

Example 75 (unknown):
```unknown
{
│   'layer': None,
│   'batch_key': None,
│   'labels_key': None,
│   'size_factor_key': None,
│   'categorical_covariate_keys': ['batch_key', 'region'],
│   'continuous_covariate_keys': ['total_counts', 'pct_counts_mt', 'pct_counts_ribo', 'S_score', 'G2M_score']
}
```

Example 76 (unknown):
```unknown
Summary Statistics         
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┓
┃     Summary Stat Key     ┃ Value ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━┩
│         n_batch          │   1   │
│         n_cells          │ 6531  │
│ n_extra_categorical_covs │   2   │
│ n_extra_continuous_covs  │   5   │
│         n_labels         │   1   │
│          n_vars          │ 24426 │
└──────────────────────────┴───────┘
```

Example 77 (python):
```python
Data Registry                             
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃      Registry Key      ┃            scvi-tools Location             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│           X            │                  adata.X                   │
│         batch          │          adata.obs['_scvi_batch']          │
│ extra_categorical_covs │ adata.obsm['_scvi_extra_categorical_covs'] │
│ extra_continuous_covs  │ adata.obsm['_scvi_extra_continuous_covs']  │
│         labels         │         adata.obs['_scvi_labels']          │
└────────────────────────┴────────────────────────────────────────────┘
```

Example 78 (python):
```python
batch State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃     Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_batch'] │     0      │          0          │
└──────────────────────────┴────────────┴─────────────────────┘
```

Example 79 (python):
```python
labels State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 80 (python):
```python
extra_categorical_covs State Registry                       
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location     ┃           Categories            ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['batch_key'] │     C82_Nuclei_Multiome-v1      │          0          │
│                        │     C85_Nuclei_Multiome-v1      │          1          │
│                        │     C87_Nuclei_Multiome-v1      │          2          │
│                        │     C92_Nuclei_Multiome-v1      │          3          │
│                        │     C98_Nuclei_Multiome-v1      │          4          │
│                        │     C104_Nuclei_Multiome-v1     │          5          │
│                        │    Hst33_Nuclei_Multiome-v1     │          6          │
│                        │    Hst36_Nuclei_Multiome-v1     │          7          │
│                        │    Hst39_Nuclei_Multiome-v1     │          8          │
│                        │    Hst40_Nuclei_Multiome-v1     │          9          │
│                        │    Hst41_Nuclei_Multiome-v1     │         10          │
│                        │                                 │                     │
│  adata.obs['region']   │              apex               │          0          │
│                        │              atria              │          1          │
│                        │          atrial septum          │          2          │
│                        │          whole sample           │          3          │
│                        │       heart without node        │          4          │
│                        │         left ventricle          │          5          │
│                        │              node               │          6          │
│                        │          outflow tract          │          7          │
│                        │         right ventricle         │          8          │
│                        │ right ventricle and papillaries │          9          │
│                        │                                 │                     │
└────────────────────────┴─────────────────────────────────┴─────────────────────┘
```

Example 81 (python):
```python
extra_continuous_covs State   
            Registry            
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃       Source Location        ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│  adata.obs['total_counts']   │
│  adata.obs['pct_counts_mt']  │
│ adata.obs['pct_counts_ribo'] │
│     adata.obs['S_score']     │
│    adata.obs['G2M_score']    │
└──────────────────────────────┘
```

Example 82 (unknown):
```unknown
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
```

Example 83 (unknown):
```unknown
Epoch 400/400: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 400/400 [08:18<00:00,  1.30s/it, loss=6.25e+03, v_num=1]
```

Example 84 (unknown):
```unknown
`Trainer.fit` stopped: `max_epochs=400` reached.
```

Example 85 (unknown):
```unknown
Epoch 400/400: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 400/400 [08:18<00:00,  1.25s/it, loss=6.25e+03, v_num=1]
```

Example 86 (unknown):
```unknown
##############################
##### one-layer: Glia #####
##############################
filtered out 15066 genes that are detected in less than 3 cells
```

Example 87 (python):
```python
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/scanpy/preprocessing/_simple.py:251: ImplicitModificationWarning: Trying to modify attribute `.var` of view, initializing view as actual.
  adata.var['n_cells'] = number
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/scvi/data/_utils.py:168: UserWarning: Category 5 in adata.obs['batch_key'] has fewer than 3 cells. Models may not train properly.
  warnings.warn(
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/scvi/data/_utils.py:168: UserWarning: Category 8 in adata.obs['region'] has fewer than 3 cells. Models may not train properly.
  warnings.warn(
```

Example 88 (unknown):
```unknown
SCVI Model with the following params: 
n_hidden: 128, n_latent: 50, n_layers: 1, dropout_rate: 0.1, dispersion: gene-batch, gene_likelihood: zinb, 
latent_distribution: normal
Training status: Not Trained
```

Example 89 (unknown):
```unknown
Anndata setup with scvi-tools version 0.19.0.
```

Example 90 (unknown):
```unknown
Setup via `SCVI.setup_anndata` with arguments:
```

Example 91 (unknown):
```unknown
{
│   'layer': None,
│   'batch_key': None,
│   'labels_key': None,
│   'size_factor_key': None,
│   'categorical_covariate_keys': ['batch_key', 'region'],
│   'continuous_covariate_keys': ['total_counts', 'pct_counts_mt', 'pct_counts_ribo', 'S_score', 'G2M_score']
}
```

Example 92 (unknown):
```unknown
Summary Statistics         
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┓
┃     Summary Stat Key     ┃ Value ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━┩
│         n_batch          │   1   │
│         n_cells          │ 1759  │
│ n_extra_categorical_covs │   2   │
│ n_extra_continuous_covs  │   5   │
│         n_labels         │   1   │
│          n_vars          │ 21535 │
└──────────────────────────┴───────┘
```

Example 93 (python):
```python
Data Registry                             
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃      Registry Key      ┃            scvi-tools Location             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│           X            │                  adata.X                   │
│         batch          │          adata.obs['_scvi_batch']          │
│ extra_categorical_covs │ adata.obsm['_scvi_extra_categorical_covs'] │
│ extra_continuous_covs  │ adata.obsm['_scvi_extra_continuous_covs']  │
│         labels         │         adata.obs['_scvi_labels']          │
└────────────────────────┴────────────────────────────────────────────┘
```

Example 94 (python):
```python
batch State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃     Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_batch'] │     0      │          0          │
└──────────────────────────┴────────────┴─────────────────────┘
```

Example 95 (python):
```python
labels State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 96 (python):
```python
extra_categorical_covs State Registry                       
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location     ┃           Categories            ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['batch_key'] │     C82_Nuclei_Multiome-v1      │          0          │
│                        │     C85_Nuclei_Multiome-v1      │          1          │
│                        │     C87_Nuclei_Multiome-v1      │          2          │
│                        │     C92_Nuclei_Multiome-v1      │          3          │
│                        │     C98_Nuclei_Multiome-v1      │          4          │
│                        │     C104_Nuclei_Multiome-v1     │          5          │
│                        │    Hst33_Nuclei_Multiome-v1     │          6          │
│                        │    Hst36_Nuclei_Multiome-v1     │          7          │
│                        │    Hst39_Nuclei_Multiome-v1     │          8          │
│                        │    Hst40_Nuclei_Multiome-v1     │          9          │
│                        │    Hst41_Nuclei_Multiome-v1     │         10          │
│                        │                                 │                     │
│  adata.obs['region']   │              aorta              │          0          │
│                        │              atria              │          1          │
│                        │          atrial septum          │          2          │
│                        │        ductus arteriosus        │          3          │
│                        │          whole sample           │          4          │
│                        │       heart without node        │          5          │
│                        │         left ventricle          │          6          │
│                        │              node               │          7          │
│                        │          outflow tract          │          8          │
│                        │           pericardium           │          9          │
│                        │  pulmonary arches and branches  │         10          │
│                        │         right ventricle         │         11          │
│                        │ right ventricle and papillaries │         12          │
│                        │                                 │                     │
└────────────────────────┴─────────────────────────────────┴─────────────────────┘
```

Example 97 (python):
```python
extra_continuous_covs State   
            Registry            
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃       Source Location        ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│  adata.obs['total_counts']   │
│  adata.obs['pct_counts_mt']  │
│ adata.obs['pct_counts_ribo'] │
│     adata.obs['S_score']     │
│    adata.obs['G2M_score']    │
└──────────────────────────────┘
```

Example 98 (unknown):
```unknown
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
```

Example 99 (unknown):
```unknown
Epoch 400/400: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 400/400 [02:20<00:00,  2.89it/s, loss=5.82e+03, v_num=1]
```

Example 100 (unknown):
```unknown
`Trainer.fit` stopped: `max_epochs=400` reached.
```

Example 101 (unknown):
```unknown
Epoch 400/400: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 400/400 [02:20<00:00,  2.84it/s, loss=5.82e+03, v_num=1]
```

Example 102 (unknown):
```unknown
##############################
##### one-layer: MyeloidCells #####
##############################
filtered out 12311 genes that are detected in less than 3 cells
```

Example 103 (python):
```python
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/scanpy/preprocessing/_simple.py:251: ImplicitModificationWarning: Trying to modify attribute `.var` of view, initializing view as actual.
  adata.var['n_cells'] = number
```

Example 104 (unknown):
```unknown
SCVI Model with the following params: 
n_hidden: 128, n_latent: 50, n_layers: 1, dropout_rate: 0.1, dispersion: gene-batch, gene_likelihood: zinb, 
latent_distribution: normal
Training status: Not Trained
```

Example 105 (unknown):
```unknown
Anndata setup with scvi-tools version 0.19.0.
```

Example 106 (unknown):
```unknown
Setup via `SCVI.setup_anndata` with arguments:
```

Example 107 (unknown):
```unknown
{
│   'layer': None,
│   'batch_key': None,
│   'labels_key': None,
│   'size_factor_key': None,
│   'categorical_covariate_keys': ['batch_key', 'region'],
│   'continuous_covariate_keys': ['total_counts', 'pct_counts_mt', 'pct_counts_ribo', 'S_score', 'G2M_score']
}
```

Example 108 (unknown):
```unknown
Summary Statistics         
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┓
┃     Summary Stat Key     ┃ Value ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━┩
│         n_batch          │   1   │
│         n_cells          │ 4321  │
│ n_extra_categorical_covs │   2   │
│ n_extra_continuous_covs  │   5   │
│         n_labels         │   1   │
│          n_vars          │ 24290 │
└──────────────────────────┴───────┘
```

Example 109 (python):
```python
Data Registry                             
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃      Registry Key      ┃            scvi-tools Location             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│           X            │                  adata.X                   │
│         batch          │          adata.obs['_scvi_batch']          │
│ extra_categorical_covs │ adata.obsm['_scvi_extra_categorical_covs'] │
│ extra_continuous_covs  │ adata.obsm['_scvi_extra_continuous_covs']  │
│         labels         │         adata.obs['_scvi_labels']          │
└────────────────────────┴────────────────────────────────────────────┘
```

Example 110 (python):
```python
batch State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃     Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_batch'] │     0      │          0          │
└──────────────────────────┴────────────┴─────────────────────┘
```

Example 111 (python):
```python
labels State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 112 (python):
```python
extra_categorical_covs State Registry                       
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location     ┃           Categories            ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['batch_key'] │     C82_Nuclei_Multiome-v1      │          0          │
│                        │     C85_Nuclei_Multiome-v1      │          1          │
│                        │     C87_Nuclei_Multiome-v1      │          2          │
│                        │     C92_Nuclei_Multiome-v1      │          3          │
│                        │     C98_Nuclei_Multiome-v1      │          4          │
│                        │     C104_Nuclei_Multiome-v1     │          5          │
│                        │    Hst33_Nuclei_Multiome-v1     │          6          │
│                        │    Hst36_Nuclei_Multiome-v1     │          7          │
│                        │    Hst39_Nuclei_Multiome-v1     │          8          │
│                        │    Hst40_Nuclei_Multiome-v1     │          9          │
│                        │    Hst41_Nuclei_Multiome-v1     │         10          │
│                        │                                 │                     │
│  adata.obs['region']   │              aorta              │          0          │
│                        │              apex               │          1          │
│                        │              atria              │          2          │
│                        │          atrial septum          │          3          │
│                        │        ductus arteriosus        │          4          │
│                        │          whole sample           │          5          │
│                        │       heart without node        │          6          │
│                        │         left ventricle          │          7          │
│                        │              node               │          8          │
│                        │          outflow tract          │          9          │
│                        │           pericardium           │         10          │
│                        │  pulmonary arches and branches  │         11          │
│                        │         right ventricle         │         12          │
│                        │ right ventricle and papillaries │         13          │
│                        │                                 │                     │
└────────────────────────┴─────────────────────────────────┴─────────────────────┘
```

Example 113 (python):
```python
extra_continuous_covs State   
            Registry            
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃       Source Location        ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│  adata.obs['total_counts']   │
│  adata.obs['pct_counts_mt']  │
│ adata.obs['pct_counts_ribo'] │
│     adata.obs['S_score']     │
│    adata.obs['G2M_score']    │
└──────────────────────────────┘
```

Example 114 (unknown):
```unknown
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
```

Example 115 (unknown):
```unknown
Epoch 400/400: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 400/400 [05:34<00:00,  1.15it/s, loss=5.32e+03, v_num=1]
```

Example 116 (unknown):
```unknown
`Trainer.fit` stopped: `max_epochs=400` reached.
```

Example 117 (unknown):
```unknown
Epoch 400/400: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 400/400 [05:34<00:00,  1.20it/s, loss=5.32e+03, v_num=1]
```

Example 118 (unknown):
```unknown
##############################
##### one-layer: PericardialCells #####
##############################
filtered out 14085 genes that are detected in less than 3 cells
```

Example 119 (python):
```python
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/scanpy/preprocessing/_simple.py:251: ImplicitModificationWarning: Trying to modify attribute `.var` of view, initializing view as actual.
  adata.var['n_cells'] = number
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/scvi/data/_utils.py:168: UserWarning: Category 3 in adata.obs['batch_key'] has fewer than 3 cells. Models may not train properly.
  warnings.warn(
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/scvi/data/_utils.py:168: UserWarning: Category 0 in adata.obs['region'] has fewer than 3 cells. Models may not train properly.
  warnings.warn(
```

Example 120 (unknown):
```unknown
SCVI Model with the following params: 
n_hidden: 128, n_latent: 50, n_layers: 1, dropout_rate: 0.1, dispersion: gene-batch, gene_likelihood: zinb, 
latent_distribution: normal
Training status: Not Trained
```

Example 121 (unknown):
```unknown
Anndata setup with scvi-tools version 0.19.0.
```

Example 122 (unknown):
```unknown
Setup via `SCVI.setup_anndata` with arguments:
```

Example 123 (unknown):
```unknown
{
│   'layer': None,
│   'batch_key': None,
│   'labels_key': None,
│   'size_factor_key': None,
│   'categorical_covariate_keys': ['batch_key', 'region'],
│   'continuous_covariate_keys': ['total_counts', 'pct_counts_mt', 'pct_counts_ribo', 'S_score', 'G2M_score']
}
```

Example 124 (unknown):
```unknown
Summary Statistics         
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┓
┃     Summary Stat Key     ┃ Value ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━┩
│         n_batch          │   1   │
│         n_cells          │ 2187  │
│ n_extra_categorical_covs │   2   │
│ n_extra_continuous_covs  │   5   │
│         n_labels         │   1   │
│          n_vars          │ 22516 │
└──────────────────────────┴───────┘
```

Example 125 (python):
```python
Data Registry                             
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃      Registry Key      ┃            scvi-tools Location             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│           X            │                  adata.X                   │
│         batch          │          adata.obs['_scvi_batch']          │
│ extra_categorical_covs │ adata.obsm['_scvi_extra_categorical_covs'] │
│ extra_continuous_covs  │ adata.obsm['_scvi_extra_continuous_covs']  │
│         labels         │         adata.obs['_scvi_labels']          │
└────────────────────────┴────────────────────────────────────────────┘
```

Example 126 (python):
```python
batch State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃     Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_batch'] │     0      │          0          │
└──────────────────────────┴────────────┴─────────────────────┘
```

Example 127 (python):
```python
labels State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 128 (python):
```python
extra_categorical_covs State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location     ┃          Categories           ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['batch_key'] │    C87_Nuclei_Multiome-v1     │          0          │
│                        │    C92_Nuclei_Multiome-v1     │          1          │
│                        │    C98_Nuclei_Multiome-v1     │          2          │
│                        │    C104_Nuclei_Multiome-v1    │          3          │
│                        │   Hst39_Nuclei_Multiome-v1    │          4          │
│                        │   Hst40_Nuclei_Multiome-v1    │          5          │
│                        │   Hst41_Nuclei_Multiome-v1    │          6          │
│                        │                               │                     │
│  adata.obs['region']   │             aorta             │          0          │
│                        │         atrial septum         │          1          │
│                        │       ductus arteriosus       │          2          │
│                        │         whole sample          │          3          │
│                        │      heart without node       │          4          │
│                        │         outflow tract         │          5          │
│                        │          pericardium          │          6          │
│                        │ pulmonary arches and branches │          7          │
│                        │                               │                     │
└────────────────────────┴───────────────────────────────┴─────────────────────┘
```

Example 129 (python):
```python
extra_continuous_covs State   
            Registry            
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃       Source Location        ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│  adata.obs['total_counts']   │
│  adata.obs['pct_counts_mt']  │
│ adata.obs['pct_counts_ribo'] │
│     adata.obs['S_score']     │
│    adata.obs['G2M_score']    │
└──────────────────────────────┘
```

Example 130 (unknown):
```unknown
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
```

Example 131 (unknown):
```unknown
Epoch 400/400: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 400/400 [02:53<00:00,  2.29it/s, loss=5.98e+03, v_num=1]
```

Example 132 (unknown):
```unknown
`Trainer.fit` stopped: `max_epochs=400` reached.
```

Example 133 (unknown):
```unknown
Epoch 400/400: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 400/400 [02:53<00:00,  2.31it/s, loss=5.98e+03, v_num=1]
```

Example 134 (unknown):
```unknown
##############################
##### one-layer: BloodVesselEndothelialCells #####
##############################
filtered out 12573 genes that are detected in less than 3 cells
```

Example 135 (python):
```python
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/scanpy/preprocessing/_simple.py:251: ImplicitModificationWarning: Trying to modify attribute `.var` of view, initializing view as actual.
  adata.var['n_cells'] = number
```

Example 136 (unknown):
```unknown
SCVI Model with the following params: 
n_hidden: 128, n_latent: 50, n_layers: 1, dropout_rate: 0.1, dispersion: gene-batch, gene_likelihood: zinb, 
latent_distribution: normal
Training status: Not Trained
```

Example 137 (unknown):
```unknown
Anndata setup with scvi-tools version 0.19.0.
```

Example 138 (unknown):
```unknown
Setup via `SCVI.setup_anndata` with arguments:
```

Example 139 (unknown):
```unknown
{
│   'layer': None,
│   'batch_key': None,
│   'labels_key': None,
│   'size_factor_key': None,
│   'categorical_covariate_keys': ['batch_key', 'region'],
│   'continuous_covariate_keys': ['total_counts', 'pct_counts_mt', 'pct_counts_ribo', 'S_score', 'G2M_score']
}
```

Example 140 (unknown):
```unknown
Summary Statistics         
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┓
┃     Summary Stat Key     ┃ Value ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━┩
│         n_batch          │   1   │
│         n_cells          │ 3028  │
│ n_extra_categorical_covs │   2   │
│ n_extra_continuous_covs  │   5   │
│         n_labels         │   1   │
│          n_vars          │ 24028 │
└──────────────────────────┴───────┘
```

Example 141 (python):
```python
Data Registry                             
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃      Registry Key      ┃            scvi-tools Location             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│           X            │                  adata.X                   │
│         batch          │          adata.obs['_scvi_batch']          │
│ extra_categorical_covs │ adata.obsm['_scvi_extra_categorical_covs'] │
│ extra_continuous_covs  │ adata.obsm['_scvi_extra_continuous_covs']  │
│         labels         │         adata.obs['_scvi_labels']          │
└────────────────────────┴────────────────────────────────────────────┘
```

Example 142 (python):
```python
batch State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃     Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_batch'] │     0      │          0          │
└──────────────────────────┴────────────┴─────────────────────┘
```

Example 143 (python):
```python
labels State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 144 (python):
```python
extra_categorical_covs State Registry                       
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location     ┃           Categories            ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['batch_key'] │     C82_Nuclei_Multiome-v1      │          0          │
│                        │     C85_Nuclei_Multiome-v1      │          1          │
│                        │     C87_Nuclei_Multiome-v1      │          2          │
│                        │     C92_Nuclei_Multiome-v1      │          3          │
│                        │     C98_Nuclei_Multiome-v1      │          4          │
│                        │     C104_Nuclei_Multiome-v1     │          5          │
│                        │    Hst33_Nuclei_Multiome-v1     │          6          │
│                        │    Hst36_Nuclei_Multiome-v1     │          7          │
│                        │    Hst39_Nuclei_Multiome-v1     │          8          │
│                        │    Hst40_Nuclei_Multiome-v1     │          9          │
│                        │    Hst41_Nuclei_Multiome-v1     │         10          │
│                        │                                 │                     │
│  adata.obs['region']   │              aorta              │          0          │
│                        │              apex               │          1          │
│                        │              atria              │          2          │
│                        │          atrial septum          │          3          │
│                        │        ductus arteriosus        │          4          │
│                        │          whole sample           │          5          │
│                        │       heart without node        │          6          │
│                        │         left ventricle          │          7          │
│                        │              node               │          8          │
│                        │          outflow tract          │          9          │
│                        │           pericardium           │         10          │
│                        │  pulmonary arches and branches  │         11          │
│                        │         right ventricle         │         12          │
│                        │ right ventricle and papillaries │         13          │
│                        │                                 │                     │
└────────────────────────┴─────────────────────────────────┴─────────────────────┘
```

Example 145 (python):
```python
extra_continuous_covs State   
            Registry            
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃       Source Location        ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│  adata.obs['total_counts']   │
│  adata.obs['pct_counts_mt']  │
│ adata.obs['pct_counts_ribo'] │
│     adata.obs['S_score']     │
│    adata.obs['G2M_score']    │
└──────────────────────────────┘
```

Example 146 (unknown):
```unknown
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
```

Example 147 (unknown):
```unknown
Epoch 400/400: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 400/400 [03:59<00:00,  1.74it/s, loss=6.45e+03, v_num=1]
```

Example 148 (unknown):
```unknown
`Trainer.fit` stopped: `max_epochs=400` reached.
```

Example 149 (unknown):
```unknown
Epoch 400/400: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 400/400 [03:59<00:00,  1.67it/s, loss=6.45e+03, v_num=1]
```

Example 150 (unknown):
```unknown
##############################
##### one-layer: EndocardialCells #####
##############################
filtered out 15447 genes that are detected in less than 3 cells
```

Example 151 (python):
```python
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/scanpy/preprocessing/_simple.py:251: ImplicitModificationWarning: Trying to modify attribute `.var` of view, initializing view as actual.
  adata.var['n_cells'] = number
```

Example 152 (unknown):
```unknown
SCVI Model with the following params: 
n_hidden: 128, n_latent: 50, n_layers: 1, dropout_rate: 0.1, dispersion: gene-batch, gene_likelihood: zinb, 
latent_distribution: normal
Training status: Not Trained
```

Example 153 (unknown):
```unknown
Anndata setup with scvi-tools version 0.19.0.
```

Example 154 (unknown):
```unknown
Setup via `SCVI.setup_anndata` with arguments:
```

Example 155 (unknown):
```unknown
{
│   'layer': None,
│   'batch_key': None,
│   'labels_key': None,
│   'size_factor_key': None,
│   'categorical_covariate_keys': ['batch_key', 'region'],
│   'continuous_covariate_keys': ['total_counts', 'pct_counts_mt', 'pct_counts_ribo', 'S_score', 'G2M_score']
}
```

Example 156 (unknown):
```unknown
Summary Statistics         
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┓
┃     Summary Stat Key     ┃ Value ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━┩
│         n_batch          │   1   │
│         n_cells          │ 1410  │
│ n_extra_categorical_covs │   2   │
│ n_extra_continuous_covs  │   5   │
│         n_labels         │   1   │
│          n_vars          │ 21154 │
└──────────────────────────┴───────┘
```

Example 157 (python):
```python
Data Registry                             
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃      Registry Key      ┃            scvi-tools Location             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│           X            │                  adata.X                   │
│         batch          │          adata.obs['_scvi_batch']          │
│ extra_categorical_covs │ adata.obsm['_scvi_extra_categorical_covs'] │
│ extra_continuous_covs  │ adata.obsm['_scvi_extra_continuous_covs']  │
│         labels         │         adata.obs['_scvi_labels']          │
└────────────────────────┴────────────────────────────────────────────┘
```

Example 158 (python):
```python
batch State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃     Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_batch'] │     0      │          0          │
└──────────────────────────┴────────────┴─────────────────────┘
```

Example 159 (python):
```python
labels State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 160 (python):
```python
extra_categorical_covs State Registry                       
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location     ┃           Categories            ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['batch_key'] │     C82_Nuclei_Multiome-v1      │          0          │
│                        │     C85_Nuclei_Multiome-v1      │          1          │
│                        │     C87_Nuclei_Multiome-v1      │          2          │
│                        │     C92_Nuclei_Multiome-v1      │          3          │
│                        │     C98_Nuclei_Multiome-v1      │          4          │
│                        │     C104_Nuclei_Multiome-v1     │          5          │
│                        │    Hst33_Nuclei_Multiome-v1     │          6          │
│                        │    Hst36_Nuclei_Multiome-v1     │          7          │
│                        │    Hst39_Nuclei_Multiome-v1     │          8          │
│                        │    Hst40_Nuclei_Multiome-v1     │          9          │
│                        │    Hst41_Nuclei_Multiome-v1     │         10          │
│                        │                                 │                     │
│  adata.obs['region']   │              aorta              │          0          │
│                        │              apex               │          1          │
│                        │              atria              │          2          │
│                        │          atrial septum          │          3          │
│                        │          whole sample           │          4          │
│                        │       heart without node        │          5          │
│                        │         left ventricle          │          6          │
│                        │              node               │          7          │
│                        │          outflow tract          │          8          │
│                        │         right ventricle         │          9          │
│                        │ right ventricle and papillaries │         10          │
│                        │                                 │                     │
└────────────────────────┴─────────────────────────────────┴─────────────────────┘
```

Example 161 (python):
```python
extra_continuous_covs State   
            Registry            
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃       Source Location        ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│  adata.obs['total_counts']   │
│  adata.obs['pct_counts_mt']  │
│ adata.obs['pct_counts_ribo'] │
│     adata.obs['S_score']     │
│    adata.obs['G2M_score']    │
└──────────────────────────────┘
```

Example 162 (unknown):
```unknown
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
```

Example 163 (unknown):
```unknown
Epoch 400/400: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 400/400 [01:50<00:00,  3.57it/s, loss=6.83e+03, v_num=1]
```

Example 164 (unknown):
```unknown
`Trainer.fit` stopped: `max_epochs=400` reached.
```

Example 165 (unknown):
```unknown
Epoch 400/400: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 400/400 [01:50<00:00,  3.61it/s, loss=6.83e+03, v_num=1]
```

Example 166 (unknown):
```unknown
##############################
##### one-layer: EpicardialCells #####
##############################
filtered out 13029 genes that are detected in less than 3 cells
```

Example 167 (python):
```python
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/scanpy/preprocessing/_simple.py:251: ImplicitModificationWarning: Trying to modify attribute `.var` of view, initializing view as actual.
  adata.var['n_cells'] = number
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/scvi/data/_utils.py:168: UserWarning: Category 13 in adata.obs['region'] has fewer than 3 cells. Models may not train properly.
  warnings.warn(
```

Example 168 (unknown):
```unknown
SCVI Model with the following params: 
n_hidden: 128, n_latent: 50, n_layers: 1, dropout_rate: 0.1, dispersion: gene-batch, gene_likelihood: zinb, 
latent_distribution: normal
Training status: Not Trained
```

Example 169 (unknown):
```unknown
Anndata setup with scvi-tools version 0.19.0.
```

Example 170 (unknown):
```unknown
Setup via `SCVI.setup_anndata` with arguments:
```

Example 171 (unknown):
```unknown
{
│   'layer': None,
│   'batch_key': None,
│   'labels_key': None,
│   'size_factor_key': None,
│   'categorical_covariate_keys': ['batch_key', 'region'],
│   'continuous_covariate_keys': ['total_counts', 'pct_counts_mt', 'pct_counts_ribo', 'S_score', 'G2M_score']
}
```

Example 172 (unknown):
```unknown
Summary Statistics         
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┓
┃     Summary Stat Key     ┃ Value ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━┩
│         n_batch          │   1   │
│         n_cells          │ 1340  │
│ n_extra_categorical_covs │   2   │
│ n_extra_continuous_covs  │   5   │
│         n_labels         │   1   │
│          n_vars          │ 23572 │
└──────────────────────────┴───────┘
```

Example 173 (python):
```python
Data Registry                             
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃      Registry Key      ┃            scvi-tools Location             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│           X            │                  adata.X                   │
│         batch          │          adata.obs['_scvi_batch']          │
│ extra_categorical_covs │ adata.obsm['_scvi_extra_categorical_covs'] │
│ extra_continuous_covs  │ adata.obsm['_scvi_extra_continuous_covs']  │
│         labels         │         adata.obs['_scvi_labels']          │
└────────────────────────┴────────────────────────────────────────────┘
```

Example 174 (python):
```python
batch State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃     Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_batch'] │     0      │          0          │
└──────────────────────────┴────────────┴─────────────────────┘
```

Example 175 (python):
```python
labels State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 176 (python):
```python
extra_categorical_covs State Registry                       
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location     ┃           Categories            ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['batch_key'] │     C82_Nuclei_Multiome-v1      │          0          │
│                        │     C85_Nuclei_Multiome-v1      │          1          │
│                        │     C87_Nuclei_Multiome-v1      │          2          │
│                        │     C92_Nuclei_Multiome-v1      │          3          │
│                        │     C98_Nuclei_Multiome-v1      │          4          │
│                        │     C104_Nuclei_Multiome-v1     │          5          │
│                        │    Hst33_Nuclei_Multiome-v1     │          6          │
│                        │    Hst36_Nuclei_Multiome-v1     │          7          │
│                        │    Hst39_Nuclei_Multiome-v1     │          8          │
│                        │    Hst40_Nuclei_Multiome-v1     │          9          │
│                        │    Hst41_Nuclei_Multiome-v1     │         10          │
│                        │                                 │                     │
│  adata.obs['region']   │              aorta              │          0          │
│                        │              apex               │          1          │
│                        │              atria              │          2          │
│                        │          atrial septum          │          3          │
│                        │        ductus arteriosus        │          4          │
│                        │          whole sample           │          5          │
│                        │       heart without node        │          6          │
│                        │         left ventricle          │          7          │
│                        │              node               │          8          │
│                        │          outflow tract          │          9          │
│                        │           pericardium           │         10          │
│                        │  pulmonary arches and branches  │         11          │
│                        │         right ventricle         │         12          │
│                        │ right ventricle and papillaries │         13          │
│                        │                                 │                     │
└────────────────────────┴─────────────────────────────────┴─────────────────────┘
```

Example 177 (python):
```python
extra_continuous_covs State   
            Registry            
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃       Source Location        ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│  adata.obs['total_counts']   │
│  adata.obs['pct_counts_mt']  │
│ adata.obs['pct_counts_ribo'] │
│     adata.obs['S_score']     │
│    adata.obs['G2M_score']    │
└──────────────────────────────┘
```

Example 178 (unknown):
```unknown
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
```

Example 179 (unknown):
```unknown
Epoch 264/400:  66%|████████████████████████████████████████████████████████████████████████████████████████████▍                                               | 264/400 [01:16<00:38,  3.51it/s, loss=9.89e+03, v_num=1]
```

Example 180 (python):
```python
len(set(adata.obs[celltype_col]))
```

---

## Import required modules¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig2_Overview_of_cellular_niches_and_TissueTypist/tissuetypist_VisiumHD_TissueTypist-prediction_plotting.html

**Contents:**
- Import required modules¶
- Read in adata¶
- Plotting tissue labels¶
- Plotting cropped images - predicted labels¶
  - Atrium¶
  - Ventricle¶
  - Great vessel¶
- Plotting cropped images - cell types¶
  - Atrium¶
- UMAP¶

**Examples:**

Example 1 (python):
```python
import warnings
warnings.simplefilter(action='ignore', category=FutureWarning)
warnings.filterwarnings('ignore')

import scanpy as sc
import anndata
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 
import seaborn as sns
import os 
from mpl_toolkits.axes_grid1 import make_axes_locatable

import celltypist
```

Example 2 (python):
```python
# import session_info
# session_info.show()
```

Example 3 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)

spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/tissueaxis.py")
tissueaxis = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = tissueaxis
spec.loader.exec_module(tissueaxis)
```

Example 4 (python):
```python
sc.settings.verbosity = 1          # verbosity: errors (0), warnings (1), info (2), hints (3)
sc.settings.set_figure_params(dpi=80,dpi_save=300)
```

Example 5 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk/Aug2024'
table_dir = '/nfs/team205/heart/fetal_ms_figures/tables/Aug2024'
```

Example 6 (unknown):
```unknown
os.getcwd()
```

Example 7 (unknown):
```unknown
'/nfs/team205/kk18/notebooks/Foetal/VisiumHD/re-seq_mpp0p3'
```

Example 8 (unknown):
```unknown
# set palette and categorie order
AnnotationPalette={ 
    'Atrium':'#eac075',
        'Ventricle - Compact':'#f1acfa',
     'Ventricle - Trabeculated':'#f09951',
        'SAnode':'#8A2BE2',
        'AVnode':'#062b4a',
    'VCS - Proximal':'#FFFF00',
        'VCS - Distal':'#7FFFD4',
        'Epicardium':'#01ccc8',
        'Endocardium - Atrial':'#DC143C',
        'Endocardium - Ventricular':'#f53527',
        'Endocardial cusion':'#ff7e97',
        'Great vessel':'#ff9b6e',
    'Ductus arteriosus':'#8a6800',
        'Coronary vessel':'#206601',
        'Adventitia - Coronary':'#CAFF70',
     'Adventitia - GV':'#005aa0',
        'AV ring':'#e478ff',
    'Valves':'#3300FF',
    'LN':'#9088ff',
}
```

Example 9 (python):
```python
sample_id = 'HEA_FOET14880396'
path_adata = f'/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/{sample_id}_b2c_cells_filtered_celltype-selected_lognorm.h5ad'
```

Example 10 (python):
```python
adata = sc.read_h5ad(path_adata)
adata
```

Example 11 (unknown):
```unknown
AnnData object with n_obs × n_vars = 92164 × 18047
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'fine_grain_4Jan2025', 'HistologicalAnnotation', 'cluster_cellcharter_k-8', 'cluster_cellcharter_k-10', 'cluster_cellcharter_k-12', 'cluster_cellcharter_k-14', 'cluster_cellcharter_k-16', 'cluster_cellcharter_k-18', 'cluster_cellcharter_k-20', 'cluster_cellcharter_k-25', 'cluster_cellcharter_k-30', 'cluster_cellcharter_k-35', 'cluster_cellcharter_k-40', 'cluster_cellcharter_k-45', 'cluster_cellcharter_k-50', 'predicted_labels', 'conf_score', 'predicted_labels_cellcharter-cluster-k50', 'conf_score_cellcharter-cluster-k50', 'spatial_leiden_1', 'spatial_leiden_5', 'spatial_leiden_0.5', 'spatial_leiden_2', 'spatial_leiden_10', 'spatial_leiden_100', 'spatial_leiden_1000', 'KMeans_k9216_cluster', 'predicted_labels_spatial_KMeans_k9216', 'conf_score_spatial_KMeans_k9216', 'KMeans_k4608_cluster', 'predicted_labels_spatial_KMeans_k4608', 'conf_score_spatial_KMeans_k4608', 'KMeans_k3072_cluster', 'KMeans_k1843_cluster', 'predicted_labels_spatial_KMeans_k3072', 'conf_score_spatial_KMeans_k3072', 'predicted_labels_spatial_KMeans_k1843', 'conf_score_spatial_KMeans_k1843', 'sliding_window_assignment', 'predicted_labels_sliding_window_assignment', 'conf_score_sliding_window_assignment', 'tt_prediction'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'HistologicalAnnotation_colors', 'KMeans_k1843_centroids', 'KMeans_k3072_centroids', 'KMeans_k4608_centroids', 'KMeans_k9216_centroids', 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'cluster_cellcharter_k-10_colors', 'cluster_cellcharter_k-12_colors', 'cluster_cellcharter_k-14_colors', 'cluster_cellcharter_k-16_colors', 'cluster_cellcharter_k-18_colors', 'cluster_cellcharter_k-20_colors', 'cluster_cellcharter_k-25_colors', 'cluster_cellcharter_k-30_colors', 'cluster_cellcharter_k-35_colors', 'cluster_cellcharter_k-40_colors', 'cluster_cellcharter_k-45_colors', 'cluster_cellcharter_k-50_colors', 'cluster_cellcharter_k-8_colors', 'conf_score_cellcharter-cluster-k50_colors', 'conf_score_sliding_window_assignment_colors', 'fine_grain_4Jan2025_colors', 'hvg', 'log1p', 'neighbors', 'pca', 'predicted_labels_cellcharter-cluster-k50_colors', 'predicted_labels_colors', 'predicted_labels_sliding_window_assignment_colors', 'predicted_labels_spatial_KMeans_k1843_colors', 'predicted_labels_spatial_KMeans_k3072_colors', 'predicted_labels_spatial_KMeans_k4608_colors', 'predicted_labels_spatial_KMeans_k9216_colors', 'sliding_window_assignment_colors', 'spatial', 'spatial_leiden_0.5', 'spatial_leiden_1', 'spatial_leiden_10', 'spatial_leiden_100', 'spatial_leiden_1000', 'spatial_leiden_1_colors', 'spatial_leiden_2', 'spatial_leiden_5', 'spatial_leiden_5_colors', 'spatial_neighbors', 'tt_prediction_colors', 'umap'
    obsm: 'X_cellcharter', 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances', 'spatial_connectivities', 'spatial_distances'
```

Example 12 (python):
```python
adata.X.data[:5]
```

Example 13 (unknown):
```unknown
array([1.27187559, 1.95641568, 1.75023687, 1.96760567, 2.7810601 ])
```

Example 14 (unknown):
```unknown
group_col = 'tt_prediction'
```

Example 15 (python):
```python
# plot
sc.pl.spatial(adata, 
                  color=[group_col], 
                  img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer",
                  size=2,palette=AnnotationPalette,
                  frameon=False,show=False
                 )
plt.savefig(f'{figdir}/VisiumHD_euploid_TissueTypist_predicted-labels.pdf', bbox_inches="tight",dpi=300)
```

Example 16 (unknown):
```unknown
maxp pruned
cmap pruned
kern dropped
post pruned
FFTM dropped
GPOS pruned
GSUB pruned
glyf pruned
Added gid0 to subset
Added first four glyphs to subset
Closing glyph list over 'MATH': 35 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'C', 'D', 'E', 'G', 'N', 'P', 'S', 'T', 'V', 'a', 'b', 'c', 'd', 'e', 'g', 'hyphen', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'p', 'r', 's', 'space', 't', 'u', 'underscore', 'v', 'x', 'y']
Glyph IDs:   [0, 1, 2, 3, 16, 36, 38, 39, 40, 42, 49, 51, 54, 55, 57, 66, 68, 69, 70, 71, 72, 74, 76, 79, 80, 81, 82, 83, 85, 86, 87, 88, 89, 91, 92]
Closed glyph list over 'MATH': 35 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'C', 'D', 'E', 'G', 'N', 'P', 'S', 'T', 'V', 'a', 'b', 'c', 'd', 'e', 'g', 'hyphen', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'p', 'r', 's', 'space', 't', 'u', 'underscore', 'v', 'x', 'y']
Glyph IDs:   [0, 1, 2, 3, 16, 36, 38, 39, 40, 42, 49, 51, 54, 55, 57, 66, 68, 69, 70, 71, 72, 74, 76, 79, 80, 81, 82, 83, 85, 86, 87, 88, 89, 91, 92]
Closing glyph list over 'GSUB': 35 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'C', 'D', 'E', 'G', 'N', 'P', 'S', 'T', 'V', 'a', 'b', 'c', 'd', 'e', 'g', 'hyphen', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'p', 'r', 's', 'space', 't', 'u', 'underscore', 'v', 'x', 'y']
Glyph IDs:   [0, 1, 2, 3, 16, 36, 38, 39, 40, 42, 49, 51, 54, 55, 57, 66, 68, 69, 70, 71, 72, 74, 76, 79, 80, 81, 82, 83, 85, 86, 87, 88, 89, 91, 92]
Closed glyph list over 'GSUB': 35 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'C', 'D', 'E', 'G', 'N', 'P', 'S', 'T', 'V', 'a', 'b', 'c', 'd', 'e', 'g', 'hyphen', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'p', 'r', 's', 'space', 't', 'u', 'underscore', 'v', 'x', 'y']
Glyph IDs:   [0, 1, 2, 3, 16, 36, 38, 39, 40, 42, 49, 51, 54, 55, 57, 66, 68, 69, 70, 71, 72, 74, 76, 79, 80, 81, 82, 83, 85, 86, 87, 88, 89, 91, 92]
Closing glyph list over 'glyf': 35 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'C', 'D', 'E', 'G', 'N', 'P', 'S', 'T', 'V', 'a', 'b', 'c', 'd', 'e', 'g', 'hyphen', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'p', 'r', 's', 'space', 't', 'u', 'underscore', 'v', 'x', 'y']
Glyph IDs:   [0, 1, 2, 3, 16, 36, 38, 39, 40, 42, 49, 51, 54, 55, 57, 66, 68, 69, 70, 71, 72, 74, 76, 79, 80, 81, 82, 83, 85, 86, 87, 88, 89, 91, 92]
Closed glyph list over 'glyf': 35 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'C', 'D', 'E', 'G', 'N', 'P', 'S', 'T', 'V', 'a', 'b', 'c', 'd', 'e', 'g', 'hyphen', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'p', 'r', 's', 'space', 't', 'u', 'underscore', 'v', 'x', 'y']
Glyph IDs:   [0, 1, 2, 3, 16, 36, 38, 39, 40, 42, 49, 51, 54, 55, 57, 66, 68, 69, 70, 71, 72, 74, 76, 79, 80, 81, 82, 83, 85, 86, 87, 88, 89, 91, 92]
Retaining 35 glyphs
head subsetting not needed
hhea subsetting not needed
maxp subsetting not needed
OS/2 subsetting not needed
hmtx subsetted
cmap subsetted
fpgm subsetting not needed
prep subsetting not needed
cvt  subsetting not needed
loca subsetting not needed
post subsetted
gasp subsetting not needed
MATH subsetted
GDEF subsetted
GPOS subsetted
GSUB subsetted
name subsetting not needed
glyf subsetted
head pruned
OS/2 Unicode ranges pruned: [0]
OS/2 CodePage ranges pruned: [0]
glyf pruned
GDEF pruned
GPOS pruned
GSUB pruned
name pruned
```

Example 17 (python):
```python
labels_to_plot = adata.obs[group_col].cat.categories.copy()
print(len(labels_to_plot))
labels_to_plot
```

Example 18 (unknown):
```unknown
Index(['AV ring', 'AVnode', 'Adventitia - GV', 'Atrium', 'Coronary vessel',
       'Ductus arteriosus', 'Endocardial cusion', 'Endocardium - Atrial',
       'Endocardium - Ventricular', 'Epicardium', 'Great vessel', 'SAnode',
       'VCS - Distal', 'VCS - Proximal', 'Valves', 'Ventricle - Compact',
       'Ventricle - Trabeculated'],
      dtype='object')
```

Example 19 (python):
```python
fig,axs=plt.subplots(nrows=3,ncols=6,figsize=(24,9))
# Flatten the axs array for easier iteration in case of multiple rows and columns
axs_flat = axs.flatten()

for i,label in enumerate(labels_to_plot):
    ad = adata.copy()
    # update colors
    # get color in adata
    ind = list(ad.obs[group_col].cat.categories).index(label)
    ad.uns[f'{group_col}_colors'][ind] = 'greenyellow'
    # plot
    sc.pl.spatial(ad, 
                  color=group_col, groups=label,
                  img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer",
                  size=7,legend_loc=None,
                  frameon=False,title=label,
                  show=False, ax=axs_flat[i],
                 )
# Turn off any remaining subplots
for j in range(len(labels_to_plot), len(axs_flat)):
    fig.delaxes(axs_flat[j])

# fig.suptitle('Main Title for the Figure')
fig.tight_layout()
```

Example 20 (python):
```python
mask = ((adata.obs['array_row'] >= 1500) & 
        (adata.obs['array_row'] <= 2300) & 
        (adata.obs['array_col'] >= 400) & 
        (adata.obs['array_col'] <= 1200)
       )
bdata = adata[mask]

sc.pl.spatial(bdata, 
              color=[
                      'HistologicalAnnotation'
                        ], 
                  img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer",
                  size=0,alpha_img=1,legend_loc=None,
                  frameon=False, show=False
                 )
plt.savefig(f'{figdir}/VisiumHD_euploid_Atrium_HE.pdf', bbox_inches="tight",dpi=300)
```

Example 21 (unknown):
```unknown
maxp pruned
cmap pruned
kern dropped
post pruned
FFTM dropped
GPOS pruned
GSUB pruned
glyf pruned
Added gid0 to subset
Added first four glyphs to subset
Closing glyph list over 'MATH': 15 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'H', 'a', 'c', 'g', 'i', 'l', 'n', 'nonmarkingreturn', 'o', 's', 'space', 't']
Glyph IDs:   [0, 1, 2, 3, 36, 43, 68, 70, 74, 76, 79, 81, 82, 86, 87]
Closed glyph list over 'MATH': 15 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'H', 'a', 'c', 'g', 'i', 'l', 'n', 'nonmarkingreturn', 'o', 's', 'space', 't']
Glyph IDs:   [0, 1, 2, 3, 36, 43, 68, 70, 74, 76, 79, 81, 82, 86, 87]
Closing glyph list over 'GSUB': 15 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'H', 'a', 'c', 'g', 'i', 'l', 'n', 'nonmarkingreturn', 'o', 's', 'space', 't']
Glyph IDs:   [0, 1, 2, 3, 36, 43, 68, 70, 74, 76, 79, 81, 82, 86, 87]
Closed glyph list over 'GSUB': 15 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'H', 'a', 'c', 'g', 'i', 'l', 'n', 'nonmarkingreturn', 'o', 's', 'space', 't']
Glyph IDs:   [0, 1, 2, 3, 36, 43, 68, 70, 74, 76, 79, 81, 82, 86, 87]
Closing glyph list over 'glyf': 15 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'H', 'a', 'c', 'g', 'i', 'l', 'n', 'nonmarkingreturn', 'o', 's', 'space', 't']
Glyph IDs:   [0, 1, 2, 3, 36, 43, 68, 70, 74, 76, 79, 81, 82, 86, 87]
Closed glyph list over 'glyf': 15 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'H', 'a', 'c', 'g', 'i', 'l', 'n', 'nonmarkingreturn', 'o', 's', 'space', 't']
Glyph IDs:   [0, 1, 2, 3, 36, 43, 68, 70, 74, 76, 79, 81, 82, 86, 87]
Retaining 15 glyphs
head subsetting not needed
hhea subsetting not needed
maxp subsetting not needed
OS/2 subsetting not needed
hmtx subsetted
cmap subsetted
fpgm subsetting not needed
prep subsetting not needed
cvt  subsetting not needed
loca subsetting not needed
post subsetted
gasp subsetting not needed
MATH subsetted
GDEF subsetted
GPOS subsetted
GSUB subsetted
name subsetting not needed
glyf subsetted
head pruned
OS/2 Unicode ranges pruned: [0]
OS/2 CodePage ranges pruned: [0]
glyf pruned
GDEF pruned
GPOS pruned
GSUB pruned
name pruned
```

Example 22 (python):
```python
n_annotation = len(bdata.obs['HistologicalAnnotation'].cat.categories)
bdata.uns['HistologicalAnnotation'] = list(np.repeat('#ffff00',n_annotation))

# update colors
for c in bdata.obs['HistologicalAnnotation'].cat.categories:
    # get color in adata
    ind = list(adata.obs['HistologicalAnnotation'].cat.categories).index(c)
    color = adata.uns['HistologicalAnnotation_colors'][ind]
    # update color in bdata
    ind = list(bdata.obs['HistologicalAnnotation'].cat.categories).index(c)
    bdata.uns['HistologicalAnnotation_colors'][ind] = color

sc.pl.spatial(bdata, 
                  color=[
                      'HistologicalAnnotation'
                        ], 
                  img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer",
                  size=1.5,alpha_img=0.3,
                  frameon=False, show=False
                 )
plt.savefig(f'{figdir}/VisiumHD_euploid_Atrium_HistologicalAnnotation.pdf', bbox_inches="tight",dpi=300)
```

Example 23 (unknown):
```unknown
maxp pruned
cmap pruned
kern dropped
post pruned
FFTM dropped
GPOS pruned
GSUB pruned
glyf pruned
Added gid0 to subset
Added first four glyphs to subset
Closing glyph list over 'MATH': 24 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'G', 'H', 'N', 'V', 'a', 'c', 'e', 'g', 'hyphen', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'r', 's', 'space', 't', 'u', 'v']
Glyph IDs:   [0, 1, 2, 3, 16, 36, 42, 43, 49, 57, 68, 70, 72, 74, 76, 79, 80, 81, 82, 85, 86, 87, 88, 89]
Closed glyph list over 'MATH': 24 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'G', 'H', 'N', 'V', 'a', 'c', 'e', 'g', 'hyphen', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'r', 's', 'space', 't', 'u', 'v']
Glyph IDs:   [0, 1, 2, 3, 16, 36, 42, 43, 49, 57, 68, 70, 72, 74, 76, 79, 80, 81, 82, 85, 86, 87, 88, 89]
Closing glyph list over 'GSUB': 24 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'G', 'H', 'N', 'V', 'a', 'c', 'e', 'g', 'hyphen', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'r', 's', 'space', 't', 'u', 'v']
Glyph IDs:   [0, 1, 2, 3, 16, 36, 42, 43, 49, 57, 68, 70, 72, 74, 76, 79, 80, 81, 82, 85, 86, 87, 88, 89]
Closed glyph list over 'GSUB': 24 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'G', 'H', 'N', 'V', 'a', 'c', 'e', 'g', 'hyphen', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'r', 's', 'space', 't', 'u', 'v']
Glyph IDs:   [0, 1, 2, 3, 16, 36, 42, 43, 49, 57, 68, 70, 72, 74, 76, 79, 80, 81, 82, 85, 86, 87, 88, 89]
Closing glyph list over 'glyf': 24 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'G', 'H', 'N', 'V', 'a', 'c', 'e', 'g', 'hyphen', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'r', 's', 'space', 't', 'u', 'v']
Glyph IDs:   [0, 1, 2, 3, 16, 36, 42, 43, 49, 57, 68, 70, 72, 74, 76, 79, 80, 81, 82, 85, 86, 87, 88, 89]
Closed glyph list over 'glyf': 24 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'G', 'H', 'N', 'V', 'a', 'c', 'e', 'g', 'hyphen', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'r', 's', 'space', 't', 'u', 'v']
Glyph IDs:   [0, 1, 2, 3, 16, 36, 42, 43, 49, 57, 68, 70, 72, 74, 76, 79, 80, 81, 82, 85, 86, 87, 88, 89]
Retaining 24 glyphs
head subsetting not needed
hhea subsetting not needed
maxp subsetting not needed
OS/2 subsetting not needed
hmtx subsetted
cmap subsetted
fpgm subsetting not needed
prep subsetting not needed
cvt  subsetting not needed
loca subsetting not needed
post subsetted
gasp subsetting not needed
MATH subsetted
GDEF subsetted
GPOS subsetted
GSUB subsetted
name subsetting not needed
glyf subsetted
head pruned
OS/2 Unicode ranges pruned: [0]
OS/2 CodePage ranges pruned: [0]
glyf pruned
GDEF pruned
GPOS pruned
GSUB pruned
name pruned
```

Example 24 (python):
```python
sc.pl.spatial(bdata, 
                  color=[
                      group_col
                        ], 
                  img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer",
                  size=1.5,alpha_img=0.3,palette=AnnotationPalette,
                  frameon=False, show=False
                 )
plt.savefig(f'{figdir}/VisiumHD_euploid_Atrium_TissueTypist_window_predicted-labels.pdf', bbox_inches="tight",dpi=300)
```

Example 25 (unknown):
```unknown
maxp pruned
cmap pruned
kern dropped
post pruned
FFTM dropped
GPOS pruned
GSUB pruned
glyf pruned
Added gid0 to subset
Added first four glyphs to subset
Closing glyph list over 'MATH': 34 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'C', 'D', 'E', 'G', 'N', 'P', 'S', 'T', 'V', 'a', 'b', 'c', 'd', 'e', 'hyphen', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'p', 'r', 's', 'space', 't', 'u', 'underscore', 'v', 'x', 'y']
Glyph IDs:   [0, 1, 2, 3, 16, 36, 38, 39, 40, 42, 49, 51, 54, 55, 57, 66, 68, 69, 70, 71, 72, 76, 79, 80, 81, 82, 83, 85, 86, 87, 88, 89, 91, 92]
Closed glyph list over 'MATH': 34 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'C', 'D', 'E', 'G', 'N', 'P', 'S', 'T', 'V', 'a', 'b', 'c', 'd', 'e', 'hyphen', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'p', 'r', 's', 'space', 't', 'u', 'underscore', 'v', 'x', 'y']
Glyph IDs:   [0, 1, 2, 3, 16, 36, 38, 39, 40, 42, 49, 51, 54, 55, 57, 66, 68, 69, 70, 71, 72, 76, 79, 80, 81, 82, 83, 85, 86, 87, 88, 89, 91, 92]
Closing glyph list over 'GSUB': 34 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'C', 'D', 'E', 'G', 'N', 'P', 'S', 'T', 'V', 'a', 'b', 'c', 'd', 'e', 'hyphen', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'p', 'r', 's', 'space', 't', 'u', 'underscore', 'v', 'x', 'y']
Glyph IDs:   [0, 1, 2, 3, 16, 36, 38, 39, 40, 42, 49, 51, 54, 55, 57, 66, 68, 69, 70, 71, 72, 76, 79, 80, 81, 82, 83, 85, 86, 87, 88, 89, 91, 92]
Closed glyph list over 'GSUB': 34 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'C', 'D', 'E', 'G', 'N', 'P', 'S', 'T', 'V', 'a', 'b', 'c', 'd', 'e', 'hyphen', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'p', 'r', 's', 'space', 't', 'u', 'underscore', 'v', 'x', 'y']
Glyph IDs:   [0, 1, 2, 3, 16, 36, 38, 39, 40, 42, 49, 51, 54, 55, 57, 66, 68, 69, 70, 71, 72, 76, 79, 80, 81, 82, 83, 85, 86, 87, 88, 89, 91, 92]
Closing glyph list over 'glyf': 34 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'C', 'D', 'E', 'G', 'N', 'P', 'S', 'T', 'V', 'a', 'b', 'c', 'd', 'e', 'hyphen', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'p', 'r', 's', 'space', 't', 'u', 'underscore', 'v', 'x', 'y']
Glyph IDs:   [0, 1, 2, 3, 16, 36, 38, 39, 40, 42, 49, 51, 54, 55, 57, 66, 68, 69, 70, 71, 72, 76, 79, 80, 81, 82, 83, 85, 86, 87, 88, 89, 91, 92]
Closed glyph list over 'glyf': 34 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'C', 'D', 'E', 'G', 'N', 'P', 'S', 'T', 'V', 'a', 'b', 'c', 'd', 'e', 'hyphen', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'p', 'r', 's', 'space', 't', 'u', 'underscore', 'v', 'x', 'y']
Glyph IDs:   [0, 1, 2, 3, 16, 36, 38, 39, 40, 42, 49, 51, 54, 55, 57, 66, 68, 69, 70, 71, 72, 76, 79, 80, 81, 82, 83, 85, 86, 87, 88, 89, 91, 92]
Retaining 34 glyphs
head subsetting not needed
hhea subsetting not needed
maxp subsetting not needed
OS/2 subsetting not needed
hmtx subsetted
cmap subsetted
fpgm subsetting not needed
prep subsetting not needed
cvt  subsetting not needed
loca subsetting not needed
post subsetted
gasp subsetting not needed
MATH subsetted
GDEF subsetted
GPOS subsetted
GSUB subsetted
name subsetting not needed
glyf subsetted
head pruned
OS/2 Unicode ranges pruned: [0]
OS/2 CodePage ranges pruned: [0]
glyf pruned
GDEF pruned
GPOS pruned
GSUB pruned
name pruned
```

Example 26 (python):
```python
mask = ((adata.obs['array_row'] >= 1300) & 
        (adata.obs['array_row'] <= 2100) & 
        (adata.obs['array_col'] >= 2400) & 
        (adata.obs['array_col'] <= 3400)
       )
bdata = adata[mask]

sc.pl.spatial(bdata, 
              color=[
                      'HistologicalAnnotation'
                        ], 
                  img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer",
                  size=0,alpha_img=1,legend_loc=None,
                  frameon=False, show=False
                 )
plt.savefig(f'{figdir}/VisiumHD_euploid_Ventricle_HE.pdf', bbox_inches="tight",dpi=300)
```

Example 27 (unknown):
```unknown
maxp pruned
cmap pruned
kern dropped
post pruned
FFTM dropped
GPOS pruned
GSUB pruned
glyf pruned
Added gid0 to subset
Added first four glyphs to subset
Closing glyph list over 'MATH': 15 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'H', 'a', 'c', 'g', 'i', 'l', 'n', 'nonmarkingreturn', 'o', 's', 'space', 't']
Glyph IDs:   [0, 1, 2, 3, 36, 43, 68, 70, 74, 76, 79, 81, 82, 86, 87]
Closed glyph list over 'MATH': 15 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'H', 'a', 'c', 'g', 'i', 'l', 'n', 'nonmarkingreturn', 'o', 's', 'space', 't']
Glyph IDs:   [0, 1, 2, 3, 36, 43, 68, 70, 74, 76, 79, 81, 82, 86, 87]
Closing glyph list over 'GSUB': 15 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'H', 'a', 'c', 'g', 'i', 'l', 'n', 'nonmarkingreturn', 'o', 's', 'space', 't']
Glyph IDs:   [0, 1, 2, 3, 36, 43, 68, 70, 74, 76, 79, 81, 82, 86, 87]
Closed glyph list over 'GSUB': 15 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'H', 'a', 'c', 'g', 'i', 'l', 'n', 'nonmarkingreturn', 'o', 's', 'space', 't']
Glyph IDs:   [0, 1, 2, 3, 36, 43, 68, 70, 74, 76, 79, 81, 82, 86, 87]
Closing glyph list over 'glyf': 15 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'H', 'a', 'c', 'g', 'i', 'l', 'n', 'nonmarkingreturn', 'o', 's', 'space', 't']
Glyph IDs:   [0, 1, 2, 3, 36, 43, 68, 70, 74, 76, 79, 81, 82, 86, 87]
Closed glyph list over 'glyf': 15 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'H', 'a', 'c', 'g', 'i', 'l', 'n', 'nonmarkingreturn', 'o', 's', 'space', 't']
Glyph IDs:   [0, 1, 2, 3, 36, 43, 68, 70, 74, 76, 79, 81, 82, 86, 87]
Retaining 15 glyphs
head subsetting not needed
hhea subsetting not needed
maxp subsetting not needed
OS/2 subsetting not needed
hmtx subsetted
cmap subsetted
fpgm subsetting not needed
prep subsetting not needed
cvt  subsetting not needed
loca subsetting not needed
post subsetted
gasp subsetting not needed
MATH subsetted
GDEF subsetted
GPOS subsetted
GSUB subsetted
name subsetting not needed
glyf subsetted
head pruned
OS/2 Unicode ranges pruned: [0]
OS/2 CodePage ranges pruned: [0]
glyf pruned
GDEF pruned
GPOS pruned
GSUB pruned
name pruned
```

Example 28 (python):
```python
n_annotation = len(bdata.obs['HistologicalAnnotation'].cat.categories)
bdata.uns['HistologicalAnnotation'] = list(np.repeat('#ffff00',n_annotation))

# update colors
for c in bdata.obs['HistologicalAnnotation'].cat.categories:
    # get color in adata
    ind = list(adata.obs['HistologicalAnnotation'].cat.categories).index(c)
    color = adata.uns['HistologicalAnnotation_colors'][ind]
    # update color in bdata
    ind = list(bdata.obs['HistologicalAnnotation'].cat.categories).index(c)
    bdata.uns['HistologicalAnnotation_colors'][ind] = color

sc.pl.spatial(bdata, 
                  color=[
                      'HistologicalAnnotation'
                        ], 
                  img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer",
                  size=1.5,alpha_img=0.3,
                  frameon=False, show=False
                 )
plt.savefig(f'{figdir}/VisiumHD_euploid_Ventricle_HistologicalAnnotation.pdf', bbox_inches="tight",dpi=300)
```

Example 29 (unknown):
```unknown
maxp pruned
cmap pruned
kern dropped
post pruned
FFTM dropped
GPOS pruned
GSUB pruned
glyf pruned
Added gid0 to subset
Added first four glyphs to subset
Closing glyph list over 'MATH': 18 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'H', 'V', 'a', 'c', 'e', 'g', 'i', 'l', 'n', 'nonmarkingreturn', 'o', 'r', 's', 'space', 't']
Glyph IDs:   [0, 1, 2, 3, 36, 43, 57, 68, 70, 72, 74, 76, 79, 81, 82, 85, 86, 87]
Closed glyph list over 'MATH': 18 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'H', 'V', 'a', 'c', 'e', 'g', 'i', 'l', 'n', 'nonmarkingreturn', 'o', 'r', 's', 'space', 't']
Glyph IDs:   [0, 1, 2, 3, 36, 43, 57, 68, 70, 72, 74, 76, 79, 81, 82, 85, 86, 87]
Closing glyph list over 'GSUB': 18 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'H', 'V', 'a', 'c', 'e', 'g', 'i', 'l', 'n', 'nonmarkingreturn', 'o', 'r', 's', 'space', 't']
Glyph IDs:   [0, 1, 2, 3, 36, 43, 57, 68, 70, 72, 74, 76, 79, 81, 82, 85, 86, 87]
Closed glyph list over 'GSUB': 18 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'H', 'V', 'a', 'c', 'e', 'g', 'i', 'l', 'n', 'nonmarkingreturn', 'o', 'r', 's', 'space', 't']
Glyph IDs:   [0, 1, 2, 3, 36, 43, 57, 68, 70, 72, 74, 76, 79, 81, 82, 85, 86, 87]
Closing glyph list over 'glyf': 18 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'H', 'V', 'a', 'c', 'e', 'g', 'i', 'l', 'n', 'nonmarkingreturn', 'o', 'r', 's', 'space', 't']
Glyph IDs:   [0, 1, 2, 3, 36, 43, 57, 68, 70, 72, 74, 76, 79, 81, 82, 85, 86, 87]
Closed glyph list over 'glyf': 18 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'H', 'V', 'a', 'c', 'e', 'g', 'i', 'l', 'n', 'nonmarkingreturn', 'o', 'r', 's', 'space', 't']
Glyph IDs:   [0, 1, 2, 3, 36, 43, 57, 68, 70, 72, 74, 76, 79, 81, 82, 85, 86, 87]
Retaining 18 glyphs
head subsetting not needed
hhea subsetting not needed
maxp subsetting not needed
OS/2 subsetting not needed
hmtx subsetted
cmap subsetted
fpgm subsetting not needed
prep subsetting not needed
cvt  subsetting not needed
loca subsetting not needed
post subsetted
gasp subsetting not needed
MATH subsetted
GDEF subsetted
GPOS subsetted
GSUB subsetted
name subsetting not needed
glyf subsetted
head pruned
OS/2 Unicode ranges pruned: [0]
OS/2 CodePage ranges pruned: [0]
glyf pruned
GDEF pruned
GPOS pruned
GSUB pruned
name pruned
```

Example 30 (python):
```python
sc.pl.spatial(bdata, 
                  color=[
                      group_col
                        ], 
                  img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer",
                  size=1.5,alpha_img=0.3,palette=AnnotationPalette,
                  frameon=False, show=False
                 )
plt.savefig(f'{figdir}/VisiumHD_euploid_Ventricle_TissueTypist_window_predicted-labels.pdf', bbox_inches="tight",dpi=300)
```

Example 31 (unknown):
```unknown
maxp pruned
cmap pruned
kern dropped
post pruned
FFTM dropped
GPOS pruned
GSUB pruned
glyf pruned
Added gid0 to subset
Added first four glyphs to subset
Closing glyph list over 'MATH': 27 glyphs before
Glyph names: ['.notdef', '.null', 'C', 'D', 'E', 'S', 'T', 'V', 'a', 'b', 'c', 'd', 'e', 'hyphen', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'p', 'r', 's', 'space', 't', 'u', 'underscore']
Glyph IDs:   [0, 1, 2, 3, 16, 38, 39, 40, 54, 55, 57, 66, 68, 69, 70, 71, 72, 76, 79, 80, 81, 82, 83, 85, 86, 87, 88]
Closed glyph list over 'MATH': 27 glyphs after
Glyph names: ['.notdef', '.null', 'C', 'D', 'E', 'S', 'T', 'V', 'a', 'b', 'c', 'd', 'e', 'hyphen', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'p', 'r', 's', 'space', 't', 'u', 'underscore']
Glyph IDs:   [0, 1, 2, 3, 16, 38, 39, 40, 54, 55, 57, 66, 68, 69, 70, 71, 72, 76, 79, 80, 81, 82, 83, 85, 86, 87, 88]
Closing glyph list over 'GSUB': 27 glyphs before
Glyph names: ['.notdef', '.null', 'C', 'D', 'E', 'S', 'T', 'V', 'a', 'b', 'c', 'd', 'e', 'hyphen', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'p', 'r', 's', 'space', 't', 'u', 'underscore']
Glyph IDs:   [0, 1, 2, 3, 16, 38, 39, 40, 54, 55, 57, 66, 68, 69, 70, 71, 72, 76, 79, 80, 81, 82, 83, 85, 86, 87, 88]
Closed glyph list over 'GSUB': 27 glyphs after
Glyph names: ['.notdef', '.null', 'C', 'D', 'E', 'S', 'T', 'V', 'a', 'b', 'c', 'd', 'e', 'hyphen', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'p', 'r', 's', 'space', 't', 'u', 'underscore']
Glyph IDs:   [0, 1, 2, 3, 16, 38, 39, 40, 54, 55, 57, 66, 68, 69, 70, 71, 72, 76, 79, 80, 81, 82, 83, 85, 86, 87, 88]
Closing glyph list over 'glyf': 27 glyphs before
Glyph names: ['.notdef', '.null', 'C', 'D', 'E', 'S', 'T', 'V', 'a', 'b', 'c', 'd', 'e', 'hyphen', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'p', 'r', 's', 'space', 't', 'u', 'underscore']
Glyph IDs:   [0, 1, 2, 3, 16, 38, 39, 40, 54, 55, 57, 66, 68, 69, 70, 71, 72, 76, 79, 80, 81, 82, 83, 85, 86, 87, 88]
Closed glyph list over 'glyf': 27 glyphs after
Glyph names: ['.notdef', '.null', 'C', 'D', 'E', 'S', 'T', 'V', 'a', 'b', 'c', 'd', 'e', 'hyphen', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'p', 'r', 's', 'space', 't', 'u', 'underscore']
Glyph IDs:   [0, 1, 2, 3, 16, 38, 39, 40, 54, 55, 57, 66, 68, 69, 70, 71, 72, 76, 79, 80, 81, 82, 83, 85, 86, 87, 88]
Retaining 27 glyphs
head subsetting not needed
hhea subsetting not needed
maxp subsetting not needed
OS/2 subsetting not needed
hmtx subsetted
cmap subsetted
fpgm subsetting not needed
prep subsetting not needed
cvt  subsetting not needed
loca subsetting not needed
post subsetted
gasp subsetting not needed
MATH subsetted
GDEF subsetted
GPOS subsetted
GSUB subsetted
name subsetting not needed
glyf subsetted
head pruned
OS/2 Unicode ranges pruned: [0]
OS/2 CodePage ranges pruned: [0]
glyf pruned
GDEF pruned
GPOS pruned
GSUB pruned
name pruned
```

Example 32 (python):
```python
mask = ((adata.obs['array_row'] >= 2600) & 
        (adata.obs['array_row'] <= 3000) & 
        (adata.obs['array_col'] >= 1200) & 
        (adata.obs['array_col'] <= 1650)
       )
bdata = adata[mask]

sc.pl.spatial(bdata, 
              color=[
                      'HistologicalAnnotation'
                        ], 
                  img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer",
                  size=0,alpha_img=1,legend_loc=None,
                  frameon=False, show=False
                 )
plt.savefig(f'{figdir}/VisiumHD_euploid_GreatVessel_HE.pdf', bbox_inches="tight",dpi=300)
```

Example 33 (unknown):
```unknown
maxp pruned
cmap pruned
kern dropped
post pruned
FFTM dropped
GPOS pruned
GSUB pruned
glyf pruned
Added gid0 to subset
Added first four glyphs to subset
Closing glyph list over 'MATH': 15 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'H', 'a', 'c', 'g', 'i', 'l', 'n', 'nonmarkingreturn', 'o', 's', 'space', 't']
Glyph IDs:   [0, 1, 2, 3, 36, 43, 68, 70, 74, 76, 79, 81, 82, 86, 87]
Closed glyph list over 'MATH': 15 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'H', 'a', 'c', 'g', 'i', 'l', 'n', 'nonmarkingreturn', 'o', 's', 'space', 't']
Glyph IDs:   [0, 1, 2, 3, 36, 43, 68, 70, 74, 76, 79, 81, 82, 86, 87]
Closing glyph list over 'GSUB': 15 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'H', 'a', 'c', 'g', 'i', 'l', 'n', 'nonmarkingreturn', 'o', 's', 'space', 't']
Glyph IDs:   [0, 1, 2, 3, 36, 43, 68, 70, 74, 76, 79, 81, 82, 86, 87]
Closed glyph list over 'GSUB': 15 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'H', 'a', 'c', 'g', 'i', 'l', 'n', 'nonmarkingreturn', 'o', 's', 'space', 't']
Glyph IDs:   [0, 1, 2, 3, 36, 43, 68, 70, 74, 76, 79, 81, 82, 86, 87]
Closing glyph list over 'glyf': 15 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'H', 'a', 'c', 'g', 'i', 'l', 'n', 'nonmarkingreturn', 'o', 's', 'space', 't']
Glyph IDs:   [0, 1, 2, 3, 36, 43, 68, 70, 74, 76, 79, 81, 82, 86, 87]
Closed glyph list over 'glyf': 15 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'H', 'a', 'c', 'g', 'i', 'l', 'n', 'nonmarkingreturn', 'o', 's', 'space', 't']
Glyph IDs:   [0, 1, 2, 3, 36, 43, 68, 70, 74, 76, 79, 81, 82, 86, 87]
Retaining 15 glyphs
head subsetting not needed
hhea subsetting not needed
maxp subsetting not needed
OS/2 subsetting not needed
hmtx subsetted
cmap subsetted
fpgm subsetting not needed
prep subsetting not needed
cvt  subsetting not needed
loca subsetting not needed
post subsetted
gasp subsetting not needed
MATH subsetted
GDEF subsetted
GPOS subsetted
GSUB subsetted
name subsetting not needed
glyf subsetted
head pruned
OS/2 Unicode ranges pruned: [0]
OS/2 CodePage ranges pruned: [0]
glyf pruned
GDEF pruned
GPOS pruned
GSUB pruned
name pruned
```

Example 34 (python):
```python
n_annotation = len(bdata.obs['HistologicalAnnotation'].cat.categories)
bdata.uns['HistologicalAnnotation'] = list(np.repeat('#ffff00',n_annotation))

# update colors
for c in bdata.obs['HistologicalAnnotation'].cat.categories:
    # get color in adata
    ind = list(adata.obs['HistologicalAnnotation'].cat.categories).index(c)
    color = adata.uns['HistologicalAnnotation_colors'][ind]
    # update color in bdata
    ind = list(bdata.obs['HistologicalAnnotation'].cat.categories).index(c)
    bdata.uns['HistologicalAnnotation_colors'][ind] = color

sc.pl.spatial(bdata, 
                  color=[
                      'HistologicalAnnotation'
                        ], 
                  img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer",
                  size=1.5,alpha_img=0.3,
                  frameon=False, show=False
                 )
plt.savefig(f'{figdir}/VisiumHD_euploid_GreatVessel_HistologicalAnnotation.pdf', bbox_inches="tight",dpi=300)
```

Example 35 (unknown):
```unknown
maxp pruned
cmap pruned
kern dropped
post pruned
FFTM dropped
GPOS pruned
GSUB pruned
glyf pruned
Added gid0 to subset
Added first four glyphs to subset
Closing glyph list over 'MATH': 29 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'D', 'G', 'H', 'L', 'N', 'P', 'a', 'c', 'd', 'e', 'f', 'g', 'hyphen', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'r', 's', 'space', 't', 'u', 'v', 'y']
Glyph IDs:   [0, 1, 2, 3, 16, 36, 39, 42, 43, 47, 49, 51, 68, 70, 71, 72, 73, 74, 76, 79, 80, 81, 82, 85, 86, 87, 88, 89, 92]
Closed glyph list over 'MATH': 29 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'D', 'G', 'H', 'L', 'N', 'P', 'a', 'c', 'd', 'e', 'f', 'g', 'hyphen', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'r', 's', 'space', 't', 'u', 'v', 'y']
Glyph IDs:   [0, 1, 2, 3, 16, 36, 39, 42, 43, 47, 49, 51, 68, 70, 71, 72, 73, 74, 76, 79, 80, 81, 82, 85, 86, 87, 88, 89, 92]
Closing glyph list over 'GSUB': 29 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'D', 'G', 'H', 'L', 'N', 'P', 'a', 'c', 'd', 'e', 'f', 'g', 'hyphen', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'r', 's', 'space', 't', 'u', 'v', 'y']
Glyph IDs:   [0, 1, 2, 3, 16, 36, 39, 42, 43, 47, 49, 51, 68, 70, 71, 72, 73, 74, 76, 79, 80, 81, 82, 85, 86, 87, 88, 89, 92]
Closed glyph list over 'GSUB': 34 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'D', 'G', 'H', 'L', 'N', 'P', 'a', 'c', 'd', 'e', 'f', 'fi', 'fl', 'g', 'hyphen', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'r', 's', 'space', 't', 'u', 'uniFB00', 'uniFB03', 'uniFB04', 'v', 'y']
Glyph IDs:   [0, 1, 2, 3, 16, 36, 39, 42, 43, 47, 49, 51, 68, 70, 71, 72, 73, 74, 76, 79, 80, 81, 82, 85, 86, 87, 88, 89, 92, 5038, 5039, 5040, 5041, 5042]
Closing glyph list over 'glyf': 34 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'D', 'G', 'H', 'L', 'N', 'P', 'a', 'c', 'd', 'e', 'f', 'fi', 'fl', 'g', 'hyphen', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'r', 's', 'space', 't', 'u', 'uniFB00', 'uniFB03', 'uniFB04', 'v', 'y']
Glyph IDs:   [0, 1, 2, 3, 16, 36, 39, 42, 43, 47, 49, 51, 68, 70, 71, 72, 73, 74, 76, 79, 80, 81, 82, 85, 86, 87, 88, 89, 92, 5038, 5039, 5040, 5041, 5042]
Closed glyph list over 'glyf': 34 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'D', 'G', 'H', 'L', 'N', 'P', 'a', 'c', 'd', 'e', 'f', 'fi', 'fl', 'g', 'hyphen', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'r', 's', 'space', 't', 'u', 'uniFB00', 'uniFB03', 'uniFB04', 'v', 'y']
Glyph IDs:   [0, 1, 2, 3, 16, 36, 39, 42, 43, 47, 49, 51, 68, 70, 71, 72, 73, 74, 76, 79, 80, 81, 82, 85, 86, 87, 88, 89, 92, 5038, 5039, 5040, 5041, 5042]
Retaining 34 glyphs
head subsetting not needed
hhea subsetting not needed
maxp subsetting not needed
OS/2 subsetting not needed
hmtx subsetted
cmap subsetted
fpgm subsetting not needed
prep subsetting not needed
cvt  subsetting not needed
loca subsetting not needed
post subsetted
gasp subsetting not needed
MATH subsetted
GDEF subsetted
GPOS subsetted
GSUB subsetted
name subsetting not needed
glyf subsetted
head pruned
OS/2 Unicode ranges pruned: [0]
OS/2 CodePage ranges pruned: [0]
glyf pruned
GDEF pruned
GPOS pruned
GSUB pruned
name pruned
```

Example 36 (python):
```python
sc.pl.spatial(bdata, 
                  color=[
                      group_col
                        ], 
                  img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer",
                  size=1.5,alpha_img=0.3,palette=AnnotationPalette,
                  frameon=False, show=False
                 )
plt.savefig(f'{figdir}/VisiumHD_euploid_GreatVessel_TissueTypist_window_predicted-labels.pdf', bbox_inches="tight",dpi=300)
```

Example 37 (unknown):
```unknown
maxp pruned
cmap pruned
kern dropped
post pruned
FFTM dropped
GPOS pruned
GSUB pruned
glyf pruned
Added gid0 to subset
Added first four glyphs to subset
Closing glyph list over 'MATH': 26 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'D', 'E', 'G', 'V', 'a', 'c', 'd', 'e', 'hyphen', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'p', 'r', 's', 'space', 't', 'u', 'underscore', 'v']
Glyph IDs:   [0, 1, 2, 3, 16, 36, 39, 40, 42, 57, 66, 68, 70, 71, 72, 76, 79, 80, 81, 82, 83, 85, 86, 87, 88, 89]
Closed glyph list over 'MATH': 26 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'D', 'E', 'G', 'V', 'a', 'c', 'd', 'e', 'hyphen', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'p', 'r', 's', 'space', 't', 'u', 'underscore', 'v']
Glyph IDs:   [0, 1, 2, 3, 16, 36, 39, 40, 42, 57, 66, 68, 70, 71, 72, 76, 79, 80, 81, 82, 83, 85, 86, 87, 88, 89]
Closing glyph list over 'GSUB': 26 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'D', 'E', 'G', 'V', 'a', 'c', 'd', 'e', 'hyphen', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'p', 'r', 's', 'space', 't', 'u', 'underscore', 'v']
Glyph IDs:   [0, 1, 2, 3, 16, 36, 39, 40, 42, 57, 66, 68, 70, 71, 72, 76, 79, 80, 81, 82, 83, 85, 86, 87, 88, 89]
Closed glyph list over 'GSUB': 26 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'D', 'E', 'G', 'V', 'a', 'c', 'd', 'e', 'hyphen', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'p', 'r', 's', 'space', 't', 'u', 'underscore', 'v']
Glyph IDs:   [0, 1, 2, 3, 16, 36, 39, 40, 42, 57, 66, 68, 70, 71, 72, 76, 79, 80, 81, 82, 83, 85, 86, 87, 88, 89]
Closing glyph list over 'glyf': 26 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'D', 'E', 'G', 'V', 'a', 'c', 'd', 'e', 'hyphen', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'p', 'r', 's', 'space', 't', 'u', 'underscore', 'v']
Glyph IDs:   [0, 1, 2, 3, 16, 36, 39, 40, 42, 57, 66, 68, 70, 71, 72, 76, 79, 80, 81, 82, 83, 85, 86, 87, 88, 89]
Closed glyph list over 'glyf': 26 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'D', 'E', 'G', 'V', 'a', 'c', 'd', 'e', 'hyphen', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'p', 'r', 's', 'space', 't', 'u', 'underscore', 'v']
Glyph IDs:   [0, 1, 2, 3, 16, 36, 39, 40, 42, 57, 66, 68, 70, 71, 72, 76, 79, 80, 81, 82, 83, 85, 86, 87, 88, 89]
Retaining 26 glyphs
head subsetting not needed
hhea subsetting not needed
maxp subsetting not needed
OS/2 subsetting not needed
hmtx subsetted
cmap subsetted
fpgm subsetting not needed
prep subsetting not needed
cvt  subsetting not needed
loca subsetting not needed
post subsetted
gasp subsetting not needed
MATH subsetted
GDEF subsetted
GPOS subsetted
GSUB subsetted
name subsetting not needed
glyf subsetted
head pruned
OS/2 Unicode ranges pruned: [0]
OS/2 CodePage ranges pruned: [0]
glyf pruned
GDEF pruned
GPOS pruned
GSUB pruned
name pruned
```

Example 38 (python):
```python
# celltype name mapping for plotting
df = pd.read_csv('/nfs/team205/heart/anndata_objects/Foetal/finegrain_name_mapping.csv')
celltype_mapping = df.set_index('Full_name')['Short_name_finalised'].to_dict()
adata.obs.replace({'fine_grain_4Jan2025':celltype_mapping},inplace=True)
```

Example 39 (python):
```python
celltype_key = 'fine_grain_4Jan2025'
set(adata.obs[celltype_key])
```

Example 40 (unknown):
```unknown
{'AVNPC',
 'B',
 'ChromC',
 'CorArtEC',
 'CorCapEC',
 'CorPeri',
 'CorSMC',
 'CorVenEC',
 'DASMC',
 'EPDC',
 'EndoCush',
 'Endocard',
 'Eryth',
 'FB',
 'FBCOL2A1',
 'GVArtEC',
 'GVSMC',
 'GVSMCP',
 'GVSMCT',
 'GVVenEC',
 'ILC',
 'LymphEC',
 'MPLYVE1',
 'MacCX3',
 'MegaK',
 'MesEpiC',
 'MoDC',
 'Mono',
 'NeuroPre',
 'ParaN',
 'PericFib',
 'PericInt',
 'SANPCHd',
 'SANPCHn',
 'SANPCT',
 'SchwC',
 'SchwNeuro',
 'SchwP',
 'SympN',
 'VCSDist',
 'VCSProx',
 'ValveEC',
 'ValveIC',
 'aCML',
 'aCMR',
 'vCMC',
 'vCMT'}
```

Example 41 (python):
```python
# HE
mask = ((adata.obs['array_row'] >= 1700) & 
        (adata.obs['array_row'] <= 2300) & 
        (adata.obs['array_col'] >= 600) & 
        (adata.obs['array_col'] <= 1100)
       )
bdata = adata[mask]

sc.pl.spatial(bdata, 
              color=['fine_grain_4Jan2025'],
                  img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer",
                  size=0,alpha_img=1,
                  legend_loc=None, frameon=False, show=False
                 )
plt.savefig(f'{figdir}/VisiumHD_euploid_SAN_for-Fig1_HE.pdf',bbox_inches="tight",dpi=300)
```

Example 42 (unknown):
```unknown
maxp pruned
cmap pruned
kern dropped
post pruned
FFTM dropped
GPOS pruned
GSUB pruned
glyf pruned
Added gid0 to subset
Added first four glyphs to subset
Closing glyph list over 'MATH': 17 glyphs before
Glyph names: ['.notdef', '.null', 'J', 'a', 'e', 'f', 'five', 'four', 'g', 'i', 'n', 'nonmarkingreturn', 'r', 'space', 'two', 'underscore', 'zero']
Glyph IDs:   [0, 1, 2, 3, 19, 21, 23, 24, 45, 66, 68, 72, 73, 74, 76, 81, 85]
Closed glyph list over 'MATH': 17 glyphs after
Glyph names: ['.notdef', '.null', 'J', 'a', 'e', 'f', 'five', 'four', 'g', 'i', 'n', 'nonmarkingreturn', 'r', 'space', 'two', 'underscore', 'zero']
Glyph IDs:   [0, 1, 2, 3, 19, 21, 23, 24, 45, 66, 68, 72, 73, 74, 76, 81, 85]
Closing glyph list over 'GSUB': 17 glyphs before
Glyph names: ['.notdef', '.null', 'J', 'a', 'e', 'f', 'five', 'four', 'g', 'i', 'n', 'nonmarkingreturn', 'r', 'space', 'two', 'underscore', 'zero']
Glyph IDs:   [0, 1, 2, 3, 19, 21, 23, 24, 45, 66, 68, 72, 73, 74, 76, 81, 85]
Closed glyph list over 'GSUB': 20 glyphs after
Glyph names: ['.notdef', '.null', 'J', 'a', 'e', 'f', 'fi', 'five', 'four', 'g', 'i', 'n', 'nonmarkingreturn', 'r', 'space', 'two', 'underscore', 'uniFB00', 'uniFB03', 'zero']
Glyph IDs:   [0, 1, 2, 3, 19, 21, 23, 24, 45, 66, 68, 72, 73, 74, 76, 81, 85, 5038, 5039, 5041]
Closing glyph list over 'glyf': 20 glyphs before
Glyph names: ['.notdef', '.null', 'J', 'a', 'e', 'f', 'fi', 'five', 'four', 'g', 'i', 'n', 'nonmarkingreturn', 'r', 'space', 'two', 'underscore', 'uniFB00', 'uniFB03', 'zero']
Glyph IDs:   [0, 1, 2, 3, 19, 21, 23, 24, 45, 66, 68, 72, 73, 74, 76, 81, 85, 5038, 5039, 5041]
Closed glyph list over 'glyf': 20 glyphs after
Glyph names: ['.notdef', '.null', 'J', 'a', 'e', 'f', 'fi', 'five', 'four', 'g', 'i', 'n', 'nonmarkingreturn', 'r', 'space', 'two', 'underscore', 'uniFB00', 'uniFB03', 'zero']
Glyph IDs:   [0, 1, 2, 3, 19, 21, 23, 24, 45, 66, 68, 72, 73, 74, 76, 81, 85, 5038, 5039, 5041]
Retaining 20 glyphs
head subsetting not needed
hhea subsetting not needed
maxp subsetting not needed
OS/2 subsetting not needed
hmtx subsetted
cmap subsetted
fpgm subsetting not needed
prep subsetting not needed
cvt  subsetting not needed
loca subsetting not needed
post subsetted
gasp subsetting not needed
MATH subsetted
GDEF subsetted
GPOS subsetted
GSUB subsetted
name subsetting not needed
glyf subsetted
head pruned
OS/2 Unicode ranges pruned: [0]
OS/2 CodePage ranges pruned: [0]
glyf pruned
GDEF pruned
GPOS pruned
GSUB pruned
name pruned
```

Example 43 (python):
```python
mask = ((adata.obs['array_row'] >= 1700) & 
        (adata.obs['array_row'] <= 2300) & 
        (adata.obs['array_col'] >= 600) & 
        (adata.obs['array_col'] <= 1100)
       )
bdata = adata[mask]

bdata.obs[celltype_key] = bdata.obs[celltype_key].astype('str')
bdata.obs.replace({celltype_key:{
    'SANPCHn':'SANPC',
    'SANPCHd':'SANPC',
    'SANPCT':'SANPC'
}},inplace=True)
bdata.obs[celltype_key] = bdata.obs[celltype_key].astype('category')

celltypes_to_plot = ['SANPC',
                     'CorCapEC',
                     'CorPeri',
                     'GVVenEC',
                     'MesEpiC',
                     'FB',
                     'aCMR',
                     'AVNPC',
                     'ParaN',
                     'SchwP',
                    ]

# modify colors
bdata.uns[f"{celltype_key}_colors"][list(bdata.obs[celltype_key].cat.categories).index('GVVenEC')]='red'
bdata.uns[f"{celltype_key}_colors"][list(bdata.obs[celltype_key].cat.categories).index('ParaN')]='blue'
bdata.uns[f"{celltype_key}_colors"][list(bdata.obs[celltype_key].cat.categories).index('SANPC')]=(0.5, 0.0, 0.5, 0.50) # purple, 30% opacity

sc.pl.spatial(bdata, 
              color=['fine_grain_4Jan2025'], groups=celltypes_to_plot,
                  img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer",
                  size=1.5,alpha_img=0.3,
                  frameon=False, show=False
                 )
plt.savefig(f'{figdir}/VisiumHD_euploid_SAN_for-Fig1.pdf',bbox_inches="tight",dpi=300)
```

Example 44 (unknown):
```unknown
maxp pruned
cmap pruned
kern dropped
post pruned
FFTM dropped
GPOS pruned
GSUB pruned
glyf pruned
Added gid0 to subset
Added first four glyphs to subset
Closing glyph list over 'MATH': 32 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'B', 'C', 'E', 'F', 'G', 'J', 'M', 'N', 'P', 'R', 'S', 'V', 'a', 'e', 'f', 'five', 'four', 'g', 'i', 'n', 'nonmarkingreturn', 'o', 'p', 'r', 's', 'space', 'two', 'underscore', 'zero']
Glyph IDs:   [0, 1, 2, 3, 19, 21, 23, 24, 36, 37, 38, 40, 41, 42, 45, 48, 49, 51, 53, 54, 57, 66, 68, 72, 73, 74, 76, 81, 82, 83, 85, 86]
Closed glyph list over 'MATH': 32 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'B', 'C', 'E', 'F', 'G', 'J', 'M', 'N', 'P', 'R', 'S', 'V', 'a', 'e', 'f', 'five', 'four', 'g', 'i', 'n', 'nonmarkingreturn', 'o', 'p', 'r', 's', 'space', 'two', 'underscore', 'zero']
Glyph IDs:   [0, 1, 2, 3, 19, 21, 23, 24, 36, 37, 38, 40, 41, 42, 45, 48, 49, 51, 53, 54, 57, 66, 68, 72, 73, 74, 76, 81, 82, 83, 85, 86]
Closing glyph list over 'GSUB': 32 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'B', 'C', 'E', 'F', 'G', 'J', 'M', 'N', 'P', 'R', 'S', 'V', 'a', 'e', 'f', 'five', 'four', 'g', 'i', 'n', 'nonmarkingreturn', 'o', 'p', 'r', 's', 'space', 'two', 'underscore', 'zero']
Glyph IDs:   [0, 1, 2, 3, 19, 21, 23, 24, 36, 37, 38, 40, 41, 42, 45, 48, 49, 51, 53, 54, 57, 66, 68, 72, 73, 74, 76, 81, 82, 83, 85, 86]
Closed glyph list over 'GSUB': 35 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'B', 'C', 'E', 'F', 'G', 'J', 'M', 'N', 'P', 'R', 'S', 'V', 'a', 'e', 'f', 'fi', 'five', 'four', 'g', 'i', 'n', 'nonmarkingreturn', 'o', 'p', 'r', 's', 'space', 'two', 'underscore', 'uniFB00', 'uniFB03', 'zero']
Glyph IDs:   [0, 1, 2, 3, 19, 21, 23, 24, 36, 37, 38, 40, 41, 42, 45, 48, 49, 51, 53, 54, 57, 66, 68, 72, 73, 74, 76, 81, 82, 83, 85, 86, 5038, 5039, 5041]
Closing glyph list over 'glyf': 35 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'B', 'C', 'E', 'F', 'G', 'J', 'M', 'N', 'P', 'R', 'S', 'V', 'a', 'e', 'f', 'fi', 'five', 'four', 'g', 'i', 'n', 'nonmarkingreturn', 'o', 'p', 'r', 's', 'space', 'two', 'underscore', 'uniFB00', 'uniFB03', 'zero']
Glyph IDs:   [0, 1, 2, 3, 19, 21, 23, 24, 36, 37, 38, 40, 41, 42, 45, 48, 49, 51, 53, 54, 57, 66, 68, 72, 73, 74, 76, 81, 82, 83, 85, 86, 5038, 5039, 5041]
Closed glyph list over 'glyf': 35 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'B', 'C', 'E', 'F', 'G', 'J', 'M', 'N', 'P', 'R', 'S', 'V', 'a', 'e', 'f', 'fi', 'five', 'four', 'g', 'i', 'n', 'nonmarkingreturn', 'o', 'p', 'r', 's', 'space', 'two', 'underscore', 'uniFB00', 'uniFB03', 'zero']
Glyph IDs:   [0, 1, 2, 3, 19, 21, 23, 24, 36, 37, 38, 40, 41, 42, 45, 48, 49, 51, 53, 54, 57, 66, 68, 72, 73, 74, 76, 81, 82, 83, 85, 86, 5038, 5039, 5041]
Retaining 35 glyphs
head subsetting not needed
hhea subsetting not needed
maxp subsetting not needed
OS/2 subsetting not needed
hmtx subsetted
cmap subsetted
fpgm subsetting not needed
prep subsetting not needed
cvt  subsetting not needed
loca subsetting not needed
post subsetted
gasp subsetting not needed
MATH subsetted
GDEF subsetted
GPOS subsetted
GSUB subsetted
name subsetting not needed
glyf subsetted
head pruned
OS/2 Unicode ranges pruned: [0]
OS/2 CodePage ranges pruned: [0]
glyf pruned
GDEF pruned
GPOS pruned
GSUB pruned
name pruned
```

Example 45 (python):
```python
print(bdata.obs[celltype_key].cat.categories)
bdata.uns[f"{celltype_key}_colors"]
```

Example 46 (unknown):
```unknown
Index(['AVNPC', 'CorCapEC', 'CorPeri', 'DASMC', 'EPDC', 'EndoCush', 'Endocard',
       'Eryth', 'FB', 'GVSMC', 'GVSMCT', 'GVVenEC', 'LymphEC', 'MPLYVE1',
       'MacCX3', 'MesEpiC', 'MoDC', 'ParaN', 'PericFib', 'SANPC', 'SchwC',
       'SchwNeuro', 'ValveEC', 'ValveIC', 'aCML', 'aCMR'],
      dtype='object')
```

Example 47 (unknown):
```unknown
['#ffff00',
 '#1ce6ff',
 '#ff34ff',
 '#a30059',
 '#ffdbe5',
 '#63ffac',
 '#b79762',
 '#004d43',
 '#8fb0ff',
 '#997d87',
 '#5a0007',
 'red',
 '#4a3b53',
 '#ff2f80',
 '#ba0900',
 '#6b7900',
 '#00c2a0',
 'blue',
 '#b903aa',
 (0.5, 0.0, 0.5, 0.5),
 '#7b4f4b',
 '#013349',
 '#00846f',
 '#372101',
 '#ffb500',
 '#c2ffed',
 '#cc0744',
 '#c0b9b2']
```

Example 48 (python):
```python
celltypes_plot = {
    'SANPC':"#8000801A", # purple, 15% opacity
    'ParaN':'blue',
    'MacCX3':'red'
}

for celltype,color in celltypes_plot.items():
    # update colors
    bdata.uns[f"{celltype_key}_colors"][list(bdata.obs[celltype_key].cat.categories).index(celltype)]=color

sc.pl.spatial(bdata, 
              color=[celltype_key], 
              groups=list(celltypes_plot.keys()),
                  img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer",
                  size=3,alpha_img=0.3,
                  frameon=False, show=False
                 )
plt.savefig(f'{figdir}/VisiumHD_euploid_SAN_{"-".join(list(celltypes_plot.keys()))}.pdf',bbox_inches="tight",dpi=300)
```

Example 49 (unknown):
```unknown
maxp pruned
cmap pruned
kern dropped
post pruned
FFTM dropped
GPOS pruned
GSUB pruned
glyf pruned
Added gid0 to subset
Added first four glyphs to subset
Closing glyph list over 'MATH': 26 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'C', 'J', 'M', 'N', 'P', 'S', 'X', 'a', 'c', 'e', 'f', 'five', 'four', 'g', 'i', 'n', 'nonmarkingreturn', 'r', 'space', 'three', 'two', 'underscore', 'zero']
Glyph IDs:   [0, 1, 2, 3, 19, 21, 22, 23, 24, 36, 38, 45, 48, 49, 51, 54, 59, 66, 68, 70, 72, 73, 74, 76, 81, 85]
Closed glyph list over 'MATH': 26 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'C', 'J', 'M', 'N', 'P', 'S', 'X', 'a', 'c', 'e', 'f', 'five', 'four', 'g', 'i', 'n', 'nonmarkingreturn', 'r', 'space', 'three', 'two', 'underscore', 'zero']
Glyph IDs:   [0, 1, 2, 3, 19, 21, 22, 23, 24, 36, 38, 45, 48, 49, 51, 54, 59, 66, 68, 70, 72, 73, 74, 76, 81, 85]
Closing glyph list over 'GSUB': 26 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'C', 'J', 'M', 'N', 'P', 'S', 'X', 'a', 'c', 'e', 'f', 'five', 'four', 'g', 'i', 'n', 'nonmarkingreturn', 'r', 'space', 'three', 'two', 'underscore', 'zero']
Glyph IDs:   [0, 1, 2, 3, 19, 21, 22, 23, 24, 36, 38, 45, 48, 49, 51, 54, 59, 66, 68, 70, 72, 73, 74, 76, 81, 85]
Closed glyph list over 'GSUB': 29 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'C', 'J', 'M', 'N', 'P', 'S', 'X', 'a', 'c', 'e', 'f', 'fi', 'five', 'four', 'g', 'i', 'n', 'nonmarkingreturn', 'r', 'space', 'three', 'two', 'underscore', 'uniFB00', 'uniFB03', 'zero']
Glyph IDs:   [0, 1, 2, 3, 19, 21, 22, 23, 24, 36, 38, 45, 48, 49, 51, 54, 59, 66, 68, 70, 72, 73, 74, 76, 81, 85, 5038, 5039, 5041]
Closing glyph list over 'glyf': 29 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'C', 'J', 'M', 'N', 'P', 'S', 'X', 'a', 'c', 'e', 'f', 'fi', 'five', 'four', 'g', 'i', 'n', 'nonmarkingreturn', 'r', 'space', 'three', 'two', 'underscore', 'uniFB00', 'uniFB03', 'zero']
Glyph IDs:   [0, 1, 2, 3, 19, 21, 22, 23, 24, 36, 38, 45, 48, 49, 51, 54, 59, 66, 68, 70, 72, 73, 74, 76, 81, 85, 5038, 5039, 5041]
Closed glyph list over 'glyf': 29 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'C', 'J', 'M', 'N', 'P', 'S', 'X', 'a', 'c', 'e', 'f', 'fi', 'five', 'four', 'g', 'i', 'n', 'nonmarkingreturn', 'r', 'space', 'three', 'two', 'underscore', 'uniFB00', 'uniFB03', 'zero']
Glyph IDs:   [0, 1, 2, 3, 19, 21, 22, 23, 24, 36, 38, 45, 48, 49, 51, 54, 59, 66, 68, 70, 72, 73, 74, 76, 81, 85, 5038, 5039, 5041]
Retaining 29 glyphs
head subsetting not needed
hhea subsetting not needed
maxp subsetting not needed
OS/2 subsetting not needed
hmtx subsetted
cmap subsetted
fpgm subsetting not needed
prep subsetting not needed
cvt  subsetting not needed
loca subsetting not needed
post subsetted
gasp subsetting not needed
MATH subsetted
GDEF subsetted
GPOS subsetted
GSUB subsetted
name subsetting not needed
glyf subsetted
head pruned
OS/2 Unicode ranges pruned: [0]
OS/2 CodePage ranges pruned: [0]
glyf pruned
GDEF pruned
GPOS pruned
GSUB pruned
name pruned
```

Example 50 (python):
```python
ADATA_PATH = "/nfs/team205/heart/anndata_objects/Foetal/Feb28ObjectRaw_finegrain_updated.h5ad"
ad = sc.read_h5ad(ADATA_PATH, backed='r')

# Step 1: Create mapping (already done)
cat = "mid_grain"
mid2colours = dict(zip(ad.obs[cat].cat.categories, ad.uns[f"{cat}_colors"]))

# Step 2: Extract the labels and colors in the correct order
mid_grains = list(mid2colours.keys())
palette = list(mid2colours.values())
num_mid_grains = len(mid_grains)

# Step 3: Plot
plt.figure(figsize=(num_mid_grains * 0.4, 2))  # wide figure
sns.palplot(palette)
plt.xticks(np.arange(num_mid_grains), mid_grains, rotation=90)
plt.title("Mid Grain to Color Mapping")
plt.tight_layout()
plt.show()
```

Example 51 (unknown):
```unknown
<Figure size 448x160 with 0 Axes>
```

Example 52 (python):
```python
adata.obs['celltypist_mid_fin'] = pd.Categorical(
    adata.obs['celltypist_mid_fin'],
    categories=ad.obs[cat].cat.categories,
    ordered=True
)
```

Example 53 (python):
```python
sc.pl.embedding(adata,
                    basis=f"X_umap",
                    color='celltypist_mid_fin',palette=palette,
                show=False
                   )
plt.savefig(f'{figdir}/VisiumHD_UMAP-midgrain_for-Fig1.pdf',bbox_inches="tight",dpi=300)
```

Example 54 (unknown):
```unknown
maxp pruned
cmap pruned
kern dropped
post pruned
FFTM dropped
GPOS pruned
GSUB pruned
glyf pruned
Added gid0 to subset
Added first four glyphs to subset
Closing glyph list over 'MATH': 38 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'B', 'C', 'E', 'F', 'G', 'L', 'M', 'N', 'P', 'S', 'V', 'X', 'a', 'b', 'c', 'd', 'e', 'f', 'h', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'one', 'p', 'r', 's', 'space', 't', 'two', 'u', 'underscore', 'y']
Glyph IDs:   [0, 1, 2, 3, 20, 21, 36, 37, 38, 40, 41, 42, 47, 48, 49, 51, 54, 57, 59, 66, 68, 69, 70, 71, 72, 73, 75, 76, 79, 80, 81, 82, 83, 85, 86, 87, 88, 92]
Closed glyph list over 'MATH': 38 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'B', 'C', 'E', 'F', 'G', 'L', 'M', 'N', 'P', 'S', 'V', 'X', 'a', 'b', 'c', 'd', 'e', 'f', 'h', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'one', 'p', 'r', 's', 'space', 't', 'two', 'u', 'underscore', 'y']
Glyph IDs:   [0, 1, 2, 3, 20, 21, 36, 37, 38, 40, 41, 42, 47, 48, 49, 51, 54, 57, 59, 66, 68, 69, 70, 71, 72, 73, 75, 76, 79, 80, 81, 82, 83, 85, 86, 87, 88, 92]
Closing glyph list over 'GSUB': 38 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'B', 'C', 'E', 'F', 'G', 'L', 'M', 'N', 'P', 'S', 'V', 'X', 'a', 'b', 'c', 'd', 'e', 'f', 'h', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'one', 'p', 'r', 's', 'space', 't', 'two', 'u', 'underscore', 'y']
Glyph IDs:   [0, 1, 2, 3, 20, 21, 36, 37, 38, 40, 41, 42, 47, 48, 49, 51, 54, 57, 59, 66, 68, 69, 70, 71, 72, 73, 75, 76, 79, 80, 81, 82, 83, 85, 86, 87, 88, 92]
Closed glyph list over 'GSUB': 43 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'B', 'C', 'E', 'F', 'G', 'L', 'M', 'N', 'P', 'S', 'V', 'X', 'a', 'b', 'c', 'd', 'e', 'f', 'fi', 'fl', 'h', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'one', 'p', 'r', 's', 'space', 't', 'two', 'u', 'underscore', 'uniFB00', 'uniFB03', 'uniFB04', 'y']
Glyph IDs:   [0, 1, 2, 3, 20, 21, 36, 37, 38, 40, 41, 42, 47, 48, 49, 51, 54, 57, 59, 66, 68, 69, 70, 71, 72, 73, 75, 76, 79, 80, 81, 82, 83, 85, 86, 87, 88, 92, 5038, 5039, 5040, 5041, 5042]
Closing glyph list over 'glyf': 43 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'B', 'C', 'E', 'F', 'G', 'L', 'M', 'N', 'P', 'S', 'V', 'X', 'a', 'b', 'c', 'd', 'e', 'f', 'fi', 'fl', 'h', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'one', 'p', 'r', 's', 'space', 't', 'two', 'u', 'underscore', 'uniFB00', 'uniFB03', 'uniFB04', 'y']
Glyph IDs:   [0, 1, 2, 3, 20, 21, 36, 37, 38, 40, 41, 42, 47, 48, 49, 51, 54, 57, 59, 66, 68, 69, 70, 71, 72, 73, 75, 76, 79, 80, 81, 82, 83, 85, 86, 87, 88, 92, 5038, 5039, 5040, 5041, 5042]
Closed glyph list over 'glyf': 43 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'B', 'C', 'E', 'F', 'G', 'L', 'M', 'N', 'P', 'S', 'V', 'X', 'a', 'b', 'c', 'd', 'e', 'f', 'fi', 'fl', 'h', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'one', 'p', 'r', 's', 'space', 't', 'two', 'u', 'underscore', 'uniFB00', 'uniFB03', 'uniFB04', 'y']
Glyph IDs:   [0, 1, 2, 3, 20, 21, 36, 37, 38, 40, 41, 42, 47, 48, 49, 51, 54, 57, 59, 66, 68, 69, 70, 71, 72, 73, 75, 76, 79, 80, 81, 82, 83, 85, 86, 87, 88, 92, 5038, 5039, 5040, 5041, 5042]
Retaining 43 glyphs
head subsetting not needed
hhea subsetting not needed
maxp subsetting not needed
OS/2 subsetting not needed
hmtx subsetted
cmap subsetted
fpgm subsetting not needed
prep subsetting not needed
cvt  subsetting not needed
loca subsetting not needed
post subsetted
gasp subsetting not needed
MATH subsetted
GDEF subsetted
GPOS subsetted
GSUB subsetted
name subsetting not needed
glyf subsetted
head pruned
OS/2 Unicode ranges pruned: [0]
OS/2 CodePage ranges pruned: [0]
glyf pruned
GDEF pruned
GPOS pruned
GSUB pruned
name pruned
```

---

## Import libraries¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig4_Cardiomyocytes/VentricularCardiomyocytes/CellOracle/CellOracle_no1_construct-GRN_Left-cardiomyocytes.html

**Contents:**
- Import libraries¶
- Load gene-expression data¶
- Make oracle object¶
- Add scenicplus base GRN¶
- KNN imputation¶
- GRN calculation¶
- Network preprocessing for perturbation simulation¶
- Save the oracle object¶
- Network preprocessing¶
- Network analysis; Network score for each gene¶
- Find parameters for n_grid and min_mass for perturbation simulation¶
- Make predictive modelos for simulation¶
- In silico TF perturbation analysis¶
- Visualisation¶
- Find parameters for n_grid and min_mass¶
- Plot vector field¶
- Compare simulation vector with development vectors¶
- Focus on a single development lineage to interpret the results in detail¶

ref: https://morris-lab.github.io/CellOracle.documentation/notebooks/04_Network_analysis/Network_analysis_with_Paul_etal_2015_data.html

CellOracle uses the same strategy as velocyto for visualizing cell transitions. This process requires KNN imputation in advance.

For the KNN imputation, we first need to calculate and select PCs.

Estimate the optimal number of nearest neighbors for KNN imputation.

67559 rows × 6 columns

from below ref: https://morris-lab.github.io/CellOracle.documentation/notebooks/05_simulation/Gata1_KO_simulation_with_Paul_etal_2015_data.html

Here, we will need to fit the ridge regression models again. This process will take less time than the GRN inference in the previous notebook, because we are using the filtered GRN models.

The next step calculates the probability of cell state transitions based on the simulation data. You can use the transition probabilities between cells to predict how cells will change after a perturbation.

This transition probability will be used later.

Caution: It is very important to find the optimal scale parameter.

If the vectors are not visible, you can try a smaller scale parameter to magnify the vector length. However, if you see large vectors in the randomized results (right panel), it means that the scale parameter is too small.

Again, we need to adjust the scale parameter. Please seek to find the optimal scale parameter that provides good visualization.

If you don’t see any vector, you can try the smaller scale parameter to magnify the vector length. However, if you see large vectors in the randomized results (right panel), it means the scale parameter is too small.

Here, we will calculate the development vector field using pseudotime gradient as follows.

Note: Other methods may be used to represent a continuous scRNA-seq trajectory flow. For example, RNA velocity analysis is a good way to estimate the direction of cell differentiation. Choose the method that best suits your data.

In the analysis below, we will use pseudotime data. Pseudotime data is included in the demo data. If you are analyzing your own scRNA-seq data, please calculate pseudotime before starting this analysis.

Next, we convert the pseudotime data into grid points. For this calculation we can chose one of two methods.

knn: K-Nearesr Neighbor regression. You will need to set number of neighbors. Please adjust n_knn for best results.This will depend on the population size and density of your scRNA-seq data.

gradient.transfer_data_into_grid(args={"method": "knn", "n_knn":50})

polynomial: Polynomial regression using x-axis and y-axis of dimensional reduction space.

In general, this method will be more robust. Please use this method if knn method does not work. n_poly is the number of degrees for the polynomial regression model. Please try to find appropriaten_poly searching for best results.

gradient.transfer_data_into_grid(args={"method": "polynomial", "n_poly":3})

Calculate the 2D vector map to represents the pseudotime gradient. After the gradient calculation, the length of the vector will be normalized automatically.

Please adjust the scale parameter to adjust vector length visualization.

The inner product represents the similarity between two vectors.

Using the inner product, we compare the 2D vector field of perturbation simulation and development flow.

The inner product will be a positive value when two vectors are pointing in the same direction.

Conversely, the inner product will be a negative value when two vectors are pointing in the opposing directions.

The length of the vectors also affects the magnitude of inner product value.

In summary, we quantitatively compare the directionality and size of vectors between perturbation simulation and natural differentiation using inner product, and we define the score as perturbation score (PS).

A negative PS means that the TF perturbation would block differentiation.

A positive PS means that the TF perturbation would promote differentiation.

Here, need to adjust the vm parameter for the PS score color visualization. Please seek to find the optimal vm parameter that provides good visualization.

If you don’t see any color in the left panel below, you can try the smaller vm parameter to magnify the scale of vm visualization. However, if you see colors in the randomized results (right panel), it means the vm parameter is too small.

**Examples:**

Example 1 (python):
```python
import os
import sys

import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import scanpy as sc
import seaborn as sns
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
-----
matplotlib          3.6.3
numpy               1.23.5
pandas              1.5.3
scanpy              1.9.2
seaborn             0.12.2
session_info        1.0.0
-----
```

Example 4 (python):
```python
PIL                 9.4.0
anndata             0.8.0
asttokens           NA
backcall            0.2.0
beta_ufunc          NA
binom_ufunc         NA
cffi                1.15.1
colorama            0.4.6
comm                0.1.2
cycler              0.10.0
cython_runtime      NA
dateutil            2.8.2
debugpy             1.6.6
decorator           5.1.1
defusedxml          0.7.1
executing           1.2.0
google              NA
h5py                3.8.0
hypergeom_ufunc     NA
igraph              0.10.4
importlib_metadata  NA
invgauss_ufunc      NA
ipykernel           6.21.2
ipython_genutils    0.2.0
ipywidgets          8.0.4
jedi                0.18.2
joblib              1.2.0
kiwisolver          1.4.4
llvmlite            0.39.1
louvain             0.8.0
mpl_toolkits        NA
natsort             8.2.0
nbinom_ufunc        NA
ncf_ufunc           NA
nct_ufunc           NA
ncx2_ufunc          NA
numba               0.56.4
packaging           23.0
parso               0.8.3
pexpect             4.8.0
pickleshare         0.7.5
pkg_resources       NA
platformdirs        3.0.0
prompt_toolkit      3.0.36
psutil              5.9.4
ptyprocess          0.7.0
pure_eval           0.2.2
pyarrow             11.0.0
pydev_ipython       NA
pydevconsole        NA
pydevd              2.9.5
pydevd_file_utils   NA
pydevd_plugins      NA
pydevd_tracing      NA
pygments            2.14.0
pyparsing           3.0.9
pytz                2022.7.1
scipy               1.10.1
setuptools          67.4.0
six                 1.16.0
skewnorm_ufunc      NA
sklearn             1.2.1
stack_data          0.6.2
statsmodels         0.13.5
texttable           1.6.7
threadpoolctl       3.1.0
tornado             6.2
traitlets           5.9.0
typing_extensions   NA
wcwidth             0.2.6
yaml                6.0
zipp                NA
zmq                 25.0.0
```

Example 5 (unknown):
```unknown
-----
IPython             8.4.0
jupyter_client      8.0.3
jupyter_core        5.2.0
notebook            6.5.2
-----
Python 3.8.16 | packaged by conda-forge | (default, Feb  1 2023, 16:01:55) [GCC 11.3.0]
Linux-4.15.0-213-generic-x86_64-with-glibc2.10
-----
Session information updated at 2023-10-17 13:23
```

Example 6 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 7 (python):
```python
sc.settings.set_figure_params(dpi=120)
```

Example 8 (python):
```python
# read in adata
# whole data
adata = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/vCM/post_scFate/foetal_vCM_onlyLeft.h5ad')

# subset gene expression data
adata = adata[:,adata.var['modality']=='Gene Expression']

# keep raw cont data before log normalisation
adata.raw = adata
adata.layers["raw_count"] = adata.raw.X.copy()

# filter and log-normalise
sc.pp.filter_genes(adata, min_cells=3)
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

print(adata.X.data[:10])
adata
```

Example 9 (unknown):
```unknown
[1.0619993 1.5653387 1.8985853 1.5653387 2.347629  1.8985853 1.5653387
 1.0619993 1.0619993 1.0619993]
```

Example 10 (unknown):
```unknown
AnnData object with n_obs × n_vars = 16451 × 26187
    obs: 'sangerID', 'combinedID', 'donor', 'region', 'age', 'sex', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'week', 'trimester', 'cycling', 'S_score', 'G2M_score', 'phase', 'fine_grain', 'mid_grain', 'coarse_grain', 'batch_key', 'total_counts', 'total_counts_mt', 'total_counts_ribo', 'n_genes_by_counts', 'pct_counts_mt', 'pct_counts_ribo', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'log1p_total_counts_mt', 'log1p_total_counts_ribo', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', '_indices', '_scvi_batch', '_scvi_labels', 'leiden_MultiVI', 't', 'seg', 'edge', 'milestones', 'age_days', 'week_cont'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id', 'seqnames', 'start', 'end', 'width', 'strand', 'score', 'replicateScoreQuantile', 'groupScoreQuantile', 'Reproducibility', 'GroupReplicate', 'distToGeneStart', 'nearestGene', 'peakType', 'distToTSS', 'nearestTSS', 'GC', 'idx', 'N', 'n_cells'
    uns: '_scvi_manager_uuid', '_scvi_uuid', 'cycling_colors', 'donor_colors', 'draw_graph', 'fine_grain_colors', 'graph', 'leiden', 'leiden_MultiVI_colors', 'milestones_colors', 'neighbors', 'ppt', 'pseudotime_list', 'region_colors', 'umap', 'week_colors', 'log1p'
    obsm: 'MultiVI_latent', 'X_R', 'X_draw_graph_fa', 'X_palantir', 'X_umap', 'X_umap13', '_scvi_extra_categorical_covs', '_scvi_extra_continuous_covs'
    layers: 'raw_count'
    obsp: 'connectivities', 'distances'
```

Example 11 (python):
```python
# select genes which are targeted or the regulators in the regulons
# 
ereg_signatures = utils.read_pkl("/nfs/team205/heart/pyscenic/plus/eRegulon_signatures/vCM_onlyLeft_Gene_eRegulon_signatures.pkl")
genes_sel = []
for key,value in ereg_signatures.items():
    genes_sel = genes_sel + value
    genes_sel = genes_sel + [key.split('_')[0]]
genes_sel = list(set(genes_sel))

# hvg and targeted
# hvg_targeted = list(set(adata.var_names[adata.var.highly_variable]).intersection(genes_targeted))
# len(hvg_targeted)
len(genes_sel)
```

Example 12 (python):
```python
# subset genes
# adata = adata[:,hvg_targeted]
adata = adata[:,genes_sel]
adata
```

Example 13 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 16451 × 11229
    obs: 'sangerID', 'combinedID', 'donor', 'region', 'age', 'sex', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'week', 'trimester', 'cycling', 'S_score', 'G2M_score', 'phase', 'fine_grain', 'mid_grain', 'coarse_grain', 'batch_key', 'total_counts', 'total_counts_mt', 'total_counts_ribo', 'n_genes_by_counts', 'pct_counts_mt', 'pct_counts_ribo', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'log1p_total_counts_mt', 'log1p_total_counts_ribo', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', '_indices', '_scvi_batch', '_scvi_labels', 'leiden_MultiVI', 't', 'seg', 'edge', 'milestones', 'age_days', 'week_cont'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id', 'seqnames', 'start', 'end', 'width', 'strand', 'score', 'replicateScoreQuantile', 'groupScoreQuantile', 'Reproducibility', 'GroupReplicate', 'distToGeneStart', 'nearestGene', 'peakType', 'distToTSS', 'nearestTSS', 'GC', 'idx', 'N', 'n_cells'
    uns: '_scvi_manager_uuid', '_scvi_uuid', 'cycling_colors', 'donor_colors', 'draw_graph', 'fine_grain_colors', 'graph', 'leiden', 'leiden_MultiVI_colors', 'milestones_colors', 'neighbors', 'ppt', 'pseudotime_list', 'region_colors', 'umap', 'week_colors', 'log1p'
    obsm: 'MultiVI_latent', 'X_R', 'X_draw_graph_fa', 'X_palantir', 'X_umap', 'X_umap13', '_scvi_extra_categorical_covs', '_scvi_extra_continuous_covs'
    layers: 'raw_count'
    obsp: 'connectivities', 'distances'
```

Example 14 (python):
```python
print(f"Cell number is :{adata.shape[0]}")
print(f"Gene number is :{adata.shape[1]}")
```

Example 15 (unknown):
```unknown
Cell number is :16451
Gene number is :11229
```

Example 16 (python):
```python
# add t_binned category
adata.obs['t_binned']=np.nan

# compact
adata_compact =  sc.read_h5ad('/nfs/team205/heart/anndata_objects/Vitro/post_multiVI_full-features/Foetal_vCM_Left-compact_rna.h5ad',
                              backed='r')
adata.obs.loc[adata_compact.obs_names, 't_binned'] = adata_compact.obs['t_binned'].copy()

# trabeculated
adata_trab =  sc.read_h5ad('/nfs/team205/heart/anndata_objects/Vitro/post_multiVI_full-features/Foetal_vCM_Left-trabeculated_rna.h5ad',
                              backed='r')
adata.obs.loc[adata_trab.obs_names, 't_binned'] = adata_trab.obs['t_binned'].copy()

adata.obs['t_binned'].value_counts()

# combine fine_grain and t_binned category
adata.obs['fine_grain_time-binned'] = adata.obs['fine_grain'].astype('str')+'_'+adata.obs['t_binned'].astype('str')
```

Example 17 (python):
```python
/tmp/ipykernel_900/724224780.py:2: ImplicitModificationWarning: Trying to modify attribute `.obs` of view, initializing view as actual.
  adata.obs['t_binned']=np.nan
```

Example 18 (python):
```python
set(adata.obs['t_binned'])
```

Example 19 (unknown):
```unknown
{'t1', 't2', 't3'}
```

Example 20 (python):
```python
sc.pl.draw_graph(adata, color=['fine_grain','t','fine_grain_time-binned'], size=5, wspace=0.3, ncols=3)
```

Example 21 (unknown):
```unknown
/home/jovyan/my-conda-envs/celloracle_env/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/home/jovyan/my-conda-envs/celloracle_env/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 22 (python):
```python
import celloracle as co
co.__version__
```

Example 23 (unknown):
```unknown
# Instantiate Oracle object
oracle = co.Oracle()
```

Example 24 (python):
```python
# Check data in anndata
print("Metadata columns :", list(adata.obs.columns))
print("Dimensional reduction: ", list(adata.obsm.keys()))
```

Example 25 (python):
```python
Metadata columns : ['sangerID', 'combinedID', 'donor', 'region', 'age', 'sex', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'week', 'trimester', 'cycling', 'S_score', 'G2M_score', 'phase', 'fine_grain', 'mid_grain', 'coarse_grain', 'batch_key', 'total_counts', 'total_counts_mt', 'total_counts_ribo', 'n_genes_by_counts', 'pct_counts_mt', 'pct_counts_ribo', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'log1p_total_counts_mt', 'log1p_total_counts_ribo', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', '_indices', '_scvi_batch', '_scvi_labels', 'leiden_MultiVI', 't', 'seg', 'edge', 'milestones', 'age_days', 'week_cont', 't_binned', 'fine_grain_time-binned']
Dimensional reduction:  ['MultiVI_latent', 'X_R', 'X_draw_graph_fa', 'X_palantir', 'X_umap', 'X_umap13', '_scvi_extra_categorical_covs', '_scvi_extra_continuous_covs']
```

Example 26 (python):
```python
# In this notebook, we use the unscaled mRNA count for the nput of Oracle object.
adata.X = adata.layers["raw_count"].copy()

# Instantiate Oracle object.
oracle.import_anndata_as_raw_count(adata=adata,
                                   cluster_column_name="fine_grain_time-binned",
                                   embedding_name="X_draw_graph_fa")
```

Example 27 (python):
```python
11229 genes were found in the adata. Note that Celloracle is intended to use around 1000-3000 genes, so the behavior with this number of genes may differ from what is expected.
WARNING: adata.X seems to be already log-transformed.
```

Example 28 (unknown):
```unknown
# read in scenicplus regulons
regulons = utils.read_pkl('/nfs/team205/heart/pyscenic/plus/baseGRN_forCO/vCM_onlyLeft_baseGRN_act.pkl')
TG_to_TF_dictionary = co.utility.inverse_dictionary(regulons)
```

Example 29 (unknown):
```unknown
0%|          | 0/11200 [00:00<?, ?it/s]
```

Example 30 (unknown):
```unknown
# add
oracle.import_TF_data(TFdict=TG_to_TF_dictionary)
```

Example 31 (unknown):
```unknown
TG_to_TF_dictionary['SGSM2']
```

Example 32 (unknown):
```unknown
['CREB1',
 'HAND2',
 'MEF2A',
 'PLAG1',
 'PRDM16',
 'SREBF2',
 'TEAD4',
 'ZNF148',
 'ZNF316',
 'ZNF576']
```

Example 33 (python):
```python
%%time
%matplotlib inline
# Perform PCA
oracle.perform_PCA()

# Select important PCs
plt.plot(np.cumsum(oracle.pca.explained_variance_ratio_)[:100])
n_comps = np.where(np.diff(np.diff(np.cumsum(oracle.pca.explained_variance_ratio_))>0.002))[0][0]
plt.axvline(n_comps, c="k")
plt.show()
print(n_comps)
n_comps = min(n_comps, 50)
```

Example 34 (unknown):
```unknown
6
CPU times: user 1h 11min 45s, sys: 15min 10s, total: 1h 26min 56s
Wall time: 8min 54s
```

Example 35 (python):
```python
oracle.pcs.shape
```

Example 36 (unknown):
```unknown
(16451, 11229)
```

Example 37 (python):
```python
n_cell = oracle.adata.shape[0]
print(f"cell number is :{n_cell}")
```

Example 38 (unknown):
```unknown
cell number is :16451
```

Example 39 (python):
```python
k = int(0.025*n_cell)
print(f"Auto-selected k is :{k}")
```

Example 40 (unknown):
```unknown
Auto-selected k is :411
```

Example 41 (unknown):
```unknown
%%time
oracle.knn_imputation(n_pca_dims=n_comps, k=k, balanced=True, b_sight=k*8,
                      b_maxl=k*4, n_jobs=4)
```

Example 42 (unknown):
```unknown
CPU times: user 4min 13s, sys: 5.82 s, total: 4min 18s
Wall time: 4min 4s
```

Example 43 (unknown):
```unknown
# Save oracle object.
oracle.to_hdf5("/nfs/team205/heart/anndata_objects/Foetal/vCM/post_scFate/foetal_vCM_onlyLeft.celloracle.oracle")
```

Example 44 (unknown):
```unknown
# Load
oracle = co.load_hdf5("/nfs/team205/heart/anndata_objects/Foetal/vCM/post_scFate/foetal_vCM_onlyLeft.celloracle.oracle")
```

Example 45 (python):
```python
%matplotlib inline
# Check clustering data
sc.pl.draw_graph(oracle.adata, color='fine_grain_time-binned')
```

Example 46 (unknown):
```unknown
%%time
# Calculate GRN for each population in "louvain_annot" clustering unit.
# This step may take some time.(~30 minutes)
links = oracle.get_links(cluster_name_for_GRN_unit='fine_grain_time-binned', alpha=10,
                         verbose_level=10)
```

Example 47 (unknown):
```unknown
0%|          | 0/6 [00:00<?, ?it/s]
```

Example 48 (unknown):
```unknown
Inferring GRN for VentricularCardiomyocytesLeftCompact_t1...
```

Example 49 (unknown):
```unknown
0%|          | 0/11200 [00:00<?, ?it/s]
```

Example 50 (unknown):
```unknown
Inferring GRN for VentricularCardiomyocytesLeftCompact_t2...
```

Example 51 (unknown):
```unknown
0%|          | 0/11200 [00:00<?, ?it/s]
```

Example 52 (unknown):
```unknown
Inferring GRN for VentricularCardiomyocytesLeftCompact_t3...
```

Example 53 (unknown):
```unknown
0%|          | 0/11200 [00:00<?, ?it/s]
```

Example 54 (unknown):
```unknown
Inferring GRN for VentricularCardiomyocytesLeftTrabeculated_t1...
```

Example 55 (unknown):
```unknown
0%|          | 0/11200 [00:00<?, ?it/s]
```

Example 56 (unknown):
```unknown
Inferring GRN for VentricularCardiomyocytesLeftTrabeculated_t2...
```

Example 57 (unknown):
```unknown
0%|          | 0/11200 [00:00<?, ?it/s]
```

Example 58 (unknown):
```unknown
Inferring GRN for VentricularCardiomyocytesLeftTrabeculated_t3...
```

Example 59 (unknown):
```unknown
0%|          | 0/11200 [00:00<?, ?it/s]
```

Example 60 (unknown):
```unknown
CPU times: user 29min 21s, sys: 1min 38s, total: 30min 59s
Wall time: 36min 53s
```

Example 61 (unknown):
```unknown
links.links_dict.keys()
```

Example 62 (unknown):
```unknown
dict_keys(['VentricularCardiomyocytesLeftCompact_t1', 'VentricularCardiomyocytesLeftCompact_t2', 'VentricularCardiomyocytesLeftCompact_t3', 'VentricularCardiomyocytesLeftTrabeculated_t1', 'VentricularCardiomyocytesLeftTrabeculated_t2', 'VentricularCardiomyocytesLeftTrabeculated_t3'])
```

Example 63 (unknown):
```unknown
links.links_dict['VentricularCardiomyocytesLeftCompact_t3']
```

Example 64 (unknown):
```unknown
# Save Links object.
links.to_hdf5(file_path="/nfs/team205/heart/anndata_objects/Foetal/vCM/post_scFate/vCM_onlyLeft_links.celloracle.links")
```

Example 65 (unknown):
```unknown
# load the saved links
links = co.load_hdf5("/nfs/team205/heart/anndata_objects/Foetal/vCM/post_scFate/vCM_onlyLeft_links.celloracle.links")
```

Example 66 (unknown):
```unknown
%%time

links.filter_links(p=0.001, weight="coef_abs", threshold_number=10000)

# Calculate network scores.
links.get_network_score()
display(links.merged_score.head())

# Make predictive modelos for simulation
# Here, we will need to fit the ridge regression models again.
# This process will take less time than the GRN inference in the previous notebook, because we are using the filtered GRN models.
oracle.get_cluster_specific_TFdict_from_Links(links_object=links)
oracle.fit_GRN_for_simulation(alpha=10,
                              use_cluster_specific_TFdict=True)
```

Example 67 (unknown):
```unknown
0%|          | 0/6 [00:00<?, ?it/s]
```

Example 68 (unknown):
```unknown
CPU times: user 16min 9s, sys: 16min 33s, total: 32min 42s
Wall time: 5min 24s
```

Example 69 (unknown):
```unknown
Oracle object

Meta data
    celloracle version used for instantiation: 0.11.0
    n_cells: 16451
    n_genes: 11229
    cluster_name: fine_grain_time-binned
    dimensional_reduction_name: X_draw_graph_fa
    n_target_genes_in_TFdict: 11200 genes
    n_regulatory_in_TFdict: 236 genes
    n_regulatory_in_both_TFdict_and_scRNA-seq: 236 genes
    n_target_genes_both_TFdict_and_scRNA-seq: 11200 genes
    k_for_knn_imputation: 411
Status
    Gene expression matrix: Ready
    BaseGRN: Ready
    PCA calculation: Done
    Knn imputation: Done
    GRN calculation for simulation: Done
```

Example 70 (unknown):
```unknown
oracle.to_hdf5("/nfs/team205/heart/anndata_objects/Foetal/vCM/post_scFate/foetal_vCM_onlyLeft.celloracle.oracle")
```

Example 71 (unknown):
```unknown
links.filter_links(p=0.001, weight="coef_abs", threshold_number=10000)
```

Example 72 (python):
```python
plt.rcParams["figure.figsize"] = [7, 3]
```

Example 73 (unknown):
```unknown
links.plot_degree_distributions(plot_model=True,
                                               #save=f"{save_folder}/degree_distribution/",
                                               )
```

Example 74 (unknown):
```unknown
VentricularCardiomyocytesLeftCompact_t1
```

Example 75 (unknown):
```unknown
VentricularCardiomyocytesLeftCompact_t2
```

Example 76 (unknown):
```unknown
VentricularCardiomyocytesLeftCompact_t3
```

Example 77 (unknown):
```unknown
VentricularCardiomyocytesLeftTrabeculated_t1
```

Example 78 (unknown):
```unknown
VentricularCardiomyocytesLeftTrabeculated_t2
```

Example 79 (unknown):
```unknown
VentricularCardiomyocytesLeftTrabeculated_t3
```

Example 80 (unknown):
```unknown
# Calculate network scores.
links.get_network_score()
```

Example 81 (unknown):
```unknown
links.merged_score.head()
```

Example 82 (unknown):
```unknown
# Check cluster name
links.cluster
```

Example 83 (unknown):
```unknown
['VentricularCardiomyocytesLeftCompact_t1',
 'VentricularCardiomyocytesLeftCompact_t2',
 'VentricularCardiomyocytesLeftCompact_t3',
 'VentricularCardiomyocytesLeftTrabeculated_t1',
 'VentricularCardiomyocytesLeftTrabeculated_t2',
 'VentricularCardiomyocytesLeftTrabeculated_t3']
```

Example 84 (python):
```python
# Visualize top n-th genes with high scores.
plt.rcParams["figure.figsize"] = [5, 7]
links.plot_scores_as_rank(cluster="VentricularCardiomyocytesLeftCompact_t3", n_gene=30, save=None)
```

Example 85 (python):
```python
# Compare GRN score between two clusters
plt.rcParams["figure.figsize"] = [5, 5]
links.plot_score_comparison_2D(value="betweenness_centrality",
                               cluster1='VentricularCardiomyocytesLeftCompact_t3', cluster2='VentricularCardiomyocytesLeftCompact_t1',
                               percentile=98,
                               save=None)
```

Example 86 (unknown):
```unknown
links.plot_score_comparison_2D(value="betweenness_centrality",
                               cluster1='VentricularCardiomyocytesLeftCompact_t3', cluster2='VentricularCardiomyocytesLeftTrabeculated_t3',
                               percentile=98,
                               save=None)
```

Example 87 (unknown):
```unknown
# set a gene for this trial
goi = "TBX20"
# Enter perturbation conditions to simulate signal propagation after the perturbation.
oracle.simulate_shift(perturb_condition={goi: 0.0}, # this is KO simulation so expression=0
                      n_propagation=3)
```

Example 88 (unknown):
```unknown
%%time
# Get transition probability
oracle.estimate_transition_prob(n_neighbors=200,
                                knn_random=True,
                                sampled_fraction=1)

# Calculate embedding
oracle.calculate_embedding_shift(sigma_corr=0.05)
```

Example 89 (python):
```python
# visualisation
fig, ax = plt.subplots(1, 2,  figsize=[13, 6])

scale = 20
# Show quiver plot
oracle.plot_quiver(scale=scale, ax=ax[0])
ax[0].set_title(f"Simulated cell identity shift vector: {goi} KO")

# Show quiver plot that was calculated with randomized graph.
oracle.plot_quiver_random(scale=scale, ax=ax[1])
ax[1].set_title(f"Randomized simulation vector")

plt.show()
```

Example 90 (unknown):
```unknown
# n_grid = 40 is a good starting value.
n_grid = 40
oracle.calculate_p_mass(smooth=0.8, n_grid=n_grid, n_neighbors=200)
```

Example 91 (unknown):
```unknown
# Search for best min_mass.
oracle.suggest_mass_thresholds(n_suggestion=12)
```

Example 92 (unknown):
```unknown
min_mass = 0.011
oracle.calculate_mass_filter(min_mass=min_mass, plot=True)
```

Example 93 (unknown):
```unknown
%%time
links.filter_links()
oracle.get_cluster_specific_TFdict_from_Links(links_object=links)
oracle.fit_GRN_for_simulation(alpha=10,
                              use_cluster_specific_TFdict=True)
```

Example 94 (python):
```python
# Check gene expression
goi = "TBX20"
sc.pl.draw_graph(oracle.adata, color=[goi, oracle.cluster_column_name],
                 layer="imputed_count", use_raw=False, cmap="viridis")
```

Example 95 (python):
```python
# Plot gene expression in histogram
sc.get.obs_df(oracle.adata, keys=[goi], layer="imputed_count").hist()
plt.show()
```

Example 96 (unknown):
```unknown
# Enter perturbation conditions to simulate signal propagation after the perturbation.
oracle.simulate_shift(perturb_condition={goi: 0.0}, # this is KO simulation so expression=0
                      n_propagation=3)
```

Example 97 (unknown):
```unknown
%%time
# Get transition probability
oracle.estimate_transition_prob(n_neighbors=200,
                                knn_random=True,
                                sampled_fraction=1)

# Calculate embedding
oracle.calculate_embedding_shift(sigma_corr=0.05)
```

Example 98 (python):
```python
fig, ax = plt.subplots(1, 2,  figsize=[13, 6])

scale = 20
# Show quiver plot
oracle.plot_quiver(scale=scale, ax=ax[0])
ax[0].set_title(f"Simulated cell identity shift vector: {goi} KO")

# Show quiver plot that was calculated with randomized graph.
oracle.plot_quiver_random(scale=scale, ax=ax[1])
ax[1].set_title(f"Randomized simulation vector")

plt.show()
```

Example 99 (unknown):
```unknown
# n_grid = 40 is a good starting value.
n_grid = 40
oracle.calculate_p_mass(smooth=0.8, n_grid=n_grid, n_neighbors=200)
```

Example 100 (unknown):
```unknown
# Search for best min_mass.
oracle.suggest_mass_thresholds(n_suggestion=12)
```

Example 101 (unknown):
```unknown
min_mass = 0.011
oracle.calculate_mass_filter(min_mass=min_mass, plot=True)
```

Example 102 (python):
```python
fig, ax = plt.subplots(1, 2,  figsize=[13, 6])

scale_simulation = 0.1
# Show quiver plot
oracle.plot_simulation_flow_on_grid(scale=scale_simulation, ax=ax[0])
ax[0].set_title(f"Simulated cell identity shift vector: {goi} KO")

# Show quiver plot that was calculated with randomized graph.
oracle.plot_simulation_flow_random_on_grid(scale=scale_simulation, ax=ax[1])
ax[1].set_title(f"Randomized simulation vector")

plt.show()
```

Example 103 (python):
```python
# Plot vector field with cell cluster
fig, ax = plt.subplots(figsize=[8, 8])

oracle.plot_cluster_whole(ax=ax, s=10)
oracle.plot_simulation_flow_on_grid(scale=scale_simulation, ax=ax, show_background=False)
```

Example 104 (python):
```python
# Visualize pseudotime
fig, ax = plt.subplots(figsize=[6,6])

sc.pl.embedding(adata=oracle.adata, basis=oracle.embedding_name, ax=ax, cmap="rainbow",
                color=["t"])
```

Example 105 (python):
```python
from celloracle.applications import Gradient_calculator

# Instantiate Gradient calculator object
gradient = Gradient_calculator(oracle_object=oracle, pseudotime_key="t")
```

Example 106 (unknown):
```unknown
gradient.calculate_p_mass(smooth=0.8, n_grid=n_grid, n_neighbors=200)
gradient.calculate_mass_filter(min_mass=min_mass, plot=True)
```

Example 107 (unknown):
```unknown
gradient.transfer_data_into_grid(args={"method": "knn", "n_knn":50})
```

Example 108 (unknown):
```unknown
# Calculate graddient
gradient.calculate_gradient()

# Show results
scale_dev = 40
gradient.visualize_results(scale=scale_dev, s=5)
```

Example 109 (python):
```python
# Visualize results
fig, ax = plt.subplots(figsize=[6, 6])
gradient.plot_dev_flow_on_grid(scale=scale_dev, ax=ax)
```

Example 110 (python):
```python
from celloracle.applications import Oracle_development_module

# Make Oracle_development_module to compare two vector field
dev = Oracle_development_module()

# Load development flow
dev.load_differentiation_reference_data(gradient_object=gradient)

# Load simulation result
dev.load_perturb_simulation_data(oracle_object=oracle)


# Calculate inner produc scores
dev.calculate_inner_product()
dev.calculate_digitized_ip(n_bins=10)
```

Example 111 (python):
```python
# Show perturbation scores
vm = 0.01

fig, ax = plt.subplots(1, 2, figsize=[12, 6])
dev.plot_inner_product_on_grid(vm=vm, s=50, ax=ax[0])
ax[0].set_title(f"PS")

dev.plot_inner_product_random_on_grid(vm=vm, s=50, ax=ax[1])
ax[1].set_title(f"PS calculated with Randomized simulation vector")
plt.show()
```

Example 112 (python):
```python
# Show perturbation scores with perturbation simulation vector field
fig, ax = plt.subplots(figsize=[6, 6])
dev.plot_inner_product_on_grid(vm=vm, s=50, ax=ax)
dev.plot_simulation_flow_on_grid(scale=scale_simulation, show_background=False, ax=ax)
```

Example 113 (unknown):
```unknown
# Let's visualize the results
dev.visualize_development_module_layout_0(s=5,
                                          scale_for_simulation=scale_simulation,
                                          s_grid=50,
                                          scale_for_pseudotime=scale_dev,
                                          vm=vm)
```

Example 114 (unknown):
```unknown
dev.inner_product_df
```

Example 115 (unknown):
```unknown
dev.cell_idx_use
```

Example 116 (python):
```python
# Get cell index list for the cells of interest
clusters = ['VentricularCardiomyocytesLeftCompact_t1',
 'VentricularCardiomyocytesLeftCompact_t2',
 'VentricularCardiomyocytesLeftCompact_t3']
cell_idx = np.where(oracle.adata.obs["fine_grain_time-binned"].isin(clusters))[0]

# Check index
print(cell_idx)
```

Example 117 (unknown):
```unknown
dev = Oracle_development_module()

# Load development flow
dev.load_differentiation_reference_data(gradient_object=gradient)

# Load simulation result
dev.load_perturb_simulation_data(oracle_object=oracle,
                                 cell_idx_use=cell_idx, # Enter cell id list
                                 name="Lineage_Compact" # Name of this cell group. You can enter any name.
                                 )

# Calculation
dev.calculate_inner_product()
dev.calculate_digitized_ip(n_bins=10)
```

Example 118 (unknown):
```unknown
# Let's visualize the results
dev.visualize_development_module_layout_0(s=5,
                                          scale_for_simulation=scale_simulation,
                                          s_grid=50,
                                          scale_for_pseudotime=scale_dev,
                                          vm=vm)
```

Example 119 (unknown):
```unknown
dev.inner_product_df
```

Example 120 (unknown):
```unknown
df = dev.inner_product_df.copy()
df[['score','pseudotime']].corr(method='pearson')
```

Example 121 (unknown):
```unknown
dev.inner_product_df
```

Example 122 (unknown):
```unknown
len(dev.inner_product)
```

---

## Variables¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/Xenium/no2_integration_and_cell-type-annotation/no6_manual-annotation_EpicardialCells.html

**Contents:**
- Variables¶
- Read in data¶
- Markers¶
- 1st round annotation¶
- Save¶

--> cluster 0 has lower total counts and smaller cell area, but has mesothelial marker gene expression --> retain

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os

import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 

import scanpy.external as sce
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/session_info/main.py:213: UserWarning: The '__version__' attribute is deprecated and will be removed in MarkupSafe 3.1. Use feature detection, or `importlib.metadata.version("markupsafe")`, instead.
  mod_version = _find_version(mod.__version__)
```

Example 4 (python):
```python
-----
matplotlib          3.10.1
numpy               2.1.3
pandas              2.2.3
scanpy              1.11.1
seaborn             0.13.2
session_info        v1.0.1
-----
```

Example 5 (python):
```python
Cython              3.1.1
PIL                 11.1.0
anndata             0.11.4
asciitree           NA
asttokens           NA
charset_normalizer  3.4.1
cloudpickle         3.1.1
comm                0.2.2
cycler              0.12.1
cython              3.1.1
cython_runtime      NA
dask                2024.11.2
dateutil            2.9.0.post0
debugpy             1.8.13
decorator           5.2.1
exceptiongroup      1.2.2
executing           2.2.0
h5py                3.13.0
igraph              0.11.8
importlib_metadata  NA
ipykernel           6.29.5
jedi                0.19.2
jinja2              3.1.6
joblib              1.4.2
kiwisolver          1.4.8
legacy_api_wrap     NA
leidenalg           0.10.2
llvmlite            0.44.0
markupsafe          3.0.2
mpl_toolkits        NA
msgpack             1.1.0
natsort             8.4.0
numba               0.61.0
numcodecs           0.13.1
packaging           24.2
parso               0.8.4
patsy               1.0.1
platformdirs        4.3.7
prompt_toolkit      3.0.50
psutil              7.0.0
pure_eval           0.2.3
pyarrow             19.0.1
pydev_ipython       NA
pydevconsole        NA
pydevd              3.2.3
pydevd_file_utils   NA
pydevd_plugins      NA
pydevd_tracing      NA
pygments            2.19.1
pyparsing           3.2.3
pytz                2025.2
scipy               1.15.2
six                 1.17.0
sklearn             1.5.2
stack_data          0.6.3
statsmodels         0.14.4
tblib               3.1.0
texttable           1.7.0
threadpoolctl       3.6.0
tlz                 1.0.0
toolz               1.0.0
tornado             6.4.2
traitlets           5.14.3
typing_extensions   NA
wcwidth             0.2.13
yaml                6.0.2
zarr                2.18.3
zipp                NA
zmq                 26.3.0
zoneinfo            NA
```

Example 6 (unknown):
```unknown
-----
IPython             8.34.0
jupyter_client      8.6.3
jupyter_core        5.7.2
-----
Python 3.10.14 | packaged by conda-forge | (main, Mar 20 2024, 12:45:18) [GCC 12.3.0]
Linux-4.18.0-553.53.1.el8_10.x86_64-x86_64-with-glibc2.28
-----
Session information updated at 2025-06-09 22:38
```

Example 7 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/home/kk837/rfs/rfs-teichlab-nfs-iCNyzSAaucw/kk837/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 8 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 9 (python):
```python
adata_dir = '/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium/subsets'
sample_id = 'C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat'
celltype = 'EpicardialCells'
```

Example 10 (unknown):
```unknown
# figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk/Aug2024'
# table_dir = '/nfs/team205/heart/fetal_ms_figures/tables/Aug2024'
```

Example 11 (unknown):
```unknown
latent_space = 'pca_harmony'
```

Example 12 (python):
```python
# read in
adata = sc.read_h5ad(f'{adata_dir}/{sample_id}_5K_{celltype}_lognorm.h5ad')
adata
```

Example 13 (unknown):
```unknown
AnnData object with n_obs × n_vars = 5206 × 4657
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_3.0', 'multi_celltypes_coarse', 'leiden_0.2', 'leiden_0.4', 'leiden_0.6', 'leiden_1'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1', 'n_cells', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'dendrogram_pca_harmony_leiden_1.0', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1_colors', 'log1p', 'multi_celltypes_coarse_colors', 'pca', 'pca_harmony', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.2_colors', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.3_colors', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.4_colors', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.5_colors', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_0.8_colors', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.0_colors', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_1.5_colors', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_2.0_colors', 'pca_harmony_leiden_3.0', 'pca_harmony_leiden_3.0_colors', 'tissue_block_id_colors', 'umap'
    obsm: 'X_pca', 'X_pca_harmony', 'X_umap_pca_harmony', 'spatial'
    varm: 'PCs'
    obsp: 'pca_harmony_connectivities', 'pca_harmony_distances'
```

Example 14 (python):
```python
adata.X.data[:5]
```

Example 15 (unknown):
```unknown
array([3.773983, 3.773983, 3.773983, 3.773983, 3.773983], dtype=float32)
```

Example 16 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=['total_counts','n_genes_by_counts','cell_area','tissue_block_id'],
                    wspace=0.2,cmap='RdPu',vmax='p100')
```

Example 17 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=['multi_celltypes_coarse',"conf_score_midmod2fine","celltypist_midmod2fine",],
                    wspace=0.2,cmap='RdPu',vmax='p100')
resolutions_list = [0.2,0.4,0.6,1]
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                    color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                    wspace=0.2)
```

Example 18 (python):
```python
%%time
# leiden clustering
resolutions_list = [# 0.2,0.4,0.6,1, # already done
                    1.2,1.5,2]
for resolution in resolutions_list:
    sc.tl.leiden(adata,neighbors_key=latent_space,
                     resolution=resolution,key_added=f'leiden_{str(resolution)}',n_iterations=2)
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                wspace=0.3)
```

Example 19 (unknown):
```unknown
<timed exec>:5: FutureWarning: In the future, the default backend for leiden will be igraph instead of leidenalg.

 To achieve the future defaults please pass: flavor="igraph" and n_iterations=2.  directed must also be False to work with igraph's implementation.
```

Example 20 (unknown):
```unknown
CPU times: user 3.38 s, sys: 79.9 ms, total: 3.46 s
Wall time: 3.47 s
```

Example 21 (python):
```python
# read in markers identified using single-nucleus data
markers_df = pd.read_csv(f'/home/kk837/rfs/rfs-teichlab-nfs-iCNyzSAaucw/kk837/notebooks/Foetal/VisiumHD/re-seq_mpp0p3/sc_{celltype}_finegrain_markers.csv')

mask1 = markers_df['group_mean']>1
mask2 = markers_df['logfoldchanges']>1
markers_df = markers_df[mask1&mask2]

markers_df
```

Example 22 (python):
```python
markers = {}
n_top = 100
for c in set(markers_df['group']):
    markers_df_sub = markers_df[markers_df['group']==c]
    markers_df_sub.sort_values('logfoldchanges',ascending=False)
    markers[c]=list(markers_df_sub[:n_top]['names'])

# manually add genes
markers['Cycling'] = ['MKI67','TOP2A'] 

genes = []
for g in markers.values():
    genes = genes + [x for x in g if (x in list(adata.var_names))&(x not in genes)]
for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in list(adata.var_names)]
len(genes)
```

Example 23 (python):
```python
sc.pl.embedding(adata,basis=f"X_umap_{latent_space}",color=genes, size=20,
               ncols=6, wspace=0.2,cmap='RdPu',vmax='p100')
```

Example 24 (python):
```python
sc.pl.embedding(adata,basis=f"X_umap_{latent_space}",
                color=['celltypist_midmod2fine', 'conf_score_midmod2fine'],
                wspace=0.7)
```

Example 25 (python):
```python
# umap per celltype to highlight
group_col = 'celltypist_midmod2fine'
groups_to_plot = list(set(adata.obs[group_col]))

plt.rcParams['axes.grid'] = False
fig,axs=plt.subplots(nrows=2,ncols=4,figsize=(15,7))
# Flatten the axs array for easier iteration in case of multiple rows and columns
axs_flat = axs.flatten()

for i,grp in enumerate(groups_to_plot):
    sc.pl.embedding(adata,basis=f"X_umap_{latent_space}",
                    color=[group_col], groups=grp, 
               size=30, frameon=False,
               show=False, ax=axs_flat[i], legend_loc=None, legend_fontsize=5,
               title=grp
              )
# Turn off any remaining subplots
for j in range(len(groups_to_plot), len(axs_flat)):
    fig.delaxes(axs_flat[j])

# fig.suptitle('Main Title for the Figure')
fig.tight_layout()
```

Example 26 (unknown):
```unknown
resolution_sel = 0.6
```

Example 27 (python):
```python
markers_others = {
    'CM': ['TTN', 'TNNT2','MYH6',
        'MYH7', 'MYL2', 'FHL2', # vCN
            'NPPA', 'MYL7', 'MYL4'], #aCM
    'FB': ['DCN', 'GCN', 'PDGFRA','COL1A1','COL1A2'],
    'EC': ['VWF', 'PECAM1', 'CDH5','RGCC', 'FABP5',],
    'Peri': ['RGS5', 'ABCC9', 'KCNJ8'],
    'SMC': ['MYH11', 'TAGLN', 'ACTA2'],
    'Meso': ['MSLN', 'WT1', 'BNC1'],
    'Neuro': ['PLP1', 'NRXN1', 'NRXN3','PRPH', 'NEFL', 'NEFM', 'NEFH', 'STMN2', 'SST', 'SLC10A4', 'SLC18A3', 'TH', 'NPY'],
    'Adipo': ['GPAM', 'FASN', 'LEP'],
    'Myelo': ['CD14', 'C1QA', 'CD68','LYVE1','TIMD4'],
    'Lymph': ['CD8A', 'IL7R', 'CD40LG'],
    'B': ['CD19','SDC1','IGHG1','IGHA1'],
    'Mast': ['KIT', 'CPA3'],
}

for key in markers_others.keys():
    markers_others[key]=[x for x in markers_others[key] if x in adata.var_names]

sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
sc.pl.dotplot(adata,
              markers_others, 
              groupby=f'leiden_{str(resolution_sel)}',
              dendrogram=True,
              standard_scale="var",
              color_map="Reds",
              swap_axes=False,
             )
```

Example 28 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: CM, FB, EC, etc.
```

Example 29 (python):
```python
sc.tl.dendrogram(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.pl.dotplot(adata,
              var_names=markers,
              groupby=f'leiden_{str(resolution_sel)}',
              standard_scale='var',
              dendrogram=True
             )
```

Example 30 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: MesothelialEpicardialCells, EpicardiumDerivedCells, Cycling
```

Example 31 (python):
```python
print(adata.X.data[:5])
adata.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata, dendrogram=True, standard_scale='var', return_fig=True)
dp.add_totals().show()
```

Example 32 (unknown):
```unknown
[3.773983 3.773983 3.773983 3.773983 3.773983]
```

Example 33 (python):
```python
sc.pl.violin(adata,keys=['total_counts','cell_area'],groupby=f'leiden_{str(resolution_sel)}')
```

Example 34 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=f'leiden_{str(resolution_sel)}',legend_loc='on data',
                )
```

Example 35 (python):
```python
# annotate
adata.obs['fine_grain_manual'] = adata.obs[f'leiden_{str(resolution_sel)}'].copy()
adata.obs.replace({'fine_grain_manual':{
    '0':'MesothelialEpicardialCells',
    '1':'MesothelialEpicardialCells',
    '2':'?EpithelialCells',
    '3':'MesothelialEpicardialCells',
    '4':'MesothelialEpicardialCells',
    '5':'?ThyroidCells',
    '6':'EpicardiumDerivedCells',
}},inplace=True)
adata.obs['fine_grain_manual'].value_counts()
```

Example 36 (python):
```python
/tmp/ipykernel_70466/3275159660.py:3: FutureWarning: The behavior of Series.replace (and DataFrame.replace) with CategoricalDtype is deprecated. In a future version, replace will only be used for cases that preserve the categories. To change the categories, use ser.cat.rename_categories instead.
  adata.obs.replace({'fine_grain_manual':{
```

Example 37 (unknown):
```unknown
fine_grain_manual
MesothelialEpicardialCells    3513
?EpithelialCells               786
?ThyroidCells                  518
EpicardiumDerivedCells         389
Name: count, dtype: int64
```

Example 38 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color='fine_grain_manual'
                )
```

Example 39 (python):
```python
sc.tl.dendrogram(adata,groupby='fine_grain_manual')
sc.pl.dotplot(adata,
              var_names=markers,
              groupby='fine_grain_manual',
              standard_scale='var',
              dendrogram=True
             )
```

Example 40 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: MesothelialEpicardialCells, ?EpithelialCells, ?ThyroidCells, etc.
var_group_labels: MesothelialEpicardialCells, EpicardiumDerivedCells, Cycling
```

Example 41 (python):
```python
for ct in set(adata.obs['fine_grain_manual']):
    for tissue_id in ['C194-HEA-0-FFPE-1', 'Hst45-HEA-0-FFPE-1']:
        slide = adata[adata.obs['tissue_block_id']==tissue_id]
        sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
                      spot_size=60,vmax='p100')
```

Example 42 (python):
```python
/tmp/ipykernel_70466/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 43 (python):
```python
/tmp/ipykernel_70466/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 44 (python):
```python
/tmp/ipykernel_70466/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 45 (python):
```python
/tmp/ipykernel_70466/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 46 (python):
```python
/tmp/ipykernel_70466/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 47 (python):
```python
/tmp/ipykernel_70466/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 48 (python):
```python
/tmp/ipykernel_70466/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 49 (python):
```python
/tmp/ipykernel_70466/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 50 (python):
```python
# save
adata.write(f'{adata_dir}/{sample_id}_5K_{celltype}_lognorm.h5ad')
print(f'{adata_dir}/{sample_id}_5K_{celltype}_lognorm.h5ad')
```

Example 51 (unknown):
```unknown
/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium/subsets/C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat_5K_EpicardialCells_lognorm.h5ad
```

---

## Set Up¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig4_Cardiomyocytes/AtrialCardiomyocytes/2_multivi_compute_latent_space.html

**Contents:**
- Set Up¶
- Read in¶
- Organise anndata¶
- Setup and training MultiVI (with continuous covariates)¶

5664 rows × 40 columns

544641 rows × 3 columns

MultiVI requires the main batch annotation to correspond to the modality of the samples. Other batch annotation, such as in the case of multiple RNA-only batches, can be specified using categorical_covariate_keys.

**Examples:**

Example 1 (python):
```python
job_name = None
n_layers = None
developing=None
paired_adata_path=None
unimodality_adata_path=None
multiome_only=None
combination=None
```

Example 2 (python):
```python
# Parameters
job_name = "aCMs_WholeSample"
n_layers = 1
developing = False
paired_adata_path = "/lustre/scratch126/cellgen/team205/heart/objects/fetal/RNA/aCMs_WholeSample_paired.h5ad"
unimodality_adata_path = "none"
combination = "multivi_aCMs_WholeSample_n_layers1"
```

Example 3 (python):
```python
import scvi
import anndata
import numpy as np
import pandas as pd
import scanpy as sc
import matplotlib.pyplot as plt
import os
```

Example 4 (unknown):
```unknown
Global seed set to 0
```

Example 5 (python):
```python
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm
```

Example 6 (python):
```python
import scanpy as sc

def all_integers_in_float_list(float_list):
    """Check if all values in a list of floats are integers."""
    return all(float_val.is_integer() for float_val in float_list)

def check_counts_layer(adata):
    """
    Check if the "counts" layer exists in the given AnnData object.
    
    Parameters:
    - adata (anndata.AnnData): The AnnData object to check.
    
    Returns:
    - bool: True if "counts" layer exists, False otherwise.
    """
    return "counts" in adata.layers.keys()

def makecountslayer(adata, apply_norm_log=True):
    """
    Ensure that the AnnData object has a "counts" layer and optionally log-normalize .X
    
    Parameters:
    - adata (anndata.AnnData): The AnnData object to be processed.
    - apply_norm_log (bool): Whether to apply normalization and log1p transformation. Default is True.
    
    Returns:
    - None: Modifies the AnnData object in place.
    """
    # Check if 'counts' layer exists
    if check_counts_layer(adata):
        print("Counts layer found, copying counts to .X...")
        # Copy data from 'counts' layer to .X
        adata.X = adata.layers["counts"].copy()
    else:
        print("No counts layer found, checking whether .X contains count data (integers)...")
        # Sample first 1000 elements from .X for integer check
        adata_X = adata.X.data[:1000].tolist()

        # Check if sampled data are all integers
        if all_integers_in_float_list(adata_X):
            print("Counts found in adata.X, saving them to a counts layer...")
            # Copy data from .X to 'counts' layer
            adata.layers["counts"] = adata.X.copy()
        else:
            raise ValueError("No count data found, cannot proceed")
    
    if apply_norm_log:
        print("Norm-logging .X...")
        # Normalize and log-transform data
        sc.pp.normalize_total(adata, target_sum=1e4)
        sc.pp.log1p(adata)
    else:
        print("NOT norm-logging")
```

Example 7 (python):
```python
from collections import Counter

def ensure_unique_var_names(adata):
    """
    Ensure that the variable names in an AnnData object are unique.
    
    Parameters:
    -----------
    adata : AnnData
        The AnnData object to be checked and processed.

    Returns:
    --------
    None
        Modifies the AnnData object in place if non-unique variable names are found.
    """
    
    # Find non-unique variable names
    var_names = list(adata.var_names)
    counter = Counter(var_names)
    non_unique_var_names = [name for name, count in counter.items() if count > 1]

    # If non-unique variable names are found, make them unique
    if non_unique_var_names:
        print("The following variable names are NOT unique:", non_unique_var_names)
        print("Running adata.var_names_make_unique()...")
        adata.var_names_make_unique()
    else:
        print("All variable names are unique.")
```

Example 8 (python):
```python
if unimodality_adata_path=="none":
    multiome_only=True
else:
    multiome_only=False
```

Example 9 (unknown):
```unknown
continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo','S_score','G2M_score']
```

Example 10 (unknown):
```unknown
model_dir=os.path.join("/lustre/scratch126/cellgen/team205/heart/objects/fetal/RNA","models") # for model
latent_variables_dir=os.path.join("/lustre/scratch126/cellgen/team205/heart/objects/fetal/RNA","latent_variables") # for latent variables
```

Example 11 (unknown):
```unknown
if job_name=="global":
    max_epochs=200
else:
    max_epochs=500
if developing:
    max_epochs=2
```

Example 12 (python):
```python
import session_info
session_info.show()
```

Example 13 (python):
```python
-----
anndata             0.8.0
matplotlib          3.7.2
numpy               1.24.4
pandas              1.5.3
scanpy              1.9.3
scvi                0.20.3
session_info        1.0.0
-----
```

Example 14 (unknown):
```unknown
PIL                         10.0.0
absl                        NA
anyio                       NA
arrow                       1.2.3
asttokens                   NA
attr                        23.1.0
attrs                       23.1.0
babel                       2.12.1
backcall                    0.2.0
brotli                      1.0.9
certifi                     2023.07.22
cffi                        1.15.1
charset_normalizer          3.2.0
chex                        0.1.7
comm                        0.1.3
contextlib2                 NA
cycler                      0.10.0
cython_runtime              NA
dateutil                    2.8.2
debugpy                     1.6.7
decorator                   5.1.1
defusedxml                  0.7.1
docrep                      0.3.2
etils                       1.4.0
executing                   1.2.0
fastjsonschema              NA
flax                        0.7.0
fqdn                        NA
fsspec                      2023.6.0
graphviz                    0.20.1
h5py                        3.9.0
idna                        3.4
igraph                      0.10.6
importlib_resources         NA
ipykernel                   6.25.0
isoduration                 NA
jax                         0.4.14
jaxlib                      0.4.14
jedi                        0.18.2
jinja2                      3.1.2
joblib                      1.3.1
json5                       NA
jsonpointer                 2.0
jsonschema                  4.18.4
jsonschema_specifications   NA
jupyter_events              0.6.3
jupyter_server              2.7.0
jupyterlab_server           2.24.0
kiwisolver                  1.4.4
leidenalg                   0.10.1
lightning_fabric            1.9.5
lightning_utilities         0.9.0
llvmlite                    0.40.1
markupsafe                  2.1.3
ml_collections              NA
ml_dtypes                   0.2.0
mpl_toolkits                NA
mpmath                      1.3.0
msgpack                     1.0.5
mudata                      0.2.3
multipledispatch            0.6.0
natsort                     8.4.0
nbformat                    5.9.1
numba                       0.57.1
numpyro                     0.12.1
nvfuser                     NA
opt_einsum                  v3.3.0
optax                       0.1.7
overrides                   NA
packaging                   23.1
parso                       0.8.3
pexpect                     4.8.0
pickleshare                 0.7.5
pkg_resources               NA
platformdirs                3.9.1
plotly                      5.16.1
prometheus_client           NA
prompt_toolkit              3.0.39
psutil                      5.9.5
ptyprocess                  0.7.0
pure_eval                   0.2.2
pyarrow                     12.0.1
pydev_ipython               NA
pydevconsole                NA
pydevd                      2.9.5
pydevd_file_utils           NA
pydevd_plugins              NA
pydevd_tracing              NA
pygments                    2.15.1
pyparsing                   3.0.9
pyro                        1.8.6
pythonjsonlogger            NA
pytorch_lightning           1.9.5
pytz                        2023.3
referencing                 NA
regex                       2.5.129
requests                    2.31.0
rfc3339_validator           0.1.4
rfc3986_validator           0.1.1
rich                        NA
rpds                        NA
scipy                       1.11.1
send2trash                  NA
setuptools                  68.0.0
six                         1.16.0
sklearn                     1.3.0
sniffio                     1.3.0
socks                       1.7.1
stack_data                  0.6.2
sympy                       1.12
texttable                   1.6.7
threadpoolctl               3.2.0
toolz                       0.12.0
torch                       2.0.1+cu117
torchmetrics                1.0.1
tornado                     6.3.2
tqdm                        4.65.0
traitlets                   5.9.0
tree                        0.1.8
typing_extensions           NA
uri_template                NA
urllib3                     1.26.16
wcwidth                     0.2.6
webcolors                   1.13
websocket                   1.6.1
yaml                        6.0
zmq                         25.1.0
```

Example 15 (unknown):
```unknown
-----
IPython             8.14.0
jupyter_client      8.3.0
jupyter_core        5.3.1
jupyterlab          4.0.3
notebook            7.0.0
-----
Python 3.11.4 | packaged by conda-forge | (main, Jun 10 2023, 18:08:17) [GCC 12.2.0]
Linux-5.4.0-159-generic-x86_64-with-glibc2.27
-----
Session information updated at 2023-10-19 16:46
```

Example 16 (python):
```python
sc.settings.verbosity = 3             # verbosity: errors (0), warnings (1), info (2), hints (3)
sc.settings.set_figure_params(dpi=50, facecolor='white', color_map = 'RdPu',figsize=(4, 4))
sc.set_figure_params(dpi=80, dpi_save=150, figsize=(6,6))
```

Example 17 (unknown):
```unknown
!nvidia-smi
```

Example 18 (unknown):
```unknown
Thu Oct 19 16:46:43 2023       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 510.108.03   Driver Version: 510.108.03   CUDA Version: 11.6     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
```

Example 19 (unknown):
```unknown
|   0  NVIDIA A100-SXM...  On   | 00000000:07:00.0 Off |                    0 |
| N/A   44C    P0   104W / 400W |  21381MiB / 81920MiB |     75%      Default |
|                               |                      |             Disabled |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
```

Example 20 (unknown):
```unknown
|    0   N/A  N/A    247245      C   python                          21379MiB |
+-----------------------------------------------------------------------------+
```

Example 21 (python):
```python
adata_paired = sc.read_h5ad(paired_adata_path)
print(adata_paired.shape)

if developing:
    adata_paired=adata_paired[::10, :] # slices to keep every Xth cell
    
print(adata_paired.shape)
adata_paired
```

Example 22 (unknown):
```unknown
(5664, 544641)
(5664, 544641)
```

Example 23 (unknown):
```unknown
AnnData object with n_obs × n_vars = 5664 × 544641
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id'
```

Example 24 (python):
```python
adata_paired.obs
```

Example 25 (python):
```python
adata_paired.var
```

Example 26 (python):
```python
sum(adata_paired.var['modality']=='Gene Expression')
```

Example 27 (python):
```python
ensure_unique_var_names(adata_paired)
```

Example 28 (unknown):
```unknown
All variable names are unique.
```

Example 29 (python):
```python
makecountslayer(adata_paired, apply_norm_log=False)
```

Example 30 (python):
```python
No counts layer found, checking whether .X contains count data (integers)...
Counts found in adata.X, saving them to a counts layer...
NOT norm-logging
```

Example 31 (python):
```python
if multiome_only:
    adata_mvi=scvi.data.organize_multiome_anndatas(multi_anndata=adata_paired)
    adata_mvi = adata_mvi[:, adata_mvi.var["modality"].argsort()].copy()
if multiome_only==False:
    unimodality_adata = sc.read_h5ad(unimodality_adata_path)
    print(unimodality_adata.shape)
    if developing:
        unimodality_adata=unimodality_adata[::10, :] # slices to keep every Xth cell
    ensure_unique_var_names(unimodality_adata)
    makecountslayer(unimodality_adata, apply_norm_log=False)
    adata_mvi=scvi.data.organize_multiome_anndatas(multi_anndata=adata_paired,rna_anndata=unimodality_adata)
    adata_mvi=adata_mvi[:, adata_mvi.var["modality"].argsort()].copy()
```

Example 32 (python):
```python
# We also filter features to remove those that appear in fewer than 1% of the cells
print(adata_mvi.shape)
sc.pp.filter_genes(adata_mvi, min_cells=int(adata_mvi.shape[0] * 0.01))
print(adata_mvi.shape)
```

Example 33 (unknown):
```unknown
(5664, 544641)
```

Example 34 (unknown):
```unknown
filtered out 364821 genes that are detected in less than 56 cells
```

Example 35 (unknown):
```unknown
(5664, 179820)
```

Example 36 (python):
```python
adata_mvi.obs['modality'].value_counts()
```

Example 37 (unknown):
```unknown
paired    5664
Name: modality, dtype: int64
```

Example 38 (python):
```python
adata_mvi.obs['batch_key'].value_counts()
```

Example 39 (unknown):
```unknown
Hst33_Nuclei_Multiome-v1    3925
C92_Nuclei_Multiome-v1       443
C85_Nuclei_Multiome-v1       380
Hst36_Nuclei_Multiome-v1     372
C87_Nuclei_Multiome-v1       321
C98_Nuclei_Multiome-v1       186
C82_Nuclei_Multiome-v1        37
Name: batch_key, dtype: int64
```

Example 40 (python):
```python
scvi.model.MULTIVI.setup_anndata(adata_mvi, batch_key='modality',
                                categorical_covariate_keys=['batch_key'],
                                continuous_covariate_keys=continuous_covariate_keys)
```

Example 41 (unknown):
```unknown
No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.)
```

Example 42 (python):
```python
# When creating the object, we need to specify how many of the features are genes, and how many are genomic regions. 
# This is so MultiVI can determine the exact architecture for each modality.
mvi = scvi.model.MULTIVI(
    adata_mvi,
    n_genes=(adata_mvi.var['modality']=='Gene Expression').sum(),
    n_regions=(adata_mvi.var['modality']=='Peaks').sum(),
    n_layers_encoder=n_layers,
    n_layers_decoder=n_layers,
)
mvi.view_anndata_setup()
```

Example 43 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/torch/nn/init.py:405: UserWarning: Initializing zero-element tensors is a no-op
  warnings.warn("Initializing zero-element tensors is a no-op")
```

Example 44 (unknown):
```unknown
Anndata setup with scvi-tools version 0.20.3.
```

Example 45 (unknown):
```unknown
Setup via `MULTIVI.setup_anndata` with arguments:
```

Example 46 (unknown):
```unknown
{
│   'layer': None,
│   'batch_key': 'modality',
│   'size_factor_key': None,
│   'categorical_covariate_keys': ['batch_key'],
│   'continuous_covariate_keys': ['total_counts', 'pct_counts_mt', 'pct_counts_ribo', 'S_score', 'G2M_score'],
│   'protein_expression_obsm_key': None,
│   'protein_names_uns_key': None
}
```

Example 47 (unknown):
```unknown
Summary Statistics          
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃     Summary Stat Key     ┃ Value  ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│         n_batch          │   1    │
│         n_cells          │  5664  │
│ n_extra_categorical_covs │   1    │
│ n_extra_continuous_covs  │   5    │
│         n_labels         │   1    │
│          n_vars          │ 179820 │
└──────────────────────────┴────────┘
```

Example 48 (python):
```python
Data Registry                             
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃      Registry Key      ┃            scvi-tools Location             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│           X            │                  adata.X                   │
│         batch          │          adata.obs['_scvi_batch']          │
│ extra_categorical_covs │ adata.obsm['_scvi_extra_categorical_covs'] │
│ extra_continuous_covs  │ adata.obsm['_scvi_extra_continuous_covs']  │
│         ind_x          │           adata.obs['_indices']            │
│         labels         │         adata.obs['_scvi_labels']          │
└────────────────────────┴────────────────────────────────────────────┘
```

Example 49 (python):
```python
batch State Registry                    
┏━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location    ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['modality'] │   paired   │          0          │
└───────────────────────┴────────────┴─────────────────────┘
```

Example 50 (python):
```python
labels State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 51 (python):
```python
batch State Registry                    
┏━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location    ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['modality'] │   paired   │          0          │
└───────────────────────┴────────────┴─────────────────────┘
```

Example 52 (python):
```python
extra_categorical_covs State Registry                   
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location     ┃        Categories        ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['batch_key'] │  C82_Nuclei_Multiome-v1  │          0          │
│                        │  C85_Nuclei_Multiome-v1  │          1          │
│                        │  C87_Nuclei_Multiome-v1  │          2          │
│                        │  C92_Nuclei_Multiome-v1  │          3          │
│                        │  C98_Nuclei_Multiome-v1  │          4          │
│                        │ Hst33_Nuclei_Multiome-v1 │          5          │
│                        │ Hst36_Nuclei_Multiome-v1 │          6          │
│                        │                          │                     │
└────────────────────────┴──────────────────────────┴─────────────────────┘
```

Example 53 (python):
```python
extra_continuous_covs State   
            Registry            
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃       Source Location        ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│  adata.obs['total_counts']   │
│  adata.obs['pct_counts_mt']  │
│ adata.obs['pct_counts_ribo'] │
│     adata.obs['S_score']     │
│    adata.obs['G2M_score']    │
└──────────────────────────────┘
```

Example 54 (unknown):
```unknown
mvi.train(max_epochs=max_epochs)
```

Example 55 (unknown):
```unknown
GPU available: True (cuda), used: True
```

Example 56 (unknown):
```unknown
TPU available: False, using: 0 TPU cores
```

Example 57 (unknown):
```unknown
IPU available: False, using: 0 IPUs
```

Example 58 (unknown):
```unknown
HPU available: False, using: 0 HPUs
```

Example 59 (unknown):
```unknown
You are using a CUDA device ('NVIDIA A100-SXM4-80GB') that has Tensor Cores. To properly utilize them, you should set `torch.set_float32_matmul_precision('medium' | 'high')` which will trade-off precision for performance. For more details, read https://pytorch.org/docs/stable/generated/torch.set_float32_matmul_precision.html#torch.set_float32_matmul_precision
```

Example 60 (unknown):
```unknown
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
```

Example 61 (unknown):
```unknown
Training:   0%|          | 0/500 [00:00<?, ?it/s]
```

Example 62 (unknown):
```unknown
Epoch 1/500:   0%|          | 0/500 [00:00<?, ?it/s]
```

Example 63 (unknown):
```unknown
Epoch 1/500:   0%|          | 1/500 [00:17<2:22:02, 17.08s/it]
```

Example 64 (unknown):
```unknown
Epoch 1/500:   0%|          | 1/500 [00:17<2:22:02, 17.08s/it, loss=2.18e+04, v_num=1]
```

Example 65 (unknown):
```unknown
Epoch 2/500:   0%|          | 1/500 [00:17<2:22:02, 17.08s/it, loss=2.18e+04, v_num=1]
```

Example 66 (unknown):
```unknown
Epoch 2/500:   0%|          | 2/500 [00:24<1:32:11, 11.11s/it, loss=2.18e+04, v_num=1]
```

Example 67 (unknown):
```unknown
Epoch 2/500:   0%|          | 2/500 [00:24<1:32:11, 11.11s/it, loss=1.87e+04, v_num=1]
```

Example 68 (unknown):
```unknown
Epoch 3/500:   0%|          | 2/500 [00:24<1:32:11, 11.11s/it, loss=1.87e+04, v_num=1]
```

Example 69 (unknown):
```unknown
Epoch 3/500:   1%|          | 3/500 [00:31<1:16:30,  9.24s/it, loss=1.87e+04, v_num=1]
```

Example 70 (unknown):
```unknown
Epoch 3/500:   1%|          | 3/500 [00:31<1:16:30,  9.24s/it, loss=1.7e+04, v_num=1]
```

Example 71 (unknown):
```unknown
Epoch 4/500:   1%|          | 3/500 [00:31<1:16:30,  9.24s/it, loss=1.7e+04, v_num=1]
```

Example 72 (unknown):
```unknown
Epoch 4/500:   1%|          | 4/500 [00:38<1:08:59,  8.35s/it, loss=1.7e+04, v_num=1]
```

Example 73 (unknown):
```unknown
Epoch 4/500:   1%|          | 4/500 [00:38<1:08:59,  8.35s/it, loss=1.65e+04, v_num=1]
```

Example 74 (unknown):
```unknown
Epoch 5/500:   1%|          | 4/500 [00:38<1:08:59,  8.35s/it, loss=1.65e+04, v_num=1]
```

Example 75 (unknown):
```unknown
Epoch 5/500:   1%|          | 5/500 [00:44<1:04:07,  7.77s/it, loss=1.65e+04, v_num=1]
```

Example 76 (unknown):
```unknown
Epoch 5/500:   1%|          | 5/500 [00:44<1:04:07,  7.77s/it, loss=1.62e+04, v_num=1]
```

Example 77 (unknown):
```unknown
Epoch 6/500:   1%|          | 5/500 [00:44<1:04:07,  7.77s/it, loss=1.62e+04, v_num=1]
```

Example 78 (unknown):
```unknown
Epoch 6/500:   1%|          | 6/500 [00:51<1:00:48,  7.39s/it, loss=1.62e+04, v_num=1]
```

Example 79 (unknown):
```unknown
Epoch 6/500:   1%|          | 6/500 [00:51<1:00:48,  7.39s/it, loss=1.58e+04, v_num=1]
```

Example 80 (unknown):
```unknown
Epoch 7/500:   1%|          | 6/500 [00:51<1:00:48,  7.39s/it, loss=1.58e+04, v_num=1]
```

Example 81 (unknown):
```unknown
Epoch 7/500:   1%|▏         | 7/500 [00:58<59:08,  7.20s/it, loss=1.58e+04, v_num=1]
```

Example 82 (unknown):
```unknown
Epoch 7/500:   1%|▏         | 7/500 [00:58<59:08,  7.20s/it, loss=1.59e+04, v_num=1]
```

Example 83 (unknown):
```unknown
Epoch 8/500:   1%|▏         | 7/500 [00:58<59:08,  7.20s/it, loss=1.59e+04, v_num=1]
```

Example 84 (unknown):
```unknown
Epoch 8/500:   2%|▏         | 8/500 [01:05<58:21,  7.12s/it, loss=1.59e+04, v_num=1]
```

Example 85 (unknown):
```unknown
Epoch 8/500:   2%|▏         | 8/500 [01:05<58:21,  7.12s/it, loss=1.58e+04, v_num=1]
```

Example 86 (unknown):
```unknown
Epoch 9/500:   2%|▏         | 8/500 [01:05<58:21,  7.12s/it, loss=1.58e+04, v_num=1]
```

Example 87 (unknown):
```unknown
Epoch 9/500:   2%|▏         | 9/500 [01:11<57:03,  6.97s/it, loss=1.58e+04, v_num=1]
```

Example 88 (unknown):
```unknown
Epoch 9/500:   2%|▏         | 9/500 [01:11<57:03,  6.97s/it, loss=1.55e+04, v_num=1]
```

Example 89 (unknown):
```unknown
Epoch 10/500:   2%|▏         | 9/500 [01:11<57:03,  6.97s/it, loss=1.55e+04, v_num=1]
```

Example 90 (unknown):
```unknown
Epoch 10/500:   2%|▏         | 10/500 [01:18<56:49,  6.96s/it, loss=1.55e+04, v_num=1]
```

Example 91 (unknown):
```unknown
Epoch 10/500:   2%|▏         | 10/500 [01:18<56:49,  6.96s/it, loss=1.55e+04, v_num=1]
```

Example 92 (unknown):
```unknown
Epoch 11/500:   2%|▏         | 10/500 [01:18<56:49,  6.96s/it, loss=1.55e+04, v_num=1]
```

Example 93 (unknown):
```unknown
Epoch 11/500:   2%|▏         | 11/500 [01:25<56:10,  6.89s/it, loss=1.55e+04, v_num=1]
```

Example 94 (unknown):
```unknown
Epoch 11/500:   2%|▏         | 11/500 [01:25<56:10,  6.89s/it, loss=1.53e+04, v_num=1]
```

Example 95 (unknown):
```unknown
Epoch 12/500:   2%|▏         | 11/500 [01:25<56:10,  6.89s/it, loss=1.53e+04, v_num=1]
```

Example 96 (unknown):
```unknown
Epoch 12/500:   2%|▏         | 12/500 [01:32<56:16,  6.92s/it, loss=1.53e+04, v_num=1]
```

Example 97 (unknown):
```unknown
Epoch 12/500:   2%|▏         | 12/500 [01:32<56:16,  6.92s/it, loss=1.57e+04, v_num=1]
```

Example 98 (unknown):
```unknown
Epoch 13/500:   2%|▏         | 12/500 [01:32<56:16,  6.92s/it, loss=1.57e+04, v_num=1]
```

Example 99 (unknown):
```unknown
Epoch 13/500:   3%|▎         | 13/500 [01:39<56:19,  6.94s/it, loss=1.57e+04, v_num=1]
```

Example 100 (unknown):
```unknown
Epoch 13/500:   3%|▎         | 13/500 [01:39<56:19,  6.94s/it, loss=1.55e+04, v_num=1]
```

Example 101 (unknown):
```unknown
Epoch 14/500:   3%|▎         | 13/500 [01:39<56:19,  6.94s/it, loss=1.55e+04, v_num=1]
```

Example 102 (unknown):
```unknown
Epoch 14/500:   3%|▎         | 14/500 [01:46<56:15,  6.95s/it, loss=1.55e+04, v_num=1]
```

Example 103 (unknown):
```unknown
Epoch 14/500:   3%|▎         | 14/500 [01:46<56:15,  6.95s/it, loss=1.53e+04, v_num=1]
```

Example 104 (unknown):
```unknown
Epoch 15/500:   3%|▎         | 14/500 [01:46<56:15,  6.95s/it, loss=1.53e+04, v_num=1]
```

Example 105 (unknown):
```unknown
Epoch 15/500:   3%|▎         | 15/500 [01:53<56:12,  6.95s/it, loss=1.53e+04, v_num=1]
```

Example 106 (unknown):
```unknown
Epoch 15/500:   3%|▎         | 15/500 [01:53<56:12,  6.95s/it, loss=1.55e+04, v_num=1]
```

Example 107 (unknown):
```unknown
Epoch 16/500:   3%|▎         | 15/500 [01:53<56:12,  6.95s/it, loss=1.55e+04, v_num=1]
```

Example 108 (unknown):
```unknown
Epoch 16/500:   3%|▎         | 16/500 [02:00<56:05,  6.95s/it, loss=1.55e+04, v_num=1]
```

Example 109 (unknown):
```unknown
Epoch 16/500:   3%|▎         | 16/500 [02:00<56:05,  6.95s/it, loss=1.53e+04, v_num=1]
```

Example 110 (unknown):
```unknown
Epoch 17/500:   3%|▎         | 16/500 [02:00<56:05,  6.95s/it, loss=1.53e+04, v_num=1]
```

Example 111 (unknown):
```unknown
Epoch 17/500:   3%|▎         | 17/500 [02:07<55:57,  6.95s/it, loss=1.53e+04, v_num=1]
```

Example 112 (unknown):
```unknown
Epoch 17/500:   3%|▎         | 17/500 [02:07<55:57,  6.95s/it, loss=1.53e+04, v_num=1]
```

Example 113 (unknown):
```unknown
Epoch 18/500:   3%|▎         | 17/500 [02:07<55:57,  6.95s/it, loss=1.53e+04, v_num=1]
```

Example 114 (unknown):
```unknown
Epoch 18/500:   4%|▎         | 18/500 [02:14<55:49,  6.95s/it, loss=1.53e+04, v_num=1]
```

Example 115 (unknown):
```unknown
Epoch 18/500:   4%|▎         | 18/500 [02:14<55:49,  6.95s/it, loss=1.55e+04, v_num=1]
```

Example 116 (unknown):
```unknown
Epoch 19/500:   4%|▎         | 18/500 [02:14<55:49,  6.95s/it, loss=1.55e+04, v_num=1]
```

Example 117 (unknown):
```unknown
Epoch 19/500:   4%|▍         | 19/500 [02:20<55:17,  6.90s/it, loss=1.55e+04, v_num=1]
```

Example 118 (unknown):
```unknown
Epoch 19/500:   4%|▍         | 19/500 [02:20<55:17,  6.90s/it, loss=1.53e+04, v_num=1]
```

Example 119 (unknown):
```unknown
Epoch 20/500:   4%|▍         | 19/500 [02:20<55:17,  6.90s/it, loss=1.53e+04, v_num=1]
```

Example 120 (unknown):
```unknown
Epoch 20/500:   4%|▍         | 20/500 [02:27<55:16,  6.91s/it, loss=1.53e+04, v_num=1]
```

Example 121 (unknown):
```unknown
Epoch 20/500:   4%|▍         | 20/500 [02:27<55:16,  6.91s/it, loss=1.55e+04, v_num=1]
```

Example 122 (unknown):
```unknown
Epoch 21/500:   4%|▍         | 20/500 [02:27<55:16,  6.91s/it, loss=1.55e+04, v_num=1]
```

Example 123 (unknown):
```unknown
Epoch 21/500:   4%|▍         | 21/500 [02:34<54:41,  6.85s/it, loss=1.55e+04, v_num=1]
```

Example 124 (unknown):
```unknown
Epoch 21/500:   4%|▍         | 21/500 [02:34<54:41,  6.85s/it, loss=1.55e+04, v_num=1]
```

Example 125 (unknown):
```unknown
Epoch 22/500:   4%|▍         | 21/500 [02:34<54:41,  6.85s/it, loss=1.55e+04, v_num=1]
```

Example 126 (unknown):
```unknown
Epoch 22/500:   4%|▍         | 22/500 [02:41<54:20,  6.82s/it, loss=1.55e+04, v_num=1]
```

Example 127 (unknown):
```unknown
Epoch 22/500:   4%|▍         | 22/500 [02:41<54:20,  6.82s/it, loss=1.52e+04, v_num=1]
```

Example 128 (unknown):
```unknown
Epoch 23/500:   4%|▍         | 22/500 [02:41<54:20,  6.82s/it, loss=1.52e+04, v_num=1]
```

Example 129 (unknown):
```unknown
Epoch 23/500:   5%|▍         | 23/500 [02:48<54:26,  6.85s/it, loss=1.52e+04, v_num=1]
```

Example 130 (unknown):
```unknown
Epoch 23/500:   5%|▍         | 23/500 [02:48<54:26,  6.85s/it, loss=1.52e+04, v_num=1]
```

Example 131 (unknown):
```unknown
Epoch 24/500:   5%|▍         | 23/500 [02:48<54:26,  6.85s/it, loss=1.52e+04, v_num=1]
```

Example 132 (unknown):
```unknown
Epoch 24/500:   5%|▍         | 24/500 [02:54<53:41,  6.77s/it, loss=1.52e+04, v_num=1]
```

Example 133 (unknown):
```unknown
Epoch 24/500:   5%|▍         | 24/500 [02:54<53:41,  6.77s/it, loss=1.52e+04, v_num=1]
```

Example 134 (unknown):
```unknown
Epoch 25/500:   5%|▍         | 24/500 [02:54<53:41,  6.77s/it, loss=1.52e+04, v_num=1]
```

Example 135 (unknown):
```unknown
Epoch 25/500:   5%|▌         | 25/500 [03:01<53:52,  6.80s/it, loss=1.52e+04, v_num=1]
```

Example 136 (unknown):
```unknown
Epoch 25/500:   5%|▌         | 25/500 [03:01<53:52,  6.80s/it, loss=1.53e+04, v_num=1]
```

Example 137 (unknown):
```unknown
Epoch 26/500:   5%|▌         | 25/500 [03:01<53:52,  6.80s/it, loss=1.53e+04, v_num=1]
```

Example 138 (unknown):
```unknown
Epoch 26/500:   5%|▌         | 26/500 [03:08<53:36,  6.79s/it, loss=1.53e+04, v_num=1]
```

Example 139 (unknown):
```unknown
Epoch 26/500:   5%|▌         | 26/500 [03:08<53:36,  6.79s/it, loss=1.53e+04, v_num=1]
```

Example 140 (unknown):
```unknown
Epoch 27/500:   5%|▌         | 26/500 [03:08<53:36,  6.79s/it, loss=1.53e+04, v_num=1]
```

Example 141 (unknown):
```unknown
Epoch 27/500:   5%|▌         | 27/500 [03:15<53:54,  6.84s/it, loss=1.53e+04, v_num=1]
```

Example 142 (unknown):
```unknown
Epoch 27/500:   5%|▌         | 27/500 [03:15<53:54,  6.84s/it, loss=1.57e+04, v_num=1]
```

Example 143 (unknown):
```unknown
Epoch 28/500:   5%|▌         | 27/500 [03:15<53:54,  6.84s/it, loss=1.57e+04, v_num=1]
```

Example 144 (unknown):
```unknown
Epoch 28/500:   6%|▌         | 28/500 [03:22<54:03,  6.87s/it, loss=1.57e+04, v_num=1]
```

Example 145 (unknown):
```unknown
Epoch 28/500:   6%|▌         | 28/500 [03:22<54:03,  6.87s/it, loss=1.57e+04, v_num=1]
```

Example 146 (unknown):
```unknown
Epoch 29/500:   6%|▌         | 28/500 [03:22<54:03,  6.87s/it, loss=1.57e+04, v_num=1]
```

Example 147 (unknown):
```unknown
Epoch 29/500:   6%|▌         | 29/500 [03:29<54:12,  6.91s/it, loss=1.57e+04, v_num=1]
```

Example 148 (unknown):
```unknown
Epoch 29/500:   6%|▌         | 29/500 [03:29<54:12,  6.91s/it, loss=1.53e+04, v_num=1]
```

Example 149 (unknown):
```unknown
Epoch 30/500:   6%|▌         | 29/500 [03:29<54:12,  6.91s/it, loss=1.53e+04, v_num=1]
```

Example 150 (unknown):
```unknown
Epoch 30/500:   6%|▌         | 30/500 [03:36<54:11,  6.92s/it, loss=1.53e+04, v_num=1]
```

Example 151 (unknown):
```unknown
Epoch 30/500:   6%|▌         | 30/500 [03:36<54:11,  6.92s/it, loss=1.5e+04, v_num=1]
```

Example 152 (unknown):
```unknown
Epoch 31/500:   6%|▌         | 30/500 [03:36<54:11,  6.92s/it, loss=1.5e+04, v_num=1]
```

Example 153 (unknown):
```unknown
Epoch 31/500:   6%|▌         | 31/500 [03:43<53:31,  6.85s/it, loss=1.5e+04, v_num=1]
```

Example 154 (unknown):
```unknown
Epoch 31/500:   6%|▌         | 31/500 [03:43<53:31,  6.85s/it, loss=1.52e+04, v_num=1]
```

Example 155 (unknown):
```unknown
Epoch 32/500:   6%|▌         | 31/500 [03:43<53:31,  6.85s/it, loss=1.52e+04, v_num=1]
```

Example 156 (unknown):
```unknown
Epoch 32/500:   6%|▋         | 32/500 [03:49<53:22,  6.84s/it, loss=1.52e+04, v_num=1]
```

Example 157 (unknown):
```unknown
Epoch 32/500:   6%|▋         | 32/500 [03:49<53:22,  6.84s/it, loss=1.51e+04, v_num=1]
```

Example 158 (unknown):
```unknown
Epoch 33/500:   6%|▋         | 32/500 [03:49<53:22,  6.84s/it, loss=1.51e+04, v_num=1]
```

Example 159 (unknown):
```unknown
Epoch 33/500:   7%|▋         | 33/500 [03:56<53:03,  6.82s/it, loss=1.51e+04, v_num=1]
```

Example 160 (unknown):
```unknown
Epoch 33/500:   7%|▋         | 33/500 [03:56<53:03,  6.82s/it, loss=1.56e+04, v_num=1]
```

Example 161 (unknown):
```unknown
Epoch 34/500:   7%|▋         | 33/500 [03:56<53:03,  6.82s/it, loss=1.56e+04, v_num=1]
```

Example 162 (unknown):
```unknown
Epoch 34/500:   7%|▋         | 34/500 [04:03<52:46,  6.79s/it, loss=1.56e+04, v_num=1]
```

Example 163 (unknown):
```unknown
Epoch 34/500:   7%|▋         | 34/500 [04:03<52:46,  6.79s/it, loss=1.58e+04, v_num=1]
```

Example 164 (unknown):
```unknown
Epoch 35/500:   7%|▋         | 34/500 [04:03<52:46,  6.79s/it, loss=1.58e+04, v_num=1]
```

Example 165 (unknown):
```unknown
Epoch 35/500:   7%|▋         | 35/500 [04:10<52:30,  6.78s/it, loss=1.58e+04, v_num=1]
```

Example 166 (unknown):
```unknown
Epoch 35/500:   7%|▋         | 35/500 [04:10<52:30,  6.78s/it, loss=1.57e+04, v_num=1]
```

Example 167 (unknown):
```unknown
Epoch 36/500:   7%|▋         | 35/500 [04:10<52:30,  6.78s/it, loss=1.57e+04, v_num=1]
```

Example 168 (unknown):
```unknown
Epoch 36/500:   7%|▋         | 36/500 [04:17<52:50,  6.83s/it, loss=1.57e+04, v_num=1]
```

Example 169 (unknown):
```unknown
Epoch 36/500:   7%|▋         | 36/500 [04:17<52:50,  6.83s/it, loss=1.54e+04, v_num=1]
```

Example 170 (unknown):
```unknown
Epoch 37/500:   7%|▋         | 36/500 [04:17<52:50,  6.83s/it, loss=1.54e+04, v_num=1]
```

Example 171 (unknown):
```unknown
Epoch 37/500:   7%|▋         | 37/500 [04:23<52:20,  6.78s/it, loss=1.54e+04, v_num=1]
```

Example 172 (unknown):
```unknown
Epoch 37/500:   7%|▋         | 37/500 [04:23<52:20,  6.78s/it, loss=1.52e+04, v_num=1]
```

Example 173 (unknown):
```unknown
Epoch 38/500:   7%|▋         | 37/500 [04:23<52:20,  6.78s/it, loss=1.52e+04, v_num=1]
```

Example 174 (unknown):
```unknown
Epoch 38/500:   8%|▊         | 38/500 [04:30<51:51,  6.74s/it, loss=1.52e+04, v_num=1]
```

Example 175 (unknown):
```unknown
Epoch 38/500:   8%|▊         | 38/500 [04:30<51:51,  6.74s/it, loss=1.53e+04, v_num=1]
```

Example 176 (unknown):
```unknown
Epoch 39/500:   8%|▊         | 38/500 [04:30<51:51,  6.74s/it, loss=1.53e+04, v_num=1]
```

Example 177 (unknown):
```unknown
Epoch 39/500:   8%|▊         | 39/500 [04:37<51:50,  6.75s/it, loss=1.53e+04, v_num=1]
```

Example 178 (unknown):
```unknown
Epoch 39/500:   8%|▊         | 39/500 [04:37<51:50,  6.75s/it, loss=1.55e+04, v_num=1]
```

Example 179 (unknown):
```unknown
Epoch 40/500:   8%|▊         | 39/500 [04:37<51:50,  6.75s/it, loss=1.55e+04, v_num=1]
```

Example 180 (unknown):
```unknown
Epoch 40/500:   8%|▊         | 40/500 [04:43<51:42,  6.74s/it, loss=1.55e+04, v_num=1]
```

Example 181 (unknown):
```unknown
Epoch 40/500:   8%|▊         | 40/500 [04:43<51:42,  6.74s/it, loss=1.52e+04, v_num=1]
```

Example 182 (unknown):
```unknown
Epoch 41/500:   8%|▊         | 40/500 [04:43<51:42,  6.74s/it, loss=1.52e+04, v_num=1]
```

Example 183 (unknown):
```unknown
Epoch 41/500:   8%|▊         | 41/500 [04:50<52:01,  6.80s/it, loss=1.52e+04, v_num=1]
```

Example 184 (unknown):
```unknown
Epoch 41/500:   8%|▊         | 41/500 [04:50<52:01,  6.80s/it, loss=1.54e+04, v_num=1]
```

Example 185 (unknown):
```unknown
Epoch 42/500:   8%|▊         | 41/500 [04:50<52:01,  6.80s/it, loss=1.54e+04, v_num=1]
```

Example 186 (unknown):
```unknown
Epoch 42/500:   8%|▊         | 42/500 [04:57<52:18,  6.85s/it, loss=1.54e+04, v_num=1]
```

Example 187 (unknown):
```unknown
Epoch 42/500:   8%|▊         | 42/500 [04:57<52:18,  6.85s/it, loss=1.53e+04, v_num=1]
```

Example 188 (unknown):
```unknown
Epoch 43/500:   8%|▊         | 42/500 [04:57<52:18,  6.85s/it, loss=1.53e+04, v_num=1]
```

Example 189 (unknown):
```unknown
Epoch 43/500:   9%|▊         | 43/500 [05:04<52:36,  6.91s/it, loss=1.53e+04, v_num=1]
```

Example 190 (unknown):
```unknown
Epoch 43/500:   9%|▊         | 43/500 [05:04<52:36,  6.91s/it, loss=1.51e+04, v_num=1]
```

Example 191 (unknown):
```unknown
Epoch 44/500:   9%|▊         | 43/500 [05:04<52:36,  6.91s/it, loss=1.51e+04, v_num=1]
```

Example 192 (unknown):
```unknown
Epoch 44/500:   9%|▉         | 44/500 [05:11<52:33,  6.92s/it, loss=1.51e+04, v_num=1]
```

Example 193 (unknown):
```unknown
Epoch 44/500:   9%|▉         | 44/500 [05:11<52:33,  6.92s/it, loss=1.52e+04, v_num=1]
```

Example 194 (unknown):
```unknown
Epoch 45/500:   9%|▉         | 44/500 [05:11<52:33,  6.92s/it, loss=1.52e+04, v_num=1]
```

Example 195 (unknown):
```unknown
Epoch 45/500:   9%|▉         | 45/500 [05:18<52:33,  6.93s/it, loss=1.52e+04, v_num=1]
```

Example 196 (unknown):
```unknown
Epoch 45/500:   9%|▉         | 45/500 [05:18<52:33,  6.93s/it, loss=1.54e+04, v_num=1]
```

Example 197 (unknown):
```unknown
Epoch 46/500:   9%|▉         | 45/500 [05:18<52:33,  6.93s/it, loss=1.54e+04, v_num=1]
```

Example 198 (unknown):
```unknown
Epoch 46/500:   9%|▉         | 46/500 [05:25<52:17,  6.91s/it, loss=1.54e+04, v_num=1]
```

Example 199 (unknown):
```unknown
Epoch 46/500:   9%|▉         | 46/500 [05:25<52:17,  6.91s/it, loss=1.55e+04, v_num=1]
```

Example 200 (unknown):
```unknown
Epoch 47/500:   9%|▉         | 46/500 [05:25<52:17,  6.91s/it, loss=1.55e+04, v_num=1]
```

Example 201 (unknown):
```unknown
Epoch 47/500:   9%|▉         | 47/500 [05:32<52:17,  6.93s/it, loss=1.55e+04, v_num=1]
```

Example 202 (unknown):
```unknown
Epoch 47/500:   9%|▉         | 47/500 [05:32<52:17,  6.93s/it, loss=1.54e+04, v_num=1]
```

Example 203 (unknown):
```unknown
Epoch 48/500:   9%|▉         | 47/500 [05:32<52:17,  6.93s/it, loss=1.54e+04, v_num=1]
```

Example 204 (unknown):
```unknown
Epoch 48/500:  10%|▉         | 48/500 [05:39<52:03,  6.91s/it, loss=1.54e+04, v_num=1]
```

Example 205 (unknown):
```unknown
Epoch 48/500:  10%|▉         | 48/500 [05:39<52:03,  6.91s/it, loss=1.54e+04, v_num=1]
```

Example 206 (unknown):
```unknown
Epoch 49/500:  10%|▉         | 48/500 [05:39<52:03,  6.91s/it, loss=1.54e+04, v_num=1]
```

Example 207 (unknown):
```unknown
Epoch 49/500:  10%|▉         | 49/500 [05:46<51:31,  6.85s/it, loss=1.54e+04, v_num=1]
```

Example 208 (unknown):
```unknown
Epoch 49/500:  10%|▉         | 49/500 [05:46<51:31,  6.85s/it, loss=1.53e+04, v_num=1]
```

Example 209 (unknown):
```unknown
Epoch 50/500:  10%|▉         | 49/500 [05:46<51:31,  6.85s/it, loss=1.53e+04, v_num=1]
```

Example 210 (unknown):
```unknown
Epoch 50/500:  10%|█         | 50/500 [05:53<51:32,  6.87s/it, loss=1.53e+04, v_num=1]
```

Example 211 (unknown):
```unknown
Epoch 50/500:  10%|█         | 50/500 [05:53<51:32,  6.87s/it, loss=1.54e+04, v_num=1]
```

Example 212 (unknown):
```unknown
Epoch 51/500:  10%|█         | 50/500 [05:53<51:32,  6.87s/it, loss=1.54e+04, v_num=1]
```

Example 213 (unknown):
```unknown
Epoch 51/500:  10%|█         | 51/500 [05:59<51:35,  6.89s/it, loss=1.54e+04, v_num=1]
```

Example 214 (unknown):
```unknown
Epoch 51/500:  10%|█         | 51/500 [06:00<51:35,  6.89s/it, loss=1.53e+04, v_num=1]
```

Example 215 (unknown):
```unknown
Epoch 52/500:  10%|█         | 51/500 [06:00<51:35,  6.89s/it, loss=1.53e+04, v_num=1]
```

Example 216 (unknown):
```unknown
Epoch 52/500:  10%|█         | 52/500 [06:06<51:39,  6.92s/it, loss=1.53e+04, v_num=1]
```

Example 217 (unknown):
```unknown
Epoch 52/500:  10%|█         | 52/500 [06:06<51:39,  6.92s/it, loss=1.54e+04, v_num=1]
```

Example 218 (unknown):
```unknown
Epoch 53/500:  10%|█         | 52/500 [06:06<51:39,  6.92s/it, loss=1.54e+04, v_num=1]
```

Example 219 (unknown):
```unknown
Epoch 53/500:  11%|█         | 53/500 [06:13<50:53,  6.83s/it, loss=1.54e+04, v_num=1]
```

Example 220 (unknown):
```unknown
Epoch 53/500:  11%|█         | 53/500 [06:13<50:53,  6.83s/it, loss=1.51e+04, v_num=1]
```

Example 221 (unknown):
```unknown
Epoch 54/500:  11%|█         | 53/500 [06:13<50:53,  6.83s/it, loss=1.51e+04, v_num=1]
```

Example 222 (unknown):
```unknown
Epoch 54/500:  11%|█         | 54/500 [06:20<50:20,  6.77s/it, loss=1.51e+04, v_num=1]
```

Example 223 (unknown):
```unknown
Epoch 54/500:  11%|█         | 54/500 [06:20<50:20,  6.77s/it, loss=1.54e+04, v_num=1]
```

Example 224 (unknown):
```unknown
Epoch 55/500:  11%|█         | 54/500 [06:20<50:20,  6.77s/it, loss=1.54e+04, v_num=1]
```

Example 225 (unknown):
```unknown
Epoch 55/500:  11%|█         | 55/500 [06:27<50:13,  6.77s/it, loss=1.54e+04, v_num=1]
```

Example 226 (unknown):
```unknown
Epoch 55/500:  11%|█         | 55/500 [06:27<50:13,  6.77s/it, loss=1.55e+04, v_num=1]
```

Example 227 (unknown):
```unknown
Epoch 56/500:  11%|█         | 55/500 [06:27<50:13,  6.77s/it, loss=1.55e+04, v_num=1]
```

Example 228 (unknown):
```unknown
Epoch 56/500:  11%|█         | 56/500 [06:33<50:14,  6.79s/it, loss=1.55e+04, v_num=1]
```

Example 229 (unknown):
```unknown
Epoch 56/500:  11%|█         | 56/500 [06:33<50:14,  6.79s/it, loss=1.52e+04, v_num=1]
```

Example 230 (unknown):
```unknown
Epoch 57/500:  11%|█         | 56/500 [06:33<50:14,  6.79s/it, loss=1.52e+04, v_num=1]
```

Example 231 (unknown):
```unknown
Epoch 57/500:  11%|█▏        | 57/500 [06:40<50:39,  6.86s/it, loss=1.52e+04, v_num=1]
```

Example 232 (unknown):
```unknown
Epoch 57/500:  11%|█▏        | 57/500 [06:40<50:39,  6.86s/it, loss=1.52e+04, v_num=1]
```

Example 233 (unknown):
```unknown
Epoch 58/500:  11%|█▏        | 57/500 [06:40<50:39,  6.86s/it, loss=1.52e+04, v_num=1]
```

Example 234 (unknown):
```unknown
Epoch 58/500:  12%|█▏        | 58/500 [06:47<50:46,  6.89s/it, loss=1.52e+04, v_num=1]
```

Example 235 (unknown):
```unknown
Epoch 58/500:  12%|█▏        | 58/500 [06:47<50:46,  6.89s/it, loss=1.56e+04, v_num=1]
```

Example 236 (unknown):
```unknown
Epoch 59/500:  12%|█▏        | 58/500 [06:47<50:46,  6.89s/it, loss=1.56e+04, v_num=1]
```

Example 237 (unknown):
```unknown
Epoch 59/500:  12%|█▏        | 59/500 [06:54<49:50,  6.78s/it, loss=1.56e+04, v_num=1]
```

Example 238 (unknown):
```unknown
Epoch 59/500:  12%|█▏        | 59/500 [06:54<49:50,  6.78s/it, loss=1.57e+04, v_num=1]
```

Example 239 (unknown):
```unknown
Epoch 60/500:  12%|█▏        | 59/500 [06:54<49:50,  6.78s/it, loss=1.57e+04, v_num=1]
```

Example 240 (unknown):
```unknown
Epoch 60/500:  12%|█▏        | 60/500 [07:01<50:12,  6.85s/it, loss=1.57e+04, v_num=1]
```

Example 241 (unknown):
```unknown
Epoch 60/500:  12%|█▏        | 60/500 [07:01<50:12,  6.85s/it, loss=1.54e+04, v_num=1]
```

Example 242 (unknown):
```unknown
Epoch 61/500:  12%|█▏        | 60/500 [07:01<50:12,  6.85s/it, loss=1.54e+04, v_num=1]
```

Example 243 (unknown):
```unknown
Epoch 61/500:  12%|█▏        | 61/500 [07:08<50:16,  6.87s/it, loss=1.54e+04, v_num=1]
```

Example 244 (unknown):
```unknown
Epoch 61/500:  12%|█▏        | 61/500 [07:08<50:16,  6.87s/it, loss=1.52e+04, v_num=1]
```

Example 245 (unknown):
```unknown
Epoch 62/500:  12%|█▏        | 61/500 [07:08<50:16,  6.87s/it, loss=1.52e+04, v_num=1]
```

Example 246 (unknown):
```unknown
Epoch 62/500:  12%|█▏        | 62/500 [07:15<49:53,  6.84s/it, loss=1.52e+04, v_num=1]
```

Example 247 (unknown):
```unknown
Epoch 62/500:  12%|█▏        | 62/500 [07:15<49:53,  6.84s/it, loss=1.52e+04, v_num=1]
```

Example 248 (unknown):
```unknown
Epoch 63/500:  12%|█▏        | 62/500 [07:15<49:53,  6.84s/it, loss=1.52e+04, v_num=1]
```

Example 249 (unknown):
```unknown
Epoch 63/500:  13%|█▎        | 63/500 [07:22<50:12,  6.89s/it, loss=1.52e+04, v_num=1]
```

Example 250 (unknown):
```unknown
Epoch 63/500:  13%|█▎        | 63/500 [07:22<50:12,  6.89s/it, loss=1.54e+04, v_num=1]
```

Example 251 (unknown):
```unknown
Epoch 64/500:  13%|█▎        | 63/500 [07:22<50:12,  6.89s/it, loss=1.54e+04, v_num=1]
```

Example 252 (unknown):
```unknown
Epoch 64/500:  13%|█▎        | 64/500 [07:28<49:46,  6.85s/it, loss=1.54e+04, v_num=1]
```

Example 253 (unknown):
```unknown
Epoch 64/500:  13%|█▎        | 64/500 [07:28<49:46,  6.85s/it, loss=1.54e+04, v_num=1]
```

Example 254 (unknown):
```unknown
Epoch 65/500:  13%|█▎        | 64/500 [07:28<49:46,  6.85s/it, loss=1.54e+04, v_num=1]
```

Example 255 (unknown):
```unknown
Epoch 65/500:  13%|█▎        | 65/500 [07:35<49:37,  6.84s/it, loss=1.54e+04, v_num=1]
```

Example 256 (unknown):
```unknown
Epoch 65/500:  13%|█▎        | 65/500 [07:35<49:37,  6.84s/it, loss=1.56e+04, v_num=1]
```

Example 257 (unknown):
```unknown
Epoch 66/500:  13%|█▎        | 65/500 [07:35<49:37,  6.84s/it, loss=1.56e+04, v_num=1]
```

Example 258 (unknown):
```unknown
Epoch 66/500:  13%|█▎        | 66/500 [07:42<49:48,  6.88s/it, loss=1.56e+04, v_num=1]
```

Example 259 (unknown):
```unknown
Epoch 66/500:  13%|█▎        | 66/500 [07:42<49:48,  6.88s/it, loss=1.5e+04, v_num=1]
```

Example 260 (unknown):
```unknown
Epoch 67/500:  13%|█▎        | 66/500 [07:42<49:48,  6.88s/it, loss=1.5e+04, v_num=1]
```

Example 261 (unknown):
```unknown
Epoch 67/500:  13%|█▎        | 67/500 [07:49<49:52,  6.91s/it, loss=1.5e+04, v_num=1]
```

Example 262 (unknown):
```unknown
Epoch 67/500:  13%|█▎        | 67/500 [07:49<49:52,  6.91s/it, loss=1.53e+04, v_num=1]
```

Example 263 (unknown):
```unknown
Epoch 68/500:  13%|█▎        | 67/500 [07:49<49:52,  6.91s/it, loss=1.53e+04, v_num=1]
```

Example 264 (unknown):
```unknown
Epoch 68/500:  14%|█▎        | 68/500 [07:56<49:22,  6.86s/it, loss=1.53e+04, v_num=1]
```

Example 265 (unknown):
```unknown
Epoch 68/500:  14%|█▎        | 68/500 [07:56<49:22,  6.86s/it, loss=1.55e+04, v_num=1]
```

Example 266 (unknown):
```unknown
Epoch 69/500:  14%|█▎        | 68/500 [07:56<49:22,  6.86s/it, loss=1.55e+04, v_num=1]
```

Example 267 (unknown):
```unknown
Epoch 69/500:  14%|█▍        | 69/500 [08:03<49:19,  6.87s/it, loss=1.55e+04, v_num=1]
```

Example 268 (unknown):
```unknown
Epoch 69/500:  14%|█▍        | 69/500 [08:03<49:19,  6.87s/it, loss=1.5e+04, v_num=1]
```

Example 269 (unknown):
```unknown
Epoch 70/500:  14%|█▍        | 69/500 [08:03<49:19,  6.87s/it, loss=1.5e+04, v_num=1]
```

Example 270 (unknown):
```unknown
Epoch 70/500:  14%|█▍        | 70/500 [08:10<49:21,  6.89s/it, loss=1.5e+04, v_num=1]
```

Example 271 (unknown):
```unknown
Epoch 70/500:  14%|█▍        | 70/500 [08:10<49:21,  6.89s/it, loss=1.56e+04, v_num=1]
```

Example 272 (unknown):
```unknown
Epoch 71/500:  14%|█▍        | 70/500 [08:10<49:21,  6.89s/it, loss=1.56e+04, v_num=1]
```

Example 273 (unknown):
```unknown
Epoch 71/500:  14%|█▍        | 71/500 [08:17<49:25,  6.91s/it, loss=1.56e+04, v_num=1]
```

Example 274 (unknown):
```unknown
Epoch 71/500:  14%|█▍        | 71/500 [08:17<49:25,  6.91s/it, loss=1.52e+04, v_num=1]
```

Example 275 (unknown):
```unknown
Epoch 72/500:  14%|█▍        | 71/500 [08:17<49:25,  6.91s/it, loss=1.52e+04, v_num=1]
```

Example 276 (unknown):
```unknown
Epoch 72/500:  14%|█▍        | 72/500 [08:23<49:07,  6.89s/it, loss=1.52e+04, v_num=1]
```

Example 277 (unknown):
```unknown
Epoch 72/500:  14%|█▍        | 72/500 [08:23<49:07,  6.89s/it, loss=1.52e+04, v_num=1]
```

Example 278 (unknown):
```unknown
Epoch 73/500:  14%|█▍        | 72/500 [08:23<49:07,  6.89s/it, loss=1.52e+04, v_num=1]
```

Example 279 (unknown):
```unknown
Epoch 73/500:  15%|█▍        | 73/500 [08:30<48:26,  6.81s/it, loss=1.52e+04, v_num=1]
```

Example 280 (unknown):
```unknown
Epoch 73/500:  15%|█▍        | 73/500 [08:30<48:26,  6.81s/it, loss=1.53e+04, v_num=1]
```

Example 281 (unknown):
```unknown
Epoch 74/500:  15%|█▍        | 73/500 [08:30<48:26,  6.81s/it, loss=1.53e+04, v_num=1]
```

Example 282 (unknown):
```unknown
Epoch 74/500:  15%|█▍        | 74/500 [08:37<48:35,  6.84s/it, loss=1.53e+04, v_num=1]
```

Example 283 (unknown):
```unknown
Epoch 74/500:  15%|█▍        | 74/500 [08:37<48:35,  6.84s/it, loss=1.51e+04, v_num=1]
```

Example 284 (unknown):
```unknown
Epoch 75/500:  15%|█▍        | 74/500 [08:37<48:35,  6.84s/it, loss=1.51e+04, v_num=1]
```

Example 285 (unknown):
```unknown
Epoch 75/500:  15%|█▌        | 75/500 [08:44<48:16,  6.82s/it, loss=1.51e+04, v_num=1]
```

Example 286 (unknown):
```unknown
Epoch 75/500:  15%|█▌        | 75/500 [08:44<48:16,  6.82s/it, loss=1.53e+04, v_num=1]
```

Example 287 (unknown):
```unknown
Epoch 76/500:  15%|█▌        | 75/500 [08:44<48:16,  6.82s/it, loss=1.53e+04, v_num=1]
```

Example 288 (unknown):
```unknown
Epoch 76/500:  15%|█▌        | 76/500 [08:50<47:59,  6.79s/it, loss=1.53e+04, v_num=1]
```

Example 289 (unknown):
```unknown
Epoch 76/500:  15%|█▌        | 76/500 [08:50<47:59,  6.79s/it, loss=1.54e+04, v_num=1]
```

Example 290 (unknown):
```unknown
Epoch 77/500:  15%|█▌        | 76/500 [08:50<47:59,  6.79s/it, loss=1.54e+04, v_num=1]
```

Example 291 (unknown):
```unknown
Epoch 77/500:  15%|█▌        | 77/500 [08:57<47:03,  6.67s/it, loss=1.54e+04, v_num=1]
```

Example 292 (unknown):
```unknown
Epoch 77/500:  15%|█▌        | 77/500 [08:57<47:03,  6.67s/it, loss=1.51e+04, v_num=1]
```

Example 293 (unknown):
```unknown
Epoch 78/500:  15%|█▌        | 77/500 [08:57<47:03,  6.67s/it, loss=1.51e+04, v_num=1]
```

Example 294 (unknown):
```unknown
Epoch 78/500:  16%|█▌        | 78/500 [09:04<47:17,  6.72s/it, loss=1.51e+04, v_num=1]
```

Example 295 (unknown):
```unknown
Epoch 78/500:  16%|█▌        | 78/500 [09:04<47:17,  6.72s/it, loss=1.53e+04, v_num=1]
```

Example 296 (unknown):
```unknown
Epoch 79/500:  16%|█▌        | 78/500 [09:04<47:17,  6.72s/it, loss=1.53e+04, v_num=1]
```

Example 297 (unknown):
```unknown
Epoch 79/500:  16%|█▌        | 79/500 [09:10<47:12,  6.73s/it, loss=1.53e+04, v_num=1]
```

Example 298 (unknown):
```unknown
Epoch 79/500:  16%|█▌        | 79/500 [09:10<47:12,  6.73s/it, loss=1.53e+04, v_num=1]
```

Example 299 (unknown):
```unknown
Epoch 80/500:  16%|█▌        | 79/500 [09:10<47:12,  6.73s/it, loss=1.53e+04, v_num=1]
```

Example 300 (unknown):
```unknown
Epoch 80/500:  16%|█▌        | 80/500 [09:17<46:25,  6.63s/it, loss=1.53e+04, v_num=1]
```

Example 301 (unknown):
```unknown
Epoch 80/500:  16%|█▌        | 80/500 [09:17<46:25,  6.63s/it, loss=1.54e+04, v_num=1]
```

Example 302 (unknown):
```unknown
Epoch 81/500:  16%|█▌        | 80/500 [09:17<46:25,  6.63s/it, loss=1.54e+04, v_num=1]
```

Example 303 (unknown):
```unknown
Epoch 81/500:  16%|█▌        | 81/500 [09:23<46:00,  6.59s/it, loss=1.54e+04, v_num=1]
```

Example 304 (unknown):
```unknown
Epoch 81/500:  16%|█▌        | 81/500 [09:23<46:00,  6.59s/it, loss=1.53e+04, v_num=1]
```

Example 305 (unknown):
```unknown
Epoch 82/500:  16%|█▌        | 81/500 [09:23<46:00,  6.59s/it, loss=1.53e+04, v_num=1]
```

Example 306 (unknown):
```unknown
Epoch 82/500:  16%|█▋        | 82/500 [09:30<45:25,  6.52s/it, loss=1.53e+04, v_num=1]
```

Example 307 (unknown):
```unknown
Epoch 82/500:  16%|█▋        | 82/500 [09:30<45:25,  6.52s/it, loss=1.55e+04, v_num=1]
```

Example 308 (unknown):
```unknown
Epoch 83/500:  16%|█▋        | 82/500 [09:30<45:25,  6.52s/it, loss=1.55e+04, v_num=1]
```

Example 309 (unknown):
```unknown
Epoch 83/500:  17%|█▋        | 83/500 [09:36<45:49,  6.59s/it, loss=1.55e+04, v_num=1]
```

Example 310 (unknown):
```unknown
Epoch 83/500:  17%|█▋        | 83/500 [09:36<45:49,  6.59s/it, loss=1.57e+04, v_num=1]
```

Example 311 (unknown):
```unknown
Epoch 84/500:  17%|█▋        | 83/500 [09:36<45:49,  6.59s/it, loss=1.57e+04, v_num=1]
```

Example 312 (unknown):
```unknown
Epoch 84/500:  17%|█▋        | 84/500 [09:43<46:03,  6.64s/it, loss=1.57e+04, v_num=1]
```

Example 313 (unknown):
```unknown
Epoch 84/500:  17%|█▋        | 84/500 [09:43<46:03,  6.64s/it, loss=1.51e+04, v_num=1]
```

Example 314 (unknown):
```unknown
Epoch 85/500:  17%|█▋        | 84/500 [09:43<46:03,  6.64s/it, loss=1.51e+04, v_num=1]
```

Example 315 (unknown):
```unknown
Epoch 85/500:  17%|█▋        | 85/500 [09:50<46:38,  6.74s/it, loss=1.51e+04, v_num=1]
```

Example 316 (unknown):
```unknown
Epoch 85/500:  17%|█▋        | 85/500 [09:50<46:38,  6.74s/it, loss=1.54e+04, v_num=1]
```

Example 317 (unknown):
```unknown
Epoch 86/500:  17%|█▋        | 85/500 [09:50<46:38,  6.74s/it, loss=1.54e+04, v_num=1]
```

Example 318 (unknown):
```unknown
Epoch 86/500:  17%|█▋        | 86/500 [09:57<47:00,  6.81s/it, loss=1.54e+04, v_num=1]
```

Example 319 (unknown):
```unknown
Epoch 86/500:  17%|█▋        | 86/500 [09:57<47:00,  6.81s/it, loss=1.53e+04, v_num=1]
```

Example 320 (unknown):
```unknown
Epoch 87/500:  17%|█▋        | 86/500 [09:57<47:00,  6.81s/it, loss=1.53e+04, v_num=1]
```

Example 321 (unknown):
```unknown
Epoch 87/500:  17%|█▋        | 87/500 [10:04<46:55,  6.82s/it, loss=1.53e+04, v_num=1]
```

Example 322 (unknown):
```unknown
Epoch 87/500:  17%|█▋        | 87/500 [10:04<46:55,  6.82s/it, loss=1.51e+04, v_num=1]
```

Example 323 (unknown):
```unknown
Epoch 88/500:  17%|█▋        | 87/500 [10:04<46:55,  6.82s/it, loss=1.51e+04, v_num=1]
```

Example 324 (unknown):
```unknown
Epoch 88/500:  18%|█▊        | 88/500 [10:11<46:19,  6.75s/it, loss=1.51e+04, v_num=1]
```

Example 325 (unknown):
```unknown
Epoch 88/500:  18%|█▊        | 88/500 [10:11<46:19,  6.75s/it, loss=1.52e+04, v_num=1]
```

Example 326 (unknown):
```unknown
Epoch 89/500:  18%|█▊        | 88/500 [10:11<46:19,  6.75s/it, loss=1.52e+04, v_num=1]
```

Example 327 (unknown):
```unknown
Epoch 89/500:  18%|█▊        | 89/500 [10:17<45:07,  6.59s/it, loss=1.52e+04, v_num=1]
```

Example 328 (unknown):
```unknown
Epoch 89/500:  18%|█▊        | 89/500 [10:17<45:07,  6.59s/it, loss=1.51e+04, v_num=1]
```

Example 329 (unknown):
```unknown
Epoch 90/500:  18%|█▊        | 89/500 [10:17<45:07,  6.59s/it, loss=1.51e+04, v_num=1]
```

Example 330 (unknown):
```unknown
Epoch 90/500:  18%|█▊        | 90/500 [10:24<45:23,  6.64s/it, loss=1.51e+04, v_num=1]
```

Example 331 (unknown):
```unknown
Epoch 90/500:  18%|█▊        | 90/500 [10:24<45:23,  6.64s/it, loss=1.54e+04, v_num=1]
```

Example 332 (unknown):
```unknown
Epoch 91/500:  18%|█▊        | 90/500 [10:24<45:23,  6.64s/it, loss=1.54e+04, v_num=1]
```

Example 333 (unknown):
```unknown
Epoch 91/500:  18%|█▊        | 91/500 [10:30<45:44,  6.71s/it, loss=1.54e+04, v_num=1]
```

Example 334 (unknown):
```unknown
Epoch 91/500:  18%|█▊        | 91/500 [10:30<45:44,  6.71s/it, loss=1.53e+04, v_num=1]
```

Example 335 (unknown):
```unknown
Epoch 92/500:  18%|█▊        | 91/500 [10:30<45:44,  6.71s/it, loss=1.53e+04, v_num=1]
```

Example 336 (unknown):
```unknown
Epoch 92/500:  18%|█▊        | 92/500 [10:37<45:56,  6.76s/it, loss=1.53e+04, v_num=1]
```

Example 337 (unknown):
```unknown
Epoch 92/500:  18%|█▊        | 92/500 [10:37<45:56,  6.76s/it, loss=1.51e+04, v_num=1]
```

Example 338 (unknown):
```unknown
Epoch 93/500:  18%|█▊        | 92/500 [10:37<45:56,  6.76s/it, loss=1.51e+04, v_num=1]
```

Example 339 (unknown):
```unknown
Epoch 93/500:  19%|█▊        | 93/500 [10:44<46:11,  6.81s/it, loss=1.51e+04, v_num=1]
```

Example 340 (unknown):
```unknown
Epoch 93/500:  19%|█▊        | 93/500 [10:44<46:11,  6.81s/it, loss=1.53e+04, v_num=1]
```

Example 341 (unknown):
```unknown
Epoch 94/500:  19%|█▊        | 93/500 [10:44<46:11,  6.81s/it, loss=1.53e+04, v_num=1]
```

Example 342 (unknown):
```unknown
Epoch 94/500:  19%|█▉        | 94/500 [10:51<45:41,  6.75s/it, loss=1.53e+04, v_num=1]
```

Example 343 (unknown):
```unknown
Epoch 94/500:  19%|█▉        | 94/500 [10:51<45:41,  6.75s/it, loss=1.54e+04, v_num=1]
```

Example 344 (unknown):
```unknown
Epoch 95/500:  19%|█▉        | 94/500 [10:51<45:41,  6.75s/it, loss=1.54e+04, v_num=1]
```

Example 345 (unknown):
```unknown
Epoch 95/500:  19%|█▉        | 95/500 [10:57<45:02,  6.67s/it, loss=1.54e+04, v_num=1]
```

Example 346 (unknown):
```unknown
Epoch 95/500:  19%|█▉        | 95/500 [10:57<45:02,  6.67s/it, loss=1.53e+04, v_num=1]
```

Example 347 (unknown):
```unknown
Epoch 96/500:  19%|█▉        | 95/500 [10:57<45:02,  6.67s/it, loss=1.53e+04, v_num=1]
```

Example 348 (unknown):
```unknown
Epoch 96/500:  19%|█▉        | 96/500 [11:04<45:05,  6.70s/it, loss=1.53e+04, v_num=1]
```

Example 349 (unknown):
```unknown
Epoch 96/500:  19%|█▉        | 96/500 [11:04<45:05,  6.70s/it, loss=1.49e+04, v_num=1]
```

Example 350 (unknown):
```unknown
Epoch 97/500:  19%|█▉        | 96/500 [11:04<45:05,  6.70s/it, loss=1.49e+04, v_num=1]
```

Example 351 (unknown):
```unknown
Epoch 97/500:  19%|█▉        | 97/500 [11:11<45:26,  6.77s/it, loss=1.49e+04, v_num=1]
```

Example 352 (unknown):
```unknown
Epoch 97/500:  19%|█▉        | 97/500 [11:11<45:26,  6.77s/it, loss=1.53e+04, v_num=1]
```

Example 353 (unknown):
```unknown
Epoch 98/500:  19%|█▉        | 97/500 [11:11<45:26,  6.77s/it, loss=1.53e+04, v_num=1]
```

Example 354 (unknown):
```unknown
Epoch 98/500:  20%|█▉        | 98/500 [11:18<45:20,  6.77s/it, loss=1.53e+04, v_num=1]
```

Example 355 (unknown):
```unknown
Epoch 98/500:  20%|█▉        | 98/500 [11:18<45:20,  6.77s/it, loss=1.54e+04, v_num=1]
```

Example 356 (unknown):
```unknown
Epoch 99/500:  20%|█▉        | 98/500 [11:18<45:20,  6.77s/it, loss=1.54e+04, v_num=1]
```

Example 357 (unknown):
```unknown
Epoch 99/500:  20%|█▉        | 99/500 [11:25<45:34,  6.82s/it, loss=1.54e+04, v_num=1]
```

Example 358 (unknown):
```unknown
Epoch 99/500:  20%|█▉        | 99/500 [11:25<45:34,  6.82s/it, loss=1.55e+04, v_num=1]
```

Example 359 (unknown):
```unknown
Epoch 100/500:  20%|█▉        | 99/500 [11:25<45:34,  6.82s/it, loss=1.55e+04, v_num=1]
```

Example 360 (unknown):
```unknown
Epoch 100/500:  20%|██        | 100/500 [11:32<45:28,  6.82s/it, loss=1.55e+04, v_num=1]
```

Example 361 (unknown):
```unknown
Epoch 100/500:  20%|██        | 100/500 [11:32<45:28,  6.82s/it, loss=1.54e+04, v_num=1]
```

Example 362 (unknown):
```unknown
Epoch 101/500:  20%|██        | 100/500 [11:32<45:28,  6.82s/it, loss=1.54e+04, v_num=1]
```

Example 363 (unknown):
```unknown
Epoch 101/500:  20%|██        | 101/500 [11:39<45:39,  6.87s/it, loss=1.54e+04, v_num=1]
```

Example 364 (unknown):
```unknown
Epoch 101/500:  20%|██        | 101/500 [11:39<45:39,  6.87s/it, loss=1.54e+04, v_num=1]
```

Example 365 (unknown):
```unknown
Epoch 102/500:  20%|██        | 101/500 [11:39<45:39,  6.87s/it, loss=1.54e+04, v_num=1]
```

Example 366 (unknown):
```unknown
Epoch 102/500:  20%|██        | 102/500 [11:46<45:44,  6.90s/it, loss=1.54e+04, v_num=1]
```

Example 367 (unknown):
```unknown
Epoch 102/500:  20%|██        | 102/500 [11:46<45:44,  6.90s/it, loss=1.52e+04, v_num=1]
```

Example 368 (unknown):
```unknown
Epoch 103/500:  20%|██        | 102/500 [11:46<45:44,  6.90s/it, loss=1.52e+04, v_num=1]
```

Example 369 (unknown):
```unknown
Epoch 103/500:  21%|██        | 103/500 [11:52<45:45,  6.91s/it, loss=1.52e+04, v_num=1]
```

Example 370 (unknown):
```unknown
Epoch 103/500:  21%|██        | 103/500 [11:52<45:45,  6.91s/it, loss=1.55e+04, v_num=1]
```

Example 371 (unknown):
```unknown
Epoch 104/500:  21%|██        | 103/500 [11:52<45:45,  6.91s/it, loss=1.55e+04, v_num=1]
```

Example 372 (unknown):
```unknown
Epoch 104/500:  21%|██        | 104/500 [11:59<45:28,  6.89s/it, loss=1.55e+04, v_num=1]
```

Example 373 (unknown):
```unknown
Epoch 104/500:  21%|██        | 104/500 [11:59<45:28,  6.89s/it, loss=1.52e+04, v_num=1]
```

Example 374 (unknown):
```unknown
Epoch 105/500:  21%|██        | 104/500 [11:59<45:28,  6.89s/it, loss=1.52e+04, v_num=1]
```

Example 375 (unknown):
```unknown
Epoch 105/500:  21%|██        | 105/500 [12:06<45:30,  6.91s/it, loss=1.52e+04, v_num=1]
```

Example 376 (unknown):
```unknown
Epoch 105/500:  21%|██        | 105/500 [12:06<45:30,  6.91s/it, loss=1.52e+04, v_num=1]
```

Example 377 (unknown):
```unknown
Epoch 106/500:  21%|██        | 105/500 [12:06<45:30,  6.91s/it, loss=1.52e+04, v_num=1]
```

Example 378 (unknown):
```unknown
Epoch 106/500:  21%|██        | 106/500 [12:13<45:05,  6.87s/it, loss=1.52e+04, v_num=1]
```

Example 379 (unknown):
```unknown
Epoch 106/500:  21%|██        | 106/500 [12:13<45:05,  6.87s/it, loss=1.54e+04, v_num=1]
```

Example 380 (unknown):
```unknown
Epoch 107/500:  21%|██        | 106/500 [12:13<45:05,  6.87s/it, loss=1.54e+04, v_num=1]
```

Example 381 (unknown):
```unknown
Epoch 107/500:  21%|██▏       | 107/500 [12:20<45:03,  6.88s/it, loss=1.54e+04, v_num=1]
```

Example 382 (unknown):
```unknown
Epoch 107/500:  21%|██▏       | 107/500 [12:20<45:03,  6.88s/it, loss=1.54e+04, v_num=1]
```

Example 383 (unknown):
```unknown
Epoch 108/500:  21%|██▏       | 107/500 [12:20<45:03,  6.88s/it, loss=1.54e+04, v_num=1]
```

Example 384 (unknown):
```unknown
Epoch 108/500:  22%|██▏       | 108/500 [12:27<44:56,  6.88s/it, loss=1.54e+04, v_num=1]
```

Example 385 (unknown):
```unknown
Epoch 108/500:  22%|██▏       | 108/500 [12:27<44:56,  6.88s/it, loss=1.53e+04, v_num=1]
```

Example 386 (unknown):
```unknown
Epoch 109/500:  22%|██▏       | 108/500 [12:27<44:56,  6.88s/it, loss=1.53e+04, v_num=1]
```

Example 387 (unknown):
```unknown
Epoch 109/500:  22%|██▏       | 109/500 [12:34<45:03,  6.91s/it, loss=1.53e+04, v_num=1]
```

Example 388 (unknown):
```unknown
Epoch 109/500:  22%|██▏       | 109/500 [12:34<45:03,  6.91s/it, loss=1.53e+04, v_num=1]
```

Example 389 (unknown):
```unknown
Epoch 110/500:  22%|██▏       | 109/500 [12:34<45:03,  6.91s/it, loss=1.53e+04, v_num=1]
```

Example 390 (unknown):
```unknown
Epoch 110/500:  22%|██▏       | 110/500 [12:41<44:58,  6.92s/it, loss=1.53e+04, v_num=1]
```

Example 391 (unknown):
```unknown
Epoch 110/500:  22%|██▏       | 110/500 [12:41<44:58,  6.92s/it, loss=1.53e+04, v_num=1]
```

Example 392 (unknown):
```unknown
Epoch 111/500:  22%|██▏       | 110/500 [12:41<44:58,  6.92s/it, loss=1.53e+04, v_num=1]
```

Example 393 (unknown):
```unknown
Epoch 111/500:  22%|██▏       | 111/500 [12:48<44:43,  6.90s/it, loss=1.53e+04, v_num=1]
```

Example 394 (unknown):
```unknown
Epoch 111/500:  22%|██▏       | 111/500 [12:48<44:43,  6.90s/it, loss=1.51e+04, v_num=1]
```

Example 395 (unknown):
```unknown
Epoch 112/500:  22%|██▏       | 111/500 [12:48<44:43,  6.90s/it, loss=1.51e+04, v_num=1]
```

Example 396 (unknown):
```unknown
Epoch 112/500:  22%|██▏       | 112/500 [12:55<44:37,  6.90s/it, loss=1.51e+04, v_num=1]
```

Example 397 (unknown):
```unknown
Epoch 112/500:  22%|██▏       | 112/500 [12:55<44:37,  6.90s/it, loss=1.54e+04, v_num=1]
```

Example 398 (unknown):
```unknown
Epoch 113/500:  22%|██▏       | 112/500 [12:55<44:37,  6.90s/it, loss=1.54e+04, v_num=1]
```

Example 399 (unknown):
```unknown
Epoch 113/500:  23%|██▎       | 113/500 [13:01<44:35,  6.91s/it, loss=1.54e+04, v_num=1]
```

Example 400 (unknown):
```unknown
Epoch 113/500:  23%|██▎       | 113/500 [13:01<44:35,  6.91s/it, loss=1.55e+04, v_num=1]
```

Example 401 (unknown):
```unknown
Epoch 114/500:  23%|██▎       | 113/500 [13:01<44:35,  6.91s/it, loss=1.55e+04, v_num=1]
```

Example 402 (unknown):
```unknown
Epoch 114/500:  23%|██▎       | 114/500 [13:08<44:35,  6.93s/it, loss=1.55e+04, v_num=1]
```

Example 403 (unknown):
```unknown
Epoch 114/500:  23%|██▎       | 114/500 [13:08<44:35,  6.93s/it, loss=1.5e+04, v_num=1]
```

Example 404 (unknown):
```unknown
Epoch 115/500:  23%|██▎       | 114/500 [13:08<44:35,  6.93s/it, loss=1.5e+04, v_num=1]
```

Example 405 (unknown):
```unknown
Epoch 115/500:  23%|██▎       | 115/500 [13:15<43:32,  6.79s/it, loss=1.5e+04, v_num=1]
```

Example 406 (unknown):
```unknown
Epoch 115/500:  23%|██▎       | 115/500 [13:15<43:32,  6.79s/it, loss=1.52e+04, v_num=1]
```

Example 407 (unknown):
```unknown
Epoch 116/500:  23%|██▎       | 115/500 [13:15<43:32,  6.79s/it, loss=1.52e+04, v_num=1]
```

Example 408 (unknown):
```unknown
Epoch 116/500:  23%|██▎       | 116/500 [13:22<43:51,  6.85s/it, loss=1.52e+04, v_num=1]
```

Example 409 (unknown):
```unknown
Epoch 116/500:  23%|██▎       | 116/500 [13:22<43:51,  6.85s/it, loss=1.54e+04, v_num=1]
```

Example 410 (unknown):
```unknown
Epoch 117/500:  23%|██▎       | 116/500 [13:22<43:51,  6.85s/it, loss=1.54e+04, v_num=1]
```

Example 411 (unknown):
```unknown
Epoch 117/500:  23%|██▎       | 117/500 [13:29<44:14,  6.93s/it, loss=1.54e+04, v_num=1]
```

Example 412 (unknown):
```unknown
Epoch 117/500:  23%|██▎       | 117/500 [13:29<44:14,  6.93s/it, loss=1.51e+04, v_num=1]
```

Example 413 (unknown):
```unknown
Epoch 118/500:  23%|██▎       | 117/500 [13:29<44:14,  6.93s/it, loss=1.51e+04, v_num=1]
```

Example 414 (unknown):
```unknown
Epoch 118/500:  24%|██▎       | 118/500 [13:36<43:55,  6.90s/it, loss=1.51e+04, v_num=1]
```

Example 415 (unknown):
```unknown
Epoch 118/500:  24%|██▎       | 118/500 [13:36<43:55,  6.90s/it, loss=1.52e+04, v_num=1]
```

Example 416 (unknown):
```unknown
Epoch 119/500:  24%|██▎       | 118/500 [13:36<43:55,  6.90s/it, loss=1.52e+04, v_num=1]
```

Example 417 (unknown):
```unknown
Epoch 119/500:  24%|██▍       | 119/500 [13:43<43:48,  6.90s/it, loss=1.52e+04, v_num=1]
```

Example 418 (unknown):
```unknown
Epoch 119/500:  24%|██▍       | 119/500 [13:43<43:48,  6.90s/it, loss=1.54e+04, v_num=1]
```

Example 419 (unknown):
```unknown
Epoch 120/500:  24%|██▍       | 119/500 [13:43<43:48,  6.90s/it, loss=1.54e+04, v_num=1]
```

Example 420 (unknown):
```unknown
Epoch 120/500:  24%|██▍       | 120/500 [13:50<43:57,  6.94s/it, loss=1.54e+04, v_num=1]
```

Example 421 (unknown):
```unknown
Epoch 120/500:  24%|██▍       | 120/500 [13:50<43:57,  6.94s/it, loss=1.55e+04, v_num=1]
```

Example 422 (unknown):
```unknown
Epoch 121/500:  24%|██▍       | 120/500 [13:50<43:57,  6.94s/it, loss=1.55e+04, v_num=1]
```

Example 423 (unknown):
```unknown
Epoch 121/500:  24%|██▍       | 121/500 [13:56<43:06,  6.82s/it, loss=1.55e+04, v_num=1]
```

Example 424 (unknown):
```unknown
Epoch 121/500:  24%|██▍       | 121/500 [13:56<43:06,  6.82s/it, loss=1.52e+04, v_num=1]
```

Example 425 (unknown):
```unknown
Epoch 122/500:  24%|██▍       | 121/500 [13:56<43:06,  6.82s/it, loss=1.52e+04, v_num=1]
```

Example 426 (unknown):
```unknown
Epoch 122/500:  24%|██▍       | 122/500 [14:03<43:16,  6.87s/it, loss=1.52e+04, v_num=1]
```

Example 427 (unknown):
```unknown
Epoch 122/500:  24%|██▍       | 122/500 [14:03<43:16,  6.87s/it, loss=1.55e+04, v_num=1]
```

Example 428 (unknown):
```unknown
Epoch 123/500:  24%|██▍       | 122/500 [14:03<43:16,  6.87s/it, loss=1.55e+04, v_num=1]
```

Example 429 (unknown):
```unknown
Epoch 123/500:  25%|██▍       | 123/500 [14:10<43:16,  6.89s/it, loss=1.55e+04, v_num=1]
```

Example 430 (unknown):
```unknown
Epoch 123/500:  25%|██▍       | 123/500 [14:10<43:16,  6.89s/it, loss=1.51e+04, v_num=1]
```

Example 431 (unknown):
```unknown
Epoch 124/500:  25%|██▍       | 123/500 [14:10<43:16,  6.89s/it, loss=1.51e+04, v_num=1]
```

Example 432 (unknown):
```unknown
Epoch 124/500:  25%|██▍       | 124/500 [14:17<43:11,  6.89s/it, loss=1.51e+04, v_num=1]
```

Example 433 (unknown):
```unknown
Epoch 124/500:  25%|██▍       | 124/500 [14:17<43:11,  6.89s/it, loss=1.54e+04, v_num=1]
```

Example 434 (unknown):
```unknown
Epoch 125/500:  25%|██▍       | 124/500 [14:17<43:11,  6.89s/it, loss=1.54e+04, v_num=1]
```

Example 435 (unknown):
```unknown
Epoch 125/500:  25%|██▌       | 125/500 [14:24<43:12,  6.91s/it, loss=1.54e+04, v_num=1]
```

Example 436 (unknown):
```unknown
Epoch 125/500:  25%|██▌       | 125/500 [14:24<43:12,  6.91s/it, loss=1.52e+04, v_num=1]
```

Example 437 (unknown):
```unknown
Epoch 126/500:  25%|██▌       | 125/500 [14:24<43:12,  6.91s/it, loss=1.52e+04, v_num=1]
```

Example 438 (unknown):
```unknown
Epoch 126/500:  25%|██▌       | 126/500 [14:31<42:49,  6.87s/it, loss=1.52e+04, v_num=1]
```

Example 439 (unknown):
```unknown
Epoch 126/500:  25%|██▌       | 126/500 [14:31<42:49,  6.87s/it, loss=1.51e+04, v_num=1]
```

Example 440 (unknown):
```unknown
Epoch 127/500:  25%|██▌       | 126/500 [14:31<42:49,  6.87s/it, loss=1.51e+04, v_num=1]
```

Example 441 (unknown):
```unknown
Epoch 127/500:  25%|██▌       | 127/500 [14:38<42:32,  6.84s/it, loss=1.51e+04, v_num=1]
```

Example 442 (unknown):
```unknown
Epoch 127/500:  25%|██▌       | 127/500 [14:38<42:32,  6.84s/it, loss=1.54e+04, v_num=1]
```

Example 443 (unknown):
```unknown
Epoch 128/500:  25%|██▌       | 127/500 [14:38<42:32,  6.84s/it, loss=1.54e+04, v_num=1]
```

Example 444 (unknown):
```unknown
Epoch 128/500:  26%|██▌       | 128/500 [14:45<42:39,  6.88s/it, loss=1.54e+04, v_num=1]
```

Example 445 (unknown):
```unknown
Epoch 128/500:  26%|██▌       | 128/500 [14:45<42:39,  6.88s/it, loss=1.52e+04, v_num=1]
```

Example 446 (unknown):
```unknown
Epoch 129/500:  26%|██▌       | 128/500 [14:45<42:39,  6.88s/it, loss=1.52e+04, v_num=1]
```

Example 447 (unknown):
```unknown
Epoch 129/500:  26%|██▌       | 129/500 [14:52<42:36,  6.89s/it, loss=1.52e+04, v_num=1]
```

Example 448 (unknown):
```unknown
Epoch 129/500:  26%|██▌       | 129/500 [14:52<42:36,  6.89s/it, loss=1.54e+04, v_num=1]
```

Example 449 (unknown):
```unknown
Epoch 130/500:  26%|██▌       | 129/500 [14:52<42:36,  6.89s/it, loss=1.54e+04, v_num=1]
```

Example 450 (unknown):
```unknown
Epoch 130/500:  26%|██▌       | 130/500 [14:58<42:20,  6.87s/it, loss=1.54e+04, v_num=1]
```

Example 451 (unknown):
```unknown
Epoch 130/500:  26%|██▌       | 130/500 [14:58<42:20,  6.87s/it, loss=1.54e+04, v_num=1]
```

Example 452 (unknown):
```unknown
Epoch 131/500:  26%|██▌       | 130/500 [14:58<42:20,  6.87s/it, loss=1.54e+04, v_num=1]
```

Example 453 (unknown):
```unknown
Epoch 131/500:  26%|██▌       | 131/500 [15:05<41:29,  6.75s/it, loss=1.54e+04, v_num=1]
```

Example 454 (unknown):
```unknown
Epoch 131/500:  26%|██▌       | 131/500 [15:05<41:29,  6.75s/it, loss=1.5e+04, v_num=1]
```

Example 455 (unknown):
```unknown
Epoch 132/500:  26%|██▌       | 131/500 [15:05<41:29,  6.75s/it, loss=1.5e+04, v_num=1]
```

Example 456 (unknown):
```unknown
Epoch 132/500:  26%|██▋       | 132/500 [15:11<41:06,  6.70s/it, loss=1.5e+04, v_num=1]
```

Example 457 (unknown):
```unknown
Epoch 132/500:  26%|██▋       | 132/500 [15:11<41:06,  6.70s/it, loss=1.54e+04, v_num=1]
```

Example 458 (unknown):
```unknown
Epoch 133/500:  26%|██▋       | 132/500 [15:11<41:06,  6.70s/it, loss=1.54e+04, v_num=1]
```

Example 459 (unknown):
```unknown
Epoch 133/500:  27%|██▋       | 133/500 [15:18<40:19,  6.59s/it, loss=1.54e+04, v_num=1]
```

Example 460 (unknown):
```unknown
Epoch 133/500:  27%|██▋       | 133/500 [15:18<40:19,  6.59s/it, loss=1.53e+04, v_num=1]
```

Example 461 (unknown):
```unknown
Epoch 134/500:  27%|██▋       | 133/500 [15:18<40:19,  6.59s/it, loss=1.53e+04, v_num=1]
```

Example 462 (unknown):
```unknown
Epoch 134/500:  27%|██▋       | 134/500 [15:24<40:02,  6.57s/it, loss=1.53e+04, v_num=1]
```

Example 463 (unknown):
```unknown
Epoch 134/500:  27%|██▋       | 134/500 [15:24<40:02,  6.57s/it, loss=1.51e+04, v_num=1]
```

Example 464 (unknown):
```unknown
Epoch 135/500:  27%|██▋       | 134/500 [15:24<40:02,  6.57s/it, loss=1.51e+04, v_num=1]
```

Example 465 (unknown):
```unknown
Epoch 135/500:  27%|██▋       | 135/500 [15:31<39:37,  6.51s/it, loss=1.51e+04, v_num=1]
```

Example 466 (unknown):
```unknown
Epoch 135/500:  27%|██▋       | 135/500 [15:31<39:37,  6.51s/it, loss=1.52e+04, v_num=1]
```

Example 467 (unknown):
```unknown
Epoch 136/500:  27%|██▋       | 135/500 [15:31<39:37,  6.51s/it, loss=1.52e+04, v_num=1]
```

Example 468 (unknown):
```unknown
Epoch 136/500:  27%|██▋       | 136/500 [15:37<40:04,  6.61s/it, loss=1.52e+04, v_num=1]
```

Example 469 (unknown):
```unknown
Epoch 136/500:  27%|██▋       | 136/500 [15:37<40:04,  6.61s/it, loss=1.53e+04, v_num=1]
```

Example 470 (unknown):
```unknown
Epoch 137/500:  27%|██▋       | 136/500 [15:37<40:04,  6.61s/it, loss=1.53e+04, v_num=1]
```

Example 471 (unknown):
```unknown
Epoch 137/500:  27%|██▋       | 137/500 [15:44<40:40,  6.72s/it, loss=1.53e+04, v_num=1]
```

Example 472 (unknown):
```unknown
Epoch 137/500:  27%|██▋       | 137/500 [15:44<40:40,  6.72s/it, loss=1.51e+04, v_num=1]
```

Example 473 (unknown):
```unknown
Epoch 138/500:  27%|██▋       | 137/500 [15:44<40:40,  6.72s/it, loss=1.51e+04, v_num=1]
```

Example 474 (unknown):
```unknown
Epoch 138/500:  28%|██▊       | 138/500 [15:51<40:57,  6.79s/it, loss=1.51e+04, v_num=1]
```

Example 475 (unknown):
```unknown
Epoch 138/500:  28%|██▊       | 138/500 [15:51<40:57,  6.79s/it, loss=1.52e+04, v_num=1]
```

Example 476 (unknown):
```unknown
Epoch 139/500:  28%|██▊       | 138/500 [15:51<40:57,  6.79s/it, loss=1.52e+04, v_num=1]
```

Example 477 (unknown):
```unknown
Epoch 139/500:  28%|██▊       | 139/500 [15:58<40:45,  6.78s/it, loss=1.52e+04, v_num=1]
```

Example 478 (unknown):
```unknown
Epoch 139/500:  28%|██▊       | 139/500 [15:58<40:45,  6.78s/it, loss=1.51e+04, v_num=1]
```

Example 479 (unknown):
```unknown
Epoch 140/500:  28%|██▊       | 139/500 [15:58<40:45,  6.78s/it, loss=1.51e+04, v_num=1]
```

Example 480 (unknown):
```unknown
Epoch 140/500:  28%|██▊       | 140/500 [16:05<41:00,  6.83s/it, loss=1.51e+04, v_num=1]
```

Example 481 (unknown):
```unknown
Epoch 140/500:  28%|██▊       | 140/500 [16:05<41:00,  6.83s/it, loss=1.54e+04, v_num=1]
```

Example 482 (unknown):
```unknown
Epoch 141/500:  28%|██▊       | 140/500 [16:05<41:00,  6.83s/it, loss=1.54e+04, v_num=1]
```

Example 483 (unknown):
```unknown
Epoch 141/500:  28%|██▊       | 141/500 [16:12<40:36,  6.79s/it, loss=1.54e+04, v_num=1]
```

Example 484 (unknown):
```unknown
Epoch 141/500:  28%|██▊       | 141/500 [16:12<40:36,  6.79s/it, loss=1.52e+04, v_num=1]
```

Example 485 (unknown):
```unknown
Epoch 142/500:  28%|██▊       | 141/500 [16:12<40:36,  6.79s/it, loss=1.52e+04, v_num=1]
```

Example 486 (unknown):
```unknown
Epoch 142/500:  28%|██▊       | 142/500 [16:18<39:48,  6.67s/it, loss=1.52e+04, v_num=1]
```

Example 487 (unknown):
```unknown
Epoch 142/500:  28%|██▊       | 142/500 [16:18<39:48,  6.67s/it, loss=1.51e+04, v_num=1]
```

Example 488 (unknown):
```unknown
Epoch 143/500:  28%|██▊       | 142/500 [16:18<39:48,  6.67s/it, loss=1.51e+04, v_num=1]
```

Example 489 (unknown):
```unknown
Epoch 143/500:  29%|██▊       | 143/500 [16:25<39:48,  6.69s/it, loss=1.51e+04, v_num=1]
```

Example 490 (unknown):
```unknown
Epoch 143/500:  29%|██▊       | 143/500 [16:25<39:48,  6.69s/it, loss=1.52e+04, v_num=1]
```

Example 491 (unknown):
```unknown
Epoch 144/500:  29%|██▊       | 143/500 [16:25<39:48,  6.69s/it, loss=1.52e+04, v_num=1]
```

Example 492 (unknown):
```unknown
Epoch 144/500:  29%|██▉       | 144/500 [16:32<39:42,  6.69s/it, loss=1.52e+04, v_num=1]
```

Example 493 (unknown):
```unknown
Epoch 144/500:  29%|██▉       | 144/500 [16:32<39:42,  6.69s/it, loss=1.51e+04, v_num=1]
```

Example 494 (unknown):
```unknown
Epoch 145/500:  29%|██▉       | 144/500 [16:32<39:42,  6.69s/it, loss=1.51e+04, v_num=1]
```

Example 495 (unknown):
```unknown
Epoch 145/500:  29%|██▉       | 145/500 [16:38<39:47,  6.73s/it, loss=1.51e+04, v_num=1]
```

Example 496 (unknown):
```unknown
Epoch 145/500:  29%|██▉       | 145/500 [16:38<39:47,  6.73s/it, loss=1.52e+04, v_num=1]
```

Example 497 (unknown):
```unknown
Epoch 146/500:  29%|██▉       | 145/500 [16:38<39:47,  6.73s/it, loss=1.52e+04, v_num=1]
```

Example 498 (unknown):
```unknown
Epoch 146/500:  29%|██▉       | 146/500 [16:45<40:06,  6.80s/it, loss=1.52e+04, v_num=1]
```

Example 499 (unknown):
```unknown
Epoch 146/500:  29%|██▉       | 146/500 [16:45<40:06,  6.80s/it, loss=1.52e+04, v_num=1]
```

Example 500 (unknown):
```unknown
Epoch 147/500:  29%|██▉       | 146/500 [16:45<40:06,  6.80s/it, loss=1.52e+04, v_num=1]
```

Example 501 (unknown):
```unknown
Epoch 147/500:  29%|██▉       | 147/500 [16:52<39:38,  6.74s/it, loss=1.52e+04, v_num=1]
```

Example 502 (unknown):
```unknown
Epoch 147/500:  29%|██▉       | 147/500 [16:52<39:38,  6.74s/it, loss=1.52e+04, v_num=1]
```

Example 503 (unknown):
```unknown
Epoch 148/500:  29%|██▉       | 147/500 [16:52<39:38,  6.74s/it, loss=1.52e+04, v_num=1]
```

Example 504 (unknown):
```unknown
Epoch 148/500:  30%|██▉       | 148/500 [16:59<39:58,  6.81s/it, loss=1.52e+04, v_num=1]
```

Example 505 (unknown):
```unknown
Epoch 148/500:  30%|██▉       | 148/500 [16:59<39:58,  6.81s/it, loss=1.5e+04, v_num=1]
```

Example 506 (unknown):
```unknown
Epoch 149/500:  30%|██▉       | 148/500 [16:59<39:58,  6.81s/it, loss=1.5e+04, v_num=1]
```

Example 507 (unknown):
```unknown
Epoch 149/500:  30%|██▉       | 149/500 [17:06<40:06,  6.86s/it, loss=1.5e+04, v_num=1]
```

Example 508 (unknown):
```unknown
Epoch 149/500:  30%|██▉       | 149/500 [17:06<40:06,  6.86s/it, loss=1.5e+04, v_num=1]
```

Example 509 (unknown):
```unknown
Epoch 150/500:  30%|██▉       | 149/500 [17:06<40:06,  6.86s/it, loss=1.5e+04, v_num=1]
```

Example 510 (unknown):
```unknown
Epoch 150/500:  30%|███       | 150/500 [17:13<40:17,  6.91s/it, loss=1.5e+04, v_num=1]
```

Example 511 (unknown):
```unknown
Epoch 150/500:  30%|███       | 150/500 [17:13<40:17,  6.91s/it, loss=1.5e+04, v_num=1]
```

Example 512 (unknown):
```unknown
Epoch 151/500:  30%|███       | 150/500 [17:13<40:17,  6.91s/it, loss=1.5e+04, v_num=1]
```

Example 513 (unknown):
```unknown
Epoch 151/500:  30%|███       | 151/500 [17:20<40:17,  6.93s/it, loss=1.5e+04, v_num=1]
```

Example 514 (unknown):
```unknown
Epoch 151/500:  30%|███       | 151/500 [17:20<40:17,  6.93s/it, loss=1.54e+04, v_num=1]
```

Example 515 (unknown):
```unknown
Epoch 152/500:  30%|███       | 151/500 [17:20<40:17,  6.93s/it, loss=1.54e+04, v_num=1]
```

Example 516 (unknown):
```unknown
Epoch 152/500:  30%|███       | 152/500 [17:27<39:53,  6.88s/it, loss=1.54e+04, v_num=1]
```

Example 517 (unknown):
```unknown
Epoch 152/500:  30%|███       | 152/500 [17:27<39:53,  6.88s/it, loss=1.52e+04, v_num=1]
```

Example 518 (unknown):
```unknown
Epoch 153/500:  30%|███       | 152/500 [17:27<39:53,  6.88s/it, loss=1.52e+04, v_num=1]
```

Example 519 (unknown):
```unknown
Epoch 153/500:  31%|███       | 153/500 [17:33<39:32,  6.84s/it, loss=1.52e+04, v_num=1]
```

Example 520 (unknown):
```unknown
Epoch 153/500:  31%|███       | 153/500 [17:33<39:32,  6.84s/it, loss=1.49e+04, v_num=1]
```

Example 521 (unknown):
```unknown
Epoch 154/500:  31%|███       | 153/500 [17:33<39:32,  6.84s/it, loss=1.49e+04, v_num=1]
```

Example 522 (unknown):
```unknown
Epoch 154/500:  31%|███       | 154/500 [17:40<39:37,  6.87s/it, loss=1.49e+04, v_num=1]
```

Example 523 (unknown):
```unknown
Epoch 154/500:  31%|███       | 154/500 [17:40<39:37,  6.87s/it, loss=1.52e+04, v_num=1]
```

Example 524 (unknown):
```unknown
Epoch 155/500:  31%|███       | 154/500 [17:40<39:37,  6.87s/it, loss=1.52e+04, v_num=1]
```

Example 525 (unknown):
```unknown
Epoch 155/500:  31%|███       | 155/500 [17:47<39:27,  6.86s/it, loss=1.52e+04, v_num=1]
```

Example 526 (unknown):
```unknown
Epoch 155/500:  31%|███       | 155/500 [17:47<39:27,  6.86s/it, loss=1.53e+04, v_num=1]
```

Example 527 (unknown):
```unknown
Epoch 156/500:  31%|███       | 155/500 [17:47<39:27,  6.86s/it, loss=1.53e+04, v_num=1]
```

Example 528 (unknown):
```unknown
Epoch 156/500:  31%|███       | 156/500 [17:54<39:22,  6.87s/it, loss=1.53e+04, v_num=1]
```

Example 529 (unknown):
```unknown
Epoch 156/500:  31%|███       | 156/500 [17:54<39:22,  6.87s/it, loss=1.56e+04, v_num=1]
```

Example 530 (unknown):
```unknown
Epoch 157/500:  31%|███       | 156/500 [17:54<39:22,  6.87s/it, loss=1.56e+04, v_num=1]
```

Example 531 (unknown):
```unknown
Epoch 157/500:  31%|███▏      | 157/500 [18:01<39:28,  6.90s/it, loss=1.56e+04, v_num=1]
```

Example 532 (unknown):
```unknown
Epoch 157/500:  31%|███▏      | 157/500 [18:01<39:28,  6.90s/it, loss=1.51e+04, v_num=1]
```

Example 533 (unknown):
```unknown
Epoch 158/500:  31%|███▏      | 157/500 [18:01<39:28,  6.90s/it, loss=1.51e+04, v_num=1]
```

Example 534 (unknown):
```unknown
Epoch 158/500:  32%|███▏      | 158/500 [18:08<39:08,  6.87s/it, loss=1.51e+04, v_num=1]
```

Example 535 (unknown):
```unknown
Epoch 158/500:  32%|███▏      | 158/500 [18:08<39:08,  6.87s/it, loss=1.51e+04, v_num=1]
```

Example 536 (unknown):
```unknown
Epoch 159/500:  32%|███▏      | 158/500 [18:08<39:08,  6.87s/it, loss=1.51e+04, v_num=1]
```

Example 537 (unknown):
```unknown
Epoch 159/500:  32%|███▏      | 159/500 [18:15<38:45,  6.82s/it, loss=1.51e+04, v_num=1]
```

Example 538 (unknown):
```unknown
Epoch 159/500:  32%|███▏      | 159/500 [18:15<38:45,  6.82s/it, loss=1.5e+04, v_num=1]
```

Example 539 (unknown):
```unknown
Epoch 160/500:  32%|███▏      | 159/500 [18:15<38:45,  6.82s/it, loss=1.5e+04, v_num=1]
```

Example 540 (unknown):
```unknown
Epoch 160/500:  32%|███▏      | 160/500 [18:21<38:30,  6.80s/it, loss=1.5e+04, v_num=1]
```

Example 541 (unknown):
```unknown
Epoch 160/500:  32%|███▏      | 160/500 [18:21<38:30,  6.80s/it, loss=1.51e+04, v_num=1]
```

Example 542 (unknown):
```unknown
Epoch 161/500:  32%|███▏      | 160/500 [18:21<38:30,  6.80s/it, loss=1.51e+04, v_num=1]
```

Example 543 (unknown):
```unknown
Epoch 161/500:  32%|███▏      | 161/500 [18:28<38:36,  6.83s/it, loss=1.51e+04, v_num=1]
```

Example 544 (unknown):
```unknown
Epoch 161/500:  32%|███▏      | 161/500 [18:28<38:36,  6.83s/it, loss=1.52e+04, v_num=1]
```

Example 545 (unknown):
```unknown
Epoch 162/500:  32%|███▏      | 161/500 [18:28<38:36,  6.83s/it, loss=1.52e+04, v_num=1]
```

Example 546 (unknown):
```unknown
Epoch 162/500:  32%|███▏      | 162/500 [18:35<38:45,  6.88s/it, loss=1.52e+04, v_num=1]
```

Example 547 (unknown):
```unknown
Epoch 162/500:  32%|███▏      | 162/500 [18:35<38:45,  6.88s/it, loss=1.51e+04, v_num=1]
```

Example 548 (unknown):
```unknown
Epoch 163/500:  32%|███▏      | 162/500 [18:35<38:45,  6.88s/it, loss=1.51e+04, v_num=1]
```

Example 549 (unknown):
```unknown
Epoch 163/500:  33%|███▎      | 163/500 [18:42<38:46,  6.90s/it, loss=1.51e+04, v_num=1]
```

Example 550 (unknown):
```unknown
Epoch 163/500:  33%|███▎      | 163/500 [18:42<38:46,  6.90s/it, loss=1.51e+04, v_num=1]
```

Example 551 (unknown):
```unknown
Epoch 164/500:  33%|███▎      | 163/500 [18:42<38:46,  6.90s/it, loss=1.51e+04, v_num=1]
```

Example 552 (unknown):
```unknown
Epoch 164/500:  33%|███▎      | 164/500 [18:49<38:43,  6.92s/it, loss=1.51e+04, v_num=1]
```

Example 553 (unknown):
```unknown
Epoch 164/500:  33%|███▎      | 164/500 [18:49<38:43,  6.92s/it, loss=1.54e+04, v_num=1]
```

Example 554 (unknown):
```unknown
Epoch 165/500:  33%|███▎      | 164/500 [18:49<38:43,  6.92s/it, loss=1.54e+04, v_num=1]
```

Example 555 (unknown):
```unknown
Epoch 165/500:  33%|███▎      | 165/500 [18:56<38:18,  6.86s/it, loss=1.54e+04, v_num=1]
```

Example 556 (unknown):
```unknown
Epoch 165/500:  33%|███▎      | 165/500 [18:56<38:18,  6.86s/it, loss=1.55e+04, v_num=1]
```

Example 557 (unknown):
```unknown
Epoch 166/500:  33%|███▎      | 165/500 [18:56<38:18,  6.86s/it, loss=1.55e+04, v_num=1]
```

Example 558 (unknown):
```unknown
Epoch 166/500:  33%|███▎      | 166/500 [19:03<38:19,  6.88s/it, loss=1.55e+04, v_num=1]
```

Example 559 (unknown):
```unknown
Epoch 166/500:  33%|███▎      | 166/500 [19:03<38:19,  6.88s/it, loss=1.56e+04, v_num=1]
```

Example 560 (unknown):
```unknown
Epoch 167/500:  33%|███▎      | 166/500 [19:03<38:19,  6.88s/it, loss=1.56e+04, v_num=1]
```

Example 561 (unknown):
```unknown
Epoch 167/500:  33%|███▎      | 167/500 [19:10<37:57,  6.84s/it, loss=1.56e+04, v_num=1]
```

Example 562 (unknown):
```unknown
Epoch 167/500:  33%|███▎      | 167/500 [19:10<37:57,  6.84s/it, loss=1.51e+04, v_num=1]
```

Example 563 (unknown):
```unknown
Epoch 168/500:  33%|███▎      | 167/500 [19:10<37:57,  6.84s/it, loss=1.51e+04, v_num=1]
```

Example 564 (unknown):
```unknown
Epoch 168/500:  34%|███▎      | 168/500 [19:17<38:11,  6.90s/it, loss=1.51e+04, v_num=1]
```

Example 565 (unknown):
```unknown
Epoch 168/500:  34%|███▎      | 168/500 [19:17<38:11,  6.90s/it, loss=1.54e+04, v_num=1]
```

Example 566 (unknown):
```unknown
Epoch 169/500:  34%|███▎      | 168/500 [19:17<38:11,  6.90s/it, loss=1.54e+04, v_num=1]
```

Example 567 (unknown):
```unknown
Epoch 169/500:  34%|███▍      | 169/500 [19:23<38:06,  6.91s/it, loss=1.54e+04, v_num=1]
```

Example 568 (unknown):
```unknown
Epoch 169/500:  34%|███▍      | 169/500 [19:23<38:06,  6.91s/it, loss=1.51e+04, v_num=1]
```

Example 569 (unknown):
```unknown
Epoch 170/500:  34%|███▍      | 169/500 [19:23<38:06,  6.91s/it, loss=1.51e+04, v_num=1]
```

Example 570 (unknown):
```unknown
Epoch 170/500:  34%|███▍      | 170/500 [19:30<38:05,  6.93s/it, loss=1.51e+04, v_num=1]
```

Example 571 (unknown):
```unknown
Epoch 170/500:  34%|███▍      | 170/500 [19:30<38:05,  6.93s/it, loss=1.53e+04, v_num=1]
```

Example 572 (unknown):
```unknown
Epoch 171/500:  34%|███▍      | 170/500 [19:30<38:05,  6.93s/it, loss=1.53e+04, v_num=1]
```

Example 573 (unknown):
```unknown
Epoch 171/500:  34%|███▍      | 171/500 [19:37<38:01,  6.94s/it, loss=1.53e+04, v_num=1]
```

Example 574 (unknown):
```unknown
Epoch 171/500:  34%|███▍      | 171/500 [19:37<38:01,  6.94s/it, loss=1.52e+04, v_num=1]
```

Example 575 (unknown):
```unknown
Epoch 172/500:  34%|███▍      | 171/500 [19:37<38:01,  6.94s/it, loss=1.52e+04, v_num=1]
```

Example 576 (unknown):
```unknown
Epoch 172/500:  34%|███▍      | 172/500 [19:44<37:43,  6.90s/it, loss=1.52e+04, v_num=1]
```

Example 577 (unknown):
```unknown
Epoch 172/500:  34%|███▍      | 172/500 [19:44<37:43,  6.90s/it, loss=1.49e+04, v_num=1]
```

Example 578 (unknown):
```unknown
Epoch 173/500:  34%|███▍      | 172/500 [19:44<37:43,  6.90s/it, loss=1.49e+04, v_num=1]
```

Example 579 (unknown):
```unknown
Epoch 173/500:  35%|███▍      | 173/500 [19:51<37:44,  6.93s/it, loss=1.49e+04, v_num=1]
```

Example 580 (unknown):
```unknown
Epoch 173/500:  35%|███▍      | 173/500 [19:51<37:44,  6.93s/it, loss=1.48e+04, v_num=1]
```

Example 581 (unknown):
```unknown
Epoch 174/500:  35%|███▍      | 173/500 [19:51<37:44,  6.93s/it, loss=1.48e+04, v_num=1]
```

Example 582 (unknown):
```unknown
Epoch 174/500:  35%|███▍      | 174/500 [19:58<37:35,  6.92s/it, loss=1.48e+04, v_num=1]
```

Example 583 (unknown):
```unknown
Epoch 174/500:  35%|███▍      | 174/500 [19:58<37:35,  6.92s/it, loss=1.5e+04, v_num=1]
```

Example 584 (unknown):
```unknown
Epoch 175/500:  35%|███▍      | 174/500 [19:58<37:35,  6.92s/it, loss=1.5e+04, v_num=1]
```

Example 585 (unknown):
```unknown
Epoch 175/500:  35%|███▌      | 175/500 [20:05<37:16,  6.88s/it, loss=1.5e+04, v_num=1]
```

Example 586 (unknown):
```unknown
Epoch 175/500:  35%|███▌      | 175/500 [20:05<37:16,  6.88s/it, loss=1.51e+04, v_num=1]
```

Example 587 (unknown):
```unknown
Epoch 176/500:  35%|███▌      | 175/500 [20:05<37:16,  6.88s/it, loss=1.51e+04, v_num=1]
```

Example 588 (unknown):
```unknown
Epoch 176/500:  35%|███▌      | 176/500 [20:12<37:15,  6.90s/it, loss=1.51e+04, v_num=1]
```

Example 589 (unknown):
```unknown
Epoch 176/500:  35%|███▌      | 176/500 [20:12<37:15,  6.90s/it, loss=1.52e+04, v_num=1]
```

Example 590 (unknown):
```unknown
Epoch 177/500:  35%|███▌      | 176/500 [20:12<37:15,  6.90s/it, loss=1.52e+04, v_num=1]
```

Example 591 (unknown):
```unknown
Epoch 177/500:  35%|███▌      | 177/500 [20:19<37:10,  6.91s/it, loss=1.52e+04, v_num=1]
```

Example 592 (unknown):
```unknown
Epoch 177/500:  35%|███▌      | 177/500 [20:19<37:10,  6.91s/it, loss=1.52e+04, v_num=1]
```

Example 593 (unknown):
```unknown
Epoch 178/500:  35%|███▌      | 177/500 [20:19<37:10,  6.91s/it, loss=1.52e+04, v_num=1]
```

Example 594 (unknown):
```unknown
Epoch 178/500:  36%|███▌      | 178/500 [20:25<36:45,  6.85s/it, loss=1.52e+04, v_num=1]
```

Example 595 (unknown):
```unknown
Epoch 178/500:  36%|███▌      | 178/500 [20:25<36:45,  6.85s/it, loss=1.51e+04, v_num=1]
```

Example 596 (unknown):
```unknown
Epoch 179/500:  36%|███▌      | 178/500 [20:26<36:45,  6.85s/it, loss=1.51e+04, v_num=1]
```

Example 597 (unknown):
```unknown
Epoch 179/500:  36%|███▌      | 179/500 [20:32<36:50,  6.89s/it, loss=1.51e+04, v_num=1]
```

Example 598 (unknown):
```unknown
Epoch 179/500:  36%|███▌      | 179/500 [20:32<36:50,  6.89s/it, loss=1.51e+04, v_num=1]
```

Example 599 (unknown):
```unknown
Epoch 180/500:  36%|███▌      | 179/500 [20:32<36:50,  6.89s/it, loss=1.51e+04, v_num=1]
```

Example 600 (unknown):
```unknown
Epoch 180/500:  36%|███▌      | 180/500 [20:39<36:44,  6.89s/it, loss=1.51e+04, v_num=1]
```

Example 601 (unknown):
```unknown
Epoch 180/500:  36%|███▌      | 180/500 [20:39<36:44,  6.89s/it, loss=1.48e+04, v_num=1]
```

Example 602 (unknown):
```unknown
Epoch 181/500:  36%|███▌      | 180/500 [20:39<36:44,  6.89s/it, loss=1.48e+04, v_num=1]
```

Example 603 (unknown):
```unknown
Epoch 181/500:  36%|███▌      | 181/500 [20:46<36:25,  6.85s/it, loss=1.48e+04, v_num=1]
```

Example 604 (unknown):
```unknown
Epoch 181/500:  36%|███▌      | 181/500 [20:46<36:25,  6.85s/it, loss=1.51e+04, v_num=1]
```

Example 605 (unknown):
```unknown
Epoch 182/500:  36%|███▌      | 181/500 [20:46<36:25,  6.85s/it, loss=1.51e+04, v_num=1]
```

Example 606 (unknown):
```unknown
Epoch 182/500:  36%|███▋      | 182/500 [20:53<36:26,  6.88s/it, loss=1.51e+04, v_num=1]
```

Example 607 (unknown):
```unknown
Epoch 182/500:  36%|███▋      | 182/500 [20:53<36:26,  6.88s/it, loss=1.51e+04, v_num=1]
```

Example 608 (unknown):
```unknown
Epoch 183/500:  36%|███▋      | 182/500 [20:53<36:26,  6.88s/it, loss=1.51e+04, v_num=1]
```

Example 609 (unknown):
```unknown
Epoch 183/500:  37%|███▋      | 183/500 [21:00<36:26,  6.90s/it, loss=1.51e+04, v_num=1]
```

Example 610 (unknown):
```unknown
Epoch 183/500:  37%|███▋      | 183/500 [21:00<36:26,  6.90s/it, loss=1.48e+04, v_num=1]
```

Example 611 (unknown):
```unknown
Epoch 184/500:  37%|███▋      | 183/500 [21:00<36:26,  6.90s/it, loss=1.48e+04, v_num=1]
```

Example 612 (unknown):
```unknown
Epoch 184/500:  37%|███▋      | 184/500 [21:07<36:15,  6.88s/it, loss=1.48e+04, v_num=1]
```

Example 613 (unknown):
```unknown
Epoch 184/500:  37%|███▋      | 184/500 [21:07<36:15,  6.88s/it, loss=1.51e+04, v_num=1]
```

Example 614 (unknown):
```unknown
Epoch 185/500:  37%|███▋      | 184/500 [21:07<36:15,  6.88s/it, loss=1.51e+04, v_num=1]
```

Example 615 (unknown):
```unknown
Epoch 185/500:  37%|███▋      | 185/500 [21:14<36:02,  6.86s/it, loss=1.51e+04, v_num=1]
```

Example 616 (unknown):
```unknown
Epoch 185/500:  37%|███▋      | 185/500 [21:14<36:02,  6.86s/it, loss=1.51e+04, v_num=1]
```

Example 617 (unknown):
```unknown
Epoch 186/500:  37%|███▋      | 185/500 [21:14<36:02,  6.86s/it, loss=1.51e+04, v_num=1]
```

Example 618 (unknown):
```unknown
Epoch 186/500:  37%|███▋      | 186/500 [21:21<36:02,  6.89s/it, loss=1.51e+04, v_num=1]
```

Example 619 (unknown):
```unknown
Epoch 186/500:  37%|███▋      | 186/500 [21:21<36:02,  6.89s/it, loss=1.5e+04, v_num=1]
```

Example 620 (unknown):
```unknown
Epoch 187/500:  37%|███▋      | 186/500 [21:21<36:02,  6.89s/it, loss=1.5e+04, v_num=1]
```

Example 621 (unknown):
```unknown
Epoch 187/500:  37%|███▋      | 187/500 [21:28<35:59,  6.90s/it, loss=1.5e+04, v_num=1]
```

Example 622 (unknown):
```unknown
Epoch 187/500:  37%|███▋      | 187/500 [21:28<35:59,  6.90s/it, loss=1.52e+04, v_num=1]
```

Example 623 (unknown):
```unknown
Epoch 188/500:  37%|███▋      | 187/500 [21:28<35:59,  6.90s/it, loss=1.52e+04, v_num=1]
```

Example 624 (unknown):
```unknown
Epoch 188/500:  38%|███▊      | 188/500 [21:35<36:03,  6.93s/it, loss=1.52e+04, v_num=1]
```

Example 625 (unknown):
```unknown
Epoch 188/500:  38%|███▊      | 188/500 [21:35<36:03,  6.93s/it, loss=1.52e+04, v_num=1]
```

Example 626 (unknown):
```unknown
Epoch 189/500:  38%|███▊      | 188/500 [21:35<36:03,  6.93s/it, loss=1.52e+04, v_num=1]
```

Example 627 (unknown):
```unknown
Epoch 189/500:  38%|███▊      | 189/500 [21:42<35:58,  6.94s/it, loss=1.52e+04, v_num=1]
```

Example 628 (unknown):
```unknown
Epoch 189/500:  38%|███▊      | 189/500 [21:42<35:58,  6.94s/it, loss=1.54e+04, v_num=1]
```

Example 629 (unknown):
```unknown
Epoch 190/500:  38%|███▊      | 189/500 [21:42<35:58,  6.94s/it, loss=1.54e+04, v_num=1]
```

Example 630 (unknown):
```unknown
Epoch 190/500:  38%|███▊      | 190/500 [21:48<35:16,  6.83s/it, loss=1.54e+04, v_num=1]
```

Example 631 (unknown):
```unknown
Epoch 190/500:  38%|███▊      | 190/500 [21:48<35:16,  6.83s/it, loss=1.53e+04, v_num=1]
```

Example 632 (unknown):
```unknown
Epoch 191/500:  38%|███▊      | 190/500 [21:48<35:16,  6.83s/it, loss=1.53e+04, v_num=1]
```

Example 633 (unknown):
```unknown
Epoch 191/500:  38%|███▊      | 191/500 [21:55<35:20,  6.86s/it, loss=1.53e+04, v_num=1]
```

Example 634 (unknown):
```unknown
Epoch 191/500:  38%|███▊      | 191/500 [21:55<35:20,  6.86s/it, loss=1.52e+04, v_num=1]
```

Example 635 (unknown):
```unknown
Epoch 192/500:  38%|███▊      | 191/500 [21:55<35:20,  6.86s/it, loss=1.52e+04, v_num=1]
```

Example 636 (unknown):
```unknown
Epoch 192/500:  38%|███▊      | 192/500 [22:02<35:22,  6.89s/it, loss=1.52e+04, v_num=1]
```

Example 637 (unknown):
```unknown
Epoch 192/500:  38%|███▊      | 192/500 [22:02<35:22,  6.89s/it, loss=1.5e+04, v_num=1]
```

Example 638 (unknown):
```unknown
Epoch 193/500:  38%|███▊      | 192/500 [22:02<35:22,  6.89s/it, loss=1.5e+04, v_num=1]
```

Example 639 (unknown):
```unknown
Epoch 193/500:  39%|███▊      | 193/500 [22:09<34:49,  6.81s/it, loss=1.5e+04, v_num=1]
```

Example 640 (unknown):
```unknown
Epoch 193/500:  39%|███▊      | 193/500 [22:09<34:49,  6.81s/it, loss=1.51e+04, v_num=1]
```

Example 641 (unknown):
```unknown
Epoch 194/500:  39%|███▊      | 193/500 [22:09<34:49,  6.81s/it, loss=1.51e+04, v_num=1]
```

Example 642 (unknown):
```unknown
Epoch 194/500:  39%|███▉      | 194/500 [22:15<34:50,  6.83s/it, loss=1.51e+04, v_num=1]
```

Example 643 (unknown):
```unknown
Epoch 194/500:  39%|███▉      | 194/500 [22:15<34:50,  6.83s/it, loss=1.51e+04, v_num=1]
```

Example 644 (unknown):
```unknown
Epoch 195/500:  39%|███▉      | 194/500 [22:15<34:50,  6.83s/it, loss=1.51e+04, v_num=1]
```

Example 645 (unknown):
```unknown
Epoch 195/500:  39%|███▉      | 195/500 [22:23<35:03,  6.90s/it, loss=1.51e+04, v_num=1]
```

Example 646 (unknown):
```unknown
Epoch 195/500:  39%|███▉      | 195/500 [22:23<35:03,  6.90s/it, loss=1.5e+04, v_num=1]
```

Example 647 (unknown):
```unknown
Epoch 196/500:  39%|███▉      | 195/500 [22:23<35:03,  6.90s/it, loss=1.5e+04, v_num=1]
```

Example 648 (unknown):
```unknown
Epoch 196/500:  39%|███▉      | 196/500 [22:30<35:09,  6.94s/it, loss=1.5e+04, v_num=1]
```

Example 649 (unknown):
```unknown
Epoch 196/500:  39%|███▉      | 196/500 [22:30<35:09,  6.94s/it, loss=1.52e+04, v_num=1]
```

Example 650 (unknown):
```unknown
Epoch 197/500:  39%|███▉      | 196/500 [22:30<35:09,  6.94s/it, loss=1.52e+04, v_num=1]
```

Example 651 (unknown):
```unknown
Epoch 197/500:  39%|███▉      | 197/500 [22:36<34:43,  6.88s/it, loss=1.52e+04, v_num=1]
```

Example 652 (unknown):
```unknown
Epoch 197/500:  39%|███▉      | 197/500 [22:36<34:43,  6.88s/it, loss=1.51e+04, v_num=1]
```

Example 653 (unknown):
```unknown
Epoch 198/500:  39%|███▉      | 197/500 [22:36<34:43,  6.88s/it, loss=1.51e+04, v_num=1]
```

Example 654 (unknown):
```unknown
Epoch 198/500:  40%|███▉      | 198/500 [22:43<34:31,  6.86s/it, loss=1.51e+04, v_num=1]
```

Example 655 (unknown):
```unknown
Epoch 198/500:  40%|███▉      | 198/500 [22:43<34:31,  6.86s/it, loss=1.51e+04, v_num=1]
```

Example 656 (unknown):
```unknown
Epoch 199/500:  40%|███▉      | 198/500 [22:43<34:31,  6.86s/it, loss=1.51e+04, v_num=1]
```

Example 657 (unknown):
```unknown
Epoch 199/500:  40%|███▉      | 199/500 [22:50<33:58,  6.77s/it, loss=1.51e+04, v_num=1]
```

Example 658 (unknown):
```unknown
Epoch 199/500:  40%|███▉      | 199/500 [22:50<33:58,  6.77s/it, loss=1.49e+04, v_num=1]
```

Example 659 (unknown):
```unknown
Epoch 200/500:  40%|███▉      | 199/500 [22:50<33:58,  6.77s/it, loss=1.49e+04, v_num=1]
```

Example 660 (unknown):
```unknown
Epoch 200/500:  40%|████      | 200/500 [22:57<34:01,  6.80s/it, loss=1.49e+04, v_num=1]
```

Example 661 (unknown):
```unknown
Epoch 200/500:  40%|████      | 200/500 [22:57<34:01,  6.80s/it, loss=1.5e+04, v_num=1]
```

Example 662 (unknown):
```unknown
Epoch 201/500:  40%|████      | 200/500 [22:57<34:01,  6.80s/it, loss=1.5e+04, v_num=1]
```

Example 663 (unknown):
```unknown
Epoch 201/500:  40%|████      | 201/500 [23:04<34:05,  6.84s/it, loss=1.5e+04, v_num=1]
```

Example 664 (unknown):
```unknown
Epoch 201/500:  40%|████      | 201/500 [23:04<34:05,  6.84s/it, loss=1.5e+04, v_num=1]
```

Example 665 (unknown):
```unknown
Epoch 202/500:  40%|████      | 201/500 [23:04<34:05,  6.84s/it, loss=1.5e+04, v_num=1]
```

Example 666 (unknown):
```unknown
Epoch 202/500:  40%|████      | 202/500 [23:10<34:06,  6.87s/it, loss=1.5e+04, v_num=1]
```

Example 667 (unknown):
```unknown
Epoch 202/500:  40%|████      | 202/500 [23:10<34:06,  6.87s/it, loss=1.52e+04, v_num=1]
```

Example 668 (unknown):
```unknown
Epoch 203/500:  40%|████      | 202/500 [23:10<34:06,  6.87s/it, loss=1.52e+04, v_num=1]
```

Example 669 (unknown):
```unknown
Epoch 203/500:  41%|████      | 203/500 [23:17<34:08,  6.90s/it, loss=1.52e+04, v_num=1]
```

Example 670 (unknown):
```unknown
Epoch 203/500:  41%|████      | 203/500 [23:17<34:08,  6.90s/it, loss=1.51e+04, v_num=1]
```

Example 671 (unknown):
```unknown
Epoch 204/500:  41%|████      | 203/500 [23:17<34:08,  6.90s/it, loss=1.51e+04, v_num=1]
```

Example 672 (unknown):
```unknown
Epoch 204/500:  41%|████      | 204/500 [23:24<33:53,  6.87s/it, loss=1.51e+04, v_num=1]
```

Example 673 (unknown):
```unknown
Epoch 204/500:  41%|████      | 204/500 [23:24<33:53,  6.87s/it, loss=1.51e+04, v_num=1]
```

Example 674 (unknown):
```unknown
Epoch 205/500:  41%|████      | 204/500 [23:24<33:53,  6.87s/it, loss=1.51e+04, v_num=1]
```

Example 675 (unknown):
```unknown
Epoch 205/500:  41%|████      | 205/500 [23:31<33:23,  6.79s/it, loss=1.51e+04, v_num=1]
```

Example 676 (unknown):
```unknown
Epoch 205/500:  41%|████      | 205/500 [23:31<33:23,  6.79s/it, loss=1.54e+04, v_num=1]
```

Example 677 (unknown):
```unknown
Epoch 206/500:  41%|████      | 205/500 [23:31<33:23,  6.79s/it, loss=1.54e+04, v_num=1]
```

Example 678 (unknown):
```unknown
Epoch 206/500:  41%|████      | 206/500 [23:38<33:26,  6.83s/it, loss=1.54e+04, v_num=1]
```

Example 679 (unknown):
```unknown
Epoch 206/500:  41%|████      | 206/500 [23:38<33:26,  6.83s/it, loss=1.51e+04, v_num=1]
```

Example 680 (unknown):
```unknown
Epoch 207/500:  41%|████      | 206/500 [23:38<33:26,  6.83s/it, loss=1.51e+04, v_num=1]
```

Example 681 (unknown):
```unknown
Epoch 207/500:  41%|████▏     | 207/500 [23:45<33:35,  6.88s/it, loss=1.51e+04, v_num=1]
```

Example 682 (unknown):
```unknown
Epoch 207/500:  41%|████▏     | 207/500 [23:45<33:35,  6.88s/it, loss=1.52e+04, v_num=1]
```

Example 683 (unknown):
```unknown
Epoch 208/500:  41%|████▏     | 207/500 [23:45<33:35,  6.88s/it, loss=1.52e+04, v_num=1]
```

Example 684 (unknown):
```unknown
Epoch 208/500:  42%|████▏     | 208/500 [23:52<33:38,  6.91s/it, loss=1.52e+04, v_num=1]
```

Example 685 (unknown):
```unknown
Epoch 208/500:  42%|████▏     | 208/500 [23:52<33:38,  6.91s/it, loss=1.49e+04, v_num=1]
```

Example 686 (unknown):
```unknown
Epoch 209/500:  42%|████▏     | 208/500 [23:52<33:38,  6.91s/it, loss=1.49e+04, v_num=1]
```

Example 687 (unknown):
```unknown
Epoch 209/500:  42%|████▏     | 209/500 [23:58<33:08,  6.83s/it, loss=1.49e+04, v_num=1]
```

Example 688 (unknown):
```unknown
Epoch 209/500:  42%|████▏     | 209/500 [23:58<33:08,  6.83s/it, loss=1.49e+04, v_num=1]
```

Example 689 (unknown):
```unknown
Epoch 210/500:  42%|████▏     | 209/500 [23:58<33:08,  6.83s/it, loss=1.49e+04, v_num=1]
```

Example 690 (unknown):
```unknown
Epoch 210/500:  42%|████▏     | 210/500 [24:05<33:00,  6.83s/it, loss=1.49e+04, v_num=1]
```

Example 691 (unknown):
```unknown
Epoch 210/500:  42%|████▏     | 210/500 [24:05<33:00,  6.83s/it, loss=1.52e+04, v_num=1]
```

Example 692 (unknown):
```unknown
Epoch 211/500:  42%|████▏     | 210/500 [24:05<33:00,  6.83s/it, loss=1.52e+04, v_num=1]
```

Example 693 (unknown):
```unknown
Epoch 211/500:  42%|████▏     | 211/500 [24:12<32:40,  6.78s/it, loss=1.52e+04, v_num=1]
```

Example 694 (unknown):
```unknown
Epoch 211/500:  42%|████▏     | 211/500 [24:12<32:40,  6.78s/it, loss=1.49e+04, v_num=1]
```

Example 695 (unknown):
```unknown
Epoch 212/500:  42%|████▏     | 211/500 [24:12<32:40,  6.78s/it, loss=1.49e+04, v_num=1]
```

Example 696 (unknown):
```unknown
Epoch 212/500:  42%|████▏     | 212/500 [24:19<32:50,  6.84s/it, loss=1.49e+04, v_num=1]
```

Example 697 (unknown):
```unknown
Epoch 212/500:  42%|████▏     | 212/500 [24:19<32:50,  6.84s/it, loss=1.52e+04, v_num=1]
```

Example 698 (unknown):
```unknown
Epoch 213/500:  42%|████▏     | 212/500 [24:19<32:50,  6.84s/it, loss=1.52e+04, v_num=1]
```

Example 699 (unknown):
```unknown
Epoch 213/500:  43%|████▎     | 213/500 [24:26<32:39,  6.83s/it, loss=1.52e+04, v_num=1]
```

Example 700 (unknown):
```unknown
Epoch 213/500:  43%|████▎     | 213/500 [24:26<32:39,  6.83s/it, loss=1.49e+04, v_num=1]
```

Example 701 (unknown):
```unknown
Epoch 214/500:  43%|████▎     | 213/500 [24:26<32:39,  6.83s/it, loss=1.49e+04, v_num=1]
```

Example 702 (unknown):
```unknown
Epoch 214/500:  43%|████▎     | 214/500 [24:32<32:24,  6.80s/it, loss=1.49e+04, v_num=1]
```

Example 703 (unknown):
```unknown
Epoch 214/500:  43%|████▎     | 214/500 [24:32<32:24,  6.80s/it, loss=1.49e+04, v_num=1]
```

Example 704 (unknown):
```unknown
Epoch 215/500:  43%|████▎     | 214/500 [24:32<32:24,  6.80s/it, loss=1.49e+04, v_num=1]
```

Example 705 (unknown):
```unknown
Epoch 215/500:  43%|████▎     | 215/500 [24:39<32:23,  6.82s/it, loss=1.49e+04, v_num=1]
```

Example 706 (unknown):
```unknown
Epoch 215/500:  43%|████▎     | 215/500 [24:39<32:23,  6.82s/it, loss=1.52e+04, v_num=1]
```

Example 707 (unknown):
```unknown
Epoch 216/500:  43%|████▎     | 215/500 [24:39<32:23,  6.82s/it, loss=1.52e+04, v_num=1]
```

Example 708 (unknown):
```unknown
Epoch 216/500:  43%|████▎     | 216/500 [24:46<32:32,  6.87s/it, loss=1.52e+04, v_num=1]
```

Example 709 (unknown):
```unknown
Epoch 216/500:  43%|████▎     | 216/500 [24:46<32:32,  6.87s/it, loss=1.53e+04, v_num=1]
```

Example 710 (unknown):
```unknown
Epoch 217/500:  43%|████▎     | 216/500 [24:46<32:32,  6.87s/it, loss=1.53e+04, v_num=1]
```

Example 711 (unknown):
```unknown
Epoch 217/500:  43%|████▎     | 217/500 [24:53<32:23,  6.87s/it, loss=1.53e+04, v_num=1]
```

Example 712 (unknown):
```unknown
Epoch 217/500:  43%|████▎     | 217/500 [24:53<32:23,  6.87s/it, loss=1.48e+04, v_num=1]
```

Example 713 (unknown):
```unknown
Epoch 218/500:  43%|████▎     | 217/500 [24:53<32:23,  6.87s/it, loss=1.48e+04, v_num=1]
```

Example 714 (unknown):
```unknown
Epoch 218/500:  44%|████▎     | 218/500 [25:00<32:29,  6.91s/it, loss=1.48e+04, v_num=1]
```

Example 715 (unknown):
```unknown
Epoch 218/500:  44%|████▎     | 218/500 [25:00<32:29,  6.91s/it, loss=1.5e+04, v_num=1]
```

Example 716 (unknown):
```unknown
Epoch 219/500:  44%|████▎     | 218/500 [25:00<32:29,  6.91s/it, loss=1.5e+04, v_num=1]
```

Example 717 (unknown):
```unknown
Epoch 219/500:  44%|████▍     | 219/500 [25:07<32:05,  6.85s/it, loss=1.5e+04, v_num=1]
```

Example 718 (unknown):
```unknown
Epoch 219/500:  44%|████▍     | 219/500 [25:07<32:05,  6.85s/it, loss=1.56e+04, v_num=1]
```

Example 719 (unknown):
```unknown
Epoch 220/500:  44%|████▍     | 219/500 [25:07<32:05,  6.85s/it, loss=1.56e+04, v_num=1]
```

Example 720 (unknown):
```unknown
Epoch 220/500:  44%|████▍     | 220/500 [25:14<32:04,  6.87s/it, loss=1.56e+04, v_num=1]
```

Example 721 (unknown):
```unknown
Epoch 220/500:  44%|████▍     | 220/500 [25:14<32:04,  6.87s/it, loss=1.5e+04, v_num=1]
```

Example 722 (unknown):
```unknown
Epoch 221/500:  44%|████▍     | 220/500 [25:14<32:04,  6.87s/it, loss=1.5e+04, v_num=1]
```

Example 723 (unknown):
```unknown
Epoch 221/500:  44%|████▍     | 221/500 [25:21<31:56,  6.87s/it, loss=1.5e+04, v_num=1]
```

Example 724 (unknown):
```unknown
Epoch 221/500:  44%|████▍     | 221/500 [25:21<31:56,  6.87s/it, loss=1.5e+04, v_num=1]
```

Example 725 (unknown):
```unknown
Epoch 222/500:  44%|████▍     | 221/500 [25:21<31:56,  6.87s/it, loss=1.5e+04, v_num=1]
```

Example 726 (unknown):
```unknown
Epoch 222/500:  44%|████▍     | 222/500 [25:28<31:53,  6.88s/it, loss=1.5e+04, v_num=1]
```

Example 727 (unknown):
```unknown
Epoch 222/500:  44%|████▍     | 222/500 [25:28<31:53,  6.88s/it, loss=1.55e+04, v_num=1]
```

Example 728 (unknown):
```unknown
Epoch 223/500:  44%|████▍     | 222/500 [25:28<31:53,  6.88s/it, loss=1.55e+04, v_num=1]
```

Example 729 (unknown):
```unknown
Epoch 223/500:  45%|████▍     | 223/500 [25:35<31:55,  6.92s/it, loss=1.55e+04, v_num=1]
```

Example 730 (unknown):
```unknown
Epoch 223/500:  45%|████▍     | 223/500 [25:35<31:55,  6.92s/it, loss=1.49e+04, v_num=1]
```

Example 731 (unknown):
```unknown
Epoch 224/500:  45%|████▍     | 223/500 [25:35<31:55,  6.92s/it, loss=1.49e+04, v_num=1]
```

Example 732 (unknown):
```unknown
Epoch 224/500:  45%|████▍     | 224/500 [25:42<32:02,  6.97s/it, loss=1.49e+04, v_num=1]
```

Example 733 (unknown):
```unknown
Epoch 224/500:  45%|████▍     | 224/500 [25:42<32:02,  6.97s/it, loss=1.5e+04, v_num=1]
```

Example 734 (unknown):
```unknown
Epoch 225/500:  45%|████▍     | 224/500 [25:42<32:02,  6.97s/it, loss=1.5e+04, v_num=1]
```

Example 735 (unknown):
```unknown
Epoch 225/500:  45%|████▌     | 225/500 [25:49<31:56,  6.97s/it, loss=1.5e+04, v_num=1]
```

Example 736 (unknown):
```unknown
Epoch 225/500:  45%|████▌     | 225/500 [25:49<31:56,  6.97s/it, loss=1.51e+04, v_num=1]
```

Example 737 (unknown):
```unknown
Epoch 226/500:  45%|████▌     | 225/500 [25:49<31:56,  6.97s/it, loss=1.51e+04, v_num=1]
```

Example 738 (unknown):
```unknown
Epoch 226/500:  45%|████▌     | 226/500 [25:56<31:49,  6.97s/it, loss=1.51e+04, v_num=1]
```

Example 739 (unknown):
```unknown
Epoch 226/500:  45%|████▌     | 226/500 [25:56<31:49,  6.97s/it, loss=1.47e+04, v_num=1]
```

Example 740 (unknown):
```unknown
Epoch 227/500:  45%|████▌     | 226/500 [25:56<31:49,  6.97s/it, loss=1.47e+04, v_num=1]
```

Example 741 (unknown):
```unknown
Epoch 227/500:  45%|████▌     | 227/500 [26:02<31:40,  6.96s/it, loss=1.47e+04, v_num=1]
```

Example 742 (unknown):
```unknown
Epoch 227/500:  45%|████▌     | 227/500 [26:02<31:40,  6.96s/it, loss=1.51e+04, v_num=1]
```

Example 743 (unknown):
```unknown
Epoch 228/500:  45%|████▌     | 227/500 [26:02<31:40,  6.96s/it, loss=1.51e+04, v_num=1]
```

Example 744 (unknown):
```unknown
Epoch 228/500:  46%|████▌     | 228/500 [26:09<31:01,  6.84s/it, loss=1.51e+04, v_num=1]
```

Example 745 (unknown):
```unknown
Epoch 228/500:  46%|████▌     | 228/500 [26:09<31:01,  6.84s/it, loss=1.5e+04, v_num=1]
```

Example 746 (unknown):
```unknown
Epoch 229/500:  46%|████▌     | 228/500 [26:09<31:01,  6.84s/it, loss=1.5e+04, v_num=1]
```

Example 747 (unknown):
```unknown
Epoch 229/500:  46%|████▌     | 229/500 [26:16<30:52,  6.84s/it, loss=1.5e+04, v_num=1]
```

Example 748 (unknown):
```unknown
Epoch 229/500:  46%|████▌     | 229/500 [26:16<30:52,  6.84s/it, loss=1.52e+04, v_num=1]
```

Example 749 (unknown):
```unknown
Epoch 230/500:  46%|████▌     | 229/500 [26:16<30:52,  6.84s/it, loss=1.52e+04, v_num=1]
```

Example 750 (unknown):
```unknown
Epoch 230/500:  46%|████▌     | 230/500 [26:23<30:37,  6.81s/it, loss=1.52e+04, v_num=1]
```

Example 751 (unknown):
```unknown
Epoch 230/500:  46%|████▌     | 230/500 [26:23<30:37,  6.81s/it, loss=1.52e+04, v_num=1]
```

Example 752 (unknown):
```unknown
Epoch 231/500:  46%|████▌     | 230/500 [26:23<30:37,  6.81s/it, loss=1.52e+04, v_num=1]
```

Example 753 (unknown):
```unknown
Epoch 231/500:  46%|████▌     | 231/500 [26:30<30:39,  6.84s/it, loss=1.52e+04, v_num=1]
```

Example 754 (unknown):
```unknown
Epoch 231/500:  46%|████▌     | 231/500 [26:30<30:39,  6.84s/it, loss=1.51e+04, v_num=1]
```

Example 755 (unknown):
```unknown
Epoch 232/500:  46%|████▌     | 231/500 [26:30<30:39,  6.84s/it, loss=1.51e+04, v_num=1]
```

Example 756 (unknown):
```unknown
Epoch 232/500:  46%|████▋     | 232/500 [26:36<30:43,  6.88s/it, loss=1.51e+04, v_num=1]
```

Example 757 (unknown):
```unknown
Epoch 232/500:  46%|████▋     | 232/500 [26:36<30:43,  6.88s/it, loss=1.54e+04, v_num=1]
```

Example 758 (unknown):
```unknown
Epoch 233/500:  46%|████▋     | 232/500 [26:36<30:43,  6.88s/it, loss=1.54e+04, v_num=1]
```

Example 759 (unknown):
```unknown
Epoch 233/500:  47%|████▋     | 233/500 [26:43<30:45,  6.91s/it, loss=1.54e+04, v_num=1]
```

Example 760 (unknown):
```unknown
Epoch 233/500:  47%|████▋     | 233/500 [26:43<30:45,  6.91s/it, loss=1.5e+04, v_num=1]
```

Example 761 (unknown):
```unknown
Epoch 234/500:  47%|████▋     | 233/500 [26:43<30:45,  6.91s/it, loss=1.5e+04, v_num=1]
```

Example 762 (unknown):
```unknown
Epoch 234/500:  47%|████▋     | 234/500 [26:50<30:41,  6.92s/it, loss=1.5e+04, v_num=1]
```

Example 763 (unknown):
```unknown
Epoch 234/500:  47%|████▋     | 234/500 [26:50<30:41,  6.92s/it, loss=1.49e+04, v_num=1]
```

Example 764 (unknown):
```unknown
Epoch 235/500:  47%|████▋     | 234/500 [26:50<30:41,  6.92s/it, loss=1.49e+04, v_num=1]
```

Example 765 (unknown):
```unknown
Epoch 235/500:  47%|████▋     | 235/500 [26:57<30:34,  6.92s/it, loss=1.49e+04, v_num=1]
```

Example 766 (unknown):
```unknown
Epoch 235/500:  47%|████▋     | 235/500 [26:57<30:34,  6.92s/it, loss=1.51e+04, v_num=1]
```

Example 767 (unknown):
```unknown
Epoch 236/500:  47%|████▋     | 235/500 [26:57<30:34,  6.92s/it, loss=1.51e+04, v_num=1]
```

Example 768 (unknown):
```unknown
Epoch 236/500:  47%|████▋     | 236/500 [27:04<30:31,  6.94s/it, loss=1.51e+04, v_num=1]
```

Example 769 (unknown):
```unknown
Epoch 236/500:  47%|████▋     | 236/500 [27:04<30:31,  6.94s/it, loss=1.52e+04, v_num=1]
```

Example 770 (unknown):
```unknown
Epoch 237/500:  47%|████▋     | 236/500 [27:04<30:31,  6.94s/it, loss=1.52e+04, v_num=1]
```

Example 771 (unknown):
```unknown
Epoch 237/500:  47%|████▋     | 237/500 [27:11<30:31,  6.97s/it, loss=1.52e+04, v_num=1]
```

Example 772 (unknown):
```unknown
Epoch 237/500:  47%|████▋     | 237/500 [27:11<30:31,  6.97s/it, loss=1.48e+04, v_num=1]
```

Example 773 (unknown):
```unknown
Epoch 238/500:  47%|████▋     | 237/500 [27:11<30:31,  6.97s/it, loss=1.48e+04, v_num=1]
```

Example 774 (unknown):
```unknown
Epoch 238/500:  48%|████▊     | 238/500 [27:18<30:23,  6.96s/it, loss=1.48e+04, v_num=1]
```

Example 775 (unknown):
```unknown
Epoch 238/500:  48%|████▊     | 238/500 [27:18<30:23,  6.96s/it, loss=1.51e+04, v_num=1]
```

Example 776 (unknown):
```unknown
Epoch 239/500:  48%|████▊     | 238/500 [27:18<30:23,  6.96s/it, loss=1.51e+04, v_num=1]
```

Example 777 (unknown):
```unknown
Epoch 239/500:  48%|████▊     | 239/500 [27:25<30:11,  6.94s/it, loss=1.51e+04, v_num=1]
```

Example 778 (unknown):
```unknown
Epoch 239/500:  48%|████▊     | 239/500 [27:25<30:11,  6.94s/it, loss=1.49e+04, v_num=1]
```

Example 779 (unknown):
```unknown
Epoch 240/500:  48%|████▊     | 239/500 [27:25<30:11,  6.94s/it, loss=1.49e+04, v_num=1]
```

Example 780 (unknown):
```unknown
Epoch 240/500:  48%|████▊     | 240/500 [27:32<30:06,  6.95s/it, loss=1.49e+04, v_num=1]
```

Example 781 (unknown):
```unknown
Epoch 240/500:  48%|████▊     | 240/500 [27:32<30:06,  6.95s/it, loss=1.51e+04, v_num=1]
```

Example 782 (unknown):
```unknown
Epoch 241/500:  48%|████▊     | 240/500 [27:32<30:06,  6.95s/it, loss=1.51e+04, v_num=1]
```

Example 783 (unknown):
```unknown
Epoch 241/500:  48%|████▊     | 241/500 [27:39<30:03,  6.96s/it, loss=1.51e+04, v_num=1]
```

Example 784 (unknown):
```unknown
Epoch 241/500:  48%|████▊     | 241/500 [27:39<30:03,  6.96s/it, loss=1.5e+04, v_num=1]
```

Example 785 (unknown):
```unknown
Epoch 242/500:  48%|████▊     | 241/500 [27:39<30:03,  6.96s/it, loss=1.5e+04, v_num=1]
```

Example 786 (unknown):
```unknown
Epoch 242/500:  48%|████▊     | 242/500 [27:46<29:46,  6.93s/it, loss=1.5e+04, v_num=1]
```

Example 787 (unknown):
```unknown
Epoch 242/500:  48%|████▊     | 242/500 [27:46<29:46,  6.93s/it, loss=1.53e+04, v_num=1]
```

Example 788 (unknown):
```unknown
Epoch 243/500:  48%|████▊     | 242/500 [27:46<29:46,  6.93s/it, loss=1.53e+04, v_num=1]
```

Example 789 (unknown):
```unknown
Epoch 243/500:  49%|████▊     | 243/500 [27:53<29:37,  6.92s/it, loss=1.53e+04, v_num=1]
```

Example 790 (unknown):
```unknown
Epoch 243/500:  49%|████▊     | 243/500 [27:53<29:37,  6.92s/it, loss=1.54e+04, v_num=1]
```

Example 791 (unknown):
```unknown
Epoch 244/500:  49%|████▊     | 243/500 [27:53<29:37,  6.92s/it, loss=1.54e+04, v_num=1]
```

Example 792 (unknown):
```unknown
Epoch 244/500:  49%|████▉     | 244/500 [28:00<29:11,  6.84s/it, loss=1.54e+04, v_num=1]
```

Example 793 (unknown):
```unknown
Epoch 244/500:  49%|████▉     | 244/500 [28:00<29:11,  6.84s/it, loss=1.48e+04, v_num=1]
```

Example 794 (unknown):
```unknown
Epoch 245/500:  49%|████▉     | 244/500 [28:00<29:11,  6.84s/it, loss=1.48e+04, v_num=1]
```

Example 795 (unknown):
```unknown
Epoch 245/500:  49%|████▉     | 245/500 [28:07<29:12,  6.87s/it, loss=1.48e+04, v_num=1]
```

Example 796 (unknown):
```unknown
Epoch 245/500:  49%|████▉     | 245/500 [28:07<29:12,  6.87s/it, loss=1.49e+04, v_num=1]
```

Example 797 (unknown):
```unknown
Epoch 246/500:  49%|████▉     | 245/500 [28:07<29:12,  6.87s/it, loss=1.49e+04, v_num=1]
```

Example 798 (unknown):
```unknown
Epoch 246/500:  49%|████▉     | 246/500 [28:13<29:05,  6.87s/it, loss=1.49e+04, v_num=1]
```

Example 799 (unknown):
```unknown
Epoch 246/500:  49%|████▉     | 246/500 [28:13<29:05,  6.87s/it, loss=1.51e+04, v_num=1]
```

Example 800 (unknown):
```unknown
Epoch 247/500:  49%|████▉     | 246/500 [28:13<29:05,  6.87s/it, loss=1.51e+04, v_num=1]
```

Example 801 (unknown):
```unknown
Epoch 247/500:  49%|████▉     | 247/500 [28:20<29:06,  6.90s/it, loss=1.51e+04, v_num=1]
```

Example 802 (unknown):
```unknown
Epoch 247/500:  49%|████▉     | 247/500 [28:20<29:06,  6.90s/it, loss=1.52e+04, v_num=1]
```

Example 803 (unknown):
```unknown
Epoch 248/500:  49%|████▉     | 247/500 [28:20<29:06,  6.90s/it, loss=1.52e+04, v_num=1]
```

Example 804 (unknown):
```unknown
Epoch 248/500:  50%|████▉     | 248/500 [28:27<28:54,  6.88s/it, loss=1.52e+04, v_num=1]
```

Example 805 (unknown):
```unknown
Epoch 248/500:  50%|████▉     | 248/500 [28:27<28:54,  6.88s/it, loss=1.51e+04, v_num=1]
```

Example 806 (unknown):
```unknown
Epoch 249/500:  50%|████▉     | 248/500 [28:27<28:54,  6.88s/it, loss=1.51e+04, v_num=1]
```

Example 807 (unknown):
```unknown
Epoch 249/500:  50%|████▉     | 249/500 [28:34<28:36,  6.84s/it, loss=1.51e+04, v_num=1]
```

Example 808 (unknown):
```unknown
Epoch 249/500:  50%|████▉     | 249/500 [28:34<28:36,  6.84s/it, loss=1.49e+04, v_num=1]
```

Example 809 (unknown):
```unknown
Epoch 250/500:  50%|████▉     | 249/500 [28:34<28:36,  6.84s/it, loss=1.49e+04, v_num=1]
```

Example 810 (unknown):
```unknown
Epoch 250/500:  50%|█████     | 250/500 [28:41<28:14,  6.78s/it, loss=1.49e+04, v_num=1]
```

Example 811 (unknown):
```unknown
Epoch 250/500:  50%|█████     | 250/500 [28:41<28:14,  6.78s/it, loss=1.51e+04, v_num=1]
```

Example 812 (unknown):
```unknown
Epoch 251/500:  50%|█████     | 250/500 [28:41<28:14,  6.78s/it, loss=1.51e+04, v_num=1]
```

Example 813 (unknown):
```unknown
Epoch 251/500:  50%|█████     | 251/500 [28:48<28:21,  6.83s/it, loss=1.51e+04, v_num=1]
```

Example 814 (unknown):
```unknown
Epoch 251/500:  50%|█████     | 251/500 [28:48<28:21,  6.83s/it, loss=1.53e+04, v_num=1]
```

Example 815 (unknown):
```unknown
Epoch 252/500:  50%|█████     | 251/500 [28:48<28:21,  6.83s/it, loss=1.53e+04, v_num=1]
```

Example 816 (unknown):
```unknown
Epoch 252/500:  50%|█████     | 252/500 [28:54<28:06,  6.80s/it, loss=1.53e+04, v_num=1]
```

Example 817 (unknown):
```unknown
Epoch 252/500:  50%|█████     | 252/500 [28:54<28:06,  6.80s/it, loss=1.48e+04, v_num=1]
```

Example 818 (unknown):
```unknown
Epoch 253/500:  50%|█████     | 252/500 [28:54<28:06,  6.80s/it, loss=1.48e+04, v_num=1]
```

Example 819 (unknown):
```unknown
Epoch 253/500:  51%|█████     | 253/500 [29:01<28:07,  6.83s/it, loss=1.48e+04, v_num=1]
```

Example 820 (unknown):
```unknown
Epoch 253/500:  51%|█████     | 253/500 [29:01<28:07,  6.83s/it, loss=1.54e+04, v_num=1]
```

Example 821 (unknown):
```unknown
Epoch 254/500:  51%|█████     | 253/500 [29:01<28:07,  6.83s/it, loss=1.54e+04, v_num=1]
```

Example 822 (unknown):
```unknown
Epoch 254/500:  51%|█████     | 254/500 [29:08<28:00,  6.83s/it, loss=1.54e+04, v_num=1]
```

Example 823 (unknown):
```unknown
Epoch 254/500:  51%|█████     | 254/500 [29:08<28:00,  6.83s/it, loss=1.47e+04, v_num=1]
```

Example 824 (unknown):
```unknown
Epoch 255/500:  51%|█████     | 254/500 [29:08<28:00,  6.83s/it, loss=1.47e+04, v_num=1]
```

Example 825 (unknown):
```unknown
Epoch 255/500:  51%|█████     | 255/500 [29:15<27:48,  6.81s/it, loss=1.47e+04, v_num=1]
```

Example 826 (unknown):
```unknown
Epoch 255/500:  51%|█████     | 255/500 [29:15<27:48,  6.81s/it, loss=1.49e+04, v_num=1]
```

Example 827 (unknown):
```unknown
Epoch 256/500:  51%|█████     | 255/500 [29:15<27:48,  6.81s/it, loss=1.49e+04, v_num=1]
```

Example 828 (unknown):
```unknown
Epoch 256/500:  51%|█████     | 256/500 [29:22<27:44,  6.82s/it, loss=1.49e+04, v_num=1]
```

Example 829 (unknown):
```unknown
Epoch 256/500:  51%|█████     | 256/500 [29:22<27:44,  6.82s/it, loss=1.5e+04, v_num=1]
```

Example 830 (unknown):
```unknown
Epoch 257/500:  51%|█████     | 256/500 [29:22<27:44,  6.82s/it, loss=1.5e+04, v_num=1]
```

Example 831 (unknown):
```unknown
Epoch 257/500:  51%|█████▏    | 257/500 [29:29<27:50,  6.88s/it, loss=1.5e+04, v_num=1]
```

Example 832 (unknown):
```unknown
Epoch 257/500:  51%|█████▏    | 257/500 [29:29<27:50,  6.88s/it, loss=1.51e+04, v_num=1]
```

Example 833 (unknown):
```unknown
Epoch 258/500:  51%|█████▏    | 257/500 [29:29<27:50,  6.88s/it, loss=1.51e+04, v_num=1]
```

Example 834 (unknown):
```unknown
Epoch 258/500:  52%|█████▏    | 258/500 [29:36<27:50,  6.90s/it, loss=1.51e+04, v_num=1]
```

Example 835 (unknown):
```unknown
Epoch 258/500:  52%|█████▏    | 258/500 [29:36<27:50,  6.90s/it, loss=1.51e+04, v_num=1]
```

Example 836 (unknown):
```unknown
Epoch 259/500:  52%|█████▏    | 258/500 [29:36<27:50,  6.90s/it, loss=1.51e+04, v_num=1]
```

Example 837 (unknown):
```unknown
Epoch 259/500:  52%|█████▏    | 259/500 [29:43<27:50,  6.93s/it, loss=1.51e+04, v_num=1]
```

Example 838 (unknown):
```unknown
Epoch 259/500:  52%|█████▏    | 259/500 [29:43<27:50,  6.93s/it, loss=1.53e+04, v_num=1]
```

Example 839 (unknown):
```unknown
Epoch 260/500:  52%|█████▏    | 259/500 [29:43<27:50,  6.93s/it, loss=1.53e+04, v_num=1]
```

Example 840 (unknown):
```unknown
Epoch 260/500:  52%|█████▏    | 260/500 [29:49<27:44,  6.93s/it, loss=1.53e+04, v_num=1]
```

Example 841 (unknown):
```unknown
Epoch 260/500:  52%|█████▏    | 260/500 [29:50<27:44,  6.93s/it, loss=1.52e+04, v_num=1]
```

Example 842 (unknown):
```unknown
Epoch 261/500:  52%|█████▏    | 260/500 [29:50<27:44,  6.93s/it, loss=1.52e+04, v_num=1]
```

Example 843 (unknown):
```unknown
Epoch 261/500:  52%|█████▏    | 261/500 [29:56<27:19,  6.86s/it, loss=1.52e+04, v_num=1]
```

Example 844 (unknown):
```unknown
Epoch 261/500:  52%|█████▏    | 261/500 [29:56<27:19,  6.86s/it, loss=1.51e+04, v_num=1]
```

Example 845 (unknown):
```unknown
Epoch 262/500:  52%|█████▏    | 261/500 [29:56<27:19,  6.86s/it, loss=1.51e+04, v_num=1]
```

Example 846 (unknown):
```unknown
Epoch 262/500:  52%|█████▏    | 262/500 [30:03<26:42,  6.74s/it, loss=1.51e+04, v_num=1]
```

Example 847 (unknown):
```unknown
Epoch 262/500:  52%|█████▏    | 262/500 [30:03<26:42,  6.74s/it, loss=1.52e+04, v_num=1]
```

Example 848 (unknown):
```unknown
Epoch 263/500:  52%|█████▏    | 262/500 [30:03<26:42,  6.74s/it, loss=1.52e+04, v_num=1]
```

Example 849 (unknown):
```unknown
Epoch 263/500:  53%|█████▎    | 263/500 [30:09<26:45,  6.77s/it, loss=1.52e+04, v_num=1]
```

Example 850 (unknown):
```unknown
Epoch 263/500:  53%|█████▎    | 263/500 [30:09<26:45,  6.77s/it, loss=1.5e+04, v_num=1]
```

Example 851 (unknown):
```unknown
Epoch 264/500:  53%|█████▎    | 263/500 [30:09<26:45,  6.77s/it, loss=1.5e+04, v_num=1]
```

Example 852 (unknown):
```unknown
Epoch 264/500:  53%|█████▎    | 264/500 [30:16<26:36,  6.77s/it, loss=1.5e+04, v_num=1]
```

Example 853 (unknown):
```unknown
Epoch 264/500:  53%|█████▎    | 264/500 [30:16<26:36,  6.77s/it, loss=1.51e+04, v_num=1]
```

Example 854 (unknown):
```unknown
Epoch 265/500:  53%|█████▎    | 264/500 [30:16<26:36,  6.77s/it, loss=1.51e+04, v_num=1]
```

Example 855 (unknown):
```unknown
Epoch 265/500:  53%|█████▎    | 265/500 [30:23<26:44,  6.83s/it, loss=1.51e+04, v_num=1]
```

Example 856 (unknown):
```unknown
Epoch 265/500:  53%|█████▎    | 265/500 [30:23<26:44,  6.83s/it, loss=1.51e+04, v_num=1]
```

Example 857 (unknown):
```unknown
Epoch 266/500:  53%|█████▎    | 265/500 [30:23<26:44,  6.83s/it, loss=1.51e+04, v_num=1]
```

Example 858 (unknown):
```unknown
Epoch 266/500:  53%|█████▎    | 266/500 [30:30<26:26,  6.78s/it, loss=1.51e+04, v_num=1]
```

Example 859 (unknown):
```unknown
Epoch 266/500:  53%|█████▎    | 266/500 [30:30<26:26,  6.78s/it, loss=1.5e+04, v_num=1]
```

Example 860 (unknown):
```unknown
Epoch 267/500:  53%|█████▎    | 266/500 [30:30<26:26,  6.78s/it, loss=1.5e+04, v_num=1]
```

Example 861 (unknown):
```unknown
Epoch 267/500:  53%|█████▎    | 267/500 [30:37<26:32,  6.83s/it, loss=1.5e+04, v_num=1]
```

Example 862 (unknown):
```unknown
Epoch 267/500:  53%|█████▎    | 267/500 [30:37<26:32,  6.83s/it, loss=1.52e+04, v_num=1]
```

Example 863 (unknown):
```unknown
Epoch 268/500:  53%|█████▎    | 267/500 [30:37<26:32,  6.83s/it, loss=1.52e+04, v_num=1]
```

Example 864 (unknown):
```unknown
Epoch 268/500:  54%|█████▎    | 268/500 [30:44<26:35,  6.88s/it, loss=1.52e+04, v_num=1]
```

Example 865 (unknown):
```unknown
Epoch 268/500:  54%|█████▎    | 268/500 [30:44<26:35,  6.88s/it, loss=1.48e+04, v_num=1]
```

Example 866 (unknown):
```unknown
Epoch 269/500:  54%|█████▎    | 268/500 [30:44<26:35,  6.88s/it, loss=1.48e+04, v_num=1]
```

Example 867 (unknown):
```unknown
Epoch 269/500:  54%|█████▍    | 269/500 [30:51<26:30,  6.88s/it, loss=1.48e+04, v_num=1]
```

Example 868 (unknown):
```unknown
Epoch 269/500:  54%|█████▍    | 269/500 [30:51<26:30,  6.88s/it, loss=1.5e+04, v_num=1]
```

Example 869 (unknown):
```unknown
Epoch 270/500:  54%|█████▍    | 269/500 [30:51<26:30,  6.88s/it, loss=1.5e+04, v_num=1]
```

Example 870 (unknown):
```unknown
Epoch 270/500:  54%|█████▍    | 270/500 [30:58<26:31,  6.92s/it, loss=1.5e+04, v_num=1]
```

Example 871 (unknown):
```unknown
Epoch 270/500:  54%|█████▍    | 270/500 [30:58<26:31,  6.92s/it, loss=1.51e+04, v_num=1]
```

Example 872 (unknown):
```unknown
Epoch 271/500:  54%|█████▍    | 270/500 [30:58<26:31,  6.92s/it, loss=1.51e+04, v_num=1]
```

Example 873 (unknown):
```unknown
Epoch 271/500:  54%|█████▍    | 271/500 [31:05<26:27,  6.93s/it, loss=1.51e+04, v_num=1]
```

Example 874 (unknown):
```unknown
Epoch 271/500:  54%|█████▍    | 271/500 [31:05<26:27,  6.93s/it, loss=1.47e+04, v_num=1]
```

Example 875 (unknown):
```unknown
Epoch 272/500:  54%|█████▍    | 271/500 [31:05<26:27,  6.93s/it, loss=1.47e+04, v_num=1]
```

Example 876 (unknown):
```unknown
Epoch 272/500:  54%|█████▍    | 272/500 [31:12<26:13,  6.90s/it, loss=1.47e+04, v_num=1]
```

Example 877 (unknown):
```unknown
Epoch 272/500:  54%|█████▍    | 272/500 [31:12<26:13,  6.90s/it, loss=1.51e+04, v_num=1]
```

Example 878 (unknown):
```unknown
Epoch 273/500:  54%|█████▍    | 272/500 [31:12<26:13,  6.90s/it, loss=1.51e+04, v_num=1]
```

Example 879 (unknown):
```unknown
Epoch 273/500:  55%|█████▍    | 273/500 [31:18<26:11,  6.92s/it, loss=1.51e+04, v_num=1]
```

Example 880 (unknown):
```unknown
Epoch 273/500:  55%|█████▍    | 273/500 [31:18<26:11,  6.92s/it, loss=1.5e+04, v_num=1]
```

Example 881 (unknown):
```unknown
Epoch 274/500:  55%|█████▍    | 273/500 [31:18<26:11,  6.92s/it, loss=1.5e+04, v_num=1]
```

Example 882 (unknown):
```unknown
Epoch 274/500:  55%|█████▍    | 274/500 [31:25<25:41,  6.82s/it, loss=1.5e+04, v_num=1]
```

Example 883 (unknown):
```unknown
Epoch 274/500:  55%|█████▍    | 274/500 [31:25<25:41,  6.82s/it, loss=1.5e+04, v_num=1]
```

Example 884 (unknown):
```unknown
Epoch 275/500:  55%|█████▍    | 274/500 [31:25<25:41,  6.82s/it, loss=1.5e+04, v_num=1]
```

Example 885 (unknown):
```unknown
Epoch 275/500:  55%|█████▌    | 275/500 [31:32<25:16,  6.74s/it, loss=1.5e+04, v_num=1]
```

Example 886 (unknown):
```unknown
Epoch 275/500:  55%|█████▌    | 275/500 [31:32<25:16,  6.74s/it, loss=1.48e+04, v_num=1]
```

Example 887 (unknown):
```unknown
Epoch 276/500:  55%|█████▌    | 275/500 [31:32<25:16,  6.74s/it, loss=1.48e+04, v_num=1]
```

Example 888 (unknown):
```unknown
Epoch 276/500:  55%|█████▌    | 276/500 [31:38<25:18,  6.78s/it, loss=1.48e+04, v_num=1]
```

Example 889 (unknown):
```unknown
Epoch 276/500:  55%|█████▌    | 276/500 [31:38<25:18,  6.78s/it, loss=1.51e+04, v_num=1]
```

Example 890 (unknown):
```unknown
Epoch 277/500:  55%|█████▌    | 276/500 [31:38<25:18,  6.78s/it, loss=1.51e+04, v_num=1]
```

Example 891 (unknown):
```unknown
Epoch 277/500:  55%|█████▌    | 277/500 [31:45<25:12,  6.78s/it, loss=1.51e+04, v_num=1]
```

Example 892 (unknown):
```unknown
Epoch 277/500:  55%|█████▌    | 277/500 [31:45<25:12,  6.78s/it, loss=1.51e+04, v_num=1]
```

Example 893 (unknown):
```unknown
Epoch 278/500:  55%|█████▌    | 277/500 [31:45<25:12,  6.78s/it, loss=1.51e+04, v_num=1]
```

Example 894 (unknown):
```unknown
Epoch 278/500:  56%|█████▌    | 278/500 [31:52<25:04,  6.78s/it, loss=1.51e+04, v_num=1]
```

Example 895 (unknown):
```unknown
Epoch 278/500:  56%|█████▌    | 278/500 [31:52<25:04,  6.78s/it, loss=1.5e+04, v_num=1]
```

Example 896 (unknown):
```unknown
Epoch 279/500:  56%|█████▌    | 278/500 [31:52<25:04,  6.78s/it, loss=1.5e+04, v_num=1]
```

Example 897 (unknown):
```unknown
Epoch 279/500:  56%|█████▌    | 279/500 [31:59<25:06,  6.82s/it, loss=1.5e+04, v_num=1]
```

Example 898 (unknown):
```unknown
Epoch 279/500:  56%|█████▌    | 279/500 [31:59<25:06,  6.82s/it, loss=1.5e+04, v_num=1]
```

Example 899 (unknown):
```unknown
Epoch 280/500:  56%|█████▌    | 279/500 [31:59<25:06,  6.82s/it, loss=1.5e+04, v_num=1]
```

Example 900 (unknown):
```unknown
Epoch 280/500:  56%|█████▌    | 280/500 [32:06<25:12,  6.88s/it, loss=1.5e+04, v_num=1]
```

Example 901 (unknown):
```unknown
Epoch 280/500:  56%|█████▌    | 280/500 [32:06<25:12,  6.88s/it, loss=1.48e+04, v_num=1]
```

Example 902 (unknown):
```unknown
Epoch 281/500:  56%|█████▌    | 280/500 [32:06<25:12,  6.88s/it, loss=1.48e+04, v_num=1]
```

Example 903 (unknown):
```unknown
Epoch 281/500:  56%|█████▌    | 281/500 [32:13<24:58,  6.84s/it, loss=1.48e+04, v_num=1]
```

Example 904 (unknown):
```unknown
Epoch 281/500:  56%|█████▌    | 281/500 [32:13<24:58,  6.84s/it, loss=1.52e+04, v_num=1]
```

Example 905 (unknown):
```unknown
Epoch 282/500:  56%|█████▌    | 281/500 [32:13<24:58,  6.84s/it, loss=1.52e+04, v_num=1]
```

Example 906 (unknown):
```unknown
Epoch 282/500:  56%|█████▋    | 282/500 [32:20<24:55,  6.86s/it, loss=1.52e+04, v_num=1]
```

Example 907 (unknown):
```unknown
Epoch 282/500:  56%|█████▋    | 282/500 [32:20<24:55,  6.86s/it, loss=1.5e+04, v_num=1]
```

Example 908 (unknown):
```unknown
Epoch 283/500:  56%|█████▋    | 282/500 [32:20<24:55,  6.86s/it, loss=1.5e+04, v_num=1]
```

Example 909 (unknown):
```unknown
Epoch 283/500:  57%|█████▋    | 283/500 [32:27<25:03,  6.93s/it, loss=1.5e+04, v_num=1]
```

Example 910 (unknown):
```unknown
Epoch 283/500:  57%|█████▋    | 283/500 [32:27<25:03,  6.93s/it, loss=1.51e+04, v_num=1]
```

Example 911 (unknown):
```unknown
Epoch 284/500:  57%|█████▋    | 283/500 [32:27<25:03,  6.93s/it, loss=1.51e+04, v_num=1]
```

Example 912 (unknown):
```unknown
Epoch 284/500:  57%|█████▋    | 284/500 [32:33<24:35,  6.83s/it, loss=1.51e+04, v_num=1]
```

Example 913 (unknown):
```unknown
Epoch 284/500:  57%|█████▋    | 284/500 [32:33<24:35,  6.83s/it, loss=1.53e+04, v_num=1]
```

Example 914 (unknown):
```unknown
Epoch 285/500:  57%|█████▋    | 284/500 [32:33<24:35,  6.83s/it, loss=1.53e+04, v_num=1]
```

Example 915 (unknown):
```unknown
Epoch 285/500:  57%|█████▋    | 285/500 [32:40<24:35,  6.86s/it, loss=1.53e+04, v_num=1]
```

Example 916 (unknown):
```unknown
Epoch 285/500:  57%|█████▋    | 285/500 [32:40<24:35,  6.86s/it, loss=1.49e+04, v_num=1]
```

Example 917 (unknown):
```unknown
Epoch 286/500:  57%|█████▋    | 285/500 [32:40<24:35,  6.86s/it, loss=1.49e+04, v_num=1]
```

Example 918 (unknown):
```unknown
Epoch 286/500:  57%|█████▋    | 286/500 [32:47<24:35,  6.89s/it, loss=1.49e+04, v_num=1]
```

Example 919 (unknown):
```unknown
Epoch 286/500:  57%|█████▋    | 286/500 [32:47<24:35,  6.89s/it, loss=1.47e+04, v_num=1]
```

Example 920 (unknown):
```unknown
Epoch 287/500:  57%|█████▋    | 286/500 [32:47<24:35,  6.89s/it, loss=1.47e+04, v_num=1]
```

Example 921 (unknown):
```unknown
Epoch 287/500:  57%|█████▋    | 287/500 [32:54<24:07,  6.80s/it, loss=1.47e+04, v_num=1]
```

Example 922 (unknown):
```unknown
Epoch 287/500:  57%|█████▋    | 287/500 [32:54<24:07,  6.80s/it, loss=1.5e+04, v_num=1]
```

Example 923 (unknown):
```unknown
Epoch 288/500:  57%|█████▋    | 287/500 [32:54<24:07,  6.80s/it, loss=1.5e+04, v_num=1]
```

Example 924 (unknown):
```unknown
Epoch 288/500:  58%|█████▊    | 288/500 [33:01<24:13,  6.85s/it, loss=1.5e+04, v_num=1]
```

Example 925 (unknown):
```unknown
Epoch 288/500:  58%|█████▊    | 288/500 [33:01<24:13,  6.85s/it, loss=1.5e+04, v_num=1]
```

Example 926 (unknown):
```unknown
Epoch 289/500:  58%|█████▊    | 288/500 [33:01<24:13,  6.85s/it, loss=1.5e+04, v_num=1]
```

Example 927 (unknown):
```unknown
Epoch 289/500:  58%|█████▊    | 289/500 [33:08<24:03,  6.84s/it, loss=1.5e+04, v_num=1]
```

Example 928 (unknown):
```unknown
Epoch 289/500:  58%|█████▊    | 289/500 [33:08<24:03,  6.84s/it, loss=1.49e+04, v_num=1]
```

Example 929 (unknown):
```unknown
Epoch 290/500:  58%|█████▊    | 289/500 [33:08<24:03,  6.84s/it, loss=1.49e+04, v_num=1]
```

Example 930 (unknown):
```unknown
Epoch 290/500:  58%|█████▊    | 290/500 [33:15<24:01,  6.87s/it, loss=1.49e+04, v_num=1]
```

Example 931 (unknown):
```unknown
Epoch 290/500:  58%|█████▊    | 290/500 [33:15<24:01,  6.87s/it, loss=1.48e+04, v_num=1]
```

Example 932 (unknown):
```unknown
Epoch 291/500:  58%|█████▊    | 290/500 [33:15<24:01,  6.87s/it, loss=1.48e+04, v_num=1]
```

Example 933 (unknown):
```unknown
Epoch 291/500:  58%|█████▊    | 291/500 [33:22<24:11,  6.94s/it, loss=1.48e+04, v_num=1]
```

Example 934 (unknown):
```unknown
Epoch 291/500:  58%|█████▊    | 291/500 [33:22<24:11,  6.94s/it, loss=1.5e+04, v_num=1]
```

Example 935 (unknown):
```unknown
Epoch 292/500:  58%|█████▊    | 291/500 [33:22<24:11,  6.94s/it, loss=1.5e+04, v_num=1]
```

Example 936 (unknown):
```unknown
Epoch 292/500:  58%|█████▊    | 292/500 [33:28<23:55,  6.90s/it, loss=1.5e+04, v_num=1]
```

Example 937 (unknown):
```unknown
Epoch 292/500:  58%|█████▊    | 292/500 [33:28<23:55,  6.90s/it, loss=1.48e+04, v_num=1]
```

Example 938 (unknown):
```unknown
Epoch 293/500:  58%|█████▊    | 292/500 [33:28<23:55,  6.90s/it, loss=1.48e+04, v_num=1]
```

Example 939 (unknown):
```unknown
Epoch 293/500:  59%|█████▊    | 293/500 [33:35<23:26,  6.79s/it, loss=1.48e+04, v_num=1]
```

Example 940 (unknown):
```unknown
Epoch 293/500:  59%|█████▊    | 293/500 [33:35<23:26,  6.79s/it, loss=1.52e+04, v_num=1]
```

Example 941 (unknown):
```unknown
Epoch 294/500:  59%|█████▊    | 293/500 [33:35<23:26,  6.79s/it, loss=1.52e+04, v_num=1]
```

Example 942 (unknown):
```unknown
Epoch 294/500:  59%|█████▉    | 294/500 [33:42<23:27,  6.83s/it, loss=1.52e+04, v_num=1]
```

Example 943 (unknown):
```unknown
Epoch 294/500:  59%|█████▉    | 294/500 [33:42<23:27,  6.83s/it, loss=1.51e+04, v_num=1]
```

Example 944 (unknown):
```unknown
Epoch 295/500:  59%|█████▉    | 294/500 [33:42<23:27,  6.83s/it, loss=1.51e+04, v_num=1]
```

Example 945 (unknown):
```unknown
Epoch 295/500:  59%|█████▉    | 295/500 [33:49<23:31,  6.88s/it, loss=1.51e+04, v_num=1]
```

Example 946 (unknown):
```unknown
Epoch 295/500:  59%|█████▉    | 295/500 [33:49<23:31,  6.88s/it, loss=1.49e+04, v_num=1]
```

Example 947 (unknown):
```unknown
Epoch 296/500:  59%|█████▉    | 295/500 [33:49<23:31,  6.88s/it, loss=1.49e+04, v_num=1]
```

Example 948 (unknown):
```unknown
Epoch 296/500:  59%|█████▉    | 296/500 [33:56<23:22,  6.87s/it, loss=1.49e+04, v_num=1]
```

Example 949 (unknown):
```unknown
Epoch 296/500:  59%|█████▉    | 296/500 [33:56<23:22,  6.87s/it, loss=1.48e+04, v_num=1]
```

Example 950 (unknown):
```unknown
Epoch 297/500:  59%|█████▉    | 296/500 [33:56<23:22,  6.87s/it, loss=1.48e+04, v_num=1]
```

Example 951 (unknown):
```unknown
Epoch 297/500:  59%|█████▉    | 297/500 [34:03<23:27,  6.93s/it, loss=1.48e+04, v_num=1]
```

Example 952 (unknown):
```unknown
Epoch 297/500:  59%|█████▉    | 297/500 [34:03<23:27,  6.93s/it, loss=1.5e+04, v_num=1]
```

Example 953 (unknown):
```unknown
Epoch 298/500:  59%|█████▉    | 297/500 [34:03<23:27,  6.93s/it, loss=1.5e+04, v_num=1]
```

Example 954 (unknown):
```unknown
Epoch 298/500:  60%|█████▉    | 298/500 [34:10<23:19,  6.93s/it, loss=1.5e+04, v_num=1]
```

Example 955 (unknown):
```unknown
Epoch 298/500:  60%|█████▉    | 298/500 [34:10<23:19,  6.93s/it, loss=1.49e+04, v_num=1]
```

Example 956 (unknown):
```unknown
Epoch 299/500:  60%|█████▉    | 298/500 [34:10<23:19,  6.93s/it, loss=1.49e+04, v_num=1]
```

Example 957 (unknown):
```unknown
Epoch 299/500:  60%|█████▉    | 299/500 [34:16<22:49,  6.81s/it, loss=1.49e+04, v_num=1]
```

Example 958 (unknown):
```unknown
Epoch 299/500:  60%|█████▉    | 299/500 [34:16<22:49,  6.81s/it, loss=1.47e+04, v_num=1]
```

Example 959 (unknown):
```unknown
Epoch 300/500:  60%|█████▉    | 299/500 [34:16<22:49,  6.81s/it, loss=1.47e+04, v_num=1]
```

Example 960 (unknown):
```unknown
Epoch 300/500:  60%|██████    | 300/500 [34:23<22:42,  6.81s/it, loss=1.47e+04, v_num=1]
```

Example 961 (unknown):
```unknown
Epoch 300/500:  60%|██████    | 300/500 [34:23<22:42,  6.81s/it, loss=1.49e+04, v_num=1]
```

Example 962 (unknown):
```unknown
Epoch 301/500:  60%|██████    | 300/500 [34:23<22:42,  6.81s/it, loss=1.49e+04, v_num=1]
```

Example 963 (unknown):
```unknown
Epoch 301/500:  60%|██████    | 301/500 [34:30<22:19,  6.73s/it, loss=1.49e+04, v_num=1]
```

Example 964 (unknown):
```unknown
Epoch 301/500:  60%|██████    | 301/500 [34:30<22:19,  6.73s/it, loss=1.5e+04, v_num=1]
```

Example 965 (unknown):
```unknown
Epoch 302/500:  60%|██████    | 301/500 [34:30<22:19,  6.73s/it, loss=1.5e+04, v_num=1]
```

Example 966 (unknown):
```unknown
Epoch 302/500:  60%|██████    | 302/500 [34:37<22:25,  6.80s/it, loss=1.5e+04, v_num=1]
```

Example 967 (unknown):
```unknown
Epoch 302/500:  60%|██████    | 302/500 [34:37<22:25,  6.80s/it, loss=1.48e+04, v_num=1]
```

Example 968 (unknown):
```unknown
Epoch 303/500:  60%|██████    | 302/500 [34:37<22:25,  6.80s/it, loss=1.48e+04, v_num=1]
```

Example 969 (unknown):
```unknown
Epoch 303/500:  61%|██████    | 303/500 [34:44<22:26,  6.84s/it, loss=1.48e+04, v_num=1]
```

Example 970 (unknown):
```unknown
Epoch 303/500:  61%|██████    | 303/500 [34:44<22:26,  6.84s/it, loss=1.48e+04, v_num=1]
```

Example 971 (unknown):
```unknown
Epoch 304/500:  61%|██████    | 303/500 [34:44<22:26,  6.84s/it, loss=1.48e+04, v_num=1]
```

Example 972 (unknown):
```unknown
Epoch 304/500:  61%|██████    | 304/500 [34:50<22:21,  6.84s/it, loss=1.48e+04, v_num=1]
```

Example 973 (unknown):
```unknown
Epoch 304/500:  61%|██████    | 304/500 [34:50<22:21,  6.84s/it, loss=1.52e+04, v_num=1]
```

Example 974 (unknown):
```unknown
Epoch 305/500:  61%|██████    | 304/500 [34:50<22:21,  6.84s/it, loss=1.52e+04, v_num=1]
```

Example 975 (unknown):
```unknown
Epoch 305/500:  61%|██████    | 305/500 [34:57<22:19,  6.87s/it, loss=1.52e+04, v_num=1]
```

Example 976 (unknown):
```unknown
Epoch 305/500:  61%|██████    | 305/500 [34:57<22:19,  6.87s/it, loss=1.51e+04, v_num=1]
```

Example 977 (unknown):
```unknown
Epoch 306/500:  61%|██████    | 305/500 [34:57<22:19,  6.87s/it, loss=1.51e+04, v_num=1]
```

Example 978 (unknown):
```unknown
Epoch 306/500:  61%|██████    | 306/500 [35:04<22:04,  6.83s/it, loss=1.51e+04, v_num=1]
```

Example 979 (unknown):
```unknown
Epoch 306/500:  61%|██████    | 306/500 [35:04<22:04,  6.83s/it, loss=1.48e+04, v_num=1]
```

Example 980 (unknown):
```unknown
Epoch 307/500:  61%|██████    | 306/500 [35:04<22:04,  6.83s/it, loss=1.48e+04, v_num=1]
```

Example 981 (unknown):
```unknown
Epoch 307/500:  61%|██████▏   | 307/500 [35:11<22:04,  6.86s/it, loss=1.48e+04, v_num=1]
```

Example 982 (unknown):
```unknown
Epoch 307/500:  61%|██████▏   | 307/500 [35:11<22:04,  6.86s/it, loss=1.48e+04, v_num=1]
```

Example 983 (unknown):
```unknown
Epoch 308/500:  61%|██████▏   | 307/500 [35:11<22:04,  6.86s/it, loss=1.48e+04, v_num=1]
```

Example 984 (unknown):
```unknown
Epoch 308/500:  62%|██████▏   | 308/500 [35:18<21:45,  6.80s/it, loss=1.48e+04, v_num=1]
```

Example 985 (unknown):
```unknown
Epoch 308/500:  62%|██████▏   | 308/500 [35:18<21:45,  6.80s/it, loss=1.52e+04, v_num=1]
```

Example 986 (unknown):
```unknown
Epoch 309/500:  62%|██████▏   | 308/500 [35:18<21:45,  6.80s/it, loss=1.52e+04, v_num=1]
```

Example 987 (unknown):
```unknown
Epoch 309/500:  62%|██████▏   | 309/500 [35:25<21:49,  6.85s/it, loss=1.52e+04, v_num=1]
```

Example 988 (unknown):
```unknown
Epoch 309/500:  62%|██████▏   | 309/500 [35:25<21:49,  6.85s/it, loss=1.51e+04, v_num=1]
```

Example 989 (unknown):
```unknown
Epoch 310/500:  62%|██████▏   | 309/500 [35:25<21:49,  6.85s/it, loss=1.51e+04, v_num=1]
```

Example 990 (unknown):
```unknown
Epoch 310/500:  62%|██████▏   | 310/500 [35:32<21:44,  6.87s/it, loss=1.51e+04, v_num=1]
```

Example 991 (unknown):
```unknown
Epoch 310/500:  62%|██████▏   | 310/500 [35:32<21:44,  6.87s/it, loss=1.48e+04, v_num=1]
```

Example 992 (unknown):
```unknown
Epoch 311/500:  62%|██████▏   | 310/500 [35:32<21:44,  6.87s/it, loss=1.48e+04, v_num=1]
```

Example 993 (unknown):
```unknown
Epoch 311/500:  62%|██████▏   | 311/500 [35:38<21:35,  6.85s/it, loss=1.48e+04, v_num=1]
```

Example 994 (unknown):
```unknown
Epoch 311/500:  62%|██████▏   | 311/500 [35:38<21:35,  6.85s/it, loss=1.51e+04, v_num=1]
```

Example 995 (unknown):
```unknown
Epoch 312/500:  62%|██████▏   | 311/500 [35:38<21:35,  6.85s/it, loss=1.51e+04, v_num=1]
```

Example 996 (unknown):
```unknown
Epoch 312/500:  62%|██████▏   | 312/500 [35:45<21:32,  6.87s/it, loss=1.51e+04, v_num=1]
```

Example 997 (unknown):
```unknown
Epoch 312/500:  62%|██████▏   | 312/500 [35:45<21:32,  6.87s/it, loss=1.53e+04, v_num=1]
```

Example 998 (unknown):
```unknown
Epoch 313/500:  62%|██████▏   | 312/500 [35:45<21:32,  6.87s/it, loss=1.53e+04, v_num=1]
```

Example 999 (unknown):
```unknown
Epoch 313/500:  63%|██████▎   | 313/500 [35:52<21:32,  6.91s/it, loss=1.53e+04, v_num=1]
```

Example 1000 (unknown):
```unknown
Epoch 313/500:  63%|██████▎   | 313/500 [35:52<21:32,  6.91s/it, loss=1.5e+04, v_num=1]
```

Example 1001 (unknown):
```unknown
Epoch 314/500:  63%|██████▎   | 313/500 [35:52<21:32,  6.91s/it, loss=1.5e+04, v_num=1]
```

Example 1002 (unknown):
```unknown
Epoch 314/500:  63%|██████▎   | 314/500 [35:59<21:11,  6.83s/it, loss=1.5e+04, v_num=1]
```

Example 1003 (unknown):
```unknown
Epoch 314/500:  63%|██████▎   | 314/500 [35:59<21:11,  6.83s/it, loss=1.5e+04, v_num=1]
```

Example 1004 (unknown):
```unknown
Epoch 315/500:  63%|██████▎   | 314/500 [35:59<21:11,  6.83s/it, loss=1.5e+04, v_num=1]
```

Example 1005 (unknown):
```unknown
Epoch 315/500:  63%|██████▎   | 315/500 [36:06<21:09,  6.86s/it, loss=1.5e+04, v_num=1]
```

Example 1006 (unknown):
```unknown
Epoch 315/500:  63%|██████▎   | 315/500 [36:06<21:09,  6.86s/it, loss=1.48e+04, v_num=1]
```

Example 1007 (unknown):
```unknown
Epoch 316/500:  63%|██████▎   | 315/500 [36:06<21:09,  6.86s/it, loss=1.48e+04, v_num=1]
```

Example 1008 (unknown):
```unknown
Epoch 316/500:  63%|██████▎   | 316/500 [36:13<21:01,  6.86s/it, loss=1.48e+04, v_num=1]
```

Example 1009 (unknown):
```unknown
Epoch 316/500:  63%|██████▎   | 316/500 [36:13<21:01,  6.86s/it, loss=1.5e+04, v_num=1]
```

Example 1010 (unknown):
```unknown
Epoch 317/500:  63%|██████▎   | 316/500 [36:13<21:01,  6.86s/it, loss=1.5e+04, v_num=1]
```

Example 1011 (unknown):
```unknown
Epoch 317/500:  63%|██████▎   | 317/500 [36:20<21:06,  6.92s/it, loss=1.5e+04, v_num=1]
```

Example 1012 (unknown):
```unknown
Epoch 317/500:  63%|██████▎   | 317/500 [36:20<21:06,  6.92s/it, loss=1.51e+04, v_num=1]
```

Example 1013 (unknown):
```unknown
Epoch 318/500:  63%|██████▎   | 317/500 [36:20<21:06,  6.92s/it, loss=1.51e+04, v_num=1]
```

Example 1014 (unknown):
```unknown
Epoch 318/500:  64%|██████▎   | 318/500 [36:26<20:49,  6.86s/it, loss=1.51e+04, v_num=1]
```

Example 1015 (unknown):
```unknown
Epoch 318/500:  64%|██████▎   | 318/500 [36:26<20:49,  6.86s/it, loss=1.47e+04, v_num=1]
```

Example 1016 (unknown):
```unknown
Epoch 319/500:  64%|██████▎   | 318/500 [36:26<20:49,  6.86s/it, loss=1.47e+04, v_num=1]
```

Example 1017 (unknown):
```unknown
Epoch 319/500:  64%|██████▍   | 319/500 [36:33<20:27,  6.78s/it, loss=1.47e+04, v_num=1]
```

Example 1018 (unknown):
```unknown
Epoch 319/500:  64%|██████▍   | 319/500 [36:33<20:27,  6.78s/it, loss=1.52e+04, v_num=1]
```

Example 1019 (unknown):
```unknown
Epoch 320/500:  64%|██████▍   | 319/500 [36:33<20:27,  6.78s/it, loss=1.52e+04, v_num=1]
```

Example 1020 (unknown):
```unknown
Epoch 320/500:  64%|██████▍   | 320/500 [36:40<20:33,  6.85s/it, loss=1.52e+04, v_num=1]
```

Example 1021 (unknown):
```unknown
Epoch 320/500:  64%|██████▍   | 320/500 [36:40<20:33,  6.85s/it, loss=1.49e+04, v_num=1]
```

Example 1022 (unknown):
```unknown
Epoch 321/500:  64%|██████▍   | 320/500 [36:40<20:33,  6.85s/it, loss=1.49e+04, v_num=1]
```

Example 1023 (unknown):
```unknown
Epoch 321/500:  64%|██████▍   | 321/500 [36:47<20:34,  6.90s/it, loss=1.49e+04, v_num=1]
```

Example 1024 (unknown):
```unknown
Epoch 321/500:  64%|██████▍   | 321/500 [36:47<20:34,  6.90s/it, loss=1.51e+04, v_num=1]
```

Example 1025 (unknown):
```unknown
Epoch 322/500:  64%|██████▍   | 321/500 [36:47<20:34,  6.90s/it, loss=1.51e+04, v_num=1]
```

Example 1026 (unknown):
```unknown
Epoch 322/500:  64%|██████▍   | 322/500 [36:54<20:21,  6.86s/it, loss=1.51e+04, v_num=1]
```

Example 1027 (unknown):
```unknown
Epoch 322/500:  64%|██████▍   | 322/500 [36:54<20:21,  6.86s/it, loss=1.5e+04, v_num=1]
```

Example 1028 (unknown):
```unknown
Epoch 323/500:  64%|██████▍   | 322/500 [36:54<20:21,  6.86s/it, loss=1.5e+04, v_num=1]
```

Example 1029 (unknown):
```unknown
Epoch 323/500:  65%|██████▍   | 323/500 [37:01<20:21,  6.90s/it, loss=1.5e+04, v_num=1]
```

Example 1030 (unknown):
```unknown
Epoch 323/500:  65%|██████▍   | 323/500 [37:01<20:21,  6.90s/it, loss=1.5e+04, v_num=1]
```

Example 1031 (unknown):
```unknown
Epoch 324/500:  65%|██████▍   | 323/500 [37:01<20:21,  6.90s/it, loss=1.5e+04, v_num=1]
```

Example 1032 (unknown):
```unknown
Epoch 324/500:  65%|██████▍   | 324/500 [37:07<19:38,  6.70s/it, loss=1.5e+04, v_num=1]
```

Example 1033 (unknown):
```unknown
Epoch 324/500:  65%|██████▍   | 324/500 [37:07<19:38,  6.70s/it, loss=1.48e+04, v_num=1]
```

Example 1034 (unknown):
```unknown
Epoch 325/500:  65%|██████▍   | 324/500 [37:07<19:38,  6.70s/it, loss=1.48e+04, v_num=1]
```

Example 1035 (unknown):
```unknown
Epoch 325/500:  65%|██████▌   | 325/500 [37:14<19:18,  6.62s/it, loss=1.48e+04, v_num=1]
```

Example 1036 (unknown):
```unknown
Epoch 325/500:  65%|██████▌   | 325/500 [37:14<19:18,  6.62s/it, loss=1.5e+04, v_num=1]
```

Example 1037 (unknown):
```unknown
Epoch 326/500:  65%|██████▌   | 325/500 [37:14<19:18,  6.62s/it, loss=1.5e+04, v_num=1]
```

Example 1038 (unknown):
```unknown
Epoch 326/500:  65%|██████▌   | 326/500 [37:21<19:32,  6.74s/it, loss=1.5e+04, v_num=1]
```

Example 1039 (unknown):
```unknown
Epoch 326/500:  65%|██████▌   | 326/500 [37:21<19:32,  6.74s/it, loss=1.5e+04, v_num=1]
```

Example 1040 (unknown):
```unknown
Epoch 327/500:  65%|██████▌   | 326/500 [37:21<19:32,  6.74s/it, loss=1.5e+04, v_num=1]
```

Example 1041 (unknown):
```unknown
Epoch 327/500:  65%|██████▌   | 327/500 [37:28<19:37,  6.81s/it, loss=1.5e+04, v_num=1]
```

Example 1042 (unknown):
```unknown
Epoch 327/500:  65%|██████▌   | 327/500 [37:28<19:37,  6.81s/it, loss=1.48e+04, v_num=1]
```

Example 1043 (unknown):
```unknown
Epoch 328/500:  65%|██████▌   | 327/500 [37:28<19:37,  6.81s/it, loss=1.48e+04, v_num=1]
```

Example 1044 (unknown):
```unknown
Epoch 328/500:  66%|██████▌   | 328/500 [37:35<19:40,  6.86s/it, loss=1.48e+04, v_num=1]
```

Example 1045 (unknown):
```unknown
Epoch 328/500:  66%|██████▌   | 328/500 [37:35<19:40,  6.86s/it, loss=1.49e+04, v_num=1]
```

Example 1046 (unknown):
```unknown
Epoch 329/500:  66%|██████▌   | 328/500 [37:35<19:40,  6.86s/it, loss=1.49e+04, v_num=1]
```

Example 1047 (unknown):
```unknown
Epoch 329/500:  66%|██████▌   | 329/500 [37:41<19:34,  6.87s/it, loss=1.49e+04, v_num=1]
```

Example 1048 (unknown):
```unknown
Epoch 329/500:  66%|██████▌   | 329/500 [37:41<19:34,  6.87s/it, loss=1.49e+04, v_num=1]
```

Example 1049 (unknown):
```unknown
Epoch 330/500:  66%|██████▌   | 329/500 [37:41<19:34,  6.87s/it, loss=1.49e+04, v_num=1]
```

Example 1050 (unknown):
```unknown
Epoch 330/500:  66%|██████▌   | 330/500 [37:48<19:28,  6.87s/it, loss=1.49e+04, v_num=1]
```

Example 1051 (unknown):
```unknown
Epoch 330/500:  66%|██████▌   | 330/500 [37:48<19:28,  6.87s/it, loss=1.47e+04, v_num=1]
```

Example 1052 (unknown):
```unknown
Epoch 331/500:  66%|██████▌   | 330/500 [37:48<19:28,  6.87s/it, loss=1.47e+04, v_num=1]
```

Example 1053 (unknown):
```unknown
Epoch 331/500:  66%|██████▌   | 331/500 [37:55<19:16,  6.84s/it, loss=1.47e+04, v_num=1]
```

Example 1054 (unknown):
```unknown
Epoch 331/500:  66%|██████▌   | 331/500 [37:55<19:16,  6.84s/it, loss=1.48e+04, v_num=1]
```

Example 1055 (unknown):
```unknown
Epoch 332/500:  66%|██████▌   | 331/500 [37:55<19:16,  6.84s/it, loss=1.48e+04, v_num=1]
```

Example 1056 (unknown):
```unknown
Epoch 332/500:  66%|██████▋   | 332/500 [38:02<19:03,  6.81s/it, loss=1.48e+04, v_num=1]
```

Example 1057 (unknown):
```unknown
Epoch 332/500:  66%|██████▋   | 332/500 [38:02<19:03,  6.81s/it, loss=1.5e+04, v_num=1]
```

Example 1058 (unknown):
```unknown
Epoch 333/500:  66%|██████▋   | 332/500 [38:02<19:03,  6.81s/it, loss=1.5e+04, v_num=1]
```

Example 1059 (unknown):
```unknown
Epoch 333/500:  67%|██████▋   | 333/500 [38:08<18:41,  6.71s/it, loss=1.5e+04, v_num=1]
```

Example 1060 (unknown):
```unknown
Epoch 333/500:  67%|██████▋   | 333/500 [38:08<18:41,  6.71s/it, loss=1.5e+04, v_num=1]
```

Example 1061 (unknown):
```unknown
Epoch 334/500:  67%|██████▋   | 333/500 [38:08<18:41,  6.71s/it, loss=1.5e+04, v_num=1]
```

Example 1062 (unknown):
```unknown
Epoch 334/500:  67%|██████▋   | 334/500 [38:15<18:47,  6.79s/it, loss=1.5e+04, v_num=1]
```

Example 1063 (unknown):
```unknown
Epoch 334/500:  67%|██████▋   | 334/500 [38:15<18:47,  6.79s/it, loss=1.48e+04, v_num=1]
```

Example 1064 (unknown):
```unknown
Epoch 335/500:  67%|██████▋   | 334/500 [38:15<18:47,  6.79s/it, loss=1.48e+04, v_num=1]
```

Example 1065 (unknown):
```unknown
Epoch 335/500:  67%|██████▋   | 335/500 [38:22<18:49,  6.85s/it, loss=1.48e+04, v_num=1]
```

Example 1066 (unknown):
```unknown
Epoch 335/500:  67%|██████▋   | 335/500 [38:22<18:49,  6.85s/it, loss=1.5e+04, v_num=1]
```

Example 1067 (unknown):
```unknown
Epoch 336/500:  67%|██████▋   | 335/500 [38:22<18:49,  6.85s/it, loss=1.5e+04, v_num=1]
```

Example 1068 (unknown):
```unknown
Epoch 336/500:  67%|██████▋   | 336/500 [38:29<18:40,  6.83s/it, loss=1.5e+04, v_num=1]
```

Example 1069 (unknown):
```unknown
Epoch 336/500:  67%|██████▋   | 336/500 [38:29<18:40,  6.83s/it, loss=1.48e+04, v_num=1]
```

Example 1070 (unknown):
```unknown
Epoch 337/500:  67%|██████▋   | 336/500 [38:29<18:40,  6.83s/it, loss=1.48e+04, v_num=1]
```

Example 1071 (unknown):
```unknown
Epoch 337/500:  67%|██████▋   | 337/500 [38:36<18:27,  6.80s/it, loss=1.48e+04, v_num=1]
```

Example 1072 (unknown):
```unknown
Epoch 337/500:  67%|██████▋   | 337/500 [38:36<18:27,  6.80s/it, loss=1.5e+04, v_num=1]
```

Example 1073 (unknown):
```unknown
Epoch 338/500:  67%|██████▋   | 337/500 [38:36<18:27,  6.80s/it, loss=1.5e+04, v_num=1]
```

Example 1074 (unknown):
```unknown
Epoch 338/500:  68%|██████▊   | 338/500 [38:43<18:28,  6.84s/it, loss=1.5e+04, v_num=1]
```

Example 1075 (unknown):
```unknown
Epoch 338/500:  68%|██████▊   | 338/500 [38:43<18:28,  6.84s/it, loss=1.51e+04, v_num=1]
```

Example 1076 (unknown):
```unknown
Epoch 339/500:  68%|██████▊   | 338/500 [38:43<18:28,  6.84s/it, loss=1.51e+04, v_num=1]
```

Example 1077 (unknown):
```unknown
Epoch 339/500:  68%|██████▊   | 339/500 [38:50<18:22,  6.85s/it, loss=1.51e+04, v_num=1]
```

Example 1078 (unknown):
```unknown
Epoch 339/500:  68%|██████▊   | 339/500 [38:50<18:22,  6.85s/it, loss=1.49e+04, v_num=1]
```

Example 1079 (unknown):
```unknown
Epoch 340/500:  68%|██████▊   | 339/500 [38:50<18:22,  6.85s/it, loss=1.49e+04, v_num=1]
```

Example 1080 (unknown):
```unknown
Epoch 340/500:  68%|██████▊   | 340/500 [38:56<18:15,  6.85s/it, loss=1.49e+04, v_num=1]
```

Example 1081 (unknown):
```unknown
Epoch 340/500:  68%|██████▊   | 340/500 [38:56<18:15,  6.85s/it, loss=1.48e+04, v_num=1]
```

Example 1082 (unknown):
```unknown
Epoch 341/500:  68%|██████▊   | 340/500 [38:56<18:15,  6.85s/it, loss=1.48e+04, v_num=1]
```

Example 1083 (unknown):
```unknown
Epoch 341/500:  68%|██████▊   | 341/500 [39:03<18:09,  6.85s/it, loss=1.48e+04, v_num=1]
```

Example 1084 (unknown):
```unknown
Epoch 341/500:  68%|██████▊   | 341/500 [39:03<18:09,  6.85s/it, loss=1.49e+04, v_num=1]
```

Example 1085 (unknown):
```unknown
Epoch 342/500:  68%|██████▊   | 341/500 [39:03<18:09,  6.85s/it, loss=1.49e+04, v_num=1]
```

Example 1086 (unknown):
```unknown
Epoch 342/500:  68%|██████▊   | 342/500 [39:10<18:01,  6.84s/it, loss=1.49e+04, v_num=1]
```

Example 1087 (unknown):
```unknown
Epoch 342/500:  68%|██████▊   | 342/500 [39:10<18:01,  6.84s/it, loss=1.47e+04, v_num=1]
```

Example 1088 (unknown):
```unknown
Epoch 343/500:  68%|██████▊   | 342/500 [39:10<18:01,  6.84s/it, loss=1.47e+04, v_num=1]
```

Example 1089 (unknown):
```unknown
Epoch 343/500:  69%|██████▊   | 343/500 [39:17<17:48,  6.81s/it, loss=1.47e+04, v_num=1]
```

Example 1090 (unknown):
```unknown
Epoch 343/500:  69%|██████▊   | 343/500 [39:17<17:48,  6.81s/it, loss=1.5e+04, v_num=1]
```

Example 1091 (unknown):
```unknown
Epoch 344/500:  69%|██████▊   | 343/500 [39:17<17:48,  6.81s/it, loss=1.5e+04, v_num=1]
```

Example 1092 (unknown):
```unknown
Epoch 344/500:  69%|██████▉   | 344/500 [39:24<17:48,  6.85s/it, loss=1.5e+04, v_num=1]
```

Example 1093 (unknown):
```unknown
Epoch 344/500:  69%|██████▉   | 344/500 [39:24<17:48,  6.85s/it, loss=1.49e+04, v_num=1]
```

Example 1094 (unknown):
```unknown
Epoch 345/500:  69%|██████▉   | 344/500 [39:24<17:48,  6.85s/it, loss=1.49e+04, v_num=1]
```

Example 1095 (unknown):
```unknown
Epoch 345/500:  69%|██████▉   | 345/500 [39:30<17:30,  6.78s/it, loss=1.49e+04, v_num=1]
```

Example 1096 (unknown):
```unknown
Epoch 345/500:  69%|██████▉   | 345/500 [39:30<17:30,  6.78s/it, loss=1.47e+04, v_num=1]
```

Example 1097 (unknown):
```unknown
Epoch 346/500:  69%|██████▉   | 345/500 [39:30<17:30,  6.78s/it, loss=1.47e+04, v_num=1]
```

Example 1098 (unknown):
```unknown
Epoch 346/500:  69%|██████▉   | 346/500 [39:37<17:30,  6.82s/it, loss=1.47e+04, v_num=1]
```

Example 1099 (unknown):
```unknown
Epoch 346/500:  69%|██████▉   | 346/500 [39:37<17:30,  6.82s/it, loss=1.48e+04, v_num=1]
```

Example 1100 (unknown):
```unknown
Epoch 347/500:  69%|██████▉   | 346/500 [39:37<17:30,  6.82s/it, loss=1.48e+04, v_num=1]
```

Example 1101 (unknown):
```unknown
Epoch 347/500:  69%|██████▉   | 347/500 [39:44<17:21,  6.81s/it, loss=1.48e+04, v_num=1]
```

Example 1102 (unknown):
```unknown
Epoch 347/500:  69%|██████▉   | 347/500 [39:44<17:21,  6.81s/it, loss=1.49e+04, v_num=1]
```

Example 1103 (unknown):
```unknown
Epoch 348/500:  69%|██████▉   | 347/500 [39:44<17:21,  6.81s/it, loss=1.49e+04, v_num=1]
```

Example 1104 (unknown):
```unknown
Epoch 348/500:  70%|██████▉   | 348/500 [39:51<17:22,  6.86s/it, loss=1.49e+04, v_num=1]
```

Example 1105 (unknown):
```unknown
Epoch 348/500:  70%|██████▉   | 348/500 [39:51<17:22,  6.86s/it, loss=1.47e+04, v_num=1]
```

Example 1106 (unknown):
```unknown
Epoch 349/500:  70%|██████▉   | 348/500 [39:51<17:22,  6.86s/it, loss=1.47e+04, v_num=1]
```

Example 1107 (unknown):
```unknown
Epoch 349/500:  70%|██████▉   | 349/500 [39:58<17:20,  6.89s/it, loss=1.47e+04, v_num=1]
```

Example 1108 (unknown):
```unknown
Epoch 349/500:  70%|██████▉   | 349/500 [39:58<17:20,  6.89s/it, loss=1.5e+04, v_num=1]
```

Example 1109 (unknown):
```unknown
Epoch 350/500:  70%|██████▉   | 349/500 [39:58<17:20,  6.89s/it, loss=1.5e+04, v_num=1]
```

Example 1110 (unknown):
```unknown
Epoch 350/500:  70%|███████   | 350/500 [40:05<17:13,  6.89s/it, loss=1.5e+04, v_num=1]
```

Example 1111 (unknown):
```unknown
Epoch 350/500:  70%|███████   | 350/500 [40:05<17:13,  6.89s/it, loss=1.47e+04, v_num=1]
```

Example 1112 (unknown):
```unknown
Epoch 351/500:  70%|███████   | 350/500 [40:05<17:13,  6.89s/it, loss=1.47e+04, v_num=1]
```

Example 1113 (unknown):
```unknown
Epoch 351/500:  70%|███████   | 351/500 [40:12<17:10,  6.92s/it, loss=1.47e+04, v_num=1]
```

Example 1114 (unknown):
```unknown
Epoch 351/500:  70%|███████   | 351/500 [40:12<17:10,  6.92s/it, loss=1.47e+04, v_num=1]
```

Example 1115 (unknown):
```unknown
Epoch 352/500:  70%|███████   | 351/500 [40:12<17:10,  6.92s/it, loss=1.47e+04, v_num=1]
```

Example 1116 (unknown):
```unknown
Epoch 352/500:  70%|███████   | 352/500 [40:19<17:09,  6.96s/it, loss=1.47e+04, v_num=1]
```

Example 1117 (unknown):
```unknown
Epoch 352/500:  70%|███████   | 352/500 [40:19<17:09,  6.96s/it, loss=1.47e+04, v_num=1]
```

Example 1118 (unknown):
```unknown
Epoch 353/500:  70%|███████   | 352/500 [40:19<17:09,  6.96s/it, loss=1.47e+04, v_num=1]
```

Example 1119 (unknown):
```unknown
Epoch 353/500:  71%|███████   | 353/500 [40:26<16:53,  6.89s/it, loss=1.47e+04, v_num=1]
```

Example 1120 (unknown):
```unknown
Epoch 353/500:  71%|███████   | 353/500 [40:26<16:53,  6.89s/it, loss=1.49e+04, v_num=1]
```

Example 1121 (unknown):
```unknown
Epoch 354/500:  71%|███████   | 353/500 [40:26<16:53,  6.89s/it, loss=1.49e+04, v_num=1]
```

Example 1122 (unknown):
```unknown
Epoch 354/500:  71%|███████   | 354/500 [40:33<16:49,  6.92s/it, loss=1.49e+04, v_num=1]
```

Example 1123 (unknown):
```unknown
Epoch 354/500:  71%|███████   | 354/500 [40:33<16:49,  6.92s/it, loss=1.47e+04, v_num=1]
```

Example 1124 (unknown):
```unknown
Epoch 355/500:  71%|███████   | 354/500 [40:33<16:49,  6.92s/it, loss=1.47e+04, v_num=1]
```

Example 1125 (unknown):
```unknown
Epoch 355/500:  71%|███████   | 355/500 [40:40<16:43,  6.92s/it, loss=1.47e+04, v_num=1]
```

Example 1126 (unknown):
```unknown
Epoch 355/500:  71%|███████   | 355/500 [40:40<16:43,  6.92s/it, loss=1.48e+04, v_num=1]
```

Example 1127 (unknown):
```unknown
Epoch 356/500:  71%|███████   | 355/500 [40:40<16:43,  6.92s/it, loss=1.48e+04, v_num=1]
```

Example 1128 (unknown):
```unknown
Epoch 356/500:  71%|███████   | 356/500 [40:46<16:35,  6.91s/it, loss=1.48e+04, v_num=1]
```

Example 1129 (unknown):
```unknown
Epoch 356/500:  71%|███████   | 356/500 [40:46<16:35,  6.91s/it, loss=1.48e+04, v_num=1]
```

Example 1130 (unknown):
```unknown
Epoch 357/500:  71%|███████   | 356/500 [40:46<16:35,  6.91s/it, loss=1.48e+04, v_num=1]
```

Example 1131 (unknown):
```unknown
Epoch 357/500:  71%|███████▏  | 357/500 [40:53<16:07,  6.77s/it, loss=1.48e+04, v_num=1]
```

Example 1132 (unknown):
```unknown
Epoch 357/500:  71%|███████▏  | 357/500 [40:53<16:07,  6.77s/it, loss=1.47e+04, v_num=1]
```

Example 1133 (unknown):
```unknown
Epoch 358/500:  71%|███████▏  | 357/500 [40:53<16:07,  6.77s/it, loss=1.47e+04, v_num=1]
```

Example 1134 (unknown):
```unknown
Epoch 358/500:  72%|███████▏  | 358/500 [40:59<15:51,  6.70s/it, loss=1.47e+04, v_num=1]
```

Example 1135 (unknown):
```unknown
Epoch 358/500:  72%|███████▏  | 358/500 [40:59<15:51,  6.70s/it, loss=1.49e+04, v_num=1]
```

Example 1136 (unknown):
```unknown
Epoch 359/500:  72%|███████▏  | 358/500 [40:59<15:51,  6.70s/it, loss=1.49e+04, v_num=1]
```

Example 1137 (unknown):
```unknown
Epoch 359/500:  72%|███████▏  | 359/500 [41:06<15:54,  6.77s/it, loss=1.49e+04, v_num=1]
```

Example 1138 (unknown):
```unknown
Epoch 359/500:  72%|███████▏  | 359/500 [41:06<15:54,  6.77s/it, loss=1.47e+04, v_num=1]
```

Example 1139 (unknown):
```unknown
Epoch 360/500:  72%|███████▏  | 359/500 [41:06<15:54,  6.77s/it, loss=1.47e+04, v_num=1]
```

Example 1140 (unknown):
```unknown
Epoch 360/500:  72%|███████▏  | 360/500 [41:13<15:56,  6.83s/it, loss=1.47e+04, v_num=1]
```

Example 1141 (unknown):
```unknown
Epoch 360/500:  72%|███████▏  | 360/500 [41:13<15:56,  6.83s/it, loss=1.47e+04, v_num=1]
```

Example 1142 (unknown):
```unknown
Epoch 361/500:  72%|███████▏  | 360/500 [41:13<15:56,  6.83s/it, loss=1.47e+04, v_num=1]
```

Example 1143 (unknown):
```unknown
Epoch 361/500:  72%|███████▏  | 361/500 [41:20<15:53,  6.86s/it, loss=1.47e+04, v_num=1]
```

Example 1144 (unknown):
```unknown
Epoch 361/500:  72%|███████▏  | 361/500 [41:20<15:53,  6.86s/it, loss=1.5e+04, v_num=1]
```

Example 1145 (unknown):
```unknown
Epoch 362/500:  72%|███████▏  | 361/500 [41:20<15:53,  6.86s/it, loss=1.5e+04, v_num=1]
```

Example 1146 (unknown):
```unknown
Epoch 362/500:  72%|███████▏  | 362/500 [41:27<15:49,  6.88s/it, loss=1.5e+04, v_num=1]
```

Example 1147 (unknown):
```unknown
Epoch 362/500:  72%|███████▏  | 362/500 [41:27<15:49,  6.88s/it, loss=1.49e+04, v_num=1]
```

Example 1148 (unknown):
```unknown
Epoch 363/500:  72%|███████▏  | 362/500 [41:27<15:49,  6.88s/it, loss=1.49e+04, v_num=1]
```

Example 1149 (unknown):
```unknown
Epoch 363/500:  73%|███████▎  | 363/500 [41:34<15:48,  6.92s/it, loss=1.49e+04, v_num=1]
```

Example 1150 (unknown):
```unknown
Epoch 363/500:  73%|███████▎  | 363/500 [41:34<15:48,  6.92s/it, loss=1.47e+04, v_num=1]
```

Example 1151 (unknown):
```unknown
Epoch 364/500:  73%|███████▎  | 363/500 [41:34<15:48,  6.92s/it, loss=1.47e+04, v_num=1]
```

Example 1152 (unknown):
```unknown
Epoch 364/500:  73%|███████▎  | 364/500 [41:41<15:44,  6.95s/it, loss=1.47e+04, v_num=1]
```

Example 1153 (unknown):
```unknown
Epoch 364/500:  73%|███████▎  | 364/500 [41:41<15:44,  6.95s/it, loss=1.49e+04, v_num=1]
```

Example 1154 (unknown):
```unknown
Epoch 365/500:  73%|███████▎  | 364/500 [41:41<15:44,  6.95s/it, loss=1.49e+04, v_num=1]
```

Example 1155 (unknown):
```unknown
Epoch 365/500:  73%|███████▎  | 365/500 [41:48<15:33,  6.91s/it, loss=1.49e+04, v_num=1]
```

Example 1156 (unknown):
```unknown
Epoch 365/500:  73%|███████▎  | 365/500 [41:48<15:33,  6.91s/it, loss=1.49e+04, v_num=1]
```

Example 1157 (unknown):
```unknown
Epoch 366/500:  73%|███████▎  | 365/500 [41:48<15:33,  6.91s/it, loss=1.49e+04, v_num=1]
```

Example 1158 (unknown):
```unknown
Epoch 366/500:  73%|███████▎  | 366/500 [41:55<15:28,  6.93s/it, loss=1.49e+04, v_num=1]
```

Example 1159 (unknown):
```unknown
Epoch 366/500:  73%|███████▎  | 366/500 [41:55<15:28,  6.93s/it, loss=1.47e+04, v_num=1]
```

Example 1160 (unknown):
```unknown
Epoch 367/500:  73%|███████▎  | 366/500 [41:55<15:28,  6.93s/it, loss=1.47e+04, v_num=1]
```

Example 1161 (unknown):
```unknown
Epoch 367/500:  73%|███████▎  | 367/500 [42:02<15:28,  6.98s/it, loss=1.47e+04, v_num=1]
```

Example 1162 (unknown):
```unknown
Epoch 367/500:  73%|███████▎  | 367/500 [42:02<15:28,  6.98s/it, loss=1.46e+04, v_num=1]
```

Example 1163 (unknown):
```unknown
Epoch 368/500:  73%|███████▎  | 367/500 [42:02<15:28,  6.98s/it, loss=1.46e+04, v_num=1]
```

Example 1164 (unknown):
```unknown
Epoch 368/500:  74%|███████▎  | 368/500 [42:09<15:14,  6.93s/it, loss=1.46e+04, v_num=1]
```

Example 1165 (unknown):
```unknown
Epoch 368/500:  74%|███████▎  | 368/500 [42:09<15:14,  6.93s/it, loss=1.52e+04, v_num=1]
```

Example 1166 (unknown):
```unknown
Epoch 369/500:  74%|███████▎  | 368/500 [42:09<15:14,  6.93s/it, loss=1.52e+04, v_num=1]
```

Example 1167 (unknown):
```unknown
Epoch 369/500:  74%|███████▍  | 369/500 [42:16<14:59,  6.87s/it, loss=1.52e+04, v_num=1]
```

Example 1168 (unknown):
```unknown
Epoch 369/500:  74%|███████▍  | 369/500 [42:16<14:59,  6.87s/it, loss=1.48e+04, v_num=1]
```

Example 1169 (unknown):
```unknown
Epoch 370/500:  74%|███████▍  | 369/500 [42:16<14:59,  6.87s/it, loss=1.48e+04, v_num=1]
```

Example 1170 (unknown):
```unknown
Epoch 370/500:  74%|███████▍  | 370/500 [42:22<14:49,  6.84s/it, loss=1.48e+04, v_num=1]
```

Example 1171 (unknown):
```unknown
Epoch 370/500:  74%|███████▍  | 370/500 [42:22<14:49,  6.84s/it, loss=1.49e+04, v_num=1]
```

Example 1172 (unknown):
```unknown
Epoch 371/500:  74%|███████▍  | 370/500 [42:22<14:49,  6.84s/it, loss=1.49e+04, v_num=1]
```

Example 1173 (unknown):
```unknown
Epoch 371/500:  74%|███████▍  | 371/500 [42:29<14:31,  6.75s/it, loss=1.49e+04, v_num=1]
```

Example 1174 (unknown):
```unknown
Epoch 371/500:  74%|███████▍  | 371/500 [42:29<14:31,  6.75s/it, loss=1.48e+04, v_num=1]
```

Example 1175 (unknown):
```unknown
Epoch 372/500:  74%|███████▍  | 371/500 [42:29<14:31,  6.75s/it, loss=1.48e+04, v_num=1]
```

Example 1176 (unknown):
```unknown
Epoch 372/500:  74%|███████▍  | 372/500 [42:35<14:08,  6.63s/it, loss=1.48e+04, v_num=1]
```

Example 1177 (unknown):
```unknown
Epoch 372/500:  74%|███████▍  | 372/500 [42:35<14:08,  6.63s/it, loss=1.43e+04, v_num=1]
```

Example 1178 (unknown):
```unknown
Epoch 373/500:  74%|███████▍  | 372/500 [42:35<14:08,  6.63s/it, loss=1.43e+04, v_num=1]
```

Example 1179 (unknown):
```unknown
Epoch 373/500:  75%|███████▍  | 373/500 [42:42<14:15,  6.73s/it, loss=1.43e+04, v_num=1]
```

Example 1180 (unknown):
```unknown
Epoch 373/500:  75%|███████▍  | 373/500 [42:42<14:15,  6.73s/it, loss=1.48e+04, v_num=1]
```

Example 1181 (unknown):
```unknown
Epoch 374/500:  75%|███████▍  | 373/500 [42:42<14:15,  6.73s/it, loss=1.48e+04, v_num=1]
```

Example 1182 (unknown):
```unknown
Epoch 374/500:  75%|███████▍  | 374/500 [42:49<14:07,  6.73s/it, loss=1.48e+04, v_num=1]
```

Example 1183 (unknown):
```unknown
Epoch 374/500:  75%|███████▍  | 374/500 [42:49<14:07,  6.73s/it, loss=1.49e+04, v_num=1]
```

Example 1184 (unknown):
```unknown
Epoch 375/500:  75%|███████▍  | 374/500 [42:49<14:07,  6.73s/it, loss=1.49e+04, v_num=1]
```

Example 1185 (unknown):
```unknown
Epoch 375/500:  75%|███████▌  | 375/500 [42:56<14:01,  6.73s/it, loss=1.49e+04, v_num=1]
```

Example 1186 (unknown):
```unknown
Epoch 375/500:  75%|███████▌  | 375/500 [42:56<14:01,  6.73s/it, loss=1.5e+04, v_num=1]
```

Example 1187 (unknown):
```unknown
Epoch 376/500:  75%|███████▌  | 375/500 [42:56<14:01,  6.73s/it, loss=1.5e+04, v_num=1]
```

Example 1188 (unknown):
```unknown
Epoch 376/500:  75%|███████▌  | 376/500 [43:03<14:02,  6.79s/it, loss=1.5e+04, v_num=1]
```

Example 1189 (unknown):
```unknown
Epoch 376/500:  75%|███████▌  | 376/500 [43:03<14:02,  6.79s/it, loss=1.46e+04, v_num=1]
```

Example 1190 (unknown):
```unknown
Epoch 377/500:  75%|███████▌  | 376/500 [43:03<14:02,  6.79s/it, loss=1.46e+04, v_num=1]
```

Example 1191 (unknown):
```unknown
Epoch 377/500:  75%|███████▌  | 377/500 [43:09<13:48,  6.73s/it, loss=1.46e+04, v_num=1]
```

Example 1192 (unknown):
```unknown
Epoch 377/500:  75%|███████▌  | 377/500 [43:09<13:48,  6.73s/it, loss=1.46e+04, v_num=1]
```

Example 1193 (unknown):
```unknown
Epoch 378/500:  75%|███████▌  | 377/500 [43:09<13:48,  6.73s/it, loss=1.46e+04, v_num=1]
```

Example 1194 (unknown):
```unknown
Epoch 378/500:  76%|███████▌  | 378/500 [43:16<13:48,  6.79s/it, loss=1.46e+04, v_num=1]
```

Example 1195 (unknown):
```unknown
Epoch 378/500:  76%|███████▌  | 378/500 [43:16<13:48,  6.79s/it, loss=1.47e+04, v_num=1]
```

Example 1196 (unknown):
```unknown
Epoch 379/500:  76%|███████▌  | 378/500 [43:16<13:48,  6.79s/it, loss=1.47e+04, v_num=1]
```

Example 1197 (unknown):
```unknown
Epoch 379/500:  76%|███████▌  | 379/500 [43:23<13:42,  6.80s/it, loss=1.47e+04, v_num=1]
```

Example 1198 (unknown):
```unknown
Epoch 379/500:  76%|███████▌  | 379/500 [43:23<13:42,  6.80s/it, loss=1.51e+04, v_num=1]
```

Example 1199 (unknown):
```unknown
Epoch 380/500:  76%|███████▌  | 379/500 [43:23<13:42,  6.80s/it, loss=1.51e+04, v_num=1]
```

Example 1200 (unknown):
```unknown
Epoch 380/500:  76%|███████▌  | 380/500 [43:30<13:34,  6.79s/it, loss=1.51e+04, v_num=1]
```

Example 1201 (unknown):
```unknown
Epoch 380/500:  76%|███████▌  | 380/500 [43:30<13:34,  6.79s/it, loss=1.46e+04, v_num=1]
```

Example 1202 (unknown):
```unknown
Epoch 381/500:  76%|███████▌  | 380/500 [43:30<13:34,  6.79s/it, loss=1.46e+04, v_num=1]
```

Example 1203 (unknown):
```unknown
Epoch 381/500:  76%|███████▌  | 381/500 [43:37<13:33,  6.84s/it, loss=1.46e+04, v_num=1]
```

Example 1204 (unknown):
```unknown
Epoch 381/500:  76%|███████▌  | 381/500 [43:37<13:33,  6.84s/it, loss=1.49e+04, v_num=1]
```

Example 1205 (unknown):
```unknown
Epoch 382/500:  76%|███████▌  | 381/500 [43:37<13:33,  6.84s/it, loss=1.49e+04, v_num=1]
```

Example 1206 (unknown):
```unknown
Epoch 382/500:  76%|███████▋  | 382/500 [43:43<13:18,  6.77s/it, loss=1.49e+04, v_num=1]
```

Example 1207 (unknown):
```unknown
Epoch 382/500:  76%|███████▋  | 382/500 [43:43<13:18,  6.77s/it, loss=1.45e+04, v_num=1]
```

Example 1208 (unknown):
```unknown
Epoch 383/500:  76%|███████▋  | 382/500 [43:43<13:18,  6.77s/it, loss=1.45e+04, v_num=1]
```

Example 1209 (unknown):
```unknown
Epoch 383/500:  77%|███████▋  | 383/500 [43:50<13:22,  6.86s/it, loss=1.45e+04, v_num=1]
```

Example 1210 (unknown):
```unknown
Epoch 383/500:  77%|███████▋  | 383/500 [43:50<13:22,  6.86s/it, loss=1.49e+04, v_num=1]
```

Example 1211 (unknown):
```unknown
Epoch 384/500:  77%|███████▋  | 383/500 [43:50<13:22,  6.86s/it, loss=1.49e+04, v_num=1]
```

Example 1212 (unknown):
```unknown
Epoch 384/500:  77%|███████▋  | 384/500 [43:57<13:22,  6.92s/it, loss=1.49e+04, v_num=1]
```

Example 1213 (unknown):
```unknown
Epoch 384/500:  77%|███████▋  | 384/500 [43:57<13:22,  6.92s/it, loss=1.47e+04, v_num=1]
```

Example 1214 (unknown):
```unknown
Epoch 385/500:  77%|███████▋  | 384/500 [43:57<13:22,  6.92s/it, loss=1.47e+04, v_num=1]
```

Example 1215 (unknown):
```unknown
Epoch 385/500:  77%|███████▋  | 385/500 [44:04<13:16,  6.92s/it, loss=1.47e+04, v_num=1]
```

Example 1216 (unknown):
```unknown
Epoch 385/500:  77%|███████▋  | 385/500 [44:04<13:16,  6.92s/it, loss=1.52e+04, v_num=1]
```

Example 1217 (unknown):
```unknown
Epoch 386/500:  77%|███████▋  | 385/500 [44:04<13:16,  6.92s/it, loss=1.52e+04, v_num=1]
```

Example 1218 (unknown):
```unknown
Epoch 386/500:  77%|███████▋  | 386/500 [44:11<13:11,  6.94s/it, loss=1.52e+04, v_num=1]
```

Example 1219 (unknown):
```unknown
Epoch 386/500:  77%|███████▋  | 386/500 [44:11<13:11,  6.94s/it, loss=1.47e+04, v_num=1]
```

Example 1220 (unknown):
```unknown
Epoch 387/500:  77%|███████▋  | 386/500 [44:11<13:11,  6.94s/it, loss=1.47e+04, v_num=1]
```

Example 1221 (unknown):
```unknown
Epoch 387/500:  77%|███████▋  | 387/500 [44:18<13:07,  6.97s/it, loss=1.47e+04, v_num=1]
```

Example 1222 (unknown):
```unknown
Epoch 387/500:  77%|███████▋  | 387/500 [44:18<13:07,  6.97s/it, loss=1.47e+04, v_num=1]
```

Example 1223 (unknown):
```unknown
Epoch 388/500:  77%|███████▋  | 387/500 [44:18<13:07,  6.97s/it, loss=1.47e+04, v_num=1]
```

Example 1224 (unknown):
```unknown
Epoch 388/500:  78%|███████▊  | 388/500 [44:25<12:59,  6.96s/it, loss=1.47e+04, v_num=1]
```

Example 1225 (unknown):
```unknown
Epoch 388/500:  78%|███████▊  | 388/500 [44:25<12:59,  6.96s/it, loss=1.5e+04, v_num=1]
```

Example 1226 (unknown):
```unknown
Epoch 389/500:  78%|███████▊  | 388/500 [44:25<12:59,  6.96s/it, loss=1.5e+04, v_num=1]
```

Example 1227 (unknown):
```unknown
Epoch 389/500:  78%|███████▊  | 389/500 [44:32<12:52,  6.96s/it, loss=1.5e+04, v_num=1]
```

Example 1228 (unknown):
```unknown
Epoch 389/500:  78%|███████▊  | 389/500 [44:32<12:52,  6.96s/it, loss=1.47e+04, v_num=1]
```

Example 1229 (unknown):
```unknown
Epoch 390/500:  78%|███████▊  | 389/500 [44:32<12:52,  6.96s/it, loss=1.47e+04, v_num=1]
```

Example 1230 (unknown):
```unknown
Epoch 390/500:  78%|███████▊  | 390/500 [44:39<12:42,  6.93s/it, loss=1.47e+04, v_num=1]
```

Example 1231 (unknown):
```unknown
Epoch 390/500:  78%|███████▊  | 390/500 [44:39<12:42,  6.93s/it, loss=1.48e+04, v_num=1]
```

Example 1232 (unknown):
```unknown
Epoch 391/500:  78%|███████▊  | 390/500 [44:39<12:42,  6.93s/it, loss=1.48e+04, v_num=1]
```

Example 1233 (unknown):
```unknown
Epoch 391/500:  78%|███████▊  | 391/500 [44:46<12:32,  6.90s/it, loss=1.48e+04, v_num=1]
```

Example 1234 (unknown):
```unknown
Epoch 391/500:  78%|███████▊  | 391/500 [44:46<12:32,  6.90s/it, loss=1.48e+04, v_num=1]
```

Example 1235 (unknown):
```unknown
Epoch 392/500:  78%|███████▊  | 391/500 [44:46<12:32,  6.90s/it, loss=1.48e+04, v_num=1]
```

Example 1236 (unknown):
```unknown
Epoch 392/500:  78%|███████▊  | 392/500 [44:53<12:20,  6.86s/it, loss=1.48e+04, v_num=1]
```

Example 1237 (unknown):
```unknown
Epoch 392/500:  78%|███████▊  | 392/500 [44:53<12:20,  6.86s/it, loss=1.48e+04, v_num=1]
```

Example 1238 (unknown):
```unknown
Epoch 393/500:  78%|███████▊  | 392/500 [44:53<12:20,  6.86s/it, loss=1.48e+04, v_num=1]
```

Example 1239 (unknown):
```unknown
Epoch 393/500:  79%|███████▊  | 393/500 [45:00<12:10,  6.82s/it, loss=1.48e+04, v_num=1]
```

Example 1240 (unknown):
```unknown
Epoch 393/500:  79%|███████▊  | 393/500 [45:00<12:10,  6.82s/it, loss=1.45e+04, v_num=1]
```

Example 1241 (unknown):
```unknown
Epoch 394/500:  79%|███████▊  | 393/500 [45:00<12:10,  6.82s/it, loss=1.45e+04, v_num=1]
```

Example 1242 (unknown):
```unknown
Epoch 394/500:  79%|███████▉  | 394/500 [45:06<12:07,  6.86s/it, loss=1.45e+04, v_num=1]
```

Example 1243 (unknown):
```unknown
Epoch 394/500:  79%|███████▉  | 394/500 [45:06<12:07,  6.86s/it, loss=1.47e+04, v_num=1]
```

Example 1244 (unknown):
```unknown
Epoch 395/500:  79%|███████▉  | 394/500 [45:06<12:07,  6.86s/it, loss=1.47e+04, v_num=1]
```

Example 1245 (unknown):
```unknown
Epoch 395/500:  79%|███████▉  | 395/500 [45:13<12:05,  6.91s/it, loss=1.47e+04, v_num=1]
```

Example 1246 (unknown):
```unknown
Epoch 395/500:  79%|███████▉  | 395/500 [45:13<12:05,  6.91s/it, loss=1.5e+04, v_num=1]
```

Example 1247 (unknown):
```unknown
Epoch 396/500:  79%|███████▉  | 395/500 [45:13<12:05,  6.91s/it, loss=1.5e+04, v_num=1]
```

Example 1248 (unknown):
```unknown
Epoch 396/500:  79%|███████▉  | 396/500 [45:20<12:01,  6.93s/it, loss=1.5e+04, v_num=1]
```

Example 1249 (unknown):
```unknown
Epoch 396/500:  79%|███████▉  | 396/500 [45:20<12:01,  6.93s/it, loss=1.47e+04, v_num=1]
```

Example 1250 (unknown):
```unknown
Epoch 397/500:  79%|███████▉  | 396/500 [45:20<12:01,  6.93s/it, loss=1.47e+04, v_num=1]
```

Example 1251 (unknown):
```unknown
Epoch 397/500:  79%|███████▉  | 397/500 [45:27<11:49,  6.88s/it, loss=1.47e+04, v_num=1]
```

Example 1252 (unknown):
```unknown
Epoch 397/500:  79%|███████▉  | 397/500 [45:27<11:49,  6.88s/it, loss=1.46e+04, v_num=1]
```

Example 1253 (unknown):
```unknown
Epoch 398/500:  79%|███████▉  | 397/500 [45:27<11:49,  6.88s/it, loss=1.46e+04, v_num=1]
```

Example 1254 (unknown):
```unknown
Epoch 398/500:  80%|███████▉  | 398/500 [45:34<11:47,  6.94s/it, loss=1.46e+04, v_num=1]
```

Example 1255 (unknown):
```unknown
Epoch 398/500:  80%|███████▉  | 398/500 [45:34<11:47,  6.94s/it, loss=1.48e+04, v_num=1]
```

Example 1256 (unknown):
```unknown
Epoch 399/500:  80%|███████▉  | 398/500 [45:34<11:47,  6.94s/it, loss=1.48e+04, v_num=1]
```

Example 1257 (unknown):
```unknown
Epoch 399/500:  80%|███████▉  | 399/500 [45:41<11:43,  6.97s/it, loss=1.48e+04, v_num=1]
```

Example 1258 (unknown):
```unknown
Epoch 399/500:  80%|███████▉  | 399/500 [45:41<11:43,  6.97s/it, loss=1.49e+04, v_num=1]
```

Example 1259 (unknown):
```unknown
Epoch 400/500:  80%|███████▉  | 399/500 [45:41<11:43,  6.97s/it, loss=1.49e+04, v_num=1]
```

Example 1260 (unknown):
```unknown
Epoch 400/500:  80%|████████  | 400/500 [45:48<11:31,  6.91s/it, loss=1.49e+04, v_num=1]
```

Example 1261 (unknown):
```unknown
Epoch 400/500:  80%|████████  | 400/500 [45:48<11:31,  6.91s/it, loss=1.5e+04, v_num=1]
```

Example 1262 (unknown):
```unknown
Epoch 401/500:  80%|████████  | 400/500 [45:48<11:31,  6.91s/it, loss=1.5e+04, v_num=1]
```

Example 1263 (unknown):
```unknown
Epoch 401/500:  80%|████████  | 401/500 [45:55<11:24,  6.91s/it, loss=1.5e+04, v_num=1]
```

Example 1264 (unknown):
```unknown
Epoch 401/500:  80%|████████  | 401/500 [45:55<11:24,  6.91s/it, loss=1.48e+04, v_num=1]
```

Example 1265 (unknown):
```unknown
Epoch 402/500:  80%|████████  | 401/500 [45:55<11:24,  6.91s/it, loss=1.48e+04, v_num=1]
```

Example 1266 (unknown):
```unknown
Epoch 402/500:  80%|████████  | 402/500 [46:02<11:14,  6.88s/it, loss=1.48e+04, v_num=1]
```

Example 1267 (unknown):
```unknown
Epoch 402/500:  80%|████████  | 402/500 [46:02<11:14,  6.88s/it, loss=1.45e+04, v_num=1]
```

Example 1268 (unknown):
```unknown
Epoch 403/500:  80%|████████  | 402/500 [46:02<11:14,  6.88s/it, loss=1.45e+04, v_num=1]
```

Example 1269 (unknown):
```unknown
Epoch 403/500:  81%|████████  | 403/500 [46:09<11:10,  6.91s/it, loss=1.45e+04, v_num=1]
```

Example 1270 (unknown):
```unknown
Epoch 403/500:  81%|████████  | 403/500 [46:09<11:10,  6.91s/it, loss=1.47e+04, v_num=1]
```

Example 1271 (unknown):
```unknown
Epoch 404/500:  81%|████████  | 403/500 [46:09<11:10,  6.91s/it, loss=1.47e+04, v_num=1]
```

Example 1272 (unknown):
```unknown
Epoch 404/500:  81%|████████  | 404/500 [46:16<11:04,  6.92s/it, loss=1.47e+04, v_num=1]
```

Example 1273 (unknown):
```unknown
Epoch 404/500:  81%|████████  | 404/500 [46:16<11:04,  6.92s/it, loss=1.45e+04, v_num=1]
```

Example 1274 (unknown):
```unknown
Epoch 405/500:  81%|████████  | 404/500 [46:16<11:04,  6.92s/it, loss=1.45e+04, v_num=1]
```

Example 1275 (unknown):
```unknown
Epoch 405/500:  81%|████████  | 405/500 [46:23<10:57,  6.92s/it, loss=1.45e+04, v_num=1]
```

Example 1276 (unknown):
```unknown
Epoch 405/500:  81%|████████  | 405/500 [46:23<10:57,  6.92s/it, loss=1.46e+04, v_num=1]
```

Example 1277 (unknown):
```unknown
Epoch 406/500:  81%|████████  | 405/500 [46:23<10:57,  6.92s/it, loss=1.46e+04, v_num=1]
```

Example 1278 (unknown):
```unknown
Epoch 406/500:  81%|████████  | 406/500 [46:30<10:51,  6.93s/it, loss=1.46e+04, v_num=1]
```

Example 1279 (unknown):
```unknown
Epoch 406/500:  81%|████████  | 406/500 [46:30<10:51,  6.93s/it, loss=1.46e+04, v_num=1]
```

Example 1280 (unknown):
```unknown
Epoch 407/500:  81%|████████  | 406/500 [46:30<10:51,  6.93s/it, loss=1.46e+04, v_num=1]
```

Example 1281 (unknown):
```unknown
Epoch 407/500:  81%|████████▏ | 407/500 [46:37<10:43,  6.92s/it, loss=1.46e+04, v_num=1]
```

Example 1282 (unknown):
```unknown
Epoch 407/500:  81%|████████▏ | 407/500 [46:37<10:43,  6.92s/it, loss=1.51e+04, v_num=1]
```

Example 1283 (unknown):
```unknown
Epoch 408/500:  81%|████████▏ | 407/500 [46:37<10:43,  6.92s/it, loss=1.51e+04, v_num=1]
```

Example 1284 (unknown):
```unknown
Epoch 408/500:  82%|████████▏ | 408/500 [46:44<10:38,  6.94s/it, loss=1.51e+04, v_num=1]
```

Example 1285 (unknown):
```unknown
Epoch 408/500:  82%|████████▏ | 408/500 [46:44<10:38,  6.94s/it, loss=1.5e+04, v_num=1]
```

Example 1286 (unknown):
```unknown
Epoch 409/500:  82%|████████▏ | 408/500 [46:44<10:38,  6.94s/it, loss=1.5e+04, v_num=1]
```

Example 1287 (unknown):
```unknown
Epoch 409/500:  82%|████████▏ | 409/500 [46:51<10:33,  6.96s/it, loss=1.5e+04, v_num=1]
```

Example 1288 (unknown):
```unknown
Epoch 409/500:  82%|████████▏ | 409/500 [46:51<10:33,  6.96s/it, loss=1.48e+04, v_num=1]
```

Example 1289 (unknown):
```unknown
Epoch 410/500:  82%|████████▏ | 409/500 [46:51<10:33,  6.96s/it, loss=1.48e+04, v_num=1]
```

Example 1290 (unknown):
```unknown
Epoch 410/500:  82%|████████▏ | 410/500 [46:57<10:22,  6.91s/it, loss=1.48e+04, v_num=1]
```

Example 1291 (unknown):
```unknown
Epoch 410/500:  82%|████████▏ | 410/500 [46:57<10:22,  6.91s/it, loss=1.48e+04, v_num=1]
```

Example 1292 (unknown):
```unknown
Epoch 411/500:  82%|████████▏ | 410/500 [46:57<10:22,  6.91s/it, loss=1.48e+04, v_num=1]
```

Example 1293 (unknown):
```unknown
Epoch 411/500:  82%|████████▏ | 411/500 [47:04<10:07,  6.82s/it, loss=1.48e+04, v_num=1]
```

Example 1294 (unknown):
```unknown
Epoch 411/500:  82%|████████▏ | 411/500 [47:04<10:07,  6.82s/it, loss=1.48e+04, v_num=1]
```

Example 1295 (unknown):
```unknown
Epoch 412/500:  82%|████████▏ | 411/500 [47:04<10:07,  6.82s/it, loss=1.48e+04, v_num=1]
```

Example 1296 (unknown):
```unknown
Epoch 412/500:  82%|████████▏ | 412/500 [47:11<10:03,  6.86s/it, loss=1.48e+04, v_num=1]
```

Example 1297 (unknown):
```unknown
Epoch 412/500:  82%|████████▏ | 412/500 [47:11<10:03,  6.86s/it, loss=1.47e+04, v_num=1]
```

Example 1298 (unknown):
```unknown
Epoch 413/500:  82%|████████▏ | 412/500 [47:11<10:03,  6.86s/it, loss=1.47e+04, v_num=1]
```

Example 1299 (unknown):
```unknown
Epoch 413/500:  83%|████████▎ | 413/500 [47:18<09:55,  6.85s/it, loss=1.47e+04, v_num=1]
```

Example 1300 (unknown):
```unknown
Epoch 413/500:  83%|████████▎ | 413/500 [47:18<09:55,  6.85s/it, loss=1.47e+04, v_num=1]
```

Example 1301 (unknown):
```unknown
Epoch 414/500:  83%|████████▎ | 413/500 [47:18<09:55,  6.85s/it, loss=1.47e+04, v_num=1]
```

Example 1302 (unknown):
```unknown
Epoch 414/500:  83%|████████▎ | 414/500 [47:24<09:42,  6.77s/it, loss=1.47e+04, v_num=1]
```

Example 1303 (unknown):
```unknown
Epoch 414/500:  83%|████████▎ | 414/500 [47:24<09:42,  6.77s/it, loss=1.47e+04, v_num=1]
```

Example 1304 (unknown):
```unknown
Epoch 415/500:  83%|████████▎ | 414/500 [47:24<09:42,  6.77s/it, loss=1.47e+04, v_num=1]
```

Example 1305 (unknown):
```unknown
Epoch 415/500:  83%|████████▎ | 415/500 [47:31<09:40,  6.83s/it, loss=1.47e+04, v_num=1]
```

Example 1306 (unknown):
```unknown
Epoch 415/500:  83%|████████▎ | 415/500 [47:31<09:40,  6.83s/it, loss=1.47e+04, v_num=1]
```

Example 1307 (unknown):
```unknown
Epoch 416/500:  83%|████████▎ | 415/500 [47:31<09:40,  6.83s/it, loss=1.47e+04, v_num=1]
```

Example 1308 (unknown):
```unknown
Epoch 416/500:  83%|████████▎ | 416/500 [47:38<09:37,  6.88s/it, loss=1.47e+04, v_num=1]
```

Example 1309 (unknown):
```unknown
Epoch 416/500:  83%|████████▎ | 416/500 [47:38<09:37,  6.88s/it, loss=1.47e+04, v_num=1]
```

Example 1310 (unknown):
```unknown
Epoch 417/500:  83%|████████▎ | 416/500 [47:38<09:37,  6.88s/it, loss=1.47e+04, v_num=1]
```

Example 1311 (unknown):
```unknown
Epoch 417/500:  83%|████████▎ | 417/500 [47:45<09:36,  6.94s/it, loss=1.47e+04, v_num=1]
```

Example 1312 (unknown):
```unknown
Epoch 417/500:  83%|████████▎ | 417/500 [47:45<09:36,  6.94s/it, loss=1.49e+04, v_num=1]
```

Example 1313 (unknown):
```unknown
Epoch 418/500:  83%|████████▎ | 417/500 [47:45<09:36,  6.94s/it, loss=1.49e+04, v_num=1]
```

Example 1314 (unknown):
```unknown
Epoch 418/500:  84%|████████▎ | 418/500 [47:52<09:29,  6.94s/it, loss=1.49e+04, v_num=1]
```

Example 1315 (unknown):
```unknown
Epoch 418/500:  84%|████████▎ | 418/500 [47:52<09:29,  6.94s/it, loss=1.48e+04, v_num=1]
```

Example 1316 (unknown):
```unknown
Epoch 419/500:  84%|████████▎ | 418/500 [47:52<09:29,  6.94s/it, loss=1.48e+04, v_num=1]
```

Example 1317 (unknown):
```unknown
Epoch 419/500:  84%|████████▍ | 419/500 [47:59<09:19,  6.91s/it, loss=1.48e+04, v_num=1]
```

Example 1318 (unknown):
```unknown
Epoch 419/500:  84%|████████▍ | 419/500 [47:59<09:19,  6.91s/it, loss=1.48e+04, v_num=1]
```

Example 1319 (unknown):
```unknown
Epoch 420/500:  84%|████████▍ | 419/500 [47:59<09:19,  6.91s/it, loss=1.48e+04, v_num=1]
```

Example 1320 (unknown):
```unknown
Epoch 420/500:  84%|████████▍ | 420/500 [48:06<09:15,  6.94s/it, loss=1.48e+04, v_num=1]
```

Example 1321 (unknown):
```unknown
Epoch 420/500:  84%|████████▍ | 420/500 [48:06<09:15,  6.94s/it, loss=1.46e+04, v_num=1]
```

Example 1322 (unknown):
```unknown
Epoch 421/500:  84%|████████▍ | 420/500 [48:06<09:15,  6.94s/it, loss=1.46e+04, v_num=1]
```

Example 1323 (unknown):
```unknown
Epoch 421/500:  84%|████████▍ | 421/500 [48:13<09:08,  6.95s/it, loss=1.46e+04, v_num=1]
```

Example 1324 (unknown):
```unknown
Epoch 421/500:  84%|████████▍ | 421/500 [48:13<09:08,  6.95s/it, loss=1.46e+04, v_num=1]
```

Example 1325 (unknown):
```unknown
Epoch 422/500:  84%|████████▍ | 421/500 [48:13<09:08,  6.95s/it, loss=1.46e+04, v_num=1]
```

Example 1326 (unknown):
```unknown
Epoch 422/500:  84%|████████▍ | 422/500 [48:20<08:59,  6.92s/it, loss=1.46e+04, v_num=1]
```

Example 1327 (unknown):
```unknown
Epoch 422/500:  84%|████████▍ | 422/500 [48:20<08:59,  6.92s/it, loss=1.47e+04, v_num=1]
```

Example 1328 (unknown):
```unknown
Epoch 423/500:  84%|████████▍ | 422/500 [48:20<08:59,  6.92s/it, loss=1.47e+04, v_num=1]
```

Example 1329 (unknown):
```unknown
Epoch 423/500:  85%|████████▍ | 423/500 [48:27<08:52,  6.92s/it, loss=1.47e+04, v_num=1]
```

Example 1330 (unknown):
```unknown
Epoch 423/500:  85%|████████▍ | 423/500 [48:27<08:52,  6.92s/it, loss=1.47e+04, v_num=1]
```

Example 1331 (unknown):
```unknown
Epoch 424/500:  85%|████████▍ | 423/500 [48:27<08:52,  6.92s/it, loss=1.47e+04, v_num=1]
```

Example 1332 (unknown):
```unknown
Epoch 424/500:  85%|████████▍ | 424/500 [48:34<08:47,  6.93s/it, loss=1.47e+04, v_num=1]
```

Example 1333 (unknown):
```unknown
Epoch 424/500:  85%|████████▍ | 424/500 [48:34<08:47,  6.93s/it, loss=1.46e+04, v_num=1]
```

Example 1334 (unknown):
```unknown
Epoch 425/500:  85%|████████▍ | 424/500 [48:34<08:47,  6.93s/it, loss=1.46e+04, v_num=1]
```

Example 1335 (unknown):
```unknown
Epoch 425/500:  85%|████████▌ | 425/500 [48:41<08:41,  6.95s/it, loss=1.46e+04, v_num=1]
```

Example 1336 (unknown):
```unknown
Epoch 425/500:  85%|████████▌ | 425/500 [48:41<08:41,  6.95s/it, loss=1.41e+04, v_num=1]
```

Example 1337 (unknown):
```unknown
Epoch 426/500:  85%|████████▌ | 425/500 [48:41<08:41,  6.95s/it, loss=1.41e+04, v_num=1]
```

Example 1338 (unknown):
```unknown
Epoch 426/500:  85%|████████▌ | 426/500 [48:48<08:34,  6.95s/it, loss=1.41e+04, v_num=1]
```

Example 1339 (unknown):
```unknown
Epoch 426/500:  85%|████████▌ | 426/500 [48:48<08:34,  6.95s/it, loss=1.43e+04, v_num=1]
```

Example 1340 (unknown):
```unknown
Epoch 427/500:  85%|████████▌ | 426/500 [48:48<08:34,  6.95s/it, loss=1.43e+04, v_num=1]
```

Example 1341 (unknown):
```unknown
Epoch 427/500:  85%|████████▌ | 427/500 [48:55<08:25,  6.93s/it, loss=1.43e+04, v_num=1]
```

Example 1342 (unknown):
```unknown
Epoch 427/500:  85%|████████▌ | 427/500 [48:55<08:25,  6.93s/it, loss=1.49e+04, v_num=1]
```

Example 1343 (unknown):
```unknown
Epoch 428/500:  85%|████████▌ | 427/500 [48:55<08:25,  6.93s/it, loss=1.49e+04, v_num=1]
```

Example 1344 (unknown):
```unknown
Epoch 428/500:  86%|████████▌ | 428/500 [49:01<08:11,  6.83s/it, loss=1.49e+04, v_num=1]
```

Example 1345 (unknown):
```unknown
Epoch 428/500:  86%|████████▌ | 428/500 [49:01<08:11,  6.83s/it, loss=1.44e+04, v_num=1]
```

Example 1346 (unknown):
```unknown
Epoch 429/500:  86%|████████▌ | 428/500 [49:01<08:11,  6.83s/it, loss=1.44e+04, v_num=1]
```

Example 1347 (unknown):
```unknown
Epoch 429/500:  86%|████████▌ | 429/500 [49:08<07:59,  6.75s/it, loss=1.44e+04, v_num=1]
```

Example 1348 (unknown):
```unknown
Epoch 429/500:  86%|████████▌ | 429/500 [49:08<07:59,  6.75s/it, loss=1.47e+04, v_num=1]
```

Example 1349 (unknown):
```unknown
Epoch 430/500:  86%|████████▌ | 429/500 [49:08<07:59,  6.75s/it, loss=1.47e+04, v_num=1]
```

Example 1350 (unknown):
```unknown
Epoch 430/500:  86%|████████▌ | 430/500 [49:15<07:56,  6.80s/it, loss=1.47e+04, v_num=1]
```

Example 1351 (unknown):
```unknown
Epoch 430/500:  86%|████████▌ | 430/500 [49:15<07:56,  6.80s/it, loss=1.47e+04, v_num=1]
```

Example 1352 (unknown):
```unknown
Epoch 431/500:  86%|████████▌ | 430/500 [49:15<07:56,  6.80s/it, loss=1.47e+04, v_num=1]
```

Example 1353 (unknown):
```unknown
Epoch 431/500:  86%|████████▌ | 431/500 [49:22<07:52,  6.85s/it, loss=1.47e+04, v_num=1]
```

Example 1354 (unknown):
```unknown
Epoch 431/500:  86%|████████▌ | 431/500 [49:22<07:52,  6.85s/it, loss=1.46e+04, v_num=1]
```

Example 1355 (unknown):
```unknown
Epoch 432/500:  86%|████████▌ | 431/500 [49:22<07:52,  6.85s/it, loss=1.46e+04, v_num=1]
```

Example 1356 (unknown):
```unknown
Epoch 432/500:  86%|████████▋ | 432/500 [49:29<07:46,  6.86s/it, loss=1.46e+04, v_num=1]
```

Example 1357 (unknown):
```unknown
Epoch 432/500:  86%|████████▋ | 432/500 [49:29<07:46,  6.86s/it, loss=1.46e+04, v_num=1]
```

Example 1358 (unknown):
```unknown
Epoch 433/500:  86%|████████▋ | 432/500 [49:29<07:46,  6.86s/it, loss=1.46e+04, v_num=1]
```

Example 1359 (unknown):
```unknown
Epoch 433/500:  87%|████████▋ | 433/500 [49:36<07:41,  6.89s/it, loss=1.46e+04, v_num=1]
```

Example 1360 (unknown):
```unknown
Epoch 433/500:  87%|████████▋ | 433/500 [49:36<07:41,  6.89s/it, loss=1.44e+04, v_num=1]
```

Example 1361 (unknown):
```unknown
Epoch 434/500:  87%|████████▋ | 433/500 [49:36<07:41,  6.89s/it, loss=1.44e+04, v_num=1]
```

Example 1362 (unknown):
```unknown
Epoch 434/500:  87%|████████▋ | 434/500 [49:42<07:25,  6.75s/it, loss=1.44e+04, v_num=1]
```

Example 1363 (unknown):
```unknown
Epoch 434/500:  87%|████████▋ | 434/500 [49:42<07:25,  6.75s/it, loss=1.47e+04, v_num=1]
```

Example 1364 (unknown):
```unknown
Epoch 435/500:  87%|████████▋ | 434/500 [49:42<07:25,  6.75s/it, loss=1.47e+04, v_num=1]
```

Example 1365 (unknown):
```unknown
Epoch 435/500:  87%|████████▋ | 435/500 [49:49<07:19,  6.76s/it, loss=1.47e+04, v_num=1]
```

Example 1366 (unknown):
```unknown
Epoch 435/500:  87%|████████▋ | 435/500 [49:49<07:19,  6.76s/it, loss=1.45e+04, v_num=1]
```

Example 1367 (unknown):
```unknown
Epoch 436/500:  87%|████████▋ | 435/500 [49:49<07:19,  6.76s/it, loss=1.45e+04, v_num=1]
```

Example 1368 (unknown):
```unknown
Epoch 436/500:  87%|████████▋ | 436/500 [49:55<07:11,  6.75s/it, loss=1.45e+04, v_num=1]
```

Example 1369 (unknown):
```unknown
Epoch 436/500:  87%|████████▋ | 436/500 [49:55<07:11,  6.75s/it, loss=1.48e+04, v_num=1]
```

Example 1370 (unknown):
```unknown
Epoch 437/500:  87%|████████▋ | 436/500 [49:55<07:11,  6.75s/it, loss=1.48e+04, v_num=1]
```

Example 1371 (unknown):
```unknown
Epoch 437/500:  87%|████████▋ | 437/500 [50:02<07:04,  6.73s/it, loss=1.48e+04, v_num=1]
```

Example 1372 (unknown):
```unknown
Epoch 437/500:  87%|████████▋ | 437/500 [50:02<07:04,  6.73s/it, loss=1.49e+04, v_num=1]
```

Example 1373 (unknown):
```unknown
Epoch 438/500:  87%|████████▋ | 437/500 [50:02<07:04,  6.73s/it, loss=1.49e+04, v_num=1]
```

Example 1374 (unknown):
```unknown
Epoch 438/500:  88%|████████▊ | 438/500 [50:09<07:03,  6.82s/it, loss=1.49e+04, v_num=1]
```

Example 1375 (unknown):
```unknown
Epoch 438/500:  88%|████████▊ | 438/500 [50:09<07:03,  6.82s/it, loss=1.49e+04, v_num=1]
```

Example 1376 (unknown):
```unknown
Epoch 439/500:  88%|████████▊ | 438/500 [50:09<07:03,  6.82s/it, loss=1.49e+04, v_num=1]
```

Example 1377 (unknown):
```unknown
Epoch 439/500:  88%|████████▊ | 439/500 [50:16<06:53,  6.77s/it, loss=1.49e+04, v_num=1]
```

Example 1378 (unknown):
```unknown
Epoch 439/500:  88%|████████▊ | 439/500 [50:16<06:53,  6.77s/it, loss=1.47e+04, v_num=1]
```

Example 1379 (unknown):
```unknown
Epoch 440/500:  88%|████████▊ | 439/500 [50:16<06:53,  6.77s/it, loss=1.47e+04, v_num=1]
```

Example 1380 (unknown):
```unknown
Epoch 440/500:  88%|████████▊ | 440/500 [50:23<06:48,  6.80s/it, loss=1.47e+04, v_num=1]
```

Example 1381 (unknown):
```unknown
Epoch 440/500:  88%|████████▊ | 440/500 [50:23<06:48,  6.80s/it, loss=1.46e+04, v_num=1]
```

Example 1382 (unknown):
```unknown
Epoch 441/500:  88%|████████▊ | 440/500 [50:23<06:48,  6.80s/it, loss=1.46e+04, v_num=1]
```

Example 1383 (unknown):
```unknown
Epoch 441/500:  88%|████████▊ | 441/500 [50:30<06:44,  6.85s/it, loss=1.46e+04, v_num=1]
```

Example 1384 (unknown):
```unknown
Epoch 441/500:  88%|████████▊ | 441/500 [50:30<06:44,  6.85s/it, loss=1.47e+04, v_num=1]
```

Example 1385 (unknown):
```unknown
Epoch 442/500:  88%|████████▊ | 441/500 [50:30<06:44,  6.85s/it, loss=1.47e+04, v_num=1]
```

Example 1386 (unknown):
```unknown
Epoch 442/500:  88%|████████▊ | 442/500 [50:37<06:38,  6.87s/it, loss=1.47e+04, v_num=1]
```

Example 1387 (unknown):
```unknown
Epoch 442/500:  88%|████████▊ | 442/500 [50:37<06:38,  6.87s/it, loss=1.45e+04, v_num=1]
```

Example 1388 (unknown):
```unknown
Epoch 443/500:  88%|████████▊ | 442/500 [50:37<06:38,  6.87s/it, loss=1.45e+04, v_num=1]
```

Example 1389 (unknown):
```unknown
Epoch 443/500:  89%|████████▊ | 443/500 [50:43<06:31,  6.86s/it, loss=1.45e+04, v_num=1]
```

Example 1390 (unknown):
```unknown
Epoch 443/500:  89%|████████▊ | 443/500 [50:43<06:31,  6.86s/it, loss=1.47e+04, v_num=1]
```

Example 1391 (unknown):
```unknown
Epoch 444/500:  89%|████████▊ | 443/500 [50:43<06:31,  6.86s/it, loss=1.47e+04, v_num=1]
```

Example 1392 (unknown):
```unknown
Epoch 444/500:  89%|████████▉ | 444/500 [50:50<06:21,  6.82s/it, loss=1.47e+04, v_num=1]
```

Example 1393 (unknown):
```unknown
Epoch 444/500:  89%|████████▉ | 444/500 [50:50<06:21,  6.82s/it, loss=1.46e+04, v_num=1]
```

Example 1394 (unknown):
```unknown
Epoch 445/500:  89%|████████▉ | 444/500 [50:50<06:21,  6.82s/it, loss=1.46e+04, v_num=1]
```

Example 1395 (unknown):
```unknown
Epoch 445/500:  89%|████████▉ | 445/500 [50:57<06:19,  6.90s/it, loss=1.46e+04, v_num=1]
```

Example 1396 (unknown):
```unknown
Epoch 445/500:  89%|████████▉ | 445/500 [50:57<06:19,  6.90s/it, loss=1.46e+04, v_num=1]
```

Example 1397 (unknown):
```unknown
Epoch 446/500:  89%|████████▉ | 445/500 [50:57<06:19,  6.90s/it, loss=1.46e+04, v_num=1]
```

Example 1398 (unknown):
```unknown
Epoch 446/500:  89%|████████▉ | 446/500 [51:04<06:13,  6.91s/it, loss=1.46e+04, v_num=1]
```

Example 1399 (unknown):
```unknown
Epoch 446/500:  89%|████████▉ | 446/500 [51:04<06:13,  6.91s/it, loss=1.46e+04, v_num=1]
```

Example 1400 (unknown):
```unknown
Epoch 447/500:  89%|████████▉ | 446/500 [51:04<06:13,  6.91s/it, loss=1.46e+04, v_num=1]
```

Example 1401 (unknown):
```unknown
Epoch 447/500:  89%|████████▉ | 447/500 [51:11<06:05,  6.90s/it, loss=1.46e+04, v_num=1]
```

Example 1402 (unknown):
```unknown
Epoch 447/500:  89%|████████▉ | 447/500 [51:11<06:05,  6.90s/it, loss=1.45e+04, v_num=1]
```

Example 1403 (unknown):
```unknown
Epoch 448/500:  89%|████████▉ | 447/500 [51:11<06:05,  6.90s/it, loss=1.45e+04, v_num=1]
```

Example 1404 (unknown):
```unknown
Epoch 448/500:  90%|████████▉ | 448/500 [51:18<06:00,  6.93s/it, loss=1.45e+04, v_num=1]
```

Example 1405 (unknown):
```unknown
Epoch 448/500:  90%|████████▉ | 448/500 [51:18<06:00,  6.93s/it, loss=1.45e+04, v_num=1]
```

Example 1406 (unknown):
```unknown
Epoch 449/500:  90%|████████▉ | 448/500 [51:18<06:00,  6.93s/it, loss=1.45e+04, v_num=1]
```

Example 1407 (unknown):
```unknown
Epoch 449/500:  90%|████████▉ | 449/500 [51:25<05:52,  6.91s/it, loss=1.45e+04, v_num=1]
```

Example 1408 (unknown):
```unknown
Epoch 449/500:  90%|████████▉ | 449/500 [51:25<05:52,  6.91s/it, loss=1.49e+04, v_num=1]
```

Example 1409 (unknown):
```unknown
Epoch 450/500:  90%|████████▉ | 449/500 [51:25<05:52,  6.91s/it, loss=1.49e+04, v_num=1]
```

Example 1410 (unknown):
```unknown
Epoch 450/500:  90%|█████████ | 450/500 [51:32<05:46,  6.93s/it, loss=1.49e+04, v_num=1]
```

Example 1411 (unknown):
```unknown
Epoch 450/500:  90%|█████████ | 450/500 [51:32<05:46,  6.93s/it, loss=1.46e+04, v_num=1]
```

Example 1412 (unknown):
```unknown
Epoch 451/500:  90%|█████████ | 450/500 [51:32<05:46,  6.93s/it, loss=1.46e+04, v_num=1]
```

Example 1413 (unknown):
```unknown
Epoch 451/500:  90%|█████████ | 451/500 [51:39<05:37,  6.90s/it, loss=1.46e+04, v_num=1]
```

Example 1414 (unknown):
```unknown
Epoch 451/500:  90%|█████████ | 451/500 [51:39<05:37,  6.90s/it, loss=1.49e+04, v_num=1]
```

Example 1415 (unknown):
```unknown
Epoch 452/500:  90%|█████████ | 451/500 [51:39<05:37,  6.90s/it, loss=1.49e+04, v_num=1]
```

Example 1416 (unknown):
```unknown
Epoch 452/500:  90%|█████████ | 452/500 [51:45<05:28,  6.85s/it, loss=1.49e+04, v_num=1]
```

Example 1417 (unknown):
```unknown
Epoch 452/500:  90%|█████████ | 452/500 [51:45<05:28,  6.85s/it, loss=1.48e+04, v_num=1]
```

Example 1418 (unknown):
```unknown
Epoch 453/500:  90%|█████████ | 452/500 [51:45<05:28,  6.85s/it, loss=1.48e+04, v_num=1]
```

Example 1419 (unknown):
```unknown
Epoch 453/500:  91%|█████████ | 453/500 [51:52<05:23,  6.88s/it, loss=1.48e+04, v_num=1]
```

Example 1420 (unknown):
```unknown
Epoch 453/500:  91%|█████████ | 453/500 [51:52<05:23,  6.88s/it, loss=1.45e+04, v_num=1]
```

Example 1421 (unknown):
```unknown
Epoch 454/500:  91%|█████████ | 453/500 [51:52<05:23,  6.88s/it, loss=1.45e+04, v_num=1]
```

Example 1422 (unknown):
```unknown
Epoch 454/500:  91%|█████████ | 454/500 [51:59<05:17,  6.91s/it, loss=1.45e+04, v_num=1]
```

Example 1423 (unknown):
```unknown
Epoch 454/500:  91%|█████████ | 454/500 [51:59<05:17,  6.91s/it, loss=1.46e+04, v_num=1]
```

Example 1424 (unknown):
```unknown
Epoch 455/500:  91%|█████████ | 454/500 [51:59<05:17,  6.91s/it, loss=1.46e+04, v_num=1]
```

Example 1425 (unknown):
```unknown
Epoch 455/500:  91%|█████████ | 455/500 [52:06<05:11,  6.92s/it, loss=1.46e+04, v_num=1]
```

Example 1426 (unknown):
```unknown
Epoch 455/500:  91%|█████████ | 455/500 [52:06<05:11,  6.92s/it, loss=1.45e+04, v_num=1]
```

Example 1427 (unknown):
```unknown
Epoch 456/500:  91%|█████████ | 455/500 [52:06<05:11,  6.92s/it, loss=1.45e+04, v_num=1]
```

Example 1428 (unknown):
```unknown
Epoch 456/500:  91%|█████████ | 456/500 [52:13<05:04,  6.91s/it, loss=1.45e+04, v_num=1]
```

Example 1429 (unknown):
```unknown
Epoch 456/500:  91%|█████████ | 456/500 [52:13<05:04,  6.91s/it, loss=1.46e+04, v_num=1]
```

Example 1430 (unknown):
```unknown
Epoch 457/500:  91%|█████████ | 456/500 [52:13<05:04,  6.91s/it, loss=1.46e+04, v_num=1]
```

Example 1431 (unknown):
```unknown
Epoch 457/500:  91%|█████████▏| 457/500 [52:20<04:57,  6.92s/it, loss=1.46e+04, v_num=1]
```

Example 1432 (unknown):
```unknown
Epoch 457/500:  91%|█████████▏| 457/500 [52:20<04:57,  6.92s/it, loss=1.43e+04, v_num=1]
```

Example 1433 (unknown):
```unknown
Epoch 458/500:  91%|█████████▏| 457/500 [52:20<04:57,  6.92s/it, loss=1.43e+04, v_num=1]
```

Example 1434 (unknown):
```unknown
Epoch 458/500:  92%|█████████▏| 458/500 [52:27<04:50,  6.91s/it, loss=1.43e+04, v_num=1]
```

Example 1435 (unknown):
```unknown
Epoch 458/500:  92%|█████████▏| 458/500 [52:27<04:50,  6.91s/it, loss=1.46e+04, v_num=1]
```

Example 1436 (unknown):
```unknown
Epoch 459/500:  92%|█████████▏| 458/500 [52:27<04:50,  6.91s/it, loss=1.46e+04, v_num=1]
```

Example 1437 (unknown):
```unknown
Epoch 459/500:  92%|█████████▏| 459/500 [52:34<04:40,  6.85s/it, loss=1.46e+04, v_num=1]
```

Example 1438 (unknown):
```unknown
Epoch 459/500:  92%|█████████▏| 459/500 [52:34<04:40,  6.85s/it, loss=1.47e+04, v_num=1]
```

Example 1439 (unknown):
```unknown
Epoch 460/500:  92%|█████████▏| 459/500 [52:34<04:40,  6.85s/it, loss=1.47e+04, v_num=1]
```

Example 1440 (unknown):
```unknown
Epoch 460/500:  92%|█████████▏| 460/500 [52:40<04:31,  6.79s/it, loss=1.47e+04, v_num=1]
```

Example 1441 (unknown):
```unknown
Epoch 460/500:  92%|█████████▏| 460/500 [52:40<04:31,  6.79s/it, loss=1.44e+04, v_num=1]
```

Example 1442 (unknown):
```unknown
Epoch 461/500:  92%|█████████▏| 460/500 [52:40<04:31,  6.79s/it, loss=1.44e+04, v_num=1]
```

Example 1443 (unknown):
```unknown
Epoch 461/500:  92%|█████████▏| 461/500 [52:47<04:26,  6.83s/it, loss=1.44e+04, v_num=1]
```

Example 1444 (unknown):
```unknown
Epoch 461/500:  92%|█████████▏| 461/500 [52:47<04:26,  6.83s/it, loss=1.41e+04, v_num=1]
```

Example 1445 (unknown):
```unknown
Epoch 462/500:  92%|█████████▏| 461/500 [52:47<04:26,  6.83s/it, loss=1.41e+04, v_num=1]
```

Example 1446 (unknown):
```unknown
Epoch 462/500:  92%|█████████▏| 462/500 [52:54<04:22,  6.91s/it, loss=1.41e+04, v_num=1]
```

Example 1447 (unknown):
```unknown
Epoch 462/500:  92%|█████████▏| 462/500 [52:54<04:22,  6.91s/it, loss=1.46e+04, v_num=1]
```

Example 1448 (unknown):
```unknown
Epoch 463/500:  92%|█████████▏| 462/500 [52:54<04:22,  6.91s/it, loss=1.46e+04, v_num=1]
```

Example 1449 (unknown):
```unknown
Epoch 463/500:  93%|█████████▎| 463/500 [53:01<04:16,  6.93s/it, loss=1.46e+04, v_num=1]
```

Example 1450 (unknown):
```unknown
Epoch 463/500:  93%|█████████▎| 463/500 [53:01<04:16,  6.93s/it, loss=1.47e+04, v_num=1]
```

Example 1451 (unknown):
```unknown
Epoch 464/500:  93%|█████████▎| 463/500 [53:01<04:16,  6.93s/it, loss=1.47e+04, v_num=1]
```

Example 1452 (unknown):
```unknown
Epoch 464/500:  93%|█████████▎| 464/500 [53:08<04:08,  6.92s/it, loss=1.47e+04, v_num=1]
```

Example 1453 (unknown):
```unknown
Epoch 464/500:  93%|█████████▎| 464/500 [53:08<04:08,  6.92s/it, loss=1.44e+04, v_num=1]
```

Example 1454 (unknown):
```unknown
Epoch 465/500:  93%|█████████▎| 464/500 [53:08<04:08,  6.92s/it, loss=1.44e+04, v_num=1]
```

Example 1455 (unknown):
```unknown
Epoch 465/500:  93%|█████████▎| 465/500 [53:15<04:03,  6.95s/it, loss=1.44e+04, v_num=1]
```

Example 1456 (unknown):
```unknown
Epoch 465/500:  93%|█████████▎| 465/500 [53:15<04:03,  6.95s/it, loss=1.46e+04, v_num=1]
```

Example 1457 (unknown):
```unknown
Epoch 466/500:  93%|█████████▎| 465/500 [53:15<04:03,  6.95s/it, loss=1.46e+04, v_num=1]
```

Example 1458 (unknown):
```unknown
Epoch 466/500:  93%|█████████▎| 466/500 [53:22<03:56,  6.95s/it, loss=1.46e+04, v_num=1]
```

Example 1459 (unknown):
```unknown
Epoch 466/500:  93%|█████████▎| 466/500 [53:22<03:56,  6.95s/it, loss=1.46e+04, v_num=1]
```

Example 1460 (unknown):
```unknown
Epoch 467/500:  93%|█████████▎| 466/500 [53:22<03:56,  6.95s/it, loss=1.46e+04, v_num=1]
```

Example 1461 (unknown):
```unknown
Epoch 467/500:  93%|█████████▎| 467/500 [53:29<03:48,  6.93s/it, loss=1.46e+04, v_num=1]
```

Example 1462 (unknown):
```unknown
Epoch 467/500:  93%|█████████▎| 467/500 [53:29<03:48,  6.93s/it, loss=1.45e+04, v_num=1]
```

Example 1463 (unknown):
```unknown
Epoch 468/500:  93%|█████████▎| 467/500 [53:29<03:48,  6.93s/it, loss=1.45e+04, v_num=1]
```

Example 1464 (unknown):
```unknown
Epoch 468/500:  94%|█████████▎| 468/500 [53:36<03:42,  6.94s/it, loss=1.45e+04, v_num=1]
```

Example 1465 (unknown):
```unknown
Epoch 468/500:  94%|█████████▎| 468/500 [53:36<03:42,  6.94s/it, loss=1.44e+04, v_num=1]
```

Example 1466 (unknown):
```unknown
Epoch 469/500:  94%|█████████▎| 468/500 [53:36<03:42,  6.94s/it, loss=1.44e+04, v_num=1]
```

Example 1467 (unknown):
```unknown
Epoch 469/500:  94%|█████████▍| 469/500 [53:43<03:32,  6.84s/it, loss=1.44e+04, v_num=1]
```

Example 1468 (unknown):
```unknown
Epoch 469/500:  94%|█████████▍| 469/500 [53:43<03:32,  6.84s/it, loss=1.47e+04, v_num=1]
```

Example 1469 (unknown):
```unknown
Epoch 470/500:  94%|█████████▍| 469/500 [53:43<03:32,  6.84s/it, loss=1.47e+04, v_num=1]
```

Example 1470 (unknown):
```unknown
Epoch 470/500:  94%|█████████▍| 470/500 [53:50<03:25,  6.84s/it, loss=1.47e+04, v_num=1]
```

Example 1471 (unknown):
```unknown
Epoch 470/500:  94%|█████████▍| 470/500 [53:50<03:25,  6.84s/it, loss=1.47e+04, v_num=1]
```

Example 1472 (unknown):
```unknown
Epoch 471/500:  94%|█████████▍| 470/500 [53:50<03:25,  6.84s/it, loss=1.47e+04, v_num=1]
```

Example 1473 (unknown):
```unknown
Epoch 471/500:  94%|█████████▍| 471/500 [53:56<03:17,  6.81s/it, loss=1.47e+04, v_num=1]
```

Example 1474 (unknown):
```unknown
Epoch 471/500:  94%|█████████▍| 471/500 [53:56<03:17,  6.81s/it, loss=1.47e+04, v_num=1]
```

Example 1475 (unknown):
```unknown
Epoch 472/500:  94%|█████████▍| 471/500 [53:56<03:17,  6.81s/it, loss=1.47e+04, v_num=1]
```

Example 1476 (unknown):
```unknown
Epoch 472/500:  94%|█████████▍| 472/500 [54:03<03:10,  6.79s/it, loss=1.47e+04, v_num=1]
```

Example 1477 (unknown):
```unknown
Epoch 472/500:  94%|█████████▍| 472/500 [54:03<03:10,  6.79s/it, loss=1.48e+04, v_num=1]
```

Example 1478 (unknown):
```unknown
Epoch 473/500:  94%|█████████▍| 472/500 [54:03<03:10,  6.79s/it, loss=1.48e+04, v_num=1]
```

Example 1479 (unknown):
```unknown
Epoch 473/500:  95%|█████████▍| 473/500 [54:10<03:04,  6.84s/it, loss=1.48e+04, v_num=1]
```

Example 1480 (unknown):
```unknown
Epoch 473/500:  95%|█████████▍| 473/500 [54:10<03:04,  6.84s/it, loss=1.45e+04, v_num=1]
```

Example 1481 (unknown):
```unknown
Epoch 474/500:  95%|█████████▍| 473/500 [54:10<03:04,  6.84s/it, loss=1.45e+04, v_num=1]
```

Example 1482 (unknown):
```unknown
Epoch 474/500:  95%|█████████▍| 474/500 [54:17<02:58,  6.85s/it, loss=1.45e+04, v_num=1]
```

Example 1483 (unknown):
```unknown
Epoch 474/500:  95%|█████████▍| 474/500 [54:17<02:58,  6.85s/it, loss=1.44e+04, v_num=1]
```

Example 1484 (unknown):
```unknown
Epoch 475/500:  95%|█████████▍| 474/500 [54:17<02:58,  6.85s/it, loss=1.44e+04, v_num=1]
```

Example 1485 (unknown):
```unknown
Epoch 475/500:  95%|█████████▌| 475/500 [54:24<02:51,  6.85s/it, loss=1.44e+04, v_num=1]
```

Example 1486 (unknown):
```unknown
Epoch 475/500:  95%|█████████▌| 475/500 [54:24<02:51,  6.85s/it, loss=1.46e+04, v_num=1]
```

Example 1487 (unknown):
```unknown
Epoch 476/500:  95%|█████████▌| 475/500 [54:24<02:51,  6.85s/it, loss=1.46e+04, v_num=1]
```

Example 1488 (unknown):
```unknown
Epoch 476/500:  95%|█████████▌| 476/500 [54:31<02:44,  6.87s/it, loss=1.46e+04, v_num=1]
```

Example 1489 (unknown):
```unknown
Epoch 476/500:  95%|█████████▌| 476/500 [54:31<02:44,  6.87s/it, loss=1.45e+04, v_num=1]
```

Example 1490 (unknown):
```unknown
Epoch 477/500:  95%|█████████▌| 476/500 [54:31<02:44,  6.87s/it, loss=1.45e+04, v_num=1]
```

Example 1491 (unknown):
```unknown
Epoch 477/500:  95%|█████████▌| 477/500 [54:38<02:38,  6.90s/it, loss=1.45e+04, v_num=1]
```

Example 1492 (unknown):
```unknown
Epoch 477/500:  95%|█████████▌| 477/500 [54:38<02:38,  6.90s/it, loss=1.46e+04, v_num=1]
```

Example 1493 (unknown):
```unknown
Epoch 478/500:  95%|█████████▌| 477/500 [54:38<02:38,  6.90s/it, loss=1.46e+04, v_num=1]
```

Example 1494 (unknown):
```unknown
Epoch 478/500:  96%|█████████▌| 478/500 [54:44<02:27,  6.72s/it, loss=1.46e+04, v_num=1]
```

Example 1495 (unknown):
```unknown
Epoch 478/500:  96%|█████████▌| 478/500 [54:44<02:27,  6.72s/it, loss=1.43e+04, v_num=1]
```

Example 1496 (unknown):
```unknown
Epoch 479/500:  96%|█████████▌| 478/500 [54:44<02:27,  6.72s/it, loss=1.43e+04, v_num=1]
```

Example 1497 (unknown):
```unknown
Epoch 479/500:  96%|█████████▌| 479/500 [54:50<02:20,  6.67s/it, loss=1.43e+04, v_num=1]
```

Example 1498 (unknown):
```unknown
Epoch 479/500:  96%|█████████▌| 479/500 [54:50<02:20,  6.67s/it, loss=1.45e+04, v_num=1]
```

Example 1499 (unknown):
```unknown
Epoch 480/500:  96%|█████████▌| 479/500 [54:50<02:20,  6.67s/it, loss=1.45e+04, v_num=1]
```

Example 1500 (unknown):
```unknown
Epoch 480/500:  96%|█████████▌| 480/500 [54:56<02:05,  6.26s/it, loss=1.45e+04, v_num=1]
```

Example 1501 (unknown):
```unknown
Epoch 480/500:  96%|█████████▌| 480/500 [54:56<02:05,  6.26s/it, loss=1.45e+04, v_num=1]
```

Example 1502 (unknown):
```unknown
Epoch 481/500:  96%|█████████▌| 480/500 [54:56<02:05,  6.26s/it, loss=1.45e+04, v_num=1]
```

Example 1503 (unknown):
```unknown
Epoch 481/500:  96%|█████████▌| 481/500 [55:02<01:59,  6.28s/it, loss=1.45e+04, v_num=1]
```

Example 1504 (unknown):
```unknown
Epoch 481/500:  96%|█████████▌| 481/500 [55:02<01:59,  6.28s/it, loss=1.45e+04, v_num=1]
```

Example 1505 (unknown):
```unknown
Epoch 482/500:  96%|█████████▌| 481/500 [55:02<01:59,  6.28s/it, loss=1.45e+04, v_num=1]
```

Example 1506 (unknown):
```unknown
Epoch 482/500:  96%|█████████▋| 482/500 [55:06<01:42,  5.71s/it, loss=1.45e+04, v_num=1]
```

Example 1507 (unknown):
```unknown
Epoch 482/500:  96%|█████████▋| 482/500 [55:06<01:42,  5.71s/it, loss=1.47e+04, v_num=1]
```

Example 1508 (unknown):
```unknown
Epoch 483/500:  96%|█████████▋| 482/500 [55:06<01:42,  5.71s/it, loss=1.47e+04, v_num=1]
```

Example 1509 (unknown):
```unknown
Epoch 483/500:  97%|█████████▋| 483/500 [55:11<01:28,  5.22s/it, loss=1.47e+04, v_num=1]
```

Example 1510 (unknown):
```unknown
Epoch 483/500:  97%|█████████▋| 483/500 [55:11<01:28,  5.22s/it, loss=1.43e+04, v_num=1]
```

Example 1511 (unknown):
```unknown
Epoch 484/500:  97%|█████████▋| 483/500 [55:11<01:28,  5.22s/it, loss=1.43e+04, v_num=1]
```

Example 1512 (unknown):
```unknown
Epoch 484/500:  97%|█████████▋| 484/500 [55:15<01:18,  4.89s/it, loss=1.43e+04, v_num=1]
```

Example 1513 (unknown):
```unknown
Epoch 484/500:  97%|█████████▋| 484/500 [55:15<01:18,  4.89s/it, loss=1.44e+04, v_num=1]
```

Example 1514 (unknown):
```unknown
Epoch 485/500:  97%|█████████▋| 484/500 [55:15<01:18,  4.89s/it, loss=1.44e+04, v_num=1]
```

Example 1515 (unknown):
```unknown
Epoch 485/500:  97%|█████████▋| 485/500 [55:19<01:09,  4.65s/it, loss=1.44e+04, v_num=1]
```

Example 1516 (unknown):
```unknown
Epoch 485/500:  97%|█████████▋| 485/500 [55:19<01:09,  4.65s/it, loss=1.45e+04, v_num=1]
```

Example 1517 (unknown):
```unknown
Epoch 486/500:  97%|█████████▋| 485/500 [55:19<01:09,  4.65s/it, loss=1.45e+04, v_num=1]
```

Example 1518 (unknown):
```unknown
Epoch 486/500:  97%|█████████▋| 486/500 [55:23<01:02,  4.48s/it, loss=1.45e+04, v_num=1]
```

Example 1519 (unknown):
```unknown
Epoch 486/500:  97%|█████████▋| 486/500 [55:23<01:02,  4.48s/it, loss=1.47e+04, v_num=1]
```

Example 1520 (unknown):
```unknown
Epoch 487/500:  97%|█████████▋| 486/500 [55:23<01:02,  4.48s/it, loss=1.47e+04, v_num=1]
```

Example 1521 (unknown):
```unknown
Epoch 487/500:  97%|█████████▋| 487/500 [55:27<00:56,  4.36s/it, loss=1.47e+04, v_num=1]
```

Example 1522 (unknown):
```unknown
Epoch 487/500:  97%|█████████▋| 487/500 [55:27<00:56,  4.36s/it, loss=1.44e+04, v_num=1]
```

Example 1523 (unknown):
```unknown
Epoch 488/500:  97%|█████████▋| 487/500 [55:27<00:56,  4.36s/it, loss=1.44e+04, v_num=1]
```

Example 1524 (unknown):
```unknown
Epoch 488/500:  98%|█████████▊| 488/500 [55:31<00:51,  4.28s/it, loss=1.44e+04, v_num=1]
```

Example 1525 (unknown):
```unknown
Epoch 488/500:  98%|█████████▊| 488/500 [55:31<00:51,  4.28s/it, loss=1.47e+04, v_num=1]
```

Example 1526 (unknown):
```unknown
Epoch 489/500:  98%|█████████▊| 488/500 [55:31<00:51,  4.28s/it, loss=1.47e+04, v_num=1]
```

Example 1527 (unknown):
```unknown
Epoch 489/500:  98%|█████████▊| 489/500 [55:35<00:46,  4.22s/it, loss=1.47e+04, v_num=1]
```

Example 1528 (unknown):
```unknown
Epoch 489/500:  98%|█████████▊| 489/500 [55:35<00:46,  4.22s/it, loss=1.44e+04, v_num=1]
```

Example 1529 (unknown):
```unknown
Epoch 490/500:  98%|█████████▊| 489/500 [55:35<00:46,  4.22s/it, loss=1.44e+04, v_num=1]
```

Example 1530 (unknown):
```unknown
Epoch 490/500:  98%|█████████▊| 490/500 [55:39<00:42,  4.21s/it, loss=1.44e+04, v_num=1]
```

Example 1531 (unknown):
```unknown
Epoch 490/500:  98%|█████████▊| 490/500 [55:39<00:42,  4.21s/it, loss=1.45e+04, v_num=1]
```

Example 1532 (unknown):
```unknown
Epoch 491/500:  98%|█████████▊| 490/500 [55:39<00:42,  4.21s/it, loss=1.45e+04, v_num=1]
```

Example 1533 (unknown):
```unknown
Epoch 491/500:  98%|█████████▊| 491/500 [55:43<00:37,  4.17s/it, loss=1.45e+04, v_num=1]
```

Example 1534 (unknown):
```unknown
Epoch 491/500:  98%|█████████▊| 491/500 [55:43<00:37,  4.17s/it, loss=1.45e+04, v_num=1]
```

Example 1535 (unknown):
```unknown
Epoch 491/500:  98%|█████████▊| 491/500 [55:43<01:01,  6.81s/it, loss=1.45e+04, v_num=1]
```

Example 1536 (unknown):
```unknown
Monitored metric reconstruction_loss_validation did not improve in the last 50 records. Best score: 31435.986. Signaling Trainer to stop.
```

Example 1537 (unknown):
```unknown
# save model
if developing==False:
    mvi.save(f'{model_dir}/mvi_{combination}',overwrite=True)
```

Example 1538 (python):
```python
# Extract the latent variables and normalized expression matrices
latent_variables = mvi.get_latent_representation()

# Create new anndata objects with the extracted matrices
latent_variables_adata = anndata.AnnData(X=latent_variables)

# display
print(latent_variables_adata)

# save
if developing==False:
    latent_variables_adata.write(os.path.join(latent_variables_dir, f"mvi_{combination}_latent_variables.h5ad"))
```

Example 1539 (unknown):
```unknown
AnnData object with n_obs × n_vars = 5664 × 20
```

Example 1540 (python):
```python
%%time
# add the latent space to adata with original data
adata_mvi.obsm["X_multiVI"] = mvi.get_latent_representation()
```

Example 1541 (unknown):
```unknown
CPU times: user 1.13 s, sys: 1.44 s, total: 2.57 s
Wall time: 2.58 s
```

Example 1542 (python):
```python
%%time
sc.pp.neighbors(adata_mvi,use_rep="X_multiVI")
```

Example 1543 (unknown):
```unknown
computing neighbors
```

Example 1544 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/umap/distances.py:1063: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/umap/distances.py:1071: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/umap/distances.py:1086: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
```

Example 1545 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/umap/umap_.py:660: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
```

Example 1546 (unknown):
```unknown
finished: added to `.uns['neighbors']`
    `.obsp['distances']`, distances for each pair of neighbors
    `.obsp['connectivities']`, weighted adjacency matrix (0:00:06)
```

Example 1547 (unknown):
```unknown
CPU times: user 5.33 s, sys: 363 ms, total: 5.7 s
Wall time: 6.96 s
```

Example 1548 (python):
```python
%%time
sc.tl.umap(adata_mvi, min_dist=0.1)
```

Example 1549 (unknown):
```unknown
computing UMAP
```

Example 1550 (python):
```python
finished: added
    'X_umap', UMAP coordinates (adata.obsm) (0:00:10)
```

Example 1551 (unknown):
```unknown
CPU times: user 9.89 s, sys: 5.12 ms, total: 9.9 s
Wall time: 10.3 s
```

Example 1552 (python):
```python
if developing==False:
    sc.pl.umap(adata_mvi,title=combination, color=['donor','cell_or_nuclei','kit_10x','scrublet_score','donor', 'week', 'sex', 'region','modality'],
           wspace=0.8,
           ncols=4,
           save=f"_{combination}.png")
else:
    sc.pl.umap(adata_mvi,title=combination, color=['donor','cell_or_nuclei','kit_10x','scrublet_score','donor', 'week', 'sex', 'region','modality'],
           wspace=0.8,
           ncols=4)
```

Example 1553 (unknown):
```unknown
WARNING: The title list is shorter than the number of panels. Using 'color' value instead for some plots.
```

Example 1554 (unknown):
```unknown
WARNING: The title list is shorter than the number of panels. Using 'color' value instead for some plots.
```

Example 1555 (unknown):
```unknown
WARNING: The title list is shorter than the number of panels. Using 'color' value instead for some plots.
```

Example 1556 (unknown):
```unknown
WARNING: The title list is shorter than the number of panels. Using 'color' value instead for some plots.
```

Example 1557 (unknown):
```unknown
WARNING: The title list is shorter than the number of panels. Using 'color' value instead for some plots.
```

Example 1558 (unknown):
```unknown
WARNING: The title list is shorter than the number of panels. Using 'color' value instead for some plots.
```

Example 1559 (unknown):
```unknown
WARNING: The title list is shorter than the number of panels. Using 'color' value instead for some plots.
```

Example 1560 (unknown):
```unknown
WARNING: The title list is shorter than the number of panels. Using 'color' value instead for some plots.
```

Example 1561 (unknown):
```unknown
WARNING: saving figure to file figures/umap_multivi_aCMs_WholeSample_n_layers1.png
```

Example 1562 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

---

## Import modules¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/RNA/scnRNA_Integration-Annotation/Leukocytes/No2_Immune_post-scVI.html

**Contents:**
- Import modules¶
- Function¶
- Read in data¶
- Log-normalise¶
- Clustering, UMAP¶
  - Deciding resolution¶
- Marker gene expressions¶
- DEG analysis¶
- Annotation¶
- Recover row counts and save¶
- Celltypist, pan-foetal immune¶

gene expression: normalise + logtransform

**Examples:**

Example 1 (python):
```python
import warnings
warnings.simplefilter(action='ignore', category=FutureWarning)

import numpy as np
import numpy.random as random
import pandas as pd
import scanpy as sc
import matplotlib.pyplot as plt
import anndata


import celltypist
from celltypist import models

import sctk
import os
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
-----
anndata             0.8.0
celltypist          0.1.9
matplotlib          3.5.2
numpy               1.21.5
pandas              1.3.5
scanpy              1.8.2
sctk                NA
session_info        1.0.0
-----
```

Example 4 (python):
```python
PIL                 9.1.1
asciitree           NA
backcall            0.2.0
beta_ufunc          NA
binom_ufunc         NA
brotli              NA
cached_property     1.5.1
cairo               1.20.0
certifi             2020.12.05
cffi                1.14.4
chardet             3.0.4
charset_normalizer  2.0.10
cloudpickle         1.6.0
colorama            0.4.4
cycler              0.10.0
cython_runtime      NA
cytoolz             0.11.2
dask                2022.01.0
dateutil            2.8.1
decorator           4.4.2
defusedxml          0.6.0
fasteners           0.17.3
fsspec              2022.01.0
google              NA
h5py                3.1.0
idna                2.10
igraph              0.10.2
importlib_metadata  1.7.0
ipykernel           5.4.2
ipython_genutils    0.2.0
ipywidgets          7.6.3
jedi                0.17.2
jinja2              3.1.2
joblib              1.0.0
kiwisolver          1.3.1
leidenalg           0.9.0
llvmlite            0.35.0
louvain             0.7.0
markupsafe          2.1.1
mpl_toolkits        NA
msgpack             1.0.3
natsort             7.1.0
nbinom_ufunc        NA
numba               0.52.0
numcodecs           0.9.1
numexpr             2.7.2
numpy_groupies      0.9.13
packaging           21.3
parso               0.7.1
pexpect             4.8.0
pickleshare         0.7.5
pkg_resources       NA
plotly              5.10.0
prompt_toolkit      3.0.10
psutil              5.9.0
ptyprocess          0.7.0
pyarrow             9.0.0
pygments            2.13.0
pyparsing           2.4.7
pytz                2020.5
requests            2.27.1
scipy               1.7.3
scvelo              0.2.4
seaborn             0.11.1
setuptools_scm      NA
sinfo               0.3.1
six                 1.14.0
sklearn             1.0.2
socks               1.7.1
sphinxcontrib       NA
statsmodels         0.12.1
storemagic          NA
tables              3.6.1
tblib               1.7.0
texttable           1.6.3
threadpoolctl       2.1.0
tlz                 0.11.2
toolz               0.11.2
tornado             6.1
traitlets           5.0.5
typing_extensions   NA
urllib3             1.26.9
wcwidth             0.2.5
yaml                5.4.1
zarr                2.11.3
zipp                NA
zmq                 21.0.1
zope                NA
```

Example 5 (unknown):
```unknown
-----
IPython             7.19.0
jupyter_client      6.1.11
jupyter_core        4.7.0
notebook            6.2.0
-----
Python 3.7.9 | packaged by conda-forge | (default, Dec  9 2020, 21:08:20) [GCC 9.3.0]
Linux-4.15.0-112-generic-x86_64-with-debian-bullseye-sid
-----
Session information updated at 2023-02-17 14:35
```

Example 6 (python):
```python
sc.settings.set_figure_params(dpi=120, facecolor='white', color_map = 'RdPu',)
```

Example 7 (python):
```python
#>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
# extract rank_genes_groups result 
#>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

import scanpy as sc
import pandas as pd

def rank_genes_groups_to_df(adata, cell):
    
    dic = adata.uns['rank_genes_groups'] 
    
    df = pd.DataFrame({
        'names' : dic['names'][cell],
        'scores' : dic['scores'][cell],
        'pvals' : dic['pvals'][cell],
        'pvals_adj' : dic['pvals_adj'][cell],
        'logfoldchanges' : dic['logfoldchanges'][cell]
    })
    
    return df
```

Example 8 (python):
```python
adata = sc.read_h5ad('/home/jovyan/mount/gdrive/Colab/data/scVI/subsets/RNA_foetal_raw_immune_post-scVI_16Feb2023.h5ad')
adata
```

Example 9 (unknown):
```unknown
AnnData object with n_obs × n_vars = 41881 × 31326
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', '_scvi_labels', '2Jan_coarse_grain', '2Jan_mid_grain', '2Jan_fine_grain', 'leiden_scVI', 'FACSgate', 'sex', 'week', 'trimester'
    var: 'gene_name_scRNA-0', 'gene_id', 'n_cells'
    uns: '2Jan_fine_grain_colors', '2Jan_mid_grain_colors', 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'age_colors', 'cell_or_nuclei_colors', 'donor_colors', 'leiden_scVI_colors', 'modality_colors', 'sex_colors', 'trimester_colors', 'week_colors'
    obsm: 'X_scVI', 'X_umap', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 10 (python):
```python
# normalise and logtransform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)
```

Example 11 (python):
```python
%%time
sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata, min_dist=0.2, spread=1.0, n_components=3)
sc.tl.leiden(adata, resolution=2, key_added='leiden_scVI')
```

Example 12 (unknown):
```unknown
CPU times: user 1min 45s, sys: 44.8 s, total: 2min 30s
Wall time: 1min 17s
```

Example 13 (python):
```python
sc.pl.umap(adata, color=['leiden_scVI'], size=5, wspace=0.1, legend_loc='on data', legend_fontsize=10,components='all')
```

Example 14 (python):
```python
sc.pl.umap(adata, color=['region','donor','cell_or_nuclei','kit_10x'], size=5, wspace=0.7)
```

Example 15 (python):
```python
pd.crosstab(adata.obs['cell_or_nuclei'],adata.obs['donor']).T
```

Example 16 (python):
```python
sc.pl.umap(adata, color=['2Jan_fine_grain','week'], size=5, wspace=0.1, legend_loc='on data', legend_fontsize=5)
```

Example 17 (python):
```python
set(adata.obs['2Jan_fine_grain'])
```

Example 18 (unknown):
```unknown
{'B1Cells',
 'BCells',
 'BProCells',
 'Cardiofibromyocytes',
 'CardiofibromyocytesSecretory',
 'CoronaryCapillaryEndothelialCells',
 'CoronaryPericytes',
 'CoronaryVesselAdventitialFibroblasts',
 'CoronaryVesselSmoothMuscleCells',
 'DC1',
 'DC2',
 'DCsMature',
 'DCsPlasmocytoids',
 'EarlyGreatVesselFLRT2',
 'EarlyGreatVesselHOXB4',
 'EndocardialCells',
 'GVAdFibOCA2',
 'GVAdFibOther',
 'GreatVesselArterialEndothelialCells',
 'GreatVesselSmoothMuscleCells',
 'InnateLymphoidCells',
 'LymphaticEndothelialCells',
 'MacrophagesCX3CR1pos',
 'MacrophagesInflammasome',
 'MacrophagesLYVE1pos',
 'MacrophagesTIMD4pos',
 'MacrophagesTREM2pos',
 'MastCells',
 'Monocytes',
 'MonocytesMPOpos',
 'MyocIntFibROBO2',
 'MyocIntFibSLIT3',
 'Myofibroblasts',
 'NaturalKillerCells',
 'SchwannCells',
 'TCellsCD4pos',
 'TCellsCD8pos',
 'Tregs',
 'ValveOther',
 'ValvePENK',
 nan}
```

Example 19 (python):
```python
sc.pl.umap(adata, color=['2Jan_fine_grain'], groups='DCsPlasmocytoids', size=5, wspace=0.1, legend_loc='on data', legend_fontsize=5)
```

Example 20 (python):
```python
sc.pl.umap(adata, color=['scrublet_score','cluster_scrublet_score','doublet_pval','doublet_bh_pval','pct_counts_mt','pct_counts_ribo'], size=2, wspace=0.2, ncols=6)
```

Example 21 (python):
```python
'''
# set symbol as var_names
adata.var.set_index('gene_name_multiome-1',inplace=True)
adata.var_names = adata.var_names.astype('str')
adata.var_names_make_unique()
'''
```

Example 22 (python):
```python
# Checking glabal markers
markers = {
    'vCM': ['MYH7', 'MYL2', 'FHL2'],
    'aCM': ['NPPA', 'MYL7', 'MYL4'],
    'FB': ['DCN', 'GCN', 'PDGFRA'],
    'EC': ['VWF', 'PECAM1', 'CDH5'],
    'Peri': ['RGS5', 'ABCC9', 'KCNJ8'],
    'SMC': ['MYH11', 'TAGLN', 'ACTA2'],
    'Meso': ['MSLN', 'WT1', 'BNC1'],
    'Neuro': ['PLP1', 'NRXN1', 'NRXN3'],
    'Adipo': ['GPAM', 'FASN', 'LEP'],
    'Myelo': ['CD14', 'C1QA', 'CD68','LYVE1','TIMD4'],
    'Lymph': ['CD8A', 'IL7R', 'CD40LG'],
    'B': ['CD19','SDC1','IGHG1','IGHA1'],
    'Mast': ['KIT', 'CPA3'],
}

for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in adata.var_names]

sc.tl.dendrogram(adata, groupby='leiden_scVI')
sc.pl.dotplot(adata,
              markers, 
              groupby='leiden_scVI',
              dendrogram=True,
              standard_scale="var",
              color_map="Reds",
              # swap_axes=True,
             )
```

Example 23 (python):
```python
WARNING: You’re trying to run this on 31326 dimensions of `.X`, if you really want this, set `use_rep='X'`.
         Falling back to preprocessing with `sc.pp.pca` and default params.
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: vCM, aCM, FB, etc.
```

Example 24 (python):
```python
# lymphocytes markers

markers={
    'B cell': ['CD19', 'CD79A', 'CD79B', 'IGHM', 'IGHG1', 'IGHA1'],
    'T cell': ['CD3E', 'CD3D', 'TRAC', 'CD4', 'CD8A', 'CD8B'],
    'naive': ['CCR7', 'TCF7', 'LEF1', 'SELL'],
    'act': ['IL2','CD69'],
    'CD4T_reg': ['FOXP3', 'CTLA4', 'IL2RA', 'IKZF2'],
    'cytotoxic': ['PRF1', 'GZMB', 'GZMH', 'NKG7'],
    'trans': ['GZMK', 'TOX'],
    'te': ['TIGIT', 'LAG3', 'PDCD1'],
    'em': ['IL7R', 'RUNX2'],
    'MAIT': ['SLC4A10', 'KLRB1', 'CCR6'],
    'NK': ['KLRF1', 'KLRD1', 'NCAM1', 'FCGR3A'],
    'ILC': ['KIT', 'IL18R1'],
    'cycling': ['MKI67', 'TOP2A', 'UBE2C'],
    'TFs': ['TBX21', 'GATA3', 'RORC', 'BCL6',],
}


for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in adata.var_names]

sc.pl.dotplot(adata,
              markers, 
              groupby='leiden_scVI',
              dendrogram=True,
              standard_scale="var",
              color_map="Reds",
              # swap_axes=True,
             )
```

Example 25 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: B cell, T cell, naive, etc.
```

Example 26 (python):
```python
markers={
     'T cell': ['CD3E', 'CD3D','CD4', 'CD8A', 'CD8B','TRAC','TRBC1','TRDC','TRGC1'],
    'naive': ['TCF7', 'LEF1', 'SELL'],
    'cytotoxic': ['PRF1', 'GZMB', 'GZMH', 'NKG7'],
    'NK': ['KLRF1', 'KLRD1', 'NCAM1','FCGR3A','IFNG'],
    'ILC3': ['KIT', 'IL18R1','RORC','IL17A'],
    
    'Mono':['CCR2','CD14','FCGR3A','CX3CR1','FCGR1A','FCN1','TREM2'],
    'MP':['C1QA','C1QB','CD68','CD163','MERTK','LYVE1','IGF1','TIMD4','DOCK4'],
    'DC':['FLT3','FCER1A','CD1C','HLA-DPA1','HLA-DPB1','CCR7'],
    'DC1':['ID2','IRF8','BATF3','CLEC9A'],
    'DC2':['CLEC10A','CD1C','FCER1A',],
    'Neutro':['MYB', 'MPO', 'CEACAM8','CXCR1','CXCR2'],
    
    'B': ['CD19','CD20','CD22','CD79A', 'CD79B','MS4A1','IL7R',],
    '':['CCR7','TNFRSF13B',],
    'ProB':['KIT','SPN','CD34'],
    'Plasma': ['CD27','CCR10','CD38','SDC1','JCAIN'],
    'Abs':['IGHM','IGHD', 'IGHG1','IGHG2', 'IGHG3','IGHG4', 'IGHA1','IGHE','IGLC1','IGKC'],
    'Gene editing': ['AICDA', 'RAG1','RAG2'],
    
    
    'Cycling':['MKI67','TOP2A'],

    'TFs': ['TBX21', 'GATA3', 'RORC', 'BCL6',],
    'Mast': ['KIT', 'CPA3'],
    'Megakaryocyte':['ITGA2B','MPL','ITGB3', 'TUBB1', 'PPBP'],
    'pDC':['IRF8','IRF4','IL3RA']
}


for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in adata.var_names]

sc.pl.dotplot(adata,
              markers, 
              groupby='leiden_scVI',
              dendrogram=True,
              standard_scale="var",
              color_map="Reds",
              # swap_axes=True,
             )
```

Example 27 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: T cell, naive, cytotoxic, etc.
```

Example 28 (python):
```python
sc.settings.set_figure_params(dpi=120, facecolor='white', color_map = 'RdPu',)
sc.pl.umap(adata, color=['CCR2','CD14','FCGR3A','CX3CR1','FCGR1A','FCN1','TREM2',
                'C1QA','C1QB','CD68','CD163','MERTK','LYVE1','IGF1','TIMD4','FOLR2','DOCK4',
                'FLT3','FCER1A','CD1C','HLA-DPA1','HLA-DPB1',
                'CD3D','CD3E','CD4','CD8A','CD8B','CD19','CD19','CD22','CD79A', 'CD79B','MKI67','TOP2A',], size=20, wspace=0.2, ncols=6, components='all')
```

Example 29 (python):
```python
adata.X.data[:10]
```

Example 30 (unknown):
```unknown
array([0.20843568, 0.38083044, 0.20843568, 0.20843568, 0.20843568,
       0.38083044, 0.38083044, 0.76952875, 0.38083044, 0.38083044],
      dtype=float32)
```

Example 31 (python):
```python
sc.tl.rank_genes_groups(adata, 'leiden_scVI', method='t-test')

DEG_dict = {}
for cluster in adata.obs['leiden_scVI'].cat.categories:
    df=rank_genes_groups_to_df(adata, cluster)
    # select genes: significant and up-regulated ones
    df=df[(df['pvals_adj']<0.05)&(df['logfoldchanges']>0)]
    # sort based on the logFC
    df.sort_values('scores',ascending=False,inplace=True)
    DEG_dict[cluster]=list(df[:5]['names'])
    del df

sc.pl.dotplot(adata,
              DEG_dict, 
              groupby='leiden_scVI',
              dendrogram=True,
              standard_scale="var",
              color_map="Reds",
              swap_axes=False,
             )
```

Example 32 (python):
```python
# marker of cluster 44
# migratory DC?
DEG_dict = {}
for cluster in adata.obs['leiden_scVI'].cat.categories:
    df=rank_genes_groups_to_df(adata, cluster)
    # select genes: significant and up-regulated ones
    df=df[(df['pvals_adj']<0.05)&(df['logfoldchanges']>0)]
    # sort based on the logFC
    df.sort_values('scores',ascending=False,inplace=True)
    DEG_dict[cluster]=list(df[:50]['names'])
    del df

sc.pl.dotplot(adata,
              DEG_dict['42']+['FLT3','ID2','IRF4','IRF8','CCR7','CIITA','PTPRC'], 
              groupby='leiden_scVI',
              dendrogram=True,
              standard_scale="var",
              color_map="Reds",
              swap_axes=False,
             )
```

Example 33 (python):
```python
sc.pl.umap(adata, color=['leiden_scVI'], size=3, wspace=0.1, legend_loc='on data', legend_fontsize=5, legend_fontweight='normal')
# components='all',
```

Example 34 (python):
```python
sc.pl.umap(adata, color=['leiden_scVI'], groups=['32'], size=5, wspace=0.1, legend_loc='on data', legend_fontsize=7)
```

Example 35 (python):
```python
adata.obs['cell_type2']=adata.obs['leiden_scVI'].astype('str').copy()
adata.obs.replace({'cell_type2':{
    '0':'Lymphoid', 
    '1':'Myeloid', 
    '2':'Myeloid', 
    '3':'Lymphoid', 
    '4':'Lymphoid', 
    '5':'Myeloid',  
    '6':'Lymphoid', 
    '7':'Lymphoid',  
    '8':'Lymphoid', 
    '9':'Lymphoid',  
    '10':'Myeloid',  
    '11':'Myeloid',  
    '12':'Lymphoid', 
    '13':'Myeloid', 
    '14':'Lymphoid', 
    '15':'Myeloid', 
    '16':'Myeloid', 
    '17':'Lymphoid', 
    '18':'Myeloid', 
    '19':'Myeloid', 
    '20':'Myeloid', 
    '21':'Lymphoid', 
     '22':'Lymphoid', 
     '23':'Mast cell', 
     '24':'Lymphoid', 
     '25':'Lymphoid', 
     '26':'Myeloid', 
    '27':'Myeloid', 
    '28':'Lymphoid', 
    '29':'Myeloid', 
    '30':'Lymphoid', 
    '31':'Myeloid', 
    '32':'Myeloid', 
    '33':'Lymphoid', 
    '34':'Lymphoid', 
    '35':'Plasmacytoid DC',  
    '36':'Myeloid', 
    '37':'Myeloid', 
    '38':'Lymphoid', 
    '39':'Lymphoid', 
    '40':'Megakaryocyte', 
    '41':'Lymphoid', 
    '42':'DC_mature', 
    '43':'Plasmacytoid DC', 
}}, inplace=True)


adata.obs['cell_type2']=adata.obs['cell_type2'].astype('category')
adata.obs['cell_type2'].cat.reorder_categories([
   'Lymphoid', 
    'Myeloid',
    'Plasmacytoid DC',
    # 'DC1',
    'DC_mature',
    'Megakaryocyte',
     'Mast cell',
    # 'Doublet',
], inplace=True)

adata.obs['cell_type2'].value_counts()
```

Example 36 (unknown):
```unknown
Lymphoid           21336
Myeloid            18527
Mast cell            866
Plasmacytoid DC      608
Megakaryocyte        360
DC_mature            184
Name: cell_type2, dtype: int64
```

Example 37 (python):
```python
markers={
     'T cell': ['CD3E', 'CD3D','CD4', 'CD8A', 'CD8B','TRAC','TRBC1','TRDC','TRGC1'],
    'naive': ['TCF7', 'LEF1', 'SELL'],
    'cytotoxic': ['PRF1', 'GZMB', 'GZMH', 'NKG7'],
    'NK': ['KLRF1', 'KLRD1', 'NCAM1','FCGR3A','IFNG'],
    'ILC3': ['KIT', 'IL18R1','RORC','IL17A'],
    
    'Mono':['CCR2','CD14','FCGR3A','CX3CR1','FCGR1A','FCN1','TREM2'],
    'MP':['C1QA','C1QB','CD68','CD163','MERTK','LYVE1','IGF1','TIMD4','CX3CR1','DOCK4','COL1A1','COL1A2','COL3A1'],
    'DC':['FLT3','FCER1A','CD1C','HLA-DPA1','HLA-DPB1','CCR7'],
    'DC1':['ID2','IRF8','BATF3','CLEC9A'],
    'DC2':['CLEC10A','CD1C','FCER1A',],
    'DC_mature':['FLT3', 'CCR7','LAMP3', 'IL15', 'CD83', 'LY75', ],
    'Neutro':['MYB', 'MPO', 'CEACAM8','CXCR1','CXCR2'],
    
    'B': ['CD19','CD20','CD22','CD79A', 'CD79B','MS4A1','IL7R',],
    '':['CCR7','TNFRSF13B',],
    'ProB':['KIT','SPN','CD34'],
    'Plasma': ['CD27','CCR10','CD38','SDC1','JCAIN'],
    'Abs':['IGHM','IGHD', 'IGHG1','IGHG2', 'IGHG3','IGHG4', 'IGHA1','IGHE','IGLC1','IGKC'],
    'Gene editing': ['AICDA', 'RAG1','RAG2'],
    
     'pDC':['IRF8','IRF4','IL3RA'],
    
    

    'TFs': ['TBX21', 'GATA3', 'RORC', 'BCL6',],
    'Mast': ['KIT', 'CPA3','HDC','GATA2'],
    'Megakaryocyte':['ITGA2B','MPL','ITGB3', 'TUBB1', 'PPBP'],
    'Cycling':['MKI67','TOP2A'],
}


for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in adata.var_names]

sc.pl.dotplot(adata,
              markers, 
              groupby='cell_type2',
              dendrogram=False,
              standard_scale="var",
              color_map="Reds",
              # swap_axes=True,
             )
```

Example 38 (python):
```python
markers={
     'T cell': ['CD3E', 'CD3D','TRAC','TRBC1',],
     'B': ['CD19','CD20','CD22','CD79A', 'CD79B','MS4A1','IL7R',],
   
    
    'Mono':['CCR2','CD14','FCGR3A','CX3CR1','FCGR1A','FCN1','TREM2'],
    'MP':['C1QA','C1QB','CD68','CD163','MERTK','LYVE1','IGF1','TIMD4','CX3CR1','DOCK4','COL1A1','COL1A2','COL3A1'],
    'DC1':['ID2','IRF8','BATF3','CLEC9A'],
    'DC2':['CLEC10A','CD1C','FCER1A',],
     'DC_mature':['FLT3', 'CCR7','LAMP3', 'IL15', 'CD83', 'LY75', ],
    'pDC':['IRF8','IRF4','IL3RA'],
    
   
    'Mast': ['KIT', 'CPA3','HDC','GATA2'],
    'Megakaryocyte':['ITGA2B','MPL','ITGB3',],
    'mesenchymalMP':['COL1A1','COL1A2','COL3A1','C1QA','C1QC','C1QD','VSIG4','COL3A1', 'MDK', 'SPARC', 'ELN'],
    'vCM': ['MYH7', 'MYL2', 'FHL2'],
    'aCM': ['NPPA', 'MYL7', 'MYL4'],
}


for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in adata.var_names]

sc.pl.dotplot(adata,
              markers, 
              groupby='cell_type2',
              dendrogram=False,
              standard_scale="var",
              color_map="Reds",
              # swap_axes=True,
             )
```

Example 39 (python):
```python
sc.settings.set_figure_params(dpi=120,  color_map = 'RdPu',)
sc.pl.umap(adata, color=['cell_type2'], size=5, legend_fontsize=10, frameon=False, components='all', wspace=0.5)
```

Example 40 (python):
```python
sc.pl.umap(adata, color=['cell_type2'], groups='Myeloid', size=5, legend_fontsize=10, frameon=False, components='all', wspace=0.5)
```

Example 41 (python):
```python
sc.pl.umap(adata, color=['cell_type2'], groups='Lymphoid', size=5, legend_fontsize=10, frameon=False, components='all', wspace=0.5)
```

Example 42 (python):
```python
sc.tl.rank_genes_groups(adata, 'cell_type2', method='t-test')

DEG_dict = {}
for cluster in adata.obs['cell_type2'].cat.categories:
    df=rank_genes_groups_to_df(adata, cluster)
    # select genes: significant and up-regulated ones
    df=df[(df['pvals_adj']<0.05)&(df['logfoldchanges']>0)]
    # sort based on the logFC
    df.sort_values('scores',ascending=False,inplace=True)
    DEG_dict[cluster]=list(df[:10]['names'])
    del df

sc.pl.dotplot(adata,
              DEG_dict, 
              groupby='cell_type2',
              dendrogram=False,
              standard_scale="var",
              color_map="Reds",
              swap_axes=False,
             )
```

Example 43 (python):
```python
adata_raw=sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/14Feb_fromSemih/14FebImmuneForKazumasa.h5ad')
print(adata_raw.shape)

adata_raw=anndata.AnnData(X=adata_raw[adata.obs_names,:].X,
                      obs=adata.obs, # from annotated or subsetted data
                      var=adata_raw.var,
                      obsm=adata.obsm, # from annotated or subsetted data
                      obsp=adata.obsp # from annotated or subsetted data
                               )
print(adata_raw.shape)
```

Example 44 (unknown):
```unknown
(49042, 36601)
(41881, 36601)
```

Example 45 (python):
```python
adata_raw.write('/nfs/team205/heart/anndata_objects/Foetal/Immune/RNA_foetal_raw_immune_16Feb2023.h5ad')
adata_raw
```

Example 46 (unknown):
```unknown
AnnData object with n_obs × n_vars = 41881 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', '_scvi_labels', '2Jan_coarse_grain', '2Jan_mid_grain', '2Jan_fine_grain', 'leiden_scVI', 'FACSgate', 'sex', 'week', 'trimester', 'cell_type2'
    var: 'gene_name_scRNA-0', 'gene_id'
    obsm: 'X_scVI', 'X_umap', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding', 'X_pca'
    obsp: 'connectivities', 'distances'
```

Example 47 (python):
```python
# save to gdrive
# update RNA_foetal_raw_immune_16Feb2023.h5ad
adata_raw.write('/home/jovyan/mount/gdrive/Colab/data/anndata/RNA_foetal_raw_immune_16Feb2023.h5ad')
```

Example 48 (python):
```python
sc.pl.umap(adata_raw, color=['cell_type2'], size=5, legend_fontsize=10, frameon=False, components='all', wspace=0.5)
```

Example 49 (python):
```python
adata_raw.obs['cell_type2'].value_counts()
```

Example 50 (unknown):
```unknown
# load model
panfetal_model = models.Model.load('/nfs/team205/kk18/Analysis/celltypist/models/celltypist_model.Pan_Fetal_Human.pkl')
```

Example 51 (python):
```python
prediction=celltypist.annotate(adata, model = panfetal_model, majority_voting = False)
```

Example 52 (unknown):
```unknown
celltypist.dotplot(prediction, use_as_reference = 'cell_type2', use_as_prediction = 'predicted_labels')
```

Example 53 (python):
```python
adata_pre = sc.read_h5ad('/home/jovyan/mount/gdrive/Colab/data/anndata/RNA_foetal_raw_immune_4Feb2023.h5ad')
adata_pre
```

Example 54 (unknown):
```unknown
AnnData object with n_obs × n_vars = 42624 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', '_scvi_labels', '2Jan_coarse_grain', '2Jan_mid_grain', '2Jan_fine_grain', 'leiden_scVI', 'FACSgate', 'immune', 'age_pcw', 'week_pcw', 'cell_type2'
    var: 'gene_name_scRNA-0', 'gene_name_multiome-1'
    uns: 'cell_type2_colors'
    obsm: 'X_pca', 'X_scVI', 'X_umap', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 55 (python):
```python
pd.crosstab(adata_pre.obs['cell_type2'],adata_pre.obs['week_pcw'])
```

Example 56 (python):
```python
pd.crosstab(adata_raw.obs['cell_type2'],adata_raw.obs['week'])
```

---

## Download¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/RNA/QC/Demultiplex/No2_CMO_download_results.html

**Contents:**
- Download¶
- Read in path file¶
- Download from iRODs¶
- Read in Multiome data¶
- CMO result¶
- Save¶
- Proportions¶

**Examples:**

Example 1 (python):
```python
import warnings
warnings.simplefilter(action='ignore', category=FutureWarning)

import numpy as np
import pandas as pd
import scanpy as sc
import session_info
import matplotlib.pyplot as plt

import os
```

Example 2 (unknown):
```unknown
session_info.show()
```

Example 3 (python):
```python
-----
matplotlib          3.5.2
numpy               1.21.5
pandas              1.3.5
scanpy              1.8.2
session_info        1.0.0
-----
```

Example 4 (python):
```python
PIL                 9.1.1
anndata             0.8.0
asciitree           NA
backcall            0.2.0
beta_ufunc          NA
binom_ufunc         NA
cached_property     1.5.1
cairo               1.20.0
cffi                1.14.4
cloudpickle         1.6.0
colorama            0.4.4
cycler              0.10.0
cython_runtime      NA
cytoolz             0.11.2
dask                2022.01.0
dateutil            2.8.1
decorator           4.4.2
defusedxml          0.6.0
fasteners           0.17.3
fsspec              2022.01.0
google              NA
h5py                3.1.0
igraph              0.10.2
importlib_metadata  1.7.0
ipykernel           5.4.2
ipython_genutils    0.2.0
jedi                0.17.2
jinja2              3.1.2
joblib              1.0.0
kiwisolver          1.3.1
leidenalg           0.9.0
llvmlite            0.35.0
louvain             0.7.0
markupsafe          2.1.1
mpl_toolkits        NA
msgpack             1.0.3
natsort             7.1.0
nbinom_ufunc        NA
numba               0.52.0
numcodecs           0.9.1
numexpr             2.7.2
packaging           21.3
parso               0.7.1
pexpect             4.8.0
pickleshare         0.7.5
pkg_resources       NA
plotly              5.10.0
prompt_toolkit      3.0.10
psutil              5.9.0
ptyprocess          0.7.0
pyarrow             9.0.0
pygments            2.13.0
pyparsing           2.4.7
pytz                2020.5
scipy               1.7.3
setuptools_scm      NA
sinfo               0.3.1
six                 1.14.0
sklearn             1.0.2
sphinxcontrib       NA
storemagic          NA
tables              3.6.1
tblib               1.7.0
texttable           1.6.3
threadpoolctl       2.1.0
tlz                 0.11.2
toolz               0.11.2
tornado             6.1
traitlets           5.0.5
typing_extensions   NA
wcwidth             0.2.5
yaml                5.4.1
zarr                2.11.3
zipp                NA
zmq                 21.0.1
zope                NA
```

Example 5 (unknown):
```unknown
-----
IPython             7.19.0
jupyter_client      6.1.11
jupyter_core        4.7.0
notebook            6.2.0
-----
Python 3.7.9 | packaged by conda-forge | (default, Dec  9 2020, 21:08:20) [GCC 9.3.0]
Linux-4.15.0-112-generic-x86_64-with-debian-bullseye-sid
-----
Session information updated at 2023-02-07 23:34
```

Example 6 (python):
```python
samplefile = pd.read_csv('/nfs/team205/heart/mapped/CMO/csv/multiome-foetal_20221221.csv')
samplefile.set_index('CMO_ID',inplace=True)
samplefile
```

Example 7 (unknown):
```unknown
cd /nfs/team205/heart/mapped/CMO
sed 1d ./csv/multiome-foetal_20221221.csv | while IFS=, read -r mID cID IRODSPATH
do
echo $cID

# create output folder if it does not exist
if [[ ! -d "$cID" ]]; then
    mkdir -p "$cID"
fi

iget -K -r $IRODSPATH/raw_feature_bc_matrix $cID
iget -K $IRODSPATH/web_summary.html ./websummary/multiome-foetal_20221221/${cID}_web_summary.html
done
```

Example 8 (unknown):
```unknown
cID=7089STDY13216919
IRODSPATH=/seq/illumina/runs/46/46301/cellranger/cellranger700_count_46301_7089STDY13216919_GRCh38-2020-A
iget -K -r $IRODSPATH/raw_feature_bc_matrix $cID
iget -K $IRODSPATH/web_summary.html ./websummary/multiome-foetal_20221221/${cID}_web_summary.html
```

Example 9 (python):
```python
# post-cellbender
adata = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/QC/multiome_RNA_foetal_prefilter.h5ad',
                     backed = 'r')
# only multiplexed data
adata = adata[adata.obs['combinedID'].isin(list(samplefile['Multiome_ID']))]

adata
```

Example 10 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 74984 × 36601 backed at '/nfs/team205/heart/anndata_objects/Foetal/QC/multiome_RNA_foetal_prefilter.h5ad'
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval'
    var: 'gene_name', 'feature_type', 'ambient_expression-0', 'ambient_expression-1', 'ambient_expression-10', 'ambient_expression-11', 'ambient_expression-12', 'ambient_expression-13', 'ambient_expression-14', 'ambient_expression-15', 'ambient_expression-16', 'ambient_expression-17', 'ambient_expression-18', 'ambient_expression-19', 'ambient_expression-2', 'ambient_expression-20', 'ambient_expression-21', 'ambient_expression-22', 'ambient_expression-23', 'ambient_expression-24', 'ambient_expression-25', 'ambient_expression-26', 'ambient_expression-27', 'ambient_expression-28', 'ambient_expression-29', 'ambient_expression-3', 'ambient_expression-30', 'ambient_expression-31', 'ambient_expression-32', 'ambient_expression-33', 'ambient_expression-34', 'ambient_expression-35', 'ambient_expression-36', 'ambient_expression-37', 'ambient_expression-4', 'ambient_expression-5', 'ambient_expression-6', 'ambient_expression-7', 'ambient_expression-8', 'ambient_expression-9'
    obsm: 'latent_gene_encoding'
```

Example 11 (python):
```python
adata.obs['combinedID'].value_counts()
```

Example 12 (unknown):
```unknown
7089STDY13216920_BHF_F_Hea13242527    27498
7089STDY13216921_BHF_F_Hea13242528    24211
7089STDY13216922_BHF_F_Hea13242529    21402
7089STDY13216923_BHF_F_Hea13242530     1873
Name: combinedID, dtype: int64
```

Example 13 (unknown):
```unknown
* donor_assignment: Determining by selecting which of its donors (tags) has the maximum count per cell
* ratio: Ratio of "the 1st maximum tag count" to "the 2nd maximum tag count" per cell.
```

Example 14 (python):
```python
# function to calculate proportion of the maximum tag count to the total tag count

def ratio(row):
    # convert to array
    arr = np.array(row)
    # sort -> reverse to make it descending
    arr.sort()
    arr = arr[::-1]
    # get 1st and 2nd max value
    first = arr[0]
    second = arr[1]
    # return ratio
    return first/second
```

Example 15 (python):
```python
for i,cmo_ID in enumerate(samplefile.index):
    # read in
    cmo = sc.read_10x_mtx(f'/nfs/team205/heart/mapped/CMO/{cmo_ID}/raw_feature_bc_matrix',
                     gex_only=False)
    
    # get multiome ID
    multi_ID = samplefile.loc[cmo_ID,'Multiome_ID']

    # modify obs_names
    cmo.obs_names = multi_ID + '_' + cmo.obs_names
    # add sample IDs
    cmo.obs['combinedID'] = multi_ID
    
    # select cells in post-cellbender adata
    obsnames = adata.obs.index[adata.obs['combinedID']==multi_ID]
    cmo = cmo[obsnames]
    # get obs
    cmo_obs = cmo.obs.copy()
    
    # prepare count dataframe
    X_df = cmo.to_df()
    X_df.columns = [x.split('-')[0] for x in X_df.columns]
    
    # calculate delta
    cmo_obs['ratio'] = X_df.apply(ratio,axis=1)
    cmo_obs['top_count'] = X_df.max(axis=1)
    
    # assign donor
    def which_is_max(row):
        # convert to array
        arr = np.array(row)
        # get position
        pos = np.where(arr == np.max(arr))[0]
        # get donor
        don = ','.join(X_df.columns[pos])
        return don
    def n_max_donor(row):
        # convert to array
        arr = np.array(row)
        # get position
        pos = np.where(arr == np.max(arr))[0]
        return len(pos)
    cmo_obs['donor_assignment'] = X_df.apply(which_is_max,axis=1)
    cmo_obs['n_donor_assigned'] = X_df.apply(n_max_donor,axis=1)
    
    # add count values
    cmo_obs[X_df.columns] = X_df.copy()
    
    # concatenate
    if i==0:
        cmo_obs_all = cmo_obs.copy()
    else:
        cmo_obs_all = pd.concat([cmo_obs_all,cmo_obs])
        
    del cmo, cmo_obs, multi_ID, obsnames, X_df

print(cmo_obs_all.shape)
cmo_obs_all.head()
```

Example 16 (unknown):
```unknown
# check samples which is 1st max == 2d max
cmo_obs_all[cmo_obs_all['n_donor_assigned']==2]
```

Example 17 (unknown):
```unknown
cmo_obs_all.to_csv('/nfs/team205/heart/mapped/CMO/result_summary/CMO-result_multiome-foetal_20230207.csv')
```

Example 18 (python):
```python
df = cmo_obs_all.copy()
plt.rcParams['figure.figsize'] = 3,3
tmp = pd.crosstab(df['combinedID'], df['donor_assignment'], normalize='index')
# tmp.plot.bar().legend(loc=1)
ax = tmp.plot.bar(stacked=True,width=0.7)
ax.grid(False)
ax.set_ylabel('proportion')
ax.legend(bbox_to_anchor=(1.0, 1.0))
ax.plot()
plt.show()

for sam in list(samplefile['Multiome_ID']):
    df_s = df[df['combinedID']==sam]
    print(f'##### {sam} #####')
    print(df_s['donor_assignment'].value_counts())
    print('')
```

Example 19 (unknown):
```unknown
##### 7089STDY13216920_BHF_F_Hea13242527 #####
Hst40          18933
Hst39           8557
Hst40,Hst39        8
Name: donor_assignment, dtype: int64

##### 7089STDY13216921_BHF_F_Hea13242528 #####
Hst40          15866
Hst39           8335
Hst40,Hst39       10
Name: donor_assignment, dtype: int64

##### 7089STDY13216922_BHF_F_Hea13242529 #####
Hst40          13755
Hst36           5466
Hst39           2178
Hst40,Hst36        2
Hst40,Hst39        1
Name: donor_assignment, dtype: int64

##### 7089STDY13216923_BHF_F_Hea13242530 #####
Hst40    1435
Hst36     394
Hst39      44
Name: donor_assignment, dtype: int64
```

---

## Loading packages¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/Visium/cell2location/wholeheart_without_DA/no1_reference-signatures_woPericardial_woDASMCs.html

**Contents:**
- Loading packages¶
- Data, parameters¶
- Loading scRNA-seq reference data¶
- Remove cells¶
- Add Atria Ventricle key
- Estimation of reference cell type signatures (NB regression)¶

**Examples:**

Example 1 (python):
```python
import sys
import scanpy as sc
import anndata
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt 
import matplotlib as mpl

import cell2location
import scvi

from matplotlib import rcParams
rcParams['pdf.fonttype'] = 42 # enables correct plotting of text
import seaborn as sns
```

Example 2 (unknown):
```unknown
Global seed set to 0
```

Example 3 (python):
```python
import session_info
session_info.show()
```

Example 4 (python):
```python
-----
anndata             0.7.8
cell2location       NA
matplotlib          3.5.0
numpy               1.20.3
pandas              1.3.4
scanpy              1.9.3
scvi                0.14.4
seaborn             0.11.2
session_info        1.0.0
-----
```

Example 5 (python):
```python
PIL                         8.4.0
absl                        NA
adjustText                  NA
arviz                       0.10.0
attr                        21.2.0
backcall                    0.2.0
beta_ufunc                  NA
binom_ufunc                 NA
cffi                        1.15.0
cftime                      1.5.1.1
colorama                    0.4.4
cycler                      0.10.0
cython_runtime              NA
dateutil                    2.8.2
debugpy                     1.5.1
decorator                   5.1.0
defusedxml                  0.7.1
deprecate                   0.3.0
descartes                   NA
dill                        0.3.4
docrep                      0.3.2
entrypoints                 0.3
fastprogress                0.2.7
fontTools                   4.28.1
fsspec                      2021.11.0
google                      NA
h5py                        3.6.0
igraph                      0.9.8
ipykernel                   6.5.1
ipython_genutils            0.2.0
ipywidgets                  7.6.5
jedi                        0.18.1
joblib                      1.1.0
kiwisolver                  1.3.2
lazylinker_ext              NA
leidenalg                   0.8.8
llvmlite                    0.37.0
matplotlib_inline           NA
mizani                      0.7.3
mpl_toolkits                NA
natsort                     8.0.0
nbinom_ufunc                NA
netCDF4                     1.5.8
numba                       0.54.1
numexpr                     2.7.3
opt_einsum                  v3.3.0
packaging                   21.3
palettable                  3.3.0
parso                       0.8.2
patsy                       0.5.2
pexpect                     4.8.0
pickleshare                 0.7.5
pkg_resources               NA
plotnine                    0.8.0
prompt_toolkit              3.0.22
ptyprocess                  0.7.0
pycparser                   2.21
pydev_ipython               NA
pydevconsole                NA
pydevd                      2.6.0
pydevd_concurrency_analyser NA
pydevd_file_utils           NA
pydevd_plugins              NA
pydevd_tracing              NA
pyexpat                     NA
pygments                    2.10.0
pymc3                       3.10.0
pynndescent                 0.5.5
pyparsing                   3.0.6
pyro                        1.7.0
pytorch_lightning           1.3.8
pytz                        2021.3
rich                        NA
scipy                       1.7.2
setuptools                  58.0.4
setuptools_scm              NA
six                         1.16.0
sklearn                     1.0.1
statsmodels                 0.13.1
storemagic                  NA
tensorboard                 2.7.0
texttable                   1.6.4
theano                      1.0.11
threadpoolctl               3.0.0
torch                       1.10.0+cu102
torchmetrics                0.6.0
tornado                     6.1
tqdm                        4.62.3
traitlets                   5.1.1
typing_extensions           NA
umap                        0.5.2
wcwidth                     0.2.5
xarray                      0.20.1
yaml                        5.4.1
zmq                         22.3.0
```

Example 6 (unknown):
```unknown
-----
IPython             7.29.0
jupyter_client      7.0.6
jupyter_core        4.9.1
notebook            6.4.6
-----
Python 3.9.7 (default, Sep 16 2021, 13:09:58) [GCC 7.5.0]
Linux-4.15.0-200-generic-x86_64-with-glibc2.27
-----
Session information updated at 2023-03-14 11:24
```

Example 7 (unknown):
```unknown
results_folder = '/nfs/team205/heart/cell2location/foetal'

# create paths and names to results folders for reference regression and cell2location models
ref_run_name = f'{results_folder}/reference_signatures/ver28Feb2023'
run_name = f'{results_folder}/cell2location_map/ver28Feb2023'
```

Example 8 (unknown):
```unknown
scRNA_path='/nfs/team205/heart/anndata_objects/Foetal/Feb28ObjectRaw.h5ad'
celltype_key='fine_grain'
```

Example 9 (python):
```python
# Read scRNA data
adata = sc.read(scRNA_path)
adata
```

Example 10 (unknown):
```unknown
AnnData object with n_obs × n_vars = 297473 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score'
    var: 'gene_name_scRNA-0', 'gene_id'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'age_colors', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'cycling_colors', 'donor_colors', 'fine_grain_colors', 'heart_or_greatvessels_colors', 'hvg', 'kit_10x_colors', 'mid_grain_colors', 'modality_colors', 'neighbors', 'phase_colors', 'region_colors', 'sex_colors', 'trimester_colors', 'umap', 'week_colors'
    obsm: 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 11 (python):
```python
adata.X.data[:10]
```

Example 12 (unknown):
```unknown
array([ 1.,  1.,  5.,  5.,  8.,  4., 13.,  4.,  3.,  1.], dtype=float32)
```

Example 13 (python):
```python
set(adata.obs[celltype_key])
```

Example 14 (unknown):
```unknown
{'AtrialCardiomyocytesCycling',
 'AtrialCardiomyocytesLeft',
 'AtrialCardiomyocytesRight',
 'AtrioventricularNodeCardiomyocytes',
 'BCells',
 'BCellsMS4A1pos',
 'ChromaffinCells',
 'CoronaryArterialEndothelialCells',
 'CoronaryCapillaryEndothelialCells',
 'CoronaryPericytes',
 'CoronarySmoothMuscleCells',
 'CoronaryVenousEndothelialCells',
 'CoronaryVesselAdventitialFibroblasts',
 'DendriticCellsMature',
 'DendriticCellsType1',
 'DuctusArteriosusSmoothMuscleCells',
 'EndocardialCells',
 'EndocardialCushionCells',
 'EpicardiumDerivedCells',
 'GreatVesselAdventitialFibroblasts',
 'GreatVesselArterialEndothelialCells',
 'GreatVesselSmoothMuscleCells',
 'GreatVesselVenousEndothelialCells',
 'InnateLymphoidCells',
 'LymphNodeFibroblasticReticularCells',
 'LymphaticEndothelialCells',
 'MacrophagesATF3pos',
 'MacrophagesCX3CR1pos',
 'MacrophagesLYVE1pos',
 'MacrophagesTIMD4pos',
 'MastCells',
 'Megakaryocytes',
 'MesothelialEpicardialCells',
 'MonocyteDerivedCells',
 'Monocytes',
 'MonocytesMPOpos',
 'MyocardialInterstitialFibroblasts1',
 'MyocardialInterstitialFibroblasts2',
 'Myofibroblasts',
 'NaturalKillerCells',
 'NeuronPrecursors',
 'ParaympatheticNeurons',
 'PericardialCells1',
 'PericardialCells2',
 'PericardialCells3',
 'PlasmacytoidDendriticCells',
 'ProBCells',
 'SchwannCellPrecursors',
 'SchwannCells',
 'SinoatrialNodeCardiomyocytes',
 'SympatheticNeurons',
 'TCellsCD4pos',
 'TCellsCD8pos',
 'TregsCD4pos',
 'ValveEndothelialCells',
 'ValveInterstitialCells',
 'VentricularCardiomyocytesCycling',
 'VentricularCardiomyocytesLeftCompact',
 'VentricularCardiomyocytesLeftTrabeculated',
 'VentricularCardiomyocytesRightCompact',
 'VentricularCardiomyocytesRightTrabeculated',
 'VentricularConductionSystemDistal',
 'VentricularConductionSystemProximal'}
```

Example 15 (python):
```python
# Make sure .var_names are ENSEMBL IDs
adata.var.head()
```

Example 16 (python):
```python
adata.obs['batch_key'].value_counts()
```

Example 17 (unknown):
```unknown
Hst41_Nuclei_Multiome-v1    104878
Hst33_Nuclei_Multiome-v1     38234
Hst39_Nuclei_Multiome-v1     25386
C99_Cell_5prime-v2           17547
C92_Nuclei_Multiome-v1       13770
C94_Cell_5prime-v2           12985
Hst40_Nuclei_Multiome-v1      9708
C97_Cell_5prime-v2            9702
BRC2251_Cell_3prime-v3        9121
BRC2256_Cell_3prime-v3        7948
Hst36_Nuclei_Multiome-v1      7268
BRC2252_Cell_3prime-v3        6854
C86_Cell_5prime-v2            6376
BRC2260_Cell_3prime-v3        6004
BRC2263_Cell_3prime-v3        5662
BRC2262_Cell_3prime-v3        4129
C87_Nuclei_Multiome-v1        3907
C98_Nuclei_Multiome-v1        2660
C104_Nuclei_Multiome-v1       2490
C85_Nuclei_Multiome-v1        2436
C82_Nuclei_Multiome-v1         408
Name: batch_key, dtype: int64
```

Example 18 (python):
```python
set(adata.obs['batch_key'])
```

Example 19 (unknown):
```unknown
{'BRC2251_Cell_3prime-v3',
 'BRC2252_Cell_3prime-v3',
 'BRC2256_Cell_3prime-v3',
 'BRC2260_Cell_3prime-v3',
 'BRC2262_Cell_3prime-v3',
 'BRC2263_Cell_3prime-v3',
 'C104_Nuclei_Multiome-v1',
 'C82_Nuclei_Multiome-v1',
 'C85_Nuclei_Multiome-v1',
 'C86_Cell_5prime-v2',
 'C87_Nuclei_Multiome-v1',
 'C92_Nuclei_Multiome-v1',
 'C94_Cell_5prime-v2',
 'C97_Cell_5prime-v2',
 'C98_Nuclei_Multiome-v1',
 'C99_Cell_5prime-v2',
 'Hst33_Nuclei_Multiome-v1',
 'Hst36_Nuclei_Multiome-v1',
 'Hst39_Nuclei_Multiome-v1',
 'Hst40_Nuclei_Multiome-v1',
 'Hst41_Nuclei_Multiome-v1'}
```

Example 20 (python):
```python
# remove pericardial cells
adata = adata[adata.obs['fine_grain'].isin(['PericardialCells1','PericardialCells2','PericardialCells3'])==False]

# remove ductus arterial SMCs
adata = adata[adata.obs['fine_grain'].isin(['DuctusArteriosusSmoothMuscleCells'])==False]

# remove cycling cells
adata = adata[adata.obs['cycling']=='no']

adata.shape
```

Example 21 (unknown):
```unknown
(185953, 36601)
```

Example 22 (python):
```python
'''
atrial_or_ventricular = {
    # 'AtrialCardiomyocytesCycling':'atrial',
 'AtrialCardiomyocytesLeft':'atrial',
 'AtrialCardiomyocytesRight':'atrial',
 'AtrioventricularNodeCardiomyocytes':'either',
 'BCells':'either',
 'BCellsMS4A1pos':'either',
 'ChromaffinCells':'either',
 'CoronaryArterialEndothelialCells':'either',
 'CoronaryCapillaryEndothelialCells':'either',
 'CoronaryPericytes':'either',
 'CoronarySmoothMuscleCells':'either',
 'CoronaryVenousEndothelialCells':'either',
 'CoronaryVesselAdventitialFibroblasts':'either',
 'DendriticCellsMature':'either',
 'DendriticCellsType1':'either',
 # 'DuctusArteriosusSmoothMuscleCells':'atrial',
 'EndocardialCells':'either',
 'EndocardialCushionCells':'either',
 'EpicardiumDerivedCells':'either',
 'GreatVesselAdventitialFibroblasts':'atrial',
 'GreatVesselArterialEndothelialCells':'atrial',
 'GreatVesselSmoothMuscleCells':'atrial',
 'GreatVesselVenousEndothelialCells':'atrial',
 'InnateLymphoidCells':'either',
 'LymphNodeFibroblasticReticularCells':'either',
 'LymphaticEndothelialCells':'either',
 'MacrophagesATF3pos':'either',
 'MacrophagesCX3CR1pos':'either',
 'MacrophagesLYVE1pos':'either',
 'MacrophagesTIMD4pos':'either',
 'MastCells':'either',
 'Megakaryocytes':'either',
 'MesothelialEpicardialCells':'either',
 'MonocyteDerivedCells':'either',
 'Monocytes':'either',
 'MonocytesMPOpos':'either',
 'MyocardialInterstitialFibroblasts1':'either',
 'MyocardialInterstitialFibroblasts2':'either',
 'Myofibroblasts':'either',
 'NaturalKillerCells':'either',
 'NeuronPrecursors':'either',
 'ParaympatheticNeurons':'either',
 # 'PericardialCells1':'either',
 # 'PericardialCells2':'either',
 # 'PericardialCells3':'either',
 'PlasmacytoidDendriticCells':'either',
 'ProBCells':'either',
 'SchwannCellPrecursors':'either',
 'SchwannCells':'either',
 'SinoatrialNodeCardiomyocytes':'atrial',
 'SympatheticNeurons':'either',
 'TCellsCD4pos':'either',
 'TCellsCD8pos':'either',
 'TregsCD4pos':'either',
 'ValveEndothelialCells':'either',
 'ValveInterstitialCells':'either',
 # 'VentricularCardiomyocytesCycling':'ventricular',
 'VentricularCardiomyocytesLeftCompact':'ventricular',
 'VentricularCardiomyocytesLeftTrabeculated':'ventricular',
 'VentricularCardiomyocytesRightCompact':'ventricular',
 'VentricularCardiomyocytesRightTrabeculated':'ventricular',
 'VentricularConductionSystemDistal':'ventricular',
 'VentricularConductionSystemProximal':'ventricular'}

if set(adata.obs['fine_grain']) == set(atrial_or_ventricular.keys()):
    adata.obs['atrialorventricular']=adata.obs['fine_grain'].astype('str').copy()
    adata.obs.replace({'atrialorventricular':atrial_or_ventricular},inplace=True)
else:
    print('ERROR: the cell type names are not matching')

pd.set_option("max_rows", 100)
pd.crosstab(adata.obs['fine_grain'],adata.obs['atrialorventricular'])
'''
```

Example 23 (unknown):
```unknown
regions = ['whole_woPeriCell_woDASMC'] #,'atrial','ventricular'
```

Example 24 (python):
```python
%%time

from cell2location.utils.filtering import filter_genes
from cell2location.models import RegressionModel

for reg in regions:
    print(f'##### {reg} #####')
    
    # subset adata
    if reg == 'whole_woPeriCell_woDASMC':
        adata_ref = adata.copy()
    else:
        adata_ref = adata[adata.obs['atrialorventricular'].isin(['either',reg])]
    
    # show cell types
    adata_ref.obs['fine_grain'].value_counts()
    
    # before we estimate the reference cell type signature we recommend to perform very permissive genes selection
    # in this 2D histogram orange rectangle lays over excluded genes.
    # In this case, the downloaded dataset was already filtered using this method,
    # hence no density under the orange rectangle
    selected = filter_genes(adata_ref, cell_count_cutoff=5, cell_percentage_cutoff2=0.03, nonz_mean_cutoff=1.12)
    # filter the object
    adata_ref = adata_ref[:, selected].copy()

    # remove cell_states, which have only 2 cells or less
    counts = adata_ref.obs[celltype_key].value_counts()
    adata_ref = adata_ref[adata_ref.obs[celltype_key].isin(list(counts.keys()[counts>2]))]

    # prepare anndata for the regression model
    adata_ref=adata_ref.copy()
    scvi.data.setup_anndata(adata=adata_ref, 
                            # 10X reaction / sample / batch
                            batch_key='batch_key', 
                            # cell type, covariate used for constructing signatures
                            labels_key=celltype_key, 
                            # multiplicative technical effects (platform, 3' vs 5', donor effect)
                            # categorical_covariate_keys=['Method']
                           )
    scvi.data.view_anndata_setup(adata_ref)

    # create and train the regression model
    mod = RegressionModel(adata_ref) 

    # Use all data for training (validation not implemented yet, train_size=1)
    mod.train(max_epochs=1000, batch_size=2500, train_size=1, lr=0.002, use_gpu=True)

    # plot ELBO loss history during training, removing first 20 epochs from the plot
    mod.plot_history(20)

    # In this section, we export the estimated cell abundance (summary of the posterior distribution).
    adata_ref = mod.export_posterior(
        adata_ref, sample_kwargs={'num_samples': 1000, 'batch_size': 2500, 'use_gpu': True}
    )

    # Save model
    mod.save(f"{ref_run_name}/{reg}/", overwrite=True)

    # Save anndata object with results
    # adata_file = f"{ref_run_name}/{reg}/sc.h5ad"
    adata_ref.write(f"{ref_run_name}/{reg}/sc.h5ad")

    mod.plot_QC()

    # export estimated expression in each cluster
    if 'means_per_cluster_mu_fg' in adata_ref.varm.keys():
        inf_aver = adata_ref.varm['means_per_cluster_mu_fg'][[f'means_per_cluster_mu_fg_{i}' 
                                        for i in adata_ref.uns['mod']['factor_names']]].copy()
    else:
        inf_aver = adata_ref.var[[f'means_per_cluster_mu_fg_{i}' 
                                        for i in adata_ref.uns['mod']['factor_names']]].copy()
    inf_aver.columns = adata_ref.uns['mod']['factor_names']
    inf_aver.iloc[0:5, 0:5]

    # save inf_aver
    inf_aver.to_csv(f"{ref_run_name}/{reg}/inf_aver.csv")
    
    del adata_ref,mod,inf_aver,selected,counts
```

Example 25 (unknown):
```unknown
##### whole_woPeriCell_woDASMC #####
```

Example 26 (python):
```python
INFO     Using batches from adata.obs["batch_key"]                                           
INFO     Using labels from adata.obs["fine_grain"]                                           
INFO     Using data from adata.X
```

Example 27 (unknown):
```unknown
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function setup_anndata is deprecated; Please use the model-specific setup_anndata methods instead. The global method will be removed in version 0.15.0.
```

Example 28 (python):
```python
INFO     Successfully registered anndata object containing 185953 cells, 17286 vars, 21      
         batches, 57 labels, and 0 proteins. Also registered 0 extra categorical covariates  
         and 0 extra continuous covariates.                                                  
INFO     Please do not further modify adata until model is trained.
```

Example 29 (unknown):
```unknown
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function view_anndata_setup is deprecated; This method will be removed in 0.15.0. Please avoid building any new dependencies on it.
```

Example 30 (unknown):
```unknown
Anndata setup with scvi-tools version 0.14.4.
```

Example 31 (unknown):
```unknown
Data Summary               
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃             Data             ┃ Count  ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│            Cells             │ 185953 │
│             Vars             │ 17286  │
│            Labels            │   57   │
│           Batches            │   21   │
│           Proteins           │   0    │
│ Extra Categorical Covariates │   0    │
│ Extra Continuous Covariates  │   0    │
└──────────────────────────────┴────────┘
```

Example 32 (python):
```python
SCVI Data Registry              
┏━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃     Data      ┃    scvi-tools Location    ┃
┡━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│       X       │          adata.X          │
│ batch_indices │ adata.obs['_scvi_batch']  │
│    labels     │ adata.obs['_scvi_labels'] │
└───────────────┴───────────────────────────┘
```

Example 33 (python):
```python
Label Categories                                       
┏━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┓
┃     Source Location     ┃                 Categories                 ┃ scvi-tools Encodin ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['fine_grain'] │          AtrialCardiomyocytesLeft          │         0          │
│                         │         AtrialCardiomyocytesRight          │         1          │
│                         │    VentricularCardiomyocytesLeftCompact    │         2          │
│                         │   VentricularCardiomyocytesRightCompact    │         3          │
│                         │ VentricularCardiomyocytesLeftTrabeculated  │         4          │
│                         │ VentricularCardiomyocytesRightTrabeculated │         5          │
│                         │        SinoatrialNodeCardiomyocytes        │         6          │
│                         │     AtrioventricularNodeCardiomyocytes     │         7          │
│                         │    VentricularConductionSystemProximal     │         8          │
│                         │     VentricularConductionSystemDistal      │         9          │
│                         │     GreatVesselAdventitialFibroblasts      │         10         │
│                         │    CoronaryVesselAdventitialFibroblasts    │         11         │
│                         │     MyocardialInterstitialFibroblasts1     │         12         │
│                         │     MyocardialInterstitialFibroblasts2     │         13         │
│                         │               Myofibroblasts               │         14         │
│                         │    LymphNodeFibroblasticReticularCells     │         15         │
│                         │           ValveInterstitialCells           │         16         │
│                         │        GreatVesselSmoothMuscleCells        │         17         │
│                         │         CoronarySmoothMuscleCells          │         18         │
│                         │             CoronaryPericytes              │         19         │
│                         │    GreatVesselArterialEndothelialCells     │         20         │
│                         │     GreatVesselVenousEndothelialCells      │         21         │
│                         │      CoronaryArterialEndothelialCells      │         22         │
│                         │       CoronaryVenousEndothelialCells       │         23         │
│                         │     CoronaryCapillaryEndothelialCells      │         24         │
│                         │              EndocardialCells              │         25         │
│                         │          EndocardialCushionCells           │         26         │
│                         │           ValveEndothelialCells            │         27         │
│                         │         LymphaticEndothelialCells          │         28         │
│                         │         MesothelialEpicardialCells         │         29         │
│                         │           EpicardiumDerivedCells           │         30         │
│                         │              NeuronPrecursors              │         31         │
│                         │              ChromaffinCells               │         32         │
│                         │             SympatheticNeurons             │         33         │
│                         │           ParaympatheticNeurons            │         34         │
│                         │           SchwannCellPrecursors            │         35         │
│                         │                SchwannCells                │         36         │
│                         │              MonocytesMPOpos               │         37         │
│                         │                 Monocytes                  │         38         │
│                         │            MonocyteDerivedCells            │         39         │
│                         │            MacrophagesCX3CR1pos            │         40         │
│                         │            MacrophagesTIMD4pos             │         41         │
│                         │            MacrophagesLYVE1pos             │         42         │
│                         │             MacrophagesATF3pos             │         43         │
│                         │            DendriticCellsType1             │         44         │
│                         │            DendriticCellsMature            │         45         │
│                         │         PlasmacytoidDendriticCells         │         46         │
│                         │                 MastCells                  │         47         │
│                         │               Megakaryocytes               │         48         │
│                         │                TCellsCD4pos                │         49         │
│                         │                TCellsCD8pos                │         50         │
│                         │                TregsCD4pos                 │         51         │
│                         │                 ProBCells                  │         52         │
│                         │                   BCells                   │         53         │
│                         │               BCellsMS4A1pos               │         54         │
│                         │             NaturalKillerCells             │         55         │
│                         │            InnateLymphoidCells             │         56         │
└─────────────────────────┴────────────────────────────────────────────┴────────────────────┘
```

Example 34 (python):
```python
Batch Categories                              
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location     ┃        Categories        ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['batch_key'] │  BRC2251_Cell_3prime-v3  │          0          │
│                        │  BRC2252_Cell_3prime-v3  │          1          │
│                        │  BRC2256_Cell_3prime-v3  │          2          │
│                        │  BRC2260_Cell_3prime-v3  │          3          │
│                        │  BRC2262_Cell_3prime-v3  │          4          │
│                        │  BRC2263_Cell_3prime-v3  │          5          │
│                        │  C82_Nuclei_Multiome-v1  │          6          │
│                        │  C85_Nuclei_Multiome-v1  │          7          │
│                        │    C86_Cell_5prime-v2    │          8          │
│                        │  C87_Nuclei_Multiome-v1  │          9          │
│                        │  C92_Nuclei_Multiome-v1  │         10          │
│                        │    C94_Cell_5prime-v2    │         11          │
│                        │    C97_Cell_5prime-v2    │         12          │
│                        │  C98_Nuclei_Multiome-v1  │         13          │
│                        │    C99_Cell_5prime-v2    │         14          │
│                        │ C104_Nuclei_Multiome-v1  │         15          │
│                        │ Hst33_Nuclei_Multiome-v1 │         16          │
│                        │ Hst36_Nuclei_Multiome-v1 │         17          │
│                        │ Hst39_Nuclei_Multiome-v1 │         18          │
│                        │ Hst40_Nuclei_Multiome-v1 │         19          │
│                        │ Hst41_Nuclei_Multiome-v1 │         20          │
└────────────────────────┴──────────────────────────┴─────────────────────┘
```

Example 35 (unknown):
```unknown
GPU available: True, used: True
TPU available: False, using: 0 TPU cores
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/pytorch_lightning/trainer/configuration_validator.py:99: UserWarning: you passed in a val_dataloader but have no validation_step. Skipping val loop
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
```

Example 36 (unknown):
```unknown
Epoch 1000/1000: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1000/1000 [6:02:33<00:00, 21.75s/it, v_num=1, elbo_train=1.19e+9]
Sampling local variables, batch:   0%|                                                                                                                                                                             | 0/75 [00:00<?, ?it/s]
Sampling global variables, sample: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 999/999 [00:16<00:00, 59.54it/s]
```

Example 37 (unknown):
```unknown
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'atrialorventricular' as categorical
```

Example 38 (unknown):
```unknown
CPU times: user 5h 33min 38s, sys: 2h 59min 2s, total: 8h 32min 41s
Wall time: 6h 5min 40s
```

---

## Read in adata¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/Xenium/no2_integration_and_cell-type-annotation/no7_collect_fine-grained-annotations.html

**Contents:**
- Read in adata¶
- Add fine-grained annotation¶
- Save¶
- Output file for XeniumExplorer¶
- Plotting¶
- Cropping¶

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os
import scanpy.external as sce
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/session_info/main.py:213: UserWarning: The '__version__' attribute is deprecated and will be removed in MarkupSafe 3.1. Use feature detection, or `importlib.metadata.version("markupsafe")`, instead.
  mod_version = _find_version(mod.__version__)
```

Example 4 (python):
```python
-----
matplotlib          3.10.1
numpy               2.1.3
pandas              2.2.3
scanpy              1.11.1
seaborn             0.13.2
session_info        v1.0.1
-----
```

Example 5 (python):
```python
Cython              3.1.1
PIL                 11.1.0
anndata             0.11.4
asciitree           NA
asttokens           NA
charset_normalizer  3.4.1
cloudpickle         3.1.1
comm                0.2.2
cycler              0.12.1
cython              3.1.1
cython_runtime      NA
dask                2024.11.2
dateutil            2.9.0.post0
debugpy             1.8.13
decorator           5.2.1
exceptiongroup      1.2.2
executing           2.2.0
h5py                3.13.0
igraph              0.11.8
importlib_metadata  NA
ipykernel           6.29.5
jedi                0.19.2
jinja2              3.1.6
joblib              1.4.2
kiwisolver          1.4.8
legacy_api_wrap     NA
leidenalg           0.10.2
llvmlite            0.44.0
markupsafe          3.0.2
mpl_toolkits        NA
msgpack             1.1.0
natsort             8.4.0
numba               0.61.0
numcodecs           0.13.1
packaging           24.2
parso               0.8.4
patsy               1.0.1
platformdirs        4.3.7
prompt_toolkit      3.0.50
psutil              7.0.0
pure_eval           0.2.3
pyarrow             19.0.1
pydev_ipython       NA
pydevconsole        NA
pydevd              3.2.3
pydevd_file_utils   NA
pydevd_plugins      NA
pydevd_tracing      NA
pygments            2.19.1
pyparsing           3.2.3
pytz                2025.2
scipy               1.15.2
six                 1.17.0
sklearn             1.5.2
stack_data          0.6.3
statsmodels         0.14.4
tblib               3.1.0
texttable           1.7.0
threadpoolctl       3.6.0
tlz                 1.0.0
toolz               1.0.0
tornado             6.4.2
traitlets           5.14.3
typing_extensions   NA
wcwidth             0.2.13
yaml                6.0.2
zarr                2.18.3
zipp                NA
zmq                 26.3.0
zoneinfo            NA
```

Example 6 (unknown):
```unknown
-----
IPython             8.34.0
jupyter_client      8.6.3
jupyter_core        5.7.2
-----
Python 3.10.14 | packaged by conda-forge | (main, Mar 20 2024, 12:45:18) [GCC 12.3.0]
Linux-4.18.0-553.53.1.el8_10.x86_64-x86_64-with-glibc2.28
-----
Session information updated at 2025-06-12 20:59
```

Example 7 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 8 (python):
```python
objects_dir = '/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium'
path_adata = f'{objects_dir}/C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat_5K_filtered_raw.h5ad'
adata = sc.read_h5ad(path_adata)
adata
```

Example 9 (unknown):
```unknown
AnnData object with n_obs × n_vars = 362277 × 5001
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_3.0', 'multi_celltypes_coarse'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_mid_fin_colors', 'dendrogram_pca_harmony_leiden_1.0', 'multi_celltypes_coarse_colors', 'pca', 'pca_harmony', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.2_colors', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.3_colors', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.4_colors', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.5_colors', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_0.8_colors', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.0_colors', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_1.5_colors', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_2.0_colors', 'pca_harmony_leiden_3.0', 'pca_harmony_leiden_3.0_colors', 'tissue_block_id_colors', 'umap'
    obsm: 'X_pca', 'X_pca_harmony', 'X_umap_pca_harmony', 'spatial'
    varm: 'PCs'
    obsp: 'pca_harmony_connectivities', 'pca_harmony_distances'
```

Example 10 (unknown):
```unknown
os.getcwd()
```

Example 11 (unknown):
```unknown
'/rfs/project/rfs-iCNyzSAaucw/kk837/notebooks/Foetal/Xenium/5K/C194_Hst45/CamU'
```

Example 12 (python):
```python
adata.X.data[:5]
```

Example 13 (unknown):
```unknown
array([5., 1., 1., 2., 2.], dtype=float32)
```

Example 14 (python):
```python
latent_space = 'pca_harmony'
sc.pl.embedding(adata,
                    basis=f"X_umap_{latent_space}",
                    color='celltypist_mid_fin',
                   )
```

Example 15 (python):
```python
adata.obs['celltypist_mid_fin'].value_counts()
```

Example 16 (unknown):
```unknown
celltypist_mid_fin
VentricularCardiomyocytes      188122
Fibroblasts                     49242
AtrialCardiomyocytes            47076
BloodVesselEndothelialCells     20055
MuralCells                      10986
PericardialCells                 8698
unassigned                       7967
Glia                             6976
EpicardialCells                  5206
EndocardialCells                 5202
MyeloidCells                     3965
LymphaticEndothelialCells        3674
LymphoidCells                    3116
Neurons                          1992
Name: count, dtype: int64
```

Example 17 (python):
```python
subset_adata_dir = '/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium/subsets'
fine_grain_colname = 'fine_grain_12Jun2025'
```

Example 18 (python):
```python
!ls {subset_adata_dir}
```

Example 19 (unknown):
```unknown
C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat_5K_AtrialCardiomyocytes_lognorm.h5ad
C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat_5K_BloodVesselEndothelialCells_lognorm.h5ad
C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat_5K_EndocardialCells_lognorm.h5ad
C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat_5K_EpicardialCells_lognorm.h5ad
C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat_5K_Fibroblasts_lognorm.h5ad
C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat_5K_Glia_lognorm.h5ad
C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat_5K_LymphaticEndothelialCells_lognorm.h5ad
C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat_5K_LymphoidCells_lognorm.h5ad
C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat_5K_MuralCells_lognorm.h5ad
C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat_5K_MyeloidCells_lognorm.h5ad
C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat_5K_Neurons_lognorm.h5ad
C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat_5K_PericardialCells_lognorm.h5ad
C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat_5K_VentricularCardiomyocytes_lognorm.h5ad
```

Example 20 (python):
```python
file_list = os.listdir(subset_adata_dir)

filename_head = 'C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat_5K_'
celltypes_to_add = [x.replace(filename_head,'').replace('_lognorm.h5ad','') \
                    for x in file_list if filename_head in x]
celltypes_to_add
```

Example 21 (unknown):
```unknown
['EpicardialCells',
 'PericardialCells',
 'EndocardialCells',
 'LymphaticEndothelialCells',
 'Fibroblasts',
 'AtrialCardiomyocytes',
 'MyeloidCells',
 'Glia',
 'VentricularCardiomyocytes',
 'MuralCells',
 'BloodVesselEndothelialCells',
 'Neurons',
 'LymphoidCells']
```

Example 22 (python):
```python
adata.obs[fine_grain_colname] =  adata.obs['celltypist_mid_fin'].astype('str').copy()
adata.obs['multi_celltypes_coarse'] = np.nan
```

Example 23 (python):
```python
for celltype in celltypes_to_add:
    print(celltype)
    if celltype=='LymphaticEndothelialCells':
        print('skip since the label of mid-grain and fine-grain are same')
        continue
    ad = sc.read_h5ad(f'{subset_adata_dir}/C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat_5K_{celltype}_lognorm.h5ad',backed='r')
    print(ad.shape)
    # add cell type labels
    adata.obs.loc[ad.obs_names,fine_grain_colname] = ad.obs['fine_grain_manual']
    # add 'multi_celltypes_coarse'
    adata.obs.loc[ad.obs_names,'multi_celltypes_coarse'] = ad.obs['multi_celltypes_coarse']
adata.obs[fine_grain_colname].value_counts()
```

Example 24 (unknown):
```unknown
EpicardialCells
(5206, 4657)
PericardialCells
```

Example 25 (python):
```python
/tmp/ipykernel_3225005/343326667.py:11: FutureWarning: Setting an item of incompatible dtype is deprecated and will raise an error in a future version of pandas. Value '['No', 'No', 'No', 'No', 'No', ..., 'No', 'No', 'No', 'No', 'No']
Length: 5206
Categories (2, object): ['No', 'Yes']' has dtype incompatible with float64, please explicitly cast to a compatible dtype first.
  adata.obs.loc[ad.obs_names,'multi_celltypes_coarse'] = ad.obs['multi_celltypes_coarse']
```

Example 26 (unknown):
```unknown
(8698, 4737)
EndocardialCells
(5202, 4492)
LymphaticEndothelialCells
skip since the label of mid-grain and fine-grain are same
Fibroblasts
(49242, 4999)
AtrialCardiomyocytes
(47076, 5000)
MyeloidCells
(3965, 4505)
Glia
(6976, 4699)
VentricularCardiomyocytes
(188122, 5001)
MuralCells
(10986, 4771)
BloodVesselEndothelialCells
(20055, 4950)
Neurons
(1992, 4270)
LymphoidCells
(3116, 4614)
```

Example 27 (unknown):
```unknown
fine_grain_12Jun2025
VentricularCardiomyocytes           104353
unassigned                           74123
Fibroblasts                          32842
VentricularCardiomyocytesCycling     30516
unclassified                         13129
                                     ...  
ErythrocytesCycling                     65
EntericLikeNeurons                      45
Megakaryocytes                          42
TCellsCD4pos                            41
DendriticCells                          26
Name: count, Length: 68, dtype: int64
```

Example 28 (python):
```python
adata.obs['multi_celltypes_coarse'].value_counts()
```

Example 29 (unknown):
```unknown
multi_celltypes_coarse
No     315108
Yes     35528
Name: count, dtype: int64
```

Example 30 (python):
```python
sc.pl.embedding(adata,
                    basis=f"X_umap_{latent_space}",
                    color=fine_grain_colname,
                   )
```

Example 31 (python):
```python
sc.pl.embedding(adata,
                    basis=f"X_umap_{latent_space}",
                    color='multi_celltypes_coarse',
                   )
```

Example 32 (python):
```python
adata.write(path_adata)
adata
```

Example 33 (unknown):
```unknown
AnnData object with n_obs × n_vars = 362277 × 5001
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_3.0', 'multi_celltypes_coarse', 'fine_grain_12Jun2025'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_mid_fin_colors', 'dendrogram_pca_harmony_leiden_1.0', 'multi_celltypes_coarse_colors', 'pca', 'pca_harmony', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.2_colors', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.3_colors', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.4_colors', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.5_colors', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_0.8_colors', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.0_colors', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_1.5_colors', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_2.0_colors', 'pca_harmony_leiden_3.0', 'pca_harmony_leiden_3.0_colors', 'tissue_block_id_colors', 'umap', 'fine_grain_12Jun2025_colors'
    obsm: 'X_pca', 'X_pca_harmony', 'X_umap_pca_harmony', 'spatial'
    varm: 'PCs'
    obsp: 'pca_harmony_connectivities', 'pca_harmony_distances'
```

Example 34 (unknown):
```unknown
'/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium/C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat_5K_filtered_raw.h5ad'
```

Example 35 (python):
```python
for tissue_id in ['C194-HEA-0-FFPE-1', 'Hst45-HEA-0-FFPE-1']:
    slide = adata[adata.obs['tissue_block_id']==tissue_id]
    df = slide.obs[['cell_id',fine_grain_colname]]
    df.columns = ['cell_id','group']
    df.to_csv(f'./{tissue_id}_cellgroup_XR_{fine_grain_colname}.csv',index=False)
```

Example 36 (python):
```python
for tissue_id in ['C194-HEA-0-FFPE-1', 'Hst45-HEA-0-FFPE-1']:
    slide = adata[adata.obs['tissue_block_id']==tissue_id]
    sc.pl.spatial(slide,color=[fine_grain_colname],
                  groups=['SinoatrialNodePacemakerCells','AtrioventricularNodePacemakerCells'],
                  spot_size=60,vmax='p100')
```

Example 37 (python):
```python
for tissue_id in ['C194-HEA-0-FFPE-1', 'Hst45-HEA-0-FFPE-1']:
    slide = adata[adata.obs['tissue_block_id']==tissue_id]
    sc.pl.spatial(slide,color=[fine_grain_colname],
                  groups=['MacrophagesCX3CR1pos','MacrophagesLYVE1pos'],
                  spot_size=60,vmax='p100')
```

Example 38 (python):
```python
for tissue_id in ['C194-HEA-0-FFPE-1', 'Hst45-HEA-0-FFPE-1']:
    slide = adata[adata.obs['tissue_block_id']==tissue_id]
    sc.pl.spatial(slide,color=[fine_grain_colname],
                  groups=['ParasympatheticNeurons','SympatheticNeurons','EntericLikeNeurons'],
                  spot_size=60,vmax='p100')
```

Example 39 (python):
```python
for tissue_id in ['C194-HEA-0-FFPE-1', 'Hst45-HEA-0-FFPE-1']:
    slide = adata[adata.obs['tissue_block_id']==tissue_id]
    sc.pl.spatial(slide,color=[fine_grain_colname],
                  groups=['LymphoidCells'],
                  spot_size=60,vmax='p100')
```

Example 40 (python):
```python
tissue_id = 'C194-HEA-0-FFPE-1'
slide = adata[adata.obs['tissue_block_id']==tissue_id]
sc.pl.spatial(slide,color=[fine_grain_colname],
              groups=['SinoatrialNodePacemakerCells','AtrioventricularNodePacemakerCells','ParasympatheticNeurons','MacrophagesCX3CR1pos','MacrophagesLYVE1pos'],
              spot_size=20,vmax='p100',
              crop_coord=(4500,7000,4000,6000)
             )
```

Example 41 (python):
```python
tissue_id = 'Hst45-HEA-0-FFPE-1'
slide = adata[adata.obs['tissue_block_id']==tissue_id]
sc.pl.spatial(slide,color=[fine_grain_colname],
              groups=['SinoatrialNodePacemakerCells','AtrioventricularNodePacemakerCells','ParasympatheticNeurons','MacrophagesCX3CR1pos','MacrophagesLYVE1pos'],
              spot_size=60,vmax='p100',
              crop_coord=(6000,7000,500,3000)
             )
```

Example 42 (python):
```python
tissue_id = 'Hst45-HEA-0-FFPE-1'
slide = adata[adata.obs['tissue_block_id']==tissue_id]
sc.pl.spatial(slide,color=[fine_grain_colname],
              groups=['SinoatrialNodePacemakerCells','ParasympatheticNeurons','TCellsCD8pos','LymphaticEndothelialCells'],
              spot_size=40,vmax='p100',
              crop_coord=(6000,7000,500,3000)
             )
```

---

## SMCs fro apex or base¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/RNA/scnRNA_Integration-Annotation/Mesenchymal/Figure2SMCs.html

**Contents:**
- SMCs fro apex or base¶
- Pulmonary with only GVSMCs¶
- PA related cell types¶

**Examples:**

Example 1 (unknown):
```unknown
#Added the keys for age and sex
```

Example 2 (python):
```python
# Import packages, initial settings

import numpy as np
import matplotlib.pyplot as plt
import scanpy as sc
import pandas as pd
import anndata
import scrublet as scr
import seaborn as sns
import magic
import pybiomart
import six
import sys
sys.modules['sklearn.externals.six'] = six
import scanorama
import harmonypy
import bbknn
import gseapy
from gseapy.plot import barplot, dotplot
import scvelo as scv
import scvi

plt.rcParams['pdf.fonttype'] = 42
plt.rcParams['ps.fonttype'] = 42

sc.settings.verbosity = 3  # verbosity: errors (0), warnings (1), info (2), hints (3)

from matplotlib.colors import LinearSegmentedColormap
cmap = LinearSegmentedColormap.from_list(name='gene_cmap', colors=['lightgrey', 'thistle', 'red', 'darkred']) 

sc.settings.set_figure_params(dpi=150, color_map='viridis', vector_friendly=False,  dpi_save=300, facecolor='white')
sc.logging.print_version_and_date()
#sc.logging.print_versions()
```

Example 3 (unknown):
```unknown
Global seed set to 0
```

Example 4 (unknown):
```unknown
Running Scanpy 1.8.1, on 2023-09-18 17:07.
```

Example 5 (python):
```python
sc.settings.set_figure_params(dpi=150, color_map='viridis', vector_friendly=False,  dpi_save=300, facecolor='white')
```

Example 6 (python):
```python
adata=sc.read('./Objects/Feb28ObjectRaw.h5ad')
print(adata.shape)
set(adata.obs.fine_grain)
```

Example 7 (unknown):
```unknown
(297473, 36601)
```

Example 8 (unknown):
```unknown
{'AtrialCardiomyocytesCycling',
 'AtrialCardiomyocytesLeft',
 'AtrialCardiomyocytesRight',
 'AtrioventricularNodePacemakerCells',
 'BCells',
 'BCellsMS4A1pos',
 'ChromaffinCells',
 'CoronaryArterialEndothelialCells',
 'CoronaryCapillaryEndothelialCells',
 'CoronaryPericytes',
 'CoronarySmoothMuscleCells',
 'CoronaryVenousEndothelialCells',
 'CoronaryVesselAdventitialFibroblasts',
 'DendriticCellsMature',
 'DendriticCellsType1',
 'DuctusArteriosusSmoothMuscleCells',
 'EndocardialCells',
 'EndocardialCushionCells',
 'EpicardiumDerivedCells',
 'GreatVesselAdventitialFibroblasts',
 'GreatVesselArterialEndothelialCells',
 'GreatVesselSmoothMuscleCells',
 'GreatVesselVenousEndothelialCells',
 'InnateLymphoidCells',
 'LymphNodeFibroblasticReticularCells',
 'LymphaticEndothelialCells',
 'MacrophagesATF3pos',
 'MacrophagesCX3CR1pos',
 'MacrophagesLYVE1pos',
 'MacrophagesTIMD4pos',
 'MastCells',
 'Megakaryocytes',
 'MesothelialEpicardialCells',
 'MonocyteDerivedCells',
 'Monocytes',
 'MonocytesMPOpos',
 'MyocardialInterstitialFibroblasts',
 'Myofibroblasts',
 'NaturalKillerCells',
 'NeuronPrecursors',
 'ParasympatheticNeurons',
 'PericardialCellsFibrous',
 'PericardialCellsIntermediate',
 'PericardialCellsParietal',
 'PlasmacytoidDendriticCells',
 'ProBCells',
 'SchwannCellPrecursors',
 'SchwannCells',
 'SinoatrialNodePacemakerCells',
 'SubEpicardialFibroblasts',
 'SympatheticNeurons',
 'TCellsCD4pos',
 'TCellsCD8pos',
 'TregsCD4pos',
 'ValveEndothelialCells',
 'ValveInterstitialCells',
 'VentricularCardiomyocytesCycling',
 'VentricularCardiomyocytesLeftCompact',
 'VentricularCardiomyocytesLeftTrabeculated',
 'VentricularCardiomyocytesRightCompact',
 'VentricularCardiomyocytesRightTrabeculated',
 'VentricularConductionSystemDistal',
 'VentricularConductionSystemProximal'}
```

Example 9 (python):
```python
#Expression: "YlGn" (e.g. DE plots)
#Divergent expression: "PRGn" (e.g. log2FC)
#Age: "afmhot_r"
#Others: "viridis" (e.g. UMAP expression)

#Define color palette
FineGrainPalette={
    'AtrialCardiomyocytesLeft':'#eac075',
    'AtrialCardiomyocytesRight':'#4c6b8a',
    'AtrialCardiomyocytesCycling':'#e94bc7',
    'VentricularCardiomyocytesLeftCompact':'#ab1c3c',
    'VentricularCardiomyocytesRightCompact':'#fdba45',
    'VentricularCardiomyocytesLeftTrabeculated':'#2b9200',
    'VentricularCardiomyocytesRightTrabeculated':'#0273a8',
    'VentricularCardiomyocytesCycling':'#e1acff',
    'SinoatrialNodePacemakerCells':'#854834',
    'AtrioventricularNodePacemakerCells':'#688500',
    'VentricularConductionSystemProximal':'#01ccc8',
    'VentricularConductionSystemDistal':'#b58fab',
    'GreatVesselAdventitialFibroblasts':'#ff7e97',
    'CoronaryVesselAdventitialFibroblasts':'#ff9b6e',
    'MyocardialInterstitialFibroblasts':'#8a6800',
    'SubEpicardialFibroblasts':'#206601',
    'Myofibroblasts':'#1bdebf',
    'LymphNodeFibroblasticReticularCells':'#005aa0',
    'ValveInterstitialCells':'#e478ff',
    'GreatVesselSmoothMuscleCells':'#ff665f',
    'CoronarySmoothMuscleCells':'#d2cb18',
    'DuctusArteriosusSmoothMuscleCells':'#83d7b3',
    'CoronaryPericytes':'#9088ff',
    'PericardialCellsIntermediate':'#c95e00',
    'PericardialCellsFibrous':'#01e077',
    'PericardialCellsParietal':'#ff88e4',
    'GreatVesselArterialEndothelialCells':'#d4004b',
    'GreatVesselVenousEndothelialCells':'#8f4300',
    'CoronaryArterialEndothelialCells':'#89da5a',
    'CoronaryVenousEndothelialCells':'#006f86',
    'CoronaryCapillaryEndothelialCells':'#aa7dff',
    'EndocardialCells':'#ff8241',
    'EndocardialCushionCells':'#46603a',
    'ValveEndothelialCells':'#414eb6',
    'LymphaticEndothelialCells':'#8b3f60',
    'MesothelialEpicardialCells':'#0295fe',
    'EpicardiumDerivedCells':'#ff92c4',
    'NeuronPrecursors':'#d30038',
    'ChromaffinCells':'#726800',
    'SympatheticNeurons':'#01894b',
    'ParasympatheticNeurons':'#545191',
    'SchwannCellPrecursors':'#b0d276',
    'SchwannCells':'#b30079',
    'MonocytesMPOpos':'#c50066',
    'Monocytes':'#fab4b6',
    'MonocyteDerivedCells':'#d14008',
    'MacrophagesCX3CR1pos':'#eabe97',
    'MacrophagesTIMD4pos':'#b2a200',
    'MacrophagesLYVE1pos':'#4a6005',
    'MacrophagesATF3pos':'#3db833',
    'DendriticCellsType1':'#007859',
    'DendriticCellsMature':'#01a9db',
    'PlasmacytoidDendriticCells':'#2c54ab',
    'MastCells':'#d0bef2',
    'Megakaryocytes':'#94298f',
    'TCellsCD4pos':'#ff387c',
    'TCellsCD8pos':'#9e3429',
    'TregsCD4pos':'#b37700',
    'ProBCells':'#8dab00',
    'BCells':'#a8d1a1',
    'BCellsMS4A1pos':'#00d2f0',
    'NaturalKillerCells':'#abb4ff',
    'InnateLymphoidCells':'#72497f',
}
sc.pl.umap(adata, color='fine_grain',palette=FineGrainPalette)

MidGrainPalette={
    'AtrialCardiomyocytes':'#af3a4d',
    'VentricularCardiomyocytes':'#b1756c',
    'CardiacConductionSystem':'#d45633',
    'Fibroblasts':'#9b6e4e',
    'MuralCells':'#ffb92f',
    'PericardialCells':'#b4f03b',
    'BloodVesselEndothelialCells':'#72cc57',
    'EndocardialCells':'#436e43',
    'LymphaticEndothelialCells':'#85cca0',
    'EpicardialCells':'#5eabb0',
    'Neurons':'#7bbdff',
    'Glia':'#403c90',
    'MyeloidCells':'#420066',
    'LymphoidCells':'#f553f6',
}
sc.pl.umap(adata, color='mid_grain',palette=MidGrainPalette)

CoarseGrainPalette={
    'Cardiomyocytes':'#bc5950',
    'Mesenchymal':'#af9c4c',
    'Endothelium':'#599e58',
    'Epicardium':'#5eabb0',
    'Neural':'#5e75b1',
    'Leukocytes':'#9a52a0',
}
sc.pl.umap(adata, color='coarse_grain',palette=CoarseGrainPalette)

RegionGrainPalette={
    'aorta':'#ffa2bc',
    'apex':'#fe0049',
    'atria':'#732d00',
    'atrial septum':'#ffd54c',
    'base':'#837100',
    'ductus arteriosus':'#bdff8b',
    'whole sample':'#1b6c00',
    'heart without node':'#02e6c6',
    'left ventricle':'#01c1f5',
    'node':'#00466d',
    'outflow tract':'#020011',
    'pericardium':'#7a75ff',
    'pulmonary arches and branches':'#30003e',
    'right ventricle':'#ff88e1',
    'right ventricle and papillaries':'#98004f',
}
sc.pl.umap(adata, color='region',palette=RegionGrainPalette)
```

Example 10 (python):
```python
cell_cycle_genes = ['MCM5','PCNA','TYMS','FEN1','MCM2','MCM4','RRM1','UNG','GINS2','MCM6','CDCA7','DTL','PRIM1','UHRF1','MLF1IP','HELLS','RFC2','RPA2','NASP','RAD51AP1','GMNN','WDR76','SLBP','CCNE2','UBR7','POLD3','MSH2','ATAD2','RAD51','RRM2','CDC45','CDC6','EXO1','TIPIN','DSCC1','BLM','CASP8AP2','USP1','CLSPN','POLA1','CHAF1B','BRIP1','E2F8','HMGB2','CDK1','NUSAP1','UBE2C','BIRC5','TPX2','TOP2A','NDC80','CKS2','NUF2','CKS1B','MKI67','TMPO','CENPF','TACC3','FAM64A','SMC4','CCNB2','CKAP2L','CKAP2','AURKB','BUB1','KIF11','ANP32E','TUBB4B','GTSE1','KIF20B','HJURP','CDCA3','HN1','CDC20','TTK','CDC25C','KIF2C','RANGAP1','NCAPD2','DLGAP5','CDCA2','CDCA8','ECT2','KIF23','HMMR','AURKA','PSRC1','ANLN','LBR','CKAP5','CENPE','CTCF','NEK2','G2E3','GAS2L3','CBX5','CENPA']
s_genes = cell_cycle_genes[:43]
s_genes = [x for x in s_genes if x in adata.var_names]
print('S genes:', s_genes, '\n')
g2m_genes = cell_cycle_genes[43:]
g2m_genes = [x for x in g2m_genes if x in adata.var_names]
print('G2 genes:', g2m_genes, '\n')
cell_cycle_genes = [x for x in cell_cycle_genes if x in adata.var_names]

hb_genes1 = ['HBB','HBD','HBG1','HBG2','HBE1','HBZ','HBM','HBA2','HBA1', 'HBQ1']
hb_genes1 = [x for x in hb_genes1 if x in adata.var_names]
print('Hemoglobin genes set 1:', hb_genes1, '\n')

hb_genes2 = ['AHSP','GYPA','RHAG','GFI1B','KEL','EPB42','ALAS2','TSPO2','GYPB','GYPE','SPTA1','RHCE','KLF1','RHD','SLC4A1','SNX22','SQSTM1','SOD1','FTH1','MAP1LC3B','SLC25A37','TRIM58','HMBS']
hb_genes2 = [x for x in hb_genes2 if x in adata.var_names]
print('Hemoglobin genes set 2:', hb_genes2, '\n')

mito_genes=['MT-ND1','MT-ND2','MT-CO1','MT-CO2','MT-ATP8','MT-ATP6','MT-CO3','MT-ND3','MT-ND4L', 'MT-ND4', 'MT-ND5', 'MT-ND6', 'MT-CYB']
mito_genes = [x for x in mito_genes if x in adata.var_names]
print('Mitochondrial genes:', mito_genes, '\n')

ribo_genes = ['RPLP0','RPLP1','RPLP2','RPL3','RPL3L','RPL4','RPL5','RPL6','RPL7','RPL7A','RPL7L1','RPL8','RPL9','RPL10','RPL10A','RPL10L','RPL11','RPL12','RPL13','RPL13A','RPL14','RPL15','RPL17','RPL18','RPL18A','RPL19','RPL21','RPL22','RPL22L1','RPL23','RPL23A','RPL24','RPL26','RPL26L1','RPL27','RPL27A','RPL28','RPL29','RPL30','RPL31','RPL32','RPL34','RPL35','RPL35A','RPL36','RPL36A','RPL36AL','RPL37','RPL37A','RPL38','RPL39','RPL39L','UBA52','RPL41','RPSA','RPS2','RPS3','RPS3A','RPS4X','RPS4Y1','RPS4Y2','RPS5','RPS6','RPS7','RPS8','RPS9','RPS10','RPS11','RPS12','RPS13','RPS14','RPS15','RPS15A','RPS16','RPS17','RPS18','RPS19','RPS20','RPS21','RPS23','RPS24','RPS25','RPS26','RPS27','RPS27A','RPS27L','RPS28','RPS29','FAU']
ribo_genes = [x for x in ribo_genes if x in adata.var_names]
print('Ribo genes:', ribo_genes, '\n')

stress_genes=['FOS','FOSB','JUN','JUNB','JUND','ZFP36','SOCS3','CEBPD','CEBPB','ATF3','EGR1','HSPA1A','HSPA1B','HSP90AA1','HSP90AB1','HSPA8','HSPB1','HSPE1','IER3','IER2','BTG1','BTG2','DUSP1']
stress_genes = [x for x in stress_genes if x in adata.var_names]
print('Tissue dissociation stress associated genes:', stress_genes, '\n')

FoxProteins=['FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1']
chrY_genes=['SRY', 'RPS4Y1', 'ZFY', 'ZFY-AS1', 'LINC00278', 'PCDH11Y', 'TTTY2B', 'TBL1Y', 'PRKY', 'USP9Y', 'DDX3Y', 'UTY', 'TMSB4Y', 'NLGN4Y', 'TTTY14', 'KDM5D', 'TTTY10', 'EIF1AY', 'PRORY', 'LINC00266-4P']
MembraneProteins1=['AADAT','AAK1','ABCA12','ABCA7','ABCB1','ABCB11','ABCB4','ABCB6','ABCC1','ABCC3','ABCC4','ABCC5','ABCC6','ABCG2','ABHD3','ABI1','ABLIM3','AC004691.2','AC004922.1','AC005324.3','AC005726.1','AC008403.1','AC138696.1','ACIN1','ACKR3','ACLY','ACOT13','ACOT9','ACTN1','ACTR10','ACTR6','ADA','ADAM10','ADAM12','ADAM22','ADAM28','ADAM33','ADAMTS1','ADAMTS2','ADAMTSL4','ADAP1','ADCK5','ADD1','ADD2','ADGRB2','ADGRD1','ADGRG2','ADGRG7','ADGRL2','ADH1A','ADH1B','ADH1C','ADH7','ADK','ADORA1','ADRA1B','ADRM1','ADSS','AFAP1L2','AFDN','AFG1L','AGER','AGRN','AHNAK','AHNAK2','AIFM2','AKAP10','AKAP11','AKAP12','AKR1B10','AKR1B15','AL132780.3','AL139300.1','AL355315.1','AL358113.1','AL645922.1','ALDH3A1','ALK','ALOX15B','ALOXE3','ALPI','ALPP','ALPPL2','ALS2CR12','AMACR','AMDHD1','AMER1','AMER2','AMIGO2','AMOT','AMOTL1','AMOTL2','AMPH','ANK2','ANK3','ANKEF1','ANKFN1','ANKIB1','ANKLE2','ANKRA2','ANKRD13A','ANKRD35','ANKRD42','ANKRD45','ANKS1B','ANO1','ANO10','ANO2','ANO6','ANPEP','ANXA1','ANXA13','ANXA2','ANXA3','AP2M1','APBB1','APBB1IP','APC','APLP1','APMAP','APOC3','APOD','APOLD1','AQP1','AQP10','AQP3','AQP4','AQP5','ARAP1','ARF1','ARHGAP18','ARHGAP19','ARHGAP21','ARHGAP22','ARHGAP23','ARHGAP24','ARHGAP28','ARHGAP29','ARHGAP33','ARHGAP36','ARHGAP40','ARHGAP9','ARHGDIG','ARHGEF1','ARHGEF11','ARHGEF12','ARHGEF15','ARHGEF26','ARHGEF28','ARHGEF40','ARHGEF5','ARID2','ARID4A','ARID5B','ARL13A','ARL15','ARL4C','ARMC12','ARMCX5','ARPC1A','ARPC5','ARRB2','ARRDC4','ARVCF','ASAP1','ASB5','ASGR1','ASGR2','ASH2L','ASIC1','ASIC4','ASPM','ASPSCR1','ATG13','ATG3','ATG7','ATIC','ATOX1','ATP10D','ATP12A','ATP13A3','ATP1B3','ATP23','ATP2B1','ATP2B4','ATP2C2','ATP6V0A2','ATP6V1H','ATP8A2','ATP8B1','ATXN10','ATXN3','ATXN7L3','AVEN','AVPI1','AXIN2','AXL','B2M','B3GALT2','BACE1','BAG3','BAIAP2','BAIAP2L1','BANK1','BASP1','BBOF1','BCAR1','BCAR3','BCL2L12','BCORL1','BCR','BDKRB2','BEND6','BFSP1','BFSP2','BICD2','BIN2','BLNK','BLVRB','BMPR2','BMX','BOC','BOLL','BORCS5','BORCS8-MEF2B','BRK1','BRPF1','BRSK1','BSDC1','BST2','BTK','BTN2A1','BTRC','BUB1B-PAK6','BVES','BZW2','C10orf90','C11orf42','C11orf52','C12orf73','C12orf76','C14orf37','C14orf93','C16orf74','C17orf58','C17orf62','C18orf21','C19orf33','C19orf38','C19orf48','C19orf71','C1orf116','C1orf189','C1orf35','C1orf54','C1orf68','C1orf74','C1QBP','C2orf72','C3orf14','C3orf20','C3orf38','C4orf19','C4orf22','C7orf43','C8orf33','C8orf37','C9orf153','C9orf3','CA14','CA9','CABLES2','CACNA1E','CACNG4','CADM1','CALCRL','CALD1','CALML5','CALN1','CAMK2A','CAMK2B','CAMK2D','CAMK2G','CAP2','CAPS','CASKIN2','CASP4','CAST','CAV3','CAVIN1','CAVIN2','CAVIN3','CAVIN4','CBWD1','CBWD2','CBWD3','CBWD5','CBWD6','CC2D1A','CCBE1','CCDC107','CCDC112','CCDC115','CCDC124','CCDC136','CCDC144NL','CCDC15','CCDC154','CCDC170','CCDC24','CCDC28A','CCDC59','CCDC62','CCDC66','CCDC69','CCDC8','CCDC83','CCDC85C','CCDC93','CCDC97','CCNT2','CCNYL1','CCR3','CCR4','CCT3','CD109','CD163','CD164L2','CD200R1','CD226','CD248','CD274','CD2AP','CD300A','CD300LB','CD33','CD34','CD37','CD38','CD3E','CD3G','CD4','CD44','CD46','CD47','CD48','CD79B','CD81','CD84','CD86','CD8A','CD8B','CD9','CD93','CD99L2','CDAN1','CDC42EP1','CDC42EP3','CDC42EP4','CDC42SE1','CDCA4','CDH1','CDH13','CDH15','CDH17','CDH2','CDH26','CDH3','CDH4','CDH5','CDH6','CDH8','CDIPT','CDK16','CDK5','CDK5RAP2','CDK7','CDRT15','CDV3','CEACAM5','CELSR1','CENPJ','CEP162','CEP295','CEP55','CEP57L1','CEP89','CERCAM','CFAP157','CFAP206','CFAP58','CFAP65','CFB','CFL2','CFLAR','CGN','CGNL1','CHCHD7','CHIC2','CHMP2A','CHRM3','CHRNA1','CHRNA5','CHRNB4','CHRND','CIB1','CIB3','CIP2A','CKAP5','CLCA2','CLCA4','CLCN5','CLCN6','CLDN11','CLDN18','CLDN2','CLDN23','CLDN3','CLDN4','CLDN7','CLDN9','CLEC2A','CLEC4F','CLEC5A','CLECL1','CLIC2','CLIC5','CLTB','CLUL1','CMTM4','CMTM7','CMTR2','CMYA5','CNGA1','CNKSR2','CNNM1','CNNM4','CNP','CNR1','CNTD1','COBL','COBLL1','COG3','COL16A1','COL17A1','COL4A5','COL6A2','COLEC12','COLQ','COMMD4','COMTD1','COPRS','COQ7','CORIN','CORO1B','CORO2A','CORO2B','COX6B2','CPEB3','CPPED1','CPZ','CRB3','CREB5','CRIP1','CRIP2','CRK','CRLF3','CROCC','CRTC1','CRYAB','CRYBG3','CRYL1','CSDE1','CSF1','CSF1R','CSPG4','CST6','CTNNA1','CTNNB1','CTNNBIP1','CTNND1','CTNND2','CTSF','CTSV','CTTN','CTXN1','CUEDC1','CUTA','CX3CL1','CX3CR1','CXADR','CXCR2','CYB5D2','CYBRD1','CYFIP2','CYP20A1','CYP24A1','CYTH2','CYTH4','DAAM1','DAB2','DAG1','DAGLB','DAW1','DBN1','DCAF13','DCAF16','DCBLD2','DCHS1','DCHS2','DCP2','DCT','DCTN6','DCUN1D2','DDAH1','DDB2','DDIAS','DDIT4L','DDR1','DDR2','DDX58','DDX60L','DENND6B','DFFA','DGKG','DGKH','DHDDS','DHPS','DHRS7B','DIAPH1','DIAPH3','DIO2','DIS3L','DLG1','DLG2','DLG5','DLGAP3','DLL3','DMWD','DNAAF1','DNAAF4','DNAH17','DNAJA4','DNAJB13','DNAJB4','DNAJC18','DNAJC5','DNAJC6','DNAJC9','DOC2A','DOCK4','DOK3','DONSON','DPEP1','DPH1','DPPA5','DPYSL2','DQX1','DRD1','DRP2','DSC3','DSEL','DSG2','DSP','DTNA','DUS1L','DUSP15','DXO','DYSF','ECSCR','EDA2R','EDNRA','EDNRB','EEF1AKMT1','EEF2','EFCAB1','EFCAB11','EFCAB2','EFCAB7','EFHC2','EFNB1','EFR3A','EFR3B','EGFR','EHBP1','EHD1','EHD2','EHD3','EHD4','EIF2B2','EIF5','EIF5B','ELFN1','ELMO3','ELN','ELOC','EMP3','ENAH','ENC1','ENG','ENKD1','ENO1','ENO2','ENO3','ENOX1','ENOX2','EPB41','EPB41L1','EPB41L2','EPB41L3','EPB41L4A','EPB41L4B','EPB41L5','EPCAM','EPHA2','EPHA3','EPHB1','EPHB2','EPN1','EPOR','EPS8L2','ERAP1','ERBB2','ERBB3','ERBIN','ERC1','ERCC4','ERMN','ESCO2','ETNK1','ETS2','EVA1A','EVI2B','EXOC1','EXOC3L1','EXOC7','EZR','F11R','F2RL1','F2RL3','FAAP20','FABP2','FABP3','FABP5','FAIM','FAM126A','FAM126B','FAM129A','FAM129B','FAM13A','FAM163B','FAM171A1','FAM193A','FAM205A','FAM210B','FAM222A','FAM45A','FAM47E','FAM47E-STBD1','FAM72A','FAM72B','FAM72C','FAM72D','FANCG','FAS','FASN','FBLIM1','FBLN2','FBLN5','FBLN7','FBP2','FBRSL1','FBXL2','FBXO16','FBXO38','FBXO41','FBXW11','FCAR','FCGBP','FCGR1A','FCGR1B','FCGR2A','FCHSD2','FFAR4','FGD1','FGD5','FGFR2','FGR','FHIT','FHL1','FIBCD1','FIGN','FILIP1','FILIP1L','FKBP2','FLAD1','FLCN','FLNA','FLNB','FLNC','FLOT1','FLOT2','FLRT3','FLT1','FLT4','FLVCR1','FMN2','FMNL2','FMNL3','FNBP1','FOXA2','FOXN3','FP565260.3','FRMD3','FRMD7','FRMD8','FRS2','FSCN1','FTCD','FUT2','FXYD3','FYB1','FYN','FZD1','FZD2','FZD4','FZD6','GAB1','GAB2','GABRA1','GABRA2','GABRA3','GABRA5','GABRG1','GAD1','GADL1','GAGE1','GAGE10','GAGE12C','GAGE12D','GAGE12E','GAGE12F','GAGE12G','GAGE12H','GAGE12J','GAGE13','GAGE2A','GAGE2E','GALNT10','GALR1','GAP43','GAPDH','GAPT','GAPVD1','GAS2','GAS2L1','GAS7','GAS8','GBP4','GCA','GCC1','GCLM','GCSAML-AS1','GFI1B','GFOD1','GFRA3','GHR','GID8','GINM1','GIPC1','GJA1','GJB3','GJB4','GJB6','GJB7','GJC1','GJC2','GLO1','GLP2R','GLRX','GLUL','GMIP','GNAI2','GNAQ','GNAS','GNB1','GNB2','GNB3','GNB4','GNG2','GP9','GPAT3','GPATCH1','GPBP1','GPBP1L1','GPC1','GPC3','GPC4','GPI','GPR1','GPR142','GPR156','GPR157','GPR160','GPR173','GPR20','GPR27','GPR50','GPR63','GPRC5A','GPRIN1','GPRIN2','GPX8','GRAMD1A','GRAMD1B','GRAMD1C','GRAP2','GRB7','GRID2IP','GRIK5','GRK3','GRK5','GRPR','GSDMA','GTF2F1','GTSE1','GUCA1A','GYPA','GYPC','HAPLN3','HARBI1','HAS1','HAUS4','HAUS7','HCAR2','HCAR3','HCFC2','HCK','HCLS1','HCST','HDAC8','HEG1','HELZ','HENMT1','HERC2','HERPUD1','HFE','HIF1A','HIF3A','HINT1','HIP1R','HLA-A','HLA-DPB1','HMGCS1','HMOX1','HOMER3','HOXC5','HOXD4','HPS3','HRASLS5','HRH1','HS6ST3','HSH2D','HSPB1','HSPG2','HTR1B','HTR7','HTRA1','HUNK','HYDIN','HYI','HYLS1','ICAM1','ICAM2','ICOS','ICOSLG','IFFO2','IFIT5','IFNGR1','IFRD1','IFT122','IGFL2','IGSF11','IGSF23','IKZF5','IL10RA','IL12RB1','IL12RB2','IL13','IL13RA2','IL16','IL17RB','IL1R1','IL1RL1','IL21R','IL4R','IL6ST','IL7R','ILDR2','ILK','INPP5A','INSR','INTS12','IP6K2','IPCEF1','IQGAP1','IQGAP2','IREB2','IRF2','IRS1','ISLR','ITGA2B','ITGA3','ITGA4','ITGA7','ITGA8','ITGA9','ITGB1','ITGB2','ITGB3','ITGB4','ITGB5','ITGB6','ITGB7','ITGB8','ITK','ITM2A','ITPKA','ITSN1','ITSN2','IYD','JAG1','JAK2','JAK3','JAM2','JAML','JCAD','JMJD4','JPT2','JUP','KANK1','KANK3','KANSL2','KARS','KATNB1','KCNAB1','KCNB2','KCNH8','KCNJ15','KCNK13','KCNK17','KCNK6','KCNK7','KCNK9','KCNN2','KCNS3','KCNU1','KDF1','KDR','KEL','KIAA0556','KIAA0895','KIAA1324','KIAA1522','KIAA1958','KIF17','KIF19','KIF21A','KIF21B','KIF26A','KIF26B','KISS1R','KIT','KLC1','KLC2','KLF9','KLHDC7B','KLHL41','KLHL6','KLHL7','KLK7','KLRC1','KLRC2','KLRF1','KLRK1','KMT2D','KNDC1','KNSTRN','KNTC1','KPTN','KRT75','KRT77','KRTAP25-1','L1CAM','L3MBTL1','LAIR1','LAMA4','LAMC1','LAMP2','LAMTOR1','LAPTM4A','LASP1','LAT','LAT2','LAX1','LCP1','LCTL','LDB2','LDLRAD3','LENG1','LGALS3','LGALS4','LGALS8','LGR6','LHFPL2','LHX6','LIMA1','LINGO1','LIPJ','LMCD1','LMOD2','LMTK2','LMTK3','LNX1','LNX2','LONRF1','LONRF3','LOXHD1','LPAR1','LPAR3','LPAR4','LPIN1','LPIN2','LRCH2','LRP11','LRP12','LRRC15','LRRC26','LRRC34','LRRC45','LRRC69','LRRC7','LRRC9','LRRFIP2','LRRIQ3','LSMEM2','LSR','LTB4R2','LUZP2','LY6K','LYN','LYPD1','LYST','LZTS1','LZTS2','MACF1','MADD','MAGI1','MAGI3','MAMDC2','MAP1S','MAP2','MAP2K1','MAP2K4','MAP2K7','MAP3K12','MAP3K21','MAP3K6','MAP3K7','MAP4','MAP4K1','MAP4K5','MAP6','MAPK15','MAPK8IP1','MAPT','MARCH10','MARCH7','MARCKS','MARK2','MARVELD2','MAS1L','MAT2B','MBP','MC5R','MCAM','MCEMP1','MCF2L2','MCMBP','MCTS1','MDGA1','ME1','MEF2B','MEIOC','MEPCE','MERTK','MET','METAP1','METAP2','METTL15','METTL25','METTL9','MFAP3L','MFSD1','MFSD2A','MGAT4C','MIB1','MICAL2','MICAL3','MICALL1','MICB','MIEN1','MIGA2','MISP','MKLN1','MKRN3','MLF2','MLKL','MLST8','MMACHC','MMP10','MMP15','MMP24','MMP25','MOAP1','MPHOSPH8','MPHOSPH9','MPI','MPIG6B','MPL','MPP1','MPP5','MPP7','MPZL1','MPZL2','MR1','MRGPRF','MRNIP','MRPL42','MRPL44','MRPL46','MRPS18B','MS4A1','MS4A12','MS4A14','MS4A15','MSN','MSRA','MTCP1','MTFR1L','MTHFR','MTM1','MTPN','MTSS1','MUC1','MUC20','MVD','MYH11','MYH9','MYL4','MYLK','MYO10','MYO1B','MYO1C','MYO1E','MYO1G','MYO3A','MYO5B','MYO9A','MYOF','N4BP2','NAA35','NAB1','NAIF1','NAMPT','NAP1L2','NAPG','NARFL','NCAM1','NCAM2','NCK1','NCKAP1L','NCKAP5','NCKIPSD','NCOA1','NCS1','NDC1','NECTIN2','NECTIN3','NECTIN4','NEDD1','NEDD9','NEO1','NEU1','NEXN','NF2','NFE2L2','NFKBID','NGEF']
MembraneProteins2=['NGFR','NHS','NHSL2','NID2','NINJ2','NKTR','NKX2-8','NLGN1','NLGN3','NLK','NLRX1','NMB','NMBR','NME1','NME2','NMT1','NMT2','NMUR2','NOS1','NOTCH2','NPFFR2','NPL','NPNT','NPR1','NPTN','NPTXR','NR2C1','NRAS','NRCAM','NRP2','NRSN1','NRXN1','NSD1','NSFL1C','NT5E','NTNG1','NTNG2','NTSR1','NUDCD3','NUDT16L1','NUDT9','NUMB','NUP35','NUS1','OAS3','OBSCN','OCA2','OCLN','OCM','OCM2','ODC1','ODR4','OGT','OLFM4','OLIG2','OLR1','OPHN1','OPRK1','OR10P1','OR1N2','OR2C3','OR6B2','OR6B3','OR6C6','OR8S1','OSBPL10','OSBPL6','OSBPL7','OSCAR','OSCP1','OSGEP','OSR2','OTOP2','OTULIN','OXGR1','P3H3','PACSIN2','PACSIN3','PAFAH1B2','PAG1','PAIP1','PAK1','PAK4','PAK6','PALLD','PALM','PALM2','PALM2-AKAP2','PALM3','PALMD','PANX1','PANX2','PAQR8','PARD3','PARD3B','PARD6A','PARD6G','PARP6','PATJ','PAWR','PCBD2','PCDH1','PCDH17','PCDH7','PCDHA1','PCDHA11','PCDHA12','PCDHA4','PCDHA5','PCDHB1','PCDHB11','PCDHB12','PCDHB16','PCDHB4','PCDHB5','PCDHGA1','PCDHGA10','PCDHGA11','PCDHGA12','PCDHGA2','PCDHGA3','PCDHGA4','PCDHGA5','PCDHGA6','PCDHGA7','PCDHGA8','PCDHGA9','PCDHGB1','PCDHGB2','PCDHGB3','PCDHGB4','PCDHGB5','PCDHGB6','PCDHGB7','PCDHGC3','PCDHGC4','PCDHGC5','PCLO','PCMTD1','PDAP1','PDCD7','PDE3A','PDE4A','PDE4D','PDE9A','PDGFC','PDGFRA','PDHX','PDLIM1','PDLIM5','PDPN','PDZD2','PDZD8','PEAR1','PEBP1','PFDN1','PGAP3','PGLYRP2','PGLYRP4','PGRMC2','PHACTR1','PHACTR4','PHF20L1','PHGDH','PHLDB2','PHLDB3','PHOSPHO1','PHPT1','PI3','PI4K2A','PI4KA','PIEZO2','PIGB','PIGN','PIGW','PIGX','PIK3AP1','PIK3CG','PIK3R6','PIP4K2A','PIP4K2C','PIP5K1A','PIRT','PITPNM3','PKD2L2','PKN1','PKN2','PKP1','PKP2','PKP3','PKP4','PLAA','PLAUR','PLCB1','PLCB2','PLCD3','PLCD4','PLEKHA4','PLEKHA6','PLEKHA7','PLEKHG3','PLEKHG4','PLEKHG6','PLEKHH2','PLEKHO2','PLIN4','PLPP1','PLPPR5','PLS3','PLSCR1','PLVAP','PLXNA3','PLXNA4','PNCK','PNISR','PODXL','POLE','POU2F3','PPEF1','PPFIBP1','PPIP5K1','PPL','PPM1A','PPM1F','PPP1CA','PPP1CB','PPP1R12B','PPP1R16A','PPP1R16B','PPP1R18','PPP1R27','PPP1R9A','PPP2R1B','PPP3R1','PPP3R2','PPP4C','PPP6R3','PPTC7','PRAME','PRC1','PRDX6','PRIMA1','PRKCA','PRKCE','PRKCH','PRKCZ','PRKD1','PROM2','PROSER2','PRR15','PRR32','PRR7','PRRC2A','PRRG1','PRRT2','PRRT3','PRRT4','PRTG','PRX','PSCA','PSD','PSD2','PSEN1','PSMC5','PSMC6','PSMD9','PSTPIP2','PTAFR','PTCHD4','PTK2B','PTK6','PTPN12','PTPN21','PTPN22','PTPN4','PTPRB','PTPRG','PTPRK','PTPRM','PTPRR','PTPRS','PTX3','PURG','PVALB','PVR','PXDC1','PXK','PYGL','PYGO2','PYM1','RAB13','RAB23','RAB25','RAB26','RAB27B','RAB3B','RAB3GAP2','RAB43','RAB8A','RABEPK','RAD51C','RAD52','RAI2','RALA','RALGAPA2','RAPGEF2','RAPH1','RAPSN','RASA4','RASA4B','RASAL1','RASAL2','RASAL3','RASD1','RASD2','RASGRP1','RASGRP2','RASGRP4','RASSF2','RASSF3','RASSF4','RBM46','RC3H1','RCCD1','RCSD1','RD3L','RDX','RECK','RELL1','REPS1','RET','RFLNB','RFPL1','RFPL2','RFPL3','RGP1','RGS10','RGS14','RGS19','RGS3','RGS7BP','RGS9BP','RHCE','RHD','RHEX','RHOA','RHOBTB2','RHOJ','RHOXF2','RHOXF2B','RIC8A','RIC8B','RIF1','RILPL1','RIMBP3','RIMBP3B','RIMBP3C','RIMS2','RIOK2','RIOK3','RIPK1','RND1','RNF114','RNF145','RNF146','RNF149','RNF165','RNF166','RNF39','ROBO1','ROBO3','ROBO4','ROM1','RP2','RPE','RPEL1','RPGRIP1L','RPIA','RPL36A','RPL36A-HNRNPH2','RPL36AL','RPSA','RPUSD1','RRAD','RRP12','RRP7A','RSBN1L','RSC1A1','RSPH3','RTL3','RTL4','RTN4R','RTN4RL1','RYR2','S100A13','S100A14','S100A16','S100A2','S100A3','S100A4','S100A6','S100A9','SAMD4A','SAMHD1','SAMSN1','SAPCD1','SASH3','SC5D','SCAMP1','SCAMP4','SCEL','SCHIP1','SCIN','SCN1A','SCN5A','SCN8A','SCN9A','SCNN1D','SCNN1G','SCRIB','SCUBE3','SDC2','SDC4','SDE2','SDHB','SEC16B','SEMA3C','SEMA3D','SEMA4B','SEMA4F','SEMA6C','SEMA6D','SEPT2','SEPT3','SERINC2','SERINC5','SERPINB3','SFSWAP','SGCE','SGK494','SH3BGR','SH3D21','SHANK1','SHANK3','SHARPIN','SHISA6','SHKBP1','SHROOM2','SHTN1','SIGLEC10','SIGLEC14','SIGLEC6','SIGLEC7','SIN3B','SIPA1','SIPA1L1','SIPA1L3','SKAP1','SLAMF6','SLC10A5','SLC12A2','SLC13A3','SLC13A5','SLC14A1','SLC16A1','SLC16A10','SLC16A2','SLC16A3','SLC16A7','SLC16A9','SLC17A5','SLC19A1','SLC19A3','SLC1A5','SLC1A6','SLC22A16','SLC23A3','SLC24A4','SLC26A9','SLC27A1','SLC28A1','SLC2A1','SLC2A11','SLC2A12','SLC2A14','SLC2A2','SLC2A3','SLC30A1','SLC31A2','SLC34A1','SLC34A2','SLC35F5','SLC35G2','SLC36A4','SLC38A5','SLC38A6','SLC38A7','SLC39A10','SLC39A14','SLC39A2','SLC39A6','SLC3A2','SLC40A1','SLC41A3','SLC43A1','SLC43A3','SLC44A2','SLC45A4','SLC46A1','SLC46A3','SLC4A1AP','SLC4A2','SLC4A7','SLC4A8','SLC5A12','SLC5A3','SLC5A6','SLC5A7','SLC6A6','SLC7A14','SLC7A2','SLC7A5','SLC7A6','SLC8A1','SLC9A1','SLC9A2','SLC9A3','SLC9A3R1','SLC9A3R2','SLC9A4','SLC9A5','SLC9B2','SLC9C2','SLCO1A2','SLCO1B3','SLCO4A1','SLCO4C1','SLCO5A1','SLIT1','SLIT3','SLITRK6','SLK','SLX4','SMAGP','SMAP1','SMOC1','SMPD2','SMPX','SNAP23','SNAP25','SNRK','SNTB2','SNTG2','SNX15','SNX32','SNX9','SOCS3','SOCS5','SOD1','SORBS1','SORCS1','SORT1','SOX11','SPAG16','SPANXN4','SPART','SPATA12','SPATC1L','SPERT','SPG11','SPHK1','SPINK13','SPINT1','SPINT2','SPN','SPPL2B','SPPL3','SPRED3','SPRYD3','SPTBN2','SRC','SRCIN1','SRD5A3','SRPK1','SRPRB','SRRM3','SSFA2','SSH1','SSH3','SSTR1','SSTR5','SSX2IP','ST8SIA5','STAB1','STAC','STAC2','STAMBP','STAMBPL1','STARD4','STAT2','STEAP4','STIL','STIP1','STK10','STK16','STK17A','STK17B','STOM','STOML2','STOML3','STX18','STX4','STXBP4','STYK1','SUCLG1','SUCLG2','SUGT1','SUMF2','SUPT6H','SUSD2','SVIL','SWAP70','SYCP2','SYK','SYMPK','SYNC','SYNGR3','SYNPO2L','SYPL2','SYT4','SYTL2','SYVN1','TACC2','TACR2','TACSTD2','TASP1','TAZ','TBC1D10A','TBC1D2','TBC1D24','TBC1D30','TBC1D9B','TBXA2R','TCAF2','TCAF2C','TCHP','TCIM','TDG','TDO2','TDP1','TEC','TEDC2','TEK','TES','TEX14','TEX9','TGFBR1','TGFBR2','TGM1','TGM2','THBS1','THBS2','THEGL','THY1','TIAM1','TICAM1','TIGD7','TIMD4','TJP1','TJP2','TJP3','TLN1','TLR4','TM2D2','TM4SF20','TM4SF4','TM4SF5','TMC5','TMED8','TMEM101','TMEM107','TMEM108','TMEM121B','TMEM139','TMEM141','TMEM164','TMEM176B','TMEM183A','TMEM186','TMEM2','TMEM218','TMEM222','TMEM229A','TMEM245','TMEM251','TMEM254','TMEM258','TMEM266','TMEM268','TMEM50A','TMEM63B','TMPRSS15','TMPRSS2','TMTC2','TNFRSF10D','TNFRSF11A','TNFRSF12A','TNFRSF21','TNFRSF25','TNFRSF8','TNFRSF9','TNFSF13B','TNK1','TNKS1BP1','TNNI2','TOGARAM2','TOLLIP','TOM1','TP53BP2','TPD52L1','TPRN','TRAF7','TRAIP','TRAPPC13','TRAT1','TRDN','TREML1','TRIB1','TRIM16','TRIM16L','TRIM38','TRIM4','TRIM44','TRIM54','TRIM71','TRIML2','TRIP6','TRMT1','TRMT10B','TRMT13','TRMT6','TRO','TRPM1','TRPM4','TRPM7','TRPV2','TRPV3','TSG101','TSGA13','TSHZ3','TSPAN15','TSPAN6','TSPO2','TSR2','TSSK4','TTC17','TTC23','TTC7B','TTLL12','TTLL5','TULP3','TWF2','TXNDC11','TXNIP','TYROBP','TYW3','UAP1','UBA3','UBA52','UBAC1','UBALD1','UBAP1','UBASH3B','UBE2A','UBE2B','UBE2C','UBE2D1','UBE2D2','UBE2D3','UBE2D4','UBE2K','UBE2QL1','UBE2S','UBE3C','UBQLN2','UBR2','UBTD1','UCP3','ULBP1','ULBP2','UNC5A','UNC79','UQCC1','USP14','USP15','USP2','USP20','USP21','USP4','USP53','UTP20','UTRN','VAMP5','VAMP8','VANGL1','VASP','VCAM1','VCPIP1','VIL1','VPS13B','VPS36','VPS50','VPS8','VSIG4','VTCN1','VWA5B1','VWA7','WAS','WASF2','WBP4','WDFY3','WDPCP','WDR1','WDR12','WDR13','WDR53','WDR6','WDR66','WIPF1','WWP1','XIAP','XIRP1','XIRP2','XPC','XPNPEP2','XPO6','XRN1','XYLT1','YES1','YIPF1','YIPF4','YOD1','YTHDC1','ZBED3','ZBTB33','ZBTB8OS','ZDHHC2','ZDHHC20','ZDHHC22','ZDHHC5','ZFP41','ZFP90','ZFYVE28','ZGPAT','ZMYND10','ZNF10','ZNF18','ZNF185','ZNF2','ZNF20','ZNF227','ZNF229','ZNF236','ZNF276','ZNF277','ZNF337','ZNF397','ZNF407','ZNF497','ZNF503','ZNF510','ZNF536','ZNF571','ZNF598','ZNF599','ZNF646','ZNF655','ZNF688','ZNF706','ZNF717','ZNF783','ZNF79','ZNF839','ZSCAN22','ZSCAN29','ZSWIM1','ZSWIM8','ZYX']
MembraneProteins=MembraneProteins1+MembraneProteins2

TFs1=['AC008770.3','AC023509.3','AC092835.1','AC138696.1','ADNP','ADNP2','AEBP1','AEBP2','AHCTF1','AHDC1','AHR','AHRR','AIRE','AKAP8','AKAP8L','AKNA','ALX1','ALX3','ALX4','ANHX','ANKZF1','AR','ARGFX','ARHGAP35','ARID2','ARID3A','ARID3B','ARID3C','ARID5A','ARID5B','ARNT','ARNT2','ARNTL','ARNTL2','ARX','ASCL1','ASCL2','ASCL3','ASCL4','ASCL5','ASH1L','ATF1','ATF2','ATF3','ATF4','ATF5','ATF6','ATF6B','ATF7','ATMIN','ATOH1','ATOH7','ATOH8','BACH1','BACH2','BARHL1','BARHL2','BARX1','BARX2','BATF','BATF2','BATF3','BAZ2A','BAZ2B','BBX','BCL11A','BCL11B','BCL6','BCL6B','BHLHA15','BHLHA9','BHLHE22','BHLHE23','BHLHE40','BHLHE41','BNC1','BNC2','BORCS8-MEF2B','BPTF','BRF2','BSX','C11orf95','CAMTA1','CAMTA2','CARF','CASZ1','CBX2','CC2D1A','CCDC169-SOHLH2','CCDC17','CDC5L','CDX1','CDX2','CDX4','CEBPA','CEBPB','CEBPD','CEBPE','CEBPG','CEBPZ','CENPA','CENPB','CENPBD1','CENPS','CENPT','CENPX','CGGBP1','CHAMP1','CHCHD3','CIC','CLOCK','CPEB1','CPXCR1','CREB1','CREB3','CREB3L1','CREB3L2','CREB3L3','CREB3L4','CREB5','CREBL2','CREBZF','CREM','CRX','CSRNP1','CSRNP2','CSRNP3','CTCF','CTCFL','CUX1','CUX2','CXXC1','CXXC4','CXXC5','DACH1','DACH2','DBP','DBX1','DBX2','DDIT3','DEAF1','DLX1','DLX2','DLX3','DLX4','DLX5','DLX6','DMBX1','DMRT1','DMRT2','DMRT3','DMRTA1','DMRTA2','DMRTB1','DMRTC2','DMTF1','DNMT1','DNTTIP1','DOT1L','DPF1','DPF3','DPRX','DR1','DRAP1','DRGX','DUX1','DUX3','DUX4','DUXA','DZIP1','E2F1','E2F2','E2F3','E2F4','E2F5','E2F6','E2F7','E2F8','E4F1','EBF1','EBF2','EBF3','EBF4','EEA1','EGR1','EGR2','EGR3','EGR4','EHF','ELF1','ELF2','ELF3','ELF4','ELF5','ELK1','ELK3','ELK4','EMX1','EMX2','EN1','EN2','EOMES','EPAS1','ERF','ERG','ESR1','ESR2','ESRRA','ESRRB','ESRRG','ESX1','ETS1','ETS2','ETV1','ETV2','ETV3','ETV3L','ETV4','ETV5','ETV6','ETV7','EVX1','EVX2','FAM170A','FAM200B','FBXL19','FERD3L','FEV','FEZF1','FEZF2','FIGLA','FIZ1','FLI1','FLYWCH1','FOS','FOSB','FOSL1','FOSL2','FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXN4','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1','GABPA','GATA1','GATA2','GATA3','GATA4','GATA5','GATA6','GATAD2A','GATAD2B','GBX1','GBX2','GCM1','GCM2','GFI1','GFI1B','GLI1','GLI2','GLI3','GLI4','GLIS1','GLIS2','GLIS3','GLMP','GLYR1','GMEB1','GMEB2','GPBP1','GPBP1L1','GRHL1','GRHL2','GRHL3','GSC','GSC2','GSX1','GSX2','GTF2B','GTF2I','GTF2IRD1','GTF2IRD2','GTF2IRD2B','GTF3A','GZF1','HAND1','HAND2','HBP1','HDX','HELT','HES1','HES2','HES3','HES4','HES5','HES6','HES7','HESX1','HEY1','HEY2','HEYL','HHEX','HIC1','HIC2','HIF1A','HIF3A','HINFP','HIVEP1','HIVEP2','HIVEP3','HKR1','HLF','HLX','HMBOX1','HMG20A','HMG20B','HMGA1','HMGA2','HMGN3','HMX1','HMX2','HMX3','HNF1A','HNF1B','HNF4A','HNF4G','HOMEZ','HOXA1','HOXA10','HOXA11','HOXA13','HOXA2','HOXA3','HOXA4','HOXA5','HOXA6','HOXA7','HOXA9','HOXB1','HOXB13','HOXB2','HOXB3','HOXB4','HOXB5','HOXB6','HOXB7','HOXB8','HOXB9','HOXC10','HOXC11','HOXC12','HOXC13','HOXC4','HOXC5','HOXC6','HOXC8','HOXC9','HOXD1','HOXD10','HOXD11','HOXD12','HOXD13','HOXD3','HOXD4','HOXD8','HOXD9','HSF1','HSF2','HSF4','HSF5','HSFX1','HSFX2','HSFY1','HSFY2','IKZF1','IKZF2','IKZF3','IKZF4','IKZF5','INSM1','INSM2','IRF1','IRF2','IRF3','IRF4','IRF5','IRF6','IRF7','IRF8','IRF9','IRX1','IRX2','IRX3','IRX4','IRX5','IRX6','ISL1','ISL2','ISX','JAZF1','JDP2','JRK','JRKL','JUN','JUNB','JUND','KAT7','KCMF1','KCNIP3','KDM2A','KDM2B','KDM5B','KIN','KLF1','KLF10','KLF11','KLF12','KLF13','KLF14','KLF15','KLF16','KLF17','KLF2','KLF3','KLF4','KLF5','KLF6','KLF7','KLF8','KLF9','KMT2A','KMT2B','L3MBTL1','L3MBTL3','L3MBTL4','LBX1','LBX2','LCOR','LCORL','LEF1','LEUTX','LHX1','LHX2','LHX3','LHX4','LHX5','LHX6','LHX8','LHX9','LIN28A','LIN28B','LIN54','LMX1A','LMX1B','LTF','LYL1','MAF','MAFA','MAFB','MAFF','MAFG','MAFK','MAX','MAZ','MBD1','MBD2','MBD3','MBD4','MBD6','MBNL2','MECOM','MECP2','MEF2A','MEF2B','MEF2C','MEF2D','MEIS1','MEIS2','MEIS3','MEOX1','MEOX2','MESP1','MESP2','MGA','MITF','MIXL1','MKX','MLX','MLXIP','MLXIPL','MNT','MNX1','MSANTD1','MSANTD3','MSANTD4','MSC','MSGN1','MSX1','MSX2','MTERF1','MTERF2','MTERF3','MTERF4','MTF1','MTF2','MXD1','MXD3','MXD4','MXI1','MYB','MYBL1','MYBL2','MYC','MYCL','MYCN','MYF5','MYF6','MYNN','MYOD1','MYOG','MYPOP','MYRF','MYRFL','MYSM1','MYT1','MYT1L','MZF1','NACC2','NAIF1','NANOG','NANOGNB','NANOGP8','NCOA1','NCOA2','NCOA3','NEUROD1','NEUROD2','NEUROD4','NEUROD6','NEUROG1','NEUROG2','NEUROG3','NFAT5','NFATC1','NFATC2','NFATC3','NFATC4','NFE2','NFE2L1','NFE2L2','NFE2L3','NFE4','NFIA','NFIB','NFIC','NFIL3','NFIX','NFKB1','NFKB2','NFX1','NFXL1','NFYA','NFYB','NFYC','NHLH1','NHLH2','NKRF','NKX1-1','NKX1-2','NKX2-1','NKX2-2','NKX2-3','NKX2-4','NKX2-5','NKX2-6','NKX2-8','NKX3-1','NKX3-2','NKX6-1','NKX6-2','NKX6-3','NME2','NOBOX','NOTO','NPAS1','NPAS2','NPAS3','NPAS4','NR0B1','NR1D1','NR1D2','NR1H2','NR1H3','NR1H4','NR1I2','NR1I3','NR2C1','NR2C2','NR2E1','NR2E3','NR2F1','NR2F2','NR2F6','NR3C1','NR3C2','NR4A1','NR4A2','NR4A3','NR5A1','NR5A2','NR6A1','NRF1','NRL','OLIG1','OLIG2','OLIG3','ONECUT1','ONECUT2','ONECUT3','OSR1','OSR2','OTP','OTX1','OTX2','OVOL1','OVOL2','OVOL3','PA2G4','PATZ1','PAX1','PAX2','PAX3','PAX4','PAX5','PAX6','PAX7','PAX8','PAX9','PBX1','PBX2','PBX3','PBX4','PCGF2','PCGF6','PDX1','PEG3','PGR','PHF1','PHF19 ','PHF20','PHF21A','PHOX2A','PHOX2B','PIN1','PITX1','PITX2','PITX3','PKNOX1','PKNOX2','PLAG1','PLAGL1','PLAGL2','PLSCR1','POGK','POU1F1','POU2AF1','POU2F1','POU2F2','POU2F3','POU3F1','POU3F2','POU3F3','POU3F4','POU4F1','POU4F2','POU4F3','POU5F1','POU5F1B','POU5F2','POU6F1','POU6F2','PPARA','PPARD','PPARG','PRDM1','PRDM10','PRDM12','PRDM13','PRDM14','PRDM15','PRDM16','PRDM2','PRDM4','PRDM5','PRDM6','PRDM8','PRDM9','PREB','PRMT3','PROP1','PROX1','PROX2','PRR12','PRRX1','PRRX2','PTF1A','PURA','PURB','PURG','RAG1','RARA','RARB','RARG','RAX','RAX2','RBAK','RBCK1','RBPJ','RBPJL','RBSN','REL','RELA','RELB','REPIN1','REST','REXO4','RFX1','RFX2','RFX3','RFX4','RFX5','RFX6','RFX7','RFX8','RHOXF1','RHOXF2','RHOXF2B','RLF','RORA','RORB','RORC','RREB1','RUNX1','RUNX2','RUNX3','RXRA','RXRB','RXRG','SAFB','SAFB2','SALL1','SALL2','SALL3','SALL4','SATB1','SATB2','SCMH1','SCML4','SCRT1','SCRT2','SCX','SEBOX','SETBP1','SETDB1','SETDB2','SGSM2','SHOX','SHOX2','SIM1','SIM2','SIX1','SIX2','SIX3','SIX4','SIX5','SIX6','SKI','SKIL','SKOR1','SKOR2','SLC2A4RG','SMAD1','SMAD3','SMAD4','SMAD5','SMAD9','SMYD3','SNAI1','SNAI2','SNAI3','SNAPC2','SNAPC4','SNAPC5','SOHLH1','SOHLH2','SON','SOX1','SOX10','SOX11']
TFs2=['SOX12','SOX13','SOX14','SOX15','SOX17','SOX18','SOX2','SOX21','SOX3','SOX30','SOX4','SOX5','SOX6','SOX7','SOX8','SOX9','SP1','SP100','SP110','SP140','SP140L','SP2','SP3','SP4','SP5','SP6','SP7','SP8','SP9','SPDEF','SPEN','SPI1','SPIB','SPIC','SPZ1','SRCAP','SREBF1','SREBF2','SRF','SRY','ST18','STAT1','STAT2','STAT3','STAT4','STAT5A','STAT5B','STAT6','T','TAL1','TAL2','TBP','TBPL1','TBPL2','TBR1','TBX1','TBX10','TBX15','TBX18','TBX19','TBX2','TBX20','TBX21','TBX22','TBX3','TBX4','TBX5','TBX6','TCF12','TCF15','TCF20','TCF21','TCF23','TCF24','TCF3','TCF4','TCF7','TCF7L1','TCF7L2','TCFL5','TEAD1','TEAD2','TEAD3','TEAD4','TEF','TERB1','TERF1','TERF2','TET1','TET2','TET3','TFAP2A','TFAP2B','TFAP2C','TFAP2D','TFAP2E','TFAP4','TFCP2','TFCP2L1','TFDP1','TFDP2','TFDP3','TFE3','TFEB','TFEC','TGIF1','TGIF2','TGIF2LX','TGIF2LY','THAP1','THAP10','THAP11','THAP12','THAP2','THAP3','THAP4','THAP5','THAP6','THAP7','THAP8','THAP9','THRA','THRB','THYN1','TIGD1','TIGD2','TIGD3','TIGD4','TIGD5','TIGD6','TIGD7','TLX1','TLX2','TLX3','TMF1','TOPORS','TP53','TP63','TP73','TPRX1','TRAFD1','TRERF1','TRPS1','TSC22D1','TSHZ1','TSHZ2','TSHZ3','TTF1','TWIST1','TWIST2','UBP1','UNCX','USF1','USF2','USF3','VAX1','VAX2','VDR','VENTX','VEZF1','VSX1','VSX2','WIZ','WT1','XBP1','XPA','YBX1','YBX2','YBX3','YY1','YY2','ZBED1','ZBED2','ZBED3','ZBED4','ZBED5','ZBED6','ZBED9','ZBTB1','ZBTB10','ZBTB11','ZBTB12','ZBTB14','ZBTB16','ZBTB17','ZBTB18','ZBTB2','ZBTB20','ZBTB21','ZBTB22','ZBTB24','ZBTB25','ZBTB26','ZBTB3','ZBTB32','ZBTB33','ZBTB34','ZBTB37','ZBTB38','ZBTB39','ZBTB4','ZBTB40','ZBTB41','ZBTB42','ZBTB43','ZBTB44','ZBTB45','ZBTB46','ZBTB47','ZBTB48','ZBTB49','ZBTB5','ZBTB6','ZBTB7A','ZBTB7B','ZBTB7C','ZBTB8A','ZBTB8B','ZBTB9','ZC3H8','ZEB1','ZEB2','ZFAT','ZFHX2','ZFHX3','ZFHX4','ZFP1','ZFP14','ZFP2','ZFP28','ZFP3','ZFP30','ZFP37','ZFP41','ZFP42','ZFP57','ZFP62','ZFP64','ZFP69','ZFP69B','ZFP82','ZFP90','ZFP91','ZFP92','ZFPM1','ZFPM2','ZFX','ZFY','ZGLP1','ZGPAT','ZHX1','ZHX2','ZHX3','ZIC1','ZIC2','ZIC3','ZIC4','ZIC5','ZIK1','ZIM2','ZIM3','ZKSCAN1','ZKSCAN2','ZKSCAN3','ZKSCAN4','ZKSCAN5','ZKSCAN7','ZKSCAN8','ZMAT1','ZMAT4','ZNF10','ZNF100','ZNF101','ZNF107','ZNF112','ZNF114','ZNF117','ZNF12','ZNF121','ZNF124','ZNF131','ZNF132','ZNF133','ZNF134','ZNF135','ZNF136','ZNF138','ZNF14','ZNF140','ZNF141','ZNF142','ZNF143','ZNF146','ZNF148','ZNF154','ZNF155','ZNF157','ZNF16','ZNF160','ZNF165','ZNF169','ZNF17','ZNF174','ZNF175','ZNF177','ZNF18','ZNF180','ZNF181','ZNF182','ZNF184','ZNF189','ZNF19','ZNF195','ZNF197','ZNF2','ZNF20','ZNF200','ZNF202','ZNF205','ZNF207','ZNF208','ZNF211','ZNF212','ZNF213','ZNF214','ZNF215','ZNF217','ZNF219','ZNF22','ZNF221','ZNF222','ZNF223','ZNF224','ZNF225','ZNF226','ZNF227','ZNF229','ZNF23','ZNF230','ZNF232','ZNF233','ZNF234','ZNF235','ZNF236','ZNF239','ZNF24','ZNF248','ZNF25','ZNF250','ZNF251','ZNF253','ZNF254','ZNF256','ZNF257','ZNF26','ZNF260','ZNF263','ZNF264','ZNF266','ZNF267','ZNF268','ZNF273','ZNF274','ZNF275','ZNF276','ZNF277','ZNF28','ZNF280A','ZNF280B','ZNF280C','ZNF280D','ZNF281','ZNF282','ZNF283','ZNF284','ZNF285','ZNF286A','ZNF286B','ZNF287','ZNF292','ZNF296','ZNF3','ZNF30','ZNF300','ZNF302','ZNF304','ZNF311','ZNF316','ZNF317','ZNF318','ZNF319','ZNF32','ZNF320','ZNF322','ZNF324','ZNF324B','ZNF326','ZNF329','ZNF331','ZNF333','ZNF334','ZNF335','ZNF337','ZNF33A','ZNF33B','ZNF34','ZNF341','ZNF343','ZNF345','ZNF346','ZNF347','ZNF35','ZNF350','ZNF354A','ZNF354B','ZNF354C','ZNF358','ZNF362','ZNF365','ZNF366','ZNF367','ZNF37A','ZNF382','ZNF383','ZNF384','ZNF385A','ZNF385B','ZNF385C','ZNF385D','ZNF391','ZNF394','ZNF395','ZNF396','ZNF397','ZNF398','ZNF404','ZNF407','ZNF408','ZNF41','ZNF410','ZNF414','ZNF415','ZNF416','ZNF417','ZNF418','ZNF419','ZNF420','ZNF423','ZNF425','ZNF426','ZNF428','ZNF429','ZNF43','ZNF430','ZNF431','ZNF432','ZNF433','ZNF436','ZNF438','ZNF439','ZNF44','ZNF440','ZNF441','ZNF442','ZNF443','ZNF444','ZNF445','ZNF446','ZNF449','ZNF45','ZNF451','ZNF454','ZNF460','ZNF461','ZNF462','ZNF467','ZNF468','ZNF469','ZNF470','ZNF471','ZNF473','ZNF474','ZNF479','ZNF48','ZNF480','ZNF483','ZNF484','ZNF485','ZNF486','ZNF487','ZNF488','ZNF490','ZNF491','ZNF492','ZNF493','ZNF496','ZNF497','ZNF500','ZNF501','ZNF502','ZNF503','ZNF506','ZNF507','ZNF510','ZNF511','ZNF512','ZNF512B','ZNF513','ZNF514','ZNF516','ZNF517','ZNF518A','ZNF518B','ZNF519','ZNF521','ZNF524','ZNF525','ZNF526','ZNF527','ZNF528','ZNF529','ZNF530','ZNF532','ZNF534','ZNF536','ZNF540','ZNF541','ZNF543','ZNF544','ZNF546','ZNF547','ZNF548','ZNF549','ZNF550','ZNF551','ZNF552','ZNF554','ZNF555','ZNF556','ZNF557','ZNF558','ZNF559','ZNF560','ZNF561','ZNF562','ZNF563','ZNF564','ZNF565','ZNF566','ZNF567','ZNF568','ZNF569','ZNF57','ZNF570','ZNF571','ZNF572','ZNF573','ZNF574','ZNF575','ZNF576','ZNF577','ZNF578','ZNF579','ZNF580','ZNF581','ZNF582','ZNF583','ZNF584','ZNF585A','ZNF585B','ZNF586','ZNF587','ZNF587B','ZNF589','ZNF592','ZNF594','ZNF595','ZNF596','ZNF597','ZNF598','ZNF599','ZNF600','ZNF605','ZNF606','ZNF607','ZNF608','ZNF609','ZNF610','ZNF611','ZNF613','ZNF614','ZNF615','ZNF616','ZNF618','ZNF619','ZNF620','ZNF621','ZNF623','ZNF624','ZNF625','ZNF626','ZNF627','ZNF628','ZNF629','ZNF630','ZNF639','ZNF641','ZNF644','ZNF645','ZNF646','ZNF648','ZNF649','ZNF652','ZNF653','ZNF654','ZNF655','ZNF658','ZNF66','ZNF660','ZNF662','ZNF664','ZNF665','ZNF667','ZNF668','ZNF669','ZNF670','ZNF671','ZNF672','ZNF674','ZNF675','ZNF676','ZNF677','ZNF678','ZNF679','ZNF680','ZNF681','ZNF682','ZNF683','ZNF684','ZNF687','ZNF688','ZNF689','ZNF69','ZNF691','ZNF692','ZNF695','ZNF696','ZNF697','ZNF699','ZNF7','ZNF70','ZNF700','ZNF701','ZNF703','ZNF704','ZNF705A','ZNF705B','ZNF705D','ZNF705E','ZNF705G','ZNF706','ZNF707','ZNF708','ZNF709','ZNF71','ZNF710','ZNF711','ZNF713','ZNF714','ZNF716','ZNF717','ZNF718','ZNF721','ZNF724','ZNF726','ZNF727','ZNF728','ZNF729','ZNF730','ZNF732','ZNF735','ZNF736','ZNF737','ZNF74','ZNF740','ZNF746','ZNF747','ZNF749','ZNF750','ZNF75A','ZNF75D','ZNF76','ZNF761','ZNF763','ZNF764','ZNF765','ZNF766','ZNF768','ZNF77','ZNF770','ZNF771','ZNF772','ZNF773','ZNF774','ZNF775','ZNF776','ZNF777','ZNF778','ZNF780A','ZNF780B','ZNF781','ZNF782','ZNF783','ZNF784','ZNF785','ZNF786','ZNF787','ZNF788','ZNF789','ZNF79','ZNF790','ZNF791','ZNF792','ZNF793','ZNF799','ZNF8','ZNF80','ZNF800','ZNF804A','ZNF804B','ZNF805','ZNF808','ZNF81','ZNF813','ZNF814','ZNF816','ZNF821','ZNF823','ZNF827','ZNF829','ZNF83','ZNF830','ZNF831','ZNF835','ZNF836','ZNF837','ZNF84','ZNF841','ZNF843','ZNF844','ZNF845','ZNF846','ZNF85','ZNF850','ZNF852','ZNF853','ZNF860','ZNF865','ZNF878','ZNF879','ZNF880','ZNF883','ZNF888','ZNF891','ZNF90','ZNF91','ZNF92','ZNF93','ZNF98','ZNF99','ZSCAN1','ZSCAN10','ZSCAN12','ZSCAN16','ZSCAN18','ZSCAN2','ZSCAN20','ZSCAN21','ZSCAN22','ZSCAN23','ZSCAN25','ZSCAN26','ZSCAN29','ZSCAN30','ZSCAN31','ZSCAN32','ZSCAN4','ZSCAN5A','ZSCAN5B','ZSCAN5C','ZSCAN9','ZUFSP','ZXDA','ZXDB','ZXDC','ZZZ3']
TFs=TFs1+TFs2

#TFs other, inclusive of the above list, not necessarily DNA binding proteins but affect transcription
TFs1Other=['AATF','ABL1','ABRA','AC008770.3','AC020915.1','AC023509.3','AC092835.1','AC105001.2','AC138696.1','ACOD1','ACTR3B','ACTR3C','ADAM8','ADAMTS17','ADAMTS19','ADAR','ADGRA1','ADGRG3','ADNP','ADNP2','AEBP1','AEBP2','AES','AFAP1 ','AFF1','AFF3','AFF4','AGER','AGT','AHCTF1','AHDC1','AHR','AHRR','AIM2','AIRE','AKAP8','AKAP8L','AKNA','AKT1','ALK','ALKBH2','ALX1','ALX3','ALX4','AMH','ANAPC2','ANHX','ANKRD30A','ANKZF1','ANXA3','ANXA4','AP5Z1','APBB3','APC2','APEX1','APOA4','APOBEC3A','APOBEC3B','APOBEC3G','APTX','AR','ARGFX','ARHGAP35','ARHGEF2','ARHGEF5','ARID1A','ARID1B','ARID2','ARID3A','ARID3B','ARID3C','ARID4A','ARID4B','ARID5A','ARID5B','ARNT','ARNT2','ARNTL','ARNTL2','ARRB1','ARRB2','ARX','ASCC1','ASCL1','ASCL2','ASCL3','ASCL4','ASCL5','ASH1L','ATF1','ATF2','ATF3','ATF4','ATF5','ATF6','ATF6B','ATF7','ATF7IP','ATM','ATMIN','ATOH1','ATOH7','ATOH8','ATR','ATRX','ATXN7','BACH1','BACH2','BAG1','BANF1','BANP','BARHL1','BARHL2','BARX1','BARX2','BATF','BATF2','BATF3','BAZ2A','BAZ2B','BBX','BCL10','BCL11A','BCL11B','BCL3','BCL6','BCL6B','BCLAF1','BCOR','BDP1','BEND4','BEX1','BHLHA15','BHLHA9','BHLHE22','BHLHE23','BHLHE40','BHLHE41','BHMG1','BIN1','BLM','BLZF1','BMI1','BMP2','BMP4','BMP7','BMPR1A','BNC1','BNC2','BOD1L1','BOLA1','BOLA2','BOLA2B','BOLA3','BORCS8-MEF2B','BPNT1','BPTF','BRCA1','BRD4','BRD7','BRD8','BRD9','BRF1','BRF2','BRMS1','BRPF1','BSX','BTAF1','BTBD8','BTG2','BTK','BTRC','BUD31','C11orf95','C14orf39','C17orf49','C1D','C20orf194','C3orf33','CACTIN','CAMK1D','CAMK2A','CAMTA1','CAMTA2','CAPN15','CAPN3','CARD11','CARD14','CARF','CARHSP1','CASP8AP2','CASZ1','CAT','CBFA2T2','CBFA2T3','CBFB','CBL','CBLL1','CBX2','CC2D1A','CC2D1B','CCDC124','CCDC169-SOHLH2','CCDC17','CCDC3','CCDC71','CCDC83','CCNT1','CCNT2','CD36','CD40','CD40LG','CDC5L','CDC6','CDK5RAP3','CDK7','CDK8','CDK9','CDKN1A','CDKN2A','CDX1','CDX2','CDX4','CEBPA','CEBPB','CEBPD','CEBPE','CEBPG','CEBPZ','CELF3','CENPA','CENPB','CENPBD1','CENPS','CENPT','CENPX','CERS2','CERS3','CERS4','CERS5','CERS6','CFLAR','CGGBP1','CHAMP1','CHCHD3','CHD1','CHD2','CHD3','CHD4','CHD6','CHD7','CHD9','CHP1','CHRAC1','CHUK','CHURC1','CIB1','CIC','CIITA','CIR1','CITED1','CITED2','CITED4','CIZ1','CLOCK','CLU','CMKLR1','CNBP','CNOT3','CNOT4','CNOT7','CNOT8','COMMD1','COMMD6','COMMD7','COPS2','COPS3','COPS4','CPEB1','CPSF4','CPSF4L','CPXCR1','CRAMP1','CREB1','CREB3','CREB3L1','CREB3L2','CREB3L3','CREB3L4','CREB5','CREBBP','CREBL2','CREBRF','CREBZF','CREG1','CREM','CRTC1','CRTC2','CRTC3','CRX','CSDC2','CSDE1','CSRNP1','CSRNP2','CSRNP3','CTBP1','CTBP2','CTCF','CTCFL','CTH','CTNNB1','CTNNBIP1','CUL1','CUL2','CUL3','CUL4A','CUL4B','CUL5','CUX1','CUX2','CXXC1','CXXC4','CXXC5','CYLD','CYP1B1','CYTL1','DAB2IP','DACH1','DACH2','DAP','DAXX','DBP','DBX1','DBX2','DCP1A','DDB1','DDB2','DDIT3','DDN','DDR2','DDRGK1','DDX3X','DDX5','DDX58','DEAF1','DEK','DEPDC1','DEPDC1B','DEPDC4','DEPDC5','DEPDC7','DEPTOR','DHX34','DHX36','DHX57','DISP3','DLX1','DLX2','DLX3','DLX4','DLX5','DLX6','DMAP1','DMBX1','DMC1','DMRT1','DMRT2','DMRT3','DMRTA1','DMRTA2','DMRTB1','DMRTC1','DMRTC2','DMTF1','DNA2','DNAJA3','DNAJC1','DNAJC2','DNAJC21','DNMT1','DNMT3A','DNMT3B','DNMT3L','DNTT','DNTTIP1','DOT1L','DPF1','DPF2','DPF3','DPRX','DR1','DRAP1','DRGX','DSP','DTX1','DUS3L','DUSP12','DUX1','DUX3','DUX4','DUXA','DVL1','DVL2','DVL3','DZIP1','DZIP1L','E2F1','E2F2','E2F3','E2F4','E2F5','E2F6','E2F7','E2F8','E4F1','EAF2','EBF1','EBF2','EBF3','EBF4','EDA','EDA2R','EDF1','EDN1','EEA1','EED','EGLN1','EGR1','EGR2','EGR3','EGR4','EHF','EID1','EID2','EIF2AK2','EIF2AK4','EIF3C','EIF3CL','EIF3K','ELF1','ELF2','ELF3','ELF4','ELF5','ELK1','ELK3','ELK4','ELMSAN1','EME1','EMX1','EMX2','EN1','EN2','ENO1','EOMES','EP300','EP400','EPAS1','EPHA5','EPM2AIP1','ERBIN','ERC1','ERCC3','ERCC4','ERF','ERG','ESR1','ESR2','ESRRA','ESRRB','ESRRG','ESX1','ETS1','ETS2','ETV1','ETV2','ETV3','ETV3L','ETV4','ETV5','ETV6','ETV7','EVX1','EVX2','EWSR1','EXO1','EXOC2','EXOG','EZH1','EZH2','FAAP24','FAM153C','FAM170A','FAM171B','FAM200A','FAM200B','FAN1','FANCA','FANCD2','FANK1','FARSA','FARSB','FBN1','FBXL19','FBXO41','FEN1','FER','FERD3L','FEV','FEZF1','FEZF2','FGD1','FHL2','FIGLA','FIZ1','FLI1','FLNA','FLYWCH1','FMNL2','FOS','FOSB','FOSL1','FOSL2','FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L2','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXN4','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1','FUBP1','FUBP3','FUS','FZD1','FZD2','FZD4','FZD6','GABPA','GABPB1','GABPB2','GADD45A','GAS6','GAS7','GATA1','GATA2','GATA3','GATA4','GATA5','GATA6','GATAD1','GATAD2A','GATAD2B','GBX1','GBX2','GCFC2','GCM1','GCM2','GEN1','GFI1','GFI1B','GLA','GLI1','GLI2','GLI3','GLI4','GLIS1','GLIS2','GLIS3','GLMP','GLYR1','GMEB1','GMEB2','GOLGB1','GON4L','GPATCH8','GPBP1','GPBP1L1','GPER1','GPR155','GREM1','GRHL1','GRHL2','GRHL3','GRM6','GSC','GSC2','GSX1','GSX2','GTF2A1','GTF2A1L','GTF2A2','GTF2B','GTF2E2','GTF2F1','GTF2F2','GTF2H1','GTF2H2','GTF2H2C','GTF2H3','GTF2H4','GTF2I','GTF2IRD1','GTF2IRD2','GTF2IRD2B','GTF3A','GTF3C1','GTF3C2','GULP1','GZF1','H1F0','H1FOO','H1FX','HAND1','HAND2','HAVCR2','HBP1','HCFC1','HCK','HDAC1','HDAC10','HDAC2','HDAC3','HDAC4','HDAC5','HDAC6','HDAC7','HDAC8','HDAC9','HDX','HELT','HELZ','HELZ2','HES1','HES2','HES3','HES4','HES5','HES6','HES7','HESX1','HEY1','HEY2','HEYL','HHEX','HIC1','HIC2','HIF1A','HIF3A','HINFP','HIPK2','HIRA','HIST1H1A','HIST1H1B','HIST1H1C','HIST1H1D','HIST1H1E','HIST1H1T','HIVEP1','HIVEP2','HIVEP3','HKR1','HLA-DQB1','HLA-DQB2','HLA-DRB1','HLA-DRB3','HLA-DRB5','HLF','HLTF','HLX','HMBOX1','HMG20A','HMG20B','HMGA1','HMGA2','HMGB1','HMGB2','HMGB3','HMGB4','HMGN1','HMGN3','HMGXB3','HMGXB4','HMOX1','HMX1','HMX2','HMX3','HNF1A','HNF1B','HNF4A','HNF4G','HNRNPA1','HNRNPA3','HNRNPAB','HNRNPCL2','HNRNPD','HNRNPDL','HNRNPK','HNRNPL','HNRNPU','HOMEZ','HOOK2','HOPX','HOXA1','HOXA10','HOXA11','HOXA13','HOXA2','HOXA3','HOXA4','HOXA5','HOXA6','HOXA7','HOXA9','HOXB1','HOXB13','HOXB2','HOXB3','HOXB4','HOXB5','HOXB6','HOXB7','HOXB8','HOXB9','HOXC10','HOXC11','HOXC12','HOXC13','HOXC4','HOXC5','HOXC6','HOXC8','HOXC9','HOXD1','HOXD10','HOXD11','HOXD12','HOXD13','HOXD3','HOXD4','HOXD8','HOXD9','HP1BP3','HR','HSBP1','HSF1','HSF2','HSF4','HSF5','HSFX1','HSFX2','HSFY1','HSFY2','HSPA1A','HSPA1B','ICAM1','ID1','ID2','ID3','ID4','IFI16','IFNB1','IGHM','IKBKB','IKBKG','IKZF1','IKZF2','IKZF3','IKZF4','IKZF5','IL10','IL1B','IL1RAP','IL4','IL5','IL6','INF2','ING4','INS','INSM1','INSM2','IQGAP1','IRAK1','IRAK2','IRAK3','IRF1','IRF2','IRF2BP1','IRF3','IRF4','IRF5','IRF6','IRF7','IRF8','IRF9','IRX1','IRX2','IRX3','IRX4','IRX5','IRX6','ISL1','ISL2','ISX','ITCH','ITGB2','ITGB3BP','IZUMO2','JADE1','JAK2','JARID2','JAZF1','JDP2','JMY','JRK','JRKL','JUN','JUNB','JUND','JUP','KANSL2','KAT2A','KAT2B','KAT5','KAT6A','KAT6B','KAT7','KAT8','KCMF1','KCNIP3','KDM1A','KDM2A','KDM2B','KDM3A','KDM5A','KDM5B','KDM5C','KDM5D','KEAP1','KIAA1549','KIAA1683','KIAA1841','KIAA1958','KIF15','KIN','KIT','KLF1','KLF10','KLF11','KLF12','KLF13','KLF14','KLF15','KLF16','KLF17','KLF2','KLF3','KLF4','KLF5','KLF6','KLF7','KLF8','KLF9','KMT2A','KMT2B','KMT2C','KMT2D','KNTC1','KRAS','KRBOX1','KRT13','KRTAP5-1','KRTAP5-9','KRTAP9-9','L3MBTL1','L3MBTL3','L3MBTL4','LARP1','LARP1B','LARP4','LARP4B','LARP6','LARP7','LBX1','LBX2','LCOR','LCORL','LEF1','LEUTX','LGALS9','LGR4','LHX1','LHX2','LHX3','LHX4','LHX5','LHX6','LHX8','LHX9','LIG1','LIG3','LIN28A','LIN28B','LIN54','LMO2','LMO4','LMX1A','LMX1B','LPP','LRP5','LRP6','LRP8','LRRFIP1','LRRFIP2','LSR','LTF','LTN1','LYAR','LYL1','LZTFL1','LZTR1','LZTS1','MACC1','MACF1','MAD2L2','MAEL','MAF','MAFA','MAFB','MAFF','MAFG','MAFK','MALT1','MAML1','MAML3','MAP2K5','MAP3K10','MAP3K13','MAP3K7','MAPK1','MAPK10','MAPK11','MAPK14','MAPK3','MAPK8','MAPK9','MATR3','MAVS','MAX','MAZ','MB21D1','MBD1','MBD2','MBD3','MBD4','MBD6','MBNL1','MBNL2','MBNL3','MBTPS2','MDFI','MDM2','MDM4','MECOM','MECP2','MED1','MED12','MED13','MED13L','MED14','MED15','MED16','MED17','MED21','MED23','MED24','MED6','MEF2A','MEF2B','MEF2C','MEF2D','MEIS1','MEIS2','MEIS3','MEN1','MEOX1','MEOX2','MESP1','MESP2','MET','MGA','MGMT','MID2','MIER1','MIER2','MIER3','MINK1','MIS18BP1','MITF','MIXL1','MKL1','MKL2','MKRN1','MKRN2','MKRN3','MKX','MLLT1','MLLT10','MLLT11','MLLT3','MLLT6','MLX','MLXIP','MLXIPL','MN1','MNAT1','MNT','MNX1','MORC3','MPG','MRPL28','MRRF','MSANTD1','MSANTD3','MSANTD4','MSC','MSGN1','MSH2','MSH3','MSH6','MSL3','MSRB2','MST1R','MSX1','MSX2','MTA1','MTA2','MTA3','MTDH','MTERF1','MTERF2','MTERF3','MTERF4','MTF1','MTF2','MTPN','MUS81','MXD1','MXD3','MXD4','MXI1','MYB','MYBBP1A','MYBL1','MYBL2','MYC','MYCBP','MYCL','MYCN','MYD88','MYEF2','MYF5','MYF6','MYNN','MYOCD','MYOD1','MYOG','MYPOP','MYRF','MYRFL','MYSM1','MYT1','MYT1L','MZF1','NAB2','NABP2','NACC2','NAIF1','NANOG','NANOGNB','NANOGP8','NCOA1','NCOA2','NCOA3','NCOA4','NCOA5','NCOA6','NCOA7','NCOR1','NCOR2','NDC80','NDN','NDP','NEB','NEIL1','NEUROD1','NEUROD2','NEUROD4','NEUROD6','NEUROG1','NEUROG2','NEUROG3','NFAM1','NFAT5','NFATC1','NFATC2','NFATC3','NFATC4','NFE2','NFE2L1','NFE2L2','NFE2L3','NFE4','NFIA','NFIB','NFIC','NFIL3','NFIX','NFKB1','NFKB2','NFKBIA','NFKBIB','NFKBID','NFKBIL1','NFKBIZ','NFRKB','NFX1','NFXL1','NFYA','NFYB','NFYC','NHLH1','NHLH2','NKRF','NKX1-1','NKX1-2','NKX2-1','NKX2-2','NKX2-3','NKX2-4','NKX2-5','NKX2-6','NKX2-8','NKX3-1','NKX3-2','NKX6-1','NKX6-2','NKX6-3','NLRC3','NLRC4','NLRC5','NLRP12','NLRP2B','NLRP3','NME2','NOBOX','NOC3L','NOC4L','NOCT','NOD1','NOD2','NODAL','NONO','NOP2','NOTCH1','NOTCH2','NOTCH3','NOTO','NPAS1','NPAS2','NPAS3','NPAS4','NPM1','NR0B1','NR0B2','NR1D1','NR1D2','NR1H2','NR1H3','NR1H4','NR1I2','NR1I3','NR2C1','NR2C2','NR2E1','NR2E3','NR2F1','NR2F2','NR2F6','NR3C1','NR3C2','NR4A1','NR4A2','NR4A3','NR5A1','NR5A2','NR6A1','NRBF2','NRF1','NRG1','NRIP1','NRIP3','NRK','NRL','NSD1','NSD2','NTRK1','NUCKS1','NUFIP1','NUP98','NUPL2']
TFs2Other=['OGG1','OLIG1','OLIG2','OLIG3','ONECUT1','ONECUT2','ONECUT3','OPRD1','OSR1','OSR2','OTOP3','OTP','OTULIN','OTX1','OTX2','OVOL1','OVOL2','OVOL3','PA2G4','PABPN1','PARK7','PARP1','PARP10','PARP12','PATZ1','PAWR','PAX1','PAX2','PAX3','PAX4','PAX5','PAX6','PAX7','PAX8','PAX9','PAXBP1','PAXIP1','PBRM1','PBX1','PBX2','PBX3','PBX4','PCBD1','PCBP1','PCBP2','PCBP3','PCGF2','PCGF6','PCNA','PCSK6','PDCD7','PDS5B','PDX1','PEG3','PELI1','PER1','PER2','PER3','PEX14','PFDN1','PGBD1','PGR','PGS1','PHB','PHB2','PHF1','PHF19 ','PHF20','PHF21A','PHF5A','PHOX2A','PHOX2B','PHTF1','PHTF2','PIAS1','PIAS2','PIAS3','PIAS4','PIDD1','PIKFYVE','PIM1','PIN1','PINK1','PITX1','PITX2','PITX3','PKHD1','PKHD1L1','PKNOX1','PKNOX2','PLA2G10','PLA2G1B','PLAG1','PLAGL1','PLAGL2','PLEK','PLEK2','PLEKHA4','PLPP3','PLSCR1','PLXNA1','PLXNA2','PLXNA3','PLXNA4','PLXNB1','PLXNB2','PLXNB3','PLXNC1','PLXND1','PML','PMS1','PNKP','POGK','POGZ','POLA1','POLB','POLE3','POLE4','POLG','POLH','POLI','POLK','POLL','POLM','POLN','POLQ','POLR1B','POLR2A','POLR2B','POLR2E','POLR2J','POLR2K','POLR2L','POLR3A','POLRMT','POT1','POU1F1','POU2AF1','POU2F1','POU2F2','POU2F3','POU3F1','POU3F2','POU3F3','POU3F4','POU4F1','POU4F2','POU4F3','POU5F1','POU5F1B','POU5F2','POU6F1','POU6F2','PPARA','PPARD','PPARG','PPARGC1A','PPARGC1B','PPP1R10','PPP1R13L','PPP1R21','PPP2R3B','PPP2R5B','PPP3CA','PPRC1','PQBP1','PRAM1','PRAP1','PRB3','PRDM1','PRDM10','PRDM11','PRDM12','PRDM13','PRDM14','PRDM15','PRDM16','PRDM2','PRDM4','PRDM5','PRDM6','PRDM7','PRDM8','PRDM9','PRDX3','PREB','PREX1','PREX2','PRH2','PRIM2','PRIMPOL','PRKCB','PRKCH','PRKCI','PRKCQ','PRKCZ','PRKD1','PRKD2','PRKN','PRMT2','PRMT3','PRNP','PROP1','PROX1','PROX2','PRR12','PRR3','PRRX1','PRRX2','PSMA6','PSMC5','PSMD10','PSMD11','PSMD12','PSPC1','PTCH1','PTEN','PTF1A','PTGIS','PTH','PTTG1','PUM3','PURA','PURB','PURG','PYCARD','PYDC1','PYDC2','QRICH1','RAB11FIP4','RAB7B','RAD21','RAD51','RAG1','RAI1','RAPGEF3','RAPGEF4','RAPGEF5','RARA','RARB','RARG','RAX','RAX2','RB1','RBAK','RBBP7','RBCK1','RBL1','RBL2','RBM10','RBM20','RBM22','RBM26','RBM27','RBM5','RBM6','RBPJ','RBPJL','RBSN','RC3H1','RC3H2','RCAN1','RCOR1','RCOR2','RCOR3','RECQL','REL','RELA','RELB','RELL2','RELN','REPIN1','RERE','REST','REV1','REXO4','RFC1','RFX1','RFX2','RFX3','RFX4','RFX5','RFX6','RFX7','RFX8','RFXANK','RFXAP','RGCC','RGS11','RGS6','RGS7','RGS9','RHEBL1','RHOXF1','RHOXF2','RHOXF2B','RING1','RIOK2','RIPK1','RIPK2','RIPK3','RIPK4','RLF','RLIM','RNASE2','RNASEH1','RNF113A','RNF113B','RNF114','RNF125','RNF138','RNF14','RNF166','RNF2','RNF219','RNF25','RNF31','RNF4','RNF41','RORA','RORB','RORC','RPA2','RPA4','RPL36A','RPL7','RPL7A','RPS27A','RPS6KA4','RPS6KA5','RREB1','RRN3','RUNX1','RUNX1T1','RUNX2','RUNX3','RWDD3','RXRA','RXRB','RXRG','RYBP','S100A12','S100A8','S100A9','SAFB','SAFB2','SALL1','SALL2','SALL3','SALL4','SART1','SATB1','SATB2','SAV1','SCAND1','SCAND2P','SCAPER','SCMH1','SCML1','SCML2','SCML4','SCOC','SCRT1','SCRT2','SCX','SEBOX','SEMA4A','SERTAD1','SERTAD2','SETBP1','SETD6','SETDB1','SETDB2','SF3A2','SF3A3','SFPQ','SFRP4','SFRP5','SFTPB','SGK1','SGK3','SGSM2','SHH','SHOX','SHOX2','SHPRH','SIGIRR','SIK1','SIM1','SIM2','SIN3A','SIN3B','SIRT1','SIRT2','SIRT3','SIRT6','SIVA1','SIX1','SIX2','SIX3','SIX4','SIX5','SIX6','SKI','SKIDA1','SKIL','SKOR1','SKOR2','SLC22A4','SLC26A10','SLC26A3','SLC2A4RG','SLC30A9','SLC39A10','SLC4A10','SMAD1','SMAD2','SMAD3','SMAD4','SMAD5','SMAD6','SMAD7','SMAD9','SMARCA1','SMARCA2','SMARCA4','SMARCA5','SMARCAL1','SMARCB1','SMARCC1','SMARCC2','SMARCD1','SMARCD2','SMARCD3','SMARCE1','SMC3','SMO','SMUG1','SMYD1','SMYD3','SNAI1','SNAI2','SNAI3','SNAPC2','SNAPC4','SNAPC5','SNRPC','SNURF','SNW1','SOGA3','SOHLH1','SOHLH2','SON','SORBS2','SOX1','SOX10','SOX11','SOX12','SOX13','SOX14','SOX15','SOX17','SOX18','SOX2','SOX21','SOX3','SOX30','SOX4','SOX5','SOX6','SOX7','SOX8','SOX9','SP1','SP100','SP110','SP140','SP140L','SP2','SP3','SP4','SP5','SP6','SP7','SP8','SP9','SPDEF','SPEN','SPHK1','SPI1','SPIB','SPIC','SPZ1','SRCAP','SREBF1','SREBF2','SRF','SRY','SSB','SSBP2','SSBP3','SSBP4','SSH1','SSH2','SSH3','SSRP1','SSX1','ST18','STAG1','STAT1','STAT2','STAT3','STAT4','STAT5A','STAT5B','STAT6','STK16','STK3','STK36','STRN','STRN3','SUB1','SUFU','SUMO1','SUPT16H','SUPT4H1','SUPT5H','SUPT6H','SUV39H1','SUV39H2','SUZ12','SYK','T','TAB1','TAB2','TAB3','TADA2A','TADA2B','TADA3','TAF1','TAF10','TAF11','TAF12','TAF13','TAF1A','TAF1B','TAF1C','TAF1L','TAF2','TAF3','TAF4','TAF4B','TAF5','TAF5L','TAF6','TAF6L','TAF7','TAF7L','TAF8','TAF9','TAF9B','TAL1','TAL2','TARDBP','TAX1BP1','TBL1Y','TBP','TBPL1','TBPL2','TBR1','TBX1','TBX10','TBX15','TBX18','TBX19','TBX2','TBX20','TBX21','TBX22','TBX3','TBX4','TBX5','TBX6','TCEA2','TCEAL1','TCEAL7','TCEAL8','TCF12','TCF15','TCF19','TCF20','TCF21','TCF23','TCF24','TCF25','TCF3','TCF4','TCF7','TCF7L1','TCF7L2','TCFL5','TDG','TDP1','TDP2','TEAD1','TEAD2','TEAD3','TEAD4','TEF','TERB1','TERF1','TERF2','TERF2IP','TESMIN','TET1','TET2','TET3','TFAM','TFAP2A','TFAP2B','TFAP2C','TFAP2D','TFAP2E','TFAP4','TFCP2','TFCP2L1','TFDP1','TFDP2','TFDP3','TFE3','TFEB','TFEC','TGFB1','TGFB1I1','TGIF1','TGIF2','TGIF2LX','TGIF2LY','TGS1','THAP1','THAP10','THAP11','THAP12','THAP2','THAP3','THAP4','THAP5','THAP6','THAP7','THAP8','THAP9','THRA','THRAP3','THRB','THYN1','TIA1','TICAM1','TIGD1','TIGD2','TIGD3','TIGD4','TIGD5','TIGD6','TIGD7','TIPARP','TIRAP','TLE1','TLE2','TLE3','TLE4','TLE6','TLR2','TLR3','TLR4','TLR9','TLX1','TLX2','TLX3','TMEM235','TMF1','TNF','TNFAIP3','TNFRSF11A','TNFRSF4','TNFSF11','TNFSF18','TNFSF4','TOE1','TONSL','TOP1','TOP2A','TOP2B','TOPORS','TOX','TOX2','TOX3','TOX4','TP53','TP53BP1','TP63','TP73','TPRX1','TRADD','TRAF1','TRAF2','TRAF3','TRAF4','TRAF5','TRAF6','TRAFD1','TRAK2','TRAPPC9','TRERF1','TREX1','TRIB1','TRIB3','TRIM13','TRIM14','TRIM15','TRIM21','TRIM22','TRIM23','TRIM24','TRIM25','TRIM26','TRIM27','TRIM28','TRIM29','TRIM3','TRIM31','TRIM32','TRIM33','TRIM34','TRIM37','TRIM38','TRIM40','TRIM5','TRIM52','TRIM62','TRIM8','TRIP11','TRIT1','TRMT1','TRMT1L','TRPS1','TRRAP','TSC2','TSC22D1','TSC22D2','TSC22D3','TSC22D4','TSHZ1','TSHZ2','TSHZ3','TSN','TSSK4','TTF1','TUB','TULP4','TUT1','TWIST1','TWIST2','TXK','U2AF1','U2AF1L4','UBA52','UBB','UBC','UBE2K','UBE2N','UBE2V1','UBN1','UBP1','UBR4','UBTF','UBTFL1','UBTFL6','UFL1','UHRF1','UHRF2','UNCX','UNG','UNK','UNKL','USF1','USF2','USF3','USP39','USP7','UTF1','VAV1','VAX1','VAX2','VDR','VEGFA','VENTX','VEZF1','VGLL1','VGLL2','VGLL3','VHLL','VPS72','VSX1','VSX2','WDHD1','WFS1','WIZ','WNT1','WNT10B','WNT2','WNT3A','WNT5A','WNT8B','WRN','WT1','WWOX','WWP2','WWTR1','XBP1','XCL1','XPA','YBX1','YBX2','YBX3','YEATS2','YEATS4','YOD1','YY1','YY2','ZBED1','ZBED2','ZBED3','ZBED4','ZBED5','ZBED6','ZBED6CL','ZBED8','ZBED9','ZBP1','ZBTB1','ZBTB10','ZBTB11','ZBTB12','ZBTB14','ZBTB16','ZBTB17','ZBTB18','ZBTB2','ZBTB20','ZBTB21','ZBTB22','ZBTB24','ZBTB25','ZBTB26','ZBTB3','ZBTB32','ZBTB33','ZBTB34','ZBTB37','ZBTB38','ZBTB39','ZBTB4','ZBTB40','ZBTB41','ZBTB42','ZBTB43','ZBTB44','ZBTB45','ZBTB46','ZBTB47','ZBTB48','ZBTB49','ZBTB5','ZBTB6','ZBTB7A','ZBTB7B','ZBTB7C','ZBTB8A','ZBTB8B','ZBTB9','ZC3H10','ZC3H11A','ZC3H12A','ZC3H13','ZC3H14','ZC3H15','ZC3H18','ZC3H3','ZC3H4','ZC3H6','ZC3H7A','ZC3H7B','ZC3H8','ZC3HAV1','ZCCHC11','ZCCHC6','ZDHHC11','ZDHHC19','ZEB1','ZEB2','ZFAND5','ZFAT','ZFHX2','ZFHX3','ZFHX4','ZFP1','ZFP14','ZFP2','ZFP28','ZFP3','ZFP30','ZFP36','ZFP36L1','ZFP36L2','ZFP37','ZFP41','ZFP42','ZFP57','ZFP62','ZFP64','ZFP69','ZFP69B','ZFP82','ZFP90','ZFP91','ZFP92','ZFPL1','ZFPM1','ZFPM2','ZFR','ZFR2','ZFX','ZFY','ZFYVE26','ZGLP1','ZGPAT','ZHX1','ZHX2','ZHX3','ZIC1','ZIC2','ZIC3','ZIC4','ZIC5','ZIK1','ZIM2','ZIM3','ZKSCAN1','ZKSCAN2','ZKSCAN3','ZKSCAN4','ZKSCAN5','ZKSCAN7','ZKSCAN8','ZMAT1','ZMAT2','ZMAT3','ZMAT4','ZMAT5','ZMYM1','ZMYM2','ZMYM3','ZMYM4','ZMYM5','ZMYM6','ZMYND10','ZNF10','ZNF100','ZNF101','ZNF106','ZNF107','ZNF112','ZNF114','ZNF117','ZNF12','ZNF121','ZNF124','ZNF131','ZNF132','ZNF133','ZNF134','ZNF135','ZNF136','ZNF137P','ZNF138','ZNF14','ZNF140','ZNF141','ZNF142','ZNF143','ZNF146','ZNF148','ZNF154','ZNF155','ZNF157','ZNF16','ZNF160','ZNF165','ZNF169','ZNF17','ZNF174','ZNF175','ZNF177','ZNF18','ZNF180','ZNF181','ZNF182','ZNF184','ZNF185','ZNF189','ZNF19','ZNF195','ZNF197','ZNF2','ZNF20','ZNF200','ZNF202','ZNF205','ZNF207','ZNF208','ZNF211','ZNF212','ZNF213','ZNF214','ZNF215','ZNF217','ZNF219','ZNF22','ZNF221','ZNF222','ZNF223','ZNF224','ZNF225','ZNF226','ZNF227','ZNF229','ZNF23','ZNF230','ZNF232','ZNF233','ZNF234','ZNF235','ZNF236','ZNF239','ZNF24','ZNF248','ZNF25','ZNF250','ZNF251','ZNF253','ZNF254','ZNF256','ZNF257','ZNF26','ZNF260','ZNF263','ZNF264','ZNF266','ZNF267','ZNF268','ZNF273','ZNF274','ZNF275','ZNF276','ZNF277','ZNF28','ZNF280A','ZNF280B','ZNF280C','ZNF280D','ZNF281','ZNF282','ZNF283','ZNF284','ZNF285','ZNF286A','ZNF286B','ZNF287','ZNF292','ZNF296','ZNF3','ZNF30','ZNF300','ZNF302','ZNF304','ZNF311','ZNF316','ZNF317','ZNF318','ZNF319','ZNF32','ZNF320','ZNF322','ZNF324','ZNF324B','ZNF326','ZNF329','ZNF330','ZNF331','ZNF333','ZNF334','ZNF335','ZNF337','ZNF33A','ZNF33B','ZNF34','ZNF341','ZNF343','ZNF345','ZNF346','ZNF347','ZNF35','ZNF350','ZNF354A','ZNF354B','ZNF354C','ZNF355P','ZNF358','ZNF362','ZNF365','ZNF366','ZNF367','ZNF37A','ZNF382','ZNF383','ZNF384','ZNF385A','ZNF385B','ZNF385C','ZNF385D','ZNF391','ZNF394','ZNF395','ZNF396','ZNF397','ZNF398','ZNF404','ZNF407','ZNF408','ZNF41','ZNF410','ZNF414','ZNF415','ZNF416','ZNF417','ZNF418','ZNF419','ZNF420','ZNF423','ZNF425','ZNF426','ZNF428','ZNF429','ZNF43','ZNF430','ZNF431','ZNF432','ZNF433','ZNF436','ZNF438','ZNF439','ZNF44','ZNF440','ZNF441','ZNF442','ZNF443','ZNF444','ZNF445','ZNF446','ZNF449','ZNF45','ZNF451','ZNF454','ZNF460','ZNF461','ZNF462','ZNF467','ZNF468','ZNF469','ZNF470','ZNF471','ZNF473','ZNF474','ZNF479','ZNF48','ZNF480','ZNF483','ZNF484','ZNF485','ZNF486','ZNF487','ZNF488','ZNF490','ZNF491','ZNF492','ZNF493','ZNF496','ZNF497','ZNF500','ZNF501','ZNF502','ZNF503','ZNF506','ZNF507','ZNF510','ZNF511','ZNF512','ZNF512B','ZNF513','ZNF514','ZNF516','ZNF517','ZNF518A','ZNF518B','ZNF519']
TFs3Other=['ZNF521','ZNF524','ZNF525','ZNF526','ZNF527','ZNF528','ZNF529','ZNF530','ZNF532','ZNF534','ZNF536','ZNF540','ZNF541','ZNF542P','ZNF543','ZNF544','ZNF546','ZNF547','ZNF548','ZNF549','ZNF550','ZNF551','ZNF552','ZNF554','ZNF555','ZNF556','ZNF557','ZNF558','ZNF559','ZNF560','ZNF561','ZNF562','ZNF563','ZNF564','ZNF565','ZNF566','ZNF567','ZNF568','ZNF569','ZNF57','ZNF570','ZNF571','ZNF572','ZNF573','ZNF574','ZNF575','ZNF576','ZNF577','ZNF578','ZNF579','ZNF580','ZNF581','ZNF582','ZNF583','ZNF584','ZNF585A','ZNF585B','ZNF586','ZNF587','ZNF587B','ZNF589','ZNF592','ZNF593','ZNF594','ZNF595','ZNF596','ZNF597','ZNF598','ZNF599','ZNF600','ZNF605','ZNF606','ZNF607','ZNF608','ZNF609','ZNF610','ZNF611','ZNF613','ZNF614','ZNF615','ZNF616','ZNF618','ZNF619','ZNF620','ZNF621','ZNF622','ZNF623','ZNF624','ZNF625','ZNF626','ZNF627','ZNF628','ZNF629','ZNF630','ZNF638','ZNF639','ZNF641','ZNF644','ZNF645','ZNF646','ZNF648','ZNF649','ZNF652','ZNF653','ZNF654','ZNF655','ZNF658','ZNF66','ZNF660','ZNF662','ZNF664','ZNF665','ZNF667','ZNF668','ZNF669','ZNF670','ZNF671','ZNF672','ZNF674','ZNF675','ZNF676','ZNF677','ZNF678','ZNF679','ZNF680','ZNF681','ZNF682','ZNF683','ZNF684','ZNF687','ZNF688','ZNF689','ZNF69','ZNF691','ZNF692','ZNF695','ZNF696','ZNF697','ZNF699','ZNF7','ZNF70','ZNF700','ZNF701','ZNF703','ZNF704','ZNF705A','ZNF705B','ZNF705D','ZNF705E','ZNF705G','ZNF706','ZNF707','ZNF708','ZNF709','ZNF71','ZNF710','ZNF711','ZNF713','ZNF714','ZNF716','ZNF717','ZNF718','ZNF720','ZNF721','ZNF724','ZNF726','ZNF727','ZNF728','ZNF729','ZNF73','ZNF730','ZNF732','ZNF735','ZNF736','ZNF737','ZNF738','ZNF74','ZNF740','ZNF746','ZNF747','ZNF749','ZNF750','ZNF75A','ZNF75D','ZNF76','ZNF761','ZNF763','ZNF764','ZNF765','ZNF766','ZNF768','ZNF77','ZNF770','ZNF771','ZNF772','ZNF773','ZNF774','ZNF775','ZNF776','ZNF777','ZNF778','ZNF780A','ZNF780B','ZNF781','ZNF782','ZNF783','ZNF784','ZNF785','ZNF786','ZNF787','ZNF788','ZNF789','ZNF79','ZNF790','ZNF791','ZNF792','ZNF793','ZNF799','ZNF8','ZNF80','ZNF800','ZNF804A','ZNF804B','ZNF805','ZNF808','ZNF81','ZNF812P','ZNF813','ZNF814','ZNF816','ZNF821','ZNF823','ZNF827','ZNF829','ZNF83','ZNF830','ZNF831','ZNF835','ZNF836','ZNF837','ZNF839','ZNF84','ZNF841','ZNF843','ZNF844','ZNF845','ZNF846','ZNF85','ZNF850','ZNF852','ZNF853','ZNF860','ZNF862','ZNF865','ZNF878','ZNF879','ZNF880','ZNF883','ZNF888','ZNF891','ZNF90','ZNF91','ZNF92','ZNF93','ZNF98','ZNF99','ZNFX1','ZRANB2','ZRSR1','ZRSR2','ZSCAN1','ZSCAN10','ZSCAN12','ZSCAN16','ZSCAN18','ZSCAN2','ZSCAN20','ZSCAN21','ZSCAN22','ZSCAN23','ZSCAN25','ZSCAN26','ZSCAN29','ZSCAN30','ZSCAN31','ZSCAN32','ZSCAN4','ZSCAN5A','ZSCAN5B','ZSCAN5C','ZSCAN5DP','ZSCAN9','ZUFSP','ZXDA','ZXDB','ZXDC','ZZZ3']
TFs4Other=['ATF2','ATF6B','ATN1','BHLHA15','BHLHB2','BHLHB4 ','BHLHB5','BHLHB9','C13ORF8','C20ORF194','CDX2','CRAMP1L','ELP3','ERVK3-1','ETF1','FLJ12895','FLJ20321','FLJ25476','FLJ35784','FOXM1','FOXO2','FOXR1','FOXR2','GATA1','GRLF1','KDM4C','KDM4E','KIAA0863','KIAA0961','KLRG1','LBXCOR1','LEF1','LHX7','LOC126295','LOC51058','LOC91661','MGC4728','MYOD1','MYOG','MYRF','NEUROD4','NEUROG1','NFAT2','NFAT4','NFE2L1','NR1A2','NR1B1','NR1B2','NR1B3','NR1C1','NR1C2','NR1C3','NR1F3','NR1I1','NR2A1','NR2B3','NR3A1','NR3A2','NR3B3','OTEX','PEPP-2','PHF2','POU2F1','POU2F2','POU4F3','POU5F1','POU6F1','PRDM3','PRKRIP1','PRKRIR','SETD1A','SMAD8','SMYD2','SOX12','SOX14','SOX15','TBX14','TCF1','TCF17','TCF2','TCF9','TEAD1','TPRXL','UBTFL1','WHSC1','ZBTB14','ZBTB18','ZBTB19','ZBTB21','ZBTB27','ZBTB28','ZBTB35','ZBTB49','ZFP161','ZFP28','ZKSCAN10','ZKSCAN11','ZKSCAN12','ZKSCAN13','ZKSCAN14','ZKSCAN16','ZKSCAN19','ZKSCAN21','ZKSCAN24','ZKSCAN6','ZKSCAN8','ZNF10','ZNF112','ZNF187','ZNF193','ZNF204P','ZNF238','ZNF271','ZNF295','ZNF305','ZNF323','ZNF434','ZNF499','ZNF542','ZNF545','ZNF553','ZNF56','ZNF625','ZNF643','ZNF698','ZNF705F','ZNF751','ZNF752','ZNF753','ZNF785','ZNF806','ZNF816','ZNF833P','ZNF906','ZSCAN1','ZSCAN25','ZSCAN26','ZSCAN31','ZSCAN32','ZSCAN5D','ZSCAN9']
TFsOther=TFs1Other+TFs2Other+TFs3Other+TFs4Other
```

Example 11 (unknown):
```unknown
S genes: ['MCM5', 'PCNA', 'TYMS', 'FEN1', 'MCM2', 'MCM4', 'RRM1', 'UNG', 'GINS2', 'MCM6', 'CDCA7', 'DTL', 'PRIM1', 'UHRF1', 'HELLS', 'RFC2', 'RPA2', 'NASP', 'RAD51AP1', 'GMNN', 'WDR76', 'SLBP', 'CCNE2', 'UBR7', 'POLD3', 'MSH2', 'ATAD2', 'RAD51', 'RRM2', 'CDC45', 'CDC6', 'EXO1', 'TIPIN', 'DSCC1', 'BLM', 'CASP8AP2', 'USP1', 'CLSPN', 'POLA1', 'CHAF1B', 'BRIP1', 'E2F8'] 

G2 genes: ['HMGB2', 'CDK1', 'NUSAP1', 'UBE2C', 'BIRC5', 'TPX2', 'TOP2A', 'NDC80', 'CKS2', 'NUF2', 'CKS1B', 'MKI67', 'TMPO', 'CENPF', 'TACC3', 'SMC4', 'CCNB2', 'CKAP2L', 'CKAP2', 'AURKB', 'BUB1', 'KIF11', 'ANP32E', 'TUBB4B', 'GTSE1', 'KIF20B', 'HJURP', 'CDCA3', 'CDC20', 'TTK', 'CDC25C', 'KIF2C', 'RANGAP1', 'NCAPD2', 'DLGAP5', 'CDCA2', 'CDCA8', 'ECT2', 'KIF23', 'HMMR', 'AURKA', 'PSRC1', 'ANLN', 'LBR', 'CKAP5', 'CENPE', 'CTCF', 'NEK2', 'G2E3', 'GAS2L3', 'CBX5', 'CENPA'] 

Hemoglobin genes set 1: ['HBB', 'HBD', 'HBG1', 'HBG2', 'HBE1', 'HBZ', 'HBM', 'HBA2', 'HBA1', 'HBQ1'] 

Hemoglobin genes set 2: ['AHSP', 'GYPA', 'RHAG', 'GFI1B', 'KEL', 'EPB42', 'ALAS2', 'TSPO2', 'GYPB', 'GYPE', 'SPTA1', 'RHCE', 'KLF1', 'RHD', 'SLC4A1', 'SNX22', 'SQSTM1', 'SOD1', 'FTH1', 'MAP1LC3B', 'SLC25A37', 'TRIM58', 'HMBS'] 

Mitochondrial genes: ['MT-ND1', 'MT-ND2', 'MT-CO1', 'MT-CO2', 'MT-ATP8', 'MT-ATP6', 'MT-CO3', 'MT-ND3', 'MT-ND4L', 'MT-ND4', 'MT-ND5', 'MT-ND6', 'MT-CYB'] 

Ribo genes: ['RPLP0', 'RPLP1', 'RPLP2', 'RPL3', 'RPL3L', 'RPL4', 'RPL5', 'RPL6', 'RPL7', 'RPL7A', 'RPL7L1', 'RPL8', 'RPL9', 'RPL10', 'RPL10A', 'RPL10L', 'RPL11', 'RPL12', 'RPL13', 'RPL13A', 'RPL14', 'RPL15', 'RPL17', 'RPL18', 'RPL18A', 'RPL19', 'RPL21', 'RPL22', 'RPL22L1', 'RPL23', 'RPL23A', 'RPL24', 'RPL26', 'RPL26L1', 'RPL27', 'RPL27A', 'RPL28', 'RPL29', 'RPL30', 'RPL31', 'RPL32', 'RPL34', 'RPL35', 'RPL35A', 'RPL36', 'RPL36A', 'RPL36AL', 'RPL37', 'RPL37A', 'RPL38', 'RPL39', 'RPL39L', 'UBA52', 'RPL41', 'RPSA', 'RPS2', 'RPS3', 'RPS3A', 'RPS4X', 'RPS4Y1', 'RPS4Y2', 'RPS5', 'RPS6', 'RPS7', 'RPS8', 'RPS9', 'RPS10', 'RPS11', 'RPS12', 'RPS13', 'RPS14', 'RPS15', 'RPS15A', 'RPS16', 'RPS17', 'RPS18', 'RPS19', 'RPS20', 'RPS21', 'RPS23', 'RPS24', 'RPS25', 'RPS26', 'RPS27', 'RPS27A', 'RPS27L', 'RPS28', 'RPS29', 'FAU'] 

Tissue dissociation stress associated genes: ['FOS', 'FOSB', 'JUN', 'JUNB', 'JUND', 'ZFP36', 'SOCS3', 'CEBPD', 'CEBPB', 'ATF3', 'EGR1', 'HSPA1A', 'HSPA1B', 'HSP90AA1', 'HSP90AB1', 'HSPA8', 'HSPB1', 'HSPE1', 'IER3', 'IER2', 'BTG1', 'BTG2', 'DUSP1']
```

Example 12 (python):
```python
set(adata.obs.fine_grain)
```

Example 13 (unknown):
```unknown
{'AtrialCardiomyocytesCycling',
 'AtrialCardiomyocytesLeft',
 'AtrialCardiomyocytesRight',
 'AtrioventricularNodePacemakerCells',
 'BCells',
 'BCellsMS4A1pos',
 'ChromaffinCells',
 'CoronaryArterialEndothelialCells',
 'CoronaryCapillaryEndothelialCells',
 'CoronaryPericytes',
 'CoronarySmoothMuscleCells',
 'CoronaryVenousEndothelialCells',
 'CoronaryVesselAdventitialFibroblasts',
 'DendriticCellsMature',
 'DendriticCellsType1',
 'DuctusArteriosusSmoothMuscleCells',
 'EndocardialCells',
 'EndocardialCushionCells',
 'EpicardiumDerivedCells',
 'GreatVesselAdventitialFibroblasts',
 'GreatVesselArterialEndothelialCells',
 'GreatVesselSmoothMuscleCells',
 'GreatVesselVenousEndothelialCells',
 'InnateLymphoidCells',
 'LymphNodeFibroblasticReticularCells',
 'LymphaticEndothelialCells',
 'MacrophagesATF3pos',
 'MacrophagesCX3CR1pos',
 'MacrophagesLYVE1pos',
 'MacrophagesTIMD4pos',
 'MastCells',
 'Megakaryocytes',
 'MesothelialEpicardialCells',
 'MonocyteDerivedCells',
 'Monocytes',
 'MonocytesMPOpos',
 'MyocardialInterstitialFibroblasts',
 'Myofibroblasts',
 'NaturalKillerCells',
 'NeuronPrecursors',
 'ParasympatheticNeurons',
 'PericardialCellsFibrous',
 'PericardialCellsIntermediate',
 'PericardialCellsParietal',
 'PlasmacytoidDendriticCells',
 'ProBCells',
 'SchwannCellPrecursors',
 'SchwannCells',
 'SinoatrialNodePacemakerCells',
 'SubEpicardialFibroblasts',
 'SympatheticNeurons',
 'TCellsCD4pos',
 'TCellsCD8pos',
 'TregsCD4pos',
 'ValveEndothelialCells',
 'ValveInterstitialCells',
 'VentricularCardiomyocytesCycling',
 'VentricularCardiomyocytesLeftCompact',
 'VentricularCardiomyocytesLeftTrabeculated',
 'VentricularCardiomyocytesRightCompact',
 'VentricularCardiomyocytesRightTrabeculated',
 'VentricularConductionSystemDistal',
 'VentricularConductionSystemProximal'}
```

Example 14 (python):
```python
set(adata.obs.region)
```

Example 15 (unknown):
```unknown
{'aorta',
 'atria',
 'atrial septum',
 'ductus arteriosus',
 'left ventricle',
 'pericardium',
 'pulmonary arches and branches',
 'right ventricle',
 'right ventricle and papillaries'}
```

Example 16 (python):
```python
adata = adata[adata.obs['fine_grain'].isin(['GreatVesselSmoothMuscleCells','DuctusArteriosusSmoothMuscleCells','CoronarySmoothMuscleCells','CoronaryPericytes','CoronaryArterialEndothelialCells','CoronaryCapillaryEndothelialCells','CoronaryVenousEndothelialCells','GreatVesselArterialEndothelialCells','GreatVesselVenousEndothelialCells','GreatVesselAdventitialFibroblasts','CoronaryVesselAdventitialFibroblasts','Myofibroblasts','SubEpicardialFibroblasts','MyocardialInterstitialFibroblasts'])].copy()
set(adata.obs.fine_grain)
adata
```

Example 17 (python):
```python
# preserve raw count
adata.layers["counts"] = adata.X.copy()

# normalise and log-transform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key='batch_key')
#sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5)

print(sum(adata.var.highly_variable))
sc.pl.highly_variable_genes(adata)

# freeze the state in `.raw`
adata.raw = adata

# subset hvgs
adata = adata[:, adata.var.highly_variable]

adata=adata.copy()

sc.tl.score_genes_cell_cycle(adata, s_genes=s_genes, g2m_genes=g2m_genes)
#sc.pp.regress_out(adata, ['S_score','G2M_score'])

#batch correction
scvi.model.SCVI.setup_anndata(adata, 
                              layer="counts", 
                              batch_key="batch_key",
                              categorical_covariate_keys=['phase'],
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo'])

vae = scvi.model.SCVI(adata, n_hidden = 128, n_latent = 50, n_layers = 3, dispersion = 'gene-batch')
vae.train(max_epochs=50)
    
# Convergence plot
x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
plt.plot(x, vae.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
plt.clf()

# add the latent space to adata
adata.obsm["X_scVI"] = vae.get_latent_representation()

sc.tl.score_genes(adata, stress_genes, score_name='stress_score')
sc.tl.score_genes(adata, hb_genes1, score_name='hb1_score')

sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata,min_dist=0.1)
#sc.tl.tsne(adata)
sc.pl.umap(adata, color=['log1p_total_counts','n_genes','n_genes_by_counts','total_counts','pct_counts_mt','pct_counts_ribo'],ncols=5)
sc.pl.umap(adata, color=['scrublet_score','stress_score', 'hb1_score','phase'], ncols=5,)
sc.pl.umap(adata, color=['donor','age','week','trimester','region','phase','cycling','cell_or_nuclei','modality','kit_10x','FACSgate','heart_or_greatvessels','coarse_grain','mid_grain','fine_grain'],wspace=0.8,ncols=5)
```

Example 18 (python):
```python
normalizing counts per cell
    finished (0:00:01)
extracting highly variable genes
    finished (0:00:16)
--> added
    'highly_variable', boolean vector (adata.var)
    'means', float vector (adata.var)
    'dispersions', float vector (adata.var)
    'dispersions_norm', float vector (adata.var)
2520
```

Example 19 (python):
```python
calculating cell cycle phase
computing score 'S_score'
WARNING: genes are not in var_names and ignored: ['MCM5', 'PCNA', 'FEN1', 'MCM2', 'MCM4', 'RRM1', 'UNG', 'GINS2', 'MCM6', 'CDCA7', 'PRIM1', 'RFC2', 'RPA2', 'NASP', 'GMNN', 'SLBP', 'CCNE2', 'UBR7', 'POLD3', 'MSH2', 'RAD51', 'CDC45', 'CDC6', 'EXO1', 'TIPIN', 'DSCC1', 'BLM', 'CASP8AP2', 'USP1', 'POLA1', 'CHAF1B', 'E2F8']
    finished: added
    'S_score', score of gene set (adata.obs).
    288 total control genes are used. (0:00:01)
computing score 'G2M_score'
WARNING: genes are not in var_names and ignored: ['CKS1B', 'TMPO', 'ANP32E', 'RANGAP1', 'NCAPD2', 'LBR', 'CKAP5', 'CTCF', 'G2E3', 'CBX5']
    finished: added
    'G2M_score', score of gene set (adata.obs).
    572 total control genes are used. (0:00:01)
-->     'phase', cell cycle phase (adata.obs)
```

Example 20 (unknown):
```unknown
GPU available: False, used: False
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
```

Example 21 (unknown):
```unknown
Epoch 50/50: 100%|███████████| 50/50 [11:57<00:00, 14.35s/it, loss=921, v_num=1]
```

Example 22 (python):
```python
computing score 'stress_score'
WARNING: genes are not in var_names and ignored: ['JUND', 'CEBPB', 'HSP90AA1', 'HSP90AB1', 'HSPA8', 'HSPB1', 'HSPE1', 'BTG1']
    finished: added
    'stress_score', score of gene set (adata.obs).
    342 total control genes are used. (0:00:01)
computing score 'hb1_score'
WARNING: genes are not in var_names and ignored: ['HBD', 'HBM', 'HBQ1']
    finished: added
    'hb1_score', score of gene set (adata.obs).
    348 total control genes are used. (0:00:01)
computing neighbors
    finished: added to `.uns['neighbors']`
    `.obsp['distances']`, distances for each pair of neighbors
    `.obsp['connectivities']`, weighted adjacency matrix (0:00:13)
computing UMAP
    finished: added
    'X_umap', UMAP coordinates (adata.obsm) (0:00:45)
```

Example 23 (unknown):
```unknown
... storing 'phase' as categorical
```

Example 24 (unknown):
```unknown
<Figure size 320x320 with 0 Axes>
```

Example 25 (python):
```python
#adata.write('./25MayAllECSMCandFib.h5ad')
```

Example 26 (python):
```python
adata=sc.read('./Objects/Feb28ObjectRaw.h5ad')
```

Example 27 (python):
```python
adata = adata[adata.obs['fine_grain'].isin(['GreatVesselSmoothMuscleCells','DuctusArteriosusSmoothMuscleCells','CoronarySmoothMuscleCells','CoronaryPericytes'])].copy()
set(adata.obs.fine_grain)
adata



# preserve raw count
adata.layers["counts"] = adata.X.copy()

# normalise and log-transform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key='batch_key')
#sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5)

print(sum(adata.var.highly_variable))
sc.pl.highly_variable_genes(adata)

# freeze the state in `.raw`
adata.raw = adata

# subset hvgs
adata = adata[:, adata.var.highly_variable]

adata=adata.copy()

sc.tl.score_genes_cell_cycle(adata, s_genes=s_genes, g2m_genes=g2m_genes)
#sc.pp.regress_out(adata, ['S_score','G2M_score'])

#batch correction
scvi.model.SCVI.setup_anndata(adata, 
                              layer="counts", 
                              batch_key="batch_key",
                              categorical_covariate_keys=['phase'],
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo'])

vae = scvi.model.SCVI(adata, n_hidden = 128, n_latent = 50, n_layers = 3, dispersion = 'gene-batch')
vae.train(max_epochs=50)
    
# Convergence plot
x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
plt.plot(x, vae.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
plt.clf()

# add the latent space to adata
adata.obsm["X_scVI"] = vae.get_latent_representation()

sc.tl.score_genes(adata, stress_genes, score_name='stress_score')
sc.tl.score_genes(adata, hb_genes1, score_name='hb1_score')

sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata,min_dist=0.1)
#sc.tl.tsne(adata)
sc.pl.umap(adata, color=['log1p_total_counts','n_genes','n_genes_by_counts','total_counts','pct_counts_mt','pct_counts_ribo'],ncols=5)
sc.pl.umap(adata, color=['scrublet_score','stress_score', 'hb1_score','phase'], ncols=5,)
sc.pl.umap(adata, color=['donor','age','week','trimester','region','phase','cycling','cell_or_nuclei','modality','kit_10x','FACSgate','heart_or_greatvessels','coarse_grain','mid_grain','fine_grain'],wspace=0.8,ncols=5)
```

Example 28 (python):
```python
normalizing counts per cell
    finished (0:00:00)
extracting highly variable genes
    finished (0:00:10)
--> added
    'highly_variable', boolean vector (adata.var)
    'means', float vector (adata.var)
    'dispersions', float vector (adata.var)
    'dispersions_norm', float vector (adata.var)
1505
```

Example 29 (python):
```python
calculating cell cycle phase
computing score 'S_score'
WARNING: genes are not in var_names and ignored: ['MCM5', 'FEN1', 'RRM1', 'UNG', 'GINS2', 'MCM6', 'CDCA7', 'PRIM1', 'RFC2', 'RPA2', 'NASP', 'RAD51AP1', 'GMNN', 'WDR76', 'SLBP', 'CCNE2', 'UBR7', 'POLD3', 'MSH2', 'RAD51', 'CDC45', 'CDC6', 'EXO1', 'TIPIN', 'DSCC1', 'BLM', 'CASP8AP2', 'USP1', 'POLA1', 'CHAF1B', 'E2F8']
    finished: added
    'S_score', score of gene set (adata.obs).
    328 total control genes are used. (0:00:00)
computing score 'G2M_score'
WARNING: genes are not in var_names and ignored: ['ANP32E', 'CDC25C', 'RANGAP1', 'LBR', 'CKAP5', 'CTCF', 'NEK2', 'G2E3', 'CBX5']
    finished: added
    'G2M_score', score of gene set (adata.obs).
    598 total control genes are used. (0:00:00)
-->     'phase', cell cycle phase (adata.obs)
```

Example 30 (unknown):
```unknown
GPU available: False, used: False
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
```

Example 31 (unknown):
```unknown
Epoch 15/50:  28%|███        | 14/50 [00:35<01:31,  2.54s/it, loss=710, v_num=1]
```

Example 32 (python):
```python
computing score 'stress_score'
WARNING: genes are not in var_names and ignored: ['JUND', 'CEBPB', 'HSP90AA1', 'HSP90AB1', 'HSPA8', 'HSPB1', 'HSPE1', 'BTG1']
    finished: added
    'stress_score', score of gene set (adata.obs).
    341 total control genes are used. (0:00:00)
computing score 'hb1_score'
WARNING: genes are not in var_names and ignored: ['HBD', 'HBE1', 'HBM', 'HBQ1']
    finished: added
    'hb1_score', score of gene set (adata.obs).
    245 total control genes are used. (0:00:00)
computing neighbors
    finished: added to `.uns['neighbors']`
    `.obsp['distances']`, distances for each pair of neighbors
    `.obsp['connectivities']`, weighted adjacency matrix (0:00:03)
computing UMAP
```

Example 33 (python):
```python
---------------------------------------------------------------------------
KeyboardInterrupt                         Traceback (most recent call last)
/tmp/ipykernel_61186/1485025011.py in <module>
     54 
     55 sc.pp.neighbors(adata, use_rep="X_scVI")
---> 56 sc.tl.umap(adata,min_dist=0.1)
     57 #sc.tl.tsne(adata)
     58 sc.pl.umap(adata, color=['log1p_total_counts','n_genes','n_genes_by_counts','total_counts','pct_counts_mt','pct_counts_ribo'],ncols=5)

~/miniconda3/envs/scanpynew/lib/python3.8/site-packages/scanpy/tools/_umap.py in umap(adata, min_dist, spread, n_components, maxiter, alpha, gamma, negative_sample_rate, init_pos, random_state, a, b, copy, method, neighbors_key)
    191         # for the init condition in the UMAP embedding
    192         n_epochs = 0 if maxiter is None else maxiter
--> 193         X_umap = simplicial_set_embedding(
    194             X,
    195             neighbors['connectivities'].tocoo(),

~/miniconda3/envs/scanpynew/lib/python3.8/site-packages/umap/umap_.py in simplicial_set_embedding(data, graph, n_components, initial_alpha, a, b, gamma, negative_sample_rate, n_epochs, init, random_state, metric, metric_kwds, verbose)
    954     elif isinstance(init, str) and init == "spectral":
    955         # We add a little noise to avoid local minima for optimization to come
--> 956         initialisation = spectral_layout(
    957             data,
    958             graph,

~/miniconda3/envs/scanpynew/lib/python3.8/site-packages/umap/spectral.py in spectral_layout(data, graph, dim, random_state, metric, metric_kwds)
    256     try:
    257         if L.shape[0] < 2000000:
--> 258             eigenvalues, eigenvectors = scipy.sparse.linalg.eigsh(
    259                 L,
    260                 k,

~/miniconda3/envs/scanpynew/lib/python3.8/site-packages/scipy/sparse/linalg/eigen/arpack/arpack.py in eigsh(A, k, M, sigma, which, v0, ncv, maxiter, tol, return_eigenvectors, Minv, OPinv, mode)
   1688     with _ARPACK_LOCK:
   1689         while not params.converged:
-> 1690             params.iterate()
   1691 
   1692         return params.extract(return_eigenvectors)

~/miniconda3/envs/scanpynew/lib/python3.8/site-packages/scipy/sparse/linalg/eigen/arpack/arpack.py in iterate(self)
    534     def iterate(self):
    535         self.ido, self.tol, self.resid, self.v, self.iparam, self.ipntr, self.info = \
--> 536             self._arpack_solver(self.ido, self.bmat, self.which, self.k,
    537                                 self.tol, self.resid, self.v, self.iparam,
    538                                 self.ipntr, self.workd, self.workl, self.info)

KeyboardInterrupt:
```

Example 34 (unknown):
```unknown
<Figure size 600x600 with 0 Axes>
```

Example 35 (python):
```python
gene = 'PDE1C'

key_1 = 'week'
key_2 = 'fine_grain'
sc.settings.set_figure_params(dpi=80, facecolor='white')
plt.rcParams['figure.figsize'] =4,3  # width, height

# subset anndata of a gene of interest
ad = adata[:,list(adata.var_names).index(gene)]
# add gene expression to .obs
ad.obs[gene] = ad.to_df()
# groupby two keys
# ths is mean expression
df = ad.obs[[key_1,key_2,gene]]

sns.lineplot(x=key_1, y=gene,
             hue=key_2,
             data=df, err_style='band').legend(loc='center left',bbox_to_anchor=(1.0,0.5))
plt.xticks(rotation=45)
plt.ylabel("PDE1C expression")
plt.savefig('PDE1CExpression.pdf')
```

Example 36 (unknown):
```unknown
Trying to set attribute `.obs` of view, copying.
```

Example 37 (python):
```python
sc.settings.set_figure_params(dpi=150, color_map='viridis', vector_friendly=False,  dpi_save=300, facecolor='white')
```

Example 38 (python):
```python
sc.pl.dotplot(adata, ['ROBO2','DCLK2','PDE1C','TAFA1','CSMD1','PTGER4','PTGER3','PDE3A','PDE4D','PDE5A','EDNRA','EDNRB'],groupby='fine_grain',standard_scale='var',cmap='YlGn')
```

Example 39 (python):
```python
sc.pl.dotplot(adata, ['ROBO2','DCLK2','PDE1C','TAFA1','CSMD1','PTGER4','EDNRA','EDNRB','PDE3A','PDE4D','PDE5A'],groupby='fine_grain',standard_scale='var',cmap='YlGn')
```

Example 40 (python):
```python
FineGrainCategoriesOrder=['GreatVesselSmoothMuscleCells','DuctusArteriosusSmoothMuscleCells','GreatVesselArterialEndothelialCells','GreatVesselVenousEndothelialCells','GreatVesselAdventitialFibroblasts','CoronarySmoothMuscleCells','CoronaryPericytes','CoronaryArterialEndothelialCells','CoronaryCapillaryEndothelialCells','CoronaryVenousEndothelialCells','CoronaryVesselAdventitialFibroblasts','SubEpicardialFibroblasts','MyocardialInterstitialFibroblasts','Myofibroblasts']
sc.pl.dotplot(adata, ['NLGN1','FOS','FOSB','PCDH7','COLEC12','FN1','FBN1','ELN','FBLN5','HSPG2','DCN','AGRN','TNC'], var_group_rotation=30,groupby='fine_grain',categories_order=FineGrainCategoriesOrder,standard_scale='var',cmap='YlGn',var_group_positions=[(0,4),(5,12)], var_group_labels=['great vessel enriched','extracellular matrix'])
```

Example 41 (python):
```python
FineGrainCategoriesOrder=['GreatVesselSmoothMuscleCells','DuctusArteriosusSmoothMuscleCells','GreatVesselArterialEndothelialCells','GreatVesselVenousEndothelialCells','GreatVesselAdventitialFibroblasts','CoronarySmoothMuscleCells','CoronaryPericytes','CoronaryArterialEndothelialCells','CoronaryCapillaryEndothelialCells','CoronaryVenousEndothelialCells','CoronaryVesselAdventitialFibroblasts','SubEpicardialFibroblasts','MyocardialInterstitialFibroblasts','Myofibroblasts']
sc.pl.dotplot(adata, ['NLGN1','FOS','FOSB','PCDH7','COLEC12','FN1','FBN1','ELN','FBLN5','HSPG2','DCN','AGRN','TNC'], var_group_rotation=30,groupby='fine_grain',categories_order=FineGrainCategoriesOrder,standard_scale='var',cmap='YlGn',var_group_positions=[(0,4),(5,12)], var_group_labels=['great vessel enriched','extracellular matrix'])
```

Example 42 (python):
```python
adata = adata[adata.obs['fine_grain'].isin(['GreatVesselSmoothMuscleCells'])].copy()
```

Example 43 (python):
```python
sc.pl.umap(adata, color='fine_grain',palette=FineGrainPalette)
sc.pl.umap(adata, color='region',groups=['apex','left ventricle','right ventricle','right ventricle and papillaries'])
sc.pl.umap(adata, color='region',groups=['pulmonary arches and branches'])
```

Example 44 (python):
```python
#Pulmonary Vessel Smooth Muscle Cells
sc.tl.leiden(adata, resolution=0.3, key_added='leiden')
sc.pl.umap(adata, color='leiden',legend_loc='on data')
sc.pl.umap(adata, color='leiden',legend_loc='on data',groups='2')
```

Example 45 (python):
```python
running Leiden clustering
    finished: found 7 clusters and added
    'leiden', the cluster labels (adata.obs, categorical) (0:00:02)
```

Example 46 (python):
```python
adata.obs['fine_grain2']= 'na'
adata.obs['fine_grain2'][[x for x in adata.obs_names if adata.obs.fine_grain[x]=='DuctusArteriosusSmoothMuscleCells']]='DuctusArteriosusSmoothMuscleCells'
adata.obs['fine_grain2'][[x for x in adata.obs_names if adata.obs.fine_grain[x]=='CoronarySmoothMuscleCells']]='CoronarySmoothMuscleCells'
adata.obs['fine_grain2'][[x for x in adata.obs_names if adata.obs.fine_grain[x]=='CoronaryPericytes']]='CoronaryPericytes'
adata.obs['fine_grain2'][[x for x in adata.obs_names if adata.obs.fine_grain[x]=='GreatVesselSmoothMuscleCells']]='OtherGreatVesselSmoothMuscleCells'
adata.obs['fine_grain2'][[x for x in adata.obs_names if adata.obs.leiden[x] in ['2']]]='PulmonaryArterySmoothMuscleCells'
sc.pl.umap(adata, color='fine_grain2')
```

Example 47 (unknown):
```unknown
... storing 'fine_grain2' as categorical
```

Example 48 (python):
```python
CategoriesOrder=['PulmonaryArterySmoothMuscleCells','OtherGreatVesselSmoothMuscleCells','DuctusArteriosusSmoothMuscleCells','CoronarySmoothMuscleCells','CoronaryPericytes']
sc.pl.dotplot(adata, ['MYH11','ARL15','LAMA2','INPP4B','PXDNL','ALDH1A2','ESRRG','HPSE2','CACNA2D3'] ,groupby='fine_grain2',categories_order=CategoriesOrder,standard_scale='var',cmap='YlGn')
sc.pl.umap(adata, color='BMPR2')
```

Example 49 (python):
```python
adata.obs['Pulmonary']= 'OtherMuralCells'
adata.obs['Pulmonary'][[x for x in adata.obs_names if adata.obs.leiden[x] in ['2']]]='PulmonaryVesselSmoothMuscleCells'
#Final pulmonary genes
sc.pl.umap(adata, color='Pulmonary')
sc.pl.umap(adata, color='region',groups=['pulmonary arches and branches'])
sc.pl.dotplot(adata, ['PXDNL','ALDH1A2','ESRRG','HPSE2','CACNA2D3','ARL15','INPP4B','LAMA2','BMPR2'] ,groupby='Pulmonary',standard_scale='var',cmap='YlGn')
```

Example 50 (unknown):
```unknown
... storing 'Pulmonary' as categorical
```

Example 51 (python):
```python
ClusterOfInterest='CoronarySmoothMuscleCells'
sc.tl.rank_genes_groups(adata, 'fine_grain2', method='t-test', key_added = "t-test")
DEGenesForCluster = [x[ClusterOfInterest] for x in adata.uns['t-test']['names'][:50]]
sc.pl.dotplot(adata, DEGenesForCluster ,groupby='fine_grain2',standard_scale='var',cmap='PRGn')
#sc.pl.umap(adata, color=DEGenesForCluster)
```

Example 52 (python):
```python
ranking genes
    finished: added to `.uns['t-test']`
    'names', sorted np.recarray to be indexed by group ids
    'scores', sorted np.recarray to be indexed by group ids
    'logfoldchanges', sorted np.recarray to be indexed by group ids
    'pvals', sorted np.recarray to be indexed by group ids
    'pvals_adj', sorted np.recarray to be indexed by group ids (0:00:01)
```

Example 53 (python):
```python
bdata=adata[adata.obs.fine_grain2=='PulmonaryArterySmoothMuscleCells',:].copy()
bdatanames=bdata.obs_names

np.savetxt('./PulmonarySMCRelatedCellIndex/PulmonaryArterySmoothMuscleCells.txt', bdatanames, delimiter="\n", fmt="%s")
```

Example 54 (python):
```python
adata=sc.read('./Objects/Feb28ObjectRaw.h5ad')
adata = adata[adata.obs['fine_grain'].isin(['GreatVesselSmoothMuscleCells','DuctusArteriosusSmoothMuscleCells','CoronarySmoothMuscleCells','CoronaryPericytes'])].copy()
adata = adata[adata.obs['donor'].isin(['BRC2253','BRC2256','BRC2260','BRC2262','BRC2251','BRC2263'])].copy()
print(adata.shape)

# preserve raw count
adata.layers["counts"] = adata.X.copy()

# normalise and log-transform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key='batch_key')

print(sum(adata.var.highly_variable))
sc.pl.highly_variable_genes(adata)

# freeze the state in `.raw`
adata.raw = adata

# subset hvgs
adata = adata[:, adata.var.highly_variable]

adata=adata.copy()

sc.tl.score_genes_cell_cycle(adata, s_genes=s_genes, g2m_genes=g2m_genes)
#sc.pp.regress_out(adata, ['S_score','G2M_score'])

#batch correction
scvi.model.SCVI.setup_anndata(adata, 
                              layer="counts", 
                              batch_key="batch_key",
                              categorical_covariate_keys=['phase'],
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo'])

vae = scvi.model.SCVI(adata, n_hidden = 128, n_latent = 50, n_layers = 3, dispersion = 'gene-batch')
vae.train(max_epochs=50)
    
# Convergence plot
x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
plt.plot(x, vae.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
plt.clf()

# add the latent space to adata
adata.obsm["X_scVI"] = vae.get_latent_representation()

sc.tl.score_genes(adata, stress_genes, score_name='stress_score')
sc.tl.score_genes(adata, hb_genes1, score_name='hb1_score')

sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata,min_dist=0.2)
#sc.tl.tsne(adata)
sc.pl.umap(adata, color=['log1p_total_counts','n_genes','n_genes_by_counts','total_counts','pct_counts_mt','pct_counts_ribo'],ncols=5)
sc.pl.umap(adata, color=['scrublet_score','stress_score', 'hb1_score','phase'], ncols=5,)
sc.pl.umap(adata, color=['donor','age','week','trimester','region','phase','cycling','cell_or_nuclei','modality','kit_10x','FACSgate','heart_or_greatvessels','coarse_grain','mid_grain','fine_grain'],wspace=0.8,ncols=5)
sc.pl.umap(adata, color='region')
```

Example 55 (python):
```python
(5989, 36601)
normalizing counts per cell
    finished (0:00:00)
extracting highly variable genes
    finished (0:00:02)
--> added
    'highly_variable', boolean vector (adata.var)
    'means', float vector (adata.var)
    'dispersions', float vector (adata.var)
    'dispersions_norm', float vector (adata.var)
3391
```

Example 56 (python):
```python
calculating cell cycle phase
computing score 'S_score'
WARNING: genes are not in var_names and ignored: ['MCM2', 'RRM1', 'UNG', 'PRIM1', 'RFC2', 'RPA2', 'GMNN', 'WDR76', 'SLBP', 'UBR7', 'CDC45', 'CDC6', 'EXO1', 'TIPIN', 'BLM', 'CASP8AP2', 'USP1', 'POLA1', 'CHAF1B', 'BRIP1', 'E2F8']
    finished: added
    'S_score', score of gene set (adata.obs).
    579 total control genes are used. (0:00:00)
computing score 'G2M_score'
WARNING: genes are not in var_names and ignored: ['CDC25C', 'CDCA2', 'LBR', 'CTCF', 'CBX5']
    finished: added
    'G2M_score', score of gene set (adata.obs).
    658 total control genes are used. (0:00:00)
-->     'phase', cell cycle phase (adata.obs)
```

Example 57 (unknown):
```unknown
GPU available: False, used: False
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
```

Example 58 (unknown):
```unknown
Epoch 50/50: 100%|██████| 50/50 [00:36<00:00,  1.36it/s, loss=1.38e+03, v_num=1]
```

Example 59 (python):
```python
computing score 'stress_score'
WARNING: genes are not in var_names and ignored: ['JUND', 'HSP90AA1', 'HSP90AB1', 'HSPA8', 'HSPB1', 'HSPE1']
    finished: added
    'stress_score', score of gene set (adata.obs).
    495 total control genes are used. (0:00:00)
computing score 'hb1_score'
WARNING: genes are not in var_names and ignored: ['HBD', 'HBM', 'HBQ1']
    finished: added
    'hb1_score', score of gene set (adata.obs).
    248 total control genes are used. (0:00:00)
computing neighbors
    finished: added to `.uns['neighbors']`
    `.obsp['distances']`, distances for each pair of neighbors
    `.obsp['connectivities']`, weighted adjacency matrix (0:00:01)
computing UMAP
    finished: added
    'X_umap', UMAP coordinates (adata.obsm) (0:00:04)
```

Example 60 (unknown):
```unknown
... storing 'phase' as categorical
```

Example 61 (unknown):
```unknown
<Figure size 320x320 with 0 Axes>
```

Example 62 (python):
```python
sc.tl.leiden(adata, resolution=0.2, key_added='leiden')
sc.pl.umap(adata, color='leiden',legend_loc='on data')
```

Example 63 (python):
```python
running Leiden clustering
    finished: found 5 clusters and added
    'leiden', the cluster labels (adata.obs, categorical) (0:00:00)
```

Example 64 (python):
```python
sc.pl.umap(adata, color='fine_grain',palette=FineGrainPalette)
sc.pl.umap(adata, color='region',groups=['aorta','base','pulmonary arches and branches','atria','atrial septum','node','outflow tract'])
sc.pl.umap(adata, color='region',groups=['pulmonary arches and branches'])
```

Example 65 (python):
```python
sc.pl.umap(adata, color='SMCRegions',groups='DuctusArteriosusRegion')
OtherRegions=['aorta','atria','atrial septum','base','heart without node','node','outflow tract','pericardium','pulmonary arches and branches','whole sample']
ApexAssociatedRegions=['apex','left ventricle','right ventricle','right ventricle and papillaries']
DuctusArteriosusRegion=['ductus arteriosus']

adata.obs['SMCRegions']= 'na'
adata.obs['SMCRegions'][[x for x in adata.obs_names if adata.obs.region[x] in OtherRegions]]='OtherRegions'
adata.obs['SMCRegions'][[x for x in adata.obs_names if adata.obs.region[x] in ApexAssociatedRegions]]='ApexAssociatedRegions'
adata.obs['SMCRegions'][[x for x in adata.obs_names if adata.obs.region[x] in DuctusArteriosusRegion]]='DuctusArteriosusRegion'
sc.pl.umap(adata, color='SMCRegions')
```

Example 66 (python):
```python
adata = adata[adata.obs['week'].isin(['20W'])].copy()
adata = adata[adata.obs['region'].isin(['pulmonary arches and branches','aorta'])].copy()
adata = adata[adata.obs['fine_grain'].isin(['GreatVesselSmoothMuscleCells','GreatVesselAdventitialFibroblasts','GreatVesselArterialEndothelialCells','GreatVesselVenousEndothelialCells'])].copy()
set(adata.obs.fine_grain)
adata
```

Example 67 (unknown):
```unknown
AnnData object with n_obs × n_vars = 22161 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score'
    var: 'gene_name_scRNA-0', 'gene_id'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'age_colors', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'cycling_colors', 'donor_colors', 'fine_grain_colors', 'heart_or_greatvessels_colors', 'hvg', 'kit_10x_colors', 'mid_grain_colors', 'modality_colors', 'neighbors', 'phase_colors', 'region_colors', 'sex_colors', 'trimester_colors', 'umap', 'week_colors'
    obsm: 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 68 (python):
```python
set(adata.obs.fine_grain)
```

Example 69 (unknown):
```unknown
{'GreatVesselAdventitialFibroblasts',
 'GreatVesselArterialEndothelialCells',
 'GreatVesselSmoothMuscleCells',
 'GreatVesselVenousEndothelialCells'}
```

Example 70 (python):
```python
adata = adata[adata.obs['fine_grain'].isin(['GreatVesselArterialEndothelialCells','GreatVesselVenousEndothelialCells'])].copy()
adata
```

Example 71 (unknown):
```unknown
AnnData object with n_obs × n_vars = 2977 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score'
    var: 'gene_name_scRNA-0', 'gene_id'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'age_colors', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'cycling_colors', 'donor_colors', 'fine_grain_colors', 'heart_or_greatvessels_colors', 'hvg', 'kit_10x_colors', 'mid_grain_colors', 'modality_colors', 'neighbors', 'phase_colors', 'region_colors', 'sex_colors', 'trimester_colors', 'umap', 'week_colors'
    obsm: 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 72 (python):
```python
# preserve raw count
adata.layers["counts"] = adata.X.copy()

# normalise and log-transform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key='batch_key')
#sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5)

print(sum(adata.var.highly_variable))
sc.pl.highly_variable_genes(adata)

# freeze the state in `.raw`
adata.raw = adata

# subset hvgs
adata = adata[:, adata.var.highly_variable]

adata=adata.copy()

sc.tl.score_genes_cell_cycle(adata, s_genes=s_genes, g2m_genes=g2m_genes)
#sc.pp.regress_out(adata, ['S_score','G2M_score'])

#batch correction
scvi.model.SCVI.setup_anndata(adata, 
                              layer="counts", 
                              batch_key="batch_key",
                              categorical_covariate_keys=['phase'],
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo'])

vae = scvi.model.SCVI(adata, n_hidden = 128, n_latent = 50, n_layers = 3, dispersion = 'gene-batch')
vae.train(max_epochs=50)
    
# Convergence plot
x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
plt.plot(x, vae.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
plt.clf()

# add the latent space to adata
adata.obsm["X_scVI"] = vae.get_latent_representation()

sc.tl.score_genes(adata, stress_genes, score_name='stress_score')
sc.tl.score_genes(adata, hb_genes1, score_name='hb1_score')

sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata,min_dist=0.1)
#sc.tl.tsne(adata)
sc.pl.umap(adata, color=['log1p_total_counts','n_genes','n_genes_by_counts','total_counts','pct_counts_mt','pct_counts_ribo'],ncols=5)
sc.pl.umap(adata, color=['scrublet_score','stress_score', 'hb1_score','phase'], ncols=5,)
sc.pl.umap(adata, color=['donor','age','week','trimester','region','phase','cycling','cell_or_nuclei','modality','kit_10x','FACSgate','heart_or_greatvessels','coarse_grain','mid_grain','fine_grain'],wspace=0.8,ncols=5)
```

Example 73 (python):
```python
normalizing counts per cell
    finished (0:00:00)
extracting highly variable genes
    finished (0:00:06)
--> added
    'highly_variable', boolean vector (adata.var)
    'means', float vector (adata.var)
    'dispersions', float vector (adata.var)
    'dispersions_norm', float vector (adata.var)
915
```

Example 74 (python):
```python
calculating cell cycle phase
computing score 'S_score'
WARNING: genes are not in var_names and ignored: ['FEN1', 'MCM2', 'MCM4', 'RRM1', 'MCM6', 'CDCA7', 'DTL', 'PRIM1', 'UHRF1', 'RFC2', 'RPA2', 'NASP', 'RAD51AP1', 'GMNN', 'WDR76', 'SLBP', 'CCNE2', 'UBR7', 'POLD3', 'MSH2', 'RAD51', 'CDC45', 'EXO1', 'TIPIN', 'DSCC1', 'BLM', 'CASP8AP2', 'USP1', 'POLA1', 'CHAF1B', 'BRIP1', 'E2F8']
    finished: added
    'S_score', score of gene set (adata.obs).
    293 total control genes are used. (0:00:00)
computing score 'G2M_score'
WARNING: genes are not in var_names and ignored: ['CKS1B', 'TMPO', 'TACC3', 'CKAP2L', 'AURKB', 'BUB1', 'KIF11', 'ANP32E', 'HJURP', 'CDCA3', 'TTK', 'CDC25C', 'KIF2C', 'RANGAP1', 'CDCA2', 'CDCA8', 'AURKA', 'PSRC1', 'LBR', 'CKAP5', 'CTCF', 'NEK2', 'G2E3', 'GAS2L3', 'CBX5', 'CENPA']
    finished: added
    'G2M_score', score of gene set (adata.obs).
    392 total control genes are used. (0:00:00)
-->     'phase', cell cycle phase (adata.obs)
```

Example 75 (unknown):
```unknown
GPU available: False, used: False
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
```

Example 76 (unknown):
```unknown
Epoch 50/50: 100%|███████████| 50/50 [00:10<00:00,  4.87it/s, loss=655, v_num=1]
```

Example 77 (python):
```python
computing score 'stress_score'
WARNING: genes are not in var_names and ignored: ['JUND', 'CEBPB', 'HSP90AA1', 'HSP90AB1', 'HSPA8', 'HSPB1', 'HSPE1', 'BTG1', 'BTG2']
    finished: added
    'stress_score', score of gene set (adata.obs).
    252 total control genes are used. (0:00:00)
computing score 'hb1_score'
WARNING: genes are not in var_names and ignored: ['HBB', 'HBD', 'HBE1', 'HBM', 'HBQ1']
    finished: added
    'hb1_score', score of gene set (adata.obs).
    185 total control genes are used. (0:00:00)
computing neighbors
    finished: added to `.uns['neighbors']`
    `.obsp['distances']`, distances for each pair of neighbors
    `.obsp['connectivities']`, weighted adjacency matrix (0:00:00)
computing UMAP
    finished: added
    'X_umap', UMAP coordinates (adata.obsm) (0:00:01)
```

Example 78 (unknown):
```unknown
... storing 'phase' as categorical
```

Example 79 (unknown):
```unknown
<Figure size 320x320 with 0 Axes>
```

Example 80 (python):
```python
sc.tl.leiden(adata, resolution=0.1, key_added='leiden')
sc.pl.umap(adata, color='leiden',legend_loc='on data')
```

Example 81 (python):
```python
running Leiden clustering
    finished: found 3 clusters and added
    'leiden', the cluster labels (adata.obs, categorical) (0:00:00)
```

Example 82 (python):
```python
ClusterOfInterest='0'
sc.tl.rank_genes_groups(adata, 'leiden', method='t-test', key_added = "t-test")
DEGenesForCluster = [x[ClusterOfInterest] for x in adata.uns['t-test']['names'][:50]]
sc.pl.dotplot(adata, DEGenesForCluster ,groupby='leiden',standard_scale='var',cmap='PRGn')
#sc.pl.umap(adata, color=DEGenesForCluster)
```

Example 83 (python):
```python
ranking genes
    finished: added to `.uns['t-test']`
    'names', sorted np.recarray to be indexed by group ids
    'scores', sorted np.recarray to be indexed by group ids
    'logfoldchanges', sorted np.recarray to be indexed by group ids
    'pvals', sorted np.recarray to be indexed by group ids
    'pvals_adj', sorted np.recarray to be indexed by group ids (0:00:00)
```

Example 84 (python):
```python
adata.obs['type']= 'na'
adata.obs['type'][[x for x in adata.obs_names if adata.obs.fine_grain[x] in ['GreatVesselAdventitialFibroblasts']]]='Fibroblasts'
adata.obs['type'][[x for x in adata.obs_names if adata.obs.fine_grain[x] in ['GreatVesselSmoothMuscleCells']]]='SmoothMusleCells'
adata.obs['type'][[x for x in adata.obs_names if adata.obs.fine_grain[x] in ['GreatVesselArterialEndothelialCells','GreatVesselArterialEndothelialCells']]]='EndothelialCells'
```

Example 85 (python):
```python
set(adata.obs.mid_grain)
```

Example 86 (unknown):
```unknown
{'BloodVesselEndothelialCells', 'Fibroblasts', 'MuralCells'}
```

Example 87 (python):
```python
set(adata.obs.region)
```

Example 88 (unknown):
```unknown
{'aorta', 'pulmonary arches and branches'}
```

Example 89 (python):
```python
PulmonaryEndothelial=adata[adata.obs.mid_grain=='BloodVesselEndothelialCells',:].copy()
PulmonaryEndothelial=PulmonaryEndothelial[PulmonaryEndothelial.obs.region=='pulmonary arches and branches',:].copy()
PulmonaryEndothelial.obs['type']= 'PulmonaryEndothelial'

AorticEndothelial=adata[adata.obs.mid_grain=='BloodVesselEndothelialCells',:].copy()
AorticEndothelial=AorticEndothelial[AorticEndothelial.obs.region=='aorta',:].copy()
AorticEndothelial.obs['type']= 'AorticEndothelial'

PulmonarySmoothMuscle=adata[adata.obs.mid_grain=='MuralCells',:].copy()
PulmonarySmoothMuscle=PulmonarySmoothMuscle[PulmonarySmoothMuscle.obs.region=='pulmonary arches and branches',:].copy()
PulmonarySmoothMuscle.obs['type']= 'PulmonarySmoothMuscle'

AorticSmoothMuscle=adata[adata.obs.mid_grain=='MuralCells',:].copy()
AorticSmoothMuscle=AorticSmoothMuscle[AorticSmoothMuscle.obs.region=='aorta',:].copy()
AorticSmoothMuscle.obs['type']= 'AorticSmoothMuscle'

PulmonaryAdventitialFibroblasts=adata[adata.obs.mid_grain=='Fibroblasts',:].copy()
PulmonaryAdventitialFibroblasts=PulmonaryAdventitialFibroblasts[PulmonaryAdventitialFibroblasts.obs.region=='pulmonary arches and branches',:].copy()
PulmonaryAdventitialFibroblasts.obs['type']= 'PulmonaryAdventitialFibroblasts'

AorticAdventitialFibroblasts=adata[adata.obs.mid_grain=='Fibroblasts',:].copy()
AorticAdventitialFibroblasts=AorticAdventitialFibroblasts[AorticAdventitialFibroblasts.obs.region=='aorta',:].copy()
AorticAdventitialFibroblasts.obs['type']= 'AorticAdventitialFibroblasts'

bdata = PulmonaryEndothelial.concatenate(AorticEndothelial,PulmonarySmoothMuscle,AorticSmoothMuscle,PulmonaryAdventitialFibroblasts,AorticAdventitialFibroblasts)
```

Example 90 (python):
```python
sc.pl.umap(bdata, color=['type'])
```

Example 91 (unknown):
```unknown
... storing 'sangerID' as categorical
... storing 'combinedID' as categorical
... storing 'region' as categorical
... storing 'fine_grain' as categorical
... storing 'mid_grain' as categorical
... storing 'coarse_grain' as categorical
... storing 'type' as categorical
```

Example 92 (python):
```python
adata = bdata[bdata.obs['type'].isin(['PulmonaryEndothelial','AorticEndothelial'])].copy()
```

Example 93 (python):
```python
# Pulmonary region ECs have both arterial and venous markers
Genes=['KCNIP4','ADGRG6','MPP7','TSHZ2','PDE7B','NTS','DKK2','FBLN5','ENOX1','HEY2','PDGFD','RELN','GJA5']
CategoriesOrder=['PulmonaryEndothelial','AorticEndothelial']
sc.pl.dotplot(adata,Genes, groupby='type',categories_order=CategoriesOrder,cmap='YlGn',var_group_rotation=45)
```

Example 94 (python):
```python
tmp = pd.crosstab(adata.obs['type'],adata.obs['fine_grain'], normalize='index')
plt.rcParams['figure.figsize'] = [3,2]
tmp.plot.bar(stacked=True).legend(loc='center left', bbox_to_anchor=(1.0, 0.5))
```

Example 95 (python):
```python
<matplotlib.legend.Legend at 0x7fdc606270a0>
```

Example 96 (python):
```python
ClusterOfInterest='GreatVesselArterialEndothelialCells'
sc.tl.rank_genes_groups(adata, 'fine_grain', method='t-test', key_added = "t-test")
DEGenesForCluster = [x[ClusterOfInterest] for x in adata.uns['t-test']['names'][:50]]
sc.pl.dotplot(adata, DEGenesForCluster ,groupby='fine_grain',standard_scale='var',cmap='PRGn')
#sc.pl.umap(adata, color=DEGenesForCluster)
```

Example 97 (python):
```python
ranking genes
    finished: added to `.uns['t-test']`
    'names', sorted np.recarray to be indexed by group ids
    'scores', sorted np.recarray to be indexed by group ids
    'logfoldchanges', sorted np.recarray to be indexed by group ids
    'pvals', sorted np.recarray to be indexed by group ids
    'pvals_adj', sorted np.recarray to be indexed by group ids (0:00:00)
```

Example 98 (python):
```python
ClusterOfInterest='GreatVesselVenousEndothelialCells'
sc.tl.rank_genes_groups(adata, 'fine_grain', method='t-test', key_added = "t-test")
DEGenesForCluster = [x[ClusterOfInterest] for x in adata.uns['t-test']['names'][:50]]
sc.pl.dotplot(adata, DEGenesForCluster ,groupby='fine_grain',standard_scale='var',cmap='PRGn')
#sc.pl.umap(adata, color=DEGenesForCluster)
```

Example 99 (python):
```python
ranking genes
    finished: added to `.uns['t-test']`
    'names', sorted np.recarray to be indexed by group ids
    'scores', sorted np.recarray to be indexed by group ids
    'logfoldchanges', sorted np.recarray to be indexed by group ids
    'pvals', sorted np.recarray to be indexed by group ids
    'pvals_adj', sorted np.recarray to be indexed by group ids (0:00:00)
```

Example 100 (python):
```python
Genes=['MTRNR2L12','NAP1L1','HSP90AA1','NUCKS1','HDLBP','SERBP1','NPM1','MTDH','NCL','NRG1','IL33','EDN1','PVT1','RAPGEF4','PDE7B','KCNIP4','ADGRG6','DPYD','IGFBP3','NTS','INPP4B','LAMA2','PXDNL','ALDH1A2','HPSE2','CACNA2D3','TMEM132C','ESRRG','SIPA1L2','PRRX1','NR2F2','IGFBP5','HSPA1A','SLC4A4']
CategoriesOrder=['PulmonaryEndothelial','PulmonarySmoothMuscle','PulmonaryAdventitialFibroblasts','AorticEndothelial','AorticSmoothMuscle','AorticAdventitialFibroblasts']
sc.pl.dotplot(bdata,Genes, groupby='type',standard_scale='var',categories_order=CategoriesOrder,cmap='YlGn',var_group_positions=[(9,19),(20,28),(29,32)], var_group_labels=['pulmonary endothelial','pulmonary smooth muscle','pulmonary adventitial'], var_group_rotation=45)
```

Example 101 (python):
```python
sc.pl.dotplot(bdata,['NR2F2'], groupby='type',standard_scale='var',categories_order=CategoriesOrder,cmap='YlGn', var_group_rotation=45)
```

Example 102 (python):
```python
bdata.obs['AorticOrPulmonary']= 'na'
bdata.obs['AorticOrPulmonary'][[x for x in bdata.obs_names if bdata.obs.type[x] in ['AorticAdventitialFibroblasts','AorticEndothelial','AorticSmoothMuscle']]]='Aortic'
bdata.obs['AorticOrPulmonary'][[x for x in bdata.obs_names if bdata.obs.type[x] in ['PulmonaryAdventitialFibroblasts','PulmonaryEndothelial','PulmonarySmoothMuscle']]]='Pulmonary'
sc.pl.umap(bdata, color=['AorticOrPulmonary'])
```

Example 103 (python):
```python
ranking genes
    finished: added to `.uns['t-test']`
    'names', sorted np.recarray to be indexed by group ids
    'scores', sorted np.recarray to be indexed by group ids
    'logfoldchanges', sorted np.recarray to be indexed by group ids
    'pvals', sorted np.recarray to be indexed by group ids
    'pvals_adj', sorted np.recarray to be indexed by group ids (0:00:00)
```

Example 104 (python):
```python
ClusterOfInterest='Pulmonary'
sc.tl.rank_genes_groups(bdata, 'AorticOrPulmonary', method='t-test', key_added = "t-test")
DEGenesForCluster = [x[ClusterOfInterest] for x in bdata.uns['t-test']['names'][:50]]
CategoriesOrder=['AorticEndothelial','AorticSmoothMuscle','AorticAdventitialFibroblasts','PulmonaryEndothelial','PulmonarySmoothMuscle','PulmonaryAdventitialFibroblasts']
sc.pl.dotplot(bdata, DEGenesForCluster ,groupby='type',categories_order=CategoriesOrder,standard_scale='var',cmap='PRGn')
#sc.pl.umap(bdata, color=DEGenesForCluster)
```

Example 105 (python):
```python
ranking genes
    finished: added to `.uns['t-test']`
    'names', sorted np.recarray to be indexed by group ids
    'scores', sorted np.recarray to be indexed by group ids
    'logfoldchanges', sorted np.recarray to be indexed by group ids
    'pvals', sorted np.recarray to be indexed by group ids
    'pvals_adj', sorted np.recarray to be indexed by group ids (0:00:00)
```

Example 106 (python):
```python
sc.pl.dotplot(bdata, ['MTRNR2L12','NAP1L1','NUCKS1','HDLBP','SERBP1','NPM1','MTDH','NCL',] ,groupby='type',categories_order=CategoriesOrder,standard_scale='var',cmap='YlGn')
```

Example 107 (python):
```python
set(adata.obs.region)
```

Example 108 (unknown):
```unknown
{'aorta',
 'apex',
 'atria',
 'atrial septum',
 'base',
 'ductus arteriosus',
 'heart without node',
 'left ventricle',
 'node',
 'outflow tract',
 'pericardium',
 'pulmonary arches and branches',
 'right ventricle',
 'right ventricle and papillaries',
 'whole sample'}
```

Example 109 (python):
```python
adata = adata[adata.obs['week'].isin(['20W'])].copy()
adata = adata[adata.obs['region'].isin(['pulmonary arches and branches','aorta','ductus arteriosus'])].copy()
adata = adata[adata.obs['fine_grain'].isin(['GreatVesselArterialEndothelialCells','GreatVesselVenousEndothelialCells'])].copy()
set(adata.obs.fine_grain)
adata
```

Example 110 (unknown):
```unknown
AnnData object with n_obs × n_vars = 1272 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score'
    var: 'gene_name_scRNA-0', 'gene_id'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'age_colors', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'cycling_colors', 'donor_colors', 'fine_grain_colors', 'heart_or_greatvessels_colors', 'hvg', 'kit_10x_colors', 'modality_colors', 'neighbors', 'phase_colors', 'region_colors', 'sex_colors', 'trimester_colors', 'umap', 'week_colors'
    obsm: 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 111 (python):
```python
# preserve raw count
adata.layers["counts"] = adata.X.copy()

# normalise and log-transform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key='batch_key')
#sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5)

print(sum(adata.var.highly_variable))
sc.pl.highly_variable_genes(adata)

# freeze the state in `.raw`
adata.raw = adata

# subset hvgs
adata = adata[:, adata.var.highly_variable]

adata=adata.copy()

sc.tl.score_genes_cell_cycle(adata, s_genes=s_genes, g2m_genes=g2m_genes)
#sc.pp.regress_out(adata, ['S_score','G2M_score'])

#batch correction
scvi.model.SCVI.setup_anndata(adata, 
                              layer="counts", 
                              batch_key="batch_key",
                              categorical_covariate_keys=['phase'],
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo'])

vae = scvi.model.SCVI(adata, n_hidden = 128, n_latent = 50, n_layers = 3, dispersion = 'gene-batch')
vae.train(max_epochs=50)
    
# Convergence plot
x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
plt.plot(x, vae.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
plt.clf()

# add the latent space to adata
adata.obsm["X_scVI"] = vae.get_latent_representation()

sc.tl.score_genes(adata, stress_genes, score_name='stress_score')
sc.tl.score_genes(adata, hb_genes1, score_name='hb1_score')

sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata,min_dist=0.1)
#sc.tl.tsne(adata)
sc.pl.umap(adata, color=['log1p_total_counts','n_genes','n_genes_by_counts','total_counts','pct_counts_mt','pct_counts_ribo'],ncols=5)
sc.pl.umap(adata, color=['scrublet_score','stress_score', 'hb1_score','phase'], ncols=5,)
sc.pl.umap(adata, color=['donor','age','week','trimester','region','phase','cycling','cell_or_nuclei','modality','kit_10x','FACSgate','heart_or_greatvessels','coarse_grain','mid_grain','fine_grain'],wspace=0.8,ncols=5)
```

Example 112 (python):
```python
normalizing counts per cell
    finished (0:00:00)
extracting highly variable genes
    finished (0:00:01)
--> added
    'highly_variable', boolean vector (adata.var)
    'means', float vector (adata.var)
    'dispersions', float vector (adata.var)
    'dispersions_norm', float vector (adata.var)
5300
```

Example 113 (python):
```python
calculating cell cycle phase
computing score 'S_score'
WARNING: genes are not in var_names and ignored: ['MCM5', 'MCM2', 'UNG', 'MCM6', 'CDCA7', 'DTL', 'UHRF1', 'RFC2', 'RPA2', 'NASP', 'RAD51AP1', 'SLBP', 'CCNE2', 'UBR7', 'MSH2', 'RAD51', 'CDC45', 'TIPIN', 'DSCC1', 'BLM', 'CASP8AP2', 'USP1', 'POLA1', 'CHAF1B', 'E2F8']
    finished: added
    'S_score', score of gene set (adata.obs).
    500 total control genes are used. (0:00:00)
computing score 'G2M_score'
WARNING: genes are not in var_names and ignored: ['TMPO', 'CKAP2L', 'HJURP', 'CDCA3', 'TTK', 'RANGAP1', 'AURKA', 'PSRC1', 'LBR', 'CKAP5', 'CTCF', 'G2E3', 'CBX5', 'CENPA']
    finished: added
    'G2M_score', score of gene set (adata.obs).
    707 total control genes are used. (0:00:00)
-->     'phase', cell cycle phase (adata.obs)
```

Example 114 (unknown):
```unknown
GPU available: False, used: False
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
```

Example 115 (unknown):
```unknown
Epoch 50/50: 100%|██████| 50/50 [00:12<00:00,  4.05it/s, loss=1.83e+03, v_num=1]
```

Example 116 (python):
```python
computing score 'stress_score'
WARNING: genes are not in var_names and ignored: ['FOS', 'FOSB', 'JUN', 'JUNB', 'JUND', 'CEBPB', 'EGR1', 'HSP90AA1', 'HSP90AB1', 'HSPA8', 'HSPE1', 'IER2', 'BTG1', 'DUSP1']
    finished: added
    'stress_score', score of gene set (adata.obs).
    200 total control genes are used. (0:00:00)
computing score 'hb1_score'
WARNING: genes are not in var_names and ignored: ['HBD', 'HBE1', 'HBZ', 'HBM', 'HBQ1']
    finished: added
    'hb1_score', score of gene set (adata.obs).
    198 total control genes are used. (0:00:00)
computing neighbors
    finished: added to `.uns['neighbors']`
    `.obsp['distances']`, distances for each pair of neighbors
    `.obsp['connectivities']`, weighted adjacency matrix (0:00:00)
computing UMAP
    finished: added
    'X_umap', UMAP coordinates (adata.obsm) (0:00:00)
```

Example 117 (unknown):
```unknown
... storing 'phase' as categorical
```

Example 118 (unknown):
```unknown
<Figure size 600x600 with 0 Axes>
```

Example 119 (python):
```python
sc.tl.leiden(adata, resolution=0.25, key_added='leiden')
sc.pl.umap(adata, color='leiden',legend_loc='on data')
```

Example 120 (python):
```python
running Leiden clustering
    finished: found 6 clusters and added
    'leiden', the cluster labels (adata.obs, categorical) (0:00:00)
```

Example 121 (python):
```python
sc.pl.dotplot(adata, ['PDE4D','GADD45B','CCNL1','ICAM1','POU4F1','IER3','CHRM3','GHR','TNFRSF10D','GPRC5A','NAMPT','PEX5L','DEPP1','ADCY2','SLC2A3','TBX1'], categories_order=['ductus arteriosus','pulmonary arches and branches','aorta'],groupby='region',standard_scale='var',cmap='YlGn',title='great vessel endothelial cells')
```

Example 122 (python):
```python
sc.pl.dotplot(adata, ['PDE4D','GADD45B','CCNL1','ICAM1','POU4F1','IER3','CHRM3','GHR','TNFRSF10D','GPRC5A','NAMPT','PEX5L','DEPP1','ADCY2','SLC2A3','TBX1','HIF1A','EPAS1','NFE2L2','PTGS2','PTGES3','PTGDS','PTGDS'], categories_order=['ductus arteriosus','pulmonary arches and branches','aorta'],groupby='region',standard_scale='var',cmap='YlGn',title='great vessel endothelial cells')
```

---

## Variables¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig3_SAN_niche/VisiumHD_CCSaxis_no2_calculate_axis.html

**Contents:**
- Variables¶
- Read in data¶
- Calculate axis¶
- Save¶
- Plot for figure¶

bin2cell output which has celltypist output as well

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os

import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 

import tissue_tag as tt
```

Example 2 (python):
```python
# import session_info
# session_info.show()
```

Example 3 (python):
```python
sc.settings.set_figure_params(dpi=120)
```

Example 4 (unknown):
```unknown
os.getcwd()
```

Example 5 (unknown):
```unknown
'/nfs/team205/kk18/notebooks/Foetal/VisiumHD/re-seq_mpp0p3'
```

Example 6 (python):
```python
adata_dir = '/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects'
sample_id = 'HEA_FOET14880396'
```

Example 7 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk/Aug2024'
table_dir = '/nfs/team205/heart/fetal_ms_figures/tables/Aug2024'
```

Example 8 (python):
```python
# read in 
adata = sc.read_h5ad(f'{adata_dir}/{sample_id}_b2c_cells_filtered_lognorm.h5ad')
adata
```

Example 9 (unknown):
```unknown
AnnData object with n_obs × n_vars = 129803 × 18047
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'fine_grain_4Jan2025', 'HistologicalAnnotation', 'SANaxis_landmarks'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'HistologicalAnnotation_colors', 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'fine_grain_4Jan2025_colors', 'hvg', 'log1p', 'neighbors', 'pca', 'spatial', 'umap'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 10 (python):
```python
# select cells to calculate axis
cells = [
     'SinoatrialNodePacemakerCellsHorn',
    'SinoatrialNodePacemakerCellsHead',
    'SinoatrialNodePacemakerCellsTail',
]
bdata = adata[adata.obs['fine_grain_4Jan2025'].isin(cells)]
obs_df_sel = bdata.obs.copy()
obs_df_sel.rename(columns={'array_row':'y','array_col':'x'},inplace=True)
```

Example 11 (python):
```python
# calculate distance
print('calculating distances')
dist2clusterall = tt.dist2cluster_fast(obs_df_sel, annotation='SANaxis_landmarks', KNN=1, logscale=False)
dist2clusterall
```

Example 12 (unknown):
```unknown
calculating distances
calculating distance matrix with cKDTree
end
nan
start
```

Example 13 (unknown):
```unknown
{'end': array([1251.38301643, 1505.08749736, 1600.76852645, ...,  223.18975964,
         355.30243964,  435.07815142]),
 'nan': array([0., 0., 0., ..., 0., 0., 0.]),
 'start': array([1113.68595365, 1453.53120161, 1529.98277576, ...,  183.28824696,
         717.60829512,  805.5865061 ])}
```

Example 14 (unknown):
```unknown
obs_df_sel = tt.calculate_axis_2p(obs_df_sel, 'SANaxis_landmarks', ['start','end'], output_col='out',prefix='L2_dist_')
```

Example 15 (python):
```python
bdata.obs['SAN_axis'] = np.nan
bdata.obs.loc[obs_df_sel.index,'SAN_axis'] = obs_df_sel['out']
```

Example 16 (python):
```python
/tmp/ipykernel_1233989/1043464966.py:1: ImplicitModificationWarning: Trying to modify attribute `.obs` of view, initializing view as actual.
  bdata.obs['SAN_axis'] = np.nan
```

Example 17 (python):
```python
# add to adata
adata.obs['SAN_axis'] = np.nan
adata.obs.loc[bdata.obs_names,'SAN_axis'] = bdata.obs['SAN_axis'].copy()
```

Example 18 (python):
```python
mask = ((adata.obs['array_row'] >= 1800) & 
        (adata.obs['array_row'] <= 2300) & 
        (adata.obs['array_col'] >= 600) & 
        (adata.obs['array_col'] <= 1000)
       )
bdata = adata[mask]

sc.pl.spatial(bdata, color='SAN_axis',cmap='gist_rainbow',
              img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer",
              size=7,legend_fontsize=30, spot_size=10,
                  frameon=False) # save=f'_{c}_b2c',
```

Example 19 (python):
```python
adata.write(f'{adata_dir}/{sample_id}_b2c_cells_filtered_lognorm.h5ad')
adata
```

Example 20 (unknown):
```unknown
AnnData object with n_obs × n_vars = 129803 × 18047
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'fine_grain_4Jan2025', 'HistologicalAnnotation', 'SANaxis_landmarks', 'SAN_axis'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'HistologicalAnnotation_colors', 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'fine_grain_4Jan2025_colors', 'hvg', 'log1p', 'neighbors', 'pca', 'spatial', 'umap'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 21 (python):
```python
# read in 
adata = sc.read_h5ad(f'{adata_dir}/{sample_id}_b2c_cells_filtered_lognorm.h5ad')

mask = ((adata.obs['array_row'] >= 1800) & 
        (adata.obs['array_row'] <= 2300) & 
        (adata.obs['array_col'] >= 600) & 
        (adata.obs['array_col'] <= 1000)
       )
bdata = adata[mask]

sc.pl.spatial(bdata, color='SAN_axis',cmap='gist_rainbow',
              img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer",
              size=7,legend_fontsize=30, spot_size=10,
                  frameon=False,show=False)
plt.savefig(f'{figdir}/VisiumHD_SANaxis.pdf', bbox_inches="tight",dpi=300)
```

---

## Subcluster¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/RNA/scnRNA_Integration-Annotation/Cardiomyocytes/CardiomyocyteLVRVAtriaTrabeculatedFigures.html

**Contents:**
- Subcluster¶
- Compact trabeculated GO¶
- Analysis on ratios¶
- Any markers of cycling CMs (apart from cycling genes)¶
- If you wish to investigate from the global object¶
- Fetal CMs only, for UMAP figure¶

**Examples:**

Example 1 (unknown):
```unknown
#Added the keys for age and sex
```

Example 2 (python):
```python
# Import packages, initial settings

import numpy as np
import matplotlib.pyplot as plt
import scanpy as sc
import pandas as pd
import anndata
import scrublet as scr
import seaborn as sns
import magic
import pybiomart
import six
import sys
sys.modules['sklearn.externals.six'] = six
import scanorama
import harmonypy
import bbknn
import gseapy
from gseapy.plot import barplot, dotplot
import scvelo as scv
import scvi

plt.rcParams['pdf.fonttype'] = 42
plt.rcParams['ps.fonttype'] = 42

sc.settings.verbosity = 3  # verbosity: errors (0), warnings (1), info (2), hints (3)

from matplotlib.colors import LinearSegmentedColormap
cmap = LinearSegmentedColormap.from_list(name='gene_cmap', colors=['lightgrey', 'thistle', 'red', 'darkred']) 

sc.settings.set_figure_params(dpi=80, color_map='viridis', vector_friendly=False,  dpi_save=300, facecolor='white')
sc.logging.print_version_and_date()
#sc.logging.print_versions()
```

Example 3 (unknown):
```unknown
Global seed set to 0
```

Example 4 (unknown):
```unknown
Running Scanpy 1.8.1, on 2023-09-18 10:12.
```

Example 5 (python):
```python
adata=sc.read('./Objects/Feb28ObjectRaw.h5ad')
print(adata.shape)
```

Example 6 (unknown):
```unknown
(297473, 36601)
```

Example 7 (python):
```python
cell_cycle_genes = ['MCM5','PCNA','TYMS','FEN1','MCM2','MCM4','RRM1','UNG','GINS2','MCM6','CDCA7','DTL','PRIM1','UHRF1','MLF1IP','HELLS','RFC2','RPA2','NASP','RAD51AP1','GMNN','WDR76','SLBP','CCNE2','UBR7','POLD3','MSH2','ATAD2','RAD51','RRM2','CDC45','CDC6','EXO1','TIPIN','DSCC1','BLM','CASP8AP2','USP1','CLSPN','POLA1','CHAF1B','BRIP1','E2F8','HMGB2','CDK1','NUSAP1','UBE2C','BIRC5','TPX2','TOP2A','NDC80','CKS2','NUF2','CKS1B','MKI67','TMPO','CENPF','TACC3','FAM64A','SMC4','CCNB2','CKAP2L','CKAP2','AURKB','BUB1','KIF11','ANP32E','TUBB4B','GTSE1','KIF20B','HJURP','CDCA3','HN1','CDC20','TTK','CDC25C','KIF2C','RANGAP1','NCAPD2','DLGAP5','CDCA2','CDCA8','ECT2','KIF23','HMMR','AURKA','PSRC1','ANLN','LBR','CKAP5','CENPE','CTCF','NEK2','G2E3','GAS2L3','CBX5','CENPA']
s_genes = cell_cycle_genes[:43]
s_genes = [x for x in s_genes if x in adata.var_names]
print('S genes:', s_genes, '\n')
g2m_genes = cell_cycle_genes[43:]
g2m_genes = [x for x in g2m_genes if x in adata.var_names]
print('G2 genes:', g2m_genes, '\n')
cell_cycle_genes = [x for x in cell_cycle_genes if x in adata.var_names]

hb_genes1 = ['HBB','HBD','HBG1','HBG2','HBE1','HBZ','HBM','HBA2','HBA1', 'HBQ1']
hb_genes1 = [x for x in hb_genes1 if x in adata.var_names]
print('Hemoglobin genes set 1:', hb_genes1, '\n')

hb_genes2 = ['AHSP','GYPA','RHAG','GFI1B','KEL','EPB42','ALAS2','TSPO2','GYPB','GYPE','SPTA1','RHCE','KLF1','RHD','SLC4A1','SNX22','SQSTM1','SOD1','FTH1','MAP1LC3B','SLC25A37','TRIM58','HMBS']
hb_genes2 = [x for x in hb_genes2 if x in adata.var_names]
print('Hemoglobin genes set 2:', hb_genes2, '\n')

mito_genes=['MT-ND1','MT-ND2','MT-CO1','MT-CO2','MT-ATP8','MT-ATP6','MT-CO3','MT-ND3','MT-ND4L', 'MT-ND4', 'MT-ND5', 'MT-ND6', 'MT-CYB']
mito_genes = [x for x in mito_genes if x in adata.var_names]
print('Mitochondrial genes:', mito_genes, '\n')

ribo_genes = ['RPLP0','RPLP1','RPLP2','RPL3','RPL3L','RPL4','RPL5','RPL6','RPL7','RPL7A','RPL7L1','RPL8','RPL9','RPL10','RPL10A','RPL10L','RPL11','RPL12','RPL13','RPL13A','RPL14','RPL15','RPL17','RPL18','RPL18A','RPL19','RPL21','RPL22','RPL22L1','RPL23','RPL23A','RPL24','RPL26','RPL26L1','RPL27','RPL27A','RPL28','RPL29','RPL30','RPL31','RPL32','RPL34','RPL35','RPL35A','RPL36','RPL36A','RPL36AL','RPL37','RPL37A','RPL38','RPL39','RPL39L','UBA52','RPL41','RPSA','RPS2','RPS3','RPS3A','RPS4X','RPS4Y1','RPS4Y2','RPS5','RPS6','RPS7','RPS8','RPS9','RPS10','RPS11','RPS12','RPS13','RPS14','RPS15','RPS15A','RPS16','RPS17','RPS18','RPS19','RPS20','RPS21','RPS23','RPS24','RPS25','RPS26','RPS27','RPS27A','RPS27L','RPS28','RPS29','FAU']
ribo_genes = [x for x in ribo_genes if x in adata.var_names]
print('Ribo genes:', ribo_genes, '\n')

stress_genes=['FOS','FOSB','JUN','JUNB','JUND','ZFP36','SOCS3','CEBPD','CEBPB','ATF3','EGR1','HSPA1A','HSPA1B','HSP90AA1','HSP90AB1','HSPA8','HSPB1','HSPE1','IER3','IER2','BTG1','BTG2','DUSP1']
stress_genes = [x for x in stress_genes if x in adata.var_names]
print('Tissue dissociation stress associated genes:', stress_genes, '\n')

FoxProteins=['FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1']
chrY_genes=['SRY', 'RPS4Y1', 'ZFY', 'ZFY-AS1', 'LINC00278', 'PCDH11Y', 'TTTY2B', 'TBL1Y', 'PRKY', 'USP9Y', 'DDX3Y', 'UTY', 'TMSB4Y', 'NLGN4Y', 'TTTY14', 'KDM5D', 'TTTY10', 'EIF1AY', 'PRORY', 'LINC00266-4P']
MembraneProteins1=['AADAT','AAK1','ABCA12','ABCA7','ABCB1','ABCB11','ABCB4','ABCB6','ABCC1','ABCC3','ABCC4','ABCC5','ABCC6','ABCG2','ABHD3','ABI1','ABLIM3','AC004691.2','AC004922.1','AC005324.3','AC005726.1','AC008403.1','AC138696.1','ACIN1','ACKR3','ACLY','ACOT13','ACOT9','ACTN1','ACTR10','ACTR6','ADA','ADAM10','ADAM12','ADAM22','ADAM28','ADAM33','ADAMTS1','ADAMTS2','ADAMTSL4','ADAP1','ADCK5','ADD1','ADD2','ADGRB2','ADGRD1','ADGRG2','ADGRG7','ADGRL2','ADH1A','ADH1B','ADH1C','ADH7','ADK','ADORA1','ADRA1B','ADRM1','ADSS','AFAP1L2','AFDN','AFG1L','AGER','AGRN','AHNAK','AHNAK2','AIFM2','AKAP10','AKAP11','AKAP12','AKR1B10','AKR1B15','AL132780.3','AL139300.1','AL355315.1','AL358113.1','AL645922.1','ALDH3A1','ALK','ALOX15B','ALOXE3','ALPI','ALPP','ALPPL2','ALS2CR12','AMACR','AMDHD1','AMER1','AMER2','AMIGO2','AMOT','AMOTL1','AMOTL2','AMPH','ANK2','ANK3','ANKEF1','ANKFN1','ANKIB1','ANKLE2','ANKRA2','ANKRD13A','ANKRD35','ANKRD42','ANKRD45','ANKS1B','ANO1','ANO10','ANO2','ANO6','ANPEP','ANXA1','ANXA13','ANXA2','ANXA3','AP2M1','APBB1','APBB1IP','APC','APLP1','APMAP','APOC3','APOD','APOLD1','AQP1','AQP10','AQP3','AQP4','AQP5','ARAP1','ARF1','ARHGAP18','ARHGAP19','ARHGAP21','ARHGAP22','ARHGAP23','ARHGAP24','ARHGAP28','ARHGAP29','ARHGAP33','ARHGAP36','ARHGAP40','ARHGAP9','ARHGDIG','ARHGEF1','ARHGEF11','ARHGEF12','ARHGEF15','ARHGEF26','ARHGEF28','ARHGEF40','ARHGEF5','ARID2','ARID4A','ARID5B','ARL13A','ARL15','ARL4C','ARMC12','ARMCX5','ARPC1A','ARPC5','ARRB2','ARRDC4','ARVCF','ASAP1','ASB5','ASGR1','ASGR2','ASH2L','ASIC1','ASIC4','ASPM','ASPSCR1','ATG13','ATG3','ATG7','ATIC','ATOX1','ATP10D','ATP12A','ATP13A3','ATP1B3','ATP23','ATP2B1','ATP2B4','ATP2C2','ATP6V0A2','ATP6V1H','ATP8A2','ATP8B1','ATXN10','ATXN3','ATXN7L3','AVEN','AVPI1','AXIN2','AXL','B2M','B3GALT2','BACE1','BAG3','BAIAP2','BAIAP2L1','BANK1','BASP1','BBOF1','BCAR1','BCAR3','BCL2L12','BCORL1','BCR','BDKRB2','BEND6','BFSP1','BFSP2','BICD2','BIN2','BLNK','BLVRB','BMPR2','BMX','BOC','BOLL','BORCS5','BORCS8-MEF2B','BRK1','BRPF1','BRSK1','BSDC1','BST2','BTK','BTN2A1','BTRC','BUB1B-PAK6','BVES','BZW2','C10orf90','C11orf42','C11orf52','C12orf73','C12orf76','C14orf37','C14orf93','C16orf74','C17orf58','C17orf62','C18orf21','C19orf33','C19orf38','C19orf48','C19orf71','C1orf116','C1orf189','C1orf35','C1orf54','C1orf68','C1orf74','C1QBP','C2orf72','C3orf14','C3orf20','C3orf38','C4orf19','C4orf22','C7orf43','C8orf33','C8orf37','C9orf153','C9orf3','CA14','CA9','CABLES2','CACNA1E','CACNG4','CADM1','CALCRL','CALD1','CALML5','CALN1','CAMK2A','CAMK2B','CAMK2D','CAMK2G','CAP2','CAPS','CASKIN2','CASP4','CAST','CAV3','CAVIN1','CAVIN2','CAVIN3','CAVIN4','CBWD1','CBWD2','CBWD3','CBWD5','CBWD6','CC2D1A','CCBE1','CCDC107','CCDC112','CCDC115','CCDC124','CCDC136','CCDC144NL','CCDC15','CCDC154','CCDC170','CCDC24','CCDC28A','CCDC59','CCDC62','CCDC66','CCDC69','CCDC8','CCDC83','CCDC85C','CCDC93','CCDC97','CCNT2','CCNYL1','CCR3','CCR4','CCT3','CD109','CD163','CD164L2','CD200R1','CD226','CD248','CD274','CD2AP','CD300A','CD300LB','CD33','CD34','CD37','CD38','CD3E','CD3G','CD4','CD44','CD46','CD47','CD48','CD79B','CD81','CD84','CD86','CD8A','CD8B','CD9','CD93','CD99L2','CDAN1','CDC42EP1','CDC42EP3','CDC42EP4','CDC42SE1','CDCA4','CDH1','CDH13','CDH15','CDH17','CDH2','CDH26','CDH3','CDH4','CDH5','CDH6','CDH8','CDIPT','CDK16','CDK5','CDK5RAP2','CDK7','CDRT15','CDV3','CEACAM5','CELSR1','CENPJ','CEP162','CEP295','CEP55','CEP57L1','CEP89','CERCAM','CFAP157','CFAP206','CFAP58','CFAP65','CFB','CFL2','CFLAR','CGN','CGNL1','CHCHD7','CHIC2','CHMP2A','CHRM3','CHRNA1','CHRNA5','CHRNB4','CHRND','CIB1','CIB3','CIP2A','CKAP5','CLCA2','CLCA4','CLCN5','CLCN6','CLDN11','CLDN18','CLDN2','CLDN23','CLDN3','CLDN4','CLDN7','CLDN9','CLEC2A','CLEC4F','CLEC5A','CLECL1','CLIC2','CLIC5','CLTB','CLUL1','CMTM4','CMTM7','CMTR2','CMYA5','CNGA1','CNKSR2','CNNM1','CNNM4','CNP','CNR1','CNTD1','COBL','COBLL1','COG3','COL16A1','COL17A1','COL4A5','COL6A2','COLEC12','COLQ','COMMD4','COMTD1','COPRS','COQ7','CORIN','CORO1B','CORO2A','CORO2B','COX6B2','CPEB3','CPPED1','CPZ','CRB3','CREB5','CRIP1','CRIP2','CRK','CRLF3','CROCC','CRTC1','CRYAB','CRYBG3','CRYL1','CSDE1','CSF1','CSF1R','CSPG4','CST6','CTNNA1','CTNNB1','CTNNBIP1','CTNND1','CTNND2','CTSF','CTSV','CTTN','CTXN1','CUEDC1','CUTA','CX3CL1','CX3CR1','CXADR','CXCR2','CYB5D2','CYBRD1','CYFIP2','CYP20A1','CYP24A1','CYTH2','CYTH4','DAAM1','DAB2','DAG1','DAGLB','DAW1','DBN1','DCAF13','DCAF16','DCBLD2','DCHS1','DCHS2','DCP2','DCT','DCTN6','DCUN1D2','DDAH1','DDB2','DDIAS','DDIT4L','DDR1','DDR2','DDX58','DDX60L','DENND6B','DFFA','DGKG','DGKH','DHDDS','DHPS','DHRS7B','DIAPH1','DIAPH3','DIO2','DIS3L','DLG1','DLG2','DLG5','DLGAP3','DLL3','DMWD','DNAAF1','DNAAF4','DNAH17','DNAJA4','DNAJB13','DNAJB4','DNAJC18','DNAJC5','DNAJC6','DNAJC9','DOC2A','DOCK4','DOK3','DONSON','DPEP1','DPH1','DPPA5','DPYSL2','DQX1','DRD1','DRP2','DSC3','DSEL','DSG2','DSP','DTNA','DUS1L','DUSP15','DXO','DYSF','ECSCR','EDA2R','EDNRA','EDNRB','EEF1AKMT1','EEF2','EFCAB1','EFCAB11','EFCAB2','EFCAB7','EFHC2','EFNB1','EFR3A','EFR3B','EGFR','EHBP1','EHD1','EHD2','EHD3','EHD4','EIF2B2','EIF5','EIF5B','ELFN1','ELMO3','ELN','ELOC','EMP3','ENAH','ENC1','ENG','ENKD1','ENO1','ENO2','ENO3','ENOX1','ENOX2','EPB41','EPB41L1','EPB41L2','EPB41L3','EPB41L4A','EPB41L4B','EPB41L5','EPCAM','EPHA2','EPHA3','EPHB1','EPHB2','EPN1','EPOR','EPS8L2','ERAP1','ERBB2','ERBB3','ERBIN','ERC1','ERCC4','ERMN','ESCO2','ETNK1','ETS2','EVA1A','EVI2B','EXOC1','EXOC3L1','EXOC7','EZR','F11R','F2RL1','F2RL3','FAAP20','FABP2','FABP3','FABP5','FAIM','FAM126A','FAM126B','FAM129A','FAM129B','FAM13A','FAM163B','FAM171A1','FAM193A','FAM205A','FAM210B','FAM222A','FAM45A','FAM47E','FAM47E-STBD1','FAM72A','FAM72B','FAM72C','FAM72D','FANCG','FAS','FASN','FBLIM1','FBLN2','FBLN5','FBLN7','FBP2','FBRSL1','FBXL2','FBXO16','FBXO38','FBXO41','FBXW11','FCAR','FCGBP','FCGR1A','FCGR1B','FCGR2A','FCHSD2','FFAR4','FGD1','FGD5','FGFR2','FGR','FHIT','FHL1','FIBCD1','FIGN','FILIP1','FILIP1L','FKBP2','FLAD1','FLCN','FLNA','FLNB','FLNC','FLOT1','FLOT2','FLRT3','FLT1','FLT4','FLVCR1','FMN2','FMNL2','FMNL3','FNBP1','FOXA2','FOXN3','FP565260.3','FRMD3','FRMD7','FRMD8','FRS2','FSCN1','FTCD','FUT2','FXYD3','FYB1','FYN','FZD1','FZD2','FZD4','FZD6','GAB1','GAB2','GABRA1','GABRA2','GABRA3','GABRA5','GABRG1','GAD1','GADL1','GAGE1','GAGE10','GAGE12C','GAGE12D','GAGE12E','GAGE12F','GAGE12G','GAGE12H','GAGE12J','GAGE13','GAGE2A','GAGE2E','GALNT10','GALR1','GAP43','GAPDH','GAPT','GAPVD1','GAS2','GAS2L1','GAS7','GAS8','GBP4','GCA','GCC1','GCLM','GCSAML-AS1','GFI1B','GFOD1','GFRA3','GHR','GID8','GINM1','GIPC1','GJA1','GJB3','GJB4','GJB6','GJB7','GJC1','GJC2','GLO1','GLP2R','GLRX','GLUL','GMIP','GNAI2','GNAQ','GNAS','GNB1','GNB2','GNB3','GNB4','GNG2','GP9','GPAT3','GPATCH1','GPBP1','GPBP1L1','GPC1','GPC3','GPC4','GPI','GPR1','GPR142','GPR156','GPR157','GPR160','GPR173','GPR20','GPR27','GPR50','GPR63','GPRC5A','GPRIN1','GPRIN2','GPX8','GRAMD1A','GRAMD1B','GRAMD1C','GRAP2','GRB7','GRID2IP','GRIK5','GRK3','GRK5','GRPR','GSDMA','GTF2F1','GTSE1','GUCA1A','GYPA','GYPC','HAPLN3','HARBI1','HAS1','HAUS4','HAUS7','HCAR2','HCAR3','HCFC2','HCK','HCLS1','HCST','HDAC8','HEG1','HELZ','HENMT1','HERC2','HERPUD1','HFE','HIF1A','HIF3A','HINT1','HIP1R','HLA-A','HLA-DPB1','HMGCS1','HMOX1','HOMER3','HOXC5','HOXD4','HPS3','HRASLS5','HRH1','HS6ST3','HSH2D','HSPB1','HSPG2','HTR1B','HTR7','HTRA1','HUNK','HYDIN','HYI','HYLS1','ICAM1','ICAM2','ICOS','ICOSLG','IFFO2','IFIT5','IFNGR1','IFRD1','IFT122','IGFL2','IGSF11','IGSF23','IKZF5','IL10RA','IL12RB1','IL12RB2','IL13','IL13RA2','IL16','IL17RB','IL1R1','IL1RL1','IL21R','IL4R','IL6ST','IL7R','ILDR2','ILK','INPP5A','INSR','INTS12','IP6K2','IPCEF1','IQGAP1','IQGAP2','IREB2','IRF2','IRS1','ISLR','ITGA2B','ITGA3','ITGA4','ITGA7','ITGA8','ITGA9','ITGB1','ITGB2','ITGB3','ITGB4','ITGB5','ITGB6','ITGB7','ITGB8','ITK','ITM2A','ITPKA','ITSN1','ITSN2','IYD','JAG1','JAK2','JAK3','JAM2','JAML','JCAD','JMJD4','JPT2','JUP','KANK1','KANK3','KANSL2','KARS','KATNB1','KCNAB1','KCNB2','KCNH8','KCNJ15','KCNK13','KCNK17','KCNK6','KCNK7','KCNK9','KCNN2','KCNS3','KCNU1','KDF1','KDR','KEL','KIAA0556','KIAA0895','KIAA1324','KIAA1522','KIAA1958','KIF17','KIF19','KIF21A','KIF21B','KIF26A','KIF26B','KISS1R','KIT','KLC1','KLC2','KLF9','KLHDC7B','KLHL41','KLHL6','KLHL7','KLK7','KLRC1','KLRC2','KLRF1','KLRK1','KMT2D','KNDC1','KNSTRN','KNTC1','KPTN','KRT75','KRT77','KRTAP25-1','L1CAM','L3MBTL1','LAIR1','LAMA4','LAMC1','LAMP2','LAMTOR1','LAPTM4A','LASP1','LAT','LAT2','LAX1','LCP1','LCTL','LDB2','LDLRAD3','LENG1','LGALS3','LGALS4','LGALS8','LGR6','LHFPL2','LHX6','LIMA1','LINGO1','LIPJ','LMCD1','LMOD2','LMTK2','LMTK3','LNX1','LNX2','LONRF1','LONRF3','LOXHD1','LPAR1','LPAR3','LPAR4','LPIN1','LPIN2','LRCH2','LRP11','LRP12','LRRC15','LRRC26','LRRC34','LRRC45','LRRC69','LRRC7','LRRC9','LRRFIP2','LRRIQ3','LSMEM2','LSR','LTB4R2','LUZP2','LY6K','LYN','LYPD1','LYST','LZTS1','LZTS2','MACF1','MADD','MAGI1','MAGI3','MAMDC2','MAP1S','MAP2','MAP2K1','MAP2K4','MAP2K7','MAP3K12','MAP3K21','MAP3K6','MAP3K7','MAP4','MAP4K1','MAP4K5','MAP6','MAPK15','MAPK8IP1','MAPT','MARCH10','MARCH7','MARCKS','MARK2','MARVELD2','MAS1L','MAT2B','MBP','MC5R','MCAM','MCEMP1','MCF2L2','MCMBP','MCTS1','MDGA1','ME1','MEF2B','MEIOC','MEPCE','MERTK','MET','METAP1','METAP2','METTL15','METTL25','METTL9','MFAP3L','MFSD1','MFSD2A','MGAT4C','MIB1','MICAL2','MICAL3','MICALL1','MICB','MIEN1','MIGA2','MISP','MKLN1','MKRN3','MLF2','MLKL','MLST8','MMACHC','MMP10','MMP15','MMP24','MMP25','MOAP1','MPHOSPH8','MPHOSPH9','MPI','MPIG6B','MPL','MPP1','MPP5','MPP7','MPZL1','MPZL2','MR1','MRGPRF','MRNIP','MRPL42','MRPL44','MRPL46','MRPS18B','MS4A1','MS4A12','MS4A14','MS4A15','MSN','MSRA','MTCP1','MTFR1L','MTHFR','MTM1','MTPN','MTSS1','MUC1','MUC20','MVD','MYH11','MYH9','MYL4','MYLK','MYO10','MYO1B','MYO1C','MYO1E','MYO1G','MYO3A','MYO5B','MYO9A','MYOF','N4BP2','NAA35','NAB1','NAIF1','NAMPT','NAP1L2','NAPG','NARFL','NCAM1','NCAM2','NCK1','NCKAP1L','NCKAP5','NCKIPSD','NCOA1','NCS1','NDC1','NECTIN2','NECTIN3','NECTIN4','NEDD1','NEDD9','NEO1','NEU1','NEXN','NF2','NFE2L2','NFKBID','NGEF']
MembraneProteins2=['NGFR','NHS','NHSL2','NID2','NINJ2','NKTR','NKX2-8','NLGN1','NLGN3','NLK','NLRX1','NMB','NMBR','NME1','NME2','NMT1','NMT2','NMUR2','NOS1','NOTCH2','NPFFR2','NPL','NPNT','NPR1','NPTN','NPTXR','NR2C1','NRAS','NRCAM','NRP2','NRSN1','NRXN1','NSD1','NSFL1C','NT5E','NTNG1','NTNG2','NTSR1','NUDCD3','NUDT16L1','NUDT9','NUMB','NUP35','NUS1','OAS3','OBSCN','OCA2','OCLN','OCM','OCM2','ODC1','ODR4','OGT','OLFM4','OLIG2','OLR1','OPHN1','OPRK1','OR10P1','OR1N2','OR2C3','OR6B2','OR6B3','OR6C6','OR8S1','OSBPL10','OSBPL6','OSBPL7','OSCAR','OSCP1','OSGEP','OSR2','OTOP2','OTULIN','OXGR1','P3H3','PACSIN2','PACSIN3','PAFAH1B2','PAG1','PAIP1','PAK1','PAK4','PAK6','PALLD','PALM','PALM2','PALM2-AKAP2','PALM3','PALMD','PANX1','PANX2','PAQR8','PARD3','PARD3B','PARD6A','PARD6G','PARP6','PATJ','PAWR','PCBD2','PCDH1','PCDH17','PCDH7','PCDHA1','PCDHA11','PCDHA12','PCDHA4','PCDHA5','PCDHB1','PCDHB11','PCDHB12','PCDHB16','PCDHB4','PCDHB5','PCDHGA1','PCDHGA10','PCDHGA11','PCDHGA12','PCDHGA2','PCDHGA3','PCDHGA4','PCDHGA5','PCDHGA6','PCDHGA7','PCDHGA8','PCDHGA9','PCDHGB1','PCDHGB2','PCDHGB3','PCDHGB4','PCDHGB5','PCDHGB6','PCDHGB7','PCDHGC3','PCDHGC4','PCDHGC5','PCLO','PCMTD1','PDAP1','PDCD7','PDE3A','PDE4A','PDE4D','PDE9A','PDGFC','PDGFRA','PDHX','PDLIM1','PDLIM5','PDPN','PDZD2','PDZD8','PEAR1','PEBP1','PFDN1','PGAP3','PGLYRP2','PGLYRP4','PGRMC2','PHACTR1','PHACTR4','PHF20L1','PHGDH','PHLDB2','PHLDB3','PHOSPHO1','PHPT1','PI3','PI4K2A','PI4KA','PIEZO2','PIGB','PIGN','PIGW','PIGX','PIK3AP1','PIK3CG','PIK3R6','PIP4K2A','PIP4K2C','PIP5K1A','PIRT','PITPNM3','PKD2L2','PKN1','PKN2','PKP1','PKP2','PKP3','PKP4','PLAA','PLAUR','PLCB1','PLCB2','PLCD3','PLCD4','PLEKHA4','PLEKHA6','PLEKHA7','PLEKHG3','PLEKHG4','PLEKHG6','PLEKHH2','PLEKHO2','PLIN4','PLPP1','PLPPR5','PLS3','PLSCR1','PLVAP','PLXNA3','PLXNA4','PNCK','PNISR','PODXL','POLE','POU2F3','PPEF1','PPFIBP1','PPIP5K1','PPL','PPM1A','PPM1F','PPP1CA','PPP1CB','PPP1R12B','PPP1R16A','PPP1R16B','PPP1R18','PPP1R27','PPP1R9A','PPP2R1B','PPP3R1','PPP3R2','PPP4C','PPP6R3','PPTC7','PRAME','PRC1','PRDX6','PRIMA1','PRKCA','PRKCE','PRKCH','PRKCZ','PRKD1','PROM2','PROSER2','PRR15','PRR32','PRR7','PRRC2A','PRRG1','PRRT2','PRRT3','PRRT4','PRTG','PRX','PSCA','PSD','PSD2','PSEN1','PSMC5','PSMC6','PSMD9','PSTPIP2','PTAFR','PTCHD4','PTK2B','PTK6','PTPN12','PTPN21','PTPN22','PTPN4','PTPRB','PTPRG','PTPRK','PTPRM','PTPRR','PTPRS','PTX3','PURG','PVALB','PVR','PXDC1','PXK','PYGL','PYGO2','PYM1','RAB13','RAB23','RAB25','RAB26','RAB27B','RAB3B','RAB3GAP2','RAB43','RAB8A','RABEPK','RAD51C','RAD52','RAI2','RALA','RALGAPA2','RAPGEF2','RAPH1','RAPSN','RASA4','RASA4B','RASAL1','RASAL2','RASAL3','RASD1','RASD2','RASGRP1','RASGRP2','RASGRP4','RASSF2','RASSF3','RASSF4','RBM46','RC3H1','RCCD1','RCSD1','RD3L','RDX','RECK','RELL1','REPS1','RET','RFLNB','RFPL1','RFPL2','RFPL3','RGP1','RGS10','RGS14','RGS19','RGS3','RGS7BP','RGS9BP','RHCE','RHD','RHEX','RHOA','RHOBTB2','RHOJ','RHOXF2','RHOXF2B','RIC8A','RIC8B','RIF1','RILPL1','RIMBP3','RIMBP3B','RIMBP3C','RIMS2','RIOK2','RIOK3','RIPK1','RND1','RNF114','RNF145','RNF146','RNF149','RNF165','RNF166','RNF39','ROBO1','ROBO3','ROBO4','ROM1','RP2','RPE','RPEL1','RPGRIP1L','RPIA','RPL36A','RPL36A-HNRNPH2','RPL36AL','RPSA','RPUSD1','RRAD','RRP12','RRP7A','RSBN1L','RSC1A1','RSPH3','RTL3','RTL4','RTN4R','RTN4RL1','RYR2','S100A13','S100A14','S100A16','S100A2','S100A3','S100A4','S100A6','S100A9','SAMD4A','SAMHD1','SAMSN1','SAPCD1','SASH3','SC5D','SCAMP1','SCAMP4','SCEL','SCHIP1','SCIN','SCN1A','SCN5A','SCN8A','SCN9A','SCNN1D','SCNN1G','SCRIB','SCUBE3','SDC2','SDC4','SDE2','SDHB','SEC16B','SEMA3C','SEMA3D','SEMA4B','SEMA4F','SEMA6C','SEMA6D','SEPT2','SEPT3','SERINC2','SERINC5','SERPINB3','SFSWAP','SGCE','SGK494','SH3BGR','SH3D21','SHANK1','SHANK3','SHARPIN','SHISA6','SHKBP1','SHROOM2','SHTN1','SIGLEC10','SIGLEC14','SIGLEC6','SIGLEC7','SIN3B','SIPA1','SIPA1L1','SIPA1L3','SKAP1','SLAMF6','SLC10A5','SLC12A2','SLC13A3','SLC13A5','SLC14A1','SLC16A1','SLC16A10','SLC16A2','SLC16A3','SLC16A7','SLC16A9','SLC17A5','SLC19A1','SLC19A3','SLC1A5','SLC1A6','SLC22A16','SLC23A3','SLC24A4','SLC26A9','SLC27A1','SLC28A1','SLC2A1','SLC2A11','SLC2A12','SLC2A14','SLC2A2','SLC2A3','SLC30A1','SLC31A2','SLC34A1','SLC34A2','SLC35F5','SLC35G2','SLC36A4','SLC38A5','SLC38A6','SLC38A7','SLC39A10','SLC39A14','SLC39A2','SLC39A6','SLC3A2','SLC40A1','SLC41A3','SLC43A1','SLC43A3','SLC44A2','SLC45A4','SLC46A1','SLC46A3','SLC4A1AP','SLC4A2','SLC4A7','SLC4A8','SLC5A12','SLC5A3','SLC5A6','SLC5A7','SLC6A6','SLC7A14','SLC7A2','SLC7A5','SLC7A6','SLC8A1','SLC9A1','SLC9A2','SLC9A3','SLC9A3R1','SLC9A3R2','SLC9A4','SLC9A5','SLC9B2','SLC9C2','SLCO1A2','SLCO1B3','SLCO4A1','SLCO4C1','SLCO5A1','SLIT1','SLIT3','SLITRK6','SLK','SLX4','SMAGP','SMAP1','SMOC1','SMPD2','SMPX','SNAP23','SNAP25','SNRK','SNTB2','SNTG2','SNX15','SNX32','SNX9','SOCS3','SOCS5','SOD1','SORBS1','SORCS1','SORT1','SOX11','SPAG16','SPANXN4','SPART','SPATA12','SPATC1L','SPERT','SPG11','SPHK1','SPINK13','SPINT1','SPINT2','SPN','SPPL2B','SPPL3','SPRED3','SPRYD3','SPTBN2','SRC','SRCIN1','SRD5A3','SRPK1','SRPRB','SRRM3','SSFA2','SSH1','SSH3','SSTR1','SSTR5','SSX2IP','ST8SIA5','STAB1','STAC','STAC2','STAMBP','STAMBPL1','STARD4','STAT2','STEAP4','STIL','STIP1','STK10','STK16','STK17A','STK17B','STOM','STOML2','STOML3','STX18','STX4','STXBP4','STYK1','SUCLG1','SUCLG2','SUGT1','SUMF2','SUPT6H','SUSD2','SVIL','SWAP70','SYCP2','SYK','SYMPK','SYNC','SYNGR3','SYNPO2L','SYPL2','SYT4','SYTL2','SYVN1','TACC2','TACR2','TACSTD2','TASP1','TAZ','TBC1D10A','TBC1D2','TBC1D24','TBC1D30','TBC1D9B','TBXA2R','TCAF2','TCAF2C','TCHP','TCIM','TDG','TDO2','TDP1','TEC','TEDC2','TEK','TES','TEX14','TEX9','TGFBR1','TGFBR2','TGM1','TGM2','THBS1','THBS2','THEGL','THY1','TIAM1','TICAM1','TIGD7','TIMD4','TJP1','TJP2','TJP3','TLN1','TLR4','TM2D2','TM4SF20','TM4SF4','TM4SF5','TMC5','TMED8','TMEM101','TMEM107','TMEM108','TMEM121B','TMEM139','TMEM141','TMEM164','TMEM176B','TMEM183A','TMEM186','TMEM2','TMEM218','TMEM222','TMEM229A','TMEM245','TMEM251','TMEM254','TMEM258','TMEM266','TMEM268','TMEM50A','TMEM63B','TMPRSS15','TMPRSS2','TMTC2','TNFRSF10D','TNFRSF11A','TNFRSF12A','TNFRSF21','TNFRSF25','TNFRSF8','TNFRSF9','TNFSF13B','TNK1','TNKS1BP1','TNNI2','TOGARAM2','TOLLIP','TOM1','TP53BP2','TPD52L1','TPRN','TRAF7','TRAIP','TRAPPC13','TRAT1','TRDN','TREML1','TRIB1','TRIM16','TRIM16L','TRIM38','TRIM4','TRIM44','TRIM54','TRIM71','TRIML2','TRIP6','TRMT1','TRMT10B','TRMT13','TRMT6','TRO','TRPM1','TRPM4','TRPM7','TRPV2','TRPV3','TSG101','TSGA13','TSHZ3','TSPAN15','TSPAN6','TSPO2','TSR2','TSSK4','TTC17','TTC23','TTC7B','TTLL12','TTLL5','TULP3','TWF2','TXNDC11','TXNIP','TYROBP','TYW3','UAP1','UBA3','UBA52','UBAC1','UBALD1','UBAP1','UBASH3B','UBE2A','UBE2B','UBE2C','UBE2D1','UBE2D2','UBE2D3','UBE2D4','UBE2K','UBE2QL1','UBE2S','UBE3C','UBQLN2','UBR2','UBTD1','UCP3','ULBP1','ULBP2','UNC5A','UNC79','UQCC1','USP14','USP15','USP2','USP20','USP21','USP4','USP53','UTP20','UTRN','VAMP5','VAMP8','VANGL1','VASP','VCAM1','VCPIP1','VIL1','VPS13B','VPS36','VPS50','VPS8','VSIG4','VTCN1','VWA5B1','VWA7','WAS','WASF2','WBP4','WDFY3','WDPCP','WDR1','WDR12','WDR13','WDR53','WDR6','WDR66','WIPF1','WWP1','XIAP','XIRP1','XIRP2','XPC','XPNPEP2','XPO6','XRN1','XYLT1','YES1','YIPF1','YIPF4','YOD1','YTHDC1','ZBED3','ZBTB33','ZBTB8OS','ZDHHC2','ZDHHC20','ZDHHC22','ZDHHC5','ZFP41','ZFP90','ZFYVE28','ZGPAT','ZMYND10','ZNF10','ZNF18','ZNF185','ZNF2','ZNF20','ZNF227','ZNF229','ZNF236','ZNF276','ZNF277','ZNF337','ZNF397','ZNF407','ZNF497','ZNF503','ZNF510','ZNF536','ZNF571','ZNF598','ZNF599','ZNF646','ZNF655','ZNF688','ZNF706','ZNF717','ZNF783','ZNF79','ZNF839','ZSCAN22','ZSCAN29','ZSWIM1','ZSWIM8','ZYX']
MembraneProteins=MembraneProteins1+MembraneProteins2

TFs1=['AC008770.3','AC023509.3','AC092835.1','AC138696.1','ADNP','ADNP2','AEBP1','AEBP2','AHCTF1','AHDC1','AHR','AHRR','AIRE','AKAP8','AKAP8L','AKNA','ALX1','ALX3','ALX4','ANHX','ANKZF1','AR','ARGFX','ARHGAP35','ARID2','ARID3A','ARID3B','ARID3C','ARID5A','ARID5B','ARNT','ARNT2','ARNTL','ARNTL2','ARX','ASCL1','ASCL2','ASCL3','ASCL4','ASCL5','ASH1L','ATF1','ATF2','ATF3','ATF4','ATF5','ATF6','ATF6B','ATF7','ATMIN','ATOH1','ATOH7','ATOH8','BACH1','BACH2','BARHL1','BARHL2','BARX1','BARX2','BATF','BATF2','BATF3','BAZ2A','BAZ2B','BBX','BCL11A','BCL11B','BCL6','BCL6B','BHLHA15','BHLHA9','BHLHE22','BHLHE23','BHLHE40','BHLHE41','BNC1','BNC2','BORCS8-MEF2B','BPTF','BRF2','BSX','C11orf95','CAMTA1','CAMTA2','CARF','CASZ1','CBX2','CC2D1A','CCDC169-SOHLH2','CCDC17','CDC5L','CDX1','CDX2','CDX4','CEBPA','CEBPB','CEBPD','CEBPE','CEBPG','CEBPZ','CENPA','CENPB','CENPBD1','CENPS','CENPT','CENPX','CGGBP1','CHAMP1','CHCHD3','CIC','CLOCK','CPEB1','CPXCR1','CREB1','CREB3','CREB3L1','CREB3L2','CREB3L3','CREB3L4','CREB5','CREBL2','CREBZF','CREM','CRX','CSRNP1','CSRNP2','CSRNP3','CTCF','CTCFL','CUX1','CUX2','CXXC1','CXXC4','CXXC5','DACH1','DACH2','DBP','DBX1','DBX2','DDIT3','DEAF1','DLX1','DLX2','DLX3','DLX4','DLX5','DLX6','DMBX1','DMRT1','DMRT2','DMRT3','DMRTA1','DMRTA2','DMRTB1','DMRTC2','DMTF1','DNMT1','DNTTIP1','DOT1L','DPF1','DPF3','DPRX','DR1','DRAP1','DRGX','DUX1','DUX3','DUX4','DUXA','DZIP1','E2F1','E2F2','E2F3','E2F4','E2F5','E2F6','E2F7','E2F8','E4F1','EBF1','EBF2','EBF3','EBF4','EEA1','EGR1','EGR2','EGR3','EGR4','EHF','ELF1','ELF2','ELF3','ELF4','ELF5','ELK1','ELK3','ELK4','EMX1','EMX2','EN1','EN2','EOMES','EPAS1','ERF','ERG','ESR1','ESR2','ESRRA','ESRRB','ESRRG','ESX1','ETS1','ETS2','ETV1','ETV2','ETV3','ETV3L','ETV4','ETV5','ETV6','ETV7','EVX1','EVX2','FAM170A','FAM200B','FBXL19','FERD3L','FEV','FEZF1','FEZF2','FIGLA','FIZ1','FLI1','FLYWCH1','FOS','FOSB','FOSL1','FOSL2','FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXN4','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1','GABPA','GATA1','GATA2','GATA3','GATA4','GATA5','GATA6','GATAD2A','GATAD2B','GBX1','GBX2','GCM1','GCM2','GFI1','GFI1B','GLI1','GLI2','GLI3','GLI4','GLIS1','GLIS2','GLIS3','GLMP','GLYR1','GMEB1','GMEB2','GPBP1','GPBP1L1','GRHL1','GRHL2','GRHL3','GSC','GSC2','GSX1','GSX2','GTF2B','GTF2I','GTF2IRD1','GTF2IRD2','GTF2IRD2B','GTF3A','GZF1','HAND1','HAND2','HBP1','HDX','HELT','HES1','HES2','HES3','HES4','HES5','HES6','HES7','HESX1','HEY1','HEY2','HEYL','HHEX','HIC1','HIC2','HIF1A','HIF3A','HINFP','HIVEP1','HIVEP2','HIVEP3','HKR1','HLF','HLX','HMBOX1','HMG20A','HMG20B','HMGA1','HMGA2','HMGN3','HMX1','HMX2','HMX3','HNF1A','HNF1B','HNF4A','HNF4G','HOMEZ','HOXA1','HOXA10','HOXA11','HOXA13','HOXA2','HOXA3','HOXA4','HOXA5','HOXA6','HOXA7','HOXA9','HOXB1','HOXB13','HOXB2','HOXB3','HOXB4','HOXB5','HOXB6','HOXB7','HOXB8','HOXB9','HOXC10','HOXC11','HOXC12','HOXC13','HOXC4','HOXC5','HOXC6','HOXC8','HOXC9','HOXD1','HOXD10','HOXD11','HOXD12','HOXD13','HOXD3','HOXD4','HOXD8','HOXD9','HSF1','HSF2','HSF4','HSF5','HSFX1','HSFX2','HSFY1','HSFY2','IKZF1','IKZF2','IKZF3','IKZF4','IKZF5','INSM1','INSM2','IRF1','IRF2','IRF3','IRF4','IRF5','IRF6','IRF7','IRF8','IRF9','IRX1','IRX2','IRX3','IRX4','IRX5','IRX6','ISL1','ISL2','ISX','JAZF1','JDP2','JRK','JRKL','JUN','JUNB','JUND','KAT7','KCMF1','KCNIP3','KDM2A','KDM2B','KDM5B','KIN','KLF1','KLF10','KLF11','KLF12','KLF13','KLF14','KLF15','KLF16','KLF17','KLF2','KLF3','KLF4','KLF5','KLF6','KLF7','KLF8','KLF9','KMT2A','KMT2B','L3MBTL1','L3MBTL3','L3MBTL4','LBX1','LBX2','LCOR','LCORL','LEF1','LEUTX','LHX1','LHX2','LHX3','LHX4','LHX5','LHX6','LHX8','LHX9','LIN28A','LIN28B','LIN54','LMX1A','LMX1B','LTF','LYL1','MAF','MAFA','MAFB','MAFF','MAFG','MAFK','MAX','MAZ','MBD1','MBD2','MBD3','MBD4','MBD6','MBNL2','MECOM','MECP2','MEF2A','MEF2B','MEF2C','MEF2D','MEIS1','MEIS2','MEIS3','MEOX1','MEOX2','MESP1','MESP2','MGA','MITF','MIXL1','MKX','MLX','MLXIP','MLXIPL','MNT','MNX1','MSANTD1','MSANTD3','MSANTD4','MSC','MSGN1','MSX1','MSX2','MTERF1','MTERF2','MTERF3','MTERF4','MTF1','MTF2','MXD1','MXD3','MXD4','MXI1','MYB','MYBL1','MYBL2','MYC','MYCL','MYCN','MYF5','MYF6','MYNN','MYOD1','MYOG','MYPOP','MYRF','MYRFL','MYSM1','MYT1','MYT1L','MZF1','NACC2','NAIF1','NANOG','NANOGNB','NANOGP8','NCOA1','NCOA2','NCOA3','NEUROD1','NEUROD2','NEUROD4','NEUROD6','NEUROG1','NEUROG2','NEUROG3','NFAT5','NFATC1','NFATC2','NFATC3','NFATC4','NFE2','NFE2L1','NFE2L2','NFE2L3','NFE4','NFIA','NFIB','NFIC','NFIL3','NFIX','NFKB1','NFKB2','NFX1','NFXL1','NFYA','NFYB','NFYC','NHLH1','NHLH2','NKRF','NKX1-1','NKX1-2','NKX2-1','NKX2-2','NKX2-3','NKX2-4','NKX2-5','NKX2-6','NKX2-8','NKX3-1','NKX3-2','NKX6-1','NKX6-2','NKX6-3','NME2','NOBOX','NOTO','NPAS1','NPAS2','NPAS3','NPAS4','NR0B1','NR1D1','NR1D2','NR1H2','NR1H3','NR1H4','NR1I2','NR1I3','NR2C1','NR2C2','NR2E1','NR2E3','NR2F1','NR2F2','NR2F6','NR3C1','NR3C2','NR4A1','NR4A2','NR4A3','NR5A1','NR5A2','NR6A1','NRF1','NRL','OLIG1','OLIG2','OLIG3','ONECUT1','ONECUT2','ONECUT3','OSR1','OSR2','OTP','OTX1','OTX2','OVOL1','OVOL2','OVOL3','PA2G4','PATZ1','PAX1','PAX2','PAX3','PAX4','PAX5','PAX6','PAX7','PAX8','PAX9','PBX1','PBX2','PBX3','PBX4','PCGF2','PCGF6','PDX1','PEG3','PGR','PHF1','PHF19 ','PHF20','PHF21A','PHOX2A','PHOX2B','PIN1','PITX1','PITX2','PITX3','PKNOX1','PKNOX2','PLAG1','PLAGL1','PLAGL2','PLSCR1','POGK','POU1F1','POU2AF1','POU2F1','POU2F2','POU2F3','POU3F1','POU3F2','POU3F3','POU3F4','POU4F1','POU4F2','POU4F3','POU5F1','POU5F1B','POU5F2','POU6F1','POU6F2','PPARA','PPARD','PPARG','PRDM1','PRDM10','PRDM12','PRDM13','PRDM14','PRDM15','PRDM16','PRDM2','PRDM4','PRDM5','PRDM6','PRDM8','PRDM9','PREB','PRMT3','PROP1','PROX1','PROX2','PRR12','PRRX1','PRRX2','PTF1A','PURA','PURB','PURG','RAG1','RARA','RARB','RARG','RAX','RAX2','RBAK','RBCK1','RBPJ','RBPJL','RBSN','REL','RELA','RELB','REPIN1','REST','REXO4','RFX1','RFX2','RFX3','RFX4','RFX5','RFX6','RFX7','RFX8','RHOXF1','RHOXF2','RHOXF2B','RLF','RORA','RORB','RORC','RREB1','RUNX1','RUNX2','RUNX3','RXRA','RXRB','RXRG','SAFB','SAFB2','SALL1','SALL2','SALL3','SALL4','SATB1','SATB2','SCMH1','SCML4','SCRT1','SCRT2','SCX','SEBOX','SETBP1','SETDB1','SETDB2','SGSM2','SHOX','SHOX2','SIM1','SIM2','SIX1','SIX2','SIX3','SIX4','SIX5','SIX6','SKI','SKIL','SKOR1','SKOR2','SLC2A4RG','SMAD1','SMAD3','SMAD4','SMAD5','SMAD9','SMYD3','SNAI1','SNAI2','SNAI3','SNAPC2','SNAPC4','SNAPC5','SOHLH1','SOHLH2','SON','SOX1','SOX10','SOX11']
TFs2=['SOX12','SOX13','SOX14','SOX15','SOX17','SOX18','SOX2','SOX21','SOX3','SOX30','SOX4','SOX5','SOX6','SOX7','SOX8','SOX9','SP1','SP100','SP110','SP140','SP140L','SP2','SP3','SP4','SP5','SP6','SP7','SP8','SP9','SPDEF','SPEN','SPI1','SPIB','SPIC','SPZ1','SRCAP','SREBF1','SREBF2','SRF','SRY','ST18','STAT1','STAT2','STAT3','STAT4','STAT5A','STAT5B','STAT6','T','TAL1','TAL2','TBP','TBPL1','TBPL2','TBR1','TBX1','TBX10','TBX15','TBX18','TBX19','TBX2','TBX20','TBX21','TBX22','TBX3','TBX4','TBX5','TBX6','TCF12','TCF15','TCF20','TCF21','TCF23','TCF24','TCF3','TCF4','TCF7','TCF7L1','TCF7L2','TCFL5','TEAD1','TEAD2','TEAD3','TEAD4','TEF','TERB1','TERF1','TERF2','TET1','TET2','TET3','TFAP2A','TFAP2B','TFAP2C','TFAP2D','TFAP2E','TFAP4','TFCP2','TFCP2L1','TFDP1','TFDP2','TFDP3','TFE3','TFEB','TFEC','TGIF1','TGIF2','TGIF2LX','TGIF2LY','THAP1','THAP10','THAP11','THAP12','THAP2','THAP3','THAP4','THAP5','THAP6','THAP7','THAP8','THAP9','THRA','THRB','THYN1','TIGD1','TIGD2','TIGD3','TIGD4','TIGD5','TIGD6','TIGD7','TLX1','TLX2','TLX3','TMF1','TOPORS','TP53','TP63','TP73','TPRX1','TRAFD1','TRERF1','TRPS1','TSC22D1','TSHZ1','TSHZ2','TSHZ3','TTF1','TWIST1','TWIST2','UBP1','UNCX','USF1','USF2','USF3','VAX1','VAX2','VDR','VENTX','VEZF1','VSX1','VSX2','WIZ','WT1','XBP1','XPA','YBX1','YBX2','YBX3','YY1','YY2','ZBED1','ZBED2','ZBED3','ZBED4','ZBED5','ZBED6','ZBED9','ZBTB1','ZBTB10','ZBTB11','ZBTB12','ZBTB14','ZBTB16','ZBTB17','ZBTB18','ZBTB2','ZBTB20','ZBTB21','ZBTB22','ZBTB24','ZBTB25','ZBTB26','ZBTB3','ZBTB32','ZBTB33','ZBTB34','ZBTB37','ZBTB38','ZBTB39','ZBTB4','ZBTB40','ZBTB41','ZBTB42','ZBTB43','ZBTB44','ZBTB45','ZBTB46','ZBTB47','ZBTB48','ZBTB49','ZBTB5','ZBTB6','ZBTB7A','ZBTB7B','ZBTB7C','ZBTB8A','ZBTB8B','ZBTB9','ZC3H8','ZEB1','ZEB2','ZFAT','ZFHX2','ZFHX3','ZFHX4','ZFP1','ZFP14','ZFP2','ZFP28','ZFP3','ZFP30','ZFP37','ZFP41','ZFP42','ZFP57','ZFP62','ZFP64','ZFP69','ZFP69B','ZFP82','ZFP90','ZFP91','ZFP92','ZFPM1','ZFPM2','ZFX','ZFY','ZGLP1','ZGPAT','ZHX1','ZHX2','ZHX3','ZIC1','ZIC2','ZIC3','ZIC4','ZIC5','ZIK1','ZIM2','ZIM3','ZKSCAN1','ZKSCAN2','ZKSCAN3','ZKSCAN4','ZKSCAN5','ZKSCAN7','ZKSCAN8','ZMAT1','ZMAT4','ZNF10','ZNF100','ZNF101','ZNF107','ZNF112','ZNF114','ZNF117','ZNF12','ZNF121','ZNF124','ZNF131','ZNF132','ZNF133','ZNF134','ZNF135','ZNF136','ZNF138','ZNF14','ZNF140','ZNF141','ZNF142','ZNF143','ZNF146','ZNF148','ZNF154','ZNF155','ZNF157','ZNF16','ZNF160','ZNF165','ZNF169','ZNF17','ZNF174','ZNF175','ZNF177','ZNF18','ZNF180','ZNF181','ZNF182','ZNF184','ZNF189','ZNF19','ZNF195','ZNF197','ZNF2','ZNF20','ZNF200','ZNF202','ZNF205','ZNF207','ZNF208','ZNF211','ZNF212','ZNF213','ZNF214','ZNF215','ZNF217','ZNF219','ZNF22','ZNF221','ZNF222','ZNF223','ZNF224','ZNF225','ZNF226','ZNF227','ZNF229','ZNF23','ZNF230','ZNF232','ZNF233','ZNF234','ZNF235','ZNF236','ZNF239','ZNF24','ZNF248','ZNF25','ZNF250','ZNF251','ZNF253','ZNF254','ZNF256','ZNF257','ZNF26','ZNF260','ZNF263','ZNF264','ZNF266','ZNF267','ZNF268','ZNF273','ZNF274','ZNF275','ZNF276','ZNF277','ZNF28','ZNF280A','ZNF280B','ZNF280C','ZNF280D','ZNF281','ZNF282','ZNF283','ZNF284','ZNF285','ZNF286A','ZNF286B','ZNF287','ZNF292','ZNF296','ZNF3','ZNF30','ZNF300','ZNF302','ZNF304','ZNF311','ZNF316','ZNF317','ZNF318','ZNF319','ZNF32','ZNF320','ZNF322','ZNF324','ZNF324B','ZNF326','ZNF329','ZNF331','ZNF333','ZNF334','ZNF335','ZNF337','ZNF33A','ZNF33B','ZNF34','ZNF341','ZNF343','ZNF345','ZNF346','ZNF347','ZNF35','ZNF350','ZNF354A','ZNF354B','ZNF354C','ZNF358','ZNF362','ZNF365','ZNF366','ZNF367','ZNF37A','ZNF382','ZNF383','ZNF384','ZNF385A','ZNF385B','ZNF385C','ZNF385D','ZNF391','ZNF394','ZNF395','ZNF396','ZNF397','ZNF398','ZNF404','ZNF407','ZNF408','ZNF41','ZNF410','ZNF414','ZNF415','ZNF416','ZNF417','ZNF418','ZNF419','ZNF420','ZNF423','ZNF425','ZNF426','ZNF428','ZNF429','ZNF43','ZNF430','ZNF431','ZNF432','ZNF433','ZNF436','ZNF438','ZNF439','ZNF44','ZNF440','ZNF441','ZNF442','ZNF443','ZNF444','ZNF445','ZNF446','ZNF449','ZNF45','ZNF451','ZNF454','ZNF460','ZNF461','ZNF462','ZNF467','ZNF468','ZNF469','ZNF470','ZNF471','ZNF473','ZNF474','ZNF479','ZNF48','ZNF480','ZNF483','ZNF484','ZNF485','ZNF486','ZNF487','ZNF488','ZNF490','ZNF491','ZNF492','ZNF493','ZNF496','ZNF497','ZNF500','ZNF501','ZNF502','ZNF503','ZNF506','ZNF507','ZNF510','ZNF511','ZNF512','ZNF512B','ZNF513','ZNF514','ZNF516','ZNF517','ZNF518A','ZNF518B','ZNF519','ZNF521','ZNF524','ZNF525','ZNF526','ZNF527','ZNF528','ZNF529','ZNF530','ZNF532','ZNF534','ZNF536','ZNF540','ZNF541','ZNF543','ZNF544','ZNF546','ZNF547','ZNF548','ZNF549','ZNF550','ZNF551','ZNF552','ZNF554','ZNF555','ZNF556','ZNF557','ZNF558','ZNF559','ZNF560','ZNF561','ZNF562','ZNF563','ZNF564','ZNF565','ZNF566','ZNF567','ZNF568','ZNF569','ZNF57','ZNF570','ZNF571','ZNF572','ZNF573','ZNF574','ZNF575','ZNF576','ZNF577','ZNF578','ZNF579','ZNF580','ZNF581','ZNF582','ZNF583','ZNF584','ZNF585A','ZNF585B','ZNF586','ZNF587','ZNF587B','ZNF589','ZNF592','ZNF594','ZNF595','ZNF596','ZNF597','ZNF598','ZNF599','ZNF600','ZNF605','ZNF606','ZNF607','ZNF608','ZNF609','ZNF610','ZNF611','ZNF613','ZNF614','ZNF615','ZNF616','ZNF618','ZNF619','ZNF620','ZNF621','ZNF623','ZNF624','ZNF625','ZNF626','ZNF627','ZNF628','ZNF629','ZNF630','ZNF639','ZNF641','ZNF644','ZNF645','ZNF646','ZNF648','ZNF649','ZNF652','ZNF653','ZNF654','ZNF655','ZNF658','ZNF66','ZNF660','ZNF662','ZNF664','ZNF665','ZNF667','ZNF668','ZNF669','ZNF670','ZNF671','ZNF672','ZNF674','ZNF675','ZNF676','ZNF677','ZNF678','ZNF679','ZNF680','ZNF681','ZNF682','ZNF683','ZNF684','ZNF687','ZNF688','ZNF689','ZNF69','ZNF691','ZNF692','ZNF695','ZNF696','ZNF697','ZNF699','ZNF7','ZNF70','ZNF700','ZNF701','ZNF703','ZNF704','ZNF705A','ZNF705B','ZNF705D','ZNF705E','ZNF705G','ZNF706','ZNF707','ZNF708','ZNF709','ZNF71','ZNF710','ZNF711','ZNF713','ZNF714','ZNF716','ZNF717','ZNF718','ZNF721','ZNF724','ZNF726','ZNF727','ZNF728','ZNF729','ZNF730','ZNF732','ZNF735','ZNF736','ZNF737','ZNF74','ZNF740','ZNF746','ZNF747','ZNF749','ZNF750','ZNF75A','ZNF75D','ZNF76','ZNF761','ZNF763','ZNF764','ZNF765','ZNF766','ZNF768','ZNF77','ZNF770','ZNF771','ZNF772','ZNF773','ZNF774','ZNF775','ZNF776','ZNF777','ZNF778','ZNF780A','ZNF780B','ZNF781','ZNF782','ZNF783','ZNF784','ZNF785','ZNF786','ZNF787','ZNF788','ZNF789','ZNF79','ZNF790','ZNF791','ZNF792','ZNF793','ZNF799','ZNF8','ZNF80','ZNF800','ZNF804A','ZNF804B','ZNF805','ZNF808','ZNF81','ZNF813','ZNF814','ZNF816','ZNF821','ZNF823','ZNF827','ZNF829','ZNF83','ZNF830','ZNF831','ZNF835','ZNF836','ZNF837','ZNF84','ZNF841','ZNF843','ZNF844','ZNF845','ZNF846','ZNF85','ZNF850','ZNF852','ZNF853','ZNF860','ZNF865','ZNF878','ZNF879','ZNF880','ZNF883','ZNF888','ZNF891','ZNF90','ZNF91','ZNF92','ZNF93','ZNF98','ZNF99','ZSCAN1','ZSCAN10','ZSCAN12','ZSCAN16','ZSCAN18','ZSCAN2','ZSCAN20','ZSCAN21','ZSCAN22','ZSCAN23','ZSCAN25','ZSCAN26','ZSCAN29','ZSCAN30','ZSCAN31','ZSCAN32','ZSCAN4','ZSCAN5A','ZSCAN5B','ZSCAN5C','ZSCAN9','ZUFSP','ZXDA','ZXDB','ZXDC','ZZZ3']
TFs=TFs1+TFs2

#TFs other, inclusive of the above list, not necessarily DNA binding proteins but affect transcription
TFs1Other=['AATF','ABL1','ABRA','AC008770.3','AC020915.1','AC023509.3','AC092835.1','AC105001.2','AC138696.1','ACOD1','ACTR3B','ACTR3C','ADAM8','ADAMTS17','ADAMTS19','ADAR','ADGRA1','ADGRG3','ADNP','ADNP2','AEBP1','AEBP2','AES','AFAP1 ','AFF1','AFF3','AFF4','AGER','AGT','AHCTF1','AHDC1','AHR','AHRR','AIM2','AIRE','AKAP8','AKAP8L','AKNA','AKT1','ALK','ALKBH2','ALX1','ALX3','ALX4','AMH','ANAPC2','ANHX','ANKRD30A','ANKZF1','ANXA3','ANXA4','AP5Z1','APBB3','APC2','APEX1','APOA4','APOBEC3A','APOBEC3B','APOBEC3G','APTX','AR','ARGFX','ARHGAP35','ARHGEF2','ARHGEF5','ARID1A','ARID1B','ARID2','ARID3A','ARID3B','ARID3C','ARID4A','ARID4B','ARID5A','ARID5B','ARNT','ARNT2','ARNTL','ARNTL2','ARRB1','ARRB2','ARX','ASCC1','ASCL1','ASCL2','ASCL3','ASCL4','ASCL5','ASH1L','ATF1','ATF2','ATF3','ATF4','ATF5','ATF6','ATF6B','ATF7','ATF7IP','ATM','ATMIN','ATOH1','ATOH7','ATOH8','ATR','ATRX','ATXN7','BACH1','BACH2','BAG1','BANF1','BANP','BARHL1','BARHL2','BARX1','BARX2','BATF','BATF2','BATF3','BAZ2A','BAZ2B','BBX','BCL10','BCL11A','BCL11B','BCL3','BCL6','BCL6B','BCLAF1','BCOR','BDP1','BEND4','BEX1','BHLHA15','BHLHA9','BHLHE22','BHLHE23','BHLHE40','BHLHE41','BHMG1','BIN1','BLM','BLZF1','BMI1','BMP2','BMP4','BMP7','BMPR1A','BNC1','BNC2','BOD1L1','BOLA1','BOLA2','BOLA2B','BOLA3','BORCS8-MEF2B','BPNT1','BPTF','BRCA1','BRD4','BRD7','BRD8','BRD9','BRF1','BRF2','BRMS1','BRPF1','BSX','BTAF1','BTBD8','BTG2','BTK','BTRC','BUD31','C11orf95','C14orf39','C17orf49','C1D','C20orf194','C3orf33','CACTIN','CAMK1D','CAMK2A','CAMTA1','CAMTA2','CAPN15','CAPN3','CARD11','CARD14','CARF','CARHSP1','CASP8AP2','CASZ1','CAT','CBFA2T2','CBFA2T3','CBFB','CBL','CBLL1','CBX2','CC2D1A','CC2D1B','CCDC124','CCDC169-SOHLH2','CCDC17','CCDC3','CCDC71','CCDC83','CCNT1','CCNT2','CD36','CD40','CD40LG','CDC5L','CDC6','CDK5RAP3','CDK7','CDK8','CDK9','CDKN1A','CDKN2A','CDX1','CDX2','CDX4','CEBPA','CEBPB','CEBPD','CEBPE','CEBPG','CEBPZ','CELF3','CENPA','CENPB','CENPBD1','CENPS','CENPT','CENPX','CERS2','CERS3','CERS4','CERS5','CERS6','CFLAR','CGGBP1','CHAMP1','CHCHD3','CHD1','CHD2','CHD3','CHD4','CHD6','CHD7','CHD9','CHP1','CHRAC1','CHUK','CHURC1','CIB1','CIC','CIITA','CIR1','CITED1','CITED2','CITED4','CIZ1','CLOCK','CLU','CMKLR1','CNBP','CNOT3','CNOT4','CNOT7','CNOT8','COMMD1','COMMD6','COMMD7','COPS2','COPS3','COPS4','CPEB1','CPSF4','CPSF4L','CPXCR1','CRAMP1','CREB1','CREB3','CREB3L1','CREB3L2','CREB3L3','CREB3L4','CREB5','CREBBP','CREBL2','CREBRF','CREBZF','CREG1','CREM','CRTC1','CRTC2','CRTC3','CRX','CSDC2','CSDE1','CSRNP1','CSRNP2','CSRNP3','CTBP1','CTBP2','CTCF','CTCFL','CTH','CTNNB1','CTNNBIP1','CUL1','CUL2','CUL3','CUL4A','CUL4B','CUL5','CUX1','CUX2','CXXC1','CXXC4','CXXC5','CYLD','CYP1B1','CYTL1','DAB2IP','DACH1','DACH2','DAP','DAXX','DBP','DBX1','DBX2','DCP1A','DDB1','DDB2','DDIT3','DDN','DDR2','DDRGK1','DDX3X','DDX5','DDX58','DEAF1','DEK','DEPDC1','DEPDC1B','DEPDC4','DEPDC5','DEPDC7','DEPTOR','DHX34','DHX36','DHX57','DISP3','DLX1','DLX2','DLX3','DLX4','DLX5','DLX6','DMAP1','DMBX1','DMC1','DMRT1','DMRT2','DMRT3','DMRTA1','DMRTA2','DMRTB1','DMRTC1','DMRTC2','DMTF1','DNA2','DNAJA3','DNAJC1','DNAJC2','DNAJC21','DNMT1','DNMT3A','DNMT3B','DNMT3L','DNTT','DNTTIP1','DOT1L','DPF1','DPF2','DPF3','DPRX','DR1','DRAP1','DRGX','DSP','DTX1','DUS3L','DUSP12','DUX1','DUX3','DUX4','DUXA','DVL1','DVL2','DVL3','DZIP1','DZIP1L','E2F1','E2F2','E2F3','E2F4','E2F5','E2F6','E2F7','E2F8','E4F1','EAF2','EBF1','EBF2','EBF3','EBF4','EDA','EDA2R','EDF1','EDN1','EEA1','EED','EGLN1','EGR1','EGR2','EGR3','EGR4','EHF','EID1','EID2','EIF2AK2','EIF2AK4','EIF3C','EIF3CL','EIF3K','ELF1','ELF2','ELF3','ELF4','ELF5','ELK1','ELK3','ELK4','ELMSAN1','EME1','EMX1','EMX2','EN1','EN2','ENO1','EOMES','EP300','EP400','EPAS1','EPHA5','EPM2AIP1','ERBIN','ERC1','ERCC3','ERCC4','ERF','ERG','ESR1','ESR2','ESRRA','ESRRB','ESRRG','ESX1','ETS1','ETS2','ETV1','ETV2','ETV3','ETV3L','ETV4','ETV5','ETV6','ETV7','EVX1','EVX2','EWSR1','EXO1','EXOC2','EXOG','EZH1','EZH2','FAAP24','FAM153C','FAM170A','FAM171B','FAM200A','FAM200B','FAN1','FANCA','FANCD2','FANK1','FARSA','FARSB','FBN1','FBXL19','FBXO41','FEN1','FER','FERD3L','FEV','FEZF1','FEZF2','FGD1','FHL2','FIGLA','FIZ1','FLI1','FLNA','FLYWCH1','FMNL2','FOS','FOSB','FOSL1','FOSL2','FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L2','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXN4','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1','FUBP1','FUBP3','FUS','FZD1','FZD2','FZD4','FZD6','GABPA','GABPB1','GABPB2','GADD45A','GAS6','GAS7','GATA1','GATA2','GATA3','GATA4','GATA5','GATA6','GATAD1','GATAD2A','GATAD2B','GBX1','GBX2','GCFC2','GCM1','GCM2','GEN1','GFI1','GFI1B','GLA','GLI1','GLI2','GLI3','GLI4','GLIS1','GLIS2','GLIS3','GLMP','GLYR1','GMEB1','GMEB2','GOLGB1','GON4L','GPATCH8','GPBP1','GPBP1L1','GPER1','GPR155','GREM1','GRHL1','GRHL2','GRHL3','GRM6','GSC','GSC2','GSX1','GSX2','GTF2A1','GTF2A1L','GTF2A2','GTF2B','GTF2E2','GTF2F1','GTF2F2','GTF2H1','GTF2H2','GTF2H2C','GTF2H3','GTF2H4','GTF2I','GTF2IRD1','GTF2IRD2','GTF2IRD2B','GTF3A','GTF3C1','GTF3C2','GULP1','GZF1','H1F0','H1FOO','H1FX','HAND1','HAND2','HAVCR2','HBP1','HCFC1','HCK','HDAC1','HDAC10','HDAC2','HDAC3','HDAC4','HDAC5','HDAC6','HDAC7','HDAC8','HDAC9','HDX','HELT','HELZ','HELZ2','HES1','HES2','HES3','HES4','HES5','HES6','HES7','HESX1','HEY1','HEY2','HEYL','HHEX','HIC1','HIC2','HIF1A','HIF3A','HINFP','HIPK2','HIRA','HIST1H1A','HIST1H1B','HIST1H1C','HIST1H1D','HIST1H1E','HIST1H1T','HIVEP1','HIVEP2','HIVEP3','HKR1','HLA-DQB1','HLA-DQB2','HLA-DRB1','HLA-DRB3','HLA-DRB5','HLF','HLTF','HLX','HMBOX1','HMG20A','HMG20B','HMGA1','HMGA2','HMGB1','HMGB2','HMGB3','HMGB4','HMGN1','HMGN3','HMGXB3','HMGXB4','HMOX1','HMX1','HMX2','HMX3','HNF1A','HNF1B','HNF4A','HNF4G','HNRNPA1','HNRNPA3','HNRNPAB','HNRNPCL2','HNRNPD','HNRNPDL','HNRNPK','HNRNPL','HNRNPU','HOMEZ','HOOK2','HOPX','HOXA1','HOXA10','HOXA11','HOXA13','HOXA2','HOXA3','HOXA4','HOXA5','HOXA6','HOXA7','HOXA9','HOXB1','HOXB13','HOXB2','HOXB3','HOXB4','HOXB5','HOXB6','HOXB7','HOXB8','HOXB9','HOXC10','HOXC11','HOXC12','HOXC13','HOXC4','HOXC5','HOXC6','HOXC8','HOXC9','HOXD1','HOXD10','HOXD11','HOXD12','HOXD13','HOXD3','HOXD4','HOXD8','HOXD9','HP1BP3','HR','HSBP1','HSF1','HSF2','HSF4','HSF5','HSFX1','HSFX2','HSFY1','HSFY2','HSPA1A','HSPA1B','ICAM1','ID1','ID2','ID3','ID4','IFI16','IFNB1','IGHM','IKBKB','IKBKG','IKZF1','IKZF2','IKZF3','IKZF4','IKZF5','IL10','IL1B','IL1RAP','IL4','IL5','IL6','INF2','ING4','INS','INSM1','INSM2','IQGAP1','IRAK1','IRAK2','IRAK3','IRF1','IRF2','IRF2BP1','IRF3','IRF4','IRF5','IRF6','IRF7','IRF8','IRF9','IRX1','IRX2','IRX3','IRX4','IRX5','IRX6','ISL1','ISL2','ISX','ITCH','ITGB2','ITGB3BP','IZUMO2','JADE1','JAK2','JARID2','JAZF1','JDP2','JMY','JRK','JRKL','JUN','JUNB','JUND','JUP','KANSL2','KAT2A','KAT2B','KAT5','KAT6A','KAT6B','KAT7','KAT8','KCMF1','KCNIP3','KDM1A','KDM2A','KDM2B','KDM3A','KDM5A','KDM5B','KDM5C','KDM5D','KEAP1','KIAA1549','KIAA1683','KIAA1841','KIAA1958','KIF15','KIN','KIT','KLF1','KLF10','KLF11','KLF12','KLF13','KLF14','KLF15','KLF16','KLF17','KLF2','KLF3','KLF4','KLF5','KLF6','KLF7','KLF8','KLF9','KMT2A','KMT2B','KMT2C','KMT2D','KNTC1','KRAS','KRBOX1','KRT13','KRTAP5-1','KRTAP5-9','KRTAP9-9','L3MBTL1','L3MBTL3','L3MBTL4','LARP1','LARP1B','LARP4','LARP4B','LARP6','LARP7','LBX1','LBX2','LCOR','LCORL','LEF1','LEUTX','LGALS9','LGR4','LHX1','LHX2','LHX3','LHX4','LHX5','LHX6','LHX8','LHX9','LIG1','LIG3','LIN28A','LIN28B','LIN54','LMO2','LMO4','LMX1A','LMX1B','LPP','LRP5','LRP6','LRP8','LRRFIP1','LRRFIP2','LSR','LTF','LTN1','LYAR','LYL1','LZTFL1','LZTR1','LZTS1','MACC1','MACF1','MAD2L2','MAEL','MAF','MAFA','MAFB','MAFF','MAFG','MAFK','MALT1','MAML1','MAML3','MAP2K5','MAP3K10','MAP3K13','MAP3K7','MAPK1','MAPK10','MAPK11','MAPK14','MAPK3','MAPK8','MAPK9','MATR3','MAVS','MAX','MAZ','MB21D1','MBD1','MBD2','MBD3','MBD4','MBD6','MBNL1','MBNL2','MBNL3','MBTPS2','MDFI','MDM2','MDM4','MECOM','MECP2','MED1','MED12','MED13','MED13L','MED14','MED15','MED16','MED17','MED21','MED23','MED24','MED6','MEF2A','MEF2B','MEF2C','MEF2D','MEIS1','MEIS2','MEIS3','MEN1','MEOX1','MEOX2','MESP1','MESP2','MET','MGA','MGMT','MID2','MIER1','MIER2','MIER3','MINK1','MIS18BP1','MITF','MIXL1','MKL1','MKL2','MKRN1','MKRN2','MKRN3','MKX','MLLT1','MLLT10','MLLT11','MLLT3','MLLT6','MLX','MLXIP','MLXIPL','MN1','MNAT1','MNT','MNX1','MORC3','MPG','MRPL28','MRRF','MSANTD1','MSANTD3','MSANTD4','MSC','MSGN1','MSH2','MSH3','MSH6','MSL3','MSRB2','MST1R','MSX1','MSX2','MTA1','MTA2','MTA3','MTDH','MTERF1','MTERF2','MTERF3','MTERF4','MTF1','MTF2','MTPN','MUS81','MXD1','MXD3','MXD4','MXI1','MYB','MYBBP1A','MYBL1','MYBL2','MYC','MYCBP','MYCL','MYCN','MYD88','MYEF2','MYF5','MYF6','MYNN','MYOCD','MYOD1','MYOG','MYPOP','MYRF','MYRFL','MYSM1','MYT1','MYT1L','MZF1','NAB2','NABP2','NACC2','NAIF1','NANOG','NANOGNB','NANOGP8','NCOA1','NCOA2','NCOA3','NCOA4','NCOA5','NCOA6','NCOA7','NCOR1','NCOR2','NDC80','NDN','NDP','NEB','NEIL1','NEUROD1','NEUROD2','NEUROD4','NEUROD6','NEUROG1','NEUROG2','NEUROG3','NFAM1','NFAT5','NFATC1','NFATC2','NFATC3','NFATC4','NFE2','NFE2L1','NFE2L2','NFE2L3','NFE4','NFIA','NFIB','NFIC','NFIL3','NFIX','NFKB1','NFKB2','NFKBIA','NFKBIB','NFKBID','NFKBIL1','NFKBIZ','NFRKB','NFX1','NFXL1','NFYA','NFYB','NFYC','NHLH1','NHLH2','NKRF','NKX1-1','NKX1-2','NKX2-1','NKX2-2','NKX2-3','NKX2-4','NKX2-5','NKX2-6','NKX2-8','NKX3-1','NKX3-2','NKX6-1','NKX6-2','NKX6-3','NLRC3','NLRC4','NLRC5','NLRP12','NLRP2B','NLRP3','NME2','NOBOX','NOC3L','NOC4L','NOCT','NOD1','NOD2','NODAL','NONO','NOP2','NOTCH1','NOTCH2','NOTCH3','NOTO','NPAS1','NPAS2','NPAS3','NPAS4','NPM1','NR0B1','NR0B2','NR1D1','NR1D2','NR1H2','NR1H3','NR1H4','NR1I2','NR1I3','NR2C1','NR2C2','NR2E1','NR2E3','NR2F1','NR2F2','NR2F6','NR3C1','NR3C2','NR4A1','NR4A2','NR4A3','NR5A1','NR5A2','NR6A1','NRBF2','NRF1','NRG1','NRIP1','NRIP3','NRK','NRL','NSD1','NSD2','NTRK1','NUCKS1','NUFIP1','NUP98','NUPL2']
TFs2Other=['OGG1','OLIG1','OLIG2','OLIG3','ONECUT1','ONECUT2','ONECUT3','OPRD1','OSR1','OSR2','OTOP3','OTP','OTULIN','OTX1','OTX2','OVOL1','OVOL2','OVOL3','PA2G4','PABPN1','PARK7','PARP1','PARP10','PARP12','PATZ1','PAWR','PAX1','PAX2','PAX3','PAX4','PAX5','PAX6','PAX7','PAX8','PAX9','PAXBP1','PAXIP1','PBRM1','PBX1','PBX2','PBX3','PBX4','PCBD1','PCBP1','PCBP2','PCBP3','PCGF2','PCGF6','PCNA','PCSK6','PDCD7','PDS5B','PDX1','PEG3','PELI1','PER1','PER2','PER3','PEX14','PFDN1','PGBD1','PGR','PGS1','PHB','PHB2','PHF1','PHF19 ','PHF20','PHF21A','PHF5A','PHOX2A','PHOX2B','PHTF1','PHTF2','PIAS1','PIAS2','PIAS3','PIAS4','PIDD1','PIKFYVE','PIM1','PIN1','PINK1','PITX1','PITX2','PITX3','PKHD1','PKHD1L1','PKNOX1','PKNOX2','PLA2G10','PLA2G1B','PLAG1','PLAGL1','PLAGL2','PLEK','PLEK2','PLEKHA4','PLPP3','PLSCR1','PLXNA1','PLXNA2','PLXNA3','PLXNA4','PLXNB1','PLXNB2','PLXNB3','PLXNC1','PLXND1','PML','PMS1','PNKP','POGK','POGZ','POLA1','POLB','POLE3','POLE4','POLG','POLH','POLI','POLK','POLL','POLM','POLN','POLQ','POLR1B','POLR2A','POLR2B','POLR2E','POLR2J','POLR2K','POLR2L','POLR3A','POLRMT','POT1','POU1F1','POU2AF1','POU2F1','POU2F2','POU2F3','POU3F1','POU3F2','POU3F3','POU3F4','POU4F1','POU4F2','POU4F3','POU5F1','POU5F1B','POU5F2','POU6F1','POU6F2','PPARA','PPARD','PPARG','PPARGC1A','PPARGC1B','PPP1R10','PPP1R13L','PPP1R21','PPP2R3B','PPP2R5B','PPP3CA','PPRC1','PQBP1','PRAM1','PRAP1','PRB3','PRDM1','PRDM10','PRDM11','PRDM12','PRDM13','PRDM14','PRDM15','PRDM16','PRDM2','PRDM4','PRDM5','PRDM6','PRDM7','PRDM8','PRDM9','PRDX3','PREB','PREX1','PREX2','PRH2','PRIM2','PRIMPOL','PRKCB','PRKCH','PRKCI','PRKCQ','PRKCZ','PRKD1','PRKD2','PRKN','PRMT2','PRMT3','PRNP','PROP1','PROX1','PROX2','PRR12','PRR3','PRRX1','PRRX2','PSMA6','PSMC5','PSMD10','PSMD11','PSMD12','PSPC1','PTCH1','PTEN','PTF1A','PTGIS','PTH','PTTG1','PUM3','PURA','PURB','PURG','PYCARD','PYDC1','PYDC2','QRICH1','RAB11FIP4','RAB7B','RAD21','RAD51','RAG1','RAI1','RAPGEF3','RAPGEF4','RAPGEF5','RARA','RARB','RARG','RAX','RAX2','RB1','RBAK','RBBP7','RBCK1','RBL1','RBL2','RBM10','RBM20','RBM22','RBM26','RBM27','RBM5','RBM6','RBPJ','RBPJL','RBSN','RC3H1','RC3H2','RCAN1','RCOR1','RCOR2','RCOR3','RECQL','REL','RELA','RELB','RELL2','RELN','REPIN1','RERE','REST','REV1','REXO4','RFC1','RFX1','RFX2','RFX3','RFX4','RFX5','RFX6','RFX7','RFX8','RFXANK','RFXAP','RGCC','RGS11','RGS6','RGS7','RGS9','RHEBL1','RHOXF1','RHOXF2','RHOXF2B','RING1','RIOK2','RIPK1','RIPK2','RIPK3','RIPK4','RLF','RLIM','RNASE2','RNASEH1','RNF113A','RNF113B','RNF114','RNF125','RNF138','RNF14','RNF166','RNF2','RNF219','RNF25','RNF31','RNF4','RNF41','RORA','RORB','RORC','RPA2','RPA4','RPL36A','RPL7','RPL7A','RPS27A','RPS6KA4','RPS6KA5','RREB1','RRN3','RUNX1','RUNX1T1','RUNX2','RUNX3','RWDD3','RXRA','RXRB','RXRG','RYBP','S100A12','S100A8','S100A9','SAFB','SAFB2','SALL1','SALL2','SALL3','SALL4','SART1','SATB1','SATB2','SAV1','SCAND1','SCAND2P','SCAPER','SCMH1','SCML1','SCML2','SCML4','SCOC','SCRT1','SCRT2','SCX','SEBOX','SEMA4A','SERTAD1','SERTAD2','SETBP1','SETD6','SETDB1','SETDB2','SF3A2','SF3A3','SFPQ','SFRP4','SFRP5','SFTPB','SGK1','SGK3','SGSM2','SHH','SHOX','SHOX2','SHPRH','SIGIRR','SIK1','SIM1','SIM2','SIN3A','SIN3B','SIRT1','SIRT2','SIRT3','SIRT6','SIVA1','SIX1','SIX2','SIX3','SIX4','SIX5','SIX6','SKI','SKIDA1','SKIL','SKOR1','SKOR2','SLC22A4','SLC26A10','SLC26A3','SLC2A4RG','SLC30A9','SLC39A10','SLC4A10','SMAD1','SMAD2','SMAD3','SMAD4','SMAD5','SMAD6','SMAD7','SMAD9','SMARCA1','SMARCA2','SMARCA4','SMARCA5','SMARCAL1','SMARCB1','SMARCC1','SMARCC2','SMARCD1','SMARCD2','SMARCD3','SMARCE1','SMC3','SMO','SMUG1','SMYD1','SMYD3','SNAI1','SNAI2','SNAI3','SNAPC2','SNAPC4','SNAPC5','SNRPC','SNURF','SNW1','SOGA3','SOHLH1','SOHLH2','SON','SORBS2','SOX1','SOX10','SOX11','SOX12','SOX13','SOX14','SOX15','SOX17','SOX18','SOX2','SOX21','SOX3','SOX30','SOX4','SOX5','SOX6','SOX7','SOX8','SOX9','SP1','SP100','SP110','SP140','SP140L','SP2','SP3','SP4','SP5','SP6','SP7','SP8','SP9','SPDEF','SPEN','SPHK1','SPI1','SPIB','SPIC','SPZ1','SRCAP','SREBF1','SREBF2','SRF','SRY','SSB','SSBP2','SSBP3','SSBP4','SSH1','SSH2','SSH3','SSRP1','SSX1','ST18','STAG1','STAT1','STAT2','STAT3','STAT4','STAT5A','STAT5B','STAT6','STK16','STK3','STK36','STRN','STRN3','SUB1','SUFU','SUMO1','SUPT16H','SUPT4H1','SUPT5H','SUPT6H','SUV39H1','SUV39H2','SUZ12','SYK','T','TAB1','TAB2','TAB3','TADA2A','TADA2B','TADA3','TAF1','TAF10','TAF11','TAF12','TAF13','TAF1A','TAF1B','TAF1C','TAF1L','TAF2','TAF3','TAF4','TAF4B','TAF5','TAF5L','TAF6','TAF6L','TAF7','TAF7L','TAF8','TAF9','TAF9B','TAL1','TAL2','TARDBP','TAX1BP1','TBL1Y','TBP','TBPL1','TBPL2','TBR1','TBX1','TBX10','TBX15','TBX18','TBX19','TBX2','TBX20','TBX21','TBX22','TBX3','TBX4','TBX5','TBX6','TCEA2','TCEAL1','TCEAL7','TCEAL8','TCF12','TCF15','TCF19','TCF20','TCF21','TCF23','TCF24','TCF25','TCF3','TCF4','TCF7','TCF7L1','TCF7L2','TCFL5','TDG','TDP1','TDP2','TEAD1','TEAD2','TEAD3','TEAD4','TEF','TERB1','TERF1','TERF2','TERF2IP','TESMIN','TET1','TET2','TET3','TFAM','TFAP2A','TFAP2B','TFAP2C','TFAP2D','TFAP2E','TFAP4','TFCP2','TFCP2L1','TFDP1','TFDP2','TFDP3','TFE3','TFEB','TFEC','TGFB1','TGFB1I1','TGIF1','TGIF2','TGIF2LX','TGIF2LY','TGS1','THAP1','THAP10','THAP11','THAP12','THAP2','THAP3','THAP4','THAP5','THAP6','THAP7','THAP8','THAP9','THRA','THRAP3','THRB','THYN1','TIA1','TICAM1','TIGD1','TIGD2','TIGD3','TIGD4','TIGD5','TIGD6','TIGD7','TIPARP','TIRAP','TLE1','TLE2','TLE3','TLE4','TLE6','TLR2','TLR3','TLR4','TLR9','TLX1','TLX2','TLX3','TMEM235','TMF1','TNF','TNFAIP3','TNFRSF11A','TNFRSF4','TNFSF11','TNFSF18','TNFSF4','TOE1','TONSL','TOP1','TOP2A','TOP2B','TOPORS','TOX','TOX2','TOX3','TOX4','TP53','TP53BP1','TP63','TP73','TPRX1','TRADD','TRAF1','TRAF2','TRAF3','TRAF4','TRAF5','TRAF6','TRAFD1','TRAK2','TRAPPC9','TRERF1','TREX1','TRIB1','TRIB3','TRIM13','TRIM14','TRIM15','TRIM21','TRIM22','TRIM23','TRIM24','TRIM25','TRIM26','TRIM27','TRIM28','TRIM29','TRIM3','TRIM31','TRIM32','TRIM33','TRIM34','TRIM37','TRIM38','TRIM40','TRIM5','TRIM52','TRIM62','TRIM8','TRIP11','TRIT1','TRMT1','TRMT1L','TRPS1','TRRAP','TSC2','TSC22D1','TSC22D2','TSC22D3','TSC22D4','TSHZ1','TSHZ2','TSHZ3','TSN','TSSK4','TTF1','TUB','TULP4','TUT1','TWIST1','TWIST2','TXK','U2AF1','U2AF1L4','UBA52','UBB','UBC','UBE2K','UBE2N','UBE2V1','UBN1','UBP1','UBR4','UBTF','UBTFL1','UBTFL6','UFL1','UHRF1','UHRF2','UNCX','UNG','UNK','UNKL','USF1','USF2','USF3','USP39','USP7','UTF1','VAV1','VAX1','VAX2','VDR','VEGFA','VENTX','VEZF1','VGLL1','VGLL2','VGLL3','VHLL','VPS72','VSX1','VSX2','WDHD1','WFS1','WIZ','WNT1','WNT10B','WNT2','WNT3A','WNT5A','WNT8B','WRN','WT1','WWOX','WWP2','WWTR1','XBP1','XCL1','XPA','YBX1','YBX2','YBX3','YEATS2','YEATS4','YOD1','YY1','YY2','ZBED1','ZBED2','ZBED3','ZBED4','ZBED5','ZBED6','ZBED6CL','ZBED8','ZBED9','ZBP1','ZBTB1','ZBTB10','ZBTB11','ZBTB12','ZBTB14','ZBTB16','ZBTB17','ZBTB18','ZBTB2','ZBTB20','ZBTB21','ZBTB22','ZBTB24','ZBTB25','ZBTB26','ZBTB3','ZBTB32','ZBTB33','ZBTB34','ZBTB37','ZBTB38','ZBTB39','ZBTB4','ZBTB40','ZBTB41','ZBTB42','ZBTB43','ZBTB44','ZBTB45','ZBTB46','ZBTB47','ZBTB48','ZBTB49','ZBTB5','ZBTB6','ZBTB7A','ZBTB7B','ZBTB7C','ZBTB8A','ZBTB8B','ZBTB9','ZC3H10','ZC3H11A','ZC3H12A','ZC3H13','ZC3H14','ZC3H15','ZC3H18','ZC3H3','ZC3H4','ZC3H6','ZC3H7A','ZC3H7B','ZC3H8','ZC3HAV1','ZCCHC11','ZCCHC6','ZDHHC11','ZDHHC19','ZEB1','ZEB2','ZFAND5','ZFAT','ZFHX2','ZFHX3','ZFHX4','ZFP1','ZFP14','ZFP2','ZFP28','ZFP3','ZFP30','ZFP36','ZFP36L1','ZFP36L2','ZFP37','ZFP41','ZFP42','ZFP57','ZFP62','ZFP64','ZFP69','ZFP69B','ZFP82','ZFP90','ZFP91','ZFP92','ZFPL1','ZFPM1','ZFPM2','ZFR','ZFR2','ZFX','ZFY','ZFYVE26','ZGLP1','ZGPAT','ZHX1','ZHX2','ZHX3','ZIC1','ZIC2','ZIC3','ZIC4','ZIC5','ZIK1','ZIM2','ZIM3','ZKSCAN1','ZKSCAN2','ZKSCAN3','ZKSCAN4','ZKSCAN5','ZKSCAN7','ZKSCAN8','ZMAT1','ZMAT2','ZMAT3','ZMAT4','ZMAT5','ZMYM1','ZMYM2','ZMYM3','ZMYM4','ZMYM5','ZMYM6','ZMYND10','ZNF10','ZNF100','ZNF101','ZNF106','ZNF107','ZNF112','ZNF114','ZNF117','ZNF12','ZNF121','ZNF124','ZNF131','ZNF132','ZNF133','ZNF134','ZNF135','ZNF136','ZNF137P','ZNF138','ZNF14','ZNF140','ZNF141','ZNF142','ZNF143','ZNF146','ZNF148','ZNF154','ZNF155','ZNF157','ZNF16','ZNF160','ZNF165','ZNF169','ZNF17','ZNF174','ZNF175','ZNF177','ZNF18','ZNF180','ZNF181','ZNF182','ZNF184','ZNF185','ZNF189','ZNF19','ZNF195','ZNF197','ZNF2','ZNF20','ZNF200','ZNF202','ZNF205','ZNF207','ZNF208','ZNF211','ZNF212','ZNF213','ZNF214','ZNF215','ZNF217','ZNF219','ZNF22','ZNF221','ZNF222','ZNF223','ZNF224','ZNF225','ZNF226','ZNF227','ZNF229','ZNF23','ZNF230','ZNF232','ZNF233','ZNF234','ZNF235','ZNF236','ZNF239','ZNF24','ZNF248','ZNF25','ZNF250','ZNF251','ZNF253','ZNF254','ZNF256','ZNF257','ZNF26','ZNF260','ZNF263','ZNF264','ZNF266','ZNF267','ZNF268','ZNF273','ZNF274','ZNF275','ZNF276','ZNF277','ZNF28','ZNF280A','ZNF280B','ZNF280C','ZNF280D','ZNF281','ZNF282','ZNF283','ZNF284','ZNF285','ZNF286A','ZNF286B','ZNF287','ZNF292','ZNF296','ZNF3','ZNF30','ZNF300','ZNF302','ZNF304','ZNF311','ZNF316','ZNF317','ZNF318','ZNF319','ZNF32','ZNF320','ZNF322','ZNF324','ZNF324B','ZNF326','ZNF329','ZNF330','ZNF331','ZNF333','ZNF334','ZNF335','ZNF337','ZNF33A','ZNF33B','ZNF34','ZNF341','ZNF343','ZNF345','ZNF346','ZNF347','ZNF35','ZNF350','ZNF354A','ZNF354B','ZNF354C','ZNF355P','ZNF358','ZNF362','ZNF365','ZNF366','ZNF367','ZNF37A','ZNF382','ZNF383','ZNF384','ZNF385A','ZNF385B','ZNF385C','ZNF385D','ZNF391','ZNF394','ZNF395','ZNF396','ZNF397','ZNF398','ZNF404','ZNF407','ZNF408','ZNF41','ZNF410','ZNF414','ZNF415','ZNF416','ZNF417','ZNF418','ZNF419','ZNF420','ZNF423','ZNF425','ZNF426','ZNF428','ZNF429','ZNF43','ZNF430','ZNF431','ZNF432','ZNF433','ZNF436','ZNF438','ZNF439','ZNF44','ZNF440','ZNF441','ZNF442','ZNF443','ZNF444','ZNF445','ZNF446','ZNF449','ZNF45','ZNF451','ZNF454','ZNF460','ZNF461','ZNF462','ZNF467','ZNF468','ZNF469','ZNF470','ZNF471','ZNF473','ZNF474','ZNF479','ZNF48','ZNF480','ZNF483','ZNF484','ZNF485','ZNF486','ZNF487','ZNF488','ZNF490','ZNF491','ZNF492','ZNF493','ZNF496','ZNF497','ZNF500','ZNF501','ZNF502','ZNF503','ZNF506','ZNF507','ZNF510','ZNF511','ZNF512','ZNF512B','ZNF513','ZNF514','ZNF516','ZNF517','ZNF518A','ZNF518B','ZNF519']
TFs3Other=['ZNF521','ZNF524','ZNF525','ZNF526','ZNF527','ZNF528','ZNF529','ZNF530','ZNF532','ZNF534','ZNF536','ZNF540','ZNF541','ZNF542P','ZNF543','ZNF544','ZNF546','ZNF547','ZNF548','ZNF549','ZNF550','ZNF551','ZNF552','ZNF554','ZNF555','ZNF556','ZNF557','ZNF558','ZNF559','ZNF560','ZNF561','ZNF562','ZNF563','ZNF564','ZNF565','ZNF566','ZNF567','ZNF568','ZNF569','ZNF57','ZNF570','ZNF571','ZNF572','ZNF573','ZNF574','ZNF575','ZNF576','ZNF577','ZNF578','ZNF579','ZNF580','ZNF581','ZNF582','ZNF583','ZNF584','ZNF585A','ZNF585B','ZNF586','ZNF587','ZNF587B','ZNF589','ZNF592','ZNF593','ZNF594','ZNF595','ZNF596','ZNF597','ZNF598','ZNF599','ZNF600','ZNF605','ZNF606','ZNF607','ZNF608','ZNF609','ZNF610','ZNF611','ZNF613','ZNF614','ZNF615','ZNF616','ZNF618','ZNF619','ZNF620','ZNF621','ZNF622','ZNF623','ZNF624','ZNF625','ZNF626','ZNF627','ZNF628','ZNF629','ZNF630','ZNF638','ZNF639','ZNF641','ZNF644','ZNF645','ZNF646','ZNF648','ZNF649','ZNF652','ZNF653','ZNF654','ZNF655','ZNF658','ZNF66','ZNF660','ZNF662','ZNF664','ZNF665','ZNF667','ZNF668','ZNF669','ZNF670','ZNF671','ZNF672','ZNF674','ZNF675','ZNF676','ZNF677','ZNF678','ZNF679','ZNF680','ZNF681','ZNF682','ZNF683','ZNF684','ZNF687','ZNF688','ZNF689','ZNF69','ZNF691','ZNF692','ZNF695','ZNF696','ZNF697','ZNF699','ZNF7','ZNF70','ZNF700','ZNF701','ZNF703','ZNF704','ZNF705A','ZNF705B','ZNF705D','ZNF705E','ZNF705G','ZNF706','ZNF707','ZNF708','ZNF709','ZNF71','ZNF710','ZNF711','ZNF713','ZNF714','ZNF716','ZNF717','ZNF718','ZNF720','ZNF721','ZNF724','ZNF726','ZNF727','ZNF728','ZNF729','ZNF73','ZNF730','ZNF732','ZNF735','ZNF736','ZNF737','ZNF738','ZNF74','ZNF740','ZNF746','ZNF747','ZNF749','ZNF750','ZNF75A','ZNF75D','ZNF76','ZNF761','ZNF763','ZNF764','ZNF765','ZNF766','ZNF768','ZNF77','ZNF770','ZNF771','ZNF772','ZNF773','ZNF774','ZNF775','ZNF776','ZNF777','ZNF778','ZNF780A','ZNF780B','ZNF781','ZNF782','ZNF783','ZNF784','ZNF785','ZNF786','ZNF787','ZNF788','ZNF789','ZNF79','ZNF790','ZNF791','ZNF792','ZNF793','ZNF799','ZNF8','ZNF80','ZNF800','ZNF804A','ZNF804B','ZNF805','ZNF808','ZNF81','ZNF812P','ZNF813','ZNF814','ZNF816','ZNF821','ZNF823','ZNF827','ZNF829','ZNF83','ZNF830','ZNF831','ZNF835','ZNF836','ZNF837','ZNF839','ZNF84','ZNF841','ZNF843','ZNF844','ZNF845','ZNF846','ZNF85','ZNF850','ZNF852','ZNF853','ZNF860','ZNF862','ZNF865','ZNF878','ZNF879','ZNF880','ZNF883','ZNF888','ZNF891','ZNF90','ZNF91','ZNF92','ZNF93','ZNF98','ZNF99','ZNFX1','ZRANB2','ZRSR1','ZRSR2','ZSCAN1','ZSCAN10','ZSCAN12','ZSCAN16','ZSCAN18','ZSCAN2','ZSCAN20','ZSCAN21','ZSCAN22','ZSCAN23','ZSCAN25','ZSCAN26','ZSCAN29','ZSCAN30','ZSCAN31','ZSCAN32','ZSCAN4','ZSCAN5A','ZSCAN5B','ZSCAN5C','ZSCAN5DP','ZSCAN9','ZUFSP','ZXDA','ZXDB','ZXDC','ZZZ3']
TFs4Other=['ATF2','ATF6B','ATN1','BHLHA15','BHLHB2','BHLHB4 ','BHLHB5','BHLHB9','C13ORF8','C20ORF194','CDX2','CRAMP1L','ELP3','ERVK3-1','ETF1','FLJ12895','FLJ20321','FLJ25476','FLJ35784','FOXM1','FOXO2','FOXR1','FOXR2','GATA1','GRLF1','KDM4C','KDM4E','KIAA0863','KIAA0961','KLRG1','LBXCOR1','LEF1','LHX7','LOC126295','LOC51058','LOC91661','MGC4728','MYOD1','MYOG','MYRF','NEUROD4','NEUROG1','NFAT2','NFAT4','NFE2L1','NR1A2','NR1B1','NR1B2','NR1B3','NR1C1','NR1C2','NR1C3','NR1F3','NR1I1','NR2A1','NR2B3','NR3A1','NR3A2','NR3B3','OTEX','PEPP-2','PHF2','POU2F1','POU2F2','POU4F3','POU5F1','POU6F1','PRDM3','PRKRIP1','PRKRIR','SETD1A','SMAD8','SMYD2','SOX12','SOX14','SOX15','TBX14','TCF1','TCF17','TCF2','TCF9','TEAD1','TPRXL','UBTFL1','WHSC1','ZBTB14','ZBTB18','ZBTB19','ZBTB21','ZBTB27','ZBTB28','ZBTB35','ZBTB49','ZFP161','ZFP28','ZKSCAN10','ZKSCAN11','ZKSCAN12','ZKSCAN13','ZKSCAN14','ZKSCAN16','ZKSCAN19','ZKSCAN21','ZKSCAN24','ZKSCAN6','ZKSCAN8','ZNF10','ZNF112','ZNF187','ZNF193','ZNF204P','ZNF238','ZNF271','ZNF295','ZNF305','ZNF323','ZNF434','ZNF499','ZNF542','ZNF545','ZNF553','ZNF56','ZNF625','ZNF643','ZNF698','ZNF705F','ZNF751','ZNF752','ZNF753','ZNF785','ZNF806','ZNF816','ZNF833P','ZNF906','ZSCAN1','ZSCAN25','ZSCAN26','ZSCAN31','ZSCAN32','ZSCAN5D','ZSCAN9']
TFsOther=TFs1Other+TFs2Other+TFs3Other+TFs4Other
```

Example 8 (unknown):
```unknown
S genes: ['MCM5', 'PCNA', 'TYMS', 'FEN1', 'MCM2', 'MCM4', 'RRM1', 'UNG', 'GINS2', 'MCM6', 'CDCA7', 'DTL', 'PRIM1', 'UHRF1', 'HELLS', 'RFC2', 'RPA2', 'NASP', 'RAD51AP1', 'GMNN', 'WDR76', 'SLBP', 'CCNE2', 'UBR7', 'POLD3', 'MSH2', 'ATAD2', 'RAD51', 'RRM2', 'CDC45', 'CDC6', 'EXO1', 'TIPIN', 'DSCC1', 'BLM', 'CASP8AP2', 'USP1', 'CLSPN', 'POLA1', 'CHAF1B', 'BRIP1', 'E2F8'] 

G2 genes: ['HMGB2', 'CDK1', 'NUSAP1', 'UBE2C', 'BIRC5', 'TPX2', 'TOP2A', 'NDC80', 'CKS2', 'NUF2', 'CKS1B', 'MKI67', 'TMPO', 'CENPF', 'TACC3', 'SMC4', 'CCNB2', 'CKAP2L', 'CKAP2', 'AURKB', 'BUB1', 'KIF11', 'ANP32E', 'TUBB4B', 'GTSE1', 'KIF20B', 'HJURP', 'CDCA3', 'CDC20', 'TTK', 'CDC25C', 'KIF2C', 'RANGAP1', 'NCAPD2', 'DLGAP5', 'CDCA2', 'CDCA8', 'ECT2', 'KIF23', 'HMMR', 'AURKA', 'PSRC1', 'ANLN', 'LBR', 'CKAP5', 'CENPE', 'CTCF', 'NEK2', 'G2E3', 'GAS2L3', 'CBX5', 'CENPA'] 

Hemoglobin genes set 1: ['HBB', 'HBD', 'HBG1', 'HBG2', 'HBE1', 'HBZ', 'HBM', 'HBA2', 'HBA1', 'HBQ1'] 

Hemoglobin genes set 2: ['AHSP', 'GYPA', 'RHAG', 'GFI1B', 'KEL', 'EPB42', 'ALAS2', 'TSPO2', 'GYPB', 'GYPE', 'SPTA1', 'RHCE', 'KLF1', 'RHD', 'SLC4A1', 'SNX22', 'SQSTM1', 'SOD1', 'FTH1', 'MAP1LC3B', 'SLC25A37', 'TRIM58', 'HMBS'] 

Mitochondrial genes: ['MT-ND1', 'MT-ND2', 'MT-CO1', 'MT-CO2', 'MT-ATP8', 'MT-ATP6', 'MT-CO3', 'MT-ND3', 'MT-ND4L', 'MT-ND4', 'MT-ND5', 'MT-ND6', 'MT-CYB'] 

Ribo genes: ['RPLP0', 'RPLP1', 'RPLP2', 'RPL3', 'RPL3L', 'RPL4', 'RPL5', 'RPL6', 'RPL7', 'RPL7A', 'RPL7L1', 'RPL8', 'RPL9', 'RPL10', 'RPL10A', 'RPL10L', 'RPL11', 'RPL12', 'RPL13', 'RPL13A', 'RPL14', 'RPL15', 'RPL17', 'RPL18', 'RPL18A', 'RPL19', 'RPL21', 'RPL22', 'RPL22L1', 'RPL23', 'RPL23A', 'RPL24', 'RPL26', 'RPL26L1', 'RPL27', 'RPL27A', 'RPL28', 'RPL29', 'RPL30', 'RPL31', 'RPL32', 'RPL34', 'RPL35', 'RPL35A', 'RPL36', 'RPL36A', 'RPL36AL', 'RPL37', 'RPL37A', 'RPL38', 'RPL39', 'RPL39L', 'UBA52', 'RPL41', 'RPSA', 'RPS2', 'RPS3', 'RPS3A', 'RPS4X', 'RPS4Y1', 'RPS4Y2', 'RPS5', 'RPS6', 'RPS7', 'RPS8', 'RPS9', 'RPS10', 'RPS11', 'RPS12', 'RPS13', 'RPS14', 'RPS15', 'RPS15A', 'RPS16', 'RPS17', 'RPS18', 'RPS19', 'RPS20', 'RPS21', 'RPS23', 'RPS24', 'RPS25', 'RPS26', 'RPS27', 'RPS27A', 'RPS27L', 'RPS28', 'RPS29', 'FAU'] 

Tissue dissociation stress associated genes: ['FOS', 'FOSB', 'JUN', 'JUNB', 'JUND', 'ZFP36', 'SOCS3', 'CEBPD', 'CEBPB', 'ATF3', 'EGR1', 'HSPA1A', 'HSPA1B', 'HSP90AA1', 'HSP90AB1', 'HSPA8', 'HSPB1', 'HSPE1', 'IER3', 'IER2', 'BTG1', 'BTG2', 'DUSP1']
```

Example 9 (python):
```python
#Expression: "YlGn" (e.g. DE plots)
#Divergent expression: "PRGn" (e.g. log2FC)
#Age: "afmhot_r"
#Others: "viridis" (e.g. UMAP expression)

#Define color palette
FineGrainPalette={
    'AtrialCardiomyocytesLeft':'#eac075',
    'AtrialCardiomyocytesRight':'#4c6b8a',
    'AtrialCardiomyocytesCycling':'#e94bc7',
    'VentricularCardiomyocytesLeftCompact':'#ab1c3c',
    'VentricularCardiomyocytesRightCompact':'#fdba45',
    'VentricularCardiomyocytesLeftTrabeculated':'#2b9200',
    'VentricularCardiomyocytesRightTrabeculated':'#0273a8',
    'VentricularCardiomyocytesCycling':'#e1acff',
    'SinoatrialNodePacemakerCells':'#854834',
    'AtrioventricularNodePacemakerCells':'#688500',
    'VentricularConductionSystemProximal':'#01ccc8',
    'VentricularConductionSystemDistal':'#b58fab',
    'GreatVesselAdventitialFibroblasts':'#ff7e97',
    'CoronaryVesselAdventitialFibroblasts':'#ff9b6e',
    'MyocardialInterstitialFibroblasts':'#8a6800',
    'SubEpicardialFibroblasts':'#206601',
    'Myofibroblasts':'#1bdebf',
    'LymphNodeFibroblasticReticularCells':'#005aa0',
    'ValveInterstitialCells':'#e478ff',
    'GreatVesselSmoothMuscleCells':'#ff665f',
    'CoronarySmoothMuscleCells':'#d2cb18',
    'DuctusArteriosusSmoothMuscleCells':'#83d7b3',
    'CoronaryPericytes':'#9088ff',
    'PericardialCellsIntermediate':'#c95e00',
    'PericardialCellsFibrous':'#01e077',
    'PericardialCellsParietal':'#ff88e4',
    'GreatVesselArterialEndothelialCells':'#d4004b',
    'GreatVesselVenousEndothelialCells':'#8f4300',
    'CoronaryArterialEndothelialCells':'#89da5a',
    'CoronaryVenousEndothelialCells':'#006f86',
    'CoronaryCapillaryEndothelialCells':'#aa7dff',
    'EndocardialCells':'#ff8241',
    'EndocardialCushionCells':'#46603a',
    'ValveEndothelialCells':'#414eb6',
    'LymphaticEndothelialCells':'#8b3f60',
    'MesothelialEpicardialCells':'#0295fe',
    'EpicardiumDerivedCells':'#ff92c4',
    'NeuronPrecursors':'#d30038',
    'ChromaffinCells':'#726800',
    'SympatheticNeurons':'#01894b',
    'ParasympatheticNeurons':'#545191',
    'SchwannCellPrecursors':'#b0d276',
    'SchwannCells':'#b30079',
    'MonocytesMPOpos':'#c50066',
    'Monocytes':'#fab4b6',
    'MonocyteDerivedCells':'#d14008',
    'MacrophagesCX3CR1pos':'#eabe97',
    'MacrophagesTIMD4pos':'#b2a200',
    'MacrophagesLYVE1pos':'#4a6005',
    'MacrophagesATF3pos':'#3db833',
    'DendriticCellsType1':'#007859',
    'DendriticCellsMature':'#01a9db',
    'PlasmacytoidDendriticCells':'#2c54ab',
    'MastCells':'#d0bef2',
    'Megakaryocytes':'#94298f',
    'TCellsCD4pos':'#ff387c',
    'TCellsCD8pos':'#9e3429',
    'TregsCD4pos':'#b37700',
    'ProBCells':'#8dab00',
    'BCells':'#a8d1a1',
    'BCellsMS4A1pos':'#00d2f0',
    'NaturalKillerCells':'#abb4ff',
    'InnateLymphoidCells':'#72497f',
}
sc.pl.umap(adata, color='fine_grain',palette=FineGrainPalette)

MidGrainPalette={
    'AtrialCardiomyocytes':'#af3a4d',
    'VentricularCardiomyocytes':'#b1756c',
    'CardiacConductionSystem':'#d45633',
    'Fibroblasts':'#9b6e4e',
    'MuralCells':'#ffb92f',
    'PericardialCells':'#b4f03b',
    'BloodVesselEndothelialCells':'#72cc57',
    'EndocardialCells':'#436e43',
    'LymphaticEndothelialCells':'#85cca0',
    'EpicardialCells':'#5eabb0',
    'Neurons':'#7bbdff',
    'Glia':'#403c90',
    'MyeloidCells':'#420066',
    'LymphoidCells':'#f553f6',
}
sc.pl.umap(adata, color='mid_grain',palette=MidGrainPalette)

CoarseGrainPalette={
    'Cardiomyocytes':'#bc5950',
    'Mesenchymal':'#af9c4c',
    'Endothelium':'#599e58',
    'Epicardium':'#5eabb0',
    'Neural':'#5e75b1',
    'Leukocytes':'#9a52a0',
}
sc.pl.umap(adata, color='coarse_grain',palette=CoarseGrainPalette)

RegionGrainPalette={
    'aorta':'#ffa2bc',
    'apex':'#fe0049',
    'atria':'#732d00',
    'atrial septum':'#ffd54c',
    'base':'#837100',
    'ductus arteriosus':'#bdff8b',
    'whole sample':'#1b6c00',
    'heart without node':'#02e6c6',
    'left ventricle':'#01c1f5',
    'node':'#00466d',
    'outflow tract':'#020011',
    'pericardium':'#7a75ff',
    'pulmonary arches and branches':'#30003e',
    'right ventricle':'#ff88e1',
    'right ventricle and papillaries':'#98004f',
}
sc.pl.umap(adata, color='region',palette=RegionGrainPalette)
```

Example 10 (python):
```python
# Adding hierarchy to the fine_grain mid_grain and coarse_grain; this will be propagated when plotting
coarsegrain_order = ['Cardiomyocytes',
                     'Mesenchymal',
                     'Endothelium',
                     'Epicardium',
                     'Neural',
                     'Leukocytes']

midgrain_order = ['AtrialCardiomyocytes',
                'VentricularCardiomyocytes', 
                'CardiacConductionSystem', 
                  
                'Fibroblasts',
                 'MuralCells',
                  'PericardialCells',
                  
                'BloodVesselEndothelialCells',
                'EndocardialCells',
                  'LymphaticEndothelialCells',
                    
                  'EpicardialCells',
                  
                  'Neurons',
                 'Glia',
              
                    'MyeloidCells',  
                 'LymphoidCells',
                ]

finegrain_order = [ 'AtrialCardiomyocytesLeft',
                    'AtrialCardiomyocytesRight',
                   'AtrialCardiomyocytesCycling',
                     
                     'VentricularCardiomyocytesLeftCompact',
                     'VentricularCardiomyocytesRightCompact',
                   'VentricularCardiomyocytesLeftTrabeculated',
                     'VentricularCardiomyocytesRightTrabeculated',
                   'VentricularCardiomyocytesCycling',
                   
                   'SinoatrialNodePacemakerCells',
                   'AtrioventricularNodePacemakerCells',
                     'VentricularConductionSystemProximal',
                     'VentricularConductionSystemDistal',
                     
                    'GreatVesselAdventitialFibroblasts',
                   'CoronaryVesselAdventitialFibroblasts',
                 'MyocardialInterstitialFibroblasts',
                 'SubEpicardialFibroblasts',
                 'Myofibroblasts',
                    'LymphNodeFibroblasticReticularCells',
                        'ValveInterstitialCells',
               
                   'GreatVesselSmoothMuscleCells',
                 'CoronarySmoothMuscleCells',
                 'DuctusArteriosusSmoothMuscleCells',
                   'CoronaryPericytes',
                   
                   'PericardialCellsIntermediate', 
                   'PericardialCellsFibrous', 
                   'PericardialCellsParietal',
                   
                   'GreatVesselArterialEndothelialCells',
                     'GreatVesselVenousEndothelialCells',
                  'CoronaryArterialEndothelialCells',
                     'CoronaryVenousEndothelialCells',
                   'CoronaryCapillaryEndothelialCells',
 
                  'EndocardialCells',
                   'EndocardialCushionCells',
                   'ValveEndothelialCells',
                  
                'LymphaticEndothelialCells',
                
                'MesothelialEpicardialCells',
                   'EpicardiumDerivedCells', 
                  
                   'NeuronPrecursors',
                    'ChromaffinCells',
                     'SympatheticNeurons',
                     'ParasympatheticNeurons',
                     
                  'SchwannCellPrecursors', 
                   'SchwannCells',
                  
                   'MonocytesMPOpos',
                   'Monocytes',
                   'MonocyteDerivedCells',
                   'MacrophagesCX3CR1pos',
                   'MacrophagesTIMD4pos',
                    'MacrophagesLYVE1pos',
                   'MacrophagesATF3pos',
                    'DendriticCellsType1',
                  'DendriticCellsMature',
                   'PlasmacytoidDendriticCells',
                   'MastCells',
                 'Megakaryocytes',

                   'TCellsCD4pos',
                 'TCellsCD8pos',
                 'TregsCD4pos',
                   'ProBCells',
                   'BCells',
                 'BCellsMS4A1pos',
                   'NaturalKillerCells',
                 'InnateLymphoidCells',
                ]

adata.obs['coarse_grain'].cat.reorder_categories(coarsegrain_order,inplace=True)
adata.obs['mid_grain'].cat.reorder_categories(midgrain_order,inplace=True)
adata.obs['fine_grain'].cat.reorder_categories(finegrain_order,inplace=True)

sc.pl.umap(adata, color='coarse_grain')
sc.pl.umap(adata, color='mid_grain')
sc.pl.umap(adata, color='fine_grain')
```

Example 11 (python):
```python
set(adata.obs.fine_grain)
```

Example 12 (unknown):
```unknown
{'AtrialCardiomyocytesCycling',
 'AtrialCardiomyocytesLeft',
 'AtrialCardiomyocytesRight',
 'AtrioventricularNodePacemakerCells',
 'BCells',
 'BCellsMS4A1pos',
 'ChromaffinCells',
 'CoronaryArterialEndothelialCells',
 'CoronaryCapillaryEndothelialCells',
 'CoronaryPericytes',
 'CoronarySmoothMuscleCells',
 'CoronaryVenousEndothelialCells',
 'CoronaryVesselAdventitialFibroblasts',
 'DendriticCellsMature',
 'DendriticCellsType1',
 'DuctusArteriosusSmoothMuscleCells',
 'EndocardialCells',
 'EndocardialCushionCells',
 'EpicardiumDerivedCells',
 'GreatVesselAdventitialFibroblasts',
 'GreatVesselArterialEndothelialCells',
 'GreatVesselSmoothMuscleCells',
 'GreatVesselVenousEndothelialCells',
 'InnateLymphoidCells',
 'LymphNodeFibroblasticReticularCells',
 'LymphaticEndothelialCells',
 'MacrophagesATF3pos',
 'MacrophagesCX3CR1pos',
 'MacrophagesLYVE1pos',
 'MacrophagesTIMD4pos',
 'MastCells',
 'Megakaryocytes',
 'MesothelialEpicardialCells',
 'MonocyteDerivedCells',
 'Monocytes',
 'MonocytesMPOpos',
 'MyocardialInterstitialFibroblasts',
 'Myofibroblasts',
 'NaturalKillerCells',
 'NeuronPrecursors',
 'ParaympatheticNeurons',
 'PericardialCellsFibrous',
 'PericardialCellsIntermediate',
 'PericardialCellsParietal',
 'PlasmacytoidDendriticCells',
 'ProBCells',
 'SchwannCellPrecursors',
 'SchwannCells',
 'SinoatrialNodePacemakerCells',
 'SubEpicardialFibroblasts',
 'SympatheticNeurons',
 'TCellsCD4pos',
 'TCellsCD8pos',
 'TregsCD4pos',
 'ValveEndothelialCells',
 'ValveInterstitialCells',
 'VentricularCardiomyocytesCycling',
 'VentricularCardiomyocytesLeftCompact',
 'VentricularCardiomyocytesLeftTrabeculated',
 'VentricularCardiomyocytesRightCompact',
 'VentricularCardiomyocytesRightTrabeculated',
 'VentricularConductionSystemDistal',
 'VentricularConductionSystemProximal'}
```

Example 13 (python):
```python
set(adata.obs.mid_grain)
```

Example 14 (unknown):
```unknown
{'AtrialCardiomyocytes',
 'BloodVesselEndothelialCells',
 'CardiacConductionSystem',
 'EndocardialCells',
 'EpicardialCells',
 'Fibroblasts',
 'Glia',
 'LymphaticEndothelialCells',
 'LymphoidCells',
 'MuralCells',
 'MyeloidCells',
 'Neurons',
 'PericardialCells',
 'VentricularCardiomyocytes'}
```

Example 15 (python):
```python
#adata = adata[adata.obs['fine_grain'].isin(['CoronaryVesselAdventitialFibroblasts','MyocardialInterstitialFibroblasts1','MyocardialInterstitialFibroblasts2','Myofibroblasts','GreatVesselAdventitialFibroblasts'])].copy()
#adata = adata[adata.obs['fine_grain'].isin(['CoronaryArterialEndothelialCells','CoronaryCapillaryEndothelialCells','CoronaryVenousEndothelialCells','GreatVesselArterialEndothelialCells','GreatVesselVenousEndothelialCells'])].copy()
#adata = adata[adata.obs['fine_grain'].isin(['CoronaryPericytes','CoronarySmoothMuscleCells','DuctusArteriosusSmoothMuscleCells','GreatVesselSmoothMuscleCells'])].copy()
#adata = adata[adata.obs['mid_grain'].isin(['CardiacConductionSystem'])].copy()
adata=adata[adata.obs.coarse_grain=='Cardiomyocytes',:].copy()
```

Example 16 (unknown):
```unknown
AnnData object with n_obs × n_vars = 5087 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score'
    var: 'gene_name_scRNA-0', 'gene_id'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'age_colors', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'cycling_colors', 'donor_colors', 'fine_grain_colors', 'heart_or_greatvessels_colors', 'hvg', 'kit_10x_colors', 'mid_grain_colors', 'modality_colors', 'neighbors', 'phase_colors', 'region_colors', 'sex_colors', 'trimester_colors', 'umap', 'week_colors'
    obsm: 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 17 (python):
```python
adata = adata[adata.obs['donor'].isin(['BRC2251','BRC2253','BRC2256','BRC2260','BRC2262','BRC2263'])].copy()
```

Example 18 (python):
```python
adata = adata[adata.obs['fine_grain'].isin(['VentricularConductionSystemProximal','VentricularConductionSystemDistal','VentricularCardiomyocytesRightTrabeculated','VentricularCardiomyocytesRightCompact','VentricularCardiomyocytesLeftTrabeculated','VentricularCardiomyocytesLeftCompact'])].copy()
```

Example 19 (python):
```python
set(adata.obs.fine_grain)
```

Example 20 (unknown):
```unknown
{'AtrialCardiomyocytesCycling',
 'AtrialCardiomyocytesLeft',
 'AtrialCardiomyocytesRight',
 'AtrioventricularNodePacemakerCells',
 'SinoatrialNodePacemakerCells',
 'VentricularCardiomyocytesCycling',
 'VentricularCardiomyocytesLeftCompact',
 'VentricularCardiomyocytesLeftTrabeculated',
 'VentricularCardiomyocytesRightCompact',
 'VentricularCardiomyocytesRightTrabeculated',
 'VentricularConductionSystemDistal',
 'VentricularConductionSystemProximal'}
```

Example 21 (python):
```python
# preserve raw count
adata.layers["counts"] = adata.X.copy()

# normalise and log-transform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key='batch_key')
#sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5)

print(sum(adata.var.highly_variable))
sc.pl.highly_variable_genes(adata)

# freeze the state in `.raw`
adata.raw = adata

# subset hvgs
adata = adata[:, adata.var.highly_variable]

adata=adata.copy()

sc.tl.score_genes_cell_cycle(adata, s_genes=s_genes, g2m_genes=g2m_genes)
#sc.pp.regress_out(adata, ['S_score','G2M_score'])

#batch correction
scvi.model.SCVI.setup_anndata(adata, 
                              layer="counts", 
                              batch_key="batch_key",
                              categorical_covariate_keys=['phase'],
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo'])

vae = scvi.model.SCVI(adata, n_hidden = 128, n_latent = 50, n_layers = 3, dispersion = 'gene-batch')
vae.train(max_epochs=50)
    
# Convergence plot
x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
plt.plot(x, vae.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
plt.clf()

# add the latent space to adata
adata.obsm["X_scVI"] = vae.get_latent_representation()

sc.tl.score_genes(adata, stress_genes, score_name='stress_score')
sc.tl.score_genes(adata, hb_genes1, score_name='hb1_score')

sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata,min_dist=0.1)
#sc.tl.tsne(adata)
sc.pl.umap(adata, color=['log1p_total_counts','n_genes','n_genes_by_counts','total_counts','pct_counts_mt','pct_counts_ribo'],ncols=5)
sc.pl.umap(adata, color=['scrublet_score','stress_score', 'hb1_score','phase'], ncols=5,)

sc.pl.umap(adata, color=['donor','age','week','trimester','region','phase','cycling','cell_or_nuclei','modality','kit_10x','FACSgate','heart_or_greatvessels','coarse_grain','mid_grain','fine_grain'],wspace=0.8,ncols=5)
Genes=['TTN','TNNT2','MYOM1','LDB3','ALDH1A2','TNNT1','KRT19','UPK3B','BNC1','COL6A3','FBLN1','PCOLCE','LRRC17','PCDH18','EMCN','ECSCR','PECAM1','CDH5','ESAM', 'LAPTM5','PTPRC','LCP1','CD53','WAS','PHOX2B','NRXN1','STMN2','GRIK2','L1CAM']
sc.pl.umap(adata, color=Genes)
```

Example 22 (python):
```python
normalizing counts per cell
    finished (0:00:00)
extracting highly variable genes
    finished (0:00:09)
--> added
    'highly_variable', boolean vector (adata.var)
    'means', float vector (adata.var)
    'dispersions', float vector (adata.var)
    'dispersions_norm', float vector (adata.var)
1125
```

Example 23 (python):
```python
calculating cell cycle phase
computing score 'S_score'
WARNING: genes are not in var_names and ignored: ['PCNA', 'TYMS', 'FEN1', 'MCM2', 'MCM4', 'RRM1', 'UNG', 'MCM6', 'CDCA7', 'DTL', 'PRIM1', 'UHRF1', 'HELLS', 'RFC2', 'RPA2', 'NASP', 'RAD51AP1', 'GMNN', 'WDR76', 'CCNE2', 'UBR7', 'MSH2', 'ATAD2', 'RAD51', 'RRM2', 'CDC45', 'CDC6', 'EXO1', 'DSCC1', 'BLM', 'CASP8AP2', 'USP1', 'CLSPN', 'POLA1', 'CHAF1B', 'BRIP1', 'E2F8']
    finished: added
    'S_score', score of gene set (adata.obs).
    205 total control genes are used. (0:00:00)
computing score 'G2M_score'
WARNING: genes are not in var_names and ignored: ['HMGB2', 'CDK1', 'NUSAP1', 'UBE2C', 'BIRC5', 'TPX2', 'TOP2A', 'NDC80', 'CKS2', 'NUF2', 'CKS1B', 'MKI67', 'TMPO', 'CENPF', 'TACC3', 'SMC4', 'CCNB2', 'CKAP2L', 'CKAP2', 'AURKB', 'BUB1', 'KIF11', 'ANP32E', 'TUBB4B', 'GTSE1', 'KIF20B', 'HJURP', 'CDCA3', 'CDC20', 'TTK', 'CDC25C', 'KIF2C', 'RANGAP1', 'DLGAP5', 'CDCA2', 'CDCA8', 'ECT2', 'KIF23', 'HMMR', 'AURKA', 'PSRC1', 'ANLN', 'LBR', 'CKAP5', 'CENPE', 'CTCF', 'NEK2', 'G2E3', 'GAS2L3', 'CBX5', 'CENPA']
    finished: added
    'G2M_score', score of gene set (adata.obs).
    41 total control genes are used. (0:00:00)
-->     'phase', cell cycle phase (adata.obs)
```

Example 24 (unknown):
```unknown
GPU available: False, used: False
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
```

Example 25 (unknown):
```unknown
Epoch 50/50: 100%|███████████| 50/50 [01:48<00:00,  2.17s/it, loss=581, v_num=1]
```

Example 26 (python):
```python
computing score 'stress_score'
WARNING: genes are not in var_names and ignored: ['JUND', 'ZFP36', 'SOCS3', 'CEBPB', 'EGR1', 'HSPA1B', 'HSP90AA1', 'HSP90AB1', 'HSPA8', 'HSPB1', 'HSPE1', 'BTG1', 'BTG2', 'DUSP1']
    finished: added
    'stress_score', score of gene set (adata.obs).
    272 total control genes are used. (0:00:00)
computing score 'hb1_score'
WARNING: genes are not in var_names and ignored: ['HBB', 'HBD', 'HBE1', 'HBZ', 'HBM', 'HBQ1']
    finished: added
    'hb1_score', score of gene set (adata.obs).
    184 total control genes are used. (0:00:00)
computing neighbors
    finished: added to `.uns['neighbors']`
    `.obsp['distances']`, distances for each pair of neighbors
    `.obsp['connectivities']`, weighted adjacency matrix (0:00:04)
computing UMAP
    finished: added
    'X_umap', UMAP coordinates (adata.obsm) (0:00:08)
```

Example 27 (unknown):
```unknown
... storing 'phase' as categorical
```

Example 28 (unknown):
```unknown
<Figure size 320x320 with 0 Axes>
```

Example 29 (python):
```python
sc.pl.umap(adata, color=['PRRX1','fine_grain'])
```

Example 30 (python):
```python
#adata.write('./29MarchVentricularCMs.h5ad')
#adata.write('./29MarchAtrialCMs.h5ad')
#adata.write('./11MayAllCMs.h5ad')
#adata.write('./29MarchCardiacConductionSystem.h5ad')
```

Example 31 (python):
```python
adata=sc.read('./11MayAllCMs.h5ad')
print(adata.shape)
```

Example 32 (unknown):
```unknown
(74884, 1821)
```

Example 33 (python):
```python
adata=adata[adata.obs.cycling=='no',:].copy()
```

Example 34 (python):
```python
sc.pl.umap(adata, color='mid_grain')
```

Example 35 (python):
```python
#Getting the UMAPs for clusterofinterest
ClusterOfInterest='VentricularCardiomyocytes'
sc.tl.rank_genes_groups(adata, 'mid_grain', method='t-test', key_added = "t-test")
DEGenesForCluster = [x[ClusterOfInterest] for x in adata.uns['t-test']['names'][:50]]
sc.pl.dotplot(adata, DEGenesForCluster ,groupby='fine_grain',standard_scale='var',cmap='YlGn')
#sc.pl.umap(adata, color=DEGenesForCluster)
```

Example 36 (python):
```python
ranking genes
    finished: added to `.uns['t-test']`
    'names', sorted np.recarray to be indexed by group ids
    'scores', sorted np.recarray to be indexed by group ids
    'logfoldchanges', sorted np.recarray to be indexed by group ids
    'pvals', sorted np.recarray to be indexed by group ids
    'pvals_adj', sorted np.recarray to be indexed by group ids (0:00:01)
```

Example 37 (python):
```python
adata=adata[adata.obs.cycling=='no',:].copy()
adata=adata[adata.obs.mid_grain=='VentricularCardiomyocytes',:].copy()
```

Example 38 (python):
```python
Genes=['SLC1A3','ADGRL2','MGAT4C','CDH13','CRIM1','ANKRD1','MYL2','PRRX1','GPNMB','ZNF385D','RABGAP1L','CPVL','SYNDIG1','ADCY5','SEMA5A','ADAMTSL1','SGCD','MYOM2','SPOCK1','SLIT3','TBX20','KCNQ5','HEY2','LRMDA','BRINP3','CSMD1','CRNDE','OPCML','CYP2J2','CNN1','NPPB','XIRP2','NCALD','ENOX1','KCNIP4','SORCS3']
print(set([x for x in Genes if Genes.count(x) > 1]))
sc.pl.dotplot(adata,Genes,groupby='fine_grain',var_group_positions=[(0,4),(5,9),(10,12),(13,19)], var_group_labels=['left','right','compact','trabeculated'],standard_scale='var',cmap='YlGn',var_group_rotation=0)
```

Example 39 (python):
```python
#VCMs all
bdata=adata[adata.obs.mid_grain=='VentricularCardiomyocytes',:].copy()
Genes=['TBX20','KCNQ5','HEY2','LRMDA','BRINP3','CSMD1','CRNDE','OPCML','CYP2J2','CNN1','NPPB','XIRP2','NCALD','ENOX1','KCNIP4','SORCS3']
print(set([x for x in Genes if Genes.count(x) > 1]))
sc.pl.dotplot(bdata,Genes,groupby='fine_grain',standard_scale='var',cmap='YlGn')
```

Example 40 (python):
```python
#VCMs right/left and traveculated/compact
bdata=adata[adata.obs.mid_grain=='VentricularCardiomyocytes',:].copy()
Genes=['RABGAP1L','CPVL','SYNDIG1','ADCY5','SEMA5A','ADAMTSL1','SGCD','MYOM2','SPOCK1','SLIT3']
print(set([x for x in Genes if Genes.count(x) > 1]))
sc.pl.dotplot(bdata, Genes,var_group_positions=[(0,2),(3,9)], var_group_labels=['compact','trabeculated'], var_group_rotation=0 ,groupby='fine_grain',standard_scale='var',cmap='YlGn')
Genes= ['SLC1A3','ADGRL2','MGAT4C','CDH13','CRIM1','ANKRD1','MYL2','PRRX1','GPNMB','ZNF385D']
print(set([x for x in Genes if Genes.count(x) > 1]))
sc.pl.dotplot(bdata,Genes ,var_group_positions=[(0,4),(5,9)], var_group_labels=['left','right'], var_group_rotation=0, groupby='fine_grain',standard_scale='var',cmap='YlGn')
```

Example 41 (python):
```python
#ACMs
bdata=adata[adata.obs.mid_grain=='AtrialCardiomyocytes',:].copy()
Genes=['KCNMB2','PCDH7','RYR3','PANCR','MBP','NAV3','PITX2','NTM','ROR1','KIF26B','PLCL1','ANGPT1','BRINP3','BMP10']
print(set([x for x in Genes if Genes.count(x) > 1]))
sc.pl.dotplot(bdata, Genes ,groupby='fine_grain',standard_scale='var',cmap='YlGn')
```

Example 42 (python):
```python
#CardiacConductionSystem
bdata=adata[adata.obs.mid_grain=='CardiacConductionSystem',:].copy()
Genes=['MYH6','CACNA1D','KCNJ3','NR2F2','NRXN1','CAMK1D','HCN1','TBX3','SHOX2','PDE1A','TENM4','MYH11','TBX18','GREB1L','GNG11','NRXN3','RSPO3','ZNF536','MYH7','MYL2','NAV1','ADAMTSL1','GJA1','GCNT2','LPL','PCDH7','PRDM16','NIBAN1','PHACTR1','FGF18','OPCML','NTN1','IRX3','IRX2','DGKG']
print(set([x for x in Genes if Genes.count(x) > 1]))
sc.pl.dotplot(bdata, Genes,categories_order=['SinoatrialNodePacemakerCells','AtrioventricularNodePacemakerCells','VentricularConductionSystemProximal','VentricularConductionSystemDistal'], var_group_positions=[(0,17),(18,34)], var_group_labels=['atrial conduction','ventricular conduction'], var_group_rotation=0,groupby='fine_grain',standard_scale='var',cmap='YlGn')
```

Example 43 (python):
```python
#!!No good markres of conduction system

#Getting the UMAPs for clusterofinterest
ClusterOfInterest='CardiacConductionSystem'
sc.tl.rank_genes_groups(adata, 'mid_grain', method='t-test', key_added = "t-test")
DEGenesForCluster = [x[ClusterOfInterest] for x in adata.uns['t-test']['names'][:50]]
sc.pl.dotplot(adata, DEGenesForCluster ,groupby='fine_grain', categories_order=['SinoatrialNodePacemakerCells','AtrioventricularNodePacemakerCells','VentricularConductionSystemProximal','VentricularConductionSystemDistal','AtrialCardiomyocytesLeft','AtrialCardiomyocytesRight','VentricularCardiomyocytesLeftCompact','VentricularCardiomyocytesLeftTrabeculated','VentricularCardiomyocytesRightCompact','VentricularCardiomyocytesRightTrabeculated'],standard_scale='var',cmap='YlGn')
#sc.pl.umap(adata, color=DEGenesForCluster)
```

Example 44 (python):
```python
ranking genes
    finished: added to `.uns['t-test']`
    'names', sorted np.recarray to be indexed by group ids
    'scores', sorted np.recarray to be indexed by group ids
    'logfoldchanges', sorted np.recarray to be indexed by group ids
    'pvals', sorted np.recarray to be indexed by group ids
    'pvals_adj', sorted np.recarray to be indexed by group ids (0:00:01)
```

Example 45 (python):
```python
sc.pl.dotplot(adata, ['CNTN2','NPTN','NTM'] ,groupby='fine_grain', categories_order=['SinoatrialNodePacemakerCells','AtrioventricularNodePacemakerCells','VentricularConductionSystemProximal','VentricularConductionSystemDistal','AtrialCardiomyocytesLeft','AtrialCardiomyocytesRight','VentricularCardiomyocytesLeftCompact','VentricularCardiomyocytesLeftTrabeculated','VentricularCardiomyocytesRightCompact','VentricularCardiomyocytesRightTrabeculated'],standard_scale='var',cmap='YlGn')
```

Example 46 (python):
```python
adata = adata[adata.obs['mid_grain'].isin(['VentricularCardiomyocytes'])].copy()
adata = adata[adata.obs['cycling'].isin(['no'])].copy()
```

Example 47 (python):
```python
adata.obs['type']= 'compact'
adata.obs['type'][[x for x in adata.obs_names if adata.obs.fine_grain[x] in ['VentricularCardiomyocytesLeftTrabeculated','VentricularCardiomyocytesRightTrabeculated']]]='trabeculated'
sc.pl.umap(adata, color='type')
```

Example 48 (unknown):
```unknown
... storing 'type' as categorical
```

Example 49 (python):
```python
#This box to run enrichment right away
sc.tl.rank_genes_groups(adata, 'type', method='t-test', key_added = "t-test")
gene_set_names = gseapy.get_library_name(organism='Human')
print(gene_set_names)
glist = sc.get.rank_genes_groups_df(adata, group='trabeculated', 
                                    key='t-test', log2fc_min=0.25, 
                                    pval_cutoff=0.05)['names'].squeeze().str.strip().tolist()
print(len(glist))                                  
#GO_Biological_Process_2021,GO_Cellular_Component_2021,GO_Molecular_Function_2021
enr_res = gseapy.enrichr(gene_list=glist,
                     organism='Human',
                     gene_sets='GO_Biological_Process_2021',
                     cutoff = 0.5)
enr_res.results.head(20)
gseapy.barplot(enr_res.res2d,title='GO_Biological_Process_2021',top_term=20,figsize=(3,20))
```

Example 50 (python):
```python
ranking genes
    finished: added to `.uns['wilcoxon']`
    'names', sorted np.recarray to be indexed by group ids
    'scores', sorted np.recarray to be indexed by group ids
    'logfoldchanges', sorted np.recarray to be indexed by group ids
    'pvals', sorted np.recarray to be indexed by group ids
    'pvals_adj', sorted np.recarray to be indexed by group ids (0:00:32)
['ARCHS4_Cell-lines', 'ARCHS4_IDG_Coexp', 'ARCHS4_Kinases_Coexp', 'ARCHS4_TFs_Coexp', 'ARCHS4_Tissues', 'Achilles_fitness_decrease', 'Achilles_fitness_increase', 'Aging_Perturbations_from_GEO_down', 'Aging_Perturbations_from_GEO_up', 'Allen_Brain_Atlas_10x_scRNA_2021', 'Allen_Brain_Atlas_down', 'Allen_Brain_Atlas_up', 'Azimuth_Cell_Types_2021', 'BioCarta_2013', 'BioCarta_2015', 'BioCarta_2016', 'BioPlanet_2019', 'BioPlex_2017', 'CCLE_Proteomics_2020', 'CORUM', 'COVID-19_Related_Gene_Sets', 'COVID-19_Related_Gene_Sets_2021', 'Cancer_Cell_Line_Encyclopedia', 'CellMarker_Augmented_2021', 'ChEA_2013', 'ChEA_2015', 'ChEA_2016', 'ChEA_2022', 'Chromosome_Location', 'Chromosome_Location_hg19', 'ClinVar_2019', 'DSigDB', 'Data_Acquisition_Method_Most_Popular_Genes', 'DepMap_WG_CRISPR_Screens_Broad_CellLines_2019', 'DepMap_WG_CRISPR_Screens_Sanger_CellLines_2019', 'Descartes_Cell_Types_and_Tissue_2021', 'Diabetes_Perturbations_GEO_2022', 'DisGeNET', 'Disease_Perturbations_from_GEO_down', 'Disease_Perturbations_from_GEO_up', 'Disease_Signatures_from_GEO_down_2014', 'Disease_Signatures_from_GEO_up_2014', 'DrugMatrix', 'Drug_Perturbations_from_GEO_2014', 'Drug_Perturbations_from_GEO_down', 'Drug_Perturbations_from_GEO_up', 'ENCODE_Histone_Modifications_2013', 'ENCODE_Histone_Modifications_2015', 'ENCODE_TF_ChIP-seq_2014', 'ENCODE_TF_ChIP-seq_2015', 'ENCODE_and_ChEA_Consensus_TFs_from_ChIP-X', 'ESCAPE', 'Elsevier_Pathway_Collection', 'Enrichr_Libraries_Most_Popular_Genes', 'Enrichr_Submissions_TF-Gene_Coocurrence', 'Enrichr_Users_Contributed_Lists_2020', 'Epigenomics_Roadmap_HM_ChIP-seq', 'FANTOM6_lncRNA_KD_DEGs', 'GO_Biological_Process_2013', 'GO_Biological_Process_2015', 'GO_Biological_Process_2017', 'GO_Biological_Process_2017b', 'GO_Biological_Process_2018', 'GO_Biological_Process_2021', 'GO_Cellular_Component_2013', 'GO_Cellular_Component_2015', 'GO_Cellular_Component_2017', 'GO_Cellular_Component_2017b', 'GO_Cellular_Component_2018', 'GO_Cellular_Component_2021', 'GO_Molecular_Function_2013', 'GO_Molecular_Function_2015', 'GO_Molecular_Function_2017', 'GO_Molecular_Function_2017b', 'GO_Molecular_Function_2018', 'GO_Molecular_Function_2021', 'GTEx_Aging_Signatures_2021', 'GTEx_Tissue_Expression_Down', 'GTEx_Tissue_Expression_Up', 'GTEx_Tissues_V8_2023', 'GWAS_Catalog_2019', 'GeneSigDB', 'Gene_Perturbations_from_GEO_down', 'Gene_Perturbations_from_GEO_up', 'Genes_Associated_with_NIH_Grants', 'Genome_Browser_PWMs', 'GlyGen_Glycosylated_Proteins_2022', 'HDSigDB_Human_2021', 'HDSigDB_Mouse_2021', 'HMDB_Metabolites', 'HMS_LINCS_KinomeScan', 'HomoloGene', 'HuBMAP_ASCT_plus_B_augmented_w_RNAseq_Coexpression', 'HuBMAP_ASCTplusB_augmented_2022', 'HumanCyc_2015', 'HumanCyc_2016', 'Human_Gene_Atlas', 'Human_Phenotype_Ontology', 'IDG_Drug_Targets_2022', 'InterPro_Domains_2019', 'Jensen_COMPARTMENTS', 'Jensen_DISEASES', 'Jensen_TISSUES', 'KEA_2013', 'KEA_2015', 'KEGG_2013', 'KEGG_2015', 'KEGG_2016', 'KEGG_2019_Human', 'KEGG_2019_Mouse', 'KEGG_2021_Human', 'KOMP2_Mouse_Phenotypes_2022', 'Kinase_Perturbations_from_GEO_down', 'Kinase_Perturbations_from_GEO_up', 'L1000_Kinase_and_GPCR_Perturbations_down', 'L1000_Kinase_and_GPCR_Perturbations_up', 'LINCS_L1000_CRISPR_KO_Consensus_Sigs', 'LINCS_L1000_Chem_Pert_Consensus_Sigs', 'LINCS_L1000_Chem_Pert_down', 'LINCS_L1000_Chem_Pert_up', 'LINCS_L1000_Ligand_Perturbations_down', 'LINCS_L1000_Ligand_Perturbations_up', 'Ligand_Perturbations_from_GEO_down', 'Ligand_Perturbations_from_GEO_up', 'MAGMA_Drugs_and_Diseases', 'MCF7_Perturbations_from_GEO_down', 'MCF7_Perturbations_from_GEO_up', 'MGI_Mammalian_Phenotype_2013', 'MGI_Mammalian_Phenotype_2017', 'MGI_Mammalian_Phenotype_Level_3', 'MGI_Mammalian_Phenotype_Level_4', 'MGI_Mammalian_Phenotype_Level_4_2019', 'MGI_Mammalian_Phenotype_Level_4_2021', 'MSigDB_Computational', 'MSigDB_Hallmark_2020', 'MSigDB_Oncogenic_Signatures', 'Metabolomics_Workbench_Metabolites_2022', 'Microbe_Perturbations_from_GEO_down', 'Microbe_Perturbations_from_GEO_up', 'Mouse_Gene_Atlas', 'NCI-60_Cancer_Cell_Lines', 'NCI-Nature_2015', 'NCI-Nature_2016', 'NIH_Funded_PIs_2017_AutoRIF_ARCHS4_Predictions', 'NIH_Funded_PIs_2017_GeneRIF_ARCHS4_Predictions', 'NIH_Funded_PIs_2017_Human_AutoRIF', 'NIH_Funded_PIs_2017_Human_GeneRIF', 'NURSA_Human_Endogenous_Complexome', 'OMIM_Disease', 'OMIM_Expanded', 'Old_CMAP_down', 'Old_CMAP_up', 'Orphanet_Augmented_2021', 'PFOCR_Pathways', 'PPI_Hub_Proteins', 'PanglaoDB_Augmented_2021', 'Panther_2015', 'Panther_2016', 'Pfam_Domains_2019', 'Pfam_InterPro_Domains', 'PheWeb_2019', 'PhenGenI_Association_2021', 'Phosphatase_Substrates_from_DEPOD', 'ProteomicsDB_2020', 'Proteomics_Drug_Atlas_2023', 'RNA-Seq_Disease_Gene_and_Drug_Signatures_from_GEO', 'RNAseq_Automatic_GEO_Signatures_Human_Down', 'RNAseq_Automatic_GEO_Signatures_Human_Up', 'RNAseq_Automatic_GEO_Signatures_Mouse_Down', 'RNAseq_Automatic_GEO_Signatures_Mouse_Up', 'Rare_Diseases_AutoRIF_ARCHS4_Predictions', 'Rare_Diseases_AutoRIF_Gene_Lists', 'Rare_Diseases_GeneRIF_ARCHS4_Predictions', 'Rare_Diseases_GeneRIF_Gene_Lists', 'Reactome_2013', 'Reactome_2015', 'Reactome_2016', 'Reactome_2022', 'SILAC_Phosphoproteomics', 'Serine_Threonine_Kinome_Atlas_2023', 'SubCell_BarCode', 'SynGO_2022', 'SysMyo_Muscle_Gene_Sets', 'TF-LOF_Expression_from_GEO', 'TF_Perturbations_Followed_by_Expression', 'TG_GATES_2020', 'TRANSFAC_and_JASPAR_PWMs', 'TRRUST_Transcription_Factors_2019', 'Table_Mining_of_CRISPR_Studies', 'Tabula_Muris', 'Tabula_Sapiens', 'TargetScan_microRNA', 'TargetScan_microRNA_2017', 'Tissue_Protein_Expression_from_Human_Proteome_Map', 'Tissue_Protein_Expression_from_ProteomicsDB', 'Transcription_Factor_PPIs', 'UK_Biobank_GWAS_v1', 'Virus-Host_PPI_P-HIPSTer_2020', 'VirusMINT', 'Virus_Perturbations_from_GEO_down', 'Virus_Perturbations_from_GEO_up', 'WikiPathway_2021_Human', 'WikiPathways_2013', 'WikiPathways_2015', 'WikiPathways_2016', 'WikiPathways_2019_Human', 'WikiPathways_2019_Mouse', 'dbGaP', 'huMAP', 'lncHUB_lncRNA_Co-Expression', 'miRTarBase_2017']
1134
```

Example 51 (python):
```python
sc.settings.set_figure_params(dpi=80, color_map='viridis', vector_friendly=False,  dpi_save=300, facecolor='white')
gseapy.barplot(enr_res.res2d,title='GO_Biological_Process_2021',top_term=20,figsize=(3,5))
```

Example 52 (python):
```python
glist = sc.get.rank_genes_groups_df(adata, group='compact', 
                                    key='wilcoxon', log2fc_min=0.25, 
                                    pval_cutoff=0.05)['names'].squeeze().str.strip().tolist()
print(len(glist))                                  
#GO_Biological_Process_2021,GO_Cellular_Component_2021,GO_Molecular_Function_2021
enr_res = gseapy.enrichr(gene_list=glist,
                     organism='Human',
                     gene_sets='GO_Biological_Process_2021',
                     cutoff = 0.5)
enr_res.results.head(20)
gseapy.barplot(enr_res.res2d,title='GO_Biological_Process_2021',top_term=20,figsize=(3,5))
```

Example 53 (python):
```python
adata=sc.read('./11MayAllCMs.h5ad')
print(adata.shape)
```

Example 54 (unknown):
```unknown
(74884, 1821)
```

Example 55 (python):
```python
plt.rcParams['figure.figsize'] = [3,2]
adata.obs['cycling2']= 'na'
adata.obs['cycling2'][[x for x in adata.obs_names if adata.obs.cycling[x] in ['no']]]='non-cycling'
adata.obs['cycling2'][[x for x in adata.obs_names if adata.obs.cycling[x] in ['yes']]]='cycling'
tmp = pd.crosstab(adata.obs['week'],adata.obs['cycling2'], normalize='index')
tmp.plot.bar(stacked=True).legend(loc='lower left')
plt.title('all cardiomyocytes')
plt.ylabel('proportion')
```

Example 56 (unknown):
```unknown
Text(0, 0.5, 'proportion')
```

Example 57 (python):
```python
kdata=adata[adata.obs.mid_grain=='AtrialCardiomyocytes',:].copy()
kdata.obs['cycling2']= 'na'
kdata.obs['cycling2'][[x for x in kdata.obs_names if kdata.obs.cycling[x] in ['no']]]='non-cycling'
kdata.obs['cycling2'][[x for x in kdata.obs_names if kdata.obs.cycling[x] in ['yes']]]='cycling'

plt.rcParams['figure.figsize'] = [3,2]
tmp = pd.crosstab(kdata.obs['week'],kdata.obs['cycling2'], normalize='index')
tmp.plot.bar(stacked=True).legend(loc='lower left')
plt.title('atrial cardiomyocytes')
plt.ylabel('proportion')
```

Example 58 (unknown):
```unknown
Text(0, 0.5, 'proportion')
```

Example 59 (python):
```python
kdata=adata[adata.obs.mid_grain=='VentricularCardiomyocytes',:].copy()
kdata.obs['cycling2']= 'na'
kdata.obs['cycling2'][[x for x in kdata.obs_names if kdata.obs.cycling[x] in ['no']]]='non-cycling'
kdata.obs['cycling2'][[x for x in kdata.obs_names if kdata.obs.cycling[x] in ['yes']]]='cycling'
plt.rcParams['figure.figsize'] = [3,2]
tmp = pd.crosstab(kdata.obs['week'],kdata.obs['cycling2'], normalize='index')
tmp.plot.bar(stacked=True).legend(loc='lower left')
plt.title('ventricular cardiomyocytes')
plt.ylabel('proportion')
```

Example 60 (unknown):
```unknown
Text(0, 0.5, 'proportion')
```

Example 61 (python):
```python
kdata=adata[adata.obs.mid_grain=='CardiacConductionSystem',:].copy()
kdata.obs['cycling2']= 'na'
kdata.obs['cycling2'][[x for x in kdata.obs_names if kdata.obs.cycling[x] in ['no']]]='non-cycling'
kdata.obs['cycling2'][[x for x in kdata.obs_names if kdata.obs.cycling[x] in ['yes']]]='cycling'
plt.rcParams['figure.figsize'] = [3,2]
tmp = pd.crosstab(kdata.obs['week'],kdata.obs['cycling2'], normalize='index')
tmp.plot.bar(stacked=True).legend(loc='lower left')
plt.title('cardiac conduction system')
plt.ylabel('proportion')
```

Example 62 (unknown):
```unknown
Text(0, 0.5, 'proportion')
```

Example 63 (python):
```python
kdata = adata[adata.obs['fine_grain'].isin(['VentricularCardiomyocytesRightCompact','VentricularCardiomyocytesLeftCompact'])].copy()
kdata.obs['cycling2']= 'na'
kdata.obs['cycling2'][[x for x in kdata.obs_names if kdata.obs.cycling[x] in ['no']]]='non-cycling'
kdata.obs['cycling2'][[x for x in kdata.obs_names if kdata.obs.cycling[x] in ['yes']]]='cycling'
plt.rcParams['figure.figsize'] = [3,2]
tmp = pd.crosstab(kdata.obs['week'],kdata.obs['cycling2'], normalize='index')
tmp.plot.bar(stacked=True).legend(loc='lower left')
plt.title('ventricular compact cardiomyocytes')
plt.ylabel('proportion')
```

Example 64 (unknown):
```unknown
Text(0, 0.5, 'proportion')
```

Example 65 (python):
```python
kdata = adata[adata.obs['fine_grain'].isin(['VentricularCardiomyocytesRightTrabeculated','VentricularCardiomyocytesLeftTrabeculated'])].copy()
kdata.obs['cycling2']= 'na'
kdata.obs['cycling2'][[x for x in kdata.obs_names if kdata.obs.cycling[x] in ['no']]]='non-cycling'
kdata.obs['cycling2'][[x for x in kdata.obs_names if kdata.obs.cycling[x] in ['yes']]]='cycling'
plt.rcParams['figure.figsize'] = [3,2]
tmp = pd.crosstab(kdata.obs['week'],kdata.obs['cycling2'], normalize='index')
tmp.plot.bar(stacked=True).legend(loc='lower left')
plt.title('ventricular trabeculated cardiomyocytes')
plt.ylabel('proportion')
```

Example 66 (unknown):
```unknown
Text(0, 0.5, 'proportion')
```

Example 67 (python):
```python
kdata = adata[adata.obs['fine_grain'].isin(['VentricularCardiomyocytesRightTrabeculated','VentricularCardiomyocytesRightCompact'])].copy()
kdata.obs['cycling2']= 'na'
kdata.obs['cycling2'][[x for x in kdata.obs_names if kdata.obs.cycling[x] in ['no']]]='non-cycling'
kdata.obs['cycling2'][[x for x in kdata.obs_names if kdata.obs.cycling[x] in ['yes']]]='cycling'
plt.rcParams['figure.figsize'] = [3,2]
tmp = pd.crosstab(kdata.obs['week'],kdata.obs['cycling2'], normalize='index')
tmp.plot.bar(stacked=True).legend(loc='lower left')
plt.title('right ventricular cardiomyocytes')
plt.ylabel('proportion')
```

Example 68 (unknown):
```unknown
Text(0, 0.5, 'proportion')
```

Example 69 (python):
```python
kdata = adata[adata.obs['fine_grain'].isin(['VentricularCardiomyocytesLeftTrabeculated','VentricularCardiomyocytesLeftCompact'])].copy()
kdata.obs['cycling2']= 'na'
kdata.obs['cycling2'][[x for x in kdata.obs_names if kdata.obs.cycling[x] in ['no']]]='non-cycling'
kdata.obs['cycling2'][[x for x in kdata.obs_names if kdata.obs.cycling[x] in ['yes']]]='cycling'
plt.rcParams['figure.figsize'] = [3,2]
tmp = pd.crosstab(kdata.obs['week'],kdata.obs['cycling2'], normalize='index')
tmp.plot.bar(stacked=True).legend(loc='lower left')
plt.title('left ventricular cardiomyocytes')
plt.ylabel('proportion')
```

Example 70 (unknown):
```unknown
Text(0, 0.5, 'proportion')
```

Example 71 (python):
```python
kdata = adata[adata.obs['fine_grain'].isin(['VentricularCardiomyocytesRightTrabeculated','VentricularCardiomyocytesLeftTrabeculated','VentricularCardiomyocytesRightCompact','VentricularCardiomyocytesLeftCompact'])].copy()
kdata.obs['compacttrabeculated']= 'na'
kdata.obs['compacttrabeculated'][[x for x in kdata.obs_names if kdata.obs.fine_grain[x] in ['VentricularCardiomyocytesLeftTrabeculated','VentricularCardiomyocytesRightTrabeculated']]]='trabeculated'
kdata.obs['compacttrabeculated'][[x for x in kdata.obs_names if kdata.obs.fine_grain[x] in ['VentricularCardiomyocytesRightCompact','VentricularCardiomyocytesLeftCompact']]]='compact'

plt.rcParams['figure.figsize'] = [3,2]
tmp = pd.crosstab(kdata.obs['week'],kdata.obs['compacttrabeculated'], normalize='index')
tmp.plot.bar(stacked=True).legend(loc='lower left')
plt.title('ventricular cardiomyocytes')
plt.ylabel('proportion')
```

Example 72 (unknown):
```unknown
Text(0, 0.5, 'proportion')
```

Example 73 (python):
```python
#Getting the UMAPs for clusterofinterest
ClusterOfInterest='yes'
#sc.tl.rank_genes_groups(adata, 'cycling', method='t-test', key_added = "t-test")
DEGenesForCluster = [x[ClusterOfInterest] for x in adata.uns['t-test']['names'][:50]]
DEGenesForCluster=list(set(DEGenesForCluster)-set(cell_cycle_genes)-set(ribo_genes))
sc.pl.dotplot(adata, DEGenesForCluster ,groupby='cycling',standard_scale='var',cmap='YlGn')
#sc.pl.umap(adata, color=DEGenesForCluster)
```

Example 74 (python):
```python
bdata=sc.read('./AdultAtrialCMs.h5ad')
bdata.obs['batch_key'] = bdata.obs['donor']
bdata.obs['total_counts'] = bdata.obs['n_counts']
bdata.obs['pct_counts_mt'] = bdata.obs['percent_mito']
bdata.obs['pct_counts_ribo'] = bdata.obs['percent_ribo']
print(bdata.shape)

cdata=sc.read('./AdultVentricularCMs.h5ad')
cdata.obs['batch_key'] = cdata.obs['donor']
cdata.obs['total_counts'] = cdata.obs['n_counts']
cdata.obs['pct_counts_mt'] = cdata.obs['percent_mito']
cdata.obs['pct_counts_ribo'] = cdata.obs['percent_ribo']
print(cdata.shape)

adata = adata.concatenate(bdata,cdata) 
print(adata)
del bdata
del cdata
```

Example 75 (unknown):
```unknown
(23483, 33538)
(125289, 33538)
AnnData object with n_obs × n_vars = 223656 × 31915
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score', 'NRP', 'age_group', 'cell_source', 'cell_type', 'gender', 'percent_mito', 'percent_ribo', 'sample', 'source', 'type', 'version', 'cell_states', 'Used', 'batch'
    var: 'gene_name_scRNA-0-0', 'gene_id-0', 'gene_ids-Harvard-Nuclei-1', 'feature_types-Harvard-Nuclei-1', 'gene_ids-Sanger-Nuclei-1', 'feature_types-Sanger-Nuclei-1', 'gene_ids-Sanger-Cells-1', 'feature_types-Sanger-Cells-1', 'gene_ids-Sanger-CD45-1', 'feature_types-Sanger-CD45-1', 'gene_ids-Harvard-Nuclei-2', 'feature_types-Harvard-Nuclei-2', 'gene_ids-Sanger-Nuclei-2', 'feature_types-Sanger-Nuclei-2', 'gene_ids-Sanger-Cells-2', 'feature_types-Sanger-Cells-2', 'gene_ids-Sanger-CD45-2', 'feature_types-Sanger-CD45-2'
    obsm: 'X_umap'
```

Example 76 (python):
```python
# preserve raw count
adata.layers["counts"] = adata.X.copy()

# normalise and log-transform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key='batch_key')
#sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5)

print(sum(adata.var.highly_variable))
sc.pl.highly_variable_genes(adata)

# freeze the state in `.raw`
adata.raw = adata

# subset hvgs
adata = adata[:, adata.var.highly_variable]

adata=adata.copy()

sc.tl.score_genes_cell_cycle(adata, s_genes=s_genes, g2m_genes=g2m_genes)
#sc.pp.regress_out(adata, ['S_score','G2M_score'])

#batch correction
scvi.model.SCVI.setup_anndata(adata, 
                              layer="counts", 
                              batch_key="batch_key",
                              categorical_covariate_keys=['phase'],
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo'])

vae = scvi.model.SCVI(adata, n_hidden = 128, n_latent = 50, n_layers = 3, dispersion = 'gene-batch')
vae.train(max_epochs=50)
    
# Convergence plot
x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
plt.plot(x, vae.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
plt.clf()

# add the latent space to adata
adata.obsm["X_scVI"] = vae.get_latent_representation()

sc.tl.score_genes(adata, stress_genes, score_name='stress_score')
sc.tl.score_genes(adata, hb_genes1, score_name='hb1_score')

sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata,min_dist=0.1)
#sc.tl.tsne(adata)
sc.pl.umap(adata, color=['log1p_total_counts','n_genes','n_genes_by_counts','total_counts','pct_counts_mt','pct_counts_ribo'],ncols=5)
sc.pl.umap(adata, color=['scrublet_score','stress_score', 'hb1_score','phase'], ncols=5,)

sc.pl.umap(adata, color=['donor','age','week','trimester','region','phase','cycling','cell_or_nuclei','modality','kit_10x','FACSgate','heart_or_greatvessels','coarse_grain','mid_grain','fine_grain'],wspace=0.8,ncols=5)
Genes=['TTN','TNNT2','MYOM1','LDB3','ALDH1A2','TNNT1','KRT19','UPK3B','BNC1','COL6A3','FBLN1','PCOLCE','LRRC17','PCDH18','EMCN','ECSCR','PECAM1','CDH5','ESAM', 'LAPTM5','PTPRC','LCP1','CD53','WAS','PHOX2B','NRXN1','STMN2','GRIK2','L1CAM']
sc.pl.umap(adata, color=Genes)
```

Example 77 (unknown):
```unknown
normalizing counts per cell
    finished (0:00:01)
extracting highly variable genes
```

Example 78 (unknown):
```unknown
... storing 'donor' as categorical
... storing 'region' as categorical
... storing 'batch_key' as categorical
... storing 'cell_type' as categorical
... storing 'sample' as categorical
... storing 'cell_states' as categorical
```

Example 79 (python):
```python
finished (0:00:40)
--> added
    'highly_variable', boolean vector (adata.var)
    'means', float vector (adata.var)
    'dispersions', float vector (adata.var)
    'dispersions_norm', float vector (adata.var)
1412
```

Example 80 (python):
```python
calculating cell cycle phase
computing score 'S_score'
WARNING: genes are not in var_names and ignored: ['MCM5', 'FEN1', 'MCM2', 'MCM4', 'RRM1', 'UNG', 'GINS2', 'MCM6', 'CDCA7', 'PRIM1', 'UHRF1', 'RFC2', 'RPA2', 'NASP', 'RAD51AP1', 'GMNN', 'SLBP', 'CCNE2', 'UBR7', 'POLD3', 'MSH2', 'RAD51', 'CDC45', 'CDC6', 'EXO1', 'TIPIN', 'DSCC1', 'BLM', 'CASP8AP2', 'USP1', 'CHAF1B', 'E2F8']
    finished: added
    'S_score', score of gene set (adata.obs).
    246 total control genes are used. (0:00:01)
computing score 'G2M_score'
WARNING: genes are not in var_names and ignored: ['TMPO', 'AURKB', 'ANP32E', 'HJURP', 'TTK', 'CDC25C', 'RANGAP1', 'PSRC1', 'LBR', 'CKAP5', 'CTCF', 'NEK2', 'CBX5']
    finished: added
    'G2M_score', score of gene set (adata.obs).
    599 total control genes are used. (0:00:01)
-->     'phase', cell cycle phase (adata.obs)
```

Example 81 (unknown):
```unknown
GPU available: False, used: False
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
```

Example 82 (unknown):
```unknown
Epoch 50/50: 100%|███████████| 50/50 [14:37<00:00, 17.54s/it, loss=600, v_num=1]
```

Example 83 (python):
```python
computing score 'stress_score'
WARNING: genes are not in var_names and ignored: ['JUND', 'CEBPD', 'CEBPB', 'HSP90AB1', 'HSPA8', 'HSPE1']
    finished: added
    'stress_score', score of gene set (adata.obs).
    635 total control genes are used. (0:00:01)
computing score 'hb1_score'
WARNING: genes are not in var_names and ignored: ['HBD', 'HBE1', 'HBZ', 'HBM', 'HBQ1']
    finished: added
    'hb1_score', score of gene set (adata.obs).
    245 total control genes are used. (0:00:01)
computing neighbors
    finished: added to `.uns['neighbors']`
    `.obsp['distances']`, distances for each pair of neighbors
    `.obsp['connectivities']`, weighted adjacency matrix (0:00:23)
computing UMAP
    finished: added
    'X_umap', UMAP coordinates (adata.obsm) (0:01:27)
```

Example 84 (unknown):
```unknown
... storing 'phase' as categorical
```

Example 85 (unknown):
```unknown
<Figure size 320x320 with 0 Axes>
```

Example 86 (python):
```python
#adata.write('./29MarchAdultandFetalCMsProcessedTogether.h5ad')
```

Example 87 (python):
```python
sc.pl.umap(adata, color='fine_grain')
```

Example 88 (python):
```python
AtrialCMsleiden=['1']
VentricularCMsleiden=['0']
adata.obs['cmtype']= 'na'
adata.obs['cmtype'][[x for x in adata.obs_names if adata.obs.leiden[x] in AtrialCMsleiden]]='atrial'
adata.obs['cmtype'][[x for x in adata.obs_names if adata.obs.leiden[x] in VentricularCMsleiden]]='ventricular'
sc.pl.umap(adata, color='cmtype')

FetalDonors=['Hst41','Hst39','BRC2263','BRC2262','BRC2251','Hst33','C92','C97','C98','Hst36','C99','BRC2260','C104','BRC2256','C94','C87','BRC2253','C85','C86','Hst40','C82']
AdultDonors=['D7','D6','D2','D1','H7','H5','D3','D5','H6','H3','D11','H4','D4','H2']
adata.obs['donortype']= 'na'
adata.obs['donortype'][[x for x in adata.obs_names if adata.obs.donor[x] in FetalDonors]]='fetal'
adata.obs['donortype'][[x for x in adata.obs_names if adata.obs.donor[x] in AdultDonors]]='adult'
sc.pl.umap(adata, color='donortype')
```

Example 89 (unknown):
```unknown
... storing 'cmtype' as categorical
```

Example 90 (unknown):
```unknown
... storing 'donortype' as categorical
```

Example 91 (python):
```python
bdata=adata[adata.obs.cmtype=='ventricular',:].copy()
ClusterOfInterest='fetal'
sc.tl.rank_genes_groups(bdata, 'donortype', method='t-test', key_added = "t-test")
DEGenesForCluster = [x[ClusterOfInterest] for x in bdata.uns['t-test']['names'][:150]]
DEGenesForCluster=list(set(DEGenesForCluster)-set(mito_genes)-set(ribo_genes))
FetalVentricularList=DEGenesForCluster
sc.pl.dotplot(bdata, DEGenesForCluster ,groupby='donortype',standard_scale='var',cmap='PRGn')
#sc.pl.umap(bdata, color=DEGenesForCluster)

ClusterOfInterest='adult'
DEGenesForCluster = [x[ClusterOfInterest] for x in bdata.uns['t-test']['names'][:150]]
DEGenesForCluster=list(set(DEGenesForCluster)-set(mito_genes)-set(ribo_genes))
AdultVentricularList=DEGenesForCluster
sc.pl.dotplot(bdata, DEGenesForCluster ,groupby='donortype',standard_scale='var',cmap='PRGn')

bdata=adata[adata.obs.cmtype=='atrial',:].copy()
ClusterOfInterest='fetal'
sc.tl.rank_genes_groups(bdata, 'donortype', method='t-test', key_added = "t-test")
DEGenesForCluster = [x[ClusterOfInterest] for x in bdata.uns['t-test']['names'][:150]]
DEGenesForCluster=list(set(DEGenesForCluster)-set(mito_genes)-set(ribo_genes))
FetalAtrialList=DEGenesForCluster
sc.pl.dotplot(bdata, DEGenesForCluster ,groupby='donortype',standard_scale='var',cmap='PRGn')
#sc.pl.umap(bdata, color=DEGenesForCluster)

ClusterOfInterest='adult'
DEGenesForCluster = [x[ClusterOfInterest] for x in bdata.uns['t-test']['names'][:150]]
DEGenesForCluster=list(set(DEGenesForCluster)-set(mito_genes)-set(ribo_genes))
AdultAtrialList=DEGenesForCluster
sc.pl.dotplot(bdata, DEGenesForCluster ,groupby='donortype',standard_scale='var',cmap='PRGn')
```

Example 92 (python):
```python
ranking genes
    finished: added to `.uns['t-test']`
    'names', sorted np.recarray to be indexed by group ids
    'scores', sorted np.recarray to be indexed by group ids
    'logfoldchanges', sorted np.recarray to be indexed by group ids
    'pvals', sorted np.recarray to be indexed by group ids
    'pvals_adj', sorted np.recarray to be indexed by group ids (0:00:04)
```

Example 93 (python):
```python
ranking genes
    finished: added to `.uns['t-test']`
    'names', sorted np.recarray to be indexed by group ids
    'scores', sorted np.recarray to be indexed by group ids
    'logfoldchanges', sorted np.recarray to be indexed by group ids
    'pvals', sorted np.recarray to be indexed by group ids
    'pvals_adj', sorted np.recarray to be indexed by group ids (0:00:01)
```

Example 94 (python):
```python
CommonAdult=list(set(AdultVentricularList)&set(AdultAtrialList))
CommonFetal=list(set(FetalVentricularList)&set(FetalAtrialList))

sc.pl.dotplot(adata, CommonAdult ,groupby='donortype',standard_scale='var',cmap='PRGn')
sc.pl.dotplot(adata, CommonFetal ,groupby='donortype',standard_scale='var',cmap='PRGn')
```

Example 95 (python):
```python
bdata=adata[adata.obs.cmtype=='ventricular',:].copy()
ClusterOfInterest='fetal'
sc.tl.rank_genes_groups(bdata, 'donortype', method='t-test', key_added = "t-test")
DEGenesForCluster = [x[ClusterOfInterest] for x in bdata.uns['t-test']['names'][:500]]
#DEGenesForCluster=list(set(DEGenesForCluster)-set(mito_genes)-set(ribo_genes))
FetalVentricularList=DEGenesForCluster
#sc.pl.dotplot(bdata, DEGenesForCluster ,groupby='donortype',standard_scale='var',cmap='PRGn')
#sc.pl.umap(bdata, color=DEGenesForCluster)

ClusterOfInterest='adult'
DEGenesForCluster = [x[ClusterOfInterest] for x in bdata.uns['t-test']['names'][:500]]
#DEGenesForCluster=list(set(DEGenesForCluster)-set(mito_genes)-set(ribo_genes))
AdultVentricularList=DEGenesForCluster
#sc.pl.dotplot(bdata, DEGenesForCluster ,groupby='donortype',standard_scale='var',cmap='PRGn')

bdata=adata[adata.obs.cmtype=='atrial',:].copy()
ClusterOfInterest='fetal'
sc.tl.rank_genes_groups(bdata, 'donortype', method='t-test', key_added = "t-test")
DEGenesForCluster = [x[ClusterOfInterest] for x in bdata.uns['t-test']['names'][:500]]
#DEGenesForCluster=list(set(DEGenesForCluster)-set(mito_genes)-set(ribo_genes))
FetalAtrialList=DEGenesForCluster
#sc.pl.dotplot(bdata, DEGenesForCluster ,groupby='donortype',standard_scale='var',cmap='PRGn')
#sc.pl.umap(bdata, color=DEGenesForCluster)

ClusterOfInterest='adult'
DEGenesForCluster = [x[ClusterOfInterest] for x in bdata.uns['t-test']['names'][:500]]
#DEGenesForCluster=list(set(DEGenesForCluster)-set(mito_genes)-set(ribo_genes))
AdultAtrialList=DEGenesForCluster
#sc.pl.dotplot(bdata, DEGenesForCluster ,groupby='donortype',standard_scale='var',cmap='PRGn')

CommonAdult=list(set(AdultVentricularList)&set(AdultAtrialList))
CommonFetal=list(set(FetalVentricularList)&set(FetalAtrialList))

CommonAdultTFs=list(set(CommonAdult)&set(TFs))
CommonFetalTFs=list(set(CommonFetal)&set(TFs))

sc.pl.dotplot(adata, CommonAdultTFs ,groupby='donortype',standard_scale='var',cmap='PRGn')
sc.pl.dotplot(adata, CommonFetalTFs ,groupby='donortype',standard_scale='var',cmap='PRGn')
```

Example 96 (python):
```python
ranking genes
    finished: added to `.uns['t-test']`
    'names', sorted np.recarray to be indexed by group ids
    'scores', sorted np.recarray to be indexed by group ids
    'logfoldchanges', sorted np.recarray to be indexed by group ids
    'pvals', sorted np.recarray to be indexed by group ids
    'pvals_adj', sorted np.recarray to be indexed by group ids (0:00:05)
ranking genes
    finished: added to `.uns['t-test']`
    'names', sorted np.recarray to be indexed by group ids
    'scores', sorted np.recarray to be indexed by group ids
    'logfoldchanges', sorted np.recarray to be indexed by group ids
    'pvals', sorted np.recarray to be indexed by group ids
    'pvals_adj', sorted np.recarray to be indexed by group ids (0:00:01)
```

Example 97 (python):
```python
sc.pl.dotplot(adata, 'SOX4' ,groupby='fine_grain',standard_scale='var',cmap='PRGn')
```

Example 98 (python):
```python
adata = adata[adata.obs['mid_grain'].isin(['AtrialCardiomyocytes','VentricularCardiomyocytes','CardiacConductionSystem'])].copy()
```

Example 99 (unknown):
```unknown
AnnData object with n_obs × n_vars = 74884 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score'
    var: 'gene_name_scRNA-0', 'gene_id'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'age_colors', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'cycling_colors', 'donor_colors', 'fine_grain_colors', 'heart_or_greatvessels_colors', 'hvg', 'kit_10x_colors', 'mid_grain_colors', 'modality_colors', 'neighbors', 'phase_colors', 'region_colors', 'sex_colors', 'trimester_colors', 'umap', 'week_colors'
    obsm: 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 100 (python):
```python
# preserve raw count
adata.layers["counts"] = adata.X.copy()

# normalise and log-transform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key='batch_key')
#sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5)

print(sum(adata.var.highly_variable))
sc.pl.highly_variable_genes(adata)

# freeze the state in `.raw`
adata.raw = adata

# subset hvgs
adata = adata[:, adata.var.highly_variable]

adata=adata.copy()

sc.tl.score_genes_cell_cycle(adata, s_genes=s_genes, g2m_genes=g2m_genes)
#sc.pp.regress_out(adata, ['S_score','G2M_score'])

#batch correction
scvi.model.SCVI.setup_anndata(adata, 
                              layer="counts", 
                              batch_key="batch_key",
                              categorical_covariate_keys=['phase'],
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo'])

vae = scvi.model.SCVI(adata, n_hidden = 128, n_latent = 50, n_layers = 3, dispersion = 'gene-batch')
vae.train(max_epochs=50)
    
# Convergence plot
x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
plt.plot(x, vae.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
plt.clf()

# add the latent space to adata
adata.obsm["X_scVI"] = vae.get_latent_representation()

sc.tl.score_genes(adata, stress_genes, score_name='stress_score')
sc.tl.score_genes(adata, hb_genes1, score_name='hb1_score')

sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata,min_dist=0.1)
#sc.tl.tsne(adata)
sc.pl.umap(adata, color=['log1p_total_counts','n_genes','n_genes_by_counts','total_counts','pct_counts_mt','pct_counts_ribo'],ncols=5)
sc.pl.umap(adata, color=['scrublet_score','stress_score', 'hb1_score','phase'], ncols=5,)

sc.pl.umap(adata, color=['donor','age','week','trimester','region','phase','cycling','cell_or_nuclei','modality','kit_10x','FACSgate','heart_or_greatvessels','coarse_grain','mid_grain','fine_grain'],wspace=0.8,ncols=5)
Genes=['TTN','TNNT2','MYOM1','LDB3','ALDH1A2','TNNT1','KRT19','UPK3B','BNC1','COL6A3','FBLN1','PCOLCE','LRRC17','PCDH18','EMCN','ECSCR','PECAM1','CDH5','ESAM', 'LAPTM5','PTPRC','LCP1','CD53','WAS','PHOX2B','NRXN1','STMN2','GRIK2','L1CAM']
sc.pl.umap(adata, color=Genes)
```

Example 101 (python):
```python
normalizing counts per cell
    finished (0:00:00)
extracting highly variable genes
    finished (0:00:13)
--> added
    'highly_variable', boolean vector (adata.var)
    'means', float vector (adata.var)
    'dispersions', float vector (adata.var)
    'dispersions_norm', float vector (adata.var)
1821
```

Example 102 (python):
```python
calculating cell cycle phase
computing score 'S_score'
WARNING: genes are not in var_names and ignored: ['FEN1', 'MCM2', 'MCM4', 'RRM1', 'UNG', 'CDCA7', 'PRIM1', 'UHRF1', 'RPA2', 'NASP', 'GMNN', 'SLBP', 'CCNE2', 'UBR7', 'POLD3', 'MSH2', 'RAD51', 'CDC45', 'CDC6', 'EXO1', 'DSCC1', 'BLM', 'CASP8AP2', 'USP1', 'CHAF1B', 'E2F8']
    finished: added
    'S_score', score of gene set (adata.obs).
    369 total control genes are used. (0:00:00)
computing score 'G2M_score'
WARNING: genes are not in var_names and ignored: ['TMPO', 'ANP32E', 'RANGAP1', 'LBR', 'CTCF', 'CBX5']
    finished: added
    'G2M_score', score of gene set (adata.obs).
    768 total control genes are used. (0:00:00)
-->     'phase', cell cycle phase (adata.obs)
```

Example 103 (unknown):
```unknown
GPU available: False, used: False
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
```

Example 104 (unknown):
```unknown
Epoch 50/50: 100%|███████████| 50/50 [05:18<00:00,  6.37s/it, loss=800, v_num=1]
```

Example 105 (python):
```python
computing score 'stress_score'
WARNING: genes are not in var_names and ignored: ['JUND', 'SOCS3', 'CEBPD', 'CEBPB', 'HSP90AA1', 'HSP90AB1', 'HSPA8', 'HSPB1', 'HSPE1']
    finished: added
    'stress_score', score of gene set (adata.obs).
    596 total control genes are used. (0:00:00)
computing score 'hb1_score'
WARNING: genes are not in var_names and ignored: ['HBB', 'HBD', 'HBM', 'HBQ1']
    finished: added
    'hb1_score', score of gene set (adata.obs).
    248 total control genes are used. (0:00:00)
computing neighbors
    finished: added to `.uns['neighbors']`
    `.obsp['distances']`, distances for each pair of neighbors
    `.obsp['connectivities']`, weighted adjacency matrix (0:00:07)
computing UMAP
    finished: added
    'X_umap', UMAP coordinates (adata.obsm) (0:00:21)
```

Example 106 (unknown):
```unknown
... storing 'phase' as categorical
```

Example 107 (unknown):
```unknown
<Figure size 320x320 with 0 Axes>
```

Example 108 (python):
```python
sc.pl.umap(adata, color='fine_grain')
```

Example 109 (python):
```python
adata = adata[adata.obs['fine_grain'].isin(['VentricularCardiomyocytesRightTrabeculated','VentricularCardiomyocytesRightCompact','VentricularCardiomyocytesLeftTrabeculated','VentricularCardiomyocytesLeftCompact'])].copy()
```

Example 110 (python):
```python
adata.obs['compactortrabeculated']= 'na'
adata.obs['compactortrabeculated'][[x for x in adata.obs_names if adata.obs.fine_grain[x] in ['VentricularCardiomyocytesLeftTrabeculated','VentricularCardiomyocytesRightTrabeculated']]]='trabeculated'
adata.obs['compactortrabeculated'][[x for x in adata.obs_names if adata.obs.fine_grain[x] in ['VentricularCardiomyocytesLeftCompact','VentricularCardiomyocytesRightCompact']]]='compact'
sc.pl.umap(adata, color='compactortrabeculated')
```

Example 111 (unknown):
```unknown
... storing 'compactortrabeculated' as categorical
```

Example 112 (python):
```python
ClusterOfInterest='trabeculated'
sc.tl.rank_genes_groups(adata, 'compactortrabeculated', method='t-test', key_added = "t-test")
DEGenesForCluster = [x[ClusterOfInterest] for x in adata.uns['t-test']['names'][:50]]
sc.pl.dotplot(adata, DEGenesForCluster ,groupby='compactortrabeculated',standard_scale='var',cmap='YlGn')
#sc.pl.umap(adata, color=DEGenesForCluster)
```

Example 113 (python):
```python
ranking genes
    finished: added to `.uns['t-test']`
    'names', sorted np.recarray to be indexed by group ids
    'scores', sorted np.recarray to be indexed by group ids
    'logfoldchanges', sorted np.recarray to be indexed by group ids
    'pvals', sorted np.recarray to be indexed by group ids
    'pvals_adj', sorted np.recarray to be indexed by group ids (0:00:01)
```

Example 114 (python):
```python
DEGenesForCluster = [x[ClusterOfInterest] for x in adata.uns['t-test']['names'][50:100]]
sc.pl.dotplot(adata, DEGenesForCluster ,groupby='fine_grain',cmap='YlGn')
```

Example 115 (python):
```python
sc.pl.dotplot(adata, ['SEMA5A','ADCY5','CORIN','SIPA1L1','ADAMTSL1','CRNDE','SGCD','MYOM2','VCAN','SPOCK1','PPP2R2B','ANGPT1'], categories_order=['VentricularCardiomyocytesLeftTrabeculated','VentricularCardiomyocytesRightTrabeculated','VentricularCardiomyocytesLeftCompact','VentricularCardiomyocytesRightCompact'],groupby='fine_grain',standard_scale='var',cmap='YlGn')
```

Example 116 (python):
```python
set(adata.obs.fine_grain)
```

Example 117 (unknown):
```unknown
{'VentricularCardiomyocytesLeftCompact',
 'VentricularCardiomyocytesLeftTrabeculated',
 'VentricularCardiomyocytesRightCompact',
 'VentricularCardiomyocytesRightTrabeculated'}
```

---

## Read in ID metadata¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/RNA/QC/Demultiplex/No3_SOC-CMO-comparison-with-GEX.html

**Contents:**
- Read in ID metadata¶
- Read in Multiome data¶
- Add SOC result¶
- Add CMO result¶
- Comparison of SOC and CMO result with confusion matrix¶
- Comparison of SOC and CMO result with GEX (UMAP)¶
- "CMO | ratio" of matched and unmatched nuclei¶
- setting a threshold with matched and unmatched nuclei¶
  - Option 1: to be max(tpr-fpr)¶
  - Option 2: fpr lower than 0.05¶
- Additional work¶
  - Only SOC-assigned ones¶

--> similar patter between SOC and CMO

--> CMO is lacking assignments in the left island

--> CMO is over-assigning nuclei in the lower right part of the largest island

--> Unmatched samples has smaller "prop_delta" value, which suggest this metrics is usable for rescuing nuclei using CMO result (for those nuclei whose donor were not assigned by SOC)

27498 rows × 35 columns

194705 rows × 35 columns

73111 rows × 37 columns

**Examples:**

Example 1 (python):
```python
import warnings
warnings.simplefilter(action='ignore', category=FutureWarning)

import numpy as np
import pandas as pd
import scanpy as sc
import session_info
import matplotlib.pyplot as plt
import seaborn as sns


import os
```

Example 2 (unknown):
```unknown
session_info.show()
```

Example 3 (python):
```python
-----
matplotlib          3.5.2
numpy               1.21.5
pandas              1.3.5
scanpy              1.8.2
seaborn             0.11.1
session_info        1.0.0
-----
```

Example 4 (python):
```python
PIL                 9.1.1
anndata             0.8.0
asciitree           NA
backcall            0.2.0
beta_ufunc          NA
binom_ufunc         NA
cached_property     1.5.1
cairo               1.20.0
cffi                1.14.4
cloudpickle         1.6.0
colorama            0.4.4
cycler              0.10.0
cython_runtime      NA
cytoolz             0.11.2
dask                2022.01.0
dateutil            2.8.1
decorator           4.4.2
defusedxml          0.6.0
fasteners           0.17.3
fsspec              2022.01.0
google              NA
h5py                3.1.0
igraph              0.10.2
importlib_metadata  1.7.0
ipykernel           5.4.2
ipython_genutils    0.2.0
ipywidgets          7.6.3
jedi                0.17.2
jinja2              3.1.2
joblib              1.0.0
kiwisolver          1.3.1
leidenalg           0.9.0
llvmlite            0.35.0
louvain             0.7.0
markupsafe          2.1.1
mpl_toolkits        NA
msgpack             1.0.3
natsort             7.1.0
nbinom_ufunc        NA
numba               0.52.0
numcodecs           0.9.1
numexpr             2.7.2
packaging           21.3
parso               0.7.1
pexpect             4.8.0
pickleshare         0.7.5
pkg_resources       NA
plotly              5.10.0
prompt_toolkit      3.0.10
psutil              5.9.0
ptyprocess          0.7.0
pyarrow             9.0.0
pygments            2.13.0
pyparsing           2.4.7
pytz                2020.5
scipy               1.7.3
setuptools_scm      NA
sinfo               0.3.1
six                 1.14.0
sklearn             1.0.2
sphinxcontrib       NA
statsmodels         0.12.1
storemagic          NA
tables              3.6.1
tblib               1.7.0
texttable           1.6.3
threadpoolctl       2.1.0
tlz                 0.11.2
toolz               0.11.2
tornado             6.1
traitlets           5.0.5
typing_extensions   NA
wcwidth             0.2.5
yaml                5.4.1
zarr                2.11.3
zipp                NA
zmq                 21.0.1
zope                NA
```

Example 5 (unknown):
```unknown
-----
IPython             7.19.0
jupyter_client      6.1.11
jupyter_core        4.7.0
notebook            6.2.0
-----
Python 3.7.9 | packaged by conda-forge | (default, Dec  9 2020, 21:08:20) [GCC 9.3.0]
Linux-4.15.0-112-generic-x86_64-with-debian-bullseye-sid
-----
Session information updated at 2023-02-08 18:37
```

Example 6 (python):
```python
sc.settings.set_figure_params(dpi=120)
```

Example 7 (python):
```python
multiplexed_samples = pd.read_csv('/nfs/team205/heart/mapped/cellranger-arc200/csv/multiome-foetal_20221221.csv')
multiplexed_samples

# read in metadata
meta = pd.read_csv('/nfs/team205/heart/anndata_objects/Foetal/metadata/HeartTeamSamples_Mappeddata_20221223.csv', sep = ',', index_col = None)
meta = meta[meta['Publication']=='Foetal']
meta = meta[['sangerID', 'combinedID',
       'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender',
       'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed',
       'status']]

# select samples
meta = meta[meta['combinedID'].isin(multiplexed_samples['combinedID'])]

# select Multiome-ATAC
meta = meta[meta['modality']=='Multiome-ATAC']

# reindex
meta.index = range(len(meta))

pd.set_option('max_colwidth', None)
meta[['combinedID',
       'donor', 'region', 'age', 'gender']]
```

Example 8 (unknown):
```unknown
multiplexed_samples = ['7089STDY13216920_BHF_F_Hea13242527','7089STDY13216921_BHF_F_Hea13242528',
                       '7089STDY13216922_BHF_F_Hea13242529','7089STDY13216923_BHF_F_Hea13242530']
```

Example 9 (python):
```python
# post-cellbender
adata = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/QC/multiome_RNA_foetal_prefilter.h5ad')

# only multiplexed data (plus relevant single genotype samples)
adata = adata[adata.obs['combinedID'].isin(list(meta['combinedID']))]

adata
```

Example 10 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 194705 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval'
    var: 'gene_name', 'feature_type', 'ambient_expression-0', 'ambient_expression-1', 'ambient_expression-10', 'ambient_expression-11', 'ambient_expression-12', 'ambient_expression-13', 'ambient_expression-14', 'ambient_expression-15', 'ambient_expression-16', 'ambient_expression-17', 'ambient_expression-18', 'ambient_expression-19', 'ambient_expression-2', 'ambient_expression-20', 'ambient_expression-21', 'ambient_expression-22', 'ambient_expression-23', 'ambient_expression-24', 'ambient_expression-25', 'ambient_expression-26', 'ambient_expression-27', 'ambient_expression-28', 'ambient_expression-29', 'ambient_expression-3', 'ambient_expression-30', 'ambient_expression-31', 'ambient_expression-32', 'ambient_expression-33', 'ambient_expression-34', 'ambient_expression-35', 'ambient_expression-36', 'ambient_expression-37', 'ambient_expression-4', 'ambient_expression-5', 'ambient_expression-6', 'ambient_expression-7', 'ambient_expression-8', 'ambient_expression-9'
    obsm: 'latent_gene_encoding'
```

Example 11 (python):
```python
adata.obs['combinedID'].value_counts()
```

Example 12 (unknown):
```unknown
7089STDY13216924_BHF_F_Hea13242531    40000
7089STDY13216925_BHF_F_Hea13242532    30453
7089STDY13216920_BHF_F_Hea13242527    27498
7089STDY13216927_BHF_F_Hea13242534    24666
7089STDY13216926_BHF_F_Hea13242533    24602
7089STDY13216921_BHF_F_Hea13242528    24211
7089STDY13216922_BHF_F_Hea13242529    21402
7089STDY13216923_BHF_F_Hea13242530     1873
Name: combinedID, dtype: int64
```

Example 13 (python):
```python
adata.obs[adata.obs['combinedID'].isin(multiplexed_samples)]['combinedID'].value_counts()
```

Example 14 (unknown):
```unknown
7089STDY13216920_BHF_F_Hea13242527    27498
7089STDY13216921_BHF_F_Hea13242528    24211
7089STDY13216922_BHF_F_Hea13242529    21402
7089STDY13216923_BHF_F_Hea13242530     1873
7089STDY13216924_BHF_F_Hea13242531        0
7089STDY13216925_BHF_F_Hea13242532        0
7089STDY13216926_BHF_F_Hea13242533        0
7089STDY13216927_BHF_F_Hea13242534        0
Name: combinedID, dtype: int64
```

Example 15 (python):
```python
# read in
soc = pd.read_csv('/nfs/team205/heart/mapped/cellranger-arc200/SOC_out/SOC-clusters_multiome-foetal_20221221.csv',
                 index_col=0)
print(soc.shape)
soc.head()
```

Example 16 (unknown):
```unknown
(74984, 10)
```

Example 17 (python):
```python
# add
col_to_add = ['status', 'assignment', 'log_prob_singleton',
               'log_prob_doublet', 'cluster0', 'cluster1', 'cluster2',
               'donor_assignment']
adata.obs[[f'SOC | {x}' for x in col_to_add]] = soc[col_to_add].reindex(adata.obs_names)
adata.obs.head()
```

Example 18 (unknown):
```unknown
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/ipykernel_launcher.py:5: ImplicitModificationWarning: Trying to modify attribute `.obs` of view, initializing view as actual.
  """
```

Example 19 (python):
```python
adata.obs['SOC | donor_assignment'].value_counts()
```

Example 20 (unknown):
```unknown
Hst39               36323
Hst40               20246
donor_unassigned    13322
Hst36                5093
Name: SOC | donor_assignment, dtype: int64
```

Example 21 (python):
```python
adata.obs[adata.obs['SOC | donor_assignment']!='donor_unassigned']['SOC | donor_assignment'].value_counts()
```

Example 22 (unknown):
```unknown
Hst39    36323
Hst40    20246
Hst36     5093
Name: SOC | donor_assignment, dtype: int64
```

Example 23 (python):
```python
adata.obs['SOC | status'].value_counts()
```

Example 24 (javascript):
```javascript
singlet       63170
doublet        8383
unassigned     3431
Name: SOC | status, dtype: int64
```

Example 25 (python):
```python
# read in
cmo = pd.read_csv('/nfs/team205/heart/mapped/CMO/result_summary/CMO-result_multiome-foetal_20230207.csv',
                 index_col=0)
print(cmo.shape)
cmo.head()
```

Example 26 (python):
```python
# add
col_to_add = ['ratio','top_count', 'donor_assignment', 'n_donor_assigned','Hst40', 'Hst39', 'Hst36']
adata.obs[[f'CMO | {x}' for x in col_to_add]] = cmo[col_to_add].reindex(adata.obs_names)
adata.obs.head()
```

Example 27 (python):
```python
# nuclei number
df = adata.obs[adata.obs['combinedID'].isin(multiplexed_samples)]
pd.crosstab(df['SOC | donor_assignment'],df['CMO | donor_assignment'])
```

Example 28 (python):
```python
# nuclei number in each sample
df_cmo = adata.obs[adata.obs['combinedID'].isin(multiplexed_samples)]
for sam in multiplexed_samples:
    print(f'##### {sam} #####')
    df = df_cmo[df_cmo['combinedID']==sam]
    print(pd.crosstab(df['SOC | donor_assignment'],df['CMO | donor_assignment']))
    print('')
```

Example 29 (unknown):
```unknown
##### 7089STDY13216920_BHF_F_Hea13242527 #####
CMO | donor_assignment  Hst39  Hst40  Hst40,Hst39
SOC | donor_assignment                           
Hst39                    7777   8886            7
Hst40                     177   6716            1
donor_unassigned          603   3331            0

##### 7089STDY13216921_BHF_F_Hea13242528 #####
CMO | donor_assignment  Hst39  Hst40  Hst40,Hst39
SOC | donor_assignment                           
Hst39                    7675   7182           10
Hst40                     129   5815            0
donor_unassigned          531   2869            0

##### 7089STDY13216922_BHF_F_Hea13242529 #####
CMO | donor_assignment  Hst36  Hst39  Hst40  Hst40,Hst36  Hst40,Hst39
SOC | donor_assignment                                               
Hst36                    3559     73   1459            2            0
Hst39                     444   1692   2649            0            1
Hst40                     362     72   6974            0            0
donor_unassigned         1101    341   2673            0            0

##### 7089STDY13216923_BHF_F_Hea13242530 #####
CMO | donor_assignment  Hst36  Hst39  Hst40
SOC | donor_assignment                     
donor_unassigned          394     44   1435
```

Example 30 (python):
```python
%%time
adata_test = adata.copy()

# remove doublets
dbl_score_thresh=0.15
adata_test = adata_test[adata_test.obs['scrublet_score']<dbl_score_thresh]
# select regions
adata_test = adata_test[adata_test.obs['region'].isin([
    'Heart',
    'Heart (Hst36), Heart-wo-nodes (Hst39), Heart-wo-nodes (Hst40)',
    'Heart-wo-nodes'])]

# sc.pp.filter_cells(adata_test, min_genes=200)
sc.pp.filter_genes(adata_test, min_cells=3)

# QC metrics
adata_test.var['mt'] = adata_test.var_names.str.startswith('MT-')  # annotate the group of mitochondrial genes as 'mt'
sc.pp.calculate_qc_metrics(adata_test, qc_vars=['mt'], percent_top=None, log1p=True, inplace=True)
## adata_test = adata_test[adata_test.obs.pct_counts_mt < 5, :]

# log-normalise
sc.pp.normalize_total(adata_test, target_sum=1e4)
sc.pp.log1p(adata_test)

# HVGs
sc.pp.highly_variable_genes(adata_test, min_mean=0.0125, max_mean=3, min_disp=0.5)
print(f'n_hvgs: {sum(adata_test.var.highly_variable)}')
adata_test.raw = adata_test
adata_test = adata_test[:, adata_test.var.highly_variable]

# scale, pce, neighbors, umap
## sc.pp.regress_out(adata_test, ['total_counts', 'pct_counts_mt'])
sc.pp.scale(adata_test, max_value=10)
sc.tl.pca(adata_test, svd_solver='arpack')
sc.pp.neighbors(adata_test)
sc.tl.umap(adata_test, min_dist=0.1)
```

Example 31 (python):
```python
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/scanpy/preprocessing/_simple.py:251: ImplicitModificationWarning: Trying to modify attribute `.var` of view, initializing view as actual.
  adata.var['n_cells'] = number
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/scanpy/preprocessing/_normalization.py:182: UserWarning: Some cells have zero counts
  warn(UserWarning('Some cells have zero counts'))
```

Example 32 (unknown):
```unknown
n_hvgs: 5536
```

Example 33 (python):
```python
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/scanpy/preprocessing/_simple.py:843: UserWarning: Revieved a view of an AnnData. Making a copy.
  view_to_actual(adata)
```

Example 34 (unknown):
```unknown
CPU times: user 8min 4s, sys: 3min 11s, total: 11min 16s
Wall time: 2min 59s
```

Example 35 (python):
```python
fig, axs = plt.subplots(nrows=1, ncols=2, figsize=(16,4)) 
sc.pl.umap(adata_test, color = ['combinedID',], show=False, ax=axs[0], size=0.3)
sc.pl.umap(adata_test, color = ['combinedID',], groups=['7089STDY13216922_BHF_F_Hea13242529','7089STDY13216923_BHF_F_Hea13242530'], show=False, ax=axs[1])
plt.tight_layout()
```

Example 36 (python):
```python
adata_test.obs['multiplexed'] = 'No'
multi_obsnames = adata_test.obs_names[adata_test.obs['combinedID'].isin(multiplexed_samples)]
adata_test.obs.loc[multi_obsnames,'multiplexed'] = 'Yes'

fig, axs = plt.subplots(nrows=1, ncols=2, figsize=(9,4)) 
sc.pl.umap(adata_test, color = ['multiplexed'], groups='No', wspace=0.5, size=2, show=False, ax=axs[0])
sc.pl.umap(adata_test, color = ['multiplexed'], groups='Yes', wspace=0.5, size=2, show=False, ax=axs[1])
plt.tight_layout()
```

Example 37 (python):
```python
adata_test.uns['donor_colors'] = ['#1f77b4', '#d62728', '#ff7f0e', '#279e68']
sc.pl.umap(adata_test, color = ['donor','SOC | donor_assignment','CMO | donor_assignment',], wspace=0.6)
```

Example 38 (python):
```python
don = 'Hst36'
size = 3

fig, axs = plt.subplots(nrows=1, ncols=3, figsize=(14,4)) 
sc.pl.umap(adata_test, color = ['donor'], groups=don, size=size, show=False, ax=axs[0], title='donor, singleplex sample')
sc.pl.umap(adata_test, color = ['SOC | donor_assignment'], groups=don, size=size, show=False, ax=axs[1])
sc.pl.umap(adata_test, color = ['CMO | donor_assignment'], groups=don, size=size, show=False, ax=axs[2])
plt.tight_layout()
```

Example 39 (python):
```python
don = 'Hst39'
size = 3

fig, axs = plt.subplots(nrows=1, ncols=3, figsize=(14,4)) 
sc.pl.umap(adata_test, color = ['donor'], groups=don, size=size, show=False, ax=axs[0], title='donor, singleplex sample')
sc.pl.umap(adata_test, color = ['SOC | donor_assignment'], groups=don, size=size, show=False, ax=axs[1])
sc.pl.umap(adata_test, color = ['CMO | donor_assignment'], groups=don, size=size, show=False, ax=axs[2])
plt.tight_layout()
```

Example 40 (python):
```python
don = 'Hst40'
size = 3

fig, axs = plt.subplots(nrows=1, ncols=3, figsize=(14,4)) 
sc.pl.umap(adata_test, color = ['donor'], groups=don, size=size, show=False, ax=axs[0], title='donor, singleplex sample')
sc.pl.umap(adata_test, color = ['SOC | donor_assignment'], groups=don, size=size, show=False, ax=axs[1])
sc.pl.umap(adata_test, color = ['CMO | donor_assignment'], groups=don, size=size, show=False, ax=axs[2])
plt.tight_layout()
```

Example 41 (python):
```python
sc.pl.umap(adata_test, color = ['CMO | prop_delta',], wspace=0.5)
```

Example 42 (python):
```python
# multiplexed samples apart from '7089STDY13216923_BHF_F_Hea13242530'(no SOC assignment)
df = adata.obs[adata.obs['combinedID'].isin(['7089STDY13216920_BHF_F_Hea13242527',
                                             '7089STDY13216921_BHF_F_Hea13242528',
                                             '7089STDY13216922_BHF_F_Hea13242529',])]

pd.crosstab(df['SOC | donor_assignment'],df['CMO | donor_assignment'])
```

Example 43 (python):
```python
# new labels for the comparison
matched_obsnames = df.index[df['SOC | donor_assignment']==df['CMO | donor_assignment']]
unmatched_obsnames =  df.index[(df['SOC | donor_assignment']!=df['CMO | donor_assignment'])&(df['SOC | donor_assignment']!='donor_unassigned')]
unassigned_obsnames = df.index[df['SOC | status']=='unassigned']
doublet_obsnames = df.index[df['SOC | status']=='doublet']

df['match'] = np.nan
df.loc[matched_obsnames,'match'] = 'SOC-COM, matched'
df.loc[unmatched_obsnames,'match'] = 'SOC-COM, unmatched'
df.loc[unassigned_obsnames,'match'] = 'SOC status, unassigned'
df.loc[doublet_obsnames,'match'] = 'SOC status, doublet'
set(df['match'])
```

Example 44 (python):
```python
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/ipykernel_launcher.py:7: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  import sys
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/pandas/core/indexing.py:1817: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  self._setitem_single_column(loc, value, pi)
```

Example 45 (unknown):
```unknown
{'SOC status, doublet',
 'SOC status, unassigned',
 'SOC-COM, matched',
 'SOC-COM, unmatched'}
```

Example 46 (python):
```python
sns.violinplot(data=df, x="CMO | ratio", y="match")
plt.xlim(0.5, 4)
plt.show()

df['match'].value_counts()
```

Example 47 (javascript):
```javascript
SOC-COM, matched          40208
SOC-COM, unmatched        21454
SOC status, doublet        8103
SOC status, unassigned     3346
Name: match, dtype: int64
```

Example 48 (python):
```python
from sklearn.metrics import roc_curve
```

Example 49 (python):
```python
df_s = df[df['match'].isin(['SOC-COM, matched','SOC-COM, unmatched'])]
X = np.array(df_s['CMO | ratio']) # .reshape(-1, 1)
y = np.array(df_s['match'] == 'SOC-COM, matched')
```

Example 50 (unknown):
```unknown
fpr,tpr,thresholds = roc_curve(y, X)
fpr,tpr,thresholds
```

Example 51 (unknown):
```unknown
(array([0.        , 0.        , 0.        , ..., 0.99888133, 0.99902116,
        1.        ]),
 array([0.00000000e+00, 2.48706725e-05, 1.09430959e-03, ...,
        1.00000000e+00, 1.00000000e+00, 1.00000000e+00]),
 array([15.5164995, 14.5164995,  8.282574 , ...,  1.0002261,  1.0001545,
         1.       ]))
```

Example 52 (python):
```python
optimal_idx = np.argmax(tpr - fpr)
print(f'optimal threshold to make the "tpr - fpr" maximum: {thresholds[optimal_idx]}')
print(f'true positive rate with the threshold: {tpr[optimal_idx]}')
print(f'false positive rate with the threshold: {fpr[optimal_idx]}')
```

Example 53 (unknown):
```unknown
optimal threshold to make the "tpr - fpr" maximum: 1.4452686
true positive rate with the threshold: 0.5862763629128531
false positive rate with the threshold: 0.2780367297473665
```

Example 54 (python):
```python
sns.violinplot(data=df_s, x="CMO | ratio", y="match")
plt.axvline(thresholds[optimal_idx], color='red', linestyle='dashed', linewidth=1)
plt.xlim(0.5, 4)
plt.show()
# red dashed line is the threshold which gives maximum "tpr - fpr" value
```

Example 55 (python):
```python
optimal_idx = sum(fpr < 0.05)-1
print(f'optimal threshold to make the fpr lower than 0.05: {thresholds[optimal_idx]}')
print(f'true positive rate with the threshold: {tpr[optimal_idx]}')
print(f'false positive rate with the threshold: {fpr[optimal_idx]}')
```

Example 56 (unknown):
```unknown
optimal threshold to make the fpr lower than 0.05: 2.4282365
true positive rate with the threshold: 0.23104854755272583
false positive rate with the threshold: 0.049967372051831827
```

Example 57 (python):
```python
sns.violinplot(data=df_s, x="CMO | ratio", y="match")
plt.axvline(thresholds[optimal_idx], color='red', linestyle='dashed', linewidth=1)
plt.xlim(0.5, 4)
plt.show()
# red dashed line is the threshold which gives maximum "tpr - fpr" value
```

Example 58 (unknown):
```unknown
df = df['combinedID']==''
```

Example 59 (python):
```python
# multiplexed samples apart from '7089STDY13216923_BHF_F_Hea13242530'(no SOC assignment)
df = adata.obs[adata.obs['combinedID'].isin(['7089STDY13216920_BHF_F_Hea13242527',
                                             '7089STDY13216921_BHF_F_Hea13242528',
                                             '7089STDY13216922_BHF_F_Hea13242529',])]
# select only SOC | donor-assigned ones
df = df[df['SOC | donor_assignment'] != 'donor_unassigned']
print(df.shape)

donors = ['Hst36','Hst39','Hst40']
samples = ['7089STDY13216920_BHF_F_Hea13242527',
           '7089STDY13216921_BHF_F_Hea13242528',
           '7089STDY13216922_BHF_F_Hea13242529']
# set order
df['SOC | donor_assignment'] = df['SOC | donor_assignment'].astype('category')
df['SOC | donor_assignment'].cat.reorder_categories(donors, inplace=True)

fig,axs = plt.subplots(nrows=3, ncols=3, figsize=(17,10)) 
for sam in samples:
    df_s = df[df['combinedID']==sam]
    
    for don in donors:
        row_no = samples.index(sam)
        col_no = donors.index(don)
        
        sns.violinplot(data=df_s, 
                       x=f'CMO | {don}',
                       y=f'SOC | donor_assignment',
                       ax=axs[row_no,col_no])
        axs[row_no,col_no].set_xlabel(f'{don} tag count')
        axs[row_no,col_no].set_title(sam)
        axs[row_no,col_no].set_xlim(-1000, 30000)

plt.tight_layout(h_pad=3.0)
```

Example 60 (unknown):
```unknown
(61662, 35)
```

Example 61 (python):
```python
# multiplexed samples apart from '7089STDY13216923_BHF_F_Hea13242530'(no SOC assignment)
df = adata.obs[adata.obs['combinedID'].isin(['7089STDY13216920_BHF_F_Hea13242527',
                                             '7089STDY13216921_BHF_F_Hea13242528',
                                             '7089STDY13216922_BHF_F_Hea13242529',])]
# select only SOC | donor-assigned ones
df = df[df['SOC | donor_assignment'] != 'donor_unassigned']

# remove double-assigned nuclei
# df = df[[',' not in x for x in df['CMO | donor_assignment']]]
# print(df.shape)

donors = ['Hst36','Hst39','Hst40']
samples = ['7089STDY13216920_BHF_F_Hea13242527',
           '7089STDY13216921_BHF_F_Hea13242528',
           '7089STDY13216922_BHF_F_Hea13242529']

# set color
df['color'] = 'grey'
df.loc[df.index[df['SOC | donor_assignment']=='Hst39'],'color'] = 'blue'
df.loc[df.index[df['SOC | donor_assignment']=='Hst40'],'color'] = 'red'

# set order
# df['CMO | donor_assignment'] = df['CMO | donor_assignment'].astype('category')
# df['CMO | donor_assignment'].cat.reorder_categories(donors, inplace=True)

fig,axs = plt.subplots(nrows=1, ncols=3, figsize=(20,6)) 

for sam in samples:
    # subset each sample
    df_s = df[df['combinedID']==sam]
    
    for don in donors:
        row_no = samples.index(sam)
        # col_no = donors.index(don)
        
        # subset each donor (assume SOC is giving the correct donor)
        # df_ss = df_s[df_s['SOC | donor_assignment']==don]
        
        axs[row_no].scatter( 
                       x=np.log10(df_s['CMO | Hst39']),
                       y=np.log10(df_s['CMO | Hst40']),
                       # x=df_s['CMO | Hst39'],
                       # y=df_s['CMO | Hst40'],
                       c=df_s['color'],
                        s=2
                       )
        axs[row_no].set_xlabel('CMO | Hst39 count')
        axs[row_no].set_ylabel('CMO | Hst40 count')
        axs[row_no].set_title(f'{sam}, SOC|{don}')
        # axs[row_no,col_no].set_xlim(-1000, 50000)

plt.tight_layout(w_pad=5.0, h_pad=3.0)
```

Example 62 (python):
```python
np.mean(df['CMO | Hst39'])
```

Example 63 (unknown):
```unknown
3688.686516817489
```

Example 64 (python):
```python
# multiplexed samples apart from '7089STDY13216923_BHF_F_Hea13242530'(no SOC assignment)
df = adata.obs[adata.obs['combinedID'].isin(['7089STDY13216920_BHF_F_Hea13242527',
                                             '7089STDY13216921_BHF_F_Hea13242528',
                                             '7089STDY13216922_BHF_F_Hea13242529',])]
# select only SOC | donor-assigned ones
df = df[df['SOC | donor_assignment'] != 'donor_unassigned']

# remove double-assigned nuclei
df = df[[',' not in x for x in df['CMO | donor_assignment']]]
print(df.shape)

donors = ['Hst36','Hst39','Hst40']
samples = ['7089STDY13216920_BHF_F_Hea13242527',
           '7089STDY13216921_BHF_F_Hea13242528',
           '7089STDY13216922_BHF_F_Hea13242529']

# set order
df['CMO | donor_assignment'] = df['CMO | donor_assignment'].astype('category')
df['CMO | donor_assignment'].cat.reorder_categories(donors, inplace=True)

fig,axs = plt.subplots(nrows=3, ncols=3, figsize=(20,17)) 

for sam in samples:
    # subset each sample
    df_s = df[df['combinedID']==sam]
    
    for don in donors:
        row_no = samples.index(sam)
        col_no = donors.index(don)
        
        # subset each donor (assume SOC is giving the correct donor)
        df_ss = df_s[df_s['SOC | donor_assignment']==don]
        
        if len(df_ss)==0:
            continue
        
        axs[row_no,col_no].scatter(data=df_ss, 
                       x='CMO | Hst39',
                       y='CMO | Hst40',
                       )
        axs[row_no,col_no].set_xlabel('CMO | Hst39 count')
        axs[row_no,col_no].set_ylabel('CMO | Hst40 count')
        axs[row_no,col_no].set_title(f'{sam}, SOC|{don}')
        axs[row_no,col_no].set_xlim(-1000, 50000)

plt.tight_layout(w_pad=5.0, h_pad=3.0)
```

Example 65 (unknown):
```unknown
(61641, 37)
```

Example 66 (python):
```python
samplefile = pd.read_csv('/nfs/team205/heart/mapped/CMO/csv/multiome-foetal_20221221.csv')
samplefile.set_index('CMO_ID',inplace=True)
samplefile
```

Example 67 (python):
```python
cmo_ID = '7089STDY13216916'
cmo_raw = sc.read_10x_mtx(f'/nfs/team205/heart/mapped/CMO/{cmo_ID}/raw_feature_bc_matrix',
                     gex_only=False)
cmo_raw.var_names = [x.split('-')[0] for x in cmo_raw.var_names]


# get multiome ID
multi_ID = samplefile.loc[cmo_ID,'Multiome_ID']

# modify obs_names
cmo_raw.obs_names = multi_ID + '_' + cmo_raw.obs_names

# label
cmo_raw.obs['label'] = 'not-cell'
shared = set(adata.obs_names).intersection(cmo_raw.obs_names)
cmo_raw.obs.loc[shared,'label'] = adata.obs.loc[shared, 'SOC | donor_assignment']

# color
cmo_raw.obs['color'] = cmo_raw.obs['label'].copy()
cmo_raw.obs.replace({'color':{
    'Hst36':'green',
    'Hst39':'blue',
    'Hst40':'red',
    'donor_unassigned':'grey',
    'not-cell':'grey'
}},inplace=True)

tag = 'Hst40'
df = cmo_raw.to_df()[[tag]]
df.sort_values(tag,ascending=False,inplace=True)
df['rank'] = np.array(range(len(df)))+1

plt.rcParams['figure.figsize'] = 6,4
plt.scatter(x=df['rank'], y=np.log10(df[tag]), c=cmo_raw.obs['color'].reindex(df.index), s=5)
# plt.scatter(x=df['rank'], y=df[tag], c=cmo_raw.obs['color'].reindex(df.index), s=5)
plt.xscale('log')
plt.xlabel('Droplet ID ranked by count')
plt.ylabel(f'CMO count per droplet (log10)')
plt.title(tag)
plt.show()
plt.clf()
```

Example 68 (unknown):
```unknown
<Figure size 432x288 with 0 Axes>
```

Example 69 (python):
```python
cmo_ID = '7089STDY13216916'
cmo_raw = sc.read_10x_mtx(f'/nfs/team205/heart/mapped/CMO/{cmo_ID}/raw_feature_bc_matrix',
                     gex_only=False)
cmo_raw.var_names = [x.split('-')[0] for x in cmo_raw.var_names]


# get multiome ID
multi_ID = samplefile.loc[cmo_ID,'Multiome_ID']

# modify obs_names
cmo_raw.obs_names = multi_ID + '_' + cmo_raw.obs_names

# label
cmo_raw.obs['label'] = 'not-cell'
shared = set(adata.obs_names).intersection(cmo_raw.obs_names)
cmo_raw.obs.loc[shared,'label'] = adata.obs.loc[shared, 'SOC | donor_assignment']

# color
cmo_raw.obs['color'] = cmo_raw.obs['label'].copy()
cmo_raw.obs.replace({'color':{
    'Hst36':'green',
    'Hst39':'blue',
    'Hst40':'red',
    'donor_unassigned':'grey',
    'not-cell':'grey'
}},inplace=True)

tags = ['Hst39 (tag6)', 'Hst40 (tag4)']

fig,axs = plt.subplots(nrows=1, ncols=2, figsize=(12,5))
fig.suptitle(multi_ID)
for i,tg in enumerate(tags):
    don = tg.split(' (')[0]
    df = cmo_raw.to_df()[[don]]
    df.sort_values(don,ascending=False,inplace=True)
    df['rank'] = np.array(range(len(df)))+1

    axs[i].scatter(x=df['rank'], y=np.log10(df[don]), c=cmo_raw.obs['color'].reindex(df.index), s=10)
    # axs[i].scatter(x=df['rank'], y=df[don], c=cmo_raw.obs['color'].reindex(df.index), s=1)
    axs[i].set_xscale('log')
    axs[i].set_xlabel('Droplets ranked by count')
    axs[i].set_ylabel(f'CMO count per droplet (log10)')
    axs[i].set_title(tg)

plt.tight_layout(w_pad=3.0)
```

Example 70 (unknown):
```unknown
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/pandas/core/arraylike.py:364: RuntimeWarning: divide by zero encountered in log10
  result = getattr(ufunc, method)(*inputs, **kwargs)
```

Example 71 (unknown):
```unknown
cmo_raw.to_df()
```

Example 72 (python):
```python
cmo_ID = '7089STDY13216917'
cmo_raw = sc.read_10x_mtx(f'/nfs/team205/heart/mapped/CMO/{cmo_ID}/raw_feature_bc_matrix',
                     gex_only=False)
cmo_raw.var_names = [x.split('-')[0] for x in cmo_raw.var_names]


# get multiome ID
multi_ID = samplefile.loc[cmo_ID,'Multiome_ID']

# modify obs_names
cmo_raw.obs_names = multi_ID + '_' + cmo_raw.obs_names

# label
cmo_raw.obs['label'] = 'not-cell'
shared = set(adata.obs_names).intersection(cmo_raw.obs_names)
cmo_raw.obs.loc[shared,'label'] = adata.obs.loc[shared, 'SOC | donor_assignment']

# color
cmo_raw.obs['color'] = cmo_raw.obs['label'].copy()
cmo_raw.obs.replace({'color':{
    'Hst36':'green',
    'Hst39':'blue',
    'Hst40':'red',
    'donor_unassigned':'white',
    'not-cell':'grey'
}},inplace=True)

tags = ['Hst39 (tag6)', 'Hst40 (tag4)']

fig,axs = plt.subplots(nrows=1, ncols=2, figsize=(12,5))
fig.suptitle(multi_ID)
for i,tg in enumerate(tags):
    don = tg.split(' (')[0]
    df = cmo_raw.to_df()[[don]]
    df.sort_values(don,ascending=False,inplace=True)
    df['rank'] = np.array(range(len(df)))+1

    axs[i].scatter(x=df['rank'], y=np.log10(df[don]), c=cmo_raw.obs['color'].reindex(df.index), s=10)
    # axs[i].scatter(x=df['rank'], y=df[don], c=cmo_raw.obs['color'].reindex(df.index), s=1)
    axs[i].set_xscale('log')
    axs[i].set_xlabel('Droplets ranked by count')
    axs[i].set_ylabel(f'CMO count per droplet (log10)')
    axs[i].set_title(tg)

plt.tight_layout(w_pad=3.0)
```

Example 73 (python):
```python
cmo_ID = '7089STDY13216918'
cmo_raw = sc.read_10x_mtx(f'/nfs/team205/heart/mapped/CMO/{cmo_ID}/raw_feature_bc_matrix',
                     gex_only=False)
cmo_raw.var_names = [x.split('-')[0] for x in cmo_raw.var_names]


# get multiome ID
multi_ID = samplefile.loc[cmo_ID,'Multiome_ID']

# modify obs_names
cmo_raw.obs_names = multi_ID + '_' + cmo_raw.obs_names

# label
cmo_raw.obs['label'] = 'not-cell'
shared = set(adata.obs_names).intersection(cmo_raw.obs_names)
cmo_raw.obs.loc[shared,'label'] = adata.obs.loc[shared, 'SOC | donor_assignment']

# color
cmo_raw.obs['color'] = cmo_raw.obs['label'].copy()
cmo_raw.obs.replace({'color':{
    'Hst36':'green',
    'Hst39':'blue',
    'Hst40':'red',
    'donor_unassigned':'grey',
    'not-cell':'grey'
}},inplace=True)

tags = ['Hst36 (tag9)', 'Hst39 (tag8)', 'Hst40 (tag5)']

fig,axs = plt.subplots(nrows=1, ncols=3, figsize=(17,5))
fig.suptitle(multi_ID)
for i,tg in enumerate(tags):
    don = tg.split(' (')[0]
    df = cmo_raw.to_df()[[don]]
    df.sort_values(don,ascending=False,inplace=True)
    df['rank'] = np.array(range(len(df)))+1

    axs[i].scatter(x=df['rank'], y=np.log10(df[don]), c=cmo_raw.obs['color'].reindex(df.index), s=10)
    # axs[i].scatter(x=df['rank'], y=df[don], c=cmo_raw.obs['color'].reindex(df.index), s=1)
    axs[i].set_xscale('log')
    axs[i].set_xlabel('Droplets ranked by count')
    axs[i].set_ylabel(f'CMO count per droplet (log10)')
    axs[i].set_title(tg)

plt.tight_layout(w_pad=3.0)
```

Example 74 (python):
```python
# multiplexed samples apart from '7089STDY13216923_BHF_F_Hea13242530'(no SOC assignment)
df = adata.obs[adata.obs['combinedID'].isin(['7089STDY13216920_BHF_F_Hea13242527',
                                             '7089STDY13216921_BHF_F_Hea13242528',
                                             '7089STDY13216922_BHF_F_Hea13242529',])]
# select only SOC | donor-assigned ones
df = df[df['SOC | donor_assignment'] != 'donor_unassigned']

# remove double-assigned nuclei
df = df[[',' not in x for x in df['CMO | donor_assignment']]]
print(df.shape)

donors = ['Hst36','Hst39','Hst40']
samples = ['7089STDY13216920_BHF_F_Hea13242527',
           '7089STDY13216921_BHF_F_Hea13242528',
           '7089STDY13216922_BHF_F_Hea13242529']

# set order
df['CMO | donor_assignment'] = df['CMO | donor_assignment'].astype('category')
df['CMO | donor_assignment'].cat.reorder_categories(donors, inplace=True)

fig,axs = plt.subplots(nrows=3, ncols=3, figsize=(20,10)) 

for sam in samples:
    # subset each sample
    df_s = df[df['combinedID']==sam]
    
    for don in donors:
        row_no = samples.index(sam)
        col_no = donors.index(don)
        
        # subset each donor (assume SOC is giving the correct donor)
        df_ss = df_s[df_s['SOC | donor_assignment']==don]
        
        if len(df_ss)==0:
            continue
        
        sns.violinplot(data=df_ss, 
                       x=f'CMO | top_count',
                       y=f'CMO | donor_assignment',
                       ax=axs[row_no,col_no])
        axs[row_no,col_no].set_xlabel(f'top_count')
        axs[row_no,col_no].set_title(f'{sam}, SOC|{don}')
        axs[row_no,col_no].set_xlim(-1000, 50000)

plt.tight_layout(w_pad=5.0, h_pad=3.0)
```

Example 75 (unknown):
```unknown
(61641, 37)
```

Example 76 (python):
```python
# multiplexed samples apart from '7089STDY13216923_BHF_F_Hea13242530'(no SOC assignment)
df = adata.obs[adata.obs['combinedID'].isin(['7089STDY13216920_BHF_F_Hea13242527',
                                             '7089STDY13216921_BHF_F_Hea13242528',
                                             '7089STDY13216922_BHF_F_Hea13242529',])]
# select only SOC | donor-assigned ones
df = df[df['SOC | donor_assignment'] != 'donor_unassigned']

# remove double-assigned nuclei
df = df[[',' not in x for x in df['CMO | donor_assignment']]]
print(df.shape)

donors = ['Hst36','Hst39','Hst40']
samples = ['7089STDY13216920_BHF_F_Hea13242527',
           '7089STDY13216921_BHF_F_Hea13242528',
           '7089STDY13216922_BHF_F_Hea13242529']

# set order
df['CMO | donor_assignment'] = df['CMO | donor_assignment'].astype('category')
df['CMO | donor_assignment'].cat.reorder_categories(donors, inplace=True)

fig,axs = plt.subplots(nrows=3, ncols=3, figsize=(20,10)) 

for sam in samples:
    # subset each sample
    df_s = df[df['combinedID']==sam]
    
    for don in donors:
        row_no = samples.index(sam)
        col_no = donors.index(don)
        
        # subset each donor (assume SOC is giving the correct donor)
        df_ss = df_s[df_s['SOC | donor_assignment']==don]
        
        if len(df_ss)==0:
            continue
        
        sns.violinplot(data=df_ss, 
                       x=f'CMO | delta',
                       y=f'CMO | donor_assignment',
                       ax=axs[row_no,col_no])
        axs[row_no,col_no].set_xlabel(f'delta')
        axs[row_no,col_no].set_title(f'{sam}, SOC|{don}')
        axs[row_no,col_no].set_xlim(-5000, 30000)

plt.tight_layout(w_pad=5.0, h_pad=3.0)
```

Example 77 (unknown):
```unknown
(61641, 36)
```

Example 78 (python):
```python
np.mean(df['CMO | Hst39'])
```

Example 79 (unknown):
```unknown
3688.8545935335246
```

Example 80 (python):
```python
# multiplexed samples apart from '7089STDY13216923_BHF_F_Hea13242530'(no SOC assignment)
df = adata.obs[adata.obs['combinedID'].isin(['7089STDY13216920_BHF_F_Hea13242527',
                                             '7089STDY13216921_BHF_F_Hea13242528',
                                             '7089STDY13216922_BHF_F_Hea13242529',])]
# select only SOC | donor-assigned ones
df = df[df['SOC | donor_assignment'] != 'donor_unassigned']

# remove double-assigned nuclei
df = df[[',' not in x for x in df['CMO | donor_assignment']]]
print(df.shape)

donors = ['Hst36','Hst39','Hst40']
samples = ['7089STDY13216920_BHF_F_Hea13242527',
           '7089STDY13216921_BHF_F_Hea13242528',
           '7089STDY13216922_BHF_F_Hea13242529']

# set order
df['CMO | donor_assignment'] = df['CMO | donor_assignment'].astype('category')
df['CMO | donor_assignment'].cat.reorder_categories(donors, inplace=True)

fig,axs = plt.subplots(nrows=3, ncols=3, figsize=(20,10)) 

for sam in samples:
    # subset each sample
    df_s = df[df['combinedID']==sam]
    
    for don in donors:
        row_no = samples.index(sam)
        col_no = donors.index(don)
        
        # subset each donor (assume SOC is giving the correct donor)
        df_ss = df_s[df_s['SOC | donor_assignment']==don]
        
        if len(df_ss)==0:
            continue
        
        sns.violinplot(data=df_ss, 
                       x=f'CMO | prop_delta',
                       y=f'CMO | donor_assignment',
                       ax=axs[row_no,col_no])
        axs[row_no,col_no].set_xlabel(f'prop_delta')
        axs[row_no,col_no].set_title(f'{sam}, SOC|{don}')

plt.tight_layout(w_pad=5.0, h_pad=3.0)
```

Example 81 (unknown):
```unknown
(61641, 35)
```

Example 82 (python):
```python
# multiplexed samples apart from '7089STDY13216923_BHF_F_Hea13242530'(no SOC assignment)
df = adata.obs[adata.obs['combinedID'].isin(['7089STDY13216920_BHF_F_Hea13242527',
                                             '7089STDY13216921_BHF_F_Hea13242528',
                                             '7089STDY13216922_BHF_F_Hea13242529',])]
# select only SOC | donor-assigned ones
df = df[df['SOC | donor_assignment'] != 'donor_unassigned']

# remove double-assigned nuclei
df = df[[',' not in x for x in df['CMO | donor_assignment']]]
print(df.shape)

donors = ['Hst36','Hst39','Hst40']
samples = ['7089STDY13216920_BHF_F_Hea13242527',
           '7089STDY13216921_BHF_F_Hea13242528',
           '7089STDY13216922_BHF_F_Hea13242529']

# set order
df['CMO | donor_assignment'] = df['CMO | donor_assignment'].astype('category')
df['CMO | donor_assignment'].cat.reorder_categories(donors, inplace=True)

fig,axs = plt.subplots(nrows=3, ncols=3, figsize=(20,10)) 

for sam in samples:
    # subset each sample
    df_s = df[df['combinedID']==sam]
    
    for don in donors:
        row_no = samples.index(sam)
        col_no = donors.index(don)
        
        # subset each donor (assume SOC is giving the correct donor)
        df_ss = df_s[df_s['SOC | donor_assignment']==don]
        
        if len(df_ss)==0:
            continue
        
        sns.violinplot(data=df_ss, 
                       x=f'CMO | prop_delta',
                       y=f'CMO | donor_assignment',
                       ax=axs[row_no,col_no])
        axs[row_no,col_no].set_xlabel(f'prop_delta')
        axs[row_no,col_no].set_title(f'{sam}, SOC|{don}')

plt.tight_layout(w_pad=5.0, h_pad=3.0)
```

Example 83 (python):
```python
# multiplexed samples apart from '7089STDY13216923_BHF_F_Hea13242530'(no SOC assignment)
df = adata.obs[adata.obs['combinedID'].isin(['7089STDY13216920_BHF_F_Hea13242527',
                                             '7089STDY13216921_BHF_F_Hea13242528',
                                             '7089STDY13216922_BHF_F_Hea13242529',])]
# select only SOC | donor-assigned ones
df = df[df['SOC | donor_assignment'] != 'donor_unassigned']
print(df.shape)

donors = ['Hst36','Hst39','Hst40']
samples = ['7089STDY13216920_BHF_F_Hea13242527',
           '7089STDY13216921_BHF_F_Hea13242528',
           '7089STDY13216922_BHF_F_Hea13242529']

# label correct one and uncorrect



df['match'] = 

fig,axs = plt.subplots(nrows=3, ncols=3, figsize=(17,10)) 
for sam in samples:
    # subset each sample
    df_s = df[df['combinedID']==sam]
    
    # label correct one and uncorrect
    df
    
    for don in donors:
        row_no = samples.index(sam)
        col_no = donors.index(don)
        
        sns.violinplot(data=df_s, 
                       x=f'CMO | {don}',
                       y=f'SOC | donor_assignment',
                       ax=axs[row_no,col_no])
        axs[row_no,col_no].set_xlabel(f'{don} tag count')
        axs[row_no,col_no].set_title(sam)
        axs[row_no,col_no].set_xlim(-1000, 30000)

plt.tight_layout(h_pad=3.0)
```

Example 84 (python):
```python
# multiplexed samples apart from '7089STDY13216923_BHF_F_Hea13242530'(no SOC assignment)
df = adata.obs[adata.obs['combinedID'].isin(['7089STDY13216920_BHF_F_Hea13242527',
                                             '7089STDY13216921_BHF_F_Hea13242528',
                                             '7089STDY13216922_BHF_F_Hea13242529',])]
print(df.shape)

donors = ['Hst36','Hst39','Hst40']
samples = ['7089STDY13216920_BHF_F_Hea13242527',
           '7089STDY13216921_BHF_F_Hea13242528',
           '7089STDY13216922_BHF_F_Hea13242529']

fig,axs = plt.subplots(nrows=3, ncols=3, figsize=(20,10)) 

for sam in samples:
    df_s = df[df['combinedID']==sam]
    
    for don in donors:
        row_no = samples.index(sam)
        col_no = donors.index(don)
        
        sns.violinplot(data=df_s, 
                       x=f'CMO | {don}',
                       y=f'CMO | donor_assignment',
                       ax=axs[row_no,col_no])
        axs[row_no,col_no].set_xlabel(f'{don} tag count')
        axs[row_no,col_no].set_title(sam)
        axs[row_no,col_no].set_xlim(-1000, 30000)

plt.tight_layout(h_pad=3.0)
```

Example 85 (unknown):
```unknown
(73111, 35)
```

Example 86 (python):
```python
# multiplexed samples apart from '7089STDY13216923_BHF_F_Hea13242530'(no SOC assignment)
df = adata.obs[adata.obs['combinedID'].isin(['7089STDY13216920_BHF_F_Hea13242527',
                                             '7089STDY13216921_BHF_F_Hea13242528',
                                             '7089STDY13216922_BHF_F_Hea13242529',])]
print(df.shape)

donors = ['Hst36','Hst39','Hst40']
samples = ['7089STDY13216920_BHF_F_Hea13242527',
           '7089STDY13216921_BHF_F_Hea13242528',
           '7089STDY13216922_BHF_F_Hea13242529']

for sam in samples:
    df_s = df[df['combinedID']==sam]
        
    sns.violinplot(data=df_s, 
                   x=f'CMO | prop_delta',
                   y=f'SOC | donor_assignment')
    plt.xlabel('prop_delta')
    plt.title(sam)
    plt.show()
```

Example 87 (unknown):
```unknown
(73111, 35)
```

Example 88 (unknown):
```unknown
samples = ['7089STDY13216920_BHF_F_Hea13242527',
           '7089STDY13216921_BHF_F_Hea13242528',
           '7089STDY13216922_BHF_F_Hea13242529']
samples.index('7089STDY13216921_BHF_F_Hea13242528')
```

Example 89 (unknown):
```unknown
df[df['combinedID']=='7089STDY13216920_BHF_F_Hea13242527']
```

Example 90 (unknown):
```unknown
df['combinedID']
```

Example 91 (unknown):
```unknown
{'7089STDY13216920_BHF_F_Hea13242527',
 '7089STDY13216921_BHF_F_Hea13242528',
 '7089STDY13216922_BHF_F_Hea13242529',
 '7089STDY13216923_BHF_F_Hea13242530'}
```

Example 92 (python):
```python
df = adata.obs[adata.obs['combinedID'].isin(multiplexed_samples)]
df['combinedID'] = df['combinedID'].astype('str')
```

Example 93 (python):
```python
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/ipykernel_launcher.py:2: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
```

Example 94 (python):
```python
donor = 'Hst36'
sns.violinplot(data=df, x=f'CMO | {donor}', y="combinedID", vmax=5000)
plt.title(f'{donor} tag count')
plt.show()
```

Example 95 (python):
```python
donor = 'Hst36'
sns.violinplot(data=df, x=f'CMO | {donor}', y="combinedID", vmax=5000)
plt.xlim(-1000, 30000)
plt.title(f'{donor} tag count')
plt.show()

donor = 'Hst39'
sns.violinplot(data=df, x=f'CMO | {donor}', y="combinedID", vmax=5000)
plt.xlim(-1000, 30000)
plt.title(f'{donor} tag count')
plt.show()

donor = 'Hst40'
sns.violinplot(data=df, x=f'CMO | {donor}', y="combinedID", vmax=5000)
plt.xlim(-1000, 30000)
plt.title(f'{donor} tag count')
plt.show()
```

Example 96 (unknown):
```unknown
multiplexed_samples
```

Example 97 (unknown):
```unknown
['7089STDY13216920_BHF_F_Hea13242527',
 '7089STDY13216921_BHF_F_Hea13242528',
 '7089STDY13216922_BHF_F_Hea13242529',
 '7089STDY13216923_BHF_F_Hea13242530']
```

Example 98 (python):
```python
pd.crosstab(df['SOC | donor_assignment'],df['CMO | donor_assignment'])
```

Example 99 (python):
```python
SOC_39_CMO_39 = df.index[(df['SOC | donor_assignment']=='Hst39')&(df['CMO | donor_assignment']=='Hst39')]
SOC_39_CMO_40 = df.index[(df['SOC | donor_assignment']=='Hst39')&(df['CMO | donor_assignment']=='Hst40')]
SOC_40_CMO_40 = df.index[(df['SOC | donor_assignment']=='Hst40')&(df['CMO | donor_assignment']=='Hst40')]
SOC_40_CMO_39 = df.index[(df['SOC | donor_assignment']=='Hst40')&(df['CMO | donor_assignment']=='Hst39')]

df['match_39-40'] = np.nan
df.loc[SOC_39_CMO_39,'match_39-40'] = 'SOC-39_CMO-39'
df.loc[SOC_39_CMO_40,'match_39-40'] = 'SOC-39_CMO-40'
df.loc[SOC_40_CMO_40,'match_39-40'] = 'SOC-40_CMO-40'
df.loc[SOC_40_CMO_39,'match_39-40'] = 'SOC-40_CMO-39'
```

Example 100 (python):
```python
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/ipykernel_launcher.py:6: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/pandas/core/indexing.py:1817: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  self._setitem_single_column(loc, value, pi)
```

Example 101 (unknown):
```unknown
df['match_39-40'] = df['match_39-40'].astype('category')
df['match_39-40'].cat.reorder_categories([
    'SOC-39_CMO-39',
    'SOC-39_CMO-40',
    'SOC-40_CMO-40',
    'SOC-40_CMO-39'
], inplace=True)
```

Example 102 (python):
```python
/home/jovyan/my-conda-envs/cellpymc/lib/python3.7/site-packages/ipykernel_launcher.py:1: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  """Entry point for launching an IPython kernel.
```

Example 103 (python):
```python
for sam in ['7089STDY13216920_BHF_F_Hea13242527',
 '7089STDY13216921_BHF_F_Hea13242528',
 '7089STDY13216922_BHF_F_Hea13242529']:
    df_s = df[df['combinedID']==sam]
    sns.violinplot(data=df_s, x='CMO | Hst39', y="match_39-40", vmax=5000)
    plt.xlim(-2000, 30000)
    plt.title(sam)
    plt.show()
```

Example 104 (python):
```python
for sam in ['7089STDY13216920_BHF_F_Hea13242527',
 '7089STDY13216921_BHF_F_Hea13242528',
 '7089STDY13216922_BHF_F_Hea13242529']:
    df_s = df[df['combinedID']==sam]
    sns.violinplot(data=df_s, x='CMO | Hst40', y="match_39-40", vmax=5000)
    plt.xlim(-2000, 30000)
    plt.title(sam)
    plt.show()
```

Example 105 (python):
```python
for sam in ['7089STDY13216920_BHF_F_Hea13242527',
 '7089STDY13216921_BHF_F_Hea13242528',
 '7089STDY13216922_BHF_F_Hea13242529']:
    df_s = df[df['combinedID']==sam]
    sns.violinplot(data=df_s, x='CMO | Hst39', y="match", vmax=5000)
    plt.xlim(-1000, 20000)
    plt.title(sam)
    plt.show()
```

Example 106 (python):
```python
df - cmo

for don in set(df['CMO | donor_assignment']):
    
    if ',' not in don:
    
        df_s = df[df['CMO | donor_assignment']==don]
        plt.scatter(df_s['CMO | prop_delta'],df_s[f'CMO | {don}'])
        plt.title(f'nuclei which had max-count={don} tag')
        plt.xlabel('prop_delta')
        plt.ylabel(f'{don} count')
        plt.show()
```

---

## Aims¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig4_Cardiomyocytes/AtrialCardiomyocytes/LvR_aCM_biology_fitted_time_divergence_CHD_enrichment.html

**Contents:**
- Aims¶
- Read in data with all features¶
- Split into GEX and ATAC objects¶
- Make AUC objects¶
- Find features which are Left or Right specific¶
- GEX¶
  - Functional interpretation of Left and Right CM sigantures¶
- ATAC¶
- AUCgene¶
- AUCregion¶
  - Plot features which are specific to Left or Right aCMs¶
- Find features which increase over time¶
  - Functional interpretation of genes which increase over time across all aCMs¶
- Find features which are associated with time and are increasing over time¶
- Fit expression of features which change (increase or decrease) over time¶
- GEX¶
- ATAC¶
- AUCgene¶
- AUCregion¶

We want to demonstrate:

LvR: features which are different at all time (DE testing)

L: features which are associated and increase over time L: features which are associated and decrease over time

R: features which are associated and increase over time R: features which are associated and decrease over time

We start by definign a stricter set of L and R specific genes

**Examples:**

Example 1 (unknown):
```unknown
jctk_dir="/lustre/scratch126/cellgen/team205/jc48/jupyter/jctk"
```

Example 2 (python):
```python
import importlib.util
import sys
import types
import inspect

# ensure the `jctk_dir` path is already specified

def import_and_list_functions(module_name, file_path):
    spec = importlib.util.spec_from_file_location(module_name, file_path)
    module = importlib.util.module_from_spec(spec)
    sys.modules[module_name] = module
    spec.loader.exec_module(module)
    
    function_names = []
    for name, obj in inspect.getmembers(module):
        if isinstance(obj, types.FunctionType):
            function_names.append(name)
    
    print(f"Imported from {module_name}: {function_names}")
    
    return module  # Return the module object

# Import and list functions, then assign to a variable
plot = import_and_list_functions("plot", f"{jctk_dir}/jctk/plot.py")
utils = import_and_list_functions("utils", f"{jctk_dir}/jctk/utils.py")
api = import_and_list_functions("api", f"{jctk_dir}/jctk/api.py")
```

Example 3 (python):
```python
Imported from plot: ['GO_plot', 'highlight_all_categories', 'highlight_categories', 'normalized_stacked_bar_chart']
Imported from utils: ['check_update', 'compute_median_count', 'downsample_adata_proportionately', 'downsample_adata_randomly', 'lognorm_to_counts_with_progress', 'rank_genes_vs_nearest', 'read_gsheet', 'remove_small_batches', 'rename_adata_var_index', 'rules_based_annotation', 'shapiro', 'test_normality']
Imported from api: ['add_genomic_coordinates', 'getClinVargenes', 'getHGNCgroup', 'getHGNCgroup_dict']
```

Example 4 (python):
```python
import anndata

def split_adata_by_modality(adata, modality_column, modality_values):
    """
    Splits an AnnData object into multiple AnnData objects based on a given modality column.
    
    Parameters:
        adata (anndata.AnnData): The original AnnData object.
        modality_column (str): The name of the column in adata.var that contains the modality information.
        modality_values (list): List of unique modalities that you want to split the adata into.
        
    Returns:
        dict: A dictionary where the keys are the modality names and the values are the new AnnData objects.
    """
    # Initialize an empty dictionary to hold the new AnnData objects
    splitted_adatas = {}
    
    # Loop through each modality value to create a new AnnData object
    for modality in modality_values:
        # Get the indices of the features belonging to the current modality
        feature_idx = adata.var[modality_column] == modality
        
        # Create a new AnnData object with only the features belonging to the current modality
        new_adata = anndata.AnnData(X=adata[:, feature_idx].X,
                                    obs=adata.obs,
                                    var=adata.var.loc[feature_idx],
                                    obsm=adata.obsm,
                                    uns=adata.uns)
        
        # Add the new AnnData object to the dictionary
        splitted_adatas[modality] = new_adata
    
    return splitted_adatas
```

Example 5 (python):
```python
path_to_auc_gene="/nfs/team205/heart/pyscenic/plus/AUCscore/foetal_aCM_n570_eRegulon_AUC_gene.csv"
path_to_auc_region="/nfs/team205/heart/pyscenic/plus/AUCscore/foetal_aCM_n570_eRegulon_AUC_region.csv"

paired_adata_with_time_path = "/lustre/scratch126/cellgen/team205/heart/objects/fetal/RNA/aCMs_WholeSample_paired_with_time.h5ad"

temporal_ordering='age_days_smoothed_once'
```

Example 6 (python):
```python
import scanpy as sc
import pandas as pd
```

Example 7 (python):
```python
adata=sc.read(paired_adata_with_time_path)
adata
```

Example 8 (unknown):
```unknown
AnnData object with n_obs × n_vars = 5664 × 544641
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', 'age_days', 'age_days_smoothed_once', 'age_days_normalized', 'age_days_normalized_smoothed_once', 'age_days_smoothed_iteratively', 'age_days_normalized_smoothed_iteratively', 't', 'seg', 'edge', 't_sd', 'milestones'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id'
    uns: 'donor_colors', 'fine_grain_colors', 'graph', 'milestones_colors', 'neighbors', 'ppt', 'pseudotime_list', 'seg_colors', 'umap', 'week_colors'
    obsm: 'X_R', 'X_multiVI', 'X_umap'
    obsp: 'connectivities', 'distances'
```

Example 9 (python):
```python
adata.X.data[:5]
```

Example 10 (unknown):
```unknown
array([2., 1., 1., 1., 1.], dtype=float32)
```

Example 11 (python):
```python
sc.pl.umap(adata,color=["seg","milestones","t",temporal_ordering,"fine_grain"],ncols=2)
```

Example 12 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 13 (python):
```python
sc.pl.umap(adata,color=["fine_grain"],ncols=1)
sc.pl.umap(adata,color=[temporal_ordering,"t"],ncols=1,title="sample age")

sc.pl.umap(adata,color=["fine_grain","age_days","t"],ncols=1, legend_loc=None)
```

Example 14 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 15 (unknown):
```unknown
WARNING: The title list is shorter than the number of panels. Using 'color' value instead for some plots.
```

Example 16 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 17 (python):
```python
adata.shape
```

Example 18 (unknown):
```unknown
(5664, 544641)
```

Example 19 (python):
```python
modality_column = "modality"
modality_values = ["Gene Expression", "Peaks"]
splitted_adatas = split_adata_by_modality(adata, modality_column, modality_values)

adata_gex = splitted_adatas["Gene Expression"]
print(adata_gex)

adata_atac = splitted_adatas["Peaks"]
print(adata_atac)
```

Example 20 (unknown):
```unknown
AnnData object with n_obs × n_vars = 5664 × 36601
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', 'age_days', 'age_days_smoothed_once', 'age_days_normalized', 'age_days_normalized_smoothed_once', 'age_days_smoothed_iteratively', 'age_days_normalized_smoothed_iteratively', 't', 'seg', 'edge', 't_sd', 'milestones'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id'
    uns: 'donor_colors', 'fine_grain_colors', 'graph', 'milestones_colors', 'neighbors', 'ppt', 'pseudotime_list', 'seg_colors', 'umap', 'week_colors'
    obsm: 'X_R', 'X_multiVI', 'X_umap'
AnnData object with n_obs × n_vars = 5664 × 508040
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', 'age_days', 'age_days_smoothed_once', 'age_days_normalized', 'age_days_normalized_smoothed_once', 'age_days_smoothed_iteratively', 'age_days_normalized_smoothed_iteratively', 't', 'seg', 'edge', 't_sd', 'milestones'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id'
    uns: 'donor_colors', 'fine_grain_colors', 'graph', 'milestones_colors', 'neighbors', 'ppt', 'pseudotime_list', 'seg_colors', 'umap', 'week_colors'
    obsm: 'X_R', 'X_multiVI', 'X_umap'
```

Example 21 (python):
```python
adata_gex.X.data[:10]
```

Example 22 (unknown):
```unknown
array([2., 1., 1., 1., 1., 1., 8., 2., 1., 1.], dtype=float32)
```

Example 23 (python):
```python
adata_atac.X.data[:10]
```

Example 24 (unknown):
```unknown
array([2., 1., 1., 1., 1., 2., 2., 1., 1., 2.], dtype=float32)
```

Example 25 (python):
```python
adata_gex.layers["counts"]=adata_gex.X.copy()
adata_atac.layers["counts"]=adata_atac.X.copy()

sc.pp.normalize_total(adata_gex, target_sum=1e4)
sc.pp.normalize_total(adata_atac, target_sum=1e4)

sc.pp.log1p(adata_gex)
sc.pp.log1p(adata_atac)
```

Example 26 (python):
```python
WARNING: adata.X seems to be already log-transformed.
```

Example 27 (python):
```python
adata_gex.X.data[:10]
```

Example 28 (unknown):
```unknown
array([1.559231 , 1.0569503, 1.0569503, 1.0569503, 1.0569503, 1.0569503,
       2.7738788, 1.559231 , 1.0569503, 1.0569503], dtype=float32)
```

Example 29 (python):
```python
adata_atac.X.data[:10]
```

Example 30 (unknown):
```unknown
array([0.6963374 , 0.40759304, 0.40759304, 0.40759304, 0.40759304,
       0.6963374 , 0.6963374 , 0.40759304, 0.40759304, 0.6963374 ],
      dtype=float32)
```

Example 31 (python):
```python
# read in regulon scores
auc_gene = pd.read_csv(path_to_auc_gene, index_col=0)
print(auc_gene.shape)
auc_region = pd.read_csv(path_to_auc_region, index_col=0)
print(auc_region.shape)
```

Example 32 (unknown):
```unknown
(17302, 278)
(12964, 278)
```

Example 33 (python):
```python
# make regulon adata, AUC gene
# AUC Gene
# create anndata
adata_aucg = anndata.AnnData(
    X=auc_gene.loc[adata_gex.obs_names],
    obs=adata_gex.obs,
    var=pd.DataFrame(auc_gene.columns).set_index(0),
    obsm=adata_gex.obsm,
    uns= adata_gex.uns
)
# modify varnames
adata_aucg.var_names = [x.split('_(')[0] for x in adata_aucg.var_names]
print(adata_aucg.shape)

# AUC Region
# create anndata
adata_aucr = anndata.AnnData(
    X=auc_region.loc[adata_gex.obs_names],
    obs=adata_gex.obs,
    var=pd.DataFrame(auc_region.columns).set_index(0),
    obsm=adata_gex.obsm
)
# modify varnames
adata_aucr.var_names = [x.split('_(')[0] for x in adata_aucr.var_names]
print(adata_aucr.shape)
```

Example 34 (unknown):
```unknown
(5664, 278)
(5664, 278)
```

Example 35 (unknown):
```unknown
LEFT_or_RIGHT_features={}
```

Example 36 (unknown):
```unknown
log2fc_min=0.5
pval_cutoff=0.1
```

Example 37 (python):
```python
sc.tl.dendrogram(adata_gex,groupby="fine_grain",use_rep="X_multiVI")
sc.tl.rank_genes_groups(adata_gex,groupby="fine_grain")
sc.pl.rank_genes_groups_dotplot(adata_gex,groupby="fine_grain",n_genes=20)

df=sc.get.rank_genes_groups_df(adata,group="AtrialCardiomyocytesLeft",log2fc_min=log2fc_min)
df=df[df.pvals_adj<pval_cutoff]
LEFT_or_RIGHT_features["LEFT_genes"]=df.names.tolist()

df=sc.get.rank_genes_groups_df(adata,group="AtrialCardiomyocytesRight",log2fc_min=log2fc_min)
df=df[df.pvals_adj<pval_cutoff]
LEFT_or_RIGHT_features["RIGHT_genes"]=df.names.tolist()
```

Example 38 (python):
```python
WARNING: Default of the method has been changed to 't-test' from 't-test_overestim_var'
WARNING: Dendrogram not added. Dendrogram is added only when the number of categories to plot > 2
```

Example 39 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_dotplot.py:749: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap', 'norm' will be ignored
  dot_ax.scatter(x, y, **kwds)
```

Example 40 (unknown):
```unknown
plot.GO_plot(gene_list=LEFT_or_RIGHT_features["RIGHT_genes"],display_table=True,
             gene_sets=['GO_Biological_Process_2021'],
             title_suffix=" AtrialCardiomyocytesRight",
            n_rank=10)
```

Example 41 (unknown):
```unknown
GO_Biological_Process_2021
```

Example 42 (unknown):
```unknown
plot.GO_plot(gene_list=LEFT_or_RIGHT_features["LEFT_genes"],display_table=True,gene_sets=['GO_Biological_Process_2023'],title_suffix=" AtrialCardiomyocytesLeft")
```

Example 43 (unknown):
```unknown
GO_Biological_Process_2023
```

Example 44 (python):
```python
sc.tl.dendrogram(adata_atac,groupby="fine_grain",use_rep="X_multiVI")
sc.tl.rank_genes_groups(adata_atac,groupby="fine_grain")
sc.pl.rank_genes_groups_dotplot(adata_atac,groupby="fine_grain",n_genes=20)

df=sc.get.rank_genes_groups_df(adata,group="AtrialCardiomyocytesLeft",log2fc_min=log2fc_min)
df=df[df.pvals_adj<pval_cutoff]
print(f"adding {len(df.names.tolist())} features for LEFT...")
LEFT_or_RIGHT_features["LEFT_peaks"]=df.names.tolist()

df=sc.get.rank_genes_groups_df(adata,group="AtrialCardiomyocytesRight",log2fc_min=log2fc_min)
df=df[df.pvals_adj<pval_cutoff]
print(f"adding {len(df.names.tolist())} features for RIGHT...")
LEFT_or_RIGHT_features["RIGHT_peaks"]=df.names.tolist()
```

Example 45 (python):
```python
WARNING: Default of the method has been changed to 't-test' from 't-test_overestim_var'
WARNING: Dendrogram not added. Dendrogram is added only when the number of categories to plot > 2
```

Example 46 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_dotplot.py:749: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap', 'norm' will be ignored
  dot_ax.scatter(x, y, **kwds)
```

Example 47 (unknown):
```unknown
adding 5316 features for LEFT...
adding 3871 features for RIGHT...
```

Example 48 (python):
```python
# Instead of doing scanpy DE testing, we use the results of ArchR for this step
df=pd.read_csv("./DAR_testing/L-vs-R_aCM.csv")
df=df[(df.Log2FC>log2fc_min)&(df.FDR<pval_cutoff)]
df["names"]=df["seqnames"].astype(str)+":"+df["start"].astype(str)+"_"+df["end"].astype(str)
print(f"adding {len(df.names.tolist())} features for LEFT...")
LEFT_or_RIGHT_features["LEFT_peaks"]=df.names.tolist()

df=pd.read_csv("./DAR_testing/R-vs-L_aCM.csv")
df=df[(df.Log2FC>log2fc_min)&(df.FDR<pval_cutoff)]
df["names"]=df["seqnames"].astype(str)+":"+df["start"].astype(str)+"_"+df["end"].astype(str)
print(f"adding {len(df.names.tolist())} features for RIGHT...")
LEFT_or_RIGHT_features["RIGHT_peaks"]=df.names.tolist()
```

Example 49 (unknown):
```unknown
adding 132 features for LEFT...
adding 57 features for RIGHT...
```

Example 50 (python):
```python
sc.tl.dendrogram(adata_aucg,groupby="fine_grain",use_rep="X_multiVI")
sc.tl.rank_genes_groups(adata_aucg,groupby="fine_grain")
sc.pl.rank_genes_groups_dotplot(adata_aucg,groupby="fine_grain",n_genes=20)

df=sc.get.rank_genes_groups_df(adata_aucg,group="AtrialCardiomyocytesLeft",log2fc_min=log2fc_min)
df=df[df.pvals_adj<pval_cutoff]
print(f"adding {len(df.names.tolist())} features for LEFT...")
LEFT_or_RIGHT_features["LEFT_AUCgene"]=df.names.tolist()

df=sc.get.rank_genes_groups_df(adata_aucg,group="AtrialCardiomyocytesRight",log2fc_min=log2fc_min)
df=df[df.pvals_adj<pval_cutoff]
print(f"adding {len(df.names.tolist())} features for RIGHT...")
LEFT_or_RIGHT_features["RIGHT_AUCgene"]=df.names.tolist()
```

Example 51 (python):
```python
WARNING: Default of the method has been changed to 't-test' from 't-test_overestim_var'
WARNING: Dendrogram not added. Dendrogram is added only when the number of categories to plot > 2
```

Example 52 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_dotplot.py:749: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap', 'norm' will be ignored
  dot_ax.scatter(x, y, **kwds)
```

Example 53 (unknown):
```unknown
adding 11 features for LEFT...
adding 4 features for RIGHT...
```

Example 54 (python):
```python
sc.tl.dendrogram(adata_aucr,groupby="fine_grain",use_rep="X_multiVI")
sc.tl.rank_genes_groups(adata_aucr,groupby="fine_grain")
sc.pl.rank_genes_groups_dotplot(adata_aucr,groupby="fine_grain",n_genes=20)

df=sc.get.rank_genes_groups_df(adata_aucr,group="AtrialCardiomyocytesLeft",log2fc_min=log2fc_min)
df=df[df.pvals_adj<pval_cutoff]
print(f"adding {len(df.names.tolist())} features for LEFT...")
LEFT_or_RIGHT_features["LEFT_AUCregion"]=df.names.tolist()

df=sc.get.rank_genes_groups_df(adata_aucr,group="AtrialCardiomyocytesRight",log2fc_min=log2fc_min)
df=df[df.pvals_adj<pval_cutoff]
print(f"adding {len(df.names.tolist())} features for RIGHT...")
LEFT_or_RIGHT_features["RIGHT_AUCregion"]=df.names.tolist()
```

Example 55 (python):
```python
WARNING: Default of the method has been changed to 't-test' from 't-test_overestim_var'
WARNING: Dendrogram not added. Dendrogram is added only when the number of categories to plot > 2
```

Example 56 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_dotplot.py:749: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap', 'norm' will be ignored
  dot_ax.scatter(x, y, **kwds)
```

Example 57 (unknown):
```unknown
adding 15 features for LEFT...
adding 0 features for RIGHT...
```

Example 58 (python):
```python
import scanpy as sc  # Import scanpy, assuming it's used for gene scoring and plotting

def plot_sided_features(adata, feature_type, associated_increasing_features_dict, cmap="RdPu", sides=["LEFT", "RIGHT", "COMMON","NON_SIDE_SPECIFIC"]):
    """
    Plots sided features for a given AnnData object and feature type.
    
    Args:
    adata (AnnData): An AnnData object containing gene expression data.
    feature_type (str): The type of feature to plot.
    associated_increasing_features_dict (dict): A dictionary containing associated increasing features.
    """
    # Copy the adata object to avoid modifying the original data
    plot_adata = adata.copy()

    # Initialize an empty list to store score names
    scores = []

    # Iterate over the sides and calculate scores for each
    for side in sides:
        # Retrieve the gene list from the dictionary
        gene_list = associated_increasing_features_dict[f"{feature_type}_{side}"]

        if gene_list:
            # Construct a unique name for the score
            score_name = f"{feature_type}_{side}_({len(gene_list)} features)"
            # Add the score name to the list
            scores.append(score_name)
            # Calculate the score using scanpy's score_genes function
            sc.tl.score_genes(plot_adata, gene_list=gene_list, score_name=score_name)

    # Plot the UMAP with the scores
    if len(scores)>0:
        sc.pl.umap(plot_adata, color=scores, ncols=3, wspace=0.8, cmap=cmap)
```

Example 59 (unknown):
```unknown
# New dictionary to store converted keys
LEFT_or_RIGHT_features_converted = {}

# Iterate through each item in LEFT_or_RIGHT_features
for key, value in LEFT_or_RIGHT_features.items():
    # Split the key on the underscore
    parts = key.split('_')
    
    # Rearrange the parts to match the desired format
    # New key format: 'genes_LEFT', 'genes_RIGHT', etc.
    new_key = f"{parts[1]}_{parts[0]}"

    # Add to the converted dictionary
    LEFT_or_RIGHT_features_converted[new_key] = value
LEFT_or_RIGHT_features_converted.keys()
```

Example 60 (unknown):
```unknown
dict_keys(['genes_LEFT', 'genes_RIGHT', 'peaks_LEFT', 'peaks_RIGHT', 'AUCgene_LEFT', 'AUCgene_RIGHT', 'AUCregion_LEFT', 'AUCregion_RIGHT'])
```

Example 61 (python):
```python
plot_sided_features(adata_gex,
                    sides=["LEFT","RIGHT"],
                    feature_type="genes",
                    associated_increasing_features_dict=LEFT_or_RIGHT_features_converted)

plot_sided_features(adata_atac,
                    sides=["LEFT","RIGHT"],
                    feature_type="peaks",
                    associated_increasing_features_dict=LEFT_or_RIGHT_features_converted)

plot_sided_features(adata_aucg,
                    sides=["LEFT","RIGHT"],
                    feature_type="AUCgene",
                    associated_increasing_features_dict=LEFT_or_RIGHT_features_converted)

plot_sided_features(adata_aucr,
                    sides=["LEFT","RIGHT"],
                    feature_type="AUCregion",
                    associated_increasing_features_dict=LEFT_or_RIGHT_features_converted)
```

Example 62 (python):
```python
print(LEFT_or_RIGHT_features_converted["AUCgene_LEFT"])
print(LEFT_or_RIGHT_features_converted["AUCregion_LEFT"])
print(LEFT_or_RIGHT_features_converted["AUCgene_RIGHT"])
```

Example 63 (unknown):
```unknown
['PITX2_+', 'ETS2_-', 'MAF_-', 'GATA5_+', 'SIX5_extended_+', 'ZNF337_+', 'PEG3_+', 'NFATC1_+', 'ZNF383_-', 'ARID3A_extended_-', 'MEF2A_-']
['GTF2IRD1_+', 'PITX2_+', 'IKZF2_+', 'NFE2L1_+', 'NFE2L2_+', 'TCF7L2_+', 'MGA_+', 'FOXJ2_+', 'MAF_-', 'STAT5B_+', 'MESP1_extended_+', 'ETS2_-', 'STAT4_+', 'ZNF337_+', 'ZNF470_+']
['ETS2_+', 'MAF_+', 'MYCN_+', 'ZNF526_+']
```

Example 64 (python):
```python
import pandas as pd
import numpy as np
from typing import Any

def bin_pseudotime(adata: Any, pseudotime_column: str, n_bins: int) -> Any:
    """
    Bin cells in an AnnData object into n_bins based on a specified pseudotime column.

    Parameters:
    adata (AnnData): The AnnData object containing single-cell data.
    pseudotime_column (str): The name of the column in adata.obs that contains pseudotime values.
    n_bins (int): The number of bins to divide the pseudotime values into.

    Returns:
    AnnData: The AnnData object with an additional column in adata.obs indicating the bin each cell belongs to.
    """
    # Calculate the percentiles that will be used as bin edges
    percentiles = [i*(100/n_bins) for i in range(1, n_bins)]
    bin_edges = np.percentile(adata.obs[pseudotime_column], percentiles)
    
    # Bin values into discrete intervals using pandas.cut
    adata.obs[f'{pseudotime_column}_bins'] = pd.cut(
        adata.obs[pseudotime_column],
        bins=np.insert(bin_edges, [0, len(bin_edges)], [adata.obs[pseudotime_column].min(), adata.obs[pseudotime_column].max()]),  # Include the extreme values of the pseudotime range
        include_lowest=True,
        right=True,
        labels=[str(i) for i in range(n_bins)]
    )
    
    # Return the modified AnnData object
    return adata
```

Example 65 (unknown):
```unknown
n_bins=2 # number of temporal bins
```

Example 66 (python):
```python
adata_gex = bin_pseudotime(adata_gex, 't', n_bins)
adata_atac = bin_pseudotime(adata_atac, 't', n_bins)

adata_gex = bin_pseudotime(adata_gex, temporal_ordering, n_bins)
adata_atac = bin_pseudotime(adata_atac, temporal_ordering, n_bins)

adata_aucg = bin_pseudotime(adata_aucg, 't', n_bins)
adata_aucr = bin_pseudotime(adata_aucg, 't', n_bins)

adata_aucg = bin_pseudotime(adata_aucr, temporal_ordering, n_bins)
adata_aucr = bin_pseudotime(adata_aucr, temporal_ordering, n_bins)

sc.pl.umap(adata_gex,color=["seg","milestones","t",temporal_ordering,"fine_grain","t_bins",f"{temporal_ordering}_bins"],ncols=2,wspace=0.8)
```

Example 67 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 68 (unknown):
```unknown
max_group=n_bins-1
max_group=str(max_group)
max_group
```

Example 69 (python):
```python
adata_gex.obs[f"{temporal_ordering}_bins"].value_counts()
```

Example 70 (unknown):
```unknown
0    2832
1    2832
Name: age_days_smoothed_once_bins, dtype: int64
```

Example 71 (python):
```python
increasing_features_dict={}

sc.tl.rank_genes_groups(adata_gex,groupby=f"{temporal_ordering}_bins",groups=[max_group],reference="0")
sc.pl.rank_genes_groups_dotplot(adata_gex,groupby=f"{temporal_ordering}_bins",n_genes=40,dendrogram=False)

df=sc.get.rank_genes_groups_df(adata_gex,group=max_group,log2fc_min=log2fc_min)
df=df[df.pvals_adj<pval_cutoff]
increasing_features_dict["INCREASING_genes"]=df.names.tolist()




sc.tl.rank_genes_groups(adata_atac,groupby=f"{temporal_ordering}_bins",groups=[max_group],reference="0")
sc.pl.rank_genes_groups_dotplot(adata_atac,groupby=f"{temporal_ordering}_bins",n_genes=40,dendrogram=False)

df=sc.get.rank_genes_groups_df(adata_atac,group=max_group,log2fc_min=log2fc_min)
#df=df[df.pvals_adj<pval_cutoff]
df=df[df.pvals<pval_cutoff] #pvals
increasing_features_dict["INCREASING_peaks"]=df.names.tolist()




sc.tl.rank_genes_groups(adata_aucg,groupby=f"{temporal_ordering}_bins",groups=[max_group],reference="0")
sc.pl.rank_genes_groups_dotplot(adata_aucg,groupby=f"{temporal_ordering}_bins",n_genes=40,dendrogram=False)

df=sc.get.rank_genes_groups_df(adata_aucg,group=max_group,log2fc_min=log2fc_min)
#df=df[df.pvals_adj<pval_cutoff]
df=df[df.pvals<pval_cutoff] #pvals
increasing_features_dict["INCREASING_AUCgene"]=df.names.tolist()




sc.tl.rank_genes_groups(adata_aucr,groupby=f"{temporal_ordering}_bins",groups=[max_group],reference="0")
sc.pl.rank_genes_groups_dotplot(adata_aucr,groupby=f"{temporal_ordering}_bins",n_genes=40,dendrogram=False)

df=sc.get.rank_genes_groups_df(adata_aucr,group=max_group,log2fc_min=log2fc_min)
#df=df[df.pvals_adj<pval_cutoff]
df=df[df.pvals<pval_cutoff] #pvals
increasing_features_dict["INCREASING_AUCregion"]=df.names.tolist()
```

Example 72 (python):
```python
WARNING: Default of the method has been changed to 't-test' from 't-test_overestim_var'
```

Example 73 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_dotplot.py:749: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap', 'norm' will be ignored
  dot_ax.scatter(x, y, **kwds)
```

Example 74 (python):
```python
WARNING: Default of the method has been changed to 't-test' from 't-test_overestim_var'
```

Example 75 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_dotplot.py:749: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap', 'norm' will be ignored
  dot_ax.scatter(x, y, **kwds)
```

Example 76 (python):
```python
WARNING: Default of the method has been changed to 't-test' from 't-test_overestim_var'
```

Example 77 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_dotplot.py:749: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap', 'norm' will be ignored
  dot_ax.scatter(x, y, **kwds)
```

Example 78 (python):
```python
WARNING: Default of the method has been changed to 't-test' from 't-test_overestim_var'
```

Example 79 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_dotplot.py:749: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap', 'norm' will be ignored
  dot_ax.scatter(x, y, **kwds)
```

Example 80 (unknown):
```unknown
plot.GO_plot(gene_list=increasing_features_dict["INCREASING_genes"],display_table=True,
             gene_sets=['MSigDB_Hallmark_2020'],
             title_suffix=" AtrialCardiomyocytesCommon increasing over time")
```

Example 81 (unknown):
```unknown
MSigDB_Hallmark_2020
```

Example 82 (unknown):
```unknown
increasing_features_dict
```

Example 83 (python):
```python
from tqdm import tqdm
import pandas as pd

sort_col = "A"
cutoff_col = 'fdr'
cutoff_value = 1e-1

associated_increasing_features_dict = {}

for feature in tqdm(["genes", "peaks", "AUCgene", "AUCregion"]):
    print("\n######### {} #########\n".format(feature))
    
    associated_increasing_NONsidespecific_features = []

    for side in ["LEFT", "RIGHT"]:
        # Load and sort the data
        df = pd.read_csv(f"./fitted_features/aCMs_WholeSample_scfatespseudotime_{feature}_{side}.csv", index_col=0)
        df = df.sort_values(sort_col, ascending=False).drop(columns="activator_repressor")
        print(f"Total number of {feature}_{side} before filtering: {len(df)}")
        
        # Apply cutoff and filter positive 'A' values
        associated_features = df[(df[cutoff_col] < cutoff_value) & (df["A"] > 0.5)].index.tolist()
        print(f"Number of {feature}_{side} after cutoff and 'A' filtering: {len(associated_features)}")
        
        # Filter for increasing features
        increasing_features = increasing_features_dict[f"INCREASING_{feature}"]
        print(f"Number of increasing {feature}: {len(increasing_features)}")
        
        # Find associated increasing features
        associated_increasing_features = [f for f in associated_features if f in increasing_features]
        print(f"Number of increasing {feature}_{side} after associating: {len(associated_increasing_features)}")
        
        # Filter for side-specific features
        side_specific_features = LEFT_or_RIGHT_features[f"{side}_{feature}"]
        print(f"Number of {side}-specific {feature}: {len(side_specific_features)}")
        
        # Find associated increasing side-specific features
        associated_increasing_sidespecific_features = [f for f in associated_increasing_features if f in side_specific_features]
        print(f"Number of associated increasing {side}-specific {feature}: {len(associated_increasing_sidespecific_features)}")
        
        # Find non-side-specific features
        non_side_specific_features = [f for f in associated_increasing_features if f not in side_specific_features]
        print(f"Number of associated increasing non-{side}-specific {feature}: {len(non_side_specific_features)}")
        
        # Combine non-side-specific features for later processing
        associated_increasing_NONsidespecific_features.extend(non_side_specific_features)
        
        # Store side-specific features
        associated_increasing_features_dict[f"{feature}_{side}"] = associated_increasing_sidespecific_features

    # Deduplicate the list of non-side-specific features
    non_side_specific_set = set(associated_increasing_NONsidespecific_features)
    print(f"Total number of associated increasing non-side-specific {feature}: {len(non_side_specific_set)}")

    # Store these NonSideSpecific features
    associated_increasing_features_dict[f"{feature}_NON_SIDE_SPECIFIC"] = list(non_side_specific_set)

    # Identify common features (present in both LEFT and RIGHT)
    common_features = non_side_specific_set.intersection(
        associated_increasing_features_dict[f"{feature}_LEFT"],
        associated_increasing_features_dict[f"{feature}_RIGHT"]
    )
    associated_increasing_features_dict[f"{feature}_COMMON"] = list(common_features)
    print(f"Total number of common non-side-specific {feature}: {len(common_features)}")

# Calculate and display counts for each category
counts_dict = {key: len(value) for key, value in associated_increasing_features_dict.items()}
print("\nCounts per category:")
for key, value in counts_dict.items():
    print(f"{key}: {value}")
```

Example 84 (unknown):
```unknown
0%|                                                                                                                                                                                                                     | 0/4 [00:00<?, ?it/s]
```

Example 85 (unknown):
```unknown
######### genes #########

Total number of genes_LEFT before filtering: 21922
```

Example 86 (unknown):
```unknown
25%|███████████████████████████████████████████████████▎                                                                                                                                                         | 1/4 [00:00<00:00,  7.55it/s]
```

Example 87 (unknown):
```unknown
Number of genes_LEFT after cutoff and 'A' filtering: 3626
Number of increasing genes: 1178
Number of increasing genes_LEFT after associating: 577
Number of LEFT-specific genes: 1440
Number of associated increasing LEFT-specific genes: 98
Number of associated increasing non-LEFT-specific genes: 479
Total number of genes_RIGHT before filtering: 21024
Number of genes_RIGHT after cutoff and 'A' filtering: 3439
Number of increasing genes: 1178
Number of increasing genes_RIGHT after associating: 485
Number of RIGHT-specific genes: 936
Number of associated increasing RIGHT-specific genes: 83
Number of associated increasing non-RIGHT-specific genes: 402
Total number of associated increasing non-side-specific genes: 569
Total number of common non-side-specific genes: 0

######### peaks #########

Total number of peaks_LEFT before filtering: 167061
Number of peaks_LEFT after cutoff and 'A' filtering: 4835
Number of increasing peaks: 5856
Number of increasing peaks_LEFT after associating: 113
Number of LEFT-specific peaks: 132
Number of associated increasing LEFT-specific peaks: 1
Number of associated increasing non-LEFT-specific peaks: 112
Total number of peaks_RIGHT before filtering: 166063
Number of peaks_RIGHT after cutoff and 'A' filtering: 7171
Number of increasing peaks: 5856
```

Example 88 (unknown):
```unknown
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:01<00:00,  2.87it/s]
```

Example 89 (unknown):
```unknown
Number of increasing peaks_RIGHT after associating: 166
Number of RIGHT-specific peaks: 57
Number of associated increasing RIGHT-specific peaks: 0
Number of associated increasing non-RIGHT-specific peaks: 166
Total number of associated increasing non-side-specific peaks: 262
Total number of common non-side-specific peaks: 0

######### AUCgene #########

Total number of AUCgene_LEFT before filtering: 278
Number of AUCgene_LEFT after cutoff and 'A' filtering: 83
Number of increasing AUCgene: 4
Number of increasing AUCgene_LEFT after associating: 4
Number of LEFT-specific AUCgene: 11
Number of associated increasing LEFT-specific AUCgene: 0
Number of associated increasing non-LEFT-specific AUCgene: 4
Total number of AUCgene_RIGHT before filtering: 278
Number of AUCgene_RIGHT after cutoff and 'A' filtering: 85
Number of increasing AUCgene: 4
Number of increasing AUCgene_RIGHT after associating: 4
Number of RIGHT-specific AUCgene: 4
Number of associated increasing RIGHT-specific AUCgene: 0
Number of associated increasing non-RIGHT-specific AUCgene: 4
Total number of associated increasing non-side-specific AUCgene: 4
Total number of common non-side-specific AUCgene: 0

######### AUCregion #########

Total number of AUCregion_LEFT before filtering: 278
Number of AUCregion_LEFT after cutoff and 'A' filtering: 75
Number of increasing AUCregion: 4
Number of increasing AUCregion_LEFT after associating: 3
Number of LEFT-specific AUCregion: 15
Number of associated increasing LEFT-specific AUCregion: 0
Number of associated increasing non-LEFT-specific AUCregion: 3
Total number of AUCregion_RIGHT before filtering: 278
Number of AUCregion_RIGHT after cutoff and 'A' filtering: 72
Number of increasing AUCregion: 4
Number of increasing AUCregion_RIGHT after associating: 3
Number of RIGHT-specific AUCregion: 0
Number of associated increasing RIGHT-specific AUCregion: 0
Number of associated increasing non-RIGHT-specific AUCregion: 3
Total number of associated increasing non-side-specific AUCregion: 4
Total number of common non-side-specific AUCregion: 0

Counts per category:
genes_LEFT: 98
genes_RIGHT: 83
genes_NON_SIDE_SPECIFIC: 569
genes_COMMON: 0
peaks_LEFT: 1
peaks_RIGHT: 0
peaks_NON_SIDE_SPECIFIC: 262
peaks_COMMON: 0
AUCgene_LEFT: 0
AUCgene_RIGHT: 0
AUCgene_NON_SIDE_SPECIFIC: 4
AUCgene_COMMON: 0
AUCregion_LEFT: 0
AUCregion_RIGHT: 0
AUCregion_NON_SIDE_SPECIFIC: 4
AUCregion_COMMON: 0
```

Example 90 (python):
```python
plot_sided_features(adata_gex,
                    feature_type="genes",
                    associated_increasing_features_dict=associated_increasing_features_dict)

plot_sided_features(adata_atac,
                    feature_type="peaks",
                    associated_increasing_features_dict=associated_increasing_features_dict)

plot_sided_features(adata_aucg,
                    feature_type="AUCgene",
                    associated_increasing_features_dict=associated_increasing_features_dict)

plot_sided_features(adata_aucr,
                    feature_type="AUCregion",
                    associated_increasing_features_dict=associated_increasing_features_dict)
```

Example 91 (unknown):
```unknown
LEFT_or_RIGHT_features={}
```

Example 92 (unknown):
```unknown
log2fc_min=2
pval_cutoff=0.05
```

Example 93 (python):
```python
sc.tl.dendrogram(adata_gex,groupby="fine_grain",use_rep="X_multiVI")
sc.tl.rank_genes_groups(adata_gex,groupby="fine_grain")
sc.pl.rank_genes_groups_dotplot(adata_gex,groupby="fine_grain",n_genes=20)

df=sc.get.rank_genes_groups_df(adata,group="AtrialCardiomyocytesLeft",log2fc_min=log2fc_min)
df=df[df.pvals_adj<pval_cutoff]
LEFT_or_RIGHT_features["LEFT_genes"]=df.names.tolist()

df=sc.get.rank_genes_groups_df(adata,group="AtrialCardiomyocytesRight",log2fc_min=log2fc_min)
df=df[df.pvals_adj<pval_cutoff]
LEFT_or_RIGHT_features["RIGHT_genes"]=df.names.tolist()
```

Example 94 (python):
```python
WARNING: Default of the method has been changed to 't-test' from 't-test_overestim_var'
WARNING: Dendrogram not added. Dendrogram is added only when the number of categories to plot > 2
```

Example 95 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_dotplot.py:749: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap', 'norm' will be ignored
  dot_ax.scatter(x, y, **kwds)
```

Example 96 (python):
```python
sc.tl.dendrogram(adata_atac,groupby="fine_grain",use_rep="X_multiVI")
sc.tl.rank_genes_groups(adata_atac,groupby="fine_grain")
sc.pl.rank_genes_groups_dotplot(adata_atac,groupby="fine_grain",n_genes=20)

df=sc.get.rank_genes_groups_df(adata,group="AtrialCardiomyocytesLeft",log2fc_min=log2fc_min)
df=df[df.pvals_adj<pval_cutoff]
print(f"adding {len(df.names.tolist())} features for LEFT...")
LEFT_or_RIGHT_features["LEFT_peaks"]=df.names.tolist()

df=sc.get.rank_genes_groups_df(adata,group="AtrialCardiomyocytesRight",log2fc_min=log2fc_min)
df=df[df.pvals_adj<pval_cutoff]
print(f"adding {len(df.names.tolist())} features for RIGHT...")
LEFT_or_RIGHT_features["RIGHT_peaks"]=df.names.tolist()
```

Example 97 (python):
```python
WARNING: Default of the method has been changed to 't-test' from 't-test_overestim_var'
WARNING: Dendrogram not added. Dendrogram is added only when the number of categories to plot > 2
```

Example 98 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_dotplot.py:749: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap', 'norm' will be ignored
  dot_ax.scatter(x, y, **kwds)
```

Example 99 (unknown):
```unknown
adding 1100 features for LEFT...
adding 414 features for RIGHT...
```

Example 100 (python):
```python
# Instead of doing scanpy DE testing, we use the results of ArchR for this step
df=pd.read_csv("./DAR_testing/L-vs-R_aCM.csv")
df=df[(df.Log2FC>log2fc_min)&(df.FDR<pval_cutoff)]
df["names"]=df["seqnames"].astype(str)+":"+df["start"].astype(str)+"_"+df["end"].astype(str)
print(f"adding {len(df.names.tolist())} features for LEFT...")
LEFT_or_RIGHT_features["LEFT_peaks"]=df.names.tolist()

df=pd.read_csv("./DAR_testing/R-vs-L_aCM.csv")
df=df[(df.Log2FC>log2fc_min)&(df.FDR<pval_cutoff)]
df["names"]=df["seqnames"].astype(str)+":"+df["start"].astype(str)+"_"+df["end"].astype(str)
print(f"adding {len(df.names.tolist())} features for RIGHT...")
LEFT_or_RIGHT_features["RIGHT_peaks"]=df.names.tolist()
```

Example 101 (unknown):
```unknown
adding 71 features for LEFT...
adding 19 features for RIGHT...
```

Example 102 (python):
```python
sc.tl.dendrogram(adata_aucg,groupby="fine_grain",use_rep="X_multiVI")
sc.tl.rank_genes_groups(adata_aucg,groupby="fine_grain")
sc.pl.rank_genes_groups_dotplot(adata_aucg,groupby="fine_grain",n_genes=20)

df=sc.get.rank_genes_groups_df(adata_aucg,group="AtrialCardiomyocytesLeft",log2fc_min=log2fc_min)
df=df[df.pvals_adj<pval_cutoff]
print(f"adding {len(df.names.tolist())} features for LEFT...")
LEFT_or_RIGHT_features["LEFT_AUCgene"]=df.names.tolist()

df=sc.get.rank_genes_groups_df(adata_aucg,group="AtrialCardiomyocytesRight",log2fc_min=log2fc_min)
df=df[df.pvals_adj<pval_cutoff]
print(f"adding {len(df.names.tolist())} features for RIGHT...")
LEFT_or_RIGHT_features["RIGHT_AUCgene"]=df.names.tolist()
```

Example 103 (python):
```python
WARNING: Default of the method has been changed to 't-test' from 't-test_overestim_var'
WARNING: Dendrogram not added. Dendrogram is added only when the number of categories to plot > 2
```

Example 104 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_dotplot.py:749: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap', 'norm' will be ignored
  dot_ax.scatter(x, y, **kwds)
```

Example 105 (unknown):
```unknown
adding 0 features for LEFT...
adding 0 features for RIGHT...
```

Example 106 (python):
```python
sc.tl.dendrogram(adata_aucr,groupby="fine_grain",use_rep="X_multiVI")
sc.tl.rank_genes_groups(adata_aucr,groupby="fine_grain")
sc.pl.rank_genes_groups_dotplot(adata_aucr,groupby="fine_grain",n_genes=20)

df=sc.get.rank_genes_groups_df(adata_aucr,group="AtrialCardiomyocytesLeft",log2fc_min=log2fc_min)
df=df[df.pvals_adj<pval_cutoff]
print(f"adding {len(df.names.tolist())} features for LEFT...")
LEFT_or_RIGHT_features["LEFT_AUCregion"]=df.names.tolist()

df=sc.get.rank_genes_groups_df(adata_aucr,group="AtrialCardiomyocytesRight",log2fc_min=log2fc_min)
df=df[df.pvals_adj<pval_cutoff]
print(f"adding {len(df.names.tolist())} features for RIGHT...")
LEFT_or_RIGHT_features["RIGHT_AUCregion"]=df.names.tolist()
```

Example 107 (python):
```python
WARNING: Default of the method has been changed to 't-test' from 't-test_overestim_var'
WARNING: Dendrogram not added. Dendrogram is added only when the number of categories to plot > 2
```

Example 108 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_dotplot.py:749: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap', 'norm' will be ignored
  dot_ax.scatter(x, y, **kwds)
```

Example 109 (unknown):
```unknown
adding 0 features for LEFT...
adding 0 features for RIGHT...
```

Example 110 (python):
```python
from tqdm import tqdm
import pandas as pd

sort_col="A"
cutff_col = 'fdr'
cutoff_value = 1e-1

associated_features_dict = {}

for feature in tqdm(["genes", "peaks", "AUCgene", "AUCregion"]):
    print("\n")
    print(f"######### {feature} #########")
    print("\n")
    
    associated_increasing_NONsidespecific_features = []

    for side in ["LEFT", "RIGHT"]:
        # Read and sort the CSV file
        df = pd.read_csv(f"./fitted_features/aCMs_WholeSample_scfatespseudotime_{feature}_{side}.csv", index_col=0).sort_values(sort_col,ascending=False)
        df = df.drop(columns="activator_repressor")
        print(f"Total number of {feature}_{side} before filtering: {len(df)}")
        
        associated_features = df[(df[cutff_col] < cutoff_value) & (df["A"] > 1)].index.tolist()
        print(f"Number of {feature}_{side} after cutoff and 'A' filtering: {len(associated_features)}")
        
        # Filter for side-specific features
        side_specific_features = LEFT_or_RIGHT_features[f"{side}_{feature}"]
        print(f"Number of {side}-specific {feature}: {len(side_specific_features)}")
        
        associated_sidespecific_features = [f for f in associated_features if f in side_specific_features]
        print(f"Number of associated {side}-specific {feature}: {len(associated_sidespecific_features)}")
        
        associated_features_dict[f"{feature}_{side}"] = associated_sidespecific_features

    # Read in COMMON features which are associated with pseudotime
    df = pd.read_csv(f"./fitted_features/aCMs_WholeSample_scfatespseudotime_{feature}_COMMON.csv", index_col=0).sort_values(sort_col,ascending=False)
    df = df.drop(columns="activator_repressor")
    print(f"Total number of {feature}_COMMON before filtering: {len(df)}")
    
    associated_features = df[(df[cutff_col] < cutoff_value) & (df["A"] > 1)].index.tolist()
    print(f"Number of {feature}_COMMON after cutoff and 'A' filtering: {len(associated_features)}")

    # Filter out any features which are known to be side-specific
    LEFT_side_specific_features = LEFT_or_RIGHT_features[f"LEFT_{feature}"]
    RIGHT_side_specific_features = LEFT_or_RIGHT_features[f"RIGHT_{feature}"]
    LEFT_or_RIGHT_side_specific_features=LEFT_side_specific_features+RIGHT_side_specific_features
    print(f"Number of (either) side-specific {feature}: {len(LEFT_or_RIGHT_side_specific_features)}")

    associated_non_sidespecific_features = [f for f in associated_features if f not in LEFT_or_RIGHT_side_specific_features]
    print(f"Number of associated non-side-specific {feature}: {len(associated_non_sidespecific_features)}")

    associated_features_dict[f"{feature}_COMMON"]=associated_non_sidespecific_features

# Calculate the counts for each category
counts_dict = {key: len(value) for key, value in associated_features_dict.items()}
print("\nCounts per category:")
for key, value in counts_dict.items():
    print(f"{key}: {value}")
```

Example 111 (unknown):
```unknown
0%|                                                                                                                                                                                                                     | 0/4 [00:00<?, ?it/s]
```

Example 112 (unknown):
```unknown
######### genes #########


Total number of genes_LEFT before filtering: 21922
Number of genes_LEFT after cutoff and 'A' filtering: 2519
Number of LEFT-specific genes: 209
Number of associated LEFT-specific genes: 28
Total number of genes_RIGHT before filtering: 21024
Number of genes_RIGHT after cutoff and 'A' filtering: 2584
Number of RIGHT-specific genes: 129
Number of associated RIGHT-specific genes: 29
```

Example 113 (unknown):
```unknown
25%|███████████████████████████████████████████████████▎                                                                                                                                                         | 1/4 [00:00<00:00,  4.54it/s]
```

Example 114 (unknown):
```unknown
Total number of genes_COMMON before filtering: 23953
Number of genes_COMMON after cutoff and 'A' filtering: 1081
Number of (either) side-specific genes: 338
Number of associated non-side-specific genes: 1048


######### peaks #########


Total number of peaks_LEFT before filtering: 167061
Number of peaks_LEFT after cutoff and 'A' filtering: 1122
Number of LEFT-specific peaks: 71
Number of associated LEFT-specific peaks: 0
Total number of peaks_RIGHT before filtering: 166063
Number of peaks_RIGHT after cutoff and 'A' filtering: 3933
Number of RIGHT-specific peaks: 19
Number of associated RIGHT-specific peaks: 0
```

Example 115 (unknown):
```unknown
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:01<00:00,  3.70it/s]
```

Example 116 (unknown):
```unknown
Total number of peaks_COMMON before filtering: 165795
Number of peaks_COMMON after cutoff and 'A' filtering: 0
Number of (either) side-specific peaks: 90
Number of associated non-side-specific peaks: 0


######### AUCgene #########


Total number of AUCgene_LEFT before filtering: 278
Number of AUCgene_LEFT after cutoff and 'A' filtering: 62
Number of LEFT-specific AUCgene: 0
Number of associated LEFT-specific AUCgene: 0
Total number of AUCgene_RIGHT before filtering: 278
Number of AUCgene_RIGHT after cutoff and 'A' filtering: 69
Number of RIGHT-specific AUCgene: 0
Number of associated RIGHT-specific AUCgene: 0
Total number of AUCgene_COMMON before filtering: 278
Number of AUCgene_COMMON after cutoff and 'A' filtering: 52
Number of (either) side-specific AUCgene: 0
Number of associated non-side-specific AUCgene: 52


######### AUCregion #########


Total number of AUCregion_LEFT before filtering: 278
Number of AUCregion_LEFT after cutoff and 'A' filtering: 50
Number of LEFT-specific AUCregion: 0
Number of associated LEFT-specific AUCregion: 0
Total number of AUCregion_RIGHT before filtering: 278
Number of AUCregion_RIGHT after cutoff and 'A' filtering: 58
Number of RIGHT-specific AUCregion: 0
Number of associated RIGHT-specific AUCregion: 0
Total number of AUCregion_COMMON before filtering: 278
Number of AUCregion_COMMON after cutoff and 'A' filtering: 56
Number of (either) side-specific AUCregion: 0
Number of associated non-side-specific AUCregion: 56

Counts per category:
genes_LEFT: 28
genes_RIGHT: 29
genes_COMMON: 1048
peaks_LEFT: 0
peaks_RIGHT: 0
peaks_COMMON: 0
AUCgene_LEFT: 0
AUCgene_RIGHT: 0
AUCgene_COMMON: 52
AUCregion_LEFT: 0
AUCregion_RIGHT: 0
AUCregion_COMMON: 56
```

Example 117 (python):
```python
# similar to before but we do not filter using the increasing features dictionary
from tqdm import tqdm
import pandas as pd

sort_col = "A"
cutoff_col = 'fdr'
cutoff_value = 1e-1

associated_increasing_features_dict = {}

for feature in tqdm(["genes", "peaks", "AUCgene", "AUCregion"]):
    print("\n######### {} #########\n".format(feature))
    
    associated_increasing_NONsidespecific_features = []

    for side in ["LEFT", "RIGHT"]:
        # Load and sort the data
        df = pd.read_csv(f"./fitted_features/aCMs_WholeSample_scfatespseudotime_{feature}_{side}.csv", index_col=0)
        df = df.sort_values(sort_col, ascending=False).drop(columns="activator_repressor")
        print(f"Total number of {feature}_{side} before filtering: {len(df)}")
        
        # Apply cutoff and filter positive 'A' values
        associated_features = df[(df[cutoff_col] < cutoff_value) & (df["A"] > 0.5)].index.tolist()
        print(f"Number of {feature}_{side} after cutoff and 'A' filtering: {len(associated_features)}")
        
        # Filter for increasing features
        #increasing_features = increasing_features_dict[f"INCREASING_{feature}"]
        #print(f"Number of increasing {feature}: {len(increasing_features)}")
        
        # Find associated increasing features
        #associated_increasing_features = [f for f in associated_features if f in increasing_features]
        #print(f"Number of increasing {feature}_{side} after associating: {len(associated_increasing_features)}")

        associated_increasing_features=associated_features
        
        # Filter for side-specific features
        side_specific_features = LEFT_or_RIGHT_features[f"{side}_{feature}"]
        print(f"Number of {side}-specific {feature}: {len(side_specific_features)}")
        
        # Find associated increasing side-specific features
        associated_increasing_sidespecific_features = [f for f in associated_increasing_features if f in side_specific_features]
        print(f"Number of associated increasing {side}-specific {feature}: {len(associated_increasing_sidespecific_features)}")
        
        # Find non-side-specific features
        non_side_specific_features = [f for f in associated_increasing_features if f not in side_specific_features]
        print(f"Number of associated increasing non-{side}-specific {feature}: {len(non_side_specific_features)}")
        
        # Combine non-side-specific features for later processing
        associated_increasing_NONsidespecific_features.extend(non_side_specific_features)
        
        # Store side-specific features
        associated_increasing_features_dict[f"{feature}_{side}"] = associated_increasing_sidespecific_features

    # Deduplicate the list of non-side-specific features
    non_side_specific_set = set(associated_increasing_NONsidespecific_features)
    print(f"Total number of associated increasing non-side-specific {feature}: {len(non_side_specific_set)}")

    # Store these NonSideSpecific features
    associated_increasing_features_dict[f"{feature}_NON_SIDE_SPECIFIC"] = list(non_side_specific_set)

    # Identify common features (present in both LEFT and RIGHT)
    common_features = non_side_specific_set.intersection(
        associated_increasing_features_dict[f"{feature}_LEFT"],
        associated_increasing_features_dict[f"{feature}_RIGHT"]
    )
    associated_increasing_features_dict[f"{feature}_COMMON"] = list(common_features)
    print(f"Total number of common non-side-specific {feature}: {len(common_features)}")

# Calculate and display counts for each category
counts_dict = {key: len(value) for key, value in associated_increasing_features_dict.items()}
print("\nCounts per category:")
for key, value in counts_dict.items():
    print(f"{key}: {value}")
```

Example 118 (unknown):
```unknown
0%|                                                                                                                                                                                                                     | 0/4 [00:00<?, ?it/s]
```

Example 119 (unknown):
```unknown
######### genes #########

Total number of genes_LEFT before filtering: 21922
Number of genes_LEFT after cutoff and 'A' filtering: 3626
Number of LEFT-specific genes: 209
Number of associated increasing LEFT-specific genes: 50
Number of associated increasing non-LEFT-specific genes: 3576
Total number of genes_RIGHT before filtering: 21024
Number of genes_RIGHT after cutoff and 'A' filtering: 3439
Number of RIGHT-specific genes: 129
Number of associated increasing RIGHT-specific genes: 42
Number of associated increasing non-RIGHT-specific genes: 3397
Total number of associated increasing non-side-specific genes: 4858
Total number of common non-side-specific genes: 0

######### peaks #########

Total number of peaks_LEFT before filtering: 167061
Number of peaks_LEFT after cutoff and 'A' filtering: 4835
Number of LEFT-specific peaks: 71
Number of associated increasing LEFT-specific peaks: 0
Number of associated increasing non-LEFT-specific peaks: 4835
```

Example 120 (unknown):
```unknown
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:00<00:00, 11.14it/s]
```

Example 121 (unknown):
```unknown
Total number of peaks_RIGHT before filtering: 166063
Number of peaks_RIGHT after cutoff and 'A' filtering: 7171
Number of RIGHT-specific peaks: 19
Number of associated increasing RIGHT-specific peaks: 0
Number of associated increasing non-RIGHT-specific peaks: 7171
Total number of associated increasing non-side-specific peaks: 11469
Total number of common non-side-specific peaks: 0

######### AUCgene #########

Total number of AUCgene_LEFT before filtering: 278
Number of AUCgene_LEFT after cutoff and 'A' filtering: 83
Number of LEFT-specific AUCgene: 0
Number of associated increasing LEFT-specific AUCgene: 0
Number of associated increasing non-LEFT-specific AUCgene: 83
Total number of AUCgene_RIGHT before filtering: 278
Number of AUCgene_RIGHT after cutoff and 'A' filtering: 85
Number of RIGHT-specific AUCgene: 0
Number of associated increasing RIGHT-specific AUCgene: 0
Number of associated increasing non-RIGHT-specific AUCgene: 85
Total number of associated increasing non-side-specific AUCgene: 104
Total number of common non-side-specific AUCgene: 0

######### AUCregion #########

Total number of AUCregion_LEFT before filtering: 278
Number of AUCregion_LEFT after cutoff and 'A' filtering: 75
Number of LEFT-specific AUCregion: 0
Number of associated increasing LEFT-specific AUCregion: 0
Number of associated increasing non-LEFT-specific AUCregion: 75
Total number of AUCregion_RIGHT before filtering: 278
Number of AUCregion_RIGHT after cutoff and 'A' filtering: 72
Number of RIGHT-specific AUCregion: 0
Number of associated increasing RIGHT-specific AUCregion: 0
Number of associated increasing non-RIGHT-specific AUCregion: 72
Total number of associated increasing non-side-specific AUCregion: 99
Total number of common non-side-specific AUCregion: 0

Counts per category:
genes_LEFT: 50
genes_RIGHT: 42
genes_NON_SIDE_SPECIFIC: 4858
genes_COMMON: 0
peaks_LEFT: 0
peaks_RIGHT: 0
peaks_NON_SIDE_SPECIFIC: 11469
peaks_COMMON: 0
AUCgene_LEFT: 0
AUCgene_RIGHT: 0
AUCgene_NON_SIDE_SPECIFIC: 104
AUCgene_COMMON: 0
AUCregion_LEFT: 0
AUCregion_RIGHT: 0
AUCregion_NON_SIDE_SPECIFIC: 99
AUCregion_COMMON: 0
```

Example 122 (python):
```python
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import cross_val_score
from matplotlib import pyplot as plt
from matplotlib import cm
import numpy as np


def plot_fitted_expression(adata, gene_list, title, figsize, fname, show_colorbar=True):
    """
    Plots fitted gene expression over pseudotime for a list of genes.

    Parameters:
    adata (AnnData): Annotated data matrix of shape n_obs × n_vars. Rows correspond to cells and columns to genes.
    gene_list (list): List of genes to be plotted.
    title (str): Title of the plot.
    figsize (tuple): Figure size in the format (width, height).
    show_colorbar (bool): If True, display the color bar. Defaults to True.

    The function plots each gene's fitted expression as a function of pseudotime.
    The genes are ordered based on the pseudotime of their maximal expression.
    """
    pseudotime = adata.obs["t"].values  # Convert to numpy array
    pseudotime_reshaped = pseudotime.reshape(-1, 1)
    pseudotime_plot = np.linspace(min(pseudotime), max(pseudotime), 500)
    pseudotime_plot_reshaped = pseudotime_plot.reshape(-1, 1)
    gene_info = {}

    for gene in tqdm(gene_list, desc="Fitting expression..."):
        gene_expression = adata[:, gene].X.toarray()  # Adjust indexing based on your Anndata structure
        degrees = [1, 2, 3, 4, 5]
        best_score = -np.inf
        best_degree = 0

        for degree in degrees:
            model = make_pipeline(PolynomialFeatures(degree), LinearRegression())
            score = cross_val_score(model, pseudotime_reshaped, gene_expression, cv=5).mean()
            if score > best_score:
                best_score = score
                best_degree = degree

        best_model = make_pipeline(PolynomialFeatures(best_degree), LinearRegression())
        best_model.fit(pseudotime_reshaped, gene_expression)
        fitted_expression_plot = best_model.predict(pseudotime_plot_reshaped)
        max_expression_idx = np.argmax(fitted_expression_plot)
        max_expression_time = pseudotime_plot[max_expression_idx]
        gene_info[gene] = {'model': best_model, 'max_time': max_expression_time}

    sorted_genes = sorted(gene_list, key=lambda gene: gene_info[gene]['max_time'], reverse=True)

    fig, ax = plt.subplots(figsize=figsize)
    cmap = cm.RdPu

    for i, gene in enumerate(sorted_genes):
        best_model = gene_info[gene]['model']
        fitted_expression_plot = best_model.predict(pseudotime_plot_reshaped)
        norm = plt.Normalize(fitted_expression_plot.min(), fitted_expression_plot.max())
        sc = plt.scatter(pseudotime_plot, np.ones_like(pseudotime_plot) * (i + 1), 
                         c=fitted_expression_plot, cmap=cmap, norm=norm, marker='s')

    if show_colorbar:
        plt.colorbar(sc, label='Fitted Gene Expression')

    # Remove the box around the plot
    ax.spines['top'].set_visible(False)
    ax.spines['right'].set_visible(False)
    ax.spines['left'].set_visible(False)

    # Hide x-axis ticks and labels
    ax.set_xticks([])  # This removes the x-axis ticks
    ax.set_xlabel('')  # This removes the x-axis label

    # Adjust x-axis position
    ax.set_ylim(0.5, len(sorted_genes) + 0.5)

    plt.xlabel('Pseudotime')
    plt.ylabel('Gene')
    plt.yticks(np.arange(1, len(sorted_genes) + 1), sorted_genes, fontsize='xx-small', fontstyle='italic')
    plt.title(title)
    plt.savefig(fname=fname, bbox_inches='tight')
    plt.show()
```

Example 123 (python):
```python
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import cross_val_score
from matplotlib import pyplot as plt
from matplotlib import cm
import numpy as np


def plot_fitted_expression(adata, gene_list, title, figsize, fname, show_colorbar=True):
    """
    Plots fitted gene expression over pseudotime for a list of genes.

    Parameters:
    adata (AnnData): Annotated data matrix of shape n_obs × n_vars. Rows correspond to cells and columns to genes.
    gene_list (list): List of genes to be plotted.
    title (str): Title of the plot.
    figsize (tuple): Figure size in the format (width, height).
    show_colorbar (bool): If True, display the color bar. Defaults to True.

    The function plots each gene's fitted expression as a function of pseudotime.
    The genes are ordered based on the pseudotime of their maximal expression.
    """
    pseudotime = adata.obs["t"].values  # Convert to numpy array

    # Normalize pseudotime to [0, 1] range
    pseudotime_min = min(pseudotime)
    pseudotime_max = max(pseudotime)
    pseudotime_normalized = (pseudotime - pseudotime_min) / (pseudotime_max - pseudotime_min)

    # Reshape for model fitting
    pseudotime_reshaped = pseudotime_normalized.reshape(-1, 1)
    
    # Create equally spaced values in [0, 1] for plotting
    pseudotime_plot = np.linspace(0, 1, 500)
    pseudotime_plot_reshaped = pseudotime_plot.reshape(-1, 1)
    
    gene_info = {}

    for gene in tqdm(gene_list, desc="Fitting expression..."):
        gene_expression = adata[:, gene].X.toarray()  # Adjust indexing based on your Anndata structure
        degrees = [1, 2, 3, 4, 5]
        best_score = -np.inf
        best_degree = 0

        for degree in degrees:
            model = make_pipeline(PolynomialFeatures(degree), LinearRegression())
            score = cross_val_score(model, pseudotime_reshaped, gene_expression, cv=5).mean()
            if score > best_score:
                best_score = score
                best_degree = degree

        best_model = make_pipeline(PolynomialFeatures(best_degree), LinearRegression())
        best_model.fit(pseudotime_reshaped, gene_expression)
        fitted_expression_plot = best_model.predict(pseudotime_plot_reshaped)
        max_expression_idx = np.argmax(fitted_expression_plot)
        max_expression_time = pseudotime_plot[max_expression_idx]
        gene_info[gene] = {'model': best_model, 'max_time': max_expression_time}

    sorted_genes = sorted(gene_list, key=lambda gene: gene_info[gene]['max_time'], reverse=True)

    fig, ax = plt.subplots(figsize=figsize)
    cmap = cm.RdPu

    for i, gene in enumerate(sorted_genes):
        best_model = gene_info[gene]['model']
        fitted_expression_plot = best_model.predict(pseudotime_plot_reshaped)
        norm = plt.Normalize(fitted_expression_plot.min(), fitted_expression_plot.max())
        sc = plt.scatter(pseudotime_plot, np.ones_like(pseudotime_plot) * (i + 1), 
                         c=fitted_expression_plot, cmap=cmap, norm=norm, marker='s')

    if show_colorbar:
        plt.colorbar(sc, label='Fitted Gene Expression')

    # Remove the box around the plot
    ax.spines['top'].set_visible(False)
    ax.spines['right'].set_visible(False)
    ax.spines['left'].set_visible(False)

    # If you wish, hide x-axis ticks and labels by uncommenting
    #ax.set_xticks([])  # This removes the x-axis ticks
    #ax.set_xlabel('')  # This removes the x-axis label
    
    ax.set_xlim(0, 1)
    ax.set_ylim(0.5, len(sorted_genes) + 0.5)

    plt.xlabel('Pseudotime')
    plt.ylabel('Gene')
    plt.yticks(np.arange(1, len(sorted_genes) + 1), sorted_genes, fontsize='xx-small', fontstyle='italic')
    plt.title(title)
    plt.savefig(fname=fname, bbox_inches='tight')
    plt.show()
```

Example 124 (unknown):
```unknown
associated_increasing_features_dict.keys()
```

Example 125 (unknown):
```unknown
dict_keys(['genes_LEFT', 'genes_RIGHT', 'genes_NON_SIDE_SPECIFIC', 'genes_COMMON', 'peaks_LEFT', 'peaks_RIGHT', 'peaks_NON_SIDE_SPECIFIC', 'peaks_COMMON', 'AUCgene_LEFT', 'AUCgene_RIGHT', 'AUCgene_NON_SIDE_SPECIFIC', 'AUCgene_COMMON', 'AUCregion_LEFT', 'AUCregion_RIGHT', 'AUCregion_NON_SIDE_SPECIFIC', 'AUCregion_COMMON'])
```

Example 126 (python):
```python
figsize=(2,4.5)
top_n_features=28
file_extension=".png"
file_extension=".svg"

plot_fitted_expression(adata_gex[adata_gex.obs.fine_grain=="AtrialCardiomyocytesLeft"], gene_list=associated_features_dict["genes_LEFT"][:top_n_features], title="LEFT atrial cardiomyocyte maturation", figsize=figsize, show_colorbar=False,fname=f"genes_LEFT{file_extension}")
plot_fitted_expression(adata_gex[adata_gex.obs.fine_grain=="AtrialCardiomyocytesRight"], gene_list=associated_features_dict["genes_RIGHT"][:top_n_features], title="RIGHT atrial cardiomyocyte maturation", figsize=figsize, show_colorbar=False,fname=f"genes_RIGHT{file_extension}")

#top_n_features=80
#figsize=(2,10)

plot_fitted_expression(adata_gex, gene_list=associated_features_dict["genes_COMMON"][:top_n_features], title="COMMON atrial cardiomyocyte maturation", figsize=figsize, show_colorbar=False,fname=f"genes_COMMON{file_extension}")
```

Example 127 (unknown):
```unknown
Fitting expression...: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:02<00:00, 11.82it/s]
```

Example 128 (unknown):
```unknown
Fitting expression...: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:01<00:00, 17.55it/s]
```

Example 129 (unknown):
```unknown
Fitting expression...: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:01<00:00, 15.48it/s]
```

Example 130 (python):
```python
gene_list=associated_features_dict["genes_LEFT"]
adata_subset=adata_gex[adata_gex.obs.fine_grain=="AtrialCardiomyocytesLeft"].copy()
distance_threshold=4.5
```

Example 131 (python):
```python
import seaborn as sns
import pandas as pd
import numpy as np
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import PolynomialFeatures, MinMaxScaler
from sklearn.linear_model import LinearRegression
from scipy.spatial.distance import pdist
from scipy.cluster.hierarchy import linkage, fcluster
from tqdm import tqdm
import matplotlib.pyplot as plt

def create_clustermap(adata, gene_list, n_bins, figsize, desired_n_clusters):
    """
    Creates a clustermap of fitted gene expression across pseudotime bins for a list of genes.

    Parameters:
    adata (AnnData): Annotated data matrix.
    gene_list (list): List of genes to be included.
    n_bins (int): Number of bins to divide pseudotime into.
    figsize (tuple): Figure size in the format (width, height).
    """
    # Normalize and reshape pseudotime
    pseudotime = adata.obs["t"].values
    pseudotime_min = min(pseudotime)
    pseudotime_max = max(pseudotime)
    pseudotime_normalized = (pseudotime - pseudotime_min) / (pseudotime_max - pseudotime_min)
    pseudotime_reshaped = pseudotime_normalized.reshape(-1, 1)

    # Create pseudotime bins
    bins = np.linspace(0, 1, n_bins + 1)
    pseudotime_binned = np.digitize(pseudotime_normalized, bins) - 1

    # DataFrame to store fitted expressions
    expression_data = pd.DataFrame(index=gene_list, columns=range(n_bins))

    print(f"Creating clustermap for {len(gene_list)} features across {n_bins} bins")

    for gene in tqdm(gene_list, desc="Fitting expression..."):
        gene_expression = adata[:, gene].X.toarray()

        # Check if gene_expression is empty or almost empty
        if gene_expression.size == 0 or np.count_nonzero(gene_expression) < 5:
            print(f"Insufficient data for gene: {gene}")
            continue

        # Fit the model
        model = make_pipeline(PolynomialFeatures(2), LinearRegression())
        model.fit(pseudotime_reshaped, gene_expression)

        # Predict expression for each bin
        for i in range(n_bins):
            bin_range = np.linspace(bins[i], bins[i + 1], 10).reshape(-1, 1)
            predicted_expression = model.predict(bin_range).mean()
            expression_data.loc[gene, i] = predicted_expression  # Direct assignment
    
    expression_data = expression_data.astype(float).fillna(0)

    # Perform min-max normalization
    scaler = MinMaxScaler()
    expression_data_normalized = pd.DataFrame(scaler.fit_transform(expression_data.T).T, index=gene_list, columns=expression_data.columns) # row-wise (per gene) min-max normalisation


    # Compute the distance matrix for the rows (genes) and then the linkage matrix
    distance_matrix = pdist(expression_data_normalized)
    row_linkage = linkage(distance_matrix, method='average')

    # Define clusters using the row linkage
    clusters = fcluster(row_linkage, desired_n_clusters, criterion='maxclust')

    # Define clusters using a distance threshold
    clusters = fcluster(row_linkage, distance_threshold, criterion='distance')

    # Generate a color palette with seaborn and map cluster labels to colors
    cluster_colors = sns.color_palette("tab20", n_colors=len(np.unique(clusters)))
    row_colors = pd.Series(clusters, index=gene_list).map(dict(zip(np.unique(clusters), cluster_colors)))

    # Create clustermap with row colors
    g = sns.clustermap(expression_data_normalized, row_linkage=row_linkage, figsize=figsize, cmap='RdPu', yticklabels=True, xticklabels=True, col_cluster=False, row_colors=row_colors)
    g.cax.set_visible(False)
    
    plt.setp(g.ax_heatmap.get_yticklabels(), fontsize='xx-small', fontstyle='italic')
    plt.show()

    return row_linkage, clusters

row_linkage, clusters = create_clustermap(adata=adata_subset, gene_list=gene_list, n_bins=500, figsize=(4,30), desired_n_clusters=3)




from collections import defaultdict

def create_cluster_dict(clusters, gene_list):
    """
    Creates a dictionary mapping cluster names to lists of genes, adjusting for zero-based indexing.

    Parameters:
    clusters (array): Array of cluster assignments for each gene.
    gene_list (list): List of genes corresponding to the clusters.

    Returns:
    dict: Dictionary with cluster names as keys and lists of genes as values.
    """
    cluster_dict = defaultdict(list)
    for gene, cluster in zip(gene_list, clusters):
        #print(f"Gene: {gene}, Cluster: {cluster}")  # Diagnostic print
        cluster_name=f"cluster_{cluster - 1}"
        cluster_dict[cluster_name].append(gene)

    return dict(cluster_dict)

# Example usage
cluster_dict = create_cluster_dict(clusters, gene_list=gene_list)

# Check the clusters and the number of genes in each
for cluster, genes in cluster_dict.items():
    print(f"{cluster}: {len(genes)} genes")

cluster_dict.keys()
scores=[]
for cluster_name,cluster_genes in cluster_dict.items():
    score_name=f"{cluster_name}_({len(cluster_genes)} genes)"
    scores.append(score_name)
    sc.tl.score_genes(adata_gex,gene_list=cluster_genes,score_name=f"{score_name}")

sc.pl.umap(adata_gex, color=scores, ncols=4, wspace=0.8, cmap="RdPu")








import numpy as np
from sklearn.metrics import mean_squared_error
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model import LinearRegression
from sklearn.pipeline import make_pipeline

optimal_orders = {}

for score in scores:
    # Prepare the data
    tmp_df = adata_gex.obs[["t", score]]
    X = tmp_df["t"].values.reshape(-1, 1)
    y = tmp_df[score].values

    # Split the data into training and testing sets
    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

    best_order = 1
    best_mse = float('inf')

    # Test different polynomial orders
    for order in range(1, 6):
        model = make_pipeline(PolynomialFeatures(order), LinearRegression())
        model.fit(X_train, y_train)
        predictions = model.predict(X_test)
        mse = mean_squared_error(y_test, predictions)

        if mse < best_mse:
            best_mse = mse
            best_order = order

    optimal_orders[score] = best_order






import matplotlib.pyplot as plt
import seaborn as sns

# Number of scores (subplots)
num_scores = len(scores)

# Create a figure with a row of subplots
fig, axes = plt.subplots(1, num_scores, figsize=(5 * num_scores, 5), sharey=True)

# Track the global min and max for the y-axis
global_min, global_max = float('inf'), float('-inf')

# Plot each score
for i, score in enumerate(scores):
    # Prepare the data
    tmp_df = adata_gex.obs[["t", score, "fine_grain"]]
    
    # Plot for each fine_grain category
    fine_grain_categories = tmp_df["fine_grain"].unique()
    for category in fine_grain_categories:
        category_data = tmp_df[tmp_df["fine_grain"] == category]
        sns.regplot(ax=axes[i], data=category_data, x="t", y=score, order=optimal_orders[score], scatter=False, label=category)
    
    # Set the title and add legend
    axes[i].set_title(score)
    axes[i].legend()

    # Update global min and max
    current_min, current_max = tmp_df[score].min(), tmp_df[score].max()
    global_min, global_max = min(global_min, current_min), max(global_max, current_max)

# Set the same y-axis limits for all subplots
for ax in axes:
    ax.set_ylim(global_min, global_max)

# Adjust layout
plt.tight_layout()
plt.show()
```

Example 132 (unknown):
```unknown
Creating clustermap for 28 features across 500 bins
```

Example 133 (unknown):
```unknown
Fitting expression...: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:02<00:00, 10.55it/s]
```

Example 134 (unknown):
```unknown
cluster_3: 7 genes
cluster_2: 6 genes
cluster_4: 2 genes
cluster_6: 1 genes
cluster_0: 8 genes
cluster_5: 3 genes
cluster_1: 1 genes
```

Example 135 (unknown):
```unknown
cluster_dict["cluster_3"]
```

Example 136 (unknown):
```unknown
['MBP', 'KCNMB2', 'GAN', 'SLC8A3', 'A2M', 'KCNMB2-AS1', 'KSR1']
```

Example 137 (unknown):
```unknown
gene_set_A=cluster_dict["cluster_3"]+cluster_dict["cluster_4"]+cluster_dict["cluster_6"]
gene_set_A
```

Example 138 (unknown):
```unknown
['MBP',
 'KCNMB2',
 'GAN',
 'SLC8A3',
 'A2M',
 'KCNMB2-AS1',
 'KSR1',
 'PANCR',
 'PITX2',
 'DCBLD2']
```

Example 139 (python):
```python
gene_list=associated_features_dict["genes_RIGHT"]
adata_subset=adata_gex[adata_gex.obs.fine_grain=="AtrialCardiomyocytesRight"].copy()
distance_threshold=4.5
```

Example 140 (python):
```python
import seaborn as sns
import pandas as pd
import numpy as np
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import PolynomialFeatures, MinMaxScaler
from sklearn.linear_model import LinearRegression
from scipy.spatial.distance import pdist
from scipy.cluster.hierarchy import linkage, fcluster
from tqdm import tqdm
import matplotlib.pyplot as plt

def create_clustermap(adata, gene_list, n_bins, figsize, desired_n_clusters):
    """
    Creates a clustermap of fitted gene expression across pseudotime bins for a list of genes.

    Parameters:
    adata (AnnData): Annotated data matrix.
    gene_list (list): List of genes to be included.
    n_bins (int): Number of bins to divide pseudotime into.
    figsize (tuple): Figure size in the format (width, height).
    """
    # Normalize and reshape pseudotime
    pseudotime = adata.obs["t"].values
    pseudotime_min = min(pseudotime)
    pseudotime_max = max(pseudotime)
    pseudotime_normalized = (pseudotime - pseudotime_min) / (pseudotime_max - pseudotime_min)
    pseudotime_reshaped = pseudotime_normalized.reshape(-1, 1)

    # Create pseudotime bins
    bins = np.linspace(0, 1, n_bins + 1)
    pseudotime_binned = np.digitize(pseudotime_normalized, bins) - 1

    # DataFrame to store fitted expressions
    expression_data = pd.DataFrame(index=gene_list, columns=range(n_bins))

    print(f"Creating clustermap for {len(gene_list)} features across {n_bins} bins")

    for gene in tqdm(gene_list, desc="Fitting expression..."):
        gene_expression = adata[:, gene].X.toarray()

        # Check if gene_expression is empty or almost empty
        if gene_expression.size == 0 or np.count_nonzero(gene_expression) < 5:
            print(f"Insufficient data for gene: {gene}")
            continue

        # Fit the model
        model = make_pipeline(PolynomialFeatures(2), LinearRegression())
        model.fit(pseudotime_reshaped, gene_expression)

        # Predict expression for each bin
        for i in range(n_bins):
            bin_range = np.linspace(bins[i], bins[i + 1], 10).reshape(-1, 1)
            predicted_expression = model.predict(bin_range).mean()
            expression_data.loc[gene, i] = predicted_expression  # Direct assignment
    
    expression_data = expression_data.astype(float).fillna(0)

    # Perform min-max normalization
    scaler = MinMaxScaler()
    expression_data_normalized = pd.DataFrame(scaler.fit_transform(expression_data.T).T, index=gene_list, columns=expression_data.columns) # row-wise (per gene) min-max normalisation


    # Compute the distance matrix for the rows (genes) and then the linkage matrix
    distance_matrix = pdist(expression_data_normalized)
    row_linkage = linkage(distance_matrix, method='average')

    # Define clusters using the row linkage
    clusters = fcluster(row_linkage, desired_n_clusters, criterion='maxclust')

    # Define clusters using a distance threshold
    clusters = fcluster(row_linkage, distance_threshold, criterion='distance')

    # Generate a color palette with seaborn and map cluster labels to colors
    cluster_colors = sns.color_palette("tab20", n_colors=len(np.unique(clusters)))
    row_colors = pd.Series(clusters, index=gene_list).map(dict(zip(np.unique(clusters), cluster_colors)))

    # Create clustermap with row colors
    g = sns.clustermap(expression_data_normalized, row_linkage=row_linkage, figsize=figsize, cmap='RdPu', yticklabels=True, xticklabels=True, col_cluster=False, row_colors=row_colors)
    g.cax.set_visible(False)
    
    plt.setp(g.ax_heatmap.get_yticklabels(), fontsize='xx-small', fontstyle='italic')
    plt.show()

    return row_linkage, clusters

row_linkage, clusters = create_clustermap(adata=adata_subset, gene_list=gene_list, n_bins=500, figsize=(4,30), desired_n_clusters=3)




from collections import defaultdict

def create_cluster_dict(clusters, gene_list):
    """
    Creates a dictionary mapping cluster names to lists of genes, adjusting for zero-based indexing.

    Parameters:
    clusters (array): Array of cluster assignments for each gene.
    gene_list (list): List of genes corresponding to the clusters.

    Returns:
    dict: Dictionary with cluster names as keys and lists of genes as values.
    """
    cluster_dict = defaultdict(list)
    for gene, cluster in zip(gene_list, clusters):
        #print(f"Gene: {gene}, Cluster: {cluster}")  # Diagnostic print
        cluster_name=f"cluster_{cluster - 1}"
        cluster_dict[cluster_name].append(gene)

    return dict(cluster_dict)

# Example usage
cluster_dict = create_cluster_dict(clusters, gene_list=gene_list)

# Check the clusters and the number of genes in each
for cluster, genes in cluster_dict.items():
    print(f"{cluster}: {len(genes)} genes")

cluster_dict.keys()
scores=[]
for cluster_name,cluster_genes in cluster_dict.items():
    score_name=f"{cluster_name}_({len(cluster_genes)} genes)"
    scores.append(score_name)
    sc.tl.score_genes(adata_gex,gene_list=cluster_genes,score_name=f"{score_name}")

sc.pl.umap(adata_gex, color=scores, ncols=4, wspace=0.8, cmap="RdPu")








import numpy as np
from sklearn.metrics import mean_squared_error
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model import LinearRegression
from sklearn.pipeline import make_pipeline

optimal_orders = {}

for score in scores:
    # Prepare the data
    tmp_df = adata_gex.obs[["t", score]]
    X = tmp_df["t"].values.reshape(-1, 1)
    y = tmp_df[score].values

    # Split the data into training and testing sets
    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

    best_order = 1
    best_mse = float('inf')

    # Test different polynomial orders
    for order in range(1, 6):
        model = make_pipeline(PolynomialFeatures(order), LinearRegression())
        model.fit(X_train, y_train)
        predictions = model.predict(X_test)
        mse = mean_squared_error(y_test, predictions)

        if mse < best_mse:
            best_mse = mse
            best_order = order

    optimal_orders[score] = best_order






import matplotlib.pyplot as plt
import seaborn as sns

# Number of scores (subplots)
num_scores = len(scores)

# Create a figure with a row of subplots
fig, axes = plt.subplots(1, num_scores, figsize=(5 * num_scores, 5), sharey=True)

# Track the global min and max for the y-axis
global_min, global_max = float('inf'), float('-inf')

# Plot each score
for i, score in enumerate(scores):
    # Prepare the data
    tmp_df = adata_gex.obs[["t", score, "fine_grain"]]
    
    # Plot for each fine_grain category
    fine_grain_categories = tmp_df["fine_grain"].unique()
    for category in fine_grain_categories:
        category_data = tmp_df[tmp_df["fine_grain"] == category]
        sns.regplot(ax=axes[i], data=category_data, x="t", y=score, order=optimal_orders[score], scatter=False, label=category)
    
    # Set the title and add legend
    axes[i].set_title(score)
    axes[i].legend()

    # Update global min and max
    current_min, current_max = tmp_df[score].min(), tmp_df[score].max()
    global_min, global_max = min(global_min, current_min), max(global_max, current_max)

# Set the same y-axis limits for all subplots
for ax in axes:
    ax.set_ylim(global_min, global_max)

# Adjust layout
plt.tight_layout()
plt.show()
```

Example 141 (unknown):
```unknown
Creating clustermap for 29 features across 500 bins
```

Example 142 (unknown):
```unknown
Fitting expression...: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 29/29 [00:02<00:00, 10.81it/s]
```

Example 143 (unknown):
```unknown
cluster_1: 8 genes
cluster_4: 5 genes
cluster_2: 4 genes
cluster_3: 10 genes
cluster_0: 2 genes
```

Example 144 (unknown):
```unknown
gene_set_B=cluster_dict["cluster_2"]+cluster_dict["cluster_1"]
gene_set_B
```

Example 145 (unknown):
```unknown
['DAB1',
 'UST',
 'ANGPT1',
 'GNAL',
 'KIF26B',
 'KCNQ5',
 'HEY1',
 'HSPA6',
 'RFTN1',
 'MAF',
 'OTOGL',
 'PDE1A']
```

Example 146 (python):
```python
sc.pl.umap(adata_gex, color=["milestones","seg"], ncols=4, wspace=0.8)
```

Example 147 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 148 (python):
```python
import pandas as pd

# Assuming adata_gex is your AnnData object containing scRNAseq data
# and it has 't' and 'milestones' in adata_gex.obs

# Filter the rows where 'milestones' is '60'
filtered_data = adata_gex.obs[adata_gex.obs['milestones'] == '60']

# Find the maximum 't' value in this filtered group
bifurcation_point = filtered_data['t'].median()

# Output the bifurcation point
print("Bifurcation Point:", bifurcation_point)
```

Example 149 (unknown):
```unknown
Bifurcation Point: 1.9802904538320618
```

Example 150 (python):
```python
import numpy as np
import pandas as pd

# Assuming adata_gex is your AnnData object containing scRNAseq data
# and bifurcation_point is already defined as shown previously

# Define the number of bins you want
X = 4  # Adjust this value as needed

# Get the minimum 't' value from adata_gex
min_t_value = adata_gex.obs['t'].min()

# Create bins between the minimum 't' value and the bifurcation point
bins = np.linspace(min_t_value, bifurcation_point, X + 1)  # X+1 because we need X bins

# Assign each cell to a pseudotime bin or "post_bifurcation"
adata_gex.obs['pseudotime_bin'] = pd.cut(adata_gex.obs['t'], bins=bins, include_lowest=True, 
                                         labels=[f'pseudotime_bin_{i}' for i in range(1, X + 1)])
adata_gex.obs['pseudotime_bin'] = adata_gex.obs['pseudotime_bin'].cat.add_categories('post_bifurcation')
adata_gex.obs.loc[adata_gex.obs['t'] > bifurcation_point, 'pseudotime_bin'] = 'post_bifurcation'

# You can now work with the 'pseudotime_bin' column in adata_gex.obs for further analysis
sc.pl.umap(adata_gex, color=["pseudotime_bin"], ncols=1, wspace=0.8, size=50)
```

Example 151 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 152 (python):
```python
adata_gex.obs.fine_grain.value_counts()
```

Example 153 (unknown):
```unknown
AtrialCardiomyocytesLeft     3041
AtrialCardiomyocytesRight    2623
Name: fine_grain, dtype: int64
```

Example 154 (python):
```python
# Step 1: Copy values from 'fine_grain' to the new column
adata_gex.obs['AtrialCardiomyocytesImmature_annotation'] = adata_gex.obs['fine_grain']

# Add new category to the column
new_category = "AtrialCardiomyocytesImmature"
if new_category not in adata_gex.obs['AtrialCardiomyocytesImmature_annotation'].cat.categories:
    adata_gex.obs['AtrialCardiomyocytesImmature_annotation'] = adata_gex.obs['AtrialCardiomyocytesImmature_annotation'].cat.add_categories(new_category)

# Step 2: Modify values for cells not equal to "post_bifurcation" in 'pseudotime_bin'
adata_gex.obs.loc[adata_gex.obs['pseudotime_bin'] != 'post_bifurcation', 'AtrialCardiomyocytesImmature_annotation'] = new_category

# Verify the value counts
print(adata_gex.obs['AtrialCardiomyocytesImmature_annotation'].value_counts())
print("saving annotation for use with c2l...")
adata_gex.obs['AtrialCardiomyocytesImmature_annotation'].to_csv("AtrialCardiomyocytesImmature_annotation.csv")
sc.pl.umap(adata_gex,color="AtrialCardiomyocytesImmature_annotation")
```

Example 155 (unknown):
```unknown
AtrialCardiomyocytesLeft        2828
AtrialCardiomyocytesRight       2509
AtrialCardiomyocytesImmature     327
Name: AtrialCardiomyocytesImmature_annotation, dtype: int64
saving annotation for use with c2l...
```

Example 156 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 157 (python):
```python
adata_gex.X=adata_gex.layers["counts"].copy()
sc.pp.normalize_total(adata_gex, target_sum=1e4)
sc.pp.log1p(adata_gex)
```

Example 158 (python):
```python
WARNING: adata.X seems to be already log-transformed.
```

Example 159 (python):
```python
import matplotlib.pyplot as plt
import seaborn as sns
import pandas as pd
import scipy.stats
from scipy.sparse import issparse
from tqdm import tqdm
import scanpy as sc
import numpy as np

# Function to calculate the correlation matrix and scatter plot data
def calculate_correlation(bin_cells, gene_set_A, gene_set_B, gene_indices):
    combined_gene_set = gene_set_A + gene_set_B
    # Initialize an empty DataFrame for the correlation matrix
    correlation_matrix = pd.DataFrame(index=combined_gene_set, columns=combined_gene_set)
    scatter_data = []

    # Calculate correlations for each pair of genes
    for gene1 in tqdm(combined_gene_set):
        for gene2 in combined_gene_set:
            # Extract expression values for gene1 and gene2
            x_values = bin_cells.X[:, gene_indices[gene1]].toarray().flatten() if issparse(bin_cells.X) else bin_cells.X[:, gene_indices[gene1]].flatten()
            y_values = bin_cells.X[:, gene_indices[gene2]].toarray().flatten() if issparse(bin_cells.X) else bin_cells.X[:, gene_indices[gene2]].flatten()

            # Calculate Pearson correlation
            correlation, _ = scipy.stats.pearsonr(x_values, y_values)

            # Store the correlation in the matrix
            correlation_matrix.at[gene1, gene2] = correlation

    # Calculate mean correlation for each gene with gene_set_A and gene_set_B
    for gene in combined_gene_set:
        mean_corr_with_A = correlation_matrix.loc[gene, gene_set_A].mean()
        mean_corr_with_B = correlation_matrix.loc[gene, gene_set_B].mean()
        gene_set_label = 'Left aCM early module' if gene in gene_set_A else 'Right aCM early module'

        scatter_data.append({'gene': gene, 'mean_corr_with_A': mean_corr_with_A, 'mean_corr_with_B': mean_corr_with_B, 'gene_set': gene_set_label})

    scatter_df = pd.DataFrame(scatter_data)

    return scatter_df

# Function to add labels to the genes closest to the bottom right and top left of each scatter plot
def label_extreme_genes(scatter_df, ax):
    # Define bottom right and top left points
    bottom_right_point = (1, -1)
    top_left_point = (-1, 1)

    # Function to calculate Euclidean distance
    def euclidean_distance(point1, point2):
        return np.sqrt((point1[0] - point2[0])**2 + (point1[1] - point2[1])**2)

    # Find the gene closest to the bottom right
    scatter_df['distance_to_bottom_right'] = scatter_df.apply(lambda row: euclidean_distance((row['mean_corr_with_A'], row['mean_corr_with_B']), bottom_right_point), axis=1)
    bottom_right_gene = scatter_df.loc[scatter_df['distance_to_bottom_right'].idxmin()]

    # Find the gene closest to the top left
    scatter_df['distance_to_top_left'] = scatter_df.apply(lambda row: euclidean_distance((row['mean_corr_with_A'], row['mean_corr_with_B']), top_left_point), axis=1)
    top_left_gene = scatter_df.loc[scatter_df['distance_to_top_left'].idxmin()]

    # Add labels to the plot
    ax.text(bottom_right_gene['mean_corr_with_A'], bottom_right_gene['mean_corr_with_B'], bottom_right_gene['gene'], color='blue', ha='right')
    ax.text(top_left_gene['mean_corr_with_A'], top_left_gene['mean_corr_with_B'], top_left_gene['gene'], color='red', ha='left')

# Assuming gene_set_A and gene_set_B are lists of gene names
# Assuming adata_gex is your AnnData object containing scRNAseq data
gene_indices = {gene: adata_gex.var.index.get_loc(gene) for gene in set(gene_set_A + gene_set_B)}

# Get all unique categories from 'pseudotime_bin' and move 'post_bifurcation' to the end
bin_categories = [cat for cat in adata_gex.obs['pseudotime_bin'].cat.categories if cat != 'post_bifurcation']
bin_categories.append('post_bifurcation')

# Number of bins to create subplots for
num_bins = len(adata_gex.obs['pseudotime_bin'].cat.categories)

# Create a figure with two rows of subplots: one for UMAP and one for scatter plots
fig, axes = plt.subplots(2, num_bins, figsize=(6 * num_bins, 12))

# Loop over each pseudotime bin to create UMAP and scatter plots
for i, bin_label in enumerate(adata_gex.obs['pseudotime_bin'].cat.categories):
    # Skip the post_bifurcation bin if needed
    #if bin_label == 'post_bifurcation':
    #    continue

    # Filter cells in the current bin for scatter plot
    bin_cells = adata_gex[adata_gex.obs['pseudotime_bin'] == bin_label]

    # Calculate scatter plot data
    scatter_df = calculate_correlation(bin_cells, gene_set_A, gene_set_B, gene_indices)

    # Create UMAP plot in the first row
    ax_umap = axes[0, i]
    sc.pl.umap(adata_gex, color="pseudotime_bin", groups=[bin_label], ax=ax_umap, show=False, legend_loc=None, title="", size=50)

    # Create specific scatter plot in the second row
    ax_scatter = axes[1, i]
    sns.scatterplot(data=scatter_df, x='mean_corr_with_A', y='mean_corr_with_B', hue='gene_set', ax=ax_scatter)
    ax_scatter.set_title(f'{bin_label}')
    ax_scatter.axhline(0, color='grey', linestyle='--')
    ax_scatter.axvline(0, color='grey', linestyle='--')
    ax_scatter.set_xlabel('Correlation with Left aCM early module')
    ax_scatter.set_ylabel('Correlation with Right aCM early module')
    ax_scatter.set_xlim(-1, 1)
    ax_scatter.set_ylim(-1, 1)
    ax_scatter.legend(title='Gene Module')

    # Label the extreme genes
    label_extreme_genes(scatter_df, ax_scatter)

plt.tight_layout()
plt.show()
```

Example 160 (unknown):
```unknown
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 22/22 [00:01<00:00, 18.84it/s]
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
  0%|                                                                                                                                                                                                                    | 0/22 [00:00<?, ?it/s]/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scipy/stats/_stats_py.py:4781: ConstantInputWarning: An input array is constant; the correlation coefficient is not defined.
  warnings.warn(stats.ConstantInputWarning(msg))
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 22/22 [00:01<00:00, 20.25it/s]
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
  0%|                                                                                                                                                                                                                    | 0/22 [00:00<?, ?it/s]/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scipy/stats/_stats_py.py:4781: ConstantInputWarning: An input array is constant; the correlation coefficient is not defined.
  warnings.warn(stats.ConstantInputWarning(msg))
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 22/22 [00:01<00:00, 21.55it/s]
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
  0%|                                                                                                                                                                                                                    | 0/22 [00:00<?, ?it/s]/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scipy/stats/_stats_py.py:4781: ConstantInputWarning: An input array is constant; the correlation coefficient is not defined.
  warnings.warn(stats.ConstantInputWarning(msg))
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 22/22 [00:01<00:00, 21.89it/s]
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 22/22 [00:48<00:00,  2.20s/it]
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 161 (unknown):
```unknown
early_modules_dict={}
early_modules_dict["LEFT"]=gene_set_A
early_modules_dict["RIGHT"]=gene_set_B
```

Example 162 (python):
```python
import pickle

key="aCM_n570" # or (aCM_n1140 or aCM_full)
key="aCM_n1140"
key="aCM_full"
target_type="Gene"
# File path
file_path = f"/nfs/team205/heart/pyscenic/plus/eRegulon_signatures/{key}_{target_type}_eRegulon_signatures.pkl"

# Open the file and load the dictionary
with open(file_path, 'rb') as file:
    TF_TG_dict = pickle.load(file)
```

Example 163 (unknown):
```unknown
results_list=[]
```

Example 164 (python):
```python
side="LEFT"
associated_features_dict.keys()
total_transcriptome_genes=36000
genes_of_interest=early_modules_dict[f"{side}"]

import pandas as pd
from scipy.stats import fisher_exact


# Initialize a list to store the results
results = []

for regulon, regulon_genes in TF_TG_dict.items():
    # Contingency table
    overlap = len(set(regulon_genes) & set(genes_of_interest))
    only_genes_of_interest = len(genes_of_interest) - overlap
    only_regulon_genes = len(regulon_genes) - overlap
    neither = total_transcriptome_genes - (overlap + only_genes_of_interest + only_regulon_genes)
    table = [[overlap, only_genes_of_interest], [only_regulon_genes, neither]]

    # Perform Fisher's Exact Test
    odds_ratio, p_value = fisher_exact(table)
    
    # Append the results
    results.append({"Regulon": regulon, "Odds Ratio": odds_ratio, "pval": p_value})

# Convert the list of dictionaries to a DataFrame
results_df = pd.DataFrame(results)

# Sort the DataFrame by p-value
results_df.sort_values(by="pval", inplace=True)

# Reset index after sorting
results_df.reset_index(drop=True, inplace=True)

# Display the DataFrame
results_df=results_df[results_df.pval<0.01]
results_df["side"]=side
results_df=results_df.reset_index(names='rank')
results_list.append(results_df)
results_df
```

Example 165 (python):
```python
side="RIGHT"
associated_features_dict.keys()
total_transcriptome_genes=36000
genes_of_interest=early_modules_dict[f"{side}"]

import pandas as pd
from scipy.stats import fisher_exact


# Initialize a list to store the results
results = []

for regulon, regulon_genes in TF_TG_dict.items():
    # Contingency table
    overlap = len(set(regulon_genes) & set(genes_of_interest))
    only_genes_of_interest = len(genes_of_interest) - overlap
    only_regulon_genes = len(regulon_genes) - overlap
    neither = total_transcriptome_genes - (overlap + only_genes_of_interest + only_regulon_genes)
    table = [[overlap, only_genes_of_interest], [only_regulon_genes, neither]]

    # Perform Fisher's Exact Test
    odds_ratio, p_value = fisher_exact(table)
    
    # Append the results
    results.append({"Regulon": regulon, "Odds Ratio": odds_ratio, "pval": p_value})

# Convert the list of dictionaries to a DataFrame
results_df = pd.DataFrame(results)

# Sort the DataFrame by p-value
results_df.sort_values(by="pval", inplace=True)

# Reset index after sorting
results_df.reset_index(drop=True, inplace=True)

# Display the DataFrame
results_df=results_df[results_df.pval<0.01]
results_df["side"]=side
results_df=results_df.reset_index(names='rank')
results_list.append(results_df)
results_df
```

Example 166 (python):
```python
results_concat=pd.concat(results_list)
print(results_concat.shape)
results_concat["TF"]=results_concat.Regulon.str.split("_",expand=True)[0]
results_concat
```

Example 167 (python):
```python
chdgene_table=pd.read_csv("chdgene_table.csv")
results_concat=results_concat.merge(chdgene_table, left_on="TF",right_on="Gene", how='left')
results_concat
```

Example 168 (python):
```python
import matplotlib
matplotlib.rcParams['pdf.fonttype'] = 42
matplotlib.rcParams['ps.fonttype'] = 42
```

Example 169 (python):
```python
# Transforming p-values to -log10(p)
results_concat['-log10(pval)'] = -np.log10(results_concat['pval'])

# Create a pivot table for the heatmap
heatmap_data = results_concat.pivot("TF", "side", "-log10(pval)")
heatmap_data = heatmap_data.fillna(value=0)
heatmap_data = heatmap_data.sort_values(by=["LEFT","RIGHT"], ascending=False)

# Initialize with empty strings
annotations = pd.DataFrame('', index=heatmap_data.index, columns=heatmap_data.columns)

# Loop through the original dataframe to populate the annotations
for _, row in results_concat.iterrows():
    tf = row['TF']
    side = row['side']
    chd_classification = row['CHD classification']
    if pd.notna(chd_classification):
        if "ASD" in chd_classification:
            annotations.loc[tf, side] = '**'  # Two asterisks if "ASD" in CHD classification
        else:
            annotations.loc[tf, side] = '*'   # One asterisk otherwise

# Create the heatmap
plt.figure(figsize=(10, 10))
sns.heatmap(heatmap_data, annot=annotations, fmt="s", cmap="RdPu", square=True, cbar=True)
plt.title("Enrichment of early gene modules for regulons\n '*': associated with any CHD, '**' associated with ASD\ncolor scale = -log10(p-value)")
plt.xlabel("Side")
plt.ylabel("Regulon (Transcription Factor)")
plt.savefig("CHD_enrichment.pdf", format='pdf', bbox_inches='tight')
plt.show()
```

Example 170 (unknown):
```unknown
/tmp/ipykernel_615173/3468513932.py:5: FutureWarning: In a future version of pandas all arguments of DataFrame.pivot will be keyword-only.
  heatmap_data = results_concat.pivot("TF", "side", "-log10(pval)")
```

Example 171 (unknown):
```unknown
[reg for reg in list(TF_TG_dict.keys()) if "PITX" in reg]
```

Example 172 (unknown):
```unknown
['PITX2_+_(130g)']
```

Example 173 (unknown):
```unknown
plot.GO_plot(gene_list=TF_TG_dict["PITX2_+_(130g)"],display_table=True,gene_sets=['GO_Biological_Process_2023'])
```

Example 174 (unknown):
```unknown
GO_Biological_Process_2023
```

Example 175 (python):
```python
sc.pl.dotplot(adata_gex, groupby="AtrialCardiomyocytesImmature_annotation",var_names=TF_TG_dict["PITX2_+_(130g)"])
```

Example 176 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omni_env/lib/python3.11/site-packages/scanpy/plotting/_dotplot.py:749: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap', 'norm' will be ignored
  dot_ax.scatter(x, y, **kwds)
```

---

## Investigate Immune cluster¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/RNA/scnRNA_Integration-Annotation/T21/T21HeartsFurtherQC.html

**Contents:**
- Investigate Immune cluster¶
- Investigate epicardium¶
- Investigate endothelium¶
- Investigate neural¶
- Investigate mesenchymal¶
- Investigate cardiomyocytes¶
- Investigate the overall object with the other cells removed¶
- Process the dataset with all contaminating cells removed, add metadata information¶

**Examples:**

Example 1 (python):
```python
# Import packages, initial settings

import numpy as np
import matplotlib.pyplot as plt
import scanpy as sc
import pandas as pd
import anndata
import scrublet as scr
import seaborn as sns
import magic
import pybiomart
import six
import sys
sys.modules['sklearn.externals.six'] = six
import scanorama
import harmonypy
import bbknn
import gseapy
from gseapy.plot import barplot, dotplot
import scvelo as scv
import scvi

plt.rcParams['pdf.fonttype'] = 42
plt.rcParams['ps.fonttype'] = 42

sc.settings.verbosity = 3  # verbosity: errors (0), warnings (1), info (2), hints (3)

from matplotlib.colors import LinearSegmentedColormap
cmap = LinearSegmentedColormap.from_list(name='gene_cmap', colors=['lightgrey', 'thistle', 'red', 'darkred']) 

sc.settings.set_figure_params(dpi=80, color_map='viridis', vector_friendly=False,  dpi_save=300, facecolor='white')
sc.logging.print_version_and_date()
#sc.logging.print_versions()
```

Example 2 (python):
```python
adata=sc.read('./Objects/T21_multiome_RNA_foetal.h5ad')
print(adata.shape)
```

Example 3 (python):
```python
# set gene name as var_names 
adata.var['gene_id']=adata.var_names.copy()
adata.var.set_index('gene_name',inplace=True)
adata.var_names=adata.var_names.astype('str')
adata.var_names_make_unique()
```

Example 4 (python):
```python
cell_cycle_genes = ['MCM5','PCNA','TYMS','FEN1','MCM2','MCM4','RRM1','UNG','GINS2','MCM6','CDCA7','DTL','PRIM1','UHRF1','MLF1IP','HELLS','RFC2','RPA2','NASP','RAD51AP1','GMNN','WDR76','SLBP','CCNE2','UBR7','POLD3','MSH2','ATAD2','RAD51','RRM2','CDC45','CDC6','EXO1','TIPIN','DSCC1','BLM','CASP8AP2','USP1','CLSPN','POLA1','CHAF1B','BRIP1','E2F8','HMGB2','CDK1','NUSAP1','UBE2C','BIRC5','TPX2','TOP2A','NDC80','CKS2','NUF2','CKS1B','MKI67','TMPO','CENPF','TACC3','FAM64A','SMC4','CCNB2','CKAP2L','CKAP2','AURKB','BUB1','KIF11','ANP32E','TUBB4B','GTSE1','KIF20B','HJURP','CDCA3','HN1','CDC20','TTK','CDC25C','KIF2C','RANGAP1','NCAPD2','DLGAP5','CDCA2','CDCA8','ECT2','KIF23','HMMR','AURKA','PSRC1','ANLN','LBR','CKAP5','CENPE','CTCF','NEK2','G2E3','GAS2L3','CBX5','CENPA']
s_genes = cell_cycle_genes[:43]
s_genes = [x for x in s_genes if x in adata.var_names]
print('S genes:', s_genes, '\n')
g2m_genes = cell_cycle_genes[43:]
g2m_genes = [x for x in g2m_genes if x in adata.var_names]
print('G2 genes:', g2m_genes, '\n')
cell_cycle_genes = [x for x in cell_cycle_genes if x in adata.var_names]

hb_genes1 = ['HBB','HBD','HBG1','HBG2','HBE1','HBZ','HBM','HBA2','HBA1', 'HBQ1']
hb_genes1 = [x for x in hb_genes1 if x in adata.var_names]
print('Hemoglobin genes set 1:', hb_genes1, '\n')

hb_genes2 = ['AHSP','GYPA','RHAG','GFI1B','KEL','EPB42','ALAS2','TSPO2','GYPB','GYPE','SPTA1','RHCE','KLF1','RHD','SLC4A1','SNX22','SQSTM1','SOD1','FTH1','MAP1LC3B','SLC25A37','TRIM58','HMBS']
hb_genes2 = [x for x in hb_genes2 if x in adata.var_names]
print('Hemoglobin genes set 2:', hb_genes2, '\n')

mito_genes=['MT-ND1','MT-ND2','MT-CO1','MT-CO2','MT-ATP8','MT-ATP6','MT-CO3','MT-ND3','MT-ND4L', 'MT-ND4', 'MT-ND5', 'MT-ND6', 'MT-CYB']
mito_genes = [x for x in mito_genes if x in adata.var_names]
print('Mitochondrial genes:', mito_genes, '\n')

ribo_genes = ['RPLP0','RPLP1','RPLP2','RPL3','RPL3L','RPL4','RPL5','RPL6','RPL7','RPL7A','RPL7L1','RPL8','RPL9','RPL10','RPL10A','RPL10L','RPL11','RPL12','RPL13','RPL13A','RPL14','RPL15','RPL17','RPL18','RPL18A','RPL19','RPL21','RPL22','RPL22L1','RPL23','RPL23A','RPL24','RPL26','RPL26L1','RPL27','RPL27A','RPL28','RPL29','RPL30','RPL31','RPL32','RPL34','RPL35','RPL35A','RPL36','RPL36A','RPL36AL','RPL37','RPL37A','RPL38','RPL39','RPL39L','UBA52','RPL41','RPSA','RPS2','RPS3','RPS3A','RPS4X','RPS4Y1','RPS4Y2','RPS5','RPS6','RPS7','RPS8','RPS9','RPS10','RPS11','RPS12','RPS13','RPS14','RPS15','RPS15A','RPS16','RPS17','RPS18','RPS19','RPS20','RPS21','RPS23','RPS24','RPS25','RPS26','RPS27','RPS27A','RPS27L','RPS28','RPS29','FAU']
ribo_genes = [x for x in ribo_genes if x in adata.var_names]
print('Ribo genes:', ribo_genes, '\n')

stress_genes=['FOS','FOSB','JUN','JUNB','JUND','ZFP36','SOCS3','CEBPD','CEBPB','ATF3','EGR1','HSPA1A','HSPA1B','HSP90AA1','HSP90AB1','HSPA8','HSPB1','HSPE1','IER3','IER2','BTG1','BTG2','DUSP1']
stress_genes = [x for x in stress_genes if x in adata.var_names]
print('Tissue dissociation stress associated genes:', stress_genes, '\n')

FoxProteins=['FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1']
chrY_genes=['SRY', 'RPS4Y1', 'ZFY', 'ZFY-AS1', 'LINC00278', 'PCDH11Y', 'TTTY2B', 'TBL1Y', 'PRKY', 'USP9Y', 'DDX3Y', 'UTY', 'TMSB4Y', 'NLGN4Y', 'TTTY14', 'KDM5D', 'TTTY10', 'EIF1AY', 'PRORY', 'LINC00266-4P']
MembraneProteins1=['AADAT','AAK1','ABCA12','ABCA7','ABCB1','ABCB11','ABCB4','ABCB6','ABCC1','ABCC3','ABCC4','ABCC5','ABCC6','ABCG2','ABHD3','ABI1','ABLIM3','AC004691.2','AC004922.1','AC005324.3','AC005726.1','AC008403.1','AC138696.1','ACIN1','ACKR3','ACLY','ACOT13','ACOT9','ACTN1','ACTR10','ACTR6','ADA','ADAM10','ADAM12','ADAM22','ADAM28','ADAM33','ADAMTS1','ADAMTS2','ADAMTSL4','ADAP1','ADCK5','ADD1','ADD2','ADGRB2','ADGRD1','ADGRG2','ADGRG7','ADGRL2','ADH1A','ADH1B','ADH1C','ADH7','ADK','ADORA1','ADRA1B','ADRM1','ADSS','AFAP1L2','AFDN','AFG1L','AGER','AGRN','AHNAK','AHNAK2','AIFM2','AKAP10','AKAP11','AKAP12','AKR1B10','AKR1B15','AL132780.3','AL139300.1','AL355315.1','AL358113.1','AL645922.1','ALDH3A1','ALK','ALOX15B','ALOXE3','ALPI','ALPP','ALPPL2','ALS2CR12','AMACR','AMDHD1','AMER1','AMER2','AMIGO2','AMOT','AMOTL1','AMOTL2','AMPH','ANK2','ANK3','ANKEF1','ANKFN1','ANKIB1','ANKLE2','ANKRA2','ANKRD13A','ANKRD35','ANKRD42','ANKRD45','ANKS1B','ANO1','ANO10','ANO2','ANO6','ANPEP','ANXA1','ANXA13','ANXA2','ANXA3','AP2M1','APBB1','APBB1IP','APC','APLP1','APMAP','APOC3','APOD','APOLD1','AQP1','AQP10','AQP3','AQP4','AQP5','ARAP1','ARF1','ARHGAP18','ARHGAP19','ARHGAP21','ARHGAP22','ARHGAP23','ARHGAP24','ARHGAP28','ARHGAP29','ARHGAP33','ARHGAP36','ARHGAP40','ARHGAP9','ARHGDIG','ARHGEF1','ARHGEF11','ARHGEF12','ARHGEF15','ARHGEF26','ARHGEF28','ARHGEF40','ARHGEF5','ARID2','ARID4A','ARID5B','ARL13A','ARL15','ARL4C','ARMC12','ARMCX5','ARPC1A','ARPC5','ARRB2','ARRDC4','ARVCF','ASAP1','ASB5','ASGR1','ASGR2','ASH2L','ASIC1','ASIC4','ASPM','ASPSCR1','ATG13','ATG3','ATG7','ATIC','ATOX1','ATP10D','ATP12A','ATP13A3','ATP1B3','ATP23','ATP2B1','ATP2B4','ATP2C2','ATP6V0A2','ATP6V1H','ATP8A2','ATP8B1','ATXN10','ATXN3','ATXN7L3','AVEN','AVPI1','AXIN2','AXL','B2M','B3GALT2','BACE1','BAG3','BAIAP2','BAIAP2L1','BANK1','BASP1','BBOF1','BCAR1','BCAR3','BCL2L12','BCORL1','BCR','BDKRB2','BEND6','BFSP1','BFSP2','BICD2','BIN2','BLNK','BLVRB','BMPR2','BMX','BOC','BOLL','BORCS5','BORCS8-MEF2B','BRK1','BRPF1','BRSK1','BSDC1','BST2','BTK','BTN2A1','BTRC','BUB1B-PAK6','BVES','BZW2','C10orf90','C11orf42','C11orf52','C12orf73','C12orf76','C14orf37','C14orf93','C16orf74','C17orf58','C17orf62','C18orf21','C19orf33','C19orf38','C19orf48','C19orf71','C1orf116','C1orf189','C1orf35','C1orf54','C1orf68','C1orf74','C1QBP','C2orf72','C3orf14','C3orf20','C3orf38','C4orf19','C4orf22','C7orf43','C8orf33','C8orf37','C9orf153','C9orf3','CA14','CA9','CABLES2','CACNA1E','CACNG4','CADM1','CALCRL','CALD1','CALML5','CALN1','CAMK2A','CAMK2B','CAMK2D','CAMK2G','CAP2','CAPS','CASKIN2','CASP4','CAST','CAV3','CAVIN1','CAVIN2','CAVIN3','CAVIN4','CBWD1','CBWD2','CBWD3','CBWD5','CBWD6','CC2D1A','CCBE1','CCDC107','CCDC112','CCDC115','CCDC124','CCDC136','CCDC144NL','CCDC15','CCDC154','CCDC170','CCDC24','CCDC28A','CCDC59','CCDC62','CCDC66','CCDC69','CCDC8','CCDC83','CCDC85C','CCDC93','CCDC97','CCNT2','CCNYL1','CCR3','CCR4','CCT3','CD109','CD163','CD164L2','CD200R1','CD226','CD248','CD274','CD2AP','CD300A','CD300LB','CD33','CD34','CD37','CD38','CD3E','CD3G','CD4','CD44','CD46','CD47','CD48','CD79B','CD81','CD84','CD86','CD8A','CD8B','CD9','CD93','CD99L2','CDAN1','CDC42EP1','CDC42EP3','CDC42EP4','CDC42SE1','CDCA4','CDH1','CDH13','CDH15','CDH17','CDH2','CDH26','CDH3','CDH4','CDH5','CDH6','CDH8','CDIPT','CDK16','CDK5','CDK5RAP2','CDK7','CDRT15','CDV3','CEACAM5','CELSR1','CENPJ','CEP162','CEP295','CEP55','CEP57L1','CEP89','CERCAM','CFAP157','CFAP206','CFAP58','CFAP65','CFB','CFL2','CFLAR','CGN','CGNL1','CHCHD7','CHIC2','CHMP2A','CHRM3','CHRNA1','CHRNA5','CHRNB4','CHRND','CIB1','CIB3','CIP2A','CKAP5','CLCA2','CLCA4','CLCN5','CLCN6','CLDN11','CLDN18','CLDN2','CLDN23','CLDN3','CLDN4','CLDN7','CLDN9','CLEC2A','CLEC4F','CLEC5A','CLECL1','CLIC2','CLIC5','CLTB','CLUL1','CMTM4','CMTM7','CMTR2','CMYA5','CNGA1','CNKSR2','CNNM1','CNNM4','CNP','CNR1','CNTD1','COBL','COBLL1','COG3','COL16A1','COL17A1','COL4A5','COL6A2','COLEC12','COLQ','COMMD4','COMTD1','COPRS','COQ7','CORIN','CORO1B','CORO2A','CORO2B','COX6B2','CPEB3','CPPED1','CPZ','CRB3','CREB5','CRIP1','CRIP2','CRK','CRLF3','CROCC','CRTC1','CRYAB','CRYBG3','CRYL1','CSDE1','CSF1','CSF1R','CSPG4','CST6','CTNNA1','CTNNB1','CTNNBIP1','CTNND1','CTNND2','CTSF','CTSV','CTTN','CTXN1','CUEDC1','CUTA','CX3CL1','CX3CR1','CXADR','CXCR2','CYB5D2','CYBRD1','CYFIP2','CYP20A1','CYP24A1','CYTH2','CYTH4','DAAM1','DAB2','DAG1','DAGLB','DAW1','DBN1','DCAF13','DCAF16','DCBLD2','DCHS1','DCHS2','DCP2','DCT','DCTN6','DCUN1D2','DDAH1','DDB2','DDIAS','DDIT4L','DDR1','DDR2','DDX58','DDX60L','DENND6B','DFFA','DGKG','DGKH','DHDDS','DHPS','DHRS7B','DIAPH1','DIAPH3','DIO2','DIS3L','DLG1','DLG2','DLG5','DLGAP3','DLL3','DMWD','DNAAF1','DNAAF4','DNAH17','DNAJA4','DNAJB13','DNAJB4','DNAJC18','DNAJC5','DNAJC6','DNAJC9','DOC2A','DOCK4','DOK3','DONSON','DPEP1','DPH1','DPPA5','DPYSL2','DQX1','DRD1','DRP2','DSC3','DSEL','DSG2','DSP','DTNA','DUS1L','DUSP15','DXO','DYSF','ECSCR','EDA2R','EDNRA','EDNRB','EEF1AKMT1','EEF2','EFCAB1','EFCAB11','EFCAB2','EFCAB7','EFHC2','EFNB1','EFR3A','EFR3B','EGFR','EHBP1','EHD1','EHD2','EHD3','EHD4','EIF2B2','EIF5','EIF5B','ELFN1','ELMO3','ELN','ELOC','EMP3','ENAH','ENC1','ENG','ENKD1','ENO1','ENO2','ENO3','ENOX1','ENOX2','EPB41','EPB41L1','EPB41L2','EPB41L3','EPB41L4A','EPB41L4B','EPB41L5','EPCAM','EPHA2','EPHA3','EPHB1','EPHB2','EPN1','EPOR','EPS8L2','ERAP1','ERBB2','ERBB3','ERBIN','ERC1','ERCC4','ERMN','ESCO2','ETNK1','ETS2','EVA1A','EVI2B','EXOC1','EXOC3L1','EXOC7','EZR','F11R','F2RL1','F2RL3','FAAP20','FABP2','FABP3','FABP5','FAIM','FAM126A','FAM126B','FAM129A','FAM129B','FAM13A','FAM163B','FAM171A1','FAM193A','FAM205A','FAM210B','FAM222A','FAM45A','FAM47E','FAM47E-STBD1','FAM72A','FAM72B','FAM72C','FAM72D','FANCG','FAS','FASN','FBLIM1','FBLN2','FBLN5','FBLN7','FBP2','FBRSL1','FBXL2','FBXO16','FBXO38','FBXO41','FBXW11','FCAR','FCGBP','FCGR1A','FCGR1B','FCGR2A','FCHSD2','FFAR4','FGD1','FGD5','FGFR2','FGR','FHIT','FHL1','FIBCD1','FIGN','FILIP1','FILIP1L','FKBP2','FLAD1','FLCN','FLNA','FLNB','FLNC','FLOT1','FLOT2','FLRT3','FLT1','FLT4','FLVCR1','FMN2','FMNL2','FMNL3','FNBP1','FOXA2','FOXN3','FP565260.3','FRMD3','FRMD7','FRMD8','FRS2','FSCN1','FTCD','FUT2','FXYD3','FYB1','FYN','FZD1','FZD2','FZD4','FZD6','GAB1','GAB2','GABRA1','GABRA2','GABRA3','GABRA5','GABRG1','GAD1','GADL1','GAGE1','GAGE10','GAGE12C','GAGE12D','GAGE12E','GAGE12F','GAGE12G','GAGE12H','GAGE12J','GAGE13','GAGE2A','GAGE2E','GALNT10','GALR1','GAP43','GAPDH','GAPT','GAPVD1','GAS2','GAS2L1','GAS7','GAS8','GBP4','GCA','GCC1','GCLM','GCSAML-AS1','GFI1B','GFOD1','GFRA3','GHR','GID8','GINM1','GIPC1','GJA1','GJB3','GJB4','GJB6','GJB7','GJC1','GJC2','GLO1','GLP2R','GLRX','GLUL','GMIP','GNAI2','GNAQ','GNAS','GNB1','GNB2','GNB3','GNB4','GNG2','GP9','GPAT3','GPATCH1','GPBP1','GPBP1L1','GPC1','GPC3','GPC4','GPI','GPR1','GPR142','GPR156','GPR157','GPR160','GPR173','GPR20','GPR27','GPR50','GPR63','GPRC5A','GPRIN1','GPRIN2','GPX8','GRAMD1A','GRAMD1B','GRAMD1C','GRAP2','GRB7','GRID2IP','GRIK5','GRK3','GRK5','GRPR','GSDMA','GTF2F1','GTSE1','GUCA1A','GYPA','GYPC','HAPLN3','HARBI1','HAS1','HAUS4','HAUS7','HCAR2','HCAR3','HCFC2','HCK','HCLS1','HCST','HDAC8','HEG1','HELZ','HENMT1','HERC2','HERPUD1','HFE','HIF1A','HIF3A','HINT1','HIP1R','HLA-A','HLA-DPB1','HMGCS1','HMOX1','HOMER3','HOXC5','HOXD4','HPS3','HRASLS5','HRH1','HS6ST3','HSH2D','HSPB1','HSPG2','HTR1B','HTR7','HTRA1','HUNK','HYDIN','HYI','HYLS1','ICAM1','ICAM2','ICOS','ICOSLG','IFFO2','IFIT5','IFNGR1','IFRD1','IFT122','IGFL2','IGSF11','IGSF23','IKZF5','IL10RA','IL12RB1','IL12RB2','IL13','IL13RA2','IL16','IL17RB','IL1R1','IL1RL1','IL21R','IL4R','IL6ST','IL7R','ILDR2','ILK','INPP5A','INSR','INTS12','IP6K2','IPCEF1','IQGAP1','IQGAP2','IREB2','IRF2','IRS1','ISLR','ITGA2B','ITGA3','ITGA4','ITGA7','ITGA8','ITGA9','ITGB1','ITGB2','ITGB3','ITGB4','ITGB5','ITGB6','ITGB7','ITGB8','ITK','ITM2A','ITPKA','ITSN1','ITSN2','IYD','JAG1','JAK2','JAK3','JAM2','JAML','JCAD','JMJD4','JPT2','JUP','KANK1','KANK3','KANSL2','KARS','KATNB1','KCNAB1','KCNB2','KCNH8','KCNJ15','KCNK13','KCNK17','KCNK6','KCNK7','KCNK9','KCNN2','KCNS3','KCNU1','KDF1','KDR','KEL','KIAA0556','KIAA0895','KIAA1324','KIAA1522','KIAA1958','KIF17','KIF19','KIF21A','KIF21B','KIF26A','KIF26B','KISS1R','KIT','KLC1','KLC2','KLF9','KLHDC7B','KLHL41','KLHL6','KLHL7','KLK7','KLRC1','KLRC2','KLRF1','KLRK1','KMT2D','KNDC1','KNSTRN','KNTC1','KPTN','KRT75','KRT77','KRTAP25-1','L1CAM','L3MBTL1','LAIR1','LAMA4','LAMC1','LAMP2','LAMTOR1','LAPTM4A','LASP1','LAT','LAT2','LAX1','LCP1','LCTL','LDB2','LDLRAD3','LENG1','LGALS3','LGALS4','LGALS8','LGR6','LHFPL2','LHX6','LIMA1','LINGO1','LIPJ','LMCD1','LMOD2','LMTK2','LMTK3','LNX1','LNX2','LONRF1','LONRF3','LOXHD1','LPAR1','LPAR3','LPAR4','LPIN1','LPIN2','LRCH2','LRP11','LRP12','LRRC15','LRRC26','LRRC34','LRRC45','LRRC69','LRRC7','LRRC9','LRRFIP2','LRRIQ3','LSMEM2','LSR','LTB4R2','LUZP2','LY6K','LYN','LYPD1','LYST','LZTS1','LZTS2','MACF1','MADD','MAGI1','MAGI3','MAMDC2','MAP1S','MAP2','MAP2K1','MAP2K4','MAP2K7','MAP3K12','MAP3K21','MAP3K6','MAP3K7','MAP4','MAP4K1','MAP4K5','MAP6','MAPK15','MAPK8IP1','MAPT','MARCH10','MARCH7','MARCKS','MARK2','MARVELD2','MAS1L','MAT2B','MBP','MC5R','MCAM','MCEMP1','MCF2L2','MCMBP','MCTS1','MDGA1','ME1','MEF2B','MEIOC','MEPCE','MERTK','MET','METAP1','METAP2','METTL15','METTL25','METTL9','MFAP3L','MFSD1','MFSD2A','MGAT4C','MIB1','MICAL2','MICAL3','MICALL1','MICB','MIEN1','MIGA2','MISP','MKLN1','MKRN3','MLF2','MLKL','MLST8','MMACHC','MMP10','MMP15','MMP24','MMP25','MOAP1','MPHOSPH8','MPHOSPH9','MPI','MPIG6B','MPL','MPP1','MPP5','MPP7','MPZL1','MPZL2','MR1','MRGPRF','MRNIP','MRPL42','MRPL44','MRPL46','MRPS18B','MS4A1','MS4A12','MS4A14','MS4A15','MSN','MSRA','MTCP1','MTFR1L','MTHFR','MTM1','MTPN','MTSS1','MUC1','MUC20','MVD','MYH11','MYH9','MYL4','MYLK','MYO10','MYO1B','MYO1C','MYO1E','MYO1G','MYO3A','MYO5B','MYO9A','MYOF','N4BP2','NAA35','NAB1','NAIF1','NAMPT','NAP1L2','NAPG','NARFL','NCAM1','NCAM2','NCK1','NCKAP1L','NCKAP5','NCKIPSD','NCOA1','NCS1','NDC1','NECTIN2','NECTIN3','NECTIN4','NEDD1','NEDD9','NEO1','NEU1','NEXN','NF2','NFE2L2','NFKBID','NGEF']
MembraneProteins2=['NGFR','NHS','NHSL2','NID2','NINJ2','NKTR','NKX2-8','NLGN1','NLGN3','NLK','NLRX1','NMB','NMBR','NME1','NME2','NMT1','NMT2','NMUR2','NOS1','NOTCH2','NPFFR2','NPL','NPNT','NPR1','NPTN','NPTXR','NR2C1','NRAS','NRCAM','NRP2','NRSN1','NRXN1','NSD1','NSFL1C','NT5E','NTNG1','NTNG2','NTSR1','NUDCD3','NUDT16L1','NUDT9','NUMB','NUP35','NUS1','OAS3','OBSCN','OCA2','OCLN','OCM','OCM2','ODC1','ODR4','OGT','OLFM4','OLIG2','OLR1','OPHN1','OPRK1','OR10P1','OR1N2','OR2C3','OR6B2','OR6B3','OR6C6','OR8S1','OSBPL10','OSBPL6','OSBPL7','OSCAR','OSCP1','OSGEP','OSR2','OTOP2','OTULIN','OXGR1','P3H3','PACSIN2','PACSIN3','PAFAH1B2','PAG1','PAIP1','PAK1','PAK4','PAK6','PALLD','PALM','PALM2','PALM2-AKAP2','PALM3','PALMD','PANX1','PANX2','PAQR8','PARD3','PARD3B','PARD6A','PARD6G','PARP6','PATJ','PAWR','PCBD2','PCDH1','PCDH17','PCDH7','PCDHA1','PCDHA11','PCDHA12','PCDHA4','PCDHA5','PCDHB1','PCDHB11','PCDHB12','PCDHB16','PCDHB4','PCDHB5','PCDHGA1','PCDHGA10','PCDHGA11','PCDHGA12','PCDHGA2','PCDHGA3','PCDHGA4','PCDHGA5','PCDHGA6','PCDHGA7','PCDHGA8','PCDHGA9','PCDHGB1','PCDHGB2','PCDHGB3','PCDHGB4','PCDHGB5','PCDHGB6','PCDHGB7','PCDHGC3','PCDHGC4','PCDHGC5','PCLO','PCMTD1','PDAP1','PDCD7','PDE3A','PDE4A','PDE4D','PDE9A','PDGFC','PDGFRA','PDHX','PDLIM1','PDLIM5','PDPN','PDZD2','PDZD8','PEAR1','PEBP1','PFDN1','PGAP3','PGLYRP2','PGLYRP4','PGRMC2','PHACTR1','PHACTR4','PHF20L1','PHGDH','PHLDB2','PHLDB3','PHOSPHO1','PHPT1','PI3','PI4K2A','PI4KA','PIEZO2','PIGB','PIGN','PIGW','PIGX','PIK3AP1','PIK3CG','PIK3R6','PIP4K2A','PIP4K2C','PIP5K1A','PIRT','PITPNM3','PKD2L2','PKN1','PKN2','PKP1','PKP2','PKP3','PKP4','PLAA','PLAUR','PLCB1','PLCB2','PLCD3','PLCD4','PLEKHA4','PLEKHA6','PLEKHA7','PLEKHG3','PLEKHG4','PLEKHG6','PLEKHH2','PLEKHO2','PLIN4','PLPP1','PLPPR5','PLS3','PLSCR1','PLVAP','PLXNA3','PLXNA4','PNCK','PNISR','PODXL','POLE','POU2F3','PPEF1','PPFIBP1','PPIP5K1','PPL','PPM1A','PPM1F','PPP1CA','PPP1CB','PPP1R12B','PPP1R16A','PPP1R16B','PPP1R18','PPP1R27','PPP1R9A','PPP2R1B','PPP3R1','PPP3R2','PPP4C','PPP6R3','PPTC7','PRAME','PRC1','PRDX6','PRIMA1','PRKCA','PRKCE','PRKCH','PRKCZ','PRKD1','PROM2','PROSER2','PRR15','PRR32','PRR7','PRRC2A','PRRG1','PRRT2','PRRT3','PRRT4','PRTG','PRX','PSCA','PSD','PSD2','PSEN1','PSMC5','PSMC6','PSMD9','PSTPIP2','PTAFR','PTCHD4','PTK2B','PTK6','PTPN12','PTPN21','PTPN22','PTPN4','PTPRB','PTPRG','PTPRK','PTPRM','PTPRR','PTPRS','PTX3','PURG','PVALB','PVR','PXDC1','PXK','PYGL','PYGO2','PYM1','RAB13','RAB23','RAB25','RAB26','RAB27B','RAB3B','RAB3GAP2','RAB43','RAB8A','RABEPK','RAD51C','RAD52','RAI2','RALA','RALGAPA2','RAPGEF2','RAPH1','RAPSN','RASA4','RASA4B','RASAL1','RASAL2','RASAL3','RASD1','RASD2','RASGRP1','RASGRP2','RASGRP4','RASSF2','RASSF3','RASSF4','RBM46','RC3H1','RCCD1','RCSD1','RD3L','RDX','RECK','RELL1','REPS1','RET','RFLNB','RFPL1','RFPL2','RFPL3','RGP1','RGS10','RGS14','RGS19','RGS3','RGS7BP','RGS9BP','RHCE','RHD','RHEX','RHOA','RHOBTB2','RHOJ','RHOXF2','RHOXF2B','RIC8A','RIC8B','RIF1','RILPL1','RIMBP3','RIMBP3B','RIMBP3C','RIMS2','RIOK2','RIOK3','RIPK1','RND1','RNF114','RNF145','RNF146','RNF149','RNF165','RNF166','RNF39','ROBO1','ROBO3','ROBO4','ROM1','RP2','RPE','RPEL1','RPGRIP1L','RPIA','RPL36A','RPL36A-HNRNPH2','RPL36AL','RPSA','RPUSD1','RRAD','RRP12','RRP7A','RSBN1L','RSC1A1','RSPH3','RTL3','RTL4','RTN4R','RTN4RL1','RYR2','S100A13','S100A14','S100A16','S100A2','S100A3','S100A4','S100A6','S100A9','SAMD4A','SAMHD1','SAMSN1','SAPCD1','SASH3','SC5D','SCAMP1','SCAMP4','SCEL','SCHIP1','SCIN','SCN1A','SCN5A','SCN8A','SCN9A','SCNN1D','SCNN1G','SCRIB','SCUBE3','SDC2','SDC4','SDE2','SDHB','SEC16B','SEMA3C','SEMA3D','SEMA4B','SEMA4F','SEMA6C','SEMA6D','SEPT2','SEPT3','SERINC2','SERINC5','SERPINB3','SFSWAP','SGCE','SGK494','SH3BGR','SH3D21','SHANK1','SHANK3','SHARPIN','SHISA6','SHKBP1','SHROOM2','SHTN1','SIGLEC10','SIGLEC14','SIGLEC6','SIGLEC7','SIN3B','SIPA1','SIPA1L1','SIPA1L3','SKAP1','SLAMF6','SLC10A5','SLC12A2','SLC13A3','SLC13A5','SLC14A1','SLC16A1','SLC16A10','SLC16A2','SLC16A3','SLC16A7','SLC16A9','SLC17A5','SLC19A1','SLC19A3','SLC1A5','SLC1A6','SLC22A16','SLC23A3','SLC24A4','SLC26A9','SLC27A1','SLC28A1','SLC2A1','SLC2A11','SLC2A12','SLC2A14','SLC2A2','SLC2A3','SLC30A1','SLC31A2','SLC34A1','SLC34A2','SLC35F5','SLC35G2','SLC36A4','SLC38A5','SLC38A6','SLC38A7','SLC39A10','SLC39A14','SLC39A2','SLC39A6','SLC3A2','SLC40A1','SLC41A3','SLC43A1','SLC43A3','SLC44A2','SLC45A4','SLC46A1','SLC46A3','SLC4A1AP','SLC4A2','SLC4A7','SLC4A8','SLC5A12','SLC5A3','SLC5A6','SLC5A7','SLC6A6','SLC7A14','SLC7A2','SLC7A5','SLC7A6','SLC8A1','SLC9A1','SLC9A2','SLC9A3','SLC9A3R1','SLC9A3R2','SLC9A4','SLC9A5','SLC9B2','SLC9C2','SLCO1A2','SLCO1B3','SLCO4A1','SLCO4C1','SLCO5A1','SLIT1','SLIT3','SLITRK6','SLK','SLX4','SMAGP','SMAP1','SMOC1','SMPD2','SMPX','SNAP23','SNAP25','SNRK','SNTB2','SNTG2','SNX15','SNX32','SNX9','SOCS3','SOCS5','SOD1','SORBS1','SORCS1','SORT1','SOX11','SPAG16','SPANXN4','SPART','SPATA12','SPATC1L','SPERT','SPG11','SPHK1','SPINK13','SPINT1','SPINT2','SPN','SPPL2B','SPPL3','SPRED3','SPRYD3','SPTBN2','SRC','SRCIN1','SRD5A3','SRPK1','SRPRB','SRRM3','SSFA2','SSH1','SSH3','SSTR1','SSTR5','SSX2IP','ST8SIA5','STAB1','STAC','STAC2','STAMBP','STAMBPL1','STARD4','STAT2','STEAP4','STIL','STIP1','STK10','STK16','STK17A','STK17B','STOM','STOML2','STOML3','STX18','STX4','STXBP4','STYK1','SUCLG1','SUCLG2','SUGT1','SUMF2','SUPT6H','SUSD2','SVIL','SWAP70','SYCP2','SYK','SYMPK','SYNC','SYNGR3','SYNPO2L','SYPL2','SYT4','SYTL2','SYVN1','TACC2','TACR2','TACSTD2','TASP1','TAZ','TBC1D10A','TBC1D2','TBC1D24','TBC1D30','TBC1D9B','TBXA2R','TCAF2','TCAF2C','TCHP','TCIM','TDG','TDO2','TDP1','TEC','TEDC2','TEK','TES','TEX14','TEX9','TGFBR1','TGFBR2','TGM1','TGM2','THBS1','THBS2','THEGL','THY1','TIAM1','TICAM1','TIGD7','TIMD4','TJP1','TJP2','TJP3','TLN1','TLR4','TM2D2','TM4SF20','TM4SF4','TM4SF5','TMC5','TMED8','TMEM101','TMEM107','TMEM108','TMEM121B','TMEM139','TMEM141','TMEM164','TMEM176B','TMEM183A','TMEM186','TMEM2','TMEM218','TMEM222','TMEM229A','TMEM245','TMEM251','TMEM254','TMEM258','TMEM266','TMEM268','TMEM50A','TMEM63B','TMPRSS15','TMPRSS2','TMTC2','TNFRSF10D','TNFRSF11A','TNFRSF12A','TNFRSF21','TNFRSF25','TNFRSF8','TNFRSF9','TNFSF13B','TNK1','TNKS1BP1','TNNI2','TOGARAM2','TOLLIP','TOM1','TP53BP2','TPD52L1','TPRN','TRAF7','TRAIP','TRAPPC13','TRAT1','TRDN','TREML1','TRIB1','TRIM16','TRIM16L','TRIM38','TRIM4','TRIM44','TRIM54','TRIM71','TRIML2','TRIP6','TRMT1','TRMT10B','TRMT13','TRMT6','TRO','TRPM1','TRPM4','TRPM7','TRPV2','TRPV3','TSG101','TSGA13','TSHZ3','TSPAN15','TSPAN6','TSPO2','TSR2','TSSK4','TTC17','TTC23','TTC7B','TTLL12','TTLL5','TULP3','TWF2','TXNDC11','TXNIP','TYROBP','TYW3','UAP1','UBA3','UBA52','UBAC1','UBALD1','UBAP1','UBASH3B','UBE2A','UBE2B','UBE2C','UBE2D1','UBE2D2','UBE2D3','UBE2D4','UBE2K','UBE2QL1','UBE2S','UBE3C','UBQLN2','UBR2','UBTD1','UCP3','ULBP1','ULBP2','UNC5A','UNC79','UQCC1','USP14','USP15','USP2','USP20','USP21','USP4','USP53','UTP20','UTRN','VAMP5','VAMP8','VANGL1','VASP','VCAM1','VCPIP1','VIL1','VPS13B','VPS36','VPS50','VPS8','VSIG4','VTCN1','VWA5B1','VWA7','WAS','WASF2','WBP4','WDFY3','WDPCP','WDR1','WDR12','WDR13','WDR53','WDR6','WDR66','WIPF1','WWP1','XIAP','XIRP1','XIRP2','XPC','XPNPEP2','XPO6','XRN1','XYLT1','YES1','YIPF1','YIPF4','YOD1','YTHDC1','ZBED3','ZBTB33','ZBTB8OS','ZDHHC2','ZDHHC20','ZDHHC22','ZDHHC5','ZFP41','ZFP90','ZFYVE28','ZGPAT','ZMYND10','ZNF10','ZNF18','ZNF185','ZNF2','ZNF20','ZNF227','ZNF229','ZNF236','ZNF276','ZNF277','ZNF337','ZNF397','ZNF407','ZNF497','ZNF503','ZNF510','ZNF536','ZNF571','ZNF598','ZNF599','ZNF646','ZNF655','ZNF688','ZNF706','ZNF717','ZNF783','ZNF79','ZNF839','ZSCAN22','ZSCAN29','ZSWIM1','ZSWIM8','ZYX']
MembraneProteins=MembraneProteins1+MembraneProteins2

TFs1=['AC008770.3','AC023509.3','AC092835.1','AC138696.1','ADNP','ADNP2','AEBP1','AEBP2','AHCTF1','AHDC1','AHR','AHRR','AIRE','AKAP8','AKAP8L','AKNA','ALX1','ALX3','ALX4','ANHX','ANKZF1','AR','ARGFX','ARHGAP35','ARID2','ARID3A','ARID3B','ARID3C','ARID5A','ARID5B','ARNT','ARNT2','ARNTL','ARNTL2','ARX','ASCL1','ASCL2','ASCL3','ASCL4','ASCL5','ASH1L','ATF1','ATF2','ATF3','ATF4','ATF5','ATF6','ATF6B','ATF7','ATMIN','ATOH1','ATOH7','ATOH8','BACH1','BACH2','BARHL1','BARHL2','BARX1','BARX2','BATF','BATF2','BATF3','BAZ2A','BAZ2B','BBX','BCL11A','BCL11B','BCL6','BCL6B','BHLHA15','BHLHA9','BHLHE22','BHLHE23','BHLHE40','BHLHE41','BNC1','BNC2','BORCS8-MEF2B','BPTF','BRF2','BSX','C11orf95','CAMTA1','CAMTA2','CARF','CASZ1','CBX2','CC2D1A','CCDC169-SOHLH2','CCDC17','CDC5L','CDX1','CDX2','CDX4','CEBPA','CEBPB','CEBPD','CEBPE','CEBPG','CEBPZ','CENPA','CENPB','CENPBD1','CENPS','CENPT','CENPX','CGGBP1','CHAMP1','CHCHD3','CIC','CLOCK','CPEB1','CPXCR1','CREB1','CREB3','CREB3L1','CREB3L2','CREB3L3','CREB3L4','CREB5','CREBL2','CREBZF','CREM','CRX','CSRNP1','CSRNP2','CSRNP3','CTCF','CTCFL','CUX1','CUX2','CXXC1','CXXC4','CXXC5','DACH1','DACH2','DBP','DBX1','DBX2','DDIT3','DEAF1','DLX1','DLX2','DLX3','DLX4','DLX5','DLX6','DMBX1','DMRT1','DMRT2','DMRT3','DMRTA1','DMRTA2','DMRTB1','DMRTC2','DMTF1','DNMT1','DNTTIP1','DOT1L','DPF1','DPF3','DPRX','DR1','DRAP1','DRGX','DUX1','DUX3','DUX4','DUXA','DZIP1','E2F1','E2F2','E2F3','E2F4','E2F5','E2F6','E2F7','E2F8','E4F1','EBF1','EBF2','EBF3','EBF4','EEA1','EGR1','EGR2','EGR3','EGR4','EHF','ELF1','ELF2','ELF3','ELF4','ELF5','ELK1','ELK3','ELK4','EMX1','EMX2','EN1','EN2','EOMES','EPAS1','ERF','ERG','ESR1','ESR2','ESRRA','ESRRB','ESRRG','ESX1','ETS1','ETS2','ETV1','ETV2','ETV3','ETV3L','ETV4','ETV5','ETV6','ETV7','EVX1','EVX2','FAM170A','FAM200B','FBXL19','FERD3L','FEV','FEZF1','FEZF2','FIGLA','FIZ1','FLI1','FLYWCH1','FOS','FOSB','FOSL1','FOSL2','FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXN4','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1','GABPA','GATA1','GATA2','GATA3','GATA4','GATA5','GATA6','GATAD2A','GATAD2B','GBX1','GBX2','GCM1','GCM2','GFI1','GFI1B','GLI1','GLI2','GLI3','GLI4','GLIS1','GLIS2','GLIS3','GLMP','GLYR1','GMEB1','GMEB2','GPBP1','GPBP1L1','GRHL1','GRHL2','GRHL3','GSC','GSC2','GSX1','GSX2','GTF2B','GTF2I','GTF2IRD1','GTF2IRD2','GTF2IRD2B','GTF3A','GZF1','HAND1','HAND2','HBP1','HDX','HELT','HES1','HES2','HES3','HES4','HES5','HES6','HES7','HESX1','HEY1','HEY2','HEYL','HHEX','HIC1','HIC2','HIF1A','HIF3A','HINFP','HIVEP1','HIVEP2','HIVEP3','HKR1','HLF','HLX','HMBOX1','HMG20A','HMG20B','HMGA1','HMGA2','HMGN3','HMX1','HMX2','HMX3','HNF1A','HNF1B','HNF4A','HNF4G','HOMEZ','HOXA1','HOXA10','HOXA11','HOXA13','HOXA2','HOXA3','HOXA4','HOXA5','HOXA6','HOXA7','HOXA9','HOXB1','HOXB13','HOXB2','HOXB3','HOXB4','HOXB5','HOXB6','HOXB7','HOXB8','HOXB9','HOXC10','HOXC11','HOXC12','HOXC13','HOXC4','HOXC5','HOXC6','HOXC8','HOXC9','HOXD1','HOXD10','HOXD11','HOXD12','HOXD13','HOXD3','HOXD4','HOXD8','HOXD9','HSF1','HSF2','HSF4','HSF5','HSFX1','HSFX2','HSFY1','HSFY2','IKZF1','IKZF2','IKZF3','IKZF4','IKZF5','INSM1','INSM2','IRF1','IRF2','IRF3','IRF4','IRF5','IRF6','IRF7','IRF8','IRF9','IRX1','IRX2','IRX3','IRX4','IRX5','IRX6','ISL1','ISL2','ISX','JAZF1','JDP2','JRK','JRKL','JUN','JUNB','JUND','KAT7','KCMF1','KCNIP3','KDM2A','KDM2B','KDM5B','KIN','KLF1','KLF10','KLF11','KLF12','KLF13','KLF14','KLF15','KLF16','KLF17','KLF2','KLF3','KLF4','KLF5','KLF6','KLF7','KLF8','KLF9','KMT2A','KMT2B','L3MBTL1','L3MBTL3','L3MBTL4','LBX1','LBX2','LCOR','LCORL','LEF1','LEUTX','LHX1','LHX2','LHX3','LHX4','LHX5','LHX6','LHX8','LHX9','LIN28A','LIN28B','LIN54','LMX1A','LMX1B','LTF','LYL1','MAF','MAFA','MAFB','MAFF','MAFG','MAFK','MAX','MAZ','MBD1','MBD2','MBD3','MBD4','MBD6','MBNL2','MECOM','MECP2','MEF2A','MEF2B','MEF2C','MEF2D','MEIS1','MEIS2','MEIS3','MEOX1','MEOX2','MESP1','MESP2','MGA','MITF','MIXL1','MKX','MLX','MLXIP','MLXIPL','MNT','MNX1','MSANTD1','MSANTD3','MSANTD4','MSC','MSGN1','MSX1','MSX2','MTERF1','MTERF2','MTERF3','MTERF4','MTF1','MTF2','MXD1','MXD3','MXD4','MXI1','MYB','MYBL1','MYBL2','MYC','MYCL','MYCN','MYF5','MYF6','MYNN','MYOD1','MYOG','MYPOP','MYRF','MYRFL','MYSM1','MYT1','MYT1L','MZF1','NACC2','NAIF1','NANOG','NANOGNB','NANOGP8','NCOA1','NCOA2','NCOA3','NEUROD1','NEUROD2','NEUROD4','NEUROD6','NEUROG1','NEUROG2','NEUROG3','NFAT5','NFATC1','NFATC2','NFATC3','NFATC4','NFE2','NFE2L1','NFE2L2','NFE2L3','NFE4','NFIA','NFIB','NFIC','NFIL3','NFIX','NFKB1','NFKB2','NFX1','NFXL1','NFYA','NFYB','NFYC','NHLH1','NHLH2','NKRF','NKX1-1','NKX1-2','NKX2-1','NKX2-2','NKX2-3','NKX2-4','NKX2-5','NKX2-6','NKX2-8','NKX3-1','NKX3-2','NKX6-1','NKX6-2','NKX6-3','NME2','NOBOX','NOTO','NPAS1','NPAS2','NPAS3','NPAS4','NR0B1','NR1D1','NR1D2','NR1H2','NR1H3','NR1H4','NR1I2','NR1I3','NR2C1','NR2C2','NR2E1','NR2E3','NR2F1','NR2F2','NR2F6','NR3C1','NR3C2','NR4A1','NR4A2','NR4A3','NR5A1','NR5A2','NR6A1','NRF1','NRL','OLIG1','OLIG2','OLIG3','ONECUT1','ONECUT2','ONECUT3','OSR1','OSR2','OTP','OTX1','OTX2','OVOL1','OVOL2','OVOL3','PA2G4','PATZ1','PAX1','PAX2','PAX3','PAX4','PAX5','PAX6','PAX7','PAX8','PAX9','PBX1','PBX2','PBX3','PBX4','PCGF2','PCGF6','PDX1','PEG3','PGR','PHF1','PHF19 ','PHF20','PHF21A','PHOX2A','PHOX2B','PIN1','PITX1','PITX2','PITX3','PKNOX1','PKNOX2','PLAG1','PLAGL1','PLAGL2','PLSCR1','POGK','POU1F1','POU2AF1','POU2F1','POU2F2','POU2F3','POU3F1','POU3F2','POU3F3','POU3F4','POU4F1','POU4F2','POU4F3','POU5F1','POU5F1B','POU5F2','POU6F1','POU6F2','PPARA','PPARD','PPARG','PRDM1','PRDM10','PRDM12','PRDM13','PRDM14','PRDM15','PRDM16','PRDM2','PRDM4','PRDM5','PRDM6','PRDM8','PRDM9','PREB','PRMT3','PROP1','PROX1','PROX2','PRR12','PRRX1','PRRX2','PTF1A','PURA','PURB','PURG','RAG1','RARA','RARB','RARG','RAX','RAX2','RBAK','RBCK1','RBPJ','RBPJL','RBSN','REL','RELA','RELB','REPIN1','REST','REXO4','RFX1','RFX2','RFX3','RFX4','RFX5','RFX6','RFX7','RFX8','RHOXF1','RHOXF2','RHOXF2B','RLF','RORA','RORB','RORC','RREB1','RUNX1','RUNX2','RUNX3','RXRA','RXRB','RXRG','SAFB','SAFB2','SALL1','SALL2','SALL3','SALL4','SATB1','SATB2','SCMH1','SCML4','SCRT1','SCRT2','SCX','SEBOX','SETBP1','SETDB1','SETDB2','SGSM2','SHOX','SHOX2','SIM1','SIM2','SIX1','SIX2','SIX3','SIX4','SIX5','SIX6','SKI','SKIL','SKOR1','SKOR2','SLC2A4RG','SMAD1','SMAD3','SMAD4','SMAD5','SMAD9','SMYD3','SNAI1','SNAI2','SNAI3','SNAPC2','SNAPC4','SNAPC5','SOHLH1','SOHLH2','SON','SOX1','SOX10','SOX11']
TFs2=['SOX12','SOX13','SOX14','SOX15','SOX17','SOX18','SOX2','SOX21','SOX3','SOX30','SOX4','SOX5','SOX6','SOX7','SOX8','SOX9','SP1','SP100','SP110','SP140','SP140L','SP2','SP3','SP4','SP5','SP6','SP7','SP8','SP9','SPDEF','SPEN','SPI1','SPIB','SPIC','SPZ1','SRCAP','SREBF1','SREBF2','SRF','SRY','ST18','STAT1','STAT2','STAT3','STAT4','STAT5A','STAT5B','STAT6','T','TAL1','TAL2','TBP','TBPL1','TBPL2','TBR1','TBX1','TBX10','TBX15','TBX18','TBX19','TBX2','TBX20','TBX21','TBX22','TBX3','TBX4','TBX5','TBX6','TCF12','TCF15','TCF20','TCF21','TCF23','TCF24','TCF3','TCF4','TCF7','TCF7L1','TCF7L2','TCFL5','TEAD1','TEAD2','TEAD3','TEAD4','TEF','TERB1','TERF1','TERF2','TET1','TET2','TET3','TFAP2A','TFAP2B','TFAP2C','TFAP2D','TFAP2E','TFAP4','TFCP2','TFCP2L1','TFDP1','TFDP2','TFDP3','TFE3','TFEB','TFEC','TGIF1','TGIF2','TGIF2LX','TGIF2LY','THAP1','THAP10','THAP11','THAP12','THAP2','THAP3','THAP4','THAP5','THAP6','THAP7','THAP8','THAP9','THRA','THRB','THYN1','TIGD1','TIGD2','TIGD3','TIGD4','TIGD5','TIGD6','TIGD7','TLX1','TLX2','TLX3','TMF1','TOPORS','TP53','TP63','TP73','TPRX1','TRAFD1','TRERF1','TRPS1','TSC22D1','TSHZ1','TSHZ2','TSHZ3','TTF1','TWIST1','TWIST2','UBP1','UNCX','USF1','USF2','USF3','VAX1','VAX2','VDR','VENTX','VEZF1','VSX1','VSX2','WIZ','WT1','XBP1','XPA','YBX1','YBX2','YBX3','YY1','YY2','ZBED1','ZBED2','ZBED3','ZBED4','ZBED5','ZBED6','ZBED9','ZBTB1','ZBTB10','ZBTB11','ZBTB12','ZBTB14','ZBTB16','ZBTB17','ZBTB18','ZBTB2','ZBTB20','ZBTB21','ZBTB22','ZBTB24','ZBTB25','ZBTB26','ZBTB3','ZBTB32','ZBTB33','ZBTB34','ZBTB37','ZBTB38','ZBTB39','ZBTB4','ZBTB40','ZBTB41','ZBTB42','ZBTB43','ZBTB44','ZBTB45','ZBTB46','ZBTB47','ZBTB48','ZBTB49','ZBTB5','ZBTB6','ZBTB7A','ZBTB7B','ZBTB7C','ZBTB8A','ZBTB8B','ZBTB9','ZC3H8','ZEB1','ZEB2','ZFAT','ZFHX2','ZFHX3','ZFHX4','ZFP1','ZFP14','ZFP2','ZFP28','ZFP3','ZFP30','ZFP37','ZFP41','ZFP42','ZFP57','ZFP62','ZFP64','ZFP69','ZFP69B','ZFP82','ZFP90','ZFP91','ZFP92','ZFPM1','ZFPM2','ZFX','ZFY','ZGLP1','ZGPAT','ZHX1','ZHX2','ZHX3','ZIC1','ZIC2','ZIC3','ZIC4','ZIC5','ZIK1','ZIM2','ZIM3','ZKSCAN1','ZKSCAN2','ZKSCAN3','ZKSCAN4','ZKSCAN5','ZKSCAN7','ZKSCAN8','ZMAT1','ZMAT4','ZNF10','ZNF100','ZNF101','ZNF107','ZNF112','ZNF114','ZNF117','ZNF12','ZNF121','ZNF124','ZNF131','ZNF132','ZNF133','ZNF134','ZNF135','ZNF136','ZNF138','ZNF14','ZNF140','ZNF141','ZNF142','ZNF143','ZNF146','ZNF148','ZNF154','ZNF155','ZNF157','ZNF16','ZNF160','ZNF165','ZNF169','ZNF17','ZNF174','ZNF175','ZNF177','ZNF18','ZNF180','ZNF181','ZNF182','ZNF184','ZNF189','ZNF19','ZNF195','ZNF197','ZNF2','ZNF20','ZNF200','ZNF202','ZNF205','ZNF207','ZNF208','ZNF211','ZNF212','ZNF213','ZNF214','ZNF215','ZNF217','ZNF219','ZNF22','ZNF221','ZNF222','ZNF223','ZNF224','ZNF225','ZNF226','ZNF227','ZNF229','ZNF23','ZNF230','ZNF232','ZNF233','ZNF234','ZNF235','ZNF236','ZNF239','ZNF24','ZNF248','ZNF25','ZNF250','ZNF251','ZNF253','ZNF254','ZNF256','ZNF257','ZNF26','ZNF260','ZNF263','ZNF264','ZNF266','ZNF267','ZNF268','ZNF273','ZNF274','ZNF275','ZNF276','ZNF277','ZNF28','ZNF280A','ZNF280B','ZNF280C','ZNF280D','ZNF281','ZNF282','ZNF283','ZNF284','ZNF285','ZNF286A','ZNF286B','ZNF287','ZNF292','ZNF296','ZNF3','ZNF30','ZNF300','ZNF302','ZNF304','ZNF311','ZNF316','ZNF317','ZNF318','ZNF319','ZNF32','ZNF320','ZNF322','ZNF324','ZNF324B','ZNF326','ZNF329','ZNF331','ZNF333','ZNF334','ZNF335','ZNF337','ZNF33A','ZNF33B','ZNF34','ZNF341','ZNF343','ZNF345','ZNF346','ZNF347','ZNF35','ZNF350','ZNF354A','ZNF354B','ZNF354C','ZNF358','ZNF362','ZNF365','ZNF366','ZNF367','ZNF37A','ZNF382','ZNF383','ZNF384','ZNF385A','ZNF385B','ZNF385C','ZNF385D','ZNF391','ZNF394','ZNF395','ZNF396','ZNF397','ZNF398','ZNF404','ZNF407','ZNF408','ZNF41','ZNF410','ZNF414','ZNF415','ZNF416','ZNF417','ZNF418','ZNF419','ZNF420','ZNF423','ZNF425','ZNF426','ZNF428','ZNF429','ZNF43','ZNF430','ZNF431','ZNF432','ZNF433','ZNF436','ZNF438','ZNF439','ZNF44','ZNF440','ZNF441','ZNF442','ZNF443','ZNF444','ZNF445','ZNF446','ZNF449','ZNF45','ZNF451','ZNF454','ZNF460','ZNF461','ZNF462','ZNF467','ZNF468','ZNF469','ZNF470','ZNF471','ZNF473','ZNF474','ZNF479','ZNF48','ZNF480','ZNF483','ZNF484','ZNF485','ZNF486','ZNF487','ZNF488','ZNF490','ZNF491','ZNF492','ZNF493','ZNF496','ZNF497','ZNF500','ZNF501','ZNF502','ZNF503','ZNF506','ZNF507','ZNF510','ZNF511','ZNF512','ZNF512B','ZNF513','ZNF514','ZNF516','ZNF517','ZNF518A','ZNF518B','ZNF519','ZNF521','ZNF524','ZNF525','ZNF526','ZNF527','ZNF528','ZNF529','ZNF530','ZNF532','ZNF534','ZNF536','ZNF540','ZNF541','ZNF543','ZNF544','ZNF546','ZNF547','ZNF548','ZNF549','ZNF550','ZNF551','ZNF552','ZNF554','ZNF555','ZNF556','ZNF557','ZNF558','ZNF559','ZNF560','ZNF561','ZNF562','ZNF563','ZNF564','ZNF565','ZNF566','ZNF567','ZNF568','ZNF569','ZNF57','ZNF570','ZNF571','ZNF572','ZNF573','ZNF574','ZNF575','ZNF576','ZNF577','ZNF578','ZNF579','ZNF580','ZNF581','ZNF582','ZNF583','ZNF584','ZNF585A','ZNF585B','ZNF586','ZNF587','ZNF587B','ZNF589','ZNF592','ZNF594','ZNF595','ZNF596','ZNF597','ZNF598','ZNF599','ZNF600','ZNF605','ZNF606','ZNF607','ZNF608','ZNF609','ZNF610','ZNF611','ZNF613','ZNF614','ZNF615','ZNF616','ZNF618','ZNF619','ZNF620','ZNF621','ZNF623','ZNF624','ZNF625','ZNF626','ZNF627','ZNF628','ZNF629','ZNF630','ZNF639','ZNF641','ZNF644','ZNF645','ZNF646','ZNF648','ZNF649','ZNF652','ZNF653','ZNF654','ZNF655','ZNF658','ZNF66','ZNF660','ZNF662','ZNF664','ZNF665','ZNF667','ZNF668','ZNF669','ZNF670','ZNF671','ZNF672','ZNF674','ZNF675','ZNF676','ZNF677','ZNF678','ZNF679','ZNF680','ZNF681','ZNF682','ZNF683','ZNF684','ZNF687','ZNF688','ZNF689','ZNF69','ZNF691','ZNF692','ZNF695','ZNF696','ZNF697','ZNF699','ZNF7','ZNF70','ZNF700','ZNF701','ZNF703','ZNF704','ZNF705A','ZNF705B','ZNF705D','ZNF705E','ZNF705G','ZNF706','ZNF707','ZNF708','ZNF709','ZNF71','ZNF710','ZNF711','ZNF713','ZNF714','ZNF716','ZNF717','ZNF718','ZNF721','ZNF724','ZNF726','ZNF727','ZNF728','ZNF729','ZNF730','ZNF732','ZNF735','ZNF736','ZNF737','ZNF74','ZNF740','ZNF746','ZNF747','ZNF749','ZNF750','ZNF75A','ZNF75D','ZNF76','ZNF761','ZNF763','ZNF764','ZNF765','ZNF766','ZNF768','ZNF77','ZNF770','ZNF771','ZNF772','ZNF773','ZNF774','ZNF775','ZNF776','ZNF777','ZNF778','ZNF780A','ZNF780B','ZNF781','ZNF782','ZNF783','ZNF784','ZNF785','ZNF786','ZNF787','ZNF788','ZNF789','ZNF79','ZNF790','ZNF791','ZNF792','ZNF793','ZNF799','ZNF8','ZNF80','ZNF800','ZNF804A','ZNF804B','ZNF805','ZNF808','ZNF81','ZNF813','ZNF814','ZNF816','ZNF821','ZNF823','ZNF827','ZNF829','ZNF83','ZNF830','ZNF831','ZNF835','ZNF836','ZNF837','ZNF84','ZNF841','ZNF843','ZNF844','ZNF845','ZNF846','ZNF85','ZNF850','ZNF852','ZNF853','ZNF860','ZNF865','ZNF878','ZNF879','ZNF880','ZNF883','ZNF888','ZNF891','ZNF90','ZNF91','ZNF92','ZNF93','ZNF98','ZNF99','ZSCAN1','ZSCAN10','ZSCAN12','ZSCAN16','ZSCAN18','ZSCAN2','ZSCAN20','ZSCAN21','ZSCAN22','ZSCAN23','ZSCAN25','ZSCAN26','ZSCAN29','ZSCAN30','ZSCAN31','ZSCAN32','ZSCAN4','ZSCAN5A','ZSCAN5B','ZSCAN5C','ZSCAN9','ZUFSP','ZXDA','ZXDB','ZXDC','ZZZ3']
TFs=TFs1+TFs2

#TFs other, inclusive of the above list, not necessarily DNA binding proteins but affect transcription
TFs1Other=['AATF','ABL1','ABRA','AC008770.3','AC020915.1','AC023509.3','AC092835.1','AC105001.2','AC138696.1','ACOD1','ACTR3B','ACTR3C','ADAM8','ADAMTS17','ADAMTS19','ADAR','ADGRA1','ADGRG3','ADNP','ADNP2','AEBP1','AEBP2','AES','AFAP1 ','AFF1','AFF3','AFF4','AGER','AGT','AHCTF1','AHDC1','AHR','AHRR','AIM2','AIRE','AKAP8','AKAP8L','AKNA','AKT1','ALK','ALKBH2','ALX1','ALX3','ALX4','AMH','ANAPC2','ANHX','ANKRD30A','ANKZF1','ANXA3','ANXA4','AP5Z1','APBB3','APC2','APEX1','APOA4','APOBEC3A','APOBEC3B','APOBEC3G','APTX','AR','ARGFX','ARHGAP35','ARHGEF2','ARHGEF5','ARID1A','ARID1B','ARID2','ARID3A','ARID3B','ARID3C','ARID4A','ARID4B','ARID5A','ARID5B','ARNT','ARNT2','ARNTL','ARNTL2','ARRB1','ARRB2','ARX','ASCC1','ASCL1','ASCL2','ASCL3','ASCL4','ASCL5','ASH1L','ATF1','ATF2','ATF3','ATF4','ATF5','ATF6','ATF6B','ATF7','ATF7IP','ATM','ATMIN','ATOH1','ATOH7','ATOH8','ATR','ATRX','ATXN7','BACH1','BACH2','BAG1','BANF1','BANP','BARHL1','BARHL2','BARX1','BARX2','BATF','BATF2','BATF3','BAZ2A','BAZ2B','BBX','BCL10','BCL11A','BCL11B','BCL3','BCL6','BCL6B','BCLAF1','BCOR','BDP1','BEND4','BEX1','BHLHA15','BHLHA9','BHLHE22','BHLHE23','BHLHE40','BHLHE41','BHMG1','BIN1','BLM','BLZF1','BMI1','BMP2','BMP4','BMP7','BMPR1A','BNC1','BNC2','BOD1L1','BOLA1','BOLA2','BOLA2B','BOLA3','BORCS8-MEF2B','BPNT1','BPTF','BRCA1','BRD4','BRD7','BRD8','BRD9','BRF1','BRF2','BRMS1','BRPF1','BSX','BTAF1','BTBD8','BTG2','BTK','BTRC','BUD31','C11orf95','C14orf39','C17orf49','C1D','C20orf194','C3orf33','CACTIN','CAMK1D','CAMK2A','CAMTA1','CAMTA2','CAPN15','CAPN3','CARD11','CARD14','CARF','CARHSP1','CASP8AP2','CASZ1','CAT','CBFA2T2','CBFA2T3','CBFB','CBL','CBLL1','CBX2','CC2D1A','CC2D1B','CCDC124','CCDC169-SOHLH2','CCDC17','CCDC3','CCDC71','CCDC83','CCNT1','CCNT2','CD36','CD40','CD40LG','CDC5L','CDC6','CDK5RAP3','CDK7','CDK8','CDK9','CDKN1A','CDKN2A','CDX1','CDX2','CDX4','CEBPA','CEBPB','CEBPD','CEBPE','CEBPG','CEBPZ','CELF3','CENPA','CENPB','CENPBD1','CENPS','CENPT','CENPX','CERS2','CERS3','CERS4','CERS5','CERS6','CFLAR','CGGBP1','CHAMP1','CHCHD3','CHD1','CHD2','CHD3','CHD4','CHD6','CHD7','CHD9','CHP1','CHRAC1','CHUK','CHURC1','CIB1','CIC','CIITA','CIR1','CITED1','CITED2','CITED4','CIZ1','CLOCK','CLU','CMKLR1','CNBP','CNOT3','CNOT4','CNOT7','CNOT8','COMMD1','COMMD6','COMMD7','COPS2','COPS3','COPS4','CPEB1','CPSF4','CPSF4L','CPXCR1','CRAMP1','CREB1','CREB3','CREB3L1','CREB3L2','CREB3L3','CREB3L4','CREB5','CREBBP','CREBL2','CREBRF','CREBZF','CREG1','CREM','CRTC1','CRTC2','CRTC3','CRX','CSDC2','CSDE1','CSRNP1','CSRNP2','CSRNP3','CTBP1','CTBP2','CTCF','CTCFL','CTH','CTNNB1','CTNNBIP1','CUL1','CUL2','CUL3','CUL4A','CUL4B','CUL5','CUX1','CUX2','CXXC1','CXXC4','CXXC5','CYLD','CYP1B1','CYTL1','DAB2IP','DACH1','DACH2','DAP','DAXX','DBP','DBX1','DBX2','DCP1A','DDB1','DDB2','DDIT3','DDN','DDR2','DDRGK1','DDX3X','DDX5','DDX58','DEAF1','DEK','DEPDC1','DEPDC1B','DEPDC4','DEPDC5','DEPDC7','DEPTOR','DHX34','DHX36','DHX57','DISP3','DLX1','DLX2','DLX3','DLX4','DLX5','DLX6','DMAP1','DMBX1','DMC1','DMRT1','DMRT2','DMRT3','DMRTA1','DMRTA2','DMRTB1','DMRTC1','DMRTC2','DMTF1','DNA2','DNAJA3','DNAJC1','DNAJC2','DNAJC21','DNMT1','DNMT3A','DNMT3B','DNMT3L','DNTT','DNTTIP1','DOT1L','DPF1','DPF2','DPF3','DPRX','DR1','DRAP1','DRGX','DSP','DTX1','DUS3L','DUSP12','DUX1','DUX3','DUX4','DUXA','DVL1','DVL2','DVL3','DZIP1','DZIP1L','E2F1','E2F2','E2F3','E2F4','E2F5','E2F6','E2F7','E2F8','E4F1','EAF2','EBF1','EBF2','EBF3','EBF4','EDA','EDA2R','EDF1','EDN1','EEA1','EED','EGLN1','EGR1','EGR2','EGR3','EGR4','EHF','EID1','EID2','EIF2AK2','EIF2AK4','EIF3C','EIF3CL','EIF3K','ELF1','ELF2','ELF3','ELF4','ELF5','ELK1','ELK3','ELK4','ELMSAN1','EME1','EMX1','EMX2','EN1','EN2','ENO1','EOMES','EP300','EP400','EPAS1','EPHA5','EPM2AIP1','ERBIN','ERC1','ERCC3','ERCC4','ERF','ERG','ESR1','ESR2','ESRRA','ESRRB','ESRRG','ESX1','ETS1','ETS2','ETV1','ETV2','ETV3','ETV3L','ETV4','ETV5','ETV6','ETV7','EVX1','EVX2','EWSR1','EXO1','EXOC2','EXOG','EZH1','EZH2','FAAP24','FAM153C','FAM170A','FAM171B','FAM200A','FAM200B','FAN1','FANCA','FANCD2','FANK1','FARSA','FARSB','FBN1','FBXL19','FBXO41','FEN1','FER','FERD3L','FEV','FEZF1','FEZF2','FGD1','FHL2','FIGLA','FIZ1','FLI1','FLNA','FLYWCH1','FMNL2','FOS','FOSB','FOSL1','FOSL2','FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L2','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXN4','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1','FUBP1','FUBP3','FUS','FZD1','FZD2','FZD4','FZD6','GABPA','GABPB1','GABPB2','GADD45A','GAS6','GAS7','GATA1','GATA2','GATA3','GATA4','GATA5','GATA6','GATAD1','GATAD2A','GATAD2B','GBX1','GBX2','GCFC2','GCM1','GCM2','GEN1','GFI1','GFI1B','GLA','GLI1','GLI2','GLI3','GLI4','GLIS1','GLIS2','GLIS3','GLMP','GLYR1','GMEB1','GMEB2','GOLGB1','GON4L','GPATCH8','GPBP1','GPBP1L1','GPER1','GPR155','GREM1','GRHL1','GRHL2','GRHL3','GRM6','GSC','GSC2','GSX1','GSX2','GTF2A1','GTF2A1L','GTF2A2','GTF2B','GTF2E2','GTF2F1','GTF2F2','GTF2H1','GTF2H2','GTF2H2C','GTF2H3','GTF2H4','GTF2I','GTF2IRD1','GTF2IRD2','GTF2IRD2B','GTF3A','GTF3C1','GTF3C2','GULP1','GZF1','H1F0','H1FOO','H1FX','HAND1','HAND2','HAVCR2','HBP1','HCFC1','HCK','HDAC1','HDAC10','HDAC2','HDAC3','HDAC4','HDAC5','HDAC6','HDAC7','HDAC8','HDAC9','HDX','HELT','HELZ','HELZ2','HES1','HES2','HES3','HES4','HES5','HES6','HES7','HESX1','HEY1','HEY2','HEYL','HHEX','HIC1','HIC2','HIF1A','HIF3A','HINFP','HIPK2','HIRA','HIST1H1A','HIST1H1B','HIST1H1C','HIST1H1D','HIST1H1E','HIST1H1T','HIVEP1','HIVEP2','HIVEP3','HKR1','HLA-DQB1','HLA-DQB2','HLA-DRB1','HLA-DRB3','HLA-DRB5','HLF','HLTF','HLX','HMBOX1','HMG20A','HMG20B','HMGA1','HMGA2','HMGB1','HMGB2','HMGB3','HMGB4','HMGN1','HMGN3','HMGXB3','HMGXB4','HMOX1','HMX1','HMX2','HMX3','HNF1A','HNF1B','HNF4A','HNF4G','HNRNPA1','HNRNPA3','HNRNPAB','HNRNPCL2','HNRNPD','HNRNPDL','HNRNPK','HNRNPL','HNRNPU','HOMEZ','HOOK2','HOPX','HOXA1','HOXA10','HOXA11','HOXA13','HOXA2','HOXA3','HOXA4','HOXA5','HOXA6','HOXA7','HOXA9','HOXB1','HOXB13','HOXB2','HOXB3','HOXB4','HOXB5','HOXB6','HOXB7','HOXB8','HOXB9','HOXC10','HOXC11','HOXC12','HOXC13','HOXC4','HOXC5','HOXC6','HOXC8','HOXC9','HOXD1','HOXD10','HOXD11','HOXD12','HOXD13','HOXD3','HOXD4','HOXD8','HOXD9','HP1BP3','HR','HSBP1','HSF1','HSF2','HSF4','HSF5','HSFX1','HSFX2','HSFY1','HSFY2','HSPA1A','HSPA1B','ICAM1','ID1','ID2','ID3','ID4','IFI16','IFNB1','IGHM','IKBKB','IKBKG','IKZF1','IKZF2','IKZF3','IKZF4','IKZF5','IL10','IL1B','IL1RAP','IL4','IL5','IL6','INF2','ING4','INS','INSM1','INSM2','IQGAP1','IRAK1','IRAK2','IRAK3','IRF1','IRF2','IRF2BP1','IRF3','IRF4','IRF5','IRF6','IRF7','IRF8','IRF9','IRX1','IRX2','IRX3','IRX4','IRX5','IRX6','ISL1','ISL2','ISX','ITCH','ITGB2','ITGB3BP','IZUMO2','JADE1','JAK2','JARID2','JAZF1','JDP2','JMY','JRK','JRKL','JUN','JUNB','JUND','JUP','KANSL2','KAT2A','KAT2B','KAT5','KAT6A','KAT6B','KAT7','KAT8','KCMF1','KCNIP3','KDM1A','KDM2A','KDM2B','KDM3A','KDM5A','KDM5B','KDM5C','KDM5D','KEAP1','KIAA1549','KIAA1683','KIAA1841','KIAA1958','KIF15','KIN','KIT','KLF1','KLF10','KLF11','KLF12','KLF13','KLF14','KLF15','KLF16','KLF17','KLF2','KLF3','KLF4','KLF5','KLF6','KLF7','KLF8','KLF9','KMT2A','KMT2B','KMT2C','KMT2D','KNTC1','KRAS','KRBOX1','KRT13','KRTAP5-1','KRTAP5-9','KRTAP9-9','L3MBTL1','L3MBTL3','L3MBTL4','LARP1','LARP1B','LARP4','LARP4B','LARP6','LARP7','LBX1','LBX2','LCOR','LCORL','LEF1','LEUTX','LGALS9','LGR4','LHX1','LHX2','LHX3','LHX4','LHX5','LHX6','LHX8','LHX9','LIG1','LIG3','LIN28A','LIN28B','LIN54','LMO2','LMO4','LMX1A','LMX1B','LPP','LRP5','LRP6','LRP8','LRRFIP1','LRRFIP2','LSR','LTF','LTN1','LYAR','LYL1','LZTFL1','LZTR1','LZTS1','MACC1','MACF1','MAD2L2','MAEL','MAF','MAFA','MAFB','MAFF','MAFG','MAFK','MALT1','MAML1','MAML3','MAP2K5','MAP3K10','MAP3K13','MAP3K7','MAPK1','MAPK10','MAPK11','MAPK14','MAPK3','MAPK8','MAPK9','MATR3','MAVS','MAX','MAZ','MB21D1','MBD1','MBD2','MBD3','MBD4','MBD6','MBNL1','MBNL2','MBNL3','MBTPS2','MDFI','MDM2','MDM4','MECOM','MECP2','MED1','MED12','MED13','MED13L','MED14','MED15','MED16','MED17','MED21','MED23','MED24','MED6','MEF2A','MEF2B','MEF2C','MEF2D','MEIS1','MEIS2','MEIS3','MEN1','MEOX1','MEOX2','MESP1','MESP2','MET','MGA','MGMT','MID2','MIER1','MIER2','MIER3','MINK1','MIS18BP1','MITF','MIXL1','MKL1','MKL2','MKRN1','MKRN2','MKRN3','MKX','MLLT1','MLLT10','MLLT11','MLLT3','MLLT6','MLX','MLXIP','MLXIPL','MN1','MNAT1','MNT','MNX1','MORC3','MPG','MRPL28','MRRF','MSANTD1','MSANTD3','MSANTD4','MSC','MSGN1','MSH2','MSH3','MSH6','MSL3','MSRB2','MST1R','MSX1','MSX2','MTA1','MTA2','MTA3','MTDH','MTERF1','MTERF2','MTERF3','MTERF4','MTF1','MTF2','MTPN','MUS81','MXD1','MXD3','MXD4','MXI1','MYB','MYBBP1A','MYBL1','MYBL2','MYC','MYCBP','MYCL','MYCN','MYD88','MYEF2','MYF5','MYF6','MYNN','MYOCD','MYOD1','MYOG','MYPOP','MYRF','MYRFL','MYSM1','MYT1','MYT1L','MZF1','NAB2','NABP2','NACC2','NAIF1','NANOG','NANOGNB','NANOGP8','NCOA1','NCOA2','NCOA3','NCOA4','NCOA5','NCOA6','NCOA7','NCOR1','NCOR2','NDC80','NDN','NDP','NEB','NEIL1','NEUROD1','NEUROD2','NEUROD4','NEUROD6','NEUROG1','NEUROG2','NEUROG3','NFAM1','NFAT5','NFATC1','NFATC2','NFATC3','NFATC4','NFE2','NFE2L1','NFE2L2','NFE2L3','NFE4','NFIA','NFIB','NFIC','NFIL3','NFIX','NFKB1','NFKB2','NFKBIA','NFKBIB','NFKBID','NFKBIL1','NFKBIZ','NFRKB','NFX1','NFXL1','NFYA','NFYB','NFYC','NHLH1','NHLH2','NKRF','NKX1-1','NKX1-2','NKX2-1','NKX2-2','NKX2-3','NKX2-4','NKX2-5','NKX2-6','NKX2-8','NKX3-1','NKX3-2','NKX6-1','NKX6-2','NKX6-3','NLRC3','NLRC4','NLRC5','NLRP12','NLRP2B','NLRP3','NME2','NOBOX','NOC3L','NOC4L','NOCT','NOD1','NOD2','NODAL','NONO','NOP2','NOTCH1','NOTCH2','NOTCH3','NOTO','NPAS1','NPAS2','NPAS3','NPAS4','NPM1','NR0B1','NR0B2','NR1D1','NR1D2','NR1H2','NR1H3','NR1H4','NR1I2','NR1I3','NR2C1','NR2C2','NR2E1','NR2E3','NR2F1','NR2F2','NR2F6','NR3C1','NR3C2','NR4A1','NR4A2','NR4A3','NR5A1','NR5A2','NR6A1','NRBF2','NRF1','NRG1','NRIP1','NRIP3','NRK','NRL','NSD1','NSD2','NTRK1','NUCKS1','NUFIP1','NUP98','NUPL2']
TFs2Other=['OGG1','OLIG1','OLIG2','OLIG3','ONECUT1','ONECUT2','ONECUT3','OPRD1','OSR1','OSR2','OTOP3','OTP','OTULIN','OTX1','OTX2','OVOL1','OVOL2','OVOL3','PA2G4','PABPN1','PARK7','PARP1','PARP10','PARP12','PATZ1','PAWR','PAX1','PAX2','PAX3','PAX4','PAX5','PAX6','PAX7','PAX8','PAX9','PAXBP1','PAXIP1','PBRM1','PBX1','PBX2','PBX3','PBX4','PCBD1','PCBP1','PCBP2','PCBP3','PCGF2','PCGF6','PCNA','PCSK6','PDCD7','PDS5B','PDX1','PEG3','PELI1','PER1','PER2','PER3','PEX14','PFDN1','PGBD1','PGR','PGS1','PHB','PHB2','PHF1','PHF19 ','PHF20','PHF21A','PHF5A','PHOX2A','PHOX2B','PHTF1','PHTF2','PIAS1','PIAS2','PIAS3','PIAS4','PIDD1','PIKFYVE','PIM1','PIN1','PINK1','PITX1','PITX2','PITX3','PKHD1','PKHD1L1','PKNOX1','PKNOX2','PLA2G10','PLA2G1B','PLAG1','PLAGL1','PLAGL2','PLEK','PLEK2','PLEKHA4','PLPP3','PLSCR1','PLXNA1','PLXNA2','PLXNA3','PLXNA4','PLXNB1','PLXNB2','PLXNB3','PLXNC1','PLXND1','PML','PMS1','PNKP','POGK','POGZ','POLA1','POLB','POLE3','POLE4','POLG','POLH','POLI','POLK','POLL','POLM','POLN','POLQ','POLR1B','POLR2A','POLR2B','POLR2E','POLR2J','POLR2K','POLR2L','POLR3A','POLRMT','POT1','POU1F1','POU2AF1','POU2F1','POU2F2','POU2F3','POU3F1','POU3F2','POU3F3','POU3F4','POU4F1','POU4F2','POU4F3','POU5F1','POU5F1B','POU5F2','POU6F1','POU6F2','PPARA','PPARD','PPARG','PPARGC1A','PPARGC1B','PPP1R10','PPP1R13L','PPP1R21','PPP2R3B','PPP2R5B','PPP3CA','PPRC1','PQBP1','PRAM1','PRAP1','PRB3','PRDM1','PRDM10','PRDM11','PRDM12','PRDM13','PRDM14','PRDM15','PRDM16','PRDM2','PRDM4','PRDM5','PRDM6','PRDM7','PRDM8','PRDM9','PRDX3','PREB','PREX1','PREX2','PRH2','PRIM2','PRIMPOL','PRKCB','PRKCH','PRKCI','PRKCQ','PRKCZ','PRKD1','PRKD2','PRKN','PRMT2','PRMT3','PRNP','PROP1','PROX1','PROX2','PRR12','PRR3','PRRX1','PRRX2','PSMA6','PSMC5','PSMD10','PSMD11','PSMD12','PSPC1','PTCH1','PTEN','PTF1A','PTGIS','PTH','PTTG1','PUM3','PURA','PURB','PURG','PYCARD','PYDC1','PYDC2','QRICH1','RAB11FIP4','RAB7B','RAD21','RAD51','RAG1','RAI1','RAPGEF3','RAPGEF4','RAPGEF5','RARA','RARB','RARG','RAX','RAX2','RB1','RBAK','RBBP7','RBCK1','RBL1','RBL2','RBM10','RBM20','RBM22','RBM26','RBM27','RBM5','RBM6','RBPJ','RBPJL','RBSN','RC3H1','RC3H2','RCAN1','RCOR1','RCOR2','RCOR3','RECQL','REL','RELA','RELB','RELL2','RELN','REPIN1','RERE','REST','REV1','REXO4','RFC1','RFX1','RFX2','RFX3','RFX4','RFX5','RFX6','RFX7','RFX8','RFXANK','RFXAP','RGCC','RGS11','RGS6','RGS7','RGS9','RHEBL1','RHOXF1','RHOXF2','RHOXF2B','RING1','RIOK2','RIPK1','RIPK2','RIPK3','RIPK4','RLF','RLIM','RNASE2','RNASEH1','RNF113A','RNF113B','RNF114','RNF125','RNF138','RNF14','RNF166','RNF2','RNF219','RNF25','RNF31','RNF4','RNF41','RORA','RORB','RORC','RPA2','RPA4','RPL36A','RPL7','RPL7A','RPS27A','RPS6KA4','RPS6KA5','RREB1','RRN3','RUNX1','RUNX1T1','RUNX2','RUNX3','RWDD3','RXRA','RXRB','RXRG','RYBP','S100A12','S100A8','S100A9','SAFB','SAFB2','SALL1','SALL2','SALL3','SALL4','SART1','SATB1','SATB2','SAV1','SCAND1','SCAND2P','SCAPER','SCMH1','SCML1','SCML2','SCML4','SCOC','SCRT1','SCRT2','SCX','SEBOX','SEMA4A','SERTAD1','SERTAD2','SETBP1','SETD6','SETDB1','SETDB2','SF3A2','SF3A3','SFPQ','SFRP4','SFRP5','SFTPB','SGK1','SGK3','SGSM2','SHH','SHOX','SHOX2','SHPRH','SIGIRR','SIK1','SIM1','SIM2','SIN3A','SIN3B','SIRT1','SIRT2','SIRT3','SIRT6','SIVA1','SIX1','SIX2','SIX3','SIX4','SIX5','SIX6','SKI','SKIDA1','SKIL','SKOR1','SKOR2','SLC22A4','SLC26A10','SLC26A3','SLC2A4RG','SLC30A9','SLC39A10','SLC4A10','SMAD1','SMAD2','SMAD3','SMAD4','SMAD5','SMAD6','SMAD7','SMAD9','SMARCA1','SMARCA2','SMARCA4','SMARCA5','SMARCAL1','SMARCB1','SMARCC1','SMARCC2','SMARCD1','SMARCD2','SMARCD3','SMARCE1','SMC3','SMO','SMUG1','SMYD1','SMYD3','SNAI1','SNAI2','SNAI3','SNAPC2','SNAPC4','SNAPC5','SNRPC','SNURF','SNW1','SOGA3','SOHLH1','SOHLH2','SON','SORBS2','SOX1','SOX10','SOX11','SOX12','SOX13','SOX14','SOX15','SOX17','SOX18','SOX2','SOX21','SOX3','SOX30','SOX4','SOX5','SOX6','SOX7','SOX8','SOX9','SP1','SP100','SP110','SP140','SP140L','SP2','SP3','SP4','SP5','SP6','SP7','SP8','SP9','SPDEF','SPEN','SPHK1','SPI1','SPIB','SPIC','SPZ1','SRCAP','SREBF1','SREBF2','SRF','SRY','SSB','SSBP2','SSBP3','SSBP4','SSH1','SSH2','SSH3','SSRP1','SSX1','ST18','STAG1','STAT1','STAT2','STAT3','STAT4','STAT5A','STAT5B','STAT6','STK16','STK3','STK36','STRN','STRN3','SUB1','SUFU','SUMO1','SUPT16H','SUPT4H1','SUPT5H','SUPT6H','SUV39H1','SUV39H2','SUZ12','SYK','T','TAB1','TAB2','TAB3','TADA2A','TADA2B','TADA3','TAF1','TAF10','TAF11','TAF12','TAF13','TAF1A','TAF1B','TAF1C','TAF1L','TAF2','TAF3','TAF4','TAF4B','TAF5','TAF5L','TAF6','TAF6L','TAF7','TAF7L','TAF8','TAF9','TAF9B','TAL1','TAL2','TARDBP','TAX1BP1','TBL1Y','TBP','TBPL1','TBPL2','TBR1','TBX1','TBX10','TBX15','TBX18','TBX19','TBX2','TBX20','TBX21','TBX22','TBX3','TBX4','TBX5','TBX6','TCEA2','TCEAL1','TCEAL7','TCEAL8','TCF12','TCF15','TCF19','TCF20','TCF21','TCF23','TCF24','TCF25','TCF3','TCF4','TCF7','TCF7L1','TCF7L2','TCFL5','TDG','TDP1','TDP2','TEAD1','TEAD2','TEAD3','TEAD4','TEF','TERB1','TERF1','TERF2','TERF2IP','TESMIN','TET1','TET2','TET3','TFAM','TFAP2A','TFAP2B','TFAP2C','TFAP2D','TFAP2E','TFAP4','TFCP2','TFCP2L1','TFDP1','TFDP2','TFDP3','TFE3','TFEB','TFEC','TGFB1','TGFB1I1','TGIF1','TGIF2','TGIF2LX','TGIF2LY','TGS1','THAP1','THAP10','THAP11','THAP12','THAP2','THAP3','THAP4','THAP5','THAP6','THAP7','THAP8','THAP9','THRA','THRAP3','THRB','THYN1','TIA1','TICAM1','TIGD1','TIGD2','TIGD3','TIGD4','TIGD5','TIGD6','TIGD7','TIPARP','TIRAP','TLE1','TLE2','TLE3','TLE4','TLE6','TLR2','TLR3','TLR4','TLR9','TLX1','TLX2','TLX3','TMEM235','TMF1','TNF','TNFAIP3','TNFRSF11A','TNFRSF4','TNFSF11','TNFSF18','TNFSF4','TOE1','TONSL','TOP1','TOP2A','TOP2B','TOPORS','TOX','TOX2','TOX3','TOX4','TP53','TP53BP1','TP63','TP73','TPRX1','TRADD','TRAF1','TRAF2','TRAF3','TRAF4','TRAF5','TRAF6','TRAFD1','TRAK2','TRAPPC9','TRERF1','TREX1','TRIB1','TRIB3','TRIM13','TRIM14','TRIM15','TRIM21','TRIM22','TRIM23','TRIM24','TRIM25','TRIM26','TRIM27','TRIM28','TRIM29','TRIM3','TRIM31','TRIM32','TRIM33','TRIM34','TRIM37','TRIM38','TRIM40','TRIM5','TRIM52','TRIM62','TRIM8','TRIP11','TRIT1','TRMT1','TRMT1L','TRPS1','TRRAP','TSC2','TSC22D1','TSC22D2','TSC22D3','TSC22D4','TSHZ1','TSHZ2','TSHZ3','TSN','TSSK4','TTF1','TUB','TULP4','TUT1','TWIST1','TWIST2','TXK','U2AF1','U2AF1L4','UBA52','UBB','UBC','UBE2K','UBE2N','UBE2V1','UBN1','UBP1','UBR4','UBTF','UBTFL1','UBTFL6','UFL1','UHRF1','UHRF2','UNCX','UNG','UNK','UNKL','USF1','USF2','USF3','USP39','USP7','UTF1','VAV1','VAX1','VAX2','VDR','VEGFA','VENTX','VEZF1','VGLL1','VGLL2','VGLL3','VHLL','VPS72','VSX1','VSX2','WDHD1','WFS1','WIZ','WNT1','WNT10B','WNT2','WNT3A','WNT5A','WNT8B','WRN','WT1','WWOX','WWP2','WWTR1','XBP1','XCL1','XPA','YBX1','YBX2','YBX3','YEATS2','YEATS4','YOD1','YY1','YY2','ZBED1','ZBED2','ZBED3','ZBED4','ZBED5','ZBED6','ZBED6CL','ZBED8','ZBED9','ZBP1','ZBTB1','ZBTB10','ZBTB11','ZBTB12','ZBTB14','ZBTB16','ZBTB17','ZBTB18','ZBTB2','ZBTB20','ZBTB21','ZBTB22','ZBTB24','ZBTB25','ZBTB26','ZBTB3','ZBTB32','ZBTB33','ZBTB34','ZBTB37','ZBTB38','ZBTB39','ZBTB4','ZBTB40','ZBTB41','ZBTB42','ZBTB43','ZBTB44','ZBTB45','ZBTB46','ZBTB47','ZBTB48','ZBTB49','ZBTB5','ZBTB6','ZBTB7A','ZBTB7B','ZBTB7C','ZBTB8A','ZBTB8B','ZBTB9','ZC3H10','ZC3H11A','ZC3H12A','ZC3H13','ZC3H14','ZC3H15','ZC3H18','ZC3H3','ZC3H4','ZC3H6','ZC3H7A','ZC3H7B','ZC3H8','ZC3HAV1','ZCCHC11','ZCCHC6','ZDHHC11','ZDHHC19','ZEB1','ZEB2','ZFAND5','ZFAT','ZFHX2','ZFHX3','ZFHX4','ZFP1','ZFP14','ZFP2','ZFP28','ZFP3','ZFP30','ZFP36','ZFP36L1','ZFP36L2','ZFP37','ZFP41','ZFP42','ZFP57','ZFP62','ZFP64','ZFP69','ZFP69B','ZFP82','ZFP90','ZFP91','ZFP92','ZFPL1','ZFPM1','ZFPM2','ZFR','ZFR2','ZFX','ZFY','ZFYVE26','ZGLP1','ZGPAT','ZHX1','ZHX2','ZHX3','ZIC1','ZIC2','ZIC3','ZIC4','ZIC5','ZIK1','ZIM2','ZIM3','ZKSCAN1','ZKSCAN2','ZKSCAN3','ZKSCAN4','ZKSCAN5','ZKSCAN7','ZKSCAN8','ZMAT1','ZMAT2','ZMAT3','ZMAT4','ZMAT5','ZMYM1','ZMYM2','ZMYM3','ZMYM4','ZMYM5','ZMYM6','ZMYND10','ZNF10','ZNF100','ZNF101','ZNF106','ZNF107','ZNF112','ZNF114','ZNF117','ZNF12','ZNF121','ZNF124','ZNF131','ZNF132','ZNF133','ZNF134','ZNF135','ZNF136','ZNF137P','ZNF138','ZNF14','ZNF140','ZNF141','ZNF142','ZNF143','ZNF146','ZNF148','ZNF154','ZNF155','ZNF157','ZNF16','ZNF160','ZNF165','ZNF169','ZNF17','ZNF174','ZNF175','ZNF177','ZNF18','ZNF180','ZNF181','ZNF182','ZNF184','ZNF185','ZNF189','ZNF19','ZNF195','ZNF197','ZNF2','ZNF20','ZNF200','ZNF202','ZNF205','ZNF207','ZNF208','ZNF211','ZNF212','ZNF213','ZNF214','ZNF215','ZNF217','ZNF219','ZNF22','ZNF221','ZNF222','ZNF223','ZNF224','ZNF225','ZNF226','ZNF227','ZNF229','ZNF23','ZNF230','ZNF232','ZNF233','ZNF234','ZNF235','ZNF236','ZNF239','ZNF24','ZNF248','ZNF25','ZNF250','ZNF251','ZNF253','ZNF254','ZNF256','ZNF257','ZNF26','ZNF260','ZNF263','ZNF264','ZNF266','ZNF267','ZNF268','ZNF273','ZNF274','ZNF275','ZNF276','ZNF277','ZNF28','ZNF280A','ZNF280B','ZNF280C','ZNF280D','ZNF281','ZNF282','ZNF283','ZNF284','ZNF285','ZNF286A','ZNF286B','ZNF287','ZNF292','ZNF296','ZNF3','ZNF30','ZNF300','ZNF302','ZNF304','ZNF311','ZNF316','ZNF317','ZNF318','ZNF319','ZNF32','ZNF320','ZNF322','ZNF324','ZNF324B','ZNF326','ZNF329','ZNF330','ZNF331','ZNF333','ZNF334','ZNF335','ZNF337','ZNF33A','ZNF33B','ZNF34','ZNF341','ZNF343','ZNF345','ZNF346','ZNF347','ZNF35','ZNF350','ZNF354A','ZNF354B','ZNF354C','ZNF355P','ZNF358','ZNF362','ZNF365','ZNF366','ZNF367','ZNF37A','ZNF382','ZNF383','ZNF384','ZNF385A','ZNF385B','ZNF385C','ZNF385D','ZNF391','ZNF394','ZNF395','ZNF396','ZNF397','ZNF398','ZNF404','ZNF407','ZNF408','ZNF41','ZNF410','ZNF414','ZNF415','ZNF416','ZNF417','ZNF418','ZNF419','ZNF420','ZNF423','ZNF425','ZNF426','ZNF428','ZNF429','ZNF43','ZNF430','ZNF431','ZNF432','ZNF433','ZNF436','ZNF438','ZNF439','ZNF44','ZNF440','ZNF441','ZNF442','ZNF443','ZNF444','ZNF445','ZNF446','ZNF449','ZNF45','ZNF451','ZNF454','ZNF460','ZNF461','ZNF462','ZNF467','ZNF468','ZNF469','ZNF470','ZNF471','ZNF473','ZNF474','ZNF479','ZNF48','ZNF480','ZNF483','ZNF484','ZNF485','ZNF486','ZNF487','ZNF488','ZNF490','ZNF491','ZNF492','ZNF493','ZNF496','ZNF497','ZNF500','ZNF501','ZNF502','ZNF503','ZNF506','ZNF507','ZNF510','ZNF511','ZNF512','ZNF512B','ZNF513','ZNF514','ZNF516','ZNF517','ZNF518A','ZNF518B','ZNF519']
TFs3Other=['ZNF521','ZNF524','ZNF525','ZNF526','ZNF527','ZNF528','ZNF529','ZNF530','ZNF532','ZNF534','ZNF536','ZNF540','ZNF541','ZNF542P','ZNF543','ZNF544','ZNF546','ZNF547','ZNF548','ZNF549','ZNF550','ZNF551','ZNF552','ZNF554','ZNF555','ZNF556','ZNF557','ZNF558','ZNF559','ZNF560','ZNF561','ZNF562','ZNF563','ZNF564','ZNF565','ZNF566','ZNF567','ZNF568','ZNF569','ZNF57','ZNF570','ZNF571','ZNF572','ZNF573','ZNF574','ZNF575','ZNF576','ZNF577','ZNF578','ZNF579','ZNF580','ZNF581','ZNF582','ZNF583','ZNF584','ZNF585A','ZNF585B','ZNF586','ZNF587','ZNF587B','ZNF589','ZNF592','ZNF593','ZNF594','ZNF595','ZNF596','ZNF597','ZNF598','ZNF599','ZNF600','ZNF605','ZNF606','ZNF607','ZNF608','ZNF609','ZNF610','ZNF611','ZNF613','ZNF614','ZNF615','ZNF616','ZNF618','ZNF619','ZNF620','ZNF621','ZNF622','ZNF623','ZNF624','ZNF625','ZNF626','ZNF627','ZNF628','ZNF629','ZNF630','ZNF638','ZNF639','ZNF641','ZNF644','ZNF645','ZNF646','ZNF648','ZNF649','ZNF652','ZNF653','ZNF654','ZNF655','ZNF658','ZNF66','ZNF660','ZNF662','ZNF664','ZNF665','ZNF667','ZNF668','ZNF669','ZNF670','ZNF671','ZNF672','ZNF674','ZNF675','ZNF676','ZNF677','ZNF678','ZNF679','ZNF680','ZNF681','ZNF682','ZNF683','ZNF684','ZNF687','ZNF688','ZNF689','ZNF69','ZNF691','ZNF692','ZNF695','ZNF696','ZNF697','ZNF699','ZNF7','ZNF70','ZNF700','ZNF701','ZNF703','ZNF704','ZNF705A','ZNF705B','ZNF705D','ZNF705E','ZNF705G','ZNF706','ZNF707','ZNF708','ZNF709','ZNF71','ZNF710','ZNF711','ZNF713','ZNF714','ZNF716','ZNF717','ZNF718','ZNF720','ZNF721','ZNF724','ZNF726','ZNF727','ZNF728','ZNF729','ZNF73','ZNF730','ZNF732','ZNF735','ZNF736','ZNF737','ZNF738','ZNF74','ZNF740','ZNF746','ZNF747','ZNF749','ZNF750','ZNF75A','ZNF75D','ZNF76','ZNF761','ZNF763','ZNF764','ZNF765','ZNF766','ZNF768','ZNF77','ZNF770','ZNF771','ZNF772','ZNF773','ZNF774','ZNF775','ZNF776','ZNF777','ZNF778','ZNF780A','ZNF780B','ZNF781','ZNF782','ZNF783','ZNF784','ZNF785','ZNF786','ZNF787','ZNF788','ZNF789','ZNF79','ZNF790','ZNF791','ZNF792','ZNF793','ZNF799','ZNF8','ZNF80','ZNF800','ZNF804A','ZNF804B','ZNF805','ZNF808','ZNF81','ZNF812P','ZNF813','ZNF814','ZNF816','ZNF821','ZNF823','ZNF827','ZNF829','ZNF83','ZNF830','ZNF831','ZNF835','ZNF836','ZNF837','ZNF839','ZNF84','ZNF841','ZNF843','ZNF844','ZNF845','ZNF846','ZNF85','ZNF850','ZNF852','ZNF853','ZNF860','ZNF862','ZNF865','ZNF878','ZNF879','ZNF880','ZNF883','ZNF888','ZNF891','ZNF90','ZNF91','ZNF92','ZNF93','ZNF98','ZNF99','ZNFX1','ZRANB2','ZRSR1','ZRSR2','ZSCAN1','ZSCAN10','ZSCAN12','ZSCAN16','ZSCAN18','ZSCAN2','ZSCAN20','ZSCAN21','ZSCAN22','ZSCAN23','ZSCAN25','ZSCAN26','ZSCAN29','ZSCAN30','ZSCAN31','ZSCAN32','ZSCAN4','ZSCAN5A','ZSCAN5B','ZSCAN5C','ZSCAN5DP','ZSCAN9','ZUFSP','ZXDA','ZXDB','ZXDC','ZZZ3']
TFs4Other=['ATF2','ATF6B','ATN1','BHLHA15','BHLHB2','BHLHB4 ','BHLHB5','BHLHB9','C13ORF8','C20ORF194','CDX2','CRAMP1L','ELP3','ERVK3-1','ETF1','FLJ12895','FLJ20321','FLJ25476','FLJ35784','FOXM1','FOXO2','FOXR1','FOXR2','GATA1','GRLF1','KDM4C','KDM4E','KIAA0863','KIAA0961','KLRG1','LBXCOR1','LEF1','LHX7','LOC126295','LOC51058','LOC91661','MGC4728','MYOD1','MYOG','MYRF','NEUROD4','NEUROG1','NFAT2','NFAT4','NFE2L1','NR1A2','NR1B1','NR1B2','NR1B3','NR1C1','NR1C2','NR1C3','NR1F3','NR1I1','NR2A1','NR2B3','NR3A1','NR3A2','NR3B3','OTEX','PEPP-2','PHF2','POU2F1','POU2F2','POU4F3','POU5F1','POU6F1','PRDM3','PRKRIP1','PRKRIR','SETD1A','SMAD8','SMYD2','SOX12','SOX14','SOX15','TBX14','TCF1','TCF17','TCF2','TCF9','TEAD1','TPRXL','UBTFL1','WHSC1','ZBTB14','ZBTB18','ZBTB19','ZBTB21','ZBTB27','ZBTB28','ZBTB35','ZBTB49','ZFP161','ZFP28','ZKSCAN10','ZKSCAN11','ZKSCAN12','ZKSCAN13','ZKSCAN14','ZKSCAN16','ZKSCAN19','ZKSCAN21','ZKSCAN24','ZKSCAN6','ZKSCAN8','ZNF10','ZNF112','ZNF187','ZNF193','ZNF204P','ZNF238','ZNF271','ZNF295','ZNF305','ZNF323','ZNF434','ZNF499','ZNF542','ZNF545','ZNF553','ZNF56','ZNF625','ZNF643','ZNF698','ZNF705F','ZNF751','ZNF752','ZNF753','ZNF785','ZNF806','ZNF816','ZNF833P','ZNF906','ZSCAN1','ZSCAN25','ZSCAN26','ZSCAN31','ZSCAN32','ZSCAN5D','ZSCAN9']
TFsOther=TFs1Other+TFs2Other+TFs3Other+TFs4Other
```

Example 5 (python):
```python
# preserve raw count
adata.layers["counts"] = adata.X.copy()

# normalise and log-transform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key='batch_key')

print(sum(adata.var.highly_variable))
sc.pl.highly_variable_genes(adata)

# freeze the state in `.raw`
adata.raw = adata

# subset hvgs
adata = adata[:, adata.var.highly_variable]

adata=adata.copy()

sc.tl.score_genes_cell_cycle(adata, s_genes=s_genes, g2m_genes=g2m_genes)
#sc.pp.regress_out(adata, ['S_score','G2M_score'])
```

Example 6 (python):
```python
#batch correction
scvi.model.SCVI.setup_anndata(adata, 
                              layer="counts", 
                              batch_key="batch_key",
                              categorical_covariate_keys=['phase'],
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo'])

vae = scvi.model.SCVI(adata, n_hidden = 128, n_latent = 50, n_layers = 3, dispersion = 'gene-batch')
vae.train(max_epochs=50)
    
# Convergence plot
x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
plt.plot(x, vae.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
plt.clf()
```

Example 7 (python):
```python
# add the latent space to adata
adata.obsm["X_scVI"] = vae.get_latent_representation()

sc.tl.score_genes(adata, stress_genes, score_name='stress_score')
sc.tl.score_genes(adata, hb_genes1, score_name='hb1_score')

sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata,min_dist=0.1)
#sc.tl.tsne(adata)
```

Example 8 (python):
```python
sc.pl.umap(adata, color=['log1p_total_counts','n_genes','n_genes_by_counts','total_counts','pct_counts_mt','pct_counts_ribo'],ncols=5)
sc.pl.umap(adata, color=['scrublet_score','stress_score', 'hb1_score','phase'])
sc.pl.umap(adata, color=['donor','age','region','phase','cell_or_nuclei','modality','kit_10x'],wspace=0.8)
Genes=['TTN','TNNT2','MYOM1','LDB3','ALDH1A2','TNNT1','KRT19','UPK3B','BNC1','COL6A3','FBLN1','PCOLCE','LRRC17','PCDH18','EMCN','ECSCR','PECAM1','CDH5','ESAM', 'LAPTM5','PTPRC','LCP1','CD53','WAS','PHOX2B','NRXN1','STMN2','GRIK2','L1CAM']
sc.pl.umap(adata, color=Genes)
```

Example 9 (python):
```python
sc.pl.umap(adata, color=['LYVE1','PECAM1'])
```

Example 10 (python):
```python
for k in list(set(adata.obs.sangerID)):
    sc.pl.umap(adata, color='sangerID',groups=k)
```

Example 11 (python):
```python
for k in list(set(adata.obs.region)):
    sc.pl.umap(adata, color='region',groups=k)
```

Example 12 (python):
```python
sc.tl.leiden(adata, resolution=1, key_added='leiden')
sc.pl.umap(adata, color='leiden',legend_loc='on data')
```

Example 13 (python):
```python
CardiomyocyteClusters=['1','15','28','20','5','11','3']
MesenchymalClusters=['2','23','10','0','12','6','33','14','27','29','4','17','22','24','26','17','13','16','31']
EndotheliumClusters=['8','35','7','25']
EpicardiumClusters=['19']
ImmuneClusters=['9','18','37','21','34','38','36']
NeuralClusters=['32','30']
adata.obs['coarse_grain']= 'na'
adata.obs['coarse_grain'][[x for x in adata.obs_names if adata.obs.leiden[x] in CardiomyocyteClusters]]='Cardiomyocytes'
adata.obs['coarse_grain'][[x for x in adata.obs_names if adata.obs.leiden[x] in MesenchymalClusters]]='Mesenchymal'
adata.obs['coarse_grain'][[x for x in adata.obs_names if adata.obs.leiden[x] in EndotheliumClusters]]='Endothelium'
adata.obs['coarse_grain'][[x for x in adata.obs_names if adata.obs.leiden[x] in EpicardiumClusters]]='Epicardium'
adata.obs['coarse_grain'][[x for x in adata.obs_names if adata.obs.leiden[x] in ImmuneClusters]]='Immune'
adata.obs['coarse_grain'][[x for x in adata.obs_names if adata.obs.leiden[x] in NeuralClusters]]='Neural'
sc.pl.umap(adata, color='coarse_grain')
```

Example 14 (python):
```python
#Ito cells
sc.pl.umap(adata, color=['OIT3','CRHBP','FCN3'])

#Parathyroid cells
sc.pl.umap(adata, color=['PTH','MAFB','GATA3'])

#Spleen
sc.pl.umap(adata, color=['VPREB3','MZB1','CD79B'])

#Hepatocytes
sc.pl.umap(adata, color=['ALB','APOA1','TTR'])

#Melanocytes
sc.pl.umap(adata, color=['MLANA','DCT','DCD','ALX1','CCER2'])

#Adipocytes
sc.pl.umap(adata, color=['GPAM','FASN','ADIPOQ'])

#Tenocytes
sc.pl.umap(adata, color=['TNMD','SCX','TNC','CNMD','THBS4'])

#Chondrocyte markers
sc.pl.umap(adata, color=['MATN4','ACAN','CHAD','COL6A3','COL9A1','COL9A3','HAPLN1','CYTL1','COL2A1','COMP','MKX','SOX9','CD74','EFEMP1','TGM2'])

#AT/12 cells
sc.pl.umap(adata, color=['EPCAM','HOPX','LAMP3','SFTPC'])

#Neural crest cells
sc.pl.umap(adata, color=['FOXD3','SNAI1','SNAI2','SOX8','SOX9','SOX10','TFAP2A'])

#Placenta cytotrophoblasts
sc.pl.umap(adata, color=['PAGE4','GSTA3'])

#Esophagus epithelial
sc.pl.umap(adata, color=['SOX2','TP63','TRH'])
```

Example 15 (python):
```python
adata.write('./Objects/T21_multiome_RNA_foetal_scVIProcessed.h5ad')
```

Example 16 (python):
```python
adata=sc.read('./Objects/T21_multiome_RNA_foetal.h5ad')
print(adata.shape)

bdata=sc.read('./Objects/T21_multiome_RNA_foetal_scVIProcessed.h5ad')
print(bdata.shape)
#Select Cardiomyocyte, Mesenchymal, Endothelium, Epicardium, Immune, Neural
bdata=bdata[bdata.obs.coarse_grain=='Immune',:].copy()

adata=adata[[x for x in adata.obs_names if x in bdata.obs_names],:].copy()
print(adata.shape)
del bdata

# set gene name as var_names 
adata.var['gene_id']=adata.var_names.copy()
adata.var.set_index('gene_name',inplace=True)
adata.var_names=adata.var_names.astype('str')
adata.var_names_make_unique()

cell_cycle_genes = ['MCM5','PCNA','TYMS','FEN1','MCM2','MCM4','RRM1','UNG','GINS2','MCM6','CDCA7','DTL','PRIM1','UHRF1','MLF1IP','HELLS','RFC2','RPA2','NASP','RAD51AP1','GMNN','WDR76','SLBP','CCNE2','UBR7','POLD3','MSH2','ATAD2','RAD51','RRM2','CDC45','CDC6','EXO1','TIPIN','DSCC1','BLM','CASP8AP2','USP1','CLSPN','POLA1','CHAF1B','BRIP1','E2F8','HMGB2','CDK1','NUSAP1','UBE2C','BIRC5','TPX2','TOP2A','NDC80','CKS2','NUF2','CKS1B','MKI67','TMPO','CENPF','TACC3','FAM64A','SMC4','CCNB2','CKAP2L','CKAP2','AURKB','BUB1','KIF11','ANP32E','TUBB4B','GTSE1','KIF20B','HJURP','CDCA3','HN1','CDC20','TTK','CDC25C','KIF2C','RANGAP1','NCAPD2','DLGAP5','CDCA2','CDCA8','ECT2','KIF23','HMMR','AURKA','PSRC1','ANLN','LBR','CKAP5','CENPE','CTCF','NEK2','G2E3','GAS2L3','CBX5','CENPA']
s_genes = cell_cycle_genes[:43]
s_genes = [x for x in s_genes if x in adata.var_names]
print('S genes:', s_genes, '\n')
g2m_genes = cell_cycle_genes[43:]
g2m_genes = [x for x in g2m_genes if x in adata.var_names]
print('G2 genes:', g2m_genes, '\n')
cell_cycle_genes = [x for x in cell_cycle_genes if x in adata.var_names]

hb_genes1 = ['HBB','HBD','HBG1','HBG2','HBE1','HBZ','HBM','HBA2','HBA1', 'HBQ1']
hb_genes1 = [x for x in hb_genes1 if x in adata.var_names]
print('Hemoglobin genes set 1:', hb_genes1, '\n')

hb_genes2 = ['AHSP','GYPA','RHAG','GFI1B','KEL','EPB42','ALAS2','TSPO2','GYPB','GYPE','SPTA1','RHCE','KLF1','RHD','SLC4A1','SNX22','SQSTM1','SOD1','FTH1','MAP1LC3B','SLC25A37','TRIM58','HMBS']
hb_genes2 = [x for x in hb_genes2 if x in adata.var_names]
print('Hemoglobin genes set 2:', hb_genes2, '\n')

mito_genes=['MT-ND1','MT-ND2','MT-CO1','MT-CO2','MT-ATP8','MT-ATP6','MT-CO3','MT-ND3','MT-ND4L', 'MT-ND4', 'MT-ND5', 'MT-ND6', 'MT-CYB']
mito_genes = [x for x in mito_genes if x in adata.var_names]
print('Mitochondrial genes:', mito_genes, '\n')

ribo_genes = ['RPLP0','RPLP1','RPLP2','RPL3','RPL3L','RPL4','RPL5','RPL6','RPL7','RPL7A','RPL7L1','RPL8','RPL9','RPL10','RPL10A','RPL10L','RPL11','RPL12','RPL13','RPL13A','RPL14','RPL15','RPL17','RPL18','RPL18A','RPL19','RPL21','RPL22','RPL22L1','RPL23','RPL23A','RPL24','RPL26','RPL26L1','RPL27','RPL27A','RPL28','RPL29','RPL30','RPL31','RPL32','RPL34','RPL35','RPL35A','RPL36','RPL36A','RPL36AL','RPL37','RPL37A','RPL38','RPL39','RPL39L','UBA52','RPL41','RPSA','RPS2','RPS3','RPS3A','RPS4X','RPS4Y1','RPS4Y2','RPS5','RPS6','RPS7','RPS8','RPS9','RPS10','RPS11','RPS12','RPS13','RPS14','RPS15','RPS15A','RPS16','RPS17','RPS18','RPS19','RPS20','RPS21','RPS23','RPS24','RPS25','RPS26','RPS27','RPS27A','RPS27L','RPS28','RPS29','FAU']
ribo_genes = [x for x in ribo_genes if x in adata.var_names]
print('Ribo genes:', ribo_genes, '\n')

stress_genes=['FOS','FOSB','JUN','JUNB','JUND','ZFP36','SOCS3','CEBPD','CEBPB','ATF3','EGR1','HSPA1A','HSPA1B','HSP90AA1','HSP90AB1','HSPA8','HSPB1','HSPE1','IER3','IER2','BTG1','BTG2','DUSP1']
stress_genes = [x for x in stress_genes if x in adata.var_names]
print('Tissue dissociation stress associated genes:', stress_genes, '\n')

# preserve raw count
adata.layers["counts"] = adata.X.copy()

# normalise and log-transform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key='batch_key')

print(sum(adata.var.highly_variable))
sc.pl.highly_variable_genes(adata)

# freeze the state in `.raw`
adata.raw = adata

# subset hvgs
adata = adata[:, adata.var.highly_variable]

adata=adata.copy()

sc.tl.score_genes_cell_cycle(adata, s_genes=s_genes, g2m_genes=g2m_genes)
#sc.pp.regress_out(adata, ['S_score','G2M_score'])

#batch correction
scvi.model.SCVI.setup_anndata(adata, 
                              layer="counts", 
                              batch_key="batch_key",
                              categorical_covariate_keys=['phase'],
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo'])

vae = scvi.model.SCVI(adata, n_hidden = 128, n_latent = 50, n_layers = 3, dispersion = 'gene-batch')
vae.train(max_epochs=50)
    
# Convergence plot
x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
plt.plot(x, vae.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
plt.clf()

# add the latent space to adata
adata.obsm["X_scVI"] = vae.get_latent_representation()

#sc.tl.score_genes(adata, stress_genes, score_name='stress_score')

sc.tl.score_genes(adata, hb_genes1, score_name='hb1_score')

sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata,min_dist=0.1)
#sc.tl.tsne(adata)

sc.pl.umap(adata, color=['log1p_total_counts','n_genes','n_genes_by_counts','total_counts','pct_counts_mt','pct_counts_ribo'],ncols=5)
sc.pl.umap(adata, color=['scrublet_score', 'hb1_score','phase'])
sc.pl.umap(adata, color=['donor','age','region','phase','cell_or_nuclei','modality','kit_10x'],wspace=0.8)
```

Example 17 (python):
```python
sc.pl.umap(adata, color=hb_genes1)
sc.pl.umap(adata, color=hb_genes2)
```

Example 18 (python):
```python
Genes=['TTN','TNNT2','MYOM1','LDB3','ALDH1A2','TNNT1','KRT19','UPK3B','BNC1','COL6A3','FBLN1','PCOLCE','LRRC17','PCDH18','EMCN','ECSCR','PECAM1','CDH5','ESAM', 'LAPTM5','PTPRC','LCP1','CD53','WAS','PHOX2B','NRXN1','STMN2','GRIK2','L1CAM']
sc.pl.umap(adata, color=Genes)

#Ito cells
sc.pl.umap(adata, color=['OIT3','CRHBP','FCN3'])

#Parathyroid cells
sc.pl.umap(adata, color=['PTH','MAFB','GATA3'])

#Spleen
sc.pl.umap(adata, color=['VPREB3','MZB1','CD79B'])

#Hepatocytes
sc.pl.umap(adata, color=['ALB','APOA1','TTR'])

#Melanocytes
sc.pl.umap(adata, color=['MLANA','DCT','DCD','ALX1','CCER2'])

#Adipocytes
sc.pl.umap(adata, color=['GPAM','FASN','ADIPOQ'])

#Tenocytes
sc.pl.umap(adata, color=['TNMD','SCX','TNC','CNMD','THBS4'])

#Chondrocyte markers
sc.pl.umap(adata, color=['MATN4','ACAN','CHAD','COL6A3','COL9A1','COL9A3','HAPLN1','CYTL1','COL2A1','COMP','MKX','SOX9','CD74','EFEMP1','TGM2'])

#AT/12 cells
sc.pl.umap(adata, color=['EPCAM','HOPX','LAMP3','SFTPC'])

#Neural crest cells
sc.pl.umap(adata, color=['FOXD3','SNAI1','SNAI2','SOX8','SOX9','SOX10','TFAP2A'])

#Placenta cytotrophoblasts
sc.pl.umap(adata, color=['PAGE4','GSTA3'])

#Esophagus epithelial
sc.pl.umap(adata, color=['SOX2','TP63','TRH'])
```

Example 19 (python):
```python
sc.tl.leiden(adata, resolution=0.75, key_added='leiden')  #Cluster 11 CMs, cluster 19 RBCs
sc.pl.umap(adata, color='leiden',legend_loc='on data')
```

Example 20 (python):
```python
sc.pl.dotplot(adata, ['PTPRC','MYL7','NPPA','TNNT2'] ,groupby='leiden',standard_scale='var',cmap='PRGn')
sc.pl.umap(adata, color=['PTPRC','MYL7','NPPA','TNNT2'])
```

Example 21 (python):
```python
#this block is to see where the cells over a certain threshold with TNNT2 expression lies in the UMAP
#enter what gene to set threshold for
#when running this box for the first time, comment the starting with 'del'
#then uncomment it for the other ones --- !!!

sc.pl.umap(adata, color=['TNNT2'])
CellName='Cardio_toremove'
gene='TNNT2'
exp_thresh=1.5
gene_pos=adata.var_names.get_loc(gene)
adata_sel = adata[adata.X[:,gene_pos]>exp_thresh].copy()

l=adata_sel.obs_names
set(l).intersection(adata.obs_names)
print(len(l))

#del adata.obs[CellName]
adata.obs.loc[set(l).intersection(adata.obs_names),CellName]='Yes'
sc.settings.set_figure_params(dpi=200, facecolor='white', color_map = 'RdPu',)
sc.pl.umap(adata, color=[CellName], groups='Yes',size=10, wspace=0.45, legend_fontsize=5, legend_loc='on data',frameon=False)
```

Example 22 (python):
```python
bdata = adata[adata.obs['leiden'].isin(['11','19'])].copy()
bdatanames=bdata.obs_names #this is for the obvious clusters to remove directly
cdatanames=adata_sel.obs_names #this is from the removed cells above

p=set(cdatanames)
r=set(bdatanames)
p.update(r)
p=list(p)

print(len(p))
np.savetxt('./Trisomy/RemovedCells/RBCandCardioRemovedFromImmune.txt', p, delimiter="\n", fmt="%s")
```

Example 23 (python):
```python
adata=sc.read('./Objects/T21_multiome_RNA_foetal.h5ad')
print(adata.shape)

bdata=sc.read('./Objects/T21_multiome_RNA_foetal_scVIProcessed.h5ad')
print(bdata.shape)
#Select Cardiomyocyte, Mesenchymal, Endothelium, Epicardium, Immune, Neural
bdata=bdata[bdata.obs.coarse_grain=='Epicardium',:].copy()

adata=adata[[x for x in adata.obs_names if x in bdata.obs_names],:].copy()
print(adata.shape)
del bdata

# set gene name as var_names 
adata.var['gene_id']=adata.var_names.copy()
adata.var.set_index('gene_name',inplace=True)
adata.var_names=adata.var_names.astype('str')
adata.var_names_make_unique()

cell_cycle_genes = ['MCM5','PCNA','TYMS','FEN1','MCM2','MCM4','RRM1','UNG','GINS2','MCM6','CDCA7','DTL','PRIM1','UHRF1','MLF1IP','HELLS','RFC2','RPA2','NASP','RAD51AP1','GMNN','WDR76','SLBP','CCNE2','UBR7','POLD3','MSH2','ATAD2','RAD51','RRM2','CDC45','CDC6','EXO1','TIPIN','DSCC1','BLM','CASP8AP2','USP1','CLSPN','POLA1','CHAF1B','BRIP1','E2F8','HMGB2','CDK1','NUSAP1','UBE2C','BIRC5','TPX2','TOP2A','NDC80','CKS2','NUF2','CKS1B','MKI67','TMPO','CENPF','TACC3','FAM64A','SMC4','CCNB2','CKAP2L','CKAP2','AURKB','BUB1','KIF11','ANP32E','TUBB4B','GTSE1','KIF20B','HJURP','CDCA3','HN1','CDC20','TTK','CDC25C','KIF2C','RANGAP1','NCAPD2','DLGAP5','CDCA2','CDCA8','ECT2','KIF23','HMMR','AURKA','PSRC1','ANLN','LBR','CKAP5','CENPE','CTCF','NEK2','G2E3','GAS2L3','CBX5','CENPA']
s_genes = cell_cycle_genes[:43]
s_genes = [x for x in s_genes if x in adata.var_names]
print('S genes:', s_genes, '\n')
g2m_genes = cell_cycle_genes[43:]
g2m_genes = [x for x in g2m_genes if x in adata.var_names]
print('G2 genes:', g2m_genes, '\n')
cell_cycle_genes = [x for x in cell_cycle_genes if x in adata.var_names]

hb_genes1 = ['HBB','HBD','HBG1','HBG2','HBE1','HBZ','HBM','HBA2','HBA1', 'HBQ1']
hb_genes1 = [x for x in hb_genes1 if x in adata.var_names]
print('Hemoglobin genes set 1:', hb_genes1, '\n')

hb_genes2 = ['AHSP','GYPA','RHAG','GFI1B','KEL','EPB42','ALAS2','TSPO2','GYPB','GYPE','SPTA1','RHCE','KLF1','RHD','SLC4A1','SNX22','SQSTM1','SOD1','FTH1','MAP1LC3B','SLC25A37','TRIM58','HMBS']
hb_genes2 = [x for x in hb_genes2 if x in adata.var_names]
print('Hemoglobin genes set 2:', hb_genes2, '\n')

mito_genes=['MT-ND1','MT-ND2','MT-CO1','MT-CO2','MT-ATP8','MT-ATP6','MT-CO3','MT-ND3','MT-ND4L', 'MT-ND4', 'MT-ND5', 'MT-ND6', 'MT-CYB']
mito_genes = [x for x in mito_genes if x in adata.var_names]
print('Mitochondrial genes:', mito_genes, '\n')

ribo_genes = ['RPLP0','RPLP1','RPLP2','RPL3','RPL3L','RPL4','RPL5','RPL6','RPL7','RPL7A','RPL7L1','RPL8','RPL9','RPL10','RPL10A','RPL10L','RPL11','RPL12','RPL13','RPL13A','RPL14','RPL15','RPL17','RPL18','RPL18A','RPL19','RPL21','RPL22','RPL22L1','RPL23','RPL23A','RPL24','RPL26','RPL26L1','RPL27','RPL27A','RPL28','RPL29','RPL30','RPL31','RPL32','RPL34','RPL35','RPL35A','RPL36','RPL36A','RPL36AL','RPL37','RPL37A','RPL38','RPL39','RPL39L','UBA52','RPL41','RPSA','RPS2','RPS3','RPS3A','RPS4X','RPS4Y1','RPS4Y2','RPS5','RPS6','RPS7','RPS8','RPS9','RPS10','RPS11','RPS12','RPS13','RPS14','RPS15','RPS15A','RPS16','RPS17','RPS18','RPS19','RPS20','RPS21','RPS23','RPS24','RPS25','RPS26','RPS27','RPS27A','RPS27L','RPS28','RPS29','FAU']
ribo_genes = [x for x in ribo_genes if x in adata.var_names]
print('Ribo genes:', ribo_genes, '\n')

stress_genes=['FOS','FOSB','JUN','JUNB','JUND','ZFP36','SOCS3','CEBPD','CEBPB','ATF3','EGR1','HSPA1A','HSPA1B','HSP90AA1','HSP90AB1','HSPA8','HSPB1','HSPE1','IER3','IER2','BTG1','BTG2','DUSP1']
stress_genes = [x for x in stress_genes if x in adata.var_names]
print('Tissue dissociation stress associated genes:', stress_genes, '\n')

# preserve raw count
adata.layers["counts"] = adata.X.copy()

# normalise and log-transform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key='batch_key')

print(sum(adata.var.highly_variable))
sc.pl.highly_variable_genes(adata)

# freeze the state in `.raw`
adata.raw = adata

# subset hvgs
adata = adata[:, adata.var.highly_variable]

adata=adata.copy()

sc.tl.score_genes_cell_cycle(adata, s_genes=s_genes, g2m_genes=g2m_genes)
#sc.pp.regress_out(adata, ['S_score','G2M_score'])

#batch correction
scvi.model.SCVI.setup_anndata(adata, 
                              layer="counts", 
                              batch_key="batch_key",
                              categorical_covariate_keys=['phase'],
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo'])

vae = scvi.model.SCVI(adata, n_hidden = 128, n_latent = 50, n_layers = 3, dispersion = 'gene-batch')
vae.train(max_epochs=100)
    
# Convergence plot
x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
plt.plot(x, vae.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
plt.clf()

# add the latent space to adata
adata.obsm["X_scVI"] = vae.get_latent_representation()

#sc.tl.score_genes(adata, stress_genes, score_name='stress_score')

sc.tl.score_genes(adata, hb_genes1, score_name='hb1_score')

sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata,min_dist=0.1)
#sc.tl.tsne(adata)

sc.pl.umap(adata, color=['log1p_total_counts','n_genes','n_genes_by_counts','total_counts','pct_counts_mt','pct_counts_ribo'],ncols=5)
sc.pl.umap(adata, color=['scrublet_score', 'hb1_score','phase'])
sc.pl.umap(adata, color=['donor','age','region','phase','cell_or_nuclei','modality','kit_10x'],wspace=0.8)
```

Example 24 (python):
```python
sc.pl.umap(adata, color=['scrublet_score', 'hb1_score','phase'])
sc.pl.umap(adata, color=['donor','age','region','phase','cell_or_nuclei','modality','kit_10x'],wspace=0.8)
```

Example 25 (python):
```python
#Hepatocyte
sc.pl.umap(adata, color=['ALB','APOA1','TTR'])
```

Example 26 (python):
```python
#Epi markers
sc.pl.umap(adata, color=['MSLN','TCF21','POSTN','ITLN1','DCN','TBX18','BNC1','GPC6'])


sc.pl.umap(adata, color=hb_genes1)
sc.pl.umap(adata, color=hb_genes2)

Genes=['TTN','TNNT2','MYOM1','LDB3','ALDH1A2','TNNT1','KRT19','UPK3B','BNC1','COL6A3','FBLN1','PCOLCE','LRRC17','PCDH18','EMCN','ECSCR','PECAM1','CDH5','ESAM', 'LAPTM5','PTPRC','LCP1','CD53','WAS','PHOX2B','NRXN1','STMN2','GRIK2','L1CAM']
sc.pl.umap(adata, color=Genes)
```

Example 27 (python):
```python
sc.pl.umap(adata, color='hb1_score')
```

Example 28 (python):
```python
sc.pl.umap(adata, color=hb_genes1)
sc.pl.umap(adata, color=hb_genes2)

for k in list(set(adata.obs.region)):
    sc.pl.umap(adata, color='region',groups=k)
    
for k in list(set(adata.obs.sangerID)):
    sc.pl.umap(adata, color='sangerID',groups=k)
```

Example 29 (python):
```python
#Ito cells
sc.pl.umap(adata, color=['OIT3','CRHBP','FCN3'])

#Parathyroid cells
sc.pl.umap(adata, color=['PTH','MAFB','GATA3'])

#Spleen
sc.pl.umap(adata, color=['VPREB3','MZB1','CD79B'])

#Hepatocytes
sc.pl.umap(adata, color=['ALB','APOA1','TTR'])

#Melanocytes
sc.pl.umap(adata, color=['MLANA','DCT','DCD','ALX1','CCER2'])

#Adipocytes
sc.pl.umap(adata, color=['GPAM','FASN','ADIPOQ'])

#Tenocytes
sc.pl.umap(adata, color=['TNMD','SCX','TNC','CNMD','THBS4'])

#Chondrocyte markers
sc.pl.umap(adata, color=['MATN4','ACAN','CHAD','COL6A3','COL9A1','COL9A3','HAPLN1','CYTL1','COL2A1','COMP','MKX','SOX9','CD74','EFEMP1','TGM2'])

#AT/12 cells
sc.pl.umap(adata, color=['EPCAM','HOPX','LAMP3','SFTPC'])

#Neural crest cells
sc.pl.umap(adata, color=['FOXD3','SNAI1','SNAI2','SOX8','SOX9','SOX10','TFAP2A'])

#Placenta cytotrophoblasts
sc.pl.umap(adata, color=['PAGE4','GSTA3'])

#Esophagus epithelial
sc.pl.umap(adata, color=['SOX2','TP63','TRH'])
```

Example 30 (python):
```python
sc.tl.leiden(adata, resolution=0.05, key_added='leiden') #cluster 12 is RBCs, cluster 13 and 7 are CMs
sc.pl.umap(adata, color='leiden',legend_loc='on data')

ClusterOfInterest='2'
sc.tl.rank_genes_groups(adata, 'leiden', method='t-test', key_added = "t-test")
DEGenesForCluster = [x[ClusterOfInterest] for x in adata.uns['t-test']['names'][:50]]
sc.pl.dotplot(adata, DEGenesForCluster ,groupby='leiden',standard_scale='var',cmap='PRGn')
#sc.pl.umap(adata, color=DEGenesForCluster)
```

Example 31 (python):
```python
#this block is to see where the cells over a certain threshold with TNNT2 expression lies in the UMAP
#enter what gene to set threshold for
#when running this box for the first time, comment the starting with 'del'
#then uncomment it for the other ones --- !!!

sc.pl.umap(adata, color=['TNNT2'])
CellName='Cardio_toremove'
gene='TNNT2'
exp_thresh=1.3
gene_pos=adata.var_names.get_loc(gene)
adata_sel = adata[adata.X[:,gene_pos]>exp_thresh].copy()

l=adata_sel.obs_names
set(l).intersection(adata.obs_names)
print(len(l))

del adata.obs[CellName]
adata.obs.loc[set(l).intersection(adata.obs_names),CellName]='Yes'
sc.settings.set_figure_params(dpi=200, facecolor='white', color_map = 'RdPu',)
sc.pl.umap(adata, color=[CellName], groups='Yes',size=10, wspace=0.45, legend_fontsize=5, legend_loc='on data',frameon=False)
```

Example 32 (python):
```python
bdatanames=adata_sel.obs_names #this is for the obvious clusters to remove directly


print(len(bdatanames))
np.savetxt('./Trisomy/RemovedCells/RBCandCardioRemovedFromEpi.txt', bdatanames, delimiter="\n", fmt="%s")
```

Example 33 (python):
```python
adata=sc.read('./Objects/T21_multiome_RNA_foetal.h5ad')
print(adata.shape)

bdata=sc.read('./Objects/T21_multiome_RNA_foetal_scVIProcessed.h5ad')
print(bdata.shape)
#Select Cardiomyocyte, Mesenchymal, Endothelium, Epicardium, Immune, Neural
bdata=bdata[bdata.obs.coarse_grain=='Endothelium',:].copy()

adata=adata[[x for x in adata.obs_names if x in bdata.obs_names],:].copy()
print(adata.shape)
del bdata

# set gene name as var_names 
adata.var['gene_id']=adata.var_names.copy()
adata.var.set_index('gene_name',inplace=True)
adata.var_names=adata.var_names.astype('str')
adata.var_names_make_unique()

cell_cycle_genes = ['MCM5','PCNA','TYMS','FEN1','MCM2','MCM4','RRM1','UNG','GINS2','MCM6','CDCA7','DTL','PRIM1','UHRF1','MLF1IP','HELLS','RFC2','RPA2','NASP','RAD51AP1','GMNN','WDR76','SLBP','CCNE2','UBR7','POLD3','MSH2','ATAD2','RAD51','RRM2','CDC45','CDC6','EXO1','TIPIN','DSCC1','BLM','CASP8AP2','USP1','CLSPN','POLA1','CHAF1B','BRIP1','E2F8','HMGB2','CDK1','NUSAP1','UBE2C','BIRC5','TPX2','TOP2A','NDC80','CKS2','NUF2','CKS1B','MKI67','TMPO','CENPF','TACC3','FAM64A','SMC4','CCNB2','CKAP2L','CKAP2','AURKB','BUB1','KIF11','ANP32E','TUBB4B','GTSE1','KIF20B','HJURP','CDCA3','HN1','CDC20','TTK','CDC25C','KIF2C','RANGAP1','NCAPD2','DLGAP5','CDCA2','CDCA8','ECT2','KIF23','HMMR','AURKA','PSRC1','ANLN','LBR','CKAP5','CENPE','CTCF','NEK2','G2E3','GAS2L3','CBX5','CENPA']
s_genes = cell_cycle_genes[:43]
s_genes = [x for x in s_genes if x in adata.var_names]
print('S genes:', s_genes, '\n')
g2m_genes = cell_cycle_genes[43:]
g2m_genes = [x for x in g2m_genes if x in adata.var_names]
print('G2 genes:', g2m_genes, '\n')
cell_cycle_genes = [x for x in cell_cycle_genes if x in adata.var_names]

hb_genes1 = ['HBB','HBD','HBG1','HBG2','HBE1','HBZ','HBM','HBA2','HBA1', 'HBQ1']
hb_genes1 = [x for x in hb_genes1 if x in adata.var_names]
print('Hemoglobin genes set 1:', hb_genes1, '\n')

hb_genes2 = ['AHSP','GYPA','RHAG','GFI1B','KEL','EPB42','ALAS2','TSPO2','GYPB','GYPE','SPTA1','RHCE','KLF1','RHD','SLC4A1','SNX22','SQSTM1','SOD1','FTH1','MAP1LC3B','SLC25A37','TRIM58','HMBS']
hb_genes2 = [x for x in hb_genes2 if x in adata.var_names]
print('Hemoglobin genes set 2:', hb_genes2, '\n')

mito_genes=['MT-ND1','MT-ND2','MT-CO1','MT-CO2','MT-ATP8','MT-ATP6','MT-CO3','MT-ND3','MT-ND4L', 'MT-ND4', 'MT-ND5', 'MT-ND6', 'MT-CYB']
mito_genes = [x for x in mito_genes if x in adata.var_names]
print('Mitochondrial genes:', mito_genes, '\n')

ribo_genes = ['RPLP0','RPLP1','RPLP2','RPL3','RPL3L','RPL4','RPL5','RPL6','RPL7','RPL7A','RPL7L1','RPL8','RPL9','RPL10','RPL10A','RPL10L','RPL11','RPL12','RPL13','RPL13A','RPL14','RPL15','RPL17','RPL18','RPL18A','RPL19','RPL21','RPL22','RPL22L1','RPL23','RPL23A','RPL24','RPL26','RPL26L1','RPL27','RPL27A','RPL28','RPL29','RPL30','RPL31','RPL32','RPL34','RPL35','RPL35A','RPL36','RPL36A','RPL36AL','RPL37','RPL37A','RPL38','RPL39','RPL39L','UBA52','RPL41','RPSA','RPS2','RPS3','RPS3A','RPS4X','RPS4Y1','RPS4Y2','RPS5','RPS6','RPS7','RPS8','RPS9','RPS10','RPS11','RPS12','RPS13','RPS14','RPS15','RPS15A','RPS16','RPS17','RPS18','RPS19','RPS20','RPS21','RPS23','RPS24','RPS25','RPS26','RPS27','RPS27A','RPS27L','RPS28','RPS29','FAU']
ribo_genes = [x for x in ribo_genes if x in adata.var_names]
print('Ribo genes:', ribo_genes, '\n')

stress_genes=['FOS','FOSB','JUN','JUNB','JUND','ZFP36','SOCS3','CEBPD','CEBPB','ATF3','EGR1','HSPA1A','HSPA1B','HSP90AA1','HSP90AB1','HSPA8','HSPB1','HSPE1','IER3','IER2','BTG1','BTG2','DUSP1']
stress_genes = [x for x in stress_genes if x in adata.var_names]
print('Tissue dissociation stress associated genes:', stress_genes, '\n')

# preserve raw count
adata.layers["counts"] = adata.X.copy()

# normalise and log-transform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key='batch_key')

print(sum(adata.var.highly_variable))
sc.pl.highly_variable_genes(adata)

# freeze the state in `.raw`
adata.raw = adata

# subset hvgs
adata = adata[:, adata.var.highly_variable]

adata=adata.copy()

sc.tl.score_genes_cell_cycle(adata, s_genes=s_genes, g2m_genes=g2m_genes)
#sc.pp.regress_out(adata, ['S_score','G2M_score'])

#batch correction
scvi.model.SCVI.setup_anndata(adata, 
                              layer="counts", 
                              batch_key="batch_key",
                              categorical_covariate_keys=['phase'],
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo'])

vae = scvi.model.SCVI(adata, n_hidden = 128, n_latent = 50, n_layers = 3, dispersion = 'gene-batch')
vae.train(max_epochs=50)
    
# Convergence plot
x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
plt.plot(x, vae.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
plt.clf()

# add the latent space to adata
adata.obsm["X_scVI"] = vae.get_latent_representation()

#sc.tl.score_genes(adata, stress_genes, score_name='stress_score')

sc.tl.score_genes(adata, hb_genes1, score_name='hb1_score')

sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata,min_dist=0.1)
#sc.tl.tsne(adata)

sc.pl.umap(adata, color=['log1p_total_counts','n_genes','n_genes_by_counts','total_counts','pct_counts_mt','pct_counts_ribo'],ncols=5)
sc.pl.umap(adata, color=['scrublet_score', 'hb1_score','phase'])
sc.pl.umap(adata, color=['donor','age','region','phase','cell_or_nuclei','modality','kit_10x'],wspace=0.8)
```

Example 34 (python):
```python
Genes=['TTN','TNNT2','MYOM1','LDB3','ALDH1A2','TNNT1','KRT19','UPK3B','BNC1','COL6A3','FBLN1','PCOLCE','LRRC17','PCDH18','EMCN','ECSCR','PECAM1','CDH5','ESAM', 'LAPTM5','PTPRC','LCP1','CD53','WAS','PHOX2B','NRXN1','STMN2','GRIK2','L1CAM']
sc.pl.umap(adata, color=Genes)

#Ito cells
sc.pl.umap(adata, color=['OIT3','CRHBP','FCN3'])

#Parathyroid cells
sc.pl.umap(adata, color=['PTH','MAFB','GATA3'])

#Spleen
sc.pl.umap(adata, color=['VPREB3','MZB1','CD79B'])

#Hepatocytes
sc.pl.umap(adata, color=['ALB','APOA1','TTR'])

#Melanocytes
sc.pl.umap(adata, color=['MLANA','DCT','DCD','ALX1','CCER2'])

#Adipocytes
sc.pl.umap(adata, color=['GPAM','FASN','ADIPOQ'])

#Tenocytes
sc.pl.umap(adata, color=['TNMD','SCX','TNC','CNMD','THBS4'])

#Chondrocyte markers
sc.pl.umap(adata, color=['MATN4','ACAN','CHAD','COL6A3','COL9A1','COL9A3','HAPLN1','CYTL1','COL2A1','COMP','MKX','SOX9','CD74','EFEMP1','TGM2'])

#AT/12 cells
sc.pl.umap(adata, color=['EPCAM','HOPX','LAMP3','SFTPC'])

#Neural crest cells
sc.pl.umap(adata, color=['FOXD3','SNAI1','SNAI2','SOX8','SOX9','SOX10','TFAP2A'])

#Placenta cytotrophoblasts
sc.pl.umap(adata, color=['PAGE4','GSTA3'])

#Esophagus epithelial
sc.pl.umap(adata, color=['SOX2','TP63','TRH'])
```

Example 35 (python):
```python
sc.pl.umap(adata, color=['LYVE1','NPR3','NFATC1','FBLN5','RGCC','HAS2','ASPN','KCNJ2'])
```

Example 36 (python):
```python
sc.pl.umap(adata, color=['MYOM1'])
```

Example 37 (python):
```python
sc.tl.leiden(adata, resolution=0.4, key_added='leiden') #Cluster 14 hepatocytes, cluster 10 and 6 cardios
sc.pl.umap(adata, color='leiden',legend_loc='on data')
```

Example 38 (python):
```python
#this block is to see where the cells over a certain threshold with TNNT2 expression lies in the UMAP
#enter what gene to set threshold for
#when running this box for the first time, comment the starting with 'del'
#then uncomment it for the other ones --- !!!

sc.pl.umap(adata, color=['TNNT2'])
CellName='Cardio_toremove'
gene='TNNT2'
exp_thresh=2
gene_pos=adata.var_names.get_loc(gene)
adata_sel = adata[adata.X[:,gene_pos]>exp_thresh].copy()

l=adata_sel.obs_names
set(l).intersection(adata.obs_names)
print(len(l))

#del adata.obs[CellName]
adata.obs.loc[set(l).intersection(adata.obs_names),CellName]='Yes'
sc.settings.set_figure_params(dpi=200, facecolor='white', color_map = 'RdPu',)
sc.pl.umap(adata, color=[CellName], groups='Yes',size=10, wspace=0.45, legend_fontsize=5, legend_loc='on data',frameon=False)
```

Example 39 (python):
```python
bdata = adata[adata.obs['leiden'].isin(['14','6','10'])].copy()
bdatanames=bdata.obs_names #this is for the obvious clusters to remove directly
cdatanames=adata_sel.obs_names #this is from the removed cells above

p=set(cdatanames)
r=set(bdatanames)
p.update(r)
p=list(p)

print(len(p))
np.savetxt('./Trisomy/RemovedCells/RBCandCardioRemovedFromEndo.txt', p, delimiter="\n", fmt="%s")
```

Example 40 (python):
```python
adata=sc.read('./Objects/T21_multiome_RNA_foetal.h5ad')
print(adata.shape)

bdata=sc.read('./Objects/T21_multiome_RNA_foetal_scVIProcessed.h5ad')
print(bdata.shape)
#Select Cardiomyocyte, Mesenchymal, Endothelium, Epicardium, Immune, Neural
bdata=bdata[bdata.obs.coarse_grain=='Neural',:].copy()

adata=adata[[x for x in adata.obs_names if x in bdata.obs_names],:].copy()
print(adata.shape)
del bdata

# set gene name as var_names 
adata.var['gene_id']=adata.var_names.copy()
adata.var.set_index('gene_name',inplace=True)
adata.var_names=adata.var_names.astype('str')
adata.var_names_make_unique()

cell_cycle_genes = ['MCM5','PCNA','TYMS','FEN1','MCM2','MCM4','RRM1','UNG','GINS2','MCM6','CDCA7','DTL','PRIM1','UHRF1','MLF1IP','HELLS','RFC2','RPA2','NASP','RAD51AP1','GMNN','WDR76','SLBP','CCNE2','UBR7','POLD3','MSH2','ATAD2','RAD51','RRM2','CDC45','CDC6','EXO1','TIPIN','DSCC1','BLM','CASP8AP2','USP1','CLSPN','POLA1','CHAF1B','BRIP1','E2F8','HMGB2','CDK1','NUSAP1','UBE2C','BIRC5','TPX2','TOP2A','NDC80','CKS2','NUF2','CKS1B','MKI67','TMPO','CENPF','TACC3','FAM64A','SMC4','CCNB2','CKAP2L','CKAP2','AURKB','BUB1','KIF11','ANP32E','TUBB4B','GTSE1','KIF20B','HJURP','CDCA3','HN1','CDC20','TTK','CDC25C','KIF2C','RANGAP1','NCAPD2','DLGAP5','CDCA2','CDCA8','ECT2','KIF23','HMMR','AURKA','PSRC1','ANLN','LBR','CKAP5','CENPE','CTCF','NEK2','G2E3','GAS2L3','CBX5','CENPA']
s_genes = cell_cycle_genes[:43]
s_genes = [x for x in s_genes if x in adata.var_names]
print('S genes:', s_genes, '\n')
g2m_genes = cell_cycle_genes[43:]
g2m_genes = [x for x in g2m_genes if x in adata.var_names]
print('G2 genes:', g2m_genes, '\n')
cell_cycle_genes = [x for x in cell_cycle_genes if x in adata.var_names]

hb_genes1 = ['HBB','HBD','HBG1','HBG2','HBE1','HBZ','HBM','HBA2','HBA1', 'HBQ1']
hb_genes1 = [x for x in hb_genes1 if x in adata.var_names]
print('Hemoglobin genes set 1:', hb_genes1, '\n')

hb_genes2 = ['AHSP','GYPA','RHAG','GFI1B','KEL','EPB42','ALAS2','TSPO2','GYPB','GYPE','SPTA1','RHCE','KLF1','RHD','SLC4A1','SNX22','SQSTM1','SOD1','FTH1','MAP1LC3B','SLC25A37','TRIM58','HMBS']
hb_genes2 = [x for x in hb_genes2 if x in adata.var_names]
print('Hemoglobin genes set 2:', hb_genes2, '\n')

mito_genes=['MT-ND1','MT-ND2','MT-CO1','MT-CO2','MT-ATP8','MT-ATP6','MT-CO3','MT-ND3','MT-ND4L', 'MT-ND4', 'MT-ND5', 'MT-ND6', 'MT-CYB']
mito_genes = [x for x in mito_genes if x in adata.var_names]
print('Mitochondrial genes:', mito_genes, '\n')

ribo_genes = ['RPLP0','RPLP1','RPLP2','RPL3','RPL3L','RPL4','RPL5','RPL6','RPL7','RPL7A','RPL7L1','RPL8','RPL9','RPL10','RPL10A','RPL10L','RPL11','RPL12','RPL13','RPL13A','RPL14','RPL15','RPL17','RPL18','RPL18A','RPL19','RPL21','RPL22','RPL22L1','RPL23','RPL23A','RPL24','RPL26','RPL26L1','RPL27','RPL27A','RPL28','RPL29','RPL30','RPL31','RPL32','RPL34','RPL35','RPL35A','RPL36','RPL36A','RPL36AL','RPL37','RPL37A','RPL38','RPL39','RPL39L','UBA52','RPL41','RPSA','RPS2','RPS3','RPS3A','RPS4X','RPS4Y1','RPS4Y2','RPS5','RPS6','RPS7','RPS8','RPS9','RPS10','RPS11','RPS12','RPS13','RPS14','RPS15','RPS15A','RPS16','RPS17','RPS18','RPS19','RPS20','RPS21','RPS23','RPS24','RPS25','RPS26','RPS27','RPS27A','RPS27L','RPS28','RPS29','FAU']
ribo_genes = [x for x in ribo_genes if x in adata.var_names]
print('Ribo genes:', ribo_genes, '\n')

stress_genes=['FOS','FOSB','JUN','JUNB','JUND','ZFP36','SOCS3','CEBPD','CEBPB','ATF3','EGR1','HSPA1A','HSPA1B','HSP90AA1','HSP90AB1','HSPA8','HSPB1','HSPE1','IER3','IER2','BTG1','BTG2','DUSP1']
stress_genes = [x for x in stress_genes if x in adata.var_names]
print('Tissue dissociation stress associated genes:', stress_genes, '\n')

# preserve raw count
adata.layers["counts"] = adata.X.copy()

# normalise and log-transform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key='batch_key')

print(sum(adata.var.highly_variable))
sc.pl.highly_variable_genes(adata)

# freeze the state in `.raw`
adata.raw = adata

# subset hvgs
adata = adata[:, adata.var.highly_variable]

adata=adata.copy()

sc.tl.score_genes_cell_cycle(adata, s_genes=s_genes, g2m_genes=g2m_genes)
#sc.pp.regress_out(adata, ['S_score','G2M_score'])

#batch correction
scvi.model.SCVI.setup_anndata(adata, 
                              layer="counts", 
                              batch_key="batch_key",
                              categorical_covariate_keys=['phase'],
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo'])

vae = scvi.model.SCVI(adata, n_hidden = 128, n_latent = 50, n_layers = 3, dispersion = 'gene-batch')
vae.train(max_epochs=50)
    
# Convergence plot
x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
plt.plot(x, vae.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
plt.clf()

# add the latent space to adata
adata.obsm["X_scVI"] = vae.get_latent_representation()

#sc.tl.score_genes(adata, stress_genes, score_name='stress_score')

sc.tl.score_genes(adata, hb_genes1, score_name='hb1_score')

sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata,min_dist=0.1)
#sc.tl.tsne(adata)

sc.pl.umap(adata, color=['log1p_total_counts','n_genes','n_genes_by_counts','total_counts','pct_counts_mt','pct_counts_ribo'],ncols=5)
sc.pl.umap(adata, color=['scrublet_score', 'hb1_score','phase'])
sc.pl.umap(adata, color=['donor','age','region','phase','cell_or_nuclei','modality','kit_10x'],wspace=0.8)
```

Example 41 (python):
```python
Genes=['TTN','TNNT2','MYOM1','LDB3','ALDH1A2','TNNT1','KRT19','UPK3B','BNC1','COL6A3','FBLN1','PCOLCE','LRRC17','PCDH18','EMCN','ECSCR','PECAM1','CDH5','ESAM', 'LAPTM5','PTPRC','LCP1','CD53','WAS','PHOX2B','NRXN1','STMN2','GRIK2','L1CAM']
sc.pl.umap(adata, color=Genes)

#Ito cells
sc.pl.umap(adata, color=['OIT3','CRHBP','FCN3'])

#Parathyroid cells
sc.pl.umap(adata, color=['PTH','MAFB','GATA3'])

#Spleen
sc.pl.umap(adata, color=['VPREB3','MZB1','CD79B'])

#Hepatocytes
sc.pl.umap(adata, color=['ALB','APOA1','TTR'])

#Melanocytes
sc.pl.umap(adata, color=['MLANA','DCT','DCD','ALX1','CCER2'])

#Adipocytes
sc.pl.umap(adata, color=['GPAM','FASN','ADIPOQ'])

#Tenocytes
sc.pl.umap(adata, color=['TNMD','SCX','TNC','CNMD','THBS4'])

#Chondrocyte markers
sc.pl.umap(adata, color=['MATN4','ACAN','CHAD','COL6A3','COL9A1','COL9A3','HAPLN1','CYTL1','COL2A1','COMP','MKX','SOX9','CD74','EFEMP1','TGM2'])

#AT/12 cells
sc.pl.umap(adata, color=['EPCAM','HOPX','LAMP3','SFTPC'])

#Neural crest cells
sc.pl.umap(adata, color=['FOXD3','SNAI1','SNAI2','SOX8','SOX9','SOX10','TFAP2A'])

#Placenta cytotrophoblasts
sc.pl.umap(adata, color=['PAGE4','GSTA3'])

#Esophagus epithelial
sc.pl.umap(adata, color=['SOX2','TP63','TRH'])
```

Example 42 (python):
```python
sc.pl.umap(adata, color=['MYOM1','TNNT2'])
sc.tl.leiden(adata, resolution=0.5, key_added='leiden') #Cluster 14 hepatocytes, cluster 10 and 6 cardios
sc.pl.umap(adata, color='leiden',legend_loc='on data')
```

Example 43 (python):
```python
#this block is to see where the cells over a certain threshold with TNNT2 expression lies in the UMAP
#enter what gene to set threshold for
#when running this box for the first time, comment the starting with 'del'
#then uncomment it for the other ones --- !!!

sc.pl.umap(adata, color=['TNNT2'])
CellName='Cardio_toremove'
gene='TNNT2'
exp_thresh=2
gene_pos=adata.var_names.get_loc(gene)
adata_sel = adata[adata.X[:,gene_pos]>exp_thresh].copy()

l=adata_sel.obs_names
set(l).intersection(adata.obs_names)
print(len(l))

#del adata.obs[CellName]
adata.obs.loc[set(l).intersection(adata.obs_names),CellName]='Yes'
sc.settings.set_figure_params(dpi=200, facecolor='white', color_map = 'RdPu',)
sc.pl.umap(adata, color=[CellName], groups='Yes',size=10, wspace=0.45, legend_fontsize=5, legend_loc='on data',frameon=False)
```

Example 44 (python):
```python
bdata = adata[adata.obs['leiden'].isin(['6','11'])].copy()
bdatanames=bdata.obs_names #this is for the obvious clusters to remove directly
cdatanames=adata_sel.obs_names #this is from the removed cells above

p=set(cdatanames)
r=set(bdatanames)
p.update(r)
p=list(p)

print(len(p))
np.savetxt('./Trisomy/RemovedCells/RBCandCardioRemovedFromNeural.txt', p, delimiter="\n", fmt="%s")
```

Example 45 (python):
```python
adata=sc.read('./Objects/T21_multiome_RNA_foetal.h5ad')
print(adata.shape)

bdata=sc.read('./Objects/T21_multiome_RNA_foetal_scVIProcessed.h5ad')
print(bdata.shape)
#Select Cardiomyocyte, Mesenchymal, Endothelium, Epicardium, Immune, Neural
bdata=bdata[bdata.obs.coarse_grain=='Mesenchymal',:].copy()

adata=adata[[x for x in adata.obs_names if x in bdata.obs_names],:].copy()
print(adata.shape)
del bdata

# set gene name as var_names 
adata.var['gene_id']=adata.var_names.copy()
adata.var.set_index('gene_name',inplace=True)
adata.var_names=adata.var_names.astype('str')
adata.var_names_make_unique()

cell_cycle_genes = ['MCM5','PCNA','TYMS','FEN1','MCM2','MCM4','RRM1','UNG','GINS2','MCM6','CDCA7','DTL','PRIM1','UHRF1','MLF1IP','HELLS','RFC2','RPA2','NASP','RAD51AP1','GMNN','WDR76','SLBP','CCNE2','UBR7','POLD3','MSH2','ATAD2','RAD51','RRM2','CDC45','CDC6','EXO1','TIPIN','DSCC1','BLM','CASP8AP2','USP1','CLSPN','POLA1','CHAF1B','BRIP1','E2F8','HMGB2','CDK1','NUSAP1','UBE2C','BIRC5','TPX2','TOP2A','NDC80','CKS2','NUF2','CKS1B','MKI67','TMPO','CENPF','TACC3','FAM64A','SMC4','CCNB2','CKAP2L','CKAP2','AURKB','BUB1','KIF11','ANP32E','TUBB4B','GTSE1','KIF20B','HJURP','CDCA3','HN1','CDC20','TTK','CDC25C','KIF2C','RANGAP1','NCAPD2','DLGAP5','CDCA2','CDCA8','ECT2','KIF23','HMMR','AURKA','PSRC1','ANLN','LBR','CKAP5','CENPE','CTCF','NEK2','G2E3','GAS2L3','CBX5','CENPA']
s_genes = cell_cycle_genes[:43]
s_genes = [x for x in s_genes if x in adata.var_names]
print('S genes:', s_genes, '\n')
g2m_genes = cell_cycle_genes[43:]
g2m_genes = [x for x in g2m_genes if x in adata.var_names]
print('G2 genes:', g2m_genes, '\n')
cell_cycle_genes = [x for x in cell_cycle_genes if x in adata.var_names]

hb_genes1 = ['HBB','HBD','HBG1','HBG2','HBE1','HBZ','HBM','HBA2','HBA1', 'HBQ1']
hb_genes1 = [x for x in hb_genes1 if x in adata.var_names]
print('Hemoglobin genes set 1:', hb_genes1, '\n')

hb_genes2 = ['AHSP','GYPA','RHAG','GFI1B','KEL','EPB42','ALAS2','TSPO2','GYPB','GYPE','SPTA1','RHCE','KLF1','RHD','SLC4A1','SNX22','SQSTM1','SOD1','FTH1','MAP1LC3B','SLC25A37','TRIM58','HMBS']
hb_genes2 = [x for x in hb_genes2 if x in adata.var_names]
print('Hemoglobin genes set 2:', hb_genes2, '\n')

mito_genes=['MT-ND1','MT-ND2','MT-CO1','MT-CO2','MT-ATP8','MT-ATP6','MT-CO3','MT-ND3','MT-ND4L', 'MT-ND4', 'MT-ND5', 'MT-ND6', 'MT-CYB']
mito_genes = [x for x in mito_genes if x in adata.var_names]
print('Mitochondrial genes:', mito_genes, '\n')

ribo_genes = ['RPLP0','RPLP1','RPLP2','RPL3','RPL3L','RPL4','RPL5','RPL6','RPL7','RPL7A','RPL7L1','RPL8','RPL9','RPL10','RPL10A','RPL10L','RPL11','RPL12','RPL13','RPL13A','RPL14','RPL15','RPL17','RPL18','RPL18A','RPL19','RPL21','RPL22','RPL22L1','RPL23','RPL23A','RPL24','RPL26','RPL26L1','RPL27','RPL27A','RPL28','RPL29','RPL30','RPL31','RPL32','RPL34','RPL35','RPL35A','RPL36','RPL36A','RPL36AL','RPL37','RPL37A','RPL38','RPL39','RPL39L','UBA52','RPL41','RPSA','RPS2','RPS3','RPS3A','RPS4X','RPS4Y1','RPS4Y2','RPS5','RPS6','RPS7','RPS8','RPS9','RPS10','RPS11','RPS12','RPS13','RPS14','RPS15','RPS15A','RPS16','RPS17','RPS18','RPS19','RPS20','RPS21','RPS23','RPS24','RPS25','RPS26','RPS27','RPS27A','RPS27L','RPS28','RPS29','FAU']
ribo_genes = [x for x in ribo_genes if x in adata.var_names]
print('Ribo genes:', ribo_genes, '\n')

stress_genes=['FOS','FOSB','JUN','JUNB','JUND','ZFP36','SOCS3','CEBPD','CEBPB','ATF3','EGR1','HSPA1A','HSPA1B','HSP90AA1','HSP90AB1','HSPA8','HSPB1','HSPE1','IER3','IER2','BTG1','BTG2','DUSP1']
stress_genes = [x for x in stress_genes if x in adata.var_names]
print('Tissue dissociation stress associated genes:', stress_genes, '\n')

# preserve raw count
adata.layers["counts"] = adata.X.copy()

# normalise and log-transform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key='batch_key')

print(sum(adata.var.highly_variable))
sc.pl.highly_variable_genes(adata)

# freeze the state in `.raw`
adata.raw = adata

# subset hvgs
adata = adata[:, adata.var.highly_variable]

adata=adata.copy()

sc.tl.score_genes_cell_cycle(adata, s_genes=s_genes, g2m_genes=g2m_genes)
#sc.pp.regress_out(adata, ['S_score','G2M_score'])

#batch correction
scvi.model.SCVI.setup_anndata(adata, 
                              layer="counts", 
                              batch_key="batch_key",
                              categorical_covariate_keys=['phase'],
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo'])

vae = scvi.model.SCVI(adata, n_hidden = 128, n_latent = 50, n_layers = 3, dispersion = 'gene-batch')
vae.train(max_epochs=50)
    
# Convergence plot
x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
plt.plot(x, vae.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
plt.clf()

# add the latent space to adata
adata.obsm["X_scVI"] = vae.get_latent_representation()

#sc.tl.score_genes(adata, stress_genes, score_name='stress_score')

sc.tl.score_genes(adata, hb_genes1, score_name='hb1_score')

sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata,min_dist=0.1)
#sc.tl.tsne(adata)

sc.pl.umap(adata, color=['log1p_total_counts','n_genes','n_genes_by_counts','total_counts','pct_counts_mt','pct_counts_ribo'],ncols=5)
sc.pl.umap(adata, color=['scrublet_score', 'hb1_score','phase'])
sc.pl.umap(adata, color=['donor','age','region','phase','cell_or_nuclei','modality','kit_10x'],wspace=0.8)
```

Example 46 (python):
```python
Genes=['TTN','TNNT2','MYOM1','LDB3','ALDH1A2','TNNT1','KRT19','UPK3B','BNC1','COL6A3','FBLN1','PCOLCE','LRRC17','PCDH18','EMCN','ECSCR','PECAM1','CDH5','ESAM', 'LAPTM5','PTPRC','LCP1','CD53','WAS','PHOX2B','NRXN1','STMN2','GRIK2','L1CAM']
sc.pl.umap(adata, color=Genes)

#Ito cells
sc.pl.umap(adata, color=['OIT3','CRHBP','FCN3'])

#Parathyroid cells
sc.pl.umap(adata, color=['PTH','MAFB','GATA3'])

#Spleen
sc.pl.umap(adata, color=['VPREB3','MZB1','CD79B'])

#Hepatocytes
sc.pl.umap(adata, color=['ALB','APOA1','TTR'])

#Melanocytes
sc.pl.umap(adata, color=['MLANA','DCT','DCD','ALX1','CCER2'])

#Adipocytes
sc.pl.umap(adata, color=['GPAM','FASN','ADIPOQ'])

#Tenocytes
sc.pl.umap(adata, color=['TNMD','SCX','TNC','CNMD','THBS4'])

#Chondrocyte markers
sc.pl.umap(adata, color=['MATN4','ACAN','CHAD','COL6A3','COL9A1','COL9A3','HAPLN1','CYTL1','COL2A1','COMP','MKX','SOX9','CD74','EFEMP1','TGM2'])

#AT/12 cells
sc.pl.umap(adata, color=['EPCAM','HOPX','LAMP3','SFTPC'])

#Neural crest cells
sc.pl.umap(adata, color=['FOXD3','SNAI1','SNAI2','SOX8','SOX9','SOX10','TFAP2A'])

#Placenta cytotrophoblasts
sc.pl.umap(adata, color=['PAGE4','GSTA3'])

#Esophagus epithelial
sc.pl.umap(adata, color=['SOX2','TP63','TRH'])
```

Example 47 (python):
```python
sc.pl.umap(adata, color=['PDGFRA','MYH11','DCN','POSTN','TCF21','KCNJ8','PLA2G5','MT2A','HIGD1B','CLU','PDGFRB'])
```

Example 48 (python):
```python
sc.pl.umap(adata, color=hb_genes1)
sc.pl.umap(adata, color=hb_genes2)

for k in list(set(adata.obs.region)):
    sc.pl.umap(adata, color='region',groups=k)
    
for k in list(set(adata.obs.sangerID)):
    sc.pl.umap(adata, color='sangerID',groups=k)
```

Example 49 (python):
```python
sc.tl.leiden(adata, resolution=0.3, key_added='leiden')
sc.pl.umap(adata, color='leiden',legend_loc='on data')
```

Example 50 (python):
```python
adata = adata[adata.obs['leiden'].isin(['0','1','2','3','4','5','6','7','8','10','11','13'])].copy()
```

Example 51 (python):
```python
#this block is to see where the cells over a certain threshold with TNNT2 expression lies in the UMAP
#enter what gene to set threshold for
#when running this box for the first time, comment the starting with 'del'
#then uncomment it for the other ones --- !!!

sc.pl.umap(adata, color=['TNNT2'])
CellName='Cardio_toremove'
gene='TNNT2'
exp_thresh=2
gene_pos=adata.var_names.get_loc(gene)
adata_sel = adata[adata.X[:,gene_pos]>exp_thresh].copy()

l=adata_sel.obs_names
set(l).intersection(adata.obs_names)
print(len(l))

#del adata.obs[CellName]
adata.obs.loc[set(l).intersection(adata.obs_names),CellName]='Yes'
sc.settings.set_figure_params(dpi=200, facecolor='white', color_map = 'RdPu',)
sc.pl.umap(adata, color=[CellName], groups='Yes',size=10, wspace=0.45, legend_fontsize=5, legend_loc='on data',frameon=False)
```

Example 52 (python):
```python
bdatanames=adata_sel.obs_names #this is from the removed cells above
len(bdatanames)
np.savetxt('./Trisomy/RemovedCells/RBCandCardioRemovedMesenchymal.txt', bdatanames, delimiter="\n", fmt="%s")
```

Example 53 (python):
```python
adata=sc.read('./Objects/T21_multiome_RNA_foetal.h5ad')
print(adata.shape)

bdata=sc.read('./Objects/T21_multiome_RNA_foetal_scVIProcessed.h5ad')
print(bdata.shape)
#Select Cardiomyocyte, Mesenchymal, Endothelium, Epicardium, Immune, Neural
bdata=bdata[bdata.obs.coarse_grain=='Cardiomyocytes',:].copy()

adata=adata[[x for x in adata.obs_names if x in bdata.obs_names],:].copy()
print(adata.shape)
del bdata

# set gene name as var_names 
adata.var['gene_id']=adata.var_names.copy()
adata.var.set_index('gene_name',inplace=True)
adata.var_names=adata.var_names.astype('str')
adata.var_names_make_unique()

cell_cycle_genes = ['MCM5','PCNA','TYMS','FEN1','MCM2','MCM4','RRM1','UNG','GINS2','MCM6','CDCA7','DTL','PRIM1','UHRF1','MLF1IP','HELLS','RFC2','RPA2','NASP','RAD51AP1','GMNN','WDR76','SLBP','CCNE2','UBR7','POLD3','MSH2','ATAD2','RAD51','RRM2','CDC45','CDC6','EXO1','TIPIN','DSCC1','BLM','CASP8AP2','USP1','CLSPN','POLA1','CHAF1B','BRIP1','E2F8','HMGB2','CDK1','NUSAP1','UBE2C','BIRC5','TPX2','TOP2A','NDC80','CKS2','NUF2','CKS1B','MKI67','TMPO','CENPF','TACC3','FAM64A','SMC4','CCNB2','CKAP2L','CKAP2','AURKB','BUB1','KIF11','ANP32E','TUBB4B','GTSE1','KIF20B','HJURP','CDCA3','HN1','CDC20','TTK','CDC25C','KIF2C','RANGAP1','NCAPD2','DLGAP5','CDCA2','CDCA8','ECT2','KIF23','HMMR','AURKA','PSRC1','ANLN','LBR','CKAP5','CENPE','CTCF','NEK2','G2E3','GAS2L3','CBX5','CENPA']
s_genes = cell_cycle_genes[:43]
s_genes = [x for x in s_genes if x in adata.var_names]
print('S genes:', s_genes, '\n')
g2m_genes = cell_cycle_genes[43:]
g2m_genes = [x for x in g2m_genes if x in adata.var_names]
print('G2 genes:', g2m_genes, '\n')
cell_cycle_genes = [x for x in cell_cycle_genes if x in adata.var_names]

hb_genes1 = ['HBB','HBD','HBG1','HBG2','HBE1','HBZ','HBM','HBA2','HBA1', 'HBQ1']
hb_genes1 = [x for x in hb_genes1 if x in adata.var_names]
print('Hemoglobin genes set 1:', hb_genes1, '\n')

hb_genes2 = ['AHSP','GYPA','RHAG','GFI1B','KEL','EPB42','ALAS2','TSPO2','GYPB','GYPE','SPTA1','RHCE','KLF1','RHD','SLC4A1','SNX22','SQSTM1','SOD1','FTH1','MAP1LC3B','SLC25A37','TRIM58','HMBS']
hb_genes2 = [x for x in hb_genes2 if x in adata.var_names]
print('Hemoglobin genes set 2:', hb_genes2, '\n')

mito_genes=['MT-ND1','MT-ND2','MT-CO1','MT-CO2','MT-ATP8','MT-ATP6','MT-CO3','MT-ND3','MT-ND4L', 'MT-ND4', 'MT-ND5', 'MT-ND6', 'MT-CYB']
mito_genes = [x for x in mito_genes if x in adata.var_names]
print('Mitochondrial genes:', mito_genes, '\n')

ribo_genes = ['RPLP0','RPLP1','RPLP2','RPL3','RPL3L','RPL4','RPL5','RPL6','RPL7','RPL7A','RPL7L1','RPL8','RPL9','RPL10','RPL10A','RPL10L','RPL11','RPL12','RPL13','RPL13A','RPL14','RPL15','RPL17','RPL18','RPL18A','RPL19','RPL21','RPL22','RPL22L1','RPL23','RPL23A','RPL24','RPL26','RPL26L1','RPL27','RPL27A','RPL28','RPL29','RPL30','RPL31','RPL32','RPL34','RPL35','RPL35A','RPL36','RPL36A','RPL36AL','RPL37','RPL37A','RPL38','RPL39','RPL39L','UBA52','RPL41','RPSA','RPS2','RPS3','RPS3A','RPS4X','RPS4Y1','RPS4Y2','RPS5','RPS6','RPS7','RPS8','RPS9','RPS10','RPS11','RPS12','RPS13','RPS14','RPS15','RPS15A','RPS16','RPS17','RPS18','RPS19','RPS20','RPS21','RPS23','RPS24','RPS25','RPS26','RPS27','RPS27A','RPS27L','RPS28','RPS29','FAU']
ribo_genes = [x for x in ribo_genes if x in adata.var_names]
print('Ribo genes:', ribo_genes, '\n')

stress_genes=['FOS','FOSB','JUN','JUNB','JUND','ZFP36','SOCS3','CEBPD','CEBPB','ATF3','EGR1','HSPA1A','HSPA1B','HSP90AA1','HSP90AB1','HSPA8','HSPB1','HSPE1','IER3','IER2','BTG1','BTG2','DUSP1']
stress_genes = [x for x in stress_genes if x in adata.var_names]
print('Tissue dissociation stress associated genes:', stress_genes, '\n')

# preserve raw count
adata.layers["counts"] = adata.X.copy()

# normalise and log-transform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key='batch_key')

print(sum(adata.var.highly_variable))
sc.pl.highly_variable_genes(adata)

# freeze the state in `.raw`
adata.raw = adata

# subset hvgs
adata = adata[:, adata.var.highly_variable]

adata=adata.copy()

sc.tl.score_genes_cell_cycle(adata, s_genes=s_genes, g2m_genes=g2m_genes)
#sc.pp.regress_out(adata, ['S_score','G2M_score'])

#batch correction
scvi.model.SCVI.setup_anndata(adata, 
                              layer="counts", 
                              batch_key="batch_key",
                              categorical_covariate_keys=['phase'],
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo'])

vae = scvi.model.SCVI(adata, n_hidden = 128, n_latent = 50, n_layers = 3, dispersion = 'gene-batch')
vae.train(max_epochs=50)
    
# Convergence plot
x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
plt.plot(x, vae.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
plt.clf()

# add the latent space to adata
adata.obsm["X_scVI"] = vae.get_latent_representation()

#sc.tl.score_genes(adata, stress_genes, score_name='stress_score')

sc.tl.score_genes(adata, hb_genes1, score_name='hb1_score')

sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata,min_dist=0.1)
#sc.tl.tsne(adata)

sc.pl.umap(adata, color=['log1p_total_counts','n_genes','n_genes_by_counts','total_counts','pct_counts_mt','pct_counts_ribo'],ncols=5)
sc.pl.umap(adata, color=['scrublet_score', 'hb1_score','phase'])
sc.pl.umap(adata, color=['donor','age','region','phase','cell_or_nuclei','modality','kit_10x'],wspace=0.8)
```

Example 54 (python):
```python
for k in list(set(adata.obs.region)):
    sc.pl.umap(adata, color='region',groups=k)
    
for k in list(set(adata.obs.sangerID)):
    sc.pl.umap(adata, color='sangerID',groups=k)
```

Example 55 (python):
```python
Genes=['TTN','TNNT2','MYOM1','LDB3','ALDH1A2','TNNT1','KRT19','UPK3B','BNC1','COL6A3','FBLN1','PCOLCE','LRRC17','PCDH18','EMCN','ECSCR','PECAM1','CDH5','ESAM', 'LAPTM5','PTPRC','LCP1','CD53','WAS','PHOX2B','NRXN1','STMN2','GRIK2','L1CAM']
sc.pl.umap(adata, color=Genes)

#Ito cells
sc.pl.umap(adata, color=['OIT3','CRHBP','FCN3'])

#Parathyroid cells
sc.pl.umap(adata, color=['PTH','MAFB','GATA3'])

#Spleen
sc.pl.umap(adata, color=['VPREB3','MZB1','CD79B'])

#Hepatocytes
sc.pl.umap(adata, color=['ALB','APOA1','TTR'])

#Melanocytes
sc.pl.umap(adata, color=['MLANA','DCT','DCD','ALX1','CCER2'])

#Adipocytes
sc.pl.umap(adata, color=['GPAM','FASN','ADIPOQ'])

#Tenocytes
sc.pl.umap(adata, color=['TNMD','SCX','TNC','CNMD','THBS4'])

#Chondrocyte markers
sc.pl.umap(adata, color=['MATN4','ACAN','CHAD','COL6A3','COL9A1','COL9A3','HAPLN1','CYTL1','COL2A1','COMP','MKX','SOX9','CD74','EFEMP1','TGM2'])

#AT/12 cells
sc.pl.umap(adata, color=['EPCAM','HOPX','LAMP3','SFTPC'])

#Neural crest cells
sc.pl.umap(adata, color=['FOXD3','SNAI1','SNAI2','SOX8','SOX9','SOX10','TFAP2A'])

#Placenta cytotrophoblasts
sc.pl.umap(adata, color=['PAGE4','GSTA3'])

#Esophagus epithelial
sc.pl.umap(adata, color=['SOX2','TP63','TRH'])
```

Example 56 (python):
```python
sc.pl.umap(adata, color=['SHOX2', 'TBX3','BMP10','PITX2'])
```

Example 57 (python):
```python
sc.tl.leiden(adata, resolution=0.22, key_added='leiden')
sc.pl.umap(adata, color='leiden',legend_loc='on data')
```

Example 58 (python):
```python
#Getting the UMAPs for clusterofinterest
ClusterOfInterest='3'
sc.tl.rank_genes_groups(adata, 'leiden', method='t-test', key_added = "t-test")
DEGenesForCluster = [x[ClusterOfInterest] for x in adata.uns['t-test']['names'][:50]]
sc.pl.dotplot(adata, DEGenesForCluster ,groupby='leiden',standard_scale='var',cmap='PRGn')
#sc.pl.umap(adata, color=DEGenesForCluster)
```

Example 59 (python):
```python
sc.tl.rank_genes_groups(adata, 'leiden', groups=['3'], reference='1', method='t-test')
sc.pl.rank_genes_groups(adata, groups=['3'], n_genes=20)
```

Example 60 (python):
```python
adata=sc.read('./Objects/T21_multiome_RNA_foetal.h5ad')
print(adata.shape)

RBCandCardioRemovedFromEndo=[x.strip() for x in open('./Trisomy/RemovedCells/RBCandCardioRemovedFromEndo.txt')]
RBCandCardioRemovedFromEpi=[x.strip() for x in open('./Trisomy/RemovedCells/RBCandCardioRemovedFromEpi.txt')]
RBCandCardioRemovedFromImmune=[x.strip() for x in open('./Trisomy/RemovedCells/RBCandCardioRemovedFromImmune.txt')]
RBCandCardioRemovedFromNeural=[x.strip() for x in open('./Trisomy/RemovedCells/RBCandCardioRemovedFromNeural.txt')]
RBCandCardioRemovedMesenchymal=[x.strip() for x in open('./Trisomy/RemovedCells/RBCandCardioRemovedMesenchymal.txt')]
ShouldNotBeIn=RBCandCardioRemovedFromEndo+RBCandCardioRemovedFromEpi+RBCandCardioRemovedFromImmune+RBCandCardioRemovedFromNeural+RBCandCardioRemovedMesenchymal
print(len(ShouldNotBeIn))

print(adata)
adata=adata[[x for x in adata.obs_names if x not in ShouldNotBeIn],:].copy()
print(adata)

# set gene name as var_names 
adata.var['gene_id']=adata.var_names.copy()
adata.var.set_index('gene_name',inplace=True)
adata.var_names=adata.var_names.astype('str')
adata.var_names_make_unique()

cell_cycle_genes = ['MCM5','PCNA','TYMS','FEN1','MCM2','MCM4','RRM1','UNG','GINS2','MCM6','CDCA7','DTL','PRIM1','UHRF1','MLF1IP','HELLS','RFC2','RPA2','NASP','RAD51AP1','GMNN','WDR76','SLBP','CCNE2','UBR7','POLD3','MSH2','ATAD2','RAD51','RRM2','CDC45','CDC6','EXO1','TIPIN','DSCC1','BLM','CASP8AP2','USP1','CLSPN','POLA1','CHAF1B','BRIP1','E2F8','HMGB2','CDK1','NUSAP1','UBE2C','BIRC5','TPX2','TOP2A','NDC80','CKS2','NUF2','CKS1B','MKI67','TMPO','CENPF','TACC3','FAM64A','SMC4','CCNB2','CKAP2L','CKAP2','AURKB','BUB1','KIF11','ANP32E','TUBB4B','GTSE1','KIF20B','HJURP','CDCA3','HN1','CDC20','TTK','CDC25C','KIF2C','RANGAP1','NCAPD2','DLGAP5','CDCA2','CDCA8','ECT2','KIF23','HMMR','AURKA','PSRC1','ANLN','LBR','CKAP5','CENPE','CTCF','NEK2','G2E3','GAS2L3','CBX5','CENPA']
s_genes = cell_cycle_genes[:43]
s_genes = [x for x in s_genes if x in adata.var_names]
print('S genes:', s_genes, '\n')
g2m_genes = cell_cycle_genes[43:]
g2m_genes = [x for x in g2m_genes if x in adata.var_names]
print('G2 genes:', g2m_genes, '\n')
cell_cycle_genes = [x for x in cell_cycle_genes if x in adata.var_names]

hb_genes1 = ['HBB','HBD','HBG1','HBG2','HBE1','HBZ','HBM','HBA2','HBA1', 'HBQ1']
hb_genes1 = [x for x in hb_genes1 if x in adata.var_names]
print('Hemoglobin genes set 1:', hb_genes1, '\n')

hb_genes2 = ['AHSP','GYPA','RHAG','GFI1B','KEL','EPB42','ALAS2','TSPO2','GYPB','GYPE','SPTA1','RHCE','KLF1','RHD','SLC4A1','SNX22','SQSTM1','SOD1','FTH1','MAP1LC3B','SLC25A37','TRIM58','HMBS']
hb_genes2 = [x for x in hb_genes2 if x in adata.var_names]
print('Hemoglobin genes set 2:', hb_genes2, '\n')

mito_genes=['MT-ND1','MT-ND2','MT-CO1','MT-CO2','MT-ATP8','MT-ATP6','MT-CO3','MT-ND3','MT-ND4L', 'MT-ND4', 'MT-ND5', 'MT-ND6', 'MT-CYB']
mito_genes = [x for x in mito_genes if x in adata.var_names]
print('Mitochondrial genes:', mito_genes, '\n')

ribo_genes = ['RPLP0','RPLP1','RPLP2','RPL3','RPL3L','RPL4','RPL5','RPL6','RPL7','RPL7A','RPL7L1','RPL8','RPL9','RPL10','RPL10A','RPL10L','RPL11','RPL12','RPL13','RPL13A','RPL14','RPL15','RPL17','RPL18','RPL18A','RPL19','RPL21','RPL22','RPL22L1','RPL23','RPL23A','RPL24','RPL26','RPL26L1','RPL27','RPL27A','RPL28','RPL29','RPL30','RPL31','RPL32','RPL34','RPL35','RPL35A','RPL36','RPL36A','RPL36AL','RPL37','RPL37A','RPL38','RPL39','RPL39L','UBA52','RPL41','RPSA','RPS2','RPS3','RPS3A','RPS4X','RPS4Y1','RPS4Y2','RPS5','RPS6','RPS7','RPS8','RPS9','RPS10','RPS11','RPS12','RPS13','RPS14','RPS15','RPS15A','RPS16','RPS17','RPS18','RPS19','RPS20','RPS21','RPS23','RPS24','RPS25','RPS26','RPS27','RPS27A','RPS27L','RPS28','RPS29','FAU']
ribo_genes = [x for x in ribo_genes if x in adata.var_names]
print('Ribo genes:', ribo_genes, '\n')

stress_genes=['FOS','FOSB','JUN','JUNB','JUND','ZFP36','SOCS3','CEBPD','CEBPB','ATF3','EGR1','HSPA1A','HSPA1B','HSP90AA1','HSP90AB1','HSPA8','HSPB1','HSPE1','IER3','IER2','BTG1','BTG2','DUSP1']
stress_genes = [x for x in stress_genes if x in adata.var_names]
print('Tissue dissociation stress associated genes:', stress_genes, '\n')

# preserve raw count
adata.layers["counts"] = adata.X.copy()

# normalise and log-transform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key='batch_key')

print(sum(adata.var.highly_variable))
sc.pl.highly_variable_genes(adata)

# freeze the state in `.raw`
adata.raw = adata

# subset hvgs
adata = adata[:, adata.var.highly_variable]

adata=adata.copy()

sc.tl.score_genes_cell_cycle(adata, s_genes=s_genes, g2m_genes=g2m_genes)
#sc.pp.regress_out(adata, ['S_score','G2M_score'])

#batch correction
scvi.model.SCVI.setup_anndata(adata, 
                              layer="counts", 
                              batch_key="batch_key",
                              categorical_covariate_keys=['phase'],
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo'])

vae = scvi.model.SCVI(adata, n_hidden = 128, n_latent = 50, n_layers = 3, dispersion = 'gene-batch')
vae.train(max_epochs=50)
    
# Convergence plot
x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
plt.plot(x, vae.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
plt.clf()

# add the latent space to adata
adata.obsm["X_scVI"] = vae.get_latent_representation()

#sc.tl.score_genes(adata, stress_genes, score_name='stress_score')

sc.tl.score_genes(adata, hb_genes1, score_name='hb1_score')

sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata,min_dist=0.1)
#sc.tl.tsne(adata)

sc.pl.umap(adata, color=['log1p_total_counts','n_genes','n_genes_by_counts','total_counts','pct_counts_mt','pct_counts_ribo'],ncols=5)
sc.pl.umap(adata, color=['scrublet_score', 'hb1_score','phase'])
sc.pl.umap(adata, color=['donor','age','region','phase','cell_or_nuclei','modality','kit_10x'],wspace=0.8)
```

Example 61 (python):
```python
sc.pl.umap(adata, color=hb_genes1)
sc.pl.umap(adata, color=hb_genes2)

for k in list(set(adata.obs.region)):
    sc.pl.umap(adata, color='region',groups=k)
    
for k in list(set(adata.obs.sangerID)):
    sc.pl.umap(adata, color='sangerID',groups=k)
```

Example 62 (python):
```python
Genes=['TTN','TNNT2','MYOM1','LDB3','ALDH1A2','TNNT1','KRT19','UPK3B','BNC1','COL6A3','FBLN1','PCOLCE','LRRC17','PCDH18','EMCN','ECSCR','PECAM1','CDH5','ESAM', 'LAPTM5','PTPRC','LCP1','CD53','WAS','PHOX2B','NRXN1','STMN2','GRIK2','L1CAM']
sc.pl.umap(adata, color=Genes)

#Ito cells
sc.pl.umap(adata, color=['OIT3','CRHBP','FCN3'])

#Parathyroid cells
sc.pl.umap(adata, color=['PTH','MAFB','GATA3'])

#Spleen
sc.pl.umap(adata, color=['VPREB3','MZB1','CD79B'])

#Hepatocytes
sc.pl.umap(adata, color=['ALB','APOA1','TTR'])

#Melanocytes
sc.pl.umap(adata, color=['MLANA','DCT','DCD','ALX1','CCER2'])

#Adipocytes
sc.pl.umap(adata, color=['GPAM','FASN','ADIPOQ'])

#Tenocytes
sc.pl.umap(adata, color=['TNMD','SCX','TNC','CNMD','THBS4'])

#Chondrocyte markers
sc.pl.umap(adata, color=['MATN4','ACAN','CHAD','COL6A3','COL9A1','COL9A3','HAPLN1','CYTL1','COL2A1','COMP','MKX','SOX9','CD74','EFEMP1','TGM2'])

#AT/12 cells
sc.pl.umap(adata, color=['EPCAM','HOPX','LAMP3','SFTPC'])

#Neural crest cells
sc.pl.umap(adata, color=['FOXD3','SNAI1','SNAI2','SOX8','SOX9','SOX10','TFAP2A'])

#Placenta cytotrophoblasts
sc.pl.umap(adata, color=['PAGE4','GSTA3'])

#Esophagus epithelial
sc.pl.umap(adata, color=['SOX2','TP63','TRH'])
```

Example 63 (python):
```python
sc.tl.leiden(adata, resolution=1, key_added='leiden')
sc.pl.umap(adata, color='leiden',legend_loc='on data')
```

Example 64 (python):
```python
sc.pl.umap(adata, color='leiden',groups='33',legend_loc='on data')
```

Example 65 (python):
```python
ClusterOfInterest='40'
#sc.tl.rank_genes_groups(adata, 'leiden', method='t-test', key_added = "t-test")
DEGenesForCluster = [x[ClusterOfInterest] for x in adata.uns['t-test']['names'][:50]]
sc.pl.dotplot(adata, DEGenesForCluster ,groupby='leiden',standard_scale='var',cmap='PRGn')
#sc.pl.umap(adata, color=DEGenesForCluster)
```

Example 66 (python):
```python
ClusterOfInterest='16' #This is skeletal muscle cluster (17 is obtained because it is from a different run)
sc.tl.rank_genes_groups(adata, 'leiden', method='t-test', key_added = "t-test")
DEGenesForCluster = [x[ClusterOfInterest] for x in adata.uns['t-test']['names'][:50]]
sc.pl.dotplot(adata, DEGenesForCluster ,groupby='leiden',standard_scale='var',cmap='PRGn')
#sc.pl.umap(adata, color=DEGenesForCluster)
```

Example 67 (python):
```python
bdata = adata[adata.obs['leiden'].isin(['33','40'])].copy() #33 other cardio in the mesenchymal cluster; 40 is unknown (tried all the known markers)
bdatanames=bdata.obs_names
print(len(bdatanames))
np.savetxt('./Trisomy/RemovedCells/RBCandCardioRemovedGeneral.txt', bdatanames, delimiter="\n", fmt="%s")
```

Example 68 (python):
```python
adata=sc.read('./Objects/T21_multiome_RNA_foetal.h5ad')
print(adata.shape)

RBCandCardioRemovedFromEndo=[x.strip() for x in open('./Trisomy/RemovedCells/RBCandCardioRemovedFromEndo.txt')]
RBCandCardioRemovedFromEpi=[x.strip() for x in open('./Trisomy/RemovedCells/RBCandCardioRemovedFromEpi.txt')]
RBCandCardioRemovedFromImmune=[x.strip() for x in open('./Trisomy/RemovedCells/RBCandCardioRemovedFromImmune.txt')]
RBCandCardioRemovedFromNeural=[x.strip() for x in open('./Trisomy/RemovedCells/RBCandCardioRemovedFromNeural.txt')]
RBCandCardioRemovedMesenchymal=[x.strip() for x in open('./Trisomy/RemovedCells/RBCandCardioRemovedMesenchymal.txt')]
RBCandCardioRemovedGeneral=[x.strip() for x in open('./Trisomy/RemovedCells/RBCandCardioRemovedGeneral.txt')]
SkeletalMuscle=[x.strip() for x in open('./Trisomy/RemovedCells/SkeletalMuscle.txt')]
ShouldNotBeIn=RBCandCardioRemovedFromEndo+RBCandCardioRemovedFromEpi+RBCandCardioRemovedFromImmune+RBCandCardioRemovedFromNeural+RBCandCardioRemovedMesenchymal+RBCandCardioRemovedGeneral+SkeletalMuscle
print(len(ShouldNotBeIn))

print(adata)
adata=adata[[x for x in adata.obs_names if x not in ShouldNotBeIn],:].copy()
print(adata)

# set gene name as var_names 
adata.var['gene_id']=adata.var_names.copy()
adata.var.set_index('gene_name',inplace=True)
adata.var_names=adata.var_names.astype('str')
adata.var_names_make_unique()

cell_cycle_genes = ['MCM5','PCNA','TYMS','FEN1','MCM2','MCM4','RRM1','UNG','GINS2','MCM6','CDCA7','DTL','PRIM1','UHRF1','MLF1IP','HELLS','RFC2','RPA2','NASP','RAD51AP1','GMNN','WDR76','SLBP','CCNE2','UBR7','POLD3','MSH2','ATAD2','RAD51','RRM2','CDC45','CDC6','EXO1','TIPIN','DSCC1','BLM','CASP8AP2','USP1','CLSPN','POLA1','CHAF1B','BRIP1','E2F8','HMGB2','CDK1','NUSAP1','UBE2C','BIRC5','TPX2','TOP2A','NDC80','CKS2','NUF2','CKS1B','MKI67','TMPO','CENPF','TACC3','FAM64A','SMC4','CCNB2','CKAP2L','CKAP2','AURKB','BUB1','KIF11','ANP32E','TUBB4B','GTSE1','KIF20B','HJURP','CDCA3','HN1','CDC20','TTK','CDC25C','KIF2C','RANGAP1','NCAPD2','DLGAP5','CDCA2','CDCA8','ECT2','KIF23','HMMR','AURKA','PSRC1','ANLN','LBR','CKAP5','CENPE','CTCF','NEK2','G2E3','GAS2L3','CBX5','CENPA']
s_genes = cell_cycle_genes[:43]
s_genes = [x for x in s_genes if x in adata.var_names]
print('S genes:', s_genes, '\n')
g2m_genes = cell_cycle_genes[43:]
g2m_genes = [x for x in g2m_genes if x in adata.var_names]
print('G2 genes:', g2m_genes, '\n')
cell_cycle_genes = [x for x in cell_cycle_genes if x in adata.var_names]

hb_genes1 = ['HBB','HBD','HBG1','HBG2','HBE1','HBZ','HBM','HBA2','HBA1', 'HBQ1']
hb_genes1 = [x for x in hb_genes1 if x in adata.var_names]
print('Hemoglobin genes set 1:', hb_genes1, '\n')

hb_genes2 = ['AHSP','GYPA','RHAG','GFI1B','KEL','EPB42','ALAS2','TSPO2','GYPB','GYPE','SPTA1','RHCE','KLF1','RHD','SLC4A1','SNX22','SQSTM1','SOD1','FTH1','MAP1LC3B','SLC25A37','TRIM58','HMBS']
hb_genes2 = [x for x in hb_genes2 if x in adata.var_names]
print('Hemoglobin genes set 2:', hb_genes2, '\n')

mito_genes=['MT-ND1','MT-ND2','MT-CO1','MT-CO2','MT-ATP8','MT-ATP6','MT-CO3','MT-ND3','MT-ND4L', 'MT-ND4', 'MT-ND5', 'MT-ND6', 'MT-CYB']
mito_genes = [x for x in mito_genes if x in adata.var_names]
print('Mitochondrial genes:', mito_genes, '\n')

ribo_genes = ['RPLP0','RPLP1','RPLP2','RPL3','RPL3L','RPL4','RPL5','RPL6','RPL7','RPL7A','RPL7L1','RPL8','RPL9','RPL10','RPL10A','RPL10L','RPL11','RPL12','RPL13','RPL13A','RPL14','RPL15','RPL17','RPL18','RPL18A','RPL19','RPL21','RPL22','RPL22L1','RPL23','RPL23A','RPL24','RPL26','RPL26L1','RPL27','RPL27A','RPL28','RPL29','RPL30','RPL31','RPL32','RPL34','RPL35','RPL35A','RPL36','RPL36A','RPL36AL','RPL37','RPL37A','RPL38','RPL39','RPL39L','UBA52','RPL41','RPSA','RPS2','RPS3','RPS3A','RPS4X','RPS4Y1','RPS4Y2','RPS5','RPS6','RPS7','RPS8','RPS9','RPS10','RPS11','RPS12','RPS13','RPS14','RPS15','RPS15A','RPS16','RPS17','RPS18','RPS19','RPS20','RPS21','RPS23','RPS24','RPS25','RPS26','RPS27','RPS27A','RPS27L','RPS28','RPS29','FAU']
ribo_genes = [x for x in ribo_genes if x in adata.var_names]
print('Ribo genes:', ribo_genes, '\n')

stress_genes=['FOS','FOSB','JUN','JUNB','JUND','ZFP36','SOCS3','CEBPD','CEBPB','ATF3','EGR1','HSPA1A','HSPA1B','HSP90AA1','HSP90AB1','HSPA8','HSPB1','HSPE1','IER3','IER2','BTG1','BTG2','DUSP1']
stress_genes = [x for x in stress_genes if x in adata.var_names]
print('Tissue dissociation stress associated genes:', stress_genes, '\n')

# preserve raw count
adata.layers["counts"] = adata.X.copy()

# normalise and log-transform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key='batch_key')

print(sum(adata.var.highly_variable))
sc.pl.highly_variable_genes(adata)

# freeze the state in `.raw`
adata.raw = adata

# subset hvgs
adata = adata[:, adata.var.highly_variable]

adata=adata.copy()

sc.tl.score_genes_cell_cycle(adata, s_genes=s_genes, g2m_genes=g2m_genes)
#sc.pp.regress_out(adata, ['S_score','G2M_score'])

#batch correction
scvi.model.SCVI.setup_anndata(adata, 
                              layer="counts", 
                              batch_key="batch_key",
                              categorical_covariate_keys=['phase'],
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo'])

vae = scvi.model.SCVI(adata, n_hidden = 128, n_latent = 50, n_layers = 3, dispersion = 'gene-batch')
vae.train(max_epochs=50)
    
# Convergence plot
x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
plt.plot(x, vae.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
plt.clf()

# add the latent space to adata
adata.obsm["X_scVI"] = vae.get_latent_representation()

#sc.tl.score_genes(adata, stress_genes, score_name='stress_score')

sc.tl.score_genes(adata, hb_genes1, score_name='hb1_score')

sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata,min_dist=0.1)
#sc.tl.tsne(adata)

sc.pl.umap(adata, color=['log1p_total_counts','n_genes','n_genes_by_counts','total_counts','pct_counts_mt','pct_counts_ribo'],ncols=5)
sc.pl.umap(adata, color=['scrublet_score', 'hb1_score','phase'])
sc.pl.umap(adata, color=['donor','age','region','phase','cell_or_nuclei','modality','kit_10x'],wspace=0.8)
```

Example 69 (python):
```python
Genes=['TTN','TNNT2','MYOM1','LDB3','ALDH1A2','TNNT1','KRT19','UPK3B','BNC1','COL6A3','FBLN1','PCOLCE','LRRC17','PCDH18','EMCN','ECSCR','PECAM1','CDH5','ESAM', 'LAPTM5','PTPRC','LCP1','CD53','WAS','PHOX2B','NRXN1','STMN2','GRIK2','L1CAM']
sc.pl.umap(adata, color=Genes)
```

Example 70 (python):
```python
sc.pl.umap(adata, color=['TTN','BNC1','COL6A3','PECAM1','PTPRC','LYVE1','NRXN1'])
```

Example 71 (python):
```python
sc.tl.leiden(adata, resolution=1, key_added='leiden')
sc.pl.umap(adata, color='leiden',legend_loc='on data')
```

Example 72 (python):
```python
#add coarse_grain
CardiomyocyteClusters=['0','5','28','22','32','15','19','1']
MesenchymalClusters=['23','3','10','2','14','7','12','30','27','4','20','8','16','21','9','25']
EndotheliumClusters=['34','11','13','24']
EpicardiumClusters=['17']
ImmuneClusters=['33','31','6','18']
NeuralClusters=['29','26']
adata.obs['coarse_grain']= 'na'
adata.obs['coarse_grain'][[x for x in adata.obs_names if adata.obs.leiden[x] in CardiomyocyteClusters]]='Cardiomyocytes'
adata.obs['coarse_grain'][[x for x in adata.obs_names if adata.obs.leiden[x] in MesenchymalClusters]]='Mesenchymal'
adata.obs['coarse_grain'][[x for x in adata.obs_names if adata.obs.leiden[x] in EndotheliumClusters]]='Endothelium'
adata.obs['coarse_grain'][[x for x in adata.obs_names if adata.obs.leiden[x] in EpicardiumClusters]]='Epicardium'
adata.obs['coarse_grain'][[x for x in adata.obs_names if adata.obs.leiden[x] in ImmuneClusters]]='Immune'
adata.obs['coarse_grain'][[x for x in adata.obs_names if adata.obs.leiden[x] in NeuralClusters]]='Neural'
sc.pl.umap(adata, color='coarse_grain')
```

Example 73 (python):
```python
set(adata.obs.donor)
```

Example 74 (python):
```python
adata.obs['cell_or_nuclei']= 'na'

adata.obs['cell_or_nuclei'][adata.obs['donor']=='Hst42']='nuclei'
adata.obs['cell_or_nuclei'][adata.obs['donor']=='Hst44']='nuclei'

sc.pl.umap(adata, color='cell_or_nuclei')
```

Example 75 (python):
```python
adata.obs['modality']= 'na'

adata.obs['modality'][adata.obs['cell_or_nuclei']=='nuclei']='scMultiome'

sc.pl.umap(adata, color='modality')
```

Example 76 (python):
```python
adata.obs['sex']= 'na'

adata.obs['sex'][adata.obs['donor']=='Hst42']='male'
adata.obs['sex'][adata.obs['donor']=='Hst44']='male'

sc.pl.umap(adata, color='sex')
```

Example 77 (python):
```python
adata.obs['FACSgate']= 'na'
sc.pl.umap(adata, color='FACSgate')
```

Example 78 (python):
```python
#all age taken in PCW and not GA
adata.obs['age']= 'na'

adata.obs['age'][adata.obs['donor']=='Hst42']='14W0D'
adata.obs['age'][adata.obs['donor']=='Hst44']='13W0D'

sc.pl.umap(adata, color='age')
```

Example 79 (python):
```python
#all age taken in PCW and not GA
adata.obs['week']= 'na'

adata.obs['week'][adata.obs['donor']=='Hst42']='14W'
adata.obs['week'][adata.obs['donor']=='Hst44']='13W'

sc.pl.umap(adata, color='week')
```

Example 80 (python):
```python
#all age taken in PCW and not GA
adata.obs['trimester']= 'na'

adata.obs['trimester'][adata.obs['week']=='14W']='secondtrimester'
adata.obs['trimester'][adata.obs['week']=='13W']='secondtrimester'

sc.pl.umap(adata, color='trimester')
```

Example 81 (python):
```python
set(adata.obs.region)
```

Example 82 (python):
```python
adata.obs.replace({'region':{'Great vessels':'great vessels'}}, inplace=True)
adata.obs.replace({'region':{'Heart':'whole sample'}}, inplace=True)
adata.obs.replace({'region':{'Left atria':'left atria'}}, inplace=True)
adata.obs.replace({'region':{'Left ventricle':'left ventricle'}}, inplace=True)
adata.obs.replace({'region':{'Pericardium':'pericardium'}}, inplace=True)
adata.obs.replace({'region':{'Right ventricle':'right ventricle'}}, inplace=True)

sc.pl.umap(adata, color='region')
```

Example 83 (python):
```python
sc.pl.umap(adata,color=['phase'])
adata.obs['cycling']= 'no'
adata.obs['cycling'][[x for x in adata.obs_names if adata.obs.phase[x] in ['G2M','S']]]='yes'
sc.pl.umap(adata,color=['cycling'])
```

Example 84 (python):
```python
del(adata.obs['donor_type'])
del(adata.obs['region_finest'])
del(adata.obs['gender'])
del(adata.obs['flushed'])
del(adata.obs['scrublet_leiden'])
del(adata.obs['cluster_scrublet_score'])
del(adata.obs['_scvi_labels'])
```

Example 85 (python):
```python
sc.pl.umap(adata, color=['donor','age','week','trimester','region','phase','cell_or_nuclei','cycling','modality','kit_10x','FACSgate','coarse_grain'],wspace=0.8,ncols=5)
```

Example 86 (python):
```python
adata.write('./Objects/T21HeartsExtracardiacRemovedIntegratedProcessed.h5ad')
```

Example 87 (python):
```python
adata_raw=sc.read('./Objects/T21_multiome_RNA_foetal.h5ad')
print(adata_raw)

# constructing new anndata with maintaining .obs, .obsm, .obsp, .uns
# overwriting adata_raw
adata_raw=anndata.AnnData(X=adata_raw[adata.obs_names,:].X,
                      obs=adata.obs, # from annotated or subsetted data
                      var=adata_raw.var,
                      obsm=adata.obsm, # from annotated or subsetted data
                      obsp=adata.obsp, # from annotated or subsetted data
                    uns=adata.uns # from annotated or subsetted data
                               )

# set gene name as var_names 
adata_raw.var['gene_id']=adata_raw.var_names.copy()
adata_raw.var.set_index('gene_name',inplace=True)
adata_raw.var_names=adata_raw.var_names.astype('str')
adata_raw.var_names_make_unique()
sc.pl.umap(adata_raw, color=['TNNT2'])

adata=adata_raw.copy()
print(adata)
adata.write('./Objects/T21HeartsExtracardiacRemovedRaw.h5ad')
```

---

## Samples¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/RNA/QC/Demultiplex/No1_SOC_results_multiome-ATAC.html

**Contents:**
- Samples¶
- Read in and concatenate clusters.tsv¶
- Genotype matching¶
- 7089STDY13216920_BHF_F_Hea13242527¶
- 7089STDY13216921_BHF_F_Hea13242528¶
- 7089STDY13216922_BHF_F_Hea13242529¶
- 7089STDY13216923_BHF_F_Hea13242530¶
- Save clusters dataframe¶
- Proportions¶
- Doublets and scrublet score¶

74984 rows × 9 columns

will assign donors as below

**Examples:**

Example 1 (python):
```python
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from scipy.stats import gaussian_kde

import scanpy as sc
```

Example 2 (unknown):
```unknown
# to use magic command %%R
%load_ext rpy2.ipython
```

Example 3 (unknown):
```unknown
%%R
library(GenotypeMixtures)
library(tidyverse)
```

Example 4 (unknown):
```unknown
── Attaching packages ─────────────────────────────────────── tidyverse 1.3.2 ──
✔ ggplot2 3.4.0      ✔ purrr   0.3.4 
✔ tibble  3.1.8      ✔ dplyr   1.0.10
✔ tidyr   1.2.1      ✔ stringr 1.4.0 
✔ readr   2.1.3      ✔ forcats 0.5.1 
── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter() masks stats::filter()
✖ dplyr::lag()    masks stats::lag()
```

Example 5 (python):
```python
multiplexed_samples = pd.read_csv('/nfs/team205/heart/mapped/cellranger-arc200/csv/multiome-foetal_20221221.csv')
multiplexed_samples
```

Example 6 (python):
```python
# read in metadata
meta = pd.read_csv('/nfs/team205/heart/anndata_objects/Foetal/metadata/HeartTeamSamples_Mappeddata_20221223.csv', sep = ',', index_col = None)
meta = meta[meta['Publication']=='Foetal']
meta = meta[['sangerID', 'combinedID',
       'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender',
       'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed',
       'status']]

# select samples
meta = meta[meta['combinedID'].isin(multiplexed_samples['combinedID'])]

# select Multiome-ATAC
meta = meta[meta['modality']=='Multiome-ATAC']

# reindex
meta.index = range(len(meta))

meta
```

Example 7 (python):
```python
# min-ref/alt = 4
clusters_all = {}
multiplexed_IDs = ['7089STDY13216920_BHF_F_Hea13242527',
                   '7089STDY13216921_BHF_F_Hea13242528',
                   '7089STDY13216922_BHF_F_Hea13242529',
                   '7089STDY13216923_BHF_F_Hea13242530']

for i,ID in enumerate(multiplexed_IDs):
    df = pd.read_csv(f'/nfs/team205/heart/mapped/cellranger-arc200/SOC_out/{ID}/ATAC_SOC_common_variants/clusters.tsv', sep='\t')
    df['sample'] = ID
    if i==0:
        clusters_all = df.copy()
    else:
        clusters_all = pd.concat([clusters_all,df])
clusters_all = clusters_all[['sample', 'barcode', 'status', 'assignment', 'log_prob_singleton',
                           'log_prob_doublet', 'cluster0', 'cluster1', 'cluster2']]

# combine sample and barcode name: adata.obs_names
clusters_all['obsnames'] = clusters_all['sample'].astype('str')+'_'+clusters_all['barcode'].astype('str')
clusters_all.set_index('obsnames',inplace=True)

print(clusters_all['status'].value_counts())
clusters_all
```

Example 8 (javascript):
```javascript
singlet       63170
doublet        8383
unassigned     3431
Name: status, dtype: int64
```

Example 9 (unknown):
```unknown
set(clusters_all['sample'])
```

Example 10 (unknown):
```unknown
{'7089STDY13216920_BHF_F_Hea13242527',
 '7089STDY13216921_BHF_F_Hea13242528',
 '7089STDY13216922_BHF_F_Hea13242529',
 '7089STDY13216923_BHF_F_Hea13242530'}
```

Example 11 (python):
```python
# proportion of status in each genotype assignment
# most of them are singlet in single genotype assignment which is good
df = clusters_all.copy()

fig, axs = plt.subplots(nrows=1, ncols=len(multiplexed_IDs), figsize=(20,4)) 

for i,sam in enumerate(multiplexed_IDs):

    df = clusters_all[clusters_all['sample']==sam]
    tmp = pd.crosstab(df['assignment'], df['status'], normalize='index')
    # tmp.plot.bar().legend(loc=1)
    tmp.plot.bar(stacked=True,width=0.7,ax=axs[i])
    axs[i].grid(False)
    axs[i].set_ylabel('proportion')
    axs[i].set_title(sam)
    axs[i].legend(bbox_to_anchor=(1.0, 1.0))
    
plt.tight_layout()
```

Example 12 (python):
```python
# Proportion of status in each sample
# most are singlet
df = clusters_all.copy()
plt.rcParams['figure.figsize'] = 3,3
tmp = pd.crosstab(df['sample'], df['status'], normalize='index')
# tmp.plot.bar().legend(loc=1)
ax = tmp.plot.bar(stacked=True,width=0.7)
ax.grid(False)
ax.set_ylabel('proportion')
ax.legend(bbox_to_anchor=(1.0, 1.0))
ax.plot()
plt.show()
```

Example 13 (unknown):
```unknown
# prepare the column for donor assignment
clusters_all['donor_assignment'] = 'donor_unassigned'
```

Example 14 (javascript):
```javascript
%%R -i meta
# prepare wrapper function
plot_result = function(ID, Donor){
    output_dir = '/nfs/team205/heart/mapped/cellranger-arc200/SOC_out/'
    
    if (Donor == 'Hst39'){
        ID_ref = '7089STDY13216926_BHF_F_Hea13242533'
    }else{
        ID_ref = meta %>% filter(donor==Donor) %>% pull(combinedID)
    }
    
    plot_cross_vaf(experiment_1_path = paste0(output_dir,ID,'/ATAC_SOC_common_variants/'), 
                   experiment_2_path = paste0(output_dir,ID_ref,'/ATAC_SOC_common_variants/'), 
                   experiment_1_name = strsplit(ID,'_')[[1]][1],
                   experiment_2_name = Donor)
}
```

Example 15 (unknown):
```unknown
targetID = '7089STDY13216920_BHF_F_Hea13242527'
```

Example 16 (unknown):
```unknown
%%R -i targetID
plot_result(ID=targetID, Donor='Hst39')
```

Example 17 (unknown):
```unknown
# assign donors
obsnames = clusters_all.index[(clusters_all['sample']==targetID)&\
                              (clusters_all['status']=='singlet')&\
                              (clusters_all['assignment']=='0')]
clusters_all.loc[obsnames,'donor_assignment'] = 'Hst39'
targetID
```

Example 18 (unknown):
```unknown
'7089STDY13216920_BHF_F_Hea13242527'
```

Example 19 (unknown):
```unknown
%%R -i targetID
plot_result(ID=targetID, Donor='Hst40')
```

Example 20 (unknown):
```unknown
# assign donors
obsnames = clusters_all.index[(clusters_all['sample']==targetID)&\
                              (clusters_all['status']=='singlet')&\
                              (clusters_all['assignment']=='1')]
clusters_all.loc[obsnames,'donor_assignment'] = 'Hst40'
targetID
```

Example 21 (unknown):
```unknown
'7089STDY13216920_BHF_F_Hea13242527'
```

Example 22 (unknown):
```unknown
targetID = '7089STDY13216921_BHF_F_Hea13242528'
```

Example 23 (unknown):
```unknown
%%R -i targetID
plot_result(ID=targetID, Donor='Hst39')
```

Example 24 (unknown):
```unknown
# assign donors
obsnames = clusters_all.index[(clusters_all['sample']==targetID)&\
                              (clusters_all['status']=='singlet')&\
                              (clusters_all['assignment']=='1')]
clusters_all.loc[obsnames,'donor_assignment'] = 'Hst39'
targetID
```

Example 25 (unknown):
```unknown
'7089STDY13216921_BHF_F_Hea13242528'
```

Example 26 (unknown):
```unknown
%%R -i targetID
plot_result(ID=targetID, Donor='Hst40')
```

Example 27 (unknown):
```unknown
# assign donors
obsnames = clusters_all.index[(clusters_all['sample']==targetID)&\
                              (clusters_all['status']=='singlet')&\
                              (clusters_all['assignment']=='0')]
clusters_all.loc[obsnames,'donor_assignment'] = 'Hst40'
targetID
```

Example 28 (unknown):
```unknown
'7089STDY13216921_BHF_F_Hea13242528'
```

Example 29 (unknown):
```unknown
%%R
output_dir = '/nfs/team205/heart/mapped/cellranger-arc200/SOC_out/'
plot_cross_vaf(experiment_1_path = paste0(output_dir,'7089STDY13216920_BHF_F_Hea13242527','/ATAC_SOC_common_variants/'), 
                   experiment_2_path = paste0(output_dir,'7089STDY13216921_BHF_F_Hea13242528','/ATAC_SOC_common_variants/'), 
                   experiment_1_name = '7089STDY13216920',
                   experiment_2_name = '7089STDY13216921')
```

Example 30 (unknown):
```unknown
targetID = '7089STDY13216922_BHF_F_Hea13242529'
```

Example 31 (unknown):
```unknown
%%R -i targetID
plot_result(ID=targetID, Donor='Hst36')
```

Example 32 (unknown):
```unknown
# assign donors
obsnames = clusters_all.index[(clusters_all['sample']==targetID)&\
                              (clusters_all['status']=='singlet')&\
                              (clusters_all['assignment']=='2')]
clusters_all.loc[obsnames,'donor_assignment'] = 'Hst36'
targetID
```

Example 33 (unknown):
```unknown
'7089STDY13216922_BHF_F_Hea13242529'
```

Example 34 (unknown):
```unknown
%%R -i targetID
plot_result(ID=targetID, Donor='Hst39')
```

Example 35 (unknown):
```unknown
# assign donors
obsnames = clusters_all.index[(clusters_all['sample']==targetID)&\
                              (clusters_all['status']=='singlet')&\
                              (clusters_all['assignment']=='0')]
clusters_all.loc[obsnames,'donor_assignment'] = 'Hst39'
targetID
```

Example 36 (unknown):
```unknown
'7089STDY13216922_BHF_F_Hea13242529'
```

Example 37 (unknown):
```unknown
%%R -i targetID
plot_result(ID=targetID, Donor='Hst40')
```

Example 38 (unknown):
```unknown
# assign donors
obsnames = clusters_all.index[(clusters_all['sample']==targetID)&\
                              (clusters_all['status']=='singlet')&\
                              (clusters_all['assignment']=='1')]
clusters_all.loc[obsnames,'donor_assignment'] = 'Hst40'
targetID
```

Example 39 (unknown):
```unknown
'7089STDY13216922_BHF_F_Hea13242529'
```

Example 40 (unknown):
```unknown
targetID = '7089STDY13216923_BHF_F_Hea13242530'
```

Example 41 (unknown):
```unknown
%%R -i targetID
plot_result(ID=targetID, Donor='Hst36')
```

Example 42 (unknown):
```unknown
'''
obsnames = clusters_all.index[(clusters_all['sample']==targetID)&(clusters_all['assignment']==    )]
clusters_all.loc[obsnames,'donor_assignment'] = 'Hst36'
targetID
'''
```

Example 43 (unknown):
```unknown
%%R -i targetID
plot_result(ID=targetID, Donor='Hst39')
```

Example 44 (unknown):
```unknown
'''
obsnames = clusters_all.index[(clusters_all['sample']==targetID)&(clusters_all['assignment']==    )]
clusters_all.loc[obsnames,'donor_assignment'] = 'Hst39'
targetID
'''
```

Example 45 (unknown):
```unknown
%%R -i targetID
plot_result(ID=targetID, Donor='Hst40')
```

Example 46 (unknown):
```unknown
'''
obsnames = clusters_all.index[(clusters_all['sample']==targetID)&(clusters_all['assignment']==    )]
clusters_all.loc[obsnames,'donor_assignment'] = 'Hst40'
targetID
'''
```

Example 47 (unknown):
```unknown
%%R
output_dir = '/nfs/team205/heart/mapped/cellranger-arc200/SOC_out/'
plot_cross_vaf(experiment_1_path = paste0(output_dir,'7089STDY13216922_BHF_F_Hea13242529','/ATAC_SOC_common_variants/'), 
                   experiment_2_path = paste0(output_dir,'7089STDY13216923_BHF_F_Hea13242530','/ATAC_SOC_common_variants/'), 
                   experiment_1_name = '7089STDY13216922',
                   experiment_2_name = '7089STDY13216923')
```

Example 48 (unknown):
```unknown
clusters_all.head()
```

Example 49 (unknown):
```unknown
clusters_all.to_csv('/nfs/team205/heart/mapped/cellranger-arc200/SOC_out/SOC-clusters_multiome-foetal_20221221.csv')
```

Example 50 (python):
```python
df = clusters_all.copy()
plt.rcParams['figure.figsize'] = 3,3
tmp = pd.crosstab(df['sample'], df['donor_assignment'], normalize='index')
# tmp.plot.bar().legend(loc=1)
ax = tmp.plot.bar(stacked=True,width=0.7)
ax.grid(False)
ax.set_ylabel('proportion')
ax.legend(bbox_to_anchor=(1.0, 1.0))
ax.plot()
plt.show()

for sam in multiplexed_IDs:
    df_s = df[df['sample']==sam]
    print(f'##### {sam} #####')
    print(df_s['donor_assignment'].value_counts())
    print('')
```

Example 51 (unknown):
```unknown
##### 7089STDY13216920_BHF_F_Hea13242527 #####
Hst39               16670
Hst40                6894
donor_unassigned     3934
Name: donor_assignment, dtype: int64

##### 7089STDY13216921_BHF_F_Hea13242528 #####
Hst39               14867
Hst40                5944
donor_unassigned     3400
Name: donor_assignment, dtype: int64

##### 7089STDY13216922_BHF_F_Hea13242529 #####
Hst40               7408
Hst36               5093
Hst39               4786
donor_unassigned    4115
Name: donor_assignment, dtype: int64

##### 7089STDY13216923_BHF_F_Hea13242530 #####
donor_unassigned    1873
Name: donor_assignment, dtype: int64
```

Example 52 (python):
```python
# reason why donor was unassigned
df = clusters_all[clusters_all['donor_assignment']=='donor_unassigned']
plt.rcParams['figure.figsize'] = 3,3
tmp = pd.crosstab(df['sample'], df['status'], normalize='index')
# tmp.plot.bar().legend(loc=1)
ax = tmp.plot.bar(stacked=True,width=0.7)
ax.grid(False)
ax.set_ylabel('proportion')
ax.legend(bbox_to_anchor=(1.0, 1.0))
ax.plot()
plt.show()

for sam in multiplexed_IDs:
    df_s = df[df['sample']==sam]
    print(f'##### {sam} #####')
    print(df_s['status'].value_counts())
    print('')
```

Example 53 (javascript):
```javascript
##### 7089STDY13216920_BHF_F_Hea13242527 #####
doublet       3086
unassigned     848
Name: status, dtype: int64

##### 7089STDY13216921_BHF_F_Hea13242528 #####
doublet       2655
unassigned     745
Name: status, dtype: int64

##### 7089STDY13216922_BHF_F_Hea13242529 #####
doublet       2362
unassigned    1753
Name: status, dtype: int64

##### 7089STDY13216923_BHF_F_Hea13242530 #####
singlet       1508
doublet        280
unassigned      85
Name: status, dtype: int64
```

Example 54 (python):
```python
# post-cellbender anndata
adata = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/QC/multiome_RNA_foetal_prefilter.h5ad',
                     backed = 'r')
# only multiplexed data
adata = adata[adata.obs['combinedID'].isin(multiplexed_IDs)]

adata
```

Example 55 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 74984 × 36601 backed at '/nfs/team205/heart/anndata_objects/Foetal/QC/multiome_RNA_foetal_prefilter.h5ad'
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval'
    var: 'gene_name', 'feature_type', 'ambient_expression-0', 'ambient_expression-1', 'ambient_expression-10', 'ambient_expression-11', 'ambient_expression-12', 'ambient_expression-13', 'ambient_expression-14', 'ambient_expression-15', 'ambient_expression-16', 'ambient_expression-17', 'ambient_expression-18', 'ambient_expression-19', 'ambient_expression-2', 'ambient_expression-20', 'ambient_expression-21', 'ambient_expression-22', 'ambient_expression-23', 'ambient_expression-24', 'ambient_expression-25', 'ambient_expression-26', 'ambient_expression-27', 'ambient_expression-28', 'ambient_expression-29', 'ambient_expression-3', 'ambient_expression-30', 'ambient_expression-31', 'ambient_expression-32', 'ambient_expression-33', 'ambient_expression-34', 'ambient_expression-35', 'ambient_expression-36', 'ambient_expression-37', 'ambient_expression-4', 'ambient_expression-5', 'ambient_expression-6', 'ambient_expression-7', 'ambient_expression-8', 'ambient_expression-9'
    obsm: 'latent_gene_encoding'
```

Example 56 (python):
```python
sc.settings.set_figure_params(dpi=120)
```

Example 57 (python):
```python
# add
col_to_add = ['status', 'assignment', 'log_prob_singleton',
               'log_prob_doublet', 'cluster0', 'cluster1', 'cluster2',
               'donor_assignment']
df = adata.obs.copy()
df[[f'SOC | {x}' for x in col_to_add]] = clusters_all[col_to_add].reindex(df.index)

sns.violinplot(data=df, x="scrublet_score", y="SOC | status")
plt.show()

df['SOC | status'].value_counts()
```

Example 58 (javascript):
```javascript
singlet       63170
doublet        8383
unassigned     3431
Name: SOC | status, dtype: int64
```

---

## Function¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig4_Cardiomyocytes/VentricularCardiomyocytes/no2_plotting-transition_pathway_analysis.html

**Contents:**
- Function¶
- Read in gene signatures and define target genes¶
- Pathway analysis¶

ref: https://yoseflab.github.io/Hotspot/CD4_Tutorial.html

**Examples:**

Example 1 (python):
```python
# Parameters
working_dir = "/nfs/team205/heart/Vitro/TF_selection"
path_to_adata_gex_dir = "/nfs/team205/heart/anndata_objects/Vitro/post_multiVI_full-features"
path_to_adata_peak_tfidf = "/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/PeakMatrix/Foetal_Peaks_tfidf.h5ad"
group_key = "t_binned"
group_a = "t3"
group_b = "t1"
time_key = "t"
subset_object = False
geneset_list = ["MSigDB_Hallmark_2020","GO_Biological_Process_2023"]

path_to_eRegulon_signatures_gene = "/nfs/team205/heart/pyscenic/plus/eRegulon_signatures/vCM_onlyLeft_Gene_eRegulon_signatures.pkl"
path_to_eRegulon_signatures_region = "/nfs/team205/heart/pyscenic/plus/eRegulon_signatures/vCM_onlyLeft_Region_eRegulon_signatures.pkl"

path_to_DAR_compact = '/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/MarkerPeaks/vCM_Left_Compact-vs-Trab.csv'
path_to_DAR_trab = '/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/MarkerPeaks/vCM_Left_Trab-vs-Compact.csv'
path_to_eRegulon_metadata = "/nfs/team205/heart/pyscenic/plus/eRegulon_metadata/vCM_onlyLeft_eRegulon_metadata_filtered.csv" # filtered metadata

path_to_gene_signatures = '/nfs/team205/heart/Vitro/TF_selection/foetal_vCM_left_compact-vs-trabeculated_downstream/gene_signatures.pkl'
```

Example 2 (python):
```python
import scanpy as sc
import pandas as pd
import numpy as np
import anndata

import matplotlib.pyplot as plt
import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 
import seaborn as sns

import hotspot
```

Example 3 (python):
```python
import session_info
session_info.show()
```

Example 4 (python):
```python
-----
anndata             0.9.2
hotspot             1.1.1
matplotlib          3.7.2
numpy               1.24.4
pandas              1.5.3
scanpy              1.9.3
seaborn             0.12.2
session_info        1.0.0
-----
```

Example 5 (python):
```python
PIL                         10.0.0
anyio                       NA
arrow                       1.2.3
asttokens                   NA
attr                        23.1.0
attrs                       23.1.0
babel                       2.12.1
backcall                    0.2.0
brotli                      1.0.9
certifi                     2023.11.17
cffi                        1.15.1
charset_normalizer          3.2.0
colorama                    0.4.6
comm                        0.1.4
cycler                      0.10.0
cython_runtime              NA
dateutil                    2.8.2
debugpy                     1.6.8
decorator                   5.1.1
defusedxml                  0.7.1
executing                   1.2.0
fastjsonschema              NA
fqdn                        NA
h5py                        3.9.0
idna                        3.4
igraph                      0.10.6
importlib_metadata          NA
importlib_resources         NA
ipykernel                   6.25.1
isoduration                 NA
jedi                        0.19.0
jinja2                      3.1.2
joblib                      1.3.2
json5                       NA
jsonpointer                 2.0
jsonschema                  4.19.0
jsonschema_specifications   NA
jupyter_events              0.7.0
jupyter_server              2.7.1
jupyterlab_server           2.24.0
kiwisolver                  1.4.5
leidenalg                   0.10.1
llvmlite                    0.40.1
markupsafe                  2.1.3
mpl_toolkits                NA
natsort                     8.4.0
nbformat                    5.9.2
numba                       0.57.1
overrides                   NA
packaging                   23.1
parso                       0.8.3
patsy                       0.5.3
pexpect                     4.8.0
pickleshare                 0.7.5
pkg_resources               NA
plotly                      5.16.1
prometheus_client           NA
prompt_toolkit              3.0.39
psutil                      5.9.5
ptyprocess                  0.7.0
pure_eval                   0.2.2
pycparser                   2.21
pydev_ipython               NA
pydevconsole                NA
pydevd                      2.9.5
pydevd_file_utils           NA
pydevd_plugins              NA
pydevd_tracing              NA
pygments                    2.16.1
pynndescent                 0.5.10
pyparsing                   3.0.9
pythonjsonlogger            NA
pytz                        2023.3
referencing                 NA
requests                    2.31.0
rfc3339_validator           0.1.4
rfc3986_validator           0.1.1
rpds                        NA
ruamel                      NA
scipy                       1.10.1
send2trash                  NA
six                         1.16.0
sklearn                     1.3.0
sniffio                     1.3.0
socks                       1.7.1
sphinxcontrib               NA
stack_data                  0.6.2
statsmodels                 0.14.0
texttable                   1.6.7
threadpoolctl               3.2.0
tornado                     6.3.3
tqdm                        4.66.1
traitlets                   5.9.0
typing_extensions           NA
uri_template                NA
urllib3                     2.0.4
wcwidth                     0.2.6
webcolors                   1.13
websocket                   1.6.2
yaml                        6.0.1
zipp                        NA
zmq                         25.1.1
zstandard                   0.19.0
```

Example 6 (unknown):
```unknown
-----
IPython             8.12.0
jupyter_client      8.3.0
jupyter_core        4.12.0
jupyterlab          4.0.5
notebook            7.0.2
-----
Python 3.8.17 (default, Jul  5 2023, 21:04:15) [GCC 11.2.0]
Linux-5.4.0-171-generic-x86_64-with-glibc2.17
-----
Session information updated at 2024-02-22 22:51
```

Example 7 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)

spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/regulon_plot.py")
regulon_plot = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = regulon_plot
spec.loader.exec_module(regulon_plot)

spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/tissueaxis.py")
tissueaxis = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = tissueaxis
spec.loader.exec_module(tissueaxis)
```

Example 8 (python):
```python
sc.settings.set_figure_params(dpi=80,dpi_save=300)
```

Example 9 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk'
```

Example 10 (python):
```python
gene_signatures = utils.read_pkl(path_to_gene_signatures)
for k,v in gene_signatures.items():
    print(f'{k}: {len(v)} genes')
```

Example 11 (unknown):
```unknown
compact_specific_up: 130 genes
trab_specific_up: 107 genes
common_up: 336 genes
compact_specific_down: 185 genes
trab_specific_down: 384 genes
common_down: 738 genes
```

Example 12 (python):
```python
import gseapy as gp
```

Example 13 (python):
```python
key = 'compact_specific_up'
adj_pval_thresh = 0.1
metrics = 'n_genes'
n_rank_plot = 10

enr = gp.enrichr(gene_list=gene_signatures[key],
                 gene_sets=geneset_list,
                 organism='human', # don't forget to set organism to the one you desired! e.g. Yeast
                 outdir=None, # don't write to disk
                )
enr.results['n_genes'] = [int(x.split('/')[0]) for x in enr.results['Overlap']]

for geneset in geneset_list:
    # select based on thresholds
    df = enr.results[enr.results['Gene_set'] == geneset]
    df = df[df['Adjusted P-value'] < adj_pval_thresh]
    if len(df)>0:
        print(geneset)
        df = df.sort_values(metrics,ascending=False)
        display(df[:30])
        
        # plot
        plt.rcParams['axes.grid'] = False
        plt.rcParams['figure.figsize'] = 4,2
        plt.barh(width=df[:n_rank_plot][metrics],
                 y=[x.split(' (')[0] for x in df[:n_rank_plot]['Term']],
                 color='darkred')
        plt.gca().invert_yaxis()
        plt.xlabel(metrics)
        plt.title(f'{key}, {geneset}')
        plt.margins(y=0.03)
        plt.savefig(f'{figdir}/Pathway-enr_vCM-left_{key}_{geneset}.pdf', bbox_inches="tight",dpi=300)
        plt.show()
```

Example 14 (unknown):
```unknown
MSigDB_Hallmark_2020
```

Example 15 (unknown):
```unknown
GO_Biological_Process_2023
```

Example 16 (python):
```python
key = 'trab_specific_up'
adj_pval_thresh = 0.1
metrics = 'n_genes'
n_rank_plot = 10

enr = gp.enrichr(gene_list=gene_signatures[key],
                 gene_sets=geneset_list,
                 organism='human', # don't forget to set organism to the one you desired! e.g. Yeast
                 outdir=None, # don't write to disk
                )
enr.results['n_genes'] = [int(x.split('/')[0]) for x in enr.results['Overlap']]

for geneset in geneset_list:
    # select based on thresholds
    df = enr.results[enr.results['Gene_set'] == geneset]
    df = df[df['Adjusted P-value'] < adj_pval_thresh]
    if len(df)>0:
        print(geneset)
        df = df.sort_values(metrics,ascending=False)
        display(df[:30])
        
        # plot
        plt.rcParams['axes.grid'] = False
        plt.rcParams['figure.figsize'] = 4,3
        plt.barh(width=df[:n_rank_plot][metrics],
                 y=[x.split(' (')[0] for x in df[:n_rank_plot]['Term']],
                 color='darkred')
        plt.gca().invert_yaxis()
        plt.xlabel(metrics)
        plt.title(f'{key}, {geneset}')
        plt.margins(y=0.03)
        plt.savefig(f'{figdir}/Pathway-enr_vCM-left_{key}_{geneset}.pdf', bbox_inches="tight",dpi=300)
        plt.show()
```

Example 17 (unknown):
```unknown
MSigDB_Hallmark_2020
```

Example 18 (python):
```python
key = 'common_up'
adj_pval_thresh = 0.1
metrics = 'n_genes'
n_rank_plot = 10

enr = gp.enrichr(gene_list=gene_signatures[key],
                 gene_sets=geneset_list,
                 organism='human', # don't forget to set organism to the one you desired! e.g. Yeast
                 outdir=None, # don't write to disk
                )
enr.results['n_genes'] = [int(x.split('/')[0]) for x in enr.results['Overlap']]

for geneset in geneset_list:
    # select based on thresholds
    df = enr.results[enr.results['Gene_set'] == geneset]
    df = df[df['Adjusted P-value'] < adj_pval_thresh]
    if len(df)>0:
        print(geneset)
        df = df.sort_values(metrics,ascending=False)
        display(df[:30])
        
        # plot
        plt.rcParams['axes.grid'] = False
        plt.rcParams['figure.figsize'] = 4,3
        plt.barh(width=df[:n_rank_plot][metrics],
                 y=[x.split(' (')[0] for x in df[:n_rank_plot]['Term']],
                 color='darkred')
        plt.gca().invert_yaxis()
        plt.xlabel(metrics)
        plt.title(f'{key}, {geneset}')
        plt.margins(y=0.03)
        plt.savefig(f'{figdir}/Pathway-enr_vCM-left_{key}_{geneset}.pdf', bbox_inches="tight",dpi=300)
        plt.show()
```

Example 19 (unknown):
```unknown
MSigDB_Hallmark_2020
```

Example 20 (unknown):
```unknown
GO_Biological_Process_2023
```

Example 21 (python):
```python
key = 'compact_specific_down'
adj_pval_thresh = 0.1
metrics = 'n_genes'
n_rank_plot = 10

enr = gp.enrichr(gene_list=gene_signatures[key],
                 gene_sets=geneset_list,
                 organism='human', # don't forget to set organism to the one you desired! e.g. Yeast
                 outdir=None, # don't write to disk
                )
enr.results['n_genes'] = [int(x.split('/')[0]) for x in enr.results['Overlap']]

for geneset in geneset_list:
    # select based on thresholds
    df = enr.results[enr.results['Gene_set'] == geneset]
    df = df[df['Adjusted P-value'] < adj_pval_thresh]
    if len(df)>0:
        print(geneset)
        df = df.sort_values(metrics,ascending=False)
        display(df[:30])
        
        # plot
        plt.rcParams['axes.grid'] = False
        plt.rcParams['figure.figsize'] = 4,3
        plt.barh(width=df[:n_rank_plot][metrics],
                 y=[x.split(' (')[0] for x in df[:n_rank_plot]['Term']],
                 color='darkred')
        plt.gca().invert_yaxis()
        plt.xlabel(metrics)
        plt.title(f'{key}, {geneset}')
        plt.margins(y=0.03)
        plt.show()
```

Example 22 (unknown):
```unknown
MSigDB_Hallmark_2020
```

Example 23 (unknown):
```unknown
GO_Biological_Process_2023
```

Example 24 (python):
```python
key = 'common_down'
adj_pval_thresh = 0.1
metrics = 'n_genes'
n_rank_plot = 10

enr = gp.enrichr(gene_list=gene_signatures[key],
                 gene_sets=geneset_list,
                 organism='human', # don't forget to set organism to the one you desired! e.g. Yeast
                 outdir=None, # don't write to disk
                )
enr.results['n_genes'] = [int(x.split('/')[0]) for x in enr.results['Overlap']]

for geneset in geneset_list:
    # select based on thresholds
    df = enr.results[enr.results['Gene_set'] == geneset]
    df = df[df['Adjusted P-value'] < adj_pval_thresh]
    if len(df)>0:
        print(geneset)
        df = df.sort_values(metrics,ascending=False)
        display(df[:30])
        
        # plot
        plt.rcParams['axes.grid'] = False
        plt.rcParams['figure.figsize'] = 4,3
        plt.barh(width=df[:n_rank_plot][metrics],
                 y=[x.split(' (')[0] for x in df[:n_rank_plot]['Term']],
                 color='darkred')
        plt.gca().invert_yaxis()
        plt.xlabel(metrics)
        plt.title(f'{key}, {geneset}')
        plt.margins(y=0.03)
        plt.show()
```

Example 25 (unknown):
```unknown
MSigDB_Hallmark_2020
```

Example 26 (unknown):
```unknown
GO_Biological_Process_2023
```

---

## Variables¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/Xenium/no2_integration_and_cell-type-annotation/no6_manual-annotation_Glia.html

**Contents:**
- Variables¶
- Read in data¶
- 1st round annotation¶
- Subset and reclustering¶
- Transfer annotation to the original adata¶
- Save¶

--> cluster 1 has lower total counts and smaller cell area, and no specific gene expression --> annotate as "unassigned"

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os

import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 

import scanpy.external as sce
```

Example 2 (python):
```python
# import session_info
# session_info.show()
```

Example 3 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/home/kk837/rfs/rfs-teichlab-nfs-iCNyzSAaucw/kk837/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 4 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 5 (unknown):
```unknown
!ls /lustre/scratch126/cellgen/team205/kk18/xenium/objects/foetal/subsets
```

Example 6 (unknown):
```unknown
ls: cannot access '/lustre/scratch126/cellgen/team205/kk18/xenium/objects/foetal/subsets': No such file or directory
```

Example 7 (python):
```python
adata_dir = '/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium/subsets'
sample_id = 'C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat'
celltype = 'Glia'
```

Example 8 (unknown):
```unknown
# figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk/Aug2024'
# table_dir = '/nfs/team205/heart/fetal_ms_figures/tables/Aug2024'
```

Example 9 (unknown):
```unknown
latent_space = 'pca_harmony'
```

Example 10 (python):
```python
# read in
adata = sc.read_h5ad(f'{adata_dir}/{sample_id}_5K_{celltype}_lognorm.h5ad')
adata
```

Example 11 (unknown):
```unknown
AnnData object with n_obs × n_vars = 6976 × 4699
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_3.0', 'multi_celltypes_coarse', 'leiden_0.2', 'leiden_0.4', 'leiden_0.6', 'leiden_1'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1', 'n_cells', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'dendrogram_pca_harmony_leiden_1.0', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1_colors', 'log1p', 'multi_celltypes_coarse_colors', 'pca', 'pca_harmony', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.2_colors', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.3_colors', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.4_colors', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.5_colors', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_0.8_colors', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.0_colors', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_1.5_colors', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_2.0_colors', 'pca_harmony_leiden_3.0', 'pca_harmony_leiden_3.0_colors', 'tissue_block_id_colors', 'umap'
    obsm: 'X_pca', 'X_pca_harmony', 'X_umap_pca_harmony', 'spatial'
    varm: 'PCs'
    obsp: 'pca_harmony_connectivities', 'pca_harmony_distances'
```

Example 12 (python):
```python
adata.X.data[:5]
```

Example 13 (unknown):
```unknown
array([3.036935, 3.036935, 3.036935, 3.705799, 3.036935], dtype=float32)
```

Example 14 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=['total_counts','n_genes_by_counts','cell_area','tissue_block_id'],
                    wspace=0.2,cmap='RdPu',vmax='p100')
```

Example 15 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=['multi_celltypes_coarse',"conf_score_midmod2fine","celltypist_midmod2fine",],
                    wspace=0.2,cmap='RdPu',vmax='p100')
resolutions_list = [0.2,0.4,0.6,1]
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                    color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                    wspace=0.2)
```

Example 16 (python):
```python
%%time
# leiden clustering
resolutions_list = [# 0.2,0.4,0.6,1, # already done
                    1.2,1.5,2]
for resolution in resolutions_list:
    sc.tl.leiden(adata,neighbors_key=latent_space,
                     resolution=resolution,key_added=f'leiden_{str(resolution)}',n_iterations=2)
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                wspace=0.3)
```

Example 17 (unknown):
```unknown
<timed exec>:5: FutureWarning: In the future, the default backend for leiden will be igraph instead of leidenalg.

 To achieve the future defaults please pass: flavor="igraph" and n_iterations=2.  directed must also be False to work with igraph's implementation.
```

Example 18 (unknown):
```unknown
CPU times: user 4.52 s, sys: 75.7 ms, total: 4.6 s
Wall time: 4.78 s
```

Example 19 (python):
```python
# markers for fine-grained cell types
markers = {
    "astrocytes": ["GFAP", "S100B", "ALDH1L1"],
    "oligodendrocytes": ["OLIG2", "MBP", "PDGFRA"],
    "microglia": ["AIF1"],  # IBA1 protein is encoded by the AIF1 gene
    "schwann_cells": ["SOX10", "MPZ", "S100B"],
    "schwann_cell_precursors": ["NGFR", "SOX10", "ERBB3"],
    'Cycling':['MKI67','TOP2A'],
    'Others':['NGF']
}

genes = []
for g in markers.values():
    genes = genes + [x for x in g if (x in adata.var_names)&(x not in genes)]
for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in adata.var_names]
    
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=genes,
               ncols=5, wspace=0.2,cmap='RdPu',vmax='p100')
```

Example 20 (unknown):
```unknown
AnnData object with n_obs × n_vars = 6976 × 4699
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_3.0', 'multi_celltypes_coarse', 'leiden_0.2', 'leiden_0.4', 'leiden_0.6', 'leiden_1', 'leiden_1.2', 'leiden_1.5', 'leiden_2'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1', 'n_cells', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'dendrogram_pca_harmony_leiden_1.0', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1_colors', 'log1p', 'multi_celltypes_coarse_colors', 'pca', 'pca_harmony', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.2_colors', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.3_colors', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.4_colors', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.5_colors', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_0.8_colors', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.0_colors', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_1.5_colors', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_2.0_colors', 'pca_harmony_leiden_3.0', 'pca_harmony_leiden_3.0_colors', 'tissue_block_id_colors', 'umap', 'leiden_1.2', 'leiden_1.5', 'leiden_2', 'leiden_1.2_colors', 'leiden_1.5_colors', 'leiden_2_colors'
    obsm: 'X_pca', 'X_pca_harmony', 'X_umap_pca_harmony', 'spatial'
    varm: 'PCs'
    obsp: 'pca_harmony_connectivities', 'pca_harmony_distances'
```

Example 21 (unknown):
```unknown
resolution_sel = 1
```

Example 22 (python):
```python
sc.tl.dendrogram(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.pl.dotplot(adata,
              var_names=markers,
              groupby=f'leiden_{str(resolution_sel)}',
              standard_scale='var',
              dendrogram=True
             )
```

Example 23 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: astrocytes, oligodendrocytes, microglia, etc.
```

Example 24 (python):
```python
print(adata.X.data[:5])
adata.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata, dendrogram=True, standard_scale='var', return_fig=True)
dp.add_totals().show()
```

Example 25 (unknown):
```unknown
[3.036935 3.036935 3.036935 3.705799 3.036935]
```

Example 26 (python):
```python
sc.pl.violin(adata,keys=['total_counts','cell_area'],groupby=f'leiden_{str(resolution_sel)}')
```

Example 27 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=f'leiden_{str(resolution_sel)}',legend_loc='on data',
                )
```

Example 28 (python):
```python
# annotate
adata.obs['tmp_annotation'] = adata.obs[f'leiden_{str(resolution_sel)}'].copy()
adata.obs.replace({'tmp_annotation':{
    '0':'Glia',
    '1':'unassigned',
    '2':'Glia',
    '3':'Glia',
    '4':'Glia',
    '5':'Glia',
    '6':'Glia',
    '7':'Glia',
}},inplace=True)
adata.obs['tmp_annotation'].value_counts()
```

Example 29 (python):
```python
/tmp/ipykernel_4110952/2206290592.py:3: FutureWarning: The behavior of Series.replace (and DataFrame.replace) with CategoricalDtype is deprecated. In a future version, replace will only be used for cases that preserve the categories. To change the categories, use ser.cat.rename_categories instead.
  adata.obs.replace({'tmp_annotation':{
```

Example 30 (unknown):
```unknown
tmp_annotation
Glia          5708
unassigned    1268
Name: count, dtype: int64
```

Example 31 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color='tmp_annotation'
                )
```

Example 32 (python):
```python
adata_sub = adata[adata.obs['tmp_annotation'].isin(['Glia'])]
```

Example 33 (python):
```python
# drop leiden clustering results
resolutions_list = [0.2,0.4,0.6,1, 1.2,1.5,2]
for resolution in resolutions_list:
    adata_sub.obs.drop(f'leiden_{str(resolution)}',axis=1,inplace=True)
adata_sub
```

Example 34 (unknown):
```unknown
AnnData object with n_obs × n_vars = 5708 × 4699
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_3.0', 'multi_celltypes_coarse', 'tmp_annotation'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1', 'n_cells', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'dendrogram_pca_harmony_leiden_1.0', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1_colors', 'log1p', 'multi_celltypes_coarse_colors', 'pca', 'pca_harmony', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.2_colors', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.3_colors', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.4_colors', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.5_colors', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_0.8_colors', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.0_colors', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_1.5_colors', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_2.0_colors', 'pca_harmony_leiden_3.0', 'pca_harmony_leiden_3.0_colors', 'tissue_block_id_colors', 'umap', 'leiden_1.2', 'leiden_1.5', 'leiden_2', 'leiden_1.2_colors', 'leiden_1.5_colors', 'leiden_2_colors', 'dendrogram_leiden_1', 'rank_genes_groups', 'tmp_annotation_colors'
    obsm: 'X_pca', 'X_pca_harmony', 'X_umap_pca_harmony', 'spatial'
    varm: 'PCs'
    obsp: 'pca_harmony_connectivities', 'pca_harmony_distances'
```

Example 35 (python):
```python
## Remove mid-grain markers from the feature
# read in markers
midgrain_markers = pd.read_csv('/home/kk837/rfs/rfs-teichlab-nfs-iCNyzSAaucw/kk837/notebooks/Foetal/VisiumHD/re-seq_mpp0p3/sc_midgrain_markers.csv',
                               index_col=0)
# remove the markers of celltype of interest
midgrain_markers = midgrain_markers[midgrain_markers['group']!=celltype]
# remove
mask = [x not in list(midgrain_markers['names']) for x in adata_sub.var_names]
bdata = adata_sub[:,mask]
bdata
```

Example 36 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 5708 × 4233
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_3.0', 'multi_celltypes_coarse', 'tmp_annotation'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1', 'n_cells', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'dendrogram_pca_harmony_leiden_1.0', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1_colors', 'log1p', 'multi_celltypes_coarse_colors', 'pca', 'pca_harmony', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.2_colors', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.3_colors', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.4_colors', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.5_colors', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_0.8_colors', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.0_colors', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_1.5_colors', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_2.0_colors', 'pca_harmony_leiden_3.0', 'pca_harmony_leiden_3.0_colors', 'tissue_block_id_colors', 'umap', 'leiden_1.2', 'leiden_1.5', 'leiden_2', 'leiden_1.2_colors', 'leiden_1.5_colors', 'leiden_2_colors', 'dendrogram_leiden_1', 'rank_genes_groups', 'tmp_annotation_colors'
    obsm: 'X_pca', 'X_pca_harmony', 'X_umap_pca_harmony', 'spatial'
    varm: 'PCs'
    obsp: 'pca_harmony_connectivities', 'pca_harmony_distances'
```

Example 37 (python):
```python
'''
# remove cell cycle genes
cell_cycle_genes = ['MCM5','PCNA','TYMS','FEN1','MCM2','MCM4','RRM1','UNG','GINS2','MCM6','CDCA7','DTL','PRIM1','UHRF1','MLF1IP','HELLS','RFC2','RPA2','NASP',
                'RAD51AP1','GMNN','WDR76','SLBP','CCNE2','UBR7','POLD3','MSH2','ATAD2','RAD51','RRM2','CDC45','CDC6','EXO1','TIPIN','DSCC1','BLM','CASP8AP2',
                'USP1','CLSPN','POLA1','CHAF1B','BRIP1','E2F8','HMGB2','CDK1','NUSAP1','UBE2C','BIRC5','TPX2','TOP2A','NDC80','CKS2','NUF2','CKS1B','MKI67',
                'TMPO','CENPF','TACC3','FAM64A','SMC4','CCNB2','CKAP2L','CKAP2','AURKB','BUB1','KIF11','ANP32E','TUBB4B','GTSE1','KIF20B','HJURP','CDCA3','HN1',
                'CDC20','TTK','CDC25C','KIF2C','RANGAP1','NCAPD2','DLGAP5','CDCA2','CDCA8','ECT2','KIF23','HMMR','AURKA','PSRC1','ANLN','LBR','CKAP5','CENPE','CTCF',
                'NEK2','G2E3','GAS2L3','CBX5','CENPA']
mask = [x not in cell_cycle_genes for x in bdata.var_names]
bdata = bdata[:,mask]
bdata.shape
'''
```

Example 38 (python):
```python
"\n# remove cell cycle genes\ncell_cycle_genes = ['MCM5','PCNA','TYMS','FEN1','MCM2','MCM4','RRM1','UNG','GINS2','MCM6','CDCA7','DTL','PRIM1','UHRF1','MLF1IP','HELLS','RFC2','RPA2','NASP',\n                'RAD51AP1','GMNN','WDR76','SLBP','CCNE2','UBR7','POLD3','MSH2','ATAD2','RAD51','RRM2','CDC45','CDC6','EXO1','TIPIN','DSCC1','BLM','CASP8AP2',\n                'USP1','CLSPN','POLA1','CHAF1B','BRIP1','E2F8','HMGB2','CDK1','NUSAP1','UBE2C','BIRC5','TPX2','TOP2A','NDC80','CKS2','NUF2','CKS1B','MKI67',\n                'TMPO','CENPF','TACC3','FAM64A','SMC4','CCNB2','CKAP2L','CKAP2','AURKB','BUB1','KIF11','ANP32E','TUBB4B','GTSE1','KIF20B','HJURP','CDCA3','HN1',\n                'CDC20','TTK','CDC25C','KIF2C','RANGAP1','NCAPD2','DLGAP5','CDCA2','CDCA8','ECT2','KIF23','HMMR','AURKA','PSRC1','ANLN','LBR','CKAP5','CENPE','CTCF',\n                'NEK2','G2E3','GAS2L3','CBX5','CENPA']\nmask = [x not in cell_cycle_genes for x in bdata.var_names]\nbdata = bdata[:,mask]\nbdata.shape\n"
```

Example 39 (python):
```python
# HVGs, using the object which doesn't have midgrain markers
sc.pp.highly_variable_genes(bdata, flavor='seurat', min_mean=0.0125, max_mean=3, min_disp=0.5)
print(f'number of hvgs: {sum(bdata.var.highly_variable)}')
# PCA using the degected HVGs
sc.tl.pca(bdata, use_highly_variable=True)
adata_sub.obsm['X_pca'] = bdata.obsm['X_pca'].copy()
# harmony integration
sce.pp.harmony_integrate(adata_sub, 'tissue_block_id', basis='X_pca', adjusted_basis='X_pca_harmony')
# create neighbour graph
latent_space = 'pca_harmony'
sc.pp.neighbors(adata_sub,n_neighbors=15,use_rep=f"X_{latent_space}",key_added=latent_space)
# visualisation with umap
sc.tl.umap(adata_sub,min_dist=0.5,neighbors_key=latent_space)
adata_sub.obsm[f"X_umap_{latent_space}"]=adata_sub.obsm["X_umap"]
del adata_sub.obsm["X_umap"]
```

Example 40 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_highly_variable_genes.py:698: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  adata.uns["hvg"] = {"flavor": flavor}
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_pca/__init__.py:438: FutureWarning: Argument `use_highly_variable` is deprecated, consider using the mask argument. Use_highly_variable=True can be called through mask_var="highly_variable". Use_highly_variable=False can be called through mask_var=None
  warn(msg, FutureWarning)
```

Example 41 (unknown):
```unknown
number of hvgs: 738
```

Example 42 (python):
```python
2025-06-12 09:34:41,633 - harmonypy - INFO - Computing initial centroids with sklearn.KMeans...
2025-06-12 09:34:43,290 - harmonypy - INFO - sklearn.KMeans initialization complete.
2025-06-12 09:34:43,307 - harmonypy - INFO - Iteration 1 of 10
2025-06-12 09:34:43,925 - harmonypy - INFO - Iteration 2 of 10
2025-06-12 09:34:44,533 - harmonypy - INFO - Converged after 2 iterations
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm
```

Example 43 (python):
```python
%%time
# reclustering
# leiden clustering
resolutions_list = [0.2,0.4,0.6,1, 1.2,1.5,2]
for resolution in resolutions_list:
    sc.tl.leiden(adata_sub,neighbors_key=latent_space,
                 resolution=resolution,key_added=f'leiden_{str(resolution)}',n_iterations=2)
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                wspace=0.3)
```

Example 44 (unknown):
```unknown
CPU times: user 7.57 s, sys: 120 ms, total: 7.69 s
Wall time: 7.71 s
```

Example 45 (python):
```python
markers = {
    "astrocytes": ["GFAP", "S100B", "ALDH1L1"],
    "oligodendrocytes": ["OLIG2", "MBP", "PDGFRA"],
    "microglia": ["AIF1"],  # IBA1 protein is encoded by the AIF1 gene
    "schwann_cells": ["SOX10", "MPZ", "S100B"],
    "schwann_cell_precursors": [
                                "PLS3",'SLITRK6','IL1RAPL2'
                               ],
    "ependymal cells":[
                    "FOXJ1",  # Forkhead box J1
                    "S100B",  # S100 Calcium Binding Protein B
                    "VIM",    # Vimentin
                    "PROM1",  # Prominin-1 (CD133)
                    "CDH1",   # Cadherin 1 (E-cadherin)
                    "EPCAM"   # Epithelial Cell Adhesion Molecule
    ],
    'Cycling':['MKI67','TOP2A'],
    'Others':['NGF'],
    'Neuronal':['NEFL','RBFOX3']
}
genes = []
for g in markers.values():
    genes = genes + [x for x in g if (x in adata.var_names)&(x not in genes)]
for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in adata.var_names]

sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color=genes,
               ncols=5, wspace=0.2,cmap='RdPu',vmax='p100')
```

Example 46 (unknown):
```unknown
sub_resolution_sel = 0.4
```

Example 47 (python):
```python
sc.tl.dendrogram(adata_sub,groupby=f'leiden_{str(sub_resolution_sel)}')
sc.pl.dotplot(adata_sub,
              var_names=markers,
              groupby=f'leiden_{str(sub_resolution_sel)}',
              standard_scale='var',
              dendrogram=True
             )
```

Example 48 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: astrocytes, oligodendrocytes, microglia, etc.
```

Example 49 (python):
```python
# DEG analysis
print(adata_sub.X.data[:5])
adata_sub.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata_sub,groupby=f'leiden_{str(sub_resolution_sel)}')
sc.tl.dendrogram(adata_sub, groupby=f'leiden_{str(sub_resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata_sub, dendrogram=True, standard_scale='var', return_fig=True)
dp.add_totals().show()
```

Example 50 (unknown):
```unknown
[2.439362  1.8298357 2.439362  1.8298357 2.439362 ]
```

Example 51 (python):
```python
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color=f'leiden_{str(sub_resolution_sel)}',legend_loc='on data',
                wspace=0.3)
```

Example 52 (python):
```python
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color=['celltypist_midmod2fine', 'conf_score_midmod2fine'],
                wspace=0.7)
```

Example 53 (python):
```python
# annotate
adata_sub.obs['fine_grain_manual'] = adata_sub.obs[f'leiden_{str(sub_resolution_sel)}'].copy()
adata_sub.obs.replace({'fine_grain_manual':{
    '0':'SchwannCellPrecursors',
    '1':'SchwannCells',
    '2':'SchwannCellsCycling',
    '3':'SchwannCells_DoubletsWithNeurons',
    '4':'?EpendymalCells',
}},inplace=True)
adata_sub.obs['fine_grain_manual'].value_counts()
```

Example 54 (python):
```python
/tmp/ipykernel_4110952/1316203814.py:3: FutureWarning: The behavior of Series.replace (and DataFrame.replace) with CategoricalDtype is deprecated. In a future version, replace will only be used for cases that preserve the categories. To change the categories, use ser.cat.rename_categories instead.
  adata_sub.obs.replace({'fine_grain_manual':{
```

Example 55 (unknown):
```unknown
fine_grain_manual
SchwannCellPrecursors               2202
SchwannCells                        2036
SchwannCellsCycling                  871
SchwannCells_DoubletsWithNeurons     370
?EpendymalCells                      229
Name: count, dtype: int64
```

Example 56 (python):
```python
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color='fine_grain_manual',
                wspace=0.3)
```

Example 57 (python):
```python
sc.tl.dendrogram(adata_sub,groupby='fine_grain_manual')
sc.pl.dotplot(adata_sub,
              var_names=markers,
              groupby='fine_grain_manual',
              standard_scale='var',
              dendrogram=True
             )
```

Example 58 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: SchwannCellPrecursors, SchwannCells, SchwannCellsCycling, etc.
var_group_labels: astrocytes, oligodendrocytes, microglia, etc.
```

Example 59 (python):
```python
adata.obs['fine_grain_manual'] = adata.obs['tmp_annotation'].astype('str').copy()
adata.obs.loc[adata_sub.obs_names,'fine_grain_manual'] = adata_sub.obs['fine_grain_manual']
```

Example 60 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color='fine_grain_manual'
                )
```

Example 61 (python):
```python
for tissue_id in ['C194-HEA-0-FFPE-1', 'Hst45-HEA-0-FFPE-1']:
    slide = adata[adata.obs['tissue_block_id']==tissue_id]
    sc.pl.spatial(slide,color=['fine_grain_manual'],# groups='SinoatrialNodePacemakerCells',
                  spot_size=60,vmax='p100')
```

Example 62 (python):
```python
/tmp/ipykernel_4110952/4023824168.py:3: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],# groups='SinoatrialNodePacemakerCells',
```

Example 63 (python):
```python
/tmp/ipykernel_4110952/4023824168.py:3: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],# groups='SinoatrialNodePacemakerCells',
```

Example 64 (python):
```python
for ct in set(adata.obs['fine_grain_manual']):
    for tissue_id in ['C194-HEA-0-FFPE-1', 'Hst45-HEA-0-FFPE-1']:
        slide = adata[adata.obs['tissue_block_id']==tissue_id]
        sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
                      spot_size=80,vmax='p100')
```

Example 65 (python):
```python
/tmp/ipykernel_4110952/1166615315.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 66 (python):
```python
/tmp/ipykernel_4110952/1166615315.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 67 (python):
```python
/tmp/ipykernel_4110952/1166615315.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 68 (python):
```python
/tmp/ipykernel_4110952/1166615315.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 69 (python):
```python
/tmp/ipykernel_4110952/1166615315.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 70 (python):
```python
/tmp/ipykernel_4110952/1166615315.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 71 (python):
```python
/tmp/ipykernel_4110952/1166615315.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 72 (python):
```python
/tmp/ipykernel_4110952/1166615315.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 73 (python):
```python
/tmp/ipykernel_4110952/1166615315.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 74 (python):
```python
/tmp/ipykernel_4110952/1166615315.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 75 (python):
```python
/tmp/ipykernel_4110952/1166615315.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 76 (python):
```python
/tmp/ipykernel_4110952/1166615315.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 77 (python):
```python
# save
adata.write(f'{adata_dir}/{sample_id}_5K_{celltype}_lognorm.h5ad')
print(f'{adata_dir}/{sample_id}_5K_{celltype}_lognorm.h5ad')
```

Example 78 (unknown):
```unknown
/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium/subsets/C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat_5K_Glia_lognorm.h5ad
```

---

## Import libraries¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig4_Cardiomyocytes/AtrialCardiomyocytes/CellOracle/CellOracle_no1-1_construct-GRN.html

**Contents:**
- Import libraries¶
- Load gene-expression data¶
- Make oracle object¶
- Add scenicplus base GRN¶
- KNN imputation¶
- GRN calculation¶
- Network preprocessing for perturbation simulation¶
- Save the oracle object¶
- Network preprocessing¶
- Network analysis; Network score for each gene¶
- Find parameters for n_grid and min_mass for perturbation simulation¶
- Make predictive modelos for simulation¶
- In silico TF perturbation analysis¶
- Visualisation¶
- Find parameters for n_grid and min_mass¶
- Plot vector field¶
- Compare simulation vector with development vectors¶
- Focus on a single development lineage to interpret the results in detail¶

ref: https://morris-lab.github.io/CellOracle.documentation/notebooks/04_Network_analysis/Network_analysis_with_Paul_etal_2015_data.html

CellOracle uses the same strategy as velocyto for visualizing cell transitions. This process requires KNN imputation in advance.

For the KNN imputation, we first need to calculate and select PCs.

Estimate the optimal number of nearest neighbors for KNN imputation.

59522 rows × 6 columns

from below ref: https://morris-lab.github.io/CellOracle.documentation/notebooks/05_simulation/Gata1_KO_simulation_with_Paul_etal_2015_data.html

Here, we will need to fit the ridge regression models again. This process will take less time than the GRN inference in the previous notebook, because we are using the filtered GRN models.

The next step calculates the probability of cell state transitions based on the simulation data. You can use the transition probabilities between cells to predict how cells will change after a perturbation.

This transition probability will be used later.

Caution: It is very important to find the optimal scale parameter.

If the vectors are not visible, you can try a smaller scale parameter to magnify the vector length. However, if you see large vectors in the randomized results (right panel), it means that the scale parameter is too small.

Again, we need to adjust the scale parameter. Please seek to find the optimal scale parameter that provides good visualization.

If you don’t see any vector, you can try the smaller scale parameter to magnify the vector length. However, if you see large vectors in the randomized results (right panel), it means the scale parameter is too small.

Here, we will calculate the development vector field using pseudotime gradient as follows.

Note: Other methods may be used to represent a continuous scRNA-seq trajectory flow. For example, RNA velocity analysis is a good way to estimate the direction of cell differentiation. Choose the method that best suits your data.

In the analysis below, we will use pseudotime data. Pseudotime data is included in the demo data. If you are analyzing your own scRNA-seq data, please calculate pseudotime before starting this analysis.

Next, we convert the pseudotime data into grid points. For this calculation we can chose one of two methods.

knn: K-Nearesr Neighbor regression. You will need to set number of neighbors. Please adjust n_knn for best results.This will depend on the population size and density of your scRNA-seq data.

gradient.transfer_data_into_grid(args={"method": "knn", "n_knn":50})

polynomial: Polynomial regression using x-axis and y-axis of dimensional reduction space.

In general, this method will be more robust. Please use this method if knn method does not work. n_poly is the number of degrees for the polynomial regression model. Please try to find appropriaten_poly searching for best results.

gradient.transfer_data_into_grid(args={"method": "polynomial", "n_poly":3})

Calculate the 2D vector map to represents the pseudotime gradient. After the gradient calculation, the length of the vector will be normalized automatically.

Please adjust the scale parameter to adjust vector length visualization.

The inner product represents the similarity between two vectors.

Using the inner product, we compare the 2D vector field of perturbation simulation and development flow.

The inner product will be a positive value when two vectors are pointing in the same direction.

Conversely, the inner product will be a negative value when two vectors are pointing in the opposing directions.

The length of the vectors also affects the magnitude of inner product value.

In summary, we quantitatively compare the directionality and size of vectors between perturbation simulation and natural differentiation using inner product, and we define the score as perturbation score (PS).

A negative PS means that the TF perturbation would block differentiation.

A positive PS means that the TF perturbation would promote differentiation.

Here, need to adjust the vm parameter for the PS score color visualization. Please seek to find the optimal vm parameter that provides good visualization.

If you don’t see any color in the left panel below, you can try the smaller vm parameter to magnify the scale of vm visualization. However, if you see colors in the randomized results (right panel), it means the vm parameter is too small.

**Examples:**

Example 1 (python):
```python
import os
import sys

import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import scanpy as sc
import seaborn as sns
```

Example 2 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 3 (python):
```python
sc.settings.set_figure_params(dpi=120)
```

Example 4 (python):
```python
# read in adata
# whole data
adata = sc.read_h5ad('/lustre/scratch126/cellgen/team205/heart/objects/fetal/RNA/aCMs_WholeSample_paired_with_time.h5ad')
adata
```

Example 5 (unknown):
```unknown
AnnData object with n_obs × n_vars = 5664 × 544641
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', 'age_days', 'age_days_smoothed_once', 'age_days_normalized', 'age_days_normalized_smoothed_once', 'age_days_smoothed_iteratively', 'age_days_normalized_smoothed_iteratively', 't', 'seg', 'edge', 't_sd', 'milestones'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id'
    uns: 'donor_colors', 'fine_grain_colors', 'graph', 'milestones_colors', 'neighbors', 'ppt', 'pseudotime_list', 'seg_colors', 'umap', 'week_colors'
    obsm: 'X_R', 'X_multiVI', 'X_umap'
    obsp: 'connectivities', 'distances'
```

Example 6 (python):
```python
print(adata.X.data[:5])
adata.var['modality'].value_counts()
```

Example 7 (unknown):
```unknown
[2. 1. 1. 1. 1.]
```

Example 8 (unknown):
```unknown
Peaks              508040
Gene Expression     36601
Name: modality, dtype: int64
```

Example 9 (python):
```python
# subset gene expression data
adata = adata[:,adata.var['modality']=='Gene Expression']

# keep raw cont data before log normalisation
adata.raw = adata
adata.layers["raw_count"] = adata.raw.X.copy()

# filter and log-normalise
sc.pp.filter_genes(adata, min_cells=3)
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

print(adata.X.data[:10])
adata
```

Example 10 (unknown):
```unknown
[1.559231  1.0569503 1.0569503 1.0569503 1.0569503 1.0569503 2.7738788
 1.559231  1.0569503 1.0569503]
```

Example 11 (unknown):
```unknown
AnnData object with n_obs × n_vars = 5664 × 23953
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', 'age_days', 'age_days_smoothed_once', 'age_days_normalized', 'age_days_normalized_smoothed_once', 'age_days_smoothed_iteratively', 'age_days_normalized_smoothed_iteratively', 't', 'seg', 'edge', 't_sd', 'milestones'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id', 'n_cells'
    uns: 'donor_colors', 'fine_grain_colors', 'graph', 'milestones_colors', 'neighbors', 'ppt', 'pseudotime_list', 'seg_colors', 'umap', 'week_colors', 'log1p'
    obsm: 'X_R', 'X_multiVI', 'X_umap'
    layers: 'raw_count'
    obsp: 'connectivities', 'distances'
```

Example 12 (python):
```python
# select genes which are targeted or the regulators in the regulons
# 
ereg_signatures = utils.read_pkl("/nfs/team205/heart/pyscenic/plus/eRegulon_signatures/aCM_n570_Gene_eRegulon_signatures.pkl")
genes_sel = []
for key,value in ereg_signatures.items():
    genes_sel = genes_sel + value
    genes_sel = genes_sel + [key.split('_')[0]]
genes_sel = list(set(genes_sel))

# hvg and targeted
# hvg_targeted = list(set(adata.var_names[adata.var.highly_variable]).intersection(genes_targeted))
# len(hvg_targeted)
len(genes_sel)
```

Example 13 (python):
```python
# subset genes
# adata = adata[:,hvg_targeted]
adata = adata[:,genes_sel]
adata
```

Example 14 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 5664 × 10870
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', 'age_days', 'age_days_smoothed_once', 'age_days_normalized', 'age_days_normalized_smoothed_once', 'age_days_smoothed_iteratively', 'age_days_normalized_smoothed_iteratively', 't', 'seg', 'edge', 't_sd', 'milestones'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id', 'n_cells'
    uns: 'donor_colors', 'fine_grain_colors', 'graph', 'milestones_colors', 'neighbors', 'ppt', 'pseudotime_list', 'seg_colors', 'umap', 'week_colors', 'log1p'
    obsm: 'X_R', 'X_multiVI', 'X_umap'
    layers: 'raw_count'
    obsp: 'connectivities', 'distances'
```

Example 15 (python):
```python
print(f"Cell number is :{adata.shape[0]}")
print(f"Gene number is :{adata.shape[1]}")
```

Example 16 (unknown):
```unknown
Cell number is :5664
Gene number is :10870
```

Example 17 (python):
```python
# add t_binned category
n_bin_time = 3
adata.obs['t_binned'] = pd.cut(adata.obs['t'],
                               bins=n_bin_time,
                               labels=[f't{str(x+1)}' for x in range(n_bin_time)])
print(set(adata.obs['t_binned']))

# combine fine_grain and t_binned category
adata.obs['fine_grain_time-binned'] = adata.obs['fine_grain'].astype('str')+'_'+adata.obs['t_binned'].astype('str')
```

Example 18 (python):
```python
/tmp/ipykernel_3282310/3161589083.py:3: ImplicitModificationWarning: Trying to modify attribute `.obs` of view, initializing view as actual.
  adata.obs['t_binned'] = pd.cut(adata.obs['t'],
```

Example 19 (unknown):
```unknown
{'t1', 't2', 't3'}
```

Example 20 (python):
```python
sc.pl.umap(adata,color=['fine_grain','t','fine_grain_time-binned'], size=5, wspace=0.3, ncols=3)
```

Example 21 (python):
```python
import celloracle as co
co.__version__
```

Example 22 (unknown):
```unknown
2025-05-05 21:07:46,185 - INFO - BioProspector no longer found.
2025-05-05 21:07:46,186 - INFO - HMS no longer found.
2025-05-05 21:07:46,188 - INFO - MDmodule no longer found.
2025-05-05 21:07:46,189 - INFO - Posmo no longer found.
2025-05-05 21:07:46,194 - INFO - Using included version of BioProspector.
2025-05-05 21:07:46,198 - INFO - Using included version of HMS.
2025-05-05 21:07:46,202 - INFO - Using included version of MDmodule.
2025-05-05 21:07:46,205 - INFO - Using included version of Posmo.
2025-05-05 21:07:46,211 - INFO - Configuration file: /nfs/users/nfs_k/kk18/.config/gimmemotifs/gimmemotifs.cfg
```

Example 23 (unknown):
```unknown
# Instantiate Oracle object
oracle = co.Oracle()
```

Example 24 (python):
```python
# Check data in anndata
print("Metadata columns :", list(adata.obs.columns))
print("Dimensional reduction: ", list(adata.obsm.keys()))
```

Example 25 (python):
```python
Metadata columns : ['sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', 'age_days', 'age_days_smoothed_once', 'age_days_normalized', 'age_days_normalized_smoothed_once', 'age_days_smoothed_iteratively', 'age_days_normalized_smoothed_iteratively', 't', 'seg', 'edge', 't_sd', 'milestones', 't_binned', 'fine_grain_time-binned']
Dimensional reduction:  ['X_R', 'X_multiVI', 'X_umap']
```

Example 26 (python):
```python
# In this notebook, we use the unscaled mRNA count for the nput of Oracle object.
adata.X = adata.layers["raw_count"].copy()

# Instantiate Oracle object.
oracle.import_anndata_as_raw_count(adata=adata,
                                   cluster_column_name="fine_grain_time-binned",
                                   embedding_name="X_umap")
```

Example 27 (python):
```python
10870 genes were found in the adata. Note that Celloracle is intended to use around 1000-3000 genes, so the behavior with this number of genes may differ from what is expected.
WARNING: adata.X seems to be already log-transformed.
```

Example 28 (unknown):
```unknown
# read in scenicplus regulons
regulons = utils.read_pkl('/nfs/team205/heart/pyscenic/plus/baseGRN_forCO/AtrialCM_baseGRN_act.pkl')
TG_to_TF_dictionary = co.utility.inverse_dictionary(regulons)
```

Example 29 (unknown):
```unknown
0%|          | 0/10836 [00:00<?, ?it/s]
```

Example 30 (unknown):
```unknown
# add
oracle.import_TF_data(TFdict=TG_to_TF_dictionary)
```

Example 31 (unknown):
```unknown
TG_to_TF_dictionary['SGSM2']
```

Example 32 (unknown):
```unknown
['KLF13', 'KLF15', 'MBD1', 'RELA', 'SREBF1', 'VEZF1', 'ZNF28', 'ZNF644']
```

Example 33 (python):
```python
%%time
%matplotlib inline
# Perform PCA
oracle.perform_PCA()

# Select important PCs
plt.plot(np.cumsum(oracle.pca.explained_variance_ratio_)[:100])
n_comps = np.where(np.diff(np.diff(np.cumsum(oracle.pca.explained_variance_ratio_))>0.002))[0][0]
plt.axvline(n_comps, c="k")
plt.show()
print(n_comps)
n_comps = min(n_comps, 50)
```

Example 34 (unknown):
```unknown
10
CPU times: user 1min 32s, sys: 994 ms, total: 1min 33s
Wall time: 1min 36s
```

Example 35 (python):
```python
oracle.pcs.shape
```

Example 36 (unknown):
```unknown
(5664, 5664)
```

Example 37 (python):
```python
n_cell = oracle.adata.shape[0]
print(f"cell number is :{n_cell}")
```

Example 38 (unknown):
```unknown
cell number is :5664
```

Example 39 (python):
```python
k = int(0.025*n_cell)
print(f"Auto-selected k is :{k}")
```

Example 40 (unknown):
```unknown
Auto-selected k is :141
```

Example 41 (unknown):
```unknown
%%time
oracle.knn_imputation(n_pca_dims=n_comps, k=k, balanced=True, b_sight=k*8,
                      b_maxl=k*4, n_jobs=4)
```

Example 42 (unknown):
```unknown
CPU times: user 8.42 s, sys: 1.53 s, total: 9.94 s
Wall time: 10.2 s
```

Example 43 (unknown):
```unknown
# Save oracle object.
oracle.to_hdf5("/nfs/team205/heart/anndata_objects/Foetal/celloracle/foetal_AtrialCM.celloracle.oracle")
```

Example 44 (unknown):
```unknown
# Load
oracle = co.load_hdf5("/nfs/team205/heart/anndata_objects/Foetal/celloracle/foetal_AtrialCM.celloracle.oracle")
```

Example 45 (python):
```python
%matplotlib inline
# Check clustering data
sc.pl.umap(oracle.adata, color='fine_grain_time-binned')
```

Example 46 (unknown):
```unknown
%%time
# Calculate GRN for each population in "louvain_annot" clustering unit.
# This step may take some time.(~30 minutes)
links = oracle.get_links(cluster_name_for_GRN_unit='fine_grain_time-binned', alpha=10,
                         verbose_level=10)
```

Example 47 (unknown):
```unknown
0%|          | 0/6 [00:00<?, ?it/s]
```

Example 48 (unknown):
```unknown
Inferring GRN for AtrialCardiomyocytesLeft_t1...
```

Example 49 (unknown):
```unknown
0%|          | 0/10836 [00:00<?, ?it/s]
```

Example 50 (unknown):
```unknown
Inferring GRN for AtrialCardiomyocytesLeft_t2...
```

Example 51 (unknown):
```unknown
0%|          | 0/10836 [00:00<?, ?it/s]
```

Example 52 (unknown):
```unknown
Inferring GRN for AtrialCardiomyocytesLeft_t3...
```

Example 53 (unknown):
```unknown
0%|          | 0/10836 [00:00<?, ?it/s]
```

Example 54 (unknown):
```unknown
Inferring GRN for AtrialCardiomyocytesRight_t1...
```

Example 55 (unknown):
```unknown
0%|          | 0/10836 [00:00<?, ?it/s]
```

Example 56 (unknown):
```unknown
Inferring GRN for AtrialCardiomyocytesRight_t2...
```

Example 57 (unknown):
```unknown
0%|          | 0/10836 [00:00<?, ?it/s]
```

Example 58 (unknown):
```unknown
Inferring GRN for AtrialCardiomyocytesRight_t3...
```

Example 59 (unknown):
```unknown
0%|          | 0/10836 [00:00<?, ?it/s]
```

Example 60 (unknown):
```unknown
CPU times: user 54min 47s, sys: 12.5 s, total: 55min
Wall time: 56min 58s
```

Example 61 (unknown):
```unknown
links.links_dict.keys()
```

Example 62 (unknown):
```unknown
dict_keys(['AtrialCardiomyocytesLeft_t1', 'AtrialCardiomyocytesLeft_t2', 'AtrialCardiomyocytesLeft_t3', 'AtrialCardiomyocytesRight_t1', 'AtrialCardiomyocytesRight_t2', 'AtrialCardiomyocytesRight_t3'])
```

Example 63 (unknown):
```unknown
links.links_dict['AtrialCardiomyocytesRight_t3']
```

Example 64 (unknown):
```unknown
# Save Links object.
links.to_hdf5(file_path="/nfs/team205/heart/anndata_objects/Foetal/celloracle/foetal_AtrialCM_links.celloracle.links")
```

Example 65 (unknown):
```unknown
# load the saved links
links = co.load_hdf5("/nfs/team205/heart/anndata_objects/Foetal/celloracle/foetal_AtrialCM_links.celloracle.links")
```

Example 66 (unknown):
```unknown
%%time

links.filter_links(p=0.001, weight="coef_abs", threshold_number=10000)

# Calculate network scores.
links.get_network_score()
display(links.merged_score.head())

# Make predictive modelos for simulation
# Here, we will need to fit the ridge regression models again.
# This process will take less time than the GRN inference in the previous notebook, because we are using the filtered GRN models.
oracle.get_cluster_specific_TFdict_from_Links(links_object=links)
oracle.fit_GRN_for_simulation(alpha=10,
                              use_cluster_specific_TFdict=True)
```

Example 67 (unknown):
```unknown
0%|          | 0/6 [00:00<?, ?it/s]
```

Example 68 (unknown):
```unknown
CPU times: user 3min 12s, sys: 3.54 s, total: 3min 15s
Wall time: 3min 21s
```

Example 69 (unknown):
```unknown
oracle.to_hdf5("/nfs/team205/heart/anndata_objects/Foetal/celloracle/foetal_AtrialCM.celloracle.oracle")
oracle
```

Example 70 (unknown):
```unknown
Oracle object

Meta data
    celloracle version used for instantiation: 0.20.0
    n_cells: 5664
    n_genes: 10870
    cluster_name: fine_grain_time-binned
    dimensional_reduction_name: X_umap
    n_target_genes_in_TFdict: 10836 genes
    n_regulatory_in_TFdict: 263 genes
    n_regulatory_in_both_TFdict_and_scRNA-seq: 263 genes
    n_target_genes_both_TFdict_and_scRNA-seq: 10836 genes
    k_for_knn_imputation: 141
Status
    Gene expression matrix: Ready
    BaseGRN: Ready
    PCA calculation: Done
    Knn imputation: Done
    GRN calculation for simulation: Done
```

Example 71 (unknown):
```unknown
links.filter_links(p=0.001, weight="coef_abs", threshold_number=10000)
```

Example 72 (python):
```python
plt.rcParams["figure.figsize"] = [7, 3]
```

Example 73 (unknown):
```unknown
links.plot_degree_distributions(plot_model=True,
                                               #save=f"{save_folder}/degree_distribution/",
                                               )
```

Example 74 (unknown):
```unknown
AtrialCardiomyocytesLeft_t1
```

Example 75 (unknown):
```unknown
AtrialCardiomyocytesLeft_t2
```

Example 76 (unknown):
```unknown
AtrialCardiomyocytesLeft_t3
```

Example 77 (unknown):
```unknown
AtrialCardiomyocytesRight_t1
```

Example 78 (unknown):
```unknown
AtrialCardiomyocytesRight_t2
```

Example 79 (unknown):
```unknown
AtrialCardiomyocytesRight_t3
```

Example 80 (unknown):
```unknown
# Calculate network scores.
links.get_network_score()
```

Example 81 (unknown):
```unknown
links.merged_score.head()
```

Example 82 (unknown):
```unknown
# Check cluster name
links.cluster
```

Example 83 (unknown):
```unknown
['AtrialCardiomyocytesLeft_t1',
 'AtrialCardiomyocytesLeft_t2',
 'AtrialCardiomyocytesLeft_t3',
 'AtrialCardiomyocytesRight_t1',
 'AtrialCardiomyocytesRight_t2',
 'AtrialCardiomyocytesRight_t3']
```

Example 84 (python):
```python
# Visualize top n-th genes with high scores.
plt.rcParams["figure.figsize"] = [5, 7]
links.plot_scores_as_rank(cluster='AtrialCardiomyocytesRight_t3', n_gene=30, save=None)
```

Example 85 (python):
```python
# Visualize top n-th genes with high scores.
plt.rcParams["figure.figsize"] = [5, 7]
links.plot_scores_as_rank(cluster='AtrialCardiomyocytesLeft_t3', n_gene=30, save=None)
```

Example 86 (python):
```python
# Compare GRN score between two clusters
plt.rcParams["figure.figsize"] = [5, 5]
links.plot_score_comparison_2D(value="betweenness_centrality",
                               cluster1='AtrialCardiomyocytesRight_t3', cluster2='AtrialCardiomyocytesRight_t1',
                               percentile=98,
                               save=None)
```

Example 87 (unknown):
```unknown
links.plot_score_comparison_2D(value="betweenness_centrality",
                               cluster1='AtrialCardiomyocytesLeft_t3', cluster2='AtrialCardiomyocytesLeft_t1',
                               percentile=98,
                               save=None)
```

Example 88 (unknown):
```unknown
links.plot_score_comparison_2D(value="betweenness_centrality",
                               cluster1='AtrialCardiomyocytesLeft_t3', cluster2='AtrialCardiomyocytesRight_t3',
                               percentile=98,
                               save=None)
```

Example 89 (unknown):
```unknown
# set a gene for this trial
goi = "PITX2"
# Enter perturbation conditions to simulate signal propagation after the perturbation.
oracle.simulate_shift(perturb_condition={goi: 0.0}, # this is KO simulation so expression=0
                      n_propagation=3)
```

Example 90 (unknown):
```unknown
%%time
# Get transition probability
oracle.estimate_transition_prob(n_neighbors=200,
                                knn_random=True,
                                sampled_fraction=1)

# Calculate embedding
oracle.calculate_embedding_shift(sigma_corr=0.05)
```

Example 91 (python):
```python
# visualisation
fig, ax = plt.subplots(1, 2,  figsize=[13, 6])

scale = 20
# Show quiver plot
oracle.plot_quiver(scale=scale, ax=ax[0])
ax[0].set_title(f"Simulated cell identity shift vector: {goi} KO")

# Show quiver plot that was calculated with randomized graph.
oracle.plot_quiver_random(scale=scale, ax=ax[1])
ax[1].set_title(f"Randomized simulation vector")

plt.show()
```

Example 92 (unknown):
```unknown
# n_grid = 40 is a good starting value.
n_grid = 40
oracle.calculate_p_mass(smooth=0.8, n_grid=n_grid, n_neighbors=200)
```

Example 93 (unknown):
```unknown
# Search for best min_mass.
oracle.suggest_mass_thresholds(n_suggestion=12)
```

Example 94 (unknown):
```unknown
min_mass = 0.011
oracle.calculate_mass_filter(min_mass=min_mass, plot=True)
```

Example 95 (unknown):
```unknown
%%time
links.filter_links()
oracle.get_cluster_specific_TFdict_from_Links(links_object=links)
oracle.fit_GRN_for_simulation(alpha=10,
                              use_cluster_specific_TFdict=True)
```

Example 96 (python):
```python
# Check gene expression
goi = "TBX20"
sc.pl.draw_graph(oracle.adata, color=[goi, oracle.cluster_column_name],
                 layer="imputed_count", use_raw=False, cmap="viridis")
```

Example 97 (python):
```python
# Plot gene expression in histogram
sc.get.obs_df(oracle.adata, keys=[goi], layer="imputed_count").hist()
plt.show()
```

Example 98 (unknown):
```unknown
# Enter perturbation conditions to simulate signal propagation after the perturbation.
oracle.simulate_shift(perturb_condition={goi: 0.0}, # this is KO simulation so expression=0
                      n_propagation=3)
```

Example 99 (unknown):
```unknown
%%time
# Get transition probability
oracle.estimate_transition_prob(n_neighbors=200,
                                knn_random=True,
                                sampled_fraction=1)

# Calculate embedding
oracle.calculate_embedding_shift(sigma_corr=0.05)
```

Example 100 (python):
```python
fig, ax = plt.subplots(1, 2,  figsize=[13, 6])

scale = 20
# Show quiver plot
oracle.plot_quiver(scale=scale, ax=ax[0])
ax[0].set_title(f"Simulated cell identity shift vector: {goi} KO")

# Show quiver plot that was calculated with randomized graph.
oracle.plot_quiver_random(scale=scale, ax=ax[1])
ax[1].set_title(f"Randomized simulation vector")

plt.show()
```

Example 101 (unknown):
```unknown
# n_grid = 40 is a good starting value.
n_grid = 40
oracle.calculate_p_mass(smooth=0.8, n_grid=n_grid, n_neighbors=200)
```

Example 102 (unknown):
```unknown
# Search for best min_mass.
oracle.suggest_mass_thresholds(n_suggestion=12)
```

Example 103 (unknown):
```unknown
min_mass = 0.011
oracle.calculate_mass_filter(min_mass=min_mass, plot=True)
```

Example 104 (python):
```python
fig, ax = plt.subplots(1, 2,  figsize=[13, 6])

scale_simulation = 0.1
# Show quiver plot
oracle.plot_simulation_flow_on_grid(scale=scale_simulation, ax=ax[0])
ax[0].set_title(f"Simulated cell identity shift vector: {goi} KO")

# Show quiver plot that was calculated with randomized graph.
oracle.plot_simulation_flow_random_on_grid(scale=scale_simulation, ax=ax[1])
ax[1].set_title(f"Randomized simulation vector")

plt.show()
```

Example 105 (python):
```python
# Plot vector field with cell cluster
fig, ax = plt.subplots(figsize=[8, 8])

oracle.plot_cluster_whole(ax=ax, s=10)
oracle.plot_simulation_flow_on_grid(scale=scale_simulation, ax=ax, show_background=False)
```

Example 106 (python):
```python
# Visualize pseudotime
fig, ax = plt.subplots(figsize=[6,6])

sc.pl.embedding(adata=oracle.adata, basis=oracle.embedding_name, ax=ax, cmap="rainbow",
                color=["t"])
```

Example 107 (python):
```python
from celloracle.applications import Gradient_calculator

# Instantiate Gradient calculator object
gradient = Gradient_calculator(oracle_object=oracle, pseudotime_key="t")
```

Example 108 (unknown):
```unknown
gradient.calculate_p_mass(smooth=0.8, n_grid=n_grid, n_neighbors=200)
gradient.calculate_mass_filter(min_mass=min_mass, plot=True)
```

Example 109 (unknown):
```unknown
gradient.transfer_data_into_grid(args={"method": "knn", "n_knn":50})
```

Example 110 (unknown):
```unknown
# Calculate graddient
gradient.calculate_gradient()

# Show results
scale_dev = 40
gradient.visualize_results(scale=scale_dev, s=5)
```

Example 111 (python):
```python
# Visualize results
fig, ax = plt.subplots(figsize=[6, 6])
gradient.plot_dev_flow_on_grid(scale=scale_dev, ax=ax)
```

Example 112 (python):
```python
from celloracle.applications import Oracle_development_module

# Make Oracle_development_module to compare two vector field
dev = Oracle_development_module()

# Load development flow
dev.load_differentiation_reference_data(gradient_object=gradient)

# Load simulation result
dev.load_perturb_simulation_data(oracle_object=oracle)


# Calculate inner produc scores
dev.calculate_inner_product()
dev.calculate_digitized_ip(n_bins=10)
```

Example 113 (python):
```python
# Show perturbation scores
vm = 0.01

fig, ax = plt.subplots(1, 2, figsize=[12, 6])
dev.plot_inner_product_on_grid(vm=vm, s=50, ax=ax[0])
ax[0].set_title(f"PS")

dev.plot_inner_product_random_on_grid(vm=vm, s=50, ax=ax[1])
ax[1].set_title(f"PS calculated with Randomized simulation vector")
plt.show()
```

Example 114 (python):
```python
# Show perturbation scores with perturbation simulation vector field
fig, ax = plt.subplots(figsize=[6, 6])
dev.plot_inner_product_on_grid(vm=vm, s=50, ax=ax)
dev.plot_simulation_flow_on_grid(scale=scale_simulation, show_background=False, ax=ax)
```

Example 115 (unknown):
```unknown
# Let's visualize the results
dev.visualize_development_module_layout_0(s=5,
                                          scale_for_simulation=scale_simulation,
                                          s_grid=50,
                                          scale_for_pseudotime=scale_dev,
                                          vm=vm)
```

Example 116 (unknown):
```unknown
dev.inner_product_df
```

Example 117 (unknown):
```unknown
dev.cell_idx_use
```

Example 118 (python):
```python
# Get cell index list for the cells of interest
clusters = ['VentricularCardiomyocytesLeftCompact_t1',
 'VentricularCardiomyocytesLeftCompact_t2',
 'VentricularCardiomyocytesLeftCompact_t3']
cell_idx = np.where(oracle.adata.obs["fine_grain_time-binned"].isin(clusters))[0]

# Check index
print(cell_idx)
```

Example 119 (unknown):
```unknown
dev = Oracle_development_module()

# Load development flow
dev.load_differentiation_reference_data(gradient_object=gradient)

# Load simulation result
dev.load_perturb_simulation_data(oracle_object=oracle,
                                 cell_idx_use=cell_idx, # Enter cell id list
                                 name="Lineage_Compact" # Name of this cell group. You can enter any name.
                                 )

# Calculation
dev.calculate_inner_product()
dev.calculate_digitized_ip(n_bins=10)
```

Example 120 (unknown):
```unknown
# Let's visualize the results
dev.visualize_development_module_layout_0(s=5,
                                          scale_for_simulation=scale_simulation,
                                          s_grid=50,
                                          scale_for_pseudotime=scale_dev,
                                          vm=vm)
```

Example 121 (unknown):
```unknown
dev.inner_product_df
```

Example 122 (unknown):
```unknown
df = dev.inner_product_df.copy()
df[['score','pseudotime']].corr(method='pearson')
```

Example 123 (unknown):
```unknown
dev.inner_product_df
```

Example 124 (unknown):
```unknown
len(dev.inner_product)
```

---

## Read in data¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/VisiumHD/no3_bin2cell_GEX-segmentation_sigma5.html

**Contents:**
- Read in data¶
- Cell detection using gene expression data¶
- Joint HE and GEX labels¶
- ~Remove too small cells~¶
- Group bins into cells!¶
- Save¶

The H&E segmentation is not guaranteed to be perfect. There may be some regions that have expression data but lack a visible nuclei to seed a cell. Alternately, nuclei may take on unusual shapes and not be detected by the model. Performing segmentation on a representation of total expression per bin may be able to detect some of the missed cells. However, this kind of segmentation only performs well on sparse tissue, and struggles to differentiate individual cells in dense regions. As such, we use it as secondary means of object identification, preferring H&E where able.

The input image is going to be a representation of the total counts per bin, with a Gaussian filter with a sigma of 5 (measured in pixels) applied for a little smoothing.

The segmentation is performed via StarDist's fluorescence model, and identifies cells rather than nuclei, meaning subsequent label expasion is not necessary. Just like with the H&E model, lowering prob_thresh to make the calls less stringent is recommended. Increasing nms_thresh requires more of the putative objects to be overlapping for them to be merged into a single label, which may help a bit in denser regions.

The resulting calls are loaded like the H&E ones were, except the input image was based on array rather than spatial coordinates.

We can visualise the cells identified in the GEX space segmentation in our previously defined region.

Armed with both the H&E and gene expression segmentation results, we can now fill in the gaps in the H&E with GEX calls. We only add GEX cells that do not overlap with any H&E labels.

A few extra GEX-derived cells show up in our test region. Note the presence of .obs["labels_joint_source"] which specifies whether any given label came from the primary or secondary labelling.

At this point, the counts have been destriped and bins have been assigned to cells based on both H&E and GEX segmentation. Time to group the bins into cells!

The object features the sum of gene expression of the constituent bins, and means of the array and spatial coordinates to represent the cell centroids. .obs["labels_joint_source"] is taken as well, providing information on which segmentation any given cell stems from. Let's visualise our test fragment of the tissue, now in cell space!

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import os

import matplotlib.pyplot as plt
import seaborn as sns

import bin2cell as b2c

#create directory for stardist input/output files
os.makedirs("stardist", exist_ok=True)
```

Example 2 (python):
```python
adata = sc.read_h5ad('/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/HEA_FOET14880396_b2c.h5ad')
adata
```

Example 3 (unknown):
```unknown
AnnData object with n_obs × n_vars = 5001851 × 18051
    obs: 'in_tissue', 'array_row', 'array_col', 'n_counts', 'destripe_factor', 'n_counts_adjusted', 'labels_he_0p01', 'labels_he_0p03', 'labels_he_0p05', 'labels_he_0p1', 'labels_he_default', 'labels_he_sel', 'labels_he_expanded_1', 'labels_he_expanded_2', 'labels_he_expanded_3', 'labels_he_expanded_4', 'labels_he_expanded_sel'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells'
    uns: 'bin2cell', 'spatial'
    obsm: 'spatial', 'spatial_cropped_150_buffer'
```

Example 4 (python):
```python
# to re-start gex segmentation
# not required if initial attemp
# adata.obs.drop(columns=['labels_gex_0p002', 'labels_gex_0p01', 'labels_gex_0p05', 'labels_gex_sel'],inplace=True)
```

Example 5 (python):
```python
%%time
mpp = 0.3
b2c.grid_image(adata, "n_counts_adjusted", mpp=mpp, sigma=5, save_path="stardist/gex.tiff")
```

Example 6 (unknown):
```unknown
CPU times: user 34.7 s, sys: 14 s, total: 48.7 s
Wall time: 1min 10s
```

Example 7 (python):
```python
%%time
prob_thresh_dict = {
    '0p005':0.005,
    '0p01':0.01,
     '0p03':0.03,
    '0p05':0.05,
}

for key,prob_thresh in prob_thresh_dict.items():
    print(f'### {key} ###')
    b2c.stardist(image_path="stardist/gex.tiff", 
                 labels_npz_path=f"stardist/gex_prob{key}.npz", 
                 stardist_model="2D_versatile_fluo", 
                 prob_thresh=prob_thresh, 
                 nms_thresh=0.5
                )
    print('')
```

Example 8 (unknown):
```unknown
### 0p005 ###
```

Example 9 (unknown):
```unknown
2024-12-05 17:50:10.809546: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2024-12-05 17:50:11.174332: I external/local_xla/xla/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2024-12-05 17:50:11.723242: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:477] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1733421011.999613  690313 cuda_dnn.cc:8310] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
E0000 00:00:1733421012.075924  690313 cuda_blas.cc:1418] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-12-05 17:50:12.987381: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
```

Example 10 (unknown):
```unknown
Found model '2D_versatile_fluo' for 'StarDist2D'.
```

Example 11 (unknown):
```unknown
2024-12-05 17:51:24.727765: E external/local_xla/xla/stream_executor/cuda/cuda_driver.cc:152] failed call to cuInit: INTERNAL: CUDA error: Failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected
```

Example 12 (python):
```python
Loading network weights from 'weights_best.h5'.
Loading thresholds from 'thresholds.json'.
Using default values: prob_thresh=0.479071, nms_thresh=0.3.
effective: block_size=(4096, 4096), min_overlap=(128, 128), context=(128, 128)
```

Example 13 (unknown):
```unknown
functional.py (238): The structure of `inputs` doesn't match the expected structure.
Expected: ['input']
Received: inputs=Tensor(shape=(1, 256, 256, 1))
  0%|          | 0/36 [00:00<?, ?it/s]functional.py (238): The structure of `inputs` doesn't match the expected structure.
Expected: ['input']
Received: inputs=Tensor(shape=(1, None, None, 1))
100%|██████████| 36/36 [49:28<00:00, 82.46s/it]
```

Example 14 (python):
```python
Found 155283 objects

### 0p01 ###
Found model '2D_versatile_fluo' for 'StarDist2D'.
Loading network weights from 'weights_best.h5'.
Loading thresholds from 'thresholds.json'.
Using default values: prob_thresh=0.479071, nms_thresh=0.3.
effective: block_size=(4096, 4096), min_overlap=(128, 128), context=(128, 128)
```

Example 15 (unknown):
```unknown
100%|██████████| 36/36 [50:34<00:00, 84.30s/it]
```

Example 16 (python):
```python
Found 123972 objects

### 0p03 ###
Found model '2D_versatile_fluo' for 'StarDist2D'.
Loading network weights from 'weights_best.h5'.
Loading thresholds from 'thresholds.json'.
Using default values: prob_thresh=0.479071, nms_thresh=0.3.
effective: block_size=(4096, 4096), min_overlap=(128, 128), context=(128, 128)
```

Example 17 (unknown):
```unknown
100%|██████████| 36/36 [43:22<00:00, 72.29s/it]
```

Example 18 (python):
```python
Found 83831 objects

### 0p05 ###
Found model '2D_versatile_fluo' for 'StarDist2D'.
Loading network weights from 'weights_best.h5'.
Loading thresholds from 'thresholds.json'.
Using default values: prob_thresh=0.479071, nms_thresh=0.3.
effective: block_size=(4096, 4096), min_overlap=(128, 128), context=(128, 128)
```

Example 19 (unknown):
```unknown
100%|██████████| 36/36 [42:00<00:00, 70.02s/it]
```

Example 20 (unknown):
```unknown
Found 69281 objects

CPU times: user 1h 56min 24s, sys: 25min 44s, total: 2h 22min 8s
Wall time: 3h 9min 6s
```

Example 21 (python):
```python
prob_thresh_dict = {
    '0p005':0.005,
    '0p01':0.01,
     '0p03':0.03,
    '0p05':0.05,
}

for key,prob_thresh in prob_thresh_dict.items():
    print(f'### {key} ###')
    b2c.insert_labels(adata, 
                  labels_npz_path=f"stardist/gex_prob{key}.npz", 
                  basis="array", 
                  mpp=mpp, 
                  labels_key=f"labels_gex_{key}"
                 )
```

Example 22 (unknown):
```unknown
### 0p005 ###
### 0p01 ###
### 0p03 ###
### 0p05 ###
```

Example 23 (python):
```python
mask = ((adata.obs['array_row'] >= 1450) & 
        (adata.obs['array_row'] <= 1550) & 
        (adata.obs['array_col'] >= 1400) & 
        (adata.obs['array_col'] <= 1600)
       )
bdata = adata[mask]

for key in prob_thresh_dict.keys():
    #0 means unassigned
    bdata = bdata[bdata.obs[f"labels_gex_{key}"]>0]
    bdata.obs[f"labels_gex_{key}"] = bdata.obs[f"labels_gex_{key}"].astype(str)
    print(len(set(bdata.obs[f"labels_gex_{key}"])))
    sc.pl.spatial(bdata, color=[None, f"labels_gex_{key}"], img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer",
                  legend_loc=None,ncols=2
                 )
```

Example 24 (unknown):
```unknown
2949655472.py (11): Trying to modify attribute `.obs` of view, initializing view as actual.
```

Example 25 (unknown):
```unknown
2949655472.py (11): Trying to modify attribute `.obs` of view, initializing view as actual.
```

Example 26 (unknown):
```unknown
2949655472.py (11): Trying to modify attribute `.obs` of view, initializing view as actual.
```

Example 27 (unknown):
```unknown
2949655472.py (11): Trying to modify attribute `.obs` of view, initializing view as actual.
```

Example 28 (python):
```python
mask = ((adata.obs['array_row'] >= 1450) & 
        (adata.obs['array_row'] <= 1550) & 
        (adata.obs['array_col'] >= 1400) & 
        (adata.obs['array_col'] <= 1600)
       )
bdata = adata[mask]
sc.pl.spatial(bdata, color=[None, 'n_counts_adjusted'], img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer",
                  legend_loc=None,ncols=2
                 )
```

Example 29 (python):
```python
#the label viewer wants a crop of the processed image
#get the corresponding coordinates spanning the subset object
crop = b2c.get_crop(bdata, basis="array", mpp=mpp)

#GEX pops better with percentile normalisation performed
rendered = b2c.view_labels(image_path="stardist/gex.tiff", 
                           labels_npz_path="stardist/gex_prob0p005.npz", 
                           crop=crop,
                           stardist_normalize=True
                          )
plt.imshow(rendered)
plt.title('sigma: 5, gex_prob_thresh: 0p005')
```

Example 30 (unknown):
```unknown
Text(0.5, 1.0, 'sigma: 5, gex_prob_thresh: 0p005')
```

Example 31 (python):
```python
#the label viewer wants a crop of the processed image
#get the corresponding coordinates spanning the subset object
crop = b2c.get_crop(bdata, basis="array", mpp=mpp)

#GEX pops better with percentile normalisation performed
rendered = b2c.view_labels(image_path="stardist/gex.tiff", 
                           labels_npz_path="stardist/gex_prob0p05.npz", 
                           crop=crop,
                           stardist_normalize=True
                          )
plt.imshow(rendered)
plt.title('sigma: 5, gex_prob_thresh: 0p05')
```

Example 32 (unknown):
```unknown
Text(0.5, 1.0, 'sigma: 5, gex_prob_thresh: 0p05')
```

Example 33 (python):
```python
gex_prob_thresh_sel = '0p05' # sigma: 5
adata.obs['labels_gex_sel'] = adata.obs[f'labels_gex_{gex_prob_thresh_sel}']
```

Example 34 (python):
```python
# save
adata.write('/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/HEA_FOET14880396_b2c.h5ad')
```

Example 35 (unknown):
```unknown
AnnData object with n_obs × n_vars = 5001851 × 18051
    obs: 'in_tissue', 'array_row', 'array_col', 'n_counts', 'destripe_factor', 'n_counts_adjusted', 'labels_he_0p01', 'labels_he_0p03', 'labels_he_0p05', 'labels_he_0p1', 'labels_he_default', 'labels_he_sel', 'labels_he_expanded_1', 'labels_he_expanded_2', 'labels_he_expanded_3', 'labels_he_expanded_4', 'labels_he_expanded_sel', 'labels_gex_0p005', 'labels_gex_0p01', 'labels_gex_0p03', 'labels_gex_0p05', 'labels_gex_sel'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells'
    uns: 'bin2cell', 'spatial'
    obsm: 'spatial', 'spatial_cropped_150_buffer'
```

Example 36 (python):
```python
# read in
mpp=0.3
adata = sc.read_h5ad('/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/HEA_FOET14880396_b2c.h5ad')
adata
```

Example 37 (unknown):
```unknown
AnnData object with n_obs × n_vars = 5001851 × 18051
    obs: 'in_tissue', 'array_row', 'array_col', 'n_counts', 'destripe_factor', 'n_counts_adjusted', 'labels_he_0p01', 'labels_he_0p03', 'labels_he_0p05', 'labels_he_0p1', 'labels_he_default', 'labels_he_sel', 'labels_he_expanded_1', 'labels_he_expanded_2', 'labels_he_expanded_3', 'labels_he_expanded_4', 'labels_he_expanded_sel', 'labels_gex_0p005', 'labels_gex_0p01', 'labels_gex_0p03', 'labels_gex_0p05', 'labels_gex_sel'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells'
    uns: 'bin2cell', 'spatial'
    obsm: 'spatial', 'spatial_cropped_150_buffer'
```

Example 38 (python):
```python
b2c.salvage_secondary_labels(adata, 
                             primary_label="labels_he_expanded_sel", 
                             secondary_label="labels_gex_sel", 
                             labels_key="labels_joint"
                            )
```

Example 39 (unknown):
```unknown
Salvaged 10198 secondary labels
```

Example 40 (python):
```python
mask = ((adata.obs['array_row'] >= 1450) & 
        (adata.obs['array_row'] <= 1550) & 
        (adata.obs['array_col'] >= 1400) & 
        (adata.obs['array_col'] <= 1600)
       )
bdata = adata[mask]

#0 means unassigned
bdata = bdata[bdata.obs['labels_joint']>0]
bdata.obs['labels_joint'] = bdata.obs['labels_joint'].astype(str)

sc.pl.spatial(bdata, color=[None, "labels_joint_source"],
              img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer")
```

Example 41 (unknown):
```unknown
2034932217.py (10): Trying to modify attribute `.obs` of view, initializing view as actual.
```

Example 42 (python):
```python
#0 means unassigned
adata_assigned = adata[adata.obs['labels_joint']>0]
counts = adata_assigned.obs['labels_joint'].value_counts()
plt.figure(figsize=(10, 3))
sns.histplot(counts, kde=False, bins=200)
```

Example 43 (unknown):
```unknown
<Axes: xlabel='count', ylabel='Count'>
```

Example 44 (python):
```python
counts_1st = adata_assigned[adata_assigned.obs['labels_joint_source']=='primary'].obs['labels_joint'].value_counts()
plt.figure(figsize=(10, 3))
sns.histplot(counts_1st, kde=False, bins=200)
```

Example 45 (unknown):
```unknown
<Axes: xlabel='count', ylabel='Count'>
```

Example 46 (python):
```python
counts_2nd = adata_assigned[adata_assigned.obs['labels_joint_source']=='secondary'].obs['labels_joint'].value_counts()
plt.figure(figsize=(10, 3))
sns.histplot(counts_2nd, kde=False, bins=200)
```

Example 47 (unknown):
```unknown
<Axes: xlabel='count', ylabel='Count'>
```

Example 48 (python):
```python
'''
# remove
bin_counts_thresh = 5
labels_to_remove = counts.index[counts<bin_counts_thresh]
mask = adata.obs['labels_joint'].isin(labels_to_remove)
bins_to_remove = adata.obs_names[mask]
adata.obs.loc[bins_to_remove,'labels_joint'] = 0

adata_assigned = adata[adata.obs['labels_joint']>0]
counts = adata_assigned.obs['labels_joint'].value_counts()
plt.figure(figsize=(10, 3))
sns.histplot(counts, kde=False, bins=200)
'''
```

Example 49 (python):
```python
'''
mask = ((adata.obs['array_row'] >= 1450) & 
        (adata.obs['array_row'] <= 1550) & 
        (adata.obs['array_col'] >= 1400) & 
        (adata.obs['array_col'] <= 1600)
       )
bdata = adata[mask]

#0 means unassigned
bdata = bdata[bdata.obs['labels_joint']>0]
bdata.obs['labels_joint'] = bdata.obs['labels_joint'].astype(str)

sc.pl.spatial(bdata, color=[None, "labels_joint_source"], img_key="0.3_mpp", basis="spatial_cropped")
'''
```

Example 50 (python):
```python
# save adata
adata.write('/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/HEA_FOET14880396_b2c.h5ad')
adata
```

Example 51 (unknown):
```unknown
AnnData object with n_obs × n_vars = 5001851 × 18051
    obs: 'in_tissue', 'array_row', 'array_col', 'n_counts', 'destripe_factor', 'n_counts_adjusted', 'labels_he_0p01', 'labels_he_0p03', 'labels_he_0p05', 'labels_he_0p1', 'labels_he_default', 'labels_he_sel', 'labels_he_expanded_1', 'labels_he_expanded_2', 'labels_he_expanded_3', 'labels_he_expanded_4', 'labels_he_expanded_sel', 'labels_gex_0p005', 'labels_gex_0p01', 'labels_gex_0p03', 'labels_gex_0p05', 'labels_gex_sel', 'labels_joint', 'labels_joint_source'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells'
    uns: 'bin2cell', 'spatial'
    obsm: 'spatial', 'spatial_cropped_150_buffer'
```

Example 52 (python):
```python
cdata = b2c.bin_to_cell(adata, labels_key="labels_joint", spatial_keys=["spatial", "spatial_cropped_150_buffer"])
cdata
```

Example 53 (unknown):
```unknown
AnnData object with n_obs × n_vars = 134590 × 18051
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells'
    uns: 'spatial'
    obsm: 'spatial', 'spatial_cropped_150_buffer'
```

Example 54 (python):
```python
cell_mask = ((cdata.obs['array_row'] >= 1450) & 
        (cdata.obs['array_row'] <= 1550) & 
        (cdata.obs['array_col'] >= 1400) & 
        (cdata.obs['array_col'] <= 1600)
       )
ddata = cdata[cell_mask]
sc.pl.spatial(ddata, color=["bin_count", "labels_joint_source"], 
              img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer", ncols=2)
```

Example 55 (unknown):
```unknown
anndata.py (1146): Trying to modify attribute `.obs` of view, initializing view as actual.
```

Example 56 (unknown):
```unknown
cdata.write('/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/HEA_FOET14880396_b2c_cells.h5ad')
```

Example 57 (unknown):
```unknown
cdata.X.data[:5]
```

Example 58 (unknown):
```unknown
array([0.55027094, 1.30175513, 1.01929175, 1.31881525, 3.24395094])
```

---

## Variables¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/Xenium/no2_integration_and_cell-type-annotation/no6_manual-annotation_BloodVesselEC.html

**Contents:**
- Variables¶
- Read in data¶
- Markers¶
- 1st round annotation¶
- Subset and reclustering¶
- Transfer annotation to the original adata¶
- Save¶

--> cluster 0 has lower total counts and smaller cell area, and no specific gene expression --> annotate as "unassigned"

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os

import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 

import scanpy.external as sce
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/session_info/main.py:213: UserWarning: The '__version__' attribute is deprecated and will be removed in MarkupSafe 3.1. Use feature detection, or `importlib.metadata.version("markupsafe")`, instead.
  mod_version = _find_version(mod.__version__)
```

Example 4 (python):
```python
-----
matplotlib          3.10.1
numpy               2.1.3
pandas              2.2.3
scanpy              1.11.1
seaborn             0.13.2
session_info        v1.0.1
-----
```

Example 5 (python):
```python
Cython              3.1.1
PIL                 11.1.0
anndata             0.11.4
asciitree           NA
asttokens           NA
charset_normalizer  3.4.1
cloudpickle         3.1.1
comm                0.2.2
cycler              0.12.1
cython              3.1.1
cython_runtime      NA
dask                2024.11.2
dateutil            2.9.0.post0
debugpy             1.8.13
decorator           5.2.1
exceptiongroup      1.2.2
executing           2.2.0
h5py                3.13.0
igraph              0.11.8
importlib_metadata  NA
ipykernel           6.29.5
jedi                0.19.2
jinja2              3.1.6
joblib              1.4.2
kiwisolver          1.4.8
legacy_api_wrap     NA
leidenalg           0.10.2
llvmlite            0.44.0
markupsafe          3.0.2
mpl_toolkits        NA
msgpack             1.1.0
natsort             8.4.0
numba               0.61.0
numcodecs           0.13.1
packaging           24.2
parso               0.8.4
patsy               1.0.1
platformdirs        4.3.7
prompt_toolkit      3.0.50
psutil              7.0.0
pure_eval           0.2.3
pyarrow             19.0.1
pydev_ipython       NA
pydevconsole        NA
pydevd              3.2.3
pydevd_file_utils   NA
pydevd_plugins      NA
pydevd_tracing      NA
pygments            2.19.1
pyparsing           3.2.3
pytz                2025.2
scipy               1.15.2
six                 1.17.0
sklearn             1.5.2
stack_data          0.6.3
statsmodels         0.14.4
tblib               3.1.0
texttable           1.7.0
threadpoolctl       3.6.0
tlz                 1.0.0
toolz               1.0.0
tornado             6.4.2
traitlets           5.14.3
typing_extensions   NA
wcwidth             0.2.13
yaml                6.0.2
zarr                2.18.3
zipp                NA
zmq                 26.3.0
zoneinfo            NA
```

Example 6 (unknown):
```unknown
-----
IPython             8.34.0
jupyter_client      8.6.3
jupyter_core        5.7.2
-----
Python 3.10.14 | packaged by conda-forge | (main, Mar 20 2024, 12:45:18) [GCC 12.3.0]
Linux-4.18.0-553.53.1.el8_10.x86_64-x86_64-with-glibc2.28
-----
Session information updated at 2025-06-09 22:01
```

Example 7 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/home/kk837/rfs/rfs-teichlab-nfs-iCNyzSAaucw/kk837/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 8 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 9 (python):
```python
adata_dir = '/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium/subsets'
sample_id = 'C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat'
celltype = 'BloodVesselEndothelialCells'
```

Example 10 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk/Aug2024'
table_dir = '/nfs/team205/heart/fetal_ms_figures/tables/Aug2024'
```

Example 11 (unknown):
```unknown
latent_space = 'pca_harmony'
```

Example 12 (python):
```python
# read in
adata = sc.read_h5ad(f'{adata_dir}/{sample_id}_5K_{celltype}_lognorm.h5ad')
adata
```

Example 13 (unknown):
```unknown
AnnData object with n_obs × n_vars = 20055 × 4950
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_3.0', 'multi_celltypes_coarse', 'leiden_0.2', 'leiden_0.4', 'leiden_0.6', 'leiden_1'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1', 'n_cells', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'dendrogram_pca_harmony_leiden_1.0', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1_colors', 'log1p', 'multi_celltypes_coarse_colors', 'pca', 'pca_harmony', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.2_colors', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.3_colors', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.4_colors', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.5_colors', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_0.8_colors', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.0_colors', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_1.5_colors', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_2.0_colors', 'pca_harmony_leiden_3.0', 'pca_harmony_leiden_3.0_colors', 'tissue_block_id_colors', 'umap'
    obsm: 'X_pca', 'X_pca_harmony', 'X_umap_pca_harmony', 'spatial'
    varm: 'PCs'
    obsp: 'pca_harmony_connectivities', 'pca_harmony_distances'
```

Example 14 (python):
```python
adata.X.data[:5]
```

Example 15 (unknown):
```unknown
array([3.0598898, 3.0598898, 3.0598898, 3.0598898, 3.0598898],
      dtype=float32)
```

Example 16 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=['total_counts','n_genes_by_counts','cell_area','tissue_block_id'],
                    wspace=0.2,cmap='RdPu',vmax='p100')
```

Example 17 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=['multi_celltypes_coarse',"conf_score_midmod2fine","celltypist_midmod2fine",],
                    wspace=0.2,cmap='RdPu',vmax='p100')
resolutions_list = [0.2,0.4,0.6,1]
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                    color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                    wspace=0.2)
```

Example 18 (python):
```python
%%time
# leiden clustering
resolutions_list = [# 0.2,0.4,0.6,1, # already done
                    1.2,1.5,2]
for resolution in resolutions_list:
    sc.tl.leiden(adata,neighbors_key=latent_space,
                     resolution=resolution,key_added=f'leiden_{str(resolution)}',n_iterations=2)
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                wspace=0.3)
```

Example 19 (unknown):
```unknown
<timed exec>:5: FutureWarning: In the future, the default backend for leiden will be igraph instead of leidenalg.

 To achieve the future defaults please pass: flavor="igraph" and n_iterations=2.  directed must also be False to work with igraph's implementation.
```

Example 20 (unknown):
```unknown
CPU times: user 16.9 s, sys: 190 ms, total: 17.1 s
Wall time: 17.1 s
```

Example 21 (python):
```python
# read in markers identified using single-nucleus data
markers_df = pd.read_csv(f'/home/kk837/rfs/rfs-teichlab-nfs-iCNyzSAaucw/kk837/notebooks/Foetal/VisiumHD/re-seq_mpp0p3/sc_{celltype}_finegrain_markers.csv')

mask1 = markers_df['group_mean']>1
mask2 = markers_df['logfoldchanges']>1
markers_df = markers_df[mask1&mask2]

markers_df
```

Example 22 (python):
```python
markers = {}
n_top = 15
for c in set(markers_df['group']):
    markers_df_sub = markers_df[markers_df['group']==c]
    markers_df_sub.sort_values('logfoldchanges',ascending=False)
    markers[c]=list(markers_df_sub[:n_top]['names'])

# manually add genes
markers['Cycling'] = ['MKI67','TOP2A'] 

genes = []
for g in markers.values():
    genes = genes + [x for x in g if (x in list(adata.var_names))&(x not in genes)]
for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in list(adata.var_names)]
len(genes)
```

Example 23 (python):
```python
sc.pl.embedding(adata,basis=f"X_umap_{latent_space}",color=genes, size=10,
               ncols=6, wspace=0.2,cmap='RdPu',vmax='p100')
```

Example 24 (unknown):
```unknown
resolution_sel = 0.6
```

Example 25 (python):
```python
markers_others = {
    'CM': ['TTN', 'TNNT2','MYH6',
        'MYH7', 'MYL2', 'FHL2', # vCN
            'NPPA', 'MYL7', 'MYL4'], #aCM
    'FB': ['DCN', 'GCN', 'PDGFRA','COL1A1','COL1A2'],
    'EC': ['VWF', 'PECAM1', 'CDH5','RGCC', 'FABP5',],
    'Peri': ['RGS5', 'ABCC9', 'KCNJ8'],
    'SMC': ['MYH11', 'TAGLN', 'ACTA2'],
    'Meso': ['MSLN', 'WT1', 'BNC1'],
    'Neuro': ['PLP1', 'NRXN1', 'NRXN3','PRPH', 'NEFL', 'NEFM', 'NEFH', 'STMN2', 'SST', 'SLC10A4', 'SLC18A3', 'TH', 'NPY'],
    'Adipo': ['GPAM', 'FASN', 'LEP'],
    'Myelo': ['CD14', 'C1QA', 'CD68','LYVE1','TIMD4'],
    'Lymph': ['CD8A', 'IL7R', 'CD40LG'],
    'B': ['CD19','SDC1','IGHG1','IGHA1'],
    'Mast': ['KIT', 'CPA3'],
}

for key in markers_others.keys():
    markers_others[key]=[x for x in markers_others[key] if x in adata.var_names]

sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
sc.pl.dotplot(adata,
              markers_others, 
              groupby=f'leiden_{str(resolution_sel)}',
              dendrogram=True,
              standard_scale="var",
              color_map="Reds",
              swap_axes=False,
             )
```

Example 26 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: CM, FB, EC, etc.
```

Example 27 (python):
```python
sc.tl.dendrogram(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.pl.dotplot(adata,
              var_names=markers,
              groupby=f'leiden_{str(resolution_sel)}',
              standard_scale='var',
              dendrogram=True
             )
```

Example 28 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: CoronaryCapillaryEndothelialCells, CoronaryArterialEndothelialCells, GreatVesselVenousEndothelialCells, etc.
```

Example 29 (python):
```python
print(adata.X.data[:5])
adata.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata, dendrogram=True, standard_scale='var', return_fig=True)
dp.add_totals().show()
```

Example 30 (unknown):
```unknown
[3.0598898 3.0598898 3.0598898 3.0598898 3.0598898]
```

Example 31 (python):
```python
sc.pl.violin(adata,keys=['total_counts','cell_area'],groupby=f'leiden_{str(resolution_sel)}')
```

Example 32 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=f'leiden_{str(resolution_sel)}',legend_loc='on data',
                )
```

Example 33 (unknown):
```unknown
'BloodVesselEndothelialCells'
```

Example 34 (python):
```python
# annotate
adata.obs['tmp_annotation'] = adata.obs[f'leiden_{str(resolution_sel)}'].copy()
adata.obs.replace({'tmp_annotation':{
    '0':'unassigned',
    '1':'BloodVesselEndothelialCells',
    '2':'BloodVesselEndothelialCells',
    '3':'BloodVesselEndothelialCells',
    '4':'BloodVesselEndothelialCells',
    '5':'?ThyroidCells',
    '6':'BloodVesselEndothelialCells',
    '7':'BloodVesselEndothelialCells',
}},inplace=True)
adata.obs['tmp_annotation'].value_counts()
```

Example 35 (python):
```python
/tmp/ipykernel_68424/4049359585.py:3: FutureWarning: The behavior of Series.replace (and DataFrame.replace) with CategoricalDtype is deprecated. In a future version, replace will only be used for cases that preserve the categories. To change the categories, use ser.cat.rename_categories instead.
  adata.obs.replace({'tmp_annotation':{
```

Example 36 (unknown):
```unknown
tmp_annotation
BloodVesselEndothelialCells    14065
unassigned                      5199
?ThyroidCells                    791
Name: count, dtype: int64
```

Example 37 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color='tmp_annotation'
                )
```

Example 38 (python):
```python
adata_sub = adata[adata.obs['tmp_annotation'].isin(['BloodVesselEndothelialCells'])]
```

Example 39 (python):
```python
# drop leiden clustering results
resolutions_list = [0.2,0.4,0.6,1, 1.2,1.5,2]
for resolution in resolutions_list:
    adata_sub.obs.drop(f'leiden_{str(resolution)}',axis=1,inplace=True)
adata_sub
```

Example 40 (unknown):
```unknown
AnnData object with n_obs × n_vars = 14065 × 4950
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_3.0', 'multi_celltypes_coarse', 'tmp_annotation'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1', 'n_cells', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'dendrogram_pca_harmony_leiden_1.0', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1_colors', 'log1p', 'multi_celltypes_coarse_colors', 'pca', 'pca_harmony', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.2_colors', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.3_colors', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.4_colors', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.5_colors', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_0.8_colors', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.0_colors', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_1.5_colors', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_2.0_colors', 'pca_harmony_leiden_3.0', 'pca_harmony_leiden_3.0_colors', 'tissue_block_id_colors', 'umap', 'leiden_1.2', 'leiden_1.5', 'leiden_2', 'leiden_1.2_colors', 'leiden_1.5_colors', 'leiden_2_colors', 'dendrogram_leiden_0.6', 'rank_genes_groups', 'tmp_annotation_colors'
    obsm: 'X_pca', 'X_pca_harmony', 'X_umap_pca_harmony', 'spatial'
    varm: 'PCs'
    obsp: 'pca_harmony_connectivities', 'pca_harmony_distances'
```

Example 41 (python):
```python
## Remove mid-grain markers from the feature
# read in markers
midgrain_markers = pd.read_csv('/home/kk837/rfs/rfs-teichlab-nfs-iCNyzSAaucw/kk837/notebooks/Foetal/VisiumHD/re-seq_mpp0p3/sc_midgrain_markers.csv',
                               index_col=0)
# remove the markers of celltype of interest
midgrain_markers = midgrain_markers[midgrain_markers['group']!=celltype]
# remove
mask = [x not in list(midgrain_markers['names']) for x in adata_sub.var_names]
bdata = adata_sub[:,mask]
bdata
```

Example 42 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 14065 × 4477
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_3.0', 'multi_celltypes_coarse', 'tmp_annotation'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1', 'n_cells', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'dendrogram_pca_harmony_leiden_1.0', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1_colors', 'log1p', 'multi_celltypes_coarse_colors', 'pca', 'pca_harmony', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.2_colors', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.3_colors', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.4_colors', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.5_colors', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_0.8_colors', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.0_colors', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_1.5_colors', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_2.0_colors', 'pca_harmony_leiden_3.0', 'pca_harmony_leiden_3.0_colors', 'tissue_block_id_colors', 'umap', 'leiden_1.2', 'leiden_1.5', 'leiden_2', 'leiden_1.2_colors', 'leiden_1.5_colors', 'leiden_2_colors', 'dendrogram_leiden_0.6', 'rank_genes_groups', 'tmp_annotation_colors'
    obsm: 'X_pca', 'X_pca_harmony', 'X_umap_pca_harmony', 'spatial'
    varm: 'PCs'
    obsp: 'pca_harmony_connectivities', 'pca_harmony_distances'
```

Example 43 (python):
```python
'''
# remove cell cycle genes
cell_cycle_genes = ['MCM5','PCNA','TYMS','FEN1','MCM2','MCM4','RRM1','UNG','GINS2','MCM6','CDCA7','DTL','PRIM1','UHRF1','MLF1IP','HELLS','RFC2','RPA2','NASP',
                'RAD51AP1','GMNN','WDR76','SLBP','CCNE2','UBR7','POLD3','MSH2','ATAD2','RAD51','RRM2','CDC45','CDC6','EXO1','TIPIN','DSCC1','BLM','CASP8AP2',
                'USP1','CLSPN','POLA1','CHAF1B','BRIP1','E2F8','HMGB2','CDK1','NUSAP1','UBE2C','BIRC5','TPX2','TOP2A','NDC80','CKS2','NUF2','CKS1B','MKI67',
                'TMPO','CENPF','TACC3','FAM64A','SMC4','CCNB2','CKAP2L','CKAP2','AURKB','BUB1','KIF11','ANP32E','TUBB4B','GTSE1','KIF20B','HJURP','CDCA3','HN1',
                'CDC20','TTK','CDC25C','KIF2C','RANGAP1','NCAPD2','DLGAP5','CDCA2','CDCA8','ECT2','KIF23','HMMR','AURKA','PSRC1','ANLN','LBR','CKAP5','CENPE','CTCF',
                'NEK2','G2E3','GAS2L3','CBX5','CENPA']
mask = [x not in cell_cycle_genes for x in bdata.var_names]
bdata = bdata[:,mask]
bdata.shape
'''
```

Example 44 (python):
```python
# HVGs, using the object which doesn't have midgrain markers
sc.pp.highly_variable_genes(bdata, flavor='seurat', min_mean=0.0125, max_mean=3, min_disp=0.5)
print(f'number of hvgs: {sum(bdata.var.highly_variable)}')
# PCA using the degected HVGs
sc.tl.pca(bdata, use_highly_variable=True)
adata_sub.obsm['X_pca'] = bdata.obsm['X_pca'].copy()
# harmony integration
sce.pp.harmony_integrate(adata_sub, 'tissue_block_id', basis='X_pca', adjusted_basis='X_pca_harmony')
# create neighbour graph
latent_space = 'pca_harmony'
sc.pp.neighbors(adata_sub,n_neighbors=15,use_rep=f"X_{latent_space}",key_added=latent_space)
# visualisation with umap
sc.tl.umap(adata_sub,min_dist=0.5,neighbors_key=latent_space)
adata_sub.obsm[f"X_umap_{latent_space}"]=adata_sub.obsm["X_umap"]
del adata_sub.obsm["X_umap"]
```

Example 45 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_highly_variable_genes.py:698: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  adata.uns["hvg"] = {"flavor": flavor}
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_pca/__init__.py:438: FutureWarning: Argument `use_highly_variable` is deprecated, consider using the mask argument. Use_highly_variable=True can be called through mask_var="highly_variable". Use_highly_variable=False can be called through mask_var=None
  warn(msg, FutureWarning)
```

Example 46 (unknown):
```unknown
number of hvgs: 678
```

Example 47 (python):
```python
2025-06-09 22:05:14,014 - harmonypy - INFO - Computing initial centroids with sklearn.KMeans...
2025-06-09 22:05:16,262 - harmonypy - INFO - sklearn.KMeans initialization complete.
2025-06-09 22:05:16,351 - harmonypy - INFO - Iteration 1 of 10
2025-06-09 22:05:18,155 - harmonypy - INFO - Iteration 2 of 10
2025-06-09 22:05:19,990 - harmonypy - INFO - Converged after 2 iterations
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm
```

Example 48 (python):
```python
%%time
# reclustering
# leiden clustering
resolutions_list = [0.2,0.4,0.6,1, 1.2,1.5,2]
for resolution in resolutions_list:
    sc.tl.leiden(adata_sub,neighbors_key=latent_space,
                 resolution=resolution,key_added=f'leiden_{str(resolution)}',n_iterations=2)
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                wspace=0.3)
```

Example 49 (unknown):
```unknown
CPU times: user 22.4 s, sys: 190 ms, total: 22.6 s
Wall time: 22.7 s
```

Example 50 (python):
```python
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",size=10,
                color=genes,
               ncols=6, wspace=0.2,cmap='RdPu',vmax='p100')
```

Example 51 (python):
```python
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color=["conf_score_midmod2fine","celltypist_midmod2fine",],
                    wspace=0.2,cmap='RdPu',vmax='p100')
```

Example 52 (python):
```python
# umap per celltype to highlight
group_col = 'celltypist_midmod2fine'
groups_to_plot = list(set(adata_sub.obs[group_col]))

plt.rcParams['axes.grid'] = False
fig,axs=plt.subplots(nrows=2,ncols=4,figsize=(15,8))
# Flatten the axs array for easier iteration in case of multiple rows and columns
axs_flat = axs.flatten()

for i,grp in enumerate(groups_to_plot):
    sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                    color=[group_col], groups=grp, 
               size=10, frameon=False,
               show=False, ax=axs_flat[i], legend_loc=None, legend_fontsize=5,
               title=grp
              )
# Turn off any remaining subplots
for j in range(len(groups_to_plot), len(axs_flat)):
    fig.delaxes(axs_flat[j])

# fig.suptitle('Main Title for the Figure')
fig.tight_layout()
```

Example 53 (unknown):
```unknown
sub_resolution_sel = 2
```

Example 54 (python):
```python
markers_others = {
    'CM': ['TTN', 'TNNT2','MYH6',
        'MYH7', 'MYL2', 'FHL2', # vCN
            'NPPA', 'MYL7', 'MYL4'], #aCM
    'FB': ['DCN', 'GCN', 'PDGFRA','COL1A1','COL1A2'],
    'EC': ['VWF', 'PECAM1', 'CDH5','RGCC', 'FABP5',],
    'Peri': ['RGS5', 'ABCC9', 'KCNJ8'],
    'SMC': ['MYH11', 'TAGLN', 'ACTA2'],
    'Meso': ['MSLN', 'WT1', 'BNC1'],
    'Neuro': ['PLP1', 'NRXN1', 'NRXN3','PRPH', 'NEFL', 'NEFM', 'NEFH', 'STMN2', 'SST', 'SLC10A4', 'SLC18A3', 'TH', 'NPY'],
    'Adipo': ['GPAM', 'FASN', 'LEP'],
    'Myelo': ['CD14', 'C1QA', 'CD68','LYVE1','TIMD4'],
    'Lymph': ['CD8A', 'IL7R', 'CD40LG'],
    'B': ['CD19','SDC1','IGHG1','IGHA1'],
    'Mast': ['KIT', 'CPA3'],
}

for key in markers_others.keys():
    markers_others[key]=[x for x in markers_others[key] if x in adata.var_names]

sc.tl.dendrogram(adata_sub, groupby=f'leiden_{str(sub_resolution_sel)}')
sc.pl.dotplot(adata_sub,
              markers_others, 
              groupby=f'leiden_{str(sub_resolution_sel)}',
              dendrogram=True,
              standard_scale="var",
              color_map="Reds",
              swap_axes=False,
             )
```

Example 55 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: CM, FB, EC, etc.
```

Example 56 (python):
```python
sc.tl.dendrogram(adata_sub,groupby=f'leiden_{str(sub_resolution_sel)}')
sc.pl.dotplot(adata_sub,
              var_names=markers,
              groupby=f'leiden_{str(sub_resolution_sel)}',
              standard_scale='var',
              dendrogram=True
             )
```

Example 57 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: CoronaryCapillaryEndothelialCells, CoronaryArterialEndothelialCells, GreatVesselVenousEndothelialCells, etc.
```

Example 58 (python):
```python
# DEG analysis
print(adata_sub.X.data[:5])
adata_sub.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata_sub,groupby=f'leiden_{str(sub_resolution_sel)}')
sc.tl.dendrogram(adata_sub, groupby=f'leiden_{str(sub_resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata_sub, dendrogram=True, standard_scale='var', return_fig=True)
dp.add_totals().show()
```

Example 59 (unknown):
```unknown
[2.8347206 2.8347206 2.8347206 3.893389  2.8347206]
```

Example 60 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:458: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "names"] = self.var_names[global_indices]
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:460: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "scores"] = scores[global_indices]
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:463: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "pvals"] = pvals[global_indices]
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:473: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "pvals_adj"] = pvals_adj[global_indices]
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:484: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "logfoldchanges"] = np.log2(
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:458: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "names"] = self.var_names[global_indices]
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:460: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "scores"] = scores[global_indices]
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:463: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "pvals"] = pvals[global_indices]
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:473: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "pvals_adj"] = pvals_adj[global_indices]
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:484: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "logfoldchanges"] = np.log2(
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:458: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "names"] = self.var_names[global_indices]
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:460: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "scores"] = scores[global_indices]
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:463: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "pvals"] = pvals[global_indices]
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:473: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "pvals_adj"] = pvals_adj[global_indices]
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:484: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  self.stats[group_name, "logfoldchanges"] = np.log2(
```

Example 61 (python):
```python
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color=f'leiden_{str(sub_resolution_sel)}',legend_loc='on data',
                wspace=0.3)
```

Example 62 (python):
```python
# annotate
adata_sub.obs['fine_grain_manual'] = adata_sub.obs[f'leiden_{str(sub_resolution_sel)}'].copy()
adata_sub.obs.replace({'fine_grain_manual':{
    '0':'CoronaryCapillaryEndothelialCells',
    '1':'CoronaryCapillaryEndothelialCellsCycling',
    '2':'GreatVesselVenousEndothelialCells',
    '3':'CoronaryCapillaryEndothelialCellsNOTCHpos',
    '4':'CoronaryCapillaryEndothelialCells',
    '5':'CoronaryCapillaryEndothelialCells',
    '6':'CoronaryVenousEndothelialCells',
    '7':'CoronaryCapillaryEndothelialCells',
    '8':'CoronaryCapillaryEndothelialCells',
    '9':'CoronaryVenousEndothelialCells',
    '10':'CoronaryCapillaryEndothelialCellsNOTCHpos',
    '11':'CoronaryCapillaryEndothelialCells',
    '12':'CoronaryArterialEndothelialCells',
    '13':'CoronaryCapillaryEndothelialCells',
    '14':'GreatVesselArterialEndothelialCells',
    '15':'CoronaryCapillaryEndothelialCells',
    '16':'CoronaryCapillaryEndothelialCells',
    '17':'CoronaryCapillaryEndothelialCells',
    '18':'CoronaryCapillaryEndothelialCells',
    '19':'CoronaryCapillaryEndothelialCells_DoubletsWithEryth',
    '20':'CoronaryCapillaryEndothelialCells',
    '21':'CoronaryCapillaryEndothelialCellsCycling',
    '22':'CoronaryCapillaryEndothelialCells',
}},inplace=True)
adata_sub.obs['fine_grain_manual'].value_counts()
```

Example 63 (python):
```python
/tmp/ipykernel_68424/23694716.py:3: FutureWarning: The behavior of Series.replace (and DataFrame.replace) with CategoricalDtype is deprecated. In a future version, replace will only be used for cases that preserve the categories. To change the categories, use ser.cat.rename_categories instead.
  adata_sub.obs.replace({'fine_grain_manual':{
```

Example 64 (unknown):
```unknown
fine_grain_manual
CoronaryCapillaryEndothelialCells                      7259
CoronaryCapillaryEndothelialCellsNOTCHpos              1579
CoronaryVenousEndothelialCells                         1493
CoronaryCapillaryEndothelialCellsCycling               1414
GreatVesselVenousEndothelialCells                      1145
CoronaryArterialEndothelialCells                        504
GreatVesselArterialEndothelialCells                     472
CoronaryCapillaryEndothelialCells_DoubletsWithEryth     199
Name: count, dtype: int64
```

Example 65 (python):
```python
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color='fine_grain_manual',
                wspace=0.3)
```

Example 66 (python):
```python
adata.obs['fine_grain_manual'] = adata.obs['tmp_annotation'].astype('str').copy()
adata.obs.loc[adata_sub.obs_names,'fine_grain_manual'] = adata_sub.obs['fine_grain_manual']
```

Example 67 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color='fine_grain_manual'
                )
```

Example 68 (python):
```python
for ct in set(adata.obs['fine_grain_manual']):
    for tissue_id in ['C194-HEA-0-FFPE-1', 'Hst45-HEA-0-FFPE-1']:
        slide = adata[adata.obs['tissue_block_id']==tissue_id]
        sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
                      spot_size=60,vmax='p100')
```

Example 69 (python):
```python
/tmp/ipykernel_68424/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 70 (python):
```python
/tmp/ipykernel_68424/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 71 (python):
```python
/tmp/ipykernel_68424/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 72 (python):
```python
/tmp/ipykernel_68424/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 73 (python):
```python
/tmp/ipykernel_68424/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 74 (python):
```python
/tmp/ipykernel_68424/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 75 (python):
```python
/tmp/ipykernel_68424/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 76 (python):
```python
/tmp/ipykernel_68424/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 77 (python):
```python
/tmp/ipykernel_68424/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 78 (python):
```python
/tmp/ipykernel_68424/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 79 (python):
```python
/tmp/ipykernel_68424/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 80 (python):
```python
/tmp/ipykernel_68424/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 81 (python):
```python
/tmp/ipykernel_68424/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 82 (python):
```python
/tmp/ipykernel_68424/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 83 (python):
```python
/tmp/ipykernel_68424/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 84 (python):
```python
/tmp/ipykernel_68424/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 85 (python):
```python
/tmp/ipykernel_68424/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 86 (python):
```python
/tmp/ipykernel_68424/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 87 (python):
```python
/tmp/ipykernel_68424/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 88 (python):
```python
/tmp/ipykernel_68424/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 89 (python):
```python
# save
adata.write(f'{adata_dir}/{sample_id}_5K_{celltype}_lognorm.h5ad')
print(f'{adata_dir}/{sample_id}_5K_{celltype}_lognorm.h5ad')
```

Example 90 (unknown):
```unknown
/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium/subsets/C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat_5K_BloodVesselEndothelialCells_lognorm.h5ad
```

---

## Functions¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig4_Cardiomyocytes/AtrialCardiomyocytes/Visium_mapping/c2l_pt2_mappings.html

**Contents:**
  - Loading packages¶
- Functions¶
- Read in metadata¶
- Data, parameters¶
- Loading visium data¶
- Filter out low quality spots¶
- Remove mitochondrial genes¶
- Add metadata and annotation¶
- Remove spots based on manual annotation¶
- Combine pcw13w-Atrial and pcw13w-Ventricle data to have whole heart in one sample¶
- Run per group: cell2location mapping¶

**Examples:**

Example 1 (python):
```python
import sys
import scanpy as sc
import anndata
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt 
import matplotlib as mpl

import cell2location
import scvi

from matplotlib import rcParams
rcParams['pdf.fonttype'] = 42 # enables correct plotting of text
import seaborn as sns
```

Example 2 (python):
```python
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm
Global seed set to 0
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/umap/distances.py:1063: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/umap/distances.py:1071: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/umap/distances.py:1086: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/umap/umap_.py:660: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
```

Example 3 (python):
```python
import gc
import torch
```

Example 4 (python):
```python
def read_and_qc(sample_name, path):
    r""" This function reads the data for one 10X spatial experiment into the anndata object.
    It also calculates QC metrics. Modify this function if required by your workflow.

    :param sample_name: Name of the sample
    :param path: path to data
    """

    adata = sc.read_visium(path + str(sample_name),
                           count_file='filtered_feature_bc_matrix.h5', load_images=True)
    adata.var_names_make_unique(join='-')
    
    adata.obs['sample'] = sample_name
    
    # rename genes to ENSEMBL
    adata.var['SYMBOL'] = adata.var_names
    adata.var_names = adata.var['gene_ids']
    adata.var_names.name = None

    # Calculate QC metrics
    sc.pp.calculate_qc_metrics(adata, inplace=True)
    adata.var['mt'] = [gene.startswith('MT-') for gene in adata.var['SYMBOL']]
    adata.obs['mt_frac'] = adata[:, adata.var['mt'].tolist()].X.sum(1).A.squeeze()/adata.obs['total_counts']

    # add sample name to obs names
    adata.obs["sample"] = [str(i) for i in adata.obs['sample']]
    adata.obs_names = adata.obs["sample"] \
                          + '_' + adata.obs_names
    adata.obs.index.name = 'spot_id'

    return adata

def select_slide(adata, s, s_col='sample'):
    r""" This function selects the data for one slide from the spatial anndata object.

    :param adata: Anndata object with multiple spatial experiments
    :param s: name of selected experiment
    :param s_col: column in adata.obs listing experiment name for each location
    """

    slide = adata[adata.obs[s_col].isin([s]), :]
    s_keys = list(slide.uns['spatial'].keys())
    s_spatial = np.array(s_keys)[[s in k for k in s_keys]][0]

    slide.uns['spatial'] = {s_spatial: slide.uns['spatial'][s_spatial]}

    return slide
```

Example 5 (python):
```python
meta=pd.read_csv('/nfs/team205/heart/anndata_objects/Foetal/metadata/HeartTeamSamples_Mappeddata_20230208_for-cell2loc.csv',
                index_col=0)
meta
```

Example 6 (python):
```python
pd.crosstab(meta['donor'],meta['cell2loc-map_group'])
```

Example 7 (unknown):
```unknown
# select slides
# Healthy, OCT, and without DA
meta = meta[meta['cell2loc-map_group'].isin(['4PCW','5PCW','7PCW','13PCW','20PCW'])]

sampleIDs = list(meta['sangerID'])
len(sampleIDs)
```

Example 8 (unknown):
```unknown
run = "20Mar2024_fetal_OCT"
```

Example 9 (python):
```python
import os
results_folder = os.path.join(os.getcwd(),"analysis/Visium/cell2location_results")

ref_run_name = f'{results_folder}/reference_signatures/{run}'
run_name = f'{results_folder}/cell2location_map/{run}'
```

Example 10 (unknown):
```unknown
visium_path='/nfs/team205/heart/mapped/spaceranger110/'

celltype_key='fine_grain_modified'
```

Example 11 (python):
```python
# Read the data of the interested region into anndata objects
slides = []
for i in sampleIDs:
    slides.append(read_and_qc(i, path=visium_path))

# Combine anndata objects together
adata_vis = slides[0].concatenate(
    slides[1:],
    batch_key="sample",
    uns_merge="unique",
    batch_categories=sampleIDs,
    index_unique=None
)
```

Example 12 (python):
```python
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/_core/anndata.py:1830: UserWarning: Variable names are not unique. To make them unique, call `.var_names_make_unique`.
  utils.warn_names_duplicates("var")
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/_core/anndata.py:1830: UserWarning: Variable names are not unique. To make them unique, call `.var_names_make_unique`.
  utils.warn_names_duplicates("var")
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/_core/anndata.py:1830: UserWarning: Variable names are not unique. To make them unique, call `.var_names_make_unique`.
  utils.warn_names_duplicates("var")
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/_core/anndata.py:1830: UserWarning: Variable names are not unique. To make them unique, call `.var_names_make_unique`.
  utils.warn_names_duplicates("var")
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/_core/anndata.py:1830: UserWarning: Variable names are not unique. To make them unique, call `.var_names_make_unique`.
  utils.warn_names_duplicates("var")
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/_core/anndata.py:1830: UserWarning: Variable names are not unique. To make them unique, call `.var_names_make_unique`.
  utils.warn_names_duplicates("var")
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/_core/anndata.py:1830: UserWarning: Variable names are not unique. To make them unique, call `.var_names_make_unique`.
  utils.warn_names_duplicates("var")
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/_core/anndata.py:1830: UserWarning: Variable names are not unique. To make them unique, call `.var_names_make_unique`.
  utils.warn_names_duplicates("var")
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/_core/anndata.py:1830: UserWarning: Variable names are not unique. To make them unique, call `.var_names_make_unique`.
  utils.warn_names_duplicates("var")
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/_core/anndata.py:1830: UserWarning: Variable names are not unique. To make them unique, call `.var_names_make_unique`.
  utils.warn_names_duplicates("var")
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/_core/anndata.py:1830: UserWarning: Variable names are not unique. To make them unique, call `.var_names_make_unique`.
  utils.warn_names_duplicates("var")
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/_core/anndata.py:1830: UserWarning: Variable names are not unique. To make them unique, call `.var_names_make_unique`.
  utils.warn_names_duplicates("var")
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/_core/anndata.py:1830: UserWarning: Variable names are not unique. To make them unique, call `.var_names_make_unique`.
  utils.warn_names_duplicates("var")
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/_core/anndata.py:1830: UserWarning: Variable names are not unique. To make them unique, call `.var_names_make_unique`.
  utils.warn_names_duplicates("var")
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/_core/anndata.py:1830: UserWarning: Variable names are not unique. To make them unique, call `.var_names_make_unique`.
  utils.warn_names_duplicates("var")
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/_core/anndata.py:1830: UserWarning: Variable names are not unique. To make them unique, call `.var_names_make_unique`.
  utils.warn_names_duplicates("var")
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/_core/anndata.py:1830: UserWarning: Variable names are not unique. To make them unique, call `.var_names_make_unique`.
  utils.warn_names_duplicates("var")
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/_core/anndata.py:1830: UserWarning: Variable names are not unique. To make them unique, call `.var_names_make_unique`.
  utils.warn_names_duplicates("var")
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/_core/anndata.py:1785: FutureWarning: X.dtype being converted to np.float32 from float64. In the next version of anndata (0.9) conversion will not be automatic. Pass dtype explicitly to avoid this warning. Pass `AnnData(X, dtype=X.dtype, ...)` to get the future behavour.
  [AnnData(sparse.csr_matrix(a.shape), obs=a.obs) for a in all_adatas],
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/_core/anndata.py:1785: FutureWarning: X.dtype being converted to np.float32 from float64. In the next version of anndata (0.9) conversion will not be automatic. Pass dtype explicitly to avoid this warning. Pass `AnnData(X, dtype=X.dtype, ...)` to get the future behavour.
  [AnnData(sparse.csr_matrix(a.shape), obs=a.obs) for a in all_adatas],
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/_core/anndata.py:1785: FutureWarning: X.dtype being converted to np.float32 from float64. In the next version of anndata (0.9) conversion will not be automatic. Pass dtype explicitly to avoid this warning. Pass `AnnData(X, dtype=X.dtype, ...)` to get the future behavour.
  [AnnData(sparse.csr_matrix(a.shape), obs=a.obs) for a in all_adatas],
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/_core/anndata.py:1785: FutureWarning: X.dtype being converted to np.float32 from float64. In the next version of anndata (0.9) conversion will not be automatic. Pass dtype explicitly to avoid this warning. Pass `AnnData(X, dtype=X.dtype, ...)` to get the future behavour.
  [AnnData(sparse.csr_matrix(a.shape), obs=a.obs) for a in all_adatas],
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/_core/anndata.py:1785: FutureWarning: X.dtype being converted to np.float32 from float64. In the next version of anndata (0.9) conversion will not be automatic. Pass dtype explicitly to avoid this warning. Pass `AnnData(X, dtype=X.dtype, ...)` to get the future behavour.
  [AnnData(sparse.csr_matrix(a.shape), obs=a.obs) for a in all_adatas],
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/_core/anndata.py:1785: FutureWarning: X.dtype being converted to np.float32 from float64. In the next version of anndata (0.9) conversion will not be automatic. Pass dtype explicitly to avoid this warning. Pass `AnnData(X, dtype=X.dtype, ...)` to get the future behavour.
  [AnnData(sparse.csr_matrix(a.shape), obs=a.obs) for a in all_adatas],
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/_core/anndata.py:1785: FutureWarning: X.dtype being converted to np.float32 from float64. In the next version of anndata (0.9) conversion will not be automatic. Pass dtype explicitly to avoid this warning. Pass `AnnData(X, dtype=X.dtype, ...)` to get the future behavour.
  [AnnData(sparse.csr_matrix(a.shape), obs=a.obs) for a in all_adatas],
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/_core/anndata.py:1785: FutureWarning: X.dtype being converted to np.float32 from float64. In the next version of anndata (0.9) conversion will not be automatic. Pass dtype explicitly to avoid this warning. Pass `AnnData(X, dtype=X.dtype, ...)` to get the future behavour.
  [AnnData(sparse.csr_matrix(a.shape), obs=a.obs) for a in all_adatas],
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/_core/anndata.py:1785: FutureWarning: X.dtype being converted to np.float32 from float64. In the next version of anndata (0.9) conversion will not be automatic. Pass dtype explicitly to avoid this warning. Pass `AnnData(X, dtype=X.dtype, ...)` to get the future behavour.
  [AnnData(sparse.csr_matrix(a.shape), obs=a.obs) for a in all_adatas],
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/_core/anndata.py:1785: FutureWarning: X.dtype being converted to np.float32 from float64. In the next version of anndata (0.9) conversion will not be automatic. Pass dtype explicitly to avoid this warning. Pass `AnnData(X, dtype=X.dtype, ...)` to get the future behavour.
  [AnnData(sparse.csr_matrix(a.shape), obs=a.obs) for a in all_adatas],
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/_core/anndata.py:1785: FutureWarning: X.dtype being converted to np.float32 from float64. In the next version of anndata (0.9) conversion will not be automatic. Pass dtype explicitly to avoid this warning. Pass `AnnData(X, dtype=X.dtype, ...)` to get the future behavour.
  [AnnData(sparse.csr_matrix(a.shape), obs=a.obs) for a in all_adatas],
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/_core/anndata.py:1785: FutureWarning: X.dtype being converted to np.float32 from float64. In the next version of anndata (0.9) conversion will not be automatic. Pass dtype explicitly to avoid this warning. Pass `AnnData(X, dtype=X.dtype, ...)` to get the future behavour.
  [AnnData(sparse.csr_matrix(a.shape), obs=a.obs) for a in all_adatas],
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/_core/anndata.py:1785: FutureWarning: X.dtype being converted to np.float32 from float64. In the next version of anndata (0.9) conversion will not be automatic. Pass dtype explicitly to avoid this warning. Pass `AnnData(X, dtype=X.dtype, ...)` to get the future behavour.
  [AnnData(sparse.csr_matrix(a.shape), obs=a.obs) for a in all_adatas],
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/_core/anndata.py:1785: FutureWarning: X.dtype being converted to np.float32 from float64. In the next version of anndata (0.9) conversion will not be automatic. Pass dtype explicitly to avoid this warning. Pass `AnnData(X, dtype=X.dtype, ...)` to get the future behavour.
  [AnnData(sparse.csr_matrix(a.shape), obs=a.obs) for a in all_adatas],
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/_core/anndata.py:1785: FutureWarning: X.dtype being converted to np.float32 from float64. In the next version of anndata (0.9) conversion will not be automatic. Pass dtype explicitly to avoid this warning. Pass `AnnData(X, dtype=X.dtype, ...)` to get the future behavour.
  [AnnData(sparse.csr_matrix(a.shape), obs=a.obs) for a in all_adatas],
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/_core/anndata.py:1785: FutureWarning: X.dtype being converted to np.float32 from float64. In the next version of anndata (0.9) conversion will not be automatic. Pass dtype explicitly to avoid this warning. Pass `AnnData(X, dtype=X.dtype, ...)` to get the future behavour.
  [AnnData(sparse.csr_matrix(a.shape), obs=a.obs) for a in all_adatas],
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/_core/anndata.py:1785: FutureWarning: X.dtype being converted to np.float32 from float64. In the next version of anndata (0.9) conversion will not be automatic. Pass dtype explicitly to avoid this warning. Pass `AnnData(X, dtype=X.dtype, ...)` to get the future behavour.
  [AnnData(sparse.csr_matrix(a.shape), obs=a.obs) for a in all_adatas],
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/_core/anndata.py:1785: FutureWarning: X.dtype being converted to np.float32 from float64. In the next version of anndata (0.9) conversion will not be automatic. Pass dtype explicitly to avoid this warning. Pass `AnnData(X, dtype=X.dtype, ...)` to get the future behavour.
  [AnnData(sparse.csr_matrix(a.shape), obs=a.obs) for a in all_adatas],
```

Example 13 (python):
```python
# PLOT QC FOR EACH SAMPLE
fig, axs = plt.subplots(len(slides), 4, figsize=(15, 4*len(slides)-4))
for i, s in enumerate(adata_vis.obs['sample'].unique()):
    #fig.suptitle('Covariates for filtering')

    slide = select_slide(adata_vis, s)
    sns.distplot(slide.obs['total_counts'],
                 kde=False, ax = axs[i, 0])
    axs[i, 0].set_xlim(0, adata_vis.obs['total_counts'].max())
    axs[i, 0].set_xlabel(f'total_counts | {s}')

    sns.distplot(slide.obs['total_counts']\
                 [slide.obs['total_counts']<2000],
                 kde=False, bins=40, ax = axs[i, 1])
    axs[i, 1].set_xlim(0, 2000)
    axs[i, 1].set_xlabel(f'total_counts | {s}')

    sns.distplot(slide.obs['n_genes_by_counts'],
                 kde=False, bins=60, ax = axs[i, 2])
    axs[i, 2].set_xlim(0, adata_vis.obs['n_genes_by_counts'].max())
    axs[i, 2].set_xlabel(f'n_genes_by_counts | {s}')

    sns.distplot(slide.obs['n_genes_by_counts']\
                 [slide.obs['n_genes_by_counts']<1000],
                 kde=False, bins=60, ax = axs[i, 3])
    axs[i, 3].set_xlim(0, 1000)
    axs[i, 3].set_xlabel(f'n_genes_by_counts | {s}')

plt.tight_layout()
```

Example 14 (python):
```python
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
/tmp/ipykernel_8715/2051608433.py:7: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['total_counts'],
/tmp/ipykernel_8715/2051608433.py:12: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['total_counts']\
/tmp/ipykernel_8715/2051608433.py:18: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['n_genes_by_counts'],
/tmp/ipykernel_8715/2051608433.py:23: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['n_genes_by_counts']\
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
/tmp/ipykernel_8715/2051608433.py:7: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['total_counts'],
/tmp/ipykernel_8715/2051608433.py:12: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['total_counts']\
/tmp/ipykernel_8715/2051608433.py:18: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['n_genes_by_counts'],
/tmp/ipykernel_8715/2051608433.py:23: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['n_genes_by_counts']\
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
/tmp/ipykernel_8715/2051608433.py:7: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['total_counts'],
/tmp/ipykernel_8715/2051608433.py:12: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['total_counts']\
/tmp/ipykernel_8715/2051608433.py:18: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['n_genes_by_counts'],
/tmp/ipykernel_8715/2051608433.py:23: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['n_genes_by_counts']\
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
/tmp/ipykernel_8715/2051608433.py:7: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['total_counts'],
/tmp/ipykernel_8715/2051608433.py:12: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['total_counts']\
/tmp/ipykernel_8715/2051608433.py:18: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['n_genes_by_counts'],
/tmp/ipykernel_8715/2051608433.py:23: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['n_genes_by_counts']\
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
/tmp/ipykernel_8715/2051608433.py:7: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['total_counts'],
/tmp/ipykernel_8715/2051608433.py:12: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['total_counts']\
/tmp/ipykernel_8715/2051608433.py:18: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['n_genes_by_counts'],
/tmp/ipykernel_8715/2051608433.py:23: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['n_genes_by_counts']\
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
/tmp/ipykernel_8715/2051608433.py:7: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['total_counts'],
/tmp/ipykernel_8715/2051608433.py:12: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['total_counts']\
/tmp/ipykernel_8715/2051608433.py:18: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['n_genes_by_counts'],
/tmp/ipykernel_8715/2051608433.py:23: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['n_genes_by_counts']\
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
/tmp/ipykernel_8715/2051608433.py:7: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['total_counts'],
/tmp/ipykernel_8715/2051608433.py:12: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['total_counts']\
/tmp/ipykernel_8715/2051608433.py:18: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['n_genes_by_counts'],
/tmp/ipykernel_8715/2051608433.py:23: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['n_genes_by_counts']\
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
/tmp/ipykernel_8715/2051608433.py:7: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['total_counts'],
/tmp/ipykernel_8715/2051608433.py:12: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['total_counts']\
/tmp/ipykernel_8715/2051608433.py:18: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['n_genes_by_counts'],
/tmp/ipykernel_8715/2051608433.py:23: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['n_genes_by_counts']\
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
/tmp/ipykernel_8715/2051608433.py:7: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['total_counts'],
/tmp/ipykernel_8715/2051608433.py:12: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['total_counts']\
/tmp/ipykernel_8715/2051608433.py:18: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['n_genes_by_counts'],
/tmp/ipykernel_8715/2051608433.py:23: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['n_genes_by_counts']\
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
/tmp/ipykernel_8715/2051608433.py:7: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['total_counts'],
/tmp/ipykernel_8715/2051608433.py:12: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['total_counts']\
/tmp/ipykernel_8715/2051608433.py:18: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['n_genes_by_counts'],
/tmp/ipykernel_8715/2051608433.py:23: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['n_genes_by_counts']\
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
/tmp/ipykernel_8715/2051608433.py:7: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['total_counts'],
/tmp/ipykernel_8715/2051608433.py:12: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['total_counts']\
/tmp/ipykernel_8715/2051608433.py:18: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['n_genes_by_counts'],
/tmp/ipykernel_8715/2051608433.py:23: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['n_genes_by_counts']\
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
/tmp/ipykernel_8715/2051608433.py:7: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['total_counts'],
/tmp/ipykernel_8715/2051608433.py:12: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['total_counts']\
/tmp/ipykernel_8715/2051608433.py:18: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['n_genes_by_counts'],
/tmp/ipykernel_8715/2051608433.py:23: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['n_genes_by_counts']\
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
/tmp/ipykernel_8715/2051608433.py:7: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['total_counts'],
/tmp/ipykernel_8715/2051608433.py:12: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['total_counts']\
/tmp/ipykernel_8715/2051608433.py:18: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['n_genes_by_counts'],
/tmp/ipykernel_8715/2051608433.py:23: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['n_genes_by_counts']\
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
/tmp/ipykernel_8715/2051608433.py:7: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['total_counts'],
/tmp/ipykernel_8715/2051608433.py:12: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['total_counts']\
/tmp/ipykernel_8715/2051608433.py:18: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['n_genes_by_counts'],
/tmp/ipykernel_8715/2051608433.py:23: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['n_genes_by_counts']\
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
/tmp/ipykernel_8715/2051608433.py:7: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['total_counts'],
/tmp/ipykernel_8715/2051608433.py:12: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['total_counts']\
/tmp/ipykernel_8715/2051608433.py:18: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['n_genes_by_counts'],
/tmp/ipykernel_8715/2051608433.py:23: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['n_genes_by_counts']\
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
/tmp/ipykernel_8715/2051608433.py:7: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['total_counts'],
/tmp/ipykernel_8715/2051608433.py:12: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['total_counts']\
/tmp/ipykernel_8715/2051608433.py:18: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['n_genes_by_counts'],
/tmp/ipykernel_8715/2051608433.py:23: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['n_genes_by_counts']\
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
/tmp/ipykernel_8715/2051608433.py:7: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['total_counts'],
/tmp/ipykernel_8715/2051608433.py:12: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['total_counts']\
/tmp/ipykernel_8715/2051608433.py:18: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['n_genes_by_counts'],
/tmp/ipykernel_8715/2051608433.py:23: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['n_genes_by_counts']\
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
/tmp/ipykernel_8715/2051608433.py:7: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['total_counts'],
/tmp/ipykernel_8715/2051608433.py:12: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['total_counts']\
/tmp/ipykernel_8715/2051608433.py:18: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['n_genes_by_counts'],
/tmp/ipykernel_8715/2051608433.py:23: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['n_genes_by_counts']\
```

Example 15 (python):
```python
# filter out low quality spots
sc.pp.filter_cells(adata_vis, min_counts = 500)
sc.pp.filter_cells(adata_vis, min_genes = 300)

# Spot number per slide
adata_vis.obs['sample'].value_counts()
```

Example 16 (unknown):
```unknown
HCAHeartST13162330    2806
HCAHeartST11605165    2429
HCAHeartST13162331    2338
HCAHeartST13162329    1777
HCAHeartST10317190    1337
HCAHeartST11605166    1321
HCAHeartST10298116    1219
HCAHeartST10298120    1170
HCAHeartST10298117    1126
HCAHeartST11605168    1082
HCAHeartST11605167    1044
HCAHeartST10298115    1025
HCAHeartST10317189    1004
HCAHeartST10317184     897
HCAHeartST10298114     872
HCAHeartST10317186     855
HCAHeartST10317185     824
HCAHeartST13162332     505
Name: sample, dtype: int64
```

Example 17 (python):
```python
# find mitochondria-encoded (MT) genes
adata_vis.var['MT_gene'] = [gene.startswith('MT-') for gene in adata_vis.var['SYMBOL']]
print(sum(adata_vis.var['MT_gene']))
    
# remove MT genes for spatial mapping (keeping their counts in the object)
adata_vis.obsm['MT'] = adata_vis[:, adata_vis.var['MT_gene'].values].X.toarray()
adata_vis = adata_vis[:, ~adata_vis.var['MT_gene'].values]
print(sum(adata_vis.var['MT_gene']))
```

Example 18 (unknown):
```unknown
meta.head()
```

Example 19 (python):
```python
adata_vis.obs = adata_vis.obs.reset_index().merge(meta, how='left', left_on='sample', right_on='sangerID').set_index('spot_id')
adata_vis
```

Example 20 (unknown):
```unknown
AnnData object with n_obs × n_vars = 23631 × 33525
    obs: 'in_tissue', 'array_row', 'array_col', 'sample', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'mt_frac', 'n_counts', 'n_genes', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'status', 'cell2loc-map_group'
    var: 'gene_ids', 'feature_types', 'genome', 'SYMBOL', 'mt', 'n_cells_by_counts-HCAHeartST10298114', 'mean_counts-HCAHeartST10298114', 'log1p_mean_counts-HCAHeartST10298114', 'pct_dropout_by_counts-HCAHeartST10298114', 'total_counts-HCAHeartST10298114', 'log1p_total_counts-HCAHeartST10298114', 'n_cells_by_counts-HCAHeartST10298115', 'mean_counts-HCAHeartST10298115', 'log1p_mean_counts-HCAHeartST10298115', 'pct_dropout_by_counts-HCAHeartST10298115', 'total_counts-HCAHeartST10298115', 'log1p_total_counts-HCAHeartST10298115', 'n_cells_by_counts-HCAHeartST10298116', 'mean_counts-HCAHeartST10298116', 'log1p_mean_counts-HCAHeartST10298116', 'pct_dropout_by_counts-HCAHeartST10298116', 'total_counts-HCAHeartST10298116', 'log1p_total_counts-HCAHeartST10298116', 'n_cells_by_counts-HCAHeartST10298117', 'mean_counts-HCAHeartST10298117', 'log1p_mean_counts-HCAHeartST10298117', 'pct_dropout_by_counts-HCAHeartST10298117', 'total_counts-HCAHeartST10298117', 'log1p_total_counts-HCAHeartST10298117', 'n_cells_by_counts-HCAHeartST10298120', 'mean_counts-HCAHeartST10298120', 'log1p_mean_counts-HCAHeartST10298120', 'pct_dropout_by_counts-HCAHeartST10298120', 'total_counts-HCAHeartST10298120', 'log1p_total_counts-HCAHeartST10298120', 'n_cells_by_counts-HCAHeartST10317184', 'mean_counts-HCAHeartST10317184', 'log1p_mean_counts-HCAHeartST10317184', 'pct_dropout_by_counts-HCAHeartST10317184', 'total_counts-HCAHeartST10317184', 'log1p_total_counts-HCAHeartST10317184', 'n_cells_by_counts-HCAHeartST10317185', 'mean_counts-HCAHeartST10317185', 'log1p_mean_counts-HCAHeartST10317185', 'pct_dropout_by_counts-HCAHeartST10317185', 'total_counts-HCAHeartST10317185', 'log1p_total_counts-HCAHeartST10317185', 'n_cells_by_counts-HCAHeartST10317186', 'mean_counts-HCAHeartST10317186', 'log1p_mean_counts-HCAHeartST10317186', 'pct_dropout_by_counts-HCAHeartST10317186', 'total_counts-HCAHeartST10317186', 'log1p_total_counts-HCAHeartST10317186', 'n_cells_by_counts-HCAHeartST10317189', 'mean_counts-HCAHeartST10317189', 'log1p_mean_counts-HCAHeartST10317189', 'pct_dropout_by_counts-HCAHeartST10317189', 'total_counts-HCAHeartST10317189', 'log1p_total_counts-HCAHeartST10317189', 'n_cells_by_counts-HCAHeartST10317190', 'mean_counts-HCAHeartST10317190', 'log1p_mean_counts-HCAHeartST10317190', 'pct_dropout_by_counts-HCAHeartST10317190', 'total_counts-HCAHeartST10317190', 'log1p_total_counts-HCAHeartST10317190', 'n_cells_by_counts-HCAHeartST11605165', 'mean_counts-HCAHeartST11605165', 'log1p_mean_counts-HCAHeartST11605165', 'pct_dropout_by_counts-HCAHeartST11605165', 'total_counts-HCAHeartST11605165', 'log1p_total_counts-HCAHeartST11605165', 'n_cells_by_counts-HCAHeartST11605166', 'mean_counts-HCAHeartST11605166', 'log1p_mean_counts-HCAHeartST11605166', 'pct_dropout_by_counts-HCAHeartST11605166', 'total_counts-HCAHeartST11605166', 'log1p_total_counts-HCAHeartST11605166', 'n_cells_by_counts-HCAHeartST11605167', 'mean_counts-HCAHeartST11605167', 'log1p_mean_counts-HCAHeartST11605167', 'pct_dropout_by_counts-HCAHeartST11605167', 'total_counts-HCAHeartST11605167', 'log1p_total_counts-HCAHeartST11605167', 'n_cells_by_counts-HCAHeartST11605168', 'mean_counts-HCAHeartST11605168', 'log1p_mean_counts-HCAHeartST11605168', 'pct_dropout_by_counts-HCAHeartST11605168', 'total_counts-HCAHeartST11605168', 'log1p_total_counts-HCAHeartST11605168', 'n_cells_by_counts-HCAHeartST13162329', 'mean_counts-HCAHeartST13162329', 'log1p_mean_counts-HCAHeartST13162329', 'pct_dropout_by_counts-HCAHeartST13162329', 'total_counts-HCAHeartST13162329', 'log1p_total_counts-HCAHeartST13162329', 'n_cells_by_counts-HCAHeartST13162330', 'mean_counts-HCAHeartST13162330', 'log1p_mean_counts-HCAHeartST13162330', 'pct_dropout_by_counts-HCAHeartST13162330', 'total_counts-HCAHeartST13162330', 'log1p_total_counts-HCAHeartST13162330', 'n_cells_by_counts-HCAHeartST13162331', 'mean_counts-HCAHeartST13162331', 'log1p_mean_counts-HCAHeartST13162331', 'pct_dropout_by_counts-HCAHeartST13162331', 'total_counts-HCAHeartST13162331', 'log1p_total_counts-HCAHeartST13162331', 'n_cells_by_counts-HCAHeartST13162332', 'mean_counts-HCAHeartST13162332', 'log1p_mean_counts-HCAHeartST13162332', 'pct_dropout_by_counts-HCAHeartST13162332', 'total_counts-HCAHeartST13162332', 'log1p_total_counts-HCAHeartST13162332', 'MT_gene'
    uns: 'spatial'
    obsm: 'spatial', 'MT'
```

Example 21 (python):
```python
adata_vis.obs.head()
```

Example 22 (python):
```python
# annotation
path_to_annot='/nfs/team205/heart/visium_annotations/foetal'
category_name='JC_edit'

# read in annotation csv file and concatenate
for i, ID in enumerate(sampleIDs):
    if f'{ID}_JC.csv' in os.listdir(path_to_annot):
    
        print(ID)
        annot_df = pd.read_csv(f'{path_to_annot}/{ID}_JC.csv')
        annot_df['spot_id']=ID+'_'+annot_df['Barcode']

        if i==0:
            annot_df_all=annot_df.copy()
        else:
            annot_df_all=pd.concat([annot_df_all,annot_df])
    else:
        print(f'{ID} is not annotated')

# add annotations to adata_vis
adata_vis.obs=pd.concat([adata_vis.obs,
                     annot_df_all.set_index('spot_id')[[category_name]].reindex(adata_vis.obs.index)],
                     axis=1)
# replace nan to 'none'
adata_vis.obs[category_name].fillna('none', inplace=True)

# minor modification
annotation_edit_dict = {
     'AV Ring':'AV ring',
    'Aorta':'Great vessels',
    'Coronary Artery':'Coronary artery',
    'Mitral Valve':'AV valves',
    'Mitral valve':'AV valves',
    'Myocardium - Ventricular - Trabecular':'Myocardium - Ventricular - Trabeculated',
    'Myocardim - Ventricular - Trabeculated':'Myocardium - Ventricular - Trabeculated',
    'Myocardim - Ventricular - Compact':'Myocardium - Ventricular - Compact',
    'unknown':'Unannotated',
    'Unknown':'Unannotated',
}
adata_vis.obs.replace({category_name:annotation_edit_dict}, inplace=True)
set(adata_vis.obs[category_name])
```

Example 23 (unknown):
```unknown
HCAHeartST10298114
HCAHeartST10298115
HCAHeartST10298116
HCAHeartST10298117
HCAHeartST10298120
HCAHeartST10317184
HCAHeartST10317185
HCAHeartST10317186
HCAHeartST10317189
HCAHeartST10317190
HCAHeartST11605165
HCAHeartST11605166
HCAHeartST11605167
HCAHeartST11605168
HCAHeartST13162329
HCAHeartST13162330
HCAHeartST13162331
HCAHeartST13162332
```

Example 24 (unknown):
```unknown
{'AV ring',
 'AV valves',
 'Aortic valve',
 'Cavity',
 'Connective tissue',
 'Coronary artery',
 'Epicardium',
 'Great vessels',
 'Myocardium - Atrial',
 'Myocardium - Papillary Muscle',
 'Myocardium - Ventricular - Compact',
 'Myocardium - Ventricular - Trabeculated',
 'Out_of_tissue',
 'Unannotated',
 'Valve apparatus',
 'none'}
```

Example 25 (python):
```python
# remove 'none' and 'no_tissue'
adata_vis = adata_vis[adata_vis.obs[category_name].isin(['none','Out_of_tissue'])==False]

# remove 'Cavity'
adata_vis = adata_vis[adata_vis.obs[category_name].isin(['Cavity'])==False]

# remove 'Unannotated'
adata_vis = adata_vis[adata_vis.obs[category_name].isin(['Unannotated'])==False]
```

Example 26 (python):
```python
# order annotations
annotations_order = ['Myocardium - Atrial',
                     'Myocardium - Ventricular - Compact',
                     'Myocardium - Ventricular - Trabeculated',
                    'Myocardium - Papillary Muscle',
                    'Epicardium',
                     'AV valves',
                     'Aortic valve',
                    'Valve apparatus',
                   'AV ring',
                    'Connective tissue',
                     'Great vessels',
                     'Coronary artery',
                     # 'Nerve',
                     ]
adata_vis.obs[category_name] = adata_vis.obs[category_name].astype('category')
adata_vis.obs[category_name].cat.reorder_categories(annotations_order,inplace=True)
```

Example 27 (python):
```python
/tmp/ipykernel_8715/3572501826.py:16: ImplicitModificationWarning: Trying to modify attribute `.obs` of view, initializing view as actual.
  adata_vis.obs[category_name] = adata_vis.obs[category_name].astype('category')
/tmp/ipykernel_8715/3572501826.py:17: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
  adata_vis.obs[category_name].cat.reorder_categories(annotations_order,inplace=True)
```

Example 28 (python):
```python
pd.crosstab(adata_vis.obs['cell2loc-map_group'],adata_vis.obs[category_name])
```

Example 29 (python):
```python
# check the slides for 'PCW-13w'
mask = adata_vis.obs['age']=='PCW-13w'
adata_vis[mask].obs['sample'].value_counts()
# 'HCAHeartST11605166': Atrium
# 'HCAHeartST11605165': Ventricle
```

Example 30 (unknown):
```unknown
HCAHeartST11605165    2428
HCAHeartST11605166    1120
Name: sample, dtype: int64
```

Example 31 (python):
```python
# combine
adata_vis.obs['sample_for_cell2loc']=adata_vis.obs['sample'].astype('str')
adata_vis.obs.replace({'sample_for_cell2loc':{
    'HCAHeartST11605165':'HCAHeartST11605165_HCAHeartST11605166',
    'HCAHeartST11605166':'HCAHeartST11605165_HCAHeartST11605166'
}},inplace=True)
adata_vis.obs['sample_for_cell2loc'].value_counts()
```

Example 32 (unknown):
```unknown
HCAHeartST11605165_HCAHeartST11605166    3548
HCAHeartST13162330                       2806
HCAHeartST13162331                       2338
HCAHeartST13162329                       1623
HCAHeartST10298116                       1187
HCAHeartST10298120                       1108
HCAHeartST10298117                       1085
HCAHeartST10298115                        991
HCAHeartST10317189                        944
HCAHeartST10317184                        836
HCAHeartST10298114                        812
HCAHeartST10317185                        704
HCAHeartST10317186                        693
HCAHeartST11605167                        692
HCAHeartST11605168                        639
HCAHeartST10317190                        609
HCAHeartST13162332                        413
Name: sample_for_cell2loc, dtype: int64
```

Example 33 (python):
```python
adata_vis.obs.columns
```

Example 34 (unknown):
```unknown
Index(['in_tissue', 'array_row', 'array_col', 'sample', 'n_genes_by_counts',
       'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts',
       'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes',
       'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'mt_frac',
       'n_counts', 'n_genes', 'sangerID', 'combinedID', 'donor', 'donor_type',
       'region', 'region_finest', 'age', 'gender', 'facility',
       'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'status',
       'cell2loc-map_group', 'JC_edit', 'sample_for_cell2loc'],
      dtype='object')
```

Example 35 (unknown):
```unknown
set(meta['cell2loc-map_group'])
```

Example 36 (unknown):
```unknown
{'13PCW', '20PCW', '4PCW', '5PCW', '7PCW'}
```

Example 37 (python):
```python
# convert adata_vis.var_names to gene_id
adata_vis.var.set_index('gene_ids',inplace=True)
adata_vis.var.head()
```

Example 38 (python):
```python
%%time
import cell2location
# read in reference signatures
inf_aver = pd.read_csv(f"{ref_run_name}/all_data/inf_aver.csv", index_col=0)

# find shared genes and subset both anndata and reference signatures
intersect = np.intersect1d(adata_vis.var_names, inf_aver.index)
adata_vis = adata_vis[:, intersect].copy()
inf_aver = inf_aver.loc[intersect, :].copy()
print(inf_aver.shape)

for group in set(meta['cell2loc-map_group']):
    print(f'####### {group} #######')
    
    # subset adata
    adata_vis_sub=adata_vis[adata_vis.obs['cell2loc-map_group']==group].copy()
    samples = list(set(adata_vis_sub.obs['sample_for_cell2loc']))
    samples.sort()
    print(f'samples in this run: {samples}')
    adata_vis_sub=adata_vis_sub.copy()
    # prepare anndata for cell2location model
    cell2location.models.Cell2location.setup_anndata(adata=adata_vis_sub, batch_key="sample_for_cell2loc")
    #scvi.data.setup_anndata(adata=adata_vis_sub, batch_key="sample_for_cell2loc")
    #scvi.data.view_anndata_setup(adata_vis_sub)

    # create and train the model
    mod = cell2location.models.Cell2location(
        adata_vis_sub, cell_state_df=inf_aver, 
        # the expected average cell abundance: tissue-dependent 
        # hyper-prior which can be estimated from paired histology:
        N_cells_per_location=20,
        # hyperparameter controlling normalisation of
        # within-experiment variation in RNA detection (using default here):
        detection_alpha=20
    ) 

    mod.train(max_epochs=20000, 
              # train using full data (batch_size=None)
              batch_size=None, 
              # use all data points in training because 
              # we need to estimate cell abundance at all locations
              train_size=1,
              use_gpu=True)
    mod.view_anndata_setup()
    # plot ELBO loss history during training, removing first 100 epochs from the plot
    mod.plot_history(1000)
    plt.legend(labels=['full data training']);

    # In this section, we export the estimated cell abundance (summary of the posterior distribution).
    adata_vis_sub = mod.export_posterior(
        adata_vis_sub, sample_kwargs={'num_samples': 1000, 'batch_size': mod.adata.n_obs, 'use_gpu': True}
    )

    # Save model
    mod.save(f"{run_name}/{group}", overwrite=True)

    # Save anndata object with results
    adata_vis_sub.write(f"{run_name}/{group}/sp.h5ad")

    # Examine reconstruction accuracy to assess if there are any issues with mapping
    # the plot should be roughly diagonal, strong deviations will signal problems
    mod.plot_QC()
    
    del mod,adata_vis_sub
    gc.collect()
    torch.cuda.empty_cache()
```

Example 39 (unknown):
```unknown
(17072, 64)
####### 5PCW #######
samples in this run: ['HCAHeartST11605167', 'HCAHeartST11605168']
```

Example 40 (unknown):
```unknown
No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.)
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/pytorch_lightning/trainer/configuration_validator.py:106: UserWarning: You passed in a `val_dataloader` but have no `validation_step`. Skipping val loop.
  rank_zero_warn("You passed in a `val_dataloader` but have no `validation_step`. Skipping val loop.")
You are using a CUDA device ('NVIDIA A100-SXM4-80GB') that has Tensor Cores. To properly utilize them, you should set `torch.set_float32_matmul_precision('medium' | 'high')` which will trade-off precision for performance. For more details, read https://pytorch.org/docs/stable/generated/torch.set_float32_matmul_precision.html#torch.set_float32_matmul_precision
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/pytorch_lightning/trainer/trainer.py:1609: PossibleUserWarning: The number of training batches (1) is smaller than the logging interval Trainer(log_every_n_steps=10). Set a lower value for log_every_n_steps if you want to see logs for the training epoch.
  rank_zero_warn(
```

Example 41 (unknown):
```unknown
Epoch 1511/20000:   8%|███████████▍                                                                                                                                           | 1511/20000 [00:58<11:30, 26.77it/s, v_num=1, elbo_train=1.84e+7]
```

---

## Functions¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/Visium/cell2location/wholeheart_without_DA/no2_mapping_FFPE.html

**Contents:**
  - Loading packages¶
- Functions¶
- Read in metadata¶
- Data, parameters¶
- Loading visium data¶
- Remove mitochondrial genes¶
- Run cell2location mapping¶

**Examples:**

Example 1 (python):
```python
import sys
import scanpy as sc
import anndata
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt 
import matplotlib as mpl

import cell2location
import scvi

from matplotlib import rcParams
rcParams['pdf.fonttype'] = 42 # enables correct plotting of text
import seaborn as sns

import os
```

Example 2 (unknown):
```unknown
Global seed set to 0
```

Example 3 (python):
```python
def read_and_qc(sample_name, path):
    r""" This function reads the data for one 10X spatial experiment into the anndata object.
    It also calculates QC metrics. Modify this function if required by your workflow.

    :param sample_name: Name of the sample
    :param path: path to data
    """

    adata = sc.read_visium(path + str(sample_name),
                           count_file='filtered_feature_bc_matrix.h5', load_images=True)
    adata.var_names_make_unique(join='-')
    
    adata.obs['sample'] = sample_name
    
    # rename genes to ENSEMBL
    adata.var['SYMBOL'] = adata.var_names
    adata.var_names = adata.var['gene_ids']
    adata.var_names.name = None

    # Calculate QC metrics
    sc.pp.calculate_qc_metrics(adata, inplace=True)
    adata.var['mt'] = [gene.startswith('MT-') for gene in adata.var['SYMBOL']]
    adata.obs['mt_frac'] = adata[:, adata.var['mt'].tolist()].X.sum(1).A.squeeze()/adata.obs['total_counts']

    # add sample name to obs names
    adata.obs["sample"] = [str(i) for i in adata.obs['sample']]
    adata.obs_names = adata.obs["sample"] \
                          + '_' + adata.obs_names
    adata.obs.index.name = 'spot_id'

    return adata

def select_slide(adata, s, s_col='sample'):
    r""" This function selects the data for one slide from the spatial anndata object.

    :param adata: Anndata object with multiple spatial experiments
    :param s: name of selected experiment
    :param s_col: column in adata.obs listing experiment name for each location
    """

    slide = adata[adata.obs[s_col].isin([s]), :]
    s_keys = list(slide.uns['spatial'].keys())
    s_spatial = np.array(s_keys)[[s in k for k in s_keys]][0]

    slide.uns['spatial'] = {s_spatial: slide.uns['spatial'][s_spatial]}

    return slide
```

Example 4 (python):
```python
meta=pd.read_csv('/nfs/team205/heart/anndata_objects/Foetal/metadata/HeartTeamSamples_Mappeddata_20230208.csv',
                index_col=0)
meta=meta[['Publication','sangerID', 'combinedID',
       'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender',
       'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'status']]

# select slides for foetal publication
meta=meta[meta['Publication'].isin(['Foetal'])]
# select only FFPE slides
meta=meta[meta['modality'].isin(['Visium-FFPE'])]
# select only Healthy slides
meta=meta[meta['status'].isin(['Healthy'])]

meta = meta.sort_values(['Publication','sangerID'])
```

Example 5 (unknown):
```unknown
sampleIDs = list(meta['sangerID'])
len(sampleIDs)
```

Example 6 (unknown):
```unknown
results_folder = '/nfs/team205/heart/cell2location/foetal'

# create paths and names to results folders for reference regression and cell2location models
ref_run_name = f'{results_folder}/reference_signatures/ver28Feb2023'
run_name = f'{results_folder}/cell2location_map/ver28Feb2023'
```

Example 7 (unknown):
```unknown
visium_path='/nfs/team205/heart/mapped/spaceranger200/'

celltype_key='fine_grain'
```

Example 8 (python):
```python
# Read the data of the interested region into anndata objects
# rename spatial/tissue_positions.csv as spatial/tissue_positions_list.csv
# Read the data of the interested region into anndata objects
slides = []
for i in sampleIDs:
    slides.append(read_and_qc(i, path=visium_path))

# Combine anndata objects together
adata_vis = slides[0].concatenate(
    slides[1:],
    batch_key="sample",
    uns_merge="unique",
    batch_categories=sampleIDs,
    index_unique=None
)
adata_vis
```

Example 9 (unknown):
```unknown
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
```

Example 10 (unknown):
```unknown
AnnData object with n_obs × n_vars = 23294 × 18085
    obs: 'in_tissue', 'array_row', 'array_col', 'sample', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'mt_frac'
    var: 'gene_ids', 'feature_types', 'genome', 'SYMBOL', 'mt', 'n_cells_by_counts-BHF_F_Hea13295830', 'mean_counts-BHF_F_Hea13295830', 'log1p_mean_counts-BHF_F_Hea13295830', 'pct_dropout_by_counts-BHF_F_Hea13295830', 'total_counts-BHF_F_Hea13295830', 'log1p_total_counts-BHF_F_Hea13295830', 'n_cells_by_counts-BHF_F_Hea13295831', 'mean_counts-BHF_F_Hea13295831', 'log1p_mean_counts-BHF_F_Hea13295831', 'pct_dropout_by_counts-BHF_F_Hea13295831', 'total_counts-BHF_F_Hea13295831', 'log1p_total_counts-BHF_F_Hea13295831'
    uns: 'spatial'
    obsm: 'spatial'
```

Example 11 (python):
```python
# filter out low quality spots
sc.pp.filter_cells(adata_vis, min_counts = 5000)
sc.pp.filter_cells(adata_vis, min_genes = 1000)

# Spot number per slide
adata_vis.obs['sample'].value_counts()
```

Example 12 (unknown):
```unknown
BHF_F_Hea13295830    11111
BHF_F_Hea13295831     8862
Name: sample, dtype: int64
```

Example 13 (python):
```python
# find mitochondria-encoded (MT) genes
adata_vis.var['MT_gene'] = [gene.startswith('MT-') for gene in adata_vis.var['SYMBOL']]
print(sum(adata_vis.var['MT_gene']))
    
# remove MT genes for spatial mapping (keeping their counts in the object)
adata_vis.obsm['MT'] = adata_vis[:, adata_vis.var['MT_gene'].values].X.toarray()
adata_vis = adata_vis[:, ~adata_vis.var['MT_gene'].values]
print(sum(adata_vis.var['MT_gene']))
```

Example 14 (python):
```python
# convert adata_vis.var_names to symbol
adata_vis.var.set_index('SYMBOL',inplace=True)
```

Example 15 (python):
```python
%%time


inf_aver = pd.read_csv(f"{ref_run_name}/whole_woPeriCell_woDASMC/inf_aver.csv", index_col=0)

# find shared genes and subset both anndata and reference signatures
intersect = np.intersect1d(adata_vis.var_names, inf_aver.index)
adata_vis = adata_vis[:, intersect].copy()
inf_aver = inf_aver.loc[intersect, :].copy()
print(f'shape of inf_aver: {inf_aver.shape}')

print('reference cell types')
print(inf_aver.columns)

# subset adata
adata_vis=adata_vis.copy()
# prepare anndata for cell2location model
scvi.data.setup_anndata(adata=adata_vis, batch_key="sample")
scvi.data.view_anndata_setup(adata_vis)

# create and train the model
mod = cell2location.models.Cell2location(
    adata_vis, cell_state_df=inf_aver, 
    # the expected average cell abundance: tissue-dependent 
    # hyper-prior which can be estimated from paired histology:
    N_cells_per_location=20,
    # hyperparameter controlling normalisation of
    # within-experiment variation in RNA detection (using default here):
    detection_alpha=20
) 

mod.train(max_epochs=20000, 
          # train using full data (batch_size=None)
          batch_size=None, 
          # use all data points in training because 
          # we need to estimate cell abundance at all locations
          train_size=1,
          use_gpu=True)

# plot ELBO loss history during training, removing first 100 epochs from the plot
mod.plot_history(1000)
plt.legend(labels=['full data training']);

# In this section, we export the estimated cell abundance (summary of the posterior distribution).
adata_vis = mod.export_posterior(
    adata_vis, sample_kwargs={'num_samples': 1000, 'batch_size': mod.adata.n_obs, 'use_gpu': True}
)

# Save model
mod.save(f"{run_name}/FFPE_16PCW", overwrite=True)

# Save anndata object with results
adata_vis.write(f"{run_name}/FFPE_16PCW/sp.h5ad")

# Examine reconstruction accuracy to assess if there are any issues with mapping
# the plot should be roughly diagonal, strong deviations will signal problems
mod.plot_QC()
```

Example 16 (python):
```python
shape of inf_aver: (13540, 57)
reference cell types
Index(['AtrialCardiomyocytesLeft', 'AtrialCardiomyocytesRight',
       'VentricularCardiomyocytesLeftCompact',
       'VentricularCardiomyocytesRightCompact',
       'VentricularCardiomyocytesLeftTrabeculated',
       'VentricularCardiomyocytesRightTrabeculated',
       'SinoatrialNodeCardiomyocytes', 'AtrioventricularNodeCardiomyocytes',
       'VentricularConductionSystemProximal',
       'VentricularConductionSystemDistal',
       'GreatVesselAdventitialFibroblasts',
       'CoronaryVesselAdventitialFibroblasts',
       'MyocardialInterstitialFibroblasts1',
       'MyocardialInterstitialFibroblasts2', 'Myofibroblasts',
       'LymphNodeFibroblasticReticularCells', 'ValveInterstitialCells',
       'GreatVesselSmoothMuscleCells', 'CoronarySmoothMuscleCells',
       'CoronaryPericytes', 'GreatVesselArterialEndothelialCells',
       'GreatVesselVenousEndothelialCells', 'CoronaryArterialEndothelialCells',
       'CoronaryVenousEndothelialCells', 'CoronaryCapillaryEndothelialCells',
       'EndocardialCells', 'EndocardialCushionCells', 'ValveEndothelialCells',
       'LymphaticEndothelialCells', 'MesothelialEpicardialCells',
       'EpicardiumDerivedCells', 'NeuronPrecursors', 'ChromaffinCells',
       'SympatheticNeurons', 'ParaympatheticNeurons', 'SchwannCellPrecursors',
       'SchwannCells', 'MonocytesMPOpos', 'Monocytes', 'MonocyteDerivedCells',
       'MacrophagesCX3CR1pos', 'MacrophagesTIMD4pos', 'MacrophagesLYVE1pos',
       'MacrophagesATF3pos', 'DendriticCellsType1', 'DendriticCellsMature',
       'PlasmacytoidDendriticCells', 'MastCells', 'Megakaryocytes',
       'TCellsCD4pos', 'TCellsCD8pos', 'TregsCD4pos', 'ProBCells', 'BCells',
       'BCellsMS4A1pos', 'NaturalKillerCells', 'InnateLymphoidCells'],
      dtype='object')
INFO     Using batches from adata.obs["sample"]                                              
INFO     No label_key inputted, assuming all cells have same label                           
INFO     Using data from adata.X
```

Example 17 (unknown):
```unknown
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function setup_anndata is deprecated; Please use the model-specific setup_anndata methods instead. The global method will be removed in version 0.15.0.
```

Example 18 (python):
```python
INFO     Successfully registered anndata object containing 19973 cells, 13540 vars, 2        
         batches, 1 labels, and 0 proteins. Also registered 0 extra categorical covariates   
         and 0 extra continuous covariates.                                                  
INFO     Please do not further modify adata until model is trained.
```

Example 19 (unknown):
```unknown
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function view_anndata_setup is deprecated; This method will be removed in 0.15.0. Please avoid building any new dependencies on it.
```

Example 20 (unknown):
```unknown
Anndata setup with scvi-tools version 0.14.4.
```

Example 21 (unknown):
```unknown
Data Summary              
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┓
┃             Data             ┃ Count ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━┩
│            Cells             │ 19973 │
│             Vars             │ 13540 │
│            Labels            │   1   │
│           Batches            │   2   │
│           Proteins           │   0   │
│ Extra Categorical Covariates │   0   │
│ Extra Continuous Covariates  │   0   │
└──────────────────────────────┴───────┘
```

Example 22 (python):
```python
SCVI Data Registry              
┏━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃     Data      ┃    scvi-tools Location    ┃
┡━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│       X       │          adata.X          │
│ batch_indices │ adata.obs['_scvi_batch']  │
│    labels     │ adata.obs['_scvi_labels'] │
└───────────────┴───────────────────────────┘
```

Example 23 (python):
```python
Label Categories                        
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 24 (python):
```python
Batch Categories                         
┏━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃   Source Location   ┃    Categories     ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['sample'] │ BHF_F_Hea13295830 │          0          │
│                     │ BHF_F_Hea13295831 │          1          │
└─────────────────────┴───────────────────┴─────────────────────┘
```

Example 25 (unknown):
```unknown
GPU available: True, used: True
TPU available: False, using: 0 TPU cores
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/pytorch_lightning/trainer/configuration_validator.py:99: UserWarning: you passed in a val_dataloader but have no validation_step. Skipping val loop
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
```

Example 26 (unknown):
```unknown
Epoch 20000/20000: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20000/20000 [1:34:26<00:00,  3.53it/s, v_num=1, elbo_train=2.93e+8]
Sampling local variables, batch: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [01:08<00:00, 68.84s/it]
Sampling global variables, sample: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 999/999 [00:57<00:00, 17.35it/s]
```

Example 27 (unknown):
```unknown
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'in_tissue' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'array_row' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'array_col' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'feature_types' as categorical
/lustre/scratch117/cellgen/team205/kk18/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1228: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
... storing 'genome' as categorical
```

Example 28 (unknown):
```unknown
CPU times: user 1h 16min 33s, sys: 22min 38s, total: 1h 39min 12s
Wall time: 1h 39min 37s
```

---

## Read in ArchR project¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/ATAC/ATAC_preprocess/ArchR_no3_peak-calling.html

**Contents:**
- Read in ArchR project¶
- Check cell number per sample¶
- Create one "Sample" label¶
- Create Pseudo-bulk Replicates¶
- Calling peaks¶
- Add Peak matrix¶
- Estract peakset as Granges object¶

Ref: https://www.archrproject.com/bookdown/creating-an-archrproject-1.html

**Examples:**

Example 1 (unknown):
```unknown
library(ArchR)
library(tidyverse)
library(BSgenome.Hsapiens.UCSC.hg38)
```

Example 2 (python):
```python
/ |
                                                 /    \
            .                                  /      |.
            \\\                              /        |.
              \\\                          /           `|.
                \\\                      /              |.
                  \                    /                |\
                  \\#####\           /                  ||
                ==###########>      /                   ||
                 \\##==......\    /                     ||
            ______ =       =|__ /__                     ||      \\\
        ,--' ,----`-,__ ___/'  --,-`-===================##========>
       \               '        ##_______ _____ ,--,__,=##,__   ///
        ,    __==    ___,-,__,--'#'  ==='      `-'    | ##,-/
        -,____,---'       \\####\\________________,--\\_##,/
           ___      .______        ______  __    __  .______      
          /   \     |   _  \      /      ||  |  |  | |   _  \     
         /  ^  \    |  |_)  |    |  ,----'|  |__|  | |  |_)  |    
        /  /_\  \   |      /     |  |     |   __   | |      /     
       /  _____  \  |  |\  \\___ |  `----.|  |  |  | |  |\  \\___.
      /__/     \__\ | _| `._____| \______||__|  |__| | _| `._____|
    

ArchR : Version 1.0.2
For more information see our website : www.ArchRProject.com
If you encounter a bug please report : https://github.com/GreenleafLab/ArchR/issues

Loading Required Packages...

	Loading Package : grid v4.0.4

	Loading Package : gridExtra v2.3

	Loading Package : gtools v3.9.3

	Loading Package : gtable v0.3.0

	Loading Package : ggplot2 v3.4.0

	Loading Package : magrittr v2.0.3

	Loading Package : plyr v1.8.8

	Loading Package : stringr v1.4.0

	Loading Package : data.table v1.14.4

	Loading Package : matrixStats v0.62.0

	Loading Package : S4Vectors v0.28.1

	Loading Package : GenomicRanges v1.42.0

	Loading Package : BiocGenerics v0.36.1

	Loading Package : Matrix v1.3.2

	Loading Package : Rcpp v1.0.9

	Loading Package : SummarizedExperiment v1.20.0

	Loading Package : rhdf5 v2.34.0

Warning message in system("timedatectl", intern = TRUE):
“running command 'timedatectl' had status 1”
── Attaching packages ─────────────────────────────────────── tidyverse 1.3.2 ──
✔ tibble  3.1.8      ✔ purrr   0.3.4 
✔ tidyr   1.2.1      ✔ dplyr   1.0.10
✔ readr   2.1.3      ✔ forcats 0.5.1 
── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::arrange()         masks plyr::arrange()
✖ dplyr::between()         masks data.table::between()
✖ dplyr::collapse()        masks IRanges::collapse()
✖ dplyr::combine()         masks Biobase::combine(), BiocGenerics::combine(), gridExtra::combine()
✖ purrr::compact()         masks plyr::compact()
✖ dplyr::count()           masks matrixStats::count(), plyr::count()
✖ dplyr::desc()            masks IRanges::desc(), plyr::desc()
✖ tidyr::expand()          masks Matrix::expand(), S4Vectors::expand()
✖ tidyr::extract()         masks magrittr::extract()
✖ dplyr::failwith()        masks plyr::failwith()
✖ dplyr::filter()          masks stats::filter()
✖ dplyr::first()           masks S4Vectors::first(), data.table::first()
✖ dplyr::id()              masks plyr::id()
✖ dplyr::lag()             masks stats::lag()
✖ dplyr::last()            masks data.table::last()
✖ dplyr::mutate()          masks plyr::mutate()
✖ tidyr::pack()            masks Matrix::pack()
✖ BiocGenerics::Position() masks ggplot2::Position(), base::Position()
✖ purrr::reduce()          masks GenomicRanges::reduce(), IRanges::reduce()
✖ dplyr::rename()          masks S4Vectors::rename(), plyr::rename()
✖ purrr::set_names()       masks magrittr::set_names()
✖ dplyr::slice()           masks IRanges::slice()
✖ dplyr::summarise()       masks plyr::summarise()
✖ dplyr::summarize()       masks plyr::summarize()
✖ purrr::transpose()       masks data.table::transpose()
✖ tidyr::unpack()          masks Matrix::unpack()
Loading required package: BSgenome

Loading required package: Biostrings

Loading required package: XVector


Attaching package: ‘XVector’


The following object is masked from ‘package:purrr’:

    compact


The following object is masked from ‘package:plyr’:

    compact



Attaching package: ‘Biostrings’


The following object is masked from ‘package:base’:

    strsplit


Loading required package: rtracklayer
```

Example 3 (unknown):
```unknown
setwd('/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR')
getwd()
```

Example 4 (unknown):
```unknown
# before starting a project we must set the ArchRGenome and default threads for parallelization.
# Setting default genome to Hg38.
addArchRGenome("hg38")
```

Example 5 (unknown):
```unknown
Setting default genome to Hg38.
```

Example 6 (unknown):
```unknown
# Setting default number of Parallel threads to 16
addArchRThreads(threads = 10)
```

Example 7 (unknown):
```unknown
Setting default number of Parallel threads to 10.
```

Example 8 (unknown):
```unknown
archr_project_path = '/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output'
proj = loadArchRProject(path = archr_project_path, showLogo = FALSE)
proj
```

Example 9 (unknown):
```unknown
Successfully loaded ArchRProject!


           ___      .______        ______  __    __  .______      
          /   \     |   _  \      /      ||  |  |  | |   _  \     
         /  ^  \    |  |_)  |    |  ,----'|  |__|  | |  |_)  |    
        /  /_\  \   |      /     |  |     |   __   | |      /     
       /  _____  \  |  |\  \\___ |  `----.|  |  |  | |  |\  \\___.
      /__/     \__\ | _| `._____| \______||__|  |__| | _| `._____|
```

Example 10 (unknown):
```unknown
class: ArchRProject 
outputDirectory: /nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output 
samples(37): BHF_F_Hea11064670_BHF_F_Hea11031823
  BHF_F_Hea11064671_BHF_F_Hea11031824 ...
  HCAHeartST13386009_HCAHeartST13303419
  HCAHeartST13386010_HCAHeartST13303420
sampleColData names(1): ArrowFiles
cellColData names(56): Sample TSSEnrichment ... hb1_score Clusters
numberOfCells(1): 167022
medianTSS(1): 11.614
medianFrags(1): 10338.5
```

Example 11 (unknown):
```unknown
# check
library(repr)
options(repr.plot.width=15, repr.plot.height=15)

cM <- confusionMatrix(paste0(proj$Sample), paste0(proj$fine_grain))
df = t(as.data.frame(as.matrix(cM)))
colnames(df) = c(1:ncol(df)) 
df
```

Example 12 (unknown):
```unknown
# check
library(repr)
options(repr.plot.width=15, repr.plot.height=15)

cM <- confusionMatrix(paste0(proj$Sample), paste0(proj$week))
df = t(as.data.frame(as.matrix(cM)))
colnames(df) = c(1:ncol(df)) 
df
```

Example 13 (unknown):
```unknown
library(repr)
options(repr.plot.width=15, repr.plot.height=15)

cM <- confusionMatrix(paste0(proj$fine_grain), paste0(proj$week))
df = t(as.data.frame(as.matrix(cM)))
# colnames(df) = c(1:ncol(df)) 
t(df)
```

Example 14 (unknown):
```unknown
proj$Sample2 = "1"
```

Example 15 (unknown):
```unknown
proj = addGroupCoverages(ArchRProj = proj, groupBy = "fine_grain", sampleLabels = "Sample2",
                        minReplicates = 3,
                        maxReplicates = 3,
                         minCells = 40,
                      maxCells = 500,
                        force = TRUE)
# maxReplicate = sample number
```

Example 16 (unknown):
```unknown
ArchR logging to : ArchRLogs/ArchR-addGroupCoverages-ab73c00ce4-Date-2023-03-09_Time-18-45-28.log
If there is an issue, please report to github with logFile!

AtrialCardiomyocytesCycling (1 of 63) : CellGroups N = 2

AtrialCardiomyocytesLeft (2 of 63) : CellGroups N = 2

AtrialCardiomyocytesRight (3 of 63) : CellGroups N = 2

AtrioventricularNodeCardiomyocytes (4 of 63) : CellGroups N = 2

BCells (5 of 63) : CellGroups N = 2

BCellsMS4A1pos (6 of 63) : CellGroups N = 2

ChromaffinCells (7 of 63) : CellGroups N = 3

CoronaryArterialEndothelialCells (8 of 63) : CellGroups N = 2

CoronaryCapillaryEndothelialCells (9 of 63) : CellGroups N = 2

CoronaryPericytes (10 of 63) : CellGroups N = 2

CoronarySmoothMuscleCells (11 of 63) : CellGroups N = 2

CoronaryVenousEndothelialCells (12 of 63) : CellGroups N = 2

CoronaryVesselAdventitialFibroblasts (13 of 63) : CellGroups N = 2

DendriticCellsMature (14 of 63) : CellGroups N = 2

DendriticCellsType1 (15 of 63) : CellGroups N = 2

DuctusArteriosusSmoothMuscleCells (16 of 63) : CellGroups N = 2

EndocardialCells (17 of 63) : CellGroups N = 2

EndocardialCushionCells (18 of 63) : CellGroups N = 2

EpicardiumDerivedCells (19 of 63) : CellGroups N = 2

GreatVesselAdventitialFibroblasts (20 of 63) : CellGroups N = 2

GreatVesselArterialEndothelialCells (21 of 63) : CellGroups N = 2

GreatVesselSmoothMuscleCells (22 of 63) : CellGroups N = 2

GreatVesselVenousEndothelialCells (23 of 63) : CellGroups N = 2

InnateLymphoidCells (24 of 63) : CellGroups N = 2

LymphaticEndothelialCells (25 of 63) : CellGroups N = 2

LymphNodeFibroblasticReticularCells (26 of 63) : CellGroups N = 2

MacrophagesATF3pos (27 of 63) : CellGroups N = 2

MacrophagesCX3CR1pos (28 of 63) : CellGroups N = 2

MacrophagesLYVE1pos (29 of 63) : CellGroups N = 2

MacrophagesTIMD4pos (30 of 63) : CellGroups N = 2

MastCells (31 of 63) : CellGroups N = 2

Megakaryocytes (32 of 63) : CellGroups N = 3

MesothelialEpicardialCells (33 of 63) : CellGroups N = 2

MonocyteDerivedCells (34 of 63) : CellGroups N = 2

Monocytes (35 of 63) : CellGroups N = 3

MonocytesMPOpos (36 of 63) : CellGroups N = 3

MyocardialInterstitialFibroblasts1 (37 of 63) : CellGroups N = 2

MyocardialInterstitialFibroblasts2 (38 of 63) : CellGroups N = 2

Myofibroblasts (39 of 63) : CellGroups N = 2

NaturalKillerCells (40 of 63) : CellGroups N = 2

NeuronPrecursors (41 of 63) : CellGroups N = 2

ParaympatheticNeurons (42 of 63) : CellGroups N = 2

PericardialCells1 (43 of 63) : CellGroups N = 2

PericardialCells2 (44 of 63) : CellGroups N = 2

PericardialCells3 (45 of 63) : CellGroups N = 2

PlasmacytoidDendriticCells (46 of 63) : CellGroups N = 2

ProBCells (47 of 63) : CellGroups N = 2

SchwannCellPrecursors (48 of 63) : CellGroups N = 2

SchwannCells (49 of 63) : CellGroups N = 2

SinoatrialNodeCardiomyocytes (50 of 63) : CellGroups N = 2

SympatheticNeurons (51 of 63) : CellGroups N = 2

TCellsCD4pos (52 of 63) : CellGroups N = 2

TCellsCD8pos (53 of 63) : CellGroups N = 2

TregsCD4pos (54 of 63) : CellGroups N = 2

ValveEndothelialCells (55 of 63) : CellGroups N = 2

ValveInterstitialCells (56 of 63) : CellGroups N = 2

VentricularCardiomyocytesCycling (57 of 63) : CellGroups N = 2

VentricularCardiomyocytesLeftCompact (58 of 63) : CellGroups N = 2

VentricularCardiomyocytesLeftTrabeculated (59 of 63) : CellGroups N = 2

VentricularCardiomyocytesRightCompact (60 of 63) : CellGroups N = 2

VentricularCardiomyocytesRightTrabeculated (61 of 63) : CellGroups N = 2

VentricularConductionSystemDistal (62 of 63) : CellGroups N = 2

VentricularConductionSystemProximal (63 of 63) : CellGroups N = 2

2023-03-09 18:47:08 : Creating Coverage Files!, 1.658 mins elapsed.

2023-03-09 18:47:08 : Batch Execution w/ safelapply!, 1.658 mins elapsed.

2023-03-09 20:44:56 : Adding Kmer Bias to Coverage Files!, 119.458 mins elapsed.

Completed Kmer Bias Calculation

Adding Kmer Bias (1 of 130)

Adding Kmer Bias (2 of 130)

Adding Kmer Bias (3 of 130)

Adding Kmer Bias (4 of 130)

Adding Kmer Bias (5 of 130)

Adding Kmer Bias (6 of 130)

Adding Kmer Bias (7 of 130)

Adding Kmer Bias (8 of 130)

Adding Kmer Bias (9 of 130)

Adding Kmer Bias (10 of 130)

Adding Kmer Bias (11 of 130)

Adding Kmer Bias (12 of 130)

Adding Kmer Bias (13 of 130)

Adding Kmer Bias (14 of 130)

Adding Kmer Bias (15 of 130)

Adding Kmer Bias (16 of 130)

Adding Kmer Bias (17 of 130)

Adding Kmer Bias (18 of 130)

Adding Kmer Bias (19 of 130)

Adding Kmer Bias (20 of 130)

Adding Kmer Bias (21 of 130)

Adding Kmer Bias (22 of 130)

Adding Kmer Bias (23 of 130)

Adding Kmer Bias (24 of 130)

Adding Kmer Bias (25 of 130)

Adding Kmer Bias (26 of 130)

Adding Kmer Bias (27 of 130)

Adding Kmer Bias (28 of 130)

Adding Kmer Bias (29 of 130)

Adding Kmer Bias (30 of 130)

Adding Kmer Bias (31 of 130)

Adding Kmer Bias (32 of 130)

Adding Kmer Bias (33 of 130)

Adding Kmer Bias (34 of 130)

Adding Kmer Bias (35 of 130)

Adding Kmer Bias (36 of 130)

Adding Kmer Bias (37 of 130)

Adding Kmer Bias (38 of 130)

Adding Kmer Bias (39 of 130)

Adding Kmer Bias (40 of 130)

Adding Kmer Bias (41 of 130)

Adding Kmer Bias (42 of 130)

Adding Kmer Bias (43 of 130)

Adding Kmer Bias (44 of 130)

Adding Kmer Bias (45 of 130)

Adding Kmer Bias (46 of 130)

Adding Kmer Bias (47 of 130)

Adding Kmer Bias (48 of 130)

Adding Kmer Bias (49 of 130)

Adding Kmer Bias (50 of 130)

Adding Kmer Bias (51 of 130)

Adding Kmer Bias (52 of 130)

Adding Kmer Bias (53 of 130)

Adding Kmer Bias (54 of 130)

Adding Kmer Bias (55 of 130)

Adding Kmer Bias (56 of 130)

Adding Kmer Bias (57 of 130)

Adding Kmer Bias (58 of 130)

Adding Kmer Bias (59 of 130)

Adding Kmer Bias (60 of 130)

Adding Kmer Bias (61 of 130)

Adding Kmer Bias (62 of 130)

Adding Kmer Bias (63 of 130)

Adding Kmer Bias (64 of 130)

Adding Kmer Bias (65 of 130)

Adding Kmer Bias (66 of 130)

Adding Kmer Bias (67 of 130)

Adding Kmer Bias (68 of 130)

Adding Kmer Bias (69 of 130)

Adding Kmer Bias (70 of 130)

Adding Kmer Bias (71 of 130)

Adding Kmer Bias (72 of 130)

Adding Kmer Bias (73 of 130)

Adding Kmer Bias (74 of 130)

Adding Kmer Bias (75 of 130)

Adding Kmer Bias (76 of 130)

Adding Kmer Bias (77 of 130)

Adding Kmer Bias (78 of 130)

Adding Kmer Bias (79 of 130)

Adding Kmer Bias (80 of 130)

Adding Kmer Bias (81 of 130)

Adding Kmer Bias (82 of 130)

Adding Kmer Bias (83 of 130)

Adding Kmer Bias (84 of 130)

Adding Kmer Bias (85 of 130)

Adding Kmer Bias (86 of 130)

Adding Kmer Bias (87 of 130)

Adding Kmer Bias (88 of 130)

Adding Kmer Bias (89 of 130)

Adding Kmer Bias (90 of 130)

Adding Kmer Bias (91 of 130)

Adding Kmer Bias (92 of 130)

Adding Kmer Bias (93 of 130)

Adding Kmer Bias (94 of 130)

Adding Kmer Bias (95 of 130)

Adding Kmer Bias (96 of 130)

Adding Kmer Bias (97 of 130)

Adding Kmer Bias (98 of 130)

Adding Kmer Bias (99 of 130)

Adding Kmer Bias (100 of 130)

Adding Kmer Bias (101 of 130)

Adding Kmer Bias (102 of 130)

Adding Kmer Bias (103 of 130)

Adding Kmer Bias (104 of 130)

Adding Kmer Bias (105 of 130)

Adding Kmer Bias (106 of 130)

Adding Kmer Bias (107 of 130)

Adding Kmer Bias (108 of 130)

Adding Kmer Bias (109 of 130)

Adding Kmer Bias (110 of 130)

Adding Kmer Bias (111 of 130)

Adding Kmer Bias (112 of 130)

Adding Kmer Bias (113 of 130)

Adding Kmer Bias (114 of 130)

Adding Kmer Bias (115 of 130)

Adding Kmer Bias (116 of 130)

Adding Kmer Bias (117 of 130)

Adding Kmer Bias (118 of 130)

Adding Kmer Bias (119 of 130)

Adding Kmer Bias (120 of 130)

Adding Kmer Bias (121 of 130)

Adding Kmer Bias (122 of 130)

Adding Kmer Bias (123 of 130)

Adding Kmer Bias (124 of 130)

Adding Kmer Bias (125 of 130)

Adding Kmer Bias (126 of 130)

Adding Kmer Bias (127 of 130)

Adding Kmer Bias (128 of 130)

Adding Kmer Bias (129 of 130)

Adding Kmer Bias (130 of 130)

2023-03-09 20:54:24 : Finished Creation of Coverage Files!, 128.931 mins elapsed.

ArchR logging successful to : ArchRLogs/ArchR-addGroupCoverages-ab73c00ce4-Date-2023-03-09_Time-18-45-28.log
```

Example 17 (unknown):
```unknown
# need to make sure MACS2 is the latest version
pip install --upgrade --force-reinstall MACS2
```

Example 18 (unknown):
```unknown
?addReproduciblePeakSet
```

Example 19 (unknown):
```unknown
start_time <- Sys.time()

proj <- addReproduciblePeakSet(
    ArchRProj = proj, 
    groupBy = "fine_grain", 
    threads = 10,
    reproducibility = "2",
)

end_time <- Sys.time()
end_time - start_time
```

Example 20 (unknown):
```unknown
Searching For MACS2..

Found with $path!

ArchR logging to : ArchRLogs/ArchR-addReproduciblePeakSet-ab18b19e47-Date-2023-03-09_Time-20-54-24.log
If there is an issue, please report to github with logFile!

Calling Peaks with Macs2

2023-03-09 20:54:26 : Peak Calling Parameters!, 0.028 mins elapsed.
```

Example 21 (unknown):
```unknown
Group
AtrialCardiomyocytesCycling                               AtrialCardiomyocytesCycling
AtrialCardiomyocytesLeft                                     AtrialCardiomyocytesLeft
AtrialCardiomyocytesRight                                   AtrialCardiomyocytesRight
AtrioventricularNodeCardiomyocytes                 AtrioventricularNodeCardiomyocytes
BCells                                                                         BCells
BCellsMS4A1pos                                                         BCellsMS4A1pos
ChromaffinCells                                                       ChromaffinCells
CoronaryArterialEndothelialCells                     CoronaryArterialEndothelialCells
CoronaryCapillaryEndothelialCells                   CoronaryCapillaryEndothelialCells
CoronaryPericytes                                                   CoronaryPericytes
CoronarySmoothMuscleCells                                   CoronarySmoothMuscleCells
CoronaryVenousEndothelialCells                         CoronaryVenousEndothelialCells
CoronaryVesselAdventitialFibroblasts             CoronaryVesselAdventitialFibroblasts
DendriticCellsMature                                             DendriticCellsMature
DendriticCellsType1                                               DendriticCellsType1
DuctusArteriosusSmoothMuscleCells                   DuctusArteriosusSmoothMuscleCells
EndocardialCells                                                     EndocardialCells
EndocardialCushionCells                                       EndocardialCushionCells
EpicardiumDerivedCells                                         EpicardiumDerivedCells
GreatVesselAdventitialFibroblasts                   GreatVesselAdventitialFibroblasts
GreatVesselArterialEndothelialCells               GreatVesselArterialEndothelialCells
GreatVesselSmoothMuscleCells                             GreatVesselSmoothMuscleCells
GreatVesselVenousEndothelialCells                   GreatVesselVenousEndothelialCells
InnateLymphoidCells                                               InnateLymphoidCells
LymphaticEndothelialCells                                   LymphaticEndothelialCells
LymphNodeFibroblasticReticularCells               LymphNodeFibroblasticReticularCells
MacrophagesATF3pos                                                 MacrophagesATF3pos
MacrophagesCX3CR1pos                                             MacrophagesCX3CR1pos
MacrophagesLYVE1pos                                               MacrophagesLYVE1pos
MacrophagesTIMD4pos                                               MacrophagesTIMD4pos
MastCells                                                                   MastCells
Megakaryocytes                                                         Megakaryocytes
MesothelialEpicardialCells                                 MesothelialEpicardialCells
MonocyteDerivedCells                                             MonocyteDerivedCells
Monocytes                                                                   Monocytes
MonocytesMPOpos                                                       MonocytesMPOpos
MyocardialInterstitialFibroblasts1                 MyocardialInterstitialFibroblasts1
MyocardialInterstitialFibroblasts2                 MyocardialInterstitialFibroblasts2
Myofibroblasts                                                         Myofibroblasts
NaturalKillerCells                                                 NaturalKillerCells
NeuronPrecursors                                                     NeuronPrecursors
ParaympatheticNeurons                                           ParaympatheticNeurons
PericardialCells1                                                   PericardialCells1
PericardialCells2                                                   PericardialCells2
PericardialCells3                                                   PericardialCells3
PlasmacytoidDendriticCells                                 PlasmacytoidDendriticCells
ProBCells                                                                   ProBCells
SchwannCellPrecursors                                           SchwannCellPrecursors
SchwannCells                                                             SchwannCells
SinoatrialNodeCardiomyocytes                             SinoatrialNodeCardiomyocytes
SympatheticNeurons                                                 SympatheticNeurons
TCellsCD4pos                                                             TCellsCD4pos
TCellsCD8pos                                                             TCellsCD8pos
TregsCD4pos                                                               TregsCD4pos
ValveEndothelialCells                                           ValveEndothelialCells
ValveInterstitialCells                                         ValveInterstitialCells
VentricularCardiomyocytesCycling                     VentricularCardiomyocytesCycling
VentricularCardiomyocytesLeftCompact             VentricularCardiomyocytesLeftCompact
VentricularCardiomyocytesLeftTrabeculated   VentricularCardiomyocytesLeftTrabeculated
VentricularCardiomyocytesRightCompact           VentricularCardiomyocytesRightCompact
VentricularCardiomyocytesRightTrabeculated VentricularCardiomyocytesRightTrabeculated
VentricularConductionSystemDistal                   VentricularConductionSystemDistal
VentricularConductionSystemProximal               VentricularConductionSystemProximal
                                           nCells nCellsUsed nReplicates nMin
AtrialCardiomyocytesCycling                  1597        580           2   80
AtrialCardiomyocytesLeft                     4777        580           2   80
AtrialCardiomyocytesRight                    8187        580           2   80
AtrioventricularNodeCardiomyocytes            643        580           2   80
BCells                                       1369        580           2   80
BCellsMS4A1pos                               2559        580           2   80
ChromaffinCells                               112         84           3   40
CoronaryArterialEndothelialCells              353        353           2   80
CoronaryCapillaryEndothelialCells            2449        580           2   80
CoronaryPericytes                             688        580           2   80
CoronarySmoothMuscleCells                     415        415           2   80
CoronaryVenousEndothelialCells                539        539           2   80
CoronaryVesselAdventitialFibroblasts         1613        580           2   80
DendriticCellsMature                          156        156           2   76
DendriticCellsType1                           425        425           2   80
DuctusArteriosusSmoothMuscleCells            1159        580           2   80
EndocardialCells                             2690        580           2   80
EndocardialCushionCells                       224        224           2   80
EpicardiumDerivedCells                        340        340           2   80
GreatVesselAdventitialFibroblasts           31525        580           2   80
GreatVesselArterialEndothelialCells           929        580           2   80
GreatVesselSmoothMuscleCells                14126        580           2   80
GreatVesselVenousEndothelialCells             207        207           2   80
InnateLymphoidCells                           547        547           2   80
LymphaticEndothelialCells                    1282        580           2   80
LymphNodeFibroblasticReticularCells           969        580           2   80
MacrophagesATF3pos                            611        580           2   80
MacrophagesCX3CR1pos                          551        551           2   80
MacrophagesLYVE1pos                          4157        580           2   80
MacrophagesTIMD4pos                           303        303           2   80
MastCells                                     190        190           2   80
Megakaryocytes                                118         86           3   40
MesothelialEpicardialCells                   1214        580           2   80
MonocyteDerivedCells                          386        386           2   80
Monocytes                                     117         85           3   40
MonocytesMPOpos                                34         33           3   20
MyocardialInterstitialFibroblasts1           7671        580           2   80
MyocardialInterstitialFibroblasts2           5123        580           2   80
Myofibroblasts                               3214        580           2   80
NaturalKillerCells                           1406        580           2   80
NeuronPrecursors                              371        371           2   80
ParaympatheticNeurons                         264        264           2   80
PericardialCells1                            8967        580           2   80
PericardialCells2                             187        187           2   80
PericardialCells3                            1040        580           2   80
PlasmacytoidDendriticCells                    430        430           2   80
ProBCells                                     459        459           2   80
SchwannCellPrecursors                         759        580           2   80
SchwannCells                                 1166        580           2   80
SinoatrialNodeCardiomyocytes                  738        580           2   80
SympatheticNeurons                            354        354           2   80
TCellsCD4pos                                 4251        580           2   80
TCellsCD8pos                                 1810        580           2   80
TregsCD4pos                                  1461        580           2   80
ValveEndothelialCells                         186        186           2   80
ValveInterstitialCells                       2034        580           2   80
VentricularCardiomyocytesCycling             4977        580           2   80
VentricularCardiomyocytesLeftCompact        11397        580           2   80
VentricularCardiomyocytesLeftTrabeculated    5054        580           2   80
VentricularCardiomyocytesRightCompact       12008        580           2   80
VentricularCardiomyocytesRightTrabeculated   2269        580           2   80
VentricularConductionSystemDistal             531        531           2   80
VentricularConductionSystemProximal          1334        580           2   80
                                           nMax maxPeaks
AtrialCardiomyocytesCycling                 500   150000
AtrialCardiomyocytesLeft                    500   150000
AtrialCardiomyocytesRight                   500   150000
AtrioventricularNodeCardiomyocytes          500   150000
BCells                                      500   150000
BCellsMS4A1pos                              500   150000
ChromaffinCells                              40    42000
CoronaryArterialEndothelialCells            273   150000
CoronaryCapillaryEndothelialCells           500   150000
CoronaryPericytes                           500   150000
CoronarySmoothMuscleCells                   335   150000
CoronaryVenousEndothelialCells              459   150000
CoronaryVesselAdventitialFibroblasts        500   150000
DendriticCellsMature                         80    78000
DendriticCellsType1                         345   150000
DuctusArteriosusSmoothMuscleCells           500   150000
EndocardialCells                            500   150000
EndocardialCushionCells                     144   112000
EpicardiumDerivedCells                      260   150000
GreatVesselAdventitialFibroblasts           500   150000
GreatVesselArterialEndothelialCells         500   150000
GreatVesselSmoothMuscleCells                500   150000
GreatVesselVenousEndothelialCells           127   103500
InnateLymphoidCells                         467   150000
LymphaticEndothelialCells                   500   150000
LymphNodeFibroblasticReticularCells         500   150000
MacrophagesATF3pos                          500   150000
MacrophagesCX3CR1pos                        471   150000
MacrophagesLYVE1pos                         500   150000
MacrophagesTIMD4pos                         223   150000
MastCells                                   110    95000
Megakaryocytes                               40    43000
MesothelialEpicardialCells                  500   150000
MonocyteDerivedCells                        306   150000
Monocytes                                    40    42500
MonocytesMPOpos                              27    16500
MyocardialInterstitialFibroblasts1          500   150000
MyocardialInterstitialFibroblasts2          500   150000
Myofibroblasts                              500   150000
NaturalKillerCells                          500   150000
NeuronPrecursors                            291   150000
ParaympatheticNeurons                       184   132000
PericardialCells1                           500   150000
PericardialCells2                           107    93500
PericardialCells3                           500   150000
PlasmacytoidDendriticCells                  350   150000
ProBCells                                   379   150000
SchwannCellPrecursors                       500   150000
SchwannCells                                500   150000
SinoatrialNodeCardiomyocytes                500   150000
SympatheticNeurons                          274   150000
TCellsCD4pos                                500   150000
TCellsCD8pos                                500   150000
TregsCD4pos                                 500   150000
ValveEndothelialCells                       106    93000
ValveInterstitialCells                      500   150000
VentricularCardiomyocytesCycling            500   150000
VentricularCardiomyocytesLeftCompact        500   150000
VentricularCardiomyocytesLeftTrabeculated   500   150000
VentricularCardiomyocytesRightCompact       500   150000
VentricularCardiomyocytesRightTrabeculated  500   150000
VentricularConductionSystemDistal           451   150000
VentricularConductionSystemProximal         500   150000
```

Example 22 (unknown):
```unknown
2023-03-09 20:54:26 : Batching Peak Calls!, 0.028 mins elapsed.

2023-03-09 20:54:26 : Batch Execution w/ safelapply!, 0 mins elapsed.

2023-03-09 21:13:11 : Identifying Reproducible Peaks!, 18.778 mins elapsed.

2023-03-09 21:14:40 : Creating Union Peak Set!, 20.261 mins elapsed.

Converged after 14 iterations!

Plotting Ggplot!

2023-03-09 21:15:06 : Finished Creating Union Peak Set (508040)!, 20.694 mins elapsed.
```

Example 23 (unknown):
```unknown
Time difference of 20.703 mins
```

Example 24 (unknown):
```unknown
?addReproduciblePeakSet
```

Example 25 (unknown):
```unknown
proj = addPeakMatrix(proj)
```

Example 26 (unknown):
```unknown
ArchR logging to : ArchRLogs/ArchR-addPeakMatrix-ab40ab39a8-Date-2023-03-09_Time-21-15-06.log
If there is an issue, please report to github with logFile!

2023-03-09 21:15:07 : Batch Execution w/ safelapply!, 0 mins elapsed.

ArchR logging successful to : ArchRLogs/ArchR-addPeakMatrix-ab40ab39a8-Date-2023-03-09_Time-21-15-06.log
```

Example 27 (unknown):
```unknown
saveArchRProject(ArchRProj = proj, outputDirectory = "project_output", load = FALSE)
```

Example 28 (unknown):
```unknown
Saving ArchRProject...
```

Example 29 (unknown):
```unknown
PeakSet = getPeakSet(proj)
```

Example 30 (unknown):
```unknown
# cell number
n_cell = table(proj@cellColData$fine_grain)

# peak number
n_peak = sort(table(rownames(mcols(PeakSet))),decreasing = TRUE)
```

Example 31 (unknown):
```unknown
library(ggplot2)
library(ggrepel)

options(repr.plot.width=10, repr.plot.height=10)

df = data.frame(fine_grain=names(n_peak),
           n_peak=as.numeric(unname(n_peak)),
           n_cell=as.numeric(unname(n_cell[names(n_peak)])))
df = df[order(-df$n_peak),]

ggplot(df, aes(n_cell,n_peak)) +
  geom_point()
```

Example 32 (unknown):
```unknown
ggplot(df, aes(n_cell,n_peak)) +
  geom_point() +
  geom_text_repel(aes(label = fine_grain))
```

Example 33 (unknown):
```unknown
sessionInfo()
```

---

## Import required modules¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig2_Overview_of_cellular_niches_and_TissueTypist/Visium_tissue-by-celltype_dotplot.html

**Contents:**
- Import required modules¶
- Read in anndata¶
- Preapare cell2location output anndata¶
- Select regions and cells to plot¶
- Dot plot¶

**Examples:**

Example 1 (python):
```python
import warnings
warnings.simplefilter(action='ignore', category=FutureWarning)

import scanpy as sc
import anndata
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 

import seaborn as sns
import os 
from mpl_toolkits.axes_grid1 import make_axes_locatable

import scipy
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/session_info/main.py:213: UserWarning: The '__version__' attribute is deprecated and will be removed in MarkupSafe 3.1. Use feature detection, or `importlib.metadata.version("markupsafe")`, instead.
  mod_version = _find_version(mod.__version__)
```

Example 4 (python):
```python
-----
anndata             0.11.4
matplotlib          3.10.1
mpl_toolkits        NA
numpy               2.1.3
pandas              2.2.3
scanpy              1.11.1
scipy               1.15.2
seaborn             0.13.2
session_info        v1.0.1
-----
```

Example 5 (python):
```python
Cython              3.1.1
PIL                 11.1.0
asciitree           NA
asttokens           NA
charset_normalizer  3.4.1
cloudpickle         3.1.1
comm                0.2.2
cycler              0.12.1
cython              3.1.1
cython_runtime      NA
dask                2024.11.2
dateutil            2.9.0.post0
debugpy             1.8.13
decorator           5.2.1
exceptiongroup      1.2.2
executing           2.2.0
h5py                3.13.0
igraph              0.11.8
importlib_metadata  NA
ipykernel           6.29.5
jedi                0.19.2
jinja2              3.1.6
joblib              1.4.2
kiwisolver          1.4.8
legacy_api_wrap     NA
leidenalg           0.10.2
llvmlite            0.44.0
markupsafe          3.0.2
msgpack             1.1.0
natsort             8.4.0
numba               0.61.0
numcodecs           0.13.1
packaging           24.2
parso               0.8.4
patsy               1.0.1
platformdirs        4.3.7
prompt_toolkit      3.0.50
psutil              7.0.0
pure_eval           0.2.3
pyarrow             19.0.1
pydev_ipython       NA
pydevconsole        NA
pydevd              3.2.3
pydevd_file_utils   NA
pydevd_plugins      NA
pydevd_tracing      NA
pygments            2.19.1
pyparsing           3.2.3
pytz                2025.2
six                 1.17.0
sklearn             1.5.2
stack_data          0.6.3
statsmodels         0.14.4
tblib               3.1.0
texttable           1.7.0
threadpoolctl       3.6.0
tlz                 1.0.0
toolz               1.0.0
tornado             6.4.2
traitlets           5.14.3
typing_extensions   NA
wcwidth             0.2.13
yaml                6.0.2
zarr                2.18.3
zipp                NA
zmq                 26.3.0
zoneinfo            NA
```

Example 6 (unknown):
```unknown
-----
IPython             8.34.0
jupyter_client      8.6.3
jupyter_core        5.7.2
-----
Python 3.10.14 | packaged by conda-forge | (main, Mar 20 2024, 12:45:18) [GCC 12.3.0]
Linux-4.18.0-553.53.1.el8_10.x86_64-x86_64-with-glibc2.28
-----
Session information updated at 2025-07-14 22:15
```

Example 7 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/rfs/project/rfs-iCNyzSAaucw/kk837/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 8 (python):
```python
sc.settings.verbosity = 1          # verbosity: errors (0), warnings (1), info (2), hints (3)
sc.settings.set_figure_params(dpi=80,dpi_save=300)
```

Example 9 (unknown):
```unknown
figdir = '/rfs/project/rfs-iCNyzSAaucw/kk837/notebooks/Foetal/fetal_ms_figures'
```

Example 10 (python):
```python
adata_vis = sc.read_h5ad('/rfs/project/rfs-iCNyzSAaucw/kk837/data_objects/Foetal/Visium/visium_foetal_healthy-all_raw.h5ad')
adata_vis
```

Example 11 (unknown):
```unknown
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/anndata/_core/anndata.py:1758: UserWarning: Variable names are not unique. To make them unique, call `.var_names_make_unique`.
  utils.warn_names_duplicates("var")
```

Example 12 (unknown):
```unknown
AnnData object with n_obs × n_vars = 39223 × 33562
    obs: 'array_row', 'array_col', 'sample', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'mt_frac', 'sangerID', 'Publication', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'status', 'cell2loc-map_group', 'JC_edit', 'sample_for_cell2loc', 'week', 'leiden', 'region_cluster', 'AtrialCardiomyocytesLeft', 'AtrialCardiomyocytesRight', 'VentricularCardiomyocytesLeftCompact', 'VentricularCardiomyocytesRightCompact', 'VentricularCardiomyocytesLeftTrabeculated', 'VentricularCardiomyocytesRightTrabeculated', 'SinoatrialNodeCardiomyocytes', 'AtrioventricularNodeCardiomyocytes', 'VentricularConductionSystemProximal', 'VentricularConductionSystemDistal', 'GreatVesselAdventitialFibroblasts', 'CoronaryVesselAdventitialFibroblasts', 'MyocardialInterstitialFibroblasts1', 'MyocardialInterstitialFibroblasts2', 'Myofibroblasts', 'LymphNodeFibroblasticReticularCells', 'ValveInterstitialCells', 'GreatVesselSmoothMuscleCells', 'CoronarySmoothMuscleCells', 'CoronaryPericytes', 'GreatVesselArterialEndothelialCells', 'GreatVesselVenousEndothelialCells', 'CoronaryArterialEndothelialCells', 'CoronaryVenousEndothelialCells', 'CoronaryCapillaryEndothelialCells', 'EndocardialCells', 'EndocardialCushionCells', 'ValveEndothelialCells', 'LymphaticEndothelialCells', 'MesothelialEpicardialCells', 'EpicardiumDerivedCells', 'NeuronPrecursors', 'ChromaffinCells', 'SympatheticNeurons', 'ParaympatheticNeurons', 'SchwannCellPrecursors', 'SchwannCells', 'MonocytesMPOpos', 'Monocytes', 'MonocyteDerivedCells', 'MacrophagesCX3CR1pos', 'MacrophagesTIMD4pos', 'MacrophagesLYVE1pos', 'MacrophagesATF3pos', 'DendriticCellsType1', 'DendriticCellsMature', 'PlasmacytoidDendriticCells', 'MastCells', 'Megakaryocytes', 'TCellsCD4pos', 'TCellsCD8pos', 'TregsCD4pos', 'ProBCells', 'BCells', 'BCellsMS4A1pos', 'NaturalKillerCells', 'InnateLymphoidCells', 'DuctusArteriosusSmoothMuscleCells', 'AtrialCardiomyocytesLeft_minmax', 'AtrialCardiomyocytesRight_minmax', 'VentricularCardiomyocytesLeftCompact_minmax', 'VentricularCardiomyocytesRightCompact_minmax', 'VentricularCardiomyocytesLeftTrabeculated_minmax', 'VentricularCardiomyocytesRightTrabeculated_minmax', 'SinoatrialNodeCardiomyocytes_minmax', 'AtrioventricularNodeCardiomyocytes_minmax', 'VentricularConductionSystemProximal_minmax', 'VentricularConductionSystemDistal_minmax', 'GreatVesselAdventitialFibroblasts_minmax', 'CoronaryVesselAdventitialFibroblasts_minmax', 'MyocardialInterstitialFibroblasts1_minmax', 'MyocardialInterstitialFibroblasts2_minmax', 'Myofibroblasts_minmax', 'LymphNodeFibroblasticReticularCells_minmax', 'ValveInterstitialCells_minmax', 'GreatVesselSmoothMuscleCells_minmax', 'CoronarySmoothMuscleCells_minmax', 'CoronaryPericytes_minmax', 'GreatVesselArterialEndothelialCells_minmax', 'GreatVesselVenousEndothelialCells_minmax', 'CoronaryArterialEndothelialCells_minmax', 'CoronaryVenousEndothelialCells_minmax', 'CoronaryCapillaryEndothelialCells_minmax', 'EndocardialCells_minmax', 'EndocardialCushionCells_minmax', 'ValveEndothelialCells_minmax', 'LymphaticEndothelialCells_minmax', 'MesothelialEpicardialCells_minmax', 'EpicardiumDerivedCells_minmax', 'NeuronPrecursors_minmax', 'ChromaffinCells_minmax', 'SympatheticNeurons_minmax', 'ParaympatheticNeurons_minmax', 'SchwannCellPrecursors_minmax', 'SchwannCells_minmax', 'MonocytesMPOpos_minmax', 'Monocytes_minmax', 'MonocyteDerivedCells_minmax', 'MacrophagesCX3CR1pos_minmax', 'MacrophagesTIMD4pos_minmax', 'MacrophagesLYVE1pos_minmax', 'MacrophagesATF3pos_minmax', 'DendriticCellsType1_minmax', 'DendriticCellsMature_minmax', 'PlasmacytoidDendriticCells_minmax', 'MastCells_minmax', 'Megakaryocytes_minmax', 'TCellsCD4pos_minmax', 'TCellsCD8pos_minmax', 'TregsCD4pos_minmax', 'ProBCells_minmax', 'BCells_minmax', 'BCellsMS4A1pos_minmax', 'NaturalKillerCells_minmax', 'InnateLymphoidCells_minmax', 'DuctusArteriosusSmoothMuscleCells_minmax', 'annotation', 'annotation_final', 'annotation_final_coarse', 'annotation_final_mod'
    var: 'gene_ids'
    uns: 'JC_edit_colors', 'annotation_colors', 'annotation_final_colors', 'cell2loc-map_group_colors', 'leiden', 'neighbors', 'region_cluster_colors', 'sample_for_cell2loc_colors', 'umap'
    obsm: 'MT', 'X_umap', 'means_cell_abundance_w_sf', 'q05_cell_abundance_w_sf', 'q05_cell_abundance_w_sf_ndarray', 'q95_cell_abundance_w_sf', 'spatial', 'stds_cell_abundance_w_sf'
    obsp: 'connectivities', 'distances'
```

Example 13 (python):
```python
# modify annotations for matching with factor analysis
annotation_edit_dict = {
   'AV valves':'Valves',
    'Aortic valve':'Valves',
    'Valve apparatus':'Valves',
    'Atrium - Left':'Atrium',
     'Atrium - Right':'Atrium',
    'Ventricle - Compact - Left':'Ventricle - Compact',
    'Ventricle - Compact - Right':'Ventricle - Compact',
    'Ventricle - Trabeculated - Left':'Ventricle - Trabeculated',
    'Ventricle - Trabeculated - Right':'Ventricle - Trabeculated',
}
adata_vis.obs['annotation_final_mod']=adata_vis.obs['annotation_final'].astype('str').copy()
adata_vis.obs.replace({'annotation_final_mod':annotation_edit_dict}, 
                      inplace=True)
adata_vis.obs['annotation_final_mod'] = adata_vis.obs['annotation_final_mod'].astype('category')

#reorder annotations
annotations_order = [
   'Atrium',
        'Ventricle - Compact',
     'Ventricle - Trabeculated',
        'SAnode',
        'AVnode',
    'VCS - Proximal',
        'VCS - Distal',
        'Great vessel',
        'Coronary vessel',
    'Ductus arteriosus',
        'Adventitia - Coronary',
     'Adventitia - GV',
    'Epicardium',
        'Endocardium - Atrial', 
    'Endocardium - Ventricular',
        'Endocardial cusion',
        'AV ring',
    'Valves',
    'LN',
            ]
adata_vis.obs['annotation_final_mod'] = adata_vis.obs['annotation_final_mod'].cat.reorder_categories(annotations_order)
```

Example 14 (python):
```python
# get the cell2loc output
cell2loc_out = adata_vis.obsm['q05_cell_abundance_w_sf'].copy()
## read in cell type name update dictionary
celltype_rename = utils.read_pkl('/rfs/project/rfs-iCNyzSAaucw/kk837/data_objects/Foetal/fine_grain_rename_dictionary.pkl')
## update cell type name
cellnames = adata_vis.obsm['q05_cell_abundance_w_sf'].columns
cellnames = [x.split('q05cell_abundance_w_sf_')[1] for x in cellnames]
cellnames = [celltype_rename[x] for x in cellnames]

# select obs columns
obs_columns = ['array_row', 'array_col', 'sample', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes',
               'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'mt_frac', 'sangerID', 'Publication', 'combinedID', 'donor', 
               'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei',
               'modality', 'kit_10x', 'flushed', 'status', 'cell2loc-map_group', 'JC_edit', 'sample_for_cell2loc', 'week', 'leiden', 'annotation_final','annotation_final_mod']

# make anndata
adata_c2l = anndata.AnnData(
    X=adata_vis.obsm['q05_cell_abundance_w_sf_ndarray'].copy(),
    obs=adata_vis.obs[obs_columns],
    var=pd.DataFrame(cellnames).set_index(0),
    obsm=adata_vis.obsm.copy(),
    obsp=adata_vis.obsp.copy(),
    uns=adata_vis.uns.copy()
)
adata_c2l
```

Example 15 (unknown):
```unknown
AnnData object with n_obs × n_vars = 39223 × 58
    obs: 'array_row', 'array_col', 'sample', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'mt_frac', 'sangerID', 'Publication', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'status', 'cell2loc-map_group', 'JC_edit', 'sample_for_cell2loc', 'week', 'leiden', 'annotation_final', 'annotation_final_mod'
    uns: 'JC_edit_colors', 'annotation_colors', 'annotation_final_colors', 'cell2loc-map_group_colors', 'leiden', 'neighbors', 'region_cluster_colors', 'sample_for_cell2loc_colors', 'umap'
    obsm: 'MT', 'X_umap', 'means_cell_abundance_w_sf', 'q05_cell_abundance_w_sf', 'q05_cell_abundance_w_sf_ndarray', 'q95_cell_abundance_w_sf', 'spatial', 'stds_cell_abundance_w_sf'
    obsp: 'connectivities', 'distances'
```

Example 16 (python):
```python
df = pd.read_csv('/rfs/project/rfs-iCNyzSAaucw/kk837/notebooks/Foetal/finegrain_name_mapping.csv').set_index('Full_name')
mapping = df['Short_name_finalised'].to_dict()
'AtrialCardiomyocytesLeft' in mapping.keys()
```

Example 17 (python):
```python
# map fine-grained cell types to short version# map cell type name to short versions
df = pd.read_csv('/rfs/project/rfs-iCNyzSAaucw/kk837/notebooks/Foetal/finegrain_name_mapping.csv').set_index('Full_name')
mapping = df['Short_name_finalised'].to_dict()
adata_c2l.var_names = [mapping[x] if x in mapping.keys() else x for x in adata_c2l.var_names]
adata_c2l.var
```

Example 18 (python):
```python
# make the value 0 below an arbitrary threshold
# to calculate proportion
abundance_thresh = 0.05
adata_c2l.raw = adata_c2l
adata_c2l.X[adata_c2l.X<abundance_thresh] = 0
adata_c2l.X = scipy.sparse.csr_matrix(adata_c2l.X)
```

Example 19 (python):
```python
/tmp/ipykernel_295047/1462688745.py:5: SparseEfficiencyWarning: Comparing a sparse matrix with a scalar greater than zero using < is inefficient, try using >= instead.
  adata_c2l.X[adata_c2l.X<abundance_thresh] = 0
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scipy/sparse/_index.py:210: SparseEfficiencyWarning: Changing the sparsity structure of a csr_matrix is expensive. lil and dok are more efficient.
  self._set_arrayXarray(i, j, x)
```

Example 20 (python):
```python
# select regions
# regions without LN
regions_sel = [ 'Great vessel',
               'Ductus arteriosus',
               'Coronary vessel',
                'Adventitia - GV',
               'Adventitia - Coronary',
               'AV ring',
               'Valves',
               'Atrium',
               'SAnode',
               'AVnode',
               'VCS - Proximal',
               'VCS - Distal',
               'Ventricle - Trabeculated',
               'Ventricle - Compact',
                'Epicardium',
               'Endocardium - Atrial',
               'Endocardium - Ventricular',        
               'Endocardial cusion',
 ]
ad = adata_c2l[adata_c2l.obs['annotation_final_mod'].isin(regions_sel)]
```

Example 21 (python):
```python
# select cells to plot based on expression proportion

## calculate expression proportion
exp_prop = utils.get_exp_proportion(ad,groupby='annotation_final_mod', use_rep='X')

## select cells which are above an arbitrary threshold at any of the niches
prop_thresh = 0.4
cells_sel = list(exp_prop.max(axis=1).index[exp_prop.max(axis=1)>prop_thresh])

# get cell order based on hierarchical clustering
## mean abundance per niche
exp_mean = utils.grouped_obs_mean(ad[:,cells_sel], group_key='annotation_final_mod').T
## min-max normalisation
exp_mean_scaled = (exp_mean-exp_mean.min())/(exp_mean.max()-exp_mean.min())
## clustering
g = sns.clustermap(exp_mean_scaled.T, figsize=(10,15), dendrogram_ratio=0.1)
row_order = g.dendrogram_row.reordered_ind
cell_order = exp_mean_scaled.columns[row_order]
```

Example 22 (python):
```python
ad = adata_c2l[adata_c2l.obs['annotation_final_mod'].isin(regions_sel)]
ad.obs['annotation_final_mod'] = ad.obs['annotation_final_mod'].astype('category')
ad.obs['annotation_final_mod'] = ad.obs['annotation_final_mod'].cat.reorder_categories(regions_sel)

sc.tl.dendrogram(ad, groupby='annotation_final_mod')

sc.pl.dotplot(ad,
              var_names=cell_order,
              standard_scale='var',
              groupby='annotation_final_mod',
                               dendrogram=True,
             cmap='YlGn',
              colorbar_title='Mean abundance\nin group',
              show=False
                  )
plt.savefig(f"{figdir}/visium_tissue-by-celltype_dotplot.pdf", bbox_inches="tight",dpi=300)
```

Example 23 (python):
```python
/tmp/ipykernel_295047/3506546109.py:2: ImplicitModificationWarning: Trying to modify attribute `.obs` of view, initializing view as actual.
  ad.obs['annotation_final_mod'] = ad.obs['annotation_final_mod'].astype('category')
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_utils.py:40: UserWarning: You’re trying to run this on 58 dimensions of `.X`, if you really want this, set `use_rep='X'`.
         Falling back to preprocessing with `sc.pp.pca` and default params.
  warnings.warn(
```

---

## Read in adata¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig5_Trisomy21/no08_reassign-nhoods_significant-nhoods.html

**Contents:**
- Read in adata¶
- Identify significant nhoods¶
- Subset cells and nhoods¶
- Assess cells which belong to multipe nhoods¶
- Re-assign cells to a neighbourhood¶
- Save¶

--> most of the nhoods contain repetitive cells

**Examples:**

Example 1 (python):
```python
import scanpy as sc
import pandas as pd
import numpy as np
import anndata
import scipy
import matplotlib.pyplot as plt
import seaborn as sns
import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42

import warnings
warnings.simplefilter("ignore", UserWarning)
warnings.simplefilter(action='ignore', category=pd.errors.PerformanceWarning)
```

Example 2 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 3 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 4 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk/Aug2024'
table_dir = '/nfs/team205/heart/fetal_ms_figures/tables/Aug2024'
```

Example 5 (python):
```python
# read in post-Milo single-cell adata
adata = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/trisomy21/Euploid_T21Hearts_Aug2024_sel_subsampled-per-cellstate-donor.h5ad')
adata
```

Example 6 (python):
```python
AnnData object with n_obs × n_vars = 83477 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score', 'euploid_or_t21', 'donor_type', 'region_finest', 'flushed', 'scrublet_leiden', 'cluster_scrublet_score', 'leiden_scVI_0.1', 'leiden_scVI_0.2', 'leiden_scVI_0.3', 'leiden_scVI_0.5', 'leiden_scVI_0.7', 'leiden_scVI_1', 'celltypist_coarse', 'conf_score_coarse', 'cells_to_remove', 'celltypist_fine', 'conf_score_fine', 'coarse_grain_merged', 'fine_grain_merged', 'nhood_ixs_random', 'nhood_ixs_refined', 'nhood_kth_distance'
    var: 'gene_id', 'gene_name_scRNA-0-0', 'feature_type-1', 'mt-1', 'ribo-1', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1', 'gene_name_GenomicFeatures', 'gene_biotype_GenomicFeatures', 'gene_seq_start_GenomicFeatures', 'gene_seq_end_GenomicFeatures', 'seq_name_GenomicFeatures', 'seq_strand_GenomicFeatures', 'seq_coord_system_GenomicFeatures', 'description_GenomicFeatures', 'gene_id_version_GenomicFeatures', 'canonical_transcript_GenomicFeatures', 'gene_name'
    uns: 'coarse_grain_merged_colors', 'correcting-donor-region_n-layers-2', 'donor_colors', 'euploid_or_t21_colors', 'fine_grain_merged_colors', 'mid_grain_colors', 'nhood_adata', 'nhood_neighbors_key', 'region_colors', 'scVI_latent_correcting-donor-region_n-layers-2-forMilo', 'umap'
    obsm: 'X_umap_correcting-donor-region_n-layers-2', 'latent_gene_encoding', 'nhoods', 'nhoods_reassigned', 'scVI_latent_correcting-donor-region_n-layers-2'
    obsp: 'correcting-donor-region_n-layers-2_connectivities', 'correcting-donor-region_n-layers-2_distances', 'scVI_latent_correcting-donor-region_n-layers-2-forMilo_connectivities', 'scVI_latent_correcting-donor-region_n-layers-2-forMilo_distances'
```

Example 7 (python):
```python
# get significant nhoods
DA_fdr_thresh = 0.1
fcneg_label = 'euploid'
fcpos_label = 'trisomy21'

##############################
nhood_adata = adata.uns['nhood_adata'].copy()
mask_sig = nhood_adata.obs['SpatialFDR'] < DA_fdr_thresh
mask_fcneg = nhood_adata.obs['logFC'] < 0
mask_fcpos = nhood_adata.obs['logFC'] > 0
nhoods_fcneg = list(nhood_adata.obs_names[mask_sig&mask_fcneg])
print(f'FC-negative nhoods: {len(nhoods_fcneg)}')
print(nhoods_fcneg[:5])
nhoods_fcpos = list(nhood_adata.obs_names[mask_sig&mask_fcpos])
print(f'FC-positive nhoods: {len(nhoods_fcpos)}')
print(nhoods_fcpos[:5])

# add to nhood_adata
adata.uns['nhood_adata'].obs['deg_group'] = 'nan'
adata.uns['nhood_adata'].obs.loc[nhoods_fcneg,'deg_group'] = fcneg_label
adata.uns['nhood_adata'].obs.loc[nhoods_fcpos,'deg_group'] = fcpos_label
adata.uns['nhood_adata'].obs['deg_group'].value_counts()
```

Example 8 (unknown):
```unknown
FC-negative nhoods: 238
['nhood58', 'nhood157', 'nhood161', 'nhood178', 'nhood185']
FC-positive nhoods: 339
['nhood2', 'nhood20', 'nhood36', 'nhood53', 'nhood105']
```

Example 9 (unknown):
```unknown
deg_group
nan          6116
trisomy21     339
euploid       238
Name: count, dtype: int64
```

Example 10 (unknown):
```unknown
# gent nhood index
mhoods_sel = nhoods_fcneg + nhoods_fcpos
nhood_index = [int(x.replace('nhood','')) for x in mhoods_sel]
nhood_index.sort()
```

Example 11 (python):
```python
# get cell obsnames
# which belongs to the significant nhoods obtained above
mask = adata.obsm['nhoods'][:,nhood_index].toarray().sum(axis=1)>0 # get cells which has at least one neighbourhod to belong
cell_obsnames = adata.obs_names[mask]
print(len(cell_obsnames))
```

Example 12 (python):
```python
# subset
## cell
adata_sub = adata[cell_obsnames]
## nhoods, in the two different data storage
adata_sub.obsm['nhoods'] = adata_sub.obsm['nhoods'][:,nhood_index]
print(adata_sub.obsm['nhoods'].shape)
adata_sub.uns['nhood_adata'] = adata_sub.uns['nhood_adata'][nhood_index]
print(adata_sub.uns['nhood_adata'].shape)
```

Example 13 (unknown):
```unknown
(14846, 577)
(577, 24)
```

Example 14 (python):
```python
mask = adata_sub.obsm['nhoods'].toarray().sum(axis=1)>1 # get cells which has at least two neighbourhod to belong
cells_rep = adata_sub.obs_names[mask]
print(f'number of cells: {adata_sub.shape[0]}')
print(f'fraction of repetitive cells: {len(cells_rep)/adata_sub.shape[0]}')
```

Example 15 (unknown):
```unknown
number of cells: 14846
fraction of repetitive cells: 0.3483766671157214
```

Example 16 (unknown):
```unknown
len(cells_rep)
```

Example 17 (python):
```python
print(adata_sub.obsm['nhoods'].shape)
plt.hist(adata_sub.obsm['nhoods'].toarray().sum(axis=1),bins=10)
plt.xlabel('number of nhoods')
plt.show()
```

Example 18 (unknown):
```unknown
(14846, 577)
```

Example 19 (python):
```python
# obtain nhoods which contains repetitive cells
mask = adata_sub[cells_rep].obsm['nhoods'].toarray().sum(axis=0)>0
nhoods_rep = np.nonzero(mask)[0]
print(len(nhoods_rep))
nhoods_rep[:5]
```

Example 20 (unknown):
```unknown
array([0, 1, 2, 3, 4])
```

Example 21 (python):
```python
# get centroid for each nhood
cell_by_nhood = adata_sub.obsm['nhoods'][:,nhoods_rep].copy()
n_cell_per_nhood = cell_by_nhood.toarray().sum(axis=0)
scVI_latent_rep = 'scVI_latent_correcting-donor-region_n-layers-2'
cell_by_scvi = adata_sub.obsm[scVI_latent_rep].copy()
nhood_by_scvi_sum = np.array(np.matmul(cell_by_nhood.T.todense(),cell_by_scvi))
nhood_by_scvi_mean = (nhood_by_scvi_sum.T/n_cell_per_nhood).T # i.e. centroid
nhood_by_scvi_mean = pd.DataFrame(nhood_by_scvi_mean,
                                  index=[f'nhood{str(x)}' for x in nhoods_rep])
print(nhood_by_scvi_mean.shape)
nhood_by_scvi_mean.head()
```

Example 22 (python):
```python
# for each repetitive cell, calculate distance to the centroids of the nhoods, and select the most closest one
import numpy as np
def assign_cells_to_neighborhoods(cell_nhood, cell_latent, nhood_centroids):
    """
    Assigns each cell to the closest neighborhood (nhood) centroid among the neighborhoods 
    the cell belongs to.
    
    Parameters:
      cell_nhood: 2D binary NumPy array of shape (n_cells, n_nhoods).
                  cell_nhood[i, j] is 1 if cell i belongs to nhood j.
      cell_latent: 2D NumPy array of shape (n_cells, latent_dim) containing the cell coordinates.
      nhood_centroids: 2D NumPy array of shape (n_nhoods, latent_dim) containing the centroids.
    
    Returns:
      A 1D array of length n_cells where each element is the index of the nearest nhood centroid 
      (among those the cell belongs to) for that cell.
      
    Explanation
    1. Distance Computation: The squared Euclidean distance between each cell (vector in latent space) and each nhood centroid is computed using a vectorized approach. This avoids explicit loops and leverages fast linear algebra routines.
    2. Masking: Since each cell only belongs to certain nhoods (as indicated by the cell_nhood binary matrix), we set the distances for non-member nhoods to infinity (np.inf). This ensures that the minimization only considers the valid nhoods.
    3. Choosing the Minimum: Finally, we use np.argmin along the nhood axis (axis=1) to find the index of the closest centroid among those the cell belongs to.  
    """
    # Compute squared Euclidean distances between each cell and each nhood centroid.
    # Using the identity: ||a-b||^2 = ||a||^2 + ||b||^2 - 2 * a.dot(b)
    cell_sq = np.sum(cell_latent**2, axis=1, keepdims=True)   # shape: (n_cells, 1)
    nhood_sq = np.sum(nhood_centroids**2, axis=1)  # shape: (n_nhoods,)
    distances = cell_sq + nhood_sq - 2 * (cell_latent @ nhood_centroids.T)  # shape: (n_cells, n_nhoods)
    
    # Mask distances for nhoods that the cell does not belong to.
    # For these entries, set the distance to a very high value (np.inf) so they are not chosen.
    mask = cell_nhood.astype(bool)
    distances[~mask] = np.inf
    
    # For each cell, find the index of the nhood with the minimal distance.
    closest_nhood = np.argmin(distances, axis=1)
    
    return closest_nhood
```

Example 23 (python):
```python
%%time
cell_by_nhood = adata_sub[cells_rep].obsm['nhoods'][:,nhoods_rep].toarray().copy()
print(cell_by_nhood.shape)
cell_by_scVI = adata_sub[cells_rep].obsm[scVI_latent_rep].toarray().copy()
print(cell_by_scVI.shape)

assignments = assign_cells_to_neighborhoods(cell_by_nhood, cell_by_scVI, np.array(nhood_by_scvi_mean))
assignments = nhood_by_scvi_mean.index[assignments]
print(len(assignments))
assignments[:5]
```

Example 24 (unknown):
```unknown
(5172, 559)
(5172, 50)
5172
CPU times: user 98.8 ms, sys: 2.6 ms, total: 101 ms
Wall time: 104 ms
```

Example 25 (unknown):
```unknown
Index(['nhood566', 'nhood398', 'nhood395', 'nhood79', 'nhood6'], dtype='object')
```

Example 26 (python):
```python
# reassign nhoods
cell_index = np.nonzero(adata_sub.obs_names.isin(cells_rep))[0]
nhood_reassigned_index = [int(x.replace('nhood','')) for x in assignments]

cell_by_nhood = adata_sub.obsm['nhoods'].toarray().copy()
cell_by_nhood[cell_index,:] = 0
cell_by_nhood[cell_index,nhood_reassigned_index] = 1
from scipy.sparse import csr_matrix
adata_sub.obsm['nhoods_reassigned'] = csr_matrix(cell_by_nhood)
adata_sub.obsm['nhoods_reassigned']
```

Example 27 (python):
```python
<14846x577 sparse matrix of type '<class 'numpy.float32'>'
	with 14846 stored elements in Compressed Sparse Row format>
```

Example 28 (python):
```python
# check
print(np.max(adata_sub.obsm['nhoods_reassigned'].toarray().sum(axis=1)))
plt.hist(adata_sub.obsm['nhoods_reassigned'].toarray().sum(axis=1))
plt.xlabel('number of nhoods')
plt.show()
```

Example 29 (python):
```python
# n_cells per nhood
plt.hist(adata_sub.obsm['nhoods'].toarray().sum(axis=0),bins=100)
plt.title('original')
plt.show()

plt.hist(adata_sub.obsm['nhoods_reassigned'].toarray().sum(axis=0),bins=100)
plt.title('reassigned')
plt.show()
```

Example 30 (python):
```python
# save
adata_sub.write('/nfs/team205/heart/anndata_objects/Foetal/trisomy21/Euploid_T21Hearts_Aug2024_sel_subsampled-per-cellstate-donor_cells-in-sig-nhoods-sfdr0p1.h5ad')
```

---

## Read in inputs¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig4_Cardiomyocytes/VentricularCardiomyocytes/no3_create-network_TF-Region-TG_compact-common_up.html

**Contents:**
- Read in inputs¶
- Read in gene signatures and define target genes¶
- Read in peak signatures¶
- Read in scenicplus output metadata¶
- Read in and prepare anndata contains gene expression¶
- Read in cardiomyocyte markers¶
- Obtain expressing genes and accessible regions¶
- Genes¶
- Peaks¶
- Prepare scenic metadata¶
- Select interactions¶
- TF-TF network¶
- Calculate centrality¶
- Save networks¶
- Plot network¶
- Correlation of peak cound and n_TF to bind¶

What this notebook does

**Examples:**

Example 1 (python):
```python
# Parameters
working_dir = "/nfs/team205/heart/Vitro/TF_selection"
path_to_adata_gex = '/nfs/team205/heart/anndata_objects/Foetal/vCM/post_scFate/foetal_vCM_onlyLeft.h5ad'
path_to_adata_peak_tfidf = "/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/PeakMatrix/Foetal_Peaks_tfidf.h5ad"
group_key = "t_binned"
group_a = "t3"
group_b = "t1"
time_key = "t"
subset_object = False
geneset_list = ["MSigDB_Hallmark_2020", "KEGG_2021_Human", "GO_Biological_Process_2023"]

path_to_eRegulon_signatures_gene = "/nfs/team205/heart/pyscenic/plus/eRegulon_signatures/vCM_onlyLeft_Gene_eRegulon_signatures.pkl"
path_to_eRegulon_signatures_region = "/nfs/team205/heart/pyscenic/plus/eRegulon_signatures/vCM_onlyLeft_Region_eRegulon_signatures.pkl"

path_to_DAR_compact = '/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/MarkerPeaks/vCM_Left_Compact-vs-Trab.csv'
path_to_DAR_trab = '/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/MarkerPeaks/vCM_Left_Trab-vs-Compact.csv'
path_to_eRegulon_metadata = "/nfs/team205/heart/pyscenic/plus/eRegulon_metadata/vCM_onlyLeft_eRegulon_metadata_filtered.csv" # filtered metadata

path_to_gene_signatures = '/nfs/team205/heart/Vitro/TF_selection/foetal_vCM_left_compact-vs-trabeculated_downstream/gene_signatures.pkl'
path_to_peak_signatures = '/nfs/team205/heart/Vitro/TF_selection/foetal_vCM_left_compact-vs-trabeculated_downstream/peak_signatures.pkl'

###### parameters for TF selection for the population ######
cell_of_intest = 'VentricularCardiomyocytesLeftCompact'
target_signatures = ['compact_specific_up','common_up']

# proportion threshold for expressing genes
exp_prop_thresh = 0.1

# proportion threshold for accesible regions
access_prop_thresh = 0.01
```

Example 2 (python):
```python
import scanpy as sc
import pandas as pd
import numpy as np
import anndata

import matplotlib.pyplot as plt
import seaborn as sns

import os
```

Example 3 (python):
```python
import session_info
session_info.show()
```

Example 4 (python):
```python
-----
anndata             0.9.2
matplotlib          3.7.2
numpy               1.24.4
pandas              1.5.3
scanpy              1.9.3
seaborn             0.12.2
session_info        1.0.0
-----
```

Example 5 (python):
```python
PIL                         10.0.0
anyio                       NA
arrow                       1.2.3
asttokens                   NA
attr                        23.1.0
attrs                       23.1.0
babel                       2.12.1
backcall                    0.2.0
brotli                      1.0.9
certifi                     2023.11.17
cffi                        1.15.1
charset_normalizer          3.2.0
colorama                    0.4.6
comm                        0.1.4
cycler                      0.10.0
cython_runtime              NA
dateutil                    2.8.2
debugpy                     1.6.8
decorator                   5.1.1
defusedxml                  0.7.1
executing                   1.2.0
fastjsonschema              NA
fqdn                        NA
h5py                        3.9.0
idna                        3.4
igraph                      0.10.6
importlib_metadata          NA
importlib_resources         NA
ipykernel                   6.25.1
isoduration                 NA
jedi                        0.19.0
jinja2                      3.1.2
joblib                      1.3.2
json5                       NA
jsonpointer                 2.0
jsonschema                  4.19.0
jsonschema_specifications   NA
jupyter_events              0.7.0
jupyter_server              2.7.1
jupyterlab_server           2.24.0
kiwisolver                  1.4.5
leidenalg                   0.10.1
llvmlite                    0.40.1
markupsafe                  2.1.3
mpl_toolkits                NA
natsort                     8.4.0
nbformat                    5.9.2
numba                       0.57.1
overrides                   NA
packaging                   23.1
parso                       0.8.3
patsy                       0.5.3
pexpect                     4.8.0
pickleshare                 0.7.5
pkg_resources               NA
plotly                      5.16.1
prometheus_client           NA
prompt_toolkit              3.0.39
psutil                      5.9.5
ptyprocess                  0.7.0
pure_eval                   0.2.2
pydev_ipython               NA
pydevconsole                NA
pydevd                      2.9.5
pydevd_file_utils           NA
pydevd_plugins              NA
pydevd_tracing              NA
pygments                    2.16.1
pyparsing                   3.0.9
pythonjsonlogger            NA
pytz                        2023.3
referencing                 NA
requests                    2.31.0
rfc3339_validator           0.1.4
rfc3986_validator           0.1.1
rpds                        NA
ruamel                      NA
scipy                       1.10.1
send2trash                  NA
six                         1.16.0
sklearn                     1.3.0
sniffio                     1.3.0
socks                       1.7.1
sphinxcontrib               NA
stack_data                  0.6.2
statsmodels                 0.14.0
texttable                   1.6.7
threadpoolctl               3.2.0
tornado                     6.3.3
traitlets                   5.9.0
typing_extensions           NA
uri_template                NA
urllib3                     2.0.4
wcwidth                     0.2.6
webcolors                   1.13
websocket                   1.6.2
yaml                        6.0.1
zipp                        NA
zmq                         25.1.1
zstandard                   0.19.0
```

Example 6 (unknown):
```unknown
-----
IPython             8.12.0
jupyter_client      8.3.0
jupyter_core        4.12.0
jupyterlab          4.0.5
notebook            7.0.2
-----
Python 3.8.17 (default, Jul  5 2023, 21:04:15) [GCC 11.2.0]
Linux-5.4.0-169-generic-x86_64-with-glibc2.17
-----
Session information updated at 2024-03-25 12:27
```

Example 7 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)

spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/regulon_plot.py")
regulon_plot = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = regulon_plot
spec.loader.exec_module(regulon_plot)
```

Example 8 (python):
```python
sc.settings.set_figure_params(dpi=120)
```

Example 9 (unknown):
```unknown
gene_signatures = utils.read_pkl(path_to_gene_signatures)
gene_signatures.keys()
```

Example 10 (unknown):
```unknown
dict_keys(['compact_specific_up', 'trab_specific_up', 'common_up', 'compact_specific_down', 'trab_specific_down', 'common_down'])
```

Example 11 (python):
```python
sig_genes = []
for sig in target_signatures:
    print(sig)
    sig_genes = sig_genes + gene_signatures[sig]
print(len(sig_genes))
sig_genes[:5]
```

Example 12 (unknown):
```unknown
compact_specific_up
common_up
466
```

Example 13 (unknown):
```unknown
['KIDINS220', 'ANKRD18A', 'MT-ATP6', 'ZNF385D', 'VEGFA']
```

Example 14 (unknown):
```unknown
peak_signatures = utils.read_pkl(path_to_peak_signatures)
peak_signatures.keys()
```

Example 15 (unknown):
```unknown
dict_keys(['compact_up', 'trab_up', 'compact_down', 'trab_down'])
```

Example 16 (python):
```python
sig_peaks = peak_signatures['compact_up']
print(len(sig_peaks))
sig_peaks[:5]
```

Example 17 (unknown):
```unknown
['chr1:240234383_240234883',
 'chr9:93673338_93673838',
 'chr22:42772683_42773183',
 'chr12:48124648_48125148',
 'chr12:46438700_46439200']
```

Example 18 (python):
```python
# scenicplus output
scenic_meta = pd.read_csv(path_to_eRegulon_metadata,index_col=0)
scenic_meta['Region'] = [x.replace('-','_') for x in scenic_meta['Region']]

# select 
scenic_meta.head()
```

Example 19 (python):
```python
# read in adata
# whole data
adata = sc.read_h5ad(path_to_adata_gex)

# subset gene expression data
adata = adata[:,adata.var['modality']=='Gene Expression']

# filter and log-normalise
sc.pp.filter_genes(adata, min_cells=3)
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

print(adata.X.data[:10])
adata
```

Example 20 (python):
```python
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/preprocessing/_simple.py:251: ImplicitModificationWarning: Trying to modify attribute `.var` of view, initializing view as actual.
  adata.var['n_cells'] = number
```

Example 21 (unknown):
```unknown
[1.0619993 1.5653387 1.8985853 1.5653387 2.347629  1.8985853 1.5653387
 1.0619993 1.0619993 1.0619993]
```

Example 22 (unknown):
```unknown
AnnData object with n_obs × n_vars = 16451 × 26187
    obs: 'sangerID', 'combinedID', 'donor', 'region', 'age', 'sex', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'week', 'trimester', 'cycling', 'S_score', 'G2M_score', 'phase', 'fine_grain', 'mid_grain', 'coarse_grain', 'batch_key', 'total_counts', 'total_counts_mt', 'total_counts_ribo', 'n_genes_by_counts', 'pct_counts_mt', 'pct_counts_ribo', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'log1p_total_counts_mt', 'log1p_total_counts_ribo', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', '_indices', '_scvi_batch', '_scvi_labels', 'leiden_MultiVI', 't', 'seg', 'edge', 'milestones', 'age_days', 'week_cont'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id', 'seqnames', 'start', 'end', 'width', 'strand', 'score', 'replicateScoreQuantile', 'groupScoreQuantile', 'Reproducibility', 'GroupReplicate', 'distToGeneStart', 'nearestGene', 'peakType', 'distToTSS', 'nearestTSS', 'GC', 'idx', 'N', 'n_cells'
    uns: '_scvi_manager_uuid', '_scvi_uuid', 'cycling_colors', 'donor_colors', 'draw_graph', 'fine_grain_colors', 'graph', 'leiden', 'leiden_MultiVI_colors', 'milestones_colors', 'neighbors', 'ppt', 'pseudotime_list', 'region_colors', 'umap', 'week_colors', 'log1p'
    obsm: 'MultiVI_latent', 'X_R', 'X_draw_graph_fa', 'X_palantir', 'X_umap', 'X_umap13', '_scvi_extra_categorical_covs', '_scvi_extra_continuous_covs'
    obsp: 'connectivities', 'distances'
```

Example 23 (python):
```python
# read in major TF list
markers = pd.read_csv('./Gene_markers_Maturation.csv')
print(markers.shape)
markers.head()

# get non-TF markers (since this will overlap with centrality calculation based on TF-TF network)
marker_genes = list(markers[markers['Category']!='TF']['Gene'])
print(f'number of marker genes: {len(marker_genes)}')

# known TFs
TFs_known = list(markers[markers['Category']=='TF']['Gene'])
print(f'number of known TFs: {len(TFs_known)}')
TFs_known
```

Example 24 (unknown):
```unknown
(44, 4)
number of marker genes: 35
number of known TFs: 9
```

Example 25 (unknown):
```unknown
['MEF2A',
 'MEF2C',
 'TBX20',
 'NKX2-5',
 'HEY2',
 'PRDM16',
 'PPARA',
 'RORA',
 'THRA']
```

Example 26 (python):
```python
# get expression proportions
exp_mat = utils.get_exp_proportion(adata,groupby='fine_grain', use_rep='X')
exp_genes = list(exp_mat.index[exp_mat[cell_of_intest]>exp_prop_thresh])
len(exp_genes)
```

Example 27 (unknown):
```unknown
'PRDM16' in exp_genes
```

Example 28 (python):
```python
adata_peak = sc.read_h5ad(path_to_adata_peak_tfidf)
adata_peak
```

Example 29 (unknown):
```unknown
AnnData object with n_obs × n_vars = 167022 × 508040
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels'
    var: 'seqnames', 'start', 'end', 'width', 'strand', 'score', 'replicateScoreQuantile', 'groupScoreQuantile', 'Reproducibility', 'GroupReplicate', 'distToGeneStart', 'nearestGene', 'peakType', 'distToTSS', 'nearestTSS', 'GC', 'idx', 'N'
```

Example 30 (python):
```python
# get accessible proportions
access_mat = utils.get_exp_proportion(adata_peak,groupby='fine_grain', use_rep='X')
```

Example 31 (unknown):
```unknown
access_peaks = list(access_mat.index[access_mat[cell_of_intest]>access_prop_thresh])
len(access_peaks)
```

Example 32 (python):
```python
# add columns to scenicplus metadata
# scenic_meta['Gene_isTF'] = scenic_meta['Gene'].isin(list(set(scenic_meta['TF'])))
scenic_meta['TF_isExp'] = scenic_meta['TF'].isin(exp_genes)
scenic_meta['TF_isSignature'] = (scenic_meta['TF'].isin(sig_genes))&(scenic_meta['TF'].isin(exp_genes))
scenic_meta['Region_isAccess'] = scenic_meta['Region'].isin(access_peaks)
scenic_meta['Region_isSignature'] = (scenic_meta['Region'].isin(sig_peaks))&(scenic_meta['Region'].isin(access_peaks))
scenic_meta.head()
```

Example 33 (python):
```python
# save the modified metadata
scenic_meta.to_csv(os.path.join(table_dir,'csvs/scenicplus_eRegulon_metadata_vCM_onlyLeft_modified.csv'))
```

Example 34 (python):
```python
os.path.join(table_dir,'csvs/scenicplus_eRegulon_metadata_vCM_onlyLeft_modified.csv')
```

Example 35 (python):
```python
'/nfs/team205/heart/fetal_ms_figures/tables/csvs/scenicplus_eRegulon_metadata_vCM_onlyLeft_modified.csv'
```

Example 36 (python):
```python
target_genes = sig_genes.copy()
print(len(target_genes))
target_genes[:5]
# removing TFs from the list
```

Example 37 (unknown):
```unknown
['KIDINS220', 'ANKRD18A', 'MT-ATP6', 'ZNF385D', 'VEGFA']
```

Example 38 (python):
```python
# ionchannel = list(pd.read_csv('/nfs/team205/kk18/data/geneset/HGNC/HGNC_GID177_Ion-channels.txt',sep='\t')['Approved symbol'].unique())
# gpcr = list(pd.read_csv('/nfs/team205/kk18/data/geneset/HGNC/HGNC_GID139_G-protein-coupled-receptors.txt',sep='\t')['Approved symbol'].unique())
```

Example 39 (python):
```python
# select interactions

# select interaction which target "target_genes"
scenic_meta_sel = scenic_meta[(scenic_meta['Gene'].isin(target_genes))&\
                               scenic_meta['TF_isExp'] &\
                               scenic_meta['Region_isAccess'] &\
                              (scenic_meta['TF_isSignature']|scenic_meta['Region_isSignature'])
                             ]

# get TFs and target genes in the selected interaction
direct_TFs = list(set(scenic_meta_sel['TF']))
targets = list(set(scenic_meta_sel['Gene']))
# remove TFs already in target genes
TFs_sel = list(set(direct_TFs) - set(targets))

print(len(TFs_sel))
print(f'direct TFs: {direct_TFs}')
print('')

# select higher level interactions 
print('Start looking for higher level TFs')
higher_level_TFs = []
while len(TFs_sel)>0:
    scenic_meta_sel2 = scenic_meta[(scenic_meta['Gene'].isin(TFs_sel))&\
                               scenic_meta['TF_isExp'] &\
                               scenic_meta['Region_isAccess'] &\
                              (scenic_meta['TF_isSignature']|scenic_meta['Region_isSignature'])
                                  ]
    higher_level_TFs = higher_level_TFs + list(set(scenic_meta_sel2['TF']))
    if len(scenic_meta_sel2)>0:
        # concatenate
        scenic_meta_sel = pd.concat([scenic_meta_sel,scenic_meta_sel2])
        scenic_meta_sel = scenic_meta_sel.drop_duplicates()
        # get TFs and target genes in the selected interaction
        TFs_sel = list(set(scenic_meta_sel['TF']))
        targets = list(set(scenic_meta_sel['Gene']))
        # remove TFs already in target genes
        TFs_sel = list(set(TFs_sel) - set(targets))
        print(f'{len(TFs_sel)}: {TFs_sel}')
    else:
        print('no more higher level TFs')
        TFs_sel = []

scenic_meta_sel['direct_TF'] = scenic_meta_sel['TF'].isin(direct_TFs)
higher_level_TFs = list(set(higher_level_TFs)-set(direct_TFs))

# reindex
scenic_meta_sel.index = range(len(scenic_meta_sel))

print("")
print(f'Number of selected interactions: {len(scenic_meta_sel)}')
print(f"Number of TFs: {len(set(scenic_meta_sel['TF']))}")
```

Example 40 (unknown):
```unknown
44
direct TFs: ['TFCP2', 'ZNF280D', 'AHCTF1', 'MEIS2', 'NFATC2', 'ZBTB20', 'UBP1', 'NPAS2', 'KLF12', 'SREBF2', 'MAX', 'HMBOX1', 'ESRRB', 'NR3C1', 'HAND2', 'CREB1', 'PHF21A', 'SOX5', 'ZBTB16', 'THRA', 'MEF2D', 'RORA', 'TGIF2', 'ARID3A', 'KLF6', 'MXI1', 'PLAG1', 'BHLHE40', 'ESRRA', 'NFIB', 'MEF2A', 'FOXK1', 'GTF2IRD1', 'BACH2', 'GATA6', 'MAZ', 'ZNF141', 'NCOA1', 'SMAD3', 'NFIC', 'HIF1A', 'NR4A1', 'LCORL', 'FOXN3', 'ATF1', 'GATA4', 'MEF2C', 'NFE2L1', 'MBNL2', 'YY1', 'HEY2', 'PRDM16', 'ZNF654', 'PPARA', 'TBX20', 'ZNF148', 'ZSCAN18', 'USF2', 'NFIA', 'PBX1', 'FOXO1', 'CUX1']

Start looking for higher level TFs
12: ['MEF2C', 'MAZ', 'AHCTF1', 'IRX3', 'POU2F1', 'ZNF141', 'ELF2', 'SMAD3', 'UBP1', 'HIF1A', 'ZNF586', 'FOXO1']
9: ['MEF2C', 'MAZ', 'AHCTF1', 'POU2F1', 'ZNF141', 'SMAD3', 'UBP1', 'HIF1A', 'FOXO1']
no more higher level TFs

Number of selected interactions: 4832
Number of TFs: 66
```

Example 41 (python):
```python
# get TF-TG dictionary
network = {}
for tf in set(scenic_meta_sel['TF']):
    df_sub = scenic_meta_sel[scenic_meta_sel['TF']==tf]
    network[tf] = list(set(df_sub['Gene']))
print(len(network.keys()))
```

Example 42 (unknown):
```unknown
# signature genes which are not included in the interaction
# just checking
genes_not_incl = set(sig_genes) - set(scenic_meta_sel['Gene'])
genes_not_incl
```

Example 43 (unknown):
```unknown
{'AC008415.1',
 'AC009878.2',
 'AC011346.1',
 'AC018464.1',
 'AC022784.1',
 'AC063944.1',
 'AC097522.2',
 'AC105402.3',
 'AC119673.2',
 'AC124852.1',
 'AC137770.1',
 'ADAMTS9-AS2',
 'AL590004.3',
 'BANCR',
 'CCDC93',
 'CD81-AS1',
 'DANT2',
 'DHRSX',
 'DLEU2',
 'DUSP27',
 'ERO1A',
 'FSD2',
 'GAB2',
 'GALNTL6',
 'GPCPD1',
 'HAND2-AS1',
 'HIF1A-AS3',
 'JPX',
 'LINC00342',
 'LINC00881',
 'LINC00882',
 'LINC00906',
 'LINC01128',
 'LINC02248',
 'LINC02649',
 'LRRC52-AS1',
 'MALAT1',
 'MT-ATP6',
 'MT-ATP8',
 'MT-CO1',
 'MT-CO2',
 'MT-CO3',
 'MT-CYB',
 'MT-ND1',
 'MT-ND2',
 'MT-ND3',
 'MT-ND4',
 'MT-ND4L',
 'MT-ND5',
 'MT-ND6',
 'MTRNR2L1',
 'MTRNR2L12',
 'MTRNR2L8',
 'NEAT1',
 'NPIPB5',
 'PARM1',
 'PDE4DIP',
 'SLC16A1-AS1',
 'SLC16A9',
 'SMG1',
 'SNX30',
 'SPIRE1',
 'SV2C',
 'TRDN-AS1',
 'TRPM7',
 'TSIX',
 'TTN-AS1',
 'XIST',
 'ZNF780B'}
```

Example 44 (python):
```python
# add metrics
scenic_meta_sel['TF_n_linked_region'] = np.nan
scenic_meta_sel['TF_n_target_genes'] = np.nan
scenic_meta_sel['TF_n_target_sig_regions'] = np.nan

for tf in set(scenic_meta_sel['TF']):
    tf_ind = scenic_meta_sel[scenic_meta_sel['TF']==tf].index
    targets = list(set(scenic_meta_sel.loc[tf_ind,'Gene']))
    scenic_meta_sel.loc[tf_ind,'TF_n_target_genes'] = len(targets)
    regions = list(set(scenic_meta_sel.loc[tf_ind,'Region']))
    regions = list(set(regions).intersection(sig_peaks))
    scenic_meta_sel.loc[tf_ind,'TF_n_target_sig_regions'] = len(regions)
    if tf in list(set(scenic_meta_sel['Gene'])):
        gene_ind = scenic_meta_sel[scenic_meta_sel['Gene']==tf].index
        n_region = len(set(scenic_meta_sel.loc[gene_ind,'Region']))
        scenic_meta_sel.loc[tf_ind,'TF_n_linked_region'] = n_region
    else:
        scenic_meta_sel.loc[tf_ind,'TF_n_linked_region'] = 0
        
scenic_meta_sel.head(3)
```

Example 45 (unknown):
```unknown
scenic_meta_sel_TF_metrices = scenic_meta_sel[['TF','direct_TF','TF_n_target_genes','TF_n_target_sig_regions','TF_n_linked_region']]
# df = df[df['direct_TF']]
scenic_meta_sel_TF_metrices.drop_duplicates(inplace=True)
scenic_meta_sel_TF_metrices.set_index('TF',inplace=True)
scenic_meta_sel_TF_metrices.sort_values('TF_n_target_genes',ascending=False)[:30]
```

Example 46 (python):
```python
/tmp/ipykernel_12998/2002270316.py:3: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  scenic_meta_sel_TF_metrices.drop_duplicates(inplace=True)
```

Example 47 (python):
```python
# select interactions those targets are TFs
scenic_meta_sel_TF = scenic_meta_sel[scenic_meta_sel['Gene'].isin(list(set(scenic_meta_sel['TF'])))]

network_TF = {}
for tf in set(scenic_meta_sel_TF['TF']):
    df_sub = scenic_meta_sel_TF[scenic_meta_sel_TF['TF']==tf]
    network_TF[tf] = list(set(df_sub['Gene']))
print(len(network_TF.keys()))
```

Example 48 (unknown):
```unknown
centrality_type = 'eigenvector'
centrality_dict = regulon_plot.get_centrality(network_TF,
                   directed=True,
                   centrality_type=centrality_type, # or 'degree','in_dgree','out_degree'
                   out_df=False # if False output dictionary
                  )
centrality_dict
```

Example 49 (unknown):
```unknown
{'AHCTF1': 5.261440304720337e-13,
 'CUX1': 0.1613225717321188,
 'IRX3': 0.09655257585268091,
 'NFE2L1': 0.16804307262113158,
 'MEIS2': 0.17572204778120903,
 'GATA6': 0.258982861145849,
 'NFATC2': 0.058962883832198366,
 'HAND2': 0.16435825570985327,
 'NPAS2': 0.13296167973192152,
 'FOXK1': 0.09513524891186585,
 'ZBTB16': 0.19239868969576412,
 'PRDM16': 0.26446595865456146,
 'NCOA1': 0.09755874537895777,
 'PPARA': 0.25966245320131714,
 'ZNF148': 0.08260119221864297,
 'NFIC': 0.2550226201637516,
 'THRA': 0.20941008995417718,
 'NR4A1': 0.06297964019703133,
 'FOXN3': 0.052370866840318823,
 'KLF12': 8.6203437952538e-09,
 'SREBF2': 0.1669437114745006,
 'HMBOX1': 0.02829374150769981,
 'ESRRB': 8.6203437952538e-09,
 'PHF21A': 0.02829374150769981,
 'MEF2D': 8.6203437952538e-09,
 'ARID3A': 0.07904814500448452,
 'NFIB': 0.15573842049331146,
 'MEF2A': 0.15625062567673273,
 'GTF2IRD1': 0.17970394495998632,
 'BACH2': 8.6203437952538e-09,
 'ELF2': 0.07447411104479447,
 'ZNF586': 0.047890113112582074,
 'LCORL': 8.6203437952538e-09,
 'GATA4': 0.29973001663194976,
 'ATF1': 8.6203437952538e-09,
 'ZNF654': 0.02829374150769981,
 'TBX20': 0.27305250780466267,
 'ZSCAN18': 0.12383286193783795,
 'PBX1': 0.0217718774266034,
 'TFCP2': 0.1144017967214871,
 'MAX': 0.058524058266206304,
 'CREB1': 0.1174992030532011,
 'RORA': 0.058524058266206304,
 'TGIF2': 0.040827789573739046,
 'MXI1': 0.10796855806768811,
 'KLF6': 0.08828643755436749,
 'YY1': 0.07641069491594496,
 'HEY2': 0.20525881097301638,
 'USF2': 0.05999270466701148,
 'ZNF280D': 0.04311445515750737,
 'NR3C1': 0.20979980852423905,
 'ESRRA': 0.09800823377250537,
 'SOX5': 0.02829373288788216,
 'NFIA': 0.005123344739513594,
 'POU2F1': 5.261440304720337e-13,
 'BHLHE40': 0.04789010449276443,
 'ZBTB20': 0.03341708623932124,
 'MBNL2': 0.02829373288788216,
 'MAZ': 5.261440304720337e-13,
 'ZNF141': 5.261440304720337e-13,
 'SMAD3': 5.261440304720337e-13,
 'PLAG1': 0.04944449980200794,
 'FOXO1': 5.261440304720337e-13}
```

Example 50 (python):
```python
# add to dataframe and save
scenic_meta_sel_TF_metrices.loc[centrality_dict.keys(),f'centaliry_{centrality_type}'] = list(centrality_dict.values())
scenic_meta_sel_TF_metrices[f'centaliry_{centrality_type}'].fillna(0,inplace=True)
print(len(scenic_meta_sel_TF_metrices))
display(scenic_meta_sel_TF_metrices)

# save
# scenic_meta_sel_TF_metrices.to_csv('/nfs/team205/heart/Vitro/TF_selection/foetal_vCM_left_compact-vs-trabeculated_downstream/TF_selection_df_compact-common-up_2Feb2024.csv')
```

Example 51 (unknown):
```unknown
set(marker_genes).intersection(sig_genes)
```

Example 52 (unknown):
```unknown
{'ATP2A2',
 'CACNA1C',
 'CASQ2',
 'CD36',
 'CKMT2',
 'COX6A2',
 'GJA1',
 'MLYCD',
 'PPARGC1A',
 'RYR2',
 'SCN5A'}
```

Example 53 (unknown):
```unknown
# TF-TG network
# utils.save_pkl(network,
#                '/nfs/team205/heart/Vitro/TF_selection/foetal_vCM_left_compact-vs-trabeculated_downstream/TF-TG-network_compact-common-up_2Feb2024.pkl')

# TF-TF network
# utils.save_pkl(network_TF,
#                '/nfs/team205/heart/Vitro/TF_selection/foetal_vCM_left_compact-vs-trabeculated_downstream/TF-TF-network_compact-common-up_2Feb2024.pkl')
```

Example 54 (python):
```python
import networkx as nx
```

Example 55 (unknown):
```unknown
# prepare parameters
size_scale = 7000

node_size_map = {}
for n in network_TF.keys():
    if n in centrality_dict.keys():
        node_size_map[n]=int(centrality_dict[n]*size_scale)
    else: # means no connection with other TFs
        node_size_map[n]=0

node_size_map
```

Example 56 (unknown):
```unknown
{'PPARA': 1817,
 'NFIB': 1090,
 'HEY2': 1436,
 'KLF12': 0,
 'PRDM16': 1851,
 'SREBF2': 1168,
 'ZNF141': 0,
 'CREB1': 822,
 'AHCTF1': 0,
 'CUX1': 1129,
 'GATA4': 2098,
 'TBX20': 1911,
 'LCORL': 0,
 'FOXO1': 0,
 'GATA6': 1812,
 'BHLHE40': 335,
 'ELF2': 521,
 'ZNF586': 335,
 'NFATC2': 412,
 'MAX': 409,
 'MEIS2': 1230,
 'MXI1': 755,
 'NR3C1': 1468,
 'MAZ': 0,
 'NPAS2': 930,
 'RORA': 409,
 'MEF2A': 1093,
 'BACH2': 0,
 'ZNF654': 198,
 'IRX3': 675,
 'NFIC': 1785,
 'HAND2': 1150,
 'SMAD3': 0,
 'MEF2D': 0,
 'GTF2IRD1': 1257,
 'NFE2L1': 1176,
 'KLF6': 618,
 'ESRRB': 0,
 'HMBOX1': 198,
 'POU2F1': 0,
 'SOX5': 198,
 'USF2': 419,
 'PHF21A': 198,
 'ZNF148': 578,
 'NFIA': 35,
 'YY1': 534}
```

Example 57 (unknown):
```unknown
regulon_plot.draw_networkx(
    network=network_TF,
    nodes_color_base='white',
    nodes_to_color=None, # list
    nodes_color_highlight=None,
    nodes_size_base=400,
    nodes_size_map=node_size_map,
     spring_layout_k=1.5,
    spring_layout_scale=1.5,
    figsize =(7,7),
    random_seed=12,
    numpy_random_seed=100
    
)
```

Example 58 (python):
```python
import networkx as nx

# Create a directed graph
G = nx.DiGraph()

# Add nodes and edges
for tf, targets in network.items():
    G.add_node(tf, type='TF', size=node_size_map.get(tf, 1))  # Add TF node
    for tg in targets:
        if tg not in G:
            G.add_node(tg, type='TG', size=500)  # Add TG node if not already added
        G.add_edge(tf, tg)  # Add edge

# Define tf_nodes
tf_nodes = [node for node, attr in G.nodes(data=True) if attr['type'] == 'TF']
tg_nodes = [node for node, attr in G.nodes(data=True) if attr['type'] == 'TG']
tg_nodes.sort()

# Identifying direct and indirect TFs
direct_tfs = set()
for tf, targets in network.items():
    for tg in targets:
        if tg in tg_nodes:
            direct_tfs.add(tf)

indirect_tfs = list(set(tf_nodes) - direct_tfs)  # TFs that are not in direct TFs
direct_tfs = list(direct_tfs)
indirect_tfs.sort()
direct_tfs.sort()

# Manually creating circular positions for indirect and direct TFs, and linear positions for TGs
def create_circular_positions(node_list, center_x, center_y, radius):
    angle_step = 2 * np.pi / len(node_list)
    return {node: (center_x + np.cos(i * angle_step) * radius, center_y + np.sin(i * angle_step) * radius) 
            for i, node in enumerate(node_list)}

# Create circular positions for indirect and direct TFs
direct_tf_pos = create_circular_positions(direct_tfs, 0.5, 0.5, 0.3)

# Linear positions for indirect TFs
indirect_tf_x = [0 for _ in indirect_tfs]  # All nodes have the same x-coordinate
indirect_tf_y = np.linspace(0.8, 0.2, len(indirect_tfs))  # Evenly spaced y-coordinates
indirect_tf_pos = {indirect_tf: (x, y) for indirect_tf, x, y in zip(indirect_tfs, indirect_tf_x, indirect_tf_y)}

# Linear positions for TGs
tg_x = [1 for _ in tg_nodes]  # All TG nodes have the same x-coordinate
tg_y = np.linspace(1, 0, len(tg_nodes))  # Evenly spaced y-coordinates
tg_pos = {tg: (x, y) for tg, x, y in zip(tg_nodes, tg_x, tg_y)}

# Combine positions
pos = {**indirect_tf_pos, **direct_tf_pos, **tg_pos}

# Update node colors: indirect TFs to "lightskyblue"
for node in G.nodes:
    if node in indirect_tfs:
        G.nodes[node]['color'] = 'mistyrose'
    else:
        G.nodes[node]['color'] = 'lightsalmon' if G.nodes[node]['type'] == 'TF' else 'white'

# node colors
node_colors = [data['color'] for node, data in G.nodes(data=True)]

# node edge colors
# change color for the ones highly expressed in foetal vs adult
node_edge_colors = ['red' if node in marker_genes else 'black' for node in G.nodes()]
node_edge_widths = [2 if node in marker_genes else 1 for node in G.nodes()]
# node_shapes = ['s' if node in tg_nodes else 'o' for node in G.nodes()]

# Plotting with updated positions
plt.figure(figsize=(10, 10))
nx.draw(G, pos, with_labels=True, node_color=node_colors, node_size=[G.nodes[node]['size'] for node in G.nodes()], 
        edge_color='grey', alpha=0.9, 
        edgecolors=node_edge_colors, linewidths=node_edge_widths, node_shape="o")
# plt.title("TF-TG Network (Three Levels: Indirect TFs, Direct TFs, TGs)")
plt.show()
```

Example 59 (unknown):
```unknown
l = ['CD36', 'OSBPL2', 'PITPNC1', 'SREBF2', 'SLC27A1']
[x for x in l if x in sig_genes]
```

Example 60 (unknown):
```unknown
['CD36', 'OSBPL2', 'PITPNC1', 'SREBF2']
```

Example 61 (unknown):
```unknown
df = scenic_meta_sel[['Region','TF']]
df.drop_duplicates(inplace=True)
df['TF_count'] = 1
df = df.groupby('Region').sum()
df.head()
```

Example 62 (python):
```python
ad = adata_peak[:,df.index]
ad = ad[ad.obs['fine_grain']==cell_of_intest]
ad
```

Example 63 (python):
```python
df[f'total_count, {cell_of_intest}'] = ad.to_df().sum().reindex(df.index)
plt.scatter(df['TF_count'],df[f'total_count, {cell_of_intest}'])
plt.xlabel('TF_count')
plt.ylabel(f'total_count, {cell_of_intest}')
```

---

## Subcluster¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/RNA/scnRNA_Integration-Annotation/Leukocytes/ImmuneDE.html

**Contents:**
- Subcluster¶

**Examples:**

Example 1 (unknown):
```unknown
#Added the keys for age and sex
```

Example 2 (python):
```python
# Import packages, initial settings

import numpy as np
import matplotlib.pyplot as plt
import scanpy as sc
import pandas as pd
import anndata
import scrublet as scr
import seaborn as sns
import magic
import pybiomart
import six
import sys
sys.modules['sklearn.externals.six'] = six
import scanorama
import harmonypy
import bbknn
import gseapy
from gseapy.plot import barplot, dotplot
import scvelo as scv
import scvi

plt.rcParams['pdf.fonttype'] = 42
plt.rcParams['ps.fonttype'] = 42

sc.settings.verbosity = 3  # verbosity: errors (0), warnings (1), info (2), hints (3)

from matplotlib.colors import LinearSegmentedColormap
cmap = LinearSegmentedColormap.from_list(name='gene_cmap', colors=['lightgrey', 'thistle', 'red', 'darkred']) 

sc.settings.set_figure_params(dpi=80, color_map='viridis', vector_friendly=False,  dpi_save=300, facecolor='white')
sc.logging.print_version_and_date()
#sc.logging.print_versions()
```

Example 3 (unknown):
```unknown
Global seed set to 0
```

Example 4 (unknown):
```unknown
Running Scanpy 1.8.1, on 2023-09-01 22:07.
```

Example 5 (python):
```python
sc.settings.set_figure_params(dpi=300, color_map='viridis', vector_friendly=False,  dpi_save=300, facecolor='white')
```

Example 6 (python):
```python
adata=sc.read('./Objects/Feb28ObjectRaw.h5ad')
print(adata.shape)
```

Example 7 (unknown):
```unknown
(297473, 36601)
```

Example 8 (python):
```python
cell_cycle_genes = ['MCM5','PCNA','TYMS','FEN1','MCM2','MCM4','RRM1','UNG','GINS2','MCM6','CDCA7','DTL','PRIM1','UHRF1','MLF1IP','HELLS','RFC2','RPA2','NASP','RAD51AP1','GMNN','WDR76','SLBP','CCNE2','UBR7','POLD3','MSH2','ATAD2','RAD51','RRM2','CDC45','CDC6','EXO1','TIPIN','DSCC1','BLM','CASP8AP2','USP1','CLSPN','POLA1','CHAF1B','BRIP1','E2F8','HMGB2','CDK1','NUSAP1','UBE2C','BIRC5','TPX2','TOP2A','NDC80','CKS2','NUF2','CKS1B','MKI67','TMPO','CENPF','TACC3','FAM64A','SMC4','CCNB2','CKAP2L','CKAP2','AURKB','BUB1','KIF11','ANP32E','TUBB4B','GTSE1','KIF20B','HJURP','CDCA3','HN1','CDC20','TTK','CDC25C','KIF2C','RANGAP1','NCAPD2','DLGAP5','CDCA2','CDCA8','ECT2','KIF23','HMMR','AURKA','PSRC1','ANLN','LBR','CKAP5','CENPE','CTCF','NEK2','G2E3','GAS2L3','CBX5','CENPA']
s_genes = cell_cycle_genes[:43]
s_genes = [x for x in s_genes if x in adata.var_names]
print('S genes:', s_genes, '\n')
g2m_genes = cell_cycle_genes[43:]
g2m_genes = [x for x in g2m_genes if x in adata.var_names]
print('G2 genes:', g2m_genes, '\n')
cell_cycle_genes = [x for x in cell_cycle_genes if x in adata.var_names]

hb_genes1 = ['HBB','HBD','HBG1','HBG2','HBE1','HBZ','HBM','HBA2','HBA1', 'HBQ1']
hb_genes1 = [x for x in hb_genes1 if x in adata.var_names]
print('Hemoglobin genes set 1:', hb_genes1, '\n')

hb_genes2 = ['AHSP','GYPA','RHAG','GFI1B','KEL','EPB42','ALAS2','TSPO2','GYPB','GYPE','SPTA1','RHCE','KLF1','RHD','SLC4A1','SNX22','SQSTM1','SOD1','FTH1','MAP1LC3B','SLC25A37','TRIM58','HMBS']
hb_genes2 = [x for x in hb_genes2 if x in adata.var_names]
print('Hemoglobin genes set 2:', hb_genes2, '\n')

mito_genes=['MT-ND1','MT-ND2','MT-CO1','MT-CO2','MT-ATP8','MT-ATP6','MT-CO3','MT-ND3','MT-ND4L', 'MT-ND4', 'MT-ND5', 'MT-ND6', 'MT-CYB']
mito_genes = [x for x in mito_genes if x in adata.var_names]
print('Mitochondrial genes:', mito_genes, '\n')

ribo_genes = ['RPLP0','RPLP1','RPLP2','RPL3','RPL3L','RPL4','RPL5','RPL6','RPL7','RPL7A','RPL7L1','RPL8','RPL9','RPL10','RPL10A','RPL10L','RPL11','RPL12','RPL13','RPL13A','RPL14','RPL15','RPL17','RPL18','RPL18A','RPL19','RPL21','RPL22','RPL22L1','RPL23','RPL23A','RPL24','RPL26','RPL26L1','RPL27','RPL27A','RPL28','RPL29','RPL30','RPL31','RPL32','RPL34','RPL35','RPL35A','RPL36','RPL36A','RPL36AL','RPL37','RPL37A','RPL38','RPL39','RPL39L','UBA52','RPL41','RPSA','RPS2','RPS3','RPS3A','RPS4X','RPS4Y1','RPS4Y2','RPS5','RPS6','RPS7','RPS8','RPS9','RPS10','RPS11','RPS12','RPS13','RPS14','RPS15','RPS15A','RPS16','RPS17','RPS18','RPS19','RPS20','RPS21','RPS23','RPS24','RPS25','RPS26','RPS27','RPS27A','RPS27L','RPS28','RPS29','FAU']
ribo_genes = [x for x in ribo_genes if x in adata.var_names]
print('Ribo genes:', ribo_genes, '\n')

stress_genes=['FOS','FOSB','JUN','JUNB','JUND','ZFP36','SOCS3','CEBPD','CEBPB','ATF3','EGR1','HSPA1A','HSPA1B','HSP90AA1','HSP90AB1','HSPA8','HSPB1','HSPE1','IER3','IER2','BTG1','BTG2','DUSP1']
stress_genes = [x for x in stress_genes if x in adata.var_names]
print('Tissue dissociation stress associated genes:', stress_genes, '\n')

FoxProteins=['FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1']
chrY_genes=['SRY', 'RPS4Y1', 'ZFY', 'ZFY-AS1', 'LINC00278', 'PCDH11Y', 'TTTY2B', 'TBL1Y', 'PRKY', 'USP9Y', 'DDX3Y', 'UTY', 'TMSB4Y', 'NLGN4Y', 'TTTY14', 'KDM5D', 'TTTY10', 'EIF1AY', 'PRORY', 'LINC00266-4P']
MembraneProteins1=['AADAT','AAK1','ABCA12','ABCA7','ABCB1','ABCB11','ABCB4','ABCB6','ABCC1','ABCC3','ABCC4','ABCC5','ABCC6','ABCG2','ABHD3','ABI1','ABLIM3','AC004691.2','AC004922.1','AC005324.3','AC005726.1','AC008403.1','AC138696.1','ACIN1','ACKR3','ACLY','ACOT13','ACOT9','ACTN1','ACTR10','ACTR6','ADA','ADAM10','ADAM12','ADAM22','ADAM28','ADAM33','ADAMTS1','ADAMTS2','ADAMTSL4','ADAP1','ADCK5','ADD1','ADD2','ADGRB2','ADGRD1','ADGRG2','ADGRG7','ADGRL2','ADH1A','ADH1B','ADH1C','ADH7','ADK','ADORA1','ADRA1B','ADRM1','ADSS','AFAP1L2','AFDN','AFG1L','AGER','AGRN','AHNAK','AHNAK2','AIFM2','AKAP10','AKAP11','AKAP12','AKR1B10','AKR1B15','AL132780.3','AL139300.1','AL355315.1','AL358113.1','AL645922.1','ALDH3A1','ALK','ALOX15B','ALOXE3','ALPI','ALPP','ALPPL2','ALS2CR12','AMACR','AMDHD1','AMER1','AMER2','AMIGO2','AMOT','AMOTL1','AMOTL2','AMPH','ANK2','ANK3','ANKEF1','ANKFN1','ANKIB1','ANKLE2','ANKRA2','ANKRD13A','ANKRD35','ANKRD42','ANKRD45','ANKS1B','ANO1','ANO10','ANO2','ANO6','ANPEP','ANXA1','ANXA13','ANXA2','ANXA3','AP2M1','APBB1','APBB1IP','APC','APLP1','APMAP','APOC3','APOD','APOLD1','AQP1','AQP10','AQP3','AQP4','AQP5','ARAP1','ARF1','ARHGAP18','ARHGAP19','ARHGAP21','ARHGAP22','ARHGAP23','ARHGAP24','ARHGAP28','ARHGAP29','ARHGAP33','ARHGAP36','ARHGAP40','ARHGAP9','ARHGDIG','ARHGEF1','ARHGEF11','ARHGEF12','ARHGEF15','ARHGEF26','ARHGEF28','ARHGEF40','ARHGEF5','ARID2','ARID4A','ARID5B','ARL13A','ARL15','ARL4C','ARMC12','ARMCX5','ARPC1A','ARPC5','ARRB2','ARRDC4','ARVCF','ASAP1','ASB5','ASGR1','ASGR2','ASH2L','ASIC1','ASIC4','ASPM','ASPSCR1','ATG13','ATG3','ATG7','ATIC','ATOX1','ATP10D','ATP12A','ATP13A3','ATP1B3','ATP23','ATP2B1','ATP2B4','ATP2C2','ATP6V0A2','ATP6V1H','ATP8A2','ATP8B1','ATXN10','ATXN3','ATXN7L3','AVEN','AVPI1','AXIN2','AXL','B2M','B3GALT2','BACE1','BAG3','BAIAP2','BAIAP2L1','BANK1','BASP1','BBOF1','BCAR1','BCAR3','BCL2L12','BCORL1','BCR','BDKRB2','BEND6','BFSP1','BFSP2','BICD2','BIN2','BLNK','BLVRB','BMPR2','BMX','BOC','BOLL','BORCS5','BORCS8-MEF2B','BRK1','BRPF1','BRSK1','BSDC1','BST2','BTK','BTN2A1','BTRC','BUB1B-PAK6','BVES','BZW2','C10orf90','C11orf42','C11orf52','C12orf73','C12orf76','C14orf37','C14orf93','C16orf74','C17orf58','C17orf62','C18orf21','C19orf33','C19orf38','C19orf48','C19orf71','C1orf116','C1orf189','C1orf35','C1orf54','C1orf68','C1orf74','C1QBP','C2orf72','C3orf14','C3orf20','C3orf38','C4orf19','C4orf22','C7orf43','C8orf33','C8orf37','C9orf153','C9orf3','CA14','CA9','CABLES2','CACNA1E','CACNG4','CADM1','CALCRL','CALD1','CALML5','CALN1','CAMK2A','CAMK2B','CAMK2D','CAMK2G','CAP2','CAPS','CASKIN2','CASP4','CAST','CAV3','CAVIN1','CAVIN2','CAVIN3','CAVIN4','CBWD1','CBWD2','CBWD3','CBWD5','CBWD6','CC2D1A','CCBE1','CCDC107','CCDC112','CCDC115','CCDC124','CCDC136','CCDC144NL','CCDC15','CCDC154','CCDC170','CCDC24','CCDC28A','CCDC59','CCDC62','CCDC66','CCDC69','CCDC8','CCDC83','CCDC85C','CCDC93','CCDC97','CCNT2','CCNYL1','CCR3','CCR4','CCT3','CD109','CD163','CD164L2','CD200R1','CD226','CD248','CD274','CD2AP','CD300A','CD300LB','CD33','CD34','CD37','CD38','CD3E','CD3G','CD4','CD44','CD46','CD47','CD48','CD79B','CD81','CD84','CD86','CD8A','CD8B','CD9','CD93','CD99L2','CDAN1','CDC42EP1','CDC42EP3','CDC42EP4','CDC42SE1','CDCA4','CDH1','CDH13','CDH15','CDH17','CDH2','CDH26','CDH3','CDH4','CDH5','CDH6','CDH8','CDIPT','CDK16','CDK5','CDK5RAP2','CDK7','CDRT15','CDV3','CEACAM5','CELSR1','CENPJ','CEP162','CEP295','CEP55','CEP57L1','CEP89','CERCAM','CFAP157','CFAP206','CFAP58','CFAP65','CFB','CFL2','CFLAR','CGN','CGNL1','CHCHD7','CHIC2','CHMP2A','CHRM3','CHRNA1','CHRNA5','CHRNB4','CHRND','CIB1','CIB3','CIP2A','CKAP5','CLCA2','CLCA4','CLCN5','CLCN6','CLDN11','CLDN18','CLDN2','CLDN23','CLDN3','CLDN4','CLDN7','CLDN9','CLEC2A','CLEC4F','CLEC5A','CLECL1','CLIC2','CLIC5','CLTB','CLUL1','CMTM4','CMTM7','CMTR2','CMYA5','CNGA1','CNKSR2','CNNM1','CNNM4','CNP','CNR1','CNTD1','COBL','COBLL1','COG3','COL16A1','COL17A1','COL4A5','COL6A2','COLEC12','COLQ','COMMD4','COMTD1','COPRS','COQ7','CORIN','CORO1B','CORO2A','CORO2B','COX6B2','CPEB3','CPPED1','CPZ','CRB3','CREB5','CRIP1','CRIP2','CRK','CRLF3','CROCC','CRTC1','CRYAB','CRYBG3','CRYL1','CSDE1','CSF1','CSF1R','CSPG4','CST6','CTNNA1','CTNNB1','CTNNBIP1','CTNND1','CTNND2','CTSF','CTSV','CTTN','CTXN1','CUEDC1','CUTA','CX3CL1','CX3CR1','CXADR','CXCR2','CYB5D2','CYBRD1','CYFIP2','CYP20A1','CYP24A1','CYTH2','CYTH4','DAAM1','DAB2','DAG1','DAGLB','DAW1','DBN1','DCAF13','DCAF16','DCBLD2','DCHS1','DCHS2','DCP2','DCT','DCTN6','DCUN1D2','DDAH1','DDB2','DDIAS','DDIT4L','DDR1','DDR2','DDX58','DDX60L','DENND6B','DFFA','DGKG','DGKH','DHDDS','DHPS','DHRS7B','DIAPH1','DIAPH3','DIO2','DIS3L','DLG1','DLG2','DLG5','DLGAP3','DLL3','DMWD','DNAAF1','DNAAF4','DNAH17','DNAJA4','DNAJB13','DNAJB4','DNAJC18','DNAJC5','DNAJC6','DNAJC9','DOC2A','DOCK4','DOK3','DONSON','DPEP1','DPH1','DPPA5','DPYSL2','DQX1','DRD1','DRP2','DSC3','DSEL','DSG2','DSP','DTNA','DUS1L','DUSP15','DXO','DYSF','ECSCR','EDA2R','EDNRA','EDNRB','EEF1AKMT1','EEF2','EFCAB1','EFCAB11','EFCAB2','EFCAB7','EFHC2','EFNB1','EFR3A','EFR3B','EGFR','EHBP1','EHD1','EHD2','EHD3','EHD4','EIF2B2','EIF5','EIF5B','ELFN1','ELMO3','ELN','ELOC','EMP3','ENAH','ENC1','ENG','ENKD1','ENO1','ENO2','ENO3','ENOX1','ENOX2','EPB41','EPB41L1','EPB41L2','EPB41L3','EPB41L4A','EPB41L4B','EPB41L5','EPCAM','EPHA2','EPHA3','EPHB1','EPHB2','EPN1','EPOR','EPS8L2','ERAP1','ERBB2','ERBB3','ERBIN','ERC1','ERCC4','ERMN','ESCO2','ETNK1','ETS2','EVA1A','EVI2B','EXOC1','EXOC3L1','EXOC7','EZR','F11R','F2RL1','F2RL3','FAAP20','FABP2','FABP3','FABP5','FAIM','FAM126A','FAM126B','FAM129A','FAM129B','FAM13A','FAM163B','FAM171A1','FAM193A','FAM205A','FAM210B','FAM222A','FAM45A','FAM47E','FAM47E-STBD1','FAM72A','FAM72B','FAM72C','FAM72D','FANCG','FAS','FASN','FBLIM1','FBLN2','FBLN5','FBLN7','FBP2','FBRSL1','FBXL2','FBXO16','FBXO38','FBXO41','FBXW11','FCAR','FCGBP','FCGR1A','FCGR1B','FCGR2A','FCHSD2','FFAR4','FGD1','FGD5','FGFR2','FGR','FHIT','FHL1','FIBCD1','FIGN','FILIP1','FILIP1L','FKBP2','FLAD1','FLCN','FLNA','FLNB','FLNC','FLOT1','FLOT2','FLRT3','FLT1','FLT4','FLVCR1','FMN2','FMNL2','FMNL3','FNBP1','FOXA2','FOXN3','FP565260.3','FRMD3','FRMD7','FRMD8','FRS2','FSCN1','FTCD','FUT2','FXYD3','FYB1','FYN','FZD1','FZD2','FZD4','FZD6','GAB1','GAB2','GABRA1','GABRA2','GABRA3','GABRA5','GABRG1','GAD1','GADL1','GAGE1','GAGE10','GAGE12C','GAGE12D','GAGE12E','GAGE12F','GAGE12G','GAGE12H','GAGE12J','GAGE13','GAGE2A','GAGE2E','GALNT10','GALR1','GAP43','GAPDH','GAPT','GAPVD1','GAS2','GAS2L1','GAS7','GAS8','GBP4','GCA','GCC1','GCLM','GCSAML-AS1','GFI1B','GFOD1','GFRA3','GHR','GID8','GINM1','GIPC1','GJA1','GJB3','GJB4','GJB6','GJB7','GJC1','GJC2','GLO1','GLP2R','GLRX','GLUL','GMIP','GNAI2','GNAQ','GNAS','GNB1','GNB2','GNB3','GNB4','GNG2','GP9','GPAT3','GPATCH1','GPBP1','GPBP1L1','GPC1','GPC3','GPC4','GPI','GPR1','GPR142','GPR156','GPR157','GPR160','GPR173','GPR20','GPR27','GPR50','GPR63','GPRC5A','GPRIN1','GPRIN2','GPX8','GRAMD1A','GRAMD1B','GRAMD1C','GRAP2','GRB7','GRID2IP','GRIK5','GRK3','GRK5','GRPR','GSDMA','GTF2F1','GTSE1','GUCA1A','GYPA','GYPC','HAPLN3','HARBI1','HAS1','HAUS4','HAUS7','HCAR2','HCAR3','HCFC2','HCK','HCLS1','HCST','HDAC8','HEG1','HELZ','HENMT1','HERC2','HERPUD1','HFE','HIF1A','HIF3A','HINT1','HIP1R','HLA-A','HLA-DPB1','HMGCS1','HMOX1','HOMER3','HOXC5','HOXD4','HPS3','HRASLS5','HRH1','HS6ST3','HSH2D','HSPB1','HSPG2','HTR1B','HTR7','HTRA1','HUNK','HYDIN','HYI','HYLS1','ICAM1','ICAM2','ICOS','ICOSLG','IFFO2','IFIT5','IFNGR1','IFRD1','IFT122','IGFL2','IGSF11','IGSF23','IKZF5','IL10RA','IL12RB1','IL12RB2','IL13','IL13RA2','IL16','IL17RB','IL1R1','IL1RL1','IL21R','IL4R','IL6ST','IL7R','ILDR2','ILK','INPP5A','INSR','INTS12','IP6K2','IPCEF1','IQGAP1','IQGAP2','IREB2','IRF2','IRS1','ISLR','ITGA2B','ITGA3','ITGA4','ITGA7','ITGA8','ITGA9','ITGB1','ITGB2','ITGB3','ITGB4','ITGB5','ITGB6','ITGB7','ITGB8','ITK','ITM2A','ITPKA','ITSN1','ITSN2','IYD','JAG1','JAK2','JAK3','JAM2','JAML','JCAD','JMJD4','JPT2','JUP','KANK1','KANK3','KANSL2','KARS','KATNB1','KCNAB1','KCNB2','KCNH8','KCNJ15','KCNK13','KCNK17','KCNK6','KCNK7','KCNK9','KCNN2','KCNS3','KCNU1','KDF1','KDR','KEL','KIAA0556','KIAA0895','KIAA1324','KIAA1522','KIAA1958','KIF17','KIF19','KIF21A','KIF21B','KIF26A','KIF26B','KISS1R','KIT','KLC1','KLC2','KLF9','KLHDC7B','KLHL41','KLHL6','KLHL7','KLK7','KLRC1','KLRC2','KLRF1','KLRK1','KMT2D','KNDC1','KNSTRN','KNTC1','KPTN','KRT75','KRT77','KRTAP25-1','L1CAM','L3MBTL1','LAIR1','LAMA4','LAMC1','LAMP2','LAMTOR1','LAPTM4A','LASP1','LAT','LAT2','LAX1','LCP1','LCTL','LDB2','LDLRAD3','LENG1','LGALS3','LGALS4','LGALS8','LGR6','LHFPL2','LHX6','LIMA1','LINGO1','LIPJ','LMCD1','LMOD2','LMTK2','LMTK3','LNX1','LNX2','LONRF1','LONRF3','LOXHD1','LPAR1','LPAR3','LPAR4','LPIN1','LPIN2','LRCH2','LRP11','LRP12','LRRC15','LRRC26','LRRC34','LRRC45','LRRC69','LRRC7','LRRC9','LRRFIP2','LRRIQ3','LSMEM2','LSR','LTB4R2','LUZP2','LY6K','LYN','LYPD1','LYST','LZTS1','LZTS2','MACF1','MADD','MAGI1','MAGI3','MAMDC2','MAP1S','MAP2','MAP2K1','MAP2K4','MAP2K7','MAP3K12','MAP3K21','MAP3K6','MAP3K7','MAP4','MAP4K1','MAP4K5','MAP6','MAPK15','MAPK8IP1','MAPT','MARCH10','MARCH7','MARCKS','MARK2','MARVELD2','MAS1L','MAT2B','MBP','MC5R','MCAM','MCEMP1','MCF2L2','MCMBP','MCTS1','MDGA1','ME1','MEF2B','MEIOC','MEPCE','MERTK','MET','METAP1','METAP2','METTL15','METTL25','METTL9','MFAP3L','MFSD1','MFSD2A','MGAT4C','MIB1','MICAL2','MICAL3','MICALL1','MICB','MIEN1','MIGA2','MISP','MKLN1','MKRN3','MLF2','MLKL','MLST8','MMACHC','MMP10','MMP15','MMP24','MMP25','MOAP1','MPHOSPH8','MPHOSPH9','MPI','MPIG6B','MPL','MPP1','MPP5','MPP7','MPZL1','MPZL2','MR1','MRGPRF','MRNIP','MRPL42','MRPL44','MRPL46','MRPS18B','MS4A1','MS4A12','MS4A14','MS4A15','MSN','MSRA','MTCP1','MTFR1L','MTHFR','MTM1','MTPN','MTSS1','MUC1','MUC20','MVD','MYH11','MYH9','MYL4','MYLK','MYO10','MYO1B','MYO1C','MYO1E','MYO1G','MYO3A','MYO5B','MYO9A','MYOF','N4BP2','NAA35','NAB1','NAIF1','NAMPT','NAP1L2','NAPG','NARFL','NCAM1','NCAM2','NCK1','NCKAP1L','NCKAP5','NCKIPSD','NCOA1','NCS1','NDC1','NECTIN2','NECTIN3','NECTIN4','NEDD1','NEDD9','NEO1','NEU1','NEXN','NF2','NFE2L2','NFKBID','NGEF']
MembraneProteins2=['NGFR','NHS','NHSL2','NID2','NINJ2','NKTR','NKX2-8','NLGN1','NLGN3','NLK','NLRX1','NMB','NMBR','NME1','NME2','NMT1','NMT2','NMUR2','NOS1','NOTCH2','NPFFR2','NPL','NPNT','NPR1','NPTN','NPTXR','NR2C1','NRAS','NRCAM','NRP2','NRSN1','NRXN1','NSD1','NSFL1C','NT5E','NTNG1','NTNG2','NTSR1','NUDCD3','NUDT16L1','NUDT9','NUMB','NUP35','NUS1','OAS3','OBSCN','OCA2','OCLN','OCM','OCM2','ODC1','ODR4','OGT','OLFM4','OLIG2','OLR1','OPHN1','OPRK1','OR10P1','OR1N2','OR2C3','OR6B2','OR6B3','OR6C6','OR8S1','OSBPL10','OSBPL6','OSBPL7','OSCAR','OSCP1','OSGEP','OSR2','OTOP2','OTULIN','OXGR1','P3H3','PACSIN2','PACSIN3','PAFAH1B2','PAG1','PAIP1','PAK1','PAK4','PAK6','PALLD','PALM','PALM2','PALM2-AKAP2','PALM3','PALMD','PANX1','PANX2','PAQR8','PARD3','PARD3B','PARD6A','PARD6G','PARP6','PATJ','PAWR','PCBD2','PCDH1','PCDH17','PCDH7','PCDHA1','PCDHA11','PCDHA12','PCDHA4','PCDHA5','PCDHB1','PCDHB11','PCDHB12','PCDHB16','PCDHB4','PCDHB5','PCDHGA1','PCDHGA10','PCDHGA11','PCDHGA12','PCDHGA2','PCDHGA3','PCDHGA4','PCDHGA5','PCDHGA6','PCDHGA7','PCDHGA8','PCDHGA9','PCDHGB1','PCDHGB2','PCDHGB3','PCDHGB4','PCDHGB5','PCDHGB6','PCDHGB7','PCDHGC3','PCDHGC4','PCDHGC5','PCLO','PCMTD1','PDAP1','PDCD7','PDE3A','PDE4A','PDE4D','PDE9A','PDGFC','PDGFRA','PDHX','PDLIM1','PDLIM5','PDPN','PDZD2','PDZD8','PEAR1','PEBP1','PFDN1','PGAP3','PGLYRP2','PGLYRP4','PGRMC2','PHACTR1','PHACTR4','PHF20L1','PHGDH','PHLDB2','PHLDB3','PHOSPHO1','PHPT1','PI3','PI4K2A','PI4KA','PIEZO2','PIGB','PIGN','PIGW','PIGX','PIK3AP1','PIK3CG','PIK3R6','PIP4K2A','PIP4K2C','PIP5K1A','PIRT','PITPNM3','PKD2L2','PKN1','PKN2','PKP1','PKP2','PKP3','PKP4','PLAA','PLAUR','PLCB1','PLCB2','PLCD3','PLCD4','PLEKHA4','PLEKHA6','PLEKHA7','PLEKHG3','PLEKHG4','PLEKHG6','PLEKHH2','PLEKHO2','PLIN4','PLPP1','PLPPR5','PLS3','PLSCR1','PLVAP','PLXNA3','PLXNA4','PNCK','PNISR','PODXL','POLE','POU2F3','PPEF1','PPFIBP1','PPIP5K1','PPL','PPM1A','PPM1F','PPP1CA','PPP1CB','PPP1R12B','PPP1R16A','PPP1R16B','PPP1R18','PPP1R27','PPP1R9A','PPP2R1B','PPP3R1','PPP3R2','PPP4C','PPP6R3','PPTC7','PRAME','PRC1','PRDX6','PRIMA1','PRKCA','PRKCE','PRKCH','PRKCZ','PRKD1','PROM2','PROSER2','PRR15','PRR32','PRR7','PRRC2A','PRRG1','PRRT2','PRRT3','PRRT4','PRTG','PRX','PSCA','PSD','PSD2','PSEN1','PSMC5','PSMC6','PSMD9','PSTPIP2','PTAFR','PTCHD4','PTK2B','PTK6','PTPN12','PTPN21','PTPN22','PTPN4','PTPRB','PTPRG','PTPRK','PTPRM','PTPRR','PTPRS','PTX3','PURG','PVALB','PVR','PXDC1','PXK','PYGL','PYGO2','PYM1','RAB13','RAB23','RAB25','RAB26','RAB27B','RAB3B','RAB3GAP2','RAB43','RAB8A','RABEPK','RAD51C','RAD52','RAI2','RALA','RALGAPA2','RAPGEF2','RAPH1','RAPSN','RASA4','RASA4B','RASAL1','RASAL2','RASAL3','RASD1','RASD2','RASGRP1','RASGRP2','RASGRP4','RASSF2','RASSF3','RASSF4','RBM46','RC3H1','RCCD1','RCSD1','RD3L','RDX','RECK','RELL1','REPS1','RET','RFLNB','RFPL1','RFPL2','RFPL3','RGP1','RGS10','RGS14','RGS19','RGS3','RGS7BP','RGS9BP','RHCE','RHD','RHEX','RHOA','RHOBTB2','RHOJ','RHOXF2','RHOXF2B','RIC8A','RIC8B','RIF1','RILPL1','RIMBP3','RIMBP3B','RIMBP3C','RIMS2','RIOK2','RIOK3','RIPK1','RND1','RNF114','RNF145','RNF146','RNF149','RNF165','RNF166','RNF39','ROBO1','ROBO3','ROBO4','ROM1','RP2','RPE','RPEL1','RPGRIP1L','RPIA','RPL36A','RPL36A-HNRNPH2','RPL36AL','RPSA','RPUSD1','RRAD','RRP12','RRP7A','RSBN1L','RSC1A1','RSPH3','RTL3','RTL4','RTN4R','RTN4RL1','RYR2','S100A13','S100A14','S100A16','S100A2','S100A3','S100A4','S100A6','S100A9','SAMD4A','SAMHD1','SAMSN1','SAPCD1','SASH3','SC5D','SCAMP1','SCAMP4','SCEL','SCHIP1','SCIN','SCN1A','SCN5A','SCN8A','SCN9A','SCNN1D','SCNN1G','SCRIB','SCUBE3','SDC2','SDC4','SDE2','SDHB','SEC16B','SEMA3C','SEMA3D','SEMA4B','SEMA4F','SEMA6C','SEMA6D','SEPT2','SEPT3','SERINC2','SERINC5','SERPINB3','SFSWAP','SGCE','SGK494','SH3BGR','SH3D21','SHANK1','SHANK3','SHARPIN','SHISA6','SHKBP1','SHROOM2','SHTN1','SIGLEC10','SIGLEC14','SIGLEC6','SIGLEC7','SIN3B','SIPA1','SIPA1L1','SIPA1L3','SKAP1','SLAMF6','SLC10A5','SLC12A2','SLC13A3','SLC13A5','SLC14A1','SLC16A1','SLC16A10','SLC16A2','SLC16A3','SLC16A7','SLC16A9','SLC17A5','SLC19A1','SLC19A3','SLC1A5','SLC1A6','SLC22A16','SLC23A3','SLC24A4','SLC26A9','SLC27A1','SLC28A1','SLC2A1','SLC2A11','SLC2A12','SLC2A14','SLC2A2','SLC2A3','SLC30A1','SLC31A2','SLC34A1','SLC34A2','SLC35F5','SLC35G2','SLC36A4','SLC38A5','SLC38A6','SLC38A7','SLC39A10','SLC39A14','SLC39A2','SLC39A6','SLC3A2','SLC40A1','SLC41A3','SLC43A1','SLC43A3','SLC44A2','SLC45A4','SLC46A1','SLC46A3','SLC4A1AP','SLC4A2','SLC4A7','SLC4A8','SLC5A12','SLC5A3','SLC5A6','SLC5A7','SLC6A6','SLC7A14','SLC7A2','SLC7A5','SLC7A6','SLC8A1','SLC9A1','SLC9A2','SLC9A3','SLC9A3R1','SLC9A3R2','SLC9A4','SLC9A5','SLC9B2','SLC9C2','SLCO1A2','SLCO1B3','SLCO4A1','SLCO4C1','SLCO5A1','SLIT1','SLIT3','SLITRK6','SLK','SLX4','SMAGP','SMAP1','SMOC1','SMPD2','SMPX','SNAP23','SNAP25','SNRK','SNTB2','SNTG2','SNX15','SNX32','SNX9','SOCS3','SOCS5','SOD1','SORBS1','SORCS1','SORT1','SOX11','SPAG16','SPANXN4','SPART','SPATA12','SPATC1L','SPERT','SPG11','SPHK1','SPINK13','SPINT1','SPINT2','SPN','SPPL2B','SPPL3','SPRED3','SPRYD3','SPTBN2','SRC','SRCIN1','SRD5A3','SRPK1','SRPRB','SRRM3','SSFA2','SSH1','SSH3','SSTR1','SSTR5','SSX2IP','ST8SIA5','STAB1','STAC','STAC2','STAMBP','STAMBPL1','STARD4','STAT2','STEAP4','STIL','STIP1','STK10','STK16','STK17A','STK17B','STOM','STOML2','STOML3','STX18','STX4','STXBP4','STYK1','SUCLG1','SUCLG2','SUGT1','SUMF2','SUPT6H','SUSD2','SVIL','SWAP70','SYCP2','SYK','SYMPK','SYNC','SYNGR3','SYNPO2L','SYPL2','SYT4','SYTL2','SYVN1','TACC2','TACR2','TACSTD2','TASP1','TAZ','TBC1D10A','TBC1D2','TBC1D24','TBC1D30','TBC1D9B','TBXA2R','TCAF2','TCAF2C','TCHP','TCIM','TDG','TDO2','TDP1','TEC','TEDC2','TEK','TES','TEX14','TEX9','TGFBR1','TGFBR2','TGM1','TGM2','THBS1','THBS2','THEGL','THY1','TIAM1','TICAM1','TIGD7','TIMD4','TJP1','TJP2','TJP3','TLN1','TLR4','TM2D2','TM4SF20','TM4SF4','TM4SF5','TMC5','TMED8','TMEM101','TMEM107','TMEM108','TMEM121B','TMEM139','TMEM141','TMEM164','TMEM176B','TMEM183A','TMEM186','TMEM2','TMEM218','TMEM222','TMEM229A','TMEM245','TMEM251','TMEM254','TMEM258','TMEM266','TMEM268','TMEM50A','TMEM63B','TMPRSS15','TMPRSS2','TMTC2','TNFRSF10D','TNFRSF11A','TNFRSF12A','TNFRSF21','TNFRSF25','TNFRSF8','TNFRSF9','TNFSF13B','TNK1','TNKS1BP1','TNNI2','TOGARAM2','TOLLIP','TOM1','TP53BP2','TPD52L1','TPRN','TRAF7','TRAIP','TRAPPC13','TRAT1','TRDN','TREML1','TRIB1','TRIM16','TRIM16L','TRIM38','TRIM4','TRIM44','TRIM54','TRIM71','TRIML2','TRIP6','TRMT1','TRMT10B','TRMT13','TRMT6','TRO','TRPM1','TRPM4','TRPM7','TRPV2','TRPV3','TSG101','TSGA13','TSHZ3','TSPAN15','TSPAN6','TSPO2','TSR2','TSSK4','TTC17','TTC23','TTC7B','TTLL12','TTLL5','TULP3','TWF2','TXNDC11','TXNIP','TYROBP','TYW3','UAP1','UBA3','UBA52','UBAC1','UBALD1','UBAP1','UBASH3B','UBE2A','UBE2B','UBE2C','UBE2D1','UBE2D2','UBE2D3','UBE2D4','UBE2K','UBE2QL1','UBE2S','UBE3C','UBQLN2','UBR2','UBTD1','UCP3','ULBP1','ULBP2','UNC5A','UNC79','UQCC1','USP14','USP15','USP2','USP20','USP21','USP4','USP53','UTP20','UTRN','VAMP5','VAMP8','VANGL1','VASP','VCAM1','VCPIP1','VIL1','VPS13B','VPS36','VPS50','VPS8','VSIG4','VTCN1','VWA5B1','VWA7','WAS','WASF2','WBP4','WDFY3','WDPCP','WDR1','WDR12','WDR13','WDR53','WDR6','WDR66','WIPF1','WWP1','XIAP','XIRP1','XIRP2','XPC','XPNPEP2','XPO6','XRN1','XYLT1','YES1','YIPF1','YIPF4','YOD1','YTHDC1','ZBED3','ZBTB33','ZBTB8OS','ZDHHC2','ZDHHC20','ZDHHC22','ZDHHC5','ZFP41','ZFP90','ZFYVE28','ZGPAT','ZMYND10','ZNF10','ZNF18','ZNF185','ZNF2','ZNF20','ZNF227','ZNF229','ZNF236','ZNF276','ZNF277','ZNF337','ZNF397','ZNF407','ZNF497','ZNF503','ZNF510','ZNF536','ZNF571','ZNF598','ZNF599','ZNF646','ZNF655','ZNF688','ZNF706','ZNF717','ZNF783','ZNF79','ZNF839','ZSCAN22','ZSCAN29','ZSWIM1','ZSWIM8','ZYX']
MembraneProteins=MembraneProteins1+MembraneProteins2

TFs1=['AC008770.3','AC023509.3','AC092835.1','AC138696.1','ADNP','ADNP2','AEBP1','AEBP2','AHCTF1','AHDC1','AHR','AHRR','AIRE','AKAP8','AKAP8L','AKNA','ALX1','ALX3','ALX4','ANHX','ANKZF1','AR','ARGFX','ARHGAP35','ARID2','ARID3A','ARID3B','ARID3C','ARID5A','ARID5B','ARNT','ARNT2','ARNTL','ARNTL2','ARX','ASCL1','ASCL2','ASCL3','ASCL4','ASCL5','ASH1L','ATF1','ATF2','ATF3','ATF4','ATF5','ATF6','ATF6B','ATF7','ATMIN','ATOH1','ATOH7','ATOH8','BACH1','BACH2','BARHL1','BARHL2','BARX1','BARX2','BATF','BATF2','BATF3','BAZ2A','BAZ2B','BBX','BCL11A','BCL11B','BCL6','BCL6B','BHLHA15','BHLHA9','BHLHE22','BHLHE23','BHLHE40','BHLHE41','BNC1','BNC2','BORCS8-MEF2B','BPTF','BRF2','BSX','C11orf95','CAMTA1','CAMTA2','CARF','CASZ1','CBX2','CC2D1A','CCDC169-SOHLH2','CCDC17','CDC5L','CDX1','CDX2','CDX4','CEBPA','CEBPB','CEBPD','CEBPE','CEBPG','CEBPZ','CENPA','CENPB','CENPBD1','CENPS','CENPT','CENPX','CGGBP1','CHAMP1','CHCHD3','CIC','CLOCK','CPEB1','CPXCR1','CREB1','CREB3','CREB3L1','CREB3L2','CREB3L3','CREB3L4','CREB5','CREBL2','CREBZF','CREM','CRX','CSRNP1','CSRNP2','CSRNP3','CTCF','CTCFL','CUX1','CUX2','CXXC1','CXXC4','CXXC5','DACH1','DACH2','DBP','DBX1','DBX2','DDIT3','DEAF1','DLX1','DLX2','DLX3','DLX4','DLX5','DLX6','DMBX1','DMRT1','DMRT2','DMRT3','DMRTA1','DMRTA2','DMRTB1','DMRTC2','DMTF1','DNMT1','DNTTIP1','DOT1L','DPF1','DPF3','DPRX','DR1','DRAP1','DRGX','DUX1','DUX3','DUX4','DUXA','DZIP1','E2F1','E2F2','E2F3','E2F4','E2F5','E2F6','E2F7','E2F8','E4F1','EBF1','EBF2','EBF3','EBF4','EEA1','EGR1','EGR2','EGR3','EGR4','EHF','ELF1','ELF2','ELF3','ELF4','ELF5','ELK1','ELK3','ELK4','EMX1','EMX2','EN1','EN2','EOMES','EPAS1','ERF','ERG','ESR1','ESR2','ESRRA','ESRRB','ESRRG','ESX1','ETS1','ETS2','ETV1','ETV2','ETV3','ETV3L','ETV4','ETV5','ETV6','ETV7','EVX1','EVX2','FAM170A','FAM200B','FBXL19','FERD3L','FEV','FEZF1','FEZF2','FIGLA','FIZ1','FLI1','FLYWCH1','FOS','FOSB','FOSL1','FOSL2','FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXN4','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1','GABPA','GATA1','GATA2','GATA3','GATA4','GATA5','GATA6','GATAD2A','GATAD2B','GBX1','GBX2','GCM1','GCM2','GFI1','GFI1B','GLI1','GLI2','GLI3','GLI4','GLIS1','GLIS2','GLIS3','GLMP','GLYR1','GMEB1','GMEB2','GPBP1','GPBP1L1','GRHL1','GRHL2','GRHL3','GSC','GSC2','GSX1','GSX2','GTF2B','GTF2I','GTF2IRD1','GTF2IRD2','GTF2IRD2B','GTF3A','GZF1','HAND1','HAND2','HBP1','HDX','HELT','HES1','HES2','HES3','HES4','HES5','HES6','HES7','HESX1','HEY1','HEY2','HEYL','HHEX','HIC1','HIC2','HIF1A','HIF3A','HINFP','HIVEP1','HIVEP2','HIVEP3','HKR1','HLF','HLX','HMBOX1','HMG20A','HMG20B','HMGA1','HMGA2','HMGN3','HMX1','HMX2','HMX3','HNF1A','HNF1B','HNF4A','HNF4G','HOMEZ','HOXA1','HOXA10','HOXA11','HOXA13','HOXA2','HOXA3','HOXA4','HOXA5','HOXA6','HOXA7','HOXA9','HOXB1','HOXB13','HOXB2','HOXB3','HOXB4','HOXB5','HOXB6','HOXB7','HOXB8','HOXB9','HOXC10','HOXC11','HOXC12','HOXC13','HOXC4','HOXC5','HOXC6','HOXC8','HOXC9','HOXD1','HOXD10','HOXD11','HOXD12','HOXD13','HOXD3','HOXD4','HOXD8','HOXD9','HSF1','HSF2','HSF4','HSF5','HSFX1','HSFX2','HSFY1','HSFY2','IKZF1','IKZF2','IKZF3','IKZF4','IKZF5','INSM1','INSM2','IRF1','IRF2','IRF3','IRF4','IRF5','IRF6','IRF7','IRF8','IRF9','IRX1','IRX2','IRX3','IRX4','IRX5','IRX6','ISL1','ISL2','ISX','JAZF1','JDP2','JRK','JRKL','JUN','JUNB','JUND','KAT7','KCMF1','KCNIP3','KDM2A','KDM2B','KDM5B','KIN','KLF1','KLF10','KLF11','KLF12','KLF13','KLF14','KLF15','KLF16','KLF17','KLF2','KLF3','KLF4','KLF5','KLF6','KLF7','KLF8','KLF9','KMT2A','KMT2B','L3MBTL1','L3MBTL3','L3MBTL4','LBX1','LBX2','LCOR','LCORL','LEF1','LEUTX','LHX1','LHX2','LHX3','LHX4','LHX5','LHX6','LHX8','LHX9','LIN28A','LIN28B','LIN54','LMX1A','LMX1B','LTF','LYL1','MAF','MAFA','MAFB','MAFF','MAFG','MAFK','MAX','MAZ','MBD1','MBD2','MBD3','MBD4','MBD6','MBNL2','MECOM','MECP2','MEF2A','MEF2B','MEF2C','MEF2D','MEIS1','MEIS2','MEIS3','MEOX1','MEOX2','MESP1','MESP2','MGA','MITF','MIXL1','MKX','MLX','MLXIP','MLXIPL','MNT','MNX1','MSANTD1','MSANTD3','MSANTD4','MSC','MSGN1','MSX1','MSX2','MTERF1','MTERF2','MTERF3','MTERF4','MTF1','MTF2','MXD1','MXD3','MXD4','MXI1','MYB','MYBL1','MYBL2','MYC','MYCL','MYCN','MYF5','MYF6','MYNN','MYOD1','MYOG','MYPOP','MYRF','MYRFL','MYSM1','MYT1','MYT1L','MZF1','NACC2','NAIF1','NANOG','NANOGNB','NANOGP8','NCOA1','NCOA2','NCOA3','NEUROD1','NEUROD2','NEUROD4','NEUROD6','NEUROG1','NEUROG2','NEUROG3','NFAT5','NFATC1','NFATC2','NFATC3','NFATC4','NFE2','NFE2L1','NFE2L2','NFE2L3','NFE4','NFIA','NFIB','NFIC','NFIL3','NFIX','NFKB1','NFKB2','NFX1','NFXL1','NFYA','NFYB','NFYC','NHLH1','NHLH2','NKRF','NKX1-1','NKX1-2','NKX2-1','NKX2-2','NKX2-3','NKX2-4','NKX2-5','NKX2-6','NKX2-8','NKX3-1','NKX3-2','NKX6-1','NKX6-2','NKX6-3','NME2','NOBOX','NOTO','NPAS1','NPAS2','NPAS3','NPAS4','NR0B1','NR1D1','NR1D2','NR1H2','NR1H3','NR1H4','NR1I2','NR1I3','NR2C1','NR2C2','NR2E1','NR2E3','NR2F1','NR2F2','NR2F6','NR3C1','NR3C2','NR4A1','NR4A2','NR4A3','NR5A1','NR5A2','NR6A1','NRF1','NRL','OLIG1','OLIG2','OLIG3','ONECUT1','ONECUT2','ONECUT3','OSR1','OSR2','OTP','OTX1','OTX2','OVOL1','OVOL2','OVOL3','PA2G4','PATZ1','PAX1','PAX2','PAX3','PAX4','PAX5','PAX6','PAX7','PAX8','PAX9','PBX1','PBX2','PBX3','PBX4','PCGF2','PCGF6','PDX1','PEG3','PGR','PHF1','PHF19 ','PHF20','PHF21A','PHOX2A','PHOX2B','PIN1','PITX1','PITX2','PITX3','PKNOX1','PKNOX2','PLAG1','PLAGL1','PLAGL2','PLSCR1','POGK','POU1F1','POU2AF1','POU2F1','POU2F2','POU2F3','POU3F1','POU3F2','POU3F3','POU3F4','POU4F1','POU4F2','POU4F3','POU5F1','POU5F1B','POU5F2','POU6F1','POU6F2','PPARA','PPARD','PPARG','PRDM1','PRDM10','PRDM12','PRDM13','PRDM14','PRDM15','PRDM16','PRDM2','PRDM4','PRDM5','PRDM6','PRDM8','PRDM9','PREB','PRMT3','PROP1','PROX1','PROX2','PRR12','PRRX1','PRRX2','PTF1A','PURA','PURB','PURG','RAG1','RARA','RARB','RARG','RAX','RAX2','RBAK','RBCK1','RBPJ','RBPJL','RBSN','REL','RELA','RELB','REPIN1','REST','REXO4','RFX1','RFX2','RFX3','RFX4','RFX5','RFX6','RFX7','RFX8','RHOXF1','RHOXF2','RHOXF2B','RLF','RORA','RORB','RORC','RREB1','RUNX1','RUNX2','RUNX3','RXRA','RXRB','RXRG','SAFB','SAFB2','SALL1','SALL2','SALL3','SALL4','SATB1','SATB2','SCMH1','SCML4','SCRT1','SCRT2','SCX','SEBOX','SETBP1','SETDB1','SETDB2','SGSM2','SHOX','SHOX2','SIM1','SIM2','SIX1','SIX2','SIX3','SIX4','SIX5','SIX6','SKI','SKIL','SKOR1','SKOR2','SLC2A4RG','SMAD1','SMAD3','SMAD4','SMAD5','SMAD9','SMYD3','SNAI1','SNAI2','SNAI3','SNAPC2','SNAPC4','SNAPC5','SOHLH1','SOHLH2','SON','SOX1','SOX10','SOX11']
TFs2=['SOX12','SOX13','SOX14','SOX15','SOX17','SOX18','SOX2','SOX21','SOX3','SOX30','SOX4','SOX5','SOX6','SOX7','SOX8','SOX9','SP1','SP100','SP110','SP140','SP140L','SP2','SP3','SP4','SP5','SP6','SP7','SP8','SP9','SPDEF','SPEN','SPI1','SPIB','SPIC','SPZ1','SRCAP','SREBF1','SREBF2','SRF','SRY','ST18','STAT1','STAT2','STAT3','STAT4','STAT5A','STAT5B','STAT6','T','TAL1','TAL2','TBP','TBPL1','TBPL2','TBR1','TBX1','TBX10','TBX15','TBX18','TBX19','TBX2','TBX20','TBX21','TBX22','TBX3','TBX4','TBX5','TBX6','TCF12','TCF15','TCF20','TCF21','TCF23','TCF24','TCF3','TCF4','TCF7','TCF7L1','TCF7L2','TCFL5','TEAD1','TEAD2','TEAD3','TEAD4','TEF','TERB1','TERF1','TERF2','TET1','TET2','TET3','TFAP2A','TFAP2B','TFAP2C','TFAP2D','TFAP2E','TFAP4','TFCP2','TFCP2L1','TFDP1','TFDP2','TFDP3','TFE3','TFEB','TFEC','TGIF1','TGIF2','TGIF2LX','TGIF2LY','THAP1','THAP10','THAP11','THAP12','THAP2','THAP3','THAP4','THAP5','THAP6','THAP7','THAP8','THAP9','THRA','THRB','THYN1','TIGD1','TIGD2','TIGD3','TIGD4','TIGD5','TIGD6','TIGD7','TLX1','TLX2','TLX3','TMF1','TOPORS','TP53','TP63','TP73','TPRX1','TRAFD1','TRERF1','TRPS1','TSC22D1','TSHZ1','TSHZ2','TSHZ3','TTF1','TWIST1','TWIST2','UBP1','UNCX','USF1','USF2','USF3','VAX1','VAX2','VDR','VENTX','VEZF1','VSX1','VSX2','WIZ','WT1','XBP1','XPA','YBX1','YBX2','YBX3','YY1','YY2','ZBED1','ZBED2','ZBED3','ZBED4','ZBED5','ZBED6','ZBED9','ZBTB1','ZBTB10','ZBTB11','ZBTB12','ZBTB14','ZBTB16','ZBTB17','ZBTB18','ZBTB2','ZBTB20','ZBTB21','ZBTB22','ZBTB24','ZBTB25','ZBTB26','ZBTB3','ZBTB32','ZBTB33','ZBTB34','ZBTB37','ZBTB38','ZBTB39','ZBTB4','ZBTB40','ZBTB41','ZBTB42','ZBTB43','ZBTB44','ZBTB45','ZBTB46','ZBTB47','ZBTB48','ZBTB49','ZBTB5','ZBTB6','ZBTB7A','ZBTB7B','ZBTB7C','ZBTB8A','ZBTB8B','ZBTB9','ZC3H8','ZEB1','ZEB2','ZFAT','ZFHX2','ZFHX3','ZFHX4','ZFP1','ZFP14','ZFP2','ZFP28','ZFP3','ZFP30','ZFP37','ZFP41','ZFP42','ZFP57','ZFP62','ZFP64','ZFP69','ZFP69B','ZFP82','ZFP90','ZFP91','ZFP92','ZFPM1','ZFPM2','ZFX','ZFY','ZGLP1','ZGPAT','ZHX1','ZHX2','ZHX3','ZIC1','ZIC2','ZIC3','ZIC4','ZIC5','ZIK1','ZIM2','ZIM3','ZKSCAN1','ZKSCAN2','ZKSCAN3','ZKSCAN4','ZKSCAN5','ZKSCAN7','ZKSCAN8','ZMAT1','ZMAT4','ZNF10','ZNF100','ZNF101','ZNF107','ZNF112','ZNF114','ZNF117','ZNF12','ZNF121','ZNF124','ZNF131','ZNF132','ZNF133','ZNF134','ZNF135','ZNF136','ZNF138','ZNF14','ZNF140','ZNF141','ZNF142','ZNF143','ZNF146','ZNF148','ZNF154','ZNF155','ZNF157','ZNF16','ZNF160','ZNF165','ZNF169','ZNF17','ZNF174','ZNF175','ZNF177','ZNF18','ZNF180','ZNF181','ZNF182','ZNF184','ZNF189','ZNF19','ZNF195','ZNF197','ZNF2','ZNF20','ZNF200','ZNF202','ZNF205','ZNF207','ZNF208','ZNF211','ZNF212','ZNF213','ZNF214','ZNF215','ZNF217','ZNF219','ZNF22','ZNF221','ZNF222','ZNF223','ZNF224','ZNF225','ZNF226','ZNF227','ZNF229','ZNF23','ZNF230','ZNF232','ZNF233','ZNF234','ZNF235','ZNF236','ZNF239','ZNF24','ZNF248','ZNF25','ZNF250','ZNF251','ZNF253','ZNF254','ZNF256','ZNF257','ZNF26','ZNF260','ZNF263','ZNF264','ZNF266','ZNF267','ZNF268','ZNF273','ZNF274','ZNF275','ZNF276','ZNF277','ZNF28','ZNF280A','ZNF280B','ZNF280C','ZNF280D','ZNF281','ZNF282','ZNF283','ZNF284','ZNF285','ZNF286A','ZNF286B','ZNF287','ZNF292','ZNF296','ZNF3','ZNF30','ZNF300','ZNF302','ZNF304','ZNF311','ZNF316','ZNF317','ZNF318','ZNF319','ZNF32','ZNF320','ZNF322','ZNF324','ZNF324B','ZNF326','ZNF329','ZNF331','ZNF333','ZNF334','ZNF335','ZNF337','ZNF33A','ZNF33B','ZNF34','ZNF341','ZNF343','ZNF345','ZNF346','ZNF347','ZNF35','ZNF350','ZNF354A','ZNF354B','ZNF354C','ZNF358','ZNF362','ZNF365','ZNF366','ZNF367','ZNF37A','ZNF382','ZNF383','ZNF384','ZNF385A','ZNF385B','ZNF385C','ZNF385D','ZNF391','ZNF394','ZNF395','ZNF396','ZNF397','ZNF398','ZNF404','ZNF407','ZNF408','ZNF41','ZNF410','ZNF414','ZNF415','ZNF416','ZNF417','ZNF418','ZNF419','ZNF420','ZNF423','ZNF425','ZNF426','ZNF428','ZNF429','ZNF43','ZNF430','ZNF431','ZNF432','ZNF433','ZNF436','ZNF438','ZNF439','ZNF44','ZNF440','ZNF441','ZNF442','ZNF443','ZNF444','ZNF445','ZNF446','ZNF449','ZNF45','ZNF451','ZNF454','ZNF460','ZNF461','ZNF462','ZNF467','ZNF468','ZNF469','ZNF470','ZNF471','ZNF473','ZNF474','ZNF479','ZNF48','ZNF480','ZNF483','ZNF484','ZNF485','ZNF486','ZNF487','ZNF488','ZNF490','ZNF491','ZNF492','ZNF493','ZNF496','ZNF497','ZNF500','ZNF501','ZNF502','ZNF503','ZNF506','ZNF507','ZNF510','ZNF511','ZNF512','ZNF512B','ZNF513','ZNF514','ZNF516','ZNF517','ZNF518A','ZNF518B','ZNF519','ZNF521','ZNF524','ZNF525','ZNF526','ZNF527','ZNF528','ZNF529','ZNF530','ZNF532','ZNF534','ZNF536','ZNF540','ZNF541','ZNF543','ZNF544','ZNF546','ZNF547','ZNF548','ZNF549','ZNF550','ZNF551','ZNF552','ZNF554','ZNF555','ZNF556','ZNF557','ZNF558','ZNF559','ZNF560','ZNF561','ZNF562','ZNF563','ZNF564','ZNF565','ZNF566','ZNF567','ZNF568','ZNF569','ZNF57','ZNF570','ZNF571','ZNF572','ZNF573','ZNF574','ZNF575','ZNF576','ZNF577','ZNF578','ZNF579','ZNF580','ZNF581','ZNF582','ZNF583','ZNF584','ZNF585A','ZNF585B','ZNF586','ZNF587','ZNF587B','ZNF589','ZNF592','ZNF594','ZNF595','ZNF596','ZNF597','ZNF598','ZNF599','ZNF600','ZNF605','ZNF606','ZNF607','ZNF608','ZNF609','ZNF610','ZNF611','ZNF613','ZNF614','ZNF615','ZNF616','ZNF618','ZNF619','ZNF620','ZNF621','ZNF623','ZNF624','ZNF625','ZNF626','ZNF627','ZNF628','ZNF629','ZNF630','ZNF639','ZNF641','ZNF644','ZNF645','ZNF646','ZNF648','ZNF649','ZNF652','ZNF653','ZNF654','ZNF655','ZNF658','ZNF66','ZNF660','ZNF662','ZNF664','ZNF665','ZNF667','ZNF668','ZNF669','ZNF670','ZNF671','ZNF672','ZNF674','ZNF675','ZNF676','ZNF677','ZNF678','ZNF679','ZNF680','ZNF681','ZNF682','ZNF683','ZNF684','ZNF687','ZNF688','ZNF689','ZNF69','ZNF691','ZNF692','ZNF695','ZNF696','ZNF697','ZNF699','ZNF7','ZNF70','ZNF700','ZNF701','ZNF703','ZNF704','ZNF705A','ZNF705B','ZNF705D','ZNF705E','ZNF705G','ZNF706','ZNF707','ZNF708','ZNF709','ZNF71','ZNF710','ZNF711','ZNF713','ZNF714','ZNF716','ZNF717','ZNF718','ZNF721','ZNF724','ZNF726','ZNF727','ZNF728','ZNF729','ZNF730','ZNF732','ZNF735','ZNF736','ZNF737','ZNF74','ZNF740','ZNF746','ZNF747','ZNF749','ZNF750','ZNF75A','ZNF75D','ZNF76','ZNF761','ZNF763','ZNF764','ZNF765','ZNF766','ZNF768','ZNF77','ZNF770','ZNF771','ZNF772','ZNF773','ZNF774','ZNF775','ZNF776','ZNF777','ZNF778','ZNF780A','ZNF780B','ZNF781','ZNF782','ZNF783','ZNF784','ZNF785','ZNF786','ZNF787','ZNF788','ZNF789','ZNF79','ZNF790','ZNF791','ZNF792','ZNF793','ZNF799','ZNF8','ZNF80','ZNF800','ZNF804A','ZNF804B','ZNF805','ZNF808','ZNF81','ZNF813','ZNF814','ZNF816','ZNF821','ZNF823','ZNF827','ZNF829','ZNF83','ZNF830','ZNF831','ZNF835','ZNF836','ZNF837','ZNF84','ZNF841','ZNF843','ZNF844','ZNF845','ZNF846','ZNF85','ZNF850','ZNF852','ZNF853','ZNF860','ZNF865','ZNF878','ZNF879','ZNF880','ZNF883','ZNF888','ZNF891','ZNF90','ZNF91','ZNF92','ZNF93','ZNF98','ZNF99','ZSCAN1','ZSCAN10','ZSCAN12','ZSCAN16','ZSCAN18','ZSCAN2','ZSCAN20','ZSCAN21','ZSCAN22','ZSCAN23','ZSCAN25','ZSCAN26','ZSCAN29','ZSCAN30','ZSCAN31','ZSCAN32','ZSCAN4','ZSCAN5A','ZSCAN5B','ZSCAN5C','ZSCAN9','ZUFSP','ZXDA','ZXDB','ZXDC','ZZZ3']
TFs=TFs1+TFs2

#TFs other, inclusive of the above list, not necessarily DNA binding proteins but affect transcription
TFs1Other=['AATF','ABL1','ABRA','AC008770.3','AC020915.1','AC023509.3','AC092835.1','AC105001.2','AC138696.1','ACOD1','ACTR3B','ACTR3C','ADAM8','ADAMTS17','ADAMTS19','ADAR','ADGRA1','ADGRG3','ADNP','ADNP2','AEBP1','AEBP2','AES','AFAP1 ','AFF1','AFF3','AFF4','AGER','AGT','AHCTF1','AHDC1','AHR','AHRR','AIM2','AIRE','AKAP8','AKAP8L','AKNA','AKT1','ALK','ALKBH2','ALX1','ALX3','ALX4','AMH','ANAPC2','ANHX','ANKRD30A','ANKZF1','ANXA3','ANXA4','AP5Z1','APBB3','APC2','APEX1','APOA4','APOBEC3A','APOBEC3B','APOBEC3G','APTX','AR','ARGFX','ARHGAP35','ARHGEF2','ARHGEF5','ARID1A','ARID1B','ARID2','ARID3A','ARID3B','ARID3C','ARID4A','ARID4B','ARID5A','ARID5B','ARNT','ARNT2','ARNTL','ARNTL2','ARRB1','ARRB2','ARX','ASCC1','ASCL1','ASCL2','ASCL3','ASCL4','ASCL5','ASH1L','ATF1','ATF2','ATF3','ATF4','ATF5','ATF6','ATF6B','ATF7','ATF7IP','ATM','ATMIN','ATOH1','ATOH7','ATOH8','ATR','ATRX','ATXN7','BACH1','BACH2','BAG1','BANF1','BANP','BARHL1','BARHL2','BARX1','BARX2','BATF','BATF2','BATF3','BAZ2A','BAZ2B','BBX','BCL10','BCL11A','BCL11B','BCL3','BCL6','BCL6B','BCLAF1','BCOR','BDP1','BEND4','BEX1','BHLHA15','BHLHA9','BHLHE22','BHLHE23','BHLHE40','BHLHE41','BHMG1','BIN1','BLM','BLZF1','BMI1','BMP2','BMP4','BMP7','BMPR1A','BNC1','BNC2','BOD1L1','BOLA1','BOLA2','BOLA2B','BOLA3','BORCS8-MEF2B','BPNT1','BPTF','BRCA1','BRD4','BRD7','BRD8','BRD9','BRF1','BRF2','BRMS1','BRPF1','BSX','BTAF1','BTBD8','BTG2','BTK','BTRC','BUD31','C11orf95','C14orf39','C17orf49','C1D','C20orf194','C3orf33','CACTIN','CAMK1D','CAMK2A','CAMTA1','CAMTA2','CAPN15','CAPN3','CARD11','CARD14','CARF','CARHSP1','CASP8AP2','CASZ1','CAT','CBFA2T2','CBFA2T3','CBFB','CBL','CBLL1','CBX2','CC2D1A','CC2D1B','CCDC124','CCDC169-SOHLH2','CCDC17','CCDC3','CCDC71','CCDC83','CCNT1','CCNT2','CD36','CD40','CD40LG','CDC5L','CDC6','CDK5RAP3','CDK7','CDK8','CDK9','CDKN1A','CDKN2A','CDX1','CDX2','CDX4','CEBPA','CEBPB','CEBPD','CEBPE','CEBPG','CEBPZ','CELF3','CENPA','CENPB','CENPBD1','CENPS','CENPT','CENPX','CERS2','CERS3','CERS4','CERS5','CERS6','CFLAR','CGGBP1','CHAMP1','CHCHD3','CHD1','CHD2','CHD3','CHD4','CHD6','CHD7','CHD9','CHP1','CHRAC1','CHUK','CHURC1','CIB1','CIC','CIITA','CIR1','CITED1','CITED2','CITED4','CIZ1','CLOCK','CLU','CMKLR1','CNBP','CNOT3','CNOT4','CNOT7','CNOT8','COMMD1','COMMD6','COMMD7','COPS2','COPS3','COPS4','CPEB1','CPSF4','CPSF4L','CPXCR1','CRAMP1','CREB1','CREB3','CREB3L1','CREB3L2','CREB3L3','CREB3L4','CREB5','CREBBP','CREBL2','CREBRF','CREBZF','CREG1','CREM','CRTC1','CRTC2','CRTC3','CRX','CSDC2','CSDE1','CSRNP1','CSRNP2','CSRNP3','CTBP1','CTBP2','CTCF','CTCFL','CTH','CTNNB1','CTNNBIP1','CUL1','CUL2','CUL3','CUL4A','CUL4B','CUL5','CUX1','CUX2','CXXC1','CXXC4','CXXC5','CYLD','CYP1B1','CYTL1','DAB2IP','DACH1','DACH2','DAP','DAXX','DBP','DBX1','DBX2','DCP1A','DDB1','DDB2','DDIT3','DDN','DDR2','DDRGK1','DDX3X','DDX5','DDX58','DEAF1','DEK','DEPDC1','DEPDC1B','DEPDC4','DEPDC5','DEPDC7','DEPTOR','DHX34','DHX36','DHX57','DISP3','DLX1','DLX2','DLX3','DLX4','DLX5','DLX6','DMAP1','DMBX1','DMC1','DMRT1','DMRT2','DMRT3','DMRTA1','DMRTA2','DMRTB1','DMRTC1','DMRTC2','DMTF1','DNA2','DNAJA3','DNAJC1','DNAJC2','DNAJC21','DNMT1','DNMT3A','DNMT3B','DNMT3L','DNTT','DNTTIP1','DOT1L','DPF1','DPF2','DPF3','DPRX','DR1','DRAP1','DRGX','DSP','DTX1','DUS3L','DUSP12','DUX1','DUX3','DUX4','DUXA','DVL1','DVL2','DVL3','DZIP1','DZIP1L','E2F1','E2F2','E2F3','E2F4','E2F5','E2F6','E2F7','E2F8','E4F1','EAF2','EBF1','EBF2','EBF3','EBF4','EDA','EDA2R','EDF1','EDN1','EEA1','EED','EGLN1','EGR1','EGR2','EGR3','EGR4','EHF','EID1','EID2','EIF2AK2','EIF2AK4','EIF3C','EIF3CL','EIF3K','ELF1','ELF2','ELF3','ELF4','ELF5','ELK1','ELK3','ELK4','ELMSAN1','EME1','EMX1','EMX2','EN1','EN2','ENO1','EOMES','EP300','EP400','EPAS1','EPHA5','EPM2AIP1','ERBIN','ERC1','ERCC3','ERCC4','ERF','ERG','ESR1','ESR2','ESRRA','ESRRB','ESRRG','ESX1','ETS1','ETS2','ETV1','ETV2','ETV3','ETV3L','ETV4','ETV5','ETV6','ETV7','EVX1','EVX2','EWSR1','EXO1','EXOC2','EXOG','EZH1','EZH2','FAAP24','FAM153C','FAM170A','FAM171B','FAM200A','FAM200B','FAN1','FANCA','FANCD2','FANK1','FARSA','FARSB','FBN1','FBXL19','FBXO41','FEN1','FER','FERD3L','FEV','FEZF1','FEZF2','FGD1','FHL2','FIGLA','FIZ1','FLI1','FLNA','FLYWCH1','FMNL2','FOS','FOSB','FOSL1','FOSL2','FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L2','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXN4','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1','FUBP1','FUBP3','FUS','FZD1','FZD2','FZD4','FZD6','GABPA','GABPB1','GABPB2','GADD45A','GAS6','GAS7','GATA1','GATA2','GATA3','GATA4','GATA5','GATA6','GATAD1','GATAD2A','GATAD2B','GBX1','GBX2','GCFC2','GCM1','GCM2','GEN1','GFI1','GFI1B','GLA','GLI1','GLI2','GLI3','GLI4','GLIS1','GLIS2','GLIS3','GLMP','GLYR1','GMEB1','GMEB2','GOLGB1','GON4L','GPATCH8','GPBP1','GPBP1L1','GPER1','GPR155','GREM1','GRHL1','GRHL2','GRHL3','GRM6','GSC','GSC2','GSX1','GSX2','GTF2A1','GTF2A1L','GTF2A2','GTF2B','GTF2E2','GTF2F1','GTF2F2','GTF2H1','GTF2H2','GTF2H2C','GTF2H3','GTF2H4','GTF2I','GTF2IRD1','GTF2IRD2','GTF2IRD2B','GTF3A','GTF3C1','GTF3C2','GULP1','GZF1','H1F0','H1FOO','H1FX','HAND1','HAND2','HAVCR2','HBP1','HCFC1','HCK','HDAC1','HDAC10','HDAC2','HDAC3','HDAC4','HDAC5','HDAC6','HDAC7','HDAC8','HDAC9','HDX','HELT','HELZ','HELZ2','HES1','HES2','HES3','HES4','HES5','HES6','HES7','HESX1','HEY1','HEY2','HEYL','HHEX','HIC1','HIC2','HIF1A','HIF3A','HINFP','HIPK2','HIRA','HIST1H1A','HIST1H1B','HIST1H1C','HIST1H1D','HIST1H1E','HIST1H1T','HIVEP1','HIVEP2','HIVEP3','HKR1','HLA-DQB1','HLA-DQB2','HLA-DRB1','HLA-DRB3','HLA-DRB5','HLF','HLTF','HLX','HMBOX1','HMG20A','HMG20B','HMGA1','HMGA2','HMGB1','HMGB2','HMGB3','HMGB4','HMGN1','HMGN3','HMGXB3','HMGXB4','HMOX1','HMX1','HMX2','HMX3','HNF1A','HNF1B','HNF4A','HNF4G','HNRNPA1','HNRNPA3','HNRNPAB','HNRNPCL2','HNRNPD','HNRNPDL','HNRNPK','HNRNPL','HNRNPU','HOMEZ','HOOK2','HOPX','HOXA1','HOXA10','HOXA11','HOXA13','HOXA2','HOXA3','HOXA4','HOXA5','HOXA6','HOXA7','HOXA9','HOXB1','HOXB13','HOXB2','HOXB3','HOXB4','HOXB5','HOXB6','HOXB7','HOXB8','HOXB9','HOXC10','HOXC11','HOXC12','HOXC13','HOXC4','HOXC5','HOXC6','HOXC8','HOXC9','HOXD1','HOXD10','HOXD11','HOXD12','HOXD13','HOXD3','HOXD4','HOXD8','HOXD9','HP1BP3','HR','HSBP1','HSF1','HSF2','HSF4','HSF5','HSFX1','HSFX2','HSFY1','HSFY2','HSPA1A','HSPA1B','ICAM1','ID1','ID2','ID3','ID4','IFI16','IFNB1','IGHM','IKBKB','IKBKG','IKZF1','IKZF2','IKZF3','IKZF4','IKZF5','IL10','IL1B','IL1RAP','IL4','IL5','IL6','INF2','ING4','INS','INSM1','INSM2','IQGAP1','IRAK1','IRAK2','IRAK3','IRF1','IRF2','IRF2BP1','IRF3','IRF4','IRF5','IRF6','IRF7','IRF8','IRF9','IRX1','IRX2','IRX3','IRX4','IRX5','IRX6','ISL1','ISL2','ISX','ITCH','ITGB2','ITGB3BP','IZUMO2','JADE1','JAK2','JARID2','JAZF1','JDP2','JMY','JRK','JRKL','JUN','JUNB','JUND','JUP','KANSL2','KAT2A','KAT2B','KAT5','KAT6A','KAT6B','KAT7','KAT8','KCMF1','KCNIP3','KDM1A','KDM2A','KDM2B','KDM3A','KDM5A','KDM5B','KDM5C','KDM5D','KEAP1','KIAA1549','KIAA1683','KIAA1841','KIAA1958','KIF15','KIN','KIT','KLF1','KLF10','KLF11','KLF12','KLF13','KLF14','KLF15','KLF16','KLF17','KLF2','KLF3','KLF4','KLF5','KLF6','KLF7','KLF8','KLF9','KMT2A','KMT2B','KMT2C','KMT2D','KNTC1','KRAS','KRBOX1','KRT13','KRTAP5-1','KRTAP5-9','KRTAP9-9','L3MBTL1','L3MBTL3','L3MBTL4','LARP1','LARP1B','LARP4','LARP4B','LARP6','LARP7','LBX1','LBX2','LCOR','LCORL','LEF1','LEUTX','LGALS9','LGR4','LHX1','LHX2','LHX3','LHX4','LHX5','LHX6','LHX8','LHX9','LIG1','LIG3','LIN28A','LIN28B','LIN54','LMO2','LMO4','LMX1A','LMX1B','LPP','LRP5','LRP6','LRP8','LRRFIP1','LRRFIP2','LSR','LTF','LTN1','LYAR','LYL1','LZTFL1','LZTR1','LZTS1','MACC1','MACF1','MAD2L2','MAEL','MAF','MAFA','MAFB','MAFF','MAFG','MAFK','MALT1','MAML1','MAML3','MAP2K5','MAP3K10','MAP3K13','MAP3K7','MAPK1','MAPK10','MAPK11','MAPK14','MAPK3','MAPK8','MAPK9','MATR3','MAVS','MAX','MAZ','MB21D1','MBD1','MBD2','MBD3','MBD4','MBD6','MBNL1','MBNL2','MBNL3','MBTPS2','MDFI','MDM2','MDM4','MECOM','MECP2','MED1','MED12','MED13','MED13L','MED14','MED15','MED16','MED17','MED21','MED23','MED24','MED6','MEF2A','MEF2B','MEF2C','MEF2D','MEIS1','MEIS2','MEIS3','MEN1','MEOX1','MEOX2','MESP1','MESP2','MET','MGA','MGMT','MID2','MIER1','MIER2','MIER3','MINK1','MIS18BP1','MITF','MIXL1','MKL1','MKL2','MKRN1','MKRN2','MKRN3','MKX','MLLT1','MLLT10','MLLT11','MLLT3','MLLT6','MLX','MLXIP','MLXIPL','MN1','MNAT1','MNT','MNX1','MORC3','MPG','MRPL28','MRRF','MSANTD1','MSANTD3','MSANTD4','MSC','MSGN1','MSH2','MSH3','MSH6','MSL3','MSRB2','MST1R','MSX1','MSX2','MTA1','MTA2','MTA3','MTDH','MTERF1','MTERF2','MTERF3','MTERF4','MTF1','MTF2','MTPN','MUS81','MXD1','MXD3','MXD4','MXI1','MYB','MYBBP1A','MYBL1','MYBL2','MYC','MYCBP','MYCL','MYCN','MYD88','MYEF2','MYF5','MYF6','MYNN','MYOCD','MYOD1','MYOG','MYPOP','MYRF','MYRFL','MYSM1','MYT1','MYT1L','MZF1','NAB2','NABP2','NACC2','NAIF1','NANOG','NANOGNB','NANOGP8','NCOA1','NCOA2','NCOA3','NCOA4','NCOA5','NCOA6','NCOA7','NCOR1','NCOR2','NDC80','NDN','NDP','NEB','NEIL1','NEUROD1','NEUROD2','NEUROD4','NEUROD6','NEUROG1','NEUROG2','NEUROG3','NFAM1','NFAT5','NFATC1','NFATC2','NFATC3','NFATC4','NFE2','NFE2L1','NFE2L2','NFE2L3','NFE4','NFIA','NFIB','NFIC','NFIL3','NFIX','NFKB1','NFKB2','NFKBIA','NFKBIB','NFKBID','NFKBIL1','NFKBIZ','NFRKB','NFX1','NFXL1','NFYA','NFYB','NFYC','NHLH1','NHLH2','NKRF','NKX1-1','NKX1-2','NKX2-1','NKX2-2','NKX2-3','NKX2-4','NKX2-5','NKX2-6','NKX2-8','NKX3-1','NKX3-2','NKX6-1','NKX6-2','NKX6-3','NLRC3','NLRC4','NLRC5','NLRP12','NLRP2B','NLRP3','NME2','NOBOX','NOC3L','NOC4L','NOCT','NOD1','NOD2','NODAL','NONO','NOP2','NOTCH1','NOTCH2','NOTCH3','NOTO','NPAS1','NPAS2','NPAS3','NPAS4','NPM1','NR0B1','NR0B2','NR1D1','NR1D2','NR1H2','NR1H3','NR1H4','NR1I2','NR1I3','NR2C1','NR2C2','NR2E1','NR2E3','NR2F1','NR2F2','NR2F6','NR3C1','NR3C2','NR4A1','NR4A2','NR4A3','NR5A1','NR5A2','NR6A1','NRBF2','NRF1','NRG1','NRIP1','NRIP3','NRK','NRL','NSD1','NSD2','NTRK1','NUCKS1','NUFIP1','NUP98','NUPL2']
TFs2Other=['OGG1','OLIG1','OLIG2','OLIG3','ONECUT1','ONECUT2','ONECUT3','OPRD1','OSR1','OSR2','OTOP3','OTP','OTULIN','OTX1','OTX2','OVOL1','OVOL2','OVOL3','PA2G4','PABPN1','PARK7','PARP1','PARP10','PARP12','PATZ1','PAWR','PAX1','PAX2','PAX3','PAX4','PAX5','PAX6','PAX7','PAX8','PAX9','PAXBP1','PAXIP1','PBRM1','PBX1','PBX2','PBX3','PBX4','PCBD1','PCBP1','PCBP2','PCBP3','PCGF2','PCGF6','PCNA','PCSK6','PDCD7','PDS5B','PDX1','PEG3','PELI1','PER1','PER2','PER3','PEX14','PFDN1','PGBD1','PGR','PGS1','PHB','PHB2','PHF1','PHF19 ','PHF20','PHF21A','PHF5A','PHOX2A','PHOX2B','PHTF1','PHTF2','PIAS1','PIAS2','PIAS3','PIAS4','PIDD1','PIKFYVE','PIM1','PIN1','PINK1','PITX1','PITX2','PITX3','PKHD1','PKHD1L1','PKNOX1','PKNOX2','PLA2G10','PLA2G1B','PLAG1','PLAGL1','PLAGL2','PLEK','PLEK2','PLEKHA4','PLPP3','PLSCR1','PLXNA1','PLXNA2','PLXNA3','PLXNA4','PLXNB1','PLXNB2','PLXNB3','PLXNC1','PLXND1','PML','PMS1','PNKP','POGK','POGZ','POLA1','POLB','POLE3','POLE4','POLG','POLH','POLI','POLK','POLL','POLM','POLN','POLQ','POLR1B','POLR2A','POLR2B','POLR2E','POLR2J','POLR2K','POLR2L','POLR3A','POLRMT','POT1','POU1F1','POU2AF1','POU2F1','POU2F2','POU2F3','POU3F1','POU3F2','POU3F3','POU3F4','POU4F1','POU4F2','POU4F3','POU5F1','POU5F1B','POU5F2','POU6F1','POU6F2','PPARA','PPARD','PPARG','PPARGC1A','PPARGC1B','PPP1R10','PPP1R13L','PPP1R21','PPP2R3B','PPP2R5B','PPP3CA','PPRC1','PQBP1','PRAM1','PRAP1','PRB3','PRDM1','PRDM10','PRDM11','PRDM12','PRDM13','PRDM14','PRDM15','PRDM16','PRDM2','PRDM4','PRDM5','PRDM6','PRDM7','PRDM8','PRDM9','PRDX3','PREB','PREX1','PREX2','PRH2','PRIM2','PRIMPOL','PRKCB','PRKCH','PRKCI','PRKCQ','PRKCZ','PRKD1','PRKD2','PRKN','PRMT2','PRMT3','PRNP','PROP1','PROX1','PROX2','PRR12','PRR3','PRRX1','PRRX2','PSMA6','PSMC5','PSMD10','PSMD11','PSMD12','PSPC1','PTCH1','PTEN','PTF1A','PTGIS','PTH','PTTG1','PUM3','PURA','PURB','PURG','PYCARD','PYDC1','PYDC2','QRICH1','RAB11FIP4','RAB7B','RAD21','RAD51','RAG1','RAI1','RAPGEF3','RAPGEF4','RAPGEF5','RARA','RARB','RARG','RAX','RAX2','RB1','RBAK','RBBP7','RBCK1','RBL1','RBL2','RBM10','RBM20','RBM22','RBM26','RBM27','RBM5','RBM6','RBPJ','RBPJL','RBSN','RC3H1','RC3H2','RCAN1','RCOR1','RCOR2','RCOR3','RECQL','REL','RELA','RELB','RELL2','RELN','REPIN1','RERE','REST','REV1','REXO4','RFC1','RFX1','RFX2','RFX3','RFX4','RFX5','RFX6','RFX7','RFX8','RFXANK','RFXAP','RGCC','RGS11','RGS6','RGS7','RGS9','RHEBL1','RHOXF1','RHOXF2','RHOXF2B','RING1','RIOK2','RIPK1','RIPK2','RIPK3','RIPK4','RLF','RLIM','RNASE2','RNASEH1','RNF113A','RNF113B','RNF114','RNF125','RNF138','RNF14','RNF166','RNF2','RNF219','RNF25','RNF31','RNF4','RNF41','RORA','RORB','RORC','RPA2','RPA4','RPL36A','RPL7','RPL7A','RPS27A','RPS6KA4','RPS6KA5','RREB1','RRN3','RUNX1','RUNX1T1','RUNX2','RUNX3','RWDD3','RXRA','RXRB','RXRG','RYBP','S100A12','S100A8','S100A9','SAFB','SAFB2','SALL1','SALL2','SALL3','SALL4','SART1','SATB1','SATB2','SAV1','SCAND1','SCAND2P','SCAPER','SCMH1','SCML1','SCML2','SCML4','SCOC','SCRT1','SCRT2','SCX','SEBOX','SEMA4A','SERTAD1','SERTAD2','SETBP1','SETD6','SETDB1','SETDB2','SF3A2','SF3A3','SFPQ','SFRP4','SFRP5','SFTPB','SGK1','SGK3','SGSM2','SHH','SHOX','SHOX2','SHPRH','SIGIRR','SIK1','SIM1','SIM2','SIN3A','SIN3B','SIRT1','SIRT2','SIRT3','SIRT6','SIVA1','SIX1','SIX2','SIX3','SIX4','SIX5','SIX6','SKI','SKIDA1','SKIL','SKOR1','SKOR2','SLC22A4','SLC26A10','SLC26A3','SLC2A4RG','SLC30A9','SLC39A10','SLC4A10','SMAD1','SMAD2','SMAD3','SMAD4','SMAD5','SMAD6','SMAD7','SMAD9','SMARCA1','SMARCA2','SMARCA4','SMARCA5','SMARCAL1','SMARCB1','SMARCC1','SMARCC2','SMARCD1','SMARCD2','SMARCD3','SMARCE1','SMC3','SMO','SMUG1','SMYD1','SMYD3','SNAI1','SNAI2','SNAI3','SNAPC2','SNAPC4','SNAPC5','SNRPC','SNURF','SNW1','SOGA3','SOHLH1','SOHLH2','SON','SORBS2','SOX1','SOX10','SOX11','SOX12','SOX13','SOX14','SOX15','SOX17','SOX18','SOX2','SOX21','SOX3','SOX30','SOX4','SOX5','SOX6','SOX7','SOX8','SOX9','SP1','SP100','SP110','SP140','SP140L','SP2','SP3','SP4','SP5','SP6','SP7','SP8','SP9','SPDEF','SPEN','SPHK1','SPI1','SPIB','SPIC','SPZ1','SRCAP','SREBF1','SREBF2','SRF','SRY','SSB','SSBP2','SSBP3','SSBP4','SSH1','SSH2','SSH3','SSRP1','SSX1','ST18','STAG1','STAT1','STAT2','STAT3','STAT4','STAT5A','STAT5B','STAT6','STK16','STK3','STK36','STRN','STRN3','SUB1','SUFU','SUMO1','SUPT16H','SUPT4H1','SUPT5H','SUPT6H','SUV39H1','SUV39H2','SUZ12','SYK','T','TAB1','TAB2','TAB3','TADA2A','TADA2B','TADA3','TAF1','TAF10','TAF11','TAF12','TAF13','TAF1A','TAF1B','TAF1C','TAF1L','TAF2','TAF3','TAF4','TAF4B','TAF5','TAF5L','TAF6','TAF6L','TAF7','TAF7L','TAF8','TAF9','TAF9B','TAL1','TAL2','TARDBP','TAX1BP1','TBL1Y','TBP','TBPL1','TBPL2','TBR1','TBX1','TBX10','TBX15','TBX18','TBX19','TBX2','TBX20','TBX21','TBX22','TBX3','TBX4','TBX5','TBX6','TCEA2','TCEAL1','TCEAL7','TCEAL8','TCF12','TCF15','TCF19','TCF20','TCF21','TCF23','TCF24','TCF25','TCF3','TCF4','TCF7','TCF7L1','TCF7L2','TCFL5','TDG','TDP1','TDP2','TEAD1','TEAD2','TEAD3','TEAD4','TEF','TERB1','TERF1','TERF2','TERF2IP','TESMIN','TET1','TET2','TET3','TFAM','TFAP2A','TFAP2B','TFAP2C','TFAP2D','TFAP2E','TFAP4','TFCP2','TFCP2L1','TFDP1','TFDP2','TFDP3','TFE3','TFEB','TFEC','TGFB1','TGFB1I1','TGIF1','TGIF2','TGIF2LX','TGIF2LY','TGS1','THAP1','THAP10','THAP11','THAP12','THAP2','THAP3','THAP4','THAP5','THAP6','THAP7','THAP8','THAP9','THRA','THRAP3','THRB','THYN1','TIA1','TICAM1','TIGD1','TIGD2','TIGD3','TIGD4','TIGD5','TIGD6','TIGD7','TIPARP','TIRAP','TLE1','TLE2','TLE3','TLE4','TLE6','TLR2','TLR3','TLR4','TLR9','TLX1','TLX2','TLX3','TMEM235','TMF1','TNF','TNFAIP3','TNFRSF11A','TNFRSF4','TNFSF11','TNFSF18','TNFSF4','TOE1','TONSL','TOP1','TOP2A','TOP2B','TOPORS','TOX','TOX2','TOX3','TOX4','TP53','TP53BP1','TP63','TP73','TPRX1','TRADD','TRAF1','TRAF2','TRAF3','TRAF4','TRAF5','TRAF6','TRAFD1','TRAK2','TRAPPC9','TRERF1','TREX1','TRIB1','TRIB3','TRIM13','TRIM14','TRIM15','TRIM21','TRIM22','TRIM23','TRIM24','TRIM25','TRIM26','TRIM27','TRIM28','TRIM29','TRIM3','TRIM31','TRIM32','TRIM33','TRIM34','TRIM37','TRIM38','TRIM40','TRIM5','TRIM52','TRIM62','TRIM8','TRIP11','TRIT1','TRMT1','TRMT1L','TRPS1','TRRAP','TSC2','TSC22D1','TSC22D2','TSC22D3','TSC22D4','TSHZ1','TSHZ2','TSHZ3','TSN','TSSK4','TTF1','TUB','TULP4','TUT1','TWIST1','TWIST2','TXK','U2AF1','U2AF1L4','UBA52','UBB','UBC','UBE2K','UBE2N','UBE2V1','UBN1','UBP1','UBR4','UBTF','UBTFL1','UBTFL6','UFL1','UHRF1','UHRF2','UNCX','UNG','UNK','UNKL','USF1','USF2','USF3','USP39','USP7','UTF1','VAV1','VAX1','VAX2','VDR','VEGFA','VENTX','VEZF1','VGLL1','VGLL2','VGLL3','VHLL','VPS72','VSX1','VSX2','WDHD1','WFS1','WIZ','WNT1','WNT10B','WNT2','WNT3A','WNT5A','WNT8B','WRN','WT1','WWOX','WWP2','WWTR1','XBP1','XCL1','XPA','YBX1','YBX2','YBX3','YEATS2','YEATS4','YOD1','YY1','YY2','ZBED1','ZBED2','ZBED3','ZBED4','ZBED5','ZBED6','ZBED6CL','ZBED8','ZBED9','ZBP1','ZBTB1','ZBTB10','ZBTB11','ZBTB12','ZBTB14','ZBTB16','ZBTB17','ZBTB18','ZBTB2','ZBTB20','ZBTB21','ZBTB22','ZBTB24','ZBTB25','ZBTB26','ZBTB3','ZBTB32','ZBTB33','ZBTB34','ZBTB37','ZBTB38','ZBTB39','ZBTB4','ZBTB40','ZBTB41','ZBTB42','ZBTB43','ZBTB44','ZBTB45','ZBTB46','ZBTB47','ZBTB48','ZBTB49','ZBTB5','ZBTB6','ZBTB7A','ZBTB7B','ZBTB7C','ZBTB8A','ZBTB8B','ZBTB9','ZC3H10','ZC3H11A','ZC3H12A','ZC3H13','ZC3H14','ZC3H15','ZC3H18','ZC3H3','ZC3H4','ZC3H6','ZC3H7A','ZC3H7B','ZC3H8','ZC3HAV1','ZCCHC11','ZCCHC6','ZDHHC11','ZDHHC19','ZEB1','ZEB2','ZFAND5','ZFAT','ZFHX2','ZFHX3','ZFHX4','ZFP1','ZFP14','ZFP2','ZFP28','ZFP3','ZFP30','ZFP36','ZFP36L1','ZFP36L2','ZFP37','ZFP41','ZFP42','ZFP57','ZFP62','ZFP64','ZFP69','ZFP69B','ZFP82','ZFP90','ZFP91','ZFP92','ZFPL1','ZFPM1','ZFPM2','ZFR','ZFR2','ZFX','ZFY','ZFYVE26','ZGLP1','ZGPAT','ZHX1','ZHX2','ZHX3','ZIC1','ZIC2','ZIC3','ZIC4','ZIC5','ZIK1','ZIM2','ZIM3','ZKSCAN1','ZKSCAN2','ZKSCAN3','ZKSCAN4','ZKSCAN5','ZKSCAN7','ZKSCAN8','ZMAT1','ZMAT2','ZMAT3','ZMAT4','ZMAT5','ZMYM1','ZMYM2','ZMYM3','ZMYM4','ZMYM5','ZMYM6','ZMYND10','ZNF10','ZNF100','ZNF101','ZNF106','ZNF107','ZNF112','ZNF114','ZNF117','ZNF12','ZNF121','ZNF124','ZNF131','ZNF132','ZNF133','ZNF134','ZNF135','ZNF136','ZNF137P','ZNF138','ZNF14','ZNF140','ZNF141','ZNF142','ZNF143','ZNF146','ZNF148','ZNF154','ZNF155','ZNF157','ZNF16','ZNF160','ZNF165','ZNF169','ZNF17','ZNF174','ZNF175','ZNF177','ZNF18','ZNF180','ZNF181','ZNF182','ZNF184','ZNF185','ZNF189','ZNF19','ZNF195','ZNF197','ZNF2','ZNF20','ZNF200','ZNF202','ZNF205','ZNF207','ZNF208','ZNF211','ZNF212','ZNF213','ZNF214','ZNF215','ZNF217','ZNF219','ZNF22','ZNF221','ZNF222','ZNF223','ZNF224','ZNF225','ZNF226','ZNF227','ZNF229','ZNF23','ZNF230','ZNF232','ZNF233','ZNF234','ZNF235','ZNF236','ZNF239','ZNF24','ZNF248','ZNF25','ZNF250','ZNF251','ZNF253','ZNF254','ZNF256','ZNF257','ZNF26','ZNF260','ZNF263','ZNF264','ZNF266','ZNF267','ZNF268','ZNF273','ZNF274','ZNF275','ZNF276','ZNF277','ZNF28','ZNF280A','ZNF280B','ZNF280C','ZNF280D','ZNF281','ZNF282','ZNF283','ZNF284','ZNF285','ZNF286A','ZNF286B','ZNF287','ZNF292','ZNF296','ZNF3','ZNF30','ZNF300','ZNF302','ZNF304','ZNF311','ZNF316','ZNF317','ZNF318','ZNF319','ZNF32','ZNF320','ZNF322','ZNF324','ZNF324B','ZNF326','ZNF329','ZNF330','ZNF331','ZNF333','ZNF334','ZNF335','ZNF337','ZNF33A','ZNF33B','ZNF34','ZNF341','ZNF343','ZNF345','ZNF346','ZNF347','ZNF35','ZNF350','ZNF354A','ZNF354B','ZNF354C','ZNF355P','ZNF358','ZNF362','ZNF365','ZNF366','ZNF367','ZNF37A','ZNF382','ZNF383','ZNF384','ZNF385A','ZNF385B','ZNF385C','ZNF385D','ZNF391','ZNF394','ZNF395','ZNF396','ZNF397','ZNF398','ZNF404','ZNF407','ZNF408','ZNF41','ZNF410','ZNF414','ZNF415','ZNF416','ZNF417','ZNF418','ZNF419','ZNF420','ZNF423','ZNF425','ZNF426','ZNF428','ZNF429','ZNF43','ZNF430','ZNF431','ZNF432','ZNF433','ZNF436','ZNF438','ZNF439','ZNF44','ZNF440','ZNF441','ZNF442','ZNF443','ZNF444','ZNF445','ZNF446','ZNF449','ZNF45','ZNF451','ZNF454','ZNF460','ZNF461','ZNF462','ZNF467','ZNF468','ZNF469','ZNF470','ZNF471','ZNF473','ZNF474','ZNF479','ZNF48','ZNF480','ZNF483','ZNF484','ZNF485','ZNF486','ZNF487','ZNF488','ZNF490','ZNF491','ZNF492','ZNF493','ZNF496','ZNF497','ZNF500','ZNF501','ZNF502','ZNF503','ZNF506','ZNF507','ZNF510','ZNF511','ZNF512','ZNF512B','ZNF513','ZNF514','ZNF516','ZNF517','ZNF518A','ZNF518B','ZNF519']
TFs3Other=['ZNF521','ZNF524','ZNF525','ZNF526','ZNF527','ZNF528','ZNF529','ZNF530','ZNF532','ZNF534','ZNF536','ZNF540','ZNF541','ZNF542P','ZNF543','ZNF544','ZNF546','ZNF547','ZNF548','ZNF549','ZNF550','ZNF551','ZNF552','ZNF554','ZNF555','ZNF556','ZNF557','ZNF558','ZNF559','ZNF560','ZNF561','ZNF562','ZNF563','ZNF564','ZNF565','ZNF566','ZNF567','ZNF568','ZNF569','ZNF57','ZNF570','ZNF571','ZNF572','ZNF573','ZNF574','ZNF575','ZNF576','ZNF577','ZNF578','ZNF579','ZNF580','ZNF581','ZNF582','ZNF583','ZNF584','ZNF585A','ZNF585B','ZNF586','ZNF587','ZNF587B','ZNF589','ZNF592','ZNF593','ZNF594','ZNF595','ZNF596','ZNF597','ZNF598','ZNF599','ZNF600','ZNF605','ZNF606','ZNF607','ZNF608','ZNF609','ZNF610','ZNF611','ZNF613','ZNF614','ZNF615','ZNF616','ZNF618','ZNF619','ZNF620','ZNF621','ZNF622','ZNF623','ZNF624','ZNF625','ZNF626','ZNF627','ZNF628','ZNF629','ZNF630','ZNF638','ZNF639','ZNF641','ZNF644','ZNF645','ZNF646','ZNF648','ZNF649','ZNF652','ZNF653','ZNF654','ZNF655','ZNF658','ZNF66','ZNF660','ZNF662','ZNF664','ZNF665','ZNF667','ZNF668','ZNF669','ZNF670','ZNF671','ZNF672','ZNF674','ZNF675','ZNF676','ZNF677','ZNF678','ZNF679','ZNF680','ZNF681','ZNF682','ZNF683','ZNF684','ZNF687','ZNF688','ZNF689','ZNF69','ZNF691','ZNF692','ZNF695','ZNF696','ZNF697','ZNF699','ZNF7','ZNF70','ZNF700','ZNF701','ZNF703','ZNF704','ZNF705A','ZNF705B','ZNF705D','ZNF705E','ZNF705G','ZNF706','ZNF707','ZNF708','ZNF709','ZNF71','ZNF710','ZNF711','ZNF713','ZNF714','ZNF716','ZNF717','ZNF718','ZNF720','ZNF721','ZNF724','ZNF726','ZNF727','ZNF728','ZNF729','ZNF73','ZNF730','ZNF732','ZNF735','ZNF736','ZNF737','ZNF738','ZNF74','ZNF740','ZNF746','ZNF747','ZNF749','ZNF750','ZNF75A','ZNF75D','ZNF76','ZNF761','ZNF763','ZNF764','ZNF765','ZNF766','ZNF768','ZNF77','ZNF770','ZNF771','ZNF772','ZNF773','ZNF774','ZNF775','ZNF776','ZNF777','ZNF778','ZNF780A','ZNF780B','ZNF781','ZNF782','ZNF783','ZNF784','ZNF785','ZNF786','ZNF787','ZNF788','ZNF789','ZNF79','ZNF790','ZNF791','ZNF792','ZNF793','ZNF799','ZNF8','ZNF80','ZNF800','ZNF804A','ZNF804B','ZNF805','ZNF808','ZNF81','ZNF812P','ZNF813','ZNF814','ZNF816','ZNF821','ZNF823','ZNF827','ZNF829','ZNF83','ZNF830','ZNF831','ZNF835','ZNF836','ZNF837','ZNF839','ZNF84','ZNF841','ZNF843','ZNF844','ZNF845','ZNF846','ZNF85','ZNF850','ZNF852','ZNF853','ZNF860','ZNF862','ZNF865','ZNF878','ZNF879','ZNF880','ZNF883','ZNF888','ZNF891','ZNF90','ZNF91','ZNF92','ZNF93','ZNF98','ZNF99','ZNFX1','ZRANB2','ZRSR1','ZRSR2','ZSCAN1','ZSCAN10','ZSCAN12','ZSCAN16','ZSCAN18','ZSCAN2','ZSCAN20','ZSCAN21','ZSCAN22','ZSCAN23','ZSCAN25','ZSCAN26','ZSCAN29','ZSCAN30','ZSCAN31','ZSCAN32','ZSCAN4','ZSCAN5A','ZSCAN5B','ZSCAN5C','ZSCAN5DP','ZSCAN9','ZUFSP','ZXDA','ZXDB','ZXDC','ZZZ3']
TFs4Other=['ATF2','ATF6B','ATN1','BHLHA15','BHLHB2','BHLHB4 ','BHLHB5','BHLHB9','C13ORF8','C20ORF194','CDX2','CRAMP1L','ELP3','ERVK3-1','ETF1','FLJ12895','FLJ20321','FLJ25476','FLJ35784','FOXM1','FOXO2','FOXR1','FOXR2','GATA1','GRLF1','KDM4C','KDM4E','KIAA0863','KIAA0961','KLRG1','LBXCOR1','LEF1','LHX7','LOC126295','LOC51058','LOC91661','MGC4728','MYOD1','MYOG','MYRF','NEUROD4','NEUROG1','NFAT2','NFAT4','NFE2L1','NR1A2','NR1B1','NR1B2','NR1B3','NR1C1','NR1C2','NR1C3','NR1F3','NR1I1','NR2A1','NR2B3','NR3A1','NR3A2','NR3B3','OTEX','PEPP-2','PHF2','POU2F1','POU2F2','POU4F3','POU5F1','POU6F1','PRDM3','PRKRIP1','PRKRIR','SETD1A','SMAD8','SMYD2','SOX12','SOX14','SOX15','TBX14','TCF1','TCF17','TCF2','TCF9','TEAD1','TPRXL','UBTFL1','WHSC1','ZBTB14','ZBTB18','ZBTB19','ZBTB21','ZBTB27','ZBTB28','ZBTB35','ZBTB49','ZFP161','ZFP28','ZKSCAN10','ZKSCAN11','ZKSCAN12','ZKSCAN13','ZKSCAN14','ZKSCAN16','ZKSCAN19','ZKSCAN21','ZKSCAN24','ZKSCAN6','ZKSCAN8','ZNF10','ZNF112','ZNF187','ZNF193','ZNF204P','ZNF238','ZNF271','ZNF295','ZNF305','ZNF323','ZNF434','ZNF499','ZNF542','ZNF545','ZNF553','ZNF56','ZNF625','ZNF643','ZNF698','ZNF705F','ZNF751','ZNF752','ZNF753','ZNF785','ZNF806','ZNF816','ZNF833P','ZNF906','ZSCAN1','ZSCAN25','ZSCAN26','ZSCAN31','ZSCAN32','ZSCAN5D','ZSCAN9']
TFsOther=TFs1Other+TFs2Other+TFs3Other+TFs4Other
```

Example 9 (unknown):
```unknown
S genes: ['MCM5', 'PCNA', 'TYMS', 'FEN1', 'MCM2', 'MCM4', 'RRM1', 'UNG', 'GINS2', 'MCM6', 'CDCA7', 'DTL', 'PRIM1', 'UHRF1', 'HELLS', 'RFC2', 'RPA2', 'NASP', 'RAD51AP1', 'GMNN', 'WDR76', 'SLBP', 'CCNE2', 'UBR7', 'POLD3', 'MSH2', 'ATAD2', 'RAD51', 'RRM2', 'CDC45', 'CDC6', 'EXO1', 'TIPIN', 'DSCC1', 'BLM', 'CASP8AP2', 'USP1', 'CLSPN', 'POLA1', 'CHAF1B', 'BRIP1', 'E2F8'] 

G2 genes: ['HMGB2', 'CDK1', 'NUSAP1', 'UBE2C', 'BIRC5', 'TPX2', 'TOP2A', 'NDC80', 'CKS2', 'NUF2', 'CKS1B', 'MKI67', 'TMPO', 'CENPF', 'TACC3', 'SMC4', 'CCNB2', 'CKAP2L', 'CKAP2', 'AURKB', 'BUB1', 'KIF11', 'ANP32E', 'TUBB4B', 'GTSE1', 'KIF20B', 'HJURP', 'CDCA3', 'CDC20', 'TTK', 'CDC25C', 'KIF2C', 'RANGAP1', 'NCAPD2', 'DLGAP5', 'CDCA2', 'CDCA8', 'ECT2', 'KIF23', 'HMMR', 'AURKA', 'PSRC1', 'ANLN', 'LBR', 'CKAP5', 'CENPE', 'CTCF', 'NEK2', 'G2E3', 'GAS2L3', 'CBX5', 'CENPA'] 

Hemoglobin genes set 1: ['HBB', 'HBD', 'HBG1', 'HBG2', 'HBE1', 'HBZ', 'HBM', 'HBA2', 'HBA1', 'HBQ1'] 

Hemoglobin genes set 2: ['AHSP', 'GYPA', 'RHAG', 'GFI1B', 'KEL', 'EPB42', 'ALAS2', 'TSPO2', 'GYPB', 'GYPE', 'SPTA1', 'RHCE', 'KLF1', 'RHD', 'SLC4A1', 'SNX22', 'SQSTM1', 'SOD1', 'FTH1', 'MAP1LC3B', 'SLC25A37', 'TRIM58', 'HMBS'] 

Mitochondrial genes: ['MT-ND1', 'MT-ND2', 'MT-CO1', 'MT-CO2', 'MT-ATP8', 'MT-ATP6', 'MT-CO3', 'MT-ND3', 'MT-ND4L', 'MT-ND4', 'MT-ND5', 'MT-ND6', 'MT-CYB'] 

Ribo genes: ['RPLP0', 'RPLP1', 'RPLP2', 'RPL3', 'RPL3L', 'RPL4', 'RPL5', 'RPL6', 'RPL7', 'RPL7A', 'RPL7L1', 'RPL8', 'RPL9', 'RPL10', 'RPL10A', 'RPL10L', 'RPL11', 'RPL12', 'RPL13', 'RPL13A', 'RPL14', 'RPL15', 'RPL17', 'RPL18', 'RPL18A', 'RPL19', 'RPL21', 'RPL22', 'RPL22L1', 'RPL23', 'RPL23A', 'RPL24', 'RPL26', 'RPL26L1', 'RPL27', 'RPL27A', 'RPL28', 'RPL29', 'RPL30', 'RPL31', 'RPL32', 'RPL34', 'RPL35', 'RPL35A', 'RPL36', 'RPL36A', 'RPL36AL', 'RPL37', 'RPL37A', 'RPL38', 'RPL39', 'RPL39L', 'UBA52', 'RPL41', 'RPSA', 'RPS2', 'RPS3', 'RPS3A', 'RPS4X', 'RPS4Y1', 'RPS4Y2', 'RPS5', 'RPS6', 'RPS7', 'RPS8', 'RPS9', 'RPS10', 'RPS11', 'RPS12', 'RPS13', 'RPS14', 'RPS15', 'RPS15A', 'RPS16', 'RPS17', 'RPS18', 'RPS19', 'RPS20', 'RPS21', 'RPS23', 'RPS24', 'RPS25', 'RPS26', 'RPS27', 'RPS27A', 'RPS27L', 'RPS28', 'RPS29', 'FAU'] 

Tissue dissociation stress associated genes: ['FOS', 'FOSB', 'JUN', 'JUNB', 'JUND', 'ZFP36', 'SOCS3', 'CEBPD', 'CEBPB', 'ATF3', 'EGR1', 'HSPA1A', 'HSPA1B', 'HSP90AA1', 'HSP90AB1', 'HSPA8', 'HSPB1', 'HSPE1', 'IER3', 'IER2', 'BTG1', 'BTG2', 'DUSP1']
```

Example 10 (python):
```python
#Expression: "YlGn" (e.g. DE plots)
#Divergent expression: "PRGn" (e.g. log2FC)
#Age: "afmhot_r"
#Others: "viridis" (e.g. UMAP expression)

#Define color palette
FineGrainPalette={
    'AtrialCardiomyocytesLeft':'#eac075',
    'AtrialCardiomyocytesRight':'#4c6b8a',
    'AtrialCardiomyocytesCycling':'#e94bc7',
    'VentricularCardiomyocytesLeftCompact':'#ab1c3c',
    'VentricularCardiomyocytesRightCompact':'#fdba45',
    'VentricularCardiomyocytesLeftTrabeculated':'#2b9200',
    'VentricularCardiomyocytesRightTrabeculated':'#0273a8',
    'VentricularCardiomyocytesCycling':'#e1acff',
    'SinoatrialNodePacemakerCells':'#854834',
    'AtrioventricularNodePacemakerCells':'#688500',
    'VentricularConductionSystemProximal':'#01ccc8',
    'VentricularConductionSystemDistal':'#b58fab',
    'GreatVesselAdventitialFibroblasts':'#ff7e97',
    'CoronaryVesselAdventitialFibroblasts':'#ff9b6e',
    'MyocardialInterstitialFibroblasts':'#8a6800',
    'SubEpicardialFibroblasts':'#206601',
    'Myofibroblasts':'#1bdebf',
    'LymphNodeFibroblasticReticularCells':'#005aa0',
    'ValveInterstitialCells':'#e478ff',
    'GreatVesselSmoothMuscleCells':'#ff665f',
    'CoronarySmoothMuscleCells':'#d2cb18',
    'DuctusArteriosusSmoothMuscleCells':'#83d7b3',
    'CoronaryPericytes':'#9088ff',
    'PericardialCellsIntermediate':'#c95e00',
    'PericardialCellsFibrous':'#01e077',
    'PericardialCellsParietal':'#ff88e4',
    'GreatVesselArterialEndothelialCells':'#d4004b',
    'GreatVesselVenousEndothelialCells':'#8f4300',
    'CoronaryArterialEndothelialCells':'#89da5a',
    'CoronaryVenousEndothelialCells':'#006f86',
    'CoronaryCapillaryEndothelialCells':'#aa7dff',
    'EndocardialCells':'#ff8241',
    'EndocardialCushionCells':'#46603a',
    'ValveEndothelialCells':'#414eb6',
    'LymphaticEndothelialCells':'#8b3f60',
    'MesothelialEpicardialCells':'#0295fe',
    'EpicardiumDerivedCells':'#ff92c4',
    'NeuronPrecursors':'#d30038',
    'ChromaffinCells':'#726800',
    'SympatheticNeurons':'#01894b',
    'ParaympatheticNeurons':'#545191',
    'SchwannCellPrecursors':'#b0d276',
    'SchwannCells':'#b30079',
    'MonocytesMPOpos':'#c50066',
    'Monocytes':'#fab4b6',
    'MonocyteDerivedCells':'#d14008',
    'MacrophagesCX3CR1pos':'#eabe97',
    'MacrophagesTIMD4pos':'#b2a200',
    'MacrophagesLYVE1pos':'#4a6005',
    'MacrophagesATF3pos':'#3db833',
    'DendriticCellsType1':'#007859',
    'DendriticCellsMature':'#01a9db',
    'PlasmacytoidDendriticCells':'#2c54ab',
    'MastCells':'#d0bef2',
    'Megakaryocytes':'#94298f',
    'TCellsCD4pos':'#ff387c',
    'TCellsCD8pos':'#9e3429',
    'TregsCD4pos':'#b37700',
    'ProBCells':'#8dab00',
    'BCells':'#a8d1a1',
    'BCellsMS4A1pos':'#00d2f0',
    'NaturalKillerCells':'#abb4ff',
    'InnateLymphoidCells':'#72497f',
}
sc.pl.umap(adata, color='fine_grain',palette=FineGrainPalette)

MidGrainPalette={
    'AtrialCardiomyocytes':'#af3a4d',
    'VentricularCardiomyocytes':'#b1756c',
    'CardiacConductionSystem':'#d45633',
    'Fibroblasts':'#9b6e4e',
    'MuralCells':'#ffb92f',
    'PericardialCells':'#b4f03b',
    'BloodVesselEndothelialCells':'#72cc57',
    'EndocardialCells':'#436e43',
    'LymphaticEndothelialCells':'#85cca0',
    'EpicardialCells':'#5eabb0',
    'Neurons':'#7bbdff',
    'Glia':'#403c90',
    'MyeloidCells':'#420066',
    'LymphoidCells':'#f553f6',
}
sc.pl.umap(adata, color='mid_grain',palette=MidGrainPalette)

CoarseGrainPalette={
    'Cardiomyocytes':'#bc5950',
    'Mesenchymal':'#af9c4c',
    'Endothelium':'#599e58',
    'Epicardium':'#5eabb0',
    'Neural':'#5e75b1',
    'Leukocytes':'#9a52a0',
}
sc.pl.umap(adata, color='coarse_grain',palette=CoarseGrainPalette)

RegionGrainPalette={
    'aorta':'#ffa2bc',
    'apex':'#fe0049',
    'atria':'#732d00',
    'atrial septum':'#ffd54c',
    'base':'#837100',
    'ductus arteriosus':'#bdff8b',
    'whole sample':'#1b6c00',
    'heart without node':'#02e6c6',
    'left ventricle':'#01c1f5',
    'node':'#00466d',
    'outflow tract':'#020011',
    'pericardium':'#7a75ff',
    'pulmonary arches and branches':'#30003e',
    'right ventricle':'#ff88e1',
    'right ventricle and papillaries':'#98004f',
}
sc.pl.umap(adata, color='region',palette=RegionGrainPalette)
```

Example 11 (python):
```python
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
/tmp/ipykernel_61971/621898565.py in <module>
     70     'InnateLymphoidCells':'#72497f',
     71 }
---> 72 sc.pl.umap(adata, color='fine_grain',palette=FineGrainPalette)
     73 
     74 MidGrainPalette={

~/miniconda3/envs/scanpynew/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py in umap(adata, **kwargs)
    657     tl.umap
    658     """
--> 659     return embedding(adata, 'umap', **kwargs)
    660 
    661 

~/miniconda3/envs/scanpynew/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py in embedding(adata, basis, color, gene_symbols, use_raw, sort_order, edges, edges_width, edges_color, neighbors_key, arrows, arrows_kwds, groups, components, layer, projection, scale_factor, color_map, cmap, palette, na_color, na_in_legend, size, frameon, legend_fontsize, legend_fontweight, legend_loc, legend_fontoutline, vmax, vmin, vcenter, norm, add_outline, outline_width, outline_color, ncols, hspace, wspace, title, show, save, ax, return_fig, **kwargs)
    250             groups=groups,
    251         )
--> 252         color_vector, categorical = _color_vector(
    253             adata,
    254             value_to_plot,

~/miniconda3/envs/scanpynew/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py in _color_vector(adata, values_key, values, palette, na_color)
   1267         color_map = {
   1268             k: to_hex(v)
-> 1269             for k, v in _get_palette(adata, values_key, palette=palette).items()
   1270         }
   1271         # If color_map does not have unique values, this can be slow as the

~/miniconda3/envs/scanpynew/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py in _get_palette(adata, values_key, palette)
   1233     values = pd.Categorical(adata.obs[values_key])
   1234     if palette:
-> 1235         _utils._set_colors_for_categorical_obs(adata, values_key, palette)
   1236     elif color_key not in adata.uns or len(adata.uns[color_key]) < len(
   1237         values.categories

~/miniconda3/envs/scanpynew/lib/python3.8/site-packages/scanpy/plotting/_utils.py in _set_colors_for_categorical_obs(adata, value_to_plot, palette)
    388         colors_list = [to_hex(x) for x in cmap(np.linspace(0, 1, len(categories)))]
    389     elif isinstance(palette, cabc.Mapping):
--> 390         colors_list = [to_hex(palette[k], keep_alpha=True) for k in categories]
    391     else:
    392         # check if palette is a list and convert it to a cycler, thus

~/miniconda3/envs/scanpynew/lib/python3.8/site-packages/scanpy/plotting/_utils.py in <listcomp>(.0)
    388         colors_list = [to_hex(x) for x in cmap(np.linspace(0, 1, len(categories)))]
    389     elif isinstance(palette, cabc.Mapping):
--> 390         colors_list = [to_hex(palette[k], keep_alpha=True) for k in categories]
    391     else:
    392         # check if palette is a list and convert it to a cycler, thus

KeyError: 'ParasympatheticNeurons'
```

Example 12 (python):
```python
set(adata.obs.fine_grain)
```

Example 13 (unknown):
```unknown
{'AtrialCardiomyocytesCycling',
 'AtrialCardiomyocytesLeft',
 'AtrialCardiomyocytesRight',
 'AtrioventricularNodePacemakerCells',
 'BCells',
 'BCellsMS4A1pos',
 'ChromaffinCells',
 'CoronaryArterialEndothelialCells',
 'CoronaryCapillaryEndothelialCells',
 'CoronaryPericytes',
 'CoronarySmoothMuscleCells',
 'CoronaryVenousEndothelialCells',
 'CoronaryVesselAdventitialFibroblasts',
 'DendriticCellsMature',
 'DendriticCellsType1',
 'DuctusArteriosusSmoothMuscleCells',
 'EndocardialCells',
 'EndocardialCushionCells',
 'EpicardiumDerivedCells',
 'GreatVesselAdventitialFibroblasts',
 'GreatVesselArterialEndothelialCells',
 'GreatVesselSmoothMuscleCells',
 'GreatVesselVenousEndothelialCells',
 'InnateLymphoidCells',
 'LymphNodeFibroblasticReticularCells',
 'LymphaticEndothelialCells',
 'MacrophagesATF3pos',
 'MacrophagesCX3CR1pos',
 'MacrophagesLYVE1pos',
 'MacrophagesTIMD4pos',
 'MastCells',
 'Megakaryocytes',
 'MesothelialEpicardialCells',
 'MonocyteDerivedCells',
 'Monocytes',
 'MonocytesMPOpos',
 'MyocardialInterstitialFibroblasts',
 'Myofibroblasts',
 'NaturalKillerCells',
 'NeuronPrecursors',
 'ParaympatheticNeurons',
 'PericardialCellsFibrous',
 'PericardialCellsIntermediate',
 'PericardialCellsParietal',
 'PlasmacytoidDendriticCells',
 'ProBCells',
 'SchwannCellPrecursors',
 'SchwannCells',
 'SinoatrialNodePacemakerCells',
 'SubEpicardialFibroblasts',
 'SympatheticNeurons',
 'TCellsCD4pos',
 'TCellsCD8pos',
 'TregsCD4pos',
 'ValveEndothelialCells',
 'ValveInterstitialCells',
 'VentricularCardiomyocytesCycling',
 'VentricularCardiomyocytesLeftCompact',
 'VentricularCardiomyocytesLeftTrabeculated',
 'VentricularCardiomyocytesRightCompact',
 'VentricularCardiomyocytesRightTrabeculated',
 'VentricularConductionSystemDistal',
 'VentricularConductionSystemProximal'}
```

Example 14 (python):
```python
set(adata.obs.mid_grain)
```

Example 15 (unknown):
```unknown
{'AtrialCardiomyocytes',
 'BloodVesselEndothelialCells',
 'CardiacConductionSystem',
 'EndocardialCells',
 'EpicardialCells',
 'Fibroblasts',
 'Glia',
 'LymphaticEndothelialCells',
 'LymphoidCells',
 'MuralCells',
 'MyeloidCells',
 'Neurons',
 'PericardialCells',
 'VentricularCardiomyocytes'}
```

Example 16 (python):
```python
set(adata.obs.coarse_grain)
```

Example 17 (unknown):
```unknown
{'Cardiomyocytes',
 'Endothelium',
 'Epicardium',
 'Leukocytes',
 'Mesenchymal',
 'Neural'}
```

Example 18 (python):
```python
#adata = adata[adata.obs['fine_grain'].isin(['CoronaryVesselAdventitialFibroblasts','MyocardialInterstitialFibroblasts1','MyocardialInterstitialFibroblasts2','Myofibroblasts','GreatVesselAdventitialFibroblasts'])].copy()
#adata = adata[adata.obs['fine_grain'].isin(['CoronaryArterialEndothelialCells','CoronaryCapillaryEndothelialCells','CoronaryVenousEndothelialCells','GreatVesselArterialEndothelialCells','GreatVesselVenousEndothelialCells'])].copy()
#adata = adata[adata.obs['fine_grain'].isin(['Myofibroblasts','CoronaryVesselAdventitialFibroblasts','SubEpicardialFibroblasts','MyocardialInterstitialFibroblasts'])].copy()
#adata = adata[adata.obs['mid_grain'].isin(['PericardialCells'])].copy()
adata=adata[adata.obs.coarse_grain=='Leukocytes',:].copy()
```

Example 19 (unknown):
```unknown
AnnData object with n_obs × n_vars = 41046 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score'
    var: 'gene_name_scRNA-0', 'gene_id'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'age_colors', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'cycling_colors', 'donor_colors', 'fine_grain_colors', 'heart_or_greatvessels_colors', 'hvg', 'kit_10x_colors', 'mid_grain_colors', 'modality_colors', 'neighbors', 'phase_colors', 'region_colors', 'sex_colors', 'trimester_colors', 'umap', 'week_colors'
    obsm: 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 20 (python):
```python
set(adata.obs.fine_grain)
```

Example 21 (unknown):
```unknown
{'CoronaryVesselAdventitialFibroblasts',
 'MyocardialInterstitialFibroblasts',
 'Myofibroblasts',
 'SubEpicardialFibroblasts'}
```

Example 22 (python):
```python
# preserve raw count
adata.layers["counts"] = adata.X.copy()

# normalise and log-transform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key='batch_key')
#sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5)

print(sum(adata.var.highly_variable))
sc.pl.highly_variable_genes(adata)

# freeze the state in `.raw`
adata.raw = adata

# subset hvgs
adata = adata[:, adata.var.highly_variable]

adata=adata.copy()

sc.tl.score_genes_cell_cycle(adata, s_genes=s_genes, g2m_genes=g2m_genes)
#sc.pp.regress_out(adata, ['S_score','G2M_score'])

#batch correction
scvi.model.SCVI.setup_anndata(adata, 
                              layer="counts", 
                              batch_key="batch_key",
                              categorical_covariate_keys=['phase'],
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo'])

vae = scvi.model.SCVI(adata, n_hidden = 128, n_latent = 50, n_layers = 3, dispersion = 'gene-batch')
vae.train(max_epochs=50)
    
# Convergence plot
x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
plt.plot(x, vae.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
plt.clf()

# add the latent space to adata
adata.obsm["X_scVI"] = vae.get_latent_representation()

sc.tl.score_genes(adata, stress_genes, score_name='stress_score')
sc.tl.score_genes(adata, hb_genes1, score_name='hb1_score')

sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata,min_dist=0.1)
#sc.tl.tsne(adata)
sc.pl.umap(adata, color=['log1p_total_counts','n_genes','n_genes_by_counts','total_counts','pct_counts_mt','pct_counts_ribo'],ncols=5)
sc.pl.umap(adata, color=['scrublet_score','stress_score', 'hb1_score','phase'], ncols=5,)

sc.pl.umap(adata, color=['donor','age','week','trimester','region','phase','cycling','cell_or_nuclei','modality','kit_10x','FACSgate','heart_or_greatvessels','coarse_grain','mid_grain','fine_grain'],wspace=0.8,ncols=5)
Genes=['TTN','TNNT2','MYOM1','LDB3','ALDH1A2','TNNT1','KRT19','UPK3B','BNC1','COL6A3','FBLN1','PCOLCE','LRRC17','PCDH18','EMCN','ECSCR','PECAM1','CDH5','ESAM', 'LAPTM5','PTPRC','LCP1','CD53','WAS','PHOX2B','NRXN1','STMN2','GRIK2','L1CAM']
sc.pl.umap(adata, color=Genes)
```

Example 23 (python):
```python
normalizing counts per cell
    finished (0:00:00)
extracting highly variable genes
    finished (0:00:09)
--> added
    'highly_variable', boolean vector (adata.var)
    'means', float vector (adata.var)
    'dispersions', float vector (adata.var)
    'dispersions_norm', float vector (adata.var)
1745
```

Example 24 (python):
```python
calculating cell cycle phase
computing score 'S_score'
WARNING: genes are not in var_names and ignored: ['FEN1', 'MCM2', 'MCM4', 'RRM1', 'UNG', 'GINS2', 'MCM6', 'CDCA7', 'DTL', 'PRIM1', 'RFC2', 'RPA2', 'NASP', 'RAD51AP1', 'GMNN', 'WDR76', 'SLBP', 'CCNE2', 'UBR7', 'POLD3', 'MSH2', 'RAD51', 'CDC45', 'CDC6', 'EXO1', 'TIPIN', 'DSCC1', 'BLM', 'CASP8AP2', 'USP1', 'CLSPN', 'POLA1', 'CHAF1B', 'E2F8']
    finished: added
    'S_score', score of gene set (adata.obs).
    121 total control genes are used. (0:00:00)
computing score 'G2M_score'
WARNING: genes are not in var_names and ignored: ['BIRC5', 'NDC80', 'CKS2', 'TMPO', 'TACC3', 'CKAP2L', 'AURKB', 'ANP32E', 'KIF20B', 'HJURP', 'CDCA3', 'TTK', 'CDC25C', 'KIF2C', 'RANGAP1', 'NCAPD2', 'CDCA2', 'CDCA8', 'ECT2', 'KIF23', 'AURKA', 'PSRC1', 'LBR', 'CKAP5', 'CTCF', 'NEK2', 'G2E3', 'CBX5']
    finished: added
    'G2M_score', score of gene set (adata.obs).
    532 total control genes are used. (0:00:00)
-->     'phase', cell cycle phase (adata.obs)
```

Example 25 (unknown):
```unknown
GPU available: False, used: False
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
```

Example 26 (unknown):
```unknown
Epoch 50/50: 100%|███████████| 50/50 [03:01<00:00,  3.63s/it, loss=625, v_num=1]
```

Example 27 (python):
```python
computing score 'stress_score'
WARNING: genes are not in var_names and ignored: ['JUND', 'ZFP36', 'CEBPD', 'CEBPB', 'HSP90AB1', 'HSPA8', 'HSPE1', 'IER2', 'BTG1']
    finished: added
    'stress_score', score of gene set (adata.obs).
    340 total control genes are used. (0:00:00)
computing score 'hb1_score'
WARNING: genes are not in var_names and ignored: ['HBZ', 'HBM', 'HBQ1']
    finished: added
    'hb1_score', score of gene set (adata.obs).
    293 total control genes are used. (0:00:00)
computing neighbors
    finished: added to `.uns['neighbors']`
    `.obsp['distances']`, distances for each pair of neighbors
    `.obsp['connectivities']`, weighted adjacency matrix (0:00:04)
computing UMAP
    finished: added
    'X_umap', UMAP coordinates (adata.obsm) (0:00:12)
```

Example 28 (unknown):
```unknown
... storing 'phase' as categorical
```

Example 29 (unknown):
```unknown
<Figure size 320x320 with 0 Axes>
```

Example 30 (python):
```python
for k in list(set(adata.obs.region)):
    sc.pl.umap(adata, color='region',groups=k,palette=['red','blue'])
```

Example 31 (unknown):
```unknown
WARNING: Length of palette colors is smaller than the number of categories (palette length: 2, categories length: 15. Some categories will have the same color.
```

Example 32 (unknown):
```unknown
WARNING: Length of palette colors is smaller than the number of categories (palette length: 2, categories length: 15. Some categories will have the same color.
```

Example 33 (unknown):
```unknown
WARNING: Length of palette colors is smaller than the number of categories (palette length: 2, categories length: 15. Some categories will have the same color.
```

Example 34 (unknown):
```unknown
WARNING: Length of palette colors is smaller than the number of categories (palette length: 2, categories length: 15. Some categories will have the same color.
```

Example 35 (unknown):
```unknown
WARNING: Length of palette colors is smaller than the number of categories (palette length: 2, categories length: 15. Some categories will have the same color.
```

Example 36 (unknown):
```unknown
WARNING: Length of palette colors is smaller than the number of categories (palette length: 2, categories length: 15. Some categories will have the same color.
```

Example 37 (unknown):
```unknown
WARNING: Length of palette colors is smaller than the number of categories (palette length: 2, categories length: 15. Some categories will have the same color.
```

Example 38 (unknown):
```unknown
WARNING: Length of palette colors is smaller than the number of categories (palette length: 2, categories length: 15. Some categories will have the same color.
```

Example 39 (unknown):
```unknown
WARNING: Length of palette colors is smaller than the number of categories (palette length: 2, categories length: 15. Some categories will have the same color.
```

Example 40 (unknown):
```unknown
WARNING: Length of palette colors is smaller than the number of categories (palette length: 2, categories length: 15. Some categories will have the same color.
```

Example 41 (unknown):
```unknown
WARNING: Length of palette colors is smaller than the number of categories (palette length: 2, categories length: 15. Some categories will have the same color.
```

Example 42 (unknown):
```unknown
WARNING: Length of palette colors is smaller than the number of categories (palette length: 2, categories length: 15. Some categories will have the same color.
```

Example 43 (unknown):
```unknown
WARNING: Length of palette colors is smaller than the number of categories (palette length: 2, categories length: 15. Some categories will have the same color.
```

Example 44 (unknown):
```unknown
WARNING: Length of palette colors is smaller than the number of categories (palette length: 2, categories length: 15. Some categories will have the same color.
```

Example 45 (unknown):
```unknown
WARNING: Length of palette colors is smaller than the number of categories (palette length: 2, categories length: 15. Some categories will have the same color.
```

Example 46 (python):
```python
sc.pl.umap(adata, color='fine_grain')
sc.pl.umap(adata, color='mid_grain')
```

Example 47 (python):
```python
sc.pl.umap(adata, color='mid_grain')
```

Example 48 (python):
```python
set(adata.obs.mid_grain)
```

Example 49 (unknown):
```unknown
{'LymphoidCells', 'MyeloidCells'}
```

Example 50 (python):
```python
set(adata.obs.coarse_grain)
```

Example 51 (python):
```python
ClusterOfInterest='InnateLymphoidCells'
#sc.tl.rank_genes_groups(adata, 'fine_grain', method='t-test', key_added = "t-test")
DEGenesForCluster = [x[ClusterOfInterest] for x in adata.uns['t-test']['names'][:50]]
CategoriesOrder=['MonocytesMPOpos','Monocytes','MonocyteDerivedCells','MacrophagesCX3CR1pos','MacrophagesTIMD4pos','MacrophagesLYVE1pos','MacrophagesATF3pos','DendriticCellsType1','DendriticCellsMature','PlasmacytoidDendriticCells','MastCells','Megakaryocytes','TCellsCD4pos','TCellsCD8pos','TregsCD4pos','ProBCells','BCells','BCellsMS4A1pos','NaturalKillerCells','InnateLymphoidCells']
sc.pl.dotplot(adata, DEGenesForCluster ,groupby='fine_grain',categories_order=CategoriesOrder,standard_scale='var',cmap='PRGn')
DEGenesForCluster = [x[ClusterOfInterest] for x in adata.uns['t-test']['names'][50:100]]
sc.pl.dotplot(adata, DEGenesForCluster ,groupby='fine_grain',categories_order=CategoriesOrder,standard_scale='var',cmap='PRGn')
DEGenesForCluster = [x[ClusterOfInterest] for x in adata.uns['t-test']['names'][100:150]]
sc.pl.dotplot(adata, DEGenesForCluster ,groupby='fine_grain',categories_order=CategoriesOrder,standard_scale='var',cmap='PRGn')
#sc.pl.umap(adata, color=DEGenesForCluster)
```

Example 52 (python):
```python
Genes=['FCN1','S100A8','S100A12','PADI4','CDA','RNASE2','MPO','RBP7','SERPINA1','CLEC10A','GPAT3','CD1C','CCR2','C3','CX3CR1','HTRA1','BHLHE41','OLFML3','IGSF21','KCNMA1','IL18','ANTXR1','ESRRG','CMKLR1','SAMD4A','MMP9','SMIM35','MYO5B','ATRNL1','F13A1','LYVE1','SELENOP','FOLR2','LILRB5','ACSL1','NR4A2','ATF3','NR4A3','CXCL8','CLNK','CADM1','CLEC9A','SLC24A4','NEGR1','ENPP1','RAB7B','ANPEP','ST18','LAMP3','ENOX1','GPC5','ADAM12','ST8SIA1','XKR4','GALNT17','EFNA5','ZFPM2','ABTB2','RUNX2','ZFAT','EPHB1','CUX2','PTPRS','CLEC4C','PHEX','CPA3','HDC','GATA2','VWA5A','IL1RL1','TPSB2','TPSAB1','MS4A2','CDK15','LIF','ADCYAP1','IL5RA','MLPH','ITGA2B','GP1BB','GATA1','ITGB3','GP9','MPIG6B','CLEC1B','BCL11B','THEMIS','TCF7','LRRN3','CD6','TRAC','TAFA1','TENM1','NELL2','CD8A','CD8B','IL2RA','CTLA4','TIGIT','FOXP3','MME','VPREB1','IGLL1','DNTT','RAG1','RAG2','SH2D4B','LCN6','PCDH9','ACSM3','ADAM23','BEST3','TCL1A','MS4A1','OSBPL10','COL19A1','CD22','FCRL2','FCRL5','KLRK1','TGFBR3','GZMA','KLRD1','IL2RB','CCL5','IL18RAP','KLRC2','TOX2','RORC','AQP3','IL1R1','HPN']
CategoriesOrder=['MonocytesMPOpos','Monocytes','MonocyteDerivedCells','MacrophagesCX3CR1pos','MacrophagesTIMD4pos','MacrophagesLYVE1pos','MacrophagesATF3pos','DendriticCellsType1','DendriticCellsMature','PlasmacytoidDendriticCells','MastCells','Megakaryocytes','TCellsCD4pos','TCellsCD8pos','TregsCD4pos','ProBCells','BCells','BCellsMS4A1pos','NaturalKillerCells','InnateLymphoidCells']
sc.pl.dotplot(adata, Genes ,groupby='fine_grain',categories_order=CategoriesOrder,cmap='YlGn',standard_scale='var',var_group_positions=[(0,84),(85,131)], var_group_labels=['myeloid cells',' lymphoid cells'],var_group_rotation=0)
```

Example 53 (python):
```python
Genes=['FCN1','S100A8','S100A12','PADI4','CDA','RNASE2','MPO','RBP7','SERPINA1','CLEC10A','GPAT3','CD1C','CCR2','C3','CX3CR1','HTRA1','BHLHE41','OLFML3','IGSF21','KCNMA1','IL18','ANTXR1','ESRRG','CMKLR1','SAMD4A','MMP9','SMIM35','MYO5B','ATRNL1','F13A1','LYVE1','SELENOP','FOLR2','LILRB5','ACSL1','NR4A2','ATF3','NR4A3','CXCL8','CLNK','CADM1','CLEC9A','SLC24A4','NEGR1','ENPP1','RAB7B','ANPEP','ST18','LAMP3','ENOX1','GPC5','ADAM12','ST8SIA1','XKR4','GALNT17','EFNA5','ZFPM2','ABTB2','RUNX2','ZFAT','EPHB1','CUX2','PTPRS','CLEC4C','PHEX','CPA3','HDC','GATA2','VWA5A','IL1RL1','TPSB2','TPSAB1','MS4A2','CDK15','LIF','ADCYAP1','IL5RA','MLPH','ITGA2B','GP1BB','GATA1','ITGB3','GP9','MPIG6B','CLEC1B']
CategoriesOrder=['MonocytesMPOpos','Monocytes','MonocyteDerivedCells','MacrophagesCX3CR1pos','MacrophagesTIMD4pos','MacrophagesLYVE1pos','MacrophagesATF3pos','DendriticCellsType1','DendriticCellsMature','PlasmacytoidDendriticCells','MastCells','Megakaryocytes','TCellsCD4pos','TCellsCD8pos','TregsCD4pos','ProBCells','BCells','BCellsMS4A1pos','NaturalKillerCells','InnateLymphoidCells']
sc.pl.dotplot(adata, Genes ,groupby='fine_grain',categories_order=CategoriesOrder,cmap='YlGn',standard_scale='var',var_group_positions=[(0,84)], var_group_labels=['myeloid cells'],var_group_rotation=0)
```

Example 54 (python):
```python
Genes=['BCL11B','THEMIS','TCF7','CD6','TRAC','LRRN3','TAFA1','TENM1','NELL2','CD8A','CD8B','IL2RA','CTLA4','TIGIT','FOXP3','MME','VPREB1','IGLL1','DNTT','RAG1','RAG2','SH2D4B','LCN6','PCDH9','ACSM3','ADAM23','BEST3','TCL1A','MS4A1','OSBPL10','COL19A1','CD22','FCRL2','FCRL5','KLRK1','TGFBR3','GZMA','KLRD1','IL2RB','CCL5','IL18RAP','KLRC2','TOX2','RORC','AQP3','IL1R1','HPN']
CategoriesOrder=['MonocytesMPOpos','Monocytes','MonocyteDerivedCells','MacrophagesCX3CR1pos','MacrophagesTIMD4pos','MacrophagesLYVE1pos','MacrophagesATF3pos','DendriticCellsType1','DendriticCellsMature','PlasmacytoidDendriticCells','MastCells','Megakaryocytes','TCellsCD4pos','TCellsCD8pos','TregsCD4pos','ProBCells','BCells','BCellsMS4A1pos','NaturalKillerCells','InnateLymphoidCells']
sc.pl.dotplot(adata, Genes ,groupby='fine_grain',categories_order=CategoriesOrder,cmap='YlGn',standard_scale='var',var_group_positions=[(0,46)], var_group_labels=[' lymphoid cells'],var_group_rotation=0)
```

Example 55 (python):
```python
sc.tl.rank_genes_groups(adata, 'fine_grain', groups=['MonocytesMPOpos'], reference='Monocytes', method='t-test')
sc.pl.rank_genes_groups(adata, groups=['MonocytesMPOpos'], n_genes=20)
```

Example 56 (python):
```python
ranking genes
    finished: added to `.uns['rank_genes_groups']`
    'names', sorted np.recarray to be indexed by group ids
    'scores', sorted np.recarray to be indexed by group ids
    'logfoldchanges', sorted np.recarray to be indexed by group ids
    'pvals', sorted np.recarray to be indexed by group ids
    'pvals_adj', sorted np.recarray to be indexed by group ids (0:00:00)
```

Example 57 (python):
```python
#adata.write('./ObjectsToUpload/Leukocytes.h5ad')
```

Example 58 (python):
```python
adata=sc.read('./ObjectsToUpload/Leukocytes.h5ad')
print(adata.shape)
```

Example 59 (unknown):
```unknown
(41046, 1745)
```

Example 60 (python):
```python
adata = adata[adata.obs['fine_grain'].isin(['MacrophagesATF3pos','MacrophagesCX3CR1pos','MacrophagesLYVE1pos','MacrophagesTIMD4pos','Monocytes','MonocyteDerivedCells','MonocytesMPOpos'])].copy()
adata
```

Example 61 (unknown):
```unknown
AnnData object with n_obs × n_vars = 17335 × 1745
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score'
    var: 'gene_name_scRNA-0', 'gene_id', 'highly_variable', 'means', 'dispersions', 'dispersions_norm', 'highly_variable_nbatches', 'highly_variable_intersection'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'age_colors', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'cycling_colors', 'donor_colors', 'fine_grain_colors', 'heart_or_greatvessels_colors', 'hvg', 'kit_10x_colors', 'mid_grain_colors', 'modality_colors', 'neighbors', 'phase_colors', 'region_colors', 'sex_colors', 'trimester_colors', 'umap', 'week_colors', 'log1p', 't-test', 'rank_genes_groups'
    obsm: 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    layers: 'counts'
    obsp: 'connectivities', 'distances'
```

Example 62 (python):
```python
set(adata.obs.fine_grain)
```

Example 63 (unknown):
```unknown
{'MacrophagesATF3pos',
 'MacrophagesCX3CR1pos',
 'MacrophagesLYVE1pos',
 'MacrophagesTIMD4pos',
 'MonocyteDerivedCells',
 'Monocytes',
 'MonocytesMPOpos'}
```

Example 64 (python):
```python
#This box to run enrichment right away
ClusterOfInterest='MacrophagesATF3pos'
sc.tl.rank_genes_groups(adata, 'fine_grain', method='wilcoxon', key_added = "wilcoxon")
gene_set_names = gseapy.get_library_name(organism='Human')
#print(gene_set_names)
glist = sc.get.rank_genes_groups_df(adata, group=ClusterOfInterest, 
                                    key='wilcoxon', log2fc_min=0.25, 
                                    pval_cutoff=0.05)['names'].squeeze().str.strip().tolist()
#print(len(glist))                                  
#GO_Biological_Process_2021,GO_Cellular_Component_2021,GO_Molecular_Function_2021
enr_res = gseapy.enrichr(gene_list=glist,
                     organism='Human',
                     gene_sets='GO_Biological_Process_2021',
                     cutoff = 0.5)
gseapy.barplot(enr_res.res2d,title=ClusterOfInterest,top_term=20,figsize=(5,5))
#enr_res.results.head(20)
```

Example 65 (python):
```python
ranking genes
    finished: added to `.uns['wilcoxon']`
    'names', sorted np.recarray to be indexed by group ids
    'scores', sorted np.recarray to be indexed by group ids
    'logfoldchanges', sorted np.recarray to be indexed by group ids
    'pvals', sorted np.recarray to be indexed by group ids
    'pvals_adj', sorted np.recarray to be indexed by group ids (0:00:19)
```

Example 66 (python):
```python
#This box to run enrichment right away
ClusterOfInterest='MacrophagesCX3CR1pos'
#sc.tl.rank_genes_groups(adata, 'fine_grain', method='wilcoxon', key_added = "wilcoxon")
gene_set_names = gseapy.get_library_name(organism='Human')
#print(gene_set_names)
glist = sc.get.rank_genes_groups_df(adata, group=ClusterOfInterest, 
                                    key='wilcoxon', log2fc_min=0.25, 
                                    pval_cutoff=0.05)['names'].squeeze().str.strip().tolist()
#print(len(glist))                                  
#GO_Biological_Process_2021,GO_Cellular_Component_2021,GO_Molecular_Function_2021
enr_res = gseapy.enrichr(gene_list=glist,
                     organism='Human',
                     gene_sets='GO_Biological_Process_2021',
                     cutoff = 0.5)
gseapy.barplot(enr_res.res2d,title=ClusterOfInterest,top_term=20,figsize=(5,5))
#enr_res.results.head(20)
```

Example 67 (python):
```python
#This box to run enrichment right away
ClusterOfInterest='MacrophagesLYVE1pos'
#sc.tl.rank_genes_groups(adata, 'fine_grain', method='wilcoxon', key_added = "wilcoxon")
gene_set_names = gseapy.get_library_name(organism='Human')
#print(gene_set_names)
glist = sc.get.rank_genes_groups_df(adata, group=ClusterOfInterest, 
                                    key='wilcoxon', log2fc_min=0.25, 
                                    pval_cutoff=0.05)['names'].squeeze().str.strip().tolist()
#print(len(glist))                                  
#GO_Biological_Process_2021,GO_Cellular_Component_2021,GO_Molecular_Function_2021
enr_res = gseapy.enrichr(gene_list=glist,
                     organism='Human',
                     gene_sets='GO_Biological_Process_2021',
                     cutoff = 0.5)
gseapy.barplot(enr_res.res2d,title=ClusterOfInterest,top_term=20,figsize=(5,5))
#enr_res.results.head(20)
```

Example 68 (python):
```python
#This box to run enrichment right away
ClusterOfInterest='MacrophagesTIMD4pos'
#sc.tl.rank_genes_groups(adata, 'fine_grain', method='wilcoxon', key_added = "wilcoxon")
gene_set_names = gseapy.get_library_name(organism='Human')
#print(gene_set_names)
glist = sc.get.rank_genes_groups_df(adata, group=ClusterOfInterest, 
                                    key='wilcoxon', log2fc_min=0.25, 
                                    pval_cutoff=0.05)['names'].squeeze().str.strip().tolist()
#print(len(glist))                                  
#GO_Biological_Process_2021,GO_Cellular_Component_2021,GO_Molecular_Function_2021
enr_res = gseapy.enrichr(gene_list=glist,
                     organism='Human',
                     gene_sets='GO_Biological_Process_2021',
                     cutoff = 0.5)
gseapy.barplot(enr_res.res2d,title=ClusterOfInterest,top_term=20,figsize=(5,5))
#enr_res.results.head(20)
```

Example 69 (python):
```python
#This box to run enrichment right away
ClusterOfInterest='MonocyteDerivedCells'
#sc.tl.rank_genes_groups(adata, 'fine_grain', method='wilcoxon', key_added = "wilcoxon")
gene_set_names = gseapy.get_library_name(organism='Human')
#print(gene_set_names)
glist = sc.get.rank_genes_groups_df(adata, group=ClusterOfInterest, 
                                    key='wilcoxon', log2fc_min=0.25, 
                                    pval_cutoff=0.05)['names'].squeeze().str.strip().tolist()
#print(len(glist))                                  
#GO_Biological_Process_2021,GO_Cellular_Component_2021,GO_Molecular_Function_2021
enr_res = gseapy.enrichr(gene_list=glist,
                     organism='Human',
                     gene_sets='GO_Biological_Process_2021',
                     cutoff = 0.5)
gseapy.barplot(enr_res.res2d,title=ClusterOfInterest,top_term=20,figsize=(5,5))
#enr_res.results.head(20)
```

Example 70 (python):
```python
#This box to run enrichment right away
ClusterOfInterest='Monocytes'
#sc.tl.rank_genes_groups(adata, 'fine_grain', method='wilcoxon', key_added = "wilcoxon")
gene_set_names = gseapy.get_library_name(organism='Human')
#print(gene_set_names)
glist = sc.get.rank_genes_groups_df(adata, group=ClusterOfInterest, 
                                    key='wilcoxon', log2fc_min=0.25, 
                                    pval_cutoff=0.05)['names'].squeeze().str.strip().tolist()
#print(len(glist))                                  
#GO_Biological_Process_2021,GO_Cellular_Component_2021,GO_Molecular_Function_2021
enr_res = gseapy.enrichr(gene_list=glist,
                     organism='Human',
                     gene_sets='GO_Biological_Process_2021',
                     cutoff = 0.5)
gseapy.barplot(enr_res.res2d,title=ClusterOfInterest,top_term=20,figsize=(5,5))
#enr_res.results.head(20)
```

Example 71 (python):
```python
#This box to run enrichment right away
ClusterOfInterest='MonocytesMPOpos'
#sc.tl.rank_genes_groups(adata, 'fine_grain', method='wilcoxon', key_added = "wilcoxon")
gene_set_names = gseapy.get_library_name(organism='Human')
#print(gene_set_names)
glist = sc.get.rank_genes_groups_df(adata, group=ClusterOfInterest, 
                                    key='wilcoxon', log2fc_min=0.25, 
                                    pval_cutoff=0.05)['names'].squeeze().str.strip().tolist()
#print(len(glist))                                  
#GO_Biological_Process_2021,GO_Cellular_Component_2021,GO_Molecular_Function_2021
enr_res = gseapy.enrichr(gene_list=glist,
                     organism='Human',
                     gene_sets='GO_Biological_Process_2021',
                     cutoff = 0.5)
gseapy.barplot(enr_res.res2d,title=ClusterOfInterest,top_term=20,figsize=(5,5))
#enr_res.results.head(20)
```

Example 72 (python):
```python
#This box to run enrichment right away
ClusterOfInterest='MonocytesMPOpos'
#sc.tl.rank_genes_groups(adata, 'fine_grain', method='wilcoxon', key_added = "wilcoxon")
gene_set_names = gseapy.get_library_name(organism='Human')
#print(gene_set_names)
glist = sc.get.rank_genes_groups_df(adata, group=ClusterOfInterest, 
                                    key='wilcoxon', log2fc_min=0.25, 
                                    pval_cutoff=0.05)['names'].squeeze().str.strip().tolist()
#print(len(glist))                                  
#GO_Biological_Process_2021,GO_Cellular_Component_2021,GO_Molecular_Function_2021
enr_res = gseapy.enrichr(gene_list=glist,
                     organism='Human',
                     gene_sets='Reactome_2022',
                     cutoff = 0.5)
gseapy.barplot(enr_res.res2d,title=ClusterOfInterest,top_term=20,figsize=(5,5))
#enr_res.results.head(20)
```

Example 73 (python):
```python
print(gene_set_names)
```

Example 74 (unknown):
```unknown
['ARCHS4_Cell-lines', 'ARCHS4_IDG_Coexp', 'ARCHS4_Kinases_Coexp', 'ARCHS4_TFs_Coexp', 'ARCHS4_Tissues', 'Achilles_fitness_decrease', 'Achilles_fitness_increase', 'Aging_Perturbations_from_GEO_down', 'Aging_Perturbations_from_GEO_up', 'Allen_Brain_Atlas_10x_scRNA_2021', 'Allen_Brain_Atlas_down', 'Allen_Brain_Atlas_up', 'Azimuth_Cell_Types_2021', 'BioCarta_2013', 'BioCarta_2015', 'BioCarta_2016', 'BioPlanet_2019', 'BioPlex_2017', 'CCLE_Proteomics_2020', 'CORUM', 'COVID-19_Related_Gene_Sets', 'COVID-19_Related_Gene_Sets_2021', 'Cancer_Cell_Line_Encyclopedia', 'CellMarker_Augmented_2021', 'ChEA_2013', 'ChEA_2015', 'ChEA_2016', 'ChEA_2022', 'Chromosome_Location', 'Chromosome_Location_hg19', 'ClinVar_2019', 'DSigDB', 'Data_Acquisition_Method_Most_Popular_Genes', 'DepMap_WG_CRISPR_Screens_Broad_CellLines_2019', 'DepMap_WG_CRISPR_Screens_Sanger_CellLines_2019', 'Descartes_Cell_Types_and_Tissue_2021', 'Diabetes_Perturbations_GEO_2022', 'DisGeNET', 'Disease_Perturbations_from_GEO_down', 'Disease_Perturbations_from_GEO_up', 'Disease_Signatures_from_GEO_down_2014', 'Disease_Signatures_from_GEO_up_2014', 'DrugMatrix', 'Drug_Perturbations_from_GEO_2014', 'Drug_Perturbations_from_GEO_down', 'Drug_Perturbations_from_GEO_up', 'ENCODE_Histone_Modifications_2013', 'ENCODE_Histone_Modifications_2015', 'ENCODE_TF_ChIP-seq_2014', 'ENCODE_TF_ChIP-seq_2015', 'ENCODE_and_ChEA_Consensus_TFs_from_ChIP-X', 'ESCAPE', 'Elsevier_Pathway_Collection', 'Enrichr_Libraries_Most_Popular_Genes', 'Enrichr_Submissions_TF-Gene_Coocurrence', 'Enrichr_Users_Contributed_Lists_2020', 'Epigenomics_Roadmap_HM_ChIP-seq', 'FANTOM6_lncRNA_KD_DEGs', 'GO_Biological_Process_2013', 'GO_Biological_Process_2015', 'GO_Biological_Process_2017', 'GO_Biological_Process_2017b', 'GO_Biological_Process_2018', 'GO_Biological_Process_2021', 'GO_Cellular_Component_2013', 'GO_Cellular_Component_2015', 'GO_Cellular_Component_2017', 'GO_Cellular_Component_2017b', 'GO_Cellular_Component_2018', 'GO_Cellular_Component_2021', 'GO_Molecular_Function_2013', 'GO_Molecular_Function_2015', 'GO_Molecular_Function_2017', 'GO_Molecular_Function_2017b', 'GO_Molecular_Function_2018', 'GO_Molecular_Function_2021', 'GTEx_Aging_Signatures_2021', 'GTEx_Tissue_Expression_Down', 'GTEx_Tissue_Expression_Up', 'GTEx_Tissues_V8_2023', 'GWAS_Catalog_2019', 'GeneSigDB', 'Gene_Perturbations_from_GEO_down', 'Gene_Perturbations_from_GEO_up', 'Genes_Associated_with_NIH_Grants', 'Genome_Browser_PWMs', 'GlyGen_Glycosylated_Proteins_2022', 'HDSigDB_Human_2021', 'HDSigDB_Mouse_2021', 'HMDB_Metabolites', 'HMS_LINCS_KinomeScan', 'HomoloGene', 'HuBMAP_ASCT_plus_B_augmented_w_RNAseq_Coexpression', 'HuBMAP_ASCTplusB_augmented_2022', 'HumanCyc_2015', 'HumanCyc_2016', 'Human_Gene_Atlas', 'Human_Phenotype_Ontology', 'IDG_Drug_Targets_2022', 'InterPro_Domains_2019', 'Jensen_COMPARTMENTS', 'Jensen_DISEASES', 'Jensen_TISSUES', 'KEA_2013', 'KEA_2015', 'KEGG_2013', 'KEGG_2015', 'KEGG_2016', 'KEGG_2019_Human', 'KEGG_2019_Mouse', 'KEGG_2021_Human', 'KOMP2_Mouse_Phenotypes_2022', 'Kinase_Perturbations_from_GEO_down', 'Kinase_Perturbations_from_GEO_up', 'L1000_Kinase_and_GPCR_Perturbations_down', 'L1000_Kinase_and_GPCR_Perturbations_up', 'LINCS_L1000_CRISPR_KO_Consensus_Sigs', 'LINCS_L1000_Chem_Pert_Consensus_Sigs', 'LINCS_L1000_Chem_Pert_down', 'LINCS_L1000_Chem_Pert_up', 'LINCS_L1000_Ligand_Perturbations_down', 'LINCS_L1000_Ligand_Perturbations_up', 'Ligand_Perturbations_from_GEO_down', 'Ligand_Perturbations_from_GEO_up', 'MAGMA_Drugs_and_Diseases', 'MCF7_Perturbations_from_GEO_down', 'MCF7_Perturbations_from_GEO_up', 'MGI_Mammalian_Phenotype_2013', 'MGI_Mammalian_Phenotype_2017', 'MGI_Mammalian_Phenotype_Level_3', 'MGI_Mammalian_Phenotype_Level_4', 'MGI_Mammalian_Phenotype_Level_4_2019', 'MGI_Mammalian_Phenotype_Level_4_2021', 'MSigDB_Computational', 'MSigDB_Hallmark_2020', 'MSigDB_Oncogenic_Signatures', 'Metabolomics_Workbench_Metabolites_2022', 'Microbe_Perturbations_from_GEO_down', 'Microbe_Perturbations_from_GEO_up', 'Mouse_Gene_Atlas', 'NCI-60_Cancer_Cell_Lines', 'NCI-Nature_2015', 'NCI-Nature_2016', 'NIH_Funded_PIs_2017_AutoRIF_ARCHS4_Predictions', 'NIH_Funded_PIs_2017_GeneRIF_ARCHS4_Predictions', 'NIH_Funded_PIs_2017_Human_AutoRIF', 'NIH_Funded_PIs_2017_Human_GeneRIF', 'NURSA_Human_Endogenous_Complexome', 'OMIM_Disease', 'OMIM_Expanded', 'Old_CMAP_down', 'Old_CMAP_up', 'Orphanet_Augmented_2021', 'PFOCR_Pathways', 'PPI_Hub_Proteins', 'PanglaoDB_Augmented_2021', 'Panther_2015', 'Panther_2016', 'Pfam_Domains_2019', 'Pfam_InterPro_Domains', 'PheWeb_2019', 'PhenGenI_Association_2021', 'Phosphatase_Substrates_from_DEPOD', 'ProteomicsDB_2020', 'Proteomics_Drug_Atlas_2023', 'RNA-Seq_Disease_Gene_and_Drug_Signatures_from_GEO', 'RNAseq_Automatic_GEO_Signatures_Human_Down', 'RNAseq_Automatic_GEO_Signatures_Human_Up', 'RNAseq_Automatic_GEO_Signatures_Mouse_Down', 'RNAseq_Automatic_GEO_Signatures_Mouse_Up', 'Rare_Diseases_AutoRIF_ARCHS4_Predictions', 'Rare_Diseases_AutoRIF_Gene_Lists', 'Rare_Diseases_GeneRIF_ARCHS4_Predictions', 'Rare_Diseases_GeneRIF_Gene_Lists', 'Reactome_2013', 'Reactome_2015', 'Reactome_2016', 'Reactome_2022', 'SILAC_Phosphoproteomics', 'Serine_Threonine_Kinome_Atlas_2023', 'SubCell_BarCode', 'SynGO_2022', 'SysMyo_Muscle_Gene_Sets', 'TF-LOF_Expression_from_GEO', 'TF_Perturbations_Followed_by_Expression', 'TG_GATES_2020', 'TRANSFAC_and_JASPAR_PWMs', 'TRRUST_Transcription_Factors_2019', 'Table_Mining_of_CRISPR_Studies', 'Tabula_Muris', 'Tabula_Sapiens', 'TargetScan_microRNA', 'TargetScan_microRNA_2017', 'Tissue_Protein_Expression_from_Human_Proteome_Map', 'Tissue_Protein_Expression_from_ProteomicsDB', 'Transcription_Factor_PPIs', 'UK_Biobank_GWAS_v1', 'Virus-Host_PPI_P-HIPSTer_2020', 'VirusMINT', 'Virus_Perturbations_from_GEO_down', 'Virus_Perturbations_from_GEO_up', 'WikiPathway_2021_Human', 'WikiPathways_2013', 'WikiPathways_2015', 'WikiPathways_2016', 'WikiPathways_2019_Human', 'WikiPathways_2019_Mouse', 'dbGaP', 'huMAP', 'lncHUB_lncRNA_Co-Expression', 'miRTarBase_2017']
```

Example 75 (python):
```python
adata=sc.read('./Objects/Feb28ObjectRaw.h5ad')
print(adata.shape)
```

Example 76 (unknown):
```unknown
(297473, 36601)
```

Example 77 (python):
```python
adata = adata[adata.obs['fine_grain'].isin(['MacrophagesATF3pos','MacrophagesCX3CR1pos','MacrophagesLYVE1pos','MacrophagesTIMD4pos','Monocytes','MonocyteDerivedCells','MonocytesMPOpos'])].copy()
adata
```

Example 78 (unknown):
```unknown
AnnData object with n_obs × n_vars = 17335 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score'
    var: 'gene_name_scRNA-0', 'gene_id'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'age_colors', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'cycling_colors', 'donor_colors', 'fine_grain_colors', 'heart_or_greatvessels_colors', 'hvg', 'kit_10x_colors', 'modality_colors', 'neighbors', 'phase_colors', 'region_colors', 'sex_colors', 'trimester_colors', 'umap', 'week_colors'
    obsm: 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 79 (python):
```python
# preserve raw count
adata.layers["counts"] = adata.X.copy()

# normalise and log-transform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key='batch_key')
#sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5)

print(sum(adata.var.highly_variable))
sc.pl.highly_variable_genes(adata)

# freeze the state in `.raw`
adata.raw = adata

# subset hvgs
adata = adata[:, adata.var.highly_variable]

adata=adata.copy()

sc.tl.score_genes_cell_cycle(adata, s_genes=s_genes, g2m_genes=g2m_genes)
#sc.pp.regress_out(adata, ['S_score','G2M_score'])

#batch correction
scvi.model.SCVI.setup_anndata(adata, 
                              layer="counts", 
                              batch_key="batch_key",
                              categorical_covariate_keys=['phase'],
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo'])

vae = scvi.model.SCVI(adata, n_hidden = 128, n_latent = 50, n_layers = 3, dispersion = 'gene-batch')
vae.train(max_epochs=50)
    
# Convergence plot
x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
plt.plot(x, vae.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
plt.clf()

# add the latent space to adata
adata.obsm["X_scVI"] = vae.get_latent_representation()

sc.tl.score_genes(adata, stress_genes, score_name='stress_score')
sc.tl.score_genes(adata, hb_genes1, score_name='hb1_score')

sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata,min_dist=0.1)
#sc.tl.tsne(adata)
sc.pl.umap(adata, color=['log1p_total_counts','n_genes','n_genes_by_counts','total_counts','pct_counts_mt','pct_counts_ribo'],ncols=5)
sc.pl.umap(adata, color=['scrublet_score','stress_score', 'hb1_score','phase'], ncols=5,)

sc.pl.umap(adata, color=['donor','age','week','trimester','region','phase','cycling','cell_or_nuclei','modality','kit_10x','FACSgate','heart_or_greatvessels','coarse_grain','mid_grain','fine_grain'],wspace=0.8,ncols=5)
Genes=['TTN','TNNT2','MYOM1','LDB3','ALDH1A2','TNNT1','KRT19','UPK3B','BNC1','COL6A3','FBLN1','PCOLCE','LRRC17','PCDH18','EMCN','ECSCR','PECAM1','CDH5','ESAM', 'LAPTM5','PTPRC','LCP1','CD53','WAS','PHOX2B','NRXN1','STMN2','GRIK2','L1CAM']
sc.pl.umap(adata, color=Genes)
```

Example 80 (python):
```python
normalizing counts per cell
    finished (0:00:00)
extracting highly variable genes
    finished (0:00:08)
--> added
    'highly_variable', boolean vector (adata.var)
    'means', float vector (adata.var)
    'dispersions', float vector (adata.var)
    'dispersions_norm', float vector (adata.var)
1387
```

Example 81 (python):
```python
calculating cell cycle phase
computing score 'S_score'
WARNING: genes are not in var_names and ignored: ['FEN1', 'MCM2', 'MCM4', 'RRM1', 'UNG', 'GINS2', 'MCM6', 'CDCA7', 'PRIM1', 'HELLS', 'RFC2', 'RPA2', 'NASP', 'RAD51AP1', 'SLBP', 'CCNE2', 'UBR7', 'POLD3', 'MSH2', 'RAD51', 'CDC45', 'CDC6', 'EXO1', 'TIPIN', 'DSCC1', 'BLM', 'CASP8AP2', 'USP1', 'CHAF1B', 'E2F8']
    finished: added
    'S_score', score of gene set (adata.obs).
    413 total control genes are used. (0:00:00)
computing score 'G2M_score'
WARNING: genes are not in var_names and ignored: ['BIRC5', 'NDC80', 'NUF2', 'TMPO', 'TACC3', 'CKAP2L', 'AURKB', 'ANP32E', 'KIF20B', 'HJURP', 'CDCA3', 'TTK', 'CDC25C', 'KIF2C', 'RANGAP1', 'NCAPD2', 'CDCA2', 'CDCA8', 'ECT2', 'KIF23', 'AURKA', 'PSRC1', 'LBR', 'CKAP5', 'CTCF', 'NEK2', 'G2E3', 'CBX5', 'CENPA']
    finished: added
    'G2M_score', score of gene set (adata.obs).
    488 total control genes are used. (0:00:00)
-->     'phase', cell cycle phase (adata.obs)
```

Example 82 (unknown):
```unknown
GPU available: False, used: False
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
```

Example 83 (unknown):
```unknown
Epoch 50/50: 100%|███████████| 50/50 [01:12<00:00,  1.44s/it, loss=659, v_num=1]
```

Example 84 (python):
```python
computing score 'stress_score'
WARNING: genes are not in var_names and ignored: ['JUND', 'ZFP36', 'CEBPD', 'CEBPB', 'HSP90AB1', 'HSPA8', 'HSPE1']
    finished: added
    'stress_score', score of gene set (adata.obs).
    337 total control genes are used. (0:00:00)
computing score 'hb1_score'
WARNING: genes are not in var_names and ignored: ['HBD', 'HBM', 'HBQ1']
    finished: added
    'hb1_score', score of gene set (adata.obs).
    343 total control genes are used. (0:00:00)
computing neighbors
    finished: added to `.uns['neighbors']`
    `.obsp['distances']`, distances for each pair of neighbors
    `.obsp['connectivities']`, weighted adjacency matrix (0:00:01)
computing UMAP
    finished: added
    'X_umap', UMAP coordinates (adata.obsm) (0:00:04)
```

Example 85 (unknown):
```unknown
... storing 'phase' as categorical
```

Example 86 (unknown):
```unknown
<Figure size 320x320 with 0 Axes>
```

Example 87 (python):
```python
adata=adata[adata.obs.cycling=='no',:].copy()
```

Example 88 (python):
```python
#This box to run enrichment right away
ClusterOfInterest='MacrophagesATF3pos'
sc.tl.rank_genes_groups(adata, 'fine_grain', method='wilcoxon', key_added = "wilcoxon")
gene_set_names = gseapy.get_library_name(organism='Human')
#print(gene_set_names)
glist = sc.get.rank_genes_groups_df(adata, group=ClusterOfInterest, 
                                    key='wilcoxon', log2fc_min=0.25, 
                                    pval_cutoff=0.05)['names'].squeeze().str.strip().tolist()
#print(len(glist))                                  
#GO_Biological_Process_2021,GO_Cellular_Component_2021,GO_Molecular_Function_2021
enr_res = gseapy.enrichr(gene_list=glist,
                     organism='Human',
                     gene_sets='GO_Biological_Process_2021',
                     cutoff = 0.5)
gseapy.barplot(enr_res.res2d,title=ClusterOfInterest,top_term=20,figsize=(5,5))
#enr_res.results.head(20)

#This box to run enrichment right away
ClusterOfInterest='MacrophagesCX3CR1pos'
#sc.tl.rank_genes_groups(adata, 'fine_grain', method='wilcoxon', key_added = "wilcoxon")
gene_set_names = gseapy.get_library_name(organism='Human')
#print(gene_set_names)
glist = sc.get.rank_genes_groups_df(adata, group=ClusterOfInterest, 
                                    key='wilcoxon', log2fc_min=0.25, 
                                    pval_cutoff=0.05)['names'].squeeze().str.strip().tolist()
#print(len(glist))                                  
#GO_Biological_Process_2021,GO_Cellular_Component_2021,GO_Molecular_Function_2021
enr_res = gseapy.enrichr(gene_list=glist,
                     organism='Human',
                     gene_sets='GO_Biological_Process_2021',
                     cutoff = 0.5)
gseapy.barplot(enr_res.res2d,title=ClusterOfInterest,top_term=20,figsize=(5,5))
#enr_res.results.head(20)

#This box to run enrichment right away
ClusterOfInterest='MacrophagesLYVE1pos'
#sc.tl.rank_genes_groups(adata, 'fine_grain', method='wilcoxon', key_added = "wilcoxon")
gene_set_names = gseapy.get_library_name(organism='Human')
#print(gene_set_names)
glist = sc.get.rank_genes_groups_df(adata, group=ClusterOfInterest, 
                                    key='wilcoxon', log2fc_min=0.25, 
                                    pval_cutoff=0.05)['names'].squeeze().str.strip().tolist()
#print(len(glist))                                  
#GO_Biological_Process_2021,GO_Cellular_Component_2021,GO_Molecular_Function_2021
enr_res = gseapy.enrichr(gene_list=glist,
                     organism='Human',
                     gene_sets='GO_Biological_Process_2021',
                     cutoff = 0.5)
gseapy.barplot(enr_res.res2d,title=ClusterOfInterest,top_term=20,figsize=(5,5))
#enr_res.results.head(20)

#This box to run enrichment right away
ClusterOfInterest='MacrophagesTIMD4pos'
#sc.tl.rank_genes_groups(adata, 'fine_grain', method='wilcoxon', key_added = "wilcoxon")
gene_set_names = gseapy.get_library_name(organism='Human')
#print(gene_set_names)
glist = sc.get.rank_genes_groups_df(adata, group=ClusterOfInterest, 
                                    key='wilcoxon', log2fc_min=0.25, 
                                    pval_cutoff=0.05)['names'].squeeze().str.strip().tolist()
#print(len(glist))                                  
#GO_Biological_Process_2021,GO_Cellular_Component_2021,GO_Molecular_Function_2021
enr_res = gseapy.enrichr(gene_list=glist,
                     organism='Human',
                     gene_sets='GO_Biological_Process_2021',
                     cutoff = 0.5)
gseapy.barplot(enr_res.res2d,title=ClusterOfInterest,top_term=20,figsize=(5,5))
#enr_res.results.head(20)

#This box to run enrichment right away
ClusterOfInterest='MonocyteDerivedCells'
#sc.tl.rank_genes_groups(adata, 'fine_grain', method='wilcoxon', key_added = "wilcoxon")
gene_set_names = gseapy.get_library_name(organism='Human')
#print(gene_set_names)
glist = sc.get.rank_genes_groups_df(adata, group=ClusterOfInterest, 
                                    key='wilcoxon', log2fc_min=0.25, 
                                    pval_cutoff=0.05)['names'].squeeze().str.strip().tolist()
#print(len(glist))                                  
#GO_Biological_Process_2021,GO_Cellular_Component_2021,GO_Molecular_Function_2021
enr_res = gseapy.enrichr(gene_list=glist,
                     organism='Human',
                     gene_sets='GO_Biological_Process_2021',
                     cutoff = 0.5)
gseapy.barplot(enr_res.res2d,title=ClusterOfInterest,top_term=20,figsize=(5,5))
#enr_res.results.head(20)

#This box to run enrichment right away
ClusterOfInterest='Monocytes'
#sc.tl.rank_genes_groups(adata, 'fine_grain', method='wilcoxon', key_added = "wilcoxon")
gene_set_names = gseapy.get_library_name(organism='Human')
#print(gene_set_names)
glist = sc.get.rank_genes_groups_df(adata, group=ClusterOfInterest, 
                                    key='wilcoxon', log2fc_min=0.25, 
                                    pval_cutoff=0.05)['names'].squeeze().str.strip().tolist()
#print(len(glist))                                  
#GO_Biological_Process_2021,GO_Cellular_Component_2021,GO_Molecular_Function_2021
enr_res = gseapy.enrichr(gene_list=glist,
                     organism='Human',
                     gene_sets='GO_Biological_Process_2021',
                     cutoff = 0.5)
gseapy.barplot(enr_res.res2d,title=ClusterOfInterest,top_term=20,figsize=(5,5))
#enr_res.results.head(20)

#This box to run enrichment right away
ClusterOfInterest='MonocytesMPOpos'
#sc.tl.rank_genes_groups(adata, 'fine_grain', method='wilcoxon', key_added = "wilcoxon")
gene_set_names = gseapy.get_library_name(organism='Human')
#print(gene_set_names)
glist = sc.get.rank_genes_groups_df(adata, group=ClusterOfInterest, 
                                    key='wilcoxon', log2fc_min=0.25, 
                                    pval_cutoff=0.05)['names'].squeeze().str.strip().tolist()
#print(len(glist))                                  
#GO_Biological_Process_2021,GO_Cellular_Component_2021,GO_Molecular_Function_2021
enr_res = gseapy.enrichr(gene_list=glist,
                     organism='Human',
                     gene_sets='GO_Biological_Process_2021',
                     cutoff = 0.5)
gseapy.barplot(enr_res.res2d,title=ClusterOfInterest,top_term=20,figsize=(5,5))
#enr_res.results.head(20)
```

Example 89 (python):
```python
ranking genes
    finished: added to `.uns['wilcoxon']`
    'names', sorted np.recarray to be indexed by group ids
    'scores', sorted np.recarray to be indexed by group ids
    'logfoldchanges', sorted np.recarray to be indexed by group ids
    'pvals', sorted np.recarray to be indexed by group ids
    'pvals_adj', sorted np.recarray to be indexed by group ids (0:00:15)
```

---

## Variables¶

**URL:** http://127.0.0.1:9020/notebooks_html/Supplementary_materials/Macrophages/Mo-MP_no8_YSMP-regulon-markers.html

**Contents:**
- Variables¶
- Read in anndata object¶
- Plotting¶

**Examples:**

Example 1 (python):
```python
import scanpy as sc
import pandas as pd
import numpy as np
import anndata
import scipy
import matplotlib.pyplot as plt
import seaborn as sns

import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
-----
anndata             0.9.2
matplotlib          3.7.2
numpy               1.24.4
pandas              1.5.3
scanpy              1.9.3
scipy               1.10.1
seaborn             0.12.2
session_info        1.0.0
-----
```

Example 4 (python):
```python
PIL                         10.0.0
anyio                       NA
arrow                       1.2.3
asttokens                   NA
attr                        23.1.0
attrs                       23.1.0
babel                       2.12.1
backcall                    0.2.0
brotli                      1.0.9
certifi                     2023.11.17
cffi                        1.15.1
charset_normalizer          3.2.0
colorama                    0.4.6
comm                        0.1.4
cycler                      0.10.0
cython_runtime              NA
dateutil                    2.8.2
debugpy                     1.6.8
decorator                   5.1.1
defusedxml                  0.7.1
executing                   1.2.0
fastjsonschema              NA
fqdn                        NA
h5py                        3.9.0
idna                        3.4
igraph                      0.10.6
importlib_metadata          NA
importlib_resources         NA
ipykernel                   6.25.1
isoduration                 NA
jedi                        0.19.0
jinja2                      3.1.2
joblib                      1.3.2
json5                       NA
jsonpointer                 2.0
jsonschema                  4.19.0
jsonschema_specifications   NA
jupyter_events              0.7.0
jupyter_server              2.7.1
jupyterlab_server           2.24.0
kiwisolver                  1.4.5
leidenalg                   0.10.1
llvmlite                    0.40.1
markupsafe                  2.1.3
mpl_toolkits                NA
natsort                     8.4.0
nbformat                    5.9.2
numba                       0.57.1
overrides                   NA
packaging                   23.1
parso                       0.8.3
patsy                       0.5.3
pexpect                     4.8.0
pickleshare                 0.7.5
pkg_resources               NA
plotly                      5.16.1
prometheus_client           NA
prompt_toolkit              3.0.39
psutil                      5.9.5
ptyprocess                  0.7.0
pure_eval                   0.2.2
pydev_ipython               NA
pydevconsole                NA
pydevd                      2.9.5
pydevd_file_utils           NA
pydevd_plugins              NA
pydevd_tracing              NA
pygments                    2.16.1
pyparsing                   3.0.9
pythonjsonlogger            NA
pytz                        2023.3
referencing                 NA
requests                    2.31.0
rfc3339_validator           0.1.4
rfc3986_validator           0.1.1
rpds                        NA
ruamel                      NA
send2trash                  NA
six                         1.16.0
sklearn                     1.3.0
sniffio                     1.3.0
socks                       1.7.1
sphinxcontrib               NA
stack_data                  0.6.2
statsmodels                 0.14.0
texttable                   1.6.7
threadpoolctl               3.2.0
tornado                     6.3.3
traitlets                   5.9.0
typing_extensions           NA
uri_template                NA
urllib3                     2.0.4
wcwidth                     0.2.6
webcolors                   1.13
websocket                   1.6.2
yaml                        6.0.1
zipp                        NA
zmq                         25.1.1
zstandard                   0.19.0
```

Example 5 (unknown):
```unknown
-----
IPython             8.12.0
jupyter_client      8.3.0
jupyter_core        4.12.0
jupyterlab          4.0.5
notebook            7.0.2
-----
Python 3.8.17 (default, Jul  5 2023, 21:04:15) [GCC 11.2.0]
Linux-5.4.0-169-generic-x86_64-with-glibc2.17
-----
Session information updated at 2024-02-05 21:15
```

Example 6 (python):
```python
sc.settings.set_figure_params(dpi=120)
```

Example 7 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 8 (python):
```python
sc.settings.set_figure_params(dpi=80,dpi_save=300)
```

Example 9 (python):
```python
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk'
adata_path_dir = '/nfs/team205/heart/anndata_objects/Foetal/Immune/MoMP_Aug2023'
```

Example 10 (python):
```python
adata = sc.read_h5ad(f'{adata_path_dir}/MultiVI-post_foetal_Mono-MP_withContCov-CCCov_13Mar2023_feature-select-0003.h5ad')
adata
```

Example 11 (python):
```python
AnnData object with n_obs × n_vars = 7674 × 142283
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', '_indices', '_scvi_batch', '_scvi_labels', 'fine_grain_withcycling', 't', 'seg', 'edge', 't_sd', 'milestones', 'fine_grain_mod'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id', 'n_cells'
    uns: '_scvi_manager_uuid', '_scvi_uuid', 'adata_auc_gene', 'adata_auc_region', 'adata_gex', 'adata_gex_lognorm', 'adata_peak_tfidf', 'draw_graph', 'fine_grain_colors', 'fine_grain_mod_colors', 'fine_grain_withcycling_colors', 'graph', 'milestones_colors', 'neighbors', 'ppt', 'pseudotime_list', 'seg_colors', 'umap', 'week_colors'
    obsm: 'MultiVI_latent', 'X_R', 'X_draw_graph_fa', 'X_palantir', 'X_umap', 'X_umap13', '_scvi_extra_categorical_covs', '_scvi_extra_continuous_covs'
    obsp: 'connectivities', 'distances'
```

Example 12 (python):
```python
# get regulon score data, gene expression based
adata_auc = adata.uns['adata_auc_gene'].copy()
adata_auc.uns['draw_graph'] = adata.uns['draw_graph']
print(adata_auc.X.data[:5])
adata_auc
```

Example 13 (unknown):
```unknown
<memory at 0x151287d85380>
```

Example 14 (unknown):
```unknown
AnnData object with n_obs × n_vars = 1899 × 280
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', '_indices', '_scvi_batch', '_scvi_labels', 'fine_grain_withcycling', 't', 'seg', 'edge', 't_sd', 'milestones'
    uns: 'draw_graph'
    obsm: 'MultiVI_latent', 'X_R', 'X_draw_graph_fa', 'X_palantir', 'X_umap', 'X_umap13', '_scvi_extra_categorical_covs', '_scvi_extra_continuous_covs'
```

Example 15 (python):
```python
markers = ['FLI1_+','MEF2C_+']
markers = [x for x in markers if x in adata_auc.var_names]

plt.rcParams['figure.figsize'] = 1.7,3.5
sc.pl.draw_graph(adata_auc,color=markers,cmap='inferno',wspace=0.45,frameon=False,show=False)
plt.savefig(f"{figdir}/Force-directed-graph_MoMP_YSMP-marker-regulon-scores.pdf", bbox_inches="tight",dpi=300)
```

---

## Import modules¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/RNA/scnRNA_Integration-Annotation/Leukocytes/No3_Myeloid_post-scVI.html

**Contents:**
- Import modules¶
- Function¶
- Read in data¶
- Log-normalise¶
- Clustering, UMAP¶
  - Deciding resolution¶
- Marker gene expressions¶
- DEG analysis¶
- CellTypist prediction¶
- Annotation¶
- Recover row counts and save¶
- Celltypist, pan-foetal immune¶
- CellTypist prediction: EL-YS¶
- Check matching with previoius annotation¶

**Examples:**

Example 1 (python):
```python
import warnings
warnings.simplefilter(action='ignore', category=FutureWarning)

import numpy as np
import numpy.random as random
import pandas as pd
import scanpy as sc
import matplotlib.pyplot as plt
import anndata


import celltypist
from celltypist import models

import sctk
import os
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
-----
anndata             0.8.0
celltypist          0.1.9
matplotlib          3.5.2
numpy               1.21.5
pandas              1.3.5
scanpy              1.8.2
sctk                NA
session_info        1.0.0
-----
```

Example 4 (python):
```python
PIL                 9.1.1
asciitree           NA
backcall            0.2.0
beta_ufunc          NA
binom_ufunc         NA
brotli              NA
cached_property     1.5.1
cairo               1.20.0
certifi             2020.12.05
cffi                1.14.4
chardet             3.0.4
charset_normalizer  2.0.10
cloudpickle         1.6.0
colorama            0.4.4
cycler              0.10.0
cython_runtime      NA
cytoolz             0.11.2
dask                2022.01.0
dateutil            2.8.1
decorator           4.4.2
defusedxml          0.6.0
fasteners           0.17.3
fsspec              2022.01.0
google              NA
h5py                3.1.0
idna                2.10
igraph              0.10.2
importlib_metadata  1.7.0
ipykernel           5.4.2
ipython_genutils    0.2.0
ipywidgets          7.6.3
jedi                0.17.2
jinja2              3.1.2
joblib              1.0.0
kiwisolver          1.3.1
leidenalg           0.9.0
llvmlite            0.35.0
louvain             0.7.0
markupsafe          2.1.1
mpl_toolkits        NA
msgpack             1.0.3
natsort             7.1.0
nbinom_ufunc        NA
numba               0.52.0
numcodecs           0.9.1
numexpr             2.7.2
numpy_groupies      0.9.13
packaging           21.3
parso               0.7.1
pexpect             4.8.0
pickleshare         0.7.5
pkg_resources       NA
plotly              5.10.0
prompt_toolkit      3.0.10
psutil              5.9.0
ptyprocess          0.7.0
pyarrow             9.0.0
pygments            2.13.0
pyparsing           2.4.7
pytz                2020.5
requests            2.27.1
scipy               1.7.3
scvelo              0.2.4
seaborn             0.11.1
setuptools_scm      NA
sinfo               0.3.1
six                 1.14.0
sklearn             1.0.2
socks               1.7.1
sphinxcontrib       NA
statsmodels         0.12.1
storemagic          NA
tables              3.6.1
tblib               1.7.0
texttable           1.6.3
threadpoolctl       2.1.0
tlz                 0.11.2
toolz               0.11.2
tornado             6.1
traitlets           5.0.5
typing_extensions   NA
urllib3             1.26.9
wcwidth             0.2.5
yaml                5.4.1
zarr                2.11.3
zipp                NA
zmq                 21.0.1
zope                NA
```

Example 5 (unknown):
```unknown
-----
IPython             7.19.0
jupyter_client      6.1.11
jupyter_core        4.7.0
notebook            6.2.0
-----
Python 3.7.9 | packaged by conda-forge | (default, Dec  9 2020, 21:08:20) [GCC 9.3.0]
Linux-4.15.0-112-generic-x86_64-with-debian-bullseye-sid
-----
Session information updated at 2023-03-09 09:25
```

Example 6 (python):
```python
sc.settings.set_figure_params(dpi=120, facecolor='white', color_map = 'RdPu',)
```

Example 7 (python):
```python
#>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
# extract rank_genes_groups result 
#>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

import scanpy as sc
import pandas as pd

def rank_genes_groups_to_df(adata, cell):
    
    dic = adata.uns['rank_genes_groups'] 
    
    df = pd.DataFrame({
        'names' : dic['names'][cell],
        'scores' : dic['scores'][cell],
        'pvals' : dic['pvals'][cell],
        'pvals_adj' : dic['pvals_adj'][cell],
        'logfoldchanges' : dic['logfoldchanges'][cell]
    })
    
    return df
```

Example 8 (unknown):
```unknown
compartment = 'Myeloid'
```

Example 9 (python):
```python
adata = sc.read_h5ad(f'/home/jovyan/mount/gdrive/Colab/data/scVI/subsets/RNA_foetal_{compartment}_post-scVI_16Feb2023.h5ad')
adata
```

Example 10 (unknown):
```unknown
AnnData object with n_obs × n_vars = 18527 × 28267
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', '_scvi_labels', '2Jan_coarse_grain', '2Jan_mid_grain', '2Jan_fine_grain', 'leiden_scVI', 'FACSgate', 'sex', 'week', 'trimester', 'cell_type2'
    var: 'gene_name_scRNA-0', 'gene_id', 'n_cells'
    uns: '_scvi_manager_uuid', '_scvi_uuid'
    obsm: 'X_pca', 'X_scVI', 'X_umap', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 11 (python):
```python
# normalise and logtransform
# adata.raw = adata
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)
```

Example 12 (python):
```python
%%time

sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata, min_dist=0.2, spread=1.0, n_components=3)

resolutions = [0.2,0.3,0.4,0.5,0.6,] # 0.7,0.8,0.9,1.0
for res in resolutions:
    print(res)
    sc.tl.leiden(adata, resolution=res, key_added=f'leiden_scVI_{res}')
    sc.pl.umap(adata, color=[f'leiden_scVI_{res}'], size=5, components='all', wspace=0.5)
```

Example 13 (python):
```python
sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata, min_dist=0.2, spread=1.0, n_components=3)
sc.tl.leiden(adata, resolution=1, key_added='leiden_scVI')
```

Example 14 (python):
```python
sc.pl.umap(adata, color=['leiden_scVI'], size=5, wspace=0.1, legend_loc='on data', legend_fontsize=10,components='all')
```

Example 15 (python):
```python
sc.pl.umap(adata, color=['region','donor','cell_or_nuclei','kit_10x'], size=5, wspace=0.7)
```

Example 16 (python):
```python
sc.pl.umap(adata, color=['region'],groups='Pericardium', size=5, wspace=0.7,components='all')
```

Example 17 (python):
```python
sc.pl.umap(adata, color=['scrublet_score','cluster_scrublet_score','doublet_pval','doublet_bh_pval','pct_counts_mt','pct_counts_ribo'], size=2, wspace=0.2, ncols=6)
```

Example 18 (python):
```python
genes=['CCR2','CLEC10A','LYVE1','FOLR2','CX3CR1','TIMD4','MERTK','TREM2','COL1A1','HLA-DPA1']
genes = [x for x in genes if x in adata.var_names]
sc.pl.umap(adata, color=genes, size=20, wspace=0.2, ncols=6, components='all')
```

Example 19 (python):
```python
# Checking glabal markers
markers = {
    'vCM': ['MYH7', 'MYL2', 'FHL2'],
    'aCM': ['NPPA', 'MYL7', 'MYL4'],
    'FB': ['DCN', 'GCN', 'PDGFRA'],
    'EC': ['VWF', 'PECAM1', 'CDH5'],
    'Peri': ['RGS5', 'ABCC9', 'KCNJ8'],
    'SMC': ['MYH11', 'TAGLN', 'ACTA2'],
    'Meso': ['MSLN', 'WT1', 'BNC1'],
    'Neuro': ['PLP1', 'NRXN1', 'NRXN3'],
    'Adipo': ['GPAM', 'FASN', 'LEP'],
    'Myelo': ['CD14', 'C1QA', 'CD68','LYVE1','TIMD4'],
    'Lymph': ['CD8A', 'IL7R', 'CD40LG'],
    'B': ['CD19','SDC1','IGHG1','IGHA1'],
    'Mast': ['KIT', 'CPA3'],
}

for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in adata.var_names]

sc.tl.dendrogram(adata, groupby='leiden_scVI')
sc.pl.dotplot(adata,
              markers, 
              groupby='leiden_scVI',
              dendrogram=True,
              standard_scale="var",
              color_map="Reds",
              # swap_axes=True,
             )
```

Example 20 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: vCM, aCM, FB, etc.
```

Example 21 (python):
```python
# lymphocytes markers

markers={
    'B cell': ['CD19', 'CD79A', 'CD79B', 'IGHM', 'IGHG1', 'IGHA1'],
    'T cell': ['CD3E', 'CD3D', 'TRAC', 'CD4', 'CD8A', 'CD8B'],
    'naive': ['CCR7', 'TCF7', 'LEF1', 'SELL'],
    'act': ['IL2','CD69'],
    'CD4T_reg': ['FOXP3', 'CTLA4', 'IL2RA', 'IKZF2'],
    'cytotoxic': ['PRF1', 'GZMB', 'GZMH', 'NKG7'],
    'trans': ['GZMK', 'TOX'],
    'te': ['TIGIT', 'LAG3', 'PDCD1'],
    'em': ['IL7R', 'RUNX2'],
    'MAIT': ['SLC4A10', 'KLRB1', 'CCR6'],
    'NK': ['KLRF1', 'KLRD1', 'NCAM1', 'FCGR3A'],
    'ILC': ['KIT', 'IL18R1'],
    'cycling': ['MKI67', 'TOP2A', 'UBE2C'],
    'TFs': ['TBX21', 'GATA3', 'RORC', 'BCL6',],
}


for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in adata.var_names]

sc.pl.dotplot(adata,
              markers, 
              groupby='leiden_scVI',
              dendrogram=True,
              standard_scale="var",
              color_map="Reds",
              # swap_axes=True,
             )
```

Example 22 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: B cell, T cell, naive, etc.
```

Example 23 (python):
```python
markers={
    'Mono':['CCR2','CD14','FCGR3A','CX3CR1','FCGR1A','FCN1','TREM2'],
    'MP':['C1QA','C1QB','CD68','LYVE1','IGF1','FOLR2','DOCK4','MRC1','P2RY12','SALL1','C3','OLFML3','HTRA1'],
    'PS-R':['TIMD4','MERTK','AXL','SIRPA','CD300A','ITGAV','ITGB3','ITGB5'],
    'M2MP':['CD163','MRC1','CLEC10A'],
    'DC':['FLT3','FCER1A','CD1C','HLA-DPA1','HLA-DPB1','CCR7'],
    'DC1':['ID2','IRF8','BATF3','CLEC9A'],
    'DC2':['CLEC10A','CD1C','FCER1A',],
    'Neutro':['MYB', 'MPO', 'CEACAM8','CXCR1','CXCR2'],
    'collagen':['COL1A1','COL1A2','COL3A1','NLRP3'],
    'FOS-JUN':['FOS','JUN'],
    'Cycling':['MKI67','TOP2A'],

}


for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in adata.var_names]

sc.pl.dotplot(adata,
              markers, 
              groupby='leiden_scVI',
              dendrogram=True,
              standard_scale="var",
              color_map="Reds",
              # swap_axes=True,
             )
```

Example 24 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: Mono, MP, PS-R, etc.
```

Example 25 (python):
```python
adata.X.data[:10]
```

Example 26 (python):
```python
sc.tl.rank_genes_groups(adata, 'leiden_scVI', method='t-test')

DEG_dict = {}
for cluster in adata.obs['leiden_scVI'].cat.categories:
    df=rank_genes_groups_to_df(adata, cluster)
    # select genes: significant and up-regulated ones
    df=df[(df['pvals_adj']<0.05)&(df['logfoldchanges']>0)]
    # sort based on the logFC
    df.sort_values('scores',ascending=False,inplace=True)
    DEG_dict[cluster]=list(df[:10]['names'])
    del df

sc.pl.dotplot(adata,
              DEG_dict, 
              groupby='leiden_scVI',
              dendrogram=True,
              standard_scale="var",
              color_map="Reds",
              swap_axes=False,
             )
```

Example 27 (python):
```python
# load model and predict
panfetal_model = models.Model.load('/nfs/team205/kk18/Analysis/celltypist/models/celltypist_model.Pan_Fetal_Human.pkl')
prediction=celltypist.annotate(adata, model = panfetal_model, majority_voting = True)
```

Example 28 (unknown):
```unknown
🔬 Input data has 18527 cells and 28267 genes
🔗 Matching reference genes in the model
🧬 6636 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
👀 Detected a neighborhood graph in the input object, will run over-clustering on the basis of it
⛓️ Over-clustering input data with resolution set to 10
🗳️ Majority voting the predictions
✅ Majority voting done!
```

Example 29 (unknown):
```unknown
celltypist.dotplot(prediction, use_as_reference = 'leiden_scVI', use_as_prediction = 'predicted_labels')
```

Example 30 (python):
```python
sc.pl.umap(adata, color=['leiden_scVI'], size=5, wspace=0.1, legend_loc='on data', components='all', legend_fontsize=10)
```

Example 31 (python):
```python
set(adata.obs['2Jan_fine_grain'])
```

Example 32 (unknown):
```unknown
{'B1Cells',
 'BCells',
 'Cardiofibromyocytes',
 'CoronaryCapillaryEndothelialCells',
 'CoronaryPericytes',
 'CoronaryVesselAdventitialFibroblasts',
 'DC1',
 'DC2',
 'DCsMature',
 'DCsPlasmocytoids',
 'EarlyGreatVesselFLRT2',
 'EndocardialCells',
 'GVAdFibOCA2',
 'GVAdFibOther',
 'InnateLymphoidCells',
 'LymphaticEndothelialCells',
 'MacrophagesCX3CR1pos',
 'MacrophagesInflammasome',
 'MacrophagesLYVE1pos',
 'MacrophagesTIMD4pos',
 'MacrophagesTREM2pos',
 'MastCells',
 'Monocytes',
 'MonocytesMPOpos',
 'MyocIntFibROBO2',
 'MyocIntFibSLIT3',
 'Myofibroblasts',
 'NaturalKillerCells',
 'SchwannCells',
 'TCellsCD8pos',
 'Tregs',
 'ValvePENK',
 nan}
```

Example 33 (python):
```python
sc.pl.umap(adata, color=['2Jan_fine_grain'], groups='MacrophagesTREM2pos' ,size=5, wspace=0.1, legend_loc='on data', components='all', legend_fontsize=10)
```

Example 34 (python):
```python
sc.pl.umap(adata, color=['TIMD4','LYVE1','FOLR2','CX3CR1','TREM2','FOS','MKI67','ATF3','TLR4'], size=10, wspace=0.1, 
           legend_loc='on data', components='all', legend_fontsize=10,ncols=3)
```

Example 35 (python):
```python
pd.crosstab(adata.obs['leiden_scVI'],adata.obs['region'])
```

Example 36 (python):
```python
adata.obs['cell_type3']=adata.obs['leiden_scVI'].astype('str').copy()
adata.obs.replace({'cell_type3':{
    '0':'MP_CX3CR1+', 
    '1':'MP_LYVE1+', 
    '2':'MP_LYVE1+', 
    '3':'MP_LYVE1+', 
    '4':'MP_LYVE1+_cycling', 
    '5':'MP_LYVE1+ATF3+', 
    '6':'MP_LYVE1+', 
    '7':'MP_LYVE1+',   
    '8':'MP_LYVE1+',  
    '9':'MP_LYVE1+', 
    '10':'Mono', 
    '11':'MP_LYVE1+', 
    '12':'MP_LYVE1+', 
    '13':'MP_LYVE1+ATF3+', 
    '14':'MP_LYVE1+', 
    '15':'Mono-derived', 
    '16':'DC1', 
    '17':'Doublet',   
    '18':'MP_LYVE1+_cycling', 
    '19':'MP_TIMD4+',
    '20':'Mono_MPO+', 
}}, inplace=True)


adata.obs['cell_type3']=adata.obs['cell_type3'].astype('category')
adata.obs['cell_type3'].cat.reorder_categories([
    'Mono', 
    'Mono_MPO+', 
    'Mono-derived', 
    'MP_CX3CR1+',
    # 'MP_TREM2+', 
    'MP_TIMD4+', 
    'MP_LYVE1+',
    # 'MP_LYVE1+MHCIIhi', 
    'MP_LYVE1+ATF3+',
    'MP_LYVE1+_cycling',
    'DC1',
    'Doublet',  
], inplace=True)

adata.obs['cell_type3'].value_counts()
```

Example 37 (javascript):
```javascript
MP_LYVE1+            9748
MP_LYVE1+ATF3+       1844
MP_LYVE1+_cycling    1650
MP_CX3CR1+           1648
Mono                  900
Mono-derived          661
DC1                   609
Doublet               583
MP_TIMD4+             470
Mono_MPO+             414
Name: cell_type3, dtype: int64
```

Example 38 (python):
```python
markers={
  'Mono':['CCR2','CD14','FCGR3A','CX3CR1','FCGR1A','FCN1','TREM2'],
    'MP':['C1QA','C1QB','CD68','LYVE1','IGF1','FOLR2','DOCK4'],
    'PS-R':['TIMD4','MERTK','AXL','SIRPA','CD300A','ITGAV','ITGB3','ITGB5'],
    'M2MP':['CD163','MRC1','CLEC10A'],
    'DC':['FLT3','FCER1A','CD1C','HLA-DPA1','HLA-DPB1','CCR7'],
    'DC1':['ID2','IRF8','BATF3','CLEC9A'],
    'DC2':['CLEC10A','CD1C','FCER1A',],
    'Neutro':['MYB', 'MPO', 'CEACAM8','CXCR1','CXCR2'],
    'collagen':['COL1A1','COL1A2','COL3A1'],
    'FOS-JUN':['FOS','JUN'],
    'Inflammasome':['NLRP1','NLRP3','PYCARD','CASP1'],
    'Cycling':['MKI67','TOP2A'],
}


for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in adata.var_names]

sc.pl.dotplot(adata,
              markers, 
              groupby='cell_type3',
              dendrogram=False,
              standard_scale="var",
              color_map="Reds",
              # swap_axes=True,
             )
```

Example 39 (python):
```python
markers={
    'Mono':['CCR2','FCN1','MPO'],
    'Mono-derived':['CLEC10A','CD1C'],
    'MP':['C1QA','C1QB','CD68','MRC1','CD163','LYVE1','FOLR2'],
    'CX3CR1+':['CX3CR1','FCGR1A','FCGR3A','C3','P2RY12','SALL1','C3','OLFML3','HTRA1','TREM2',],
    'PtdSer-R':['TIMD4','MERTK','AXL','ITGAV'],
    'ATF3':['ATF3'],
    'ATF3':['ATF3'],
    'DC general':['FLT3','HLA-DPA1','HLA-DPB1','CIITA'],
    'DC1':['ID2','BATF3','CLEC9A'],
    'Cycling':['MKI67','TOP2A'],
}

for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in adata.var_names]

sc.pl.dotplot(adata[adata.obs['cell_type3']!='Doublet'],
              markers, 
              groupby='cell_type3',
              dendrogram=False,
              standard_scale="var",
              color_map="Reds",
              # swap_axes=True,
             )
```

Example 40 (python):
```python
sc.settings.set_figure_params(dpi=120,  color_map = 'RdPu',)
sc.pl.umap(adata, color=['cell_type3'], size=5, legend_fontsize=10, frameon=False, components='all', wspace=0.5)
```

Example 41 (python):
```python
sc.tl.rank_genes_groups(adata, 'cell_type3', method='t-test')

DEG_dict = {}
for cluster in adata.obs['cell_type3'].cat.categories:
    df=rank_genes_groups_to_df(adata, cluster)
    # select genes: significant and up-regulated ones
    df=df[(df['pvals_adj']<0.05)&(df['logfoldchanges']>0)]
    # sort based on the logFC
    df.sort_values('scores',ascending=False,inplace=True)
    DEG_dict[cluster]=list(df[:10]['names'])
    del df

sc.pl.dotplot(adata,
              DEG_dict, 
              groupby='cell_type3',
              dendrogram=False,
              standard_scale="var",
              color_map="Reds",
              swap_axes=False,
             )
```

Example 42 (python):
```python
adata_raw=sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/Immune/RNA_foetal_raw_immune_16Feb2023.h5ad')
print(adata_raw.shape)

adata_raw=anndata.AnnData(X=adata_raw[adata.obs_names,:].X,
                      obs=adata.obs, # from annotated or subsetted data
                      var=adata_raw.var,
                      obsm=adata.obsm, # from annotated or subsetted data
                      obsp=adata.obsp # from annotated or subsetted data
                               )
print(adata_raw.shape)
```

Example 43 (unknown):
```unknown
(41881, 36601)
(18527, 36601)
```

Example 44 (python):
```python
adata_raw.write('/nfs/team205/heart/anndata_objects/Foetal/Immune/RNA_foetal_raw_Myeloid_16Feb2023.h5ad')
adata_raw
```

Example 45 (unknown):
```unknown
AnnData object with n_obs × n_vars = 18527 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', '_scvi_labels', '2Jan_coarse_grain', '2Jan_mid_grain', '2Jan_fine_grain', 'leiden_scVI', 'FACSgate', 'sex', 'week', 'trimester', 'cell_type2', 'cell_type3'
    var: 'gene_name_scRNA-0', 'gene_id'
    obsm: 'X_pca', 'X_scVI', 'X_umap', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 46 (python):
```python
sc.pl.umap(adata_raw, color=['cell_type3'], size=5, legend_fontsize=10, frameon=False, components='all', wspace=0.5)
```

Example 47 (python):
```python
adata_raw = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/Immune/RNA_foetal_raw_Myeloid_16Feb2023.h5ad')
adata_raw
```

Example 48 (unknown):
```unknown
AnnData object with n_obs × n_vars = 18527 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', '_scvi_labels', '2Jan_coarse_grain', '2Jan_mid_grain', '2Jan_fine_grain', 'leiden_scVI', 'FACSgate', 'sex', 'week', 'trimester', 'cell_type2', 'cell_type3'
    var: 'gene_name_scRNA-0', 'gene_id'
    obsm: 'X_pca', 'X_scVI', 'X_umap', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 49 (python):
```python
# save metadata
adata_raw.obs.to_csv('/nfs/team205/heart/anndata_objects/Foetal/Immune/RNA_foetal_raw_Myeloid_16Feb2023.obs.csv')
```

Example 50 (python):
```python
adata = adata_raw.copy()
# normalise and logtransform
# adata.raw = adata
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)
```

Example 51 (python):
```python
markers={
    'Mono':['CCR2','FCN1','MPO'],
    # 'Mono-derived':['CLEC10A','CD1C'],
    'MP':['C1QA','C1QB','CD68','MRC1','CD163'],
    'CX3CR1+':['CX3CR1','C3','TREM2','OLFMl3','P2RY12'],
    'PtdSer-R':['TIMD4','MERTK','ITGAV'],
    'LYVE1+':['LYVE1','FOLR2'],
    'ATF3':['ATF3'],
    'Cycling':['MKI67'],
}

for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in adata.var_names]

sc.pl.dotplot(adata[adata.obs['cell_type3'].isin(['Doublet','DC1'])==False],
              markers, 
              groupby='cell_type3',
              dendrogram=False,
              standard_scale="var",
              color_map="Reds",
              # swap_axes=True,
             )
```

Example 52 (python):
```python
sc.pl.umap(adata, 
           color=['AREG','CCL24'], size=10, wspace=0.1, 
           legend_loc='on data', components='all', legend_fontsize=10,ncols=3)
```

Example 53 (python):
```python
sc.pl.umap(adata, 
           color=['GATA6'], size=10, wspace=0.1, 
           legend_loc='on data', components='all', legend_fontsize=10,ncols=3)
```

Example 54 (python):
```python
sc.pl.umap(adata, 
           color=['region'], groups='Pericardium', size=10, wspace=0.1, 
           legend_loc='on data', components='all', legend_fontsize=10,ncols=3)
```

Example 55 (python):
```python
sc.pl.umap(adata[adata.obs['cell_type3']=='Mono-derived'], 
           color=['CCR2','CLEC10A','CD1C'], size=10, wspace=0.1, 
           legend_loc='on data', components='all', legend_fontsize=10,ncols=3)
```

Example 56 (python):
```python
# load model and predict
panfetal_model = models.Model.load('/nfs/team205/kk18/Analysis/celltypist/models/celltypist_model.Pan_Fetal_Human.pkl')
prediction=celltypist.annotate(adata, model = panfetal_model, majority_voting = False)
celltypist.dotplot(prediction, use_as_reference = 'cell_type3', use_as_prediction = 'predicted_labels')
```

Example 57 (unknown):
```unknown
🔬 Input data has 18527 cells and 28267 genes
🔗 Matching reference genes in the model
🧬 6636 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
```

Example 58 (python):
```python
# load model
elys_model = models.Model.load('/nfs/team205/heart/anndata_objects/Haniffa_YolkSac/celltypist_EL-YS_myeloid.pkl')
prediction=celltypist.annotate(adata, model = elys_model, majority_voting = False)
celltypist.dotplot(prediction, use_as_reference = 'cell_type3', use_as_prediction = 'predicted_labels')
```

Example 59 (unknown):
```unknown
🔬 Input data has 18527 cells and 28267 genes
🔗 Matching reference genes in the model
🧬 16762 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
```

Example 60 (python):
```python
adata = prediction.to_adata(insert_labels = True, insert_conf = True)
```

Example 61 (python):
```python
sc.pl.umap(adata, color=['predicted_labels'], size=5, legend_fontsize=10, frameon=False, wspace=1,components=['1,2','1,3'])
```

Example 62 (python):
```python
sc.pl.umap(adata, color=['conf_score'], size=5, legend_fontsize=10, frameon=False, wspace=0.5, components='all')
```

Example 63 (python):
```python
pd.crosstab(adata.obs['cell_type3'],adata.obs['2Jan_fine_grain']).T
```

Example 64 (python):
```python
import seaborn as sns
plt.rcParams['figure.figsize'] = 12,5
df=pd.crosstab(adata.obs['cell_type3'],adata.obs['2Jan_fine_grain'],normalize='columns')
sns.heatmap(df)
```

Example 65 (unknown):
```unknown
<AxesSubplot:xlabel='2Jan_fine_grain', ylabel='cell_type3'>
```

Example 66 (python):
```python
import seaborn as sns
plt.rcParams['figure.figsize'] = 13,5
ad=adata[adata.obs['2Jan_fine_grain']!='nan']
df=pd.crosstab(ad.obs['cell_type3'],ad.obs['2Jan_fine_grain'],normalize='index')
sns.heatmap(df)
```

Example 67 (unknown):
```unknown
<AxesSubplot:xlabel='2Jan_fine_grain', ylabel='cell_type3'>
```

---

## Import modules¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig5_Trisomy21/no03-1_combine-and-subsample_euploid-trisomy21_data.html

**Contents:**
- Import modules¶
- Read in anndata¶
- Euploid¶
- Trisomy21¶
- Select samples¶
- Euploid¶
- Trisomy 21¶
- Concatenate¶
- Subsample cells¶

**Examples:**

Example 1 (python):
```python
import warnings
warnings.simplefilter("ignore", UserWarning)

import anndata
import matplotlib.pyplot as plt
import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 
import seaborn as sns

import numpy as np
import numpy.random as random
import pandas as pd
import scanpy as sc
```

Example 2 (python):
```python
sc.settings.verbosity = 3             # verbosity: errors (0), warnings (1), info (2), hints (3)
sc.settings.set_figure_params(dpi=80, facecolor='white', color_map = 'RdPu',)
```

Example 3 (python):
```python
# import own function
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 4 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk/Aug2024'
table_dir = '/nfs/team205/heart/fetal_ms_figures/tables/Aug2024'
```

Example 5 (python):
```python
import os
os.getcwd()
```

Example 6 (unknown):
```unknown
'/nfs/team205/kk18/notebooks/Foetal/Trisomy21/RNA'
```

Example 7 (python):
```python
# euploid
eup = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/Feb28ObjectRaw_finegrain_updated.h5ad')
eup
```

Example 8 (unknown):
```unknown
AnnData object with n_obs × n_vars = 297473 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score'
    var: 'gene_name_scRNA-0', 'gene_id'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'age_colors', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'cycling_colors', 'donor_colors', 'fine_grain_colors', 'heart_or_greatvessels_colors', 'hvg', 'kit_10x_colors', 'mid_grain_colors', 'modality_colors', 'neighbors', 'phase_colors', 'region_colors', 'sex_colors', 'trimester_colors', 'umap', 'week_colors'
    obsm: 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 9 (python):
```python
# subset multiome
eup = eup[eup.obs['kit_10x']=='Multiome-v1']
eup.shape
```

Example 10 (unknown):
```unknown
(211145, 36601)
```

Example 11 (python):
```python
pd.crosstab(eup.obs['age'],eup.obs['sex'])
```

Example 12 (python):
```python
pd.crosstab(eup.obs['age'],eup.obs['region'])
```

Example 13 (python):
```python
pd.crosstab(eup.obs['donor'],eup.obs['region'])
```

Example 14 (python):
```python
pd.crosstab(eup.obs['donor'],eup.obs['age'])
```

Example 15 (python):
```python
t21 = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/trisomy21/T21Hearts_cleaned_July2024.h5ad')
t21
```

Example 16 (unknown):
```unknown
AnnData object with n_obs × n_vars = 110693 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'batch_key', 'leiden_scVI_0.1', 'leiden_scVI_0.2', 'leiden_scVI_0.3', 'leiden_scVI_0.5', 'leiden_scVI_0.7', 'leiden_scVI_1', 'celltypist_coarse', 'conf_score_coarse', 'cells_to_remove', 'week', 'trimester', 'sex', 'celltypist_fine', 'conf_score_fine'
    var: 'feature_type', 'ambient_expression-0', 'ambient_expression-1', 'ambient_expression-10', 'ambient_expression-11', 'ambient_expression-12', 'ambient_expression-13', 'ambient_expression-14', 'ambient_expression-2', 'ambient_expression-3', 'ambient_expression-4', 'ambient_expression-5', 'ambient_expression-6', 'ambient_expression-7', 'ambient_expression-8', 'ambient_expression-9', 'mt', 'ribo', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'gene_id'
    uns: 'age_colors', 'cells_to_remove_colors', 'celltypist_coarse_colors', 'celltypist_fine_colors', 'donor_colors', 'leiden', 'leiden_scVI_0.1_colors', 'leiden_scVI_0.2_colors', 'leiden_scVI_0.3_colors', 'leiden_scVI_0.5_colors', 'leiden_scVI_0.7_colors', 'leiden_scVI_1_colors', 'neighbors', 'region_colors', 'sangerID_colors', 'umap'
    obsm: 'X_scVI', 'X_umap', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 17 (python):
```python
pd.crosstab(t21.obs['age'],t21.obs['sex'])
```

Example 18 (python):
```python
pd.crosstab(t21.obs['age'],t21.obs['region'])
```

Example 19 (python):
```python
display(pd.crosstab(t21.obs['donor'],t21.obs['region']))
display(pd.crosstab(t21.obs['donor'],t21.obs['sex']))
display(pd.crosstab(t21.obs['donor'],t21.obs['age']))
```

Example 20 (unknown):
```unknown
age_sel = ['12W0D','13W0D','14W0D']
region_sel = ['apex','atria','atrial septum','heart without node',
                 'left ventricle',
                 'node',
                 'outflow tract','right ventricle',
                 'right ventricle and papillaries',
                 'whole sample']

mask1 = eup.obs['age'].isin(age_sel)
mask2 = eup.obs['region'].isin(region_sel)
eup = eup[mask1&mask2]
eup
```

Example 21 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 55210 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score'
    var: 'gene_name_scRNA-0', 'gene_id'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'age_colors', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'cycling_colors', 'donor_colors', 'fine_grain_colors', 'heart_or_greatvessels_colors', 'hvg', 'kit_10x_colors', 'modality_colors', 'neighbors', 'phase_colors', 'region_colors', 'sex_colors', 'trimester_colors', 'umap', 'week_colors'
    obsm: 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 22 (python):
```python
display(pd.crosstab(eup.obs['donor'],eup.obs['region']))
display(pd.crosstab(eup.obs['donor'],eup.obs['sex']))
display(pd.crosstab(eup.obs['donor'],eup.obs['age']))
```

Example 23 (unknown):
```unknown
mask = t21.obs['region'].isin(['Pericardium'])==False
t21 = t21[mask]
t21
```

Example 24 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 92284 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'batch_key', 'leiden_scVI_0.1', 'leiden_scVI_0.2', 'leiden_scVI_0.3', 'leiden_scVI_0.5', 'leiden_scVI_0.7', 'leiden_scVI_1', 'celltypist_coarse', 'conf_score_coarse', 'cells_to_remove', 'week', 'trimester', 'sex', 'celltypist_fine', 'conf_score_fine'
    var: 'feature_type', 'ambient_expression-0', 'ambient_expression-1', 'ambient_expression-10', 'ambient_expression-11', 'ambient_expression-12', 'ambient_expression-13', 'ambient_expression-14', 'ambient_expression-2', 'ambient_expression-3', 'ambient_expression-4', 'ambient_expression-5', 'ambient_expression-6', 'ambient_expression-7', 'ambient_expression-8', 'ambient_expression-9', 'mt', 'ribo', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'gene_id'
    uns: 'age_colors', 'cells_to_remove_colors', 'celltypist_coarse_colors', 'celltypist_fine_colors', 'donor_colors', 'leiden', 'leiden_scVI_0.1_colors', 'leiden_scVI_0.2_colors', 'leiden_scVI_0.3_colors', 'leiden_scVI_0.5_colors', 'leiden_scVI_0.7_colors', 'leiden_scVI_1_colors', 'neighbors', 'region_colors', 'sangerID_colors', 'umap'
    obsm: 'X_scVI', 'X_umap', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 25 (python):
```python
display(pd.crosstab(t21.obs['donor'],t21.obs['region']))
display(pd.crosstab(t21.obs['donor'],t21.obs['sex']))
display(pd.crosstab(t21.obs['donor'],t21.obs['age']))
```

Example 26 (python):
```python
eup.obs['euploid_or_t21'] = 'euploid'
t21.obs['euploid_or_t21'] = 'tri21'
t21.obs.replace({'cell_or_nuclei':{'Nuclei':'nuclei'}},inplace=True)

adata = eup.concatenate(t21, 
            join='inner', 
            batch_key=None, 
            index_unique=None)
adata
```

Example 27 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/anndata/_core/anndata.py:1763: FutureWarning: The AnnData.concatenate method is deprecated in favour of the anndata.concat function. Please use anndata.concat instead.

See the tutorial for concat at: https://anndata.readthedocs.io/en/latest/concatenation.html
  warnings.warn(
```

Example 28 (unknown):
```unknown
AnnData object with n_obs × n_vars = 147494 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score', 'euploid_or_t21', 'donor_type', 'region_finest', 'flushed', 'scrublet_leiden', 'cluster_scrublet_score', 'leiden_scVI_0.1', 'leiden_scVI_0.2', 'leiden_scVI_0.3', 'leiden_scVI_0.5', 'leiden_scVI_0.7', 'leiden_scVI_1', 'celltypist_coarse', 'conf_score_coarse', 'cells_to_remove', 'celltypist_fine', 'conf_score_fine'
    var: 'gene_id', 'gene_name_scRNA-0-0', 'feature_type-1', 'ambient_expression-0-1', 'ambient_expression-1-1', 'ambient_expression-10-1', 'ambient_expression-11-1', 'ambient_expression-12-1', 'ambient_expression-13-1', 'ambient_expression-14-1', 'ambient_expression-2-1', 'ambient_expression-3-1', 'ambient_expression-4-1', 'ambient_expression-5-1', 'ambient_expression-6-1', 'ambient_expression-7-1', 'ambient_expression-8-1', 'ambient_expression-9-1', 'mt-1', 'ribo-1', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1'
    obsm: 'X_scVI', 'X_umap', 'latent_gene_encoding'
```

Example 29 (python):
```python
# merge coarse grain labels
adata.obs['coarse_grain_merged'] = adata.obs['coarse_grain'].astype('str').copy()
t21_obsnames = adata.obs_names[adata.obs['euploid_or_t21']=='tri21']
adata.obs.loc[t21_obsnames,'coarse_grain_merged'] = adata.obs.loc[t21_obsnames,'celltypist_coarse']
set(adata.obs['coarse_grain_merged'])
```

Example 30 (unknown):
```unknown
{'Cardiomyocytes',
 'Endothelium',
 'Epicardium',
 'Leukocytes',
 'Mesenchymal',
 'Neural'}
```

Example 31 (python):
```python
# merge fine grain labels
adata.obs['fine_grain_merged'] = adata.obs['fine_grain'].astype('str').copy()
t21_obsnames = adata.obs_names[adata.obs['euploid_or_t21']=='tri21']
adata.obs.loc[t21_obsnames,'fine_grain_merged'] = adata.obs.loc[t21_obsnames,'celltypist_fine']
print(f"number of NaN in the merged label: {sum(adata.obs['fine_grain_merged'].isna())}")
with pd.option_context('display.max_rows', None):
    display(pd.crosstab(adata.obs['fine_grain_merged'],adata.obs['euploid_or_t21']))
```

Example 32 (unknown):
```unknown
number of NaN in the merged label: 0
```

Example 33 (python):
```python
# save concatenated object
adata.write('/nfs/team205/heart/anndata_objects/Foetal/trisomy21/Euploid_T21Hearts_Aug2024_sel.h5ad')
```

Example 34 (python):
```python
# read in cell IDs which passed ATAC QC: trisomy21
tri21_atac_cells = utils.txt2list("/nfs/team205/heart/anndata_objects/Foetal/trisomy21/ArchR/project_output/project_cellIDs_removed-low-cellnumer-celltypes.txt")
print(len(tri21_atac_cells))
print(tri21_atac_cells[:5])

# read in cell IDs which passed ATAC QC: euploid
eup_atac_cells = utils.txt2list('/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output_peak-revised-Feb2024/project_cellIDs.txt')
print(len(eup_atac_cells))
print(eup_atac_cells[:5])

# conbine
atac_cells = eup_atac_cells + tri21_atac_cells
print(len(atac_cells))
```

Example 35 (unknown):
```unknown
89416
['BHF_F_Hea13188320_BHF_F_Hea13187622_GGGTCACTCATAGCCG-1', 'BHF_F_Hea13188320_BHF_F_Hea13187622_TAATCACCATTGGGAG-1', 'BHF_F_Hea13188320_BHF_F_Hea13187622_CGAGCTGGTCTCACTG-1', 'BHF_F_Hea13188320_BHF_F_Hea13187622_AAGGTATAGGCTACTG-1', 'BHF_F_Hea13188320_BHF_F_Hea13187622_GCGCGATTCATAATCG-1']
167022
['BHF_F_Hea11064670_BHF_F_Hea11031823_CAGCCTAAGTCTTGAA-1', 'BHF_F_Hea11064670_BHF_F_Hea11031823_GGTGTTGTCAGGCCTA-1', 'BHF_F_Hea11064670_BHF_F_Hea11031823_CGTGTGTCAAGCCACT-1', 'BHF_F_Hea11064670_BHF_F_Hea11031823_AGCTTTAAGCTAAAGG-1', 'BHF_F_Hea11064670_BHF_F_Hea11031823_GTTTCCTCAGGTCCTG-1']
256438
```

Example 36 (python):
```python
shared_obsnames = list(set(adata.obs_names).intersection(atac_cells))
print(len(shared_obsnames))
```

Example 37 (python):
```python
print(adata.shape)
adata_sub = adata[shared_obsnames]
print(adata_sub.shape)
```

Example 38 (unknown):
```unknown
(147494, 36601)
(122174, 36601)
```

Example 39 (python):
```python
with pd.option_context('display.max_rows', None):
    display(pd.crosstab(adata_sub.obs['fine_grain_merged'],adata_sub.obs['euploid_or_t21']))
```

Example 40 (python):
```python
with pd.option_context('display.max_rows', None):
    display(pd.crosstab(adata_sub.obs['fine_grain_merged'],adata_sub.obs['donor']))
```

Example 41 (python):
```python
# select cell states which has more than 30 cells in both euploid and trisomy21
ctab = pd.crosstab(adata_sub.obs['fine_grain_merged'],adata_sub.obs['euploid_or_t21'])
mask = (ctab>=30).sum(axis=1)==2
celltype_sel = ctab.index[mask]

# for the celltypes which passed above, subset nuclei per donor to be the 3rd largest number in the list of nuclei number per donor
selected_obsnames = []
for celltype in celltype_sel:
    ad = adata_sub[adata_sub.obs['fine_grain_merged']==celltype]
    n_subsample = ad.obs['donor'].value_counts()[2] # get 3rd largest number
    ad = utils.sctk_subsample(ad, fraction=1, groupby='donor', max_n=n_subsample, random_state=0)
    selected_obsnames = selected_obsnames+list(ad.obs_names)
    del ad
adata_sub = adata_sub[selected_obsnames]
adata_sub
```

Example 42 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 83477 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score', 'euploid_or_t21', 'donor_type', 'region_finest', 'flushed', 'scrublet_leiden', 'cluster_scrublet_score', 'leiden_scVI_0.1', 'leiden_scVI_0.2', 'leiden_scVI_0.3', 'leiden_scVI_0.5', 'leiden_scVI_0.7', 'leiden_scVI_1', 'celltypist_coarse', 'conf_score_coarse', 'cells_to_remove', 'celltypist_fine', 'conf_score_fine', 'coarse_grain_merged', 'fine_grain_merged'
    var: 'gene_id', 'gene_name_scRNA-0-0', 'feature_type-1', 'ambient_expression-0-1', 'ambient_expression-1-1', 'ambient_expression-10-1', 'ambient_expression-11-1', 'ambient_expression-12-1', 'ambient_expression-13-1', 'ambient_expression-14-1', 'ambient_expression-2-1', 'ambient_expression-3-1', 'ambient_expression-4-1', 'ambient_expression-5-1', 'ambient_expression-6-1', 'ambient_expression-7-1', 'ambient_expression-8-1', 'ambient_expression-9-1', 'mt-1', 'ribo-1', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1'
    obsm: 'X_scVI', 'X_umap', 'latent_gene_encoding'
```

Example 43 (python):
```python
with pd.option_context('display.max_rows', None):
    display(pd.crosstab(adata_sub.obs['fine_grain_merged'],adata_sub.obs['euploid_or_t21']))

with pd.option_context('display.max_rows', None):
    display(pd.crosstab(adata_sub.obs['fine_grain_merged'],adata_sub.obs['donor']))
```

Example 44 (python):
```python
pd.crosstab(adata_sub.obs['donor'],adata_sub.obs['euploid_or_t21'])
```

Example 45 (python):
```python
adata_sub.X.data[:10]
```

Example 46 (unknown):
```unknown
array([1., 1., 2., 1., 2., 2., 1., 1., 1., 1.], dtype=float32)
```

Example 47 (python):
```python
# save
adata_sub.write('/nfs/team205/heart/anndata_objects/Foetal/trisomy21/Euploid_T21Hearts_Aug2024_sel_subsampled-per-cellstate-donor.h5ad')
```

Example 48 (python):
```python
# save obs for later use
adata_sub.obs.to_csv('/nfs/team205/heart/anndata_objects/Foetal/trisomy21/Euploid_T21Hearts_Aug2024_sel_subsampled-per-cellstate-donor.obs.csv')
```

---

## Variables¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/VisiumHD/no8_collect_fine-grained-annotations.html

**Contents:**
- Variables¶
- Read in data¶
- Add fine-grained annotation¶
- Save¶
- Gene expression¶

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os
```

Example 2 (python):
```python
# import session_info
# session_info.show()
```

Example 3 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 4 (python):
```python
adata_dir = '/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects'
sample_id = 'HEA_FOET14880396'
```

Example 5 (unknown):
```unknown
os.getcwd()
```

Example 6 (unknown):
```unknown
'/nfs/team205/kk18/notebooks/Foetal/VisiumHD/re-seq_mpp0p3'
```

Example 7 (python):
```python
adata= sc.read_h5ad(f'{adata_dir}/{sample_id}_b2c_cells_filtered_lognorm.h5ad')
adata
```

Example 8 (unknown):
```unknown
AnnData object with n_obs × n_vars = 129803 × 18047
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'hvg', 'log1p', 'neighbors', 'pca', 'spatial', 'umap'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 9 (python):
```python
adata.X.data[:5]
```

Example 10 (unknown):
```unknown
array([1.27187559, 1.95641568, 1.75023687, 1.96760567, 2.7810601 ])
```

Example 11 (python):
```python
sc.pl.embedding(adata,
                    basis=f"X_umap",
                    color='celltypist_mid_fin',
                # groups='unassigned',
                   ncols=4
                   )
```

Example 12 (python):
```python
subset_adata_dir = '/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/subsets'
fine_grain_colname = 'fine_grain_4Jan2025'
```

Example 13 (python):
```python
!ls {subset_adata_dir}
```

Example 14 (unknown):
```unknown
HEA_FOET14880396_AtrialCardiomyocytes_b2c_cells_filtered_lognorm.h5ad
HEA_FOET14880396_BloodVesselEndothelialCells_b2c_cells_filtered_lognorm.h5ad
HEA_FOET14880396_EndocardialCells_b2c_cells_filtered_lognorm.h5ad
HEA_FOET14880396_EpicardialCells_b2c_cells_filtered_lognorm.h5ad
HEA_FOET14880396_Fibroblasts_b2c_cells_filtered_lognorm.h5ad
HEA_FOET14880396_Glia_b2c_cells_filtered_lognorm.h5ad
HEA_FOET14880396_LymphaticEndothelialCells_b2c_cells_filtered_lognorm.h5ad
HEA_FOET14880396_LymphoidCells_b2c_cells_filtered_lognorm.h5ad
HEA_FOET14880396_MuralCells_b2c_cells_filtered_lognorm.h5ad
HEA_FOET14880396_MyeloidCells_b2c_cells_filtered_lognorm.h5ad
HEA_FOET14880396_Neurons_b2c_cells_filtered_lognorm.h5ad
HEA_FOET14880396_PericardialCells_b2c_cells_filtered_lognorm.h5ad
HEA_FOET14880396_VentricularCardiomyocytes_b2c_cells_filtered_lognorm.h5ad
```

Example 15 (python):
```python
adata.obs[fine_grain_colname] =  adata.obs['celltypist_mid_fin'].astype('str').copy()
```

Example 16 (python):
```python
file_list = os.listdir(subset_adata_dir)
celltypes_to_add = [x.split('_')[2] for x in file_list]
celltypes_to_add
```

Example 17 (unknown):
```unknown
['PericardialCells',
 'LymphaticEndothelialCells',
 'AtrialCardiomyocytes',
 'MuralCells',
 'BloodVesselEndothelialCells',
 'MyeloidCells',
 'EpicardialCells',
 'Fibroblasts',
 'LymphoidCells',
 'VentricularCardiomyocytes',
 'Glia',
 'Neurons',
 'EndocardialCells']
```

Example 18 (python):
```python
for celltype in celltypes_to_add:
    print(celltype)
    if celltype=='LymphaticEndothelialCells':
        print('skip since the label of mid-grain and fine-grain are same')
        continue
    ad = sc.read_h5ad(f'{subset_adata_dir}/{sample_id}_{celltype}_b2c_cells_filtered_lognorm.h5ad',backed='r')
    print(ad.shape)
    # add cell type labels
    adata.obs.loc[ad.obs_names,fine_grain_colname] = ad.obs['fine_grain_manual']
adata.obs[fine_grain_colname].value_counts()
```

Example 19 (unknown):
```unknown
PericardialCells
(3061, 15212)
LymphaticEndothelialCells
skip since the label of mid-grain and fine-grain are same
AtrialCardiomyocytes
(13904, 17650)
MuralCells
(10785, 17693)
BloodVesselEndothelialCells
(1765, 15377)
MyeloidCells
(526, 13502)
EpicardialCells
(603, 13050)
Fibroblasts
(20318, 17926)
LymphoidCells
(87, 10288)
VentricularCardiomyocytes
(61075, 17994)
Glia
(588, 13981)
Neurons
(589, 14494)
EndocardialCells
(1526, 14451)
```

Example 20 (unknown):
```unknown
fine_grain_4Jan2025
unassigned                                            35539
VentricularCardiomyocytesCompact                      32970
Fibroblasts                                            7849
VentricularCardiomyocytesTrabeculated                  6349
VentricularConductionSystemProximal                    5902
ValveInterstitialCells                                 5186
AtrialCardiomyocytesRight                              5060
GreatVesselSmoothMuscleCells                           4843
AtrialCardiomyocytesLeft                               3107
PericardialCellsFibrous                                2231
DuctusArteriosusSmoothMuscleCells                      2145
VentricularCardiomyocytes_DoubletsWithErythrocytes     1723
FibroblastsCOL2A1pos                                   1648
AtrioventricularNodePacemakerCells                     1602
GreatVesselSmoothMuscleCellsTFPI2pos                   1519
VentricularConductionSystemDistal                      1342
LymphaticEndothelialCells                              1025
CoronaryCapillaryEndothelialCells                       848
CoronaryPericytes                                       753
PericardialCellsIntermediate                            730
SinoatrialNodePacemakerCellsHead                        636
ValveEndothelialCells                                   585
EndocardialCushionCells                                 531
SinoatrialNodePacemakerCellsHorn                        501
MesothelialEpicardialCells                              491
CoronarySmoothMuscleCells                               424
EndocardialCells                                        399
SinoatrialNodePacemakerCellsTail                        373
SchwannCells                                            354
GreatVesselVenousEndothelialCells                       313
GreatVesselArterialEndothelialCells                     308
MacrophagesLYVE1pos                                     299
GreatVesselSmoothMuscleCellsPTGER4pos                   285
ChromaffinCells                                         265
Fibroblasts_DoubletsWithCMs                             255
CoronaryArterialEndothelialCells                        219
SchwannCellPrecursors                                   163
ParasympatheticNeurons                                  153
EpicardiumDerivedCells                                  112
SympatheticNeurons                                      110
PericardialCells_DoubletsWithCMs                        100
MacrophagesCX3CR1pos                                     90
CoronaryVenousEndothelialCells                           77
SchwannCells_and_Neurons                                 71
InnateLymphoidCells                                      69
NeuronPrecursors                                         61
Monocytes                                                47
MonocyteDerivedCells                                     40
Erythrocytes                                             37
Megakaryocytes                                           24
VentricularCardiomyocytes_Doublets                       22
BCells                                                   18
Name: count, dtype: int64
```

Example 21 (python):
```python
sc.pl.embedding(adata,basis='X_umap',color=[fine_grain_colname],wspace=0.2,cmap='RdPu')
```

Example 22 (python):
```python
adata.write(f'{adata_dir}/{sample_id}_b2c_cells_filtered_lognorm.h5ad')
f'{adata_dir}/{sample_id}_b2c_cells_filtered_lognorm.h5ad'
```

Example 23 (unknown):
```unknown
'/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/HEA_FOET14880396_b2c_cells_filtered_lognorm.h5ad'
```

Example 24 (unknown):
```unknown
AnnData object with n_obs × n_vars = 129803 × 18047
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'fine_grain_4Jan2025'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'hvg', 'log1p', 'neighbors', 'pca', 'spatial', 'umap', 'fine_grain_4Jan2025_colors'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 25 (python):
```python
adata= sc.read_h5ad(f'{adata_dir}/{sample_id}_b2c_cells_filtered_lognorm.h5ad')
adata
```

Example 26 (python):
```python
cells = [
    'SinoatrialNodePacemakerCellsHorn',
    'SinoatrialNodePacemakerCellsHead',
    'SinoatrialNodePacemakerCellsTail',
    'AtrioventricularNodePacemakerCells',
    # 'VentricularConductionSystem'
    
    # 'MacrophagesCX3CR1pos',
    # 'ParasympatheticNeurons'
]

celltype_key = 'fine_grain_8Dec2024'
adata.uns[f"{celltype_key}_colors"][list(adata.obs[celltype_key].cat.categories).index('SinoatrialNodePacemakerCellsHorn')]='magenta'
adata.uns[f"{celltype_key}_colors"][list(adata.obs[celltype_key].cat.categories).index('SinoatrialNodePacemakerCellsHead')]='deepskyblue'
adata.uns[f"{celltype_key}_colors"][list(adata.obs[celltype_key].cat.categories).index('SinoatrialNodePacemakerCellsTail',)]='darkcyan'
adata.uns[f"{celltype_key}_colors"][list(adata.obs[celltype_key].cat.categories).index('AtrioventricularNodePacemakerCells')]='orange'
# adata.uns[f"{celltype_key}_colors"][list(adata.obs[celltype_key].cat.categories).index('VentricularConductionSystem')]='mediumpurple'

sc.pl.spatial(adata, color=celltype_key, groups=cells,
              img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer",
              size=10,legend_fontsize=30,spot_size=10,
                  frameon=False,cmap='viridis') # save=f'_{c}_b2c',
```

Example 27 (python):
```python
cells = [
  'MacrophagesCX3CR1pos',
  'MacrophagesLYVE1pos',
    'MacrophagesTREM2pos',
]

celltype_key = 'fine_grain_8Dec2024'
'''
adata.uns[f"{celltype_key}_colors"][list(adata.obs[celltype_key].cat.categories).index('SinoatrialNodePacemakerCellsHorn')]='magenta'
adata.uns[f"{celltype_key}_colors"][list(adata.obs[celltype_key].cat.categories).index('SinoatrialNodePacemakerCellsHead')]='deepskyblue'
adata.uns[f"{celltype_key}_colors"][list(adata.obs[celltype_key].cat.categories).index('SinoatrialNodePacemakerCellsTail',)]='darkcyan'
adata.uns[f"{celltype_key}_colors"][list(adata.obs[celltype_key].cat.categories).index('AtrioventricularNodePacemakerCells')]='orange'
# adata.uns[f"{celltype_key}_colors"][list(adata.obs[celltype_key].cat.categories).index('VentricularConductionSystem')]='mediumpurple'
'''

sc.pl.spatial(adata, color=celltype_key, groups=cells,
              img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer",
              size=10,legend_fontsize=30,spot_size=10,
                  frameon=False,cmap='viridis') # save=f'_{c}_b2c',
```

Example 28 (python):
```python
cells = [
  'ParasympatheticNeurons',
  'SympatheticNeurons',
]

celltype_key = 'fine_grain_8Dec2024'
'''
adata.uns[f"{celltype_key}_colors"][list(adata.obs[celltype_key].cat.categories).index('SinoatrialNodePacemakerCellsHorn')]='magenta'
adata.uns[f"{celltype_key}_colors"][list(adata.obs[celltype_key].cat.categories).index('SinoatrialNodePacemakerCellsHead')]='deepskyblue'
adata.uns[f"{celltype_key}_colors"][list(adata.obs[celltype_key].cat.categories).index('SinoatrialNodePacemakerCellsTail',)]='darkcyan'
adata.uns[f"{celltype_key}_colors"][list(adata.obs[celltype_key].cat.categories).index('AtrioventricularNodePacemakerCells')]='orange'
# adata.uns[f"{celltype_key}_colors"][list(adata.obs[celltype_key].cat.categories).index('VentricularConductionSystem')]='mediumpurple'
'''

sc.pl.spatial(adata, color=celltype_key, groups=cells,
              img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer",
              size=10,legend_fontsize=30,spot_size=10,
                  frameon=False,cmap='viridis') # save=f'_{c}_b2c',
```

Example 29 (python):
```python
cells = [
     'SinoatrialNodePacemakerCellsHorn',
    'SinoatrialNodePacemakerCellsHead',
    'SinoatrialNodePacemakerCellsTail',
    'AtrioventricularNodePacemakerCells',
    # 'VentricularConductionSystem'
]
celltype_key = 'fine_grain_8Dec2024'

mask = ((adata.obs['array_row'] >= 1700) & 
        (adata.obs['array_row'] <= 2300) & 
        (adata.obs['array_col'] >= 600) & 
        (adata.obs['array_col'] <= 1200)
       )
bdata = adata[mask]
n_celltype = len(bdata.obs[celltype_key].cat.categories)
bdata.uns[celltype_key] = list(np.repeat('#ffff00',n_celltype))

# update colors
for c in cells:
    # get color in adata
    ind = list(adata.obs[celltype_key].cat.categories).index(c)
    color = adata.uns[f'{celltype_key}_colors'][ind]
    # update color in bdata
    ind = list(bdata.obs[celltype_key].cat.categories).index(c)
    bdata.uns[f'{celltype_key}_colors'][ind] = color

sc.pl.spatial(bdata, color=celltype_key, groups=cells,
              img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer",
              size=2,legend_fontsize=30,# spot_size=10,
                  frameon=False,) # save=f'_{c}_b2c',
```

Example 30 (python):
```python
cells = [
     'MacrophagesCX3CR1pos',
  'MacrophagesLYVE1pos',
]
celltype_key = 'fine_grain_8Dec2024'

mask = ((adata.obs['array_row'] >= 1700) & 
        (adata.obs['array_row'] <= 2300) & 
        (adata.obs['array_col'] >= 600) & 
        (adata.obs['array_col'] <= 1200)
       )
bdata = adata[mask]
n_celltype = len(bdata.obs[celltype_key].cat.categories)
bdata.uns[celltype_key] = list(np.repeat('#ffff00',n_celltype))

# update colors
for c in cells:
    # get color in adata
    ind = list(adata.obs[celltype_key].cat.categories).index(c)
    color = adata.uns[f'{celltype_key}_colors'][ind]
    # update color in bdata
    ind = list(bdata.obs[celltype_key].cat.categories).index(c)
    bdata.uns[f'{celltype_key}_colors'][ind] = color

sc.pl.spatial(bdata, color=celltype_key, groups=cells,
              img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer",
              size=2,legend_fontsize=30,# spot_size=10,
                  frameon=False,) # save=f'_{c}_b2c',
```

Example 31 (python):
```python
cells = [
    'ParasympatheticNeurons',
  # 'SympatheticNeurons',
]
celltype_key = 'fine_grain_8Dec2024'

mask = ((adata.obs['array_row'] >= 1700) & 
        (adata.obs['array_row'] <= 2300) & 
        (adata.obs['array_col'] >= 600) & 
        (adata.obs['array_col'] <= 1200)
       )
bdata = adata[mask]
n_celltype = len(bdata.obs[celltype_key].cat.categories)
bdata.uns[celltype_key] = list(np.repeat('#ffff00',n_celltype))

# update colors
for c in cells:
    # get color in adata
    ind = list(adata.obs[celltype_key].cat.categories).index(c)
    color = adata.uns[f'{celltype_key}_colors'][ind]
    # update color in bdata
    ind = list(bdata.obs[celltype_key].cat.categories).index(c)
    bdata.uns[f'{celltype_key}_colors'][ind] = color

sc.pl.spatial(bdata, color=celltype_key, groups=cells,
              img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer",
              size=2,legend_fontsize=30,# spot_size=10,
                  frameon=False,) # save=f'_{c}_b2c',
```

Example 32 (python):
```python
cells = [
    'ParasympatheticNeurons',
  'SchwannCellPrecursors',
    'SchwannCells',
   'SchwannCells_and_Neurons'
]

sc.pl.spatial(bdata, color=celltype_key, groups=cells,
              img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer",
              size=2,legend_fontsize=30,# spot_size=10,
                  frameon=False,) # save=f'_{c}_b2c',
```

Example 33 (python):
```python
cells = [
    'ParasympatheticNeurons',
  'SchwannCellPrecursors',
    'SchwannCells',
   'SchwannCells_and_Neurons'
]

celltype_key = 'fine_grain_8Dec2024'
mask = ((adata.obs['array_row'] >= 1700) & 
        (adata.obs['array_row'] <= 2300) & 
        (adata.obs['array_col'] >= 600) & 
        (adata.obs['array_col'] <= 1200)
       )
bdata = adata[mask]

sc.pl.spatial(bdata, color=celltype_key, groups=cells,
              img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer",
              size=2,legend_fontsize=30,# spot_size=10,
                  frameon=False,) # save=f'_{c}_b2c',
```

Example 34 (python):
```python
cells = [
   'ParasympatheticNeurons',
  'SchwannCellPrecursors',
    'SchwannCells',
   'SchwannCells_and_Neurons'
]

celltype_key = 'fine_grain_8Dec2024'
'''
adata.uns[f"{celltype_key}_colors"][list(adata.obs[celltype_key].cat.categories).index('SinoatrialNodePacemakerCellsHorn')]='magenta'
adata.uns[f"{celltype_key}_colors"][list(adata.obs[celltype_key].cat.categories).index('SinoatrialNodePacemakerCellsHead')]='deepskyblue'
adata.uns[f"{celltype_key}_colors"][list(adata.obs[celltype_key].cat.categories).index('SinoatrialNodePacemakerCellsTail',)]='darkcyan'
adata.uns[f"{celltype_key}_colors"][list(adata.obs[celltype_key].cat.categories).index('AtrioventricularNodePacemakerCells')]='orange'
# adata.uns[f"{celltype_key}_colors"][list(adata.obs[celltype_key].cat.categories).index('VentricularConductionSystem')]='mediumpurple'
'''

sc.pl.spatial(adata, color=celltype_key, groups=cells,
              img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer",
              size=30,legend_fontsize=30,spot_size=10,
                  frameon=False,cmap='viridis') # save=f'_{c}_b2c',
```

Example 35 (python):
```python
genes = ['SEMA3A', 'SLIT2', 'SLIT3', 'EFNB2', 'CXCL12', 'FGF10', 'VEGFC', 'NELL1']
genes = [x for x in genes if x in adata.var_names]
genes
```

Example 36 (python):
```python
sc.pl.spatial(adata, color=genes,
              img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer",
              size=30,legend_fontsize=0,spot_size=5,ncols=5,wspace=0.15,
                  frameon=False,cmap='viridis')
```

Example 37 (python):
```python
mask = ((adata.obs['array_row'] >= 1700) & 
        (adata.obs['array_row'] <= 2300) & 
        (adata.obs['array_col'] >= 600) & 
        (adata.obs['array_col'] <= 1200)
       )
bdata = adata[mask]

sc.pl.spatial(bdata, color=genes,
              img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer",
              size=30,legend_fontsize=0,spot_size=5,ncols=5,wspace=0.15,
                  frameon=False,cmap='viridis') # save=f'_{c}_b2c',
```

Example 38 (python):
```python
mask = ((adata.obs['array_row'] >= 1500) & 
        (adata.obs['array_row'] <= 2500) & 
        (adata.obs['array_col'] >= 400) & 
        (adata.obs['array_col'] <= 1200)
       )
bdata = adata[mask]

sc.pl.spatial(bdata, color=genes,
              img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer",
              size=30,legend_fontsize=0,spot_size=5,ncols=5,wspace=0.15,
                  frameon=False,cmap='viridis') # save=f'_{c}_b2c',
```

Example 39 (python):
```python
cells = [
     'MacrophagesCX3CR1pos',
    'SinoatrialNodePacemakerCellsHorn',
  'SinoatrialNodePacemakerCellsHead',
    'ParasympatheticNeurons',
]
celltype_key = 'fine_grain_8Dec2024'

mask = ((adata.obs['array_row'] >= 2000) & 
        (adata.obs['array_row'] <= 2200) & 
        (adata.obs['array_col'] >= 600) & 
        (adata.obs['array_col'] <= 800)
       )
bdata = adata[mask]
bdata.uns[f"{celltype_key}_colors"][list(bdata.obs[celltype_key].cat.categories).index('SinoatrialNodePacemakerCellsHorn')]='magenta'
bdata.uns[f"{celltype_key}_colors"][list(bdata.obs[celltype_key].cat.categories).index('SinoatrialNodePacemakerCellsHead')]='deepskyblue'
bdata.uns[f"{celltype_key}_colors"][list(bdata.obs[celltype_key].cat.categories).index('MacrophagesCX3CR1pos',)]='red'
bdata.uns[f"{celltype_key}_colors"][list(bdata.obs[celltype_key].cat.categories).index('ParasympatheticNeurons')]='purple'

sc.pl.spatial(bdata, color=celltype_key, groups=cells,
              img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer",
              size=4,legend_fontsize=30,spot_size=10,alpha_img=0.3,
                  frameon=False,) # save=f'_{c}_b2c',
```

Example 40 (python):
```python
cells = [
  'MacrophagesCX3CR1pos',
  'MacrophagesLYVE1pos',
    'MacrophagesTREM2pos',
]

celltype_key = 'fine_grain_8Dec2024'
'''
adata.uns[f"{celltype_key}_colors"][list(adata.obs[celltype_key].cat.categories).index('SinoatrialNodePacemakerCellsHorn')]='magenta'
adata.uns[f"{celltype_key}_colors"][list(adata.obs[celltype_key].cat.categories).index('SinoatrialNodePacemakerCellsHead')]='deepskyblue'
adata.uns[f"{celltype_key}_colors"][list(adata.obs[celltype_key].cat.categories).index('SinoatrialNodePacemakerCellsTail',)]='darkcyan'
adata.uns[f"{celltype_key}_colors"][list(adata.obs[celltype_key].cat.categories).index('AtrioventricularNodePacemakerCells')]='orange'
# adata.uns[f"{celltype_key}_colors"][list(adata.obs[celltype_key].cat.categories).index('VentricularConductionSystem')]='mediumpurple'
'''

sc.pl.spatial(adata, color=celltype_key, groups=cells,
              img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer",
              size=10,legend_fontsize=30,spot_size=10,
                  frameon=False,cmap='viridis') # save=f'_{c}_b2c',
```

Example 41 (python):
```python
genes = ['SHOX2', 'TBX3', 
         'GJA1', # CX43
         'GJA5', # CX40
         'RSPO3'
        ]
genes = [x for x in genes if x in adata.var_names]
genes
```

Example 42 (python):
```python
mask = ((adata.obs['array_row'] >= 1700) & 
        (adata.obs['array_row'] <= 2300) & 
        (adata.obs['array_col'] >= 600) & 
        (adata.obs['array_col'] <= 1200)
       )
bdata = adata[mask]

sc.pl.spatial(bdata, color=genes,
              img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer",
              size=30,legend_fontsize=0,spot_size=5,ncols=5,wspace=0.15,
                  frameon=False,cmap='viridis') # save=f'_{c}_b2c',
```

Example 43 (python):
```python
mask = ((adata.obs['array_row'] >= 1500) & 
        (adata.obs['array_row'] <= 2500) & 
        (adata.obs['array_col'] >= 400) & 
        (adata.obs['array_col'] <= 1200)
       )
bdata = adata[mask]

sc.pl.spatial(bdata, color=genes,
              img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer",
              size=30,legend_fontsize=0,spot_size=5,ncols=5,wspace=0.15,
                  frameon=False,cmap='viridis') # save=f'_{c}_b2c',
```

---

## Read in original adata¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig5_Trisomy21/no05_euploid-trisomy21_post-scVI_incpect-all-conditions.html

**Contents:**
- Read in original adata¶
- Add scVI latent space¶
- Find neighbors, UMAP/Clustering¶

**Examples:**

Example 1 (python):
```python
import scanpy as sc
import pandas as pd
import numpy as np
import anndata
import scipy

import os

import matplotlib.pyplot as plt
import seaborn as sns

import warnings
warnings.simplefilter("ignore", UserWarning)
```

Example 2 (python):
```python
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm
```

Example 3 (python):
```python
import session_info
session_info.show()
```

Example 4 (python):
```python
-----
anndata             0.9.2
matplotlib          3.7.2
numpy               1.22.4
pandas              1.5.3
scanpy              1.9.3
scipy               1.10.1
seaborn             0.12.2
session_info        1.0.0
-----
```

Example 5 (python):
```python
PIL                         10.0.0
anyio                       NA
arrow                       1.2.3
asciitree                   NA
asttokens                   NA
attr                        23.1.0
attrs                       23.1.0
babel                       2.12.1
backcall                    0.2.0
brotli                      1.0.9
certifi                     2023.11.17
cffi                        1.15.1
charset_normalizer          3.2.0
cloudpickle                 3.0.0
colorama                    0.4.6
comm                        0.1.4
cycler                      0.10.0
cython_runtime              NA
dask                        2023.5.0
dateutil                    2.8.2
debugpy                     1.6.8
decorator                   5.1.1
defusedxml                  0.7.1
executing                   1.2.0
fasteners                   0.19
fastjsonschema              NA
fqdn                        NA
h5py                        3.9.0
idna                        3.4
igraph                      0.10.6
importlib_metadata          NA
importlib_resources         NA
ipykernel                   6.25.1
isoduration                 NA
jedi                        0.19.0
jinja2                      3.1.2
joblib                      1.3.2
json5                       NA
jsonpointer                 2.0
jsonschema                  4.19.0
jsonschema_specifications   NA
jupyter_events              0.7.0
jupyter_server              2.7.1
jupyterlab_server           2.24.0
kiwisolver                  1.4.5
leidenalg                   0.10.1
llvmlite                    0.38.1
markupsafe                  2.1.3
mpl_toolkits                NA
natsort                     8.4.0
nbformat                    5.9.2
numba                       0.55.2
numcodecs                   0.12.1
opt_einsum                  v3.3.0
overrides                   NA
packaging                   23.1
parso                       0.8.3
patsy                       0.5.3
pexpect                     4.8.0
pickleshare                 0.7.5
pkg_resources               NA
plotly                      5.16.1
prometheus_client           NA
prompt_toolkit              3.0.39
psutil                      5.9.5
ptyprocess                  0.7.0
pure_eval                   0.2.2
pyarrow                     15.0.2
pydev_ipython               NA
pydevconsole                NA
pydevd                      2.9.5
pydevd_file_utils           NA
pydevd_plugins              NA
pydevd_tracing              NA
pygments                    2.16.1
pyparsing                   3.0.9
pythonjsonlogger            NA
pytz                        2023.3
referencing                 NA
requests                    2.31.0
rfc3339_validator           0.1.4
rfc3986_validator           0.1.1
rpds                        NA
ruamel                      NA
send2trash                  NA
six                         1.16.0
sklearn                     1.3.0
sniffio                     1.3.0
socks                       1.7.1
sphinxcontrib               NA
stack_data                  0.6.2
statsmodels                 0.14.0
texttable                   1.6.7
threadpoolctl               3.2.0
tlz                         0.12.1
toolz                       0.12.1
torch                       1.13.1+cu117
tornado                     6.3.3
tqdm                        4.66.1
traitlets                   5.9.0
typing_extensions           NA
uri_template                NA
urllib3                     2.0.4
wcwidth                     0.2.6
webcolors                   1.13
websocket                   1.6.2
yaml                        6.0.1
zarr                        2.16.1
zipp                        NA
zmq                         25.1.1
zstandard                   0.19.0
```

Example 6 (unknown):
```unknown
-----
IPython             8.12.0
jupyter_client      8.3.0
jupyter_core        4.12.0
jupyterlab          4.0.5
notebook            7.0.2
-----
Python 3.8.17 (default, Jul  5 2023, 21:04:15) [GCC 11.2.0]
Linux-5.15.0-116-generic-x86_64-with-glibc2.17
-----
Session information updated at 2024-08-20 13:52
```

Example 7 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 8 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 9 (unknown):
```unknown
!ls /nfs/team205/heart/anndata_objects/Foetal/trisomy21
```

Example 10 (unknown):
```unknown
archive_ArchR
ArchR
AtlasT21IntegratedRaw.h5ad
AtlasT21IntegratedRaw.obs.csv
AtlasT21ToFIntegratedRaw.h5ad
CellTypist_labelling
Euploid_T21Hearts_Aug2024_sel.h5ad
Euploid_T21Hearts_Aug2024_sel_subsampled-per-cellstate-donor.h5ad
Euploid_T21Hearts_Aug2024_sel_subsampled-per-cellstate-donor.obs.csv
Euploid_T21Hearts_July2024_sel_subsampled-donor-10000.h5ad
Euploid_T21Hearts_July2024_sel_subsampled-donor-10000.obs.csv
Euploid_T21Hearts_July2024_sel_subsampled-donor-10000_pseudobulked.h5ad
Euploid_T21Hearts_July2024_sel_subsampled-donor-10000_pseudobulked-perdonor-sum.h5ad
Euploid_T21Hearts_July2024_sel_subsampled-donor-10000_pseudobulked-sum.h5ad
T21-Healthy_RNA_raw_forArchR.h5ad
T21-Healthy_RNA_raw_forArchR.rds
T21Hearts_cleaned_July2024.h5ad
T21Hearts_cleaned_July2024.obs.csv
T21HeartsExtracardiacRemovedIntegratedProcessed.h5ad
T21HeartsExtracardiacRemovedRaw.h5ad
T21HeartsExtracardiacRemovedRaw.obs.csv
T21Hearts_July2024.h5ad
```

Example 11 (python):
```python
# atlas
adata = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/trisomy21/Euploid_T21Hearts_Aug2024_sel_subsampled-per-cellstate-donor.h5ad')
adata
```

Example 12 (unknown):
```unknown
AnnData object with n_obs × n_vars = 83477 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score', 'euploid_or_t21', 'donor_type', 'region_finest', 'flushed', 'scrublet_leiden', 'cluster_scrublet_score', 'leiden_scVI_0.1', 'leiden_scVI_0.2', 'leiden_scVI_0.3', 'leiden_scVI_0.5', 'leiden_scVI_0.7', 'leiden_scVI_1', 'celltypist_coarse', 'conf_score_coarse', 'cells_to_remove', 'celltypist_fine', 'conf_score_fine', 'coarse_grain_merged', 'fine_grain_merged'
    var: 'gene_id', 'gene_name_scRNA-0-0', 'feature_type-1', 'ambient_expression-0-1', 'ambient_expression-1-1', 'ambient_expression-10-1', 'ambient_expression-11-1', 'ambient_expression-12-1', 'ambient_expression-13-1', 'ambient_expression-14-1', 'ambient_expression-2-1', 'ambient_expression-3-1', 'ambient_expression-4-1', 'ambient_expression-5-1', 'ambient_expression-6-1', 'ambient_expression-7-1', 'ambient_expression-8-1', 'ambient_expression-9-1', 'mt-1', 'ribo-1', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1'
    obsm: 'X_scVI', 'X_umap', 'latent_gene_encoding'
```

Example 13 (unknown):
```unknown
latent_file_name_list = os.listdir('/nfs/team205/heart/anndata_objects/Foetal/scVI/latent_variables')
latent_file_name_list = [x.split('.h5ad')[0] for x in latent_file_name_list if (x!='archive')&('subsampled-per-cellstate-donor_' in x)]
latent_file_name_list
```

Example 14 (unknown):
```unknown
['Euploid_T21Hearts_Aug2024_sel_subsampled-per-cellstate-donor_correcting-region_n-layers-2',
 'Euploid_T21Hearts_Aug2024_sel_subsampled-per-cellstate-donor_correcting-donor-region_n-layers-3',
 'Euploid_T21Hearts_Aug2024_sel_subsampled-per-cellstate-donor_correcting-donor-region_n-layers-2',
 'Euploid_T21Hearts_Aug2024_sel_subsampled-per-cellstate-donor_correcting-donor_n-layers-2',
 'Euploid_T21Hearts_Aug2024_sel_subsampled-per-cellstate-donor_correcting-donor_n-layers-1',
 'Euploid_T21Hearts_Aug2024_sel_subsampled-per-cellstate-donor_correcting-donor-region_n-layers-1',
 'Euploid_T21Hearts_Aug2024_sel_subsampled-per-cellstate-donor_correcting-region_n-layers-1',
 'Euploid_T21Hearts_Aug2024_sel_subsampled-per-cellstate-donor_correcting-region_n-layers-3',
 'Euploid_T21Hearts_Aug2024_sel_subsampled-per-cellstate-donor_correcting-donor_n-layers-3']
```

Example 15 (python):
```python
for latent_file_name in latent_file_name_list:
    latent_space_condition = latent_file_name.split('subsampled-per-cellstate-donor_')[1]
    latent = sc.read_h5ad(f'/nfs/team205/heart/anndata_objects/Foetal/scVI/latent_variables/{latent_file_name}.h5ad')
    # add latent space
    adata.obsm[f'scVI_latent_{latent_space_condition}']=latent.X.copy()
```

Example 16 (unknown):
```unknown
AnnData object with n_obs × n_vars = 83477 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score', 'euploid_or_t21', 'donor_type', 'region_finest', 'flushed', 'scrublet_leiden', 'cluster_scrublet_score', 'leiden_scVI_0.1', 'leiden_scVI_0.2', 'leiden_scVI_0.3', 'leiden_scVI_0.5', 'leiden_scVI_0.7', 'leiden_scVI_1', 'celltypist_coarse', 'conf_score_coarse', 'cells_to_remove', 'celltypist_fine', 'conf_score_fine', 'coarse_grain_merged', 'fine_grain_merged'
    var: 'gene_id', 'gene_name_scRNA-0-0', 'feature_type-1', 'ambient_expression-0-1', 'ambient_expression-1-1', 'ambient_expression-10-1', 'ambient_expression-11-1', 'ambient_expression-12-1', 'ambient_expression-13-1', 'ambient_expression-14-1', 'ambient_expression-2-1', 'ambient_expression-3-1', 'ambient_expression-4-1', 'ambient_expression-5-1', 'ambient_expression-6-1', 'ambient_expression-7-1', 'ambient_expression-8-1', 'ambient_expression-9-1', 'mt-1', 'ribo-1', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1'
    obsm: 'X_scVI', 'X_umap', 'latent_gene_encoding', 'scVI_latent_correcting-region_n-layers-2', 'scVI_latent_correcting-donor-region_n-layers-3', 'scVI_latent_correcting-donor-region_n-layers-2', 'scVI_latent_correcting-donor_n-layers-2', 'scVI_latent_correcting-donor_n-layers-1', 'scVI_latent_correcting-donor-region_n-layers-1', 'scVI_latent_correcting-region_n-layers-1', 'scVI_latent_correcting-region_n-layers-3', 'scVI_latent_correcting-donor_n-layers-3'
```

Example 17 (unknown):
```unknown
latent_file_name_list.sort()
latent_file_name_list
```

Example 18 (unknown):
```unknown
['Euploid_T21Hearts_Aug2024_sel_subsampled-per-cellstate-donor_correcting-donor-region_n-layers-1',
 'Euploid_T21Hearts_Aug2024_sel_subsampled-per-cellstate-donor_correcting-donor-region_n-layers-2',
 'Euploid_T21Hearts_Aug2024_sel_subsampled-per-cellstate-donor_correcting-donor-region_n-layers-3',
 'Euploid_T21Hearts_Aug2024_sel_subsampled-per-cellstate-donor_correcting-donor_n-layers-1',
 'Euploid_T21Hearts_Aug2024_sel_subsampled-per-cellstate-donor_correcting-donor_n-layers-2',
 'Euploid_T21Hearts_Aug2024_sel_subsampled-per-cellstate-donor_correcting-donor_n-layers-3',
 'Euploid_T21Hearts_Aug2024_sel_subsampled-per-cellstate-donor_correcting-region_n-layers-1',
 'Euploid_T21Hearts_Aug2024_sel_subsampled-per-cellstate-donor_correcting-region_n-layers-2',
 'Euploid_T21Hearts_Aug2024_sel_subsampled-per-cellstate-donor_correcting-region_n-layers-3']
```

Example 19 (python):
```python
for latent_file_name in latent_file_name_list:
    latent_space_condition = latent_file_name.split('subsampled-per-cellstate-donor_')[1]
    print(f'##### {latent_space_condition} #####')
    sc.pp.neighbors(adata, use_rep=f'scVI_latent_{latent_space_condition}',n_neighbors=15)
    sc.tl.umap(adata, min_dist=0.2, spread=1.0)# 
    # check batches
    sc.pl.umap(adata, color=['total_counts','pct_counts_mt','pct_counts_ribo','donor','euploid_or_t21'], 
               size=5, wspace=0.3, ncols=6, components='all',cmap='RdPu')
    print('')
    print('')
```

Example 20 (unknown):
```unknown
##### correcting-donor-region_n-layers-1 #####
```

Example 21 (unknown):
```unknown
##### correcting-donor-region_n-layers-2 #####
```

Example 22 (unknown):
```unknown
##### correcting-donor-region_n-layers-3 #####
```

Example 23 (unknown):
```unknown
##### correcting-donor_n-layers-1 #####
```

Example 24 (unknown):
```unknown
##### correcting-donor_n-layers-2 #####
```

Example 25 (unknown):
```unknown
##### correcting-donor_n-layers-3 #####
```

Example 26 (unknown):
```unknown
##### correcting-region_n-layers-1 #####
```

Example 27 (unknown):
```unknown
##### correcting-region_n-layers-2 #####
```

Example 28 (unknown):
```unknown
##### correcting-region_n-layers-3 #####
```

---

## Prepare Arrow files¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/ATAC/ATAC_preprocess/ArchR_no2_global_CreateArchRproject_Filtering_Clustering.html

**Contents:**
- Prepare Arrow files¶
- Create an ArchR Project¶
- Plotting¶
- Add fine-grained cell type annotation from RNA data¶
- Dimensionality reduction with ArchR¶
  - Iterative Latent Semantic Indexing (LSI)¶
  - Batch Effect Correction wtih Harmony¶
- Clustering¶
  - create a cluster confusion matrix¶
- Single-cell Embeddings¶
  - UMAP¶
- Save project¶
- Save cell index¶

Ref: https://www.archrproject.com/bookdown/creating-an-archrproject-1.html

**Examples:**

Example 1 (unknown):
```unknown
library(ArchR)
library(tidyverse)
```

Example 2 (javascript):
```javascript
/ |
                                                 /    \
            .                                  /      |.
            \\\                              /        |.
              \\\                          /           `|.
                \\\                      /              |.
                  \                    /                |\
                  \\#####\           /                  ||
                ==###########>      /                   ||
                 \\##==......\    /                     ||
            ______ =       =|__ /__                     ||      \\\
        ,--' ,----`-,__ ___/'  --,-`-===================##========>
       \               '        ##_______ _____ ,--,__,=##,__   ///
        ,    __==    ___,-,__,--'#'  ==='      `-'    | ##,-/
        -,____,---'       \\####\\________________,--\\_##,/
           ___      .______        ______  __    __  .______      
          /   \     |   _  \      /      ||  |  |  | |   _  \     
         /  ^  \    |  |_)  |    |  ,----'|  |__|  | |  |_)  |    
        /  /_\  \   |      /     |  |     |   __   | |      /     
       /  _____  \  |  |\  \\___ |  `----.|  |  |  | |  |\  \\___.
      /__/     \__\ | _| `._____| \______||__|  |__| | _| `._____|
    

ArchR : Version 1.0.2
For more information see our website : www.ArchRProject.com
If you encounter a bug please report : https://github.com/GreenleafLab/ArchR/issues

Loading Required Packages...

	Loading Package : grid v4.0.4

	Loading Package : gridExtra v2.3

	Loading Package : gtools v3.9.3

	Loading Package : gtable v0.3.0

	Loading Package : ggplot2 v3.4.0

	Loading Package : magrittr v2.0.3

	Loading Package : plyr v1.8.8

	Loading Package : stringr v1.4.0

	Loading Package : data.table v1.14.4

	Loading Package : matrixStats v0.62.0

	Loading Package : S4Vectors v0.28.1

	Loading Package : GenomicRanges v1.42.0

	Loading Package : BiocGenerics v0.36.1

	Loading Package : Matrix v1.3.2

	Loading Package : Rcpp v1.0.9

	Loading Package : SummarizedExperiment v1.20.0

	Loading Package : rhdf5 v2.34.0

Warning message in system("timedatectl", intern = TRUE):
“running command 'timedatectl' had status 1”
── Attaching packages ─────────────────────────────────────── tidyverse 1.3.2 ──
✔ tibble  3.1.8      ✔ purrr   0.3.4 
✔ tidyr   1.2.1      ✔ dplyr   1.0.10
✔ readr   2.1.3      ✔ forcats 0.5.1 
── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::arrange()         masks plyr::arrange()
✖ dplyr::between()         masks data.table::between()
✖ dplyr::collapse()        masks IRanges::collapse()
✖ dplyr::combine()         masks Biobase::combine(), BiocGenerics::combine(), gridExtra::combine()
✖ purrr::compact()         masks plyr::compact()
✖ dplyr::count()           masks matrixStats::count(), plyr::count()
✖ dplyr::desc()            masks IRanges::desc(), plyr::desc()
✖ tidyr::expand()          masks Matrix::expand(), S4Vectors::expand()
✖ tidyr::extract()         masks magrittr::extract()
✖ dplyr::failwith()        masks plyr::failwith()
✖ dplyr::filter()          masks stats::filter()
✖ dplyr::first()           masks S4Vectors::first(), data.table::first()
✖ dplyr::id()              masks plyr::id()
✖ dplyr::lag()             masks stats::lag()
✖ dplyr::last()            masks data.table::last()
✖ dplyr::mutate()          masks plyr::mutate()
✖ tidyr::pack()            masks Matrix::pack()
✖ BiocGenerics::Position() masks ggplot2::Position(), base::Position()
✖ purrr::reduce()          masks GenomicRanges::reduce(), IRanges::reduce()
✖ dplyr::rename()          masks S4Vectors::rename(), plyr::rename()
✖ purrr::set_names()       masks magrittr::set_names()
✖ dplyr::slice()           masks IRanges::slice()
✖ dplyr::summarise()       masks plyr::summarise()
✖ dplyr::summarize()       masks plyr::summarize()
✖ purrr::transpose()       masks data.table::transpose()
✖ tidyr::unpack()          masks Matrix::unpack()
```

Example 3 (unknown):
```unknown
# before starting a project we must set the ArchRGenome and default threads for parallelization.
# Setting default genome to Hg38.
addArchRGenome("hg38")
```

Example 4 (unknown):
```unknown
Setting default genome to Hg38.
```

Example 5 (unknown):
```unknown
# Setting default number of Parallel threads to 16
addArchRThreads(threads = 10)
```

Example 6 (unknown):
```unknown
Setting default number of Parallel threads to 10.
```

Example 7 (python):
```python
# read in rna metadata
obs = read.csv('/nfs/team205/heart/anndata_objects/Foetal/Feb28ObjectRaw.obs.csv',row.names=1)

# select modality (multiome)
print(table(obs$kit_10x))
obs = obs %>% filter(kit_10x == "Multiome-v1")

# select columns
obs = obs %>% select('sangerID', 'combinedID', 'donor', 
       'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 
       'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts',
       'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts',
       'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt',
       'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo',
       'pct_counts_ribo', 'HB_score', 'donor_by_library.prep','multiplexed',
        'batch_key','region','FACSgate','sex','week','trimester',
                     'fine_grain','mid_grain','coarse_grain',
                     'heart_or_greatvessels','cycling','S_score','G2M_score','phase','stress_score','hb1_score')

print(dim(obs))
head(obs)
```

Example 8 (unknown):
```unknown
3prime-v3   5prime-v2 Multiome-v1 
      39718       46610      211145 
[1] 211145     41
```

Example 9 (unknown):
```unknown
# make arrow file list using combinedID
ArrowFiles = paste0(unique(obs$combinedID),'.arrow')
length(ArrowFiles)
ArrowFiles
```

Example 10 (unknown):
```unknown
proj <- ArchRProject(
  ArrowFiles = ArrowFiles, 
  outputDirectory = "project_output",
  copyArrows = FALSE #This is recommened so that if you modify the Arrow files you have an original copy for later usage.
)
# copyArrows = FALSE since copy Arrows will be created when we save project file
```

Example 11 (python):
```python
Using GeneAnnotation set by addArchRGenome(Hg38)!

Using GeneAnnotation set by addArchRGenome(Hg38)!

Validating Arrows...

Getting SampleNames...



Getting Cell Metadata...



Merging Cell Metadata...

Initializing ArchRProject...


                                                   / |
                                                 /    \
            .                                  /      |.
            \\\                              /        |.
              \\\                          /           `|.
                \\\                      /              |.
                  \                    /                |\
                  \\#####\           /                  ||
                ==###########>      /                   ||
                 \\##==......\    /                     ||
            ______ =       =|__ /__                     ||      \\\
        ,--' ,----`-,__ ___/'  --,-`-===================##========>
       \               '        ##_______ _____ ,--,__,=##,__   ///
        ,    __==    ___,-,__,--'#'  ==='      `-'    | ##,-/
        -,____,---'       \\####\\________________,--\\_##,/
           ___      .______        ______  __    __  .______      
          /   \     |   _  \      /      ||  |  |  | |   _  \     
         /  ^  \    |  |_)  |    |  ,----'|  |__|  | |  |_)  |    
        /  /_\  \   |      /     |  |     |   __   | |      /     
       /  _____  \  |  |\  \\___ |  `----.|  |  |  | |  |\  \\___.
      /__/     \__\ | _| `._____| \______||__|  |__| | _| `._____|
```

Example 12 (unknown):
```unknown
___      .______        ______  __    __  .______      
          /   \     |   _  \      /      ||  |  |  | |   _  \     
         /  ^  \    |  |_)  |    |  ,----'|  |__|  | |  |_)  |    
        /  /_\  \   |      /     |  |     |   __   | |      /     
       /  _____  \  |  |\  \\___ |  `----.|  |  |  | |  |\  \\___.
      /__/     \__\ | _| `._____| \______||__|  |__| | _| `._____|
```

Example 13 (unknown):
```unknown
class: ArchRProject 
outputDirectory: /nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output 
samples(37): BHF_F_Hea11064670_BHF_F_Hea11031823
  BHF_F_Hea11064671_BHF_F_Hea11031824 ...
  HCAHeartST13386009_HCAHeartST13303419
  HCAHeartST13386010_HCAHeartST13303420
sampleColData names(1): ArrowFiles
cellColData names(13): Sample TSSEnrichment ... nDiFrags BlacklistRatio
numberOfCells(1): 497607
medianTSS(1): 11.159
medianFrags(1): 8628
```

Example 14 (unknown):
```unknown
colnames(proj@cellColData)
```

Example 15 (unknown):
```unknown
paste0("Memory Size = ", round(object.size(proj) / 10^6, 3), " MB")
```

Example 16 (unknown):
```unknown
# getAvailableMatrices(proj)
```

Example 17 (unknown):
```unknown
# TSS enrichment score
p1 <- plotGroups(
    ArchRProj = proj, 
    groupBy = "Sample", 
    colorBy = "cellColData", 
    name = "TSSEnrichment",
    plotAs = "violin",
    alpha = 0.4,
    addBoxPlot = TRUE
   )
p1
```

Example 18 (unknown):
```unknown
Length of unique values greater than palette, interpolating..

1
```

Example 19 (unknown):
```unknown
# log10(unique nuclear fragments)
p2 <- plotGroups(
    ArchRProj = proj, 
    groupBy = "Sample", 
    colorBy = "cellColData", 
    name = "log10(nFrags)",
    plotAs = "violin",
    alpha = 0.4,
    addBoxPlot = TRUE
   )
p2
```

Example 20 (unknown):
```unknown
Length of unique values greater than palette, interpolating..

1
```

Example 21 (unknown):
```unknown
plotPDF(p1,p2, name = "QC-Sample-Statistics.pdf", ArchRProj = proj, addDOC = FALSE, width = 8, height = 5)
```

Example 22 (unknown):
```unknown
Plotting Ggplot!

Plotting Ggplot!
```

Example 23 (python):
```python
# change cell names to match with .obs metadata
proj$cellNames_mod = gsub("#", "_", proj$cellNames)
proj$cellNames_mod[1:10]
```

Example 24 (python):
```python
# keep cellColData dataframe for later use
cellColData_prefilter = proj@cellColData

# select shared cells between RNA and ATAC
shared_cells = intersect(proj$cellNames_mod, rownames(obs))
length(shared_cells)

# select shared cells fromr the ArchR pobject
idxSample <- BiocGenerics::which(proj$cellNames_mod %in% shared_cells)
cellsSample <- proj$cellNames[idxSample]
proj = proj[cellsSample, ]
proj

# select shared cells fromr the .obs metadata
obs = obs[proj$cellNames_mod, ]
dim(obs)
```

Example 25 (unknown):
```unknown
___      .______        ______  __    __  .______      
          /   \     |   _  \      /      ||  |  |  | |   _  \     
         /  ^  \    |  |_)  |    |  ,----'|  |__|  | |  |_)  |    
        /  /_\  \   |      /     |  |     |   __   | |      /     
       /  _____  \  |  |\  \\___ |  `----.|  |  |  | |  |\  \\___.
      /__/     \__\ | _| `._____| \______||__|  |__| | _| `._____|
```

Example 26 (unknown):
```unknown
class: ArchRProject 
outputDirectory: /nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output 
samples(37): BHF_F_Hea11064670_BHF_F_Hea11031823
  BHF_F_Hea11064671_BHF_F_Hea11031824 ...
  HCAHeartST13386009_HCAHeartST13303419
  HCAHeartST13386010_HCAHeartST13303420
sampleColData names(1): ArrowFiles
cellColData names(14): Sample TSSEnrichment ... BlacklistRatio
  cellNames_mod
numberOfCells(1): 167022
medianTSS(1): 11.614
medianFrags(1): 10338.5
```

Example 27 (python):
```python
# add metadata
for (c in colnames(obs)){
    proj = addCellColData(ArchRProj = proj, data = obs[[c]], cells = proj$cellNames, name = c)
}
```

Example 28 (unknown):
```unknown
table(proj$coarse_grain)
```

Example 29 (unknown):
```unknown
Cardiomyocytes    Endothelium     Epicardium     Leukocytes    Mesenchymal 
         53512           8859           1554          21340          78731 
        Neural 
          3026
```

Example 30 (unknown):
```unknown
table(proj$fine_grain)
```

Example 31 (unknown):
```unknown
AtrialCardiomyocytesCycling 
                                      1597 
                  AtrialCardiomyocytesLeft 
                                      4777 
                 AtrialCardiomyocytesRight 
                                      8187 
        AtrioventricularNodeCardiomyocytes 
                                       643 
                                    BCells 
                                      1369 
                            BCellsMS4A1pos 
                                      2559 
                           ChromaffinCells 
                                       112 
          CoronaryArterialEndothelialCells 
                                       353 
         CoronaryCapillaryEndothelialCells 
                                      2449 
                         CoronaryPericytes 
                                       688 
                 CoronarySmoothMuscleCells 
                                       415 
            CoronaryVenousEndothelialCells 
                                       539 
      CoronaryVesselAdventitialFibroblasts 
                                      1613 
                      DendriticCellsMature 
                                       156 
                       DendriticCellsType1 
                                       425 
         DuctusArteriosusSmoothMuscleCells 
                                      1159 
                          EndocardialCells 
                                      2690 
                   EndocardialCushionCells 
                                       224 
                    EpicardiumDerivedCells 
                                       340 
         GreatVesselAdventitialFibroblasts 
                                     31525 
       GreatVesselArterialEndothelialCells 
                                       929 
              GreatVesselSmoothMuscleCells 
                                     14126 
         GreatVesselVenousEndothelialCells 
                                       207 
                       InnateLymphoidCells 
                                       547 
                 LymphaticEndothelialCells 
                                      1282 
       LymphNodeFibroblasticReticularCells 
                                       969 
                        MacrophagesATF3pos 
                                       611 
                      MacrophagesCX3CR1pos 
                                       551 
                       MacrophagesLYVE1pos 
                                      4157 
                       MacrophagesTIMD4pos 
                                       303 
                                 MastCells 
                                       190 
                            Megakaryocytes 
                                       118 
                MesothelialEpicardialCells 
                                      1214 
                      MonocyteDerivedCells 
                                       386 
                                 Monocytes 
                                       117 
                           MonocytesMPOpos 
                                        34 
        MyocardialInterstitialFibroblasts1 
                                      7671 
        MyocardialInterstitialFibroblasts2 
                                      5123 
                            Myofibroblasts 
                                      3214 
                        NaturalKillerCells 
                                      1406 
                          NeuronPrecursors 
                                       371 
                     ParaympatheticNeurons 
                                       264 
                         PericardialCells1 
                                      8967 
                         PericardialCells2 
                                       187 
                         PericardialCells3 
                                      1040 
                PlasmacytoidDendriticCells 
                                       430 
                                 ProBCells 
                                       459 
                     SchwannCellPrecursors 
                                       759 
                              SchwannCells 
                                      1166 
              SinoatrialNodeCardiomyocytes 
                                       738 
                        SympatheticNeurons 
                                       354 
                              TCellsCD4pos 
                                      4251 
                              TCellsCD8pos 
                                      1810 
                               TregsCD4pos 
                                      1461 
                     ValveEndothelialCells 
                                       186 
                    ValveInterstitialCells 
                                      2034 
          VentricularCardiomyocytesCycling 
                                      4977 
      VentricularCardiomyocytesLeftCompact 
                                     11397 
 VentricularCardiomyocytesLeftTrabeculated 
                                      5054 
     VentricularCardiomyocytesRightCompact 
                                     12008 
VentricularCardiomyocytesRightTrabeculated 
                                      2269 
         VentricularConductionSystemDistal 
                                       531 
       VentricularConductionSystemProximal 
                                      1334
```

Example 32 (unknown):
```unknown
proj <- addIterativeLSI(
    ArchRProj = proj,
    useMatrix = "TileMatrix", 
    name = "IterativeLSI", 
    iterations = 2, 
    clusterParams = list( #See Seurat::FindClusters
        resolution = c(0.2), 
        sampleCells = 10000, 
        n.start = 10
    ), 
    varFeatures = 25000, 
    dimsToUse = 1:30
)
```

Example 33 (unknown):
```unknown
Checking Inputs...

ArchR logging to : ArchRLogs/ArchR-addIterativeLSI-1c3a2773b85e-Date-2023-03-09_Time-10-33-14.log
If there is an issue, please report to github with logFile!

2023-03-09 10:33:16 : Computing Total Across All Features, 0.028 mins elapsed.

2023-03-09 10:34:39 : Computing Top Features, 1.395 mins elapsed.

###########
2023-03-09 10:34:42 : Running LSI (1 of 2) on Top Features, 1.451 mins elapsed.
###########

2023-03-09 10:34:42 : Sampling Cells (N = 10017) for Estimated LSI, 1.455 mins elapsed.

2023-03-09 10:34:42 : Creating Sampled Partial Matrix, 1.455 mins elapsed.

2023-03-09 10:38:30 : Computing Estimated LSI (projectAll = FALSE), 5.252 mins elapsed.

2023-03-09 10:39:00 : Identifying Clusters, 5.756 mins elapsed.

2023-03-09 10:39:26 : Identified 6 Clusters, 6.194 mins elapsed.

2023-03-09 10:39:27 : Saving LSI Iteration, 6.195 mins elapsed.

Length of unique values greater than palette, interpolating..

2023-03-09 10:39:54 : Creating Cluster Matrix on the total Group Features, 6.661 mins elapsed.

2023-03-09 10:41:01 : Computing Variable Features, 7.777 mins elapsed.

###########
2023-03-09 10:41:02 : Running LSI (2 of 2) on Variable Features, 7.78 mins elapsed.
###########

2023-03-09 10:41:02 : Creating Partial Matrix, 7.782 mins elapsed.

2023-03-09 10:45:38 : Computing LSI, 12.388 mins elapsed.

2023-03-09 10:49:34 : Finished Running IterativeLSI, 16.324 mins elapsed.
```

Example 34 (unknown):
```unknown
proj <- addHarmony(
    ArchRProj = proj,
    reducedDims = "IterativeLSI",
    name = "Harmony",
    groupBy = "batch_key"
)
```

Example 35 (unknown):
```unknown
Warning message:
“Quick-TRANSfer stage steps exceeded maximum (= 8351100)”
Warning message:
“Quick-TRANSfer stage steps exceeded maximum (= 8351100)”
Warning message:
“Quick-TRANSfer stage steps exceeded maximum (= 8351100)”
Warning message:
“Quick-TRANSfer stage steps exceeded maximum (= 8351100)”
Harmony 1/10

Harmony 2/10

Harmony 3/10

Harmony converged after 3 iterations
```

Example 36 (unknown):
```unknown
table(proj$batch_key)
```

Example 37 (unknown):
```unknown
C104_Nuclei_Multiome-v1   C82_Nuclei_Multiome-v1   C85_Nuclei_Multiome-v1 
                    2068                      350                     2218 
  C87_Nuclei_Multiome-v1   C92_Nuclei_Multiome-v1   C98_Nuclei_Multiome-v1 
                    3537                    12468                     2430 
Hst33_Nuclei_Multiome-v1 Hst36_Nuclei_Multiome-v1 Hst39_Nuclei_Multiome-v1 
                   32750                     6351                    23258 
Hst40_Nuclei_Multiome-v1 Hst41_Nuclei_Multiome-v1 
                    8705                    72887
```

Example 38 (unknown):
```unknown
___      .______        ______  __    __  .______      
          /   \     |   _  \      /      ||  |  |  | |   _  \     
         /  ^  \    |  |_)  |    |  ,----'|  |__|  | |  |_)  |    
        /  /_\  \   |      /     |  |     |   __   | |      /     
       /  _____  \  |  |\  \\___ |  `----.|  |  |  | |  |\  \\___.
      /__/     \__\ | _| `._____| \______||__|  |__| | _| `._____|
```

Example 39 (unknown):
```unknown
class: ArchRProject 
outputDirectory: /nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output 
samples(37): BHF_F_Hea11064670_BHF_F_Hea11031823
  BHF_F_Hea11064671_BHF_F_Hea11031824 ...
  HCAHeartST13386009_HCAHeartST13303419
  HCAHeartST13386010_HCAHeartST13303420
sampleColData names(1): ArrowFiles
cellColData names(55): Sample TSSEnrichment ... stress_score hb1_score
numberOfCells(1): 167022
medianTSS(1): 11.614
medianFrags(1): 10338.5
```

Example 40 (unknown):
```unknown
proj <- addClusters(
    input =proj,
    reducedDims = "Harmony",
    method = "Seurat",
    name = "Clusters",
    resolution = 0.8
)
```

Example 41 (unknown):
```unknown
ArchR logging to : ArchRLogs/ArchR-addClusters-1c3a682a96e4-Date-2023-03-09_Time-10-55-55.log
If there is an issue, please report to github with logFile!

2023-03-09 10:55:58 : Running Seurats FindClusters (Stuart et al. Cell 2019), 0.007 mins elapsed.

Computing nearest neighbor graph

Computing SNN
```

Example 42 (unknown):
```unknown
Modularity Optimizer version 1.3.0 by Ludo Waltman and Nees Jan van Eck

Number of nodes: 167022
Number of edges: 5103432

Running Louvain algorithm...
Maximum modularity in 10 random starts: 0.9367
Number of communities: 272
Elapsed time: 89 seconds
```

Example 43 (unknown):
```unknown
234 singletons identified. 38 final clusters.

2023-03-09 11:07:23 : Testing Biased Clusters, 11.426 mins elapsed.

2023-03-09 11:07:24 : Testing Outlier Clusters, 11.455 mins elapsed.

2023-03-09 11:07:25 : Assigning Outlier Clusters (n = 3, nOutlier < 5 cells) to Neighbors, 11.455 mins elapsed.

2023-03-09 11:07:25 : Identified more clusters than maxClusters allowed (n = 3). Merging clusters to maxClusters (n = 25).
If this is not desired set maxClusters = NULL!, 11.469 mins elapsed.

2023-03-09 11:07:26 : Assigning Cluster Names to 25 Clusters, 11.472 mins elapsed.

2023-03-09 11:07:26 : Finished addClusters, 11.477 mins elapsed.
```

Example 44 (unknown):
```unknown
head(proj$Clusters)
```

Example 45 (unknown):
```unknown
library(repr)

# Change plot size to 4 x 3
options(repr.plot.width=10, repr.plot.height=10)
```

Example 46 (unknown):
```unknown
cM <- confusionMatrix(paste0(proj$Clusters), paste0(proj$fine_grain))

library(pheatmap)
cM <- cM / Matrix::rowSums(cM)
p <- pheatmap::pheatmap(
    mat = as.matrix(cM), 
    color = paletteContinuous("whiteBlue"), 
    border_color = "black"
)
p
```

Example 47 (unknown):
```unknown
proj <- addUMAP(
    ArchRProj = proj, 
    reducedDims = "Harmony", 
    name = "UMAP", 
    nNeighbors = 30, 
    minDist = 0.5, 
    metric = "cosine"
)
```

Example 48 (python):
```python
11:07:26 UMAP embedding parameters a = 0.583 b = 1.334

11:07:27 Read 167022 rows and found 30 numeric columns

11:07:27 Using Annoy for neighbor search, n_neighbors = 30

11:07:27 Building Annoy index with metric = cosine, n_trees = 50

0%   10   20   30   40   50   60   70   80   90   100%

[----|----|----|----|----|----|----|----|----|----|

*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
*
|

11:08:15 Writing NN index file to temp file /tmp/RtmpSjr0IA/file1c3a1d0b0164

11:08:16 Searching Annoy index using 13 threads, search_k = 3000

11:08:26 Annoy recall = 99.89%

11:08:29 Commencing smooth kNN distance calibration using 13 threads

11:08:39 Initializing from normalized Laplacian + noise

11:09:02 Commencing optimization for 200 epochs, with 8626560 positive edges

11:11:10 Optimization finished

11:11:10 Creating temp model dir /tmp/RtmpSjr0IA/dir1c3a4f664611

11:11:10 Creating dir /tmp/RtmpSjr0IA/dir1c3a4f664611

11:11:20 Changing to /tmp/RtmpSjr0IA/dir1c3a4f664611

11:11:20 Creating /nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/Embeddings/Save-Uwot-UMAP-Params-Harmony-1c3a5c37709c-Date-2023-03-09_Time-11-11-10.tar
```

Example 49 (unknown):
```unknown
options(repr.plot.width=30, repr.plot.height=10)

p1 <- plotEmbedding(ArchRProj = proj, colorBy = "cellColData", name = "fine_grain", embedding = "UMAP")
p2 <- plotEmbedding(ArchRProj = proj, colorBy = "cellColData", name = "Clusters", embedding = "UMAP")
p3 <- plotEmbedding(ArchRProj = proj, colorBy = "cellColData", name = "donor", embedding = "UMAP")
p4 <- plotEmbedding(ArchRProj = proj, colorBy = "cellColData", name = "week", embedding = "UMAP")
ggAlignPlots(p1,p2,p3,p4, type = "h")
```

Example 50 (unknown):
```unknown
ArchR logging to : ArchRLogs/ArchR-plotEmbedding-1c3aca214ca-Date-2023-03-09_Time-11-11-32.log
If there is an issue, please report to github with logFile!

Getting UMAP Embedding

ColorBy = cellColData

Plotting Embedding

1 
Length of unique values greater than palette, interpolating..



ArchR logging successful to : ArchRLogs/ArchR-plotEmbedding-1c3aca214ca-Date-2023-03-09_Time-11-11-32.log

ArchR logging to : ArchRLogs/ArchR-plotEmbedding-1c3a58bc8fe7-Date-2023-03-09_Time-11-11-34.log
If there is an issue, please report to github with logFile!

Getting UMAP Embedding

ColorBy = cellColData

Plotting Embedding

1 
Length of unique values greater than palette, interpolating..



ArchR logging successful to : ArchRLogs/ArchR-plotEmbedding-1c3a58bc8fe7-Date-2023-03-09_Time-11-11-34.log

ArchR logging to : ArchRLogs/ArchR-plotEmbedding-1c3a7fd8bf0b-Date-2023-03-09_Time-11-11-37.log
If there is an issue, please report to github with logFile!

Getting UMAP Embedding

ColorBy = cellColData

Plotting Embedding

1 


ArchR logging successful to : ArchRLogs/ArchR-plotEmbedding-1c3a7fd8bf0b-Date-2023-03-09_Time-11-11-37.log

ArchR logging to : ArchRLogs/ArchR-plotEmbedding-1c3a69343d9c-Date-2023-03-09_Time-11-11-39.log
If there is an issue, please report to github with logFile!

Getting UMAP Embedding

ColorBy = cellColData

Plotting Embedding

1 


ArchR logging successful to : ArchRLogs/ArchR-plotEmbedding-1c3a69343d9c-Date-2023-03-09_Time-11-11-39.log
```

Example 51 (unknown):
```unknown
plotPDF(p1,p2,p3,p4, name = "Global_Plot-UMAPs.pdf", ArchRProj = proj, addDOC = FALSE, width = 8, height = 8)
```

Example 52 (unknown):
```unknown
Plotting Ggplot!

Plotting Ggplot!

Plotting Ggplot!

Plotting Ggplot!
```

Example 53 (unknown):
```unknown
saveArchRProject(ArchRProj = proj, outputDirectory = "project_output", load = FALSE)
```

Example 54 (unknown):
```unknown
Copying Arrow Files...

Copying Arrow Files (1 of 37)

Copying Arrow Files (2 of 37)

Copying Arrow Files (3 of 37)

Copying Arrow Files (4 of 37)

Copying Arrow Files (5 of 37)

Copying Arrow Files (6 of 37)

Copying Arrow Files (7 of 37)

Copying Arrow Files (8 of 37)

Copying Arrow Files (9 of 37)

Copying Arrow Files (10 of 37)

Copying Arrow Files (11 of 37)

Copying Arrow Files (12 of 37)

Copying Arrow Files (13 of 37)

Copying Arrow Files (14 of 37)

Copying Arrow Files (15 of 37)

Copying Arrow Files (16 of 37)

Copying Arrow Files (17 of 37)

Copying Arrow Files (18 of 37)

Copying Arrow Files (19 of 37)

Copying Arrow Files (20 of 37)

Copying Arrow Files (21 of 37)

Copying Arrow Files (22 of 37)

Copying Arrow Files (23 of 37)

Copying Arrow Files (24 of 37)

Copying Arrow Files (25 of 37)

Copying Arrow Files (26 of 37)

Copying Arrow Files (27 of 37)

Copying Arrow Files (28 of 37)

Copying Arrow Files (29 of 37)

Copying Arrow Files (30 of 37)

Copying Arrow Files (31 of 37)

Copying Arrow Files (32 of 37)

Copying Arrow Files (33 of 37)

Copying Arrow Files (34 of 37)

Copying Arrow Files (35 of 37)

Copying Arrow Files (36 of 37)

Copying Arrow Files (37 of 37)

Saving ArchRProject...
```

Example 55 (unknown):
```unknown
___      .______        ______  __    __  .______      
          /   \     |   _  \      /      ||  |  |  | |   _  \     
         /  ^  \    |  |_)  |    |  ,----'|  |__|  | |  |_)  |    
        /  /_\  \   |      /     |  |     |   __   | |      /     
       /  _____  \  |  |\  \\___ |  `----.|  |  |  | |  |\  \\___.
      /__/     \__\ | _| `._____| \______||__|  |__| | _| `._____|
```

Example 56 (unknown):
```unknown
class: ArchRProject 
outputDirectory: /nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output 
samples(37): BHF_F_Hea11064670_BHF_F_Hea11031823
  BHF_F_Hea11064671_BHF_F_Hea11031824 ...
  HCAHeartST13386009_HCAHeartST13303419
  HCAHeartST13386010_HCAHeartST13303420
sampleColData names(1): ArrowFiles
cellColData names(56): Sample TSSEnrichment ... hb1_score Clusters
numberOfCells(1): 167022
medianTSS(1): 11.614
medianFrags(1): 10338.5
```

Example 57 (unknown):
```unknown
filetxt<-file("foetal_ATAC_index.txt")
writeLines(proj$cellNames_mod, filetxt)
close(filetxt)
```

---

## Read in adata¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/Xenium/plotting_for-Fig.html

**Contents:**
- Read in adata¶
- Log-normalisation¶
- Plot cell types¶
- UMAP¶
- TissueTypist niches¶

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os

import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42
```

Example 2 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 3 (unknown):
```unknown
os.getcwd()
```

Example 4 (unknown):
```unknown
'/rfs/project/rfs-iCNyzSAaucw/kk837/notebooks/Foetal/Xenium/5K/C194_Hst45/CamU'
```

Example 5 (unknown):
```unknown
figdir = '/rfs/project/rfs-iCNyzSAaucw/kk837/notebooks/Foetal/fetal_ms_figures'
# table_dir = '/nfs/team205/heart/fetal_ms_figures/tables/Aug2024'
```

Example 6 (unknown):
```unknown
celltype_key = 'fine_grain_12Jun2025'
```

Example 7 (python):
```python
objects_dir = '/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium'
path_adata = f'{objects_dir}/C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat_5K_filtered_raw.h5ad'
adata = sc.read_h5ad(path_adata)
adata
```

Example 8 (unknown):
```unknown
AnnData object with n_obs × n_vars = 362277 × 5001
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_3.0', 'multi_celltypes_coarse', 'fine_grain_12Jun2025', 'sliding_window_assignment', 'window_col', 'window_row', 'globalX', 'globalY', 'tt_prediction'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_mid_fin_colors', 'dendrogram_pca_harmony_leiden_1.0', 'fine_grain_12Jun2025_colors', 'multi_celltypes_coarse_colors', 'pca', 'pca_harmony', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.2_colors', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.3_colors', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.4_colors', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.5_colors', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_0.8_colors', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.0_colors', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_1.5_colors', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_2.0_colors', 'pca_harmony_leiden_3.0', 'pca_harmony_leiden_3.0_colors', 'tissue_block_id_colors', 'umap'
    obsm: 'X_pca', 'X_pca_harmony', 'X_umap_pca_harmony', 'spatial'
    varm: 'PCs'
    obsp: 'pca_harmony_connectivities', 'pca_harmony_distances'
```

Example 9 (unknown):
```unknown
os.getcwd()
```

Example 10 (unknown):
```unknown
'/rfs/project/rfs-iCNyzSAaucw/kk837/notebooks/Foetal/Xenium/5K/C194_Hst45/CamU'
```

Example 11 (python):
```python
adata.X.data[:5]
```

Example 12 (unknown):
```unknown
array([5., 1., 1., 2., 2.], dtype=float32)
```

Example 13 (python):
```python
# celltype name mapping for plotting
df = pd.read_csv('/rfs/project/rfs-iCNyzSAaucw/kk837/notebooks/Foetal/finegrain_name_mapping.csv')
celltype_mapping = df.set_index('Full_name')['Short_name_finalised'].to_dict()
adata.obs.replace({celltype_key:celltype_mapping},inplace=True)
```

Example 14 (python):
```python
/tmp/ipykernel_968450/407166641.py:4: FutureWarning: The behavior of Series.replace (and DataFrame.replace) with CategoricalDtype is deprecated. In a future version, replace will only be used for cases that preserve the categories. To change the categories, use ser.cat.rename_categories instead.
  adata.obs.replace({celltype_key:celltype_mapping},inplace=True)
```

Example 15 (python):
```python
# log-normalise the count
sc.pp.normalize_total(adata,target_sum=1e4)
sc.pp.log1p(adata)
adata.X.data[:5]
```

Example 16 (unknown):
```unknown
array([4.2093534, 2.6576407, 2.6576407, 3.3151019, 3.3151019],
      dtype=float32)
```

Example 17 (python):
```python
set(adata.obs[celltype_key])
```

Example 18 (unknown):
```unknown
{'?EpendymalCells',
 '?EpithelialCells',
 '?SkeletalMuscle',
 '?ThyroidCells',
 'AVNPC',
 'AtrialCardiomyocytes_DoubletWithEC',
 'AtrialCardiomyocytes_DoubletWithFB',
 'B',
 'ChromC',
 'CorArtEC',
 'CorCapEC',
 'CorCapECCyc',
 'CorCapECN',
 'CorPeri',
 'CorSMC',
 'CorVenEC',
 'CoronaryCapillaryEndothelialCells_DoubletsWithEryth',
 'DASMC',
 'DC',
 'ELNeuro',
 'EPDC',
 'EndoCush',
 'Endocard',
 'EndocardCyc',
 'EndocardialCells_Doublets',
 'Eryth',
 'ErythCyc',
 'FB',
 'FBCOL2A1',
 'FBCyc',
 'GVArtEC',
 'GVSMC',
 'GVSMCCyc',
 'GVVenEC',
 'ILC',
 'LymphEC',
 'MPLYVE1',
 'MacCX3',
 'MastC',
 'MegaK',
 'MesEpiC',
 'Mono',
 'MyeloidCells_Doublets',
 'NeuroPre',
 'ParaN',
 'PericCyc',
 'PericFib',
 'PericInt',
 'SANPC',
 'SOX2_LAMB3_pos',
 'SchwC',
 'SchwCyc',
 'SchwP',
 'SchwannCells_DoubletsWithNeurons',
 'SympN',
 'TCD4',
 'TCD8',
 'ThyroidEpithelialCells',
 'VCSDist',
 'ValveEC',
 'aCMCyc',
 'aCML',
 'aCMR',
 'doublets',
 'unassigned',
 'unclassified',
 'vCM',
 'vCMCyc'}
```

Example 19 (python):
```python
tissue_id = 'C194-HEA-0-FFPE-1'
slide = adata[adata.obs['tissue_block_id']==tissue_id]

sc.pl.spatial(slide,color=[celltype_key],
              spot_size=15,vmax='p100',
              crop_coord=(4600,5600,4250,5250)
             )
```

Example 20 (python):
```python
/tmp/ipykernel_3227243/3010556783.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=[celltype_col],
```

Example 21 (python):
```python
tissue_id = 'C194-HEA-0-FFPE-1'
slide = adata[adata.obs['tissue_block_id']==tissue_id]

celltypes_list = ['AVNPC', 'CorCapEC', 'CorPeri', 'DASMC', 'EPDC', 'EndoCush', 'Endocard',
       'Eryth', 'FB', 'GVSMC', 'GVSMCT', 'GVVenEC', 'LymphEC', 'MPLYVE1',
       'MacCX3', 'MesEpiC', 'MoDC', 'ParaN', 'PericFib', 'SANPC', 'SchwC',
       'SchwNeuro', 'ValveEC', 'ValveIC', 'aCML', 'aCMR']
color_list = ['#ffff00',
 '#1ce6ff',
 '#ff34ff',
 '#a30059',
 '#ffdbe5',
 '#63ffac',
 '#b79762',
 '#004d43',
 '#8fb0ff',
 '#997d87',
 '#5a0007',
 'red',
 '#4a3b53',
 '#ff2f80',
 '#ba0900',
 '#6b7900',
 '#00c2a0',
 'blue',
 '#b903aa',
 (0.5, 0.0, 0.5, 0.7),
 '#7b4f4b',
 '#013349',
 '#00846f',
 '#372101',
 '#ffb500',
 '#c2ffed',
 '#cc0744',
 '#c0b9b2']

for celltype,color in zip(celltypes_list,color_list):
    # update colors
    if celltype in set(slide.obs[celltype_key]):
        slide.uns[f"{celltype_key}_colors"][list(slide.obs[celltype_key].cat.categories).index(celltype)]=color

celltypes_to_plot = ['SANPC',
                     'CorCapEC',
                     'CorPeri',
                     'GVVenEC',
                     'MesEpiC',
                     'FB',
                     'aCMR',
                     'AVNPC',
                     'ParaN',
                     'SchwP',
                    ]

sc.pl.spatial(slide,color=[celltype_key],groups=celltypes_to_plot,
              spot_size=15,vmax='p100',
              crop_coord=(4600,5600,4250,5250),
              show=False
             )
plt.savefig(f'{figdir}/Xenium5K_euploid_SAN_for-Fig1.pdf',bbox_inches="tight",dpi=300)
```

Example 22 (python):
```python
/tmp/ipykernel_3227243/2717981830.py:54: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=[celltype_key],groups=celltypes_to_plot,
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/plotting/_utils.py:406: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  adata.uns[color_key] = _palette
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/plotting/_tools/scatterplots.py:1148: UserWarning: *c* argument looks like a single numeric RGB or RGBA sequence, which should be avoided as value-mapping will have precedence in case its length matches with *x* & *y*.  Please use the *color* keyword-argument or provide a 2D array with a single row if you intend to specify the same RGB or RGBA value for all points.
  ax.scatter([], [], c=palette[label], label=label)
```

Example 23 (python):
```python
ADATA_PATH = "/rfs/project/rfs-iCNyzSAaucw/kk837/data_objects/Foetal/RNA/Feb28ObjectRaw_finegrain_updated.h5ad"
ad = sc.read_h5ad(ADATA_PATH, backed='r')

# Step 1: Create mapping (already done)
cat = "mid_grain"
mid2colours = dict(zip(ad.obs[cat].cat.categories, ad.uns[f"{cat}_colors"]))

# Step 2: Extract the labels and colors in the correct order
mid_grains = list(mid2colours.keys())
palette = list(mid2colours.values())
num_mid_grains = len(mid_grains)

# Step 3: Plot
plt.figure(figsize=(num_mid_grains * 0.4, 2))  # wide figure
sns.palplot(palette)
plt.xticks(np.arange(num_mid_grains), mid_grains, rotation=90)
plt.title("Mid Grain to Color Mapping")
plt.tight_layout()
plt.show()
```

Example 24 (python):
```python
/tmp/ipykernel_794362/455462603.py:18: UserWarning: Tight layout not applied. The bottom and top margins cannot be made large enough to accommodate all Axes decorations.
  plt.tight_layout()
```

Example 25 (unknown):
```unknown
<Figure size 448x160 with 0 Axes>
```

Example 26 (python):
```python
adata.obs['celltypist_mid_fin'] = pd.Categorical(
    adata.obs['celltypist_mid_fin'],
    categories=ad.obs[cat].cat.categories,
    ordered=True
)
```

Example 27 (python):
```python
latent_space = 'pca_harmony'
sc.pl.embedding(adata,
                    basis=f"X_umap_{latent_space}",
                    color='celltypist_mid_fin',palette=palette,
                show=False
                   )
plt.savefig(f'{figdir}/Xenium5K_UMAP-midgrain_for-Fig1.pdf',bbox_inches="tight",dpi=300)
```

Example 28 (python):
```python
tissue_id = 'C194-HEA-0-FFPE-1'
slide = adata[adata.obs['tissue_block_id']==tissue_id]

sc.pl.spatial(slide,color=['celltypist_mid_fin'],
              spot_size=15,vmax='p100',
              # crop_coord=(4600,5600,4250,5250),
              show=False
             )
# plt.savefig(f'{figdir}/Xenium5K_euploid_SAN_for-Fig1.pdf',bbox_inches="tight",dpi=300)
```

Example 29 (python):
```python
/tmp/ipykernel_794362/1888614849.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['celltypist_mid_fin'],
```

Example 30 (unknown):
```unknown
[<Axes: title={'center': 'celltypist_mid_fin'}, xlabel='spatial1', ylabel='spatial2'>]
```

Example 31 (unknown):
```unknown
AnnotationPalette={ 
    'Atrium':'#eac075',
        'Ventricle - Compact':'#f1acfa',
     'Ventricle - Trabeculated':'#f09951',
        'SAnode':'#8A2BE2',
        'AVnode':'#062b4a',
    'VCS - Proximal':'#FFFF00',
        'VCS - Distal':'#7FFFD4',
        'Epicardium':'#01ccc8',
        'Endocardium - Atrial':'#DC143C',
        'Endocardium - Ventricular':'#f53527',
        'Endocardial cusion':'#ff7e97',
        'Great vessel':'#ff9b6e',
    'Ductus arteriosus':'#8a6800',
        'Coronary vessel':'#206601',
        'Adventitia - Coronary':'#CAFF70',
     'Adventitia - GV':'#005aa0',
        'AV ring':'#e478ff',
    'Valves':'#3300FF',
    'LN':'#9088ff',
}
```

Example 32 (python):
```python
tissue_id = 'C194-HEA-0-FFPE-1'
slide = adata[adata.obs['tissue_block_id']==tissue_id]

sc.pl.spatial(slide,color=['tt_prediction'],
              spot_size=15,vmax='p100',
              # crop_coord=(4600,5600,4250,5250),
              show=False,
              palette=AnnotationPalette
             )
plt.savefig(f'{figdir}/TissueTypist_Xenium5K_whole-heart.pdf',bbox_inches="tight",dpi=300)
```

Example 33 (python):
```python
/tmp/ipykernel_968450/2063101229.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['tt_prediction'],
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/plotting/_utils.py:487: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  adata.uns[value_to_plot + "_colors"] = colors_list
```

Example 34 (python):
```python
sc.pl.spatial(slide,color=['tt_prediction'],
              spot_size=15,vmax='p100',
              crop_coord=(5500,6800,4250,5500),
              show=False,
              palette=AnnotationPalette
             )
plt.savefig(f'{figdir}/TissueTypist_Xenium5K_Atrium-AVN.pdf',bbox_inches="tight",dpi=300)
```

Example 35 (python):
```python
/tmp/ipykernel_968450/2129560865.py:1: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['tt_prediction'],
```

Example 36 (python):
```python
sc.pl.spatial(slide,color=['tt_prediction'],
              spot_size=15,vmax='p100',
              crop_coord=(8000,9500,2500,4000),
              show=False,
              palette=AnnotationPalette
             )
plt.savefig(f'{figdir}/TissueTypist_Xenium5K_Ventricle.pdf',bbox_inches="tight",dpi=300)
```

Example 37 (python):
```python
/tmp/ipykernel_968450/2311424067.py:1: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['tt_prediction'],
```

Example 38 (python):
```python
sc.pl.spatial(slide,color=['tt_prediction'],
              spot_size=15,vmax='p100',
              crop_coord=(3500,5000,1800,3300),
              show=False,
              palette=AnnotationPalette
             )
plt.savefig(f'{figdir}/TissueTypist_Xenium5K_GV.pdf',bbox_inches="tight",dpi=300)
```

Example 39 (python):
```python
/tmp/ipykernel_968450/3875201264.py:1: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['tt_prediction'],
```

---

## Read in peak matrix¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/ATAC/ATAC_preprocess/ArchR_no5-1_Project-to-Matrix_TFIDF.html

**Contents:**
- Read in peak matrix¶
- TF-IDF normalisation¶

Ref: https://www.archrproject.com/bookdown/creating-an-archrproject-1.html

Using Signac function ref: https://stuartlab.org/signac/reference/runtfidf

**Examples:**

Example 1 (unknown):
```unknown
library(ArchR)
library(tidyverse)
library(BSgenome.Hsapiens.UCSC.hg38)
library(SingleCellExperiment)
library(Signac)
```

Example 2 (python):
```python
/ |
                                                 /    \
            .                                  /      |.
            \\\                              /        |.
              \\\                          /           `|.
                \\\                      /              |.
                  \                    /                |\
                  \\#####\           /                  ||
                ==###########>      /                   ||
                 \\##==......\    /                     ||
            ______ =       =|__ /__                     ||      \\\
        ,--' ,----`-,__ ___/'  --,-`-===================##========>
       \               '        ##_______ _____ ,--,__,=##,__   ///
        ,    __==    ___,-,__,--'#'  ==='      `-'    | ##,-/
        -,____,---'       \\####\\________________,--\\_##,/
           ___      .______        ______  __    __  .______      
          /   \     |   _  \      /      ||  |  |  | |   _  \     
         /  ^  \    |  |_)  |    |  ,----'|  |__|  | |  |_)  |    
        /  /_\  \   |      /     |  |     |   __   | |      /     
       /  _____  \  |  |\  \\___ |  `----.|  |  |  | |  |\  \\___.
      /__/     \__\ | _| `._____| \______||__|  |__| | _| `._____|
    

ArchR : Version 1.0.2
For more information see our website : www.ArchRProject.com
If you encounter a bug please report : https://github.com/GreenleafLab/ArchR/issues

Loading Required Packages...

	Loading Package : grid v4.0.4

	Loading Package : gridExtra v2.3

	Loading Package : gtools v3.9.3

	Loading Package : gtable v0.3.0

	Loading Package : ggplot2 v3.4.0

	Loading Package : magrittr v2.0.3

	Loading Package : plyr v1.8.8

	Loading Package : stringr v1.4.0

	Loading Package : data.table v1.14.4

	Loading Package : matrixStats v0.62.0

	Loading Package : S4Vectors v0.28.1

	Loading Package : GenomicRanges v1.42.0

	Loading Package : BiocGenerics v0.36.1

	Loading Package : Matrix v1.3.2

	Loading Package : Rcpp v1.0.9

	Loading Package : SummarizedExperiment v1.20.0

	Loading Package : rhdf5 v2.34.0

Warning message in system("timedatectl", intern = TRUE):
“running command 'timedatectl' had status 1”
── Attaching packages ─────────────────────────────────────── tidyverse 1.3.2 ──
✔ tibble  3.1.8      ✔ purrr   0.3.4 
✔ tidyr   1.2.1      ✔ dplyr   1.0.10
✔ readr   2.1.3      ✔ forcats 0.5.1 
── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::arrange()         masks plyr::arrange()
✖ dplyr::between()         masks data.table::between()
✖ dplyr::collapse()        masks IRanges::collapse()
✖ dplyr::combine()         masks Biobase::combine(), BiocGenerics::combine(), gridExtra::combine()
✖ purrr::compact()         masks plyr::compact()
✖ dplyr::count()           masks matrixStats::count(), plyr::count()
✖ dplyr::desc()            masks IRanges::desc(), plyr::desc()
✖ tidyr::expand()          masks Matrix::expand(), S4Vectors::expand()
✖ tidyr::extract()         masks magrittr::extract()
✖ dplyr::failwith()        masks plyr::failwith()
✖ dplyr::filter()          masks stats::filter()
✖ dplyr::first()           masks S4Vectors::first(), data.table::first()
✖ dplyr::id()              masks plyr::id()
✖ dplyr::lag()             masks stats::lag()
✖ dplyr::last()            masks data.table::last()
✖ dplyr::mutate()          masks plyr::mutate()
✖ tidyr::pack()            masks Matrix::pack()
✖ BiocGenerics::Position() masks ggplot2::Position(), base::Position()
✖ purrr::reduce()          masks GenomicRanges::reduce(), IRanges::reduce()
✖ dplyr::rename()          masks S4Vectors::rename(), plyr::rename()
✖ purrr::set_names()       masks magrittr::set_names()
✖ dplyr::slice()           masks IRanges::slice()
✖ dplyr::summarise()       masks plyr::summarise()
✖ dplyr::summarize()       masks plyr::summarize()
✖ purrr::transpose()       masks data.table::transpose()
✖ tidyr::unpack()          masks Matrix::unpack()
Loading required package: BSgenome

Loading required package: Biostrings

Loading required package: XVector


Attaching package: ‘XVector’


The following object is masked from ‘package:purrr’:

    compact


The following object is masked from ‘package:plyr’:

    compact



Attaching package: ‘Biostrings’


The following object is masked from ‘package:base’:

    strsplit


Loading required package: rtracklayer

Registered S3 method overwritten by 'GGally':
  method from   
  +.gg   ggplot2
```

Example 3 (unknown):
```unknown
# before starting a project we must set the ArchRGenome and default threads for parallelization.
# Setting default genome to Hg38.
addArchRGenome("hg38")
```

Example 4 (unknown):
```unknown
Setting default genome to Hg38.
```

Example 5 (unknown):
```unknown
# Setting default number of Parallel threads to 16
addArchRThreads(threads = 1)
```

Example 6 (unknown):
```unknown
Setting default number of Parallel threads to 1.
```

Example 7 (unknown):
```unknown
mat = readMM(file="/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/PeakMatrix/matrix.mtx.gz")
```

Example 8 (unknown):
```unknown
mat_norm = RunTFIDF(object = mat) # default scale factor is "scale.factor = 10000"
```

Example 9 (unknown):
```unknown
Performing TF-IDF normalization
```

Example 10 (unknown):
```unknown
dim(mat_norm)
```

Example 11 (unknown):
```unknown
mat_norm[1:10,1:10]
```

Example 12 (python):
```python
10 x 10 sparse Matrix of class "dgCMatrix"
                                                                            
 [1,] . . .        .        .        .         .        .        2.9902939 .
 [2,] . . .        .        .        .         .        .        .         .
 [3,] . . .        .        .        .         .        .        .         .
 [4,] . . .        .        .        .         .        .        .         .
 [5,] . . .        .        .        .         .        .        2.9349327 .
 [6,] . . 1.130476 1.087975 1.056641 0.7904304 1.256202 .        0.8824317 .
 [7,] . . .        .        .        .         .        .        .         .
 [8,] . . .        .        .        .         .        .        .         .
 [9,] . . 2.641712 .        .        .         .        2.138465 .         .
[10,] . . .        .        .        .         .        .        .         .
```

Example 13 (unknown):
```unknown
# save
writeMM(obj = mat_norm, file="/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/PeakMatrix/matrix_tfidf.mtx")
```

Example 14 (unknown):
```unknown
# at terminal
gzip -v /nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/PeakMatrix/matrix_tfidf.mtx
```

---

## Variables¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/VisiumHD/no7_manual-annotation_fibroblasts.html

**Contents:**
- Variables¶
- Read in data¶
- Remove mid-grain markers from the feature¶
- PCA, Clustering, UMAP¶
- Annotation¶
- Save¶

2543 rows × 7 columns

--> cluster 1 has low total_counts and bin_counts. and no specific marker expression --> will annotated as "unassigned"

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os

import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 

import celltypist
```

Example 2 (python):
```python
# import session_info
# session_info.show()
```

Example 3 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 4 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 5 (python):
```python
adata_dir = '/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/subsets'
sample_id = 'HEA_FOET14880396'
celltype = 'Fibroblasts'
```

Example 6 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk/Aug2024'
table_dir = '/nfs/team205/heart/fetal_ms_figures/tables/Aug2024'
```

Example 7 (python):
```python
# read in
adata = sc.read_h5ad(f'{adata_dir}/{sample_id}_{celltype}_b2c_cells_filtered_lognorm.h5ad')
adata
```

Example 8 (unknown):
```unknown
AnnData object with n_obs × n_vars = 20318 × 17926
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'leiden_0.2', 'leiden_0.4', 'leiden_0.6', 'leiden_1', 'leiden_1.2', 'leiden_1.5', 'leiden_2'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1.2', 'leiden_1.2_colors', 'leiden_1.5', 'leiden_1.5_colors', 'leiden_1_colors', 'leiden_2', 'leiden_2_colors', 'log1p', 'neighbors', 'pca', 'spatial', 'umap'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 9 (python):
```python
# read in
midgrain_markers = pd.read_csv('/nfs/team205/kk18/notebooks/Foetal/VisiumHD/re-seq_mpp0p3/sc_midgrain_markers.csv',
                               index_col=0
                              )
# remove the markers of celltype of interest
midgrain_markers = midgrain_markers[midgrain_markers['group']!=celltype]
midgrain_markers
```

Example 10 (python):
```python
# remove
mask = [x not in list(midgrain_markers['names']) for x in adata.var_names]
bdata = adata[:,mask]
bdata
```

Example 11 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 20318 × 16642
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'leiden_0.2', 'leiden_0.4', 'leiden_0.6', 'leiden_1', 'leiden_1.2', 'leiden_1.5', 'leiden_2'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1.2', 'leiden_1.2_colors', 'leiden_1.5', 'leiden_1.5_colors', 'leiden_1_colors', 'leiden_2', 'leiden_2_colors', 'log1p', 'neighbors', 'pca', 'spatial', 'umap'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 12 (python):
```python
# HVGs, using the object which doesn't have midgrain markers
sc.pp.highly_variable_genes(bdata, flavor='seurat', min_mean=0.0125, max_mean=3, min_disp=0.5)
print(f'number of hvgs: {sum(bdata.var.highly_variable)}')
# PCA using the degected HVGs
sc.tl.pca(bdata, use_highly_variable=True)
adata.obsm['X_pca'] = bdata.obsm['X_pca'].copy()
# create neighbour graph
sc.pp.neighbors(adata,n_neighbors=15,use_rep='X_pca')
# visualisation with umap
sc.tl.umap(adata,min_dist=0.5)
sc.pl.embedding(adata,basis='X_umap',color=['bin_count','total_counts','n_genes_by_counts',"conf_score_midmod2fine","celltypist_midmod2fine",],
                wspace=0.2,cmap='RdPu',vmax='p100',ncols=5)
# leiden clustering
resolutions_list = [0.2,0.4,0.6,1,1.2,1.5,2]
for resolution in resolutions_list:
    sc.tl.leiden(adata,resolution=resolution,key_added=f'leiden_{str(resolution)}',n_iterations=2)
sc.pl.embedding(adata,basis='X_umap',
                color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                wspace=0.3)
```

Example 13 (unknown):
```unknown
number of hvgs: 3277
```

Example 14 (python):
```python
sc.pl.embedding(adata,basis='X_umap',
                color=['celltypist_midmod2fine', 'conf_score_midmod2fine'],
                wspace=0.7)
```

Example 15 (python):
```python
'SLC18A3' in adata.var_names
```

Example 16 (python):
```python
# markers for fine-grained cell types
sc_markers = pd.read_csv(f'/nfs/team205/kk18/notebooks/Foetal/VisiumHD/re-seq_mpp0p3/sc_{celltype}_finegrain_markers.csv',
                         index_col=0)
markers = {}
n_top = 10
for c in set(sc_markers['group']):
    markers_sub = sc_markers[sc_markers['group']==c]
    markers_sub.sort_values('logfoldchanges',ascending=False)
    markers[c]=list(markers_sub[:n_top]['names'])

genes = []
for g in markers.values():
    genes = genes + [x for x in g if (x in adata.var_names)&(x not in genes)]
for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in adata.var_names]
genes = list(set(genes))
len(genes)
```

Example 17 (python):
```python
sc.pl.embedding(adata,basis='X_umap',color=genes,
               ncols=6, wspace=0.2,cmap='RdPu',vmax='p99.9')
```

Example 18 (unknown):
```unknown
resolution_sel = 0.6
```

Example 19 (python):
```python
for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in adata.var_names]

sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
sc.pl.dotplot(adata,
              markers, 
              groupby=f'leiden_{str(resolution_sel)}',
              dendrogram=True,
              standard_scale="var",
              color_map="Reds",
              swap_axes=False,
             )
```

Example 20 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: Myofibroblasts, MyocardialInterstitialFibroblasts, GreatVesselAdventitialFibroblasts, etc.
```

Example 21 (python):
```python
# DEG analysis
print(adata.X.data[:5])
adata.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata, dendrogram=True, standard_scale='var', return_fig=True)
dp.add_totals().show()
```

Example 22 (unknown):
```unknown
[1.59681474 1.90298047 2.95755701 1.65405184 3.1673657 ]
```

Example 23 (python):
```python
# DEG analysis without other cell type markers
# with removing mid_grain markers
# remove mid_grain markers
mask = [x not in list(midgrain_markers['names']) for x in adata.var_names]
bdata = adata[:,mask]

print(bdata.X.data[:5])
bdata.uns['log1p']['base']=None
sc.tl.rank_genes_groups(bdata,groupby=f'leiden_{str(resolution_sel)}')
sc.tl.dendrogram(bdata, groupby=f'leiden_{str(resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(bdata, dendrogram=True, standard_scale='var', return_fig=True)
dp.add_totals().show()
```

Example 24 (unknown):
```unknown
[1.59681474 2.95755701 1.65405184 3.1673657  1.84266849]
```

Example 25 (python):
```python
sc.pl.dotplot(adata,
              ['COL1A1','COL1A2','COL3A1','IL11'], 
              groupby=f'leiden_{str(resolution_sel)}',
              dendrogram=True,
              standard_scale="var",
              color_map="Reds",
              swap_axes=False,
             )
```

Example 26 (python):
```python
sc.pl.embedding(adata,basis='X_umap',
                color=f'leiden_{str(resolution_sel)}',
                wspace=0.3, legend_loc='on data')
```

Example 27 (python):
```python
sc.pl.violin(adata,keys=['total_counts','bin_count'],groupby=f'leiden_{str(resolution_sel)}')
```

Example 28 (unknown):
```unknown
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 29 (python):
```python
# annotate
adata.obs['fine_grain_manual'] = adata.obs[f'leiden_{str(resolution_sel)}'].copy()
adata.obs.replace({'fine_grain_manual':{
    '0':'Fibroblasts',
    '1':'unassigned',
    '2':'ValveInterstitialCells',
    '3':'Fibroblasts',
    '4':'FibroblastsCOL2A1pos',
     '5':'Fibroblasts_DoubletsWithCMs',
}},inplace=True)
adata.obs['fine_grain_manual'].value_counts()
```

Example 30 (unknown):
```unknown
fine_grain_manual
Fibroblasts                    7849
unassigned                     5380
ValveInterstitialCells         5186
FibroblastsCOL2A1pos           1648
Fibroblasts_DoubletsWithCMs     255
Name: count, dtype: int64
```

Example 31 (python):
```python
sc.pl.embedding(adata,basis='X_umap',
                color='fine_grain_manual',
                frameon=False,
                wspace=0.3)
```

Example 32 (python):
```python
# DEG analysis
print(adata.X.data[:5])
adata.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata,groupby='fine_grain_manual')
sc.tl.dendrogram(adata, groupby='fine_grain_manual')
dp=sc.pl.rank_genes_groups_dotplot(adata, dendrogram=True, standard_scale='var', return_fig=True)
dp.add_totals().show()
```

Example 33 (unknown):
```unknown
[1.59681474 1.90298047 2.95755701 1.65405184 3.1673657 ]
```

Example 34 (python):
```python
# save
adata.write(f'{adata_dir}/{sample_id}_{celltype}_b2c_cells_filtered_lognorm.h5ad')
print(f'{adata_dir}/{sample_id}_{celltype}_b2c_cells_filtered_lognorm.h5ad')
```

Example 35 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/subsets/HEA_FOET14880396_Fibroblasts_b2c_cells_filtered_lognorm.h5ad
```

---

## Import modules¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/RNA/QC/Foetal_scRNA_No2_QC-filter.html

**Contents:**
- Import modules¶
- Functions¶
- Read in anndata¶
- Remove doublets¶
- Remove low quality cells¶
- Compute QC stats¶
- Remove cells exceeding the ribo/mito threshold¶
- Calculate RBC score¶
- Remove RBCs¶
- Save¶

**Examples:**

Example 1 (python):
```python
import numpy as np
import pandas as pd
import scanpy as sc
import matplotlib.pyplot as plt
import scrublet as scr
import session_info
```

Example 2 (python):
```python
sc.settings.verbosity = 3
sc.settings.set_figure_params(dpi = 120, color_map = 'RdPu', dpi_save = 180, vector_friendly = True, format = 'svg')
session_info.show()
```

Example 3 (python):
```python
-----
matplotlib          3.3.4
numpy               1.20.1
pandas              1.1.4
scanpy              1.7.1
scrublet            NA
session_info        1.0.0
-----
```

Example 4 (unknown):
```unknown
PIL                 8.1.2
anndata             0.7.5
anyio               NA
attr                20.3.0
babel               2.9.0
backcall            0.2.0
brotli              NA
cairo               1.20.0
certifi             2020.12.05
cffi                1.14.5
chardet             4.0.0
cloudpickle         1.6.0
colorama            0.4.4
cycler              0.10.0
cython_runtime      NA
cytoolz             0.11.0
dask                2021.03.1
dateutil            2.8.1
decorator           4.4.2
fsspec              0.8.7
get_version         2.1
google              NA
h5py                2.10.0
idna                2.10
igraph              0.8.3
ipykernel           5.5.0
ipython_genutils    0.2.0
jedi                0.18.0
jinja2              2.11.3
joblib              1.0.1
json5               NA
jsonschema          3.2.0
jupyter_server      1.4.1
jupyterlab_server   2.3.0
kiwisolver          1.3.1
legacy_api_wrap     0.0.0
leidenalg           0.8.3
llvmlite            0.34.0
louvain             0.7.0
markupsafe          1.1.1
mpl_toolkits        NA
natsort             7.1.1
nbclassic           NA
nbformat            5.1.2
numba               0.51.2
numexpr             2.7.3
packaging           20.9
parso               0.8.1
pexpect             4.8.0
pickleshare         0.7.5
pkg_resources       NA
prometheus_client   NA
prompt_toolkit      3.0.16
psutil              5.8.0
ptyprocess          0.7.0
pvectorc            NA
pyarrow             0.16.0
pygments            2.8.0
pyparsing           2.4.7
pyrsistent          NA
pytoml              NA
pytz                2021.1
requests            2.25.1
ruamel              NA
scipy               1.6.1
send2trash          NA
setuptools_scm      NA
simplejson          3.17.2
sinfo               0.3.1
six                 1.15.0
sklearn             0.22
sniffio             1.2.0
socks               1.7.1
sphinxcontrib       NA
storemagic          NA
tables              3.6.1
tblib               1.7.0
texttable           1.6.3
tlz                 0.11.0
toolz               0.11.1
tornado             6.1
traitlets           5.0.5
typing_extensions   NA
uritemplate         3.0.1
urllib3             1.26.3
wcwidth             0.2.5
yaml                5.1.2
zmq                 22.0.3
```

Example 5 (unknown):
```unknown
-----
IPython             7.21.0
jupyter_client      6.1.11
jupyter_core        4.7.1
jupyterlab          3.0.9
notebook            6.2.0
-----
Python 3.8.8 | packaged by conda-forge | (default, Feb 20 2021, 16:22:27) [GCC 9.3.0]
Linux-4.15.0-158-generic-x86_64-with-glibc2.10
-----
Session information updated at 2022-12-30 10:13
```

Example 6 (python):
```python
adata = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/QC/scRNA_foetal_prefilter.h5ad')
adata
```

Example 7 (unknown):
```unknown
AnnData object with n_obs × n_vars = 205457 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval'
    var: 'gene_name', 'feature_type', 'ambient_expression-0', 'ambient_expression-1', 'ambient_expression-10', 'ambient_expression-11', 'ambient_expression-12', 'ambient_expression-13', 'ambient_expression-14', 'ambient_expression-15', 'ambient_expression-16', 'ambient_expression-17', 'ambient_expression-18', 'ambient_expression-19', 'ambient_expression-2', 'ambient_expression-3', 'ambient_expression-4', 'ambient_expression-5', 'ambient_expression-6', 'ambient_expression-7', 'ambient_expression-8', 'ambient_expression-9'
    obsm: 'latent_gene_encoding'
```

Example 8 (python):
```python
adata.obs['sangerID'].value_counts()
```

Example 9 (unknown):
```unknown
Sinha_F2             24275
Sinha_E2             24227
BHF_F_Hea11192328    21714
Sinha_E8             17219
BHF_F_Hea11192324    16688
BHF_F_Hea11192326    14117
BHF_F_Hea11192325    10676
Sinha_F4             10652
BHF_F_Hea11192327     9130
BHF_F_Hea10402918     8010
Sinha_F7              6973
Sinha_D8              6714
Sinha_A1              6640
Sinha_B1              5988
BHF_F_Hea11192323     4893
Sinha_C8              4587
BHF_F_Hea10402917     3752
Sinha_G4              3389
Sinha_F8              3203
Sinha_G7              2610
Name: sangerID, dtype: int64
```

Example 10 (python):
```python
adata.obs['scrublet_score'].hist(bins=100)
```

Example 11 (unknown):
```unknown
<AxesSubplot:>
```

Example 12 (python):
```python
# set threshold (arbitrary)
dbl_score_thresh=0.15

prop = np.sum(adata.obs['scrublet_score']>=dbl_score_thresh)/len(adata)
print(f'proportion of doublets: {prop}')
```

Example 13 (unknown):
```unknown
proportion of doublets: 0.15717644081243276
```

Example 14 (python):
```python
# Save doublet ids
doublets = adata.obs_names[(adata.obs['scrublet_score']<dbl_score_thresh)==False]
with open('/nfs/team205/heart/anndata_objects/Foetal/QC/scRNA_rm-doublets.txt', 'w') as f:
    for line in doublets:
        f.write(line)
        f.write('\n')

# Remove doublets
adata = adata[adata.obs['scrublet_score']<dbl_score_thresh]
print(adata.shape)
```

Example 15 (unknown):
```unknown
(173164, 36601)
```

Example 16 (python):
```python
sc.pp.filter_cells(adata, min_genes=500, inplace=True)
sc.pp.filter_cells(adata, min_counts=1000, inplace=True)
print(adata.shape)
```

Example 17 (unknown):
```unknown
filtered out 56470 cells that have less than 500 genes expressed
Trying to set attribute `.obs` of view, copying.
filtered out 1964 cells that have less than 1000 counts
```

Example 18 (unknown):
```unknown
(114730, 36601)
```

Example 19 (python):
```python
adata.obs['sangerID'].value_counts()
```

Example 20 (unknown):
```unknown
BHF_F_Hea11192328    12134
BHF_F_Hea11192324    10697
BHF_F_Hea11192326     9926
Sinha_E2              8388
Sinha_F4              8149
Sinha_E8              8089
BHF_F_Hea11192325     7532
Sinha_F2              6849
BHF_F_Hea11192327     5895
Sinha_D8              5213
Sinha_F7              4983
BHF_F_Hea10402918     4402
Sinha_A1              4202
Sinha_C8              3745
BHF_F_Hea11192323     3533
Sinha_B1              3361
Sinha_G4              2404
BHF_F_Hea10402917     2238
Sinha_G7              1660
Sinha_F8              1330
Name: sangerID, dtype: int64
```

Example 21 (python):
```python
adata.var['mt'] = adata.var['gene_name'].str.startswith('MT-')  # annotate the group of mitochondrial genes as 'mt'
adata.var['ribo'] = adata.var['gene_name'].str.startswith('RPS','RPL')  # annotate the group of ribosomal genes as 'mt'
sc.pp.calculate_qc_metrics(adata, qc_vars=['mt','ribo'], percent_top=None, log1p=True, inplace=True)
```

Example 22 (python):
```python
plt.rcParams['figure.figsize'] = [3, 2]

print('### Sanger ###')
fig,axs=plt.subplots(nrows=1,ncols=3,figsize=(10,3))
sc.pl.scatter(adata[adata.obs['facility']=='Sanger'], x='total_counts', y='n_genes', show=False, ax=axs[0])
sc.pl.scatter(adata[adata.obs['facility']=='Sanger'], x='total_counts', y='pct_counts_mt', show=False, ax=axs[1])
sc.pl.scatter(adata[adata.obs['facility']=='Sanger'], x='total_counts', y='pct_counts_ribo', show=False, ax=axs[2])

fig.tight_layout()
plt.show()

plt.rcParams['figure.figsize'] = [3, 2]

print('### CSCI ###')
fig,axs=plt.subplots(nrows=1,ncols=3,figsize=(10,3))
sc.pl.scatter(adata[adata.obs['facility']=='CSCI'], x='total_counts', y='n_genes', show=False, ax=axs[0])
sc.pl.scatter(adata[adata.obs['facility']=='CSCI'], x='total_counts', y='pct_counts_mt', show=False, ax=axs[1])
sc.pl.scatter(adata[adata.obs['facility']=='CSCI'], x='total_counts', y='pct_counts_ribo', show=False, ax=axs[2])

fig.tight_layout()
plt.show()
```

Example 23 (unknown):
```unknown
### Sanger ###
```

Example 24 (unknown):
```unknown
### CSCI ###
```

Example 25 (python):
```python
plt.rcParams['figure.figsize'] = [2, 2]
adata = adata[adata.obs['pct_counts_mt'] < 20, :]
adata = adata[adata.obs['pct_counts_ribo'] < 20, :]
sc.pl.violin(adata, ['n_genes', 'total_counts','log1p_total_counts', 'pct_counts_mt', 'pct_counts_ribo'],jitter = 0.2, multi_panel = True)
```

Example 26 (python):
```python
sc.settings.set_figure_params(dpi = 120, color_map = 'RdPu')
plt.rcParams['figure.figsize'] = [8, 2]
sc.pl.violin(adata,keys=['log1p_total_counts'],groupby='sangerID',rotation=90,jitter=False)
```

Example 27 (unknown):
```unknown
Trying to set attribute `.uns` of view, copying.
```

Example 28 (python):
```python
print(adata.shape)
adata.obs['sangerID'].value_counts()
```

Example 29 (unknown):
```unknown
(113279, 36601)
```

Example 30 (unknown):
```unknown
BHF_F_Hea11192328    12086
BHF_F_Hea11192324    10568
BHF_F_Hea11192326     9896
Sinha_E2              8327
Sinha_E8              8079
Sinha_F4              8052
BHF_F_Hea11192325     7515
Sinha_F2              6807
BHF_F_Hea11192327     5869
Sinha_D8              4987
Sinha_F7              4837
BHF_F_Hea10402918     4371
Sinha_A1              4118
Sinha_C8              3551
BHF_F_Hea11192323     3420
Sinha_B1              3293
Sinha_G4              2352
BHF_F_Hea10402917     2232
Sinha_G7              1623
Sinha_F8              1296
Name: sangerID, dtype: int64
```

Example 31 (python):
```python
adata_c = adata.copy()

adata_c.var.set_index('gene_name',inplace=True)
adata_c.var_names = adata_c.var_names.astype('str')
adata_c.var_names_make_unique()

sc.pp.filter_genes(adata_c, min_cells=3)
sc.pp.normalize_total(adata_c, target_sum=1e4)
sc.pp.log1p(adata_c)
```

Example 32 (unknown):
```unknown
filtered out 5479 genes that are detected in less than 3 cells
normalizing counts per cell
    finished (0:00:01)
```

Example 33 (python):
```python
# scoring hemoglobin genes
# HGNC, Hemoglobin subunits, groupID:940
genes=['HBA1','HBA2','HBB','HBD','HBE1','HBG1','HBG2','HBM','HBQ1','HBZ']
sc.tl.score_genes(adata_c,gene_list=genes,score_name='HB_score')
```

Example 34 (python):
```python
computing score 'HB_score'
    finished: added
    'HB_score', score of gene set (adata.obs).
    199 total control genes are used. (0:00:04)
```

Example 35 (python):
```python
len(adata_c.obs['sangerID'].unique())
```

Example 36 (python):
```python
sc.settings.set_figure_params(dpi = 80, color_map = 'RdPu', dpi_save = 180, vector_friendly = True, format = 'svg')

fig,axs=plt.subplots(nrows=4,ncols=5,figsize=(15,7))

for i,ID in enumerate(adata_c.obs['sangerID'].unique()):
    sample = adata_c[adata_c.obs['sangerID']==ID]
    axs[i//5,i%5].hist(sample.obs['HB_score'],bins=100)
    axs[i//5,i%5].set_title(ID)

fig.tight_layout()
plt.show()
```

Example 37 (python):
```python
# add HB score to original adata
adata.obs.loc[adata_c.obs_names,'HB_score']=adata_c.obs['HB_score']
```

Example 38 (python):
```python
HBscore_thresh=2

adata=adata[adata.obs['HB_score']<HBscore_thresh]
print(adata.shape)
```

Example 39 (unknown):
```unknown
(100463, 36601)
```

Example 40 (python):
```python
# save
adata.write('/nfs/team205/heart/anndata_objects/Foetal/QC/scRNA_foetal.h5ad')
adata
```

Example 41 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 100463 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score'
    var: 'gene_name', 'feature_type', 'ambient_expression-0', 'ambient_expression-1', 'ambient_expression-10', 'ambient_expression-11', 'ambient_expression-12', 'ambient_expression-13', 'ambient_expression-14', 'ambient_expression-15', 'ambient_expression-16', 'ambient_expression-17', 'ambient_expression-18', 'ambient_expression-19', 'ambient_expression-2', 'ambient_expression-3', 'ambient_expression-4', 'ambient_expression-5', 'ambient_expression-6', 'ambient_expression-7', 'ambient_expression-8', 'ambient_expression-9', 'mt', 'ribo', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts'
    uns: 'sangerID_colors'
    obsm: 'latent_gene_encoding'
```

Example 42 (python):
```python
adata.obs['sangerID'].value_counts()
```

Example 43 (unknown):
```unknown
BHF_F_Hea11192328    12056
BHF_F_Hea11192324    10553
BHF_F_Hea11192326     9876
Sinha_F4              7621
BHF_F_Hea11192325     7512
Sinha_E8              7511
BHF_F_Hea11192327     5865
Sinha_F7              4545
Sinha_F2              4495
Sinha_D8              4377
BHF_F_Hea10402918     4340
Sinha_E2              3913
BHF_F_Hea11192323     3418
Sinha_C8              2989
Sinha_A1              2604
BHF_F_Hea10402917     2227
Sinha_B1              2145
Sinha_G4              2030
Sinha_G7              1372
Sinha_F8              1014
Name: sangerID, dtype: int64
```

---

## Read in adata¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/Xenium/no2_integration_and_cell-type-annotation/no8_plotting_for-Fig.html

**Contents:**
- Read in adata¶
- Log-normalisation¶
- Plot cell types¶
- UMAP¶
- TissueTypist niches¶

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os

import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42
```

Example 2 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 3 (unknown):
```unknown
os.getcwd()
```

Example 4 (unknown):
```unknown
'/rfs/project/rfs-iCNyzSAaucw/kk837/notebooks/Foetal/Xenium/5K/C194_Hst45/CamU'
```

Example 5 (unknown):
```unknown
figdir = '/rfs/project/rfs-iCNyzSAaucw/kk837/notebooks/Foetal/fetal_ms_figures'
# table_dir = '/nfs/team205/heart/fetal_ms_figures/tables/Aug2024'
```

Example 6 (unknown):
```unknown
celltype_key = 'fine_grain_12Jun2025'
```

Example 7 (python):
```python
objects_dir = '/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium'
path_adata = f'{objects_dir}/C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat_5K_filtered_raw.h5ad'
adata = sc.read_h5ad(path_adata)
adata
```

Example 8 (unknown):
```unknown
AnnData object with n_obs × n_vars = 362277 × 5001
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_3.0', 'multi_celltypes_coarse', 'fine_grain_12Jun2025', 'sliding_window_assignment', 'window_col', 'window_row', 'globalX', 'globalY', 'tt_prediction'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_mid_fin_colors', 'dendrogram_pca_harmony_leiden_1.0', 'fine_grain_12Jun2025_colors', 'multi_celltypes_coarse_colors', 'pca', 'pca_harmony', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.2_colors', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.3_colors', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.4_colors', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.5_colors', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_0.8_colors', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.0_colors', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_1.5_colors', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_2.0_colors', 'pca_harmony_leiden_3.0', 'pca_harmony_leiden_3.0_colors', 'tissue_block_id_colors', 'umap'
    obsm: 'X_pca', 'X_pca_harmony', 'X_umap_pca_harmony', 'spatial'
    varm: 'PCs'
    obsp: 'pca_harmony_connectivities', 'pca_harmony_distances'
```

Example 9 (unknown):
```unknown
os.getcwd()
```

Example 10 (unknown):
```unknown
'/rfs/project/rfs-iCNyzSAaucw/kk837/notebooks/Foetal/Xenium/5K/C194_Hst45/CamU'
```

Example 11 (python):
```python
adata.X.data[:5]
```

Example 12 (unknown):
```unknown
array([5., 1., 1., 2., 2.], dtype=float32)
```

Example 13 (python):
```python
# celltype name mapping for plotting
df = pd.read_csv('/rfs/project/rfs-iCNyzSAaucw/kk837/notebooks/Foetal/finegrain_name_mapping.csv')
celltype_mapping = df.set_index('Full_name')['Short_name_finalised'].to_dict()
adata.obs.replace({celltype_key:celltype_mapping},inplace=True)
```

Example 14 (python):
```python
/tmp/ipykernel_968450/407166641.py:4: FutureWarning: The behavior of Series.replace (and DataFrame.replace) with CategoricalDtype is deprecated. In a future version, replace will only be used for cases that preserve the categories. To change the categories, use ser.cat.rename_categories instead.
  adata.obs.replace({celltype_key:celltype_mapping},inplace=True)
```

Example 15 (python):
```python
# log-normalise the count
sc.pp.normalize_total(adata,target_sum=1e4)
sc.pp.log1p(adata)
adata.X.data[:5]
```

Example 16 (unknown):
```unknown
array([4.2093534, 2.6576407, 2.6576407, 3.3151019, 3.3151019],
      dtype=float32)
```

Example 17 (python):
```python
set(adata.obs[celltype_key])
```

Example 18 (unknown):
```unknown
{'?EpendymalCells',
 '?EpithelialCells',
 '?SkeletalMuscle',
 '?ThyroidCells',
 'AVNPC',
 'AtrialCardiomyocytes_DoubletWithEC',
 'AtrialCardiomyocytes_DoubletWithFB',
 'B',
 'ChromC',
 'CorArtEC',
 'CorCapEC',
 'CorCapECCyc',
 'CorCapECN',
 'CorPeri',
 'CorSMC',
 'CorVenEC',
 'CoronaryCapillaryEndothelialCells_DoubletsWithEryth',
 'DASMC',
 'DC',
 'ELNeuro',
 'EPDC',
 'EndoCush',
 'Endocard',
 'EndocardCyc',
 'EndocardialCells_Doublets',
 'Eryth',
 'ErythCyc',
 'FB',
 'FBCOL2A1',
 'FBCyc',
 'GVArtEC',
 'GVSMC',
 'GVSMCCyc',
 'GVVenEC',
 'ILC',
 'LymphEC',
 'MPLYVE1',
 'MacCX3',
 'MastC',
 'MegaK',
 'MesEpiC',
 'Mono',
 'MyeloidCells_Doublets',
 'NeuroPre',
 'ParaN',
 'PericCyc',
 'PericFib',
 'PericInt',
 'SANPC',
 'SOX2_LAMB3_pos',
 'SchwC',
 'SchwCyc',
 'SchwP',
 'SchwannCells_DoubletsWithNeurons',
 'SympN',
 'TCD4',
 'TCD8',
 'ThyroidEpithelialCells',
 'VCSDist',
 'ValveEC',
 'aCMCyc',
 'aCML',
 'aCMR',
 'doublets',
 'unassigned',
 'unclassified',
 'vCM',
 'vCMCyc'}
```

Example 19 (python):
```python
tissue_id = 'C194-HEA-0-FFPE-1'
slide = adata[adata.obs['tissue_block_id']==tissue_id]

sc.pl.spatial(slide,color=[celltype_key],
              spot_size=15,vmax='p100',
              crop_coord=(4600,5600,4250,5250)
             )
```

Example 20 (python):
```python
/tmp/ipykernel_3227243/3010556783.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=[celltype_col],
```

Example 21 (python):
```python
tissue_id = 'C194-HEA-0-FFPE-1'
slide = adata[adata.obs['tissue_block_id']==tissue_id]

celltypes_list = ['AVNPC', 'CorCapEC', 'CorPeri', 'DASMC', 'EPDC', 'EndoCush', 'Endocard',
       'Eryth', 'FB', 'GVSMC', 'GVSMCT', 'GVVenEC', 'LymphEC', 'MPLYVE1',
       'MacCX3', 'MesEpiC', 'MoDC', 'ParaN', 'PericFib', 'SANPC', 'SchwC',
       'SchwNeuro', 'ValveEC', 'ValveIC', 'aCML', 'aCMR']
color_list = ['#ffff00',
 '#1ce6ff',
 '#ff34ff',
 '#a30059',
 '#ffdbe5',
 '#63ffac',
 '#b79762',
 '#004d43',
 '#8fb0ff',
 '#997d87',
 '#5a0007',
 'red',
 '#4a3b53',
 '#ff2f80',
 '#ba0900',
 '#6b7900',
 '#00c2a0',
 'blue',
 '#b903aa',
 (0.5, 0.0, 0.5, 0.7),
 '#7b4f4b',
 '#013349',
 '#00846f',
 '#372101',
 '#ffb500',
 '#c2ffed',
 '#cc0744',
 '#c0b9b2']

for celltype,color in zip(celltypes_list,color_list):
    # update colors
    if celltype in set(slide.obs[celltype_key]):
        slide.uns[f"{celltype_key}_colors"][list(slide.obs[celltype_key].cat.categories).index(celltype)]=color

celltypes_to_plot = ['SANPC',
                     'CorCapEC',
                     'CorPeri',
                     'GVVenEC',
                     'MesEpiC',
                     'FB',
                     'aCMR',
                     'AVNPC',
                     'ParaN',
                     'SchwP',
                    ]

sc.pl.spatial(slide,color=[celltype_key],groups=celltypes_to_plot,
              spot_size=15,vmax='p100',
              crop_coord=(4600,5600,4250,5250),
              show=False
             )
plt.savefig(f'{figdir}/Xenium5K_euploid_SAN_for-Fig1.pdf',bbox_inches="tight",dpi=300)
```

Example 22 (python):
```python
/tmp/ipykernel_3227243/2717981830.py:54: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=[celltype_key],groups=celltypes_to_plot,
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/plotting/_utils.py:406: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  adata.uns[color_key] = _palette
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/plotting/_tools/scatterplots.py:1148: UserWarning: *c* argument looks like a single numeric RGB or RGBA sequence, which should be avoided as value-mapping will have precedence in case its length matches with *x* & *y*.  Please use the *color* keyword-argument or provide a 2D array with a single row if you intend to specify the same RGB or RGBA value for all points.
  ax.scatter([], [], c=palette[label], label=label)
```

Example 23 (python):
```python
ADATA_PATH = "/rfs/project/rfs-iCNyzSAaucw/kk837/data_objects/Foetal/RNA/Feb28ObjectRaw_finegrain_updated.h5ad"
ad = sc.read_h5ad(ADATA_PATH, backed='r')

# Step 1: Create mapping (already done)
cat = "mid_grain"
mid2colours = dict(zip(ad.obs[cat].cat.categories, ad.uns[f"{cat}_colors"]))

# Step 2: Extract the labels and colors in the correct order
mid_grains = list(mid2colours.keys())
palette = list(mid2colours.values())
num_mid_grains = len(mid_grains)

# Step 3: Plot
plt.figure(figsize=(num_mid_grains * 0.4, 2))  # wide figure
sns.palplot(palette)
plt.xticks(np.arange(num_mid_grains), mid_grains, rotation=90)
plt.title("Mid Grain to Color Mapping")
plt.tight_layout()
plt.show()
```

Example 24 (python):
```python
/tmp/ipykernel_794362/455462603.py:18: UserWarning: Tight layout not applied. The bottom and top margins cannot be made large enough to accommodate all Axes decorations.
  plt.tight_layout()
```

Example 25 (unknown):
```unknown
<Figure size 448x160 with 0 Axes>
```

Example 26 (python):
```python
adata.obs['celltypist_mid_fin'] = pd.Categorical(
    adata.obs['celltypist_mid_fin'],
    categories=ad.obs[cat].cat.categories,
    ordered=True
)
```

Example 27 (python):
```python
latent_space = 'pca_harmony'
sc.pl.embedding(adata,
                    basis=f"X_umap_{latent_space}",
                    color='celltypist_mid_fin',palette=palette,
                show=False
                   )
plt.savefig(f'{figdir}/Xenium5K_UMAP-midgrain_for-Fig1.pdf',bbox_inches="tight",dpi=300)
```

Example 28 (python):
```python
tissue_id = 'C194-HEA-0-FFPE-1'
slide = adata[adata.obs['tissue_block_id']==tissue_id]

sc.pl.spatial(slide,color=['celltypist_mid_fin'],
              spot_size=15,vmax='p100',
              # crop_coord=(4600,5600,4250,5250),
              show=False
             )
# plt.savefig(f'{figdir}/Xenium5K_euploid_SAN_for-Fig1.pdf',bbox_inches="tight",dpi=300)
```

Example 29 (python):
```python
/tmp/ipykernel_794362/1888614849.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['celltypist_mid_fin'],
```

Example 30 (unknown):
```unknown
[<Axes: title={'center': 'celltypist_mid_fin'}, xlabel='spatial1', ylabel='spatial2'>]
```

Example 31 (unknown):
```unknown
AnnotationPalette={ 
    'Atrium':'#eac075',
        'Ventricle - Compact':'#f1acfa',
     'Ventricle - Trabeculated':'#f09951',
        'SAnode':'#8A2BE2',
        'AVnode':'#062b4a',
    'VCS - Proximal':'#FFFF00',
        'VCS - Distal':'#7FFFD4',
        'Epicardium':'#01ccc8',
        'Endocardium - Atrial':'#DC143C',
        'Endocardium - Ventricular':'#f53527',
        'Endocardial cusion':'#ff7e97',
        'Great vessel':'#ff9b6e',
    'Ductus arteriosus':'#8a6800',
        'Coronary vessel':'#206601',
        'Adventitia - Coronary':'#CAFF70',
     'Adventitia - GV':'#005aa0',
        'AV ring':'#e478ff',
    'Valves':'#3300FF',
    'LN':'#9088ff',
}
```

Example 32 (python):
```python
tissue_id = 'C194-HEA-0-FFPE-1'
slide = adata[adata.obs['tissue_block_id']==tissue_id]

sc.pl.spatial(slide,color=['tt_prediction'],
              spot_size=15,vmax='p100',
              # crop_coord=(4600,5600,4250,5250),
              show=False,
              palette=AnnotationPalette
             )
plt.savefig(f'{figdir}/TissueTypist_Xenium5K_whole-heart.pdf',bbox_inches="tight",dpi=300)
```

Example 33 (python):
```python
/tmp/ipykernel_968450/2063101229.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['tt_prediction'],
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/plotting/_utils.py:487: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  adata.uns[value_to_plot + "_colors"] = colors_list
```

Example 34 (python):
```python
sc.pl.spatial(slide,color=['tt_prediction'],
              spot_size=15,vmax='p100',
              crop_coord=(5500,6800,4250,5500),
              show=False,
              palette=AnnotationPalette
             )
plt.savefig(f'{figdir}/TissueTypist_Xenium5K_Atrium-AVN.pdf',bbox_inches="tight",dpi=300)
```

Example 35 (python):
```python
/tmp/ipykernel_968450/2129560865.py:1: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['tt_prediction'],
```

Example 36 (python):
```python
sc.pl.spatial(slide,color=['tt_prediction'],
              spot_size=15,vmax='p100',
              crop_coord=(8000,9500,2500,4000),
              show=False,
              palette=AnnotationPalette
             )
plt.savefig(f'{figdir}/TissueTypist_Xenium5K_Ventricle.pdf',bbox_inches="tight",dpi=300)
```

Example 37 (python):
```python
/tmp/ipykernel_968450/2311424067.py:1: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['tt_prediction'],
```

Example 38 (python):
```python
sc.pl.spatial(slide,color=['tt_prediction'],
              spot_size=15,vmax='p100',
              crop_coord=(3500,5000,1800,3300),
              show=False,
              palette=AnnotationPalette
             )
plt.savefig(f'{figdir}/TissueTypist_Xenium5K_GV.pdf',bbox_inches="tight",dpi=300)
```

Example 39 (python):
```python
/tmp/ipykernel_968450/3875201264.py:1: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['tt_prediction'],
```

---

## Import module¶

**URL:** http://127.0.0.1:9020/notebooks_html/Supplementary_materials/Epigenetic_stability/no2b_ATAC-nhoods_peakVI-embedding.html

**Contents:**
- Import module¶
- Parameters¶
- Differential acundance test (Milo)¶
- Define nhoods: Young, Old, Non-significant¶

**Examples:**

Example 1 (python):
```python
import numpy as np
import pandas as pd 
import scanpy as sc
import matplotlib.pyplot as plt
import seaborn as sns
# import sctk
```

Example 2 (python):
```python
# Configure rpy2 to Use the Correct R Installation:
import os
os.environ['R_HOME'] = '/nfs/team205/kk18/miniconda3/envs/vitro/lib/R'
# check
from rpy2.rinterface_lib import openrlib
print(openrlib.R_HOME)
```

Example 3 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/R
```

Example 4 (python):
```python
import milopy.core as milo
import milopy.plot as milopl
import milopy.utils
```

Example 5 (python):
```python
import session_info
session_info.show()
```

Example 6 (python):
```python
-----
matplotlib          3.7.2
milopy              0.1.1
numpy               1.24.4
pandas              1.5.3
rpy2                3.5.14
scanpy              1.9.3
seaborn             0.12.2
session_info        1.0.0
-----
```

Example 7 (python):
```python
PIL                         10.0.0
anndata                     0.9.2
anyio                       NA
arrow                       1.2.3
asttokens                   NA
attr                        23.1.0
attrs                       23.1.0
babel                       2.12.1
backcall                    0.2.0
backports                   NA
brotli                      1.0.9
certifi                     2023.11.17
cffi                        1.15.1
charset_normalizer          3.2.0
colorama                    0.4.6
comm                        0.1.4
cycler                      0.10.0
cython_runtime              NA
dateutil                    2.8.2
debugpy                     1.6.8
decorator                   5.1.1
defusedxml                  0.7.1
executing                   1.2.0
fastjsonschema              NA
fqdn                        NA
h5py                        3.9.0
idna                        3.4
igraph                      0.10.6
importlib_metadata          NA
importlib_resources         NA
ipykernel                   6.25.1
isoduration                 NA
jedi                        0.19.0
jinja2                      3.1.2
joblib                      1.3.2
json5                       NA
jsonpointer                 2.0
jsonschema                  4.19.0
jsonschema_specifications   NA
jupyter_events              0.7.0
jupyter_server              2.7.1
jupyterlab_server           2.24.0
kiwisolver                  1.4.5
leidenalg                   0.10.1
llvmlite                    0.40.1
markupsafe                  2.1.3
mpl_toolkits                NA
natsort                     8.4.0
nbformat                    5.9.2
numba                       0.57.1
overrides                   NA
packaging                   23.1
parso                       0.8.3
patsy                       0.5.3
pexpect                     4.8.0
pickleshare                 0.7.5
pkg_resources               NA
plotly                      5.16.1
prometheus_client           NA
prompt_toolkit              3.0.39
psutil                      5.9.5
ptyprocess                  0.7.0
pure_eval                   0.2.2
pydev_ipython               NA
pydevconsole                NA
pydevd                      2.9.5
pydevd_file_utils           NA
pydevd_plugins              NA
pydevd_tracing              NA
pygments                    2.16.1
pyparsing                   3.0.9
pythonjsonlogger            NA
pytz                        2023.3
referencing                 NA
requests                    2.31.0
rfc3339_validator           0.1.4
rfc3986_validator           0.1.1
rpds                        NA
ruamel                      NA
scipy                       1.10.1
send2trash                  NA
six                         1.16.0
sklearn                     1.3.0
sniffio                     1.3.0
socks                       1.7.1
sphinxcontrib               NA
stack_data                  0.6.2
statsmodels                 0.14.0
texttable                   1.6.7
threadpoolctl               3.2.0
tornado                     6.3.3
traitlets                   5.9.0
typing_extensions           NA
tzlocal                     NA
uri_template                NA
urllib3                     2.0.4
wcwidth                     0.2.6
webcolors                   1.13
websocket                   1.6.2
yaml                        6.0.1
zipp                        NA
zmq                         25.1.1
zstandard                   0.19.0
```

Example 8 (unknown):
```unknown
-----
IPython             8.12.0
jupyter_client      8.3.0
jupyter_core        4.12.0
jupyterlab          4.0.5
notebook            7.0.2
-----
Python 3.8.17 (default, Jul  5 2023, 21:04:15) [GCC 11.2.0]
Linux-4.15.0-220-generic-x86_64-with-glibc2.17
-----
Session information updated at 2023-12-31 21:01
```

Example 9 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 10 (python):
```python
sc.settings.set_figure_params(dpi=120)
```

Example 11 (unknown):
```unknown
outdir = '/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/Milo_nhoods/mid_grain_mod'
```

Example 12 (unknown):
```unknown
!ls /nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/PeakMatrix/mid_grain_mod
```

Example 13 (unknown):
```unknown
Foetal_AtrialCardiomyocytes_plusCCS_added-cov-region_subsampled-n1000_peakVI-model
Foetal_AtrialCardiomyocytes_plusCCS_Peaks_post-peakVI_added-cov-region_subsampled-n1000.h5ad
Foetal_AtrialCardiomyocytes_plusCCS_Peaks_post-peakVI.h5ad
Foetal_AtrialCardiomyocytes_plusCCS_Peaks_post-peakVI_subsampled-n1000.h5ad
Foetal_AtrialCardiomyocytes_plusCCS_peakVI-model
Foetal_AtrialCardiomyocytes_plusCCS_subsampled-n1000_peakVI-model
Foetal_BloodVesselEndothelialCells_added-cov-region_subsampled-n1000_peakVI-model
Foetal_BloodVesselEndothelialCells_Peaks_post-peakVI_added-cov-region_subsampled-n1000.h5ad
Foetal_BloodVesselEndothelialCells_Peaks_post-peakVI.h5ad
Foetal_BloodVesselEndothelialCells_Peaks_post-peakVI_subsampled-n1000.h5ad
Foetal_BloodVesselEndothelialCells_peakVI-model
Foetal_BloodVesselEndothelialCells_subsampled-n1000_peakVI-model
Foetal_EndocardialCells_added-cov-region_subsampled-n1000_peakVI-model
Foetal_EndocardialCells_Peaks_post-peakVI_added-cov-region_subsampled-n1000.h5ad
Foetal_EndocardialCells_Peaks_post-peakVI.h5ad
Foetal_EndocardialCells_Peaks_post-peakVI_subsampled-n1000.h5ad
Foetal_EndocardialCells_peakVI-model
Foetal_EndocardialCells_subsampled-n1000_peakVI-model
Foetal_EpicardialCells_added-cov-region_subsampled-n1000_peakVI-model
Foetal_EpicardialCells_Peaks_post-peakVI_added-cov-region_subsampled-n1000.h5ad
Foetal_EpicardialCells_Peaks_post-peakVI.h5ad
Foetal_EpicardialCells_Peaks_post-peakVI_subsampled-n1000.h5ad
Foetal_EpicardialCells_peakVI-model
Foetal_EpicardialCells_subsampled-n1000_peakVI-model
Foetal_Fibroblasts_added-cov-region_subsampled-n1000_peakVI-model
Foetal_Fibroblasts_Peaks_post-peakVI_added-cov-region_subsampled-n1000.h5ad
Foetal_Fibroblasts_Peaks_post-peakVI.h5ad
Foetal_Fibroblasts_Peaks_post-peakVI_subsampled-n1000.h5ad
Foetal_Fibroblasts_peakVI-model
Foetal_Fibroblasts_subsampled-n1000_peakVI-model
Foetal_Glia_added-cov-region_subsampled-n1000_peakVI-model
Foetal_Glia_Peaks_post-peakVI_added-cov-region_subsampled-n1000.h5ad
Foetal_Glia_Peaks_post-peakVI.h5ad
Foetal_Glia_Peaks_post-peakVI_subsampled-n1000.h5ad
Foetal_Glia_peakVI-model
Foetal_Glia_subsampled-n1000_peakVI-model
Foetal_LymphaticEndothelialCells_added-cov-region_subsampled-n1000_peakVI-model
Foetal_LymphaticEndothelialCells_Peaks_post-peakVI_added-cov-region_subsampled-n1000.h5ad
Foetal_LymphaticEndothelialCells_Peaks_post-peakVI.h5ad
Foetal_LymphaticEndothelialCells_Peaks_post-peakVI_subsampled-n1000.h5ad
Foetal_LymphaticEndothelialCells_peakVI-model
Foetal_LymphaticEndothelialCells_subsampled-n1000_peakVI-model
Foetal_LymphoidCells_added-cov-region_subsampled-n1000_peakVI-model
Foetal_LymphoidCells_Peaks_post-peakVI_added-cov-region_subsampled-n1000.h5ad
Foetal_LymphoidCells_Peaks_post-peakVI.h5ad
Foetal_LymphoidCells_Peaks_post-peakVI_subsampled-n1000.h5ad
Foetal_LymphoidCells_peakVI-model
Foetal_LymphoidCells_subsampled-n1000_peakVI-model
Foetal_MuralCells_added-cov-region_subsampled-n1000_peakVI-model
Foetal_MuralCells_Peaks_post-peakVI_added-cov-region_subsampled-n1000.h5ad
Foetal_MuralCells_Peaks_post-peakVI.h5ad
Foetal_MuralCells_Peaks_post-peakVI_subsampled-n1000.h5ad
Foetal_MuralCells_peakVI-model
Foetal_MuralCells_subsampled-n1000_peakVI-model
Foetal_MyeloidCells_added-cov-region_subsampled-n1000_peakVI-model
Foetal_MyeloidCells_Peaks_post-peakVI_added-cov-region_subsampled-n1000.h5ad
Foetal_MyeloidCells_Peaks_post-peakVI.h5ad
Foetal_MyeloidCells_Peaks_post-peakVI_subsampled-n1000.h5ad
Foetal_MyeloidCells_peakVI-model
Foetal_MyeloidCells_subsampled-n1000_peakVI-model
Foetal_Neurons_added-cov-region_subsampled-n1000_peakVI-model
Foetal_Neurons_Peaks_post-peakVI_added-cov-region_subsampled-n1000.h5ad
Foetal_Neurons_Peaks_post-peakVI.h5ad
Foetal_Neurons_Peaks_post-peakVI_subsampled-n1000.h5ad
Foetal_Neurons_peakVI-model
Foetal_Neurons_subsampled-n1000_peakVI-model
Foetal_PericardialCells_added-cov-region_subsampled-n1000_peakVI-model
Foetal_PericardialCells_Peaks_post-peakVI_added-cov-region_subsampled-n1000.h5ad
Foetal_PericardialCells_Peaks_post-peakVI.h5ad
Foetal_PericardialCells_Peaks_post-peakVI_subsampled-n1000.h5ad
Foetal_PericardialCells_peakVI-model
Foetal_PericardialCells_subsampled-n1000_peakVI-model
Foetal_VentricularCardiomyocytes_plusCCS_added-cov-region_subsampled-n1000_peakVI-model
Foetal_VentricularCardiomyocytes_plusCCS_Peaks_post-peakVI_added-cov-region_subsampled-n1000.h5ad
Foetal_VentricularCardiomyocytes_plusCCS_Peaks_post-peakVI.h5ad
Foetal_VentricularCardiomyocytes_plusCCS_Peaks_post-peakVI_subsampled-n1000.h5ad
Foetal_VentricularCardiomyocytes_plusCCS_peakVI-model
Foetal_VentricularCardiomyocytes_plusCCS_subsampled-n1000_peakVI-model
```

Example 14 (python):
```python
%%time
adata_dict = {}
for i,celltype in enumerate(['AtrialCardiomyocytes_plusCCS',
                            'VentricularCardiomyocytes_plusCCS',
                             'BloodVesselEndothelialCells',
                             'EndocardialCells',
                            'LymphaticEndothelialCells',
                             'EpicardialCells',
                             'Fibroblasts',
                             'PericardialCells',
                             'MuralCells',
                             'Neurons',
                            'Glia',
                            'MyeloidCells',
                            'LymphoidCells']
                             ):
    print(f'####### {celltype} #######')
    
    adata_dict[celltype] = sc.read_h5ad(f'/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/PeakMatrix/mid_grain_mod/Foetal_{celltype}_Peaks_post-peakVI_added-cov-region_subsampled-n1000.h5ad')
    print(adata_dict[celltype].shape)
    
    # qc
    sc.pp.calculate_qc_metrics(adata_dict[celltype], log1p=True, inplace=True)
    # compute the k-nearest-neighbor graph that is used in both clustering and umap algorithms
    sc.pp.neighbors(adata_dict[celltype], use_rep="X_PeakVI", n_neighbors=100, key_added="peakvi")
    # compute the umap
    sc.tl.umap(adata_dict[celltype], min_dist=0.2, neighbors_key="peakvi")
    # cluster the space (we use a lower resolution to get fewer clusters than the default)
    # sc.tl.leiden(adata_dict[celltype], key_added="cluster_pvi", resolution=1, neighbors_key="peakvi")
    # plot
    sc.pl.umap(adata_dict[celltype], color=['week','donor','fine_grain'], size=20, legend_fontsize=12, wspace=0.3,
              cmap='RdPu')
    
    ## Construct neighbourhoods
    milo.make_nhoods(adata_dict[celltype], neighbors_key="peakvi", prop=0.05)
    ## Count cells in neighbourhoods
    ## Milo leverages the variation in cell numbers between replicates for the same experimental condition to test for differential abundance. 
    milo.count_nhoods(adata_dict[celltype], sample_col="donor")
    # differential abundance testing
    adata_dict[celltype].obs["week_continuous"] = [int(x.split('W')[0]) for x in adata_dict[celltype].obs["week"]] #rather than adata.obs["week"].cat.codes
    milo.DA_nhoods(adata_dict[celltype], design="~ week_continuous")
    milopy.utils.build_nhood_graph(adata_dict[celltype],basis='X_umap')
    # plot
    milopl.plot_nhood_graph(adata_dict[celltype], 
                        alpha=0.1, ## SpatialFDR level (1%) 
                        min_size=3, ## Size of smallest dot
                        title = 'DA log-Fold Change\n(FDR < 0.1)'
                       )
    # store logFC
    logfcs = abs(adata_dict[celltype].uns['nhood_adata'].obs['logFC'])
    df = pd.DataFrame({'Celltype':celltype,'logFC':logfcs})
    if i==0:
        logfc_df=df.copy()
    else:
        logfc_df=pd.concat([logfc_df,df])

# save adata dictionary
utils.save_pkl(adata_dict, f'{outdir}/added-cov-region_subsampled-n1000_post_milo_adata_dictionary.pkl')
```

Example 15 (unknown):
```unknown
####### AtrialCardiomyocytes_plusCCS #######
(4381, 508040)
```

Example 16 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 17 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/anndata/_core/anndata.py:121: ImplicitModificationWarning: Transforming to str index.
  warnings.warn("Transforming to str index.", ImplicitModificationWarning)
```

Example 18 (unknown):
```unknown
####### VentricularCardiomyocytes_plusCCS #######
(6531, 508040)
```

Example 19 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 20 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/anndata/_core/anndata.py:121: ImplicitModificationWarning: Transforming to str index.
  warnings.warn("Transforming to str index.", ImplicitModificationWarning)
```

Example 21 (unknown):
```unknown
####### BloodVesselEndothelialCells #######
(3028, 508040)
```

Example 22 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 23 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/anndata/_core/anndata.py:121: ImplicitModificationWarning: Transforming to str index.
  warnings.warn("Transforming to str index.", ImplicitModificationWarning)
```

Example 24 (unknown):
```unknown
####### EndocardialCells #######
(1410, 508040)
```

Example 25 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 26 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/anndata/_core/anndata.py:121: ImplicitModificationWarning: Transforming to str index.
  warnings.warn("Transforming to str index.", ImplicitModificationWarning)
```

Example 27 (unknown):
```unknown
####### LymphaticEndothelialCells #######
(1000, 508040)
```

Example 28 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 29 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/anndata/_core/anndata.py:121: ImplicitModificationWarning: Transforming to str index.
  warnings.warn("Transforming to str index.", ImplicitModificationWarning)
```

Example 30 (unknown):
```unknown
####### EpicardialCells #######
(1340, 508040)
```

Example 31 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 32 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/anndata/_core/anndata.py:121: ImplicitModificationWarning: Transforming to str index.
  warnings.warn("Transforming to str index.", ImplicitModificationWarning)
```

Example 33 (unknown):
```unknown
####### Fibroblasts #######
(6969, 508040)
```

Example 34 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 35 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/anndata/_core/anndata.py:121: ImplicitModificationWarning: Transforming to str index.
  warnings.warn("Transforming to str index.", ImplicitModificationWarning)
```

Example 36 (unknown):
```unknown
####### PericardialCells #######
(2187, 508040)
```

Example 37 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 38 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/anndata/_core/anndata.py:121: ImplicitModificationWarning: Transforming to str index.
  warnings.warn("Transforming to str index.", ImplicitModificationWarning)
```

Example 39 (unknown):
```unknown
####### MuralCells #######
(3103, 508040)
```

Example 40 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 41 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/anndata/_core/anndata.py:121: ImplicitModificationWarning: Transforming to str index.
  warnings.warn("Transforming to str index.", ImplicitModificationWarning)
```

Example 42 (unknown):
```unknown
####### Neurons #######
(1101, 508040)
```

Example 43 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 44 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/anndata/_core/anndata.py:121: ImplicitModificationWarning: Transforming to str index.
  warnings.warn("Transforming to str index.", ImplicitModificationWarning)
```

Example 45 (unknown):
```unknown
####### Glia #######
(1759, 508040)
```

Example 46 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 47 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/anndata/_core/anndata.py:121: ImplicitModificationWarning: Transforming to str index.
  warnings.warn("Transforming to str index.", ImplicitModificationWarning)
```

Example 48 (unknown):
```unknown
####### MyeloidCells #######
(4321, 508040)
```

Example 49 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 50 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/anndata/_core/anndata.py:121: ImplicitModificationWarning: Transforming to str index.
  warnings.warn("Transforming to str index.", ImplicitModificationWarning)
```

Example 51 (unknown):
```unknown
####### LymphoidCells #######
(7006, 508040)
```

Example 52 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 53 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/anndata/_core/anndata.py:121: ImplicitModificationWarning: Transforming to str index.
  warnings.warn("Transforming to str index.", ImplicitModificationWarning)
```

Example 54 (unknown):
```unknown
CPU times: user 4min 33s, sys: 11.4 s, total: 4min 45s
Wall time: 5min 8s
```

Example 55 (python):
```python
adata_dict.keys()
```

Example 56 (unknown):
```unknown
dict_keys(['AtrialCardiomyocytes_plusCCS', 'VentricularCardiomyocytes_plusCCS', 'BloodVesselEndothelialCells', 'EndocardialCells', 'LymphaticEndothelialCells', 'EpicardialCells', 'Fibroblasts', 'PericardialCells', 'MuralCells', 'Neurons', 'Glia', 'MyeloidCells', 'LymphoidCells'])
```

Example 57 (python):
```python
for celltype in adata_dict.keys():
    print(celltype)
    # get adata
    ad = adata_dict[celltype]
    # define nhoods based on the enrichment
    nhoods_res = ad.uns['nhood_adata'].obs.copy()
    nhoods_res['nhood'] = [f'nhood_{x+1}' for x in range(len(nhoods_res))]
    nhoods_res.set_index('nhood',inplace=True)
    nhoods_res['enrichment'] = 'nonsignificant'
    old_inds = nhoods_res.index[(nhoods_res['logFC']>0)&(nhoods_res['FDR']<0.1)]
    nhoods_res.loc[old_inds,'enrichment'] = 'old'
    young_inds = nhoods_res.index[(nhoods_res['logFC']<0)&(nhoods_res['FDR']<0.1)]
    nhoods_res.loc[young_inds,'enrichment'] = 'young'
    print(f'old nhoods: {len(old_inds)}')
    print(f'young nhoods: {len(young_inds)}')
    # get cells for each nhoods
    nhood_df = pd.DataFrame(ad.obsm['nhoods'].todense())
    nhood_df.index = ad.obs_names
    nhood_df.columns = nhoods_res.index
    nhoods_res['cells'] = [','.join(nhood_df.index[nhood_df[x]==1]) for x in nhood_df.columns]
    # save
    nhoods_res.to_csv(f'{outdir}/{celltype}_added-cov-region_subsampled-n1000_milo_nhoods_result.csv')
    
    nhoods_res['Nhood_size'].hist(bins=30)
    plt.title(celltype)
    plt.show()
```

Example 58 (unknown):
```unknown
AtrialCardiomyocytes_plusCCS
old nhoods: 79
young nhoods: 15
```

Example 59 (unknown):
```unknown
VentricularCardiomyocytes_plusCCS
old nhoods: 166
young nhoods: 26
```

Example 60 (unknown):
```unknown
BloodVesselEndothelialCells
old nhoods: 49
young nhoods: 0
```

Example 61 (unknown):
```unknown
EndocardialCells
old nhoods: 16
young nhoods: 8
```

Example 62 (unknown):
```unknown
LymphaticEndothelialCells
old nhoods: 17
young nhoods: 0
```

Example 63 (unknown):
```unknown
EpicardialCells
old nhoods: 24
young nhoods: 9
```

Example 64 (unknown):
```unknown
Fibroblasts
old nhoods: 197
young nhoods: 33
```

Example 65 (unknown):
```unknown
PericardialCells
old nhoods: 56
young nhoods: 3
```

Example 66 (unknown):
```unknown
MuralCells
old nhoods: 61
young nhoods: 5
```

Example 67 (unknown):
```unknown
Neurons
old nhoods: 6
young nhoods: 15
```

Example 68 (unknown):
```unknown
Glia
old nhoods: 22
young nhoods: 19
```

Example 69 (unknown):
```unknown
MyeloidCells
old nhoods: 70
young nhoods: 8
```

Example 70 (unknown):
```unknown
LymphoidCells
old nhoods: 256
young nhoods: 0
```

Example 71 (python):
```python
# adata_dict = utils.read_pkl(f'{outdir}/post_milo_adata_dictionary.pkl')
```

Example 72 (python):
```python
for celltype in adata_dict.keys():
    print(celltype)
    # get adata
    ad = adata_dict[celltype]
    # get nhoods
    nhoods_res = ad.uns['nhood_adata'].obs.copy()
    nhoods_no = [f'nhood_{x+1}' for x in range(len(nhoods_res))]
    # prepare dataframe: cell-by-nhoods, boolean
    df = pd.DataFrame(ad.obsm['nhoods'].todense(), columns=nhoods_no, index=ad.obs_names)
    # save
    df.to_csv(f'{outdir}/{celltype}_added-cov-region_subsampled-n1000_milo_cell-by-nhood.csv')
```

Example 73 (unknown):
```unknown
AtrialCardiomyocytes_plusCCS
VentricularCardiomyocytes_plusCCS
BloodVesselEndothelialCells
EndocardialCells
LymphaticEndothelialCells
EpicardialCells
Fibroblasts
PericardialCells
MuralCells
Neurons
Glia
MyeloidCells
LymphoidCells
```

Example 74 (python):
```python
midfix = '_added-cov-region_subsampled-n1000'
suffix = '_milo_nhoods_result.csv'

nhoods_res = {}
for file in os.listdir(outdir):
    if (midfix in file)&(suffix in file):
        celltype = file.replace(midfix,'').replace(suffix,'')
        print(celltype)
        nhoods_res[celltype] = pd.read_csv(f'{outdir}/{celltype}{midfix}{suffix}',
                                           index_col=0
                                          )
```

Example 75 (python):
```python
celltypes = [
    'AtrialCardiomyocytes_plusCCS',
    'VentricularCardiomyocytes_plusCCS',
    'Fibroblasts',
    'MuralCells',
    'PericardialCells', 
    'EpicardialCells',
    'BloodVesselEndothelialCells',
    'EndocardialCells', 
     'Neurons',
    'Glia', 
     'MyeloidCells',
    'LymphoidCells',   
             ]

for celltype in celltypes:
    nhoods_res[celltype]['Nhood_size'].hist(bins=30)
    plt.title(celltype)
    plt.show()
```

---

## Functions¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig4_Cardiomyocytes/AtrialCardiomyocytes/Visium_mapping/c2l_pt2_mappings_FFPE.html

**Contents:**
  - Loading packages¶
- Functions¶
- Read in metadata¶
- Data, parameters¶
- Loading visium data¶
- Filter out low quality spots¶
- Remove mitochondrial genes¶
- Add metadata and annotation¶
- Remove spots based on manual annotation¶
- Combine pcw13w-Atrial and pcw13w-Ventricle data to have whole heart in one sample¶
- Run per group: cell2location mapping¶

**Examples:**

Example 1 (unknown):
```unknown
run = "20Mar2024_fetal_FFPE"
```

Example 2 (python):
```python
import sys
import scanpy as sc
import anndata
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt 
import matplotlib as mpl

import cell2location
import scvi

from matplotlib import rcParams
rcParams['pdf.fonttype'] = 42 # enables correct plotting of text
import seaborn as sns
```

Example 3 (python):
```python
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm
Global seed set to 0
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/umap/distances.py:1063: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/umap/distances.py:1071: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/umap/distances.py:1086: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/umap/umap_.py:660: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
```

Example 4 (python):
```python
import gc
import torch
```

Example 5 (python):
```python
def read_and_qc(sample_name, path):
    r""" This function reads the data for one 10X spatial experiment into the anndata object.
    It also calculates QC metrics. Modify this function if required by your workflow.

    :param sample_name: Name of the sample
    :param path: path to data
    """

    adata = sc.read_visium(path + str(sample_name),
                           count_file='filtered_feature_bc_matrix.h5', load_images=True)
    adata.var_names_make_unique(join='-')
    
    adata.obs['sample'] = sample_name
    
    # rename genes to ENSEMBL
    adata.var['SYMBOL'] = adata.var_names
    adata.var_names = adata.var['gene_ids']
    adata.var_names.name = None

    # Calculate QC metrics
    sc.pp.calculate_qc_metrics(adata, inplace=True)
    adata.var['mt'] = [gene.startswith('MT-') for gene in adata.var['SYMBOL']]
    adata.obs['mt_frac'] = adata[:, adata.var['mt'].tolist()].X.sum(1).A.squeeze()/adata.obs['total_counts']

    # add sample name to obs names
    adata.obs["sample"] = [str(i) for i in adata.obs['sample']]
    adata.obs_names = adata.obs["sample"] \
                          + '_' + adata.obs_names
    adata.obs.index.name = 'spot_id'

    return adata

def select_slide(adata, s, s_col='sample'):
    r""" This function selects the data for one slide from the spatial anndata object.

    :param adata: Anndata object with multiple spatial experiments
    :param s: name of selected experiment
    :param s_col: column in adata.obs listing experiment name for each location
    """

    slide = adata[adata.obs[s_col].isin([s]), :]
    s_keys = list(slide.uns['spatial'].keys())
    s_spatial = np.array(s_keys)[[s in k for k in s_keys]][0]

    slide.uns['spatial'] = {s_spatial: slide.uns['spatial'][s_spatial]}

    return slide
```

Example 6 (python):
```python
meta=pd.read_csv('/nfs/team205/heart/anndata_objects/Foetal/metadata/HeartTeamSamples_Mappeddata_20230208_for-cell2loc.csv',
                index_col=0)
meta
```

Example 7 (python):
```python
pd.crosstab(meta['donor'],meta['cell2loc-map_group'])
```

Example 8 (unknown):
```unknown
# select slides
# Healthy, FFPE
meta = meta[meta['cell2loc-map_group'].isin(['FFPE_16PCW'])]

sampleIDs = list(meta['sangerID'])
len(sampleIDs)
```

Example 9 (python):
```python
import os
results_folder = os.path.join(os.getcwd(),"analysis/Visium/cell2location_results")

ref_run_name = f'{results_folder}/reference_signatures/{run}'
run_name = f'{results_folder}/cell2location_map/{run}'
```

Example 10 (unknown):
```unknown
visium_path='/nfs/team205/heart/mapped/spaceranger200/'

celltype_key='fine_grain_modified'
```

Example 11 (python):
```python
# Read the data of the interested region into anndata objects
slides = []
for i in sampleIDs:
    slides.append(read_and_qc(i, path=visium_path))

# Combine anndata objects together
adata_vis = slides[0].concatenate(
    slides[1:],
    batch_key="sample",
    uns_merge="unique",
    batch_categories=sampleIDs,
    index_unique=None
)
```

Example 12 (python):
```python
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/_core/anndata.py:1830: UserWarning: Variable names are not unique. To make them unique, call `.var_names_make_unique`.
  utils.warn_names_duplicates("var")
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/_core/anndata.py:1830: UserWarning: Variable names are not unique. To make them unique, call `.var_names_make_unique`.
  utils.warn_names_duplicates("var")
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/_core/anndata.py:1785: FutureWarning: X.dtype being converted to np.float32 from float64. In the next version of anndata (0.9) conversion will not be automatic. Pass dtype explicitly to avoid this warning. Pass `AnnData(X, dtype=X.dtype, ...)` to get the future behavour.
  [AnnData(sparse.csr_matrix(a.shape), obs=a.obs) for a in all_adatas],
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/_core/anndata.py:1785: FutureWarning: X.dtype being converted to np.float32 from float64. In the next version of anndata (0.9) conversion will not be automatic. Pass dtype explicitly to avoid this warning. Pass `AnnData(X, dtype=X.dtype, ...)` to get the future behavour.
  [AnnData(sparse.csr_matrix(a.shape), obs=a.obs) for a in all_adatas],
```

Example 13 (python):
```python
# PLOT QC FOR EACH SAMPLE
fig, axs = plt.subplots(len(slides), 4, figsize=(15, 4*len(slides)-4))
for i, s in enumerate(adata_vis.obs['sample'].unique()):
    #fig.suptitle('Covariates for filtering')

    slide = select_slide(adata_vis, s)
    sns.distplot(slide.obs['total_counts'],
                 kde=False, ax = axs[i, 0])
    axs[i, 0].set_xlim(0, adata_vis.obs['total_counts'].max())
    axs[i, 0].set_xlabel(f'total_counts | {s}')

    sns.distplot(slide.obs['total_counts']\
                 [slide.obs['total_counts']<2000],
                 kde=False, bins=40, ax = axs[i, 1])
    axs[i, 1].set_xlim(0, 2000)
    axs[i, 1].set_xlabel(f'total_counts | {s}')

    sns.distplot(slide.obs['n_genes_by_counts'],
                 kde=False, bins=60, ax = axs[i, 2])
    axs[i, 2].set_xlim(0, adata_vis.obs['n_genes_by_counts'].max())
    axs[i, 2].set_xlabel(f'n_genes_by_counts | {s}')

    sns.distplot(slide.obs['n_genes_by_counts']\
                 [slide.obs['n_genes_by_counts']<1000],
                 kde=False, bins=60, ax = axs[i, 3])
    axs[i, 3].set_xlim(0, 1000)
    axs[i, 3].set_xlabel(f'n_genes_by_counts | {s}')

plt.tight_layout()
```

Example 14 (python):
```python
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
/tmp/ipykernel_1547584/2051608433.py:7: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['total_counts'],
/tmp/ipykernel_1547584/2051608433.py:12: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['total_counts']\
/tmp/ipykernel_1547584/2051608433.py:18: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['n_genes_by_counts'],
/tmp/ipykernel_1547584/2051608433.py:23: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['n_genes_by_counts']\
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
/tmp/ipykernel_1547584/2051608433.py:7: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['total_counts'],
/tmp/ipykernel_1547584/2051608433.py:12: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['total_counts']\
/tmp/ipykernel_1547584/2051608433.py:18: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['n_genes_by_counts'],
/tmp/ipykernel_1547584/2051608433.py:23: UserWarning: 

`distplot` is a deprecated function and will be removed in seaborn v0.14.0.

Please adapt your code to use either `displot` (a figure-level function with
similar flexibility) or `histplot` (an axes-level function for histograms).

For a guide to updating your code to use the new functions, please see
https://gist.github.com/mwaskom/de44147ed2974457ad6372750bbe5751

  sns.distplot(slide.obs['n_genes_by_counts']\
```

Example 15 (python):
```python
# filter out low quality spots
sc.pp.filter_cells(adata_vis, min_counts = 500)
sc.pp.filter_cells(adata_vis, min_genes = 300)

# Spot number per slide
adata_vis.obs['sample'].value_counts()
```

Example 16 (unknown):
```unknown
BHF_F_Hea13295830    12280
BHF_F_Hea13295831     8899
Name: sample, dtype: int64
```

Example 17 (python):
```python
# find mitochondria-encoded (MT) genes
adata_vis.var['MT_gene'] = [gene.startswith('MT-') for gene in adata_vis.var['SYMBOL']]
print(sum(adata_vis.var['MT_gene']))
    
# remove MT genes for spatial mapping (keeping their counts in the object)
adata_vis.obsm['MT'] = adata_vis[:, adata_vis.var['MT_gene'].values].X.toarray()
adata_vis = adata_vis[:, ~adata_vis.var['MT_gene'].values]
print(sum(adata_vis.var['MT_gene']))
```

Example 18 (unknown):
```unknown
meta.head()
```

Example 19 (python):
```python
adata_vis.obs = adata_vis.obs.reset_index().merge(meta, how='left', left_on='sample', right_on='sangerID').set_index('spot_id')
adata_vis
```

Example 20 (unknown):
```unknown
AnnData object with n_obs × n_vars = 21179 × 18074
    obs: 'in_tissue', 'array_row', 'array_col', 'sample', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'mt_frac', 'n_counts', 'n_genes', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'status', 'cell2loc-map_group'
    var: 'gene_ids', 'feature_types', 'genome', 'SYMBOL', 'mt', 'n_cells_by_counts-BHF_F_Hea13295830', 'mean_counts-BHF_F_Hea13295830', 'log1p_mean_counts-BHF_F_Hea13295830', 'pct_dropout_by_counts-BHF_F_Hea13295830', 'total_counts-BHF_F_Hea13295830', 'log1p_total_counts-BHF_F_Hea13295830', 'n_cells_by_counts-BHF_F_Hea13295831', 'mean_counts-BHF_F_Hea13295831', 'log1p_mean_counts-BHF_F_Hea13295831', 'pct_dropout_by_counts-BHF_F_Hea13295831', 'total_counts-BHF_F_Hea13295831', 'log1p_total_counts-BHF_F_Hea13295831', 'MT_gene'
    uns: 'spatial'
    obsm: 'spatial', 'MT'
```

Example 21 (python):
```python
adata_vis.obs.head()
```

Example 22 (python):
```python
# annotation
path_to_annot='/nfs/team205/heart/visium_annotations/foetal'
category_name='JC_edit'

# read in annotation csv file and concatenate
for i, ID in enumerate(sampleIDs):
    if f'{ID}_JC.csv' in os.listdir(path_to_annot):
    
        print(ID)
        annot_df = pd.read_csv(f'{path_to_annot}/{ID}_JC.csv')
        annot_df['spot_id']=ID+'_'+annot_df['Barcode']

        if i==0:
            annot_df_all=annot_df.copy()
        else:
            annot_df_all=pd.concat([annot_df_all,annot_df])
    else:
        print(f'{ID} is not annotated')

# add annotations to adata_vis
adata_vis.obs=pd.concat([adata_vis.obs,
                     annot_df_all.set_index('spot_id')[[category_name]].reindex(adata_vis.obs.index)],
                     axis=1)
# replace nan to 'none'
adata_vis.obs[category_name].fillna('none', inplace=True)

# minor modification
annotation_edit_dict = {
     'AV Ring':'AV ring',
    'Aorta':'Great vessels',
    'Coronary Artery':'Coronary artery',
    'Mitral Valve':'AV valves',
    'Mitral valve':'AV valves',
    'Myocardium - Ventricular - Trabecular':'Myocardium - Ventricular - Trabeculated',
    'Myocardim - Ventricular - Trabeculated':'Myocardium - Ventricular - Trabeculated',
    'Myocardim - Ventricular - Compact':'Myocardium - Ventricular - Compact',
    'unknown':'Unannotated',
    'Unknown':'Unannotated',
}
adata_vis.obs.replace({category_name:annotation_edit_dict}, inplace=True)
set(adata_vis.obs[category_name])
```

Example 23 (unknown):
```unknown
BHF_F_Hea13295830
BHF_F_Hea13295831
```

Example 24 (unknown):
```unknown
{'AV ring',
 'AV valves',
 'AVN',
 'Aortic valve',
 'Bundle branch - left',
 'Bundle branch - right',
 'Cavity',
 'Connective tissue',
 'Coronary artery',
 'Epicardium',
 'Great vessels',
 'Great vessels ',
 'LN',
 'Myocardium - Atrial',
 'Myocardium - Papillary Muscle',
 'Myocardium - Ventricular - Compact',
 'Myocardium - Ventricular - Compact 1',
 'Myocardium - Ventricular - Trabeculated',
 'Valve apparatus'}
```

Example 25 (python):
```python
# remove 'none' and 'no_tissue'
adata_vis = adata_vis[adata_vis.obs[category_name].isin(['none','Out_of_tissue'])==False]

# remove 'Cavity'
adata_vis = adata_vis[adata_vis.obs[category_name].isin(['Cavity'])==False]

# remove 'Unannotated'
adata_vis = adata_vis[adata_vis.obs[category_name].isin(['Unannotated'])==False]
```

Example 26 (python):
```python
# order annotations
annotations_order = ['Myocardium - Atrial',
                     'Myocardium - Ventricular - Compact',
                     'Myocardium - Ventricular - Trabeculated',
                    'Myocardium - Papillary Muscle',
                    'Epicardium',
                     'AV valves',
                     'Aortic valve',
                    'Valve apparatus',
                   'AV ring',
                    'Connective tissue',
                     'Great vessels',
                     'Coronary artery',
                     # 'Nerve',
                     ]
#adata_vis.obs[category_name] = adata_vis.obs[category_name].astype('category')
#adata_vis.obs[category_name].cat.reorder_categories(annotations_order,inplace=True)
```

Example 27 (python):
```python
pd.crosstab(adata_vis.obs['cell2loc-map_group'],adata_vis.obs[category_name])
```

Example 28 (python):
```python
# check the slides for 'PCW-13w'
#mask = adata_vis.obs['age']=='PCW-13w'
#adata_vis[mask].obs['sample'].value_counts()
# 'HCAHeartST11605166': Atrium
# 'HCAHeartST11605165': Ventricle
```

Example 29 (python):
```python
# combine
adata_vis.obs['sample_for_cell2loc']=adata_vis.obs['sample'].astype('str')
#adata_vis.obs.replace({'sample_for_cell2loc':{
#    'HCAHeartST11605165':'HCAHeartST11605165_HCAHeartST11605166',
#    'HCAHeartST11605166':'HCAHeartST11605165_HCAHeartST11605166'
#}},inplace=True)
adata_vis.obs['sample_for_cell2loc'].value_counts()
```

Example 30 (python):
```python
/tmp/ipykernel_1547584/283590942.py:2: ImplicitModificationWarning: Trying to modify attribute `.obs` of view, initializing view as actual.
  adata_vis.obs['sample_for_cell2loc']=adata_vis.obs['sample'].astype('str')
```

Example 31 (unknown):
```unknown
BHF_F_Hea13295830    10848
BHF_F_Hea13295831     8899
Name: sample_for_cell2loc, dtype: int64
```

Example 32 (python):
```python
adata_vis.obs.columns
```

Example 33 (unknown):
```unknown
Index(['in_tissue', 'array_row', 'array_col', 'sample', 'n_genes_by_counts',
       'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts',
       'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes',
       'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'mt_frac',
       'n_counts', 'n_genes', 'sangerID', 'combinedID', 'donor', 'donor_type',
       'region', 'region_finest', 'age', 'gender', 'facility',
       'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'status',
       'cell2loc-map_group', 'JC_edit', 'sample_for_cell2loc'],
      dtype='object')
```

Example 34 (unknown):
```unknown
set(meta['cell2loc-map_group'])
```

Example 35 (unknown):
```unknown
{'FFPE_16PCW'}
```

Example 36 (python):
```python
# convert adata_vis.var_names to gene_id
adata_vis.var.set_index('gene_ids',inplace=True)
adata_vis.var.head()
```

Example 37 (python):
```python
adata_vis.obs.head(3)
```

Example 38 (python):
```python
%%time
import cell2location
# read in reference signatures
inf_aver = pd.read_csv(f"{ref_run_name}/all_data/inf_aver.csv", index_col=0)

# find shared genes and subset both anndata and reference signatures
intersect = np.intersect1d(adata_vis.var_names, inf_aver.index)
adata_vis = adata_vis[:, intersect].copy()
inf_aver = inf_aver.loc[intersect, :].copy()
print(inf_aver.shape)

for group in set(meta['cell2loc-map_group']):
    print(f'####### {group} #######')
    
    # subset adata
    adata_vis_sub=adata_vis[adata_vis.obs['cell2loc-map_group']==group].copy()
    samples = list(set(adata_vis_sub.obs['sample_for_cell2loc']))
    samples.sort()
    print(f'samples in this run: {samples}')
    adata_vis_sub=adata_vis_sub.copy()
    # prepare anndata for cell2location model
    cell2location.models.Cell2location.setup_anndata(adata=adata_vis_sub, batch_key="sample_for_cell2loc")
    #scvi.data.setup_anndata(adata=adata_vis_sub, batch_key="sample_for_cell2loc")
    #scvi.data.view_anndata_setup(adata_vis_sub)

    # create and train the model
    mod = cell2location.models.Cell2location(
        adata_vis_sub, cell_state_df=inf_aver, 
        # the expected average cell abundance: tissue-dependent 
        # hyper-prior which can be estimated from paired histology:
        N_cells_per_location=20,
        # hyperparameter controlling normalisation of
        # within-experiment variation in RNA detection (using default here):
        detection_alpha=20
    ) 

    mod.train(max_epochs=20000, 
              # train using full data (batch_size=None)
              batch_size=None, 
              # use all data points in training because 
              # we need to estimate cell abundance at all locations
              train_size=1,
              use_gpu=True)
    mod.view_anndata_setup()
    # plot ELBO loss history during training, removing first 100 epochs from the plot
    mod.plot_history(1000)
    plt.legend(labels=['full data training']);

    # In this section, we export the estimated cell abundance (summary of the posterior distribution).
    adata_vis_sub = mod.export_posterior(
        adata_vis_sub, sample_kwargs={'num_samples': 1000, 'batch_size': mod.adata.n_obs, 'use_gpu': True}
    )

    # Save model
    mod.save(f"{run_name}/{group}", overwrite=True)

    # Save anndata object with results
    adata_vis_sub.write(f"{run_name}/{group}/sp.h5ad")

    # Examine reconstruction accuracy to assess if there are any issues with mapping
    # the plot should be roughly diagonal, strong deviations will signal problems
    mod.plot_QC()
    
    del mod,adata_vis_sub
    gc.collect()
    torch.cuda.empty_cache()
```

Example 39 (unknown):
```unknown
(13904, 64)
####### FFPE_16PCW #######
samples in this run: ['BHF_F_Hea13295830', 'BHF_F_Hea13295831']
```

Example 40 (unknown):
```unknown
No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.)
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/pytorch_lightning/trainer/configuration_validator.py:106: UserWarning: You passed in a `val_dataloader` but have no `validation_step`. Skipping val loop.
  rank_zero_warn("You passed in a `val_dataloader` but have no `validation_step`. Skipping val loop.")
You are using a CUDA device ('NVIDIA A100-SXM4-80GB') that has Tensor Cores. To properly utilize them, you should set `torch.set_float32_matmul_precision('medium' | 'high')` which will trade-off precision for performance. For more details, read https://pytorch.org/docs/stable/generated/torch.set_float32_matmul_precision.html#torch.set_float32_matmul_precision
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/pytorch_lightning/trainer/trainer.py:1609: PossibleUserWarning: The number of training batches (1) is smaller than the logging interval Trainer(log_every_n_steps=10). Set a lower value for log_every_n_steps if you want to see logs for the training epoch.
  rank_zero_warn(
```

Example 41 (unknown):
```unknown
Epoch 20000/20000: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20000/20000 [47:34<00:00,  6.62it/s, v_num=1, elbo_train=2.96e+8]
```

Example 42 (unknown):
```unknown
`Trainer.fit` stopped: `max_epochs=20000` reached.
```

Example 43 (unknown):
```unknown
Epoch 20000/20000: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20000/20000 [47:34<00:00,  7.01it/s, v_num=1, elbo_train=2.96e+8]
```

Example 44 (unknown):
```unknown
Anndata setup with scvi-tools version 0.20.3.
```

Example 45 (unknown):
```unknown
Setup via `Cell2location.setup_anndata` with arguments:
```

Example 46 (unknown):
```unknown
{
│   'layer': None,
│   'batch_key': 'sample_for_cell2loc',
│   'labels_key': None,
│   'categorical_covariate_keys': None,
│   'continuous_covariate_keys': None
}
```

Example 47 (unknown):
```unknown
Summary Statistics         
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┓
┃     Summary Stat Key     ┃ Value ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━┩
│         n_batch          │   2   │
│         n_cells          │ 19747 │
│ n_extra_categorical_covs │   0   │
│ n_extra_continuous_covs  │   0   │
│         n_labels         │   1   │
│          n_vars          │ 13904 │
└──────────────────────────┴───────┘
```

Example 48 (python):
```python
Data Registry                
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Registry Key ┃    scvi-tools Location    ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│      X       │          adata.X          │
│    batch     │ adata.obs['_scvi_batch']  │
│    ind_x     │   adata.obs['_indices']   │
│    labels    │ adata.obs['_scvi_labels'] │
└──────────────┴───────────────────────────┘
```

Example 49 (python):
```python
batch State Registry                             
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃         Source Location          ┃    Categories     ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['sample_for_cell2loc'] │ BHF_F_Hea13295830 │          0          │
│                                  │ BHF_F_Hea13295831 │          1          │
└──────────────────────────────────┴───────────────────┴─────────────────────┘
```

Example 50 (python):
```python
labels State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 51 (unknown):
```unknown
Sampling local variables, batch: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:35<00:00, 35.02s/it]
Sampling global variables, sample: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 999/999 [00:25<00:00, 39.42it/s]
```

Example 52 (python):
```python
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/cell2location/models/base/_pyro_mixin.py:407: UserWarning: The figure layout has changed to tight
  plt.tight_layout()
```

Example 53 (unknown):
```unknown
CPU times: user 50min 37s, sys: 30.9 s, total: 51min 8s
Wall time: 51min 8s
```

---

## 1. Read in inputs¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig3_SAN_niche/snMultime_no3_create-network_TF-Region-TG_AxonGuidance.html

**Contents:**
- 1. Read in inputs¶
- Read in and prepare anndata contains gene expression¶
- Read in peak signatures¶
- Read in scenicplus output metadata¶
- 2. DEG: SANCM vs other CMs¶
- 3. DEG: Foetal vs Adult¶
- Read in adult anndata and concatenate with foetal data¶
- Prepare anndata for pseudobulk wilcoxon test¶
- Pseudobulk wilcoxon test¶
- 4. Obtain expressing genes and accessible regions¶
- Genes¶
- Peaks¶
- Prepare scenic metadata¶
- Select interactions¶
- TF-TF network¶
- Save networks¶
- Plot network¶

What this notebook does

**Examples:**

Example 1 (python):
```python
# Parameters
working_dir = "/nfs/team205/heart/Vitro/TF_selection/foetal_SANPcell_vs_workingCM"
path_to_adata_gex = "/nfs/team205/heart/anndata_objects/Foetal/Feb28ObjectRaw_Multiome.h5ad"
path_to_adata_peak_tfidf = "/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/PeakMatrix/Foetal_Peaks_tfidf.h5ad"
path_to_DAR_1 = "/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/MarkerPeaks/SANP-vs-aCM.csv"
path_to_DAR_2 = "/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/MarkerPeaks/aCM-vs-SANP.csv"
path_to_p2g_ArchR = ""
group_key = "fine_grain_aCMs_merged"
group_a = "SinoatrialNodeCardiomyocytes"
group_b = "AtrialCardiomyocytes"
subset_object = True
geneset_list = ["MSigDB_Hallmark_2020", "KEGG_2021_Human", "GO_Biological_Process_2023"]

path_to_eRegulon_signatures_gene = "/nfs/team205/heart/pyscenic/plus/eRegulon_signatures/Cardiomyocytes_Gene_eRegulon_signatures.pkl"
path_to_eRegulon_signatures_region = "/nfs/team205/heart/pyscenic/plus/eRegulon_signatures/Cardiomyocytes_Region_eRegulon_signatures.pkl"

path_to_gene_signatures='/nfs/team205/heart/Vitro/TF_selection/foetal_SANPcell_vs_workingCM/outs/gene_signatures.pkl'
path_to_peak_signatures='/nfs/team205/heart/Vitro/TF_selection/foetal_SANPcell_vs_workingCM/outs/peak_signatures.pkl'

path_to_eRegulon_metadata = "/nfs/team205/heart/pyscenic/plus/eRegulon_metadata/Cardiomyocytes_eRegulon_metadata_filtered.csv" # filtered metadata

###### parameters for TF selection for the population ######
cell_of_intest = 'SinoatrialNodeCardiomyocytes'
groups_for_signatures = ['SinoatrialNodeCardiomyocytes']

# proportion threshold for expressing genes
exp_prop_thresh = 0.1

# proportion threshold for accesible regions
access_prop_thresh = 0.01
```

Example 2 (python):
```python
import scanpy as sc
import pandas as pd
import numpy as np
import anndata

import matplotlib.pyplot as plt
import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 
import seaborn as sns

# import hotspot
```

Example 3 (python):
```python
import session_info
session_info.show()
```

Example 4 (python):
```python
-----
anndata             0.9.2
matplotlib          3.7.2
numpy               1.24.4
pandas              1.5.3
scanpy              1.9.3
seaborn             0.12.2
session_info        1.0.0
-----
```

Example 5 (python):
```python
PIL                         10.0.0
anyio                       NA
arrow                       1.2.3
asttokens                   NA
attr                        23.1.0
attrs                       23.1.0
babel                       2.12.1
backcall                    0.2.0
brotli                      1.0.9
certifi                     2023.11.17
cffi                        1.15.1
charset_normalizer          3.2.0
colorama                    0.4.6
comm                        0.1.4
cycler                      0.10.0
cython_runtime              NA
dateutil                    2.8.2
debugpy                     1.6.8
decorator                   5.1.1
defusedxml                  0.7.1
executing                   1.2.0
fastjsonschema              NA
fqdn                        NA
h5py                        3.9.0
idna                        3.4
igraph                      0.10.6
importlib_metadata          NA
importlib_resources         NA
ipykernel                   6.25.1
isoduration                 NA
jedi                        0.19.0
jinja2                      3.1.2
joblib                      1.3.2
json5                       NA
jsonpointer                 2.0
jsonschema                  4.19.0
jsonschema_specifications   NA
jupyter_events              0.7.0
jupyter_server              2.7.1
jupyterlab_server           2.24.0
kiwisolver                  1.4.5
leidenalg                   0.10.1
llvmlite                    0.40.1
markupsafe                  2.1.3
mpl_toolkits                NA
natsort                     8.4.0
nbformat                    5.9.2
numba                       0.57.1
overrides                   NA
packaging                   23.1
parso                       0.8.3
patsy                       0.5.3
pexpect                     4.8.0
pickleshare                 0.7.5
pkg_resources               NA
plotly                      5.16.1
prometheus_client           NA
prompt_toolkit              3.0.39
psutil                      5.9.5
ptyprocess                  0.7.0
pure_eval                   0.2.2
pydev_ipython               NA
pydevconsole                NA
pydevd                      2.9.5
pydevd_file_utils           NA
pydevd_plugins              NA
pydevd_tracing              NA
pygments                    2.16.1
pyparsing                   3.0.9
pythonjsonlogger            NA
pytz                        2023.3
referencing                 NA
requests                    2.31.0
rfc3339_validator           0.1.4
rfc3986_validator           0.1.1
rpds                        NA
ruamel                      NA
scipy                       1.10.1
send2trash                  NA
six                         1.16.0
sklearn                     1.3.0
sniffio                     1.3.0
socks                       1.7.1
sphinxcontrib               NA
stack_data                  0.6.2
statsmodels                 0.14.0
texttable                   1.6.7
threadpoolctl               3.2.0
tornado                     6.3.3
traitlets                   5.9.0
typing_extensions           NA
uri_template                NA
urllib3                     2.0.4
wcwidth                     0.2.6
webcolors                   1.13
websocket                   1.6.2
yaml                        6.0.1
zipp                        NA
zmq                         25.1.1
zstandard                   0.19.0
```

Example 6 (unknown):
```unknown
-----
IPython             8.12.0
jupyter_client      8.3.0
jupyter_core        4.12.0
jupyterlab          4.0.5
notebook            7.0.2
-----
Python 3.8.17 (default, Jul  5 2023, 21:04:15) [GCC 11.2.0]
Linux-5.4.0-169-generic-x86_64-with-glibc2.17
-----
Session information updated at 2024-03-25 16:38
```

Example 7 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)

spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/regulon_plot.py")
regulon_plot = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = regulon_plot
spec.loader.exec_module(regulon_plot)
```

Example 8 (python):
```python
sc.settings.set_figure_params(dpi=80,dpi_save=300)
```

Example 9 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk'
```

Example 10 (python):
```python
# read in adata
# whole data
adata = sc.read_h5ad(path_to_adata_gex)

# filter and log-normalise
sc.pp.filter_genes(adata, min_cells=3)
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

print(adata.X.data[:10])
adata
```

Example 11 (unknown):
```unknown
[0.41374123 0.22811745 0.22811745 0.41374123 0.22811745 0.41374123
 0.22811745 0.41374123 0.22811745 0.22811745]
```

Example 12 (unknown):
```unknown
AnnData object with n_obs × n_vars = 211145 × 34377
    obs: 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score', 'fine_grain_aCMs_merged', 'fine_grain_aCMs-node_merged'
    var: 'gene_id', 'n_cells'
    uns: 'log1p'
```

Example 13 (unknown):
```unknown
peak_signatures = utils.read_pkl(path_to_peak_signatures)
peak_signatures.keys()
```

Example 14 (unknown):
```unknown
dict_keys(['SinoatrialNodeCardiomyocytes_up', 'SinoatrialNodeCardiomyocytes_down'])
```

Example 15 (python):
```python
sig_peaks = peak_signatures['SinoatrialNodeCardiomyocytes_up']
print(len(sig_peaks))
sig_peaks[:5]
```

Example 16 (unknown):
```unknown
['chr1:55555893_55556393',
 'chr12:115794713_115795213',
 'chr3:113167147_113167647',
 'chr17:65228782_65229282',
 'chr11:116485972_116486472']
```

Example 17 (python):
```python
# scenicplus output
scenic_meta = pd.read_csv(path_to_eRegulon_metadata,index_col=0)
scenic_meta['Region'] = [x.replace('-','_') for x in scenic_meta['Region']]

# select active regulons
scenic_meta = scenic_meta[['_+_' in x for x in scenic_meta['Gene_signature_name']]]
scenic_meta.head()
```

Example 18 (unknown):
```unknown
# select based on R2G_rho
rho_thresh = 0.2
scenic_meta = scenic_meta[scenic_meta['R2G_rho']>rho_thresh]
```

Example 19 (python):
```python
logFC_threshold = 1
group_mean_threshold = 0.1

adata_sub = adata[adata.obs['coarse_grain']=='Cardiomyocytes']
adata_sub = adata_sub[adata_sub.obs['kit_10x']=='Multiome-v1']

res = utils.rank_genes_groups_with_mean(adata_sub,
                                 groupby = 'fine_grain',
                                  groups = ['SinoatrialNodeCardiomyocytes'],
                                  reference = 'rest',
                                  method = 't-test',
                                  pval_cutoff = 0.05,
                                        log2fc_min=logFC_threshold
                                 )
sns.scatterplot(x=res['logfoldchanges'],y=res['group_mean'])
plt.show()
```

Example 20 (python):
```python
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/tools/_rank_genes_groups.py:580: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  adata.uns[key_added] = {}
```

Example 21 (unknown):
```unknown
# filter genes based on the group mean expression
res = res[res['group_mean']>group_mean_threshold]
display(res.head())
degs_sancm_vs_others = list(res['names'])
len(degs_sancm_vs_others)
```

Example 22 (unknown):
```unknown
sig_genes = degs_sancm_vs_others.copy()
```

Example 23 (unknown):
```unknown
'ISL1' in sig_genes
```

Example 24 (python):
```python
# prepare adult anndata
adata_adult = sc.read_h5ad('/nfs/team205/heart/anndata_objects/8regions/RNA_adult-8reg_full_raw_cellstate-annotated.h5ad')

# subset SAN_P_cell
adata_adult = adata_adult[adata_adult.obs['cell_state']=='SAN_P_cell']
adata_adult = adata_adult[adata_adult.obs['kit_10x']=='Multiome-v1']

# log normalise
sc.pp.normalize_total(adata_adult, target_sum=1e4)
sc.pp.log1p(adata_adult)
print(adata_adult.X.data[:10])

# rename
adata_adult.var['gene_id'] =  adata_adult.var_names.copy()
adata_adult.var.set_index('gene_name-new',inplace=True)
adata_adult.var_names = adata_adult.var_names.astype('str')
adata_adult.var_names_make_unique()

adata_adult
```

Example 25 (unknown):
```unknown
[0.911473   0.31626672 0.31626672 0.31626672 0.74952203 0.911473
 0.31626672 0.31626672 0.31626672 0.31626672]
```

Example 26 (python):
```python
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/preprocessing/_normalization.py:170: UserWarning: Received a view of an AnnData. Making a copy.
  view_to_actual(adata)
```

Example 27 (unknown):
```unknown
AnnData object with n_obs × n_vars = 245 × 32732
    obs: 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'n_genes', 'n_genes_by_counts', 'total_counts', 'total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'pct_counts_ribo', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'batch_key', 'leiden_scVI', 'cell_type', 'cell_state_HCAv1', 'cell_state_scNym', 'cell_state_scNym_confidence', 'cell_state', 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'n_counts', '_scvi_batch', '_scvi_labels', 'clus20', 'doublet_cls', 'original_or_new', 'batch', 'scANVI_predictions', 'leiden_scArches'
    var: 'gene_name_scRNA-0-original', 'gene_name_snRNA-1-original', 'gene_name_multiome-2-original', 'gene_id'
    uns: 'age_colors', 'cell_or_nuclei_colors', 'cell_state_colors', 'cell_type_colors', 'donor_colors', 'donor_type_colors', 'facility_colors', 'flushed_colors', 'gender_colors', 'kit_10x_colors', 'leiden', 'modality_colors', 'neighbors', 'original_or_new_colors', 'region_colors', 'region_finest_colors', 'scANVI_predictions_colors', 'umap', 'log1p'
    obsm: 'X_scArches', 'X_umap'
    obsp: 'connectivities', 'distances'
```

Example 28 (python):
```python
# concatenate
## shared features
shared = list(set(adata_sub.var_names).intersection(adata_adult.var_names))

adata_sub.obs['stage'] = 'foetal'
adata_adult.obs['stage'] = 'adult'
adata_adult.obs['week'] = 'adult'
adata_adult.obs['fine_grain'] = 'adult - SAN_P_cell'

adata_conc = adata_sub[:,shared].concatenate(adata_adult[:,shared],
                                             join='inner',
                                             batch_key=None,
                                             index_unique=None
                                            )
```

Example 29 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/anndata/_core/anndata.py:1763: FutureWarning: The AnnData.concatenate method is deprecated in favour of the anndata.concat function. Please use anndata.concat instead.

See the tutorial for concat at: https://anndata.readthedocs.io/en/latest/concatenation.html
  warnings.warn(
```

Example 30 (python):
```python
# combine donor and stage key
adata_conc.obs['donor_new'] = adata_conc.obs['stage'].astype('str')+'_'+adata_conc.obs['donor'].astype('str')
set(adata_conc.obs['donor_new'])
```

Example 31 (unknown):
```unknown
{'adult_A61',
 'adult_AH1',
 'adult_AH2',
 'adult_AV10',
 'adult_AV14',
 'adult_AV3',
 'foetal_C104',
 'foetal_C82',
 'foetal_C85',
 'foetal_C87',
 'foetal_C92',
 'foetal_C98',
 'foetal_Hst33',
 'foetal_Hst36',
 'foetal_Hst39',
 'foetal_Hst40',
 'foetal_Hst41'}
```

Example 32 (python):
```python
# add drimester 
donor_trimester_df = adata_conc.obs[['donor','trimester']].copy()
donor_trimester_df = donor_trimester_df.drop_duplicates()
donor_trimester_df['trimester'] = [x if x==x else 'adult' for x in donor_trimester_df['trimester']]
donor_trimester_df
```

Example 33 (python):
```python
groupby = 'donor_new'
celltypes = ['SinoatrialNodeCardiomyocytes','adult - SAN_P_cell']
n_cell_thresh = 10

# subset anndata
ad = adata_conc[adata_conc.obs['fine_grain'].isin(celltypes)]
# remove group which only has less than a threshold
vc = ad.obs[groupby].value_counts()
grp_to_drop = list(vc.index[vc < 10])
ad = ad[~ad.obs[groupby].isin(grp_to_drop)]
print(f'dropped groups: {grp_to_drop}')

# mean expression per group
exp_mean = utils.grouped_obs_mean(ad, group_key=groupby)

# make anndata
import anndata
adata_pb = anndata.AnnData(
    X=exp_mean.T,
    obs=pd.DataFrame(exp_mean.columns).set_index(0),
    var=pd.DataFrame(exp_mean.index).set_index(0)
)

# add stage key
adata_pb.obs['stage'] = [x.split('_')[0] for x in adata_pb.obs_names]
adata_pb.obs['stage'] = adata_pb.obs['stage'].astype('category')
adata_pb.obs['stage'].cat.reorder_categories(['foetal','adult'],inplace=True)

# add donor and trimester key
adata_pb.obs['donor'] = [x.split('_')[1] for x in adata_pb.obs_names]
obs = adata_pb.obs.reset_index().merge(donor_trimester_df,how='left',on='donor').set_index(0)
obs.index.name = 'index'
adata_pb.obs = obs
adata_pb.obs['trimester'] = adata_pb.obs['trimester'].astype('category')
adata_pb.obs['trimester'].cat.reorder_categories(['firsttrimester','secondtrimester','adult'],inplace=True)


print(adata_pb.obs['stage'].value_counts())
print(adata_pb)
```

Example 34 (unknown):
```unknown
dropped groups: ['foetal_C82', 'foetal_Hst36']
foetal    8
adult     6
Name: stage, dtype: int64
AnnData object with n_obs × n_vars = 14 × 30749
    obs: 'stage', 'donor', 'trimester'
```

Example 35 (python):
```python
/tmp/ipykernel_58626/3068665136.py:27: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
  adata_pb.obs['stage'].cat.reorder_categories(['foetal','adult'],inplace=True)
/tmp/ipykernel_58626/3068665136.py:35: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
  adata_pb.obs['trimester'].cat.reorder_categories(['firsttrimester','secondtrimester','adult'],inplace=True)
```

Example 36 (python):
```python
# rank test
logFC_threshold_FvsA = 1
group_mean_threshold = 0.1

res_foetal_vs_adult = utils.rank_genes_groups_with_mean(adata_pb,
                                 groupby = 'stage',
                                  groups = ['foetal'],
                                  reference = 'adult',
                                  method = 'wilcoxon',
                                  pval_cutoff = 0.1,
                                        log2fc_min=logFC_threshold_FvsA
                                 )

sns.scatterplot(x=res_foetal_vs_adult['logfoldchanges'],y=res_foetal_vs_adult['group_mean'])
plt.show()
```

Example 37 (python):
```python
res_foetal_vs_adult = res_foetal_vs_adult[res_foetal_vs_adult['group_mean']>group_mean_threshold]
degs_foetal_vs_adult = list(res_foetal_vs_adult['names'])
print(len(degs_foetal_vs_adult))
display(res_foetal_vs_adult.head())
```

Example 38 (unknown):
```unknown
res_foetal_vs_adult[res_foetal_vs_adult['names']=='CACNA1D']
```

Example 39 (unknown):
```unknown
len(degs_foetal_vs_adult)
```

Example 40 (python):
```python
# get expression proportions
exp_mat = utils.get_exp_proportion(adata,groupby='fine_grain', use_rep='X')
exp_genes = list(exp_mat.index[exp_mat[cell_of_intest]>exp_prop_thresh])
len(exp_genes)
```

Example 41 (unknown):
```unknown
'ISL1' in exp_genes
```

Example 42 (python):
```python
adata_peak = sc.read_h5ad(path_to_adata_peak_tfidf)
adata_peak
```

Example 43 (unknown):
```unknown
AnnData object with n_obs × n_vars = 167022 × 508040
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels'
    var: 'seqnames', 'start', 'end', 'width', 'strand', 'score', 'replicateScoreQuantile', 'groupScoreQuantile', 'Reproducibility', 'GroupReplicate', 'distToGeneStart', 'nearestGene', 'peakType', 'distToTSS', 'nearestTSS', 'GC', 'idx', 'N'
```

Example 44 (python):
```python
# get accessible proportions
access_mat = utils.get_exp_proportion(adata_peak,groupby='fine_grain', use_rep='X')
```

Example 45 (unknown):
```unknown
access_peaks = list(access_mat.index[access_mat[cell_of_intest]>access_prop_thresh])
len(access_peaks)
```

Example 46 (python):
```python
# add columns to scenicplus metadata
# scenic_meta['Gene_isTF'] = scenic_meta['Gene'].isin(list(set(scenic_meta['TF'])))
scenic_meta['TF_isExp'] = scenic_meta['TF'].isin(exp_genes)
scenic_meta['TF_isSignature'] = (scenic_meta['TF'].isin(sig_genes))&(scenic_meta['TF'].isin(exp_genes))
scenic_meta['Region_isAccess'] = scenic_meta['Region'].isin(access_peaks)
scenic_meta['Region_isSignature'] = (scenic_meta['Region'].isin(sig_peaks))&(scenic_meta['Region'].isin(access_peaks))
scenic_meta.head()
```

Example 47 (unknown):
```unknown
target_genes = ['SEMA3A', 'SLIT2', 'SLIT3', 'EFNB2', 'CXCL12', 'FGF10', 'VEGFC', 'NELL1']
# based on the previous analysis
```

Example 48 (python):
```python
# select interactions

# select interaction which target "target_genes"
scenic_meta_sel = scenic_meta[(scenic_meta['Gene'].isin(target_genes))&\
                               scenic_meta['TF_isExp'] &\
                               scenic_meta['Region_isAccess'] &\
                              (scenic_meta['TF_isSignature']|scenic_meta['Region_isSignature'])
                             ]

# get TFs and target genes in the selected interaction
direct_TFs = list(set(scenic_meta_sel['TF']))
targets = list(set(scenic_meta_sel['Gene']))
# remove TFs already in target genes
TFs_sel = list(set(direct_TFs) - set(targets))

print(len(TFs_sel))
print(f'direct TFs: {direct_TFs}')
print('')

# select higher level interactions 
print('Start looking for higher level TFs')
higher_level_TFs = []
while len(TFs_sel)>0:
    scenic_meta_sel2 = scenic_meta[(scenic_meta['Gene'].isin(TFs_sel))&\
                               scenic_meta['TF_isExp'] &\
                               scenic_meta['Region_isAccess'] &\
                              (scenic_meta['TF_isSignature']|scenic_meta['Region_isSignature'])
                                  ]
    higher_level_TFs = higher_level_TFs + list(set(scenic_meta_sel2['TF']))
    if len(scenic_meta_sel2)>0:
        # concatenate
        scenic_meta_sel = pd.concat([scenic_meta_sel,scenic_meta_sel2])
        scenic_meta_sel = scenic_meta_sel.drop_duplicates()
        # get TFs and target genes in the selected interaction
        TFs_sel = list(set(scenic_meta_sel['TF']))
        targets = list(set(scenic_meta_sel['Gene']))
        # remove TFs already in target genes
        TFs_sel = list(set(TFs_sel) - set(targets))
        print(f'{len(TFs_sel)}: {TFs_sel}')
    else:
        print('no more higher level TFs')
        TFs_sel = []

scenic_meta_sel['direct_TF'] = scenic_meta_sel['TF'].isin(direct_TFs)
higher_level_TFs = list(set(higher_level_TFs)-set(direct_TFs))

# reindex
scenic_meta_sel.index = range(len(scenic_meta_sel))

print("")
print(f'Number of selected interactions: {len(scenic_meta_sel)}')
print(f"Number of TFs: {len(set(scenic_meta_sel['TF']))}")
```

Example 49 (unknown):
```unknown
14
direct TFs: ['SMAD9', 'MEF2C', 'BNC2', 'NR2F1', 'NFIB', 'PKNOX2', 'THRB', 'HMGA2', 'TBX5', 'POU6F2', 'NR2F2', 'FOS', 'TRPS1', 'TBX18']

Start looking for higher level TFs
11: ['RARB', 'NFIA', 'SHOX2', 'TBX3', 'JUN', 'NFIB', 'ZNF41', 'MAX', 'ETV6', 'MITF', 'ISL1']
13: ['KLF13', 'ZNF529', 'FOXO1', 'NFIB', 'JUN', 'GLIS3', 'ZNF41', 'MAX', 'TEAD1', 'MEIS1', 'SP4', 'TEAD3', 'BHLHE40']
10: ['KLF13', 'ZNF529', 'NFIB', 'JUN', 'ZNF41', 'MAX', 'TEAD1', 'TEAD3', 'SP4', 'BHLHE40']
no more higher level TFs

Number of selected interactions: 275
Number of TFs: 33
```

Example 50 (python):
```python
# get TF-TG dictionary
network = {}
for tf in set(scenic_meta_sel['TF']):
    df_sub = scenic_meta_sel[scenic_meta_sel['TF']==tf]
    network[tf] = list(set(df_sub['Gene']))
print(len(network.keys()))
```

Example 51 (python):
```python
# add metrics
scenic_meta_sel['TF_n_linked_region'] = np.nan
scenic_meta_sel['TF_n_target_genes'] = np.nan
scenic_meta_sel['TF_n_target_sig_regions'] = np.nan

for tf in set(scenic_meta_sel['TF']):
    tf_ind = scenic_meta_sel[scenic_meta_sel['TF']==tf].index
    targets = list(set(scenic_meta_sel.loc[tf_ind,'Gene']))
    scenic_meta_sel.loc[tf_ind,'TF_n_target_genes'] = len(targets)
    regions = list(set(scenic_meta_sel.loc[tf_ind,'Region']))
    regions = list(set(regions).intersection(sig_peaks))
    scenic_meta_sel.loc[tf_ind,'TF_n_target_sig_regions'] = len(regions)
    if tf in list(set(scenic_meta_sel['Gene'])):
        gene_ind = scenic_meta_sel[scenic_meta_sel['Gene']==tf].index
        n_region = len(set(scenic_meta_sel.loc[gene_ind,'Region']))
        scenic_meta_sel.loc[tf_ind,'TF_n_linked_region'] = n_region
    else:
        scenic_meta_sel.loc[tf_ind,'TF_n_linked_region'] = 0
        
scenic_meta_sel.head(3)
```

Example 52 (unknown):
```unknown
scenic_meta_sel_TF_metrices = scenic_meta_sel[['TF','direct_TF','TF_n_target_genes','TF_n_target_sig_regions','TF_n_linked_region']]
# df = df[df['direct_TF']]
scenic_meta_sel_TF_metrices.drop_duplicates(inplace=True)
scenic_meta_sel_TF_metrices.set_index('TF',inplace=True)
scenic_meta_sel_TF_metrices.sort_values('TF_n_target_genes',ascending=False)[:30]
```

Example 53 (python):
```python
/tmp/ipykernel_58626/2002270316.py:3: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  scenic_meta_sel_TF_metrices.drop_duplicates(inplace=True)
```

Example 54 (python):
```python
# select interactions those targets are TFs
scenic_meta_sel_TF = scenic_meta_sel[scenic_meta_sel['Gene'].isin(list(set(scenic_meta_sel['TF'])))]

network_TF = {}
for tf in set(scenic_meta_sel['TF']): # for all the TFs in the network
    if tf in set(scenic_meta_sel_TF['TF']): # whether the TF has TF target
        df_sub = scenic_meta_sel_TF[scenic_meta_sel_TF['TF']==tf]
        network_TF[tf] = list(set(df_sub['Gene']))
    else:
        network_TF[tf] = []
print(len(network_TF.keys()))
```

Example 55 (unknown):
```unknown
# calculate centrality
centrality_type='eigenvector'
centrality_dict = regulon_plot.get_centrality(network_TF,
                   directed=True,
                   centrality_type=centrality_type, # or 'degree','in_dgree','out_degree'
                   out_df=False # if False output dictionary
                  )
centrality_dict
```

Example 56 (unknown):
```unknown
{'NFIA': 0.10256624163167062,
 'THRB': 0.3488051331071077,
 'TBX3': 0.15738853720889912,
 'SMAD9': 0.03078880142438707,
 'SHOX2': 0.5915619010224106,
 'ISL1': 0.2778552011352862,
 'TBX18': 0.305669956425896,
 'KLF13': 2.56222194632898e-15,
 'ZNF529': 2.56222194632898e-15,
 'PKNOX2': 0.2577801506738265,
 'NR2F1': 0.16878368729177762,
 'ETV6': 0.17901962703762447,
 'TBX5': 0.16878368729177762,
 'NR2F2': 0.16878368729177762,
 'TRPS1': 0.12463027719678846,
 'ZNF41': 2.56222194632898e-15,
 'BNC2': 0.2274036004364482,
 'GLIS3': 0.04863127661046923,
 'MEF2C': 0.1798423413704834,
 'RARB': 0.07029778563893391,
 'HMGA2': 0.04152299632772208,
 'MITF': 0.05275112280562701,
 'POU6F2': 0.10567624115934543,
 'MEIS1': 0.11163015809739468,
 'JUN': 2.56222194632898e-15,
 'SP4': 2.56222194632898e-15,
 'BHLHE40': 2.56222194632898e-15,
 'FOS': 1.3433422197969282e-09,
 'NFIB': 2.56222194632898e-15,
 'MAX': 2.56222194632898e-15,
 'TEAD1': 2.56222194632898e-15,
 'FOXO1': 0.050197031129579024,
 'TEAD3': 2.56222194632898e-15}
```

Example 57 (python):
```python
# add to dataframe and save
scenic_meta_sel_TF_metrices.loc[centrality_dict.keys(),f'centaliry_{centrality_type}'] = list(centrality_dict.values())
scenic_meta_sel_TF_metrices[f'centaliry_{centrality_type}'].fillna(0,inplace=True)
print(len(scenic_meta_sel_TF_metrices))
display(scenic_meta_sel_TF_metrices)

# save
scenic_meta_sel_TF_metrices.to_csv('/nfs/team205/heart/Vitro/TF_selection/foetal_SANPcell_vs_workingCM/TF_selection_df_target-AxonGuidances.csv')
```

Example 58 (python):
```python
/tmp/ipykernel_58626/851482954.py:2: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  scenic_meta_sel_TF_metrices.loc[centrality_dict.keys(),f'centaliry_{centrality_type}'] = list(centrality_dict.values())
/tmp/ipykernel_58626/851482954.py:3: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  scenic_meta_sel_TF_metrices[f'centaliry_{centrality_type}'].fillna(0,inplace=True)
```

Example 59 (unknown):
```unknown
# TF-TG network
utils.save_pkl(network,
               '/nfs/team205/heart/Vitro/TF_selection/foetal_SANPcell_vs_workingCM/TF-TG-network_target-AxonGuidances.pkl')

# TF-TF network
utils.save_pkl(network_TF,
               '/nfs/team205/heart/Vitro/TF_selection/foetal_SANPcell_vs_workingCM/TF-TF-network_target-AxonGuidances.pkl')
```

Example 60 (python):
```python
import networkx as nx
```

Example 61 (unknown):
```unknown
# prepare parameters
size_scale = 7000
base_size = 200

node_size_map = {}
for n in network.keys():
    if n in centrality_dict.keys():
        node_size_map[n]=int(centrality_dict[n]*size_scale + base_size)
    else: # means no connection with other TFs
        node_size_map[n]=base_size
        

node_size_map
```

Example 62 (unknown):
```unknown
{'NFIA': 917,
 'SHOX2': 4340,
 'KLF13': 200,
 'ZNF529': 200,
 'PKNOX2': 2004,
 'ZNF41': 200,
 'THRB': 2641,
 'SMAD9': 415,
 'MEF2C': 1458,
 'TBX3': 1301,
 'JUN': 200,
 'MITF': 569,
 'SP4': 200,
 'BHLHE40': 200,
 'FOS': 200,
 'TRPS1': 1072,
 'NFIB': 200,
 'MAX': 200,
 'TEAD1': 200,
 'HMGA2': 490,
 'ETV6': 1453,
 'POU6F2': 939,
 'NR2F2': 1381,
 'RARB': 692,
 'BNC2': 1791,
 'FOXO1': 551,
 'NR2F1': 1381,
 'GLIS3': 540,
 'TBX5': 1381,
 'MEIS1': 981,
 'TEAD3': 200,
 'TBX18': 2339,
 'ISL1': 2144}
```

Example 63 (python):
```python
# Create a directed graph
G = nx.DiGraph()

# Add nodes and edges
for tf, targets in network.items():
    G.add_node(tf, type='TF', size=node_size_map.get(tf, 1))  # Add TF node
    for tg in targets:
        if tg not in G:
            G.add_node(tg, type='TG', size=0)  # Add TG node if not already added
        G.add_edge(tf, tg)  # Add edge

# Define tf_nodes
tf_nodes = [node for node, attr in G.nodes(data=True) if attr['type'] == 'TF']
tg_nodes = [node for node, attr in G.nodes(data=True) if attr['type'] == 'TG']
tg_nodes.sort()

# Identifying direct and indirect TFs
direct_tfs = set()
for tf, targets in network.items():
    for tg in targets:
        if tg in tg_nodes:
            direct_tfs.add(tf)

indirect_tfs = list(set(tf_nodes) - direct_tfs)  # TFs that are not in direct TFs
direct_tfs = list(direct_tfs)
indirect_tfs.sort()
direct_tfs.sort()

# Manually creating circular positions for indirect and direct TFs, and linear positions for TGs
def create_circular_positions(node_list, center_x, center_y, radius):
    angle_step = 2 * np.pi / len(node_list)
    return {node: (center_x + np.cos(i * angle_step) * radius, center_y + np.sin(i * angle_step) * radius) 
            for i, node in enumerate(node_list)}

# Create circular positions for indirect plus direct TFs
all_tfs = direct_tfs+indirect_tfs
all_tfs.sort()
tf_pos = create_circular_positions(all_tfs, 0.5, 0.5, 0.25)

# Linear positions for TGs
tg_x = [1 for _ in tg_nodes]  # All TG nodes have the same x-coordinate
tg_y = np.linspace(0.7, 0.3, len(tg_nodes))  # Evenly spaced y-coordinates
tg_pos = {tg: (x, y) for tg, x, y in zip(tg_nodes, tg_x, tg_y)}

# Combine positions
pos = {**tf_pos, **tg_pos}

# Update node colors: indirect TFs to "lightskyblue"
for node in G.nodes:
    if node in indirect_tfs:
        G.nodes[node]['color'] = 'white'
    else:
        G.nodes[node]['color'] = 'peachpuff' if G.nodes[node]['type'] == 'TF' else 'white'

# node colors
node_colors = [data['color'] for node, data in G.nodes(data=True)]

# node label colors
label_colors = dict((node,'red') if node in degs_foetal_vs_adult else (node,'black') for node in G.nodes())

# Plotting with updated positions
plt.figure(figsize=(8, 5.5))
nx.draw(G, pos, with_labels=False, node_color=node_colors, node_size=[G.nodes[node]['size'] for node in G.nodes()], 
        edge_color='darkgrey', alpha=0.9, 
        edgecolors="black", linewidths=1, node_shape="o")
# Draw labels with specified colors
for node, color in label_colors.items():
    plt.text(pos[node][0], pos[node][1], s=node, 
             bbox=dict(facecolor='white', alpha=0),
             horizontalalignment='center', verticalalignment='center',
             color=color)

plt.title("SANPC GRN targeting axon guidance genes")
plt.savefig(f"{figdir}/SANPC_GRN_Axon-guidance-genes.pdf", bbox_inches="tight",dpi=300)
plt.show()
```

---

## Variables¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig4_Cardiomyocytes/VentricularCardiomyocytes/multiVI_scFate/no1_MultiVI_prep.html

**Contents:**
- Variables¶
- Read in¶
- Prepare paired anndata¶
- Prepare RNA data¶

35705 rows × 18 columns

36601 rows × 3 columns

**Examples:**

Example 1 (python):
```python
import scanpy as sc
import pandas as pd
import numpy as np
import anndata
import scipy

from scipy.io import mmwrite, mmread
from scipy.sparse import csr_matrix

import sctk
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (unknown):
```unknown
-----
anndata             0.8.0
numpy               1.21.5
pandas              1.3.5
scanpy              1.8.2
scipy               1.7.3
sctk                NA
session_info        1.0.0
-----
```

Example 4 (python):
```python
PIL                 9.1.1
asciitree           NA
backcall            0.2.0
beta_ufunc          NA
binom_ufunc         NA
cached_property     1.5.1
cairo               1.20.0
cffi                1.14.4
cloudpickle         1.6.0
colorama            0.4.4
cycler              0.10.0
cython_runtime      NA
cytoolz             0.11.2
dask                2022.01.0
dateutil            2.8.1
decorator           4.4.2
defusedxml          0.6.0
fasteners           0.17.3
fsspec              2022.01.0
google              NA
h5py                3.1.0
igraph              0.10.2
importlib_metadata  1.7.0
ipykernel           5.4.2
ipython_genutils    0.2.0
ipywidgets          7.6.3
jedi                0.17.2
jinja2              3.1.2
joblib              1.0.0
kiwisolver          1.3.1
leidenalg           0.9.0
llvmlite            0.35.0
louvain             0.7.0
markupsafe          2.1.1
matplotlib          3.5.2
mpl_toolkits        NA
msgpack             1.0.3
natsort             7.1.0
nbinom_ufunc        NA
numba               0.52.0
numcodecs           0.9.1
numexpr             2.7.2
numpy_groupies      0.9.13
packaging           21.3
parso               0.7.1
pexpect             4.8.0
pickleshare         0.7.5
pkg_resources       NA
plotly              5.10.0
prompt_toolkit      3.0.10
psutil              5.9.0
ptyprocess          0.7.0
pyarrow             9.0.0
pygments            2.13.0
pyparsing           2.4.7
pytz                2020.5
scvelo              0.2.4
seaborn             0.11.1
setuptools_scm      NA
sinfo               0.3.1
six                 1.14.0
sklearn             1.0.2
sphinxcontrib       NA
statsmodels         0.12.1
storemagic          NA
tables              3.6.1
tblib               1.7.0
texttable           1.6.3
threadpoolctl       2.1.0
tlz                 0.11.2
toolz               0.11.2
tornado             6.1
traitlets           5.0.5
typing_extensions   NA
wcwidth             0.2.5
yaml                5.4.1
zarr                2.11.3
zipp                NA
zmq                 21.0.1
zope                NA
```

Example 5 (unknown):
```unknown
-----
IPython             7.19.0
jupyter_client      6.1.11
jupyter_core        4.7.0
notebook            6.2.0
-----
Python 3.7.9 | packaged by conda-forge | (default, Dec  9 2020, 21:08:20) [GCC 9.3.0]
Linux-4.15.0-112-generic-x86_64-with-debian-bullseye-sid
-----
Session information updated at 2023-08-10 21:16
```

Example 6 (python):
```python
sc.settings.set_figure_params(dpi=120)
```

Example 7 (unknown):
```unknown
cells_to_analyse = ['VentricularCardiomyocytesLeftCompact',
                     'VentricularCardiomyocytesLeftTrabeculated',
                    'VentricularCardiomyocytesRightCompact',
                     'VentricularCardiomyocytesRightTrabeculated',
                    'VentricularCardiomyocytesCycling']

celltype_key = 'fine_grain'

# region_to_analyse = ['whole sample']
```

Example 8 (unknown):
```unknown
!ls /nfs/team205/heart/anndata_objects/Foetal
```

Example 9 (python):
```python
14Feb_fromSemih		      QC
archive			      RNA_foetal_post-scVI_raw.h5ad
EC			      RNA_foetal_raw.h5ad
Feb28ObjectRaw.h5ad	      scArches
Feb28ObjectRaw_Multiome.h5ad  scVI
Feb28ObjectRaw_Multiome.rds   TipAndStalkAndGVSMCsPartitionedForCell2Loc.h5ad
Feb28ObjectRaw.obs.csv	      trisomy21
Feb28ObjectRaw_original.h5ad  visium-FFPE_foetal_raw.h5ad
Feb28ObjectRaw_re-scVI.h5ad   visium_foetal_healthy-all_raw.h5ad
FetalAtlas.h5ad		      visium_foetal_healthy-all_spot-annotation.csv
He_Cell_foetal-lung	      visium_foetal_raw.h5ad
Hill2022.h5ad		      visium-OCT_foetal_acrossage_raw.h5ad
Hill2022_QCapplied.h5ad       visium-OCT_foetal_DuctusA_raw.h5ad
Immune			      visium-OCT_foetal_raw.h5ad
metadata		      visium-OCT-LV_foetal_raw.h5ad
multiome_ATAC		      visium-OCT-T21_foetal_raw.h5ad
multiVI
```

Example 10 (python):
```python
# rna
adata_rna = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/Feb28ObjectRaw.h5ad')
print(adata_rna.obs[celltype_key].value_counts())

# subset cells
adata_rna = adata_rna[adata_rna.obs[celltype_key].isin(cells_to_analyse)]

# subset regions
# adata_rna = adata_rna[adata_rna.obs['region'].isin(region_to_analyse)]

# add modality to .var
adata_rna.var['modality']='Gene Expression'

adata_rna
```

Example 11 (unknown):
```unknown
GreatVesselAdventitialFibroblasts        61593
GreatVesselSmoothMuscleCells             22992
VentricularCardiomyocytesRightCompact    16215
VentricularCardiomyocytesLeftCompact     15591
MyocardialInterstitialFibroblasts1       13880
                                         ...  
MacrophagesTIMD4pos                        470
MonocytesMPOpos                            414
Megakaryocytes                             360
ChromaffinCells                            228
DendriticCellsMature                       184
Name: fine_grain, Length: 63, dtype: int64
```

Example 12 (unknown):
```unknown
AnnData object with n_obs × n_vars = 50099 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score'
    var: 'gene_name_scRNA-0', 'gene_id', 'modality'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'age_colors', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'cycling_colors', 'donor_colors', 'fine_grain_colors', 'heart_or_greatvessels_colors', 'hvg', 'kit_10x_colors', 'modality_colors', 'neighbors', 'phase_colors', 'region_colors', 'sex_colors', 'trimester_colors', 'umap', 'week_colors'
    obsm: 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 13 (python):
```python
pd.crosstab(adata_rna.obs[celltype_key],adata_rna.obs['region'])
```

Example 14 (python):
```python
pd.crosstab(adata_rna.obs[celltype_key],adata_rna.obs['kit_10x'])
```

Example 15 (python):
```python
pd.crosstab(adata_rna.obs[celltype_key],adata_rna.obs['cell_or_nuclei'])
```

Example 16 (python):
```python
pd.crosstab(adata_rna.obs['age'],adata_rna.obs['cell_or_nuclei'])
```

Example 17 (python):
```python
pd.crosstab(adata_rna.obs[celltype_key],adata_rna.obs['cycling'])
```

Example 18 (python):
```python
adata_rna.obs['kit_10x'].value_counts()
```

Example 19 (unknown):
```unknown
Multiome-v1    41279
5prime-v2       4788
3prime-v3       4032
Name: kit_10x, dtype: int64
```

Example 20 (python):
```python
# atac
# ie only Multiome data
adata_atac = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/PeakMatrix/Foetal_Peaks.h5ad')

# subset cells
adata_atac = adata_atac[adata_atac.obs[celltype_key].isin(cells_to_analyse)]
print(adata_atac.obs[celltype_key].value_counts())

# subset regions
# adata_atac = adata_atac[adata_atac.obs['region'].isin(region_to_analyse)]

# add modality to .var
adata_atac.var['modality']='Peaks'

adata_atac
```

Example 21 (unknown):
```unknown
VentricularCardiomyocytesRightCompact         12008
VentricularCardiomyocytesLeftCompact          11397
VentricularCardiomyocytesLeftTrabeculated      5054
VentricularCardiomyocytesCycling               4977
VentricularCardiomyocytesRightTrabeculated     2269
Name: fine_grain, dtype: int64
```

Example 22 (unknown):
```unknown
AnnData object with n_obs × n_vars = 35705 × 508040
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels'
    var: 'seqnames', 'start', 'end', 'width', 'strand', 'score', 'replicateScoreQuantile', 'groupScoreQuantile', 'Reproducibility', 'GroupReplicate', 'distToGeneStart', 'nearestGene', 'peakType', 'distToTSS', 'nearestTSS', 'GC', 'idx', 'N', 'modality'
```

Example 23 (python):
```python
pd.crosstab(adata_atac.obs[celltype_key],adata_atac.obs['region'])
```

Example 24 (python):
```python
adata_atac.obs
```

Example 25 (python):
```python
# shared barcodes, ie Multiome data (not all, since MultiomeRNA has a bit more data than MultiomeATAC)
barcodes_inter=list(set(adata_rna.obs_names).intersection(adata_atac.obs_names))
len(barcodes_inter)

# take shared cells
multiome_rna=adata_rna[barcodes_inter,:]
multiome_atac=adata_atac[barcodes_inter,:]

# concatenate
adata_paired=anndata.AnnData(
    X=scipy.sparse.hstack([multiome_rna.X,multiome_atac.X]).tocsr(), # concatenate sparse matrix and convert to Compressed Sparse Row format
    obs=multiome_rna.obs[['sangerID', 'combinedID', 'donor', 'region', 'age', 'sex', 'facility',
                           'cell_or_nuclei', 'modality', 'kit_10x', 'week', 'trimester', 'cycling',
                           'S_score', 'G2M_score', 'phase', 'fine_grain', 'mid_grain','coarse_grain','batch_key',
                          # ATAC QC

                          # RNA QC
                            'total_counts',
                     'total_counts_mt',
                     'total_counts_ribo',
                            'n_genes_by_counts',
                     'pct_counts_mt',
                     'pct_counts_ribo',
                            'log1p_n_genes_by_counts',
                     'log1p_total_counts',
                     'log1p_total_counts_mt',
                     'log1p_total_counts_ribo',
                     'scrublet_score',
                        'doublet_pval',
                        'doublet_bh_pval',]],
    var=pd.concat([multiome_rna.var[['modality','gene_name_scRNA-0','gene_id']], \
                   multiome_atac.var
                  ],axis=0)
)
adata_paired
```

Example 26 (unknown):
```unknown
AnnData object with n_obs × n_vars = 35705 × 544641
    obs: 'sangerID', 'combinedID', 'donor', 'region', 'age', 'sex', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'week', 'trimester', 'cycling', 'S_score', 'G2M_score', 'phase', 'fine_grain', 'mid_grain', 'coarse_grain', 'batch_key', 'total_counts', 'total_counts_mt', 'total_counts_ribo', 'n_genes_by_counts', 'pct_counts_mt', 'pct_counts_ribo', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'log1p_total_counts_mt', 'log1p_total_counts_ribo', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id', 'seqnames', 'start', 'end', 'width', 'strand', 'score', 'replicateScoreQuantile', 'groupScoreQuantile', 'Reproducibility', 'GroupReplicate', 'distToGeneStart', 'nearestGene', 'peakType', 'distToTSS', 'nearestTSS', 'GC', 'idx', 'N'
```

Example 27 (python):
```python
# modify modality columns in .obs
adata_paired.obs['modality'] = 'Multiome'
adata_paired.obs['modality'].value_counts()
```

Example 28 (unknown):
```unknown
Multiome    35705
Name: modality, dtype: int64
```

Example 29 (python):
```python
# save
adata_paired.write('/nfs/team205/heart/anndata_objects/Foetal/vCM/MultiVI-pre_paired_foetal_vCMs_raw.h5ad')
```

Example 30 (python):
```python
adata_paired.obs.head()
```

Example 31 (python):
```python
adata_paired.obs['batch_key'].value_counts()
```

Example 32 (unknown):
```unknown
Hst33_Nuclei_Multiome-v1    17373
Hst41_Nuclei_Multiome-v1     9591
Hst36_Nuclei_Multiome-v1     2152
Hst39_Nuclei_Multiome-v1     1932
C87_Nuclei_Multiome-v1       1440
Hst40_Nuclei_Multiome-v1     1429
C104_Nuclei_Multiome-v1       720
C98_Nuclei_Multiome-v1        421
C92_Nuclei_Multiome-v1        322
C85_Nuclei_Multiome-v1        280
C82_Nuclei_Multiome-v1         45
Name: batch_key, dtype: int64
```

Example 33 (python):
```python
# select single-cell data (3p and 5p)
adata_scrna = adata_rna[adata_rna.obs['kit_10x'].isin(['3prime-v3','5prime-v2'])]
adata_scrna.obs['kit_10x'].value_counts()
```

Example 34 (unknown):
```unknown
5prime-v2    4788
3prime-v3    4032
Name: kit_10x, dtype: int64
```

Example 35 (python):
```python
adata_scrna.X.data[:10]
```

Example 36 (unknown):
```unknown
array([6., 1., 3., 1., 1., 1., 8., 3., 1., 1.], dtype=float32)
```

Example 37 (python):
```python
# recreate anndata
adata_scrna=anndata.AnnData(
    X=adata_scrna.X, # concatenate sparse matrix and convert to Compressed Sparse Row format
    obs=adata_scrna.obs[['sangerID', 'combinedID', 'donor', 'region', 'age', 'sex', 'facility',
                           'cell_or_nuclei', 'modality', 'kit_10x', 'week', 'trimester', 'cycling',
                           'S_score', 'G2M_score', 'phase', 'fine_grain', 'mid_grain','coarse_grain','batch_key',
                          # ATAC QC

                          # RNA QC
                            'total_counts',
                     'total_counts_mt',
                     'total_counts_ribo',
                            'n_genes_by_counts',
                     'pct_counts_mt',
                     'pct_counts_ribo',
                            'log1p_n_genes_by_counts',
                     'log1p_total_counts',
                     'log1p_total_counts_mt',
                     'log1p_total_counts_ribo',
                     'scrublet_score',
                        'doublet_pval',
                        'doublet_bh_pval',]],
    var=adata_scrna.var[['modality','gene_name_scRNA-0','gene_id']]
)
adata_scrna
```

Example 38 (unknown):
```unknown
AnnData object with n_obs × n_vars = 8820 × 36601
    obs: 'sangerID', 'combinedID', 'donor', 'region', 'age', 'sex', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'week', 'trimester', 'cycling', 'S_score', 'G2M_score', 'phase', 'fine_grain', 'mid_grain', 'coarse_grain', 'batch_key', 'total_counts', 'total_counts_mt', 'total_counts_ribo', 'n_genes_by_counts', 'pct_counts_mt', 'pct_counts_ribo', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'log1p_total_counts_mt', 'log1p_total_counts_ribo', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id'
```

Example 39 (python):
```python
# modify modality columns in .obs
adata_scrna.obs['modality'] = 'scRNA'
adata_scrna.obs['modality'].value_counts()
```

Example 40 (unknown):
```unknown
scRNA    8820
Name: modality, dtype: int64
```

Example 41 (python):
```python
adata_scrna.var
```

Example 42 (python):
```python
# save
adata_scrna.write('/nfs/team205/heart/anndata_objects/Foetal/vCM/MultiVI-pre_scRNA_foetal_vCMs_raw.h5ad')
```

---

## Older analysis on tip stalk cells¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/RNA/scnRNA_Integration-Annotation/Endothelial/EndothelialTipInvestigation.html

**Contents:**
- Older analysis on tip stalk cells¶

**Examples:**

Example 1 (python):
```python
# Import packages, initial settings

import numpy as np
import matplotlib.pyplot as plt
import scanpy as sc
import pandas as pd
import anndata
import scrublet as scr
import seaborn as sns
import magic
import pybiomart
import six
import sys
sys.modules['sklearn.externals.six'] = six
import scanorama
import harmonypy
import bbknn
import gseapy
from gseapy.plot import barplot, dotplot
import scvelo as scv
import scvi

plt.rcParams['pdf.fonttype'] = 42
plt.rcParams['ps.fonttype'] = 42

sc.settings.verbosity = 3  # verbosity: errors (0), warnings (1), info (2), hints (3)

from matplotlib.colors import LinearSegmentedColormap
cmap = LinearSegmentedColormap.from_list(name='gene_cmap', colors=['lightgrey', 'thistle', 'red', 'darkred']) 

sc.settings.set_figure_params(dpi=80, color_map='viridis', vector_friendly=False,  dpi_save=300, facecolor='white')
sc.logging.print_version_and_date()
#sc.logging.print_versions()
```

Example 2 (unknown):
```unknown
Global seed set to 0
```

Example 3 (unknown):
```unknown
Running Scanpy 1.8.1, on 2023-06-01 14:25.
```

Example 4 (python):
```python
adata=sc.read('./Objects/Feb28ObjectRaw.h5ad')
print(adata.shape)
```

Example 5 (unknown):
```unknown
(297473, 36601)
```

Example 6 (python):
```python
cell_cycle_genes = ['MCM5','PCNA','TYMS','FEN1','MCM2','MCM4','RRM1','UNG','GINS2','MCM6','CDCA7','DTL','PRIM1','UHRF1','MLF1IP','HELLS','RFC2','RPA2','NASP','RAD51AP1','GMNN','WDR76','SLBP','CCNE2','UBR7','POLD3','MSH2','ATAD2','RAD51','RRM2','CDC45','CDC6','EXO1','TIPIN','DSCC1','BLM','CASP8AP2','USP1','CLSPN','POLA1','CHAF1B','BRIP1','E2F8','HMGB2','CDK1','NUSAP1','UBE2C','BIRC5','TPX2','TOP2A','NDC80','CKS2','NUF2','CKS1B','MKI67','TMPO','CENPF','TACC3','FAM64A','SMC4','CCNB2','CKAP2L','CKAP2','AURKB','BUB1','KIF11','ANP32E','TUBB4B','GTSE1','KIF20B','HJURP','CDCA3','HN1','CDC20','TTK','CDC25C','KIF2C','RANGAP1','NCAPD2','DLGAP5','CDCA2','CDCA8','ECT2','KIF23','HMMR','AURKA','PSRC1','ANLN','LBR','CKAP5','CENPE','CTCF','NEK2','G2E3','GAS2L3','CBX5','CENPA']
s_genes = cell_cycle_genes[:43]
s_genes = [x for x in s_genes if x in adata.var_names]
print('S genes:', s_genes, '\n')
g2m_genes = cell_cycle_genes[43:]
g2m_genes = [x for x in g2m_genes if x in adata.var_names]
print('G2 genes:', g2m_genes, '\n')
cell_cycle_genes = [x for x in cell_cycle_genes if x in adata.var_names]

hb_genes1 = ['HBB','HBD','HBG1','HBG2','HBE1','HBZ','HBM','HBA2','HBA1', 'HBQ1']
hb_genes1 = [x for x in hb_genes1 if x in adata.var_names]
print('Hemoglobin genes set 1:', hb_genes1, '\n')

hb_genes2 = ['AHSP','GYPA','RHAG','GFI1B','KEL','EPB42','ALAS2','TSPO2','GYPB','GYPE','SPTA1','RHCE','KLF1','RHD','SLC4A1','SNX22','SQSTM1','SOD1','FTH1','MAP1LC3B','SLC25A37','TRIM58','HMBS']
hb_genes2 = [x for x in hb_genes2 if x in adata.var_names]
print('Hemoglobin genes set 2:', hb_genes2, '\n')

mito_genes=['MT-ND1','MT-ND2','MT-CO1','MT-CO2','MT-ATP8','MT-ATP6','MT-CO3','MT-ND3','MT-ND4L', 'MT-ND4', 'MT-ND5', 'MT-ND6', 'MT-CYB']
mito_genes = [x for x in mito_genes if x in adata.var_names]
print('Mitochondrial genes:', mito_genes, '\n')

ribo_genes = ['RPLP0','RPLP1','RPLP2','RPL3','RPL3L','RPL4','RPL5','RPL6','RPL7','RPL7A','RPL7L1','RPL8','RPL9','RPL10','RPL10A','RPL10L','RPL11','RPL12','RPL13','RPL13A','RPL14','RPL15','RPL17','RPL18','RPL18A','RPL19','RPL21','RPL22','RPL22L1','RPL23','RPL23A','RPL24','RPL26','RPL26L1','RPL27','RPL27A','RPL28','RPL29','RPL30','RPL31','RPL32','RPL34','RPL35','RPL35A','RPL36','RPL36A','RPL36AL','RPL37','RPL37A','RPL38','RPL39','RPL39L','UBA52','RPL41','RPSA','RPS2','RPS3','RPS3A','RPS4X','RPS4Y1','RPS4Y2','RPS5','RPS6','RPS7','RPS8','RPS9','RPS10','RPS11','RPS12','RPS13','RPS14','RPS15','RPS15A','RPS16','RPS17','RPS18','RPS19','RPS20','RPS21','RPS23','RPS24','RPS25','RPS26','RPS27','RPS27A','RPS27L','RPS28','RPS29','FAU']
ribo_genes = [x for x in ribo_genes if x in adata.var_names]
print('Ribo genes:', ribo_genes, '\n')

stress_genes=['FOS','FOSB','JUN','JUNB','JUND','ZFP36','SOCS3','CEBPD','CEBPB','ATF3','EGR1','HSPA1A','HSPA1B','HSP90AA1','HSP90AB1','HSPA8','HSPB1','HSPE1','IER3','IER2','BTG1','BTG2','DUSP1']
stress_genes = [x for x in stress_genes if x in adata.var_names]
print('Tissue dissociation stress associated genes:', stress_genes, '\n')

FoxProteins=['FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1']
chrY_genes=['SRY', 'RPS4Y1', 'ZFY', 'ZFY-AS1', 'LINC00278', 'PCDH11Y', 'TTTY2B', 'TBL1Y', 'PRKY', 'USP9Y', 'DDX3Y', 'UTY', 'TMSB4Y', 'NLGN4Y', 'TTTY14', 'KDM5D', 'TTTY10', 'EIF1AY', 'PRORY', 'LINC00266-4P']
MembraneProteins1=['AADAT','AAK1','ABCA12','ABCA7','ABCB1','ABCB11','ABCB4','ABCB6','ABCC1','ABCC3','ABCC4','ABCC5','ABCC6','ABCG2','ABHD3','ABI1','ABLIM3','AC004691.2','AC004922.1','AC005324.3','AC005726.1','AC008403.1','AC138696.1','ACIN1','ACKR3','ACLY','ACOT13','ACOT9','ACTN1','ACTR10','ACTR6','ADA','ADAM10','ADAM12','ADAM22','ADAM28','ADAM33','ADAMTS1','ADAMTS2','ADAMTSL4','ADAP1','ADCK5','ADD1','ADD2','ADGRB2','ADGRD1','ADGRG2','ADGRG7','ADGRL2','ADH1A','ADH1B','ADH1C','ADH7','ADK','ADORA1','ADRA1B','ADRM1','ADSS','AFAP1L2','AFDN','AFG1L','AGER','AGRN','AHNAK','AHNAK2','AIFM2','AKAP10','AKAP11','AKAP12','AKR1B10','AKR1B15','AL132780.3','AL139300.1','AL355315.1','AL358113.1','AL645922.1','ALDH3A1','ALK','ALOX15B','ALOXE3','ALPI','ALPP','ALPPL2','ALS2CR12','AMACR','AMDHD1','AMER1','AMER2','AMIGO2','AMOT','AMOTL1','AMOTL2','AMPH','ANK2','ANK3','ANKEF1','ANKFN1','ANKIB1','ANKLE2','ANKRA2','ANKRD13A','ANKRD35','ANKRD42','ANKRD45','ANKS1B','ANO1','ANO10','ANO2','ANO6','ANPEP','ANXA1','ANXA13','ANXA2','ANXA3','AP2M1','APBB1','APBB1IP','APC','APLP1','APMAP','APOC3','APOD','APOLD1','AQP1','AQP10','AQP3','AQP4','AQP5','ARAP1','ARF1','ARHGAP18','ARHGAP19','ARHGAP21','ARHGAP22','ARHGAP23','ARHGAP24','ARHGAP28','ARHGAP29','ARHGAP33','ARHGAP36','ARHGAP40','ARHGAP9','ARHGDIG','ARHGEF1','ARHGEF11','ARHGEF12','ARHGEF15','ARHGEF26','ARHGEF28','ARHGEF40','ARHGEF5','ARID2','ARID4A','ARID5B','ARL13A','ARL15','ARL4C','ARMC12','ARMCX5','ARPC1A','ARPC5','ARRB2','ARRDC4','ARVCF','ASAP1','ASB5','ASGR1','ASGR2','ASH2L','ASIC1','ASIC4','ASPM','ASPSCR1','ATG13','ATG3','ATG7','ATIC','ATOX1','ATP10D','ATP12A','ATP13A3','ATP1B3','ATP23','ATP2B1','ATP2B4','ATP2C2','ATP6V0A2','ATP6V1H','ATP8A2','ATP8B1','ATXN10','ATXN3','ATXN7L3','AVEN','AVPI1','AXIN2','AXL','B2M','B3GALT2','BACE1','BAG3','BAIAP2','BAIAP2L1','BANK1','BASP1','BBOF1','BCAR1','BCAR3','BCL2L12','BCORL1','BCR','BDKRB2','BEND6','BFSP1','BFSP2','BICD2','BIN2','BLNK','BLVRB','BMPR2','BMX','BOC','BOLL','BORCS5','BORCS8-MEF2B','BRK1','BRPF1','BRSK1','BSDC1','BST2','BTK','BTN2A1','BTRC','BUB1B-PAK6','BVES','BZW2','C10orf90','C11orf42','C11orf52','C12orf73','C12orf76','C14orf37','C14orf93','C16orf74','C17orf58','C17orf62','C18orf21','C19orf33','C19orf38','C19orf48','C19orf71','C1orf116','C1orf189','C1orf35','C1orf54','C1orf68','C1orf74','C1QBP','C2orf72','C3orf14','C3orf20','C3orf38','C4orf19','C4orf22','C7orf43','C8orf33','C8orf37','C9orf153','C9orf3','CA14','CA9','CABLES2','CACNA1E','CACNG4','CADM1','CALCRL','CALD1','CALML5','CALN1','CAMK2A','CAMK2B','CAMK2D','CAMK2G','CAP2','CAPS','CASKIN2','CASP4','CAST','CAV3','CAVIN1','CAVIN2','CAVIN3','CAVIN4','CBWD1','CBWD2','CBWD3','CBWD5','CBWD6','CC2D1A','CCBE1','CCDC107','CCDC112','CCDC115','CCDC124','CCDC136','CCDC144NL','CCDC15','CCDC154','CCDC170','CCDC24','CCDC28A','CCDC59','CCDC62','CCDC66','CCDC69','CCDC8','CCDC83','CCDC85C','CCDC93','CCDC97','CCNT2','CCNYL1','CCR3','CCR4','CCT3','CD109','CD163','CD164L2','CD200R1','CD226','CD248','CD274','CD2AP','CD300A','CD300LB','CD33','CD34','CD37','CD38','CD3E','CD3G','CD4','CD44','CD46','CD47','CD48','CD79B','CD81','CD84','CD86','CD8A','CD8B','CD9','CD93','CD99L2','CDAN1','CDC42EP1','CDC42EP3','CDC42EP4','CDC42SE1','CDCA4','CDH1','CDH13','CDH15','CDH17','CDH2','CDH26','CDH3','CDH4','CDH5','CDH6','CDH8','CDIPT','CDK16','CDK5','CDK5RAP2','CDK7','CDRT15','CDV3','CEACAM5','CELSR1','CENPJ','CEP162','CEP295','CEP55','CEP57L1','CEP89','CERCAM','CFAP157','CFAP206','CFAP58','CFAP65','CFB','CFL2','CFLAR','CGN','CGNL1','CHCHD7','CHIC2','CHMP2A','CHRM3','CHRNA1','CHRNA5','CHRNB4','CHRND','CIB1','CIB3','CIP2A','CKAP5','CLCA2','CLCA4','CLCN5','CLCN6','CLDN11','CLDN18','CLDN2','CLDN23','CLDN3','CLDN4','CLDN7','CLDN9','CLEC2A','CLEC4F','CLEC5A','CLECL1','CLIC2','CLIC5','CLTB','CLUL1','CMTM4','CMTM7','CMTR2','CMYA5','CNGA1','CNKSR2','CNNM1','CNNM4','CNP','CNR1','CNTD1','COBL','COBLL1','COG3','COL16A1','COL17A1','COL4A5','COL6A2','COLEC12','COLQ','COMMD4','COMTD1','COPRS','COQ7','CORIN','CORO1B','CORO2A','CORO2B','COX6B2','CPEB3','CPPED1','CPZ','CRB3','CREB5','CRIP1','CRIP2','CRK','CRLF3','CROCC','CRTC1','CRYAB','CRYBG3','CRYL1','CSDE1','CSF1','CSF1R','CSPG4','CST6','CTNNA1','CTNNB1','CTNNBIP1','CTNND1','CTNND2','CTSF','CTSV','CTTN','CTXN1','CUEDC1','CUTA','CX3CL1','CX3CR1','CXADR','CXCR2','CYB5D2','CYBRD1','CYFIP2','CYP20A1','CYP24A1','CYTH2','CYTH4','DAAM1','DAB2','DAG1','DAGLB','DAW1','DBN1','DCAF13','DCAF16','DCBLD2','DCHS1','DCHS2','DCP2','DCT','DCTN6','DCUN1D2','DDAH1','DDB2','DDIAS','DDIT4L','DDR1','DDR2','DDX58','DDX60L','DENND6B','DFFA','DGKG','DGKH','DHDDS','DHPS','DHRS7B','DIAPH1','DIAPH3','DIO2','DIS3L','DLG1','DLG2','DLG5','DLGAP3','DLL3','DMWD','DNAAF1','DNAAF4','DNAH17','DNAJA4','DNAJB13','DNAJB4','DNAJC18','DNAJC5','DNAJC6','DNAJC9','DOC2A','DOCK4','DOK3','DONSON','DPEP1','DPH1','DPPA5','DPYSL2','DQX1','DRD1','DRP2','DSC3','DSEL','DSG2','DSP','DTNA','DUS1L','DUSP15','DXO','DYSF','ECSCR','EDA2R','EDNRA','EDNRB','EEF1AKMT1','EEF2','EFCAB1','EFCAB11','EFCAB2','EFCAB7','EFHC2','EFNB1','EFR3A','EFR3B','EGFR','EHBP1','EHD1','EHD2','EHD3','EHD4','EIF2B2','EIF5','EIF5B','ELFN1','ELMO3','ELN','ELOC','EMP3','ENAH','ENC1','ENG','ENKD1','ENO1','ENO2','ENO3','ENOX1','ENOX2','EPB41','EPB41L1','EPB41L2','EPB41L3','EPB41L4A','EPB41L4B','EPB41L5','EPCAM','EPHA2','EPHA3','EPHB1','EPHB2','EPN1','EPOR','EPS8L2','ERAP1','ERBB2','ERBB3','ERBIN','ERC1','ERCC4','ERMN','ESCO2','ETNK1','ETS2','EVA1A','EVI2B','EXOC1','EXOC3L1','EXOC7','EZR','F11R','F2RL1','F2RL3','FAAP20','FABP2','FABP3','FABP5','FAIM','FAM126A','FAM126B','FAM129A','FAM129B','FAM13A','FAM163B','FAM171A1','FAM193A','FAM205A','FAM210B','FAM222A','FAM45A','FAM47E','FAM47E-STBD1','FAM72A','FAM72B','FAM72C','FAM72D','FANCG','FAS','FASN','FBLIM1','FBLN2','FBLN5','FBLN7','FBP2','FBRSL1','FBXL2','FBXO16','FBXO38','FBXO41','FBXW11','FCAR','FCGBP','FCGR1A','FCGR1B','FCGR2A','FCHSD2','FFAR4','FGD1','FGD5','FGFR2','FGR','FHIT','FHL1','FIBCD1','FIGN','FILIP1','FILIP1L','FKBP2','FLAD1','FLCN','FLNA','FLNB','FLNC','FLOT1','FLOT2','FLRT3','FLT1','FLT4','FLVCR1','FMN2','FMNL2','FMNL3','FNBP1','FOXA2','FOXN3','FP565260.3','FRMD3','FRMD7','FRMD8','FRS2','FSCN1','FTCD','FUT2','FXYD3','FYB1','FYN','FZD1','FZD2','FZD4','FZD6','GAB1','GAB2','GABRA1','GABRA2','GABRA3','GABRA5','GABRG1','GAD1','GADL1','GAGE1','GAGE10','GAGE12C','GAGE12D','GAGE12E','GAGE12F','GAGE12G','GAGE12H','GAGE12J','GAGE13','GAGE2A','GAGE2E','GALNT10','GALR1','GAP43','GAPDH','GAPT','GAPVD1','GAS2','GAS2L1','GAS7','GAS8','GBP4','GCA','GCC1','GCLM','GCSAML-AS1','GFI1B','GFOD1','GFRA3','GHR','GID8','GINM1','GIPC1','GJA1','GJB3','GJB4','GJB6','GJB7','GJC1','GJC2','GLO1','GLP2R','GLRX','GLUL','GMIP','GNAI2','GNAQ','GNAS','GNB1','GNB2','GNB3','GNB4','GNG2','GP9','GPAT3','GPATCH1','GPBP1','GPBP1L1','GPC1','GPC3','GPC4','GPI','GPR1','GPR142','GPR156','GPR157','GPR160','GPR173','GPR20','GPR27','GPR50','GPR63','GPRC5A','GPRIN1','GPRIN2','GPX8','GRAMD1A','GRAMD1B','GRAMD1C','GRAP2','GRB7','GRID2IP','GRIK5','GRK3','GRK5','GRPR','GSDMA','GTF2F1','GTSE1','GUCA1A','GYPA','GYPC','HAPLN3','HARBI1','HAS1','HAUS4','HAUS7','HCAR2','HCAR3','HCFC2','HCK','HCLS1','HCST','HDAC8','HEG1','HELZ','HENMT1','HERC2','HERPUD1','HFE','HIF1A','HIF3A','HINT1','HIP1R','HLA-A','HLA-DPB1','HMGCS1','HMOX1','HOMER3','HOXC5','HOXD4','HPS3','HRASLS5','HRH1','HS6ST3','HSH2D','HSPB1','HSPG2','HTR1B','HTR7','HTRA1','HUNK','HYDIN','HYI','HYLS1','ICAM1','ICAM2','ICOS','ICOSLG','IFFO2','IFIT5','IFNGR1','IFRD1','IFT122','IGFL2','IGSF11','IGSF23','IKZF5','IL10RA','IL12RB1','IL12RB2','IL13','IL13RA2','IL16','IL17RB','IL1R1','IL1RL1','IL21R','IL4R','IL6ST','IL7R','ILDR2','ILK','INPP5A','INSR','INTS12','IP6K2','IPCEF1','IQGAP1','IQGAP2','IREB2','IRF2','IRS1','ISLR','ITGA2B','ITGA3','ITGA4','ITGA7','ITGA8','ITGA9','ITGB1','ITGB2','ITGB3','ITGB4','ITGB5','ITGB6','ITGB7','ITGB8','ITK','ITM2A','ITPKA','ITSN1','ITSN2','IYD','JAG1','JAK2','JAK3','JAM2','JAML','JCAD','JMJD4','JPT2','JUP','KANK1','KANK3','KANSL2','KARS','KATNB1','KCNAB1','KCNB2','KCNH8','KCNJ15','KCNK13','KCNK17','KCNK6','KCNK7','KCNK9','KCNN2','KCNS3','KCNU1','KDF1','KDR','KEL','KIAA0556','KIAA0895','KIAA1324','KIAA1522','KIAA1958','KIF17','KIF19','KIF21A','KIF21B','KIF26A','KIF26B','KISS1R','KIT','KLC1','KLC2','KLF9','KLHDC7B','KLHL41','KLHL6','KLHL7','KLK7','KLRC1','KLRC2','KLRF1','KLRK1','KMT2D','KNDC1','KNSTRN','KNTC1','KPTN','KRT75','KRT77','KRTAP25-1','L1CAM','L3MBTL1','LAIR1','LAMA4','LAMC1','LAMP2','LAMTOR1','LAPTM4A','LASP1','LAT','LAT2','LAX1','LCP1','LCTL','LDB2','LDLRAD3','LENG1','LGALS3','LGALS4','LGALS8','LGR6','LHFPL2','LHX6','LIMA1','LINGO1','LIPJ','LMCD1','LMOD2','LMTK2','LMTK3','LNX1','LNX2','LONRF1','LONRF3','LOXHD1','LPAR1','LPAR3','LPAR4','LPIN1','LPIN2','LRCH2','LRP11','LRP12','LRRC15','LRRC26','LRRC34','LRRC45','LRRC69','LRRC7','LRRC9','LRRFIP2','LRRIQ3','LSMEM2','LSR','LTB4R2','LUZP2','LY6K','LYN','LYPD1','LYST','LZTS1','LZTS2','MACF1','MADD','MAGI1','MAGI3','MAMDC2','MAP1S','MAP2','MAP2K1','MAP2K4','MAP2K7','MAP3K12','MAP3K21','MAP3K6','MAP3K7','MAP4','MAP4K1','MAP4K5','MAP6','MAPK15','MAPK8IP1','MAPT','MARCH10','MARCH7','MARCKS','MARK2','MARVELD2','MAS1L','MAT2B','MBP','MC5R','MCAM','MCEMP1','MCF2L2','MCMBP','MCTS1','MDGA1','ME1','MEF2B','MEIOC','MEPCE','MERTK','MET','METAP1','METAP2','METTL15','METTL25','METTL9','MFAP3L','MFSD1','MFSD2A','MGAT4C','MIB1','MICAL2','MICAL3','MICALL1','MICB','MIEN1','MIGA2','MISP','MKLN1','MKRN3','MLF2','MLKL','MLST8','MMACHC','MMP10','MMP15','MMP24','MMP25','MOAP1','MPHOSPH8','MPHOSPH9','MPI','MPIG6B','MPL','MPP1','MPP5','MPP7','MPZL1','MPZL2','MR1','MRGPRF','MRNIP','MRPL42','MRPL44','MRPL46','MRPS18B','MS4A1','MS4A12','MS4A14','MS4A15','MSN','MSRA','MTCP1','MTFR1L','MTHFR','MTM1','MTPN','MTSS1','MUC1','MUC20','MVD','MYH11','MYH9','MYL4','MYLK','MYO10','MYO1B','MYO1C','MYO1E','MYO1G','MYO3A','MYO5B','MYO9A','MYOF','N4BP2','NAA35','NAB1','NAIF1','NAMPT','NAP1L2','NAPG','NARFL','NCAM1','NCAM2','NCK1','NCKAP1L','NCKAP5','NCKIPSD','NCOA1','NCS1','NDC1','NECTIN2','NECTIN3','NECTIN4','NEDD1','NEDD9','NEO1','NEU1','NEXN','NF2','NFE2L2','NFKBID','NGEF']
MembraneProteins2=['NGFR','NHS','NHSL2','NID2','NINJ2','NKTR','NKX2-8','NLGN1','NLGN3','NLK','NLRX1','NMB','NMBR','NME1','NME2','NMT1','NMT2','NMUR2','NOS1','NOTCH2','NPFFR2','NPL','NPNT','NPR1','NPTN','NPTXR','NR2C1','NRAS','NRCAM','NRP2','NRSN1','NRXN1','NSD1','NSFL1C','NT5E','NTNG1','NTNG2','NTSR1','NUDCD3','NUDT16L1','NUDT9','NUMB','NUP35','NUS1','OAS3','OBSCN','OCA2','OCLN','OCM','OCM2','ODC1','ODR4','OGT','OLFM4','OLIG2','OLR1','OPHN1','OPRK1','OR10P1','OR1N2','OR2C3','OR6B2','OR6B3','OR6C6','OR8S1','OSBPL10','OSBPL6','OSBPL7','OSCAR','OSCP1','OSGEP','OSR2','OTOP2','OTULIN','OXGR1','P3H3','PACSIN2','PACSIN3','PAFAH1B2','PAG1','PAIP1','PAK1','PAK4','PAK6','PALLD','PALM','PALM2','PALM2-AKAP2','PALM3','PALMD','PANX1','PANX2','PAQR8','PARD3','PARD3B','PARD6A','PARD6G','PARP6','PATJ','PAWR','PCBD2','PCDH1','PCDH17','PCDH7','PCDHA1','PCDHA11','PCDHA12','PCDHA4','PCDHA5','PCDHB1','PCDHB11','PCDHB12','PCDHB16','PCDHB4','PCDHB5','PCDHGA1','PCDHGA10','PCDHGA11','PCDHGA12','PCDHGA2','PCDHGA3','PCDHGA4','PCDHGA5','PCDHGA6','PCDHGA7','PCDHGA8','PCDHGA9','PCDHGB1','PCDHGB2','PCDHGB3','PCDHGB4','PCDHGB5','PCDHGB6','PCDHGB7','PCDHGC3','PCDHGC4','PCDHGC5','PCLO','PCMTD1','PDAP1','PDCD7','PDE3A','PDE4A','PDE4D','PDE9A','PDGFC','PDGFRA','PDHX','PDLIM1','PDLIM5','PDPN','PDZD2','PDZD8','PEAR1','PEBP1','PFDN1','PGAP3','PGLYRP2','PGLYRP4','PGRMC2','PHACTR1','PHACTR4','PHF20L1','PHGDH','PHLDB2','PHLDB3','PHOSPHO1','PHPT1','PI3','PI4K2A','PI4KA','PIEZO2','PIGB','PIGN','PIGW','PIGX','PIK3AP1','PIK3CG','PIK3R6','PIP4K2A','PIP4K2C','PIP5K1A','PIRT','PITPNM3','PKD2L2','PKN1','PKN2','PKP1','PKP2','PKP3','PKP4','PLAA','PLAUR','PLCB1','PLCB2','PLCD3','PLCD4','PLEKHA4','PLEKHA6','PLEKHA7','PLEKHG3','PLEKHG4','PLEKHG6','PLEKHH2','PLEKHO2','PLIN4','PLPP1','PLPPR5','PLS3','PLSCR1','PLVAP','PLXNA3','PLXNA4','PNCK','PNISR','PODXL','POLE','POU2F3','PPEF1','PPFIBP1','PPIP5K1','PPL','PPM1A','PPM1F','PPP1CA','PPP1CB','PPP1R12B','PPP1R16A','PPP1R16B','PPP1R18','PPP1R27','PPP1R9A','PPP2R1B','PPP3R1','PPP3R2','PPP4C','PPP6R3','PPTC7','PRAME','PRC1','PRDX6','PRIMA1','PRKCA','PRKCE','PRKCH','PRKCZ','PRKD1','PROM2','PROSER2','PRR15','PRR32','PRR7','PRRC2A','PRRG1','PRRT2','PRRT3','PRRT4','PRTG','PRX','PSCA','PSD','PSD2','PSEN1','PSMC5','PSMC6','PSMD9','PSTPIP2','PTAFR','PTCHD4','PTK2B','PTK6','PTPN12','PTPN21','PTPN22','PTPN4','PTPRB','PTPRG','PTPRK','PTPRM','PTPRR','PTPRS','PTX3','PURG','PVALB','PVR','PXDC1','PXK','PYGL','PYGO2','PYM1','RAB13','RAB23','RAB25','RAB26','RAB27B','RAB3B','RAB3GAP2','RAB43','RAB8A','RABEPK','RAD51C','RAD52','RAI2','RALA','RALGAPA2','RAPGEF2','RAPH1','RAPSN','RASA4','RASA4B','RASAL1','RASAL2','RASAL3','RASD1','RASD2','RASGRP1','RASGRP2','RASGRP4','RASSF2','RASSF3','RASSF4','RBM46','RC3H1','RCCD1','RCSD1','RD3L','RDX','RECK','RELL1','REPS1','RET','RFLNB','RFPL1','RFPL2','RFPL3','RGP1','RGS10','RGS14','RGS19','RGS3','RGS7BP','RGS9BP','RHCE','RHD','RHEX','RHOA','RHOBTB2','RHOJ','RHOXF2','RHOXF2B','RIC8A','RIC8B','RIF1','RILPL1','RIMBP3','RIMBP3B','RIMBP3C','RIMS2','RIOK2','RIOK3','RIPK1','RND1','RNF114','RNF145','RNF146','RNF149','RNF165','RNF166','RNF39','ROBO1','ROBO3','ROBO4','ROM1','RP2','RPE','RPEL1','RPGRIP1L','RPIA','RPL36A','RPL36A-HNRNPH2','RPL36AL','RPSA','RPUSD1','RRAD','RRP12','RRP7A','RSBN1L','RSC1A1','RSPH3','RTL3','RTL4','RTN4R','RTN4RL1','RYR2','S100A13','S100A14','S100A16','S100A2','S100A3','S100A4','S100A6','S100A9','SAMD4A','SAMHD1','SAMSN1','SAPCD1','SASH3','SC5D','SCAMP1','SCAMP4','SCEL','SCHIP1','SCIN','SCN1A','SCN5A','SCN8A','SCN9A','SCNN1D','SCNN1G','SCRIB','SCUBE3','SDC2','SDC4','SDE2','SDHB','SEC16B','SEMA3C','SEMA3D','SEMA4B','SEMA4F','SEMA6C','SEMA6D','SEPT2','SEPT3','SERINC2','SERINC5','SERPINB3','SFSWAP','SGCE','SGK494','SH3BGR','SH3D21','SHANK1','SHANK3','SHARPIN','SHISA6','SHKBP1','SHROOM2','SHTN1','SIGLEC10','SIGLEC14','SIGLEC6','SIGLEC7','SIN3B','SIPA1','SIPA1L1','SIPA1L3','SKAP1','SLAMF6','SLC10A5','SLC12A2','SLC13A3','SLC13A5','SLC14A1','SLC16A1','SLC16A10','SLC16A2','SLC16A3','SLC16A7','SLC16A9','SLC17A5','SLC19A1','SLC19A3','SLC1A5','SLC1A6','SLC22A16','SLC23A3','SLC24A4','SLC26A9','SLC27A1','SLC28A1','SLC2A1','SLC2A11','SLC2A12','SLC2A14','SLC2A2','SLC2A3','SLC30A1','SLC31A2','SLC34A1','SLC34A2','SLC35F5','SLC35G2','SLC36A4','SLC38A5','SLC38A6','SLC38A7','SLC39A10','SLC39A14','SLC39A2','SLC39A6','SLC3A2','SLC40A1','SLC41A3','SLC43A1','SLC43A3','SLC44A2','SLC45A4','SLC46A1','SLC46A3','SLC4A1AP','SLC4A2','SLC4A7','SLC4A8','SLC5A12','SLC5A3','SLC5A6','SLC5A7','SLC6A6','SLC7A14','SLC7A2','SLC7A5','SLC7A6','SLC8A1','SLC9A1','SLC9A2','SLC9A3','SLC9A3R1','SLC9A3R2','SLC9A4','SLC9A5','SLC9B2','SLC9C2','SLCO1A2','SLCO1B3','SLCO4A1','SLCO4C1','SLCO5A1','SLIT1','SLIT3','SLITRK6','SLK','SLX4','SMAGP','SMAP1','SMOC1','SMPD2','SMPX','SNAP23','SNAP25','SNRK','SNTB2','SNTG2','SNX15','SNX32','SNX9','SOCS3','SOCS5','SOD1','SORBS1','SORCS1','SORT1','SOX11','SPAG16','SPANXN4','SPART','SPATA12','SPATC1L','SPERT','SPG11','SPHK1','SPINK13','SPINT1','SPINT2','SPN','SPPL2B','SPPL3','SPRED3','SPRYD3','SPTBN2','SRC','SRCIN1','SRD5A3','SRPK1','SRPRB','SRRM3','SSFA2','SSH1','SSH3','SSTR1','SSTR5','SSX2IP','ST8SIA5','STAB1','STAC','STAC2','STAMBP','STAMBPL1','STARD4','STAT2','STEAP4','STIL','STIP1','STK10','STK16','STK17A','STK17B','STOM','STOML2','STOML3','STX18','STX4','STXBP4','STYK1','SUCLG1','SUCLG2','SUGT1','SUMF2','SUPT6H','SUSD2','SVIL','SWAP70','SYCP2','SYK','SYMPK','SYNC','SYNGR3','SYNPO2L','SYPL2','SYT4','SYTL2','SYVN1','TACC2','TACR2','TACSTD2','TASP1','TAZ','TBC1D10A','TBC1D2','TBC1D24','TBC1D30','TBC1D9B','TBXA2R','TCAF2','TCAF2C','TCHP','TCIM','TDG','TDO2','TDP1','TEC','TEDC2','TEK','TES','TEX14','TEX9','TGFBR1','TGFBR2','TGM1','TGM2','THBS1','THBS2','THEGL','THY1','TIAM1','TICAM1','TIGD7','TIMD4','TJP1','TJP2','TJP3','TLN1','TLR4','TM2D2','TM4SF20','TM4SF4','TM4SF5','TMC5','TMED8','TMEM101','TMEM107','TMEM108','TMEM121B','TMEM139','TMEM141','TMEM164','TMEM176B','TMEM183A','TMEM186','TMEM2','TMEM218','TMEM222','TMEM229A','TMEM245','TMEM251','TMEM254','TMEM258','TMEM266','TMEM268','TMEM50A','TMEM63B','TMPRSS15','TMPRSS2','TMTC2','TNFRSF10D','TNFRSF11A','TNFRSF12A','TNFRSF21','TNFRSF25','TNFRSF8','TNFRSF9','TNFSF13B','TNK1','TNKS1BP1','TNNI2','TOGARAM2','TOLLIP','TOM1','TP53BP2','TPD52L1','TPRN','TRAF7','TRAIP','TRAPPC13','TRAT1','TRDN','TREML1','TRIB1','TRIM16','TRIM16L','TRIM38','TRIM4','TRIM44','TRIM54','TRIM71','TRIML2','TRIP6','TRMT1','TRMT10B','TRMT13','TRMT6','TRO','TRPM1','TRPM4','TRPM7','TRPV2','TRPV3','TSG101','TSGA13','TSHZ3','TSPAN15','TSPAN6','TSPO2','TSR2','TSSK4','TTC17','TTC23','TTC7B','TTLL12','TTLL5','TULP3','TWF2','TXNDC11','TXNIP','TYROBP','TYW3','UAP1','UBA3','UBA52','UBAC1','UBALD1','UBAP1','UBASH3B','UBE2A','UBE2B','UBE2C','UBE2D1','UBE2D2','UBE2D3','UBE2D4','UBE2K','UBE2QL1','UBE2S','UBE3C','UBQLN2','UBR2','UBTD1','UCP3','ULBP1','ULBP2','UNC5A','UNC79','UQCC1','USP14','USP15','USP2','USP20','USP21','USP4','USP53','UTP20','UTRN','VAMP5','VAMP8','VANGL1','VASP','VCAM1','VCPIP1','VIL1','VPS13B','VPS36','VPS50','VPS8','VSIG4','VTCN1','VWA5B1','VWA7','WAS','WASF2','WBP4','WDFY3','WDPCP','WDR1','WDR12','WDR13','WDR53','WDR6','WDR66','WIPF1','WWP1','XIAP','XIRP1','XIRP2','XPC','XPNPEP2','XPO6','XRN1','XYLT1','YES1','YIPF1','YIPF4','YOD1','YTHDC1','ZBED3','ZBTB33','ZBTB8OS','ZDHHC2','ZDHHC20','ZDHHC22','ZDHHC5','ZFP41','ZFP90','ZFYVE28','ZGPAT','ZMYND10','ZNF10','ZNF18','ZNF185','ZNF2','ZNF20','ZNF227','ZNF229','ZNF236','ZNF276','ZNF277','ZNF337','ZNF397','ZNF407','ZNF497','ZNF503','ZNF510','ZNF536','ZNF571','ZNF598','ZNF599','ZNF646','ZNF655','ZNF688','ZNF706','ZNF717','ZNF783','ZNF79','ZNF839','ZSCAN22','ZSCAN29','ZSWIM1','ZSWIM8','ZYX']
MembraneProteins=MembraneProteins1+MembraneProteins2

TFs1=['AC008770.3','AC023509.3','AC092835.1','AC138696.1','ADNP','ADNP2','AEBP1','AEBP2','AHCTF1','AHDC1','AHR','AHRR','AIRE','AKAP8','AKAP8L','AKNA','ALX1','ALX3','ALX4','ANHX','ANKZF1','AR','ARGFX','ARHGAP35','ARID2','ARID3A','ARID3B','ARID3C','ARID5A','ARID5B','ARNT','ARNT2','ARNTL','ARNTL2','ARX','ASCL1','ASCL2','ASCL3','ASCL4','ASCL5','ASH1L','ATF1','ATF2','ATF3','ATF4','ATF5','ATF6','ATF6B','ATF7','ATMIN','ATOH1','ATOH7','ATOH8','BACH1','BACH2','BARHL1','BARHL2','BARX1','BARX2','BATF','BATF2','BATF3','BAZ2A','BAZ2B','BBX','BCL11A','BCL11B','BCL6','BCL6B','BHLHA15','BHLHA9','BHLHE22','BHLHE23','BHLHE40','BHLHE41','BNC1','BNC2','BORCS8-MEF2B','BPTF','BRF2','BSX','C11orf95','CAMTA1','CAMTA2','CARF','CASZ1','CBX2','CC2D1A','CCDC169-SOHLH2','CCDC17','CDC5L','CDX1','CDX2','CDX4','CEBPA','CEBPB','CEBPD','CEBPE','CEBPG','CEBPZ','CENPA','CENPB','CENPBD1','CENPS','CENPT','CENPX','CGGBP1','CHAMP1','CHCHD3','CIC','CLOCK','CPEB1','CPXCR1','CREB1','CREB3','CREB3L1','CREB3L2','CREB3L3','CREB3L4','CREB5','CREBL2','CREBZF','CREM','CRX','CSRNP1','CSRNP2','CSRNP3','CTCF','CTCFL','CUX1','CUX2','CXXC1','CXXC4','CXXC5','DACH1','DACH2','DBP','DBX1','DBX2','DDIT3','DEAF1','DLX1','DLX2','DLX3','DLX4','DLX5','DLX6','DMBX1','DMRT1','DMRT2','DMRT3','DMRTA1','DMRTA2','DMRTB1','DMRTC2','DMTF1','DNMT1','DNTTIP1','DOT1L','DPF1','DPF3','DPRX','DR1','DRAP1','DRGX','DUX1','DUX3','DUX4','DUXA','DZIP1','E2F1','E2F2','E2F3','E2F4','E2F5','E2F6','E2F7','E2F8','E4F1','EBF1','EBF2','EBF3','EBF4','EEA1','EGR1','EGR2','EGR3','EGR4','EHF','ELF1','ELF2','ELF3','ELF4','ELF5','ELK1','ELK3','ELK4','EMX1','EMX2','EN1','EN2','EOMES','EPAS1','ERF','ERG','ESR1','ESR2','ESRRA','ESRRB','ESRRG','ESX1','ETS1','ETS2','ETV1','ETV2','ETV3','ETV3L','ETV4','ETV5','ETV6','ETV7','EVX1','EVX2','FAM170A','FAM200B','FBXL19','FERD3L','FEV','FEZF1','FEZF2','FIGLA','FIZ1','FLI1','FLYWCH1','FOS','FOSB','FOSL1','FOSL2','FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXN4','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1','GABPA','GATA1','GATA2','GATA3','GATA4','GATA5','GATA6','GATAD2A','GATAD2B','GBX1','GBX2','GCM1','GCM2','GFI1','GFI1B','GLI1','GLI2','GLI3','GLI4','GLIS1','GLIS2','GLIS3','GLMP','GLYR1','GMEB1','GMEB2','GPBP1','GPBP1L1','GRHL1','GRHL2','GRHL3','GSC','GSC2','GSX1','GSX2','GTF2B','GTF2I','GTF2IRD1','GTF2IRD2','GTF2IRD2B','GTF3A','GZF1','HAND1','HAND2','HBP1','HDX','HELT','HES1','HES2','HES3','HES4','HES5','HES6','HES7','HESX1','HEY1','HEY2','HEYL','HHEX','HIC1','HIC2','HIF1A','HIF3A','HINFP','HIVEP1','HIVEP2','HIVEP3','HKR1','HLF','HLX','HMBOX1','HMG20A','HMG20B','HMGA1','HMGA2','HMGN3','HMX1','HMX2','HMX3','HNF1A','HNF1B','HNF4A','HNF4G','HOMEZ','HOXA1','HOXA10','HOXA11','HOXA13','HOXA2','HOXA3','HOXA4','HOXA5','HOXA6','HOXA7','HOXA9','HOXB1','HOXB13','HOXB2','HOXB3','HOXB4','HOXB5','HOXB6','HOXB7','HOXB8','HOXB9','HOXC10','HOXC11','HOXC12','HOXC13','HOXC4','HOXC5','HOXC6','HOXC8','HOXC9','HOXD1','HOXD10','HOXD11','HOXD12','HOXD13','HOXD3','HOXD4','HOXD8','HOXD9','HSF1','HSF2','HSF4','HSF5','HSFX1','HSFX2','HSFY1','HSFY2','IKZF1','IKZF2','IKZF3','IKZF4','IKZF5','INSM1','INSM2','IRF1','IRF2','IRF3','IRF4','IRF5','IRF6','IRF7','IRF8','IRF9','IRX1','IRX2','IRX3','IRX4','IRX5','IRX6','ISL1','ISL2','ISX','JAZF1','JDP2','JRK','JRKL','JUN','JUNB','JUND','KAT7','KCMF1','KCNIP3','KDM2A','KDM2B','KDM5B','KIN','KLF1','KLF10','KLF11','KLF12','KLF13','KLF14','KLF15','KLF16','KLF17','KLF2','KLF3','KLF4','KLF5','KLF6','KLF7','KLF8','KLF9','KMT2A','KMT2B','L3MBTL1','L3MBTL3','L3MBTL4','LBX1','LBX2','LCOR','LCORL','LEF1','LEUTX','LHX1','LHX2','LHX3','LHX4','LHX5','LHX6','LHX8','LHX9','LIN28A','LIN28B','LIN54','LMX1A','LMX1B','LTF','LYL1','MAF','MAFA','MAFB','MAFF','MAFG','MAFK','MAX','MAZ','MBD1','MBD2','MBD3','MBD4','MBD6','MBNL2','MECOM','MECP2','MEF2A','MEF2B','MEF2C','MEF2D','MEIS1','MEIS2','MEIS3','MEOX1','MEOX2','MESP1','MESP2','MGA','MITF','MIXL1','MKX','MLX','MLXIP','MLXIPL','MNT','MNX1','MSANTD1','MSANTD3','MSANTD4','MSC','MSGN1','MSX1','MSX2','MTERF1','MTERF2','MTERF3','MTERF4','MTF1','MTF2','MXD1','MXD3','MXD4','MXI1','MYB','MYBL1','MYBL2','MYC','MYCL','MYCN','MYF5','MYF6','MYNN','MYOD1','MYOG','MYPOP','MYRF','MYRFL','MYSM1','MYT1','MYT1L','MZF1','NACC2','NAIF1','NANOG','NANOGNB','NANOGP8','NCOA1','NCOA2','NCOA3','NEUROD1','NEUROD2','NEUROD4','NEUROD6','NEUROG1','NEUROG2','NEUROG3','NFAT5','NFATC1','NFATC2','NFATC3','NFATC4','NFE2','NFE2L1','NFE2L2','NFE2L3','NFE4','NFIA','NFIB','NFIC','NFIL3','NFIX','NFKB1','NFKB2','NFX1','NFXL1','NFYA','NFYB','NFYC','NHLH1','NHLH2','NKRF','NKX1-1','NKX1-2','NKX2-1','NKX2-2','NKX2-3','NKX2-4','NKX2-5','NKX2-6','NKX2-8','NKX3-1','NKX3-2','NKX6-1','NKX6-2','NKX6-3','NME2','NOBOX','NOTO','NPAS1','NPAS2','NPAS3','NPAS4','NR0B1','NR1D1','NR1D2','NR1H2','NR1H3','NR1H4','NR1I2','NR1I3','NR2C1','NR2C2','NR2E1','NR2E3','NR2F1','NR2F2','NR2F6','NR3C1','NR3C2','NR4A1','NR4A2','NR4A3','NR5A1','NR5A2','NR6A1','NRF1','NRL','OLIG1','OLIG2','OLIG3','ONECUT1','ONECUT2','ONECUT3','OSR1','OSR2','OTP','OTX1','OTX2','OVOL1','OVOL2','OVOL3','PA2G4','PATZ1','PAX1','PAX2','PAX3','PAX4','PAX5','PAX6','PAX7','PAX8','PAX9','PBX1','PBX2','PBX3','PBX4','PCGF2','PCGF6','PDX1','PEG3','PGR','PHF1','PHF19 ','PHF20','PHF21A','PHOX2A','PHOX2B','PIN1','PITX1','PITX2','PITX3','PKNOX1','PKNOX2','PLAG1','PLAGL1','PLAGL2','PLSCR1','POGK','POU1F1','POU2AF1','POU2F1','POU2F2','POU2F3','POU3F1','POU3F2','POU3F3','POU3F4','POU4F1','POU4F2','POU4F3','POU5F1','POU5F1B','POU5F2','POU6F1','POU6F2','PPARA','PPARD','PPARG','PRDM1','PRDM10','PRDM12','PRDM13','PRDM14','PRDM15','PRDM16','PRDM2','PRDM4','PRDM5','PRDM6','PRDM8','PRDM9','PREB','PRMT3','PROP1','PROX1','PROX2','PRR12','PRRX1','PRRX2','PTF1A','PURA','PURB','PURG','RAG1','RARA','RARB','RARG','RAX','RAX2','RBAK','RBCK1','RBPJ','RBPJL','RBSN','REL','RELA','RELB','REPIN1','REST','REXO4','RFX1','RFX2','RFX3','RFX4','RFX5','RFX6','RFX7','RFX8','RHOXF1','RHOXF2','RHOXF2B','RLF','RORA','RORB','RORC','RREB1','RUNX1','RUNX2','RUNX3','RXRA','RXRB','RXRG','SAFB','SAFB2','SALL1','SALL2','SALL3','SALL4','SATB1','SATB2','SCMH1','SCML4','SCRT1','SCRT2','SCX','SEBOX','SETBP1','SETDB1','SETDB2','SGSM2','SHOX','SHOX2','SIM1','SIM2','SIX1','SIX2','SIX3','SIX4','SIX5','SIX6','SKI','SKIL','SKOR1','SKOR2','SLC2A4RG','SMAD1','SMAD3','SMAD4','SMAD5','SMAD9','SMYD3','SNAI1','SNAI2','SNAI3','SNAPC2','SNAPC4','SNAPC5','SOHLH1','SOHLH2','SON','SOX1','SOX10','SOX11']
TFs2=['SOX12','SOX13','SOX14','SOX15','SOX17','SOX18','SOX2','SOX21','SOX3','SOX30','SOX4','SOX5','SOX6','SOX7','SOX8','SOX9','SP1','SP100','SP110','SP140','SP140L','SP2','SP3','SP4','SP5','SP6','SP7','SP8','SP9','SPDEF','SPEN','SPI1','SPIB','SPIC','SPZ1','SRCAP','SREBF1','SREBF2','SRF','SRY','ST18','STAT1','STAT2','STAT3','STAT4','STAT5A','STAT5B','STAT6','T','TAL1','TAL2','TBP','TBPL1','TBPL2','TBR1','TBX1','TBX10','TBX15','TBX18','TBX19','TBX2','TBX20','TBX21','TBX22','TBX3','TBX4','TBX5','TBX6','TCF12','TCF15','TCF20','TCF21','TCF23','TCF24','TCF3','TCF4','TCF7','TCF7L1','TCF7L2','TCFL5','TEAD1','TEAD2','TEAD3','TEAD4','TEF','TERB1','TERF1','TERF2','TET1','TET2','TET3','TFAP2A','TFAP2B','TFAP2C','TFAP2D','TFAP2E','TFAP4','TFCP2','TFCP2L1','TFDP1','TFDP2','TFDP3','TFE3','TFEB','TFEC','TGIF1','TGIF2','TGIF2LX','TGIF2LY','THAP1','THAP10','THAP11','THAP12','THAP2','THAP3','THAP4','THAP5','THAP6','THAP7','THAP8','THAP9','THRA','THRB','THYN1','TIGD1','TIGD2','TIGD3','TIGD4','TIGD5','TIGD6','TIGD7','TLX1','TLX2','TLX3','TMF1','TOPORS','TP53','TP63','TP73','TPRX1','TRAFD1','TRERF1','TRPS1','TSC22D1','TSHZ1','TSHZ2','TSHZ3','TTF1','TWIST1','TWIST2','UBP1','UNCX','USF1','USF2','USF3','VAX1','VAX2','VDR','VENTX','VEZF1','VSX1','VSX2','WIZ','WT1','XBP1','XPA','YBX1','YBX2','YBX3','YY1','YY2','ZBED1','ZBED2','ZBED3','ZBED4','ZBED5','ZBED6','ZBED9','ZBTB1','ZBTB10','ZBTB11','ZBTB12','ZBTB14','ZBTB16','ZBTB17','ZBTB18','ZBTB2','ZBTB20','ZBTB21','ZBTB22','ZBTB24','ZBTB25','ZBTB26','ZBTB3','ZBTB32','ZBTB33','ZBTB34','ZBTB37','ZBTB38','ZBTB39','ZBTB4','ZBTB40','ZBTB41','ZBTB42','ZBTB43','ZBTB44','ZBTB45','ZBTB46','ZBTB47','ZBTB48','ZBTB49','ZBTB5','ZBTB6','ZBTB7A','ZBTB7B','ZBTB7C','ZBTB8A','ZBTB8B','ZBTB9','ZC3H8','ZEB1','ZEB2','ZFAT','ZFHX2','ZFHX3','ZFHX4','ZFP1','ZFP14','ZFP2','ZFP28','ZFP3','ZFP30','ZFP37','ZFP41','ZFP42','ZFP57','ZFP62','ZFP64','ZFP69','ZFP69B','ZFP82','ZFP90','ZFP91','ZFP92','ZFPM1','ZFPM2','ZFX','ZFY','ZGLP1','ZGPAT','ZHX1','ZHX2','ZHX3','ZIC1','ZIC2','ZIC3','ZIC4','ZIC5','ZIK1','ZIM2','ZIM3','ZKSCAN1','ZKSCAN2','ZKSCAN3','ZKSCAN4','ZKSCAN5','ZKSCAN7','ZKSCAN8','ZMAT1','ZMAT4','ZNF10','ZNF100','ZNF101','ZNF107','ZNF112','ZNF114','ZNF117','ZNF12','ZNF121','ZNF124','ZNF131','ZNF132','ZNF133','ZNF134','ZNF135','ZNF136','ZNF138','ZNF14','ZNF140','ZNF141','ZNF142','ZNF143','ZNF146','ZNF148','ZNF154','ZNF155','ZNF157','ZNF16','ZNF160','ZNF165','ZNF169','ZNF17','ZNF174','ZNF175','ZNF177','ZNF18','ZNF180','ZNF181','ZNF182','ZNF184','ZNF189','ZNF19','ZNF195','ZNF197','ZNF2','ZNF20','ZNF200','ZNF202','ZNF205','ZNF207','ZNF208','ZNF211','ZNF212','ZNF213','ZNF214','ZNF215','ZNF217','ZNF219','ZNF22','ZNF221','ZNF222','ZNF223','ZNF224','ZNF225','ZNF226','ZNF227','ZNF229','ZNF23','ZNF230','ZNF232','ZNF233','ZNF234','ZNF235','ZNF236','ZNF239','ZNF24','ZNF248','ZNF25','ZNF250','ZNF251','ZNF253','ZNF254','ZNF256','ZNF257','ZNF26','ZNF260','ZNF263','ZNF264','ZNF266','ZNF267','ZNF268','ZNF273','ZNF274','ZNF275','ZNF276','ZNF277','ZNF28','ZNF280A','ZNF280B','ZNF280C','ZNF280D','ZNF281','ZNF282','ZNF283','ZNF284','ZNF285','ZNF286A','ZNF286B','ZNF287','ZNF292','ZNF296','ZNF3','ZNF30','ZNF300','ZNF302','ZNF304','ZNF311','ZNF316','ZNF317','ZNF318','ZNF319','ZNF32','ZNF320','ZNF322','ZNF324','ZNF324B','ZNF326','ZNF329','ZNF331','ZNF333','ZNF334','ZNF335','ZNF337','ZNF33A','ZNF33B','ZNF34','ZNF341','ZNF343','ZNF345','ZNF346','ZNF347','ZNF35','ZNF350','ZNF354A','ZNF354B','ZNF354C','ZNF358','ZNF362','ZNF365','ZNF366','ZNF367','ZNF37A','ZNF382','ZNF383','ZNF384','ZNF385A','ZNF385B','ZNF385C','ZNF385D','ZNF391','ZNF394','ZNF395','ZNF396','ZNF397','ZNF398','ZNF404','ZNF407','ZNF408','ZNF41','ZNF410','ZNF414','ZNF415','ZNF416','ZNF417','ZNF418','ZNF419','ZNF420','ZNF423','ZNF425','ZNF426','ZNF428','ZNF429','ZNF43','ZNF430','ZNF431','ZNF432','ZNF433','ZNF436','ZNF438','ZNF439','ZNF44','ZNF440','ZNF441','ZNF442','ZNF443','ZNF444','ZNF445','ZNF446','ZNF449','ZNF45','ZNF451','ZNF454','ZNF460','ZNF461','ZNF462','ZNF467','ZNF468','ZNF469','ZNF470','ZNF471','ZNF473','ZNF474','ZNF479','ZNF48','ZNF480','ZNF483','ZNF484','ZNF485','ZNF486','ZNF487','ZNF488','ZNF490','ZNF491','ZNF492','ZNF493','ZNF496','ZNF497','ZNF500','ZNF501','ZNF502','ZNF503','ZNF506','ZNF507','ZNF510','ZNF511','ZNF512','ZNF512B','ZNF513','ZNF514','ZNF516','ZNF517','ZNF518A','ZNF518B','ZNF519','ZNF521','ZNF524','ZNF525','ZNF526','ZNF527','ZNF528','ZNF529','ZNF530','ZNF532','ZNF534','ZNF536','ZNF540','ZNF541','ZNF543','ZNF544','ZNF546','ZNF547','ZNF548','ZNF549','ZNF550','ZNF551','ZNF552','ZNF554','ZNF555','ZNF556','ZNF557','ZNF558','ZNF559','ZNF560','ZNF561','ZNF562','ZNF563','ZNF564','ZNF565','ZNF566','ZNF567','ZNF568','ZNF569','ZNF57','ZNF570','ZNF571','ZNF572','ZNF573','ZNF574','ZNF575','ZNF576','ZNF577','ZNF578','ZNF579','ZNF580','ZNF581','ZNF582','ZNF583','ZNF584','ZNF585A','ZNF585B','ZNF586','ZNF587','ZNF587B','ZNF589','ZNF592','ZNF594','ZNF595','ZNF596','ZNF597','ZNF598','ZNF599','ZNF600','ZNF605','ZNF606','ZNF607','ZNF608','ZNF609','ZNF610','ZNF611','ZNF613','ZNF614','ZNF615','ZNF616','ZNF618','ZNF619','ZNF620','ZNF621','ZNF623','ZNF624','ZNF625','ZNF626','ZNF627','ZNF628','ZNF629','ZNF630','ZNF639','ZNF641','ZNF644','ZNF645','ZNF646','ZNF648','ZNF649','ZNF652','ZNF653','ZNF654','ZNF655','ZNF658','ZNF66','ZNF660','ZNF662','ZNF664','ZNF665','ZNF667','ZNF668','ZNF669','ZNF670','ZNF671','ZNF672','ZNF674','ZNF675','ZNF676','ZNF677','ZNF678','ZNF679','ZNF680','ZNF681','ZNF682','ZNF683','ZNF684','ZNF687','ZNF688','ZNF689','ZNF69','ZNF691','ZNF692','ZNF695','ZNF696','ZNF697','ZNF699','ZNF7','ZNF70','ZNF700','ZNF701','ZNF703','ZNF704','ZNF705A','ZNF705B','ZNF705D','ZNF705E','ZNF705G','ZNF706','ZNF707','ZNF708','ZNF709','ZNF71','ZNF710','ZNF711','ZNF713','ZNF714','ZNF716','ZNF717','ZNF718','ZNF721','ZNF724','ZNF726','ZNF727','ZNF728','ZNF729','ZNF730','ZNF732','ZNF735','ZNF736','ZNF737','ZNF74','ZNF740','ZNF746','ZNF747','ZNF749','ZNF750','ZNF75A','ZNF75D','ZNF76','ZNF761','ZNF763','ZNF764','ZNF765','ZNF766','ZNF768','ZNF77','ZNF770','ZNF771','ZNF772','ZNF773','ZNF774','ZNF775','ZNF776','ZNF777','ZNF778','ZNF780A','ZNF780B','ZNF781','ZNF782','ZNF783','ZNF784','ZNF785','ZNF786','ZNF787','ZNF788','ZNF789','ZNF79','ZNF790','ZNF791','ZNF792','ZNF793','ZNF799','ZNF8','ZNF80','ZNF800','ZNF804A','ZNF804B','ZNF805','ZNF808','ZNF81','ZNF813','ZNF814','ZNF816','ZNF821','ZNF823','ZNF827','ZNF829','ZNF83','ZNF830','ZNF831','ZNF835','ZNF836','ZNF837','ZNF84','ZNF841','ZNF843','ZNF844','ZNF845','ZNF846','ZNF85','ZNF850','ZNF852','ZNF853','ZNF860','ZNF865','ZNF878','ZNF879','ZNF880','ZNF883','ZNF888','ZNF891','ZNF90','ZNF91','ZNF92','ZNF93','ZNF98','ZNF99','ZSCAN1','ZSCAN10','ZSCAN12','ZSCAN16','ZSCAN18','ZSCAN2','ZSCAN20','ZSCAN21','ZSCAN22','ZSCAN23','ZSCAN25','ZSCAN26','ZSCAN29','ZSCAN30','ZSCAN31','ZSCAN32','ZSCAN4','ZSCAN5A','ZSCAN5B','ZSCAN5C','ZSCAN9','ZUFSP','ZXDA','ZXDB','ZXDC','ZZZ3']
TFs=TFs1+TFs2

#TFs other, inclusive of the above list, not necessarily DNA binding proteins but affect transcription
TFs1Other=['AATF','ABL1','ABRA','AC008770.3','AC020915.1','AC023509.3','AC092835.1','AC105001.2','AC138696.1','ACOD1','ACTR3B','ACTR3C','ADAM8','ADAMTS17','ADAMTS19','ADAR','ADGRA1','ADGRG3','ADNP','ADNP2','AEBP1','AEBP2','AES','AFAP1 ','AFF1','AFF3','AFF4','AGER','AGT','AHCTF1','AHDC1','AHR','AHRR','AIM2','AIRE','AKAP8','AKAP8L','AKNA','AKT1','ALK','ALKBH2','ALX1','ALX3','ALX4','AMH','ANAPC2','ANHX','ANKRD30A','ANKZF1','ANXA3','ANXA4','AP5Z1','APBB3','APC2','APEX1','APOA4','APOBEC3A','APOBEC3B','APOBEC3G','APTX','AR','ARGFX','ARHGAP35','ARHGEF2','ARHGEF5','ARID1A','ARID1B','ARID2','ARID3A','ARID3B','ARID3C','ARID4A','ARID4B','ARID5A','ARID5B','ARNT','ARNT2','ARNTL','ARNTL2','ARRB1','ARRB2','ARX','ASCC1','ASCL1','ASCL2','ASCL3','ASCL4','ASCL5','ASH1L','ATF1','ATF2','ATF3','ATF4','ATF5','ATF6','ATF6B','ATF7','ATF7IP','ATM','ATMIN','ATOH1','ATOH7','ATOH8','ATR','ATRX','ATXN7','BACH1','BACH2','BAG1','BANF1','BANP','BARHL1','BARHL2','BARX1','BARX2','BATF','BATF2','BATF3','BAZ2A','BAZ2B','BBX','BCL10','BCL11A','BCL11B','BCL3','BCL6','BCL6B','BCLAF1','BCOR','BDP1','BEND4','BEX1','BHLHA15','BHLHA9','BHLHE22','BHLHE23','BHLHE40','BHLHE41','BHMG1','BIN1','BLM','BLZF1','BMI1','BMP2','BMP4','BMP7','BMPR1A','BNC1','BNC2','BOD1L1','BOLA1','BOLA2','BOLA2B','BOLA3','BORCS8-MEF2B','BPNT1','BPTF','BRCA1','BRD4','BRD7','BRD8','BRD9','BRF1','BRF2','BRMS1','BRPF1','BSX','BTAF1','BTBD8','BTG2','BTK','BTRC','BUD31','C11orf95','C14orf39','C17orf49','C1D','C20orf194','C3orf33','CACTIN','CAMK1D','CAMK2A','CAMTA1','CAMTA2','CAPN15','CAPN3','CARD11','CARD14','CARF','CARHSP1','CASP8AP2','CASZ1','CAT','CBFA2T2','CBFA2T3','CBFB','CBL','CBLL1','CBX2','CC2D1A','CC2D1B','CCDC124','CCDC169-SOHLH2','CCDC17','CCDC3','CCDC71','CCDC83','CCNT1','CCNT2','CD36','CD40','CD40LG','CDC5L','CDC6','CDK5RAP3','CDK7','CDK8','CDK9','CDKN1A','CDKN2A','CDX1','CDX2','CDX4','CEBPA','CEBPB','CEBPD','CEBPE','CEBPG','CEBPZ','CELF3','CENPA','CENPB','CENPBD1','CENPS','CENPT','CENPX','CERS2','CERS3','CERS4','CERS5','CERS6','CFLAR','CGGBP1','CHAMP1','CHCHD3','CHD1','CHD2','CHD3','CHD4','CHD6','CHD7','CHD9','CHP1','CHRAC1','CHUK','CHURC1','CIB1','CIC','CIITA','CIR1','CITED1','CITED2','CITED4','CIZ1','CLOCK','CLU','CMKLR1','CNBP','CNOT3','CNOT4','CNOT7','CNOT8','COMMD1','COMMD6','COMMD7','COPS2','COPS3','COPS4','CPEB1','CPSF4','CPSF4L','CPXCR1','CRAMP1','CREB1','CREB3','CREB3L1','CREB3L2','CREB3L3','CREB3L4','CREB5','CREBBP','CREBL2','CREBRF','CREBZF','CREG1','CREM','CRTC1','CRTC2','CRTC3','CRX','CSDC2','CSDE1','CSRNP1','CSRNP2','CSRNP3','CTBP1','CTBP2','CTCF','CTCFL','CTH','CTNNB1','CTNNBIP1','CUL1','CUL2','CUL3','CUL4A','CUL4B','CUL5','CUX1','CUX2','CXXC1','CXXC4','CXXC5','CYLD','CYP1B1','CYTL1','DAB2IP','DACH1','DACH2','DAP','DAXX','DBP','DBX1','DBX2','DCP1A','DDB1','DDB2','DDIT3','DDN','DDR2','DDRGK1','DDX3X','DDX5','DDX58','DEAF1','DEK','DEPDC1','DEPDC1B','DEPDC4','DEPDC5','DEPDC7','DEPTOR','DHX34','DHX36','DHX57','DISP3','DLX1','DLX2','DLX3','DLX4','DLX5','DLX6','DMAP1','DMBX1','DMC1','DMRT1','DMRT2','DMRT3','DMRTA1','DMRTA2','DMRTB1','DMRTC1','DMRTC2','DMTF1','DNA2','DNAJA3','DNAJC1','DNAJC2','DNAJC21','DNMT1','DNMT3A','DNMT3B','DNMT3L','DNTT','DNTTIP1','DOT1L','DPF1','DPF2','DPF3','DPRX','DR1','DRAP1','DRGX','DSP','DTX1','DUS3L','DUSP12','DUX1','DUX3','DUX4','DUXA','DVL1','DVL2','DVL3','DZIP1','DZIP1L','E2F1','E2F2','E2F3','E2F4','E2F5','E2F6','E2F7','E2F8','E4F1','EAF2','EBF1','EBF2','EBF3','EBF4','EDA','EDA2R','EDF1','EDN1','EEA1','EED','EGLN1','EGR1','EGR2','EGR3','EGR4','EHF','EID1','EID2','EIF2AK2','EIF2AK4','EIF3C','EIF3CL','EIF3K','ELF1','ELF2','ELF3','ELF4','ELF5','ELK1','ELK3','ELK4','ELMSAN1','EME1','EMX1','EMX2','EN1','EN2','ENO1','EOMES','EP300','EP400','EPAS1','EPHA5','EPM2AIP1','ERBIN','ERC1','ERCC3','ERCC4','ERF','ERG','ESR1','ESR2','ESRRA','ESRRB','ESRRG','ESX1','ETS1','ETS2','ETV1','ETV2','ETV3','ETV3L','ETV4','ETV5','ETV6','ETV7','EVX1','EVX2','EWSR1','EXO1','EXOC2','EXOG','EZH1','EZH2','FAAP24','FAM153C','FAM170A','FAM171B','FAM200A','FAM200B','FAN1','FANCA','FANCD2','FANK1','FARSA','FARSB','FBN1','FBXL19','FBXO41','FEN1','FER','FERD3L','FEV','FEZF1','FEZF2','FGD1','FHL2','FIGLA','FIZ1','FLI1','FLNA','FLYWCH1','FMNL2','FOS','FOSB','FOSL1','FOSL2','FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L2','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXN4','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1','FUBP1','FUBP3','FUS','FZD1','FZD2','FZD4','FZD6','GABPA','GABPB1','GABPB2','GADD45A','GAS6','GAS7','GATA1','GATA2','GATA3','GATA4','GATA5','GATA6','GATAD1','GATAD2A','GATAD2B','GBX1','GBX2','GCFC2','GCM1','GCM2','GEN1','GFI1','GFI1B','GLA','GLI1','GLI2','GLI3','GLI4','GLIS1','GLIS2','GLIS3','GLMP','GLYR1','GMEB1','GMEB2','GOLGB1','GON4L','GPATCH8','GPBP1','GPBP1L1','GPER1','GPR155','GREM1','GRHL1','GRHL2','GRHL3','GRM6','GSC','GSC2','GSX1','GSX2','GTF2A1','GTF2A1L','GTF2A2','GTF2B','GTF2E2','GTF2F1','GTF2F2','GTF2H1','GTF2H2','GTF2H2C','GTF2H3','GTF2H4','GTF2I','GTF2IRD1','GTF2IRD2','GTF2IRD2B','GTF3A','GTF3C1','GTF3C2','GULP1','GZF1','H1F0','H1FOO','H1FX','HAND1','HAND2','HAVCR2','HBP1','HCFC1','HCK','HDAC1','HDAC10','HDAC2','HDAC3','HDAC4','HDAC5','HDAC6','HDAC7','HDAC8','HDAC9','HDX','HELT','HELZ','HELZ2','HES1','HES2','HES3','HES4','HES5','HES6','HES7','HESX1','HEY1','HEY2','HEYL','HHEX','HIC1','HIC2','HIF1A','HIF3A','HINFP','HIPK2','HIRA','HIST1H1A','HIST1H1B','HIST1H1C','HIST1H1D','HIST1H1E','HIST1H1T','HIVEP1','HIVEP2','HIVEP3','HKR1','HLA-DQB1','HLA-DQB2','HLA-DRB1','HLA-DRB3','HLA-DRB5','HLF','HLTF','HLX','HMBOX1','HMG20A','HMG20B','HMGA1','HMGA2','HMGB1','HMGB2','HMGB3','HMGB4','HMGN1','HMGN3','HMGXB3','HMGXB4','HMOX1','HMX1','HMX2','HMX3','HNF1A','HNF1B','HNF4A','HNF4G','HNRNPA1','HNRNPA3','HNRNPAB','HNRNPCL2','HNRNPD','HNRNPDL','HNRNPK','HNRNPL','HNRNPU','HOMEZ','HOOK2','HOPX','HOXA1','HOXA10','HOXA11','HOXA13','HOXA2','HOXA3','HOXA4','HOXA5','HOXA6','HOXA7','HOXA9','HOXB1','HOXB13','HOXB2','HOXB3','HOXB4','HOXB5','HOXB6','HOXB7','HOXB8','HOXB9','HOXC10','HOXC11','HOXC12','HOXC13','HOXC4','HOXC5','HOXC6','HOXC8','HOXC9','HOXD1','HOXD10','HOXD11','HOXD12','HOXD13','HOXD3','HOXD4','HOXD8','HOXD9','HP1BP3','HR','HSBP1','HSF1','HSF2','HSF4','HSF5','HSFX1','HSFX2','HSFY1','HSFY2','HSPA1A','HSPA1B','ICAM1','ID1','ID2','ID3','ID4','IFI16','IFNB1','IGHM','IKBKB','IKBKG','IKZF1','IKZF2','IKZF3','IKZF4','IKZF5','IL10','IL1B','IL1RAP','IL4','IL5','IL6','INF2','ING4','INS','INSM1','INSM2','IQGAP1','IRAK1','IRAK2','IRAK3','IRF1','IRF2','IRF2BP1','IRF3','IRF4','IRF5','IRF6','IRF7','IRF8','IRF9','IRX1','IRX2','IRX3','IRX4','IRX5','IRX6','ISL1','ISL2','ISX','ITCH','ITGB2','ITGB3BP','IZUMO2','JADE1','JAK2','JARID2','JAZF1','JDP2','JMY','JRK','JRKL','JUN','JUNB','JUND','JUP','KANSL2','KAT2A','KAT2B','KAT5','KAT6A','KAT6B','KAT7','KAT8','KCMF1','KCNIP3','KDM1A','KDM2A','KDM2B','KDM3A','KDM5A','KDM5B','KDM5C','KDM5D','KEAP1','KIAA1549','KIAA1683','KIAA1841','KIAA1958','KIF15','KIN','KIT','KLF1','KLF10','KLF11','KLF12','KLF13','KLF14','KLF15','KLF16','KLF17','KLF2','KLF3','KLF4','KLF5','KLF6','KLF7','KLF8','KLF9','KMT2A','KMT2B','KMT2C','KMT2D','KNTC1','KRAS','KRBOX1','KRT13','KRTAP5-1','KRTAP5-9','KRTAP9-9','L3MBTL1','L3MBTL3','L3MBTL4','LARP1','LARP1B','LARP4','LARP4B','LARP6','LARP7','LBX1','LBX2','LCOR','LCORL','LEF1','LEUTX','LGALS9','LGR4','LHX1','LHX2','LHX3','LHX4','LHX5','LHX6','LHX8','LHX9','LIG1','LIG3','LIN28A','LIN28B','LIN54','LMO2','LMO4','LMX1A','LMX1B','LPP','LRP5','LRP6','LRP8','LRRFIP1','LRRFIP2','LSR','LTF','LTN1','LYAR','LYL1','LZTFL1','LZTR1','LZTS1','MACC1','MACF1','MAD2L2','MAEL','MAF','MAFA','MAFB','MAFF','MAFG','MAFK','MALT1','MAML1','MAML3','MAP2K5','MAP3K10','MAP3K13','MAP3K7','MAPK1','MAPK10','MAPK11','MAPK14','MAPK3','MAPK8','MAPK9','MATR3','MAVS','MAX','MAZ','MB21D1','MBD1','MBD2','MBD3','MBD4','MBD6','MBNL1','MBNL2','MBNL3','MBTPS2','MDFI','MDM2','MDM4','MECOM','MECP2','MED1','MED12','MED13','MED13L','MED14','MED15','MED16','MED17','MED21','MED23','MED24','MED6','MEF2A','MEF2B','MEF2C','MEF2D','MEIS1','MEIS2','MEIS3','MEN1','MEOX1','MEOX2','MESP1','MESP2','MET','MGA','MGMT','MID2','MIER1','MIER2','MIER3','MINK1','MIS18BP1','MITF','MIXL1','MKL1','MKL2','MKRN1','MKRN2','MKRN3','MKX','MLLT1','MLLT10','MLLT11','MLLT3','MLLT6','MLX','MLXIP','MLXIPL','MN1','MNAT1','MNT','MNX1','MORC3','MPG','MRPL28','MRRF','MSANTD1','MSANTD3','MSANTD4','MSC','MSGN1','MSH2','MSH3','MSH6','MSL3','MSRB2','MST1R','MSX1','MSX2','MTA1','MTA2','MTA3','MTDH','MTERF1','MTERF2','MTERF3','MTERF4','MTF1','MTF2','MTPN','MUS81','MXD1','MXD3','MXD4','MXI1','MYB','MYBBP1A','MYBL1','MYBL2','MYC','MYCBP','MYCL','MYCN','MYD88','MYEF2','MYF5','MYF6','MYNN','MYOCD','MYOD1','MYOG','MYPOP','MYRF','MYRFL','MYSM1','MYT1','MYT1L','MZF1','NAB2','NABP2','NACC2','NAIF1','NANOG','NANOGNB','NANOGP8','NCOA1','NCOA2','NCOA3','NCOA4','NCOA5','NCOA6','NCOA7','NCOR1','NCOR2','NDC80','NDN','NDP','NEB','NEIL1','NEUROD1','NEUROD2','NEUROD4','NEUROD6','NEUROG1','NEUROG2','NEUROG3','NFAM1','NFAT5','NFATC1','NFATC2','NFATC3','NFATC4','NFE2','NFE2L1','NFE2L2','NFE2L3','NFE4','NFIA','NFIB','NFIC','NFIL3','NFIX','NFKB1','NFKB2','NFKBIA','NFKBIB','NFKBID','NFKBIL1','NFKBIZ','NFRKB','NFX1','NFXL1','NFYA','NFYB','NFYC','NHLH1','NHLH2','NKRF','NKX1-1','NKX1-2','NKX2-1','NKX2-2','NKX2-3','NKX2-4','NKX2-5','NKX2-6','NKX2-8','NKX3-1','NKX3-2','NKX6-1','NKX6-2','NKX6-3','NLRC3','NLRC4','NLRC5','NLRP12','NLRP2B','NLRP3','NME2','NOBOX','NOC3L','NOC4L','NOCT','NOD1','NOD2','NODAL','NONO','NOP2','NOTCH1','NOTCH2','NOTCH3','NOTO','NPAS1','NPAS2','NPAS3','NPAS4','NPM1','NR0B1','NR0B2','NR1D1','NR1D2','NR1H2','NR1H3','NR1H4','NR1I2','NR1I3','NR2C1','NR2C2','NR2E1','NR2E3','NR2F1','NR2F2','NR2F6','NR3C1','NR3C2','NR4A1','NR4A2','NR4A3','NR5A1','NR5A2','NR6A1','NRBF2','NRF1','NRG1','NRIP1','NRIP3','NRK','NRL','NSD1','NSD2','NTRK1','NUCKS1','NUFIP1','NUP98','NUPL2']
TFs2Other=['OGG1','OLIG1','OLIG2','OLIG3','ONECUT1','ONECUT2','ONECUT3','OPRD1','OSR1','OSR2','OTOP3','OTP','OTULIN','OTX1','OTX2','OVOL1','OVOL2','OVOL3','PA2G4','PABPN1','PARK7','PARP1','PARP10','PARP12','PATZ1','PAWR','PAX1','PAX2','PAX3','PAX4','PAX5','PAX6','PAX7','PAX8','PAX9','PAXBP1','PAXIP1','PBRM1','PBX1','PBX2','PBX3','PBX4','PCBD1','PCBP1','PCBP2','PCBP3','PCGF2','PCGF6','PCNA','PCSK6','PDCD7','PDS5B','PDX1','PEG3','PELI1','PER1','PER2','PER3','PEX14','PFDN1','PGBD1','PGR','PGS1','PHB','PHB2','PHF1','PHF19 ','PHF20','PHF21A','PHF5A','PHOX2A','PHOX2B','PHTF1','PHTF2','PIAS1','PIAS2','PIAS3','PIAS4','PIDD1','PIKFYVE','PIM1','PIN1','PINK1','PITX1','PITX2','PITX3','PKHD1','PKHD1L1','PKNOX1','PKNOX2','PLA2G10','PLA2G1B','PLAG1','PLAGL1','PLAGL2','PLEK','PLEK2','PLEKHA4','PLPP3','PLSCR1','PLXNA1','PLXNA2','PLXNA3','PLXNA4','PLXNB1','PLXNB2','PLXNB3','PLXNC1','PLXND1','PML','PMS1','PNKP','POGK','POGZ','POLA1','POLB','POLE3','POLE4','POLG','POLH','POLI','POLK','POLL','POLM','POLN','POLQ','POLR1B','POLR2A','POLR2B','POLR2E','POLR2J','POLR2K','POLR2L','POLR3A','POLRMT','POT1','POU1F1','POU2AF1','POU2F1','POU2F2','POU2F3','POU3F1','POU3F2','POU3F3','POU3F4','POU4F1','POU4F2','POU4F3','POU5F1','POU5F1B','POU5F2','POU6F1','POU6F2','PPARA','PPARD','PPARG','PPARGC1A','PPARGC1B','PPP1R10','PPP1R13L','PPP1R21','PPP2R3B','PPP2R5B','PPP3CA','PPRC1','PQBP1','PRAM1','PRAP1','PRB3','PRDM1','PRDM10','PRDM11','PRDM12','PRDM13','PRDM14','PRDM15','PRDM16','PRDM2','PRDM4','PRDM5','PRDM6','PRDM7','PRDM8','PRDM9','PRDX3','PREB','PREX1','PREX2','PRH2','PRIM2','PRIMPOL','PRKCB','PRKCH','PRKCI','PRKCQ','PRKCZ','PRKD1','PRKD2','PRKN','PRMT2','PRMT3','PRNP','PROP1','PROX1','PROX2','PRR12','PRR3','PRRX1','PRRX2','PSMA6','PSMC5','PSMD10','PSMD11','PSMD12','PSPC1','PTCH1','PTEN','PTF1A','PTGIS','PTH','PTTG1','PUM3','PURA','PURB','PURG','PYCARD','PYDC1','PYDC2','QRICH1','RAB11FIP4','RAB7B','RAD21','RAD51','RAG1','RAI1','RAPGEF3','RAPGEF4','RAPGEF5','RARA','RARB','RARG','RAX','RAX2','RB1','RBAK','RBBP7','RBCK1','RBL1','RBL2','RBM10','RBM20','RBM22','RBM26','RBM27','RBM5','RBM6','RBPJ','RBPJL','RBSN','RC3H1','RC3H2','RCAN1','RCOR1','RCOR2','RCOR3','RECQL','REL','RELA','RELB','RELL2','RELN','REPIN1','RERE','REST','REV1','REXO4','RFC1','RFX1','RFX2','RFX3','RFX4','RFX5','RFX6','RFX7','RFX8','RFXANK','RFXAP','RGCC','RGS11','RGS6','RGS7','RGS9','RHEBL1','RHOXF1','RHOXF2','RHOXF2B','RING1','RIOK2','RIPK1','RIPK2','RIPK3','RIPK4','RLF','RLIM','RNASE2','RNASEH1','RNF113A','RNF113B','RNF114','RNF125','RNF138','RNF14','RNF166','RNF2','RNF219','RNF25','RNF31','RNF4','RNF41','RORA','RORB','RORC','RPA2','RPA4','RPL36A','RPL7','RPL7A','RPS27A','RPS6KA4','RPS6KA5','RREB1','RRN3','RUNX1','RUNX1T1','RUNX2','RUNX3','RWDD3','RXRA','RXRB','RXRG','RYBP','S100A12','S100A8','S100A9','SAFB','SAFB2','SALL1','SALL2','SALL3','SALL4','SART1','SATB1','SATB2','SAV1','SCAND1','SCAND2P','SCAPER','SCMH1','SCML1','SCML2','SCML4','SCOC','SCRT1','SCRT2','SCX','SEBOX','SEMA4A','SERTAD1','SERTAD2','SETBP1','SETD6','SETDB1','SETDB2','SF3A2','SF3A3','SFPQ','SFRP4','SFRP5','SFTPB','SGK1','SGK3','SGSM2','SHH','SHOX','SHOX2','SHPRH','SIGIRR','SIK1','SIM1','SIM2','SIN3A','SIN3B','SIRT1','SIRT2','SIRT3','SIRT6','SIVA1','SIX1','SIX2','SIX3','SIX4','SIX5','SIX6','SKI','SKIDA1','SKIL','SKOR1','SKOR2','SLC22A4','SLC26A10','SLC26A3','SLC2A4RG','SLC30A9','SLC39A10','SLC4A10','SMAD1','SMAD2','SMAD3','SMAD4','SMAD5','SMAD6','SMAD7','SMAD9','SMARCA1','SMARCA2','SMARCA4','SMARCA5','SMARCAL1','SMARCB1','SMARCC1','SMARCC2','SMARCD1','SMARCD2','SMARCD3','SMARCE1','SMC3','SMO','SMUG1','SMYD1','SMYD3','SNAI1','SNAI2','SNAI3','SNAPC2','SNAPC4','SNAPC5','SNRPC','SNURF','SNW1','SOGA3','SOHLH1','SOHLH2','SON','SORBS2','SOX1','SOX10','SOX11','SOX12','SOX13','SOX14','SOX15','SOX17','SOX18','SOX2','SOX21','SOX3','SOX30','SOX4','SOX5','SOX6','SOX7','SOX8','SOX9','SP1','SP100','SP110','SP140','SP140L','SP2','SP3','SP4','SP5','SP6','SP7','SP8','SP9','SPDEF','SPEN','SPHK1','SPI1','SPIB','SPIC','SPZ1','SRCAP','SREBF1','SREBF2','SRF','SRY','SSB','SSBP2','SSBP3','SSBP4','SSH1','SSH2','SSH3','SSRP1','SSX1','ST18','STAG1','STAT1','STAT2','STAT3','STAT4','STAT5A','STAT5B','STAT6','STK16','STK3','STK36','STRN','STRN3','SUB1','SUFU','SUMO1','SUPT16H','SUPT4H1','SUPT5H','SUPT6H','SUV39H1','SUV39H2','SUZ12','SYK','T','TAB1','TAB2','TAB3','TADA2A','TADA2B','TADA3','TAF1','TAF10','TAF11','TAF12','TAF13','TAF1A','TAF1B','TAF1C','TAF1L','TAF2','TAF3','TAF4','TAF4B','TAF5','TAF5L','TAF6','TAF6L','TAF7','TAF7L','TAF8','TAF9','TAF9B','TAL1','TAL2','TARDBP','TAX1BP1','TBL1Y','TBP','TBPL1','TBPL2','TBR1','TBX1','TBX10','TBX15','TBX18','TBX19','TBX2','TBX20','TBX21','TBX22','TBX3','TBX4','TBX5','TBX6','TCEA2','TCEAL1','TCEAL7','TCEAL8','TCF12','TCF15','TCF19','TCF20','TCF21','TCF23','TCF24','TCF25','TCF3','TCF4','TCF7','TCF7L1','TCF7L2','TCFL5','TDG','TDP1','TDP2','TEAD1','TEAD2','TEAD3','TEAD4','TEF','TERB1','TERF1','TERF2','TERF2IP','TESMIN','TET1','TET2','TET3','TFAM','TFAP2A','TFAP2B','TFAP2C','TFAP2D','TFAP2E','TFAP4','TFCP2','TFCP2L1','TFDP1','TFDP2','TFDP3','TFE3','TFEB','TFEC','TGFB1','TGFB1I1','TGIF1','TGIF2','TGIF2LX','TGIF2LY','TGS1','THAP1','THAP10','THAP11','THAP12','THAP2','THAP3','THAP4','THAP5','THAP6','THAP7','THAP8','THAP9','THRA','THRAP3','THRB','THYN1','TIA1','TICAM1','TIGD1','TIGD2','TIGD3','TIGD4','TIGD5','TIGD6','TIGD7','TIPARP','TIRAP','TLE1','TLE2','TLE3','TLE4','TLE6','TLR2','TLR3','TLR4','TLR9','TLX1','TLX2','TLX3','TMEM235','TMF1','TNF','TNFAIP3','TNFRSF11A','TNFRSF4','TNFSF11','TNFSF18','TNFSF4','TOE1','TONSL','TOP1','TOP2A','TOP2B','TOPORS','TOX','TOX2','TOX3','TOX4','TP53','TP53BP1','TP63','TP73','TPRX1','TRADD','TRAF1','TRAF2','TRAF3','TRAF4','TRAF5','TRAF6','TRAFD1','TRAK2','TRAPPC9','TRERF1','TREX1','TRIB1','TRIB3','TRIM13','TRIM14','TRIM15','TRIM21','TRIM22','TRIM23','TRIM24','TRIM25','TRIM26','TRIM27','TRIM28','TRIM29','TRIM3','TRIM31','TRIM32','TRIM33','TRIM34','TRIM37','TRIM38','TRIM40','TRIM5','TRIM52','TRIM62','TRIM8','TRIP11','TRIT1','TRMT1','TRMT1L','TRPS1','TRRAP','TSC2','TSC22D1','TSC22D2','TSC22D3','TSC22D4','TSHZ1','TSHZ2','TSHZ3','TSN','TSSK4','TTF1','TUB','TULP4','TUT1','TWIST1','TWIST2','TXK','U2AF1','U2AF1L4','UBA52','UBB','UBC','UBE2K','UBE2N','UBE2V1','UBN1','UBP1','UBR4','UBTF','UBTFL1','UBTFL6','UFL1','UHRF1','UHRF2','UNCX','UNG','UNK','UNKL','USF1','USF2','USF3','USP39','USP7','UTF1','VAV1','VAX1','VAX2','VDR','VEGFA','VENTX','VEZF1','VGLL1','VGLL2','VGLL3','VHLL','VPS72','VSX1','VSX2','WDHD1','WFS1','WIZ','WNT1','WNT10B','WNT2','WNT3A','WNT5A','WNT8B','WRN','WT1','WWOX','WWP2','WWTR1','XBP1','XCL1','XPA','YBX1','YBX2','YBX3','YEATS2','YEATS4','YOD1','YY1','YY2','ZBED1','ZBED2','ZBED3','ZBED4','ZBED5','ZBED6','ZBED6CL','ZBED8','ZBED9','ZBP1','ZBTB1','ZBTB10','ZBTB11','ZBTB12','ZBTB14','ZBTB16','ZBTB17','ZBTB18','ZBTB2','ZBTB20','ZBTB21','ZBTB22','ZBTB24','ZBTB25','ZBTB26','ZBTB3','ZBTB32','ZBTB33','ZBTB34','ZBTB37','ZBTB38','ZBTB39','ZBTB4','ZBTB40','ZBTB41','ZBTB42','ZBTB43','ZBTB44','ZBTB45','ZBTB46','ZBTB47','ZBTB48','ZBTB49','ZBTB5','ZBTB6','ZBTB7A','ZBTB7B','ZBTB7C','ZBTB8A','ZBTB8B','ZBTB9','ZC3H10','ZC3H11A','ZC3H12A','ZC3H13','ZC3H14','ZC3H15','ZC3H18','ZC3H3','ZC3H4','ZC3H6','ZC3H7A','ZC3H7B','ZC3H8','ZC3HAV1','ZCCHC11','ZCCHC6','ZDHHC11','ZDHHC19','ZEB1','ZEB2','ZFAND5','ZFAT','ZFHX2','ZFHX3','ZFHX4','ZFP1','ZFP14','ZFP2','ZFP28','ZFP3','ZFP30','ZFP36','ZFP36L1','ZFP36L2','ZFP37','ZFP41','ZFP42','ZFP57','ZFP62','ZFP64','ZFP69','ZFP69B','ZFP82','ZFP90','ZFP91','ZFP92','ZFPL1','ZFPM1','ZFPM2','ZFR','ZFR2','ZFX','ZFY','ZFYVE26','ZGLP1','ZGPAT','ZHX1','ZHX2','ZHX3','ZIC1','ZIC2','ZIC3','ZIC4','ZIC5','ZIK1','ZIM2','ZIM3','ZKSCAN1','ZKSCAN2','ZKSCAN3','ZKSCAN4','ZKSCAN5','ZKSCAN7','ZKSCAN8','ZMAT1','ZMAT2','ZMAT3','ZMAT4','ZMAT5','ZMYM1','ZMYM2','ZMYM3','ZMYM4','ZMYM5','ZMYM6','ZMYND10','ZNF10','ZNF100','ZNF101','ZNF106','ZNF107','ZNF112','ZNF114','ZNF117','ZNF12','ZNF121','ZNF124','ZNF131','ZNF132','ZNF133','ZNF134','ZNF135','ZNF136','ZNF137P','ZNF138','ZNF14','ZNF140','ZNF141','ZNF142','ZNF143','ZNF146','ZNF148','ZNF154','ZNF155','ZNF157','ZNF16','ZNF160','ZNF165','ZNF169','ZNF17','ZNF174','ZNF175','ZNF177','ZNF18','ZNF180','ZNF181','ZNF182','ZNF184','ZNF185','ZNF189','ZNF19','ZNF195','ZNF197','ZNF2','ZNF20','ZNF200','ZNF202','ZNF205','ZNF207','ZNF208','ZNF211','ZNF212','ZNF213','ZNF214','ZNF215','ZNF217','ZNF219','ZNF22','ZNF221','ZNF222','ZNF223','ZNF224','ZNF225','ZNF226','ZNF227','ZNF229','ZNF23','ZNF230','ZNF232','ZNF233','ZNF234','ZNF235','ZNF236','ZNF239','ZNF24','ZNF248','ZNF25','ZNF250','ZNF251','ZNF253','ZNF254','ZNF256','ZNF257','ZNF26','ZNF260','ZNF263','ZNF264','ZNF266','ZNF267','ZNF268','ZNF273','ZNF274','ZNF275','ZNF276','ZNF277','ZNF28','ZNF280A','ZNF280B','ZNF280C','ZNF280D','ZNF281','ZNF282','ZNF283','ZNF284','ZNF285','ZNF286A','ZNF286B','ZNF287','ZNF292','ZNF296','ZNF3','ZNF30','ZNF300','ZNF302','ZNF304','ZNF311','ZNF316','ZNF317','ZNF318','ZNF319','ZNF32','ZNF320','ZNF322','ZNF324','ZNF324B','ZNF326','ZNF329','ZNF330','ZNF331','ZNF333','ZNF334','ZNF335','ZNF337','ZNF33A','ZNF33B','ZNF34','ZNF341','ZNF343','ZNF345','ZNF346','ZNF347','ZNF35','ZNF350','ZNF354A','ZNF354B','ZNF354C','ZNF355P','ZNF358','ZNF362','ZNF365','ZNF366','ZNF367','ZNF37A','ZNF382','ZNF383','ZNF384','ZNF385A','ZNF385B','ZNF385C','ZNF385D','ZNF391','ZNF394','ZNF395','ZNF396','ZNF397','ZNF398','ZNF404','ZNF407','ZNF408','ZNF41','ZNF410','ZNF414','ZNF415','ZNF416','ZNF417','ZNF418','ZNF419','ZNF420','ZNF423','ZNF425','ZNF426','ZNF428','ZNF429','ZNF43','ZNF430','ZNF431','ZNF432','ZNF433','ZNF436','ZNF438','ZNF439','ZNF44','ZNF440','ZNF441','ZNF442','ZNF443','ZNF444','ZNF445','ZNF446','ZNF449','ZNF45','ZNF451','ZNF454','ZNF460','ZNF461','ZNF462','ZNF467','ZNF468','ZNF469','ZNF470','ZNF471','ZNF473','ZNF474','ZNF479','ZNF48','ZNF480','ZNF483','ZNF484','ZNF485','ZNF486','ZNF487','ZNF488','ZNF490','ZNF491','ZNF492','ZNF493','ZNF496','ZNF497','ZNF500','ZNF501','ZNF502','ZNF503','ZNF506','ZNF507','ZNF510','ZNF511','ZNF512','ZNF512B','ZNF513','ZNF514','ZNF516','ZNF517','ZNF518A','ZNF518B','ZNF519']
TFs3Other=['ZNF521','ZNF524','ZNF525','ZNF526','ZNF527','ZNF528','ZNF529','ZNF530','ZNF532','ZNF534','ZNF536','ZNF540','ZNF541','ZNF542P','ZNF543','ZNF544','ZNF546','ZNF547','ZNF548','ZNF549','ZNF550','ZNF551','ZNF552','ZNF554','ZNF555','ZNF556','ZNF557','ZNF558','ZNF559','ZNF560','ZNF561','ZNF562','ZNF563','ZNF564','ZNF565','ZNF566','ZNF567','ZNF568','ZNF569','ZNF57','ZNF570','ZNF571','ZNF572','ZNF573','ZNF574','ZNF575','ZNF576','ZNF577','ZNF578','ZNF579','ZNF580','ZNF581','ZNF582','ZNF583','ZNF584','ZNF585A','ZNF585B','ZNF586','ZNF587','ZNF587B','ZNF589','ZNF592','ZNF593','ZNF594','ZNF595','ZNF596','ZNF597','ZNF598','ZNF599','ZNF600','ZNF605','ZNF606','ZNF607','ZNF608','ZNF609','ZNF610','ZNF611','ZNF613','ZNF614','ZNF615','ZNF616','ZNF618','ZNF619','ZNF620','ZNF621','ZNF622','ZNF623','ZNF624','ZNF625','ZNF626','ZNF627','ZNF628','ZNF629','ZNF630','ZNF638','ZNF639','ZNF641','ZNF644','ZNF645','ZNF646','ZNF648','ZNF649','ZNF652','ZNF653','ZNF654','ZNF655','ZNF658','ZNF66','ZNF660','ZNF662','ZNF664','ZNF665','ZNF667','ZNF668','ZNF669','ZNF670','ZNF671','ZNF672','ZNF674','ZNF675','ZNF676','ZNF677','ZNF678','ZNF679','ZNF680','ZNF681','ZNF682','ZNF683','ZNF684','ZNF687','ZNF688','ZNF689','ZNF69','ZNF691','ZNF692','ZNF695','ZNF696','ZNF697','ZNF699','ZNF7','ZNF70','ZNF700','ZNF701','ZNF703','ZNF704','ZNF705A','ZNF705B','ZNF705D','ZNF705E','ZNF705G','ZNF706','ZNF707','ZNF708','ZNF709','ZNF71','ZNF710','ZNF711','ZNF713','ZNF714','ZNF716','ZNF717','ZNF718','ZNF720','ZNF721','ZNF724','ZNF726','ZNF727','ZNF728','ZNF729','ZNF73','ZNF730','ZNF732','ZNF735','ZNF736','ZNF737','ZNF738','ZNF74','ZNF740','ZNF746','ZNF747','ZNF749','ZNF750','ZNF75A','ZNF75D','ZNF76','ZNF761','ZNF763','ZNF764','ZNF765','ZNF766','ZNF768','ZNF77','ZNF770','ZNF771','ZNF772','ZNF773','ZNF774','ZNF775','ZNF776','ZNF777','ZNF778','ZNF780A','ZNF780B','ZNF781','ZNF782','ZNF783','ZNF784','ZNF785','ZNF786','ZNF787','ZNF788','ZNF789','ZNF79','ZNF790','ZNF791','ZNF792','ZNF793','ZNF799','ZNF8','ZNF80','ZNF800','ZNF804A','ZNF804B','ZNF805','ZNF808','ZNF81','ZNF812P','ZNF813','ZNF814','ZNF816','ZNF821','ZNF823','ZNF827','ZNF829','ZNF83','ZNF830','ZNF831','ZNF835','ZNF836','ZNF837','ZNF839','ZNF84','ZNF841','ZNF843','ZNF844','ZNF845','ZNF846','ZNF85','ZNF850','ZNF852','ZNF853','ZNF860','ZNF862','ZNF865','ZNF878','ZNF879','ZNF880','ZNF883','ZNF888','ZNF891','ZNF90','ZNF91','ZNF92','ZNF93','ZNF98','ZNF99','ZNFX1','ZRANB2','ZRSR1','ZRSR2','ZSCAN1','ZSCAN10','ZSCAN12','ZSCAN16','ZSCAN18','ZSCAN2','ZSCAN20','ZSCAN21','ZSCAN22','ZSCAN23','ZSCAN25','ZSCAN26','ZSCAN29','ZSCAN30','ZSCAN31','ZSCAN32','ZSCAN4','ZSCAN5A','ZSCAN5B','ZSCAN5C','ZSCAN5DP','ZSCAN9','ZUFSP','ZXDA','ZXDB','ZXDC','ZZZ3']
TFs4Other=['ATF2','ATF6B','ATN1','BHLHA15','BHLHB2','BHLHB4 ','BHLHB5','BHLHB9','C13ORF8','C20ORF194','CDX2','CRAMP1L','ELP3','ERVK3-1','ETF1','FLJ12895','FLJ20321','FLJ25476','FLJ35784','FOXM1','FOXO2','FOXR1','FOXR2','GATA1','GRLF1','KDM4C','KDM4E','KIAA0863','KIAA0961','KLRG1','LBXCOR1','LEF1','LHX7','LOC126295','LOC51058','LOC91661','MGC4728','MYOD1','MYOG','MYRF','NEUROD4','NEUROG1','NFAT2','NFAT4','NFE2L1','NR1A2','NR1B1','NR1B2','NR1B3','NR1C1','NR1C2','NR1C3','NR1F3','NR1I1','NR2A1','NR2B3','NR3A1','NR3A2','NR3B3','OTEX','PEPP-2','PHF2','POU2F1','POU2F2','POU4F3','POU5F1','POU6F1','PRDM3','PRKRIP1','PRKRIR','SETD1A','SMAD8','SMYD2','SOX12','SOX14','SOX15','TBX14','TCF1','TCF17','TCF2','TCF9','TEAD1','TPRXL','UBTFL1','WHSC1','ZBTB14','ZBTB18','ZBTB19','ZBTB21','ZBTB27','ZBTB28','ZBTB35','ZBTB49','ZFP161','ZFP28','ZKSCAN10','ZKSCAN11','ZKSCAN12','ZKSCAN13','ZKSCAN14','ZKSCAN16','ZKSCAN19','ZKSCAN21','ZKSCAN24','ZKSCAN6','ZKSCAN8','ZNF10','ZNF112','ZNF187','ZNF193','ZNF204P','ZNF238','ZNF271','ZNF295','ZNF305','ZNF323','ZNF434','ZNF499','ZNF542','ZNF545','ZNF553','ZNF56','ZNF625','ZNF643','ZNF698','ZNF705F','ZNF751','ZNF752','ZNF753','ZNF785','ZNF806','ZNF816','ZNF833P','ZNF906','ZSCAN1','ZSCAN25','ZSCAN26','ZSCAN31','ZSCAN32','ZSCAN5D','ZSCAN9']
TFsOther=TFs1Other+TFs2Other+TFs3Other+TFs4Other
```

Example 7 (unknown):
```unknown
S genes: ['MCM5', 'PCNA', 'TYMS', 'FEN1', 'MCM2', 'MCM4', 'RRM1', 'UNG', 'GINS2', 'MCM6', 'CDCA7', 'DTL', 'PRIM1', 'UHRF1', 'HELLS', 'RFC2', 'RPA2', 'NASP', 'RAD51AP1', 'GMNN', 'WDR76', 'SLBP', 'CCNE2', 'UBR7', 'POLD3', 'MSH2', 'ATAD2', 'RAD51', 'RRM2', 'CDC45', 'CDC6', 'EXO1', 'TIPIN', 'DSCC1', 'BLM', 'CASP8AP2', 'USP1', 'CLSPN', 'POLA1', 'CHAF1B', 'BRIP1', 'E2F8'] 

G2 genes: ['HMGB2', 'CDK1', 'NUSAP1', 'UBE2C', 'BIRC5', 'TPX2', 'TOP2A', 'NDC80', 'CKS2', 'NUF2', 'CKS1B', 'MKI67', 'TMPO', 'CENPF', 'TACC3', 'SMC4', 'CCNB2', 'CKAP2L', 'CKAP2', 'AURKB', 'BUB1', 'KIF11', 'ANP32E', 'TUBB4B', 'GTSE1', 'KIF20B', 'HJURP', 'CDCA3', 'CDC20', 'TTK', 'CDC25C', 'KIF2C', 'RANGAP1', 'NCAPD2', 'DLGAP5', 'CDCA2', 'CDCA8', 'ECT2', 'KIF23', 'HMMR', 'AURKA', 'PSRC1', 'ANLN', 'LBR', 'CKAP5', 'CENPE', 'CTCF', 'NEK2', 'G2E3', 'GAS2L3', 'CBX5', 'CENPA'] 

Hemoglobin genes set 1: ['HBB', 'HBD', 'HBG1', 'HBG2', 'HBE1', 'HBZ', 'HBM', 'HBA2', 'HBA1', 'HBQ1'] 

Hemoglobin genes set 2: ['AHSP', 'GYPA', 'RHAG', 'GFI1B', 'KEL', 'EPB42', 'ALAS2', 'TSPO2', 'GYPB', 'GYPE', 'SPTA1', 'RHCE', 'KLF1', 'RHD', 'SLC4A1', 'SNX22', 'SQSTM1', 'SOD1', 'FTH1', 'MAP1LC3B', 'SLC25A37', 'TRIM58', 'HMBS'] 

Mitochondrial genes: ['MT-ND1', 'MT-ND2', 'MT-CO1', 'MT-CO2', 'MT-ATP8', 'MT-ATP6', 'MT-CO3', 'MT-ND3', 'MT-ND4L', 'MT-ND4', 'MT-ND5', 'MT-ND6', 'MT-CYB'] 

Ribo genes: ['RPLP0', 'RPLP1', 'RPLP2', 'RPL3', 'RPL3L', 'RPL4', 'RPL5', 'RPL6', 'RPL7', 'RPL7A', 'RPL7L1', 'RPL8', 'RPL9', 'RPL10', 'RPL10A', 'RPL10L', 'RPL11', 'RPL12', 'RPL13', 'RPL13A', 'RPL14', 'RPL15', 'RPL17', 'RPL18', 'RPL18A', 'RPL19', 'RPL21', 'RPL22', 'RPL22L1', 'RPL23', 'RPL23A', 'RPL24', 'RPL26', 'RPL26L1', 'RPL27', 'RPL27A', 'RPL28', 'RPL29', 'RPL30', 'RPL31', 'RPL32', 'RPL34', 'RPL35', 'RPL35A', 'RPL36', 'RPL36A', 'RPL36AL', 'RPL37', 'RPL37A', 'RPL38', 'RPL39', 'RPL39L', 'UBA52', 'RPL41', 'RPSA', 'RPS2', 'RPS3', 'RPS3A', 'RPS4X', 'RPS4Y1', 'RPS4Y2', 'RPS5', 'RPS6', 'RPS7', 'RPS8', 'RPS9', 'RPS10', 'RPS11', 'RPS12', 'RPS13', 'RPS14', 'RPS15', 'RPS15A', 'RPS16', 'RPS17', 'RPS18', 'RPS19', 'RPS20', 'RPS21', 'RPS23', 'RPS24', 'RPS25', 'RPS26', 'RPS27', 'RPS27A', 'RPS27L', 'RPS28', 'RPS29', 'FAU'] 

Tissue dissociation stress associated genes: ['FOS', 'FOSB', 'JUN', 'JUNB', 'JUND', 'ZFP36', 'SOCS3', 'CEBPD', 'CEBPB', 'ATF3', 'EGR1', 'HSPA1A', 'HSPA1B', 'HSP90AA1', 'HSP90AB1', 'HSPA8', 'HSPB1', 'HSPE1', 'IER3', 'IER2', 'BTG1', 'BTG2', 'DUSP1']
```

Example 8 (python):
```python
#Define color palette
FineGrainPalette={
    'AtrialCardiomyocytesLeft':'#eac075',
    'AtrialCardiomyocytesRight':'#4c6b8a',
    'AtrialCardiomyocytesCycling':'#e94bc7',
    'VentricularCardiomyocytesLeftCompact':'#ab1c3c',
    'VentricularCardiomyocytesRightCompact':'#fdba45',
    'VentricularCardiomyocytesLeftTrabeculated':'#2b9200',
    'VentricularCardiomyocytesRightTrabeculated':'#0273a8',
    'VentricularCardiomyocytesCycling':'#e1acff',
    'SinoatrialNodeCardiomyocytes':'#854834',
    'AtrioventricularNodeCardiomyocytes':'#688500',
    'VentricularConductionSystemProximal':'#01ccc8',
    'VentricularConductionSystemDistal':'#b58fab',
    'GreatVesselAdventitialFibroblasts':'#ff7e97',
    'CoronaryVesselAdventitialFibroblasts':'#ff9b6e',
    'MyocardialInterstitialFibroblasts1':'#8a6800',
    'MyocardialInterstitialFibroblasts2':'#206601',
    'Myofibroblasts':'#1bdebf',
    'LymphNodeFibroblasticReticularCells':'#005aa0',
    'ValveInterstitialCells':'#e478ff',
    'GreatVesselSmoothMuscleCells':'#ff665f',
    'CoronarySmoothMuscleCells':'#d2cb18',
    'DuctusArteriosusSmoothMuscleCells':'#83d7b3',
    'CoronaryPericytes':'#9088ff',
    'PericardialCells1':'#c95e00',
    'PericardialCells2':'#01e077',
    'PericardialCells3':'#ff88e4',
    'GreatVesselArterialEndothelialCells':'#d4004b',
    'GreatVesselVenousEndothelialCells':'#8f4300',
    'CoronaryArterialEndothelialCells':'#89da5a',
    'CoronaryVenousEndothelialCells':'#006f86',
    'CoronaryCapillaryEndothelialCells':'#aa7dff',
    'EndocardialCells':'#ff8241',
    'EndocardialCushionCells':'#46603a',
    'ValveEndothelialCells':'#414eb6',
    'LymphaticEndothelialCells':'#8b3f60',
    'MesothelialEpicardialCells':'#0295fe',
    'EpicardiumDerivedCells':'#ff92c4',
    'NeuronPrecursors':'#d30038',
    'ChromaffinCells':'#726800',
    'SympatheticNeurons':'#01894b',
    'ParaympatheticNeurons':'#545191',
    'SchwannCellPrecursors':'#b0d276',
    'SchwannCells':'#b30079',
    'MonocytesMPOpos':'#c50066',
    'Monocytes':'#fab4b6',
    'MonocyteDerivedCells':'#d14008',
    'MacrophagesCX3CR1pos':'#eabe97',
    'MacrophagesTIMD4pos':'#b2a200',
    'MacrophagesLYVE1pos':'#4a6005',
    'MacrophagesATF3pos':'#3db833',
    'DendriticCellsType1':'#007859',
    'DendriticCellsMature':'#01a9db',
    'PlasmacytoidDendriticCells':'#2c54ab',
    'MastCells':'#d0bef2',
    'Megakaryocytes':'#94298f',
    'TCellsCD4pos':'#ff387c',
    'TCellsCD8pos':'#9e3429',
    'TregsCD4pos':'#b37700',
    'ProBCells':'#8dab00',
    'BCells':'#a8d1a1',
    'BCellsMS4A1pos':'#00d2f0',
    'NaturalKillerCells':'#abb4ff',
    'InnateLymphoidCells':'#72497f',
}
sc.pl.umap(adata, color='fine_grain',palette=FineGrainPalette)

MidGrainPalette={
    'AtrialCardiomyocytes':'#af3a4d',
    'VentricularCardiomyocytes':'#b1756c',
    'CardiacConductionSystem':'#d45633',
    'Fibroblasts':'#9b6e4e',
    'MuralCells':'#ffb92f',
    'PericardialCells':'#b4f03b',
    'BloodVesselEndothelialCells':'#72cc57',
    'EndocardialCells':'#436e43',
    'LymphaticEndothelialCells':'#85cca0',
    'EpicardialCells':'#5eabb0',
    'Neurons':'#7bbdff',
    'Glia':'#403c90',
    'MyeloidCells':'#420066',
    'LymphoidCells':'#f553f6',
}
sc.pl.umap(adata, color='mid_grain',palette=MidGrainPalette)

CoarseGrainPalette={
    'Cardiomyocytes':'#bc5950',
    'Mesenchymal':'#af9c4c',
    'Endothelium':'#599e58',
    'Epicardium':'#5eabb0',
    'Neural':'#5e75b1',
    'Leukocytes':'#9a52a0',
}
sc.pl.umap(adata, color='coarse_grain',palette=CoarseGrainPalette)

RegionGrainPalette={
    'aorta':'#ffa2bc',
    'apex':'#fe0049',
    'atria':'#732d00',
    'atrial septum':'#ffd54c',
    'base':'#837100',
    'ductus arteriosus':'#bdff8b',
    'whole sample':'#1b6c00',
    'heart without node':'#02e6c6',
    'left ventricle':'#01c1f5',
    'node':'#00466d',
    'outflow tract':'#020011',
    'pericardium':'#7a75ff',
    'pulmonary arches and branches':'#30003e',
    'right ventricle':'#ff88e1',
    'right ventricle and papillaries':'#98004f',
}
sc.pl.umap(adata, color='region',palette=RegionGrainPalette)
```

Example 9 (python):
```python
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
/tmp/ipykernel_299748/1379983619.py in <module>
     65     'InnateLymphoidCells':'#72497f',
     66 }
---> 67 sc.pl.umap(adata, color='fine_grain',palette=FineGrainPalette)
     68 
     69 MidGrainPalette={

~/miniconda3/envs/scanpynew/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py in umap(adata, **kwargs)
    657     tl.umap
    658     """
--> 659     return embedding(adata, 'umap', **kwargs)
    660 
    661 

~/miniconda3/envs/scanpynew/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py in embedding(adata, basis, color, gene_symbols, use_raw, sort_order, edges, edges_width, edges_color, neighbors_key, arrows, arrows_kwds, groups, components, layer, projection, scale_factor, color_map, cmap, palette, na_color, na_in_legend, size, frameon, legend_fontsize, legend_fontweight, legend_loc, legend_fontoutline, vmax, vmin, vcenter, norm, add_outline, outline_width, outline_color, ncols, hspace, wspace, title, show, save, ax, return_fig, **kwargs)
    250             groups=groups,
    251         )
--> 252         color_vector, categorical = _color_vector(
    253             adata,
    254             value_to_plot,

~/miniconda3/envs/scanpynew/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py in _color_vector(adata, values_key, values, palette, na_color)
   1267         color_map = {
   1268             k: to_hex(v)
-> 1269             for k, v in _get_palette(adata, values_key, palette=palette).items()
   1270         }
   1271         # If color_map does not have unique values, this can be slow as the

~/miniconda3/envs/scanpynew/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py in _get_palette(adata, values_key, palette)
   1233     values = pd.Categorical(adata.obs[values_key])
   1234     if palette:
-> 1235         _utils._set_colors_for_categorical_obs(adata, values_key, palette)
   1236     elif color_key not in adata.uns or len(adata.uns[color_key]) < len(
   1237         values.categories

~/miniconda3/envs/scanpynew/lib/python3.8/site-packages/scanpy/plotting/_utils.py in _set_colors_for_categorical_obs(adata, value_to_plot, palette)
    388         colors_list = [to_hex(x) for x in cmap(np.linspace(0, 1, len(categories)))]
    389     elif isinstance(palette, cabc.Mapping):
--> 390         colors_list = [to_hex(palette[k], keep_alpha=True) for k in categories]
    391     else:
    392         # check if palette is a list and convert it to a cycler, thus

~/miniconda3/envs/scanpynew/lib/python3.8/site-packages/scanpy/plotting/_utils.py in <listcomp>(.0)
    388         colors_list = [to_hex(x) for x in cmap(np.linspace(0, 1, len(categories)))]
    389     elif isinstance(palette, cabc.Mapping):
--> 390         colors_list = [to_hex(palette[k], keep_alpha=True) for k in categories]
    391     else:
    392         # check if palette is a list and convert it to a cycler, thus

KeyError: 'AtrioventricularNodePacemakerCells'
```

Example 10 (python):
```python
set(adata.obs.region)
```

Example 11 (unknown):
```unknown
{'aorta',
 'apex',
 'atria',
 'atrial septum',
 'base',
 'ductus arteriosus',
 'heart without node',
 'left ventricle',
 'node',
 'outflow tract',
 'pericardium',
 'pulmonary arches and branches',
 'right ventricle',
 'right ventricle and papillaries',
 'whole sample'}
```

Example 12 (python):
```python
#adata = adata[adata.obs['fine_grain'].isin(['CoronaryVesselAdventitialFibroblasts','MyocardialInterstitialFibroblasts1','MyocardialInterstitialFibroblasts2','Myofibroblasts','GreatVesselAdventitialFibroblasts'])].copy()
#adata = adata[adata.obs['fine_grain'].isin(['CoronaryArterialEndothelialCells','CoronaryCapillaryEndothelialCells','CoronaryVenousEndothelialCells','GreatVesselArterialEndothelialCells','GreatVesselVenousEndothelialCells'])].copy()
#adata = adata[adata.obs['fine_grain'].isin(['CoronaryPericytes','CoronarySmoothMuscleCells','DuctusArteriosusSmoothMuscleCells','GreatVesselSmoothMuscleCells'])].copy()
#adata = adata[adata.obs['coarse_grain'].isin(['Endothelium','CoronaryCapillaryEndothelialCells','CoronaryVenousEndothelialCells'])].copy()
adata = adata[adata.obs['fine_grain'].isin(['CoronaryCapillaryEndothelialCells'])].copy()
#adata=adata[adata.obs.mid_grain=='MuralCells',:].copy()
```

Example 13 (unknown):
```unknown
AnnData object with n_obs × n_vars = 0 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score'
    var: 'gene_name_scRNA-0', 'gene_id'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'age_colors', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'cycling_colors', 'donor_colors', 'fine_grain_colors', 'heart_or_greatvessels_colors', 'hvg', 'kit_10x_colors', 'modality_colors', 'neighbors', 'phase_colors', 'region_colors', 'sex_colors', 'trimester_colors', 'umap', 'week_colors'
    obsm: 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 14 (python):
```python
adata = adata[adata.obs['coarse_grain'].isin(['Endothelium'])].copy()
adata = adata[adata.obs['region'].isin(['whole sample'])].copy()
```

Example 15 (python):
```python
tmp = pd.crosstab(adata.obs['week'],adata.obs['fine_grain'], normalize='index')
tmp.plot.bar(stacked=True).legend(loc='center left', bbox_to_anchor=(1.0, 0.5))
```

Example 16 (python):
```python
<matplotlib.legend.Legend at 0x7fe61da6fe50>
```

Example 17 (python):
```python
tmp = pd.crosstab(adata.obs['week'],adata.obs['fine_grain'], normalize='index')
tmp.plot.bar(stacked=True).legend(loc='center left', bbox_to_anchor=(1.0, 0.5))
```

Example 18 (python):
```python
<matplotlib.legend.Legend at 0x7fe6237ee4c0>
```

Example 19 (python):
```python
# preserve raw count
adata.layers["counts"] = adata.X.copy()

# normalise and log-transform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key='batch_key')
#sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5)

print(sum(adata.var.highly_variable))
sc.pl.highly_variable_genes(adata)

# freeze the state in `.raw`
adata.raw = adata

# subset hvgs
adata = adata[:, adata.var.highly_variable]

adata=adata.copy()

sc.tl.score_genes_cell_cycle(adata, s_genes=s_genes, g2m_genes=g2m_genes)
#sc.pp.regress_out(adata, ['S_score','G2M_score'])

#batch correction
scvi.model.SCVI.setup_anndata(adata, 
                              layer="counts", 
                              batch_key="batch_key",
                              categorical_covariate_keys=['phase'],
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo'])

vae = scvi.model.SCVI(adata, n_hidden = 128, n_latent = 50, n_layers = 3, dispersion = 'gene-batch')
vae.train(max_epochs=50)
    
# Convergence plot
x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
plt.plot(x, vae.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
plt.clf()

# add the latent space to adata
adata.obsm["X_scVI"] = vae.get_latent_representation()

sc.tl.score_genes(adata, stress_genes, score_name='stress_score')
sc.tl.score_genes(adata, hb_genes1, score_name='hb1_score')

sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata,min_dist=0.1)
#sc.tl.tsne(adata)
sc.pl.umap(adata, color=['log1p_total_counts','n_genes','n_genes_by_counts','total_counts','pct_counts_mt','pct_counts_ribo'],ncols=5)
sc.pl.umap(adata, color=['scrublet_score','stress_score', 'hb1_score','phase'], ncols=5,)

sc.pl.umap(adata, color=['donor','age','week','trimester','region','phase','cycling','cell_or_nuclei','modality','kit_10x','FACSgate','heart_or_greatvessels','coarse_grain','mid_grain','fine_grain'],wspace=0.8,ncols=5)
Genes=['TTN','TNNT2','MYOM1','LDB3','ALDH1A2','TNNT1','KRT19','UPK3B','BNC1','COL6A3','FBLN1','PCOLCE','LRRC17','PCDH18','EMCN','ECSCR','PECAM1','CDH5','ESAM', 'LAPTM5','PTPRC','LCP1','CD53','WAS','PHOX2B','NRXN1','STMN2','GRIK2','L1CAM']
sc.pl.umap(adata, color=Genes)
```

Example 20 (python):
```python
normalizing counts per cell
    finished (0:00:00)
extracting highly variable genes
    finished (0:00:07)
--> added
    'highly_variable', boolean vector (adata.var)
    'means', float vector (adata.var)
    'dispersions', float vector (adata.var)
    'dispersions_norm', float vector (adata.var)
1116
```

Example 21 (python):
```python
calculating cell cycle phase
computing score 'S_score'
WARNING: genes are not in var_names and ignored: ['MCM5', 'FEN1', 'MCM2', 'MCM4', 'RRM1', 'UNG', 'GINS2', 'MCM6', 'CDCA7', 'PRIM1', 'RFC2', 'RPA2', 'NASP', 'GMNN', 'WDR76', 'SLBP', 'CCNE2', 'UBR7', 'POLD3', 'MSH2', 'RAD51', 'CDC45', 'CDC6', 'EXO1', 'TIPIN', 'DSCC1', 'CASP8AP2', 'USP1', 'POLA1', 'CHAF1B', 'E2F8']
    finished: added
    'S_score', score of gene set (adata.obs).
    326 total control genes are used. (0:00:00)
computing score 'G2M_score'
WARNING: genes are not in var_names and ignored: ['TMPO', 'CKAP2L', 'ANP32E', 'HJURP', 'CDCA3', 'RANGAP1', 'NCAPD2', 'CDCA2', 'PSRC1', 'LBR', 'CKAP5', 'CTCF', 'NEK2', 'G2E3', 'CBX5', 'CENPA']
    finished: added
    'G2M_score', score of gene set (adata.obs).
    724 total control genes are used. (0:00:00)
-->     'phase', cell cycle phase (adata.obs)
```

Example 22 (unknown):
```unknown
GPU available: False, used: False
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
```

Example 23 (unknown):
```unknown
Epoch 50/50: 100%|███████████| 50/50 [00:23<00:00,  2.14it/s, loss=576, v_num=1]
```

Example 24 (python):
```python
computing score 'stress_score'
WARNING: genes are not in var_names and ignored: ['JUND', 'HSP90AA1', 'HSP90AB1', 'HSPA8', 'HSPB1', 'HSPE1', 'BTG1']
    finished: added
    'stress_score', score of gene set (adata.obs).
    536 total control genes are used. (0:00:00)
computing score 'hb1_score'
WARNING: genes are not in var_names and ignored: ['HBD', 'HBE1', 'HBM', 'HBQ1']
    finished: added
    'hb1_score', score of gene set (adata.obs).
    224 total control genes are used. (0:00:00)
computing neighbors
    finished: added to `.uns['neighbors']`
    `.obsp['distances']`, distances for each pair of neighbors
    `.obsp['connectivities']`, weighted adjacency matrix (0:00:01)
computing UMAP
    finished: added
    'X_umap', UMAP coordinates (adata.obsm) (0:00:04)
```

Example 25 (unknown):
```unknown
... storing 'phase' as categorical
```

Example 26 (unknown):
```unknown
<Figure size 320x320 with 0 Axes>
```

Example 27 (python):
```python
EndothelialStalkGenes=['ACKR1','AQP1','C1QTNF9','CD36','CSRP2','EHD4','FBLN5','HSPB1','IL6ST','JAM2','LGALS3','LRG1','MEOX2','PLSCR2','CAVIN2','SELP','SPINT2','TGFBI','TGM2','TMEM176A','TMEM176B','TMEM252','TSPAN7','FLT1','VWF']
EndothelialStalkGenes = [x for x in EndothelialStalkGenes if x in adata.var_names]
print(len(EndothelialStalkGenes))
EndothelialTipGenes=['ADM','ANKRD37','C1QTNF6','CLDN5','COL4A1','COL4A2','COTL1','DLL4','EDNRB','FSCN1','GPIHBP1','HSPG2','IGFBP3','INHBB','JUP','KCNE3','KCNJ8','LAMA4','LAMB1','LXN','MARCKSL1','MCAM','MEST','N4BP3','NID2','NOTCH4','PLOD1','PLXND1','PMEPA1','PTN','RAMP3','RBP1','RGCC','RHOC','UNC5B','KDR','FLT4']
EndothelialTipGenes = [x for x in EndothelialTipGenes if x in adata.var_names]
print(len(EndothelialTipGenes))

sc.tl.score_genes(adata, EndothelialStalkGenes, score_name='EndothelialStalkScore',ctrl_size=50, gene_pool=None, n_bins=25, random_state=0, copy=False, use_raw=None)
sc.tl.score_genes(adata, EndothelialTipGenes, score_name='EndothelialTipScore',ctrl_size=50, gene_pool=None, n_bins=25, random_state=0, copy=False, use_raw=None)
sc.pl.umap(adata, color=['EndothelialStalkScore','EndothelialTipScore'])

adata.obs['TipOrStalk']= 'Tip'
adata.obs['TipOrStalk'][[x for x in adata.obs_names if adata.obs.EndothelialStalkScore[x]>adata.obs.EndothelialTipScore[x]]]='Stalk'
sc.pl.umap(adata, color=['TipOrStalk'])

plt.rcParams['figure.figsize'] = [3,2]
tmp=pd.crosstab(adata.obs['week'],adata.obs['TipOrStalk'],normalize='index')
tmp.plot.bar(stacked=True,color = ["gold","coral"]).legend(loc='center left',bbox_to_anchor=(1.0,0.5))
plt.title('Coronary Capillary Endothelial Cells')
plt.ylabel('proportion')
```

Example 28 (python):
```python
9
14
computing score 'EndothelialStalkScore'
    finished: added
    'EndothelialStalkScore', score of gene set (adata.obs).
    267 total control genes are used. (0:00:00)
computing score 'EndothelialTipScore'
    finished: added
    'EndothelialTipScore', score of gene set (adata.obs).
    275 total control genes are used. (0:00:00)
```

Example 29 (unknown):
```unknown
... storing 'TipOrStalk' as categorical
```

Example 30 (unknown):
```unknown
Text(0, 0.5, 'proportion')
```

Example 31 (python):
```python
sc.tl.rank_genes_groups(adata, 'TipOrStalk', method='t-test', key_added = "t-test")

ClusterOfInterest='Tip'
DEGenesForCluster = [x[ClusterOfInterest] for x in adata.uns['t-test']['names'][:50]]
sc.pl.dotplot(adata, DEGenesForCluster ,groupby='TipOrStalk',standard_scale='var',cmap='YlGn')

ClusterOfInterest='Stalk'
DEGenesForCluster = [x[ClusterOfInterest] for x in adata.uns['t-test']['names'][:50]]
sc.pl.dotplot(adata, DEGenesForCluster ,groupby='TipOrStalk',standard_scale='var',cmap='YlGn')
```

Example 32 (python):
```python
ranking genes
    finished: added to `.uns['t-test']`
    'names', sorted np.recarray to be indexed by group ids
    'scores', sorted np.recarray to be indexed by group ids
    'logfoldchanges', sorted np.recarray to be indexed by group ids
    'pvals', sorted np.recarray to be indexed by group ids
    'pvals_adj', sorted np.recarray to be indexed by group ids (0:00:00)
```

Example 33 (python):
```python
sc.tl.leiden(adata, resolution=0.25, key_added='leiden')
sc.pl.umap(adata, color='leiden',legend_loc='on data')

sc.tl.rank_genes_groups(adata, 'leiden', method='t-test', key_added = "t-test")

ClusterOfInterest='0'
DEGenesForCluster = [x[ClusterOfInterest] for x in adata.uns['t-test']['names'][:50]]
sc.pl.dotplot(adata, DEGenesForCluster ,groupby='leiden',standard_scale='var',cmap='YlGn')
```

Example 34 (python):
```python
running Leiden clustering
    finished: found 5 clusters and added
    'leiden', the cluster labels (adata.obs, categorical) (0:00:00)
```

Example 35 (python):
```python
ranking genes
    finished: added to `.uns['t-test']`
    'names', sorted np.recarray to be indexed by group ids
    'scores', sorted np.recarray to be indexed by group ids
    'logfoldchanges', sorted np.recarray to be indexed by group ids
    'pvals', sorted np.recarray to be indexed by group ids
    'pvals_adj', sorted np.recarray to be indexed by group ids (0:00:00)
```

Example 36 (python):
```python
plt.rcParams['figure.figsize'] = [3,2]
tmp=pd.crosstab(adata.obs['week'],adata.obs['TipOrStalk'],normalize='index')
tmp.plot.bar(stacked=True,color = ["gold","coral"]).legend(loc='center left',bbox_to_anchor=(1.0,0.5))
plt.title('Coronary Capillary Endothelial Cells')
plt.ylabel('proportion')
```

Example 37 (unknown):
```unknown
Text(0, 0.5, 'proportion')
```

Example 38 (python):
```python
bdata=adata[adata.obs.TipOrStalk=='Tip',:].copy()
bdatanames=bdata.obs_names
np.savetxt('Index/TipVsStalkCapillaryEndothelial/TipBarcodes.txt', bdatanames, delimiter="\n", fmt="%s")

bdata=adata[adata.obs.TipOrStalk=='Stalk',:].copy()
bdatanames=bdata.obs_names
np.savetxt('Index/TipVsStalkCapillaryEndothelial/StalkBarcodes.txt', bdatanames, delimiter="\n", fmt="%s")
```

Example 39 (python):
```python
tmp = pd.crosstab(adata.obs['week'],adata.obs['fine_grain'], normalize='index')
tmp.plot.bar(stacked=True).legend(loc='lower right')
```

Example 40 (python):
```python
<matplotlib.legend.Legend at 0x7f306f482c40>
```

Example 41 (python):
```python
StalkGenes=['ACKR1','AQP1','C1QTNF9','CD36','CSRP2','EHD4','FBLN5','HSPB1','IL6ST','JAM2','LGALS3','LRG1','MEOX2','PLSCR2','CAVIN2','SELP','SPINT2','TGFBI','TGM2','TMEM176A','TMEM176B','TMEM252','TSPAN7','FLT1','VWF']
TipGenes=['ADM','ANKRD37','C1QTNF6','CLDN5','COL4A1','COL4A2','COTL1','DLL4','EDNRB','FSCN1','GPIHBP1','HSPG2','IGFBP3','INHBB','JUP','KCNE3','KCNJ8','LAMA4','LAMB1','LXN','MARCKSL1','MCAM','MEST','N4BP3','NID2','NOTCH4','PLOD1','PLXND1','PMEPA1','PTN','RAMP3','RBP1','RGCC','RHOC','UNC5B','KDR','FLT4']
sc.tl.score_genes(adata, StalkGenes, score_name='stalk_score')
sc.tl.score_genes(adata, TipGenes, score_name='tip_score')
sc.pl.umap(adata, color=['tip_score','week','fine_grain'])
sc.pl.dotplot(adata, ['tip_score','stalk_score'] ,groupby='week',categories_order=['4W', '5W', '6W', '7W', '8W', '9W', '10W', '12W', '13W', '14W', '15W', '20W'],standard_scale='var',cmap='YlGn')
sc.pl.dotplot(adata, ['tip_score'] ,groupby='week',categories_order=['4W', '5W', '6W', '7W', '8W', '9W', '10W', '12W', '13W', '14W', '15W', '20W'],standard_scale='var',cmap='YlGn')
```

Example 42 (python):
```python
computing score 'stalk_score'
WARNING: genes are not in var_names and ignored: ['CSRP2', 'EHD4', 'HSPB1', 'IL6ST', 'JAM2', 'LRG1', 'PLSCR2', 'CAVIN2', 'TMEM176A', 'TMEM176B', 'TMEM252']
    finished: added
    'stalk_score', score of gene set (adata.obs).
    483 total control genes are used. (0:00:00)
computing score 'tip_score'
WARNING: genes are not in var_names and ignored: ['C1QTNF6', 'COL4A1', 'COL4A2', 'COTL1', 'FSCN1', 'INHBB', 'JUP', 'KCNJ8', 'LAMB1', 'LXN', 'MARCKSL1', 'MEST', 'N4BP3', 'NID2', 'PLOD1', 'PLXND1', 'PMEPA1', 'PTN', 'RBP1', 'RHOC', 'KDR', 'FLT4']
    finished: added
    'tip_score', score of gene set (adata.obs).
    333 total control genes are used. (0:00:00)
```

Example 43 (python):
```python
sc.tl.leiden(adata, resolution=0.25, key_added='leiden')
sc.pl.umap(adata, color='leiden',legend_loc='on data')
```

Example 44 (python):
```python
running Leiden clustering
    finished: found 5 clusters and added
    'leiden', the cluster labels (adata.obs, categorical) (0:00:00)
```

Example 45 (python):
```python
bdata=adata[adata.obs.fine_grain=='CoronaryCapillaryEndothelialCells',:].copy()
bdata=bdata[bdata.obs.leiden!='4',:].copy()
StalkGenes=['ACKR1','AQP1','C1QTNF9','CD36','CSRP2','EHD4','FBLN5','HSPB1','IL6ST','JAM2','LGALS3','LRG1','MEOX2','PLSCR2','CAVIN2','SELP','SPINT2','TGFBI','TGM2','TMEM176A','TMEM176B','TMEM252','TSPAN7','FLT1','VWF']
TipGenes=['ADM','ANKRD37','C1QTNF6','CLDN5','COL4A1','COL4A2','COTL1','DLL4','EDNRB','FSCN1','GPIHBP1','HSPG2','IGFBP3','INHBB','JUP','KCNE3','KCNJ8','LAMA4','LAMB1','LXN','MARCKSL1','MCAM','MEST','N4BP3','NID2','NOTCH4','PLOD1','PLXND1','PMEPA1','PTN','RAMP3','RBP1','RGCC','RHOC','UNC5B','KDR','FLT4']
sc.tl.score_genes(bdata, StalkGenes, score_name='stalk_score')
sc.tl.score_genes(bdata, TipGenes, score_name='tip_score')
sc.pl.umap(bdata, color=['tip_score','stalk_score','week','fine_grain'])
sc.pl.dotplot(bdata, ['tip_score','stalk_score'] ,groupby='week',categories_order=['4W', '5W', '6W', '7W', '8W', '9W', '10W', '12W', '13W', '14W', '15W', '20W'],standard_scale='var',cmap='YlGn')
sc.pl.dotplot(bdata, ['tip_score'] ,groupby='week',categories_order=['4W', '5W', '6W', '7W', '8W', '9W', '10W', '12W', '13W', '14W', '15W', '20W'],standard_scale='var',cmap='YlGn')
```

Example 46 (python):
```python
computing score 'stalk_score'
WARNING: genes are not in var_names and ignored: ['CSRP2', 'EHD4', 'HSPB1', 'IL6ST', 'JAM2', 'LRG1', 'PLSCR2', 'CAVIN2', 'TMEM176A', 'TMEM176B', 'TMEM252']
    finished: added
    'stalk_score', score of gene set (adata.obs).
    583 total control genes are used. (0:00:00)
computing score 'tip_score'
WARNING: genes are not in var_names and ignored: ['C1QTNF6', 'COL4A1', 'COL4A2', 'COTL1', 'FSCN1', 'INHBB', 'JUP', 'KCNJ8', 'LAMB1', 'LXN', 'MARCKSL1', 'MEST', 'N4BP3', 'NID2', 'PLOD1', 'PLXND1', 'PMEPA1', 'PTN', 'RBP1', 'RHOC', 'KDR', 'FLT4']
    finished: added
    'tip_score', score of gene set (adata.obs).
    330 total control genes are used. (0:00:00)
```

Example 47 (python):
```python
sc.pp.neighbors(bdata, use_rep="X_scVI")
sc.tl.umap(bdata,min_dist=0.1)
sc.tl.leiden(bdata, resolution=0.25, key_added='leiden')
sc.pl.umap(bdata, color='leiden',legend_loc='on data')
```

Example 48 (python):
```python
computing neighbors
    finished: added to `.uns['neighbors']`
    `.obsp['distances']`, distances for each pair of neighbors
    `.obsp['connectivities']`, weighted adjacency matrix (0:00:00)
computing UMAP
    finished: added
    'X_umap', UMAP coordinates (adata.obsm) (0:00:03)
running Leiden clustering
    finished: found 4 clusters and added
    'leiden', the cluster labels (adata.obs, categorical) (0:00:00)
```

Example 49 (python):
```python
cdata=bdata[bdata.obs.leiden=='1',:].copy()
StalkGenes=['ACKR1','AQP1','C1QTNF9','CD36','CSRP2','EHD4','FBLN5','HSPB1','IL6ST','JAM2','LGALS3','LRG1','MEOX2','PLSCR2','CAVIN2','SELP','SPINT2','TGFBI','TGM2','TMEM176A','TMEM176B','TMEM252','TSPAN7','FLT1','VWF']
TipGenes=['ADM','ANKRD37','C1QTNF6','CLDN5','COL4A1','COL4A2','COTL1','DLL4','EDNRB','FSCN1','GPIHBP1','HSPG2','IGFBP3','INHBB','JUP','KCNE3','KCNJ8','LAMA4','LAMB1','LXN','MARCKSL1','MCAM','MEST','N4BP3','NID2','NOTCH4','PLOD1','PLXND1','PMEPA1','PTN','RAMP3','RBP1','RGCC','RHOC','UNC5B','KDR','FLT4']
sc.tl.score_genes(cdata, StalkGenes, score_name='stalk_score')
sc.tl.score_genes(cdata, TipGenes, score_name='tip_score')
sc.pl.umap(cdata, color=['tip_score','stalk_score','week','fine_grain'])
sc.pl.dotplot(cdata, ['tip_score','stalk_score'] ,groupby='week',categories_order=['4W', '5W', '6W', '7W', '8W', '9W', '10W', '12W', '13W', '14W', '15W', '20W'],standard_scale='var',cmap='YlGn')
sc.pl.dotplot(cdata, ['tip_score'] ,groupby='week',categories_order=['4W', '5W', '6W', '7W', '8W', '9W', '10W', '12W', '13W', '14W', '15W', '20W'],standard_scale='var',cmap='YlGn')
```

Example 50 (python):
```python
computing score 'stalk_score'
WARNING: genes are not in var_names and ignored: ['CSRP2', 'EHD4', 'HSPB1', 'IL6ST', 'JAM2', 'LRG1', 'PLSCR2', 'CAVIN2', 'TMEM176A', 'TMEM176B', 'TMEM252']
    finished: added
    'stalk_score', score of gene set (adata.obs).
    586 total control genes are used. (0:00:00)
computing score 'tip_score'
WARNING: genes are not in var_names and ignored: ['C1QTNF6', 'COL4A1', 'COL4A2', 'COTL1', 'FSCN1', 'INHBB', 'JUP', 'KCNJ8', 'LAMB1', 'LXN', 'MARCKSL1', 'MEST', 'N4BP3', 'NID2', 'PLOD1', 'PLXND1', 'PMEPA1', 'PTN', 'RBP1', 'RHOC', 'KDR', 'FLT4']
    finished: added
    'tip_score', score of gene set (adata.obs).
    330 total control genes are used. (0:00:00)
```

Example 51 (python):
```python
sc.pl.umap(bdata, color=['tip_score','stalk_score','week','fine_grain'])
```

Example 52 (python):
```python
ClusterOfInterest='1'
sc.tl.rank_genes_groups(bdata, 'leiden', method='t-test', key_added = "t-test")
DEGenesForCluster = [x[ClusterOfInterest] for x in bdata.uns['t-test']['names'][:50]]
sc.pl.dotplot(bdata, DEGenesForCluster ,groupby='leiden',standard_scale='var',cmap='PRGn')
#sc.pl.umap(adata, color=DEGenesForCluster)
```

Example 53 (python):
```python
ranking genes
    finished: added to `.uns['t-test']`
    'names', sorted np.recarray to be indexed by group ids
    'scores', sorted np.recarray to be indexed by group ids
    'logfoldchanges', sorted np.recarray to be indexed by group ids
    'pvals', sorted np.recarray to be indexed by group ids
    'pvals_adj', sorted np.recarray to be indexed by group ids (0:00:00)
```

Example 54 (python):
```python
bdata=adata[adata.obs.fine_grain=='CoronaryCapillaryEndothelialCells',:].copy()
StalkandTipGenes=['ACKR1','AQP1','C1QTNF9','CD36','CSRP2','EHD4','FBLN5','HSPB1','IL6ST','JAM2','LGALS3','LRG1','MEOX2','PLSCR2','CAVIN2','SELP','SPINT2','TGFBI','TGM2','TMEM176A','TMEM176B','TMEM252','TSPAN7','FLT1','VWF','ADM','ANKRD37','C1QTNF6','CLDN5','COL4A1','COL4A2','COTL1','DLL4','EDNRB','FSCN1','GPIHBP1','HSPG2','IGFBP3','INHBB','JUP','KCNE3','KCNJ8','LAMA4','LAMB1','LXN','MARCKSL1','MCAM','MEST','N4BP3','NID2','NOTCH4','PLOD1','PLXND1','PMEPA1','PTN','RAMP3','RBP1','RGCC','RHOC','UNC5B','KDR','FLT4']
bdata=bdata[:,[x for x in bdata.var_names if x  in StalkandTipGenes]].copy()
```

Example 55 (python):
```python
sc.pp.neighbors(bdata, use_rep="X_scVI")
sc.tl.umap(bdata,min_dist=0.1)
```

Example 56 (python):
```python
computing neighbors
    finished: added to `.uns['neighbors']`
    `.obsp['distances']`, distances for each pair of neighbors
    `.obsp['connectivities']`, weighted adjacency matrix (0:00:01)
computing UMAP
    finished: added
    'X_umap', UMAP coordinates (adata.obsm) (0:00:03)
```

Example 57 (python):
```python
sc.pl.umap(bdata, color='tip_score')
```

Example 58 (python):
```python
sc.pl.umap(bdata, color='stalk_score')
```

Example 59 (python):
```python
ClusterOfInterest='0'
sc.tl.rank_genes_groups(adata, 'leiden', method='t-test', key_added = "t-test")
DEGenesForCluster = [x[ClusterOfInterest] for x in adata.uns['t-test']['names'][:50]]
sc.pl.dotplot(adata, DEGenesForCluster ,groupby='leiden',standard_scale='var',cmap='PRGn')
#sc.pl.umap(adata, color=DEGenesForCluster)
```

Example 60 (python):
```python
ranking genes
    finished: added to `.uns['t-test']`
    'names', sorted np.recarray to be indexed by group ids
    'scores', sorted np.recarray to be indexed by group ids
    'logfoldchanges', sorted np.recarray to be indexed by group ids
    'pvals', sorted np.recarray to be indexed by group ids
    'pvals_adj', sorted np.recarray to be indexed by group ids (0:00:00)
```

Example 61 (python):
```python
set(adata.obs.fine_grain)
```

Example 62 (unknown):
```unknown
{'CoronaryArterialEndothelialCells',
 'CoronaryCapillaryEndothelialCells',
 'CoronaryVenousEndothelialCells',
 'EndocardialCells',
 'EndocardialCushionCells',
 'GreatVesselArterialEndothelialCells',
 'GreatVesselVenousEndothelialCells',
 'LymphaticEndothelialCells',
 'ValveEndothelialCells'}
```

Example 63 (python):
```python
pd.crosstab(adata.obs['fine_grain'],adata.obs['age'])
```

Example 64 (python):
```python
adata.obs['type']= 'na'
adata.obs['type'][[x for x in adata.obs_names if adata.obs.fine_grain[x] in ['LymphaticEndothelialCells']]]='Yes'
tmp = pd.crosstab(adata.obs['week'],adata.obs['type'], normalize='index')
tmp.plot.bar(stacked=True).legend(loc='lower right')
```

Example 65 (python):
```python
<matplotlib.legend.Legend at 0x7f2df6be6be0>
```

Example 66 (python):
```python
adata.obs['type']= 'na'
adata.obs['type'][[x for x in adata.obs_names if adata.obs.fine_grain[x] in ['CoronaryCapillaryEndothelialCells']]]='Yes'
tmp = pd.crosstab(adata.obs['week'],adata.obs['fine_grain'], normalize='index')
tmp.plot.bar(stacked=True).legend(loc='center left', bbox_to_anchor=(1.0, 0.5))
```

Example 67 (python):
```python
<matplotlib.legend.Legend at 0x7f2df9a5aa90>
```

---

## Import required modules¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/Visium/Visium_Intagration-Annotation/Visium_integration_no5_plotting_temporal-tissue-annotations.html

**Contents:**
- Import required modules¶
- Prepare functions¶
- Read in anndata¶
- Plot the uncropped section¶
- Add annotation¶
- Plots for figures¶
- HE, whole heart, with scale¶
- Tissue annotations¶

**Examples:**

Example 1 (python):
```python
# this is for the jupyter notebook at JupyterLab
# to make sys.path between the terminal and Jupyter same
import sys

print(sys.path) # especially the path to site-packages are different... so thats why couldn't find correct package...

# remove weird paths...
sys.path = sys.path[:4]+[sys.path[6]]
print(sys.path)
```

Example 2 (unknown):
```unknown
['/nfs/team205/kk18/notebooks/P48_Foetal-heart_Visium/notebooks/notebooks_for_manuscript', '/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python39.zip', '/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python3.9', '/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python3.9/lib-dynload', '', '/nfs/users/nfs_k/kk18/.local/lib/python3.9/site-packages', '/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python3.9/site-packages']
['/nfs/team205/kk18/notebooks/P48_Foetal-heart_Visium/notebooks/notebooks_for_manuscript', '/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python39.zip', '/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python3.9', '/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python3.9/lib-dynload', '/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python3.9/site-packages']
```

Example 3 (python):
```python
import warnings
warnings.simplefilter(action='ignore', category=FutureWarning)

import scanpy as sc
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 
import seaborn as sns
import os 
from mpl_toolkits.axes_grid1 import make_axes_locatable

from cell2location.plt import plot_spatial
```

Example 4 (python):
```python
/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python3.9/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm
Global seed set to 0
/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python3.9/site-packages/umap/distances.py:1063: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python3.9/site-packages/umap/distances.py:1071: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python3.9/site-packages/umap/distances.py:1086: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python3.9/site-packages/umap/umap_.py:660: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
```

Example 5 (python):
```python
import session_info
session_info.show()
```

Example 6 (python):
```python
-----
cell2location       0.1.3
matplotlib          3.7.1
mpl_toolkits        NA
numpy               1.24.3
pandas              2.0.1
scanpy              1.9.3
seaborn             0.12.2
session_info        1.0.0
-----
```

Example 7 (python):
```python
PIL                 9.5.0
absl                NA
anndata             0.9.1
asttokens           NA
attr                23.1.0
backcall            0.2.0
certifi             2023.05.07
cffi                1.15.1
charset_normalizer  3.1.0
chex                0.1.7
comm                0.1.3
contextlib2         NA
cycler              0.10.0
cython_runtime      NA
dateutil            2.8.2
debugpy             1.6.7
decorator           5.1.1
defusedxml          0.7.1
docrep              0.3.2
etils               1.2.0
executing           1.2.0
flax                0.6.10
fsspec              2023.5.0
google              NA
h5py                3.8.0
idna                3.4
igraph              0.10.4
importlib_metadata  NA
importlib_resources NA
ipykernel           6.23.0
ipython_genutils    0.2.0
jax                 0.4.9
jaxlib              0.4.9
jedi                0.18.2
joblib              1.2.0
kiwisolver          1.4.4
leidenalg           0.9.1
lightning_fabric    1.9.5
lightning_utilities 0.8.0
llvmlite            0.40.0
ml_collections      NA
ml_dtypes           0.1.0
mpmath              1.3.0
msgpack             1.0.5
mudata              0.2.2
multipledispatch    0.6.0
natsort             8.3.1
numba               0.57.0
numpyro             0.11.0
nvfuser             NA
opt_einsum          v3.3.0
optax               0.1.5
packaging           23.1
parso               0.8.3
patsy               0.5.3
pexpect             4.8.0
pickleshare         0.7.5
pkg_resources       NA
platformdirs        3.5.0
prompt_toolkit      3.0.38
psutil              5.9.5
ptyprocess          0.7.0
pure_eval           0.2.2
pycparser           2.21
pydev_ipython       NA
pydevconsole        NA
pydevd              2.9.5
pydevd_file_utils   NA
pydevd_plugins      NA
pydevd_tracing      NA
pygments            2.15.1
pynndescent         0.5.10
pyparsing           3.0.9
pyro                1.8.4
pytorch_lightning   1.9.5
pytz                2023.3
requests            2.30.0
rich                NA
scipy               1.10.1
scvi                0.20.3
setuptools          66.0.0
six                 1.16.0
sklearn             1.2.2
stack_data          0.6.2
statsmodels         0.14.0
sympy               1.12
texttable           1.6.7
threadpoolctl       3.1.0
toolz               0.12.0
torch               2.0.1+cu117
torchmetrics        0.11.4
tornado             6.3.1
tqdm                4.65.0
traitlets           5.9.0
tree                0.1.8
typing_extensions   NA
umap                0.5.3
urllib3             2.0.2
wcwidth             0.2.6
yaml                6.0
zipp                NA
zmq                 25.0.2
zoneinfo            NA
```

Example 8 (unknown):
```unknown
-----
IPython             8.13.2
jupyter_client      8.2.0
jupyter_core        5.3.0
notebook            6.5.4
-----
Python 3.9.16 (main, Mar  8 2023, 14:00:05) [GCC 11.2.0]
Linux-5.4.0-171-generic-x86_64-with-glibc2.27
-----
Session information updated at 2024-03-21 08:36
```

Example 9 (python):
```python
sc.settings.set_figure_params(dpi=80,dpi_save=300)
```

Example 10 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk'
```

Example 11 (python):
```python
# This function selects the data for multiple slides from the spatial anndata object.
# Modified `select_slide` in cell2loc for multiple slides

# @param adata: anndata object with multiple spatial experiments
# @param s: list of selected slides
# @param s_col: column in adata.obs listing experiment name for each location

def select_multislide(adata, s, s_col='sample'):

    slides = adata[adata.obs[s_col].isin(s), :]
    s_keys = list(slides.uns['spatial'].keys())
    
    uns_dict = {}
    for ss in s:
        s_spatial=np.array(s_keys)[[ss in k for k in s_keys]][0]
        uns_dict[s_spatial]=slides.uns['spatial'][s_spatial]
    
    slides.uns['spatial'] = uns_dict

    return slides
```

Example 12 (python):
```python
from matplotlib_scalebar.scalebar import ScaleBar
def get_pixel_size_visium(adata, library_id, spot_diameter_real = 55, img='hires'):
    '''
    Utility function to get size of pixels from visium image in AnnData.
    Useful to plot scalebars with matplotlib_scalebar.ScaleBar
    
    Params:
    -------
    - adata: AnnData object storing image info in adata.uns[spatial]
    - library_id: string storing ID for image, must be a key in adata.uns[spatial]
    - spot_diameter_real: numeric storing real diameter of spot, in whatever unit you need 
        (default is 55, for 55 microns of Visium 10X spots)
    - img: which image to use (default: lowres)
    '''

    ## get scale factor converting original pixel positions (adata.obsm['spatial']) to 
    # pixel positions in image
    scalef = adata.uns['spatial'][library_id]['scalefactors']['tissue_{i}_scalef'.format(i=img)]
    ## get spot diameter in image pixels
    spot_diameter_img = adata.uns['spatial'][library_id]['scalefactors']['spot_diameter_fullres'] * scalef
    ## Calculate pixel size
    pixel_size_real = spot_diameter_real/spot_diameter_img
    return(pixel_size_real)
```

Example 13 (python):
```python
adata_ffpe = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/visium-FFPE_foetal_raw.h5ad')

# add abundance to .obs
df = adata_ffpe.obsm['q05_cell_abundance_w_sf'].copy()
## rename columns
df.columns = [x.replace('q05cell_abundance_w_sf_','') for x in df.columns]
adata_ffpe.obs[df.columns]=df.copy()

# set symboßl as var_names
adata_ffpe.var.set_index('SYMBOL',inplace=True)
adata_ffpe
```

Example 14 (unknown):
```unknown
AnnData object with n_obs × n_vars = 19973 × 18085
    obs: 'in_tissue', 'array_row', 'array_col', 'sample', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'mt_frac', 'sangerID', 'Publication', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'status', 'JC_edit', 'AtrialCardiomyocytesLeft', 'AtrialCardiomyocytesRight', 'VentricularCardiomyocytesLeftCompact', 'VentricularCardiomyocytesRightCompact', 'VentricularCardiomyocytesLeftTrabeculated', 'VentricularCardiomyocytesRightTrabeculated', 'SinoatrialNodeCardiomyocytes', 'AtrioventricularNodeCardiomyocytes', 'VentricularConductionSystemProximal', 'VentricularConductionSystemDistal', 'GreatVesselAdventitialFibroblasts', 'CoronaryVesselAdventitialFibroblasts', 'MyocardialInterstitialFibroblasts1', 'MyocardialInterstitialFibroblasts2', 'Myofibroblasts', 'LymphNodeFibroblasticReticularCells', 'ValveInterstitialCells', 'GreatVesselSmoothMuscleCells', 'CoronarySmoothMuscleCells', 'CoronaryPericytes', 'GreatVesselArterialEndothelialCells', 'GreatVesselVenousEndothelialCells', 'CoronaryArterialEndothelialCells', 'CoronaryVenousEndothelialCells', 'CoronaryCapillaryEndothelialCells', 'EndocardialCells', 'EndocardialCushionCells', 'ValveEndothelialCells', 'LymphaticEndothelialCells', 'MesothelialEpicardialCells', 'EpicardiumDerivedCells', 'NeuronPrecursors', 'ChromaffinCells', 'SympatheticNeurons', 'ParaympatheticNeurons', 'SchwannCellPrecursors', 'SchwannCells', 'MonocytesMPOpos', 'Monocytes', 'MonocyteDerivedCells', 'MacrophagesCX3CR1pos', 'MacrophagesTIMD4pos', 'MacrophagesLYVE1pos', 'MacrophagesATF3pos', 'DendriticCellsType1', 'DendriticCellsMature', 'PlasmacytoidDendriticCells', 'MastCells', 'Megakaryocytes', 'TCellsCD4pos', 'TCellsCD8pos', 'TregsCD4pos', 'ProBCells', 'BCells', 'BCellsMS4A1pos', 'NaturalKillerCells', 'InnateLymphoidCells'
    var: 'gene_ids', 'feature_types', 'genome', 'mt'
    uns: 'spatial'
    obsm: 'MT', 'means_cell_abundance_w_sf', 'q05_cell_abundance_w_sf', 'q95_cell_abundance_w_sf', 'spatial', 'stds_cell_abundance_w_sf'
```

Example 15 (python):
```python
adata_oct = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/visium-OCT_foetal_raw.h5ad')

# add abundance to .obs
df = adata_oct.obsm['q05_cell_abundance_w_sf'].copy()
## rename columns
df.columns = [x.replace('q05cell_abundance_w_sf_','') for x in df.columns]
adata_oct.obs[df.columns]=df.copy()

# set symboßl as var_names
adata_oct.var.set_index('SYMBOL',inplace=True)
adata_oct
```

Example 16 (unknown):
```unknown
AnnData object with n_obs × n_vars = 14261 × 33538
    obs: 'in_tissue', 'array_row', 'array_col', 'sample', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'mt_frac', 'sangerID', 'Publication', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'status', 'cell2loc-map_group', 'JC_edit', 'sample_for_cell2loc', 'AtrialCardiomyocytesLeft', 'AtrialCardiomyocytesRight', 'VentricularCardiomyocytesLeftCompact', 'VentricularCardiomyocytesRightCompact', 'VentricularCardiomyocytesLeftTrabeculated', 'VentricularCardiomyocytesRightTrabeculated', 'SinoatrialNodeCardiomyocytes', 'AtrioventricularNodeCardiomyocytes', 'VentricularConductionSystemProximal', 'VentricularConductionSystemDistal', 'GreatVesselAdventitialFibroblasts', 'CoronaryVesselAdventitialFibroblasts', 'MyocardialInterstitialFibroblasts1', 'MyocardialInterstitialFibroblasts2', 'Myofibroblasts', 'LymphNodeFibroblasticReticularCells', 'ValveInterstitialCells', 'GreatVesselSmoothMuscleCells', 'CoronarySmoothMuscleCells', 'CoronaryPericytes', 'GreatVesselArterialEndothelialCells', 'GreatVesselVenousEndothelialCells', 'CoronaryArterialEndothelialCells', 'CoronaryVenousEndothelialCells', 'CoronaryCapillaryEndothelialCells', 'EndocardialCells', 'EndocardialCushionCells', 'ValveEndothelialCells', 'LymphaticEndothelialCells', 'MesothelialEpicardialCells', 'EpicardiumDerivedCells', 'NeuronPrecursors', 'ChromaffinCells', 'SympatheticNeurons', 'ParaympatheticNeurons', 'SchwannCellPrecursors', 'SchwannCells', 'MonocytesMPOpos', 'Monocytes', 'MonocyteDerivedCells', 'MacrophagesCX3CR1pos', 'MacrophagesTIMD4pos', 'MacrophagesLYVE1pos', 'MacrophagesATF3pos', 'DendriticCellsType1', 'DendriticCellsMature', 'PlasmacytoidDendriticCells', 'MastCells', 'Megakaryocytes', 'TCellsCD4pos', 'TCellsCD8pos', 'TregsCD4pos', 'ProBCells', 'BCells', 'BCellsMS4A1pos', 'NaturalKillerCells', 'InnateLymphoidCells'
    var: 'gene_ids', 'feature_types', 'genome', 'mt'
    uns: 'spatial'
    obsm: 'MT', 'means_cell_abundance_w_sf', 'q05_cell_abundance_w_sf', 'q95_cell_abundance_w_sf', 'spatial', 'stds_cell_abundance_w_sf'
```

Example 17 (python):
```python
adata_oct_20pcw = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/visium-OCT-LV_foetal_raw.h5ad')

# add abundance to .obs
df = adata_oct_20pcw.obsm['q05_cell_abundance_w_sf'].copy()
## rename columns
df.columns = [x.replace('q05cell_abundance_w_sf_','') for x in df.columns]
adata_oct_20pcw.obs[df.columns]=df.copy()

# set symboßl as var_names
adata_oct_20pcw.var.set_index('SYMBOL',inplace=True)
adata_oct_20pcw
```

Example 18 (unknown):
```unknown
AnnData object with n_obs × n_vars = 6767 × 33538
    obs: 'in_tissue', 'array_row', 'array_col', 'sample', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'mt_frac', 'sangerID', 'Publication', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'status', 'cell2loc-map_group', 'JC_edit', 'sample_for_cell2loc', 'VentricularCardiomyocytesLeftCompact', 'VentricularCardiomyocytesLeftTrabeculated', 'VentricularConductionSystemProximal', 'VentricularConductionSystemDistal', 'CoronaryVesselAdventitialFibroblasts', 'MyocardialInterstitialFibroblasts1', 'MyocardialInterstitialFibroblasts2', 'Myofibroblasts', 'LymphNodeFibroblasticReticularCells', 'CoronarySmoothMuscleCells', 'CoronaryPericytes', 'CoronaryArterialEndothelialCells', 'CoronaryVenousEndothelialCells', 'CoronaryCapillaryEndothelialCells', 'EndocardialCells', 'LymphaticEndothelialCells', 'MesothelialEpicardialCells', 'EpicardiumDerivedCells', 'NeuronPrecursors', 'ChromaffinCells', 'SympatheticNeurons', 'ParaympatheticNeurons', 'SchwannCellPrecursors', 'SchwannCells', 'MonocytesMPOpos', 'Monocytes', 'MonocyteDerivedCells', 'MacrophagesCX3CR1pos', 'MacrophagesTIMD4pos', 'MacrophagesLYVE1pos', 'MacrophagesATF3pos', 'DendriticCellsType1', 'DendriticCellsMature', 'PlasmacytoidDendriticCells', 'MastCells', 'Megakaryocytes', 'TCellsCD4pos', 'TCellsCD8pos', 'TregsCD4pos', 'ProBCells', 'BCells', 'BCellsMS4A1pos', 'NaturalKillerCells', 'InnateLymphoidCells'
    var: 'gene_ids', 'feature_types', 'genome', 'mt'
    uns: 'spatial'
    obsm: 'MT', 'means_cell_abundance_w_sf', 'q05_cell_abundance_w_sf', 'q95_cell_abundance_w_sf', 'spatial', 'stds_cell_abundance_w_sf'
```

Example 19 (unknown):
```unknown
slide_ids = {
    '5PCW':'HCAHeartST11605168',
    '7PCW':'HCAHeartST10317185',
    '13PCW':'HCAHeartST11605165',
    '16PCW':'BHF_F_Hea13295830', # FFPE
    '20PCW':'HCAHeartST13162330'
}
```

Example 20 (python):
```python
for week,slide_id in slide_ids.items():
    if week=='16PCW':
        slide = select_multislide(adata_ffpe, [slide_id])
        slide_copy = slide.copy()
        slide.uns['spatial'] = {}
        slide.uns['spatial'][slide_id] = [slide_copy.uns['spatial'][x] for x in slide_copy.uns['spatial'].keys() if slide_id in x][0]
    elif week=='20PCW':
        slide = select_multislide(adata_oct_20pcw, [slide_id])
    else:
        slide = select_multislide(adata_oct, [slide_id])
    
    # get pixel size
    pix_size = get_pixel_size_visium(slide, slide_id, spot_diameter_real = 55)
        
    with mpl.rc_context({#'axes.facecolor':  'black',
                         'figure.figsize': [4.5, 5]}):
        fig, ax = plt.subplots()
        sc.pl.spatial(slide, cmap='RdPu',
                      color=['JC_edit'], 
                      ncols=5, size=0, 
                      img_key='hires', alpha_img=1,wspace=0.3,
                      vmin=0, vmax='p99.5',# palette=palette
                      legend_loc=None,
                      title=f'{week}, {slide_id}, H&E',
                      show=False,ax=ax
                     )
        scalebar = ScaleBar(pix_size, "um", length_fraction=0.25, frameon=False, location='upper right')
        ax.add_artist(scalebar)
        plt.show()
```

Example 21 (unknown):
```unknown
/tmp/ipykernel_108365/2805736760.py:18: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  slides.uns['spatial'] = uns_dict
```

Example 22 (unknown):
```unknown
/tmp/ipykernel_108365/2805736760.py:18: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  slides.uns['spatial'] = uns_dict
```

Example 23 (unknown):
```unknown
/tmp/ipykernel_108365/2805736760.py:18: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  slides.uns['spatial'] = uns_dict
```

Example 24 (unknown):
```unknown
/tmp/ipykernel_108365/2805736760.py:18: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  slides.uns['spatial'] = uns_dict
```

Example 25 (unknown):
```unknown
/tmp/ipykernel_108365/2805736760.py:18: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  slides.uns['spatial'] = uns_dict
```

Example 26 (python):
```python
# read in modify annotations
adata_annot = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/visium_foetal_healthy-all_raw.h5ad',backed='r')

# modify annotations for matching with factor analysis
annotation_edit_dict = {
   'AV valves':'Valves',
    'Aortic valve':'Valves',
    'Valve apparatus':'Valves',
    'Atrium - Left':'Atrium',
     'Atrium - Right':'Atrium',
    'Ventricle - Compact - Left':'Ventricle - Compact',
    'Ventricle - Compact - Right':'Ventricle - Compact',
    'Ventricle - Trabeculated - Left':'Ventricle - Trabeculated',
    'Ventricle - Trabeculated - Right':'Ventricle - Trabeculated',
}
adata_annot.obs['annotation_final_mod']=adata_annot.obs['annotation_final'].astype('str').copy()
adata_annot.obs.replace({'annotation_final_mod':annotation_edit_dict}, 
                      inplace=True)
adata_annot.obs['annotation_final_mod'] = adata_annot.obs['annotation_final_mod'].astype('category')

# set palette and categorie order
AnnotationPalette={ 
    'Atrium':'#eac075',
        'Ventricle - Compact':'#f1acfa',
     'Ventricle - Trabeculated':'#f09951',
        'SAnode':'#8A2BE2',
        'AVnode':'#062b4a',
    'VCS - Proximal':'#FFFF00',
        'VCS - Distal':'#7FFFD4',
        'Epicardium':'#01ccc8',
        'Endocardium - Atrial':'#DC143C',
        'Endocardium - Ventricular':'#f53527',
        'Endocardial cusion':'#ff7e97',
        'Great vessel':'#ff9b6e',
    'Ductus arteriosus':'#8a6800',
        'Coronary vessel':'#206601',
        'Adventitia - Coronary':'#CAFF70',
     'Adventitia - GV':'#005aa0',
        'AV ring':'#e478ff',
    'Valves':'#3300FF',
    'LN':'#9088ff',
}
adata_annot.obs['annotation_final_mod'] = adata_annot.obs['annotation_final_mod'].astype('category')
adata_annot.obs['annotation_final_mod'] = adata_annot.obs['annotation_final_mod'].cat.reorder_categories(AnnotationPalette.keys())
```

Example 27 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python3.9/site-packages/anndata/_core/anndata.py:1832: UserWarning: Variable names are not unique. To make them unique, call `.var_names_make_unique`.
  utils.warn_names_duplicates("var")
```

Example 28 (python):
```python
# add
adata_ffpe.obs['annotation_final_mod'] = adata_annot.obs['annotation_final_mod'].reindex(adata_ffpe.obs_names)
adata_oct.obs['annotation_final_mod'] = adata_annot.obs['annotation_final_mod'].reindex(adata_oct.obs_names)
adata_oct_20pcw.obs['annotation_final_mod'] = adata_annot.obs['annotation_final_mod'].reindex(adata_oct_20pcw.obs_names)
```

Example 29 (unknown):
```unknown
slide_ids_sel = {}
for key,value in slide_ids.items():
    if key in ['5PCW','7PCW','16PCW']:
        slide_ids_sel[key] = value
slide_ids_sel
```

Example 30 (unknown):
```unknown
{'5PCW': 'HCAHeartST11605168',
 '7PCW': 'HCAHeartST10317185',
 '16PCW': 'BHF_F_Hea13295830'}
```

Example 31 (unknown):
```unknown
# cropping parameters
cropp_cord_list = {
    '5PCW': [8000,14500,7800,14000],
    '7PCW': [3000,14000,7000,15000],
    '16PCW': [0,17500,500,16500],
}
```

Example 32 (python):
```python
for week,slide_id in slide_ids_sel.items():
    if week=='16PCW':
        slide = select_multislide(adata_ffpe, [slide_id])
        slide_copy = slide.copy()
        slide.uns['spatial'] = {}
        slide.uns['spatial'][slide_id] = [slide_copy.uns['spatial'][x] for x in slide_copy.uns['spatial'].keys() if slide_id in x][0]
    else:
        slide = select_multislide(adata_oct, [slide_id])
    
    # get pixel size
    pix_size = get_pixel_size_visium(slide, slide_id, spot_diameter_real = 55)
        
    crop_coord = cropp_cord_list[week]

    # plot in spatial coordinates
    with mpl.rc_context({#'axes.facecolor':  'black',
                         #'figure.figsize': [4.5, 5]
    }):
        
        fig, ax = plt.subplots()
        sc.pl.spatial(slide, cmap='RdPu',
                      color=['JC_edit'], 
                      ncols=5, size=0, 
                      img_key='hires', alpha_img=1,wspace=0.3,
                      vmin=0, vmax='p99.5',# palette=palette
                      crop_coord = crop_coord,
                      legend_loc=None,
                      title=week,
                      show=False,ax=ax
                     )
        scalebar = ScaleBar(pix_size, "um", length_fraction=0.25, frameon=False, location='upper right')
        ax.add_artist(scalebar)
        plt.savefig(f"{figdir}/visium_temporal_HE_{slide_id}.pdf", bbox_inches="tight",dpi=300)
        plt.show()
```

Example 33 (unknown):
```unknown
/tmp/ipykernel_108365/2805736760.py:18: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  slides.uns['spatial'] = uns_dict
```

Example 34 (unknown):
```unknown
/tmp/ipykernel_108365/2805736760.py:18: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  slides.uns['spatial'] = uns_dict
```

Example 35 (unknown):
```unknown
/tmp/ipykernel_108365/2805736760.py:18: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  slides.uns['spatial'] = uns_dict
```

Example 36 (python):
```python
for week,slide_id in slide_ids_sel.items():
    if week=='16PCW':
        slide = select_multislide(adata_ffpe, [slide_id])
        slide_copy = slide.copy()
        slide.uns['spatial'] = {}
        slide.uns['spatial'][slide_id] = [slide_copy.uns['spatial'][x] for x in slide_copy.uns['spatial'].keys() if slide_id in x][0]
    else:
        slide = select_multislide(adata_oct, [slide_id])
        
    crop_coord = cropp_cord_list[week]

    # plot in spatial coordinates
    with mpl.rc_context({#'axes.facecolor':  'black',
                         #'figure.figsize': [4.5, 5]
    }):
        
        sc.pl.spatial(slide, cmap='RdPu',
                      color=['annotation_final_mod'], 
                      ncols=5, size=1.2, 
                      img_key='hires', alpha_img=0.3,wspace=0.3,
                      vmin=0, vmax='p99.5', palette=AnnotationPalette,
                      crop_coord = crop_coord,
                      legend_loc=None,
                      title=week,
                      show=False
                     )
        plt.savefig(f"{figdir}/visium_temporal_annotations_{slide_id}.pdf", bbox_inches="tight",dpi=300)
```

Example 37 (unknown):
```unknown
/tmp/ipykernel_108365/2805736760.py:18: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  slides.uns['spatial'] = uns_dict
/tmp/ipykernel_108365/2805736760.py:18: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  slides.uns['spatial'] = uns_dict
/tmp/ipykernel_108365/2805736760.py:18: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  slides.uns['spatial'] = uns_dict
```

---

## 

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/RNA/scnRNA_Integration-Annotation/Epicardium/RenamingPericardialCells.html

**Examples:**

Example 1 (unknown):
```unknown
#Added the keys for age and sex
```

Example 2 (python):
```python
# Import packages, initial settings

import numpy as np
import matplotlib.pyplot as plt
import scanpy as sc
import pandas as pd
import anndata
import scrublet as scr
import seaborn as sns
import magic
import pybiomart
import six
import sys
sys.modules['sklearn.externals.six'] = six
import scanorama
import harmonypy
import bbknn
import gseapy
from gseapy.plot import barplot, dotplot
import scvelo as scv
import scvi

plt.rcParams['pdf.fonttype'] = 42
plt.rcParams['ps.fonttype'] = 42

sc.settings.verbosity = 3  # verbosity: errors (0), warnings (1), info (2), hints (3)

from matplotlib.colors import LinearSegmentedColormap
cmap = LinearSegmentedColormap.from_list(name='gene_cmap', colors=['lightgrey', 'thistle', 'red', 'darkred']) 

sc.settings.set_figure_params(dpi=80, color_map='viridis', vector_friendly=False,  dpi_save=300, facecolor='white')
sc.logging.print_version_and_date()
#sc.logging.print_versions()
```

Example 3 (unknown):
```unknown
Global seed set to 0
```

Example 4 (unknown):
```unknown
Running Scanpy 1.8.1, on 2023-06-23 12:09.
```

Example 5 (python):
```python
adata=sc.read('./Objects/Feb28ObjectProcessed.h5ad')
print(adata.shape)
```

Example 6 (unknown):
```unknown
(297473, 3392)
```

Example 7 (python):
```python
adata=sc.read('./Objects/Feb28ObjectRaw.h5ad')
print(adata.shape)
```

Example 8 (unknown):
```unknown
(297473, 36601)
```

Example 9 (python):
```python
#Combine all barcodes
CoronaryPericytes=[x.strip() for x in open('./Index/Mesenchymal/CoronaryPericytes.txt')]
CoronarySmoothMuscleCells=[x.strip() for x in open('./Index/Mesenchymal/CoronarySmoothMuscleCells.txt')]
CoronaryVesselAdventitialFibroblasts=[x.strip() for x in open('./Index/Mesenchymal/CoronaryVesselAdventitialFibroblasts.txt')]
DuctusArteriosusSmoothMuscleCells=[x.strip() for x in open('./Index/Mesenchymal/DuctusArteriosusSmoothMuscleCells.txt')]
GreatVesselAdventitialFibroblasts=[x.strip() for x in open('./Index/Mesenchymal/GreatVesselAdventitialFibroblasts.txt')]
GreatVesselSmoothMuscleCells=[x.strip() for x in open('./Index/Mesenchymal/GreatVesselSmoothMuscleCells.txt')]
LymphNodeFibroblasticReticularCells=[x.strip() for x in open('./Index/Mesenchymal/LymphNodeFibroblasticReticularCells.txt')]
MyocardialInterstitialFibroblasts1=[x.strip() for x in open('./Index/Mesenchymal/MyocardialInterstitialFibroblasts1.txt')]
MyocardialInterstitialFibroblasts2=[x.strip() for x in open('./Index/Mesenchymal/MyocardialInterstitialFibroblasts2.txt')]
Myofibroblasts=[x.strip() for x in open('./Index/Mesenchymal/Myofibroblasts.txt')]
PericardialCells1=[x.strip() for x in open('./Index/Mesenchymal/PericardialCells1.txt')]
PericardialCells2=[x.strip() for x in open('./Index/Mesenchymal/PericardialCells2.txt')]
PericardialCells3=[x.strip() for x in open('./Index/Mesenchymal/PericardialCells3.txt')]
ValveInterstitialCells=[x.strip() for x in open('./Index/Mesenchymal/ValveInterstitialCells.txt')]
MesenchymalIndex=CoronaryPericytes+CoronarySmoothMuscleCells+CoronaryVesselAdventitialFibroblasts+DuctusArteriosusSmoothMuscleCells+GreatVesselAdventitialFibroblasts+LymphNodeFibroblasticReticularCells+GreatVesselSmoothMuscleCells+MyocardialInterstitialFibroblasts1+MyocardialInterstitialFibroblasts2+Myofibroblasts+PericardialCells1+PericardialCells2+PericardialCells3+ValveInterstitialCells

AtrialCardiomyocytesCycling=[x.strip() for x in open('./Index/Cardiomyocytes/AtrialCardiomyocytesCycling.txt')]
AtrialCardiomyocytesLeft=[x.strip() for x in open('./Index/Cardiomyocytes/AtrialCardiomyocytesLeft.txt')]
AtrialCardiomyocytesRight=[x.strip() for x in open('./Index/Cardiomyocytes/AtrialCardiomyocytesRight.txt')]
AtrioventricularNodeCardiomyocytes=[x.strip() for x in open('./Index/Cardiomyocytes/AtrioventricularNodeCardiomyocytes.txt')]
SinoatrialNodeCardiomyocytes=[x.strip() for x in open('./Index/Cardiomyocytes/SinoatrialNodeCardiomyocytes.txt')]
VentricularCardiomyocytesCycling=[x.strip() for x in open('./Index/Cardiomyocytes/VentricularCardiomyocytesCycling.txt')]
VentricularCardiomyocytesLeftCompact=[x.strip() for x in open('./Index/Cardiomyocytes/VentricularCardiomyocytesLeftCompact.txt')]
VentricularCardiomyocytesLeftTrabeculated=[x.strip() for x in open('./Index/Cardiomyocytes/VentricularCardiomyocytesLeftTrabeculated.txt')]
VentricularCardiomyocytesRightCompact=[x.strip() for x in open('./Index/Cardiomyocytes/VentricularCardiomyocytesRightCompact.txt')]
VentricularCardiomyocytesRightTrabeculated=[x.strip() for x in open('./Index/Cardiomyocytes/VentricularCardiomyocytesRightTrabeculated.txt')]
VentricularConductionSystemDistal=[x.strip() for x in open('./Index/Cardiomyocytes/VentricularConductionSystemDistal.txt')]
VentricularConductionSystemProximal=[x.strip() for x in open('./Index/Cardiomyocytes/VentricularConductionSystemProximal.txt')]
CardiomyocyteIndex=AtrialCardiomyocytesCycling+AtrialCardiomyocytesLeft+AtrialCardiomyocytesRight+AtrioventricularNodeCardiomyocytes+SinoatrialNodeCardiomyocytes+VentricularCardiomyocytesLeftCompact+VentricularCardiomyocytesCycling+VentricularCardiomyocytesRightTrabeculated+VentricularCardiomyocytesRightCompact+VentricularCardiomyocytesLeftTrabeculated+VentricularConductionSystemDistal+VentricularConductionSystemProximal

CoronaryArterialEndothelialCells=[x.strip() for x in open('./Index/Endothelium/CoronaryArterialEndothelialCells.txt')]
CoronaryCapillaryEndothelialCells=[x.strip() for x in open('./Index/Endothelium/CoronaryCapillaryEndothelialCells.txt')]
CoronaryVenousEndothelialCells=[x.strip() for x in open('./Index/Endothelium/CoronaryVenousEndothelialCells.txt')]
EndocardialCells=[x.strip() for x in open('./Index/Endothelium/EndocardialCells.txt')]
EndocardialCushionCells=[x.strip() for x in open('./Index/Endothelium/EndocardialCushionCells.txt')]
GreatVesselArterialEndothelialCells=[x.strip() for x in open('./Index/Endothelium/GreatVesselArterialEndothelialCells.txt')]
GreatVesselVenousEndothelialCells=[x.strip() for x in open('./Index/Endothelium/GreatVesselVenousEndothelialCells.txt')]
LymphaticEndothelialCells=[x.strip() for x in open('./Index/Endothelium/LymphaticEndothelialCells.txt')]
ValveEndothelialCells=[x.strip() for x in open('./Index/Endothelium/ValveEndothelialCells.txt')]
EndotheliumIndex=CoronaryArterialEndothelialCells+CoronaryCapillaryEndothelialCells+CoronaryVenousEndothelialCells+EndocardialCells+EndocardialCushionCells+GreatVesselArterialEndothelialCells+GreatVesselVenousEndothelialCells+LymphaticEndothelialCells+ValveEndothelialCells

EpicardiumDerivedCells=[x.strip() for x in open('./Index/Epicardium/EpicardiumDerivedCells.txt')]
MesothelialEpicardialCells=[x.strip() for x in open('./Index/Epicardium/MesothelialEpicardialCells.txt')]
EpicardiumIndex=EpicardiumDerivedCells+MesothelialEpicardialCells

SchwannCells=[x.strip() for x in open('./Index/Neural/SchwannCells.txt')]
SchwannCellPrecursors=[x.strip() for x in open('./Index/Neural/SchwannCellPrecursors.txt')]
ChromaffinCells=[x.strip() for x in open('./Index/Neural/ChromaffinCells.txt')]
NeuronPrecursors=[x.strip() for x in open('./Index/Neural/NeuronPrecursors.txt')]
ParasympatheticNeurons=[x.strip() for x in open('./Index/Neural/ParaympatheticNeurons.txt')]
SympatheticNeurons=[x.strip() for x in open('./Index/Neural/SympatheticNeurons.txt')]
NeuralIndex=ChromaffinCells+NeuronPrecursors+ParasympatheticNeurons+SchwannCellPrecursors+SchwannCells+SympatheticNeurons

BCells=[x.strip() for x in open('./Index/Immune/BCells.txt')]
BCellsMS4A1pos=[x.strip() for x in open('./Index/Immune/BCellsMS4A1pos.txt')]
TCellsCD4pos=[x.strip() for x in open('./Index/Immune/TCellsCD4pos.txt')]
TregsCD4pos=[x.strip() for x in open('./Index/Immune/TregsCD4pos.txt')]
TCellsCD8pos=[x.strip() for x in open('./Index/Immune/TCellsCD8pos.txt')]
DendriticCellsType1=[x.strip() for x in open('./Index/Immune/DendriticCellsType1.txt')]
DendriticCellsMature=[x.strip() for x in open('./Index/Immune/DendriticCellsMature.txt')]
InnateLymphoidCells=[x.strip() for x in open('./Index/Immune/InnateLymphoidCells.txt')]
MacrophagesATF3pos=[x.strip() for x in open('./Index/Immune/MacrophagesATF3pos.txt')]
MacrophagesCX3CR1pos=[x.strip() for x in open('./Index/Immune/MacrophagesCX3CR1pos.txt')]
MacrophagesLYVE1pos=[x.strip() for x in open('./Index/Immune/MacrophagesLYVE1pos.txt')]
MacrophagesTIMD4pos=[x.strip() for x in open('./Index/Immune/MacrophagesTIMD4pos.txt')]
MastCells=[x.strip() for x in open('./Index/Immune/MastCells.txt')]
Megakaryocytes=[x.strip() for x in open('./Index/Immune/Megakaryocytes.txt')]
MonocyteDerivedCells=[x.strip() for x in open('./Index/Immune/MonocyteDerivedCells.txt')]
Monocytes=[x.strip() for x in open('./Index/Immune/Monocytes.txt')]
MonocytesMPOpos=[x.strip() for x in open('./Index/Immune/MonocytesMPOpos.txt')]
NaturalKillerCells=[x.strip() for x in open('./Index/Immune/NaturalKillerCells.txt')]
PlasmacytoidDendriticCells=[x.strip() for x in open('./Index/Immune/PlasmacytoidDendriticCells.txt')]
ProBCells=[x.strip() for x in open('./Index/Immune/ProBCells.txt')]

ImmuneIndex=BCells+BCellsMS4A1pos+TCellsCD4pos+TregsCD4pos+TCellsCD8pos+DendriticCellsType1+DendriticCellsMature+InnateLymphoidCells+MacrophagesATF3pos+MacrophagesCX3CR1pos+MacrophagesLYVE1pos+MacrophagesTIMD4pos+MastCells+Megakaryocytes+MonocyteDerivedCells+Monocytes+MonocytesMPOpos+NaturalKillerCells+PlasmacytoidDendriticCells+ProBCells

AllIndex=MesenchymalIndex+CardiomyocyteIndex+EndotheliumIndex+ImmuneIndex+EpicardiumIndex+NeuralIndex
print(len(AllIndex))
```

Example 10 (python):
```python
adata.obs['fine_grain']= 'na'

adata.obs['fine_grain'][[x for x in adata.obs_names if x in CoronaryPericytes]]='CoronaryPericytes'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in CoronarySmoothMuscleCells]]='CoronarySmoothMuscleCells'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in CoronaryVesselAdventitialFibroblasts]]='CoronaryVesselAdventitialFibroblasts'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in DuctusArteriosusSmoothMuscleCells]]='DuctusArteriosusSmoothMuscleCells'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in GreatVesselAdventitialFibroblasts]]='GreatVesselAdventitialFibroblasts'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in LymphNodeFibroblasticReticularCells]]='LymphNodeFibroblasticReticularCells'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in GreatVesselSmoothMuscleCells]]='GreatVesselSmoothMuscleCells'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in MyocardialInterstitialFibroblasts1]]='MyocardialInterstitialFibroblasts'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in MyocardialInterstitialFibroblasts2]]='SubEpicardialFibroblasts'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in Myofibroblasts]]='Myofibroblasts'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in PericardialCells1]]='PericardialCellsIntermediate'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in PericardialCells2]]='PericardialCellsFibrous'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in PericardialCells3]]='PericardialCellsParietal'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in ValveInterstitialCells]]='ValveInterstitialCells'

adata.obs['fine_grain'][[x for x in adata.obs_names if x in AtrialCardiomyocytesCycling]]='AtrialCardiomyocytesCycling'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in AtrialCardiomyocytesLeft]]='AtrialCardiomyocytesLeft'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in AtrialCardiomyocytesRight]]='AtrialCardiomyocytesRight'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in AtrioventricularNodeCardiomyocytes]]='AtrioventricularNodePacemakerCells'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in SinoatrialNodeCardiomyocytes]]='SinoatrialNodePacemakerCells'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in VentricularCardiomyocytesLeftCompact]]='VentricularCardiomyocytesLeftCompact'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in VentricularCardiomyocytesCycling]]='VentricularCardiomyocytesCycling'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in VentricularCardiomyocytesRightTrabeculated]]='VentricularCardiomyocytesRightTrabeculated'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in VentricularCardiomyocytesRightCompact]]='VentricularCardiomyocytesRightCompact'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in VentricularCardiomyocytesLeftTrabeculated]]='VentricularCardiomyocytesLeftTrabeculated'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in VentricularConductionSystemDistal]]='VentricularConductionSystemDistal'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in VentricularConductionSystemProximal]]='VentricularConductionSystemProximal'

adata.obs['fine_grain'][[x for x in adata.obs_names if x in CoronaryArterialEndothelialCells]]='CoronaryArterialEndothelialCells'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in CoronaryCapillaryEndothelialCells]]='CoronaryCapillaryEndothelialCells'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in CoronaryVenousEndothelialCells]]='CoronaryVenousEndothelialCells'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in EndocardialCells]]='EndocardialCells'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in EndocardialCushionCells]]='EndocardialCushionCells'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in GreatVesselArterialEndothelialCells]]='GreatVesselArterialEndothelialCells'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in GreatVesselVenousEndothelialCells]]='GreatVesselVenousEndothelialCells'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in LymphaticEndothelialCells]]='LymphaticEndothelialCells'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in ValveEndothelialCells]]='ValveEndothelialCells'

adata.obs['fine_grain'][[x for x in adata.obs_names if x in EpicardiumDerivedCells]]='EpicardiumDerivedCells'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in MesothelialEpicardialCells]]='MesothelialEpicardialCells'

adata.obs['fine_grain'][[x for x in adata.obs_names if x in ChromaffinCells]]='ChromaffinCells'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in NeuronPrecursors]]='NeuronPrecursors'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in ParasympatheticNeurons]]='ParasympatheticNeurons'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in SchwannCellPrecursors]]='SchwannCellPrecursors'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in SchwannCells]]='SchwannCells'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in SympatheticNeurons]]='SympatheticNeurons'

adata.obs['fine_grain'][[x for x in adata.obs_names if x in BCells]]='BCells'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in BCellsMS4A1pos]]='BCellsMS4A1pos'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in DendriticCellsType1]]='DendriticCellsType1'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in DendriticCellsMature]]='DendriticCellsMature'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in InnateLymphoidCells]]='InnateLymphoidCells'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in MacrophagesATF3pos]]='MacrophagesATF3pos'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in MacrophagesCX3CR1pos]]='MacrophagesCX3CR1pos'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in MacrophagesLYVE1pos]]='MacrophagesLYVE1pos'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in MacrophagesTIMD4pos]]='MacrophagesTIMD4pos'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in MastCells]]='MastCells'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in Megakaryocytes]]='Megakaryocytes'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in MonocyteDerivedCells]]='MonocyteDerivedCells'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in Monocytes]]='Monocytes'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in MonocytesMPOpos]]='MonocytesMPOpos'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in NaturalKillerCells]]='NaturalKillerCells'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in PlasmacytoidDendriticCells]]='PlasmacytoidDendriticCells'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in ProBCells]]='ProBCells'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in TCellsCD4pos]]='TCellsCD4pos'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in TregsCD4pos]]='TregsCD4pos'
adata.obs['fine_grain'][[x for x in adata.obs_names if x in TCellsCD8pos]]='TCellsCD8pos'

sc.pl.umap(adata, color=['fine_grain'])
```

Example 11 (unknown):
```unknown
... storing 'fine_grain' as categorical
```

Example 12 (python):
```python
sc.pl.umap(adata, color=['mid_grain'])
adata
```

Example 13 (unknown):
```unknown
AnnData object with n_obs × n_vars = 297473 × 3392
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score'
    var: 'gene_name_scRNA-0', 'gene_id', 'highly_variable', 'means', 'dispersions', 'dispersions_norm', 'highly_variable_nbatches', 'highly_variable_intersection'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'age_colors', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'cycling_colors', 'donor_colors', 'fine_grain_colors', 'heart_or_greatvessels_colors', 'hvg', 'kit_10x_colors', 'mid_grain_colors', 'modality_colors', 'neighbors', 'phase_colors', 'region_colors', 'sex_colors', 'trimester_colors', 'umap', 'week_colors'
    obsm: 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    layers: 'counts'
    obsp: 'connectivities', 'distances'
```

Example 14 (python):
```python
adata.write('./Objects/Feb28ObjectProcessed.h5ad')
#adata
```

Example 15 (python):
```python
adata.write('./Objects/Feb28ObjectRaw.h5ad')
#adata
```

Example 16 (unknown):
```unknown
AnnData object with n_obs × n_vars = 297473 × 3392
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score'
    var: 'gene_name_scRNA-0', 'gene_id', 'highly_variable', 'means', 'dispersions', 'dispersions_norm', 'highly_variable_nbatches', 'highly_variable_intersection'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'age_colors', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'cycling_colors', 'donor_colors', 'fine_grain_colors', 'heart_or_greatvessels_colors', 'hvg', 'kit_10x_colors', 'mid_grain_colors', 'modality_colors', 'neighbors', 'phase_colors', 'region_colors', 'sex_colors', 'trimester_colors', 'umap', 'week_colors'
    obsm: 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    layers: 'counts'
    obsp: 'connectivities', 'distances'
```

Example 17 (python):
```python
sc.pl.umap(adata, color=['fine_grain'])
```

Example 18 (unknown):
```unknown
AnnData object with n_obs × n_vars = 297473 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score'
    var: 'gene_name_scRNA-0', 'gene_id'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'age_colors', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'cycling_colors', 'donor_colors', 'fine_grain_colors', 'heart_or_greatvessels_colors', 'hvg', 'kit_10x_colors', 'mid_grain_colors', 'modality_colors', 'neighbors', 'phase_colors', 'region_colors', 'sex_colors', 'trimester_colors', 'umap', 'week_colors'
    obsm: 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

---

## Import libraries¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig4_Cardiomyocytes/VentricularCardiomyocytes/CellOracle/CellOracle_no3_compact-common-up_summary.html

**Contents:**
- Import libraries¶
- Parameters¶
- Read in celloracle in-silico perturbation output¶
- Plot inner product scores¶
- Correlation of centrality and total inner product (effect of the perturbation)¶
- Read in CHD genes¶
- Project PS score on TF-TF network¶
- Plot representative TFs¶

ref: https://morris-lab.github.io/CellOracle.documentation/notebooks/04_Network_analysis/Network_analysis_with_Paul_etal_2015_data.html

**Examples:**

Example 1 (python):
```python
import os
import sys

import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import scanpy as sc
import seaborn as sns

import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42
```

Example 2 (python):
```python
import celloracle as co
co.__version__
```

Example 3 (python):
```python
/nfs/team205/kk18/miniconda3/envs/celloracle_env/lib/python3.8/site-packages/loompy/bus_file.py:68: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  def twobit_to_dna(twobit: int, size: int) -> str:
/nfs/team205/kk18/miniconda3/envs/celloracle_env/lib/python3.8/site-packages/loompy/bus_file.py:85: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  def dna_to_twobit(dna: str) -> int:
/nfs/team205/kk18/miniconda3/envs/celloracle_env/lib/python3.8/site-packages/loompy/bus_file.py:102: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  def twobit_1hamming(twobit: int, size: int) -> List[int]:
```

Example 4 (python):
```python
import session_info
session_info.show()
```

Example 5 (python):
```python
-----
celloracle          0.14.0
matplotlib          3.6.3
numpy               1.24.3
pandas              1.5.3
scanpy              1.9.5
seaborn             0.13.0
session_info        1.0.0
-----
```

Example 6 (python):
```python
Bio                         1.81
PIL                         10.0.1
anndata                     0.9.2
anyio                       NA
appdirs                     1.4.4
arrow                       1.3.0
asttokens                   NA
attr                        23.1.0
attrs                       23.1.0
babel                       2.13.0
backcall                    0.2.0
beta_ufunc                  NA
binom_ufunc                 NA
biothings_client            0.3.0
bottleneck                  1.3.5
brotli                      NA
certifi                     2023.07.22
cffi                        1.15.1
charset_normalizer          2.0.4
colorama                    0.4.6
comm                        0.1.4
cycler                      0.10.0
cython_runtime              NA
dateutil                    2.8.2
debugpy                     1.8.0
decorator                   5.1.1
defusedxml                  0.7.1
diskcache                   5.6.3
exceptiongroup              1.1.3
executing                   2.0.0
fastjsonschema              NA
filelock                    3.12.4
fqdn                        NA
genomepy                    0.16.1
gimmemotifs                 0.17.0
goatools                    1.3.9
h5py                        3.10.0
hypergeom_ufunc             NA
idna                        3.4
igraph                      0.10.8
importlib_metadata          NA
importlib_resources         NA
invgauss_ufunc              NA
ipykernel                   6.25.2
ipywidgets                  8.1.1
isoduration                 NA
iteround                    NA
jedi                        0.19.1
jinja2                      3.1.2
joblib                      1.2.0
json5                       NA
jsonpointer                 2.4
jsonschema                  4.19.1
jsonschema_specifications   NA
jupyter_events              0.8.0
jupyter_server              2.8.0
jupyterlab_server           2.25.0
kiwisolver                  1.4.4
llvmlite                    0.41.0
logomaker                   NA
loguru                      0.7.2
loompy                      3.0.7
louvain                     0.8.0
markupsafe                  2.1.3
mkl                         2.4.0
mpl_toolkits                NA
mygene                      3.2.2
mysql                       NA
natsort                     8.4.0
nbformat                    5.9.2
nbinom_ufunc                NA
ncf_ufunc                   NA
nct_ufunc                   NA
ncx2_ufunc                  NA
networkx                    3.1
norns                       NA
numba                       0.58.0
numexpr                     2.8.4
numpy_groupies              0.9.22
overrides                   NA
packaging                   23.1
parso                       0.8.3
patsy                       0.5.3
pexpect                     4.8.0
pickleshare                 0.7.5
pkg_resources               NA
platformdirs                3.10.0
prometheus_client           NA
prompt_toolkit              3.0.39
psutil                      5.9.6
ptyprocess                  0.7.0
pure_eval                   0.2.2
pyarrow                     13.0.0
pybedtools                  0.9.1
pycparser                   2.21
pydev_ipython               NA
pydevconsole                NA
pydevd                      2.9.5
pydevd_file_utils           NA
pydevd_plugins              NA
pydevd_tracing              NA
pyfaidx                     0.7.2.2
pygments                    2.16.1
pyparsing                   3.0.9
pysam                       0.20.0
pythonjsonlogger            NA
pytz                        2023.3.post1
referencing                 NA
requests                    2.31.0
rfc3339_validator           0.1.4
rfc3986_validator           0.1.1
rpds                        NA
scipy                       1.10.1
send2trash                  NA
setuptools                  68.2.2
six                         1.16.0
skewnorm_ufunc              NA
sklearn                     1.3.0
sniffio                     1.3.0
socks                       1.7.1
stack_data                  0.6.3
statsmodels                 0.14.0
texttable                   1.7.0
threadpoolctl               2.2.0
tornado                     6.3.3
tqdm                        4.66.1
traitlets                   5.11.2
typing_extensions           NA
uri_template                NA
urllib3                     1.26.16
velocyto                    0.17.17
wcwidth                     0.2.8
webcolors                   1.13
websocket                   1.6.4
xdg                         NA
xgboost                     2.0.0
xxhash                      NA
yaml                        6.0.1
zipp                        NA
zmq                         25.1.1
```

Example 7 (unknown):
```unknown
-----
IPython             8.12.3
jupyter_client      8.4.0
jupyter_core        5.4.0
jupyterlab          4.0.7
notebook            7.0.6
-----
Python 3.8.18 (default, Sep 11 2023, 13:40:15) [GCC 11.2.0]
Linux-5.4.0-169-generic-x86_64-with-glibc2.17
-----
Session information updated at 2024-03-25 12:47
```

Example 8 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)

spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/regulon_plot.py")
regulon_plot = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = regulon_plot
spec.loader.exec_module(regulon_plot)
```

Example 9 (python):
```python
sc.settings.set_figure_params(dpi=120)
```

Example 10 (python):
```python
%matplotlib inline
```

Example 11 (unknown):
```unknown
# Parameters
working_dir = "/nfs/team205/heart/Vitro/TF_selection/foetal_vCM_left_compact-vs-trabeculated_downstream/CO_in-silico-perturbation_papermill"
path_to_oracle_object = "/nfs/team205/heart/anndata_objects/Foetal/vCM/post_scFate/foetal_vCM_onlyLeft.celloracle.oracle"
path_to_oracle_links = "/nfs/team205/heart/anndata_objects/Foetal/vCM/post_scFate/vCM_onlyLeft_links.celloracle.links"
n_grid = 40
min_mass = 0.011
path_to_TFs = "/nfs/team205/heart/Vitro/TF_selection/foetal_vCM_left_compact-vs-trabeculated_downstream/TF_selection_df_compact-common-up_2Feb2024.csv"
path_to_res_dict = "/nfs/team205/heart/anndata_objects/Foetal/vCM/post_scFate/vCM_onlyLeft_compact-common_silico_perturbation_2Feb2024.pkl"
```

Example 12 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk'
table_dir = '/nfs/team205/heart/fetal_ms_figures/tables'
```

Example 13 (python):
```python
dev_dict = utils.read_pkl(path_to_res_dict)
print(len(dev_dict.keys()))
dev_dict.keys()
```

Example 14 (unknown):
```unknown
dict_keys(['ARID3A', 'ATF1', 'BACH2', 'BHLHE40', 'CUX1', 'ESRRA', 'ESRRB', 'FOXN3', 'FOXO1', 'GATA4', 'GATA6', 'GTF2IRD1', 'HAND2', 'HEY2', 'HIF1A', 'HMBOX1', 'KLF12', 'KLF6', 'LCORL', 'MAX', 'MAZ', 'MBNL2', 'MEF2A', 'MEF2C', 'MEF2D', 'MEIS2', 'MXI1', 'NCOA1', 'NFATC2', 'NFE2L1', 'NFIA', 'NFIB', 'NFIC', 'NPAS2', 'NR3C1', 'NR4A1', 'PBX1', 'PLAG1', 'PPARA', 'PRDM16', 'RORA', 'SMAD3', 'SREBF2', 'TBX20', 'TFCP2', 'TGIF2', 'THRA', 'USF2', 'YY1', 'ZBTB16', 'ZBTB20', 'ZNF148', 'ZNF280D', 'ZNF654', 'ZSCAN18', 'AHCTF1', 'CREB1', 'FOXK1', 'PHF21A', 'SOX5', 'UBP1', 'ZNF141', 'ELF2', 'POU2F1', 'ZNF586', 'IRX3'])
```

Example 15 (python):
```python
%matplotlib inline

vm = 0.01
scale_simulation = 0.1
scale_dev = 40

tf = 'PRDM16'
dev = dev_dict[tf]
dev.visualize_development_module_layout_0(s=5,
                                          scale_for_simulation=scale_simulation,
                                          s_grid=50,
                                          scale_for_pseudotime=scale_dev,
                                          vm=vm)
```

Example 16 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 17 (python):
```python
from scipy.interpolate import UnivariateSpline

nrows = 10
ncols = 7
fig,axs=plt.subplots(nrows=nrows,ncols=ncols,figsize=(25,25)) 
for i,tf in enumerate(dev_dict.keys()):
    dev = dev_dict[tf]
    # get inner product results
    inner_product_df = dev.inner_product_df.copy()
    # normalise pseudotime (min-max)
    arr = np.array(inner_product_df['pseudotime'])
    inner_product_df['pseudotime_norm'] = (arr-np.min(arr))/(np.max(arr)-np.min(arr))
    # sort
    inner_product_df.sort_values('pseudotime_norm',ascending=True,inplace=True)
    # smoothing inner product score
    spl_res = UnivariateSpline(inner_product_df['pseudotime_norm'], inner_product_df['score'], k=2)
    inner_product_df['score_smoothed'] = spl_res(inner_product_df['pseudotime_norm'])
    # plot
    axs[i//ncols,i%ncols].scatter(x=inner_product_df['pseudotime_norm'],y=inner_product_df['score'])
    axs[i//ncols,i%ncols].plot(inner_product_df['pseudotime_norm'],inner_product_df['score_smoothed'],color='red')
    axs[i//ncols,i%ncols].set_title(tf)
    axs[i//ncols,i%ncols].set_xlabel('pseudotime')
    axs[i//ncols,i%ncols].set_ylabel('inner product score')
# fig.suptitle('')
fig.tight_layout()
```

Example 18 (python):
```python
from scipy.stats import pearsonr
```

Example 19 (python):
```python
# raed in TF data
TF_df = pd.read_csv(path_to_TFs)
TF_df = TF_df.set_index('TF')

# get total inner product
total_inner_product = {}
for tf in dev_dict.keys():
    dev = dev_dict[tf]
    # get inner product results
    inner_product_df = dev.inner_product_df.copy()
    # take sum of the score
    TF_df.loc[tf,'total_PS_score'] = inner_product_df['score'].sum()

# save the dataframe
TF_df.to_csv(os.path.join(table_dir,'csvs/TF_metrics_vCM_onlyLeft.csv'))

TF_df
```

Example 20 (unknown):
```unknown
os.path.join(table_dir,'csvs/TF_metrics_vCM_onlyLeft.csv')
```

Example 21 (unknown):
```unknown
'/nfs/team205/heart/fetal_ms_figures/tables/csvs/TF_metrics_vCM_onlyLeft.csv'
```

Example 22 (python):
```python
r,p = pearsonr(TF_df['centaliry_eigenvector'],TF_df['total_PS_score'])
sns.regplot(data=TF_df, x='centaliry_eigenvector', y='total_PS_score')
plt.xlabel('centrality')
plt.ylabel('total PS score')
plt.title(f"Pearson's r: {round(r,3)}, p-value: {round(p,3)}")
plt.savefig(f"{figdir}/TF-network_compact-common-up_correlation-centrality-PS.pdf",bbox_inches="tight",dpi=300)
plt.show()
```

Example 23 (unknown):
```unknown
INFO:fontTools.subset:maxp pruned
INFO:fontTools.subset:cmap pruned
INFO:fontTools.subset:kern dropped
INFO:fontTools.subset:post pruned
INFO:fontTools.subset:FFTM dropped
INFO:fontTools.subset:GPOS pruned
INFO:fontTools.subset:GSUB pruned
INFO:fontTools.subset:name pruned
INFO:fontTools.subset:glyf pruned
INFO:fontTools.subset:Added gid0 to subset
INFO:fontTools.subset:Added first four glyphs to subset
INFO:fontTools.subset:Closing glyph list over 'GSUB': 32 glyphs before
INFO:fontTools.subset:Glyph names: ['.notdef', '.null', 'P', 'S', 'a', 'c', 'colon', 'comma', 'e', 'five', 'four', 'hyphen', 'i', 'l', 'minus', 'n', 'nonmarkingreturn', 'o', 'one', 'p', 'period', 'quotesingle', 'r', 's', 'space', 't', 'three', 'two', 'u', 'v', 'y', 'zero']
INFO:fontTools.subset:Glyph IDs:   [0, 1, 2, 3, 10, 15, 16, 17, 19, 20, 21, 22, 23, 24, 29, 51, 54, 68, 70, 72, 76, 79, 81, 82, 83, 85, 86, 87, 88, 89, 92, 3228]
INFO:fontTools.subset:Closed glyph list over 'GSUB': 32 glyphs after
INFO:fontTools.subset:Glyph names: ['.notdef', '.null', 'P', 'S', 'a', 'c', 'colon', 'comma', 'e', 'five', 'four', 'hyphen', 'i', 'l', 'minus', 'n', 'nonmarkingreturn', 'o', 'one', 'p', 'period', 'quotesingle', 'r', 's', 'space', 't', 'three', 'two', 'u', 'v', 'y', 'zero']
INFO:fontTools.subset:Glyph IDs:   [0, 1, 2, 3, 10, 15, 16, 17, 19, 20, 21, 22, 23, 24, 29, 51, 54, 68, 70, 72, 76, 79, 81, 82, 83, 85, 86, 87, 88, 89, 92, 3228]
INFO:fontTools.subset:Closing glyph list over 'MATH': 32 glyphs before
INFO:fontTools.subset:Glyph names: ['.notdef', '.null', 'P', 'S', 'a', 'c', 'colon', 'comma', 'e', 'five', 'four', 'hyphen', 'i', 'l', 'minus', 'n', 'nonmarkingreturn', 'o', 'one', 'p', 'period', 'quotesingle', 'r', 's', 'space', 't', 'three', 'two', 'u', 'v', 'y', 'zero']
INFO:fontTools.subset:Glyph IDs:   [0, 1, 2, 3, 10, 15, 16, 17, 19, 20, 21, 22, 23, 24, 29, 51, 54, 68, 70, 72, 76, 79, 81, 82, 83, 85, 86, 87, 88, 89, 92, 3228]
INFO:fontTools.subset:Closed glyph list over 'MATH': 32 glyphs after
INFO:fontTools.subset:Glyph names: ['.notdef', '.null', 'P', 'S', 'a', 'c', 'colon', 'comma', 'e', 'five', 'four', 'hyphen', 'i', 'l', 'minus', 'n', 'nonmarkingreturn', 'o', 'one', 'p', 'period', 'quotesingle', 'r', 's', 'space', 't', 'three', 'two', 'u', 'v', 'y', 'zero']
INFO:fontTools.subset:Glyph IDs:   [0, 1, 2, 3, 10, 15, 16, 17, 19, 20, 21, 22, 23, 24, 29, 51, 54, 68, 70, 72, 76, 79, 81, 82, 83, 85, 86, 87, 88, 89, 92, 3228]
INFO:fontTools.subset:Closing glyph list over 'glyf': 32 glyphs before
INFO:fontTools.subset:Glyph names: ['.notdef', '.null', 'P', 'S', 'a', 'c', 'colon', 'comma', 'e', 'five', 'four', 'hyphen', 'i', 'l', 'minus', 'n', 'nonmarkingreturn', 'o', 'one', 'p', 'period', 'quotesingle', 'r', 's', 'space', 't', 'three', 'two', 'u', 'v', 'y', 'zero']
INFO:fontTools.subset:Glyph IDs:   [0, 1, 2, 3, 10, 15, 16, 17, 19, 20, 21, 22, 23, 24, 29, 51, 54, 68, 70, 72, 76, 79, 81, 82, 83, 85, 86, 87, 88, 89, 92, 3228]
INFO:fontTools.subset:Closed glyph list over 'glyf': 32 glyphs after
INFO:fontTools.subset:Glyph names: ['.notdef', '.null', 'P', 'S', 'a', 'c', 'colon', 'comma', 'e', 'five', 'four', 'hyphen', 'i', 'l', 'minus', 'n', 'nonmarkingreturn', 'o', 'one', 'p', 'period', 'quotesingle', 'r', 's', 'space', 't', 'three', 'two', 'u', 'v', 'y', 'zero']
INFO:fontTools.subset:Glyph IDs:   [0, 1, 2, 3, 10, 15, 16, 17, 19, 20, 21, 22, 23, 24, 29, 51, 54, 68, 70, 72, 76, 79, 81, 82, 83, 85, 86, 87, 88, 89, 92, 3228]
INFO:fontTools.subset:Retaining 32 glyphs
INFO:fontTools.subset:head subsetting not needed
INFO:fontTools.subset:hhea subsetting not needed
INFO:fontTools.subset:maxp subsetting not needed
INFO:fontTools.subset:OS/2 subsetting not needed
INFO:fontTools.subset:hmtx subsetted
INFO:fontTools.subset:cmap subsetted
INFO:fontTools.subset:fpgm subsetting not needed
INFO:fontTools.subset:prep subsetting not needed
INFO:fontTools.subset:cvt  subsetting not needed
INFO:fontTools.subset:loca subsetting not needed
INFO:fontTools.subset:post subsetted
INFO:fontTools.subset:gasp subsetting not needed
INFO:fontTools.subset:GDEF subsetted
INFO:fontTools.subset:GPOS subsetted
INFO:fontTools.subset:GSUB subsetted
INFO:fontTools.subset:MATH subsetted
INFO:fontTools.subset:name subsetting not needed
INFO:fontTools.subset:glyf subsetted
INFO:fontTools.subset:head pruned
INFO:fontTools.subset:OS/2 Unicode ranges pruned: [0, 38]
INFO:fontTools.subset:glyf pruned
INFO:fontTools.subset:GDEF pruned
INFO:fontTools.subset:GPOS pruned
INFO:fontTools.subset:GSUB pruned
```

Example 24 (python):
```python
CHDs = pd.read_csv('./combined_tidied_disease_gene_with_phenotype_annotation_KKmod.csv',index_col=0)
print(CHDs.shape)
CHDs.head(n=3)
```

Example 25 (unknown):
```unknown
CHDs.loc['PRDM16']
```

Example 26 (unknown):
```unknown
TF                                                                                          1
source                                                                                ClinVar
phenotype_Heterotaxy                                                                        0
phenotype_ASD                                                                               0
phenotype_VSD                                                                               0
phenotype_AVSD                                                                              0
phenotype_Malformation of outflow tracts                                                    0
phenotype_Functional single ventricle                                                       0
phenotype_Cardiomyopathy                                                                    1
phenotype_DCM                                                                               0
phenotype_HCM                                                                               0
phenotype_LVNC                                                                              1
phenotype_Venous anomaly                                                                    0
phenotype_Pulmonary artery                                                                  0
phenotype_Aortic arch                                                                       0
phenotype_Aorta                                                                             0
phenotype_Coronary anomaly                                                                  0
phenotype_VA valves                                                                         0
phenotype_AV valves                                                                         0
phenotype                                   Left ventricular noncompaction 8|Left ventricu...
Name: PRDM16, dtype: object
```

Example 27 (python):
```python
# subset table based on phenotypes
phenotypes = ['phenotype_VSD','phenotype_AVSD','phenotype_LVNC','phenotype_Cardiomyopathy','phenotype_Functional single ventricle','phenotype_DCM','phenotype_HCM']
mask = CHDs[phenotypes].sum(axis=1)>0
CHDs_sel = CHDs[mask]
print(CHDs_sel.shape)
CHDs_sel.head(n=3)
```

Example 28 (unknown):
```unknown
CHDs_sel['TF'].value_counts()
```

Example 29 (unknown):
```unknown
0    2511
1      30
Name: TF, dtype: int64
```

Example 30 (python):
```python
# select TF list inplicated in the phenotypes
CHD_TFs = list(CHDs_sel[CHDs_sel['TF']==1].index)
print(len(CHD_TFs))
CHD_TFs[:5]
```

Example 31 (unknown):
```unknown
['FOXC2', 'FOXP1', 'GATA4', 'GATA5', 'GATA6']
```

Example 32 (unknown):
```unknown
# intersect with the TFs in the network
'PRDM16' in CHD_TFs
```

Example 33 (unknown):
```unknown
# read in TF-TF network
network_TF = utils.read_pkl('/nfs/team205/heart/Vitro/TF_selection/foetal_vCM_left_compact-vs-trabeculated_downstream/TF-TF-network_compact-common-up_2Feb2024.csv')
len(network_TF.keys())
```

Example 34 (python):
```python
# prepare color map
# total PS score

from matplotlib.colors import TwoSlopeNorm

# vector of continuous values
values = np.array(TF_df['total_PS_score'])

# Find the maximum absolute value for normalization
max_abs_value = np.max(np.abs(values))
# Normalize the values manually to ensure zero is centered
# Map the range [-max_abs_value, max_abs_value] to [0, 1] with 0.5 at zero
normalized_values = 0.5 + (values / (2 * max_abs_value))

# Use a diverging colormap
cmap = plt.get_cmap('PiYG')
colors = cmap(normalized_values)

# make mapping dictionary
nodes_color_map = dict(zip(TF_df.index,colors))
```

Example 35 (unknown):
```unknown
# prepare parameters
size_scale = 7000

centrality_type = 'eigenvector'
centrality_dict = regulon_plot.get_centrality(network_TF,
                   directed=True,
                   centrality_type=centrality_type, # or 'degree','in_dgree','out_degree'
                   out_df=False # if False output dictionary
                  )

node_size_map = {}
for n in network_TF.keys():
    if n in centrality_dict.keys():
        node_size_map[n]=int(centrality_dict[n]*size_scale)
    else: # means no connection with other TFs
        node_size_map[n]=0
```

Example 36 (python):
```python
# select CHD TFs which are in the network constructed
all_TFs = []
for x in network_TF.values():
    all_TFs = all_TFs + x
all_TFs = all_TFs + list(network_TF.keys())

CHD_TFs_sel = list(set(CHD_TFs).intersection(all_TFs))
print(len(CHD_TFs_sel))
CHD_TFs_sel
```

Example 37 (unknown):
```unknown
['PBX1', 'HAND2', 'PRDM16', 'GATA6', 'SMAD3', 'TBX20', 'MEIS2', 'GATA4']
```

Example 38 (python):
```python
from matplotlib.colors import Normalize
from matplotlib.cm import ScalarMappable
from matplotlib.colorbar import ColorbarBase
from mpl_toolkits.axes_grid1 import make_axes_locatable

fig, ax = plt.subplots(figsize=(8,8))

regulon_plot.draw_networkx(
    network=network_TF,
    nodes_color_base='white',
    nodes_color_map=nodes_color_map,
    nodes_size_base=400,
    nodes_size_map=node_size_map,
    nodes_edge_highlight=CHD_TFs_sel,
    nodes_edge_color='blue',
    nodes_edge_thickness=3,
     spring_layout_k=1.5,
    spring_layout_scale=1,
    font_size=12,
    pos_seed=3,
    ax=ax
)

# Create an axes on the right side of ax. The width of cax will be 5%
# of ax and the padding between cax and ax will be fixed at 0.05 inch.
divider = make_axes_locatable(ax)
cbar_ax = divider.append_axes("right", size="2%", pad=0.01)
# cbar_ax = fig.add_axes([0.9, 0.1, 0.05, 0.3])

# Create a colorbar with correct scaling
# Setup the normalization and colormap for the colorbar
norm = Normalize(vmin=-max_abs_value, vmax=max_abs_value)
sm = ScalarMappable(norm=norm, cmap=cmap)
sm.set_array([])
cbar = plt.colorbar(sm, cax=cbar_ax, orientation='vertical')
cbar.set_label('Perturbation score (sum)')

fig.suptitle('TF network for compact CM maturation')
fig.tight_layout()
# fig.savefig(f"{figdir}/TF-network_compact-common-up.pdf",bbox_inches="tight",dpi=300)
```

Example 39 (unknown):
```unknown
<Figure size 1200x1200 with 0 Axes>
```

Example 40 (python):
```python
TFs_to_plot = ['HEY2','PRDM16','NFE2L1','SREBF2']

fig,axs=plt.subplots(nrows=1,ncols=len(TFs_to_plot),figsize=(14,4)) 
for i,tf in enumerate(TFs_to_plot):
    dev = dev_dict[tf]
    dev.plot_inner_product_on_grid(vm=vm, s=50, ax=axs[i])
    dev.plot_simulation_flow_on_grid(scale=scale_simulation, show_background=False, ax=axs[i])
    axs[i].set_title(tf)
fig.tight_layout(w_pad=-0.3)
fig.savefig(f"{figdir}/vCM-left_celloracle_PS.pdf",bbox_inches="tight",dpi=300)
```

Example 41 (unknown):
```unknown
INFO:fontTools.subset:maxp pruned
INFO:fontTools.subset:cmap pruned
INFO:fontTools.subset:kern dropped
INFO:fontTools.subset:post pruned
INFO:fontTools.subset:FFTM dropped
INFO:fontTools.subset:GPOS pruned
INFO:fontTools.subset:GSUB pruned
INFO:fontTools.subset:name pruned
INFO:fontTools.subset:glyf pruned
INFO:fontTools.subset:Added gid0 to subset
INFO:fontTools.subset:Added first four glyphs to subset
INFO:fontTools.subset:Closing glyph list over 'GSUB': 19 glyphs before
INFO:fontTools.subset:Glyph names: ['.notdef', '.null', 'B', 'D', 'E', 'F', 'H', 'L', 'M', 'N', 'P', 'R', 'S', 'Y', 'nonmarkingreturn', 'one', 'six', 'space', 'two']
INFO:fontTools.subset:Glyph IDs:   [0, 1, 2, 3, 20, 21, 25, 37, 39, 40, 41, 43, 47, 48, 49, 51, 53, 54, 60]
INFO:fontTools.subset:Closed glyph list over 'GSUB': 19 glyphs after
INFO:fontTools.subset:Glyph names: ['.notdef', '.null', 'B', 'D', 'E', 'F', 'H', 'L', 'M', 'N', 'P', 'R', 'S', 'Y', 'nonmarkingreturn', 'one', 'six', 'space', 'two']
INFO:fontTools.subset:Glyph IDs:   [0, 1, 2, 3, 20, 21, 25, 37, 39, 40, 41, 43, 47, 48, 49, 51, 53, 54, 60]
INFO:fontTools.subset:Closing glyph list over 'MATH': 19 glyphs before
INFO:fontTools.subset:Glyph names: ['.notdef', '.null', 'B', 'D', 'E', 'F', 'H', 'L', 'M', 'N', 'P', 'R', 'S', 'Y', 'nonmarkingreturn', 'one', 'six', 'space', 'two']
INFO:fontTools.subset:Glyph IDs:   [0, 1, 2, 3, 20, 21, 25, 37, 39, 40, 41, 43, 47, 48, 49, 51, 53, 54, 60]
INFO:fontTools.subset:Closed glyph list over 'MATH': 19 glyphs after
INFO:fontTools.subset:Glyph names: ['.notdef', '.null', 'B', 'D', 'E', 'F', 'H', 'L', 'M', 'N', 'P', 'R', 'S', 'Y', 'nonmarkingreturn', 'one', 'six', 'space', 'two']
INFO:fontTools.subset:Glyph IDs:   [0, 1, 2, 3, 20, 21, 25, 37, 39, 40, 41, 43, 47, 48, 49, 51, 53, 54, 60]
INFO:fontTools.subset:Closing glyph list over 'glyf': 19 glyphs before
INFO:fontTools.subset:Glyph names: ['.notdef', '.null', 'B', 'D', 'E', 'F', 'H', 'L', 'M', 'N', 'P', 'R', 'S', 'Y', 'nonmarkingreturn', 'one', 'six', 'space', 'two']
INFO:fontTools.subset:Glyph IDs:   [0, 1, 2, 3, 20, 21, 25, 37, 39, 40, 41, 43, 47, 48, 49, 51, 53, 54, 60]
INFO:fontTools.subset:Closed glyph list over 'glyf': 19 glyphs after
INFO:fontTools.subset:Glyph names: ['.notdef', '.null', 'B', 'D', 'E', 'F', 'H', 'L', 'M', 'N', 'P', 'R', 'S', 'Y', 'nonmarkingreturn', 'one', 'six', 'space', 'two']
INFO:fontTools.subset:Glyph IDs:   [0, 1, 2, 3, 20, 21, 25, 37, 39, 40, 41, 43, 47, 48, 49, 51, 53, 54, 60]
INFO:fontTools.subset:Retaining 19 glyphs
INFO:fontTools.subset:head subsetting not needed
INFO:fontTools.subset:hhea subsetting not needed
INFO:fontTools.subset:maxp subsetting not needed
INFO:fontTools.subset:OS/2 subsetting not needed
INFO:fontTools.subset:hmtx subsetted
INFO:fontTools.subset:cmap subsetted
INFO:fontTools.subset:fpgm subsetting not needed
INFO:fontTools.subset:prep subsetting not needed
INFO:fontTools.subset:cvt  subsetting not needed
INFO:fontTools.subset:loca subsetting not needed
INFO:fontTools.subset:post subsetted
INFO:fontTools.subset:gasp subsetting not needed
INFO:fontTools.subset:GDEF subsetted
INFO:fontTools.subset:GPOS subsetted
INFO:fontTools.subset:GSUB subsetted
INFO:fontTools.subset:MATH subsetted
INFO:fontTools.subset:name subsetting not needed
INFO:fontTools.subset:glyf subsetted
INFO:fontTools.subset:head pruned
INFO:fontTools.subset:OS/2 Unicode ranges pruned: [0]
INFO:fontTools.subset:glyf pruned
INFO:fontTools.subset:GDEF pruned
INFO:fontTools.subset:GPOS pruned
INFO:fontTools.subset:GSUB pruned
```

Example 42 (unknown):
```unknown
?fig.tight_layout
```

Example 43 (python):
```python
Signature: fig.tight_layout(*, pad=1.08, h_pad=None, w_pad=None, rect=None)
Docstring:
Adjust the padding between and around subplots.

To exclude an artist on the Axes from the bounding box calculation
that determines the subplot parameters (i.e. legend, or annotation),
set ``a.set_in_layout(False)`` for that artist.

Parameters
----------
pad : float, default: 1.08
    Padding between the figure edge and the edges of subplots,
    as a fraction of the font size.
h_pad, w_pad : float, default: *pad*
    Padding (height/width) between edges of adjacent subplots,
    as a fraction of the font size.
rect : tuple (left, bottom, right, top), default: (0, 0, 1, 1)
    A rectangle in normalized figure coordinates into which the whole
    subplots area (including labels) will fit.

See Also
--------
.Figure.set_layout_engine
.pyplot.tight_layout
File:      /nfs/team205/kk18/miniconda3/envs/celloracle_env/lib/python3.8/site-packages/matplotlib/figure.py
Type:      method
```

---

## Import modules¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig3_SAN_niche/cellphonedb-scnRNAseq_plot_SAN.html

**Contents:**
- Import modules¶
- Variables¶
- Read in gene expression data¶
- Read in cpdb outputs¶
- Update the fine-grained cell type name for plotting¶
- Plot¶
  - Receptor expressed in PSNS¶

5 rows × 3981 columns

**Examples:**

Example 1 (python):
```python
import scanpy as sc
import pandas as pd
pd.set_option('display.max_rows', 100)
import numpy as np
import matplotlib.pyplot as plt
import matplotlib as mpl
import seaborn as sns

import ktplotspy as kpy

import os 
from matplotlib.patches import Rectangle
import itertools
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
-----
ktplotspy           0.2.4
matplotlib          3.7.5
numpy               1.24.4
pandas              2.0.3
scanpy              1.9.8
seaborn             0.13.2
session_info        1.0.0
-----
```

Example 4 (python):
```python
Bio                 1.83
PIL                 10.4.0
anndata             0.9.2
asttokens           NA
backcall            0.2.0
backports           NA
certifi             2024.12.14
charset_normalizer  3.4.1
comm                0.2.2
cycler              0.12.1
cython_runtime      NA
dateutil            2.9.0.post0
debugpy             1.8.12
decorator           5.1.1
executing           2.2.0
h5py                3.11.0
idna                3.10
importlib_metadata  NA
importlib_resources NA
ipykernel           6.29.5
jaraco              NA
jedi                0.19.2
joblib              1.4.2
kiwisolver          1.4.7
llvmlite            0.41.1
mizani              0.9.3
more_itertools      10.3.0
mpl_toolkits        NA
natsort             8.4.0
numba               0.58.1
packaging           24.2
parso               0.8.4
patsy               1.0.1
pexpect             4.9.0
pickleshare         0.7.5
pkg_resources       NA
platformdirs        4.3.6
plotnine            0.12.4
prompt_toolkit      3.0.50
psutil              6.1.1
ptyprocess          0.7.0
pure_eval           0.2.3
pycircos            NA
pydev_ipython       NA
pydevconsole        NA
pydevd              3.2.3
pydevd_file_utils   NA
pydevd_plugins      NA
pydevd_tracing      NA
pygments            2.19.1
pyparsing           3.1.4
pytz                2024.2
requests            2.32.3
scipy               1.10.1
six                 1.17.0
sklearn             1.3.2
stack_data          0.6.3
statsmodels         0.14.1
threadpoolctl       3.5.0
tornado             6.4.2
traitlets           5.14.3
typing_extensions   NA
urllib3             2.2.3
wcwidth             0.2.13
zipp                NA
zmq                 26.2.0
```

Example 5 (unknown):
```unknown
-----
IPython             8.12.3
jupyter_client      8.6.3
jupyter_core        5.7.2
-----
Python 3.8.20 | packaged by conda-forge | (default, Sep 30 2024, 17:52:49) [GCC 13.3.0]
Linux-5.15.0-126-generic-x86_64-with-glibc2.17
-----
Session information updated at 2025-04-27 11:07
```

Example 6 (python):
```python
sc.settings.set_figure_params(dpi=80,dpi_save=300)
```

Example 7 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk/Aug2024'
table_dir = '/nfs/team205/heart/fetal_ms_figures/tables/Aug2024'
```

Example 8 (python):
```python
path_to_cpdb = f'/nfs/team205/heart/cellphonedb/outputs/foetal/stat/all_cells_min0p05'
path_to_adata = f'/nfs/team205/heart/cellphonedb/inputs/foetal/log_norm_counts_all.h5ad'
celltype_col = 'fine_grain'
```

Example 9 (python):
```python
# read in adata
adata = sc.read_h5ad(path_to_adata)
adata
```

Example 10 (unknown):
```unknown
AnnData object with n_obs × n_vars = 297473 × 34805
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score', 'cell_id'
    var: 'gene_name_scRNA-0', 'gene_id', 'n_cells'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'age_colors', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'cycling_colors', 'donor_colors', 'fine_grain_colors', 'heart_or_greatvessels_colors', 'hvg', 'kit_10x_colors', 'modality_colors', 'neighbors', 'phase_colors', 'region_colors', 'sex_colors', 'trimester_colors', 'umap', 'week_colors'
    obsm: 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 11 (unknown):
```unknown
timestamp = [x for x in os.listdir(path_to_cpdb) if '_means_' in x][0].split('_means_')[1].replace('.txt','')
timestamp
```

Example 12 (unknown):
```unknown
'01_26_2024_03:48:49'
```

Example 13 (python):
```python
cpdbout_means =  pd.read_table(f'{path_to_cpdb}/statistical_analysis_means_{timestamp}.txt',index_col=0)
cpdbout_pvals =  pd.read_table(f'{path_to_cpdb}/statistical_analysis_pvalues_{timestamp}.txt',index_col=0)
cpdbout_sigmeans =  pd.read_table(f'{path_to_cpdb}/statistical_analysis_significant_means_{timestamp}.txt',index_col=0)
interaction_metadata = pd.read_csv('/nfs/team205/heart/cellphonedb/customDB/interaction_meta/interaction_metadata_v5.0.0_2024-1-26.csv')
```

Example 14 (python):
```python
/tmp/ipykernel_705256/3444635095.py:1: DtypeWarning: Columns (4) have mixed types. Specify dtype option on import or set low_memory=False.
/tmp/ipykernel_705256/3444635095.py:2: DtypeWarning: Columns (4) have mixed types. Specify dtype option on import or set low_memory=False.
```

Example 15 (python):
```python
# add gene names to cpdb outputs
cpdbout_means['gene_name_1'] = list(interaction_metadata.set_index('id_cp_interaction').loc[cpdbout_means.index,'gene_name_1'])
cpdbout_means['gene_name_2'] = list(interaction_metadata.set_index('id_cp_interaction').loc[cpdbout_means.index,'gene_name_2'])
cpdbout_pvals['gene_name_1'] = list(interaction_metadata.set_index('id_cp_interaction').loc[cpdbout_pvals.index,'gene_name_1'])
cpdbout_pvals['gene_name_2'] = list(interaction_metadata.set_index('id_cp_interaction').loc[cpdbout_pvals.index,'gene_name_2'])
cpdbout_sigmeans['gene_name_1'] = list(interaction_metadata.set_index('id_cp_interaction').loc[cpdbout_sigmeans.index,'gene_name_1'])
cpdbout_sigmeans['gene_name_2'] = list(interaction_metadata.set_index('id_cp_interaction').loc[cpdbout_sigmeans.index,'gene_name_2'])
```

Example 16 (python):
```python
# insert gene_name columns to the cpdb output
# to select interactions based on the list
cpdbout_means[['gene_name_1','gene_name_2']] = interaction_metadata.set_index('id_cp_interaction').loc[cpdbout_means.index,['gene_name_1','gene_name_2']]
cpdbout_pvals[['gene_name_1','gene_name_2']] = interaction_metadata.set_index('id_cp_interaction').loc[cpdbout_pvals.index,['gene_name_1','gene_name_2']]
cpdbout_sigmeans[['gene_name_1','gene_name_2']] = interaction_metadata.set_index('id_cp_interaction').loc[cpdbout_sigmeans.index,['gene_name_1','gene_name_2']]
```

Example 17 (python):
```python
# query a gene
gene = 'ADGRG1'
mask1 = [gene in x.split(',') for x in interaction_metadata['gene_name_1']]
mask2 = [gene in x.split(',') for x in interaction_metadata['gene_name_2']]
mask = [any(x) for x in zip(mask1,mask2)]
interaction_metadata[mask][['id_cp_interaction','gene_name_1','gene_name_2']]
```

Example 18 (python):
```python
df = pd.read_csv('/nfs/team205/heart/anndata_objects/Foetal/finegrain_name_mapping.csv')
celltype_mapping = df.set_index('Full_name')['Short_name_finalised'].to_dict()

# additional for Xenium
celltype_mapping['SinoatrialNodePacemakerCells']='SANPC'
```

Example 19 (python):
```python
# update cell type name in adata
adata.obs.replace({celltype_col:celltype_mapping},inplace=True)

# update cell type name in means dataframe
updated_columns = []
for col in cpdbout_means.columns:
    for key, value in celltype_mapping.items():
        col = col.replace(key, value)
    updated_columns.append(col)
cpdbout_means.columns = updated_columns.copy()

# update cell type name in pvals dataframe
updated_columns = []
for col in cpdbout_pvals.columns:
    for key, value in celltype_mapping.items():
        col = col.replace(key, value)
    updated_columns.append(col)
cpdbout_pvals.columns = updated_columns.copy()

# update cell type name in sig_means dataframe
updated_columns = []
for col in cpdbout_sigmeans.columns:
    for key, value in celltype_mapping.items():
        col = col.replace(key, value)
    updated_columns.append(col)
cpdbout_sigmeans.columns = updated_columns.copy()
```

Example 20 (unknown):
```unknown
cpdbout_means.head()
```

Example 21 (unknown):
```unknown
receptor_genes = [
    # Netrin receptors
    "DCC", "NEO1", "UNC5A", "UNC5B", "UNC5C", "UNC5D",
    # Slit (Robo) receptors
    "ROBO1", "ROBO2", "ROBO3", "ROBO4",
    # Semaphorin receptors
    "NRP1", "NRP2",
    "PLXNA1", "PLXNA2", "PLXNA3", "PLXNA4",
    "PLXNB1", "PLXNB2", "PLXNB3",
    "PLXNC1", "PLXND1",
    # Ephrin (Eph) receptors
    "EPHA1", "EPHA2", "EPHA3", "EPHA4", "EPHA5", "EPHA6", "EPHA7", "EPHA8", "EPHA10",
    "EPHB1", "EPHB2", "EPHB3", "EPHB4", "EPHB6",
    # Neurotrophin / GDNF family receptors
    "NTRK1", "NTRK2", "NTRK3", "NGFR", "RET",
    "GFRA1", "GFRA2", "GFRA3", "GFRA4",
    # Neuregulin (ErbB) receptors
    "ERBB2", "ERBB3", "ERBB4",
    # Additional innervation‑related receptors
    "EDNRB", "LRP8", "VLDLR", "L1CAM", "NCAM1",
    # Others
    "CXCR4","CNTNAP4","FGFR2"
]
receptor_celltype = 'ParaN'

# ligand_genes = ['SEMA3A', 'SLIT2', 'SLIT3', 'EFNB2', 'CXCL12', 'FGF10', 'VEGFC', 'NELL1']
ligand_celltypes = ['SANPCHn','SANPCHd','SANPCT','aCMR']
```

Example 22 (python):
```python
def receiving_signal(
    cpdbout_means,
    cpdbout_pvals,
    receptor_celltype,
    receptor_genes,
    ligand_celltypes=None
):
    # Subset results
    
    ### get columns
    cellpair_cols = [x for x in cpdbout_means.columns if '|' in x]
    other_cols = [x for x in cpdbout_means.columns if '|' not in x]
    
    ### get dataframe which has receptor_celltype and the receptor_genes at the "b/2" side (majority will be like this)
    mask_row = (
        cpdbout_means['gene_name_2']
        .str.split(',')
        .apply(lambda names: any(g in receptor_genes for g in names))
    )
    if ligand_celltypes!=None:
        cellpair_cols_sel = [x for x in cellpair_cols if (f'|{receptor_celltype}' in x)&(x.split('|')[0] in ligand_celltypes)]
    else:
        cellpair_cols_sel = [x for x in cellpair_cols if (f'|{receptor_celltype}' in x)]
    col_sel = other_cols + cellpair_cols_sel
    means_2 = cpdbout_means.loc[mask_row, col_sel]
    pvals_2 = cpdbout_pvals.loc[mask_row, col_sel]

    ### get dataframe which has receptor_celltype and the receptor_genes at the "a/1" side
    mask_row = (
        cpdbout_means['gene_name_1']
        .str.split(',')
        .apply(lambda names: any(g in receptor_genes for g in names))
    )
    if ligand_celltypes!=None:
        cellpair_cols_sel = [x for x in cellpair_cols if (f'{receptor_celltype}|' in x)&(x.split('|')[1] in ligand_celltypes)]
    else:
        cellpair_cols_sel = [x for x in cellpair_cols if (f'{receptor_celltype}|' in x)]
    col_sel = other_cols + cellpair_cols_sel
    means_1 = cpdbout_means.loc[mask_row, col_sel]
    pvals_1 = cpdbout_pvals.loc[mask_row, col_sel]
    # remove rows which are already exist in means_2
    index_keep = list(set(means_1.index)-set(means_2.index))
    means_1 = means_1.loc[index_keep]
    pvals_1 = pvals_1.loc[index_keep]
    # reverse gene_a and gene_b
    mapper = {'partner_a':'partner_b',
              'partner_b':'partner_a',
              'gene_a':'gene_b',
              'gene_b':'gene_a',
              'receptor_a':'receptor_b',
              'receptor_b':'receptor_a',
              'directionality':'directionality_reverse'}
    means_1.rename(columns=mapper,inplace=True)
    pvals_1.rename(columns=mapper,inplace=True)

    new_columns = []
    for col in means_1.columns:
        if '|' in col:
            cell_a = col.split('|')[0]
            cell_b = col.split('|')[1]
            new_columns.append(f'{cell_b}|{cell_a}')
        else:
            new_columns.append(col)
    means_1.columns = new_columns
    pvals_1.columns = new_columns

    new_interacting_pair = []
    for ind in means_1.index:
        partner_a = means_1.loc[ind,'partner_a']
        partner_b = means_1.loc[ind,'partner_b']
        if 'complex:' in partner_a:
            partner_a = partner_a.replace('complex:','')
        else:
            partner_a = means_1.loc[ind,'gene_a']
        if 'complex:' in partner_b:
            partner_b = partner_b.replace('complex:','')
        else:
            partner_b = means_1.loc[ind,'gene_b']
        new_interacting_pair.append(f'{partner_a}_{partner_b}')
    means_1['interacting_pair'] = new_interacting_pair
    pvals_1['interacting_pair'] = new_interacting_pair

    ### concatenate
    means = pd.concat([means_2,means_1])
    pvals = pd.concat([pvals_2,pvals_1])

    ### remove NaN
    means = means.dropna(axis=0, how='all')
    means = means.dropna(axis=1, how='all')
    pvals = pvals.dropna(axis=0, how='all')
    pvals = pvals.dropna(axis=1, how='all')

    ### formatting
    means = means.reset_index()
    pvals = pvals.reset_index()
    # means.drop(labels=['gene_name_1','gene_name_2','directionality_reverse'],axis=1,inplace=True)
    # pvals.drop(labels=['gene_name_1','gene_name_2','directionality_reverse'],axis=1,inplace=True)
    means.drop(labels=['gene_name_1','gene_name_2'],axis=1,inplace=True)
    pvals.drop(labels=['gene_name_1','gene_name_2'],axis=1,inplace=True)
    
    return means, pvals
```

Example 23 (python):
```python
means,pvals = receiving_signal(
    cpdbout_means,
    cpdbout_pvals,
    receptor_celltype,
    receptor_genes,
    ligand_celltypes=['SANPC','aCMR','vCMCR','vCMTR']
)

plot = kpy.plot_cpdb(
    adata=adata,
    cell_type1='.',
    cell_type2='.',  # this means all cell-types
    means=means,
    pvals=pvals,
    alpha=0.05,
    celltype_key=celltype_col,
    genes=None,
    figsize=(5.2, 9),
    title='SAN, CMs-to-ParaN,\nall AxonG',
    highlight_size =1.5,
    standard_scale=False
)
plot.save(f"{figdir}/snRNAseq_CellPhoneDB_SAN_CM-AxonG-to-ParaN_all-ligands.pdf", bbox_inches="tight",dpi=300)
plot
```

Example 24 (unknown):
```unknown
/software/cellgen/team205/kk18/envs/cpdb/lib/python3.8/site-packages/plotnine/ggplot.py:587: PlotnineWarning: Saving 5.2 x 9 in image.
/software/cellgen/team205/kk18/envs/cpdb/lib/python3.8/site-packages/plotnine/ggplot.py:588: PlotnineWarning: Filename: /nfs/team205/heart/fetal_ms_figures/raw_plots_kk/Aug2024/snRNAseq_CellPhoneDB_SAN_CM-AxonG-to-ParaN_all-ligands.pdf
```

Example 25 (unknown):
```unknown
<Figure Size: (520 x 900)>
```

---

## Function¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig5_Trisomy21/no10_scoring_maturation-compaction-signatures.html

**Contents:**
- Function¶
- Read in adata¶
- Normalisation¶
- vCM Maturation signature genes¶

**Examples:**

Example 1 (python):
```python
import scanpy as sc
import pandas as pd
import numpy as np
import anndata
import scipy
import matplotlib.pyplot as plt
import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 
import seaborn as sns

import warnings
warnings.simplefilter("ignore", UserWarning)
warnings.filterwarnings("ignore", category=DeprecationWarning)
```

Example 2 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 3 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/rfs/project/rfs-iCNyzSAaucw/kk837/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 4 (unknown):
```unknown
figdir = '/rfs/project/rfs-iCNyzSAaucw/kk837/notebooks/Foetal/fetal_ms_figures'
```

Example 5 (python):
```python
from scipy.stats import ranksums
from statsmodels.stats.multitest import multipletests

def plot_boxplot(adata, 
                 celltype,
                 score_name,
                 logFC_direction = 'euploid', # euploid or tri21 or both
                 fdr_thresh=0.1,
                 palette = ['gray','lightgray'],
                 savepath = None
                ):
    
    # subset adata
    ad = adata[adata.obs['nhood_annotation']==celltype]
    # target nhoods
    mask_sig = ad.obs['SpatialFDR']<fdr_thresh
    mask_eup = ad.obs['logFC']<0
    mask_t21 = ad.obs['logFC']>0
    nhoods_eup = list(ad.obs[mask_sig&mask_eup].index)
    nhoods_t21 = list(ad.obs[mask_sig&mask_t21].index)
    # add nhood labels
    ad.obs['group_to_compare'] = 'reference'
    ad.obs.loc[nhoods_eup,'group_to_compare'] = 'euploid_nhoods'
    ad.obs.loc[nhoods_t21,'group_to_compare'] = 'tri21_nhoods'
    # subset adata
    if logFC_direction=='both':
        ad.obs['group_to_compare'] = ad.obs['group_to_compare'].astype('category')
        ad.obs['group_to_compare'] = ad.obs['group_to_compare'].cat.reorder_categories(
            ['euploid_nhoods','reference','tri21_nhoods'])
    elif logFC_direction in ['tri21','euploid']:
        ad = ad[ad.obs['group_to_compare'].isin(['reference',f'{logFC_direction}_nhoods'])]
        ad.obs['group_to_compare'] = ad.obs['group_to_compare'].astype('category')
        ad.obs['group_to_compare'] = ad.obs['group_to_compare'].cat.reorder_categories(
            [f'{logFC_direction}_nhoods','reference'])
    else:
        return print('ERROR: logFC_direction is not appropriate')
    
    # prepare datafrane for plotting
    data = ad.obs.copy()
    
    ### plot ###
    split_col = 'group_to_compare'
    # Create a catplot with boxplots for each gene
    ncols=1
    g = sns.catplot(x=split_col, y=score_name, col='nhood_annotation', data=data, kind='box', palette=palette,
                    flierprops=dict(marker=''),
                    col_wrap=ncols, sharey=False, sharex=False, height=5, aspect=0.6)
    # Overlay swarmplot on each Axes
    # We'll use the 'hue' parameter to match the subsets and turn off dodge to align the points with the boxes.
    for celltype, ax in zip(data['nhood_annotation'].unique(), g.axes.flat):
        subset = data[data['nhood_annotation'] == celltype]
        sns.swarmplot(x=split_col, y=score_name, data=subset, color='k', size=2, ax=ax, dodge=False)

    # Update titles to remove "gene ="
    for ax, celltype in zip(g.axes.flat, data['nhood_annotation'].unique()):
        middle_ind = round(len(celltype)/2)
        celltype_title = f'{celltype[:middle_ind]}\n{celltype[middle_ind:]}'
        ax.set_title(celltype_title)

    # Rotate x-tick labels and remove x-axis labels
    for i,ax in enumerate(g.axes.flat):
        ax.set_xticklabels(ax.get_xticklabels(), rotation=45, ha="right", rotation_mode="anchor")  # Rotate x-tick labels
        # ax.set_xticklabels('')
        ax.set_xlabel('')  # Remove x-axis labels
        if i!=0:
            ax.set_ylabel('')

    # Adjust the spacing of the plots
    g.fig.subplots_adjust(wspace=0.4, hspace=0.4)
    
    # Show the plot
    if savepath!=None:
        plt.savefig(savepath,bbox_inches="tight",dpi=300)
    plt.ylabel(score_name)
    plt.show()

    # statistical test
    if logFC_direction=='both':
        # Wilcoxon Rank-Sum (Mann-Whitney U) Test
        # euploid vs reference
        ranksum_eup_vs_ref = ranksums(data[data['group_to_compare']=='reference'][score_name],
                                      data[data['group_to_compare']=='euploid_nhoods'][score_name])
        # trisomy21 vs reference
        ranksum_t21_vs_ref = ranksums(data[data['group_to_compare']=='reference'][score_name],
                                      data[data['group_to_compare']=='tri21_nhoods'][score_name])
        # trisomy21 vs euploid
        ranksum_t21_vs_eup = ranksums(data[data['group_to_compare']=='euploid_nhoods'][score_name],
                                      data[data['group_to_compare']=='tri21_nhoods'][score_name])
        # multipletest correction
        pvals_adjusted = multipletests([ranksum_eup_vs_ref[1],
                                        ranksum_t21_vs_ref[1],
                                        ranksum_t21_vs_eup[1]],alpha=0.05, method='fdr_bh')[1]
        print(f'adjusted p-values of eup-vs-ref, t21-vs-ref, t21-vs-eup: {pvals_adjusted}')
        
    elif logFC_direction in ['tri21','euploid']:
        print(ranksums(data[data['group_to_compare']=='reference'][score_name],
             data[data['group_to_compare']==f'{logFC_direction}_nhoods'][score_name]))
```

Example 6 (python):
```python
# pseudobulked per nhood
# nhood-by-gene
adata = sc.read_h5ad('/rfs/project/rfs-iCNyzSAaucw/kk837/data_objects/Foetal/trisomy21/Euploid_T21Hearts_Aug2024_sel_subsampled-per-cellstate-donor_pseudobulked-nhood-sum.h5ad')
adata
```

Example 7 (unknown):
```unknown
AnnData object with n_obs × n_vars = 6693 × 36601
    obs: 'index_cell', 'kth_distance', 'logFC', 'logCPM', 'F', 'PValue', 'FDR', 'SpatialFDR', 'Nhood_size', 'nhood_annotation', 'nhood_annotation_frac'
    var: 'gene_id', 'gene_name_scRNA-0-0', 'feature_type-1', 'mt-1', 'ribo-1', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1', 'gene_name_GenomicFeatures', 'gene_biotype_GenomicFeatures', 'gene_seq_start_GenomicFeatures', 'gene_seq_end_GenomicFeatures', 'seq_name_GenomicFeatures', 'seq_strand_GenomicFeatures', 'seq_coord_system_GenomicFeatures', 'description_GenomicFeatures', 'gene_id_version_GenomicFeatures', 'canonical_transcript_GenomicFeatures', 'gene_name'
```

Example 8 (python):
```python
sc.pp.filter_genes(adata, min_cells=3)
# log1pCPM
sc.pp.normalize_total(adata, target_sum=1e6)
sc.pp.log1p(adata)
```

Example 9 (python):
```python
# read in gene signature
path_to_gene_signatures = '/rfs/project/rfs-iCNyzSAaucw/kk837/notebooks/Foetal/vCMLeft/gene_signatures.pkl'
target_signatures = ['compact_specific_up','common_up']

gene_signatures = utils.read_pkl(path_to_gene_signatures)
print(gene_signatures.keys())

sig_genes = []
for sig in target_signatures:
    print(sig)
    sig_genes = sig_genes + gene_signatures[sig]
print(len(sig_genes))
sig_genes[:5]
```

Example 10 (unknown):
```unknown
dict_keys(['compact_specific_up', 'trab_specific_up', 'common_up', 'compact_specific_down', 'trab_specific_down', 'common_down'])
compact_specific_up
common_up
466
```

Example 11 (unknown):
```unknown
['KIDINS220', 'ANKRD18A', 'MT-ATP6', 'ZNF385D', 'VEGFA']
```

Example 12 (python):
```python
# set some variables
score_name = 'vCMcompact_maturation_score'
celltypes_to_analyse = ['VentricularCardiomyocytesLeftCompact',
                         'VentricularCardiomyocytesLeftTrabeculated',
                         'VentricularCardiomyocytesRightCompact',
                         'VentricularCardiomyocytesRightTrabeculated',
                        'VentricularCardiomyocytesCycling']

if celltypes_to_analyse!=None:
    print('subsetting')
    adata_sub = adata[adata.obs['nhood_annotation'].isin(celltypes_to_analyse)]
else:
    print('just removing "Mixed"')
    adata_sub = adata[adata.obs['nhood_annotation']!='Mixed']
```

Example 13 (python):
```python
# obtain gene score
sc.tl.score_genes(adata_sub, gene_list=sig_genes, 
                  score_name=score_name, random_state=0)
```

Example 14 (python):
```python
# boxplot
plot_boxplot(adata_sub, 
                 celltype='VentricularCardiomyocytesLeftCompact',
                 score_name=score_name,
                 logFC_direction = 'both', # euploid or tri21
                 fdr_thresh=0.1,
                 palette = ['lightblue','lightgray','lightpink'],
                 savepath = f'{figdir}/Tri21_Milo-nhoods_Boxplot_vCMleftcompact_vCM-compaction-maturation.pdf'
                )
```

Example 15 (python):
```python
/tmp/ipykernel_2045939/1831124056.py:45: FutureWarning: 

Passing `palette` without assigning `hue` is deprecated and will be removed in v0.14.0. Assign the `x` variable to `hue` and set `legend=False` for the same effect.

  g = sns.catplot(x=split_col, y=score_name, col='nhood_annotation', data=data, kind='box', palette=palette,
```

Example 16 (unknown):
```unknown
adjusted p-values of eup-vs-ref, t21-vs-ref, t21-vs-eup: [2.24125208e-15 5.07696046e-03 2.04802511e-08]
```

Example 17 (python):
```python
# boxplot
plot_boxplot(adata_sub, 
                 celltype='VentricularCardiomyocytesRightCompact',
                  score_name=score_name,
                 logFC_direction = 'both', # euploid or tri21
                 fdr_thresh=0.1,
                 palette = ['lightblue','lightgray','lightpink'],
                 savepath = f'{figdir}/Tri21_Milo-nhoods_Boxplot_vCMrightcompact_vCM-compaction-maturation.pdf'
                )
```

Example 18 (python):
```python
/tmp/ipykernel_2045939/1831124056.py:45: FutureWarning: 

Passing `palette` without assigning `hue` is deprecated and will be removed in v0.14.0. Assign the `x` variable to `hue` and set `legend=False` for the same effect.

  g = sns.catplot(x=split_col, y=score_name, col='nhood_annotation', data=data, kind='box', palette=palette,
```

Example 19 (unknown):
```unknown
adjusted p-values of eup-vs-ref, t21-vs-ref, t21-vs-eup: [8.18327508e-09 4.29946937e-03 3.74536337e-03]
```

Example 20 (unknown):
```unknown
3.74536337e-03
```

Example 21 (unknown):
```unknown
0.00374536337
```

---

## Variables¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig4_Cardiomyocytes/AtrialCardiomyocytes/1_prepare_for_multivi.html

**Contents:**
- Variables¶
- Prepare paired anndata¶
- Prepare unimodality data¶

12964 rows × 40 columns

544641 rows × 3 columns

**Examples:**

Example 1 (python):
```python
import scanpy as sc
import pandas as pd
import numpy as np
import anndata
import scipy
import os

from scipy.io import mmwrite, mmread
from scipy.sparse import csr_matrix

import sctk
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (unknown):
```unknown
-----
anndata             0.8.0
numpy               1.24.4
pandas              1.5.3
scanpy              1.9.3
scipy               1.11.1
sctk                NA
session_info        1.0.0
-----
```

Example 4 (python):
```python
PIL                         10.0.0
anyio                       NA
arrow                       1.2.3
asttokens                   NA
attr                        23.1.0
attrs                       23.1.0
babel                       2.12.1
backcall                    0.2.0
brotli                      1.0.9
certifi                     2023.07.22
cffi                        1.15.1
charset_normalizer          3.2.0
comm                        0.1.3
cycler                      0.10.0
cython_runtime              NA
dateutil                    2.8.2
debugpy                     1.6.7
decorator                   5.1.1
defusedxml                  0.7.1
executing                   1.2.0
fastjsonschema              NA
fqdn                        NA
h5py                        3.9.0
idna                        3.4
igraph                      0.10.6
ipykernel                   6.25.0
isoduration                 NA
jedi                        0.18.2
jinja2                      3.1.2
joblib                      1.3.1
json5                       NA
jsonpointer                 2.0
jsonschema                  4.18.4
jsonschema_specifications   NA
jupyter_events              0.6.3
jupyter_server              2.7.0
jupyterlab_server           2.24.0
kiwisolver                  1.4.4
leidenalg                   0.10.1
llvmlite                    0.40.1
markupsafe                  2.1.3
matplotlib                  3.7.2
mpl_toolkits                NA
natsort                     8.4.0
nbformat                    5.9.1
numba                       0.57.1
numpy_groupies              0.9.22
overrides                   NA
packaging                   23.1
parso                       0.8.3
patsy                       0.5.3
pexpect                     4.8.0
pickleshare                 0.7.5
pkg_resources               NA
platformdirs                3.9.1
plotly                      5.16.1
prometheus_client           NA
prompt_toolkit              3.0.39
psutil                      5.9.5
ptyprocess                  0.7.0
pure_eval                   0.2.2
pyarrow                     12.0.1
pydev_ipython               NA
pydevconsole                NA
pydevd                      2.9.5
pydevd_file_utils           NA
pydevd_plugins              NA
pydevd_tracing              NA
pygments                    2.15.1
pyparsing                   3.0.9
pythonjsonlogger            NA
pytz                        2023.3
referencing                 NA
requests                    2.31.0
rfc3339_validator           0.1.4
rfc3986_validator           0.1.1
rpds                        NA
seaborn                     0.12.2
send2trash                  NA
six                         1.16.0
sklearn                     1.3.0
sniffio                     1.3.0
socks                       1.7.1
stack_data                  0.6.2
statsmodels                 0.14.0
texttable                   1.6.7
threadpoolctl               3.2.0
tornado                     6.3.2
traitlets                   5.9.0
typing_extensions           NA
uri_template                NA
urllib3                     1.26.16
wcwidth                     0.2.6
webcolors                   1.13
websocket                   1.6.1
yaml                        6.0
zmq                         25.1.0
```

Example 5 (unknown):
```unknown
-----
IPython             8.14.0
jupyter_client      8.3.0
jupyter_core        5.3.1
jupyterlab          4.0.3
notebook            7.0.0
-----
Python 3.11.4 | packaged by conda-forge | (main, Jun 10 2023, 18:08:17) [GCC 12.2.0]
Linux-5.4.0-163-generic-x86_64-with-glibc2.27
-----
Session information updated at 2023-10-19 18:44
```

Example 6 (python):
```python
sc.settings.set_figure_params(dpi=120)
```

Example 7 (python):
```python
job_name="aCMs_AnyRegion" #  corresponds to name on input adata(s) .h5ad files. E.g. cell type or bespoke job like "aCMs_WholeSample"
```

Example 8 (unknown):
```unknown
path_to_RNA_object='/nfs/team205/heart/anndata_objects/Foetal/Feb28ObjectRaw.h5ad' # global object
path_to_ATAC_object='/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/PeakMatrix/Foetal_Peaks.h5ad' # global object
```

Example 9 (unknown):
```unknown
celltype_key = 'fine_grain'
celltypes_to_analyse = ['AtrialCardiomyocytesRight','AtrialCardiomyocytesLeft']
```

Example 10 (unknown):
```unknown
region_subset=False

if region_subset:
    region_key='region'
    regions_to_analyse = ['whole sample']
```

Example 11 (unknown):
```unknown
obs_to_keep=['sangerID', 'combinedID', 'donor', 
                       'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 
                       'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts',
                       'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts',
                       'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt',
                       'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo',
                       'pct_counts_ribo', 'HB_score', 'multiplexed',
                        'batch_key','region','FACSgate','sex','week','trimester',
                     'fine_grain','mid_grain','coarse_grain',
                     'heart_or_greatvessels','cycling','S_score','G2M_score','phase','stress_score','hb1_score']
```

Example 12 (unknown):
```unknown
RNA_var_to_keep=['gene_name_scRNA-0','gene_id']
```

Example 13 (python):
```python
paired_adata_out_path=os.path.join("/lustre/scratch126/cellgen/team205/heart/objects/fetal/RNA",f"{job_name}_paired.h5ad")
paired_adata_out_path
```

Example 14 (unknown):
```unknown
'/lustre/scratch126/cellgen/team205/heart/objects/fetal/RNA/aCMs_AnyRegion_paired.h5ad'
```

Example 15 (unknown):
```unknown
multiome_only=True
```

Example 16 (python):
```python
# rna
adata_rna = sc.read_h5ad(path_to_RNA_object)

# subset cells
adata_rna = adata_rna[adata_rna.obs[celltype_key].isin(celltypes_to_analyse)]

# subset regions
if region_subset:
    adata_rna = adata_rna[adata_rna.obs[region_key].isin(regions_to_analyse)]

# add modality to .var
adata_rna.var['modality']='Gene Expression'

adata_rna
```

Example 17 (python):
```python
/tmp/ipykernel_211566/3076047041.py:12: ImplicitModificationWarning: Trying to modify attribute `.var` of view, initializing view as actual.
  adata_rna.var['modality']='Gene Expression'
```

Example 18 (unknown):
```unknown
AnnData object with n_obs × n_vars = 17302 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score'
    var: 'gene_name_scRNA-0', 'gene_id', 'modality'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'age_colors', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'cycling_colors', 'donor_colors', 'fine_grain_colors', 'heart_or_greatvessels_colors', 'hvg', 'kit_10x_colors', 'modality_colors', 'neighbors', 'phase_colors', 'region_colors', 'sex_colors', 'trimester_colors', 'umap', 'week_colors'
    obsm: 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 19 (python):
```python
pd.crosstab(adata_rna.obs[celltype_key],adata_rna.obs['region'])
```

Example 20 (python):
```python
pd.crosstab(adata_rna.obs[celltype_key],adata_rna.obs['kit_10x'])
```

Example 21 (python):
```python
pd.crosstab(adata_rna.obs[celltype_key],adata_rna.obs['cycling'])
```

Example 22 (python):
```python
# atac
# ie only Multiome data
adata_atac = sc.read_h5ad(path_to_ATAC_object)

# subset cells
adata_atac = adata_atac[adata_atac.obs[celltype_key].isin(celltypes_to_analyse)]
print(adata_atac.obs[celltype_key].value_counts())

# subset regions
if region_subset:
    adata_atac = adata_atac[adata_atac.obs[region_key].isin(regions_to_analyse)]

# add modality to .var
adata_atac.var['modality']='Peaks'

adata_atac
```

Example 23 (unknown):
```unknown
AtrialCardiomyocytesRight    8187
AtrialCardiomyocytesLeft     4777
Name: fine_grain, dtype: int64
```

Example 24 (python):
```python
/tmp/ipykernel_211566/3253878154.py:14: ImplicitModificationWarning: Trying to modify attribute `.var` of view, initializing view as actual.
  adata_atac.var['modality']='Peaks'
```

Example 25 (unknown):
```unknown
AnnData object with n_obs × n_vars = 12964 × 508040
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels'
    var: 'seqnames', 'start', 'end', 'width', 'strand', 'score', 'replicateScoreQuantile', 'groupScoreQuantile', 'Reproducibility', 'GroupReplicate', 'distToGeneStart', 'nearestGene', 'peakType', 'distToTSS', 'nearestTSS', 'GC', 'idx', 'N', 'modality'
```

Example 26 (python):
```python
pd.crosstab(adata_atac.obs[celltype_key],adata_atac.obs['region'])
```

Example 27 (python):
```python
# shared barcodes, ie Multiome data (not all, since MultiomeRNA has a bit more data than MultiomeATAC)
barcodes_inter=list(set(adata_rna.obs_names).intersection(adata_atac.obs_names))
len(barcodes_inter)

# take shared cells
multiome_rna=adata_rna[barcodes_inter,:]
multiome_atac=adata_atac[barcodes_inter,:]

# concatenate
adata_paired=anndata.AnnData(
    X=scipy.sparse.hstack([multiome_rna.X,multiome_atac.X]).tocsr(), # concatenate sparse matrix and convert to Compressed Sparse Row format
    obs=multiome_rna.obs[obs_to_keep],
    var=pd.concat([multiome_rna.var[['modality']+RNA_var_to_keep], \
                   multiome_atac.var[['modality']]
                  ],axis=0)
)
adata_paired
```

Example 28 (unknown):
```unknown
AnnData object with n_obs × n_vars = 12964 × 544641
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id'
```

Example 29 (python):
```python
# modify modality columns in .obs
adata_paired.obs['modality'] = 'Multiome'
adata_paired.obs['modality'].value_counts()
```

Example 30 (unknown):
```unknown
Multiome    12964
Name: modality, dtype: int64
```

Example 31 (python):
```python
adata_paired.obs
```

Example 32 (python):
```python
adata_paired.var
```

Example 33 (python):
```python
adata_paired.X.data[:10]
```

Example 34 (unknown):
```unknown
array([1., 1., 1., 1., 1., 1., 2., 1., 2., 3.], dtype=float32)
```

Example 35 (python):
```python
# save
adata_paired.write(paired_adata_out_path)
print(f"saved to {paired_adata_out_path}")
```

Example 36 (unknown):
```unknown
saved to /lustre/scratch126/cellgen/team205/heart/objects/fetal/RNA/aCMs_AnyRegion_paired.h5ad
```

Example 37 (python):
```python
# If there is an object of different modality e.g. scRNAseq then make that too
if multiome_only:
    print("only making a single multiome object...")
else:
    print("making unimodality data too")
    modality_col="scRNA" # define what you want to go into modality column
    unimodality_adata_out_path=os.path.join("/lustre/scratch126/cellgen/team205/heart/objects/fetal/RNA",f"{job_name}_unimodality.h5ad")

    # select single-cell data (3p and 5p)
    adata_unimodality = adata_rna[adata_rna.obs['kit_10x'].isin(['3prime-v3','5prime-v2'])]
    adata_unimodality.obs['kit_10x'].value_counts()

    # make adata
    adata_unimodality=anndata.AnnData(
        X=adata_scrna.X, # concatenate sparse matrix and convert to Compressed Sparse Row format
        obs=adata_scrna.obs[obs_to_keep],
        var=adata_scrna.var[['modality']+RNA_var_to_keep]
    )

    # modify modality columns in .obs
    adata_unimodality.obs['modality'] = modality_col
    adata_unimodality.obs['modality'].value_counts()

    adata_unimodality.write(unimodality_adata_out_path)
    print(f"saved to {unimodality_adata_out_path}")
```

Example 38 (unknown):
```unknown
only making a single multiome object...
```

---

## Import module¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/ATAC/ATAC/post_peakVI_UMAPs.html

**Contents:**
- Import module¶
- Read in data¶
- Prepare color palette¶
- Plotting¶

**Examples:**

Example 1 (python):
```python
import numpy as np
import pandas as pd 
import scanpy as sc
import matplotlib.pyplot as plt
import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 
import seaborn as sns
# import sctk
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
-----
anndata             0.9.2
matplotlib          3.7.2
numpy               1.24.4
pandas              1.5.3
scanpy              1.9.3
seaborn             0.12.2
session_info        1.0.0
-----
```

Example 4 (python):
```python
PIL                         10.0.0
anyio                       NA
arrow                       1.2.3
asttokens                   NA
attr                        23.1.0
attrs                       23.1.0
babel                       2.12.1
backcall                    0.2.0
brotli                      1.0.9
certifi                     2023.11.17
cffi                        1.15.1
charset_normalizer          3.2.0
colorama                    0.4.6
comm                        0.1.4
cycler                      0.10.0
cython_runtime              NA
dateutil                    2.8.2
debugpy                     1.6.8
decorator                   5.1.1
defusedxml                  0.7.1
executing                   1.2.0
fastjsonschema              NA
fqdn                        NA
h5py                        3.9.0
idna                        3.4
igraph                      0.10.6
importlib_metadata          NA
importlib_resources         NA
ipykernel                   6.25.1
isoduration                 NA
jedi                        0.19.0
jinja2                      3.1.2
joblib                      1.3.2
json5                       NA
jsonpointer                 2.0
jsonschema                  4.19.0
jsonschema_specifications   NA
jupyter_events              0.7.0
jupyter_server              2.7.1
jupyterlab_server           2.24.0
kiwisolver                  1.4.5
leidenalg                   0.10.1
llvmlite                    0.40.1
markupsafe                  2.1.3
matplotlib_inline           0.1.6
mpl_toolkits                NA
natsort                     8.4.0
nbformat                    5.9.2
numba                       0.57.1
overrides                   NA
packaging                   23.1
parso                       0.8.3
patsy                       0.5.3
pexpect                     4.8.0
pickleshare                 0.7.5
pkg_resources               NA
plotly                      5.16.1
prometheus_client           NA
prompt_toolkit              3.0.39
psutil                      5.9.5
ptyprocess                  0.7.0
pure_eval                   0.2.2
pydev_ipython               NA
pydevconsole                NA
pydevd                      2.9.5
pydevd_file_utils           NA
pydevd_plugins              NA
pydevd_tracing              NA
pygments                    2.16.1
pyparsing                   3.0.9
pythonjsonlogger            NA
pytz                        2023.3
referencing                 NA
requests                    2.31.0
rfc3339_validator           0.1.4
rfc3986_validator           0.1.1
rpds                        NA
ruamel                      NA
scipy                       1.10.1
send2trash                  NA
six                         1.16.0
sklearn                     1.3.0
sniffio                     1.3.0
socks                       1.7.1
sphinxcontrib               NA
stack_data                  0.6.2
statsmodels                 0.14.0
texttable                   1.6.7
threadpoolctl               3.2.0
tornado                     6.3.3
traitlets                   5.9.0
typing_extensions           NA
uri_template                NA
urllib3                     2.0.4
wcwidth                     0.2.6
webcolors                   1.13
websocket                   1.6.2
yaml                        6.0.1
zipp                        NA
zmq                         25.1.1
zstandard                   0.19.0
```

Example 5 (unknown):
```unknown
-----
IPython             8.12.0
jupyter_client      8.3.0
jupyter_core        4.12.0
jupyterlab          4.0.5
notebook            7.0.2
-----
Python 3.8.17 (default, Jul  5 2023, 21:04:15) [GCC 11.2.0]
Linux-5.4.0-169-generic-x86_64-with-glibc2.17
-----
Session information updated at 2024-02-05 10:41
```

Example 6 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 7 (python):
```python
sc.settings.set_figure_params(dpi=120,dpi_save=300)
```

Example 8 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk'
```

Example 9 (python):
```python
adata = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/PeakMatrix/Foetal_Peaks_post-peakVI_umap.h5ad')
adata
```

Example 10 (unknown):
```unknown
AnnData object with n_obs × n_vars = 167022 × 508040
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', '_scvi_batch', '_scvi_labels', 'cluster_pvi'
    uns: '_scvi_manager_uuid', '_scvi_uuid', 'leiden', 'neighbors', 'umap'
    obsm: 'X_PeakVI', 'X_umap'
    obsp: 'connectivities', 'distances'
```

Example 11 (python):
```python
adata.obs['fine_grain'].cat.categories
```

Example 12 (unknown):
```unknown
Index(['InnateLymphoidCells', 'AtrialCardiomyocytesLeft',
       'AtrialCardiomyocytesRight', 'AtrialCardiomyocytesCycling',
       'VentricularCardiomyocytesLeftCompact',
       'VentricularCardiomyocytesRightCompact',
       'VentricularCardiomyocytesLeftTrabeculated',
       'VentricularCardiomyocytesRightTrabeculated',
       'VentricularCardiomyocytesCycling', 'SinoatrialNodePacemakerCells',
       'AtrioventriclarNodePacemakerCells',
       'VentricularConductionSystemProximal',
       'VentricularConductionSystemDistal',
       'GreatVesselAdventitialFibroblasts',
       'CoronaryVesselAdventitialFibroblasts',
       'MyocardialInterstitialFibroblasts', 'SubEpicardialFibroblasts',
       'Myofibroblasts', 'LymphNodeFibroblasticReticularCells',
       'ValveInterstitialCells', 'GreatVesselSmoothMuscleCells',
       'CoronarySmoothMuscleCells', 'DuctusArteriosusSmoothMuscleCells',
       'CoronaryPericytes', 'PericardialCellsIntermediate',
       'PericardialCellsFibrous', 'PericardialCellsParietal',
       'GreatVesselArterialEndothelialCells',
       'GreatVesselVenousEndothelialCells', 'CoronaryArterialEndothelialCells',
       'CoronaryVenousEndothelialCells', 'CoronaryCapillaryEndothelialCells',
       'EndocardialCells', 'EndocardialCushionCells', 'ValveEndothelialCells',
       'LymphaticEndothelialCells', 'MesothelialEpicardialCells',
       'EpicardiumDerivedCells', 'NeuronPrecursors', 'ChromaffinCells',
       'SympatheticNeurons', 'ParasympatheticNeurons', 'SchwannCellPrecursors',
       'SchwannCells', 'MonocytesMPOpos', 'Monocytes', 'MonocyteDerivedCells',
       'MacrophagesCX3CR1pos', 'MacrophagesTIMD4pos', 'MacrophagesLYVE1pos',
       'MacrophagesATF3pos', 'DendriticCellsType1', 'DendriticCellsMature',
       'PlasmacytoidDendriticCells', 'MastCells', 'Megakaryocytes',
       'TCellsCD4pos', 'TCellsCD8pos', 'TregsCD4pos', 'ProBCells', 'BCells',
       'BCellsMS4A1pos', 'NaturalKillerCells'],
      dtype='object')
```

Example 13 (python):
```python
FineGrainPalette={
    'InnateLymphoidCells':'#72497f', # since want to start the numbering from 1
    
    'AtrialCardiomyocytesLeft':'#eac075',
    'AtrialCardiomyocytesRight':'#4c6b8a',
    'AtrialCardiomyocytesCycling':'#e94bc7',
    'VentricularCardiomyocytesLeftCompact':'#ab1c3c',
    'VentricularCardiomyocytesRightCompact':'#fdba45',
    'VentricularCardiomyocytesLeftTrabeculated':'#2b9200',
    'VentricularCardiomyocytesRightTrabeculated':'#0273a8',
    'VentricularCardiomyocytesCycling':'#e1acff',
    'SinoatrialNodeCardiomyocytes':'#854834',
    'AtrioventricularNodeCardiomyocytes':'#688500',
    'VentricularConductionSystemProximal':'#01ccc8',
    'VentricularConductionSystemDistal':'#b58fab',
    'GreatVesselAdventitialFibroblasts':'#ff7e97',
    'CoronaryVesselAdventitialFibroblasts':'#ff9b6e',
    'MyocardialInterstitialFibroblasts1':'#8a6800',
    'MyocardialInterstitialFibroblasts2':'#206601',
    'Myofibroblasts':'#1bdebf',
    'LymphNodeFibroblasticReticularCells':'#005aa0',
    'ValveInterstitialCells':'#e478ff',
    'GreatVesselSmoothMuscleCells':'#ff665f',
    'CoronarySmoothMuscleCells':'#d2cb18',
    'DuctusArteriosusSmoothMuscleCells':'#83d7b3',
    'CoronaryPericytes':'#9088ff',
    'PericardialCells1':'#c95e00',
    'PericardialCells2':'#01e077',
    'PericardialCells3':'#ff88e4',
    'GreatVesselArterialEndothelialCells':'#d4004b',
    'GreatVesselVenousEndothelialCells':'#8f4300',
    'CoronaryArterialEndothelialCells':'#89da5a',
    'CoronaryVenousEndothelialCells':'#006f86',
    'CoronaryCapillaryEndothelialCells':'#aa7dff',
    'EndocardialCells':'#ff8241',
    'EndocardialCushionCells':'#46603a',
    'ValveEndothelialCells':'#414eb6',
    'LymphaticEndothelialCells':'#8b3f60',
    'MesothelialEpicardialCells':'#0295fe',
    'EpicardiumDerivedCells':'#ff92c4',
    'NeuronPrecursors':'#d30038',
    'ChromaffinCells':'#726800',
    'SympatheticNeurons':'#01894b',
    'ParaympatheticNeurons':'#545191',
    'SchwannCellPrecursors':'#b0d276',
    'SchwannCells':'#b30079',
    'MonocytesMPOpos':'#c50066',
    'Monocytes':'#fab4b6',
    'MonocyteDerivedCells':'#d14008',
    'MacrophagesCX3CR1pos':'#eabe97',
    'MacrophagesTIMD4pos':'#b2a200',
    'MacrophagesLYVE1pos':'#4a6005',
    'MacrophagesATF3pos':'#3db833',
    'DendriticCellsType1':'#007859',
    'DendriticCellsMature':'#01a9db',
    'PlasmacytoidDendriticCells':'#2c54ab',
    'MastCells':'#d0bef2',
    'Megakaryocytes':'#94298f',
    'TCellsCD4pos':'#ff387c',
    'TCellsCD8pos':'#9e3429',
    'TregsCD4pos':'#b37700',
    'ProBCells':'#8dab00',
    'BCells':'#a8d1a1',
    'BCellsMS4A1pos':'#00d2f0',
    'NaturalKillerCells':'#abb4ff',
}

FineGrainPalette_mod = {}
for i,celltype in enumerate(FineGrainPalette.keys()):
    if i < 10:
        celltype_mod = f'  {i}   {celltype}'
    else:
        celltype_mod = f' {i}   {celltype}'
    FineGrainPalette_mod[celltype_mod] = FineGrainPalette[celltype]
```

Example 14 (python):
```python
sc.pl.umap(adata, color=['mid_grain'], frameon=False)
```

Example 15 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 16 (python):
```python
sc.pl.umap(adata, color=['trimester'], frameon=False, show=False)
plt.savefig(f'{figdir}/Peaks_UMAP_trimesters.pdf', bbox_inches="tight",dpi=300)
```

Example 17 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

---

## Variables¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/Xenium/no1_per-sample-processing/Hst45-HEA-0-FFPE-1/no2_celltypist_prediction.html

**Contents:**
- Variables¶
- Read in data¶
- Normalise the counts¶
- CellTypist prediction: coarse-grain¶
- CellTypist prediction: mid-grain¶
- celltypist prediction: fine-grain¶
- Save¶

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os

import celltypist
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 4 (python):
```python
sample_id = 'Hst45-HEA-0-FFPE-1'
path_adata = f'/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium/{sample_id}_5K_filtered_raw.h5ad'
celltypist_model_dir = '/rfs/project/rfs-iCNyzSAaucw/kk837/notebooks/Foetal/Xenium/5K/celltypist_models'
```

Example 5 (unknown):
```unknown
os.getcwd()
```

Example 6 (python):
```python
adata = sc.read_h5ad(path_adata)
adata
```

Example 7 (python):
```python
sc.pp.filter_genes(adata, min_cells=3)
```

Example 8 (python):
```python
adata.shape
```

Example 9 (python):
```python
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)
```

Example 10 (python):
```python
adata.X.data[:5]
```

Example 11 (python):
```python
# coarse-grained
predictions_b2c = celltypist.annotate(adata, 
                                  model = f'{celltypist_model_dir}/global2coarse.pkl', 
                                  majority_voting = False)

# add to anndata
adata = predictions_b2c.to_adata(insert_labels = True, insert_conf = True)
adata.obs.rename(columns={'predicted_labels':'celltypist_coarse',
                              'conf_score':'conf_score_coarse'
                             },inplace=True)
adata.obs.head()
```

Example 12 (python):
```python
sc.pl.spatial(adata,color=['celltypist_coarse','conf_score_coarse'],cmap='RdPu',spot_size=20,vmax='p99',ncols=1)
```

Example 13 (python):
```python
# celltypist confidence
plt.figure(figsize=(6, 2))
sns.histplot(adata.obs[f'conf_score_coarse'], kde=False, bins=100)
# plt.yscale('log')
plt.title('coarse_grained')
plt.show()
```

Example 14 (python):
```python
# check
conf_thresh = 0.5 # starting from relaxed threshold
pd.crosstab(adata.obs['celltypist_coarse'],adata.obs[f'conf_score_coarse']<conf_thresh)
```

Example 15 (python):
```python
# finalised coarse-grained labels
conf_thresh = 0.5
adata.obs['celltypist_coarse_fin'] = adata.obs['celltypist_coarse'].astype('str').copy()
low_quality_cells = adata.obs_names[adata.obs[f'conf_score_coarse']<conf_thresh]
adata.obs.loc[low_quality_cells,'celltypist_coarse_fin'] = 'unassigned'
adata.obs['celltypist_coarse_fin'].value_counts()
```

Example 16 (python):
```python
sc.pl.spatial(adata,color=['celltypist_coarse_fin'],cmap='RdPu',spot_size=20,vmax='p99',ncols=1)
```

Example 17 (python):
```python
adata.obs['celltypist_coarse2midmod'] = np.nan
adata.obs['conf_score_coarse2midmod'] = np.nan

for celltype in adata.obs['celltypist_coarse_fin'].unique():
    if celltype not in ['unassigned', 'Epicardium']: # 'Epicardium' is also 'Epicardium' in mid-grained
        print(celltype)
        # subset
        adata_sub = adata[adata.obs['celltypist_coarse_fin']==celltype]
        # predict
        predictions = celltypist.annotate(adata_sub, 
                                      model = f'{celltypist_model_dir}/coarse2midmod_{celltype}.pkl', 
                                      majority_voting = False)
        # add to anndata
        adata_sub = predictions.to_adata(insert_labels = True, insert_conf = True)
        adata.obs.loc[adata_sub.obs_names,'celltypist_coarse2midmod'] = adata_sub.obs['predicted_labels']
        adata.obs.loc[adata_sub.obs_names,'conf_score_coarse2midmod'] = adata_sub.obs['conf_score']
        print('')
```

Example 18 (python):
```python
sc.pl.spatial(adata,color=['celltypist_coarse2midmod','conf_score_coarse2midmod'],
              cmap='RdPu',spot_size=20,vmax='p99',ncols=1)
```

Example 19 (python):
```python
# check
conf_thresh = 0.8
pd.crosstab(adata.obs['celltypist_coarse2midmod'],adata.obs[f'conf_score_coarse2midmod']<conf_thresh)
```

Example 20 (python):
```python
# finalised mid-grained labels
conf_thresh = 0.8
adata.obs['celltypist_mid_fin'] = adata.obs['celltypist_coarse_fin'].astype('str').copy()
adata.obs.replace({'celltypist_mid_fin':{'Epicardium':'EpicardialCells'}},inplace=True)
passed = adata.obs_names[adata.obs['conf_score_coarse2midmod']>=conf_thresh]
low_quality_cells = adata.obs_names[adata.obs[f'conf_score_coarse2midmod']<conf_thresh]

adata.obs.loc[passed,'celltypist_mid_fin'] = adata.obs.loc[passed,'celltypist_coarse2midmod'].copy()
adata.obs.loc[low_quality_cells,'celltypist_mid_fin'] = 'unassigned'
adata.obs['celltypist_mid_fin'].value_counts()
```

Example 21 (python):
```python
sc.pl.spatial(adata,color=['celltypist_mid_fin'],
              cmap='RdPu',spot_size=20,vmax='p99',ncols=1)
```

Example 22 (python):
```python
adata.obs['celltypist_midmod2fine'] = np.nan
adata.obs['conf_score_midmod2fine'] = np.nan

for celltype in adata.obs['celltypist_mid_fin'].unique():
    if celltype not in ['unassigned', 'LymphaticEndothelialCells']: # 'LymphaticEndothelialCells' is also 'LymphaticEndothelialCells' in fine-grained
        print(celltype)
        # subset
        adata_sub = adata[adata.obs['celltypist_mid_fin']==celltype]
        # predict
        predictions = celltypist.annotate(adata_sub, 
                                      model = f'{celltypist_model_dir}/midmod2fine_{celltype}.pkl', 
                                      majority_voting = False)
        # add to anndata
        adata_sub = predictions.to_adata(insert_labels = True, insert_conf = True)
        adata.obs.loc[adata_sub.obs_names,'celltypist_midmod2fine'] = adata_sub.obs['predicted_labels']
        adata.obs.loc[adata_sub.obs_names,'conf_score_midmod2fine'] = adata_sub.obs['conf_score']
        print('')
```

Example 23 (python):
```python
sc.pl.spatial(adata,color=['celltypist_midmod2fine','conf_score_midmod2fine'],
              cmap='RdPu',spot_size=20,vmax='p99',ncols=1)
```

Example 24 (python):
```python
sc.pl.spatial(adata,color=['celltypist_midmod2fine'],
              groups=['SinoatrialNodePacemakerCells','AtrioventricularNodePacemakerCells'],
              cmap='RdPu',spot_size=20,vmax='p99',ncols=3)
```

Example 25 (python):
```python
sc.pl.spatial(adata,color=['celltypist_midmod2fine'],
              groups=['MacrophagesCX3CR1pos','MacrophagesLYVE1pos'],
              cmap='RdPu',spot_size=20,vmax='p99',ncols=3)
```

Example 26 (python):
```python
adata.write(f'/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium/{sample_id}_5K_filtered_lognorm.h5ad')
```

Example 27 (python):
```python
print(f'/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium/{sample_id}_5K_filtered_lognorm.h5ad')
```

---

## Import libraries¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig4_Cardiomyocytes/AtrialCardiomyocytes/CellOracle/CellOracle_no2_in-silico-perturbation_AtrialCM-all_out_12May2025.html

**Contents:**
- Import libraries¶
- Make out directory¶
- Load oracle object¶
- In silico TF perturbation analysis¶
- Set parameters¶
- Calculating pseudotime gradient vectors¶
- Perturbation¶

ref: https://morris-lab.github.io/CellOracle.documentation/notebooks/04_Network_analysis/Network_analysis_with_Paul_etal_2015_data.html

**Examples:**

Example 1 (unknown):
```unknown
# Parameters
# working_dir = "/nfs/team205/heart/Vitro/TF_selection/foetal_vCM_left_compact-vs-trabeculated_downstream/CO_in-silico-perturbation_papermill"
path_to_oracle_object = "/nfs/team205/heart/anndata_objects/Foetal/celloracle/foetal_AtrialCM.celloracle.oracle"
path_to_oracle_links = "/nfs/team205/heart/anndata_objects/Foetal/celloracle/foetal_AtrialCM_links.celloracle.links"

# path_to_TFs = 

cell_group_name = "AtrialCM"

path_to_res_dict = f"/nfs/team205/heart/anndata_objects/Foetal/celloracle/{cell_group_name}_in-silico_perturbation_May2025.pkl"
path_to_plots = f"/nfs/team205/heart/anndata_objects/Foetal/celloracle/figures_pm/{cell_group_name}_May2025"
```

Example 2 (python):
```python
import os
import sys

import matplotlib
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import scanpy as sc
import seaborn as sns
```

Example 3 (python):
```python
import celloracle as co
co.__version__
```

Example 4 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 5 (python):
```python
sc.settings.set_figure_params(dpi=120)
```

Example 6 (python):
```python
if not os.path.exists(path_to_plots):
    # Create the directory
    os.makedirs(path_to_plots)
    print("Directory created successfully")
else:
    print("Directory already exists")
```

Example 7 (unknown):
```unknown
Directory already exists
```

Example 8 (unknown):
```unknown
# Load
oracle = co.load_hdf5(path_to_oracle_object)
```

Example 9 (python):
```python
n_cell = oracle.adata.shape[0]
print(f"cell number is :{n_cell}")

k = int(0.025*n_cell)
print(f"Auto-selected k is :{k}")
```

Example 10 (unknown):
```unknown
cell number is :5664
Auto-selected k is :141
```

Example 11 (python):
```python
%matplotlib inline
# Check clustering data
sc.pl.umap(oracle.adata, color='fine_grain_time-binned')
```

Example 12 (python):
```python
# read in post-g2g-selection csv file and get selected TFs
# log2FC<0 means TFs specific to Right-aCM
# log2FC>0 means TFs specific to Left-aCM
g2g_sel = pd.read_csv('/nfs/team205/kk18/notebooks/Foetal/AtrialCM_celloracle/AtrialCM_post-g2g_selected-recgulons.csv')
display(g2g_sel)

# TFs, only activator
list_of_TFs = [x.split('_')[0] for x in g2g_sel['Gene'] if '_+' in x]
print(len(list_of_TFs))
```

Example 13 (python):
```python
# select TFs which are in the oracle object
list_of_TFs = [x for x in list_of_TFs if x in oracle.adata.var_names]

print(len(list_of_TFs))
list_of_TFs[:5]
```

Example 14 (unknown):
```unknown
['ETS2', 'MAF', 'IKZF2', 'ETV6', 'BHLHE41']
```

Example 15 (python):
```python
# Check gene expressions
# will be saved to a separated file
sc.settings.set_figure_params(dpi=80)
sc.pl.umap(oracle.adata, color=list_of_TFs, ncols=6,
                 layer="imputed_count", use_raw=False, cmap="RdPu",
                 # show=False, save='_TF_expression.png'
                )
```

Example 16 (unknown):
```unknown
n_grid = 40
n_neighbors = 100
min_mass = 10
scale_simulation = 5
```

Example 17 (python):
```python
from celloracle.applications import Gradient_calculator
# Instantiate Gradient calculator object
gradient = Gradient_calculator(oracle_object=oracle, pseudotime_key="t")
gradient.calculate_p_mass(smooth=0.8, n_grid=n_grid, n_neighbors=n_neighbors)
gradient.calculate_mass_filter(min_mass=min_mass, plot=True)

# convert the pseudotime data into grid points.
# For this calculation we can chose one of two methods.
gradient.transfer_data_into_grid(args={"method": "knn", "n_knn":50})

# Calculate graddient
gradient.calculate_gradient()
# Show results
scale_dev = 40
gradient.visualize_results(scale=scale_dev, s=5)
# Visualize results
fig, ax = plt.subplots(figsize=[6, 6])
gradient.plot_dev_flow_on_grid(scale=scale_dev, ax=ax)
```

Example 18 (python):
```python
from tqdm import tqdm
```

Example 19 (python):
```python
from celloracle.applications import Oracle_development_module

# not showing plots
matplotlib.use('Agg')

dev_dict = {}
for goi in tqdm(list_of_TFs):
    # Enter perturbation conditions to simulate signal propagation after the perturbation.
    try:
        oracle.simulate_shift(perturb_condition={goi: 0.0}, # this is KO simulation so expression=0
                          n_propagation=3)
        print(f'simulating {goi}')
    except:
        print(f'skipping {goi} due to an error')
        continue
    
    # Get transition probability
    oracle.estimate_transition_prob(n_neighbors=n_neighbors,
                                    knn_random=True,
                                    sampled_fraction=1)

    # Calculate embedding
    oracle.calculate_embedding_shift(sigma_corr=0.05)
    
    # prepare grid
    oracle.calculate_p_mass(smooth=0.8, n_grid=n_grid, n_neighbors=n_neighbors)
    oracle.calculate_mass_filter(min_mass=min_mass, plot=False)
    
    # plot vector field
    fig, ax = plt.subplots(1, 2,  figsize=[13, 6])
    # Show quiver plot
    oracle.plot_simulation_flow_on_grid(scale=scale_simulation, ax=ax[0])
    ax[0].set_title(f"Simulated cell identity shift vector: {goi} KO")
    # Show quiver plot that was calculated with randomized graph.
    oracle.plot_simulation_flow_random_on_grid(scale=scale_simulation, ax=ax[1])
    ax[1].set_title(f"Randomized simulation vector")
    plt.savefig(f'{path_to_plots}/{goi}_simulation_vector.pdf')
    
    # Plot vector field with cell cluster
    fig, ax = plt.subplots(figsize=[8, 8])
    oracle.plot_cluster_whole(ax=ax, s=10)
    oracle.plot_simulation_flow_on_grid(scale=scale_simulation, ax=ax, show_background=False)
    plt.savefig(f'{path_to_plots}/{goi}_simulation_vector_with-clusters.pdf')
    
    
    dev = Oracle_development_module()
    # Load development flow
    dev.load_differentiation_reference_data(gradient_object=gradient)

    # Load simulation result
    dev.load_perturb_simulation_data(oracle_object=oracle,
                                     cell_idx_use=None, # Enter cell id list
                                     name=cell_group_name # Name of this cell group. You can enter any name.
                                     )

    # Calculation
    dev.calculate_inner_product()
    dev.calculate_digitized_ip(n_bins=10)
    
    # save to the dictionary
    dev_dict[goi] = dev.copy()
    
    # Let's visualize the results
    vm = 0.01
    dev.visualize_development_module_layout_0(s=5,
                                              scale_for_simulation=scale_simulation,
                                              s_grid=50,
                                              scale_for_pseudotime=scale_dev,
                                              vm=vm)
    plt.savefig(f'{path_to_plots}/{goi}_results.pdf')
```

Example 20 (unknown):
```unknown
0%|          | 0/9 [00:00<?, ?it/s]
```

Example 21 (unknown):
```unknown
simulating ETS2
```

Example 22 (unknown):
```unknown
2025-05-12 11:57:36,089 - INFO - Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
2025-05-12 11:57:36,112 - INFO - Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
 11%|█         | 1/9 [05:46<46:14, 346.82s/it]
```

Example 23 (unknown):
```unknown
simulating MAF
```

Example 24 (unknown):
```unknown
2025-05-12 12:03:27,557 - INFO - Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
2025-05-12 12:03:27,574 - INFO - Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
 22%|██▏       | 2/9 [11:38<40:46, 349.51s/it]
```

Example 25 (unknown):
```unknown
simulating IKZF2
```

Example 26 (unknown):
```unknown
2025-05-12 12:09:36,927 - INFO - Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
2025-05-12 12:09:36,939 - INFO - Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
 33%|███▎      | 3/9 [17:47<35:51, 358.54s/it]
```

Example 27 (unknown):
```unknown
simulating ETV6
```

Example 28 (unknown):
```unknown
2025-05-12 12:15:33,121 - INFO - Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
2025-05-12 12:15:33,139 - INFO - Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
 44%|████▍     | 4/9 [23:43<29:48, 357.62s/it]
```

Example 29 (unknown):
```unknown
simulating BHLHE41
```

Example 30 (unknown):
```unknown
2025-05-12 12:21:18,159 - INFO - Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
2025-05-12 12:21:18,185 - INFO - Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
 56%|█████▌    | 5/9 [29:29<23:32, 353.20s/it]
```

Example 31 (unknown):
```unknown
simulating GTF2IRD1
```

Example 32 (unknown):
```unknown
2025-05-12 12:27:09,019 - INFO - Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
2025-05-12 12:27:09,033 - INFO - Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
 67%|██████▋   | 6/9 [35:19<17:36, 352.29s/it]
```

Example 33 (unknown):
```unknown
simulating GATA5
```

Example 34 (unknown):
```unknown
2025-05-12 12:31:14,606 - INFO - Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
2025-05-12 12:31:14,617 - INFO - Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
 78%|███████▊  | 7/9 [39:25<10:34, 317.39s/it]
```

Example 35 (unknown):
```unknown
simulating PITX2
```

Example 36 (unknown):
```unknown
2025-05-12 12:35:16,104 - INFO - Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
2025-05-12 12:35:16,115 - INFO - Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
 89%|████████▉ | 8/9 [43:26<04:53, 293.21s/it]
```

Example 37 (unknown):
```unknown
simulating SIX5
```

Example 38 (unknown):
```unknown
2025-05-12 12:39:20,904 - INFO - Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
2025-05-12 12:39:20,928 - INFO - Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
100%|██████████| 9/9 [47:31<00:00, 316.84s/it]
```

Example 39 (unknown):
```unknown
# save
utils.save_pkl(dev_dict, path_to_res_dict)
```

---

## Import modules¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/Xenium/no2_integration_and_cell-type-annotation/Xenium5K_corr_with_scnRNAseq.html

**Contents:**
- Import modules¶
- Read in qeury adata¶
- Read in reference adata¶
- Harmonise cell type names¶
- Prepare celltype-by-gene dataframes¶

**Examples:**

Example 1 (python):
```python
import numpy as np
import pandas as pd
import scanpy as sc
import anndata
import os
import matplotlib.pyplot as plt
import seaborn as sns
import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 

sc.settings.verbosity = 1  # verbosity: errors (0), warnings (1), info (2), hints (3)
```

Example 2 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/rfs/project/rfs-iCNyzSAaucw/kk837/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 3 (unknown):
```unknown
figdir = '/rfs/project/rfs-iCNyzSAaucw/kk837/notebooks/Foetal/fetal_ms_figures'
# table_dir = '/nfs/team205/heart/fetal_ms_figures/tables/Aug2024'
```

Example 4 (python):
```python
# read in 
objects_dir = '/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium'
path_adata = f'{objects_dir}/C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat_5K_filtered_raw.h5ad'
adata = sc.read_h5ad(path_adata)
adata
```

Example 5 (unknown):
```unknown
AnnData object with n_obs × n_vars = 362277 × 5001
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_3.0', 'multi_celltypes_coarse', 'fine_grain_12Jun2025'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_mid_fin_colors', 'dendrogram_pca_harmony_leiden_1.0', 'fine_grain_12Jun2025_colors', 'multi_celltypes_coarse_colors', 'pca', 'pca_harmony', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.2_colors', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.3_colors', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.4_colors', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.5_colors', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_0.8_colors', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.0_colors', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_1.5_colors', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_2.0_colors', 'pca_harmony_leiden_3.0', 'pca_harmony_leiden_3.0_colors', 'tissue_block_id_colors', 'umap'
    obsm: 'X_pca', 'X_pca_harmony', 'X_umap_pca_harmony', 'spatial'
    varm: 'PCs'
    obsp: 'pca_harmony_connectivities', 'pca_harmony_distances'
```

Example 6 (unknown):
```unknown
celltype_col = 'fine_grain_12Jun2025'
```

Example 7 (python):
```python
# log-normalise the count
print(adata.X.data[:5])
sc.pp.filter_genes(adata, min_cells=3)
sc.pp.normalize_total(adata,target_sum=1e4)
sc.pp.log1p(adata)
print(adata.X.data[:5])
```

Example 8 (unknown):
```unknown
[5. 1. 1. 2. 2.]
[4.2093534 2.6576407 2.6576407 3.3151019 3.3151019]
```

Example 9 (python):
```python
# update cell type name for plotting
df = pd.read_csv('/rfs/project/rfs-iCNyzSAaucw/kk837/notebooks/Foetal/finegrain_name_mapping.csv')
celltype_mapping = df.set_index('Full_name')['Short_name_finalised'].to_dict()
adata.obs.replace({celltype_col:celltype_mapping},inplace=True)
```

Example 10 (python):
```python
/tmp/ipykernel_3438200/966943319.py:4: FutureWarning: The behavior of Series.replace (and DataFrame.replace) with CategoricalDtype is deprecated. In a future version, replace will only be used for cases that preserve the categories. To change the categories, use ser.cat.rename_categories instead.
  adata.obs.replace({celltype_col:celltype_mapping},inplace=True)
```

Example 11 (python):
```python
# remove doublets and non-cardiac cells
mask = [x in celltype_mapping.values() for x in adata.obs[celltype_col]]
adata = adata[mask]
```

Example 12 (python):
```python
adata_ref = sc.read_h5ad('/rfs/project/rfs-iCNyzSAaucw/kk837/data_objects/Foetal/RNA/Feb28ObjectRaw_finegrain_updated.h5ad')
adata_ref
```

Example 13 (unknown):
```unknown
AnnData object with n_obs × n_vars = 297473 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score'
    var: 'gene_name_scRNA-0', 'gene_id'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'age_colors', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'cycling_colors', 'donor_colors', 'fine_grain_colors', 'heart_or_greatvessels_colors', 'hvg', 'kit_10x_colors', 'mid_grain_colors', 'modality_colors', 'neighbors', 'phase_colors', 'region_colors', 'sex_colors', 'trimester_colors', 'umap', 'week_colors'
    obsm: 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 14 (python):
```python
# subset genes and log-normalise
adata_ref = adata_ref[:,adata.var_names]
print(adata_ref.X.data[:5])
sc.pp.normalize_total(adata_ref,target_sum=1e4)
sc.pp.log1p(adata_ref)
print(adata_ref.X.data[:5])
```

Example 15 (unknown):
```unknown
[1. 1. 3. 4. 1.]
```

Example 16 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_normalization.py:216: UserWarning: Received a view of an AnnData. Making a copy.
  view_to_actual(adata)
```

Example 17 (unknown):
```unknown
[0.44856235 0.44856235 0.9925765  1.1830258  0.44856235]
```

Example 18 (python):
```python
# update cell type name
df = pd.read_csv('/rfs/project/rfs-iCNyzSAaucw/kk837/notebooks/Foetal/finegrain_name_mapping.csv')
celltype_mapping = df.set_index('Full_name')['Short_name_finalised'].to_dict()
adata_ref.obs.replace({'fine_grain':celltype_mapping},inplace=True)
```

Example 19 (python):
```python
/tmp/ipykernel_3438200/3465093248.py:4: FutureWarning: The behavior of Series.replace (and DataFrame.replace) with CategoricalDtype is deprecated. In a future version, replace will only be used for cases that preserve the categories. To change the categories, use ser.cat.rename_categories instead.
  adata_ref.obs.replace({'fine_grain':celltype_mapping},inplace=True)
```

Example 20 (python):
```python
celltype_mapping_2 = {
    'BMS4':'B',
    'CorAdvFB':'FB',
    'GVAdvFB':'FB',
    'MonoMPO':'Mono',
    'MyoFB':'FB',
    'MyoFb':'FB',
     'ProB':'B',
    'SubEpiFb':'FB',
    'vCMCL':'vCM',
    'vCMCR':'vCM',
    'vCMTL':'vCM',
    'vCMTR':'vCM',
    'FBCOL2A1':'FB',
    'GVSMCP':'GVSMC',
    'GVSMCT':'GVSMC',
    'SANPCHd':'SANPC',
     'SANPCHn':'SANPC',
     'SANPCT':'SANPC',
    'CorCapECN':'CorCapEC',
    'DCMat':'DC',
 'DCT1':'DC',
}
adata.obs[celltype_col] = [celltype_mapping_2[x] if x in celltype_mapping_2.keys() else x for x in adata.obs[celltype_col]]
adata_ref.obs['fine_grain'] = [celltype_mapping_2[x] if x in celltype_mapping_2.keys() else x for x in adata_ref.obs['fine_grain']]
```

Example 21 (python):
```python
/tmp/ipykernel_3438200/752604715.py:24: ImplicitModificationWarning: Trying to modify attribute `.obs` of view, initializing view as actual.
  adata.obs[celltype_col] = [celltype_mapping_2[x] if x in celltype_mapping_2.keys() else x for x in adata.obs[celltype_col]]
```

Example 22 (python):
```python
adata_agg = sc.get.aggregate(adata, by=celltype_col, func=["mean"])
adata_agg.X = adata_agg.layers['mean']
query_df = adata_agg.to_df()
print(f'original query df, shape: {query_df.shape}')

adata_ref_agg = sc.get.aggregate(adata_ref, by='fine_grain', func=["mean"])
adata_ref_agg.X = adata_ref_agg.layers['mean']
ref_df = adata_ref_agg.to_df()
print(f'original ref_df, shape: {ref_df.shape}')

shared_celltypes = list(set(query_df.index).intersection(ref_df.index))
query_df = query_df.loc[shared_celltypes].T
ref_df = ref_df.loc[shared_celltypes].T
print(f'post-subset query df, shape: {query_df.shape}')
print(f'post-subset ref_df, shape: {ref_df.shape}')
```

Example 23 (unknown):
```unknown
original query df, shape: (51, 5001)
original ref_df, shape: (52, 5001)
post-subset query df, shape: (5001, 42)
post-subset ref_df, shape: (5001, 42)
```

Example 24 (python):
```python
%%time
figsize = (10,10)
cmap = 'inferno'
corr, pval = utils.df_pearson_corr_with_p_vectorized(query_df, ref_df, p_adjust='fdr_bh')

# draw the initial clustermap (for ordering only)
cg = sns.clustermap(
    corr,
    cmap=cmap,
    # center=0,
    metric='euclidean',
    method='average',
    figsize=figsize
)

# 4) extract the leaf order and reindex both matrices
order = cg.dendrogram_row.reordered_ind
labels = list(corr.index[order])

corr_ord = corr.loc[labels, labels]
pval_ord = pval.loc[labels, labels]

# 5) replot as a heatmap and annotate significance
fig, ax = plt.subplots(figsize=figsize)
sns.heatmap(
    corr_ord,
    cmap=cmap,
    # center=0,
    square=True,
    cbar_kws={'label': 'Pearson r'},
    ax=ax
)

'''
# add asterisks for p < 0.01
n = corr_ord.shape[0]
for i in range(n):
    for j in range(n):
        if pval_ord.iat[i, j] < 0.0001:
            ax.text(j + 0.5, i + 0.5, '*',
                    ha='center', va='center',
                    fontsize=10)
'''

ax.set_xticklabels(labels, rotation=90)
ax.set_xlabel('scnRNA-seq')
ax.set_yticklabels(labels, rotation=0)
ax.set_ylabel('Xenium5K')
plt.tight_layout()
plt.savefig(f'{figdir}/Xenium5K-vs-scnRNAseq_celltype-gex-corr.pdf', bbox_inches="tight",dpi=300)
plt.show()
```

Example 25 (unknown):
```unknown
CPU times: user 2.05 s, sys: 252 ms, total: 2.3 s
Wall time: 2.54 s
```

---

## Import modules¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig5_Trisomy21/no02_t21_post-scVI_cleanup.html

**Contents:**
- Import modules¶
- Read in data¶
- Filter and log-normalise¶
- CellTypist prediction¶
- Select cells to remove¶
- Remove from the raw data¶
- recalculate UMAP, etc¶
- Modify obs¶
- CellTypist prediction for fine-grained labels (run per coarse-grained)¶

"Pericardium" region is not represented in "Heart" and other region samples. In addition, "Pericardium" sample is coming from one donor "Pericardium" will be removed at downstream analysis

**Examples:**

Example 1 (python):
```python
import numpy as np
import numpy.random as random
import pandas as pd
import scanpy as sc
import matplotlib.pyplot as plt
import seaborn as sns
import celltypist
```

Example 2 (python):
```python
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm
```

Example 3 (python):
```python
import session_info
session_info.show()
```

Example 4 (python):
```python
-----
celltypist          1.6.2
matplotlib          3.7.2
numpy               1.22.4
pandas              1.5.3
scanpy              1.9.3
seaborn             0.12.2
session_info        1.0.0
-----
```

Example 5 (python):
```python
PIL                         10.0.0
anndata                     0.9.2
anyio                       NA
arrow                       1.2.3
asciitree                   NA
asttokens                   NA
attr                        23.1.0
attrs                       23.1.0
babel                       2.12.1
backcall                    0.2.0
brotli                      1.0.9
certifi                     2023.11.17
cffi                        1.15.1
charset_normalizer          3.2.0
cloudpickle                 3.0.0
colorama                    0.4.6
comm                        0.1.4
cycler                      0.10.0
cython_runtime              NA
dask                        2023.5.0
dateutil                    2.8.2
debugpy                     1.6.8
decorator                   5.1.1
defusedxml                  0.7.1
executing                   1.2.0
fasteners                   0.19
fastjsonschema              NA
fqdn                        NA
h5py                        3.9.0
idna                        3.4
igraph                      0.10.6
importlib_metadata          NA
importlib_resources         NA
ipykernel                   6.25.1
isoduration                 NA
jedi                        0.19.0
jinja2                      3.1.2
joblib                      1.3.2
json5                       NA
jsonpointer                 2.0
jsonschema                  4.19.0
jsonschema_specifications   NA
jupyter_events              0.7.0
jupyter_server              2.7.1
jupyterlab_server           2.24.0
kiwisolver                  1.4.5
leidenalg                   0.10.1
llvmlite                    0.38.1
markupsafe                  2.1.3
mpl_toolkits                NA
natsort                     8.4.0
nbformat                    5.9.2
numba                       0.55.2
numcodecs                   0.12.1
opt_einsum                  v3.3.0
overrides                   NA
packaging                   23.1
parso                       0.8.3
patsy                       0.5.3
pexpect                     4.8.0
pickleshare                 0.7.5
pkg_resources               NA
plotly                      5.16.1
prometheus_client           NA
prompt_toolkit              3.0.39
psutil                      5.9.5
ptyprocess                  0.7.0
pure_eval                   0.2.2
pyarrow                     15.0.2
pydev_ipython               NA
pydevconsole                NA
pydevd                      2.9.5
pydevd_file_utils           NA
pydevd_plugins              NA
pydevd_tracing              NA
pygments                    2.16.1
pyparsing                   3.0.9
pythonjsonlogger            NA
pytz                        2023.3
referencing                 NA
requests                    2.31.0
rfc3339_validator           0.1.4
rfc3986_validator           0.1.1
rpds                        NA
ruamel                      NA
scipy                       1.10.1
send2trash                  NA
six                         1.16.0
sklearn                     1.3.0
sniffio                     1.3.0
socks                       1.7.1
sphinxcontrib               NA
stack_data                  0.6.2
statsmodels                 0.14.0
texttable                   1.6.7
threadpoolctl               3.2.0
tlz                         0.12.1
toolz                       0.12.1
torch                       1.13.1+cu117
tornado                     6.3.3
tqdm                        4.66.1
traitlets                   5.9.0
typing_extensions           NA
uri_template                NA
urllib3                     2.0.4
wcwidth                     0.2.6
webcolors                   1.13
websocket                   1.6.2
yaml                        6.0.1
zarr                        2.16.1
zipp                        NA
zmq                         25.1.1
zstandard                   0.19.0
```

Example 6 (unknown):
```unknown
-----
IPython             8.12.0
jupyter_client      8.3.0
jupyter_core        4.12.0
jupyterlab          4.0.5
notebook            7.0.2
-----
Python 3.8.17 (default, Jul  5 2023, 21:04:15) [GCC 11.2.0]
Linux-5.15.0-113-generic-x86_64-with-glibc2.17
-----
Session information updated at 2024-08-01 08:21
```

Example 7 (python):
```python
# import own function
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 8 (python):
```python
sc.settings.verbosity = 3             # verbosity: errors (0), warnings (1), info (2), hints (3)
sc.settings.set_figure_params(dpi=120)
```

Example 9 (python):
```python
adata = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/trisomy21/T21Hearts_July2024.h5ad')
adata
```

Example 10 (unknown):
```unknown
AnnData object with n_obs × n_vars = 121452 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'batch_key', 'leiden_scVI_0.1', 'leiden_scVI_0.2', 'leiden_scVI_0.3', 'leiden_scVI_0.5', 'leiden_scVI_0.7', 'leiden_scVI_1'
    var: 'gene_name', 'feature_type', 'ambient_expression-0', 'ambient_expression-1', 'ambient_expression-10', 'ambient_expression-11', 'ambient_expression-12', 'ambient_expression-13', 'ambient_expression-14', 'ambient_expression-2', 'ambient_expression-3', 'ambient_expression-4', 'ambient_expression-5', 'ambient_expression-6', 'ambient_expression-7', 'ambient_expression-8', 'ambient_expression-9', 'mt', 'ribo', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts'
    uns: 'age_colors', 'donor_colors', 'leiden', 'leiden_scVI_0.1_colors', 'leiden_scVI_0.2_colors', 'leiden_scVI_0.3_colors', 'leiden_scVI_0.5_colors', 'leiden_scVI_0.7_colors', 'leiden_scVI_1_colors', 'neighbors', 'region_colors', 'sangerID_colors', 'umap'
    obsm: 'X_scVI', 'X_umap', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 11 (python):
```python
adata.var['gene_id'] = adata.var_names.copy()
adata.var.set_index('gene_name',inplace=True)
adata.var_names = adata.var_names.astype('str')
adata.var_names_make_unique()
```

Example 12 (python):
```python
sc.pp.filter_genes(adata, min_cells=3)
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)
```

Example 13 (unknown):
```unknown
filtered out 3535 genes that are detected in less than 3 cells
normalizing counts per cell
    finished (0:00:01)
```

Example 14 (python):
```python
print(adata.shape)
adata.X.data[:5]
```

Example 15 (unknown):
```unknown
(121452, 33066)
```

Example 16 (unknown):
```unknown
array([0.0534004 , 0.10409318, 0.19836526, 0.0534004 , 0.15233971],
      dtype=float32)
```

Example 17 (python):
```python
# coarse-grained
predictions = celltypist.annotate(adata, 
                                  model = '/nfs/team205/kk18/notebooks/Foetal/VisiumHD/mpp0p3/celltypist_models/from_foetal_portal/foetal_heart_coarse.pkl', 
                                  majority_voting = False)

# add to anndata
adata = predictions.to_adata(insert_labels = True, insert_conf = True)
adata.obs.rename(columns={'predicted_labels':'celltypist_coarse',
                              'conf_score':'conf_score_coarse'
                             },inplace=True)
adata.obs.head()
```

Example 18 (unknown):
```unknown
🔬 Input data has 121452 cells and 33066 genes
🔗 Matching reference genes in the model
🧬 29480 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
```

Example 19 (python):
```python
sc.pl.umap(adata, color=['celltypist_coarse','conf_score_coarse','leiden_scVI_1'], size=1, legend_fontsize=6, wspace=0.2)
```

Example 20 (python):
```python
# umap per celltype to highlight
group_col = 'region'
groups_to_plot = ['Heart',
                  'Great vessels', 
                 'Left atria',
                 'Left ventricle',
                 'Pericardial sac + RA + LA + IAS',
                 'Right ventricle',
                  'Pericardium',
                 ]

plt.rcParams['axes.grid'] = False
fig,axs=plt.subplots(nrows=2,ncols=4,figsize=(15,7))
# Flatten the axs array for easier iteration in case of multiple rows and columns
axs_flat = axs.flatten()

for i,grp in enumerate(groups_to_plot):
    sc.pl.umap(adata, color=[group_col], groups=grp, 
               size=10, frameon=False,
               show=False, ax=axs_flat[i], legend_loc=None, legend_fontsize=5,
               title=grp
              )
# Turn off any remaining subplots
for j in range(len(groups_to_plot), len(axs_flat)):
    fig.delaxes(axs_flat[j])

# fig.suptitle('Main Title for the Figure')
fig.tight_layout()
```

Example 21 (python):
```python
# mean condidence score
group_col = 'leiden_scVI_1'
conf_score_col = 'conf_score_coarse'
df = adata.obs[[group_col,conf_score_col]].groupby(group_col).mean()
df.sort_values(conf_score_col,ascending=False)
```

Example 22 (python):
```python
# cell type marker expression
markers = {
    'vCM': ['MYH7', 'MYL2', 'FHL2'],
    'aCM': ['NPPA', 'MYL7', 'MYL4'],
    'FB': ['DCN', 'GCN', 'PDGFRA'],
    'EC': ['VWF', 'PECAM1', 'CDH5'],
    'Peri': ['RGS5', 'ABCC9', 'KCNJ8'],
    'SMC': ['MYH11', 'TAGLN', 'ACTA2'],
    'Meso': ['MSLN', 'WT1', 'BNC1'],
    'Neuro': ['PLP1', 'NRXN1', 'NRXN3'],
    'Adipo': ['GPAM', 'FASN', 'LEP'],
    'Myelo': ['CD14', 'C1QA', 'CD68','LYVE1','TIMD4'],
    'Lymph': ['CD8A', 'IL7R', 'CD40LG'],
    'B': ['CD19','SDC1','IGHG1','IGHA1'],
    'Mast': ['KIT', 'CPA3'],
}

for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in adata.var_names]

sc.tl.dendrogram(adata, groupby=group_col)
sc.pl.dotplot(adata,
              markers, 
              groupby=group_col,
              dendrogram=True,
              standard_scale="var",
              color_map="Reds",
              # swap_axes=True,
             )
```

Example 23 (unknown):
```unknown
using 'X_pca' with n_pcs = 50
Storing dendrogram info using `.uns['dendrogram_leiden_scVI_1']`
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: vCM, aCM, FB, etc.
```

Example 24 (python):
```python
# sangerID proportion
crosstab=pd.crosstab(adata.obs[group_col],adata.obs["region"])
# Normalize each row to sum to 1 (i.e., to get fractions)
crosstab_normalized = crosstab.div(crosstab.sum(axis=1), axis=0)

# Plotting the stacked bar chart
ax = crosstab_normalized.plot(kind='bar',stacked=True,figsize=(10, 4),legend=False)

# Adding labels and title
plt.ylabel('Fraction')
plt.title('Normalized Stacked Bar Chart')
plt.legend(bbox_to_anchor=(0.95, 0.5, 0.5, 0.5))

# Show the plot
plt.show()
```

Example 25 (python):
```python
# donor proportion
crosstab=pd.crosstab(adata.obs[group_col],adata.obs["donor"])
# Normalize each row to sum to 1 (i.e., to get fractions)
crosstab_normalized = crosstab.div(crosstab.sum(axis=1), axis=0)

# Plotting the stacked bar chart
ax = crosstab_normalized.plot(kind='bar',stacked=True,figsize=(10, 4),legend=False)

# Adding labels and title
plt.ylabel('Fraction')
plt.title('Normalized Stacked Bar Chart')
plt.legend(bbox_to_anchor=(0.8, 0., 0.5, 0.5))

# Show the plot
plt.show()
```

Example 26 (python):
```python
# sangerID proportion
crosstab=pd.crosstab(adata.obs[group_col],adata.obs["sangerID"])
# Normalize each row to sum to 1 (i.e., to get fractions)
crosstab_normalized = crosstab.div(crosstab.sum(axis=1), axis=0)

# Plotting the stacked bar chart
ax = crosstab_normalized.plot(kind='bar',stacked=True,figsize=(10, 4),legend=False)

# Adding labels and title
plt.ylabel('Fraction')
plt.title('Normalized Stacked Bar Chart')
plt.legend(bbox_to_anchor=(0.8, 0.5, 0.5, 0.5))

# Show the plot
plt.show()
```

Example 27 (python):
```python
sc.settings.set_figure_params(dpi=120)
sc.pl.umap(adata, color=['leiden_scVI_1'], size=1, legend_fontsize=6, wspace=0.2, legend_loc='on data')
```

Example 28 (python):
```python
conf_score_thresh = 0.6

# identify single-sample clusters
crosstab=pd.crosstab(adata.obs[group_col],adata.obs["sangerID"])
crosstab_normalized = crosstab.div(crosstab.sum(axis=1), axis=0)
res = crosstab_normalized.max(axis=1)
single_sample_clusters = list(res.index[res>0.9])
print(f'single_sample_clusters: {single_sample_clusters}')

# identify pericardium clusters
crosstab=pd.crosstab(adata.obs[group_col],adata.obs["region"])
crosstab_normalized = crosstab.div(crosstab.sum(axis=1), axis=0)
res = crosstab_normalized['Pericardium']
pericardium_clusters = list(res.index[res>0.9])
print(f'pericardium_clusters: {pericardium_clusters}')

single_butnot_peri_clusters = list(set(single_sample_clusters)-set(pericardium_clusters))
print(f'single_butnot_peri_clusters: {single_butnot_peri_clusters}')

mask1 = adata.obs[group_col].isin(['4','19'])
mask2 = adata.obs[conf_score_col] < conf_score_thresh
mask3 = adata.obs[group_col].isin(single_butnot_peri_clusters)
adata.obs['cells_to_remove'] = 'No'
adata.obs.loc[adata.obs_names[mask1|mask2|mask3],'cells_to_remove'] = 'Yes'
sc.pl.umap(adata, color=['cells_to_remove'], size=1, legend_fontsize=6, wspace=0.2)
```

Example 29 (unknown):
```unknown
... storing 'cells_to_remove' as categorical
```

Example 30 (unknown):
```unknown
single_sample_clusters: ['4', '12', '14', '17', '19', '21', '27', '30', '36']
pericardium_clusters: ['12', '14', '17', '19', '21', '27', '30']
single_butnot_peri_clusters: ['36', '4']
```

Example 31 (python):
```python
adata_raw = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/trisomy21/T21Hearts_July2024.h5ad')
adata_raw
```

Example 32 (unknown):
```unknown
AnnData object with n_obs × n_vars = 121452 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'batch_key', 'leiden_scVI_0.1', 'leiden_scVI_0.2', 'leiden_scVI_0.3', 'leiden_scVI_0.5', 'leiden_scVI_0.7', 'leiden_scVI_1'
    var: 'gene_name', 'feature_type', 'ambient_expression-0', 'ambient_expression-1', 'ambient_expression-10', 'ambient_expression-11', 'ambient_expression-12', 'ambient_expression-13', 'ambient_expression-14', 'ambient_expression-2', 'ambient_expression-3', 'ambient_expression-4', 'ambient_expression-5', 'ambient_expression-6', 'ambient_expression-7', 'ambient_expression-8', 'ambient_expression-9', 'mt', 'ribo', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts'
    uns: 'age_colors', 'donor_colors', 'leiden', 'leiden_scVI_0.1_colors', 'leiden_scVI_0.2_colors', 'leiden_scVI_0.3_colors', 'leiden_scVI_0.5_colors', 'leiden_scVI_0.7_colors', 'leiden_scVI_1_colors', 'neighbors', 'region_colors', 'sangerID_colors', 'umap'
    obsm: 'X_scVI', 'X_umap', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 33 (python):
```python
adata_raw.var['gene_id'] = adata_raw.var_names.copy()
adata_raw.var.set_index('gene_name',inplace=True)
adata_raw.var_names = adata_raw.var_names.astype('str')
adata_raw.var_names_make_unique()
```

Example 34 (python):
```python
# add prediction results, etc
adata_raw.obs[['celltypist_coarse','conf_score_coarse','cells_to_remove']] = adata[adata_raw.obs_names].obs[['celltypist_coarse','conf_score_coarse','cells_to_remove']].copy()
```

Example 35 (python):
```python
# remove
adata_raw = adata_raw[adata_raw.obs['cells_to_remove']=='No']
sc.pl.umap(adata_raw, color=['cells_to_remove'], size=1, legend_fontsize=6, wspace=0.2)
```

Example 36 (python):
```python
# save
adata_raw.write('/nfs/team205/heart/anndata_objects/Foetal/trisomy21/T21Hearts_cleaned_July2024.h5ad')
```

Example 37 (python):
```python
%%time
sc.pp.neighbors(adata_raw, use_rep="X_scVI")
sc.tl.umap(adata_raw, min_dist=0.2)
```

Example 38 (python):
```python
computing neighbors
    finished: added to `.uns['neighbors']`
    `.obsp['distances']`, distances for each pair of neighbors
    `.obsp['connectivities']`, weighted adjacency matrix (0:00:37)
computing UMAP
    finished: added
    'X_umap', UMAP coordinates (adata.obsm) (0:02:11)
CPU times: user 2min 46s, sys: 733 ms, total: 2min 47s
Wall time: 2min 49s
```

Example 39 (python):
```python
sc.pl.umap(adata_raw, color=['celltypist_coarse'], frameon=False)
```

Example 40 (python):
```python
sc.pl.umap(adata_raw, color=['donor'], frameon=False)
```

Example 41 (python):
```python
# save
adata_raw.write('/nfs/team205/heart/anndata_objects/Foetal/trisomy21/T21Hearts_cleaned_July2024.h5ad')
```

Example 42 (python):
```python
# age
adata_raw.obs.replace({'age':{
    'PCW-11w':'11W0D',
    'PCW-13w':'13W0D',
    'PCW-14w':'14W0D',
}},inplace=True)

adata_raw.obs['week'] = [x.split('0D')[0] for x in adata_raw.obs['age']]
adata_raw.obs['trimester'] = adata_raw.obs['age'].astype('str').copy()
adata_raw.obs.replace({'trimester':{
    '11W0D':'firsttrimester',
    '13W0D':'secondtrimester',
    '14W0D':'secondtrimester',
}},inplace=True)
```

Example 43 (python):
```python
adata_raw.obs.drop(['gender'],axis=1,inplace=True)
```

Example 44 (python):
```python
# sex
adata_raw.obs['sex'] = adata_raw.obs['donor'].astype('str').copy()
adata_raw.obs.replace({'sex':{
    'Hst32':'female',
    'Hst42':'male',
    'Hst44':'male',
    'Hst48':'male'
}},inplace=True)
```

Example 45 (python):
```python
# save
adata_raw.write('/nfs/team205/heart/anndata_objects/Foetal/trisomy21/T21Hearts_cleaned_July2024.h5ad')
adata_raw
```

Example 46 (unknown):
```unknown
... storing 'week' as categorical
... storing 'trimester' as categorical
... storing 'sex' as categorical
```

Example 47 (unknown):
```unknown
AnnData object with n_obs × n_vars = 110693 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'batch_key', 'leiden_scVI_0.1', 'leiden_scVI_0.2', 'leiden_scVI_0.3', 'leiden_scVI_0.5', 'leiden_scVI_0.7', 'leiden_scVI_1', 'celltypist_coarse', 'conf_score_coarse', 'cells_to_remove', 'week', 'trimester', 'sex'
    var: 'feature_type', 'ambient_expression-0', 'ambient_expression-1', 'ambient_expression-10', 'ambient_expression-11', 'ambient_expression-12', 'ambient_expression-13', 'ambient_expression-14', 'ambient_expression-2', 'ambient_expression-3', 'ambient_expression-4', 'ambient_expression-5', 'ambient_expression-6', 'ambient_expression-7', 'ambient_expression-8', 'ambient_expression-9', 'mt', 'ribo', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'gene_id'
    uns: 'age_colors', 'donor_colors', 'leiden', 'leiden_scVI_0.1_colors', 'leiden_scVI_0.2_colors', 'leiden_scVI_0.3_colors', 'leiden_scVI_0.5_colors', 'leiden_scVI_0.7_colors', 'leiden_scVI_1_colors', 'neighbors', 'region_colors', 'sangerID_colors', 'umap', 'cells_to_remove_colors', 'celltypist_coarse_colors'
    obsm: 'X_scVI', 'X_umap', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 48 (python):
```python
adata_raw = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/trisomy21/T21Hearts_cleaned_July2024.h5ad')
adata_raw
```

Example 49 (unknown):
```unknown
AnnData object with n_obs × n_vars = 110693 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'batch_key', 'leiden_scVI_0.1', 'leiden_scVI_0.2', 'leiden_scVI_0.3', 'leiden_scVI_0.5', 'leiden_scVI_0.7', 'leiden_scVI_1', 'celltypist_coarse', 'conf_score_coarse', 'cells_to_remove', 'week', 'trimester', 'sex', 'celltypist_fine', 'conf_score_fine'
    var: 'feature_type', 'ambient_expression-0', 'ambient_expression-1', 'ambient_expression-10', 'ambient_expression-11', 'ambient_expression-12', 'ambient_expression-13', 'ambient_expression-14', 'ambient_expression-2', 'ambient_expression-3', 'ambient_expression-4', 'ambient_expression-5', 'ambient_expression-6', 'ambient_expression-7', 'ambient_expression-8', 'ambient_expression-9', 'mt', 'ribo', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'gene_id'
    uns: 'age_colors', 'cells_to_remove_colors', 'celltypist_coarse_colors', 'donor_colors', 'leiden', 'leiden_scVI_0.1_colors', 'leiden_scVI_0.2_colors', 'leiden_scVI_0.3_colors', 'leiden_scVI_0.5_colors', 'leiden_scVI_0.7_colors', 'leiden_scVI_1_colors', 'neighbors', 'region_colors', 'sangerID_colors', 'umap'
    obsm: 'X_scVI', 'X_umap', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 50 (python):
```python
adata = adata_raw.copy()
sc.pp.filter_genes(adata, min_cells=3)
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)
adata.X.data[:5]
```

Example 51 (unknown):
```unknown
filtered out 3663 genes that are detected in less than 3 cells
normalizing counts per cell
    finished (0:00:00)
```

Example 52 (unknown):
```unknown
array([0.0534004 , 0.10409318, 0.19836526, 0.0534004 , 0.15233971],
      dtype=float32)
```

Example 53 (python):
```python
adata.obs['celltypist_coarse'].value_counts()
```

Example 54 (unknown):
```unknown
Mesenchymal       49009
Cardiomyocytes    42792
Endothelium        8089
Leukocytes         7262
Epicardium         2071
Neural             1470
Name: celltypist_coarse, dtype: int64
```

Example 55 (python):
```python
# predict per coarse_grained cell type
for celltype in adata.obs['celltypist_coarse'].unique():
    print(celltype)
    # subset adata
    adata_sub = adata[adata.obs['celltypist_coarse']==celltype]
    # drop all the obs columns
    adata_sub.obs.drop(columns=adata_sub.obs.columns,inplace=True)
    # predict
    predictions = celltypist.annotate(adata_sub, 
                                      model = f'/nfs/team205/kk18/notebooks/Foetal/Trisomy21/celltypist_models/coarse2fine_{celltype}.pkl', 
                                      majority_voting = False)

    # add to subsetted anndata
    adata_sub = predictions.to_adata(insert_labels = True, insert_conf = True)
    adata_sub.obs.rename(columns={'predicted_labels':'celltypist_fine',
                                  'conf_score':'conf_score_fine'
                                 },inplace=True)
    # add to the original anndata
    adata.obs.loc[adata_sub.obs_names,'celltypist_fine'] = list(adata_sub.obs['celltypist_fine'])
    adata.obs.loc[adata_sub.obs_names,'conf_score_fine'] = list(adata_sub.obs['conf_score_fine'])
```

Example 56 (unknown):
```unknown
Mesenchymal
```

Example 57 (unknown):
```unknown
🔬 Input data has 49009 cells and 32938 genes
🔗 Matching reference genes in the model
🧬 32502 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
🔬 Input data has 8089 cells and 32938 genes
🔗 Matching reference genes in the model
```

Example 58 (unknown):
```unknown
Endothelium
```

Example 59 (unknown):
```unknown
🧬 31696 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
🔬 Input data has 2071 cells and 32938 genes
🔗 Matching reference genes in the model
```

Example 60 (unknown):
```unknown
🧬 30529 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
```

Example 61 (unknown):
```unknown
Cardiomyocytes
```

Example 62 (unknown):
```unknown
🔬 Input data has 42792 cells and 32938 genes
🔗 Matching reference genes in the model
🧬 31533 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
🔬 Input data has 1470 cells and 32938 genes
🔗 Matching reference genes in the model
```

Example 63 (unknown):
```unknown
🧬 29361 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
🔬 Input data has 7262 cells and 32938 genes
🔗 Matching reference genes in the model
```

Example 64 (unknown):
```unknown
🧬 31793 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
```

Example 65 (python):
```python
# add prediction results
adata_raw.obs[['celltypist_fine','conf_score_fine']] = adata[adata_raw.obs_names].obs[['celltypist_fine','conf_score_fine']].copy()
```

Example 66 (python):
```python
sc.pl.umap(adata_raw, color=['conf_score_fine','celltypist_fine'], frameon=False)
```

Example 67 (python):
```python
sc.pl.umap(adata_raw, color=['celltypist_coarse'], frameon=False)
```

Example 68 (python):
```python
# save
adata_raw.write('/nfs/team205/heart/anndata_objects/Foetal/trisomy21/T21Hearts_cleaned_July2024.h5ad')
adata_raw
```

Example 69 (unknown):
```unknown
AnnData object with n_obs × n_vars = 110693 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'batch_key', 'leiden_scVI_0.1', 'leiden_scVI_0.2', 'leiden_scVI_0.3', 'leiden_scVI_0.5', 'leiden_scVI_0.7', 'leiden_scVI_1', 'celltypist_coarse', 'conf_score_coarse', 'cells_to_remove', 'week', 'trimester', 'sex', 'celltypist_fine', 'conf_score_fine'
    var: 'feature_type', 'ambient_expression-0', 'ambient_expression-1', 'ambient_expression-10', 'ambient_expression-11', 'ambient_expression-12', 'ambient_expression-13', 'ambient_expression-14', 'ambient_expression-2', 'ambient_expression-3', 'ambient_expression-4', 'ambient_expression-5', 'ambient_expression-6', 'ambient_expression-7', 'ambient_expression-8', 'ambient_expression-9', 'mt', 'ribo', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'gene_id'
    uns: 'age_colors', 'cells_to_remove_colors', 'celltypist_coarse_colors', 'donor_colors', 'leiden', 'leiden_scVI_0.1_colors', 'leiden_scVI_0.2_colors', 'leiden_scVI_0.3_colors', 'leiden_scVI_0.5_colors', 'leiden_scVI_0.7_colors', 'leiden_scVI_1_colors', 'neighbors', 'region_colors', 'sangerID_colors', 'umap', 'celltypist_fine_colors'
    obsm: 'X_scVI', 'X_umap', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 70 (python):
```python
# save .obs for later use
adata_raw.obs.to_csv('/nfs/team205/heart/anndata_objects/Foetal/trisomy21/T21Hearts_cleaned_July2024.obs.csv')
```

---

## Import modules¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig4_Cardiomyocytes/VentricularCardiomyocytes/TransmuralAxis/tm-axis_cell2loc_t21.html

**Contents:**
- Import modules¶
- Functions¶
- Read in adata¶
- Add axis¶
- Plot transmural axis¶
- Plot cells along the axis and time¶

**Examples:**

Example 1 (python):
```python
import os
import numpy as np
import scanpy as sc
import anndata
import pandas as pd
import matplotlib as mpl
import matplotlib.pyplot as plt
import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 
import seaborn as sns

import warnings
warnings.simplefilter(action='ignore', category=FutureWarning)
warnings.simplefilter("ignore", UserWarning)
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
-----
anndata             0.9.2
matplotlib          3.7.2
numpy               1.24.4
pandas              1.5.3
scanpy              1.9.3
seaborn             0.12.2
session_info        1.0.0
-----
```

Example 4 (python):
```python
PIL                         10.0.0
anyio                       NA
arrow                       1.2.3
asttokens                   NA
attr                        23.1.0
attrs                       23.1.0
babel                       2.12.1
backcall                    0.2.0
brotli                      1.0.9
certifi                     2023.11.17
cffi                        1.15.1
charset_normalizer          3.2.0
colorama                    0.4.6
comm                        0.1.4
cycler                      0.10.0
cython_runtime              NA
dateutil                    2.8.2
debugpy                     1.6.8
decorator                   5.1.1
defusedxml                  0.7.1
executing                   1.2.0
fastjsonschema              NA
fqdn                        NA
h5py                        3.9.0
idna                        3.4
igraph                      0.10.6
importlib_metadata          NA
importlib_resources         NA
ipykernel                   6.25.1
isoduration                 NA
jedi                        0.19.0
jinja2                      3.1.2
joblib                      1.3.2
json5                       NA
jsonpointer                 2.0
jsonschema                  4.19.0
jsonschema_specifications   NA
jupyter_events              0.7.0
jupyter_server              2.7.1
jupyterlab_server           2.24.0
kiwisolver                  1.4.5
leidenalg                   0.10.1
llvmlite                    0.40.1
markupsafe                  2.1.3
mpl_toolkits                NA
natsort                     8.4.0
nbformat                    5.9.2
numba                       0.57.1
overrides                   NA
packaging                   23.1
parso                       0.8.3
patsy                       0.5.3
pexpect                     4.8.0
pickleshare                 0.7.5
pkg_resources               NA
plotly                      5.16.1
prometheus_client           NA
prompt_toolkit              3.0.39
psutil                      5.9.5
ptyprocess                  0.7.0
pure_eval                   0.2.2
pydev_ipython               NA
pydevconsole                NA
pydevd                      2.9.5
pydevd_file_utils           NA
pydevd_plugins              NA
pydevd_tracing              NA
pygments                    2.16.1
pyparsing                   3.0.9
pythonjsonlogger            NA
pytz                        2023.3
referencing                 NA
requests                    2.31.0
rfc3339_validator           0.1.4
rfc3986_validator           0.1.1
rpds                        NA
ruamel                      NA
scipy                       1.10.1
send2trash                  NA
six                         1.16.0
sklearn                     1.3.0
sniffio                     1.3.0
socks                       1.7.1
sphinxcontrib               NA
stack_data                  0.6.2
statsmodels                 0.14.0
texttable                   1.6.7
threadpoolctl               3.2.0
tornado                     6.3.3
traitlets                   5.9.0
typing_extensions           NA
uri_template                NA
urllib3                     2.0.4
wcwidth                     0.2.6
webcolors                   1.13
websocket                   1.6.2
yaml                        6.0.1
zipp                        NA
zmq                         25.1.1
zstandard                   0.19.0
```

Example 5 (unknown):
```unknown
-----
IPython             8.12.0
jupyter_client      8.3.0
jupyter_core        4.12.0
jupyterlab          4.0.5
notebook            7.0.2
-----
Python 3.8.17 (default, Jul  5 2023, 21:04:15) [GCC 11.2.0]
Linux-4.15.0-220-generic-x86_64-with-glibc2.17
-----
Session information updated at 2024-02-15 10:02
```

Example 6 (python):
```python
import importlib.util
import sys

spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)

spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/tissueaxis.py")
tissueaxis = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = tissueaxis
spec.loader.exec_module(tissueaxis)
```

Example 7 (python):
```python
sc.settings.set_figure_params(dpi=80,dpi_save=300)
```

Example 8 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk'
```

Example 9 (python):
```python
# This function selects the data for multiple slides from the spatial anndata object.
# Modified `select_slide` in cell2loc for multiple slides

# @param adata: anndata object with multiple spatial experiments
# @param s: list of selected slides
# @param s_col: column in adata.obs listing experiment name for each location

def select_multislide(adata, s, s_col='sample'):

    slides = adata[adata.obs[s_col].isin(s), :]
    s_keys = list(slides.uns['spatial'].keys())
    
    uns_dict = {}
    for ss in s:
        s_spatial=np.array(s_keys)[[ss in k for k in s_keys]][0]
        uns_dict[s_spatial]=slides.uns['spatial'][s_spatial]
    
    slides.uns['spatial'] = uns_dict

    return slides
```

Example 10 (python):
```python
adata_oct_healthy = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/visium-OCT_foetal_raw.h5ad')
print(adata_oct_healthy.shape)
adata_oct_t21 = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/visium-OCT-T21_foetal_raw.h5ad')
print(adata_oct_t21.shape)
```

Example 11 (unknown):
```unknown
(14261, 33538)
(11238, 33538)
```

Example 12 (python):
```python
# concatenate oct adata
adata_oct_healthy.obs['status'] = 'Euploid'
adata_oct_t21.obs['status'] = 'Trisomy21'

adata_oct = adata_oct_healthy.concatenate(
    [adata_oct_t21],
    batch_key=None,
    index_unique=None,
    join="inner"
)

adata_oct.obs['status'].value_counts()
```

Example 13 (unknown):
```unknown
Euploid      14261
Trisomy21    11238
Name: status, dtype: int64
```

Example 14 (python):
```python
# log-normalise
sc.pp.normalize_total(adata_oct, target_sum=1e4)
sc.pp.log1p(adata_oct)

# reset var_names
adata_oct.var.set_index('SYMBOL',inplace=True)
```

Example 15 (python):
```python
# get all celltypes
all_celltypes = adata_oct.obsm['q05_cell_abundance_w_sf'].columns
all_celltypes = [x.replace('q05cell_abundance_w_sf_','') for x in all_celltypes]
print(len(all_celltypes))
print(all_celltypes[:5])
```

Example 16 (unknown):
```unknown
54
['AtrialCardiomyocytesLeft', 'AtrialCardiomyocytesRight', 'VentricularCardiomyocytesLeftCompact', 'VentricularCardiomyocytesRightCompact', 'VentricularCardiomyocytesLeftTrabeculated']
```

Example 17 (python):
```python
# subset cell types in ffpe
adata_oct.obsm['q05_cell_abundance_w_sf'] = adata_oct.obsm['q05_cell_abundance_w_sf'][[f'q05cell_abundance_w_sf_{x}' for x in all_celltypes]]

# modify cell type names
celltype_rename = utils.read_pkl('/nfs/team205/heart/anndata_objects/Foetal/fine_grain_rename_dictionary.pkl')
adata_oct.obsm['q05_cell_abundance_w_sf'].columns = [x.replace(x.split('q05cell_abundance_w_sf_')[1],celltype_rename[x.split('q05cell_abundance_w_sf_')[1]]) \
                                                      for x in adata_oct.obsm['q05_cell_abundance_w_sf'].columns]
```

Example 18 (python):
```python
# add
oct_samples = list(set(adata_oct.obs['sample']))

adata_oct.obs['transmural_axis'] = np.nan

axis_files = os.listdir('/nfs/team205/heart/visium_axis/foetal/transmural_LV')
for file in axis_files:
    # read in
    axis = pd.read_csv(f'/nfs/team205/heart/visium_axis/foetal/transmural_LV/{file}',index_col=0)
    # add to anndata
    if file.split('.csv')[0] in oct_samples:
        adata_oct.obs.loc[axis.index,'transmural_axis'] = axis.copy()
        if file.split('.csv')[0]=='HCAHeartST11605165':
            adata_oct_healthy.obs.loc[axis.index,'transmural_axis'] = axis.copy()
        elif file.split('.csv')[0]=='HCAHeartST13162336':
            adata_oct_t21.obs.loc[axis.index,'transmural_axis'] = axis.copy()

# select spots with axis
adata_oct = adata_oct[adata_oct.obs['transmural_axis'].isna()==False]
```

Example 19 (python):
```python
pd.crosstab(adata_oct.obs['sample'],adata_oct.obs['cell2loc-map_group'])
```

Example 20 (python):
```python
slide_id = 'HCAHeartST11605165'
slide = select_multislide(adata_oct_healthy, [slide_id])
sc.pl.spatial(slide,color=['transmural_axis'],cmap='gist_rainbow',show=False)
plt.savefig(f"{figdir}/visium_tm-axis_euploid_{slide_id}.pdf", bbox_inches="tight",dpi=300)
```

Example 21 (python):
```python
slide_id = 'HCAHeartST13162336'
slide = select_multislide(adata_oct_t21, [slide_id])
sc.pl.spatial(slide,color=['transmural_axis'],cmap='gist_rainbow',show=False)
plt.savefig(f"{figdir}/visium_tm-axis_t21_{slide_id}.pdf", bbox_inches="tight",dpi=300)
```

Example 22 (python):
```python
features_to_plot = ['VentricularCardiomyocytesLeftCompact',
                    'CoronaryCapillaryEndothelialCells','CoronaryPericytes'
                   ]

for feature in features_to_plot:
    tissueaxis.plot_axis_split(adata_oct,
                               feature_to_plot=feature,
                               feature_type='celltype',
                               min_max_scale=False,
                               splitby='cell2loc-map_group',
                               groups_order=['13PCW','T21_13PCW'],
                               figsize=(5, 3),
                               cbar_pos=(0, 0.7, 0.05, 0.18),
                               save=f'{figdir}/tm-axis_cell2loc_t21_{feature}.pdf'
                              )
```

Example 23 (unknown):
```unknown
?tissueaxis.plot_axis_split
```

Example 24 (python):
```python
Signature:
tissueaxis.plot_axis_split(
    adata,
    feature_to_plot,
    splitby,
    groups_order=None,
    axis_col='transmural_axis',
    n_bin=15,
    feature_type='gene',
    min_max_scale=True,
    row_cluster=False,
    figsize=(3, 8),
    cbar_pos=(0, 0.5, 0.05, 0.18),
    cmap='viridis',
    xticklabels=True,
    yticklabels=True,
    title=None,
    save=None,
    save_dpi=300,
    **kwargs,
)
Docstring: <no docstring>
File:      /nfs/team205/kk18/function/python/tissueaxis.py
Type:      function
```

---

## Variables¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/Xenium/no2_integration_and_cell-type-annotation/no6_manual-annotation_fibroblasts.html

**Contents:**
- Variables¶
- Read in data¶
- 1st round annotation¶
- Subset and reclustering¶
- Transfer annotation to the original adata¶
- Save¶

--> cluster 0 has lower total counts and smaller cell area, and no specific gene expression --> annotate as "unassigned"

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os

import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 

import scanpy.external as sce
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/session_info/main.py:213: UserWarning: The '__version__' attribute is deprecated and will be removed in MarkupSafe 3.1. Use feature detection, or `importlib.metadata.version("markupsafe")`, instead.
  mod_version = _find_version(mod.__version__)
```

Example 4 (python):
```python
-----
matplotlib          3.10.1
numpy               2.1.3
pandas              2.2.3
scanpy              1.11.1
seaborn             0.13.2
session_info        v1.0.1
-----
```

Example 5 (python):
```python
Cython              3.1.1
PIL                 11.1.0
anndata             0.11.4
asciitree           NA
asttokens           NA
charset_normalizer  3.4.1
cloudpickle         3.1.1
comm                0.2.2
cycler              0.12.1
cython              3.1.1
cython_runtime      NA
dask                2024.11.2
dateutil            2.9.0.post0
debugpy             1.8.13
decorator           5.2.1
exceptiongroup      1.2.2
executing           2.2.0
h5py                3.13.0
igraph              0.11.8
importlib_metadata  NA
ipykernel           6.29.5
jedi                0.19.2
jinja2              3.1.6
joblib              1.4.2
kiwisolver          1.4.8
legacy_api_wrap     NA
leidenalg           0.10.2
llvmlite            0.44.0
markupsafe          3.0.2
mpl_toolkits        NA
msgpack             1.1.0
natsort             8.4.0
numba               0.61.0
numcodecs           0.13.1
packaging           24.2
parso               0.8.4
patsy               1.0.1
platformdirs        4.3.7
prompt_toolkit      3.0.50
psutil              7.0.0
pure_eval           0.2.3
pyarrow             19.0.1
pydev_ipython       NA
pydevconsole        NA
pydevd              3.2.3
pydevd_file_utils   NA
pydevd_plugins      NA
pydevd_tracing      NA
pygments            2.19.1
pyparsing           3.2.3
pytz                2025.2
scipy               1.15.2
six                 1.17.0
sklearn             1.5.2
stack_data          0.6.3
statsmodels         0.14.4
tblib               3.1.0
texttable           1.7.0
threadpoolctl       3.6.0
tlz                 1.0.0
toolz               1.0.0
tornado             6.4.2
traitlets           5.14.3
typing_extensions   NA
wcwidth             0.2.13
yaml                6.0.2
zarr                2.18.3
zipp                NA
zmq                 26.3.0
zoneinfo            NA
```

Example 6 (unknown):
```unknown
-----
IPython             8.34.0
jupyter_client      8.6.3
jupyter_core        5.7.2
-----
Python 3.10.14 | packaged by conda-forge | (main, Mar 20 2024, 12:45:18) [GCC 12.3.0]
Linux-4.18.0-553.53.1.el8_10.x86_64-x86_64-with-glibc2.28
-----
Session information updated at 2025-06-09 22:58
```

Example 7 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/home/kk837/rfs/rfs-teichlab-nfs-iCNyzSAaucw/kk837/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 8 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 9 (python):
```python
adata_dir = '/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium/subsets'
sample_id = 'C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat'
celltype = 'Fibroblasts'
```

Example 10 (unknown):
```unknown
# figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk/Aug2024'
# table_dir = '/nfs/team205/heart/fetal_ms_figures/tables/Aug2024'
```

Example 11 (unknown):
```unknown
latent_space = 'pca_harmony'
```

Example 12 (python):
```python
# read in
adata = sc.read_h5ad(f'{adata_dir}/{sample_id}_5K_{celltype}_lognorm.h5ad')
adata
```

Example 13 (unknown):
```unknown
AnnData object with n_obs × n_vars = 49242 × 4999
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_3.0', 'multi_celltypes_coarse', 'leiden_0.2', 'leiden_0.4', 'leiden_0.6', 'leiden_1'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1', 'n_cells', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'dendrogram_pca_harmony_leiden_1.0', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1_colors', 'log1p', 'multi_celltypes_coarse_colors', 'pca', 'pca_harmony', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.2_colors', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.3_colors', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.4_colors', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.5_colors', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_0.8_colors', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.0_colors', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_1.5_colors', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_2.0_colors', 'pca_harmony_leiden_3.0', 'pca_harmony_leiden_3.0_colors', 'tissue_block_id_colors', 'umap'
    obsm: 'X_pca', 'X_pca_harmony', 'X_umap_pca_harmony', 'spatial'
    varm: 'PCs'
    obsp: 'pca_harmony_connectivities', 'pca_harmony_distances'
```

Example 14 (python):
```python
adata.X.data[:5]
```

Example 15 (unknown):
```unknown
array([4.2093534, 2.6576407, 2.6576407, 3.3151019, 3.3151019],
      dtype=float32)
```

Example 16 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=['total_counts','n_genes_by_counts','cell_area','tissue_block_id'],
                    wspace=0.2,cmap='RdPu',vmax='p100')
```

Example 17 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=['multi_celltypes_coarse',"conf_score_midmod2fine","celltypist_midmod2fine",],
                    wspace=0.2,cmap='RdPu',vmax='p100')
resolutions_list = [0.2,0.4,0.6,1]
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                    color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                    wspace=0.2)
```

Example 18 (python):
```python
%%time
# leiden clustering
resolutions_list = [# 0.2,0.4,0.6,1, # already done
                    1.2,1.5,2]
for resolution in resolutions_list:
    sc.tl.leiden(adata,neighbors_key=latent_space,
                     resolution=resolution,key_added=f'leiden_{str(resolution)}',n_iterations=2)
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                wspace=0.3)
```

Example 19 (unknown):
```unknown
<timed exec>:5: FutureWarning: In the future, the default backend for leiden will be igraph instead of leidenalg.

 To achieve the future defaults please pass: flavor="igraph" and n_iterations=2.  directed must also be False to work with igraph's implementation.
```

Example 20 (unknown):
```unknown
CPU times: user 36.3 s, sys: 367 ms, total: 36.7 s
Wall time: 36.8 s
```

Example 21 (python):
```python
# markers for fine-grained cell types
sc_markers = pd.read_csv(f'/home/kk837/rfs/rfs-teichlab-nfs-iCNyzSAaucw/kk837/notebooks/Foetal/VisiumHD/re-seq_mpp0p3/sc_{celltype}_finegrain_markers.csv',
                         index_col=0)
markers = {}
n_top = 30
for c in set(sc_markers['group']):
    markers_sub = sc_markers[sc_markers['group']==c]
    markers_sub.sort_values('logfoldchanges',ascending=False)
    markers[c]=list(markers_sub[:n_top]['names'])

genes = []
for g in markers.values():
    genes = genes + [x for x in g if (x in adata.var_names)&(x not in genes)]
for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in adata.var_names]
genes = list(set(genes))
len(genes)
```

Example 22 (unknown):
```unknown
resolution_sel = 0.6
```

Example 23 (python):
```python
sc.tl.dendrogram(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.pl.dotplot(adata,
              var_names=markers,
              groupby=f'leiden_{str(resolution_sel)}',
              standard_scale='var',
              dendrogram=True
             )
```

Example 24 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: ValveInterstitialCells, Myofibroblasts, LymphNodeFibroblasticReticularCells, etc.
```

Example 25 (python):
```python
print(adata.X.data[:5])
adata.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata, dendrogram=True, standard_scale='var', return_fig=True)
dp.add_totals().show()
```

Example 26 (unknown):
```unknown
[4.2093534 2.6576407 2.6576407 3.3151019 3.3151019]
```

Example 27 (python):
```python
sc.pl.violin(adata,keys=['total_counts','cell_area'],groupby=f'leiden_{str(resolution_sel)}')
```

Example 28 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=f'leiden_{str(resolution_sel)}',legend_loc='on data',
                )
```

Example 29 (python):
```python
# annotate
adata.obs['tmp_annotation'] = adata.obs[f'leiden_{str(resolution_sel)}'].copy()
adata.obs.replace({'tmp_annotation':{
    '0':'unassigned',
    '1':'Fibroblasts',
    '2':'Fibroblasts',
    '3':'Fibroblasts',
    '4':'Fibroblasts',
    '5':'Fibroblasts',
    '6':'Fibroblasts',
    '7':'Fibroblasts',
    '8':'Fibroblasts',
    '9':'Fibroblasts',
    '10':'Fibroblasts',
}},inplace=True)
adata.obs['tmp_annotation'].value_counts()
```

Example 30 (python):
```python
/tmp/ipykernel_71001/632585045.py:3: FutureWarning: The behavior of Series.replace (and DataFrame.replace) with CategoricalDtype is deprecated. In a future version, replace will only be used for cases that preserve the categories. To change the categories, use ser.cat.rename_categories instead.
  adata.obs.replace({'tmp_annotation':{
```

Example 31 (unknown):
```unknown
tmp_annotation
Fibroblasts    39604
unassigned      9638
Name: count, dtype: int64
```

Example 32 (unknown):
```unknown
'Fibroblasts'
```

Example 33 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color='tmp_annotation'
                )
```

Example 34 (python):
```python
adata_sub = adata[adata.obs['tmp_annotation'].isin(['Fibroblasts'])]
```

Example 35 (python):
```python
# drop leiden clustering results
resolutions_list = [0.2,0.4,0.6,1, 1.2,1.5,2]
for resolution in resolutions_list:
    adata_sub.obs.drop(f'leiden_{str(resolution)}',axis=1,inplace=True)
adata_sub
```

Example 36 (unknown):
```unknown
AnnData object with n_obs × n_vars = 39604 × 4999
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_3.0', 'multi_celltypes_coarse', 'tmp_annotation'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1', 'n_cells', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'dendrogram_pca_harmony_leiden_1.0', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1_colors', 'log1p', 'multi_celltypes_coarse_colors', 'pca', 'pca_harmony', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.2_colors', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.3_colors', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.4_colors', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.5_colors', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_0.8_colors', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.0_colors', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_1.5_colors', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_2.0_colors', 'pca_harmony_leiden_3.0', 'pca_harmony_leiden_3.0_colors', 'tissue_block_id_colors', 'umap', 'leiden_1.2', 'leiden_1.5', 'leiden_2', 'leiden_1.2_colors', 'leiden_1.5_colors', 'leiden_2_colors', 'dendrogram_leiden_0.6', 'rank_genes_groups', 'tmp_annotation_colors'
    obsm: 'X_pca', 'X_pca_harmony', 'X_umap_pca_harmony', 'spatial'
    varm: 'PCs'
    obsp: 'pca_harmony_connectivities', 'pca_harmony_distances'
```

Example 37 (python):
```python
## Remove mid-grain markers from the feature
# read in markers
midgrain_markers = pd.read_csv('/home/kk837/rfs/rfs-teichlab-nfs-iCNyzSAaucw/kk837/notebooks/Foetal/VisiumHD/re-seq_mpp0p3/sc_midgrain_markers.csv',
                               index_col=0)
# remove the markers of celltype of interest
midgrain_markers = midgrain_markers[midgrain_markers['group']!=celltype]
# remove
mask = [x not in list(midgrain_markers['names']) for x in adata_sub.var_names]
bdata = adata_sub[:,mask]
bdata
```

Example 38 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 39604 × 4522
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_3.0', 'multi_celltypes_coarse', 'tmp_annotation'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1', 'n_cells', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'dendrogram_pca_harmony_leiden_1.0', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1_colors', 'log1p', 'multi_celltypes_coarse_colors', 'pca', 'pca_harmony', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.2_colors', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.3_colors', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.4_colors', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.5_colors', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_0.8_colors', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.0_colors', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_1.5_colors', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_2.0_colors', 'pca_harmony_leiden_3.0', 'pca_harmony_leiden_3.0_colors', 'tissue_block_id_colors', 'umap', 'leiden_1.2', 'leiden_1.5', 'leiden_2', 'leiden_1.2_colors', 'leiden_1.5_colors', 'leiden_2_colors', 'dendrogram_leiden_0.6', 'rank_genes_groups', 'tmp_annotation_colors'
    obsm: 'X_pca', 'X_pca_harmony', 'X_umap_pca_harmony', 'spatial'
    varm: 'PCs'
    obsp: 'pca_harmony_connectivities', 'pca_harmony_distances'
```

Example 39 (python):
```python
'''
# remove cell cycle genes
cell_cycle_genes = ['MCM5','PCNA','TYMS','FEN1','MCM2','MCM4','RRM1','UNG','GINS2','MCM6','CDCA7','DTL','PRIM1','UHRF1','MLF1IP','HELLS','RFC2','RPA2','NASP',
                'RAD51AP1','GMNN','WDR76','SLBP','CCNE2','UBR7','POLD3','MSH2','ATAD2','RAD51','RRM2','CDC45','CDC6','EXO1','TIPIN','DSCC1','BLM','CASP8AP2',
                'USP1','CLSPN','POLA1','CHAF1B','BRIP1','E2F8','HMGB2','CDK1','NUSAP1','UBE2C','BIRC5','TPX2','TOP2A','NDC80','CKS2','NUF2','CKS1B','MKI67',
                'TMPO','CENPF','TACC3','FAM64A','SMC4','CCNB2','CKAP2L','CKAP2','AURKB','BUB1','KIF11','ANP32E','TUBB4B','GTSE1','KIF20B','HJURP','CDCA3','HN1',
                'CDC20','TTK','CDC25C','KIF2C','RANGAP1','NCAPD2','DLGAP5','CDCA2','CDCA8','ECT2','KIF23','HMMR','AURKA','PSRC1','ANLN','LBR','CKAP5','CENPE','CTCF',
                'NEK2','G2E3','GAS2L3','CBX5','CENPA']
mask = [x not in cell_cycle_genes for x in bdata.var_names]
bdata = bdata[:,mask]
bdata.shape
'''
```

Example 40 (python):
```python
# HVGs, using the object which doesn't have midgrain markers
sc.pp.highly_variable_genes(bdata, flavor='seurat', min_mean=0.0125, max_mean=3, min_disp=0.5)
print(f'number of hvgs: {sum(bdata.var.highly_variable)}')
# PCA using the degected HVGs
sc.tl.pca(bdata, use_highly_variable=True)
adata_sub.obsm['X_pca'] = bdata.obsm['X_pca'].copy()
# harmony integration
sce.pp.harmony_integrate(adata_sub, 'tissue_block_id', basis='X_pca', adjusted_basis='X_pca_harmony')
# create neighbour graph
latent_space = 'pca_harmony'
sc.pp.neighbors(adata_sub,n_neighbors=15,use_rep=f"X_{latent_space}",key_added=latent_space)
# visualisation with umap
sc.tl.umap(adata_sub,min_dist=0.5,neighbors_key=latent_space)
adata_sub.obsm[f"X_umap_{latent_space}"]=adata_sub.obsm["X_umap"]
del adata_sub.obsm["X_umap"]
```

Example 41 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_highly_variable_genes.py:698: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  adata.uns["hvg"] = {"flavor": flavor}
```

Example 42 (unknown):
```unknown
number of hvgs: 732
```

Example 43 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_pca/__init__.py:438: FutureWarning: Argument `use_highly_variable` is deprecated, consider using the mask argument. Use_highly_variable=True can be called through mask_var="highly_variable". Use_highly_variable=False can be called through mask_var=None
  warn(msg, FutureWarning)
2025-06-09 23:01:48,070 - harmonypy - INFO - Computing initial centroids with sklearn.KMeans...
2025-06-09 23:01:51,731 - harmonypy - INFO - sklearn.KMeans initialization complete.
2025-06-09 23:01:51,859 - harmonypy - INFO - Iteration 1 of 10
2025-06-09 23:01:58,543 - harmonypy - INFO - Iteration 2 of 10
2025-06-09 23:02:06,203 - harmonypy - INFO - Converged after 2 iterations
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm
```

Example 44 (python):
```python
%%time
# reclustering
# leiden clustering
resolutions_list = [0.2,0.4,0.6,1, 1.2,1.5,2]
for resolution in resolutions_list:
    sc.tl.leiden(adata_sub,neighbors_key=latent_space,
                 resolution=resolution,key_added=f'leiden_{str(resolution)}',n_iterations=2)
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                wspace=0.3)
```

Example 45 (unknown):
```unknown
CPU times: user 1min 18s, sys: 734 ms, total: 1min 19s
Wall time: 1min 19s
```

Example 46 (python):
```python
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color=genes,
               ncols=5, wspace=0.2,cmap='RdPu',vmax='p100')
```

Example 47 (python):
```python
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color=['TOP2A','MKI67'],
               ncols=5, wspace=0.2,cmap='RdPu',vmax='p100')
```

Example 48 (python):
```python
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color=["conf_score_midmod2fine","celltypist_midmod2fine",],
                    wspace=0.2,cmap='RdPu',vmax='p100')
```

Example 49 (unknown):
```unknown
sub_resolution_sel = 0.6
```

Example 50 (python):
```python
sc.tl.dendrogram(adata_sub,groupby=f'leiden_{str(sub_resolution_sel)}')
sc.pl.dotplot(adata_sub,
              var_names=markers,
              groupby=f'leiden_{str(sub_resolution_sel)}',
              standard_scale='var',
              dendrogram=True
             )
```

Example 51 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: ValveInterstitialCells, Myofibroblasts, LymphNodeFibroblasticReticularCells, etc.
```

Example 52 (python):
```python
# DEG analysis
print(adata_sub.X.data[:5])
adata_sub.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata_sub,groupby=f'leiden_{str(sub_resolution_sel)}')
sc.tl.dendrogram(adata_sub, groupby=f'leiden_{str(sub_resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata_sub, dendrogram=True, standard_scale='var', return_fig=True)
dp.add_totals().show()
```

Example 53 (unknown):
```unknown
[4.2093534 2.6576407 2.6576407 3.3151019 3.3151019]
```

Example 54 (python):
```python
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color=f'leiden_{str(sub_resolution_sel)}',legend_loc='on data',
                wspace=0.3)
```

Example 55 (python):
```python
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color=['celltypist_midmod2fine', 'conf_score_midmod2fine'],
                wspace=0.7)
```

Example 56 (python):
```python
# annotate
adata_sub.obs['fine_grain_manual'] = adata_sub.obs[f'leiden_{str(sub_resolution_sel)}'].copy()
adata_sub.obs.replace({'fine_grain_manual':{
    '0':'Fibroblasts',
    '1':'Fibroblasts',
    '2':'Fibroblasts',
    '3':'FibroblastsCycling',
    '4':'Fibroblasts',
    '5':'Fibroblasts',
    '6':'Fibroblasts',
    '7':'FibroblastsCOL2A1pos',
    '8':'Fibroblasts',
    '9':'Fibroblasts',
}},inplace=True)
adata_sub.obs['fine_grain_manual'].value_counts()
```

Example 57 (python):
```python
/tmp/ipykernel_71001/4045297513.py:3: FutureWarning: The behavior of Series.replace (and DataFrame.replace) with CategoricalDtype is deprecated. In a future version, replace will only be used for cases that preserve the categories. To change the categories, use ser.cat.rename_categories instead.
  adata_sub.obs.replace({'fine_grain_manual':{
```

Example 58 (unknown):
```unknown
fine_grain_manual
Fibroblasts             32842
FibroblastsCycling       4133
FibroblastsCOL2A1pos     2629
Name: count, dtype: int64
```

Example 59 (unknown):
```unknown
'Fibroblasts'
```

Example 60 (python):
```python
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color='fine_grain_manual',
                wspace=0.3)
```

Example 61 (python):
```python
adata.obs['fine_grain_manual'] = adata.obs['tmp_annotation'].astype('str').copy()
adata.obs.loc[adata_sub.obs_names,'fine_grain_manual'] = adata_sub.obs['fine_grain_manual']
```

Example 62 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color='fine_grain_manual'
                )
```

Example 63 (python):
```python
for tissue_id in ['C194-HEA-0-FFPE-1', 'Hst45-HEA-0-FFPE-1']:
    slide = adata[adata.obs['tissue_block_id']==tissue_id]
    sc.pl.spatial(slide,color=['fine_grain_manual'],# groups='SinoatrialNodePacemakerCells',
                  spot_size=60,vmax='p100')
```

Example 64 (python):
```python
/tmp/ipykernel_71001/4023824168.py:3: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],# groups='SinoatrialNodePacemakerCells',
```

Example 65 (python):
```python
/tmp/ipykernel_71001/4023824168.py:3: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],# groups='SinoatrialNodePacemakerCells',
```

Example 66 (python):
```python
for ct in set(adata.obs['fine_grain_manual']):
    for tissue_id in ['C194-HEA-0-FFPE-1', 'Hst45-HEA-0-FFPE-1']:
        slide = adata[adata.obs['tissue_block_id']==tissue_id]
        sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
                      spot_size=80,vmax='p100')
```

Example 67 (python):
```python
/tmp/ipykernel_71001/1166615315.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 68 (python):
```python
/tmp/ipykernel_71001/1166615315.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 69 (python):
```python
/tmp/ipykernel_71001/1166615315.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 70 (python):
```python
/tmp/ipykernel_71001/1166615315.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 71 (python):
```python
/tmp/ipykernel_71001/1166615315.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 72 (python):
```python
/tmp/ipykernel_71001/1166615315.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 73 (python):
```python
/tmp/ipykernel_71001/1166615315.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 74 (python):
```python
/tmp/ipykernel_71001/1166615315.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 75 (python):
```python
# save
adata.write(f'{adata_dir}/{sample_id}_5K_{celltype}_lognorm.h5ad')
print(f'{adata_dir}/{sample_id}_5K_{celltype}_lognorm.h5ad')
```

Example 76 (unknown):
```unknown
/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium/subsets/C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat_5K_Fibroblasts_lognorm.h5ad
```

---

## Violin plots¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/RNA/scnRNA_Integration-Annotation/Mesenchymal/ViolinPlots.html

**Contents:**
- Violin plots¶

**Examples:**

Example 1 (unknown):
```unknown
#General figures codes
```

Example 2 (python):
```python
# Import packages, initial settings

import numpy as np
import matplotlib.pyplot as plt
import scanpy as sc
import pandas as pd
import anndata
import scrublet as scr
import seaborn as sns
import magic
import pybiomart
import six
import sys
sys.modules['sklearn.externals.six'] = six
import scanorama
import harmonypy
import bbknn
import gseapy
from gseapy.plot import barplot, dotplot
import scvelo as scv
import scvi

plt.rcParams['pdf.fonttype'] = 42
plt.rcParams['ps.fonttype'] = 42

sc.settings.verbosity = 3  # verbosity: errors (0), warnings (1), info (2), hints (3)

from matplotlib.colors import LinearSegmentedColormap
cmap = LinearSegmentedColormap.from_list(name='gene_cmap', colors=['lightgrey', 'thistle', 'red', 'darkred']) 

sc.settings.set_figure_params(dpi=80, color_map='viridis', vector_friendly=False,  dpi_save=300, facecolor='white')
sc.logging.print_version_and_date()
#sc.logging.print_versions()
```

Example 3 (unknown):
```unknown
Global seed set to 0
```

Example 4 (unknown):
```unknown
Running Scanpy 1.8.1, on 2023-05-11 19:31.
```

Example 5 (python):
```python
adata=sc.read('./Objects/Feb28ObjectRaw.h5ad')
print(adata.shape)
```

Example 6 (unknown):
```unknown
(297473, 36601)
```

Example 7 (python):
```python
set(adata.obs.fine_grain)
```

Example 8 (unknown):
```unknown
{'AtrialCardiomyocytesCycling',
 'AtrialCardiomyocytesLeft',
 'AtrialCardiomyocytesRight',
 'AtrioventricularNodePacemakerCells',
 'BCells',
 'BCellsMS4A1pos',
 'ChromaffinCells',
 'CoronaryArterialEndothelialCells',
 'CoronaryCapillaryEndothelialCells',
 'CoronaryPericytes',
 'CoronarySmoothMuscleCells',
 'CoronaryVenousEndothelialCells',
 'CoronaryVesselAdventitialFibroblasts',
 'DendriticCellsMature',
 'DendriticCellsType1',
 'DuctusArteriosusSmoothMuscleCells',
 'EndocardialCells',
 'EndocardialCushionCells',
 'EpicardiumDerivedCells',
 'GreatVesselAdventitialFibroblasts',
 'GreatVesselArterialEndothelialCells',
 'GreatVesselSmoothMuscleCells',
 'GreatVesselVenousEndothelialCells',
 'InnateLymphoidCells',
 'LymphNodeFibroblasticReticularCells',
 'LymphaticEndothelialCells',
 'MacrophagesATF3pos',
 'MacrophagesCX3CR1pos',
 'MacrophagesLYVE1pos',
 'MacrophagesTIMD4pos',
 'MastCells',
 'Megakaryocytes',
 'MesothelialEpicardialCells',
 'MonocyteDerivedCells',
 'Monocytes',
 'MonocytesMPOpos',
 'MyocardialInterstitialFibroblasts',
 'Myofibroblasts',
 'NaturalKillerCells',
 'NeuronPrecursors',
 'ParaympatheticNeurons',
 'PericardialCellsFibrous',
 'PericardialCellsIntermediate',
 'PericardialCellsParietal',
 'PlasmacytoidDendriticCells',
 'ProBCells',
 'SchwannCellPrecursors',
 'SchwannCells',
 'SinoatrialNodePacemakerCells',
 'SubEpicardialFibroblasts',
 'SympatheticNeurons',
 'TCellsCD4pos',
 'TCellsCD8pos',
 'TregsCD4pos',
 'ValveEndothelialCells',
 'ValveInterstitialCells',
 'VentricularCardiomyocytesCycling',
 'VentricularCardiomyocytesLeftCompact',
 'VentricularCardiomyocytesLeftTrabeculated',
 'VentricularCardiomyocytesRightCompact',
 'VentricularCardiomyocytesRightTrabeculated',
 'VentricularConductionSystemDistal',
 'VentricularConductionSystemProximal'}
```

Example 9 (python):
```python
cell_cycle_genes = ['MCM5','PCNA','TYMS','FEN1','MCM2','MCM4','RRM1','UNG','GINS2','MCM6','CDCA7','DTL','PRIM1','UHRF1','MLF1IP','HELLS','RFC2','RPA2','NASP','RAD51AP1','GMNN','WDR76','SLBP','CCNE2','UBR7','POLD3','MSH2','ATAD2','RAD51','RRM2','CDC45','CDC6','EXO1','TIPIN','DSCC1','BLM','CASP8AP2','USP1','CLSPN','POLA1','CHAF1B','BRIP1','E2F8','HMGB2','CDK1','NUSAP1','UBE2C','BIRC5','TPX2','TOP2A','NDC80','CKS2','NUF2','CKS1B','MKI67','TMPO','CENPF','TACC3','FAM64A','SMC4','CCNB2','CKAP2L','CKAP2','AURKB','BUB1','KIF11','ANP32E','TUBB4B','GTSE1','KIF20B','HJURP','CDCA3','HN1','CDC20','TTK','CDC25C','KIF2C','RANGAP1','NCAPD2','DLGAP5','CDCA2','CDCA8','ECT2','KIF23','HMMR','AURKA','PSRC1','ANLN','LBR','CKAP5','CENPE','CTCF','NEK2','G2E3','GAS2L3','CBX5','CENPA']
s_genes = cell_cycle_genes[:43]
s_genes = [x for x in s_genes if x in adata.var_names]
print('S genes:', s_genes, '\n')
g2m_genes = cell_cycle_genes[43:]
g2m_genes = [x for x in g2m_genes if x in adata.var_names]
print('G2 genes:', g2m_genes, '\n')
cell_cycle_genes = [x for x in cell_cycle_genes if x in adata.var_names]

hb_genes1 = ['HBB','HBD','HBG1','HBG2','HBE1','HBZ','HBM','HBA2','HBA1', 'HBQ1']
hb_genes1 = [x for x in hb_genes1 if x in adata.var_names]
print('Hemoglobin genes set 1:', hb_genes1, '\n')

hb_genes2 = ['AHSP','GYPA','RHAG','GFI1B','KEL','EPB42','ALAS2','TSPO2','GYPB','GYPE','SPTA1','RHCE','KLF1','RHD','SLC4A1','SNX22','SQSTM1','SOD1','FTH1','MAP1LC3B','SLC25A37','TRIM58','HMBS']
hb_genes2 = [x for x in hb_genes2 if x in adata.var_names]
print('Hemoglobin genes set 2:', hb_genes2, '\n')

mito_genes=['MT-ND1','MT-ND2','MT-CO1','MT-CO2','MT-ATP8','MT-ATP6','MT-CO3','MT-ND3','MT-ND4L', 'MT-ND4', 'MT-ND5', 'MT-ND6', 'MT-CYB']
mito_genes = [x for x in mito_genes if x in adata.var_names]
print('Mitochondrial genes:', mito_genes, '\n')

ribo_genes = ['RPLP0','RPLP1','RPLP2','RPL3','RPL3L','RPL4','RPL5','RPL6','RPL7','RPL7A','RPL7L1','RPL8','RPL9','RPL10','RPL10A','RPL10L','RPL11','RPL12','RPL13','RPL13A','RPL14','RPL15','RPL17','RPL18','RPL18A','RPL19','RPL21','RPL22','RPL22L1','RPL23','RPL23A','RPL24','RPL26','RPL26L1','RPL27','RPL27A','RPL28','RPL29','RPL30','RPL31','RPL32','RPL34','RPL35','RPL35A','RPL36','RPL36A','RPL36AL','RPL37','RPL37A','RPL38','RPL39','RPL39L','UBA52','RPL41','RPSA','RPS2','RPS3','RPS3A','RPS4X','RPS4Y1','RPS4Y2','RPS5','RPS6','RPS7','RPS8','RPS9','RPS10','RPS11','RPS12','RPS13','RPS14','RPS15','RPS15A','RPS16','RPS17','RPS18','RPS19','RPS20','RPS21','RPS23','RPS24','RPS25','RPS26','RPS27','RPS27A','RPS27L','RPS28','RPS29','FAU']
ribo_genes = [x for x in ribo_genes if x in adata.var_names]
print('Ribo genes:', ribo_genes, '\n')

stress_genes=['FOS','FOSB','JUN','JUNB','JUND','ZFP36','SOCS3','CEBPD','CEBPB','ATF3','EGR1','HSPA1A','HSPA1B','HSP90AA1','HSP90AB1','HSPA8','HSPB1','HSPE1','IER3','IER2','BTG1','BTG2','DUSP1']
stress_genes = [x for x in stress_genes if x in adata.var_names]
print('Tissue dissociation stress associated genes:', stress_genes, '\n')

FoxProteins=['FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1']
chrY_genes=['SRY', 'RPS4Y1', 'ZFY', 'ZFY-AS1', 'LINC00278', 'PCDH11Y', 'TTTY2B', 'TBL1Y', 'PRKY', 'USP9Y', 'DDX3Y', 'UTY', 'TMSB4Y', 'NLGN4Y', 'TTTY14', 'KDM5D', 'TTTY10', 'EIF1AY', 'PRORY', 'LINC00266-4P']
MembraneProteins1=['AADAT','AAK1','ABCA12','ABCA7','ABCB1','ABCB11','ABCB4','ABCB6','ABCC1','ABCC3','ABCC4','ABCC5','ABCC6','ABCG2','ABHD3','ABI1','ABLIM3','AC004691.2','AC004922.1','AC005324.3','AC005726.1','AC008403.1','AC138696.1','ACIN1','ACKR3','ACLY','ACOT13','ACOT9','ACTN1','ACTR10','ACTR6','ADA','ADAM10','ADAM12','ADAM22','ADAM28','ADAM33','ADAMTS1','ADAMTS2','ADAMTSL4','ADAP1','ADCK5','ADD1','ADD2','ADGRB2','ADGRD1','ADGRG2','ADGRG7','ADGRL2','ADH1A','ADH1B','ADH1C','ADH7','ADK','ADORA1','ADRA1B','ADRM1','ADSS','AFAP1L2','AFDN','AFG1L','AGER','AGRN','AHNAK','AHNAK2','AIFM2','AKAP10','AKAP11','AKAP12','AKR1B10','AKR1B15','AL132780.3','AL139300.1','AL355315.1','AL358113.1','AL645922.1','ALDH3A1','ALK','ALOX15B','ALOXE3','ALPI','ALPP','ALPPL2','ALS2CR12','AMACR','AMDHD1','AMER1','AMER2','AMIGO2','AMOT','AMOTL1','AMOTL2','AMPH','ANK2','ANK3','ANKEF1','ANKFN1','ANKIB1','ANKLE2','ANKRA2','ANKRD13A','ANKRD35','ANKRD42','ANKRD45','ANKS1B','ANO1','ANO10','ANO2','ANO6','ANPEP','ANXA1','ANXA13','ANXA2','ANXA3','AP2M1','APBB1','APBB1IP','APC','APLP1','APMAP','APOC3','APOD','APOLD1','AQP1','AQP10','AQP3','AQP4','AQP5','ARAP1','ARF1','ARHGAP18','ARHGAP19','ARHGAP21','ARHGAP22','ARHGAP23','ARHGAP24','ARHGAP28','ARHGAP29','ARHGAP33','ARHGAP36','ARHGAP40','ARHGAP9','ARHGDIG','ARHGEF1','ARHGEF11','ARHGEF12','ARHGEF15','ARHGEF26','ARHGEF28','ARHGEF40','ARHGEF5','ARID2','ARID4A','ARID5B','ARL13A','ARL15','ARL4C','ARMC12','ARMCX5','ARPC1A','ARPC5','ARRB2','ARRDC4','ARVCF','ASAP1','ASB5','ASGR1','ASGR2','ASH2L','ASIC1','ASIC4','ASPM','ASPSCR1','ATG13','ATG3','ATG7','ATIC','ATOX1','ATP10D','ATP12A','ATP13A3','ATP1B3','ATP23','ATP2B1','ATP2B4','ATP2C2','ATP6V0A2','ATP6V1H','ATP8A2','ATP8B1','ATXN10','ATXN3','ATXN7L3','AVEN','AVPI1','AXIN2','AXL','B2M','B3GALT2','BACE1','BAG3','BAIAP2','BAIAP2L1','BANK1','BASP1','BBOF1','BCAR1','BCAR3','BCL2L12','BCORL1','BCR','BDKRB2','BEND6','BFSP1','BFSP2','BICD2','BIN2','BLNK','BLVRB','BMPR2','BMX','BOC','BOLL','BORCS5','BORCS8-MEF2B','BRK1','BRPF1','BRSK1','BSDC1','BST2','BTK','BTN2A1','BTRC','BUB1B-PAK6','BVES','BZW2','C10orf90','C11orf42','C11orf52','C12orf73','C12orf76','C14orf37','C14orf93','C16orf74','C17orf58','C17orf62','C18orf21','C19orf33','C19orf38','C19orf48','C19orf71','C1orf116','C1orf189','C1orf35','C1orf54','C1orf68','C1orf74','C1QBP','C2orf72','C3orf14','C3orf20','C3orf38','C4orf19','C4orf22','C7orf43','C8orf33','C8orf37','C9orf153','C9orf3','CA14','CA9','CABLES2','CACNA1E','CACNG4','CADM1','CALCRL','CALD1','CALML5','CALN1','CAMK2A','CAMK2B','CAMK2D','CAMK2G','CAP2','CAPS','CASKIN2','CASP4','CAST','CAV3','CAVIN1','CAVIN2','CAVIN3','CAVIN4','CBWD1','CBWD2','CBWD3','CBWD5','CBWD6','CC2D1A','CCBE1','CCDC107','CCDC112','CCDC115','CCDC124','CCDC136','CCDC144NL','CCDC15','CCDC154','CCDC170','CCDC24','CCDC28A','CCDC59','CCDC62','CCDC66','CCDC69','CCDC8','CCDC83','CCDC85C','CCDC93','CCDC97','CCNT2','CCNYL1','CCR3','CCR4','CCT3','CD109','CD163','CD164L2','CD200R1','CD226','CD248','CD274','CD2AP','CD300A','CD300LB','CD33','CD34','CD37','CD38','CD3E','CD3G','CD4','CD44','CD46','CD47','CD48','CD79B','CD81','CD84','CD86','CD8A','CD8B','CD9','CD93','CD99L2','CDAN1','CDC42EP1','CDC42EP3','CDC42EP4','CDC42SE1','CDCA4','CDH1','CDH13','CDH15','CDH17','CDH2','CDH26','CDH3','CDH4','CDH5','CDH6','CDH8','CDIPT','CDK16','CDK5','CDK5RAP2','CDK7','CDRT15','CDV3','CEACAM5','CELSR1','CENPJ','CEP162','CEP295','CEP55','CEP57L1','CEP89','CERCAM','CFAP157','CFAP206','CFAP58','CFAP65','CFB','CFL2','CFLAR','CGN','CGNL1','CHCHD7','CHIC2','CHMP2A','CHRM3','CHRNA1','CHRNA5','CHRNB4','CHRND','CIB1','CIB3','CIP2A','CKAP5','CLCA2','CLCA4','CLCN5','CLCN6','CLDN11','CLDN18','CLDN2','CLDN23','CLDN3','CLDN4','CLDN7','CLDN9','CLEC2A','CLEC4F','CLEC5A','CLECL1','CLIC2','CLIC5','CLTB','CLUL1','CMTM4','CMTM7','CMTR2','CMYA5','CNGA1','CNKSR2','CNNM1','CNNM4','CNP','CNR1','CNTD1','COBL','COBLL1','COG3','COL16A1','COL17A1','COL4A5','COL6A2','COLEC12','COLQ','COMMD4','COMTD1','COPRS','COQ7','CORIN','CORO1B','CORO2A','CORO2B','COX6B2','CPEB3','CPPED1','CPZ','CRB3','CREB5','CRIP1','CRIP2','CRK','CRLF3','CROCC','CRTC1','CRYAB','CRYBG3','CRYL1','CSDE1','CSF1','CSF1R','CSPG4','CST6','CTNNA1','CTNNB1','CTNNBIP1','CTNND1','CTNND2','CTSF','CTSV','CTTN','CTXN1','CUEDC1','CUTA','CX3CL1','CX3CR1','CXADR','CXCR2','CYB5D2','CYBRD1','CYFIP2','CYP20A1','CYP24A1','CYTH2','CYTH4','DAAM1','DAB2','DAG1','DAGLB','DAW1','DBN1','DCAF13','DCAF16','DCBLD2','DCHS1','DCHS2','DCP2','DCT','DCTN6','DCUN1D2','DDAH1','DDB2','DDIAS','DDIT4L','DDR1','DDR2','DDX58','DDX60L','DENND6B','DFFA','DGKG','DGKH','DHDDS','DHPS','DHRS7B','DIAPH1','DIAPH3','DIO2','DIS3L','DLG1','DLG2','DLG5','DLGAP3','DLL3','DMWD','DNAAF1','DNAAF4','DNAH17','DNAJA4','DNAJB13','DNAJB4','DNAJC18','DNAJC5','DNAJC6','DNAJC9','DOC2A','DOCK4','DOK3','DONSON','DPEP1','DPH1','DPPA5','DPYSL2','DQX1','DRD1','DRP2','DSC3','DSEL','DSG2','DSP','DTNA','DUS1L','DUSP15','DXO','DYSF','ECSCR','EDA2R','EDNRA','EDNRB','EEF1AKMT1','EEF2','EFCAB1','EFCAB11','EFCAB2','EFCAB7','EFHC2','EFNB1','EFR3A','EFR3B','EGFR','EHBP1','EHD1','EHD2','EHD3','EHD4','EIF2B2','EIF5','EIF5B','ELFN1','ELMO3','ELN','ELOC','EMP3','ENAH','ENC1','ENG','ENKD1','ENO1','ENO2','ENO3','ENOX1','ENOX2','EPB41','EPB41L1','EPB41L2','EPB41L3','EPB41L4A','EPB41L4B','EPB41L5','EPCAM','EPHA2','EPHA3','EPHB1','EPHB2','EPN1','EPOR','EPS8L2','ERAP1','ERBB2','ERBB3','ERBIN','ERC1','ERCC4','ERMN','ESCO2','ETNK1','ETS2','EVA1A','EVI2B','EXOC1','EXOC3L1','EXOC7','EZR','F11R','F2RL1','F2RL3','FAAP20','FABP2','FABP3','FABP5','FAIM','FAM126A','FAM126B','FAM129A','FAM129B','FAM13A','FAM163B','FAM171A1','FAM193A','FAM205A','FAM210B','FAM222A','FAM45A','FAM47E','FAM47E-STBD1','FAM72A','FAM72B','FAM72C','FAM72D','FANCG','FAS','FASN','FBLIM1','FBLN2','FBLN5','FBLN7','FBP2','FBRSL1','FBXL2','FBXO16','FBXO38','FBXO41','FBXW11','FCAR','FCGBP','FCGR1A','FCGR1B','FCGR2A','FCHSD2','FFAR4','FGD1','FGD5','FGFR2','FGR','FHIT','FHL1','FIBCD1','FIGN','FILIP1','FILIP1L','FKBP2','FLAD1','FLCN','FLNA','FLNB','FLNC','FLOT1','FLOT2','FLRT3','FLT1','FLT4','FLVCR1','FMN2','FMNL2','FMNL3','FNBP1','FOXA2','FOXN3','FP565260.3','FRMD3','FRMD7','FRMD8','FRS2','FSCN1','FTCD','FUT2','FXYD3','FYB1','FYN','FZD1','FZD2','FZD4','FZD6','GAB1','GAB2','GABRA1','GABRA2','GABRA3','GABRA5','GABRG1','GAD1','GADL1','GAGE1','GAGE10','GAGE12C','GAGE12D','GAGE12E','GAGE12F','GAGE12G','GAGE12H','GAGE12J','GAGE13','GAGE2A','GAGE2E','GALNT10','GALR1','GAP43','GAPDH','GAPT','GAPVD1','GAS2','GAS2L1','GAS7','GAS8','GBP4','GCA','GCC1','GCLM','GCSAML-AS1','GFI1B','GFOD1','GFRA3','GHR','GID8','GINM1','GIPC1','GJA1','GJB3','GJB4','GJB6','GJB7','GJC1','GJC2','GLO1','GLP2R','GLRX','GLUL','GMIP','GNAI2','GNAQ','GNAS','GNB1','GNB2','GNB3','GNB4','GNG2','GP9','GPAT3','GPATCH1','GPBP1','GPBP1L1','GPC1','GPC3','GPC4','GPI','GPR1','GPR142','GPR156','GPR157','GPR160','GPR173','GPR20','GPR27','GPR50','GPR63','GPRC5A','GPRIN1','GPRIN2','GPX8','GRAMD1A','GRAMD1B','GRAMD1C','GRAP2','GRB7','GRID2IP','GRIK5','GRK3','GRK5','GRPR','GSDMA','GTF2F1','GTSE1','GUCA1A','GYPA','GYPC','HAPLN3','HARBI1','HAS1','HAUS4','HAUS7','HCAR2','HCAR3','HCFC2','HCK','HCLS1','HCST','HDAC8','HEG1','HELZ','HENMT1','HERC2','HERPUD1','HFE','HIF1A','HIF3A','HINT1','HIP1R','HLA-A','HLA-DPB1','HMGCS1','HMOX1','HOMER3','HOXC5','HOXD4','HPS3','HRASLS5','HRH1','HS6ST3','HSH2D','HSPB1','HSPG2','HTR1B','HTR7','HTRA1','HUNK','HYDIN','HYI','HYLS1','ICAM1','ICAM2','ICOS','ICOSLG','IFFO2','IFIT5','IFNGR1','IFRD1','IFT122','IGFL2','IGSF11','IGSF23','IKZF5','IL10RA','IL12RB1','IL12RB2','IL13','IL13RA2','IL16','IL17RB','IL1R1','IL1RL1','IL21R','IL4R','IL6ST','IL7R','ILDR2','ILK','INPP5A','INSR','INTS12','IP6K2','IPCEF1','IQGAP1','IQGAP2','IREB2','IRF2','IRS1','ISLR','ITGA2B','ITGA3','ITGA4','ITGA7','ITGA8','ITGA9','ITGB1','ITGB2','ITGB3','ITGB4','ITGB5','ITGB6','ITGB7','ITGB8','ITK','ITM2A','ITPKA','ITSN1','ITSN2','IYD','JAG1','JAK2','JAK3','JAM2','JAML','JCAD','JMJD4','JPT2','JUP','KANK1','KANK3','KANSL2','KARS','KATNB1','KCNAB1','KCNB2','KCNH8','KCNJ15','KCNK13','KCNK17','KCNK6','KCNK7','KCNK9','KCNN2','KCNS3','KCNU1','KDF1','KDR','KEL','KIAA0556','KIAA0895','KIAA1324','KIAA1522','KIAA1958','KIF17','KIF19','KIF21A','KIF21B','KIF26A','KIF26B','KISS1R','KIT','KLC1','KLC2','KLF9','KLHDC7B','KLHL41','KLHL6','KLHL7','KLK7','KLRC1','KLRC2','KLRF1','KLRK1','KMT2D','KNDC1','KNSTRN','KNTC1','KPTN','KRT75','KRT77','KRTAP25-1','L1CAM','L3MBTL1','LAIR1','LAMA4','LAMC1','LAMP2','LAMTOR1','LAPTM4A','LASP1','LAT','LAT2','LAX1','LCP1','LCTL','LDB2','LDLRAD3','LENG1','LGALS3','LGALS4','LGALS8','LGR6','LHFPL2','LHX6','LIMA1','LINGO1','LIPJ','LMCD1','LMOD2','LMTK2','LMTK3','LNX1','LNX2','LONRF1','LONRF3','LOXHD1','LPAR1','LPAR3','LPAR4','LPIN1','LPIN2','LRCH2','LRP11','LRP12','LRRC15','LRRC26','LRRC34','LRRC45','LRRC69','LRRC7','LRRC9','LRRFIP2','LRRIQ3','LSMEM2','LSR','LTB4R2','LUZP2','LY6K','LYN','LYPD1','LYST','LZTS1','LZTS2','MACF1','MADD','MAGI1','MAGI3','MAMDC2','MAP1S','MAP2','MAP2K1','MAP2K4','MAP2K7','MAP3K12','MAP3K21','MAP3K6','MAP3K7','MAP4','MAP4K1','MAP4K5','MAP6','MAPK15','MAPK8IP1','MAPT','MARCH10','MARCH7','MARCKS','MARK2','MARVELD2','MAS1L','MAT2B','MBP','MC5R','MCAM','MCEMP1','MCF2L2','MCMBP','MCTS1','MDGA1','ME1','MEF2B','MEIOC','MEPCE','MERTK','MET','METAP1','METAP2','METTL15','METTL25','METTL9','MFAP3L','MFSD1','MFSD2A','MGAT4C','MIB1','MICAL2','MICAL3','MICALL1','MICB','MIEN1','MIGA2','MISP','MKLN1','MKRN3','MLF2','MLKL','MLST8','MMACHC','MMP10','MMP15','MMP24','MMP25','MOAP1','MPHOSPH8','MPHOSPH9','MPI','MPIG6B','MPL','MPP1','MPP5','MPP7','MPZL1','MPZL2','MR1','MRGPRF','MRNIP','MRPL42','MRPL44','MRPL46','MRPS18B','MS4A1','MS4A12','MS4A14','MS4A15','MSN','MSRA','MTCP1','MTFR1L','MTHFR','MTM1','MTPN','MTSS1','MUC1','MUC20','MVD','MYH11','MYH9','MYL4','MYLK','MYO10','MYO1B','MYO1C','MYO1E','MYO1G','MYO3A','MYO5B','MYO9A','MYOF','N4BP2','NAA35','NAB1','NAIF1','NAMPT','NAP1L2','NAPG','NARFL','NCAM1','NCAM2','NCK1','NCKAP1L','NCKAP5','NCKIPSD','NCOA1','NCS1','NDC1','NECTIN2','NECTIN3','NECTIN4','NEDD1','NEDD9','NEO1','NEU1','NEXN','NF2','NFE2L2','NFKBID','NGEF']
MembraneProteins2=['NGFR','NHS','NHSL2','NID2','NINJ2','NKTR','NKX2-8','NLGN1','NLGN3','NLK','NLRX1','NMB','NMBR','NME1','NME2','NMT1','NMT2','NMUR2','NOS1','NOTCH2','NPFFR2','NPL','NPNT','NPR1','NPTN','NPTXR','NR2C1','NRAS','NRCAM','NRP2','NRSN1','NRXN1','NSD1','NSFL1C','NT5E','NTNG1','NTNG2','NTSR1','NUDCD3','NUDT16L1','NUDT9','NUMB','NUP35','NUS1','OAS3','OBSCN','OCA2','OCLN','OCM','OCM2','ODC1','ODR4','OGT','OLFM4','OLIG2','OLR1','OPHN1','OPRK1','OR10P1','OR1N2','OR2C3','OR6B2','OR6B3','OR6C6','OR8S1','OSBPL10','OSBPL6','OSBPL7','OSCAR','OSCP1','OSGEP','OSR2','OTOP2','OTULIN','OXGR1','P3H3','PACSIN2','PACSIN3','PAFAH1B2','PAG1','PAIP1','PAK1','PAK4','PAK6','PALLD','PALM','PALM2','PALM2-AKAP2','PALM3','PALMD','PANX1','PANX2','PAQR8','PARD3','PARD3B','PARD6A','PARD6G','PARP6','PATJ','PAWR','PCBD2','PCDH1','PCDH17','PCDH7','PCDHA1','PCDHA11','PCDHA12','PCDHA4','PCDHA5','PCDHB1','PCDHB11','PCDHB12','PCDHB16','PCDHB4','PCDHB5','PCDHGA1','PCDHGA10','PCDHGA11','PCDHGA12','PCDHGA2','PCDHGA3','PCDHGA4','PCDHGA5','PCDHGA6','PCDHGA7','PCDHGA8','PCDHGA9','PCDHGB1','PCDHGB2','PCDHGB3','PCDHGB4','PCDHGB5','PCDHGB6','PCDHGB7','PCDHGC3','PCDHGC4','PCDHGC5','PCLO','PCMTD1','PDAP1','PDCD7','PDE3A','PDE4A','PDE4D','PDE9A','PDGFC','PDGFRA','PDHX','PDLIM1','PDLIM5','PDPN','PDZD2','PDZD8','PEAR1','PEBP1','PFDN1','PGAP3','PGLYRP2','PGLYRP4','PGRMC2','PHACTR1','PHACTR4','PHF20L1','PHGDH','PHLDB2','PHLDB3','PHOSPHO1','PHPT1','PI3','PI4K2A','PI4KA','PIEZO2','PIGB','PIGN','PIGW','PIGX','PIK3AP1','PIK3CG','PIK3R6','PIP4K2A','PIP4K2C','PIP5K1A','PIRT','PITPNM3','PKD2L2','PKN1','PKN2','PKP1','PKP2','PKP3','PKP4','PLAA','PLAUR','PLCB1','PLCB2','PLCD3','PLCD4','PLEKHA4','PLEKHA6','PLEKHA7','PLEKHG3','PLEKHG4','PLEKHG6','PLEKHH2','PLEKHO2','PLIN4','PLPP1','PLPPR5','PLS3','PLSCR1','PLVAP','PLXNA3','PLXNA4','PNCK','PNISR','PODXL','POLE','POU2F3','PPEF1','PPFIBP1','PPIP5K1','PPL','PPM1A','PPM1F','PPP1CA','PPP1CB','PPP1R12B','PPP1R16A','PPP1R16B','PPP1R18','PPP1R27','PPP1R9A','PPP2R1B','PPP3R1','PPP3R2','PPP4C','PPP6R3','PPTC7','PRAME','PRC1','PRDX6','PRIMA1','PRKCA','PRKCE','PRKCH','PRKCZ','PRKD1','PROM2','PROSER2','PRR15','PRR32','PRR7','PRRC2A','PRRG1','PRRT2','PRRT3','PRRT4','PRTG','PRX','PSCA','PSD','PSD2','PSEN1','PSMC5','PSMC6','PSMD9','PSTPIP2','PTAFR','PTCHD4','PTK2B','PTK6','PTPN12','PTPN21','PTPN22','PTPN4','PTPRB','PTPRG','PTPRK','PTPRM','PTPRR','PTPRS','PTX3','PURG','PVALB','PVR','PXDC1','PXK','PYGL','PYGO2','PYM1','RAB13','RAB23','RAB25','RAB26','RAB27B','RAB3B','RAB3GAP2','RAB43','RAB8A','RABEPK','RAD51C','RAD52','RAI2','RALA','RALGAPA2','RAPGEF2','RAPH1','RAPSN','RASA4','RASA4B','RASAL1','RASAL2','RASAL3','RASD1','RASD2','RASGRP1','RASGRP2','RASGRP4','RASSF2','RASSF3','RASSF4','RBM46','RC3H1','RCCD1','RCSD1','RD3L','RDX','RECK','RELL1','REPS1','RET','RFLNB','RFPL1','RFPL2','RFPL3','RGP1','RGS10','RGS14','RGS19','RGS3','RGS7BP','RGS9BP','RHCE','RHD','RHEX','RHOA','RHOBTB2','RHOJ','RHOXF2','RHOXF2B','RIC8A','RIC8B','RIF1','RILPL1','RIMBP3','RIMBP3B','RIMBP3C','RIMS2','RIOK2','RIOK3','RIPK1','RND1','RNF114','RNF145','RNF146','RNF149','RNF165','RNF166','RNF39','ROBO1','ROBO3','ROBO4','ROM1','RP2','RPE','RPEL1','RPGRIP1L','RPIA','RPL36A','RPL36A-HNRNPH2','RPL36AL','RPSA','RPUSD1','RRAD','RRP12','RRP7A','RSBN1L','RSC1A1','RSPH3','RTL3','RTL4','RTN4R','RTN4RL1','RYR2','S100A13','S100A14','S100A16','S100A2','S100A3','S100A4','S100A6','S100A9','SAMD4A','SAMHD1','SAMSN1','SAPCD1','SASH3','SC5D','SCAMP1','SCAMP4','SCEL','SCHIP1','SCIN','SCN1A','SCN5A','SCN8A','SCN9A','SCNN1D','SCNN1G','SCRIB','SCUBE3','SDC2','SDC4','SDE2','SDHB','SEC16B','SEMA3C','SEMA3D','SEMA4B','SEMA4F','SEMA6C','SEMA6D','SEPT2','SEPT3','SERINC2','SERINC5','SERPINB3','SFSWAP','SGCE','SGK494','SH3BGR','SH3D21','SHANK1','SHANK3','SHARPIN','SHISA6','SHKBP1','SHROOM2','SHTN1','SIGLEC10','SIGLEC14','SIGLEC6','SIGLEC7','SIN3B','SIPA1','SIPA1L1','SIPA1L3','SKAP1','SLAMF6','SLC10A5','SLC12A2','SLC13A3','SLC13A5','SLC14A1','SLC16A1','SLC16A10','SLC16A2','SLC16A3','SLC16A7','SLC16A9','SLC17A5','SLC19A1','SLC19A3','SLC1A5','SLC1A6','SLC22A16','SLC23A3','SLC24A4','SLC26A9','SLC27A1','SLC28A1','SLC2A1','SLC2A11','SLC2A12','SLC2A14','SLC2A2','SLC2A3','SLC30A1','SLC31A2','SLC34A1','SLC34A2','SLC35F5','SLC35G2','SLC36A4','SLC38A5','SLC38A6','SLC38A7','SLC39A10','SLC39A14','SLC39A2','SLC39A6','SLC3A2','SLC40A1','SLC41A3','SLC43A1','SLC43A3','SLC44A2','SLC45A4','SLC46A1','SLC46A3','SLC4A1AP','SLC4A2','SLC4A7','SLC4A8','SLC5A12','SLC5A3','SLC5A6','SLC5A7','SLC6A6','SLC7A14','SLC7A2','SLC7A5','SLC7A6','SLC8A1','SLC9A1','SLC9A2','SLC9A3','SLC9A3R1','SLC9A3R2','SLC9A4','SLC9A5','SLC9B2','SLC9C2','SLCO1A2','SLCO1B3','SLCO4A1','SLCO4C1','SLCO5A1','SLIT1','SLIT3','SLITRK6','SLK','SLX4','SMAGP','SMAP1','SMOC1','SMPD2','SMPX','SNAP23','SNAP25','SNRK','SNTB2','SNTG2','SNX15','SNX32','SNX9','SOCS3','SOCS5','SOD1','SORBS1','SORCS1','SORT1','SOX11','SPAG16','SPANXN4','SPART','SPATA12','SPATC1L','SPERT','SPG11','SPHK1','SPINK13','SPINT1','SPINT2','SPN','SPPL2B','SPPL3','SPRED3','SPRYD3','SPTBN2','SRC','SRCIN1','SRD5A3','SRPK1','SRPRB','SRRM3','SSFA2','SSH1','SSH3','SSTR1','SSTR5','SSX2IP','ST8SIA5','STAB1','STAC','STAC2','STAMBP','STAMBPL1','STARD4','STAT2','STEAP4','STIL','STIP1','STK10','STK16','STK17A','STK17B','STOM','STOML2','STOML3','STX18','STX4','STXBP4','STYK1','SUCLG1','SUCLG2','SUGT1','SUMF2','SUPT6H','SUSD2','SVIL','SWAP70','SYCP2','SYK','SYMPK','SYNC','SYNGR3','SYNPO2L','SYPL2','SYT4','SYTL2','SYVN1','TACC2','TACR2','TACSTD2','TASP1','TAZ','TBC1D10A','TBC1D2','TBC1D24','TBC1D30','TBC1D9B','TBXA2R','TCAF2','TCAF2C','TCHP','TCIM','TDG','TDO2','TDP1','TEC','TEDC2','TEK','TES','TEX14','TEX9','TGFBR1','TGFBR2','TGM1','TGM2','THBS1','THBS2','THEGL','THY1','TIAM1','TICAM1','TIGD7','TIMD4','TJP1','TJP2','TJP3','TLN1','TLR4','TM2D2','TM4SF20','TM4SF4','TM4SF5','TMC5','TMED8','TMEM101','TMEM107','TMEM108','TMEM121B','TMEM139','TMEM141','TMEM164','TMEM176B','TMEM183A','TMEM186','TMEM2','TMEM218','TMEM222','TMEM229A','TMEM245','TMEM251','TMEM254','TMEM258','TMEM266','TMEM268','TMEM50A','TMEM63B','TMPRSS15','TMPRSS2','TMTC2','TNFRSF10D','TNFRSF11A','TNFRSF12A','TNFRSF21','TNFRSF25','TNFRSF8','TNFRSF9','TNFSF13B','TNK1','TNKS1BP1','TNNI2','TOGARAM2','TOLLIP','TOM1','TP53BP2','TPD52L1','TPRN','TRAF7','TRAIP','TRAPPC13','TRAT1','TRDN','TREML1','TRIB1','TRIM16','TRIM16L','TRIM38','TRIM4','TRIM44','TRIM54','TRIM71','TRIML2','TRIP6','TRMT1','TRMT10B','TRMT13','TRMT6','TRO','TRPM1','TRPM4','TRPM7','TRPV2','TRPV3','TSG101','TSGA13','TSHZ3','TSPAN15','TSPAN6','TSPO2','TSR2','TSSK4','TTC17','TTC23','TTC7B','TTLL12','TTLL5','TULP3','TWF2','TXNDC11','TXNIP','TYROBP','TYW3','UAP1','UBA3','UBA52','UBAC1','UBALD1','UBAP1','UBASH3B','UBE2A','UBE2B','UBE2C','UBE2D1','UBE2D2','UBE2D3','UBE2D4','UBE2K','UBE2QL1','UBE2S','UBE3C','UBQLN2','UBR2','UBTD1','UCP3','ULBP1','ULBP2','UNC5A','UNC79','UQCC1','USP14','USP15','USP2','USP20','USP21','USP4','USP53','UTP20','UTRN','VAMP5','VAMP8','VANGL1','VASP','VCAM1','VCPIP1','VIL1','VPS13B','VPS36','VPS50','VPS8','VSIG4','VTCN1','VWA5B1','VWA7','WAS','WASF2','WBP4','WDFY3','WDPCP','WDR1','WDR12','WDR13','WDR53','WDR6','WDR66','WIPF1','WWP1','XIAP','XIRP1','XIRP2','XPC','XPNPEP2','XPO6','XRN1','XYLT1','YES1','YIPF1','YIPF4','YOD1','YTHDC1','ZBED3','ZBTB33','ZBTB8OS','ZDHHC2','ZDHHC20','ZDHHC22','ZDHHC5','ZFP41','ZFP90','ZFYVE28','ZGPAT','ZMYND10','ZNF10','ZNF18','ZNF185','ZNF2','ZNF20','ZNF227','ZNF229','ZNF236','ZNF276','ZNF277','ZNF337','ZNF397','ZNF407','ZNF497','ZNF503','ZNF510','ZNF536','ZNF571','ZNF598','ZNF599','ZNF646','ZNF655','ZNF688','ZNF706','ZNF717','ZNF783','ZNF79','ZNF839','ZSCAN22','ZSCAN29','ZSWIM1','ZSWIM8','ZYX']
MembraneProteins=MembraneProteins1+MembraneProteins2

TFs1=['AC008770.3','AC023509.3','AC092835.1','AC138696.1','ADNP','ADNP2','AEBP1','AEBP2','AHCTF1','AHDC1','AHR','AHRR','AIRE','AKAP8','AKAP8L','AKNA','ALX1','ALX3','ALX4','ANHX','ANKZF1','AR','ARGFX','ARHGAP35','ARID2','ARID3A','ARID3B','ARID3C','ARID5A','ARID5B','ARNT','ARNT2','ARNTL','ARNTL2','ARX','ASCL1','ASCL2','ASCL3','ASCL4','ASCL5','ASH1L','ATF1','ATF2','ATF3','ATF4','ATF5','ATF6','ATF6B','ATF7','ATMIN','ATOH1','ATOH7','ATOH8','BACH1','BACH2','BARHL1','BARHL2','BARX1','BARX2','BATF','BATF2','BATF3','BAZ2A','BAZ2B','BBX','BCL11A','BCL11B','BCL6','BCL6B','BHLHA15','BHLHA9','BHLHE22','BHLHE23','BHLHE40','BHLHE41','BNC1','BNC2','BORCS8-MEF2B','BPTF','BRF2','BSX','C11orf95','CAMTA1','CAMTA2','CARF','CASZ1','CBX2','CC2D1A','CCDC169-SOHLH2','CCDC17','CDC5L','CDX1','CDX2','CDX4','CEBPA','CEBPB','CEBPD','CEBPE','CEBPG','CEBPZ','CENPA','CENPB','CENPBD1','CENPS','CENPT','CENPX','CGGBP1','CHAMP1','CHCHD3','CIC','CLOCK','CPEB1','CPXCR1','CREB1','CREB3','CREB3L1','CREB3L2','CREB3L3','CREB3L4','CREB5','CREBL2','CREBZF','CREM','CRX','CSRNP1','CSRNP2','CSRNP3','CTCF','CTCFL','CUX1','CUX2','CXXC1','CXXC4','CXXC5','DACH1','DACH2','DBP','DBX1','DBX2','DDIT3','DEAF1','DLX1','DLX2','DLX3','DLX4','DLX5','DLX6','DMBX1','DMRT1','DMRT2','DMRT3','DMRTA1','DMRTA2','DMRTB1','DMRTC2','DMTF1','DNMT1','DNTTIP1','DOT1L','DPF1','DPF3','DPRX','DR1','DRAP1','DRGX','DUX1','DUX3','DUX4','DUXA','DZIP1','E2F1','E2F2','E2F3','E2F4','E2F5','E2F6','E2F7','E2F8','E4F1','EBF1','EBF2','EBF3','EBF4','EEA1','EGR1','EGR2','EGR3','EGR4','EHF','ELF1','ELF2','ELF3','ELF4','ELF5','ELK1','ELK3','ELK4','EMX1','EMX2','EN1','EN2','EOMES','EPAS1','ERF','ERG','ESR1','ESR2','ESRRA','ESRRB','ESRRG','ESX1','ETS1','ETS2','ETV1','ETV2','ETV3','ETV3L','ETV4','ETV5','ETV6','ETV7','EVX1','EVX2','FAM170A','FAM200B','FBXL19','FERD3L','FEV','FEZF1','FEZF2','FIGLA','FIZ1','FLI1','FLYWCH1','FOS','FOSB','FOSL1','FOSL2','FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXN4','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1','GABPA','GATA1','GATA2','GATA3','GATA4','GATA5','GATA6','GATAD2A','GATAD2B','GBX1','GBX2','GCM1','GCM2','GFI1','GFI1B','GLI1','GLI2','GLI3','GLI4','GLIS1','GLIS2','GLIS3','GLMP','GLYR1','GMEB1','GMEB2','GPBP1','GPBP1L1','GRHL1','GRHL2','GRHL3','GSC','GSC2','GSX1','GSX2','GTF2B','GTF2I','GTF2IRD1','GTF2IRD2','GTF2IRD2B','GTF3A','GZF1','HAND1','HAND2','HBP1','HDX','HELT','HES1','HES2','HES3','HES4','HES5','HES6','HES7','HESX1','HEY1','HEY2','HEYL','HHEX','HIC1','HIC2','HIF1A','HIF3A','HINFP','HIVEP1','HIVEP2','HIVEP3','HKR1','HLF','HLX','HMBOX1','HMG20A','HMG20B','HMGA1','HMGA2','HMGN3','HMX1','HMX2','HMX3','HNF1A','HNF1B','HNF4A','HNF4G','HOMEZ','HOXA1','HOXA10','HOXA11','HOXA13','HOXA2','HOXA3','HOXA4','HOXA5','HOXA6','HOXA7','HOXA9','HOXB1','HOXB13','HOXB2','HOXB3','HOXB4','HOXB5','HOXB6','HOXB7','HOXB8','HOXB9','HOXC10','HOXC11','HOXC12','HOXC13','HOXC4','HOXC5','HOXC6','HOXC8','HOXC9','HOXD1','HOXD10','HOXD11','HOXD12','HOXD13','HOXD3','HOXD4','HOXD8','HOXD9','HSF1','HSF2','HSF4','HSF5','HSFX1','HSFX2','HSFY1','HSFY2','IKZF1','IKZF2','IKZF3','IKZF4','IKZF5','INSM1','INSM2','IRF1','IRF2','IRF3','IRF4','IRF5','IRF6','IRF7','IRF8','IRF9','IRX1','IRX2','IRX3','IRX4','IRX5','IRX6','ISL1','ISL2','ISX','JAZF1','JDP2','JRK','JRKL','JUN','JUNB','JUND','KAT7','KCMF1','KCNIP3','KDM2A','KDM2B','KDM5B','KIN','KLF1','KLF10','KLF11','KLF12','KLF13','KLF14','KLF15','KLF16','KLF17','KLF2','KLF3','KLF4','KLF5','KLF6','KLF7','KLF8','KLF9','KMT2A','KMT2B','L3MBTL1','L3MBTL3','L3MBTL4','LBX1','LBX2','LCOR','LCORL','LEF1','LEUTX','LHX1','LHX2','LHX3','LHX4','LHX5','LHX6','LHX8','LHX9','LIN28A','LIN28B','LIN54','LMX1A','LMX1B','LTF','LYL1','MAF','MAFA','MAFB','MAFF','MAFG','MAFK','MAX','MAZ','MBD1','MBD2','MBD3','MBD4','MBD6','MBNL2','MECOM','MECP2','MEF2A','MEF2B','MEF2C','MEF2D','MEIS1','MEIS2','MEIS3','MEOX1','MEOX2','MESP1','MESP2','MGA','MITF','MIXL1','MKX','MLX','MLXIP','MLXIPL','MNT','MNX1','MSANTD1','MSANTD3','MSANTD4','MSC','MSGN1','MSX1','MSX2','MTERF1','MTERF2','MTERF3','MTERF4','MTF1','MTF2','MXD1','MXD3','MXD4','MXI1','MYB','MYBL1','MYBL2','MYC','MYCL','MYCN','MYF5','MYF6','MYNN','MYOD1','MYOG','MYPOP','MYRF','MYRFL','MYSM1','MYT1','MYT1L','MZF1','NACC2','NAIF1','NANOG','NANOGNB','NANOGP8','NCOA1','NCOA2','NCOA3','NEUROD1','NEUROD2','NEUROD4','NEUROD6','NEUROG1','NEUROG2','NEUROG3','NFAT5','NFATC1','NFATC2','NFATC3','NFATC4','NFE2','NFE2L1','NFE2L2','NFE2L3','NFE4','NFIA','NFIB','NFIC','NFIL3','NFIX','NFKB1','NFKB2','NFX1','NFXL1','NFYA','NFYB','NFYC','NHLH1','NHLH2','NKRF','NKX1-1','NKX1-2','NKX2-1','NKX2-2','NKX2-3','NKX2-4','NKX2-5','NKX2-6','NKX2-8','NKX3-1','NKX3-2','NKX6-1','NKX6-2','NKX6-3','NME2','NOBOX','NOTO','NPAS1','NPAS2','NPAS3','NPAS4','NR0B1','NR1D1','NR1D2','NR1H2','NR1H3','NR1H4','NR1I2','NR1I3','NR2C1','NR2C2','NR2E1','NR2E3','NR2F1','NR2F2','NR2F6','NR3C1','NR3C2','NR4A1','NR4A2','NR4A3','NR5A1','NR5A2','NR6A1','NRF1','NRL','OLIG1','OLIG2','OLIG3','ONECUT1','ONECUT2','ONECUT3','OSR1','OSR2','OTP','OTX1','OTX2','OVOL1','OVOL2','OVOL3','PA2G4','PATZ1','PAX1','PAX2','PAX3','PAX4','PAX5','PAX6','PAX7','PAX8','PAX9','PBX1','PBX2','PBX3','PBX4','PCGF2','PCGF6','PDX1','PEG3','PGR','PHF1','PHF19 ','PHF20','PHF21A','PHOX2A','PHOX2B','PIN1','PITX1','PITX2','PITX3','PKNOX1','PKNOX2','PLAG1','PLAGL1','PLAGL2','PLSCR1','POGK','POU1F1','POU2AF1','POU2F1','POU2F2','POU2F3','POU3F1','POU3F2','POU3F3','POU3F4','POU4F1','POU4F2','POU4F3','POU5F1','POU5F1B','POU5F2','POU6F1','POU6F2','PPARA','PPARD','PPARG','PRDM1','PRDM10','PRDM12','PRDM13','PRDM14','PRDM15','PRDM16','PRDM2','PRDM4','PRDM5','PRDM6','PRDM8','PRDM9','PREB','PRMT3','PROP1','PROX1','PROX2','PRR12','PRRX1','PRRX2','PTF1A','PURA','PURB','PURG','RAG1','RARA','RARB','RARG','RAX','RAX2','RBAK','RBCK1','RBPJ','RBPJL','RBSN','REL','RELA','RELB','REPIN1','REST','REXO4','RFX1','RFX2','RFX3','RFX4','RFX5','RFX6','RFX7','RFX8','RHOXF1','RHOXF2','RHOXF2B','RLF','RORA','RORB','RORC','RREB1','RUNX1','RUNX2','RUNX3','RXRA','RXRB','RXRG','SAFB','SAFB2','SALL1','SALL2','SALL3','SALL4','SATB1','SATB2','SCMH1','SCML4','SCRT1','SCRT2','SCX','SEBOX','SETBP1','SETDB1','SETDB2','SGSM2','SHOX','SHOX2','SIM1','SIM2','SIX1','SIX2','SIX3','SIX4','SIX5','SIX6','SKI','SKIL','SKOR1','SKOR2','SLC2A4RG','SMAD1','SMAD3','SMAD4','SMAD5','SMAD9','SMYD3','SNAI1','SNAI2','SNAI3','SNAPC2','SNAPC4','SNAPC5','SOHLH1','SOHLH2','SON','SOX1','SOX10','SOX11']
TFs2=['SOX12','SOX13','SOX14','SOX15','SOX17','SOX18','SOX2','SOX21','SOX3','SOX30','SOX4','SOX5','SOX6','SOX7','SOX8','SOX9','SP1','SP100','SP110','SP140','SP140L','SP2','SP3','SP4','SP5','SP6','SP7','SP8','SP9','SPDEF','SPEN','SPI1','SPIB','SPIC','SPZ1','SRCAP','SREBF1','SREBF2','SRF','SRY','ST18','STAT1','STAT2','STAT3','STAT4','STAT5A','STAT5B','STAT6','T','TAL1','TAL2','TBP','TBPL1','TBPL2','TBR1','TBX1','TBX10','TBX15','TBX18','TBX19','TBX2','TBX20','TBX21','TBX22','TBX3','TBX4','TBX5','TBX6','TCF12','TCF15','TCF20','TCF21','TCF23','TCF24','TCF3','TCF4','TCF7','TCF7L1','TCF7L2','TCFL5','TEAD1','TEAD2','TEAD3','TEAD4','TEF','TERB1','TERF1','TERF2','TET1','TET2','TET3','TFAP2A','TFAP2B','TFAP2C','TFAP2D','TFAP2E','TFAP4','TFCP2','TFCP2L1','TFDP1','TFDP2','TFDP3','TFE3','TFEB','TFEC','TGIF1','TGIF2','TGIF2LX','TGIF2LY','THAP1','THAP10','THAP11','THAP12','THAP2','THAP3','THAP4','THAP5','THAP6','THAP7','THAP8','THAP9','THRA','THRB','THYN1','TIGD1','TIGD2','TIGD3','TIGD4','TIGD5','TIGD6','TIGD7','TLX1','TLX2','TLX3','TMF1','TOPORS','TP53','TP63','TP73','TPRX1','TRAFD1','TRERF1','TRPS1','TSC22D1','TSHZ1','TSHZ2','TSHZ3','TTF1','TWIST1','TWIST2','UBP1','UNCX','USF1','USF2','USF3','VAX1','VAX2','VDR','VENTX','VEZF1','VSX1','VSX2','WIZ','WT1','XBP1','XPA','YBX1','YBX2','YBX3','YY1','YY2','ZBED1','ZBED2','ZBED3','ZBED4','ZBED5','ZBED6','ZBED9','ZBTB1','ZBTB10','ZBTB11','ZBTB12','ZBTB14','ZBTB16','ZBTB17','ZBTB18','ZBTB2','ZBTB20','ZBTB21','ZBTB22','ZBTB24','ZBTB25','ZBTB26','ZBTB3','ZBTB32','ZBTB33','ZBTB34','ZBTB37','ZBTB38','ZBTB39','ZBTB4','ZBTB40','ZBTB41','ZBTB42','ZBTB43','ZBTB44','ZBTB45','ZBTB46','ZBTB47','ZBTB48','ZBTB49','ZBTB5','ZBTB6','ZBTB7A','ZBTB7B','ZBTB7C','ZBTB8A','ZBTB8B','ZBTB9','ZC3H8','ZEB1','ZEB2','ZFAT','ZFHX2','ZFHX3','ZFHX4','ZFP1','ZFP14','ZFP2','ZFP28','ZFP3','ZFP30','ZFP37','ZFP41','ZFP42','ZFP57','ZFP62','ZFP64','ZFP69','ZFP69B','ZFP82','ZFP90','ZFP91','ZFP92','ZFPM1','ZFPM2','ZFX','ZFY','ZGLP1','ZGPAT','ZHX1','ZHX2','ZHX3','ZIC1','ZIC2','ZIC3','ZIC4','ZIC5','ZIK1','ZIM2','ZIM3','ZKSCAN1','ZKSCAN2','ZKSCAN3','ZKSCAN4','ZKSCAN5','ZKSCAN7','ZKSCAN8','ZMAT1','ZMAT4','ZNF10','ZNF100','ZNF101','ZNF107','ZNF112','ZNF114','ZNF117','ZNF12','ZNF121','ZNF124','ZNF131','ZNF132','ZNF133','ZNF134','ZNF135','ZNF136','ZNF138','ZNF14','ZNF140','ZNF141','ZNF142','ZNF143','ZNF146','ZNF148','ZNF154','ZNF155','ZNF157','ZNF16','ZNF160','ZNF165','ZNF169','ZNF17','ZNF174','ZNF175','ZNF177','ZNF18','ZNF180','ZNF181','ZNF182','ZNF184','ZNF189','ZNF19','ZNF195','ZNF197','ZNF2','ZNF20','ZNF200','ZNF202','ZNF205','ZNF207','ZNF208','ZNF211','ZNF212','ZNF213','ZNF214','ZNF215','ZNF217','ZNF219','ZNF22','ZNF221','ZNF222','ZNF223','ZNF224','ZNF225','ZNF226','ZNF227','ZNF229','ZNF23','ZNF230','ZNF232','ZNF233','ZNF234','ZNF235','ZNF236','ZNF239','ZNF24','ZNF248','ZNF25','ZNF250','ZNF251','ZNF253','ZNF254','ZNF256','ZNF257','ZNF26','ZNF260','ZNF263','ZNF264','ZNF266','ZNF267','ZNF268','ZNF273','ZNF274','ZNF275','ZNF276','ZNF277','ZNF28','ZNF280A','ZNF280B','ZNF280C','ZNF280D','ZNF281','ZNF282','ZNF283','ZNF284','ZNF285','ZNF286A','ZNF286B','ZNF287','ZNF292','ZNF296','ZNF3','ZNF30','ZNF300','ZNF302','ZNF304','ZNF311','ZNF316','ZNF317','ZNF318','ZNF319','ZNF32','ZNF320','ZNF322','ZNF324','ZNF324B','ZNF326','ZNF329','ZNF331','ZNF333','ZNF334','ZNF335','ZNF337','ZNF33A','ZNF33B','ZNF34','ZNF341','ZNF343','ZNF345','ZNF346','ZNF347','ZNF35','ZNF350','ZNF354A','ZNF354B','ZNF354C','ZNF358','ZNF362','ZNF365','ZNF366','ZNF367','ZNF37A','ZNF382','ZNF383','ZNF384','ZNF385A','ZNF385B','ZNF385C','ZNF385D','ZNF391','ZNF394','ZNF395','ZNF396','ZNF397','ZNF398','ZNF404','ZNF407','ZNF408','ZNF41','ZNF410','ZNF414','ZNF415','ZNF416','ZNF417','ZNF418','ZNF419','ZNF420','ZNF423','ZNF425','ZNF426','ZNF428','ZNF429','ZNF43','ZNF430','ZNF431','ZNF432','ZNF433','ZNF436','ZNF438','ZNF439','ZNF44','ZNF440','ZNF441','ZNF442','ZNF443','ZNF444','ZNF445','ZNF446','ZNF449','ZNF45','ZNF451','ZNF454','ZNF460','ZNF461','ZNF462','ZNF467','ZNF468','ZNF469','ZNF470','ZNF471','ZNF473','ZNF474','ZNF479','ZNF48','ZNF480','ZNF483','ZNF484','ZNF485','ZNF486','ZNF487','ZNF488','ZNF490','ZNF491','ZNF492','ZNF493','ZNF496','ZNF497','ZNF500','ZNF501','ZNF502','ZNF503','ZNF506','ZNF507','ZNF510','ZNF511','ZNF512','ZNF512B','ZNF513','ZNF514','ZNF516','ZNF517','ZNF518A','ZNF518B','ZNF519','ZNF521','ZNF524','ZNF525','ZNF526','ZNF527','ZNF528','ZNF529','ZNF530','ZNF532','ZNF534','ZNF536','ZNF540','ZNF541','ZNF543','ZNF544','ZNF546','ZNF547','ZNF548','ZNF549','ZNF550','ZNF551','ZNF552','ZNF554','ZNF555','ZNF556','ZNF557','ZNF558','ZNF559','ZNF560','ZNF561','ZNF562','ZNF563','ZNF564','ZNF565','ZNF566','ZNF567','ZNF568','ZNF569','ZNF57','ZNF570','ZNF571','ZNF572','ZNF573','ZNF574','ZNF575','ZNF576','ZNF577','ZNF578','ZNF579','ZNF580','ZNF581','ZNF582','ZNF583','ZNF584','ZNF585A','ZNF585B','ZNF586','ZNF587','ZNF587B','ZNF589','ZNF592','ZNF594','ZNF595','ZNF596','ZNF597','ZNF598','ZNF599','ZNF600','ZNF605','ZNF606','ZNF607','ZNF608','ZNF609','ZNF610','ZNF611','ZNF613','ZNF614','ZNF615','ZNF616','ZNF618','ZNF619','ZNF620','ZNF621','ZNF623','ZNF624','ZNF625','ZNF626','ZNF627','ZNF628','ZNF629','ZNF630','ZNF639','ZNF641','ZNF644','ZNF645','ZNF646','ZNF648','ZNF649','ZNF652','ZNF653','ZNF654','ZNF655','ZNF658','ZNF66','ZNF660','ZNF662','ZNF664','ZNF665','ZNF667','ZNF668','ZNF669','ZNF670','ZNF671','ZNF672','ZNF674','ZNF675','ZNF676','ZNF677','ZNF678','ZNF679','ZNF680','ZNF681','ZNF682','ZNF683','ZNF684','ZNF687','ZNF688','ZNF689','ZNF69','ZNF691','ZNF692','ZNF695','ZNF696','ZNF697','ZNF699','ZNF7','ZNF70','ZNF700','ZNF701','ZNF703','ZNF704','ZNF705A','ZNF705B','ZNF705D','ZNF705E','ZNF705G','ZNF706','ZNF707','ZNF708','ZNF709','ZNF71','ZNF710','ZNF711','ZNF713','ZNF714','ZNF716','ZNF717','ZNF718','ZNF721','ZNF724','ZNF726','ZNF727','ZNF728','ZNF729','ZNF730','ZNF732','ZNF735','ZNF736','ZNF737','ZNF74','ZNF740','ZNF746','ZNF747','ZNF749','ZNF750','ZNF75A','ZNF75D','ZNF76','ZNF761','ZNF763','ZNF764','ZNF765','ZNF766','ZNF768','ZNF77','ZNF770','ZNF771','ZNF772','ZNF773','ZNF774','ZNF775','ZNF776','ZNF777','ZNF778','ZNF780A','ZNF780B','ZNF781','ZNF782','ZNF783','ZNF784','ZNF785','ZNF786','ZNF787','ZNF788','ZNF789','ZNF79','ZNF790','ZNF791','ZNF792','ZNF793','ZNF799','ZNF8','ZNF80','ZNF800','ZNF804A','ZNF804B','ZNF805','ZNF808','ZNF81','ZNF813','ZNF814','ZNF816','ZNF821','ZNF823','ZNF827','ZNF829','ZNF83','ZNF830','ZNF831','ZNF835','ZNF836','ZNF837','ZNF84','ZNF841','ZNF843','ZNF844','ZNF845','ZNF846','ZNF85','ZNF850','ZNF852','ZNF853','ZNF860','ZNF865','ZNF878','ZNF879','ZNF880','ZNF883','ZNF888','ZNF891','ZNF90','ZNF91','ZNF92','ZNF93','ZNF98','ZNF99','ZSCAN1','ZSCAN10','ZSCAN12','ZSCAN16','ZSCAN18','ZSCAN2','ZSCAN20','ZSCAN21','ZSCAN22','ZSCAN23','ZSCAN25','ZSCAN26','ZSCAN29','ZSCAN30','ZSCAN31','ZSCAN32','ZSCAN4','ZSCAN5A','ZSCAN5B','ZSCAN5C','ZSCAN9','ZUFSP','ZXDA','ZXDB','ZXDC','ZZZ3']
TFs=TFs1+TFs2

#TFs other, inclusive of the above list, not necessarily DNA binding proteins but affect transcription
TFs1Other=['AATF','ABL1','ABRA','AC008770.3','AC020915.1','AC023509.3','AC092835.1','AC105001.2','AC138696.1','ACOD1','ACTR3B','ACTR3C','ADAM8','ADAMTS17','ADAMTS19','ADAR','ADGRA1','ADGRG3','ADNP','ADNP2','AEBP1','AEBP2','AES','AFAP1 ','AFF1','AFF3','AFF4','AGER','AGT','AHCTF1','AHDC1','AHR','AHRR','AIM2','AIRE','AKAP8','AKAP8L','AKNA','AKT1','ALK','ALKBH2','ALX1','ALX3','ALX4','AMH','ANAPC2','ANHX','ANKRD30A','ANKZF1','ANXA3','ANXA4','AP5Z1','APBB3','APC2','APEX1','APOA4','APOBEC3A','APOBEC3B','APOBEC3G','APTX','AR','ARGFX','ARHGAP35','ARHGEF2','ARHGEF5','ARID1A','ARID1B','ARID2','ARID3A','ARID3B','ARID3C','ARID4A','ARID4B','ARID5A','ARID5B','ARNT','ARNT2','ARNTL','ARNTL2','ARRB1','ARRB2','ARX','ASCC1','ASCL1','ASCL2','ASCL3','ASCL4','ASCL5','ASH1L','ATF1','ATF2','ATF3','ATF4','ATF5','ATF6','ATF6B','ATF7','ATF7IP','ATM','ATMIN','ATOH1','ATOH7','ATOH8','ATR','ATRX','ATXN7','BACH1','BACH2','BAG1','BANF1','BANP','BARHL1','BARHL2','BARX1','BARX2','BATF','BATF2','BATF3','BAZ2A','BAZ2B','BBX','BCL10','BCL11A','BCL11B','BCL3','BCL6','BCL6B','BCLAF1','BCOR','BDP1','BEND4','BEX1','BHLHA15','BHLHA9','BHLHE22','BHLHE23','BHLHE40','BHLHE41','BHMG1','BIN1','BLM','BLZF1','BMI1','BMP2','BMP4','BMP7','BMPR1A','BNC1','BNC2','BOD1L1','BOLA1','BOLA2','BOLA2B','BOLA3','BORCS8-MEF2B','BPNT1','BPTF','BRCA1','BRD4','BRD7','BRD8','BRD9','BRF1','BRF2','BRMS1','BRPF1','BSX','BTAF1','BTBD8','BTG2','BTK','BTRC','BUD31','C11orf95','C14orf39','C17orf49','C1D','C20orf194','C3orf33','CACTIN','CAMK1D','CAMK2A','CAMTA1','CAMTA2','CAPN15','CAPN3','CARD11','CARD14','CARF','CARHSP1','CASP8AP2','CASZ1','CAT','CBFA2T2','CBFA2T3','CBFB','CBL','CBLL1','CBX2','CC2D1A','CC2D1B','CCDC124','CCDC169-SOHLH2','CCDC17','CCDC3','CCDC71','CCDC83','CCNT1','CCNT2','CD36','CD40','CD40LG','CDC5L','CDC6','CDK5RAP3','CDK7','CDK8','CDK9','CDKN1A','CDKN2A','CDX1','CDX2','CDX4','CEBPA','CEBPB','CEBPD','CEBPE','CEBPG','CEBPZ','CELF3','CENPA','CENPB','CENPBD1','CENPS','CENPT','CENPX','CERS2','CERS3','CERS4','CERS5','CERS6','CFLAR','CGGBP1','CHAMP1','CHCHD3','CHD1','CHD2','CHD3','CHD4','CHD6','CHD7','CHD9','CHP1','CHRAC1','CHUK','CHURC1','CIB1','CIC','CIITA','CIR1','CITED1','CITED2','CITED4','CIZ1','CLOCK','CLU','CMKLR1','CNBP','CNOT3','CNOT4','CNOT7','CNOT8','COMMD1','COMMD6','COMMD7','COPS2','COPS3','COPS4','CPEB1','CPSF4','CPSF4L','CPXCR1','CRAMP1','CREB1','CREB3','CREB3L1','CREB3L2','CREB3L3','CREB3L4','CREB5','CREBBP','CREBL2','CREBRF','CREBZF','CREG1','CREM','CRTC1','CRTC2','CRTC3','CRX','CSDC2','CSDE1','CSRNP1','CSRNP2','CSRNP3','CTBP1','CTBP2','CTCF','CTCFL','CTH','CTNNB1','CTNNBIP1','CUL1','CUL2','CUL3','CUL4A','CUL4B','CUL5','CUX1','CUX2','CXXC1','CXXC4','CXXC5','CYLD','CYP1B1','CYTL1','DAB2IP','DACH1','DACH2','DAP','DAXX','DBP','DBX1','DBX2','DCP1A','DDB1','DDB2','DDIT3','DDN','DDR2','DDRGK1','DDX3X','DDX5','DDX58','DEAF1','DEK','DEPDC1','DEPDC1B','DEPDC4','DEPDC5','DEPDC7','DEPTOR','DHX34','DHX36','DHX57','DISP3','DLX1','DLX2','DLX3','DLX4','DLX5','DLX6','DMAP1','DMBX1','DMC1','DMRT1','DMRT2','DMRT3','DMRTA1','DMRTA2','DMRTB1','DMRTC1','DMRTC2','DMTF1','DNA2','DNAJA3','DNAJC1','DNAJC2','DNAJC21','DNMT1','DNMT3A','DNMT3B','DNMT3L','DNTT','DNTTIP1','DOT1L','DPF1','DPF2','DPF3','DPRX','DR1','DRAP1','DRGX','DSP','DTX1','DUS3L','DUSP12','DUX1','DUX3','DUX4','DUXA','DVL1','DVL2','DVL3','DZIP1','DZIP1L','E2F1','E2F2','E2F3','E2F4','E2F5','E2F6','E2F7','E2F8','E4F1','EAF2','EBF1','EBF2','EBF3','EBF4','EDA','EDA2R','EDF1','EDN1','EEA1','EED','EGLN1','EGR1','EGR2','EGR3','EGR4','EHF','EID1','EID2','EIF2AK2','EIF2AK4','EIF3C','EIF3CL','EIF3K','ELF1','ELF2','ELF3','ELF4','ELF5','ELK1','ELK3','ELK4','ELMSAN1','EME1','EMX1','EMX2','EN1','EN2','ENO1','EOMES','EP300','EP400','EPAS1','EPHA5','EPM2AIP1','ERBIN','ERC1','ERCC3','ERCC4','ERF','ERG','ESR1','ESR2','ESRRA','ESRRB','ESRRG','ESX1','ETS1','ETS2','ETV1','ETV2','ETV3','ETV3L','ETV4','ETV5','ETV6','ETV7','EVX1','EVX2','EWSR1','EXO1','EXOC2','EXOG','EZH1','EZH2','FAAP24','FAM153C','FAM170A','FAM171B','FAM200A','FAM200B','FAN1','FANCA','FANCD2','FANK1','FARSA','FARSB','FBN1','FBXL19','FBXO41','FEN1','FER','FERD3L','FEV','FEZF1','FEZF2','FGD1','FHL2','FIGLA','FIZ1','FLI1','FLNA','FLYWCH1','FMNL2','FOS','FOSB','FOSL1','FOSL2','FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L2','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXN4','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1','FUBP1','FUBP3','FUS','FZD1','FZD2','FZD4','FZD6','GABPA','GABPB1','GABPB2','GADD45A','GAS6','GAS7','GATA1','GATA2','GATA3','GATA4','GATA5','GATA6','GATAD1','GATAD2A','GATAD2B','GBX1','GBX2','GCFC2','GCM1','GCM2','GEN1','GFI1','GFI1B','GLA','GLI1','GLI2','GLI3','GLI4','GLIS1','GLIS2','GLIS3','GLMP','GLYR1','GMEB1','GMEB2','GOLGB1','GON4L','GPATCH8','GPBP1','GPBP1L1','GPER1','GPR155','GREM1','GRHL1','GRHL2','GRHL3','GRM6','GSC','GSC2','GSX1','GSX2','GTF2A1','GTF2A1L','GTF2A2','GTF2B','GTF2E2','GTF2F1','GTF2F2','GTF2H1','GTF2H2','GTF2H2C','GTF2H3','GTF2H4','GTF2I','GTF2IRD1','GTF2IRD2','GTF2IRD2B','GTF3A','GTF3C1','GTF3C2','GULP1','GZF1','H1F0','H1FOO','H1FX','HAND1','HAND2','HAVCR2','HBP1','HCFC1','HCK','HDAC1','HDAC10','HDAC2','HDAC3','HDAC4','HDAC5','HDAC6','HDAC7','HDAC8','HDAC9','HDX','HELT','HELZ','HELZ2','HES1','HES2','HES3','HES4','HES5','HES6','HES7','HESX1','HEY1','HEY2','HEYL','HHEX','HIC1','HIC2','HIF1A','HIF3A','HINFP','HIPK2','HIRA','HIST1H1A','HIST1H1B','HIST1H1C','HIST1H1D','HIST1H1E','HIST1H1T','HIVEP1','HIVEP2','HIVEP3','HKR1','HLA-DQB1','HLA-DQB2','HLA-DRB1','HLA-DRB3','HLA-DRB5','HLF','HLTF','HLX','HMBOX1','HMG20A','HMG20B','HMGA1','HMGA2','HMGB1','HMGB2','HMGB3','HMGB4','HMGN1','HMGN3','HMGXB3','HMGXB4','HMOX1','HMX1','HMX2','HMX3','HNF1A','HNF1B','HNF4A','HNF4G','HNRNPA1','HNRNPA3','HNRNPAB','HNRNPCL2','HNRNPD','HNRNPDL','HNRNPK','HNRNPL','HNRNPU','HOMEZ','HOOK2','HOPX','HOXA1','HOXA10','HOXA11','HOXA13','HOXA2','HOXA3','HOXA4','HOXA5','HOXA6','HOXA7','HOXA9','HOXB1','HOXB13','HOXB2','HOXB3','HOXB4','HOXB5','HOXB6','HOXB7','HOXB8','HOXB9','HOXC10','HOXC11','HOXC12','HOXC13','HOXC4','HOXC5','HOXC6','HOXC8','HOXC9','HOXD1','HOXD10','HOXD11','HOXD12','HOXD13','HOXD3','HOXD4','HOXD8','HOXD9','HP1BP3','HR','HSBP1','HSF1','HSF2','HSF4','HSF5','HSFX1','HSFX2','HSFY1','HSFY2','HSPA1A','HSPA1B','ICAM1','ID1','ID2','ID3','ID4','IFI16','IFNB1','IGHM','IKBKB','IKBKG','IKZF1','IKZF2','IKZF3','IKZF4','IKZF5','IL10','IL1B','IL1RAP','IL4','IL5','IL6','INF2','ING4','INS','INSM1','INSM2','IQGAP1','IRAK1','IRAK2','IRAK3','IRF1','IRF2','IRF2BP1','IRF3','IRF4','IRF5','IRF6','IRF7','IRF8','IRF9','IRX1','IRX2','IRX3','IRX4','IRX5','IRX6','ISL1','ISL2','ISX','ITCH','ITGB2','ITGB3BP','IZUMO2','JADE1','JAK2','JARID2','JAZF1','JDP2','JMY','JRK','JRKL','JUN','JUNB','JUND','JUP','KANSL2','KAT2A','KAT2B','KAT5','KAT6A','KAT6B','KAT7','KAT8','KCMF1','KCNIP3','KDM1A','KDM2A','KDM2B','KDM3A','KDM5A','KDM5B','KDM5C','KDM5D','KEAP1','KIAA1549','KIAA1683','KIAA1841','KIAA1958','KIF15','KIN','KIT','KLF1','KLF10','KLF11','KLF12','KLF13','KLF14','KLF15','KLF16','KLF17','KLF2','KLF3','KLF4','KLF5','KLF6','KLF7','KLF8','KLF9','KMT2A','KMT2B','KMT2C','KMT2D','KNTC1','KRAS','KRBOX1','KRT13','KRTAP5-1','KRTAP5-9','KRTAP9-9','L3MBTL1','L3MBTL3','L3MBTL4','LARP1','LARP1B','LARP4','LARP4B','LARP6','LARP7','LBX1','LBX2','LCOR','LCORL','LEF1','LEUTX','LGALS9','LGR4','LHX1','LHX2','LHX3','LHX4','LHX5','LHX6','LHX8','LHX9','LIG1','LIG3','LIN28A','LIN28B','LIN54','LMO2','LMO4','LMX1A','LMX1B','LPP','LRP5','LRP6','LRP8','LRRFIP1','LRRFIP2','LSR','LTF','LTN1','LYAR','LYL1','LZTFL1','LZTR1','LZTS1','MACC1','MACF1','MAD2L2','MAEL','MAF','MAFA','MAFB','MAFF','MAFG','MAFK','MALT1','MAML1','MAML3','MAP2K5','MAP3K10','MAP3K13','MAP3K7','MAPK1','MAPK10','MAPK11','MAPK14','MAPK3','MAPK8','MAPK9','MATR3','MAVS','MAX','MAZ','MB21D1','MBD1','MBD2','MBD3','MBD4','MBD6','MBNL1','MBNL2','MBNL3','MBTPS2','MDFI','MDM2','MDM4','MECOM','MECP2','MED1','MED12','MED13','MED13L','MED14','MED15','MED16','MED17','MED21','MED23','MED24','MED6','MEF2A','MEF2B','MEF2C','MEF2D','MEIS1','MEIS2','MEIS3','MEN1','MEOX1','MEOX2','MESP1','MESP2','MET','MGA','MGMT','MID2','MIER1','MIER2','MIER3','MINK1','MIS18BP1','MITF','MIXL1','MKL1','MKL2','MKRN1','MKRN2','MKRN3','MKX','MLLT1','MLLT10','MLLT11','MLLT3','MLLT6','MLX','MLXIP','MLXIPL','MN1','MNAT1','MNT','MNX1','MORC3','MPG','MRPL28','MRRF','MSANTD1','MSANTD3','MSANTD4','MSC','MSGN1','MSH2','MSH3','MSH6','MSL3','MSRB2','MST1R','MSX1','MSX2','MTA1','MTA2','MTA3','MTDH','MTERF1','MTERF2','MTERF3','MTERF4','MTF1','MTF2','MTPN','MUS81','MXD1','MXD3','MXD4','MXI1','MYB','MYBBP1A','MYBL1','MYBL2','MYC','MYCBP','MYCL','MYCN','MYD88','MYEF2','MYF5','MYF6','MYNN','MYOCD','MYOD1','MYOG','MYPOP','MYRF','MYRFL','MYSM1','MYT1','MYT1L','MZF1','NAB2','NABP2','NACC2','NAIF1','NANOG','NANOGNB','NANOGP8','NCOA1','NCOA2','NCOA3','NCOA4','NCOA5','NCOA6','NCOA7','NCOR1','NCOR2','NDC80','NDN','NDP','NEB','NEIL1','NEUROD1','NEUROD2','NEUROD4','NEUROD6','NEUROG1','NEUROG2','NEUROG3','NFAM1','NFAT5','NFATC1','NFATC2','NFATC3','NFATC4','NFE2','NFE2L1','NFE2L2','NFE2L3','NFE4','NFIA','NFIB','NFIC','NFIL3','NFIX','NFKB1','NFKB2','NFKBIA','NFKBIB','NFKBID','NFKBIL1','NFKBIZ','NFRKB','NFX1','NFXL1','NFYA','NFYB','NFYC','NHLH1','NHLH2','NKRF','NKX1-1','NKX1-2','NKX2-1','NKX2-2','NKX2-3','NKX2-4','NKX2-5','NKX2-6','NKX2-8','NKX3-1','NKX3-2','NKX6-1','NKX6-2','NKX6-3','NLRC3','NLRC4','NLRC5','NLRP12','NLRP2B','NLRP3','NME2','NOBOX','NOC3L','NOC4L','NOCT','NOD1','NOD2','NODAL','NONO','NOP2','NOTCH1','NOTCH2','NOTCH3','NOTO','NPAS1','NPAS2','NPAS3','NPAS4','NPM1','NR0B1','NR0B2','NR1D1','NR1D2','NR1H2','NR1H3','NR1H4','NR1I2','NR1I3','NR2C1','NR2C2','NR2E1','NR2E3','NR2F1','NR2F2','NR2F6','NR3C1','NR3C2','NR4A1','NR4A2','NR4A3','NR5A1','NR5A2','NR6A1','NRBF2','NRF1','NRG1','NRIP1','NRIP3','NRK','NRL','NSD1','NSD2','NTRK1','NUCKS1','NUFIP1','NUP98','NUPL2']
TFs2Other=['OGG1','OLIG1','OLIG2','OLIG3','ONECUT1','ONECUT2','ONECUT3','OPRD1','OSR1','OSR2','OTOP3','OTP','OTULIN','OTX1','OTX2','OVOL1','OVOL2','OVOL3','PA2G4','PABPN1','PARK7','PARP1','PARP10','PARP12','PATZ1','PAWR','PAX1','PAX2','PAX3','PAX4','PAX5','PAX6','PAX7','PAX8','PAX9','PAXBP1','PAXIP1','PBRM1','PBX1','PBX2','PBX3','PBX4','PCBD1','PCBP1','PCBP2','PCBP3','PCGF2','PCGF6','PCNA','PCSK6','PDCD7','PDS5B','PDX1','PEG3','PELI1','PER1','PER2','PER3','PEX14','PFDN1','PGBD1','PGR','PGS1','PHB','PHB2','PHF1','PHF19 ','PHF20','PHF21A','PHF5A','PHOX2A','PHOX2B','PHTF1','PHTF2','PIAS1','PIAS2','PIAS3','PIAS4','PIDD1','PIKFYVE','PIM1','PIN1','PINK1','PITX1','PITX2','PITX3','PKHD1','PKHD1L1','PKNOX1','PKNOX2','PLA2G10','PLA2G1B','PLAG1','PLAGL1','PLAGL2','PLEK','PLEK2','PLEKHA4','PLPP3','PLSCR1','PLXNA1','PLXNA2','PLXNA3','PLXNA4','PLXNB1','PLXNB2','PLXNB3','PLXNC1','PLXND1','PML','PMS1','PNKP','POGK','POGZ','POLA1','POLB','POLE3','POLE4','POLG','POLH','POLI','POLK','POLL','POLM','POLN','POLQ','POLR1B','POLR2A','POLR2B','POLR2E','POLR2J','POLR2K','POLR2L','POLR3A','POLRMT','POT1','POU1F1','POU2AF1','POU2F1','POU2F2','POU2F3','POU3F1','POU3F2','POU3F3','POU3F4','POU4F1','POU4F2','POU4F3','POU5F1','POU5F1B','POU5F2','POU6F1','POU6F2','PPARA','PPARD','PPARG','PPARGC1A','PPARGC1B','PPP1R10','PPP1R13L','PPP1R21','PPP2R3B','PPP2R5B','PPP3CA','PPRC1','PQBP1','PRAM1','PRAP1','PRB3','PRDM1','PRDM10','PRDM11','PRDM12','PRDM13','PRDM14','PRDM15','PRDM16','PRDM2','PRDM4','PRDM5','PRDM6','PRDM7','PRDM8','PRDM9','PRDX3','PREB','PREX1','PREX2','PRH2','PRIM2','PRIMPOL','PRKCB','PRKCH','PRKCI','PRKCQ','PRKCZ','PRKD1','PRKD2','PRKN','PRMT2','PRMT3','PRNP','PROP1','PROX1','PROX2','PRR12','PRR3','PRRX1','PRRX2','PSMA6','PSMC5','PSMD10','PSMD11','PSMD12','PSPC1','PTCH1','PTEN','PTF1A','PTGIS','PTH','PTTG1','PUM3','PURA','PURB','PURG','PYCARD','PYDC1','PYDC2','QRICH1','RAB11FIP4','RAB7B','RAD21','RAD51','RAG1','RAI1','RAPGEF3','RAPGEF4','RAPGEF5','RARA','RARB','RARG','RAX','RAX2','RB1','RBAK','RBBP7','RBCK1','RBL1','RBL2','RBM10','RBM20','RBM22','RBM26','RBM27','RBM5','RBM6','RBPJ','RBPJL','RBSN','RC3H1','RC3H2','RCAN1','RCOR1','RCOR2','RCOR3','RECQL','REL','RELA','RELB','RELL2','RELN','REPIN1','RERE','REST','REV1','REXO4','RFC1','RFX1','RFX2','RFX3','RFX4','RFX5','RFX6','RFX7','RFX8','RFXANK','RFXAP','RGCC','RGS11','RGS6','RGS7','RGS9','RHEBL1','RHOXF1','RHOXF2','RHOXF2B','RING1','RIOK2','RIPK1','RIPK2','RIPK3','RIPK4','RLF','RLIM','RNASE2','RNASEH1','RNF113A','RNF113B','RNF114','RNF125','RNF138','RNF14','RNF166','RNF2','RNF219','RNF25','RNF31','RNF4','RNF41','RORA','RORB','RORC','RPA2','RPA4','RPL36A','RPL7','RPL7A','RPS27A','RPS6KA4','RPS6KA5','RREB1','RRN3','RUNX1','RUNX1T1','RUNX2','RUNX3','RWDD3','RXRA','RXRB','RXRG','RYBP','S100A12','S100A8','S100A9','SAFB','SAFB2','SALL1','SALL2','SALL3','SALL4','SART1','SATB1','SATB2','SAV1','SCAND1','SCAND2P','SCAPER','SCMH1','SCML1','SCML2','SCML4','SCOC','SCRT1','SCRT2','SCX','SEBOX','SEMA4A','SERTAD1','SERTAD2','SETBP1','SETD6','SETDB1','SETDB2','SF3A2','SF3A3','SFPQ','SFRP4','SFRP5','SFTPB','SGK1','SGK3','SGSM2','SHH','SHOX','SHOX2','SHPRH','SIGIRR','SIK1','SIM1','SIM2','SIN3A','SIN3B','SIRT1','SIRT2','SIRT3','SIRT6','SIVA1','SIX1','SIX2','SIX3','SIX4','SIX5','SIX6','SKI','SKIDA1','SKIL','SKOR1','SKOR2','SLC22A4','SLC26A10','SLC26A3','SLC2A4RG','SLC30A9','SLC39A10','SLC4A10','SMAD1','SMAD2','SMAD3','SMAD4','SMAD5','SMAD6','SMAD7','SMAD9','SMARCA1','SMARCA2','SMARCA4','SMARCA5','SMARCAL1','SMARCB1','SMARCC1','SMARCC2','SMARCD1','SMARCD2','SMARCD3','SMARCE1','SMC3','SMO','SMUG1','SMYD1','SMYD3','SNAI1','SNAI2','SNAI3','SNAPC2','SNAPC4','SNAPC5','SNRPC','SNURF','SNW1','SOGA3','SOHLH1','SOHLH2','SON','SORBS2','SOX1','SOX10','SOX11','SOX12','SOX13','SOX14','SOX15','SOX17','SOX18','SOX2','SOX21','SOX3','SOX30','SOX4','SOX5','SOX6','SOX7','SOX8','SOX9','SP1','SP100','SP110','SP140','SP140L','SP2','SP3','SP4','SP5','SP6','SP7','SP8','SP9','SPDEF','SPEN','SPHK1','SPI1','SPIB','SPIC','SPZ1','SRCAP','SREBF1','SREBF2','SRF','SRY','SSB','SSBP2','SSBP3','SSBP4','SSH1','SSH2','SSH3','SSRP1','SSX1','ST18','STAG1','STAT1','STAT2','STAT3','STAT4','STAT5A','STAT5B','STAT6','STK16','STK3','STK36','STRN','STRN3','SUB1','SUFU','SUMO1','SUPT16H','SUPT4H1','SUPT5H','SUPT6H','SUV39H1','SUV39H2','SUZ12','SYK','T','TAB1','TAB2','TAB3','TADA2A','TADA2B','TADA3','TAF1','TAF10','TAF11','TAF12','TAF13','TAF1A','TAF1B','TAF1C','TAF1L','TAF2','TAF3','TAF4','TAF4B','TAF5','TAF5L','TAF6','TAF6L','TAF7','TAF7L','TAF8','TAF9','TAF9B','TAL1','TAL2','TARDBP','TAX1BP1','TBL1Y','TBP','TBPL1','TBPL2','TBR1','TBX1','TBX10','TBX15','TBX18','TBX19','TBX2','TBX20','TBX21','TBX22','TBX3','TBX4','TBX5','TBX6','TCEA2','TCEAL1','TCEAL7','TCEAL8','TCF12','TCF15','TCF19','TCF20','TCF21','TCF23','TCF24','TCF25','TCF3','TCF4','TCF7','TCF7L1','TCF7L2','TCFL5','TDG','TDP1','TDP2','TEAD1','TEAD2','TEAD3','TEAD4','TEF','TERB1','TERF1','TERF2','TERF2IP','TESMIN','TET1','TET2','TET3','TFAM','TFAP2A','TFAP2B','TFAP2C','TFAP2D','TFAP2E','TFAP4','TFCP2','TFCP2L1','TFDP1','TFDP2','TFDP3','TFE3','TFEB','TFEC','TGFB1','TGFB1I1','TGIF1','TGIF2','TGIF2LX','TGIF2LY','TGS1','THAP1','THAP10','THAP11','THAP12','THAP2','THAP3','THAP4','THAP5','THAP6','THAP7','THAP8','THAP9','THRA','THRAP3','THRB','THYN1','TIA1','TICAM1','TIGD1','TIGD2','TIGD3','TIGD4','TIGD5','TIGD6','TIGD7','TIPARP','TIRAP','TLE1','TLE2','TLE3','TLE4','TLE6','TLR2','TLR3','TLR4','TLR9','TLX1','TLX2','TLX3','TMEM235','TMF1','TNF','TNFAIP3','TNFRSF11A','TNFRSF4','TNFSF11','TNFSF18','TNFSF4','TOE1','TONSL','TOP1','TOP2A','TOP2B','TOPORS','TOX','TOX2','TOX3','TOX4','TP53','TP53BP1','TP63','TP73','TPRX1','TRADD','TRAF1','TRAF2','TRAF3','TRAF4','TRAF5','TRAF6','TRAFD1','TRAK2','TRAPPC9','TRERF1','TREX1','TRIB1','TRIB3','TRIM13','TRIM14','TRIM15','TRIM21','TRIM22','TRIM23','TRIM24','TRIM25','TRIM26','TRIM27','TRIM28','TRIM29','TRIM3','TRIM31','TRIM32','TRIM33','TRIM34','TRIM37','TRIM38','TRIM40','TRIM5','TRIM52','TRIM62','TRIM8','TRIP11','TRIT1','TRMT1','TRMT1L','TRPS1','TRRAP','TSC2','TSC22D1','TSC22D2','TSC22D3','TSC22D4','TSHZ1','TSHZ2','TSHZ3','TSN','TSSK4','TTF1','TUB','TULP4','TUT1','TWIST1','TWIST2','TXK','U2AF1','U2AF1L4','UBA52','UBB','UBC','UBE2K','UBE2N','UBE2V1','UBN1','UBP1','UBR4','UBTF','UBTFL1','UBTFL6','UFL1','UHRF1','UHRF2','UNCX','UNG','UNK','UNKL','USF1','USF2','USF3','USP39','USP7','UTF1','VAV1','VAX1','VAX2','VDR','VEGFA','VENTX','VEZF1','VGLL1','VGLL2','VGLL3','VHLL','VPS72','VSX1','VSX2','WDHD1','WFS1','WIZ','WNT1','WNT10B','WNT2','WNT3A','WNT5A','WNT8B','WRN','WT1','WWOX','WWP2','WWTR1','XBP1','XCL1','XPA','YBX1','YBX2','YBX3','YEATS2','YEATS4','YOD1','YY1','YY2','ZBED1','ZBED2','ZBED3','ZBED4','ZBED5','ZBED6','ZBED6CL','ZBED8','ZBED9','ZBP1','ZBTB1','ZBTB10','ZBTB11','ZBTB12','ZBTB14','ZBTB16','ZBTB17','ZBTB18','ZBTB2','ZBTB20','ZBTB21','ZBTB22','ZBTB24','ZBTB25','ZBTB26','ZBTB3','ZBTB32','ZBTB33','ZBTB34','ZBTB37','ZBTB38','ZBTB39','ZBTB4','ZBTB40','ZBTB41','ZBTB42','ZBTB43','ZBTB44','ZBTB45','ZBTB46','ZBTB47','ZBTB48','ZBTB49','ZBTB5','ZBTB6','ZBTB7A','ZBTB7B','ZBTB7C','ZBTB8A','ZBTB8B','ZBTB9','ZC3H10','ZC3H11A','ZC3H12A','ZC3H13','ZC3H14','ZC3H15','ZC3H18','ZC3H3','ZC3H4','ZC3H6','ZC3H7A','ZC3H7B','ZC3H8','ZC3HAV1','ZCCHC11','ZCCHC6','ZDHHC11','ZDHHC19','ZEB1','ZEB2','ZFAND5','ZFAT','ZFHX2','ZFHX3','ZFHX4','ZFP1','ZFP14','ZFP2','ZFP28','ZFP3','ZFP30','ZFP36','ZFP36L1','ZFP36L2','ZFP37','ZFP41','ZFP42','ZFP57','ZFP62','ZFP64','ZFP69','ZFP69B','ZFP82','ZFP90','ZFP91','ZFP92','ZFPL1','ZFPM1','ZFPM2','ZFR','ZFR2','ZFX','ZFY','ZFYVE26','ZGLP1','ZGPAT','ZHX1','ZHX2','ZHX3','ZIC1','ZIC2','ZIC3','ZIC4','ZIC5','ZIK1','ZIM2','ZIM3','ZKSCAN1','ZKSCAN2','ZKSCAN3','ZKSCAN4','ZKSCAN5','ZKSCAN7','ZKSCAN8','ZMAT1','ZMAT2','ZMAT3','ZMAT4','ZMAT5','ZMYM1','ZMYM2','ZMYM3','ZMYM4','ZMYM5','ZMYM6','ZMYND10','ZNF10','ZNF100','ZNF101','ZNF106','ZNF107','ZNF112','ZNF114','ZNF117','ZNF12','ZNF121','ZNF124','ZNF131','ZNF132','ZNF133','ZNF134','ZNF135','ZNF136','ZNF137P','ZNF138','ZNF14','ZNF140','ZNF141','ZNF142','ZNF143','ZNF146','ZNF148','ZNF154','ZNF155','ZNF157','ZNF16','ZNF160','ZNF165','ZNF169','ZNF17','ZNF174','ZNF175','ZNF177','ZNF18','ZNF180','ZNF181','ZNF182','ZNF184','ZNF185','ZNF189','ZNF19','ZNF195','ZNF197','ZNF2','ZNF20','ZNF200','ZNF202','ZNF205','ZNF207','ZNF208','ZNF211','ZNF212','ZNF213','ZNF214','ZNF215','ZNF217','ZNF219','ZNF22','ZNF221','ZNF222','ZNF223','ZNF224','ZNF225','ZNF226','ZNF227','ZNF229','ZNF23','ZNF230','ZNF232','ZNF233','ZNF234','ZNF235','ZNF236','ZNF239','ZNF24','ZNF248','ZNF25','ZNF250','ZNF251','ZNF253','ZNF254','ZNF256','ZNF257','ZNF26','ZNF260','ZNF263','ZNF264','ZNF266','ZNF267','ZNF268','ZNF273','ZNF274','ZNF275','ZNF276','ZNF277','ZNF28','ZNF280A','ZNF280B','ZNF280C','ZNF280D','ZNF281','ZNF282','ZNF283','ZNF284','ZNF285','ZNF286A','ZNF286B','ZNF287','ZNF292','ZNF296','ZNF3','ZNF30','ZNF300','ZNF302','ZNF304','ZNF311','ZNF316','ZNF317','ZNF318','ZNF319','ZNF32','ZNF320','ZNF322','ZNF324','ZNF324B','ZNF326','ZNF329','ZNF330','ZNF331','ZNF333','ZNF334','ZNF335','ZNF337','ZNF33A','ZNF33B','ZNF34','ZNF341','ZNF343','ZNF345','ZNF346','ZNF347','ZNF35','ZNF350','ZNF354A','ZNF354B','ZNF354C','ZNF355P','ZNF358','ZNF362','ZNF365','ZNF366','ZNF367','ZNF37A','ZNF382','ZNF383','ZNF384','ZNF385A','ZNF385B','ZNF385C','ZNF385D','ZNF391','ZNF394','ZNF395','ZNF396','ZNF397','ZNF398','ZNF404','ZNF407','ZNF408','ZNF41','ZNF410','ZNF414','ZNF415','ZNF416','ZNF417','ZNF418','ZNF419','ZNF420','ZNF423','ZNF425','ZNF426','ZNF428','ZNF429','ZNF43','ZNF430','ZNF431','ZNF432','ZNF433','ZNF436','ZNF438','ZNF439','ZNF44','ZNF440','ZNF441','ZNF442','ZNF443','ZNF444','ZNF445','ZNF446','ZNF449','ZNF45','ZNF451','ZNF454','ZNF460','ZNF461','ZNF462','ZNF467','ZNF468','ZNF469','ZNF470','ZNF471','ZNF473','ZNF474','ZNF479','ZNF48','ZNF480','ZNF483','ZNF484','ZNF485','ZNF486','ZNF487','ZNF488','ZNF490','ZNF491','ZNF492','ZNF493','ZNF496','ZNF497','ZNF500','ZNF501','ZNF502','ZNF503','ZNF506','ZNF507','ZNF510','ZNF511','ZNF512','ZNF512B','ZNF513','ZNF514','ZNF516','ZNF517','ZNF518A','ZNF518B','ZNF519']
TFs3Other=['ZNF521','ZNF524','ZNF525','ZNF526','ZNF527','ZNF528','ZNF529','ZNF530','ZNF532','ZNF534','ZNF536','ZNF540','ZNF541','ZNF542P','ZNF543','ZNF544','ZNF546','ZNF547','ZNF548','ZNF549','ZNF550','ZNF551','ZNF552','ZNF554','ZNF555','ZNF556','ZNF557','ZNF558','ZNF559','ZNF560','ZNF561','ZNF562','ZNF563','ZNF564','ZNF565','ZNF566','ZNF567','ZNF568','ZNF569','ZNF57','ZNF570','ZNF571','ZNF572','ZNF573','ZNF574','ZNF575','ZNF576','ZNF577','ZNF578','ZNF579','ZNF580','ZNF581','ZNF582','ZNF583','ZNF584','ZNF585A','ZNF585B','ZNF586','ZNF587','ZNF587B','ZNF589','ZNF592','ZNF593','ZNF594','ZNF595','ZNF596','ZNF597','ZNF598','ZNF599','ZNF600','ZNF605','ZNF606','ZNF607','ZNF608','ZNF609','ZNF610','ZNF611','ZNF613','ZNF614','ZNF615','ZNF616','ZNF618','ZNF619','ZNF620','ZNF621','ZNF622','ZNF623','ZNF624','ZNF625','ZNF626','ZNF627','ZNF628','ZNF629','ZNF630','ZNF638','ZNF639','ZNF641','ZNF644','ZNF645','ZNF646','ZNF648','ZNF649','ZNF652','ZNF653','ZNF654','ZNF655','ZNF658','ZNF66','ZNF660','ZNF662','ZNF664','ZNF665','ZNF667','ZNF668','ZNF669','ZNF670','ZNF671','ZNF672','ZNF674','ZNF675','ZNF676','ZNF677','ZNF678','ZNF679','ZNF680','ZNF681','ZNF682','ZNF683','ZNF684','ZNF687','ZNF688','ZNF689','ZNF69','ZNF691','ZNF692','ZNF695','ZNF696','ZNF697','ZNF699','ZNF7','ZNF70','ZNF700','ZNF701','ZNF703','ZNF704','ZNF705A','ZNF705B','ZNF705D','ZNF705E','ZNF705G','ZNF706','ZNF707','ZNF708','ZNF709','ZNF71','ZNF710','ZNF711','ZNF713','ZNF714','ZNF716','ZNF717','ZNF718','ZNF720','ZNF721','ZNF724','ZNF726','ZNF727','ZNF728','ZNF729','ZNF73','ZNF730','ZNF732','ZNF735','ZNF736','ZNF737','ZNF738','ZNF74','ZNF740','ZNF746','ZNF747','ZNF749','ZNF750','ZNF75A','ZNF75D','ZNF76','ZNF761','ZNF763','ZNF764','ZNF765','ZNF766','ZNF768','ZNF77','ZNF770','ZNF771','ZNF772','ZNF773','ZNF774','ZNF775','ZNF776','ZNF777','ZNF778','ZNF780A','ZNF780B','ZNF781','ZNF782','ZNF783','ZNF784','ZNF785','ZNF786','ZNF787','ZNF788','ZNF789','ZNF79','ZNF790','ZNF791','ZNF792','ZNF793','ZNF799','ZNF8','ZNF80','ZNF800','ZNF804A','ZNF804B','ZNF805','ZNF808','ZNF81','ZNF812P','ZNF813','ZNF814','ZNF816','ZNF821','ZNF823','ZNF827','ZNF829','ZNF83','ZNF830','ZNF831','ZNF835','ZNF836','ZNF837','ZNF839','ZNF84','ZNF841','ZNF843','ZNF844','ZNF845','ZNF846','ZNF85','ZNF850','ZNF852','ZNF853','ZNF860','ZNF862','ZNF865','ZNF878','ZNF879','ZNF880','ZNF883','ZNF888','ZNF891','ZNF90','ZNF91','ZNF92','ZNF93','ZNF98','ZNF99','ZNFX1','ZRANB2','ZRSR1','ZRSR2','ZSCAN1','ZSCAN10','ZSCAN12','ZSCAN16','ZSCAN18','ZSCAN2','ZSCAN20','ZSCAN21','ZSCAN22','ZSCAN23','ZSCAN25','ZSCAN26','ZSCAN29','ZSCAN30','ZSCAN31','ZSCAN32','ZSCAN4','ZSCAN5A','ZSCAN5B','ZSCAN5C','ZSCAN5DP','ZSCAN9','ZUFSP','ZXDA','ZXDB','ZXDC','ZZZ3']
TFs4Other=['ATF2','ATF6B','ATN1','BHLHA15','BHLHB2','BHLHB4 ','BHLHB5','BHLHB9','C13ORF8','C20ORF194','CDX2','CRAMP1L','ELP3','ERVK3-1','ETF1','FLJ12895','FLJ20321','FLJ25476','FLJ35784','FOXM1','FOXO2','FOXR1','FOXR2','GATA1','GRLF1','KDM4C','KDM4E','KIAA0863','KIAA0961','KLRG1','LBXCOR1','LEF1','LHX7','LOC126295','LOC51058','LOC91661','MGC4728','MYOD1','MYOG','MYRF','NEUROD4','NEUROG1','NFAT2','NFAT4','NFE2L1','NR1A2','NR1B1','NR1B2','NR1B3','NR1C1','NR1C2','NR1C3','NR1F3','NR1I1','NR2A1','NR2B3','NR3A1','NR3A2','NR3B3','OTEX','PEPP-2','PHF2','POU2F1','POU2F2','POU4F3','POU5F1','POU6F1','PRDM3','PRKRIP1','PRKRIR','SETD1A','SMAD8','SMYD2','SOX12','SOX14','SOX15','TBX14','TCF1','TCF17','TCF2','TCF9','TEAD1','TPRXL','UBTFL1','WHSC1','ZBTB14','ZBTB18','ZBTB19','ZBTB21','ZBTB27','ZBTB28','ZBTB35','ZBTB49','ZFP161','ZFP28','ZKSCAN10','ZKSCAN11','ZKSCAN12','ZKSCAN13','ZKSCAN14','ZKSCAN16','ZKSCAN19','ZKSCAN21','ZKSCAN24','ZKSCAN6','ZKSCAN8','ZNF10','ZNF112','ZNF187','ZNF193','ZNF204P','ZNF238','ZNF271','ZNF295','ZNF305','ZNF323','ZNF434','ZNF499','ZNF542','ZNF545','ZNF553','ZNF56','ZNF625','ZNF643','ZNF698','ZNF705F','ZNF751','ZNF752','ZNF753','ZNF785','ZNF806','ZNF816','ZNF833P','ZNF906','ZSCAN1','ZSCAN25','ZSCAN26','ZSCAN31','ZSCAN32','ZSCAN5D','ZSCAN9']
TFsOther=TFs1Other+TFs2Other+TFs3Other+TFs4Other
```

Example 10 (unknown):
```unknown
S genes: ['MCM5', 'PCNA', 'TYMS', 'FEN1', 'MCM2', 'MCM4', 'RRM1', 'UNG', 'GINS2', 'MCM6', 'CDCA7', 'DTL', 'PRIM1', 'UHRF1', 'HELLS', 'RFC2', 'RPA2', 'NASP', 'RAD51AP1', 'GMNN', 'WDR76', 'SLBP', 'CCNE2', 'UBR7', 'POLD3', 'MSH2', 'ATAD2', 'RAD51', 'RRM2', 'CDC45', 'CDC6', 'EXO1', 'TIPIN', 'DSCC1', 'BLM', 'CASP8AP2', 'USP1', 'CLSPN', 'POLA1', 'CHAF1B', 'BRIP1', 'E2F8'] 

G2 genes: ['HMGB2', 'CDK1', 'NUSAP1', 'UBE2C', 'BIRC5', 'TPX2', 'TOP2A', 'NDC80', 'CKS2', 'NUF2', 'CKS1B', 'MKI67', 'TMPO', 'CENPF', 'TACC3', 'SMC4', 'CCNB2', 'CKAP2L', 'CKAP2', 'AURKB', 'BUB1', 'KIF11', 'ANP32E', 'TUBB4B', 'GTSE1', 'KIF20B', 'HJURP', 'CDCA3', 'CDC20', 'TTK', 'CDC25C', 'KIF2C', 'RANGAP1', 'NCAPD2', 'DLGAP5', 'CDCA2', 'CDCA8', 'ECT2', 'KIF23', 'HMMR', 'AURKA', 'PSRC1', 'ANLN', 'LBR', 'CKAP5', 'CENPE', 'CTCF', 'NEK2', 'G2E3', 'GAS2L3', 'CBX5', 'CENPA'] 

Hemoglobin genes set 1: ['HBB', 'HBD', 'HBG1', 'HBG2', 'HBE1', 'HBZ', 'HBM', 'HBA2', 'HBA1', 'HBQ1'] 

Hemoglobin genes set 2: ['AHSP', 'GYPA', 'RHAG', 'GFI1B', 'KEL', 'EPB42', 'ALAS2', 'TSPO2', 'GYPB', 'GYPE', 'SPTA1', 'RHCE', 'KLF1', 'RHD', 'SLC4A1', 'SNX22', 'SQSTM1', 'SOD1', 'FTH1', 'MAP1LC3B', 'SLC25A37', 'TRIM58', 'HMBS'] 

Mitochondrial genes: ['MT-ND1', 'MT-ND2', 'MT-CO1', 'MT-CO2', 'MT-ATP8', 'MT-ATP6', 'MT-CO3', 'MT-ND3', 'MT-ND4L', 'MT-ND4', 'MT-ND5', 'MT-ND6', 'MT-CYB'] 

Ribo genes: ['RPLP0', 'RPLP1', 'RPLP2', 'RPL3', 'RPL3L', 'RPL4', 'RPL5', 'RPL6', 'RPL7', 'RPL7A', 'RPL7L1', 'RPL8', 'RPL9', 'RPL10', 'RPL10A', 'RPL10L', 'RPL11', 'RPL12', 'RPL13', 'RPL13A', 'RPL14', 'RPL15', 'RPL17', 'RPL18', 'RPL18A', 'RPL19', 'RPL21', 'RPL22', 'RPL22L1', 'RPL23', 'RPL23A', 'RPL24', 'RPL26', 'RPL26L1', 'RPL27', 'RPL27A', 'RPL28', 'RPL29', 'RPL30', 'RPL31', 'RPL32', 'RPL34', 'RPL35', 'RPL35A', 'RPL36', 'RPL36A', 'RPL36AL', 'RPL37', 'RPL37A', 'RPL38', 'RPL39', 'RPL39L', 'UBA52', 'RPL41', 'RPSA', 'RPS2', 'RPS3', 'RPS3A', 'RPS4X', 'RPS4Y1', 'RPS4Y2', 'RPS5', 'RPS6', 'RPS7', 'RPS8', 'RPS9', 'RPS10', 'RPS11', 'RPS12', 'RPS13', 'RPS14', 'RPS15', 'RPS15A', 'RPS16', 'RPS17', 'RPS18', 'RPS19', 'RPS20', 'RPS21', 'RPS23', 'RPS24', 'RPS25', 'RPS26', 'RPS27', 'RPS27A', 'RPS27L', 'RPS28', 'RPS29', 'FAU'] 

Tissue dissociation stress associated genes: ['FOS', 'FOSB', 'JUN', 'JUNB', 'JUND', 'ZFP36', 'SOCS3', 'CEBPD', 'CEBPB', 'ATF3', 'EGR1', 'HSPA1A', 'HSPA1B', 'HSP90AA1', 'HSP90AB1', 'HSPA8', 'HSPB1', 'HSPE1', 'IER3', 'IER2', 'BTG1', 'BTG2', 'DUSP1']
```

Example 11 (python):
```python
#Expression: "YlGn" (e.g. DE plots)
#Divergent expression: "PRGn" (e.g. log2FC)
#Age: "afmhot_r"
#Others: "viridis" (e.g. UMAP expression)

#Define color palette
FineGrainPalette={
    'AtrialCardiomyocytesLeft':'#eac075',
    'AtrialCardiomyocytesRight':'#4c6b8a',
    'AtrialCardiomyocytesCycling':'#e94bc7',
    'VentricularCardiomyocytesLeftCompact':'#ab1c3c',
    'VentricularCardiomyocytesRightCompact':'#fdba45',
    'VentricularCardiomyocytesLeftTrabeculated':'#2b9200',
    'VentricularCardiomyocytesRightTrabeculated':'#0273a8',
    'VentricularCardiomyocytesCycling':'#e1acff',
    'SinoatrialNodePacemakerCells':'#854834',
    'AtrioventricularNodePacemakerCells':'#688500',
    'VentricularConductionSystemProximal':'#01ccc8',
    'VentricularConductionSystemDistal':'#b58fab',
    'GreatVesselAdventitialFibroblasts':'#ff7e97',
    'CoronaryVesselAdventitialFibroblasts':'#ff9b6e',
    'MyocardialInterstitialFibroblasts':'#8a6800',
    'SubEpicardialFibroblasts':'#206601',
    'Myofibroblasts':'#1bdebf',
    'LymphNodeFibroblasticReticularCells':'#005aa0',
    'ValveInterstitialCells':'#e478ff',
    'GreatVesselSmoothMuscleCells':'#ff665f',
    'CoronarySmoothMuscleCells':'#d2cb18',
    'DuctusArteriosusSmoothMuscleCells':'#83d7b3',
    'CoronaryPericytes':'#9088ff',
    'PericardialCellsIntermediate':'#c95e00',
    'PericardialCellsFibrous':'#01e077',
    'PericardialCellsParietal':'#ff88e4',
    'GreatVesselArterialEndothelialCells':'#d4004b',
    'GreatVesselVenousEndothelialCells':'#8f4300',
    'CoronaryArterialEndothelialCells':'#89da5a',
    'CoronaryVenousEndothelialCells':'#006f86',
    'CoronaryCapillaryEndothelialCells':'#aa7dff',
    'EndocardialCells':'#ff8241',
    'EndocardialCushionCells':'#46603a',
    'ValveEndothelialCells':'#414eb6',
    'LymphaticEndothelialCells':'#8b3f60',
    'MesothelialEpicardialCells':'#0295fe',
    'EpicardiumDerivedCells':'#ff92c4',
    'NeuronPrecursors':'#d30038',
    'ChromaffinCells':'#726800',
    'SympatheticNeurons':'#01894b',
    'ParaympatheticNeurons':'#545191',
    'SchwannCellPrecursors':'#b0d276',
    'SchwannCells':'#b30079',
    'MonocytesMPOpos':'#c50066',
    'Monocytes':'#fab4b6',
    'MonocyteDerivedCells':'#d14008',
    'MacrophagesCX3CR1pos':'#eabe97',
    'MacrophagesTIMD4pos':'#b2a200',
    'MacrophagesLYVE1pos':'#4a6005',
    'MacrophagesATF3pos':'#3db833',
    'DendriticCellsType1':'#007859',
    'DendriticCellsMature':'#01a9db',
    'PlasmacytoidDendriticCells':'#2c54ab',
    'MastCells':'#d0bef2',
    'Megakaryocytes':'#94298f',
    'TCellsCD4pos':'#ff387c',
    'TCellsCD8pos':'#9e3429',
    'TregsCD4pos':'#b37700',
    'ProBCells':'#8dab00',
    'BCells':'#a8d1a1',
    'BCellsMS4A1pos':'#00d2f0',
    'NaturalKillerCells':'#abb4ff',
    'InnateLymphoidCells':'#72497f',
}
sc.pl.umap(adata, color='fine_grain',palette=FineGrainPalette)

MidGrainPalette={
    'AtrialCardiomyocytes':'#af3a4d',
    'VentricularCardiomyocytes':'#b1756c',
    'CardiacConductionSystem':'#d45633',
    'Fibroblasts':'#9b6e4e',
    'MuralCells':'#ffb92f',
    'PericardialCells':'#b4f03b',
    'BloodVesselEndothelialCells':'#72cc57',
    'EndocardialCells':'#436e43',
    'LymphaticEndothelialCells':'#85cca0',
    'EpicardialCells':'#5eabb0',
    'Neurons':'#7bbdff',
    'Glia':'#403c90',
    'MyeloidCells':'#420066',
    'LymphoidCells':'#f553f6',
}
sc.pl.umap(adata, color='mid_grain',palette=MidGrainPalette)

CoarseGrainPalette={
    'Cardiomyocytes':'#bc5950',
    'Mesenchymal':'#af9c4c',
    'Endothelium':'#599e58',
    'Epicardium':'#5eabb0',
    'Neural':'#5e75b1',
    'Leukocytes':'#9a52a0',
}
sc.pl.umap(adata, color='coarse_grain',palette=CoarseGrainPalette)

RegionGrainPalette={
    'aorta':'#ffa2bc',
    'apex':'#fe0049',
    'atria':'#732d00',
    'atrial septum':'#ffd54c',
    'base':'#837100',
    'ductus arteriosus':'#bdff8b',
    'whole sample':'#1b6c00',
    'heart without node':'#02e6c6',
    'left ventricle':'#01c1f5',
    'node':'#00466d',
    'outflow tract':'#020011',
    'pericardium':'#7a75ff',
    'pulmonary arches and branches':'#30003e',
    'right ventricle':'#ff88e1',
    'right ventricle and papillaries':'#98004f',
}
sc.pl.umap(adata, color='region',palette=RegionGrainPalette)
```

Example 12 (python):
```python
#adata = adata[adata.obs['fine_grain'].isin(['CoronaryVesselAdventitialFibroblasts','MyocardialInterstitialFibroblasts1','MyocardialInterstitialFibroblasts2','Myofibroblasts','GreatVesselAdventitialFibroblasts'])].copy()
#adata = adata[adata.obs['fine_grain'].isin(['CoronaryArterialEndothelialCells','CoronaryCapillaryEndothelialCells','CoronaryVenousEndothelialCells','GreatVesselArterialEndothelialCells','GreatVesselVenousEndothelialCells'])].copy()
#adata = adata[adata.obs['fine_grain'].isin(['CoronaryPericytes','CoronarySmoothMuscleCells','DuctusArteriosusSmoothMuscleCells','GreatVesselSmoothMuscleCells'])].copy()
adata=adata[adata.obs.coarse_grain=='Cardiomyocytea',:].copy()
set(adata.obs.fine_grain)
```

Example 13 (unknown):
```unknown
{'AtrialCardiomyocytesCycling',
 'AtrialCardiomyocytesLeft',
 'AtrialCardiomyocytesRight',
 'AtrioventricularNodePacemakerCells',
 'SinoatrialNodePacemakerCells',
 'VentricularCardiomyocytesCycling',
 'VentricularCardiomyocytesLeftCompact',
 'VentricularCardiomyocytesLeftTrabeculated',
 'VentricularCardiomyocytesRightCompact',
 'VentricularCardiomyocytesRightTrabeculated',
 'VentricularConductionSystemDistal',
 'VentricularConductionSystemProximal'}
```

Example 14 (python):
```python
# preserve raw count
adata.layers["counts"] = adata.X.copy()

# normalise and log-transform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key='batch_key')
#sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5)

print(sum(adata.var.highly_variable))
sc.pl.highly_variable_genes(adata)

# freeze the state in `.raw`
adata.raw = adata

# subset hvgs
adata = adata[:, adata.var.highly_variable]

adata=adata.copy()

sc.tl.score_genes_cell_cycle(adata, s_genes=s_genes, g2m_genes=g2m_genes)
#sc.pp.regress_out(adata, ['S_score','G2M_score'])

#batch correction
scvi.model.SCVI.setup_anndata(adata, 
                              layer="counts", 
                              batch_key="batch_key",
                              categorical_covariate_keys=['phase'],
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo'])

vae = scvi.model.SCVI(adata, n_hidden = 128, n_latent = 50, n_layers = 3, dispersion = 'gene-batch')
vae.train(max_epochs=50)
    
# Convergence plot
x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
plt.plot(x, vae.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
plt.clf()

# add the latent space to adata
adata.obsm["X_scVI"] = vae.get_latent_representation()

sc.tl.score_genes(adata, stress_genes, score_name='stress_score')
sc.tl.score_genes(adata, hb_genes1, score_name='hb1_score')

sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata,min_dist=0.1)
#sc.tl.tsne(adata)
sc.pl.umap(adata, color=['log1p_total_counts','n_genes','n_genes_by_counts','total_counts','pct_counts_mt','pct_counts_ribo'],ncols=5)
sc.pl.umap(adata, color=['scrublet_score','stress_score', 'hb1_score','phase'], ncols=5,)

sc.pl.umap(adata, color=['donor','age','week','trimester','region','phase','cycling','cell_or_nuclei','modality','kit_10x','FACSgate','heart_or_greatvessels','coarse_grain','mid_grain','fine_grain'],wspace=0.8,ncols=5)
Genes=['TTN','TNNT2','MYOM1','LDB3','ALDH1A2','TNNT1','KRT19','UPK3B','BNC1','COL6A3','FBLN1','PCOLCE','LRRC17','PCDH18','EMCN','ECSCR','PECAM1','CDH5','ESAM', 'LAPTM5','PTPRC','LCP1','CD53','WAS','PHOX2B','NRXN1','STMN2','GRIK2','L1CAM']
sc.pl.umap(adata, color=Genes)
```

Example 15 (unknown):
```unknown
normalizing counts per cell
    finished (0:00:00)
```

Example 16 (python):
```python
---------------------------------------------------------------------------
KeyboardInterrupt                         Traceback (most recent call last)
/tmp/ipykernel_189866/3224076617.py in <module>
      4 # normalise and log-transform
      5 sc.pp.normalize_total(adata, target_sum=1e4)
----> 6 sc.pp.log1p(adata)
      7 
      8 sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key='batch_key')

~/miniconda3/envs/scanpynew/lib/python3.8/functools.py in wrapper(*args, **kw)
    873                             '1 positional argument')
    874 
--> 875         return dispatch(args[0].__class__)(*args, **kw)
    876 
    877     funcname = getattr(func, '__name__', 'singledispatch function')

~/miniconda3/envs/scanpynew/lib/python3.8/site-packages/scanpy/preprocessing/_simple.py in log1p_anndata(adata, base, copy, chunked, chunk_size, layer, obsm)
    382     else:
    383         X = _get_obs_rep(adata, layer=layer, obsm=obsm)
--> 384         X = log1p(X, copy=False, base=base)
    385         _set_obs_rep(adata, X, layer=layer, obsm=obsm)
    386 

~/miniconda3/envs/scanpynew/lib/python3.8/functools.py in wrapper(*args, **kw)
    873                             '1 positional argument')
    874 
--> 875         return dispatch(args[0].__class__)(*args, **kw)
    876 
    877     funcname = getattr(func, '__name__', 'singledispatch function')

~/miniconda3/envs/scanpynew/lib/python3.8/site-packages/scanpy/preprocessing/_simple.py in log1p_sparse(X, base, copy)
    335         X, accept_sparse=("csr", "csc"), dtype=(np.float64, np.float32), copy=copy
    336     )
--> 337     X.data = log1p(X.data, copy=False, base=base)
    338     return X
    339 

~/miniconda3/envs/scanpynew/lib/python3.8/functools.py in wrapper(*args, **kw)
    873                             '1 positional argument')
    874 
--> 875         return dispatch(args[0].__class__)(*args, **kw)
    876 
    877     funcname = getattr(func, '__name__', 'singledispatch function')

~/miniconda3/envs/scanpynew/lib/python3.8/site-packages/scanpy/preprocessing/_simple.py in log1p_array(X, base, copy)
    350     elif not (np.issubdtype(X.dtype, np.floating) or np.issubdtype(X.dtype, complex)):
    351         X = X.astype(float)
--> 352     np.log1p(X, out=X)
    353     if base is not None:
    354         np.divide(X, np.log(base), out=X)

KeyboardInterrupt:
```

Example 17 (python):
```python
#Fibroblasts
#adata.obs['type']= 'na'
#adata.obs['type'][[x for x in adata.obs_names if adata.obs.fine_grain[x] in ['CoronaryVesselAdventitialFibroblasts','MyocardialInterstitialFibroblasts1','MyocardialInterstitialFibroblasts2','Myofibroblasts']]]='Heart'
#adata.obs['type'][[x for x in adata.obs_names if adata.obs.fine_grain[x] in ['GreatVesselAdventitialFibroblasts']]]='GreatVessels'
#sc.pl.umap(adata, color='type')

#Endothelium
#adata.obs['type']= 'na'
#adata.obs['type'][[x for x in adata.obs_names if adata.obs.fine_grain[x] in ['CoronaryArterialEndothelialCells','CoronaryCapillaryEndothelialCells','CoronaryVenousEndothelialCells']]]='Heart'
#adata.obs['type'][[x for x in adata.obs_names if adata.obs.fine_grain[x] in ['GreatVesselArterialEndothelialCells','GreatVesselVenousEndothelialCells']]]='GreatVessels'
#sc.pl.umap(adata, color='type')

#SmoothMuscleCells
adata.obs['type']= 'na'
adata.obs['type'][[x for x in adata.obs_names if adata.obs.fine_grain[x] in ['CoronaryPericytes','CoronarySmoothMuscleCells']]]='Heart'
adata.obs['type'][[x for x in adata.obs_names if adata.obs.fine_grain[x] in ['DuctusArteriosusSmoothMuscleCells','GreatVesselSmoothMuscleCells']]]='GreatVessels'
sc.pl.umap(adata, color='type')
```

Example 18 (unknown):
```unknown
... storing 'type' as categorical
```

Example 19 (python):
```python
sc.tl.rank_genes_groups(adata, 'type', method='wilcoxon', key_added = "wilcoxon")
result = adata.uns['wilcoxon']
groups = result['names'].dtype.names
df = pd.DataFrame(
{group + '_' + key[:5]: result[key][group]
for group in groups for key in ['names','scores','logfoldchanges','pvals','pvals_adj']})
df.to_csv('./FibroblastsDE.csv')
```

Example 20 (python):
```python
ranking genes
    finished: added to `.uns['wilcoxon']`
    'names', sorted np.recarray to be indexed by group ids
    'scores', sorted np.recarray to be indexed by group ids
    'logfoldchanges', sorted np.recarray to be indexed by group ids
    'pvals', sorted np.recarray to be indexed by group ids
    'pvals_adj', sorted np.recarray to be indexed by group ids (0:02:21)
```

Example 21 (python):
```python
#This box to run enrichment right away
gene_set_names = gseapy.get_library_name(organism='Human')
print(gene_set_names)

glist = sc.get.rank_genes_groups_df(adata, group='GreatVessels', 
                                    key='wilcoxon', log2fc_min=0, 
                                    pval_cutoff=1)['names'].squeeze().str.strip().tolist()
print(len(glist))

#GO_Biological_Process_2021,GO_Cellular_Component_2021,GO_Molecular_Function_2021
enr_res = gseapy.enrichr(gene_list=glist,
                     organism='Human',
                     gene_sets='GO_Biological_Process_2021',
                     cutoff = 0.5)
enr_res.results.head()
```

Example 22 (python):
```python
adata=sc.read('./ViolinPlot/FibroblastViolinFigures.h5ad')
adata = adata[adata.obs['fine_grain'].isin(['CoronaryVesselAdventitialFibroblasts','GreatVesselAdventitialFibroblasts'])].copy()
sc.tl.rank_genes_groups(adata, 'heart_or_greatvessels', groups=['heart'], reference='greatvessels', method='t-test')
plt.rcParams['figure.figsize'] = 6,5
sc.pl.rank_genes_groups_violin(adata, groups='heart',gene_names=['TBX20','SCN7A','MOXD1','SPON2','TCF21','TSHZ2','ROR1','PIEZO2','MEIS1','SFRP2'],strip=False)
#adata.write('./ViolinPlot/FibroblastViolinFigures.h5ad')
```

Example 23 (python):
```python
ranking genes
    finished: added to `.uns['rank_genes_groups']`
    'names', sorted np.recarray to be indexed by group ids
    'scores', sorted np.recarray to be indexed by group ids
    'logfoldchanges', sorted np.recarray to be indexed by group ids
    'pvals', sorted np.recarray to be indexed by group ids
    'pvals_adj', sorted np.recarray to be indexed by group ids (0:00:01)
```

Example 24 (python):
```python
ClusterOfInterest='GreatVesselAdventitialFibroblasts'
#sc.tl.rank_genes_groups(adata, 'fine_grain', method='t-test', key_added = "t-test")
DEGenesForCluster = [x[ClusterOfInterest] for x in adata.uns['t-test']['names'][:50]]
sc.pl.dotplot(adata, DEGenesForCluster ,groupby='fine_grain',standard_scale='var',cmap='PRGn')
#sc.pl.umap(adata, color=DEGenesForCluster)
```

Example 25 (python):
```python
#Muralcells
adata=sc.read('./ViolinPlot/MuralViolinFigures.h5ad')
print(adata.shape)
sc.tl.rank_genes_groups(adata, 'heart_or_greatvessels', groups=['heart'], reference='greatvessels', method='t-test')
plt.rcParams['figure.figsize'] = 6,5
sc.pl.rank_genes_groups_violin(adata, groups='heart',gene_names=['NR2F2','TFPI','TBX2','ENPEP','COL18A1','ERBB4','THSD4','SFRP1','FBLN5','PRDM6'],strip=False)
#adata.write('./ViolinPlot/MuralViolinFigures.h5ad')
```

Example 26 (python):
```python
(28497, 1505)
ranking genes
    finished: added to `.uns['rank_genes_groups']`
    'names', sorted np.recarray to be indexed by group ids
    'scores', sorted np.recarray to be indexed by group ids
    'logfoldchanges', sorted np.recarray to be indexed by group ids
    'pvals', sorted np.recarray to be indexed by group ids
    'pvals_adj', sorted np.recarray to be indexed by group ids (0:00:00)
```

Example 27 (python):
```python
#Endothelial
adata=sc.read('./ViolinPlot/EndothelialViolinFigures.h5ad')
print(adata.shape)
sc.tl.rank_genes_groups(adata, 'heart_or_greatvessels', groups=['heart'], reference='greatvessels', method='t-test')
plt.rcParams['figure.figsize'] = 6,5
sc.pl.rank_genes_groups_violin(adata, groups='heart',gene_names=['COL4A1','COL4A2','H19','A2M','ADGRL4','HMCN1','COL3A1','NRG3','MEIS2','PLXNA4'],strip=False)
#adata.write('./ViolinPlot/EndothelialViolinFigures.h5ad')
```

Example 28 (python):
```python
(11352, 1634)
ranking genes
    finished: added to `.uns['rank_genes_groups']`
    'names', sorted np.recarray to be indexed by group ids
    'scores', sorted np.recarray to be indexed by group ids
    'logfoldchanges', sorted np.recarray to be indexed by group ids
    'pvals', sorted np.recarray to be indexed by group ids
    'pvals_adj', sorted np.recarray to be indexed by group ids (0:00:00)
```

---

## Variables¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/VisiumHD/no7_manual-annotation_EpicardialCells.html

**Contents:**
- Variables¶
- Read in data¶
- Remove mid-grain markers from the feature¶
- PCA, Clustering, UMAP¶
- Markers¶
- Save¶

2392 rows × 7 columns

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os

import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 

import celltypist
```

Example 2 (python):
```python
# import session_info
# session_info.show()
```

Example 3 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 4 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 5 (python):
```python
adata_dir = '/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/subsets'
sample_id = 'HEA_FOET14880396'
celltype = 'EpicardialCells'
```

Example 6 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk/Aug2024'
table_dir = '/nfs/team205/heart/fetal_ms_figures/tables/Aug2024'
```

Example 7 (python):
```python
# read in
adata = sc.read_h5ad(f'{adata_dir}/{sample_id}_{celltype}_b2c_cells_filtered_lognorm.h5ad')
adata
```

Example 8 (unknown):
```unknown
AnnData object with n_obs × n_vars = 603 × 13050
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'leiden_0.2', 'leiden_0.4', 'leiden_0.6', 'leiden_1', 'leiden_1.2', 'leiden_1.5', 'leiden_2'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1.2', 'leiden_1.2_colors', 'leiden_1.5', 'leiden_1.5_colors', 'leiden_1_colors', 'leiden_2', 'leiden_2_colors', 'log1p', 'neighbors', 'pca', 'spatial', 'umap'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 9 (python):
```python
# read in
midgrain_markers = pd.read_csv('/nfs/team205/kk18/notebooks/Foetal/VisiumHD/re-seq_mpp0p3/sc_midgrain_markers.csv',
                               index_col=0
                              )
# remove the markers of celltype of interest
midgrain_markers = midgrain_markers[midgrain_markers['group']!=celltype]
midgrain_markers
```

Example 10 (python):
```python
# remove
mask = [x not in list(midgrain_markers['names']) for x in adata.var_names]
bdata = adata[:,mask]
bdata
```

Example 11 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 603 × 11843
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'leiden_0.2', 'leiden_0.4', 'leiden_0.6', 'leiden_1', 'leiden_1.2', 'leiden_1.5', 'leiden_2'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1.2', 'leiden_1.2_colors', 'leiden_1.5', 'leiden_1.5_colors', 'leiden_1_colors', 'leiden_2', 'leiden_2_colors', 'log1p', 'neighbors', 'pca', 'spatial', 'umap'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 12 (python):
```python
# HVGs, using the object which doesn't have midgrain markers
sc.pp.highly_variable_genes(bdata, flavor='seurat', min_mean=0.0125, max_mean=3, min_disp=0.5)
print(f'number of hvgs: {sum(bdata.var.highly_variable)}')
# PCA using the degected HVGs
sc.tl.pca(bdata, use_highly_variable=True)
adata.obsm['X_pca'] = bdata.obsm['X_pca'].copy()
# create neighbour graph
sc.pp.neighbors(adata,n_neighbors=15,use_rep='X_pca')
# visualisation with umap
sc.tl.umap(adata,min_dist=0.5)
sc.pl.embedding(adata,basis='X_umap',color=['bin_count','total_counts','n_genes_by_counts',"conf_score_midmod2fine","celltypist_midmod2fine",],
                wspace=0.2,cmap='RdPu',vmax='p100',ncols=5)
# leiden clustering
resolutions_list = [0.2,0.4,0.6,1,1.2,1.5,2]
for resolution in resolutions_list:
    sc.tl.leiden(adata,resolution=resolution,key_added=f'leiden_{str(resolution)}',n_iterations=2)
sc.pl.embedding(adata,basis='X_umap',
                color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                wspace=0.3)
```

Example 13 (unknown):
```unknown
number of hvgs: 3273
```

Example 14 (python):
```python
# read in markers identified using single-nucleus data
markers_df = pd.read_csv(f'/nfs/team205/kk18/notebooks/Foetal/VisiumHD/re-seq_mpp0p3/sc_{celltype}_finegrain_markers.csv')

mask1 = markers_df['group_mean']>1
mask2 = markers_df['logfoldchanges']>1
markers_df = markers_df[mask1&mask2]

markers_df
```

Example 15 (python):
```python
markers = {}
n_top = 10
for c in set(markers_df['group']):
    markers_df_sub = markers_df[markers_df['group']==c]
    markers_df_sub.sort_values('logfoldchanges',ascending=False)
    markers[c]=list(markers_df_sub[:n_top]['names'])

# manually add genes
markers['Cycling'] = ['MKI67','TOP2A'] 

genes = []
for g in markers.values():
    genes = genes + [x for x in g if (x in list(adata.var_names))&(x not in genes)]
for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in list(adata.var_names)]
len(genes)
```

Example 16 (python):
```python
sc.pl.embedding(adata,basis='X_umap',color=genes, # size=30,
               ncols=6, wspace=0.2,cmap='RdPu',vmax='p100')
```

Example 17 (python):
```python
sc.pl.embedding(adata,basis='X_umap',
                color=['celltypist_midmod2fine', 'conf_score_midmod2fine'],
                wspace=0.7)
```

Example 18 (python):
```python
# umap per celltype to highlight
group_col = 'celltypist_midmod2fine'
groups_to_plot = list(set(adata.obs[group_col]))

plt.rcParams['axes.grid'] = False
fig,axs=plt.subplots(nrows=2,ncols=4,figsize=(15,7))
# Flatten the axs array for easier iteration in case of multiple rows and columns
axs_flat = axs.flatten()

for i,grp in enumerate(groups_to_plot):
    sc.pl.umap(adata, color=[group_col], groups=grp, 
               size=80, frameon=False,
               show=False, ax=axs_flat[i], legend_loc=None, legend_fontsize=5,
               title=grp
              )
# Turn off any remaining subplots
for j in range(len(groups_to_plot), len(axs_flat)):
    fig.delaxes(axs_flat[j])

# fig.suptitle('Main Title for the Figure')
fig.tight_layout()
```

Example 19 (unknown):
```unknown
resolution_sel = 1
```

Example 20 (python):
```python
markers_others = {
    'CM': ['TTN', 'TNNT2','MYH6',
        'MYH7', 'MYL2', 'FHL2', # vCN
            'NPPA', 'MYL7', 'MYL4'], #aCM
    'FB': ['DCN', 'GCN', 'PDGFRA','COL1A1','COL1A2'],
    'EC': ['VWF', 'PECAM1', 'CDH5','RGCC', 'FABP5',],
    'Peri': ['RGS5', 'ABCC9', 'KCNJ8'],
    'SMC': ['MYH11', 'TAGLN', 'ACTA2'],
    'Meso': ['MSLN', 'WT1', 'BNC1'],
    'Neuro': ['PLP1', 'NRXN1', 'NRXN3','PRPH', 'NEFL', 'NEFM', 'NEFH', 'STMN2', 'SST', 'SLC10A4', 'SLC18A3', 'TH', 'NPY'],
    'Adipo': ['GPAM', 'FASN', 'LEP'],
    'Myelo': ['CD14', 'C1QA', 'CD68','LYVE1','TIMD4'],
    'Lymph': ['CD8A', 'IL7R', 'CD40LG'],
    'B': ['CD19','SDC1','IGHG1','IGHA1'],
    'Mast': ['KIT', 'CPA3'],
}

for key in markers_others.keys():
    markers_others[key]=[x for x in markers_others[key] if x in adata.var_names]

sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
sc.pl.dotplot(adata,
              markers_others, 
              groupby=f'leiden_{str(resolution_sel)}',
              dendrogram=True,
              standard_scale="var",
              color_map="Reds",
              swap_axes=False,
             )
```

Example 21 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: CM, FB, EC, etc.
```

Example 22 (python):
```python
sc.tl.dendrogram(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.pl.dotplot(adata,
              var_names=markers,
              groupby=f'leiden_{str(resolution_sel)}',
              standard_scale='var',
              dendrogram=True
             )
```

Example 23 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: EpicardiumDerivedCells, MesothelialEpicardialCells, Cycling
```

Example 24 (python):
```python
# DEG analysis
print(adata.X.data[:5])
adata.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata, dendrogram=True, standard_scale='var', return_fig=True)
dp.add_totals().show()
```

Example 25 (unknown):
```unknown
[2.04256052 2.77787835 1.93707821 2.4739898  3.19327673]
```

Example 26 (python):
```python
# DEG analysis
print(adata.X.data[:5])
adata.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata, dendrogram=True, standard_scale='var', return_fig=True, n_genes=5)
dp.add_totals().show()
```

Example 27 (unknown):
```unknown
[2.04256052 2.77787835 1.93707821 2.4739898  3.19327673]
```

Example 28 (python):
```python
sc.pl.embedding(adata,basis='X_umap',
                color=f'leiden_{str(resolution_sel)}',
                wspace=0.3, legend_loc='on data')
```

Example 29 (python):
```python
sc.pl.violin(adata,keys=['total_counts','bin_count'],groupby=f'leiden_{str(resolution_sel)}')
```

Example 30 (unknown):
```unknown
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 31 (python):
```python
# annotate
adata.obs['fine_grain_manual'] = adata.obs[f'leiden_{str(resolution_sel)}'].copy()
adata.obs.replace({'fine_grain_manual':{
    '0':'MesothelialEpicardialCells', # contamination from CMs
    '1':'MesothelialEpicardialCells',
    '2':'EpicardiumDerivedCells',
    '3':'MesothelialEpicardialCells',
    '4':'MesothelialEpicardialCells', 
    '5':'MesothelialEpicardialCells',
}},inplace=True)
adata.obs['fine_grain_manual'].value_counts()
```

Example 32 (unknown):
```unknown
fine_grain_manual
MesothelialEpicardialCells    491
EpicardiumDerivedCells        112
Name: count, dtype: int64
```

Example 33 (python):
```python
ad = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/Feb28ObjectRaw_finegrain_updated.h5ad',backed='r')
ad = ad[ad.obs['mid_grain']==celltype]
set(ad.obs['fine_grain'])
```

Example 34 (unknown):
```unknown
{'EpicardiumDerivedCells', 'MesothelialEpicardialCells'}
```

Example 35 (python):
```python
sc.pl.embedding(adata,basis='X_umap',
                color='fine_grain_manual',
                frameon=False,
                wspace=0.3)
```

Example 36 (python):
```python
markers['MesothelialEpicardialCells']=markers['MesothelialEpicardialCells']+['WT1','MSLN']

sc.tl.dendrogram(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.pl.dotplot(adata,
              var_names=markers,
              groupby='fine_grain_manual',
              standard_scale='var',
              dendrogram=True
             )
```

Example 37 (unknown):
```unknown
WARNING: Dendrogram not added. Dendrogram is added only when the number of categories to plot > 2
```

Example 38 (python):
```python
# save
adata.write(f'{adata_dir}/{sample_id}_{celltype}_b2c_cells_filtered_lognorm.h5ad')
print(f'{adata_dir}/{sample_id}_{celltype}_b2c_cells_filtered_lognorm.h5ad')
```

Example 39 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/subsets/HEA_FOET14880396_EpicardialCells_b2c_cells_filtered_lognorm.h5ad
```

---

## Variables¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/Xenium/no2_integration_and_cell-type-annotation/no6_manual-annotation_PericardialCells.html

**Contents:**
- Variables¶
- Read in data¶
- Markers¶
- 1st round annotation¶
- Subset and reclustering¶
- Transfer annotation to the original adata¶
- Save¶

--> cluster 0 has lower total counts and smaller cell area, and no specific gene expression --> annotate as "unassigned"

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os

import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 

import scanpy.external as sce
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/session_info/main.py:213: UserWarning: The '__version__' attribute is deprecated and will be removed in MarkupSafe 3.1. Use feature detection, or `importlib.metadata.version("markupsafe")`, instead.
  mod_version = _find_version(mod.__version__)
```

Example 4 (python):
```python
-----
matplotlib          3.10.1
numpy               2.1.3
pandas              2.2.3
scanpy              1.11.1
seaborn             0.13.2
session_info        v1.0.1
-----
```

Example 5 (python):
```python
Cython              3.1.1
PIL                 11.1.0
anndata             0.11.4
asciitree           NA
asttokens           NA
charset_normalizer  3.4.1
cloudpickle         3.1.1
comm                0.2.2
cycler              0.12.1
cython              3.1.1
cython_runtime      NA
dask                2024.11.2
dateutil            2.9.0.post0
debugpy             1.8.13
decorator           5.2.1
exceptiongroup      1.2.2
executing           2.2.0
h5py                3.13.0
igraph              0.11.8
importlib_metadata  NA
ipykernel           6.29.5
jedi                0.19.2
jinja2              3.1.6
joblib              1.4.2
kiwisolver          1.4.8
legacy_api_wrap     NA
leidenalg           0.10.2
llvmlite            0.44.0
markupsafe          3.0.2
mpl_toolkits        NA
msgpack             1.1.0
natsort             8.4.0
numba               0.61.0
numcodecs           0.13.1
packaging           24.2
parso               0.8.4
patsy               1.0.1
platformdirs        4.3.7
prompt_toolkit      3.0.50
psutil              7.0.0
pure_eval           0.2.3
pyarrow             19.0.1
pydev_ipython       NA
pydevconsole        NA
pydevd              3.2.3
pydevd_file_utils   NA
pydevd_plugins      NA
pydevd_tracing      NA
pygments            2.19.1
pyparsing           3.2.3
pytz                2025.2
scipy               1.15.2
six                 1.17.0
sklearn             1.5.2
stack_data          0.6.3
statsmodels         0.14.4
tblib               3.1.0
texttable           1.7.0
threadpoolctl       3.6.0
tlz                 1.0.0
toolz               1.0.0
tornado             6.4.2
traitlets           5.14.3
typing_extensions   NA
wcwidth             0.2.13
yaml                6.0.2
zarr                2.18.3
zipp                NA
zmq                 26.3.0
zoneinfo            NA
```

Example 6 (unknown):
```unknown
-----
IPython             8.34.0
jupyter_client      8.6.3
jupyter_core        5.7.2
-----
Python 3.10.14 | packaged by conda-forge | (main, Mar 20 2024, 12:45:18) [GCC 12.3.0]
Linux-4.18.0-553.53.1.el8_10.x86_64-x86_64-with-glibc2.28
-----
Session information updated at 2025-06-12 17:55
```

Example 7 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/home/kk837/rfs/rfs-teichlab-nfs-iCNyzSAaucw/kk837/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 8 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 9 (python):
```python
adata_dir = '/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium/subsets'
sample_id = 'C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat'
celltype = 'PericardialCells'
```

Example 10 (unknown):
```unknown
# figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk/Aug2024'
# table_dir = '/nfs/team205/heart/fetal_ms_figures/tables/Aug2024'
```

Example 11 (unknown):
```unknown
latent_space = 'pca_harmony'
```

Example 12 (python):
```python
# read in
adata = sc.read_h5ad(f'{adata_dir}/{sample_id}_5K_{celltype}_lognorm.h5ad')
adata
```

Example 13 (unknown):
```unknown
AnnData object with n_obs × n_vars = 8698 × 4737
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_3.0', 'multi_celltypes_coarse', 'leiden_0.2', 'leiden_0.4', 'leiden_0.6', 'leiden_1'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1', 'n_cells', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'dendrogram_pca_harmony_leiden_1.0', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1_colors', 'log1p', 'multi_celltypes_coarse_colors', 'pca', 'pca_harmony', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.2_colors', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.3_colors', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.4_colors', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.5_colors', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_0.8_colors', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.0_colors', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_1.5_colors', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_2.0_colors', 'pca_harmony_leiden_3.0', 'pca_harmony_leiden_3.0_colors', 'tissue_block_id_colors', 'umap'
    obsm: 'X_pca', 'X_pca_harmony', 'X_umap_pca_harmony', 'spatial'
    varm: 'PCs'
    obsp: 'pca_harmony_connectivities', 'pca_harmony_distances'
```

Example 14 (python):
```python
adata.X.data[:5]
```

Example 15 (unknown):
```unknown
array([2.9218001, 2.9218001, 2.9218001, 3.58766  , 2.9218001],
      dtype=float32)
```

Example 16 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=['total_counts','n_genes_by_counts','cell_area','tissue_block_id'],
                    wspace=0.2,cmap='RdPu',vmax='p100')
```

Example 17 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=['multi_celltypes_coarse',"conf_score_midmod2fine","celltypist_midmod2fine",],
                    wspace=0.2,cmap='RdPu',vmax='p100')
resolutions_list = [0.2,0.4,0.6,1]
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                    color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                    wspace=0.2)
```

Example 18 (python):
```python
%%time
# leiden clustering
resolutions_list = [# 0.2,0.4,0.6,1, # already done
                    1.2,1.5,2]
for resolution in resolutions_list:
    sc.tl.leiden(adata,neighbors_key=latent_space,
                     resolution=resolution,key_added=f'leiden_{str(resolution)}',n_iterations=2)
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                wspace=0.3)
```

Example 19 (unknown):
```unknown
<timed exec>:5: FutureWarning: In the future, the default backend for leiden will be igraph instead of leidenalg.

 To achieve the future defaults please pass: flavor="igraph" and n_iterations=2.  directed must also be False to work with igraph's implementation.
```

Example 20 (unknown):
```unknown
CPU times: user 5.86 s, sys: 110 ms, total: 5.97 s
Wall time: 5.98 s
```

Example 21 (python):
```python
# read in markers identified using single-nucleus data
markers_df = pd.read_csv(f'/home/kk837/rfs/rfs-teichlab-nfs-iCNyzSAaucw/kk837/notebooks/Foetal/VisiumHD/re-seq_mpp0p3/sc_{celltype}_finegrain_markers.csv')

mask1 = markers_df['group_mean']>1
mask2 = markers_df['logfoldchanges']>1
markers_df = markers_df[mask1&mask2]

markers_df
```

Example 22 (python):
```python
markers = {}
n_top = 50
for c in set(markers_df['group']):
    markers_df_sub = markers_df[markers_df['group']==c]
    markers_df_sub.sort_values('logfoldchanges',ascending=False)
    markers[c]=list(markers_df_sub[:n_top]['names'])

# manually add genes
markers['Cycling'] = ['MKI67','TOP2A'] 

genes = []
for g in markers.values():
    genes = genes + [x for x in g if (x in list(adata.var_names))&(x not in genes)]
for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in list(adata.var_names)]
len(genes)
```

Example 23 (python):
```python
sc.pl.embedding(adata,basis=f"X_umap_{latent_space}",color=genes, size=20,
               ncols=6, wspace=0.2,cmap='RdPu',vmax='p100')
```

Example 24 (python):
```python
sc.pl.embedding(adata,basis=f"X_umap_{latent_space}",
                color=['celltypist_midmod2fine', 'conf_score_midmod2fine'],
                wspace=0.7)
```

Example 25 (python):
```python
# umap per celltype to highlight
group_col = 'celltypist_midmod2fine'
groups_to_plot = list(set(adata.obs[group_col]))

plt.rcParams['axes.grid'] = False
fig,axs=plt.subplots(nrows=2,ncols=4,figsize=(15,7))
# Flatten the axs array for easier iteration in case of multiple rows and columns
axs_flat = axs.flatten()

for i,grp in enumerate(groups_to_plot):
    sc.pl.embedding(adata,basis=f"X_umap_{latent_space}",
                    color=[group_col], groups=grp, 
               size=30, frameon=False,
               show=False, ax=axs_flat[i], legend_loc=None, legend_fontsize=5,
               title=grp
              )
# Turn off any remaining subplots
for j in range(len(groups_to_plot), len(axs_flat)):
    fig.delaxes(axs_flat[j])

# fig.suptitle('Main Title for the Figure')
fig.tight_layout()
```

Example 26 (unknown):
```unknown
resolution_sel = 0.6
```

Example 27 (python):
```python
markers_others = {
    'CM': ['TTN', 'TNNT2','MYH6',
        'MYH7', 'MYL2', 'FHL2', # vCN
            'NPPA', 'MYL7', 'MYL4'], #aCM
    'FB': ['DCN', 'GCN', 'PDGFRA','COL1A1','COL1A2'],
    'EC': ['VWF', 'PECAM1', 'CDH5','RGCC', 'FABP5',],
    'Peri': ['RGS5', 'ABCC9', 'KCNJ8'],
    'SMC': ['MYH11', 'TAGLN', 'ACTA2'],
    'Meso': ['MSLN', 'WT1', 'BNC1'],
    'Neuro': ['PLP1', 'NRXN1', 'NRXN3','PRPH', 'NEFL', 'NEFM', 'NEFH', 'STMN2', 'SST', 'SLC10A4', 'SLC18A3', 'TH', 'NPY'],
    'Adipo': ['GPAM', 'FASN', 'LEP'],
    'Myelo': ['CD14', 'C1QA', 'CD68','LYVE1','TIMD4'],
    'Lymph': ['CD8A', 'IL7R', 'CD40LG'],
    'B': ['CD19','SDC1','IGHG1','IGHA1'],
    'Mast': ['KIT', 'CPA3'],
}

for key in markers_others.keys():
    markers_others[key]=[x for x in markers_others[key] if x in adata.var_names]

sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
sc.pl.dotplot(adata,
              markers_others, 
              groupby=f'leiden_{str(resolution_sel)}',
              dendrogram=True,
              standard_scale="var",
              color_map="Reds",
              swap_axes=False,
             )
```

Example 28 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: CM, FB, EC, etc.
```

Example 29 (python):
```python
sc.tl.dendrogram(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.pl.dotplot(adata,
              var_names=markers,
              groupby=f'leiden_{str(resolution_sel)}',
              standard_scale='var',
              dendrogram=True
             )
```

Example 30 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: PericardialCellsParietal, PericardialCellsIntermediate, PericardialCellsFibrous, etc.
```

Example 31 (python):
```python
print(adata.X.data[:5])
adata.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata, dendrogram=True, standard_scale='var', return_fig=True)
dp.add_totals().show()
```

Example 32 (unknown):
```unknown
[2.9218001 2.9218001 2.9218001 3.58766   2.9218001]
```

Example 33 (python):
```python
sc.pl.violin(adata,keys=['total_counts','cell_area'],groupby=f'leiden_{str(resolution_sel)}')
```

Example 34 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=f'leiden_{str(resolution_sel)}',legend_loc='on data',
                )
```

Example 35 (unknown):
```unknown
'PericardialCells'
```

Example 36 (python):
```python
# annotate
adata.obs['tmp_annotation'] = adata.obs[f'leiden_{str(resolution_sel)}'].copy()
adata.obs.replace({'tmp_annotation':{
    '0':'unassigned',
    '1':'PericardialCells',
    '2':'PericardialCells',
    '3':'PericardialCells',
    '4':'PericardialCells',
    '5':'?ThyroidCells',
}},inplace=True)
adata.obs['tmp_annotation'].value_counts()
```

Example 37 (python):
```python
/tmp/ipykernel_419353/351917040.py:3: FutureWarning: The behavior of Series.replace (and DataFrame.replace) with CategoricalDtype is deprecated. In a future version, replace will only be used for cases that preserve the categories. To change the categories, use ser.cat.rename_categories instead.
  adata.obs.replace({'tmp_annotation':{
```

Example 38 (unknown):
```unknown
tmp_annotation
PericardialCells    5374
unassigned          3266
?ThyroidCells         58
Name: count, dtype: int64
```

Example 39 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color='tmp_annotation'
                )
```

Example 40 (python):
```python
adata_sub = adata[adata.obs['tmp_annotation'].isin(['PericardialCells'])]
```

Example 41 (python):
```python
# drop leiden clustering results
resolutions_list = [0.2,0.4,0.6,1, 1.2,1.5,2]
for resolution in resolutions_list:
    adata_sub.obs.drop(f'leiden_{str(resolution)}',axis=1,inplace=True)
adata_sub
```

Example 42 (unknown):
```unknown
AnnData object with n_obs × n_vars = 5374 × 4737
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_3.0', 'multi_celltypes_coarse', 'tmp_annotation'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1', 'n_cells', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'dendrogram_pca_harmony_leiden_1.0', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1_colors', 'log1p', 'multi_celltypes_coarse_colors', 'pca', 'pca_harmony', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.2_colors', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.3_colors', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.4_colors', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.5_colors', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_0.8_colors', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.0_colors', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_1.5_colors', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_2.0_colors', 'pca_harmony_leiden_3.0', 'pca_harmony_leiden_3.0_colors', 'tissue_block_id_colors', 'umap', 'leiden_1.2', 'leiden_1.5', 'leiden_2', 'leiden_1.2_colors', 'leiden_1.5_colors', 'leiden_2_colors', 'dendrogram_leiden_0.6', 'rank_genes_groups', 'tmp_annotation_colors'
    obsm: 'X_pca', 'X_pca_harmony', 'X_umap_pca_harmony', 'spatial'
    varm: 'PCs'
    obsp: 'pca_harmony_connectivities', 'pca_harmony_distances'
```

Example 43 (python):
```python
## Remove mid-grain markers from the feature
# read in markers
midgrain_markers = pd.read_csv('/home/kk837/rfs/rfs-teichlab-nfs-iCNyzSAaucw/kk837/notebooks/Foetal/VisiumHD/re-seq_mpp0p3/sc_midgrain_markers.csv',
                               index_col=0)
# remove the markers of celltype of interest
midgrain_markers = midgrain_markers[midgrain_markers['group']!=celltype]
# remove
mask = [x not in list(midgrain_markers['names']) for x in adata_sub.var_names]
bdata = adata_sub[:,mask]
bdata
```

Example 44 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 5374 × 4280
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_3.0', 'multi_celltypes_coarse', 'tmp_annotation'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1', 'n_cells', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'dendrogram_pca_harmony_leiden_1.0', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1_colors', 'log1p', 'multi_celltypes_coarse_colors', 'pca', 'pca_harmony', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.2_colors', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.3_colors', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.4_colors', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.5_colors', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_0.8_colors', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.0_colors', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_1.5_colors', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_2.0_colors', 'pca_harmony_leiden_3.0', 'pca_harmony_leiden_3.0_colors', 'tissue_block_id_colors', 'umap', 'leiden_1.2', 'leiden_1.5', 'leiden_2', 'leiden_1.2_colors', 'leiden_1.5_colors', 'leiden_2_colors', 'dendrogram_leiden_0.6', 'rank_genes_groups', 'tmp_annotation_colors'
    obsm: 'X_pca', 'X_pca_harmony', 'X_umap_pca_harmony', 'spatial'
    varm: 'PCs'
    obsp: 'pca_harmony_connectivities', 'pca_harmony_distances'
```

Example 45 (python):
```python
'''
# remove cell cycle genes
cell_cycle_genes = ['MCM5','PCNA','TYMS','FEN1','MCM2','MCM4','RRM1','UNG','GINS2','MCM6','CDCA7','DTL','PRIM1','UHRF1','MLF1IP','HELLS','RFC2','RPA2','NASP',
                'RAD51AP1','GMNN','WDR76','SLBP','CCNE2','UBR7','POLD3','MSH2','ATAD2','RAD51','RRM2','CDC45','CDC6','EXO1','TIPIN','DSCC1','BLM','CASP8AP2',
                'USP1','CLSPN','POLA1','CHAF1B','BRIP1','E2F8','HMGB2','CDK1','NUSAP1','UBE2C','BIRC5','TPX2','TOP2A','NDC80','CKS2','NUF2','CKS1B','MKI67',
                'TMPO','CENPF','TACC3','FAM64A','SMC4','CCNB2','CKAP2L','CKAP2','AURKB','BUB1','KIF11','ANP32E','TUBB4B','GTSE1','KIF20B','HJURP','CDCA3','HN1',
                'CDC20','TTK','CDC25C','KIF2C','RANGAP1','NCAPD2','DLGAP5','CDCA2','CDCA8','ECT2','KIF23','HMMR','AURKA','PSRC1','ANLN','LBR','CKAP5','CENPE','CTCF',
                'NEK2','G2E3','GAS2L3','CBX5','CENPA']
mask = [x not in cell_cycle_genes for x in bdata.var_names]
bdata = bdata[:,mask]
bdata.shape
'''
```

Example 46 (python):
```python
# HVGs, using the object which doesn't have midgrain markers
sc.pp.highly_variable_genes(bdata, flavor='seurat', min_mean=0.0125, max_mean=3, min_disp=0.5)
print(f'number of hvgs: {sum(bdata.var.highly_variable)}')
# PCA using the degected HVGs
sc.tl.pca(bdata, use_highly_variable=True)
adata_sub.obsm['X_pca'] = bdata.obsm['X_pca'].copy()
# harmony integration
sce.pp.harmony_integrate(adata_sub, 'tissue_block_id', basis='X_pca', adjusted_basis='X_pca_harmony')
# create neighbour graph
latent_space = 'pca_harmony'
sc.pp.neighbors(adata_sub,n_neighbors=15,use_rep=f"X_{latent_space}",key_added=latent_space)
# visualisation with umap
sc.tl.umap(adata_sub,min_dist=0.5,neighbors_key=latent_space)
adata_sub.obsm[f"X_umap_{latent_space}"]=adata_sub.obsm["X_umap"]
del adata_sub.obsm["X_umap"]
```

Example 47 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_highly_variable_genes.py:698: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  adata.uns["hvg"] = {"flavor": flavor}
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_pca/__init__.py:438: FutureWarning: Argument `use_highly_variable` is deprecated, consider using the mask argument. Use_highly_variable=True can be called through mask_var="highly_variable". Use_highly_variable=False can be called through mask_var=None
  warn(msg, FutureWarning)
```

Example 48 (unknown):
```unknown
number of hvgs: 825
```

Example 49 (python):
```python
2025-06-12 18:00:09,972 - harmonypy - INFO - Computing initial centroids with sklearn.KMeans...
2025-06-12 18:00:11,635 - harmonypy - INFO - sklearn.KMeans initialization complete.
2025-06-12 18:00:11,737 - harmonypy - INFO - Iteration 1 of 10
2025-06-12 18:00:12,403 - harmonypy - INFO - Iteration 2 of 10
2025-06-12 18:00:13,065 - harmonypy - INFO - Converged after 2 iterations
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm
```

Example 50 (python):
```python
%%time
# reclustering
# leiden clustering
resolutions_list = [0.2,0.4,0.6,1, 1.2,1.5,2]
for resolution in resolutions_list:
    sc.tl.leiden(adata_sub,neighbors_key=latent_space,
                 resolution=resolution,key_added=f'leiden_{str(resolution)}',n_iterations=2)
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                wspace=0.3)
```

Example 51 (unknown):
```unknown
CPU times: user 7.26 s, sys: 106 ms, total: 7.37 s
Wall time: 7.38 s
```

Example 52 (python):
```python
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",size=10,
                color=genes,
               ncols=6, wspace=0.2,cmap='RdPu',vmax='p100')
```

Example 53 (python):
```python
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color=["conf_score_midmod2fine","celltypist_midmod2fine",],
                    wspace=0.2,cmap='RdPu',vmax='p100')
```

Example 54 (python):
```python
# umap per celltype to highlight
group_col = 'celltypist_midmod2fine'
groups_to_plot = list(set(adata_sub.obs[group_col]))

plt.rcParams['axes.grid'] = False
fig,axs=plt.subplots(nrows=2,ncols=4,figsize=(15,8))
# Flatten the axs array for easier iteration in case of multiple rows and columns
axs_flat = axs.flatten()

for i,grp in enumerate(groups_to_plot):
    sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                    color=[group_col], groups=grp, 
               size=20, frameon=False,
               show=False, ax=axs_flat[i], legend_loc=None, legend_fontsize=5,
               title=grp
              )
# Turn off any remaining subplots
for j in range(len(groups_to_plot), len(axs_flat)):
    fig.delaxes(axs_flat[j])

# fig.suptitle('Main Title for the Figure')
fig.tight_layout()
```

Example 55 (unknown):
```unknown
sub_resolution_sel = 0.6
```

Example 56 (python):
```python
markers_others = {
    'CM': ['TTN', 'TNNT2','MYH6',
        'MYH7', 'MYL2', 'FHL2', # vCN
            'NPPA', 'MYL7', 'MYL4'], #aCM
    'FB': ['DCN', 'GCN', 'PDGFRA','COL1A1','COL1A2'],
    'EC': ['VWF', 'PECAM1', 'CDH5','RGCC', 'FABP5',],
    'Peri': ['RGS5', 'ABCC9', 'KCNJ8'],
    'SMC': ['MYH11', 'TAGLN', 'ACTA2'],
    'Meso': ['MSLN', 'WT1', 'BNC1'],
    'Neuro': ['PLP1', 'NRXN1', 'NRXN3','PRPH', 'NEFL', 'NEFM', 'NEFH', 'STMN2', 'SST', 'SLC10A4', 'SLC18A3', 'TH', 'NPY'],
    'Adipo': ['GPAM', 'FASN', 'LEP'],
    'Myelo': ['CD14', 'C1QA', 'CD68','LYVE1','TIMD4'],
    'Lymph': ['CD8A', 'IL7R', 'CD40LG'],
    'B': ['CD19','SDC1','IGHG1','IGHA1'],
    'Mast': ['KIT', 'CPA3'],
}

for key in markers_others.keys():
    markers_others[key]=[x for x in markers_others[key] if x in adata.var_names]

sc.tl.dendrogram(adata_sub, groupby=f'leiden_{str(sub_resolution_sel)}')
sc.pl.dotplot(adata_sub,
              markers_others, 
              groupby=f'leiden_{str(sub_resolution_sel)}',
              dendrogram=True,
              standard_scale="var",
              color_map="Reds",
              swap_axes=False,
             )
```

Example 57 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: CM, FB, EC, etc.
```

Example 58 (python):
```python
sc.tl.dendrogram(adata_sub,groupby=f'leiden_{str(sub_resolution_sel)}')
sc.pl.dotplot(adata_sub,
              var_names=markers,
              groupby=f'leiden_{str(sub_resolution_sel)}',
              standard_scale='var',
              dendrogram=True
             )
```

Example 59 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: PericardialCellsParietal, PericardialCellsIntermediate, PericardialCellsFibrous, etc.
```

Example 60 (python):
```python
# DEG analysis
print(adata_sub.X.data[:5])
adata_sub.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata_sub,groupby=f'leiden_{str(sub_resolution_sel)}')
sc.tl.dendrogram(adata_sub, groupby=f'leiden_{str(sub_resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata_sub, dendrogram=True, standard_scale='var', return_fig=True)
dp.add_totals().show()
```

Example 61 (unknown):
```unknown
[2.7834363 2.7834363 2.7834363 2.7834363 2.7834363]
```

Example 62 (python):
```python
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color=f'leiden_{str(sub_resolution_sel)}',legend_loc='on data',
                wspace=0.3)
```

Example 63 (python):
```python
# annotate
adata_sub.obs['fine_grain_manual'] = adata_sub.obs[f'leiden_{str(sub_resolution_sel)}'].copy()
adata_sub.obs.replace({'fine_grain_manual':{
    '0':'PericardialCellsFibrous',
    '1':'PericardialCellsFibrous',
    '2':'PericardialCellsIntermediate',
    '3':'PericardialCellsCycling',
    '4':'PericardialCellsIntermediate',
}},inplace=True)
adata_sub.obs['fine_grain_manual'].value_counts()
```

Example 64 (python):
```python
/tmp/ipykernel_419353/2716599662.py:3: FutureWarning: The behavior of Series.replace (and DataFrame.replace) with CategoricalDtype is deprecated. In a future version, replace will only be used for cases that preserve the categories. To change the categories, use ser.cat.rename_categories instead.
  adata_sub.obs.replace({'fine_grain_manual':{
```

Example 65 (unknown):
```unknown
fine_grain_manual
PericardialCellsFibrous         3017
PericardialCellsIntermediate    1553
PericardialCellsCycling          804
Name: count, dtype: int64
```

Example 66 (python):
```python
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color='fine_grain_manual',
                wspace=0.3)
```

Example 67 (python):
```python
sc.tl.dendrogram(adata_sub,groupby='fine_grain_manual')
sc.pl.dotplot(adata_sub,
              var_names=markers,
              groupby='fine_grain_manual',
              standard_scale='var',
              dendrogram=True
             )
```

Example 68 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: PericardialCellsFibrous, PericardialCellsIntermediate, PericardialCellsCycling
var_group_labels: PericardialCellsParietal, PericardialCellsIntermediate, PericardialCellsFibrous, etc.
```

Example 69 (python):
```python
adata.obs['fine_grain_manual'] = adata.obs['tmp_annotation'].astype('str').copy()
adata.obs.loc[adata_sub.obs_names,'fine_grain_manual'] = adata_sub.obs['fine_grain_manual']
```

Example 70 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color='fine_grain_manual'
                )
```

Example 71 (python):
```python
sc.tl.dendrogram(adata,groupby='fine_grain_manual')
sc.pl.dotplot(adata,
              var_names=markers,
              groupby='fine_grain_manual',
              standard_scale='var',
              dendrogram=True
             )
```

Example 72 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: ?ThyroidCells, PericardialCellsCycling, PericardialCellsFibrous, etc.
var_group_labels: PericardialCellsParietal, PericardialCellsIntermediate, PericardialCellsFibrous, etc.
```

Example 73 (python):
```python
for ct in set(adata.obs['fine_grain_manual']):
    for tissue_id in ['C194-HEA-0-FFPE-1', 'Hst45-HEA-0-FFPE-1']:
        slide = adata[adata.obs['tissue_block_id']==tissue_id]
        sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
                      spot_size=60,vmax='p100')
```

Example 74 (python):
```python
/tmp/ipykernel_419353/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 75 (python):
```python
/tmp/ipykernel_419353/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 76 (python):
```python
/tmp/ipykernel_419353/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 77 (python):
```python
/tmp/ipykernel_419353/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 78 (python):
```python
/tmp/ipykernel_419353/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 79 (python):
```python
/tmp/ipykernel_419353/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 80 (python):
```python
/tmp/ipykernel_419353/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 81 (python):
```python
/tmp/ipykernel_419353/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 82 (python):
```python
/tmp/ipykernel_419353/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 83 (python):
```python
/tmp/ipykernel_419353/2414531429.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 84 (python):
```python
# save
adata.write(f'{adata_dir}/{sample_id}_5K_{celltype}_lognorm.h5ad')
print(f'{adata_dir}/{sample_id}_5K_{celltype}_lognorm.h5ad')
```

Example 85 (unknown):
```unknown
/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium/subsets/C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat_5K_PericardialCells_lognorm.h5ad
```

---

## Variables¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/VisiumHD/no7_manual-annotation_lymphoid.html

**Contents:**
- Variables¶
- Read in data¶
- Remove mid-grain markers from the feature¶
- PCA, Clustering, UMAP¶
- Save¶

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os

import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 

import celltypist
```

Example 2 (python):
```python
# import session_info
# session_info.show()
```

Example 3 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 4 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 5 (python):
```python
adata_dir = '/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/subsets'
sample_id = 'HEA_FOET14880396'
celltype = 'LymphoidCells'
```

Example 6 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk/Aug2024'
table_dir = '/nfs/team205/heart/fetal_ms_figures/tables/Aug2024'
```

Example 7 (python):
```python
# read in
adata = sc.read_h5ad(f'{adata_dir}/{sample_id}_{celltype}_b2c_cells_filtered_lognorm.h5ad')
adata
```

Example 8 (unknown):
```unknown
AnnData object with n_obs × n_vars = 87 × 10288
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'leiden_0.2', 'leiden_0.4', 'leiden_0.6', 'leiden_1', 'leiden_1.2', 'leiden_1.5', 'leiden_2'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1.2', 'leiden_1.2_colors', 'leiden_1.5', 'leiden_1.5_colors', 'leiden_1_colors', 'leiden_2', 'leiden_2_colors', 'log1p', 'neighbors', 'pca', 'spatial', 'umap'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 9 (python):
```python
# read in
midgrain_markers = pd.read_csv('/nfs/team205/kk18/notebooks/Foetal/VisiumHD/re-seq_mpp0p3/sc_midgrain_markers.csv',
                               index_col=0
                              )
# remove the markers of celltype of interest
midgrain_markers = midgrain_markers[midgrain_markers['group']!=celltype]
midgrain_markers.head()
```

Example 10 (unknown):
```unknown
'GATA2' in list(midgrain_markers['names'])
```

Example 11 (python):
```python
# remove
mask = [x not in list(midgrain_markers['names']) for x in adata.var_names]
bdata = adata[:,mask]
bdata
```

Example 12 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 87 × 9290
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'leiden_0.2', 'leiden_0.4', 'leiden_0.6', 'leiden_1', 'leiden_1.2', 'leiden_1.5', 'leiden_2'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1.2', 'leiden_1.2_colors', 'leiden_1.5', 'leiden_1.5_colors', 'leiden_1_colors', 'leiden_2', 'leiden_2_colors', 'log1p', 'neighbors', 'pca', 'spatial', 'umap'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 13 (python):
```python
# remove cell cycle genes
cell_cycle_genes = ['MCM5','PCNA','TYMS','FEN1','MCM2','MCM4','RRM1','UNG','GINS2','MCM6','CDCA7','DTL','PRIM1','UHRF1','MLF1IP','HELLS','RFC2','RPA2','NASP',
                'RAD51AP1','GMNN','WDR76','SLBP','CCNE2','UBR7','POLD3','MSH2','ATAD2','RAD51','RRM2','CDC45','CDC6','EXO1','TIPIN','DSCC1','BLM','CASP8AP2',
                'USP1','CLSPN','POLA1','CHAF1B','BRIP1','E2F8','HMGB2','CDK1','NUSAP1','UBE2C','BIRC5','TPX2','TOP2A','NDC80','CKS2','NUF2','CKS1B','MKI67',
                'TMPO','CENPF','TACC3','FAM64A','SMC4','CCNB2','CKAP2L','CKAP2','AURKB','BUB1','KIF11','ANP32E','TUBB4B','GTSE1','KIF20B','HJURP','CDCA3','HN1',
                'CDC20','TTK','CDC25C','KIF2C','RANGAP1','NCAPD2','DLGAP5','CDCA2','CDCA8','ECT2','KIF23','HMMR','AURKA','PSRC1','ANLN','LBR','CKAP5','CENPE','CTCF',
                'NEK2','G2E3','GAS2L3','CBX5','CENPA']
mask = [x not in cell_cycle_genes for x in bdata.var_names]
bdata = bdata[:,mask]
bdata.shape
```

Example 14 (unknown):
```unknown
'MYH11' in list(bdata.var_names)
```

Example 15 (python):
```python
# HVGs, using the object which doesn't have midgrain markers
sc.pp.highly_variable_genes(bdata, flavor='seurat', min_mean=0.0125, max_mean=3, min_disp=0.5)
print(f'number of hvgs: {sum(bdata.var.highly_variable)}')
# PCA using the degected HVGs
sc.tl.pca(bdata, use_highly_variable=True)
adata.obsm['X_pca'] = bdata.obsm['X_pca'].copy()
# create neighbour graph
sc.pp.neighbors(adata,n_neighbors=15,use_rep='X_pca')
# visualisation with umap
sc.tl.umap(adata,min_dist=0.5)
sc.pl.embedding(adata,basis='X_umap',color=['bin_count','total_counts','n_genes_by_counts',"conf_score_midmod2fine","celltypist_midmod2fine",],
                wspace=0.2,cmap='RdPu',vmax='p100',ncols=5)
# leiden clustering
resolutions_list = [0.2,0.4,0.5,0.6,1,1.2,1.5,2]
for resolution in resolutions_list:
    sc.tl.leiden(adata,resolution=resolution,key_added=f'leiden_{str(resolution)}',n_iterations=2)
sc.pl.embedding(adata,basis='X_umap',
                color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                wspace=0.3)
```

Example 16 (unknown):
```unknown
number of hvgs: 2398
```

Example 17 (python):
```python
markers={
     'T cell': ['CD3E', 'CD3D','CD4', 'CD8A', 'CD8B','TRAC','TRBC1','TRDC','TRGC1'],
    'naive': ['TCF7', 'LEF1', 'SELL'],
    'CD4T_reg': ['FOXP3', 'CTLA4', 'IL2RA', 'IKZF2'],
    'cytotoxic': ['PRF1', 'GZMB', 'GZMH', 'NKG7'],
    'NK': ['KLRF1', 'KLRD1', 'NCAM1','FCGR3A','IFNG'],
    'ILC3': ['KIT', 'IL18R1','RORC','IL17A'],
    
    'B': ['CD19','CD20','CD22','CD79A', 'CD79B','MS4A1','IL7R',],
    '':['CCR7','TNFRSF13B',],
    'ProB':['KIT','SPN','CD34'],
    'B1':['CD27','CCR10','MS4A1','SPN'],
    'Plasma': ['CD27','CCR10','CD38','SDC1','JCAIN'],
    'Abs':['IGHM','IGHD', 'IGHG1','IGHG2', 'IGHG3','IGHG4', 'IGHA1','IGHE','IGLC1','IGKC'],
    'Gene editing': ['AICDA', 'RAG1','RAG2'],
    
    
    'Cycling':['MKI67','TOP2A'],

    'TFs': ['TBX21', 'GATA3', 'RORC', 'BCL6',],
}
genes = []
for g in markers.values():
    genes = genes + [x for x in g if (x in adata.var_names)&(x not in genes)]
for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in adata.var_names]
sc.pl.embedding(adata,basis='X_umap',color=genes,
               ncols=6, wspace=0.2,cmap='RdPu',vmax='p99.7')
```

Example 18 (unknown):
```unknown
resolution_sel = 1
```

Example 19 (python):
```python
markers_others = {
    'CM': ['TTN', 'TNNT2','MYH6',
        'MYH7', 'MYL2', 'FHL2', # vCN
            'NPPA', 'MYL7', 'MYL4'], #aCM
    'FB': ['DCN', 'GCN', 'PDGFRA','COL1A1','COL1A2'],
    'EC': ['VWF', 'PECAM1', 'CDH5','RGCC', 'FABP5',],
    'Peri': ['RGS5', 'ABCC9', 'KCNJ8'],
    'SMC': ['MYH11', 'TAGLN', 'ACTA2'],
    'Meso': ['MSLN', 'WT1', 'BNC1'],
    'Neuro': ['PLP1', 'NRXN1', 'NRXN3','PRPH', 'NEFL', 'NEFM', 'NEFH', 'STMN2', 'SST', 'SLC10A4', 'SLC18A3', 'TH', 'NPY'],
    'Adipo': ['GPAM', 'FASN', 'LEP'],
    'Myelo': ['CD14', 'C1QA', 'CD68','LYVE1','TIMD4'],
    'Lymph': ['CD8A', 'IL7R', 'CD40LG'],
    'B': ['CD19','SDC1','IGHG1','IGHA1'],
    'Mast': ['KIT', 'CPA3'],
}

for key in markers_others.keys():
    markers_others[key]=[x for x in markers_others[key] if x in adata.var_names]

sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
sc.pl.dotplot(adata,
              markers_others, 
              groupby=f'leiden_{str(resolution_sel)}',
              dendrogram=True,
              standard_scale="var",
              color_map="Reds",
              swap_axes=False,
             )
```

Example 20 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: CM, FB, EC, etc.
```

Example 21 (python):
```python
sc.tl.dendrogram(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.pl.dotplot(adata,
              var_names=markers,
              groupby=f'leiden_{str(resolution_sel)}',
              standard_scale='var',
              dendrogram=True
             )
```

Example 22 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: T cell, naive, CD4T_reg, etc.
```

Example 23 (python):
```python
# DEG analysis
print(adata.X.data[:5])
adata.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata, dendrogram=True, standard_scale='var', return_fig=True)
dp.add_totals().show()
```

Example 24 (unknown):
```unknown
[1.27187559 1.95641568 1.75023687 1.96760567 2.7810601 ]
```

Example 25 (python):
```python
# DEG analysis
print(adata.X.data[:5])
adata.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata, dendrogram=True, standard_scale='var', return_fig=True, n_genes=5)
dp.add_totals().show()
```

Example 26 (unknown):
```unknown
[1.27187559 1.95641568 1.75023687 1.96760567 2.7810601 ]
```

Example 27 (python):
```python
sc.pl.embedding(adata,basis='X_umap',
                color=f'leiden_{str(resolution_sel)}',
                wspace=0.3, legend_loc='on data')
```

Example 28 (python):
```python
sc.pl.embedding(adata,basis='X_umap',
                color=['celltypist_midmod2fine', 'conf_score_midmod2fine'],
                wspace=0.7)
```

Example 29 (python):
```python
sc.pl.violin(adata,keys=['total_counts','bin_count'],groupby=f'leiden_{str(resolution_sel)}')
```

Example 30 (unknown):
```unknown
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 31 (python):
```python
# annotate
adata.obs['fine_grain_manual'] = adata.obs[f'leiden_{str(resolution_sel)}'].copy()
adata.obs.replace({'fine_grain_manual':{
    '0':'InnateLymphoidCells',
    '1':'InnateLymphoidCells',
    '2':'InnateLymphoidCells',
    '3':'BCells',
    '4':'InnateLymphoidCells',
}},inplace=True)
adata.obs['fine_grain_manual'].value_counts()
```

Example 32 (unknown):
```unknown
fine_grain_manual
InnateLymphoidCells    69
BCells                 18
Name: count, dtype: int64
```

Example 33 (python):
```python
ad = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/Feb28ObjectRaw_finegrain_updated.h5ad',backed='r')
ad[ad.obs['mid_grain']==celltype].obs['fine_grain'].value_counts()
```

Example 34 (unknown):
```unknown
fine_grain
TCellsCD4pos           5660
BCellsMS4A1pos         3470
NaturalKillerCells     2483
TCellsCD8pos           2297
TregsCD4pos            2096
BCells                 2058
InnateLymphoidCells    1719
ProBCells              1301
Name: count, dtype: int64
```

Example 35 (python):
```python
sc.pl.embedding(adata,basis='X_umap',
                color='fine_grain_manual',
                frameon=False,
                wspace=0.3)
```

Example 36 (python):
```python
sc.tl.dendrogram(adata,groupby='fine_grain_manual')
sc.pl.dotplot(adata,
              var_names=markers,
              groupby='fine_grain_manual',
              standard_scale='var',
              dendrogram=True
             )
```

Example 37 (unknown):
```unknown
WARNING: Dendrogram not added. Dendrogram is added only when the number of categories to plot > 2
```

Example 38 (python):
```python
# save
adata.write(f'{adata_dir}/{sample_id}_{celltype}_b2c_cells_filtered_lognorm.h5ad')
print(f'{adata_dir}/{sample_id}_{celltype}_b2c_cells_filtered_lognorm.h5ad')
```

---

## Read the objects¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig2_Overview_of_cellular_niches_and_TissueTypist/tissuetypist_VisiumHD_map_tt-prediction-to-b2c.html

**Contents:**
- Read the objects¶
- Import annotations from loupe browser¶
- Plot¶
- Save cdata¶

311232 rows × 1 columns

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/session_info/main.py:213: UserWarning: The '__version__' attribute is deprecated and will be removed in MarkupSafe 3.1. Use feature detection, or `importlib.metadata.version("markupsafe")`, instead.
  mod_version = _find_version(mod.__version__)
```

Example 4 (python):
```python
-----
matplotlib          3.10.1
numpy               2.1.3
pandas              2.2.3
scanpy              1.11.1
seaborn             0.13.2
session_info        v1.0.1
-----
```

Example 5 (python):
```python
Cython              3.1.1
PIL                 11.1.0
anndata             0.11.4
asciitree           NA
asttokens           NA
charset_normalizer  3.4.1
cloudpickle         3.1.1
comm                0.2.2
cycler              0.12.1
cython              3.1.1
cython_runtime      NA
dask                2024.11.2
dateutil            2.9.0.post0
debugpy             1.8.13
decorator           5.2.1
exceptiongroup      1.2.2
executing           2.2.0
h5py                3.13.0
igraph              0.11.8
importlib_metadata  NA
ipykernel           6.29.5
jedi                0.19.2
jinja2              3.1.6
joblib              1.4.2
kiwisolver          1.4.8
legacy_api_wrap     NA
leidenalg           0.10.2
llvmlite            0.44.0
markupsafe          3.0.2
mpl_toolkits        NA
msgpack             1.1.0
natsort             8.4.0
numba               0.61.0
numcodecs           0.13.1
packaging           24.2
parso               0.8.4
patsy               1.0.1
platformdirs        4.3.7
prompt_toolkit      3.0.50
psutil              7.0.0
pure_eval           0.2.3
pyarrow             19.0.1
pydev_ipython       NA
pydevconsole        NA
pydevd              3.2.3
pydevd_file_utils   NA
pydevd_plugins      NA
pydevd_tracing      NA
pygments            2.19.1
pyparsing           3.2.3
pytz                2025.2
scipy               1.15.2
six                 1.17.0
sklearn             1.5.2
stack_data          0.6.3
statsmodels         0.14.4
tblib               3.1.0
texttable           1.7.0
threadpoolctl       3.6.0
tlz                 1.0.0
toolz               1.0.0
tornado             6.4.2
traitlets           5.14.3
typing_extensions   NA
wcwidth             0.2.13
yaml                6.0.2
zarr                2.18.3
zipp                NA
zmq                 26.3.0
zoneinfo            NA
```

Example 6 (unknown):
```unknown
-----
IPython             8.34.0
jupyter_client      8.6.3
jupyter_core        5.7.2
-----
Python 3.10.14 | packaged by conda-forge | (main, Mar 20 2024, 12:45:18) [GCC 12.3.0]
Linux-4.18.0-553.53.1.el8_10.x86_64-x86_64-with-glibc2.28
-----
Session information updated at 2025-07-06 10:18
```

Example 7 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 8 (unknown):
```unknown
os.getcwd()
```

Example 9 (unknown):
```unknown
'/rfs/project/rfs-iCNyzSAaucw/kk837/notebooks/Foetal/TissueTypist/CamU/tissuetypist_package'
```

Example 10 (python):
```python
adata_dir = '/rfs/project/rfs-iCNyzSAaucw/kk837/data_objects/Foetal/VisiumHD'
sample_id = 'HEA_FOET14880396'
```

Example 11 (python):
```python
!ls {adata_dir}
```

Example 12 (unknown):
```unknown
archived-Dec2024_HEA_FOET14880396_b2c.h5ad
cellcharter
cellphonedb
for_Theo
HEA_AD14880301_b2c.h5ad
HEA_FOET14880396_8um-bins_filtered.h5ad
HEA_FOET14880396_8um-bins_filtered_lognorm.h5ad
HEA_FOET14880396_b2c_cells_filtered_celltype-selected_lognorm.h5ad
HEA_FOET14880396_b2c_cells_filtered.h5ad
HEA_FOET14880396_b2c_cells_filtered_lognorm.h5ad
HEA_FOET14880396_b2c_cells_filtered_lognorm_morpho_obs.csv
HEA_FOET14880396_b2c_cells.h5ad
HEA_FOET14880396_b2c.h5ad
HEA_FOET14880397_b2c_cells_filtered.h5ad
HEA_FOET14880397_b2c_cells_filtered_lognorm.h5ad
HEA_FOET14880397_b2c_cells.h5ad
HEA_FOET14880397_b2c.h5ad
subsets
```

Example 13 (python):
```python
# read in b2c object
adata = sc.read_h5ad(f'{adata_dir}/{sample_id}_b2c.h5ad') # read 2um object which has b2c cell grouping - see tutorial - https://nbviewer.org/github/Teichlab/bin2cell/blob/main/notebooks/demo.ipynb 
cdata = sc.read_h5ad(f'{adata_dir}/{sample_id}_b2c_cells_filtered_lognorm.h5ad') # read b2c object
```

Example 14 (python):
```python
# read in 8um object to get tt_prediction results
adata_8um = sc.read_h5ad(f'{adata_dir}/{sample_id}_8um-bins_filtered.h5ad',backed='r')
adata_8um
```

Example 15 (unknown):
```unknown
AnnData object with n_obs × n_vars = 311232 × 18085 backed at '/rfs/project/rfs-iCNyzSAaucw/kk837/data_objects/Foetal/VisiumHD/HEA_FOET14880396_8um-bins_filtered.h5ad'
    obs: 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'n_genes', 'binned_8_row', 'binned_8_col', 'section', 'sliding_window_assignment', 'window_col', 'window_row', 'tt_prediction'
    var: 'gene_ids', 'feature_types', 'genome', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts'
```

Example 16 (python):
```python
anno_df = adata_8um.obs[['tt_prediction']].copy()
anno = 'tt_prediction'
cell_grouping = 'labels_joint'
```

Example 17 (python):
```python
def map_loupe_to_b2c(
        adata,
        cdata,
        anno_df,
        anno,
        cell_grouping = 'labels_he_expanded', 
        ):
     

    # Calculate the binned row and column indices in adata
    adata.obs['binned_8_row'] = adata.obs['array_row'] // 4
    adata.obs['binned_8_col'] = adata.obs['array_col'] // 4

    # Create a combined index in adata and set it as the index
    adata.obs['mapping_index_08'] = 's_008um_' + adata.obs['binned_8_row'].astype(str).str.zfill(5) + '_' + adata.obs['binned_8_col'].astype(str).str.zfill(5) + '-1'
    adata.obs['mapping_index_08'].value_counts() # most should have 16 but there are some on the tissue edge that actually have less due to partial overlap 

    # read annotations from df and map to the 2um object 
    print('annotations to map:')
    print(anno_df.head(5))
    mapping_dict = pd.Series(anno_df[anno].values,index=anno_df.index).to_dict()
    adata.obs[anno] = adata.obs['mapping_index_08'].map(mapping_dict) # update 2um object 

    # assign annotation to cells by pandas.Series.mode(). The mode is the value that appears most often which captures the most frequent annotation for that cell from 2um data
    filtered_data = adata.obs[(adata.obs[cell_grouping] != 0) & adata.obs[anno].notna()]
    cell_annotation = (
        filtered_data.groupby(cell_grouping)[anno]
        .apply(lambda x: x.mode().iloc[0] if not x.mode().empty else None)
    )
    cell_annotation.index = cell_annotation.index.astype('str')
    cdata.obs[anno] = cdata.obs.index.map(cell_annotation).astype('category') # label cells 
    print('Annotations in b2c object:')
    print(cdata.obs[anno].value_counts())

map_loupe_to_b2c(adata,
                 cdata,
                 anno_df,
                 anno,
                 cell_grouping)
```

Example 18 (unknown):
```unknown
annotations to map:
                                  tt_prediction
s_008um_00301_00321-1                   AV ring
s_008um_00602_00290-1              Great vessel
s_008um_00377_00750-1  Ventricle - Trabeculated
s_008um_00383_00696-1  Ventricle - Trabeculated
s_008um_00526_00291-1              Great vessel
Annotations in b2c object:
tt_prediction
Ventricle - Compact          43759
Adventitia - GV              24215
Atrium                       11901
Great vessel                 11489
Ventricle - Trabeculated      8742
VCS - Distal                  5460
VCS - Proximal                4430
AV ring                       4371
Endocardium - Atrial          2857
Epicardium                    2108
Ductus arteriosus             2050
Endocardium - Ventricular     1964
SAnode                        1559
Endocardial cusion            1264
AVnode                         674
Coronary vessel                625
Valves                         526
Name: count, dtype: int64
```

Example 19 (unknown):
```unknown
AnnotationPalette={ 
    'Atrium':'#eac075',
        'Ventricle - Compact':'#f1acfa',
     'Ventricle - Trabeculated':'#f09951',
        'SAnode':'#8A2BE2',
        'AVnode':'#062b4a',
    'VCS - Proximal':'#FFFF00',
        'VCS - Distal':'#7FFFD4',
        'Epicardium':'#01ccc8',
        'Endocardium - Atrial':'#DC143C',
        'Endocardium - Ventricular':'#f53527',
        'Endocardial cusion':'#ff7e97',
        'Great vessel':'#ff9b6e',
    'Ductus arteriosus':'#8a6800',
        'Coronary vessel':'#206601',
        'Adventitia - Coronary':'#CAFF70',
     'Adventitia - GV':'#005aa0',
        'AV ring':'#e478ff',
    'Valves':'#3300FF',
    'LN':'#9088ff',
}
```

Example 20 (python):
```python
sc.settings.set_figure_params(dpi=120)
sc.pl.spatial(cdata, 
                  color=anno, 
                  img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer",
                  size=1.5,alpha_img=0.3,
                  frameon=False,palette=AnnotationPalette
                 )
```

Example 21 (python):
```python
/tmp/ipykernel_237158/3235274587.py:2: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(cdata,
```

Example 22 (python):
```python
cdata.write(f'{adata_dir}/{sample_id}_b2c_cells_filtered_lognorm.h5ad')
cdata
```

Example 23 (unknown):
```unknown
AnnData object with n_obs × n_vars = 129803 × 18047
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'fine_grain_4Jan2025', 'HistologicalAnnotation', 'SANaxis_landmarks', 'SAN_axis', 'SAN_axis_bin_annot', 'tt_prediction'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'HistologicalAnnotation_colors', 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'fine_grain_4Jan2025_colors', 'hvg', 'log1p', 'neighbors', 'pca', 'spatial', 'umap', 'tt_prediction_colors'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

---

## Read in GSEA results¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig5_Trisomy21/no13_GSEA_summary_finegrain.html

**Contents:**
- Read in GSEA results¶
- Plot: Euploid nhoods vs reference nhoods¶

**Examples:**

Example 1 (python):
```python
import scanpy as sc
import pandas as pd
import numpy as np
import pandas as pd
import anndata
import scipy
import matplotlib.pyplot as plt
import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42
import seaborn as sns

import os
import warnings
warnings.simplefilter("ignore", UserWarning)
warnings.simplefilter(action='ignore', category=pd.errors.PerformanceWarning)
```

Example 2 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 3 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk/Aug2024'
table_dir = '/nfs/team205/heart/fetal_ms_figures/tables/Aug2024'
```

Example 4 (python):
```python
gsea_res_df = pd.read_csv('/nfs/team205/kk18/notebooks/Foetal/Trisomy21/RNA/milo_Mar2025_refine-nhoods/GSEA_trisomy21-vs-euploid-nhoods_finegrain.csv',
                                index_col=0)
print(gsea_res_df.shape)
gsea_res_df.head()
```

Example 5 (unknown):
```unknown
(20860, 10)
```

Example 6 (python):
```python
from matplotlib.colors import ListedColormap
```

Example 7 (python):
```python
fdr_thresh = 0.1
geneset = 'MSigDB_Hallmark_2020'
plt_title = f'Trisomy21-nhood vs Euploid-nhood, {geneset}'
significant_only = True
figsize = (8,5.5)

#################################
# prepare dataframes
df = gsea_res_df[gsea_res_df['geneset'] == geneset]
df_nes = df.pivot(columns='Term',index='celltype',values='nes')
df_nes = df_nes.fillna(0)
df_sig = df.pivot(columns='Term',index='celltype',values='fdr')
df_sig = df_sig<fdr_thresh

if significant_only:
    # select pathways which have significant NES at least in one cell type
    pathways_sel = list(df_sig.columns[df_sig.sum()>0])
    df_nes = df_nes[pathways_sel]
    df_sig = df_sig[pathways_sel]

# plotting
plt.rcParams['axes.grid'] = False
# Custom colormap: add grey for masked areas
g = sns.clustermap(df_nes,figsize=figsize,cmap='coolwarm',center=0, # vmax=5,
                  dendrogram_ratio=0.05, cbar_pos=(-0.07, 0.9, 0.02, 0.10)
                  ) # 

# Get the reordered indices for rows and columns
row_order = g.dendrogram_row.reordered_ind
col_order = g.dendrogram_col.reordered_ind

# Reorder the mask based on the clustering result
df_sig_reorder = df_sig.iloc[row_order, :].iloc[:, col_order]

# Iterate over the mask and add asterisks
for i in range(df_nes.shape[0]):
    for j in range(df_nes.shape[1]):
        if df_sig_reorder.iloc[i, j]:
            g.ax_heatmap.text(j + 0.5, i + 0.7, '*', ha='center', va='center', color='black', fontsize=14)
            
# Add a title to the colorbar
colorbar = g.ax_heatmap.collections[0].colorbar
colorbar.set_label('NES', rotation=270, labelpad=15)
plt.title(plt_title)
plt.show()
```

Example 8 (python):
```python
fdr_thresh = 0.1
geneset = 'KEGG_2021_Human'
plt_title = f'Trisomy21-nhood vs Euploid-nhood, {geneset}'
significant_only = True
figsize = (12,6)

#################################
# prepare dataframes
df = gsea_res_df[gsea_res_df['geneset'] == geneset]
df_nes = df.pivot(columns='Term',index='celltype',values='nes')
df_nes = df_nes.fillna(0)
df_sig = df.pivot(columns='Term',index='celltype',values='fdr')
df_sig = df_sig<fdr_thresh

if significant_only:
    # select pathways which have significant NES at least in one cell type
    pathways_sel = list(df_sig.columns[df_sig.sum()>0])
    df_nes = df_nes[pathways_sel]
    df_sig = df_sig[pathways_sel]

# plotting
plt.rcParams['axes.grid'] = False
# Custom colormap: add grey for masked areas
g = sns.clustermap(df_nes,figsize=figsize,cmap='coolwarm',center=0, # vmax=5,
                  dendrogram_ratio=0.05, cbar_pos=(-0.07, 0.9, 0.02, 0.10)
                  ) # 

# Get the reordered indices for rows and columns
row_order = g.dendrogram_row.reordered_ind
col_order = g.dendrogram_col.reordered_ind

# Reorder the mask based on the clustering result
df_sig_reorder = df_sig.iloc[row_order, :].iloc[:, col_order]

# Iterate over the mask and add asterisks
for i in range(df_nes.shape[0]):
    for j in range(df_nes.shape[1]):
        if df_sig_reorder.iloc[i, j]:
            g.ax_heatmap.text(j + 0.5, i + 0.7, '*', ha='center', va='center', color='black', fontsize=14)
            
# Add a title to the colorbar
colorbar = g.ax_heatmap.collections[0].colorbar
colorbar.set_label('NES', rotation=270, labelpad=15)
plt.title(plt_title)
plt.show()
```

Example 9 (python):
```python
fdr_thresh = 0.1
genesets = ['MSigDB_Hallmark_2020','KEGG_2021_Human']
remove_keywords = ['infection','disease','cancer','cardiomyopathy','shigellosis',
                   'malaria','hepatitis','bacterial','diabetes','atherosclerosis','sclerosis',
                   'carcinogenesis'
                  ]
plt_title = f'Trisomy21-nhood vs Euploid-nhood, {geneset}'
significant_only = True
figsize = (12,6.5)
savepath = f'{figdir}/Tri21_Milo-reassigned-DEGs_T21-vs-EUPnhoods_GSEA_MSigDB-KEGG.pdf'

#################################
# prepare dataframes
df = gsea_res_df.copy()
df = df[df['geneset'].isin(genesets)]
df.replace({'geneset':{
    'MSigDB_Hallmark_2020':'MH',
    'KEGG_2021_Human':'KEGG',
    'GO_Biological_Process_2023':'GOBP',
    'Reactome_2022':'RT'
}},inplace=True)
df['Term'] = df['geneset'].astype('str')+': '+df['Term'].astype('str')
df_nes = df.pivot(columns='Term',index='celltype',values='nes')
df_nes = df_nes.fillna(0)
df_sig = df.pivot(columns='Term',index='celltype',values='fdr')
df_sig = df_sig<fdr_thresh

if significant_only:
    # select pathways which have significant NES at least in one cell type
    pathways_sel = list(df_sig.columns[df_sig.sum()>0])
    df_nes = df_nes[pathways_sel]
    df_sig = df_sig[pathways_sel]

if remove_keywords!=None:
    pathways_sel = [x for x in df_nes.columns if all([xx.lower() not in x.lower() for xx in remove_keywords])]
    df_nes = df_nes[pathways_sel]
    df_sig = df_sig[pathways_sel]

# plotting
plt.rcParams['axes.grid'] = False
# Custom colormap: add grey for masked areas
g = sns.clustermap(df_nes,figsize=figsize,cmap='coolwarm',center=0, # vmax=5,
                  dendrogram_ratio=0.05, cbar_pos=(-0.07, 0.9, 0.02, 0.10)
                  ) # 

# Get the reordered indices for rows and columns
row_order = g.dendrogram_row.reordered_ind
col_order = g.dendrogram_col.reordered_ind

# Reorder the mask based on the clustering result
df_sig_reorder = df_sig.iloc[row_order, :].iloc[:, col_order]

# Iterate over the mask and add asterisks
for i in range(df_nes.shape[0]):
    for j in range(df_nes.shape[1]):
        if df_sig_reorder.iloc[i, j]:
            g.ax_heatmap.text(j + 0.5, i + 0.7, '*', ha='center', va='center', color='black', fontsize=14)
            
# Add a title to the colorbar
colorbar = g.ax_heatmap.collections[0].colorbar
colorbar.set_label('NES', rotation=270, labelpad=15)
plt.title(plt_title)
if savepath!=None:
    plt.savefig(savepath, bbox_inches="tight",dpi=300)
plt.show()
```

---

## Functions¶

**URL:** http://127.0.0.1:9020/notebooks_html/Supplementary_materials/Macrophages/Mo-MP_no7_MP-LYVE1_fitting-genes-pathway-analysis.html

**Contents:**
- Functions¶
- Variables¶
- Read in anndata object¶
- Read in pseudotime-fitting output¶
- Read in differential test output (vs root)¶
- Select genes¶
- Gene enrichment analysis¶
- Plotting gene transition¶

15922 rows × 8 columns

15922 rows × 9 columns

111454 rows × 7 columns

**Examples:**

Example 1 (python):
```python
import scanpy as sc
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import anndata
import scipy
import os

import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
-----
anndata             0.9.2
matplotlib          3.7.2
numpy               1.24.4
pandas              1.5.3
scanpy              1.9.3
scipy               1.10.1
seaborn             0.12.2
session_info        1.0.0
-----
```

Example 4 (python):
```python
PIL                         10.0.0
anyio                       NA
arrow                       1.2.3
asttokens                   NA
attr                        23.1.0
attrs                       23.1.0
babel                       2.12.1
backcall                    0.2.0
brotli                      1.0.9
certifi                     2023.11.17
cffi                        1.15.1
charset_normalizer          3.2.0
colorama                    0.4.6
comm                        0.1.4
cycler                      0.10.0
cython_runtime              NA
dateutil                    2.8.2
debugpy                     1.6.8
decorator                   5.1.1
defusedxml                  0.7.1
executing                   1.2.0
fastjsonschema              NA
fqdn                        NA
h5py                        3.9.0
idna                        3.4
igraph                      0.10.6
importlib_metadata          NA
importlib_resources         NA
ipykernel                   6.25.1
isoduration                 NA
jedi                        0.19.0
jinja2                      3.1.2
joblib                      1.3.2
json5                       NA
jsonpointer                 2.0
jsonschema                  4.19.0
jsonschema_specifications   NA
jupyter_events              0.7.0
jupyter_server              2.7.1
jupyterlab_server           2.24.0
kiwisolver                  1.4.5
leidenalg                   0.10.1
llvmlite                    0.40.1
markupsafe                  2.1.3
mpl_toolkits                NA
natsort                     8.4.0
nbformat                    5.9.2
numba                       0.57.1
overrides                   NA
packaging                   23.1
parso                       0.8.3
patsy                       0.5.3
pexpect                     4.8.0
pickleshare                 0.7.5
pkg_resources               NA
plotly                      5.16.1
prometheus_client           NA
prompt_toolkit              3.0.39
psutil                      5.9.5
ptyprocess                  0.7.0
pure_eval                   0.2.2
pydev_ipython               NA
pydevconsole                NA
pydevd                      2.9.5
pydevd_file_utils           NA
pydevd_plugins              NA
pydevd_tracing              NA
pygments                    2.16.1
pyparsing                   3.0.9
pythonjsonlogger            NA
pytz                        2023.3
referencing                 NA
requests                    2.31.0
rfc3339_validator           0.1.4
rfc3986_validator           0.1.1
rpds                        NA
ruamel                      NA
send2trash                  NA
six                         1.16.0
sklearn                     1.3.0
sniffio                     1.3.0
socks                       1.7.1
sphinxcontrib               NA
stack_data                  0.6.2
statsmodels                 0.14.0
texttable                   1.6.7
threadpoolctl               3.2.0
tornado                     6.3.3
traitlets                   5.9.0
typing_extensions           NA
uri_template                NA
urllib3                     2.0.4
wcwidth                     0.2.6
webcolors                   1.13
websocket                   1.6.2
yaml                        6.0.1
zipp                        NA
zmq                         25.1.1
zstandard                   0.19.0
```

Example 5 (unknown):
```unknown
-----
IPython             8.12.0
jupyter_client      8.3.0
jupyter_core        4.12.0
jupyterlab          4.0.5
notebook            7.0.2
-----
Python 3.8.17 (default, Jul  5 2023, 21:04:15) [GCC 11.2.0]
Linux-5.4.0-169-generic-x86_64-with-glibc2.17
-----
Session information updated at 2024-02-07 14:28
```

Example 6 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 7 (python):
```python
sc.settings.set_figure_params(dpi=300,dpi_save=300)
```

Example 8 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk'
outdir = '/nfs/team205/heart/anndata_objects/Foetal/Immune/MoMP_Aug2023/seg_3-4-5-6-7-8-9'
os.listdir(outdir)
```

Example 9 (python):
```python
['adata.h5ad', 'differential-test_result.pkl', 'fitting_result.pkl']
```

Example 10 (python):
```python
adata = sc.read_h5ad(f'{outdir}/adata.h5ad')
adata
```

Example 11 (python):
```python
AnnData object with n_obs × n_vars = 3886 × 142283
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', '_indices', '_scvi_batch', '_scvi_labels', 'fine_grain_withcycling', 't', 'seg', 'edge', 't_sd', 'milestones', 'fine_grain_mod'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id', 'n_cells'
    uns: '_scvi_manager_uuid', '_scvi_uuid', 'adata_auc_gene', 'adata_auc_region', 'adata_gex', 'adata_gex_lognorm', 'adata_peak_tfidf', 'draw_graph', 'fine_grain_colors', 'fine_grain_mod_colors', 'fine_grain_withcycling_colors', 'graph', 'milestones_colors', 'neighbors', 'ppt', 'pseudotime_list', 'seg_colors', 'umap', 'week_colors'
    obsm: 'MultiVI_latent', 'X_R', 'X_draw_graph_fa', 'X_palantir', 'X_umap', 'X_umap13', '_scvi_extra_categorical_covs', '_scvi_extra_continuous_covs'
    obsp: 'connectivities', 'distances'
```

Example 12 (python):
```python
# scale t
# min-max scaling
adata.obs['t_scaled'] = (adata.obs['t']-adata.obs['t'].min())/(adata.obs['t'].max()-adata.obs['t'].min())
```

Example 13 (python):
```python
# summarise gene expression
adata_gex = adata.uns['adata_gex_lognorm'].copy()
print(adata_gex.X.data[:10])

# scale t
# min-max scaling
adata_gex.obs['t_scaled'] = (adata_gex.obs['t']-adata_gex.obs['t'].min())/(adata_gex.obs['t'].max()-adata_gex.obs['t'].min())
```

Example 14 (unknown):
```unknown
[1.0841573 2.6877544 1.0841573 2.1779046 1.0841573 2.1779046 1.0841573
 1.0841573 2.9238348 2.3781319]
```

Example 15 (python):
```python
group_mean_exp = utils.grouped_obs_mean(adata_gex, group_key='milestones')
group_mean_exp
```

Example 16 (unknown):
```unknown
# read in
fit_res_dict = utils.read_pkl(f'{outdir}/fitting_result.pkl')
fit_res = fit_res_dict['gene'].copy()
fit_res
```

Example 17 (unknown):
```unknown
# read in
diff_res_dict = utils.read_pkl(f'{outdir}/differential-test_result.pkl')
diff_res = diff_res_dict['gene'].copy()

# add mean gene expression
df = group_mean_exp.reset_index().melt(id_vars=['index'])
df.rename(columns={'index':'names','variable':'group','value':'mean_gene_exp'},
          inplace=True)
# add
diff_res = diff_res.merge(df, how='left', on=['group','names'])

diff_res
```

Example 18 (unknown):
```unknown
diff_res[diff_res['names']=='DOCK4']
```

Example 19 (unknown):
```unknown
set(diff_res['group'])
```

Example 20 (unknown):
```unknown
{'16', '20', '28', '32', '44', '47', '5'}
```

Example 21 (python):
```python
sns.scatterplot(data=diff_res, x='logfoldchanges',y='scores')
```

Example 22 (unknown):
```unknown
<Axes: xlabel='logfoldchanges', ylabel='scores'>
```

Example 23 (python):
```python
sns.scatterplot(data=diff_res,x='logfoldchanges',y='pvals_adj')
```

Example 24 (unknown):
```unknown
<Axes: xlabel='logfoldchanges', ylabel='pvals_adj'>
```

Example 25 (python):
```python
sns.scatterplot(data=diff_res,x='logfoldchanges',y='mean_gene_exp')
```

Example 26 (unknown):
```unknown
<Axes: xlabel='logfoldchanges', ylabel='mean_gene_exp'>
```

Example 27 (python):
```python
diff_logFC_thresh = 0.5
diff_pval_thresh = 0.05
diff_meanexp_thresh = 0.5
fit_pval_thresh = 0.05
fit_amplitude_thresh = 0.5

up_genes = list(set(diff_res[(diff_res['logfoldchanges']>diff_logFC_thresh)&\
                             (diff_res['pvals_adj']<diff_pval_thresh)&\
                             (diff_res['mean_gene_exp']>diff_meanexp_thresh)
                            ]['names']))
print(f'Upregulated gene number: {len(up_genes)}')
fit_genes = list(set(fit_res[(fit_res['fdr']<fit_pval_thresh)&(fit_res['A']>fit_amplitude_thresh)]['gene_name_scRNA-0']))
print(f'Fitted gene number: {len(fit_genes)}')

# take intersection
genes_of_interest = list(set(up_genes).intersection(fit_genes))
print(f'Final gene number: {len(genes_of_interest)}')
```

Example 28 (unknown):
```unknown
Upregulated gene number: 996
Fitted gene number: 2948
Final gene number: 956
```

Example 29 (python):
```python
# ligands
ligands = list(pd.read_table('/nfs/team205/kk18/data/geneset/HGNC/HGNC_542_Receptor_ligands.txt')['Approved symbol'])
print(len(ligands))
genes_of_interest_ligands = list(set(genes_of_interest).intersection(ligands))
print(len(genes_of_interest_ligands))
```

Example 30 (unknown):
```unknown
genes_of_interest_ligands
```

Example 31 (unknown):
```unknown
['LGALS1',
 'CFLAR',
 'TNFSF13B',
 'PSAP',
 'SPP1',
 'ICAM1',
 'RGS2',
 'ANXA1',
 'LGALS9',
 'CCL3',
 'ADM',
 'CCL4',
 'APP',
 'CCL4L2',
 'CXCL8',
 'VEGFA',
 'CCL3L1',
 'IL1B',
 'EDA']
```

Example 32 (python):
```python
import gseapy as gp
```

Example 33 (python):
```python
geneset_list = ['MSigDB_Hallmark_2020','KEGG_2021_Human','GO_Biological_Process_2023','Reactome_2022']
enr = gp.enrichr(gene_list=genes_of_interest, # or "./tests/data/gene_list.txt",
                 gene_sets=geneset_list,
                 organism='human', # don't forget to set organism to the one you desired! e.g. Yeast
                 outdir=None, # don't write to disk
                )

geneset = 'MSigDB_Hallmark_2020'
df = enr.results[enr.results['Gene_set'] == geneset]
df = df[df['Adjusted P-value'] < 0.1]
df = df.sort_values('Combined Score',ascending=False)

# plot
n_rank = 10
plt.rcParams['axes.grid'] = False
plt.rcParams['figure.figsize'] = 4,3
plt.barh(width=df[:n_rank]['Combined Score'],
         y=[x.split(' (')[0] for x in df[:n_rank]['Term']],
         color='darkred')
plt.gca().invert_yaxis()
plt.xlabel('Score')
plt.title(f'All genes\n({geneset})')
plt.margins(y=0.02)
plt.savefig(f"{figdir}/MoMP_LYVE1-up-genes_all-genes_pathways.pdf", bbox_inches="tight",dpi=300)
plt.show()

for geneset in geneset_list:
    print(geneset)
    df = enr.results[enr.results['Gene_set'] == geneset]
    display(df.sort_values('Combined Score',ascending=False)[:30])
```

Example 34 (unknown):
```unknown
MSigDB_Hallmark_2020
```

Example 35 (unknown):
```unknown
KEGG_2021_Human
```

Example 36 (unknown):
```unknown
GO_Biological_Process_2023
```

Example 37 (unknown):
```unknown
Reactome_2022
```

Example 38 (unknown):
```unknown
df = enr.results[enr.results['Gene_set'] == 'MSigDB_Hallmark_2020']
genes = df.set_index('Term').loc['TNF-alpha Signaling via NF-kB','Genes']
genes = genes.split(';')
genes
```

Example 39 (python):
```python
# ligands

geneset_list = ['MSigDB_Hallmark_2020','KEGG_2021_Human','GO_Biological_Process_2023','Reactome_2022']
enr = gp.enrichr(gene_list=genes_of_interest_ligands, # or "./tests/data/gene_list.txt",
                 gene_sets=geneset_list,
                 organism='human', # don't forget to set organism to the one you desired! e.g. Yeast
                 outdir=None, # don't write to disk
                )

geneset = 'MSigDB_Hallmark_2020'
df = enr.results[enr.results['Gene_set'] == geneset]
df = df[df['Adjusted P-value'] < 0.1]
df = df.sort_values('Combined Score',ascending=False)

# plot
n_rank = 10
plt.rcParams['axes.grid'] = False
plt.rcParams['figure.figsize'] = 4,3
plt.barh(width=df[:n_rank]['Combined Score'],
         y=[x.split(' (')[0] for x in df[:n_rank]['Term']],
         color='darkred')
plt.gca().invert_yaxis()
plt.xlabel('Score')
plt.title(f'Ligand genes\n({geneset})')
plt.margins(y=0.02)
plt.savefig(f"{figdir}/MoMP_LYVE1-up-genes_ligands_pathways.pdf", bbox_inches="tight",dpi=300)
plt.show()

for geneset in geneset_list:
    print(geneset)
    df = enr.results[enr.results['Gene_set'] == geneset]
    display(df.sort_values('Combined Score',ascending=False)[:30])
```

Example 40 (unknown):
```unknown
MSigDB_Hallmark_2020
```

Example 41 (unknown):
```unknown
KEGG_2021_Human
```

Example 42 (unknown):
```unknown
GO_Biological_Process_2023
```

Example 43 (unknown):
```unknown
Reactome_2022
```

Example 44 (python):
```python
adata_gex.X.data[:5]
```

Example 45 (unknown):
```unknown
array([1.0841573, 2.6877544, 1.0841573, 2.1779046, 1.0841573],
      dtype=float32)
```

Example 46 (python):
```python
n_pseudotime_bin=8
adata_gex.obs['t_scaled_binned'] = pd.cut(adata_gex.obs['t_scaled'],bins=n_pseudotime_bin)
exp_df = adata_gex[:,genes_of_interest_ligands].to_df()
exp_df['t_scaled_binned'] = adata_gex.obs['t_scaled_binned']
# groupby
exp_df = exp_df.groupby('t_scaled_binned').mean()
# min-max normalise
exp_df = (exp_df-exp_df.min())/(exp_df.max()-exp_df.min())
exp_df = exp_df.T.copy()

plt.rcParams['axes.grid'] = False
g = sns.clustermap(exp_df, col_cluster=False, row_cluster=True,
                   figsize=(3,7),
                  cmap='coolwarm',center=0,
               cbar_pos=(-0.05, 0.7, 0.03, 0.1),
                   xticklabels=True,yticklabels=True)
plt.savefig(f'{figdir}/MP_ligand_genes_heatmap.pdf',bbox_inches="tight",dpi=300)
plt.show()
# get gene order
row_order = g.dendrogram_row.reordered_ind
gene_order = list(exp_df.index[row_order])
```

Example 47 (python):
```python
# smoothing expression
from scipy.interpolate import UnivariateSpline

plt.rcParams['axes.grid'] = False
fig, ax = plt.subplots(figsize=(4,3))
cmap = 'RdPu'

adata_df = adata_gex[:,genes_of_interest_ligands].to_df()
adata_df['t_scaled'] = adata_gex.obs['t_scaled'].copy()
adata_df.sort_values('t_scaled',ascending=True,inplace=True)

pseudotime = adata_df['t_scaled'].values  # Convert to numpy array
pseudotime_reshaped = pseudotime.reshape(-1, 1)
pseudotime_plot = np.linspace(min(pseudotime), max(pseudotime), len(pseudotime))
pseudotime_plot_reshaped = pseudotime_plot.reshape(-1, 1)

gene_order_rev = gene_order.copy()
gene_order_rev.reverse()
for i, gene in enumerate(gene_order_rev):
    spl_res = UnivariateSpline(adata_df['t_scaled'], adata_df[gene], k=2)
    gene_smoothed_values = spl_res(adata_df['t_scaled'])
    # min-max normalise
    gene_smoothed_values_norm = (gene_smoothed_values-np.min(gene_smoothed_values))/(np.max(gene_smoothed_values)-np.min(gene_smoothed_values))
    norm = plt.Normalize(gene_smoothed_values_norm.min(), gene_smoothed_values_norm.max())
    sc = plt.scatter(pseudotime_plot, np.ones_like(pseudotime_plot) * (i + 1), 
                     c=gene_smoothed_values_norm, cmap=cmap, norm=norm, marker='s')

plt.colorbar(sc, label='Smoothed gene expression')

# Remove the box around the plot
ax.spines['top'].set_visible(False)
ax.spines['right'].set_visible(False)
ax.spines['left'].set_visible(False)

# Adjust x-axis position
# ax.set_ylim(0.5, len(sorted_genes) + 0.5)

plt.xlabel('Pseudotime')
# plt.ylabel('Gene')
plt.yticks(np.arange(1, len(gene_order_rev) + 1), gene_order_rev, fontsize='xx-small', fontstyle='italic')
# plt.title(title)
# plt.savefig(fname=fname, bbox_inches='tight')
plt.show()
```

Example 48 (python):
```python
np.min(gene_smoothed_values)
```

Example 49 (unknown):
```unknown
-0.020486605442069778
```

Example 50 (unknown):
```unknown
len(gene_order)
```

Example 51 (python):
```python
pseudotime = adata_df['t_scaled'].values  # Convert to numpy array
pseudotime_reshaped = pseudotime.reshape(-1, 1)
pseudotime_plot = np.linspace(min(pseudotime), max(pseudotime), len(pseudotime))
pseudotime_plot_reshaped = pseudotime_plot.reshape(-1, 1)

nrows = 4
ncols = 6
fig,axs=plt.subplots(nrows=nrows,ncols=ncols,figsize=(20,10)) 
for i,gene in enumerate(gene_order):
    spl_res = UnivariateSpline(adata_df['t_scaled'], adata_df[gene], k=2, s=None)
    gene_smoothed_values = spl_res(adata_df['t_scaled'])
    norm = plt.Normalize(gene_smoothed_values.min(), gene_smoothed_values.max())
    
    # plot
    axs[i//ncols,i%ncols].scatter(x=adata_df['t_scaled'],y=adata_df[gene])
    axs[i//ncols,i%ncols].plot(adata_df['t_scaled'],gene_smoothed_values,color='red')
    axs[i//ncols,i%ncols].set_title(gene)
    axs[i//ncols,i%ncols].set_xlabel('pseudotime')
    axs[i//ncols,i%ncols].set_ylabel('smoothed gene expression')
# fig.suptitle('')
fig.tight_layout()
```

Example 52 (unknown):
```unknown
* associated genes
* associated regions
* espressed genes
* expressed regions
```

---

## Import module¶

**URL:** http://127.0.0.1:9020/notebooks_html/Supplementary_materials/Epigenetic_stability/no2a_RNA-nhoods_scVI-embedding.html

**Contents:**
- Import module¶
- Parameters¶
- Differential acundance test (Milo)¶
- Define nhoods: Young, Old, Non-significant¶

**Examples:**

Example 1 (python):
```python
import numpy as np
import pandas as pd 
import scanpy as sc
import matplotlib.pyplot as plt
import seaborn as sns
# import sctk
```

Example 2 (python):
```python
# Configure rpy2 to Use the Correct R Installation:
import os
os.environ['R_HOME'] = '/nfs/team205/kk18/miniconda3/envs/vitro/lib/R'
# check
from rpy2.rinterface_lib import openrlib
print(openrlib.R_HOME)
```

Example 3 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/R
```

Example 4 (python):
```python
import milopy.core as milo
import milopy.plot as milopl
import milopy.utils
```

Example 5 (python):
```python
import session_info
session_info.show()
```

Example 6 (python):
```python
-----
matplotlib          3.7.2
milopy              0.1.1
numpy               1.24.4
pandas              1.5.3
rpy2                3.5.14
scanpy              1.9.3
seaborn             0.12.2
session_info        1.0.0
-----
```

Example 7 (python):
```python
PIL                         10.0.0
anndata                     0.9.2
anyio                       NA
arrow                       1.2.3
asttokens                   NA
attr                        23.1.0
attrs                       23.1.0
babel                       2.12.1
backcall                    0.2.0
backports                   NA
brotli                      1.0.9
certifi                     2023.11.17
cffi                        1.15.1
charset_normalizer          3.2.0
colorama                    0.4.6
comm                        0.1.4
cycler                      0.10.0
cython_runtime              NA
dateutil                    2.8.2
debugpy                     1.6.8
decorator                   5.1.1
defusedxml                  0.7.1
executing                   1.2.0
fastjsonschema              NA
fqdn                        NA
h5py                        3.9.0
idna                        3.4
igraph                      0.10.6
importlib_metadata          NA
importlib_resources         NA
ipykernel                   6.25.1
isoduration                 NA
jedi                        0.19.0
jinja2                      3.1.2
joblib                      1.3.2
json5                       NA
jsonpointer                 2.0
jsonschema                  4.19.0
jsonschema_specifications   NA
jupyter_events              0.7.0
jupyter_server              2.7.1
jupyterlab_server           2.24.0
kiwisolver                  1.4.5
leidenalg                   0.10.1
llvmlite                    0.40.1
markupsafe                  2.1.3
mpl_toolkits                NA
natsort                     8.4.0
nbformat                    5.9.2
numba                       0.57.1
overrides                   NA
packaging                   23.1
parso                       0.8.3
patsy                       0.5.3
pexpect                     4.8.0
pickleshare                 0.7.5
pkg_resources               NA
plotly                      5.16.1
prometheus_client           NA
prompt_toolkit              3.0.39
psutil                      5.9.5
ptyprocess                  0.7.0
pure_eval                   0.2.2
pydev_ipython               NA
pydevconsole                NA
pydevd                      2.9.5
pydevd_file_utils           NA
pydevd_plugins              NA
pydevd_tracing              NA
pygments                    2.16.1
pyparsing                   3.0.9
pythonjsonlogger            NA
pytz                        2023.3
referencing                 NA
requests                    2.31.0
rfc3339_validator           0.1.4
rfc3986_validator           0.1.1
rpds                        NA
ruamel                      NA
scipy                       1.10.1
send2trash                  NA
six                         1.16.0
sklearn                     1.3.0
sniffio                     1.3.0
socks                       1.7.1
sphinxcontrib               NA
stack_data                  0.6.2
statsmodels                 0.14.0
texttable                   1.6.7
threadpoolctl               3.2.0
tornado                     6.3.3
traitlets                   5.9.0
typing_extensions           NA
tzlocal                     NA
uri_template                NA
urllib3                     2.0.4
wcwidth                     0.2.6
webcolors                   1.13
websocket                   1.6.2
yaml                        6.0.1
zipp                        NA
zmq                         25.1.1
zstandard                   0.19.0
```

Example 8 (unknown):
```unknown
-----
IPython             8.12.0
jupyter_client      8.3.0
jupyter_core        4.12.0
jupyterlab          4.0.5
notebook            7.0.2
-----
Python 3.8.17 (default, Jul  5 2023, 21:04:15) [GCC 11.2.0]
Linux-4.15.0-220-generic-x86_64-with-glibc2.17
-----
Session information updated at 2024-01-30 14:05
```

Example 9 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 10 (python):
```python
sc.settings.set_figure_params(dpi=120)
```

Example 11 (unknown):
```unknown
outdir = '/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/Milo_nhoods_gex/scVI_mid_grain_mod'
```

Example 12 (unknown):
```unknown
!ls /nfs/team205/heart/anndata_objects/Foetal/scVI/subsets
```

Example 13 (unknown):
```unknown
Foetal_multiome_AtrialCardiomyocytes_plusCCS_one-layer_added-cov-region.h5ad
Foetal_multiome_AtrialCardiomyocytes_plusCCS_one-layer_added-cov-region_subsampled-n1000.h5ad
Foetal_multiome_AtrialCardiomyocytes_plusCCS_one-layer.h5ad
Foetal_multiome_AtrialCardiomyocytes_plusCCS_three-layer.h5ad
Foetal_multiome_AtrialCardiomyocytes_plusCCS_two-layer.h5ad
Foetal_multiome_BloodVesselEndothelialCells_one-layer_added-cov-region.h5ad
Foetal_multiome_BloodVesselEndothelialCells_one-layer_added-cov-region_subsampled-n1000.h5ad
Foetal_multiome_BloodVesselEndothelialCells_one-layer.h5ad
Foetal_multiome_BloodVesselEndothelialCells_three-layer.h5ad
Foetal_multiome_BloodVesselEndothelialCells_two-layer.h5ad
Foetal_multiome_EndocardialCells_one-layer_added-cov-region.h5ad
Foetal_multiome_EndocardialCells_one-layer_added-cov-region_subsampled-n1000.h5ad
Foetal_multiome_EndocardialCells_one-layer.h5ad
Foetal_multiome_EndocardialCells_three-layer.h5ad
Foetal_multiome_EndocardialCells_two-layer.h5ad
Foetal_multiome_EpicardialCells_one-layer_added-cov-region.h5ad
Foetal_multiome_EpicardialCells_one-layer_added-cov-region_subsampled-n1000.h5ad
Foetal_multiome_EpicardialCells_one-layer.h5ad
Foetal_multiome_EpicardialCells_three-layer.h5ad
Foetal_multiome_EpicardialCells_two-layer.h5ad
Foetal_multiome_Fibroblasts_one-layer_added-cov-region.h5ad
Foetal_multiome_Fibroblasts_one-layer_added-cov-region_subsampled-n1000.h5ad
Foetal_multiome_Fibroblasts_one-layer.h5ad
Foetal_multiome_Fibroblasts_three-layer.h5ad
Foetal_multiome_Fibroblasts_two-layer.h5ad
Foetal_multiome_Glia_one-layer_added-cov-region.h5ad
Foetal_multiome_Glia_one-layer_added-cov-region_subsampled-n1000.h5ad
Foetal_multiome_Glia_one-layer.h5ad
Foetal_multiome_Glia_three-layer.h5ad
Foetal_multiome_Glia_two-layer.h5ad
Foetal_multiome_LymphaticEndothelialCells_one-layer_added-cov-region.h5ad
Foetal_multiome_LymphaticEndothelialCells_one-layer_added-cov-region_subsampled-n1000.h5ad
Foetal_multiome_LymphaticEndothelialCells_one-layer.h5ad
Foetal_multiome_LymphaticEndothelialCells_three-layer.h5ad
Foetal_multiome_LymphaticEndothelialCells_two-layer.h5ad
Foetal_multiome_LymphoidCells_one-layer_added-cov-region.h5ad
Foetal_multiome_LymphoidCells_one-layer_added-cov-region_subsampled-n1000.h5ad
Foetal_multiome_LymphoidCells_one-layer.h5ad
Foetal_multiome_LymphoidCells_three-layer.h5ad
Foetal_multiome_LymphoidCells_two-layer.h5ad
Foetal_multiome_MuralCells_one-layer_added-cov-region.h5ad
Foetal_multiome_MuralCells_one-layer_added-cov-region_subsampled-n1000.h5ad
Foetal_multiome_MuralCells_one-layer.h5ad
Foetal_multiome_MuralCells_three-layer.h5ad
Foetal_multiome_MuralCells_two-layer.h5ad
Foetal_multiome_MyeloidCells_one-layer_added-cov-region.h5ad
Foetal_multiome_MyeloidCells_one-layer_added-cov-region_subsampled-n1000.h5ad
Foetal_multiome_MyeloidCells_one-layer.h5ad
Foetal_multiome_MyeloidCells_three-layer.h5ad
Foetal_multiome_MyeloidCells_two-layer.h5ad
Foetal_multiome_Neurons_one-layer_added-cov-region.h5ad
Foetal_multiome_Neurons_one-layer_added-cov-region_subsampled-n1000.h5ad
Foetal_multiome_Neurons_one-layer.h5ad
Foetal_multiome_Neurons_three-layer.h5ad
Foetal_multiome_Neurons_two-layer.h5ad
Foetal_multiome_PericardialCells_one-layer_added-cov-region.h5ad
Foetal_multiome_PericardialCells_one-layer_added-cov-region_subsampled-n1000.h5ad
Foetal_multiome_PericardialCells_one-layer.h5ad
Foetal_multiome_PericardialCells_three-layer.h5ad
Foetal_multiome_PericardialCells_two-layer.h5ad
Foetal_multiome_VentricularCardiomyocytes_plusCCS_one-layer_added-cov-region.h5ad
Foetal_multiome_VentricularCardiomyocytes_plusCCS_one-layer_added-cov-region_subsampled-n1000.h5ad
Foetal_multiome_VentricularCardiomyocytes_plusCCS_one-layer.h5ad
Foetal_multiome_VentricularCardiomyocytes_plusCCS_three-layer.h5ad
Foetal_multiome_VentricularCardiomyocytes_plusCCS_two-layer.h5ad
```

Example 14 (python):
```python
%%time
n_layer = 'one-layer'

adata_dict = {}
for i,celltype in enumerate(['AtrialCardiomyocytes_plusCCS',
                            'VentricularCardiomyocytes_plusCCS',
                             'BloodVesselEndothelialCells',
                             'EndocardialCells',
                            'LymphaticEndothelialCells',
                             'EpicardialCells',
                             'Fibroblasts',
                             'PericardialCells',
                             'MuralCells',
                             'Neurons',
                            'Glia',
                            'MyeloidCells',
                            'LymphoidCells']
                             ):
    print(f'####### {celltype} #######')
    
    adata_dict[celltype] = sc.read_h5ad(f'/nfs/team205/heart/anndata_objects/Foetal/scVI/subsets/Foetal_multiome_{celltype}_{n_layer}_added-cov-region_subsampled-n1000.h5ad')
    print(adata_dict[celltype].shape)
    
    # compute the k-nearest-neighbor graph that is used in both clustering and umap algorithms
    sc.pp.neighbors(adata_dict[celltype], use_rep="X_scVI", n_neighbors=15, key_added="scvi")
    # compute the umap
    sc.tl.umap(adata_dict[celltype], min_dist=0.2, neighbors_key="scvi")
    # cluster the space (we use a lower resolution to get fewer clusters than the default)
    # sc.tl.leiden(adata_dict[celltype], key_added="cluster_pvi", resolution=1, neighbors_key="peakvi")
    # plot
    sc.pl.umap(adata_dict[celltype], color=['week','donor','fine_grain'], size=20, legend_fontsize=12, wspace=0.3,
              cmap='RdPu')
    
    ## Construct neighbourhoods
    milo.make_nhoods(adata_dict[celltype], neighbors_key="scvi", prop=0.05)
    ## Count cells in neighbourhoods
    ## Milo leverages the variation in cell numbers between replicates for the same experimental condition to test for differential abundance. 
    milo.count_nhoods(adata_dict[celltype], sample_col="donor")
    # differential abundance testing
    adata_dict[celltype].obs["week_continuous"] = [int(x.split('W')[0]) for x in adata_dict[celltype].obs["week"]] #rather than adata.obs["week"].cat.codes
    milo.DA_nhoods(adata_dict[celltype], design="~ week_continuous")
    milopy.utils.build_nhood_graph(adata_dict[celltype],basis='X_umap')
    # plot
    milopl.plot_nhood_graph(adata_dict[celltype], 
                        alpha=0.1, ## SpatialFDR level (1%) 
                        min_size=3, ## Size of smallest dot
                        title = 'DA log-Fold Change\n(FDR < 0.1)'
                       )
    # store logFC
    logfcs = abs(adata_dict[celltype].uns['nhood_adata'].obs['logFC'])
    df = pd.DataFrame({'Celltype':celltype,'logFC':logfcs})
    if i==0:
        logfc_df=df.copy()
    else:
        logfc_df=pd.concat([logfc_df,df])

# save adata dictionary
utils.save_pkl(adata_dict, f'{outdir}/{n_layer}_added-cov-region_subsampled-n1000_post_milo_adata_dictionary.pkl')
```

Example 15 (unknown):
```unknown
####### AtrialCardiomyocytes_plusCCS #######
(4381, 24444)
```

Example 16 (python):
```python
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/umap/distances.py:1063: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/umap/distances.py:1071: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/umap/distances.py:1086: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/umap/umap_.py:660: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 17 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/anndata/_core/anndata.py:121: ImplicitModificationWarning: Transforming to str index.
  warnings.warn("Transforming to str index.", ImplicitModificationWarning)
```

Example 18 (unknown):
```unknown
####### VentricularCardiomyocytes_plusCCS #######
(6531, 24426)
```

Example 19 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 20 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/anndata/_core/anndata.py:121: ImplicitModificationWarning: Transforming to str index.
  warnings.warn("Transforming to str index.", ImplicitModificationWarning)
```

Example 21 (unknown):
```unknown
####### BloodVesselEndothelialCells #######
(3028, 24028)
```

Example 22 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 23 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/anndata/_core/anndata.py:121: ImplicitModificationWarning: Transforming to str index.
  warnings.warn("Transforming to str index.", ImplicitModificationWarning)
```

Example 24 (unknown):
```unknown
####### EndocardialCells #######
(1410, 21154)
```

Example 25 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 26 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/anndata/_core/anndata.py:121: ImplicitModificationWarning: Transforming to str index.
  warnings.warn("Transforming to str index.", ImplicitModificationWarning)
```

Example 27 (unknown):
```unknown
####### LymphaticEndothelialCells #######
(1000, 20680)
```

Example 28 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 29 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/anndata/_core/anndata.py:121: ImplicitModificationWarning: Transforming to str index.
  warnings.warn("Transforming to str index.", ImplicitModificationWarning)
```

Example 30 (unknown):
```unknown
####### EpicardialCells #######
(1340, 23572)
```

Example 31 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 32 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/anndata/_core/anndata.py:121: ImplicitModificationWarning: Transforming to str index.
  warnings.warn("Transforming to str index.", ImplicitModificationWarning)
```

Example 33 (unknown):
```unknown
####### Fibroblasts #######
(6969, 26468)
```

Example 34 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 35 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/anndata/_core/anndata.py:121: ImplicitModificationWarning: Transforming to str index.
  warnings.warn("Transforming to str index.", ImplicitModificationWarning)
```

Example 36 (unknown):
```unknown
####### PericardialCells #######
(2187, 22516)
```

Example 37 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 38 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/anndata/_core/anndata.py:121: ImplicitModificationWarning: Transforming to str index.
  warnings.warn("Transforming to str index.", ImplicitModificationWarning)
```

Example 39 (unknown):
```unknown
####### MuralCells #######
(3103, 23056)
```

Example 40 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 41 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/anndata/_core/anndata.py:121: ImplicitModificationWarning: Transforming to str index.
  warnings.warn("Transforming to str index.", ImplicitModificationWarning)
```

Example 42 (unknown):
```unknown
####### Neurons #######
(1101, 21561)
```

Example 43 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 44 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/anndata/_core/anndata.py:121: ImplicitModificationWarning: Transforming to str index.
  warnings.warn("Transforming to str index.", ImplicitModificationWarning)
```

Example 45 (unknown):
```unknown
####### Glia #######
(1759, 21535)
```

Example 46 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 47 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/anndata/_core/anndata.py:121: ImplicitModificationWarning: Transforming to str index.
  warnings.warn("Transforming to str index.", ImplicitModificationWarning)
```

Example 48 (unknown):
```unknown
####### MyeloidCells #######
(4321, 24290)
```

Example 49 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 50 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/anndata/_core/anndata.py:121: ImplicitModificationWarning: Transforming to str index.
  warnings.warn("Transforming to str index.", ImplicitModificationWarning)
```

Example 51 (unknown):
```unknown
####### LymphoidCells #######
(7006, 24441)
```

Example 52 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 53 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/anndata/_core/anndata.py:121: ImplicitModificationWarning: Transforming to str index.
  warnings.warn("Transforming to str index.", ImplicitModificationWarning)
```

Example 54 (unknown):
```unknown
CPU times: user 2min 28s, sys: 4.19 s, total: 2min 32s
Wall time: 3min 3s
```

Example 55 (python):
```python
adata_dict.keys()
```

Example 56 (unknown):
```unknown
dict_keys(['AtrialCardiomyocytes_plusCCS', 'VentricularCardiomyocytes_plusCCS', 'BloodVesselEndothelialCells', 'EndocardialCells', 'LymphaticEndothelialCells', 'EpicardialCells', 'Fibroblasts', 'PericardialCells', 'MuralCells', 'Neurons', 'Glia', 'MyeloidCells', 'LymphoidCells'])
```

Example 57 (python):
```python
for celltype in adata_dict.keys():
    print(celltype)
    # get adata
    ad = adata_dict[celltype]
    # define nhoods based on the enrichment
    nhoods_res = ad.uns['nhood_adata'].obs.copy()
    nhoods_res['nhood'] = [f'nhood_{x+1}' for x in range(len(nhoods_res))]
    nhoods_res.set_index('nhood',inplace=True)
    nhoods_res['enrichment'] = 'nonsignificant'
    old_inds = nhoods_res.index[(nhoods_res['logFC']>0)&(nhoods_res['FDR']<0.1)]
    nhoods_res.loc[old_inds,'enrichment'] = 'old'
    young_inds = nhoods_res.index[(nhoods_res['logFC']<0)&(nhoods_res['FDR']<0.1)]
    nhoods_res.loc[young_inds,'enrichment'] = 'young'
    print(f'old nhoods: {len(old_inds)}')
    print(f'young nhoods: {len(young_inds)}')
    # prepare cell-by-nhood dataframe and save
    cell_by_nhood = pd.DataFrame(ad.obsm['nhoods'].todense(), columns=nhoods_res.index, index=ad.obs_names)
    # save cell-by-nhood dataframe
    cell_by_nhood.to_csv(f'{outdir}/{n_layer}_added-cov-region_subsampled-n1000_{celltype}_milo_cell-by-nhood.csv')
    
    ### calculate variance of a nhood-cells in scVI latent space ###
    for nhood in cell_by_nhood.columns:
        # get cells of the nhood
        cells = list(cell_by_nhood.index[cell_by_nhood[nhood]==1])
        nhoods_res.loc[nhood,'cells'] = ','.join(cells)
        # get scVI latent space
        matrix = ad[cells].obsm['X_scVI']
        matrix = np.array(matrix)
        # calculate variance of neighboring cells per axis
        variance_per_axis = np.var(matrix, axis=0)
        # aggregate by taking mean
        variance = np.mean(variance_per_axis)
        # store in nhood result dataframe
        nhoods_res.loc[nhood,'variance_scVI-latent'] = variance
    
    # save nhood result dataframe
    nhoods_res.to_csv(f'{outdir}/{n_layer}_added-cov-region_subsampled-n1000_{celltype}_milo_nhoods_result.csv')
```

Example 58 (unknown):
```unknown
AtrialCardiomyocytes_plusCCS
old nhoods: 88
young nhoods: 14
VentricularCardiomyocytes_plusCCS
old nhoods: 79
young nhoods: 11
BloodVesselEndothelialCells
old nhoods: 33
young nhoods: 2
EndocardialCells
old nhoods: 13
young nhoods: 8
LymphaticEndothelialCells
old nhoods: 19
young nhoods: 0
EpicardialCells
old nhoods: 46
young nhoods: 11
Fibroblasts
old nhoods: 155
young nhoods: 26
PericardialCells
old nhoods: 42
young nhoods: 0
MuralCells
old nhoods: 73
young nhoods: 2
Neurons
old nhoods: 2
young nhoods: 16
Glia
old nhoods: 22
young nhoods: 24
MyeloidCells
old nhoods: 79
young nhoods: 0
LymphoidCells
old nhoods: 247
young nhoods: 0
```

---

## Read in ArchR project¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/ATAC/ATAC_preprocess/ArchR_no4_addGEX.html

**Contents:**
- Read in ArchR project¶
- AnnData to Seurat object¶
- Prepare RNA data¶
  - Prepare SingleCellExperiment¶
  - Add rowRanges data¶
  - Modify cellnames etc¶
- Add RNA data to ArchR object¶
- Save¶

What is rowRanges data? ref https://www.bioconductor.org/packages/devel/bioc/vignettes/SummarizedExperiment/inst/doc/SummarizedExperiment.html https://robertamezquita.github.io/orchestratingSingleCellAnalysis/data-infrastructure.html#the-essentials-of-sce

**Examples:**

Example 1 (unknown):
```unknown
library(ArchR)
library(tidyverse)
library(BSgenome.Hsapiens.UCSC.hg38)
library(SingleCellExperiment)
library(anndata)
# library(sceasy)
# library(reticulate)
# use_condaenv('cellpymc')
# loompy <- reticulate::import('loompy')
```

Example 2 (python):
```python
/ |
                                                 /    \
            .                                  /      |.
            \\\                              /        |.
              \\\                          /           `|.
                \\\                      /              |.
                  \                    /                |\
                  \\#####\           /                  ||
                ==###########>      /                   ||
                 \\##==......\    /                     ||
            ______ =       =|__ /__                     ||      \\\
        ,--' ,----`-,__ ___/'  --,-`-===================##========>
       \               '        ##_______ _____ ,--,__,=##,__   ///
        ,    __==    ___,-,__,--'#'  ==='      `-'    | ##,-/
        -,____,---'       \\####\\________________,--\\_##,/
           ___      .______        ______  __    __  .______      
          /   \     |   _  \      /      ||  |  |  | |   _  \     
         /  ^  \    |  |_)  |    |  ,----'|  |__|  | |  |_)  |    
        /  /_\  \   |      /     |  |     |   __   | |      /     
       /  _____  \  |  |\  \\___ |  `----.|  |  |  | |  |\  \\___.
      /__/     \__\ | _| `._____| \______||__|  |__| | _| `._____|
    

ArchR : Version 1.0.2
For more information see our website : www.ArchRProject.com
If you encounter a bug please report : https://github.com/GreenleafLab/ArchR/issues

Loading Required Packages...

	Loading Package : grid v4.0.4

	Loading Package : gridExtra v2.3

	Loading Package : gtools v3.9.3

	Loading Package : gtable v0.3.0

	Loading Package : ggplot2 v3.4.0

	Loading Package : magrittr v2.0.3

	Loading Package : plyr v1.8.8

	Loading Package : stringr v1.4.0

	Loading Package : data.table v1.14.4

	Loading Package : matrixStats v0.62.0

	Loading Package : S4Vectors v0.28.1

	Loading Package : GenomicRanges v1.42.0

	Loading Package : BiocGenerics v0.36.1

	Loading Package : Matrix v1.3.2

	Loading Package : Rcpp v1.0.9

	Loading Package : SummarizedExperiment v1.20.0

	Loading Package : rhdf5 v2.34.0

Warning message in system("timedatectl", intern = TRUE):
“running command 'timedatectl' had status 1”
── Attaching packages ─────────────────────────────────────── tidyverse 1.3.2 ──
✔ tibble  3.1.8      ✔ purrr   0.3.4 
✔ tidyr   1.2.1      ✔ dplyr   1.0.10
✔ readr   2.1.3      ✔ forcats 0.5.1 
── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::arrange()         masks plyr::arrange()
✖ dplyr::between()         masks data.table::between()
✖ dplyr::collapse()        masks IRanges::collapse()
✖ dplyr::combine()         masks Biobase::combine(), BiocGenerics::combine(), gridExtra::combine()
✖ purrr::compact()         masks plyr::compact()
✖ dplyr::count()           masks matrixStats::count(), plyr::count()
✖ dplyr::desc()            masks IRanges::desc(), plyr::desc()
✖ tidyr::expand()          masks Matrix::expand(), S4Vectors::expand()
✖ tidyr::extract()         masks magrittr::extract()
✖ dplyr::failwith()        masks plyr::failwith()
✖ dplyr::filter()          masks stats::filter()
✖ dplyr::first()           masks S4Vectors::first(), data.table::first()
✖ dplyr::id()              masks plyr::id()
✖ dplyr::lag()             masks stats::lag()
✖ dplyr::last()            masks data.table::last()
✖ dplyr::mutate()          masks plyr::mutate()
✖ tidyr::pack()            masks Matrix::pack()
✖ BiocGenerics::Position() masks ggplot2::Position(), base::Position()
✖ purrr::reduce()          masks GenomicRanges::reduce(), IRanges::reduce()
✖ dplyr::rename()          masks S4Vectors::rename(), plyr::rename()
✖ purrr::set_names()       masks magrittr::set_names()
✖ dplyr::slice()           masks IRanges::slice()
✖ dplyr::summarise()       masks plyr::summarise()
✖ dplyr::summarize()       masks plyr::summarize()
✖ purrr::transpose()       masks data.table::transpose()
✖ tidyr::unpack()          masks Matrix::unpack()
Loading required package: BSgenome

Loading required package: Biostrings

Loading required package: XVector


Attaching package: ‘XVector’


The following object is masked from ‘package:purrr’:

    compact


The following object is masked from ‘package:plyr’:

    compact



Attaching package: ‘Biostrings’


The following object is masked from ‘package:base’:

    strsplit


Loading required package: rtracklayer


Attaching package: ‘anndata’


The following object is masked from ‘package:readr’:

    read_csv
```

Example 3 (unknown):
```unknown
# before starting a project we must set the ArchRGenome and default threads for parallelization.
# Setting default genome to Hg38.
addArchRGenome("hg38")
```

Example 4 (unknown):
```unknown
Setting default genome to Hg38.
```

Example 5 (unknown):
```unknown
# Setting default number of Parallel threads to 16
addArchRThreads(threads = 1)
```

Example 6 (unknown):
```unknown
Setting default number of Parallel threads to 1.
```

Example 7 (unknown):
```unknown
archr_project_path = '/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output'
proj = loadArchRProject(path = archr_project_path, showLogo = FALSE)
proj
```

Example 8 (unknown):
```unknown
Successfully loaded ArchRProject!


           ___      .______        ______  __    __  .______      
          /   \     |   _  \      /      ||  |  |  | |   _  \     
         /  ^  \    |  |_)  |    |  ,----'|  |__|  | |  |_)  |    
        /  /_\  \   |      /     |  |     |   __   | |      /     
       /  _____  \  |  |\  \\___ |  `----.|  |  |  | |  |\  \\___.
      /__/     \__\ | _| `._____| \______||__|  |__| | _| `._____|
```

Example 9 (unknown):
```unknown
class: ArchRProject 
outputDirectory: /nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output 
samples(37): BHF_F_Hea11064670_BHF_F_Hea11031823
  BHF_F_Hea11064671_BHF_F_Hea11031824 ...
  HCAHeartST13386009_HCAHeartST13303419
  HCAHeartST13386010_HCAHeartST13303420
sampleColData names(1): ArrowFiles
cellColData names(59): Sample TSSEnrichment ... ReadsInPeaks FRIP
numberOfCells(1): 167022
medianTSS(1): 11.614
medianFrags(1): 10338.5
```

Example 10 (unknown):
```unknown
# convert anndata to seurat object
if(0){
    sceasy::convertFormat('/nfs/team205/heart/anndata_objects/Foetal/Feb28ObjectRaw_Multiome.h5ad', 
                      from="anndata", to="seurat",
                       outFile='/nfs/team205/heart/anndata_objects/Foetal/Feb28ObjectRaw_Multiome.rds')
}
```

Example 11 (unknown):
```unknown
X -> counts
```

Example 12 (python):
```python
An object of class Seurat 
36601 features across 211145 samples within 1 assay 
Active assay: RNA (36601 features, 0 variable features)
```

Example 13 (unknown):
```unknown
# read in seurat object
rna = readRDS('/nfs/team205/heart/anndata_objects/Foetal/Feb28ObjectRaw_Multiome.rds')
rna
```

Example 14 (python):
```python
Loading required package: SeuratObject


Attaching package: ‘SeuratObject’


The following object is masked from ‘package:SummarizedExperiment’:

    Assays
```

Example 15 (python):
```python
An object of class Seurat 
36601 features across 211145 samples within 1 assay 
Active assay: RNA (36601 features, 0 variable features)
```

Example 16 (unknown):
```unknown
# read in CellRanger h5 data for rowRanges data
rna_forRowRanges <- import10xFeatureMatrix(
    input = c("/nfs/team205/heart/mapped/cellranger-arc200/BHF_F_Hea13168898_BHF_F_Hea13168514/filtered_feature_bc_matrix.h5"),
    names = c("BHF_F_Hea13168898_BHF_F_Hea13168514")
   )
```

Example 17 (unknown):
```unknown
Importing Feature Matrix 1 of 1

Re-ordering RNA matricies for consistency.
```

Example 18 (unknown):
```unknown
# shared genes
shared_genes = intersect(rownames(rna_forRowRanges),rownames(rna))

# subset shared genes
rna = rna[shared_genes,]
rna_forRowRanges = rna_forRowRanges[shared_genes,]

# convert to SingleCellExperiment
rna = Seurat::as.SingleCellExperiment(rna)
rna
```

Example 19 (python):
```python
class: SingleCellExperiment 
dim: 36578 211145 
metadata(0):
assays(2): counts logcounts
rownames(36578): MIR1302-2HG FAM138A ... AC007325.4 AC007325.2
rowData names(1): gene_id
colnames(211145):
  BHF_F_Hea11064670_BHF_F_Hea11031823_TTGCTTAGTGAGACTC-1
  BHF_F_Hea11064670_BHF_F_Hea11031823_ATAGATGCATTGTCCT-1 ...
  HCAHeartST13386010_HCAHeartST13303420_CGTAATGGTATTGAGT-1
  HCAHeartST13386010_HCAHeartST13303420_GCTGTAAGTCAATACG-1
colData names(50): sangerID combinedID ... nFeature_RNA ident
reducedDimNames(0):
altExpNames(0):
```

Example 20 (unknown):
```unknown
rowRanges(rna_forRowRanges)
```

Example 21 (python):
```python
GRanges object with 36578 ranges and 5 metadata columns:
                seqnames          ranges strand |    feature_type genome
                   <Rle>       <IRanges>  <Rle> |           <Rle>  <Rle>
  MIR1302-2HG       chr1     29553-30267      * | Gene Expression GRCh38
      FAM138A       chr1     36080-36081      * | Gene Expression GRCh38
        OR4F5       chr1     65418-69055      * | Gene Expression GRCh38
   AL627309.3       chr1     91104-91105      * | Gene Expression GRCh38
   AL627309.1       chr1   120931-133723      * | Gene Expression GRCh38
          ...        ...             ...    ... .             ...    ...
   AC141272.1 KI270728.1 1270983-1270984      * | Gene Expression GRCh38
   AC023491.2 KI270731.1     13000-13001      * | Gene Expression GRCh38
   AC007325.1 KI270734.1     72410-72411      * | Gene Expression GRCh38
   AC007325.4 KI270734.1   131493-131494      * | Gene Expression GRCh38
   AC007325.2 KI270734.1   161749-161852      * | Gene Expression GRCh38
                           id               interval        name
                      <array>                <array>     <array>
  MIR1302-2HG ENSG00000243485       chr1:29553-30267 MIR1302-2HG
      FAM138A ENSG00000237613       chr1:36080-36081     FAM138A
        OR4F5 ENSG00000186092       chr1:65418-69055       OR4F5
   AL627309.3 ENSG00000239945       chr1:91104-91105  AL627309.3
   AL627309.1 ENSG00000238009     chr1:120931-133723  AL627309.1
          ...             ...                    ...         ...
   AC141272.1 ENSG00000277836 KI270728.1:1270983-1..  AC141272.1
   AC023491.2 ENSG00000278633 KI270731.1:13000-13001  AC023491.2
   AC007325.1 ENSG00000276017 KI270734.1:72410-72411  AC007325.1
   AC007325.4 ENSG00000278817 KI270734.1:131493-13..  AC007325.4
   AC007325.2 ENSG00000277196 KI270734.1:161749-16..  AC007325.2
  -------
  seqinfo: 39 sequences from an unspecified genome; no seqlengths
```

Example 22 (unknown):
```unknown
# add rowRanges data
rowRanges(rna) <- rowRanges(rna_forRowRanges)
```

Example 23 (javascript):
```javascript
# add barcode column
colData(rna)$cellbarcode = strsplit(colnames(rna), split = paste0(colData(rna)$combinedID,"_")) %>% lapply(function(x){x[2]}) %>% unlist()

# rename cell names to be matched with ArchR object
colnames(rna) = paste0(colData(rna)$combinedID,'#',colData(rna)$cellbarcode)
colnames(rna)[1:10]
```

Example 24 (unknown):
```unknown
# remove logcounts
assays(rna)['logcounts'] = NULL
rna
```

Example 25 (python):
```python
class: SingleCellExperiment 
dim: 36578 211145 
metadata(0):
assays(1): counts
rownames(36578): MIR1302-2HG FAM138A ... AC007325.4 AC007325.2
rowData names(5): feature_type genome id interval name
colnames(211145):
  BHF_F_Hea11064670_BHF_F_Hea11031823#TTGCTTAGTGAGACTC-1
  BHF_F_Hea11064670_BHF_F_Hea11031823#ATAGATGCATTGTCCT-1 ...
  HCAHeartST13386010_HCAHeartST13303420#CGTAATGGTATTGAGT-1
  HCAHeartST13386010_HCAHeartST13303420#GCTGTAAGTCAATACG-1
colData names(51): sangerID combinedID ... ident cellbarcode
reducedDimNames(0):
altExpNames(0):
```

Example 26 (unknown):
```unknown
head(proj$cellNames)
```

Example 27 (unknown):
```unknown
length(intersect(proj$cellNames,colnames(rna)))
```

Example 28 (unknown):
```unknown
# subset shared cells
# shared_cells = intersect(proj$cellNames,colnames(rna))
# proj_shared = proj[shared_cells, ]
```

Example 29 (unknown):
```unknown
proj <- addGeneExpressionMatrix(input = proj, seRNA = rna, force = TRUE)
proj
```

Example 30 (unknown):
```unknown
ArchR logging to : ArchRLogs/ArchR-addGeneExpressionMatrix-49213027a1f7-Date-2023-03-09_Time-22-44-06.log
If there is an issue, please report to github with logFile!

Overlap w/ scATAC = 1

2023-03-09 22:44:10 : 

Overlap Per Sample w/ scATAC : 7089STDY13216920_BHF_F_Hea13242527=10630,7089STDY13216921_BHF_F_Hea13242528=9214,7089STDY13216922_BHF_F_Hea13242529=3958,7089STDY13216923_BHF_F_Hea13242530=1,7089STDY13216924_BHF_F_Hea13242531=3783,7089STDY13216925_BHF_F_Hea13242532=2197,7089STDY13216926_BHF_F_Hea13242533=3622,7089STDY13216927_BHF_F_Hea13242534=4909,BHF_F_Hea11064670_BHF_F_Hea11031823=350,BHF_F_Hea11064671_BHF_F_Hea11031824=2218,BHF_F_Hea11064672_BHF_F_Hea11031825=3537,BHF_F_Hea11933666_BHF_F_Hea11596619=7905,BHF_F_Hea11933667_BHF_F_Hea11596620=8692,BHF_F_Hea11933668_BHF_F_Hea11596621=8194,BHF_F_Hea11933669_BHF_F_Hea11596622=7959,BHF_F_Hea11933670_BHF_F_Hea11596623=2824,BHF_F_Hea11933671_BHF_F_Hea11596624=3092,BHF_F_Hea11933672_BHF_F_Hea11596625=3143,BHF_F_Hea11933673_BHF_F_Hea11596626=3409,BHF_F_Hea11933674_BHF_F_Hea11596627=2239,BHF_F_Hea11933675_BHF_F_Hea11596628=191,BHF_F_Hea13168892_BHF_F_Hea13168508=3736,BHF_F_Hea13168893_BHF_F_Hea13168509=4405,BHF_F_Hea13168894_BHF_F_Hea13168510=5944,BHF_F_Hea13168895_BHF_F_Hea13168511=5525,BHF_F_Hea13168896_BHF_F_Hea13168512=8097,BHF_F_Hea13168897_BHF_F_Hea13168513=6693,BHF_F_Hea13168898_BHF_F_Hea13168514=8593,BHF_F_Hea13168899_BHF_F_Hea13168515=7575,BHF_F_Hea13188317_BHF_F_Hea13187619=2831,BHF_F_Hea13188318_BHF_F_Hea13187620=5195,BHF_F_Hea13188324_BHF_F_Hea13187626=5123,BHF_F_Hea13188328_BHF_F_Hea13187630=817,BHF_F_Hea13188329_BHF_F_Hea13187631=691,BHF_F_Hea13188330_BHF_F_Hea13187632=560,HCAHeartST13386009_HCAHeartST13303419=3882,HCAHeartST13386010_HCAHeartST13303420=5288

2023-03-09 22:44:11 : 

2023-03-09 22:44:52 : Batch Execution w/ safelapply!, 0 mins elapsed.

2023-03-09 22:44:55 : Adding 7089STDY13216921_BHF_F_Hea13242528 to GeneExpressionMatrix for Chr (1 of 23)!, 0.047 mins elapsed.

2023-03-09 22:44:59 : Adding 7089STDY13216921_BHF_F_Hea13242528 to GeneExpressionMatrix for Chr (2 of 23)!, 0.117 mins elapsed.

2023-03-09 22:45:02 : Adding 7089STDY13216921_BHF_F_Hea13242528 to GeneExpressionMatrix for Chr (3 of 23)!, 0.172 mins elapsed.

2023-03-09 22:45:07 : Adding 7089STDY13216921_BHF_F_Hea13242528 to GeneExpressionMatrix for Chr (4 of 23)!, 0.262 mins elapsed.

2023-03-09 22:45:11 : Adding 7089STDY13216921_BHF_F_Hea13242528 to GeneExpressionMatrix for Chr (5 of 23)!, 0.314 mins elapsed.

2023-03-09 22:45:14 : Adding 7089STDY13216921_BHF_F_Hea13242528 to GeneExpressionMatrix for Chr (6 of 23)!, 0.365 mins elapsed.

2023-03-09 22:45:18 : Adding 7089STDY13216921_BHF_F_Hea13242528 to GeneExpressionMatrix for Chr (7 of 23)!, 0.439 mins elapsed.

2023-03-09 22:45:22 : Adding 7089STDY13216921_BHF_F_Hea13242528 to GeneExpressionMatrix for Chr (8 of 23)!, 0.502 mins elapsed.

2023-03-09 22:45:26 : Adding 7089STDY13216921_BHF_F_Hea13242528 to GeneExpressionMatrix for Chr (9 of 23)!, 0.564 mins elapsed.

2023-03-09 22:45:30 : Adding 7089STDY13216921_BHF_F_Hea13242528 to GeneExpressionMatrix for Chr (10 of 23)!, 0.638 mins elapsed.

2023-03-09 22:45:33 : Adding 7089STDY13216921_BHF_F_Hea13242528 to GeneExpressionMatrix for Chr (11 of 23)!, 0.695 mins elapsed.

2023-03-09 22:45:37 : Adding 7089STDY13216921_BHF_F_Hea13242528 to GeneExpressionMatrix for Chr (12 of 23)!, 0.748 mins elapsed.

2023-03-09 22:45:41 : Adding 7089STDY13216921_BHF_F_Hea13242528 to GeneExpressionMatrix for Chr (13 of 23)!, 0.83 mins elapsed.

2023-03-09 22:45:45 : Adding 7089STDY13216921_BHF_F_Hea13242528 to GeneExpressionMatrix for Chr (14 of 23)!, 0.882 mins elapsed.

2023-03-09 22:45:48 : Adding 7089STDY13216921_BHF_F_Hea13242528 to GeneExpressionMatrix for Chr (15 of 23)!, 0.938 mins elapsed.

2023-03-09 22:45:53 : Adding 7089STDY13216921_BHF_F_Hea13242528 to GeneExpressionMatrix for Chr (16 of 23)!, 1.026 mins elapsed.

2023-03-09 22:45:57 : Adding 7089STDY13216921_BHF_F_Hea13242528 to GeneExpressionMatrix for Chr (17 of 23)!, 1.081 mins elapsed.

2023-03-09 22:46:00 : Adding 7089STDY13216921_BHF_F_Hea13242528 to GeneExpressionMatrix for Chr (18 of 23)!, 1.137 mins elapsed.

2023-03-09 22:46:05 : Adding 7089STDY13216921_BHF_F_Hea13242528 to GeneExpressionMatrix for Chr (19 of 23)!, 1.223 mins elapsed.

2023-03-09 22:46:08 : Adding 7089STDY13216921_BHF_F_Hea13242528 to GeneExpressionMatrix for Chr (20 of 23)!, 1.275 mins elapsed.

2023-03-09 22:46:11 : Adding 7089STDY13216921_BHF_F_Hea13242528 to GeneExpressionMatrix for Chr (21 of 23)!, 1.325 mins elapsed.

2023-03-09 22:46:15 : Adding 7089STDY13216921_BHF_F_Hea13242528 to GeneExpressionMatrix for Chr (22 of 23)!, 1.391 mins elapsed.

2023-03-09 22:46:18 : Adding 7089STDY13216921_BHF_F_Hea13242528 to GeneExpressionMatrix for Chr (23 of 23)!, 1.441 mins elapsed.

2023-03-09 22:46:25 : Adding 7089STDY13216920_BHF_F_Hea13242527 to GeneExpressionMatrix for Chr (1 of 23)!, 1.562 mins elapsed.

2023-03-09 22:46:29 : Adding 7089STDY13216920_BHF_F_Hea13242527 to GeneExpressionMatrix for Chr (2 of 23)!, 1.623 mins elapsed.

2023-03-09 22:46:33 : Adding 7089STDY13216920_BHF_F_Hea13242527 to GeneExpressionMatrix for Chr (3 of 23)!, 1.683 mins elapsed.

2023-03-09 22:46:37 : Adding 7089STDY13216920_BHF_F_Hea13242527 to GeneExpressionMatrix for Chr (4 of 23)!, 1.761 mins elapsed.

2023-03-09 22:46:41 : Adding 7089STDY13216920_BHF_F_Hea13242527 to GeneExpressionMatrix for Chr (5 of 23)!, 1.815 mins elapsed.

2023-03-09 22:46:44 : Adding 7089STDY13216920_BHF_F_Hea13242527 to GeneExpressionMatrix for Chr (6 of 23)!, 1.868 mins elapsed.

2023-03-09 22:46:48 : Adding 7089STDY13216920_BHF_F_Hea13242527 to GeneExpressionMatrix for Chr (7 of 23)!, 1.941 mins elapsed.

2023-03-09 22:46:52 : Adding 7089STDY13216920_BHF_F_Hea13242527 to GeneExpressionMatrix for Chr (8 of 23)!, 2.004 mins elapsed.

2023-03-09 22:46:55 : Adding 7089STDY13216920_BHF_F_Hea13242527 to GeneExpressionMatrix for Chr (9 of 23)!, 2.058 mins elapsed.

2023-03-09 22:47:00 : Adding 7089STDY13216920_BHF_F_Hea13242527 to GeneExpressionMatrix for Chr (10 of 23)!, 2.131 mins elapsed.

2023-03-09 22:47:03 : Adding 7089STDY13216920_BHF_F_Hea13242527 to GeneExpressionMatrix for Chr (11 of 23)!, 2.183 mins elapsed.

2023-03-09 22:47:06 : Adding 7089STDY13216920_BHF_F_Hea13242527 to GeneExpressionMatrix for Chr (12 of 23)!, 2.236 mins elapsed.

2023-03-09 22:47:10 : Adding 7089STDY13216920_BHF_F_Hea13242527 to GeneExpressionMatrix for Chr (13 of 23)!, 2.31 mins elapsed.

2023-03-09 22:47:13 : Adding 7089STDY13216920_BHF_F_Hea13242527 to GeneExpressionMatrix for Chr (14 of 23)!, 2.36 mins elapsed.

2023-03-09 22:47:17 : Adding 7089STDY13216920_BHF_F_Hea13242527 to GeneExpressionMatrix for Chr (15 of 23)!, 2.419 mins elapsed.

2023-03-09 22:47:22 : Adding 7089STDY13216920_BHF_F_Hea13242527 to GeneExpressionMatrix for Chr (16 of 23)!, 2.5 mins elapsed.

2023-03-09 22:47:25 : Adding 7089STDY13216920_BHF_F_Hea13242527 to GeneExpressionMatrix for Chr (17 of 23)!, 2.555 mins elapsed.

2023-03-09 22:47:28 : Adding 7089STDY13216920_BHF_F_Hea13242527 to GeneExpressionMatrix for Chr (18 of 23)!, 2.609 mins elapsed.

2023-03-09 22:47:32 : Adding 7089STDY13216920_BHF_F_Hea13242527 to GeneExpressionMatrix for Chr (19 of 23)!, 2.678 mins elapsed.

2023-03-09 22:47:35 : Adding 7089STDY13216920_BHF_F_Hea13242527 to GeneExpressionMatrix for Chr (20 of 23)!, 2.73 mins elapsed.

2023-03-09 22:47:39 : Adding 7089STDY13216920_BHF_F_Hea13242527 to GeneExpressionMatrix for Chr (21 of 23)!, 2.785 mins elapsed.

2023-03-09 22:47:44 : Adding 7089STDY13216920_BHF_F_Hea13242527 to GeneExpressionMatrix for Chr (22 of 23)!, 2.87 mins elapsed.

2023-03-09 22:47:47 : Adding 7089STDY13216920_BHF_F_Hea13242527 to GeneExpressionMatrix for Chr (23 of 23)!, 2.93 mins elapsed.

2023-03-09 22:47:55 : Adding BHF_F_Hea13188318_BHF_F_Hea13187620 to GeneExpressionMatrix for Chr (1 of 23)!, 3.05 mins elapsed.

2023-03-09 22:47:58 : Adding BHF_F_Hea13188318_BHF_F_Hea13187620 to GeneExpressionMatrix for Chr (2 of 23)!, 3.103 mins elapsed.

2023-03-09 22:48:01 : Adding BHF_F_Hea13188318_BHF_F_Hea13187620 to GeneExpressionMatrix for Chr (3 of 23)!, 3.155 mins elapsed.

2023-03-09 22:48:05 : Adding BHF_F_Hea13188318_BHF_F_Hea13187620 to GeneExpressionMatrix for Chr (4 of 23)!, 3.228 mins elapsed.

2023-03-09 22:48:08 : Adding BHF_F_Hea13188318_BHF_F_Hea13187620 to GeneExpressionMatrix for Chr (5 of 23)!, 3.278 mins elapsed.

2023-03-09 22:48:11 : Adding BHF_F_Hea13188318_BHF_F_Hea13187620 to GeneExpressionMatrix for Chr (6 of 23)!, 3.33 mins elapsed.

2023-03-09 22:48:16 : Adding BHF_F_Hea13188318_BHF_F_Hea13187620 to GeneExpressionMatrix for Chr (7 of 23)!, 3.398 mins elapsed.

2023-03-09 22:48:19 : Adding BHF_F_Hea13188318_BHF_F_Hea13187620 to GeneExpressionMatrix for Chr (8 of 23)!, 3.448 mins elapsed.

2023-03-09 22:48:21 : Adding BHF_F_Hea13188318_BHF_F_Hea13187620 to GeneExpressionMatrix for Chr (9 of 23)!, 3.497 mins elapsed.

2023-03-09 22:48:26 : Adding BHF_F_Hea13188318_BHF_F_Hea13187620 to GeneExpressionMatrix for Chr (10 of 23)!, 3.566 mins elapsed.

2023-03-09 22:48:29 : Adding BHF_F_Hea13188318_BHF_F_Hea13187620 to GeneExpressionMatrix for Chr (11 of 23)!, 3.616 mins elapsed.

2023-03-09 22:48:32 : Adding BHF_F_Hea13188318_BHF_F_Hea13187620 to GeneExpressionMatrix for Chr (12 of 23)!, 3.666 mins elapsed.

2023-03-09 22:48:36 : Adding BHF_F_Hea13188318_BHF_F_Hea13187620 to GeneExpressionMatrix for Chr (13 of 23)!, 3.734 mins elapsed.

2023-03-09 22:48:39 : Adding BHF_F_Hea13188318_BHF_F_Hea13187620 to GeneExpressionMatrix for Chr (14 of 23)!, 3.794 mins elapsed.

2023-03-09 22:48:42 : Adding BHF_F_Hea13188318_BHF_F_Hea13187620 to GeneExpressionMatrix for Chr (15 of 23)!, 3.844 mins elapsed.

2023-03-09 22:48:46 : Adding BHF_F_Hea13188318_BHF_F_Hea13187620 to GeneExpressionMatrix for Chr (16 of 23)!, 3.912 mins elapsed.

2023-03-09 22:48:49 : Adding BHF_F_Hea13188318_BHF_F_Hea13187620 to GeneExpressionMatrix for Chr (17 of 23)!, 3.961 mins elapsed.

2023-03-09 22:48:52 : Adding BHF_F_Hea13188318_BHF_F_Hea13187620 to GeneExpressionMatrix for Chr (18 of 23)!, 4.009 mins elapsed.

2023-03-09 22:48:56 : Adding BHF_F_Hea13188318_BHF_F_Hea13187620 to GeneExpressionMatrix for Chr (19 of 23)!, 4.075 mins elapsed.

2023-03-09 22:48:59 : Adding BHF_F_Hea13188318_BHF_F_Hea13187620 to GeneExpressionMatrix for Chr (20 of 23)!, 4.129 mins elapsed.

2023-03-09 22:49:02 : Adding BHF_F_Hea13188318_BHF_F_Hea13187620 to GeneExpressionMatrix for Chr (21 of 23)!, 4.177 mins elapsed.

2023-03-09 22:49:06 : Adding BHF_F_Hea13188318_BHF_F_Hea13187620 to GeneExpressionMatrix for Chr (22 of 23)!, 4.243 mins elapsed.

2023-03-09 22:49:09 : Adding BHF_F_Hea13188318_BHF_F_Hea13187620 to GeneExpressionMatrix for Chr (23 of 23)!, 4.289 mins elapsed.

2023-03-09 22:49:16 : Adding 7089STDY13216924_BHF_F_Hea13242531 to GeneExpressionMatrix for Chr (1 of 23)!, 4.405 mins elapsed.

2023-03-09 22:49:19 : Adding 7089STDY13216924_BHF_F_Hea13242531 to GeneExpressionMatrix for Chr (2 of 23)!, 4.459 mins elapsed.

2023-03-09 22:49:22 : Adding 7089STDY13216924_BHF_F_Hea13242531 to GeneExpressionMatrix for Chr (3 of 23)!, 4.51 mins elapsed.

2023-03-09 22:49:27 : Adding 7089STDY13216924_BHF_F_Hea13242531 to GeneExpressionMatrix for Chr (4 of 23)!, 4.592 mins elapsed.

2023-03-09 22:49:30 : Adding 7089STDY13216924_BHF_F_Hea13242531 to GeneExpressionMatrix for Chr (5 of 23)!, 4.645 mins elapsed.

2023-03-09 22:49:33 : Adding 7089STDY13216924_BHF_F_Hea13242531 to GeneExpressionMatrix for Chr (6 of 23)!, 4.695 mins elapsed.

2023-03-09 22:49:38 : Adding 7089STDY13216924_BHF_F_Hea13242531 to GeneExpressionMatrix for Chr (7 of 23)!, 4.779 mins elapsed.

2023-03-09 22:49:42 : Adding 7089STDY13216924_BHF_F_Hea13242531 to GeneExpressionMatrix for Chr (8 of 23)!, 4.831 mins elapsed.

2023-03-09 22:49:44 : Adding 7089STDY13216924_BHF_F_Hea13242531 to GeneExpressionMatrix for Chr (9 of 23)!, 4.879 mins elapsed.

2023-03-09 22:49:49 : Adding 7089STDY13216924_BHF_F_Hea13242531 to GeneExpressionMatrix for Chr (10 of 23)!, 4.952 mins elapsed.

2023-03-09 22:49:52 : Adding 7089STDY13216924_BHF_F_Hea13242531 to GeneExpressionMatrix for Chr (11 of 23)!, 5.001 mins elapsed.

2023-03-09 22:49:55 : Adding 7089STDY13216924_BHF_F_Hea13242531 to GeneExpressionMatrix for Chr (12 of 23)!, 5.052 mins elapsed.

2023-03-09 22:49:59 : Adding 7089STDY13216924_BHF_F_Hea13242531 to GeneExpressionMatrix for Chr (13 of 23)!, 5.123 mins elapsed.

2023-03-09 22:50:02 : Adding 7089STDY13216924_BHF_F_Hea13242531 to GeneExpressionMatrix for Chr (14 of 23)!, 5.173 mins elapsed.

2023-03-09 22:50:05 : Adding 7089STDY13216924_BHF_F_Hea13242531 to GeneExpressionMatrix for Chr (15 of 23)!, 5.222 mins elapsed.

2023-03-09 22:50:09 : Adding 7089STDY13216924_BHF_F_Hea13242531 to GeneExpressionMatrix for Chr (16 of 23)!, 5.29 mins elapsed.

2023-03-09 22:50:12 : Adding 7089STDY13216924_BHF_F_Hea13242531 to GeneExpressionMatrix for Chr (17 of 23)!, 5.338 mins elapsed.

2023-03-09 22:50:15 : Adding 7089STDY13216924_BHF_F_Hea13242531 to GeneExpressionMatrix for Chr (18 of 23)!, 5.387 mins elapsed.

2023-03-09 22:50:19 : Adding 7089STDY13216924_BHF_F_Hea13242531 to GeneExpressionMatrix for Chr (19 of 23)!, 5.453 mins elapsed.

2023-03-09 22:50:22 : Adding 7089STDY13216924_BHF_F_Hea13242531 to GeneExpressionMatrix for Chr (20 of 23)!, 5.502 mins elapsed.

2023-03-09 22:50:25 : Adding 7089STDY13216924_BHF_F_Hea13242531 to GeneExpressionMatrix for Chr (21 of 23)!, 5.55 mins elapsed.

2023-03-09 22:50:29 : Adding 7089STDY13216924_BHF_F_Hea13242531 to GeneExpressionMatrix for Chr (22 of 23)!, 5.617 mins elapsed.

2023-03-09 22:50:32 : Adding 7089STDY13216924_BHF_F_Hea13242531 to GeneExpressionMatrix for Chr (23 of 23)!, 5.664 mins elapsed.

2023-03-09 22:50:41 : Adding BHF_F_Hea11933667_BHF_F_Hea11596620 to GeneExpressionMatrix for Chr (1 of 23)!, 5.819 mins elapsed.

2023-03-09 22:50:44 : Adding BHF_F_Hea11933667_BHF_F_Hea11596620 to GeneExpressionMatrix for Chr (2 of 23)!, 5.877 mins elapsed.

2023-03-09 22:50:48 : Adding BHF_F_Hea11933667_BHF_F_Hea11596620 to GeneExpressionMatrix for Chr (3 of 23)!, 5.933 mins elapsed.

2023-03-09 22:50:52 : Adding BHF_F_Hea11933667_BHF_F_Hea11596620 to GeneExpressionMatrix for Chr (4 of 23)!, 6.008 mins elapsed.

2023-03-09 22:50:55 : Adding BHF_F_Hea11933667_BHF_F_Hea11596620 to GeneExpressionMatrix for Chr (5 of 23)!, 6.059 mins elapsed.

2023-03-09 22:50:58 : Adding BHF_F_Hea11933667_BHF_F_Hea11596620 to GeneExpressionMatrix for Chr (6 of 23)!, 6.112 mins elapsed.

2023-03-09 22:51:03 : Adding BHF_F_Hea11933667_BHF_F_Hea11596620 to GeneExpressionMatrix for Chr (7 of 23)!, 6.187 mins elapsed.

2023-03-09 22:51:06 : Adding BHF_F_Hea11933667_BHF_F_Hea11596620 to GeneExpressionMatrix for Chr (8 of 23)!, 6.241 mins elapsed.

2023-03-09 22:51:10 : Adding BHF_F_Hea11933667_BHF_F_Hea11596620 to GeneExpressionMatrix for Chr (9 of 23)!, 6.298 mins elapsed.

2023-03-09 22:51:14 : Adding BHF_F_Hea11933667_BHF_F_Hea11596620 to GeneExpressionMatrix for Chr (10 of 23)!, 6.368 mins elapsed.

2023-03-09 22:51:17 : Adding BHF_F_Hea11933667_BHF_F_Hea11596620 to GeneExpressionMatrix for Chr (11 of 23)!, 6.42 mins elapsed.

2023-03-09 22:51:20 : Adding BHF_F_Hea11933667_BHF_F_Hea11596620 to GeneExpressionMatrix for Chr (12 of 23)!, 6.473 mins elapsed.

2023-03-09 22:51:24 : Adding BHF_F_Hea11933667_BHF_F_Hea11596620 to GeneExpressionMatrix for Chr (13 of 23)!, 6.546 mins elapsed.

2023-03-09 22:51:27 : Adding BHF_F_Hea11933667_BHF_F_Hea11596620 to GeneExpressionMatrix for Chr (14 of 23)!, 6.597 mins elapsed.

2023-03-09 22:51:31 : Adding BHF_F_Hea11933667_BHF_F_Hea11596620 to GeneExpressionMatrix for Chr (15 of 23)!, 6.647 mins elapsed.

2023-03-09 22:51:35 : Adding BHF_F_Hea11933667_BHF_F_Hea11596620 to GeneExpressionMatrix for Chr (16 of 23)!, 6.718 mins elapsed.

2023-03-09 22:51:38 : Adding BHF_F_Hea11933667_BHF_F_Hea11596620 to GeneExpressionMatrix for Chr (17 of 23)!, 6.769 mins elapsed.

2023-03-09 22:51:41 : Adding BHF_F_Hea11933667_BHF_F_Hea11596620 to GeneExpressionMatrix for Chr (18 of 23)!, 6.822 mins elapsed.

2023-03-09 22:51:45 : Adding BHF_F_Hea11933667_BHF_F_Hea11596620 to GeneExpressionMatrix for Chr (19 of 23)!, 6.891 mins elapsed.

2023-03-09 22:51:48 : Adding BHF_F_Hea11933667_BHF_F_Hea11596620 to GeneExpressionMatrix for Chr (20 of 23)!, 6.945 mins elapsed.

2023-03-09 22:51:51 : Adding BHF_F_Hea11933667_BHF_F_Hea11596620 to GeneExpressionMatrix for Chr (21 of 23)!, 6.996 mins elapsed.

2023-03-09 22:51:56 : Adding BHF_F_Hea11933667_BHF_F_Hea11596620 to GeneExpressionMatrix for Chr (22 of 23)!, 7.064 mins elapsed.

2023-03-09 22:51:58 : Adding BHF_F_Hea11933667_BHF_F_Hea11596620 to GeneExpressionMatrix for Chr (23 of 23)!, 7.113 mins elapsed.

2023-03-09 22:52:05 : Adding HCAHeartST13386010_HCAHeartST13303420 to GeneExpressionMatrix for Chr (1 of 23)!, 7.227 mins elapsed.

2023-03-09 22:52:08 : Adding HCAHeartST13386010_HCAHeartST13303420 to GeneExpressionMatrix for Chr (2 of 23)!, 7.279 mins elapsed.

2023-03-09 22:52:12 : Adding HCAHeartST13386010_HCAHeartST13303420 to GeneExpressionMatrix for Chr (3 of 23)!, 7.335 mins elapsed.

2023-03-09 22:52:16 : Adding HCAHeartST13386010_HCAHeartST13303420 to GeneExpressionMatrix for Chr (4 of 23)!, 7.405 mins elapsed.

2023-03-09 22:52:19 : Adding HCAHeartST13386010_HCAHeartST13303420 to GeneExpressionMatrix for Chr (5 of 23)!, 7.454 mins elapsed.

2023-03-09 22:52:22 : Adding HCAHeartST13386010_HCAHeartST13303420 to GeneExpressionMatrix for Chr (6 of 23)!, 7.507 mins elapsed.

2023-03-09 22:52:26 : Adding HCAHeartST13386010_HCAHeartST13303420 to GeneExpressionMatrix for Chr (7 of 23)!, 7.576 mins elapsed.

2023-03-09 22:52:29 : Adding HCAHeartST13386010_HCAHeartST13303420 to GeneExpressionMatrix for Chr (8 of 23)!, 7.625 mins elapsed.

2023-03-09 22:52:32 : Adding HCAHeartST13386010_HCAHeartST13303420 to GeneExpressionMatrix for Chr (9 of 23)!, 7.674 mins elapsed.

2023-03-09 22:52:36 : Adding HCAHeartST13386010_HCAHeartST13303420 to GeneExpressionMatrix for Chr (10 of 23)!, 7.742 mins elapsed.

2023-03-09 22:52:39 : Adding HCAHeartST13386010_HCAHeartST13303420 to GeneExpressionMatrix for Chr (11 of 23)!, 7.791 mins elapsed.

2023-03-09 22:52:42 : Adding HCAHeartST13386010_HCAHeartST13303420 to GeneExpressionMatrix for Chr (12 of 23)!, 7.841 mins elapsed.

2023-03-09 22:52:46 : Adding HCAHeartST13386010_HCAHeartST13303420 to GeneExpressionMatrix for Chr (13 of 23)!, 7.91 mins elapsed.

2023-03-09 22:52:49 : Adding HCAHeartST13386010_HCAHeartST13303420 to GeneExpressionMatrix for Chr (14 of 23)!, 7.958 mins elapsed.

2023-03-09 22:52:52 : Adding HCAHeartST13386010_HCAHeartST13303420 to GeneExpressionMatrix for Chr (15 of 23)!, 8.006 mins elapsed.

2023-03-09 22:52:56 : Adding HCAHeartST13386010_HCAHeartST13303420 to GeneExpressionMatrix for Chr (16 of 23)!, 8.073 mins elapsed.

2023-03-09 22:52:59 : Adding HCAHeartST13386010_HCAHeartST13303420 to GeneExpressionMatrix for Chr (17 of 23)!, 8.124 mins elapsed.

2023-03-09 22:53:02 : Adding HCAHeartST13386010_HCAHeartST13303420 to GeneExpressionMatrix for Chr (18 of 23)!, 8.174 mins elapsed.

2023-03-09 22:53:06 : Adding HCAHeartST13386010_HCAHeartST13303420 to GeneExpressionMatrix for Chr (19 of 23)!, 8.242 mins elapsed.

2023-03-09 22:53:10 : Adding HCAHeartST13386010_HCAHeartST13303420 to GeneExpressionMatrix for Chr (20 of 23)!, 8.304 mins elapsed.

2023-03-09 22:53:13 : Adding HCAHeartST13386010_HCAHeartST13303420 to GeneExpressionMatrix for Chr (21 of 23)!, 8.358 mins elapsed.

2023-03-09 22:53:17 : Adding HCAHeartST13386010_HCAHeartST13303420 to GeneExpressionMatrix for Chr (22 of 23)!, 8.425 mins elapsed.

2023-03-09 22:53:20 : Adding HCAHeartST13386010_HCAHeartST13303420 to GeneExpressionMatrix for Chr (23 of 23)!, 8.475 mins elapsed.

2023-03-09 22:53:27 : Adding 7089STDY13216925_BHF_F_Hea13242532 to GeneExpressionMatrix for Chr (1 of 23)!, 8.582 mins elapsed.

2023-03-09 22:53:29 : Adding 7089STDY13216925_BHF_F_Hea13242532 to GeneExpressionMatrix for Chr (2 of 23)!, 8.63 mins elapsed.

2023-03-09 22:53:32 : Adding 7089STDY13216925_BHF_F_Hea13242532 to GeneExpressionMatrix for Chr (3 of 23)!, 8.677 mins elapsed.

2023-03-09 22:53:36 : Adding 7089STDY13216925_BHF_F_Hea13242532 to GeneExpressionMatrix for Chr (4 of 23)!, 8.744 mins elapsed.

2023-03-09 22:53:39 : Adding 7089STDY13216925_BHF_F_Hea13242532 to GeneExpressionMatrix for Chr (5 of 23)!, 8.791 mins elapsed.

2023-03-09 22:53:42 : Adding 7089STDY13216925_BHF_F_Hea13242532 to GeneExpressionMatrix for Chr (6 of 23)!, 8.838 mins elapsed.

2023-03-09 22:53:46 : Adding 7089STDY13216925_BHF_F_Hea13242532 to GeneExpressionMatrix for Chr (7 of 23)!, 8.904 mins elapsed.

2023-03-09 22:53:49 : Adding 7089STDY13216925_BHF_F_Hea13242532 to GeneExpressionMatrix for Chr (8 of 23)!, 8.951 mins elapsed.

2023-03-09 22:53:51 : Adding 7089STDY13216925_BHF_F_Hea13242532 to GeneExpressionMatrix for Chr (9 of 23)!, 8.996 mins elapsed.

2023-03-09 22:53:55 : Adding 7089STDY13216925_BHF_F_Hea13242532 to GeneExpressionMatrix for Chr (10 of 23)!, 9.062 mins elapsed.

2023-03-09 22:53:58 : Adding 7089STDY13216925_BHF_F_Hea13242532 to GeneExpressionMatrix for Chr (11 of 23)!, 9.108 mins elapsed.

2023-03-09 22:54:01 : Adding 7089STDY13216925_BHF_F_Hea13242532 to GeneExpressionMatrix for Chr (12 of 23)!, 9.154 mins elapsed.

2023-03-09 22:54:05 : Adding 7089STDY13216925_BHF_F_Hea13242532 to GeneExpressionMatrix for Chr (13 of 23)!, 9.223 mins elapsed.

2023-03-09 22:54:08 : Adding 7089STDY13216925_BHF_F_Hea13242532 to GeneExpressionMatrix for Chr (14 of 23)!, 9.268 mins elapsed.

2023-03-09 22:54:10 : Adding 7089STDY13216925_BHF_F_Hea13242532 to GeneExpressionMatrix for Chr (15 of 23)!, 9.313 mins elapsed.

2023-03-09 22:54:14 : Adding 7089STDY13216925_BHF_F_Hea13242532 to GeneExpressionMatrix for Chr (16 of 23)!, 9.377 mins elapsed.

2023-03-09 22:54:17 : Adding 7089STDY13216925_BHF_F_Hea13242532 to GeneExpressionMatrix for Chr (17 of 23)!, 9.422 mins elapsed.

2023-03-09 22:54:20 : Adding 7089STDY13216925_BHF_F_Hea13242532 to GeneExpressionMatrix for Chr (18 of 23)!, 9.468 mins elapsed.

2023-03-09 22:54:24 : Adding 7089STDY13216925_BHF_F_Hea13242532 to GeneExpressionMatrix for Chr (19 of 23)!, 9.532 mins elapsed.

2023-03-09 22:54:26 : Adding 7089STDY13216925_BHF_F_Hea13242532 to GeneExpressionMatrix for Chr (20 of 23)!, 9.577 mins elapsed.

2023-03-09 22:54:29 : Adding 7089STDY13216925_BHF_F_Hea13242532 to GeneExpressionMatrix for Chr (21 of 23)!, 9.624 mins elapsed.

2023-03-09 22:54:33 : Adding 7089STDY13216925_BHF_F_Hea13242532 to GeneExpressionMatrix for Chr (22 of 23)!, 9.689 mins elapsed.

2023-03-09 22:54:36 : Adding 7089STDY13216925_BHF_F_Hea13242532 to GeneExpressionMatrix for Chr (23 of 23)!, 9.734 mins elapsed.

2023-03-09 22:54:43 : Adding BHF_F_Hea11933666_BHF_F_Hea11596619 to GeneExpressionMatrix for Chr (1 of 23)!, 9.849 mins elapsed.

2023-03-09 22:54:46 : Adding BHF_F_Hea11933666_BHF_F_Hea11596619 to GeneExpressionMatrix for Chr (2 of 23)!, 9.906 mins elapsed.

2023-03-09 22:54:49 : Adding BHF_F_Hea11933666_BHF_F_Hea11596619 to GeneExpressionMatrix for Chr (3 of 23)!, 9.961 mins elapsed.

2023-03-09 22:54:54 : Adding BHF_F_Hea11933666_BHF_F_Hea11596619 to GeneExpressionMatrix for Chr (4 of 23)!, 10.034 mins elapsed.

2023-03-09 22:54:57 : Adding BHF_F_Hea11933666_BHF_F_Hea11596619 to GeneExpressionMatrix for Chr (5 of 23)!, 10.086 mins elapsed.

2023-03-09 22:55:00 : Adding BHF_F_Hea11933666_BHF_F_Hea11596619 to GeneExpressionMatrix for Chr (6 of 23)!, 10.138 mins elapsed.

2023-03-09 22:55:05 : Adding BHF_F_Hea11933666_BHF_F_Hea11596619 to GeneExpressionMatrix for Chr (7 of 23)!, 10.226 mins elapsed.

2023-03-09 22:55:08 : Adding BHF_F_Hea11933666_BHF_F_Hea11596619 to GeneExpressionMatrix for Chr (8 of 23)!, 10.279 mins elapsed.

2023-03-09 22:55:11 : Adding BHF_F_Hea11933666_BHF_F_Hea11596619 to GeneExpressionMatrix for Chr (9 of 23)!, 10.329 mins elapsed.

2023-03-09 22:55:16 : Adding BHF_F_Hea11933666_BHF_F_Hea11596619 to GeneExpressionMatrix for Chr (10 of 23)!, 10.402 mins elapsed.

2023-03-09 22:55:19 : Adding BHF_F_Hea11933666_BHF_F_Hea11596619 to GeneExpressionMatrix for Chr (11 of 23)!, 10.455 mins elapsed.

2023-03-09 22:55:22 : Adding BHF_F_Hea11933666_BHF_F_Hea11596619 to GeneExpressionMatrix for Chr (12 of 23)!, 10.508 mins elapsed.

2023-03-09 22:55:26 : Adding BHF_F_Hea11933666_BHF_F_Hea11596619 to GeneExpressionMatrix for Chr (13 of 23)!, 10.58 mins elapsed.

2023-03-09 22:55:29 : Adding BHF_F_Hea11933666_BHF_F_Hea11596619 to GeneExpressionMatrix for Chr (14 of 23)!, 10.629 mins elapsed.

2023-03-09 22:55:32 : Adding BHF_F_Hea11933666_BHF_F_Hea11596619 to GeneExpressionMatrix for Chr (15 of 23)!, 10.679 mins elapsed.

2023-03-09 22:55:36 : Adding BHF_F_Hea11933666_BHF_F_Hea11596619 to GeneExpressionMatrix for Chr (16 of 23)!, 10.746 mins elapsed.

2023-03-09 22:55:39 : Adding BHF_F_Hea11933666_BHF_F_Hea11596619 to GeneExpressionMatrix for Chr (17 of 23)!, 10.795 mins elapsed.

2023-03-09 22:55:42 : Adding BHF_F_Hea11933666_BHF_F_Hea11596619 to GeneExpressionMatrix for Chr (18 of 23)!, 10.846 mins elapsed.

2023-03-09 22:55:46 : Adding BHF_F_Hea11933666_BHF_F_Hea11596619 to GeneExpressionMatrix for Chr (19 of 23)!, 10.912 mins elapsed.

2023-03-09 22:55:49 : Adding BHF_F_Hea11933666_BHF_F_Hea11596619 to GeneExpressionMatrix for Chr (20 of 23)!, 10.962 mins elapsed.

2023-03-09 22:55:52 : Adding BHF_F_Hea11933666_BHF_F_Hea11596619 to GeneExpressionMatrix for Chr (21 of 23)!, 11.011 mins elapsed.

2023-03-09 22:55:56 : Adding BHF_F_Hea11933666_BHF_F_Hea11596619 to GeneExpressionMatrix for Chr (22 of 23)!, 11.078 mins elapsed.

2023-03-09 22:55:59 : Adding BHF_F_Hea11933666_BHF_F_Hea11596619 to GeneExpressionMatrix for Chr (23 of 23)!, 11.126 mins elapsed.

2023-03-09 22:56:06 : Adding BHF_F_Hea13188324_BHF_F_Hea13187626 to GeneExpressionMatrix for Chr (1 of 23)!, 11.236 mins elapsed.

2023-03-09 22:56:09 : Adding BHF_F_Hea13188324_BHF_F_Hea13187626 to GeneExpressionMatrix for Chr (2 of 23)!, 11.294 mins elapsed.

2023-03-09 22:56:13 : Adding BHF_F_Hea13188324_BHF_F_Hea13187626 to GeneExpressionMatrix for Chr (3 of 23)!, 11.347 mins elapsed.

2023-03-09 22:56:17 : Adding BHF_F_Hea13188324_BHF_F_Hea13187626 to GeneExpressionMatrix for Chr (4 of 23)!, 11.416 mins elapsed.

2023-03-09 22:56:20 : Adding BHF_F_Hea13188324_BHF_F_Hea13187626 to GeneExpressionMatrix for Chr (5 of 23)!, 11.464 mins elapsed.

2023-03-09 22:56:23 : Adding BHF_F_Hea13188324_BHF_F_Hea13187626 to GeneExpressionMatrix for Chr (6 of 23)!, 11.514 mins elapsed.

2023-03-09 22:56:27 : Adding BHF_F_Hea13188324_BHF_F_Hea13187626 to GeneExpressionMatrix for Chr (7 of 23)!, 11.584 mins elapsed.

2023-03-09 22:56:30 : Adding BHF_F_Hea13188324_BHF_F_Hea13187626 to GeneExpressionMatrix for Chr (8 of 23)!, 11.644 mins elapsed.

2023-03-09 22:56:34 : Adding BHF_F_Hea13188324_BHF_F_Hea13187626 to GeneExpressionMatrix for Chr (9 of 23)!, 11.704 mins elapsed.

2023-03-09 22:56:38 : Adding BHF_F_Hea13188324_BHF_F_Hea13187626 to GeneExpressionMatrix for Chr (10 of 23)!, 11.776 mins elapsed.

2023-03-09 22:56:41 : Adding BHF_F_Hea13188324_BHF_F_Hea13187626 to GeneExpressionMatrix for Chr (11 of 23)!, 11.828 mins elapsed.

2023-03-09 22:56:45 : Adding BHF_F_Hea13188324_BHF_F_Hea13187626 to GeneExpressionMatrix for Chr (12 of 23)!, 11.89 mins elapsed.

2023-03-09 22:56:50 : Adding BHF_F_Hea13188324_BHF_F_Hea13187626 to GeneExpressionMatrix for Chr (13 of 23)!, 11.972 mins elapsed.

2023-03-09 22:56:53 : Adding BHF_F_Hea13188324_BHF_F_Hea13187626 to GeneExpressionMatrix for Chr (14 of 23)!, 12.024 mins elapsed.

2023-03-09 22:56:56 : Adding BHF_F_Hea13188324_BHF_F_Hea13187626 to GeneExpressionMatrix for Chr (15 of 23)!, 12.074 mins elapsed.

2023-03-09 22:57:00 : Adding BHF_F_Hea13188324_BHF_F_Hea13187626 to GeneExpressionMatrix for Chr (16 of 23)!, 12.144 mins elapsed.

2023-03-09 22:57:03 : Adding BHF_F_Hea13188324_BHF_F_Hea13187626 to GeneExpressionMatrix for Chr (17 of 23)!, 12.192 mins elapsed.

2023-03-09 22:57:06 : Adding BHF_F_Hea13188324_BHF_F_Hea13187626 to GeneExpressionMatrix for Chr (18 of 23)!, 12.241 mins elapsed.

2023-03-09 22:57:10 : Adding BHF_F_Hea13188324_BHF_F_Hea13187626 to GeneExpressionMatrix for Chr (19 of 23)!, 12.309 mins elapsed.

2023-03-09 22:57:13 : Adding BHF_F_Hea13188324_BHF_F_Hea13187626 to GeneExpressionMatrix for Chr (20 of 23)!, 12.356 mins elapsed.

2023-03-09 22:57:16 : Adding BHF_F_Hea13188324_BHF_F_Hea13187626 to GeneExpressionMatrix for Chr (21 of 23)!, 12.404 mins elapsed.

2023-03-09 22:57:20 : Adding BHF_F_Hea13188324_BHF_F_Hea13187626 to GeneExpressionMatrix for Chr (22 of 23)!, 12.47 mins elapsed.

2023-03-09 22:57:23 : Adding BHF_F_Hea13188324_BHF_F_Hea13187626 to GeneExpressionMatrix for Chr (23 of 23)!, 12.525 mins elapsed.

2023-03-09 22:57:33 : Adding BHF_F_Hea11933668_BHF_F_Hea11596621 to GeneExpressionMatrix for Chr (1 of 23)!, 12.689 mins elapsed.

2023-03-09 22:57:37 : Adding BHF_F_Hea11933668_BHF_F_Hea11596621 to GeneExpressionMatrix for Chr (2 of 23)!, 12.753 mins elapsed.

2023-03-09 22:57:40 : Adding BHF_F_Hea11933668_BHF_F_Hea11596621 to GeneExpressionMatrix for Chr (3 of 23)!, 12.811 mins elapsed.

2023-03-09 22:57:45 : Adding BHF_F_Hea11933668_BHF_F_Hea11596621 to GeneExpressionMatrix for Chr (4 of 23)!, 12.882 mins elapsed.

2023-03-09 22:57:48 : Adding BHF_F_Hea11933668_BHF_F_Hea11596621 to GeneExpressionMatrix for Chr (5 of 23)!, 12.935 mins elapsed.

2023-03-09 22:57:51 : Adding BHF_F_Hea11933668_BHF_F_Hea11596621 to GeneExpressionMatrix for Chr (6 of 23)!, 12.988 mins elapsed.

2023-03-09 22:57:57 : Adding BHF_F_Hea11933668_BHF_F_Hea11596621 to GeneExpressionMatrix for Chr (7 of 23)!, 13.081 mins elapsed.

2023-03-09 22:58:00 : Adding BHF_F_Hea11933668_BHF_F_Hea11596621 to GeneExpressionMatrix for Chr (8 of 23)!, 13.143 mins elapsed.

2023-03-09 22:58:03 : Adding BHF_F_Hea11933668_BHF_F_Hea11596621 to GeneExpressionMatrix for Chr (9 of 23)!, 13.195 mins elapsed.

2023-03-09 22:58:08 : Adding BHF_F_Hea11933668_BHF_F_Hea11596621 to GeneExpressionMatrix for Chr (10 of 23)!, 13.276 mins elapsed.

2023-03-09 22:58:12 : Adding BHF_F_Hea11933668_BHF_F_Hea11596621 to GeneExpressionMatrix for Chr (11 of 23)!, 13.335 mins elapsed.

2023-03-09 22:58:15 : Adding BHF_F_Hea11933668_BHF_F_Hea11596621 to GeneExpressionMatrix for Chr (12 of 23)!, 13.389 mins elapsed.

2023-03-09 22:58:19 : Adding BHF_F_Hea11933668_BHF_F_Hea11596621 to GeneExpressionMatrix for Chr (13 of 23)!, 13.462 mins elapsed.

2023-03-09 22:58:22 : Adding BHF_F_Hea11933668_BHF_F_Hea11596621 to GeneExpressionMatrix for Chr (14 of 23)!, 13.511 mins elapsed.

2023-03-09 22:58:25 : Adding BHF_F_Hea11933668_BHF_F_Hea11596621 to GeneExpressionMatrix for Chr (15 of 23)!, 13.562 mins elapsed.

2023-03-09 22:58:30 : Adding BHF_F_Hea11933668_BHF_F_Hea11596621 to GeneExpressionMatrix for Chr (16 of 23)!, 13.64 mins elapsed.

2023-03-09 22:58:33 : Adding BHF_F_Hea11933668_BHF_F_Hea11596621 to GeneExpressionMatrix for Chr (17 of 23)!, 13.691 mins elapsed.

2023-03-09 22:58:37 : Adding BHF_F_Hea11933668_BHF_F_Hea11596621 to GeneExpressionMatrix for Chr (18 of 23)!, 13.756 mins elapsed.

2023-03-09 22:58:42 : Adding BHF_F_Hea11933668_BHF_F_Hea11596621 to GeneExpressionMatrix for Chr (19 of 23)!, 13.843 mins elapsed.

2023-03-09 22:58:46 : Adding BHF_F_Hea11933668_BHF_F_Hea11596621 to GeneExpressionMatrix for Chr (20 of 23)!, 13.909 mins elapsed.

2023-03-09 22:58:49 : Adding BHF_F_Hea11933668_BHF_F_Hea11596621 to GeneExpressionMatrix for Chr (21 of 23)!, 13.96 mins elapsed.

2023-03-09 22:58:53 : Adding BHF_F_Hea11933668_BHF_F_Hea11596621 to GeneExpressionMatrix for Chr (22 of 23)!, 14.029 mins elapsed.

2023-03-09 22:58:56 : Adding BHF_F_Hea11933668_BHF_F_Hea11596621 to GeneExpressionMatrix for Chr (23 of 23)!, 14.079 mins elapsed.

2023-03-09 22:59:03 : Adding HCAHeartST13386009_HCAHeartST13303419 to GeneExpressionMatrix for Chr (1 of 23)!, 14.192 mins elapsed.

2023-03-09 22:59:07 : Adding HCAHeartST13386009_HCAHeartST13303419 to GeneExpressionMatrix for Chr (2 of 23)!, 14.249 mins elapsed.

2023-03-09 22:59:10 : Adding HCAHeartST13386009_HCAHeartST13303419 to GeneExpressionMatrix for Chr (3 of 23)!, 14.3 mins elapsed.

2023-03-09 22:59:14 : Adding HCAHeartST13386009_HCAHeartST13303419 to GeneExpressionMatrix for Chr (4 of 23)!, 14.369 mins elapsed.

2023-03-09 22:59:17 : Adding HCAHeartST13386009_HCAHeartST13303419 to GeneExpressionMatrix for Chr (5 of 23)!, 14.421 mins elapsed.

2023-03-09 22:59:21 : Adding HCAHeartST13386009_HCAHeartST13303419 to GeneExpressionMatrix for Chr (6 of 23)!, 14.481 mins elapsed.

2023-03-09 22:59:25 : Adding HCAHeartST13386009_HCAHeartST13303419 to GeneExpressionMatrix for Chr (7 of 23)!, 14.557 mins elapsed.

2023-03-09 22:59:28 : Adding HCAHeartST13386009_HCAHeartST13303419 to GeneExpressionMatrix for Chr (8 of 23)!, 14.606 mins elapsed.

2023-03-09 22:59:31 : Adding HCAHeartST13386009_HCAHeartST13303419 to GeneExpressionMatrix for Chr (9 of 23)!, 14.653 mins elapsed.

2023-03-09 22:59:35 : Adding HCAHeartST13386009_HCAHeartST13303419 to GeneExpressionMatrix for Chr (10 of 23)!, 14.722 mins elapsed.

2023-03-09 22:59:38 : Adding HCAHeartST13386009_HCAHeartST13303419 to GeneExpressionMatrix for Chr (11 of 23)!, 14.77 mins elapsed.

2023-03-09 22:59:41 : Adding HCAHeartST13386009_HCAHeartST13303419 to GeneExpressionMatrix for Chr (12 of 23)!, 14.818 mins elapsed.

2023-03-09 22:59:45 : Adding HCAHeartST13386009_HCAHeartST13303419 to GeneExpressionMatrix for Chr (13 of 23)!, 14.885 mins elapsed.

2023-03-09 22:59:48 : Adding HCAHeartST13386009_HCAHeartST13303419 to GeneExpressionMatrix for Chr (14 of 23)!, 14.931 mins elapsed.

2023-03-09 22:59:50 : Adding HCAHeartST13386009_HCAHeartST13303419 to GeneExpressionMatrix for Chr (15 of 23)!, 14.978 mins elapsed.

2023-03-09 22:59:54 : Adding HCAHeartST13386009_HCAHeartST13303419 to GeneExpressionMatrix for Chr (16 of 23)!, 15.047 mins elapsed.

2023-03-09 22:59:57 : Adding HCAHeartST13386009_HCAHeartST13303419 to GeneExpressionMatrix for Chr (17 of 23)!, 15.097 mins elapsed.

2023-03-09 23:00:00 : Adding HCAHeartST13386009_HCAHeartST13303419 to GeneExpressionMatrix for Chr (18 of 23)!, 15.145 mins elapsed.

2023-03-09 23:00:05 : Adding HCAHeartST13386009_HCAHeartST13303419 to GeneExpressionMatrix for Chr (19 of 23)!, 15.22 mins elapsed.

2023-03-09 23:00:08 : Adding HCAHeartST13386009_HCAHeartST13303419 to GeneExpressionMatrix for Chr (20 of 23)!, 15.279 mins elapsed.

2023-03-09 23:00:11 : Adding HCAHeartST13386009_HCAHeartST13303419 to GeneExpressionMatrix for Chr (21 of 23)!, 15.326 mins elapsed.

2023-03-09 23:00:16 : Adding HCAHeartST13386009_HCAHeartST13303419 to GeneExpressionMatrix for Chr (22 of 23)!, 15.404 mins elapsed.

2023-03-09 23:00:19 : Adding HCAHeartST13386009_HCAHeartST13303419 to GeneExpressionMatrix for Chr (23 of 23)!, 15.45 mins elapsed.

2023-03-09 23:00:26 : Adding BHF_F_Hea11933669_BHF_F_Hea11596622 to GeneExpressionMatrix for Chr (1 of 23)!, 15.567 mins elapsed.

2023-03-09 23:00:29 : Adding BHF_F_Hea11933669_BHF_F_Hea11596622 to GeneExpressionMatrix for Chr (2 of 23)!, 15.625 mins elapsed.

2023-03-09 23:00:33 : Adding BHF_F_Hea11933669_BHF_F_Hea11596622 to GeneExpressionMatrix for Chr (3 of 23)!, 15.692 mins elapsed.

2023-03-09 23:00:38 : Adding BHF_F_Hea11933669_BHF_F_Hea11596622 to GeneExpressionMatrix for Chr (4 of 23)!, 15.765 mins elapsed.

2023-03-09 23:00:41 : Adding BHF_F_Hea11933669_BHF_F_Hea11596622 to GeneExpressionMatrix for Chr (5 of 23)!, 15.815 mins elapsed.

2023-03-09 23:00:44 : Adding BHF_F_Hea11933669_BHF_F_Hea11596622 to GeneExpressionMatrix for Chr (6 of 23)!, 15.866 mins elapsed.

2023-03-09 23:00:48 : Adding BHF_F_Hea11933669_BHF_F_Hea11596622 to GeneExpressionMatrix for Chr (7 of 23)!, 15.938 mins elapsed.

2023-03-09 23:00:51 : Adding BHF_F_Hea11933669_BHF_F_Hea11596622 to GeneExpressionMatrix for Chr (8 of 23)!, 15.99 mins elapsed.

2023-03-09 23:00:54 : Adding BHF_F_Hea11933669_BHF_F_Hea11596622 to GeneExpressionMatrix for Chr (9 of 23)!, 16.04 mins elapsed.

2023-03-09 23:00:58 : Adding BHF_F_Hea11933669_BHF_F_Hea11596622 to GeneExpressionMatrix for Chr (10 of 23)!, 16.109 mins elapsed.

2023-03-09 23:01:01 : Adding BHF_F_Hea11933669_BHF_F_Hea11596622 to GeneExpressionMatrix for Chr (11 of 23)!, 16.16 mins elapsed.

2023-03-09 23:01:04 : Adding BHF_F_Hea11933669_BHF_F_Hea11596622 to GeneExpressionMatrix for Chr (12 of 23)!, 16.213 mins elapsed.

2023-03-09 23:01:09 : Adding BHF_F_Hea11933669_BHF_F_Hea11596622 to GeneExpressionMatrix for Chr (13 of 23)!, 16.283 mins elapsed.

2023-03-09 23:01:12 : Adding BHF_F_Hea11933669_BHF_F_Hea11596622 to GeneExpressionMatrix for Chr (14 of 23)!, 16.338 mins elapsed.

2023-03-09 23:01:15 : Adding BHF_F_Hea11933669_BHF_F_Hea11596622 to GeneExpressionMatrix for Chr (15 of 23)!, 16.392 mins elapsed.

2023-03-09 23:01:19 : Adding BHF_F_Hea11933669_BHF_F_Hea11596622 to GeneExpressionMatrix for Chr (16 of 23)!, 16.462 mins elapsed.

2023-03-09 23:01:22 : Adding BHF_F_Hea11933669_BHF_F_Hea11596622 to GeneExpressionMatrix for Chr (17 of 23)!, 16.513 mins elapsed.

2023-03-09 23:01:26 : Adding BHF_F_Hea11933669_BHF_F_Hea11596622 to GeneExpressionMatrix for Chr (18 of 23)!, 16.565 mins elapsed.

2023-03-09 23:01:30 : Adding BHF_F_Hea11933669_BHF_F_Hea11596622 to GeneExpressionMatrix for Chr (19 of 23)!, 16.634 mins elapsed.

2023-03-09 23:01:33 : Adding BHF_F_Hea11933669_BHF_F_Hea11596622 to GeneExpressionMatrix for Chr (20 of 23)!, 16.686 mins elapsed.

2023-03-09 23:01:36 : Adding BHF_F_Hea11933669_BHF_F_Hea11596622 to GeneExpressionMatrix for Chr (21 of 23)!, 16.746 mins elapsed.

2023-03-09 23:01:41 : Adding BHF_F_Hea11933669_BHF_F_Hea11596622 to GeneExpressionMatrix for Chr (22 of 23)!, 16.815 mins elapsed.

2023-03-09 23:01:44 : Adding BHF_F_Hea11933669_BHF_F_Hea11596622 to GeneExpressionMatrix for Chr (23 of 23)!, 16.865 mins elapsed.

2023-03-09 23:01:51 : Adding BHF_F_Hea13168898_BHF_F_Hea13168514 to GeneExpressionMatrix for Chr (1 of 23)!, 16.984 mins elapsed.

2023-03-09 23:01:54 : Adding BHF_F_Hea13168898_BHF_F_Hea13168514 to GeneExpressionMatrix for Chr (2 of 23)!, 17.044 mins elapsed.

2023-03-09 23:01:58 : Adding BHF_F_Hea13168898_BHF_F_Hea13168514 to GeneExpressionMatrix for Chr (3 of 23)!, 17.104 mins elapsed.

2023-03-09 23:02:03 : Adding BHF_F_Hea13168898_BHF_F_Hea13168514 to GeneExpressionMatrix for Chr (4 of 23)!, 17.183 mins elapsed.

2023-03-09 23:02:06 : Adding BHF_F_Hea13168898_BHF_F_Hea13168514 to GeneExpressionMatrix for Chr (5 of 23)!, 17.234 mins elapsed.

2023-03-09 23:02:09 : Adding BHF_F_Hea13168898_BHF_F_Hea13168514 to GeneExpressionMatrix for Chr (6 of 23)!, 17.287 mins elapsed.

2023-03-09 23:02:13 : Adding BHF_F_Hea13168898_BHF_F_Hea13168514 to GeneExpressionMatrix for Chr (7 of 23)!, 17.36 mins elapsed.

2023-03-09 23:02:16 : Adding BHF_F_Hea13168898_BHF_F_Hea13168514 to GeneExpressionMatrix for Chr (8 of 23)!, 17.411 mins elapsed.

2023-03-09 23:02:19 : Adding BHF_F_Hea13168898_BHF_F_Hea13168514 to GeneExpressionMatrix for Chr (9 of 23)!, 17.46 mins elapsed.

2023-03-09 23:02:23 : Adding BHF_F_Hea13168898_BHF_F_Hea13168514 to GeneExpressionMatrix for Chr (10 of 23)!, 17.53 mins elapsed.

2023-03-09 23:02:27 : Adding BHF_F_Hea13168898_BHF_F_Hea13168514 to GeneExpressionMatrix for Chr (11 of 23)!, 17.585 mins elapsed.

2023-03-09 23:02:30 : Adding BHF_F_Hea13168898_BHF_F_Hea13168514 to GeneExpressionMatrix for Chr (12 of 23)!, 17.647 mins elapsed.

2023-03-09 23:02:35 : Adding BHF_F_Hea13168898_BHF_F_Hea13168514 to GeneExpressionMatrix for Chr (13 of 23)!, 17.717 mins elapsed.

2023-03-09 23:02:38 : Adding BHF_F_Hea13168898_BHF_F_Hea13168514 to GeneExpressionMatrix for Chr (14 of 23)!, 17.768 mins elapsed.

2023-03-09 23:02:41 : Adding BHF_F_Hea13168898_BHF_F_Hea13168514 to GeneExpressionMatrix for Chr (15 of 23)!, 17.817 mins elapsed.

2023-03-09 23:02:45 : Adding BHF_F_Hea13168898_BHF_F_Hea13168514 to GeneExpressionMatrix for Chr (16 of 23)!, 17.89 mins elapsed.

2023-03-09 23:02:48 : Adding BHF_F_Hea13168898_BHF_F_Hea13168514 to GeneExpressionMatrix for Chr (17 of 23)!, 17.94 mins elapsed.

2023-03-09 23:02:51 : Adding BHF_F_Hea13168898_BHF_F_Hea13168514 to GeneExpressionMatrix for Chr (18 of 23)!, 17.993 mins elapsed.

2023-03-09 23:02:55 : Adding BHF_F_Hea13168898_BHF_F_Hea13168514 to GeneExpressionMatrix for Chr (19 of 23)!, 18.059 mins elapsed.

2023-03-09 23:02:58 : Adding BHF_F_Hea13168898_BHF_F_Hea13168514 to GeneExpressionMatrix for Chr (20 of 23)!, 18.11 mins elapsed.

2023-03-09 23:03:02 : Adding BHF_F_Hea13168898_BHF_F_Hea13168514 to GeneExpressionMatrix for Chr (21 of 23)!, 18.169 mins elapsed.

2023-03-09 23:03:07 : Adding BHF_F_Hea13168898_BHF_F_Hea13168514 to GeneExpressionMatrix for Chr (22 of 23)!, 18.254 mins elapsed.

2023-03-09 23:03:10 : Adding BHF_F_Hea13168898_BHF_F_Hea13168514 to GeneExpressionMatrix for Chr (23 of 23)!, 18.303 mins elapsed.

2023-03-09 23:03:17 : Adding BHF_F_Hea13168896_BHF_F_Hea13168512 to GeneExpressionMatrix for Chr (1 of 23)!, 18.42 mins elapsed.

2023-03-09 23:03:20 : Adding BHF_F_Hea13168896_BHF_F_Hea13168512 to GeneExpressionMatrix for Chr (2 of 23)!, 18.479 mins elapsed.

2023-03-09 23:03:24 : Adding BHF_F_Hea13168896_BHF_F_Hea13168512 to GeneExpressionMatrix for Chr (3 of 23)!, 18.533 mins elapsed.

2023-03-09 23:03:29 : Adding BHF_F_Hea13168896_BHF_F_Hea13168512 to GeneExpressionMatrix for Chr (4 of 23)!, 18.618 mins elapsed.

2023-03-09 23:03:32 : Adding BHF_F_Hea13168896_BHF_F_Hea13168512 to GeneExpressionMatrix for Chr (5 of 23)!, 18.669 mins elapsed.

2023-03-09 23:03:35 : Adding BHF_F_Hea13168896_BHF_F_Hea13168512 to GeneExpressionMatrix for Chr (6 of 23)!, 18.721 mins elapsed.

2023-03-09 23:03:39 : Adding BHF_F_Hea13168896_BHF_F_Hea13168512 to GeneExpressionMatrix for Chr (7 of 23)!, 18.792 mins elapsed.

2023-03-09 23:03:42 : Adding BHF_F_Hea13168896_BHF_F_Hea13168512 to GeneExpressionMatrix for Chr (8 of 23)!, 18.843 mins elapsed.

2023-03-09 23:03:45 : Adding BHF_F_Hea13168896_BHF_F_Hea13168512 to GeneExpressionMatrix for Chr (9 of 23)!, 18.892 mins elapsed.

2023-03-09 23:03:49 : Adding BHF_F_Hea13168896_BHF_F_Hea13168512 to GeneExpressionMatrix for Chr (10 of 23)!, 18.961 mins elapsed.

2023-03-09 23:03:52 : Adding BHF_F_Hea13168896_BHF_F_Hea13168512 to GeneExpressionMatrix for Chr (11 of 23)!, 19.011 mins elapsed.

2023-03-09 23:03:55 : Adding BHF_F_Hea13168896_BHF_F_Hea13168512 to GeneExpressionMatrix for Chr (12 of 23)!, 19.062 mins elapsed.

2023-03-09 23:04:00 : Adding BHF_F_Hea13168896_BHF_F_Hea13168512 to GeneExpressionMatrix for Chr (13 of 23)!, 19.133 mins elapsed.

2023-03-09 23:04:03 : Adding BHF_F_Hea13168896_BHF_F_Hea13168512 to GeneExpressionMatrix for Chr (14 of 23)!, 19.181 mins elapsed.

2023-03-09 23:04:05 : Adding BHF_F_Hea13168896_BHF_F_Hea13168512 to GeneExpressionMatrix for Chr (15 of 23)!, 19.229 mins elapsed.

2023-03-09 23:04:10 : Adding BHF_F_Hea13168896_BHF_F_Hea13168512 to GeneExpressionMatrix for Chr (16 of 23)!, 19.298 mins elapsed.

2023-03-09 23:04:13 : Adding BHF_F_Hea13168896_BHF_F_Hea13168512 to GeneExpressionMatrix for Chr (17 of 23)!, 19.348 mins elapsed.

2023-03-09 23:04:16 : Adding BHF_F_Hea13168896_BHF_F_Hea13168512 to GeneExpressionMatrix for Chr (18 of 23)!, 19.397 mins elapsed.

2023-03-09 23:04:20 : Adding BHF_F_Hea13168896_BHF_F_Hea13168512 to GeneExpressionMatrix for Chr (19 of 23)!, 19.465 mins elapsed.

2023-03-09 23:04:23 : Adding BHF_F_Hea13168896_BHF_F_Hea13168512 to GeneExpressionMatrix for Chr (20 of 23)!, 19.515 mins elapsed.

2023-03-09 23:04:25 : Adding BHF_F_Hea13168896_BHF_F_Hea13168512 to GeneExpressionMatrix for Chr (21 of 23)!, 19.563 mins elapsed.

2023-03-09 23:04:29 : Adding BHF_F_Hea13168896_BHF_F_Hea13168512 to GeneExpressionMatrix for Chr (22 of 23)!, 19.629 mins elapsed.

2023-03-09 23:04:32 : Adding BHF_F_Hea13168896_BHF_F_Hea13168512 to GeneExpressionMatrix for Chr (23 of 23)!, 19.675 mins elapsed.

2023-03-09 23:04:39 : Adding BHF_F_Hea13168894_BHF_F_Hea13168510 to GeneExpressionMatrix for Chr (1 of 23)!, 19.793 mins elapsed.

2023-03-09 23:04:43 : Adding BHF_F_Hea13168894_BHF_F_Hea13168510 to GeneExpressionMatrix for Chr (2 of 23)!, 19.86 mins elapsed.

2023-03-09 23:04:46 : Adding BHF_F_Hea13168894_BHF_F_Hea13168510 to GeneExpressionMatrix for Chr (3 of 23)!, 19.911 mins elapsed.

2023-03-09 23:04:50 : Adding BHF_F_Hea13168894_BHF_F_Hea13168510 to GeneExpressionMatrix for Chr (4 of 23)!, 19.98 mins elapsed.

2023-03-09 23:04:53 : Adding BHF_F_Hea13168894_BHF_F_Hea13168510 to GeneExpressionMatrix for Chr (5 of 23)!, 20.029 mins elapsed.

2023-03-09 23:04:56 : Adding BHF_F_Hea13168894_BHF_F_Hea13168510 to GeneExpressionMatrix for Chr (6 of 23)!, 20.078 mins elapsed.

2023-03-09 23:05:01 : Adding BHF_F_Hea13168894_BHF_F_Hea13168510 to GeneExpressionMatrix for Chr (7 of 23)!, 20.152 mins elapsed.

2023-03-09 23:05:04 : Adding BHF_F_Hea13168894_BHF_F_Hea13168510 to GeneExpressionMatrix for Chr (8 of 23)!, 20.202 mins elapsed.

2023-03-09 23:05:07 : Adding BHF_F_Hea13168894_BHF_F_Hea13168510 to GeneExpressionMatrix for Chr (9 of 23)!, 20.261 mins elapsed.

2023-03-09 23:05:13 : Adding BHF_F_Hea13168894_BHF_F_Hea13168510 to GeneExpressionMatrix for Chr (10 of 23)!, 20.347 mins elapsed.

2023-03-09 23:05:16 : Adding BHF_F_Hea13168894_BHF_F_Hea13168510 to GeneExpressionMatrix for Chr (11 of 23)!, 20.397 mins elapsed.

2023-03-09 23:05:19 : Adding BHF_F_Hea13168894_BHF_F_Hea13168510 to GeneExpressionMatrix for Chr (12 of 23)!, 20.449 mins elapsed.

2023-03-09 23:05:23 : Adding BHF_F_Hea13168894_BHF_F_Hea13168510 to GeneExpressionMatrix for Chr (13 of 23)!, 20.518 mins elapsed.

2023-03-09 23:05:26 : Adding BHF_F_Hea13168894_BHF_F_Hea13168510 to GeneExpressionMatrix for Chr (14 of 23)!, 20.567 mins elapsed.

2023-03-09 23:05:29 : Adding BHF_F_Hea13168894_BHF_F_Hea13168510 to GeneExpressionMatrix for Chr (15 of 23)!, 20.621 mins elapsed.

2023-03-09 23:05:33 : Adding BHF_F_Hea13168894_BHF_F_Hea13168510 to GeneExpressionMatrix for Chr (16 of 23)!, 20.691 mins elapsed.

2023-03-09 23:05:36 : Adding BHF_F_Hea13168894_BHF_F_Hea13168510 to GeneExpressionMatrix for Chr (17 of 23)!, 20.741 mins elapsed.

2023-03-09 23:05:40 : Adding BHF_F_Hea13168894_BHF_F_Hea13168510 to GeneExpressionMatrix for Chr (18 of 23)!, 20.802 mins elapsed.

2023-03-09 23:05:44 : Adding BHF_F_Hea13168894_BHF_F_Hea13168510 to GeneExpressionMatrix for Chr (19 of 23)!, 20.868 mins elapsed.

2023-03-09 23:05:47 : Adding BHF_F_Hea13168894_BHF_F_Hea13168510 to GeneExpressionMatrix for Chr (20 of 23)!, 20.929 mins elapsed.

2023-03-09 23:05:50 : Adding BHF_F_Hea13168894_BHF_F_Hea13168510 to GeneExpressionMatrix for Chr (21 of 23)!, 20.979 mins elapsed.

2023-03-09 23:05:55 : Adding BHF_F_Hea13168894_BHF_F_Hea13168510 to GeneExpressionMatrix for Chr (22 of 23)!, 21.056 mins elapsed.

2023-03-09 23:05:58 : Adding BHF_F_Hea13168894_BHF_F_Hea13168510 to GeneExpressionMatrix for Chr (23 of 23)!, 21.103 mins elapsed.

2023-03-09 23:06:06 : Adding 7089STDY13216927_BHF_F_Hea13242534 to GeneExpressionMatrix for Chr (1 of 23)!, 21.246 mins elapsed.

2023-03-09 23:06:09 : Adding 7089STDY13216927_BHF_F_Hea13242534 to GeneExpressionMatrix for Chr (2 of 23)!, 21.296 mins elapsed.

2023-03-09 23:06:12 : Adding 7089STDY13216927_BHF_F_Hea13242534 to GeneExpressionMatrix for Chr (3 of 23)!, 21.344 mins elapsed.

2023-03-09 23:06:16 : Adding 7089STDY13216927_BHF_F_Hea13242534 to GeneExpressionMatrix for Chr (4 of 23)!, 21.412 mins elapsed.

2023-03-09 23:06:19 : Adding 7089STDY13216927_BHF_F_Hea13242534 to GeneExpressionMatrix for Chr (5 of 23)!, 21.46 mins elapsed.

2023-03-09 23:06:22 : Adding 7089STDY13216927_BHF_F_Hea13242534 to GeneExpressionMatrix for Chr (6 of 23)!, 21.51 mins elapsed.

2023-03-09 23:06:27 : Adding 7089STDY13216927_BHF_F_Hea13242534 to GeneExpressionMatrix for Chr (7 of 23)!, 21.582 mins elapsed.

2023-03-09 23:06:30 : Adding 7089STDY13216927_BHF_F_Hea13242534 to GeneExpressionMatrix for Chr (8 of 23)!, 21.631 mins elapsed.

2023-03-09 23:06:32 : Adding 7089STDY13216927_BHF_F_Hea13242534 to GeneExpressionMatrix for Chr (9 of 23)!, 21.678 mins elapsed.

2023-03-09 23:06:36 : Adding 7089STDY13216927_BHF_F_Hea13242534 to GeneExpressionMatrix for Chr (10 of 23)!, 21.745 mins elapsed.

2023-03-09 23:06:39 : Adding 7089STDY13216927_BHF_F_Hea13242534 to GeneExpressionMatrix for Chr (11 of 23)!, 21.795 mins elapsed.

2023-03-09 23:06:42 : Adding 7089STDY13216927_BHF_F_Hea13242534 to GeneExpressionMatrix for Chr (12 of 23)!, 21.844 mins elapsed.

2023-03-09 23:06:46 : Adding 7089STDY13216927_BHF_F_Hea13242534 to GeneExpressionMatrix for Chr (13 of 23)!, 21.912 mins elapsed.

2023-03-09 23:06:49 : Adding 7089STDY13216927_BHF_F_Hea13242534 to GeneExpressionMatrix for Chr (14 of 23)!, 21.96 mins elapsed.

2023-03-09 23:06:52 : Adding 7089STDY13216927_BHF_F_Hea13242534 to GeneExpressionMatrix for Chr (15 of 23)!, 22.008 mins elapsed.

2023-03-09 23:06:56 : Adding 7089STDY13216927_BHF_F_Hea13242534 to GeneExpressionMatrix for Chr (16 of 23)!, 22.076 mins elapsed.

2023-03-09 23:06:59 : Adding 7089STDY13216927_BHF_F_Hea13242534 to GeneExpressionMatrix for Chr (17 of 23)!, 22.13 mins elapsed.

2023-03-09 23:07:03 : Adding 7089STDY13216927_BHF_F_Hea13242534 to GeneExpressionMatrix for Chr (18 of 23)!, 22.185 mins elapsed.

2023-03-09 23:07:07 : Adding 7089STDY13216927_BHF_F_Hea13242534 to GeneExpressionMatrix for Chr (19 of 23)!, 22.25 mins elapsed.

2023-03-09 23:07:10 : Adding 7089STDY13216927_BHF_F_Hea13242534 to GeneExpressionMatrix for Chr (20 of 23)!, 22.298 mins elapsed.

2023-03-09 23:07:13 : Adding 7089STDY13216927_BHF_F_Hea13242534 to GeneExpressionMatrix for Chr (21 of 23)!, 22.347 mins elapsed.

2023-03-09 23:07:17 : Adding 7089STDY13216927_BHF_F_Hea13242534 to GeneExpressionMatrix for Chr (22 of 23)!, 22.42 mins elapsed.

2023-03-09 23:07:20 : Adding 7089STDY13216927_BHF_F_Hea13242534 to GeneExpressionMatrix for Chr (23 of 23)!, 22.466 mins elapsed.

2023-03-09 23:07:27 : Adding BHF_F_Hea13168895_BHF_F_Hea13168511 to GeneExpressionMatrix for Chr (1 of 23)!, 22.584 mins elapsed.

2023-03-09 23:07:30 : Adding BHF_F_Hea13168895_BHF_F_Hea13168511 to GeneExpressionMatrix for Chr (2 of 23)!, 22.638 mins elapsed.

2023-03-09 23:07:33 : Adding BHF_F_Hea13168895_BHF_F_Hea13168511 to GeneExpressionMatrix for Chr (3 of 23)!, 22.688 mins elapsed.

2023-03-09 23:07:37 : Adding BHF_F_Hea13168895_BHF_F_Hea13168511 to GeneExpressionMatrix for Chr (4 of 23)!, 22.759 mins elapsed.

2023-03-09 23:07:40 : Adding BHF_F_Hea13168895_BHF_F_Hea13168511 to GeneExpressionMatrix for Chr (5 of 23)!, 22.812 mins elapsed.

2023-03-09 23:07:43 : Adding BHF_F_Hea13168895_BHF_F_Hea13168511 to GeneExpressionMatrix for Chr (6 of 23)!, 22.863 mins elapsed.

2023-03-09 23:07:48 : Adding BHF_F_Hea13168895_BHF_F_Hea13168511 to GeneExpressionMatrix for Chr (7 of 23)!, 22.933 mins elapsed.

2023-03-09 23:07:51 : Adding BHF_F_Hea13168895_BHF_F_Hea13168511 to GeneExpressionMatrix for Chr (8 of 23)!, 22.983 mins elapsed.

2023-03-09 23:07:54 : Adding BHF_F_Hea13168895_BHF_F_Hea13168511 to GeneExpressionMatrix for Chr (9 of 23)!, 23.031 mins elapsed.

2023-03-09 23:07:58 : Adding BHF_F_Hea13168895_BHF_F_Hea13168511 to GeneExpressionMatrix for Chr (10 of 23)!, 23.1 mins elapsed.

2023-03-09 23:08:01 : Adding BHF_F_Hea13168895_BHF_F_Hea13168511 to GeneExpressionMatrix for Chr (11 of 23)!, 23.162 mins elapsed.

2023-03-09 23:08:05 : Adding BHF_F_Hea13168895_BHF_F_Hea13168511 to GeneExpressionMatrix for Chr (12 of 23)!, 23.224 mins elapsed.

2023-03-09 23:08:09 : Adding BHF_F_Hea13168895_BHF_F_Hea13168511 to GeneExpressionMatrix for Chr (13 of 23)!, 23.296 mins elapsed.

2023-03-09 23:08:12 : Adding BHF_F_Hea13168895_BHF_F_Hea13168511 to GeneExpressionMatrix for Chr (14 of 23)!, 23.344 mins elapsed.

2023-03-09 23:08:15 : Adding BHF_F_Hea13168895_BHF_F_Hea13168511 to GeneExpressionMatrix for Chr (15 of 23)!, 23.394 mins elapsed.

2023-03-09 23:08:19 : Adding BHF_F_Hea13168895_BHF_F_Hea13168511 to GeneExpressionMatrix for Chr (16 of 23)!, 23.463 mins elapsed.

2023-03-09 23:08:22 : Adding BHF_F_Hea13168895_BHF_F_Hea13168511 to GeneExpressionMatrix for Chr (17 of 23)!, 23.512 mins elapsed.

2023-03-09 23:08:25 : Adding BHF_F_Hea13168895_BHF_F_Hea13168511 to GeneExpressionMatrix for Chr (18 of 23)!, 23.561 mins elapsed.

2023-03-09 23:08:29 : Adding BHF_F_Hea13168895_BHF_F_Hea13168511 to GeneExpressionMatrix for Chr (19 of 23)!, 23.629 mins elapsed.

2023-03-09 23:08:32 : Adding BHF_F_Hea13168895_BHF_F_Hea13168511 to GeneExpressionMatrix for Chr (20 of 23)!, 23.679 mins elapsed.

2023-03-09 23:08:36 : Adding BHF_F_Hea13168895_BHF_F_Hea13168511 to GeneExpressionMatrix for Chr (21 of 23)!, 23.731 mins elapsed.

2023-03-09 23:08:40 : Adding BHF_F_Hea13168895_BHF_F_Hea13168511 to GeneExpressionMatrix for Chr (22 of 23)!, 23.799 mins elapsed.

2023-03-09 23:08:42 : Adding BHF_F_Hea13168895_BHF_F_Hea13168511 to GeneExpressionMatrix for Chr (23 of 23)!, 23.846 mins elapsed.

2023-03-09 23:08:49 : Adding BHF_F_Hea11933673_BHF_F_Hea11596626 to GeneExpressionMatrix for Chr (1 of 23)!, 23.957 mins elapsed.

2023-03-09 23:08:52 : Adding BHF_F_Hea11933673_BHF_F_Hea11596626 to GeneExpressionMatrix for Chr (2 of 23)!, 24.009 mins elapsed.

2023-03-09 23:08:55 : Adding BHF_F_Hea11933673_BHF_F_Hea11596626 to GeneExpressionMatrix for Chr (3 of 23)!, 24.059 mins elapsed.

2023-03-09 23:08:59 : Adding BHF_F_Hea11933673_BHF_F_Hea11596626 to GeneExpressionMatrix for Chr (4 of 23)!, 24.127 mins elapsed.

2023-03-09 23:09:02 : Adding BHF_F_Hea11933673_BHF_F_Hea11596626 to GeneExpressionMatrix for Chr (5 of 23)!, 24.175 mins elapsed.

2023-03-09 23:09:05 : Adding BHF_F_Hea11933673_BHF_F_Hea11596626 to GeneExpressionMatrix for Chr (6 of 23)!, 24.223 mins elapsed.

2023-03-09 23:09:09 : Adding BHF_F_Hea11933673_BHF_F_Hea11596626 to GeneExpressionMatrix for Chr (7 of 23)!, 24.291 mins elapsed.

2023-03-09 23:09:12 : Adding BHF_F_Hea11933673_BHF_F_Hea11596626 to GeneExpressionMatrix for Chr (8 of 23)!, 24.342 mins elapsed.

2023-03-09 23:09:15 : Adding BHF_F_Hea11933673_BHF_F_Hea11596626 to GeneExpressionMatrix for Chr (9 of 23)!, 24.389 mins elapsed.

2023-03-09 23:09:19 : Adding BHF_F_Hea11933673_BHF_F_Hea11596626 to GeneExpressionMatrix for Chr (10 of 23)!, 24.455 mins elapsed.

2023-03-09 23:09:22 : Adding BHF_F_Hea11933673_BHF_F_Hea11596626 to GeneExpressionMatrix for Chr (11 of 23)!, 24.503 mins elapsed.

2023-03-09 23:09:25 : Adding BHF_F_Hea11933673_BHF_F_Hea11596626 to GeneExpressionMatrix for Chr (12 of 23)!, 24.552 mins elapsed.

2023-03-09 23:09:29 : Adding BHF_F_Hea11933673_BHF_F_Hea11596626 to GeneExpressionMatrix for Chr (13 of 23)!, 24.619 mins elapsed.

2023-03-09 23:09:32 : Adding BHF_F_Hea11933673_BHF_F_Hea11596626 to GeneExpressionMatrix for Chr (14 of 23)!, 24.665 mins elapsed.

2023-03-09 23:09:34 : Adding BHF_F_Hea11933673_BHF_F_Hea11596626 to GeneExpressionMatrix for Chr (15 of 23)!, 24.713 mins elapsed.

2023-03-09 23:09:38 : Adding BHF_F_Hea11933673_BHF_F_Hea11596626 to GeneExpressionMatrix for Chr (16 of 23)!, 24.78 mins elapsed.

2023-03-09 23:09:41 : Adding BHF_F_Hea11933673_BHF_F_Hea11596626 to GeneExpressionMatrix for Chr (17 of 23)!, 24.828 mins elapsed.

2023-03-09 23:09:45 : Adding BHF_F_Hea11933673_BHF_F_Hea11596626 to GeneExpressionMatrix for Chr (18 of 23)!, 24.887 mins elapsed.

2023-03-09 23:09:49 : Adding BHF_F_Hea11933673_BHF_F_Hea11596626 to GeneExpressionMatrix for Chr (19 of 23)!, 24.954 mins elapsed.

2023-03-09 23:09:52 : Adding BHF_F_Hea11933673_BHF_F_Hea11596626 to GeneExpressionMatrix for Chr (20 of 23)!, 25.003 mins elapsed.

2023-03-09 23:09:55 : Adding BHF_F_Hea11933673_BHF_F_Hea11596626 to GeneExpressionMatrix for Chr (21 of 23)!, 25.049 mins elapsed.

2023-03-09 23:09:59 : Adding BHF_F_Hea11933673_BHF_F_Hea11596626 to GeneExpressionMatrix for Chr (22 of 23)!, 25.115 mins elapsed.

2023-03-09 23:10:01 : Adding BHF_F_Hea11933673_BHF_F_Hea11596626 to GeneExpressionMatrix for Chr (23 of 23)!, 25.163 mins elapsed.

2023-03-09 23:10:09 : Adding BHF_F_Hea13168899_BHF_F_Hea13168515 to GeneExpressionMatrix for Chr (1 of 23)!, 25.282 mins elapsed.

2023-03-09 23:10:12 : Adding BHF_F_Hea13168899_BHF_F_Hea13168515 to GeneExpressionMatrix for Chr (2 of 23)!, 25.338 mins elapsed.

2023-03-09 23:10:15 : Adding BHF_F_Hea13168899_BHF_F_Hea13168515 to GeneExpressionMatrix for Chr (3 of 23)!, 25.393 mins elapsed.

2023-03-09 23:10:20 : Adding BHF_F_Hea13168899_BHF_F_Hea13168515 to GeneExpressionMatrix for Chr (4 of 23)!, 25.466 mins elapsed.

2023-03-09 23:10:23 : Adding BHF_F_Hea13168899_BHF_F_Hea13168515 to GeneExpressionMatrix for Chr (5 of 23)!, 25.517 mins elapsed.

2023-03-09 23:10:26 : Adding BHF_F_Hea13168899_BHF_F_Hea13168515 to GeneExpressionMatrix for Chr (6 of 23)!, 25.569 mins elapsed.

2023-03-09 23:10:31 : Adding BHF_F_Hea13168899_BHF_F_Hea13168515 to GeneExpressionMatrix for Chr (7 of 23)!, 25.657 mins elapsed.

2023-03-09 23:10:35 : Adding BHF_F_Hea13168899_BHF_F_Hea13168515 to GeneExpressionMatrix for Chr (8 of 23)!, 25.72 mins elapsed.

2023-03-09 23:10:38 : Adding BHF_F_Hea13168899_BHF_F_Hea13168515 to GeneExpressionMatrix for Chr (9 of 23)!, 25.772 mins elapsed.

2023-03-09 23:10:43 : Adding BHF_F_Hea13168899_BHF_F_Hea13168515 to GeneExpressionMatrix for Chr (10 of 23)!, 25.85 mins elapsed.

2023-03-09 23:10:46 : Adding BHF_F_Hea13168899_BHF_F_Hea13168515 to GeneExpressionMatrix for Chr (11 of 23)!, 25.902 mins elapsed.

2023-03-09 23:10:49 : Adding BHF_F_Hea13168899_BHF_F_Hea13168515 to GeneExpressionMatrix for Chr (12 of 23)!, 25.954 mins elapsed.

2023-03-09 23:10:53 : Adding BHF_F_Hea13168899_BHF_F_Hea13168515 to GeneExpressionMatrix for Chr (13 of 23)!, 26.026 mins elapsed.

2023-03-09 23:10:56 : Adding BHF_F_Hea13168899_BHF_F_Hea13168515 to GeneExpressionMatrix for Chr (14 of 23)!, 26.075 mins elapsed.

2023-03-09 23:10:59 : Adding BHF_F_Hea13168899_BHF_F_Hea13168515 to GeneExpressionMatrix for Chr (15 of 23)!, 26.126 mins elapsed.

2023-03-09 23:11:03 : Adding BHF_F_Hea13168899_BHF_F_Hea13168515 to GeneExpressionMatrix for Chr (16 of 23)!, 26.196 mins elapsed.

2023-03-09 23:11:06 : Adding BHF_F_Hea13168899_BHF_F_Hea13168515 to GeneExpressionMatrix for Chr (17 of 23)!, 26.246 mins elapsed.

2023-03-09 23:11:10 : Adding BHF_F_Hea13168899_BHF_F_Hea13168515 to GeneExpressionMatrix for Chr (18 of 23)!, 26.306 mins elapsed.

2023-03-09 23:11:15 : Adding BHF_F_Hea13168899_BHF_F_Hea13168515 to GeneExpressionMatrix for Chr (19 of 23)!, 26.391 mins elapsed.

2023-03-09 23:11:19 : Adding BHF_F_Hea13168899_BHF_F_Hea13168515 to GeneExpressionMatrix for Chr (20 of 23)!, 26.45 mins elapsed.

2023-03-09 23:11:22 : Adding BHF_F_Hea13168899_BHF_F_Hea13168515 to GeneExpressionMatrix for Chr (21 of 23)!, 26.499 mins elapsed.

2023-03-09 23:11:26 : Adding BHF_F_Hea13168899_BHF_F_Hea13168515 to GeneExpressionMatrix for Chr (22 of 23)!, 26.569 mins elapsed.

2023-03-09 23:11:29 : Adding BHF_F_Hea13168899_BHF_F_Hea13168515 to GeneExpressionMatrix for Chr (23 of 23)!, 26.616 mins elapsed.

2023-03-09 23:11:37 : Adding BHF_F_Hea11933672_BHF_F_Hea11596625 to GeneExpressionMatrix for Chr (1 of 23)!, 26.749 mins elapsed.

2023-03-09 23:11:40 : Adding BHF_F_Hea11933672_BHF_F_Hea11596625 to GeneExpressionMatrix for Chr (2 of 23)!, 26.808 mins elapsed.

2023-03-09 23:11:43 : Adding BHF_F_Hea11933672_BHF_F_Hea11596625 to GeneExpressionMatrix for Chr (3 of 23)!, 26.857 mins elapsed.

2023-03-09 23:11:48 : Adding BHF_F_Hea11933672_BHF_F_Hea11596625 to GeneExpressionMatrix for Chr (4 of 23)!, 26.931 mins elapsed.

2023-03-09 23:11:51 : Adding BHF_F_Hea11933672_BHF_F_Hea11596625 to GeneExpressionMatrix for Chr (5 of 23)!, 26.989 mins elapsed.

2023-03-09 23:11:55 : Adding BHF_F_Hea11933672_BHF_F_Hea11596625 to GeneExpressionMatrix for Chr (6 of 23)!, 27.05 mins elapsed.

2023-03-09 23:11:59 : Adding BHF_F_Hea11933672_BHF_F_Hea11596625 to GeneExpressionMatrix for Chr (7 of 23)!, 27.123 mins elapsed.

2023-03-09 23:12:02 : Adding BHF_F_Hea11933672_BHF_F_Hea11596625 to GeneExpressionMatrix for Chr (8 of 23)!, 27.171 mins elapsed.

2023-03-09 23:12:05 : Adding BHF_F_Hea11933672_BHF_F_Hea11596625 to GeneExpressionMatrix for Chr (9 of 23)!, 27.217 mins elapsed.

2023-03-09 23:12:09 : Adding BHF_F_Hea11933672_BHF_F_Hea11596625 to GeneExpressionMatrix for Chr (10 of 23)!, 27.285 mins elapsed.

2023-03-09 23:12:12 : Adding BHF_F_Hea11933672_BHF_F_Hea11596625 to GeneExpressionMatrix for Chr (11 of 23)!, 27.332 mins elapsed.

2023-03-09 23:12:14 : Adding BHF_F_Hea11933672_BHF_F_Hea11596625 to GeneExpressionMatrix for Chr (12 of 23)!, 27.379 mins elapsed.

2023-03-09 23:12:19 : Adding BHF_F_Hea11933672_BHF_F_Hea11596625 to GeneExpressionMatrix for Chr (13 of 23)!, 27.447 mins elapsed.

2023-03-09 23:12:21 : Adding BHF_F_Hea11933672_BHF_F_Hea11596625 to GeneExpressionMatrix for Chr (14 of 23)!, 27.493 mins elapsed.

2023-03-09 23:12:24 : Adding BHF_F_Hea11933672_BHF_F_Hea11596625 to GeneExpressionMatrix for Chr (15 of 23)!, 27.541 mins elapsed.

2023-03-09 23:12:28 : Adding BHF_F_Hea11933672_BHF_F_Hea11596625 to GeneExpressionMatrix for Chr (16 of 23)!, 27.609 mins elapsed.

2023-03-09 23:12:31 : Adding BHF_F_Hea11933672_BHF_F_Hea11596625 to GeneExpressionMatrix for Chr (17 of 23)!, 27.657 mins elapsed.

2023-03-09 23:12:34 : Adding BHF_F_Hea11933672_BHF_F_Hea11596625 to GeneExpressionMatrix for Chr (18 of 23)!, 27.705 mins elapsed.

2023-03-09 23:12:38 : Adding BHF_F_Hea11933672_BHF_F_Hea11596625 to GeneExpressionMatrix for Chr (19 of 23)!, 27.772 mins elapsed.

2023-03-09 23:12:41 : Adding BHF_F_Hea11933672_BHF_F_Hea11596625 to GeneExpressionMatrix for Chr (20 of 23)!, 27.821 mins elapsed.

2023-03-09 23:12:44 : Adding BHF_F_Hea11933672_BHF_F_Hea11596625 to GeneExpressionMatrix for Chr (21 of 23)!, 27.867 mins elapsed.

2023-03-09 23:12:48 : Adding BHF_F_Hea11933672_BHF_F_Hea11596625 to GeneExpressionMatrix for Chr (22 of 23)!, 27.932 mins elapsed.

2023-03-09 23:12:50 : Adding BHF_F_Hea11933672_BHF_F_Hea11596625 to GeneExpressionMatrix for Chr (23 of 23)!, 27.978 mins elapsed.

2023-03-09 23:12:59 : Adding BHF_F_Hea13168897_BHF_F_Hea13168513 to GeneExpressionMatrix for Chr (1 of 23)!, 28.118 mins elapsed.

2023-03-09 23:13:02 : Adding BHF_F_Hea13168897_BHF_F_Hea13168513 to GeneExpressionMatrix for Chr (2 of 23)!, 28.172 mins elapsed.

2023-03-09 23:13:05 : Adding BHF_F_Hea13168897_BHF_F_Hea13168513 to GeneExpressionMatrix for Chr (3 of 23)!, 28.224 mins elapsed.

2023-03-09 23:13:09 : Adding BHF_F_Hea13168897_BHF_F_Hea13168513 to GeneExpressionMatrix for Chr (4 of 23)!, 28.296 mins elapsed.

2023-03-09 23:13:12 : Adding BHF_F_Hea13168897_BHF_F_Hea13168513 to GeneExpressionMatrix for Chr (5 of 23)!, 28.344 mins elapsed.

2023-03-09 23:13:15 : Adding BHF_F_Hea13168897_BHF_F_Hea13168513 to GeneExpressionMatrix for Chr (6 of 23)!, 28.393 mins elapsed.

2023-03-09 23:13:19 : Adding BHF_F_Hea13168897_BHF_F_Hea13168513 to GeneExpressionMatrix for Chr (7 of 23)!, 28.463 mins elapsed.

2023-03-09 23:13:22 : Adding BHF_F_Hea13168897_BHF_F_Hea13168513 to GeneExpressionMatrix for Chr (8 of 23)!, 28.513 mins elapsed.

2023-03-09 23:13:25 : Adding BHF_F_Hea13168897_BHF_F_Hea13168513 to GeneExpressionMatrix for Chr (9 of 23)!, 28.562 mins elapsed.

2023-03-09 23:13:30 : Adding BHF_F_Hea13168897_BHF_F_Hea13168513 to GeneExpressionMatrix for Chr (10 of 23)!, 28.636 mins elapsed.

2023-03-09 23:13:33 : Adding BHF_F_Hea13168897_BHF_F_Hea13168513 to GeneExpressionMatrix for Chr (11 of 23)!, 28.686 mins elapsed.

2023-03-09 23:13:36 : Adding BHF_F_Hea13168897_BHF_F_Hea13168513 to GeneExpressionMatrix for Chr (12 of 23)!, 28.735 mins elapsed.

2023-03-09 23:13:40 : Adding BHF_F_Hea13168897_BHF_F_Hea13168513 to GeneExpressionMatrix for Chr (13 of 23)!, 28.807 mins elapsed.

2023-03-09 23:13:43 : Adding BHF_F_Hea13168897_BHF_F_Hea13168513 to GeneExpressionMatrix for Chr (14 of 23)!, 28.856 mins elapsed.

2023-03-09 23:13:46 : Adding BHF_F_Hea13168897_BHF_F_Hea13168513 to GeneExpressionMatrix for Chr (15 of 23)!, 28.906 mins elapsed.

2023-03-09 23:13:50 : Adding BHF_F_Hea13168897_BHF_F_Hea13168513 to GeneExpressionMatrix for Chr (16 of 23)!, 28.979 mins elapsed.

2023-03-09 23:13:53 : Adding BHF_F_Hea13168897_BHF_F_Hea13168513 to GeneExpressionMatrix for Chr (17 of 23)!, 29.03 mins elapsed.

2023-03-09 23:13:57 : Adding BHF_F_Hea13168897_BHF_F_Hea13168513 to GeneExpressionMatrix for Chr (18 of 23)!, 29.081 mins elapsed.

2023-03-09 23:14:01 : Adding BHF_F_Hea13168897_BHF_F_Hea13168513 to GeneExpressionMatrix for Chr (19 of 23)!, 29.149 mins elapsed.

2023-03-09 23:14:04 : Adding BHF_F_Hea13168897_BHF_F_Hea13168513 to GeneExpressionMatrix for Chr (20 of 23)!, 29.201 mins elapsed.

2023-03-09 23:14:07 : Adding BHF_F_Hea13168897_BHF_F_Hea13168513 to GeneExpressionMatrix for Chr (21 of 23)!, 29.247 mins elapsed.

2023-03-09 23:14:11 : Adding BHF_F_Hea13168897_BHF_F_Hea13168513 to GeneExpressionMatrix for Chr (22 of 23)!, 29.324 mins elapsed.

2023-03-09 23:14:15 : Adding BHF_F_Hea13168897_BHF_F_Hea13168513 to GeneExpressionMatrix for Chr (23 of 23)!, 29.383 mins elapsed.

2023-03-09 23:14:21 : Adding BHF_F_Hea11933670_BHF_F_Hea11596623 to GeneExpressionMatrix for Chr (1 of 23)!, 29.494 mins elapsed.

2023-03-09 23:14:24 : Adding BHF_F_Hea11933670_BHF_F_Hea11596623 to GeneExpressionMatrix for Chr (2 of 23)!, 29.546 mins elapsed.

2023-03-09 23:14:27 : Adding BHF_F_Hea11933670_BHF_F_Hea11596623 to GeneExpressionMatrix for Chr (3 of 23)!, 29.594 mins elapsed.

2023-03-09 23:14:31 : Adding BHF_F_Hea11933670_BHF_F_Hea11596623 to GeneExpressionMatrix for Chr (4 of 23)!, 29.663 mins elapsed.

2023-03-09 23:14:34 : Adding BHF_F_Hea11933670_BHF_F_Hea11596623 to GeneExpressionMatrix for Chr (5 of 23)!, 29.71 mins elapsed.

2023-03-09 23:14:37 : Adding BHF_F_Hea11933670_BHF_F_Hea11596623 to GeneExpressionMatrix for Chr (6 of 23)!, 29.758 mins elapsed.

2023-03-09 23:14:41 : Adding BHF_F_Hea11933670_BHF_F_Hea11596623 to GeneExpressionMatrix for Chr (7 of 23)!, 29.825 mins elapsed.

2023-03-09 23:14:44 : Adding BHF_F_Hea11933670_BHF_F_Hea11596623 to GeneExpressionMatrix for Chr (8 of 23)!, 29.876 mins elapsed.

2023-03-09 23:14:47 : Adding BHF_F_Hea11933670_BHF_F_Hea11596623 to GeneExpressionMatrix for Chr (9 of 23)!, 29.925 mins elapsed.

2023-03-09 23:14:51 : Adding BHF_F_Hea11933670_BHF_F_Hea11596623 to GeneExpressionMatrix for Chr (10 of 23)!, 29.992 mins elapsed.

2023-03-09 23:14:55 : Adding BHF_F_Hea11933670_BHF_F_Hea11596623 to GeneExpressionMatrix for Chr (11 of 23)!, 30.049 mins elapsed.

2023-03-09 23:14:58 : Adding BHF_F_Hea11933670_BHF_F_Hea11596623 to GeneExpressionMatrix for Chr (12 of 23)!, 30.108 mins elapsed.

2023-03-09 23:15:03 : Adding BHF_F_Hea11933670_BHF_F_Hea11596623 to GeneExpressionMatrix for Chr (13 of 23)!, 30.191 mins elapsed.

2023-03-09 23:15:06 : Adding BHF_F_Hea11933670_BHF_F_Hea11596623 to GeneExpressionMatrix for Chr (14 of 23)!, 30.24 mins elapsed.

2023-03-09 23:15:09 : Adding BHF_F_Hea11933670_BHF_F_Hea11596623 to GeneExpressionMatrix for Chr (15 of 23)!, 30.287 mins elapsed.

2023-03-09 23:15:14 : Adding BHF_F_Hea11933670_BHF_F_Hea11596623 to GeneExpressionMatrix for Chr (16 of 23)!, 30.372 mins elapsed.

2023-03-09 23:15:17 : Adding BHF_F_Hea11933670_BHF_F_Hea11596623 to GeneExpressionMatrix for Chr (17 of 23)!, 30.429 mins elapsed.

2023-03-09 23:15:21 : Adding BHF_F_Hea11933670_BHF_F_Hea11596623 to GeneExpressionMatrix for Chr (18 of 23)!, 30.488 mins elapsed.

2023-03-09 23:15:26 : Adding BHF_F_Hea11933670_BHF_F_Hea11596623 to GeneExpressionMatrix for Chr (19 of 23)!, 30.57 mins elapsed.

2023-03-09 23:15:29 : Adding BHF_F_Hea11933670_BHF_F_Hea11596623 to GeneExpressionMatrix for Chr (20 of 23)!, 30.618 mins elapsed.

2023-03-09 23:15:32 : Adding BHF_F_Hea11933670_BHF_F_Hea11596623 to GeneExpressionMatrix for Chr (21 of 23)!, 30.669 mins elapsed.

2023-03-09 23:15:36 : Adding BHF_F_Hea11933670_BHF_F_Hea11596623 to GeneExpressionMatrix for Chr (22 of 23)!, 30.738 mins elapsed.

2023-03-09 23:15:39 : Adding BHF_F_Hea11933670_BHF_F_Hea11596623 to GeneExpressionMatrix for Chr (23 of 23)!, 30.786 mins elapsed.

2023-03-09 23:15:46 : Adding 7089STDY13216926_BHF_F_Hea13242533 to GeneExpressionMatrix for Chr (1 of 23)!, 30.906 mins elapsed.

2023-03-09 23:15:49 : Adding 7089STDY13216926_BHF_F_Hea13242533 to GeneExpressionMatrix for Chr (2 of 23)!, 30.956 mins elapsed.

2023-03-09 23:15:52 : Adding 7089STDY13216926_BHF_F_Hea13242533 to GeneExpressionMatrix for Chr (3 of 23)!, 31.006 mins elapsed.

2023-03-09 23:15:56 : Adding 7089STDY13216926_BHF_F_Hea13242533 to GeneExpressionMatrix for Chr (4 of 23)!, 31.073 mins elapsed.

2023-03-09 23:15:59 : Adding 7089STDY13216926_BHF_F_Hea13242533 to GeneExpressionMatrix for Chr (5 of 23)!, 31.12 mins elapsed.

2023-03-09 23:16:02 : Adding 7089STDY13216926_BHF_F_Hea13242533 to GeneExpressionMatrix for Chr (6 of 23)!, 31.167 mins elapsed.

2023-03-09 23:16:06 : Adding 7089STDY13216926_BHF_F_Hea13242533 to GeneExpressionMatrix for Chr (7 of 23)!, 31.234 mins elapsed.

2023-03-09 23:16:09 : Adding 7089STDY13216926_BHF_F_Hea13242533 to GeneExpressionMatrix for Chr (8 of 23)!, 31.281 mins elapsed.

2023-03-09 23:16:11 : Adding 7089STDY13216926_BHF_F_Hea13242533 to GeneExpressionMatrix for Chr (9 of 23)!, 31.327 mins elapsed.

2023-03-09 23:16:15 : Adding 7089STDY13216926_BHF_F_Hea13242533 to GeneExpressionMatrix for Chr (10 of 23)!, 31.392 mins elapsed.

2023-03-09 23:16:18 : Adding 7089STDY13216926_BHF_F_Hea13242533 to GeneExpressionMatrix for Chr (11 of 23)!, 31.439 mins elapsed.

2023-03-09 23:16:21 : Adding 7089STDY13216926_BHF_F_Hea13242533 to GeneExpressionMatrix for Chr (12 of 23)!, 31.486 mins elapsed.

2023-03-09 23:16:25 : Adding 7089STDY13216926_BHF_F_Hea13242533 to GeneExpressionMatrix for Chr (13 of 23)!, 31.552 mins elapsed.

2023-03-09 23:16:28 : Adding 7089STDY13216926_BHF_F_Hea13242533 to GeneExpressionMatrix for Chr (14 of 23)!, 31.597 mins elapsed.

2023-03-09 23:16:30 : Adding 7089STDY13216926_BHF_F_Hea13242533 to GeneExpressionMatrix for Chr (15 of 23)!, 31.643 mins elapsed.

2023-03-09 23:16:34 : Adding 7089STDY13216926_BHF_F_Hea13242533 to GeneExpressionMatrix for Chr (16 of 23)!, 31.71 mins elapsed.

2023-03-09 23:16:37 : Adding 7089STDY13216926_BHF_F_Hea13242533 to GeneExpressionMatrix for Chr (17 of 23)!, 31.757 mins elapsed.

2023-03-09 23:16:40 : Adding 7089STDY13216926_BHF_F_Hea13242533 to GeneExpressionMatrix for Chr (18 of 23)!, 31.804 mins elapsed.

2023-03-09 23:16:44 : Adding 7089STDY13216926_BHF_F_Hea13242533 to GeneExpressionMatrix for Chr (19 of 23)!, 31.868 mins elapsed.

2023-03-09 23:16:47 : Adding 7089STDY13216926_BHF_F_Hea13242533 to GeneExpressionMatrix for Chr (20 of 23)!, 31.915 mins elapsed.

2023-03-09 23:16:49 : Adding 7089STDY13216926_BHF_F_Hea13242533 to GeneExpressionMatrix for Chr (21 of 23)!, 31.96 mins elapsed.

2023-03-09 23:16:53 : Adding 7089STDY13216926_BHF_F_Hea13242533 to GeneExpressionMatrix for Chr (22 of 23)!, 32.028 mins elapsed.

2023-03-09 23:16:56 : Adding 7089STDY13216926_BHF_F_Hea13242533 to GeneExpressionMatrix for Chr (23 of 23)!, 32.075 mins elapsed.

2023-03-09 23:17:03 : Adding BHF_F_Hea11933671_BHF_F_Hea11596624 to GeneExpressionMatrix for Chr (1 of 23)!, 32.181 mins elapsed.

2023-03-09 23:17:06 : Adding BHF_F_Hea11933671_BHF_F_Hea11596624 to GeneExpressionMatrix for Chr (2 of 23)!, 32.232 mins elapsed.

2023-03-09 23:17:09 : Adding BHF_F_Hea11933671_BHF_F_Hea11596624 to GeneExpressionMatrix for Chr (3 of 23)!, 32.281 mins elapsed.

2023-03-09 23:17:13 : Adding BHF_F_Hea11933671_BHF_F_Hea11596624 to GeneExpressionMatrix for Chr (4 of 23)!, 32.358 mins elapsed.

2023-03-09 23:17:16 : Adding BHF_F_Hea11933671_BHF_F_Hea11596624 to GeneExpressionMatrix for Chr (5 of 23)!, 32.407 mins elapsed.

2023-03-09 23:17:19 : Adding BHF_F_Hea11933671_BHF_F_Hea11596624 to GeneExpressionMatrix for Chr (6 of 23)!, 32.463 mins elapsed.

2023-03-09 23:17:24 : Adding BHF_F_Hea11933671_BHF_F_Hea11596624 to GeneExpressionMatrix for Chr (7 of 23)!, 32.533 mins elapsed.

2023-03-09 23:17:27 : Adding BHF_F_Hea11933671_BHF_F_Hea11596624 to GeneExpressionMatrix for Chr (8 of 23)!, 32.583 mins elapsed.

2023-03-09 23:17:30 : Adding BHF_F_Hea11933671_BHF_F_Hea11596624 to GeneExpressionMatrix for Chr (9 of 23)!, 32.631 mins elapsed.

2023-03-09 23:17:34 : Adding BHF_F_Hea11933671_BHF_F_Hea11596624 to GeneExpressionMatrix for Chr (10 of 23)!, 32.697 mins elapsed.

2023-03-09 23:17:36 : Adding BHF_F_Hea11933671_BHF_F_Hea11596624 to GeneExpressionMatrix for Chr (11 of 23)!, 32.744 mins elapsed.

2023-03-09 23:17:39 : Adding BHF_F_Hea11933671_BHF_F_Hea11596624 to GeneExpressionMatrix for Chr (12 of 23)!, 32.792 mins elapsed.

2023-03-09 23:17:43 : Adding BHF_F_Hea11933671_BHF_F_Hea11596624 to GeneExpressionMatrix for Chr (13 of 23)!, 32.859 mins elapsed.

2023-03-09 23:17:46 : Adding BHF_F_Hea11933671_BHF_F_Hea11596624 to GeneExpressionMatrix for Chr (14 of 23)!, 32.906 mins elapsed.

2023-03-09 23:17:49 : Adding BHF_F_Hea11933671_BHF_F_Hea11596624 to GeneExpressionMatrix for Chr (15 of 23)!, 32.954 mins elapsed.

2023-03-09 23:17:53 : Adding BHF_F_Hea11933671_BHF_F_Hea11596624 to GeneExpressionMatrix for Chr (16 of 23)!, 33.021 mins elapsed.

2023-03-09 23:17:56 : Adding BHF_F_Hea11933671_BHF_F_Hea11596624 to GeneExpressionMatrix for Chr (17 of 23)!, 33.072 mins elapsed.

2023-03-09 23:17:59 : Adding BHF_F_Hea11933671_BHF_F_Hea11596624 to GeneExpressionMatrix for Chr (18 of 23)!, 33.121 mins elapsed.

2023-03-09 23:18:03 : Adding BHF_F_Hea11933671_BHF_F_Hea11596624 to GeneExpressionMatrix for Chr (19 of 23)!, 33.189 mins elapsed.

2023-03-09 23:18:06 : Adding BHF_F_Hea11933671_BHF_F_Hea11596624 to GeneExpressionMatrix for Chr (20 of 23)!, 33.237 mins elapsed.

2023-03-09 23:18:09 : Adding BHF_F_Hea11933671_BHF_F_Hea11596624 to GeneExpressionMatrix for Chr (21 of 23)!, 33.285 mins elapsed.

2023-03-09 23:18:13 : Adding BHF_F_Hea11933671_BHF_F_Hea11596624 to GeneExpressionMatrix for Chr (22 of 23)!, 33.352 mins elapsed.

2023-03-09 23:18:16 : Adding BHF_F_Hea11933671_BHF_F_Hea11596624 to GeneExpressionMatrix for Chr (23 of 23)!, 33.398 mins elapsed.

2023-03-09 23:18:22 : Adding BHF_F_Hea11064672_BHF_F_Hea11031825 to GeneExpressionMatrix for Chr (1 of 23)!, 33.506 mins elapsed.

2023-03-09 23:18:25 : Adding BHF_F_Hea11064672_BHF_F_Hea11031825 to GeneExpressionMatrix for Chr (2 of 23)!, 33.558 mins elapsed.

2023-03-09 23:18:29 : Adding BHF_F_Hea11064672_BHF_F_Hea11031825 to GeneExpressionMatrix for Chr (3 of 23)!, 33.614 mins elapsed.

2023-03-09 23:18:33 : Adding BHF_F_Hea11064672_BHF_F_Hea11031825 to GeneExpressionMatrix for Chr (4 of 23)!, 33.682 mins elapsed.

2023-03-09 23:18:35 : Adding BHF_F_Hea11064672_BHF_F_Hea11031825 to GeneExpressionMatrix for Chr (5 of 23)!, 33.729 mins elapsed.

2023-03-09 23:18:38 : Adding BHF_F_Hea11064672_BHF_F_Hea11031825 to GeneExpressionMatrix for Chr (6 of 23)!, 33.777 mins elapsed.

2023-03-09 23:18:42 : Adding BHF_F_Hea11064672_BHF_F_Hea11031825 to GeneExpressionMatrix for Chr (7 of 23)!, 33.844 mins elapsed.

2023-03-09 23:18:45 : Adding BHF_F_Hea11064672_BHF_F_Hea11031825 to GeneExpressionMatrix for Chr (8 of 23)!, 33.897 mins elapsed.

2023-03-09 23:18:48 : Adding BHF_F_Hea11064672_BHF_F_Hea11031825 to GeneExpressionMatrix for Chr (9 of 23)!, 33.944 mins elapsed.

2023-03-09 23:18:52 : Adding BHF_F_Hea11064672_BHF_F_Hea11031825 to GeneExpressionMatrix for Chr (10 of 23)!, 34.012 mins elapsed.

2023-03-09 23:18:55 : Adding BHF_F_Hea11064672_BHF_F_Hea11031825 to GeneExpressionMatrix for Chr (11 of 23)!, 34.06 mins elapsed.

2023-03-09 23:18:58 : Adding BHF_F_Hea11064672_BHF_F_Hea11031825 to GeneExpressionMatrix for Chr (12 of 23)!, 34.113 mins elapsed.

2023-03-09 23:19:03 : Adding BHF_F_Hea11064672_BHF_F_Hea11031825 to GeneExpressionMatrix for Chr (13 of 23)!, 34.181 mins elapsed.

2023-03-09 23:19:05 : Adding BHF_F_Hea11064672_BHF_F_Hea11031825 to GeneExpressionMatrix for Chr (14 of 23)!, 34.227 mins elapsed.

2023-03-09 23:19:08 : Adding BHF_F_Hea11064672_BHF_F_Hea11031825 to GeneExpressionMatrix for Chr (15 of 23)!, 34.273 mins elapsed.

2023-03-09 23:19:12 : Adding BHF_F_Hea11064672_BHF_F_Hea11031825 to GeneExpressionMatrix for Chr (16 of 23)!, 34.338 mins elapsed.

2023-03-09 23:19:15 : Adding BHF_F_Hea11064672_BHF_F_Hea11031825 to GeneExpressionMatrix for Chr (17 of 23)!, 34.385 mins elapsed.

2023-03-09 23:19:18 : Adding BHF_F_Hea11064672_BHF_F_Hea11031825 to GeneExpressionMatrix for Chr (18 of 23)!, 34.432 mins elapsed.

2023-03-09 23:19:22 : Adding BHF_F_Hea11064672_BHF_F_Hea11031825 to GeneExpressionMatrix for Chr (19 of 23)!, 34.499 mins elapsed.

2023-03-09 23:19:25 : Adding BHF_F_Hea11064672_BHF_F_Hea11031825 to GeneExpressionMatrix for Chr (20 of 23)!, 34.548 mins elapsed.

2023-03-09 23:19:27 : Adding BHF_F_Hea11064672_BHF_F_Hea11031825 to GeneExpressionMatrix for Chr (21 of 23)!, 34.594 mins elapsed.

2023-03-09 23:19:32 : Adding BHF_F_Hea11064672_BHF_F_Hea11031825 to GeneExpressionMatrix for Chr (22 of 23)!, 34.668 mins elapsed.

2023-03-09 23:19:35 : Adding BHF_F_Hea11064672_BHF_F_Hea11031825 to GeneExpressionMatrix for Chr (23 of 23)!, 34.715 mins elapsed.

2023-03-09 23:19:41 : Adding 7089STDY13216922_BHF_F_Hea13242529 to GeneExpressionMatrix for Chr (1 of 23)!, 34.818 mins elapsed.

2023-03-09 23:19:44 : Adding 7089STDY13216922_BHF_F_Hea13242529 to GeneExpressionMatrix for Chr (2 of 23)!, 34.867 mins elapsed.

2023-03-09 23:19:47 : Adding 7089STDY13216922_BHF_F_Hea13242529 to GeneExpressionMatrix for Chr (3 of 23)!, 34.914 mins elapsed.

2023-03-09 23:19:51 : Adding 7089STDY13216922_BHF_F_Hea13242529 to GeneExpressionMatrix for Chr (4 of 23)!, 34.996 mins elapsed.

2023-03-09 23:19:55 : Adding 7089STDY13216922_BHF_F_Hea13242529 to GeneExpressionMatrix for Chr (5 of 23)!, 35.055 mins elapsed.

2023-03-09 23:19:58 : Adding 7089STDY13216922_BHF_F_Hea13242529 to GeneExpressionMatrix for Chr (6 of 23)!, 35.113 mins elapsed.

2023-03-09 23:20:02 : Adding 7089STDY13216922_BHF_F_Hea13242529 to GeneExpressionMatrix for Chr (7 of 23)!, 35.179 mins elapsed.

2023-03-09 23:20:05 : Adding 7089STDY13216922_BHF_F_Hea13242529 to GeneExpressionMatrix for Chr (8 of 23)!, 35.226 mins elapsed.

2023-03-09 23:20:08 : Adding 7089STDY13216922_BHF_F_Hea13242529 to GeneExpressionMatrix for Chr (9 of 23)!, 35.275 mins elapsed.

2023-03-09 23:20:12 : Adding 7089STDY13216922_BHF_F_Hea13242529 to GeneExpressionMatrix for Chr (10 of 23)!, 35.342 mins elapsed.

2023-03-09 23:20:15 : Adding 7089STDY13216922_BHF_F_Hea13242529 to GeneExpressionMatrix for Chr (11 of 23)!, 35.392 mins elapsed.

2023-03-09 23:20:18 : Adding 7089STDY13216922_BHF_F_Hea13242529 to GeneExpressionMatrix for Chr (12 of 23)!, 35.438 mins elapsed.

2023-03-09 23:20:22 : Adding 7089STDY13216922_BHF_F_Hea13242529 to GeneExpressionMatrix for Chr (13 of 23)!, 35.505 mins elapsed.

2023-03-09 23:20:25 : Adding 7089STDY13216922_BHF_F_Hea13242529 to GeneExpressionMatrix for Chr (14 of 23)!, 35.562 mins elapsed.

2023-03-09 23:20:28 : Adding 7089STDY13216922_BHF_F_Hea13242529 to GeneExpressionMatrix for Chr (15 of 23)!, 35.609 mins elapsed.

2023-03-09 23:20:32 : Adding 7089STDY13216922_BHF_F_Hea13242529 to GeneExpressionMatrix for Chr (16 of 23)!, 35.674 mins elapsed.

2023-03-09 23:20:35 : Adding 7089STDY13216922_BHF_F_Hea13242529 to GeneExpressionMatrix for Chr (17 of 23)!, 35.72 mins elapsed.

2023-03-09 23:20:38 : Adding 7089STDY13216922_BHF_F_Hea13242529 to GeneExpressionMatrix for Chr (18 of 23)!, 35.768 mins elapsed.

2023-03-09 23:20:42 : Adding 7089STDY13216922_BHF_F_Hea13242529 to GeneExpressionMatrix for Chr (19 of 23)!, 35.834 mins elapsed.

2023-03-09 23:20:45 : Adding 7089STDY13216922_BHF_F_Hea13242529 to GeneExpressionMatrix for Chr (20 of 23)!, 35.892 mins elapsed.

2023-03-09 23:20:48 : Adding 7089STDY13216922_BHF_F_Hea13242529 to GeneExpressionMatrix for Chr (21 of 23)!, 35.942 mins elapsed.

2023-03-09 23:20:52 : Adding 7089STDY13216922_BHF_F_Hea13242529 to GeneExpressionMatrix for Chr (22 of 23)!, 36.008 mins elapsed.

2023-03-09 23:20:55 : Adding 7089STDY13216922_BHF_F_Hea13242529 to GeneExpressionMatrix for Chr (23 of 23)!, 36.063 mins elapsed.

2023-03-09 23:21:02 : Adding BHF_F_Hea13188317_BHF_F_Hea13187619 to GeneExpressionMatrix for Chr (1 of 23)!, 36.168 mins elapsed.

2023-03-09 23:21:05 : Adding BHF_F_Hea13188317_BHF_F_Hea13187619 to GeneExpressionMatrix for Chr (2 of 23)!, 36.216 mins elapsed.

2023-03-09 23:21:07 : Adding BHF_F_Hea13188317_BHF_F_Hea13187619 to GeneExpressionMatrix for Chr (3 of 23)!, 36.263 mins elapsed.

2023-03-09 23:21:11 : Adding BHF_F_Hea13188317_BHF_F_Hea13187619 to GeneExpressionMatrix for Chr (4 of 23)!, 36.329 mins elapsed.

2023-03-09 23:21:14 : Adding BHF_F_Hea13188317_BHF_F_Hea13187619 to GeneExpressionMatrix for Chr (5 of 23)!, 36.378 mins elapsed.

2023-03-09 23:21:17 : Adding BHF_F_Hea13188317_BHF_F_Hea13187619 to GeneExpressionMatrix for Chr (6 of 23)!, 36.427 mins elapsed.

2023-03-09 23:21:22 : Adding BHF_F_Hea13188317_BHF_F_Hea13187619 to GeneExpressionMatrix for Chr (7 of 23)!, 36.509 mins elapsed.

2023-03-09 23:21:26 : Adding BHF_F_Hea13188317_BHF_F_Hea13187619 to GeneExpressionMatrix for Chr (8 of 23)!, 36.564 mins elapsed.

2023-03-09 23:21:28 : Adding BHF_F_Hea13188317_BHF_F_Hea13187619 to GeneExpressionMatrix for Chr (9 of 23)!, 36.61 mins elapsed.

2023-03-09 23:21:32 : Adding BHF_F_Hea13188317_BHF_F_Hea13187619 to GeneExpressionMatrix for Chr (10 of 23)!, 36.679 mins elapsed.

2023-03-09 23:21:35 : Adding BHF_F_Hea13188317_BHF_F_Hea13187619 to GeneExpressionMatrix for Chr (11 of 23)!, 36.726 mins elapsed.

2023-03-09 23:21:38 : Adding BHF_F_Hea13188317_BHF_F_Hea13187619 to GeneExpressionMatrix for Chr (12 of 23)!, 36.773 mins elapsed.

2023-03-09 23:21:42 : Adding BHF_F_Hea13188317_BHF_F_Hea13187619 to GeneExpressionMatrix for Chr (13 of 23)!, 36.841 mins elapsed.

2023-03-09 23:21:45 : Adding BHF_F_Hea13188317_BHF_F_Hea13187619 to GeneExpressionMatrix for Chr (14 of 23)!, 36.888 mins elapsed.

2023-03-09 23:21:48 : Adding BHF_F_Hea13188317_BHF_F_Hea13187619 to GeneExpressionMatrix for Chr (15 of 23)!, 36.936 mins elapsed.

2023-03-09 23:21:52 : Adding BHF_F_Hea13188317_BHF_F_Hea13187619 to GeneExpressionMatrix for Chr (16 of 23)!, 37.003 mins elapsed.

2023-03-09 23:21:55 : Adding BHF_F_Hea13188317_BHF_F_Hea13187619 to GeneExpressionMatrix for Chr (17 of 23)!, 37.05 mins elapsed.

2023-03-09 23:21:58 : Adding BHF_F_Hea13188317_BHF_F_Hea13187619 to GeneExpressionMatrix for Chr (18 of 23)!, 37.098 mins elapsed.

2023-03-09 23:22:02 : Adding BHF_F_Hea13188317_BHF_F_Hea13187619 to GeneExpressionMatrix for Chr (19 of 23)!, 37.167 mins elapsed.

2023-03-09 23:22:05 : Adding BHF_F_Hea13188317_BHF_F_Hea13187619 to GeneExpressionMatrix for Chr (20 of 23)!, 37.214 mins elapsed.

2023-03-09 23:22:07 : Adding BHF_F_Hea13188317_BHF_F_Hea13187619 to GeneExpressionMatrix for Chr (21 of 23)!, 37.26 mins elapsed.

2023-03-09 23:22:11 : Adding BHF_F_Hea13188317_BHF_F_Hea13187619 to GeneExpressionMatrix for Chr (22 of 23)!, 37.325 mins elapsed.

2023-03-09 23:22:14 : Adding BHF_F_Hea13188317_BHF_F_Hea13187619 to GeneExpressionMatrix for Chr (23 of 23)!, 37.372 mins elapsed.

2023-03-09 23:22:20 : Adding BHF_F_Hea13168893_BHF_F_Hea13168509 to GeneExpressionMatrix for Chr (1 of 23)!, 37.476 mins elapsed.

2023-03-09 23:22:23 : Adding BHF_F_Hea13168893_BHF_F_Hea13168509 to GeneExpressionMatrix for Chr (2 of 23)!, 37.525 mins elapsed.

2023-03-09 23:22:26 : Adding BHF_F_Hea13168893_BHF_F_Hea13168509 to GeneExpressionMatrix for Chr (3 of 23)!, 37.574 mins elapsed.

2023-03-09 23:22:30 : Adding BHF_F_Hea13168893_BHF_F_Hea13168509 to GeneExpressionMatrix for Chr (4 of 23)!, 37.642 mins elapsed.

2023-03-09 23:22:33 : Adding BHF_F_Hea13168893_BHF_F_Hea13168509 to GeneExpressionMatrix for Chr (5 of 23)!, 37.689 mins elapsed.

2023-03-09 23:22:36 : Adding BHF_F_Hea13168893_BHF_F_Hea13168509 to GeneExpressionMatrix for Chr (6 of 23)!, 37.736 mins elapsed.

2023-03-09 23:22:40 : Adding BHF_F_Hea13168893_BHF_F_Hea13168509 to GeneExpressionMatrix for Chr (7 of 23)!, 37.803 mins elapsed.

2023-03-09 23:22:43 : Adding BHF_F_Hea13168893_BHF_F_Hea13168509 to GeneExpressionMatrix for Chr (8 of 23)!, 37.85 mins elapsed.

2023-03-09 23:22:46 : Adding BHF_F_Hea13168893_BHF_F_Hea13168509 to GeneExpressionMatrix for Chr (9 of 23)!, 37.898 mins elapsed.

2023-03-09 23:22:50 : Adding BHF_F_Hea13168893_BHF_F_Hea13168509 to GeneExpressionMatrix for Chr (10 of 23)!, 37.965 mins elapsed.

2023-03-09 23:22:52 : Adding BHF_F_Hea13168893_BHF_F_Hea13168509 to GeneExpressionMatrix for Chr (11 of 23)!, 38.013 mins elapsed.

2023-03-09 23:22:55 : Adding BHF_F_Hea13168893_BHF_F_Hea13168509 to GeneExpressionMatrix for Chr (12 of 23)!, 38.06 mins elapsed.

2023-03-09 23:22:59 : Adding BHF_F_Hea13168893_BHF_F_Hea13168509 to GeneExpressionMatrix for Chr (13 of 23)!, 38.129 mins elapsed.

2023-03-09 23:23:02 : Adding BHF_F_Hea13168893_BHF_F_Hea13168509 to GeneExpressionMatrix for Chr (14 of 23)!, 38.175 mins elapsed.

2023-03-09 23:23:05 : Adding BHF_F_Hea13168893_BHF_F_Hea13168509 to GeneExpressionMatrix for Chr (15 of 23)!, 38.223 mins elapsed.

2023-03-09 23:23:09 : Adding BHF_F_Hea13168893_BHF_F_Hea13168509 to GeneExpressionMatrix for Chr (16 of 23)!, 38.29 mins elapsed.

2023-03-09 23:23:12 : Adding BHF_F_Hea13168893_BHF_F_Hea13168509 to GeneExpressionMatrix for Chr (17 of 23)!, 38.338 mins elapsed.

2023-03-09 23:23:15 : Adding BHF_F_Hea13168893_BHF_F_Hea13168509 to GeneExpressionMatrix for Chr (18 of 23)!, 38.388 mins elapsed.

2023-03-09 23:23:19 : Adding BHF_F_Hea13168893_BHF_F_Hea13168509 to GeneExpressionMatrix for Chr (19 of 23)!, 38.453 mins elapsed.

2023-03-09 23:23:22 : Adding BHF_F_Hea13168893_BHF_F_Hea13168509 to GeneExpressionMatrix for Chr (20 of 23)!, 38.5 mins elapsed.

2023-03-09 23:23:25 : Adding BHF_F_Hea13168893_BHF_F_Hea13168509 to GeneExpressionMatrix for Chr (21 of 23)!, 38.548 mins elapsed.

2023-03-09 23:23:29 : Adding BHF_F_Hea13168893_BHF_F_Hea13168509 to GeneExpressionMatrix for Chr (22 of 23)!, 38.617 mins elapsed.

2023-03-09 23:23:32 : Adding BHF_F_Hea13168893_BHF_F_Hea13168509 to GeneExpressionMatrix for Chr (23 of 23)!, 38.664 mins elapsed.

2023-03-09 23:23:38 : Adding BHF_F_Hea11933674_BHF_F_Hea11596627 to GeneExpressionMatrix for Chr (1 of 23)!, 38.772 mins elapsed.

2023-03-09 23:23:41 : Adding BHF_F_Hea11933674_BHF_F_Hea11596627 to GeneExpressionMatrix for Chr (2 of 23)!, 38.83 mins elapsed.

2023-03-09 23:23:45 : Adding BHF_F_Hea11933674_BHF_F_Hea11596627 to GeneExpressionMatrix for Chr (3 of 23)!, 38.891 mins elapsed.

2023-03-09 23:23:50 : Adding BHF_F_Hea11933674_BHF_F_Hea11596627 to GeneExpressionMatrix for Chr (4 of 23)!, 38.965 mins elapsed.

2023-03-09 23:23:52 : Adding BHF_F_Hea11933674_BHF_F_Hea11596627 to GeneExpressionMatrix for Chr (5 of 23)!, 39.013 mins elapsed.

2023-03-09 23:23:55 : Adding BHF_F_Hea11933674_BHF_F_Hea11596627 to GeneExpressionMatrix for Chr (6 of 23)!, 39.061 mins elapsed.

2023-03-09 23:23:59 : Adding BHF_F_Hea11933674_BHF_F_Hea11596627 to GeneExpressionMatrix for Chr (7 of 23)!, 39.127 mins elapsed.

2023-03-09 23:24:02 : Adding BHF_F_Hea11933674_BHF_F_Hea11596627 to GeneExpressionMatrix for Chr (8 of 23)!, 39.175 mins elapsed.

2023-03-09 23:24:05 : Adding BHF_F_Hea11933674_BHF_F_Hea11596627 to GeneExpressionMatrix for Chr (9 of 23)!, 39.222 mins elapsed.

2023-03-09 23:24:10 : Adding BHF_F_Hea11933674_BHF_F_Hea11596627 to GeneExpressionMatrix for Chr (10 of 23)!, 39.298 mins elapsed.

2023-03-09 23:24:12 : Adding BHF_F_Hea11933674_BHF_F_Hea11596627 to GeneExpressionMatrix for Chr (11 of 23)!, 39.346 mins elapsed.

2023-03-09 23:24:16 : Adding BHF_F_Hea11933674_BHF_F_Hea11596627 to GeneExpressionMatrix for Chr (12 of 23)!, 39.403 mins elapsed.

2023-03-09 23:24:20 : Adding BHF_F_Hea11933674_BHF_F_Hea11596627 to GeneExpressionMatrix for Chr (13 of 23)!, 39.471 mins elapsed.

2023-03-09 23:24:23 : Adding BHF_F_Hea11933674_BHF_F_Hea11596627 to GeneExpressionMatrix for Chr (14 of 23)!, 39.517 mins elapsed.

2023-03-09 23:24:25 : Adding BHF_F_Hea11933674_BHF_F_Hea11596627 to GeneExpressionMatrix for Chr (15 of 23)!, 39.563 mins elapsed.

2023-03-09 23:24:30 : Adding BHF_F_Hea11933674_BHF_F_Hea11596627 to GeneExpressionMatrix for Chr (16 of 23)!, 39.637 mins elapsed.

2023-03-09 23:24:33 : Adding BHF_F_Hea11933674_BHF_F_Hea11596627 to GeneExpressionMatrix for Chr (17 of 23)!, 39.684 mins elapsed.

2023-03-09 23:24:36 : Adding BHF_F_Hea11933674_BHF_F_Hea11596627 to GeneExpressionMatrix for Chr (18 of 23)!, 39.732 mins elapsed.

2023-03-09 23:24:40 : Adding BHF_F_Hea11933674_BHF_F_Hea11596627 to GeneExpressionMatrix for Chr (19 of 23)!, 39.809 mins elapsed.

2023-03-09 23:24:44 : Adding BHF_F_Hea11933674_BHF_F_Hea11596627 to GeneExpressionMatrix for Chr (20 of 23)!, 39.868 mins elapsed.

2023-03-09 23:24:47 : Adding BHF_F_Hea11933674_BHF_F_Hea11596627 to GeneExpressionMatrix for Chr (21 of 23)!, 39.917 mins elapsed.

2023-03-09 23:24:51 : Adding BHF_F_Hea11933674_BHF_F_Hea11596627 to GeneExpressionMatrix for Chr (22 of 23)!, 39.988 mins elapsed.

2023-03-09 23:24:54 : Adding BHF_F_Hea11933674_BHF_F_Hea11596627 to GeneExpressionMatrix for Chr (23 of 23)!, 40.035 mins elapsed.

2023-03-09 23:25:01 : Adding BHF_F_Hea13168892_BHF_F_Hea13168508 to GeneExpressionMatrix for Chr (1 of 23)!, 40.156 mins elapsed.

2023-03-09 23:25:04 : Adding BHF_F_Hea13168892_BHF_F_Hea13168508 to GeneExpressionMatrix for Chr (2 of 23)!, 40.205 mins elapsed.

2023-03-09 23:25:08 : Adding BHF_F_Hea13168892_BHF_F_Hea13168508 to GeneExpressionMatrix for Chr (3 of 23)!, 40.265 mins elapsed.

2023-03-09 23:25:12 : Adding BHF_F_Hea13168892_BHF_F_Hea13168508 to GeneExpressionMatrix for Chr (4 of 23)!, 40.337 mins elapsed.

2023-03-09 23:25:15 : Adding BHF_F_Hea13168892_BHF_F_Hea13168508 to GeneExpressionMatrix for Chr (5 of 23)!, 40.391 mins elapsed.

2023-03-09 23:25:18 : Adding BHF_F_Hea13168892_BHF_F_Hea13168508 to GeneExpressionMatrix for Chr (6 of 23)!, 40.441 mins elapsed.

2023-03-09 23:25:22 : Adding BHF_F_Hea13168892_BHF_F_Hea13168508 to GeneExpressionMatrix for Chr (7 of 23)!, 40.509 mins elapsed.

2023-03-09 23:25:26 : Adding BHF_F_Hea13168892_BHF_F_Hea13168508 to GeneExpressionMatrix for Chr (8 of 23)!, 40.569 mins elapsed.

2023-03-09 23:25:29 : Adding BHF_F_Hea13168892_BHF_F_Hea13168508 to GeneExpressionMatrix for Chr (9 of 23)!, 40.628 mins elapsed.

2023-03-09 23:25:34 : Adding BHF_F_Hea13168892_BHF_F_Hea13168508 to GeneExpressionMatrix for Chr (10 of 23)!, 40.697 mins elapsed.

2023-03-09 23:25:36 : Adding BHF_F_Hea13168892_BHF_F_Hea13168508 to GeneExpressionMatrix for Chr (11 of 23)!, 40.745 mins elapsed.

2023-03-09 23:25:39 : Adding BHF_F_Hea13168892_BHF_F_Hea13168508 to GeneExpressionMatrix for Chr (12 of 23)!, 40.794 mins elapsed.

2023-03-09 23:25:43 : Adding BHF_F_Hea13168892_BHF_F_Hea13168508 to GeneExpressionMatrix for Chr (13 of 23)!, 40.863 mins elapsed.

2023-03-09 23:25:46 : Adding BHF_F_Hea13168892_BHF_F_Hea13168508 to GeneExpressionMatrix for Chr (14 of 23)!, 40.91 mins elapsed.

2023-03-09 23:25:49 : Adding BHF_F_Hea13168892_BHF_F_Hea13168508 to GeneExpressionMatrix for Chr (15 of 23)!, 40.957 mins elapsed.

2023-03-09 23:25:53 : Adding BHF_F_Hea13168892_BHF_F_Hea13168508 to GeneExpressionMatrix for Chr (16 of 23)!, 41.024 mins elapsed.

2023-03-09 23:25:56 : Adding BHF_F_Hea13168892_BHF_F_Hea13168508 to GeneExpressionMatrix for Chr (17 of 23)!, 41.072 mins elapsed.

2023-03-09 23:25:59 : Adding BHF_F_Hea13168892_BHF_F_Hea13168508 to GeneExpressionMatrix for Chr (18 of 23)!, 41.12 mins elapsed.

2023-03-09 23:26:03 : Adding BHF_F_Hea13168892_BHF_F_Hea13168508 to GeneExpressionMatrix for Chr (19 of 23)!, 41.186 mins elapsed.

2023-03-09 23:26:06 : Adding BHF_F_Hea13168892_BHF_F_Hea13168508 to GeneExpressionMatrix for Chr (20 of 23)!, 41.236 mins elapsed.

2023-03-09 23:26:09 : Adding BHF_F_Hea13168892_BHF_F_Hea13168508 to GeneExpressionMatrix for Chr (21 of 23)!, 41.284 mins elapsed.

2023-03-09 23:26:13 : Adding BHF_F_Hea13168892_BHF_F_Hea13168508 to GeneExpressionMatrix for Chr (22 of 23)!, 41.351 mins elapsed.

2023-03-09 23:26:16 : Adding BHF_F_Hea13168892_BHF_F_Hea13168508 to GeneExpressionMatrix for Chr (23 of 23)!, 41.398 mins elapsed.

2023-03-09 23:26:23 : Adding BHF_F_Hea11064671_BHF_F_Hea11031824 to GeneExpressionMatrix for Chr (1 of 23)!, 41.518 mins elapsed.

2023-03-09 23:26:26 : Adding BHF_F_Hea11064671_BHF_F_Hea11031824 to GeneExpressionMatrix for Chr (2 of 23)!, 41.577 mins elapsed.

2023-03-09 23:26:29 : Adding BHF_F_Hea11064671_BHF_F_Hea11031824 to GeneExpressionMatrix for Chr (3 of 23)!, 41.629 mins elapsed.

2023-03-09 23:26:34 : Adding BHF_F_Hea11064671_BHF_F_Hea11031824 to GeneExpressionMatrix for Chr (4 of 23)!, 41.698 mins elapsed.

2023-03-09 23:26:36 : Adding BHF_F_Hea11064671_BHF_F_Hea11031824 to GeneExpressionMatrix for Chr (5 of 23)!, 41.746 mins elapsed.

2023-03-09 23:26:40 : Adding BHF_F_Hea11064671_BHF_F_Hea11031824 to GeneExpressionMatrix for Chr (6 of 23)!, 41.798 mins elapsed.

2023-03-09 23:26:44 : Adding BHF_F_Hea11064671_BHF_F_Hea11031824 to GeneExpressionMatrix for Chr (7 of 23)!, 41.875 mins elapsed.

2023-03-09 23:26:47 : Adding BHF_F_Hea11064671_BHF_F_Hea11031824 to GeneExpressionMatrix for Chr (8 of 23)!, 41.925 mins elapsed.

2023-03-09 23:26:51 : Adding BHF_F_Hea11064671_BHF_F_Hea11031824 to GeneExpressionMatrix for Chr (9 of 23)!, 41.982 mins elapsed.

2023-03-09 23:26:55 : Adding BHF_F_Hea11064671_BHF_F_Hea11031824 to GeneExpressionMatrix for Chr (10 of 23)!, 42.05 mins elapsed.

2023-03-09 23:26:58 : Adding BHF_F_Hea11064671_BHF_F_Hea11031824 to GeneExpressionMatrix for Chr (11 of 23)!, 42.098 mins elapsed.

2023-03-09 23:27:00 : Adding BHF_F_Hea11064671_BHF_F_Hea11031824 to GeneExpressionMatrix for Chr (12 of 23)!, 42.146 mins elapsed.

2023-03-09 23:27:05 : Adding BHF_F_Hea11064671_BHF_F_Hea11031824 to GeneExpressionMatrix for Chr (13 of 23)!, 42.214 mins elapsed.

2023-03-09 23:27:07 : Adding BHF_F_Hea11064671_BHF_F_Hea11031824 to GeneExpressionMatrix for Chr (14 of 23)!, 42.26 mins elapsed.

2023-03-09 23:27:10 : Adding BHF_F_Hea11064671_BHF_F_Hea11031824 to GeneExpressionMatrix for Chr (15 of 23)!, 42.313 mins elapsed.

2023-03-09 23:27:15 : Adding BHF_F_Hea11064671_BHF_F_Hea11031824 to GeneExpressionMatrix for Chr (16 of 23)!, 42.381 mins elapsed.

2023-03-09 23:27:17 : Adding BHF_F_Hea11064671_BHF_F_Hea11031824 to GeneExpressionMatrix for Chr (17 of 23)!, 42.429 mins elapsed.

2023-03-09 23:27:20 : Adding BHF_F_Hea11064671_BHF_F_Hea11031824 to GeneExpressionMatrix for Chr (18 of 23)!, 42.479 mins elapsed.

2023-03-09 23:27:24 : Adding BHF_F_Hea11064671_BHF_F_Hea11031824 to GeneExpressionMatrix for Chr (19 of 23)!, 42.545 mins elapsed.

2023-03-09 23:27:27 : Adding BHF_F_Hea11064671_BHF_F_Hea11031824 to GeneExpressionMatrix for Chr (20 of 23)!, 42.593 mins elapsed.

2023-03-09 23:27:30 : Adding BHF_F_Hea11064671_BHF_F_Hea11031824 to GeneExpressionMatrix for Chr (21 of 23)!, 42.643 mins elapsed.

2023-03-09 23:27:35 : Adding BHF_F_Hea11064671_BHF_F_Hea11031824 to GeneExpressionMatrix for Chr (22 of 23)!, 42.726 mins elapsed.

2023-03-09 23:27:39 : Adding BHF_F_Hea11064671_BHF_F_Hea11031824 to GeneExpressionMatrix for Chr (23 of 23)!, 42.781 mins elapsed.

2023-03-09 23:27:46 : Adding BHF_F_Hea13188328_BHF_F_Hea13187630 to GeneExpressionMatrix for Chr (1 of 23)!, 42.902 mins elapsed.

2023-03-09 23:27:49 : Adding BHF_F_Hea13188328_BHF_F_Hea13187630 to GeneExpressionMatrix for Chr (2 of 23)!, 42.952 mins elapsed.

2023-03-09 23:27:52 : Adding BHF_F_Hea13188328_BHF_F_Hea13187630 to GeneExpressionMatrix for Chr (3 of 23)!, 42.999 mins elapsed.

2023-03-09 23:27:56 : Adding BHF_F_Hea13188328_BHF_F_Hea13187630 to GeneExpressionMatrix for Chr (4 of 23)!, 43.065 mins elapsed.

2023-03-09 23:27:59 : Adding BHF_F_Hea13188328_BHF_F_Hea13187630 to GeneExpressionMatrix for Chr (5 of 23)!, 43.121 mins elapsed.

2023-03-09 23:28:02 : Adding BHF_F_Hea13188328_BHF_F_Hea13187630 to GeneExpressionMatrix for Chr (6 of 23)!, 43.177 mins elapsed.

2023-03-09 23:28:07 : Adding BHF_F_Hea13188328_BHF_F_Hea13187630 to GeneExpressionMatrix for Chr (7 of 23)!, 43.259 mins elapsed.

2023-03-09 23:28:10 : Adding BHF_F_Hea13188328_BHF_F_Hea13187630 to GeneExpressionMatrix for Chr (8 of 23)!, 43.313 mins elapsed.

2023-03-09 23:28:13 : Adding BHF_F_Hea13188328_BHF_F_Hea13187630 to GeneExpressionMatrix for Chr (9 of 23)!, 43.36 mins elapsed.

2023-03-09 23:28:17 : Adding BHF_F_Hea13188328_BHF_F_Hea13187630 to GeneExpressionMatrix for Chr (10 of 23)!, 43.426 mins elapsed.

2023-03-09 23:28:20 : Adding BHF_F_Hea13188328_BHF_F_Hea13187630 to GeneExpressionMatrix for Chr (11 of 23)!, 43.471 mins elapsed.

2023-03-09 23:28:23 : Adding BHF_F_Hea13188328_BHF_F_Hea13187630 to GeneExpressionMatrix for Chr (12 of 23)!, 43.517 mins elapsed.

2023-03-09 23:28:27 : Adding BHF_F_Hea13188328_BHF_F_Hea13187630 to GeneExpressionMatrix for Chr (13 of 23)!, 43.584 mins elapsed.

2023-03-09 23:28:29 : Adding BHF_F_Hea13188328_BHF_F_Hea13187630 to GeneExpressionMatrix for Chr (14 of 23)!, 43.629 mins elapsed.

2023-03-09 23:28:32 : Adding BHF_F_Hea13188328_BHF_F_Hea13187630 to GeneExpressionMatrix for Chr (15 of 23)!, 43.676 mins elapsed.

2023-03-09 23:28:36 : Adding BHF_F_Hea13188328_BHF_F_Hea13187630 to GeneExpressionMatrix for Chr (16 of 23)!, 43.745 mins elapsed.

2023-03-09 23:28:39 : Adding BHF_F_Hea13188328_BHF_F_Hea13187630 to GeneExpressionMatrix for Chr (17 of 23)!, 43.796 mins elapsed.

2023-03-09 23:28:42 : Adding BHF_F_Hea13188328_BHF_F_Hea13187630 to GeneExpressionMatrix for Chr (18 of 23)!, 43.843 mins elapsed.

2023-03-09 23:28:46 : Adding BHF_F_Hea13188328_BHF_F_Hea13187630 to GeneExpressionMatrix for Chr (19 of 23)!, 43.911 mins elapsed.

2023-03-09 23:28:49 : Adding BHF_F_Hea13188328_BHF_F_Hea13187630 to GeneExpressionMatrix for Chr (20 of 23)!, 43.957 mins elapsed.

2023-03-09 23:28:52 : Adding BHF_F_Hea13188328_BHF_F_Hea13187630 to GeneExpressionMatrix for Chr (21 of 23)!, 44.004 mins elapsed.

2023-03-09 23:28:56 : Adding BHF_F_Hea13188328_BHF_F_Hea13187630 to GeneExpressionMatrix for Chr (22 of 23)!, 44.071 mins elapsed.

2023-03-09 23:28:59 : Adding BHF_F_Hea13188328_BHF_F_Hea13187630 to GeneExpressionMatrix for Chr (23 of 23)!, 44.118 mins elapsed.

2023-03-09 23:29:06 : Adding BHF_F_Hea13188329_BHF_F_Hea13187631 to GeneExpressionMatrix for Chr (1 of 23)!, 44.231 mins elapsed.

2023-03-09 23:29:08 : Adding BHF_F_Hea13188329_BHF_F_Hea13187631 to GeneExpressionMatrix for Chr (2 of 23)!, 44.277 mins elapsed.

2023-03-09 23:29:11 : Adding BHF_F_Hea13188329_BHF_F_Hea13187631 to GeneExpressionMatrix for Chr (3 of 23)!, 44.323 mins elapsed.

2023-03-09 23:29:15 : Adding BHF_F_Hea13188329_BHF_F_Hea13187631 to GeneExpressionMatrix for Chr (4 of 23)!, 44.393 mins elapsed.

2023-03-09 23:29:18 : Adding BHF_F_Hea13188329_BHF_F_Hea13187631 to GeneExpressionMatrix for Chr (5 of 23)!, 44.44 mins elapsed.

2023-03-09 23:29:21 : Adding BHF_F_Hea13188329_BHF_F_Hea13187631 to GeneExpressionMatrix for Chr (6 of 23)!, 44.488 mins elapsed.

2023-03-09 23:29:26 : Adding BHF_F_Hea13188329_BHF_F_Hea13187631 to GeneExpressionMatrix for Chr (7 of 23)!, 44.568 mins elapsed.

2023-03-09 23:29:29 : Adding BHF_F_Hea13188329_BHF_F_Hea13187631 to GeneExpressionMatrix for Chr (8 of 23)!, 44.62 mins elapsed.

2023-03-09 23:29:32 : Adding BHF_F_Hea13188329_BHF_F_Hea13187631 to GeneExpressionMatrix for Chr (9 of 23)!, 44.665 mins elapsed.

2023-03-09 23:29:35 : Adding BHF_F_Hea13188329_BHF_F_Hea13187631 to GeneExpressionMatrix for Chr (10 of 23)!, 44.729 mins elapsed.

2023-03-09 23:29:38 : Adding BHF_F_Hea13188329_BHF_F_Hea13187631 to GeneExpressionMatrix for Chr (11 of 23)!, 44.775 mins elapsed.

2023-03-09 23:29:41 : Adding BHF_F_Hea13188329_BHF_F_Hea13187631 to GeneExpressionMatrix for Chr (12 of 23)!, 44.82 mins elapsed.

2023-03-09 23:29:45 : Adding BHF_F_Hea13188329_BHF_F_Hea13187631 to GeneExpressionMatrix for Chr (13 of 23)!, 44.885 mins elapsed.

2023-03-09 23:29:48 : Adding BHF_F_Hea13188329_BHF_F_Hea13187631 to GeneExpressionMatrix for Chr (14 of 23)!, 44.932 mins elapsed.

2023-03-09 23:29:50 : Adding BHF_F_Hea13188329_BHF_F_Hea13187631 to GeneExpressionMatrix for Chr (15 of 23)!, 44.978 mins elapsed.

2023-03-09 23:29:54 : Adding BHF_F_Hea13188329_BHF_F_Hea13187631 to GeneExpressionMatrix for Chr (16 of 23)!, 45.044 mins elapsed.

2023-03-09 23:29:57 : Adding BHF_F_Hea13188329_BHF_F_Hea13187631 to GeneExpressionMatrix for Chr (17 of 23)!, 45.089 mins elapsed.

2023-03-09 23:30:00 : Adding BHF_F_Hea13188329_BHF_F_Hea13187631 to GeneExpressionMatrix for Chr (18 of 23)!, 45.136 mins elapsed.

2023-03-09 23:30:04 : Adding BHF_F_Hea13188329_BHF_F_Hea13187631 to GeneExpressionMatrix for Chr (19 of 23)!, 45.202 mins elapsed.

2023-03-09 23:30:07 : Adding BHF_F_Hea13188329_BHF_F_Hea13187631 to GeneExpressionMatrix for Chr (20 of 23)!, 45.248 mins elapsed.

2023-03-09 23:30:09 : Adding BHF_F_Hea13188329_BHF_F_Hea13187631 to GeneExpressionMatrix for Chr (21 of 23)!, 45.294 mins elapsed.

2023-03-09 23:30:13 : Adding BHF_F_Hea13188329_BHF_F_Hea13187631 to GeneExpressionMatrix for Chr (22 of 23)!, 45.359 mins elapsed.

2023-03-09 23:30:16 : Adding BHF_F_Hea13188329_BHF_F_Hea13187631 to GeneExpressionMatrix for Chr (23 of 23)!, 45.405 mins elapsed.

2023-03-09 23:30:22 : Adding BHF_F_Hea13188330_BHF_F_Hea13187632 to GeneExpressionMatrix for Chr (1 of 23)!, 45.504 mins elapsed.

2023-03-09 23:30:25 : Adding BHF_F_Hea13188330_BHF_F_Hea13187632 to GeneExpressionMatrix for Chr (2 of 23)!, 45.55 mins elapsed.

2023-03-09 23:30:27 : Adding BHF_F_Hea13188330_BHF_F_Hea13187632 to GeneExpressionMatrix for Chr (3 of 23)!, 45.596 mins elapsed.

2023-03-09 23:30:31 : Adding BHF_F_Hea13188330_BHF_F_Hea13187632 to GeneExpressionMatrix for Chr (4 of 23)!, 45.661 mins elapsed.

2023-03-09 23:30:34 : Adding BHF_F_Hea13188330_BHF_F_Hea13187632 to GeneExpressionMatrix for Chr (5 of 23)!, 45.707 mins elapsed.

2023-03-09 23:30:37 : Adding BHF_F_Hea13188330_BHF_F_Hea13187632 to GeneExpressionMatrix for Chr (6 of 23)!, 45.753 mins elapsed.

2023-03-09 23:30:41 : Adding BHF_F_Hea13188330_BHF_F_Hea13187632 to GeneExpressionMatrix for Chr (7 of 23)!, 45.818 mins elapsed.

2023-03-09 23:30:44 : Adding BHF_F_Hea13188330_BHF_F_Hea13187632 to GeneExpressionMatrix for Chr (8 of 23)!, 45.864 mins elapsed.

2023-03-09 23:30:46 : Adding BHF_F_Hea13188330_BHF_F_Hea13187632 to GeneExpressionMatrix for Chr (9 of 23)!, 45.909 mins elapsed.

2023-03-09 23:30:50 : Adding BHF_F_Hea13188330_BHF_F_Hea13187632 to GeneExpressionMatrix for Chr (10 of 23)!, 45.976 mins elapsed.

2023-03-09 23:30:53 : Adding BHF_F_Hea13188330_BHF_F_Hea13187632 to GeneExpressionMatrix for Chr (11 of 23)!, 46.022 mins elapsed.

2023-03-09 23:30:56 : Adding BHF_F_Hea13188330_BHF_F_Hea13187632 to GeneExpressionMatrix for Chr (12 of 23)!, 46.068 mins elapsed.

2023-03-09 23:31:00 : Adding BHF_F_Hea13188330_BHF_F_Hea13187632 to GeneExpressionMatrix for Chr (13 of 23)!, 46.134 mins elapsed.

2023-03-09 23:31:03 : Adding BHF_F_Hea13188330_BHF_F_Hea13187632 to GeneExpressionMatrix for Chr (14 of 23)!, 46.181 mins elapsed.

2023-03-09 23:31:05 : Adding BHF_F_Hea13188330_BHF_F_Hea13187632 to GeneExpressionMatrix for Chr (15 of 23)!, 46.227 mins elapsed.

2023-03-09 23:31:09 : Adding BHF_F_Hea13188330_BHF_F_Hea13187632 to GeneExpressionMatrix for Chr (16 of 23)!, 46.294 mins elapsed.

2023-03-09 23:31:12 : Adding BHF_F_Hea13188330_BHF_F_Hea13187632 to GeneExpressionMatrix for Chr (17 of 23)!, 46.34 mins elapsed.

2023-03-09 23:31:15 : Adding BHF_F_Hea13188330_BHF_F_Hea13187632 to GeneExpressionMatrix for Chr (18 of 23)!, 46.387 mins elapsed.

2023-03-09 23:31:19 : Adding BHF_F_Hea13188330_BHF_F_Hea13187632 to GeneExpressionMatrix for Chr (19 of 23)!, 46.457 mins elapsed.

2023-03-09 23:31:22 : Adding BHF_F_Hea13188330_BHF_F_Hea13187632 to GeneExpressionMatrix for Chr (20 of 23)!, 46.503 mins elapsed.

2023-03-09 23:31:25 : Adding BHF_F_Hea13188330_BHF_F_Hea13187632 to GeneExpressionMatrix for Chr (21 of 23)!, 46.549 mins elapsed.

2023-03-09 23:31:29 : Adding BHF_F_Hea13188330_BHF_F_Hea13187632 to GeneExpressionMatrix for Chr (22 of 23)!, 46.615 mins elapsed.

2023-03-09 23:31:31 : Adding BHF_F_Hea13188330_BHF_F_Hea13187632 to GeneExpressionMatrix for Chr (23 of 23)!, 46.66 mins elapsed.

2023-03-09 23:31:37 : Adding BHF_F_Hea11064670_BHF_F_Hea11031823 to GeneExpressionMatrix for Chr (1 of 23)!, 46.76 mins elapsed.

2023-03-09 23:31:40 : Adding BHF_F_Hea11064670_BHF_F_Hea11031823 to GeneExpressionMatrix for Chr (2 of 23)!, 46.806 mins elapsed.

2023-03-09 23:31:43 : Adding BHF_F_Hea11064670_BHF_F_Hea11031823 to GeneExpressionMatrix for Chr (3 of 23)!, 46.852 mins elapsed.

2023-03-09 23:31:47 : Adding BHF_F_Hea11064670_BHF_F_Hea11031823 to GeneExpressionMatrix for Chr (4 of 23)!, 46.917 mins elapsed.

2023-03-09 23:31:49 : Adding BHF_F_Hea11064670_BHF_F_Hea11031823 to GeneExpressionMatrix for Chr (5 of 23)!, 46.962 mins elapsed.

2023-03-09 23:31:52 : Adding BHF_F_Hea11064670_BHF_F_Hea11031823 to GeneExpressionMatrix for Chr (6 of 23)!, 47.007 mins elapsed.

2023-03-09 23:31:56 : Adding BHF_F_Hea11064670_BHF_F_Hea11031823 to GeneExpressionMatrix for Chr (7 of 23)!, 47.073 mins elapsed.

2023-03-09 23:31:59 : Adding BHF_F_Hea11064670_BHF_F_Hea11031823 to GeneExpressionMatrix for Chr (8 of 23)!, 47.118 mins elapsed.

2023-03-09 23:32:01 : Adding BHF_F_Hea11064670_BHF_F_Hea11031823 to GeneExpressionMatrix for Chr (9 of 23)!, 47.163 mins elapsed.

2023-03-09 23:32:05 : Adding BHF_F_Hea11064670_BHF_F_Hea11031823 to GeneExpressionMatrix for Chr (10 of 23)!, 47.228 mins elapsed.

2023-03-09 23:32:08 : Adding BHF_F_Hea11064670_BHF_F_Hea11031823 to GeneExpressionMatrix for Chr (11 of 23)!, 47.274 mins elapsed.

2023-03-09 23:32:11 : Adding BHF_F_Hea11064670_BHF_F_Hea11031823 to GeneExpressionMatrix for Chr (12 of 23)!, 47.32 mins elapsed.

2023-03-09 23:32:15 : Adding BHF_F_Hea11064670_BHF_F_Hea11031823 to GeneExpressionMatrix for Chr (13 of 23)!, 47.386 mins elapsed.

2023-03-09 23:32:18 : Adding BHF_F_Hea11064670_BHF_F_Hea11031823 to GeneExpressionMatrix for Chr (14 of 23)!, 47.431 mins elapsed.

2023-03-09 23:32:21 : Adding BHF_F_Hea11064670_BHF_F_Hea11031823 to GeneExpressionMatrix for Chr (15 of 23)!, 47.488 mins elapsed.

2023-03-09 23:32:26 : Adding BHF_F_Hea11064670_BHF_F_Hea11031823 to GeneExpressionMatrix for Chr (16 of 23)!, 47.569 mins elapsed.

2023-03-09 23:32:29 : Adding BHF_F_Hea11064670_BHF_F_Hea11031823 to GeneExpressionMatrix for Chr (17 of 23)!, 47.62 mins elapsed.

2023-03-09 23:32:32 : Adding BHF_F_Hea11064670_BHF_F_Hea11031823 to GeneExpressionMatrix for Chr (18 of 23)!, 47.668 mins elapsed.

2023-03-09 23:32:36 : Adding BHF_F_Hea11064670_BHF_F_Hea11031823 to GeneExpressionMatrix for Chr (19 of 23)!, 47.737 mins elapsed.

2023-03-09 23:32:39 : Adding BHF_F_Hea11064670_BHF_F_Hea11031823 to GeneExpressionMatrix for Chr (20 of 23)!, 47.783 mins elapsed.

2023-03-09 23:32:41 : Adding BHF_F_Hea11064670_BHF_F_Hea11031823 to GeneExpressionMatrix for Chr (21 of 23)!, 47.829 mins elapsed.

2023-03-09 23:32:45 : Adding BHF_F_Hea11064670_BHF_F_Hea11031823 to GeneExpressionMatrix for Chr (22 of 23)!, 47.895 mins elapsed.

2023-03-09 23:32:48 : Adding BHF_F_Hea11064670_BHF_F_Hea11031823 to GeneExpressionMatrix for Chr (23 of 23)!, 47.94 mins elapsed.

2023-03-09 23:32:54 : Adding BHF_F_Hea11933675_BHF_F_Hea11596628 to GeneExpressionMatrix for Chr (1 of 23)!, 48.039 mins elapsed.

2023-03-09 23:32:57 : Adding BHF_F_Hea11933675_BHF_F_Hea11596628 to GeneExpressionMatrix for Chr (2 of 23)!, 48.085 mins elapsed.

2023-03-09 23:33:00 : Adding BHF_F_Hea11933675_BHF_F_Hea11596628 to GeneExpressionMatrix for Chr (3 of 23)!, 48.136 mins elapsed.

2023-03-09 23:33:04 : Adding BHF_F_Hea11933675_BHF_F_Hea11596628 to GeneExpressionMatrix for Chr (4 of 23)!, 48.2 mins elapsed.

2023-03-09 23:33:07 : Adding BHF_F_Hea11933675_BHF_F_Hea11596628 to GeneExpressionMatrix for Chr (5 of 23)!, 48.254 mins elapsed.

2023-03-09 23:33:10 : Adding BHF_F_Hea11933675_BHF_F_Hea11596628 to GeneExpressionMatrix for Chr (6 of 23)!, 48.31 mins elapsed.

2023-03-09 23:33:14 : Adding BHF_F_Hea11933675_BHF_F_Hea11596628 to GeneExpressionMatrix for Chr (7 of 23)!, 48.375 mins elapsed.

2023-03-09 23:33:17 : Adding BHF_F_Hea11933675_BHF_F_Hea11596628 to GeneExpressionMatrix for Chr (8 of 23)!, 48.419 mins elapsed.

2023-03-09 23:33:19 : Adding BHF_F_Hea11933675_BHF_F_Hea11596628 to GeneExpressionMatrix for Chr (9 of 23)!, 48.463 mins elapsed.

2023-03-09 23:33:23 : Adding BHF_F_Hea11933675_BHF_F_Hea11596628 to GeneExpressionMatrix for Chr (10 of 23)!, 48.527 mins elapsed.

2023-03-09 23:33:26 : Adding BHF_F_Hea11933675_BHF_F_Hea11596628 to GeneExpressionMatrix for Chr (11 of 23)!, 48.572 mins elapsed.

2023-03-09 23:33:29 : Adding BHF_F_Hea11933675_BHF_F_Hea11596628 to GeneExpressionMatrix for Chr (12 of 23)!, 48.617 mins elapsed.

2023-03-09 23:33:33 : Adding BHF_F_Hea11933675_BHF_F_Hea11596628 to GeneExpressionMatrix for Chr (13 of 23)!, 48.694 mins elapsed.

2023-03-09 23:33:36 : Adding BHF_F_Hea11933675_BHF_F_Hea11596628 to GeneExpressionMatrix for Chr (14 of 23)!, 48.742 mins elapsed.

2023-03-09 23:33:39 : Adding BHF_F_Hea11933675_BHF_F_Hea11596628 to GeneExpressionMatrix for Chr (15 of 23)!, 48.787 mins elapsed.

2023-03-09 23:33:43 : Adding BHF_F_Hea11933675_BHF_F_Hea11596628 to GeneExpressionMatrix for Chr (16 of 23)!, 48.852 mins elapsed.

2023-03-09 23:33:46 : Adding BHF_F_Hea11933675_BHF_F_Hea11596628 to GeneExpressionMatrix for Chr (17 of 23)!, 48.897 mins elapsed.

2023-03-09 23:33:48 : Adding BHF_F_Hea11933675_BHF_F_Hea11596628 to GeneExpressionMatrix for Chr (18 of 23)!, 48.946 mins elapsed.

2023-03-09 23:33:52 : Adding BHF_F_Hea11933675_BHF_F_Hea11596628 to GeneExpressionMatrix for Chr (19 of 23)!, 49.012 mins elapsed.

2023-03-09 23:33:55 : Adding BHF_F_Hea11933675_BHF_F_Hea11596628 to GeneExpressionMatrix for Chr (20 of 23)!, 49.056 mins elapsed.

2023-03-09 23:33:58 : Adding BHF_F_Hea11933675_BHF_F_Hea11596628 to GeneExpressionMatrix for Chr (21 of 23)!, 49.101 mins elapsed.

2023-03-09 23:34:02 : Adding BHF_F_Hea11933675_BHF_F_Hea11596628 to GeneExpressionMatrix for Chr (22 of 23)!, 49.166 mins elapsed.

2023-03-09 23:34:04 : Adding BHF_F_Hea11933675_BHF_F_Hea11596628 to GeneExpressionMatrix for Chr (23 of 23)!, 49.211 mins elapsed.

2023-03-09 23:34:10 : Adding 7089STDY13216923_BHF_F_Hea13242530 to GeneExpressionMatrix for Chr (1 of 23)!, 49.308 mins elapsed.

2023-03-09 23:34:13 : Adding 7089STDY13216923_BHF_F_Hea13242530 to GeneExpressionMatrix for Chr (2 of 23)!, 49.353 mins elapsed.

2023-03-09 23:34:15 : Adding 7089STDY13216923_BHF_F_Hea13242530 to GeneExpressionMatrix for Chr (3 of 23)!, 49.397 mins elapsed.

2023-03-09 23:34:20 : Adding 7089STDY13216923_BHF_F_Hea13242530 to GeneExpressionMatrix for Chr (4 of 23)!, 49.468 mins elapsed.

2023-03-09 23:34:22 : Adding 7089STDY13216923_BHF_F_Hea13242530 to GeneExpressionMatrix for Chr (5 of 23)!, 49.513 mins elapsed.

2023-03-09 23:34:25 : Adding 7089STDY13216923_BHF_F_Hea13242530 to GeneExpressionMatrix for Chr (6 of 23)!, 49.557 mins elapsed.

2023-03-09 23:34:29 : Adding 7089STDY13216923_BHF_F_Hea13242530 to GeneExpressionMatrix for Chr (7 of 23)!, 49.621 mins elapsed.

2023-03-09 23:34:32 : Adding 7089STDY13216923_BHF_F_Hea13242530 to GeneExpressionMatrix for Chr (8 of 23)!, 49.665 mins elapsed.

2023-03-09 23:34:34 : Adding 7089STDY13216923_BHF_F_Hea13242530 to GeneExpressionMatrix for Chr (9 of 23)!, 49.71 mins elapsed.

2023-03-09 23:34:38 : Adding 7089STDY13216923_BHF_F_Hea13242530 to GeneExpressionMatrix for Chr (10 of 23)!, 49.773 mins elapsed.

2023-03-09 23:34:41 : Adding 7089STDY13216923_BHF_F_Hea13242530 to GeneExpressionMatrix for Chr (11 of 23)!, 49.818 mins elapsed.

2023-03-09 23:34:44 : Adding 7089STDY13216923_BHF_F_Hea13242530 to GeneExpressionMatrix for Chr (12 of 23)!, 49.873 mins elapsed.

2023-03-09 23:34:49 : Adding 7089STDY13216923_BHF_F_Hea13242530 to GeneExpressionMatrix for Chr (13 of 23)!, 49.954 mins elapsed.

2023-03-09 23:34:52 : Adding 7089STDY13216923_BHF_F_Hea13242530 to GeneExpressionMatrix for Chr (14 of 23)!, 50.002 mins elapsed.

2023-03-09 23:34:55 : Adding 7089STDY13216923_BHF_F_Hea13242530 to GeneExpressionMatrix for Chr (15 of 23)!, 50.048 mins elapsed.

2023-03-09 23:34:58 : Adding 7089STDY13216923_BHF_F_Hea13242530 to GeneExpressionMatrix for Chr (16 of 23)!, 50.112 mins elapsed.

2023-03-09 23:35:01 : Adding 7089STDY13216923_BHF_F_Hea13242530 to GeneExpressionMatrix for Chr (17 of 23)!, 50.157 mins elapsed.

2023-03-09 23:35:04 : Adding 7089STDY13216923_BHF_F_Hea13242530 to GeneExpressionMatrix for Chr (18 of 23)!, 50.202 mins elapsed.

2023-03-09 23:35:08 : Adding 7089STDY13216923_BHF_F_Hea13242530 to GeneExpressionMatrix for Chr (19 of 23)!, 50.272 mins elapsed.

2023-03-09 23:35:11 : Adding 7089STDY13216923_BHF_F_Hea13242530 to GeneExpressionMatrix for Chr (20 of 23)!, 50.319 mins elapsed.

2023-03-09 23:35:14 : Adding 7089STDY13216923_BHF_F_Hea13242530 to GeneExpressionMatrix for Chr (21 of 23)!, 50.365 mins elapsed.

2023-03-09 23:35:18 : Adding 7089STDY13216923_BHF_F_Hea13242530 to GeneExpressionMatrix for Chr (22 of 23)!, 50.43 mins elapsed.

2023-03-09 23:35:20 : Adding 7089STDY13216923_BHF_F_Hea13242530 to GeneExpressionMatrix for Chr (23 of 23)!, 50.475 mins elapsed.

ArchR logging successful to : ArchRLogs/ArchR-addGeneExpressionMatrix-49213027a1f7-Date-2023-03-09_Time-22-44-06.log


           ___      .______        ______  __    __  .______      
          /   \     |   _  \      /      ||  |  |  | |   _  \     
         /  ^  \    |  |_)  |    |  ,----'|  |__|  | |  |_)  |    
        /  /_\  \   |      /     |  |     |   __   | |      /     
       /  _____  \  |  |\  \\___ |  `----.|  |  |  | |  |\  \\___.
      /__/     \__\ | _| `._____| \______||__|  |__| | _| `._____|
```

Example 31 (unknown):
```unknown
class: ArchRProject 
outputDirectory: /nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output 
samples(37): BHF_F_Hea11064670_BHF_F_Hea11031823
  BHF_F_Hea11064671_BHF_F_Hea11031824 ...
  HCAHeartST13386009_HCAHeartST13303419
  HCAHeartST13386010_HCAHeartST13303420
sampleColData names(1): ArrowFiles
cellColData names(63): Sample TSSEnrichment ... Gex_MitoRatio
  Gex_RiboRatio
numberOfCells(1): 167022
medianTSS(1): 11.614
medianFrags(1): 10338.5
```

Example 32 (unknown):
```unknown
saveArchRProject(ArchRProj = proj, outputDirectory = "project_output", load = FALSE)
```

Example 33 (unknown):
```unknown
Saving ArchRProject...
```

---

## Variables¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/Xenium/no2_integration_and_cell-type-annotation/no02_add_celltypist_prediction.html

**Contents:**
- Variables¶
- Read in data¶
- Add celltypist prediction results¶
- Save¶

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os

import celltypist
```

Example 2 (python):
```python
# import session_info
# session_info.show()
```

Example 3 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 4 (unknown):
```unknown
os.getcwd()
```

Example 5 (unknown):
```unknown
'/rfs/project/rfs-iCNyzSAaucw/kk837/notebooks/Foetal/Xenium/5K/C194_Hst45/CamU'
```

Example 6 (unknown):
```unknown
objects_dir = '/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium'
```

Example 7 (python):
```python
adata = sc.read_h5ad(f'{objects_dir}/C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat_5K_filtered_raw.h5ad')
adata
```

Example 8 (unknown):
```unknown
AnnData object with n_obs × n_vars = 362277 × 5001
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1'
    obsm: 'spatial'
```

Example 9 (python):
```python
adata.X.data[:5]
```

Example 10 (unknown):
```unknown
array([5., 1., 1., 2., 2.], dtype=float32)
```

Example 11 (unknown):
```unknown
columns_to_transfer = ['celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod',
                       'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine']
```

Example 12 (python):
```python
c194 = sc.read_h5ad(f'{objects_dir}/C194-HEA-0-FFPE-1_5K_filtered_lognorm.h5ad',
                    backed='r'
                   )
hst45 = sc.read_h5ad(f'{objects_dir}/Hst45-HEA-0-FFPE-1_5K_filtered_lognorm.h5ad',
                    backed='r'
                   )
```

Example 13 (python):
```python
print(c194.obs[columns_to_transfer].isna().sum())
print(hst45.obs[columns_to_transfer].isna().sum())
```

Example 14 (unknown):
```unknown
celltypist_coarse              0
conf_score_coarse              0
celltypist_coarse_fin          0
celltypist_coarse2midmod    4000
conf_score_coarse2midmod    4000
celltypist_mid_fin             0
celltypist_midmod2fine      6917
conf_score_midmod2fine      6917
dtype: int64
celltypist_coarse              0
conf_score_coarse              0
celltypist_coarse_fin          0
celltypist_coarse2midmod    2081
conf_score_coarse2midmod    2081
celltypist_mid_fin             0
celltypist_midmod2fine      4724
conf_score_midmod2fine      4724
dtype: int64
```

Example 15 (unknown):
```unknown
# update obsnames
c194.obs_names = list('C194-HEA-0-FFPE-1_'+c194.obs_names.astype('str'))
hst45.obs_names = list('Hst45-HEA-0-FFPE-1_'+hst45.obs_names.astype('str'))
```

Example 16 (python):
```python
# add
adata.obs[columns_to_transfer] = np.nan
adata.obs.loc[c194.obs_names,columns_to_transfer] = c194.obs[columns_to_transfer]
adata.obs.loc[hst45.obs_names,columns_to_transfer] = hst45.obs[columns_to_transfer]
```

Example 17 (python):
```python
# check if there's rows missing prediction results
adata.obs[columns_to_transfer].isna().sum()
```

Example 18 (unknown):
```unknown
celltypist_coarse               0
conf_score_coarse               0
celltypist_coarse_fin           0
celltypist_coarse2midmod     6081
conf_score_coarse2midmod     6081
celltypist_mid_fin              0
celltypist_midmod2fine      11641
conf_score_midmod2fine      11641
dtype: int64
```

Example 19 (python):
```python
adata.write(f'{objects_dir}/C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat_5K_filtered_raw.h5ad')
f'{objects_dir}/C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat_5K_filtered_raw.h5ad'
```

Example 20 (unknown):
```unknown
... storing 'celltypist_coarse' as categorical
... storing 'celltypist_coarse_fin' as categorical
... storing 'celltypist_coarse2midmod' as categorical
... storing 'celltypist_mid_fin' as categorical
... storing 'celltypist_midmod2fine' as categorical
```

Example 21 (unknown):
```unknown
'/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium/C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat_5K_filtered_raw.h5ad'
```

Example 22 (python):
```python
adata.X.data[:5]
```

Example 23 (unknown):
```unknown
array([5., 1., 1., 2., 2.], dtype=float32)
```

---

## Read in adata¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/Xenium/no2_integration_and_cell-type-annotation/no05_clustering-umap_per-mid.html

**Contents:**
- Read in adata¶
- Log-normalisation¶
- Save UMAP and leiden clustering results¶

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os
import scanpy.external as sce
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/session_info/main.py:213: UserWarning: The '__version__' attribute is deprecated and will be removed in MarkupSafe 3.1. Use feature detection, or `importlib.metadata.version("markupsafe")`, instead.
  mod_version = _find_version(mod.__version__)
```

Example 4 (python):
```python
-----
matplotlib          3.10.1
numpy               2.1.3
pandas              2.2.3
scanpy              1.11.1
seaborn             0.13.2
session_info        v1.0.1
-----
```

Example 5 (python):
```python
PIL                 11.1.0
anndata             0.11.4
asciitree           NA
asttokens           NA
charset_normalizer  3.4.1
cloudpickle         3.1.1
comm                0.2.2
cycler              0.12.1
cython_runtime      NA
dask                2024.11.2
dateutil            2.9.0.post0
debugpy             1.8.13
decorator           5.2.1
exceptiongroup      1.2.2
executing           2.2.0
h5py                3.13.0
igraph              0.11.8
importlib_metadata  NA
ipykernel           6.29.5
jedi                0.19.2
jinja2              3.1.6
joblib              1.4.2
kiwisolver          1.4.8
legacy_api_wrap     NA
leidenalg           0.10.2
llvmlite            0.44.0
markupsafe          3.0.2
mpl_toolkits        NA
msgpack             1.1.0
natsort             8.4.0
numba               0.61.0
numcodecs           0.13.1
packaging           24.2
parso               0.8.4
patsy               1.0.1
platformdirs        4.3.7
prompt_toolkit      3.0.50
psutil              7.0.0
pure_eval           0.2.3
pyarrow             19.0.1
pydev_ipython       NA
pydevconsole        NA
pydevd              3.2.3
pydevd_file_utils   NA
pydevd_plugins      NA
pydevd_tracing      NA
pygments            2.19.1
pyparsing           3.2.3
pytz                2025.2
scipy               1.15.2
six                 1.17.0
sklearn             1.5.2
stack_data          0.6.3
statsmodels         0.14.4
tblib               3.1.0
texttable           1.7.0
threadpoolctl       3.6.0
tlz                 1.0.0
toolz               1.0.0
tornado             6.4.2
traitlets           5.14.3
typing_extensions   NA
wcwidth             0.2.13
yaml                6.0.2
zarr                2.18.3
zipp                NA
zmq                 26.3.0
zoneinfo            NA
```

Example 6 (unknown):
```unknown
-----
IPython             8.34.0
jupyter_client      8.6.3
jupyter_core        5.7.2
-----
Python 3.10.14 | packaged by conda-forge | (main, Mar 20 2024, 12:45:18) [GCC 12.3.0]
Linux-4.18.0-553.45.1.el8_10.x86_64-x86_64-with-glibc2.28
-----
Session information updated at 2025-05-30 00:17
```

Example 7 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 8 (python):
```python
objects_dir = '/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium'
path_adata = f'{objects_dir}/C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat_5K_filtered_raw.h5ad'
adata = sc.read_h5ad(path_adata)
adata
```

Example 9 (unknown):
```unknown
AnnData object with n_obs × n_vars = 362277 × 5001
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_3.0', 'multi_celltypes_coarse'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_mid_fin_colors', 'dendrogram_pca_harmony_leiden_1.0', 'multi_celltypes_coarse_colors', 'pca', 'pca_harmony', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.2_colors', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.3_colors', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.4_colors', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.5_colors', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_0.8_colors', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.0_colors', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_1.5_colors', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_2.0_colors', 'pca_harmony_leiden_3.0', 'pca_harmony_leiden_3.0_colors', 'tissue_block_id_colors', 'umap'
    obsm: 'X_pca', 'X_pca_harmony', 'X_umap_pca_harmony', 'spatial'
    varm: 'PCs'
    obsp: 'pca_harmony_connectivities', 'pca_harmony_distances'
```

Example 10 (unknown):
```unknown
os.getcwd()
```

Example 11 (unknown):
```unknown
'/rfs/project/rfs-iCNyzSAaucw/kk837/notebooks/Foetal/Xenium/5K/C194_Hst45/CamU'
```

Example 12 (python):
```python
adata.X.data[:5]
```

Example 13 (unknown):
```unknown
array([5., 1., 1., 2., 2.], dtype=float32)
```

Example 14 (python):
```python
latent_space = 'pca_harmony'
sc.pl.embedding(adata,
                    basis=f"X_umap_{latent_space}",
                    color='celltypist_mid_fin',
                   )
```

Example 15 (python):
```python
adata.obs['celltypist_mid_fin'].value_counts()
```

Example 16 (unknown):
```unknown
celltypist_mid_fin
VentricularCardiomyocytes      188122
Fibroblasts                     49242
AtrialCardiomyocytes            47076
BloodVesselEndothelialCells     20055
MuralCells                      10986
PericardialCells                 8698
unassigned                       7967
Glia                             6976
EpicardialCells                  5206
EndocardialCells                 5202
MyeloidCells                     3965
LymphaticEndothelialCells        3674
LymphoidCells                    3116
Neurons                          1992
Name: count, dtype: int64
```

Example 17 (python):
```python
# log-normalise the count
sc.pp.normalize_total(adata,target_sum=1e4)
sc.pp.log1p(adata)
adata.X.data[:5]
```

Example 18 (unknown):
```unknown
array([4.2093534, 2.6576407, 2.6576407, 3.3151019, 3.3151019],
      dtype=float32)
```

Example 19 (python):
```python
adata_dir = '/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium/subsets'
```

Example 20 (python):
```python
# adata.uns['log1p']['base'] = None
```

Example 21 (python):
```python
%%time
for celltype in adata.obs['celltypist_mid_fin'].unique():
    if celltype == 'unassigned':
        continue
    
    print(f'##### {celltype} #####')
    # subset
    adata_sub = adata[adata.obs['celltypist_mid_fin']==celltype]
    sc.pp.filter_genes(adata_sub, min_cells=3)
    print(adata_sub.shape)
    # HVGs
    sc.pp.highly_variable_genes(adata_sub, flavor='seurat', min_mean=0.0125, max_mean=3, min_disp=0.5)
    print(f'number of hvgs: {sum(adata_sub.var.highly_variable)}')
    # PCA
    sc.tl.pca(adata_sub, use_highly_variable=True)
    # harmony integration
    sce.pp.harmony_integrate(adata_sub, 'tissue_block_id', basis='X_pca', adjusted_basis='X_pca_harmony')
    # create neighbour graph
    latent_space = 'pca_harmony'
    sc.pp.neighbors(adata_sub,n_neighbors=15,use_rep=f"X_{latent_space}",key_added=latent_space)
    # visualisation with umap
    sc.tl.umap(adata_sub,min_dist=0.5,neighbors_key=latent_space)
    adata_sub.obsm[f"X_umap_{latent_space}"]=adata_sub.obsm["X_umap"]
    del adata_sub.obsm["X_umap"]
    sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                    color=['multi_celltypes_coarse',"conf_score_midmod2fine","celltypist_midmod2fine",],
                    wspace=0.2,cmap='RdPu',vmax='p100')
    # leiden clustering
    resolutions_list = [0.2,0.4,0.6,1]
    for resolution in resolutions_list:
        sc.tl.leiden(adata_sub,neighbors_key=latent_space,
                     resolution=resolution,key_added=f'leiden_{str(resolution)}',n_iterations=2)
    sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                    color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                    wspace=0.3)
    # tissue id
    sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                    color=['total_counts','tissue_block_id'],
                    wspace=0.3)
    # save
    adata_sub.write(f'{adata_dir}/C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat_5K_{celltype}_lognorm.h5ad')
    print('')
```

Example 22 (unknown):
```unknown
##### Fibroblasts #####
```

Example 23 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_simple.py:287: ImplicitModificationWarning: Trying to modify attribute `.var` of view, initializing view as actual.
  adata.var["n_cells"] = number
```

Example 24 (unknown):
```unknown
(49242, 4999)
number of hvgs: 819
```

Example 25 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_pca/__init__.py:438: FutureWarning: Argument `use_highly_variable` is deprecated, consider using the mask argument. Use_highly_variable=True can be called through mask_var="highly_variable". Use_highly_variable=False can be called through mask_var=None
  warn(msg, FutureWarning)
2025-05-30 00:19:58,064 - harmonypy - INFO - Computing initial centroids with sklearn.KMeans...
2025-05-30 00:20:03,894 - harmonypy - INFO - sklearn.KMeans initialization complete.
2025-05-30 00:20:04,127 - harmonypy - INFO - Iteration 1 of 10
2025-05-30 00:20:14,361 - harmonypy - INFO - Iteration 2 of 10
2025-05-30 00:20:24,483 - harmonypy - INFO - Converged after 2 iterations
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm
```

Example 26 (unknown):
```unknown
<timed exec>:30: FutureWarning: In the future, the default backend for leiden will be igraph instead of leidenalg.

 To achieve the future defaults please pass: flavor="igraph" and n_iterations=2.  directed must also be False to work with igraph's implementation.
```

Example 27 (unknown):
```unknown
##### PericardialCells #####
(8698, 4737)
number of hvgs: 1209
```

Example 28 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_simple.py:287: ImplicitModificationWarning: Trying to modify attribute `.var` of view, initializing view as actual.
  adata.var["n_cells"] = number
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_pca/__init__.py:438: FutureWarning: Argument `use_highly_variable` is deprecated, consider using the mask argument. Use_highly_variable=True can be called through mask_var="highly_variable". Use_highly_variable=False can be called through mask_var=None
  warn(msg, FutureWarning)
2025-05-30 00:22:08,885 - harmonypy - INFO - Computing initial centroids with sklearn.KMeans...
2025-05-30 00:22:10,837 - harmonypy - INFO - sklearn.KMeans initialization complete.
2025-05-30 00:22:10,858 - harmonypy - INFO - Iteration 1 of 10
2025-05-30 00:22:11,811 - harmonypy - INFO - Iteration 2 of 10
2025-05-30 00:22:12,765 - harmonypy - INFO - Converged after 2 iterations
```

Example 29 (unknown):
```unknown
##### MuralCells #####
(10986, 4771)
number of hvgs: 1020
```

Example 30 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_simple.py:287: ImplicitModificationWarning: Trying to modify attribute `.var` of view, initializing view as actual.
  adata.var["n_cells"] = number
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_pca/__init__.py:438: FutureWarning: Argument `use_highly_variable` is deprecated, consider using the mask argument. Use_highly_variable=True can be called through mask_var="highly_variable". Use_highly_variable=False can be called through mask_var=None
  warn(msg, FutureWarning)
2025-05-30 00:22:38,155 - harmonypy - INFO - Computing initial centroids with sklearn.KMeans...
2025-05-30 00:22:40,321 - harmonypy - INFO - sklearn.KMeans initialization complete.
2025-05-30 00:22:40,360 - harmonypy - INFO - Iteration 1 of 10
2025-05-30 00:22:41,646 - harmonypy - INFO - Iteration 2 of 10
2025-05-30 00:22:42,901 - harmonypy - INFO - Converged after 2 iterations
```

Example 31 (unknown):
```unknown
##### Glia #####
(6976, 4699)
number of hvgs: 981
```

Example 32 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_simple.py:287: ImplicitModificationWarning: Trying to modify attribute `.var` of view, initializing view as actual.
  adata.var["n_cells"] = number
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_pca/__init__.py:438: FutureWarning: Argument `use_highly_variable` is deprecated, consider using the mask argument. Use_highly_variable=True can be called through mask_var="highly_variable". Use_highly_variable=False can be called through mask_var=None
  warn(msg, FutureWarning)
2025-05-30 00:23:02,139 - harmonypy - INFO - Computing initial centroids with sklearn.KMeans...
2025-05-30 00:23:04,310 - harmonypy - INFO - sklearn.KMeans initialization complete.
2025-05-30 00:23:04,336 - harmonypy - INFO - Iteration 1 of 10
2025-05-30 00:23:05,373 - harmonypy - INFO - Iteration 2 of 10
2025-05-30 00:23:06,435 - harmonypy - INFO - Converged after 2 iterations
```

Example 33 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_simple.py:287: ImplicitModificationWarning: Trying to modify attribute `.var` of view, initializing view as actual.
  adata.var["n_cells"] = number
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_pca/__init__.py:438: FutureWarning: Argument `use_highly_variable` is deprecated, consider using the mask argument. Use_highly_variable=True can be called through mask_var="highly_variable". Use_highly_variable=False can be called through mask_var=None
  warn(msg, FutureWarning)
2025-05-30 00:23:24,923 - harmonypy - INFO - Computing initial centroids with sklearn.KMeans...
```

Example 34 (unknown):
```unknown
##### Neurons #####
(1992, 4270)
number of hvgs: 1142
```

Example 35 (unknown):
```unknown
2025-05-30 00:23:26,337 - harmonypy - INFO - sklearn.KMeans initialization complete.
2025-05-30 00:23:26,343 - harmonypy - INFO - Iteration 1 of 10
2025-05-30 00:23:26,479 - harmonypy - INFO - Iteration 2 of 10
2025-05-30 00:23:26,614 - harmonypy - INFO - Converged after 2 iterations
```

Example 36 (unknown):
```unknown
##### VentricularCardiomyocytes #####
```

Example 37 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_simple.py:287: ImplicitModificationWarning: Trying to modify attribute `.var` of view, initializing view as actual.
  adata.var["n_cells"] = number
```

Example 38 (unknown):
```unknown
(188122, 5001)
number of hvgs: 607
```

Example 39 (unknown):
```unknown
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_pca/__init__.py:438: FutureWarning: Argument `use_highly_variable` is deprecated, consider using the mask argument. Use_highly_variable=True can be called through mask_var="highly_variable". Use_highly_variable=False can be called through mask_var=None
  warn(msg, FutureWarning)
2025-05-30 00:23:39,519 - harmonypy - INFO - Computing initial centroids with sklearn.KMeans...
2025-05-30 00:24:01,431 - harmonypy - INFO - sklearn.KMeans initialization complete.
2025-05-30 00:24:02,379 - harmonypy - INFO - Iteration 1 of 10
2025-05-30 00:24:53,142 - harmonypy - INFO - Iteration 2 of 10
2025-05-30 00:25:46,356 - harmonypy - INFO - Converged after 2 iterations
```

Example 40 (unknown):
```unknown
##### MyeloidCells #####
(3965, 4505)
number of hvgs: 925
```

Example 41 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_simple.py:287: ImplicitModificationWarning: Trying to modify attribute `.var` of view, initializing view as actual.
  adata.var["n_cells"] = number
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_pca/__init__.py:438: FutureWarning: Argument `use_highly_variable` is deprecated, consider using the mask argument. Use_highly_variable=True can be called through mask_var="highly_variable". Use_highly_variable=False can be called through mask_var=None
  warn(msg, FutureWarning)
2025-05-30 00:32:35,867 - harmonypy - INFO - Computing initial centroids with sklearn.KMeans...
2025-05-30 00:32:37,357 - harmonypy - INFO - sklearn.KMeans initialization complete.
2025-05-30 00:32:37,369 - harmonypy - INFO - Iteration 1 of 10
2025-05-30 00:32:37,765 - harmonypy - INFO - Iteration 2 of 10
2025-05-30 00:32:38,171 - harmonypy - INFO - Converged after 2 iterations
```

Example 42 (unknown):
```unknown
##### LymphoidCells #####
(3116, 4614)
number of hvgs: 919
```

Example 43 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_simple.py:287: ImplicitModificationWarning: Trying to modify attribute `.var` of view, initializing view as actual.
  adata.var["n_cells"] = number
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_pca/__init__.py:438: FutureWarning: Argument `use_highly_variable` is deprecated, consider using the mask argument. Use_highly_variable=True can be called through mask_var="highly_variable". Use_highly_variable=False can be called through mask_var=None
  warn(msg, FutureWarning)
2025-05-30 00:32:48,991 - harmonypy - INFO - Computing initial centroids with sklearn.KMeans...
2025-05-30 00:32:50,433 - harmonypy - INFO - sklearn.KMeans initialization complete.
2025-05-30 00:32:50,444 - harmonypy - INFO - Iteration 1 of 10
2025-05-30 00:32:50,735 - harmonypy - INFO - Iteration 2 of 10
2025-05-30 00:32:51,040 - harmonypy - INFO - Converged after 2 iterations
```

Example 44 (unknown):
```unknown
##### BloodVesselEndothelialCells #####
(20055, 4950)
```

Example 45 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_simple.py:287: ImplicitModificationWarning: Trying to modify attribute `.var` of view, initializing view as actual.
  adata.var["n_cells"] = number
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_pca/__init__.py:438: FutureWarning: Argument `use_highly_variable` is deprecated, consider using the mask argument. Use_highly_variable=True can be called through mask_var="highly_variable". Use_highly_variable=False can be called through mask_var=None
  warn(msg, FutureWarning)
```

Example 46 (unknown):
```unknown
number of hvgs: 820
```

Example 47 (unknown):
```unknown
2025-05-30 00:33:00,376 - harmonypy - INFO - Computing initial centroids with sklearn.KMeans...
2025-05-30 00:33:03,215 - harmonypy - INFO - sklearn.KMeans initialization complete.
2025-05-30 00:33:03,256 - harmonypy - INFO - Iteration 1 of 10
2025-05-30 00:33:05,943 - harmonypy - INFO - Iteration 2 of 10
2025-05-30 00:33:08,587 - harmonypy - INFO - Converged after 2 iterations
```

Example 48 (unknown):
```unknown
##### AtrialCardiomyocytes #####
```

Example 49 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_simple.py:287: ImplicitModificationWarning: Trying to modify attribute `.var` of view, initializing view as actual.
  adata.var["n_cells"] = number
```

Example 50 (unknown):
```unknown
(47076, 5000)
number of hvgs: 627
```

Example 51 (unknown):
```unknown
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_pca/__init__.py:438: FutureWarning: Argument `use_highly_variable` is deprecated, consider using the mask argument. Use_highly_variable=True can be called through mask_var="highly_variable". Use_highly_variable=False can be called through mask_var=None
  warn(msg, FutureWarning)
2025-05-30 00:33:44,353 - harmonypy - INFO - Computing initial centroids with sklearn.KMeans...
2025-05-30 00:33:49,775 - harmonypy - INFO - sklearn.KMeans initialization complete.
2025-05-30 00:33:49,906 - harmonypy - INFO - Iteration 1 of 10
2025-05-30 00:33:58,663 - harmonypy - INFO - Iteration 2 of 10
2025-05-30 00:34:06,411 - harmonypy - INFO - Converged after 2 iterations
```

Example 52 (unknown):
```unknown
##### EndocardialCells #####
(5202, 4492)
number of hvgs: 945
```

Example 53 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_simple.py:287: ImplicitModificationWarning: Trying to modify attribute `.var` of view, initializing view as actual.
  adata.var["n_cells"] = number
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_pca/__init__.py:438: FutureWarning: Argument `use_highly_variable` is deprecated, consider using the mask argument. Use_highly_variable=True can be called through mask_var="highly_variable". Use_highly_variable=False can be called through mask_var=None
  warn(msg, FutureWarning)
2025-05-30 00:35:28,149 - harmonypy - INFO - Computing initial centroids with sklearn.KMeans...
2025-05-30 00:35:29,818 - harmonypy - INFO - sklearn.KMeans initialization complete.
2025-05-30 00:35:29,833 - harmonypy - INFO - Iteration 1 of 10
2025-05-30 00:35:30,416 - harmonypy - INFO - Iteration 2 of 10
2025-05-30 00:35:31,000 - harmonypy - INFO - Converged after 2 iterations
```

Example 54 (unknown):
```unknown
##### LymphaticEndothelialCells #####
(3674, 4576)
number of hvgs: 889
```

Example 55 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_simple.py:287: ImplicitModificationWarning: Trying to modify attribute `.var` of view, initializing view as actual.
  adata.var["n_cells"] = number
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_pca/__init__.py:438: FutureWarning: Argument `use_highly_variable` is deprecated, consider using the mask argument. Use_highly_variable=True can be called through mask_var="highly_variable". Use_highly_variable=False can be called through mask_var=None
  warn(msg, FutureWarning)
2025-05-30 00:35:44,974 - harmonypy - INFO - Computing initial centroids with sklearn.KMeans...
2025-05-30 00:35:46,515 - harmonypy - INFO - sklearn.KMeans initialization complete.
2025-05-30 00:35:46,527 - harmonypy - INFO - Iteration 1 of 10
2025-05-30 00:35:46,877 - harmonypy - INFO - Iteration 2 of 10
2025-05-30 00:35:47,224 - harmonypy - INFO - Converged after 2 iterations
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/numpy/lib/_nanfunctions_impl.py:1437: RuntimeWarning: All-NaN slice encountered
  return _nanquantile_unchecked(
```

Example 56 (unknown):
```unknown
##### EpicardialCells #####
(5206, 4657)
number of hvgs: 1052
```

Example 57 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_simple.py:287: ImplicitModificationWarning: Trying to modify attribute `.var` of view, initializing view as actual.
  adata.var["n_cells"] = number
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_pca/__init__.py:438: FutureWarning: Argument `use_highly_variable` is deprecated, consider using the mask argument. Use_highly_variable=True can be called through mask_var="highly_variable". Use_highly_variable=False can be called through mask_var=None
  warn(msg, FutureWarning)
2025-05-30 00:35:57,504 - harmonypy - INFO - Computing initial centroids with sklearn.KMeans...
2025-05-30 00:35:59,232 - harmonypy - INFO - sklearn.KMeans initialization complete.
2025-05-30 00:35:59,244 - harmonypy - INFO - Iteration 1 of 10
2025-05-30 00:35:59,798 - harmonypy - INFO - Iteration 2 of 10
2025-05-30 00:36:00,348 - harmonypy - INFO - Converged after 2 iterations
```

Example 58 (unknown):
```unknown
CPU times: user 19min 17s, sys: 21.2 s, total: 19min 39s
Wall time: 16min 21s
```

---

## Import modules¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig4_Cardiomyocytes/VentricularCardiomyocytes/TransmuralAxis/tm_axis_nuclei-density.html

**Contents:**
- Import modules¶
- Functions¶
- Read in adata¶
- Add axis¶
- Load calculated image features and add to anndata¶
- Plot¶

**Examples:**

Example 1 (python):
```python
import os
import numpy as np
import scanpy as sc
import anndata
import pandas as pd
import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 
import matplotlib.pyplot as plt
import seaborn as sns

import squidpy as sq

import warnings
warnings.simplefilter(action='ignore', category=FutureWarning)
warnings.simplefilter("ignore", UserWarning)
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
-----
anndata             0.9.2
matplotlib          3.7.2
numpy               1.22.4
pandas              2.0.3
scanpy              1.9.4
seaborn             0.12.2
session_info        1.0.0
squidpy             1.2.3
-----
```

Example 4 (python):
```python
PIL                 10.0.0
asciitree           NA
asttokens           NA
backcall            0.2.0
cloudpickle         2.2.1
comm                0.1.4
cycler              0.10.0
cython_runtime      NA
dask                2023.5.0
dask_image          2023.03.0
dateutil            2.8.2
debugpy             1.7.0
decorator           5.1.1
docrep              0.3.2
entrypoints         0.4
executing           1.2.0
fasteners           0.18
h5py                3.9.0
igraph              0.10.6
imagecodecs         2023.3.16
imageio             2.31.3
importlib_metadata  NA
importlib_resources NA
ipykernel           6.25.2
jedi                0.19.0
jinja2              3.1.2
joblib              1.3.2
kiwisolver          1.4.5
lazy_loader         NA
leidenalg           0.10.1
llvmlite            0.38.1
markupsafe          2.1.3
matplotlib_scalebar 0.8.1
mpl_toolkits        NA
mpmath              1.3.0
natsort             8.4.0
networkx            3.1
numba               0.55.2
numcodecs           0.11.0
nvfuser             NA
packaging           23.1
parso               0.8.3
patsy               0.5.3
pexpect             4.8.0
pickleshare         0.7.5
pkg_resources       NA
platformdirs        3.10.0
prompt_toolkit      3.0.39
psutil              5.9.5
ptyprocess          0.7.0
pure_eval           0.2.2
pydev_ipython       NA
pydevconsole        NA
pydevd              2.9.5
pydevd_file_utils   NA
pydevd_plugins      NA
pydevd_tracing      NA
pygments            2.16.1
pyparsing           3.0.9
pytz                2023.3.post1
pywt                1.4.1
scipy               1.10.1
six                 1.16.0
skimage             0.21.0
sklearn             1.3.0
stack_data          0.6.2
statsmodels         0.14.0
sympy               1.12
texttable           1.6.7
threadpoolctl       3.2.0
tifffile            2023.7.10
tlz                 0.12.0
toolz               0.12.0
torch               2.0.1+cu117
tornado             6.3.3
tqdm                4.66.1
traitlets           5.9.0
typing_extensions   NA
validators          0.22.0
wcwidth             0.2.6
xarray              2023.1.0
yaml                6.0.1
zarr                2.16.1
zipp                NA
zmq                 25.1.1
```

Example 5 (unknown):
```unknown
-----
IPython             8.12.2
jupyter_client      8.3.1
jupyter_core        5.3.1
-----
Python 3.8.17 (default, Jul  5 2023, 21:04:15) [GCC 11.2.0]
Linux-5.4.0-169-generic-x86_64-with-glibc2.17
-----
Session information updated at 2024-02-13 15:12
```

Example 6 (python):
```python
sc.settings.set_figure_params(dpi=80,dpi_save=300)
```

Example 7 (python):
```python
import importlib.util
import sys

spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)

spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/tissueaxis.py")
tissueaxis = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = tissueaxis
spec.loader.exec_module(tissueaxis)
```

Example 8 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk'
```

Example 9 (python):
```python
# This function selects the data for multiple slides from the spatial anndata object.
# Modified `select_slide` in cell2loc for multiple slides

# @param adata: anndata object with multiple spatial experiments
# @param s: list of selected slides
# @param s_col: column in adata.obs listing experiment name for each location

def select_multislide(adata, s, s_col='sample'):

    slides = adata[adata.obs[s_col].isin(s), :]
    s_keys = list(slides.uns['spatial'].keys())
    
    uns_dict = {}
    for ss in s:
        s_spatial=np.array(s_keys)[[ss in k for k in s_keys]][0]
        uns_dict[s_spatial]=slides.uns['spatial'][s_spatial]
    
    slides.uns['spatial'] = uns_dict

    return slides
```

Example 10 (python):
```python
adata_oct_healthy = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/visium-OCT_foetal_raw.h5ad')
print(adata_oct_healthy.shape)
adata_oct_healthy_LV = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/visium-OCT-LV_foetal_raw.h5ad')
print(adata_oct_healthy_LV.shape)
adata_oct_t21 = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/visium-OCT-T21_foetal_raw.h5ad')
print(adata_oct_t21.shape)
```

Example 11 (unknown):
```unknown
(14261, 33538)
(6767, 33538)
(11238, 33538)
```

Example 12 (python):
```python
# concatenate oct adata
adata_oct_healthy_LV.obs['status'] = 'Healthy'
adata_oct_t21.obs['status'] = 'Trisomy21'

adata_oct = adata_oct_healthy.concatenate(
    [adata_oct_healthy_LV,adata_oct_t21],
    batch_key=None,
    index_unique=None,
    join="inner"
)

adata_oct.obs['status'].value_counts()
```

Example 13 (unknown):
```unknown
status
Healthy      21028
Trisomy21    11238
Name: count, dtype: int64
```

Example 14 (python):
```python
# add
oct_samples = list(set(adata_oct.obs['sample']))

adata_oct.obs['transmural_axis'] = np.nan

axis_files = os.listdir('/nfs/team205/heart/visium_axis/foetal/transmural_LV')
for file in axis_files:
    # read in
    axis = pd.read_csv(f'/nfs/team205/heart/visium_axis/foetal/transmural_LV/{file}',index_col=0)
    # add to anndata
    if file.split('.csv')[0] in oct_samples:
        adata_oct.obs.loc[axis.index,'transmural_axis'] = axis.copy()

# select spots with axis
adata_oct = adata_oct[adata_oct.obs['transmural_axis'].isna()==False]
```

Example 15 (python):
```python
# remove nuclei segmentation failed samples: 20PCW samples
adata_oct = adata_oct[adata_oct.obs['sample'].isin(['HCAHeartST13162330','HCAHeartST13162331'])==False]
```

Example 16 (unknown):
```unknown
!ls /nfs/team205/heart/visium_axis/foetal/nuclei_segmentation/calculated_img_features
```

Example 17 (unknown):
```unknown
BHF_F_Hea13295830_img_features.csv   HCAHeartST11605167_img_features.csv
BHF_F_Hea13295831_img_features.csv   HCAHeartST11605168_img_features.csv
HCAHeartST10317184_img_features.csv  HCAHeartST13162330_img_features.csv
HCAHeartST10317185_img_features.csv  HCAHeartST13162331_img_features.csv
HCAHeartST10317186_img_features.csv  HCAHeartST13162336_img_features.csv
HCAHeartST11605165_img_features.csv
```

Example 18 (unknown):
```unknown
path_to_calculated_img_features = '/nfs/team205/heart/visium_axis/foetal/nuclei_segmentation/calculated_img_features'
img_feature_files = os.listdir(path_to_calculated_img_features)
len(img_feature_files)
```

Example 19 (python):
```python
for file in img_feature_files:
    # read in
    features = pd.read_csv(f'{path_to_calculated_img_features}/{file}',index_col=0)
    # add
    if file.split('_img')[0] in ['BHF_F_Hea13295830','BHF_F_Hea13295831']:
        continue
    else:
        cell_ids = list(set(features.index).intersection(adata_oct.obs_names))
        adata_oct.obs.loc[cell_ids,features.columns] = features

# checking whether all cells have feature
print(sum(adata_oct.obs['segmentation_label'].isna())==0)
```

Example 20 (unknown):
```unknown
features.head()
```

Example 21 (python):
```python
plt.rcParams['figure.figsize'] = 4.5,2
adata_oct.obs['segmentation_label'].hist(bins=100)
```

Example 22 (unknown):
```unknown
## remove most outer and inner bins, since those are not myocardium
# ad = ad[ad.obs["transmural_axis_bin"]!=ad.obs['transmural_axis_bin'].max()]
# ad = ad[ad.obs["transmural_axis_bin"]!=ad.obs['transmural_axis_bin'].min()]

# multiple by two for the FFPE samples since the thickness is 10um whereas OCT samples are 5um
# cell_id = ad.obs_names[ad.obs['cell2loc-map_group']=='16PCW_FFPE']
# ad.obs.loc[cell_id,'segmentation_label'] = ad.obs.loc[cell_id,'segmentation_label']*2
```

Example 23 (python):
```python
adata_all.obs['cell2loc-map_group'].value_counts()
```

Example 24 (unknown):
```unknown
cell2loc-map_group
16PCW_FFPE    4605
T21_13PCW     1940
13PCW         1837
7PCW           341
5PCW           293
Name: count, dtype: int64
```

Example 25 (python):
```python
tissueaxis.plot_axis_split(adata_all,
                           feature_to_plot='segmentation_label',
                           feature_type='obs',
                           min_max_scale=False,
                           splitby='cell2loc-map_group',
                           groups_order=[ '5PCW', '7PCW','13PCW','T21_13PCW'],
                           figsize=(5, 4),
                           cbar_pos=(0, 0.7, 0.05, 0.18),
                           cmap='Reds',
                           # save=f'{figdir}/tm-axis_{features_to_plot}_per-week.pdf'
                          )
```

Example 26 (python):
```python
tissueaxis.plot_axis_split(adata_all,
                           feature_to_plot='segmentation_label',
                           feature_type='obs',
                           min_max_scale=False,
                           splitby='cell2loc-map_group',
                           groups_order=[ '5PCW', '7PCW','13PCW'],
                           figsize=(5, 3.25),
                           cbar_pos=(0, 0.7, 0.03, 0.18),
                           cmap='Reds',
                           save=f'{figdir}/tm-axis_nuclei-segmentation_per-week.pdf'
                          )
```

---

## Read in¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/ATAC/ATAC_preprocess/anndata_normPeakMatrix_to_anndata.html

**Contents:**
- Read in¶
- Make anndata¶
- Add cell type annotation¶
- Refine metadata¶

ref: is peak matrix extracted with getMatrixFromProject normalized? NO https://github.com/GreenleafLab/ArchR/discussions/943

508040 rows × 0 columns

167022 rows × 0 columns

167022 rows × 0 columns

**Examples:**

Example 1 (python):
```python
import scanpy as sc
import pandas as pd
import numpy as np
import anndata

from scipy.io import mmwrite, mmread
from scipy.sparse import csr_matrix
```

Example 2 (unknown):
```unknown
!ls -l /nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/PeakMatrix
```

Example 3 (python):
```python
total 46364348
-rw-r--r-- 1 20383 1000624     1492397 Mar 12 20:40 barcodes.tsv.gz
-rw-r--r-- 1 20383 1000624     4082604 Mar 12 20:40 features.tsv.gz
-rw-r--r-- 1 20383 1000624  3579115352 Mar 19 18:39 Foetal_Cardiomyocytes_Peaks_post-peakVI.h5ad
drwxr-xr-x 1 20383 1000624          26 Mar 15 22:12 Foetal_Cardiomyocytes_peakVI-model
-rw-r--r-- 1 20383 1000624   538152433 Mar 19 18:40 Foetal_Endothelium_Peaks_post-peakVI.h5ad
drwxr-xr-x 1 20383 1000624          26 Mar 15 19:21 Foetal_Endothelium_peakVI-model
-rw-r--r-- 1 20383 1000624   163683392 Mar 19 18:41 Foetal_Epicardium_Peaks_post-peakVI.h5ad
drwxr-xr-x 1 20383 1000624          26 Mar 15 19:29 Foetal_Epicardium_peakVI-model
-rw-r--r-- 1 20383 1000624  1000890284 Mar 19 18:42 Foetal_Leukocytes_Peaks_post-peakVI.h5ad
drwxr-xr-x 1 20383 1000624          26 Mar 15 23:29 Foetal_Leukocytes_peakVI-model
-rw-r--r-- 1 20383 1000624  4253490105 Mar 19 18:48 Foetal_Mesenchymal_Peaks_post-peakVI.h5ad
drwxr-xr-x 1 20383 1000624          26 Mar 15 18:34 Foetal_Mesenchymal_peakVI-model
drwxr-xr-x 1 20383 1000624          26 Mar 25 22:12 Foetal_MP_peakVI-model
-rw-r--r-- 1 20383 1000624   235356930 Mar 19 18:48 Foetal_Neural_Peaks_post-peakVI.h5ad
drwxr-xr-x 1 20383 1000624          26 Mar 15 18:48 Foetal_Neural_peakVI-model
-rw-r--r-- 1 20383 1000624   611045376 Apr  5 13:29 Foetal_Peaks.h5ad
-rw-r--r-- 1 20383 1000624  9477510996 Mar 14 08:12 Foetal_Peaks_post-peakVI.h5ad
-rw-r--r-- 1 20383 1000624  9459770872 Mar 30 16:37 Foetal_Peaks_tfidf.h5ad
drwxr-xr-x 1 20383 1000624          26 Mar 14 08:12 Foetal_peakVI-model
-rw-r--r-- 1 20383 1000624  3755691976 Mar 12 20:40 matrix.mtx.gz
-rw-r--r-- 1 20383 1000624 14315705883 Mar 30 13:54 matrix_tfidf.mtx.gz
-rw-r--r-- 1 20383 1000624    81051521 Apr  4 08:21 peak_metadata.tsv
```

Example 4 (unknown):
```unknown
peakmatrix_dir = '/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/PeakMatrix'
```

Example 5 (unknown):
```unknown
# peak matrix, not-binarised
mat = mmread(f'{peakmatrix_dir}/matrix_tfidf.mtx.gz')
mat
```

Example 6 (python):
```python
<508040x167022 sparse matrix of type '<class 'numpy.float64'>'
	with 1173033588 stored elements in COOrdinate format>
```

Example 7 (python):
```python
# peak names
features = pd.read_csv(f'{peakmatrix_dir}/features.tsv.gz', header=None, sep='\t')[[0]]
features.set_index(0,inplace=True)
features
```

Example 8 (python):
```python
# cell barcodes
cells = pd.read_csv(f'{peakmatrix_dir}/barcodes.tsv.gz', header=None, sep='\t')[[1]]
cells.set_index(1,inplace=True)
cells
```

Example 9 (python):
```python
adata = anndata.AnnData(
    X=mat.T,
    obs=cells,
    var=features
)

# rename obs_names
adata.obs_names = [x.replace('#','_') for x in adata.obs_names]

# convert sparce matrix
adata.X = adata.X.tocsr()
adata.X
```

Example 10 (python):
```python
<167022x508040 sparse matrix of type '<class 'numpy.float32'>'
	with 1173033588 stored elements in Compressed Sparse Row format>
```

Example 11 (unknown):
```unknown
AnnData object with n_obs × n_vars = 167022 × 508040
```

Example 12 (python):
```python
adata.X.data[:30]
```

Example 13 (unknown):
```unknown
array([1.768611 , 1.9229858, 3.635446 , 1.3192275, 1.4049321, 1.8123279,
       1.6739427, 2.9703567, 3.3852873, 1.2617257, 2.8007214, 0.7237267,
       1.4759332, 3.1692483, 3.266629 , 3.2549186, 3.8324249, 2.7257555,
       3.126333 , 3.7325323, 3.785544 , 4.073427 , 4.067536 , 4.1232815,
       3.4213552, 3.0830379, 3.7987761, 1.0650452, 3.5462437, 0.8590813],
      dtype=float32)
```

Example 14 (python):
```python
np.max(adata.X.data)
```

Example 15 (python):
```python
adata.obs.drop[]
```

Example 16 (python):
```python
obs = pd.read_csv('/nfs/team205/heart/anndata_objects/Foetal/Feb28ObjectRaw.obs.csv',index_col=0)

# add cell type labels and other metadata
metadata_to_add = ['sangerID', 'combinedID', 'donor', 
       'facility', 'cell_or_nuclei', 'modality', 'kit_10x','multiplexed',
        'batch_key','region','FACSgate','sex','week','trimester',
                     'fine_grain','mid_grain','coarse_grain',
                     'heart_or_greatvessels','cycling','S_score','G2M_score','phase']
adata.obs[metadata_to_add] = obs[metadata_to_add].reindex(adata.obs_names).copy()
adata
```

Example 17 (python):
```python
/opt/conda/envs/teichlab/lib/python3.8/site-packages/IPython/core/interactiveshell.py:3165: DtypeWarning: Columns (31) have mixed types.Specify dtype option on import or set low_memory=False.
  has_raised = await self.run_ast_nodes(code_ast.body, cell_name,
```

Example 18 (unknown):
```unknown
AnnData object with n_obs × n_vars = 167022 × 508040
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score'
```

Example 19 (python):
```python
adata.obs['modality'] = 'Multiome_ATAC'
```

Example 20 (python):
```python
adata.var.rename_axis('Peaks', axis=0, inplace=True)
```

Example 21 (python):
```python
# save
adata.write(f'{peakmatrix_dir}/Foetal_Peaks_tfidf.h5ad')
```

Example 22 (unknown):
```unknown
... storing 'sangerID' as categorical
... storing 'combinedID' as categorical
... storing 'donor' as categorical
... storing 'facility' as categorical
... storing 'cell_or_nuclei' as categorical
... storing 'modality' as categorical
... storing 'kit_10x' as categorical
... storing 'multiplexed' as categorical
... storing 'batch_key' as categorical
... storing 'region' as categorical
... storing 'FACSgate' as categorical
... storing 'sex' as categorical
... storing 'week' as categorical
... storing 'trimester' as categorical
... storing 'fine_grain' as categorical
... storing 'mid_grain' as categorical
... storing 'coarse_grain' as categorical
... storing 'heart_or_greatvessels' as categorical
... storing 'cycling' as categorical
... storing 'phase' as categorical
```

Example 23 (unknown):
```unknown
!ls -l /nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/PeakMatrix
```

Example 24 (unknown):
```unknown
total 54926532
-rw-r--r-- 1 20383 1000624     1492397 Mar 12 20:40 barcodes.tsv.gz
-rw-r--r-- 1 20383 1000624     4082604 Mar 12 20:40 features.tsv.gz
-rw-r--r-- 1 20383 1000624  3579115352 Mar 19 18:39 Foetal_Cardiomyocytes_Peaks_post-peakVI.h5ad
drwxr-xr-x 1 20383 1000624          26 Mar 15 22:12 Foetal_Cardiomyocytes_peakVI-model
-rw-r--r-- 1 20383 1000624   538152433 Mar 19 18:40 Foetal_Endothelium_Peaks_post-peakVI.h5ad
drwxr-xr-x 1 20383 1000624          26 Mar 15 19:21 Foetal_Endothelium_peakVI-model
-rw-r--r-- 1 20383 1000624   163683392 Mar 19 18:41 Foetal_Epicardium_Peaks_post-peakVI.h5ad
drwxr-xr-x 1 20383 1000624          26 Mar 15 19:29 Foetal_Epicardium_peakVI-model
-rw-r--r-- 1 20383 1000624  1000890284 Mar 19 18:42 Foetal_Leukocytes_Peaks_post-peakVI.h5ad
drwxr-xr-x 1 20383 1000624          26 Mar 15 23:29 Foetal_Leukocytes_peakVI-model
-rw-r--r-- 1 20383 1000624  4253490105 Mar 19 18:48 Foetal_Mesenchymal_Peaks_post-peakVI.h5ad
drwxr-xr-x 1 20383 1000624          26 Mar 15 18:34 Foetal_Mesenchymal_peakVI-model
drwxr-xr-x 1 20383 1000624          26 Mar 25 22:12 Foetal_MP_peakVI-model
-rw-r--r-- 1 20383 1000624   235356930 Mar 19 18:48 Foetal_Neural_Peaks_post-peakVI.h5ad
drwxr-xr-x 1 20383 1000624          26 Mar 15 18:48 Foetal_Neural_peakVI-model
-rw-r--r-- 1 20383 1000624  9459770872 Mar 13 00:38 Foetal_Peaks.h5ad
-rw-r--r-- 1 20383 1000624  9477510996 Mar 14 08:12 Foetal_Peaks_post-peakVI.h5ad
-rw-r--r-- 1 20383 1000624  9459770872 Mar 30 16:37 Foetal_Peaks_tfidf.h5ad
drwxr-xr-x 1 20383 1000624          26 Mar 14 08:12 Foetal_peakVI-model
-rw-r--r-- 1 20383 1000624  3755691976 Mar 12 20:40 matrix.mtx.gz
-rw-r--r-- 1 20383 1000624 14315705883 Mar 30 13:54 matrix_tfidf.mtx.gz
```

Example 25 (python):
```python
adata_test = sc.read_h5ad(f'{peakmatrix_dir}/Foetal_Peaks_tfidf.h5ad')
adata_test
```

Example 26 (unknown):
```unknown
AnnData object with n_obs × n_vars = 167022 × 508040
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score'
```

Example 27 (python):
```python
adata_test.X.data[:30]
```

Example 28 (unknown):
```unknown
array([1.768611 , 1.9229858, 3.635446 , 1.3192275, 1.4049321, 1.8123279,
       1.6739427, 2.9703567, 3.3852873, 1.2617257, 2.8007214, 0.7237267,
       1.4759332, 3.1692483, 3.266629 , 3.2549186, 3.8324249, 2.7257555,
       3.126333 , 3.7325323, 3.785544 , 4.073427 , 4.067536 , 4.1232815,
       3.4213552, 3.0830379, 3.7987761, 1.0650452, 3.5462437, 0.8590813],
      dtype=float32)
```

Example 29 (python):
```python
adata = sc.read_h5ad(f'{peakmatrix_dir}/Foetal_Peaks_tfidf.h5ad')
adata
```

Example 30 (unknown):
```unknown
AnnData object with n_obs × n_vars = 167022 × 508040
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score'
```

Example 31 (python):
```python
# drop some obs columns
metadata_to_drop = [
       'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts',
       'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts',
       'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt',
       'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo',
       'pct_counts_ribo', 'HB_score', 'cycling','S_score','G2M_score','phase','stress_score','hb1_score']

adata.obs.drop(metadata_to_drop,axis=1,inplace=True)
```

Example 32 (python):
```python
# add peak metadata
meta = pd.read_csv('/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/PeakMatrix/peak_metadata.tsv',
           sep='\t')
meta.set_index('range',inplace=True)

# add to anndata
adata.var[meta.columns] = meta.reindex(adata.var_names)
adata.var.head()
```

Example 33 (python):
```python
# save
adata.write(f'{peakmatrix_dir}/Foetal_Peaks_tfidf.h5ad')
```

Example 34 (unknown):
```unknown
... storing 'seqnames' as categorical
... storing 'strand' as categorical
... storing 'GroupReplicate' as categorical
... storing 'nearestGene' as categorical
... storing 'peakType' as categorical
... storing 'nearestTSS' as categorical
```

Example 35 (python):
```python
adata.X.data[:30]
```

Example 36 (unknown):
```unknown
array([1.768611 , 1.9229858, 3.635446 , 1.3192275, 1.4049321, 1.8123279,
       1.6739427, 2.9703567, 3.3852873, 1.2617257, 2.8007214, 0.7237267,
       1.4759332, 3.1692483, 3.266629 , 3.2549186, 3.8324249, 2.7257555,
       3.126333 , 3.7325323, 3.785544 , 4.073427 , 4.067536 , 4.1232815,
       3.4213552, 3.0830379, 3.7987761, 1.0650452, 3.5462437, 0.8590813],
      dtype=float32)
```

Example 37 (unknown):
```unknown
f'{peakmatrix_dir}/Foetal_Peaks_tfidf.h5ad'
```

Example 38 (unknown):
```unknown
'/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/PeakMatrix/Foetal_Peaks_tfidf.h5ad'
```

Example 39 (unknown):
```unknown
# rclone to gdrive
rclone copy /nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/PeakMatrix/Foetal_Peaks_tfidf.h5ad \
gdrive:JamesC_Kazumasa/Foetal_Heart/SanjayShinha/toSinhalab/anndata/Peak/
```

---

## Read in adata¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/Xenium/no2_integration_and_cell-type-annotation/no8_plotting-Copy1.html

**Contents:**
- Read in adata¶
- Log-normalisation¶
- Plot cell types¶
- Plot genes¶
- Plot LR¶
- Plot axonal genes¶

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os
import scanpy.external as sce
```

Example 2 (python):
```python
# import session_info
# session_info.show()
```

Example 3 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 4 (python):
```python
path_adata = '/lustre/scratch126/cellgen/team205/kk18/xenium/objects/foetal/C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat_5K_filtered_raw.h5ad'
adata = sc.read_h5ad(path_adata)
adata
```

Example 5 (unknown):
```unknown
AnnData object with n_obs × n_vars = 362277 × 5001
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'fine_grain_8Dec2024', 'multi_celltypes_coarse', 'cluster_cellcharter', 'fine_grain_4Jan2025', 'sliding_window_assignment', 'globalX', 'globalY', 'predicted_labels_sliding_window_assignment', 'conf_score_sliding_window_assignment'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1'
    uns: 'celltypist_mid_fin_colors', 'fine_grain_4Jan2025_colors', 'fine_grain_8Dec2024_colors', 'multi_celltypes_coarse_colors', 'pca', 'pca_harmony', 'spatial_neighbors', 'tissue_block_id_colors', 'umap'
    obsm: 'X_cellcharter', 'X_pca', 'X_pca_harmony', 'X_umap_pca', 'X_umap_pca_harmony', 'spatial'
    varm: 'PCs'
    obsp: 'pca_harmony_connectivities', 'pca_harmony_distances', 'spatial_connectivities', 'spatial_distances'
```

Example 6 (unknown):
```unknown
os.getcwd()
```

Example 7 (unknown):
```unknown
'/nfs/team205/kk18/notebooks/Foetal/Xenium/5K/C194_Hst45'
```

Example 8 (python):
```python
adata.X.data[:5]
```

Example 9 (unknown):
```unknown
array([5., 1., 1., 2., 2.], dtype=float32)
```

Example 10 (python):
```python
# log-normalise the count
sc.pp.normalize_total(adata,target_sum=1e4)
sc.pp.log1p(adata)
adata.X.data[:5]
```

Example 11 (unknown):
```unknown
array([4.2093534, 2.6576407, 2.6576407, 3.3151019, 3.3151019],
      dtype=float32)
```

Example 12 (python):
```python
tissue_id = 'C194-HEA-0-FFPE-1'
slide = adata[adata.obs['tissue_block_id']==tissue_id]
sc.pl.spatial(slide,color=['fine_grain_8Dec2024'],
              groups=['SinoatrialNodePacemakerCells','ParasympatheticNeurons','SympatheticNeurons'],
              spot_size=60,vmax='p100',
             )
```

Example 13 (python):
```python
tissue_id = 'C194-HEA-0-FFPE-1'
slide = adata[adata.obs['tissue_block_id']==tissue_id]
sc.pl.spatial(slide,color=['fine_grain_8Dec2024'],
              groups=['SinoatrialNodePacemakerCells','AtrioventricularNodePacemakerCells','ParasympatheticNeurons','MacrophagesCX3CR1pos','MacrophagesLYVE1pos'],
              spot_size=20,vmax='p100',
              crop_coord=(4500,7000,4000,6000)
             )
```

Example 14 (python):
```python
tissue_id = 'C194-HEA-0-FFPE-1'
celltype_key = 'fine_grain_8Dec2024'
slide = adata[adata.obs['tissue_block_id']==tissue_id]

slide.uns[f"{celltype_key}_colors"][list(slide.obs[celltype_key].cat.categories).index('MacrophagesCX3CR1pos')]='blue'
slide.uns[f"{celltype_key}_colors"][list(slide.obs[celltype_key].cat.categories).index('MacrophagesLYVE1pos')]='orange'

sc.pl.spatial(slide,color=[celltype_key],
              groups=['SinoatrialNodePacemakerCells','ParasympatheticNeurons','MacrophagesCX3CR1pos','MacrophagesLYVE1pos'],
              spot_size=15,vmax='p100',
              crop_coord=(4600,5600,4250,5250)
             )
```

Example 15 (python):
```python
tissue_id = 'C194-HEA-0-FFPE-1'
celltype_key = 'fine_grain_8Dec2024'
slide = adata[adata.obs['tissue_block_id']==tissue_id]

slide.uns[f"{celltype_key}_colors"][list(slide.obs[celltype_key].cat.categories).index('ParasympatheticNeurons')]='springgreen'
slide.uns[f"{celltype_key}_colors"][list(slide.obs[celltype_key].cat.categories).index('SinoatrialNodePacemakerCells')]='hotpink'
slide.uns[f"{celltype_key}_colors"][list(slide.obs[celltype_key].cat.categories).index('SchwannCellPrecursors')]='blue'
slide.uns[f"{celltype_key}_colors"][list(slide.obs[celltype_key].cat.categories).index('SchwannCells')]='orange'

sc.pl.spatial(slide,color=[celltype_key],
              groups=['SinoatrialNodePacemakerCells','ParasympatheticNeurons','SchwannCells','SchwannCellPrecursors'],
              spot_size=15,vmax='p100',
              crop_coord=(4600,5600,4250,5250)
             )
```

Example 16 (python):
```python
tissue_id = 'C194-HEA-0-FFPE-1'
celltype_key = 'fine_grain_8Dec2024'
slide = adata[adata.obs['tissue_block_id']==tissue_id]

slide.uns[f"{celltype_key}_colors"][list(slide.obs[celltype_key].cat.categories).index('ParasympatheticNeurons')]='springgreen'
slide.uns[f"{celltype_key}_colors"][list(slide.obs[celltype_key].cat.categories).index('SinoatrialNodePacemakerCells')]='hotpink'
slide.uns[f"{celltype_key}_colors"][list(slide.obs[celltype_key].cat.categories).index('SchwannCellPrecursors')]='blue'
# slide.uns[f"{celltype_key}_colors"][list(slide.obs[celltype_key].cat.categories).index('SchwannCells')]='orange'

sc.pl.spatial(slide,color=[celltype_key],
              groups=['SinoatrialNodePacemakerCells','ParasympatheticNeurons','SchwannCellPrecursors'],
              spot_size=15,vmax='p100',
              crop_coord=(4600,5600,4250,5250)
             )
```

Example 17 (python):
```python
tissue_id = 'C194-HEA-0-FFPE-1'
celltype_key = 'fine_grain_8Dec2024'
slide = adata[adata.obs['tissue_block_id']==tissue_id]

slide.uns[f"{celltype_key}_colors"][list(slide.obs[celltype_key].cat.categories).index('ParasympatheticNeurons')]='springgreen'
slide.uns[f"{celltype_key}_colors"][list(slide.obs[celltype_key].cat.categories).index('SinoatrialNodePacemakerCells')]='hotpink'
slide.uns[f"{celltype_key}_colors"][list(slide.obs[celltype_key].cat.categories).index('SchwannCellPrecursors')]='blue'
slide.uns[f"{celltype_key}_colors"][list(slide.obs[celltype_key].cat.categories).index('Fibroblasts')]='burlywood'

sc.pl.spatial(slide,color=[celltype_key],
              groups=['SinoatrialNodePacemakerCells','ParasympatheticNeurons','SchwannCellPrecursors','Fibroblasts'],
              spot_size=15,vmax='p100',
              crop_coord=(4600,5600,4250,5250)
             )
```

Example 18 (python):
```python
tissue_id = 'Hst45-HEA-0-FFPE-1'
celltype_key = 'fine_grain_8Dec2024'
slide = adata[adata.obs['tissue_block_id']==tissue_id]

slide.uns[f"{celltype_key}_colors"][list(slide.obs[celltype_key].cat.categories).index('ParasympatheticNeurons')]='springgreen'
slide.uns[f"{celltype_key}_colors"][list(slide.obs[celltype_key].cat.categories).index('SinoatrialNodePacemakerCells')]='hotpink'
slide.uns[f"{celltype_key}_colors"][list(slide.obs[celltype_key].cat.categories).index('SchwannCellPrecursors')]='blue'
slide.uns[f"{celltype_key}_colors"][list(slide.obs[celltype_key].cat.categories).index('SchwannCells')]='orange'

sc.pl.spatial(slide,color=[celltype_key],
              groups=['SinoatrialNodePacemakerCells','ParasympatheticNeurons','SchwannCells','SchwannCellPrecursors'],
              spot_size=30,vmax='p100',
              crop_coord=(6000,7000,500,3000)
             )
```

Example 19 (python):
```python
tissue_id = 'Hst45-HEA-0-FFPE-1'
celltype_key = 'fine_grain_8Dec2024'
slide = adata[adata.obs['tissue_block_id']==tissue_id]

slide.uns[f"{celltype_key}_colors"][list(slide.obs[celltype_key].cat.categories).index('ParasympatheticNeurons')]='springgreen'
slide.uns[f"{celltype_key}_colors"][list(slide.obs[celltype_key].cat.categories).index('SinoatrialNodePacemakerCells')]='hotpink'
slide.uns[f"{celltype_key}_colors"][list(slide.obs[celltype_key].cat.categories).index('MacrophagesCX3CR1pos')]='blue'
slide.uns[f"{celltype_key}_colors"][list(slide.obs[celltype_key].cat.categories).index('MacrophagesLYVE1pos')]='orange'
slide.uns[f"{celltype_key}_colors"][list(slide.obs[celltype_key].cat.categories).index('TCellsCD8pos')]='purple'

sc.pl.spatial(slide,color=[celltype_key],
              groups=['SinoatrialNodePacemakerCells','ParasympatheticNeurons','MacrophagesCX3CR1pos','MacrophagesLYVE1pos','TCellsCD8pos'],
              spot_size=30,vmax='p100',
              crop_coord=(6000,7000,500,3000)
             )
```

Example 20 (python):
```python
tissue_id = 'C194-HEA-0-FFPE-1'
celltype_key = 'fine_grain_8Dec2024'
slide = adata[adata.obs['tissue_block_id']==tissue_id]

slide.uns[f"{celltype_key}_colors"][list(slide.obs[celltype_key].cat.categories).index('ParasympatheticNeurons')]='springgreen'
slide.uns[f"{celltype_key}_colors"][list(slide.obs[celltype_key].cat.categories).index('SinoatrialNodePacemakerCells')]='hotpink'
slide.uns[f"{celltype_key}_colors"][list(slide.obs[celltype_key].cat.categories).index('MacrophagesCX3CR1pos')]='blue'
slide.uns[f"{celltype_key}_colors"][list(slide.obs[celltype_key].cat.categories).index('MacrophagesLYVE1pos')]='orange'
slide.uns[f"{celltype_key}_colors"][list(slide.obs[celltype_key].cat.categories).index('TCellsCD8pos')]='purple'

sc.pl.spatial(slide,color=[celltype_key],
              groups=['SinoatrialNodePacemakerCells','ParasympatheticNeurons','MacrophagesCX3CR1pos','MacrophagesLYVE1pos','TCellsCD8pos'],
              spot_size=15,vmax='p100',
              crop_coord=(4600,5600,4250,5250)
             )
```

Example 21 (python):
```python
tissue_id = 'Hst45-HEA-0-FFPE-1'
celltype_key = 'fine_grain_8Dec2024'
slide = adata[adata.obs['tissue_block_id']==tissue_id]

slide.uns[f"{celltype_key}_colors"][list(slide.obs[celltype_key].cat.categories).index('LymphaticEndothelialCells')]='blue'
slide.uns[f"{celltype_key}_colors"][list(slide.obs[celltype_key].cat.categories).index('MacrophagesLYVE1pos')]='orange'

sc.pl.spatial(slide,color=[celltype_key],
              groups=['SinoatrialNodePacemakerCells','LymphaticEndothelialCells'],
              spot_size=50,vmax='p100',
              crop_coord=(6000,7000,500,3000)
             )
```

Example 22 (python):
```python
tissue_id = 'Hst45-HEA-0-FFPE-1'
celltype_key = 'fine_grain_8Dec2024'
slide = adata[adata.obs['tissue_block_id']==tissue_id]

slide.uns[f"{celltype_key}_colors"][list(slide.obs[celltype_key].cat.categories).index('LymphaticEndothelialCells')]='blue'
slide.uns[f"{celltype_key}_colors"][list(slide.obs[celltype_key].cat.categories).index('MacrophagesLYVE1pos')]='orange'

sc.pl.spatial(slide,color=[celltype_key],
              groups=['SinoatrialNodePacemakerCells','LymphaticEndothelialCells'],
              spot_size=50,vmax='p100',
             )
```

Example 23 (python):
```python
tissue_id = 'C194-HEA-0-FFPE-1'
celltype_key = 'fine_grain_8Dec2024'
slide = adata[adata.obs['tissue_block_id']==tissue_id]

slide.uns[f"{celltype_key}_colors"][list(slide.obs[celltype_key].cat.categories).index('ParasympatheticNeurons')]='springgreen'
slide.uns[f"{celltype_key}_colors"][list(slide.obs[celltype_key].cat.categories).index('SinoatrialNodePacemakerCells')]='hotpink'
slide.uns[f"{celltype_key}_colors"][list(slide.obs[celltype_key].cat.categories).index('Fibroblasts')]='blue'

sc.pl.spatial(slide,color=[celltype_key],
              groups=['SinoatrialNodePacemakerCells','ParasympatheticNeurons','Fibroblasts'],
              spot_size=15,vmax='p100',
              crop_coord=(4600,5600,4250,5250)
             )
```

Example 24 (python):
```python
tissue_id = 'C194-HEA-0-FFPE-1'
celltype_key = 'fine_grain_8Dec2024'
slide = adata[adata.obs['tissue_block_id']==tissue_id]

slide.uns[f"{celltype_key}_colors"][list(slide.obs[celltype_key].cat.categories).index('ParasympatheticNeurons')]='springgreen'
slide.uns[f"{celltype_key}_colors"][list(slide.obs[celltype_key].cat.categories).index('SinoatrialNodePacemakerCells')]='hotpink'
slide.uns[f"{celltype_key}_colors"][list(slide.obs[celltype_key].cat.categories).index('Fibroblasts')]='blue'

sc.pl.spatial(slide,color=[celltype_key],
              groups=['SinoatrialNodePacemakerCells','ParasympatheticNeurons','Fibroblasts'],
              spot_size=15,vmax='p100',
              crop_coord=(4500,7000,4000,6000)
             )
```

Example 25 (python):
```python
tissue_id = 'Hst45-HEA-0-FFPE-1'
celltype_key = 'fine_grain_8Dec2024'
slide = adata[adata.obs['tissue_block_id']==tissue_id]

slide.uns[f"{celltype_key}_colors"][list(slide.obs[celltype_key].cat.categories).index('ParasympatheticNeurons')]='springgreen'
slide.uns[f"{celltype_key}_colors"][list(slide.obs[celltype_key].cat.categories).index('SinoatrialNodePacemakerCells')]='hotpink'
slide.uns[f"{celltype_key}_colors"][list(slide.obs[celltype_key].cat.categories).index('Fibroblasts')]='blue'

sc.pl.spatial(slide,color=[celltype_key],
              groups=['SinoatrialNodePacemakerCells','ParasympatheticNeurons','Fibroblasts'],
              spot_size=15,vmax='p100',
              crop_coord=(6000,7000,500,3000)
             )
```

Example 26 (python):
```python
tissue_id = 'Hst45-HEA-0-FFPE-1'
celltype_key = 'fine_grain_8Dec2024'
slide = adata[adata.obs['tissue_block_id']==tissue_id]

slide.uns[f"{celltype_key}_colors"][list(slide.obs[celltype_key].cat.categories).index('ParasympatheticNeurons')]='springgreen'
slide.uns[f"{celltype_key}_colors"][list(slide.obs[celltype_key].cat.categories).index('SinoatrialNodePacemakerCells')]='hotpink'
slide.uns[f"{celltype_key}_colors"][list(slide.obs[celltype_key].cat.categories).index('Fibroblasts')]='blue'

sc.pl.spatial(slide,color=[celltype_key],
              groups=['SinoatrialNodePacemakerCells','ParasympatheticNeurons','Fibroblasts'],
              spot_size=15,vmax='p100',
             )
```

Example 27 (python):
```python
genes = ['SEMA3A', 'SLIT2', 'SLIT3', 'EFNB2', 'CXCL12', 'FGF10', 'VEGFC', 'NELL1']
genes = [x for x in genes if x in adata.var_names]
genes
```

Example 28 (unknown):
```unknown
['SEMA3A', 'SLIT2', 'SLIT3', 'EFNB2', 'CXCL12', 'FGF10', 'VEGFC', 'NELL1']
```

Example 29 (python):
```python
tissue_id = 'C194-HEA-0-FFPE-1'
slide = adata[adata.obs['tissue_block_id']==tissue_id]
sc.pl.spatial(slide,color=genes,
              spot_size=20,vmax='p100',
              crop_coord=(4700,6000,4600,5300)
             )
```

Example 30 (python):
```python
genes = ['SEMA3A', 'SLIT2', 'SLIT3', 'EFNB2', 'CXCL12', 'FGF10', 'VEGFC', 'NELL1']
genes = [x for x in genes if x in adata.var_names]
genes
```

Example 31 (unknown):
```unknown
['SEMA3A', 'SLIT2', 'SLIT3', 'EFNB2', 'CXCL12', 'FGF10', 'VEGFC', 'NELL1']
```

Example 32 (python):
```python
cpdb_metadata = pd.read_csv('/nfs/team205/heart/cellphonedb/customDB/interaction_meta/interaction_metadata_v5.0.0_2024-1-26.csv')
lr_genes = cpdb_metadata[['gene_name_1', 'gene_name_2']].copy()
print(lr_genes.shape)
lr_genes.head()
```

Example 33 (python):
```python
for i,gene in enumerate(genes):
    mask1 = np.array([gene in x.split(',') for x in lr_genes['gene_name_1']])
    mask2 = np.array([gene in x.split(',') for x in lr_genes['gene_name_2']])
    df = lr_genes.loc[mask1|mask2]
    if i==0:
        lr_genes_sel = df.copy()
    else:
        lr_genes_sel = pd.concat([lr_genes_sel,df])
lr_genes_sel.drop_duplicates(inplace=True)
lr_genes_sel.index = range(len(lr_genes_sel))
lr_genes_sel
```

Example 34 (python):
```python
tissue_id = 'C194-HEA-0-FFPE-1'
slide = adata[adata.obs['tissue_block_id']==tissue_id]

for ind in lr_genes_sel.index:
    g = lr_genes_sel.loc[ind,'gene_name_1'].split(',')+lr_genes_sel.loc[ind,'gene_name_2'].split(',')
    g = [x for x in g if x in adata.var_names]
    sc.pl.spatial(slide,color=g,
              spot_size=20,vmax='p100',
              crop_coord=(4700,6000,4600,5300)
             )
```

Example 35 (python):
```python
for tissue_id in ['C194-HEA-0-FFPE-1', 'Hst45-HEA-0-FFPE-1']:
    slide = adata[adata.obs['tissue_block_id']==tissue_id]
    df = slide.obs[['cell_id','fine_grain_8Dec2024']]
    df.columns = ['cell_id','group']
    df.to_csv(f'./{tissue_id}_cellgroup_XR_8Dec2024.csv',index=False)
```

Example 36 (python):
```python
genes = ['TUBB3', 'MAP2', 'ARC', 'SLC18A3', 'SLC5A7', 'CAMK2A']
genes = [x for x in genes if x in adata.var_names]
genes
```

Example 37 (unknown):
```unknown
['TUBB3', 'MAP2', 'ARC', 'SLC5A7', 'CAMK2A']
```

Example 38 (python):
```python
tissue_id = 'C194-HEA-0-FFPE-1'
slide = adata[adata.obs['tissue_block_id']==tissue_id]
sc.pl.spatial(slide,color=genes,
              spot_size=20,vmax='p100',
              crop_coord=(4700,6000,4600,5300)
             )
```

Example 39 (python):
```python
tissue_id = 'C194-HEA-0-FFPE-1'
slide = adata[adata.obs['tissue_block_id']==tissue_id]
sc.pl.spatial(slide,color=genes,
              spot_size=60,vmax='p100',
              # crop_coord=(4700,6000,4600,5300)
             )
```

---

## Subcluster¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/RNA/scnRNA_Integration-Annotation/Neural/NeuralDE.html

**Contents:**
- Subcluster¶

**Examples:**

Example 1 (unknown):
```unknown
#Added the keys for age and sex
```

Example 2 (python):
```python
# Import packages, initial settings

import numpy as np
import matplotlib.pyplot as plt
import scanpy as sc
import pandas as pd
import anndata
import scrublet as scr
import seaborn as sns
import magic
import pybiomart
import six
import sys
sys.modules['sklearn.externals.six'] = six
import scanorama
import harmonypy
import bbknn
import gseapy
from gseapy.plot import barplot, dotplot
import scvelo as scv
import scvi

plt.rcParams['pdf.fonttype'] = 42
plt.rcParams['ps.fonttype'] = 42

sc.settings.verbosity = 3  # verbosity: errors (0), warnings (1), info (2), hints (3)

from matplotlib.colors import LinearSegmentedColormap
cmap = LinearSegmentedColormap.from_list(name='gene_cmap', colors=['lightgrey', 'thistle', 'red', 'darkred']) 

sc.settings.set_figure_params(dpi=80, color_map='viridis', vector_friendly=False,  dpi_save=300, facecolor='white')
sc.logging.print_version_and_date()
#sc.logging.print_versions()
```

Example 3 (unknown):
```unknown
Global seed set to 0
```

Example 4 (unknown):
```unknown
Running Scanpy 1.8.1, on 2023-09-01 22:04.
```

Example 5 (python):
```python
sc.settings.set_figure_params(dpi=300, color_map='viridis', vector_friendly=False,  dpi_save=300, facecolor='white')
```

Example 6 (python):
```python
adata=sc.read('./Objects/Feb28ObjectRaw.h5ad')
print(adata.shape)
```

Example 7 (unknown):
```unknown
(297473, 36601)
```

Example 8 (python):
```python
cell_cycle_genes = ['MCM5','PCNA','TYMS','FEN1','MCM2','MCM4','RRM1','UNG','GINS2','MCM6','CDCA7','DTL','PRIM1','UHRF1','MLF1IP','HELLS','RFC2','RPA2','NASP','RAD51AP1','GMNN','WDR76','SLBP','CCNE2','UBR7','POLD3','MSH2','ATAD2','RAD51','RRM2','CDC45','CDC6','EXO1','TIPIN','DSCC1','BLM','CASP8AP2','USP1','CLSPN','POLA1','CHAF1B','BRIP1','E2F8','HMGB2','CDK1','NUSAP1','UBE2C','BIRC5','TPX2','TOP2A','NDC80','CKS2','NUF2','CKS1B','MKI67','TMPO','CENPF','TACC3','FAM64A','SMC4','CCNB2','CKAP2L','CKAP2','AURKB','BUB1','KIF11','ANP32E','TUBB4B','GTSE1','KIF20B','HJURP','CDCA3','HN1','CDC20','TTK','CDC25C','KIF2C','RANGAP1','NCAPD2','DLGAP5','CDCA2','CDCA8','ECT2','KIF23','HMMR','AURKA','PSRC1','ANLN','LBR','CKAP5','CENPE','CTCF','NEK2','G2E3','GAS2L3','CBX5','CENPA']
s_genes = cell_cycle_genes[:43]
s_genes = [x for x in s_genes if x in adata.var_names]
print('S genes:', s_genes, '\n')
g2m_genes = cell_cycle_genes[43:]
g2m_genes = [x for x in g2m_genes if x in adata.var_names]
print('G2 genes:', g2m_genes, '\n')
cell_cycle_genes = [x for x in cell_cycle_genes if x in adata.var_names]

hb_genes1 = ['HBB','HBD','HBG1','HBG2','HBE1','HBZ','HBM','HBA2','HBA1', 'HBQ1']
hb_genes1 = [x for x in hb_genes1 if x in adata.var_names]
print('Hemoglobin genes set 1:', hb_genes1, '\n')

hb_genes2 = ['AHSP','GYPA','RHAG','GFI1B','KEL','EPB42','ALAS2','TSPO2','GYPB','GYPE','SPTA1','RHCE','KLF1','RHD','SLC4A1','SNX22','SQSTM1','SOD1','FTH1','MAP1LC3B','SLC25A37','TRIM58','HMBS']
hb_genes2 = [x for x in hb_genes2 if x in adata.var_names]
print('Hemoglobin genes set 2:', hb_genes2, '\n')

mito_genes=['MT-ND1','MT-ND2','MT-CO1','MT-CO2','MT-ATP8','MT-ATP6','MT-CO3','MT-ND3','MT-ND4L', 'MT-ND4', 'MT-ND5', 'MT-ND6', 'MT-CYB']
mito_genes = [x for x in mito_genes if x in adata.var_names]
print('Mitochondrial genes:', mito_genes, '\n')

ribo_genes = ['RPLP0','RPLP1','RPLP2','RPL3','RPL3L','RPL4','RPL5','RPL6','RPL7','RPL7A','RPL7L1','RPL8','RPL9','RPL10','RPL10A','RPL10L','RPL11','RPL12','RPL13','RPL13A','RPL14','RPL15','RPL17','RPL18','RPL18A','RPL19','RPL21','RPL22','RPL22L1','RPL23','RPL23A','RPL24','RPL26','RPL26L1','RPL27','RPL27A','RPL28','RPL29','RPL30','RPL31','RPL32','RPL34','RPL35','RPL35A','RPL36','RPL36A','RPL36AL','RPL37','RPL37A','RPL38','RPL39','RPL39L','UBA52','RPL41','RPSA','RPS2','RPS3','RPS3A','RPS4X','RPS4Y1','RPS4Y2','RPS5','RPS6','RPS7','RPS8','RPS9','RPS10','RPS11','RPS12','RPS13','RPS14','RPS15','RPS15A','RPS16','RPS17','RPS18','RPS19','RPS20','RPS21','RPS23','RPS24','RPS25','RPS26','RPS27','RPS27A','RPS27L','RPS28','RPS29','FAU']
ribo_genes = [x for x in ribo_genes if x in adata.var_names]
print('Ribo genes:', ribo_genes, '\n')

stress_genes=['FOS','FOSB','JUN','JUNB','JUND','ZFP36','SOCS3','CEBPD','CEBPB','ATF3','EGR1','HSPA1A','HSPA1B','HSP90AA1','HSP90AB1','HSPA8','HSPB1','HSPE1','IER3','IER2','BTG1','BTG2','DUSP1']
stress_genes = [x for x in stress_genes if x in adata.var_names]
print('Tissue dissociation stress associated genes:', stress_genes, '\n')

FoxProteins=['FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1']
chrY_genes=['SRY', 'RPS4Y1', 'ZFY', 'ZFY-AS1', 'LINC00278', 'PCDH11Y', 'TTTY2B', 'TBL1Y', 'PRKY', 'USP9Y', 'DDX3Y', 'UTY', 'TMSB4Y', 'NLGN4Y', 'TTTY14', 'KDM5D', 'TTTY10', 'EIF1AY', 'PRORY', 'LINC00266-4P']
MembraneProteins1=['AADAT','AAK1','ABCA12','ABCA7','ABCB1','ABCB11','ABCB4','ABCB6','ABCC1','ABCC3','ABCC4','ABCC5','ABCC6','ABCG2','ABHD3','ABI1','ABLIM3','AC004691.2','AC004922.1','AC005324.3','AC005726.1','AC008403.1','AC138696.1','ACIN1','ACKR3','ACLY','ACOT13','ACOT9','ACTN1','ACTR10','ACTR6','ADA','ADAM10','ADAM12','ADAM22','ADAM28','ADAM33','ADAMTS1','ADAMTS2','ADAMTSL4','ADAP1','ADCK5','ADD1','ADD2','ADGRB2','ADGRD1','ADGRG2','ADGRG7','ADGRL2','ADH1A','ADH1B','ADH1C','ADH7','ADK','ADORA1','ADRA1B','ADRM1','ADSS','AFAP1L2','AFDN','AFG1L','AGER','AGRN','AHNAK','AHNAK2','AIFM2','AKAP10','AKAP11','AKAP12','AKR1B10','AKR1B15','AL132780.3','AL139300.1','AL355315.1','AL358113.1','AL645922.1','ALDH3A1','ALK','ALOX15B','ALOXE3','ALPI','ALPP','ALPPL2','ALS2CR12','AMACR','AMDHD1','AMER1','AMER2','AMIGO2','AMOT','AMOTL1','AMOTL2','AMPH','ANK2','ANK3','ANKEF1','ANKFN1','ANKIB1','ANKLE2','ANKRA2','ANKRD13A','ANKRD35','ANKRD42','ANKRD45','ANKS1B','ANO1','ANO10','ANO2','ANO6','ANPEP','ANXA1','ANXA13','ANXA2','ANXA3','AP2M1','APBB1','APBB1IP','APC','APLP1','APMAP','APOC3','APOD','APOLD1','AQP1','AQP10','AQP3','AQP4','AQP5','ARAP1','ARF1','ARHGAP18','ARHGAP19','ARHGAP21','ARHGAP22','ARHGAP23','ARHGAP24','ARHGAP28','ARHGAP29','ARHGAP33','ARHGAP36','ARHGAP40','ARHGAP9','ARHGDIG','ARHGEF1','ARHGEF11','ARHGEF12','ARHGEF15','ARHGEF26','ARHGEF28','ARHGEF40','ARHGEF5','ARID2','ARID4A','ARID5B','ARL13A','ARL15','ARL4C','ARMC12','ARMCX5','ARPC1A','ARPC5','ARRB2','ARRDC4','ARVCF','ASAP1','ASB5','ASGR1','ASGR2','ASH2L','ASIC1','ASIC4','ASPM','ASPSCR1','ATG13','ATG3','ATG7','ATIC','ATOX1','ATP10D','ATP12A','ATP13A3','ATP1B3','ATP23','ATP2B1','ATP2B4','ATP2C2','ATP6V0A2','ATP6V1H','ATP8A2','ATP8B1','ATXN10','ATXN3','ATXN7L3','AVEN','AVPI1','AXIN2','AXL','B2M','B3GALT2','BACE1','BAG3','BAIAP2','BAIAP2L1','BANK1','BASP1','BBOF1','BCAR1','BCAR3','BCL2L12','BCORL1','BCR','BDKRB2','BEND6','BFSP1','BFSP2','BICD2','BIN2','BLNK','BLVRB','BMPR2','BMX','BOC','BOLL','BORCS5','BORCS8-MEF2B','BRK1','BRPF1','BRSK1','BSDC1','BST2','BTK','BTN2A1','BTRC','BUB1B-PAK6','BVES','BZW2','C10orf90','C11orf42','C11orf52','C12orf73','C12orf76','C14orf37','C14orf93','C16orf74','C17orf58','C17orf62','C18orf21','C19orf33','C19orf38','C19orf48','C19orf71','C1orf116','C1orf189','C1orf35','C1orf54','C1orf68','C1orf74','C1QBP','C2orf72','C3orf14','C3orf20','C3orf38','C4orf19','C4orf22','C7orf43','C8orf33','C8orf37','C9orf153','C9orf3','CA14','CA9','CABLES2','CACNA1E','CACNG4','CADM1','CALCRL','CALD1','CALML5','CALN1','CAMK2A','CAMK2B','CAMK2D','CAMK2G','CAP2','CAPS','CASKIN2','CASP4','CAST','CAV3','CAVIN1','CAVIN2','CAVIN3','CAVIN4','CBWD1','CBWD2','CBWD3','CBWD5','CBWD6','CC2D1A','CCBE1','CCDC107','CCDC112','CCDC115','CCDC124','CCDC136','CCDC144NL','CCDC15','CCDC154','CCDC170','CCDC24','CCDC28A','CCDC59','CCDC62','CCDC66','CCDC69','CCDC8','CCDC83','CCDC85C','CCDC93','CCDC97','CCNT2','CCNYL1','CCR3','CCR4','CCT3','CD109','CD163','CD164L2','CD200R1','CD226','CD248','CD274','CD2AP','CD300A','CD300LB','CD33','CD34','CD37','CD38','CD3E','CD3G','CD4','CD44','CD46','CD47','CD48','CD79B','CD81','CD84','CD86','CD8A','CD8B','CD9','CD93','CD99L2','CDAN1','CDC42EP1','CDC42EP3','CDC42EP4','CDC42SE1','CDCA4','CDH1','CDH13','CDH15','CDH17','CDH2','CDH26','CDH3','CDH4','CDH5','CDH6','CDH8','CDIPT','CDK16','CDK5','CDK5RAP2','CDK7','CDRT15','CDV3','CEACAM5','CELSR1','CENPJ','CEP162','CEP295','CEP55','CEP57L1','CEP89','CERCAM','CFAP157','CFAP206','CFAP58','CFAP65','CFB','CFL2','CFLAR','CGN','CGNL1','CHCHD7','CHIC2','CHMP2A','CHRM3','CHRNA1','CHRNA5','CHRNB4','CHRND','CIB1','CIB3','CIP2A','CKAP5','CLCA2','CLCA4','CLCN5','CLCN6','CLDN11','CLDN18','CLDN2','CLDN23','CLDN3','CLDN4','CLDN7','CLDN9','CLEC2A','CLEC4F','CLEC5A','CLECL1','CLIC2','CLIC5','CLTB','CLUL1','CMTM4','CMTM7','CMTR2','CMYA5','CNGA1','CNKSR2','CNNM1','CNNM4','CNP','CNR1','CNTD1','COBL','COBLL1','COG3','COL16A1','COL17A1','COL4A5','COL6A2','COLEC12','COLQ','COMMD4','COMTD1','COPRS','COQ7','CORIN','CORO1B','CORO2A','CORO2B','COX6B2','CPEB3','CPPED1','CPZ','CRB3','CREB5','CRIP1','CRIP2','CRK','CRLF3','CROCC','CRTC1','CRYAB','CRYBG3','CRYL1','CSDE1','CSF1','CSF1R','CSPG4','CST6','CTNNA1','CTNNB1','CTNNBIP1','CTNND1','CTNND2','CTSF','CTSV','CTTN','CTXN1','CUEDC1','CUTA','CX3CL1','CX3CR1','CXADR','CXCR2','CYB5D2','CYBRD1','CYFIP2','CYP20A1','CYP24A1','CYTH2','CYTH4','DAAM1','DAB2','DAG1','DAGLB','DAW1','DBN1','DCAF13','DCAF16','DCBLD2','DCHS1','DCHS2','DCP2','DCT','DCTN6','DCUN1D2','DDAH1','DDB2','DDIAS','DDIT4L','DDR1','DDR2','DDX58','DDX60L','DENND6B','DFFA','DGKG','DGKH','DHDDS','DHPS','DHRS7B','DIAPH1','DIAPH3','DIO2','DIS3L','DLG1','DLG2','DLG5','DLGAP3','DLL3','DMWD','DNAAF1','DNAAF4','DNAH17','DNAJA4','DNAJB13','DNAJB4','DNAJC18','DNAJC5','DNAJC6','DNAJC9','DOC2A','DOCK4','DOK3','DONSON','DPEP1','DPH1','DPPA5','DPYSL2','DQX1','DRD1','DRP2','DSC3','DSEL','DSG2','DSP','DTNA','DUS1L','DUSP15','DXO','DYSF','ECSCR','EDA2R','EDNRA','EDNRB','EEF1AKMT1','EEF2','EFCAB1','EFCAB11','EFCAB2','EFCAB7','EFHC2','EFNB1','EFR3A','EFR3B','EGFR','EHBP1','EHD1','EHD2','EHD3','EHD4','EIF2B2','EIF5','EIF5B','ELFN1','ELMO3','ELN','ELOC','EMP3','ENAH','ENC1','ENG','ENKD1','ENO1','ENO2','ENO3','ENOX1','ENOX2','EPB41','EPB41L1','EPB41L2','EPB41L3','EPB41L4A','EPB41L4B','EPB41L5','EPCAM','EPHA2','EPHA3','EPHB1','EPHB2','EPN1','EPOR','EPS8L2','ERAP1','ERBB2','ERBB3','ERBIN','ERC1','ERCC4','ERMN','ESCO2','ETNK1','ETS2','EVA1A','EVI2B','EXOC1','EXOC3L1','EXOC7','EZR','F11R','F2RL1','F2RL3','FAAP20','FABP2','FABP3','FABP5','FAIM','FAM126A','FAM126B','FAM129A','FAM129B','FAM13A','FAM163B','FAM171A1','FAM193A','FAM205A','FAM210B','FAM222A','FAM45A','FAM47E','FAM47E-STBD1','FAM72A','FAM72B','FAM72C','FAM72D','FANCG','FAS','FASN','FBLIM1','FBLN2','FBLN5','FBLN7','FBP2','FBRSL1','FBXL2','FBXO16','FBXO38','FBXO41','FBXW11','FCAR','FCGBP','FCGR1A','FCGR1B','FCGR2A','FCHSD2','FFAR4','FGD1','FGD5','FGFR2','FGR','FHIT','FHL1','FIBCD1','FIGN','FILIP1','FILIP1L','FKBP2','FLAD1','FLCN','FLNA','FLNB','FLNC','FLOT1','FLOT2','FLRT3','FLT1','FLT4','FLVCR1','FMN2','FMNL2','FMNL3','FNBP1','FOXA2','FOXN3','FP565260.3','FRMD3','FRMD7','FRMD8','FRS2','FSCN1','FTCD','FUT2','FXYD3','FYB1','FYN','FZD1','FZD2','FZD4','FZD6','GAB1','GAB2','GABRA1','GABRA2','GABRA3','GABRA5','GABRG1','GAD1','GADL1','GAGE1','GAGE10','GAGE12C','GAGE12D','GAGE12E','GAGE12F','GAGE12G','GAGE12H','GAGE12J','GAGE13','GAGE2A','GAGE2E','GALNT10','GALR1','GAP43','GAPDH','GAPT','GAPVD1','GAS2','GAS2L1','GAS7','GAS8','GBP4','GCA','GCC1','GCLM','GCSAML-AS1','GFI1B','GFOD1','GFRA3','GHR','GID8','GINM1','GIPC1','GJA1','GJB3','GJB4','GJB6','GJB7','GJC1','GJC2','GLO1','GLP2R','GLRX','GLUL','GMIP','GNAI2','GNAQ','GNAS','GNB1','GNB2','GNB3','GNB4','GNG2','GP9','GPAT3','GPATCH1','GPBP1','GPBP1L1','GPC1','GPC3','GPC4','GPI','GPR1','GPR142','GPR156','GPR157','GPR160','GPR173','GPR20','GPR27','GPR50','GPR63','GPRC5A','GPRIN1','GPRIN2','GPX8','GRAMD1A','GRAMD1B','GRAMD1C','GRAP2','GRB7','GRID2IP','GRIK5','GRK3','GRK5','GRPR','GSDMA','GTF2F1','GTSE1','GUCA1A','GYPA','GYPC','HAPLN3','HARBI1','HAS1','HAUS4','HAUS7','HCAR2','HCAR3','HCFC2','HCK','HCLS1','HCST','HDAC8','HEG1','HELZ','HENMT1','HERC2','HERPUD1','HFE','HIF1A','HIF3A','HINT1','HIP1R','HLA-A','HLA-DPB1','HMGCS1','HMOX1','HOMER3','HOXC5','HOXD4','HPS3','HRASLS5','HRH1','HS6ST3','HSH2D','HSPB1','HSPG2','HTR1B','HTR7','HTRA1','HUNK','HYDIN','HYI','HYLS1','ICAM1','ICAM2','ICOS','ICOSLG','IFFO2','IFIT5','IFNGR1','IFRD1','IFT122','IGFL2','IGSF11','IGSF23','IKZF5','IL10RA','IL12RB1','IL12RB2','IL13','IL13RA2','IL16','IL17RB','IL1R1','IL1RL1','IL21R','IL4R','IL6ST','IL7R','ILDR2','ILK','INPP5A','INSR','INTS12','IP6K2','IPCEF1','IQGAP1','IQGAP2','IREB2','IRF2','IRS1','ISLR','ITGA2B','ITGA3','ITGA4','ITGA7','ITGA8','ITGA9','ITGB1','ITGB2','ITGB3','ITGB4','ITGB5','ITGB6','ITGB7','ITGB8','ITK','ITM2A','ITPKA','ITSN1','ITSN2','IYD','JAG1','JAK2','JAK3','JAM2','JAML','JCAD','JMJD4','JPT2','JUP','KANK1','KANK3','KANSL2','KARS','KATNB1','KCNAB1','KCNB2','KCNH8','KCNJ15','KCNK13','KCNK17','KCNK6','KCNK7','KCNK9','KCNN2','KCNS3','KCNU1','KDF1','KDR','KEL','KIAA0556','KIAA0895','KIAA1324','KIAA1522','KIAA1958','KIF17','KIF19','KIF21A','KIF21B','KIF26A','KIF26B','KISS1R','KIT','KLC1','KLC2','KLF9','KLHDC7B','KLHL41','KLHL6','KLHL7','KLK7','KLRC1','KLRC2','KLRF1','KLRK1','KMT2D','KNDC1','KNSTRN','KNTC1','KPTN','KRT75','KRT77','KRTAP25-1','L1CAM','L3MBTL1','LAIR1','LAMA4','LAMC1','LAMP2','LAMTOR1','LAPTM4A','LASP1','LAT','LAT2','LAX1','LCP1','LCTL','LDB2','LDLRAD3','LENG1','LGALS3','LGALS4','LGALS8','LGR6','LHFPL2','LHX6','LIMA1','LINGO1','LIPJ','LMCD1','LMOD2','LMTK2','LMTK3','LNX1','LNX2','LONRF1','LONRF3','LOXHD1','LPAR1','LPAR3','LPAR4','LPIN1','LPIN2','LRCH2','LRP11','LRP12','LRRC15','LRRC26','LRRC34','LRRC45','LRRC69','LRRC7','LRRC9','LRRFIP2','LRRIQ3','LSMEM2','LSR','LTB4R2','LUZP2','LY6K','LYN','LYPD1','LYST','LZTS1','LZTS2','MACF1','MADD','MAGI1','MAGI3','MAMDC2','MAP1S','MAP2','MAP2K1','MAP2K4','MAP2K7','MAP3K12','MAP3K21','MAP3K6','MAP3K7','MAP4','MAP4K1','MAP4K5','MAP6','MAPK15','MAPK8IP1','MAPT','MARCH10','MARCH7','MARCKS','MARK2','MARVELD2','MAS1L','MAT2B','MBP','MC5R','MCAM','MCEMP1','MCF2L2','MCMBP','MCTS1','MDGA1','ME1','MEF2B','MEIOC','MEPCE','MERTK','MET','METAP1','METAP2','METTL15','METTL25','METTL9','MFAP3L','MFSD1','MFSD2A','MGAT4C','MIB1','MICAL2','MICAL3','MICALL1','MICB','MIEN1','MIGA2','MISP','MKLN1','MKRN3','MLF2','MLKL','MLST8','MMACHC','MMP10','MMP15','MMP24','MMP25','MOAP1','MPHOSPH8','MPHOSPH9','MPI','MPIG6B','MPL','MPP1','MPP5','MPP7','MPZL1','MPZL2','MR1','MRGPRF','MRNIP','MRPL42','MRPL44','MRPL46','MRPS18B','MS4A1','MS4A12','MS4A14','MS4A15','MSN','MSRA','MTCP1','MTFR1L','MTHFR','MTM1','MTPN','MTSS1','MUC1','MUC20','MVD','MYH11','MYH9','MYL4','MYLK','MYO10','MYO1B','MYO1C','MYO1E','MYO1G','MYO3A','MYO5B','MYO9A','MYOF','N4BP2','NAA35','NAB1','NAIF1','NAMPT','NAP1L2','NAPG','NARFL','NCAM1','NCAM2','NCK1','NCKAP1L','NCKAP5','NCKIPSD','NCOA1','NCS1','NDC1','NECTIN2','NECTIN3','NECTIN4','NEDD1','NEDD9','NEO1','NEU1','NEXN','NF2','NFE2L2','NFKBID','NGEF']
MembraneProteins2=['NGFR','NHS','NHSL2','NID2','NINJ2','NKTR','NKX2-8','NLGN1','NLGN3','NLK','NLRX1','NMB','NMBR','NME1','NME2','NMT1','NMT2','NMUR2','NOS1','NOTCH2','NPFFR2','NPL','NPNT','NPR1','NPTN','NPTXR','NR2C1','NRAS','NRCAM','NRP2','NRSN1','NRXN1','NSD1','NSFL1C','NT5E','NTNG1','NTNG2','NTSR1','NUDCD3','NUDT16L1','NUDT9','NUMB','NUP35','NUS1','OAS3','OBSCN','OCA2','OCLN','OCM','OCM2','ODC1','ODR4','OGT','OLFM4','OLIG2','OLR1','OPHN1','OPRK1','OR10P1','OR1N2','OR2C3','OR6B2','OR6B3','OR6C6','OR8S1','OSBPL10','OSBPL6','OSBPL7','OSCAR','OSCP1','OSGEP','OSR2','OTOP2','OTULIN','OXGR1','P3H3','PACSIN2','PACSIN3','PAFAH1B2','PAG1','PAIP1','PAK1','PAK4','PAK6','PALLD','PALM','PALM2','PALM2-AKAP2','PALM3','PALMD','PANX1','PANX2','PAQR8','PARD3','PARD3B','PARD6A','PARD6G','PARP6','PATJ','PAWR','PCBD2','PCDH1','PCDH17','PCDH7','PCDHA1','PCDHA11','PCDHA12','PCDHA4','PCDHA5','PCDHB1','PCDHB11','PCDHB12','PCDHB16','PCDHB4','PCDHB5','PCDHGA1','PCDHGA10','PCDHGA11','PCDHGA12','PCDHGA2','PCDHGA3','PCDHGA4','PCDHGA5','PCDHGA6','PCDHGA7','PCDHGA8','PCDHGA9','PCDHGB1','PCDHGB2','PCDHGB3','PCDHGB4','PCDHGB5','PCDHGB6','PCDHGB7','PCDHGC3','PCDHGC4','PCDHGC5','PCLO','PCMTD1','PDAP1','PDCD7','PDE3A','PDE4A','PDE4D','PDE9A','PDGFC','PDGFRA','PDHX','PDLIM1','PDLIM5','PDPN','PDZD2','PDZD8','PEAR1','PEBP1','PFDN1','PGAP3','PGLYRP2','PGLYRP4','PGRMC2','PHACTR1','PHACTR4','PHF20L1','PHGDH','PHLDB2','PHLDB3','PHOSPHO1','PHPT1','PI3','PI4K2A','PI4KA','PIEZO2','PIGB','PIGN','PIGW','PIGX','PIK3AP1','PIK3CG','PIK3R6','PIP4K2A','PIP4K2C','PIP5K1A','PIRT','PITPNM3','PKD2L2','PKN1','PKN2','PKP1','PKP2','PKP3','PKP4','PLAA','PLAUR','PLCB1','PLCB2','PLCD3','PLCD4','PLEKHA4','PLEKHA6','PLEKHA7','PLEKHG3','PLEKHG4','PLEKHG6','PLEKHH2','PLEKHO2','PLIN4','PLPP1','PLPPR5','PLS3','PLSCR1','PLVAP','PLXNA3','PLXNA4','PNCK','PNISR','PODXL','POLE','POU2F3','PPEF1','PPFIBP1','PPIP5K1','PPL','PPM1A','PPM1F','PPP1CA','PPP1CB','PPP1R12B','PPP1R16A','PPP1R16B','PPP1R18','PPP1R27','PPP1R9A','PPP2R1B','PPP3R1','PPP3R2','PPP4C','PPP6R3','PPTC7','PRAME','PRC1','PRDX6','PRIMA1','PRKCA','PRKCE','PRKCH','PRKCZ','PRKD1','PROM2','PROSER2','PRR15','PRR32','PRR7','PRRC2A','PRRG1','PRRT2','PRRT3','PRRT4','PRTG','PRX','PSCA','PSD','PSD2','PSEN1','PSMC5','PSMC6','PSMD9','PSTPIP2','PTAFR','PTCHD4','PTK2B','PTK6','PTPN12','PTPN21','PTPN22','PTPN4','PTPRB','PTPRG','PTPRK','PTPRM','PTPRR','PTPRS','PTX3','PURG','PVALB','PVR','PXDC1','PXK','PYGL','PYGO2','PYM1','RAB13','RAB23','RAB25','RAB26','RAB27B','RAB3B','RAB3GAP2','RAB43','RAB8A','RABEPK','RAD51C','RAD52','RAI2','RALA','RALGAPA2','RAPGEF2','RAPH1','RAPSN','RASA4','RASA4B','RASAL1','RASAL2','RASAL3','RASD1','RASD2','RASGRP1','RASGRP2','RASGRP4','RASSF2','RASSF3','RASSF4','RBM46','RC3H1','RCCD1','RCSD1','RD3L','RDX','RECK','RELL1','REPS1','RET','RFLNB','RFPL1','RFPL2','RFPL3','RGP1','RGS10','RGS14','RGS19','RGS3','RGS7BP','RGS9BP','RHCE','RHD','RHEX','RHOA','RHOBTB2','RHOJ','RHOXF2','RHOXF2B','RIC8A','RIC8B','RIF1','RILPL1','RIMBP3','RIMBP3B','RIMBP3C','RIMS2','RIOK2','RIOK3','RIPK1','RND1','RNF114','RNF145','RNF146','RNF149','RNF165','RNF166','RNF39','ROBO1','ROBO3','ROBO4','ROM1','RP2','RPE','RPEL1','RPGRIP1L','RPIA','RPL36A','RPL36A-HNRNPH2','RPL36AL','RPSA','RPUSD1','RRAD','RRP12','RRP7A','RSBN1L','RSC1A1','RSPH3','RTL3','RTL4','RTN4R','RTN4RL1','RYR2','S100A13','S100A14','S100A16','S100A2','S100A3','S100A4','S100A6','S100A9','SAMD4A','SAMHD1','SAMSN1','SAPCD1','SASH3','SC5D','SCAMP1','SCAMP4','SCEL','SCHIP1','SCIN','SCN1A','SCN5A','SCN8A','SCN9A','SCNN1D','SCNN1G','SCRIB','SCUBE3','SDC2','SDC4','SDE2','SDHB','SEC16B','SEMA3C','SEMA3D','SEMA4B','SEMA4F','SEMA6C','SEMA6D','SEPT2','SEPT3','SERINC2','SERINC5','SERPINB3','SFSWAP','SGCE','SGK494','SH3BGR','SH3D21','SHANK1','SHANK3','SHARPIN','SHISA6','SHKBP1','SHROOM2','SHTN1','SIGLEC10','SIGLEC14','SIGLEC6','SIGLEC7','SIN3B','SIPA1','SIPA1L1','SIPA1L3','SKAP1','SLAMF6','SLC10A5','SLC12A2','SLC13A3','SLC13A5','SLC14A1','SLC16A1','SLC16A10','SLC16A2','SLC16A3','SLC16A7','SLC16A9','SLC17A5','SLC19A1','SLC19A3','SLC1A5','SLC1A6','SLC22A16','SLC23A3','SLC24A4','SLC26A9','SLC27A1','SLC28A1','SLC2A1','SLC2A11','SLC2A12','SLC2A14','SLC2A2','SLC2A3','SLC30A1','SLC31A2','SLC34A1','SLC34A2','SLC35F5','SLC35G2','SLC36A4','SLC38A5','SLC38A6','SLC38A7','SLC39A10','SLC39A14','SLC39A2','SLC39A6','SLC3A2','SLC40A1','SLC41A3','SLC43A1','SLC43A3','SLC44A2','SLC45A4','SLC46A1','SLC46A3','SLC4A1AP','SLC4A2','SLC4A7','SLC4A8','SLC5A12','SLC5A3','SLC5A6','SLC5A7','SLC6A6','SLC7A14','SLC7A2','SLC7A5','SLC7A6','SLC8A1','SLC9A1','SLC9A2','SLC9A3','SLC9A3R1','SLC9A3R2','SLC9A4','SLC9A5','SLC9B2','SLC9C2','SLCO1A2','SLCO1B3','SLCO4A1','SLCO4C1','SLCO5A1','SLIT1','SLIT3','SLITRK6','SLK','SLX4','SMAGP','SMAP1','SMOC1','SMPD2','SMPX','SNAP23','SNAP25','SNRK','SNTB2','SNTG2','SNX15','SNX32','SNX9','SOCS3','SOCS5','SOD1','SORBS1','SORCS1','SORT1','SOX11','SPAG16','SPANXN4','SPART','SPATA12','SPATC1L','SPERT','SPG11','SPHK1','SPINK13','SPINT1','SPINT2','SPN','SPPL2B','SPPL3','SPRED3','SPRYD3','SPTBN2','SRC','SRCIN1','SRD5A3','SRPK1','SRPRB','SRRM3','SSFA2','SSH1','SSH3','SSTR1','SSTR5','SSX2IP','ST8SIA5','STAB1','STAC','STAC2','STAMBP','STAMBPL1','STARD4','STAT2','STEAP4','STIL','STIP1','STK10','STK16','STK17A','STK17B','STOM','STOML2','STOML3','STX18','STX4','STXBP4','STYK1','SUCLG1','SUCLG2','SUGT1','SUMF2','SUPT6H','SUSD2','SVIL','SWAP70','SYCP2','SYK','SYMPK','SYNC','SYNGR3','SYNPO2L','SYPL2','SYT4','SYTL2','SYVN1','TACC2','TACR2','TACSTD2','TASP1','TAZ','TBC1D10A','TBC1D2','TBC1D24','TBC1D30','TBC1D9B','TBXA2R','TCAF2','TCAF2C','TCHP','TCIM','TDG','TDO2','TDP1','TEC','TEDC2','TEK','TES','TEX14','TEX9','TGFBR1','TGFBR2','TGM1','TGM2','THBS1','THBS2','THEGL','THY1','TIAM1','TICAM1','TIGD7','TIMD4','TJP1','TJP2','TJP3','TLN1','TLR4','TM2D2','TM4SF20','TM4SF4','TM4SF5','TMC5','TMED8','TMEM101','TMEM107','TMEM108','TMEM121B','TMEM139','TMEM141','TMEM164','TMEM176B','TMEM183A','TMEM186','TMEM2','TMEM218','TMEM222','TMEM229A','TMEM245','TMEM251','TMEM254','TMEM258','TMEM266','TMEM268','TMEM50A','TMEM63B','TMPRSS15','TMPRSS2','TMTC2','TNFRSF10D','TNFRSF11A','TNFRSF12A','TNFRSF21','TNFRSF25','TNFRSF8','TNFRSF9','TNFSF13B','TNK1','TNKS1BP1','TNNI2','TOGARAM2','TOLLIP','TOM1','TP53BP2','TPD52L1','TPRN','TRAF7','TRAIP','TRAPPC13','TRAT1','TRDN','TREML1','TRIB1','TRIM16','TRIM16L','TRIM38','TRIM4','TRIM44','TRIM54','TRIM71','TRIML2','TRIP6','TRMT1','TRMT10B','TRMT13','TRMT6','TRO','TRPM1','TRPM4','TRPM7','TRPV2','TRPV3','TSG101','TSGA13','TSHZ3','TSPAN15','TSPAN6','TSPO2','TSR2','TSSK4','TTC17','TTC23','TTC7B','TTLL12','TTLL5','TULP3','TWF2','TXNDC11','TXNIP','TYROBP','TYW3','UAP1','UBA3','UBA52','UBAC1','UBALD1','UBAP1','UBASH3B','UBE2A','UBE2B','UBE2C','UBE2D1','UBE2D2','UBE2D3','UBE2D4','UBE2K','UBE2QL1','UBE2S','UBE3C','UBQLN2','UBR2','UBTD1','UCP3','ULBP1','ULBP2','UNC5A','UNC79','UQCC1','USP14','USP15','USP2','USP20','USP21','USP4','USP53','UTP20','UTRN','VAMP5','VAMP8','VANGL1','VASP','VCAM1','VCPIP1','VIL1','VPS13B','VPS36','VPS50','VPS8','VSIG4','VTCN1','VWA5B1','VWA7','WAS','WASF2','WBP4','WDFY3','WDPCP','WDR1','WDR12','WDR13','WDR53','WDR6','WDR66','WIPF1','WWP1','XIAP','XIRP1','XIRP2','XPC','XPNPEP2','XPO6','XRN1','XYLT1','YES1','YIPF1','YIPF4','YOD1','YTHDC1','ZBED3','ZBTB33','ZBTB8OS','ZDHHC2','ZDHHC20','ZDHHC22','ZDHHC5','ZFP41','ZFP90','ZFYVE28','ZGPAT','ZMYND10','ZNF10','ZNF18','ZNF185','ZNF2','ZNF20','ZNF227','ZNF229','ZNF236','ZNF276','ZNF277','ZNF337','ZNF397','ZNF407','ZNF497','ZNF503','ZNF510','ZNF536','ZNF571','ZNF598','ZNF599','ZNF646','ZNF655','ZNF688','ZNF706','ZNF717','ZNF783','ZNF79','ZNF839','ZSCAN22','ZSCAN29','ZSWIM1','ZSWIM8','ZYX']
MembraneProteins=MembraneProteins1+MembraneProteins2

TFs1=['AC008770.3','AC023509.3','AC092835.1','AC138696.1','ADNP','ADNP2','AEBP1','AEBP2','AHCTF1','AHDC1','AHR','AHRR','AIRE','AKAP8','AKAP8L','AKNA','ALX1','ALX3','ALX4','ANHX','ANKZF1','AR','ARGFX','ARHGAP35','ARID2','ARID3A','ARID3B','ARID3C','ARID5A','ARID5B','ARNT','ARNT2','ARNTL','ARNTL2','ARX','ASCL1','ASCL2','ASCL3','ASCL4','ASCL5','ASH1L','ATF1','ATF2','ATF3','ATF4','ATF5','ATF6','ATF6B','ATF7','ATMIN','ATOH1','ATOH7','ATOH8','BACH1','BACH2','BARHL1','BARHL2','BARX1','BARX2','BATF','BATF2','BATF3','BAZ2A','BAZ2B','BBX','BCL11A','BCL11B','BCL6','BCL6B','BHLHA15','BHLHA9','BHLHE22','BHLHE23','BHLHE40','BHLHE41','BNC1','BNC2','BORCS8-MEF2B','BPTF','BRF2','BSX','C11orf95','CAMTA1','CAMTA2','CARF','CASZ1','CBX2','CC2D1A','CCDC169-SOHLH2','CCDC17','CDC5L','CDX1','CDX2','CDX4','CEBPA','CEBPB','CEBPD','CEBPE','CEBPG','CEBPZ','CENPA','CENPB','CENPBD1','CENPS','CENPT','CENPX','CGGBP1','CHAMP1','CHCHD3','CIC','CLOCK','CPEB1','CPXCR1','CREB1','CREB3','CREB3L1','CREB3L2','CREB3L3','CREB3L4','CREB5','CREBL2','CREBZF','CREM','CRX','CSRNP1','CSRNP2','CSRNP3','CTCF','CTCFL','CUX1','CUX2','CXXC1','CXXC4','CXXC5','DACH1','DACH2','DBP','DBX1','DBX2','DDIT3','DEAF1','DLX1','DLX2','DLX3','DLX4','DLX5','DLX6','DMBX1','DMRT1','DMRT2','DMRT3','DMRTA1','DMRTA2','DMRTB1','DMRTC2','DMTF1','DNMT1','DNTTIP1','DOT1L','DPF1','DPF3','DPRX','DR1','DRAP1','DRGX','DUX1','DUX3','DUX4','DUXA','DZIP1','E2F1','E2F2','E2F3','E2F4','E2F5','E2F6','E2F7','E2F8','E4F1','EBF1','EBF2','EBF3','EBF4','EEA1','EGR1','EGR2','EGR3','EGR4','EHF','ELF1','ELF2','ELF3','ELF4','ELF5','ELK1','ELK3','ELK4','EMX1','EMX2','EN1','EN2','EOMES','EPAS1','ERF','ERG','ESR1','ESR2','ESRRA','ESRRB','ESRRG','ESX1','ETS1','ETS2','ETV1','ETV2','ETV3','ETV3L','ETV4','ETV5','ETV6','ETV7','EVX1','EVX2','FAM170A','FAM200B','FBXL19','FERD3L','FEV','FEZF1','FEZF2','FIGLA','FIZ1','FLI1','FLYWCH1','FOS','FOSB','FOSL1','FOSL2','FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXN4','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1','GABPA','GATA1','GATA2','GATA3','GATA4','GATA5','GATA6','GATAD2A','GATAD2B','GBX1','GBX2','GCM1','GCM2','GFI1','GFI1B','GLI1','GLI2','GLI3','GLI4','GLIS1','GLIS2','GLIS3','GLMP','GLYR1','GMEB1','GMEB2','GPBP1','GPBP1L1','GRHL1','GRHL2','GRHL3','GSC','GSC2','GSX1','GSX2','GTF2B','GTF2I','GTF2IRD1','GTF2IRD2','GTF2IRD2B','GTF3A','GZF1','HAND1','HAND2','HBP1','HDX','HELT','HES1','HES2','HES3','HES4','HES5','HES6','HES7','HESX1','HEY1','HEY2','HEYL','HHEX','HIC1','HIC2','HIF1A','HIF3A','HINFP','HIVEP1','HIVEP2','HIVEP3','HKR1','HLF','HLX','HMBOX1','HMG20A','HMG20B','HMGA1','HMGA2','HMGN3','HMX1','HMX2','HMX3','HNF1A','HNF1B','HNF4A','HNF4G','HOMEZ','HOXA1','HOXA10','HOXA11','HOXA13','HOXA2','HOXA3','HOXA4','HOXA5','HOXA6','HOXA7','HOXA9','HOXB1','HOXB13','HOXB2','HOXB3','HOXB4','HOXB5','HOXB6','HOXB7','HOXB8','HOXB9','HOXC10','HOXC11','HOXC12','HOXC13','HOXC4','HOXC5','HOXC6','HOXC8','HOXC9','HOXD1','HOXD10','HOXD11','HOXD12','HOXD13','HOXD3','HOXD4','HOXD8','HOXD9','HSF1','HSF2','HSF4','HSF5','HSFX1','HSFX2','HSFY1','HSFY2','IKZF1','IKZF2','IKZF3','IKZF4','IKZF5','INSM1','INSM2','IRF1','IRF2','IRF3','IRF4','IRF5','IRF6','IRF7','IRF8','IRF9','IRX1','IRX2','IRX3','IRX4','IRX5','IRX6','ISL1','ISL2','ISX','JAZF1','JDP2','JRK','JRKL','JUN','JUNB','JUND','KAT7','KCMF1','KCNIP3','KDM2A','KDM2B','KDM5B','KIN','KLF1','KLF10','KLF11','KLF12','KLF13','KLF14','KLF15','KLF16','KLF17','KLF2','KLF3','KLF4','KLF5','KLF6','KLF7','KLF8','KLF9','KMT2A','KMT2B','L3MBTL1','L3MBTL3','L3MBTL4','LBX1','LBX2','LCOR','LCORL','LEF1','LEUTX','LHX1','LHX2','LHX3','LHX4','LHX5','LHX6','LHX8','LHX9','LIN28A','LIN28B','LIN54','LMX1A','LMX1B','LTF','LYL1','MAF','MAFA','MAFB','MAFF','MAFG','MAFK','MAX','MAZ','MBD1','MBD2','MBD3','MBD4','MBD6','MBNL2','MECOM','MECP2','MEF2A','MEF2B','MEF2C','MEF2D','MEIS1','MEIS2','MEIS3','MEOX1','MEOX2','MESP1','MESP2','MGA','MITF','MIXL1','MKX','MLX','MLXIP','MLXIPL','MNT','MNX1','MSANTD1','MSANTD3','MSANTD4','MSC','MSGN1','MSX1','MSX2','MTERF1','MTERF2','MTERF3','MTERF4','MTF1','MTF2','MXD1','MXD3','MXD4','MXI1','MYB','MYBL1','MYBL2','MYC','MYCL','MYCN','MYF5','MYF6','MYNN','MYOD1','MYOG','MYPOP','MYRF','MYRFL','MYSM1','MYT1','MYT1L','MZF1','NACC2','NAIF1','NANOG','NANOGNB','NANOGP8','NCOA1','NCOA2','NCOA3','NEUROD1','NEUROD2','NEUROD4','NEUROD6','NEUROG1','NEUROG2','NEUROG3','NFAT5','NFATC1','NFATC2','NFATC3','NFATC4','NFE2','NFE2L1','NFE2L2','NFE2L3','NFE4','NFIA','NFIB','NFIC','NFIL3','NFIX','NFKB1','NFKB2','NFX1','NFXL1','NFYA','NFYB','NFYC','NHLH1','NHLH2','NKRF','NKX1-1','NKX1-2','NKX2-1','NKX2-2','NKX2-3','NKX2-4','NKX2-5','NKX2-6','NKX2-8','NKX3-1','NKX3-2','NKX6-1','NKX6-2','NKX6-3','NME2','NOBOX','NOTO','NPAS1','NPAS2','NPAS3','NPAS4','NR0B1','NR1D1','NR1D2','NR1H2','NR1H3','NR1H4','NR1I2','NR1I3','NR2C1','NR2C2','NR2E1','NR2E3','NR2F1','NR2F2','NR2F6','NR3C1','NR3C2','NR4A1','NR4A2','NR4A3','NR5A1','NR5A2','NR6A1','NRF1','NRL','OLIG1','OLIG2','OLIG3','ONECUT1','ONECUT2','ONECUT3','OSR1','OSR2','OTP','OTX1','OTX2','OVOL1','OVOL2','OVOL3','PA2G4','PATZ1','PAX1','PAX2','PAX3','PAX4','PAX5','PAX6','PAX7','PAX8','PAX9','PBX1','PBX2','PBX3','PBX4','PCGF2','PCGF6','PDX1','PEG3','PGR','PHF1','PHF19 ','PHF20','PHF21A','PHOX2A','PHOX2B','PIN1','PITX1','PITX2','PITX3','PKNOX1','PKNOX2','PLAG1','PLAGL1','PLAGL2','PLSCR1','POGK','POU1F1','POU2AF1','POU2F1','POU2F2','POU2F3','POU3F1','POU3F2','POU3F3','POU3F4','POU4F1','POU4F2','POU4F3','POU5F1','POU5F1B','POU5F2','POU6F1','POU6F2','PPARA','PPARD','PPARG','PRDM1','PRDM10','PRDM12','PRDM13','PRDM14','PRDM15','PRDM16','PRDM2','PRDM4','PRDM5','PRDM6','PRDM8','PRDM9','PREB','PRMT3','PROP1','PROX1','PROX2','PRR12','PRRX1','PRRX2','PTF1A','PURA','PURB','PURG','RAG1','RARA','RARB','RARG','RAX','RAX2','RBAK','RBCK1','RBPJ','RBPJL','RBSN','REL','RELA','RELB','REPIN1','REST','REXO4','RFX1','RFX2','RFX3','RFX4','RFX5','RFX6','RFX7','RFX8','RHOXF1','RHOXF2','RHOXF2B','RLF','RORA','RORB','RORC','RREB1','RUNX1','RUNX2','RUNX3','RXRA','RXRB','RXRG','SAFB','SAFB2','SALL1','SALL2','SALL3','SALL4','SATB1','SATB2','SCMH1','SCML4','SCRT1','SCRT2','SCX','SEBOX','SETBP1','SETDB1','SETDB2','SGSM2','SHOX','SHOX2','SIM1','SIM2','SIX1','SIX2','SIX3','SIX4','SIX5','SIX6','SKI','SKIL','SKOR1','SKOR2','SLC2A4RG','SMAD1','SMAD3','SMAD4','SMAD5','SMAD9','SMYD3','SNAI1','SNAI2','SNAI3','SNAPC2','SNAPC4','SNAPC5','SOHLH1','SOHLH2','SON','SOX1','SOX10','SOX11']
TFs2=['SOX12','SOX13','SOX14','SOX15','SOX17','SOX18','SOX2','SOX21','SOX3','SOX30','SOX4','SOX5','SOX6','SOX7','SOX8','SOX9','SP1','SP100','SP110','SP140','SP140L','SP2','SP3','SP4','SP5','SP6','SP7','SP8','SP9','SPDEF','SPEN','SPI1','SPIB','SPIC','SPZ1','SRCAP','SREBF1','SREBF2','SRF','SRY','ST18','STAT1','STAT2','STAT3','STAT4','STAT5A','STAT5B','STAT6','T','TAL1','TAL2','TBP','TBPL1','TBPL2','TBR1','TBX1','TBX10','TBX15','TBX18','TBX19','TBX2','TBX20','TBX21','TBX22','TBX3','TBX4','TBX5','TBX6','TCF12','TCF15','TCF20','TCF21','TCF23','TCF24','TCF3','TCF4','TCF7','TCF7L1','TCF7L2','TCFL5','TEAD1','TEAD2','TEAD3','TEAD4','TEF','TERB1','TERF1','TERF2','TET1','TET2','TET3','TFAP2A','TFAP2B','TFAP2C','TFAP2D','TFAP2E','TFAP4','TFCP2','TFCP2L1','TFDP1','TFDP2','TFDP3','TFE3','TFEB','TFEC','TGIF1','TGIF2','TGIF2LX','TGIF2LY','THAP1','THAP10','THAP11','THAP12','THAP2','THAP3','THAP4','THAP5','THAP6','THAP7','THAP8','THAP9','THRA','THRB','THYN1','TIGD1','TIGD2','TIGD3','TIGD4','TIGD5','TIGD6','TIGD7','TLX1','TLX2','TLX3','TMF1','TOPORS','TP53','TP63','TP73','TPRX1','TRAFD1','TRERF1','TRPS1','TSC22D1','TSHZ1','TSHZ2','TSHZ3','TTF1','TWIST1','TWIST2','UBP1','UNCX','USF1','USF2','USF3','VAX1','VAX2','VDR','VENTX','VEZF1','VSX1','VSX2','WIZ','WT1','XBP1','XPA','YBX1','YBX2','YBX3','YY1','YY2','ZBED1','ZBED2','ZBED3','ZBED4','ZBED5','ZBED6','ZBED9','ZBTB1','ZBTB10','ZBTB11','ZBTB12','ZBTB14','ZBTB16','ZBTB17','ZBTB18','ZBTB2','ZBTB20','ZBTB21','ZBTB22','ZBTB24','ZBTB25','ZBTB26','ZBTB3','ZBTB32','ZBTB33','ZBTB34','ZBTB37','ZBTB38','ZBTB39','ZBTB4','ZBTB40','ZBTB41','ZBTB42','ZBTB43','ZBTB44','ZBTB45','ZBTB46','ZBTB47','ZBTB48','ZBTB49','ZBTB5','ZBTB6','ZBTB7A','ZBTB7B','ZBTB7C','ZBTB8A','ZBTB8B','ZBTB9','ZC3H8','ZEB1','ZEB2','ZFAT','ZFHX2','ZFHX3','ZFHX4','ZFP1','ZFP14','ZFP2','ZFP28','ZFP3','ZFP30','ZFP37','ZFP41','ZFP42','ZFP57','ZFP62','ZFP64','ZFP69','ZFP69B','ZFP82','ZFP90','ZFP91','ZFP92','ZFPM1','ZFPM2','ZFX','ZFY','ZGLP1','ZGPAT','ZHX1','ZHX2','ZHX3','ZIC1','ZIC2','ZIC3','ZIC4','ZIC5','ZIK1','ZIM2','ZIM3','ZKSCAN1','ZKSCAN2','ZKSCAN3','ZKSCAN4','ZKSCAN5','ZKSCAN7','ZKSCAN8','ZMAT1','ZMAT4','ZNF10','ZNF100','ZNF101','ZNF107','ZNF112','ZNF114','ZNF117','ZNF12','ZNF121','ZNF124','ZNF131','ZNF132','ZNF133','ZNF134','ZNF135','ZNF136','ZNF138','ZNF14','ZNF140','ZNF141','ZNF142','ZNF143','ZNF146','ZNF148','ZNF154','ZNF155','ZNF157','ZNF16','ZNF160','ZNF165','ZNF169','ZNF17','ZNF174','ZNF175','ZNF177','ZNF18','ZNF180','ZNF181','ZNF182','ZNF184','ZNF189','ZNF19','ZNF195','ZNF197','ZNF2','ZNF20','ZNF200','ZNF202','ZNF205','ZNF207','ZNF208','ZNF211','ZNF212','ZNF213','ZNF214','ZNF215','ZNF217','ZNF219','ZNF22','ZNF221','ZNF222','ZNF223','ZNF224','ZNF225','ZNF226','ZNF227','ZNF229','ZNF23','ZNF230','ZNF232','ZNF233','ZNF234','ZNF235','ZNF236','ZNF239','ZNF24','ZNF248','ZNF25','ZNF250','ZNF251','ZNF253','ZNF254','ZNF256','ZNF257','ZNF26','ZNF260','ZNF263','ZNF264','ZNF266','ZNF267','ZNF268','ZNF273','ZNF274','ZNF275','ZNF276','ZNF277','ZNF28','ZNF280A','ZNF280B','ZNF280C','ZNF280D','ZNF281','ZNF282','ZNF283','ZNF284','ZNF285','ZNF286A','ZNF286B','ZNF287','ZNF292','ZNF296','ZNF3','ZNF30','ZNF300','ZNF302','ZNF304','ZNF311','ZNF316','ZNF317','ZNF318','ZNF319','ZNF32','ZNF320','ZNF322','ZNF324','ZNF324B','ZNF326','ZNF329','ZNF331','ZNF333','ZNF334','ZNF335','ZNF337','ZNF33A','ZNF33B','ZNF34','ZNF341','ZNF343','ZNF345','ZNF346','ZNF347','ZNF35','ZNF350','ZNF354A','ZNF354B','ZNF354C','ZNF358','ZNF362','ZNF365','ZNF366','ZNF367','ZNF37A','ZNF382','ZNF383','ZNF384','ZNF385A','ZNF385B','ZNF385C','ZNF385D','ZNF391','ZNF394','ZNF395','ZNF396','ZNF397','ZNF398','ZNF404','ZNF407','ZNF408','ZNF41','ZNF410','ZNF414','ZNF415','ZNF416','ZNF417','ZNF418','ZNF419','ZNF420','ZNF423','ZNF425','ZNF426','ZNF428','ZNF429','ZNF43','ZNF430','ZNF431','ZNF432','ZNF433','ZNF436','ZNF438','ZNF439','ZNF44','ZNF440','ZNF441','ZNF442','ZNF443','ZNF444','ZNF445','ZNF446','ZNF449','ZNF45','ZNF451','ZNF454','ZNF460','ZNF461','ZNF462','ZNF467','ZNF468','ZNF469','ZNF470','ZNF471','ZNF473','ZNF474','ZNF479','ZNF48','ZNF480','ZNF483','ZNF484','ZNF485','ZNF486','ZNF487','ZNF488','ZNF490','ZNF491','ZNF492','ZNF493','ZNF496','ZNF497','ZNF500','ZNF501','ZNF502','ZNF503','ZNF506','ZNF507','ZNF510','ZNF511','ZNF512','ZNF512B','ZNF513','ZNF514','ZNF516','ZNF517','ZNF518A','ZNF518B','ZNF519','ZNF521','ZNF524','ZNF525','ZNF526','ZNF527','ZNF528','ZNF529','ZNF530','ZNF532','ZNF534','ZNF536','ZNF540','ZNF541','ZNF543','ZNF544','ZNF546','ZNF547','ZNF548','ZNF549','ZNF550','ZNF551','ZNF552','ZNF554','ZNF555','ZNF556','ZNF557','ZNF558','ZNF559','ZNF560','ZNF561','ZNF562','ZNF563','ZNF564','ZNF565','ZNF566','ZNF567','ZNF568','ZNF569','ZNF57','ZNF570','ZNF571','ZNF572','ZNF573','ZNF574','ZNF575','ZNF576','ZNF577','ZNF578','ZNF579','ZNF580','ZNF581','ZNF582','ZNF583','ZNF584','ZNF585A','ZNF585B','ZNF586','ZNF587','ZNF587B','ZNF589','ZNF592','ZNF594','ZNF595','ZNF596','ZNF597','ZNF598','ZNF599','ZNF600','ZNF605','ZNF606','ZNF607','ZNF608','ZNF609','ZNF610','ZNF611','ZNF613','ZNF614','ZNF615','ZNF616','ZNF618','ZNF619','ZNF620','ZNF621','ZNF623','ZNF624','ZNF625','ZNF626','ZNF627','ZNF628','ZNF629','ZNF630','ZNF639','ZNF641','ZNF644','ZNF645','ZNF646','ZNF648','ZNF649','ZNF652','ZNF653','ZNF654','ZNF655','ZNF658','ZNF66','ZNF660','ZNF662','ZNF664','ZNF665','ZNF667','ZNF668','ZNF669','ZNF670','ZNF671','ZNF672','ZNF674','ZNF675','ZNF676','ZNF677','ZNF678','ZNF679','ZNF680','ZNF681','ZNF682','ZNF683','ZNF684','ZNF687','ZNF688','ZNF689','ZNF69','ZNF691','ZNF692','ZNF695','ZNF696','ZNF697','ZNF699','ZNF7','ZNF70','ZNF700','ZNF701','ZNF703','ZNF704','ZNF705A','ZNF705B','ZNF705D','ZNF705E','ZNF705G','ZNF706','ZNF707','ZNF708','ZNF709','ZNF71','ZNF710','ZNF711','ZNF713','ZNF714','ZNF716','ZNF717','ZNF718','ZNF721','ZNF724','ZNF726','ZNF727','ZNF728','ZNF729','ZNF730','ZNF732','ZNF735','ZNF736','ZNF737','ZNF74','ZNF740','ZNF746','ZNF747','ZNF749','ZNF750','ZNF75A','ZNF75D','ZNF76','ZNF761','ZNF763','ZNF764','ZNF765','ZNF766','ZNF768','ZNF77','ZNF770','ZNF771','ZNF772','ZNF773','ZNF774','ZNF775','ZNF776','ZNF777','ZNF778','ZNF780A','ZNF780B','ZNF781','ZNF782','ZNF783','ZNF784','ZNF785','ZNF786','ZNF787','ZNF788','ZNF789','ZNF79','ZNF790','ZNF791','ZNF792','ZNF793','ZNF799','ZNF8','ZNF80','ZNF800','ZNF804A','ZNF804B','ZNF805','ZNF808','ZNF81','ZNF813','ZNF814','ZNF816','ZNF821','ZNF823','ZNF827','ZNF829','ZNF83','ZNF830','ZNF831','ZNF835','ZNF836','ZNF837','ZNF84','ZNF841','ZNF843','ZNF844','ZNF845','ZNF846','ZNF85','ZNF850','ZNF852','ZNF853','ZNF860','ZNF865','ZNF878','ZNF879','ZNF880','ZNF883','ZNF888','ZNF891','ZNF90','ZNF91','ZNF92','ZNF93','ZNF98','ZNF99','ZSCAN1','ZSCAN10','ZSCAN12','ZSCAN16','ZSCAN18','ZSCAN2','ZSCAN20','ZSCAN21','ZSCAN22','ZSCAN23','ZSCAN25','ZSCAN26','ZSCAN29','ZSCAN30','ZSCAN31','ZSCAN32','ZSCAN4','ZSCAN5A','ZSCAN5B','ZSCAN5C','ZSCAN9','ZUFSP','ZXDA','ZXDB','ZXDC','ZZZ3']
TFs=TFs1+TFs2

#TFs other, inclusive of the above list, not necessarily DNA binding proteins but affect transcription
TFs1Other=['AATF','ABL1','ABRA','AC008770.3','AC020915.1','AC023509.3','AC092835.1','AC105001.2','AC138696.1','ACOD1','ACTR3B','ACTR3C','ADAM8','ADAMTS17','ADAMTS19','ADAR','ADGRA1','ADGRG3','ADNP','ADNP2','AEBP1','AEBP2','AES','AFAP1 ','AFF1','AFF3','AFF4','AGER','AGT','AHCTF1','AHDC1','AHR','AHRR','AIM2','AIRE','AKAP8','AKAP8L','AKNA','AKT1','ALK','ALKBH2','ALX1','ALX3','ALX4','AMH','ANAPC2','ANHX','ANKRD30A','ANKZF1','ANXA3','ANXA4','AP5Z1','APBB3','APC2','APEX1','APOA4','APOBEC3A','APOBEC3B','APOBEC3G','APTX','AR','ARGFX','ARHGAP35','ARHGEF2','ARHGEF5','ARID1A','ARID1B','ARID2','ARID3A','ARID3B','ARID3C','ARID4A','ARID4B','ARID5A','ARID5B','ARNT','ARNT2','ARNTL','ARNTL2','ARRB1','ARRB2','ARX','ASCC1','ASCL1','ASCL2','ASCL3','ASCL4','ASCL5','ASH1L','ATF1','ATF2','ATF3','ATF4','ATF5','ATF6','ATF6B','ATF7','ATF7IP','ATM','ATMIN','ATOH1','ATOH7','ATOH8','ATR','ATRX','ATXN7','BACH1','BACH2','BAG1','BANF1','BANP','BARHL1','BARHL2','BARX1','BARX2','BATF','BATF2','BATF3','BAZ2A','BAZ2B','BBX','BCL10','BCL11A','BCL11B','BCL3','BCL6','BCL6B','BCLAF1','BCOR','BDP1','BEND4','BEX1','BHLHA15','BHLHA9','BHLHE22','BHLHE23','BHLHE40','BHLHE41','BHMG1','BIN1','BLM','BLZF1','BMI1','BMP2','BMP4','BMP7','BMPR1A','BNC1','BNC2','BOD1L1','BOLA1','BOLA2','BOLA2B','BOLA3','BORCS8-MEF2B','BPNT1','BPTF','BRCA1','BRD4','BRD7','BRD8','BRD9','BRF1','BRF2','BRMS1','BRPF1','BSX','BTAF1','BTBD8','BTG2','BTK','BTRC','BUD31','C11orf95','C14orf39','C17orf49','C1D','C20orf194','C3orf33','CACTIN','CAMK1D','CAMK2A','CAMTA1','CAMTA2','CAPN15','CAPN3','CARD11','CARD14','CARF','CARHSP1','CASP8AP2','CASZ1','CAT','CBFA2T2','CBFA2T3','CBFB','CBL','CBLL1','CBX2','CC2D1A','CC2D1B','CCDC124','CCDC169-SOHLH2','CCDC17','CCDC3','CCDC71','CCDC83','CCNT1','CCNT2','CD36','CD40','CD40LG','CDC5L','CDC6','CDK5RAP3','CDK7','CDK8','CDK9','CDKN1A','CDKN2A','CDX1','CDX2','CDX4','CEBPA','CEBPB','CEBPD','CEBPE','CEBPG','CEBPZ','CELF3','CENPA','CENPB','CENPBD1','CENPS','CENPT','CENPX','CERS2','CERS3','CERS4','CERS5','CERS6','CFLAR','CGGBP1','CHAMP1','CHCHD3','CHD1','CHD2','CHD3','CHD4','CHD6','CHD7','CHD9','CHP1','CHRAC1','CHUK','CHURC1','CIB1','CIC','CIITA','CIR1','CITED1','CITED2','CITED4','CIZ1','CLOCK','CLU','CMKLR1','CNBP','CNOT3','CNOT4','CNOT7','CNOT8','COMMD1','COMMD6','COMMD7','COPS2','COPS3','COPS4','CPEB1','CPSF4','CPSF4L','CPXCR1','CRAMP1','CREB1','CREB3','CREB3L1','CREB3L2','CREB3L3','CREB3L4','CREB5','CREBBP','CREBL2','CREBRF','CREBZF','CREG1','CREM','CRTC1','CRTC2','CRTC3','CRX','CSDC2','CSDE1','CSRNP1','CSRNP2','CSRNP3','CTBP1','CTBP2','CTCF','CTCFL','CTH','CTNNB1','CTNNBIP1','CUL1','CUL2','CUL3','CUL4A','CUL4B','CUL5','CUX1','CUX2','CXXC1','CXXC4','CXXC5','CYLD','CYP1B1','CYTL1','DAB2IP','DACH1','DACH2','DAP','DAXX','DBP','DBX1','DBX2','DCP1A','DDB1','DDB2','DDIT3','DDN','DDR2','DDRGK1','DDX3X','DDX5','DDX58','DEAF1','DEK','DEPDC1','DEPDC1B','DEPDC4','DEPDC5','DEPDC7','DEPTOR','DHX34','DHX36','DHX57','DISP3','DLX1','DLX2','DLX3','DLX4','DLX5','DLX6','DMAP1','DMBX1','DMC1','DMRT1','DMRT2','DMRT3','DMRTA1','DMRTA2','DMRTB1','DMRTC1','DMRTC2','DMTF1','DNA2','DNAJA3','DNAJC1','DNAJC2','DNAJC21','DNMT1','DNMT3A','DNMT3B','DNMT3L','DNTT','DNTTIP1','DOT1L','DPF1','DPF2','DPF3','DPRX','DR1','DRAP1','DRGX','DSP','DTX1','DUS3L','DUSP12','DUX1','DUX3','DUX4','DUXA','DVL1','DVL2','DVL3','DZIP1','DZIP1L','E2F1','E2F2','E2F3','E2F4','E2F5','E2F6','E2F7','E2F8','E4F1','EAF2','EBF1','EBF2','EBF3','EBF4','EDA','EDA2R','EDF1','EDN1','EEA1','EED','EGLN1','EGR1','EGR2','EGR3','EGR4','EHF','EID1','EID2','EIF2AK2','EIF2AK4','EIF3C','EIF3CL','EIF3K','ELF1','ELF2','ELF3','ELF4','ELF5','ELK1','ELK3','ELK4','ELMSAN1','EME1','EMX1','EMX2','EN1','EN2','ENO1','EOMES','EP300','EP400','EPAS1','EPHA5','EPM2AIP1','ERBIN','ERC1','ERCC3','ERCC4','ERF','ERG','ESR1','ESR2','ESRRA','ESRRB','ESRRG','ESX1','ETS1','ETS2','ETV1','ETV2','ETV3','ETV3L','ETV4','ETV5','ETV6','ETV7','EVX1','EVX2','EWSR1','EXO1','EXOC2','EXOG','EZH1','EZH2','FAAP24','FAM153C','FAM170A','FAM171B','FAM200A','FAM200B','FAN1','FANCA','FANCD2','FANK1','FARSA','FARSB','FBN1','FBXL19','FBXO41','FEN1','FER','FERD3L','FEV','FEZF1','FEZF2','FGD1','FHL2','FIGLA','FIZ1','FLI1','FLNA','FLYWCH1','FMNL2','FOS','FOSB','FOSL1','FOSL2','FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L2','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXN4','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1','FUBP1','FUBP3','FUS','FZD1','FZD2','FZD4','FZD6','GABPA','GABPB1','GABPB2','GADD45A','GAS6','GAS7','GATA1','GATA2','GATA3','GATA4','GATA5','GATA6','GATAD1','GATAD2A','GATAD2B','GBX1','GBX2','GCFC2','GCM1','GCM2','GEN1','GFI1','GFI1B','GLA','GLI1','GLI2','GLI3','GLI4','GLIS1','GLIS2','GLIS3','GLMP','GLYR1','GMEB1','GMEB2','GOLGB1','GON4L','GPATCH8','GPBP1','GPBP1L1','GPER1','GPR155','GREM1','GRHL1','GRHL2','GRHL3','GRM6','GSC','GSC2','GSX1','GSX2','GTF2A1','GTF2A1L','GTF2A2','GTF2B','GTF2E2','GTF2F1','GTF2F2','GTF2H1','GTF2H2','GTF2H2C','GTF2H3','GTF2H4','GTF2I','GTF2IRD1','GTF2IRD2','GTF2IRD2B','GTF3A','GTF3C1','GTF3C2','GULP1','GZF1','H1F0','H1FOO','H1FX','HAND1','HAND2','HAVCR2','HBP1','HCFC1','HCK','HDAC1','HDAC10','HDAC2','HDAC3','HDAC4','HDAC5','HDAC6','HDAC7','HDAC8','HDAC9','HDX','HELT','HELZ','HELZ2','HES1','HES2','HES3','HES4','HES5','HES6','HES7','HESX1','HEY1','HEY2','HEYL','HHEX','HIC1','HIC2','HIF1A','HIF3A','HINFP','HIPK2','HIRA','HIST1H1A','HIST1H1B','HIST1H1C','HIST1H1D','HIST1H1E','HIST1H1T','HIVEP1','HIVEP2','HIVEP3','HKR1','HLA-DQB1','HLA-DQB2','HLA-DRB1','HLA-DRB3','HLA-DRB5','HLF','HLTF','HLX','HMBOX1','HMG20A','HMG20B','HMGA1','HMGA2','HMGB1','HMGB2','HMGB3','HMGB4','HMGN1','HMGN3','HMGXB3','HMGXB4','HMOX1','HMX1','HMX2','HMX3','HNF1A','HNF1B','HNF4A','HNF4G','HNRNPA1','HNRNPA3','HNRNPAB','HNRNPCL2','HNRNPD','HNRNPDL','HNRNPK','HNRNPL','HNRNPU','HOMEZ','HOOK2','HOPX','HOXA1','HOXA10','HOXA11','HOXA13','HOXA2','HOXA3','HOXA4','HOXA5','HOXA6','HOXA7','HOXA9','HOXB1','HOXB13','HOXB2','HOXB3','HOXB4','HOXB5','HOXB6','HOXB7','HOXB8','HOXB9','HOXC10','HOXC11','HOXC12','HOXC13','HOXC4','HOXC5','HOXC6','HOXC8','HOXC9','HOXD1','HOXD10','HOXD11','HOXD12','HOXD13','HOXD3','HOXD4','HOXD8','HOXD9','HP1BP3','HR','HSBP1','HSF1','HSF2','HSF4','HSF5','HSFX1','HSFX2','HSFY1','HSFY2','HSPA1A','HSPA1B','ICAM1','ID1','ID2','ID3','ID4','IFI16','IFNB1','IGHM','IKBKB','IKBKG','IKZF1','IKZF2','IKZF3','IKZF4','IKZF5','IL10','IL1B','IL1RAP','IL4','IL5','IL6','INF2','ING4','INS','INSM1','INSM2','IQGAP1','IRAK1','IRAK2','IRAK3','IRF1','IRF2','IRF2BP1','IRF3','IRF4','IRF5','IRF6','IRF7','IRF8','IRF9','IRX1','IRX2','IRX3','IRX4','IRX5','IRX6','ISL1','ISL2','ISX','ITCH','ITGB2','ITGB3BP','IZUMO2','JADE1','JAK2','JARID2','JAZF1','JDP2','JMY','JRK','JRKL','JUN','JUNB','JUND','JUP','KANSL2','KAT2A','KAT2B','KAT5','KAT6A','KAT6B','KAT7','KAT8','KCMF1','KCNIP3','KDM1A','KDM2A','KDM2B','KDM3A','KDM5A','KDM5B','KDM5C','KDM5D','KEAP1','KIAA1549','KIAA1683','KIAA1841','KIAA1958','KIF15','KIN','KIT','KLF1','KLF10','KLF11','KLF12','KLF13','KLF14','KLF15','KLF16','KLF17','KLF2','KLF3','KLF4','KLF5','KLF6','KLF7','KLF8','KLF9','KMT2A','KMT2B','KMT2C','KMT2D','KNTC1','KRAS','KRBOX1','KRT13','KRTAP5-1','KRTAP5-9','KRTAP9-9','L3MBTL1','L3MBTL3','L3MBTL4','LARP1','LARP1B','LARP4','LARP4B','LARP6','LARP7','LBX1','LBX2','LCOR','LCORL','LEF1','LEUTX','LGALS9','LGR4','LHX1','LHX2','LHX3','LHX4','LHX5','LHX6','LHX8','LHX9','LIG1','LIG3','LIN28A','LIN28B','LIN54','LMO2','LMO4','LMX1A','LMX1B','LPP','LRP5','LRP6','LRP8','LRRFIP1','LRRFIP2','LSR','LTF','LTN1','LYAR','LYL1','LZTFL1','LZTR1','LZTS1','MACC1','MACF1','MAD2L2','MAEL','MAF','MAFA','MAFB','MAFF','MAFG','MAFK','MALT1','MAML1','MAML3','MAP2K5','MAP3K10','MAP3K13','MAP3K7','MAPK1','MAPK10','MAPK11','MAPK14','MAPK3','MAPK8','MAPK9','MATR3','MAVS','MAX','MAZ','MB21D1','MBD1','MBD2','MBD3','MBD4','MBD6','MBNL1','MBNL2','MBNL3','MBTPS2','MDFI','MDM2','MDM4','MECOM','MECP2','MED1','MED12','MED13','MED13L','MED14','MED15','MED16','MED17','MED21','MED23','MED24','MED6','MEF2A','MEF2B','MEF2C','MEF2D','MEIS1','MEIS2','MEIS3','MEN1','MEOX1','MEOX2','MESP1','MESP2','MET','MGA','MGMT','MID2','MIER1','MIER2','MIER3','MINK1','MIS18BP1','MITF','MIXL1','MKL1','MKL2','MKRN1','MKRN2','MKRN3','MKX','MLLT1','MLLT10','MLLT11','MLLT3','MLLT6','MLX','MLXIP','MLXIPL','MN1','MNAT1','MNT','MNX1','MORC3','MPG','MRPL28','MRRF','MSANTD1','MSANTD3','MSANTD4','MSC','MSGN1','MSH2','MSH3','MSH6','MSL3','MSRB2','MST1R','MSX1','MSX2','MTA1','MTA2','MTA3','MTDH','MTERF1','MTERF2','MTERF3','MTERF4','MTF1','MTF2','MTPN','MUS81','MXD1','MXD3','MXD4','MXI1','MYB','MYBBP1A','MYBL1','MYBL2','MYC','MYCBP','MYCL','MYCN','MYD88','MYEF2','MYF5','MYF6','MYNN','MYOCD','MYOD1','MYOG','MYPOP','MYRF','MYRFL','MYSM1','MYT1','MYT1L','MZF1','NAB2','NABP2','NACC2','NAIF1','NANOG','NANOGNB','NANOGP8','NCOA1','NCOA2','NCOA3','NCOA4','NCOA5','NCOA6','NCOA7','NCOR1','NCOR2','NDC80','NDN','NDP','NEB','NEIL1','NEUROD1','NEUROD2','NEUROD4','NEUROD6','NEUROG1','NEUROG2','NEUROG3','NFAM1','NFAT5','NFATC1','NFATC2','NFATC3','NFATC4','NFE2','NFE2L1','NFE2L2','NFE2L3','NFE4','NFIA','NFIB','NFIC','NFIL3','NFIX','NFKB1','NFKB2','NFKBIA','NFKBIB','NFKBID','NFKBIL1','NFKBIZ','NFRKB','NFX1','NFXL1','NFYA','NFYB','NFYC','NHLH1','NHLH2','NKRF','NKX1-1','NKX1-2','NKX2-1','NKX2-2','NKX2-3','NKX2-4','NKX2-5','NKX2-6','NKX2-8','NKX3-1','NKX3-2','NKX6-1','NKX6-2','NKX6-3','NLRC3','NLRC4','NLRC5','NLRP12','NLRP2B','NLRP3','NME2','NOBOX','NOC3L','NOC4L','NOCT','NOD1','NOD2','NODAL','NONO','NOP2','NOTCH1','NOTCH2','NOTCH3','NOTO','NPAS1','NPAS2','NPAS3','NPAS4','NPM1','NR0B1','NR0B2','NR1D1','NR1D2','NR1H2','NR1H3','NR1H4','NR1I2','NR1I3','NR2C1','NR2C2','NR2E1','NR2E3','NR2F1','NR2F2','NR2F6','NR3C1','NR3C2','NR4A1','NR4A2','NR4A3','NR5A1','NR5A2','NR6A1','NRBF2','NRF1','NRG1','NRIP1','NRIP3','NRK','NRL','NSD1','NSD2','NTRK1','NUCKS1','NUFIP1','NUP98','NUPL2']
TFs2Other=['OGG1','OLIG1','OLIG2','OLIG3','ONECUT1','ONECUT2','ONECUT3','OPRD1','OSR1','OSR2','OTOP3','OTP','OTULIN','OTX1','OTX2','OVOL1','OVOL2','OVOL3','PA2G4','PABPN1','PARK7','PARP1','PARP10','PARP12','PATZ1','PAWR','PAX1','PAX2','PAX3','PAX4','PAX5','PAX6','PAX7','PAX8','PAX9','PAXBP1','PAXIP1','PBRM1','PBX1','PBX2','PBX3','PBX4','PCBD1','PCBP1','PCBP2','PCBP3','PCGF2','PCGF6','PCNA','PCSK6','PDCD7','PDS5B','PDX1','PEG3','PELI1','PER1','PER2','PER3','PEX14','PFDN1','PGBD1','PGR','PGS1','PHB','PHB2','PHF1','PHF19 ','PHF20','PHF21A','PHF5A','PHOX2A','PHOX2B','PHTF1','PHTF2','PIAS1','PIAS2','PIAS3','PIAS4','PIDD1','PIKFYVE','PIM1','PIN1','PINK1','PITX1','PITX2','PITX3','PKHD1','PKHD1L1','PKNOX1','PKNOX2','PLA2G10','PLA2G1B','PLAG1','PLAGL1','PLAGL2','PLEK','PLEK2','PLEKHA4','PLPP3','PLSCR1','PLXNA1','PLXNA2','PLXNA3','PLXNA4','PLXNB1','PLXNB2','PLXNB3','PLXNC1','PLXND1','PML','PMS1','PNKP','POGK','POGZ','POLA1','POLB','POLE3','POLE4','POLG','POLH','POLI','POLK','POLL','POLM','POLN','POLQ','POLR1B','POLR2A','POLR2B','POLR2E','POLR2J','POLR2K','POLR2L','POLR3A','POLRMT','POT1','POU1F1','POU2AF1','POU2F1','POU2F2','POU2F3','POU3F1','POU3F2','POU3F3','POU3F4','POU4F1','POU4F2','POU4F3','POU5F1','POU5F1B','POU5F2','POU6F1','POU6F2','PPARA','PPARD','PPARG','PPARGC1A','PPARGC1B','PPP1R10','PPP1R13L','PPP1R21','PPP2R3B','PPP2R5B','PPP3CA','PPRC1','PQBP1','PRAM1','PRAP1','PRB3','PRDM1','PRDM10','PRDM11','PRDM12','PRDM13','PRDM14','PRDM15','PRDM16','PRDM2','PRDM4','PRDM5','PRDM6','PRDM7','PRDM8','PRDM9','PRDX3','PREB','PREX1','PREX2','PRH2','PRIM2','PRIMPOL','PRKCB','PRKCH','PRKCI','PRKCQ','PRKCZ','PRKD1','PRKD2','PRKN','PRMT2','PRMT3','PRNP','PROP1','PROX1','PROX2','PRR12','PRR3','PRRX1','PRRX2','PSMA6','PSMC5','PSMD10','PSMD11','PSMD12','PSPC1','PTCH1','PTEN','PTF1A','PTGIS','PTH','PTTG1','PUM3','PURA','PURB','PURG','PYCARD','PYDC1','PYDC2','QRICH1','RAB11FIP4','RAB7B','RAD21','RAD51','RAG1','RAI1','RAPGEF3','RAPGEF4','RAPGEF5','RARA','RARB','RARG','RAX','RAX2','RB1','RBAK','RBBP7','RBCK1','RBL1','RBL2','RBM10','RBM20','RBM22','RBM26','RBM27','RBM5','RBM6','RBPJ','RBPJL','RBSN','RC3H1','RC3H2','RCAN1','RCOR1','RCOR2','RCOR3','RECQL','REL','RELA','RELB','RELL2','RELN','REPIN1','RERE','REST','REV1','REXO4','RFC1','RFX1','RFX2','RFX3','RFX4','RFX5','RFX6','RFX7','RFX8','RFXANK','RFXAP','RGCC','RGS11','RGS6','RGS7','RGS9','RHEBL1','RHOXF1','RHOXF2','RHOXF2B','RING1','RIOK2','RIPK1','RIPK2','RIPK3','RIPK4','RLF','RLIM','RNASE2','RNASEH1','RNF113A','RNF113B','RNF114','RNF125','RNF138','RNF14','RNF166','RNF2','RNF219','RNF25','RNF31','RNF4','RNF41','RORA','RORB','RORC','RPA2','RPA4','RPL36A','RPL7','RPL7A','RPS27A','RPS6KA4','RPS6KA5','RREB1','RRN3','RUNX1','RUNX1T1','RUNX2','RUNX3','RWDD3','RXRA','RXRB','RXRG','RYBP','S100A12','S100A8','S100A9','SAFB','SAFB2','SALL1','SALL2','SALL3','SALL4','SART1','SATB1','SATB2','SAV1','SCAND1','SCAND2P','SCAPER','SCMH1','SCML1','SCML2','SCML4','SCOC','SCRT1','SCRT2','SCX','SEBOX','SEMA4A','SERTAD1','SERTAD2','SETBP1','SETD6','SETDB1','SETDB2','SF3A2','SF3A3','SFPQ','SFRP4','SFRP5','SFTPB','SGK1','SGK3','SGSM2','SHH','SHOX','SHOX2','SHPRH','SIGIRR','SIK1','SIM1','SIM2','SIN3A','SIN3B','SIRT1','SIRT2','SIRT3','SIRT6','SIVA1','SIX1','SIX2','SIX3','SIX4','SIX5','SIX6','SKI','SKIDA1','SKIL','SKOR1','SKOR2','SLC22A4','SLC26A10','SLC26A3','SLC2A4RG','SLC30A9','SLC39A10','SLC4A10','SMAD1','SMAD2','SMAD3','SMAD4','SMAD5','SMAD6','SMAD7','SMAD9','SMARCA1','SMARCA2','SMARCA4','SMARCA5','SMARCAL1','SMARCB1','SMARCC1','SMARCC2','SMARCD1','SMARCD2','SMARCD3','SMARCE1','SMC3','SMO','SMUG1','SMYD1','SMYD3','SNAI1','SNAI2','SNAI3','SNAPC2','SNAPC4','SNAPC5','SNRPC','SNURF','SNW1','SOGA3','SOHLH1','SOHLH2','SON','SORBS2','SOX1','SOX10','SOX11','SOX12','SOX13','SOX14','SOX15','SOX17','SOX18','SOX2','SOX21','SOX3','SOX30','SOX4','SOX5','SOX6','SOX7','SOX8','SOX9','SP1','SP100','SP110','SP140','SP140L','SP2','SP3','SP4','SP5','SP6','SP7','SP8','SP9','SPDEF','SPEN','SPHK1','SPI1','SPIB','SPIC','SPZ1','SRCAP','SREBF1','SREBF2','SRF','SRY','SSB','SSBP2','SSBP3','SSBP4','SSH1','SSH2','SSH3','SSRP1','SSX1','ST18','STAG1','STAT1','STAT2','STAT3','STAT4','STAT5A','STAT5B','STAT6','STK16','STK3','STK36','STRN','STRN3','SUB1','SUFU','SUMO1','SUPT16H','SUPT4H1','SUPT5H','SUPT6H','SUV39H1','SUV39H2','SUZ12','SYK','T','TAB1','TAB2','TAB3','TADA2A','TADA2B','TADA3','TAF1','TAF10','TAF11','TAF12','TAF13','TAF1A','TAF1B','TAF1C','TAF1L','TAF2','TAF3','TAF4','TAF4B','TAF5','TAF5L','TAF6','TAF6L','TAF7','TAF7L','TAF8','TAF9','TAF9B','TAL1','TAL2','TARDBP','TAX1BP1','TBL1Y','TBP','TBPL1','TBPL2','TBR1','TBX1','TBX10','TBX15','TBX18','TBX19','TBX2','TBX20','TBX21','TBX22','TBX3','TBX4','TBX5','TBX6','TCEA2','TCEAL1','TCEAL7','TCEAL8','TCF12','TCF15','TCF19','TCF20','TCF21','TCF23','TCF24','TCF25','TCF3','TCF4','TCF7','TCF7L1','TCF7L2','TCFL5','TDG','TDP1','TDP2','TEAD1','TEAD2','TEAD3','TEAD4','TEF','TERB1','TERF1','TERF2','TERF2IP','TESMIN','TET1','TET2','TET3','TFAM','TFAP2A','TFAP2B','TFAP2C','TFAP2D','TFAP2E','TFAP4','TFCP2','TFCP2L1','TFDP1','TFDP2','TFDP3','TFE3','TFEB','TFEC','TGFB1','TGFB1I1','TGIF1','TGIF2','TGIF2LX','TGIF2LY','TGS1','THAP1','THAP10','THAP11','THAP12','THAP2','THAP3','THAP4','THAP5','THAP6','THAP7','THAP8','THAP9','THRA','THRAP3','THRB','THYN1','TIA1','TICAM1','TIGD1','TIGD2','TIGD3','TIGD4','TIGD5','TIGD6','TIGD7','TIPARP','TIRAP','TLE1','TLE2','TLE3','TLE4','TLE6','TLR2','TLR3','TLR4','TLR9','TLX1','TLX2','TLX3','TMEM235','TMF1','TNF','TNFAIP3','TNFRSF11A','TNFRSF4','TNFSF11','TNFSF18','TNFSF4','TOE1','TONSL','TOP1','TOP2A','TOP2B','TOPORS','TOX','TOX2','TOX3','TOX4','TP53','TP53BP1','TP63','TP73','TPRX1','TRADD','TRAF1','TRAF2','TRAF3','TRAF4','TRAF5','TRAF6','TRAFD1','TRAK2','TRAPPC9','TRERF1','TREX1','TRIB1','TRIB3','TRIM13','TRIM14','TRIM15','TRIM21','TRIM22','TRIM23','TRIM24','TRIM25','TRIM26','TRIM27','TRIM28','TRIM29','TRIM3','TRIM31','TRIM32','TRIM33','TRIM34','TRIM37','TRIM38','TRIM40','TRIM5','TRIM52','TRIM62','TRIM8','TRIP11','TRIT1','TRMT1','TRMT1L','TRPS1','TRRAP','TSC2','TSC22D1','TSC22D2','TSC22D3','TSC22D4','TSHZ1','TSHZ2','TSHZ3','TSN','TSSK4','TTF1','TUB','TULP4','TUT1','TWIST1','TWIST2','TXK','U2AF1','U2AF1L4','UBA52','UBB','UBC','UBE2K','UBE2N','UBE2V1','UBN1','UBP1','UBR4','UBTF','UBTFL1','UBTFL6','UFL1','UHRF1','UHRF2','UNCX','UNG','UNK','UNKL','USF1','USF2','USF3','USP39','USP7','UTF1','VAV1','VAX1','VAX2','VDR','VEGFA','VENTX','VEZF1','VGLL1','VGLL2','VGLL3','VHLL','VPS72','VSX1','VSX2','WDHD1','WFS1','WIZ','WNT1','WNT10B','WNT2','WNT3A','WNT5A','WNT8B','WRN','WT1','WWOX','WWP2','WWTR1','XBP1','XCL1','XPA','YBX1','YBX2','YBX3','YEATS2','YEATS4','YOD1','YY1','YY2','ZBED1','ZBED2','ZBED3','ZBED4','ZBED5','ZBED6','ZBED6CL','ZBED8','ZBED9','ZBP1','ZBTB1','ZBTB10','ZBTB11','ZBTB12','ZBTB14','ZBTB16','ZBTB17','ZBTB18','ZBTB2','ZBTB20','ZBTB21','ZBTB22','ZBTB24','ZBTB25','ZBTB26','ZBTB3','ZBTB32','ZBTB33','ZBTB34','ZBTB37','ZBTB38','ZBTB39','ZBTB4','ZBTB40','ZBTB41','ZBTB42','ZBTB43','ZBTB44','ZBTB45','ZBTB46','ZBTB47','ZBTB48','ZBTB49','ZBTB5','ZBTB6','ZBTB7A','ZBTB7B','ZBTB7C','ZBTB8A','ZBTB8B','ZBTB9','ZC3H10','ZC3H11A','ZC3H12A','ZC3H13','ZC3H14','ZC3H15','ZC3H18','ZC3H3','ZC3H4','ZC3H6','ZC3H7A','ZC3H7B','ZC3H8','ZC3HAV1','ZCCHC11','ZCCHC6','ZDHHC11','ZDHHC19','ZEB1','ZEB2','ZFAND5','ZFAT','ZFHX2','ZFHX3','ZFHX4','ZFP1','ZFP14','ZFP2','ZFP28','ZFP3','ZFP30','ZFP36','ZFP36L1','ZFP36L2','ZFP37','ZFP41','ZFP42','ZFP57','ZFP62','ZFP64','ZFP69','ZFP69B','ZFP82','ZFP90','ZFP91','ZFP92','ZFPL1','ZFPM1','ZFPM2','ZFR','ZFR2','ZFX','ZFY','ZFYVE26','ZGLP1','ZGPAT','ZHX1','ZHX2','ZHX3','ZIC1','ZIC2','ZIC3','ZIC4','ZIC5','ZIK1','ZIM2','ZIM3','ZKSCAN1','ZKSCAN2','ZKSCAN3','ZKSCAN4','ZKSCAN5','ZKSCAN7','ZKSCAN8','ZMAT1','ZMAT2','ZMAT3','ZMAT4','ZMAT5','ZMYM1','ZMYM2','ZMYM3','ZMYM4','ZMYM5','ZMYM6','ZMYND10','ZNF10','ZNF100','ZNF101','ZNF106','ZNF107','ZNF112','ZNF114','ZNF117','ZNF12','ZNF121','ZNF124','ZNF131','ZNF132','ZNF133','ZNF134','ZNF135','ZNF136','ZNF137P','ZNF138','ZNF14','ZNF140','ZNF141','ZNF142','ZNF143','ZNF146','ZNF148','ZNF154','ZNF155','ZNF157','ZNF16','ZNF160','ZNF165','ZNF169','ZNF17','ZNF174','ZNF175','ZNF177','ZNF18','ZNF180','ZNF181','ZNF182','ZNF184','ZNF185','ZNF189','ZNF19','ZNF195','ZNF197','ZNF2','ZNF20','ZNF200','ZNF202','ZNF205','ZNF207','ZNF208','ZNF211','ZNF212','ZNF213','ZNF214','ZNF215','ZNF217','ZNF219','ZNF22','ZNF221','ZNF222','ZNF223','ZNF224','ZNF225','ZNF226','ZNF227','ZNF229','ZNF23','ZNF230','ZNF232','ZNF233','ZNF234','ZNF235','ZNF236','ZNF239','ZNF24','ZNF248','ZNF25','ZNF250','ZNF251','ZNF253','ZNF254','ZNF256','ZNF257','ZNF26','ZNF260','ZNF263','ZNF264','ZNF266','ZNF267','ZNF268','ZNF273','ZNF274','ZNF275','ZNF276','ZNF277','ZNF28','ZNF280A','ZNF280B','ZNF280C','ZNF280D','ZNF281','ZNF282','ZNF283','ZNF284','ZNF285','ZNF286A','ZNF286B','ZNF287','ZNF292','ZNF296','ZNF3','ZNF30','ZNF300','ZNF302','ZNF304','ZNF311','ZNF316','ZNF317','ZNF318','ZNF319','ZNF32','ZNF320','ZNF322','ZNF324','ZNF324B','ZNF326','ZNF329','ZNF330','ZNF331','ZNF333','ZNF334','ZNF335','ZNF337','ZNF33A','ZNF33B','ZNF34','ZNF341','ZNF343','ZNF345','ZNF346','ZNF347','ZNF35','ZNF350','ZNF354A','ZNF354B','ZNF354C','ZNF355P','ZNF358','ZNF362','ZNF365','ZNF366','ZNF367','ZNF37A','ZNF382','ZNF383','ZNF384','ZNF385A','ZNF385B','ZNF385C','ZNF385D','ZNF391','ZNF394','ZNF395','ZNF396','ZNF397','ZNF398','ZNF404','ZNF407','ZNF408','ZNF41','ZNF410','ZNF414','ZNF415','ZNF416','ZNF417','ZNF418','ZNF419','ZNF420','ZNF423','ZNF425','ZNF426','ZNF428','ZNF429','ZNF43','ZNF430','ZNF431','ZNF432','ZNF433','ZNF436','ZNF438','ZNF439','ZNF44','ZNF440','ZNF441','ZNF442','ZNF443','ZNF444','ZNF445','ZNF446','ZNF449','ZNF45','ZNF451','ZNF454','ZNF460','ZNF461','ZNF462','ZNF467','ZNF468','ZNF469','ZNF470','ZNF471','ZNF473','ZNF474','ZNF479','ZNF48','ZNF480','ZNF483','ZNF484','ZNF485','ZNF486','ZNF487','ZNF488','ZNF490','ZNF491','ZNF492','ZNF493','ZNF496','ZNF497','ZNF500','ZNF501','ZNF502','ZNF503','ZNF506','ZNF507','ZNF510','ZNF511','ZNF512','ZNF512B','ZNF513','ZNF514','ZNF516','ZNF517','ZNF518A','ZNF518B','ZNF519']
TFs3Other=['ZNF521','ZNF524','ZNF525','ZNF526','ZNF527','ZNF528','ZNF529','ZNF530','ZNF532','ZNF534','ZNF536','ZNF540','ZNF541','ZNF542P','ZNF543','ZNF544','ZNF546','ZNF547','ZNF548','ZNF549','ZNF550','ZNF551','ZNF552','ZNF554','ZNF555','ZNF556','ZNF557','ZNF558','ZNF559','ZNF560','ZNF561','ZNF562','ZNF563','ZNF564','ZNF565','ZNF566','ZNF567','ZNF568','ZNF569','ZNF57','ZNF570','ZNF571','ZNF572','ZNF573','ZNF574','ZNF575','ZNF576','ZNF577','ZNF578','ZNF579','ZNF580','ZNF581','ZNF582','ZNF583','ZNF584','ZNF585A','ZNF585B','ZNF586','ZNF587','ZNF587B','ZNF589','ZNF592','ZNF593','ZNF594','ZNF595','ZNF596','ZNF597','ZNF598','ZNF599','ZNF600','ZNF605','ZNF606','ZNF607','ZNF608','ZNF609','ZNF610','ZNF611','ZNF613','ZNF614','ZNF615','ZNF616','ZNF618','ZNF619','ZNF620','ZNF621','ZNF622','ZNF623','ZNF624','ZNF625','ZNF626','ZNF627','ZNF628','ZNF629','ZNF630','ZNF638','ZNF639','ZNF641','ZNF644','ZNF645','ZNF646','ZNF648','ZNF649','ZNF652','ZNF653','ZNF654','ZNF655','ZNF658','ZNF66','ZNF660','ZNF662','ZNF664','ZNF665','ZNF667','ZNF668','ZNF669','ZNF670','ZNF671','ZNF672','ZNF674','ZNF675','ZNF676','ZNF677','ZNF678','ZNF679','ZNF680','ZNF681','ZNF682','ZNF683','ZNF684','ZNF687','ZNF688','ZNF689','ZNF69','ZNF691','ZNF692','ZNF695','ZNF696','ZNF697','ZNF699','ZNF7','ZNF70','ZNF700','ZNF701','ZNF703','ZNF704','ZNF705A','ZNF705B','ZNF705D','ZNF705E','ZNF705G','ZNF706','ZNF707','ZNF708','ZNF709','ZNF71','ZNF710','ZNF711','ZNF713','ZNF714','ZNF716','ZNF717','ZNF718','ZNF720','ZNF721','ZNF724','ZNF726','ZNF727','ZNF728','ZNF729','ZNF73','ZNF730','ZNF732','ZNF735','ZNF736','ZNF737','ZNF738','ZNF74','ZNF740','ZNF746','ZNF747','ZNF749','ZNF750','ZNF75A','ZNF75D','ZNF76','ZNF761','ZNF763','ZNF764','ZNF765','ZNF766','ZNF768','ZNF77','ZNF770','ZNF771','ZNF772','ZNF773','ZNF774','ZNF775','ZNF776','ZNF777','ZNF778','ZNF780A','ZNF780B','ZNF781','ZNF782','ZNF783','ZNF784','ZNF785','ZNF786','ZNF787','ZNF788','ZNF789','ZNF79','ZNF790','ZNF791','ZNF792','ZNF793','ZNF799','ZNF8','ZNF80','ZNF800','ZNF804A','ZNF804B','ZNF805','ZNF808','ZNF81','ZNF812P','ZNF813','ZNF814','ZNF816','ZNF821','ZNF823','ZNF827','ZNF829','ZNF83','ZNF830','ZNF831','ZNF835','ZNF836','ZNF837','ZNF839','ZNF84','ZNF841','ZNF843','ZNF844','ZNF845','ZNF846','ZNF85','ZNF850','ZNF852','ZNF853','ZNF860','ZNF862','ZNF865','ZNF878','ZNF879','ZNF880','ZNF883','ZNF888','ZNF891','ZNF90','ZNF91','ZNF92','ZNF93','ZNF98','ZNF99','ZNFX1','ZRANB2','ZRSR1','ZRSR2','ZSCAN1','ZSCAN10','ZSCAN12','ZSCAN16','ZSCAN18','ZSCAN2','ZSCAN20','ZSCAN21','ZSCAN22','ZSCAN23','ZSCAN25','ZSCAN26','ZSCAN29','ZSCAN30','ZSCAN31','ZSCAN32','ZSCAN4','ZSCAN5A','ZSCAN5B','ZSCAN5C','ZSCAN5DP','ZSCAN9','ZUFSP','ZXDA','ZXDB','ZXDC','ZZZ3']
TFs4Other=['ATF2','ATF6B','ATN1','BHLHA15','BHLHB2','BHLHB4 ','BHLHB5','BHLHB9','C13ORF8','C20ORF194','CDX2','CRAMP1L','ELP3','ERVK3-1','ETF1','FLJ12895','FLJ20321','FLJ25476','FLJ35784','FOXM1','FOXO2','FOXR1','FOXR2','GATA1','GRLF1','KDM4C','KDM4E','KIAA0863','KIAA0961','KLRG1','LBXCOR1','LEF1','LHX7','LOC126295','LOC51058','LOC91661','MGC4728','MYOD1','MYOG','MYRF','NEUROD4','NEUROG1','NFAT2','NFAT4','NFE2L1','NR1A2','NR1B1','NR1B2','NR1B3','NR1C1','NR1C2','NR1C3','NR1F3','NR1I1','NR2A1','NR2B3','NR3A1','NR3A2','NR3B3','OTEX','PEPP-2','PHF2','POU2F1','POU2F2','POU4F3','POU5F1','POU6F1','PRDM3','PRKRIP1','PRKRIR','SETD1A','SMAD8','SMYD2','SOX12','SOX14','SOX15','TBX14','TCF1','TCF17','TCF2','TCF9','TEAD1','TPRXL','UBTFL1','WHSC1','ZBTB14','ZBTB18','ZBTB19','ZBTB21','ZBTB27','ZBTB28','ZBTB35','ZBTB49','ZFP161','ZFP28','ZKSCAN10','ZKSCAN11','ZKSCAN12','ZKSCAN13','ZKSCAN14','ZKSCAN16','ZKSCAN19','ZKSCAN21','ZKSCAN24','ZKSCAN6','ZKSCAN8','ZNF10','ZNF112','ZNF187','ZNF193','ZNF204P','ZNF238','ZNF271','ZNF295','ZNF305','ZNF323','ZNF434','ZNF499','ZNF542','ZNF545','ZNF553','ZNF56','ZNF625','ZNF643','ZNF698','ZNF705F','ZNF751','ZNF752','ZNF753','ZNF785','ZNF806','ZNF816','ZNF833P','ZNF906','ZSCAN1','ZSCAN25','ZSCAN26','ZSCAN31','ZSCAN32','ZSCAN5D','ZSCAN9']
TFsOther=TFs1Other+TFs2Other+TFs3Other+TFs4Other
```

Example 9 (unknown):
```unknown
S genes: ['MCM5', 'PCNA', 'TYMS', 'FEN1', 'MCM2', 'MCM4', 'RRM1', 'UNG', 'GINS2', 'MCM6', 'CDCA7', 'DTL', 'PRIM1', 'UHRF1', 'HELLS', 'RFC2', 'RPA2', 'NASP', 'RAD51AP1', 'GMNN', 'WDR76', 'SLBP', 'CCNE2', 'UBR7', 'POLD3', 'MSH2', 'ATAD2', 'RAD51', 'RRM2', 'CDC45', 'CDC6', 'EXO1', 'TIPIN', 'DSCC1', 'BLM', 'CASP8AP2', 'USP1', 'CLSPN', 'POLA1', 'CHAF1B', 'BRIP1', 'E2F8'] 

G2 genes: ['HMGB2', 'CDK1', 'NUSAP1', 'UBE2C', 'BIRC5', 'TPX2', 'TOP2A', 'NDC80', 'CKS2', 'NUF2', 'CKS1B', 'MKI67', 'TMPO', 'CENPF', 'TACC3', 'SMC4', 'CCNB2', 'CKAP2L', 'CKAP2', 'AURKB', 'BUB1', 'KIF11', 'ANP32E', 'TUBB4B', 'GTSE1', 'KIF20B', 'HJURP', 'CDCA3', 'CDC20', 'TTK', 'CDC25C', 'KIF2C', 'RANGAP1', 'NCAPD2', 'DLGAP5', 'CDCA2', 'CDCA8', 'ECT2', 'KIF23', 'HMMR', 'AURKA', 'PSRC1', 'ANLN', 'LBR', 'CKAP5', 'CENPE', 'CTCF', 'NEK2', 'G2E3', 'GAS2L3', 'CBX5', 'CENPA'] 

Hemoglobin genes set 1: ['HBB', 'HBD', 'HBG1', 'HBG2', 'HBE1', 'HBZ', 'HBM', 'HBA2', 'HBA1', 'HBQ1'] 

Hemoglobin genes set 2: ['AHSP', 'GYPA', 'RHAG', 'GFI1B', 'KEL', 'EPB42', 'ALAS2', 'TSPO2', 'GYPB', 'GYPE', 'SPTA1', 'RHCE', 'KLF1', 'RHD', 'SLC4A1', 'SNX22', 'SQSTM1', 'SOD1', 'FTH1', 'MAP1LC3B', 'SLC25A37', 'TRIM58', 'HMBS'] 

Mitochondrial genes: ['MT-ND1', 'MT-ND2', 'MT-CO1', 'MT-CO2', 'MT-ATP8', 'MT-ATP6', 'MT-CO3', 'MT-ND3', 'MT-ND4L', 'MT-ND4', 'MT-ND5', 'MT-ND6', 'MT-CYB'] 

Ribo genes: ['RPLP0', 'RPLP1', 'RPLP2', 'RPL3', 'RPL3L', 'RPL4', 'RPL5', 'RPL6', 'RPL7', 'RPL7A', 'RPL7L1', 'RPL8', 'RPL9', 'RPL10', 'RPL10A', 'RPL10L', 'RPL11', 'RPL12', 'RPL13', 'RPL13A', 'RPL14', 'RPL15', 'RPL17', 'RPL18', 'RPL18A', 'RPL19', 'RPL21', 'RPL22', 'RPL22L1', 'RPL23', 'RPL23A', 'RPL24', 'RPL26', 'RPL26L1', 'RPL27', 'RPL27A', 'RPL28', 'RPL29', 'RPL30', 'RPL31', 'RPL32', 'RPL34', 'RPL35', 'RPL35A', 'RPL36', 'RPL36A', 'RPL36AL', 'RPL37', 'RPL37A', 'RPL38', 'RPL39', 'RPL39L', 'UBA52', 'RPL41', 'RPSA', 'RPS2', 'RPS3', 'RPS3A', 'RPS4X', 'RPS4Y1', 'RPS4Y2', 'RPS5', 'RPS6', 'RPS7', 'RPS8', 'RPS9', 'RPS10', 'RPS11', 'RPS12', 'RPS13', 'RPS14', 'RPS15', 'RPS15A', 'RPS16', 'RPS17', 'RPS18', 'RPS19', 'RPS20', 'RPS21', 'RPS23', 'RPS24', 'RPS25', 'RPS26', 'RPS27', 'RPS27A', 'RPS27L', 'RPS28', 'RPS29', 'FAU'] 

Tissue dissociation stress associated genes: ['FOS', 'FOSB', 'JUN', 'JUNB', 'JUND', 'ZFP36', 'SOCS3', 'CEBPD', 'CEBPB', 'ATF3', 'EGR1', 'HSPA1A', 'HSPA1B', 'HSP90AA1', 'HSP90AB1', 'HSPA8', 'HSPB1', 'HSPE1', 'IER3', 'IER2', 'BTG1', 'BTG2', 'DUSP1']
```

Example 10 (python):
```python
#Expression: "YlGn" (e.g. DE plots)
#Divergent expression: "PRGn" (e.g. log2FC)
#Age: "afmhot_r"
#Others: "viridis" (e.g. UMAP expression)

#Define color palette
FineGrainPalette={
    'AtrialCardiomyocytesLeft':'#eac075',
    'AtrialCardiomyocytesRight':'#4c6b8a',
    'AtrialCardiomyocytesCycling':'#e94bc7',
    'VentricularCardiomyocytesLeftCompact':'#ab1c3c',
    'VentricularCardiomyocytesRightCompact':'#fdba45',
    'VentricularCardiomyocytesLeftTrabeculated':'#2b9200',
    'VentricularCardiomyocytesRightTrabeculated':'#0273a8',
    'VentricularCardiomyocytesCycling':'#e1acff',
    'SinoatrialNodePacemakerCells':'#854834',
    'AtrioventricularNodePacemakerCells':'#688500',
    'VentricularConductionSystemProximal':'#01ccc8',
    'VentricularConductionSystemDistal':'#b58fab',
    'GreatVesselAdventitialFibroblasts':'#ff7e97',
    'CoronaryVesselAdventitialFibroblasts':'#ff9b6e',
    'MyocardialInterstitialFibroblasts':'#8a6800',
    'SubEpicardialFibroblasts':'#206601',
    'Myofibroblasts':'#1bdebf',
    'LymphNodeFibroblasticReticularCells':'#005aa0',
    'ValveInterstitialCells':'#e478ff',
    'GreatVesselSmoothMuscleCells':'#ff665f',
    'CoronarySmoothMuscleCells':'#d2cb18',
    'DuctusArteriosusSmoothMuscleCells':'#83d7b3',
    'CoronaryPericytes':'#9088ff',
    'PericardialCellsIntermediate':'#c95e00',
    'PericardialCellsFibrous':'#01e077',
    'PericardialCellsParietal':'#ff88e4',
    'GreatVesselArterialEndothelialCells':'#d4004b',
    'GreatVesselVenousEndothelialCells':'#8f4300',
    'CoronaryArterialEndothelialCells':'#89da5a',
    'CoronaryVenousEndothelialCells':'#006f86',
    'CoronaryCapillaryEndothelialCells':'#aa7dff',
    'EndocardialCells':'#ff8241',
    'EndocardialCushionCells':'#46603a',
    'ValveEndothelialCells':'#414eb6',
    'LymphaticEndothelialCells':'#8b3f60',
    'MesothelialEpicardialCells':'#0295fe',
    'EpicardiumDerivedCells':'#ff92c4',
    'NeuronPrecursors':'#d30038',
    'ChromaffinCells':'#726800',
    'SympatheticNeurons':'#01894b',
    'ParasympatheticNeurons':'#545191',
    'SchwannCellPrecursors':'#b0d276',
    'SchwannCells':'#b30079',
    'MonocytesMPOpos':'#c50066',
    'Monocytes':'#fab4b6',
    'MonocyteDerivedCells':'#d14008',
    'MacrophagesCX3CR1pos':'#eabe97',
    'MacrophagesTIMD4pos':'#b2a200',
    'MacrophagesLYVE1pos':'#4a6005',
    'MacrophagesATF3pos':'#3db833',
    'DendriticCellsType1':'#007859',
    'DendriticCellsMature':'#01a9db',
    'PlasmacytoidDendriticCells':'#2c54ab',
    'MastCells':'#d0bef2',
    'Megakaryocytes':'#94298f',
    'TCellsCD4pos':'#ff387c',
    'TCellsCD8pos':'#9e3429',
    'TregsCD4pos':'#b37700',
    'ProBCells':'#8dab00',
    'BCells':'#a8d1a1',
    'BCellsMS4A1pos':'#00d2f0',
    'NaturalKillerCells':'#abb4ff',
    'InnateLymphoidCells':'#72497f',
}
sc.pl.umap(adata, color='fine_grain',palette=FineGrainPalette)

MidGrainPalette={
    'AtrialCardiomyocytes':'#af3a4d',
    'VentricularCardiomyocytes':'#b1756c',
    'CardiacConductionSystem':'#d45633',
    'Fibroblasts':'#9b6e4e',
    'MuralCells':'#ffb92f',
    'PericardialCells':'#b4f03b',
    'BloodVesselEndothelialCells':'#72cc57',
    'EndocardialCells':'#436e43',
    'LymphaticEndothelialCells':'#85cca0',
    'EpicardialCells':'#5eabb0',
    'Neurons':'#7bbdff',
    'Glia':'#403c90',
    'MyeloidCells':'#420066',
    'LymphoidCells':'#f553f6',
}
sc.pl.umap(adata, color='mid_grain',palette=MidGrainPalette)

CoarseGrainPalette={
    'Cardiomyocytes':'#bc5950',
    'Mesenchymal':'#af9c4c',
    'Endothelium':'#599e58',
    'Epicardium':'#5eabb0',
    'Neural':'#5e75b1',
    'Leukocytes':'#9a52a0',
}
sc.pl.umap(adata, color='coarse_grain',palette=CoarseGrainPalette)

RegionGrainPalette={
    'aorta':'#ffa2bc',
    'apex':'#fe0049',
    'atria':'#732d00',
    'atrial septum':'#ffd54c',
    'base':'#837100',
    'ductus arteriosus':'#bdff8b',
    'whole sample':'#1b6c00',
    'heart without node':'#02e6c6',
    'left ventricle':'#01c1f5',
    'node':'#00466d',
    'outflow tract':'#020011',
    'pericardium':'#7a75ff',
    'pulmonary arches and branches':'#30003e',
    'right ventricle':'#ff88e1',
    'right ventricle and papillaries':'#98004f',
}
sc.pl.umap(adata, color='region',palette=RegionGrainPalette)
```

Example 11 (python):
```python
set(adata.obs.fine_grain)
```

Example 12 (unknown):
```unknown
{'AtrialCardiomyocytesCycling',
 'AtrialCardiomyocytesLeft',
 'AtrialCardiomyocytesRight',
 'AtrioventricularNodePacemakerCells',
 'BCells',
 'BCellsMS4A1pos',
 'ChromaffinCells',
 'CoronaryArterialEndothelialCells',
 'CoronaryCapillaryEndothelialCells',
 'CoronaryPericytes',
 'CoronarySmoothMuscleCells',
 'CoronaryVenousEndothelialCells',
 'CoronaryVesselAdventitialFibroblasts',
 'DendriticCellsMature',
 'DendriticCellsType1',
 'DuctusArteriosusSmoothMuscleCells',
 'EndocardialCells',
 'EndocardialCushionCells',
 'EpicardiumDerivedCells',
 'GreatVesselAdventitialFibroblasts',
 'GreatVesselArterialEndothelialCells',
 'GreatVesselSmoothMuscleCells',
 'GreatVesselVenousEndothelialCells',
 'InnateLymphoidCells',
 'LymphNodeFibroblasticReticularCells',
 'LymphaticEndothelialCells',
 'MacrophagesATF3pos',
 'MacrophagesCX3CR1pos',
 'MacrophagesLYVE1pos',
 'MacrophagesTIMD4pos',
 'MastCells',
 'Megakaryocytes',
 'MesothelialEpicardialCells',
 'MonocyteDerivedCells',
 'Monocytes',
 'MonocytesMPOpos',
 'MyocardialInterstitialFibroblasts',
 'Myofibroblasts',
 'NaturalKillerCells',
 'NeuronPrecursors',
 'ParaympatheticNeurons',
 'PericardialCellsFibrous',
 'PericardialCellsIntermediate',
 'PericardialCellsParietal',
 'PlasmacytoidDendriticCells',
 'ProBCells',
 'SchwannCellPrecursors',
 'SchwannCells',
 'SinoatrialNodePacemakerCells',
 'SubEpicardialFibroblasts',
 'SympatheticNeurons',
 'TCellsCD4pos',
 'TCellsCD8pos',
 'TregsCD4pos',
 'ValveEndothelialCells',
 'ValveInterstitialCells',
 'VentricularCardiomyocytesCycling',
 'VentricularCardiomyocytesLeftCompact',
 'VentricularCardiomyocytesLeftTrabeculated',
 'VentricularCardiomyocytesRightCompact',
 'VentricularCardiomyocytesRightTrabeculated',
 'VentricularConductionSystemDistal',
 'VentricularConductionSystemProximal'}
```

Example 13 (python):
```python
set(adata.obs.mid_grain)
```

Example 14 (unknown):
```unknown
{'AtrialCardiomyocytes',
 'BloodVesselEndothelialCells',
 'CardiacConductionSystem',
 'EndocardialCells',
 'EpicardialCells',
 'Fibroblasts',
 'Glia',
 'LymphaticEndothelialCells',
 'LymphoidCells',
 'MuralCells',
 'MyeloidCells',
 'Neurons',
 'PericardialCells',
 'VentricularCardiomyocytes'}
```

Example 15 (python):
```python
set(adata.obs.coarse_grain)
```

Example 16 (unknown):
```unknown
{'Cardiomyocytes',
 'Endothelium',
 'Epicardium',
 'Leukocytes',
 'Mesenchymal',
 'Neural'}
```

Example 17 (python):
```python
#adata = adata[adata.obs['fine_grain'].isin(['CoronaryVesselAdventitialFibroblasts','MyocardialInterstitialFibroblasts1','MyocardialInterstitialFibroblasts2','Myofibroblasts','GreatVesselAdventitialFibroblasts'])].copy()
#adata = adata[adata.obs['fine_grain'].isin(['CoronaryArterialEndothelialCells','CoronaryCapillaryEndothelialCells','CoronaryVenousEndothelialCells','GreatVesselArterialEndothelialCells','GreatVesselVenousEndothelialCells'])].copy()
#adata = adata[adata.obs['fine_grain'].isin(['Myofibroblasts','CoronaryVesselAdventitialFibroblasts','SubEpicardialFibroblasts','MyocardialInterstitialFibroblasts'])].copy()
#adata = adata[adata.obs['mid_grain'].isin(['PericardialCells'])].copy()
adata=adata[adata.obs.coarse_grain=='Neural',:].copy()
```

Example 18 (unknown):
```unknown
AnnData object with n_obs × n_vars = 6149 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score'
    var: 'gene_name_scRNA-0', 'gene_id'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'age_colors', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'cycling_colors', 'donor_colors', 'fine_grain_colors', 'heart_or_greatvessels_colors', 'hvg', 'kit_10x_colors', 'mid_grain_colors', 'modality_colors', 'neighbors', 'phase_colors', 'region_colors', 'sex_colors', 'trimester_colors', 'umap', 'week_colors'
    obsm: 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 19 (python):
```python
set(adata.obs.fine_grain)
```

Example 20 (unknown):
```unknown
{'CoronaryVesselAdventitialFibroblasts',
 'MyocardialInterstitialFibroblasts',
 'Myofibroblasts',
 'SubEpicardialFibroblasts'}
```

Example 21 (python):
```python
# preserve raw count
adata.layers["counts"] = adata.X.copy()

# normalise and log-transform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key='batch_key')
#sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5)

print(sum(adata.var.highly_variable))
sc.pl.highly_variable_genes(adata)

# freeze the state in `.raw`
adata.raw = adata

# subset hvgs
adata = adata[:, adata.var.highly_variable]

adata=adata.copy()

sc.tl.score_genes_cell_cycle(adata, s_genes=s_genes, g2m_genes=g2m_genes)
#sc.pp.regress_out(adata, ['S_score','G2M_score'])

#batch correction
scvi.model.SCVI.setup_anndata(adata, 
                              layer="counts", 
                              batch_key="batch_key",
                              categorical_covariate_keys=['phase'],
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo'])

vae = scvi.model.SCVI(adata, n_hidden = 128, n_latent = 50, n_layers = 3, dispersion = 'gene-batch')
vae.train(max_epochs=50)
    
# Convergence plot
x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
plt.plot(x, vae.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
plt.clf()

# add the latent space to adata
adata.obsm["X_scVI"] = vae.get_latent_representation()

sc.tl.score_genes(adata, stress_genes, score_name='stress_score')
sc.tl.score_genes(adata, hb_genes1, score_name='hb1_score')

sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata,min_dist=0.1)
#sc.tl.tsne(adata)
sc.pl.umap(adata, color=['log1p_total_counts','n_genes','n_genes_by_counts','total_counts','pct_counts_mt','pct_counts_ribo'],ncols=5)
sc.pl.umap(adata, color=['scrublet_score','stress_score', 'hb1_score','phase'], ncols=5,)

sc.pl.umap(adata, color=['donor','age','week','trimester','region','phase','cycling','cell_or_nuclei','modality','kit_10x','FACSgate','heart_or_greatvessels','coarse_grain','mid_grain','fine_grain'],wspace=0.8,ncols=5)
Genes=['TTN','TNNT2','MYOM1','LDB3','ALDH1A2','TNNT1','KRT19','UPK3B','BNC1','COL6A3','FBLN1','PCOLCE','LRRC17','PCDH18','EMCN','ECSCR','PECAM1','CDH5','ESAM', 'LAPTM5','PTPRC','LCP1','CD53','WAS','PHOX2B','NRXN1','STMN2','GRIK2','L1CAM']
sc.pl.umap(adata, color=Genes)
```

Example 22 (python):
```python
normalizing counts per cell
    finished (0:00:00)
extracting highly variable genes
    finished (0:00:08)
--> added
    'highly_variable', boolean vector (adata.var)
    'means', float vector (adata.var)
    'dispersions', float vector (adata.var)
    'dispersions_norm', float vector (adata.var)
1543
```

Example 23 (python):
```python
calculating cell cycle phase
computing score 'S_score'
WARNING: genes are not in var_names and ignored: ['PCNA', 'FEN1', 'MCM4', 'RRM1', 'UNG', 'MCM6', 'CDCA7', 'PRIM1', 'RFC2', 'RPA2', 'NASP', 'RAD51AP1', 'GMNN', 'SLBP', 'CCNE2', 'UBR7', 'POLD3', 'MSH2', 'RAD51', 'CDC6', 'EXO1', 'TIPIN', 'DSCC1', 'BLM', 'CASP8AP2', 'USP1', 'POLA1', 'CHAF1B', 'E2F8']
    finished: added
    'S_score', score of gene set (adata.obs).
    369 total control genes are used. (0:00:00)
computing score 'G2M_score'
WARNING: genes are not in var_names and ignored: ['TMPO', 'ANP32E', 'CDC25C', 'RANGAP1', 'NCAPD2', 'LBR', 'CKAP5', 'CTCF', 'G2E3', 'CBX5']
    finished: added
    'G2M_score', score of gene set (adata.obs).
    811 total control genes are used. (0:00:00)
-->     'phase', cell cycle phase (adata.obs)
```

Example 24 (unknown):
```unknown
GPU available: False, used: False
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
```

Example 25 (unknown):
```unknown
Epoch 50/50: 100%|███████████| 50/50 [00:27<00:00,  1.82it/s, loss=779, v_num=1]
```

Example 26 (python):
```python
computing score 'stress_score'
WARNING: genes are not in var_names and ignored: ['FOSB', 'JUND', 'SOCS3', 'CEBPB', 'ATF3', 'HSP90AA1', 'HSP90AB1', 'HSPA8', 'HSPB1', 'HSPE1', 'IER3', 'BTG1', 'DUSP1']
    finished: added
    'stress_score', score of gene set (adata.obs).
    394 total control genes are used. (0:00:00)
computing score 'hb1_score'
WARNING: genes are not in var_names and ignored: ['HBD', 'HBM', 'HBQ1']
    finished: added
    'hb1_score', score of gene set (adata.obs).
    344 total control genes are used. (0:00:00)
computing neighbors
    finished: added to `.uns['neighbors']`
    `.obsp['distances']`, distances for each pair of neighbors
    `.obsp['connectivities']`, weighted adjacency matrix (0:00:01)
computing UMAP
    finished: added
    'X_umap', UMAP coordinates (adata.obsm) (0:00:04)
```

Example 27 (unknown):
```unknown
... storing 'phase' as categorical
```

Example 28 (unknown):
```unknown
<Figure size 320x320 with 0 Axes>
```

Example 29 (python):
```python
for k in list(set(adata.obs.region)):
    sc.pl.umap(adata, color='region',groups=k,palette=['red','blue'])
```

Example 30 (unknown):
```unknown
WARNING: Length of palette colors is smaller than the number of categories (palette length: 2, categories length: 15. Some categories will have the same color.
```

Example 31 (unknown):
```unknown
WARNING: Length of palette colors is smaller than the number of categories (palette length: 2, categories length: 15. Some categories will have the same color.
```

Example 32 (unknown):
```unknown
WARNING: Length of palette colors is smaller than the number of categories (palette length: 2, categories length: 15. Some categories will have the same color.
```

Example 33 (unknown):
```unknown
WARNING: Length of palette colors is smaller than the number of categories (palette length: 2, categories length: 15. Some categories will have the same color.
```

Example 34 (unknown):
```unknown
WARNING: Length of palette colors is smaller than the number of categories (palette length: 2, categories length: 15. Some categories will have the same color.
```

Example 35 (unknown):
```unknown
WARNING: Length of palette colors is smaller than the number of categories (palette length: 2, categories length: 15. Some categories will have the same color.
```

Example 36 (unknown):
```unknown
WARNING: Length of palette colors is smaller than the number of categories (palette length: 2, categories length: 15. Some categories will have the same color.
```

Example 37 (unknown):
```unknown
WARNING: Length of palette colors is smaller than the number of categories (palette length: 2, categories length: 15. Some categories will have the same color.
```

Example 38 (unknown):
```unknown
WARNING: Length of palette colors is smaller than the number of categories (palette length: 2, categories length: 15. Some categories will have the same color.
```

Example 39 (unknown):
```unknown
WARNING: Length of palette colors is smaller than the number of categories (palette length: 2, categories length: 15. Some categories will have the same color.
```

Example 40 (unknown):
```unknown
WARNING: Length of palette colors is smaller than the number of categories (palette length: 2, categories length: 15. Some categories will have the same color.
```

Example 41 (unknown):
```unknown
WARNING: Length of palette colors is smaller than the number of categories (palette length: 2, categories length: 15. Some categories will have the same color.
```

Example 42 (unknown):
```unknown
WARNING: Length of palette colors is smaller than the number of categories (palette length: 2, categories length: 15. Some categories will have the same color.
```

Example 43 (unknown):
```unknown
WARNING: Length of palette colors is smaller than the number of categories (palette length: 2, categories length: 15. Some categories will have the same color.
```

Example 44 (unknown):
```unknown
WARNING: Length of palette colors is smaller than the number of categories (palette length: 2, categories length: 15. Some categories will have the same color.
```

Example 45 (python):
```python
sc.pl.umap(adata, color='fine_grain')
sc.pl.umap(adata, color='mid_grain')
```

Example 46 (python):
```python
set(adata.obs.fine_grain)
```

Example 47 (unknown):
```unknown
{'ChromaffinCells',
 'NeuronPrecursors',
 'ParaympatheticNeurons',
 'SchwannCellPrecursors',
 'SchwannCells',
 'SympatheticNeurons'}
```

Example 48 (python):
```python
ClusterOfInterest='SchwannCells'
sc.tl.rank_genes_groups(adata, 'fine_grain', method='t-test', key_added = "t-test")
DEGenesForCluster = [x[ClusterOfInterest] for x in adata.uns['t-test']['names'][:50]]
CategoriesOrder=['NeuronPrecursors','SympatheticNeurons','ParaympatheticNeurons','ChromaffinCells','SchwannCellPrecursors','SchwannCells']
sc.pl.dotplot(adata, DEGenesForCluster ,groupby='fine_grain',categories_order=CategoriesOrder,standard_scale='var',cmap='PRGn')
#sc.pl.umap(adata, color=DEGenesForCluster)
```

Example 49 (python):
```python
ranking genes
    finished: added to `.uns['t-test']`
    'names', sorted np.recarray to be indexed by group ids
    'scores', sorted np.recarray to be indexed by group ids
    'logfoldchanges', sorted np.recarray to be indexed by group ids
    'pvals', sorted np.recarray to be indexed by group ids
    'pvals_adj', sorted np.recarray to be indexed by group ids (0:00:00)
```

Example 50 (python):
```python
Genes=['SYT1','PCSK1N','CD24','EML5','PCLO','PHOX2A','ASCL1','NELL2','BNC2','DLL3','FNDC5','STMN2','GATA3','ISL1','RGS5','DBH','VSTM2L','RTN1','ELAVL2','MAPT','PRPH','MTUS1','CADPS','NEFL','PLPP4','CELF4','SST','PRUNE2','SV2C','PGM2L1','TMEM59L','SLC5A7','TENM2','DLK1','EPAS1','KCNJ6','CCSER1','INSM1','VEGFA','NDUFA4L2','ADM','SLC18A2','FZD5','PTPRZ1','CDH19','ERBB3','MEF2C','CDH6','SOX5','LAMA4','COL5A2','MPZ','PLS3','SLITRK6','IL1RAPL2','GFRA1','PTN','PRKG1','COL20A1','IL1RAP','LAMB1','COL4A1','OLFML2A','NR2F2','GASK1B','EGFL8','HSPG2','COL14A1','PLEKHA4','ITGA6','SCN7A','LAMA2','ARHGAP15','NRXN3','SEMA3B','TGFBR3']
CategoriesOrder=['NeuronPrecursors','SympatheticNeurons','ParaympatheticNeurons','ChromaffinCells','SchwannCellPrecursors','SchwannCells']
sc.pl.dotplot(adata, Genes ,groupby='fine_grain',categories_order=CategoriesOrder,standard_scale='var',var_group_positions=[(0,5),(6,10),(11,16),(17,32),(33,42),(43,51),(52,59),(60,75)], var_group_labels=['neurons','neuron precursors','sympathetic neurons','parasympathetic neurons','chromaffin cells','glia','Schwann cell precursors','Schwann cells'], var_group_rotation=45,cmap='YlGn')
```

Example 51 (python):
```python
Genes=['SYT1','PCSK1N','CD24','EML5','PCLO','PHOX2A','ASCL1','NELL2','BNC2','DLL3','FNDC5','STMN2','GATA3','ISL1','RGS5','DBH','VSTM2L','RTN1','ELAVL2','MAPT','PRPH','MTUS1','CADPS','NEFL','PLPP4','CELF4','SST','PRUNE2','SV2C','PGM2L1','TMEM59L','SLC5A7','TENM2','DLK1','EPAS1','KCNJ6','CCSER1','INSM1','VEGFA','NDUFA4L2','ADM','SLC18A2','FZD5']
CategoriesOrder=['NeuronPrecursors','SympatheticNeurons','ParaympatheticNeurons','ChromaffinCells','SchwannCellPrecursors','SchwannCells']
sc.pl.dotplot(adata, Genes ,groupby='fine_grain',categories_order=CategoriesOrder,standard_scale='var',var_group_positions=[(0,5),(6,10),(11,16),(17,32),(33,42)], var_group_labels=['neurons','neuron precursors','sympathetic neurons','parasympathetic neurons','chromaffin cells'], var_group_rotation=45,cmap='YlGn')
```

Example 52 (python):
```python
Genes=['PTPRZ1','CDH19','ERBB3','MEF2C','CDH6','SOX5','LAMA4','COL5A2','MPZ','PLS3','SLITRK6','IL1RAPL2','GFRA1','PTN','PRKG1','COL20A1','IL1RAP','LAMB1','COL4A1','OLFML2A','NR2F2','GASK1B','EGFL8','HSPG2','COL14A1','PLEKHA4','ITGA6','SCN7A','LAMA2','ARHGAP15','NRXN3','SEMA3B','TGFBR3']
CategoriesOrder=['NeuronPrecursors','SympatheticNeurons','ParaympatheticNeurons','ChromaffinCells','SchwannCellPrecursors','SchwannCells']
sc.pl.dotplot(adata, Genes ,groupby='fine_grain',categories_order=CategoriesOrder,standard_scale='var',var_group_positions=[(0,8),(9,16),(17,32)], var_group_labels=['glia','Schwann cell precursors','Schwann cells'], var_group_rotation=45,cmap='YlGn')
```

Example 53 (python):
```python
#adata.write('./ObjectsToUpload/Neural.h5ad')
```

Example 54 (python):
```python
adata=sc.read('./ObjectsToUpload/Neural.h5ad')
```

---

## Read in¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/Xenium/no2_integration_and_cell-type-annotation/no03_harmony.html

**Contents:**
- Read in¶
- Log-normalise¶
- PCA¶
- Run Harmony¶
- Save¶
- kNN graph, UMAP, Clustering¶
- Save¶

**Examples:**

Example 1 (python):
```python
import scanpy as sc
import numpy as np
import pandas as pd
import os
# import rapids_singlecell as rsc
from tqdm import tqdm

import matplotlib.pyplot as plt
import seaborn as sns
```

Example 2 (python):
```python
# import session_info
# session_info.show()
```

Example 3 (python):
```python
sc.settings.set_figure_params(dpi=120)
```

Example 4 (python):
```python
objects_dir = '/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium'
path_adata = f'{objects_dir}/C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat_5K_filtered_raw.h5ad'
adata = sc.read_h5ad(path_adata)
print(adata.X.data[:5])
adata
```

Example 5 (unknown):
```unknown
[5. 1. 1. 2. 2.]
```

Example 6 (unknown):
```unknown
AnnData object with n_obs × n_vars = 362277 × 5001
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1'
    obsm: 'spatial'
```

Example 7 (python):
```python
adata_lognorm = adata.copy()
```

Example 8 (python):
```python
# filter genes
sc.pp.filter_genes(adata_lognorm, min_cells=3)
```

Example 9 (python):
```python
# log-normalise the count
sc.pp.normalize_total(adata_lognorm,target_sum=1e4)
sc.pp.log1p(adata_lognorm)
adata_lognorm.X.data[:5]
```

Example 10 (unknown):
```unknown
array([4.2093534, 2.6576407, 2.6576407, 3.3151019, 3.3151019],
      dtype=float32)
```

Example 11 (python):
```python
# Create simple reduced dimensional space using PCA
sc.tl.pca(adata_lognorm)
adata_lognorm
```

Example 12 (unknown):
```unknown
AnnData object with n_obs × n_vars = 362277 × 5001
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1', 'n_cells'
    uns: 'log1p', 'pca'
    obsm: 'spatial', 'X_pca'
    varm: 'PCs'
```

Example 13 (python):
```python
import scanpy.external as sce
```

Example 14 (python):
```python
%%time
sce.pp.harmony_integrate(adata_lognorm, 'tissue_block_id', basis='X_pca', adjusted_basis='X_pca_harmony')
```

Example 15 (unknown):
```unknown
2025-05-29 11:30:02,396 - harmonypy - INFO - Computing initial centroids with sklearn.KMeans...
2025-05-29 11:30:37,959 - harmonypy - INFO - sklearn.KMeans initialization complete.
2025-05-29 11:30:39,610 - harmonypy - INFO - Iteration 1 of 10
2025-05-29 11:32:38,887 - harmonypy - INFO - Iteration 2 of 10
2025-05-29 11:34:33,817 - harmonypy - INFO - Iteration 3 of 10
2025-05-29 11:36:26,974 - harmonypy - INFO - Iteration 4 of 10
2025-05-29 11:38:20,918 - harmonypy - INFO - Iteration 5 of 10
2025-05-29 11:40:11,757 - harmonypy - INFO - Converged after 5 iterations
```

Example 16 (unknown):
```unknown
CPU times: user 33min 39s, sys: 57.2 s, total: 34min 37s
Wall time: 10min 14s
```

Example 17 (python):
```python
# add to original adata
adata.uns['pca'] = adata_lognorm.uns['pca'].copy()
adata.obsm['X_pca'] = adata_lognorm.obsm['X_pca'].copy()
adata.obsm['X_pca_harmony'] = adata_lognorm.obsm['X_pca_harmony'].copy()
adata.varm['PCs'] = adata_lognorm.varm['PCs'].copy()
adata
```

Example 18 (unknown):
```unknown
AnnData object with n_obs × n_vars = 362277 × 5001
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1'
    uns: 'pca'
    obsm: 'spatial', 'X_pca', 'X_pca_harmony'
    varm: 'PCs'
```

Example 19 (python):
```python
adata.write(path_adata)
adata
```

Example 20 (unknown):
```unknown
AnnData object with n_obs × n_vars = 362277 × 5001
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1'
    uns: 'pca'
    obsm: 'spatial', 'X_pca', 'X_pca_harmony'
    varm: 'PCs'
```

Example 21 (python):
```python
key_added = 'pca_harmony'
print("making kNN graph...")
sc.pp.neighbors(adata,use_rep=f"X_{key_added}",key_added=key_added)
print("computing UMAP coordinates...")
sc.tl.umap(adata, min_dist=0.2, neighbors_key=key_added)
adata.obsm[f"X_umap_{key_added}"]=adata.obsm["X_umap"] # rename the generic X_umap coordinates
del adata.obsm["X_umap"]
```

Example 22 (unknown):
```unknown
making kNN graph...
```

Example 23 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm
```

Example 24 (unknown):
```unknown
computing UMAP coordinates...
```

Example 25 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{key_added}",
                color=['total_counts','tissue_block_id'], wspace=0.2, cmap='RdPu')
```

Example 26 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{key_added}",
                color=['celltypist_mid_fin'], wspace=0.2, cmap='RdPu')
```

Example 27 (python):
```python
adata.write(path_adata)
adata
```

Example 28 (unknown):
```unknown
AnnData object with n_obs × n_vars = 362277 × 5001
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1'
    uns: 'pca', 'pca_harmony', 'umap', 'tissue_block_id_colors', 'celltypist_mid_fin_colors'
    obsm: 'spatial', 'X_pca', 'X_pca_harmony', 'X_umap_pca_harmony'
    varm: 'PCs'
    obsp: 'pca_harmony_distances', 'pca_harmony_connectivities'
```

Example 29 (unknown):
```unknown
/rfs/project/rfs-iCNyzSAaucw/kk837/notebooks/Foetal/Xenium/5K/C194_Hst45/CamU
```

---

## Import modules¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig4_Cardiomyocytes/VentricularCardiomyocytes/TransmuralAxis/tm_axis_gene_cell-cycle-genes.html

**Contents:**
- Import modules¶
- Functions¶
- Read in adata¶
- Add axis¶
- Differential test¶
- Slect genes to plot¶
- Pathway analysis of outer_up genes¶
- Get genes involved in cell proliferation pathway¶
- Manually curated cardiomyocyte cell cycle genes¶
- Plot genes¶

**Examples:**

Example 1 (python):
```python
import os
import numpy as np
import scanpy as sc
import anndata
import pandas as pd
import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 
import matplotlib.pyplot as plt
import seaborn as sns

import warnings
warnings.simplefilter(action='ignore', category=FutureWarning)
warnings.simplefilter("ignore", UserWarning)
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
-----
anndata             0.9.2
matplotlib          3.7.2
numpy               1.24.4
pandas              1.5.3
scanpy              1.9.3
seaborn             0.12.2
session_info        1.0.0
-----
```

Example 4 (python):
```python
PIL                         10.0.0
anyio                       NA
arrow                       1.2.3
asttokens                   NA
attr                        23.1.0
attrs                       23.1.0
babel                       2.12.1
backcall                    0.2.0
brotli                      1.0.9
certifi                     2023.11.17
cffi                        1.15.1
charset_normalizer          3.2.0
colorama                    0.4.6
comm                        0.1.4
cycler                      0.10.0
cython_runtime              NA
dateutil                    2.8.2
debugpy                     1.6.8
decorator                   5.1.1
defusedxml                  0.7.1
executing                   1.2.0
fastjsonschema              NA
fqdn                        NA
h5py                        3.9.0
idna                        3.4
igraph                      0.10.6
importlib_metadata          NA
importlib_resources         NA
ipykernel                   6.25.1
isoduration                 NA
jedi                        0.19.0
jinja2                      3.1.2
joblib                      1.3.2
json5                       NA
jsonpointer                 2.0
jsonschema                  4.19.0
jsonschema_specifications   NA
jupyter_events              0.7.0
jupyter_server              2.7.1
jupyterlab_server           2.24.0
kiwisolver                  1.4.5
leidenalg                   0.10.1
llvmlite                    0.40.1
markupsafe                  2.1.3
mpl_toolkits                NA
natsort                     8.4.0
nbformat                    5.9.2
numba                       0.57.1
overrides                   NA
packaging                   23.1
parso                       0.8.3
patsy                       0.5.3
pexpect                     4.8.0
pickleshare                 0.7.5
pkg_resources               NA
plotly                      5.16.1
prometheus_client           NA
prompt_toolkit              3.0.39
psutil                      5.9.5
ptyprocess                  0.7.0
pure_eval                   0.2.2
pydev_ipython               NA
pydevconsole                NA
pydevd                      2.9.5
pydevd_file_utils           NA
pydevd_plugins              NA
pydevd_tracing              NA
pygments                    2.16.1
pyparsing                   3.0.9
pythonjsonlogger            NA
pytz                        2023.3
referencing                 NA
requests                    2.31.0
rfc3339_validator           0.1.4
rfc3986_validator           0.1.1
rpds                        NA
ruamel                      NA
scipy                       1.10.1
send2trash                  NA
six                         1.16.0
sklearn                     1.3.0
sniffio                     1.3.0
socks                       1.7.1
sphinxcontrib               NA
stack_data                  0.6.2
statsmodels                 0.14.0
texttable                   1.6.7
threadpoolctl               3.2.0
tornado                     6.3.3
traitlets                   5.9.0
typing_extensions           NA
uri_template                NA
urllib3                     2.0.4
wcwidth                     0.2.6
webcolors                   1.13
websocket                   1.6.2
yaml                        6.0.1
zipp                        NA
zmq                         25.1.1
zstandard                   0.19.0
```

Example 5 (unknown):
```unknown
-----
IPython             8.12.0
jupyter_client      8.3.0
jupyter_core        4.12.0
jupyterlab          4.0.5
notebook            7.0.2
-----
Python 3.8.17 (default, Jul  5 2023, 21:04:15) [GCC 11.2.0]
Linux-5.4.0-171-generic-x86_64-with-glibc2.17
-----
Session information updated at 2024-03-21 10:38
```

Example 6 (python):
```python
import importlib.util
import sys

spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)

spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/tissueaxis.py")
tissueaxis = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = tissueaxis
spec.loader.exec_module(tissueaxis)
```

Example 7 (python):
```python
sc.settings.set_figure_params(dpi=80,dpi_save=300)
```

Example 8 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk'
```

Example 9 (python):
```python
# This function selects the data for multiple slides from the spatial anndata object.
# Modified `select_slide` in cell2loc for multiple slides

# @param adata: anndata object with multiple spatial experiments
# @param s: list of selected slides
# @param s_col: column in adata.obs listing experiment name for each location

def select_multislide(adata, s, s_col='sample'):

    slides = adata[adata.obs[s_col].isin(s), :]
    s_keys = list(slides.uns['spatial'].keys())
    
    uns_dict = {}
    for ss in s:
        s_spatial=np.array(s_keys)[[ss in k for k in s_keys]][0]
        uns_dict[s_spatial]=slides.uns['spatial'][s_spatial]
    
    slides.uns['spatial'] = uns_dict

    return slides
```

Example 10 (python):
```python
adata_oct_healthy = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/visium-OCT_foetal_raw.h5ad')
print(adata_oct_healthy.shape)
adata_oct_healthy_LV = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/visium-OCT-LV_foetal_raw.h5ad')
print(adata_oct_healthy_LV.shape)
```

Example 11 (unknown):
```unknown
(14261, 33538)
(6767, 33538)
```

Example 12 (python):
```python
# concatenate oct adata
adata_oct = adata_oct_healthy.concatenate(
    [adata_oct_healthy_LV],
    batch_key=None,
    index_unique=None,
    join="inner"
)
```

Example 13 (python):
```python
# log-normalise
sc.pp.normalize_total(adata_oct, target_sum=1e4)
sc.pp.log1p(adata_oct)

# sc.pp.normalize_total(adata_ffpe, target_sum=1e4)
# sc.pp.log1p(adata_ffpe)

# reset var_names
adata_oct.var.set_index('SYMBOL',inplace=True)
# adata_ffpe.var.set_index('SYMBOL',inplace=True)
```

Example 14 (unknown):
```unknown
!ls /nfs/team205/heart/visium_axis/foetal/transmural_LV
```

Example 15 (unknown):
```unknown
BHF_F_Hea13295830.csv	HCAHeartST10317186.csv	HCAHeartST13162330.csv
BHF_F_Hea13295831.csv	HCAHeartST11605165.csv	HCAHeartST13162331.csv
HCAHeartST10317184.csv	HCAHeartST11605167.csv	HCAHeartST13162336.csv
HCAHeartST10317185.csv	HCAHeartST11605168.csv
```

Example 16 (python):
```python
# add
oct_samples = list(set(adata_oct.obs['sample']))

adata_oct.obs['transmural_axis'] = np.nan

axis_files = os.listdir('/nfs/team205/heart/visium_axis/foetal/transmural_LV')
for file in axis_files:
    # read in
    axis = pd.read_csv(f'/nfs/team205/heart/visium_axis/foetal/transmural_LV/{file}',index_col=0)
    # add to anndata
    if file.split('.csv')[0] in oct_samples:
        adata_oct.obs.loc[axis.index,'transmural_axis'] = axis.copy()

# select spots with axis
adata_oct = adata_oct[adata_oct.obs['transmural_axis'].isna()==False]
```

Example 17 (python):
```python
# make bin
adata_oct.obs['transmural_axis_bin'] = pd.cut(adata_oct.obs['transmural_axis'],bins=15)
```

Example 18 (python):
```python
# annotate bin, OCT
adata_oct.obs['tm_bin_annotations'] = adata_oct.obs['transmural_axis_bin'].astype('str').copy()
adata_oct.obs.replace({'tm_bin_annotations':{
    '(-1.002, -0.867]':'outer',
    '(-0.867, -0.733]':'outer',
    '(-0.733, -0.6]':'outer',
    '(-0.6, -0.467]':'outer',
    '(-0.467, -0.333]':'outer',
    '(-0.333, -0.2]':'middle',
    '(-0.2, -0.0667]':'middle',
    '(-0.0667, 0.0667]':'middle',
    '(0.0667, 0.2]':'middle',
    '(0.2, 0.333]':'middle',
    '(0.333, 0.467]':'inner',
    '(0.467, 0.6]':'inner',
    '(0.6, 0.733]':'inner',
    '(0.733, 0.867]':'inner',
    '(0.867, 1.0]':'inner',
}}, inplace=True)

set(adata_oct.obs['tm_bin_annotations'])
```

Example 19 (unknown):
```unknown
{'inner', 'middle', 'outer'}
```

Example 20 (python):
```python
# mean value per group
group_mean_val = utils.grouped_obs_mean(adata_oct, group_key='tm_bin_annotations')
# DE test
sc.tl.rank_genes_groups(adata_oct, 'tm_bin_annotations', method='t-test')
res = sc.get.rank_genes_groups_df(adata_oct, group=None, pval_cutoff=0.05)
# add mean gene expression
df = group_mean_val.reset_index().melt(id_vars=['SYMBOL'])
df.rename(columns={'SYMBOL':'names','variable':'group','value':'mean_gene_exp'},
          inplace=True)
diff_res = res.merge(df, how='left', on=['group','names'])
diff_res.head()
```

Example 21 (python):
```python
sns.scatterplot(data=diff_res, x='logfoldchanges',y='mean_gene_exp')
```

Example 22 (unknown):
```unknown
<Axes: xlabel='logfoldchanges', ylabel='mean_gene_exp'>
```

Example 23 (python):
```python
sns.scatterplot(data=diff_res, x='scores',y='mean_gene_exp')
```

Example 24 (unknown):
```unknown
<Axes: xlabel='scores', ylabel='mean_gene_exp'>
```

Example 25 (unknown):
```unknown
display(diff_res[diff_res['names']=='NRG1'])
```

Example 26 (unknown):
```unknown
['outer', 'middle', 'inner']
```

Example 27 (python):
```python
groups = ['outer','middle','inner']
logfc_thresh = 0.1
mean_exp_thresh = 0.05

gene_lists = {}
for grp in groups:
    gene_lists[f'{grp}_up'] = np.nan
    gene_lists[f'{grp}_down'] = np.nan

for grp in groups:
    # up genes
    genes = diff_res[(diff_res['group']==grp)&\
                (diff_res['logfoldchanges'] > logfc_thresh)&\
                (diff_res['mean_gene_exp'] > mean_exp_thresh)
               ]['names']
    gene_lists[f'{grp}_up'] = list(genes)
    print(f"{grp}_up: {len(gene_lists[f'{grp}_up'])}")

    # down genes
    genes = diff_res[(diff_res['group']==grp)&\
                (diff_res['logfoldchanges'] < -logfc_thresh)
                # (res['mean_gene_exp'] > mean_exp_thresh)
               ]['names']
    gene_lists[f'{grp}_down'] = list(genes)
    print(f"{grp}_down: {len(gene_lists[f'{grp}_down'])}")
```

Example 28 (unknown):
```unknown
outer_up: 837
outer_down: 2790
middle_up: 151
middle_down: 373
inner_up: 2447
inner_down: 1187
```

Example 29 (unknown):
```unknown
gene_lists.keys()
```

Example 30 (unknown):
```unknown
dict_keys(['outer_up', 'outer_down', 'middle_up', 'middle_down', 'inner_up', 'inner_down'])
```

Example 31 (python):
```python
import gseapy as gp
pd.set_option('display.max_rows', None)

geneset_list = ['MSigDB_Hallmark_2020','KEGG_2021_Human','GO_Biological_Process_2023']

enr_res = {}
key = 'outer_up'
geneset = 'GO_Biological_Process_2023'

enr_res = gp.enrichr(gene_list=gene_lists[key],
                 gene_sets=geneset_list,
                 organism='human', # don't forget to set organism to the one you desired! e.g. Yeast
                 outdir=None, # don't write to disk
                )

enr_res_sel = enr_res.results[enr_res.results['Gene_set'] == geneset]
enr_res_sel = enr_res_sel[enr_res_sel['Adjusted P-value'] < 0.01]
enr_res_sel['n_genes'] = [int(x.split('/')[0]) for x in df['Overlap']]
display(enr_res_sel)
```

Example 32 (unknown):
```unknown
'Positive Regulation Of Leukocyte'.lower()
```

Example 33 (unknown):
```unknown
'positive regulation of leukocyte'
```

Example 34 (python):
```python
pd.set_option('max_colwidth', -1)
keys_to_grep = ['cell cycle','proliferation']
mask = [any([y.lower() in x.lower() for y in keys_to_grep]) for x in enr_res_sel['Term']]
enr_res_sel[mask][['Term','n_genes']]
```

Example 35 (python):
```python
terms_of_interest = ['Positive Regulation Of Cell Population Proliferation (GO:0008284)','Regulation Of Cell Population Proliferation (GO:0042127)']
l = list(enr_res_sel[enr_res_sel['Term'].isin(terms_of_interest)]['Genes'])
genes_of_interest = []
for g in l:
    genes_of_interest = genes_of_interest + g.split(';')
genes_of_interest = list(set(genes_of_interest))
print(len(genes_of_interest))
genes_of_interest
```

Example 36 (unknown):
```unknown
['VIPR1',
 'PHIP',
 'PPP2R5C',
 'STRN',
 'CDK6',
 'RPS4X',
 'ZNF503',
 'PIM1',
 'LIFR',
 'CDCA7L',
 'CSF1R',
 'PDPN',
 'THBS1',
 'SQLE',
 'EGLN3',
 'PDGFD',
 'HMGB2',
 'IRF6',
 'NAMPT',
 'PDCD10',
 'PTN',
 'CTBP2',
 'JUN',
 'AIF1',
 'ERBB4',
 'CDKN1A',
 'ZNF268',
 'JAK2',
 'RAPGEF2',
 'ITGB1BP1',
 'NAP1L1',
 'PDGFRB',
 'SGK1',
 'ADGRG1',
 'PDGFC',
 'CD74',
 'ZFP36L1',
 'CD200',
 'FOLR2',
 'EMILIN2',
 'MDK',
 'FGF2',
 'CCNA2',
 'PTPRK',
 'PDGFRA',
 'VEGFC',
 'WT1',
 'KDR',
 'SH3BP4',
 'KDM1A',
 'TDGF1',
 'PDGFB',
 'RPS6',
 'HMGA2',
 'RPS15A',
 'WNT5A',
 'FGF18',
 'SMARCC1',
 'NDUFS4',
 'HSPA1B',
 'CLDN5',
 'CTSH',
 'VEGFA',
 'NR4A3',
 'SFRP1',
 'RPL17',
 'KITLG',
 'HDAC2',
 'PODN',
 'HILPDA',
 'KDM4C',
 'NCK2',
 'SDCBP',
 'ALDH1A2',
 'RGCC',
 'NPM1']
```

Example 37 (python):
```python
cc_genes_df = pd.read_csv('./Gene_markers_CellCycle.csv')
cc_genes = list(cc_genes_df[cc_genes_df['Reference related to CM'].isna()==False]['Gene'])
cc_genes
```

Example 38 (unknown):
```unknown
['CDKN1A',
 'CDKN2A',
 'CDKN2B',
 'CDKN1B',
 'CDKN1C',
 'CDK1',
 'CDK2',
 'CDK4',
 'CDK6',
 'CDK7',
 'CCNA2',
 'CCNB1',
 'CCND1',
 'CCND2',
 'CCNE1',
 'CCNH',
 'YAP1',
 'TBX20']
```

Example 39 (python):
```python
# cc_genes which are statistically enriched
# up in outer side
# or down in inner side

groups = ['outer','middle','inner']
logfc_thresh = 0
mean_exp_thresh = 0

gene_lists = {}
for grp in groups:
    gene_lists[f'{grp}_up'] = np.nan
    gene_lists[f'{grp}_down'] = np.nan

for grp in groups:
    # up genes
    genes = diff_res[(diff_res['group']==grp)&\
                (diff_res['logfoldchanges'] > logfc_thresh)&\
                (diff_res['mean_gene_exp'] > mean_exp_thresh)
               ]['names']
    gene_lists[f'{grp}_up'] = list(genes)
    print(f"{grp}_up: {len(gene_lists[f'{grp}_up'])}")

    # down genes
    genes = diff_res[(diff_res['group']==grp)&\
                (diff_res['logfoldchanges'] < -logfc_thresh)
                # (res['mean_gene_exp'] > mean_exp_thresh)
               ]['names']
    gene_lists[f'{grp}_down'] = list(genes)
    print(f"{grp}_down: {len(gene_lists[f'{grp}_down'])}")

set(cc_genes).intersection(gene_lists['outer_up']+gene_lists['inner_down'])
```

Example 40 (unknown):
```unknown
outer_up: 1058
outer_down: 2798
middle_up: 157
middle_down: 376
inner_up: 2757
inner_down: 1207
```

Example 41 (unknown):
```unknown
{'CCNA2', 'CDK1', 'CDK2', 'CDK6', 'CDKN1A', 'CDKN1C'}
```

Example 42 (unknown):
```unknown
diff_res[diff_res['names']=='CDK2']
```

Example 43 (python):
```python
tissueaxis.plot_axis(adata_oct,
              axis_col='transmural_axis',
              n_bin=15,
              feature_type='gene', # 'gene' or 'celltype'
              features_to_plot=cc_genes,
              features_to_remove=None,
              threshold_express_prop=None, # at quantile=0.9
              threshold_cell_abundance=None, # at quantile=0.9
              correlation=False,
              min_max_scale=True,
              row_cluster=False,
              figsize=(4,5),
              cbar_pos=(-0.2, 0.5, 0.05, 0.18),
              cmap='viridis',
          xticklabels=False,
          yticklabels=True,
              title=None,
              return_feature = False,
                     save=f'{figdir}/tm-axis_OCT_genes_CM-cell-cycle.pdf',
    save_dpi=300,
             )
```

---

## Read in trajectory object¶

**URL:** http://127.0.0.1:9020/notebooks_html/Supplementary_materials/Macrophages/Mo-MP_no5_milestone_pseudotime_plot.html

**Contents:**
- Read in trajectory object¶

**Examples:**

Example 1 (python):
```python
import scanpy as sc
import pandas as pd
import numpy as np
import anndata
import scipy
import os

import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42
```

Example 2 (python):
```python
import scFates as scf
```

Example 3 (python):
```python
import session_info
session_info.show()
```

Example 4 (unknown):
```unknown
-----
anndata             0.9.2
numpy               1.24.4
pandas              1.5.3
scFates             1.0.6
scanpy              1.9.3
scipy               1.10.1
session_info        1.0.0
-----
```

Example 5 (python):
```python
PIL                         10.0.0
adjustText                  0.8
anyio                       NA
arrow                       1.2.3
asttokens                   NA
attr                        23.1.0
attrs                       23.1.0
babel                       2.12.1
backcall                    0.2.0
backports                   NA
brotli                      1.0.9
certifi                     2023.11.17
cffi                        1.15.1
charset_normalizer          3.2.0
colorama                    0.4.6
comm                        0.1.4
cycler                      0.10.0
cython_runtime              NA
dateutil                    2.8.2
debugpy                     1.6.8
decorator                   5.1.1
defusedxml                  0.7.1
executing                   1.2.0
fastjsonschema              NA
fqdn                        NA
h5py                        3.9.0
idna                        3.4
igraph                      0.10.6
importlib_metadata          NA
importlib_resources         NA
ipykernel                   6.25.1
isoduration                 NA
jedi                        0.19.0
jinja2                      3.1.2
joblib                      1.3.2
json5                       NA
jsonpointer                 2.0
jsonschema                  4.19.0
jsonschema_specifications   NA
jupyter_events              0.7.0
jupyter_server              2.7.1
jupyterlab_server           2.24.0
kiwisolver                  1.4.5
leidenalg                   0.10.1
llvmlite                    0.40.1
markupsafe                  2.1.3
matplotlib                  3.7.2
mpl_toolkits                NA
natsort                     8.4.0
nbformat                    5.9.2
networkx                    2.8.8
numba                       0.57.1
overrides                   NA
packaging                   23.1
parso                       0.8.3
patsy                       0.5.3
pexpect                     4.8.0
pickleshare                 0.7.5
pkg_resources               NA
plotly                      5.16.1
prometheus_client           NA
prompt_toolkit              3.0.39
psutil                      5.9.5
ptyprocess                  0.7.0
pure_eval                   0.2.2
pydev_ipython               NA
pydevconsole                NA
pydevd                      2.9.5
pydevd_file_utils           NA
pydevd_plugins              NA
pydevd_tracing              NA
pygments                    2.16.1
pyparsing                   3.0.9
pythonjsonlogger            NA
pytz                        2023.3
referencing                 NA
requests                    2.31.0
rfc3339_validator           0.1.4
rfc3986_validator           0.1.1
rpds                        NA
rpy2                        3.5.14
ruamel                      NA
seaborn                     0.12.2
send2trash                  NA
simpleppt                   1.1.4
six                         1.16.0
sklearn                     1.3.0
skmisc                      0.2.0
sniffio                     1.3.0
socks                       1.7.1
sphinxcontrib               NA
stack_data                  0.6.2
statsmodels                 0.14.0
texttable                   1.6.7
threadpoolctl               3.2.0
tornado                     6.3.3
tqdm                        4.66.1
traitlets                   5.9.0
typing_extensions           NA
tzlocal                     NA
uri_template                NA
urllib3                     2.0.4
wcwidth                     0.2.6
webcolors                   1.13
websocket                   1.6.2
yaml                        6.0.1
zipp                        NA
zmq                         25.1.1
zstandard                   0.19.0
```

Example 6 (unknown):
```unknown
-----
IPython             8.12.0
jupyter_client      8.3.0
jupyter_core        4.12.0
jupyterlab          4.0.5
notebook            7.0.2
-----
Python 3.8.17 (default, Jul  5 2023, 21:04:15) [GCC 11.2.0]
Linux-5.4.0-169-generic-x86_64-with-glibc2.17
-----
Session information updated at 2024-02-06 15:22
```

Example 7 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 8 (python):
```python
sc.settings.set_figure_params(dpi=120, dpi_save=300, transparent=True)
```

Example 9 (python):
```python
%matplotlib inline
```

Example 10 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk'
```

Example 11 (python):
```python
adata = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/Immune/MoMP_Aug2023/MultiVI-post_foetal_Mono-MP_withContCov-CCCov_13Mar2023_feature-select-0003.h5ad')

# rename obs_names
adata.obs_names = [x.replace('_expression','').replace('_paired','') for x in adata.obs_names]
adata
```

Example 12 (python):
```python
AnnData object with n_obs × n_vars = 7674 × 142283
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', '_indices', '_scvi_batch', '_scvi_labels', 'fine_grain_withcycling', 't', 'seg', 'edge', 't_sd', 'milestones', 'fine_grain_mod'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id', 'n_cells'
    uns: '_scvi_manager_uuid', '_scvi_uuid', 'adata_auc_gene', 'adata_auc_region', 'adata_gex', 'adata_gex_lognorm', 'adata_peak_tfidf', 'draw_graph', 'fine_grain_colors', 'fine_grain_mod_colors', 'fine_grain_withcycling_colors', 'graph', 'milestones_colors', 'neighbors', 'ppt', 'pseudotime_list', 'seg_colors', 'umap', 'week_colors'
    obsm: 'MultiVI_latent', 'X_R', 'X_draw_graph_fa', 'X_palantir', 'X_umap', 'X_umap13', '_scvi_extra_categorical_covs', '_scvi_extra_continuous_covs'
    obsp: 'connectivities', 'distances'
```

Example 13 (python):
```python
sc.pl.draw_graph(adata,color=["fine_grain_mod"],color_map="RdBu_r",wspace=0.7,show=False)
```

Example 14 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 15 (unknown):
```unknown
<Axes: title={'center': 'fine_grain_mod'}, xlabel='FA1', ylabel='FA2'>
```

Example 16 (python):
```python
scf.pl.trajectory(adata,arrows=True,arrow_offset=2,scale_path=0.5)
```

Example 17 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap', 'norm' will be ignored
  cax = scatter(
```

Example 18 (python):
```python
scf.pl.graph(adata,tips=True,forks=True)
```

Example 19 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap', 'norm' will be ignored
  cax = scatter(
```

Example 20 (python):
```python
sc.pl.draw_graph(adata,color=["seg","milestones"],legend_loc='on data',legend_fontsize=10)
```

Example 21 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  cax = scatter(
```

Example 22 (python):
```python
adata.obs.milestones.value_counts()
```

Example 23 (unknown):
```unknown
47    3001
5     1111
28     875
2      635
11     624
32     522
44     341
20     340
29     207
16      18
Name: milestones, dtype: int64
```

Example 24 (python):
```python
data = adata.uns['graph']['pp_seg']
data
```

Example 25 (unknown):
```unknown
# save dataframe
data.to_csv('./graph_pp_seg.csv',index=None)
```

Example 26 (python):
```python
import networkx as nx
import matplotlib.pyplot as plt
from networkx import *

# node color: age
# node size: population size

# Create a directed graph
G = nx.from_pandas_edgelist(data, 'from', 'to', create_using=nx.DiGraph())

# Creating a dictionary for edge labels
edge_labels = dict(zip(zip(data['from'], data['to']), data['n']))

# Position of the node
coord = pd.DataFrame(adata.obsm['X_draw_graph_fa'])
coord['milestones'] = list(adata.obs['milestones'])
coord = coord.groupby('milestones').mean()
coord_dict = {}
for milestone in coord.index:
    coord_dict[int(milestone)] = np.array(coord.loc[milestone])

# size of the node: cell number
size_dict_pre = adata.obs.milestones.value_counts().to_dict()
sizes = []
for key in list(G):
    sizes.append(size_dict_pre[str(key)])
sizes = np.array(sizes)

# color of the node: psendotime
adata.obs["week_continuous"] = [int(x.split('W')[0]) for x in adata.obs["week"]]
df = adata.obs[['week_continuous','milestones']]
df = df.groupby('milestones').mean()
df['week_scale'] = (df['week_continuous'] - df['week_continuous'].min())/(df['week_continuous'].max() - df['week_continuous'].min())
df.index = df.index.astype('int')
colors = np.array(df.loc[list(G)]['week_scale'])

# Draw the network
plt.figure(figsize=(5, 5))
nx.draw(G, coord_dict, with_labels=True, node_color=colors,
        node_size=sizes*1.2, arrows=True, edgecolors='black',
        cmap='Greens',
       )

# Add edge labels
nx.draw_networkx_edge_labels(G, coord_dict, edge_labels=edge_labels,
                             font_size=10, font_color='red'
                            )
plt.savefig(f"{figdir}/MoMP_milestone_segment_plots.pdf", bbox_inches="tight",dpi=300)
plt.show()
```

---

## Subcluster¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/RNA/scnRNA_Integration-Annotation/Mesenchymal/Valve.html

**Contents:**
- Subcluster¶

**Examples:**

Example 1 (unknown):
```unknown
#Added the keys for age and sex
```

Example 2 (python):
```python
# Import packages, initial settings

import numpy as np
import matplotlib.pyplot as plt
import scanpy as sc
import pandas as pd
import anndata
import scrublet as scr
import seaborn as sns
import magic
import pybiomart
import six
import sys
sys.modules['sklearn.externals.six'] = six
import scanorama
import harmonypy
import bbknn
import gseapy
from gseapy.plot import barplot, dotplot
import scvelo as scv
import scvi

plt.rcParams['pdf.fonttype'] = 42
plt.rcParams['ps.fonttype'] = 42

sc.settings.verbosity = 3  # verbosity: errors (0), warnings (1), info (2), hints (3)

from matplotlib.colors import LinearSegmentedColormap
cmap = LinearSegmentedColormap.from_list(name='gene_cmap', colors=['lightgrey', 'thistle', 'red', 'darkred']) 

sc.settings.set_figure_params(dpi=80, color_map='viridis', vector_friendly=False,  dpi_save=300, facecolor='white')
sc.logging.print_version_and_date()
#sc.logging.print_versions()
```

Example 3 (unknown):
```unknown
Running Scanpy 1.8.1, on 2023-06-21 17:45.
```

Example 4 (python):
```python
adata=sc.read('./Objects/Feb28ObjectRaw.h5ad')
print(adata.shape)
```

Example 5 (unknown):
```unknown
(297473, 36601)
```

Example 6 (python):
```python
cell_cycle_genes = ['MCM5','PCNA','TYMS','FEN1','MCM2','MCM4','RRM1','UNG','GINS2','MCM6','CDCA7','DTL','PRIM1','UHRF1','MLF1IP','HELLS','RFC2','RPA2','NASP','RAD51AP1','GMNN','WDR76','SLBP','CCNE2','UBR7','POLD3','MSH2','ATAD2','RAD51','RRM2','CDC45','CDC6','EXO1','TIPIN','DSCC1','BLM','CASP8AP2','USP1','CLSPN','POLA1','CHAF1B','BRIP1','E2F8','HMGB2','CDK1','NUSAP1','UBE2C','BIRC5','TPX2','TOP2A','NDC80','CKS2','NUF2','CKS1B','MKI67','TMPO','CENPF','TACC3','FAM64A','SMC4','CCNB2','CKAP2L','CKAP2','AURKB','BUB1','KIF11','ANP32E','TUBB4B','GTSE1','KIF20B','HJURP','CDCA3','HN1','CDC20','TTK','CDC25C','KIF2C','RANGAP1','NCAPD2','DLGAP5','CDCA2','CDCA8','ECT2','KIF23','HMMR','AURKA','PSRC1','ANLN','LBR','CKAP5','CENPE','CTCF','NEK2','G2E3','GAS2L3','CBX5','CENPA']
s_genes = cell_cycle_genes[:43]
s_genes = [x for x in s_genes if x in adata.var_names]
print('S genes:', s_genes, '\n')
g2m_genes = cell_cycle_genes[43:]
g2m_genes = [x for x in g2m_genes if x in adata.var_names]
print('G2 genes:', g2m_genes, '\n')
cell_cycle_genes = [x for x in cell_cycle_genes if x in adata.var_names]

hb_genes1 = ['HBB','HBD','HBG1','HBG2','HBE1','HBZ','HBM','HBA2','HBA1', 'HBQ1']
hb_genes1 = [x for x in hb_genes1 if x in adata.var_names]
print('Hemoglobin genes set 1:', hb_genes1, '\n')

hb_genes2 = ['AHSP','GYPA','RHAG','GFI1B','KEL','EPB42','ALAS2','TSPO2','GYPB','GYPE','SPTA1','RHCE','KLF1','RHD','SLC4A1','SNX22','SQSTM1','SOD1','FTH1','MAP1LC3B','SLC25A37','TRIM58','HMBS']
hb_genes2 = [x for x in hb_genes2 if x in adata.var_names]
print('Hemoglobin genes set 2:', hb_genes2, '\n')

mito_genes=['MT-ND1','MT-ND2','MT-CO1','MT-CO2','MT-ATP8','MT-ATP6','MT-CO3','MT-ND3','MT-ND4L', 'MT-ND4', 'MT-ND5', 'MT-ND6', 'MT-CYB']
mito_genes = [x for x in mito_genes if x in adata.var_names]
print('Mitochondrial genes:', mito_genes, '\n')

ribo_genes = ['RPLP0','RPLP1','RPLP2','RPL3','RPL3L','RPL4','RPL5','RPL6','RPL7','RPL7A','RPL7L1','RPL8','RPL9','RPL10','RPL10A','RPL10L','RPL11','RPL12','RPL13','RPL13A','RPL14','RPL15','RPL17','RPL18','RPL18A','RPL19','RPL21','RPL22','RPL22L1','RPL23','RPL23A','RPL24','RPL26','RPL26L1','RPL27','RPL27A','RPL28','RPL29','RPL30','RPL31','RPL32','RPL34','RPL35','RPL35A','RPL36','RPL36A','RPL36AL','RPL37','RPL37A','RPL38','RPL39','RPL39L','UBA52','RPL41','RPSA','RPS2','RPS3','RPS3A','RPS4X','RPS4Y1','RPS4Y2','RPS5','RPS6','RPS7','RPS8','RPS9','RPS10','RPS11','RPS12','RPS13','RPS14','RPS15','RPS15A','RPS16','RPS17','RPS18','RPS19','RPS20','RPS21','RPS23','RPS24','RPS25','RPS26','RPS27','RPS27A','RPS27L','RPS28','RPS29','FAU']
ribo_genes = [x for x in ribo_genes if x in adata.var_names]
print('Ribo genes:', ribo_genes, '\n')

stress_genes=['FOS','FOSB','JUN','JUNB','JUND','ZFP36','SOCS3','CEBPD','CEBPB','ATF3','EGR1','HSPA1A','HSPA1B','HSP90AA1','HSP90AB1','HSPA8','HSPB1','HSPE1','IER3','IER2','BTG1','BTG2','DUSP1']
stress_genes = [x for x in stress_genes if x in adata.var_names]
print('Tissue dissociation stress associated genes:', stress_genes, '\n')

FoxProteins=['FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1']
chrY_genes=['SRY', 'RPS4Y1', 'ZFY', 'ZFY-AS1', 'LINC00278', 'PCDH11Y', 'TTTY2B', 'TBL1Y', 'PRKY', 'USP9Y', 'DDX3Y', 'UTY', 'TMSB4Y', 'NLGN4Y', 'TTTY14', 'KDM5D', 'TTTY10', 'EIF1AY', 'PRORY', 'LINC00266-4P']
MembraneProteins1=['AADAT','AAK1','ABCA12','ABCA7','ABCB1','ABCB11','ABCB4','ABCB6','ABCC1','ABCC3','ABCC4','ABCC5','ABCC6','ABCG2','ABHD3','ABI1','ABLIM3','AC004691.2','AC004922.1','AC005324.3','AC005726.1','AC008403.1','AC138696.1','ACIN1','ACKR3','ACLY','ACOT13','ACOT9','ACTN1','ACTR10','ACTR6','ADA','ADAM10','ADAM12','ADAM22','ADAM28','ADAM33','ADAMTS1','ADAMTS2','ADAMTSL4','ADAP1','ADCK5','ADD1','ADD2','ADGRB2','ADGRD1','ADGRG2','ADGRG7','ADGRL2','ADH1A','ADH1B','ADH1C','ADH7','ADK','ADORA1','ADRA1B','ADRM1','ADSS','AFAP1L2','AFDN','AFG1L','AGER','AGRN','AHNAK','AHNAK2','AIFM2','AKAP10','AKAP11','AKAP12','AKR1B10','AKR1B15','AL132780.3','AL139300.1','AL355315.1','AL358113.1','AL645922.1','ALDH3A1','ALK','ALOX15B','ALOXE3','ALPI','ALPP','ALPPL2','ALS2CR12','AMACR','AMDHD1','AMER1','AMER2','AMIGO2','AMOT','AMOTL1','AMOTL2','AMPH','ANK2','ANK3','ANKEF1','ANKFN1','ANKIB1','ANKLE2','ANKRA2','ANKRD13A','ANKRD35','ANKRD42','ANKRD45','ANKS1B','ANO1','ANO10','ANO2','ANO6','ANPEP','ANXA1','ANXA13','ANXA2','ANXA3','AP2M1','APBB1','APBB1IP','APC','APLP1','APMAP','APOC3','APOD','APOLD1','AQP1','AQP10','AQP3','AQP4','AQP5','ARAP1','ARF1','ARHGAP18','ARHGAP19','ARHGAP21','ARHGAP22','ARHGAP23','ARHGAP24','ARHGAP28','ARHGAP29','ARHGAP33','ARHGAP36','ARHGAP40','ARHGAP9','ARHGDIG','ARHGEF1','ARHGEF11','ARHGEF12','ARHGEF15','ARHGEF26','ARHGEF28','ARHGEF40','ARHGEF5','ARID2','ARID4A','ARID5B','ARL13A','ARL15','ARL4C','ARMC12','ARMCX5','ARPC1A','ARPC5','ARRB2','ARRDC4','ARVCF','ASAP1','ASB5','ASGR1','ASGR2','ASH2L','ASIC1','ASIC4','ASPM','ASPSCR1','ATG13','ATG3','ATG7','ATIC','ATOX1','ATP10D','ATP12A','ATP13A3','ATP1B3','ATP23','ATP2B1','ATP2B4','ATP2C2','ATP6V0A2','ATP6V1H','ATP8A2','ATP8B1','ATXN10','ATXN3','ATXN7L3','AVEN','AVPI1','AXIN2','AXL','B2M','B3GALT2','BACE1','BAG3','BAIAP2','BAIAP2L1','BANK1','BASP1','BBOF1','BCAR1','BCAR3','BCL2L12','BCORL1','BCR','BDKRB2','BEND6','BFSP1','BFSP2','BICD2','BIN2','BLNK','BLVRB','BMPR2','BMX','BOC','BOLL','BORCS5','BORCS8-MEF2B','BRK1','BRPF1','BRSK1','BSDC1','BST2','BTK','BTN2A1','BTRC','BUB1B-PAK6','BVES','BZW2','C10orf90','C11orf42','C11orf52','C12orf73','C12orf76','C14orf37','C14orf93','C16orf74','C17orf58','C17orf62','C18orf21','C19orf33','C19orf38','C19orf48','C19orf71','C1orf116','C1orf189','C1orf35','C1orf54','C1orf68','C1orf74','C1QBP','C2orf72','C3orf14','C3orf20','C3orf38','C4orf19','C4orf22','C7orf43','C8orf33','C8orf37','C9orf153','C9orf3','CA14','CA9','CABLES2','CACNA1E','CACNG4','CADM1','CALCRL','CALD1','CALML5','CALN1','CAMK2A','CAMK2B','CAMK2D','CAMK2G','CAP2','CAPS','CASKIN2','CASP4','CAST','CAV3','CAVIN1','CAVIN2','CAVIN3','CAVIN4','CBWD1','CBWD2','CBWD3','CBWD5','CBWD6','CC2D1A','CCBE1','CCDC107','CCDC112','CCDC115','CCDC124','CCDC136','CCDC144NL','CCDC15','CCDC154','CCDC170','CCDC24','CCDC28A','CCDC59','CCDC62','CCDC66','CCDC69','CCDC8','CCDC83','CCDC85C','CCDC93','CCDC97','CCNT2','CCNYL1','CCR3','CCR4','CCT3','CD109','CD163','CD164L2','CD200R1','CD226','CD248','CD274','CD2AP','CD300A','CD300LB','CD33','CD34','CD37','CD38','CD3E','CD3G','CD4','CD44','CD46','CD47','CD48','CD79B','CD81','CD84','CD86','CD8A','CD8B','CD9','CD93','CD99L2','CDAN1','CDC42EP1','CDC42EP3','CDC42EP4','CDC42SE1','CDCA4','CDH1','CDH13','CDH15','CDH17','CDH2','CDH26','CDH3','CDH4','CDH5','CDH6','CDH8','CDIPT','CDK16','CDK5','CDK5RAP2','CDK7','CDRT15','CDV3','CEACAM5','CELSR1','CENPJ','CEP162','CEP295','CEP55','CEP57L1','CEP89','CERCAM','CFAP157','CFAP206','CFAP58','CFAP65','CFB','CFL2','CFLAR','CGN','CGNL1','CHCHD7','CHIC2','CHMP2A','CHRM3','CHRNA1','CHRNA5','CHRNB4','CHRND','CIB1','CIB3','CIP2A','CKAP5','CLCA2','CLCA4','CLCN5','CLCN6','CLDN11','CLDN18','CLDN2','CLDN23','CLDN3','CLDN4','CLDN7','CLDN9','CLEC2A','CLEC4F','CLEC5A','CLECL1','CLIC2','CLIC5','CLTB','CLUL1','CMTM4','CMTM7','CMTR2','CMYA5','CNGA1','CNKSR2','CNNM1','CNNM4','CNP','CNR1','CNTD1','COBL','COBLL1','COG3','COL16A1','COL17A1','COL4A5','COL6A2','COLEC12','COLQ','COMMD4','COMTD1','COPRS','COQ7','CORIN','CORO1B','CORO2A','CORO2B','COX6B2','CPEB3','CPPED1','CPZ','CRB3','CREB5','CRIP1','CRIP2','CRK','CRLF3','CROCC','CRTC1','CRYAB','CRYBG3','CRYL1','CSDE1','CSF1','CSF1R','CSPG4','CST6','CTNNA1','CTNNB1','CTNNBIP1','CTNND1','CTNND2','CTSF','CTSV','CTTN','CTXN1','CUEDC1','CUTA','CX3CL1','CX3CR1','CXADR','CXCR2','CYB5D2','CYBRD1','CYFIP2','CYP20A1','CYP24A1','CYTH2','CYTH4','DAAM1','DAB2','DAG1','DAGLB','DAW1','DBN1','DCAF13','DCAF16','DCBLD2','DCHS1','DCHS2','DCP2','DCT','DCTN6','DCUN1D2','DDAH1','DDB2','DDIAS','DDIT4L','DDR1','DDR2','DDX58','DDX60L','DENND6B','DFFA','DGKG','DGKH','DHDDS','DHPS','DHRS7B','DIAPH1','DIAPH3','DIO2','DIS3L','DLG1','DLG2','DLG5','DLGAP3','DLL3','DMWD','DNAAF1','DNAAF4','DNAH17','DNAJA4','DNAJB13','DNAJB4','DNAJC18','DNAJC5','DNAJC6','DNAJC9','DOC2A','DOCK4','DOK3','DONSON','DPEP1','DPH1','DPPA5','DPYSL2','DQX1','DRD1','DRP2','DSC3','DSEL','DSG2','DSP','DTNA','DUS1L','DUSP15','DXO','DYSF','ECSCR','EDA2R','EDNRA','EDNRB','EEF1AKMT1','EEF2','EFCAB1','EFCAB11','EFCAB2','EFCAB7','EFHC2','EFNB1','EFR3A','EFR3B','EGFR','EHBP1','EHD1','EHD2','EHD3','EHD4','EIF2B2','EIF5','EIF5B','ELFN1','ELMO3','ELN','ELOC','EMP3','ENAH','ENC1','ENG','ENKD1','ENO1','ENO2','ENO3','ENOX1','ENOX2','EPB41','EPB41L1','EPB41L2','EPB41L3','EPB41L4A','EPB41L4B','EPB41L5','EPCAM','EPHA2','EPHA3','EPHB1','EPHB2','EPN1','EPOR','EPS8L2','ERAP1','ERBB2','ERBB3','ERBIN','ERC1','ERCC4','ERMN','ESCO2','ETNK1','ETS2','EVA1A','EVI2B','EXOC1','EXOC3L1','EXOC7','EZR','F11R','F2RL1','F2RL3','FAAP20','FABP2','FABP3','FABP5','FAIM','FAM126A','FAM126B','FAM129A','FAM129B','FAM13A','FAM163B','FAM171A1','FAM193A','FAM205A','FAM210B','FAM222A','FAM45A','FAM47E','FAM47E-STBD1','FAM72A','FAM72B','FAM72C','FAM72D','FANCG','FAS','FASN','FBLIM1','FBLN2','FBLN5','FBLN7','FBP2','FBRSL1','FBXL2','FBXO16','FBXO38','FBXO41','FBXW11','FCAR','FCGBP','FCGR1A','FCGR1B','FCGR2A','FCHSD2','FFAR4','FGD1','FGD5','FGFR2','FGR','FHIT','FHL1','FIBCD1','FIGN','FILIP1','FILIP1L','FKBP2','FLAD1','FLCN','FLNA','FLNB','FLNC','FLOT1','FLOT2','FLRT3','FLT1','FLT4','FLVCR1','FMN2','FMNL2','FMNL3','FNBP1','FOXA2','FOXN3','FP565260.3','FRMD3','FRMD7','FRMD8','FRS2','FSCN1','FTCD','FUT2','FXYD3','FYB1','FYN','FZD1','FZD2','FZD4','FZD6','GAB1','GAB2','GABRA1','GABRA2','GABRA3','GABRA5','GABRG1','GAD1','GADL1','GAGE1','GAGE10','GAGE12C','GAGE12D','GAGE12E','GAGE12F','GAGE12G','GAGE12H','GAGE12J','GAGE13','GAGE2A','GAGE2E','GALNT10','GALR1','GAP43','GAPDH','GAPT','GAPVD1','GAS2','GAS2L1','GAS7','GAS8','GBP4','GCA','GCC1','GCLM','GCSAML-AS1','GFI1B','GFOD1','GFRA3','GHR','GID8','GINM1','GIPC1','GJA1','GJB3','GJB4','GJB6','GJB7','GJC1','GJC2','GLO1','GLP2R','GLRX','GLUL','GMIP','GNAI2','GNAQ','GNAS','GNB1','GNB2','GNB3','GNB4','GNG2','GP9','GPAT3','GPATCH1','GPBP1','GPBP1L1','GPC1','GPC3','GPC4','GPI','GPR1','GPR142','GPR156','GPR157','GPR160','GPR173','GPR20','GPR27','GPR50','GPR63','GPRC5A','GPRIN1','GPRIN2','GPX8','GRAMD1A','GRAMD1B','GRAMD1C','GRAP2','GRB7','GRID2IP','GRIK5','GRK3','GRK5','GRPR','GSDMA','GTF2F1','GTSE1','GUCA1A','GYPA','GYPC','HAPLN3','HARBI1','HAS1','HAUS4','HAUS7','HCAR2','HCAR3','HCFC2','HCK','HCLS1','HCST','HDAC8','HEG1','HELZ','HENMT1','HERC2','HERPUD1','HFE','HIF1A','HIF3A','HINT1','HIP1R','HLA-A','HLA-DPB1','HMGCS1','HMOX1','HOMER3','HOXC5','HOXD4','HPS3','HRASLS5','HRH1','HS6ST3','HSH2D','HSPB1','HSPG2','HTR1B','HTR7','HTRA1','HUNK','HYDIN','HYI','HYLS1','ICAM1','ICAM2','ICOS','ICOSLG','IFFO2','IFIT5','IFNGR1','IFRD1','IFT122','IGFL2','IGSF11','IGSF23','IKZF5','IL10RA','IL12RB1','IL12RB2','IL13','IL13RA2','IL16','IL17RB','IL1R1','IL1RL1','IL21R','IL4R','IL6ST','IL7R','ILDR2','ILK','INPP5A','INSR','INTS12','IP6K2','IPCEF1','IQGAP1','IQGAP2','IREB2','IRF2','IRS1','ISLR','ITGA2B','ITGA3','ITGA4','ITGA7','ITGA8','ITGA9','ITGB1','ITGB2','ITGB3','ITGB4','ITGB5','ITGB6','ITGB7','ITGB8','ITK','ITM2A','ITPKA','ITSN1','ITSN2','IYD','JAG1','JAK2','JAK3','JAM2','JAML','JCAD','JMJD4','JPT2','JUP','KANK1','KANK3','KANSL2','KARS','KATNB1','KCNAB1','KCNB2','KCNH8','KCNJ15','KCNK13','KCNK17','KCNK6','KCNK7','KCNK9','KCNN2','KCNS3','KCNU1','KDF1','KDR','KEL','KIAA0556','KIAA0895','KIAA1324','KIAA1522','KIAA1958','KIF17','KIF19','KIF21A','KIF21B','KIF26A','KIF26B','KISS1R','KIT','KLC1','KLC2','KLF9','KLHDC7B','KLHL41','KLHL6','KLHL7','KLK7','KLRC1','KLRC2','KLRF1','KLRK1','KMT2D','KNDC1','KNSTRN','KNTC1','KPTN','KRT75','KRT77','KRTAP25-1','L1CAM','L3MBTL1','LAIR1','LAMA4','LAMC1','LAMP2','LAMTOR1','LAPTM4A','LASP1','LAT','LAT2','LAX1','LCP1','LCTL','LDB2','LDLRAD3','LENG1','LGALS3','LGALS4','LGALS8','LGR6','LHFPL2','LHX6','LIMA1','LINGO1','LIPJ','LMCD1','LMOD2','LMTK2','LMTK3','LNX1','LNX2','LONRF1','LONRF3','LOXHD1','LPAR1','LPAR3','LPAR4','LPIN1','LPIN2','LRCH2','LRP11','LRP12','LRRC15','LRRC26','LRRC34','LRRC45','LRRC69','LRRC7','LRRC9','LRRFIP2','LRRIQ3','LSMEM2','LSR','LTB4R2','LUZP2','LY6K','LYN','LYPD1','LYST','LZTS1','LZTS2','MACF1','MADD','MAGI1','MAGI3','MAMDC2','MAP1S','MAP2','MAP2K1','MAP2K4','MAP2K7','MAP3K12','MAP3K21','MAP3K6','MAP3K7','MAP4','MAP4K1','MAP4K5','MAP6','MAPK15','MAPK8IP1','MAPT','MARCH10','MARCH7','MARCKS','MARK2','MARVELD2','MAS1L','MAT2B','MBP','MC5R','MCAM','MCEMP1','MCF2L2','MCMBP','MCTS1','MDGA1','ME1','MEF2B','MEIOC','MEPCE','MERTK','MET','METAP1','METAP2','METTL15','METTL25','METTL9','MFAP3L','MFSD1','MFSD2A','MGAT4C','MIB1','MICAL2','MICAL3','MICALL1','MICB','MIEN1','MIGA2','MISP','MKLN1','MKRN3','MLF2','MLKL','MLST8','MMACHC','MMP10','MMP15','MMP24','MMP25','MOAP1','MPHOSPH8','MPHOSPH9','MPI','MPIG6B','MPL','MPP1','MPP5','MPP7','MPZL1','MPZL2','MR1','MRGPRF','MRNIP','MRPL42','MRPL44','MRPL46','MRPS18B','MS4A1','MS4A12','MS4A14','MS4A15','MSN','MSRA','MTCP1','MTFR1L','MTHFR','MTM1','MTPN','MTSS1','MUC1','MUC20','MVD','MYH11','MYH9','MYL4','MYLK','MYO10','MYO1B','MYO1C','MYO1E','MYO1G','MYO3A','MYO5B','MYO9A','MYOF','N4BP2','NAA35','NAB1','NAIF1','NAMPT','NAP1L2','NAPG','NARFL','NCAM1','NCAM2','NCK1','NCKAP1L','NCKAP5','NCKIPSD','NCOA1','NCS1','NDC1','NECTIN2','NECTIN3','NECTIN4','NEDD1','NEDD9','NEO1','NEU1','NEXN','NF2','NFE2L2','NFKBID','NGEF']
MembraneProteins2=['NGFR','NHS','NHSL2','NID2','NINJ2','NKTR','NKX2-8','NLGN1','NLGN3','NLK','NLRX1','NMB','NMBR','NME1','NME2','NMT1','NMT2','NMUR2','NOS1','NOTCH2','NPFFR2','NPL','NPNT','NPR1','NPTN','NPTXR','NR2C1','NRAS','NRCAM','NRP2','NRSN1','NRXN1','NSD1','NSFL1C','NT5E','NTNG1','NTNG2','NTSR1','NUDCD3','NUDT16L1','NUDT9','NUMB','NUP35','NUS1','OAS3','OBSCN','OCA2','OCLN','OCM','OCM2','ODC1','ODR4','OGT','OLFM4','OLIG2','OLR1','OPHN1','OPRK1','OR10P1','OR1N2','OR2C3','OR6B2','OR6B3','OR6C6','OR8S1','OSBPL10','OSBPL6','OSBPL7','OSCAR','OSCP1','OSGEP','OSR2','OTOP2','OTULIN','OXGR1','P3H3','PACSIN2','PACSIN3','PAFAH1B2','PAG1','PAIP1','PAK1','PAK4','PAK6','PALLD','PALM','PALM2','PALM2-AKAP2','PALM3','PALMD','PANX1','PANX2','PAQR8','PARD3','PARD3B','PARD6A','PARD6G','PARP6','PATJ','PAWR','PCBD2','PCDH1','PCDH17','PCDH7','PCDHA1','PCDHA11','PCDHA12','PCDHA4','PCDHA5','PCDHB1','PCDHB11','PCDHB12','PCDHB16','PCDHB4','PCDHB5','PCDHGA1','PCDHGA10','PCDHGA11','PCDHGA12','PCDHGA2','PCDHGA3','PCDHGA4','PCDHGA5','PCDHGA6','PCDHGA7','PCDHGA8','PCDHGA9','PCDHGB1','PCDHGB2','PCDHGB3','PCDHGB4','PCDHGB5','PCDHGB6','PCDHGB7','PCDHGC3','PCDHGC4','PCDHGC5','PCLO','PCMTD1','PDAP1','PDCD7','PDE3A','PDE4A','PDE4D','PDE9A','PDGFC','PDGFRA','PDHX','PDLIM1','PDLIM5','PDPN','PDZD2','PDZD8','PEAR1','PEBP1','PFDN1','PGAP3','PGLYRP2','PGLYRP4','PGRMC2','PHACTR1','PHACTR4','PHF20L1','PHGDH','PHLDB2','PHLDB3','PHOSPHO1','PHPT1','PI3','PI4K2A','PI4KA','PIEZO2','PIGB','PIGN','PIGW','PIGX','PIK3AP1','PIK3CG','PIK3R6','PIP4K2A','PIP4K2C','PIP5K1A','PIRT','PITPNM3','PKD2L2','PKN1','PKN2','PKP1','PKP2','PKP3','PKP4','PLAA','PLAUR','PLCB1','PLCB2','PLCD3','PLCD4','PLEKHA4','PLEKHA6','PLEKHA7','PLEKHG3','PLEKHG4','PLEKHG6','PLEKHH2','PLEKHO2','PLIN4','PLPP1','PLPPR5','PLS3','PLSCR1','PLVAP','PLXNA3','PLXNA4','PNCK','PNISR','PODXL','POLE','POU2F3','PPEF1','PPFIBP1','PPIP5K1','PPL','PPM1A','PPM1F','PPP1CA','PPP1CB','PPP1R12B','PPP1R16A','PPP1R16B','PPP1R18','PPP1R27','PPP1R9A','PPP2R1B','PPP3R1','PPP3R2','PPP4C','PPP6R3','PPTC7','PRAME','PRC1','PRDX6','PRIMA1','PRKCA','PRKCE','PRKCH','PRKCZ','PRKD1','PROM2','PROSER2','PRR15','PRR32','PRR7','PRRC2A','PRRG1','PRRT2','PRRT3','PRRT4','PRTG','PRX','PSCA','PSD','PSD2','PSEN1','PSMC5','PSMC6','PSMD9','PSTPIP2','PTAFR','PTCHD4','PTK2B','PTK6','PTPN12','PTPN21','PTPN22','PTPN4','PTPRB','PTPRG','PTPRK','PTPRM','PTPRR','PTPRS','PTX3','PURG','PVALB','PVR','PXDC1','PXK','PYGL','PYGO2','PYM1','RAB13','RAB23','RAB25','RAB26','RAB27B','RAB3B','RAB3GAP2','RAB43','RAB8A','RABEPK','RAD51C','RAD52','RAI2','RALA','RALGAPA2','RAPGEF2','RAPH1','RAPSN','RASA4','RASA4B','RASAL1','RASAL2','RASAL3','RASD1','RASD2','RASGRP1','RASGRP2','RASGRP4','RASSF2','RASSF3','RASSF4','RBM46','RC3H1','RCCD1','RCSD1','RD3L','RDX','RECK','RELL1','REPS1','RET','RFLNB','RFPL1','RFPL2','RFPL3','RGP1','RGS10','RGS14','RGS19','RGS3','RGS7BP','RGS9BP','RHCE','RHD','RHEX','RHOA','RHOBTB2','RHOJ','RHOXF2','RHOXF2B','RIC8A','RIC8B','RIF1','RILPL1','RIMBP3','RIMBP3B','RIMBP3C','RIMS2','RIOK2','RIOK3','RIPK1','RND1','RNF114','RNF145','RNF146','RNF149','RNF165','RNF166','RNF39','ROBO1','ROBO3','ROBO4','ROM1','RP2','RPE','RPEL1','RPGRIP1L','RPIA','RPL36A','RPL36A-HNRNPH2','RPL36AL','RPSA','RPUSD1','RRAD','RRP12','RRP7A','RSBN1L','RSC1A1','RSPH3','RTL3','RTL4','RTN4R','RTN4RL1','RYR2','S100A13','S100A14','S100A16','S100A2','S100A3','S100A4','S100A6','S100A9','SAMD4A','SAMHD1','SAMSN1','SAPCD1','SASH3','SC5D','SCAMP1','SCAMP4','SCEL','SCHIP1','SCIN','SCN1A','SCN5A','SCN8A','SCN9A','SCNN1D','SCNN1G','SCRIB','SCUBE3','SDC2','SDC4','SDE2','SDHB','SEC16B','SEMA3C','SEMA3D','SEMA4B','SEMA4F','SEMA6C','SEMA6D','SEPT2','SEPT3','SERINC2','SERINC5','SERPINB3','SFSWAP','SGCE','SGK494','SH3BGR','SH3D21','SHANK1','SHANK3','SHARPIN','SHISA6','SHKBP1','SHROOM2','SHTN1','SIGLEC10','SIGLEC14','SIGLEC6','SIGLEC7','SIN3B','SIPA1','SIPA1L1','SIPA1L3','SKAP1','SLAMF6','SLC10A5','SLC12A2','SLC13A3','SLC13A5','SLC14A1','SLC16A1','SLC16A10','SLC16A2','SLC16A3','SLC16A7','SLC16A9','SLC17A5','SLC19A1','SLC19A3','SLC1A5','SLC1A6','SLC22A16','SLC23A3','SLC24A4','SLC26A9','SLC27A1','SLC28A1','SLC2A1','SLC2A11','SLC2A12','SLC2A14','SLC2A2','SLC2A3','SLC30A1','SLC31A2','SLC34A1','SLC34A2','SLC35F5','SLC35G2','SLC36A4','SLC38A5','SLC38A6','SLC38A7','SLC39A10','SLC39A14','SLC39A2','SLC39A6','SLC3A2','SLC40A1','SLC41A3','SLC43A1','SLC43A3','SLC44A2','SLC45A4','SLC46A1','SLC46A3','SLC4A1AP','SLC4A2','SLC4A7','SLC4A8','SLC5A12','SLC5A3','SLC5A6','SLC5A7','SLC6A6','SLC7A14','SLC7A2','SLC7A5','SLC7A6','SLC8A1','SLC9A1','SLC9A2','SLC9A3','SLC9A3R1','SLC9A3R2','SLC9A4','SLC9A5','SLC9B2','SLC9C2','SLCO1A2','SLCO1B3','SLCO4A1','SLCO4C1','SLCO5A1','SLIT1','SLIT3','SLITRK6','SLK','SLX4','SMAGP','SMAP1','SMOC1','SMPD2','SMPX','SNAP23','SNAP25','SNRK','SNTB2','SNTG2','SNX15','SNX32','SNX9','SOCS3','SOCS5','SOD1','SORBS1','SORCS1','SORT1','SOX11','SPAG16','SPANXN4','SPART','SPATA12','SPATC1L','SPERT','SPG11','SPHK1','SPINK13','SPINT1','SPINT2','SPN','SPPL2B','SPPL3','SPRED3','SPRYD3','SPTBN2','SRC','SRCIN1','SRD5A3','SRPK1','SRPRB','SRRM3','SSFA2','SSH1','SSH3','SSTR1','SSTR5','SSX2IP','ST8SIA5','STAB1','STAC','STAC2','STAMBP','STAMBPL1','STARD4','STAT2','STEAP4','STIL','STIP1','STK10','STK16','STK17A','STK17B','STOM','STOML2','STOML3','STX18','STX4','STXBP4','STYK1','SUCLG1','SUCLG2','SUGT1','SUMF2','SUPT6H','SUSD2','SVIL','SWAP70','SYCP2','SYK','SYMPK','SYNC','SYNGR3','SYNPO2L','SYPL2','SYT4','SYTL2','SYVN1','TACC2','TACR2','TACSTD2','TASP1','TAZ','TBC1D10A','TBC1D2','TBC1D24','TBC1D30','TBC1D9B','TBXA2R','TCAF2','TCAF2C','TCHP','TCIM','TDG','TDO2','TDP1','TEC','TEDC2','TEK','TES','TEX14','TEX9','TGFBR1','TGFBR2','TGM1','TGM2','THBS1','THBS2','THEGL','THY1','TIAM1','TICAM1','TIGD7','TIMD4','TJP1','TJP2','TJP3','TLN1','TLR4','TM2D2','TM4SF20','TM4SF4','TM4SF5','TMC5','TMED8','TMEM101','TMEM107','TMEM108','TMEM121B','TMEM139','TMEM141','TMEM164','TMEM176B','TMEM183A','TMEM186','TMEM2','TMEM218','TMEM222','TMEM229A','TMEM245','TMEM251','TMEM254','TMEM258','TMEM266','TMEM268','TMEM50A','TMEM63B','TMPRSS15','TMPRSS2','TMTC2','TNFRSF10D','TNFRSF11A','TNFRSF12A','TNFRSF21','TNFRSF25','TNFRSF8','TNFRSF9','TNFSF13B','TNK1','TNKS1BP1','TNNI2','TOGARAM2','TOLLIP','TOM1','TP53BP2','TPD52L1','TPRN','TRAF7','TRAIP','TRAPPC13','TRAT1','TRDN','TREML1','TRIB1','TRIM16','TRIM16L','TRIM38','TRIM4','TRIM44','TRIM54','TRIM71','TRIML2','TRIP6','TRMT1','TRMT10B','TRMT13','TRMT6','TRO','TRPM1','TRPM4','TRPM7','TRPV2','TRPV3','TSG101','TSGA13','TSHZ3','TSPAN15','TSPAN6','TSPO2','TSR2','TSSK4','TTC17','TTC23','TTC7B','TTLL12','TTLL5','TULP3','TWF2','TXNDC11','TXNIP','TYROBP','TYW3','UAP1','UBA3','UBA52','UBAC1','UBALD1','UBAP1','UBASH3B','UBE2A','UBE2B','UBE2C','UBE2D1','UBE2D2','UBE2D3','UBE2D4','UBE2K','UBE2QL1','UBE2S','UBE3C','UBQLN2','UBR2','UBTD1','UCP3','ULBP1','ULBP2','UNC5A','UNC79','UQCC1','USP14','USP15','USP2','USP20','USP21','USP4','USP53','UTP20','UTRN','VAMP5','VAMP8','VANGL1','VASP','VCAM1','VCPIP1','VIL1','VPS13B','VPS36','VPS50','VPS8','VSIG4','VTCN1','VWA5B1','VWA7','WAS','WASF2','WBP4','WDFY3','WDPCP','WDR1','WDR12','WDR13','WDR53','WDR6','WDR66','WIPF1','WWP1','XIAP','XIRP1','XIRP2','XPC','XPNPEP2','XPO6','XRN1','XYLT1','YES1','YIPF1','YIPF4','YOD1','YTHDC1','ZBED3','ZBTB33','ZBTB8OS','ZDHHC2','ZDHHC20','ZDHHC22','ZDHHC5','ZFP41','ZFP90','ZFYVE28','ZGPAT','ZMYND10','ZNF10','ZNF18','ZNF185','ZNF2','ZNF20','ZNF227','ZNF229','ZNF236','ZNF276','ZNF277','ZNF337','ZNF397','ZNF407','ZNF497','ZNF503','ZNF510','ZNF536','ZNF571','ZNF598','ZNF599','ZNF646','ZNF655','ZNF688','ZNF706','ZNF717','ZNF783','ZNF79','ZNF839','ZSCAN22','ZSCAN29','ZSWIM1','ZSWIM8','ZYX']
MembraneProteins=MembraneProteins1+MembraneProteins2

TFs1=['AC008770.3','AC023509.3','AC092835.1','AC138696.1','ADNP','ADNP2','AEBP1','AEBP2','AHCTF1','AHDC1','AHR','AHRR','AIRE','AKAP8','AKAP8L','AKNA','ALX1','ALX3','ALX4','ANHX','ANKZF1','AR','ARGFX','ARHGAP35','ARID2','ARID3A','ARID3B','ARID3C','ARID5A','ARID5B','ARNT','ARNT2','ARNTL','ARNTL2','ARX','ASCL1','ASCL2','ASCL3','ASCL4','ASCL5','ASH1L','ATF1','ATF2','ATF3','ATF4','ATF5','ATF6','ATF6B','ATF7','ATMIN','ATOH1','ATOH7','ATOH8','BACH1','BACH2','BARHL1','BARHL2','BARX1','BARX2','BATF','BATF2','BATF3','BAZ2A','BAZ2B','BBX','BCL11A','BCL11B','BCL6','BCL6B','BHLHA15','BHLHA9','BHLHE22','BHLHE23','BHLHE40','BHLHE41','BNC1','BNC2','BORCS8-MEF2B','BPTF','BRF2','BSX','C11orf95','CAMTA1','CAMTA2','CARF','CASZ1','CBX2','CC2D1A','CCDC169-SOHLH2','CCDC17','CDC5L','CDX1','CDX2','CDX4','CEBPA','CEBPB','CEBPD','CEBPE','CEBPG','CEBPZ','CENPA','CENPB','CENPBD1','CENPS','CENPT','CENPX','CGGBP1','CHAMP1','CHCHD3','CIC','CLOCK','CPEB1','CPXCR1','CREB1','CREB3','CREB3L1','CREB3L2','CREB3L3','CREB3L4','CREB5','CREBL2','CREBZF','CREM','CRX','CSRNP1','CSRNP2','CSRNP3','CTCF','CTCFL','CUX1','CUX2','CXXC1','CXXC4','CXXC5','DACH1','DACH2','DBP','DBX1','DBX2','DDIT3','DEAF1','DLX1','DLX2','DLX3','DLX4','DLX5','DLX6','DMBX1','DMRT1','DMRT2','DMRT3','DMRTA1','DMRTA2','DMRTB1','DMRTC2','DMTF1','DNMT1','DNTTIP1','DOT1L','DPF1','DPF3','DPRX','DR1','DRAP1','DRGX','DUX1','DUX3','DUX4','DUXA','DZIP1','E2F1','E2F2','E2F3','E2F4','E2F5','E2F6','E2F7','E2F8','E4F1','EBF1','EBF2','EBF3','EBF4','EEA1','EGR1','EGR2','EGR3','EGR4','EHF','ELF1','ELF2','ELF3','ELF4','ELF5','ELK1','ELK3','ELK4','EMX1','EMX2','EN1','EN2','EOMES','EPAS1','ERF','ERG','ESR1','ESR2','ESRRA','ESRRB','ESRRG','ESX1','ETS1','ETS2','ETV1','ETV2','ETV3','ETV3L','ETV4','ETV5','ETV6','ETV7','EVX1','EVX2','FAM170A','FAM200B','FBXL19','FERD3L','FEV','FEZF1','FEZF2','FIGLA','FIZ1','FLI1','FLYWCH1','FOS','FOSB','FOSL1','FOSL2','FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXN4','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1','GABPA','GATA1','GATA2','GATA3','GATA4','GATA5','GATA6','GATAD2A','GATAD2B','GBX1','GBX2','GCM1','GCM2','GFI1','GFI1B','GLI1','GLI2','GLI3','GLI4','GLIS1','GLIS2','GLIS3','GLMP','GLYR1','GMEB1','GMEB2','GPBP1','GPBP1L1','GRHL1','GRHL2','GRHL3','GSC','GSC2','GSX1','GSX2','GTF2B','GTF2I','GTF2IRD1','GTF2IRD2','GTF2IRD2B','GTF3A','GZF1','HAND1','HAND2','HBP1','HDX','HELT','HES1','HES2','HES3','HES4','HES5','HES6','HES7','HESX1','HEY1','HEY2','HEYL','HHEX','HIC1','HIC2','HIF1A','HIF3A','HINFP','HIVEP1','HIVEP2','HIVEP3','HKR1','HLF','HLX','HMBOX1','HMG20A','HMG20B','HMGA1','HMGA2','HMGN3','HMX1','HMX2','HMX3','HNF1A','HNF1B','HNF4A','HNF4G','HOMEZ','HOXA1','HOXA10','HOXA11','HOXA13','HOXA2','HOXA3','HOXA4','HOXA5','HOXA6','HOXA7','HOXA9','HOXB1','HOXB13','HOXB2','HOXB3','HOXB4','HOXB5','HOXB6','HOXB7','HOXB8','HOXB9','HOXC10','HOXC11','HOXC12','HOXC13','HOXC4','HOXC5','HOXC6','HOXC8','HOXC9','HOXD1','HOXD10','HOXD11','HOXD12','HOXD13','HOXD3','HOXD4','HOXD8','HOXD9','HSF1','HSF2','HSF4','HSF5','HSFX1','HSFX2','HSFY1','HSFY2','IKZF1','IKZF2','IKZF3','IKZF4','IKZF5','INSM1','INSM2','IRF1','IRF2','IRF3','IRF4','IRF5','IRF6','IRF7','IRF8','IRF9','IRX1','IRX2','IRX3','IRX4','IRX5','IRX6','ISL1','ISL2','ISX','JAZF1','JDP2','JRK','JRKL','JUN','JUNB','JUND','KAT7','KCMF1','KCNIP3','KDM2A','KDM2B','KDM5B','KIN','KLF1','KLF10','KLF11','KLF12','KLF13','KLF14','KLF15','KLF16','KLF17','KLF2','KLF3','KLF4','KLF5','KLF6','KLF7','KLF8','KLF9','KMT2A','KMT2B','L3MBTL1','L3MBTL3','L3MBTL4','LBX1','LBX2','LCOR','LCORL','LEF1','LEUTX','LHX1','LHX2','LHX3','LHX4','LHX5','LHX6','LHX8','LHX9','LIN28A','LIN28B','LIN54','LMX1A','LMX1B','LTF','LYL1','MAF','MAFA','MAFB','MAFF','MAFG','MAFK','MAX','MAZ','MBD1','MBD2','MBD3','MBD4','MBD6','MBNL2','MECOM','MECP2','MEF2A','MEF2B','MEF2C','MEF2D','MEIS1','MEIS2','MEIS3','MEOX1','MEOX2','MESP1','MESP2','MGA','MITF','MIXL1','MKX','MLX','MLXIP','MLXIPL','MNT','MNX1','MSANTD1','MSANTD3','MSANTD4','MSC','MSGN1','MSX1','MSX2','MTERF1','MTERF2','MTERF3','MTERF4','MTF1','MTF2','MXD1','MXD3','MXD4','MXI1','MYB','MYBL1','MYBL2','MYC','MYCL','MYCN','MYF5','MYF6','MYNN','MYOD1','MYOG','MYPOP','MYRF','MYRFL','MYSM1','MYT1','MYT1L','MZF1','NACC2','NAIF1','NANOG','NANOGNB','NANOGP8','NCOA1','NCOA2','NCOA3','NEUROD1','NEUROD2','NEUROD4','NEUROD6','NEUROG1','NEUROG2','NEUROG3','NFAT5','NFATC1','NFATC2','NFATC3','NFATC4','NFE2','NFE2L1','NFE2L2','NFE2L3','NFE4','NFIA','NFIB','NFIC','NFIL3','NFIX','NFKB1','NFKB2','NFX1','NFXL1','NFYA','NFYB','NFYC','NHLH1','NHLH2','NKRF','NKX1-1','NKX1-2','NKX2-1','NKX2-2','NKX2-3','NKX2-4','NKX2-5','NKX2-6','NKX2-8','NKX3-1','NKX3-2','NKX6-1','NKX6-2','NKX6-3','NME2','NOBOX','NOTO','NPAS1','NPAS2','NPAS3','NPAS4','NR0B1','NR1D1','NR1D2','NR1H2','NR1H3','NR1H4','NR1I2','NR1I3','NR2C1','NR2C2','NR2E1','NR2E3','NR2F1','NR2F2','NR2F6','NR3C1','NR3C2','NR4A1','NR4A2','NR4A3','NR5A1','NR5A2','NR6A1','NRF1','NRL','OLIG1','OLIG2','OLIG3','ONECUT1','ONECUT2','ONECUT3','OSR1','OSR2','OTP','OTX1','OTX2','OVOL1','OVOL2','OVOL3','PA2G4','PATZ1','PAX1','PAX2','PAX3','PAX4','PAX5','PAX6','PAX7','PAX8','PAX9','PBX1','PBX2','PBX3','PBX4','PCGF2','PCGF6','PDX1','PEG3','PGR','PHF1','PHF19 ','PHF20','PHF21A','PHOX2A','PHOX2B','PIN1','PITX1','PITX2','PITX3','PKNOX1','PKNOX2','PLAG1','PLAGL1','PLAGL2','PLSCR1','POGK','POU1F1','POU2AF1','POU2F1','POU2F2','POU2F3','POU3F1','POU3F2','POU3F3','POU3F4','POU4F1','POU4F2','POU4F3','POU5F1','POU5F1B','POU5F2','POU6F1','POU6F2','PPARA','PPARD','PPARG','PRDM1','PRDM10','PRDM12','PRDM13','PRDM14','PRDM15','PRDM16','PRDM2','PRDM4','PRDM5','PRDM6','PRDM8','PRDM9','PREB','PRMT3','PROP1','PROX1','PROX2','PRR12','PRRX1','PRRX2','PTF1A','PURA','PURB','PURG','RAG1','RARA','RARB','RARG','RAX','RAX2','RBAK','RBCK1','RBPJ','RBPJL','RBSN','REL','RELA','RELB','REPIN1','REST','REXO4','RFX1','RFX2','RFX3','RFX4','RFX5','RFX6','RFX7','RFX8','RHOXF1','RHOXF2','RHOXF2B','RLF','RORA','RORB','RORC','RREB1','RUNX1','RUNX2','RUNX3','RXRA','RXRB','RXRG','SAFB','SAFB2','SALL1','SALL2','SALL3','SALL4','SATB1','SATB2','SCMH1','SCML4','SCRT1','SCRT2','SCX','SEBOX','SETBP1','SETDB1','SETDB2','SGSM2','SHOX','SHOX2','SIM1','SIM2','SIX1','SIX2','SIX3','SIX4','SIX5','SIX6','SKI','SKIL','SKOR1','SKOR2','SLC2A4RG','SMAD1','SMAD3','SMAD4','SMAD5','SMAD9','SMYD3','SNAI1','SNAI2','SNAI3','SNAPC2','SNAPC4','SNAPC5','SOHLH1','SOHLH2','SON','SOX1','SOX10','SOX11']
TFs2=['SOX12','SOX13','SOX14','SOX15','SOX17','SOX18','SOX2','SOX21','SOX3','SOX30','SOX4','SOX5','SOX6','SOX7','SOX8','SOX9','SP1','SP100','SP110','SP140','SP140L','SP2','SP3','SP4','SP5','SP6','SP7','SP8','SP9','SPDEF','SPEN','SPI1','SPIB','SPIC','SPZ1','SRCAP','SREBF1','SREBF2','SRF','SRY','ST18','STAT1','STAT2','STAT3','STAT4','STAT5A','STAT5B','STAT6','T','TAL1','TAL2','TBP','TBPL1','TBPL2','TBR1','TBX1','TBX10','TBX15','TBX18','TBX19','TBX2','TBX20','TBX21','TBX22','TBX3','TBX4','TBX5','TBX6','TCF12','TCF15','TCF20','TCF21','TCF23','TCF24','TCF3','TCF4','TCF7','TCF7L1','TCF7L2','TCFL5','TEAD1','TEAD2','TEAD3','TEAD4','TEF','TERB1','TERF1','TERF2','TET1','TET2','TET3','TFAP2A','TFAP2B','TFAP2C','TFAP2D','TFAP2E','TFAP4','TFCP2','TFCP2L1','TFDP1','TFDP2','TFDP3','TFE3','TFEB','TFEC','TGIF1','TGIF2','TGIF2LX','TGIF2LY','THAP1','THAP10','THAP11','THAP12','THAP2','THAP3','THAP4','THAP5','THAP6','THAP7','THAP8','THAP9','THRA','THRB','THYN1','TIGD1','TIGD2','TIGD3','TIGD4','TIGD5','TIGD6','TIGD7','TLX1','TLX2','TLX3','TMF1','TOPORS','TP53','TP63','TP73','TPRX1','TRAFD1','TRERF1','TRPS1','TSC22D1','TSHZ1','TSHZ2','TSHZ3','TTF1','TWIST1','TWIST2','UBP1','UNCX','USF1','USF2','USF3','VAX1','VAX2','VDR','VENTX','VEZF1','VSX1','VSX2','WIZ','WT1','XBP1','XPA','YBX1','YBX2','YBX3','YY1','YY2','ZBED1','ZBED2','ZBED3','ZBED4','ZBED5','ZBED6','ZBED9','ZBTB1','ZBTB10','ZBTB11','ZBTB12','ZBTB14','ZBTB16','ZBTB17','ZBTB18','ZBTB2','ZBTB20','ZBTB21','ZBTB22','ZBTB24','ZBTB25','ZBTB26','ZBTB3','ZBTB32','ZBTB33','ZBTB34','ZBTB37','ZBTB38','ZBTB39','ZBTB4','ZBTB40','ZBTB41','ZBTB42','ZBTB43','ZBTB44','ZBTB45','ZBTB46','ZBTB47','ZBTB48','ZBTB49','ZBTB5','ZBTB6','ZBTB7A','ZBTB7B','ZBTB7C','ZBTB8A','ZBTB8B','ZBTB9','ZC3H8','ZEB1','ZEB2','ZFAT','ZFHX2','ZFHX3','ZFHX4','ZFP1','ZFP14','ZFP2','ZFP28','ZFP3','ZFP30','ZFP37','ZFP41','ZFP42','ZFP57','ZFP62','ZFP64','ZFP69','ZFP69B','ZFP82','ZFP90','ZFP91','ZFP92','ZFPM1','ZFPM2','ZFX','ZFY','ZGLP1','ZGPAT','ZHX1','ZHX2','ZHX3','ZIC1','ZIC2','ZIC3','ZIC4','ZIC5','ZIK1','ZIM2','ZIM3','ZKSCAN1','ZKSCAN2','ZKSCAN3','ZKSCAN4','ZKSCAN5','ZKSCAN7','ZKSCAN8','ZMAT1','ZMAT4','ZNF10','ZNF100','ZNF101','ZNF107','ZNF112','ZNF114','ZNF117','ZNF12','ZNF121','ZNF124','ZNF131','ZNF132','ZNF133','ZNF134','ZNF135','ZNF136','ZNF138','ZNF14','ZNF140','ZNF141','ZNF142','ZNF143','ZNF146','ZNF148','ZNF154','ZNF155','ZNF157','ZNF16','ZNF160','ZNF165','ZNF169','ZNF17','ZNF174','ZNF175','ZNF177','ZNF18','ZNF180','ZNF181','ZNF182','ZNF184','ZNF189','ZNF19','ZNF195','ZNF197','ZNF2','ZNF20','ZNF200','ZNF202','ZNF205','ZNF207','ZNF208','ZNF211','ZNF212','ZNF213','ZNF214','ZNF215','ZNF217','ZNF219','ZNF22','ZNF221','ZNF222','ZNF223','ZNF224','ZNF225','ZNF226','ZNF227','ZNF229','ZNF23','ZNF230','ZNF232','ZNF233','ZNF234','ZNF235','ZNF236','ZNF239','ZNF24','ZNF248','ZNF25','ZNF250','ZNF251','ZNF253','ZNF254','ZNF256','ZNF257','ZNF26','ZNF260','ZNF263','ZNF264','ZNF266','ZNF267','ZNF268','ZNF273','ZNF274','ZNF275','ZNF276','ZNF277','ZNF28','ZNF280A','ZNF280B','ZNF280C','ZNF280D','ZNF281','ZNF282','ZNF283','ZNF284','ZNF285','ZNF286A','ZNF286B','ZNF287','ZNF292','ZNF296','ZNF3','ZNF30','ZNF300','ZNF302','ZNF304','ZNF311','ZNF316','ZNF317','ZNF318','ZNF319','ZNF32','ZNF320','ZNF322','ZNF324','ZNF324B','ZNF326','ZNF329','ZNF331','ZNF333','ZNF334','ZNF335','ZNF337','ZNF33A','ZNF33B','ZNF34','ZNF341','ZNF343','ZNF345','ZNF346','ZNF347','ZNF35','ZNF350','ZNF354A','ZNF354B','ZNF354C','ZNF358','ZNF362','ZNF365','ZNF366','ZNF367','ZNF37A','ZNF382','ZNF383','ZNF384','ZNF385A','ZNF385B','ZNF385C','ZNF385D','ZNF391','ZNF394','ZNF395','ZNF396','ZNF397','ZNF398','ZNF404','ZNF407','ZNF408','ZNF41','ZNF410','ZNF414','ZNF415','ZNF416','ZNF417','ZNF418','ZNF419','ZNF420','ZNF423','ZNF425','ZNF426','ZNF428','ZNF429','ZNF43','ZNF430','ZNF431','ZNF432','ZNF433','ZNF436','ZNF438','ZNF439','ZNF44','ZNF440','ZNF441','ZNF442','ZNF443','ZNF444','ZNF445','ZNF446','ZNF449','ZNF45','ZNF451','ZNF454','ZNF460','ZNF461','ZNF462','ZNF467','ZNF468','ZNF469','ZNF470','ZNF471','ZNF473','ZNF474','ZNF479','ZNF48','ZNF480','ZNF483','ZNF484','ZNF485','ZNF486','ZNF487','ZNF488','ZNF490','ZNF491','ZNF492','ZNF493','ZNF496','ZNF497','ZNF500','ZNF501','ZNF502','ZNF503','ZNF506','ZNF507','ZNF510','ZNF511','ZNF512','ZNF512B','ZNF513','ZNF514','ZNF516','ZNF517','ZNF518A','ZNF518B','ZNF519','ZNF521','ZNF524','ZNF525','ZNF526','ZNF527','ZNF528','ZNF529','ZNF530','ZNF532','ZNF534','ZNF536','ZNF540','ZNF541','ZNF543','ZNF544','ZNF546','ZNF547','ZNF548','ZNF549','ZNF550','ZNF551','ZNF552','ZNF554','ZNF555','ZNF556','ZNF557','ZNF558','ZNF559','ZNF560','ZNF561','ZNF562','ZNF563','ZNF564','ZNF565','ZNF566','ZNF567','ZNF568','ZNF569','ZNF57','ZNF570','ZNF571','ZNF572','ZNF573','ZNF574','ZNF575','ZNF576','ZNF577','ZNF578','ZNF579','ZNF580','ZNF581','ZNF582','ZNF583','ZNF584','ZNF585A','ZNF585B','ZNF586','ZNF587','ZNF587B','ZNF589','ZNF592','ZNF594','ZNF595','ZNF596','ZNF597','ZNF598','ZNF599','ZNF600','ZNF605','ZNF606','ZNF607','ZNF608','ZNF609','ZNF610','ZNF611','ZNF613','ZNF614','ZNF615','ZNF616','ZNF618','ZNF619','ZNF620','ZNF621','ZNF623','ZNF624','ZNF625','ZNF626','ZNF627','ZNF628','ZNF629','ZNF630','ZNF639','ZNF641','ZNF644','ZNF645','ZNF646','ZNF648','ZNF649','ZNF652','ZNF653','ZNF654','ZNF655','ZNF658','ZNF66','ZNF660','ZNF662','ZNF664','ZNF665','ZNF667','ZNF668','ZNF669','ZNF670','ZNF671','ZNF672','ZNF674','ZNF675','ZNF676','ZNF677','ZNF678','ZNF679','ZNF680','ZNF681','ZNF682','ZNF683','ZNF684','ZNF687','ZNF688','ZNF689','ZNF69','ZNF691','ZNF692','ZNF695','ZNF696','ZNF697','ZNF699','ZNF7','ZNF70','ZNF700','ZNF701','ZNF703','ZNF704','ZNF705A','ZNF705B','ZNF705D','ZNF705E','ZNF705G','ZNF706','ZNF707','ZNF708','ZNF709','ZNF71','ZNF710','ZNF711','ZNF713','ZNF714','ZNF716','ZNF717','ZNF718','ZNF721','ZNF724','ZNF726','ZNF727','ZNF728','ZNF729','ZNF730','ZNF732','ZNF735','ZNF736','ZNF737','ZNF74','ZNF740','ZNF746','ZNF747','ZNF749','ZNF750','ZNF75A','ZNF75D','ZNF76','ZNF761','ZNF763','ZNF764','ZNF765','ZNF766','ZNF768','ZNF77','ZNF770','ZNF771','ZNF772','ZNF773','ZNF774','ZNF775','ZNF776','ZNF777','ZNF778','ZNF780A','ZNF780B','ZNF781','ZNF782','ZNF783','ZNF784','ZNF785','ZNF786','ZNF787','ZNF788','ZNF789','ZNF79','ZNF790','ZNF791','ZNF792','ZNF793','ZNF799','ZNF8','ZNF80','ZNF800','ZNF804A','ZNF804B','ZNF805','ZNF808','ZNF81','ZNF813','ZNF814','ZNF816','ZNF821','ZNF823','ZNF827','ZNF829','ZNF83','ZNF830','ZNF831','ZNF835','ZNF836','ZNF837','ZNF84','ZNF841','ZNF843','ZNF844','ZNF845','ZNF846','ZNF85','ZNF850','ZNF852','ZNF853','ZNF860','ZNF865','ZNF878','ZNF879','ZNF880','ZNF883','ZNF888','ZNF891','ZNF90','ZNF91','ZNF92','ZNF93','ZNF98','ZNF99','ZSCAN1','ZSCAN10','ZSCAN12','ZSCAN16','ZSCAN18','ZSCAN2','ZSCAN20','ZSCAN21','ZSCAN22','ZSCAN23','ZSCAN25','ZSCAN26','ZSCAN29','ZSCAN30','ZSCAN31','ZSCAN32','ZSCAN4','ZSCAN5A','ZSCAN5B','ZSCAN5C','ZSCAN9','ZUFSP','ZXDA','ZXDB','ZXDC','ZZZ3']
TFs=TFs1+TFs2

#TFs other, inclusive of the above list, not necessarily DNA binding proteins but affect transcription
TFs1Other=['AATF','ABL1','ABRA','AC008770.3','AC020915.1','AC023509.3','AC092835.1','AC105001.2','AC138696.1','ACOD1','ACTR3B','ACTR3C','ADAM8','ADAMTS17','ADAMTS19','ADAR','ADGRA1','ADGRG3','ADNP','ADNP2','AEBP1','AEBP2','AES','AFAP1 ','AFF1','AFF3','AFF4','AGER','AGT','AHCTF1','AHDC1','AHR','AHRR','AIM2','AIRE','AKAP8','AKAP8L','AKNA','AKT1','ALK','ALKBH2','ALX1','ALX3','ALX4','AMH','ANAPC2','ANHX','ANKRD30A','ANKZF1','ANXA3','ANXA4','AP5Z1','APBB3','APC2','APEX1','APOA4','APOBEC3A','APOBEC3B','APOBEC3G','APTX','AR','ARGFX','ARHGAP35','ARHGEF2','ARHGEF5','ARID1A','ARID1B','ARID2','ARID3A','ARID3B','ARID3C','ARID4A','ARID4B','ARID5A','ARID5B','ARNT','ARNT2','ARNTL','ARNTL2','ARRB1','ARRB2','ARX','ASCC1','ASCL1','ASCL2','ASCL3','ASCL4','ASCL5','ASH1L','ATF1','ATF2','ATF3','ATF4','ATF5','ATF6','ATF6B','ATF7','ATF7IP','ATM','ATMIN','ATOH1','ATOH7','ATOH8','ATR','ATRX','ATXN7','BACH1','BACH2','BAG1','BANF1','BANP','BARHL1','BARHL2','BARX1','BARX2','BATF','BATF2','BATF3','BAZ2A','BAZ2B','BBX','BCL10','BCL11A','BCL11B','BCL3','BCL6','BCL6B','BCLAF1','BCOR','BDP1','BEND4','BEX1','BHLHA15','BHLHA9','BHLHE22','BHLHE23','BHLHE40','BHLHE41','BHMG1','BIN1','BLM','BLZF1','BMI1','BMP2','BMP4','BMP7','BMPR1A','BNC1','BNC2','BOD1L1','BOLA1','BOLA2','BOLA2B','BOLA3','BORCS8-MEF2B','BPNT1','BPTF','BRCA1','BRD4','BRD7','BRD8','BRD9','BRF1','BRF2','BRMS1','BRPF1','BSX','BTAF1','BTBD8','BTG2','BTK','BTRC','BUD31','C11orf95','C14orf39','C17orf49','C1D','C20orf194','C3orf33','CACTIN','CAMK1D','CAMK2A','CAMTA1','CAMTA2','CAPN15','CAPN3','CARD11','CARD14','CARF','CARHSP1','CASP8AP2','CASZ1','CAT','CBFA2T2','CBFA2T3','CBFB','CBL','CBLL1','CBX2','CC2D1A','CC2D1B','CCDC124','CCDC169-SOHLH2','CCDC17','CCDC3','CCDC71','CCDC83','CCNT1','CCNT2','CD36','CD40','CD40LG','CDC5L','CDC6','CDK5RAP3','CDK7','CDK8','CDK9','CDKN1A','CDKN2A','CDX1','CDX2','CDX4','CEBPA','CEBPB','CEBPD','CEBPE','CEBPG','CEBPZ','CELF3','CENPA','CENPB','CENPBD1','CENPS','CENPT','CENPX','CERS2','CERS3','CERS4','CERS5','CERS6','CFLAR','CGGBP1','CHAMP1','CHCHD3','CHD1','CHD2','CHD3','CHD4','CHD6','CHD7','CHD9','CHP1','CHRAC1','CHUK','CHURC1','CIB1','CIC','CIITA','CIR1','CITED1','CITED2','CITED4','CIZ1','CLOCK','CLU','CMKLR1','CNBP','CNOT3','CNOT4','CNOT7','CNOT8','COMMD1','COMMD6','COMMD7','COPS2','COPS3','COPS4','CPEB1','CPSF4','CPSF4L','CPXCR1','CRAMP1','CREB1','CREB3','CREB3L1','CREB3L2','CREB3L3','CREB3L4','CREB5','CREBBP','CREBL2','CREBRF','CREBZF','CREG1','CREM','CRTC1','CRTC2','CRTC3','CRX','CSDC2','CSDE1','CSRNP1','CSRNP2','CSRNP3','CTBP1','CTBP2','CTCF','CTCFL','CTH','CTNNB1','CTNNBIP1','CUL1','CUL2','CUL3','CUL4A','CUL4B','CUL5','CUX1','CUX2','CXXC1','CXXC4','CXXC5','CYLD','CYP1B1','CYTL1','DAB2IP','DACH1','DACH2','DAP','DAXX','DBP','DBX1','DBX2','DCP1A','DDB1','DDB2','DDIT3','DDN','DDR2','DDRGK1','DDX3X','DDX5','DDX58','DEAF1','DEK','DEPDC1','DEPDC1B','DEPDC4','DEPDC5','DEPDC7','DEPTOR','DHX34','DHX36','DHX57','DISP3','DLX1','DLX2','DLX3','DLX4','DLX5','DLX6','DMAP1','DMBX1','DMC1','DMRT1','DMRT2','DMRT3','DMRTA1','DMRTA2','DMRTB1','DMRTC1','DMRTC2','DMTF1','DNA2','DNAJA3','DNAJC1','DNAJC2','DNAJC21','DNMT1','DNMT3A','DNMT3B','DNMT3L','DNTT','DNTTIP1','DOT1L','DPF1','DPF2','DPF3','DPRX','DR1','DRAP1','DRGX','DSP','DTX1','DUS3L','DUSP12','DUX1','DUX3','DUX4','DUXA','DVL1','DVL2','DVL3','DZIP1','DZIP1L','E2F1','E2F2','E2F3','E2F4','E2F5','E2F6','E2F7','E2F8','E4F1','EAF2','EBF1','EBF2','EBF3','EBF4','EDA','EDA2R','EDF1','EDN1','EEA1','EED','EGLN1','EGR1','EGR2','EGR3','EGR4','EHF','EID1','EID2','EIF2AK2','EIF2AK4','EIF3C','EIF3CL','EIF3K','ELF1','ELF2','ELF3','ELF4','ELF5','ELK1','ELK3','ELK4','ELMSAN1','EME1','EMX1','EMX2','EN1','EN2','ENO1','EOMES','EP300','EP400','EPAS1','EPHA5','EPM2AIP1','ERBIN','ERC1','ERCC3','ERCC4','ERF','ERG','ESR1','ESR2','ESRRA','ESRRB','ESRRG','ESX1','ETS1','ETS2','ETV1','ETV2','ETV3','ETV3L','ETV4','ETV5','ETV6','ETV7','EVX1','EVX2','EWSR1','EXO1','EXOC2','EXOG','EZH1','EZH2','FAAP24','FAM153C','FAM170A','FAM171B','FAM200A','FAM200B','FAN1','FANCA','FANCD2','FANK1','FARSA','FARSB','FBN1','FBXL19','FBXO41','FEN1','FER','FERD3L','FEV','FEZF1','FEZF2','FGD1','FHL2','FIGLA','FIZ1','FLI1','FLNA','FLYWCH1','FMNL2','FOS','FOSB','FOSL1','FOSL2','FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L2','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXN4','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1','FUBP1','FUBP3','FUS','FZD1','FZD2','FZD4','FZD6','GABPA','GABPB1','GABPB2','GADD45A','GAS6','GAS7','GATA1','GATA2','GATA3','GATA4','GATA5','GATA6','GATAD1','GATAD2A','GATAD2B','GBX1','GBX2','GCFC2','GCM1','GCM2','GEN1','GFI1','GFI1B','GLA','GLI1','GLI2','GLI3','GLI4','GLIS1','GLIS2','GLIS3','GLMP','GLYR1','GMEB1','GMEB2','GOLGB1','GON4L','GPATCH8','GPBP1','GPBP1L1','GPER1','GPR155','GREM1','GRHL1','GRHL2','GRHL3','GRM6','GSC','GSC2','GSX1','GSX2','GTF2A1','GTF2A1L','GTF2A2','GTF2B','GTF2E2','GTF2F1','GTF2F2','GTF2H1','GTF2H2','GTF2H2C','GTF2H3','GTF2H4','GTF2I','GTF2IRD1','GTF2IRD2','GTF2IRD2B','GTF3A','GTF3C1','GTF3C2','GULP1','GZF1','H1F0','H1FOO','H1FX','HAND1','HAND2','HAVCR2','HBP1','HCFC1','HCK','HDAC1','HDAC10','HDAC2','HDAC3','HDAC4','HDAC5','HDAC6','HDAC7','HDAC8','HDAC9','HDX','HELT','HELZ','HELZ2','HES1','HES2','HES3','HES4','HES5','HES6','HES7','HESX1','HEY1','HEY2','HEYL','HHEX','HIC1','HIC2','HIF1A','HIF3A','HINFP','HIPK2','HIRA','HIST1H1A','HIST1H1B','HIST1H1C','HIST1H1D','HIST1H1E','HIST1H1T','HIVEP1','HIVEP2','HIVEP3','HKR1','HLA-DQB1','HLA-DQB2','HLA-DRB1','HLA-DRB3','HLA-DRB5','HLF','HLTF','HLX','HMBOX1','HMG20A','HMG20B','HMGA1','HMGA2','HMGB1','HMGB2','HMGB3','HMGB4','HMGN1','HMGN3','HMGXB3','HMGXB4','HMOX1','HMX1','HMX2','HMX3','HNF1A','HNF1B','HNF4A','HNF4G','HNRNPA1','HNRNPA3','HNRNPAB','HNRNPCL2','HNRNPD','HNRNPDL','HNRNPK','HNRNPL','HNRNPU','HOMEZ','HOOK2','HOPX','HOXA1','HOXA10','HOXA11','HOXA13','HOXA2','HOXA3','HOXA4','HOXA5','HOXA6','HOXA7','HOXA9','HOXB1','HOXB13','HOXB2','HOXB3','HOXB4','HOXB5','HOXB6','HOXB7','HOXB8','HOXB9','HOXC10','HOXC11','HOXC12','HOXC13','HOXC4','HOXC5','HOXC6','HOXC8','HOXC9','HOXD1','HOXD10','HOXD11','HOXD12','HOXD13','HOXD3','HOXD4','HOXD8','HOXD9','HP1BP3','HR','HSBP1','HSF1','HSF2','HSF4','HSF5','HSFX1','HSFX2','HSFY1','HSFY2','HSPA1A','HSPA1B','ICAM1','ID1','ID2','ID3','ID4','IFI16','IFNB1','IGHM','IKBKB','IKBKG','IKZF1','IKZF2','IKZF3','IKZF4','IKZF5','IL10','IL1B','IL1RAP','IL4','IL5','IL6','INF2','ING4','INS','INSM1','INSM2','IQGAP1','IRAK1','IRAK2','IRAK3','IRF1','IRF2','IRF2BP1','IRF3','IRF4','IRF5','IRF6','IRF7','IRF8','IRF9','IRX1','IRX2','IRX3','IRX4','IRX5','IRX6','ISL1','ISL2','ISX','ITCH','ITGB2','ITGB3BP','IZUMO2','JADE1','JAK2','JARID2','JAZF1','JDP2','JMY','JRK','JRKL','JUN','JUNB','JUND','JUP','KANSL2','KAT2A','KAT2B','KAT5','KAT6A','KAT6B','KAT7','KAT8','KCMF1','KCNIP3','KDM1A','KDM2A','KDM2B','KDM3A','KDM5A','KDM5B','KDM5C','KDM5D','KEAP1','KIAA1549','KIAA1683','KIAA1841','KIAA1958','KIF15','KIN','KIT','KLF1','KLF10','KLF11','KLF12','KLF13','KLF14','KLF15','KLF16','KLF17','KLF2','KLF3','KLF4','KLF5','KLF6','KLF7','KLF8','KLF9','KMT2A','KMT2B','KMT2C','KMT2D','KNTC1','KRAS','KRBOX1','KRT13','KRTAP5-1','KRTAP5-9','KRTAP9-9','L3MBTL1','L3MBTL3','L3MBTL4','LARP1','LARP1B','LARP4','LARP4B','LARP6','LARP7','LBX1','LBX2','LCOR','LCORL','LEF1','LEUTX','LGALS9','LGR4','LHX1','LHX2','LHX3','LHX4','LHX5','LHX6','LHX8','LHX9','LIG1','LIG3','LIN28A','LIN28B','LIN54','LMO2','LMO4','LMX1A','LMX1B','LPP','LRP5','LRP6','LRP8','LRRFIP1','LRRFIP2','LSR','LTF','LTN1','LYAR','LYL1','LZTFL1','LZTR1','LZTS1','MACC1','MACF1','MAD2L2','MAEL','MAF','MAFA','MAFB','MAFF','MAFG','MAFK','MALT1','MAML1','MAML3','MAP2K5','MAP3K10','MAP3K13','MAP3K7','MAPK1','MAPK10','MAPK11','MAPK14','MAPK3','MAPK8','MAPK9','MATR3','MAVS','MAX','MAZ','MB21D1','MBD1','MBD2','MBD3','MBD4','MBD6','MBNL1','MBNL2','MBNL3','MBTPS2','MDFI','MDM2','MDM4','MECOM','MECP2','MED1','MED12','MED13','MED13L','MED14','MED15','MED16','MED17','MED21','MED23','MED24','MED6','MEF2A','MEF2B','MEF2C','MEF2D','MEIS1','MEIS2','MEIS3','MEN1','MEOX1','MEOX2','MESP1','MESP2','MET','MGA','MGMT','MID2','MIER1','MIER2','MIER3','MINK1','MIS18BP1','MITF','MIXL1','MKL1','MKL2','MKRN1','MKRN2','MKRN3','MKX','MLLT1','MLLT10','MLLT11','MLLT3','MLLT6','MLX','MLXIP','MLXIPL','MN1','MNAT1','MNT','MNX1','MORC3','MPG','MRPL28','MRRF','MSANTD1','MSANTD3','MSANTD4','MSC','MSGN1','MSH2','MSH3','MSH6','MSL3','MSRB2','MST1R','MSX1','MSX2','MTA1','MTA2','MTA3','MTDH','MTERF1','MTERF2','MTERF3','MTERF4','MTF1','MTF2','MTPN','MUS81','MXD1','MXD3','MXD4','MXI1','MYB','MYBBP1A','MYBL1','MYBL2','MYC','MYCBP','MYCL','MYCN','MYD88','MYEF2','MYF5','MYF6','MYNN','MYOCD','MYOD1','MYOG','MYPOP','MYRF','MYRFL','MYSM1','MYT1','MYT1L','MZF1','NAB2','NABP2','NACC2','NAIF1','NANOG','NANOGNB','NANOGP8','NCOA1','NCOA2','NCOA3','NCOA4','NCOA5','NCOA6','NCOA7','NCOR1','NCOR2','NDC80','NDN','NDP','NEB','NEIL1','NEUROD1','NEUROD2','NEUROD4','NEUROD6','NEUROG1','NEUROG2','NEUROG3','NFAM1','NFAT5','NFATC1','NFATC2','NFATC3','NFATC4','NFE2','NFE2L1','NFE2L2','NFE2L3','NFE4','NFIA','NFIB','NFIC','NFIL3','NFIX','NFKB1','NFKB2','NFKBIA','NFKBIB','NFKBID','NFKBIL1','NFKBIZ','NFRKB','NFX1','NFXL1','NFYA','NFYB','NFYC','NHLH1','NHLH2','NKRF','NKX1-1','NKX1-2','NKX2-1','NKX2-2','NKX2-3','NKX2-4','NKX2-5','NKX2-6','NKX2-8','NKX3-1','NKX3-2','NKX6-1','NKX6-2','NKX6-3','NLRC3','NLRC4','NLRC5','NLRP12','NLRP2B','NLRP3','NME2','NOBOX','NOC3L','NOC4L','NOCT','NOD1','NOD2','NODAL','NONO','NOP2','NOTCH1','NOTCH2','NOTCH3','NOTO','NPAS1','NPAS2','NPAS3','NPAS4','NPM1','NR0B1','NR0B2','NR1D1','NR1D2','NR1H2','NR1H3','NR1H4','NR1I2','NR1I3','NR2C1','NR2C2','NR2E1','NR2E3','NR2F1','NR2F2','NR2F6','NR3C1','NR3C2','NR4A1','NR4A2','NR4A3','NR5A1','NR5A2','NR6A1','NRBF2','NRF1','NRG1','NRIP1','NRIP3','NRK','NRL','NSD1','NSD2','NTRK1','NUCKS1','NUFIP1','NUP98','NUPL2']
TFs2Other=['OGG1','OLIG1','OLIG2','OLIG3','ONECUT1','ONECUT2','ONECUT3','OPRD1','OSR1','OSR2','OTOP3','OTP','OTULIN','OTX1','OTX2','OVOL1','OVOL2','OVOL3','PA2G4','PABPN1','PARK7','PARP1','PARP10','PARP12','PATZ1','PAWR','PAX1','PAX2','PAX3','PAX4','PAX5','PAX6','PAX7','PAX8','PAX9','PAXBP1','PAXIP1','PBRM1','PBX1','PBX2','PBX3','PBX4','PCBD1','PCBP1','PCBP2','PCBP3','PCGF2','PCGF6','PCNA','PCSK6','PDCD7','PDS5B','PDX1','PEG3','PELI1','PER1','PER2','PER3','PEX14','PFDN1','PGBD1','PGR','PGS1','PHB','PHB2','PHF1','PHF19 ','PHF20','PHF21A','PHF5A','PHOX2A','PHOX2B','PHTF1','PHTF2','PIAS1','PIAS2','PIAS3','PIAS4','PIDD1','PIKFYVE','PIM1','PIN1','PINK1','PITX1','PITX2','PITX3','PKHD1','PKHD1L1','PKNOX1','PKNOX2','PLA2G10','PLA2G1B','PLAG1','PLAGL1','PLAGL2','PLEK','PLEK2','PLEKHA4','PLPP3','PLSCR1','PLXNA1','PLXNA2','PLXNA3','PLXNA4','PLXNB1','PLXNB2','PLXNB3','PLXNC1','PLXND1','PML','PMS1','PNKP','POGK','POGZ','POLA1','POLB','POLE3','POLE4','POLG','POLH','POLI','POLK','POLL','POLM','POLN','POLQ','POLR1B','POLR2A','POLR2B','POLR2E','POLR2J','POLR2K','POLR2L','POLR3A','POLRMT','POT1','POU1F1','POU2AF1','POU2F1','POU2F2','POU2F3','POU3F1','POU3F2','POU3F3','POU3F4','POU4F1','POU4F2','POU4F3','POU5F1','POU5F1B','POU5F2','POU6F1','POU6F2','PPARA','PPARD','PPARG','PPARGC1A','PPARGC1B','PPP1R10','PPP1R13L','PPP1R21','PPP2R3B','PPP2R5B','PPP3CA','PPRC1','PQBP1','PRAM1','PRAP1','PRB3','PRDM1','PRDM10','PRDM11','PRDM12','PRDM13','PRDM14','PRDM15','PRDM16','PRDM2','PRDM4','PRDM5','PRDM6','PRDM7','PRDM8','PRDM9','PRDX3','PREB','PREX1','PREX2','PRH2','PRIM2','PRIMPOL','PRKCB','PRKCH','PRKCI','PRKCQ','PRKCZ','PRKD1','PRKD2','PRKN','PRMT2','PRMT3','PRNP','PROP1','PROX1','PROX2','PRR12','PRR3','PRRX1','PRRX2','PSMA6','PSMC5','PSMD10','PSMD11','PSMD12','PSPC1','PTCH1','PTEN','PTF1A','PTGIS','PTH','PTTG1','PUM3','PURA','PURB','PURG','PYCARD','PYDC1','PYDC2','QRICH1','RAB11FIP4','RAB7B','RAD21','RAD51','RAG1','RAI1','RAPGEF3','RAPGEF4','RAPGEF5','RARA','RARB','RARG','RAX','RAX2','RB1','RBAK','RBBP7','RBCK1','RBL1','RBL2','RBM10','RBM20','RBM22','RBM26','RBM27','RBM5','RBM6','RBPJ','RBPJL','RBSN','RC3H1','RC3H2','RCAN1','RCOR1','RCOR2','RCOR3','RECQL','REL','RELA','RELB','RELL2','RELN','REPIN1','RERE','REST','REV1','REXO4','RFC1','RFX1','RFX2','RFX3','RFX4','RFX5','RFX6','RFX7','RFX8','RFXANK','RFXAP','RGCC','RGS11','RGS6','RGS7','RGS9','RHEBL1','RHOXF1','RHOXF2','RHOXF2B','RING1','RIOK2','RIPK1','RIPK2','RIPK3','RIPK4','RLF','RLIM','RNASE2','RNASEH1','RNF113A','RNF113B','RNF114','RNF125','RNF138','RNF14','RNF166','RNF2','RNF219','RNF25','RNF31','RNF4','RNF41','RORA','RORB','RORC','RPA2','RPA4','RPL36A','RPL7','RPL7A','RPS27A','RPS6KA4','RPS6KA5','RREB1','RRN3','RUNX1','RUNX1T1','RUNX2','RUNX3','RWDD3','RXRA','RXRB','RXRG','RYBP','S100A12','S100A8','S100A9','SAFB','SAFB2','SALL1','SALL2','SALL3','SALL4','SART1','SATB1','SATB2','SAV1','SCAND1','SCAND2P','SCAPER','SCMH1','SCML1','SCML2','SCML4','SCOC','SCRT1','SCRT2','SCX','SEBOX','SEMA4A','SERTAD1','SERTAD2','SETBP1','SETD6','SETDB1','SETDB2','SF3A2','SF3A3','SFPQ','SFRP4','SFRP5','SFTPB','SGK1','SGK3','SGSM2','SHH','SHOX','SHOX2','SHPRH','SIGIRR','SIK1','SIM1','SIM2','SIN3A','SIN3B','SIRT1','SIRT2','SIRT3','SIRT6','SIVA1','SIX1','SIX2','SIX3','SIX4','SIX5','SIX6','SKI','SKIDA1','SKIL','SKOR1','SKOR2','SLC22A4','SLC26A10','SLC26A3','SLC2A4RG','SLC30A9','SLC39A10','SLC4A10','SMAD1','SMAD2','SMAD3','SMAD4','SMAD5','SMAD6','SMAD7','SMAD9','SMARCA1','SMARCA2','SMARCA4','SMARCA5','SMARCAL1','SMARCB1','SMARCC1','SMARCC2','SMARCD1','SMARCD2','SMARCD3','SMARCE1','SMC3','SMO','SMUG1','SMYD1','SMYD3','SNAI1','SNAI2','SNAI3','SNAPC2','SNAPC4','SNAPC5','SNRPC','SNURF','SNW1','SOGA3','SOHLH1','SOHLH2','SON','SORBS2','SOX1','SOX10','SOX11','SOX12','SOX13','SOX14','SOX15','SOX17','SOX18','SOX2','SOX21','SOX3','SOX30','SOX4','SOX5','SOX6','SOX7','SOX8','SOX9','SP1','SP100','SP110','SP140','SP140L','SP2','SP3','SP4','SP5','SP6','SP7','SP8','SP9','SPDEF','SPEN','SPHK1','SPI1','SPIB','SPIC','SPZ1','SRCAP','SREBF1','SREBF2','SRF','SRY','SSB','SSBP2','SSBP3','SSBP4','SSH1','SSH2','SSH3','SSRP1','SSX1','ST18','STAG1','STAT1','STAT2','STAT3','STAT4','STAT5A','STAT5B','STAT6','STK16','STK3','STK36','STRN','STRN3','SUB1','SUFU','SUMO1','SUPT16H','SUPT4H1','SUPT5H','SUPT6H','SUV39H1','SUV39H2','SUZ12','SYK','T','TAB1','TAB2','TAB3','TADA2A','TADA2B','TADA3','TAF1','TAF10','TAF11','TAF12','TAF13','TAF1A','TAF1B','TAF1C','TAF1L','TAF2','TAF3','TAF4','TAF4B','TAF5','TAF5L','TAF6','TAF6L','TAF7','TAF7L','TAF8','TAF9','TAF9B','TAL1','TAL2','TARDBP','TAX1BP1','TBL1Y','TBP','TBPL1','TBPL2','TBR1','TBX1','TBX10','TBX15','TBX18','TBX19','TBX2','TBX20','TBX21','TBX22','TBX3','TBX4','TBX5','TBX6','TCEA2','TCEAL1','TCEAL7','TCEAL8','TCF12','TCF15','TCF19','TCF20','TCF21','TCF23','TCF24','TCF25','TCF3','TCF4','TCF7','TCF7L1','TCF7L2','TCFL5','TDG','TDP1','TDP2','TEAD1','TEAD2','TEAD3','TEAD4','TEF','TERB1','TERF1','TERF2','TERF2IP','TESMIN','TET1','TET2','TET3','TFAM','TFAP2A','TFAP2B','TFAP2C','TFAP2D','TFAP2E','TFAP4','TFCP2','TFCP2L1','TFDP1','TFDP2','TFDP3','TFE3','TFEB','TFEC','TGFB1','TGFB1I1','TGIF1','TGIF2','TGIF2LX','TGIF2LY','TGS1','THAP1','THAP10','THAP11','THAP12','THAP2','THAP3','THAP4','THAP5','THAP6','THAP7','THAP8','THAP9','THRA','THRAP3','THRB','THYN1','TIA1','TICAM1','TIGD1','TIGD2','TIGD3','TIGD4','TIGD5','TIGD6','TIGD7','TIPARP','TIRAP','TLE1','TLE2','TLE3','TLE4','TLE6','TLR2','TLR3','TLR4','TLR9','TLX1','TLX2','TLX3','TMEM235','TMF1','TNF','TNFAIP3','TNFRSF11A','TNFRSF4','TNFSF11','TNFSF18','TNFSF4','TOE1','TONSL','TOP1','TOP2A','TOP2B','TOPORS','TOX','TOX2','TOX3','TOX4','TP53','TP53BP1','TP63','TP73','TPRX1','TRADD','TRAF1','TRAF2','TRAF3','TRAF4','TRAF5','TRAF6','TRAFD1','TRAK2','TRAPPC9','TRERF1','TREX1','TRIB1','TRIB3','TRIM13','TRIM14','TRIM15','TRIM21','TRIM22','TRIM23','TRIM24','TRIM25','TRIM26','TRIM27','TRIM28','TRIM29','TRIM3','TRIM31','TRIM32','TRIM33','TRIM34','TRIM37','TRIM38','TRIM40','TRIM5','TRIM52','TRIM62','TRIM8','TRIP11','TRIT1','TRMT1','TRMT1L','TRPS1','TRRAP','TSC2','TSC22D1','TSC22D2','TSC22D3','TSC22D4','TSHZ1','TSHZ2','TSHZ3','TSN','TSSK4','TTF1','TUB','TULP4','TUT1','TWIST1','TWIST2','TXK','U2AF1','U2AF1L4','UBA52','UBB','UBC','UBE2K','UBE2N','UBE2V1','UBN1','UBP1','UBR4','UBTF','UBTFL1','UBTFL6','UFL1','UHRF1','UHRF2','UNCX','UNG','UNK','UNKL','USF1','USF2','USF3','USP39','USP7','UTF1','VAV1','VAX1','VAX2','VDR','VEGFA','VENTX','VEZF1','VGLL1','VGLL2','VGLL3','VHLL','VPS72','VSX1','VSX2','WDHD1','WFS1','WIZ','WNT1','WNT10B','WNT2','WNT3A','WNT5A','WNT8B','WRN','WT1','WWOX','WWP2','WWTR1','XBP1','XCL1','XPA','YBX1','YBX2','YBX3','YEATS2','YEATS4','YOD1','YY1','YY2','ZBED1','ZBED2','ZBED3','ZBED4','ZBED5','ZBED6','ZBED6CL','ZBED8','ZBED9','ZBP1','ZBTB1','ZBTB10','ZBTB11','ZBTB12','ZBTB14','ZBTB16','ZBTB17','ZBTB18','ZBTB2','ZBTB20','ZBTB21','ZBTB22','ZBTB24','ZBTB25','ZBTB26','ZBTB3','ZBTB32','ZBTB33','ZBTB34','ZBTB37','ZBTB38','ZBTB39','ZBTB4','ZBTB40','ZBTB41','ZBTB42','ZBTB43','ZBTB44','ZBTB45','ZBTB46','ZBTB47','ZBTB48','ZBTB49','ZBTB5','ZBTB6','ZBTB7A','ZBTB7B','ZBTB7C','ZBTB8A','ZBTB8B','ZBTB9','ZC3H10','ZC3H11A','ZC3H12A','ZC3H13','ZC3H14','ZC3H15','ZC3H18','ZC3H3','ZC3H4','ZC3H6','ZC3H7A','ZC3H7B','ZC3H8','ZC3HAV1','ZCCHC11','ZCCHC6','ZDHHC11','ZDHHC19','ZEB1','ZEB2','ZFAND5','ZFAT','ZFHX2','ZFHX3','ZFHX4','ZFP1','ZFP14','ZFP2','ZFP28','ZFP3','ZFP30','ZFP36','ZFP36L1','ZFP36L2','ZFP37','ZFP41','ZFP42','ZFP57','ZFP62','ZFP64','ZFP69','ZFP69B','ZFP82','ZFP90','ZFP91','ZFP92','ZFPL1','ZFPM1','ZFPM2','ZFR','ZFR2','ZFX','ZFY','ZFYVE26','ZGLP1','ZGPAT','ZHX1','ZHX2','ZHX3','ZIC1','ZIC2','ZIC3','ZIC4','ZIC5','ZIK1','ZIM2','ZIM3','ZKSCAN1','ZKSCAN2','ZKSCAN3','ZKSCAN4','ZKSCAN5','ZKSCAN7','ZKSCAN8','ZMAT1','ZMAT2','ZMAT3','ZMAT4','ZMAT5','ZMYM1','ZMYM2','ZMYM3','ZMYM4','ZMYM5','ZMYM6','ZMYND10','ZNF10','ZNF100','ZNF101','ZNF106','ZNF107','ZNF112','ZNF114','ZNF117','ZNF12','ZNF121','ZNF124','ZNF131','ZNF132','ZNF133','ZNF134','ZNF135','ZNF136','ZNF137P','ZNF138','ZNF14','ZNF140','ZNF141','ZNF142','ZNF143','ZNF146','ZNF148','ZNF154','ZNF155','ZNF157','ZNF16','ZNF160','ZNF165','ZNF169','ZNF17','ZNF174','ZNF175','ZNF177','ZNF18','ZNF180','ZNF181','ZNF182','ZNF184','ZNF185','ZNF189','ZNF19','ZNF195','ZNF197','ZNF2','ZNF20','ZNF200','ZNF202','ZNF205','ZNF207','ZNF208','ZNF211','ZNF212','ZNF213','ZNF214','ZNF215','ZNF217','ZNF219','ZNF22','ZNF221','ZNF222','ZNF223','ZNF224','ZNF225','ZNF226','ZNF227','ZNF229','ZNF23','ZNF230','ZNF232','ZNF233','ZNF234','ZNF235','ZNF236','ZNF239','ZNF24','ZNF248','ZNF25','ZNF250','ZNF251','ZNF253','ZNF254','ZNF256','ZNF257','ZNF26','ZNF260','ZNF263','ZNF264','ZNF266','ZNF267','ZNF268','ZNF273','ZNF274','ZNF275','ZNF276','ZNF277','ZNF28','ZNF280A','ZNF280B','ZNF280C','ZNF280D','ZNF281','ZNF282','ZNF283','ZNF284','ZNF285','ZNF286A','ZNF286B','ZNF287','ZNF292','ZNF296','ZNF3','ZNF30','ZNF300','ZNF302','ZNF304','ZNF311','ZNF316','ZNF317','ZNF318','ZNF319','ZNF32','ZNF320','ZNF322','ZNF324','ZNF324B','ZNF326','ZNF329','ZNF330','ZNF331','ZNF333','ZNF334','ZNF335','ZNF337','ZNF33A','ZNF33B','ZNF34','ZNF341','ZNF343','ZNF345','ZNF346','ZNF347','ZNF35','ZNF350','ZNF354A','ZNF354B','ZNF354C','ZNF355P','ZNF358','ZNF362','ZNF365','ZNF366','ZNF367','ZNF37A','ZNF382','ZNF383','ZNF384','ZNF385A','ZNF385B','ZNF385C','ZNF385D','ZNF391','ZNF394','ZNF395','ZNF396','ZNF397','ZNF398','ZNF404','ZNF407','ZNF408','ZNF41','ZNF410','ZNF414','ZNF415','ZNF416','ZNF417','ZNF418','ZNF419','ZNF420','ZNF423','ZNF425','ZNF426','ZNF428','ZNF429','ZNF43','ZNF430','ZNF431','ZNF432','ZNF433','ZNF436','ZNF438','ZNF439','ZNF44','ZNF440','ZNF441','ZNF442','ZNF443','ZNF444','ZNF445','ZNF446','ZNF449','ZNF45','ZNF451','ZNF454','ZNF460','ZNF461','ZNF462','ZNF467','ZNF468','ZNF469','ZNF470','ZNF471','ZNF473','ZNF474','ZNF479','ZNF48','ZNF480','ZNF483','ZNF484','ZNF485','ZNF486','ZNF487','ZNF488','ZNF490','ZNF491','ZNF492','ZNF493','ZNF496','ZNF497','ZNF500','ZNF501','ZNF502','ZNF503','ZNF506','ZNF507','ZNF510','ZNF511','ZNF512','ZNF512B','ZNF513','ZNF514','ZNF516','ZNF517','ZNF518A','ZNF518B','ZNF519']
TFs3Other=['ZNF521','ZNF524','ZNF525','ZNF526','ZNF527','ZNF528','ZNF529','ZNF530','ZNF532','ZNF534','ZNF536','ZNF540','ZNF541','ZNF542P','ZNF543','ZNF544','ZNF546','ZNF547','ZNF548','ZNF549','ZNF550','ZNF551','ZNF552','ZNF554','ZNF555','ZNF556','ZNF557','ZNF558','ZNF559','ZNF560','ZNF561','ZNF562','ZNF563','ZNF564','ZNF565','ZNF566','ZNF567','ZNF568','ZNF569','ZNF57','ZNF570','ZNF571','ZNF572','ZNF573','ZNF574','ZNF575','ZNF576','ZNF577','ZNF578','ZNF579','ZNF580','ZNF581','ZNF582','ZNF583','ZNF584','ZNF585A','ZNF585B','ZNF586','ZNF587','ZNF587B','ZNF589','ZNF592','ZNF593','ZNF594','ZNF595','ZNF596','ZNF597','ZNF598','ZNF599','ZNF600','ZNF605','ZNF606','ZNF607','ZNF608','ZNF609','ZNF610','ZNF611','ZNF613','ZNF614','ZNF615','ZNF616','ZNF618','ZNF619','ZNF620','ZNF621','ZNF622','ZNF623','ZNF624','ZNF625','ZNF626','ZNF627','ZNF628','ZNF629','ZNF630','ZNF638','ZNF639','ZNF641','ZNF644','ZNF645','ZNF646','ZNF648','ZNF649','ZNF652','ZNF653','ZNF654','ZNF655','ZNF658','ZNF66','ZNF660','ZNF662','ZNF664','ZNF665','ZNF667','ZNF668','ZNF669','ZNF670','ZNF671','ZNF672','ZNF674','ZNF675','ZNF676','ZNF677','ZNF678','ZNF679','ZNF680','ZNF681','ZNF682','ZNF683','ZNF684','ZNF687','ZNF688','ZNF689','ZNF69','ZNF691','ZNF692','ZNF695','ZNF696','ZNF697','ZNF699','ZNF7','ZNF70','ZNF700','ZNF701','ZNF703','ZNF704','ZNF705A','ZNF705B','ZNF705D','ZNF705E','ZNF705G','ZNF706','ZNF707','ZNF708','ZNF709','ZNF71','ZNF710','ZNF711','ZNF713','ZNF714','ZNF716','ZNF717','ZNF718','ZNF720','ZNF721','ZNF724','ZNF726','ZNF727','ZNF728','ZNF729','ZNF73','ZNF730','ZNF732','ZNF735','ZNF736','ZNF737','ZNF738','ZNF74','ZNF740','ZNF746','ZNF747','ZNF749','ZNF750','ZNF75A','ZNF75D','ZNF76','ZNF761','ZNF763','ZNF764','ZNF765','ZNF766','ZNF768','ZNF77','ZNF770','ZNF771','ZNF772','ZNF773','ZNF774','ZNF775','ZNF776','ZNF777','ZNF778','ZNF780A','ZNF780B','ZNF781','ZNF782','ZNF783','ZNF784','ZNF785','ZNF786','ZNF787','ZNF788','ZNF789','ZNF79','ZNF790','ZNF791','ZNF792','ZNF793','ZNF799','ZNF8','ZNF80','ZNF800','ZNF804A','ZNF804B','ZNF805','ZNF808','ZNF81','ZNF812P','ZNF813','ZNF814','ZNF816','ZNF821','ZNF823','ZNF827','ZNF829','ZNF83','ZNF830','ZNF831','ZNF835','ZNF836','ZNF837','ZNF839','ZNF84','ZNF841','ZNF843','ZNF844','ZNF845','ZNF846','ZNF85','ZNF850','ZNF852','ZNF853','ZNF860','ZNF862','ZNF865','ZNF878','ZNF879','ZNF880','ZNF883','ZNF888','ZNF891','ZNF90','ZNF91','ZNF92','ZNF93','ZNF98','ZNF99','ZNFX1','ZRANB2','ZRSR1','ZRSR2','ZSCAN1','ZSCAN10','ZSCAN12','ZSCAN16','ZSCAN18','ZSCAN2','ZSCAN20','ZSCAN21','ZSCAN22','ZSCAN23','ZSCAN25','ZSCAN26','ZSCAN29','ZSCAN30','ZSCAN31','ZSCAN32','ZSCAN4','ZSCAN5A','ZSCAN5B','ZSCAN5C','ZSCAN5DP','ZSCAN9','ZUFSP','ZXDA','ZXDB','ZXDC','ZZZ3']
TFs4Other=['ATF2','ATF6B','ATN1','BHLHA15','BHLHB2','BHLHB4 ','BHLHB5','BHLHB9','C13ORF8','C20ORF194','CDX2','CRAMP1L','ELP3','ERVK3-1','ETF1','FLJ12895','FLJ20321','FLJ25476','FLJ35784','FOXM1','FOXO2','FOXR1','FOXR2','GATA1','GRLF1','KDM4C','KDM4E','KIAA0863','KIAA0961','KLRG1','LBXCOR1','LEF1','LHX7','LOC126295','LOC51058','LOC91661','MGC4728','MYOD1','MYOG','MYRF','NEUROD4','NEUROG1','NFAT2','NFAT4','NFE2L1','NR1A2','NR1B1','NR1B2','NR1B3','NR1C1','NR1C2','NR1C3','NR1F3','NR1I1','NR2A1','NR2B3','NR3A1','NR3A2','NR3B3','OTEX','PEPP-2','PHF2','POU2F1','POU2F2','POU4F3','POU5F1','POU6F1','PRDM3','PRKRIP1','PRKRIR','SETD1A','SMAD8','SMYD2','SOX12','SOX14','SOX15','TBX14','TCF1','TCF17','TCF2','TCF9','TEAD1','TPRXL','UBTFL1','WHSC1','ZBTB14','ZBTB18','ZBTB19','ZBTB21','ZBTB27','ZBTB28','ZBTB35','ZBTB49','ZFP161','ZFP28','ZKSCAN10','ZKSCAN11','ZKSCAN12','ZKSCAN13','ZKSCAN14','ZKSCAN16','ZKSCAN19','ZKSCAN21','ZKSCAN24','ZKSCAN6','ZKSCAN8','ZNF10','ZNF112','ZNF187','ZNF193','ZNF204P','ZNF238','ZNF271','ZNF295','ZNF305','ZNF323','ZNF434','ZNF499','ZNF542','ZNF545','ZNF553','ZNF56','ZNF625','ZNF643','ZNF698','ZNF705F','ZNF751','ZNF752','ZNF753','ZNF785','ZNF806','ZNF816','ZNF833P','ZNF906','ZSCAN1','ZSCAN25','ZSCAN26','ZSCAN31','ZSCAN32','ZSCAN5D','ZSCAN9']
TFsOther=TFs1Other+TFs2Other+TFs3Other+TFs4Other
```

Example 7 (unknown):
```unknown
S genes: ['MCM5', 'PCNA', 'TYMS', 'FEN1', 'MCM2', 'MCM4', 'RRM1', 'UNG', 'GINS2', 'MCM6', 'CDCA7', 'DTL', 'PRIM1', 'UHRF1', 'HELLS', 'RFC2', 'RPA2', 'NASP', 'RAD51AP1', 'GMNN', 'WDR76', 'SLBP', 'CCNE2', 'UBR7', 'POLD3', 'MSH2', 'ATAD2', 'RAD51', 'RRM2', 'CDC45', 'CDC6', 'EXO1', 'TIPIN', 'DSCC1', 'BLM', 'CASP8AP2', 'USP1', 'CLSPN', 'POLA1', 'CHAF1B', 'BRIP1', 'E2F8'] 

G2 genes: ['HMGB2', 'CDK1', 'NUSAP1', 'UBE2C', 'BIRC5', 'TPX2', 'TOP2A', 'NDC80', 'CKS2', 'NUF2', 'CKS1B', 'MKI67', 'TMPO', 'CENPF', 'TACC3', 'SMC4', 'CCNB2', 'CKAP2L', 'CKAP2', 'AURKB', 'BUB1', 'KIF11', 'ANP32E', 'TUBB4B', 'GTSE1', 'KIF20B', 'HJURP', 'CDCA3', 'CDC20', 'TTK', 'CDC25C', 'KIF2C', 'RANGAP1', 'NCAPD2', 'DLGAP5', 'CDCA2', 'CDCA8', 'ECT2', 'KIF23', 'HMMR', 'AURKA', 'PSRC1', 'ANLN', 'LBR', 'CKAP5', 'CENPE', 'CTCF', 'NEK2', 'G2E3', 'GAS2L3', 'CBX5', 'CENPA'] 

Hemoglobin genes set 1: ['HBB', 'HBD', 'HBG1', 'HBG2', 'HBE1', 'HBZ', 'HBM', 'HBA2', 'HBA1', 'HBQ1'] 

Hemoglobin genes set 2: ['AHSP', 'GYPA', 'RHAG', 'GFI1B', 'KEL', 'EPB42', 'ALAS2', 'TSPO2', 'GYPB', 'GYPE', 'SPTA1', 'RHCE', 'KLF1', 'RHD', 'SLC4A1', 'SNX22', 'SQSTM1', 'SOD1', 'FTH1', 'MAP1LC3B', 'SLC25A37', 'TRIM58', 'HMBS'] 

Mitochondrial genes: ['MT-ND1', 'MT-ND2', 'MT-CO1', 'MT-CO2', 'MT-ATP8', 'MT-ATP6', 'MT-CO3', 'MT-ND3', 'MT-ND4L', 'MT-ND4', 'MT-ND5', 'MT-ND6', 'MT-CYB'] 

Ribo genes: ['RPLP0', 'RPLP1', 'RPLP2', 'RPL3', 'RPL3L', 'RPL4', 'RPL5', 'RPL6', 'RPL7', 'RPL7A', 'RPL7L1', 'RPL8', 'RPL9', 'RPL10', 'RPL10A', 'RPL10L', 'RPL11', 'RPL12', 'RPL13', 'RPL13A', 'RPL14', 'RPL15', 'RPL17', 'RPL18', 'RPL18A', 'RPL19', 'RPL21', 'RPL22', 'RPL22L1', 'RPL23', 'RPL23A', 'RPL24', 'RPL26', 'RPL26L1', 'RPL27', 'RPL27A', 'RPL28', 'RPL29', 'RPL30', 'RPL31', 'RPL32', 'RPL34', 'RPL35', 'RPL35A', 'RPL36', 'RPL36A', 'RPL36AL', 'RPL37', 'RPL37A', 'RPL38', 'RPL39', 'RPL39L', 'UBA52', 'RPL41', 'RPSA', 'RPS2', 'RPS3', 'RPS3A', 'RPS4X', 'RPS4Y1', 'RPS4Y2', 'RPS5', 'RPS6', 'RPS7', 'RPS8', 'RPS9', 'RPS10', 'RPS11', 'RPS12', 'RPS13', 'RPS14', 'RPS15', 'RPS15A', 'RPS16', 'RPS17', 'RPS18', 'RPS19', 'RPS20', 'RPS21', 'RPS23', 'RPS24', 'RPS25', 'RPS26', 'RPS27', 'RPS27A', 'RPS27L', 'RPS28', 'RPS29', 'FAU'] 

Tissue dissociation stress associated genes: ['FOS', 'FOSB', 'JUN', 'JUNB', 'JUND', 'ZFP36', 'SOCS3', 'CEBPD', 'CEBPB', 'ATF3', 'EGR1', 'HSPA1A', 'HSPA1B', 'HSP90AA1', 'HSP90AB1', 'HSPA8', 'HSPB1', 'HSPE1', 'IER3', 'IER2', 'BTG1', 'BTG2', 'DUSP1']
```

Example 8 (python):
```python
#Define color palette
FineGrainPalette={
    'AtrialCardiomyocytesLeft':'#eac075',
    'AtrialCardiomyocytesRight':'#4c6b8a',
    'AtrialCardiomyocytesCycling':'#e94bc7',
    'VentricularCardiomyocytesLeftCompact':'#ab1c3c',
    'VentricularCardiomyocytesRightCompact':'#fdba45',
    'VentricularCardiomyocytesLeftTrabeculated':'#2b9200',
    'VentricularCardiomyocytesRightTrabeculated':'#0273a8',
    'VentricularCardiomyocytesCycling':'#e1acff',
    'SinoatrialNodeCardiomyocytes':'#854834',
    'AtrioventricularNodeCardiomyocytes':'#688500',
    'VentricularConductionSystemProximal':'#01ccc8',
    'VentricularConductionSystemDistal':'#b58fab',
    'GreatVesselAdventitialFibroblasts':'#ff7e97',
    'CoronaryVesselAdventitialFibroblasts':'#ff9b6e',
    'MyocardialInterstitialFibroblasts1':'#8a6800',
    'MyocardialInterstitialFibroblasts2':'#206601',
    'Myofibroblasts':'#1bdebf',
    'LymphNodeFibroblasticReticularCells':'#005aa0',
    'ValveInterstitialCells':'#e478ff',
    'GreatVesselSmoothMuscleCells':'#ff665f',
    'CoronarySmoothMuscleCells':'#d2cb18',
    'DuctusArteriosusSmoothMuscleCells':'#83d7b3',
    'CoronaryPericytes':'#9088ff',
    'PericardialCellsIntermediate':'#c95e00',
    'PericardialCellsFibrous':'#01e077',
    'PericardialCellsParietal':'#ff88e4',
    'GreatVesselArterialEndothelialCells':'#d4004b',
    'GreatVesselVenousEndothelialCells':'#8f4300',
    'CoronaryArterialEndothelialCells':'#89da5a',
    'CoronaryVenousEndothelialCells':'#006f86',
    'CoronaryCapillaryEndothelialCells':'#aa7dff',
    'EndocardialCells':'#ff8241',
    'EndocardialCushionCells':'#46603a',
    'ValveEndothelialCells':'#414eb6',
    'LymphaticEndothelialCells':'#8b3f60',
    'MesothelialEpicardialCells':'#0295fe',
    'EpicardiumDerivedCells':'#ff92c4',
    'NeuronPrecursors':'#d30038',
    'ChromaffinCells':'#726800',
    'SympatheticNeurons':'#01894b',
    'ParaympatheticNeurons':'#545191',
    'SchwannCellPrecursors':'#b0d276',
    'SchwannCells':'#b30079',
    'MonocytesMPOpos':'#c50066',
    'Monocytes':'#fab4b6',
    'MonocyteDerivedCells':'#d14008',
    'MacrophagesCX3CR1pos':'#eabe97',
    'MacrophagesTIMD4pos':'#b2a200',
    'MacrophagesLYVE1pos':'#4a6005',
    'MacrophagesATF3pos':'#3db833',
    'DendriticCellsType1':'#007859',
    'DendriticCellsMature':'#01a9db',
    'PlasmacytoidDendriticCells':'#2c54ab',
    'MastCells':'#d0bef2',
    'Megakaryocytes':'#94298f',
    'TCellsCD4pos':'#ff387c',
    'TCellsCD8pos':'#9e3429',
    'TregsCD4pos':'#b37700',
    'ProBCells':'#8dab00',
    'BCells':'#a8d1a1',
    'BCellsMS4A1pos':'#00d2f0',
    'NaturalKillerCells':'#abb4ff',
    'InnateLymphoidCells':'#72497f',
}
sc.pl.umap(adata, color='fine_grain',palette=FineGrainPalette)

MidGrainPalette={
    'AtrialCardiomyocytes':'#af3a4d',
    'VentricularCardiomyocytes':'#b1756c',
    'CardiacConductionSystem':'#d45633',
    'Fibroblasts':'#9b6e4e',
    'MuralCells':'#ffb92f',
    'PericardialCells':'#b4f03b',
    'BloodVesselEndothelialCells':'#72cc57',
    'EndocardialCells':'#436e43',
    'LymphaticEndothelialCells':'#85cca0',
    'EpicardialCells':'#5eabb0',
    'Neurons':'#7bbdff',
    'Glia':'#403c90',
    'MyeloidCells':'#420066',
    'LymphoidCells':'#f553f6',
}
sc.pl.umap(adata, color='mid_grain',palette=MidGrainPalette)

CoarseGrainPalette={
    'Cardiomyocytes':'#bc5950',
    'Mesenchymal':'#af9c4c',
    'Endothelium':'#599e58',
    'Epicardium':'#5eabb0',
    'Neural':'#5e75b1',
    'Leukocytes':'#9a52a0',
}
sc.pl.umap(adata, color='coarse_grain',palette=CoarseGrainPalette)

RegionGrainPalette={
    'aorta':'#ffa2bc',
    'apex':'#fe0049',
    'atria':'#732d00',
    'atrial septum':'#ffd54c',
    'base':'#837100',
    'ductus arteriosus':'#bdff8b',
    'whole sample':'#1b6c00',
    'heart without node':'#02e6c6',
    'left ventricle':'#01c1f5',
    'node':'#00466d',
    'outflow tract':'#020011',
    'pericardium':'#7a75ff',
    'pulmonary arches and branches':'#30003e',
    'right ventricle':'#ff88e1',
    'right ventricle and papillaries':'#98004f',
}
sc.pl.umap(adata, color='region',palette=RegionGrainPalette)
```

Example 9 (python):
```python
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
/tmp/ipykernel_16996/2162444827.py in <module>
     65     'InnateLymphoidCells':'#72497f',
     66 }
---> 67 sc.pl.umap(adata, color='fine_grain',palette=FineGrainPalette)
     68 
     69 MidGrainPalette={

~/miniconda3/envs/scanpynew/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py in umap(adata, **kwargs)
    657     tl.umap
    658     """
--> 659     return embedding(adata, 'umap', **kwargs)
    660 
    661 

~/miniconda3/envs/scanpynew/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py in embedding(adata, basis, color, gene_symbols, use_raw, sort_order, edges, edges_width, edges_color, neighbors_key, arrows, arrows_kwds, groups, components, layer, projection, scale_factor, color_map, cmap, palette, na_color, na_in_legend, size, frameon, legend_fontsize, legend_fontweight, legend_loc, legend_fontoutline, vmax, vmin, vcenter, norm, add_outline, outline_width, outline_color, ncols, hspace, wspace, title, show, save, ax, return_fig, **kwargs)
    250             groups=groups,
    251         )
--> 252         color_vector, categorical = _color_vector(
    253             adata,
    254             value_to_plot,

~/miniconda3/envs/scanpynew/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py in _color_vector(adata, values_key, values, palette, na_color)
   1267         color_map = {
   1268             k: to_hex(v)
-> 1269             for k, v in _get_palette(adata, values_key, palette=palette).items()
   1270         }
   1271         # If color_map does not have unique values, this can be slow as the

~/miniconda3/envs/scanpynew/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py in _get_palette(adata, values_key, palette)
   1233     values = pd.Categorical(adata.obs[values_key])
   1234     if palette:
-> 1235         _utils._set_colors_for_categorical_obs(adata, values_key, palette)
   1236     elif color_key not in adata.uns or len(adata.uns[color_key]) < len(
   1237         values.categories

~/miniconda3/envs/scanpynew/lib/python3.8/site-packages/scanpy/plotting/_utils.py in _set_colors_for_categorical_obs(adata, value_to_plot, palette)
    388         colors_list = [to_hex(x) for x in cmap(np.linspace(0, 1, len(categories)))]
    389     elif isinstance(palette, cabc.Mapping):
--> 390         colors_list = [to_hex(palette[k], keep_alpha=True) for k in categories]
    391     else:
    392         # check if palette is a list and convert it to a cycler, thus

~/miniconda3/envs/scanpynew/lib/python3.8/site-packages/scanpy/plotting/_utils.py in <listcomp>(.0)
    388         colors_list = [to_hex(x) for x in cmap(np.linspace(0, 1, len(categories)))]
    389     elif isinstance(palette, cabc.Mapping):
--> 390         colors_list = [to_hex(palette[k], keep_alpha=True) for k in categories]
    391     else:
    392         # check if palette is a list and convert it to a cycler, thus

KeyError: 'AtrioventricularNodePacemakerCells'
```

Example 10 (python):
```python
set(adata.obs.fine_grain)
```

Example 11 (unknown):
```unknown
{'AtrialCardiomyocytesCycling',
 'AtrialCardiomyocytesLeft',
 'AtrialCardiomyocytesRight',
 'AtrioventricularNodeCardiomyocytes',
 'BCells',
 'BCellsMS4A1pos',
 'ChromaffinCells',
 'CoronaryArterialEndothelialCells',
 'CoronaryCapillaryEndothelialCells',
 'CoronaryPericytes',
 'CoronarySmoothMuscleCells',
 'CoronaryVenousEndothelialCells',
 'CoronaryVesselAdventitialFibroblasts',
 'DendriticCellsMature',
 'DendriticCellsType1',
 'DuctusArteriosusSmoothMuscleCells',
 'EndocardialCells',
 'EndocardialCushionCells',
 'EpicardiumDerivedCells',
 'GreatVesselAdventitialFibroblasts',
 'GreatVesselArterialEndothelialCells',
 'GreatVesselSmoothMuscleCells',
 'GreatVesselVenousEndothelialCells',
 'InnateLymphoidCells',
 'LymphNodeFibroblasticReticularCells',
 'LymphaticEndothelialCells',
 'MacrophagesATF3pos',
 'MacrophagesCX3CR1pos',
 'MacrophagesLYVE1pos',
 'MacrophagesTIMD4pos',
 'MastCells',
 'Megakaryocytes',
 'MesothelialEpicardialCells',
 'MonocyteDerivedCells',
 'Monocytes',
 'MonocytesMPOpos',
 'MyocardialInterstitialFibroblasts1',
 'MyocardialInterstitialFibroblasts2',
 'Myofibroblasts',
 'NaturalKillerCells',
 'NeuronPrecursors',
 'ParaympatheticNeurons',
 'PericardialCellsFibrous',
 'PericardialCellsIntermediate',
 'PericardialCellsParietal',
 'PlasmacytoidDendriticCells',
 'ProBCells',
 'SchwannCellPrecursors',
 'SchwannCells',
 'SinoatrialNodeCardiomyocytes',
 'SympatheticNeurons',
 'TCellsCD4pos',
 'TCellsCD8pos',
 'TregsCD4pos',
 'ValveEndothelialCells',
 'ValveInterstitialCells',
 'VentricularCardiomyocytesCycling',
 'VentricularCardiomyocytesLeftCompact',
 'VentricularCardiomyocytesLeftTrabeculated',
 'VentricularCardiomyocytesRightCompact',
 'VentricularCardiomyocytesRightTrabeculated',
 'VentricularConductionSystemDistal',
 'VentricularConductionSystemProximal'}
```

Example 12 (python):
```python
set(adata.obs.mid_grain)
```

Example 13 (unknown):
```unknown
{'AtrialCardiomyocytes',
 'BloodVesselEndothelialCells',
 'CardiacConductionSystem',
 'EndocardialCells',
 'EpicardialCells',
 'Fibroblasts',
 'Glia',
 'LymphaticEndothelialCells',
 'LymphoidCells',
 'MuralCells',
 'MyeloidCells',
 'Neurons',
 'PericardialCells',
 'VentricularCardiomyocytes'}
```

Example 14 (python):
```python
set(adata.obs.coarse_grain)
```

Example 15 (unknown):
```unknown
{'Cardiomyocytes',
 'Endothelium',
 'Epicardium',
 'Leukocytes',
 'Mesenchymal',
 'Neural'}
```

Example 16 (python):
```python
#adata = adata[adata.obs['fine_grain'].isin(['CoronaryVesselAdventitialFibroblasts','MyocardialInterstitialFibroblasts1','MyocardialInterstitialFibroblasts2','Myofibroblasts','GreatVesselAdventitialFibroblasts'])].copy()
#adata = adata[adata.obs['fine_grain'].isin(['CoronaryArterialEndothelialCells','CoronaryCapillaryEndothelialCells','CoronaryVenousEndothelialCells','GreatVesselArterialEndothelialCells','GreatVesselVenousEndothelialCells'])].copy()
adata = adata[adata.obs['fine_grain'].isin(['EndocardialCells','EndocardialCushionCells','ValveEndothelialCells','ValveInterstitialCells'])].copy()
#adata = adata[adata.obs['mid_grain'].isin(['PericardialCells'])].copy()
#adata=adata[adata.obs.mid_grain=='MuralCells',:].copy()
```

Example 17 (unknown):
```unknown
AnnData object with n_obs × n_vars = 16744 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score'
    var: 'gene_name_scRNA-0', 'gene_id'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'age_colors', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'cycling_colors', 'donor_colors', 'fine_grain_colors', 'heart_or_greatvessels_colors', 'hvg', 'kit_10x_colors', 'mid_grain_colors', 'modality_colors', 'neighbors', 'phase_colors', 'region_colors', 'sex_colors', 'trimester_colors', 'umap', 'week_colors'
    obsm: 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 18 (python):
```python
# preserve raw count
adata.layers["counts"] = adata.X.copy()

# normalise and log-transform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key='batch_key')
#sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5)

print(sum(adata.var.highly_variable))
sc.pl.highly_variable_genes(adata)

# freeze the state in `.raw`
adata.raw = adata

# subset hvgs
adata = adata[:, adata.var.highly_variable]

adata=adata.copy()

sc.tl.score_genes_cell_cycle(adata, s_genes=s_genes, g2m_genes=g2m_genes)
#sc.pp.regress_out(adata, ['S_score','G2M_score'])

#batch correction
scvi.model.SCVI.setup_anndata(adata, 
                              layer="counts", 
                              batch_key="batch_key",
                              categorical_covariate_keys=['phase'],
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo'])

vae = scvi.model.SCVI(adata, n_hidden = 128, n_latent = 50, n_layers = 3, dispersion = 'gene-batch')
vae.train(max_epochs=50)
    
# Convergence plot
x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
plt.plot(x, vae.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
plt.clf()

# add the latent space to adata
adata.obsm["X_scVI"] = vae.get_latent_representation()

sc.tl.score_genes(adata, stress_genes, score_name='stress_score')
sc.tl.score_genes(adata, hb_genes1, score_name='hb1_score')

sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata,min_dist=0.1)
#sc.tl.tsne(adata)
sc.pl.umap(adata, color=['log1p_total_counts','n_genes','n_genes_by_counts','total_counts','pct_counts_mt','pct_counts_ribo'],ncols=5)
sc.pl.umap(adata, color=['scrublet_score','stress_score', 'hb1_score','phase'], ncols=5,)

sc.pl.umap(adata, color=['donor','age','week','trimester','region','phase','cycling','cell_or_nuclei','modality','kit_10x','FACSgate','heart_or_greatvessels','coarse_grain','mid_grain','fine_grain'],wspace=0.8,ncols=5)
Genes=['TTN','TNNT2','MYOM1','LDB3','ALDH1A2','TNNT1','KRT19','UPK3B','BNC1','COL6A3','FBLN1','PCOLCE','LRRC17','PCDH18','EMCN','ECSCR','PECAM1','CDH5','ESAM', 'LAPTM5','PTPRC','LCP1','CD53','WAS','PHOX2B','NRXN1','STMN2','GRIK2','L1CAM']
sc.pl.umap(adata, color=Genes)
```

Example 19 (python):
```python
normalizing counts per cell
    finished (0:00:00)
extracting highly variable genes
    finished (0:00:10)
--> added
    'highly_variable', boolean vector (adata.var)
    'means', float vector (adata.var)
    'dispersions', float vector (adata.var)
    'dispersions_norm', float vector (adata.var)
2188
```

Example 20 (python):
```python
calculating cell cycle phase
computing score 'S_score'
WARNING: genes are not in var_names and ignored: ['MCM5', 'FEN1', 'MCM2', 'MCM4', 'RRM1', 'UNG', 'GINS2', 'MCM6', 'PRIM1', 'RFC2', 'RPA2', 'NASP', 'RAD51AP1', 'GMNN', 'WDR76', 'SLBP', 'CCNE2', 'UBR7', 'POLD3', 'MSH2', 'RAD51', 'CDC45', 'CDC6', 'EXO1', 'TIPIN', 'DSCC1', 'BLM', 'CASP8AP2', 'USP1', 'POLA1', 'CHAF1B', 'E2F8']
    finished: added
    'S_score', score of gene set (adata.obs).
    330 total control genes are used. (0:00:00)
computing score 'G2M_score'
WARNING: genes are not in var_names and ignored: ['CKS1B', 'TMPO', 'ANP32E', 'HJURP', 'CDC25C', 'RANGAP1', 'NCAPD2', 'LBR', 'CKAP5', 'CTCF', 'G2E3', 'CBX5']
    finished: added
    'G2M_score', score of gene set (adata.obs).
    695 total control genes are used. (0:00:00)
-->     'phase', cell cycle phase (adata.obs)
```

Example 21 (unknown):
```unknown
GPU available: False, used: False
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
```

Example 22 (unknown):
```unknown
Epoch 50/50: 100%|███████████| 50/50 [01:21<00:00,  1.63s/it, loss=945, v_num=1]
```

Example 23 (python):
```python
computing score 'stress_score'
WARNING: genes are not in var_names and ignored: ['JUND', 'CEBPB', 'HSP90AA1', 'HSP90AB1', 'HSPA8', 'HSPB1', 'HSPE1', 'BTG1', 'BTG2']
    finished: added
    'stress_score', score of gene set (adata.obs).
    442 total control genes are used. (0:00:00)
computing score 'hb1_score'
WARNING: genes are not in var_names and ignored: ['HBB', 'HBD', 'HBE1', 'HBZ', 'HBM', 'HBA1', 'HBQ1']
    finished: added
    'hb1_score', score of gene set (adata.obs).
    98 total control genes are used. (0:00:00)
computing neighbors
    finished: added to `.uns['neighbors']`
    `.obsp['distances']`, distances for each pair of neighbors
    `.obsp['connectivities']`, weighted adjacency matrix (0:00:03)
computing UMAP
    finished: added
    'X_umap', UMAP coordinates (adata.obsm) (0:00:04)
```

Example 24 (unknown):
```unknown
... storing 'phase' as categorical
```

Example 25 (unknown):
```unknown
<Figure size 320x320 with 0 Axes>
```

Example 26 (python):
```python
sc.tl.diffmap(adata)
sc.pl.diffmap(adata, color='fine_grain')
sc.pl.diffmap(adata, color='PECAM1')
```

Example 27 (python):
```python
computing Diffusion Maps using n_comps=15(=n_dcs)
computing transitions
    finished (0:00:00)
    eigenvalues of transition matrix
    [1.         0.99786    0.99074847 0.9877497  0.98542356 0.9823111
     0.98121595 0.9771292  0.9763206  0.97314173 0.97228724 0.96928436
     0.9683964  0.9643321  0.96360725]
    finished: added
    'X_diffmap', diffmap coordinates (adata.obsm)
    'diffmap_evals', eigenvalues of transition matrix (adata.uns) (0:00:00)
```

Example 28 (python):
```python
sc.pl.umap(adata, color='fine_grain')
```

Example 29 (python):
```python
set(adata.obs.fine_grain)
```

Example 30 (unknown):
```unknown
{'EndocardialCells',
 'EndocardialCushionCells',
 'ValveEndothelialCells',
 'ValveInterstitialCells'}
```

Example 31 (python):
```python
ClusterOfInterest='EndocardialCells'
#sc.tl.rank_genes_groups(adata, 'fine_grain', method='t-test', key_added = "t-test")
DEGenesForCluster = [x[ClusterOfInterest] for x in adata.uns['t-test']['names'][:50]]
sc.pl.dotplot(adata, DEGenesForCluster , groupby='fine_grain',standard_scale='var',cmap='PRGn')
#sc.pl.umap(adata, color=DEGenesForCluster)
```

Example 32 (python):
```python
sc.pl.dotplot(adata, ['EMCN','PCDH7','ADGRG6','NPR3','SMOC1','HAS2','PROX1','COL26A1','BMP4','GJA4','TMEM132D','TSPAN8','CNR1','MSX1','APCDD1','HAPLN1','KLF2','FXYD5','LTC4S','CCL21','NFATC1','NRGN','DKK2','KCNJ2','SULF1','GJA1','KISS1','PTN','MFAP4','ID4','PRRX1','BAMBI','INHBA','CLU','TWIST1','COL9A3','COL9A2','IGF1','RSPO3','MMP23B','SCX','COL9A1'] ,var_group_positions=[(0,4),(5,13),(14,26),(27,41)], var_group_labels=['endocardial','endocardial cushion','valve endothelial','valve interstitial'], var_group_rotation=90,groupby='fine_grain',standard_scale='var',cmap='YlGn')
```

Example 33 (python):
```python
sc.pl.dotplot(adata, ['EMCN','PCDH7','ADGRG6','NPR3','SMOC1','HAS2','PROX1','COL26A1','BMP4','GJA4','TMEM132D','TSPAN8','CNR1','MSX1','APCDD1','HAPLN1','KLF2','FXYD5','LTC4S','CCL21','NFATC1','NRGN','DKK2','KCNJ2','SULF1','GJA1','KISS1','PTN','MFAP4','ID4','PRRX1','BAMBI','INHBA','CLU','TWIST1','COL9A3','COL9A2','IGF1','RSPO3','MMP23B','SCX','COL9A1','POSTN'] ,var_group_positions=[(0,4),(5,13),(14,26),(27,41)], var_group_labels=['endocardial','endocardial cushion','valve endothelial','valve interstitial'], var_group_rotation=90,groupby='fine_grain',standard_scale='var',cmap='YlGn')
```

Example 34 (python):
```python
bdata=adata.copy()
```

Example 35 (python):
```python
bdata=sc.read('./Objects/Feb28ObjectProcessed.h5ad')
print(bdata.shape)
bdata = bdata[bdata.obs['mid_grain'].isin(['Fibroblasts'])].copy()
```

Example 36 (unknown):
```unknown
(297473, 3392)
```

Example 37 (python):
```python
sc.pl.dotplot(bdata, ['CLU','HAPLN1','INHBA','COL9A2','SCX'] ,groupby='fine_grain',standard_scale='var',cmap='YlGn')
```

Example 38 (unknown):
```unknown
FineGrainCategoriesOrder=['AtrialCardiomyocytesLeft','AtrialCardiomyocytesRight','AtrialCardiomyocytesCycling','VentricularCardiomyocytesLeftCompact','VentricularCardiomyocytesRightCompact','VentricularCardiomyocytesLeftTrabeculated','VentricularCardiomyocytesRightTrabeculated','VentricularCardiomyocytesCycling','SinoatrialNodeCardiomyocytes','AtrioventricularNodeCardiomyocytes','VentricularConductionSystemProximal','VentricularConductionSystemDistal','GreatVesselAdventitialFibroblasts','CoronaryVesselAdventitialFibroblasts','MyocardialInterstitialFibroblasts1','MyocardialInterstitialFibroblasts2','Myofibroblasts','LymphNodeFibroblasticReticularCells','ValveInterstitialCells','GreatVesselSmoothMuscleCells','CoronarySmoothMuscleCells','DuctusArteriosusSmoothMuscleCells','CoronaryPericytes','PericardialCellsIntermediate','PericardialCellsFibrous','PericardialCellsParietal','GreatVesselArterialEndothelialCells','GreatVesselVenousEndothelialCells','CoronaryArterialEndothelialCells','CoronaryVenousEndothelialCells','CoronaryCapillaryEndothelialCells','EndocardialCells','EndocardialCushionCells','ValveEndothelialCells','LymphaticEndothelialCells','MesothelialEpicardialCells','EpicardiumDerivedCells','NeuronPrecursors','ChromaffinCells','SympatheticNeurons','ParaympatheticNeurons','SchwannCellPrecursors','SchwannCells','MonocytesMPOpos','Monocytes','MonocyteDerivedCells','MacrophagesCX3CR1pos','MacrophagesTIMD4pos','MacrophagesLYVE1pos','MacrophagesATF3pos','DendriticCellsType1','DendriticCellsMature','PlasmacytoidDendriticCells','MastCells','Megakaryocytes','TCellsCD4pos','TCellsCD8pos','TregsCD4pos','ProBCells','BCells','BCellsMS4A1pos','NaturalKillerCells','InnateLymphoidCells']
```

---

## Variables¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/Xenium/no2_integration_and_cell-type-annotation/no6_manual-annotation_myeloid.html

**Contents:**
- Variables¶
- Read in data¶
- 1st round annotation¶
- Subset and reclustering¶
- Transfer annotation to the original adata¶
- Save¶

--> cluster 2 has lower total counts and smaller cell area, and no specific gene expression --> annotate as "unassigned"

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os

import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 

import scanpy.external as sce
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/session_info/main.py:213: UserWarning: The '__version__' attribute is deprecated and will be removed in MarkupSafe 3.1. Use feature detection, or `importlib.metadata.version("markupsafe")`, instead.
  mod_version = _find_version(mod.__version__)
```

Example 4 (python):
```python
-----
matplotlib          3.10.1
numpy               2.1.3
pandas              2.2.3
scanpy              1.11.1
seaborn             0.13.2
session_info        v1.0.1
-----
```

Example 5 (python):
```python
Cython              3.1.1
PIL                 11.1.0
anndata             0.11.4
asciitree           NA
asttokens           NA
charset_normalizer  3.4.1
cloudpickle         3.1.1
comm                0.2.2
cycler              0.12.1
cython              3.1.1
cython_runtime      NA
dask                2024.11.2
dateutil            2.9.0.post0
debugpy             1.8.13
decorator           5.2.1
exceptiongroup      1.2.2
executing           2.2.0
h5py                3.13.0
igraph              0.11.8
importlib_metadata  NA
ipykernel           6.29.5
jedi                0.19.2
jinja2              3.1.6
joblib              1.4.2
kiwisolver          1.4.8
legacy_api_wrap     NA
leidenalg           0.10.2
llvmlite            0.44.0
markupsafe          3.0.2
mpl_toolkits        NA
msgpack             1.1.0
natsort             8.4.0
numba               0.61.0
numcodecs           0.13.1
packaging           24.2
parso               0.8.4
patsy               1.0.1
platformdirs        4.3.7
prompt_toolkit      3.0.50
psutil              7.0.0
pure_eval           0.2.3
pyarrow             19.0.1
pydev_ipython       NA
pydevconsole        NA
pydevd              3.2.3
pydevd_file_utils   NA
pydevd_plugins      NA
pydevd_tracing      NA
pygments            2.19.1
pyparsing           3.2.3
pytz                2025.2
scipy               1.15.2
six                 1.17.0
sklearn             1.5.2
stack_data          0.6.3
statsmodels         0.14.4
tblib               3.1.0
texttable           1.7.0
threadpoolctl       3.6.0
tlz                 1.0.0
toolz               1.0.0
tornado             6.4.2
traitlets           5.14.3
typing_extensions   NA
wcwidth             0.2.13
yaml                6.0.2
zarr                2.18.3
zipp                NA
zmq                 26.3.0
zoneinfo            NA
```

Example 6 (unknown):
```unknown
-----
IPython             8.34.0
jupyter_client      8.6.3
jupyter_core        5.7.2
-----
Python 3.10.14 | packaged by conda-forge | (main, Mar 20 2024, 12:45:18) [GCC 12.3.0]
Linux-4.18.0-553.53.1.el8_10.x86_64-x86_64-with-glibc2.28
-----
Session information updated at 2025-06-12 17:17
```

Example 7 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/home/kk837/rfs/rfs-teichlab-nfs-iCNyzSAaucw/kk837/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 8 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 9 (python):
```python
adata_dir = '/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium/subsets'
sample_id = 'C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat'
celltype = 'MyeloidCells'
```

Example 10 (unknown):
```unknown
# figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk/Aug2024'
# table_dir = '/nfs/team205/heart/fetal_ms_figures/tables/Aug2024'
```

Example 11 (unknown):
```unknown
latent_space = 'pca_harmony'
```

Example 12 (python):
```python
# read in
adata = sc.read_h5ad(f'{adata_dir}/{sample_id}_5K_{celltype}_lognorm.h5ad')
adata
```

Example 13 (unknown):
```unknown
AnnData object with n_obs × n_vars = 3965 × 4505
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_3.0', 'multi_celltypes_coarse', 'leiden_0.2', 'leiden_0.4', 'leiden_0.6', 'leiden_1'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1', 'n_cells', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'dendrogram_pca_harmony_leiden_1.0', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1_colors', 'log1p', 'multi_celltypes_coarse_colors', 'pca', 'pca_harmony', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.2_colors', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.3_colors', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.4_colors', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.5_colors', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_0.8_colors', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.0_colors', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_1.5_colors', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_2.0_colors', 'pca_harmony_leiden_3.0', 'pca_harmony_leiden_3.0_colors', 'tissue_block_id_colors', 'umap'
    obsm: 'X_pca', 'X_pca_harmony', 'X_umap_pca_harmony', 'spatial'
    varm: 'PCs'
    obsp: 'pca_harmony_connectivities', 'pca_harmony_distances'
```

Example 14 (python):
```python
adata.X.data[:5]
```

Example 15 (unknown):
```unknown
array([2.0822465, 2.0822465, 2.0822465, 2.0822465, 2.0822465],
      dtype=float32)
```

Example 16 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=['total_counts','n_genes_by_counts','cell_area','tissue_block_id'],
                    wspace=0.2,cmap='RdPu',vmax='p100')
```

Example 17 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=['multi_celltypes_coarse',"conf_score_midmod2fine","celltypist_midmod2fine",],
                    wspace=0.2,cmap='RdPu',vmax='p100')
resolutions_list = [0.2,0.4,0.6,1]
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                    color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                    wspace=0.2)
```

Example 18 (python):
```python
%%time
# leiden clustering
resolutions_list = [# 0.2,0.4,0.6,1, # already done
                    1.2,1.5,2]
for resolution in resolutions_list:
    sc.tl.leiden(adata,neighbors_key=latent_space,
                     resolution=resolution,key_added=f'leiden_{str(resolution)}',n_iterations=2)
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                wspace=0.3)
```

Example 19 (unknown):
```unknown
<timed exec>:5: FutureWarning: In the future, the default backend for leiden will be igraph instead of leidenalg.

 To achieve the future defaults please pass: flavor="igraph" and n_iterations=2.  directed must also be False to work with igraph's implementation.
```

Example 20 (unknown):
```unknown
CPU times: user 2.62 s, sys: 71.3 ms, total: 2.69 s
Wall time: 2.7 s
```

Example 21 (python):
```python
markers={
    'Mono':['CCR2','FCN1','VCAN','MPO'],
    'MP':['C1QA','C1QB','CD68','MRC1','CD163','MERTK'],
    'CX3CR1+':['CX3CR1','C3','TREM2','P2RY12'],
    'LYVE1+':['LYVE1','FOLR2'],
     'pDC':['IRF8','IRF4','IL3RA'],
     'DC_mature':['FLT3', 'CCR7','LAMP3', 'IL15', 'CD83', 'LY75', ],
    'Megakaryocyte':['ITGA2B','MPL','ITGB3',],
    'Mast': ['KIT', 'CPA3','HDC','GATA2'],
    'Cycling':['MKI67','TOP2A'],
}
genes = []
for g in markers.values():
    genes = genes + [x for x in g if (x in adata.var_names)&(x not in genes)]
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=genes,
               ncols=5, wspace=0.2,cmap='RdPu',vmax='p100')
```

Example 22 (unknown):
```unknown
resolution_sel = 1.5
```

Example 23 (python):
```python
markers={
    'Mono':['CCR2','FCN1','VCAN','MPO'],
    'MP':['C1QA','C1QB','CD68','MRC1','CD163','MERTK'],
    'CX3CR1+':['CX3CR1','C3','TREM2','P2RY12'],
    'LYVE1+':['LYVE1','FOLR2'],
     'pDC':['IRF8','IRF4','IL3RA'],
     'DC_mature':['FLT3', 'CCR7','LAMP3', 'IL15', 'CD83', 'LY75', ],
    'Megakaryocyte':['ITGA2B','MPL','ITGB3',],
    'Mast': ['KIT', 'CPA3','HDC','GATA2'],
    'Cycling':['MKI67','TOP2A'],
}
for k,g in markers.items():
    markers[k] = [x for x in g if x in adata.var_names]

sc.tl.dendrogram(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.pl.dotplot(adata,
              var_names=markers,
              groupby=f'leiden_{str(resolution_sel)}',
              standard_scale='var',
              dendrogram=True
             )
```

Example 24 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: Mono, MP, CX3CR1+, etc.
```

Example 25 (python):
```python
markers = {
    'CM': ['TTN', 'TNNT2','MYH6',
        'MYH7', 'MYL2', 'FHL2', # vCN
            'NPPA', 'MYL7', 'MYL4'], #aCM
    'FB': ['DCN', 'GCN', 'PDGFRA','COL1A1','COL1A2'],
    'EC': ['VWF', 'PECAM1', 'CDH5','RGCC', 'FABP5',],
    'Peri': ['RGS5', 'ABCC9', 'KCNJ8'],
    'SMC': ['MYH11', 'TAGLN', 'ACTA2'],
    'Meso': ['MSLN', 'WT1', 'BNC1'],
    'Neuro': ['PLP1', 'NRXN1', 'NRXN3','PRPH', 'NEFL', 'NEFM', 'NEFH', 'STMN2', 'SST', 'SLC10A4', 'SLC18A3', 'TH', 'NPY'],
    'Adipo': ['GPAM', 'FASN', 'LEP'],
    'Myelo': ['CD14', 'C1QA', 'CD68','LYVE1','TIMD4'],
    'Lymph': ['CD8A', 'IL7R', 'CD40LG'],
    'B': ['CD19','SDC1','IGHG1','IGHA1'],
    'Mast': ['KIT', 'CPA3'],
}

for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in adata.var_names]

sc.tl.dendrogram(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.pl.dotplot(adata,
              var_names=markers,
              groupby=f'leiden_{str(resolution_sel)}',
              standard_scale='var',
              dendrogram=True
             )
```

Example 26 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: CM, FB, EC, etc.
```

Example 27 (python):
```python
print(adata.X.data[:5])
adata.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata, dendrogram=True, standard_scale='var', return_fig=True)
dp.add_totals().show()
```

Example 28 (unknown):
```unknown
[2.0822465 2.0822465 2.0822465 2.0822465 2.0822465]
```

Example 29 (python):
```python
sc.pl.violin(adata,keys=['total_counts','cell_area'],groupby=f'leiden_{str(resolution_sel)}')
```

Example 30 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=f'leiden_{str(resolution_sel)}',legend_loc='on data',
                )
```

Example 31 (python):
```python
# annotate
adata.obs['tmp_annotation'] = adata.obs[f'leiden_{str(resolution_sel)}'].copy()
adata.obs.replace({'tmp_annotation':{
    '0':'Macrophages',
    '1':'Macrophages',
    '2':'unassigned',
    '3':'Macrophages',
    '4':'Macrophages',
    '5':'Macrophages',
    '6':'Erythrocytes',
    '7':'Macrophages',
    '8':'Monocytes',
    '9':'Macrophages',
    '10':'MastCells',
    '11':'Macrophages',
    '12':'MyeloidCells_Doublets',
    '13':'ErythrocytesCycling',
    '14':'Macrophages',
    '15':'Megakaryocytes',
}},inplace=True)
adata.obs['tmp_annotation'].value_counts()
```

Example 32 (python):
```python
/tmp/ipykernel_285284/3101446981.py:3: FutureWarning: The behavior of Series.replace (and DataFrame.replace) with CategoricalDtype is deprecated. In a future version, replace will only be used for cases that preserve the categories. To change the categories, use ser.cat.rename_categories instead.
  adata.obs.replace({'tmp_annotation':{
```

Example 33 (unknown):
```unknown
tmp_annotation
Macrophages              2727
unassigned                522
Erythrocytes              231
Monocytes                 220
MastCells                  91
MyeloidCells_Doublets      67
ErythrocytesCycling        65
Megakaryocytes             42
Name: count, dtype: int64
```

Example 34 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color='tmp_annotation'
                )
```

Example 35 (python):
```python
adata_sub = adata[adata.obs['tmp_annotation'].isin(['Monocytes','Macrophages'])]
```

Example 36 (python):
```python
# drop leiden clustering results
resolutions_list = [0.2,0.4,0.6,1, 1.2,1.5,2]
for resolution in resolutions_list:
    adata_sub.obs.drop(f'leiden_{str(resolution)}',axis=1,inplace=True)
adata_sub
```

Example 37 (unknown):
```unknown
AnnData object with n_obs × n_vars = 2947 × 4505
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_3.0', 'multi_celltypes_coarse', 'tmp_annotation'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1', 'n_cells', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'dendrogram_pca_harmony_leiden_1.0', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1_colors', 'log1p', 'multi_celltypes_coarse_colors', 'pca', 'pca_harmony', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.2_colors', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.3_colors', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.4_colors', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.5_colors', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_0.8_colors', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.0_colors', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_1.5_colors', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_2.0_colors', 'pca_harmony_leiden_3.0', 'pca_harmony_leiden_3.0_colors', 'tissue_block_id_colors', 'umap', 'leiden_1.2', 'leiden_1.5', 'leiden_2', 'leiden_1.2_colors', 'leiden_1.5_colors', 'leiden_2_colors', 'dendrogram_leiden_1.5', 'rank_genes_groups', 'tmp_annotation_colors'
    obsm: 'X_pca', 'X_pca_harmony', 'X_umap_pca_harmony', 'spatial'
    varm: 'PCs'
    obsp: 'pca_harmony_connectivities', 'pca_harmony_distances'
```

Example 38 (python):
```python
## Remove mid-grain markers from the feature
# read in markers
midgrain_markers = pd.read_csv('/home/kk837/rfs/rfs-teichlab-nfs-iCNyzSAaucw/kk837/notebooks/Foetal/VisiumHD/re-seq_mpp0p3/sc_midgrain_markers.csv',
                               index_col=0)
# remove the markers of celltype of interest
midgrain_markers = midgrain_markers[midgrain_markers['group']!=celltype]
# remove
mask = [x not in list(midgrain_markers['names']) for x in adata_sub.var_names]
bdata = adata_sub[:,mask]
bdata
```

Example 39 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 2947 × 4059
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_3.0', 'multi_celltypes_coarse', 'tmp_annotation'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1', 'n_cells', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'dendrogram_pca_harmony_leiden_1.0', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1_colors', 'log1p', 'multi_celltypes_coarse_colors', 'pca', 'pca_harmony', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.2_colors', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.3_colors', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.4_colors', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.5_colors', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_0.8_colors', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.0_colors', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_1.5_colors', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_2.0_colors', 'pca_harmony_leiden_3.0', 'pca_harmony_leiden_3.0_colors', 'tissue_block_id_colors', 'umap', 'leiden_1.2', 'leiden_1.5', 'leiden_2', 'leiden_1.2_colors', 'leiden_1.5_colors', 'leiden_2_colors', 'dendrogram_leiden_1.5', 'rank_genes_groups', 'tmp_annotation_colors'
    obsm: 'X_pca', 'X_pca_harmony', 'X_umap_pca_harmony', 'spatial'
    varm: 'PCs'
    obsp: 'pca_harmony_connectivities', 'pca_harmony_distances'
```

Example 40 (python):
```python
# remove cell cycle genes
cell_cycle_genes = ['MCM5','PCNA','TYMS','FEN1','MCM2','MCM4','RRM1','UNG','GINS2','MCM6','CDCA7','DTL','PRIM1','UHRF1','MLF1IP','HELLS','RFC2','RPA2','NASP',
                'RAD51AP1','GMNN','WDR76','SLBP','CCNE2','UBR7','POLD3','MSH2','ATAD2','RAD51','RRM2','CDC45','CDC6','EXO1','TIPIN','DSCC1','BLM','CASP8AP2',
                'USP1','CLSPN','POLA1','CHAF1B','BRIP1','E2F8','HMGB2','CDK1','NUSAP1','UBE2C','BIRC5','TPX2','TOP2A','NDC80','CKS2','NUF2','CKS1B','MKI67',
                'TMPO','CENPF','TACC3','FAM64A','SMC4','CCNB2','CKAP2L','CKAP2','AURKB','BUB1','KIF11','ANP32E','TUBB4B','GTSE1','KIF20B','HJURP','CDCA3','HN1',
                'CDC20','TTK','CDC25C','KIF2C','RANGAP1','NCAPD2','DLGAP5','CDCA2','CDCA8','ECT2','KIF23','HMMR','AURKA','PSRC1','ANLN','LBR','CKAP5','CENPE','CTCF',
                'NEK2','G2E3','GAS2L3','CBX5','CENPA']
mask = [x not in cell_cycle_genes for x in bdata.var_names]
bdata = bdata[:,mask]
bdata.shape
```

Example 41 (unknown):
```unknown
(2947, 4006)
```

Example 42 (python):
```python
# HVGs, using the object which doesn't have midgrain markers
sc.pp.highly_variable_genes(bdata, flavor='seurat', min_mean=0.0125, max_mean=3, min_disp=0.5)
print(f'number of hvgs: {sum(bdata.var.highly_variable)}')
# PCA using the degected HVGs
sc.tl.pca(bdata, use_highly_variable=True)
adata_sub.obsm['X_pca'] = bdata.obsm['X_pca'].copy()
# harmony integration
sce.pp.harmony_integrate(adata_sub, 'tissue_block_id', basis='X_pca', adjusted_basis='X_pca_harmony')
# create neighbour graph
latent_space = 'pca_harmony'
sc.pp.neighbors(adata_sub,n_neighbors=15,use_rep=f"X_{latent_space}",key_added=latent_space)
# visualisation with umap
sc.tl.umap(adata_sub,min_dist=0.5,neighbors_key=latent_space)
adata_sub.obsm[f"X_umap_{latent_space}"]=adata_sub.obsm["X_umap"]
del adata_sub.obsm["X_umap"]
```

Example 43 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_highly_variable_genes.py:698: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  adata.uns["hvg"] = {"flavor": flavor}
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_pca/__init__.py:438: FutureWarning: Argument `use_highly_variable` is deprecated, consider using the mask argument. Use_highly_variable=True can be called through mask_var="highly_variable". Use_highly_variable=False can be called through mask_var=None
  warn(msg, FutureWarning)
```

Example 44 (unknown):
```unknown
number of hvgs: 742
```

Example 45 (python):
```python
2025-06-12 17:24:35,549 - harmonypy - INFO - Computing initial centroids with sklearn.KMeans...
2025-06-12 17:24:37,152 - harmonypy - INFO - sklearn.KMeans initialization complete.
2025-06-12 17:24:37,207 - harmonypy - INFO - Iteration 1 of 10
2025-06-12 17:24:37,480 - harmonypy - INFO - Iteration 2 of 10
2025-06-12 17:24:37,754 - harmonypy - INFO - Converged after 2 iterations
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm
```

Example 46 (python):
```python
%%time
# reclustering
# leiden clustering
resolutions_list = [0.2,0.4,0.6,1, 1.2,1.5,2]
for resolution in resolutions_list:
    sc.tl.leiden(adata_sub,neighbors_key=latent_space,
                 resolution=resolution,key_added=f'leiden_{str(resolution)}',n_iterations=2)
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                wspace=0.3)
```

Example 47 (unknown):
```unknown
CPU times: user 4.19 s, sys: 134 ms, total: 4.32 s
Wall time: 4.34 s
```

Example 48 (python):
```python
markers={
    'Mono':['CCR2','FCN1','VCAN','MPO'],
    'MP':['C1QA','C1QB','CD68','MRC1','CD163','MERTK'],
    'CX3CR1+':['CX3CR1','C3','TREM2','P2RY12'],
    'LYVE1+':['LYVE1','FOLR2'],
     'pDC':['IRF8','IRF4','IL3RA'],
     'DC_mature':['FLT3', 'CCR7','LAMP3', 'IL15', 'CD83', 'LY75', ],
    'Cycling':['MKI67','TOP2A'],
}
genes = []
for g in markers.values():
    genes = genes + [x for x in g if (x in adata.var_names)&(x not in genes)]
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color=genes,
               ncols=5, wspace=0.2,cmap='RdPu',vmax='p100')
```

Example 49 (unknown):
```unknown
sub_resolution_sel = 1
```

Example 50 (python):
```python
markers={
    'Mono':['CCR2','FCN1','VCAN','MPO'],
    'MP':['C1QA','C1QB','CD68','MRC1','CD163','MERTK'],
    'CX3CR1+':['CX3CR1','C3','TREM2','P2RY12'],
    'LYVE1+':['LYVE1','FOLR2'],
     'pDC':['IRF8','IRF4','IL3RA'],
     'DC_mature':['FLT3', 'CCR7','LAMP3', 'IL15', 'CD83', 'LY75', ],
    'Cycling':['MKI67','TOP2A'],
}
for k,g in markers.items():
    markers[k] = [x for x in g if x in adata_sub.var_names]

sc.tl.dendrogram(adata_sub,groupby=f'leiden_{str(sub_resolution_sel)}')
sc.pl.dotplot(adata_sub,
              var_names=markers,
              groupby=f'leiden_{str(sub_resolution_sel)}',
              standard_scale='var',
              dendrogram=True
             )
```

Example 51 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: Mono, MP, CX3CR1+, etc.
```

Example 52 (python):
```python
# DEG analysis
print(adata_sub.X.data[:5])
adata_sub.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata_sub,groupby=f'leiden_{str(sub_resolution_sel)}')
sc.tl.dendrogram(adata_sub, groupby=f'leiden_{str(sub_resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata_sub, dendrogram=True, standard_scale='var', return_fig=True)
dp.add_totals().show()
```

Example 53 (unknown):
```unknown
[2.0822465 2.0822465 2.0822465 2.0822465 2.0822465]
```

Example 54 (python):
```python
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color=f'leiden_{str(sub_resolution_sel)}', legend_loc='on data',
                wspace=0.3)
```

Example 55 (python):
```python
# annotate
adata_sub.obs['fine_grain_manual'] = adata_sub.obs[f'leiden_{str(sub_resolution_sel)}'].copy()
adata_sub.obs.replace({'fine_grain_manual':{
    '0':'MacrophagesLYVE1pos',
    '1':'MacrophagesCX3CR1pos',
    '2':'unassigned',
    '3':'MacrophagesLYVE1pos',
    '4':'Monocytes',
    '5':'Monocytes',
    '6':'MacrophagesCX3CR1pos',
     '7':'MacrophagesCX3CR1pos',
     '8':'DendriticCells',
}},inplace=True)
adata_sub.obs['fine_grain_manual'].value_counts()
```

Example 56 (python):
```python
/tmp/ipykernel_285284/1602106113.py:3: FutureWarning: The behavior of Series.replace (and DataFrame.replace) with CategoricalDtype is deprecated. In a future version, replace will only be used for cases that preserve the categories. To change the categories, use ser.cat.rename_categories instead.
  adata_sub.obs.replace({'fine_grain_manual':{
```

Example 57 (unknown):
```unknown
fine_grain_manual
MacrophagesLYVE1pos     993
MacrophagesCX3CR1pos    910
unassigned              551
Monocytes               467
DendriticCells           26
Name: count, dtype: int64
```

Example 58 (python):
```python
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color='fine_grain_manual',
                wspace=0.3)
```

Example 59 (python):
```python
markers={
    'Mono':['CCR2','FCN1','VCAN','MPO'],
    'MP':['C1QA','C1QB','CD68','MRC1','CD163','MERTK'],
    'CX3CR1+':['CX3CR1','C3','TREM2','P2RY12'],
    'LYVE1+':['LYVE1','FOLR2'],
     'pDC':['IRF8','IRF4','IL3RA'],
     'DC_mature':['FLT3', 'CCR7','LAMP3', 'IL15', 'CD83', 'LY75', ],
    'Cycling':['MKI67','TOP2A'],
}
for k,g in markers.items():
    markers[k] = [x for x in g if x in adata_sub.var_names]

sc.tl.dendrogram(adata_sub,groupby='fine_grain_manual')
sc.pl.dotplot(adata_sub,
              var_names=markers,
              groupby='fine_grain_manual',
              standard_scale='var',
              dendrogram=True
             )
```

Example 60 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: MacrophagesLYVE1pos, MacrophagesCX3CR1pos, unassigned, etc.
var_group_labels: Mono, MP, CX3CR1+, etc.
```

Example 61 (python):
```python
adata.obs['fine_grain_manual'] = adata.obs['tmp_annotation'].astype('str').copy()
adata.obs.loc[adata_sub.obs_names,'fine_grain_manual'] = adata_sub.obs['fine_grain_manual']
```

Example 62 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color='fine_grain_manual'
                )
```

Example 63 (python):
```python
for tissue_id in ['C194-HEA-0-FFPE-1', 'Hst45-HEA-0-FFPE-1']:
    slide = adata[adata.obs['tissue_block_id']==tissue_id]
    sc.pl.spatial(slide,color=['fine_grain_manual'],# groups='SinoatrialNodePacemakerCells',
                  spot_size=60,vmax='p100')
```

Example 64 (python):
```python
/tmp/ipykernel_285284/4023824168.py:3: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],# groups='SinoatrialNodePacemakerCells',
```

Example 65 (python):
```python
/tmp/ipykernel_285284/4023824168.py:3: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],# groups='SinoatrialNodePacemakerCells',
```

Example 66 (python):
```python
for tissue_id in ['C194-HEA-0-FFPE-1', 'Hst45-HEA-0-FFPE-1']:
    slide = adata[adata.obs['tissue_block_id']==tissue_id]
    sc.pl.spatial(slide,color=['fine_grain_manual'],groups='MacrophagesCX3CR1pos',
                  spot_size=80,vmax='p100')
```

Example 67 (python):
```python
/tmp/ipykernel_285284/642200447.py:3: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups='MacrophagesCX3CR1pos',
```

Example 68 (python):
```python
/tmp/ipykernel_285284/642200447.py:3: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups='MacrophagesCX3CR1pos',
```

Example 69 (python):
```python
for tissue_id in ['C194-HEA-0-FFPE-1', 'Hst45-HEA-0-FFPE-1']:
    slide = adata[adata.obs['tissue_block_id']==tissue_id]
    sc.pl.spatial(slide,color=['fine_grain_manual'],groups='MacrophagesLYVE1pos',
                  spot_size=60,vmax='p100')
```

Example 70 (python):
```python
/tmp/ipykernel_285284/895811050.py:3: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups='MacrophagesLYVE1pos',
```

Example 71 (python):
```python
/tmp/ipykernel_285284/895811050.py:3: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups='MacrophagesLYVE1pos',
```

Example 72 (python):
```python
for tissue_id in ['C194-HEA-0-FFPE-1', 'Hst45-HEA-0-FFPE-1']:
    slide = adata[adata.obs['tissue_block_id']==tissue_id]
    sc.pl.spatial(slide,color=['fine_grain_manual'],groups='DendriticCells',
                  spot_size=80,vmax='p100')
```

Example 73 (python):
```python
/tmp/ipykernel_285284/2965593451.py:3: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups='DendriticCells',
```

Example 74 (python):
```python
/tmp/ipykernel_285284/2965593451.py:3: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups='DendriticCells',
```

Example 75 (python):
```python
# save
adata.write(f'{adata_dir}/{sample_id}_5K_{celltype}_lognorm.h5ad')
print(f'{adata_dir}/{sample_id}_5K_{celltype}_lognorm.h5ad')
```

Example 76 (unknown):
```unknown
/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium/subsets/C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat_5K_MyeloidCells_lognorm.h5ad
```

Example 77 (python):
```python
sc.tl.dendrogram(adata,groupby='fine_grain_manual')
sc.pl.dotplot(adata,
              var_names=['P2RY12','CX3CR1','TREM2','SIRPA'],
              groupby='fine_grain_manual',
              standard_scale='var',
              dendrogram=True
             )
```

---

## Variables¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/VisiumHD/no7_manual-annotation_glia.html

**Contents:**
- Variables¶
- Read in data¶
- Remove mid-grain markers from the feature¶
- PCA, Clustering, UMAP¶
- Annotation¶
- Save¶

2540 rows × 7 columns

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os

import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 

import celltypist
```

Example 2 (python):
```python
# import session_info
# session_info.show()
```

Example 3 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 4 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 5 (python):
```python
adata_dir = '/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/subsets'
sample_id = 'HEA_FOET14880396'
celltype = 'Glia'
```

Example 6 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk/Aug2024'
table_dir = '/nfs/team205/heart/fetal_ms_figures/tables/Aug2024'
```

Example 7 (python):
```python
# read in
adata = sc.read_h5ad(f'{adata_dir}/{sample_id}_{celltype}_b2c_cells_filtered_lognorm.h5ad')
adata
```

Example 8 (unknown):
```unknown
AnnData object with n_obs × n_vars = 588 × 13981
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'leiden_0.2', 'leiden_0.4', 'leiden_0.6', 'leiden_1', 'leiden_1.2', 'leiden_1.5', 'leiden_2'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1.2', 'leiden_1.2_colors', 'leiden_1.5', 'leiden_1.5_colors', 'leiden_1_colors', 'leiden_2', 'leiden_2_colors', 'log1p', 'neighbors', 'pca', 'spatial', 'umap'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 9 (python):
```python
# read in
midgrain_markers = pd.read_csv('/nfs/team205/kk18/notebooks/Foetal/VisiumHD/re-seq_mpp0p3/sc_midgrain_markers.csv',
                               index_col=0
                              )
# remove the markers of celltype of interest
midgrain_markers = midgrain_markers[midgrain_markers['group']!=celltype]
midgrain_markers
```

Example 10 (python):
```python
# remove
mask = [x not in list(midgrain_markers['names']) for x in adata.var_names]
bdata = adata[:,mask]
bdata
```

Example 11 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 588 × 12746
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'leiden_0.2', 'leiden_0.4', 'leiden_0.6', 'leiden_1', 'leiden_1.2', 'leiden_1.5', 'leiden_2'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1.2', 'leiden_1.2_colors', 'leiden_1.5', 'leiden_1.5_colors', 'leiden_1_colors', 'leiden_2', 'leiden_2_colors', 'log1p', 'neighbors', 'pca', 'spatial', 'umap', 'dendrogram_leiden_0.6', 'rank_genes_groups'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 12 (python):
```python
# remove cell cycle genes
cell_cycle_genes = ['MCM5','PCNA','TYMS','FEN1','MCM2','MCM4','RRM1','UNG','GINS2','MCM6','CDCA7','DTL','PRIM1','UHRF1','MLF1IP','HELLS','RFC2','RPA2','NASP',
                'RAD51AP1','GMNN','WDR76','SLBP','CCNE2','UBR7','POLD3','MSH2','ATAD2','RAD51','RRM2','CDC45','CDC6','EXO1','TIPIN','DSCC1','BLM','CASP8AP2',
                'USP1','CLSPN','POLA1','CHAF1B','BRIP1','E2F8','HMGB2','CDK1','NUSAP1','UBE2C','BIRC5','TPX2','TOP2A','NDC80','CKS2','NUF2','CKS1B','MKI67',
                'TMPO','CENPF','TACC3','FAM64A','SMC4','CCNB2','CKAP2L','CKAP2','AURKB','BUB1','KIF11','ANP32E','TUBB4B','GTSE1','KIF20B','HJURP','CDCA3','HN1',
                'CDC20','TTK','CDC25C','KIF2C','RANGAP1','NCAPD2','DLGAP5','CDCA2','CDCA8','ECT2','KIF23','HMMR','AURKA','PSRC1','ANLN','LBR','CKAP5','CENPE','CTCF',
                'NEK2','G2E3','GAS2L3','CBX5','CENPA']
mask = [x not in cell_cycle_genes for x in bdata.var_names]
bdata = bdata[:,mask]
bdata.shape
```

Example 13 (unknown):
```unknown
(588, 12652)
```

Example 14 (python):
```python
# HVGs, using the object which doesn't have midgrain markers
sc.pp.highly_variable_genes(bdata, flavor='seurat', min_mean=0.0125, max_mean=3, min_disp=0.5)
print(f'number of hvgs: {sum(bdata.var.highly_variable)}')
# PCA using the degected HVGs
sc.tl.pca(bdata, use_highly_variable=True)
adata.obsm['X_pca'] = bdata.obsm['X_pca'].copy()
# create neighbour graph
sc.pp.neighbors(adata,n_neighbors=15,use_rep='X_pca')
# visualisation with umap
sc.tl.umap(adata,min_dist=0.5)
sc.pl.embedding(adata,basis='X_umap',color=['bin_count','total_counts','n_genes_by_counts',"conf_score_midmod2fine","celltypist_midmod2fine",],
                wspace=0.2,cmap='RdPu',vmax='p100',ncols=5)
# leiden clustering
resolutions_list = [0.2,0.4,0.6,1,1.2,1.5,2]
for resolution in resolutions_list:
    sc.tl.leiden(adata,resolution=resolution,key_added=f'leiden_{str(resolution)}',n_iterations=2)
sc.pl.embedding(adata,basis='X_umap',
                color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                wspace=0.3)
```

Example 15 (unknown):
```unknown
number of hvgs: 3367
```

Example 16 (python):
```python
sc.pl.embedding(adata,basis='X_umap',
                color=['celltypist_midmod2fine', 'conf_score_midmod2fine'],
                wspace=0.7)
```

Example 17 (python):
```python
'SLC18A3' in adata.var_names
```

Example 18 (python):
```python
# markers for fine-grained cell types
markers = {
    "astrocytes": ["GFAP", "S100B", "ALDH1L1"],
    "oligodendrocytes": ["OLIG2", "MBP", "PDGFRA"],
    "microglia": ["AIF1"],  # IBA1 protein is encoded by the AIF1 gene
    "schwann_cells": ["SOX10", "MPZ", "S100B"],
    "schwann_cell_precursors": [
                                "PLS3",'SLITRK6','IL1RAPL2'
                               ],
    'Cycling':['MKI67','TOP2A'],
    'Others':['NGF'],
    'Neuronal':['SYT1','PCSK1N']
}

genes = []
for g in markers.values():
    genes = genes + [x for x in g if (x in adata.var_names)&(x not in genes)]
sc.pl.embedding(adata,basis='X_umap',color=genes,
               ncols=5, wspace=0.2,cmap='RdPu',vmax='p99.7')
```

Example 19 (unknown):
```unknown
resolution_sel = 0.6
```

Example 20 (python):
```python
for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in adata.var_names]

sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
sc.pl.dotplot(adata,
              markers, 
              groupby=f'leiden_{str(resolution_sel)}',
              dendrogram=True,
              standard_scale="var",
              color_map="Reds",
              swap_axes=False,
             )
```

Example 21 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: astrocytes, oligodendrocytes, microglia, etc.
```

Example 22 (python):
```python
# DEG analysis
print(adata.X.data[:5])
adata.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata, dendrogram=True, standard_scale='var', n_genes=15, return_fig=True)
dp.add_totals().show()
```

Example 23 (unknown):
```unknown
[2.1933994  2.3171509  1.76055317 2.69164533 1.61733065]
```

Example 24 (python):
```python
# DEG analysis without other cell type markers
# with removing mid_grain markers
# remove mid_grain markers
mask = [x not in list(midgrain_markers['names']) for x in adata.var_names]
bdata = adata[:,mask]

print(bdata.X.data[:5])
bdata.uns['log1p']['base']=None
sc.tl.rank_genes_groups(bdata,groupby=f'leiden_{str(resolution_sel)}')
sc.tl.dendrogram(bdata, groupby=f'leiden_{str(resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(bdata, dendrogram=True, standard_scale='var', return_fig=True)
dp.add_totals().show()
```

Example 25 (unknown):
```unknown
[2.1933994  2.3171509  1.76055317 2.69164533 1.61733065]
```

Example 26 (python):
```python
sc.pl.embedding(adata,basis='X_umap',
                color=f'leiden_{str(resolution_sel)}',
                wspace=0.3, legend_loc='on data')
```

Example 27 (python):
```python
sc.pl.violin(adata,keys=['total_counts','bin_count'],groupby=f'leiden_{str(resolution_sel)}')
```

Example 28 (unknown):
```unknown
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 29 (python):
```python
# annotate
adata.obs['fine_grain_manual'] = adata.obs[f'leiden_{str(resolution_sel)}'].copy()
adata.obs.replace({'fine_grain_manual':{
    '0':'SchwannCells',
    '1':'SchwannCellPrecursors',
    '2':'SchwannCells',
    '3':'SchwannCells_and_Neurons', # likely doublets with neurons
}},inplace=True)
adata.obs['fine_grain_manual'].value_counts()
```

Example 30 (unknown):
```unknown
fine_grain_manual
SchwannCells                354
SchwannCellPrecursors       163
SchwannCells_and_Neurons     71
Name: count, dtype: int64
```

Example 31 (python):
```python
ad = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/Feb28ObjectRaw_finegrain_updated.h5ad',backed='r')
ad[ad.obs['mid_grain']==celltype].obs['fine_grain'].value_counts()
```

Example 32 (unknown):
```unknown
fine_grain
SchwannCells             2263
SchwannCellPrecursors    1278
Name: count, dtype: int64
```

Example 33 (python):
```python
sc.pl.embedding(adata,basis='X_umap',
                color='fine_grain_manual',
                frameon=False,
                wspace=0.3)
```

Example 34 (python):
```python
# markers for fine-grained cell types
markers = {
    "astrocytes": ["GFAP", "S100B", "ALDH1L1"],
    "oligodendrocytes": ["OLIG2", "MBP", "PDGFRA"],
    "microglia": ["AIF1"],  # IBA1 protein is encoded by the AIF1 gene
    "schwann_cells": ["SOX10", "MPZ", "S100B"],
    "schwann_cell_precursors": [
                                "PLS3",'SLITRK6','IL1RAPL2'
                               ],
    'Cycling':['MKI67','TOP2A'],
    'Others':['NGF'],
    'Neuronal':['SYT1','PCSK1N']
}
for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in adata.var_names]

sc.tl.dendrogram(adata,groupby='fine_grain_manual')
sc.pl.dotplot(adata,
              var_names=markers,
              groupby='fine_grain_manual',
              standard_scale='var',
              dendrogram=True
             )
```

Example 35 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: SchwannCells, SchwannCellPrecursors, SchwannCells_and_Neurons
var_group_labels: astrocytes, oligodendrocytes, microglia, etc.
```

Example 36 (python):
```python
# save
adata.write(f'{adata_dir}/{sample_id}_{celltype}_b2c_cells_filtered_lognorm.h5ad')
print(f'{adata_dir}/{sample_id}_{celltype}_b2c_cells_filtered_lognorm.h5ad')
```

Example 37 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/subsets/HEA_FOET14880396_Glia_b2c_cells_filtered_lognorm.h5ad
```

---

## Import modules¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/RNA/scnRNA_Integration-Annotation/Leukocytes/No1_Immune_pre-scVI.html

**Contents:**
- Import modules¶
- Read in data¶
- Remove C97: uncertain inflammatory signature in C97¶
- Save to gdrive¶

**Examples:**

Example 1 (python):
```python
import numpy as np
import numpy.random as random
import pandas as pd
import scanpy as sc
import anndata
import matplotlib.pyplot as plt
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
-----
anndata             0.7.5
matplotlib          3.3.4
numpy               1.20.1
pandas              1.1.4
scanpy              1.7.1
session_info        1.0.0
-----
```

Example 4 (unknown):
```unknown
PIL                 8.1.2
anyio               NA
attr                20.3.0
babel               2.9.0
backcall            0.2.0
brotli              NA
cairo               1.20.0
certifi             2020.12.05
cffi                1.14.5
chardet             4.0.0
cloudpickle         1.6.0
colorama            0.4.4
cycler              0.10.0
cython_runtime      NA
cytoolz             0.11.0
dask                2021.03.1
dateutil            2.8.1
decorator           4.4.2
fsspec              0.8.7
get_version         2.1
google              NA
h5py                2.10.0
idna                2.10
igraph              0.8.3
ipykernel           5.5.0
ipython_genutils    0.2.0
jedi                0.18.0
jinja2              2.11.3
joblib              1.0.1
json5               NA
jsonschema          3.2.0
jupyter_server      1.4.1
jupyterlab_server   2.3.0
kiwisolver          1.3.1
legacy_api_wrap     0.0.0
leidenalg           0.8.3
llvmlite            0.34.0
louvain             0.7.0
markupsafe          1.1.1
mpl_toolkits        NA
natsort             7.1.1
nbclassic           NA
nbformat            5.1.2
numba               0.51.2
numexpr             2.7.3
packaging           20.9
parso               0.8.1
pexpect             4.8.0
pickleshare         0.7.5
pkg_resources       NA
prometheus_client   NA
prompt_toolkit      3.0.16
psutil              5.8.0
ptyprocess          0.7.0
pvectorc            NA
pyarrow             0.16.0
pygments            2.8.0
pyparsing           2.4.7
pyrsistent          NA
pytoml              NA
pytz                2021.1
requests            2.25.1
ruamel              NA
scipy               1.6.1
send2trash          NA
setuptools_scm      NA
simplejson          3.17.2
sinfo               0.3.1
six                 1.15.0
sklearn             0.22
sniffio             1.2.0
socks               1.7.1
sphinxcontrib       NA
storemagic          NA
tables              3.6.1
tblib               1.7.0
texttable           1.6.3
tlz                 0.11.0
toolz               0.11.1
tornado             6.1
traitlets           5.0.5
typing_extensions   NA
uritemplate         3.0.1
urllib3             1.26.3
wcwidth             0.2.5
yaml                5.1.2
zmq                 22.0.3
```

Example 5 (unknown):
```unknown
-----
IPython             7.21.0
jupyter_client      6.1.11
jupyter_core        4.7.1
jupyterlab          3.0.9
notebook            6.2.0
-----
Python 3.8.8 | packaged by conda-forge | (default, Feb 20 2021, 16:22:27) [GCC 9.3.0]
Linux-4.15.0-112-generic-x86_64-with-glibc2.10
-----
Session information updated at 2023-02-16 10:25
```

Example 6 (python):
```python
sc.settings.verbosity = 3             # verbosity: errors (0), warnings (1), info (2), hints (3)
sc.settings.set_figure_params(dpi=200, color_map = 'RdPu',)
```

Example 7 (python):
```python
adata = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/14Feb_fromSemih/14FebImmuneForKazumasa.h5ad')
print(adata.X.data[:10])
adata
```

Example 8 (unknown):
```unknown
[1. 2. 1. 1. 1. 2. 2. 5. 2. 2.]
```

Example 9 (unknown):
```unknown
AnnData object with n_obs × n_vars = 49042 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', '_scvi_labels', '2Jan_coarse_grain', '2Jan_mid_grain', '2Jan_fine_grain', 'leiden_scVI', 'FACSgate', 'sex', 'week', 'trimester'
    var: 'gene_name_scRNA-0', 'gene_id'
    uns: 'FACSgate_colors', 'age_colors', 'cell_or_nuclei_colors', 'donor_colors', 'modality_colors', 'sex_colors', 'trimester_colors', 'week_colors'
    obsm: 'X_scVI', 'X_umap', '_scvi_extra_continuous', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 10 (python):
```python
adata.obs['week'].value_counts()
```

Example 11 (unknown):
```unknown
20W    22643
6W      8120
8W      5563
7W      5209
9W      2324
10W     1912
13W     1261
15W     1038
5W       520
14W      207
12W      188
4W        57
Name: week, dtype: int64
```

Example 12 (python):
```python
sc.settings.set_figure_params(dpi=200, facecolor='white', color_map = 'RdPu',)
sc.pl.umap(adata, color=['leiden_scVI','2Jan_mid_grain','2Jan_fine_grain'], size=0.5, wspace=0.1, legend_loc='on data', legend_fontsize=5)
```

Example 13 (python):
```python
adata_lognorm = adata.copy()
sc.pp.normalize_total(adata_lognorm, target_sum=1e4)
sc.pp.log1p(adata_lognorm)
```

Example 14 (unknown):
```unknown
normalizing counts per cell
    finished (0:00:00)
```

Example 15 (python):
```python
inflam_genes=['TNF','IL1B','IL1A','IL6','CXCL1','CXCL2','CXCL8','CCL2']
inflam_genes = [x for x in inflam_genes if x in adata_lognorm.var_names]
sc.tl.score_genes(adata_lognorm, inflam_genes, score_name='inflammatory')
sc.pl.violin(adata_lognorm, keys='inflammatory', groupby='donor', rotation=90, jitter=False)
```

Example 16 (python):
```python
computing score 'inflammatory'
    finished: added
    'inflammatory', score of gene set (adata.obs).
    200 total control genes are used. (0:00:01)
```

Example 17 (unknown):
```unknown
!ls /nfs/team205/heart/anndata_objects/Foetal/Immune
```

Example 18 (python):
```python
# Remove C97 from Immune cells
## save C97 cell ID
C97_index = adata.obs_names[adata.obs['donor']=='C97']
C97_index.to_series().to_csv('/nfs/team205/heart/anndata_objects/Foetal/Immune/C97_index.csv',index=False)

# remove
adata = adata[adata.obs['donor']!='C97']
pd.crosstab(adata.obs['week'],adata.obs['age'])
```

Example 19 (python):
```python
adata.write(f'/home/jovyan/mount/gdrive/Colab/data/anndata/RNA_foetal_raw_immune_16Feb2023.h5ad')
adata
```

Example 20 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 41881 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', '_scvi_labels', '2Jan_coarse_grain', '2Jan_mid_grain', '2Jan_fine_grain', 'leiden_scVI', 'FACSgate', 'sex', 'week', 'trimester'
    var: 'gene_name_scRNA-0', 'gene_id'
    uns: 'FACSgate_colors', 'age_colors', 'cell_or_nuclei_colors', 'donor_colors', 'modality_colors', 'sex_colors', 'trimester_colors', 'week_colors', 'leiden_scVI_colors', '2Jan_mid_grain_colors', '2Jan_fine_grain_colors'
    obsm: 'X_scVI', 'X_umap', '_scvi_extra_continuous', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

---

## Variables¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/VisiumHD/no7_manual-annotation_neurons.html

**Contents:**
- Variables¶
- Read in data¶
- Remove mid-grain markers from the feature¶
- PCA, Clustering, UMAP¶
- Annotation¶
- Save¶

2445 rows × 7 columns

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os

import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 

import celltypist
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
-----
celltypist          1.6.3
matplotlib          3.9.2
numpy               2.0.2
pandas              2.2.3
scanpy              1.10.3
seaborn             0.13.2
session_info        1.0.0
-----
```

Example 4 (python):
```python
PIL                 11.0.0
anndata             0.10.9
asttokens           NA
certifi             2024.08.30
charset_normalizer  3.4.0
comm                0.2.2
cycler              0.12.1
cython_runtime      NA
dateutil            2.9.0.post0
debugpy             1.8.9
decorator           5.1.1
exceptiongroup      1.2.2
executing           2.1.0
google              NA
h5py                3.12.1
idna                3.10
igraph              0.10.2
importlib_metadata  NA
importlib_resources NA
ipykernel           6.29.5
jedi                0.19.2
joblib              1.4.2
kiwisolver          1.4.7
legacy_api_wrap     NA
leidenalg           0.9.0
llvmlite            0.43.0
mpl_toolkits        NA
natsort             8.4.0
numba               0.60.0
packaging           24.2
parso               0.8.4
patsy               1.0.1
pexpect             4.9.0
platformdirs        4.3.6
prompt_toolkit      3.0.48
psutil              6.1.0
ptyprocess          0.7.0
pure_eval           0.2.3
pydev_ipython       NA
pydevconsole        NA
pydevd              3.2.3
pydevd_file_utils   NA
pydevd_plugins      NA
pydevd_tracing      NA
pygments            2.18.0
pyparsing           3.2.0
pytz                2024.2
requests            2.32.3
scipy               1.13.1
six                 1.16.0
sklearn             1.5.2
stack_data          0.6.3
statsmodels         0.14.4
texttable           1.6.7
threadpoolctl       3.5.0
tornado             6.4.2
traitlets           5.14.3
typing_extensions   NA
urllib3             2.2.3
wcwidth             0.2.13
yaml                6.0.2
zipp                NA
zmq                 26.2.0
zoneinfo            NA
```

Example 5 (unknown):
```unknown
-----
IPython             8.18.1
jupyter_client      8.6.3
jupyter_core        5.7.2
-----
Python 3.9.20 | packaged by conda-forge | (main, Sep 30 2024, 17:49:10) [GCC 13.3.0]
Linux-5.15.0-126-generic-x86_64-with-glibc2.35
-----
Session information updated at 2024-12-07 22:32
```

Example 6 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 7 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 8 (python):
```python
adata_dir = '/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/subsets'
sample_id = 'HEA_FOET14880396'
celltype = 'Neurons'
```

Example 9 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk/Aug2024'
table_dir = '/nfs/team205/heart/fetal_ms_figures/tables/Aug2024'
```

Example 10 (python):
```python
# read in
adata = sc.read_h5ad(f'{adata_dir}/{sample_id}_{celltype}_b2c_cells_filtered_lognorm.h5ad')
adata
```

Example 11 (unknown):
```unknown
AnnData object with n_obs × n_vars = 589 × 14494
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'leiden_0.2', 'leiden_0.4', 'leiden_0.6', 'leiden_1', 'leiden_1.2', 'leiden_1.5', 'leiden_2'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1.2', 'leiden_1.2_colors', 'leiden_1.5', 'leiden_1.5_colors', 'leiden_1_colors', 'leiden_2', 'leiden_2_colors', 'log1p', 'neighbors', 'pca', 'spatial', 'umap'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 12 (python):
```python
# read in
midgrain_markers = pd.read_csv('/nfs/team205/kk18/notebooks/Foetal/VisiumHD/re-seq_mpp0p3/sc_midgrain_markers.csv',
                               index_col=0
                              )
# remove the markers of celltype of interest
midgrain_markers = midgrain_markers[midgrain_markers['group']!=celltype]
midgrain_markers
```

Example 13 (python):
```python
# remove
mask = [x not in list(midgrain_markers['names']) for x in adata.var_names]
bdata = adata[:,mask]
bdata
```

Example 14 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 589 × 13304
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'leiden_0.2', 'leiden_0.4', 'leiden_0.6', 'leiden_1', 'leiden_1.2', 'leiden_1.5', 'leiden_2'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1.2', 'leiden_1.2_colors', 'leiden_1.5', 'leiden_1.5_colors', 'leiden_1_colors', 'leiden_2', 'leiden_2_colors', 'log1p', 'neighbors', 'pca', 'spatial', 'umap'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 15 (python):
```python
# HVGs, using the object which doesn't have midgrain markers
sc.pp.highly_variable_genes(bdata, flavor='seurat', min_mean=0.0125, max_mean=3, min_disp=0.5)
print(f'number of hvgs: {sum(bdata.var.highly_variable)}')
# PCA using the degected HVGs
sc.tl.pca(bdata, use_highly_variable=True)
adata.obsm['X_pca'] = bdata.obsm['X_pca'].copy()
# create neighbour graph
sc.pp.neighbors(adata,n_neighbors=15,use_rep='X_pca')
# visualisation with umap
sc.tl.umap(adata,min_dist=0.5)
sc.pl.embedding(adata,basis='X_umap',color=['bin_count','total_counts','n_genes_by_counts',"conf_score_midmod2fine","celltypist_midmod2fine",],
                wspace=0.2,cmap='RdPu',vmax='p100',ncols=5)
# leiden clustering
resolutions_list = [0.2,0.4,0.6,1,1.2,1.5,2]
for resolution in resolutions_list:
    sc.tl.leiden(adata,resolution=resolution,key_added=f'leiden_{str(resolution)}',n_iterations=2)
sc.pl.embedding(adata,basis='X_umap',
                color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                wspace=0.3)
```

Example 16 (unknown):
```unknown
number of hvgs: 3519
```

Example 17 (python):
```python
/software/cellgen/team205/kk18/envs/generic_env/lib/python3.9/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm
```

Example 18 (python):
```python
sc.pl.embedding(adata,basis='X_umap',
                color=['celltypist_midmod2fine', 'conf_score_midmod2fine'],
                wspace=0.7)
```

Example 19 (python):
```python
'SLC18A3' in adata.var_names
```

Example 20 (python):
```python
# markers for fine-grained cell types
markers = {
    "ChromaffinCells": [
        'DLK1',
        'EPAS1',
        'KCNJ6',
        "TH",    # Tyrosine hydroxylase
        "PNMT",  # Phenylethanolamine N-methyltransferase
        "CHGA",  # Chromogranin A
        "CHGB",  # Chromogranin B
        "PHOX2B"
    ],
    "NeuronPrecursors": [
        "NES",    # Nestin
        "PAX6",   # Paired box 6
        "TUBB3",  # Tubulin Beta 3 Class III
        "DCX",     # Doublecortin
        'ASCL1','NELL2','BNC2'
    ],
    "ParasympatheticNeurons": [
        "PHOX2B",
        "CHAT",    # Choline O-Acetyltransferase
        "SLC18A3", # Solute Carrier Family 18 Member A3 (Vesicular Acetylcholine Transporter)
        "GATA3",
        'NEFL','SST'
    ],
    "SympatheticNeurons": [
        "TH",
        "PHOX2B",
        "GATA3",
        "CHGA",
        "CHGB",
        'DBH','VSTM2L'
    ]
}

genes = []
for g in markers.values():
    genes = genes + [x for x in g if (x in adata.var_names)&(x not in genes)]
sc.pl.embedding(adata,basis='X_umap',color=genes,
               ncols=5, wspace=0.2,cmap='RdPu',vmax='p99.7')
```

Example 21 (unknown):
```unknown
resolution_sel = 1
```

Example 22 (python):
```python
for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in adata.var_names]

sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
sc.pl.dotplot(adata,
              markers, 
              groupby=f'leiden_{str(resolution_sel)}',
              dendrogram=True,
              standard_scale="var",
              color_map="Reds",
              swap_axes=False,
             )
```

Example 23 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: ChromaffinCells, NeuronPrecursors, ParasympatheticNeurons, etc.
```

Example 24 (python):
```python
# DEG analysis
print(adata.X.data[:5])
adata.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata, dendrogram=True, standard_scale='var', return_fig=True)
dp.add_totals().show()
```

Example 25 (unknown):
```unknown
[1.1620675  1.88821923 2.69821449 1.59010602 1.98053876]
```

Example 26 (python):
```python
# DEG analysis without other cell type markers
# with removing mid_grain markers
# remove mid_grain markers
mask = [x not in list(midgrain_markers['names']) for x in adata.var_names]
bdata = adata[:,mask]

print(bdata.X.data[:5])
bdata.uns['log1p']['base']=None
sc.tl.rank_genes_groups(bdata,groupby=f'leiden_{str(resolution_sel)}')
sc.tl.dendrogram(bdata, groupby=f'leiden_{str(resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(bdata, dendrogram=True, standard_scale='var', return_fig=True)
dp.add_totals().show()
```

Example 27 (unknown):
```unknown
[1.1620675  1.88821923 2.69821449 1.59010602 2.22351621]
```

Example 28 (python):
```python
sc.pl.embedding(adata,basis='X_umap',
                color=f'leiden_{str(resolution_sel)}',
                wspace=0.3, legend_loc='on data')
```

Example 29 (python):
```python
sc.pl.violin(adata,keys=['total_counts','bin_count'],groupby=f'leiden_{str(resolution_sel)}')
```

Example 30 (unknown):
```unknown
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 31 (python):
```python
# annotate
adata.obs['fine_grain_manual'] = adata.obs[f'leiden_{str(resolution_sel)}'].copy()
adata.obs.replace({'fine_grain_manual':{
    '0':'ChromaffinCells',
    '1':'SympatheticNeurons',
    '2':'ChromaffinCells',
    '3':'ParasympatheticNeurons',
    '4':'NeuronPrecursors',
     '5':'ParasympatheticNeurons',
}},inplace=True)
adata.obs['fine_grain_manual'].value_counts()
```

Example 32 (unknown):
```unknown
fine_grain_manual
ChromaffinCells           265
ParasympatheticNeurons    153
SympatheticNeurons        110
NeuronPrecursors           61
Name: count, dtype: int64
```

Example 33 (python):
```python
ad = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/Feb28ObjectRaw_finegrain_updated.h5ad',backed='r')
set(ad.obs['fine_grain'])
```

Example 34 (unknown):
```unknown
{'AtrialCardiomyocytesCycling',
 'AtrialCardiomyocytesLeft',
 'AtrialCardiomyocytesRight',
 'AtrioventricularNodePacemakerCells',
 'BCells',
 'BCellsMS4A1pos',
 'ChromaffinCells',
 'CoronaryArterialEndothelialCells',
 'CoronaryCapillaryEndothelialCells',
 'CoronaryPericytes',
 'CoronarySmoothMuscleCells',
 'CoronaryVenousEndothelialCells',
 'CoronaryVesselAdventitialFibroblasts',
 'DendriticCellsMature',
 'DendriticCellsType1',
 'DuctusArteriosusSmoothMuscleCells',
 'EndocardialCells',
 'EndocardialCushionCells',
 'EpicardiumDerivedCells',
 'GreatVesselAdventitialFibroblasts',
 'GreatVesselArterialEndothelialCells',
 'GreatVesselSmoothMuscleCells',
 'GreatVesselVenousEndothelialCells',
 'InnateLymphoidCells',
 'LymphNodeFibroblasticReticularCells',
 'LymphaticEndothelialCells',
 'MacrophagesATF3pos',
 'MacrophagesCX3CR1pos',
 'MacrophagesLYVE1pos',
 'MacrophagesTIMD4pos',
 'MastCells',
 'Megakaryocytes',
 'MesothelialEpicardialCells',
 'MonocyteDerivedCells',
 'Monocytes',
 'MonocytesMPOpos',
 'MyocardialInterstitialFibroblasts',
 'Myofibroblasts',
 'NaturalKillerCells',
 'NeuronPrecursors',
 'ParasympatheticNeurons',
 'PericardialCellsFibrous',
 'PericardialCellsIntermediate',
 'PericardialCellsParietal',
 'PlasmacytoidDendriticCells',
 'ProBCells',
 'SchwannCellPrecursors',
 'SchwannCells',
 'SinoatrialNodePacemakerCells',
 'SubEpicardialFibroblasts',
 'SympatheticNeurons',
 'TCellsCD4pos',
 'TCellsCD8pos',
 'TregsCD4pos',
 'ValveEndothelialCells',
 'ValveInterstitialCells',
 'VentricularCardiomyocytesCycling',
 'VentricularCardiomyocytesLeftCompact',
 'VentricularCardiomyocytesLeftTrabeculated',
 'VentricularCardiomyocytesRightCompact',
 'VentricularCardiomyocytesRightTrabeculated',
 'VentricularConductionSystemDistal',
 'VentricularConductionSystemProximal'}
```

Example 35 (python):
```python
sc.pl.embedding(adata,basis='X_umap',
                color='fine_grain_manual',
                frameon=False,
                wspace=0.3)
```

Example 36 (python):
```python
sc.tl.dendrogram(adata,groupby='fine_grain_manual')
sc.pl.dotplot(adata,
              var_names=markers,
              groupby='fine_grain_manual',
              standard_scale='var',
              dendrogram=True
             )
```

Example 37 (python):
```python
# save
adata.write(f'{adata_dir}/{sample_id}_{celltype}_b2c_cells_filtered_lognorm.h5ad')
print(f'{adata_dir}/{sample_id}_{celltype}_b2c_cells_filtered_lognorm.h5ad')
```

Example 38 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/subsets/HEA_FOET14880396_Neurons_b2c_cells_filtered_lognorm.h5ad
```

---

## Import required modules¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/Visium/Others/Plotting_multiple_Compact-Trabeculated.html

**Contents:**
- Import required modules¶
- Prepare functions¶
- Read in anndata¶
- Plot the uncropped section¶

**Examples:**

Example 1 (python):
```python
# this is for the jupyter notebook at JupyterLab
# to make sys.path between the terminal and Jupyter same
import sys

print(sys.path) # especially the path to site-packages are different... so thats why couldn't find correct package...

# remove weird paths...
sys.path = sys.path[:4]+[sys.path[6]]
print(sys.path)
```

Example 2 (unknown):
```unknown
['/nfs/team205/kk18/notebooks/P48_Foetal-heart_Visium/notebooks/notebooks_for_manuscript', '/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python39.zip', '/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python3.9', '/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python3.9/lib-dynload', '', '/nfs/users/nfs_k/kk18/.local/lib/python3.9/site-packages', '/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python3.9/site-packages']
['/nfs/team205/kk18/notebooks/P48_Foetal-heart_Visium/notebooks/notebooks_for_manuscript', '/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python39.zip', '/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python3.9', '/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python3.9/lib-dynload', '/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python3.9/site-packages']
```

Example 3 (python):
```python
import warnings
warnings.simplefilter(action='ignore', category=FutureWarning)

import scanpy as sc
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 
import seaborn as sns
import os 
from mpl_toolkits.axes_grid1 import make_axes_locatable

from cell2location.plt import plot_spatial
```

Example 4 (python):
```python
/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python3.9/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm
Global seed set to 0
/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python3.9/site-packages/umap/distances.py:1063: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python3.9/site-packages/umap/distances.py:1071: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python3.9/site-packages/umap/distances.py:1086: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python3.9/site-packages/umap/umap_.py:660: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
```

Example 5 (python):
```python
import session_info
session_info.show()
```

Example 6 (python):
```python
-----
cell2location       0.1.3
matplotlib          3.7.1
mpl_toolkits        NA
numpy               1.24.3
pandas              2.0.1
scanpy              1.9.3
seaborn             0.12.2
session_info        1.0.0
-----
```

Example 7 (python):
```python
PIL                 9.5.0
absl                NA
anndata             0.9.1
asttokens           NA
attr                23.1.0
backcall            0.2.0
certifi             2023.05.07
cffi                1.15.1
charset_normalizer  3.1.0
chex                0.1.7
comm                0.1.3
contextlib2         NA
cycler              0.10.0
cython_runtime      NA
dateutil            2.8.2
debugpy             1.6.7
decorator           5.1.1
defusedxml          0.7.1
docrep              0.3.2
etils               1.2.0
executing           1.2.0
flax                0.6.10
fsspec              2023.5.0
google              NA
h5py                3.8.0
idna                3.4
igraph              0.10.4
importlib_metadata  NA
importlib_resources NA
ipykernel           6.23.0
ipython_genutils    0.2.0
jax                 0.4.9
jaxlib              0.4.9
jedi                0.18.2
joblib              1.2.0
kiwisolver          1.4.4
leidenalg           0.9.1
lightning_fabric    1.9.5
lightning_utilities 0.8.0
llvmlite            0.40.0
ml_collections      NA
ml_dtypes           0.1.0
mpmath              1.3.0
msgpack             1.0.5
mudata              0.2.2
multipledispatch    0.6.0
natsort             8.3.1
numba               0.57.0
numpyro             0.11.0
nvfuser             NA
opt_einsum          v3.3.0
optax               0.1.5
packaging           23.1
parso               0.8.3
patsy               0.5.3
pexpect             4.8.0
pickleshare         0.7.5
pkg_resources       NA
platformdirs        3.5.0
prompt_toolkit      3.0.38
psutil              5.9.5
ptyprocess          0.7.0
pure_eval           0.2.2
pycparser           2.21
pydev_ipython       NA
pydevconsole        NA
pydevd              2.9.5
pydevd_file_utils   NA
pydevd_plugins      NA
pydevd_tracing      NA
pygments            2.15.1
pynndescent         0.5.10
pyparsing           3.0.9
pyro                1.8.4
pytorch_lightning   1.9.5
pytz                2023.3
requests            2.30.0
rich                NA
scipy               1.10.1
scvi                0.20.3
setuptools          66.0.0
six                 1.16.0
sklearn             1.2.2
stack_data          0.6.2
statsmodels         0.14.0
sympy               1.12
texttable           1.6.7
threadpoolctl       3.1.0
toolz               0.12.0
torch               2.0.1+cu117
torchmetrics        0.11.4
tornado             6.3.1
tqdm                4.65.0
traitlets           5.9.0
tree                0.1.8
typing_extensions   NA
umap                0.5.3
urllib3             2.0.2
wcwidth             0.2.6
yaml                6.0
zipp                NA
zmq                 25.0.2
zoneinfo            NA
```

Example 8 (unknown):
```unknown
-----
IPython             8.13.2
jupyter_client      8.2.0
jupyter_core        5.3.0
notebook            6.5.4
-----
Python 3.9.16 (main, Mar  8 2023, 14:00:05) [GCC 11.2.0]
Linux-5.4.0-169-generic-x86_64-with-glibc2.27
-----
Session information updated at 2024-02-12 20:46
```

Example 9 (python):
```python
sc.settings.set_figure_params(dpi=80,dpi_save=300)
```

Example 10 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk'
```

Example 11 (python):
```python
# This function selects the data for multiple slides from the spatial anndata object.
# Modified `select_slide` in cell2loc for multiple slides

# @param adata: anndata object with multiple spatial experiments
# @param s: list of selected slides
# @param s_col: column in adata.obs listing experiment name for each location

def select_multislide(adata, s, s_col='sample'):

    slides = adata[adata.obs[s_col].isin(s), :]
    s_keys = list(slides.uns['spatial'].keys())
    
    uns_dict = {}
    for ss in s:
        s_spatial=np.array(s_keys)[[ss in k for k in s_keys]][0]
        uns_dict[s_spatial]=slides.uns['spatial'][s_spatial]
    
    slides.uns['spatial'] = uns_dict

    return slides
```

Example 12 (python):
```python
adata_vis = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/visium-FFPE_foetal_raw.h5ad')

# add abundance to .obs
df = adata_vis.obsm['q05_cell_abundance_w_sf'].copy()
## rename columns
df.columns = [x.replace('q05cell_abundance_w_sf_','') for x in df.columns]
adata_vis.obs[df.columns]=df.copy()

# set symboßl as var_names
adata_vis.var.set_index('SYMBOL',inplace=True)

adata_vis
```

Example 13 (unknown):
```unknown
AnnData object with n_obs × n_vars = 19973 × 18085
    obs: 'in_tissue', 'array_row', 'array_col', 'sample', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'mt_frac', 'sangerID', 'Publication', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'status', 'JC_edit', 'AtrialCardiomyocytesLeft', 'AtrialCardiomyocytesRight', 'VentricularCardiomyocytesLeftCompact', 'VentricularCardiomyocytesRightCompact', 'VentricularCardiomyocytesLeftTrabeculated', 'VentricularCardiomyocytesRightTrabeculated', 'SinoatrialNodeCardiomyocytes', 'AtrioventricularNodeCardiomyocytes', 'VentricularConductionSystemProximal', 'VentricularConductionSystemDistal', 'GreatVesselAdventitialFibroblasts', 'CoronaryVesselAdventitialFibroblasts', 'MyocardialInterstitialFibroblasts1', 'MyocardialInterstitialFibroblasts2', 'Myofibroblasts', 'LymphNodeFibroblasticReticularCells', 'ValveInterstitialCells', 'GreatVesselSmoothMuscleCells', 'CoronarySmoothMuscleCells', 'CoronaryPericytes', 'GreatVesselArterialEndothelialCells', 'GreatVesselVenousEndothelialCells', 'CoronaryArterialEndothelialCells', 'CoronaryVenousEndothelialCells', 'CoronaryCapillaryEndothelialCells', 'EndocardialCells', 'EndocardialCushionCells', 'ValveEndothelialCells', 'LymphaticEndothelialCells', 'MesothelialEpicardialCells', 'EpicardiumDerivedCells', 'NeuronPrecursors', 'ChromaffinCells', 'SympatheticNeurons', 'ParaympatheticNeurons', 'SchwannCellPrecursors', 'SchwannCells', 'MonocytesMPOpos', 'Monocytes', 'MonocyteDerivedCells', 'MacrophagesCX3CR1pos', 'MacrophagesTIMD4pos', 'MacrophagesLYVE1pos', 'MacrophagesATF3pos', 'DendriticCellsType1', 'DendriticCellsMature', 'PlasmacytoidDendriticCells', 'MastCells', 'Megakaryocytes', 'TCellsCD4pos', 'TCellsCD8pos', 'TregsCD4pos', 'ProBCells', 'BCells', 'BCellsMS4A1pos', 'NaturalKillerCells', 'InnateLymphoidCells'
    var: 'gene_ids', 'feature_types', 'genome', 'mt'
    uns: 'spatial'
    obsm: 'MT', 'means_cell_abundance_w_sf', 'q05_cell_abundance_w_sf', 'q95_cell_abundance_w_sf', 'spatial', 'stds_cell_abundance_w_sf'
```

Example 14 (unknown):
```unknown
slide_id = 'BHF_F_Hea13295831'
```

Example 15 (python):
```python
slide = select_multislide(adata_vis, [slide_id])
    
# plot in spatial coordinates
with mpl.rc_context({#'axes.facecolor':  'black',
                     'figure.figsize': [4.5, 5]}):

    sc.pl.spatial(slide, cmap='RdPu',
                  color=['JC_edit'], 
                  ncols=5, size=0, 
                  img_key='hires', alpha_img=1,wspace=0.3,
                  vmin=0, vmax='p99.5',# palette=palette
                  legend_loc=None,
                  title='H&E',
                  show=False
                 )
    plt.savefig(f"{figdir}/visium_HE_{slide_id}.pdf", bbox_inches="tight",dpi=300)
```

Example 16 (unknown):
```unknown
/tmp/ipykernel_46793/2805736760.py:18: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  slides.uns['spatial'] = uns_dict
```

Example 17 (unknown):
```unknown
slide_id = 'BHF_F_Hea13295830'
```

Example 18 (python):
```python
slide = select_multislide(adata_vis, [slide_id])
    
# plot in spatial coordinates
with mpl.rc_context({#'axes.facecolor':  'black',
                     'figure.figsize': [4.5, 5]}):

    sc.pl.spatial(slide, cmap='RdPu',
                  color=['JC_edit'], 
                  ncols=5, size=0, 
                  img_key='hires', alpha_img=1,wspace=0.3,
                  vmin=0, vmax='p99.5',# palette=palette
                  legend_loc=None,
                  title='H&E',
                  show=False
                 )
    plt.savefig(f"{figdir}/visium_HE_{slide_id}.pdf", bbox_inches="tight",dpi=300)
```

Example 19 (unknown):
```unknown
/tmp/ipykernel_46793/2805736760.py:18: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  slides.uns['spatial'] = uns_dict
```

Example 20 (python):
```python
slide = select_multislide(adata_vis, [slide_id])
    
# plot in spatial coordinates
with mpl.rc_context({#'axes.facecolor':  'black',
                     'figure.figsize': [4.5, 5]}):

    sc.pl.spatial(slide, cmap='RdPu',
                  color=['JC_edit'], 
                  ncols=5, size=0, 
                  img_key='hires', alpha_img=1,wspace=0.3,
                  vmin=0, vmax='p99.5',# palette=palette
                  crop_coord = [6000,12000,11500,16000],
                  legend_loc=None,
                  title='',
                  show=False,
                 )
    plt.savefig(f"{figdir}/visium_HE_cropped-LV_{slide_id}.pdf", bbox_inches="tight",dpi=300)
```

Example 21 (unknown):
```unknown
/tmp/ipykernel_46793/2805736760.py:18: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  slides.uns['spatial'] = uns_dict
```

Example 22 (python):
```python
clust_show = ['VentricularCardiomyocytesLeftCompact','VentricularCardiomyocytesLeftTrabeculated']
order_cmap = [4,1] # 0:yellow, 1:orange, 2:blue, 3:green, 4:purple, 5:grey, 6:white

plt.rcParams['axes.grid'] = False
with mpl.rc_context({"figure.figsize": (20,20)}):

        fig = plot_spatial(slide, 
                      coords=slide.obsm['spatial'], 
                      show_img=True, img_alpha=1,
                      max_color_quantile=0.99, 
                      circle_diameter=0, alpha_scaling=1, labels=clust_show, 
                      #style='dark_background', # fast dark_background
                         colorbar_position=None,
                         colorbar_label_kw={'size': 20, 'weight': 'bold', 'y': 1.29},
                         colorbar_shape={'vertical_gaps': 2, 'horizontal_gaps': 0, 
                                         'width': 0.2, 'height': 0.35},
                         colorbar_tick_size=20,
                           color = clust_show,
                            crop_x = (500,1350),
                           crop_y = (1250,1825)
                    )
        plt.savefig(f"{figdir}/visium_HE_cropped-LV_{slide_id}.pdf", bbox_inches="tight",dpi=300)
```

Example 23 (python):
```python
clust_show = ['VentricularCardiomyocytesLeftCompact','VentricularCardiomyocytesLeftTrabeculated']
order_cmap = [4,1] # 0:yellow, 1:orange, 2:blue, 3:green, 4:purple, 5:grey, 6:white

plt.rcParams['axes.grid'] = False
with mpl.rc_context({"figure.figsize": (20,20)}):

        fig = plot_spatial(slide, 
                      coords=slide.obsm['spatial'], 
                      show_img=True, img_alpha=0.3,
                      max_color_quantile=0.99, 
                      circle_diameter=8, alpha_scaling=1, labels=clust_show, 
                      #style='dark_background', # fast dark_background
                         colorbar_position='right',
                         colorbar_label_kw={'size': 20, 'weight': 'bold', 'y': 1.29},
                         colorbar_shape={'vertical_gaps': 0.3, 'horizontal_gaps': 0.4, 
                                         'width': 0.2, 'height': 0.1},
                         colorbar_tick_size=20,
                           color = clust_show,
                           reorder_cmap = order_cmap,
                            crop_x = (500,1350),
                           crop_y = (1250,1825)
                    )
        plt.savefig(f"{figdir}/visium_cell2loc-outs_cropped-LV_{slide_id}.pdf", bbox_inches="tight",dpi=300)
```

---

## Import libraries¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig4_Cardiomyocytes/AtrialCardiomyocytes/CellOracle/CellOracle_no1-2_in-silico-perturbatio-prep.html

**Contents:**
- Import libraries¶
- Make out directory¶
- Load oracle object¶
- In silico TF perturbation analysis¶
- Search for best min_mass.¶
- Calculating pseudotime gradient vectors¶
- Perturbation¶

ref: https://morris-lab.github.io/CellOracle.documentation/notebooks/04_Network_analysis/Network_analysis_with_Paul_etal_2015_data.html

**Examples:**

Example 1 (unknown):
```unknown
# Parameters
# working_dir = "/nfs/team205/heart/Vitro/TF_selection/foetal_vCM_left_compact-vs-trabeculated_downstream/CO_in-silico-perturbation_papermill"
path_to_oracle_object = "/nfs/team205/heart/anndata_objects/Foetal/celloracle/foetal_AtrialCM.celloracle.oracle"
path_to_oracle_links = "/nfs/team205/heart/anndata_objects/Foetal/celloracle/foetal_AtrialCM_links.celloracle.links"

# path_to_TFs = 

clusters = ['AtrialCardiomyocytesLeft_t1',
 'AtrialCardiomyocytesLeft_t2',
 'AtrialCardiomyocytesLeft_t3']
cell_group_name = "AtrialCM-Left"

path_to_res_dict = f"/nfs/team205/heart/anndata_objects/Foetal/celloracle/{cell_group_name}_in-silico_perturbation_May2025_2.pkl"
path_to_plots = f"/nfs/team205/heart/anndata_objects/Foetal/celloracle/figures_pm/{cell_group_name}_May2025_2"
```

Example 2 (python):
```python
import os
import sys

import matplotlib
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import scanpy as sc
import seaborn as sns
```

Example 3 (python):
```python
import celloracle as co
co.__version__
```

Example 4 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 5 (python):
```python
sc.settings.set_figure_params(dpi=120)
```

Example 6 (python):
```python
if not os.path.exists(path_to_plots):
    # Create the directory
    os.makedirs(path_to_plots)
    print("Directory created successfully")
else:
    print("Directory already exists")
```

Example 7 (unknown):
```unknown
Directory already exists
```

Example 8 (unknown):
```unknown
# Load
oracle = co.load_hdf5(path_to_oracle_object)
```

Example 9 (python):
```python
n_cell = oracle.adata.shape[0]
print(f"cell number is :{n_cell}")

k = int(0.025*n_cell)
print(f"Auto-selected k is :{k}")
```

Example 10 (unknown):
```unknown
cell number is :5664
Auto-selected k is :141
```

Example 11 (python):
```python
%matplotlib inline
# Check clustering data
sc.pl.umap(oracle.adata, color='fine_grain_time-binned')
```

Example 12 (python):
```python
# Read in 
regulons = utils.read_pkl('/nfs/team205/heart/pyscenic/plus/baseGRN_forCO/AtrialCM_baseGRN_act.pkl')
list_of_TFs = list(regulons.keys())

# select TFs which are in the oracle object
list_of_TFs = [x for x in list_of_TFs if x in oracle.adata.var_names]

print(len(list_of_TFs))
list_of_TFs[:5]
```

Example 13 (unknown):
```unknown
['ZNF730', 'PHF21A', 'CREB1', 'CREB3L4', 'MXD1']
```

Example 14 (python):
```python
'''
# Check gene expressions
# will be saved to a separated file
sc.settings.set_figure_params(dpi=80)
sc.pl.draw_graph(oracle.adata, color=list_of_TFs, ncols=6,
                 layer="imputed_count", use_raw=False, cmap="RdPu",
                 show=False, save='_TF_expression.png'
                )
'''
```

Example 15 (python):
```python
'\n# Check gene expressions\n#\xa0will be saved to a separated file\nsc.settings.set_figure_params(dpi=80)\nsc.pl.draw_graph(oracle.adata, color=list_of_TFs, ncols=6,\n                 layer="imputed_count", use_raw=False, cmap="RdPu",\n                 show=False, save=\'_TF_expression.png\'\n                )\n'
```

Example 16 (unknown):
```unknown
n_grid = 40
n_neighbors = 100
```

Example 17 (python):
```python
# Check gene expression
goi = "PITX2"
sc.pl.umap(oracle.adata, color=[goi, oracle.cluster_column_name],
                 layer="imputed_count", use_raw=False, cmap="viridis")
```

Example 18 (unknown):
```unknown
# Enter perturbation conditions to simulate signal propagation after the perturbation.
oracle.simulate_shift(perturb_condition={goi: 0.0},
                      n_propagation=3)
```

Example 19 (unknown):
```unknown
# Get transition probability
oracle.estimate_transition_prob(n_neighbors=n_neighbors,
                                knn_random=True,
                                sampled_fraction=1)

# Calculate embedding
oracle.calculate_embedding_shift(sigma_corr=0.05)
```

Example 20 (unknown):
```unknown
oracle.calculate_p_mass(smooth=0.8, n_grid=n_grid, n_neighbors=n_neighbors)
```

Example 21 (unknown):
```unknown
oracle.suggest_mass_thresholds(n_suggestion=40)
```

Example 22 (unknown):
```unknown
min_mass = 31
scale_simulation = 5
```

Example 23 (python):
```python
%matplotlib inline
```

Example 24 (python):
```python
from celloracle.applications import Gradient_calculator
# Instantiate Gradient calculator object
gradient = Gradient_calculator(oracle_object=oracle, pseudotime_key="t")
gradient.calculate_p_mass(smooth=0.8, n_grid=n_grid, n_neighbors=n_neighbors)
gradient.calculate_mass_filter(min_mass=min_mass, plot=True)

# convert the pseudotime data into grid points.
# For this calculation we can chose one of two methods.
gradient.transfer_data_into_grid(args={"method": "knn", "n_knn":50})

# Calculate graddient
gradient.calculate_gradient()
# Show results
scale_dev = 40
gradient.visualize_results(scale=scale_dev, s=5)
# Visualize results
fig, ax = plt.subplots(figsize=[6, 6])
gradient.plot_dev_flow_on_grid(scale=scale_dev, ax=ax)
```

Example 25 (python):
```python
from tqdm import tqdm
```

Example 26 (unknown):
```unknown
list_of_TFs = ['PITX2']
```

Example 27 (python):
```python
from celloracle.applications import Oracle_development_module

# not showing plots
matplotlib.use('Agg')

# Get cell index list for the cells of interest
cell_idx = np.where(oracle.adata.obs["fine_grain_time-binned"].isin(clusters))[0]
# Check index
print(cell_idx)

dev_dict = {}
for goi in tqdm(list_of_TFs):
    # Enter perturbation conditions to simulate signal propagation after the perturbation.
    try:
        oracle.simulate_shift(perturb_condition={goi: 0.0}, # this is KO simulation so expression=0
                          n_propagation=3)
    except:
        print(f'skippint {goi} due to an error')
        continue
    
    # Get transition probability
    oracle.estimate_transition_prob(n_neighbors=n_neighbors,
                                    knn_random=True,
                                    sampled_fraction=1)

    # Calculate embedding
    oracle.calculate_embedding_shift(sigma_corr=0.05)
    
    # prepare grid
    oracle.calculate_p_mass(smooth=0.8, n_grid=n_grid, n_neighbors=n_neighbors)
    oracle.calculate_mass_filter(min_mass=min_mass, plot=False)
    
    # plot vector field
    fig, ax = plt.subplots(1, 2,  figsize=[13, 6])
    # Show quiver plot
    oracle.plot_simulation_flow_on_grid(scale=scale_simulation, ax=ax[0])
    ax[0].set_title(f"Simulated cell identity shift vector: {goi} KO")
    # Show quiver plot that was calculated with randomized graph.
    oracle.plot_simulation_flow_random_on_grid(scale=scale_simulation, ax=ax[1])
    ax[1].set_title(f"Randomized simulation vector")
    plt.savefig(f'{path_to_plots}/{goi}_simulation_vector.pdf')
    
    # Plot vector field with cell cluster
    fig, ax = plt.subplots(figsize=[8, 8])
    oracle.plot_cluster_whole(ax=ax, s=10)
    oracle.plot_simulation_flow_on_grid(scale=scale_simulation, ax=ax, show_background=False)
    plt.savefig(f'{path_to_plots}/{goi}_simulation_vector_with-clusters.pdf')
    
    
    dev = Oracle_development_module()
    # Load development flow
    dev.load_differentiation_reference_data(gradient_object=gradient)

    # Load simulation result
    dev.load_perturb_simulation_data(oracle_object=oracle,
                                     cell_idx_use=cell_idx, # Enter cell id list
                                     name=cell_group_name # Name of this cell group. You can enter any name.
                                     )

    # Calculation
    dev.calculate_inner_product()
    dev.calculate_digitized_ip(n_bins=10)
    
    # save to the dictionary
    dev_dict[goi] = dev.copy()
    
    # Let's visualize the results
    vm = 0.01
    dev.visualize_development_module_layout_0(s=5,
                                              scale_for_simulation=scale_simulation,
                                              s_grid=50,
                                              scale_for_pseudotime=scale_dev,
                                              vm=vm)
    plt.savefig(f'{path_to_plots}/{goi}_results.pdf')
```

Example 28 (unknown):
```unknown
[   1    2    4 ... 5659 5662 5663]
```

Example 29 (unknown):
```unknown
0%|          | 0/1 [00:00<?, ?it/s]2025-05-08 22:24:04,677 - INFO - Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
2025-05-08 22:24:04,686 - INFO - Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
100%|██████████| 1/1 [05:59<00:00, 359.94s/it]
```

Example 30 (python):
```python
%matplotlib inline
```

Example 31 (unknown):
```unknown
dev.visualize_development_module_layout_0(s=5,
                                          scale_for_simulation=scale_simulation,
                                          s_grid=50,
                                          scale_for_pseudotime=40,
                                          vm=vm)
```

Example 32 (unknown):
```unknown
2025-05-08 21:38:15,504 - INFO - Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
2025-05-08 21:38:15,513 - INFO - Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 33 (unknown):
```unknown
?dev.visualize_development_module_layout_0
```

Example 34 (unknown):
```unknown
Signature:
dev.visualize_development_module_layout_0(
    scale_for_pseudotime=30,
    scale_for_simulation=30,
    s=5,
    s_grid=20,
    vm=1,
    show_background=True,
    return_fig=False,
)
Docstring: <no docstring>
File:      /software/cellgen/team205/kk18/envs/celloracle_env/lib/python3.10/site-packages/celloracle/applications/development_module.py
Type:      method
```

Example 35 (unknown):
```unknown
oracle.calculate_p_mass(smooth=0.8, n_grid=40, n_neighbors=n_neighbors)
oracle.calculate_mass_filter(min_mass=min_mass, plot=True)
```

Example 36 (python):
```python
# plot vector field
fig, ax = plt.subplots(1, 2,  figsize=[13, 6])
scale_simulation = 5
# Show quiver plot
oracle.plot_simulation_flow_on_grid(scale=scale_simulation, ax=ax[0])
ax[0].set_title(f"Simulated cell identity shift vector: {goi} KO")
# Show quiver plot that was calculated with randomized graph.
oracle.plot_simulation_flow_random_on_grid(scale=scale_simulation, ax=ax[1])
ax[1].set_title(f"Randomized simulation vector")
plt.show()
```

Example 37 (python):
```python
# Plot vector field with cell cluster
fig, ax = plt.subplots(figsize=[8, 8])

oracle.plot_cluster_whole(ax=ax, s=10)
oracle.plot_simulation_flow_on_grid(scale=scale_simulation, ax=ax, show_background=False)
```

Example 38 (python):
```python
oracle.adata
```

Example 39 (unknown):
```unknown
AnnData object with n_obs × n_vars = 5664 × 10870
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', 'age_days', 'age_days_smoothed_once', 'age_days_normalized', 'age_days_normalized_smoothed_once', 'age_days_smoothed_iteratively', 'age_days_normalized_smoothed_iteratively', 't', 'seg', 'edge', 't_sd', 'milestones', 't_binned', 'fine_grain_time-binned'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id', 'n_cells', 'symbol', 'isin_top1000_var_mean_genes', 'isin_TFdict_targets', 'isin_TFdict_regulators', 'isin_actve_regulators'
    uns: 'donor_colors', 'fine_grain_colors', 'graph', 'milestones_colors', 'neighbors', 'ppt', 'pseudotime_list', 'seg_colors', 'umap', 'week_colors', 'log1p', 'fine_grain_time-binned_colors'
    obsm: 'X_R', 'X_multiVI', 'X_umap'
    layers: 'raw_count', 'normalized_count', 'imputed_count', 'simulation_input', 'simulated_count', 'delta_X'
    obsp: 'connectivities', 'distances'
```

Example 40 (python):
```python
sc.pl.umap(adata=oracle.adata, cmap="rainbow",
                color=["t"])
```

Example 41 (unknown):
```unknown
# save
utils.save_pkl(dev_dict, path_to_res_dict)
```

---

## Import modules¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig3_SAN_niche/cellphonedb-VisiumHD_no3_plotting_SAN_to-ParaN.html

**Contents:**
- Import modules¶
- Variables¶
- Read in input adata¶
- Read in cpdb outputs¶
- Plot¶
  - Receptor expressed in PSNS¶
- Slect interactions specific to Sinus Horn and SAN¶

**Examples:**

Example 1 (python):
```python
import scanpy as sc
import pandas as pd
pd.set_option('display.max_rows', 100)
import numpy as np
import matplotlib.pyplot as plt
import matplotlib as mpl
import seaborn as sns

import ktplotspy as kpy

import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 

import os 
from matplotlib.patches import Rectangle
import itertools
```

Example 2 (python):
```python
sc.settings.set_figure_params(dpi=80,dpi_save=300)
```

Example 3 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk/Aug2024'
table_dir = '/nfs/team205/heart/fetal_ms_figures/tables/Aug2024'
```

Example 4 (python):
```python
niche_of_interest = 'SAnode'
path_to_cpdb = f'/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/cellphonedb/HEA_FOET14880396/{niche_of_interest}'
path_to_adata = f'/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/cellphonedb/HEA_FOET14880396/inputs/log_norm_counts_{niche_of_interest}.h5ad'
celltype_col = 'fine_grain_4Jan2025'
```

Example 5 (python):
```python
# read in adata
adata = sc.read_h5ad(path_to_adata)
adata
```

Example 6 (unknown):
```unknown
AnnData object with n_obs × n_vars = 1289 × 18047
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'fine_grain_4Jan2025', 'HistologicalAnnotation', 'cluster_cellcharter_k-8', 'cluster_cellcharter_k-10', 'cluster_cellcharter_k-12', 'cluster_cellcharter_k-14', 'cluster_cellcharter_k-16', 'cluster_cellcharter_k-18', 'cluster_cellcharter_k-20', 'cluster_cellcharter_k-25', 'cluster_cellcharter_k-30', 'cluster_cellcharter_k-35', 'cluster_cellcharter_k-40', 'cluster_cellcharter_k-45', 'cluster_cellcharter_k-50', 'predicted_labels', 'conf_score', 'predicted_labels_cellcharter-cluster-k50', 'conf_score_cellcharter-cluster-k50', 'spatial_leiden_1', 'spatial_leiden_5', 'spatial_leiden_0.5', 'spatial_leiden_2', 'spatial_leiden_10', 'spatial_leiden_100', 'spatial_leiden_1000', 'KMeans_k9216_cluster', 'predicted_labels_spatial_KMeans_k9216', 'conf_score_spatial_KMeans_k9216', 'KMeans_k4608_cluster', 'predicted_labels_spatial_KMeans_k4608', 'conf_score_spatial_KMeans_k4608', 'KMeans_k3072_cluster', 'KMeans_k1843_cluster', 'predicted_labels_spatial_KMeans_k3072', 'conf_score_spatial_KMeans_k3072', 'predicted_labels_spatial_KMeans_k1843', 'conf_score_spatial_KMeans_k1843', 'sliding_window_assignment', 'predicted_labels_sliding_window_assignment', 'conf_score_sliding_window_assignment', 'tt_prediction'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'HistologicalAnnotation_colors', 'KMeans_k1843_centroids', 'KMeans_k3072_centroids', 'KMeans_k4608_centroids', 'KMeans_k9216_centroids', 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'cluster_cellcharter_k-10_colors', 'cluster_cellcharter_k-12_colors', 'cluster_cellcharter_k-14_colors', 'cluster_cellcharter_k-16_colors', 'cluster_cellcharter_k-18_colors', 'cluster_cellcharter_k-20_colors', 'cluster_cellcharter_k-25_colors', 'cluster_cellcharter_k-30_colors', 'cluster_cellcharter_k-35_colors', 'cluster_cellcharter_k-40_colors', 'cluster_cellcharter_k-45_colors', 'cluster_cellcharter_k-50_colors', 'cluster_cellcharter_k-8_colors', 'conf_score_cellcharter-cluster-k50_colors', 'conf_score_sliding_window_assignment_colors', 'fine_grain_4Jan2025_colors', 'hvg', 'log1p', 'neighbors', 'pca', 'predicted_labels_cellcharter-cluster-k50_colors', 'predicted_labels_colors', 'predicted_labels_spatial_KMeans_k1843_colors', 'predicted_labels_spatial_KMeans_k3072_colors', 'predicted_labels_spatial_KMeans_k4608_colors', 'predicted_labels_spatial_KMeans_k9216_colors', 'spatial', 'spatial_leiden_0.5', 'spatial_leiden_1', 'spatial_leiden_10', 'spatial_leiden_100', 'spatial_leiden_1000', 'spatial_leiden_1_colors', 'spatial_leiden_2', 'spatial_leiden_5', 'spatial_leiden_5_colors', 'spatial_neighbors', 'tt_prediction_colors', 'umap'
    obsm: 'X_cellcharter', 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances', 'spatial_connectivities', 'spatial_distances'
```

Example 7 (unknown):
```unknown
timestamp = '05_04_2025_105025'
timestamp
```

Example 8 (unknown):
```unknown
'05_04_2025_105025'
```

Example 9 (python):
```python
# read in
cpdbout_means =  pd.read_table(f'{path_to_cpdb}/statistical_analysis_means_{timestamp}.txt',index_col=0)
cpdbout_pvals =  pd.read_table(f'{path_to_cpdb}/statistical_analysis_pvalues_{timestamp}.txt',index_col=0)
cpdbout_sigmeans =  pd.read_table(f'{path_to_cpdb}/statistical_analysis_significant_means_{timestamp}.txt',index_col=0)
interaction_metadata = pd.read_csv('/nfs/team205/heart/cellphonedb/customDB/interaction_meta/interaction_metadata_v5.0.0_2024-1-26.csv')
```

Example 10 (python):
```python
# add gene names to cpdb outputs
cpdbout_means['gene_name_1'] = list(interaction_metadata.set_index('id_cp_interaction').loc[cpdbout_means.index,'gene_name_1'])
cpdbout_means['gene_name_2'] = list(interaction_metadata.set_index('id_cp_interaction').loc[cpdbout_means.index,'gene_name_2'])
cpdbout_pvals['gene_name_1'] = list(interaction_metadata.set_index('id_cp_interaction').loc[cpdbout_pvals.index,'gene_name_1'])
cpdbout_pvals['gene_name_2'] = list(interaction_metadata.set_index('id_cp_interaction').loc[cpdbout_pvals.index,'gene_name_2'])
cpdbout_sigmeans['gene_name_1'] = list(interaction_metadata.set_index('id_cp_interaction').loc[cpdbout_sigmeans.index,'gene_name_1'])
cpdbout_sigmeans['gene_name_2'] = list(interaction_metadata.set_index('id_cp_interaction').loc[cpdbout_sigmeans.index,'gene_name_2'])
```

Example 11 (python):
```python
# insert gene_name columns to the cpdb output
# to select interactions based on the list
cpdbout_means[['gene_name_1','gene_name_2']] = interaction_metadata.set_index('id_cp_interaction').loc[cpdbout_means.index,['gene_name_1','gene_name_2']]
cpdbout_pvals[['gene_name_1','gene_name_2']] = interaction_metadata.set_index('id_cp_interaction').loc[cpdbout_pvals.index,['gene_name_1','gene_name_2']]
cpdbout_sigmeans[['gene_name_1','gene_name_2']] = interaction_metadata.set_index('id_cp_interaction').loc[cpdbout_sigmeans.index,['gene_name_1','gene_name_2']]
```

Example 12 (python):
```python
# query a gene, to check of the existence of the its interactions in the database
gene = 'FGF10'
mask1 = [gene in x.split(',') for x in interaction_metadata['gene_name_1']]
mask2 = [gene in x.split(',') for x in interaction_metadata['gene_name_2']]
mask = [any(x) for x in zip(mask1,mask2)]
interaction_metadata[mask][['id_cp_interaction','gene_name_1','gene_name_2']]
```

Example 13 (unknown):
```unknown
receptor_genes = [
    # Netrin receptors
    "DCC", "NEO1", "UNC5A", "UNC5B", "UNC5C", "UNC5D",
    # Slit (Robo) receptors
    "ROBO1", "ROBO2", "ROBO3", "ROBO4",
    # Semaphorin receptors
    "NRP1", "NRP2",
    "PLXNA1", "PLXNA2", "PLXNA3", "PLXNA4",
    "PLXNB1", "PLXNB2", "PLXNB3",
    "PLXNC1", "PLXND1",
    # Ephrin (Eph) receptors
    "EPHA1", "EPHA2", "EPHA3", "EPHA4", "EPHA5", "EPHA6", "EPHA7", "EPHA8", "EPHA10",
    "EPHB1", "EPHB2", "EPHB3", "EPHB4", "EPHB6",
    # Neurotrophin / GDNF family receptors
    "NTRK1", "NTRK2", "NTRK3", "NGFR", "RET",
    "GFRA1", "GFRA2", "GFRA3", "GFRA4",
    # Neuregulin (ErbB) receptors
    "ERBB2", "ERBB3", "ERBB4",
    # Additional innervation‑related receptors
    "EDNRB", "LRP8", "VLDLR", "L1CAM", "NCAM1",
    # Others
    "CXCR4","CNTNAP4","FGFR2"
]
receptor_celltype = 'ParaN'

# ligand_genes = ['SEMA3A', 'SLIT2', 'SLIT3', 'EFNB2', 'CXCL12', 'FGF10', 'VEGFC', 'NELL1']
ligand_celltypes = ['SHPC','SANPCHd','SANPCTl','aCMR']
```

Example 14 (python):
```python
def receiving_signal(
    cpdbout_means,
    cpdbout_pvals,
    receptor_celltype,
    receptor_genes,
    ligand_celltypes=None
):
    # Subset results
    
    ### get columns
    cellpair_cols = [x for x in cpdbout_means.columns if '|' in x]
    other_cols = [x for x in cpdbout_means.columns if '|' not in x]
    
    ### get dataframe which has receptor_celltype and the receptor_genes at the "b/2" side (majority will be like this)
    mask_row = (
        cpdbout_means['gene_name_2']
        .str.split(',')
        .apply(lambda names: any(g in receptor_genes for g in names))
    )
    if ligand_celltypes!=None:
        cellpair_cols_sel = [x for x in cellpair_cols if (f'|{receptor_celltype}' in x)&(x.split('|')[0] in ligand_celltypes)]
    else:
        cellpair_cols_sel = [x for x in cellpair_cols if (f'|{receptor_celltype}' in x)]
    col_sel = other_cols + cellpair_cols_sel
    means_2 = cpdbout_means.loc[mask_row, col_sel]
    pvals_2 = cpdbout_pvals.loc[mask_row, col_sel]

    ### get dataframe which has receptor_celltype and the receptor_genes at the "a/1" side
    mask_row = (
        cpdbout_means['gene_name_1']
        .str.split(',')
        .apply(lambda names: any(g in receptor_genes for g in names))
    )
    if ligand_celltypes!=None:
        cellpair_cols_sel = [x for x in cellpair_cols if (f'{receptor_celltype}|' in x)&(x.split('|')[1] in ligand_celltypes)]
    else:
        cellpair_cols_sel = [x for x in cellpair_cols if (f'{receptor_celltype}|' in x)]
    col_sel = other_cols + cellpair_cols_sel
    means_1 = cpdbout_means.loc[mask_row, col_sel]
    pvals_1 = cpdbout_pvals.loc[mask_row, col_sel]
    # remove rows which are already exist in means_2
    index_keep = list(set(means_1.index)-set(means_2.index))
    means_1 = means_1.loc[index_keep]
    pvals_1 = pvals_1.loc[index_keep]
    # reverse gene_a and gene_b
    mapper = {'partner_a':'partner_b',
              'partner_b':'partner_a',
              'gene_a':'gene_b',
              'gene_b':'gene_a',
              'receptor_a':'receptor_b',
              'receptor_b':'receptor_a',
              'directionality':'directionality_reverse'}
    means_1.rename(columns=mapper,inplace=True)
    pvals_1.rename(columns=mapper,inplace=True)

    new_columns = []
    for col in means_1.columns:
        if '|' in col:
            cell_a = col.split('|')[0]
            cell_b = col.split('|')[1]
            new_columns.append(f'{cell_b}|{cell_a}')
        else:
            new_columns.append(col)
    means_1.columns = new_columns
    pvals_1.columns = new_columns

    new_interacting_pair = []
    for ind in means_1.index:
        partner_a = means_1.loc[ind,'partner_a']
        partner_b = means_1.loc[ind,'partner_b']
        if 'complex:' in partner_a:
            partner_a = partner_a.replace('complex:','')
        else:
            partner_a = means_1.loc[ind,'gene_a']
        if 'complex:' in partner_b:
            partner_b = partner_b.replace('complex:','')
        else:
            partner_b = means_1.loc[ind,'gene_b']
        new_interacting_pair.append(f'{partner_a}_{partner_b}')
    means_1['interacting_pair'] = new_interacting_pair
    pvals_1['interacting_pair'] = new_interacting_pair

    ### concatenate
    means = pd.concat([means_2,means_1])
    pvals = pd.concat([pvals_2,pvals_1])

    ### remove NaN
    means = means.dropna(axis=0, how='all')
    means = means.dropna(axis=1, how='all')
    pvals = pvals.dropna(axis=0, how='all')
    pvals = pvals.dropna(axis=1, how='all')

    ### formatting
    means = means.reset_index()
    pvals = pvals.reset_index()
    means.drop(labels=['gene_name_1','gene_name_2','directionality_reverse'],axis=1,inplace=True)
    pvals.drop(labels=['gene_name_1','gene_name_2','directionality_reverse'],axis=1,inplace=True)
    
    return means, pvals
```

Example 15 (python):
```python
means,pvals = receiving_signal(
    cpdbout_means,
    cpdbout_pvals,
    receptor_celltype,
    receptor_genes,
    ligand_celltypes=ligand_celltypes
)

# reorder interaction columns
non_interaction_columns = [x for x in means.columns if '|' not in x]
interaction_columns = [f'{x}|ParaN' for x in ligand_celltypes[::-1]]
means = means[non_interaction_columns+interaction_columns]
pvals = pvals[non_interaction_columns+interaction_columns]
interaction_columns = [f'{i}_{x}|ParaN' for i,x in enumerate(ligand_celltypes[::-1])]
means.columns = non_interaction_columns+interaction_columns
pvals.columns = non_interaction_columns+interaction_columns

# rename cell types of interest for regulating the order
for i,cell in enumerate(ligand_celltypes[::-1]):
    adata.obs.replace({celltype_col:{cell:f'{i}_{cell}'}},inplace=True)

plot = kpy.plot_cpdb(
    adata=adata,
    cell_type1='.',
    cell_type2='.',  # this means all cell-types
    means=means,
    pvals=pvals,
    alpha=0.1,
    celltype_key=celltype_col,
    genes=None,
    figsize=(5.2, 9),
    title='SAN, CMs-to-ParaN,\nall AxonG',
    highlight_size =1.5,
    standard_scale=False
)
plot.save(f"{figdir}/VisiumHD_CellPhoneDB_SAN_CM-AxonG-to-ParaN_all-ligands.pdf", bbox_inches="tight",dpi=300)
plot
```

Example 16 (unknown):
```unknown
/software/cellgen/team205/kk18/envs/cpdb/lib/python3.8/site-packages/plotnine/ggplot.py:587: PlotnineWarning: Saving 5.2 x 9 in image.
/software/cellgen/team205/kk18/envs/cpdb/lib/python3.8/site-packages/plotnine/ggplot.py:588: PlotnineWarning: Filename: /nfs/team205/heart/fetal_ms_figures/raw_plots_kk/Aug2024/VisiumHD_CellPhoneDB_SAN_CM-AxonG-to-ParaN_all-ligands.pdf
```

Example 17 (unknown):
```unknown
<Figure Size: (520 x 900)>
```

Example 18 (unknown):
```unknown
# select interactions only significant at SHPC or SANPCs
pval_thresh = 0.1
mask = pvals['0_aCMR|ParaN'] > pval_thresh
means = means.loc[mask]
pvals = pvals.loc[mask]
```

Example 19 (python):
```python
plot = kpy.plot_cpdb(
    adata=adata,
    cell_type1='.',
    cell_type2='.',  # this means all cell-types
    means=means,
    pvals=pvals,
    alpha=0.1,
    celltype_key=celltype_col,
    genes=None,
    figsize=(5.2, 5.5),
    title='SAN, CMs-to-ParaN,\nall AxonG',
    highlight_size =1.5,
    standard_scale=False
)
plot.save(f"{figdir}/VisiumHD_CellPhoneDB_SAN_CM-AxonG-to-ParaN_selected.pdf", bbox_inches="tight",dpi=300)
plot
```

Example 20 (unknown):
```unknown
/software/cellgen/team205/kk18/envs/cpdb/lib/python3.8/site-packages/plotnine/ggplot.py:587: PlotnineWarning: Saving 5.2 x 5.5 in image.
/software/cellgen/team205/kk18/envs/cpdb/lib/python3.8/site-packages/plotnine/ggplot.py:588: PlotnineWarning: Filename: /nfs/team205/heart/fetal_ms_figures/raw_plots_kk/Aug2024/VisiumHD_CellPhoneDB_SAN_CM-AxonG-to-ParaN_selected.pdf
```

Example 21 (unknown):
```unknown
<Figure Size: (520 x 550)>
```

---

## scFates - Tree analysis¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig4_Cardiomyocytes/AtrialCardiomyocytes/3_compute_pseudotime.html

**Contents:**
- scFates - Tree analysis¶
- Set up¶
- Functions¶
- Paths¶
- Import adata¶
- Actual time - kNN smoothing¶
- scVI¶
- Palantir¶
  - Pseudotime - scFates¶
- Compare real and pseudo-time¶
- Save object with UMAP, time and pseudotiem computed¶

ref: https://scfates.readthedocs.io/en/latest/Tree_Analysis_Bone_marrow_fates.html

**Examples:**

Example 1 (python):
```python
#Sets the 'R_HOME' environment variable to the location where Conda installed R libraries.
#You should run this code at the beginning of your Jupyter notebook, before importing any modules that rely on rpy2 or scFates.
#This will set the 'R_HOME' environment variable for the duration of your Python session
import os, sys
os.environ['R_HOME'] = sys.exec_prefix+"/lib/R/"
import scFates
```

Example 2 (python):
```python
import sys
!{sys.executable} -m pip -q install palantir fa2
```

Example 3 (python):
```python
from anndata import AnnData
import numpy as np
import pandas as pd
import scanpy as sc
import scFates as scf
import palantir
import matplotlib.pyplot as plt
import glob

sc.settings.verbosity = 3
sc.settings.logfile = sys.stdout
## fix palantir breaking down some plots
import seaborn
seaborn.reset_orig()
%matplotlib inline
import os

import warnings
warnings.filterwarnings('ignore')

sc.set_figure_params()
scf.set_figure_pubready()
```

Example 4 (python):
```python
import session_info
session_info.show()
```

Example 5 (python):
```python
-----
anndata             0.9.1
matplotlib          3.7.1
numpy               1.24.3
palantir            1.2
pandas              1.5.3
scFates             1.0.2
scanpy              1.9.3
seaborn             0.12.2
session_info        1.0.0
-----
```

Example 6 (python):
```python
PIL                 9.5.0
adjustText          0.8
anyio               NA
arrow               1.2.3
asttokens           NA
attr                23.1.0
babel               2.12.1
backcall            0.2.0
backports           NA
certifi             2023.05.07
cffi                1.15.1
charset_normalizer  3.1.0
comm                0.1.3
cycler              0.10.0
cython_runtime      NA
dateutil            2.8.2
debugpy             1.6.7
decorator           5.1.1
defusedxml          0.7.1
executing           1.2.0
fastjsonschema      NA
fcsparser           0.2.6
fqdn                NA
h5py                3.8.0
idna                3.4
igraph              0.10.4
importlib_metadata  NA
importlib_resources NA
ipykernel           6.23.1
ipython_genutils    0.2.0
ipywidgets          8.0.6
isoduration         NA
jedi                0.18.2
jinja2              3.1.2
joblib              1.2.0
json5               NA
jsonpointer         2.3
jsonschema          4.17.3
jupyter_events      0.6.3
jupyter_server      2.6.0
jupyterlab_server   2.23.0
kiwisolver          1.4.4
leidenalg           0.9.1
llvmlite            0.40.0
markupsafe          2.1.2
matplotlib_inline   0.1.6
mpl_toolkits        NA
natsort             8.3.1
nbformat            5.9.0
networkx            2.8.8
numba               0.57.0
numexpr             2.8.4
overrides           NA
packaging           23.1
parso               0.8.3
patsy               0.5.3
pexpect             4.8.0
phenograph          1.5.7
pickleshare         0.7.5
pkg_resources       NA
platformdirs        3.5.1
plotly              5.14.1
progressbar         4.2.0
prometheus_client   NA
prompt_toolkit      3.0.38
psutil              5.9.5
ptyprocess          0.7.0
pure_eval           0.2.2
pydev_ipython       NA
pydevconsole        NA
pydevd              2.9.5
pydevd_file_utils   NA
pydevd_plugins      NA
pydevd_tracing      NA
pygam               0.8.0
pygments            2.15.1
pyparsing           3.0.9
pyrsistent          NA
python_utils        NA
pythonjsonlogger    NA
pytz                2023.3
requests            2.31.0
rfc3339_validator   0.1.4
rfc3986_validator   0.1.1
rpy2                3.5.11
scipy               1.10.1
send2trash          NA
setuptools          67.7.2
simpleppt           1.1.3
six                 1.16.0
sklearn             1.2.2
skmisc              0.2.0
sniffio             1.3.0
stack_data          0.6.2
statsmodels         0.14.0
texttable           1.6.7
threadpoolctl       3.1.0
tornado             6.3.2
tqdm                4.65.0
traitlets           5.9.0
typing_extensions   NA
tzlocal             NA
uri_template        NA
urllib3             2.0.3
wcwidth             0.2.6
webcolors           1.13
websocket           1.5.2
yaml                6.0
zipp                NA
zmq                 25.1.0
```

Example 7 (unknown):
```unknown
-----
IPython             8.12.2
jupyter_client      8.2.0
jupyter_core        5.3.0
jupyterlab          4.0.2
notebook            6.5.4
-----
Python 3.8.16 | packaged by conda-forge | (default, Feb  1 2023, 16:01:55) [GCC 11.3.0]
Linux-5.4.0-162-generic-x86_64-with-glibc2.10
-----
Session information updated at 2023-10-24 21:11
```

Example 8 (python):
```python
sc.settings.verbosity = 3             # verbosity: errors (0), warnings (1), info (2), hints (3)
sc.settings.set_figure_params(dpi=80, facecolor='white', color_map = 'RdPu',figsize=(4,4))
```

Example 9 (python):
```python
from datetime import datetime

today = datetime.today().strftime('%Y-%m-%d')
print(today)
```

Example 10 (python):
```python
def plot_and_store(init_pos, adata):
    """
    This function selects the columns of the obsm layer specified by init_pos, 
    draws a graph using these coordinates as the initial positions, and stores the graph 
    coordinates in a new obsm layer named 'X_draw_graph_fa_' followed by the value of init_pos. 
    The modified AnnData object is then returned.

    If adata.obsm[f'X_draw_graph_fa_{init_pos}'] already exists, it puts that in adata.obsm['X_draw_graph_fa'].
    If adata.obsm[f'X_draw_graph_fa_{init_pos}'] does not exist, it performs sc.tl.draw_graph() as before.

    Parameters
    ----------
    init_pos : str
        Initial position for the graph. Should be in the format 'X_palantir12'.
    
    adata : AnnData
        The annotated data matrix of shape n_obs x n_vars. Rows correspond
        to cells and columns to genes.

    Returns
    -------
    adata : AnnData
        The modified annotated data matrix.
    """
    x_col = int(init_pos[-2:][0]) - 1
    y_col = int(init_pos[-2:][1]) - 1
    selected_cols = [x_col, y_col]
    adata.obsm[f'{init_pos}'] = adata.obsm[f'{init_pos[:-2]}'][:,selected_cols]
    
    if f'X_draw_graph_fa_{init_pos}' in adata.obsm:
        print(f'graph using {init_pos} has previously been calculated, retrieving that...')
        adata.obsm['X_draw_graph_fa'] = adata.obsm[f'X_draw_graph_fa_{init_pos}']
    else:
        print('calculating graph coordinates...')
        sc.tl.draw_graph(adata,init_pos=init_pos)
        adata.obsm[f'X_draw_graph_fa_{init_pos}'] = adata.obsm['X_draw_graph_fa'].copy()
    return adata
```

Example 11 (python):
```python
def plot_embedding(
    adata,
    groupby,
    basis="umap",
    color=None,
    annot=True,
    min_group_size=0,
    greyout_group=None,
    highlight_group=None,
    size=None,
    use_uns_colors=True,
    save=None,
    savedpi=300,
    figsize=(4, 4),
    **kwargs,
):
    set_figsize(figsize)
    if f"X_{basis}" not in adata.obsm.keys():
        raise KeyError(f'"X_{basis}" not found in `adata.obsm`.')
    if min_group_size:
        group_sizes = adata.obs[groupby].value_counts()
        good_groups = group_sizes[group_sizes >= min_group_size].index
        adata = adata[adata.obs[groupby].isin(good_groups)].copy()
    if isinstance(groupby, (list, tuple)):
        groupby = groupby[0]
    if groupby not in adata.obs.columns:
        raise KeyError(f'"{groupby}" not found in `adata.obs`.')
    if adata.obs[groupby].dtype.name != "category":
        if (
            isinstance(adata.obs[groupby][0], (str, bool, np.bool_))
            and adata.obs[groupby].unique().size < 100
        ):
            adata.obs[groupby] = adata.obs[groupby].astype(str).astype("category")
        else:
            sc_warn(f'"{groupby}" is not categorical.')
            plot_scatter(adata, basis=basis, color=groupby, **kwargs)
    categories = list(adata.obs[groupby].cat.categories)
    rename_dict1 = {
        ct: f"{i:^5d} {ct} (n={(adata.obs[groupby]==ct).sum()})" for i, ct in enumerate(categories)
    }
    restore_dict1 = {
        f"{i:^5d} {ct} (n={(adata.obs[groupby]==ct).sum()})": ct for i, ct in enumerate(categories)
    }
    rename_dict2 = {ct: f"{i:^5d} {ct}" for i, ct in enumerate(categories)}
    restore_dict2 = {f"{i:^5d} {ct}": ct for i, ct in enumerate(categories)}

    marker_size = size
    kwargs["show"] = False
    kwargs["save"] = False
    kwargs["frameon"] = kwargs.get("frameon", None)
    kwargs["legend_loc"] = kwargs.get("legend_loc", "right margin")

    if color is not None and color != groupby:
        use_uns_colors = False
    color = groupby if color is None else color
    offset = 0 if "diffmap" in basis else -1
    xi, yi = 1, 2
    if "components" in kwargs:
        xi, yi = kwargs["components"]
    xi += offset
    yi += offset

    if not (use_uns_colors and f"{color}_colors" in adata.uns):
        adata.uns[f"{color}_colors"] = make_palette(
            adata.obs[color].cat.categories.size, kwargs.get("palette", None)
        )
        if greyout_group is not None:
            if isinstance(greyout_group, str):
                greyout_group = [greyout_group]
            greyout_indices = np.where(
                adata.obs[color].cat.categories.isin(greyout_group)
            )[0]
            for nan_i in greyout_indices:
                adata.uns[f"{color}_colors"][nan_i] = "#E0E0E0"

        if highlight_group is not None:
            if isinstance(highlight_group, str):
                highlight_group = [highlight_group]
            nohl_indices = np.where(
                ~adata.obs[color].cat.categories.isin(highlight_group)
            )[0]
            for nohl_i in nohl_indices:
                adata.uns[f"{color}_colors"][nohl_i] = "#E0E0E0"

    if annot == "full":
        adata.obs[groupby].cat.rename_categories(rename_dict1, inplace=True)
    elif annot in (None, False, "none"):
        kwargs["title"] = ""
        kwargs["legend_loc"] = None
    else:
        adata.obs[groupby].cat.rename_categories(rename_dict2, inplace=True)

    try:
        if "ax" in kwargs:
            ax = kwargs["ax"]
            plot_scatter(adata, basis=basis, color=color, size=marker_size, **kwargs)
        else:
            ax = plot_scatter(adata, basis=basis, color=color, size=marker_size, **kwargs)
    finally:
        if annot == "full":
            adata.obs[groupby].cat.rename_categories(restore_dict1, inplace=True)
        elif annot not in (None, False, "none"):
            adata.obs[groupby].cat.rename_categories(restore_dict2, inplace=True)
    if annot not in (None, False, "none"):
        centroids = pseudo_bulk(adata, groupby, use_rep=f"X_{basis}", FUN=np.median).T
        fontsize = kwargs["legend_fontsize"] if "legend_fontsize" in kwargs else 11
        texts = [
            ax.text(x=row[xi], y=row[yi], s=f"{i:d}", fontsize=fontsize - 1.5, fontweight="bold")
            for i, row in centroids.reset_index(drop=True).iterrows()
            if row[0].astype(str) != "nan"
        ]
        from adjustText import adjust_text

        adjust_text(texts, ax=ax, text_from_points=False, autoalign=False)
        version_scale = 1 if sc.__version__.startswith("1.4") else 1
        ax.legend(
            loc="center left",
            bbox_to_anchor=(1, 0.5),
            frameon=False,
            ncol=int(np.ceil(len(categories) / int(figsize[1] * 3) / 8 * fontsize)),
            fontsize=fontsize * 1.1,
            markerscale=3 * (fontsize / 11) * version_scale,
            handletextpad=-1.9,
            labelspacing=9 / fontsize,
        )
    if save:
        plt.savefig(fname=save, dpi=savedpi, bbox_inches="tight", pad_inches=0.1)
    if "ax" not in kwargs:
        return ax
```

Example 12 (python):
```python
# Parameters
job_name = "aCMs_WholeSample"
n_layers = 1
developing = False
paired_adata_path = "/lustre/scratch126/cellgen/team205/heart/objects/fetal/RNA/aCMs_WholeSample_paired.h5ad"
unimodality_adata_path = "none"
combination = "multivi_aCMs_WholeSample_n_layers1"


latent_variables_dir=os.path.join("/lustre/scratch126/cellgen/team205/heart/objects/fetal/RNA","latent_variables") # for latent variables
latent_variables_path=os.path.join(latent_variables_dir, f"mvi_{combination}_latent_variables.h5ad")
```

Example 13 (python):
```python
adata = sc.read(paired_adata_path)
adata
```

Example 14 (unknown):
```unknown
AnnData object with n_obs × n_vars = 5664 × 544641
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id'
```

Example 15 (python):
```python
latent_variables = sc.read(latent_variables_path)
latent_variables
```

Example 16 (unknown):
```unknown
AnnData object with n_obs × n_vars = 5664 × 20
```

Example 17 (python):
```python
latent_variables.shape
```

Example 18 (python):
```python
adata.obsm["X_multiVI"]=latent_variables.X
adata
```

Example 19 (unknown):
```unknown
AnnData object with n_obs × n_vars = 5664 × 544641
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id'
    obsm: 'X_multiVI'
```

Example 20 (python):
```python
sc.pp.neighbors(adata,use_rep="X_multiVI")
```

Example 21 (unknown):
```unknown
computing neighbors
    finished: added to `.uns['neighbors']`
    `.obsp['distances']`, distances for each pair of neighbors
    `.obsp['connectivities']`, weighted adjacency matrix (0:00:07)
```

Example 22 (python):
```python
sc.tl.umap(adata,min_dist=0.01,n_components=3)
```

Example 23 (python):
```python
computing UMAP
    finished: added
    'X_umap', UMAP coordinates (adata.obsm) (0:00:11)
```

Example 24 (python):
```python
sc.pl.umap(adata,color=["fine_grain","week"])
```

Example 25 (unknown):
```unknown
AnnData object with n_obs × n_vars = 5664 × 544641
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id'
    uns: 'neighbors', 'umap', 'fine_grain_colors', 'week_colors'
    obsm: 'X_multiVI', 'X_umap'
    obsp: 'distances', 'connectivities'
```

Example 26 (python):
```python
# Function to convert weeks to days
def week_to_days(week_str):
    # Extract the numerical part and convert to integer
    week_num = int(week_str.replace('W', ''))
    return week_num * 7

# Create a new column 'age_days' in adata.obs DataFrame
adata.obs['age_days'] = adata.obs['week'].apply(week_to_days)
adata.obs['age_days'] = adata.obs['age_days'].astype(float)
adata.obs['age_days'].value_counts()
```

Example 27 (unknown):
```unknown
91.0    3925
35.0     443
63.0     417
84.0     372
56.0     321
49.0     186
Name: age_days, dtype: int64
```

Example 28 (python):
```python
sc.pl.umap(adata,color=["fine_grain","donor","age_days"],cmap='viridis',wspace=0.5)
```

Example 29 (python):
```python
import scipy.sparse

column='age_days'

# Extract the key for the connectivities matrix
conn_key = adata.uns['neighbors']['connectivities_key']

# Now get the actual connectivities matrix using this key
conn_matrix = adata.obsp[conn_key]

# Convert age_days to a NumPy array
age_days = adata.obs[column].values

# Calculate the weighted sum of neighbors for each node
weighted_sum = conn_matrix.dot(age_days)

# Calculate the sum of weights for each node (including self)
sum_weights = conn_matrix.sum(axis=1).A1  # A1 returns a flattened ndarray

# Calculate the weighted average
age_days_smoothed = weighted_sum / sum_weights

# Add the smoothed values as a new column
adata.obs[f'{column}_smoothed_once'] = age_days_smoothed
```

Example 30 (python):
```python
# Min-max normalization
min_val = adata.obs['age_days'].min()
max_val = adata.obs['age_days'].max()
adata.obs['age_days_normalized'] = (adata.obs['age_days'] - min_val) / (max_val - min_val)
```

Example 31 (python):
```python
sc.pl.umap(adata,color=["fine_grain","age_days","age_days_normalized"],cmap='rainbow',wspace=0.5)
```

Example 32 (python):
```python
# Function to perform iterative smoothing until a stable state is achieved
def iterative_smoothing(adata, feature_col, tol=1e-10, max_iter=1000):
    # Extract the key for the connectivities matrix
    conn_key = adata.uns['neighbors']['connectivities_key']

    # Get the actual connectivities matrix using this key
    conn_matrix = adata.obsp[conn_key]

    # Convert feature to a NumPy array
    feature_values = adata.obs[feature_col].values

    # Initialize variables to keep track of previous and current state
    prev_values = feature_values.copy()
    iter_count = 0

    while iter_count < max_iter:
        # Calculate the weighted sum of neighbors for each node
        weighted_sum = conn_matrix.dot(feature_values)

        # Calculate the sum of weights for each node (including self)
        sum_weights = conn_matrix.sum(axis=1).A1  # A1 returns a flattened ndarray

        # Calculate the new feature values (weighted average)
        feature_values = weighted_sum / sum_weights

        # Check for convergence (within given tolerance)
        diff = np.abs(feature_values - prev_values).max()
        if diff < tol:
            print(f"Converged after {iter_count} iterations.")
            break

        # Update previous values for the next iteration
        prev_values = feature_values.copy()

        # Increment iteration count
        iter_count += 1

    # If maximum iterations reached
    if iter_count == max_iter:
        print(f"Max iterations reached. Last diff: {diff}")

    # Add the smoothed values as a new column
    new_col_name = f"{feature_col}_smoothed_iteratively"
    adata.obs[new_col_name] = feature_values
```

Example 33 (python):
```python
# Run the function

# Function to perform one round of smoothing on a given column of an AnnData object
def smooth_column_once(adata, column):
    """
    Smooths a given column in an AnnData object's obs DataFrame based on neighborhood graph.
    
    Parameters:
        adata (anndata.AnnData): The AnnData object containing the neighborhood graph and data.
        column (str): The column in the obs DataFrame to be smoothed.
    
    Returns:
        None: The AnnData object's obs DataFrame is modified in-place with the smoothed column.
    """
    # Extract the key for the connectivities matrix
    conn_key = adata.uns['neighbors']['connectivities_key']
    
    # Get the actual connectivities matrix using this key
    conn_matrix = adata.obsp[conn_key]
    
    # Convert the column values to a NumPy array
    column_values = adata.obs[column].values
    
    # Calculate the weighted sum of neighbors for each node
    weighted_sum = conn_matrix.dot(column_values)
    
    # Calculate the sum of weights for each node (including self)
    sum_weights = conn_matrix.sum(axis=1).A1  # A1 returns a flattened ndarray
    
    # Calculate the weighted average
    smoothed_values = weighted_sum / sum_weights
    
    # Add the smoothed values as a new column
    new_col_name = f"{column}_smoothed_once"
    adata.obs[new_col_name] = smoothed_values
```

Example 34 (python):
```python
smooth_column_once(adata,'age_days')
smooth_column_once(adata,'age_days_normalized')

tol=1e-6
max_iter=100000

iterative_smoothing(adata, 'age_days', tol=tol, max_iter=max_iter)
iterative_smoothing(adata, 'age_days_normalized', tol=1e-6, max_iter=max_iter)
```

Example 35 (unknown):
```unknown
Converged after 608 iterations.
Converged after 436 iterations.
```

Example 36 (python):
```python
sc.pl.umap(adata,color=["age_days","age_days_smoothed_once","age_days_smoothed_iteratively",
                        "age_days_normalized","age_days_normalized_smoothed_once","age_days_normalized_smoothed_iteratively",
                        "fine_grain"],
           cmap='rainbow',wspace=0.5,ncols=3)
```

Example 37 (python):
```python
sc.pl.umap(adata,color=["TBX18","SHOX2","NTM","PANCR"],cmap='RdPu',wspace=0.5,ncols=2)
```

Example 38 (unknown):
```unknown
temporal_ordering="age_days_smoothed_iteratively"
```

Example 39 (python):
```python
global_adata = sc.read('/nfs/team205/heart/anndata_objects/Foetal/Feb28ObjectRaw.h5ad')
global_adata
```

Example 40 (unknown):
```unknown
AnnData object with n_obs × n_vars = 297473 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score'
    var: 'gene_name_scRNA-0', 'gene_id'
    uns: 'FACSgate_colors', '_scvi_manager_uuid', '_scvi_uuid', 'age_colors', 'cell_or_nuclei_colors', 'coarse_grain_colors', 'cycling_colors', 'donor_colors', 'fine_grain_colors', 'heart_or_greatvessels_colors', 'hvg', 'kit_10x_colors', 'mid_grain_colors', 'modality_colors', 'neighbors', 'phase_colors', 'region_colors', 'sex_colors', 'trimester_colors', 'umap', 'week_colors'
    obsm: 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding'
    obsp: 'connectivities', 'distances'
```

Example 41 (python):
```python
# Grab scVI latent variables for just the cells in adata
adata.obsm["X_scVI"] = global_adata[global_adata.obs.index.isin(adata.obs.index)].obsm["X_scVI"].copy()
```

Example 42 (python):
```python
# projectsions from scVI
projections = pd.DataFrame(adata.obsm["X_scVI"],index=adata.obs_names)
```

Example 43 (unknown):
```unknown
# fate probability mapping of the cells
dm_res = palantir.utils.run_diffusion_maps(projections)
ms_data = palantir.utils.determine_multiscale_space(dm_res,n_eigs=4)
```

Example 44 (unknown):
```unknown
Determing nearest neighbor graph...
computing neighbors
    finished: added to `.uns['neighbors']`
    `.obsp['distances']`, distances for each pair of neighbors
    `.obsp['connectivities']`, weighted adjacency matrix (0:00:01)
```

Example 45 (python):
```python
# Generate embedding from the multiscale diffusion space
adata.obsm["X_palantirscVI"]=ms_data.values
```

Example 46 (python):
```python
def plot_and_store(init_pos, adata):
    """
    This function selects the columns of the obsm layer specified by init_pos, 
    draws a graph using these coordinates as the initial positions, and stores the graph 
    coordinates in a new obsm layer named 'X_draw_graph_fa_' followed by the value of init_pos. 
    The modified AnnData object is then returned.

    If adata.obsm[f'X_draw_graph_fa_{init_pos}'] already exists, it puts that in adata.obsm['X_draw_graph_fa'].
    If adata.obsm[f'X_draw_graph_fa_{init_pos}'] does not exist, it performs sc.tl.draw_graph() as before.

    Parameters
    ----------
    init_pos : str
        Initial position for the graph. Should be in the format 'X_palantir12'.
    
    adata : AnnData
        The annotated data matrix of shape n_obs x n_vars. Rows correspond
        to cells and columns to genes.

    Returns
    -------
    adata : AnnData
        The modified annotated data matrix.
    """
    x_col = int(init_pos[-2:][0]) - 1
    y_col = int(init_pos[-2:][1]) - 1
    selected_cols = [x_col, y_col]
    adata.obsm[f'{init_pos}'] = adata.obsm[f'{init_pos[:-2]}'][:,selected_cols]
    
    if f'X_draw_graph_fa_{init_pos}' in adata.obsm:
        print(f'graph using {init_pos} has previously been calculated, retrieving that...')
        adata.obsm['X_draw_graph_fa'] = adata.obsm[f'X_draw_graph_fa_{init_pos}']
    else:
        print('calculating graph coordinates...')
        sc.tl.draw_graph(adata,init_pos=init_pos)
        adata.obsm[f'X_draw_graph_fa_{init_pos}'] = adata.obsm['X_draw_graph_fa'].copy()
    return adata
```

Example 47 (unknown):
```unknown
AnnData object with n_obs × n_vars = 5664 × 544641
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', 'age_days', 'age_days_smoothed_once', 'age_days_normalized', 'age_days_normalized_smoothed_once', 'age_days_smoothed_iteratively', 'age_days_normalized_smoothed_iteratively'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id'
    uns: 'neighbors', 'umap', 'fine_grain_colors', 'week_colors', 'donor_colors'
    obsm: 'X_multiVI', 'X_umap', 'X_scVI', 'X_palantir'
    obsp: 'distances', 'connectivities'
```

Example 48 (python):
```python
%%time
embeddings=['X_umap12','X_umap13','X_palantir12','X_palantir13', 'X_scVI12','X_scVI13','X_palantirscVI12','X_palantirscVI13']

for embedding in embeddings:
        print(embedding)
        adata = plot_and_store(embedding, adata)
        sc.pl.draw_graph(adata,color=['fine_grain','age_days'],wspace=0.8, ncols=2,cmap='bwr', alpha=0.8)
```

Example 49 (unknown):
```unknown
X_umap12
graph using X_umap12 has previously been calculated, retrieving that...
```

Example 50 (unknown):
```unknown
X_umap13
graph using X_umap13 has previously been calculated, retrieving that...
```

Example 51 (unknown):
```unknown
X_palantir12
graph using X_palantir12 has previously been calculated, retrieving that...
```

Example 52 (unknown):
```unknown
X_palantir13
graph using X_palantir13 has previously been calculated, retrieving that...
```

Example 53 (unknown):
```unknown
X_scVI12
graph using X_scVI12 has previously been calculated, retrieving that...
```

Example 54 (unknown):
```unknown
X_scVI13
graph using X_scVI13 has previously been calculated, retrieving that...
```

Example 55 (python):
```python
X_palantirscVI12
calculating graph coordinates...
drawing single-cell graph using layout 'fa'
    finished: added
    'X_draw_graph_fa', graph_drawing coordinates (adata.obsm) (0:00:34)
```

Example 56 (python):
```python
X_palantirscVI13
calculating graph coordinates...
drawing single-cell graph using layout 'fa'
    finished: added
    'X_draw_graph_fa', graph_drawing coordinates (adata.obsm) (0:00:33)
```

Example 57 (unknown):
```unknown
CPU times: user 1min 4s, sys: 3.06 s, total: 1min 7s
Wall time: 1min 15s
```

Example 58 (python):
```python
scf.tl.tree(adata,method="ppt",Nodes=50,use_rep="umap",
            device="cpu",seed=1,ppt_lambda=10000,ppt_sigma=0.4,ppt_nsteps=200)

scf.pl.graph(adata)
```

Example 59 (unknown):
```unknown
inferring a principal tree --> parameters used 
    50 principal points, sigma = 0.4, lambda = 10000, metric = euclidean
    fitting:   5%|█████████▎                                                                                                                                                                                 | 10/200 [00:01<00:21,  8.81it/s]
    converged
    finished (0:00:01) --> added 
    .uns['ppt'], dictionnary containing inferred tree.
    .obsm['X_R'] soft assignment of cells to principal points.
    .uns['graph']['B'] adjacency matrix of the principal points.
    .uns['graph']['F'] coordinates of principal points in representation space.
```

Example 60 (python):
```python
scf.tl.root(adata,49)
```

Example 61 (unknown):
```unknown
node 49 selected as a root --> added
    .uns['graph']['root'] selected root.
    .uns['graph']['pp_info'] for each PP, its distance vs root and segment assignment.
    .uns['graph']['pp_seg'] segments network information.
```

Example 62 (python):
```python
scf.tl.pseudotime(adata,n_jobs=20,n_map=100,seed=42)
```

Example 63 (unknown):
```unknown
projecting cells onto the principal graph
    mappings:   0%|                                                                                                                                                               | 0/100 [00:00<?, ?it/s]
```

Example 64 (unknown):
```unknown
Unable to determine R library path: Command '('/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/scfates_env/lib/R/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/scfates_env/lib/R/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/scfates_env/lib/R/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/scfates_env/lib/R/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/scfates_env/lib/R/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/scfates_env/lib/R/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/scfates_env/lib/R/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/scfates_env/lib/R/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/scfates_env/lib/R/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/scfates_env/lib/R/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/scfates_env/lib/R/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/scfates_env/lib/R/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/scfates_env/lib/R/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/scfates_env/lib/R/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/scfates_env/lib/R/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/scfates_env/lib/R/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/scfates_env/lib/R/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/scfates_env/lib/R/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/scfates_env/lib/R/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
Unable to determine R library path: Command '('/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/scfates_env/lib/R/bin/Rscript', '-e', 'cat(Sys.getenv("LD_LIBRARY_PATH"))')' returned non-zero exit status 2.
```

Example 65 (python):
```python
mappings: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 100/100 [00:51<00:00,  1.93it/s]
    finished (0:00:57) --> added
    .obs['edge'] assigned edge.
    .obs['t'] pseudotime value.
    .obs['seg'] segment of the tree assigned.
    .obs['milestones'] milestone assigned.
    .uns['pseudotime_list'] list of cell projection from all mappings.
```

Example 66 (python):
```python
scf.pl.trajectory(adata)
```

Example 67 (unknown):
```unknown
temporal_ordering='age_days_smoothed_once'
```

Example 68 (python):
```python
sc.pl.umap(adata,color=["seg","milestones","t",temporal_ordering],ncols=2)
```

Example 69 (python):
```python
import scipy.stats

# Assuming `adata` is your AnnData object and it has 't' and 'x' in its obs DataFrame
t_values = adata.obs['t']
x_values = adata.obs[temporal_ordering]

# Calculate the Pearson correlation coefficient and the p-value
correlation_coefficient, p_value = scipy.stats.pearsonr(t_values, x_values)

plt.scatter(t_values,x_values)
correlation_coefficient, p_value
```

Example 70 (unknown):
```unknown
(0.8275538981212454, 0.0)
```

Example 71 (python):
```python
import seaborn as sns
sns.histplot(adata.obs.age_days)
```

Example 72 (unknown):
```unknown
<Axes: xlabel='age_days', ylabel='Count'>
```

Example 73 (python):
```python
sns.histplot(adata.obs[temporal_ordering])
```

Example 74 (unknown):
```unknown
<Axes: xlabel='age_days_smoothed_once', ylabel='Count'>
```

Example 75 (python):
```python
sns.histplot(adata.obs['t'])
```

Example 76 (unknown):
```unknown
<Axes: xlabel='t', ylabel='Count'>
```

Example 77 (python):
```python
paired_adata_with_time_path = "/lustre/scratch126/cellgen/team205/heart/objects/fetal/RNA/aCMs_WholeSample_paired_with_time.h5ad"
adata.write(paired_adata_with_time_path)
adata
```

Example 78 (unknown):
```unknown
AnnData object with n_obs × n_vars = 5664 × 544641
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', 'age_days', 'age_days_smoothed_once', 'age_days_normalized', 'age_days_normalized_smoothed_once', 'age_days_smoothed_iteratively', 'age_days_normalized_smoothed_iteratively', 't', 'seg', 'edge', 't_sd', 'milestones'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id'
    uns: 'neighbors', 'umap', 'fine_grain_colors', 'week_colors', 'donor_colors', 'graph', 'ppt', 'pseudotime_list', 'milestones_colors', 'seg_colors'
    obsm: 'X_multiVI', 'X_umap', 'X_R'
    obsp: 'distances', 'connectivities'
```

---

## Function¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig4_Cardiomyocytes/VentricularCardiomyocytes/no1_compact-vs-trab_region-signatures.html

**Contents:**
- Function¶
- Read in adata¶
- ~Read in DAR result: compact-vs-trab~¶
- Read in fitted result, gene¶
- Select fitted genes which were highest in the group_a or group_b¶
- Fitted peaks¶
- Save the peaks¶
- Scoring and plot¶

i.e. separating into UP or DOWN regulated genes

4483 rows × 6 columns

**Examples:**

Example 1 (python):
```python
# Parameters
working_dir = "/nfs/team205/heart/Vitro/TF_selection"
path_to_adata_dir = "/nfs/team205/heart/anndata_objects/Vitro/post_multiVI_full-features"
path_to_adata_peak_tfidf = "/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/PeakMatrix/Foetal_Peaks_tfidf.h5ad"
group_key = "t_binned"
group_a = "t3"
group_b = "t1"
time_key = "t"
subset_object = False
geneset_list = ["MSigDB_Hallmark_2020", "KEGG_2021_Human", "GO_Biological_Process_2023"]

path_to_eRegulon_signatures_gene = "/nfs/team205/heart/pyscenic/plus/eRegulon_signatures/vCM_onlyLeft_Gene_eRegulon_signatures.pkl"
path_to_eRegulon_signatures_region = "/nfs/team205/heart/pyscenic/plus/eRegulon_signatures/vCM_onlyLeft_Region_eRegulon_signatures.pkl"

path_to_DAR_compact = '/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/MarkerPeaks/vCM_Left_Compact-vs-Trab.csv'
path_to_DAR_trab = '/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/MarkerPeaks/vCM_Left_Trab-vs-Compact.csv'
path_to_eRegulon_metadata = "/nfs/team205/heart/pyscenic/plus/eRegulon_metadata/vCM_onlyLeft_eRegulon_metadata_filtered.csv" # filtered metadata
```

Example 2 (python):
```python
import scanpy as sc
import pandas as pd
import numpy as np
import anndata

import matplotlib.pyplot as plt
import seaborn as sns
```

Example 3 (python):
```python
import session_info
session_info.show()
```

Example 4 (python):
```python
-----
anndata             0.9.2
matplotlib          3.5.2
numpy               1.22.3
pandas              1.5.3
scanpy              1.8.2
seaborn             0.12.2
session_info        1.0.0
-----
```

Example 5 (python):
```python
PIL                 10.0.0
asttokens           NA
backcall            0.2.0
beta_ufunc          NA
binom_ufunc         NA
cffi                1.15.1
colorama            0.4.6
comm                0.1.4
cpuinfo             NA
cycler              0.10.0
cython_runtime      NA
dateutil            2.8.2
debugpy             1.6.7.post1
decorator           5.1.1
executing           1.2.0
h5py                3.9.0
hypergeom_ufunc     NA
igraph              0.10.6
importlib_metadata  NA
ipykernel           6.25.1
jedi                0.19.0
joblib              1.3.2
kiwisolver          1.4.5
leidenalg           0.10.1
llvmlite            0.40.1
mpl_toolkits        NA
natsort             8.4.0
nbinom_ufunc        NA
numba               0.57.1
numexpr             2.8.5
packaging           23.1
parso               0.8.3
patsy               0.5.3
pexpect             4.8.0
pickleshare         0.7.5
pkg_resources       NA
platformdirs        3.10.0
plotly              5.16.1
prompt_toolkit      3.0.39
psutil              5.9.5
ptyprocess          0.7.0
pure_eval           0.2.2
pydev_ipython       NA
pydevconsole        NA
pydevd              2.9.5
pydevd_file_utils   NA
pydevd_plugins      NA
pydevd_tracing      NA
pygments            2.16.1
pyparsing           3.0.9
pytz                2023.3
scipy               1.8.1
sinfo               0.3.4
six                 1.16.0
sklearn             1.3.0
sphinxcontrib       NA
stack_data          0.6.2
statsmodels         0.14.0
tables              3.8.0
texttable           1.6.7
threadpoolctl       3.2.0
tornado             6.3.3
traitlets           5.9.0
typing_extensions   NA
wcwidth             0.2.6
zipp                NA
zmq                 25.1.1
```

Example 6 (unknown):
```unknown
-----
IPython             8.12.2
jupyter_client      8.3.0
jupyter_core        5.3.1
-----
Python 3.8.17 (default, Jul  5 2023, 21:04:15) [GCC 11.2.0]
Linux-4.15.0-212-generic-x86_64-with-glibc2.17
-----
Session information updated at 2023-10-05 08:46
```

Example 7 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 8 (python):
```python
sc.settings.set_figure_params(dpi=120)
```

Example 9 (python):
```python
# read in normalised data
adata_tfidf = sc.read_h5ad(path_to_adata_peak_tfidf)
print(adata_tfidf.X.data[:10])
adata_tfidf
```

Example 10 (unknown):
```unknown
[1.768611  1.9229858 3.635446  1.3192275 1.4049321 1.8123279 1.6739427
 2.9703567 3.3852873 1.2617257]
```

Example 11 (unknown):
```unknown
AnnData object with n_obs × n_vars = 167022 × 508040
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels'
    var: 'seqnames', 'start', 'end', 'width', 'strand', 'score', 'replicateScoreQuantile', 'groupScoreQuantile', 'Reproducibility', 'GroupReplicate', 'distToGeneStart', 'nearestGene', 'peakType', 'distToTSS', 'nearestTSS', 'GC', 'idx', 'N'
```

Example 12 (python):
```python
# read in compact, trabeculated, all adata
# which has pseudotime, etc

print('### compact ###')
adata_compact = sc.read_h5ad(f'{path_to_adata_dir}/Foetal_vCM_Left-compact_atac.h5ad')
# filter features to remove those that appear in fewer than 0.5% of the cells
print(adata_compact.shape)
sc.pp.filter_genes(adata_compact, min_cells=int(adata_compact.shape[0] * 0.005))
print(adata_compact.shape)
# replace .X data with tf-idf normalised data
adata_compact.X = adata_tfidf[adata_compact.obs_names,adata_compact.var_names].X
print(adata_compact.X.data[:10])
print(adata_compact)

print('### trabeculated ###')
adata_trab = sc.read_h5ad(f'{path_to_adata_dir}/Foetal_vCM_Left-trabeculated_atac.h5ad')
# filter features to remove those that appear in fewer than 0.5% of the cells
print(adata_trab.shape)
sc.pp.filter_genes(adata_trab, min_cells=int(adata_trab.shape[0] * 0.005))
print(adata_trab.shape)
# replace .X data with tf-idf normalised data
adata_trab.X = adata_tfidf[adata_trab.obs_names,adata_trab.var_names].X
print(adata_trab.X.data[:10])
print(adata_trab)

'''
print('### all ###')
adata_all = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/vCM/post_scFate/foetal_vCM_onlyLeft.h5ad')
# subset peak feature data
adata_all = adata_all[:,adata_all.var['modality']=='Peaks']
# select features
# features = list(set(list(adata_compact.var_names)+list(adata_trab.var_names)))
# adata_all = adata_all[:,features]
# replace .X data with tf-idf normalised data
adata_all.X = adata_tfidf[adata_all.obs_names,adata_all.var_names].X
print(adata_all.X.data[:10])
print(adata_all)
'''
```

Example 13 (unknown):
```unknown
### compact ###
(11397, 508040)
(11397, 242879)
[1.885513  4.3636436 1.0840797 2.524973  1.4176022 2.0811439 2.3172326
 2.7016861 0.8455251 4.038132 ]
AnnData object with n_obs × n_vars = 11397 × 242879
    obs: 'sangerID', 'combinedID', 'donor', 'region', 'age', 'sex', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'week', 'trimester', 'cycling', 'S_score', 'G2M_score', 'phase', 'fine_grain', 'mid_grain', 'coarse_grain', 'batch_key', 'total_counts', 'total_counts_mt', 'total_counts_ribo', 'n_genes_by_counts', 'pct_counts_mt', 'pct_counts_ribo', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'log1p_total_counts_mt', 'log1p_total_counts_ribo', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', '_indices', '_scvi_batch', '_scvi_labels', 'leiden_MultiVI', 't', 'seg', 'edge', 'milestones', 'age_days', 'week_cont', 't_binned'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id', 'seqnames', 'start', 'end', 'width', 'strand', 'score', 'replicateScoreQuantile', 'groupScoreQuantile', 'Reproducibility', 'GroupReplicate', 'distToGeneStart', 'nearestGene', 'peakType', 'distToTSS', 'nearestTSS', 'GC', 'idx', 'N', 'n_cells'
    uns: '_scvi_manager_uuid', '_scvi_uuid', 'cycling_colors', 'donor_colors', 'draw_graph', 'fine_grain_colors', 'graph', 'leiden', 'leiden_MultiVI_colors', 'milestones_colors', 'neighbors', 'ppt', 'pseudotime_list', 'region_colors', 't_binned_colors', 'umap', 'week_colors'
    obsm: 'MultiVI_latent', 'X_R', 'X_draw_graph_fa', 'X_palantir', 'X_umap', 'X_umap13', '_scvi_extra_categorical_covs', '_scvi_extra_continuous_covs'
    obsp: 'connectivities', 'distances'
### trabeculated ###
(5054, 508040)
(5054, 258442)
[3.2284448 4.816777  4.724808  3.9381142 2.5444632 4.71782   3.8803818
 4.8070517 3.4701438 5.037634 ]
AnnData object with n_obs × n_vars = 5054 × 258442
    obs: 'sangerID', 'combinedID', 'donor', 'region', 'age', 'sex', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'week', 'trimester', 'cycling', 'S_score', 'G2M_score', 'phase', 'fine_grain', 'mid_grain', 'coarse_grain', 'batch_key', 'total_counts', 'total_counts_mt', 'total_counts_ribo', 'n_genes_by_counts', 'pct_counts_mt', 'pct_counts_ribo', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'log1p_total_counts_mt', 'log1p_total_counts_ribo', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', '_indices', '_scvi_batch', '_scvi_labels', 'leiden_MultiVI', 't', 'seg', 'edge', 'milestones', 'age_days', 'week_cont', 't_binned'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id', 'seqnames', 'start', 'end', 'width', 'strand', 'score', 'replicateScoreQuantile', 'groupScoreQuantile', 'Reproducibility', 'GroupReplicate', 'distToGeneStart', 'nearestGene', 'peakType', 'distToTSS', 'nearestTSS', 'GC', 'idx', 'N', 'n_cells'
    uns: '_scvi_manager_uuid', '_scvi_uuid', 'cycling_colors', 'donor_colors', 'draw_graph', 'fine_grain_colors', 'graph', 'leiden', 'leiden_MultiVI_colors', 'milestones_colors', 'neighbors', 'ppt', 'pseudotime_list', 'region_colors', 't_binned_colors', 'umap', 'week_colors'
    obsm: 'MultiVI_latent', 'X_R', 'X_draw_graph_fa', 'X_palantir', 'X_umap', 'X_umap13', '_scvi_extra_categorical_covs', '_scvi_extra_continuous_covs'
    obsp: 'connectivities', 'distances'
```

Example 14 (python):
```python
"\nprint('### all ###')\nadata_all = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/vCM/post_scFate/foetal_vCM_onlyLeft.h5ad')\n#\xa0subset peak feature data\nadata_all = adata_all[:,adata_all.var['modality']=='Peaks']\n#\xa0select features\n#\xa0features = list(set(list(adata_compact.var_names)+list(adata_trab.var_names)))\n#\xa0adata_all = adata_all[:,features]\n#\xa0replace .X data with tf-idf normalised data\nadata_all.X = adata_tfidf[adata_all.obs_names,adata_all.var_names].X\nprint(adata_all.X.data[:10])\nprint(adata_all)\n"
```

Example 15 (python):
```python
sc.pl.draw_graph(adata_compact, color=[group_key, time_key], size=5, wspace=0.3, ncols=3)
```

Example 16 (python):
```python
sc.pl.draw_graph(adata_trab, color=[group_key, time_key], size=5, wspace=0.3, ncols=3)
```

Example 17 (python):
```python
'''
logFC_threshold = 0.1
FDR_threshold = 1

# compact
compact_dar = pd.read_csv(path_to_DAR_compact)
compact_dar_up = compact_dar[(compact_dar['Log2FC']> logFC_threshold)&(compact_dar['FDR']<FDR_threshold)]
compact_dar_down = compact_dar[(compact_dar['Log2FC']< -logFC_threshold)&(compact_dar['FDR']<FDR_threshold)]
print(f'compact, up: {len(compact_dar_up)} regions')
print(f'compact, down: {len(compact_dar_down)} regions')
'''
```

Example 18 (python):
```python
fit_res_compact = pd.read_csv(f'{working_dir}/foetal_vCM_left-compact/outs/Fitting_{time_key}_Peak.csv',index_col=0)
fit_res_trab = pd.read_csv(f'{working_dir}/foetal_vCM_left-trabeculated/outs/Fitting_{time_key}_Peak.csv',index_col=0)
```

Example 19 (unknown):
```unknown
fit_res_compact.head()
```

Example 20 (unknown):
```unknown
fit_res_compact.loc['CDKN1A']
```

Example 21 (python):
```python
# compact
group_mean_compact = utils.grouped_obs_mean(adata_compact,group_key=group_key)

# select upregulated (a>b) or downregulated (b>a) genes
group_a_peaks_compact = list(group_mean_compact.index[group_mean_compact[group_a] > group_mean_compact[group_b]])
group_b_peaks_compact = list(group_mean_compact.index[group_mean_compact[group_b] > group_mean_compact[group_a]])

group_mean_compact.head()
```

Example 22 (python):
```python
# trabeculated
group_mean_trab = utils.grouped_obs_mean(adata_trab,group_key=group_key)

# select upregulated (a>b) or downregulated (b>a) genes
group_a_peaks_trab = list(group_mean_trab.index[group_mean_trab[group_a] > group_mean_trab[group_b]])
group_b_peaks_trab = list(group_mean_trab.index[group_mean_trab[group_b] > group_mean_trab[group_a]])

group_mean_trab.head()
```

Example 23 (unknown):
```unknown
fit_fdr_threshold = 0.05
fit_A_threshold = 0.5
```

Example 24 (python):
```python
# compact
fit_res_sel_compact = fit_res_compact[(fit_res_compact['fdr']<fit_fdr_threshold)&\
                                      (fit_res_compact['A']>fit_A_threshold)
                                     ]
# group a, upregulated
shared = list(set(fit_res_sel_compact.index).intersection(group_a_peaks_compact))
fit_res_sel_up2a_compact = fit_res_sel_compact.loc[shared]
fit_res_sel_up2a_compact = fit_res_sel_up2a_compact.sort_values('A',ascending=False)
print(f'up towards {group_a}, number of fitted peaks: {len(fit_res_sel_up2a_compact)}')
# group b, downregulated
shared = list(set(fit_res_sel_compact.index).intersection(group_b_peaks_compact))
fit_res_sel_up2b_compact = fit_res_sel_compact.loc[shared]
fit_res_sel_up2b_compact = fit_res_sel_up2b_compact.sort_values('A',ascending=False)
print(f'up towards {group_b}, number of fitted peaks: {len(fit_res_sel_up2b_compact)}')
```

Example 25 (unknown):
```unknown
up towards t3, number of fitted peaks: 4483
up towards t1, number of fitted peaks: 14499
```

Example 26 (python):
```python
# trabeculated
fit_res_sel_trab = fit_res_trab[(fit_res_trab['fdr']<fit_fdr_threshold)&\
                                (fit_res_trab['A']>fit_A_threshold)
                                     ]
# group a, upregulated
shared = list(set(fit_res_sel_trab.index).intersection(group_a_peaks_trab))
fit_res_sel_up2a_trab = fit_res_sel_trab.loc[shared]
fit_res_sel_up2a_trab = fit_res_sel_up2a_trab.sort_values('A',ascending=False)
print(f'up towards {group_a}, number of fitted peaks: {len(fit_res_sel_up2a_trab)}')
# group b, downregulated
shared = list(set(fit_res_sel_trab.index).intersection(group_b_peaks_trab))
fit_res_sel_up2b_trab = fit_res_sel_trab.loc[shared]
fit_res_sel_up2b_trab = fit_res_sel_up2b_trab.sort_values('A',ascending=False)
print(f'up towards {group_b}, number of fitted peaks: {len(fit_res_sel_up2b_trab)}')
```

Example 27 (unknown):
```unknown
up towards t3, number of fitted peaks: 4147
up towards t1, number of fitted peaks: 21577
```

Example 28 (python):
```python
sns.scatterplot(fit_res_compact,
                x='fdr',
                y='A',
                size=0.001
                )
```

Example 29 (unknown):
```unknown
<AxesSubplot:xlabel='fdr', ylabel='A'>
```

Example 30 (python):
```python
sns.scatterplot(fit_res_trab,
                x='fdr',
                y='A',
                size=0.001
                )
```

Example 31 (unknown):
```unknown
<AxesSubplot:xlabel='fdr', ylabel='A'>
```

Example 32 (python):
```python
?sns.scatterplot
```

Example 33 (python):
```python
Signature:
sns.scatterplot(
    data=None,
    *,
    x=None,
    y=None,
    hue=None,
    size=None,
    style=None,
    palette=None,
    hue_order=None,
    hue_norm=None,
    sizes=None,
    size_order=None,
    size_norm=None,
    markers=True,
    style_order=None,
    legend='auto',
    ax=None,
    **kwargs,
)
Docstring:
Draw a scatter plot with possibility of several semantic groupings.

The relationship between `x` and `y` can be shown for different subsets
of the data using the `hue`, `size`, and `style` parameters. These
parameters control what visual semantics are used to identify the different
subsets. It is possible to show up to three dimensions independently by
using all three semantic types, but this style of plot can be hard to
interpret and is often ineffective. Using redundant semantics (i.e. both
`hue` and `style` for the same variable) can be helpful for making
graphics more accessible.

See the :ref:`tutorial <relational_tutorial>` for more information.

The default treatment of the `hue` (and to a lesser extent, `size`)
semantic, if present, depends on whether the variable is inferred to
represent "numeric" or "categorical" data. In particular, numeric variables
are represented with a sequential colormap by default, and the legend
entries show regular "ticks" with values that may or may not exist in the
data. This behavior can be controlled through various parameters, as
described and illustrated below.

Parameters
----------
data : :class:`pandas.DataFrame`, :class:`numpy.ndarray`, mapping, or sequence
    Input data structure. Either a long-form collection of vectors that can be
    assigned to named variables or a wide-form dataset that will be internally
    reshaped.
x, y : vectors or keys in ``data``
    Variables that specify positions on the x and y axes.
hue : vector or key in `data`
    Grouping variable that will produce points with different colors.
    Can be either categorical or numeric, although color mapping will
    behave differently in latter case.
size : vector or key in `data`
    Grouping variable that will produce points with different sizes.
    Can be either categorical or numeric, although size mapping will
    behave differently in latter case.
style : vector or key in `data`
    Grouping variable that will produce points with different markers.
    Can have a numeric dtype but will always be treated as categorical.
palette : string, list, dict, or :class:`matplotlib.colors.Colormap`
    Method for choosing the colors to use when mapping the ``hue`` semantic.
    String values are passed to :func:`color_palette`. List or dict values
    imply categorical mapping, while a colormap object implies numeric mapping.
hue_order : vector of strings
    Specify the order of processing and plotting for categorical levels of the
    ``hue`` semantic.
hue_norm : tuple or :class:`matplotlib.colors.Normalize`
    Either a pair of values that set the normalization range in data units
    or an object that will map from data units into a [0, 1] interval. Usage
    implies numeric mapping.
sizes : list, dict, or tuple
    An object that determines how sizes are chosen when `size` is used.
    List or dict arguments should provide a size for each unique data value,
    which forces a categorical interpretation. The argument may also be a
    min, max tuple.
size_order : list
    Specified order for appearance of the `size` variable levels,
    otherwise they are determined from the data. Not relevant when the
    `size` variable is numeric.
size_norm : tuple or Normalize object
    Normalization in data units for scaling plot objects when the
    `size` variable is numeric.
markers : boolean, list, or dictionary
    Object determining how to draw the markers for different levels of the
    `style` variable. Setting to `True` will use default markers, or
    you can pass a list of markers or a dictionary mapping levels of the
    `style` variable to markers. Setting to `False` will draw
    marker-less lines.  Markers are specified as in matplotlib.
style_order : list
    Specified order for appearance of the `style` variable levels
    otherwise they are determined from the data. Not relevant when the
    `style` variable is numeric.
legend : "auto", "brief", "full", or False
    How to draw the legend. If "brief", numeric `hue` and `size`
    variables will be represented with a sample of evenly spaced values.
    If "full", every group will get an entry in the legend. If "auto",
    choose between brief or full representation based on number of levels.
    If `False`, no legend data is added and no legend is drawn.
ax : :class:`matplotlib.axes.Axes`
    Pre-existing axes for the plot. Otherwise, call :func:`matplotlib.pyplot.gca`
    internally.
kwargs : key, value mappings
    Other keyword arguments are passed down to
    :meth:`matplotlib.axes.Axes.scatter`.

Returns
-------
:class:`matplotlib.axes.Axes`
    The matplotlib axes containing the plot.

See Also
--------
lineplot : Plot data using lines.
stripplot : Plot a categorical scatter with jitter.
swarmplot : Plot a categorical scatter with non-overlapping points.

Examples
--------

.. include:: ../docstrings/scatterplot.rst
File:      ~/my-conda-envs/vitro/lib/python3.8/site-packages/seaborn/relational.py
Type:      function
```

Example 34 (python):
```python
peak_signatures = {}
# up, towards group_a
peak_signatures['compact_up'] = list(fit_res_sel_up2a_compact.index)
peak_signatures['trab_up'] = list(fit_res_sel_up2a_trab.index)

# down, towards group_b
peak_signatures['compact_down'] = list(fit_res_sel_up2b_compact.index)
peak_signatures['trab_down'] = list(fit_res_sel_up2b_trab.index)

print(peak_signatures.keys())

# save
utils.save_pkl(peak_signatures,
           file = f'{working_dir}/foetal_vCM_left_compact-vs-trabeculated_downstream/peak_signatures.pkl'
          )
```

Example 35 (unknown):
```unknown
dict_keys(['compact_up', 'trab_up', 'compact_down', 'trab_down'])
```

Example 36 (unknown):
```unknown
f'{working_dir}/foetal_vCM_left_compact-vs-trabeculated_downstream/peak_signatures.pkl'
```

Example 37 (unknown):
```unknown
'/nfs/team205/heart/Vitro/TF_selection/foetal_vCM_left_compact-vs-trabeculated_downstream/peak_signatures.pkl'
```

Example 38 (python):
```python
adata_compact.X.data[:10]
```

Example 39 (unknown):
```unknown
array([1.885513 , 4.3636436, 1.0840797, 2.524973 , 1.4176022, 2.0811439,
       2.3172326, 2.7016861, 0.8455251, 4.038132 ], dtype=float32)
```

Example 40 (unknown):
```unknown
fit_res_sel_up2a_compact
```

Example 41 (python):
```python
# compact
sc.tl.score_genes(adata_compact,peak_signatures['compact_up'],score_name=f'compact-up ({len(peak_signatures["compact_up"])}r)')
sc.tl.score_genes(adata_compact,peak_signatures['compact_down'],score_name=f'compact-down ({len(peak_signatures["compact_down"])}r)')

sc.pl.draw_graph(adata_compact, color=['fine_grain',
                                   f'compact-up ({len(peak_signatures["compact_up"])}r)',
                                   f'compact-down ({len(peak_signatures["compact_down"])}r)',
                                  ], 
                 frameon=False, legend_loc='on data',legend_fontsize=0,
                 size=7, wspace=0.1, ncols=4, cmap='YlOrRd')
```

Example 42 (python):
```python
/home/jovyan/my-conda-envs/vitro/lib/python3.8/site-packages/scanpy/tools/_score_genes.py:151: FutureWarning: Passing a set as an indexer is deprecated and will raise in a future version. Use a list instead.
  for cut in np.unique(obs_cut.loc[gene_list]):
/home/jovyan/my-conda-envs/vitro/lib/python3.8/site-packages/scanpy/tools/_score_genes.py:151: FutureWarning: Passing a set as an indexer is deprecated and will raise in a future version. Use a list instead.
  for cut in np.unique(obs_cut.loc[gene_list]):
```

Example 43 (python):
```python
# trabeculated
sc.tl.score_genes(adata_trab,peak_signatures['trab_up'],score_name=f'trab-up ({len(peak_signatures["trab_up"])}r)')
sc.tl.score_genes(adata_trab,peak_signatures['trab_down'],score_name=f'trab-down ({len(peak_signatures["trab_down"])}r)')

sc.pl.draw_graph(adata_trab, color=['fine_grain',
                                   f'trab-up ({len(peak_signatures["trab_up"])}r)',
                                   f'trab-down ({len(peak_signatures["trab_down"])}r)',
                                  ], 
                 frameon=False, legend_loc='on data',legend_fontsize=0,
                 size=7, wspace=0.1, ncols=4, cmap='YlOrRd')
```

Example 44 (python):
```python
/home/jovyan/my-conda-envs/vitro/lib/python3.8/site-packages/scanpy/tools/_score_genes.py:151: FutureWarning: Passing a set as an indexer is deprecated and will raise in a future version. Use a list instead.
  for cut in np.unique(obs_cut.loc[gene_list]):
/home/jovyan/my-conda-envs/vitro/lib/python3.8/site-packages/scanpy/tools/_score_genes.py:151: FutureWarning: Passing a set as an indexer is deprecated and will raise in a future version. Use a list instead.
  for cut in np.unique(obs_cut.loc[gene_list]):
```

---

## Read the objects¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/VisiumHD/no9_map_loupe-annotations.html

**Contents:**
- Read the objects¶
- Import annotations from loupe browser¶
- Plot¶
- Save cdata¶

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os
import bin2cell as b2c
print(b2c.__version__)
```

Example 2 (python):
```python
# import session_info
# session_info.show()
```

Example 3 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 4 (unknown):
```unknown
os.getcwd()
```

Example 5 (unknown):
```unknown
'/nfs/team205/kk18/notebooks/Foetal/VisiumHD/re-seq_mpp0p3'
```

Example 6 (python):
```python
adata_dir = '/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects'
sample_id = 'HEA_FOET14880396'
```

Example 7 (python):
```python
!ls {adata_dir}
```

Example 8 (unknown):
```unknown
archived-Dec2024_HEA_FOET14880396_b2c.h5ad
cellcharter
HEA_AD14880301_b2c.h5ad
HEA_FOET14880396_8um-bins_filtered.h5ad
HEA_FOET14880396_8um-bins_filtered_lognorm.h5ad
HEA_FOET14880396_b2c_cells_filtered_celltype-selected_lognorm.h5ad
HEA_FOET14880396_b2c_cells_filtered.h5ad
HEA_FOET14880396_b2c_cells_filtered_lognorm.h5ad
HEA_FOET14880396_b2c_cells_filtered_lognorm_morpho_obs.csv
HEA_FOET14880396_b2c_cells.h5ad
HEA_FOET14880396_b2c.h5ad
HEA_FOET14880397_b2c.h5ad
subsets
```

Example 9 (python):
```python
adata = sc.read_h5ad(f'{adata_dir}/{sample_id}_b2c.h5ad') # read 2um object which has b2c cell grouping - see tutorial - https://nbviewer.org/github/Teichlab/bin2cell/blob/main/notebooks/demo.ipynb 
cdata = sc.read_h5ad(f'{adata_dir}/{sample_id}_b2c_cells_filtered_lognorm.h5ad') # read b2c object
anno_df = pd.read_csv(f'/nfs/team205/kk18/notebooks/Foetal/VisiumHD/re-seq_mpp0p3/loupe_annotation/{sample_id}_HistologicalAnnotation.csv',index_col=0)
anno = 'HistologicalAnnotation'
cell_grouping = 'labels_joint'
```

Example 10 (unknown):
```unknown
AnnData object with n_obs × n_vars = 5001851 × 18051
    obs: 'in_tissue', 'array_row', 'array_col', 'n_counts', 'destripe_factor', 'n_counts_adjusted', 'labels_he_0p01', 'labels_he_0p03', 'labels_he_0p05', 'labels_he_0p1', 'labels_he_default', 'labels_he_sel', 'labels_he_expanded_1', 'labels_he_expanded_2', 'labels_he_expanded_3', 'labels_he_expanded_4', 'labels_he_expanded_sel', 'labels_gex_0p005', 'labels_gex_0p01', 'labels_gex_0p03', 'labels_gex_0p05', 'labels_gex_sel', 'labels_joint', 'labels_joint_source'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells'
    uns: 'bin2cell', 'spatial'
    obsm: 'spatial', 'spatial_cropped_150_buffer'
```

Example 11 (python):
```python
def map_loupe_to_b2c(
        adata,
        cdata,
        anno_df,
        anno,
        cell_grouping = 'labels_he_expanded', 
        ):
     

    # Calculate the binned row and column indices in adata
    adata.obs['binned_8_row'] = adata.obs['array_row'] // 4
    adata.obs['binned_8_col'] = adata.obs['array_col'] // 4

    # Create a combined index in adata and set it as the index
    adata.obs['mapping_index_08'] = 's_008um_' + adata.obs['binned_8_row'].astype(str).str.zfill(5) + '_' + adata.obs['binned_8_col'].astype(str).str.zfill(5) + '-1'
    adata.obs['mapping_index_08'].value_counts() # most should have 16 but there are some on the tissue edge that actually have less due to partial overlap 

    # read annotations from df and map to the 2um object 
    print('annotations to map:')
    print(anno_df.head(5))
    mapping_dict = pd.Series(anno_df[anno].values,index=anno_df.index).to_dict()
    adata.obs[anno] = adata.obs['mapping_index_08'].map(mapping_dict) # update 2um object 

    # assign annotation to cells by pandas.Series.mode(). The mode is the value that appears most often which captures the most frequent annotation for that cell from 2um data
    filtered_data = adata.obs[(adata.obs[cell_grouping] != 0) & adata.obs[anno].notna()]
    cell_annotation = (
        filtered_data.groupby(cell_grouping)[anno]
        .apply(lambda x: x.mode().iloc[0] if not x.mode().empty else None)
    )
    cell_annotation.index = cell_annotation.index.astype('str')
    cdata.obs[anno] = cdata.obs.index.map(cell_annotation).astype('category') # label cells 
    print('Annotations in b2c object:')
    print(cdata.obs[anno].value_counts())

map_loupe_to_b2c(adata,
                 cdata,
                 anno_df,
                 anno,
                 cell_grouping)
```

Example 12 (unknown):
```unknown
annotations to map:
                                      HistologicalAnnotation
Barcode                                                     
s_008um_00301_00321-1                                 Valves
s_008um_00602_00290-1  Great vessel - Pulmonary artery Right
s_008um_00377_00750-1                              Ventricle
s_008um_00383_00696-1                              Ventricle
s_008um_00526_00291-1                   Great vessel - Aorta
Annotations in b2c object:
HistologicalAnnotation
Ventricle                                                     64624
Atrium                                                        17355
Valves                                                         6435
Great vessel - Pulmonary artery Left and Ductus arteriosus     4746
Great vessel - Pulmonary artery Right                          2838
Coronary vessel                                                2752
Great vessel - Aorta                                           2564
Name: count, dtype: int64
```

Example 13 (unknown):
```unknown
annotations_to_plot = list(set(anno_df['HistologicalAnnotation'].dropna()))
group_col = 'HistologicalAnnotation'
len(annotations_to_plot)
```

Example 14 (python):
```python
sc.settings.set_figure_params(dpi=120)
sc.pl.spatial(cdata, 
                  color=group_col, 
                  img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer",
                  size=1.5,alpha_img=0.3,
                  frameon=False,
                 )
```

Example 15 (python):
```python
fig,axs=plt.subplots(nrows=1,ncols=6,figsize=(24,4))
# Flatten the axs array for easier iteration in case of multiple rows and columns
axs_flat = axs.flatten()

for i,annotation in enumerate(annotations_to_plot):
    ad = cdata.copy()
    # update colors
    # get color in cdata
    ind = list(ad.obs[group_col].cat.categories).index(annotation)
    ad.uns[f'{group_col}_colors'][ind] = 'greenyellow'
    # plot
    sc.pl.spatial(ad, 
                  color=group_col, groups=annotation,
                  img_key="0.3_mpp_150_buffer", basis="spatial_cropped_150_buffer",
                  size=7,legend_loc=None,
                  frameon=False,title=annotation,
                  show=False, ax=axs_flat[i],
                 )
# Turn off any remaining subplots
for j in range(len(annotations_to_plot), len(axs_flat)):
    fig.delaxes(axs_flat[j])

# fig.suptitle('Main Title for the Figure')
fig.tight_layout()
```

Example 16 (unknown):
```unknown
AnnData object with n_obs × n_vars = 129803 × 18047
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'fine_grain_4Jan2025', 'HistologicalAnnotation'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'HistologicalAnnotation_colors', 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'fine_grain_4Jan2025_colors', 'hvg', 'log1p', 'neighbors', 'pca', 'spatial', 'umap'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 17 (python):
```python
cdata.write(f'{adata_dir}/{sample_id}_b2c_cells_filtered_lognorm.h5ad')
cdata
```

Example 18 (unknown):
```unknown
AnnData object with n_obs × n_vars = 129803 × 18047
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'fine_grain_4Jan2025', 'HistologicalAnnotation'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'HistologicalAnnotation_colors', 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'fine_grain_4Jan2025_colors', 'hvg', 'log1p', 'neighbors', 'pca', 'spatial', 'umap'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

---

## Read in¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/Xenium/no2_integration_and_cell-type-annotation/no01_C194-Hst45_concat.html

**Contents:**
- Read in¶
- Concatenate¶
- Update obs_names and save¶
- Save¶

**Examples:**

Example 1 (python):
```python
import scanpy as sc
import numpy as np
import pandas as pd
import os
# import rapids_singlecell as rsc
from tqdm import tqdm

import matplotlib.pyplot as plt
import seaborn as sns
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/session_info/main.py:213: UserWarning: The '__version__' attribute is deprecated and will be removed in MarkupSafe 3.1. Use feature detection, or `importlib.metadata.version("markupsafe")`, instead.
  mod_version = _find_version(mod.__version__)
```

Example 4 (python):
```python
-----
matplotlib          3.10.1
numpy               2.1.3
pandas              2.2.3
scanpy              1.11.1
seaborn             0.13.2
session_info        v1.0.1
tqdm                4.67.1
-----
```

Example 5 (python):
```python
PIL                 11.1.0
anndata             0.11.4
asciitree           NA
asttokens           NA
charset_normalizer  3.4.1
cloudpickle         3.1.1
comm                0.2.2
cycler              0.12.1
cython_runtime      NA
dask                2024.11.2
dateutil            2.9.0.post0
debugpy             1.8.13
decorator           5.2.1
exceptiongroup      1.2.2
executing           2.2.0
h5py                3.13.0
igraph              0.11.8
importlib_metadata  NA
ipykernel           6.29.5
jedi                0.19.2
jinja2              3.1.6
joblib              1.4.2
kiwisolver          1.4.8
legacy_api_wrap     NA
leidenalg           0.10.2
llvmlite            0.44.0
markupsafe          3.0.2
mpl_toolkits        NA
msgpack             1.1.0
natsort             8.4.0
numba               0.61.0
numcodecs           0.13.1
packaging           24.2
parso               0.8.4
patsy               1.0.1
platformdirs        4.3.7
prompt_toolkit      3.0.50
psutil              7.0.0
pure_eval           0.2.3
pyarrow             19.0.1
pydev_ipython       NA
pydevconsole        NA
pydevd              3.2.3
pydevd_file_utils   NA
pydevd_plugins      NA
pydevd_tracing      NA
pygments            2.19.1
pyparsing           3.2.3
pytz                2025.2
scipy               1.15.2
six                 1.17.0
sklearn             1.5.2
stack_data          0.6.3
statsmodels         0.14.4
tblib               3.1.0
texttable           1.7.0
threadpoolctl       3.6.0
tlz                 1.0.0
toolz               1.0.0
tornado             6.4.2
traitlets           5.14.3
typing_extensions   NA
wcwidth             0.2.13
yaml                6.0.2
zarr                2.18.3
zipp                NA
zmq                 26.3.0
zoneinfo            NA
```

Example 6 (unknown):
```unknown
-----
IPython             8.34.0
jupyter_client      8.6.3
jupyter_core        5.7.2
-----
Python 3.10.14 | packaged by conda-forge | (main, Mar 20 2024, 12:45:18) [GCC 12.3.0]
Linux-4.18.0-553.45.1.el8_10.x86_64-x86_64-with-glibc2.28
-----
Session information updated at 2025-05-29 11:15
```

Example 7 (python):
```python
sc.settings.set_figure_params(dpi=120)
```

Example 8 (unknown):
```unknown
objects_dir = '/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium'
```

Example 9 (python):
```python
sample_id_1 = 'C194-HEA-0-FFPE-1'
path_adata_1 = f'{objects_dir}/{sample_id_1}_5K_filtered_raw.h5ad'

sample_id_2 = 'Hst45-HEA-0-FFPE-1'
path_adata_2 = f'{objects_dir}/{sample_id_2}_5K_filtered_raw.h5ad'
```

Example 10 (python):
```python
adata_1 = sc.read_h5ad(path_adata_1)
adata_1
```

Example 11 (unknown):
```unknown
AnnData object with n_obs × n_vars = 144916 × 5001
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts'
    obsm: 'spatial'
```

Example 12 (python):
```python
adata_2 = sc.read_h5ad(path_adata_2)
adata_2
```

Example 13 (unknown):
```unknown
AnnData object with n_obs × n_vars = 217361 × 5001
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts'
    obsm: 'spatial'
```

Example 14 (python):
```python
# tidy obs columns
adata_1.obs = adata_1.obs[['cell_id', 'x_centroid', 'y_centroid', 'transcript_counts',
       'control_probe_counts', 'genomic_control_counts',
       'control_codeword_counts', 'unassigned_codeword_counts',
       'deprecated_codeword_counts', 'total_counts', 'cell_area',
       'nucleus_area', 'nucleus_count', 'segmentation_method',
       'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts',
       'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes',
       'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts',
       'n_genes']]
adata_1.obs['tissue_block_id'] = sample_id_1

adata_2.obs = adata_2.obs[['cell_id', 'x_centroid', 'y_centroid', 'transcript_counts',
       'control_probe_counts', 'genomic_control_counts',
       'control_codeword_counts', 'unassigned_codeword_counts',
       'deprecated_codeword_counts', 'total_counts', 'cell_area',
       'nucleus_area', 'nucleus_count', 'segmentation_method',
       'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts',
       'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes',
       'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts',
       'n_genes']]
adata_2.obs['tissue_block_id'] = sample_id_2
```

Example 15 (python):
```python
# concatenate
adata = adata_1.concatenate(adata_2, 
            join='inner', 
            batch_key=None, 
            index_unique=None)
adata.obs['tissue_block_id'].value_counts()
```

Example 16 (python):
```python
/tmp/ipykernel_1481351/366024108.py:2: FutureWarning: Use anndata.concat instead of AnnData.concatenate, AnnData.concatenate is deprecated and will be removed in the future. See the tutorial for concat at: https://anndata.readthedocs.io/en/latest/concatenation.html
  adata = adata_1.concatenate(adata_2,
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/anndata/_core/anndata.py:1756: UserWarning: Observation names are not unique. To make them unique, call `.obs_names_make_unique`.
  utils.warn_names_duplicates("obs")
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/anndata/_core/anndata.py:1756: UserWarning: Observation names are not unique. To make them unique, call `.obs_names_make_unique`.
  utils.warn_names_duplicates("obs")
```

Example 17 (unknown):
```unknown
tissue_block_id
Hst45-HEA-0-FFPE-1    217361
C194-HEA-0-FFPE-1     144916
Name: count, dtype: int64
```

Example 18 (python):
```python
adata.obs_names = adata.obs['tissue_block_id'].astype('str') + '_' + adata.obs_names.astype('str')
adata.obs.head()
```

Example 19 (python):
```python
path_adata = f'{objects_dir}/C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat_5K_filtered_raw.h5ad'
```

Example 20 (python):
```python
# save
adata.write(path_adata)
adata.obs.head()
```

Example 21 (python):
```python
adata.X.data[:5]
```

Example 22 (unknown):
```unknown
array([5., 1., 1., 2., 2.], dtype=float32)
```

---

## Read in original adata¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig5_Trisomy21/no06_euploid-trisomy21_post-scVI_correcting-donor-region_n-layers-2.html

**Contents:**
- Read in original adata¶
- Add scVI latent space¶
- Find neighbors, UMAP/Clustering¶
- Inspecting each cell types¶
- Plotting¶

**Examples:**

Example 1 (python):
```python
import scanpy as sc
import pandas as pd
import numpy as np
import anndata
import scipy

import os

import matplotlib.pyplot as plt
import seaborn as sns
import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 

import warnings
warnings.simplefilter("ignore", UserWarning)
```

Example 2 (python):
```python
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm
```

Example 3 (python):
```python
import session_info
session_info.show()
```

Example 4 (python):
```python
-----
anndata             0.9.2
matplotlib          3.7.2
numpy               1.22.4
pandas              1.5.3
scanpy              1.9.3
scipy               1.10.1
seaborn             0.12.2
session_info        1.0.0
-----
```

Example 5 (python):
```python
PIL                         10.0.0
anyio                       NA
arrow                       1.2.3
asciitree                   NA
asttokens                   NA
attr                        23.1.0
attrs                       23.1.0
babel                       2.12.1
backcall                    0.2.0
brotli                      1.0.9
certifi                     2023.11.17
cffi                        1.15.1
charset_normalizer          3.2.0
cloudpickle                 3.0.0
colorama                    0.4.6
comm                        0.1.4
cycler                      0.10.0
cython_runtime              NA
dask                        2023.5.0
dateutil                    2.8.2
debugpy                     1.6.8
decorator                   5.1.1
defusedxml                  0.7.1
executing                   1.2.0
fasteners                   0.19
fastjsonschema              NA
fqdn                        NA
h5py                        3.9.0
idna                        3.4
igraph                      0.10.6
importlib_metadata          NA
importlib_resources         NA
ipykernel                   6.25.1
isoduration                 NA
jedi                        0.19.0
jinja2                      3.1.2
joblib                      1.3.2
json5                       NA
jsonpointer                 2.0
jsonschema                  4.19.0
jsonschema_specifications   NA
jupyter_events              0.7.0
jupyter_server              2.7.1
jupyterlab_server           2.24.0
kiwisolver                  1.4.5
leidenalg                   0.10.1
llvmlite                    0.38.1
markupsafe                  2.1.3
mpl_toolkits                NA
natsort                     8.4.0
nbformat                    5.9.2
numba                       0.55.2
numcodecs                   0.12.1
opt_einsum                  v3.3.0
overrides                   NA
packaging                   23.1
parso                       0.8.3
patsy                       0.5.3
pexpect                     4.8.0
pickleshare                 0.7.5
pkg_resources               NA
plotly                      5.16.1
prometheus_client           NA
prompt_toolkit              3.0.39
psutil                      5.9.5
ptyprocess                  0.7.0
pure_eval                   0.2.2
pyarrow                     15.0.2
pydev_ipython               NA
pydevconsole                NA
pydevd                      2.9.5
pydevd_file_utils           NA
pydevd_plugins              NA
pydevd_tracing              NA
pygments                    2.16.1
pyparsing                   3.0.9
pythonjsonlogger            NA
pytz                        2023.3
referencing                 NA
requests                    2.31.0
rfc3339_validator           0.1.4
rfc3986_validator           0.1.1
rpds                        NA
ruamel                      NA
send2trash                  NA
six                         1.16.0
sklearn                     1.3.0
sniffio                     1.3.0
socks                       1.7.1
sphinxcontrib               NA
stack_data                  0.6.2
statsmodels                 0.14.0
texttable                   1.6.7
threadpoolctl               3.2.0
tlz                         0.12.1
toolz                       0.12.1
torch                       1.13.1+cu117
tornado                     6.3.3
tqdm                        4.66.1
traitlets                   5.9.0
typing_extensions           NA
uri_template                NA
urllib3                     2.0.4
wcwidth                     0.2.6
webcolors                   1.13
websocket                   1.6.2
yaml                        6.0.1
zarr                        2.16.1
zipp                        NA
zmq                         25.1.1
zstandard                   0.19.0
```

Example 6 (unknown):
```unknown
-----
IPython             8.12.0
jupyter_client      8.3.0
jupyter_core        4.12.0
jupyterlab          4.0.5
notebook            7.0.2
-----
Python 3.8.17 (default, Jul  5 2023, 21:04:15) [GCC 11.2.0]
Linux-5.15.0-118-generic-x86_64-with-glibc2.17
-----
Session information updated at 2024-08-26 16:12
```

Example 7 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 8 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 9 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk/Aug2024'
table_dir = '/nfs/team205/heart/fetal_ms_figures/tables/Aug2024'
```

Example 10 (python):
```python
# atlas
adata = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/trisomy21/Euploid_T21Hearts_Aug2024_sel_subsampled-per-cellstate-donor.h5ad')
adata
```

Example 11 (unknown):
```unknown
AnnData object with n_obs × n_vars = 83477 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score', 'euploid_or_t21', 'donor_type', 'region_finest', 'flushed', 'scrublet_leiden', 'cluster_scrublet_score', 'leiden_scVI_0.1', 'leiden_scVI_0.2', 'leiden_scVI_0.3', 'leiden_scVI_0.5', 'leiden_scVI_0.7', 'leiden_scVI_1', 'celltypist_coarse', 'conf_score_coarse', 'cells_to_remove', 'celltypist_fine', 'conf_score_fine', 'coarse_grain_merged', 'fine_grain_merged'
    var: 'gene_id', 'gene_name_scRNA-0-0', 'feature_type-1', 'ambient_expression-0-1', 'ambient_expression-1-1', 'ambient_expression-10-1', 'ambient_expression-11-1', 'ambient_expression-12-1', 'ambient_expression-13-1', 'ambient_expression-14-1', 'ambient_expression-2-1', 'ambient_expression-3-1', 'ambient_expression-4-1', 'ambient_expression-5-1', 'ambient_expression-6-1', 'ambient_expression-7-1', 'ambient_expression-8-1', 'ambient_expression-9-1', 'mt-1', 'ribo-1', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1'
    obsm: 'X_scVI', 'X_umap', 'latent_gene_encoding'
```

Example 12 (python):
```python
adata.obs['region'].value_counts()
```

Example 13 (unknown):
```unknown
Heart                              32788
whole sample                       24433
Pericardial sac + RA + LA + IAS     5084
node                                4298
Left ventricle                      3888
Right ventricle                     3792
heart without node                  3624
Left atria                          3475
Great vessels                       2095
Name: region, dtype: int64
```

Example 14 (unknown):
```unknown
latent_file_name_list = os.listdir('/nfs/team205/heart/anndata_objects/Foetal/scVI/latent_variables')
latent_file_name_list = [x.split('.h5ad')[0] for x in latent_file_name_list if 'correcting-donor-region_n-layers-2' in x]
latent_file_name_list
```

Example 15 (unknown):
```unknown
['Euploid_T21Hearts_Aug2024_sel_subsampled-per-cellstate-donor_correcting-donor-region_n-layers-2']
```

Example 16 (python):
```python
for latent_file_name in latent_file_name_list:
    latent_space_condition = latent_file_name.split('subsampled-per-cellstate-donor_')[1]
    latent = sc.read_h5ad(f'/nfs/team205/heart/anndata_objects/Foetal/scVI/latent_variables/{latent_file_name}.h5ad')
    # add latent space
    adata.obsm[f'scVI_latent_{latent_space_condition}']=latent.X.copy()
```

Example 17 (unknown):
```unknown
AnnData object with n_obs × n_vars = 83477 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score', 'euploid_or_t21', 'donor_type', 'region_finest', 'flushed', 'scrublet_leiden', 'cluster_scrublet_score', 'leiden_scVI_0.1', 'leiden_scVI_0.2', 'leiden_scVI_0.3', 'leiden_scVI_0.5', 'leiden_scVI_0.7', 'leiden_scVI_1', 'celltypist_coarse', 'conf_score_coarse', 'cells_to_remove', 'celltypist_fine', 'conf_score_fine', 'coarse_grain_merged', 'fine_grain_merged'
    var: 'gene_id', 'gene_name_scRNA-0-0', 'feature_type-1', 'ambient_expression-0-1', 'ambient_expression-1-1', 'ambient_expression-10-1', 'ambient_expression-11-1', 'ambient_expression-12-1', 'ambient_expression-13-1', 'ambient_expression-14-1', 'ambient_expression-2-1', 'ambient_expression-3-1', 'ambient_expression-4-1', 'ambient_expression-5-1', 'ambient_expression-6-1', 'ambient_expression-7-1', 'ambient_expression-8-1', 'ambient_expression-9-1', 'mt-1', 'ribo-1', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1'
    obsm: 'X_scVI', 'X_umap', 'latent_gene_encoding', 'scVI_latent_correcting-donor-region_n-layers-2'
```

Example 18 (python):
```python
for latent_file_name in latent_file_name_list:
    latent_space_condition = latent_file_name.split('subsampled-per-cellstate-donor_')[1]
    print(f'##### {latent_space_condition} #####')
    sc.pp.neighbors(adata, use_rep=f'scVI_latent_{latent_space_condition}',n_neighbors=15,key_added=latent_space_condition)
    sc.tl.umap(adata, min_dist=0.2, spread=1.0, neighbors_key=latent_space_condition)# 
    adata.obsm[f'X_umap_{latent_space_condition}'] = adata.obsm[f'X_umap'].copy()
    del adata.obsm[f'X_umap']
    # check batches
    sc.pl.embedding(adata, basis=f'umap_{latent_space_condition}',
                    color=['total_counts','pct_counts_mt','pct_counts_ribo','donor','euploid_or_t21'], 
               size=5, wspace=0.3, ncols=6, components='all',cmap='RdPu')
    print('')
    print('')
```

Example 19 (unknown):
```unknown
##### correcting-donor-region_n-layers-2 #####
```

Example 20 (unknown):
```unknown
f'umap_{latent_space_condition}'
```

Example 21 (unknown):
```unknown
'umap_correcting-donor-region_n-layers-2'
```

Example 22 (unknown):
```unknown
basis = 'umap_correcting-donor-region_n-layers-2'
```

Example 23 (python):
```python
set(adata.obs['coarse_grain_merged'])
```

Example 24 (unknown):
```unknown
{'Cardiomyocytes',
 'Endothelium',
 'Epicardium',
 'Leukocytes',
 'Mesenchymal',
 'Neural'}
```

Example 25 (python):
```python
sc.pl.embedding(adata, basis=basis, color=['euploid_or_t21'], 
               size=5, wspace=0.3, ncols=6, components='all',cmap='RdPu')
sc.pl.embedding(adata, basis=basis, color=['coarse_grain_merged'], 
               size=5, wspace=0.3, ncols=6, components='all',cmap='RdPu')
sc.pl.embedding(adata, basis=basis, color=['mid_grain'], 
               size=5, wspace=0.3, ncols=6, components='all',cmap='RdPu')
sc.pl.embedding(adata, basis=basis, color=['mid_grain'], groups='EndocardialCells', 
               size=5, wspace=0.3, ncols=6, components='all',cmap='RdPu')
```

Example 26 (python):
```python
group_col = 'fine_grain_merged'
groups_to_plot = ['VentricularCardiomyocytesLeftCompact','VentricularCardiomyocytesLeftTrabeculated',
             'VentricularCardiomyocytesRightCompact','VentricularCardiomyocytesRightTrabeculated']

ncols = 6
if len(groups_to_plot)%ncols==0:
    nrows = len(groups_to_plot)//ncols
else:
    nrows = len(groups_to_plot)//ncols+1

if len(groups_to_plot)<ncols:
    fig_width = len(groups_to_plot)%ncols
else:
    fig_width = ncols
    
fig,axs=plt.subplots(nrows=nrows,ncols=ncols,figsize=(fig_width*5,nrows*4))
# Flatten the axs array for easier iteration in case of multiple rows and columns
axs_flat = axs.flatten()

for i,grp in enumerate(groups_to_plot):
    # ad = adata.copy()
    # update colors
    # get color in adata
    # ind = list(ad.obs[group_col].cat.categories).index(celltype)
    # ad.uns[group_col][ind] = 'darkred'
    
    # title
    middle_ind = round(len(grp)/2)
    grp_title = f'{grp[:middle_ind]}\n{grp[middle_ind:]}'
    # plot
    sc.pl.embedding(adata, basis=basis,
                  color=group_col, groups=grp,
                  size=10,legend_loc=None,
                  frameon=False,title=grp_title,
                  show=False, ax=axs_flat[i],
                 )
# Turn off any remaining subplots
for j in range(len(groups_to_plot), len(axs_flat)):
    fig.delaxes(axs_flat[j])

# fig.suptitle('Main Title for the Figure')
fig.tight_layout()
```

Example 27 (python):
```python
# save
adata.write('/nfs/team205/heart/anndata_objects/Foetal/trisomy21/Euploid_T21Hearts_Aug2024_sel_subsampled-per-cellstate-donor.h5ad')
```

Example 28 (python):
```python
adata = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/trisomy21/Euploid_T21Hearts_Aug2024_sel_subsampled-per-cellstate-donor.h5ad')
print(adata.X.data[:5])
basis = 'umap_correcting-donor-region_n-layers-2'
adata
```

Example 29 (unknown):
```unknown
[1. 1. 2. 1. 2.]
```

Example 30 (python):
```python
AnnData object with n_obs × n_vars = 83477 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score', 'euploid_or_t21', 'donor_type', 'region_finest', 'flushed', 'scrublet_leiden', 'cluster_scrublet_score', 'leiden_scVI_0.1', 'leiden_scVI_0.2', 'leiden_scVI_0.3', 'leiden_scVI_0.5', 'leiden_scVI_0.7', 'leiden_scVI_1', 'celltypist_coarse', 'conf_score_coarse', 'cells_to_remove', 'celltypist_fine', 'conf_score_fine', 'coarse_grain_merged', 'fine_grain_merged', 'nhood_ixs_random', 'nhood_ixs_refined', 'nhood_kth_distance'
    var: 'gene_id', 'gene_name_scRNA-0-0', 'feature_type-1', 'mt-1', 'ribo-1', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1', 'gene_name_GenomicFeatures', 'gene_biotype_GenomicFeatures', 'gene_seq_start_GenomicFeatures', 'gene_seq_end_GenomicFeatures', 'seq_name_GenomicFeatures', 'seq_strand_GenomicFeatures', 'seq_coord_system_GenomicFeatures', 'description_GenomicFeatures', 'gene_id_version_GenomicFeatures', 'canonical_transcript_GenomicFeatures', 'gene_name'
    uns: 'coarse_grain_merged_colors', 'correcting-donor-region_n-layers-2', 'donor_colors', 'euploid_or_t21_colors', 'fine_grain_merged_colors', 'mid_grain_colors', 'nhood_adata', 'nhood_neighbors_key', 'region_colors', 'scVI_latent_correcting-donor-region_n-layers-2-forMilo', 'umap'
    obsm: 'X_umap_correcting-donor-region_n-layers-2', 'latent_gene_encoding', 'nhoods', 'scVI_latent_correcting-donor-region_n-layers-2'
    obsp: 'correcting-donor-region_n-layers-2_connectivities', 'correcting-donor-region_n-layers-2_distances', 'scVI_latent_correcting-donor-region_n-layers-2-forMilo_connectivities', 'scVI_latent_correcting-donor-region_n-layers-2-forMilo_distances'
```

Example 31 (python):
```python
group_col = 'euploid_or_t21'
groups_to_plot = ['euploid',
 'tri21',
                 ]

plt.rcParams['axes.grid'] = False
fig,axs=plt.subplots(nrows=2,ncols=4,figsize=(15,7))
# Flatten the axs array for easier iteration in case of multiple rows and columns
axs_flat = axs.flatten()

for i,grp in enumerate(groups_to_plot):
    sc.pl.embedding(adata, basis=basis, color=[group_col], groups=grp, 
               size=3, frameon=False,
               show=False, ax=axs_flat[i], legend_loc=None, legend_fontsize=5,
               title=grp
              )
# Turn off any remaining subplots
for j in range(len(groups_to_plot), len(axs_flat)):
    fig.delaxes(axs_flat[j])

# fig.suptitle('Main Title for the Figure')
fig.tight_layout()
plt.savefig(f'{figdir}/Tri21_RNA_UMAP_eup-t21-separated.pdf', bbox_inches="tight",dpi=300)
```

Example 32 (python):
```python
sc.pl.embedding(adata, basis=basis, color=['mid_grain'], frameon=False, show=False, title='mid_grain (euploid)')
plt.savefig(f'{figdir}/Tri21_RNA_UMAP_midgrain.pdf', bbox_inches="tight",dpi=300)
```

Example 33 (python):
```python
sc.pl.embedding(adata, basis=basis, color=['fine_grain_merged'], frameon=False, show=False)
plt.savefig(f'{figdir}/Tri21_RNA_UMAP_finegrain.pdf', bbox_inches="tight",dpi=300)
```

Example 34 (python):
```python
sc.pl.embedding(adata, basis=basis, color=['coarse_grain_merged'], frameon=False, show=False)
plt.savefig(f'{figdir}/Tri21_RNA_UMAP_coarsegrain.pdf', bbox_inches="tight",dpi=300)
```

---

## Read in¶

**URL:** http://127.0.0.1:9020/notebooks_html/Supplementary_materials/Macrophages/Mo-MP_no3_scFate.html

**Contents:**
- Read in¶
- scFates - Tree analysis¶
- Preprocessing pipeline from Palantir¶
  - Create projection dataframe¶
  - Run Palantir to obtain multiscale diffusion space¶
  - Generate embedding from the multiscale diffusion space¶
  - Plotting¶
- Tree learning with SimplePPT (or ElPiGraph)¶
  - Projecting results onto ForceAtlas2 embedding¶
  - Slecting roots and computing pseudotime¶
- Representing the trajectory and tree (on top of existing embedding)¶
- Save the object¶
- Plots for figure¶

ref: https://scfates.readthedocs.io/en/latest/Tree_Analysis_Bone_marrow_fates.html

142283 rows × 4 columns

7674 rows × 18 columns

**Examples:**

Example 1 (python):
```python
import scanpy as sc
import pandas as pd
import numpy as np
import anndata
import scipy
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (unknown):
```unknown
-----
anndata             0.8.0
numpy               1.21.5
pandas              1.3.5
scanpy              1.8.2
scipy               1.7.3
session_info        1.0.0
-----
```

Example 4 (python):
```python
PIL                 9.1.1
asciitree           NA
backcall            0.2.0
beta_ufunc          NA
binom_ufunc         NA
cached_property     1.5.1
cairo               1.20.0
cffi                1.14.4
cloudpickle         1.6.0
colorama            0.4.4
cycler              0.10.0
cython_runtime      NA
cytoolz             0.11.2
dask                2022.01.0
dateutil            2.8.1
decorator           4.4.2
defusedxml          0.6.0
fasteners           0.17.3
fsspec              2022.01.0
google              NA
h5py                3.1.0
igraph              0.10.2
importlib_metadata  1.7.0
ipykernel           5.4.2
ipython_genutils    0.2.0
jedi                0.17.2
jinja2              3.1.2
joblib              1.0.0
kiwisolver          1.3.1
leidenalg           0.9.0
llvmlite            0.35.0
louvain             0.7.0
markupsafe          2.1.1
matplotlib          3.5.2
mpl_toolkits        NA
msgpack             1.0.3
natsort             7.1.0
nbinom_ufunc        NA
numba               0.52.0
numcodecs           0.9.1
numexpr             2.7.2
packaging           21.3
parso               0.7.1
pexpect             4.8.0
pickleshare         0.7.5
pkg_resources       NA
plotly              5.10.0
prompt_toolkit      3.0.10
psutil              5.9.0
ptyprocess          0.7.0
pyarrow             9.0.0
pygments            2.13.0
pyparsing           2.4.7
pytz                2020.5
setuptools_scm      NA
sinfo               0.3.1
six                 1.14.0
sklearn             1.0.2
sphinxcontrib       NA
storemagic          NA
tables              3.6.1
tblib               1.7.0
texttable           1.6.3
threadpoolctl       2.1.0
tlz                 0.11.2
toolz               0.11.2
tornado             6.1
traitlets           5.0.5
typing_extensions   NA
wcwidth             0.2.5
yaml                5.4.1
zarr                2.11.3
zipp                NA
zmq                 21.0.1
zope                NA
```

Example 5 (unknown):
```unknown
-----
IPython             7.19.0
jupyter_client      6.1.11
jupyter_core        4.7.0
notebook            6.2.0
-----
Python 3.7.9 | packaged by conda-forge | (default, Dec  9 2020, 21:08:20) [GCC 9.3.0]
Linux-4.15.0-158-generic-x86_64-with-debian-bullseye-sid
-----
Session information updated at 2023-03-26 09:38
```

Example 6 (python):
```python
sc.settings.set_figure_params(dpi=300)
```

Example 7 (unknown):
```unknown
!ls /nfs/team205/heart/anndata_objects/Foetal/Immune
```

Example 8 (unknown):
```unknown
archive
C97_index.csv
MultiVI-model_foetal_Mono-MP_withContCov-CCCov_13Mar2023_feature-select-0003
MultiVI-model_foetal_Mono-MP_withContCov-CCCov_13Mar2023_feature-select-001
MultiVI-post_foetal_Mono-MP_withContCov-CCCov_13Mar2023_feature-select-0003.h5ad
MultiVI-post_foetal_Mono-MP_withContCov-CCCov_13Mar2023_feature-select-001.h5ad
MultiVI-pre_paired_foetal_MoMP_raw.h5ad
MultiVI-pre_scRNA_foetal_MoMP_raw.h5ad
RNA_foetal_raw_immune_16Feb2023.h5ad
RNA_foetal_raw_immune_16Feb2023_Multiome.h5ad
RNA_foetal_raw_immune_16Feb2023_Multiome.rds
RNA_foetal_raw_immune_16Feb2023.obs.csv
RNA_foetal_raw_Lymphoid_16Feb2023_CCregressed-scaled.h5ad
RNA_foetal_raw_Lymphoid_16Feb2023.h5ad
RNA_foetal_raw_Lymphoid_16Feb2023.obs.csv
RNA_foetal_raw_Myeloid_16Feb2023.h5ad
RNA_foetal_raw_Myeloid_16Feb2023.obs.csv
Tcycling
```

Example 9 (python):
```python
adata = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/Immune/MultiVI-post_foetal_Mono-MP_withContCov-CCCov_13Mar2023_feature-select-0003.h5ad')
adata
```

Example 10 (unknown):
```unknown
AnnData object with n_obs × n_vars = 7674 × 142283
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', '_indices', '_scvi_batch', '_scvi_labels', 'fine_grain_withcycling', 't', 'seg', 'edge', 't_sd'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id', 'n_cells'
    uns: '_scvi_manager_uuid', '_scvi_uuid', 'draw_graph', 'fine_grain_colors', 'fine_grain_withcycling_colors', 'graph', 'neighbors', 'ppt', 'pseudotime_list', 'seg_colors', 'umap', 'week_colors'
    obsm: 'MultiVI_latent', 'X_R', 'X_draw_graph_fa', 'X_palantir', 'X_umap', 'X_umap13', '_scvi_extra_categorical_covs', '_scvi_extra_continuous_covs'
    obsp: 'connectivities', 'distances'
```

Example 11 (python):
```python
sc.pl.umap(adata, color=['fine_grain_withcycling','week'], size=20, wspace=0.7, components='all', ncols=3)
```

Example 12 (python):
```python
sc.pl.umap(adata, color=['week'], groups=['5W','7W','9W','13W'], size=20, wspace=0.3, components='all', ncols=3)
```

Example 13 (python):
```python
import scFates as scf
import palantir
```

Example 14 (unknown):
```unknown
findfont: Font family ['Raleway'] not found. Falling back to DejaVu Sans.
findfont: Font family ['Lato'] not found. Falling back to DejaVu Sans.
```

Example 15 (python):
```python
projections = pd.DataFrame(adata.obsm["MultiVI_latent"],index=adata.obs_names)
projections
```

Example 16 (unknown):
```unknown
# fate probability mapping of the cells
dm_res = palantir.utils.run_diffusion_maps(projections,knn=50)
ms_data = palantir.utils.determine_multiscale_space(dm_res,n_eigs=5)
```

Example 17 (unknown):
```unknown
Determing nearest neighbor graph...
```

Example 18 (python):
```python
# Generate embedding from the multiscale diffusion space
adata.obsm["X_palantir"]=ms_data.values
sc.pp.neighbors(adata,n_neighbors=50,use_rep="X_palantir")
```

Example 19 (python):
```python
sc.pl.embedding(adata, basis='X_palantir',color=['fine_grain_withcycling'], components='all',ncols=3,wspace=0.6,size=10)
```

Example 20 (unknown):
```unknown
findfont: Font family ['Bitstream Vera Sans'] not found. Falling back to DejaVu Sans.
findfont: Font family ['Bitstream Vera Sans'] not found. Falling back to DejaVu Sans.
```

Example 21 (python):
```python
# draw ForceAtlas2 embedding using UMAP dim1 and dim2 as initial position
# make obsm with UMAP1 and 3
# since it looks well capturing the transition of Mono -> MoMP -> TRM and YS-MP -> TRM
adata.obsm['X_umap13'] = adata.obsm['X_umap'][:,[0,2]]
sc.tl.draw_graph(adata,init_pos='X_umap13',random_state=0)
```

Example 22 (python):
```python
adata.uns["week_colors"][list(adata.obs['week'].cat.categories).index('13W')]='purple'
sc.pl.draw_graph(adata,color="week",groups=['5W','7W', '13W'],color_map="RdBu_r")
```

Example 23 (python):
```python
sc.pl.draw_graph(adata,color="fine_grain_withcycling",color_map="RdBu_r")
```

Example 24 (python):
```python
scf.tl.tree(adata,method="ppt",Nodes=50,use_rep="palantir",
            device="cpu",seed=5)
```

Example 25 (unknown):
```unknown
inferring a principal tree --> parameters used 
    50 principal points, sigma = 0.1, lambda = 1, metric = euclidean
    fitting:  84%|████████▍ | 42/50 [00:04<00:00,  8.86it/s]
    converged
    finished (0:00:04) --> added 
    .uns['ppt'], dictionnary containing inferred tree.
    .obsm['X_R'] soft assignment of cells to principal points.
    .uns['graph']['B'] adjacency matrix of the principal points.
    .uns['graph']['F'] coordinates of principal points in representation space.
```

Example 26 (python):
```python
scf.pl.graph(adata,tips=False,forks=False,nodes=[2,5,29,11,47,28])
```

Example 27 (python):
```python
adata.uns['graph']['pp_seg']
```

Example 28 (python):
```python
# Selecting a root and computing pseudotime
scf.tl.roots(adata,[29,2],meeting=47)
# generate 100 mappings of pseudotime, to account for cell asssignment uncertainty.
# .obs will be saved the mean of all calculated pseudotimes.
scf.tl.pseudotime(adata,n_jobs=20,n_map=1,seed=40)
```

Example 29 (python):
```python
root selected --> added
    2 is the farthest root.
    .uns['graph']['root'] farthest root selected.
    .uns['graph']['root2'] 2nd root selected.
    .uns['graph']['meeting'] meeting point on the tree.
    .uns['graph']['pp_info'] for each PP, its distance vs root and segment assignment.
    .uns['graph']['pp_seg'] segments network information.
projecting cells onto the principal graph
    finished (0:00:05) --> added
    .obs['edge'] assigned edge.
    .obs['t'] pseudotime value.
    .obs['seg'] segment of the tree assigned.
    .obs['milestones'] milestone assigned.
    .uns['pseudotime_list'] list of cell projection from all mappings.
```

Example 30 (python):
```python
scf.pl.trajectory(adata,arrows=True,arrow_offset=3,scale_path=0.5,
                  # root_milestone=29,milestones=27
                 )
```

Example 31 (python):
```python
sc.pl.draw_graph(adata,color=["seg","milestones"],legend_loc='on data',legend_fontsize=10)
# scf.tl.rename_milestones(adata,["bifA","Ery","DC","Root","BifB","Mono"])
# we change the color of the root milestone for better visualisations
# adata.uns["milestones_colors"][3]="#17bece"
```

Example 32 (python):
```python
adata.uns['graph']['pp_seg']
```

Example 33 (python):
```python
# save
adata.write('/nfs/team205/heart/anndata_objects/Foetal/Immune/MultiVI-post_foetal_Mono-MP_withContCov-CCCov_13Mar2023_feature-select-0003.h5ad')
```

Example 34 (python):
```python
# read in
adata = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/Immune/MultiVI-post_foetal_Mono-MP_withContCov-CCCov_13Mar2023_feature-select-0003.h5ad')
adata
```

Example 35 (unknown):
```unknown
AnnData object with n_obs × n_vars = 7674 × 142283
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', '_indices', '_scvi_batch', '_scvi_labels', 'fine_grain_withcycling', 't', 'seg', 'edge', 't_sd', 'milestones'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id', 'n_cells'
    uns: '_scvi_manager_uuid', '_scvi_uuid', 'draw_graph', 'fine_grain_colors', 'fine_grain_withcycling_colors', 'graph', 'milestones_colors', 'neighbors', 'ppt', 'pseudotime_list', 'seg_colors', 'umap', 'week_colors'
    obsm: 'MultiVI_latent', 'X_R', 'X_draw_graph_fa', 'X_palantir', 'X_umap', 'X_umap13', '_scvi_extra_categorical_covs', '_scvi_extra_continuous_covs'
    obsp: 'connectivities', 'distances'
```

Example 36 (python):
```python
import scanpy as sc
import pandas as pd
import numpy as np
import anndata
import scipy
import scFates as scf
import matplotlib as mpl
import matplotlib.pyplot as plt
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42
```

Example 37 (python):
```python
sc.settings.set_figure_params(dpi=80,dpi_save=300)
```

Example 38 (python):
```python
%matplotlib inline
```

Example 39 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk'
```

Example 40 (python):
```python
# read in adata
adata = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/Immune/MultiVI-post_foetal_Mono-MP_withContCov-CCCov_13Mar2023_feature-select-0003.h5ad')
adata
```

Example 41 (python):
```python
AnnData object with n_obs × n_vars = 7674 × 142283
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', '_indices', '_scvi_batch', '_scvi_labels', 'fine_grain_withcycling', 't', 'seg', 'edge', 't_sd', 'milestones'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id', 'n_cells'
    uns: '_scvi_manager_uuid', '_scvi_uuid', 'adata_auc_gene', 'adata_auc_region', 'adata_gex', 'adata_gex_lognorm', 'draw_graph', 'fine_grain_colors', 'fine_grain_withcycling_colors', 'graph', 'milestones_colors', 'neighbors', 'ppt', 'pseudotime_list', 'seg_colors', 'umap', 'week_colors'
    obsm: 'MultiVI_latent', 'X_R', 'X_draw_graph_fa', 'X_palantir', 'X_umap', 'X_umap13', '_scvi_extra_categorical_covs', '_scvi_extra_continuous_covs'
    obsp: 'connectivities', 'distances'
```

Example 42 (python):
```python
scf.pl.trajectory(adata,arrows=True,arrow_offset=3,scale_path=0.5, frameon=False,show=False)
plt.savefig(f"{figdir}/Force-directed-graph_MoMP_pseudotime.pdf", bbox_inches="tight",dpi=300)
```

Example 43 (unknown):
```unknown
/nfs/team205/kk18/miniconda3/envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:392: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap', 'norm' will be ignored
  cax = scatter(
```

---

## Variables¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/VisiumHD/no7_manual-annotation_PericardialCells.html

**Contents:**
- Variables¶
- Read in data¶
- Remove mid-grain markers from the feature¶
- PCA, Clustering, UMAP¶
- Markers¶
- Save¶

2480 rows × 7 columns

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os

import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 

import celltypist
```

Example 2 (python):
```python
# import session_info
# session_info.show()
```

Example 3 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 4 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 5 (python):
```python
adata_dir = '/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/subsets'
sample_id = 'HEA_FOET14880396'
celltype = 'PericardialCells'
```

Example 6 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk/Aug2024'
table_dir = '/nfs/team205/heart/fetal_ms_figures/tables/Aug2024'
```

Example 7 (python):
```python
# read in
adata = sc.read_h5ad(f'{adata_dir}/{sample_id}_{celltype}_b2c_cells_filtered_lognorm.h5ad')
adata
```

Example 8 (unknown):
```unknown
AnnData object with n_obs × n_vars = 3061 × 15212
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'leiden_0.2', 'leiden_0.4', 'leiden_0.6', 'leiden_1', 'leiden_1.2', 'leiden_1.5', 'leiden_2'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1.2', 'leiden_1.2_colors', 'leiden_1.5', 'leiden_1.5_colors', 'leiden_1_colors', 'leiden_2', 'leiden_2_colors', 'log1p', 'neighbors', 'pca', 'spatial', 'umap'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 9 (python):
```python
# read in
midgrain_markers = pd.read_csv('/nfs/team205/kk18/notebooks/Foetal/VisiumHD/re-seq_mpp0p3/sc_midgrain_markers.csv',
                               index_col=0
                              )
# remove the markers of celltype of interest
midgrain_markers = midgrain_markers[midgrain_markers['group']!=celltype]
midgrain_markers
```

Example 10 (python):
```python
# remove
mask = [x not in list(midgrain_markers['names']) for x in adata.var_names]
bdata = adata[:,mask]
bdata
```

Example 11 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 3061 × 13976
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'leiden_0.2', 'leiden_0.4', 'leiden_0.6', 'leiden_1', 'leiden_1.2', 'leiden_1.5', 'leiden_2'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1.2', 'leiden_1.2_colors', 'leiden_1.5', 'leiden_1.5_colors', 'leiden_1_colors', 'leiden_2', 'leiden_2_colors', 'log1p', 'neighbors', 'pca', 'spatial', 'umap'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 12 (python):
```python
# HVGs, using the object which doesn't have midgrain markers
sc.pp.highly_variable_genes(bdata, flavor='seurat', min_mean=0.0125, max_mean=3, min_disp=0.5)
print(f'number of hvgs: {sum(bdata.var.highly_variable)}')
# PCA using the degected HVGs
sc.tl.pca(bdata, use_highly_variable=True)
adata.obsm['X_pca'] = bdata.obsm['X_pca'].copy()
# create neighbour graph
sc.pp.neighbors(adata,n_neighbors=15,use_rep='X_pca')
# visualisation with umap
sc.tl.umap(adata,min_dist=0.5)
sc.pl.embedding(adata,basis='X_umap',color=['bin_count','total_counts','n_genes_by_counts',"conf_score_midmod2fine","celltypist_midmod2fine",],
                wspace=0.2,cmap='RdPu',vmax='p100',ncols=5)
# leiden clustering
resolutions_list = [0.2,0.4,0.6,1,1.2,1.5,2]
for resolution in resolutions_list:
    sc.tl.leiden(adata,resolution=resolution,key_added=f'leiden_{str(resolution)}',n_iterations=2)
sc.pl.embedding(adata,basis='X_umap',
                color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                wspace=0.3)
```

Example 13 (unknown):
```unknown
number of hvgs: 3497
```

Example 14 (python):
```python
# read in markers identified using single-nucleus data
markers_df = pd.read_csv(f'/nfs/team205/kk18/notebooks/Foetal/VisiumHD/re-seq_mpp0p3/sc_{celltype}_finegrain_markers.csv')

mask1 = markers_df['group_mean']>1
mask2 = markers_df['logfoldchanges']>1
markers_df = markers_df[mask1&mask2]

markers_df
```

Example 15 (python):
```python
markers = {}
n_top = 10
for c in set(markers_df['group']):
    markers_df_sub = markers_df[markers_df['group']==c]
    markers_df_sub.sort_values('logfoldchanges',ascending=False)
    markers[c]=list(markers_df_sub[:n_top]['names'])

# manually add genes
markers['Cycling'] = ['MKI67','TOP2A'] 

genes = []
for g in markers.values():
    genes = genes + [x for x in g if (x in list(adata.var_names))&(x not in genes)]
for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in list(adata.var_names)]
len(genes)
```

Example 16 (python):
```python
sc.pl.embedding(adata,basis='X_umap',color=genes, size=20,
               ncols=6, wspace=0.2,cmap='RdPu',vmax='p99.7')
```

Example 17 (python):
```python
sc.pl.embedding(adata,basis='X_umap',
                color=['celltypist_midmod2fine', 'conf_score_midmod2fine'],
                wspace=0.7)
```

Example 18 (python):
```python
# umap per celltype to highlight
group_col = 'celltypist_midmod2fine'
groups_to_plot = list(set(adata.obs[group_col]))

plt.rcParams['axes.grid'] = False
fig,axs=plt.subplots(nrows=2,ncols=4,figsize=(15,7))
# Flatten the axs array for easier iteration in case of multiple rows and columns
axs_flat = axs.flatten()

for i,grp in enumerate(groups_to_plot):
    sc.pl.umap(adata, color=[group_col], groups=grp, 
               size=30, frameon=False,
               show=False, ax=axs_flat[i], legend_loc=None, legend_fontsize=5,
               title=grp
              )
# Turn off any remaining subplots
for j in range(len(groups_to_plot), len(axs_flat)):
    fig.delaxes(axs_flat[j])

# fig.suptitle('Main Title for the Figure')
fig.tight_layout()
```

Example 19 (unknown):
```unknown
resolution_sel = 1
```

Example 20 (python):
```python
markers_others = {
    'CM': ['TTN', 'TNNT2','MYH6',
        'MYH7', 'MYL2', 'FHL2', # vCN
            'NPPA', 'MYL7', 'MYL4'], #aCM
    'FB': ['DCN', 'GCN', 'PDGFRA','COL1A1','COL1A2'],
    'EC': ['VWF', 'PECAM1', 'CDH5','RGCC', 'FABP5',],
    'Peri': ['RGS5', 'ABCC9', 'KCNJ8'],
    'SMC': ['MYH11', 'TAGLN', 'ACTA2'],
    'Meso': ['MSLN', 'WT1', 'BNC1'],
    'Neuro': ['PLP1', 'NRXN1', 'NRXN3','PRPH', 'NEFL', 'NEFM', 'NEFH', 'STMN2', 'SST', 'SLC10A4', 'SLC18A3', 'TH', 'NPY'],
    'Adipo': ['GPAM', 'FASN', 'LEP'],
    'Myelo': ['CD14', 'C1QA', 'CD68','LYVE1','TIMD4'],
    'Lymph': ['CD8A', 'IL7R', 'CD40LG'],
    'B': ['CD19','SDC1','IGHG1','IGHA1'],
    'Mast': ['KIT', 'CPA3'],
}

for key in markers_others.keys():
    markers_others[key]=[x for x in markers_others[key] if x in adata.var_names]

sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
sc.pl.dotplot(adata,
              markers_others, 
              groupby=f'leiden_{str(resolution_sel)}',
              dendrogram=True,
              standard_scale="var",
              color_map="Reds",
              swap_axes=False,
             )
```

Example 21 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: CM, FB, EC, etc.
```

Example 22 (python):
```python
sc.tl.dendrogram(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.pl.dotplot(adata,
              var_names=markers,
              groupby=f'leiden_{str(resolution_sel)}',
              standard_scale='var',
              dendrogram=True
             )
```

Example 23 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: PericardialCellsParietal, PericardialCellsIntermediate, PericardialCellsFibrous, etc.
```

Example 24 (python):
```python
# DEG analysis
print(adata.X.data[:5])
adata.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata, dendrogram=True, standard_scale='var', return_fig=True)
dp.add_totals().show()
```

Example 25 (unknown):
```unknown
[2.1685468  2.22569867 2.00533587 2.08507898 2.52518411]
```

Example 26 (python):
```python
# DEG analysis
print(adata.X.data[:5])
adata.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata, dendrogram=True, standard_scale='var', return_fig=True, n_genes=5)
dp.add_totals().show()
```

Example 27 (unknown):
```unknown
[2.1685468  2.22569867 2.00533587 2.08507898 2.52518411]
```

Example 28 (python):
```python
sc.pl.embedding(adata,basis='X_umap',
                color=f'leiden_{str(resolution_sel)}',
                wspace=0.3, legend_loc='on data')
```

Example 29 (python):
```python
sc.pl.violin(adata,keys=['total_counts','bin_count'],groupby=f'leiden_{str(resolution_sel)}')
```

Example 30 (unknown):
```unknown
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 31 (unknown):
```unknown
'PericardialCells'
```

Example 32 (python):
```python
# annotate
adata.obs['fine_grain_manual'] = adata.obs[f'leiden_{str(resolution_sel)}'].copy()
adata.obs.replace({'fine_grain_manual':{
    '0':'PericardialCellsFibrous',
    '1':'PericardialCellsFibrous',
    '2':'PericardialCellsIntermediate',
    '3':'PericardialCellsFibrous',
    '4':'PericardialCellsFibrous', # contamination from valveECs
    '5':'PericardialCellsIntermediate',
    '6':'PericardialCells_DoubletsWithCMs',
}},inplace=True)
adata.obs['fine_grain_manual'].value_counts()
```

Example 33 (unknown):
```unknown
fine_grain_manual
PericardialCellsFibrous             2231
PericardialCellsIntermediate         730
PericardialCells_DoubletsWithCMs     100
Name: count, dtype: int64
```

Example 34 (python):
```python
ad = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/Feb28ObjectRaw_finegrain_updated.h5ad',backed='r')
ad = ad[ad.obs['mid_grain']==celltype]
set(ad.obs['fine_grain'])
```

Example 35 (unknown):
```unknown
{'PericardialCellsFibrous',
 'PericardialCellsIntermediate',
 'PericardialCellsParietal'}
```

Example 36 (python):
```python
sc.pl.embedding(adata,basis='X_umap',
                color='fine_grain_manual',
                frameon=False,
                wspace=0.3)
```

Example 37 (python):
```python
# save
adata.write(f'{adata_dir}/{sample_id}_{celltype}_b2c_cells_filtered_lognorm.h5ad')
print(f'{adata_dir}/{sample_id}_{celltype}_b2c_cells_filtered_lognorm.h5ad')
```

---

## Variables¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/Xenium/no2_integration_and_cell-type-annotation/no6_manual-annotation_lymphoid.html

**Contents:**
- Variables¶
- Read in data¶
- 1st round annotation¶
- Subset and reclustering¶
- Transfer annotation to the original adata¶
- Save¶

--> cluster 5 has lower total counts and smaller cell area, and no specific gene expression --> annotate as "unassigned"

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os

import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 

import scanpy.external as sce
```

Example 2 (python):
```python
# import session_info
# session_info.show()
```

Example 3 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/home/kk837/rfs/rfs-teichlab-nfs-iCNyzSAaucw/kk837/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 4 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 5 (unknown):
```unknown
!ls /lustre/scratch126/cellgen/team205/kk18/xenium/objects/foetal/subsets
```

Example 6 (python):
```python
adata_dir = '/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium/subsets'
sample_id = 'C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat'
celltype = 'LymphoidCells'
```

Example 7 (unknown):
```unknown
# figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk/Aug2024'
# table_dir = '/nfs/team205/heart/fetal_ms_figures/tables/Aug2024'
```

Example 8 (unknown):
```unknown
latent_space = 'pca_harmony'
```

Example 9 (python):
```python
# read in
adata = sc.read_h5ad(f'{adata_dir}/{sample_id}_5K_{celltype}_lognorm.h5ad')
adata
```

Example 10 (unknown):
```unknown
AnnData object with n_obs × n_vars = 3116 × 4614
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_3.0', 'multi_celltypes_coarse', 'leiden_0.2', 'leiden_0.4', 'leiden_0.6', 'leiden_1'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1', 'n_cells', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'dendrogram_pca_harmony_leiden_1.0', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1_colors', 'log1p', 'multi_celltypes_coarse_colors', 'pca', 'pca_harmony', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.2_colors', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.3_colors', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.4_colors', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.5_colors', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_0.8_colors', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.0_colors', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_1.5_colors', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_2.0_colors', 'pca_harmony_leiden_3.0', 'pca_harmony_leiden_3.0_colors', 'tissue_block_id_colors', 'umap'
    obsm: 'X_pca', 'X_pca_harmony', 'X_umap_pca_harmony', 'spatial'
    varm: 'PCs'
    obsp: 'pca_harmony_connectivities', 'pca_harmony_distances'
```

Example 11 (python):
```python
adata.X.data[:5]
```

Example 12 (unknown):
```unknown
array([3.1345382, 3.1345382, 3.1345382, 3.1345382, 3.1345382],
      dtype=float32)
```

Example 13 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=['total_counts','n_genes_by_counts','cell_area','tissue_block_id'],
                    wspace=0.2,cmap='RdPu',vmax='p100')
```

Example 14 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=['multi_celltypes_coarse',"conf_score_midmod2fine","celltypist_midmod2fine",],
                    wspace=0.2,cmap='RdPu',vmax='p100')
resolutions_list = [0.2,0.4,0.6,1]
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                    color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                    wspace=0.2)
```

Example 15 (python):
```python
%%time
# leiden clustering
resolutions_list = [# 0.2,0.4,0.6,1, # already done
                    1.2,1.5,2]
for resolution in resolutions_list:
    sc.tl.leiden(adata,neighbors_key=latent_space,
                     resolution=resolution,key_added=f'leiden_{str(resolution)}',n_iterations=2)
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                wspace=0.3)
```

Example 16 (unknown):
```unknown
<timed exec>:5: FutureWarning: In the future, the default backend for leiden will be igraph instead of leidenalg.

 To achieve the future defaults please pass: flavor="igraph" and n_iterations=2.  directed must also be False to work with igraph's implementation.
```

Example 17 (unknown):
```unknown
CPU times: user 1.94 s, sys: 82.8 ms, total: 2.03 s
Wall time: 2.07 s
```

Example 18 (python):
```python
# markers for fine-grained cell types
markers={
     'T cell': ['CD3E', 'CD3D','CD4', 'CD8A', 'CD8B','TRAC','TRBC1','TRDC','TRGC1'],
    'naive': ['TCF7', 'LEF1', 'SELL'],
    'CD4T_reg': ['FOXP3', 'CTLA4', 'IL2RA', 'IKZF2'],
    'cytotoxic': ['PRF1', 'GZMB', 'GZMH', 'NKG7'],
    'NK': ['KLRF1', 'KLRD1', 'NCAM1','FCGR3A','IFNG'],
    'ILC3': ['KIT', 'IL18R1','RORC','IL17A'],
    
    'B': ['CD19','CD20','CD22','CD79A', 'CD79B','MS4A1','IL7R',],
    '':['CCR7','TNFRSF13B',],
    'ProB':['KIT','SPN','CD34'],
    'B1':['CD27','CCR10','MS4A1','SPN'],
    'Plasma': ['CD27','CCR10','CD38','SDC1','JCAIN'],
    'Abs':['IGHM','IGHD', 'IGHG1','IGHG2', 'IGHG3','IGHG4', 'IGHA1','IGHE','IGLC1','IGKC'],
    'Gene editing': ['AICDA', 'RAG1','RAG2'],
    
    
    'Cycling':['MKI67','TOP2A'],

    'TFs': ['TBX21', 'GATA3', 'RORC', 'BCL6',],
}

genes = []
for g in markers.values():
    genes = genes + [x for x in g if (x in adata.var_names)&(x not in genes)]
for key in markers.keys():
    markers[key]=[x for x in markers[key] if x in adata.var_names]
    
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=genes,
               ncols=5, wspace=0.2,cmap='RdPu',vmax='p100')
```

Example 19 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color="AIRE",
               ncols=5, wspace=0.2,cmap='RdPu',vmax='p100')
```

Example 20 (unknown):
```unknown
resolution_sel = 1
```

Example 21 (python):
```python
sc.tl.dendrogram(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.pl.dotplot(adata,
              var_names=markers,
              groupby=f'leiden_{str(resolution_sel)}',
              standard_scale='var',
              dendrogram=True
             )
```

Example 22 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: T cell, naive, CD4T_reg, etc.
```

Example 23 (python):
```python
markers_others = {
    'CM': ['TTN', 'TNNT2','MYH6',
        'MYH7', 'MYL2', 'FHL2', # vCN
            'NPPA', 'MYL7', 'MYL4'], #aCM
    'FB': ['DCN', 'GCN', 'PDGFRA','COL1A1','COL1A2'],
    'EC': ['VWF', 'PECAM1', 'CDH5','RGCC', 'FABP5',],
    'Peri': ['RGS5', 'ABCC9', 'KCNJ8'],
    'SMC': ['MYH11', 'TAGLN', 'ACTA2'],
    'Meso': ['MSLN', 'WT1', 'BNC1'],
    'Neuro': ['PLP1', 'NRXN1', 'NRXN3','PRPH', 'NEFL', 'NEFM', 'NEFH', 'STMN2', 'SST', 'SLC10A4', 'SLC18A3', 'TH', 'NPY'],
    'Adipo': ['GPAM', 'FASN', 'LEP'],
    'Myelo': ['CD14', 'C1QA', 'CD68','LYVE1','TIMD4'],
    'Lymph': ['CD8A', 'IL7R', 'CD40LG'],
    'B': ['CD19','SDC1','IGHG1','IGHA1'],
    'Mast': ['KIT', 'CPA3'],
}

for key in markers_others.keys():
    markers_others[key]=[x for x in markers_others[key] if x in adata.var_names]

sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
sc.pl.dotplot(adata,
              markers_others, 
              groupby=f'leiden_{str(resolution_sel)}',
              dendrogram=True,
              standard_scale="var",
              color_map="Reds",
              swap_axes=False,
             )
```

Example 24 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: CM, FB, EC, etc.
```

Example 25 (python):
```python
print(adata.X.data[:5])
adata.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata,groupby=f'leiden_{str(resolution_sel)}')
sc.tl.dendrogram(adata, groupby=f'leiden_{str(resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata, dendrogram=True, standard_scale='var', return_fig=True)
dp.add_totals().show()
```

Example 26 (unknown):
```unknown
[3.1345382 3.1345382 3.1345382 3.1345382 3.1345382]
```

Example 27 (python):
```python
sc.pl.violin(adata,keys=['total_counts','cell_area'],groupby=f'leiden_{str(resolution_sel)}')
```

Example 28 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=f'leiden_{str(resolution_sel)}',legend_loc='on data',
                )
```

Example 29 (python):
```python
# annotate
adata.obs['tmp_annotation'] = adata.obs[f'leiden_{str(resolution_sel)}'].copy()
adata.obs.replace({'tmp_annotation':{
    '0':'ThyroidEpithelialCells',
    '1':'SOX2_LAMB3_pos',
    '2':'LymphoidCells',
    '3':'LymphoidCells',
    '4':'ThyroidEpithelialCells',
    '5':'unassigned',
    '6':'LymphoidCells',
    '7':'ThyroidEpithelialCells',
    '8':'Erythrocytes',
}},inplace=True)
adata.obs['tmp_annotation'].value_counts()
```

Example 30 (python):
```python
/tmp/ipykernel_78172/3528645312.py:3: FutureWarning: The behavior of Series.replace (and DataFrame.replace) with CategoricalDtype is deprecated. In a future version, replace will only be used for cases that preserve the categories. To change the categories, use ser.cat.rename_categories instead.
  adata.obs.replace({'tmp_annotation':{
```

Example 31 (unknown):
```unknown
tmp_annotation
ThyroidEpithelialCells    1520
LymphoidCells              800
SOX2_LAMB3_pos             461
unassigned                 283
Erythrocytes                52
Name: count, dtype: int64
```

Example 32 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color='tmp_annotation'
                )
```

Example 33 (python):
```python
adata_sub = adata[adata.obs['tmp_annotation'].isin(['LymphoidCells'])]
```

Example 34 (python):
```python
# drop leiden clustering results
resolutions_list = [0.2,0.4,0.6,1, 1.2,1.5,2]
for resolution in resolutions_list:
    adata_sub.obs.drop(f'leiden_{str(resolution)}',axis=1,inplace=True)
adata_sub
```

Example 35 (unknown):
```unknown
AnnData object with n_obs × n_vars = 800 × 4614
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_3.0', 'multi_celltypes_coarse', 'tmp_annotation'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1', 'n_cells', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'dendrogram_pca_harmony_leiden_1.0', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1_colors', 'log1p', 'multi_celltypes_coarse_colors', 'pca', 'pca_harmony', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.2_colors', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.3_colors', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.4_colors', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.5_colors', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_0.8_colors', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.0_colors', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_1.5_colors', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_2.0_colors', 'pca_harmony_leiden_3.0', 'pca_harmony_leiden_3.0_colors', 'tissue_block_id_colors', 'umap', 'leiden_1.2', 'leiden_1.5', 'leiden_2', 'leiden_1.2_colors', 'leiden_1.5_colors', 'leiden_2_colors', 'dendrogram_leiden_1', 'rank_genes_groups', 'tmp_annotation_colors'
    obsm: 'X_pca', 'X_pca_harmony', 'X_umap_pca_harmony', 'spatial'
    varm: 'PCs'
    obsp: 'pca_harmony_connectivities', 'pca_harmony_distances'
```

Example 36 (python):
```python
## Remove mid-grain markers from the feature
# read in markers
midgrain_markers = pd.read_csv('/home/kk837/rfs/rfs-teichlab-nfs-iCNyzSAaucw/kk837/notebooks/Foetal/VisiumHD/re-seq_mpp0p3/sc_midgrain_markers.csv',
                               index_col=0)
# remove the markers of celltype of interest
midgrain_markers = midgrain_markers[midgrain_markers['group']!=celltype]
# remove
mask = [x not in list(midgrain_markers['names']) for x in adata_sub.var_names]
bdata = adata_sub[:,mask]
bdata
```

Example 37 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 800 × 4178
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_3.0', 'multi_celltypes_coarse', 'tmp_annotation'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1', 'n_cells', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'dendrogram_pca_harmony_leiden_1.0', 'hvg', 'leiden_0.2', 'leiden_0.2_colors', 'leiden_0.4', 'leiden_0.4_colors', 'leiden_0.6', 'leiden_0.6_colors', 'leiden_1', 'leiden_1_colors', 'log1p', 'multi_celltypes_coarse_colors', 'pca', 'pca_harmony', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.2_colors', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.3_colors', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.4_colors', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.5_colors', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_0.8_colors', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.0_colors', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_1.5_colors', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_2.0_colors', 'pca_harmony_leiden_3.0', 'pca_harmony_leiden_3.0_colors', 'tissue_block_id_colors', 'umap', 'leiden_1.2', 'leiden_1.5', 'leiden_2', 'leiden_1.2_colors', 'leiden_1.5_colors', 'leiden_2_colors', 'dendrogram_leiden_1', 'rank_genes_groups', 'tmp_annotation_colors'
    obsm: 'X_pca', 'X_pca_harmony', 'X_umap_pca_harmony', 'spatial'
    varm: 'PCs'
    obsp: 'pca_harmony_connectivities', 'pca_harmony_distances'
```

Example 38 (python):
```python
# remove cell cycle genes
cell_cycle_genes = ['MCM5','PCNA','TYMS','FEN1','MCM2','MCM4','RRM1','UNG','GINS2','MCM6','CDCA7','DTL','PRIM1','UHRF1','MLF1IP','HELLS','RFC2','RPA2','NASP',
                'RAD51AP1','GMNN','WDR76','SLBP','CCNE2','UBR7','POLD3','MSH2','ATAD2','RAD51','RRM2','CDC45','CDC6','EXO1','TIPIN','DSCC1','BLM','CASP8AP2',
                'USP1','CLSPN','POLA1','CHAF1B','BRIP1','E2F8','HMGB2','CDK1','NUSAP1','UBE2C','BIRC5','TPX2','TOP2A','NDC80','CKS2','NUF2','CKS1B','MKI67',
                'TMPO','CENPF','TACC3','FAM64A','SMC4','CCNB2','CKAP2L','CKAP2','AURKB','BUB1','KIF11','ANP32E','TUBB4B','GTSE1','KIF20B','HJURP','CDCA3','HN1',
                'CDC20','TTK','CDC25C','KIF2C','RANGAP1','NCAPD2','DLGAP5','CDCA2','CDCA8','ECT2','KIF23','HMMR','AURKA','PSRC1','ANLN','LBR','CKAP5','CENPE','CTCF',
                'NEK2','G2E3','GAS2L3','CBX5','CENPA']
mask = [x not in cell_cycle_genes for x in bdata.var_names]
bdata = bdata[:,mask]
bdata.shape
```

Example 39 (unknown):
```unknown
(800, 4125)
```

Example 40 (python):
```python
# HVGs, using the object which doesn't have midgrain markers
sc.pp.highly_variable_genes(bdata, flavor='seurat', min_mean=0.0125, max_mean=3, min_disp=0.5)
print(f'number of hvgs: {sum(bdata.var.highly_variable)}')
# PCA using the degected HVGs
sc.tl.pca(bdata, use_highly_variable=True)
adata_sub.obsm['X_pca'] = bdata.obsm['X_pca'].copy()
# harmony integration
sce.pp.harmony_integrate(adata_sub, 'tissue_block_id', basis='X_pca', adjusted_basis='X_pca_harmony')
# create neighbour graph
latent_space = 'pca_harmony'
sc.pp.neighbors(adata_sub,n_neighbors=15,use_rep=f"X_{latent_space}",key_added=latent_space)
# visualisation with umap
sc.tl.umap(adata_sub,min_dist=0.5,neighbors_key=latent_space)
adata_sub.obsm[f"X_umap_{latent_space}"]=adata_sub.obsm["X_umap"]
del adata_sub.obsm["X_umap"]
```

Example 41 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_highly_variable_genes.py:698: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  adata.uns["hvg"] = {"flavor": flavor}
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/preprocessing/_pca/__init__.py:438: FutureWarning: Argument `use_highly_variable` is deprecated, consider using the mask argument. Use_highly_variable=True can be called through mask_var="highly_variable". Use_highly_variable=False can be called through mask_var=None
  warn(msg, FutureWarning)
```

Example 42 (unknown):
```unknown
number of hvgs: 859
```

Example 43 (python):
```python
2025-06-12 16:25:16,957 - harmonypy - INFO - Computing initial centroids with sklearn.KMeans...
2025-06-12 16:25:17,130 - harmonypy - INFO - sklearn.KMeans initialization complete.
2025-06-12 16:25:17,172 - harmonypy - INFO - Iteration 1 of 10
2025-06-12 16:25:17,255 - harmonypy - INFO - Iteration 2 of 10
2025-06-12 16:25:17,296 - harmonypy - INFO - Iteration 3 of 10
2025-06-12 16:25:17,325 - harmonypy - INFO - Converged after 3 iterations
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm
```

Example 44 (python):
```python
%%time
# reclustering
# leiden clustering
resolutions_list = [0.2,0.4,0.6,1, 1.2,1.5,2]
for resolution in resolutions_list:
    sc.tl.leiden(adata_sub,neighbors_key=latent_space,
                 resolution=resolution,key_added=f'leiden_{str(resolution)}',n_iterations=2)
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color=[f'leiden_{str(resolution)}' for resolution in resolutions_list],
                wspace=0.3)
```

Example 45 (unknown):
```unknown
CPU times: user 1.52 s, sys: 97.6 ms, total: 1.62 s
Wall time: 1.62 s
```

Example 46 (python):
```python
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color=genes,
               ncols=5, wspace=0.2,cmap='RdPu',vmax='p100')
```

Example 47 (unknown):
```unknown
sub_resolution_sel = 1
```

Example 48 (python):
```python
sc.tl.dendrogram(adata_sub,groupby=f'leiden_{str(sub_resolution_sel)}')
sc.pl.dotplot(adata_sub,
              var_names=markers,
              groupby=f'leiden_{str(sub_resolution_sel)}',
              standard_scale='var',
              dendrogram=True
             )
```

Example 49 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: T cell, naive, CD4T_reg, etc.
```

Example 50 (python):
```python
markers_others = {
    'CM': ['TTN', 'TNNT2','MYH6',
        'MYH7', 'MYL2', 'FHL2', # vCN
            'NPPA', 'MYL7', 'MYL4'], #aCM
    'FB': ['DCN', 'GCN', 'PDGFRA','COL1A1','COL1A2'],
    'EC': ['VWF', 'PECAM1', 'CDH5','RGCC', 'FABP5',],
    'Peri': ['RGS5', 'ABCC9', 'KCNJ8'],
    'SMC': ['MYH11', 'TAGLN', 'ACTA2'],
    'Meso': ['MSLN', 'WT1', 'BNC1'],
    'Neuro': ['PLP1', 'NRXN1', 'NRXN3','PRPH', 'NEFL', 'NEFM', 'NEFH', 'STMN2', 'SST', 'SLC10A4', 'SLC18A3', 'TH', 'NPY'],
    'Adipo': ['GPAM', 'FASN', 'LEP'],
    'Myelo': ['CD14', 'C1QA', 'CD68','LYVE1','TIMD4'],
    'Lymph': ['CD8A', 'IL7R', 'CD40LG'],
    'B': ['CD19','SDC1','IGHG1','IGHA1'],
    'Mast': ['KIT', 'CPA3'],
}

for key in markers_others.keys():
    markers_others[key]=[x for x in markers_others[key] if x in adata_sub.var_names]

sc.tl.dendrogram(adata_sub, groupby=f'leiden_{str(sub_resolution_sel)}')
sc.pl.dotplot(adata_sub,
              markers_others, 
              groupby=f'leiden_{str(sub_resolution_sel)}',
              dendrogram=True,
              standard_scale="var",
              color_map="Reds",
              swap_axes=False,
             )
```

Example 51 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: CM, FB, EC, etc.
```

Example 52 (python):
```python
# DEG analysis
print(adata_sub.X.data[:5])
adata_sub.uns['log1p']['base']=None
sc.tl.rank_genes_groups(adata_sub,groupby=f'leiden_{str(sub_resolution_sel)}')
sc.tl.dendrogram(adata_sub, groupby=f'leiden_{str(sub_resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(adata_sub, dendrogram=True, standard_scale='var', return_fig=True)
dp.add_totals().show()
```

Example 53 (unknown):
```unknown
[2.9061403 2.2662368 2.2662368 2.2662368 2.2662368]
```

Example 54 (python):
```python
# DEG analysis without other cell type markers
# with removing mid_grain markers
# remove mid_grain markers
mask = [x not in list(midgrain_markers['names']) for x in adata_sub.var_names]
bdata = adata_sub[:,mask]

print(bdata.X.data[:5])
bdata.uns['log1p']['base']=None
sc.tl.rank_genes_groups(bdata,groupby=f'leiden_{str(sub_resolution_sel)}')
sc.tl.dendrogram(bdata, groupby=f'leiden_{str(sub_resolution_sel)}')
dp=sc.pl.rank_genes_groups_dotplot(bdata, dendrogram=True, standard_scale='var', return_fig=True)
dp.add_totals().show()
```

Example 55 (unknown):
```unknown
[2.9061403 2.2662368 2.2662368 2.2662368 2.2662368]
```

Example 56 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/scanpy/tools/_rank_genes_groups.py:669: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  adata.uns[key_added] = {}
```

Example 57 (python):
```python
myeloid_markers={
    'Mono':['CCR2','FCN1','VCAN','MPO'],
    'MP':['C1QA','C1QB','CD68','MRC1','CD163','MERTK'],
    'CX3CR1+':['CX3CR1','C3','TREM2','P2RY12'],
    'LYVE1+':['LYVE1','FOLR2'],
     'pDC':['IRF8','IRF4','IL3RA'],
     'DC_mature':['FLT3', 'CCR7','LAMP3', 'IL15', 'CD83', 'LY75', ],
    'Megakaryocyte':['ITGA2B','MPL','ITGB3',],
    'Mast': ['KIT', 'CPA3','HDC','GATA2'],
    'Neutro':['MYB', 'MPO', 'CEACAM8','CXCR1','CXCR2'],
    'Cycling':['MKI67','TOP2A'],
}
for k,g in myeloid_markers.items():
    myeloid_markers[k] = [x for x in g if x in adata_sub.var_names]

sc.tl.dendrogram(adata_sub,groupby=f'leiden_{str(sub_resolution_sel)}')
sc.pl.dotplot(adata_sub,
              var_names=myeloid_markers,
              groupby=f'leiden_{str(sub_resolution_sel)}',
              standard_scale='var',
              dendrogram=True
             )
```

Example 58 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: 0, 1, 2, etc.
var_group_labels: Mono, MP, CX3CR1+, etc.
```

Example 59 (python):
```python
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color=f'leiden_{str(sub_resolution_sel)}',legend_loc='on data',
                wspace=0.3)
```

Example 60 (python):
```python
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color=['celltypist_midmod2fine', 'conf_score_midmod2fine'],
                wspace=0.7)
```

Example 61 (python):
```python
# annotate
adata_sub.obs['fine_grain_manual'] = adata_sub.obs[f'leiden_{str(sub_resolution_sel)}'].copy()
adata_sub.obs.replace({'fine_grain_manual':{
    '0':'doublets', 
    '1':'InnateLymphoidCells',
    '2':'TCellsCD8pos',
    '3':'doublets',
    '4':'BCells',
    '5':'TCellsCD4pos',
    '6':'doublets',
    '7':'doublets',
}},inplace=True)
adata_sub.obs['fine_grain_manual'].value_counts()
```

Example 62 (python):
```python
/tmp/ipykernel_78172/3271777684.py:3: FutureWarning: The behavior of Series.replace (and DataFrame.replace) with CategoricalDtype is deprecated. In a future version, replace will only be used for cases that preserve the categories. To change the categories, use ser.cat.rename_categories instead.
  adata_sub.obs.replace({'fine_grain_manual':{
```

Example 63 (unknown):
```unknown
fine_grain_manual
doublets               362
InnateLymphoidCells    153
TCellsCD8pos           140
BCells                 104
TCellsCD4pos            41
Name: count, dtype: int64
```

Example 64 (python):
```python
sc.pl.embedding(adata_sub, basis=f"X_umap_{latent_space}",
                color='fine_grain_manual',
                wspace=0.3)
```

Example 65 (python):
```python
sc.tl.dendrogram(adata_sub,groupby='fine_grain_manual')
sc.pl.dotplot(adata_sub,
              var_names=markers,
              groupby='fine_grain_manual',
              standard_scale='var',
              dendrogram=True
             )
```

Example 66 (unknown):
```unknown
WARNING: Groups are not reordered because the `groupby` categories and the `var_group_labels` are different.
categories: doublets, InnateLymphoidCells, TCellsCD8pos, etc.
var_group_labels: T cell, naive, CD4T_reg, etc.
```

Example 67 (python):
```python
adata.obs['fine_grain_manual'] = adata.obs['tmp_annotation'].astype('str').copy()
adata.obs.loc[adata_sub.obs_names,'fine_grain_manual'] = adata_sub.obs['fine_grain_manual']
```

Example 68 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color='fine_grain_manual'
                )
```

Example 69 (python):
```python
pd.crosstab(adata.obs['fine_grain_manual'],adata.obs['tissue_block_id'])
```

Example 70 (python):
```python
for tissue_id in ['C194-HEA-0-FFPE-1', 'Hst45-HEA-0-FFPE-1']:
    slide = adata[adata.obs['tissue_block_id']==tissue_id]
    sc.pl.spatial(slide,color=['fine_grain_manual'],# groups='SinoatrialNodePacemakerCells',
                  spot_size=60,vmax='p100')
```

Example 71 (python):
```python
/tmp/ipykernel_78172/4023824168.py:3: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],# groups='SinoatrialNodePacemakerCells',
```

Example 72 (python):
```python
/tmp/ipykernel_78172/4023824168.py:3: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],# groups='SinoatrialNodePacemakerCells',
```

Example 73 (python):
```python
for ct in set(adata.obs['fine_grain_manual']):
    for tissue_id in ['C194-HEA-0-FFPE-1', 'Hst45-HEA-0-FFPE-1']:
        slide = adata[adata.obs['tissue_block_id']==tissue_id]
        sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
                      spot_size=80,vmax='p100')
```

Example 74 (python):
```python
/tmp/ipykernel_78172/1166615315.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 75 (python):
```python
/tmp/ipykernel_78172/1166615315.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 76 (python):
```python
/tmp/ipykernel_78172/1166615315.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 77 (python):
```python
/tmp/ipykernel_78172/1166615315.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 78 (python):
```python
/tmp/ipykernel_78172/1166615315.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 79 (python):
```python
/tmp/ipykernel_78172/1166615315.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 80 (python):
```python
/tmp/ipykernel_78172/1166615315.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 81 (python):
```python
/tmp/ipykernel_78172/1166615315.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 82 (python):
```python
/tmp/ipykernel_78172/1166615315.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 83 (python):
```python
/tmp/ipykernel_78172/1166615315.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 84 (python):
```python
/tmp/ipykernel_78172/1166615315.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 85 (python):
```python
/tmp/ipykernel_78172/1166615315.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 86 (python):
```python
/tmp/ipykernel_78172/1166615315.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 87 (python):
```python
/tmp/ipykernel_78172/1166615315.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 88 (python):
```python
/tmp/ipykernel_78172/1166615315.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 89 (python):
```python
/tmp/ipykernel_78172/1166615315.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 90 (python):
```python
/tmp/ipykernel_78172/1166615315.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 91 (python):
```python
/tmp/ipykernel_78172/1166615315.py:4: FutureWarning: Use `squidpy.pl.spatial_scatter` instead.
  sc.pl.spatial(slide,color=['fine_grain_manual'],groups=ct,
```

Example 92 (python):
```python
# save
adata.write(f'{adata_dir}/{sample_id}_5K_{celltype}_lognorm.h5ad')
print(f'{adata_dir}/{sample_id}_5K_{celltype}_lognorm.h5ad')
```

Example 93 (unknown):
```unknown
/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium/subsets/C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat_5K_LymphoidCells_lognorm.h5ad
```

---

## Functions¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig4_Cardiomyocytes/AtrialCardiomyocytes/Visium_mapping/c2l_pt3_plot_immature.html

**Contents:**
- Functions¶
- Read in metadata¶
- Data, parameters¶

2055 rows × 6 columns

2055 rows × 10 columns

2055 rows × 10 columns

**Examples:**

Example 1 (python):
```python
import sys
import scanpy as sc
import anndata
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt 
import matplotlib as mpl

import cell2location
import scvi

from matplotlib import rcParams
rcParams['pdf.fonttype'] = 42 # enables correct plotting of text
import seaborn as sns
```

Example 2 (python):
```python
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm
Global seed set to 0
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/umap/distances.py:1063: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/umap/distances.py:1071: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/umap/distances.py:1086: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/umap/umap_.py:660: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
```

Example 3 (python):
```python
import gc
import torch
```

Example 4 (python):
```python
def read_and_qc(sample_name, path):
    r""" This function reads the data for one 10X spatial experiment into the anndata object.
    It also calculates QC metrics. Modify this function if required by your workflow.

    :param sample_name: Name of the sample
    :param path: path to data
    """

    adata = sc.read_visium(path + str(sample_name),
                           count_file='filtered_feature_bc_matrix.h5', load_images=True)
    adata.var_names_make_unique(join='-')
    
    adata.obs['sample'] = sample_name
    
    # rename genes to ENSEMBL
    adata.var['SYMBOL'] = adata.var_names
    adata.var_names = adata.var['gene_ids']
    adata.var_names.name = None

    # Calculate QC metrics
    sc.pp.calculate_qc_metrics(adata, inplace=True)
    adata.var['mt'] = [gene.startswith('MT-') for gene in adata.var['SYMBOL']]
    adata.obs['mt_frac'] = adata[:, adata.var['mt'].tolist()].X.sum(1).A.squeeze()/adata.obs['total_counts']

    # add sample name to obs names
    adata.obs["sample"] = [str(i) for i in adata.obs['sample']]
    adata.obs_names = adata.obs["sample"] \
                          + '_' + adata.obs_names
    adata.obs.index.name = 'spot_id'

    return adata

def select_slide(adata, s, s_col='sample'):
    r""" This function selects the data for one slide from the spatial anndata object.

    :param adata: Anndata object with multiple spatial experiments
    :param s: name of selected experiment
    :param s_col: column in adata.obs listing experiment name for each location
    """

    slide = adata[adata.obs[s_col].isin([s]), :]
    s_keys = list(slide.uns['spatial'].keys())
    s_spatial = np.array(s_keys)[[s in k for k in s_keys]][0]

    slide.uns['spatial'] = {s_spatial: slide.uns['spatial'][s_spatial]}

    return slide
```

Example 5 (python):
```python
meta=pd.read_csv('/nfs/team205/heart/anndata_objects/Foetal/metadata/HeartTeamSamples_Mappeddata_20230208_for-cell2loc.csv',
                index_col=0)
meta
```

Example 6 (python):
```python
pd.crosstab(meta['donor'],meta['cell2loc-map_group'])
```

Example 7 (unknown):
```unknown
# select slides
# Healthy, OCT, and without DA
meta = meta[meta['cell2loc-map_group'].isin(['4PCW','5PCW','7PCW','13PCW','20PCW'])]

sampleIDs = list(meta['sangerID'])
len(sampleIDs)
```

Example 8 (unknown):
```unknown
run = "20Mar2024_fetal_OCT"
```

Example 9 (python):
```python
import os
results_folder = os.path.join(os.getcwd(),"analysis/Visium/cell2location_results")

ref_run_name = f'{results_folder}/reference_signatures/{run}'
run_name = f'{results_folder}/cell2location_map/{run}'
```

Example 10 (unknown):
```unknown
visium_path='/nfs/team205/heart/mapped/spaceranger110/'

celltype_key='fine_grain_modified'

vis_subset_key='cell2loc-map_group'
```

Example 11 (python):
```python
%%time
subset_values = os.listdir(os.path.join(results_folder, "cell2location_map", run))
print(f"Found these subsets {subset_values}")

for value in subset_values:
    print(f"###### {value} ######")
    adata_sub = sc.read(f"{run_name}/{value}/sp.h5ad")
    print("visium read in")
    adata_sub.obs[adata_sub.uns['mod']['factor_names']] = adata_sub.obsm['q05_cell_abundance_w_sf']
    IDs = meta[meta[vis_subset_key] == value].sangerID.tolist()
    print(f"{value}: {IDs}")

    for sample in IDs:
        try:
            slide = select_slide(adata_sub, sample)
        except IndexError as e:
            print(f"Sample {sample} not found in adata: {e}")
            continue  # Skip this sample if not found

        slide.obs[slide.uns['mod']['factor_names']] = slide.obsm['q05_cell_abundance_w_sf']
        list_of_cell_types = slide.uns['mod']['factor_names'].tolist()
        print("plotting HE image...")
        sc.pl.spatial(slide, color=None, size=0, img_key='hires', title=f"HE:{sample}", 
                      save=f"{run}__{value}__{sample}__HE.png", show=False)

        # Plotting abundances for all relevant cell states
        if list_of_cell_types:
            print("plotting abundances for all cell states...")
            sc.pl.spatial(slide, color=list_of_cell_types, cmap="RdPu", ncols=10, size=1.3, 
                          img_key='hires', vmin=0, vmax='p99.2', alpha_img=0.3, 
                          save=f"{run}__{value}__{sample}__all_cell_states_abundances.png", show=False)
```

Example 12 (unknown):
```unknown
Found these subsets ['5PCW', '20PCW', '13PCW', '4PCW', '7PCW']
###### 5PCW ######
visium read in
5PCW: ['HCAHeartST11605167', 'HCAHeartST11605168']
```

Example 13 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
```

Example 14 (unknown):
```unknown
plotting HE image...
WARNING: saving figure to file figures/show20Mar2024_fetal_OCT__5PCW__HCAHeartST11605167__HE.png
plotting abundances for all cell states...
WARNING: saving figure to file figures/show20Mar2024_fetal_OCT__5PCW__HCAHeartST11605167__all_cell_states_abundances.png
```

Example 15 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
```

Example 16 (unknown):
```unknown
plotting HE image...
WARNING: saving figure to file figures/show20Mar2024_fetal_OCT__5PCW__HCAHeartST11605168__HE.png
plotting abundances for all cell states...
WARNING: saving figure to file figures/show20Mar2024_fetal_OCT__5PCW__HCAHeartST11605168__all_cell_states_abundances.png
###### 20PCW ######
visium read in
20PCW: ['HCAHeartST13162329', 'HCAHeartST13162330', 'HCAHeartST13162331']
```

Example 17 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
```

Example 18 (unknown):
```unknown
plotting HE image...
WARNING: saving figure to file figures/show20Mar2024_fetal_OCT__20PCW__HCAHeartST13162329__HE.png
plotting abundances for all cell states...
WARNING: saving figure to file figures/show20Mar2024_fetal_OCT__20PCW__HCAHeartST13162329__all_cell_states_abundances.png
```

Example 19 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
```

Example 20 (unknown):
```unknown
plotting HE image...
WARNING: saving figure to file figures/show20Mar2024_fetal_OCT__20PCW__HCAHeartST13162330__HE.png
plotting abundances for all cell states...
WARNING: saving figure to file figures/show20Mar2024_fetal_OCT__20PCW__HCAHeartST13162330__all_cell_states_abundances.png
```

Example 21 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
```

Example 22 (unknown):
```unknown
plotting HE image...
WARNING: saving figure to file figures/show20Mar2024_fetal_OCT__20PCW__HCAHeartST13162331__HE.png
plotting abundances for all cell states...
WARNING: saving figure to file figures/show20Mar2024_fetal_OCT__20PCW__HCAHeartST13162331__all_cell_states_abundances.png
###### 13PCW ######
visium read in
13PCW: ['HCAHeartST11605165', 'HCAHeartST11605166']
```

Example 23 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
```

Example 24 (unknown):
```unknown
plotting HE image...
WARNING: saving figure to file figures/show20Mar2024_fetal_OCT__13PCW__HCAHeartST11605165__HE.png
plotting abundances for all cell states...
WARNING: saving figure to file figures/show20Mar2024_fetal_OCT__13PCW__HCAHeartST11605165__all_cell_states_abundances.png
```

Example 25 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
```

Example 26 (unknown):
```unknown
plotting HE image...
WARNING: saving figure to file figures/show20Mar2024_fetal_OCT__13PCW__HCAHeartST11605166__HE.png
plotting abundances for all cell states...
WARNING: saving figure to file figures/show20Mar2024_fetal_OCT__13PCW__HCAHeartST11605166__all_cell_states_abundances.png
###### 4PCW ######
visium read in
4PCW: ['HCAHeartST13162332']
```

Example 27 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
```

Example 28 (unknown):
```unknown
plotting HE image...
WARNING: saving figure to file figures/show20Mar2024_fetal_OCT__4PCW__HCAHeartST13162332__HE.png
plotting abundances for all cell states...
WARNING: saving figure to file figures/show20Mar2024_fetal_OCT__4PCW__HCAHeartST13162332__all_cell_states_abundances.png
###### 7PCW ######
visium read in
7PCW: ['HCAHeartST10298114', 'HCAHeartST10298115', 'HCAHeartST10298116', 'HCAHeartST10298117', 'HCAHeartST10298120', 'HCAHeartST10317184', 'HCAHeartST10317185', 'HCAHeartST10317186', 'HCAHeartST10317189', 'HCAHeartST10317190']
```

Example 29 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
```

Example 30 (unknown):
```unknown
plotting HE image...
WARNING: saving figure to file figures/show20Mar2024_fetal_OCT__7PCW__HCAHeartST10298114__HE.png
plotting abundances for all cell states...
WARNING: saving figure to file figures/show20Mar2024_fetal_OCT__7PCW__HCAHeartST10298114__all_cell_states_abundances.png
```

Example 31 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
```

Example 32 (unknown):
```unknown
plotting HE image...
WARNING: saving figure to file figures/show20Mar2024_fetal_OCT__7PCW__HCAHeartST10298115__HE.png
plotting abundances for all cell states...
WARNING: saving figure to file figures/show20Mar2024_fetal_OCT__7PCW__HCAHeartST10298115__all_cell_states_abundances.png
```

Example 33 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
```

Example 34 (unknown):
```unknown
plotting HE image...
WARNING: saving figure to file figures/show20Mar2024_fetal_OCT__7PCW__HCAHeartST10298116__HE.png
plotting abundances for all cell states...
WARNING: saving figure to file figures/show20Mar2024_fetal_OCT__7PCW__HCAHeartST10298116__all_cell_states_abundances.png
```

Example 35 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
```

Example 36 (unknown):
```unknown
plotting HE image...
WARNING: saving figure to file figures/show20Mar2024_fetal_OCT__7PCW__HCAHeartST10298117__HE.png
plotting abundances for all cell states...
WARNING: saving figure to file figures/show20Mar2024_fetal_OCT__7PCW__HCAHeartST10298117__all_cell_states_abundances.png
```

Example 37 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
```

Example 38 (unknown):
```unknown
plotting HE image...
WARNING: saving figure to file figures/show20Mar2024_fetal_OCT__7PCW__HCAHeartST10298120__HE.png
plotting abundances for all cell states...
WARNING: saving figure to file figures/show20Mar2024_fetal_OCT__7PCW__HCAHeartST10298120__all_cell_states_abundances.png
```

Example 39 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
```

Example 40 (unknown):
```unknown
plotting HE image...
WARNING: saving figure to file figures/show20Mar2024_fetal_OCT__7PCW__HCAHeartST10317184__HE.png
plotting abundances for all cell states...
WARNING: saving figure to file figures/show20Mar2024_fetal_OCT__7PCW__HCAHeartST10317184__all_cell_states_abundances.png
```

Example 41 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
```

Example 42 (unknown):
```unknown
plotting HE image...
WARNING: saving figure to file figures/show20Mar2024_fetal_OCT__7PCW__HCAHeartST10317185__HE.png
plotting abundances for all cell states...
WARNING: saving figure to file figures/show20Mar2024_fetal_OCT__7PCW__HCAHeartST10317185__all_cell_states_abundances.png
```

Example 43 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
```

Example 44 (unknown):
```unknown
plotting HE image...
WARNING: saving figure to file figures/show20Mar2024_fetal_OCT__7PCW__HCAHeartST10317186__HE.png
plotting abundances for all cell states...
WARNING: saving figure to file figures/show20Mar2024_fetal_OCT__7PCW__HCAHeartST10317186__all_cell_states_abundances.png
```

Example 45 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
```

Example 46 (unknown):
```unknown
plotting HE image...
WARNING: saving figure to file figures/show20Mar2024_fetal_OCT__7PCW__HCAHeartST10317189__HE.png
plotting abundances for all cell states...
WARNING: saving figure to file figures/show20Mar2024_fetal_OCT__7PCW__HCAHeartST10317189__all_cell_states_abundances.png
```

Example 47 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
```

Example 48 (unknown):
```unknown
plotting HE image...
WARNING: saving figure to file figures/show20Mar2024_fetal_OCT__7PCW__HCAHeartST10317190__HE.png
plotting abundances for all cell states...
WARNING: saving figure to file figures/show20Mar2024_fetal_OCT__7PCW__HCAHeartST10317190__all_cell_states_abundances.png
CPU times: user 9min 45s, sys: 2min 46s, total: 12min 31s
Wall time: 12min 40s
```

Example 49 (python):
```python
%%time

adata_dict={}

subset_values = os.listdir(os.path.join(results_folder, "cell2location_map", run))
print(f"Found these subsets {subset_values}")

for value in subset_values:
    print(f"###### {value} ######")
    adata_sub = sc.read(f"{run_name}/{value}/sp.h5ad")
    print("visium read in")
    adata_sub.obs[adata_sub.uns['mod']['factor_names']] = adata_sub.obsm['q05_cell_abundance_w_sf']
    IDs = meta[meta[vis_subset_key] == value].sangerID.tolist()
    print(f"{value}: {IDs}")

    for sample in IDs:
        try:
            slide = select_slide(adata_sub, sample)
        except IndexError as e:
            print(f"Sample {sample} not found in adata: {e}")
            continue  # Skip this sample if not found

        slide.obs[slide.uns['mod']['factor_names']] = slide.obsm['q05_cell_abundance_w_sf']
        adata_dict[sample]=slide
```

Example 50 (unknown):
```unknown
Found these subsets ['5PCW', '20PCW', '13PCW', '4PCW', '7PCW']
###### 5PCW ######
visium read in
5PCW: ['HCAHeartST11605167', 'HCAHeartST11605168']
```

Example 51 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
```

Example 52 (unknown):
```unknown
###### 20PCW ######
visium read in
20PCW: ['HCAHeartST13162329', 'HCAHeartST13162330', 'HCAHeartST13162331']
```

Example 53 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
```

Example 54 (unknown):
```unknown
###### 13PCW ######
visium read in
13PCW: ['HCAHeartST11605165', 'HCAHeartST11605166']
```

Example 55 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
```

Example 56 (unknown):
```unknown
###### 4PCW ######
visium read in
4PCW: ['HCAHeartST13162332']
```

Example 57 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
```

Example 58 (unknown):
```unknown
###### 7PCW ######
visium read in
7PCW: ['HCAHeartST10298114', 'HCAHeartST10298115', 'HCAHeartST10298116', 'HCAHeartST10298117', 'HCAHeartST10298120', 'HCAHeartST10317184', 'HCAHeartST10317185', 'HCAHeartST10317186', 'HCAHeartST10317189', 'HCAHeartST10317190']
```

Example 59 (unknown):
```unknown
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
/lustre/scratch126/cellgen/team205/jc48/miniconda3/envs/omniGPU_env/lib/python3.11/site-packages/anndata/compat/_overloaded_dict.py:106: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  self.data[key] = value
```

Example 60 (unknown):
```unknown
CPU times: user 3.58 s, sys: 8.52 s, total: 12.1 s
Wall time: 18.3 s
```

Example 61 (python):
```python
adata_dict.keys()
```

Example 62 (unknown):
```unknown
dict_keys(['HCAHeartST11605167', 'HCAHeartST11605168', 'HCAHeartST13162329', 'HCAHeartST13162330', 'HCAHeartST13162331', 'HCAHeartST11605165', 'HCAHeartST11605166', 'HCAHeartST13162332', 'HCAHeartST10298114', 'HCAHeartST10298115', 'HCAHeartST10298116', 'HCAHeartST10298117', 'HCAHeartST10298120', 'HCAHeartST10317184', 'HCAHeartST10317185', 'HCAHeartST10317186', 'HCAHeartST10317189', 'HCAHeartST10317190'])
```

Example 63 (python):
```python
from tqdm import tqdm
atrial_data_dict={}
for sample,tmp_adata in tqdm(adata_dict.items()):
    print(f"sample: {sample},  age is {tmp_adata.obs.age.unique().tolist()[0]}")
    tmp_df=tmp_adata[tmp_adata.obs.JC_edit=="Myocardium - Atrial"].obs[['AtrialCardiomyocytesLeft_immature', 'AtrialCardiomyocytesLeft', 'AtrialCardiomyocytesRight','AtrialCardiomyocytesRight_immature']]

    if tmp_df.shape[0]>0:
        tmp_df["sample"]=sample
        tmp_df["age"]=tmp_adata.obs.age.unique().tolist()[0]
        atrial_data_dict[sample]=tmp_df
    else:
        print(f"no atrial myocardium spots annotated for sample: {sample}")
```

Example 64 (python):
```python
0%|                                                                                                                                                                                                                    | 0/18 [00:00<?, ?it/s]/tmp/ipykernel_167807/3397130741.py:8: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  tmp_df["sample"]=sample
/tmp/ipykernel_167807/3397130741.py:9: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  tmp_df["age"]=tmp_adata.obs.age.unique().tolist()[0]
/tmp/ipykernel_167807/3397130741.py:8: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  tmp_df["sample"]=sample
/tmp/ipykernel_167807/3397130741.py:9: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  tmp_df["age"]=tmp_adata.obs.age.unique().tolist()[0]
/tmp/ipykernel_167807/3397130741.py:8: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  tmp_df["sample"]=sample
/tmp/ipykernel_167807/3397130741.py:9: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  tmp_df["age"]=tmp_adata.obs.age.unique().tolist()[0]
/tmp/ipykernel_167807/3397130741.py:8: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  tmp_df["sample"]=sample
/tmp/ipykernel_167807/3397130741.py:9: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  tmp_df["age"]=tmp_adata.obs.age.unique().tolist()[0]
/tmp/ipykernel_167807/3397130741.py:8: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  tmp_df["sample"]=sample
/tmp/ipykernel_167807/3397130741.py:9: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  tmp_df["age"]=tmp_adata.obs.age.unique().tolist()[0]
/tmp/ipykernel_167807/3397130741.py:8: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  tmp_df["sample"]=sample
/tmp/ipykernel_167807/3397130741.py:9: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  tmp_df["age"]=tmp_adata.obs.age.unique().tolist()[0]
/tmp/ipykernel_167807/3397130741.py:8: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  tmp_df["sample"]=sample
/tmp/ipykernel_167807/3397130741.py:9: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  tmp_df["age"]=tmp_adata.obs.age.unique().tolist()[0]
/tmp/ipykernel_167807/3397130741.py:8: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  tmp_df["sample"]=sample
/tmp/ipykernel_167807/3397130741.py:9: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  tmp_df["age"]=tmp_adata.obs.age.unique().tolist()[0]
 67%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▋                                                                   | 12/18 [00:00<00:00, 109.51it/s]/tmp/ipykernel_167807/3397130741.py:8: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  tmp_df["sample"]=sample
/tmp/ipykernel_167807/3397130741.py:9: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  tmp_df["age"]=tmp_adata.obs.age.unique().tolist()[0]
/tmp/ipykernel_167807/3397130741.py:8: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  tmp_df["sample"]=sample
/tmp/ipykernel_167807/3397130741.py:9: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  tmp_df["age"]=tmp_adata.obs.age.unique().tolist()[0]
/tmp/ipykernel_167807/3397130741.py:8: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  tmp_df["sample"]=sample
/tmp/ipykernel_167807/3397130741.py:9: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  tmp_df["age"]=tmp_adata.obs.age.unique().tolist()[0]
/tmp/ipykernel_167807/3397130741.py:8: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  tmp_df["sample"]=sample
/tmp/ipykernel_167807/3397130741.py:9: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  tmp_df["age"]=tmp_adata.obs.age.unique().tolist()[0]
/tmp/ipykernel_167807/3397130741.py:8: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  tmp_df["sample"]=sample
/tmp/ipykernel_167807/3397130741.py:9: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  tmp_df["age"]=tmp_adata.obs.age.unique().tolist()[0]
/tmp/ipykernel_167807/3397130741.py:8: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  tmp_df["sample"]=sample
/tmp/ipykernel_167807/3397130741.py:9: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  tmp_df["age"]=tmp_adata.obs.age.unique().tolist()[0]
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 18/18 [00:00<00:00, 109.20it/s]
```

Example 65 (unknown):
```unknown
sample: HCAHeartST11605167,  age is GA-7w4d
sample: HCAHeartST11605168,  age is GA-7w4d
sample: HCAHeartST13162329,  age is PCW-20w
no atrial myocardium spots annotated for sample: HCAHeartST13162329
sample: HCAHeartST13162330,  age is PCW-20w
no atrial myocardium spots annotated for sample: HCAHeartST13162330
sample: HCAHeartST13162331,  age is PCW-20w
no atrial myocardium spots annotated for sample: HCAHeartST13162331
sample: HCAHeartST11605165,  age is PCW-13w
sample: HCAHeartST11605166,  age is PCW-13w
sample: HCAHeartST13162332,  age is GA-6w3d
no atrial myocardium spots annotated for sample: HCAHeartST13162332
sample: HCAHeartST10298114,  age is GA-9w4d
sample: HCAHeartST10298115,  age is GA-9w4d
sample: HCAHeartST10298116,  age is GA-9w4d
sample: HCAHeartST10298117,  age is GA-9w4d
sample: HCAHeartST10298120,  age is GA-9w4d
sample: HCAHeartST10317184,  age is GA-9w4d
sample: HCAHeartST10317185,  age is GA-9w4d
sample: HCAHeartST10317186,  age is GA-9w4d
sample: HCAHeartST10317189,  age is GA-9w4d
sample: HCAHeartST10317190,  age is GA-9w4d
```

Example 66 (python):
```python
data=pd.concat(list(atrial_data_dict.values()))
data
```

Example 67 (unknown):
```unknown
data.to_csv("AtrialProportionsOCT.csv")
```

Example 68 (unknown):
```unknown
# Assuming 'data' is your pandas DataFrame
# Calculate the total for each row
total = data['AtrialCardiomyocytesLeft_immature'] + data['AtrialCardiomyocytesLeft'] + data['AtrialCardiomyocytesRight'] + data['AtrialCardiomyocytesRight_immature']

# Calculate the proportions for each column
data['AtrialCardiomyocytesRight_immature_prop'] = data['AtrialCardiomyocytesRight_immature'] / total
data['AtrialCardiomyocytesLeft_immature_prop'] = data['AtrialCardiomyocytesLeft_immature'] / total
data['AtrialCardiomyocytesRight_prop'] = data['AtrialCardiomyocytesRight'] / total
data['AtrialCardiomyocytesLeft_prop'] = data['AtrialCardiomyocytesLeft'] / total

# The new DataFrame 'data' now includes the proportion columns
data
```

Example 69 (python):
```python
import seaborn as sns
import matplotlib.pyplot as plt

# Assuming 'data' is your DataFrame and it already includes an 'age' column
df_long = data.melt(id_vars='age', var_name='Cell Type', value_name='Proportion', 
                    value_vars=['AtrialCardiomyocytesLeft_immature_prop', 'AtrialCardiomyocytesLeft_prop', 
                                'AtrialCardiomyocytesRight_prop', 'AtrialCardiomyocytesRight_immature_prop'])

# Create the box plot with 'hue' for age categories
plt.figure(figsize=(12, 8))
sns.boxplot(x='age', y='Proportion', hue='Cell Type', data=df_long)
plt.title('Distribution of Cell Type Proportions Across Age Categories')
plt.xticks(rotation=45)
plt.xlabel('Cell Type')
plt.ylabel('Proportion')
plt.legend(title='Age Category', bbox_to_anchor=(1.05, 1), loc='upper left')
plt.tight_layout()
plt.show()
```

Example 70 (python):
```python
# Assuming 'data' is your pandas DataFrame
# Calculate the total for Left and Right cells separately
total_left = data['AtrialCardiomyocytesLeft_immature'] + data['AtrialCardiomyocytesLeft']
total_right = data['AtrialCardiomyocytesRight_immature'] + data['AtrialCardiomyocytesRight']

# Calculate the proportions for each cell type relative to their side
data['AtrialCardiomyocytesLeft_immature_prop'] = data['AtrialCardiomyocytesLeft_immature'] / total_left
data['AtrialCardiomyocytesLeft_prop'] = data['AtrialCardiomyocytesLeft'] / total_left

data['AtrialCardiomyocytesRight_immature_prop'] = data['AtrialCardiomyocytesRight_immature'] / total_right
data['AtrialCardiomyocytesRight_prop'] = data['AtrialCardiomyocytesRight'] / total_right

# Check the new columns in the DataFrame
data[['AtrialCardiomyocytesLeft_immature_prop', 'AtrialCardiomyocytesLeft_prop', 'AtrialCardiomyocytesRight_immature_prop', 'AtrialCardiomyocytesRight_prop']]

import numpy as np

data['AtrialCardiomyocytesLeft_immature_prop'] = np.where(total_left == 0, 0, data['AtrialCardiomyocytesLeft_immature'] / total_left)
data['AtrialCardiomyocytesLeft_prop'] = np.where(total_left == 0, 0, data['AtrialCardiomyocytesLeft'] / total_left)

data['AtrialCardiomyocytesRight_immature_prop'] = np.where(total_right == 0, 0, data['AtrialCardiomyocytesRight_immature'] / total_right)
data['AtrialCardiomyocytesRight_prop'] = np.where(total_right == 0, 0, data['AtrialCardiomyocytesRight'] / total_right)
data
```

Example 71 (python):
```python
import seaborn as sns
import matplotlib.pyplot as plt

# Assuming 'data' is your DataFrame and it already includes an 'age' column
df_long = data.melt(id_vars='age', var_name='Cell Type', value_name='Proportion', 
                    value_vars=['AtrialCardiomyocytesLeft_immature_prop', 'AtrialCardiomyocytesLeft_prop', 
                                'AtrialCardiomyocytesRight_prop', 'AtrialCardiomyocytesRight_immature_prop'])

# Create the box plot with 'hue' for age categories
plt.figure(figsize=(12, 8))
sns.boxplot(x='age', y='Proportion', hue='Cell Type', data=df_long)
plt.title('Distribution of Cell Type Proportions Across Age Categories')
plt.xticks(rotation=45)
plt.xlabel('Cell Type')
plt.ylabel('Proportion')
plt.legend(title='Age Category', bbox_to_anchor=(1.05, 1), loc='upper left')
plt.tight_layout()
plt.show()
```

---

## Read in¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig4_Cardiomyocytes/VentricularCardiomyocytes/multiVI_scFate/no4_scFate_vCM_Left.html

**Contents:**
- Read in¶
- scFates - Tree analysis¶
- Preprocessing pipeline from Palantir¶
  - Create projection dataframe¶
  - Run Palantir to obtain multiscale diffusion space¶
  - Generate embedding from the multiscale diffusion space¶
  - Plotting¶
- Tree learning with SimplePPT (or ElPiGraph)¶
  - Projecting results onto ForceAtlas2 embedding¶
  - Slecting roots and computing pseudotime¶
- Representing the trajectory and tree (on top of existing embedding)¶
- Recover full genes¶
- Save¶
  - make age continuous¶
  - save¶
- Save object separately, Compact, Trab, Gene, and ATAC¶
- Compact¶
- Trabeculated¶
- Other plots¶

ref: https://scfates.readthedocs.io/en/latest/Tree_Analysis_Bone_marrow_fates.html

16451 rows × 20 columns

**Examples:**

Example 1 (python):
```python
import scanpy as sc
import pandas as pd
import numpy as np
import anndata
import scipy

import matplotlib.pyplot as plt
import seaborn as sns
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
-----
anndata             0.9.2
matplotlib          3.5.2
numpy               1.22.3
pandas              1.5.3
scanpy              1.8.2
scipy               1.8.1
seaborn             0.12.2
session_info        1.0.0
-----
```

Example 4 (python):
```python
PIL                 10.0.0
asttokens           NA
backcall            0.2.0
beta_ufunc          NA
binom_ufunc         NA
cffi                1.15.1
colorama            0.4.6
comm                0.1.4
cpuinfo             NA
cycler              0.10.0
cython_runtime      NA
dateutil            2.8.2
debugpy             1.6.7.post1
decorator           5.1.1
executing           1.2.0
h5py                3.9.0
hypergeom_ufunc     NA
igraph              0.10.6
importlib_metadata  NA
ipykernel           6.25.1
jedi                0.19.0
joblib              1.3.2
kiwisolver          1.4.5
leidenalg           0.10.1
llvmlite            0.40.1
mpl_toolkits        NA
natsort             8.4.0
nbinom_ufunc        NA
numba               0.57.1
numexpr             2.8.5
packaging           23.1
parso               0.8.3
patsy               0.5.3
pexpect             4.8.0
pickleshare         0.7.5
pkg_resources       NA
platformdirs        3.10.0
plotly              5.16.1
prompt_toolkit      3.0.39
psutil              5.9.5
ptyprocess          0.7.0
pure_eval           0.2.2
pydev_ipython       NA
pydevconsole        NA
pydevd              2.9.5
pydevd_file_utils   NA
pydevd_plugins      NA
pydevd_tracing      NA
pygments            2.16.1
pyparsing           3.0.9
pytz                2023.3
sinfo               0.3.4
six                 1.16.0
sklearn             1.3.0
sphinxcontrib       NA
stack_data          0.6.2
statsmodels         0.14.0
tables              3.8.0
texttable           1.6.7
threadpoolctl       3.2.0
tornado             6.3.3
traitlets           5.9.0
typing_extensions   NA
wcwidth             0.2.6
zipp                NA
zmq                 25.1.1
```

Example 5 (unknown):
```unknown
-----
IPython             8.12.2
jupyter_client      8.3.0
jupyter_core        5.3.1
-----
Python 3.8.17 (default, Jul  5 2023, 21:04:15) [GCC 11.2.0]
Linux-4.15.0-197-generic-x86_64-with-glibc2.17
-----
Session information updated at 2023-09-04 09:13
```

Example 6 (python):
```python
sc.settings.set_figure_params(dpi=120)
```

Example 7 (python):
```python
adata = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/vCM/MultiVI-post_foetal_vCM_onlyMultiome_allregion_onlyLeft_feature-select-001.h5ad')
adata
```

Example 8 (unknown):
```unknown
AnnData object with n_obs × n_vars = 16451 × 174604
    obs: 'sangerID', 'combinedID', 'donor', 'region', 'age', 'sex', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'week', 'trimester', 'cycling', 'S_score', 'G2M_score', 'phase', 'fine_grain', 'mid_grain', 'coarse_grain', 'batch_key', 'total_counts', 'total_counts_mt', 'total_counts_ribo', 'n_genes_by_counts', 'pct_counts_mt', 'pct_counts_ribo', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'log1p_total_counts_mt', 'log1p_total_counts_ribo', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', '_indices', '_scvi_batch', '_scvi_labels', 'leiden_MultiVI'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id', 'seqnames', 'start', 'end', 'width', 'strand', 'score', 'replicateScoreQuantile', 'groupScoreQuantile', 'Reproducibility', 'GroupReplicate', 'distToGeneStart', 'nearestGene', 'peakType', 'distToTSS', 'nearestTSS', 'GC', 'idx', 'N', 'n_cells'
    uns: '_scvi_manager_uuid', '_scvi_uuid', 'cycling_colors', 'donor_colors', 'fine_grain_colors', 'leiden', 'leiden_MultiVI_colors', 'neighbors', 'region_colors', 'umap', 'week_colors'
    obsm: 'MultiVI_latent', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous_covs'
    obsp: 'connectivities', 'distances'
```

Example 9 (python):
```python
sc.pl.umap(adata, color=['fine_grain','week'], size=10, wspace=0.3, components='all', ncols=3)
```

Example 10 (python):
```python
sc.pl.umap(adata, color=['week'], groups=['4W','5W','7W','8W','9W',], size=10, wspace=0.3, components='all', ncols=3)
```

Example 11 (python):
```python
/home/jovyan/my-conda-envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:1227: FutureWarning: In a future version of pandas all arguments of Categorical.replace except for the argument 'value' will be keyword-only.
  values = values.replace(values.categories.difference(groups), np.nan)
/home/jovyan/my-conda-envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:1227: FutureWarning: Categorical.replace is deprecated and will be removed in a future version. Use Series.replace directly instead.
  values = values.replace(values.categories.difference(groups), np.nan)
/home/jovyan/my-conda-envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:1227: FutureWarning: In a future version of pandas all arguments of Categorical.replace except for the argument 'value' will be keyword-only.
  values = values.replace(values.categories.difference(groups), np.nan)
/home/jovyan/my-conda-envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:1227: FutureWarning: Categorical.replace is deprecated and will be removed in a future version. Use Series.replace directly instead.
  values = values.replace(values.categories.difference(groups), np.nan)
/home/jovyan/my-conda-envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:1227: FutureWarning: In a future version of pandas all arguments of Categorical.replace except for the argument 'value' will be keyword-only.
  values = values.replace(values.categories.difference(groups), np.nan)
/home/jovyan/my-conda-envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:1227: FutureWarning: Categorical.replace is deprecated and will be removed in a future version. Use Series.replace directly instead.
  values = values.replace(values.categories.difference(groups), np.nan)
```

Example 12 (python):
```python
sc.pl.umap(adata, color=['week'], groups=['12W','13W','14W','15W','20W',], size=10, wspace=0.3, components='all', ncols=3)
```

Example 13 (python):
```python
/home/jovyan/my-conda-envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:1227: FutureWarning: In a future version of pandas all arguments of Categorical.replace except for the argument 'value' will be keyword-only.
  values = values.replace(values.categories.difference(groups), np.nan)
/home/jovyan/my-conda-envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:1227: FutureWarning: Categorical.replace is deprecated and will be removed in a future version. Use Series.replace directly instead.
  values = values.replace(values.categories.difference(groups), np.nan)
/home/jovyan/my-conda-envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:1227: FutureWarning: In a future version of pandas all arguments of Categorical.replace except for the argument 'value' will be keyword-only.
  values = values.replace(values.categories.difference(groups), np.nan)
/home/jovyan/my-conda-envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:1227: FutureWarning: Categorical.replace is deprecated and will be removed in a future version. Use Series.replace directly instead.
  values = values.replace(values.categories.difference(groups), np.nan)
/home/jovyan/my-conda-envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:1227: FutureWarning: In a future version of pandas all arguments of Categorical.replace except for the argument 'value' will be keyword-only.
  values = values.replace(values.categories.difference(groups), np.nan)
/home/jovyan/my-conda-envs/vitro/lib/python3.8/site-packages/scanpy/plotting/_tools/scatterplots.py:1227: FutureWarning: Categorical.replace is deprecated and will be removed in a future version. Use Series.replace directly instead.
  values = values.replace(values.categories.difference(groups), np.nan)
```

Example 14 (python):
```python
import scFates as scf
import palantir
```

Example 15 (unknown):
```unknown
findfont: Font family ['Raleway'] not found. Falling back to DejaVu Sans.
findfont: Font family ['Lato'] not found. Falling back to DejaVu Sans.
```

Example 16 (python):
```python
projections = pd.DataFrame(adata.obsm["MultiVI_latent"],index=adata.obs_names)
projections
```

Example 17 (unknown):
```unknown
# fate probability mapping of the cells
dm_res = palantir.utils.run_diffusion_maps(projections,knn=30)
ms_data = palantir.utils.determine_multiscale_space(dm_res,n_eigs=5)
```

Example 18 (unknown):
```unknown
Determing nearest neighbor graph...
```

Example 19 (python):
```python
# Generate embedding from the multiscale diffusion space
adata.obsm["X_palantir"]=ms_data.values
sc.pp.neighbors(adata,n_neighbors=30,use_rep="X_palantir")
```

Example 20 (python):
```python
sc.pl.embedding(adata, basis='X_palantir',color=['week','fine_grain'], components='all',ncols=3,wspace=0.6,size=10)
```

Example 21 (python):
```python
%%time
# draw ForceAtlas2 embedding using UMAP dim1 and dim2 as initial position
adata.obsm['X_umap13'] = adata.obsm['X_umap'][:,[0,2]]
# adata.obsm['X_palantir12'] = adata.obsm['X_palantir'][:,[0,1]]
sc.tl.draw_graph(adata,init_pos='X_umap13',random_state=0)
```

Example 22 (unknown):
```unknown
CPU times: user 2min 23s, sys: 23.1 ms, total: 2min 23s
Wall time: 2min 23s
```

Example 23 (python):
```python
# adata.uns["week_colors"][list(adata.obs['week'].cat.categories).index('13W')]='purple'
sc.pl.draw_graph(adata,color="week",color_map="RdBu_r")
```

Example 24 (python):
```python
sc.pl.draw_graph(adata,color="fine_grain",color_map="RdBu_r")
```

Example 25 (python):
```python
scf.tl.tree(adata,method="ppt",Nodes=40,use_rep="palantir",
            device="cpu",seed=5,ppt_lambda=100,ppt_sigma=0.025,ppt_nsteps=200)
```

Example 26 (unknown):
```unknown
inferring a principal tree --> parameters used 
    40 principal points, sigma = 0.025, lambda = 100, metric = euclidean
    fitting:  29%|██▉       | 58/200 [00:04<00:10, 13.10it/s]
    converged
    finished (0:00:04) --> added 
    .uns['ppt'], dictionnary containing inferred tree.
    .obsm['X_R'] soft assignment of cells to principal points.
    .uns['graph']['B'] adjacency matrix of the principal points.
    .uns['graph']['F'] coordinates of principal points in representation space.
```

Example 27 (python):
```python
scf.pl.graph(adata,tips=True,forks=False)
```

Example 28 (python):
```python
# Selecting a root and computing pseudotime
scf.tl.root(adata,17)
# generate 100 mappings of pseudotime, to account for cell asssignment uncertainty.
# .obs will be saved the mean of all calculated pseudotimes.
scf.tl.pseudotime(adata,n_jobs=1,n_map=1,seed=40)
```

Example 29 (python):
```python
node 17 selected as a root --> added
    .uns['graph']['root'] selected root.
    .uns['graph']['pp_info'] for each PP, its distance vs root and segment assignment.
    .uns['graph']['pp_seg'] segments network information.
projecting cells onto the principal graph
    finished (0:00:16) --> added
    .obs['edge'] assigned edge.
    .obs['t'] pseudotime value.
    .obs['seg'] segment of the tree assigned.
    .obs['milestones'] milestone assigned.
    .uns['pseudotime_list'] list of cell projection from all mappings.
```

Example 30 (python):
```python
scf.pl.trajectory(adata,arrows=True,arrow_offset=3,scale_path=0.5,
                 )
```

Example 31 (python):
```python
sc.pl.draw_graph(adata, color=["t","seg","milestones"], cmap='viridis')
```

Example 32 (python):
```python
sc.pl.draw_graph(adata[adata.obs['fine_grain']=='VentricularCardiomyocytesLeftTrabeculated'], 
                 color=["t"], cmap='viridis')
```

Example 33 (python):
```python
adata.obs['fine_grain'].value_counts()
```

Example 34 (unknown):
```unknown
VentricularCardiomyocytesLeftCompact         11397
VentricularCardiomyocytesLeftTrabeculated     5054
Name: fine_grain, dtype: int64
```

Example 35 (python):
```python
# read in anndata which has all the features
adata_all = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/vCM/MultiVI-pre_paired_foetal_vCMs_raw.h5ad')

# create new anndata
adata_new = anndata.AnnData(X=adata_all[adata.obs_names,:].X, # from all feature data
                      obs=adata.obs, 
                      var=adata_all.var, # from all feature data
                      uns=adata.uns,
                      obsm=adata.obsm, 
                      obsp=adata.obsp 
                               )
adata_new
```

Example 36 (unknown):
```unknown
AnnData object with n_obs × n_vars = 16451 × 544641
    obs: 'sangerID', 'combinedID', 'donor', 'region', 'age', 'sex', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'week', 'trimester', 'cycling', 'S_score', 'G2M_score', 'phase', 'fine_grain', 'mid_grain', 'coarse_grain', 'batch_key', 'total_counts', 'total_counts_mt', 'total_counts_ribo', 'n_genes_by_counts', 'pct_counts_mt', 'pct_counts_ribo', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'log1p_total_counts_mt', 'log1p_total_counts_ribo', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', '_indices', '_scvi_batch', '_scvi_labels', 'leiden_MultiVI', 't', 'seg', 'edge', 'milestones', 'age_days', 'week_cont'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id', 'seqnames', 'start', 'end', 'width', 'strand', 'score', 'replicateScoreQuantile', 'groupScoreQuantile', 'Reproducibility', 'GroupReplicate', 'distToGeneStart', 'nearestGene', 'peakType', 'distToTSS', 'nearestTSS', 'GC', 'idx', 'N'
    uns: '_scvi_manager_uuid', '_scvi_uuid', 'cycling_colors', 'donor_colors', 'draw_graph', 'fine_grain_colors', 'graph', 'leiden', 'leiden_MultiVI_colors', 'milestones_colors', 'neighbors', 'ppt', 'pseudotime_list', 'region_colors', 'umap', 'week_colors', 'seg_colors'
    obsm: 'MultiVI_latent', 'X_R', 'X_draw_graph_fa', 'X_palantir', 'X_umap', 'X_umap13', '_scvi_extra_categorical_covs', '_scvi_extra_continuous_covs'
    obsp: 'connectivities', 'distances'
```

Example 37 (python):
```python
# make time continuous
import re
def age_to_days(age_str):
    match = re.match(r'(\d+)W(\d+)D', age_str)
    if match:
        weeks, days = map(int, match.groups())
        return float(weeks * 7 + days)
    else:
        return None

# Assuming 'adata' is an AnnData object
adata_new.obs['age_days'] = adata_new.obs['age'].apply(age_to_days)
adata_new.obs['age_days'] = adata_new.obs['age_days'].astype(float)

# week
adata_new.obs['week_cont'] = [int(x.replace('W','')) for x in adata_new.obs['week']]
```

Example 38 (python):
```python
adata_new.write('/nfs/team205/heart/anndata_objects/Foetal/vCM/post_scFate/foetal_vCM_onlyLeft.h5ad')
```

Example 39 (python):
```python
# Log-normalise
# gene expression
adata_lognorm = adata_new[:,adata_new.var['modality']=='Gene Expression'].copy()
sc.pp.normalize_total(adata_lognorm, target_sum=1e4)
sc.pp.log1p(adata_lognorm)
```

Example 40 (python):
```python
sc.pl.draw_graph(adata_lognorm,color=["t",'week','fine_grain'],legend_fontsize=10, cmap='viridis')
```

Example 41 (python):
```python
sc.pl.draw_graph(adata_lognorm,color=['age_days'],legend_fontsize=10, cmap='viridis')
```

Example 42 (python):
```python
sc.pl.draw_graph(adata_lognorm, color=["HEY2","MYOM2","MYH7","MYL2",'TNNI3','CPT1B','ACTN1','TBX20','CACNA1C','RYR2',
                                       'CD36','PRDM16','SCN5A','GJA1'],legend_loc='on data',legend_fontsize=10, cmap='RdPu')
```

Example 43 (python):
```python
adata = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/vCM/post_scFate/foetal_vCM_onlyLeft.h5ad')
print(adata.X.data[:10])
adata
```

Example 44 (unknown):
```unknown
[1. 2. 3. 2. 5. 3. 2. 1. 1. 1.]
```

Example 45 (unknown):
```unknown
AnnData object with n_obs × n_vars = 16451 × 544641
    obs: 'sangerID', 'combinedID', 'donor', 'region', 'age', 'sex', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'week', 'trimester', 'cycling', 'S_score', 'G2M_score', 'phase', 'fine_grain', 'mid_grain', 'coarse_grain', 'batch_key', 'total_counts', 'total_counts_mt', 'total_counts_ribo', 'n_genes_by_counts', 'pct_counts_mt', 'pct_counts_ribo', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'log1p_total_counts_mt', 'log1p_total_counts_ribo', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', '_indices', '_scvi_batch', '_scvi_labels', 'leiden_MultiVI', 't', 'seg', 'edge', 'milestones', 'age_days', 'week_cont'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id', 'seqnames', 'start', 'end', 'width', 'strand', 'score', 'replicateScoreQuantile', 'groupScoreQuantile', 'Reproducibility', 'GroupReplicate', 'distToGeneStart', 'nearestGene', 'peakType', 'distToTSS', 'nearestTSS', 'GC', 'idx', 'N'
    uns: '_scvi_manager_uuid', '_scvi_uuid', 'cycling_colors', 'donor_colors', 'draw_graph', 'fine_grain_colors', 'graph', 'leiden', 'leiden_MultiVI_colors', 'milestones_colors', 'neighbors', 'ppt', 'pseudotime_list', 'region_colors', 'seg_colors', 'umap', 'week_colors'
    obsm: 'MultiVI_latent', 'X_R', 'X_draw_graph_fa', 'X_palantir', 'X_umap', 'X_umap13', '_scvi_extra_categorical_covs', '_scvi_extra_continuous_covs'
    obsp: 'connectivities', 'distances'
```

Example 46 (python):
```python
adata.var['modality'].value_counts()
```

Example 47 (unknown):
```unknown
Peaks              508040
Gene Expression     36601
Name: modality, dtype: int64
```

Example 48 (python):
```python
adata.obs['fine_grain'].value_counts()
```

Example 49 (unknown):
```unknown
VentricularCardiomyocytesLeftCompact         11397
VentricularCardiomyocytesLeftTrabeculated     5054
Name: fine_grain, dtype: int64
```

Example 50 (python):
```python
# subset
adata_sub = adata[adata.obs_names[adata.obs['fine_grain']=='VentricularCardiomyocytesLeftCompact']]
print(adata_sub.shape)

# bin t
n_bin_time = 3
adata_sub.obs['t_binned'] = pd.cut(adata_sub.obs['t'],
                               bins=n_bin_time,
                               labels=[f't{str(x+1)}' for x in range(n_bin_time)])
print(adata_sub.obs['t_binned'].value_counts())
sc.pl.draw_graph(adata_sub,color=['t_binned'],color_map="viridis", wspace=0.2)
```

Example 51 (unknown):
```unknown
(11397, 544641)
```

Example 52 (python):
```python
/tmp/ipykernel_468/2720878155.py:7: ImplicitModificationWarning: Trying to modify attribute `.obs` of view, initializing view as actual.
  adata_sub.obs['t_binned'] = pd.cut(adata_sub.obs['t'],
```

Example 53 (unknown):
```unknown
t3    7754
t2    3477
t1     166
Name: t_binned, dtype: int64
```

Example 54 (python):
```python
# save
# RNA
adata_gex = adata_sub[:,adata_sub.var['modality']=='Gene Expression']
adata_gex.write('/nfs/team205/heart/anndata_objects/Vitro/post_multiVI_full-features/Foetal_vCM_Left-compact_rna.h5ad')
print(adata_gex)

# ATAC
adata_atac = adata_sub[:,adata_sub.var['modality']=='Peaks']
adata_atac.write('/nfs/team205/heart/anndata_objects/Vitro/post_multiVI_full-features/Foetal_vCM_Left-compact_atac.h5ad')
print(adata_atac)
```

Example 55 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 11397 × 36601
    obs: 'sangerID', 'combinedID', 'donor', 'region', 'age', 'sex', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'week', 'trimester', 'cycling', 'S_score', 'G2M_score', 'phase', 'fine_grain', 'mid_grain', 'coarse_grain', 'batch_key', 'total_counts', 'total_counts_mt', 'total_counts_ribo', 'n_genes_by_counts', 'pct_counts_mt', 'pct_counts_ribo', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'log1p_total_counts_mt', 'log1p_total_counts_ribo', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', '_indices', '_scvi_batch', '_scvi_labels', 'leiden_MultiVI', 't', 'seg', 'edge', 'milestones', 'age_days', 'week_cont', 't_binned'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id', 'seqnames', 'start', 'end', 'width', 'strand', 'score', 'replicateScoreQuantile', 'groupScoreQuantile', 'Reproducibility', 'GroupReplicate', 'distToGeneStart', 'nearestGene', 'peakType', 'distToTSS', 'nearestTSS', 'GC', 'idx', 'N'
    uns: '_scvi_manager_uuid', '_scvi_uuid', 'cycling_colors', 'donor_colors', 'draw_graph', 'fine_grain_colors', 'graph', 'leiden', 'leiden_MultiVI_colors', 'milestones_colors', 'neighbors', 'ppt', 'pseudotime_list', 'region_colors', 'umap', 'week_colors', 't_binned_colors'
    obsm: 'MultiVI_latent', 'X_R', 'X_draw_graph_fa', 'X_palantir', 'X_umap', 'X_umap13', '_scvi_extra_categorical_covs', '_scvi_extra_continuous_covs'
    obsp: 'connectivities', 'distances'
View of AnnData object with n_obs × n_vars = 11397 × 508040
    obs: 'sangerID', 'combinedID', 'donor', 'region', 'age', 'sex', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'week', 'trimester', 'cycling', 'S_score', 'G2M_score', 'phase', 'fine_grain', 'mid_grain', 'coarse_grain', 'batch_key', 'total_counts', 'total_counts_mt', 'total_counts_ribo', 'n_genes_by_counts', 'pct_counts_mt', 'pct_counts_ribo', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'log1p_total_counts_mt', 'log1p_total_counts_ribo', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', '_indices', '_scvi_batch', '_scvi_labels', 'leiden_MultiVI', 't', 'seg', 'edge', 'milestones', 'age_days', 'week_cont', 't_binned'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id', 'seqnames', 'start', 'end', 'width', 'strand', 'score', 'replicateScoreQuantile', 'groupScoreQuantile', 'Reproducibility', 'GroupReplicate', 'distToGeneStart', 'nearestGene', 'peakType', 'distToTSS', 'nearestTSS', 'GC', 'idx', 'N'
    uns: '_scvi_manager_uuid', '_scvi_uuid', 'cycling_colors', 'donor_colors', 'draw_graph', 'fine_grain_colors', 'graph', 'leiden', 'leiden_MultiVI_colors', 'milestones_colors', 'neighbors', 'ppt', 'pseudotime_list', 'region_colors', 'umap', 'week_colors', 't_binned_colors'
    obsm: 'MultiVI_latent', 'X_R', 'X_draw_graph_fa', 'X_palantir', 'X_umap', 'X_umap13', '_scvi_extra_categorical_covs', '_scvi_extra_continuous_covs'
    obsp: 'connectivities', 'distances'
```

Example 56 (python):
```python
# subset
adata_sub = adata[adata.obs_names[adata.obs['fine_grain']=='VentricularCardiomyocytesLeftTrabeculated']]
print(adata_sub.shape)

# bin t
n_bin_time = 3
adata_sub.obs['t_binned'] = pd.cut(adata_sub.obs['t'],
                               bins=n_bin_time,
                               labels=[f't{str(x+1)}' for x in range(n_bin_time)])
print(adata_sub.obs['t_binned'].value_counts())
sc.pl.draw_graph(adata_sub,color=['t_binned'],color_map="viridis", wspace=0.2)
```

Example 57 (unknown):
```unknown
(5054, 544641)
```

Example 58 (python):
```python
/tmp/ipykernel_468/2639444817.py:7: ImplicitModificationWarning: Trying to modify attribute `.obs` of view, initializing view as actual.
  adata_sub.obs['t_binned'] = pd.cut(adata_sub.obs['t'],
```

Example 59 (unknown):
```unknown
t3    3021
t2    1886
t1     147
Name: t_binned, dtype: int64
```

Example 60 (python):
```python
# save
# RNA
adata_gex = adata_sub[:,adata_sub.var['modality']=='Gene Expression']
adata_gex.write('/nfs/team205/heart/anndata_objects/Vitro/post_multiVI_full-features/Foetal_vCM_Left-trabeculated_rna.h5ad')
print(adata_gex)

# ATAC
adata_atac = adata_sub[:,adata_sub.var['modality']=='Peaks']
adata_atac.write('/nfs/team205/heart/anndata_objects/Vitro/post_multiVI_full-features/Foetal_vCM_Left-trabeculated_atac.h5ad')
print(adata_atac)
```

Example 61 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 5054 × 36601
    obs: 'sangerID', 'combinedID', 'donor', 'region', 'age', 'sex', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'week', 'trimester', 'cycling', 'S_score', 'G2M_score', 'phase', 'fine_grain', 'mid_grain', 'coarse_grain', 'batch_key', 'total_counts', 'total_counts_mt', 'total_counts_ribo', 'n_genes_by_counts', 'pct_counts_mt', 'pct_counts_ribo', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'log1p_total_counts_mt', 'log1p_total_counts_ribo', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', '_indices', '_scvi_batch', '_scvi_labels', 'leiden_MultiVI', 't', 'seg', 'edge', 'milestones', 'age_days', 'week_cont', 't_binned'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id', 'seqnames', 'start', 'end', 'width', 'strand', 'score', 'replicateScoreQuantile', 'groupScoreQuantile', 'Reproducibility', 'GroupReplicate', 'distToGeneStart', 'nearestGene', 'peakType', 'distToTSS', 'nearestTSS', 'GC', 'idx', 'N'
    uns: '_scvi_manager_uuid', '_scvi_uuid', 'cycling_colors', 'donor_colors', 'draw_graph', 'fine_grain_colors', 'graph', 'leiden', 'leiden_MultiVI_colors', 'milestones_colors', 'neighbors', 'ppt', 'pseudotime_list', 'region_colors', 'umap', 'week_colors', 't_binned_colors'
    obsm: 'MultiVI_latent', 'X_R', 'X_draw_graph_fa', 'X_palantir', 'X_umap', 'X_umap13', '_scvi_extra_categorical_covs', '_scvi_extra_continuous_covs'
    obsp: 'connectivities', 'distances'
View of AnnData object with n_obs × n_vars = 5054 × 508040
    obs: 'sangerID', 'combinedID', 'donor', 'region', 'age', 'sex', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'week', 'trimester', 'cycling', 'S_score', 'G2M_score', 'phase', 'fine_grain', 'mid_grain', 'coarse_grain', 'batch_key', 'total_counts', 'total_counts_mt', 'total_counts_ribo', 'n_genes_by_counts', 'pct_counts_mt', 'pct_counts_ribo', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'log1p_total_counts_mt', 'log1p_total_counts_ribo', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', '_indices', '_scvi_batch', '_scvi_labels', 'leiden_MultiVI', 't', 'seg', 'edge', 'milestones', 'age_days', 'week_cont', 't_binned'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id', 'seqnames', 'start', 'end', 'width', 'strand', 'score', 'replicateScoreQuantile', 'groupScoreQuantile', 'Reproducibility', 'GroupReplicate', 'distToGeneStart', 'nearestGene', 'peakType', 'distToTSS', 'nearestTSS', 'GC', 'idx', 'N'
    uns: '_scvi_manager_uuid', '_scvi_uuid', 'cycling_colors', 'donor_colors', 'draw_graph', 'fine_grain_colors', 'graph', 'leiden', 'leiden_MultiVI_colors', 'milestones_colors', 'neighbors', 'ppt', 'pseudotime_list', 'region_colors', 'umap', 'week_colors', 't_binned_colors'
    obsm: 'MultiVI_latent', 'X_R', 'X_draw_graph_fa', 'X_palantir', 'X_umap', 'X_umap13', '_scvi_extra_categorical_covs', '_scvi_extra_continuous_covs'
    obsp: 'connectivities', 'distances'
```

Example 62 (python):
```python
adata = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/vCM/post_scFate/foetal_vCM_onlyLeft.h5ad')
```

Example 63 (unknown):
```unknown
AnnData object with n_obs × n_vars = 16451 × 544641
    obs: 'sangerID', 'combinedID', 'donor', 'region', 'age', 'sex', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'week', 'trimester', 'cycling', 'S_score', 'G2M_score', 'phase', 'fine_grain', 'mid_grain', 'coarse_grain', 'batch_key', 'total_counts', 'total_counts_mt', 'total_counts_ribo', 'n_genes_by_counts', 'pct_counts_mt', 'pct_counts_ribo', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'log1p_total_counts_mt', 'log1p_total_counts_ribo', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', '_indices', '_scvi_batch', '_scvi_labels', 'leiden_MultiVI', 't', 'seg', 'edge', 'milestones', 'age_days', 'week_cont'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id', 'seqnames', 'start', 'end', 'width', 'strand', 'score', 'replicateScoreQuantile', 'groupScoreQuantile', 'Reproducibility', 'GroupReplicate', 'distToGeneStart', 'nearestGene', 'peakType', 'distToTSS', 'nearestTSS', 'GC', 'idx', 'N'
    uns: '_scvi_manager_uuid', '_scvi_uuid', 'cycling_colors', 'donor_colors', 'draw_graph', 'fine_grain_colors', 'graph', 'leiden', 'leiden_MultiVI_colors', 'milestones_colors', 'neighbors', 'ppt', 'pseudotime_list', 'region_colors', 'seg_colors', 'umap', 'week_colors'
    obsm: 'MultiVI_latent', 'X_R', 'X_draw_graph_fa', 'X_palantir', 'X_umap', 'X_umap13', '_scvi_extra_categorical_covs', '_scvi_extra_continuous_covs'
    obsp: 'connectivities', 'distances'
```

Example 64 (python):
```python
sns.scatterplot(adata.obs['t'],adata.obs['t'])
```

---

## Read in¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/Xenium/no2_integration_and_cell-type-annotation/notebook_outs/no03-2_harmony_clustering_out_29May2025.html

**Contents:**
- Read in¶
- Clustering¶
- Save¶

**Examples:**

Example 1 (python):
```python
import scanpy as sc
import numpy as np
import pandas as pd
import os
# import rapids_singlecell as rsc
from tqdm import tqdm

import matplotlib.pyplot as plt
import seaborn as sns
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
/home/kk837/.conda/envs/generic_env/lib/python3.10/site-packages/session_info/main.py:213: UserWarning: The '__version__' attribute is deprecated and will be removed in MarkupSafe 3.1. Use feature detection, or `importlib.metadata.version("markupsafe")`, instead.
  mod_version = _find_version(mod.__version__)
```

Example 4 (python):
```python
-----
matplotlib          3.10.1
numpy               2.1.3
pandas              2.2.3
scanpy              1.11.1
seaborn             0.13.2
session_info        v1.0.1
tqdm                4.67.1
-----
```

Example 5 (python):
```python
PIL                 11.1.0
anndata             0.11.4
asciitree           NA
asttokens           NA
charset_normalizer  3.4.1
cloudpickle         3.1.1
comm                0.2.2
cycler              0.12.1
cython_runtime      NA
dask                2024.11.2
dateutil            2.9.0.post0
debugpy             1.8.13
decorator           5.2.1
exceptiongroup      1.2.2
executing           2.2.0
h5py                3.13.0
igraph              0.11.8
importlib_metadata  NA
ipykernel           6.29.5
jedi                0.19.2
jinja2              3.1.6
joblib              1.4.2
kiwisolver          1.4.8
legacy_api_wrap     NA
leidenalg           0.10.2
llvmlite            0.44.0
markupsafe          3.0.2
mpl_toolkits        NA
msgpack             1.1.0
natsort             8.4.0
numba               0.61.0
numcodecs           0.13.1
packaging           24.2
parso               0.8.4
patsy               1.0.1
platformdirs        4.3.7
prompt_toolkit      3.0.50
psutil              7.0.0
pure_eval           0.2.3
pyarrow             19.0.1
pydev_ipython       NA
pydevconsole        NA
pydevd              3.2.3
pydevd_file_utils   NA
pydevd_plugins      NA
pydevd_tracing      NA
pygments            2.19.1
pyparsing           3.2.3
pytz                2025.2
scipy               1.15.2
six                 1.17.0
sklearn             1.5.2
stack_data          0.6.3
statsmodels         0.14.4
tblib               3.1.0
texttable           1.7.0
threadpoolctl       3.6.0
tlz                 1.0.0
toolz               1.0.0
tornado             6.4.2
traitlets           5.14.3
typing_extensions   NA
wcwidth             0.2.13
yaml                6.0.2
zarr                2.18.3
zipp                NA
zmq                 26.3.0
zoneinfo            NA
```

Example 6 (unknown):
```unknown
-----
IPython             8.34.0
jupyter_client      8.6.3
jupyter_core        5.7.2
-----
Python 3.10.14 | packaged by conda-forge | (main, Mar 20 2024, 12:45:18) [GCC 12.3.0]
Linux-4.18.0-553.45.1.el8_10.x86_64-x86_64-with-glibc2.28
-----
Session information updated at 2025-05-29 12:14
```

Example 7 (python):
```python
sc.settings.set_figure_params(dpi=120)
```

Example 8 (python):
```python
objects_dir = '/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium'
path_adata = f'{objects_dir}/C194-HEA-0-FFPE-1_Hst45-HEA-0-FFPE-1_concat_5K_filtered_raw.h5ad'
adata = sc.read_h5ad(path_adata)
adata
```

Example 9 (unknown):
```unknown
AnnData object with n_obs × n_vars = 362277 × 5001
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1'
    uns: 'celltypist_mid_fin_colors', 'pca', 'pca_harmony', 'tissue_block_id_colors', 'umap'
    obsm: 'X_pca', 'X_pca_harmony', 'X_umap_pca_harmony', 'spatial'
    varm: 'PCs'
    obsp: 'pca_harmony_connectivities', 'pca_harmony_distances'
```

Example 10 (unknown):
```unknown
latent_space = 'pca_harmony'
```

Example 11 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=['total_counts','tissue_block_id'], wspace=0.3, cmap='RdPu')
```

Example 12 (python):
```python
sc.pl.embedding(adata, basis=f"X_umap_{latent_space}",
                color=['celltypist_mid_fin'], wspace=0.2, cmap='RdPu')
```

Example 13 (python):
```python
%%time
latent_space='pca_harmony'
leiden_keys=[]
for res in tqdm([0.2,0.3,0.4,0.5,0.8,1.0,1.5,2.0,3.0]):
    leiden_key = f"leiden_{res}"
    key_added=f"{latent_space}_{leiden_key}"
    sc.tl.leiden(adata,neighbors_key=latent_space,resolution=res,key_added=key_added)
    leiden_keys.append(key_added)
```

Example 14 (unknown):
```unknown
0%|          | 0/9 [00:00<?, ?it/s]
```

Example 15 (unknown):
```unknown
<timed exec>:6: FutureWarning: In the future, the default backend for leiden will be igraph instead of leidenalg.

 To achieve the future defaults please pass: flavor="igraph" and n_iterations=2.  directed must also be False to work with igraph's implementation.
```

Example 16 (unknown):
```unknown
11%|█         | 1/9 [07:39<1:01:13, 459.19s/it]
```

Example 17 (unknown):
```unknown
22%|██▏       | 2/9 [19:30<1:10:50, 607.25s/it]
```

Example 18 (unknown):
```unknown
33%|███▎      | 3/9 [33:44<1:12:01, 720.18s/it]
```

Example 19 (unknown):
```unknown
44%|████▍     | 4/9 [47:17<1:03:03, 756.76s/it]
```

Example 20 (unknown):
```unknown
56%|█████▌    | 5/9 [54:49<43:08, 647.04s/it]
```

Example 21 (unknown):
```unknown
67%|██████▋   | 6/9 [1:11:29<38:20, 766.86s/it]
```

Example 22 (unknown):
```unknown
78%|███████▊  | 7/9 [1:34:34<32:18, 969.11s/it]
```

Example 23 (unknown):
```unknown
89%|████████▉ | 8/9 [1:47:13<15:01, 901.98s/it]
```

Example 24 (unknown):
```unknown
100%|██████████| 9/9 [2:03:02<00:00, 916.69s/it]
```

Example 25 (unknown):
```unknown
100%|██████████| 9/9 [2:03:02<00:00, 820.25s/it]
```

Example 26 (unknown):
```unknown
CPU times: user 2h 2min 7s, sys: 46 s, total: 2h 2min 53s
Wall time: 2h 3min 2s
```

Example 27 (python):
```python
sc.pl.embedding(adata,basis=f"X_umap_{latent_space}",
                color=leiden_keys,wspace=0.3,ncols=5)
```

Example 28 (python):
```python
adata.write(path_adata)
adata
```

Example 29 (unknown):
```unknown
AnnData object with n_obs × n_vars = 362277 × 5001
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes', 'tissue_block_id', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_3.0'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts-0', 'mean_counts-0', 'log1p_mean_counts-0', 'pct_dropout_by_counts-0', 'total_counts-0', 'log1p_total_counts-0', 'n_cells_by_counts-1', 'mean_counts-1', 'log1p_mean_counts-1', 'pct_dropout_by_counts-1', 'total_counts-1', 'log1p_total_counts-1'
    uns: 'celltypist_mid_fin_colors', 'pca', 'pca_harmony', 'tissue_block_id_colors', 'umap', 'pca_harmony_leiden_0.2', 'pca_harmony_leiden_0.3', 'pca_harmony_leiden_0.4', 'pca_harmony_leiden_0.5', 'pca_harmony_leiden_0.8', 'pca_harmony_leiden_1.0', 'pca_harmony_leiden_1.5', 'pca_harmony_leiden_2.0', 'pca_harmony_leiden_3.0', 'pca_harmony_leiden_0.2_colors', 'pca_harmony_leiden_0.3_colors', 'pca_harmony_leiden_0.4_colors', 'pca_harmony_leiden_0.5_colors', 'pca_harmony_leiden_0.8_colors', 'pca_harmony_leiden_1.0_colors', 'pca_harmony_leiden_1.5_colors', 'pca_harmony_leiden_2.0_colors', 'pca_harmony_leiden_3.0_colors'
    obsm: 'X_pca', 'X_pca_harmony', 'X_umap_pca_harmony', 'spatial'
    varm: 'PCs'
    obsp: 'pca_harmony_connectivities', 'pca_harmony_distances'
```

---

## Import modules¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig3_SAN_niche/cellphonedb-VisiumHD_no1_prepare-data.html

**Contents:**
- Import modules¶
- Set path¶
- Read in anndata¶
- Subset cellular niche of interest¶
- Save¶

https://github.com/ventolab/CellphoneDB/blob/master/notebooks/0_prepare_your_data_from_anndata.ipynb

**Examples:**

Example 1 (python):
```python
import numpy as np
import pandas as pd
import scanpy as sc
import anndata
import os
import sys
from scipy import sparse
import pickle
import matplotlib.pyplot as plt

sc.settings.verbosity = 1  # verbosity: errors (0), warnings (1), info (2), hints (3)
```

Example 2 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 3 (unknown):
```unknown
sample_id = 'HEA_FOET14880396'
cellphone_input_dir = f'/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/cellphonedb/{sample_id}/inputs'
cellphone_input_dir
```

Example 4 (unknown):
```unknown
'/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/cellphonedb/HEA_FOET14880396/inputs'
```

Example 5 (unknown):
```unknown
celltype_col = 'fine_grain_4Jan2025'
```

Example 6 (unknown):
```unknown
niche_label_col = 'tt_prediction'
niche_of_interest = 'SAnode'
```

Example 7 (python):
```python
# read in 
adata = sc.read_h5ad(f'/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/{sample_id}_b2c_cells_filtered_celltype-selected_lognorm.h5ad')
# rename sinus-horn cells
adata.obs.replace({celltype_col:
                   {'SinoatrialNodePacemakerCellsHorn':'SinusHornPacemakerCells'}},
                  inplace=True
                 )
adata
```

Example 8 (unknown):
```unknown
AnnData object with n_obs × n_vars = 92164 × 18047
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'fine_grain_4Jan2025', 'HistologicalAnnotation', 'cluster_cellcharter_k-8', 'cluster_cellcharter_k-10', 'cluster_cellcharter_k-12', 'cluster_cellcharter_k-14', 'cluster_cellcharter_k-16', 'cluster_cellcharter_k-18', 'cluster_cellcharter_k-20', 'cluster_cellcharter_k-25', 'cluster_cellcharter_k-30', 'cluster_cellcharter_k-35', 'cluster_cellcharter_k-40', 'cluster_cellcharter_k-45', 'cluster_cellcharter_k-50', 'predicted_labels', 'conf_score', 'predicted_labels_cellcharter-cluster-k50', 'conf_score_cellcharter-cluster-k50', 'spatial_leiden_1', 'spatial_leiden_5', 'spatial_leiden_0.5', 'spatial_leiden_2', 'spatial_leiden_10', 'spatial_leiden_100', 'spatial_leiden_1000', 'KMeans_k9216_cluster', 'predicted_labels_spatial_KMeans_k9216', 'conf_score_spatial_KMeans_k9216', 'KMeans_k4608_cluster', 'predicted_labels_spatial_KMeans_k4608', 'conf_score_spatial_KMeans_k4608', 'KMeans_k3072_cluster', 'KMeans_k1843_cluster', 'predicted_labels_spatial_KMeans_k3072', 'conf_score_spatial_KMeans_k3072', 'predicted_labels_spatial_KMeans_k1843', 'conf_score_spatial_KMeans_k1843', 'sliding_window_assignment', 'predicted_labels_sliding_window_assignment', 'conf_score_sliding_window_assignment', 'tt_prediction'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'HistologicalAnnotation_colors', 'KMeans_k1843_centroids', 'KMeans_k3072_centroids', 'KMeans_k4608_centroids', 'KMeans_k9216_centroids', 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'cluster_cellcharter_k-10_colors', 'cluster_cellcharter_k-12_colors', 'cluster_cellcharter_k-14_colors', 'cluster_cellcharter_k-16_colors', 'cluster_cellcharter_k-18_colors', 'cluster_cellcharter_k-20_colors', 'cluster_cellcharter_k-25_colors', 'cluster_cellcharter_k-30_colors', 'cluster_cellcharter_k-35_colors', 'cluster_cellcharter_k-40_colors', 'cluster_cellcharter_k-45_colors', 'cluster_cellcharter_k-50_colors', 'cluster_cellcharter_k-8_colors', 'conf_score_cellcharter-cluster-k50_colors', 'conf_score_sliding_window_assignment_colors', 'fine_grain_4Jan2025_colors', 'hvg', 'log1p', 'neighbors', 'pca', 'predicted_labels_cellcharter-cluster-k50_colors', 'predicted_labels_colors', 'predicted_labels_sliding_window_assignment_colors', 'predicted_labels_spatial_KMeans_k1843_colors', 'predicted_labels_spatial_KMeans_k3072_colors', 'predicted_labels_spatial_KMeans_k4608_colors', 'predicted_labels_spatial_KMeans_k9216_colors', 'sliding_window_assignment_colors', 'spatial', 'spatial_leiden_0.5', 'spatial_leiden_1', 'spatial_leiden_10', 'spatial_leiden_100', 'spatial_leiden_1000', 'spatial_leiden_1_colors', 'spatial_leiden_2', 'spatial_leiden_5', 'spatial_leiden_5_colors', 'spatial_neighbors', 'tt_prediction_colors', 'umap'
    obsm: 'X_cellcharter', 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances', 'spatial_connectivities', 'spatial_distances'
```

Example 9 (python):
```python
adata.X.data[:5]
```

Example 10 (unknown):
```unknown
array([1.27187559, 1.95641568, 1.75023687, 1.96760567, 2.7810601 ])
```

Example 11 (python):
```python
# update cell type name
df = pd.read_csv('/nfs/team205/heart/anndata_objects/Foetal/finegrain_name_mapping.csv')
celltype_mapping = df.set_index('Full_name')['Short_name_finalised'].to_dict()
adata.obs.replace({celltype_col:celltype_mapping},inplace=True)
```

Example 12 (python):
```python
adata_sub = adata[adata.obs[niche_label_col]==niche_of_interest]
adata_sub
```

Example 13 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 1313 × 18047
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'fine_grain_4Jan2025', 'HistologicalAnnotation', 'cluster_cellcharter_k-8', 'cluster_cellcharter_k-10', 'cluster_cellcharter_k-12', 'cluster_cellcharter_k-14', 'cluster_cellcharter_k-16', 'cluster_cellcharter_k-18', 'cluster_cellcharter_k-20', 'cluster_cellcharter_k-25', 'cluster_cellcharter_k-30', 'cluster_cellcharter_k-35', 'cluster_cellcharter_k-40', 'cluster_cellcharter_k-45', 'cluster_cellcharter_k-50', 'predicted_labels', 'conf_score', 'predicted_labels_cellcharter-cluster-k50', 'conf_score_cellcharter-cluster-k50', 'spatial_leiden_1', 'spatial_leiden_5', 'spatial_leiden_0.5', 'spatial_leiden_2', 'spatial_leiden_10', 'spatial_leiden_100', 'spatial_leiden_1000', 'KMeans_k9216_cluster', 'predicted_labels_spatial_KMeans_k9216', 'conf_score_spatial_KMeans_k9216', 'KMeans_k4608_cluster', 'predicted_labels_spatial_KMeans_k4608', 'conf_score_spatial_KMeans_k4608', 'KMeans_k3072_cluster', 'KMeans_k1843_cluster', 'predicted_labels_spatial_KMeans_k3072', 'conf_score_spatial_KMeans_k3072', 'predicted_labels_spatial_KMeans_k1843', 'conf_score_spatial_KMeans_k1843', 'sliding_window_assignment', 'predicted_labels_sliding_window_assignment', 'conf_score_sliding_window_assignment', 'tt_prediction'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'HistologicalAnnotation_colors', 'KMeans_k1843_centroids', 'KMeans_k3072_centroids', 'KMeans_k4608_centroids', 'KMeans_k9216_centroids', 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'cluster_cellcharter_k-10_colors', 'cluster_cellcharter_k-12_colors', 'cluster_cellcharter_k-14_colors', 'cluster_cellcharter_k-16_colors', 'cluster_cellcharter_k-18_colors', 'cluster_cellcharter_k-20_colors', 'cluster_cellcharter_k-25_colors', 'cluster_cellcharter_k-30_colors', 'cluster_cellcharter_k-35_colors', 'cluster_cellcharter_k-40_colors', 'cluster_cellcharter_k-45_colors', 'cluster_cellcharter_k-50_colors', 'cluster_cellcharter_k-8_colors', 'conf_score_cellcharter-cluster-k50_colors', 'conf_score_sliding_window_assignment_colors', 'fine_grain_4Jan2025_colors', 'hvg', 'log1p', 'neighbors', 'pca', 'predicted_labels_cellcharter-cluster-k50_colors', 'predicted_labels_colors', 'predicted_labels_spatial_KMeans_k1843_colors', 'predicted_labels_spatial_KMeans_k3072_colors', 'predicted_labels_spatial_KMeans_k4608_colors', 'predicted_labels_spatial_KMeans_k9216_colors', 'spatial', 'spatial_leiden_0.5', 'spatial_leiden_1', 'spatial_leiden_10', 'spatial_leiden_100', 'spatial_leiden_1000', 'spatial_leiden_1_colors', 'spatial_leiden_2', 'spatial_leiden_5', 'spatial_leiden_5_colors', 'spatial_neighbors', 'tt_prediction_colors', 'umap'
    obsm: 'X_cellcharter', 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances', 'spatial_connectivities', 'spatial_distances'
```

Example 14 (python):
```python
# filter cell types
n_cell_thresh = 10
val_counts = adata_sub.obs[celltype_col].value_counts()
celltype_sel = list(val_counts.index[val_counts>n_cell_thresh])
adata_sub = adata_sub[adata_sub.obs[celltype_col].isin(celltype_sel)]
adata_sub.obs[celltype_col].values.describe()
```

Example 15 (python):
```python
meta = adata_sub.obs.copy()
meta['cell_id'] = meta.index
meta = meta[['cell_id', celltype_col]]
meta.to_csv(f'{cellphone_input_dir}/meta_{niche_of_interest}.txt', index=False, sep = "\t")

adata_sub.write(f'{cellphone_input_dir}/log_norm_counts_{niche_of_interest}.h5ad')
```

Example 16 (unknown):
```unknown
f'{cellphone_input_dir}/meta_{niche_of_interest}.txt'
```

Example 17 (unknown):
```unknown
'/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/cellphonedb/HEA_FOET14880396/inputs/meta_SAnode.txt'
```

Example 18 (unknown):
```unknown
f'{cellphone_input_dir}/log_norm_counts_{niche_of_interest}.h5ad'
```

Example 19 (unknown):
```unknown
'/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/cellphonedb/HEA_FOET14880396/inputs/log_norm_counts_SAnode.h5ad'
```

Example 20 (unknown):
```unknown
!ls -lh {cellphone_input_dir}
```

---

## Read in adata and prepare¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/ATAC/ATAC/DAR-genes-linkage_SANPC-specific.html

**Contents:**
- Read in adata and prepare¶
- Read in differential accessible result¶
  - Select celltypes to analyse¶
- Set celltypes to analyse¶

2756709 rows × 9 columns

**Examples:**

Example 1 (python):
```python
import scanpy as sc
import pandas as pd
import numpy as np
import anndata
import matplotlib.pyplot as plt
import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 
import seaborn as sns
import os
# import snapatac2 as snap
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
-----
anndata             0.9.2
matplotlib          3.7.2
numpy               1.24.4
pandas              1.5.3
scanpy              1.9.3
seaborn             0.12.2
session_info        1.0.0
-----
```

Example 4 (python):
```python
PIL                         10.0.0
anyio                       NA
arrow                       1.2.3
asttokens                   NA
attr                        23.1.0
attrs                       23.1.0
babel                       2.12.1
backcall                    0.2.0
brotli                      1.0.9
certifi                     2023.11.17
cffi                        1.15.1
charset_normalizer          3.2.0
colorama                    0.4.6
comm                        0.1.4
cycler                      0.10.0
cython_runtime              NA
dateutil                    2.8.2
debugpy                     1.6.8
decorator                   5.1.1
defusedxml                  0.7.1
executing                   1.2.0
fastjsonschema              NA
fqdn                        NA
h5py                        3.9.0
idna                        3.4
igraph                      0.10.6
importlib_metadata          NA
importlib_resources         NA
ipykernel                   6.25.1
isoduration                 NA
jedi                        0.19.0
jinja2                      3.1.2
joblib                      1.3.2
json5                       NA
jsonpointer                 2.0
jsonschema                  4.19.0
jsonschema_specifications   NA
jupyter_events              0.7.0
jupyter_server              2.7.1
jupyterlab_server           2.24.0
kiwisolver                  1.4.5
leidenalg                   0.10.1
llvmlite                    0.40.1
markupsafe                  2.1.3
mpl_toolkits                NA
natsort                     8.4.0
nbformat                    5.9.2
numba                       0.57.1
overrides                   NA
packaging                   23.1
parso                       0.8.3
patsy                       0.5.3
pexpect                     4.8.0
pickleshare                 0.7.5
pkg_resources               NA
plotly                      5.16.1
prometheus_client           NA
prompt_toolkit              3.0.39
psutil                      5.9.5
ptyprocess                  0.7.0
pure_eval                   0.2.2
pydev_ipython               NA
pydevconsole                NA
pydevd                      2.9.5
pydevd_file_utils           NA
pydevd_plugins              NA
pydevd_tracing              NA
pygments                    2.16.1
pyparsing                   3.0.9
pythonjsonlogger            NA
pytz                        2023.3
referencing                 NA
requests                    2.31.0
rfc3339_validator           0.1.4
rfc3986_validator           0.1.1
rpds                        NA
ruamel                      NA
scipy                       1.10.1
send2trash                  NA
six                         1.16.0
sklearn                     1.3.0
sniffio                     1.3.0
socks                       1.7.1
sphinxcontrib               NA
stack_data                  0.6.2
statsmodels                 0.14.0
texttable                   1.6.7
threadpoolctl               3.2.0
tornado                     6.3.3
traitlets                   5.9.0
typing_extensions           NA
uri_template                NA
urllib3                     2.0.4
wcwidth                     0.2.6
webcolors                   1.13
websocket                   1.6.2
yaml                        6.0.1
zipp                        NA
zmq                         25.1.1
zstandard                   0.19.0
```

Example 5 (unknown):
```unknown
-----
IPython             8.12.0
jupyter_client      8.3.0
jupyter_core        4.12.0
jupyterlab          4.0.5
notebook            7.0.2
-----
Python 3.8.17 (default, Jul  5 2023, 21:04:15) [GCC 11.2.0]
Linux-4.15.0-220-generic-x86_64-with-glibc2.17
-----
Session information updated at 2024-02-09 16:26
```

Example 6 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
# spec = importlib.util.spec_from_file_location("module.name", "/home/jovyan/data_for_outage/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 7 (python):
```python
sc.settings.set_figure_params(dpi=80,dpi_save=300)
```

Example 8 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk'
```

Example 9 (python):
```python
# prepare expressing gene list
adata_gene = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/Feb28ObjectRaw.h5ad')

# rename fine_grain cell types
celltype_rename = utils.read_pkl('/nfs/team205/heart/anndata_objects/Foetal/fine_grain_rename_dictionary.pkl')
adata_gene.obs['fine_grain'] = [celltype_rename[x] for x in adata_gene.obs['fine_grain']]

# log-normalise
sc.pp.normalize_total(adata_gene, target_sum=1e4)
sc.pp.log1p(adata_gene)

# calculate proportion of expressing cells per gene
exp_mat = utils.get_exp_proportion(adata_gene, groupby='fine_grain', use_rep='X')
print(exp_mat.shape)
exp_mat.head()
```

Example 10 (unknown):
```unknown
(36601, 63)
```

Example 11 (python):
```python
# read in adata peak with back mode
adata_peak = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/PeakMatrix/Foetal_Peaks_tfidf.h5ad')

# modify peak names
adata_peak.var_names = [x.replace('_','-') for x in adata_peak.var_names]

# rename fine_grain cell types
adata_peak.obs['fine_grain'] = [celltype_rename[x] for x in adata_peak.obs['fine_grain']]

# accessible matrix
acc_mat = utils.get_exp_proportion(adata_peak, groupby='fine_grain', use_rep='X')

adata_peak
```

Example 12 (unknown):
```unknown
AnnData object with n_obs × n_vars = 167022 × 508040
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels'
    var: 'seqnames', 'start', 'end', 'width', 'strand', 'score', 'replicateScoreQuantile', 'groupScoreQuantile', 'Reproducibility', 'GroupReplicate', 'distToGeneStart', 'nearestGene', 'peakType', 'distToTSS', 'nearestTSS', 'GC', 'idx', 'N'
```

Example 13 (python):
```python
adata_peak.var_names[:10]
```

Example 14 (unknown):
```unknown
Index(['chr1:817093-817593', 'chr1:819818-820318', 'chr1:820542-821042',
       'chr1:825246-825746', 'chr1:826566-827066', 'chr1:827291-827791',
       'chr1:828703-829203', 'chr1:830674-831174', 'chr1:831213-831713',
       'chr1:832066-832566'],
      dtype='object')
```

Example 15 (unknown):
```unknown
dar_res_dir = '/nfs/team205/heart/anndata_objects/Foetal/multiome_ATAC/ArchR/project_output/MarkerPeaks'
os.listdir(dar_res_dir)
```

Example 16 (unknown):
```unknown
['SANP-vs-aCM.csv',
 'global_FDR001.csv',
 '.ipynb_checkpoints',
 'archive',
 'per-mid-grain.csv',
 'Mesenchymal_fine-grain_all.csv',
 'Cardiomyocytes_fine-grain_all.csv',
 'Leukocytes_fine-grain_all.csv',
 'Endothelium_fine-grain_all.csv',
 'Epicardium_fine-grain_all.csv',
 'Neural_fine-grain_all.csv',
 'global_coarse-grain_FDR005_logFC1.csv',
 'vCM_Left_Trab-vs-Compact.csv',
 'vCM_Left_Compact-vs-Trab.csv',
 'aCM-vs-SANP.csv']
```

Example 17 (python):
```python
DAR_res_dict={}
for file in os.listdir(dar_res_dir):
    if '_fine-grain_all.csv' in file:
        celltype = file.split('_fine-grain_all.csv')[0]
        # read in 
        df = pd.read_csv(f'{dar_res_dir}/{file}')
        # make peak names
        df['names'] = df['seqnames']+':'+df['start'].astype('str')+'-'+df['end'].astype('str')
        # rename fine_grain celltypes
        df['celltype'] = [celltype_rename[x] for x in df['celltype']]
        # add to the dictionary
        DAR_res_dict[celltype] = df.copy()
DAR_res_dict.keys()
```

Example 18 (unknown):
```unknown
dict_keys(['Mesenchymal', 'Cardiomyocytes', 'Leukocytes', 'Endothelium', 'Epicardium', 'Neural'])
```

Example 19 (unknown):
```unknown
DAR_res_dict[celltype]
```

Example 20 (unknown):
```unknown
DAR_res_dict.keys()
```

Example 21 (unknown):
```unknown
dict_keys(['Mesenchymal', 'Cardiomyocytes', 'Leukocytes', 'Endothelium', 'Epicardium', 'Neural'])
```

Example 22 (unknown):
```unknown
df = DAR_res_dict['Cardiomyocytes']

# select specific and accessible peaks
pval_thresh = 0.05
logFC_thresh_specific = 0.1

df = df[(df['FDR']<pval_thresh)&(df['Log2FC']>logFC_thresh_specific)]
df['celltype'].value_counts()
```

Example 23 (unknown):
```unknown
SinoatrialNodePacemakerCells                 3354
AtrialCardiomyocytesRight                    1213
AtrialCardiomyocytesLeft                      844
AtrialCardiomyocytesCycling                   579
AtrioventricularNodePacemakerCells            574
VentricularConductionSystemDistal             171
VentricularCardiomyocytesRightCompact          11
VentricularCardiomyocytesLeftCompact            2
VentricularCardiomyocytesLeftTrabeculated       2
VentricularConductionSystemProximal             2
VentricularCardiomyocytesCycling                1
Name: celltype, dtype: int64
```

Example 24 (python):
```python
import gseapy as gp
from gseapy import barplot, dotplot
```

Example 25 (python):
```python
celltype_reference = 'Cardiomyocytes'
celltype_of_interest = 'SinoatrialNodePacemakerCells'

# select specific and accessible peaks
accessible_proportion_thresh = 0.1
pval_thresh = 0.05
logFC_thresh_specific = 1


### Read in pyscenic region-gene metadata ###

# read in pyscenic outputs
eRegulon_metadata = pd.read_csv(f'/nfs/team205/heart/pyscenic/plus/eRegulon_metadata/{celltype_reference}_eRegulon_metadata_filtered.csv')
eRegulon_metadata['regulon_name_short'] = [x.split('_(')[0][:-2] for x in eRegulon_metadata['Region_signature_name']]
print(eRegulon_metadata.shape)

### Select accessible and specific peaks ###

# select peaks which is differentially accessible
DAR_res = DAR_res_dict[celltype_reference]
DAR_res = DAR_res[DAR_res['celltype']==celltype_of_interest]
DAR_res = DAR_res[(DAR_res['FDR']<pval_thresh)&(DAR_res['Log2FC']>logFC_thresh_specific)]

# select peaks which is accessible more than cetrain proportion of the cell type
acc_mat_bi = (acc_mat > accessible_proportion_thresh).astype('int')
acc_peaks = list(acc_mat_bi[acc_mat_bi[celltype_of_interest]==1].index)
DAR_res = DAR_res[DAR_res['names'].isin(acc_peaks)]

# add peaktype
DAR_res['peakType']=list(adata_peak.var.loc[list(DAR_res['names']),'peakType'])

# get the selected peaks
spec_peaks = list(DAR_res['names'])
print(len(spec_peaks))
print(len(acc_peaks))

### Get genes which linked to the peaks ###
# metadata of regions regulons which contain the selected peaks
eRegulon_metadata_sel = eRegulon_metadata[eRegulon_metadata['Region'].isin(spec_peaks)]
print(eRegulon_metadata_sel.shape)

# genes linked to the selected peaks
# and expressed in the cell type of interst 
expressed_proportion_thresh = 0.1
exp_genes = list(exp_mat.index[exp_mat[celltype_of_interest]>expressed_proportion_thresh])
print(f'n_genes expressed in {celltype_of_interest}: {len(exp_genes)}')

# get linked and expressed genes
df = eRegulon_metadata_sel[eRegulon_metadata_sel['Gene'].isin(exp_genes)]
df = df[['Region','Gene']]
df.drop_duplicates(inplace=True)
linked_genes = list(set(df['Gene']))
rank = df['Gene'].value_counts()
print(f'n_genes linked to specific peaks: {len(linked_genes)}')

# get genes which linked to enhancer peak
df['peakType'] = list(adata_peak.var['peakType'].loc[df['Region']])
linked_genes_enh = list(set(df[df['peakType'].isin(['Distal','Intronic'])]['Gene']))
print(f'n_genes linked to specific peaks (Distal, Intronic): {len(linked_genes_enh)}')

# plot
n_rank = 50
plt.rcParams['axes.grid'] = False
plt.rcParams['figure.figsize'] = 4.5,12
plt.barh(width=rank[:n_rank],y=rank[:n_rank].index,color='darkred')
plt.gca().invert_yaxis()
plt.title(f'Number of {celltype_of_interest} specific peaks')
plt.margins(y=0.01)
plt.show()

### Pathway enrichment analysis of the linked genes ###
# default: Human
geneset_list = ['MSigDB_Hallmark_2020','KEGG_2021_Human','GO_Biological_Process_2023','Reactome_2022']
enr = gp.enrichr(gene_list=linked_genes_enh, # or "./tests/data/gene_list.txt",
                 gene_sets=geneset_list,
                 organism='human', # don't forget to set organism to the one you desired! e.g. Yeast
                 outdir=None, # don't write to disk
                )

geneset = 'GO_Biological_Process_2023'
df = enr.results[enr.results['Gene_set'] == geneset]
df = df[df['Adjusted P-value'] < 0.1]
df = df.sort_values('Combined Score',ascending=False)

# plot
n_rank = 10
plt.rcParams['axes.grid'] = False
plt.rcParams['figure.figsize'] = 6,4
plt.barh(width=df[:n_rank]['Combined Score'],
         y=[x.split(' (')[0] for x in df[:n_rank]['Term']],
         color='darkred')
plt.gca().invert_yaxis()
plt.xlabel('Score')
plt.title(geneset)
plt.margins(y=0.05)
plt.savefig(f'{figdir}/Peak_linked-gene-pathways_{celltype_of_interest}.pdf', bbox_inches="tight",dpi=300)
plt.show()


for geneset in geneset_list:
    print(geneset)
    df = enr.results[enr.results['Gene_set'] == geneset]
    display(df.sort_values('Combined Score',ascending=False)[:20])
```

Example 26 (unknown):
```unknown
(260320, 18)
402
18427
(1621, 18)
n_genes expressed in SinoatrialNodePacemakerCells: 7427
n_genes linked to specific peaks: 211
n_genes linked to specific peaks (Distal, Intronic): 152
```

Example 27 (unknown):
```unknown
MSigDB_Hallmark_2020
```

Example 28 (unknown):
```unknown
KEGG_2021_Human
```

Example 29 (unknown):
```unknown
GO_Biological_Process_2023
```

Example 30 (unknown):
```unknown
Reactome_2022
```

Example 31 (python):
```python
# proportion of peaktypes
# specific&accessible peak and all peaks
df = pd.DataFrame({
    'type':list(np.repeat('All accessible peaks',len(acc_peaks)))+list(np.repeat('Specific peaks',len(spec_peaks))),
    'peakType':list(adata_peak.var.loc[acc_peaks,'peakType'])+list(adata_peak.var.loc[spec_peaks,'peakType'])
})
df['peakType'] = df['peakType'].astype('category')
df['peakType'].cat.reorder_categories(['Promoter','Exonic','Intronic','Distal'],
                                            inplace=True)

tmp = pd.crosstab(df['type'], df['peakType'], normalize='index')
# plot
plt.rcParams['figure.figsize'] = 2,3
tmp.plot.bar(stacked=True,width=0.7,legend=False)
plt.gca().grid(False)
plt.gca().set_title(celltype_of_interest)
plt.gca().set_xlabel('')
plt.gca().set_xticklabels(plt.gca().get_xticklabels(), rotation=45, 
                          ha='right',rotation_mode='anchor')
plt.gca().set_ylabel('proportion')
plt.gca().legend(bbox_to_anchor=(1.0, 1.0))
plt.show()
```

Example 32 (unknown):
```unknown
/tmp/ipykernel_30396/3341389757.py:8: FutureWarning: The `inplace` parameter in pandas.Categorical.reorder_categories is deprecated and will be removed in a future version. Reordering categories will always return a new Categorical object.
  df['peakType'].cat.reorder_categories(['Promoter','Exonic','Intronic','Distal'],
```

---

## Import modules¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig5_Trisomy21/no01_t21_scVI-for-cleanup.html

**Contents:**
- Import modules¶
- Read in anndata¶
- Pre-process¶
- Setup anndata for scVI¶
- Creating and training a model¶
- Save¶
- Run per condition¶

**Examples:**

Example 1 (python):
```python
# this is for the jupyter notebook at JupyterLab
# to make sys.path between the terminal and Jupyter same
import sys

print(sys.path) # especially the path to site-packages are different... so thats why couldn't find correct package...

# sys.path from terminal at scvi0190_env
# obtained by excuting `python -c "import sys; print('\n'.join(sys.path))"`
sys.path = [
    "/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python39.zip",
    "/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9",
    "/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/lib-dynload",
    "/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages"
]

print(sys.path)
```

Example 2 (unknown):
```unknown
['/nfs/team205/kk18/Analysis/scVI/notebooks', '/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python39.zip', '/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9', '/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/lib-dynload', '', '/nfs/users/nfs_k/kk18/.local/lib/python3.9/site-packages', '/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages']
['/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python39.zip', '/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9', '/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/lib-dynload', '/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages']
```

Example 3 (python):
```python
# import warnings
# warnings.simplefilter("ignore", UserWarning)

import anndata
import matplotlib.pyplot as plt
import seaborn as sns

import numpy as np
import numpy.random as random
import pandas as pd
import scanpy as sc

import scvi
```

Example 4 (python):
```python
Global seed set to 0
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/flax/core/frozen_dict.py:169: FutureWarning: jax.tree_util.register_keypaths is deprecated, and will be removed in a future release. Please use `register_pytree_with_keys()` instead.
  jax.tree_util.register_keypaths(
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/pytorch_lightning/utilities/warnings.py:53: LightningDeprecationWarning: pytorch_lightning.utilities.warnings.rank_zero_deprecation has been deprecated in v1.6 and will be removed in v1.8. Use the equivalent function from the pytorch_lightning.utilities.rank_zero module instead.
  new_rank_zero_deprecation(
/nfs/team205/kk18/miniconda3/envs/scvi0190_env/lib/python3.9/site-packages/pytorch_lightning/utilities/warnings.py:58: LightningDeprecationWarning: The `pytorch_lightning.loggers.base.rank_zero_experiment` is deprecated in v1.7 and will be removed in v1.9. Please use `pytorch_lightning.loggers.logger.rank_zero_experiment` instead.
  return new_rank_zero_deprecation(*args, **kwargs)
```

Example 5 (python):
```python
import gc
import torch
```

Example 6 (python):
```python
import session_info
session_info.show()
```

Example 7 (python):
```python
-----
anndata             0.8.0
matplotlib          3.7.1
numpy               1.23.5
pandas              1.5.3
scanpy              1.9.3
scvi                0.19.0
seaborn             0.12.2
session_info        1.0.0
torch               1.13.1+cu117
-----
```

Example 8 (python):
```python
PIL                 9.4.0
absl                NA
asttokens           NA
attr                22.2.0
backcall            0.2.0
certifi             2022.12.07
cffi                1.15.1
charset_normalizer  3.1.0
chex                0.1.6
comm                0.1.2
contextlib2         NA
cycler              0.10.0
cython_runtime      NA
dateutil            2.8.2
debugpy             1.6.6
decorator           5.1.1
defusedxml          0.7.1
deprecate           0.3.2
docrep              0.3.2
etils               1.1.0
executing           1.2.0
flax                0.6.7
fsspec              2023.3.0
google              NA
h5py                3.8.0
idna                3.4
igraph              0.10.4
importlib_metadata  NA
importlib_resources NA
ipykernel           6.21.3
ipython_genutils    0.2.0
ipywidgets          8.0.4
jax                 0.4.10
jaxlib              0.4.10
jedi                0.18.2
joblib              1.2.0
kiwisolver          1.4.4
leidenalg           0.9.1
llvmlite            0.39.1
ml_collections      NA
ml_dtypes           0.2.0
mpl_toolkits        NA
msgpack             1.0.5
mudata              0.2.1
multipledispatch    0.6.0
natsort             8.3.1
numba               0.56.4
numpyro             0.11.0
nvidia              NA
opt_einsum          v3.3.0
optax               0.1.4
packaging           23.0
parso               0.8.3
pexpect             4.8.0
pickleshare         0.7.5
pkg_resources       NA
platformdirs        3.1.1
prompt_toolkit      3.0.38
psutil              5.9.4
ptyprocess          0.7.0
pure_eval           0.2.2
pydev_ipython       NA
pydevconsole        NA
pydevd              2.9.5
pydevd_file_utils   NA
pydevd_plugins      NA
pydevd_tracing      NA
pygments            2.14.0
pyparsing           3.0.9
pyro                1.8.4
pytorch_lightning   1.7.7
pytz                2022.7.1
requests            2.28.2
rich                NA
scipy               1.10.1
setuptools          65.6.3
six                 1.16.0
sklearn             1.2.2
stack_data          0.6.2
statsmodels         0.13.5
tensorboard         2.12.0
texttable           1.6.7
threadpoolctl       3.1.0
toolz               0.12.0
torchmetrics        0.11.4
torchvision         0.14.1+cu117
tornado             6.2
tqdm                4.65.0
traitlets           5.9.0
tree                0.1.8
typing_extensions   NA
urllib3             1.26.15
wcwidth             0.2.6
yaml                6.0
zipp                NA
zmq                 25.0.1
zoneinfo            NA
```

Example 9 (unknown):
```unknown
-----
IPython             8.11.0
jupyter_client      8.0.3
jupyter_core        5.2.0
notebook            6.5.3
-----
Python 3.9.16 (main, Mar  8 2023, 14:00:05) [GCC 11.2.0]
Linux-4.15.0-222-generic-x86_64-with-glibc2.27
-----
Session information updated at 2024-04-04 22:57
```

Example 10 (python):
```python
sc.settings.verbosity = 3             # verbosity: errors (0), warnings (1), info (2), hints (3)
sc.settings.set_figure_params(dpi=80, facecolor='white', color_map = 'RdPu',)
```

Example 11 (unknown):
```unknown
!ls -lh /nfs/team205/heart/anndata_objects/Foetal/QC/
```

Example 12 (unknown):
```unknown
total 18G
drwxrwsr-x 2 kk18 team205   213 Aug 17  2022 multiome_ATAC
-rw-rw-r-- 1 kk18 team205   11M Feb  8  2023 Multiome_rm-doublets.txt
-rw-rw-r-- 1 kk18 team205  4.3G Feb  8  2023 multiome_RNA_foetal.h5ad
-rw-rw-r-- 1 kk18 team205   12G Feb  8  2023 multiome_RNA_foetal_prefilter.h5ad
-rw-rw-r-- 1 kk18 team205  2.0G Dec 30  2022 scRNA_foetal.h5ad
-rw-rw-r-- 1 kk18 team205  2.6G Dec 12 15:09 scRNA_foetal_include-doublets_13Dec2023.h5ad
-rw-rw-r-- 1 kk18 team205  2.9G Dec 23  2022 scRNA_foetal_prefilter.h5ad
-rw-rw-r-- 1 kk18 team205 1017K Dec 30  2022 scRNA_rm-doublets.txt
-rw-rw-r-- 1 kk18 team205  4.2M Apr  4 22:13 T21_Multiome_rm-doublets.txt
-rw-rw-r-- 1 kk18 team205  2.3G Apr  4 22:16 T21_multiome_RNA_foetal.h5ad
-rw-rw-r-- 1 kk18 team205  5.5G Apr  4 21:46 T21_multiome_RNA_foetal_prefilter.h5ad
```

Example 13 (python):
```python
adata = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/QC/T21_multiome_RNA_foetal.h5ad')
adata
```

Example 14 (unknown):
```unknown
AnnData object with n_obs × n_vars = 121452 × 36601
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'batch_key'
    var: 'gene_name', 'feature_type', 'ambient_expression-0', 'ambient_expression-1', 'ambient_expression-10', 'ambient_expression-11', 'ambient_expression-12', 'ambient_expression-13', 'ambient_expression-14', 'ambient_expression-2', 'ambient_expression-3', 'ambient_expression-4', 'ambient_expression-5', 'ambient_expression-6', 'ambient_expression-7', 'ambient_expression-8', 'ambient_expression-9', 'mt', 'ribo', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts'
    uns: 'sangerID_colors'
    obsm: 'latent_gene_encoding'
```

Example 15 (python):
```python
adata.X.data[:10]
```

Example 16 (unknown):
```unknown
array([ 1.,  2.,  4.,  1.,  3.,  3., 12.,  6.,  2.,  2.], dtype=float32)
```

Example 17 (python):
```python
print(adata.obs['batch_key'].value_counts())
print(adata.obs['cell_or_nuclei'].value_counts())
```

Example 18 (unknown):
```unknown
Hst42_Nuclei_Multiome-v1    70685
Hst32_Nuclei_Multiome-v1    23019
Hst44_Nuclei_Multiome-v1    22517
Hst48_Nuclei_Multiome-v1     5231
Name: batch_key, dtype: int64
Nuclei    121452
Name: cell_or_nuclei, dtype: int64
```

Example 19 (python):
```python
# preserve raw count
adata.layers["counts"] = adata.X.copy()

# filter genes
sc.pp.filter_genes(adata, min_cells=3)

# normalise and log-transform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

adata.shape
```

Example 20 (unknown):
```unknown
filtered out 3535 genes that are detected in less than 3 cells
normalizing counts per cell
    finished (0:00:03)
```

Example 21 (unknown):
```unknown
(121452, 33066)
```

Example 22 (python):
```python
# hvg
sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key='batch_key')

print(sum(adata.var.highly_variable))
sc.pl.highly_variable_genes(adata)
```

Example 23 (python):
```python
extracting highly variable genes
    finished (0:00:16)
--> added
    'highly_variable', boolean vector (adata.var)
    'means', float vector (adata.var)
    'dispersions', float vector (adata.var)
    'dispersions_norm', float vector (adata.var)
3789
```

Example 24 (python):
```python
# freeze the state in `.raw`
# adata.raw = adata
# subset hvgs
adata = adata[:, adata.var.highly_variable]
```

Example 25 (python):
```python
adata = adata.copy()
scvi.model.SCVI.setup_anndata(adata, 
                              layer="counts", 
                              batch_key="batch_key",
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo'])
```

Example 26 (unknown):
```unknown
No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.)
```

Example 27 (python):
```python
vae = scvi.model.SCVI(adata, n_hidden = 128, n_latent = 50, n_layers = 3, dispersion = 'gene-batch')
vae
```

Example 28 (unknown):
```unknown
SCVI Model with the following params: 
n_hidden: 128, n_latent: 50, n_layers: 3, dropout_rate: 0.1, dispersion: gene-batch, gene_likelihood: zinb, 
latent_distribution: normal
Training status: Not Trained
```

Example 29 (python):
```python
vae.view_anndata_setup(adata)
```

Example 30 (unknown):
```unknown
Anndata setup with scvi-tools version 0.19.0.
```

Example 31 (unknown):
```unknown
Setup via `SCVI.setup_anndata` with arguments:
```

Example 32 (unknown):
```unknown
{
│   'layer': 'counts',
│   'batch_key': 'batch_key',
│   'labels_key': None,
│   'size_factor_key': None,
│   'categorical_covariate_keys': None,
│   'continuous_covariate_keys': ['total_counts', 'pct_counts_mt', 'pct_counts_ribo']
}
```

Example 33 (unknown):
```unknown
Summary Statistics          
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃     Summary Stat Key     ┃ Value  ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│         n_batch          │   4    │
│         n_cells          │ 121452 │
│ n_extra_categorical_covs │   0    │
│ n_extra_continuous_covs  │   3    │
│         n_labels         │   1    │
│          n_vars          │  3789  │
└──────────────────────────┴────────┘
```

Example 34 (python):
```python
Data Registry                            
┏━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃     Registry Key      ┃            scvi-tools Location            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│           X           │          adata.layers['counts']           │
│         batch         │         adata.obs['_scvi_batch']          │
│ extra_continuous_covs │ adata.obsm['_scvi_extra_continuous_covs'] │
│        labels         │         adata.obs['_scvi_labels']         │
└───────────────────────┴───────────────────────────────────────────┘
```

Example 35 (python):
```python
batch State Registry                            
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃    Source Location     ┃        Categories        ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['batch_key'] │ Hst32_Nuclei_Multiome-v1 │          0          │
│                        │ Hst42_Nuclei_Multiome-v1 │          1          │
│                        │ Hst44_Nuclei_Multiome-v1 │          2          │
│                        │ Hst48_Nuclei_Multiome-v1 │          3          │
└────────────────────────┴──────────────────────────┴─────────────────────┘
```

Example 36 (python):
```python
labels State Registry                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃      Source Location      ┃ Categories ┃ scvi-tools Encoding ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ adata.obs['_scvi_labels'] │     0      │          0          │
└───────────────────────────┴────────────┴─────────────────────┘
```

Example 37 (python):
```python
extra_continuous_covs State   
            Registry            
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃       Source Location        ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│  adata.obs['total_counts']   │
│  adata.obs['pct_counts_mt']  │
│ adata.obs['pct_counts_ribo'] │
└──────────────────────────────┘
```

Example 38 (unknown):
```unknown
# train
vae.train()
```

Example 39 (unknown):
```unknown
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
```

Example 40 (unknown):
```unknown
Epoch 66/66: 100%|██████████| 66/66 [14:43<00:00, 13.02s/it, loss=1.2e+03, v_num=1]
```

Example 41 (unknown):
```unknown
`Trainer.fit` stopped: `max_epochs=66` reached.
```

Example 42 (unknown):
```unknown
Epoch 66/66: 100%|██████████| 66/66 [14:43<00:00, 13.39s/it, loss=1.2e+03, v_num=1]
```

Example 43 (unknown):
```unknown
# save scvi model
vae.save("/nfs/team205/heart/anndata_objects/Foetal/scVI/models/RNA_foetal_T21/",overwrite=True)
```

Example 44 (python):
```python
# save anndata
adata.obsm["X_scVI"] = vae.get_latent_representation()
adata.write('/nfs/team205/heart/anndata_objects/Foetal/scVI/T21_multiome_RNA_foetal_post-scVI.h5ad')
```

Example 45 (python):
```python
# Convergency plot
vae.history['elbo_train']
x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
plt.plot(x, vae.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
```

Example 46 (python):
```python
set(adata.obs[celltype_col])
```

Example 47 (python):
```python
%%time
for cond in model_architecture_conditions:
    for celltype in set(adata.obs[celltype_col]):
        print(f'##############################')
        print(f'##### {cond}: {celltype} #####')
        print(f'##############################')

        # subset
        adata_sub = adata[adata.obs[celltype_col]==celltype]
        # filter genes
        sc.pp.filter_genes(adata_sub, min_cells=3)

        # setup anndata for scVI
        adata_sub = adata_sub.copy()
        scvi.model.SCVI.setup_anndata(adata_sub, 
                                  # layer="counts", 
                                  categorical_covariate_keys=['batch_key'],
                                  continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo',
                                                             'S_score', 'G2M_score'
                                                            ])
        # creating a model
        if cond == 'three-layer':
            vae = scvi.model.SCVI(adata_sub, n_hidden = 128, n_latent = 50, n_layers = 3, dispersion = 'gene-batch')
        elif cond == 'two-layer':
            vae = scvi.model.SCVI(adata_sub, n_hidden=128, n_latent=50, n_layers=2, dispersion = 'gene-batch')
        elif cond == 'one-layer':
            vae = scvi.model.SCVI(adata_sub, n_hidden=128, n_latent=50, n_layers=1, dispersion = 'gene-batch')
        print(vae)
        vae.view_anndata_setup(adata_sub)
        # train
        vae.train()

        # save scvi model
        vae.save(f"/nfs/team205/heart/anndata_objects/Foetal/scVI/models/subsets/Foetal_multiome_{celltype}_{cond}/",overwrite=True)

        # save scVI lagent space
        adata_sub.obsm["X_scVI"] = vae.get_latent_representation()
        adata_sub.write(f'/nfs/team205/heart/anndata_objects/Foetal/scVI/subsets/Foetal_multiome_{celltype}_{cond}.h5ad')

        # Convergency plot
        vae.history['elbo_train']
        x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
        plt.plot(x, vae.history['elbo_train'], label="train")
        plt.xlabel('epoch')
        plt.ylabel('elbo_train')
        plt.title(f'{cond}: {celltype}')
        plt.show()

        del adata_sub, vae
        
        # free up GPU
        gc.collect()
        torch.cuda.empty_cache()
        print('')
```

---

## Loading packages¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/Visium/Others/Plotting_multiple_SAnode.html

**Contents:**
- Loading packages¶
- Read in adata¶
- Plotting¶

**Examples:**

Example 1 (python):
```python
# this is for the jupyter notebook at JupyterLab
# to make sys.path between the terminal and Jupyter same
import sys

print(sys.path) # especially the path to site-packages are different... so thats why couldn't find correct package...

# sys.path from terminal at scvi0190_env
# obtained by excuting `python -c "import sys; print('\n'.join(sys.path))"`
sys.path = [
    "/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python39.zip",
    "/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python3.9",
    "/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python3.9/lib-dynload",
    "/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python3.9/site-packages"
]

print(sys.path)
```

Example 2 (unknown):
```unknown
['/nfs/team205/kk18/notebooks/P48_Foetal-heart_Visium/notebooks/notebooks_for_manuscript', '/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python39.zip', '/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python3.9', '/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python3.9/lib-dynload', '', '/nfs/users/nfs_k/kk18/.local/lib/python3.9/site-packages', '/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python3.9/site-packages']
['/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python39.zip', '/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python3.9', '/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python3.9/lib-dynload', '/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python3.9/site-packages']
```

Example 3 (python):
```python
import sys
import scanpy as sc
import anndata
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt 
import matplotlib as mpl

from matplotlib import rcParams
rcParams['pdf.fonttype'] = 42 # enables correct plotting of text
import seaborn as sns

from cell2location.plt import plot_spatial

import os
```

Example 4 (python):
```python
/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python3.9/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm
Global seed set to 0
/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python3.9/site-packages/umap/distances.py:1063: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python3.9/site-packages/umap/distances.py:1071: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python3.9/site-packages/umap/distances.py:1086: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
/nfs/team205/kk18/miniconda3/envs/cell2loc_v013_env/lib/python3.9/site-packages/umap/umap_.py:660: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  @numba.jit()
```

Example 5 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)

spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/_utils.py")
_utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = _utils
spec.loader.exec_module(_utils)
```

Example 6 (python):
```python
sc.settings.verbosity = 1          # verbosity: errors (0), warnings (1), info (2), hints (3)
sc.settings.set_figure_params(dpi=80,dpi_save=300)
```

Example 7 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk'
```

Example 8 (python):
```python
adata_vis = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/visium-OCT_foetal_raw.h5ad')
adata_vis
```

Example 9 (unknown):
```unknown
AnnData object with n_obs × n_vars = 14261 × 33538
    obs: 'in_tissue', 'array_row', 'array_col', 'sample', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'mt_frac', 'sangerID', 'Publication', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'status', 'cell2loc-map_group', 'JC_edit', 'sample_for_cell2loc'
    var: 'gene_ids', 'feature_types', 'genome', 'SYMBOL', 'mt'
    uns: 'spatial'
    obsm: 'MT', 'means_cell_abundance_w_sf', 'q05_cell_abundance_w_sf', 'q95_cell_abundance_w_sf', 'spatial', 'stds_cell_abundance_w_sf'
```

Example 10 (python):
```python
'''
# filtering genes and log-normalise
sc.pp.filter_genes(adata_vis, min_cells=3)
sc.pp.normalize_total(adata_vis, target_sum=1e4)
sc.pp.log1p(adata_vis)

# var.names=SYMBOL
adata_vis.var.set_index('SYMBOL',inplace=True)
adata_vis
'''

# get the cell2loc output
cell2loc_out = adata_vis.obsm['q05_cell_abundance_w_sf'].copy()
## read in cell type name update dictionary
celltype_rename = utils.read_pkl('/nfs/team205/heart/anndata_objects/Foetal/fine_grain_rename_dictionary.pkl')
## update cell type name
cellnames = adata_vis.obsm['q05_cell_abundance_w_sf'].columns
cellnames = [x.split('q05cell_abundance_w_sf_')[1] for x in cellnames]
cellnames = [celltype_rename[x] for x in cellnames]
# add cell2loc output to .obs
adata_vis.obs[cellnames]=cell2loc_out.copy()
adata_vis
```

Example 11 (unknown):
```unknown
AnnData object with n_obs × n_vars = 14261 × 33538
    obs: 'in_tissue', 'array_row', 'array_col', 'sample', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'mt_frac', 'sangerID', 'Publication', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'status', 'cell2loc-map_group', 'JC_edit', 'sample_for_cell2loc', 'AtrialCardiomyocytesLeft', 'AtrialCardiomyocytesRight', 'VentricularCardiomyocytesLeftCompact', 'VentricularCardiomyocytesRightCompact', 'VentricularCardiomyocytesLeftTrabeculated', 'VentricularCardiomyocytesRightTrabeculated', 'SinoatrialNodePacemakerCells', 'AtrioventricularNodePacemakerCells', 'VentricularConductionSystemProximal', 'VentricularConductionSystemDistal', 'GreatVesselAdventitialFibroblasts', 'CoronaryVesselAdventitialFibroblasts', 'MyocardialInterstitialFibroblasts', 'SubEpicardialFibroblasts', 'Myofibroblasts', 'LymphNodeFibroblasticReticularCells', 'ValveInterstitialCells', 'GreatVesselSmoothMuscleCells', 'CoronarySmoothMuscleCells', 'CoronaryPericytes', 'GreatVesselArterialEndothelialCells', 'GreatVesselVenousEndothelialCells', 'CoronaryArterialEndothelialCells', 'CoronaryVenousEndothelialCells', 'CoronaryCapillaryEndothelialCells', 'EndocardialCells', 'EndocardialCushionCells', 'ValveEndothelialCells', 'LymphaticEndothelialCells', 'MesothelialEpicardialCells', 'EpicardiumDerivedCells', 'NeuronPrecursors', 'ChromaffinCells', 'SympatheticNeurons', 'ParasympatheticNeurons', 'SchwannCellPrecursors', 'SchwannCells', 'MonocytesMPOpos', 'Monocytes', 'MonocyteDerivedCells', 'MacrophagesCX3CR1pos', 'MacrophagesTIMD4pos', 'MacrophagesLYVE1pos', 'MacrophagesATF3pos', 'DendriticCellsType1', 'DendriticCellsMature', 'PlasmacytoidDendriticCells', 'MastCells', 'Megakaryocytes', 'TCellsCD4pos', 'TCellsCD8pos', 'TregsCD4pos', 'ProBCells', 'BCells', 'BCellsMS4A1pos', 'NaturalKillerCells', 'InnateLymphoidCells'
    var: 'gene_ids', 'feature_types', 'genome', 'SYMBOL', 'mt'
    uns: 'spatial'
    obsm: 'MT', 'means_cell_abundance_w_sf', 'q05_cell_abundance_w_sf', 'q95_cell_abundance_w_sf', 'spatial', 'stds_cell_abundance_w_sf'
```

Example 12 (python):
```python
slide = _utils.select_multislide(adata_vis, ['HCAHeartST10317184'])

clust_labels = ['SinoatrialNodePacemakerCells',
                'AtrioventricularNodePacemakerCells',
                # 'VentricularConductionSystemProximal',
                'VentricularConductionSystemDistal',
               'ParasympatheticNeurons',
                'MacrophagesCX3CR1pos'
                ]
clust_col = ['' + str(i) for i in clust_labels] # in case column names differ from labels

plt.rcParams['axes.grid'] = False
with mpl.rc_context({'figure.figsize': (15, 15)}):
    fig = plot_spatial(
        adata=slide,
        # labels to show on a plot
        color=clust_col, labels=clust_labels,
        show_img=True, img_alpha=0.5,
        # 'fast' (white background) or 'dark_background'
        style='fast',
        # limit color scale at 99.2% quantile of cell abundance
        max_color_quantile=0.997,
        # size of locations (adjust depending on figure size)
        circle_diameter=5,
        colorbar_position='right',
        colorbar_shape = {'vertical_gaps': 0.8, 'horizontal_gaps': 1,
                                    'width': 0.1, 'height': 0.1},
        # crop_x=(650,820),
        # crop_y=(1200,1420),
        reorder_cmap = [3,0,2,1,4] # 'yellow' 'orange' 'blue' 'green' 'purple' 'grey' 'white'
    )
    plt.savefig(f"{figdir}/Visium_SANP-PSNC-CX3CR1MP_HCAHeartST10317184.pdf", bbox_inches="tight",dpi=300)
```

Example 13 (unknown):
```unknown
/nfs/team205/kk18/function/python/_utils.py:95: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  slides.uns['spatial'] = uns_dict
```

Example 14 (python):
```python
# select up to 6 clusters
# https://github.com/BayraktarLab/cell2location/blob/bd2aae833f81d8ab68684aa44f863b36e3af0aed/cell2location/plt/plot_spatial.py#L108

slide = _utils.select_multislide(adata_vis, ['HCAHeartST10317184'])

clust_labels = ['SinoatrialNodePacemakerCells',
                'AtrioventricularNodePacemakerCells',
                # 'VentricularConductionSystemProximal',
                'VentricularConductionSystemDistal',
               'ParasympatheticNeurons',
                'MacrophagesCX3CR1pos'
                ]
clust_col = ['' + str(i) for i in clust_labels] # in case column names differ from labels

plt.rcParams['axes.grid'] = False
with mpl.rc_context({'figure.figsize': (15, 15)}):
    fig = plot_spatial(
        adata=slide,
        # labels to show on a plot
        color=clust_col, labels=clust_labels,
        show_img=True, img_alpha=0.5,
        # 'fast' (white background) or 'dark_background'
        style='fast',
        # limit color scale at 99.2% quantile of cell abundance
        max_color_quantile=0.997,
        # size of locations (adjust depending on figure size)
        circle_diameter=60,
        colorbar_position='right',
        colorbar_shape = {'vertical_gaps': 0.8, 'horizontal_gaps': 1,
                                    'width': 0.1, 'height': 0.1},
        crop_x=(650,820),
        crop_y=(1200,1420),
        reorder_cmap = [3,0,2,1,4] # 'yellow' 'orange' 'blue' 'green' 'purple' 'grey' 'white'
    )
    plt.savefig(f"{figdir}/Visium_SANP-PSNC-CX3CR1MP_cropped_HCAHeartST10317184.pdf", bbox_inches="tight",dpi=300)
```

Example 15 (unknown):
```unknown
/nfs/team205/kk18/function/python/_utils.py:95: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  slides.uns['spatial'] = uns_dict
```

---

## Display database versions¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig3_SAN_niche/cellphonedb-VisiumHD_no0_download_DB_v5.html

**Contents:**
- Display database versions¶
- Define the version and the path to download database¶
- Download¶

Ref: https://github.com/ventolab/CellphoneDB/blob/master/notebooks/T00_DownloadDB.ipynb

**Examples:**

Example 1 (python):
```python
import pandas as pd
import glob
import os
```

Example 2 (python):
```python
from IPython.display import HTML, display
from cellphonedb.utils import db_releases_utils

display(HTML(db_releases_utils.get_remote_database_versions_html()['db_releases_html_table']))
```

Example 3 (unknown):
```unknown
# -- Version of the databse
cpdb_version = 'v5.0.0'

# -- Path where the input files to generate the database are located
cpdb_target_dir = os.path.join('/nfs/team205/heart/cellphonedb/db', cpdb_version)
```

Example 4 (python):
```python
from cellphonedb.utils import db_utils

db_utils.download_database(cpdb_target_dir, cpdb_version)
```

Example 5 (unknown):
```unknown
Downloaded cellphonedb.zip into /nfs/team205/heart/cellphonedb/db/v5.0.0
Downloaded complex_input.csv into /nfs/team205/heart/cellphonedb/db/v5.0.0
Downloaded gene_input.csv into /nfs/team205/heart/cellphonedb/db/v5.0.0
Downloaded interaction_input.csv into /nfs/team205/heart/cellphonedb/db/v5.0.0
Downloaded protein_input.csv into /nfs/team205/heart/cellphonedb/db/v5.0.0
Downloaded uniprot_synonyms.tsv into /nfs/team205/heart/cellphonedb/db/v5.0.0/sources
Downloaded transcription_factor_input.csv into /nfs/team205/heart/cellphonedb/db/v5.0.0/sources
```

---

## 

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/RNA/scnRNA_Integration-Annotation/Endothelial/EndothelialCellCycleInvestigation.html

10685 rows × 1 columns

2691 rows × 51 columns

2691 rows × 7 columns

**Examples:**

Example 1 (unknown):
```unknown
#Added the keys for age and sex
```

Example 2 (python):
```python
# Import packages, initial settings

import numpy as np
import matplotlib.pyplot as plt
import scanpy as sc
import pandas as pd
import anndata
import scrublet as scr
import seaborn as sns
import magic
import pybiomart
import six
import sys
sys.modules['sklearn.externals.six'] = six
import scanorama
import harmonypy
import bbknn
import gseapy
from gseapy.plot import barplot, dotplot
import scvelo as scv
import scvi

plt.rcParams['pdf.fonttype'] = 42
plt.rcParams['ps.fonttype'] = 42

sc.settings.verbosity = 3  # verbosity: errors (0), warnings (1), info (2), hints (3)

from matplotlib.colors import LinearSegmentedColormap
cmap = LinearSegmentedColormap.from_list(name='gene_cmap', colors=['lightgrey', 'thistle', 'red', 'darkred']) 

sc.settings.set_figure_params(dpi=80, color_map='viridis', vector_friendly=False,  dpi_save=300, facecolor='white')
sc.logging.print_version_and_date()
#sc.logging.print_versions()
```

Example 3 (unknown):
```unknown
Global seed set to 0
```

Example 4 (unknown):
```unknown
Running Scanpy 1.8.1, on 2023-09-19 14:21.
```

Example 5 (python):
```python
adata=sc.read('./Objects/Feb28ObjectRaw.h5ad')
#adata = adata[adata.obs['fine_grain'].isin(['GreatVesselArterialEndothelialCells','GreatVesselVenousEndothelialCells','CoronaryArterialEndothelialCells','CoronaryCapillaryEndothelialCells','CoronaryVenousEndothelialCells'])].copy()
adata = adata[adata.obs['week'].isin(['4W','5W','6W','7W','8W'])].copy()
adata = adata[adata.obs['coarse_grain'].isin(['Endothelium'])].copy()
print(adata.shape)
```

Example 6 (unknown):
```unknown
(11063, 36601)
```

Example 7 (python):
```python
cell_cycle_genes = ['MCM5','PCNA','TYMS','FEN1','MCM2','MCM4','RRM1','UNG','GINS2','MCM6','CDCA7','DTL','PRIM1','UHRF1','MLF1IP','HELLS','RFC2','RPA2','NASP','RAD51AP1','GMNN','WDR76','SLBP','CCNE2','UBR7','POLD3','MSH2','ATAD2','RAD51','RRM2','CDC45','CDC6','EXO1','TIPIN','DSCC1','BLM','CASP8AP2','USP1','CLSPN','POLA1','CHAF1B','BRIP1','E2F8','HMGB2','CDK1','NUSAP1','UBE2C','BIRC5','TPX2','TOP2A','NDC80','CKS2','NUF2','CKS1B','MKI67','TMPO','CENPF','TACC3','FAM64A','SMC4','CCNB2','CKAP2L','CKAP2','AURKB','BUB1','KIF11','ANP32E','TUBB4B','GTSE1','KIF20B','HJURP','CDCA3','HN1','CDC20','TTK','CDC25C','KIF2C','RANGAP1','NCAPD2','DLGAP5','CDCA2','CDCA8','ECT2','KIF23','HMMR','AURKA','PSRC1','ANLN','LBR','CKAP5','CENPE','CTCF','NEK2','G2E3','GAS2L3','CBX5','CENPA']
s_genes = cell_cycle_genes[:43]
s_genes = [x for x in s_genes if x in adata.var_names]
print('S genes:', s_genes, '\n')
g2m_genes = cell_cycle_genes[43:]
g2m_genes = [x for x in g2m_genes if x in adata.var_names]
print('G2 genes:', g2m_genes, '\n')
cell_cycle_genes = [x for x in cell_cycle_genes if x in adata.var_names]

hb_genes1 = ['HBB','HBD','HBG1','HBG2','HBE1','HBZ','HBM','HBA2','HBA1', 'HBQ1']
hb_genes1 = [x for x in hb_genes1 if x in adata.var_names]
print('Hemoglobin genes set 1:', hb_genes1, '\n')

hb_genes2 = ['AHSP','GYPA','RHAG','GFI1B','KEL','EPB42','ALAS2','TSPO2','GYPB','GYPE','SPTA1','RHCE','KLF1','RHD','SLC4A1','SNX22','SQSTM1','SOD1','FTH1','MAP1LC3B','SLC25A37','TRIM58','HMBS']
hb_genes2 = [x for x in hb_genes2 if x in adata.var_names]
print('Hemoglobin genes set 2:', hb_genes2, '\n')

mito_genes=['MT-ND1','MT-ND2','MT-CO1','MT-CO2','MT-ATP8','MT-ATP6','MT-CO3','MT-ND3','MT-ND4L', 'MT-ND4', 'MT-ND5', 'MT-ND6', 'MT-CYB']
mito_genes = [x for x in mito_genes if x in adata.var_names]
print('Mitochondrial genes:', mito_genes, '\n')

ribo_genes = ['RPLP0','RPLP1','RPLP2','RPL3','RPL3L','RPL4','RPL5','RPL6','RPL7','RPL7A','RPL7L1','RPL8','RPL9','RPL10','RPL10A','RPL10L','RPL11','RPL12','RPL13','RPL13A','RPL14','RPL15','RPL17','RPL18','RPL18A','RPL19','RPL21','RPL22','RPL22L1','RPL23','RPL23A','RPL24','RPL26','RPL26L1','RPL27','RPL27A','RPL28','RPL29','RPL30','RPL31','RPL32','RPL34','RPL35','RPL35A','RPL36','RPL36A','RPL36AL','RPL37','RPL37A','RPL38','RPL39','RPL39L','UBA52','RPL41','RPSA','RPS2','RPS3','RPS3A','RPS4X','RPS4Y1','RPS4Y2','RPS5','RPS6','RPS7','RPS8','RPS9','RPS10','RPS11','RPS12','RPS13','RPS14','RPS15','RPS15A','RPS16','RPS17','RPS18','RPS19','RPS20','RPS21','RPS23','RPS24','RPS25','RPS26','RPS27','RPS27A','RPS27L','RPS28','RPS29','FAU']
ribo_genes = [x for x in ribo_genes if x in adata.var_names]
print('Ribo genes:', ribo_genes, '\n')

stress_genes=['FOS','FOSB','JUN','JUNB','JUND','ZFP36','SOCS3','CEBPD','CEBPB','ATF3','EGR1','HSPA1A','HSPA1B','HSP90AA1','HSP90AB1','HSPA8','HSPB1','HSPE1','IER3','IER2','BTG1','BTG2','DUSP1']
stress_genes = [x for x in stress_genes if x in adata.var_names]
print('Tissue dissociation stress associated genes:', stress_genes, '\n')

FoxProteins=['FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1']
chrY_genes=['SRY', 'RPS4Y1', 'ZFY', 'ZFY-AS1', 'LINC00278', 'PCDH11Y', 'TTTY2B', 'TBL1Y', 'PRKY', 'USP9Y', 'DDX3Y', 'UTY', 'TMSB4Y', 'NLGN4Y', 'TTTY14', 'KDM5D', 'TTTY10', 'EIF1AY', 'PRORY', 'LINC00266-4P']
MembraneProteins1=['AADAT','AAK1','ABCA12','ABCA7','ABCB1','ABCB11','ABCB4','ABCB6','ABCC1','ABCC3','ABCC4','ABCC5','ABCC6','ABCG2','ABHD3','ABI1','ABLIM3','AC004691.2','AC004922.1','AC005324.3','AC005726.1','AC008403.1','AC138696.1','ACIN1','ACKR3','ACLY','ACOT13','ACOT9','ACTN1','ACTR10','ACTR6','ADA','ADAM10','ADAM12','ADAM22','ADAM28','ADAM33','ADAMTS1','ADAMTS2','ADAMTSL4','ADAP1','ADCK5','ADD1','ADD2','ADGRB2','ADGRD1','ADGRG2','ADGRG7','ADGRL2','ADH1A','ADH1B','ADH1C','ADH7','ADK','ADORA1','ADRA1B','ADRM1','ADSS','AFAP1L2','AFDN','AFG1L','AGER','AGRN','AHNAK','AHNAK2','AIFM2','AKAP10','AKAP11','AKAP12','AKR1B10','AKR1B15','AL132780.3','AL139300.1','AL355315.1','AL358113.1','AL645922.1','ALDH3A1','ALK','ALOX15B','ALOXE3','ALPI','ALPP','ALPPL2','ALS2CR12','AMACR','AMDHD1','AMER1','AMER2','AMIGO2','AMOT','AMOTL1','AMOTL2','AMPH','ANK2','ANK3','ANKEF1','ANKFN1','ANKIB1','ANKLE2','ANKRA2','ANKRD13A','ANKRD35','ANKRD42','ANKRD45','ANKS1B','ANO1','ANO10','ANO2','ANO6','ANPEP','ANXA1','ANXA13','ANXA2','ANXA3','AP2M1','APBB1','APBB1IP','APC','APLP1','APMAP','APOC3','APOD','APOLD1','AQP1','AQP10','AQP3','AQP4','AQP5','ARAP1','ARF1','ARHGAP18','ARHGAP19','ARHGAP21','ARHGAP22','ARHGAP23','ARHGAP24','ARHGAP28','ARHGAP29','ARHGAP33','ARHGAP36','ARHGAP40','ARHGAP9','ARHGDIG','ARHGEF1','ARHGEF11','ARHGEF12','ARHGEF15','ARHGEF26','ARHGEF28','ARHGEF40','ARHGEF5','ARID2','ARID4A','ARID5B','ARL13A','ARL15','ARL4C','ARMC12','ARMCX5','ARPC1A','ARPC5','ARRB2','ARRDC4','ARVCF','ASAP1','ASB5','ASGR1','ASGR2','ASH2L','ASIC1','ASIC4','ASPM','ASPSCR1','ATG13','ATG3','ATG7','ATIC','ATOX1','ATP10D','ATP12A','ATP13A3','ATP1B3','ATP23','ATP2B1','ATP2B4','ATP2C2','ATP6V0A2','ATP6V1H','ATP8A2','ATP8B1','ATXN10','ATXN3','ATXN7L3','AVEN','AVPI1','AXIN2','AXL','B2M','B3GALT2','BACE1','BAG3','BAIAP2','BAIAP2L1','BANK1','BASP1','BBOF1','BCAR1','BCAR3','BCL2L12','BCORL1','BCR','BDKRB2','BEND6','BFSP1','BFSP2','BICD2','BIN2','BLNK','BLVRB','BMPR2','BMX','BOC','BOLL','BORCS5','BORCS8-MEF2B','BRK1','BRPF1','BRSK1','BSDC1','BST2','BTK','BTN2A1','BTRC','BUB1B-PAK6','BVES','BZW2','C10orf90','C11orf42','C11orf52','C12orf73','C12orf76','C14orf37','C14orf93','C16orf74','C17orf58','C17orf62','C18orf21','C19orf33','C19orf38','C19orf48','C19orf71','C1orf116','C1orf189','C1orf35','C1orf54','C1orf68','C1orf74','C1QBP','C2orf72','C3orf14','C3orf20','C3orf38','C4orf19','C4orf22','C7orf43','C8orf33','C8orf37','C9orf153','C9orf3','CA14','CA9','CABLES2','CACNA1E','CACNG4','CADM1','CALCRL','CALD1','CALML5','CALN1','CAMK2A','CAMK2B','CAMK2D','CAMK2G','CAP2','CAPS','CASKIN2','CASP4','CAST','CAV3','CAVIN1','CAVIN2','CAVIN3','CAVIN4','CBWD1','CBWD2','CBWD3','CBWD5','CBWD6','CC2D1A','CCBE1','CCDC107','CCDC112','CCDC115','CCDC124','CCDC136','CCDC144NL','CCDC15','CCDC154','CCDC170','CCDC24','CCDC28A','CCDC59','CCDC62','CCDC66','CCDC69','CCDC8','CCDC83','CCDC85C','CCDC93','CCDC97','CCNT2','CCNYL1','CCR3','CCR4','CCT3','CD109','CD163','CD164L2','CD200R1','CD226','CD248','CD274','CD2AP','CD300A','CD300LB','CD33','CD34','CD37','CD38','CD3E','CD3G','CD4','CD44','CD46','CD47','CD48','CD79B','CD81','CD84','CD86','CD8A','CD8B','CD9','CD93','CD99L2','CDAN1','CDC42EP1','CDC42EP3','CDC42EP4','CDC42SE1','CDCA4','CDH1','CDH13','CDH15','CDH17','CDH2','CDH26','CDH3','CDH4','CDH5','CDH6','CDH8','CDIPT','CDK16','CDK5','CDK5RAP2','CDK7','CDRT15','CDV3','CEACAM5','CELSR1','CENPJ','CEP162','CEP295','CEP55','CEP57L1','CEP89','CERCAM','CFAP157','CFAP206','CFAP58','CFAP65','CFB','CFL2','CFLAR','CGN','CGNL1','CHCHD7','CHIC2','CHMP2A','CHRM3','CHRNA1','CHRNA5','CHRNB4','CHRND','CIB1','CIB3','CIP2A','CKAP5','CLCA2','CLCA4','CLCN5','CLCN6','CLDN11','CLDN18','CLDN2','CLDN23','CLDN3','CLDN4','CLDN7','CLDN9','CLEC2A','CLEC4F','CLEC5A','CLECL1','CLIC2','CLIC5','CLTB','CLUL1','CMTM4','CMTM7','CMTR2','CMYA5','CNGA1','CNKSR2','CNNM1','CNNM4','CNP','CNR1','CNTD1','COBL','COBLL1','COG3','COL16A1','COL17A1','COL4A5','COL6A2','COLEC12','COLQ','COMMD4','COMTD1','COPRS','COQ7','CORIN','CORO1B','CORO2A','CORO2B','COX6B2','CPEB3','CPPED1','CPZ','CRB3','CREB5','CRIP1','CRIP2','CRK','CRLF3','CROCC','CRTC1','CRYAB','CRYBG3','CRYL1','CSDE1','CSF1','CSF1R','CSPG4','CST6','CTNNA1','CTNNB1','CTNNBIP1','CTNND1','CTNND2','CTSF','CTSV','CTTN','CTXN1','CUEDC1','CUTA','CX3CL1','CX3CR1','CXADR','CXCR2','CYB5D2','CYBRD1','CYFIP2','CYP20A1','CYP24A1','CYTH2','CYTH4','DAAM1','DAB2','DAG1','DAGLB','DAW1','DBN1','DCAF13','DCAF16','DCBLD2','DCHS1','DCHS2','DCP2','DCT','DCTN6','DCUN1D2','DDAH1','DDB2','DDIAS','DDIT4L','DDR1','DDR2','DDX58','DDX60L','DENND6B','DFFA','DGKG','DGKH','DHDDS','DHPS','DHRS7B','DIAPH1','DIAPH3','DIO2','DIS3L','DLG1','DLG2','DLG5','DLGAP3','DLL3','DMWD','DNAAF1','DNAAF4','DNAH17','DNAJA4','DNAJB13','DNAJB4','DNAJC18','DNAJC5','DNAJC6','DNAJC9','DOC2A','DOCK4','DOK3','DONSON','DPEP1','DPH1','DPPA5','DPYSL2','DQX1','DRD1','DRP2','DSC3','DSEL','DSG2','DSP','DTNA','DUS1L','DUSP15','DXO','DYSF','ECSCR','EDA2R','EDNRA','EDNRB','EEF1AKMT1','EEF2','EFCAB1','EFCAB11','EFCAB2','EFCAB7','EFHC2','EFNB1','EFR3A','EFR3B','EGFR','EHBP1','EHD1','EHD2','EHD3','EHD4','EIF2B2','EIF5','EIF5B','ELFN1','ELMO3','ELN','ELOC','EMP3','ENAH','ENC1','ENG','ENKD1','ENO1','ENO2','ENO3','ENOX1','ENOX2','EPB41','EPB41L1','EPB41L2','EPB41L3','EPB41L4A','EPB41L4B','EPB41L5','EPCAM','EPHA2','EPHA3','EPHB1','EPHB2','EPN1','EPOR','EPS8L2','ERAP1','ERBB2','ERBB3','ERBIN','ERC1','ERCC4','ERMN','ESCO2','ETNK1','ETS2','EVA1A','EVI2B','EXOC1','EXOC3L1','EXOC7','EZR','F11R','F2RL1','F2RL3','FAAP20','FABP2','FABP3','FABP5','FAIM','FAM126A','FAM126B','FAM129A','FAM129B','FAM13A','FAM163B','FAM171A1','FAM193A','FAM205A','FAM210B','FAM222A','FAM45A','FAM47E','FAM47E-STBD1','FAM72A','FAM72B','FAM72C','FAM72D','FANCG','FAS','FASN','FBLIM1','FBLN2','FBLN5','FBLN7','FBP2','FBRSL1','FBXL2','FBXO16','FBXO38','FBXO41','FBXW11','FCAR','FCGBP','FCGR1A','FCGR1B','FCGR2A','FCHSD2','FFAR4','FGD1','FGD5','FGFR2','FGR','FHIT','FHL1','FIBCD1','FIGN','FILIP1','FILIP1L','FKBP2','FLAD1','FLCN','FLNA','FLNB','FLNC','FLOT1','FLOT2','FLRT3','FLT1','FLT4','FLVCR1','FMN2','FMNL2','FMNL3','FNBP1','FOXA2','FOXN3','FP565260.3','FRMD3','FRMD7','FRMD8','FRS2','FSCN1','FTCD','FUT2','FXYD3','FYB1','FYN','FZD1','FZD2','FZD4','FZD6','GAB1','GAB2','GABRA1','GABRA2','GABRA3','GABRA5','GABRG1','GAD1','GADL1','GAGE1','GAGE10','GAGE12C','GAGE12D','GAGE12E','GAGE12F','GAGE12G','GAGE12H','GAGE12J','GAGE13','GAGE2A','GAGE2E','GALNT10','GALR1','GAP43','GAPDH','GAPT','GAPVD1','GAS2','GAS2L1','GAS7','GAS8','GBP4','GCA','GCC1','GCLM','GCSAML-AS1','GFI1B','GFOD1','GFRA3','GHR','GID8','GINM1','GIPC1','GJA1','GJB3','GJB4','GJB6','GJB7','GJC1','GJC2','GLO1','GLP2R','GLRX','GLUL','GMIP','GNAI2','GNAQ','GNAS','GNB1','GNB2','GNB3','GNB4','GNG2','GP9','GPAT3','GPATCH1','GPBP1','GPBP1L1','GPC1','GPC3','GPC4','GPI','GPR1','GPR142','GPR156','GPR157','GPR160','GPR173','GPR20','GPR27','GPR50','GPR63','GPRC5A','GPRIN1','GPRIN2','GPX8','GRAMD1A','GRAMD1B','GRAMD1C','GRAP2','GRB7','GRID2IP','GRIK5','GRK3','GRK5','GRPR','GSDMA','GTF2F1','GTSE1','GUCA1A','GYPA','GYPC','HAPLN3','HARBI1','HAS1','HAUS4','HAUS7','HCAR2','HCAR3','HCFC2','HCK','HCLS1','HCST','HDAC8','HEG1','HELZ','HENMT1','HERC2','HERPUD1','HFE','HIF1A','HIF3A','HINT1','HIP1R','HLA-A','HLA-DPB1','HMGCS1','HMOX1','HOMER3','HOXC5','HOXD4','HPS3','HRASLS5','HRH1','HS6ST3','HSH2D','HSPB1','HSPG2','HTR1B','HTR7','HTRA1','HUNK','HYDIN','HYI','HYLS1','ICAM1','ICAM2','ICOS','ICOSLG','IFFO2','IFIT5','IFNGR1','IFRD1','IFT122','IGFL2','IGSF11','IGSF23','IKZF5','IL10RA','IL12RB1','IL12RB2','IL13','IL13RA2','IL16','IL17RB','IL1R1','IL1RL1','IL21R','IL4R','IL6ST','IL7R','ILDR2','ILK','INPP5A','INSR','INTS12','IP6K2','IPCEF1','IQGAP1','IQGAP2','IREB2','IRF2','IRS1','ISLR','ITGA2B','ITGA3','ITGA4','ITGA7','ITGA8','ITGA9','ITGB1','ITGB2','ITGB3','ITGB4','ITGB5','ITGB6','ITGB7','ITGB8','ITK','ITM2A','ITPKA','ITSN1','ITSN2','IYD','JAG1','JAK2','JAK3','JAM2','JAML','JCAD','JMJD4','JPT2','JUP','KANK1','KANK3','KANSL2','KARS','KATNB1','KCNAB1','KCNB2','KCNH8','KCNJ15','KCNK13','KCNK17','KCNK6','KCNK7','KCNK9','KCNN2','KCNS3','KCNU1','KDF1','KDR','KEL','KIAA0556','KIAA0895','KIAA1324','KIAA1522','KIAA1958','KIF17','KIF19','KIF21A','KIF21B','KIF26A','KIF26B','KISS1R','KIT','KLC1','KLC2','KLF9','KLHDC7B','KLHL41','KLHL6','KLHL7','KLK7','KLRC1','KLRC2','KLRF1','KLRK1','KMT2D','KNDC1','KNSTRN','KNTC1','KPTN','KRT75','KRT77','KRTAP25-1','L1CAM','L3MBTL1','LAIR1','LAMA4','LAMC1','LAMP2','LAMTOR1','LAPTM4A','LASP1','LAT','LAT2','LAX1','LCP1','LCTL','LDB2','LDLRAD3','LENG1','LGALS3','LGALS4','LGALS8','LGR6','LHFPL2','LHX6','LIMA1','LINGO1','LIPJ','LMCD1','LMOD2','LMTK2','LMTK3','LNX1','LNX2','LONRF1','LONRF3','LOXHD1','LPAR1','LPAR3','LPAR4','LPIN1','LPIN2','LRCH2','LRP11','LRP12','LRRC15','LRRC26','LRRC34','LRRC45','LRRC69','LRRC7','LRRC9','LRRFIP2','LRRIQ3','LSMEM2','LSR','LTB4R2','LUZP2','LY6K','LYN','LYPD1','LYST','LZTS1','LZTS2','MACF1','MADD','MAGI1','MAGI3','MAMDC2','MAP1S','MAP2','MAP2K1','MAP2K4','MAP2K7','MAP3K12','MAP3K21','MAP3K6','MAP3K7','MAP4','MAP4K1','MAP4K5','MAP6','MAPK15','MAPK8IP1','MAPT','MARCH10','MARCH7','MARCKS','MARK2','MARVELD2','MAS1L','MAT2B','MBP','MC5R','MCAM','MCEMP1','MCF2L2','MCMBP','MCTS1','MDGA1','ME1','MEF2B','MEIOC','MEPCE','MERTK','MET','METAP1','METAP2','METTL15','METTL25','METTL9','MFAP3L','MFSD1','MFSD2A','MGAT4C','MIB1','MICAL2','MICAL3','MICALL1','MICB','MIEN1','MIGA2','MISP','MKLN1','MKRN3','MLF2','MLKL','MLST8','MMACHC','MMP10','MMP15','MMP24','MMP25','MOAP1','MPHOSPH8','MPHOSPH9','MPI','MPIG6B','MPL','MPP1','MPP5','MPP7','MPZL1','MPZL2','MR1','MRGPRF','MRNIP','MRPL42','MRPL44','MRPL46','MRPS18B','MS4A1','MS4A12','MS4A14','MS4A15','MSN','MSRA','MTCP1','MTFR1L','MTHFR','MTM1','MTPN','MTSS1','MUC1','MUC20','MVD','MYH11','MYH9','MYL4','MYLK','MYO10','MYO1B','MYO1C','MYO1E','MYO1G','MYO3A','MYO5B','MYO9A','MYOF','N4BP2','NAA35','NAB1','NAIF1','NAMPT','NAP1L2','NAPG','NARFL','NCAM1','NCAM2','NCK1','NCKAP1L','NCKAP5','NCKIPSD','NCOA1','NCS1','NDC1','NECTIN2','NECTIN3','NECTIN4','NEDD1','NEDD9','NEO1','NEU1','NEXN','NF2','NFE2L2','NFKBID','NGEF']
MembraneProteins2=['NGFR','NHS','NHSL2','NID2','NINJ2','NKTR','NKX2-8','NLGN1','NLGN3','NLK','NLRX1','NMB','NMBR','NME1','NME2','NMT1','NMT2','NMUR2','NOS1','NOTCH2','NPFFR2','NPL','NPNT','NPR1','NPTN','NPTXR','NR2C1','NRAS','NRCAM','NRP2','NRSN1','NRXN1','NSD1','NSFL1C','NT5E','NTNG1','NTNG2','NTSR1','NUDCD3','NUDT16L1','NUDT9','NUMB','NUP35','NUS1','OAS3','OBSCN','OCA2','OCLN','OCM','OCM2','ODC1','ODR4','OGT','OLFM4','OLIG2','OLR1','OPHN1','OPRK1','OR10P1','OR1N2','OR2C3','OR6B2','OR6B3','OR6C6','OR8S1','OSBPL10','OSBPL6','OSBPL7','OSCAR','OSCP1','OSGEP','OSR2','OTOP2','OTULIN','OXGR1','P3H3','PACSIN2','PACSIN3','PAFAH1B2','PAG1','PAIP1','PAK1','PAK4','PAK6','PALLD','PALM','PALM2','PALM2-AKAP2','PALM3','PALMD','PANX1','PANX2','PAQR8','PARD3','PARD3B','PARD6A','PARD6G','PARP6','PATJ','PAWR','PCBD2','PCDH1','PCDH17','PCDH7','PCDHA1','PCDHA11','PCDHA12','PCDHA4','PCDHA5','PCDHB1','PCDHB11','PCDHB12','PCDHB16','PCDHB4','PCDHB5','PCDHGA1','PCDHGA10','PCDHGA11','PCDHGA12','PCDHGA2','PCDHGA3','PCDHGA4','PCDHGA5','PCDHGA6','PCDHGA7','PCDHGA8','PCDHGA9','PCDHGB1','PCDHGB2','PCDHGB3','PCDHGB4','PCDHGB5','PCDHGB6','PCDHGB7','PCDHGC3','PCDHGC4','PCDHGC5','PCLO','PCMTD1','PDAP1','PDCD7','PDE3A','PDE4A','PDE4D','PDE9A','PDGFC','PDGFRA','PDHX','PDLIM1','PDLIM5','PDPN','PDZD2','PDZD8','PEAR1','PEBP1','PFDN1','PGAP3','PGLYRP2','PGLYRP4','PGRMC2','PHACTR1','PHACTR4','PHF20L1','PHGDH','PHLDB2','PHLDB3','PHOSPHO1','PHPT1','PI3','PI4K2A','PI4KA','PIEZO2','PIGB','PIGN','PIGW','PIGX','PIK3AP1','PIK3CG','PIK3R6','PIP4K2A','PIP4K2C','PIP5K1A','PIRT','PITPNM3','PKD2L2','PKN1','PKN2','PKP1','PKP2','PKP3','PKP4','PLAA','PLAUR','PLCB1','PLCB2','PLCD3','PLCD4','PLEKHA4','PLEKHA6','PLEKHA7','PLEKHG3','PLEKHG4','PLEKHG6','PLEKHH2','PLEKHO2','PLIN4','PLPP1','PLPPR5','PLS3','PLSCR1','PLVAP','PLXNA3','PLXNA4','PNCK','PNISR','PODXL','POLE','POU2F3','PPEF1','PPFIBP1','PPIP5K1','PPL','PPM1A','PPM1F','PPP1CA','PPP1CB','PPP1R12B','PPP1R16A','PPP1R16B','PPP1R18','PPP1R27','PPP1R9A','PPP2R1B','PPP3R1','PPP3R2','PPP4C','PPP6R3','PPTC7','PRAME','PRC1','PRDX6','PRIMA1','PRKCA','PRKCE','PRKCH','PRKCZ','PRKD1','PROM2','PROSER2','PRR15','PRR32','PRR7','PRRC2A','PRRG1','PRRT2','PRRT3','PRRT4','PRTG','PRX','PSCA','PSD','PSD2','PSEN1','PSMC5','PSMC6','PSMD9','PSTPIP2','PTAFR','PTCHD4','PTK2B','PTK6','PTPN12','PTPN21','PTPN22','PTPN4','PTPRB','PTPRG','PTPRK','PTPRM','PTPRR','PTPRS','PTX3','PURG','PVALB','PVR','PXDC1','PXK','PYGL','PYGO2','PYM1','RAB13','RAB23','RAB25','RAB26','RAB27B','RAB3B','RAB3GAP2','RAB43','RAB8A','RABEPK','RAD51C','RAD52','RAI2','RALA','RALGAPA2','RAPGEF2','RAPH1','RAPSN','RASA4','RASA4B','RASAL1','RASAL2','RASAL3','RASD1','RASD2','RASGRP1','RASGRP2','RASGRP4','RASSF2','RASSF3','RASSF4','RBM46','RC3H1','RCCD1','RCSD1','RD3L','RDX','RECK','RELL1','REPS1','RET','RFLNB','RFPL1','RFPL2','RFPL3','RGP1','RGS10','RGS14','RGS19','RGS3','RGS7BP','RGS9BP','RHCE','RHD','RHEX','RHOA','RHOBTB2','RHOJ','RHOXF2','RHOXF2B','RIC8A','RIC8B','RIF1','RILPL1','RIMBP3','RIMBP3B','RIMBP3C','RIMS2','RIOK2','RIOK3','RIPK1','RND1','RNF114','RNF145','RNF146','RNF149','RNF165','RNF166','RNF39','ROBO1','ROBO3','ROBO4','ROM1','RP2','RPE','RPEL1','RPGRIP1L','RPIA','RPL36A','RPL36A-HNRNPH2','RPL36AL','RPSA','RPUSD1','RRAD','RRP12','RRP7A','RSBN1L','RSC1A1','RSPH3','RTL3','RTL4','RTN4R','RTN4RL1','RYR2','S100A13','S100A14','S100A16','S100A2','S100A3','S100A4','S100A6','S100A9','SAMD4A','SAMHD1','SAMSN1','SAPCD1','SASH3','SC5D','SCAMP1','SCAMP4','SCEL','SCHIP1','SCIN','SCN1A','SCN5A','SCN8A','SCN9A','SCNN1D','SCNN1G','SCRIB','SCUBE3','SDC2','SDC4','SDE2','SDHB','SEC16B','SEMA3C','SEMA3D','SEMA4B','SEMA4F','SEMA6C','SEMA6D','SEPT2','SEPT3','SERINC2','SERINC5','SERPINB3','SFSWAP','SGCE','SGK494','SH3BGR','SH3D21','SHANK1','SHANK3','SHARPIN','SHISA6','SHKBP1','SHROOM2','SHTN1','SIGLEC10','SIGLEC14','SIGLEC6','SIGLEC7','SIN3B','SIPA1','SIPA1L1','SIPA1L3','SKAP1','SLAMF6','SLC10A5','SLC12A2','SLC13A3','SLC13A5','SLC14A1','SLC16A1','SLC16A10','SLC16A2','SLC16A3','SLC16A7','SLC16A9','SLC17A5','SLC19A1','SLC19A3','SLC1A5','SLC1A6','SLC22A16','SLC23A3','SLC24A4','SLC26A9','SLC27A1','SLC28A1','SLC2A1','SLC2A11','SLC2A12','SLC2A14','SLC2A2','SLC2A3','SLC30A1','SLC31A2','SLC34A1','SLC34A2','SLC35F5','SLC35G2','SLC36A4','SLC38A5','SLC38A6','SLC38A7','SLC39A10','SLC39A14','SLC39A2','SLC39A6','SLC3A2','SLC40A1','SLC41A3','SLC43A1','SLC43A3','SLC44A2','SLC45A4','SLC46A1','SLC46A3','SLC4A1AP','SLC4A2','SLC4A7','SLC4A8','SLC5A12','SLC5A3','SLC5A6','SLC5A7','SLC6A6','SLC7A14','SLC7A2','SLC7A5','SLC7A6','SLC8A1','SLC9A1','SLC9A2','SLC9A3','SLC9A3R1','SLC9A3R2','SLC9A4','SLC9A5','SLC9B2','SLC9C2','SLCO1A2','SLCO1B3','SLCO4A1','SLCO4C1','SLCO5A1','SLIT1','SLIT3','SLITRK6','SLK','SLX4','SMAGP','SMAP1','SMOC1','SMPD2','SMPX','SNAP23','SNAP25','SNRK','SNTB2','SNTG2','SNX15','SNX32','SNX9','SOCS3','SOCS5','SOD1','SORBS1','SORCS1','SORT1','SOX11','SPAG16','SPANXN4','SPART','SPATA12','SPATC1L','SPERT','SPG11','SPHK1','SPINK13','SPINT1','SPINT2','SPN','SPPL2B','SPPL3','SPRED3','SPRYD3','SPTBN2','SRC','SRCIN1','SRD5A3','SRPK1','SRPRB','SRRM3','SSFA2','SSH1','SSH3','SSTR1','SSTR5','SSX2IP','ST8SIA5','STAB1','STAC','STAC2','STAMBP','STAMBPL1','STARD4','STAT2','STEAP4','STIL','STIP1','STK10','STK16','STK17A','STK17B','STOM','STOML2','STOML3','STX18','STX4','STXBP4','STYK1','SUCLG1','SUCLG2','SUGT1','SUMF2','SUPT6H','SUSD2','SVIL','SWAP70','SYCP2','SYK','SYMPK','SYNC','SYNGR3','SYNPO2L','SYPL2','SYT4','SYTL2','SYVN1','TACC2','TACR2','TACSTD2','TASP1','TAZ','TBC1D10A','TBC1D2','TBC1D24','TBC1D30','TBC1D9B','TBXA2R','TCAF2','TCAF2C','TCHP','TCIM','TDG','TDO2','TDP1','TEC','TEDC2','TEK','TES','TEX14','TEX9','TGFBR1','TGFBR2','TGM1','TGM2','THBS1','THBS2','THEGL','THY1','TIAM1','TICAM1','TIGD7','TIMD4','TJP1','TJP2','TJP3','TLN1','TLR4','TM2D2','TM4SF20','TM4SF4','TM4SF5','TMC5','TMED8','TMEM101','TMEM107','TMEM108','TMEM121B','TMEM139','TMEM141','TMEM164','TMEM176B','TMEM183A','TMEM186','TMEM2','TMEM218','TMEM222','TMEM229A','TMEM245','TMEM251','TMEM254','TMEM258','TMEM266','TMEM268','TMEM50A','TMEM63B','TMPRSS15','TMPRSS2','TMTC2','TNFRSF10D','TNFRSF11A','TNFRSF12A','TNFRSF21','TNFRSF25','TNFRSF8','TNFRSF9','TNFSF13B','TNK1','TNKS1BP1','TNNI2','TOGARAM2','TOLLIP','TOM1','TP53BP2','TPD52L1','TPRN','TRAF7','TRAIP','TRAPPC13','TRAT1','TRDN','TREML1','TRIB1','TRIM16','TRIM16L','TRIM38','TRIM4','TRIM44','TRIM54','TRIM71','TRIML2','TRIP6','TRMT1','TRMT10B','TRMT13','TRMT6','TRO','TRPM1','TRPM4','TRPM7','TRPV2','TRPV3','TSG101','TSGA13','TSHZ3','TSPAN15','TSPAN6','TSPO2','TSR2','TSSK4','TTC17','TTC23','TTC7B','TTLL12','TTLL5','TULP3','TWF2','TXNDC11','TXNIP','TYROBP','TYW3','UAP1','UBA3','UBA52','UBAC1','UBALD1','UBAP1','UBASH3B','UBE2A','UBE2B','UBE2C','UBE2D1','UBE2D2','UBE2D3','UBE2D4','UBE2K','UBE2QL1','UBE2S','UBE3C','UBQLN2','UBR2','UBTD1','UCP3','ULBP1','ULBP2','UNC5A','UNC79','UQCC1','USP14','USP15','USP2','USP20','USP21','USP4','USP53','UTP20','UTRN','VAMP5','VAMP8','VANGL1','VASP','VCAM1','VCPIP1','VIL1','VPS13B','VPS36','VPS50','VPS8','VSIG4','VTCN1','VWA5B1','VWA7','WAS','WASF2','WBP4','WDFY3','WDPCP','WDR1','WDR12','WDR13','WDR53','WDR6','WDR66','WIPF1','WWP1','XIAP','XIRP1','XIRP2','XPC','XPNPEP2','XPO6','XRN1','XYLT1','YES1','YIPF1','YIPF4','YOD1','YTHDC1','ZBED3','ZBTB33','ZBTB8OS','ZDHHC2','ZDHHC20','ZDHHC22','ZDHHC5','ZFP41','ZFP90','ZFYVE28','ZGPAT','ZMYND10','ZNF10','ZNF18','ZNF185','ZNF2','ZNF20','ZNF227','ZNF229','ZNF236','ZNF276','ZNF277','ZNF337','ZNF397','ZNF407','ZNF497','ZNF503','ZNF510','ZNF536','ZNF571','ZNF598','ZNF599','ZNF646','ZNF655','ZNF688','ZNF706','ZNF717','ZNF783','ZNF79','ZNF839','ZSCAN22','ZSCAN29','ZSWIM1','ZSWIM8','ZYX']
MembraneProteins=MembraneProteins1+MembraneProteins2

TFs1=['AC008770.3','AC023509.3','AC092835.1','AC138696.1','ADNP','ADNP2','AEBP1','AEBP2','AHCTF1','AHDC1','AHR','AHRR','AIRE','AKAP8','AKAP8L','AKNA','ALX1','ALX3','ALX4','ANHX','ANKZF1','AR','ARGFX','ARHGAP35','ARID2','ARID3A','ARID3B','ARID3C','ARID5A','ARID5B','ARNT','ARNT2','ARNTL','ARNTL2','ARX','ASCL1','ASCL2','ASCL3','ASCL4','ASCL5','ASH1L','ATF1','ATF2','ATF3','ATF4','ATF5','ATF6','ATF6B','ATF7','ATMIN','ATOH1','ATOH7','ATOH8','BACH1','BACH2','BARHL1','BARHL2','BARX1','BARX2','BATF','BATF2','BATF3','BAZ2A','BAZ2B','BBX','BCL11A','BCL11B','BCL6','BCL6B','BHLHA15','BHLHA9','BHLHE22','BHLHE23','BHLHE40','BHLHE41','BNC1','BNC2','BORCS8-MEF2B','BPTF','BRF2','BSX','C11orf95','CAMTA1','CAMTA2','CARF','CASZ1','CBX2','CC2D1A','CCDC169-SOHLH2','CCDC17','CDC5L','CDX1','CDX2','CDX4','CEBPA','CEBPB','CEBPD','CEBPE','CEBPG','CEBPZ','CENPA','CENPB','CENPBD1','CENPS','CENPT','CENPX','CGGBP1','CHAMP1','CHCHD3','CIC','CLOCK','CPEB1','CPXCR1','CREB1','CREB3','CREB3L1','CREB3L2','CREB3L3','CREB3L4','CREB5','CREBL2','CREBZF','CREM','CRX','CSRNP1','CSRNP2','CSRNP3','CTCF','CTCFL','CUX1','CUX2','CXXC1','CXXC4','CXXC5','DACH1','DACH2','DBP','DBX1','DBX2','DDIT3','DEAF1','DLX1','DLX2','DLX3','DLX4','DLX5','DLX6','DMBX1','DMRT1','DMRT2','DMRT3','DMRTA1','DMRTA2','DMRTB1','DMRTC2','DMTF1','DNMT1','DNTTIP1','DOT1L','DPF1','DPF3','DPRX','DR1','DRAP1','DRGX','DUX1','DUX3','DUX4','DUXA','DZIP1','E2F1','E2F2','E2F3','E2F4','E2F5','E2F6','E2F7','E2F8','E4F1','EBF1','EBF2','EBF3','EBF4','EEA1','EGR1','EGR2','EGR3','EGR4','EHF','ELF1','ELF2','ELF3','ELF4','ELF5','ELK1','ELK3','ELK4','EMX1','EMX2','EN1','EN2','EOMES','EPAS1','ERF','ERG','ESR1','ESR2','ESRRA','ESRRB','ESRRG','ESX1','ETS1','ETS2','ETV1','ETV2','ETV3','ETV3L','ETV4','ETV5','ETV6','ETV7','EVX1','EVX2','FAM170A','FAM200B','FBXL19','FERD3L','FEV','FEZF1','FEZF2','FIGLA','FIZ1','FLI1','FLYWCH1','FOS','FOSB','FOSL1','FOSL2','FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXN4','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1','GABPA','GATA1','GATA2','GATA3','GATA4','GATA5','GATA6','GATAD2A','GATAD2B','GBX1','GBX2','GCM1','GCM2','GFI1','GFI1B','GLI1','GLI2','GLI3','GLI4','GLIS1','GLIS2','GLIS3','GLMP','GLYR1','GMEB1','GMEB2','GPBP1','GPBP1L1','GRHL1','GRHL2','GRHL3','GSC','GSC2','GSX1','GSX2','GTF2B','GTF2I','GTF2IRD1','GTF2IRD2','GTF2IRD2B','GTF3A','GZF1','HAND1','HAND2','HBP1','HDX','HELT','HES1','HES2','HES3','HES4','HES5','HES6','HES7','HESX1','HEY1','HEY2','HEYL','HHEX','HIC1','HIC2','HIF1A','HIF3A','HINFP','HIVEP1','HIVEP2','HIVEP3','HKR1','HLF','HLX','HMBOX1','HMG20A','HMG20B','HMGA1','HMGA2','HMGN3','HMX1','HMX2','HMX3','HNF1A','HNF1B','HNF4A','HNF4G','HOMEZ','HOXA1','HOXA10','HOXA11','HOXA13','HOXA2','HOXA3','HOXA4','HOXA5','HOXA6','HOXA7','HOXA9','HOXB1','HOXB13','HOXB2','HOXB3','HOXB4','HOXB5','HOXB6','HOXB7','HOXB8','HOXB9','HOXC10','HOXC11','HOXC12','HOXC13','HOXC4','HOXC5','HOXC6','HOXC8','HOXC9','HOXD1','HOXD10','HOXD11','HOXD12','HOXD13','HOXD3','HOXD4','HOXD8','HOXD9','HSF1','HSF2','HSF4','HSF5','HSFX1','HSFX2','HSFY1','HSFY2','IKZF1','IKZF2','IKZF3','IKZF4','IKZF5','INSM1','INSM2','IRF1','IRF2','IRF3','IRF4','IRF5','IRF6','IRF7','IRF8','IRF9','IRX1','IRX2','IRX3','IRX4','IRX5','IRX6','ISL1','ISL2','ISX','JAZF1','JDP2','JRK','JRKL','JUN','JUNB','JUND','KAT7','KCMF1','KCNIP3','KDM2A','KDM2B','KDM5B','KIN','KLF1','KLF10','KLF11','KLF12','KLF13','KLF14','KLF15','KLF16','KLF17','KLF2','KLF3','KLF4','KLF5','KLF6','KLF7','KLF8','KLF9','KMT2A','KMT2B','L3MBTL1','L3MBTL3','L3MBTL4','LBX1','LBX2','LCOR','LCORL','LEF1','LEUTX','LHX1','LHX2','LHX3','LHX4','LHX5','LHX6','LHX8','LHX9','LIN28A','LIN28B','LIN54','LMX1A','LMX1B','LTF','LYL1','MAF','MAFA','MAFB','MAFF','MAFG','MAFK','MAX','MAZ','MBD1','MBD2','MBD3','MBD4','MBD6','MBNL2','MECOM','MECP2','MEF2A','MEF2B','MEF2C','MEF2D','MEIS1','MEIS2','MEIS3','MEOX1','MEOX2','MESP1','MESP2','MGA','MITF','MIXL1','MKX','MLX','MLXIP','MLXIPL','MNT','MNX1','MSANTD1','MSANTD3','MSANTD4','MSC','MSGN1','MSX1','MSX2','MTERF1','MTERF2','MTERF3','MTERF4','MTF1','MTF2','MXD1','MXD3','MXD4','MXI1','MYB','MYBL1','MYBL2','MYC','MYCL','MYCN','MYF5','MYF6','MYNN','MYOD1','MYOG','MYPOP','MYRF','MYRFL','MYSM1','MYT1','MYT1L','MZF1','NACC2','NAIF1','NANOG','NANOGNB','NANOGP8','NCOA1','NCOA2','NCOA3','NEUROD1','NEUROD2','NEUROD4','NEUROD6','NEUROG1','NEUROG2','NEUROG3','NFAT5','NFATC1','NFATC2','NFATC3','NFATC4','NFE2','NFE2L1','NFE2L2','NFE2L3','NFE4','NFIA','NFIB','NFIC','NFIL3','NFIX','NFKB1','NFKB2','NFX1','NFXL1','NFYA','NFYB','NFYC','NHLH1','NHLH2','NKRF','NKX1-1','NKX1-2','NKX2-1','NKX2-2','NKX2-3','NKX2-4','NKX2-5','NKX2-6','NKX2-8','NKX3-1','NKX3-2','NKX6-1','NKX6-2','NKX6-3','NME2','NOBOX','NOTO','NPAS1','NPAS2','NPAS3','NPAS4','NR0B1','NR1D1','NR1D2','NR1H2','NR1H3','NR1H4','NR1I2','NR1I3','NR2C1','NR2C2','NR2E1','NR2E3','NR2F1','NR2F2','NR2F6','NR3C1','NR3C2','NR4A1','NR4A2','NR4A3','NR5A1','NR5A2','NR6A1','NRF1','NRL','OLIG1','OLIG2','OLIG3','ONECUT1','ONECUT2','ONECUT3','OSR1','OSR2','OTP','OTX1','OTX2','OVOL1','OVOL2','OVOL3','PA2G4','PATZ1','PAX1','PAX2','PAX3','PAX4','PAX5','PAX6','PAX7','PAX8','PAX9','PBX1','PBX2','PBX3','PBX4','PCGF2','PCGF6','PDX1','PEG3','PGR','PHF1','PHF19 ','PHF20','PHF21A','PHOX2A','PHOX2B','PIN1','PITX1','PITX2','PITX3','PKNOX1','PKNOX2','PLAG1','PLAGL1','PLAGL2','PLSCR1','POGK','POU1F1','POU2AF1','POU2F1','POU2F2','POU2F3','POU3F1','POU3F2','POU3F3','POU3F4','POU4F1','POU4F2','POU4F3','POU5F1','POU5F1B','POU5F2','POU6F1','POU6F2','PPARA','PPARD','PPARG','PRDM1','PRDM10','PRDM12','PRDM13','PRDM14','PRDM15','PRDM16','PRDM2','PRDM4','PRDM5','PRDM6','PRDM8','PRDM9','PREB','PRMT3','PROP1','PROX1','PROX2','PRR12','PRRX1','PRRX2','PTF1A','PURA','PURB','PURG','RAG1','RARA','RARB','RARG','RAX','RAX2','RBAK','RBCK1','RBPJ','RBPJL','RBSN','REL','RELA','RELB','REPIN1','REST','REXO4','RFX1','RFX2','RFX3','RFX4','RFX5','RFX6','RFX7','RFX8','RHOXF1','RHOXF2','RHOXF2B','RLF','RORA','RORB','RORC','RREB1','RUNX1','RUNX2','RUNX3','RXRA','RXRB','RXRG','SAFB','SAFB2','SALL1','SALL2','SALL3','SALL4','SATB1','SATB2','SCMH1','SCML4','SCRT1','SCRT2','SCX','SEBOX','SETBP1','SETDB1','SETDB2','SGSM2','SHOX','SHOX2','SIM1','SIM2','SIX1','SIX2','SIX3','SIX4','SIX5','SIX6','SKI','SKIL','SKOR1','SKOR2','SLC2A4RG','SMAD1','SMAD3','SMAD4','SMAD5','SMAD9','SMYD3','SNAI1','SNAI2','SNAI3','SNAPC2','SNAPC4','SNAPC5','SOHLH1','SOHLH2','SON','SOX1','SOX10','SOX11']
TFs2=['SOX12','SOX13','SOX14','SOX15','SOX17','SOX18','SOX2','SOX21','SOX3','SOX30','SOX4','SOX5','SOX6','SOX7','SOX8','SOX9','SP1','SP100','SP110','SP140','SP140L','SP2','SP3','SP4','SP5','SP6','SP7','SP8','SP9','SPDEF','SPEN','SPI1','SPIB','SPIC','SPZ1','SRCAP','SREBF1','SREBF2','SRF','SRY','ST18','STAT1','STAT2','STAT3','STAT4','STAT5A','STAT5B','STAT6','T','TAL1','TAL2','TBP','TBPL1','TBPL2','TBR1','TBX1','TBX10','TBX15','TBX18','TBX19','TBX2','TBX20','TBX21','TBX22','TBX3','TBX4','TBX5','TBX6','TCF12','TCF15','TCF20','TCF21','TCF23','TCF24','TCF3','TCF4','TCF7','TCF7L1','TCF7L2','TCFL5','TEAD1','TEAD2','TEAD3','TEAD4','TEF','TERB1','TERF1','TERF2','TET1','TET2','TET3','TFAP2A','TFAP2B','TFAP2C','TFAP2D','TFAP2E','TFAP4','TFCP2','TFCP2L1','TFDP1','TFDP2','TFDP3','TFE3','TFEB','TFEC','TGIF1','TGIF2','TGIF2LX','TGIF2LY','THAP1','THAP10','THAP11','THAP12','THAP2','THAP3','THAP4','THAP5','THAP6','THAP7','THAP8','THAP9','THRA','THRB','THYN1','TIGD1','TIGD2','TIGD3','TIGD4','TIGD5','TIGD6','TIGD7','TLX1','TLX2','TLX3','TMF1','TOPORS','TP53','TP63','TP73','TPRX1','TRAFD1','TRERF1','TRPS1','TSC22D1','TSHZ1','TSHZ2','TSHZ3','TTF1','TWIST1','TWIST2','UBP1','UNCX','USF1','USF2','USF3','VAX1','VAX2','VDR','VENTX','VEZF1','VSX1','VSX2','WIZ','WT1','XBP1','XPA','YBX1','YBX2','YBX3','YY1','YY2','ZBED1','ZBED2','ZBED3','ZBED4','ZBED5','ZBED6','ZBED9','ZBTB1','ZBTB10','ZBTB11','ZBTB12','ZBTB14','ZBTB16','ZBTB17','ZBTB18','ZBTB2','ZBTB20','ZBTB21','ZBTB22','ZBTB24','ZBTB25','ZBTB26','ZBTB3','ZBTB32','ZBTB33','ZBTB34','ZBTB37','ZBTB38','ZBTB39','ZBTB4','ZBTB40','ZBTB41','ZBTB42','ZBTB43','ZBTB44','ZBTB45','ZBTB46','ZBTB47','ZBTB48','ZBTB49','ZBTB5','ZBTB6','ZBTB7A','ZBTB7B','ZBTB7C','ZBTB8A','ZBTB8B','ZBTB9','ZC3H8','ZEB1','ZEB2','ZFAT','ZFHX2','ZFHX3','ZFHX4','ZFP1','ZFP14','ZFP2','ZFP28','ZFP3','ZFP30','ZFP37','ZFP41','ZFP42','ZFP57','ZFP62','ZFP64','ZFP69','ZFP69B','ZFP82','ZFP90','ZFP91','ZFP92','ZFPM1','ZFPM2','ZFX','ZFY','ZGLP1','ZGPAT','ZHX1','ZHX2','ZHX3','ZIC1','ZIC2','ZIC3','ZIC4','ZIC5','ZIK1','ZIM2','ZIM3','ZKSCAN1','ZKSCAN2','ZKSCAN3','ZKSCAN4','ZKSCAN5','ZKSCAN7','ZKSCAN8','ZMAT1','ZMAT4','ZNF10','ZNF100','ZNF101','ZNF107','ZNF112','ZNF114','ZNF117','ZNF12','ZNF121','ZNF124','ZNF131','ZNF132','ZNF133','ZNF134','ZNF135','ZNF136','ZNF138','ZNF14','ZNF140','ZNF141','ZNF142','ZNF143','ZNF146','ZNF148','ZNF154','ZNF155','ZNF157','ZNF16','ZNF160','ZNF165','ZNF169','ZNF17','ZNF174','ZNF175','ZNF177','ZNF18','ZNF180','ZNF181','ZNF182','ZNF184','ZNF189','ZNF19','ZNF195','ZNF197','ZNF2','ZNF20','ZNF200','ZNF202','ZNF205','ZNF207','ZNF208','ZNF211','ZNF212','ZNF213','ZNF214','ZNF215','ZNF217','ZNF219','ZNF22','ZNF221','ZNF222','ZNF223','ZNF224','ZNF225','ZNF226','ZNF227','ZNF229','ZNF23','ZNF230','ZNF232','ZNF233','ZNF234','ZNF235','ZNF236','ZNF239','ZNF24','ZNF248','ZNF25','ZNF250','ZNF251','ZNF253','ZNF254','ZNF256','ZNF257','ZNF26','ZNF260','ZNF263','ZNF264','ZNF266','ZNF267','ZNF268','ZNF273','ZNF274','ZNF275','ZNF276','ZNF277','ZNF28','ZNF280A','ZNF280B','ZNF280C','ZNF280D','ZNF281','ZNF282','ZNF283','ZNF284','ZNF285','ZNF286A','ZNF286B','ZNF287','ZNF292','ZNF296','ZNF3','ZNF30','ZNF300','ZNF302','ZNF304','ZNF311','ZNF316','ZNF317','ZNF318','ZNF319','ZNF32','ZNF320','ZNF322','ZNF324','ZNF324B','ZNF326','ZNF329','ZNF331','ZNF333','ZNF334','ZNF335','ZNF337','ZNF33A','ZNF33B','ZNF34','ZNF341','ZNF343','ZNF345','ZNF346','ZNF347','ZNF35','ZNF350','ZNF354A','ZNF354B','ZNF354C','ZNF358','ZNF362','ZNF365','ZNF366','ZNF367','ZNF37A','ZNF382','ZNF383','ZNF384','ZNF385A','ZNF385B','ZNF385C','ZNF385D','ZNF391','ZNF394','ZNF395','ZNF396','ZNF397','ZNF398','ZNF404','ZNF407','ZNF408','ZNF41','ZNF410','ZNF414','ZNF415','ZNF416','ZNF417','ZNF418','ZNF419','ZNF420','ZNF423','ZNF425','ZNF426','ZNF428','ZNF429','ZNF43','ZNF430','ZNF431','ZNF432','ZNF433','ZNF436','ZNF438','ZNF439','ZNF44','ZNF440','ZNF441','ZNF442','ZNF443','ZNF444','ZNF445','ZNF446','ZNF449','ZNF45','ZNF451','ZNF454','ZNF460','ZNF461','ZNF462','ZNF467','ZNF468','ZNF469','ZNF470','ZNF471','ZNF473','ZNF474','ZNF479','ZNF48','ZNF480','ZNF483','ZNF484','ZNF485','ZNF486','ZNF487','ZNF488','ZNF490','ZNF491','ZNF492','ZNF493','ZNF496','ZNF497','ZNF500','ZNF501','ZNF502','ZNF503','ZNF506','ZNF507','ZNF510','ZNF511','ZNF512','ZNF512B','ZNF513','ZNF514','ZNF516','ZNF517','ZNF518A','ZNF518B','ZNF519','ZNF521','ZNF524','ZNF525','ZNF526','ZNF527','ZNF528','ZNF529','ZNF530','ZNF532','ZNF534','ZNF536','ZNF540','ZNF541','ZNF543','ZNF544','ZNF546','ZNF547','ZNF548','ZNF549','ZNF550','ZNF551','ZNF552','ZNF554','ZNF555','ZNF556','ZNF557','ZNF558','ZNF559','ZNF560','ZNF561','ZNF562','ZNF563','ZNF564','ZNF565','ZNF566','ZNF567','ZNF568','ZNF569','ZNF57','ZNF570','ZNF571','ZNF572','ZNF573','ZNF574','ZNF575','ZNF576','ZNF577','ZNF578','ZNF579','ZNF580','ZNF581','ZNF582','ZNF583','ZNF584','ZNF585A','ZNF585B','ZNF586','ZNF587','ZNF587B','ZNF589','ZNF592','ZNF594','ZNF595','ZNF596','ZNF597','ZNF598','ZNF599','ZNF600','ZNF605','ZNF606','ZNF607','ZNF608','ZNF609','ZNF610','ZNF611','ZNF613','ZNF614','ZNF615','ZNF616','ZNF618','ZNF619','ZNF620','ZNF621','ZNF623','ZNF624','ZNF625','ZNF626','ZNF627','ZNF628','ZNF629','ZNF630','ZNF639','ZNF641','ZNF644','ZNF645','ZNF646','ZNF648','ZNF649','ZNF652','ZNF653','ZNF654','ZNF655','ZNF658','ZNF66','ZNF660','ZNF662','ZNF664','ZNF665','ZNF667','ZNF668','ZNF669','ZNF670','ZNF671','ZNF672','ZNF674','ZNF675','ZNF676','ZNF677','ZNF678','ZNF679','ZNF680','ZNF681','ZNF682','ZNF683','ZNF684','ZNF687','ZNF688','ZNF689','ZNF69','ZNF691','ZNF692','ZNF695','ZNF696','ZNF697','ZNF699','ZNF7','ZNF70','ZNF700','ZNF701','ZNF703','ZNF704','ZNF705A','ZNF705B','ZNF705D','ZNF705E','ZNF705G','ZNF706','ZNF707','ZNF708','ZNF709','ZNF71','ZNF710','ZNF711','ZNF713','ZNF714','ZNF716','ZNF717','ZNF718','ZNF721','ZNF724','ZNF726','ZNF727','ZNF728','ZNF729','ZNF730','ZNF732','ZNF735','ZNF736','ZNF737','ZNF74','ZNF740','ZNF746','ZNF747','ZNF749','ZNF750','ZNF75A','ZNF75D','ZNF76','ZNF761','ZNF763','ZNF764','ZNF765','ZNF766','ZNF768','ZNF77','ZNF770','ZNF771','ZNF772','ZNF773','ZNF774','ZNF775','ZNF776','ZNF777','ZNF778','ZNF780A','ZNF780B','ZNF781','ZNF782','ZNF783','ZNF784','ZNF785','ZNF786','ZNF787','ZNF788','ZNF789','ZNF79','ZNF790','ZNF791','ZNF792','ZNF793','ZNF799','ZNF8','ZNF80','ZNF800','ZNF804A','ZNF804B','ZNF805','ZNF808','ZNF81','ZNF813','ZNF814','ZNF816','ZNF821','ZNF823','ZNF827','ZNF829','ZNF83','ZNF830','ZNF831','ZNF835','ZNF836','ZNF837','ZNF84','ZNF841','ZNF843','ZNF844','ZNF845','ZNF846','ZNF85','ZNF850','ZNF852','ZNF853','ZNF860','ZNF865','ZNF878','ZNF879','ZNF880','ZNF883','ZNF888','ZNF891','ZNF90','ZNF91','ZNF92','ZNF93','ZNF98','ZNF99','ZSCAN1','ZSCAN10','ZSCAN12','ZSCAN16','ZSCAN18','ZSCAN2','ZSCAN20','ZSCAN21','ZSCAN22','ZSCAN23','ZSCAN25','ZSCAN26','ZSCAN29','ZSCAN30','ZSCAN31','ZSCAN32','ZSCAN4','ZSCAN5A','ZSCAN5B','ZSCAN5C','ZSCAN9','ZUFSP','ZXDA','ZXDB','ZXDC','ZZZ3']
TFs=TFs1+TFs2

#TFs other, inclusive of the above list, not necessarily DNA binding proteins but affect transcription
TFs1Other=['AATF','ABL1','ABRA','AC008770.3','AC020915.1','AC023509.3','AC092835.1','AC105001.2','AC138696.1','ACOD1','ACTR3B','ACTR3C','ADAM8','ADAMTS17','ADAMTS19','ADAR','ADGRA1','ADGRG3','ADNP','ADNP2','AEBP1','AEBP2','AES','AFAP1 ','AFF1','AFF3','AFF4','AGER','AGT','AHCTF1','AHDC1','AHR','AHRR','AIM2','AIRE','AKAP8','AKAP8L','AKNA','AKT1','ALK','ALKBH2','ALX1','ALX3','ALX4','AMH','ANAPC2','ANHX','ANKRD30A','ANKZF1','ANXA3','ANXA4','AP5Z1','APBB3','APC2','APEX1','APOA4','APOBEC3A','APOBEC3B','APOBEC3G','APTX','AR','ARGFX','ARHGAP35','ARHGEF2','ARHGEF5','ARID1A','ARID1B','ARID2','ARID3A','ARID3B','ARID3C','ARID4A','ARID4B','ARID5A','ARID5B','ARNT','ARNT2','ARNTL','ARNTL2','ARRB1','ARRB2','ARX','ASCC1','ASCL1','ASCL2','ASCL3','ASCL4','ASCL5','ASH1L','ATF1','ATF2','ATF3','ATF4','ATF5','ATF6','ATF6B','ATF7','ATF7IP','ATM','ATMIN','ATOH1','ATOH7','ATOH8','ATR','ATRX','ATXN7','BACH1','BACH2','BAG1','BANF1','BANP','BARHL1','BARHL2','BARX1','BARX2','BATF','BATF2','BATF3','BAZ2A','BAZ2B','BBX','BCL10','BCL11A','BCL11B','BCL3','BCL6','BCL6B','BCLAF1','BCOR','BDP1','BEND4','BEX1','BHLHA15','BHLHA9','BHLHE22','BHLHE23','BHLHE40','BHLHE41','BHMG1','BIN1','BLM','BLZF1','BMI1','BMP2','BMP4','BMP7','BMPR1A','BNC1','BNC2','BOD1L1','BOLA1','BOLA2','BOLA2B','BOLA3','BORCS8-MEF2B','BPNT1','BPTF','BRCA1','BRD4','BRD7','BRD8','BRD9','BRF1','BRF2','BRMS1','BRPF1','BSX','BTAF1','BTBD8','BTG2','BTK','BTRC','BUD31','C11orf95','C14orf39','C17orf49','C1D','C20orf194','C3orf33','CACTIN','CAMK1D','CAMK2A','CAMTA1','CAMTA2','CAPN15','CAPN3','CARD11','CARD14','CARF','CARHSP1','CASP8AP2','CASZ1','CAT','CBFA2T2','CBFA2T3','CBFB','CBL','CBLL1','CBX2','CC2D1A','CC2D1B','CCDC124','CCDC169-SOHLH2','CCDC17','CCDC3','CCDC71','CCDC83','CCNT1','CCNT2','CD36','CD40','CD40LG','CDC5L','CDC6','CDK5RAP3','CDK7','CDK8','CDK9','CDKN1A','CDKN2A','CDX1','CDX2','CDX4','CEBPA','CEBPB','CEBPD','CEBPE','CEBPG','CEBPZ','CELF3','CENPA','CENPB','CENPBD1','CENPS','CENPT','CENPX','CERS2','CERS3','CERS4','CERS5','CERS6','CFLAR','CGGBP1','CHAMP1','CHCHD3','CHD1','CHD2','CHD3','CHD4','CHD6','CHD7','CHD9','CHP1','CHRAC1','CHUK','CHURC1','CIB1','CIC','CIITA','CIR1','CITED1','CITED2','CITED4','CIZ1','CLOCK','CLU','CMKLR1','CNBP','CNOT3','CNOT4','CNOT7','CNOT8','COMMD1','COMMD6','COMMD7','COPS2','COPS3','COPS4','CPEB1','CPSF4','CPSF4L','CPXCR1','CRAMP1','CREB1','CREB3','CREB3L1','CREB3L2','CREB3L3','CREB3L4','CREB5','CREBBP','CREBL2','CREBRF','CREBZF','CREG1','CREM','CRTC1','CRTC2','CRTC3','CRX','CSDC2','CSDE1','CSRNP1','CSRNP2','CSRNP3','CTBP1','CTBP2','CTCF','CTCFL','CTH','CTNNB1','CTNNBIP1','CUL1','CUL2','CUL3','CUL4A','CUL4B','CUL5','CUX1','CUX2','CXXC1','CXXC4','CXXC5','CYLD','CYP1B1','CYTL1','DAB2IP','DACH1','DACH2','DAP','DAXX','DBP','DBX1','DBX2','DCP1A','DDB1','DDB2','DDIT3','DDN','DDR2','DDRGK1','DDX3X','DDX5','DDX58','DEAF1','DEK','DEPDC1','DEPDC1B','DEPDC4','DEPDC5','DEPDC7','DEPTOR','DHX34','DHX36','DHX57','DISP3','DLX1','DLX2','DLX3','DLX4','DLX5','DLX6','DMAP1','DMBX1','DMC1','DMRT1','DMRT2','DMRT3','DMRTA1','DMRTA2','DMRTB1','DMRTC1','DMRTC2','DMTF1','DNA2','DNAJA3','DNAJC1','DNAJC2','DNAJC21','DNMT1','DNMT3A','DNMT3B','DNMT3L','DNTT','DNTTIP1','DOT1L','DPF1','DPF2','DPF3','DPRX','DR1','DRAP1','DRGX','DSP','DTX1','DUS3L','DUSP12','DUX1','DUX3','DUX4','DUXA','DVL1','DVL2','DVL3','DZIP1','DZIP1L','E2F1','E2F2','E2F3','E2F4','E2F5','E2F6','E2F7','E2F8','E4F1','EAF2','EBF1','EBF2','EBF3','EBF4','EDA','EDA2R','EDF1','EDN1','EEA1','EED','EGLN1','EGR1','EGR2','EGR3','EGR4','EHF','EID1','EID2','EIF2AK2','EIF2AK4','EIF3C','EIF3CL','EIF3K','ELF1','ELF2','ELF3','ELF4','ELF5','ELK1','ELK3','ELK4','ELMSAN1','EME1','EMX1','EMX2','EN1','EN2','ENO1','EOMES','EP300','EP400','EPAS1','EPHA5','EPM2AIP1','ERBIN','ERC1','ERCC3','ERCC4','ERF','ERG','ESR1','ESR2','ESRRA','ESRRB','ESRRG','ESX1','ETS1','ETS2','ETV1','ETV2','ETV3','ETV3L','ETV4','ETV5','ETV6','ETV7','EVX1','EVX2','EWSR1','EXO1','EXOC2','EXOG','EZH1','EZH2','FAAP24','FAM153C','FAM170A','FAM171B','FAM200A','FAM200B','FAN1','FANCA','FANCD2','FANK1','FARSA','FARSB','FBN1','FBXL19','FBXO41','FEN1','FER','FERD3L','FEV','FEZF1','FEZF2','FGD1','FHL2','FIGLA','FIZ1','FLI1','FLNA','FLYWCH1','FMNL2','FOS','FOSB','FOSL1','FOSL2','FOXA1','FOXA2','FOXA3','FOXB1','FOXB2','FOXC1','FOXC2','FOXD1','FOXD2','FOXD3','FOXD4','FOXD4L1','FOXD4L2','FOXD4L3','FOXD4L4','FOXD4L5','FOXD4L6','FOXE1','FOXE3','FOXF1','FOXF2','FOXG1','FOXH1','FOXI1','FOXI2','FOXI3','FOXJ1','FOXJ2','FOXJ3','FOXK1','FOXK2','FOXL1','FOXL2','FOXM1','FOXN1','FOXN2','FOXN3','FOXN4','FOXO1','FOXO3','FOXO4','FOXO6','FOXP1','FOXP2','FOXP3','FOXP4','FOXQ1','FOXR1','FOXR2','FOXS1','FUBP1','FUBP3','FUS','FZD1','FZD2','FZD4','FZD6','GABPA','GABPB1','GABPB2','GADD45A','GAS6','GAS7','GATA1','GATA2','GATA3','GATA4','GATA5','GATA6','GATAD1','GATAD2A','GATAD2B','GBX1','GBX2','GCFC2','GCM1','GCM2','GEN1','GFI1','GFI1B','GLA','GLI1','GLI2','GLI3','GLI4','GLIS1','GLIS2','GLIS3','GLMP','GLYR1','GMEB1','GMEB2','GOLGB1','GON4L','GPATCH8','GPBP1','GPBP1L1','GPER1','GPR155','GREM1','GRHL1','GRHL2','GRHL3','GRM6','GSC','GSC2','GSX1','GSX2','GTF2A1','GTF2A1L','GTF2A2','GTF2B','GTF2E2','GTF2F1','GTF2F2','GTF2H1','GTF2H2','GTF2H2C','GTF2H3','GTF2H4','GTF2I','GTF2IRD1','GTF2IRD2','GTF2IRD2B','GTF3A','GTF3C1','GTF3C2','GULP1','GZF1','H1F0','H1FOO','H1FX','HAND1','HAND2','HAVCR2','HBP1','HCFC1','HCK','HDAC1','HDAC10','HDAC2','HDAC3','HDAC4','HDAC5','HDAC6','HDAC7','HDAC8','HDAC9','HDX','HELT','HELZ','HELZ2','HES1','HES2','HES3','HES4','HES5','HES6','HES7','HESX1','HEY1','HEY2','HEYL','HHEX','HIC1','HIC2','HIF1A','HIF3A','HINFP','HIPK2','HIRA','HIST1H1A','HIST1H1B','HIST1H1C','HIST1H1D','HIST1H1E','HIST1H1T','HIVEP1','HIVEP2','HIVEP3','HKR1','HLA-DQB1','HLA-DQB2','HLA-DRB1','HLA-DRB3','HLA-DRB5','HLF','HLTF','HLX','HMBOX1','HMG20A','HMG20B','HMGA1','HMGA2','HMGB1','HMGB2','HMGB3','HMGB4','HMGN1','HMGN3','HMGXB3','HMGXB4','HMOX1','HMX1','HMX2','HMX3','HNF1A','HNF1B','HNF4A','HNF4G','HNRNPA1','HNRNPA3','HNRNPAB','HNRNPCL2','HNRNPD','HNRNPDL','HNRNPK','HNRNPL','HNRNPU','HOMEZ','HOOK2','HOPX','HOXA1','HOXA10','HOXA11','HOXA13','HOXA2','HOXA3','HOXA4','HOXA5','HOXA6','HOXA7','HOXA9','HOXB1','HOXB13','HOXB2','HOXB3','HOXB4','HOXB5','HOXB6','HOXB7','HOXB8','HOXB9','HOXC10','HOXC11','HOXC12','HOXC13','HOXC4','HOXC5','HOXC6','HOXC8','HOXC9','HOXD1','HOXD10','HOXD11','HOXD12','HOXD13','HOXD3','HOXD4','HOXD8','HOXD9','HP1BP3','HR','HSBP1','HSF1','HSF2','HSF4','HSF5','HSFX1','HSFX2','HSFY1','HSFY2','HSPA1A','HSPA1B','ICAM1','ID1','ID2','ID3','ID4','IFI16','IFNB1','IGHM','IKBKB','IKBKG','IKZF1','IKZF2','IKZF3','IKZF4','IKZF5','IL10','IL1B','IL1RAP','IL4','IL5','IL6','INF2','ING4','INS','INSM1','INSM2','IQGAP1','IRAK1','IRAK2','IRAK3','IRF1','IRF2','IRF2BP1','IRF3','IRF4','IRF5','IRF6','IRF7','IRF8','IRF9','IRX1','IRX2','IRX3','IRX4','IRX5','IRX6','ISL1','ISL2','ISX','ITCH','ITGB2','ITGB3BP','IZUMO2','JADE1','JAK2','JARID2','JAZF1','JDP2','JMY','JRK','JRKL','JUN','JUNB','JUND','JUP','KANSL2','KAT2A','KAT2B','KAT5','KAT6A','KAT6B','KAT7','KAT8','KCMF1','KCNIP3','KDM1A','KDM2A','KDM2B','KDM3A','KDM5A','KDM5B','KDM5C','KDM5D','KEAP1','KIAA1549','KIAA1683','KIAA1841','KIAA1958','KIF15','KIN','KIT','KLF1','KLF10','KLF11','KLF12','KLF13','KLF14','KLF15','KLF16','KLF17','KLF2','KLF3','KLF4','KLF5','KLF6','KLF7','KLF8','KLF9','KMT2A','KMT2B','KMT2C','KMT2D','KNTC1','KRAS','KRBOX1','KRT13','KRTAP5-1','KRTAP5-9','KRTAP9-9','L3MBTL1','L3MBTL3','L3MBTL4','LARP1','LARP1B','LARP4','LARP4B','LARP6','LARP7','LBX1','LBX2','LCOR','LCORL','LEF1','LEUTX','LGALS9','LGR4','LHX1','LHX2','LHX3','LHX4','LHX5','LHX6','LHX8','LHX9','LIG1','LIG3','LIN28A','LIN28B','LIN54','LMO2','LMO4','LMX1A','LMX1B','LPP','LRP5','LRP6','LRP8','LRRFIP1','LRRFIP2','LSR','LTF','LTN1','LYAR','LYL1','LZTFL1','LZTR1','LZTS1','MACC1','MACF1','MAD2L2','MAEL','MAF','MAFA','MAFB','MAFF','MAFG','MAFK','MALT1','MAML1','MAML3','MAP2K5','MAP3K10','MAP3K13','MAP3K7','MAPK1','MAPK10','MAPK11','MAPK14','MAPK3','MAPK8','MAPK9','MATR3','MAVS','MAX','MAZ','MB21D1','MBD1','MBD2','MBD3','MBD4','MBD6','MBNL1','MBNL2','MBNL3','MBTPS2','MDFI','MDM2','MDM4','MECOM','MECP2','MED1','MED12','MED13','MED13L','MED14','MED15','MED16','MED17','MED21','MED23','MED24','MED6','MEF2A','MEF2B','MEF2C','MEF2D','MEIS1','MEIS2','MEIS3','MEN1','MEOX1','MEOX2','MESP1','MESP2','MET','MGA','MGMT','MID2','MIER1','MIER2','MIER3','MINK1','MIS18BP1','MITF','MIXL1','MKL1','MKL2','MKRN1','MKRN2','MKRN3','MKX','MLLT1','MLLT10','MLLT11','MLLT3','MLLT6','MLX','MLXIP','MLXIPL','MN1','MNAT1','MNT','MNX1','MORC3','MPG','MRPL28','MRRF','MSANTD1','MSANTD3','MSANTD4','MSC','MSGN1','MSH2','MSH3','MSH6','MSL3','MSRB2','MST1R','MSX1','MSX2','MTA1','MTA2','MTA3','MTDH','MTERF1','MTERF2','MTERF3','MTERF4','MTF1','MTF2','MTPN','MUS81','MXD1','MXD3','MXD4','MXI1','MYB','MYBBP1A','MYBL1','MYBL2','MYC','MYCBP','MYCL','MYCN','MYD88','MYEF2','MYF5','MYF6','MYNN','MYOCD','MYOD1','MYOG','MYPOP','MYRF','MYRFL','MYSM1','MYT1','MYT1L','MZF1','NAB2','NABP2','NACC2','NAIF1','NANOG','NANOGNB','NANOGP8','NCOA1','NCOA2','NCOA3','NCOA4','NCOA5','NCOA6','NCOA7','NCOR1','NCOR2','NDC80','NDN','NDP','NEB','NEIL1','NEUROD1','NEUROD2','NEUROD4','NEUROD6','NEUROG1','NEUROG2','NEUROG3','NFAM1','NFAT5','NFATC1','NFATC2','NFATC3','NFATC4','NFE2','NFE2L1','NFE2L2','NFE2L3','NFE4','NFIA','NFIB','NFIC','NFIL3','NFIX','NFKB1','NFKB2','NFKBIA','NFKBIB','NFKBID','NFKBIL1','NFKBIZ','NFRKB','NFX1','NFXL1','NFYA','NFYB','NFYC','NHLH1','NHLH2','NKRF','NKX1-1','NKX1-2','NKX2-1','NKX2-2','NKX2-3','NKX2-4','NKX2-5','NKX2-6','NKX2-8','NKX3-1','NKX3-2','NKX6-1','NKX6-2','NKX6-3','NLRC3','NLRC4','NLRC5','NLRP12','NLRP2B','NLRP3','NME2','NOBOX','NOC3L','NOC4L','NOCT','NOD1','NOD2','NODAL','NONO','NOP2','NOTCH1','NOTCH2','NOTCH3','NOTO','NPAS1','NPAS2','NPAS3','NPAS4','NPM1','NR0B1','NR0B2','NR1D1','NR1D2','NR1H2','NR1H3','NR1H4','NR1I2','NR1I3','NR2C1','NR2C2','NR2E1','NR2E3','NR2F1','NR2F2','NR2F6','NR3C1','NR3C2','NR4A1','NR4A2','NR4A3','NR5A1','NR5A2','NR6A1','NRBF2','NRF1','NRG1','NRIP1','NRIP3','NRK','NRL','NSD1','NSD2','NTRK1','NUCKS1','NUFIP1','NUP98','NUPL2']
TFs2Other=['OGG1','OLIG1','OLIG2','OLIG3','ONECUT1','ONECUT2','ONECUT3','OPRD1','OSR1','OSR2','OTOP3','OTP','OTULIN','OTX1','OTX2','OVOL1','OVOL2','OVOL3','PA2G4','PABPN1','PARK7','PARP1','PARP10','PARP12','PATZ1','PAWR','PAX1','PAX2','PAX3','PAX4','PAX5','PAX6','PAX7','PAX8','PAX9','PAXBP1','PAXIP1','PBRM1','PBX1','PBX2','PBX3','PBX4','PCBD1','PCBP1','PCBP2','PCBP3','PCGF2','PCGF6','PCNA','PCSK6','PDCD7','PDS5B','PDX1','PEG3','PELI1','PER1','PER2','PER3','PEX14','PFDN1','PGBD1','PGR','PGS1','PHB','PHB2','PHF1','PHF19 ','PHF20','PHF21A','PHF5A','PHOX2A','PHOX2B','PHTF1','PHTF2','PIAS1','PIAS2','PIAS3','PIAS4','PIDD1','PIKFYVE','PIM1','PIN1','PINK1','PITX1','PITX2','PITX3','PKHD1','PKHD1L1','PKNOX1','PKNOX2','PLA2G10','PLA2G1B','PLAG1','PLAGL1','PLAGL2','PLEK','PLEK2','PLEKHA4','PLPP3','PLSCR1','PLXNA1','PLXNA2','PLXNA3','PLXNA4','PLXNB1','PLXNB2','PLXNB3','PLXNC1','PLXND1','PML','PMS1','PNKP','POGK','POGZ','POLA1','POLB','POLE3','POLE4','POLG','POLH','POLI','POLK','POLL','POLM','POLN','POLQ','POLR1B','POLR2A','POLR2B','POLR2E','POLR2J','POLR2K','POLR2L','POLR3A','POLRMT','POT1','POU1F1','POU2AF1','POU2F1','POU2F2','POU2F3','POU3F1','POU3F2','POU3F3','POU3F4','POU4F1','POU4F2','POU4F3','POU5F1','POU5F1B','POU5F2','POU6F1','POU6F2','PPARA','PPARD','PPARG','PPARGC1A','PPARGC1B','PPP1R10','PPP1R13L','PPP1R21','PPP2R3B','PPP2R5B','PPP3CA','PPRC1','PQBP1','PRAM1','PRAP1','PRB3','PRDM1','PRDM10','PRDM11','PRDM12','PRDM13','PRDM14','PRDM15','PRDM16','PRDM2','PRDM4','PRDM5','PRDM6','PRDM7','PRDM8','PRDM9','PRDX3','PREB','PREX1','PREX2','PRH2','PRIM2','PRIMPOL','PRKCB','PRKCH','PRKCI','PRKCQ','PRKCZ','PRKD1','PRKD2','PRKN','PRMT2','PRMT3','PRNP','PROP1','PROX1','PROX2','PRR12','PRR3','PRRX1','PRRX2','PSMA6','PSMC5','PSMD10','PSMD11','PSMD12','PSPC1','PTCH1','PTEN','PTF1A','PTGIS','PTH','PTTG1','PUM3','PURA','PURB','PURG','PYCARD','PYDC1','PYDC2','QRICH1','RAB11FIP4','RAB7B','RAD21','RAD51','RAG1','RAI1','RAPGEF3','RAPGEF4','RAPGEF5','RARA','RARB','RARG','RAX','RAX2','RB1','RBAK','RBBP7','RBCK1','RBL1','RBL2','RBM10','RBM20','RBM22','RBM26','RBM27','RBM5','RBM6','RBPJ','RBPJL','RBSN','RC3H1','RC3H2','RCAN1','RCOR1','RCOR2','RCOR3','RECQL','REL','RELA','RELB','RELL2','RELN','REPIN1','RERE','REST','REV1','REXO4','RFC1','RFX1','RFX2','RFX3','RFX4','RFX5','RFX6','RFX7','RFX8','RFXANK','RFXAP','RGCC','RGS11','RGS6','RGS7','RGS9','RHEBL1','RHOXF1','RHOXF2','RHOXF2B','RING1','RIOK2','RIPK1','RIPK2','RIPK3','RIPK4','RLF','RLIM','RNASE2','RNASEH1','RNF113A','RNF113B','RNF114','RNF125','RNF138','RNF14','RNF166','RNF2','RNF219','RNF25','RNF31','RNF4','RNF41','RORA','RORB','RORC','RPA2','RPA4','RPL36A','RPL7','RPL7A','RPS27A','RPS6KA4','RPS6KA5','RREB1','RRN3','RUNX1','RUNX1T1','RUNX2','RUNX3','RWDD3','RXRA','RXRB','RXRG','RYBP','S100A12','S100A8','S100A9','SAFB','SAFB2','SALL1','SALL2','SALL3','SALL4','SART1','SATB1','SATB2','SAV1','SCAND1','SCAND2P','SCAPER','SCMH1','SCML1','SCML2','SCML4','SCOC','SCRT1','SCRT2','SCX','SEBOX','SEMA4A','SERTAD1','SERTAD2','SETBP1','SETD6','SETDB1','SETDB2','SF3A2','SF3A3','SFPQ','SFRP4','SFRP5','SFTPB','SGK1','SGK3','SGSM2','SHH','SHOX','SHOX2','SHPRH','SIGIRR','SIK1','SIM1','SIM2','SIN3A','SIN3B','SIRT1','SIRT2','SIRT3','SIRT6','SIVA1','SIX1','SIX2','SIX3','SIX4','SIX5','SIX6','SKI','SKIDA1','SKIL','SKOR1','SKOR2','SLC22A4','SLC26A10','SLC26A3','SLC2A4RG','SLC30A9','SLC39A10','SLC4A10','SMAD1','SMAD2','SMAD3','SMAD4','SMAD5','SMAD6','SMAD7','SMAD9','SMARCA1','SMARCA2','SMARCA4','SMARCA5','SMARCAL1','SMARCB1','SMARCC1','SMARCC2','SMARCD1','SMARCD2','SMARCD3','SMARCE1','SMC3','SMO','SMUG1','SMYD1','SMYD3','SNAI1','SNAI2','SNAI3','SNAPC2','SNAPC4','SNAPC5','SNRPC','SNURF','SNW1','SOGA3','SOHLH1','SOHLH2','SON','SORBS2','SOX1','SOX10','SOX11','SOX12','SOX13','SOX14','SOX15','SOX17','SOX18','SOX2','SOX21','SOX3','SOX30','SOX4','SOX5','SOX6','SOX7','SOX8','SOX9','SP1','SP100','SP110','SP140','SP140L','SP2','SP3','SP4','SP5','SP6','SP7','SP8','SP9','SPDEF','SPEN','SPHK1','SPI1','SPIB','SPIC','SPZ1','SRCAP','SREBF1','SREBF2','SRF','SRY','SSB','SSBP2','SSBP3','SSBP4','SSH1','SSH2','SSH3','SSRP1','SSX1','ST18','STAG1','STAT1','STAT2','STAT3','STAT4','STAT5A','STAT5B','STAT6','STK16','STK3','STK36','STRN','STRN3','SUB1','SUFU','SUMO1','SUPT16H','SUPT4H1','SUPT5H','SUPT6H','SUV39H1','SUV39H2','SUZ12','SYK','T','TAB1','TAB2','TAB3','TADA2A','TADA2B','TADA3','TAF1','TAF10','TAF11','TAF12','TAF13','TAF1A','TAF1B','TAF1C','TAF1L','TAF2','TAF3','TAF4','TAF4B','TAF5','TAF5L','TAF6','TAF6L','TAF7','TAF7L','TAF8','TAF9','TAF9B','TAL1','TAL2','TARDBP','TAX1BP1','TBL1Y','TBP','TBPL1','TBPL2','TBR1','TBX1','TBX10','TBX15','TBX18','TBX19','TBX2','TBX20','TBX21','TBX22','TBX3','TBX4','TBX5','TBX6','TCEA2','TCEAL1','TCEAL7','TCEAL8','TCF12','TCF15','TCF19','TCF20','TCF21','TCF23','TCF24','TCF25','TCF3','TCF4','TCF7','TCF7L1','TCF7L2','TCFL5','TDG','TDP1','TDP2','TEAD1','TEAD2','TEAD3','TEAD4','TEF','TERB1','TERF1','TERF2','TERF2IP','TESMIN','TET1','TET2','TET3','TFAM','TFAP2A','TFAP2B','TFAP2C','TFAP2D','TFAP2E','TFAP4','TFCP2','TFCP2L1','TFDP1','TFDP2','TFDP3','TFE3','TFEB','TFEC','TGFB1','TGFB1I1','TGIF1','TGIF2','TGIF2LX','TGIF2LY','TGS1','THAP1','THAP10','THAP11','THAP12','THAP2','THAP3','THAP4','THAP5','THAP6','THAP7','THAP8','THAP9','THRA','THRAP3','THRB','THYN1','TIA1','TICAM1','TIGD1','TIGD2','TIGD3','TIGD4','TIGD5','TIGD6','TIGD7','TIPARP','TIRAP','TLE1','TLE2','TLE3','TLE4','TLE6','TLR2','TLR3','TLR4','TLR9','TLX1','TLX2','TLX3','TMEM235','TMF1','TNF','TNFAIP3','TNFRSF11A','TNFRSF4','TNFSF11','TNFSF18','TNFSF4','TOE1','TONSL','TOP1','TOP2A','TOP2B','TOPORS','TOX','TOX2','TOX3','TOX4','TP53','TP53BP1','TP63','TP73','TPRX1','TRADD','TRAF1','TRAF2','TRAF3','TRAF4','TRAF5','TRAF6','TRAFD1','TRAK2','TRAPPC9','TRERF1','TREX1','TRIB1','TRIB3','TRIM13','TRIM14','TRIM15','TRIM21','TRIM22','TRIM23','TRIM24','TRIM25','TRIM26','TRIM27','TRIM28','TRIM29','TRIM3','TRIM31','TRIM32','TRIM33','TRIM34','TRIM37','TRIM38','TRIM40','TRIM5','TRIM52','TRIM62','TRIM8','TRIP11','TRIT1','TRMT1','TRMT1L','TRPS1','TRRAP','TSC2','TSC22D1','TSC22D2','TSC22D3','TSC22D4','TSHZ1','TSHZ2','TSHZ3','TSN','TSSK4','TTF1','TUB','TULP4','TUT1','TWIST1','TWIST2','TXK','U2AF1','U2AF1L4','UBA52','UBB','UBC','UBE2K','UBE2N','UBE2V1','UBN1','UBP1','UBR4','UBTF','UBTFL1','UBTFL6','UFL1','UHRF1','UHRF2','UNCX','UNG','UNK','UNKL','USF1','USF2','USF3','USP39','USP7','UTF1','VAV1','VAX1','VAX2','VDR','VEGFA','VENTX','VEZF1','VGLL1','VGLL2','VGLL3','VHLL','VPS72','VSX1','VSX2','WDHD1','WFS1','WIZ','WNT1','WNT10B','WNT2','WNT3A','WNT5A','WNT8B','WRN','WT1','WWOX','WWP2','WWTR1','XBP1','XCL1','XPA','YBX1','YBX2','YBX3','YEATS2','YEATS4','YOD1','YY1','YY2','ZBED1','ZBED2','ZBED3','ZBED4','ZBED5','ZBED6','ZBED6CL','ZBED8','ZBED9','ZBP1','ZBTB1','ZBTB10','ZBTB11','ZBTB12','ZBTB14','ZBTB16','ZBTB17','ZBTB18','ZBTB2','ZBTB20','ZBTB21','ZBTB22','ZBTB24','ZBTB25','ZBTB26','ZBTB3','ZBTB32','ZBTB33','ZBTB34','ZBTB37','ZBTB38','ZBTB39','ZBTB4','ZBTB40','ZBTB41','ZBTB42','ZBTB43','ZBTB44','ZBTB45','ZBTB46','ZBTB47','ZBTB48','ZBTB49','ZBTB5','ZBTB6','ZBTB7A','ZBTB7B','ZBTB7C','ZBTB8A','ZBTB8B','ZBTB9','ZC3H10','ZC3H11A','ZC3H12A','ZC3H13','ZC3H14','ZC3H15','ZC3H18','ZC3H3','ZC3H4','ZC3H6','ZC3H7A','ZC3H7B','ZC3H8','ZC3HAV1','ZCCHC11','ZCCHC6','ZDHHC11','ZDHHC19','ZEB1','ZEB2','ZFAND5','ZFAT','ZFHX2','ZFHX3','ZFHX4','ZFP1','ZFP14','ZFP2','ZFP28','ZFP3','ZFP30','ZFP36','ZFP36L1','ZFP36L2','ZFP37','ZFP41','ZFP42','ZFP57','ZFP62','ZFP64','ZFP69','ZFP69B','ZFP82','ZFP90','ZFP91','ZFP92','ZFPL1','ZFPM1','ZFPM2','ZFR','ZFR2','ZFX','ZFY','ZFYVE26','ZGLP1','ZGPAT','ZHX1','ZHX2','ZHX3','ZIC1','ZIC2','ZIC3','ZIC4','ZIC5','ZIK1','ZIM2','ZIM3','ZKSCAN1','ZKSCAN2','ZKSCAN3','ZKSCAN4','ZKSCAN5','ZKSCAN7','ZKSCAN8','ZMAT1','ZMAT2','ZMAT3','ZMAT4','ZMAT5','ZMYM1','ZMYM2','ZMYM3','ZMYM4','ZMYM5','ZMYM6','ZMYND10','ZNF10','ZNF100','ZNF101','ZNF106','ZNF107','ZNF112','ZNF114','ZNF117','ZNF12','ZNF121','ZNF124','ZNF131','ZNF132','ZNF133','ZNF134','ZNF135','ZNF136','ZNF137P','ZNF138','ZNF14','ZNF140','ZNF141','ZNF142','ZNF143','ZNF146','ZNF148','ZNF154','ZNF155','ZNF157','ZNF16','ZNF160','ZNF165','ZNF169','ZNF17','ZNF174','ZNF175','ZNF177','ZNF18','ZNF180','ZNF181','ZNF182','ZNF184','ZNF185','ZNF189','ZNF19','ZNF195','ZNF197','ZNF2','ZNF20','ZNF200','ZNF202','ZNF205','ZNF207','ZNF208','ZNF211','ZNF212','ZNF213','ZNF214','ZNF215','ZNF217','ZNF219','ZNF22','ZNF221','ZNF222','ZNF223','ZNF224','ZNF225','ZNF226','ZNF227','ZNF229','ZNF23','ZNF230','ZNF232','ZNF233','ZNF234','ZNF235','ZNF236','ZNF239','ZNF24','ZNF248','ZNF25','ZNF250','ZNF251','ZNF253','ZNF254','ZNF256','ZNF257','ZNF26','ZNF260','ZNF263','ZNF264','ZNF266','ZNF267','ZNF268','ZNF273','ZNF274','ZNF275','ZNF276','ZNF277','ZNF28','ZNF280A','ZNF280B','ZNF280C','ZNF280D','ZNF281','ZNF282','ZNF283','ZNF284','ZNF285','ZNF286A','ZNF286B','ZNF287','ZNF292','ZNF296','ZNF3','ZNF30','ZNF300','ZNF302','ZNF304','ZNF311','ZNF316','ZNF317','ZNF318','ZNF319','ZNF32','ZNF320','ZNF322','ZNF324','ZNF324B','ZNF326','ZNF329','ZNF330','ZNF331','ZNF333','ZNF334','ZNF335','ZNF337','ZNF33A','ZNF33B','ZNF34','ZNF341','ZNF343','ZNF345','ZNF346','ZNF347','ZNF35','ZNF350','ZNF354A','ZNF354B','ZNF354C','ZNF355P','ZNF358','ZNF362','ZNF365','ZNF366','ZNF367','ZNF37A','ZNF382','ZNF383','ZNF384','ZNF385A','ZNF385B','ZNF385C','ZNF385D','ZNF391','ZNF394','ZNF395','ZNF396','ZNF397','ZNF398','ZNF404','ZNF407','ZNF408','ZNF41','ZNF410','ZNF414','ZNF415','ZNF416','ZNF417','ZNF418','ZNF419','ZNF420','ZNF423','ZNF425','ZNF426','ZNF428','ZNF429','ZNF43','ZNF430','ZNF431','ZNF432','ZNF433','ZNF436','ZNF438','ZNF439','ZNF44','ZNF440','ZNF441','ZNF442','ZNF443','ZNF444','ZNF445','ZNF446','ZNF449','ZNF45','ZNF451','ZNF454','ZNF460','ZNF461','ZNF462','ZNF467','ZNF468','ZNF469','ZNF470','ZNF471','ZNF473','ZNF474','ZNF479','ZNF48','ZNF480','ZNF483','ZNF484','ZNF485','ZNF486','ZNF487','ZNF488','ZNF490','ZNF491','ZNF492','ZNF493','ZNF496','ZNF497','ZNF500','ZNF501','ZNF502','ZNF503','ZNF506','ZNF507','ZNF510','ZNF511','ZNF512','ZNF512B','ZNF513','ZNF514','ZNF516','ZNF517','ZNF518A','ZNF518B','ZNF519']
TFs3Other=['ZNF521','ZNF524','ZNF525','ZNF526','ZNF527','ZNF528','ZNF529','ZNF530','ZNF532','ZNF534','ZNF536','ZNF540','ZNF541','ZNF542P','ZNF543','ZNF544','ZNF546','ZNF547','ZNF548','ZNF549','ZNF550','ZNF551','ZNF552','ZNF554','ZNF555','ZNF556','ZNF557','ZNF558','ZNF559','ZNF560','ZNF561','ZNF562','ZNF563','ZNF564','ZNF565','ZNF566','ZNF567','ZNF568','ZNF569','ZNF57','ZNF570','ZNF571','ZNF572','ZNF573','ZNF574','ZNF575','ZNF576','ZNF577','ZNF578','ZNF579','ZNF580','ZNF581','ZNF582','ZNF583','ZNF584','ZNF585A','ZNF585B','ZNF586','ZNF587','ZNF587B','ZNF589','ZNF592','ZNF593','ZNF594','ZNF595','ZNF596','ZNF597','ZNF598','ZNF599','ZNF600','ZNF605','ZNF606','ZNF607','ZNF608','ZNF609','ZNF610','ZNF611','ZNF613','ZNF614','ZNF615','ZNF616','ZNF618','ZNF619','ZNF620','ZNF621','ZNF622','ZNF623','ZNF624','ZNF625','ZNF626','ZNF627','ZNF628','ZNF629','ZNF630','ZNF638','ZNF639','ZNF641','ZNF644','ZNF645','ZNF646','ZNF648','ZNF649','ZNF652','ZNF653','ZNF654','ZNF655','ZNF658','ZNF66','ZNF660','ZNF662','ZNF664','ZNF665','ZNF667','ZNF668','ZNF669','ZNF670','ZNF671','ZNF672','ZNF674','ZNF675','ZNF676','ZNF677','ZNF678','ZNF679','ZNF680','ZNF681','ZNF682','ZNF683','ZNF684','ZNF687','ZNF688','ZNF689','ZNF69','ZNF691','ZNF692','ZNF695','ZNF696','ZNF697','ZNF699','ZNF7','ZNF70','ZNF700','ZNF701','ZNF703','ZNF704','ZNF705A','ZNF705B','ZNF705D','ZNF705E','ZNF705G','ZNF706','ZNF707','ZNF708','ZNF709','ZNF71','ZNF710','ZNF711','ZNF713','ZNF714','ZNF716','ZNF717','ZNF718','ZNF720','ZNF721','ZNF724','ZNF726','ZNF727','ZNF728','ZNF729','ZNF73','ZNF730','ZNF732','ZNF735','ZNF736','ZNF737','ZNF738','ZNF74','ZNF740','ZNF746','ZNF747','ZNF749','ZNF750','ZNF75A','ZNF75D','ZNF76','ZNF761','ZNF763','ZNF764','ZNF765','ZNF766','ZNF768','ZNF77','ZNF770','ZNF771','ZNF772','ZNF773','ZNF774','ZNF775','ZNF776','ZNF777','ZNF778','ZNF780A','ZNF780B','ZNF781','ZNF782','ZNF783','ZNF784','ZNF785','ZNF786','ZNF787','ZNF788','ZNF789','ZNF79','ZNF790','ZNF791','ZNF792','ZNF793','ZNF799','ZNF8','ZNF80','ZNF800','ZNF804A','ZNF804B','ZNF805','ZNF808','ZNF81','ZNF812P','ZNF813','ZNF814','ZNF816','ZNF821','ZNF823','ZNF827','ZNF829','ZNF83','ZNF830','ZNF831','ZNF835','ZNF836','ZNF837','ZNF839','ZNF84','ZNF841','ZNF843','ZNF844','ZNF845','ZNF846','ZNF85','ZNF850','ZNF852','ZNF853','ZNF860','ZNF862','ZNF865','ZNF878','ZNF879','ZNF880','ZNF883','ZNF888','ZNF891','ZNF90','ZNF91','ZNF92','ZNF93','ZNF98','ZNF99','ZNFX1','ZRANB2','ZRSR1','ZRSR2','ZSCAN1','ZSCAN10','ZSCAN12','ZSCAN16','ZSCAN18','ZSCAN2','ZSCAN20','ZSCAN21','ZSCAN22','ZSCAN23','ZSCAN25','ZSCAN26','ZSCAN29','ZSCAN30','ZSCAN31','ZSCAN32','ZSCAN4','ZSCAN5A','ZSCAN5B','ZSCAN5C','ZSCAN5DP','ZSCAN9','ZUFSP','ZXDA','ZXDB','ZXDC','ZZZ3']
TFs4Other=['ATF2','ATF6B','ATN1','BHLHA15','BHLHB2','BHLHB4 ','BHLHB5','BHLHB9','C13ORF8','C20ORF194','CDX2','CRAMP1L','ELP3','ERVK3-1','ETF1','FLJ12895','FLJ20321','FLJ25476','FLJ35784','FOXM1','FOXO2','FOXR1','FOXR2','GATA1','GRLF1','KDM4C','KDM4E','KIAA0863','KIAA0961','KLRG1','LBXCOR1','LEF1','LHX7','LOC126295','LOC51058','LOC91661','MGC4728','MYOD1','MYOG','MYRF','NEUROD4','NEUROG1','NFAT2','NFAT4','NFE2L1','NR1A2','NR1B1','NR1B2','NR1B3','NR1C1','NR1C2','NR1C3','NR1F3','NR1I1','NR2A1','NR2B3','NR3A1','NR3A2','NR3B3','OTEX','PEPP-2','PHF2','POU2F1','POU2F2','POU4F3','POU5F1','POU6F1','PRDM3','PRKRIP1','PRKRIR','SETD1A','SMAD8','SMYD2','SOX12','SOX14','SOX15','TBX14','TCF1','TCF17','TCF2','TCF9','TEAD1','TPRXL','UBTFL1','WHSC1','ZBTB14','ZBTB18','ZBTB19','ZBTB21','ZBTB27','ZBTB28','ZBTB35','ZBTB49','ZFP161','ZFP28','ZKSCAN10','ZKSCAN11','ZKSCAN12','ZKSCAN13','ZKSCAN14','ZKSCAN16','ZKSCAN19','ZKSCAN21','ZKSCAN24','ZKSCAN6','ZKSCAN8','ZNF10','ZNF112','ZNF187','ZNF193','ZNF204P','ZNF238','ZNF271','ZNF295','ZNF305','ZNF323','ZNF434','ZNF499','ZNF542','ZNF545','ZNF553','ZNF56','ZNF625','ZNF643','ZNF698','ZNF705F','ZNF751','ZNF752','ZNF753','ZNF785','ZNF806','ZNF816','ZNF833P','ZNF906','ZSCAN1','ZSCAN25','ZSCAN26','ZSCAN31','ZSCAN32','ZSCAN5D','ZSCAN9']
TFsOther=TFs1Other+TFs2Other+TFs3Other+TFs4Other
```

Example 8 (unknown):
```unknown
S genes: ['MCM5', 'PCNA', 'TYMS', 'FEN1', 'MCM2', 'MCM4', 'RRM1', 'UNG', 'GINS2', 'MCM6', 'CDCA7', 'DTL', 'PRIM1', 'UHRF1', 'HELLS', 'RFC2', 'RPA2', 'NASP', 'RAD51AP1', 'GMNN', 'WDR76', 'SLBP', 'CCNE2', 'UBR7', 'POLD3', 'MSH2', 'ATAD2', 'RAD51', 'RRM2', 'CDC45', 'CDC6', 'EXO1', 'TIPIN', 'DSCC1', 'BLM', 'CASP8AP2', 'USP1', 'CLSPN', 'POLA1', 'CHAF1B', 'BRIP1', 'E2F8'] 

G2 genes: ['HMGB2', 'CDK1', 'NUSAP1', 'UBE2C', 'BIRC5', 'TPX2', 'TOP2A', 'NDC80', 'CKS2', 'NUF2', 'CKS1B', 'MKI67', 'TMPO', 'CENPF', 'TACC3', 'SMC4', 'CCNB2', 'CKAP2L', 'CKAP2', 'AURKB', 'BUB1', 'KIF11', 'ANP32E', 'TUBB4B', 'GTSE1', 'KIF20B', 'HJURP', 'CDCA3', 'CDC20', 'TTK', 'CDC25C', 'KIF2C', 'RANGAP1', 'NCAPD2', 'DLGAP5', 'CDCA2', 'CDCA8', 'ECT2', 'KIF23', 'HMMR', 'AURKA', 'PSRC1', 'ANLN', 'LBR', 'CKAP5', 'CENPE', 'CTCF', 'NEK2', 'G2E3', 'GAS2L3', 'CBX5', 'CENPA'] 

Hemoglobin genes set 1: ['HBB', 'HBD', 'HBG1', 'HBG2', 'HBE1', 'HBZ', 'HBM', 'HBA2', 'HBA1', 'HBQ1'] 

Hemoglobin genes set 2: ['AHSP', 'GYPA', 'RHAG', 'GFI1B', 'KEL', 'EPB42', 'ALAS2', 'TSPO2', 'GYPB', 'GYPE', 'SPTA1', 'RHCE', 'KLF1', 'RHD', 'SLC4A1', 'SNX22', 'SQSTM1', 'SOD1', 'FTH1', 'MAP1LC3B', 'SLC25A37', 'TRIM58', 'HMBS'] 

Mitochondrial genes: ['MT-ND1', 'MT-ND2', 'MT-CO1', 'MT-CO2', 'MT-ATP8', 'MT-ATP6', 'MT-CO3', 'MT-ND3', 'MT-ND4L', 'MT-ND4', 'MT-ND5', 'MT-ND6', 'MT-CYB'] 

Ribo genes: ['RPLP0', 'RPLP1', 'RPLP2', 'RPL3', 'RPL3L', 'RPL4', 'RPL5', 'RPL6', 'RPL7', 'RPL7A', 'RPL7L1', 'RPL8', 'RPL9', 'RPL10', 'RPL10A', 'RPL10L', 'RPL11', 'RPL12', 'RPL13', 'RPL13A', 'RPL14', 'RPL15', 'RPL17', 'RPL18', 'RPL18A', 'RPL19', 'RPL21', 'RPL22', 'RPL22L1', 'RPL23', 'RPL23A', 'RPL24', 'RPL26', 'RPL26L1', 'RPL27', 'RPL27A', 'RPL28', 'RPL29', 'RPL30', 'RPL31', 'RPL32', 'RPL34', 'RPL35', 'RPL35A', 'RPL36', 'RPL36A', 'RPL36AL', 'RPL37', 'RPL37A', 'RPL38', 'RPL39', 'RPL39L', 'UBA52', 'RPL41', 'RPSA', 'RPS2', 'RPS3', 'RPS3A', 'RPS4X', 'RPS4Y1', 'RPS4Y2', 'RPS5', 'RPS6', 'RPS7', 'RPS8', 'RPS9', 'RPS10', 'RPS11', 'RPS12', 'RPS13', 'RPS14', 'RPS15', 'RPS15A', 'RPS16', 'RPS17', 'RPS18', 'RPS19', 'RPS20', 'RPS21', 'RPS23', 'RPS24', 'RPS25', 'RPS26', 'RPS27', 'RPS27A', 'RPS27L', 'RPS28', 'RPS29', 'FAU'] 

Tissue dissociation stress associated genes: ['FOS', 'FOSB', 'JUN', 'JUNB', 'JUND', 'ZFP36', 'SOCS3', 'CEBPD', 'CEBPB', 'ATF3', 'EGR1', 'HSPA1A', 'HSPA1B', 'HSP90AA1', 'HSP90AB1', 'HSPA8', 'HSPB1', 'HSPE1', 'IER3', 'IER2', 'BTG1', 'BTG2', 'DUSP1']
```

Example 9 (python):
```python
#Expression: "YlGn" (e.g. DE plots)
#Divergent expression: "PRGn" (e.g. log2FC)
#Age: "afmhot_r"
#Others: "viridis" (e.g. UMAP expression)

#Define color palette
FineGrainPalette={
    'AtrialCardiomyocytesLeft':'#eac075',
    'AtrialCardiomyocytesRight':'#4c6b8a',
    'AtrialCardiomyocytesCycling':'#e94bc7',
    'VentricularCardiomyocytesLeftCompact':'#ab1c3c',
    'VentricularCardiomyocytesRightCompact':'#fdba45',
    'VentricularCardiomyocytesLeftTrabeculated':'#2b9200',
    'VentricularCardiomyocytesRightTrabeculated':'#0273a8',
    'VentricularCardiomyocytesCycling':'#e1acff',
    'SinoatrialNodePacemakerCells':'#854834',
    'AtrioventricularNodePacemakerCells':'#688500',
    'VentricularConductionSystemProximal':'#01ccc8',
    'VentricularConductionSystemDistal':'#b58fab',
    'GreatVesselAdventitialFibroblasts':'#ff7e97',
    'CoronaryVesselAdventitialFibroblasts':'#ff9b6e',
    'MyocardialInterstitialFibroblasts':'#8a6800',
    'SubEpicardialFibroblasts':'#206601',
    'Myofibroblasts':'#1bdebf',
    'LymphNodeFibroblasticReticularCells':'#005aa0',
    'ValveInterstitialCells':'#e478ff',
    'GreatVesselSmoothMuscleCells':'#ff665f',
    'CoronarySmoothMuscleCells':'#d2cb18',
    'DuctusArteriosusSmoothMuscleCells':'#83d7b3',
    'CoronaryPericytes':'#9088ff',
    'PericardialCellsIntermediate':'#c95e00',
    'PericardialCellsFibrous':'#01e077',
    'PericardialCellsParietal':'#ff88e4',
    'GreatVesselArterialEndothelialCells':'#d4004b',
    'GreatVesselVenousEndothelialCells':'#8f4300',
    'CoronaryArterialEndothelialCells':'#89da5a',
    'CoronaryVenousEndothelialCells':'#006f86',
    'CoronaryCapillaryEndothelialCells':'#aa7dff',
    'EndocardialCells':'#ff8241',
    'EndocardialCushionCells':'#46603a',
    'ValveEndothelialCells':'#414eb6',
    'LymphaticEndothelialCells':'#8b3f60',
    'MesothelialEpicardialCells':'#0295fe',
    'EpicardiumDerivedCells':'#ff92c4',
    'NeuronPrecursors':'#d30038',
    'ChromaffinCells':'#726800',
    'SympatheticNeurons':'#01894b',
    'ParasympatheticNeurons':'#545191',
    'SchwannCellPrecursors':'#b0d276',
    'SchwannCells':'#b30079',
    'MonocytesMPOpos':'#c50066',
    'Monocytes':'#fab4b6',
    'MonocyteDerivedCells':'#d14008',
    'MacrophagesCX3CR1pos':'#eabe97',
    'MacrophagesTIMD4pos':'#b2a200',
    'MacrophagesLYVE1pos':'#4a6005',
    'MacrophagesATF3pos':'#3db833',
    'DendriticCellsType1':'#007859',
    'DendriticCellsMature':'#01a9db',
    'PlasmacytoidDendriticCells':'#2c54ab',
    'MastCells':'#d0bef2',
    'Megakaryocytes':'#94298f',
    'TCellsCD4pos':'#ff387c',
    'TCellsCD8pos':'#9e3429',
    'TregsCD4pos':'#b37700',
    'ProBCells':'#8dab00',
    'BCells':'#a8d1a1',
    'BCellsMS4A1pos':'#00d2f0',
    'NaturalKillerCells':'#abb4ff',
    'InnateLymphoidCells':'#72497f',
}
sc.pl.umap(adata, color='fine_grain',palette=FineGrainPalette)

MidGrainPalette={
    'AtrialCardiomyocytes':'#af3a4d',
    'VentricularCardiomyocytes':'#b1756c',
    'CardiacConductionSystem':'#d45633',
    'Fibroblasts':'#9b6e4e',
    'MuralCells':'#ffb92f',
    'PericardialCells':'#b4f03b',
    'BloodVesselEndothelialCells':'#72cc57',
    'EndocardialCells':'#436e43',
    'LymphaticEndothelialCells':'#85cca0',
    'EpicardialCells':'#5eabb0',
    'Neurons':'#7bbdff',
    'Glia':'#403c90',
    'MyeloidCells':'#420066',
    'LymphoidCells':'#f553f6',
}
sc.pl.umap(adata, color='mid_grain',palette=MidGrainPalette)

CoarseGrainPalette={
    'Cardiomyocytes':'#bc5950',
    'Mesenchymal':'#af9c4c',
    'Endothelium':'#599e58',
    'Epicardium':'#5eabb0',
    'Neural':'#5e75b1',
    'Leukocytes':'#9a52a0',
}
sc.pl.umap(adata, color='coarse_grain',palette=CoarseGrainPalette)

RegionGrainPalette={
    'aorta':'#ffa2bc',
    'apex':'#fe0049',
    'atria':'#732d00',
    'atrial septum':'#ffd54c',
    'base':'#837100',
    'ductus arteriosus':'#bdff8b',
    'whole sample':'#1b6c00',
    'heart without node':'#02e6c6',
    'left ventricle':'#01c1f5',
    'node':'#00466d',
    'outflow tract':'#020011',
    'pericardium':'#7a75ff',
    'pulmonary arches and branches':'#30003e',
    'right ventricle':'#ff88e1',
    'right ventricle and papillaries':'#98004f',
}
sc.pl.umap(adata, color='region',palette=RegionGrainPalette)
```

Example 10 (python):
```python
# preserve raw count
adata.layers["counts"] = adata.X.copy()

# normalise and log-transform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key='batch_key')
#sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5)

print(sum(adata.var.highly_variable))
sc.pl.highly_variable_genes(adata)

# freeze the state in `.raw`
adata.raw = adata

# subset hvgs
adata = adata[:, adata.var.highly_variable]

adata=adata.copy()

sc.tl.score_genes_cell_cycle(adata, s_genes=s_genes, g2m_genes=g2m_genes)
#sc.pp.regress_out(adata, ['S_score','G2M_score'])

#batch correction
scvi.model.SCVI.setup_anndata(adata, 
                              layer="counts", 
                              batch_key="batch_key",
                              categorical_covariate_keys=['phase'],
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo'])

vae = scvi.model.SCVI(adata, n_hidden = 128, n_latent = 50, n_layers = 3, dispersion = 'gene-batch')
vae.train(max_epochs=50)
    
# Convergence plot
x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
plt.plot(x, vae.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
plt.clf()

# add the latent space to adata
adata.obsm["X_scVI"] = vae.get_latent_representation()

sc.tl.score_genes(adata, stress_genes, score_name='stress_score')
sc.tl.score_genes(adata, hb_genes1, score_name='hb1_score')

sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata,min_dist=0.1)
#sc.tl.tsne(adata)
sc.pl.umap(adata, color=['log1p_total_counts','n_genes','n_genes_by_counts','total_counts','pct_counts_mt','pct_counts_ribo'],ncols=5)
sc.pl.umap(adata, color=['scrublet_score','stress_score', 'hb1_score','phase'], ncols=5,)

sc.pl.umap(adata, color=['donor','age','week','trimester','region','phase','cycling','cell_or_nuclei','modality','kit_10x','FACSgate','heart_or_greatvessels','coarse_grain','mid_grain','fine_grain'],wspace=0.8,ncols=5)
Genes=['TTN','TNNT2','MYOM1','LDB3','ALDH1A2','TNNT1','KRT19','UPK3B','BNC1','COL6A3','FBLN1','PCOLCE','LRRC17','PCDH18','EMCN','ECSCR','PECAM1','CDH5','ESAM', 'LAPTM5','PTPRC','LCP1','CD53','WAS','PHOX2B','NRXN1','STMN2','GRIK2','L1CAM']
sc.pl.umap(adata, color=Genes)

sc.pl.umap(adata, color='fine_grain')
sc.pl.umap(adata, color='cycling')

sc.tl.diffmap(adata)
sc.pl.diffmap(adata, color='fine_grain')
sc.pl.diffmap(adata, color='cycling')

# PAGA graph construction
sc.tl.paga(adata, groups='fine_grain')
plt.rcParams["figure.figsize"] = [6, 4.5]
sc.pl.paga(adata,frameon=False)
sc.tl.draw_graph(adata, init_pos='paga', random_state=123)
sc.pl.draw_graph(adata, color='fine_grain')
sc.pl.draw_graph(adata, color='cycling')
```

Example 11 (python):
```python
normalizing counts per cell
    finished (0:00:00)
extracting highly variable genes
    finished (0:00:06)
--> added
    'highly_variable', boolean vector (adata.var)
    'means', float vector (adata.var)
    'dispersions', float vector (adata.var)
    'dispersions_norm', float vector (adata.var)
2794
```

Example 12 (python):
```python
calculating cell cycle phase
computing score 'S_score'
WARNING: genes are not in var_names and ignored: ['MCM5', 'UNG', 'MCM6', 'CDCA7', 'PRIM1', 'RFC2', 'RPA2', 'NASP', 'GMNN', 'SLBP', 'CCNE2', 'MSH2', 'RAD51', 'CDC45', 'CDC6', 'EXO1', 'TIPIN', 'DSCC1', 'CASP8AP2', 'USP1', 'POLA1', 'CHAF1B', 'E2F8']
    finished: added
    'S_score', score of gene set (adata.obs).
    536 total control genes are used. (0:00:00)
computing score 'G2M_score'
WARNING: genes are not in var_names and ignored: ['TMPO', 'ANP32E', 'CDC25C', 'RANGAP1', 'CDCA2', 'LBR', 'CKAP5', 'CTCF', 'G2E3', 'CBX5']
    finished: added
    'G2M_score', score of gene set (adata.obs).
    820 total control genes are used. (0:00:00)
-->     'phase', cell cycle phase (adata.obs)
```

Example 13 (unknown):
```unknown
GPU available: False, used: False
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
```

Example 14 (unknown):
```unknown
Epoch 50/50: 100%|███████| 50/50 [01:08<00:00,  1.36s/it, loss=1.2e+03, v_num=1]
```

Example 15 (python):
```python
computing score 'stress_score'
WARNING: genes are not in var_names and ignored: ['JUND', 'HSP90AA1', 'HSP90AB1', 'HSPA8', 'HSPB1', 'HSPE1', 'BTG1']
    finished: added
    'stress_score', score of gene set (adata.obs).
    393 total control genes are used. (0:00:00)
computing score 'hb1_score'
WARNING: genes are not in var_names and ignored: ['HBD', 'HBQ1']
    finished: added
    'hb1_score', score of gene set (adata.obs).
    294 total control genes are used. (0:00:00)
computing neighbors
    finished: added to `.uns['neighbors']`
    `.obsp['distances']`, distances for each pair of neighbors
    `.obsp['connectivities']`, weighted adjacency matrix (0:00:02)
computing UMAP
    finished: added
    'X_umap', UMAP coordinates (adata.obsm) (0:00:03)
```

Example 16 (unknown):
```unknown
... storing 'phase' as categorical
```

Example 17 (unknown):
```unknown
<Figure size 320x320 with 0 Axes>
```

Example 18 (python):
```python
computing Diffusion Maps using n_comps=15(=n_dcs)
computing transitions
    finished (0:00:00)
    eigenvalues of transition matrix
    [1.         0.99813575 0.99639726 0.99591756 0.9939016  0.99028945
     0.9895821  0.98796153 0.9864883  0.98519886 0.9835595  0.9803847
     0.9789236  0.97758424 0.97680676]
    finished: added
    'X_diffmap', diffmap coordinates (adata.obsm)
    'diffmap_evals', eigenvalues of transition matrix (adata.uns) (0:00:00)
```

Example 19 (python):
```python
running PAGA
    finished: added
    'paga/connectivities', connectivities adjacency (adata.uns)
    'paga/connectivities_tree', connectivities subtree (adata.uns) (0:00:00)
--> added 'pos', the PAGA positions (adata.uns['paga'])
```

Example 20 (unknown):
```unknown
drawing single-cell graph using layout 'fa'
```

Example 21 (python):
```python
---------------------------------------------------------------------------
KeyboardInterrupt                         Traceback (most recent call last)
/tmp/ipykernel_143680/25561144.py in <module>
     68 plt.rcParams["figure.figsize"] = [6, 4.5]
     69 sc.pl.paga(adata,frameon=False)
---> 70 sc.tl.draw_graph(adata, init_pos='paga', random_state=123)
     71 sc.pl.draw_graph(adata, color='fine_grain')
     72 sc.pl.draw_graph(adata, color='cycling')

~/miniconda3/envs/scanpynew/lib/python3.8/site-packages/scanpy/tools/_draw_graph.py in draw_graph(adata, layout, init_pos, root, random_state, n_jobs, adjacency, key_added_ext, neighbors_key, obsp, copy, **kwds)
    157         else:
    158             iterations = 500
--> 159         positions = forceatlas2.forceatlas2(
    160             adjacency, pos=init_coords, iterations=iterations
    161         )

~/miniconda3/envs/scanpynew/lib/python3.8/site-packages/fa2/forceatlas2.py in forceatlas2(self, G, pos, iterations)
    197             # parallelization should be implemented here
    198             if self.barnesHutOptimize:
--> 199                 rootRegion.applyForceOnNodes(nodes, self.barnesHutTheta, self.scalingRatio)
    200             else:
    201                 fa2util.apply_repulsion(nodes, self.scalingRatio)

KeyboardInterrupt:
```

Example 22 (python):
```python
#bdata=adata.copy()
#adata=bdata.copy()
```

Example 23 (unknown):
```unknown
new annotation
original annotation
barcode
```

Example 24 (python):
```python
sc.pl.umap(adata, color='fine_grain')
```

Example 25 (python):
```python
set(adata.obs.fine_grain)
```

Example 26 (unknown):
```unknown
{'CoronaryArterialEndothelialCells',
 'CoronaryCapillaryEndothelialCells',
 'CoronaryVenousEndothelialCells',
 'EndocardialCells',
 'EndocardialCushionCells',
 'GreatVesselArterialEndothelialCells',
 'GreatVesselVenousEndothelialCells',
 'LymphaticEndothelialCells',
 'ValveEndothelialCells'}
```

Example 27 (python):
```python
adata = adata[adata.obs['fine_grain'].isin(['EndocardialCushionCells','EndocardialCells','ValveEndothelialCells','GreatVesselArterialEndothelialCells','GreatVesselVenousEndothelialCells','CoronaryCapillaryEndothelialCells','CoronaryVenousEndothelialCells','CoronaryArterialEndothelialCells'])].copy()
```

Example 28 (python):
```python
sc.pl.umap(adata, color='fine_grain')
```

Example 29 (python):
```python
sc.tl.leiden(adata, resolution=0.75, key_added='leiden')
sc.pl.umap(adata, color='leiden',legend_loc='on data')
```

Example 30 (python):
```python
running Leiden clustering
    finished: found 19 clusters and added
    'leiden', the cluster labels (adata.obs, categorical) (0:00:00)
```

Example 31 (python):
```python
adata.obs['SVannotation']= 'na'

adata.obs['SVannotation'][[x for x in adata.obs_names if adata.obs.fine_grain[x] in ['CoronaryArterialEndothelialCells']]]='CoronaryArterialEndothelialCells'
adata.obs['SVannotation'][[x for x in adata.obs_names if adata.obs.fine_grain[x] in ['GreatVesselArterialEndothelialCells']]]='GreatVesselEndothelialCells'
adata.obs['SVannotation'][[x for x in adata.obs_names if adata.obs.fine_grain[x] in ['EndocardialCushionCells','EndocardialCells','ValveEndothelialCells']]]='EndocardialCells'
adata.obs['SVannotation'][[x for x in adata.obs_names if adata.obs.fine_grain[x] in ['CoronaryCapillaryEndothelialCells']]]='CoronaryCapillary1'
adata.obs['SVannotation'][[x for x in adata.obs_names if adata.obs.leiden[x] in ['17']]]='CoronaryCapillary2'
adata.obs['SVannotation'][[x for x in adata.obs_names if adata.obs.fine_grain[x] in ['GreatVesselVenousEndothelialCells','CoronaryVenousEndothelialCells']]]='ProgenitorCells'
sc.pl.umap(adata, color='SVannotation')
sc.pl.umap(adata, color='fine_grain')
```

Example 32 (unknown):
```unknown
... storing 'SVannotation' as categorical
```

Example 33 (python):
```python
ClusterOfInterest='17'
#sc.tl.rank_genes_groups(adata, 'leiden', method='t-test', key_added = "t-test")
DEGenesForCluster = [x[ClusterOfInterest] for x in adata.uns['t-test']['names'][:50]]
sc.pl.dotplot(adata, DEGenesForCluster ,groupby='leiden',standard_scale='var',cmap='PRGn')
#sc.pl.umap(adata, color=DEGenesForCluster)
```

Example 34 (python):
```python
sc.pl.umap(adata, color='cycling')
```

Example 35 (python):
```python
#New annotations for SV investigation

adata.obs['SVannotation']= 'na'
adata.obs['SVannotation'][[x for x in adata.obs_names if adata.obs.fine_grain[x] in ['GreatVesselVenousEndothelialCells','CoronaryVenousEndothelialCells']]]='ProgenitorCells'
adata.obs['SVannotation'][[x for x in adata.obs_names if adata.obs.fine_grain[x] in ['CoronaryCapillaryEndothelialCells','CoronaryArterialEndothelialCells']]]='CoronaryEndothelialCells'
adata.obs['SVannotation'][[x for x in adata.obs_names if adata.obs.fine_grain[x] in ['GreatVesselArterialEndothelialCells']]]='GreatVesselEndothelialCells'
adata.obs['SVannotation'][[x for x in adata.obs_names if adata.obs.fine_grain[x] in ['EndocardialCushionCells','EndocardialCells','ValveEndothelialCells']]]='EndocardialCells'
sc.pl.umap(adata, color='SVannotation')
```

Example 36 (unknown):
```unknown
... storing 'SVannotation' as categorical
```

Example 37 (python):
```python
sc.pl.umap(adata, color='ACKR1')
```

Example 38 (python):
```python
metadata=[]
for k in range(0,10685):
    a=adata.obs_names[k]+' , '+adata.obs.SVannotation[k]+' , '+adata.obs.fine_grain[k]+' , '+adata.obs.donor[k]
    metadata.append(a)
    
b=pd.DataFrame(metadata)
b.to_csv('SVTracingAnnotation.csv')
```

Example 39 (unknown):
```unknown
#trials for running RNA velocity
```

Example 40 (python):
```python
#G4
adataG4=sc.read('./Objects/Feb28ObjectRaw.h5ad')
adataG4 = adataG4[adataG4.obs['donor'].isin(['BRC2251'])].copy()
adataG4 = adataG4[adataG4.obs['fine_grain'].isin(['CoronaryArterialEndothelialCells','CoronaryCapillaryEndothelialCells','CoronaryVenousEndothelialCells'])].copy()
adataG4 = adataG4[adataG4.obs['region'].isin(['apex'])].copy()
print(adataG4.shape)
barcodes = adataG4.obs.index.tolist()
barcodes = [bc[9:len(bc)-2]+'_G4' for bc in barcodes]
adataG4.obs.index=barcodes
adataG4.var_names_make_unique()
ldataG4=scv.read('./LoomFiles/G4.loom', cache=True)
barcodes = [bc.split(':')[1] for bc in ldataG4.obs.index.tolist()]
barcodes = [bc[0:len(bc)-1] +'_G4' for bc in barcodes]
ldataG4.obs.index = barcodes
ldataG4.var_names_make_unique()
ldataG4=ldataG4[[x for x in ldataG4.obs_names if x in adataG4.obs_names],:].copy()
print(len(ldataG4.obs_names))

#F4
adataF4=sc.read('./Objects/Feb28ObjectRaw.h5ad')
adataF4 = adataF4[adataF4.obs['donor'].isin(['BRC2251'])].copy()
adataF4 = adataF4[adataF4.obs['fine_grain'].isin(['CoronaryArterialEndothelialCells','CoronaryCapillaryEndothelialCells','CoronaryVenousEndothelialCells'])].copy()
adataF4 = adataF4[adataF4.obs['region'].isin(['base'])].copy()
print(adataF4.shape)
barcodes = adataF4.obs.index.tolist()
barcodes = [bc[9:len(bc)-2]+'_F4' for bc in barcodes]
adataF4.obs.index=barcodes
adataF4.var_names_make_unique()
ldataF4=scv.read('./LoomFiles/F4.loom', cache=True)
barcodes = [bc.split(':')[1] for bc in ldataF4.obs.index.tolist()]
barcodes = [bc[0:len(bc)-1] +'_F4' for bc in barcodes]
ldataF4.obs.index = barcodes
ldataF4.var_names_make_unique()
ldataF4=ldataF4[[x for x in ldataF4.obs_names if x in adataF4.obs_names],:].copy()
print(len(ldataF4.obs_names))

#D8
adataD8=sc.read('./Objects/Feb28ObjectRaw.h5ad')
adataD8 = adataD8[adataD8.obs['donor'].isin(['BRC2253'])].copy()
adataD8 = adataD8[adataD8.obs['fine_grain'].isin(['CoronaryArterialEndothelialCells','CoronaryCapillaryEndothelialCells','CoronaryVenousEndothelialCells'])].copy()
adataD8 = adataD8[adataD8.obs['region'].isin(['apex'])].copy()
print(adataD8.shape)
barcodes = adataD8.obs.index.tolist()
barcodes = [bc[9:len(bc)-2]+'_D8' for bc in barcodes]
adataD8.obs.index=barcodes
adataD8.var_names_make_unique()
ldataD8=scv.read('./LoomFiles/D8.loom', cache=True)
barcodes = [bc.split(':')[1] for bc in ldataD8.obs.index.tolist()]
barcodes = [bc[0:len(bc)-1] +'_D8' for bc in barcodes]
ldataD8.obs.index = barcodes
ldataD8.var_names_make_unique()
ldataD8=ldataD8[[x for x in ldataD8.obs_names if x in adataD8.obs_names],:].copy()
print(len(ldataD8.obs_names))

#C8
adataC8=sc.read('./Objects/Feb28ObjectRaw.h5ad')
adataC8 = adataC8[adataC8.obs['donor'].isin(['BRC2253'])].copy()
adataC8 = adataC8[adataC8.obs['fine_grain'].isin(['CoronaryArterialEndothelialCells','CoronaryCapillaryEndothelialCells','CoronaryVenousEndothelialCells'])].copy()
adataC8 = adataC8[adataC8.obs['region'].isin(['base'])].copy()
print(adataC8.shape)
barcodes = adataC8.obs.index.tolist()
barcodes = [bc[9:len(bc)-2]+'_C8' for bc in barcodes]
adataC8.obs.index=barcodes
adataC8.var_names_make_unique()
ldataC8=scv.read('./LoomFiles/C8.loom', cache=True)
barcodes = [bc.split(':')[1] for bc in ldataC8.obs.index.tolist()]
barcodes = [bc[0:len(bc)-1] +'_C8' for bc in barcodes]
ldataC8.obs.index = barcodes
ldataC8.var_names_make_unique()
ldataC8=ldataC8[[x for x in ldataC8.obs_names if x in adataC8.obs_names],:].copy()
print(len(ldataC8.obs_names))

#F8
adataF8=sc.read('./Objects/Feb28ObjectRaw.h5ad')
adataF8 = adataF8[adataF8.obs['donor'].isin(['BRC2256'])].copy()
adataF8 = adataF8[adataF8.obs['fine_grain'].isin(['CoronaryArterialEndothelialCells','CoronaryCapillaryEndothelialCells','CoronaryVenousEndothelialCells'])].copy()
adataF8 = adataF8[adataF8.obs['region'].isin(['apex'])].copy()
print(adataF8.shape)
barcodes = adataF8.obs.index.tolist()
barcodes = [bc[9:len(bc)-2]+'_F8' for bc in barcodes]
adataF8.obs.index=barcodes
adataF8.var_names_make_unique()
ldataF8=scv.read('./LoomFiles/F8.loom', cache=True)
barcodes = [bc.split(':')[1] for bc in ldataF8.obs.index.tolist()]
barcodes = [bc[0:len(bc)-1] +'_F8' for bc in barcodes]
ldataF8.obs.index = barcodes
ldataF8.var_names_make_unique()
ldataF8=ldataF8[[x for x in ldataF8.obs_names if x in adataF8.obs_names],:].copy()
print(len(ldataF8.obs_names))

#E8
adataE8=sc.read('./Objects/Feb28ObjectRaw.h5ad')
adataE8 = adataE8[adataE8.obs['donor'].isin(['BRC2256'])].copy()
adataE8 = adataE8[adataE8.obs['fine_grain'].isin(['CoronaryArterialEndothelialCells','CoronaryCapillaryEndothelialCells','CoronaryVenousEndothelialCells'])].copy()
adataE8 = adataE8[adataE8.obs['region'].isin(['base'])].copy()
print(adataE8.shape)
barcodes = adataE8.obs.index.tolist()
barcodes = [bc[9:len(bc)-2]+'_E8' for bc in barcodes]
adataE8.obs.index=barcodes
adataE8.var_names_make_unique()
ldataE8=scv.read('./LoomFiles/E8.loom', cache=True)
barcodes = [bc.split(':')[1] for bc in ldataE8.obs.index.tolist()]
barcodes = [bc[0:len(bc)-1] +'_E8' for bc in barcodes]
ldataE8.obs.index = barcodes
ldataE8.var_names_make_unique()
ldataE8=ldataE8[[x for x in ldataE8.obs_names if x in adataE8.obs_names],:].copy()
print(len(ldataE8.obs_names))

#F2
adataF2=sc.read('./Objects/Feb28ObjectRaw.h5ad')
adataF2 = adataF2[adataF2.obs['donor'].isin(['BRC2260'])].copy()
adataF2 = adataF2[adataF2.obs['fine_grain'].isin(['CoronaryArterialEndothelialCells','CoronaryCapillaryEndothelialCells','CoronaryVenousEndothelialCells'])].copy()
adataF2 = adataF2[adataF2.obs['region'].isin(['apex'])].copy()
print(adataF2.shape)
barcodes = adataF2.obs.index.tolist()
barcodes = [bc[9:len(bc)-2]+'_F2' for bc in barcodes]
adataF2.obs.index=barcodes
adataF2.var_names_make_unique()
ldataF2=scv.read('./LoomFiles/F2.loom', cache=True)
barcodes = [bc.split(':')[1] for bc in ldataF2.obs.index.tolist()]
barcodes = [bc[0:len(bc)-1] +'_F2' for bc in barcodes]
ldataF2.obs.index = barcodes
ldataF2.var_names_make_unique()
ldataF2=ldataF2[[x for x in ldataF2.obs_names if x in adataF2.obs_names],:].copy()
print(len(ldataF2.obs_names))

#E2
adataE2=sc.read('./Objects/Feb28ObjectRaw.h5ad')
adataE2 = adataE2[adataE2.obs['donor'].isin(['BRC2260'])].copy()
adataE2 = adataE2[adataE2.obs['fine_grain'].isin(['CoronaryArterialEndothelialCells','CoronaryCapillaryEndothelialCells','CoronaryVenousEndothelialCells'])].copy()
adataE2 = adataE2[adataE2.obs['region'].isin(['base'])].copy()
print(adataE2.shape)
barcodes = adataE2.obs.index.tolist()
barcodes = [bc[9:len(bc)-2]+'_E2' for bc in barcodes]
adataE2.obs.index=barcodes
adataE2.var_names_make_unique()
ldataE2=scv.read('./LoomFiles/E2.loom', cache=True)
barcodes = [bc.split(':')[1] for bc in ldataE2.obs.index.tolist()]
barcodes = [bc[0:len(bc)-1] +'_E2' for bc in barcodes]
ldataE2.obs.index = barcodes
ldataE2.var_names_make_unique()
ldataE2=ldataE2[[x for x in ldataE2.obs_names if x in adataE2.obs_names],:].copy()
print(len(ldataE2.obs_names))

#B1
adataB1=sc.read('./Objects/Feb28ObjectRaw.h5ad')
adataB1 = adataB1[adataB1.obs['donor'].isin(['BRC2262'])].copy()
adataB1 = adataB1[adataB1.obs['fine_grain'].isin(['CoronaryArterialEndothelialCells','CoronaryCapillaryEndothelialCells','CoronaryVenousEndothelialCells'])].copy()
adataB1 = adataB1[adataB1.obs['region'].isin(['apex'])].copy()
print(adataB1.shape)
barcodes = adataB1.obs.index.tolist()
barcodes = [bc[9:len(bc)-2]+'_B1' for bc in barcodes]
adataB1.obs.index=barcodes
adataB1.var_names_make_unique()
ldataB1=scv.read('./LoomFiles/B1.loom', cache=True)
barcodes = [bc.split(':')[1] for bc in ldataB1.obs.index.tolist()]
barcodes = [bc[0:len(bc)-1] +'_B1' for bc in barcodes]
ldataB1.obs.index = barcodes
ldataB1.var_names_make_unique()
ldataB1=ldataB1[[x for x in ldataB1.obs_names if x in adataB1.obs_names],:].copy()
print(len(ldataB1.obs_names))

#A1
adataA1=sc.read('./Objects/Feb28ObjectRaw.h5ad')
adataA1 = adataA1[adataA1.obs['donor'].isin(['BRC2262'])].copy()
adataA1 = adataA1[adataA1.obs['fine_grain'].isin(['CoronaryArterialEndothelialCells','CoronaryCapillaryEndothelialCells','CoronaryVenousEndothelialCells'])].copy()
adataA1 = adataA1[adataA1.obs['region'].isin(['base'])].copy()
print(adataA1.shape)
barcodes = adataA1.obs.index.tolist()
barcodes = [bc[9:len(bc)-2]+'_A1' for bc in barcodes]
adataA1.obs.index=barcodes
adataA1.var_names_make_unique()
ldataA1=scv.read('./LoomFiles/A1.loom', cache=True)
barcodes = [bc.split(':')[1] for bc in ldataA1.obs.index.tolist()]
barcodes = [bc[0:len(bc)-1] +'_A1' for bc in barcodes]
ldataA1.obs.index = barcodes
ldataA1.var_names_make_unique()
ldataA1=ldataA1[[x for x in ldataA1.obs_names if x in adataA1.obs_names],:].copy()
print(len(ldataA1.obs_names))

#G7
adataG7=sc.read('./Objects/Feb28ObjectRaw.h5ad')
adataG7 = adataG7[adataG7.obs['donor'].isin(['BRC2263'])].copy()
adataG7 = adataG7[adataG7.obs['fine_grain'].isin(['CoronaryArterialEndothelialCells','CoronaryCapillaryEndothelialCells','CoronaryVenousEndothelialCells'])].copy()
adataG7 = adataG7[adataG7.obs['region'].isin(['apex'])].copy()
print(adataG7.shape)
barcodes = adataG7.obs.index.tolist()
barcodes = [bc[9:len(bc)-2]+'_G7' for bc in barcodes]
adataG7.obs.index=barcodes
adataG7.var_names_make_unique()
ldataG7=scv.read('./LoomFiles/G7.loom', cache=True)
barcodes = [bc.split(':')[1] for bc in ldataG7.obs.index.tolist()]
barcodes = [bc[0:len(bc)-1] +'_G7' for bc in barcodes]
ldataG7.obs.index = barcodes
ldataG7.var_names_make_unique()
ldataG7=ldataG7[[x for x in ldataG7.obs_names if x in adataG7.obs_names],:].copy()
print(len(ldataG7.obs_names))

#F7
adataF7=sc.read('./Objects/Feb28ObjectRaw.h5ad')
adataF7 = adataF7[adataF7.obs['donor'].isin(['BRC2263'])].copy()
adataF7 = adataF7[adataF7.obs['fine_grain'].isin(['CoronaryArterialEndothelialCells','CoronaryCapillaryEndothelialCells','CoronaryVenousEndothelialCells'])].copy()
adataF7 = adataF7[adataF7.obs['region'].isin(['base'])].copy()
print(adataF7.shape)
barcodes = adataF7.obs.index.tolist()
barcodes = [bc[9:len(bc)-2]+'_F7' for bc in barcodes]
adataF7.obs.index=barcodes
adataF7.var_names_make_unique()
ldataF7=scv.read('./LoomFiles/F7.loom', cache=True)
barcodes = [bc.split(':')[1] for bc in ldataF7.obs.index.tolist()]
barcodes = [bc[0:len(bc)-1] +'_F7' for bc in barcodes]
ldataF7.obs.index = barcodes
ldataF7.var_names_make_unique()
ldataF7=ldataF7[[x for x in ldataF7.obs_names if x in adataF7.obs_names],:].copy()
print(len(ldataF7.obs_names))

CSCIadata=adataG4.concatenate(adataF4,adataD8,adataC8,adataF8,adataE8,adataF2,adataE2,adataB1,adataA1,adataG7,adataF7)
ldata=ldataG4.concatenate([ldataF4,ldataD8,ldataC8,ldataF8,ldataE8,ldataF2,ldataE2,ldataB1,ldataA1,ldataG7,ldataF7])

#D8: BRC2253, apex
#C8: BRC2253, base\
#F8: BRC2256, apex
#E8: BRC2256, base
#F2: BRC2260, apex
#E2: BRC2260, base
#B1: BRC2262, apex
#A1: BRC2262, base
#G7: BRC2263, apex
#F7: BRC2263, base
```

Example 41 (python):
```python
(111, 36601)
... reading from cache file cache/LoomFiles-G4.h5ad
```

Example 42 (unknown):
```unknown
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
```

Example 43 (python):
```python
111
(256, 36601)
... reading from cache file cache/LoomFiles-F4.h5ad
```

Example 44 (unknown):
```unknown
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
```

Example 45 (python):
```python
252
(87, 36601)
... reading from cache file cache/LoomFiles-D8.h5ad
```

Example 46 (unknown):
```unknown
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
```

Example 47 (python):
```python
87
(81, 36601)
... reading from cache file cache/LoomFiles-C8.h5ad
```

Example 48 (unknown):
```unknown
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
```

Example 49 (python):
```python
81
(113, 36601)
... reading from cache file cache/LoomFiles-F8.h5ad
```

Example 50 (unknown):
```unknown
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
```

Example 51 (python):
```python
113
(433, 36601)
... reading from cache file cache/LoomFiles-E8.h5ad
```

Example 52 (unknown):
```unknown
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
```

Example 53 (python):
```python
433
(368, 36601)
... reading from cache file cache/LoomFiles-F2.h5ad
```

Example 54 (unknown):
```unknown
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
```

Example 55 (python):
```python
368
(517, 36601)
... reading from cache file cache/LoomFiles-E2.h5ad
```

Example 56 (unknown):
```unknown
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
```

Example 57 (python):
```python
517
(149, 36601)
... reading from cache file cache/LoomFiles-B1.h5ad
```

Example 58 (unknown):
```unknown
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
```

Example 59 (python):
```python
149
(163, 36601)
... reading from cache file cache/LoomFiles-A1.h5ad
```

Example 60 (unknown):
```unknown
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
```

Example 61 (python):
```python
163
(124, 36601)
... reading from cache file cache/LoomFiles-G7.h5ad
```

Example 62 (unknown):
```unknown
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
```

Example 63 (python):
```python
124
(295, 36601)
... reading from cache file cache/LoomFiles-F7.h5ad
```

Example 64 (unknown):
```unknown
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
```

Example 65 (python):
```python
CSCIadata=adataG4.concatenate(adataF4,adataD8,adataC8,adataF8,adataE8,adataF2,adataE2,adataB1,adataA1,adataG7,adataF7)
ldata=ldataG4.concatenate([ldataF4,ldataD8,ldataC8,ldataF8,ldataE8,ldataF2,ldataE2,ldataB1,ldataA1,ldataG7,ldataF7])
```

Example 66 (python):
```python
# reduced barcode names 
barcodes = CSCIadata.obs.index.tolist()
barcodes = [bc[0:19] for bc in barcodes]
CSCIadata.obs.index=barcodes

barcodes = ldata.obs.index.tolist()
barcodes = [bc[0:19] for bc in barcodes]
ldata.obs.index=barcodes
ldata.obs_names
```

Example 67 (unknown):
```unknown
Index(['AACGGGAGTACGATGG_G4', 'AAGAACAGTGATACAA_G4', 'AAGCCATCAGCTCTGG_G4',
       'ACACTGACACAAAGTA_G4', 'ACGATCATCATTACCT_G4', 'ACAGCCGTCACCCTGT_G4',
       'ACAAGCTGTGGATTTC_G4', 'ACTTCGCCATGTGCTA_G4', 'AGATCCATCAAACGTC_G4',
       'AGAACAAAGCACTCAT_G4',
       ...
       'TTGGGATAGCATCAAA_F7', 'TTGCTGCCACTGCACG_F7', 'TTGCATTTCACCTGGG_F7',
       'TTGTTGTGTGATGTAA_F7', 'TTGTGGAAGCTAGTTC_F7', 'TTTGGTTGTATCGTGT_F7',
       'TTTCGATTCAGACCGC_F7', 'TTTGGAGAGGCAGGGA_F7', 'TTTGGAGTCACCGGGT_F7',
       'TTTAGTCCAAGCAGGT_F7'],
      dtype='object', length=2691)
```

Example 68 (python):
```python
adata=sc.read('./Objects/Feb28ObjectRaw.h5ad')
print(adata.shape)
adata = adata[adata.obs['fine_grain'].isin(['CoronaryArterialEndothelialCells','CoronaryCapillaryEndothelialCells','CoronaryVenousEndothelialCells'])].copy()
adata = adata[adata.obs['facility'].isin(['Sanger'])].copy()
adata=adata.concatenate(CSCIadata)
```

Example 69 (unknown):
```unknown
(297473, 36601)
```

Example 70 (python):
```python
# preserve raw count
adata.layers["counts"] = adata.X.copy()

# normalise and log-transform
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key='batch_key')
#sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5)

print(sum(adata.var.highly_variable))
sc.pl.highly_variable_genes(adata)

# freeze the state in `.raw`
adata.raw = adata

# subset hvgs
adata = adata[:, adata.var.highly_variable]

adata=adata.copy()

sc.tl.score_genes_cell_cycle(adata, s_genes=s_genes, g2m_genes=g2m_genes)
#sc.pp.regress_out(adata, ['S_score','G2M_score'])

#batch correction
scvi.model.SCVI.setup_anndata(adata, 
                              layer="counts", 
                              batch_key="batch_key",
                              categorical_covariate_keys=['phase'],
                              continuous_covariate_keys=['total_counts','pct_counts_mt','pct_counts_ribo'])

vae = scvi.model.SCVI(adata, n_hidden = 128, n_latent = 50, n_layers = 3, dispersion = 'gene-batch')
vae.train(max_epochs=50)
    
# Convergence plot
x = np.linspace(1, (len(vae.history['elbo_train'])), (len(vae.history['elbo_train'])))
plt.plot(x, vae.history['elbo_train'], label="train")
plt.xlabel('epoch')
plt.ylabel('elbo_train')
plt.show()
plt.clf()

# add the latent space to adata
adata.obsm["X_scVI"] = vae.get_latent_representation()

sc.tl.score_genes(adata, stress_genes, score_name='stress_score')
sc.tl.score_genes(adata, hb_genes1, score_name='hb1_score')

sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata,min_dist=0.1)
#sc.tl.tsne(adata)
sc.pl.umap(adata, color=['log1p_total_counts','n_genes','n_genes_by_counts','total_counts','pct_counts_mt','pct_counts_ribo'],ncols=5)
sc.pl.umap(adata, color=['scrublet_score','stress_score', 'hb1_score','phase'], ncols=5,)

sc.pl.umap(adata, color=['donor','age','week','trimester','region','phase','cycling','cell_or_nuclei','modality','kit_10x','FACSgate','heart_or_greatvessels','coarse_grain','mid_grain','fine_grain'],wspace=0.8,ncols=5)
Genes=['TTN','TNNT2','MYOM1','LDB3','ALDH1A2','TNNT1','KRT19','UPK3B','BNC1','COL6A3','FBLN1','PCOLCE','LRRC17','PCDH18','EMCN','ECSCR','PECAM1','CDH5','ESAM', 'LAPTM5','PTPRC','LCP1','CD53','WAS','PHOX2B','NRXN1','STMN2','GRIK2','L1CAM']
sc.pl.umap(adata, color=Genes)

sc.pl.umap(adata, color='fine_grain')
sc.pl.umap(adata, color='cycling')

sc.tl.diffmap(adata)
sc.pl.diffmap(adata, color='fine_grain')
sc.pl.diffmap(adata, color='cycling')

# PAGA graph construction
sc.tl.paga(adata, groups='fine_grain')
plt.rcParams["figure.figsize"] = [6, 4.5]
sc.pl.paga(adata,frameon=False)
sc.tl.draw_graph(adata, init_pos='paga', random_state=123)
sc.pl.draw_graph(adata, color='fine_grain')
sc.pl.draw_graph(adata, color='cycling')
```

Example 71 (unknown):
```unknown
normalizing counts per cell
    finished (0:00:00)
extracting highly variable genes
```

Example 72 (unknown):
```unknown
... storing 'sangerID' as categorical
... storing 'combinedID' as categorical
... storing 'donor' as categorical
... storing 'region' as categorical
... storing 'age' as categorical
... storing 'facility' as categorical
... storing 'cell_or_nuclei' as categorical
... storing 'modality' as categorical
... storing 'kit_10x' as categorical
... storing 'donor_by_library-prep' as categorical
... storing 'multiplexed' as categorical
... storing 'SOC | status' as categorical
... storing 'batch_key' as categorical
... storing 'FACSgate' as categorical
... storing 'fine_grain' as categorical
... storing 'sex' as categorical
... storing 'week' as categorical
... storing 'trimester' as categorical
```

Example 73 (python):
```python
finished (0:00:08)
--> added
    'highly_variable', boolean vector (adata.var)
    'means', float vector (adata.var)
    'dispersions', float vector (adata.var)
    'dispersions_norm', float vector (adata.var)
1332
```

Example 74 (python):
```python
calculating cell cycle phase
computing score 'S_score'
WARNING: genes are not in var_names and ignored: ['MCM5', 'FEN1', 'MCM2', 'MCM4', 'UNG', 'GINS2', 'MCM6', 'CDCA7', 'PRIM1', 'UHRF1', 'RFC2', 'RPA2', 'NASP', 'GMNN', 'SLBP', 'CCNE2', 'UBR7', 'POLD3', 'MSH2', 'RAD51', 'CDC45', 'CDC6', 'EXO1', 'TIPIN', 'DSCC1', 'CASP8AP2', 'USP1', 'POLA1', 'CHAF1B', 'E2F8']
    finished: added
    'S_score', score of gene set (adata.obs).
    329 total control genes are used. (0:00:00)
computing score 'G2M_score'
WARNING: genes are not in var_names and ignored: ['TMPO', 'CKAP2L', 'ANP32E', 'HJURP', 'CDCA3', 'NCAPD2', 'CDCA2', 'PSRC1', 'LBR', 'CKAP5', 'CTCF', 'NEK2', 'G2E3', 'CBX5']
    finished: added
    'G2M_score', score of gene set (adata.obs).
    729 total control genes are used. (0:00:00)
-->     'phase', cell cycle phase (adata.obs)
```

Example 75 (unknown):
```unknown
GPU available: False, used: False
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
```

Example 76 (unknown):
```unknown
Epoch 50/50: 100%|███████████| 50/50 [00:31<00:00,  1.60it/s, loss=667, v_num=1]
```

Example 77 (python):
```python
computing score 'stress_score'
WARNING: genes are not in var_names and ignored: ['JUND', 'HSP90AA1', 'HSP90AB1', 'HSPA8', 'HSPB1', 'HSPE1', 'BTG1']
    finished: added
    'stress_score', score of gene set (adata.obs).
    479 total control genes are used. (0:00:00)
computing score 'hb1_score'
WARNING: genes are not in var_names and ignored: ['HBD', 'HBE1', 'HBM', 'HBQ1']
    finished: added
    'hb1_score', score of gene set (adata.obs).
    244 total control genes are used. (0:00:00)
computing neighbors
    finished: added to `.uns['neighbors']`
    `.obsp['distances']`, distances for each pair of neighbors
    `.obsp['connectivities']`, weighted adjacency matrix (0:00:02)
computing UMAP
    finished: added
    'X_umap', UMAP coordinates (adata.obsm) (0:00:05)
```

Example 78 (unknown):
```unknown
... storing 'phase' as categorical
```

Example 79 (unknown):
```unknown
<Figure size 320x320 with 0 Axes>
```

Example 80 (python):
```python
computing Diffusion Maps using n_comps=15(=n_dcs)
computing transitions
    finished (0:00:00)
    eigenvalues of transition matrix
    [1.         0.99514455 0.9879886  0.984966   0.9805877  0.9780712
     0.97638977 0.9711257  0.9674525  0.96672106 0.9656661  0.95888704
     0.9573041  0.9536183  0.95201683]
    finished: added
    'X_diffmap', diffmap coordinates (adata.obsm)
    'diffmap_evals', eigenvalues of transition matrix (adata.uns) (0:00:00)
```

Example 81 (python):
```python
running PAGA
    finished: added
    'paga/connectivities', connectivities adjacency (adata.uns)
    'paga/connectivities_tree', connectivities subtree (adata.uns) (0:00:00)
--> added 'pos', the PAGA positions (adata.uns['paga'])
```

Example 82 (python):
```python
drawing single-cell graph using layout 'fa'
    finished: added
    'X_draw_graph_fa', graph_drawing coordinates (adata.obsm) (0:00:36)
```

Example 83 (python):
```python
#bdata=adata.copy()
```

Example 84 (python):
```python
adata=bdata.copy()
adata
```

Example 85 (unknown):
```unknown
AnnData object with n_obs × n_vars = 8375 × 1332
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score', 'batch'
    var: 'gene_id', 'gene_name_scRNA-0', 'highly_variable', 'means', 'dispersions', 'dispersions_norm', 'highly_variable_nbatches', 'highly_variable_intersection'
    uns: 'log1p', 'hvg', '_scvi_uuid', '_scvi_manager_uuid', 'neighbors', 'umap', 'phase_colors', 'donor_colors', 'age_colors', 'week_colors', 'trimester_colors', 'region_colors', 'cycling_colors', 'cell_or_nuclei_colors', 'modality_colors', 'kit_10x_colors', 'FACSgate_colors', 'heart_or_greatvessels_colors', 'coarse_grain_colors', 'mid_grain_colors', 'fine_grain_colors', 'diffmap_evals', 'paga', 'fine_grain_sizes', 'draw_graph'
    obsm: 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding', 'X_diffmap', 'X_draw_graph_fa'
    layers: 'counts'
    obsp: 'distances', 'connectivities'
```

Example 86 (python):
```python
adata.obs_names
```

Example 87 (unknown):
```unknown
Index(['BHF_F_Hea10402917_TGGCTGGGTTGTGGAG-1-0',
       'BHF_F_Hea10402917_TTTGGTTTCTTTCCTC-1-0',
       'BHF_F_Hea10402917_TGCTGCTCAGTATCTG-1-0',
       'BHF_F_Hea10402917_TCGTAGATCCTATTCA-1-0',
       'BHF_F_Hea10402917_TGGCTGGAGACGCTTT-1-0',
       'BHF_F_Hea10402917_AGAATAGTCTGCAAGT-1-0',
       'BHF_F_Hea10402917_GATCGCGTCATTGCGA-1-0',
       'BHF_F_Hea10402917_ACAGCTATCGGTCTAA-1-0',
       'BHF_F_Hea10402917_GTCGGGTCAAGCCCAC-1-0',
       'BHF_F_Hea10402917_GTCTTCGCATCCCATC-1-0',
       ...
       'GGTCTGGCAAGAGATT_F7-1', 'AGAAGCGTCAGATTGC_F7-1',
       'CACGTTCCAATGGCCC_F7-1', 'CCATAAGTCAAGGAGC_F7-1',
       'TGTAGACCAAAGAACT_F7-1', 'TTCCTCTAGGCACCAA_F7-1',
       'TGCAGGCTCGGTTCAA_F7-1', 'GTTGCTCAGGTTACCT_F7-1',
       'TGGGAAGCAGAGCGTA_F7-1', 'ATCGTAGCAAGTGCAG_F7-1'],
      dtype='object', length=8375)
```

Example 88 (python):
```python
adata=adata[adata.obs.facility=='CSCI',:].copy()
```

Example 89 (python):
```python
adata.obs_names
```

Example 90 (unknown):
```unknown
Index(['CTTCTCTAGCCTCAAT_G4-1', 'CATGCAATCTGAGATC_G4-1',
       'TTTATGCTCCTAACAG_G4-1', 'CAGCACGCAATCGCAT_G4-1',
       'AGAAGCGTCTACTGAG_G4-1', 'TCAAGTGCACTACAGT_G4-1',
       'CCGGGTAAGAGTTGAT_G4-1', 'TTTACCACAGCTGAAG_G4-1',
       'TTGATGGGTCTACAAC_G4-1', 'TAGACCATCCTACTGC_G4-1',
       ...
       'GGTCTGGCAAGAGATT_F7-1', 'AGAAGCGTCAGATTGC_F7-1',
       'CACGTTCCAATGGCCC_F7-1', 'CCATAAGTCAAGGAGC_F7-1',
       'TGTAGACCAAAGAACT_F7-1', 'TTCCTCTAGGCACCAA_F7-1',
       'TGCAGGCTCGGTTCAA_F7-1', 'GTTGCTCAGGTTACCT_F7-1',
       'TGGGAAGCAGAGCGTA_F7-1', 'ATCGTAGCAAGTGCAG_F7-1'],
      dtype='object', length=2697)
```

Example 91 (python):
```python
barcodes = adata.obs.index.tolist()
barcodes = [bc[0:19] for bc in barcodes]
adata.obs.index=barcodes
```

Example 92 (python):
```python
adata.obs_names
```

Example 93 (unknown):
```unknown
Index(['CTTCTCTAGCCTCAAT_G4', 'CATGCAATCTGAGATC_G4', 'TTTATGCTCCTAACAG_G4',
       'CAGCACGCAATCGCAT_G4', 'AGAAGCGTCTACTGAG_G4', 'TCAAGTGCACTACAGT_G4',
       'CCGGGTAAGAGTTGAT_G4', 'TTTACCACAGCTGAAG_G4', 'TTGATGGGTCTACAAC_G4',
       'TAGACCATCCTACTGC_G4',
       ...
       'GGTCTGGCAAGAGATT_F7', 'AGAAGCGTCAGATTGC_F7', 'CACGTTCCAATGGCCC_F7',
       'CCATAAGTCAAGGAGC_F7', 'TGTAGACCAAAGAACT_F7', 'TTCCTCTAGGCACCAA_F7',
       'TGCAGGCTCGGTTCAA_F7', 'GTTGCTCAGGTTACCT_F7', 'TGGGAAGCAGAGCGTA_F7',
       'ATCGTAGCAAGTGCAG_F7'],
      dtype='object', length=2697)
```

Example 94 (unknown):
```unknown
ldata.obs_names
```

Example 95 (unknown):
```unknown
Index(['AACGGGAGTACGATGG_G4', 'AAGAACAGTGATACAA_G4', 'AAGCCATCAGCTCTGG_G4',
       'ACACTGACACAAAGTA_G4', 'ACGATCATCATTACCT_G4', 'ACAGCCGTCACCCTGT_G4',
       'ACAAGCTGTGGATTTC_G4', 'ACTTCGCCATGTGCTA_G4', 'AGATCCATCAAACGTC_G4',
       'AGAACAAAGCACTCAT_G4',
       ...
       'TTGGGATAGCATCAAA_F7', 'TTGCTGCCACTGCACG_F7', 'TTGCATTTCACCTGGG_F7',
       'TTGTTGTGTGATGTAA_F7', 'TTGTGGAAGCTAGTTC_F7', 'TTTGGTTGTATCGTGT_F7',
       'TTTCGATTCAGACCGC_F7', 'TTTGGAGAGGCAGGGA_F7', 'TTTGGAGTCACCGGGT_F7',
       'TTTAGTCCAAGCAGGT_F7'],
      dtype='object', length=2691)
```

Example 96 (python):
```python
a=list(set(adata.obs_names))
b=list(set(ldata.obs_names))
BarcodesToRemove=a+b
BarcodesToRemove=list(set([x for x in BarcodesToRemove if BarcodesToRemove.count(x) == 1]))
BarcodesToRemove
```

Example 97 (unknown):
```unknown
['GCCCAGATCGACGAGA_F4',
 'TGCAGGCAGCACGGAT_F4',
 'ATGACCATCTACCCAC_F4',
 'TGTAAGCGTAGCCAGA_F7',
 'CCACTTGAGAAGATCT_F4',
 'ACATCCCAGCTTCTAG_F7']
```

Example 98 (python):
```python
adata=adata[~adata.obs.index.isin(c)]
adata
```

Example 99 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 2691 × 1332
    obs: 'latent_RT_efficiency', 'latent_cell_probability', 'latent_scale', 'sangerID', 'combinedID', 'donor', 'region', 'age', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'donor_by_library-prep', 'multiplexed', 'SOC | status', 'SOC | log_prob_singleton', 'SOC | log_prob_doublet', 'batch_key', '_scvi_batch', 'FACSgate', 'fine_grain', 'mid_grain', 'coarse_grain', 'sex', 'week', 'trimester', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', '_scvi_labels', 'stress_score', 'hb1_score', 'batch'
    var: 'gene_id', 'gene_name_scRNA-0', 'highly_variable', 'means', 'dispersions', 'dispersions_norm', 'highly_variable_nbatches', 'highly_variable_intersection'
    uns: 'log1p', 'hvg', '_scvi_uuid', '_scvi_manager_uuid', 'neighbors', 'umap', 'phase_colors', 'donor_colors', 'age_colors', 'week_colors', 'trimester_colors', 'region_colors', 'cycling_colors', 'cell_or_nuclei_colors', 'modality_colors', 'kit_10x_colors', 'FACSgate_colors', 'heart_or_greatvessels_colors', 'coarse_grain_colors', 'mid_grain_colors', 'fine_grain_colors', 'diffmap_evals', 'paga', 'fine_grain_sizes', 'draw_graph'
    obsm: 'X_scVI', 'X_umap', '_scvi_extra_categorical_covs', '_scvi_extra_continuous', '_scvi_extra_continuous_covs', 'latent_gene_encoding', 'X_diffmap', 'X_draw_graph_fa'
    layers: 'counts'
    obsp: 'distances', 'connectivities'
```

Example 100 (python):
```python
sc.pl.diffmap(adata, color=['NR2F2','RGCC','HEY2','cycling','fine_grain'], ncols=5)
```

Example 101 (python):
```python
sc.pl.dotplot(adata, ['KLF2','SOD1','KLF4','THBD','NOS3'] ,groupby='fine_grain',standard_scale='var',cmap='YlGn')
```

Example 102 (python):
```python
adata = scv.utils.merge(adata, ldata)

scv.logging.print_version()
scv.settings.verbosity = 3  # show errors(0), warnings(1), info(2), hints(3)
scv.settings.presenter_view = True  # set max width size for presenter view
scv.set_figure_params('scvelo')  # for beautified visualization

###BASICS

scv.pl.proportions(adata)

scv.pp.filter_genes(adata, min_shared_counts=20)
scv.pp.normalize_per_cell(adata)
scv.pp.filter_genes_dispersion(adata, n_top_genes=len(adata.var_names))
scv.pp.log1p(adata)
scv.pp.filter_and_normalize(adata, min_shared_counts=20, n_top_genes=len(adata.var_names))
scv.pp.moments(adata, n_pcs=30, n_neighbors=30)

scv.tl.velocity(adata)
scv.tl.velocity_graph(adata)
scv.pl.velocity_embedding(adata, basis='diffmap')
scv.pl.velocity_embedding_grid(adata, basis='diffmap')
scv.pl.velocity_embedding_stream(adata, basis='diffmap')
scv.pl.velocity_embedding_stream(adata, basis='diffmap',color='fine_grain',palette=('Set2'),arrow_color='darkslategray')
scv.pl.velocity_embedding_stream(adata, basis='diffmap',color='cycling',palette=('Set2'),arrow_color='darkslategray')
scv.pl.velocity_embedding(adata, arrow_length=3, arrow_size=2, dpi=120)
```

Example 103 (python):
```python
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
/tmp/ipykernel_79139/1646599308.py in <module>
----> 1 adata = scv.utils.merge(adata, ldata)
      2 
      3 scv.logging.print_version()
      4 scv.settings.verbosity = 3  # show errors(0), warnings(1), info(2), hints(3)
      5 scv.settings.presenter_view = True  # set max width size for presenter view

~/miniconda3/envs/scanpynew/lib/python3.8/site-packages/scvelo/core/_anndata.py in merge(adata, ldata, copy)
    507     """
    508 
--> 509     adata.var_names_make_unique()
    510     ldata.var_names_make_unique()
    511 

~/miniconda3/envs/scanpynew/lib/python3.8/site-packages/anndata/_core/anndata.py in var_names_make_unique(self, join)
   1801     def var_names_make_unique(self, join: str = "-"):
   1802         # Important to go through the setter so obsm dataframes are updated too
-> 1803         self.var_names = utils.make_index_unique(self.var.index, join)
   1804 
   1805     var_names_make_unique.__doc__ = utils.make_index_unique.__doc__

~/miniconda3/envs/scanpynew/lib/python3.8/site-packages/anndata/_core/anndata.py in var_names(self, names)
    890     def var_names(self, names: Sequence[str]):
    891         names = self._prep_dim_index(names, "var")
--> 892         self._set_dim_index(names, "var")
    893 
    894     @property

~/miniconda3/envs/scanpynew/lib/python3.8/site-packages/anndata/_core/anndata.py in _set_dim_index(self, value, attr)
    840         # Assumes _prep_dim_index has been run
    841         if self.is_view:
--> 842             self._init_as_actual(self.copy())
    843         getattr(self, attr).index = value
    844         for v in getattr(self, f"{attr}m").values():

~/miniconda3/envs/scanpynew/lib/python3.8/site-packages/anndata/_core/anndata.py in copy(self, filename)
   1511             else:
   1512                 X = self.X.copy()
-> 1513             return self._mutated_copy(X=X)
   1514         else:
   1515             from .._io import read_h5ad

~/miniconda3/envs/scanpynew/lib/python3.8/site-packages/anndata/_core/anndata.py in _mutated_copy(self, **kwargs)
   1453                 new[key] = kwargs[key]
   1454             else:
-> 1455                 new[key] = getattr(self, key).copy()
   1456         if "X" in kwargs:
   1457             new["X"] = kwargs["X"]

~/miniconda3/envs/scanpynew/lib/python3.8/site-packages/anndata/_core/aligned_mapping.py in copy(self)
     85         d = self._actual_class(self.parent, self._axis)
     86         for k, v in self.items():
---> 87             d[k] = v.copy()
     88         return d
     89 

~/miniconda3/envs/scanpynew/lib/python3.8/site-packages/anndata/_core/aligned_mapping.py in __setitem__(self, key, value)
    149 
    150     def __setitem__(self, key: str, value: V):
--> 151         value = self._validate_value(value, key)
    152         self._data[key] = value
    153 

~/miniconda3/envs/scanpynew/lib/python3.8/site-packages/anndata/_core/aligned_mapping.py in _validate_value(self, val, key)
    210         ):
    211             # Could probably also re-order index if it’s contained
--> 212             raise ValueError(
    213                 f"value.index does not match parent’s axis {self.axes[0]} names"
    214             )

ValueError: value.index does not match parent’s axis 0 names
```

Example 104 (python):
```python
adata = scv.utils.merge(processeddata, ldata)

scv.logging.print_version()
scv.settings.verbosity = 3  # show errors(0), warnings(1), info(2), hints(3)
scv.settings.presenter_view = True  # set max width size for presenter view
scv.set_figure_params('scvelo')  # for beautified visualization

###BASICS

scv.pl.proportions(adata)

scv.pp.filter_genes(adata, min_shared_counts=20)
scv.pp.normalize_per_cell(adata)
scv.pp.filter_genes_dispersion(adata, n_top_genes=len(adata.var_names))
scv.pp.log1p(adata)
scv.pp.filter_and_normalize(adata, min_shared_counts=20, n_top_genes=len(adata.var_names))
scv.pp.moments(adata, n_pcs=30, n_neighbors=30)

scv.tl.velocity(adata)
scv.tl.velocity_graph(adata)
scv.pl.velocity_embedding(adata, basis='umap')
scv.pl.velocity_embedding_grid(adata, basis='umap')
scv.pl.velocity_embedding_stream(adata, basis='umap')
scv.pl.velocity_embedding_stream(adata, basis='umap',color='fine_grain',palette=('Set2'),arrow_color='darkslategray')
scv.pl.velocity_embedding_stream(adata, basis='umap',color='cycling',palette=('Set2'),arrow_color='darkslategray')
scv.pl.velocity_embedding(adata, arrow_length=3, arrow_size=2, dpi=120)
```

Example 105 (unknown):
```unknown
Running scvelo 0.2.4 (python 3.8.11) on 2023-08-31 09:11.
```

Example 106 (unknown):
```unknown
ERROR: XMLRPC request failed [code: -32500]
RuntimeError: PyPI no longer supports 'pip search' (or XML-RPC search). Please use https://pypi.org/search (via a browser) instead. See https://warehouse.pypa.io/api-reference/xml-rpc.html#deprecated-methods for more information.
```

Example 107 (python):
```python
Filtered out 1183 genes that are detected 20 counts (shared).
WARNING: Did not normalize X as it looks processed already. To enforce normalization, set `enforce=True`.
Normalized count data: spliced, unspliced.
Extracted 2490 highly variable genes.
Filtered out 591 genes that are detected 20 counts (shared).
WARNING: Did not normalize X as it looks processed already. To enforce normalization, set `enforce=True`.
WARNING: Did not normalize spliced as it looks processed already. To enforce normalization, set `enforce=True`.
WARNING: Did not normalize unspliced as it looks processed already. To enforce normalization, set `enforce=True`.
Skip filtering by dispersion since number of variables are less than `n_top_genes`.
WARNING: Did not modify X as it looks preprocessed already.
computing PCA
    on highly variable genes
    with n_comps=30
    finished (0:00:00)
computing neighbors
    finished (0:00:00) --> added 
    'distances' and 'connectivities', weighted adjacency matrices (adata.obsp)
computing moments based on connectivities
    finished (0:00:00) --> added 
    'Ms' and 'Mu', moments of un/spliced abundances (adata.layers)
computing velocities
    finished (0:00:00) --> added 
    'velocity', velocity vectors for each individual cell (adata.layers)
computing velocity graph (using 1/32 cores)
```

Example 108 (unknown):
```unknown
0%|          | 0/2691 [00:00<?, ?cells/s]
```

Example 109 (python):
```python
finished (0:00:03) --> added 
    'velocity_graph', sparse matrix with cosine correlations (adata.uns)
computing velocity embedding
    finished (0:00:00) --> added
    'velocity_umap', embedded velocity vectors (adata.obsm)
```

Example 110 (python):
```python
adata = scv.utils.merge(processeddata, ldata)

scv.logging.print_version()
scv.settings.verbosity = 3  # show errors(0), warnings(1), info(2), hints(3)
scv.settings.presenter_view = True  # set max width size for presenter view
scv.set_figure_params('scvelo')  # for beautified visualization

###BASICS

scv.pl.proportions(adata)

scv.pp.filter_genes(adata, min_shared_counts=20)
scv.pp.normalize_per_cell(adata)
scv.pp.filter_genes_dispersion(adata, n_top_genes=len(adata.var_names))
scv.pp.log1p(adata)
scv.pp.filter_and_normalize(adata, min_shared_counts=20, n_top_genes=len(adata.var_names))
scv.pp.moments(adata, n_pcs=30, n_neighbors=30)

scv.tl.velocity(adata)
scv.tl.velocity_graph(adata)
scv.pl.velocity_embedding(adata, basis='draw_graph_fa')
scv.pl.velocity_embedding_grid(adata, basis='draw_graph_fa')
scv.pl.velocity_embedding_stream(adata, basis='draw_graph_fa')
scv.pl.velocity_embedding_stream(adata, basis='draw_graph_fa',color='fine_grain',palette=('Set2'),arrow_color='darkslategray')
scv.pl.velocity_embedding_stream(adata, basis='draw_graph_fa',color='cycling',palette=('Set2'),arrow_color='darkslategray')
scv.pl.velocity_embedding(adata, arrow_length=3, arrow_size=2, dpi=120)
```

Example 111 (unknown):
```unknown
Running scvelo 0.2.4 (python 3.8.11) on 2023-08-31 09:10.
```

Example 112 (unknown):
```unknown
ERROR: XMLRPC request failed [code: -32500]
RuntimeError: PyPI no longer supports 'pip search' (or XML-RPC search). Please use https://pypi.org/search (via a browser) instead. See https://warehouse.pypa.io/api-reference/xml-rpc.html#deprecated-methods for more information.
```

Example 113 (python):
```python
Filtered out 1183 genes that are detected 20 counts (shared).
WARNING: Did not normalize X as it looks processed already. To enforce normalization, set `enforce=True`.
Normalized count data: spliced, unspliced.
Extracted 2490 highly variable genes.
Filtered out 591 genes that are detected 20 counts (shared).
WARNING: Did not normalize X as it looks processed already. To enforce normalization, set `enforce=True`.
WARNING: Did not normalize spliced as it looks processed already. To enforce normalization, set `enforce=True`.
WARNING: Did not normalize unspliced as it looks processed already. To enforce normalization, set `enforce=True`.
Skip filtering by dispersion since number of variables are less than `n_top_genes`.
WARNING: Did not modify X as it looks preprocessed already.
computing PCA
    on highly variable genes
    with n_comps=30
    finished (0:00:00)
computing neighbors
    finished (0:00:00) --> added 
    'distances' and 'connectivities', weighted adjacency matrices (adata.obsp)
computing moments based on connectivities
    finished (0:00:00) --> added 
    'Ms' and 'Mu', moments of un/spliced abundances (adata.layers)
computing velocities
    finished (0:00:00) --> added 
    'velocity', velocity vectors for each individual cell (adata.layers)
computing velocity graph (using 1/32 cores)
```

Example 114 (unknown):
```unknown
0%|          | 0/2691 [00:00<?, ?cells/s]
```

Example 115 (python):
```python
finished (0:00:03) --> added 
    'velocity_graph', sparse matrix with cosine correlations (adata.uns)
computing velocity embedding
    finished (0:00:00) --> added
    'velocity_draw_graph_fa', embedded velocity vectors (adata.obsm)
```

Example 116 (python):
```python
computing velocity embedding
    finished (0:00:00) --> added
    'velocity_umap', embedded velocity vectors (adata.obsm)
```

Example 117 (python):
```python
scv.pl.velocity(adata, [ 'ACTA2'], ncols=2, add_outline=True)

scv.tl.velocity_confidence(adata)
keys = 'velocity_length', 'velocity_confidence'
scv.pl.scatter(adata, c=keys, cmap='coolwarm', perc=[5, 95])
df = adata.obs.groupby('leiden')[keys].mean().T
df.style.background_gradient(cmap='coolwarm', axis=1)
scv.pl.velocity_graph(adata, threshold=.1)

x, y = scv.utils.get_cell_transitions(adata, basis='umap', starting_cell=70)
ax = scv.pl.velocity_graph(adata, c='lightgrey', edge_width=.05, show=False)
ax = scv.pl.scatter(adata, x=x, y=y, s=120, c='ascending', cmap='gnuplot', ax=ax)

scv.tl.velocity_pseudotime(adata)
scv.pl.scatter(adata, color='velocity_pseudotime', cmap='gnuplot')


adata.uns['neighbors']['distances'] = adata.obsp['distances']
adata.uns['neighbors']['connectivities'] = adata.obsp['connectivities']
scv.tl.paga(adata, groups='leiden')
df = scv.get_df(adata, 'paga/transitions_confidence', precision=2).T
df.style.background_gradient(cmap='Blues').format('{:.2g}')
scv.pl.paga(adata, basis='umap', size=50, alpha=.1, min_edge_width=2, node_size_scale=1.5)


### DYNAMICAL MODELLING
scv.tl.recover_dynamics(adata)
scv.tl.velocity(adata, mode='dynamical')
scv.tl.velocity_graph(adata)

df = adata.var
df = df[(df['fit_likelihood'] > .1) & df['velocity_genes'] == True]
kwargs = dict(xscale='log', fontsize=16)
with scv.GridSpec(ncols=3) as pl:
    pl.hist(df['fit_alpha'], xlabel='transcription rate', **kwargs)
    pl.hist(df['fit_beta'] * df['fit_scaling'], xlabel='splicing rate', xticks=[.1, .4, 1], **kwargs)
    pl.hist(df['fit_gamma'], xlabel='degradation rate', xticks=[.1, .4, 1], **kwargs)
scv.get_df(adata, 'fit*', dropna=True).head()

scv.tl.latent_time(adata)
scv.pl.scatter(adata, color='latent_time', color_map='gnuplot', size=80)
top_genes = adata.var['fit_likelihood'].sort_values(ascending=False).index[:300]
scv.pl.heatmap(adata, var_names=top_genes, sortby='latent_time', col_color='leiden', n_convolve=100)
top_genes = adata.var['fit_likelihood'].sort_values(ascending=False).index
scv.pl.scatter(adata, basis=top_genes[:15], ncols=5, frameon=False)

var_names = [ 'ACTA2', 'DCN']
scv.pl.scatter(adata, var_names, frameon=False)
scv.pl.scatter(adata, x='latent_time', y=var_names, frameon=False)

scv.tl.rank_dynamical_genes(adata, groupby='leiden')
df = scv.get_df(adata, 'rank_dynamical_genes/names')
df.head(5)

for cluster in ['0', '1', '2', '3','4']:
    scv.pl.scatter(adata, df[cluster][:10], ylabel=cluster, frameon=False)
    
var_names = [ 'ACTA2', 'MYH11']
scv.tl.differential_kinetic_test(adata, var_names=var_names, groupby='leiden')
```

---

## Variables¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig3_SAN_niche/VisiumHD_CCSaxis_no3_genes.html

**Contents:**
- Variables¶
- Read in data¶
- Binning axis¶
- Annotate bins¶
- Axon guidance genes¶

bin2cell output which has celltypist output as well

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os

import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42
```

Example 2 (python):
```python
# import deprecated
# import session_info
# session_info.show()
```

Example 3 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/tissueaxis.py")
tissueaxis = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = tissueaxis
spec.loader.exec_module(tissueaxis)
```

Example 4 (python):
```python
sc.settings.set_figure_params(dpi=120)
```

Example 5 (python):
```python
adata_dir = '/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects'
sample_id = 'HEA_FOET14880396'
```

Example 6 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk/Aug2024'
table_dir = '/nfs/team205/heart/fetal_ms_figures/tables/Aug2024'
```

Example 7 (python):
```python
# read in 
adata = sc.read_h5ad(f'{adata_dir}/{sample_id}_b2c_cells_filtered_lognorm.h5ad')
adata
```

Example 8 (unknown):
```unknown
AnnData object with n_obs × n_vars = 129803 × 18047
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'fine_grain_4Jan2025', 'HistologicalAnnotation', 'SANaxis_landmarks', 'SAN_axis', 'SAN_axis_bin_annot'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'HistologicalAnnotation_colors', 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'fine_grain_4Jan2025_colors', 'hvg', 'log1p', 'neighbors', 'pca', 'spatial', 'umap'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 9 (unknown):
```unknown
celltype_key = 'fine_grain_4Jan2025'
```

Example 10 (unknown):
```unknown
axis_name = 'SAN_axis'
```

Example 11 (python):
```python
# make bin
adata.obs[f'{axis_name}_bin'] = pd.cut(adata.obs[axis_name],bins=10)
```

Example 12 (python):
```python
set(adata.obs[f'{axis_name}_bin'])
```

Example 13 (unknown):
```unknown
{Interval(-0.2, 0.0, closed='right'),
 Interval(-0.4, -0.2, closed='right'),
 Interval(-0.6, -0.4, closed='right'),
 Interval(-0.8, -0.6, closed='right'),
 Interval(-1.002, -0.8, closed='right'),
 Interval(0.0, 0.2, closed='right'),
 Interval(0.2, 0.4, closed='right'),
 Interval(0.4, 0.6, closed='right'),
 Interval(0.6, 0.8, closed='right'),
 Interval(0.8, 1.0, closed='right'),
 nan}
```

Example 14 (python):
```python
bdata = adata[adata.obs[axis_name].isna()==False]
bdata
```

Example 15 (unknown):
```unknown
View of AnnData object with n_obs × n_vars = 1510 × 18047
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'fine_grain_4Jan2025', 'HistologicalAnnotation', 'SANaxis_landmarks', 'SAN_axis', 'SAN_axis_bin_annot', 'SAN_axis_bin'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'HistologicalAnnotation_colors', 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'fine_grain_4Jan2025_colors', 'hvg', 'log1p', 'neighbors', 'pca', 'spatial', 'umap'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 16 (python):
```python
sc.pp.filter_genes(bdata, min_cells=3)
bdata.shape
```

Example 17 (unknown):
```unknown
(1510, 14912)
```

Example 18 (python):
```python
# annotate bin
annot_dict = {
    '(-1.002, -0.8]':'SinusHorn_1',
     '(-0.8, -0.6]':'SinusHorn_2',
     '(-0.6, -0.4]':'SinusHorn_3',
     '(-0.4, -0.2]':'SinusHorn_4',
     '(-0.2, 0.0]':'SAN_head_1',
     '(0.0, 0.2]':'SAN_head_2',
     '(0.2, 0.4]':'SAN_head_3',
     '(0.4, 0.6]':'SAN_tail_1',
     '(0.6, 0.8]':'SAN_tail_2',
     '(0.8, 1.0]':'SAN_tail_3',
}
bdata.obs[f'{axis_name}_bin_annot'] = bdata.obs[f'{axis_name}_bin'].astype('str').copy()
bdata.obs.replace({f'{axis_name}_bin_annot':annot_dict}, inplace=True)
bdata.obs[f'{axis_name}_bin_annot'] = bdata.obs[f'{axis_name}_bin_annot'].astype('category')
bdata.obs[f'{axis_name}_bin_annot'] = bdata.obs[f'{axis_name}_bin_annot'].cat.reorder_categories(list(annot_dict.values()))
bdata.obs[f'{axis_name}_bin_annot'].cat.categories
```

Example 19 (unknown):
```unknown
Index(['SinusHorn_1', 'SinusHorn_2', 'SinusHorn_3', 'SinusHorn_4',
       'SAN_head_1', 'SAN_head_2', 'SAN_head_3', 'SAN_tail_1', 'SAN_tail_2',
       'SAN_tail_3'],
      dtype='object')
```

Example 20 (python):
```python
df = bdata.obs[[celltype_key]].copy()
df['count']=1
df.reset_index(inplace=True)
df = pd.pivot_table(df, values='count', index=['index'],
                       columns=[celltype_key])
df.fillna(0,inplace=True)
df[f'{axis_name}_bin']=bdata.obs.loc[df.index,f'{axis_name}_bin_annot']
df = df.groupby(f'{axis_name}_bin').sum()
max_arr = df.max(axis=1)
for col in df.columns:
    df[col] = df[col]/max_arr
plt.rcParams['axes.grid'] = False
plt.rcParams['figure.figsize'] = 3,5
sns.heatmap(df)
```

Example 21 (python):
```python
/tmp/ipykernel_1580563/441468744.py:4: FutureWarning: The default value of observed=False is deprecated and will change to observed=True in a future version of pandas. Specify observed=False to silence this warning and retain the current behavior
  df = pd.pivot_table(df, values='count', index=['index'],
/tmp/ipykernel_1580563/441468744.py:8: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning.
  df = df.groupby(f'{axis_name}_bin').sum()
```

Example 22 (unknown):
```unknown
<Axes: xlabel='fine_grain_4Jan2025', ylabel='SAN_axis_bin'>
```

Example 23 (python):
```python
# add annotation to original adata
adata.obs[f'{axis_name}_bin_annot'] = 'nan'
adata.obs.loc[bdata.obs_names, f'{axis_name}_bin_annot'] = bdata.obs[f'{axis_name}_bin_annot']
adata.obs[f'{axis_name}_bin_annot'] = adata.obs[f'{axis_name}_bin_annot'].astype('category')
adata.obs[f'{axis_name}_bin_annot'] = adata.obs[f'{axis_name}_bin_annot'].cat.reorder_categories(list(annot_dict.values())+['nan'])
adata.obs[f'{axis_name}_bin_annot'].cat.categories
```

Example 24 (unknown):
```unknown
Index(['SinusHorn_1', 'SinusHorn_2', 'SinusHorn_3', 'SinusHorn_4',
       'SAN_head_1', 'SAN_head_2', 'SAN_head_3', 'SAN_tail_1', 'SAN_tail_2',
       'SAN_tail_3', 'nan'],
      dtype='object')
```

Example 25 (python):
```python
# save
adata.obs.drop([f'{axis_name}_bin'],axis=1,inplace=True)
adata.write(f'{adata_dir}/{sample_id}_b2c_cells_filtered_lognorm.h5ad')
adata
```

Example 26 (python):
```python
# from embryonic/foetal vs adult comparison
genes = ['SEMA3A', 'SLIT2', 'SLIT3', 'EFNB2', 'CXCL12', 'FGF10', 'VEGFC', 'NELL1']
genes = [x for x in genes if x in bdata.var_names]
print(len(genes))
```

Example 27 (unknown):
```unknown
tissueaxis.plot_axis(bdata,
              axis_col=axis_name,
              binned_col=f'{axis_name}_bin_annot',
              feature_type='gene', # 'gene' or 'celltype'
              features_to_plot=genes,
              features_to_remove=None,
              threshold_express_prop=0.1, # at quantile=0.9
              threshold_cell_abundance=None, # at quantile=0.9
              correlation=False,
              min_max_scale=True,
              row_cluster=True,
              figsize=(4,4),
              cbar_pos=(-0, 0.65, 0.03, 0.15),
              cmap='viridis',
          xticklabels=True,
          yticklabels=True,
              title=None,
              return_feature = False,
         save=f'{figdir}/VisiumHD_SANaxis_axon-guidance-genes.pdf',
    save_dpi=300,
             )
```

Example 28 (unknown):
```unknown
/nfs/team205/kk18/function/python/tissueaxis.py:57: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning.
  df_gene_exp_prop = df_gene_exp_bi.groupby(f'{axis_col}_binned').mean()
/nfs/team205/kk18/function/python/tissueaxis.py:134: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning.
  df_plot = df_plot.set_index('index').groupby(['variable',f'{axis_col}_binned']).mean().reset_index()
```

---

## Import modules¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/RNA/QC/Foetal_T21-Multiome-RNA_No1_make-anndata_run-scrublet.html

**Contents:**
- Import modules¶
- Functions¶
- Create anndata object¶
- Run scrublet¶
- Save¶

190750 rows × 21 columns

**Examples:**

Example 1 (python):
```python
import numpy as np
import pandas as pd
import scanpy as sc
import matplotlib.pyplot as plt
import scrublet as scr
import session_info
```

Example 2 (python):
```python
sc.settings.verbosity = 3
sc.settings.set_figure_params(dpi = 160, color_map = 'RdPu', dpi_save = 180, vector_friendly = True, format = 'svg')
session_info.show()
```

Example 3 (python):
```python
-----
matplotlib          3.3.4
numpy               1.20.1
pandas              1.1.4
scanpy              1.7.1
scrublet            NA
session_info        1.0.0
-----
```

Example 4 (unknown):
```unknown
PIL                 8.1.2
anndata             0.7.5
anyio               NA
attr                20.3.0
babel               2.9.0
backcall            0.2.0
brotli              NA
cairo               1.20.0
certifi             2020.12.05
cffi                1.14.5
chardet             4.0.0
cloudpickle         1.6.0
colorama            0.4.4
cycler              0.10.0
cython_runtime      NA
cytoolz             0.11.0
dask                2021.03.1
dateutil            2.8.1
decorator           4.4.2
fsspec              0.8.7
get_version         2.1
google              NA
h5py                2.10.0
idna                2.10
igraph              0.8.3
ipykernel           5.5.0
ipython_genutils    0.2.0
jedi                0.18.0
jinja2              2.11.3
joblib              1.0.1
json5               NA
jsonschema          3.2.0
jupyter_server      1.4.1
jupyterlab_server   2.3.0
kiwisolver          1.3.1
legacy_api_wrap     0.0.0
leidenalg           0.8.3
llvmlite            0.34.0
louvain             0.7.0
markupsafe          1.1.1
mpl_toolkits        NA
natsort             7.1.1
nbclassic           NA
nbformat            5.1.2
numba               0.51.2
numexpr             2.7.3
packaging           20.9
parso               0.8.1
pexpect             4.8.0
pickleshare         0.7.5
pkg_resources       NA
prometheus_client   NA
prompt_toolkit      3.0.16
psutil              5.8.0
ptyprocess          0.7.0
pvectorc            NA
pyarrow             0.16.0
pygments            2.8.0
pyparsing           2.4.7
pyrsistent          NA
pytoml              NA
pytz                2021.1
requests            2.25.1
ruamel              NA
scipy               1.6.1
send2trash          NA
setuptools_scm      NA
simplejson          3.17.2
sinfo               0.3.1
six                 1.15.0
sklearn             0.22
sniffio             1.2.0
socks               1.7.1
sphinxcontrib       NA
storemagic          NA
tables              3.6.1
tblib               1.7.0
texttable           1.6.3
tlz                 0.11.0
toolz               0.11.1
tornado             6.1
traitlets           5.0.5
typing_extensions   NA
uritemplate         3.0.1
urllib3             1.26.3
wcwidth             0.2.5
yaml                5.1.2
zmq                 22.0.3
```

Example 5 (unknown):
```unknown
-----
IPython             7.21.0
jupyter_client      6.1.11
jupyter_core        4.7.1
jupyterlab          3.0.9
notebook            6.2.0
-----
Python 3.8.8 | packaged by conda-forge | (default, Feb 20 2021, 16:22:27) [GCC 9.3.0]
Linux-4.15.0-112-generic-x86_64-with-glibc2.10
-----
Session information updated at 2023-03-14 16:45
```

Example 6 (python):
```python
"""Functions for downstream work with outputs of remove-background."""

# to read cellbender outputs<br>
# https://github.com/broadinstitute/CellBender/issues/57

import tables
import numpy as np
import scipy.sparse as sp
from typing import Dict


def dict_from_h5(file: str) -> Dict[str, np.ndarray]:
    """Read in everything from an h5 file and put into a dictionary."""
    d = {}
    with tables.open_file(file) as f:
        # read in everything
        for array in f.walk_nodes("/", "Array"):
            d[array.name] = array.read()
    return d


def anndata_from_h5(file: str,
                    analyzed_barcodes_only: bool = True) -> 'anndata.AnnData':
    """Load an output h5 file into an AnnData object for downstream work.

    Args:
        file: The h5 file
        analyzed_barcodes_only: False to load all barcodes, so that the size of
            the AnnData object will match the size of the input raw count matrix.
            True to load a limited set of barcodes: only those analyzed by the
            algorithm. This allows relevant latent variables to be loaded
            properly into adata.obs and adata.obsm, rather than adata.uns.

    Returns:
        adata: The anndata object, populated with inferred latent variables
            and metadata.

    """

    try:
        import anndata
    except ImportError:
        raise ImportError('The anndata package must be installed to use the '
                          'function anndata_from_h5()')

    d = dict_from_h5(file)
    X = sp.csc_matrix((d.pop('data'), d.pop('indices'), d.pop('indptr')),
                      shape=d.pop('shape')).transpose().tocsr()

    if analyzed_barcodes_only:
        if 'barcodes_analyzed_inds' in d.keys():
            X = X[d['barcodes_analyzed_inds'], :]
            d['barcodes'] = d['barcodes'][d['barcodes_analyzed_inds']]
        elif 'barcode_indices_for_latents' in d.keys():
            X = X[d['barcode_indices_for_latents'], :]
            d['barcodes'] = d['barcodes'][d['barcode_indices_for_latents']]
        else:
            print('Warning: analyzed_barcodes_only=True, but the key '
                  '"barcodes_analyzed_inds" or "barcode_indices_for_latents" '
                  'is missing from the h5 file. '
                  'Will output all barcodes, and proceed as if '
                  'analyzed_barcodes_only=False')

    # Construct the count matrix.
    adata = anndata.AnnData(X=X,
                            obs={'barcode': d.pop('barcodes').astype(str)},
                            var={'gene_name': (d.pop('gene_names') if 'gene_names' in d.keys()
                                               else d.pop('name')).astype(str)})
    adata.obs.set_index('barcode', inplace=True)
    adata.var.set_index('gene_name', inplace=True)

    # Add other information to the adata object in the appropriate slot.
    for key, value in d.items():
        try:
            value = np.asarray(value)
            if len(value.shape) == 0:
                adata.uns[key] = value
            elif value.shape[0] == X.shape[0]:
                if (len(value.shape) < 2) or (value.shape[1] < 2):
                    adata.obs[key] = value
                else:
                    adata.obsm[key] = value
            elif value.shape[0] == X.shape[1]:
                if value.dtype.name.startswith('bytes'):
                    adata.var[key] = value.astype(str)
                else:
                    adata.var[key] = value
            else:
                adata.uns[key] = value
        except Exception:
            print('Unable to load data into AnnData: ', key, value, type(value))

    if analyzed_barcodes_only:
        for col in adata.obs.columns[adata.obs.columns.str.startswith('barcodes_analyzed')
                                     | adata.obs.columns.str.startswith('barcode_indices')]:
            try:
                del adata.obs[col]
            except Exception:
                pass

    return adata
```

Example 7 (python):
```python
# from https://github.com/Teichlab/mapcloud/blob/master/scripts/starsolo/postprocess.py

from statsmodels.stats.multitest import multipletests
# from emptydrops.matrix import CountMatrix
# from emptydrops import find_nonambient_barcodes
import scrublet as scr
import scanpy as sc
import pandas as pd
import numpy as np
import scipy

#some functions that Ni uses in scanpy scripts to run scrublet
#which in turn are inspired by my original notebook on the matter
#(extracted from scanpy_scripts 0.2.10 to get around scanpy version incompatibility)
def test_outlier(x, upper_mad_only=True):
	med = np.median(x)
	if upper_mad_only:
		mad = np.median(x[x>med] - med) * 1.4826
	else:
		mad = np.median(np.abs(x - med)) * 1.4826
	pvals = 1 - scipy.stats.norm.cdf(x, loc=med, scale=mad)
	bh_pvals = multipletests(pvals, method='fdr_bh')[1]
	return pvals, bh_pvals

def run_scrublet(adata, resolution_function=None):
	old_verbosity = sc.settings.verbosity
	sc.settings.verbosity = 1
	if resolution_function is None:
		resolution_function = lambda x: np.maximum(np.maximum(np.log10(x)-1, 0)**2, 0.1)
	scrub = scr.Scrublet(adata.X)
	#this has the potential to brick for poor quality data
	#if so, abort it and everything downstream
	try:
		ds, pd = scrub.scrub_doublets(verbose=False)
	except:
		return
	adata.obs['scrublet_score'] = ds

	adata_copy = adata.copy()
	sc.pp.filter_genes(adata_copy, min_cells=3)
	sc.pp.normalize_total(adata_copy, target_sum=1e4)
	sc.pp.log1p(adata_copy)
	sc.pp.highly_variable_genes(adata_copy, min_mean=0.0125, max_mean=3, min_disp=0.5, subset=True)
	sc.pp.scale(adata_copy, zero_center=False)
	sc.pp.pca(adata_copy, svd_solver='arpack', zero_center=False)
	sc.pp.neighbors(adata_copy, n_pcs=30)
	sc.tl.umap(adata_copy)
	sc.tl.leiden(adata_copy, resolution=1)
	for clst in np.unique(adata_copy.obs['leiden']):
		clst_size = sum(adata_copy.obs['leiden'] == clst)
		sc.tl.leiden(adata_copy, restrict_to=('leiden', [clst]), resolution=resolution_function(clst_size), key_added='leiden_R')
		adata_copy.obs['leiden'] = adata_copy.obs['leiden_R']
	clst_meds = []
	for clst in np.unique(adata_copy.obs['leiden']):
		k = adata_copy.obs['leiden'] == clst
		clst_med = np.median(adata_copy.obs.loc[k, 'scrublet_score'])
		adata_copy.obs.loc[k, 'cluster_scrublet_score'] = clst_med
		clst_meds.append(clst_med)
	clst_meds = np.array(clst_meds)
	pvals, bh_pvals = test_outlier(clst_meds)
	for i, clst in enumerate(np.unique(adata_copy.obs['leiden'])):
		k = adata_copy.obs['leiden'] == clst
		adata_copy.obs.loc[k, 'pval'] = pvals[i]
		adata_copy.obs.loc[k, 'bh_pval'] = bh_pvals[i]
	sc.settings.verbosity = old_verbosity
	#need to also export the clustering, for soupx purposes
	adata.obs['scrublet_leiden'] = adata_copy.obs['leiden']
	adata.obs['scrublet_score'] = adata_copy.obs['scrublet_score']
	adata.obs['cluster_scrublet_score'] = adata_copy.obs['cluster_scrublet_score']
	adata.obs['doublet_pval'] = adata_copy.obs['pval']
	adata.obs['doublet_bh_pval'] = adata_copy.obs['bh_pval']
	del adata_copy
```

Example 8 (python):
```python
# read in metadata
metadata = pd.read_csv('/nfs/team205/heart/anndata_objects/Foetal/metadata/HeartTeamSamples_Mappeddata_20230314.csv', sep = ',', index_col = None)
metadata = metadata[metadata['Publication']=='Foetal']

# select healthy data
metadata = metadata[metadata['status']=='Trisomy21']


print(metadata['modality'].value_counts())
```

Example 9 (unknown):
```unknown
Multiome-ATAC    8
Multiome-RNA     8
Visium           4
Name: modality, dtype: int64
```

Example 10 (python):
```python
# select modality
metadata = metadata[metadata['modality']=='Multiome-RNA']
metadata[['sangerID','combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age',
                'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x','flushed']]
```

Example 11 (python):
```python
########### editted: put id to var_names ###########

# read in
adatas=[]
for i in range(len(metadata)):
    print(metadata.iloc[i]['combinedID'])
    
    path=metadata.iloc[i]['CellBender_out']
    adatas.append(anndata_from_h5(path + '/' + path.split('/')[-1] + '_cellbender_out_filtered.h5', analyzed_barcodes_only=False))
    del path
    
    # replace var_names with ensembleID
    adatas[i].var.reset_index(inplace=True)
    if 'id' in adatas[i].var.columns:
        adatas[i].var.set_index('id',inplace=True)
    else:
        adatas[i].var.set_index('genes',inplace=True)
    
    # modify barcodes
    adatas[i].obs.index = metadata.iloc[i]['combinedID'] + '_' + adatas[i].obs.index 
    
    # add metadata
    for col in ['sangerID','combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age',
                'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x','flushed']:
        adatas[i].obs[col] = metadata.iloc[i][col]


# concatenate
adata = adatas[0].concatenate(adatas[1:], index_unique = None, batch_key=None)
adata.shape
```

Example 12 (unknown):
```unknown
BHF_F_Hea13188319_BHF_F_Hea13187621
BHF_F_Hea13188320_BHF_F_Hea13187622
BHF_F_Hea13188321_BHF_F_Hea13187623
BHF_F_Hea13188322_BHF_F_Hea13187624
BHF_F_Hea13188323_BHF_F_Hea13187625
BHF_F_Hea13188325_BHF_F_Hea13187627
BHF_F_Hea13188326_BHF_F_Hea13187628
BHF_F_Hea13188327_BHF_F_Hea13187629
```

Example 13 (unknown):
```unknown
(190750, 36601)
```

Example 14 (python):
```python
for i in range(len(adatas)):
    print(adatas[i].var.columns)
```

Example 15 (unknown):
```unknown
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
Index(['gene_name', 'ambient_expression', 'feature_type'], dtype='object')
```

Example 16 (python):
```python
adata.obs['combinedID'].value_counts()
```

Example 17 (unknown):
```unknown
BHF_F_Hea13188319_BHF_F_Hea13187621    50000
BHF_F_Hea13188321_BHF_F_Hea13187623    38944
BHF_F_Hea13188325_BHF_F_Hea13187627    23862
BHF_F_Hea13188322_BHF_F_Hea13187624    17452
BHF_F_Hea13188323_BHF_F_Hea13187625    17143
BHF_F_Hea13188327_BHF_F_Hea13187629    16377
BHF_F_Hea13188320_BHF_F_Hea13187622    14000
BHF_F_Hea13188326_BHF_F_Hea13187628    12972
Name: combinedID, dtype: int64
```

Example 18 (python):
```python
%%time

# per rxn
for i,ID in enumerate(adata.obs['combinedID'].unique()):
    print(ID)
    
    ad = adata[adata.obs['combinedID'] == ID].copy()
    run_scrublet(ad)
    if i==0:
        meta = ad.obs
    else:
        meta = pd.concat([meta, ad.obs])
    del ad
```

Example 19 (unknown):
```unknown
BHF_F_Hea13188319_BHF_F_Hea13187621
BHF_F_Hea13188320_BHF_F_Hea13187622
```

Example 20 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 21 (unknown):
```unknown
BHF_F_Hea13188321_BHF_F_Hea13187623
```

Example 22 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 23 (unknown):
```unknown
BHF_F_Hea13188322_BHF_F_Hea13187624
```

Example 24 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 25 (unknown):
```unknown
BHF_F_Hea13188323_BHF_F_Hea13187625
```

Example 26 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 27 (unknown):
```unknown
BHF_F_Hea13188325_BHF_F_Hea13187627
```

Example 28 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 29 (unknown):
```unknown
BHF_F_Hea13188326_BHF_F_Hea13187628
```

Example 30 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 31 (unknown):
```unknown
BHF_F_Hea13188327_BHF_F_Hea13187629
```

Example 32 (unknown):
```unknown
/opt/conda/envs/teichlab/lib/python3.8/site-packages/scrublet/helper_functions.py:321: RuntimeWarning: divide by zero encountered in true_divide
  w.setdiag(float(target_total) / tots_use)
WARNING: Some cells have total count of genes equal to zero
```

Example 33 (unknown):
```unknown
CPU times: user 38min 57s, sys: 18min 16s, total: 57min 13s
Wall time: 17min 13s
```

Example 34 (python):
```python
meta_scrub = meta[['scrublet_score', 'scrublet_leiden', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval']].copy()
meta_scrub.shape
```

Example 35 (unknown):
```unknown
(190750, 5)
```

Example 36 (python):
```python
if meta_scrub.reindex(adata.obs.index).index.equals(adata.obs.index):
    adata.obs = pd.concat([adata.obs, meta_scrub.reindex(adata.obs.index)], axis=1)
else:
    raise Exception('Different barcodes in meta and adata')
```

Example 37 (python):
```python
adata.write('/nfs/team205/heart/anndata_objects/Foetal/QC/T21_multiome_RNA_foetal_prefilter.h5ad')
```

Example 38 (unknown):
```unknown
... storing 'sangerID' as categorical
... storing 'combinedID' as categorical
... storing 'donor' as categorical
... storing 'donor_type' as categorical
... storing 'region' as categorical
... storing 'region_finest' as categorical
... storing 'age' as categorical
... storing 'gender' as categorical
... storing 'facility' as categorical
... storing 'cell_or_nuclei' as categorical
... storing 'modality' as categorical
... storing 'kit_10x' as categorical
... storing 'flushed' as categorical
... storing 'scrublet_leiden' as categorical
... storing 'gene_name' as categorical
... storing 'feature_type' as categorical
```

Example 39 (python):
```python
sc.pl.violin(adata,
             ['scrublet_score', 'cluster_scrublet_score', 'doublet_pval', 'doublet_bh_pval'],jitter = 0, multi_panel = True)
```

Example 40 (python):
```python
adata.obs['donor'].value_counts()
```

Example 41 (unknown):
```unknown
Hst42    137539
Hst44     53211
Name: donor, dtype: int64
```

---

## Variables¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig3_SAN_niche/VisiumHD_CCSaxis_no4_neighbouring-cells.html

**Contents:**
- Variables¶
- Read in data¶
- Detect spatial neighbours¶

bin2cell output which has celltypist output as well

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os

import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 

import squidpy as sq
```

Example 2 (python):
```python
/software/cellgen/team205/kk18/envs/squidpy/lib/python3.10/site-packages/dask/dataframe/__init__.py:31: FutureWarning: The legacy Dask DataFrame implementation is deprecated and will be removed in a future version. Set the configuration option `dataframe.query-planning` to `True` or None to enable the new Dask Dataframe implementation and silence this warning.
  warnings.warn(
/software/cellgen/team205/kk18/envs/squidpy/lib/python3.10/site-packages/numba/core/decorators.py:246: RuntimeWarning: nopython is set for njit and is ignored
  warnings.warn('nopython is set for njit and is ignored', RuntimeWarning)
/software/cellgen/team205/kk18/envs/squidpy/lib/python3.10/site-packages/anndata/utils.py:429: FutureWarning: Importing read_text from `anndata` is deprecated. Import anndata.io.read_text instead.
  warnings.warn(msg, FutureWarning)
```

Example 3 (python):
```python
# import session_info
# session_info.show()
```

Example 4 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/tissueaxis.py")
tissueaxis = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = tissueaxis
spec.loader.exec_module(tissueaxis)
```

Example 5 (python):
```python
sc.settings.set_figure_params(dpi=120)
```

Example 6 (python):
```python
adata_dir = '/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects'
sample_id = 'HEA_FOET14880396'
```

Example 7 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk/Aug2024'
table_dir = '/nfs/team205/heart/fetal_ms_figures/tables/Aug2024'
```

Example 8 (unknown):
```unknown
celltype_key = 'fine_grain_4Jan2025'
axis_name = 'SAN_axis'
```

Example 9 (python):
```python
# read in 
adata = sc.read_h5ad(f'{adata_dir}/{sample_id}_b2c_cells_filtered_lognorm.h5ad')
adata
```

Example 10 (unknown):
```unknown
AnnData object with n_obs × n_vars = 129803 × 18047
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'celltypist_coarse', 'conf_score_coarse', 'celltypist_coarse_fin', 'celltypist_coarse2midmod', 'conf_score_coarse2midmod', 'celltypist_mid_fin', 'celltypist_midmod2fine', 'conf_score_midmod2fine', 'fine_grain_4Jan2025', 'HistologicalAnnotation', 'SANaxis_landmarks', 'SAN_axis', 'SAN_axis_bin_annot'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
    uns: 'HistologicalAnnotation_colors', 'celltypist_coarse2midmod_colors', 'celltypist_coarse_colors', 'celltypist_coarse_fin_colors', 'celltypist_mid_fin_colors', 'celltypist_midmod2fine_colors', 'fine_grain_4Jan2025_colors', 'hvg', 'log1p', 'neighbors', 'pca', 'spatial', 'umap'
    obsm: 'X_pca', 'X_umap', 'spatial', 'spatial_cropped_150_buffer'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'
```

Example 11 (python):
```python
# celltype name mapping for plotting
df = pd.read_csv('/nfs/team205/heart/anndata_objects/Foetal/finegrain_name_mapping.csv')
celltype_mapping = df.set_index('Full_name')['Short_name_finalised'].to_dict()
adata.obs.replace({'fine_grain_4Jan2025':celltype_mapping},inplace=True)
```

Example 12 (python):
```python
/tmp/ipykernel_1580522/3810385248.py:4: FutureWarning: The behavior of Series.replace (and DataFrame.replace) with CategoricalDtype is deprecated. In a future version, replace will only be used for cases that preserve the categories. To change the categories, use ser.cat.rename_categories instead.
  adata.obs.replace({'fine_grain_4Jan2025':celltype_mapping},inplace=True)
```

Example 13 (python):
```python
%%time
sq.gr.spatial_neighbors(adata, coord_type="generic", n_neighs=20, spatial_key="spatial")
```

Example 14 (unknown):
```unknown
CPU times: user 988 ms, sys: 122 ms, total: 1.11 s
Wall time: 1.14 s
```

Example 15 (python):
```python
spatial_connect = adata.obsp['spatial_connectivities'].copy()
```

Example 16 (python):
```python
# mask for cells which has macrophage pseudotime
mask = adata.obs[f'{axis_name}_bin_annot']!='nan'
```

Example 17 (python):
```python
spatial_connect[:,mask].shape
```

Example 18 (unknown):
```unknown
(129803, 1510)
```

Example 19 (unknown):
```unknown
# Step 1: Transpose the sparse matrix
sparse_matrix_T = spatial_connect[:,mask].T
```

Example 20 (python):
```python
from tqdm import tqdm
```

Example 21 (python):
```python
# Step 2: Initialize a dictionary to store neighbor cell type counts for each pseudotime cell
neighbor_cell_types = {i: [] for i in range(sparse_matrix_T.shape[0])}
# Step 3: Identify neighbors for each pseudotime cell and aggregate cell types
cell_types = list(adata.obs[celltype_key])
for i in tqdm(range(sparse_matrix_T.shape[0])):
    neighbors = np.nonzero(sparse_matrix_T[i])[1]  # Get indices of neighbors
    neighbor_types = [cell_types[j] for j in neighbors]
    neighbor_cell_types[i].extend(neighbor_types)
# Step 4: Convert to DataFrame for easier analysis
neighbor_df = pd.DataFrame.from_dict(neighbor_cell_types, orient='index')
# Calculate frequency or proportion of each cell type for each pseudotime
cell_type_counts = neighbor_df.apply(pd.Series.value_counts, axis=1).fillna(0)
# add pseudotime
cell_type_counts.index = adata.obs_names[mask]
```

Example 22 (unknown):
```unknown
100%|██████████| 1510/1510 [00:01<00:00, 1035.96it/s]
```

Example 23 (python):
```python
cell_type_counts[f'{axis_name}_bin_annot']=adata.obs.loc[cell_type_counts.index,f'{axis_name}_bin_annot']
df = cell_type_counts.groupby(f'{axis_name}_bin_annot').mean()
df = df.dropna()
df.max().sort_values(ascending=False)
```

Example 24 (unknown):
```unknown
/tmp/ipykernel_1580522/1380053985.py:2: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning.
  df = cell_type_counts.groupby(f'{axis_name}_bin_annot').mean()
```

Example 25 (unknown):
```unknown
SANPCHd       13.281853
SANPCHn       13.113043
SANPCT        10.009346
aCMR           6.728571
unassigned     5.616822
FB             4.430769
AVNPC          2.859813
GVVenEC        1.127660
Endocard       1.014286
ParaN          0.574468
MesEpiC        0.430769
CorPeri        0.343629
MacCX3         0.324324
aCML           0.288889
CorCapEC       0.287425
SchwC          0.148936
MPLYVE1        0.142857
LymphEC        0.121739
GVSMCT         0.107692
SchwNeuro      0.095745
ValveIC        0.073333
GVSMC          0.046332
EPDC           0.034749
vCMC           0.034749
PericFib       0.023166
ValveEC        0.014286
dtype: float64
```

Example 26 (python):
```python
cell_type_counts[f'{axis_name}_bin_annot']=adata.obs.loc[cell_type_counts.index,f'{axis_name}_bin_annot']
df = cell_type_counts.groupby(f'{axis_name}_bin_annot').mean()
df = df.dropna()


mean_thresh = 0.1
df = df[df.columns[df.max()>mean_thresh]].T

# normalise with max value
max_arr = df.max(axis=1)
for col in df.columns:
    df[col] = df[col]/max_arr

# remove "unclassified"
df = df[df.index!='unassigned']
```

Example 27 (unknown):
```unknown
/tmp/ipykernel_1580522/3071229161.py:2: FutureWarning: The default of observed=False is deprecated and will be changed to True in a future version of pandas. Pass observed=False to retain current behavior or observed=True to adopt the future default and silence this warning.
  df = cell_type_counts.groupby(f'{axis_name}_bin_annot').mean()
```

Example 28 (python):
```python
plt.rcParams['axes.grid'] = False
sns.clustermap(df,col_cluster=False,figsize=(3.5,6),
               dendrogram_ratio=0.05,cbar_pos=(-0.05, 0.8, 0.02, 0.1))
plt.savefig(f'{figdir}/VisiumHD_SANaxis_celltypes.pdf', bbox_inches="tight",dpi=300)
plt.show()
```

---

## Import libraries¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig4_Cardiomyocytes/VentricularCardiomyocytes/CellOracle/CellOracle_no2_vCM-Left_compact-common-TF_in-silico-perturbation.html

**Contents:**
- Import libraries¶
- Make out directory¶
- Load oracle object¶
- In silico TF perturbation analysis¶
- Calculating pseudotime gradient vectors¶
- Perturbation¶

ref: https://morris-lab.github.io/CellOracle.documentation/notebooks/04_Network_analysis/Network_analysis_with_Paul_etal_2015_data.html

**Examples:**

Example 1 (unknown):
```unknown
# Parameters
working_dir = "/nfs/team205/heart/Vitro/TF_selection/foetal_vCM_left_compact-vs-trabeculated_downstream/CO_in-silico-perturbation_papermill"
path_to_oracle_object = "/nfs/team205/heart/anndata_objects/Foetal/vCM/post_scFate/foetal_vCM_onlyLeft.celloracle.oracle"
path_to_oracle_links = "/nfs/team205/heart/anndata_objects/Foetal/vCM/post_scFate/vCM_onlyLeft_links.celloracle.links"
n_grid = 40
min_mass = 0.011
path_to_TFs = "/nfs/team205/heart/Vitro/TF_selection/foetal_vCM_left_compact-vs-trabeculated_downstream/TF_selection_df_compact-common-up_2Feb2024.csv"
path_to_res_dict = "/nfs/team205/heart/anndata_objects/Foetal/vCM/post_scFate/vCM_onlyLeft_compact-common_silico_perturbation_2Feb2024.pkl"
path_to_plots = "/nfs/team205/heart/Vitro/TF_selection/foetal_vCM_left_compact-vs-trabeculated_downstream/CO_in-silico-perturbation_papermill/figures_pm/compact-common_2Feb2024"
```

Example 2 (python):
```python
import os
import sys

import matplotlib
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import scanpy as sc
import seaborn as sns
```

Example 3 (python):
```python
import celloracle as co
co.__version__
```

Example 4 (python):
```python
/nfs/team205/kk18/miniconda3/envs/celloracle_env/lib/python3.8/site-packages/loompy/bus_file.py:68: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  def twobit_to_dna(twobit: int, size: int) -> str:
/nfs/team205/kk18/miniconda3/envs/celloracle_env/lib/python3.8/site-packages/loompy/bus_file.py:85: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  def dna_to_twobit(dna: str) -> int:
/nfs/team205/kk18/miniconda3/envs/celloracle_env/lib/python3.8/site-packages/loompy/bus_file.py:102: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  def twobit_1hamming(twobit: int, size: int) -> List[int]:
```

Example 5 (python):
```python
WARNING:matplotlib.font_manager:Matplotlib is building the font cache; this may take a moment.
```

Example 6 (python):
```python
import session_info
session_info.show()
```

Example 7 (python):
```python
-----
celloracle          0.14.0
matplotlib          3.6.3
numpy               1.24.3
pandas              1.5.3
scanpy              1.9.5
seaborn             0.13.0
session_info        1.0.0
-----
```

Example 8 (python):
```python
Bio                         1.81
PIL                         10.0.1
anndata                     0.9.2
anyio                       NA
appdirs                     1.4.4
arrow                       1.3.0
asttokens                   NA
attr                        23.1.0
attrs                       23.1.0
babel                       2.13.0
backcall                    0.2.0
beta_ufunc                  NA
binom_ufunc                 NA
biothings_client            0.3.0
bottleneck                  1.3.5
brotli                      NA
certifi                     2023.07.22
cffi                        1.15.1
charset_normalizer          2.0.4
colorama                    0.4.6
comm                        0.1.4
cycler                      0.10.0
cython_runtime              NA
dateutil                    2.8.2
debugpy                     1.8.0
decorator                   5.1.1
defusedxml                  0.7.1
diskcache                   5.6.3
exceptiongroup              1.1.3
executing                   2.0.0
fastjsonschema              NA
filelock                    3.12.4
fqdn                        NA
genomepy                    0.16.1
gimmemotifs                 0.17.0
goatools                    1.3.9
h5py                        3.10.0
hypergeom_ufunc             NA
idna                        3.4
igraph                      0.10.8
importlib_metadata          NA
importlib_resources         NA
invgauss_ufunc              NA
ipykernel                   6.25.2
ipywidgets                  8.1.1
isoduration                 NA
iteround                    NA
jedi                        0.19.1
jinja2                      3.1.2
joblib                      1.2.0
json5                       NA
jsonpointer                 2.4
jsonschema                  4.19.1
jsonschema_specifications   NA
jupyter_events              0.8.0
jupyter_server              2.8.0
jupyterlab_server           2.25.0
kiwisolver                  1.4.4
llvmlite                    0.41.0
logomaker                   NA
loguru                      0.7.2
loompy                      3.0.7
louvain                     0.8.0
markupsafe                  2.1.3
mkl                         2.4.0
mpl_toolkits                NA
mygene                      3.2.2
mysql                       NA
natsort                     8.4.0
nbformat                    5.9.2
nbinom_ufunc                NA
ncf_ufunc                   NA
nct_ufunc                   NA
ncx2_ufunc                  NA
networkx                    3.1
norns                       NA
numba                       0.58.0
numexpr                     2.8.4
numpy_groupies              0.9.22
overrides                   NA
packaging                   23.1
parso                       0.8.3
patsy                       0.5.3
pexpect                     4.8.0
pickleshare                 0.7.5
pkg_resources               NA
platformdirs                3.10.0
prometheus_client           NA
prompt_toolkit              3.0.39
psutil                      5.9.6
ptyprocess                  0.7.0
pure_eval                   0.2.2
pyarrow                     13.0.0
pybedtools                  0.9.1
pycparser                   2.21
pydev_ipython               NA
pydevconsole                NA
pydevd                      2.9.5
pydevd_file_utils           NA
pydevd_plugins              NA
pydevd_tracing              NA
pyfaidx                     0.7.2.2
pygments                    2.16.1
pyparsing                   3.0.9
pysam                       0.20.0
pythonjsonlogger            NA
pytz                        2023.3.post1
referencing                 NA
requests                    2.31.0
rfc3339_validator           0.1.4
rfc3986_validator           0.1.1
rpds                        NA
scipy                       1.10.1
send2trash                  NA
setuptools                  68.2.2
six                         1.16.0
skewnorm_ufunc              NA
sklearn                     1.3.0
sniffio                     1.3.0
socks                       1.7.1
stack_data                  0.6.3
statsmodels                 0.14.0
texttable                   1.7.0
threadpoolctl               2.2.0
tornado                     6.3.3
tqdm                        4.66.1
traitlets                   5.11.2
typing_extensions           NA
uri_template                NA
urllib3                     1.26.16
velocyto                    0.17.17
wcwidth                     0.2.8
webcolors                   1.13
websocket                   1.6.4
xdg                         NA
xgboost                     2.0.0
xxhash                      NA
yaml                        6.0.1
zipp                        NA
zmq                         25.1.1
```

Example 9 (unknown):
```unknown
-----
IPython             8.12.3
jupyter_client      8.4.0
jupyter_core        5.4.0
jupyterlab          4.0.7
notebook            7.0.6
-----
Python 3.8.18 (default, Sep 11 2023, 13:40:15) [GCC 11.2.0]
Linux-5.4.0-169-generic-x86_64-with-glibc2.17
-----
Session information updated at 2024-02-03 00:23
```

Example 10 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 11 (python):
```python
sc.settings.set_figure_params(dpi=120)
```

Example 12 (python):
```python
if not os.path.exists(path_to_plots):
    # Create the directory
    os.makedirs(path_to_plots)
    print("Directory created successfully")
else:
    print("Directory already exists")
```

Example 13 (unknown):
```unknown
Directory created successfully
```

Example 14 (unknown):
```unknown
# Load
oracle = co.load_hdf5(path_to_oracle_object)
```

Example 15 (python):
```python
n_cell = oracle.adata.shape[0]
print(f"cell number is :{n_cell}")

k = int(0.025*n_cell)
print(f"Auto-selected k is :{k}")
```

Example 16 (unknown):
```unknown
cell number is :16451
Auto-selected k is :411
```

Example 17 (python):
```python
%matplotlib inline
# Check clustering data
sc.pl.draw_graph(oracle.adata, color='fine_grain_time-binned')
```

Example 18 (python):
```python
# Read data from txt file
df = pd.read_csv(path_to_TFs)
list_of_TFs = list(df['TF'])

# select TFs which are in the oracle object
list_of_TFs = [x for x in list_of_TFs if x in oracle.adata.var_names]

print(len(list_of_TFs))
list_of_TFs[:5]
```

Example 19 (unknown):
```unknown
['ARID3A', 'ATF1', 'BACH2', 'BHLHE40', 'CUX1']
```

Example 20 (python):
```python
# Check gene expressions
# will be saved to a separated file
sc.settings.set_figure_params(dpi=80)
sc.pl.draw_graph(oracle.adata, color=list_of_TFs, ncols=6,
                 layer="imputed_count", use_raw=False, cmap="RdPu",
                 show=False, save='_TF_expression.png'
                )
```

Example 21 (unknown):
```unknown
WARNING: saving figure to file figures/draw_graph_fa_TF_expression.png
```

Example 22 (unknown):
```unknown
[<AxesSubplot: title={'center': 'ARID3A'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'ATF1'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'BACH2'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'BHLHE40'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'CUX1'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'ESRRA'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'ESRRB'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'FOXN3'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'FOXO1'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'GATA4'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'GATA6'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'GTF2IRD1'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'HAND2'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'HEY2'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'HIF1A'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'HMBOX1'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'KLF12'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'KLF6'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'LCORL'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'MAX'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'MAZ'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'MBNL2'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'MEF2A'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'MEF2C'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'MEF2D'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'MEIS2'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'MXI1'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'NCOA1'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'NFATC2'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'NFE2L1'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'NFIA'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'NFIB'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'NFIC'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'NPAS2'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'NR3C1'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'NR4A1'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'PBX1'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'PLAG1'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'PPARA'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'PRDM16'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'RORA'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'SMAD3'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'SREBF2'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'TBX20'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'TFCP2'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'TGIF2'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'THRA'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'USF2'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'YY1'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'ZBTB16'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'ZBTB20'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'ZNF148'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'ZNF280D'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'ZNF654'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'ZSCAN18'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'AHCTF1'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'CREB1'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'FOXK1'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'PHF21A'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'SOX5'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'UBP1'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'ZNF141'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'ELF2'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'POU2F1'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'ZNF586'}, xlabel='FA1', ylabel='FA2'>,
 <AxesSubplot: title={'center': 'IRX3'}, xlabel='FA1', ylabel='FA2'>]
```

Example 23 (python):
```python
from celloracle.applications import Gradient_calculator
# Instantiate Gradient calculator object
gradient = Gradient_calculator(oracle_object=oracle, pseudotime_key="t")
gradient.calculate_p_mass(smooth=0.8, n_grid=n_grid, n_neighbors=200)
gradient.calculate_mass_filter(min_mass=min_mass, plot=True)

# convert the pseudotime data into grid points.
# For this calculation we can chose one of two methods.
gradient.transfer_data_into_grid(args={"method": "knn", "n_knn":50})

# Calculate graddient
gradient.calculate_gradient()
# Show results
scale_dev = 40
gradient.visualize_results(scale=scale_dev, s=5)
# Visualize results
fig, ax = plt.subplots(figsize=[6, 6])
gradient.plot_dev_flow_on_grid(scale=scale_dev, ax=ax)
```

Example 24 (python):
```python
from celloracle.applications import Oracle_development_module

# not showing plots
matplotlib.use('Agg')

# Get cell index list for the cells of interest
clusters = ['VentricularCardiomyocytesLeftCompact_t1',
 'VentricularCardiomyocytesLeftCompact_t2',
 'VentricularCardiomyocytesLeftCompact_t3']
cell_idx = np.where(oracle.adata.obs["fine_grain_time-binned"].isin(clusters))[0]

# Check index
print(cell_idx)

dev_dict = {}
for goi in list_of_TFs:
    print(goi)
    
    # Enter perturbation conditions to simulate signal propagation after the perturbation.
    oracle.simulate_shift(perturb_condition={goi: 0.0}, # this is KO simulation so expression=0
                          n_propagation=3)
    
    # Get transition probability
    oracle.estimate_transition_prob(n_neighbors=200,
                                    knn_random=True,
                                    sampled_fraction=1)

    # Calculate embedding
    oracle.calculate_embedding_shift(sigma_corr=0.05)
    
    # prepare grid
    oracle.calculate_p_mass(smooth=0.8, n_grid=n_grid, n_neighbors=200)
    oracle.calculate_mass_filter(min_mass=min_mass, plot=False)
    
    # plot vector field
    fig, ax = plt.subplots(1, 2,  figsize=[13, 6])
    scale_simulation = 0.1
    # Show quiver plot
    oracle.plot_simulation_flow_on_grid(scale=scale_simulation, ax=ax[0])
    ax[0].set_title(f"Simulated cell identity shift vector: {goi} KO")
    # Show quiver plot that was calculated with randomized graph.
    oracle.plot_simulation_flow_random_on_grid(scale=scale_simulation, ax=ax[1])
    ax[1].set_title(f"Randomized simulation vector")
    plt.savefig(f'{path_to_plots}/{goi}_simulation_vector.pdf')
    
    # Plot vector field with cell cluster
    fig, ax = plt.subplots(figsize=[8, 8])
    oracle.plot_cluster_whole(ax=ax, s=10)
    oracle.plot_simulation_flow_on_grid(scale=scale_simulation, ax=ax, show_background=False)
    plt.savefig(f'{path_to_plots}/{goi}_simulation_vector_with-clusters.pdf')
    
    
    dev = Oracle_development_module()
    # Load development flow
    dev.load_differentiation_reference_data(gradient_object=gradient)

    # Load simulation result
    dev.load_perturb_simulation_data(oracle_object=oracle,
                                     cell_idx_use=cell_idx, # Enter cell id list
                                     name="Lineage_Compact" # Name of this cell group. You can enter any name.
                                     )

    # Calculation
    dev.calculate_inner_product()
    dev.calculate_digitized_ip(n_bins=10)
    
    # save to the dictionary
    dev_dict[goi] = dev.copy()
    
    
    # Let's visualize the results
    vm = 0.01
    dev.visualize_development_module_layout_0(s=5,
                                              scale_for_simulation=scale_simulation,
                                              s_grid=50,
                                              scale_for_pseudotime=scale_dev,
                                              vm=vm)
    plt.savefig(f'{path_to_plots}/{goi}_results.pdf')
```

Example 25 (unknown):
```unknown
[    0     2     4 ... 16447 16448 16450]
ARID3A
```

Example 26 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 27 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 28 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 29 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 30 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 31 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 32 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 33 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 34 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 35 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 36 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 37 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 38 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 39 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 40 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 41 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 42 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 43 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 44 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 45 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 46 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 47 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 48 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 49 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 50 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 51 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 52 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 53 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 54 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 55 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 56 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 57 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 58 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 59 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 60 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 61 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 62 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 63 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 64 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 65 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 66 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 67 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 68 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 69 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 70 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 71 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 72 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 73 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 74 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 75 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 76 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 77 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 78 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 79 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 80 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 81 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 82 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 83 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 84 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 85 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 86 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 87 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 88 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 89 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 90 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 91 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 92 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 93 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 94 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 95 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 96 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 97 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 98 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 99 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 100 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 101 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 102 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 103 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 104 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 105 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 106 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 107 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 108 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 109 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 110 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 111 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 112 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 113 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 114 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 115 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 116 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 117 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 118 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 119 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 120 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 121 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 122 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 123 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 124 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 125 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 126 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 127 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 128 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 129 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 130 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 131 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 132 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 133 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 134 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 135 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 136 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 137 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 138 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 139 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 140 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 141 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 142 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 143 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 144 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 145 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 146 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 147 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 148 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 149 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 150 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 151 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 152 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 153 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 154 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 155 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 156 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 157 (python):
```python
INFO:matplotlib.category:Using categorical units to plot a list of strings that are all parsable as floats or dates. If these strings should be plotted as numbers, cast to the appropriate data type before plotting.
```

Example 158 (unknown):
```unknown
# save
utils.save_pkl(dev_dict, path_to_res_dict)
```

---

## Import required modules¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/Visium/Others/Plotting_separately_SAnode.html

**Contents:**
- Import required modules¶
- Prepare functions¶
- Read in anndata¶
- Plotting¶
- Cells¶
- Genes¶

**Examples:**

Example 1 (python):
```python
import warnings
warnings.simplefilter(action='ignore', category=FutureWarning)

import scanpy as sc
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42 
import seaborn as sns
import os 
from mpl_toolkits.axes_grid1 import make_axes_locatable
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
-----
matplotlib          3.7.2
mpl_toolkits        NA
numpy               1.24.4
pandas              1.5.3
scanpy              1.9.3
seaborn             0.12.2
session_info        1.0.0
-----
```

Example 4 (python):
```python
PIL                         10.0.0
anndata                     0.9.2
anyio                       NA
arrow                       1.2.3
asttokens                   NA
attr                        23.1.0
attrs                       23.1.0
babel                       2.12.1
backcall                    0.2.0
brotli                      1.0.9
certifi                     2023.11.17
cffi                        1.15.1
charset_normalizer          3.2.0
colorama                    0.4.6
comm                        0.1.4
cycler                      0.10.0
cython_runtime              NA
dateutil                    2.8.2
debugpy                     1.6.8
decorator                   5.1.1
defusedxml                  0.7.1
executing                   1.2.0
fastjsonschema              NA
fqdn                        NA
h5py                        3.9.0
idna                        3.4
igraph                      0.10.6
importlib_metadata          NA
importlib_resources         NA
ipykernel                   6.25.1
isoduration                 NA
jedi                        0.19.0
jinja2                      3.1.2
joblib                      1.3.2
json5                       NA
jsonpointer                 2.0
jsonschema                  4.19.0
jsonschema_specifications   NA
jupyter_events              0.7.0
jupyter_server              2.7.1
jupyterlab_server           2.24.0
kiwisolver                  1.4.5
leidenalg                   0.10.1
llvmlite                    0.40.1
markupsafe                  2.1.3
natsort                     8.4.0
nbformat                    5.9.2
numba                       0.57.1
overrides                   NA
packaging                   23.1
parso                       0.8.3
patsy                       0.5.3
pexpect                     4.8.0
pickleshare                 0.7.5
pkg_resources               NA
plotly                      5.16.1
prometheus_client           NA
prompt_toolkit              3.0.39
psutil                      5.9.5
ptyprocess                  0.7.0
pure_eval                   0.2.2
pydev_ipython               NA
pydevconsole                NA
pydevd                      2.9.5
pydevd_file_utils           NA
pydevd_plugins              NA
pydevd_tracing              NA
pygments                    2.16.1
pyparsing                   3.0.9
pythonjsonlogger            NA
pytz                        2023.3
referencing                 NA
requests                    2.31.0
rfc3339_validator           0.1.4
rfc3986_validator           0.1.1
rpds                        NA
ruamel                      NA
scipy                       1.10.1
send2trash                  NA
six                         1.16.0
sklearn                     1.3.0
sniffio                     1.3.0
socks                       1.7.1
sphinxcontrib               NA
stack_data                  0.6.2
statsmodels                 0.14.0
texttable                   1.6.7
threadpoolctl               3.2.0
tornado                     6.3.3
traitlets                   5.9.0
typing_extensions           NA
uri_template                NA
urllib3                     2.0.4
wcwidth                     0.2.6
webcolors                   1.13
websocket                   1.6.2
yaml                        6.0.1
zipp                        NA
zmq                         25.1.1
zstandard                   0.19.0
```

Example 5 (unknown):
```unknown
-----
IPython             8.12.0
jupyter_client      8.3.0
jupyter_core        4.12.0
jupyterlab          4.0.5
notebook            7.0.2
-----
Python 3.8.17 (default, Jul  5 2023, 21:04:15) [GCC 11.2.0]
Linux-5.4.0-169-generic-x86_64-with-glibc2.17
-----
Session information updated at 2024-02-10 23:20
```

Example 6 (python):
```python
# import a module given the full path?
import importlib.util
import sys

spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/_utils.py")
_utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = _utils
spec.loader.exec_module(_utils)
```

Example 7 (python):
```python
sc.settings.verbosity = 1          # verbosity: errors (0), warnings (1), info (2), hints (3)
sc.settings.set_figure_params(dpi=80,dpi_save=300)
```

Example 8 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk'
```

Example 9 (python):
```python
adata_vis = sc.read_h5ad('/nfs/team205/heart/anndata_objects/Foetal/visium-OCT_foetal_raw.h5ad')

# add abundance to .obs
df = adata_vis.obsm['q05_cell_abundance_w_sf'].copy()
## rename columns
df.columns = [x.replace('q05cell_abundance_w_sf_','') for x in df.columns]
adata_vis.obs[df.columns]=df.copy()

# add prop
## convert to proportions # should use np.nansam rather than sum
df = df.apply(lambda x: x/np.nansum(x), axis=1) 
adata_vis.obsm['prop']=df.copy()
adata_vis.obs['prop_'+df.columns] = df.copy()

# set symboßl as var_names
adata_vis.var.set_index('SYMBOL',inplace=True)

# log-normalise
sc.pp.filter_genes(adata_vis, min_cells=3)
sc.pp.normalize_total(adata_vis, target_sum=1e4)
sc.pp.log1p(adata_vis)

adata_vis
```

Example 10 (python):
```python
/tmp/ipykernel_131063/2172206593.py:13: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  adata_vis.obs['prop_'+df.columns] = df.copy()
/tmp/ipykernel_131063/2172206593.py:13: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  adata_vis.obs['prop_'+df.columns] = df.copy()
/tmp/ipykernel_131063/2172206593.py:13: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  adata_vis.obs['prop_'+df.columns] = df.copy()
/tmp/ipykernel_131063/2172206593.py:13: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  adata_vis.obs['prop_'+df.columns] = df.copy()
/tmp/ipykernel_131063/2172206593.py:13: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  adata_vis.obs['prop_'+df.columns] = df.copy()
/tmp/ipykernel_131063/2172206593.py:13: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  adata_vis.obs['prop_'+df.columns] = df.copy()
/tmp/ipykernel_131063/2172206593.py:13: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  adata_vis.obs['prop_'+df.columns] = df.copy()
/tmp/ipykernel_131063/2172206593.py:13: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  adata_vis.obs['prop_'+df.columns] = df.copy()
/tmp/ipykernel_131063/2172206593.py:13: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  adata_vis.obs['prop_'+df.columns] = df.copy()
/tmp/ipykernel_131063/2172206593.py:13: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  adata_vis.obs['prop_'+df.columns] = df.copy()
/tmp/ipykernel_131063/2172206593.py:13: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  adata_vis.obs['prop_'+df.columns] = df.copy()
/tmp/ipykernel_131063/2172206593.py:13: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  adata_vis.obs['prop_'+df.columns] = df.copy()
/tmp/ipykernel_131063/2172206593.py:13: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  adata_vis.obs['prop_'+df.columns] = df.copy()
/tmp/ipykernel_131063/2172206593.py:13: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  adata_vis.obs['prop_'+df.columns] = df.copy()
/tmp/ipykernel_131063/2172206593.py:13: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  adata_vis.obs['prop_'+df.columns] = df.copy()
/tmp/ipykernel_131063/2172206593.py:13: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  adata_vis.obs['prop_'+df.columns] = df.copy()
/tmp/ipykernel_131063/2172206593.py:13: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  adata_vis.obs['prop_'+df.columns] = df.copy()
/tmp/ipykernel_131063/2172206593.py:13: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
  adata_vis.obs['prop_'+df.columns] = df.copy()
```

Example 11 (unknown):
```unknown
AnnData object with n_obs × n_vars = 14261 × 22248
    obs: 'in_tissue', 'array_row', 'array_col', 'sample', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'mt_frac', 'sangerID', 'Publication', 'combinedID', 'donor', 'donor_type', 'region', 'region_finest', 'age', 'gender', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'flushed', 'status', 'cell2loc-map_group', 'JC_edit', 'sample_for_cell2loc', 'AtrialCardiomyocytesLeft', 'AtrialCardiomyocytesRight', 'VentricularCardiomyocytesLeftCompact', 'VentricularCardiomyocytesRightCompact', 'VentricularCardiomyocytesLeftTrabeculated', 'VentricularCardiomyocytesRightTrabeculated', 'SinoatrialNodeCardiomyocytes', 'AtrioventricularNodeCardiomyocytes', 'VentricularConductionSystemProximal', 'VentricularConductionSystemDistal', 'GreatVesselAdventitialFibroblasts', 'CoronaryVesselAdventitialFibroblasts', 'MyocardialInterstitialFibroblasts1', 'MyocardialInterstitialFibroblasts2', 'Myofibroblasts', 'LymphNodeFibroblasticReticularCells', 'ValveInterstitialCells', 'GreatVesselSmoothMuscleCells', 'CoronarySmoothMuscleCells', 'CoronaryPericytes', 'GreatVesselArterialEndothelialCells', 'GreatVesselVenousEndothelialCells', 'CoronaryArterialEndothelialCells', 'CoronaryVenousEndothelialCells', 'CoronaryCapillaryEndothelialCells', 'EndocardialCells', 'EndocardialCushionCells', 'ValveEndothelialCells', 'LymphaticEndothelialCells', 'MesothelialEpicardialCells', 'EpicardiumDerivedCells', 'NeuronPrecursors', 'ChromaffinCells', 'SympatheticNeurons', 'ParaympatheticNeurons', 'SchwannCellPrecursors', 'SchwannCells', 'MonocytesMPOpos', 'Monocytes', 'MonocyteDerivedCells', 'MacrophagesCX3CR1pos', 'MacrophagesTIMD4pos', 'MacrophagesLYVE1pos', 'MacrophagesATF3pos', 'DendriticCellsType1', 'DendriticCellsMature', 'PlasmacytoidDendriticCells', 'MastCells', 'Megakaryocytes', 'TCellsCD4pos', 'TCellsCD8pos', 'TregsCD4pos', 'ProBCells', 'BCells', 'BCellsMS4A1pos', 'NaturalKillerCells', 'InnateLymphoidCells', 'prop_AtrialCardiomyocytesLeft', 'prop_AtrialCardiomyocytesRight', 'prop_VentricularCardiomyocytesLeftCompact', 'prop_VentricularCardiomyocytesRightCompact', 'prop_VentricularCardiomyocytesLeftTrabeculated', 'prop_VentricularCardiomyocytesRightTrabeculated', 'prop_SinoatrialNodeCardiomyocytes', 'prop_AtrioventricularNodeCardiomyocytes', 'prop_VentricularConductionSystemProximal', 'prop_VentricularConductionSystemDistal', 'prop_GreatVesselAdventitialFibroblasts', 'prop_CoronaryVesselAdventitialFibroblasts', 'prop_MyocardialInterstitialFibroblasts1', 'prop_MyocardialInterstitialFibroblasts2', 'prop_Myofibroblasts', 'prop_LymphNodeFibroblasticReticularCells', 'prop_ValveInterstitialCells', 'prop_GreatVesselSmoothMuscleCells', 'prop_CoronarySmoothMuscleCells', 'prop_CoronaryPericytes', 'prop_GreatVesselArterialEndothelialCells', 'prop_GreatVesselVenousEndothelialCells', 'prop_CoronaryArterialEndothelialCells', 'prop_CoronaryVenousEndothelialCells', 'prop_CoronaryCapillaryEndothelialCells', 'prop_EndocardialCells', 'prop_EndocardialCushionCells', 'prop_ValveEndothelialCells', 'prop_LymphaticEndothelialCells', 'prop_MesothelialEpicardialCells', 'prop_EpicardiumDerivedCells', 'prop_NeuronPrecursors', 'prop_ChromaffinCells', 'prop_SympatheticNeurons', 'prop_ParaympatheticNeurons', 'prop_SchwannCellPrecursors', 'prop_SchwannCells', 'prop_MonocytesMPOpos', 'prop_Monocytes', 'prop_MonocyteDerivedCells', 'prop_MacrophagesCX3CR1pos', 'prop_MacrophagesTIMD4pos', 'prop_MacrophagesLYVE1pos', 'prop_MacrophagesATF3pos', 'prop_DendriticCellsType1', 'prop_DendriticCellsMature', 'prop_PlasmacytoidDendriticCells', 'prop_MastCells', 'prop_Megakaryocytes', 'prop_TCellsCD4pos', 'prop_TCellsCD8pos', 'prop_TregsCD4pos', 'prop_ProBCells', 'prop_BCells', 'prop_BCellsMS4A1pos', 'prop_NaturalKillerCells', 'prop_InnateLymphoidCells'
    var: 'gene_ids', 'feature_types', 'genome', 'mt', 'n_cells'
    uns: 'spatial', 'log1p'
    obsm: 'MT', 'means_cell_abundance_w_sf', 'q05_cell_abundance_w_sf', 'q95_cell_abundance_w_sf', 'spatial', 'stds_cell_abundance_w_sf', 'prop'
```

Example 12 (python):
```python
# add "annotation_final"
annot = pd.read_csv('/nfs/team205/heart/anndata_objects/Foetal/visium_foetal_healthy-all_spot-annotation.csv',
                    index_col=0
                   )
adata_vis.obs['annotation_final'] = annot[['final']].reindex(adata_vis.obs_names)
adata_vis.obs.head()
```

Example 13 (python):
```python
slide_id = 'HCAHeartST10317184'
slide = _utils.select_multislide(adata_vis, [slide_id], s_col='sample')

with mpl.rc_context({'figure.figsize': [3, 3],
                            'axes.titlesize':16,}):
    # cells
    print(slide_id)
    sc.pl.spatial(slide,cmap='YlGnBu',
                  color=['SinoatrialNodeCardiomyocytes','ParaympatheticNeurons','MacrophagesCX3CR1pos','SympatheticNeurons'], 
                  ncols=5, wspace=0.2,hspace=0.3,
                  size=1.2,img_key='hires',alpha_img=0.3,vmin=0, vmax='p99.4',
                  title=slide_id
                  # palette = palette
                  # save=f'_SAN-{ID}_core_cellstates.pdf'
                 )
```

Example 14 (unknown):
```unknown
/nfs/team205/kk18/function/python/_utils.py:95: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  slides.uns['spatial'] = uns_dict
```

Example 15 (unknown):
```unknown
HCAHeartST10317184
WARNING: The title list is shorter than the number of panels. Using 'color' value instead for some plots.
WARNING: The title list is shorter than the number of panels. Using 'color' value instead for some plots.
WARNING: The title list is shorter than the number of panels. Using 'color' value instead for some plots.
```

Example 16 (python):
```python
slide_id = 'HCAHeartST10317184'
slide = _utils.select_multislide(adata_vis, [slide_id], s_col='sample')

with mpl.rc_context({'figure.figsize': [3, 3],
                            'axes.titlesize':12,}):
    # cells
    print(slide_id)
    sc.pl.spatial(slide,cmap='YlGnBu',
                  color=['SinoatrialNodeCardiomyocytes','ParaympatheticNeurons','MacrophagesCX3CR1pos'], 
                  ncols=5, wspace=0.1,hspace=0.3,
                  size=1.2,img_key='hires',alpha_img=0.3,vmin=0, vmax='p99.4',
                  crop_coord = [5900,7500,10700,12800],
                  # title=slide_id
                  # palette = palette
                  # save=f'_SAN-{ID}_core_cellstates.pdf'
                 )
```

Example 17 (unknown):
```unknown
/nfs/team205/kk18/function/python/_utils.py:95: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  slides.uns['spatial'] = uns_dict
```

Example 18 (unknown):
```unknown
HCAHeartST10317184
```

Example 19 (python):
```python
slide_id = 'HCAHeartST10317184'
slide = _utils.select_multislide(adata_vis, [slide_id], s_col='sample')

with mpl.rc_context({'figure.figsize': [3, 3],
                            'axes.titlesize':16,}):
    # cells
    print(slide_id)
    sc.pl.spatial(slide,cmap='RdPu',
                  color=['SEMA3A', 'SLIT2', 'SLIT3', 'EFNB2', 'CXCL12'], 
                  ncols=5, wspace=0.2,hspace=0.3,
                  size=1.2,img_key='hires',alpha_img=0.3,vmin=0, vmax='p99.4',
                  # title=slide_id
                  # palette = palette
                  # save=f'_SAN-{ID}_core_cellstates.pdf'
                 )
```

Example 20 (unknown):
```unknown
/nfs/team205/kk18/function/python/_utils.py:95: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  slides.uns['spatial'] = uns_dict
```

Example 21 (unknown):
```unknown
HCAHeartST10317184
```

Example 22 (python):
```python
slide_id = 'HCAHeartST10317184'
slide = _utils.select_multislide(adata_vis, [slide_id], s_col='sample')

with mpl.rc_context({'figure.figsize': [3, 3],
                            'axes.titlesize':24,}):
    # cells
    print(slide_id)
    sc.pl.spatial(slide,cmap='RdPu',
                  color=['SEMA3A', 'SLIT2', 'SLIT3', 'EFNB2', 'CXCL12'], 
                  ncols=5, wspace=0.1,hspace=0.3,
                  size=1.2,img_key='hires',alpha_img=0.3,vmin=0, vmax='p99.4',
                  crop_coord = [5900,7500,10700,12800],
                  # title=slide_id
                  # palette = palette
                  save=f'_visium_section_axon-guidance-genes.pdf'
                 )
    # plt.savefig(f"{figdir}/visium_section_axon-guidance-genes.pdf", bbox_inches="tight",dpi=300)
```

Example 23 (unknown):
```unknown
/nfs/team205/kk18/function/python/_utils.py:95: ImplicitModificationWarning: Trying to modify attribute `._uns` of view, initializing view as actual.
  slides.uns['spatial'] = uns_dict
```

Example 24 (unknown):
```unknown
HCAHeartST10317184
WARNING: saving figure to file figures/show_visium_section_axon-guidance-genes.pdf
```

---

## Variables¶

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/VisiumHD/no4_QC-filtering.html

**Contents:**
- Variables¶
- Read in b2c data¶
- QC and preprocessing¶
- Save¶
- Read in for plotting¶

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os

import celltypist
```

Example 2 (python):
```python
adata_dir = '/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects'
sample_id = 'HEA_FOET14880396'
```

Example 3 (python):
```python
adata = sc.read_h5ad(f'{adata_dir}/{sample_id}_b2c_cells.h5ad')
adata
```

Example 4 (unknown):
```unknown
AnnData object with n_obs × n_vars = 134590 × 18051
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells'
    uns: 'spatial'
    obsm: 'spatial', 'spatial_cropped_150_buffer'
```

Example 5 (python):
```python
adata.obs.head()
```

Example 6 (python):
```python
adata.var["mt"] = adata.var_names.str.startswith("MT-")
sc.pp.calculate_qc_metrics(adata, qc_vars=["mt"], inplace=True)
```

Example 7 (python):
```python
fig, axs = plt.subplots(1, 4, figsize=(15, 3))
sns.histplot(adata.obs["total_counts"], kde=False, ax=axs[0])
sns.histplot(
    adata.obs["total_counts"][adata.obs["total_counts"] < 1000],
    kde=False,
    bins=40,
    ax=axs[1],
)
sns.histplot(adata.obs["n_genes_by_counts"], kde=False, bins=60, ax=axs[2])
sns.histplot(
    adata.obs["n_genes_by_counts"][adata.obs["n_genes_by_counts"] < 1000],
    kde=False,
    bins=60,
    ax=axs[3],
)
fig.suptitle('bin2cell')
fig.tight_layout()
```

Example 8 (python):
```python
fig, axs = plt.subplots(1, 2, figsize=(10, 3))
sns.histplot(
    adata.obs["total_counts"][adata.obs["total_counts"] < 8000],
    kde=False,
    bins=60,
    ax=axs[0],
)
sns.histplot(
    adata.obs["n_genes_by_counts"][adata.obs["n_genes_by_counts"] < 3500],
    kde=False,
    bins=60,
    ax=axs[1],
)
fig.suptitle('bin2cell')
fig.tight_layout()
```

Example 9 (unknown):
```unknown
AnnData object with n_obs × n_vars = 134590 × 18051
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts'
    uns: 'spatial'
    obsm: 'spatial', 'spatial_cropped_150_buffer'
```

Example 10 (python):
```python
plt.rcParams['figure.figsize'] = 4.5,1
# mito
sns.histplot(adata.obs['pct_counts_mt'][adata.obs['pct_counts_mt'] < 20], 
             kde=False)
plt.show()

sns.histplot(adata.obs['bin_count'], 
             kde=False)
plt.show()
sns.histplot(adata.obs['bin_count'][adata.obs['bin_count'] < 30], 
             kde=False)
plt.show()
```

Example 11 (python):
```python
# filtering out poor cuality cells/bins: n_genes
sc.pp.filter_cells(adata, min_genes=30)
print(f'shape of b2c: {adata.shape}')
```

Example 12 (unknown):
```unknown
shape of b2c: (131456, 18051)
```

Example 13 (python):
```python
# filtering out poor cuality cells/bins: bin_count (cell size)
adata = adata[adata.obs['bin_count']>3] # remove too small cells
adata = adata[adata.obs['bin_count']<60] # remove too large cells
print(f'shape of b2c: {adata.shape}')
```

Example 14 (unknown):
```unknown
shape of b2c: (129803, 18051)
```

Example 15 (python):
```python
sns.histplot(adata.obs['bin_count'], 
             kde=False)
plt.show()
```

Example 16 (python):
```python
fig, axs = plt.subplots(1, 4, figsize=(15, 3))
sns.histplot(adata.obs["total_counts"], kde=False, ax=axs[0])
sns.histplot(
    adata.obs["total_counts"][adata.obs["total_counts"] < 1000],
    kde=False,
    bins=40,
    ax=axs[1],
)
sns.histplot(adata.obs["n_genes_by_counts"], kde=False, bins=60, ax=axs[2])
sns.histplot(
    adata.obs["n_genes_by_counts"][adata.obs["n_genes_by_counts"] < 1000],
    kde=False,
    bins=60,
    ax=axs[3],
)
fig.suptitle('bin2cell')
fig.tight_layout()
```

Example 17 (python):
```python
# filtering genes
sc.pp.filter_genes(adata, min_cells=3)
print(f'shape of b2c: {adata.shape}')
```

Example 18 (unknown):
```unknown
shape of b2c: (129803, 18047)
```

Example 19 (python):
```python
# Plotting the histograms
plt.figure(figsize=(10, 3))
sns.histplot(
    adata.obs["n_genes_by_counts"][adata.obs["n_genes_by_counts"] < 4000],
    kde=False,
    bins=60,
)
plt.legend()
plt.show()
```

Example 20 (python):
```python
adata.write(f'{adata_dir}/{sample_id}_b2c_cells_filtered.h5ad')
```

Example 21 (python):
```python
f'{adata_dir}/{sample_id}_b2c_cells_filtered.h5ad'
```

Example 22 (unknown):
```unknown
'/lustre/scratch126/cellgen/team205/kk18/VisiumHD/objects/HEA_FOET14880396_b2c_cells_filtered.h5ad'
```

Example 23 (python):
```python
adata = sc.read_h5ad(f'{adata_dir}/{sample_id}_b2c_cells_filtered.h5ad')
adata
```

Example 24 (unknown):
```unknown
AnnData object with n_obs × n_vars = 129803 × 18047
    obs: 'object_id', 'bin_count', 'array_row', 'array_col', 'labels_joint_source', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts'
    uns: 'spatial'
    obsm: 'spatial', 'spatial_cropped_150_buffer'
```

Example 25 (python):
```python
fig, axs = plt.subplots(1, 2, figsize=(8, 3))
sns.histplot(adata.obs["total_counts"], kde=False, ax=axs[0])
sns.histplot(adata.obs["n_genes_by_counts"], kde=False, bins=60, ax=axs[1])
fig.suptitle('bin2cell')
fig.tight_layout()
```

Example 26 (python):
```python
# median
print(f'total_counts: {np.median(adata.obs["total_counts"])}')
print(f'n_genes: {np.median(adata.obs["n_genes_by_counts"])}')
```

Example 27 (unknown):
```unknown
total_counts: 2601.3060987148215
n_genes: 1138.0
```

---
