# Developing-Heart-Local-Improved - Visiumhd Analysis

**Pages:** 4

---

## Notebook

**URL:** http://127.0.0.1:9020/notebooks_html/Supplementary_materials/Macrophages/CellTypist_model_training_YS-MP.html

**Contents:**
- Import required modules¶
- Prepare functions¶
- Read in anndata¶
- Subset macrophages¶
- Merge non-pre, non-microglia populations¶
- Get informative genes¶
- HVGs¶
- Training CellTypist model¶
- Select cells to train¶
- Train¶

**Examples:**

Example 1 (python):
```python
import warnings
warnings.simplefilter(action='ignore', category=FutureWarning)

import scanpy as sc
import anndata
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import matplotlib as mpl
import seaborn as sns
import os 
from mpl_toolkits.axes_grid1 import make_axes_locatable

import celltypist
from celltypist import models

import scipy
import scanpy.external as sce
from fastprogress.fastprogress import progress_bar
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
-----
anndata             0.9.2
celltypist          1.6.0
matplotlib          3.5.2
mpl_toolkits        NA
numpy               1.22.3
pandas              1.5.3
scanpy              1.8.2
scipy               1.9.1
seaborn             0.12.2
session_info        1.0.0
-----
```

Example 4 (python):
```python
PIL                 10.0.0
asttokens           NA
backcall            0.2.0
beta_ufunc          NA
binom_ufunc         NA
certifi             2023.07.22
cffi                1.15.1
charset_normalizer  3.2.0
colorama            0.4.6
comm                0.1.4
cpuinfo             NA
cycler              0.10.0
cython_runtime      NA
dateutil            2.8.2
debugpy             1.6.7.post1
decorator           5.1.1
executing           1.2.0
h5py                3.9.0
hypergeom_ufunc     NA
idna                3.4
igraph              0.10.6
importlib_metadata  NA
ipykernel           6.25.1
jedi                0.19.0
joblib              1.3.2
kiwisolver          1.4.5
leidenalg           0.10.1
llvmlite            0.40.1
natsort             8.4.0
nbinom_ufunc        NA
ncf_ufunc           NA
numba               0.57.1
numexpr             2.8.5
packaging           23.1
parso               0.8.3
patsy               0.5.3
pexpect             4.8.0
pickleshare         0.7.5
pkg_resources       NA
platformdirs        3.10.0
plotly              5.16.1
prompt_toolkit      3.0.39
psutil              5.9.5
ptyprocess          0.7.0
pure_eval           0.2.2
pydev_ipython       NA
pydevconsole        NA
pydevd              2.9.5
pydevd_file_utils   NA
pydevd_plugins      NA
pydevd_tracing      NA
pygments            2.16.1
pyparsing           3.0.9
pytz                2023.3
requests            2.31.0
sinfo               0.3.4
six                 1.16.0
sklearn             1.3.0
sphinxcontrib       NA
stack_data          0.6.2
statsmodels         0.14.0
tables              3.8.0
texttable           1.6.7
threadpoolctl       3.2.0
tornado             6.3.3
traitlets           5.9.0
typing_extensions   NA
urllib3             2.0.4
wcwidth             0.2.6
zipp                NA
zmq                 25.1.1
```

Example 5 (unknown):
```unknown
-----
IPython             8.12.2
jupyter_client      8.3.0
jupyter_core        5.3.1
-----
Python 3.8.17 (default, Jul  5 2023, 21:04:15) [GCC 11.2.0]
Linux-4.15.0-213-generic-x86_64-with-glibc2.17
-----
Session information updated at 2023-11-18 10:54
```

Example 6 (python):
```python
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)
```

Example 7 (python):
```python
sc.settings.verbosity = 1          # verbosity: errors (0), warnings (1), info (2), hints (3)
sc.settings.set_figure_params(dpi=80)
```

Example 8 (python):
```python
adata = sc.read_h5ad('/lustre/scratch126/cellgen/team205/kk18/datasets/YS/ys_portal_object_20221208.h5ad')
adata
```

Example 9 (unknown):
```unknown
AnnData object with n_obs × n_vars = 169494 × 37318
    obs: 'component', 'stage', 'sex', 'sort.ids', 'fetal.ids', 'orig.dataset', 'sequencing.type', 'lanes', 'LVL1', 'LVL2', 'LVL3'
    obsm: 'X_umap'
```

Example 10 (python):
```python
sc.pl.umap(adata, color='LVL3')
```

Example 11 (python):
```python
adata.X.data[:5]
```

Example 12 (unknown):
```unknown
array([0.94222367, 0.94222367, 0.94222367, 0.94222367, 0.94222367],
      dtype=float32)
```

Example 13 (python):
```python
set(adata.obs['LVL3'])
```

Example 14 (unknown):
```unknown
{'CMP',
 'DC2_CYCLING',
 'DEF_HSPC_1',
 'DEF_HSPC_2',
 'EARLY_ERYTHROID',
 'ELP',
 'ENDODERM',
 'ENDOTHELIUM_AEC',
 'ENDOTHELIUM_PROLIF_AEC',
 'ENDOTHELIUM_PROLIF_SINUSOIDAL_EC',
 'ENDOTHELIUM_SINUSOIDAL',
 'ENDOTHELIUM_VWF',
 'EOSINOPHIL_BASOPHIL',
 'EO_BASO_MAST_PRECURSOR',
 'ERYTHROID',
 'FIBROBLAST',
 'HE',
 'HSPC_1',
 'HSPC_2',
 'ILC_PRECURSOR',
 'IMMATURE_ENDOTHELIUM',
 'LMPP',
 'LYMPHOID B LIN',
 'MAC DC2',
 'MACROPHAGE_IRON_RECYCLING',
 'MACROPHAGE_KUPFFER_LIKE',
 'MACROPHAGE_LYVE1_HIGH',
 'MACROPHAGE_MICROGLIA',
 'MACROPHAGE_PRE_MAC',
 'MACROPHAGE_PROLIFERATING',
 'MAST_CELL',
 'MEMP',
 'MESOTHELIUM',
 'MID_ERYTHROID',
 'MK',
 'MONO MAC DC2',
 'MONO MAC PRE DC2',
 'MONOCYTE_0',
 'MONOCYTE_1',
 'MONOCYTE_MACROPHAGE',
 'MOP',
 'NEUTROPHIL_PRECURSOR',
 'NK',
 'PDC PRECURSOR',
 'PRE DC2',
 'PRIM_HSPC_1',
 'PRIM_HSPC_2',
 'PROMONOCYTE',
 'SMOOTH_MUSCLE'}
```

Example 15 (python):
```python
set(adata.obs['fetal.ids'])
```

Example 16 (unknown):
```unknown
{'F138',
 'F32',
 'F35',
 'F37',
 'F61',
 'F79',
 'F80',
 'Wang_CS10',
 'Wang_CS11',
 'mikola_1'}
```

Example 17 (python):
```python
cells_to_select = ['MACROPHAGE_IRON_RECYCLING',
 'MACROPHAGE_KUPFFER_LIKE',
 'MACROPHAGE_LYVE1_HIGH',
 'MACROPHAGE_MICROGLIA',
 'MACROPHAGE_PRE_MAC']
# excluding proliferating cells

adata = adata[adata.obs['LVL3'].isin(cells_to_select)]
adata.obs['LVL3'].value_counts()
```

Example 18 (unknown):
```unknown
MACROPHAGE_IRON_RECYCLING    22946
MACROPHAGE_LYVE1_HIGH         8385
MACROPHAGE_KUPFFER_LIKE       7724
MACROPHAGE_MICROGLIA           613
MACROPHAGE_PRE_MAC              29
Name: LVL3, dtype: int64
```

Example 19 (python):
```python
# filter genes
sc.pp.filter_genes(adata, min_cells=3)
```

Example 20 (python):
```python
sc.pl.umap(adata, color='LVL3')
```

Example 21 (python):
```python
for celltype in cells_to_select:
    sc.pl.umap(adata, color='LVL3', groups=celltype, size=50)
```

Example 22 (python):
```python
sc.pl.umap(adata, color='LVL3', groups='MACROPHAGE_PRE_MAC', size=50)
```

Example 23 (python):
```python
sc.pl.dotplot(adata,var_names=['LYVE1','CX3CR1','TREM2'],groupby='LVL3')
```

Example 24 (python):
```python
adata.obs['LVL3_mod'] = adata.obs['LVL3'].copy()
adata.obs.replace({'LVL3_mod':{
    'MACROPHAGE_IRON_RECYCLING':'MACROPHAGE',
 'MACROPHAGE_KUPFFER_LIKE':'MACROPHAGE',
 'MACROPHAGE_LYVE1_HIGH':'MACROPHAGE',
 # 'MACROPHAGE_MICROGLIA':'MACROPHAGE_MICROGLIA',
 # 'MACROPHAGE_PRE_MAC':'MACROPHAGE_PRE_MAC'
}}, inplace=True)
adata.obs['LVL3_mod'].value_counts()
```

Example 25 (unknown):
```unknown
MACROPHAGE              39055
MACROPHAGE_MICROGLIA      613
MACROPHAGE_PRE_MAC         29
Name: LVL3_mod, dtype: int64
```

Example 26 (unknown):
```unknown
AnnData object with n_obs × n_vars = 39697 × 28532
    obs: 'component', 'stage', 'sex', 'sort.ids', 'fetal.ids', 'orig.dataset', 'sequencing.type', 'lanes', 'LVL1', 'LVL2', 'LVL3', 'LVL3_mod'
    var: 'n_cells'
    uns: 'LVL3_colors'
    obsm: 'X_umap'
```

Example 27 (unknown):
```unknown
groupby = 'LVL3_mod'
max_n = 30
n_iteration = 100
n_detected = 30
batch_key = 'fetal.ids'
```

Example 28 (python):
```python
# iteration
res = {}
for i in progress_bar(range(n_iteration)):
    # subsample
    adata_sub = utils.sctk_subsample(adata, fraction=1, groupby=groupby, max_n=max_n, random_state=i)
    # remove donors those cell number is 1
    vals = adata_sub.obs[batch_key].value_counts()
    samples_to_remove = list(vals.index[vals==1])
    adata_sub = adata_sub[adata_sub.obs[batch_key].isin(samples_to_remove)==False]
    # hvg detection
    sc.pp.highly_variable_genes(adata_sub, min_mean=0.0125, max_mean=3, min_disp=0.5, batch_key=batch_key)
    # print(f'n_hvg: {sum(adata_sub.var.highly_variable)}')
    # store data
    res[i] = adata_sub.var.highly_variable
    del adata_sub

# plot
l = []
for i in range(n_iteration):
    l.append(len(res_df.index[res_df.sum(axis=1)>i]))
plt.scatter(x=range(n_iteration), y=l)
```

Example 29 (unknown):
```unknown
final seleted hvgs: 3089
```

Example 30 (python):
```python
# selection
res_df = pd.DataFrame(res)
hvgs = list(res_df.index[res_df.sum(axis=1)>=n_detected])
print(f'final seleted hvgs: {len(hvgs)}')
```

Example 31 (unknown):
```unknown
final seleted hvgs: 868
```

Example 32 (python):
```python
from geosketch import gs
```

Example 33 (unknown):
```unknown
Signature: gs(X, N, **kwargs)
Docstring:
Geometric sketching.
Wrapper around `gs_gap()`, see parameter documentation in that
function.
File:      ~/my-conda-envs/vitro/lib/python3.8/site-packages/geosketch/sketch.py
Type:      function
```

Example 34 (python):
```python
groups_to_subset = ['MACROPHAGE','MACROPHAGE_MICROGLIA']
max_n = 30
batch_key = 'fetal.ids'

adata.obs['select'] = True
adata.obs['geosketch_index'] = False
for grp in groups_to_subset:
    print(f'##### {grp} #####')
    # subset
    adata_sub = adata[adata.obs['LVL3_mod']==grp]
    # remove samples those cell number is 1
    vals = adata_sub.obs[batch_key].value_counts()
    samples_to_remove = list(vals.index[vals==1])
    obsnames_to_remove = adata_sub.obs_names[adata_sub.obs[batch_key].isin(samples_to_remove)]
    adata_sub = adata_sub[adata_sub.obs[batch_key].isin(samples_to_remove)==False]
    # hvg detection
    sc.pp.highly_variable_genes(adata_sub, min_mean=0.0125, max_mean=3, min_disp=0.5, 
                                batch_key=batch_key, subset=True)
    print(f'n_hvg: {adata_sub.shape[1]}')
    print(f'n_cell: {adata_sub.shape[0]}')
    # scale, pca
    print('pca, batch correction')
    sc.pp.scale(adata_sub, max_value=10)
    sc.tl.pca(adata_sub, svd_solver='arpack')
    # harmony for batch correction
    sce.pp.harmony_integrate(adata_sub, key=batch_key, max_iter_harmony=30)
    # geosketch
    print('geosketching')
    sketch_index = gs(adata_sub.obsm['X_pca_harmony'], max_n, replace=False)
    # store to adata
    obsnames = adata_sub.obs_names[sketch_index]
    obsnames_others = adata_sub.obs_names[[x for x in range(len(adata_sub.obs_names)) if x not in sketch_index]]
    adata.obs.loc[obsnames_to_remove,'select'] = False # removed cells before hvg detection
    adata.obs.loc[obsnames_others,'select'] = False
    adata.obs.loc[obsnames,'geosketch_index'] = True
    print('')
```

Example 35 (unknown):
```unknown
##### MACROPHAGE #####
n_hvg: 1280
n_cell: 39055
pca, batch correction
```

Example 36 (unknown):
```unknown
2023-11-18 11:56:57,958 - harmonypy - INFO - Computing initial centroids with sklearn.KMeans...
Computing initial centroids with sklearn.KMeans...
2023-11-18 11:57:31,447 - harmonypy - INFO - sklearn.KMeans initialization complete.
sklearn.KMeans initialization complete.
2023-11-18 11:57:31,865 - harmonypy - INFO - Iteration 1 of 30
Iteration 1 of 30
2023-11-18 11:58:02,145 - harmonypy - INFO - Iteration 2 of 30
Iteration 2 of 30
2023-11-18 11:58:32,740 - harmonypy - INFO - Converged after 2 iterations
Converged after 2 iterations
```

Example 37 (unknown):
```unknown
geosketching

##### MACROPHAGE_MICROGLIA #####
n_hvg: 904
n_cell: 612
pca, batch correction
```

Example 38 (unknown):
```unknown
2023-11-18 11:58:43,743 - harmonypy - INFO - Computing initial centroids with sklearn.KMeans...
Computing initial centroids with sklearn.KMeans...
2023-11-18 11:58:44,653 - harmonypy - INFO - sklearn.KMeans initialization complete.
sklearn.KMeans initialization complete.
2023-11-18 11:58:44,721 - harmonypy - INFO - Iteration 1 of 30
Iteration 1 of 30
2023-11-18 11:58:45,055 - harmonypy - INFO - Iteration 2 of 30
Iteration 2 of 30
2023-11-18 11:58:45,368 - harmonypy - INFO - Iteration 3 of 30
Iteration 3 of 30
2023-11-18 11:58:45,566 - harmonypy - INFO - Iteration 4 of 30
Iteration 4 of 30
2023-11-18 11:58:45,761 - harmonypy - INFO - Iteration 5 of 30
Iteration 5 of 30
2023-11-18 11:58:45,954 - harmonypy - INFO - Iteration 6 of 30
Iteration 6 of 30
2023-11-18 11:58:46,057 - harmonypy - INFO - Iteration 7 of 30
Iteration 7 of 30
2023-11-18 11:58:46,326 - harmonypy - INFO - Iteration 8 of 30
Iteration 8 of 30
2023-11-18 11:58:46,356 - harmonypy - INFO - Iteration 9 of 30
Iteration 9 of 30
2023-11-18 11:58:46,526 - harmonypy - INFO - Iteration 10 of 30
Iteration 10 of 30
2023-11-18 11:58:46,645 - harmonypy - INFO - Iteration 11 of 30
Iteration 11 of 30
2023-11-18 11:58:46,759 - harmonypy - INFO - Iteration 12 of 30
Iteration 12 of 30
2023-11-18 11:58:46,929 - harmonypy - INFO - Iteration 13 of 30
Iteration 13 of 30
2023-11-18 11:58:47,050 - harmonypy - INFO - Iteration 14 of 30
Iteration 14 of 30
2023-11-18 11:58:47,159 - harmonypy - INFO - Iteration 15 of 30
Iteration 15 of 30
2023-11-18 11:58:47,330 - harmonypy - INFO - Iteration 16 of 30
Iteration 16 of 30
2023-11-18 11:58:47,437 - harmonypy - INFO - Iteration 17 of 30
Iteration 17 of 30
2023-11-18 11:58:47,544 - harmonypy - INFO - Converged after 17 iterations
Converged after 17 iterations
```

Example 39 (unknown):
```unknown
geosketching
```

Example 40 (python):
```python
pd.crosstab(adata.obs['select'],adata.obs['geosketch_index'])
```

Example 41 (python):
```python
# select
adata_sel = adata[adata.obs['select']]
adata_sel.obs['LVL3_mod'].value_counts()
```

Example 42 (unknown):
```unknown
MACROPHAGE              30
MACROPHAGE_MICROGLIA    30
MACROPHAGE_PRE_MAC      29
Name: LVL3_mod, dtype: int64
```

Example 43 (python):
```python
'''
# subset
max_n = 30
adata_sub = utils.sctk_subsample(adata, fraction=1, groupby=groupby, max_n=max_n, random_state=0)
adata_sub.obs[groupby].value_counts()
'''
```

Example 44 (python):
```python
%%time
model = celltypist.train(adata_sel[:,hvgs],  # selecting defined hvgs
                         labels = groupby,
                         n_jobs = 10,
                         feature_selection = False, # since manually selected
                         check_expression = False
                        )

# Save the model
model.write('/nfs/team205/heart/anndata_objects/Foetal/celltypist/model_YS_portal_object_20221208_onlyMP_hvg_18Nov2023.pkl')
```

Example 45 (unknown):
```unknown
🍳 Preparing data before training
🔬 Input data has 89 cells and 868 genes
⚖️ Scaling input data
🏋️ Training data using logistic regression
✅ Model training done!
```

Example 46 (unknown):
```unknown
CPU times: user 23.7 ms, sys: 8.92 ms, total: 32.6 ms
Wall time: 63.1 ms
```

Example 47 (python):
```python
%%time
# train with all genes
model = celltypist.train(adata_sel,  # all genes
                         labels = groupby,
                         n_jobs = 10,
                         feature_selection = False, # since manually selected
                         check_expression = False
                        )

# Save the model
model.write('/nfs/team205/heart/anndata_objects/Foetal/celltypist/model_YS_portal_object_20221208_onlyMP_allgenes_18Nov2023.pkl')
```

Example 48 (unknown):
```unknown
🍳 Preparing data before training
✂️ 11196 non-expressed genes are filtered out
🔬 Input data has 89 cells and 17336 genes
⚖️ Scaling input data
🏋️ Training data using logistic regression
✅ Model training done!
```

Example 49 (unknown):
```unknown
CPU times: user 55.9 ms, sys: 11.7 ms, total: 67.6 ms
Wall time: 697 ms
```

---

## Notebook

**URL:** http://127.0.0.1:9020/notebooks_html/Supplementary_materials/Macrophages/CellTypist_predict.html

**Contents:**
- Import modules¶
- Read in adata¶
- Subset macrophages¶
- CellTypist prediction¶
- Plot probability on umap¶
- Annotate YS-derived macrophage population¶
- Plot microglia marker genes¶

**Examples:**

Example 1 (python):
```python
import os
import numpy as np
import scanpy as sc
import anndata
import pandas as pd
import matplotlib as mpl
# enables correct plotting of text
mpl.rcParams['pdf.fonttype'] = 42
mpl.rcParams['ps.fonttype'] = 42
import matplotlib.pyplot as plt
import seaborn as sns

import celltypist
from celltypist import models

import warnings
warnings.simplefilter(action='ignore', category=FutureWarning)
warnings.simplefilter("ignore", UserWarning)
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
-----
anndata             0.9.2
celltypist          1.6.2
matplotlib          3.7.2
numpy               1.24.4
pandas              1.5.3
scanpy              1.9.3
seaborn             0.12.2
session_info        1.0.0
-----
```

Example 4 (python):
```python
PIL                         10.0.0
anyio                       NA
arrow                       1.2.3
asttokens                   NA
attr                        23.1.0
attrs                       23.1.0
babel                       2.12.1
backcall                    0.2.0
brotli                      1.0.9
certifi                     2023.11.17
cffi                        1.15.1
charset_normalizer          3.2.0
colorama                    0.4.6
comm                        0.1.4
cycler                      0.10.0
cython_runtime              NA
dateutil                    2.8.2
debugpy                     1.6.8
decorator                   5.1.1
defusedxml                  0.7.1
executing                   1.2.0
fastjsonschema              NA
fqdn                        NA
h5py                        3.9.0
idna                        3.4
igraph                      0.10.6
importlib_metadata          NA
importlib_resources         NA
ipykernel                   6.25.1
isoduration                 NA
jedi                        0.19.0
jinja2                      3.1.2
joblib                      1.3.2
json5                       NA
jsonpointer                 2.0
jsonschema                  4.19.0
jsonschema_specifications   NA
jupyter_events              0.7.0
jupyter_server              2.7.1
jupyterlab_server           2.24.0
kiwisolver                  1.4.5
leidenalg                   0.10.1
llvmlite                    0.40.1
markupsafe                  2.1.3
matplotlib_inline           0.1.6
mpl_toolkits                NA
natsort                     8.4.0
nbformat                    5.9.2
numba                       0.57.1
overrides                   NA
packaging                   23.1
parso                       0.8.3
patsy                       0.5.3
pexpect                     4.8.0
pickleshare                 0.7.5
pkg_resources               NA
plotly                      5.16.1
prometheus_client           NA
prompt_toolkit              3.0.39
psutil                      5.9.5
ptyprocess                  0.7.0
pure_eval                   0.2.2
pydev_ipython               NA
pydevconsole                NA
pydevd                      2.9.5
pydevd_file_utils           NA
pydevd_plugins              NA
pydevd_tracing              NA
pygments                    2.16.1
pyparsing                   3.0.9
pythonjsonlogger            NA
pytz                        2023.3
referencing                 NA
requests                    2.31.0
rfc3339_validator           0.1.4
rfc3986_validator           0.1.1
rpds                        NA
ruamel                      NA
scipy                       1.10.1
send2trash                  NA
six                         1.16.0
sklearn                     1.3.0
sniffio                     1.3.0
socks                       1.7.1
sphinxcontrib               NA
stack_data                  0.6.2
statsmodels                 0.14.0
texttable                   1.6.7
threadpoolctl               3.2.0
tornado                     6.3.3
traitlets                   5.9.0
typing_extensions           NA
uri_template                NA
urllib3                     2.0.4
wcwidth                     0.2.6
webcolors                   1.13
websocket                   1.6.2
yaml                        6.0.1
zipp                        NA
zmq                         25.1.1
zstandard                   0.19.0
```

Example 5 (unknown):
```unknown
-----
IPython             8.12.0
jupyter_client      8.3.0
jupyter_core        4.12.0
jupyterlab          4.0.5
notebook            7.0.2
-----
Python 3.8.17 (default, Jul  5 2023, 21:04:15) [GCC 11.2.0]
Linux-5.4.0-169-generic-x86_64-with-glibc2.17
-----
Session information updated at 2024-02-05 12:40
```

Example 6 (python):
```python
import importlib.util
import sys

spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/utils.py")
utils = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = utils
spec.loader.exec_module(utils)

spec = importlib.util.spec_from_file_location("module.name", "/nfs/team205/kk18/function/python/tissueaxis.py")
tissueaxis = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = tissueaxis
spec.loader.exec_module(tissueaxis)
```

Example 7 (python):
```python
sc.settings.set_figure_params(dpi=80,dpi_save=300)
```

Example 8 (python):
```python
%matplotlib inline
```

Example 9 (unknown):
```unknown
figdir = '/nfs/team205/heart/fetal_ms_figures/raw_plots_kk'
```

Example 10 (python):
```python
# read in
adata_dir = '/nfs/team205/heart/anndata_objects/Foetal/Immune/MoMP_Aug2023'
adata = sc.read_h5ad(f'{adata_dir}/MultiVI-post_foetal_Mono-MP_withContCov-CCCov_13Mar2023_feature-select-0003.h5ad')

# get gene expression data
adata_gex = adata.uns['adata_gex_lognorm'].copy()
print(adata_gex.X.data[:5])
adata_gex
```

Example 11 (unknown):
```unknown
[0.9766816 0.9766816 1.7862254 2.7663608 0.9766816]
```

Example 12 (python):
```python
AnnData object with n_obs × n_vars = 7674 × 15922
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', '_indices', '_scvi_batch', '_scvi_labels', 'fine_grain_withcycling', 't', 'seg', 'edge', 't_sd', 'milestones'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id', 'n_cells'
    uns: '_scvi_manager_uuid', '_scvi_uuid', 'adata_auc_gene', 'adata_auc_region', 'adata_gex', 'adata_gex_lognorm', 'draw_graph', 'fine_grain_colors', 'fine_grain_withcycling_colors', 'graph', 'log1p', 'milestones_colors', 'neighbors', 'ppt', 'pseudotime_list', 'seg_colors', 'umap', 'week_colors'
    obsm: 'MultiVI_latent', 'X_R', 'X_draw_graph_fa', 'X_palantir', 'X_umap', 'X_umap13', '_scvi_extra_categorical_covs', '_scvi_extra_continuous_covs'
    obsp: 'connectivities', 'distances'
```

Example 13 (python):
```python
adata_gex.var.head()
```

Example 14 (python):
```python
sc.pl.draw_graph(adata_gex,color=['seg','milestones'],cmap='RdPu',size=30)
```

Example 15 (python):
```python
cells_to_select = ['MacrophagesATF3pos',
 'MacrophagesCX3CR1pos',
 'MacrophagesLYVE1pos',
 'MacrophagesTIMD4pos']

adata_gex = adata_gex[adata_gex.obs['fine_grain'].isin(cells_to_select)]
adata_gex.obs['fine_grain'].value_counts()
```

Example 16 (unknown):
```unknown
MacrophagesLYVE1pos     4939
MacrophagesATF3pos       898
MacrophagesCX3CR1pos     826
MacrophagesTIMD4pos      112
Name: fine_grain, dtype: int64
```

Example 17 (python):
```python
set(adata_gex.obs['kit_10x'])
```

Example 18 (unknown):
```unknown
{'5prime-v2', 'Multiome-v1'}
```

Example 19 (python):
```python
adata_gex.obs['milestones'].value_counts()
```

Example 20 (unknown):
```unknown
47    2762
5     1105
28     872
11     608
32     518
20     340
44     339
29     205
Name: milestones, dtype: int64
```

Example 21 (python):
```python
sc.pl.draw_graph(adata_gex,color=['seg','milestones'],cmap='RdPu',size=30)
```

Example 22 (python):
```python
adata_gex.X.data[:10]
```

Example 23 (unknown):
```unknown
array([1.7789584, 1.7789584, 1.7789584, 2.3839223, 1.7789584, 1.7789584,
       1.7789584, 2.3839223, 2.7581758, 1.7789584], dtype=float32)
```

Example 24 (python):
```python
predictions = celltypist.annotate(adata_gex, 
                                  model = '/nfs/team205/heart/anndata_objects/Foetal/celltypist/model_YS_portal_object_20221208_onlyMP_hvg_18Nov2023.pkl', 
                                  majority_voting = False)

# plot
celltypist.dotplot(predictions, use_as_reference = 'milestones', use_as_prediction = 'predicted_labels', vmax=1, show=False)
plt.savefig(f'{figdir}/MoMP_celltypist-prediction_milestones.pdf', bbox_inches="tight",dpi=300)

celltypist.dotplot(predictions, use_as_reference = 'seg', use_as_prediction = 'predicted_labels', vmax=1)
```

Example 25 (unknown):
```unknown
🔬 Input data has 6749 cells and 15922 genes
🔗 Matching reference genes in the model
🧬 828 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
maxp pruned
cmap pruned
kern dropped
post pruned
FFTM dropped
GPOS pruned
GSUB pruned
glyf pruned
Added gid0 to subset
Added first four glyphs to subset
Closing glyph list over 'GSUB': 46 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'C', 'E', 'F', 'G', 'H', 'I', 'L', 'M', 'O', 'P', 'R', 'T', 'a', 'b', 'c', 'e', 'eight', 'f', 'five', 'four', 'i', 'l', 'n', 'nine', 'nonmarkingreturn', 'o', 'one', 'p', 'parenleft', 'parenright', 'percent', 'period', 'r', 's', 'seven', 'six', 'space', 't', 'three', 'two', 'underscore', 'y', 'zero']
Glyph IDs:   [0, 1, 2, 3, 8, 11, 12, 17, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 36, 38, 40, 41, 42, 43, 44, 47, 48, 50, 51, 53, 55, 66, 68, 69, 70, 72, 73, 76, 79, 81, 82, 83, 85, 86, 87, 92]
Closed glyph list over 'GSUB': 51 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'C', 'E', 'F', 'G', 'H', 'I', 'L', 'M', 'O', 'P', 'R', 'T', 'a', 'b', 'c', 'e', 'eight', 'f', 'fi', 'five', 'fl', 'four', 'i', 'l', 'n', 'nine', 'nonmarkingreturn', 'o', 'one', 'p', 'parenleft', 'parenright', 'percent', 'period', 'r', 's', 'seven', 'six', 'space', 't', 'three', 'two', 'underscore', 'uniFB00', 'uniFB03', 'uniFB04', 'y', 'zero']
Glyph IDs:   [0, 1, 2, 3, 8, 11, 12, 17, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 36, 38, 40, 41, 42, 43, 44, 47, 48, 50, 51, 53, 55, 66, 68, 69, 70, 72, 73, 76, 79, 81, 82, 83, 85, 86, 87, 92, 5038, 5039, 5040, 5041, 5042]
Closing glyph list over 'MATH': 51 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'C', 'E', 'F', 'G', 'H', 'I', 'L', 'M', 'O', 'P', 'R', 'T', 'a', 'b', 'c', 'e', 'eight', 'f', 'fi', 'five', 'fl', 'four', 'i', 'l', 'n', 'nine', 'nonmarkingreturn', 'o', 'one', 'p', 'parenleft', 'parenright', 'percent', 'period', 'r', 's', 'seven', 'six', 'space', 't', 'three', 'two', 'underscore', 'uniFB00', 'uniFB03', 'uniFB04', 'y', 'zero']
Glyph IDs:   [0, 1, 2, 3, 8, 11, 12, 17, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 36, 38, 40, 41, 42, 43, 44, 47, 48, 50, 51, 53, 55, 66, 68, 69, 70, 72, 73, 76, 79, 81, 82, 83, 85, 86, 87, 92, 5038, 5039, 5040, 5041, 5042]
Closed glyph list over 'MATH': 57 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'C', 'E', 'F', 'G', 'H', 'I', 'L', 'M', 'O', 'P', 'R', 'T', 'a', 'b', 'c', 'e', 'eight', 'f', 'fi', 'five', 'fl', 'four', 'i', 'l', 'n', 'nine', 'nonmarkingreturn', 'o', 'one', 'p', 'parenleft', 'parenright', 'percent', 'period', 'r', 's', 'seven', 'six', 'space', 't', 'three', 'two', 'underscore', 'uni239B', 'uni239C', 'uni239D', 'uni239E', 'uni239F', 'uni23A0', 'uniFB00', 'uniFB03', 'uniFB04', 'y', 'zero']
Glyph IDs:   [0, 1, 2, 3, 8, 11, 12, 17, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 36, 38, 40, 41, 42, 43, 44, 47, 48, 50, 51, 53, 55, 66, 68, 69, 70, 72, 73, 76, 79, 81, 82, 83, 85, 86, 87, 92, 3506, 3507, 3508, 3509, 3510, 3511, 5038, 5039, 5040, 5041, 5042]
Closing glyph list over 'glyf': 57 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'C', 'E', 'F', 'G', 'H', 'I', 'L', 'M', 'O', 'P', 'R', 'T', 'a', 'b', 'c', 'e', 'eight', 'f', 'fi', 'five', 'fl', 'four', 'i', 'l', 'n', 'nine', 'nonmarkingreturn', 'o', 'one', 'p', 'parenleft', 'parenright', 'percent', 'period', 'r', 's', 'seven', 'six', 'space', 't', 'three', 'two', 'underscore', 'uni239B', 'uni239C', 'uni239D', 'uni239E', 'uni239F', 'uni23A0', 'uniFB00', 'uniFB03', 'uniFB04', 'y', 'zero']
Glyph IDs:   [0, 1, 2, 3, 8, 11, 12, 17, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 36, 38, 40, 41, 42, 43, 44, 47, 48, 50, 51, 53, 55, 66, 68, 69, 70, 72, 73, 76, 79, 81, 82, 83, 85, 86, 87, 92, 3506, 3507, 3508, 3509, 3510, 3511, 5038, 5039, 5040, 5041, 5042]
Closed glyph list over 'glyf': 57 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'C', 'E', 'F', 'G', 'H', 'I', 'L', 'M', 'O', 'P', 'R', 'T', 'a', 'b', 'c', 'e', 'eight', 'f', 'fi', 'five', 'fl', 'four', 'i', 'l', 'n', 'nine', 'nonmarkingreturn', 'o', 'one', 'p', 'parenleft', 'parenright', 'percent', 'period', 'r', 's', 'seven', 'six', 'space', 't', 'three', 'two', 'underscore', 'uni239B', 'uni239C', 'uni239D', 'uni239E', 'uni239F', 'uni23A0', 'uniFB00', 'uniFB03', 'uniFB04', 'y', 'zero']
Glyph IDs:   [0, 1, 2, 3, 8, 11, 12, 17, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 36, 38, 40, 41, 42, 43, 44, 47, 48, 50, 51, 53, 55, 66, 68, 69, 70, 72, 73, 76, 79, 81, 82, 83, 85, 86, 87, 92, 3506, 3507, 3508, 3509, 3510, 3511, 5038, 5039, 5040, 5041, 5042]
Retaining 57 glyphs
head subsetting not needed
hhea subsetting not needed
maxp subsetting not needed
OS/2 subsetting not needed
hmtx subsetted
cmap subsetted
fpgm subsetting not needed
prep subsetting not needed
cvt  subsetting not needed
loca subsetting not needed
post subsetted
gasp subsetting not needed
MATH subsetted
GDEF subsetted
GPOS subsetted
GSUB subsetted
name subsetting not needed
glyf subsetted
head pruned
OS/2 Unicode ranges pruned: [0]
glyf pruned
GDEF pruned
GPOS pruned
GSUB pruned
name pruned
```

Example 26 (python):
```python
# add probability to adata
df = predictions.probability_matrix.copy()
adata_gex.obs[df.columns] = df.copy()
```

Example 27 (python):
```python
sc.pl.draw_graph(adata_gex,
                 color=['MACROPHAGE_PRE_MAC', 'MACROPHAGE_MICROGLIA', 'MACROPHAGE', ],
                 cmap='RdPu',size=30, show=False, frameon=False)
plt.savefig(f'{figdir}/Force-directed-graph_MoMP_YS_Mac_probability.pdf', bbox_inches="tight",dpi=300)
```

Example 28 (unknown):
```unknown
maxp pruned
cmap pruned
kern dropped
post pruned
FFTM dropped
GPOS pruned
GSUB pruned
glyf pruned
Added gid0 to subset
Added first four glyphs to subset
Closing glyph list over 'GSUB': 22 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'C', 'E', 'G', 'H', 'I', 'L', 'M', 'O', 'P', 'R', 'eight', 'four', 'nonmarkingreturn', 'period', 'six', 'space', 'two', 'underscore', 'zero']
Glyph IDs:   [0, 1, 2, 3, 17, 19, 21, 23, 25, 27, 36, 38, 40, 42, 43, 44, 47, 48, 50, 51, 53, 66]
Closed glyph list over 'GSUB': 22 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'C', 'E', 'G', 'H', 'I', 'L', 'M', 'O', 'P', 'R', 'eight', 'four', 'nonmarkingreturn', 'period', 'six', 'space', 'two', 'underscore', 'zero']
Glyph IDs:   [0, 1, 2, 3, 17, 19, 21, 23, 25, 27, 36, 38, 40, 42, 43, 44, 47, 48, 50, 51, 53, 66]
Closing glyph list over 'MATH': 22 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'C', 'E', 'G', 'H', 'I', 'L', 'M', 'O', 'P', 'R', 'eight', 'four', 'nonmarkingreturn', 'period', 'six', 'space', 'two', 'underscore', 'zero']
Glyph IDs:   [0, 1, 2, 3, 17, 19, 21, 23, 25, 27, 36, 38, 40, 42, 43, 44, 47, 48, 50, 51, 53, 66]
Closed glyph list over 'MATH': 22 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'C', 'E', 'G', 'H', 'I', 'L', 'M', 'O', 'P', 'R', 'eight', 'four', 'nonmarkingreturn', 'period', 'six', 'space', 'two', 'underscore', 'zero']
Glyph IDs:   [0, 1, 2, 3, 17, 19, 21, 23, 25, 27, 36, 38, 40, 42, 43, 44, 47, 48, 50, 51, 53, 66]
Closing glyph list over 'glyf': 22 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'C', 'E', 'G', 'H', 'I', 'L', 'M', 'O', 'P', 'R', 'eight', 'four', 'nonmarkingreturn', 'period', 'six', 'space', 'two', 'underscore', 'zero']
Glyph IDs:   [0, 1, 2, 3, 17, 19, 21, 23, 25, 27, 36, 38, 40, 42, 43, 44, 47, 48, 50, 51, 53, 66]
Closed glyph list over 'glyf': 22 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'C', 'E', 'G', 'H', 'I', 'L', 'M', 'O', 'P', 'R', 'eight', 'four', 'nonmarkingreturn', 'period', 'six', 'space', 'two', 'underscore', 'zero']
Glyph IDs:   [0, 1, 2, 3, 17, 19, 21, 23, 25, 27, 36, 38, 40, 42, 43, 44, 47, 48, 50, 51, 53, 66]
Retaining 22 glyphs
head subsetting not needed
hhea subsetting not needed
maxp subsetting not needed
OS/2 subsetting not needed
hmtx subsetted
cmap subsetted
fpgm subsetting not needed
prep subsetting not needed
cvt  subsetting not needed
loca subsetting not needed
post subsetted
gasp subsetting not needed
MATH subsetted
GDEF subsetted
GPOS subsetted
GSUB subsetted
name subsetting not needed
glyf subsetted
head pruned
OS/2 Unicode ranges pruned: [0]
glyf pruned
GDEF pruned
GPOS pruned
GSUB pruned
name pruned
```

Example 29 (python):
```python
pd.crosstab(adata.obs['fine_grain'],adata.obs['seg'])
```

Example 30 (python):
```python
adata.obs['fine_grain_mod'] = adata.obs['fine_grain'].astype('str').copy()
obsnames = adata.obs_names[adata.obs['milestones']=='29']
adata.obs.loc[obsnames,'fine_grain_mod'] = 'MacrophagesYSderived'
adata.obs['fine_grain_mod'].value_counts()
```

Example 31 (unknown):
```unknown
MacrophagesLYVE1pos     4757
MacrophagesATF3pos       898
MacrophagesCX3CR1pos     804
Monocytes                496
MacrophagesYSderived     207
MonocytesMPOpos          202
MonocyteDerivedCells     199
MacrophagesTIMD4pos      111
Name: fine_grain_mod, dtype: int64
```

Example 32 (python):
```python
sc.pl.draw_graph(adata,color=['fine_grain_mod'],cmap='RdPu',size=20,
                 frameon=False, show=False
                )
plt.savefig(f'{figdir}/Force-directed-graph_MoMP_celltype_annotation.pdf', bbox_inches="tight",dpi=300)
```

Example 33 (unknown):
```unknown
... storing 'fine_grain_mod' as categorical
maxp pruned
cmap pruned
kern dropped
post pruned
FFTM dropped
GPOS pruned
GSUB pruned
glyf pruned
Added gid0 to subset
Added first four glyphs to subset
Closing glyph list over 'GSUB': 42 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'C', 'D', 'E', 'F', 'I', 'L', 'M', 'O', 'P', 'R', 'S', 'T', 'V', 'X', 'Y', 'a', 'c', 'd', 'e', 'f', 'four', 'g', 'h', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'one', 'p', 'r', 's', 'space', 't', 'three', 'underscore', 'v', 'y']
Glyph IDs:   [0, 1, 2, 3, 20, 22, 23, 36, 38, 39, 40, 41, 44, 47, 48, 50, 51, 53, 54, 55, 57, 59, 60, 66, 68, 70, 71, 72, 73, 74, 75, 76, 79, 80, 81, 82, 83, 85, 86, 87, 89, 92]
Closed glyph list over 'GSUB': 47 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'C', 'D', 'E', 'F', 'I', 'L', 'M', 'O', 'P', 'R', 'S', 'T', 'V', 'X', 'Y', 'a', 'c', 'd', 'e', 'f', 'fi', 'fl', 'four', 'g', 'h', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'one', 'p', 'r', 's', 'space', 't', 'three', 'underscore', 'uniFB00', 'uniFB03', 'uniFB04', 'v', 'y']
Glyph IDs:   [0, 1, 2, 3, 20, 22, 23, 36, 38, 39, 40, 41, 44, 47, 48, 50, 51, 53, 54, 55, 57, 59, 60, 66, 68, 70, 71, 72, 73, 74, 75, 76, 79, 80, 81, 82, 83, 85, 86, 87, 89, 92, 5038, 5039, 5040, 5041, 5042]
Closing glyph list over 'MATH': 47 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'C', 'D', 'E', 'F', 'I', 'L', 'M', 'O', 'P', 'R', 'S', 'T', 'V', 'X', 'Y', 'a', 'c', 'd', 'e', 'f', 'fi', 'fl', 'four', 'g', 'h', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'one', 'p', 'r', 's', 'space', 't', 'three', 'underscore', 'uniFB00', 'uniFB03', 'uniFB04', 'v', 'y']
Glyph IDs:   [0, 1, 2, 3, 20, 22, 23, 36, 38, 39, 40, 41, 44, 47, 48, 50, 51, 53, 54, 55, 57, 59, 60, 66, 68, 70, 71, 72, 73, 74, 75, 76, 79, 80, 81, 82, 83, 85, 86, 87, 89, 92, 5038, 5039, 5040, 5041, 5042]
Closed glyph list over 'MATH': 47 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'C', 'D', 'E', 'F', 'I', 'L', 'M', 'O', 'P', 'R', 'S', 'T', 'V', 'X', 'Y', 'a', 'c', 'd', 'e', 'f', 'fi', 'fl', 'four', 'g', 'h', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'one', 'p', 'r', 's', 'space', 't', 'three', 'underscore', 'uniFB00', 'uniFB03', 'uniFB04', 'v', 'y']
Glyph IDs:   [0, 1, 2, 3, 20, 22, 23, 36, 38, 39, 40, 41, 44, 47, 48, 50, 51, 53, 54, 55, 57, 59, 60, 66, 68, 70, 71, 72, 73, 74, 75, 76, 79, 80, 81, 82, 83, 85, 86, 87, 89, 92, 5038, 5039, 5040, 5041, 5042]
Closing glyph list over 'glyf': 47 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'C', 'D', 'E', 'F', 'I', 'L', 'M', 'O', 'P', 'R', 'S', 'T', 'V', 'X', 'Y', 'a', 'c', 'd', 'e', 'f', 'fi', 'fl', 'four', 'g', 'h', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'one', 'p', 'r', 's', 'space', 't', 'three', 'underscore', 'uniFB00', 'uniFB03', 'uniFB04', 'v', 'y']
Glyph IDs:   [0, 1, 2, 3, 20, 22, 23, 36, 38, 39, 40, 41, 44, 47, 48, 50, 51, 53, 54, 55, 57, 59, 60, 66, 68, 70, 71, 72, 73, 74, 75, 76, 79, 80, 81, 82, 83, 85, 86, 87, 89, 92, 5038, 5039, 5040, 5041, 5042]
Closed glyph list over 'glyf': 47 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'C', 'D', 'E', 'F', 'I', 'L', 'M', 'O', 'P', 'R', 'S', 'T', 'V', 'X', 'Y', 'a', 'c', 'd', 'e', 'f', 'fi', 'fl', 'four', 'g', 'h', 'i', 'l', 'm', 'n', 'nonmarkingreturn', 'o', 'one', 'p', 'r', 's', 'space', 't', 'three', 'underscore', 'uniFB00', 'uniFB03', 'uniFB04', 'v', 'y']
Glyph IDs:   [0, 1, 2, 3, 20, 22, 23, 36, 38, 39, 40, 41, 44, 47, 48, 50, 51, 53, 54, 55, 57, 59, 60, 66, 68, 70, 71, 72, 73, 74, 75, 76, 79, 80, 81, 82, 83, 85, 86, 87, 89, 92, 5038, 5039, 5040, 5041, 5042]
Retaining 47 glyphs
head subsetting not needed
hhea subsetting not needed
maxp subsetting not needed
OS/2 subsetting not needed
hmtx subsetted
cmap subsetted
fpgm subsetting not needed
prep subsetting not needed
cvt  subsetting not needed
loca subsetting not needed
post subsetted
gasp subsetting not needed
MATH subsetted
GDEF subsetted
GPOS subsetted
GSUB subsetted
name subsetting not needed
glyf subsetted
head pruned
OS/2 Unicode ranges pruned: [0]
glyf pruned
GDEF pruned
GPOS pruned
GSUB pruned
name pruned
```

Example 34 (python):
```python
# save
adata.write(f'{outdir}/MultiVI-post_foetal_Mono-MP_withContCov-CCCov_13Mar2023_feature-select-0003.h5ad')
```

Example 35 (unknown):
```unknown
f'{outdir}/MultiVI-post_foetal_Mono-MP_withContCov-CCCov_13Mar2023_feature-select-0003.h5ad'
```

Example 36 (unknown):
```unknown
'/nfs/team205/heart/anndata_objects/Foetal/Immune/MoMP_Aug2023/MultiVI-post_foetal_Mono-MP_withContCov-CCCov_13Mar2023_feature-select-0003.h5ad'
```

Example 37 (python):
```python
adata_gex.obs['fine_grain_mod'] = adata.obs['fine_grain_mod'].reindex(adata_gex.obs_names)

predictions = celltypist.annotate(adata_gex, 
                                  model = '/nfs/team205/heart/anndata_objects/Foetal/celltypist/model_YS_portal_object_20221208_onlyMP_hvg_18Nov2023.pkl', 
                                  majority_voting = False
                                 )

# plot
celltypist.dotplot(predictions, use_as_reference = 'fine_grain_mod', use_as_prediction = 'predicted_labels', vmax=1, show=False)
plt.savefig(f'{figdir}/MoMP_celltypist-prediction_celltypes.pdf', bbox_inches="tight",dpi=300)
```

Example 38 (unknown):
```unknown
🔬 Input data has 6749 cells and 15922 genes
🔗 Matching reference genes in the model
🧬 828 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
maxp pruned
cmap pruned
kern dropped
post pruned
FFTM dropped
GPOS pruned
GSUB pruned
glyf pruned
Added gid0 to subset
Added first four glyphs to subset
Closing glyph list over 'GSUB': 53 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'L', 'M', 'O', 'P', 'R', 'S', 'T', 'V', 'X', 'Y', 'a', 'b', 'c', 'd', 'e', 'eight', 'f', 'five', 'four', 'g', 'h', 'i', 'l', 'n', 'nonmarkingreturn', 'o', 'one', 'p', 'parenleft', 'parenright', 'percent', 'period', 'r', 's', 'six', 'space', 't', 'three', 'two', 'underscore', 'v', 'y', 'zero']
Glyph IDs:   [0, 1, 2, 3, 8, 11, 12, 17, 19, 20, 21, 22, 23, 24, 25, 27, 36, 38, 39, 40, 41, 42, 43, 44, 47, 48, 50, 51, 53, 54, 55, 57, 59, 60, 66, 68, 69, 70, 71, 72, 73, 74, 75, 76, 79, 81, 82, 83, 85, 86, 87, 89, 92]
Closed glyph list over 'GSUB': 58 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'L', 'M', 'O', 'P', 'R', 'S', 'T', 'V', 'X', 'Y', 'a', 'b', 'c', 'd', 'e', 'eight', 'f', 'fi', 'five', 'fl', 'four', 'g', 'h', 'i', 'l', 'n', 'nonmarkingreturn', 'o', 'one', 'p', 'parenleft', 'parenright', 'percent', 'period', 'r', 's', 'six', 'space', 't', 'three', 'two', 'underscore', 'uniFB00', 'uniFB03', 'uniFB04', 'v', 'y', 'zero']
Glyph IDs:   [0, 1, 2, 3, 8, 11, 12, 17, 19, 20, 21, 22, 23, 24, 25, 27, 36, 38, 39, 40, 41, 42, 43, 44, 47, 48, 50, 51, 53, 54, 55, 57, 59, 60, 66, 68, 69, 70, 71, 72, 73, 74, 75, 76, 79, 81, 82, 83, 85, 86, 87, 89, 92, 5038, 5039, 5040, 5041, 5042]
Closing glyph list over 'MATH': 58 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'L', 'M', 'O', 'P', 'R', 'S', 'T', 'V', 'X', 'Y', 'a', 'b', 'c', 'd', 'e', 'eight', 'f', 'fi', 'five', 'fl', 'four', 'g', 'h', 'i', 'l', 'n', 'nonmarkingreturn', 'o', 'one', 'p', 'parenleft', 'parenright', 'percent', 'period', 'r', 's', 'six', 'space', 't', 'three', 'two', 'underscore', 'uniFB00', 'uniFB03', 'uniFB04', 'v', 'y', 'zero']
Glyph IDs:   [0, 1, 2, 3, 8, 11, 12, 17, 19, 20, 21, 22, 23, 24, 25, 27, 36, 38, 39, 40, 41, 42, 43, 44, 47, 48, 50, 51, 53, 54, 55, 57, 59, 60, 66, 68, 69, 70, 71, 72, 73, 74, 75, 76, 79, 81, 82, 83, 85, 86, 87, 89, 92, 5038, 5039, 5040, 5041, 5042]
Closed glyph list over 'MATH': 64 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'L', 'M', 'O', 'P', 'R', 'S', 'T', 'V', 'X', 'Y', 'a', 'b', 'c', 'd', 'e', 'eight', 'f', 'fi', 'five', 'fl', 'four', 'g', 'h', 'i', 'l', 'n', 'nonmarkingreturn', 'o', 'one', 'p', 'parenleft', 'parenright', 'percent', 'period', 'r', 's', 'six', 'space', 't', 'three', 'two', 'underscore', 'uni239B', 'uni239C', 'uni239D', 'uni239E', 'uni239F', 'uni23A0', 'uniFB00', 'uniFB03', 'uniFB04', 'v', 'y', 'zero']
Glyph IDs:   [0, 1, 2, 3, 8, 11, 12, 17, 19, 20, 21, 22, 23, 24, 25, 27, 36, 38, 39, 40, 41, 42, 43, 44, 47, 48, 50, 51, 53, 54, 55, 57, 59, 60, 66, 68, 69, 70, 71, 72, 73, 74, 75, 76, 79, 81, 82, 83, 85, 86, 87, 89, 92, 3506, 3507, 3508, 3509, 3510, 3511, 5038, 5039, 5040, 5041, 5042]
Closing glyph list over 'glyf': 64 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'L', 'M', 'O', 'P', 'R', 'S', 'T', 'V', 'X', 'Y', 'a', 'b', 'c', 'd', 'e', 'eight', 'f', 'fi', 'five', 'fl', 'four', 'g', 'h', 'i', 'l', 'n', 'nonmarkingreturn', 'o', 'one', 'p', 'parenleft', 'parenright', 'percent', 'period', 'r', 's', 'six', 'space', 't', 'three', 'two', 'underscore', 'uni239B', 'uni239C', 'uni239D', 'uni239E', 'uni239F', 'uni23A0', 'uniFB00', 'uniFB03', 'uniFB04', 'v', 'y', 'zero']
Glyph IDs:   [0, 1, 2, 3, 8, 11, 12, 17, 19, 20, 21, 22, 23, 24, 25, 27, 36, 38, 39, 40, 41, 42, 43, 44, 47, 48, 50, 51, 53, 54, 55, 57, 59, 60, 66, 68, 69, 70, 71, 72, 73, 74, 75, 76, 79, 81, 82, 83, 85, 86, 87, 89, 92, 3506, 3507, 3508, 3509, 3510, 3511, 5038, 5039, 5040, 5041, 5042]
Closed glyph list over 'glyf': 64 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'L', 'M', 'O', 'P', 'R', 'S', 'T', 'V', 'X', 'Y', 'a', 'b', 'c', 'd', 'e', 'eight', 'f', 'fi', 'five', 'fl', 'four', 'g', 'h', 'i', 'l', 'n', 'nonmarkingreturn', 'o', 'one', 'p', 'parenleft', 'parenright', 'percent', 'period', 'r', 's', 'six', 'space', 't', 'three', 'two', 'underscore', 'uni239B', 'uni239C', 'uni239D', 'uni239E', 'uni239F', 'uni23A0', 'uniFB00', 'uniFB03', 'uniFB04', 'v', 'y', 'zero']
Glyph IDs:   [0, 1, 2, 3, 8, 11, 12, 17, 19, 20, 21, 22, 23, 24, 25, 27, 36, 38, 39, 40, 41, 42, 43, 44, 47, 48, 50, 51, 53, 54, 55, 57, 59, 60, 66, 68, 69, 70, 71, 72, 73, 74, 75, 76, 79, 81, 82, 83, 85, 86, 87, 89, 92, 3506, 3507, 3508, 3509, 3510, 3511, 5038, 5039, 5040, 5041, 5042]
Retaining 64 glyphs
head subsetting not needed
hhea subsetting not needed
maxp subsetting not needed
OS/2 subsetting not needed
hmtx subsetted
cmap subsetted
fpgm subsetting not needed
prep subsetting not needed
cvt  subsetting not needed
loca subsetting not needed
post subsetted
gasp subsetting not needed
MATH subsetted
GDEF subsetted
GPOS subsetted
GSUB subsetted
name subsetting not needed
glyf subsetted
head pruned
OS/2 Unicode ranges pruned: [0]
glyf pruned
GDEF pruned
GPOS pruned
GSUB pruned
name pruned
```

Example 39 (python):
```python
# get gene expression data
ad = adata.uns['adata_gex_lognorm'].copy()
ad.obs['fine_grain_mod'] = adata.obs['fine_grain_mod'].reindex(ad.obs_names)
print(ad.X.data[:5])
ad
```

Example 40 (unknown):
```unknown
[0.9766816 0.9766816 1.7862254 2.7663608 0.9766816]
```

Example 41 (python):
```python
AnnData object with n_obs × n_vars = 7674 × 15922
    obs: 'sangerID', 'combinedID', 'donor', 'facility', 'cell_or_nuclei', 'modality', 'kit_10x', 'scrublet_score', 'doublet_pval', 'doublet_bh_pval', 'n_genes', 'n_counts', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'total_counts_mt', 'log1p_total_counts_mt', 'pct_counts_mt', 'total_counts_ribo', 'log1p_total_counts_ribo', 'pct_counts_ribo', 'HB_score', 'multiplexed', 'batch_key', 'region', 'FACSgate', 'sex', 'week', 'trimester', 'fine_grain', 'mid_grain', 'coarse_grain', 'heart_or_greatvessels', 'cycling', 'S_score', 'G2M_score', 'phase', 'stress_score', 'hb1_score', '_indices', '_scvi_batch', '_scvi_labels', 'fine_grain_withcycling', 't', 'seg', 'edge', 't_sd', 'milestones', 'fine_grain_mod'
    var: 'modality', 'gene_name_scRNA-0', 'gene_id', 'n_cells'
    uns: '_scvi_manager_uuid', '_scvi_uuid', 'adata_auc_gene', 'adata_auc_region', 'adata_gex', 'adata_gex_lognorm', 'draw_graph', 'fine_grain_colors', 'fine_grain_withcycling_colors', 'graph', 'log1p', 'milestones_colors', 'neighbors', 'ppt', 'pseudotime_list', 'seg_colors', 'umap', 'week_colors'
    obsm: 'MultiVI_latent', 'X_R', 'X_draw_graph_fa', 'X_palantir', 'X_umap', 'X_umap13', '_scvi_extra_categorical_covs', '_scvi_extra_continuous_covs'
    obsp: 'connectivities', 'distances'
```

Example 42 (python):
```python
genes = ['CXCR1','TREM2','P2RY12','C3','TGFB1','IL1B','TLR2','TLR4','P2RX7','RAGE','CD47','ADGRG1']
genes = [x for x in genes if x in ad.var_names]

sc.pl.dotplot(ad,
              var_names=genes,
              groupby='fine_grain_mod',
              standard_scale = 'var'
             )
```

Example 43 (python):
```python
genes = ['CX3CR1','TREM2','P2RY12','ADGRG1','C3','TGFB1','IL1B']
genes = [x for x in genes if x in ad.var_names]

sc.pl.dotplot(ad,
              var_names=genes,
              groupby='fine_grain_mod',
              standard_scale = 'var',
              show=False
             )
plt.savefig(f'{figdir}/MoMP_microglia-markers_dotplot.pdf', bbox_inches="tight",dpi=300)
```

Example 44 (unknown):
```unknown
maxp pruned
cmap pruned
kern dropped
post pruned
FFTM dropped
GPOS pruned
GSUB pruned
glyf pruned
Added gid0 to subset
Added first four glyphs to subset
Closing glyph list over 'GSUB': 53 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'I', 'L', 'M', 'O', 'P', 'R', 'S', 'T', 'V', 'X', 'Y', 'a', 'c', 'd', 'e', 'f', 'five', 'four', 'g', 'h', 'i', 'l', 'n', 'nine', 'nonmarkingreturn', 'o', 'one', 'p', 'parenleft', 'parenright', 'percent', 'period', 'r', 's', 'seven', 'space', 't', 'three', 'two', 'u', 'v', 'x', 'y', 'zero']
Glyph IDs:   [0, 1, 2, 3, 8, 11, 12, 17, 19, 20, 21, 22, 23, 24, 26, 28, 36, 37, 38, 39, 40, 41, 42, 44, 47, 48, 50, 51, 53, 54, 55, 57, 59, 60, 68, 70, 71, 72, 73, 74, 75, 76, 79, 81, 82, 83, 85, 86, 87, 88, 89, 91, 92]
Closed glyph list over 'GSUB': 58 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'I', 'L', 'M', 'O', 'P', 'R', 'S', 'T', 'V', 'X', 'Y', 'a', 'c', 'd', 'e', 'f', 'fi', 'five', 'fl', 'four', 'g', 'h', 'i', 'l', 'n', 'nine', 'nonmarkingreturn', 'o', 'one', 'p', 'parenleft', 'parenright', 'percent', 'period', 'r', 's', 'seven', 'space', 't', 'three', 'two', 'u', 'uniFB00', 'uniFB03', 'uniFB04', 'v', 'x', 'y', 'zero']
Glyph IDs:   [0, 1, 2, 3, 8, 11, 12, 17, 19, 20, 21, 22, 23, 24, 26, 28, 36, 37, 38, 39, 40, 41, 42, 44, 47, 48, 50, 51, 53, 54, 55, 57, 59, 60, 68, 70, 71, 72, 73, 74, 75, 76, 79, 81, 82, 83, 85, 86, 87, 88, 89, 91, 92, 5038, 5039, 5040, 5041, 5042]
Closing glyph list over 'MATH': 58 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'I', 'L', 'M', 'O', 'P', 'R', 'S', 'T', 'V', 'X', 'Y', 'a', 'c', 'd', 'e', 'f', 'fi', 'five', 'fl', 'four', 'g', 'h', 'i', 'l', 'n', 'nine', 'nonmarkingreturn', 'o', 'one', 'p', 'parenleft', 'parenright', 'percent', 'period', 'r', 's', 'seven', 'space', 't', 'three', 'two', 'u', 'uniFB00', 'uniFB03', 'uniFB04', 'v', 'x', 'y', 'zero']
Glyph IDs:   [0, 1, 2, 3, 8, 11, 12, 17, 19, 20, 21, 22, 23, 24, 26, 28, 36, 37, 38, 39, 40, 41, 42, 44, 47, 48, 50, 51, 53, 54, 55, 57, 59, 60, 68, 70, 71, 72, 73, 74, 75, 76, 79, 81, 82, 83, 85, 86, 87, 88, 89, 91, 92, 5038, 5039, 5040, 5041, 5042]
Closed glyph list over 'MATH': 64 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'I', 'L', 'M', 'O', 'P', 'R', 'S', 'T', 'V', 'X', 'Y', 'a', 'c', 'd', 'e', 'f', 'fi', 'five', 'fl', 'four', 'g', 'h', 'i', 'l', 'n', 'nine', 'nonmarkingreturn', 'o', 'one', 'p', 'parenleft', 'parenright', 'percent', 'period', 'r', 's', 'seven', 'space', 't', 'three', 'two', 'u', 'uni239B', 'uni239C', 'uni239D', 'uni239E', 'uni239F', 'uni23A0', 'uniFB00', 'uniFB03', 'uniFB04', 'v', 'x', 'y', 'zero']
Glyph IDs:   [0, 1, 2, 3, 8, 11, 12, 17, 19, 20, 21, 22, 23, 24, 26, 28, 36, 37, 38, 39, 40, 41, 42, 44, 47, 48, 50, 51, 53, 54, 55, 57, 59, 60, 68, 70, 71, 72, 73, 74, 75, 76, 79, 81, 82, 83, 85, 86, 87, 88, 89, 91, 92, 3506, 3507, 3508, 3509, 3510, 3511, 5038, 5039, 5040, 5041, 5042]
Closing glyph list over 'glyf': 64 glyphs before
Glyph names: ['.notdef', '.null', 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'I', 'L', 'M', 'O', 'P', 'R', 'S', 'T', 'V', 'X', 'Y', 'a', 'c', 'd', 'e', 'f', 'fi', 'five', 'fl', 'four', 'g', 'h', 'i', 'l', 'n', 'nine', 'nonmarkingreturn', 'o', 'one', 'p', 'parenleft', 'parenright', 'percent', 'period', 'r', 's', 'seven', 'space', 't', 'three', 'two', 'u', 'uni239B', 'uni239C', 'uni239D', 'uni239E', 'uni239F', 'uni23A0', 'uniFB00', 'uniFB03', 'uniFB04', 'v', 'x', 'y', 'zero']
Glyph IDs:   [0, 1, 2, 3, 8, 11, 12, 17, 19, 20, 21, 22, 23, 24, 26, 28, 36, 37, 38, 39, 40, 41, 42, 44, 47, 48, 50, 51, 53, 54, 55, 57, 59, 60, 68, 70, 71, 72, 73, 74, 75, 76, 79, 81, 82, 83, 85, 86, 87, 88, 89, 91, 92, 3506, 3507, 3508, 3509, 3510, 3511, 5038, 5039, 5040, 5041, 5042]
Closed glyph list over 'glyf': 64 glyphs after
Glyph names: ['.notdef', '.null', 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'I', 'L', 'M', 'O', 'P', 'R', 'S', 'T', 'V', 'X', 'Y', 'a', 'c', 'd', 'e', 'f', 'fi', 'five', 'fl', 'four', 'g', 'h', 'i', 'l', 'n', 'nine', 'nonmarkingreturn', 'o', 'one', 'p', 'parenleft', 'parenright', 'percent', 'period', 'r', 's', 'seven', 'space', 't', 'three', 'two', 'u', 'uni239B', 'uni239C', 'uni239D', 'uni239E', 'uni239F', 'uni23A0', 'uniFB00', 'uniFB03', 'uniFB04', 'v', 'x', 'y', 'zero']
Glyph IDs:   [0, 1, 2, 3, 8, 11, 12, 17, 19, 20, 21, 22, 23, 24, 26, 28, 36, 37, 38, 39, 40, 41, 42, 44, 47, 48, 50, 51, 53, 54, 55, 57, 59, 60, 68, 70, 71, 72, 73, 74, 75, 76, 79, 81, 82, 83, 85, 86, 87, 88, 89, 91, 92, 3506, 3507, 3508, 3509, 3510, 3511, 5038, 5039, 5040, 5041, 5042]
Retaining 64 glyphs
head subsetting not needed
hhea subsetting not needed
maxp subsetting not needed
OS/2 subsetting not needed
hmtx subsetted
cmap subsetted
fpgm subsetting not needed
prep subsetting not needed
cvt  subsetting not needed
loca subsetting not needed
post subsetted
gasp subsetting not needed
MATH subsetted
GDEF subsetted
GPOS subsetted
GSUB subsetted
name subsetting not needed
glyf subsetted
head pruned
OS/2 Unicode ranges pruned: [0]
glyf pruned
GDEF pruned
GPOS pruned
GSUB pruned
name pruned
```

---

## Notebook

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/Xenium/no1_per-sample-processing/Hst45-HEA-0-FFPE-1/no2_celltypist_prediction.html

**Contents:**
- Variables¶
- Read in data¶
- Normalise the counts¶
- CellTypist prediction: coarse-grain¶
- CellTypist prediction: mid-grain¶
- celltypist prediction: fine-grain¶
- Save¶

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os

import celltypist
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 4 (python):
```python
sample_id = 'Hst45-HEA-0-FFPE-1'
path_adata = f'/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium/{sample_id}_5K_filtered_raw.h5ad'
celltypist_model_dir = '/rfs/project/rfs-iCNyzSAaucw/kk837/notebooks/Foetal/Xenium/5K/celltypist_models'
```

Example 5 (unknown):
```unknown
os.getcwd()
```

Example 6 (python):
```python
adata = sc.read_h5ad(path_adata)
adata
```

Example 7 (python):
```python
sc.pp.filter_genes(adata, min_cells=3)
```

Example 8 (python):
```python
adata.shape
```

Example 9 (python):
```python
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)
```

Example 10 (python):
```python
adata.X.data[:5]
```

Example 11 (python):
```python
# coarse-grained
predictions_b2c = celltypist.annotate(adata, 
                                  model = f'{celltypist_model_dir}/global2coarse.pkl', 
                                  majority_voting = False)

# add to anndata
adata = predictions_b2c.to_adata(insert_labels = True, insert_conf = True)
adata.obs.rename(columns={'predicted_labels':'celltypist_coarse',
                              'conf_score':'conf_score_coarse'
                             },inplace=True)
adata.obs.head()
```

Example 12 (python):
```python
sc.pl.spatial(adata,color=['celltypist_coarse','conf_score_coarse'],cmap='RdPu',spot_size=20,vmax='p99',ncols=1)
```

Example 13 (python):
```python
# celltypist confidence
plt.figure(figsize=(6, 2))
sns.histplot(adata.obs[f'conf_score_coarse'], kde=False, bins=100)
# plt.yscale('log')
plt.title('coarse_grained')
plt.show()
```

Example 14 (python):
```python
# check
conf_thresh = 0.5 # starting from relaxed threshold
pd.crosstab(adata.obs['celltypist_coarse'],adata.obs[f'conf_score_coarse']<conf_thresh)
```

Example 15 (python):
```python
# finalised coarse-grained labels
conf_thresh = 0.5
adata.obs['celltypist_coarse_fin'] = adata.obs['celltypist_coarse'].astype('str').copy()
low_quality_cells = adata.obs_names[adata.obs[f'conf_score_coarse']<conf_thresh]
adata.obs.loc[low_quality_cells,'celltypist_coarse_fin'] = 'unassigned'
adata.obs['celltypist_coarse_fin'].value_counts()
```

Example 16 (python):
```python
sc.pl.spatial(adata,color=['celltypist_coarse_fin'],cmap='RdPu',spot_size=20,vmax='p99',ncols=1)
```

Example 17 (python):
```python
adata.obs['celltypist_coarse2midmod'] = np.nan
adata.obs['conf_score_coarse2midmod'] = np.nan

for celltype in adata.obs['celltypist_coarse_fin'].unique():
    if celltype not in ['unassigned', 'Epicardium']: # 'Epicardium' is also 'Epicardium' in mid-grained
        print(celltype)
        # subset
        adata_sub = adata[adata.obs['celltypist_coarse_fin']==celltype]
        # predict
        predictions = celltypist.annotate(adata_sub, 
                                      model = f'{celltypist_model_dir}/coarse2midmod_{celltype}.pkl', 
                                      majority_voting = False)
        # add to anndata
        adata_sub = predictions.to_adata(insert_labels = True, insert_conf = True)
        adata.obs.loc[adata_sub.obs_names,'celltypist_coarse2midmod'] = adata_sub.obs['predicted_labels']
        adata.obs.loc[adata_sub.obs_names,'conf_score_coarse2midmod'] = adata_sub.obs['conf_score']
        print('')
```

Example 18 (python):
```python
sc.pl.spatial(adata,color=['celltypist_coarse2midmod','conf_score_coarse2midmod'],
              cmap='RdPu',spot_size=20,vmax='p99',ncols=1)
```

Example 19 (python):
```python
# check
conf_thresh = 0.8
pd.crosstab(adata.obs['celltypist_coarse2midmod'],adata.obs[f'conf_score_coarse2midmod']<conf_thresh)
```

Example 20 (python):
```python
# finalised mid-grained labels
conf_thresh = 0.8
adata.obs['celltypist_mid_fin'] = adata.obs['celltypist_coarse_fin'].astype('str').copy()
adata.obs.replace({'celltypist_mid_fin':{'Epicardium':'EpicardialCells'}},inplace=True)
passed = adata.obs_names[adata.obs['conf_score_coarse2midmod']>=conf_thresh]
low_quality_cells = adata.obs_names[adata.obs[f'conf_score_coarse2midmod']<conf_thresh]

adata.obs.loc[passed,'celltypist_mid_fin'] = adata.obs.loc[passed,'celltypist_coarse2midmod'].copy()
adata.obs.loc[low_quality_cells,'celltypist_mid_fin'] = 'unassigned'
adata.obs['celltypist_mid_fin'].value_counts()
```

Example 21 (python):
```python
sc.pl.spatial(adata,color=['celltypist_mid_fin'],
              cmap='RdPu',spot_size=20,vmax='p99',ncols=1)
```

Example 22 (python):
```python
adata.obs['celltypist_midmod2fine'] = np.nan
adata.obs['conf_score_midmod2fine'] = np.nan

for celltype in adata.obs['celltypist_mid_fin'].unique():
    if celltype not in ['unassigned', 'LymphaticEndothelialCells']: # 'LymphaticEndothelialCells' is also 'LymphaticEndothelialCells' in fine-grained
        print(celltype)
        # subset
        adata_sub = adata[adata.obs['celltypist_mid_fin']==celltype]
        # predict
        predictions = celltypist.annotate(adata_sub, 
                                      model = f'{celltypist_model_dir}/midmod2fine_{celltype}.pkl', 
                                      majority_voting = False)
        # add to anndata
        adata_sub = predictions.to_adata(insert_labels = True, insert_conf = True)
        adata.obs.loc[adata_sub.obs_names,'celltypist_midmod2fine'] = adata_sub.obs['predicted_labels']
        adata.obs.loc[adata_sub.obs_names,'conf_score_midmod2fine'] = adata_sub.obs['conf_score']
        print('')
```

Example 23 (python):
```python
sc.pl.spatial(adata,color=['celltypist_midmod2fine','conf_score_midmod2fine'],
              cmap='RdPu',spot_size=20,vmax='p99',ncols=1)
```

Example 24 (python):
```python
sc.pl.spatial(adata,color=['celltypist_midmod2fine'],
              groups=['SinoatrialNodePacemakerCells','AtrioventricularNodePacemakerCells'],
              cmap='RdPu',spot_size=20,vmax='p99',ncols=3)
```

Example 25 (python):
```python
sc.pl.spatial(adata,color=['celltypist_midmod2fine'],
              groups=['MacrophagesCX3CR1pos','MacrophagesLYVE1pos'],
              cmap='RdPu',spot_size=20,vmax='p99',ncols=3)
```

Example 26 (python):
```python
adata.write(f'/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium/{sample_id}_5K_filtered_lognorm.h5ad')
```

Example 27 (python):
```python
print(f'/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium/{sample_id}_5K_filtered_lognorm.h5ad')
```

---

## Notebook

**URL:** http://127.0.0.1:9020/notebooks_html/Fig1_General_processing/Xenium/no1_per-sample-processing/C194-HEA-0-FFPE-1/no2_celltypist_prediction.html

**Contents:**
- Variables¶
- Read in data¶
- Normalise the counts¶
- CellTypist prediction: coarse-grain¶
- CellTypist prediction: mid-grain¶
- celltypist prediction: fine-grain¶
- Save¶

**Examples:**

Example 1 (python):
```python
import matplotlib.pyplot as plt
import scanpy as sc
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os

import celltypist
```

Example 2 (python):
```python
import session_info
session_info.show()
```

Example 3 (python):
```python
-----
celltypist          1.6.3
matplotlib          3.10.1
numpy               2.1.3
pandas              2.2.3
scanpy              1.11.1
seaborn             0.13.2
session_info        v1.0.1
-----
```

Example 4 (python):
```python
PIL                 11.1.0
anndata             0.11.4
asciitree           NA
asttokens           NA
certifi             2025.01.31
charset_normalizer  3.4.1
cloudpickle         3.1.1
comm                0.2.2
cycler              0.12.1
cython_runtime      NA
dask                2024.11.2
dateutil            2.9.0.post0
debugpy             1.8.13
decorator           5.2.1
exceptiongroup      1.2.2
executing           2.2.0
h5py                3.13.0
idna                3.10
igraph              0.11.8
importlib_metadata  NA
ipykernel           6.29.5
jedi                0.19.2
jinja2              3.1.6
joblib              1.4.2
kiwisolver          1.4.8
legacy_api_wrap     NA
leidenalg           0.10.2
llvmlite            0.44.0
markupsafe          3.0.2
mpl_toolkits        NA
msgpack             1.1.0
natsort             8.4.0
numba               0.61.0
numcodecs           0.13.1
packaging           24.2
parso               0.8.4
patsy               1.0.1
platformdirs        4.3.7
prompt_toolkit      3.0.50
psutil              7.0.0
pure_eval           0.2.3
pyarrow             19.0.1
pydev_ipython       NA
pydevconsole        NA
pydevd              3.2.3
pydevd_file_utils   NA
pydevd_plugins      NA
pydevd_tracing      NA
pygments            2.19.1
pyparsing           3.2.3
pytz                2025.2
requests            2.32.3
scipy               1.15.2
six                 1.17.0
sklearn             1.5.2
stack_data          0.6.3
statsmodels         0.14.4
tblib               3.1.0
texttable           1.7.0
threadpoolctl       3.6.0
tlz                 1.0.0
toolz               1.0.0
tornado             6.4.2
traitlets           5.14.3
typing_extensions   NA
urllib3             2.3.0
wcwidth             0.2.13
yaml                6.0.2
zarr                2.18.3
zipp                NA
zmq                 26.3.0
zoneinfo            NA
```

Example 5 (unknown):
```unknown
-----
IPython             8.34.0
jupyter_client      8.6.3
jupyter_core        5.7.2
-----
Python 3.10.14 | packaged by conda-forge | (main, Mar 20 2024, 12:45:18) [GCC 12.3.0]
Linux-4.18.0-553.45.1.el8_10.x86_64-x86_64-with-glibc2.28
-----
Session information updated at 2025-05-28 10:28
```

Example 6 (python):
```python
sc.settings.set_figure_params(dpi=80)
```

Example 7 (python):
```python
sample_id = 'C194-HEA-0-FFPE-1'
path_adata = f'/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium/{sample_id}_5K_filtered_raw.h5ad'
celltypist_model_dir = '/rfs/project/rfs-iCNyzSAaucw/kk837/notebooks/Foetal/Xenium/5K/celltypist_models'
```

Example 8 (unknown):
```unknown
os.getcwd()
```

Example 9 (unknown):
```unknown
'/rfs/project/rfs-iCNyzSAaucw/kk837/notebooks/Foetal/Xenium/5K/C194-HEA-0-FFPE-1'
```

Example 10 (python):
```python
adata = sc.read_h5ad(path_adata)
adata
```

Example 11 (unknown):
```unknown
AnnData object with n_obs × n_vars = 144916 × 5001
    obs: 'cell_id', 'x_centroid', 'y_centroid', 'transcript_counts', 'control_probe_counts', 'genomic_control_counts', 'control_codeword_counts', 'unassigned_codeword_counts', 'deprecated_codeword_counts', 'total_counts', 'cell_area', 'nucleus_area', 'nucleus_count', 'segmentation_method', 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'log1p_total_counts', 'pct_counts_in_top_10_genes', 'pct_counts_in_top_20_genes', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_150_genes', 'n_counts', 'n_genes'
    var: 'gene_ids', 'feature_types', 'genome', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts'
    obsm: 'spatial'
```

Example 12 (python):
```python
sc.pp.filter_genes(adata, min_cells=3)
```

Example 13 (python):
```python
adata.shape
```

Example 14 (unknown):
```unknown
(144916, 5001)
```

Example 15 (python):
```python
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)
```

Example 16 (python):
```python
adata.X.data[:5]
```

Example 17 (unknown):
```unknown
array([4.2093534, 2.6576407, 2.6576407, 3.3151019, 3.3151019],
      dtype=float32)
```

Example 18 (python):
```python
# coarse-grained
predictions_b2c = celltypist.annotate(adata, 
                                  model = f'{celltypist_model_dir}/global2coarse.pkl', 
                                  majority_voting = False)

# add to anndata
adata = predictions_b2c.to_adata(insert_labels = True, insert_conf = True)
adata.obs.rename(columns={'predicted_labels':'celltypist_coarse',
                              'conf_score':'conf_score_coarse'
                             },inplace=True)
adata.obs.head()
```

Example 19 (unknown):
```unknown
🔬 Input data has 144916 cells and 5001 genes
🔗 Matching reference genes in the model
🧬 4975 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
```

Example 20 (python):
```python
sc.pl.spatial(adata,color=['celltypist_coarse','conf_score_coarse'],cmap='RdPu',spot_size=20,vmax='p99',ncols=1)
```

Example 21 (python):
```python
# celltypist confidence
plt.figure(figsize=(6, 2))
sns.histplot(adata.obs[f'conf_score_coarse'], kde=False, bins=100)
# plt.yscale('log')
plt.title('coarse_grained')
plt.show()
```

Example 22 (python):
```python
# check
conf_thresh = 0.5 # starting from relaxed threshold
pd.crosstab(adata.obs['celltypist_coarse'],adata.obs[f'conf_score_coarse']<conf_thresh)
```

Example 23 (python):
```python
# finalised coarse-grained labels
conf_thresh = 0.5
adata.obs['celltypist_coarse_fin'] = adata.obs['celltypist_coarse'].astype('str').copy()
low_quality_cells = adata.obs_names[adata.obs[f'conf_score_coarse']<conf_thresh]
adata.obs.loc[low_quality_cells,'celltypist_coarse_fin'] = 'unassigned'
adata.obs['celltypist_coarse_fin'].value_counts()
```

Example 24 (unknown):
```unknown
celltypist_coarse_fin
Cardiomyocytes    73158
Mesenchymal       44487
Endothelium       12863
Neural             5842
Leukocytes         4566
Epicardium         3420
unassigned          580
Name: count, dtype: int64
```

Example 25 (python):
```python
sc.pl.spatial(adata,color=['celltypist_coarse_fin'],cmap='RdPu',spot_size=20,vmax='p99',ncols=1)
```

Example 26 (unknown):
```unknown
... storing 'celltypist_coarse_fin' as categorical
```

Example 27 (python):
```python
adata.obs['celltypist_coarse2midmod'] = np.nan
adata.obs['conf_score_coarse2midmod'] = np.nan

for celltype in adata.obs['celltypist_coarse_fin'].unique():
    if celltype not in ['unassigned', 'Epicardium']: # 'Epicardium' is also 'Epicardium' in mid-grained
        print(celltype)
        # subset
        adata_sub = adata[adata.obs['celltypist_coarse_fin']==celltype]
        # predict
        predictions = celltypist.annotate(adata_sub, 
                                      model = f'{celltypist_model_dir}/coarse2midmod_{celltype}.pkl', 
                                      majority_voting = False)
        # add to anndata
        adata_sub = predictions.to_adata(insert_labels = True, insert_conf = True)
        adata.obs.loc[adata_sub.obs_names,'celltypist_coarse2midmod'] = adata_sub.obs['predicted_labels']
        adata.obs.loc[adata_sub.obs_names,'conf_score_coarse2midmod'] = adata_sub.obs['conf_score']
        print('')
```

Example 28 (unknown):
```unknown
🔬 Input data has 44487 cells and 5001 genes
🔗 Matching reference genes in the model
```

Example 29 (unknown):
```unknown
Mesenchymal
```

Example 30 (unknown):
```unknown
🧬 4958 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
🔬 Input data has 5842 cells and 5001 genes
🔗 Matching reference genes in the model
🧬 4802 features used for prediction
```

Example 31 (unknown):
```unknown
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
```

Example 32 (unknown):
```unknown
Cardiomyocytes
```

Example 33 (unknown):
```unknown
🔬 Input data has 73158 cells and 5001 genes
🔗 Matching reference genes in the model
🧬 4867 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
🔬 Input data has 4566 cells and 5001 genes
🔗 Matching reference genes in the model
```

Example 34 (unknown):
```unknown
🧬 4947 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
🔬 Input data has 12863 cells and 5001 genes
🔗 Matching reference genes in the model
🧬 4916 features used for prediction
```

Example 35 (unknown):
```unknown
Endothelium
```

Example 36 (unknown):
```unknown
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
```

Example 37 (python):
```python
sc.pl.spatial(adata,color=['celltypist_coarse2midmod','conf_score_coarse2midmod'],
              cmap='RdPu',spot_size=20,vmax='p99',ncols=1)
```

Example 38 (unknown):
```unknown
... storing 'celltypist_coarse2midmod' as categorical
```

Example 39 (python):
```python
# check
conf_thresh = 0.8
pd.crosstab(adata.obs['celltypist_coarse2midmod'],adata.obs[f'conf_score_coarse2midmod']<conf_thresh)
```

Example 40 (python):
```python
# finalised mid-grained labels
conf_thresh = 0.8
adata.obs['celltypist_mid_fin'] = adata.obs['celltypist_coarse_fin'].astype('str').copy()
adata.obs.replace({'celltypist_mid_fin':{'Epicardium':'EpicardialCells'}},inplace=True)
passed = adata.obs_names[adata.obs['conf_score_coarse2midmod']>=conf_thresh]
low_quality_cells = adata.obs_names[adata.obs[f'conf_score_coarse2midmod']<conf_thresh]

adata.obs.loc[passed,'celltypist_mid_fin'] = adata.obs.loc[passed,'celltypist_coarse2midmod'].copy()
adata.obs.loc[low_quality_cells,'celltypist_mid_fin'] = 'unassigned'
adata.obs['celltypist_mid_fin'].value_counts()
```

Example 41 (unknown):
```unknown
celltypist_mid_fin
VentricularCardiomyocytes      57761
Fibroblasts                    28224
AtrialCardiomyocytes           15042
MuralCells                      7747
BloodVesselEndothelialCells     7183
PericardialCells                5402
unassigned                      4630
Glia                            4142
EpicardialCells                 3420
EndocardialCells                3158
LymphoidCells                   2727
LymphaticEndothelialCells       2287
MyeloidCells                    1626
Neurons                         1567
Name: count, dtype: int64
```

Example 42 (python):
```python
sc.pl.spatial(adata,color=['celltypist_mid_fin'],
              cmap='RdPu',spot_size=20,vmax='p99',ncols=1)
```

Example 43 (unknown):
```unknown
... storing 'celltypist_mid_fin' as categorical
```

Example 44 (python):
```python
adata.obs['celltypist_midmod2fine'] = np.nan
adata.obs['conf_score_midmod2fine'] = np.nan

for celltype in adata.obs['celltypist_mid_fin'].unique():
    if celltype not in ['unassigned', 'LymphaticEndothelialCells']: # 'LymphaticEndothelialCells' is also 'LymphaticEndothelialCells' in fine-grained
        print(celltype)
        # subset
        adata_sub = adata[adata.obs['celltypist_mid_fin']==celltype]
        # predict
        predictions = celltypist.annotate(adata_sub, 
                                      model = f'{celltypist_model_dir}/midmod2fine_{celltype}.pkl', 
                                      majority_voting = False)
        # add to anndata
        adata_sub = predictions.to_adata(insert_labels = True, insert_conf = True)
        adata.obs.loc[adata_sub.obs_names,'celltypist_midmod2fine'] = adata_sub.obs['predicted_labels']
        adata.obs.loc[adata_sub.obs_names,'conf_score_midmod2fine'] = adata_sub.obs['conf_score']
        print('')
```

Example 45 (unknown):
```unknown
🔬 Input data has 28224 cells and 5001 genes
🔗 Matching reference genes in the model
```

Example 46 (unknown):
```unknown
Fibroblasts
```

Example 47 (unknown):
```unknown
🧬 4923 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
🔬 Input data has 5402 cells and 5001 genes
🔗 Matching reference genes in the model
🧬 4811 features used for prediction
```

Example 48 (unknown):
```unknown
PericardialCells
```

Example 49 (unknown):
```unknown
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
🔬 Input data has 7747 cells and 5001 genes
🔗 Matching reference genes in the model
🧬 4846 features used for prediction
```

Example 50 (unknown):
```unknown
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
🔬 Input data has 4142 cells and 5001 genes
🔗 Matching reference genes in the model
🧬 4706 features used for prediction
```

Example 51 (unknown):
```unknown
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
🔬 Input data has 1567 cells and 5001 genes
🔗 Matching reference genes in the model
```

Example 52 (unknown):
```unknown
🧬 4674 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
```

Example 53 (unknown):
```unknown
VentricularCardiomyocytes
```

Example 54 (unknown):
```unknown
🔬 Input data has 57761 cells and 5001 genes
🔗 Matching reference genes in the model
🧬 4822 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
🔬 Input data has 1626 cells and 5001 genes
🔗 Matching reference genes in the model
🧬 4875 features used for prediction
```

Example 55 (unknown):
```unknown
MyeloidCells
```

Example 56 (unknown):
```unknown
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
🔬 Input data has 2727 cells and 5001 genes
🔗 Matching reference genes in the model
🧬 4889 features used for prediction
```

Example 57 (unknown):
```unknown
LymphoidCells
```

Example 58 (unknown):
```unknown
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
🔬 Input data has 7183 cells and 5001 genes
🔗 Matching reference genes in the model
🧬 4833 features used for prediction
```

Example 59 (unknown):
```unknown
BloodVesselEndothelialCells
```

Example 60 (unknown):
```unknown
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
🔬 Input data has 15042 cells and 5001 genes
🔗 Matching reference genes in the model
🧬 4709 features used for prediction
```

Example 61 (unknown):
```unknown
AtrialCardiomyocytes
```

Example 62 (unknown):
```unknown
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
🔬 Input data has 3158 cells and 5001 genes
🔗 Matching reference genes in the model
🧬 4675 features used for prediction
```

Example 63 (unknown):
```unknown
EndocardialCells
```

Example 64 (unknown):
```unknown
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
🔬 Input data has 3420 cells and 5001 genes
🔗 Matching reference genes in the model
🧬 4829 features used for prediction
```

Example 65 (unknown):
```unknown
EpicardialCells
```

Example 66 (unknown):
```unknown
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
```

Example 67 (python):
```python
sc.pl.spatial(adata,color=['celltypist_midmod2fine','conf_score_midmod2fine'],
              cmap='RdPu',spot_size=20,vmax='p99',ncols=1)
```

Example 68 (unknown):
```unknown
... storing 'celltypist_midmod2fine' as categorical
```

Example 69 (python):
```python
sc.pl.spatial(adata,color=['celltypist_midmod2fine'],
              groups=['SinoatrialNodePacemakerCells','AtrioventricularNodePacemakerCells'],
              cmap='RdPu',spot_size=20,vmax='p99',ncols=3)
```

Example 70 (python):
```python
sc.pl.spatial(adata,color=['celltypist_midmod2fine'],
              groups=['MacrophagesCX3CR1pos','MacrophagesLYVE1pos'],
              cmap='RdPu',spot_size=20,vmax='p99',ncols=3)
```

Example 71 (python):
```python
adata.write(f'/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium/{sample_id}_5K_filtered_lognorm.h5ad')
```

Example 72 (python):
```python
print(f'/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium/{sample_id}_5K_filtered_lognorm.h5ad')
```

Example 73 (unknown):
```unknown
/home/kk837/rds/rds-teichlab-C9woKbOCf2Y/kk837/Foetal/anndata_objects/Xenium/C194-HEA-0-FFPE-1_5K_filtered_lognorm.h5ad
```

---
