Tutorial 4: detect novel cells in the test data via CANAL
[1]:
import os
os.chdir('/data/wanh/CANAL/')
import sys
sys.path.append('/data/wanh/CANAL/')
import argparse
from model import *
import time
import resource
Construct and run the CANAL model for three stages
[2]:
experiments = 'Pancreas_novel_cells'
seed = 1
with open(f'./data/Pancreas/Pancreas_highly_gene_idx.csv') as csvfile:
spamreader = csv.reader(csvfile, delimiter=' ', quotechar='|')
highly_variable_idx = []
for row in spamreader:
highly_variable_idx.append(row[0])
highly_variable_idx = [int(i) for i in highly_variable_idx]
highly_variable_idx = np.array(highly_variable_idx)
[3]:
CANAL = CANAL_model(gpu_option = '3')
[4]:
dataset_stage_1 = "Muraro"
data_path_stage_1 = './data/Pancreas/{}_train.h5ad'.format(dataset_stage_1)
adata_stage_1 = sc.read_h5ad(data_path_stage_1)
adata_stage_1 = adata_stage_1[np.array(adata_stage_1.obs['cell_type1']) != 'alpha']
cell_type_stage_1 = adata_stage_1.obs['cell_type1']
print(adata_stage_1)
print(np.unique(np.array(cell_type_stage_1), return_counts=True))
View of AnnData object with n_obs × n_vars = 985 × 1000
obs: 'donor', 'cell_type1', 'library', 'organism', 'dataset_name', 'platform', 'organ', 'data_type', 'cell_ontology_class', 'cell_ontology_id', 'n_genes', 'n_counts', '__libsize__', 'baron_human_donor', 'age', 'gender', 'enge_donor', 'batch', 'muraro_donor', 'celltype'
var: 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
uns: 'hvg', 'log1p'
(array(['acinar', 'beta', 'delta', 'ductal', 'gamma', 'mesenchymal'],
dtype=object), array([168, 352, 153, 177, 76, 59]))
[5]:
CANAL.train(experiments = experiments, pre_dataset = "None", dataset = dataset_stage_1,
adata = adata_stage_1, cell_type = cell_type_stage_1, current_stage = 1,
is_final_stage = False, ckpt_dir = './ckpts/', rehearsal_size = 1000,
highly_variable_idx=highly_variable_idx, SEED = seed)
current data: View of AnnData object with n_obs × n_vars = 985 × 1000
obs: 'donor', 'cell_type1', 'library', 'organism', 'dataset_name', 'platform', 'organ', 'data_type', 'cell_ontology_class', 'cell_ontology_id', 'n_genes', 'n_counts', '__libsize__', 'baron_human_donor', 'age', 'gender', 'enge_donor', 'batch', 'muraro_donor', 'celltype'
var: 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
uns: 'hvg', 'log1p'
(array(['acinar', 'beta', 'delta', 'ductal', 'gamma', 'mesenchymal'],
dtype=object), array([168, 352, 153, 177, 76, 59]))
model constructing begin!
/data/wanh/CANAL/performer_pytorch/performer_pytorch.py:115: UserWarning: torch.qr is deprecated in favor of torch.linalg.qr and will be removed in a future PyTorch release.
The boolean parameter 'some' has been replaced with a string parameter 'mode'.
Q, R = torch.qr(A, some)
should be replaced with
Q, R = torch.linalg.qr(A, 'reduced' if some else 'complete') (Triggered internally at ../aten/src/ATen/native/BatchLinearAlgebra.cpp:2497.)
q, r = torch.qr(unstructured_block.cpu(), some = True)
/data/wanh/ENTER/envs/pytorch/lib/python3.7/site-packages/torch/nn/_reduction.py:42: UserWarning: size_average and reduce args will be deprecated, please use reduction='mean' instead.
warnings.warn(warning.format(ret))
model constructing finished!
label train: [0 1 2 3 4 5] 6
label val: [0 1 2 3 4 5] 6
== Begin finetuning: | Initial stage | Current stage: 1 | CANAL | Dataset: Pancreas_novel_cells Muraro ==
== Epoch: 1 | Classification Loss: 1.795873 | Representation DL Loss: 0.000000 | Accuracy: 13.9031% ==
== Epoch: 1 | Validation CLS Loss: 1.784394 | Validation Representation DL Loss: 0.000000 | F1 Score: 0.057665 == | Current accuracy: 0.209184 ==
[[ 0 0 0 40 0 0]
[ 0 0 0 67 0 0]
[ 0 0 0 26 0 0]
[ 0 0 0 41 0 0]
[ 0 0 0 11 0 0]
[ 0 0 0 11 0 0]]
Patience: 0 / 10
== Epoch: 2 | Classification Loss: 1.667545 | Representation DL Loss: 0.000000 | Accuracy: 31.7602% ==
== Epoch: 2 | Validation CLS Loss: 1.639460 | Validation Representation DL Loss: 0.000000 | F1 Score: 0.083969 == | Current accuracy: 0.336735 ==
[[ 0 47 0 0 0 0]
[ 0 66 0 0 0 0]
[ 0 29 0 0 0 0]
[ 0 26 0 0 0 0]
[ 0 13 0 0 0 0]
[ 0 15 0 0 0 0]]
Patience: 0 / 10
== Epoch: 3 | Classification Loss: 1.235898 | Representation DL Loss: 0.000000 | Accuracy: 63.5204% ==
== Epoch: 3 | Validation CLS Loss: 2.269022 | Validation Representation DL Loss: 0.000000 | F1 Score: 0.619616 == | Current accuracy: 0.790816 ==
[[33 0 0 1 0 0]
[ 0 74 0 1 0 0]
[ 0 25 0 1 0 0]
[ 0 1 0 32 0 0]
[ 0 12 0 0 0 0]
[ 0 0 0 0 0 16]]
Patience: 0 / 10
== Epoch: 4 | Classification Loss: 0.868006 | Representation DL Loss: 0.000000 | Accuracy: 81.7602% ==
== Epoch: 4 | Validation CLS Loss: 2.626368 | Validation Representation DL Loss: 0.000000 | F1 Score: 0.740653 == | Current accuracy: 0.852041 ==
[[46 0 0 0 0 0]
[ 0 48 0 6 0 0]
[ 0 8 26 0 0 0]
[ 0 3 2 31 0 0]
[ 0 6 4 0 0 0]
[ 0 0 0 0 0 16]]
Patience: 0 / 10
== Epoch: 5 | Classification Loss: 0.739252 | Representation DL Loss: 0.000000 | Accuracy: 86.7347% ==
== Epoch: 5 | Validation CLS Loss: 2.270708 | Validation Representation DL Loss: 0.000000 | F1 Score: 0.604147 == | Current accuracy: 0.704082 ==
[[29 0 0 0 0 0]
[ 0 64 0 2 0 0]
[ 0 37 0 0 0 0]
[ 0 3 0 37 0 0]
[ 0 16 0 0 0 0]
[ 0 0 0 0 0 8]]
Patience: 1 / 10
== Epoch: 6 | Classification Loss: 0.701946 | Representation DL Loss: 0.000000 | Accuracy: 89.7959% ==
== Epoch: 6 | Validation CLS Loss: 3.186212 | Validation Representation DL Loss: 0.000000 | F1 Score: 0.797634 == | Current accuracy: 0.892857 ==
[[39 0 0 0 0 0]
[ 0 64 2 1 0 0]
[ 0 0 27 0 0 0]
[ 0 2 0 35 0 0]
[ 0 1 15 0 1 0]
[ 0 0 0 0 0 9]]
Patience: 0 / 10
== Epoch: 7 | Classification Loss: 0.622170 | Representation DL Loss: 0.000000 | Accuracy: 94.3878% ==
== Epoch: 7 | Validation CLS Loss: 2.905367 | Validation Representation DL Loss: 0.000000 | F1 Score: 0.948557 == | Current accuracy: 0.959184 ==
[[39 0 0 0 0 0]
[ 1 63 2 0 0 0]
[ 0 0 37 0 1 0]
[ 0 0 0 26 2 0]
[ 0 0 2 0 12 0]
[ 0 0 0 0 0 11]]
Patience: 0 / 10
== Epoch: 8 | Classification Loss: 0.590825 | Representation DL Loss: 0.000000 | Accuracy: 95.5357% ==
== Epoch: 8 | Validation CLS Loss: 2.882448 | Validation Representation DL Loss: 0.000000 | F1 Score: 0.959882 == | Current accuracy: 0.969388 ==
[[33 0 0 0 0 0]
[ 0 71 0 0 2 0]
[ 0 0 22 0 1 0]
[ 0 1 0 37 0 0]
[ 0 0 2 0 15 0]
[ 0 0 0 0 0 12]]
Patience: 0 / 10
== Epoch: 9 | Classification Loss: 0.574096 | Representation DL Loss: 0.000000 | Accuracy: 95.0255% ==
== Epoch: 9 | Validation CLS Loss: 3.051565 | Validation Representation DL Loss: 0.000000 | F1 Score: 0.821441 == | Current accuracy: 0.862245 ==
[[41 0 0 0 0 0]
[ 1 60 0 1 0 0]
[ 0 16 10 2 1 0]
[ 0 0 0 39 0 0]
[ 0 6 0 0 6 0]
[ 0 0 0 0 0 13]]
Patience: 1 / 10
== Epoch: 10 | Classification Loss: 0.559775 | Representation DL Loss: 0.000000 | Accuracy: 96.0459% ==
== Epoch: 10 | Validation CLS Loss: 3.055145 | Validation Representation DL Loss: 0.000000 | F1 Score: 0.990385 == | Current accuracy: 0.989796 ==
[[32 0 0 0 0 0]
[ 0 61 0 1 1 0]
[ 0 0 28 0 0 0]
[ 0 0 0 38 0 0]
[ 0 0 0 0 17 0]
[ 0 0 0 0 0 18]]
Patience: 0 / 10
== Epoch: 11 | Classification Loss: 0.539837 | Representation DL Loss: 0.000000 | Accuracy: 95.7908% ==
== Epoch: 11 | Validation CLS Loss: 2.862482 | Validation Representation DL Loss: 0.000000 | F1 Score: 0.955436 == | Current accuracy: 0.959184 ==
[[36 0 0 0 0 0]
[ 3 70 0 0 2 0]
[ 0 0 25 0 0 0]
[ 0 1 0 34 0 0]
[ 0 0 2 0 13 0]
[ 0 0 0 0 0 10]]
Patience: 1 / 10
== Epoch: 12 | Classification Loss: 0.530643 | Representation DL Loss: 0.000000 | Accuracy: 96.8112% ==
== Epoch: 12 | Validation CLS Loss: 2.848025 | Validation Representation DL Loss: 0.000000 | F1 Score: 0.960366 == | Current accuracy: 0.964286 ==
[[35 0 0 0 0 0]
[ 1 68 0 1 3 0]
[ 0 0 31 0 1 0]
[ 0 1 0 30 0 0]
[ 0 0 0 0 13 0]
[ 0 0 0 0 0 12]]
Patience: 2 / 10
== Epoch: 13 | Classification Loss: 0.516704 | Representation DL Loss: 0.000000 | Accuracy: 96.8112% ==
== Epoch: 13 | Validation CLS Loss: 3.340173 | Validation Representation DL Loss: 0.000000 | F1 Score: 0.950108 == | Current accuracy: 0.954082 ==
[[38 0 0 0 0 0]
[ 1 61 0 0 3 0]
[ 0 4 28 0 1 0]
[ 0 0 0 35 0 0]
[ 0 0 0 0 12 0]
[ 0 0 0 0 0 13]]
Patience: 3 / 10
== Epoch: 14 | Classification Loss: 0.507660 | Representation DL Loss: 0.000000 | Accuracy: 97.8316% ==
== Epoch: 14 | Validation CLS Loss: 3.123664 | Validation Representation DL Loss: 0.000000 | F1 Score: 0.938434 == | Current accuracy: 0.943878 ==
[[29 0 0 0 0 0]
[ 1 67 0 0 4 0]
[ 0 0 27 0 1 0]
[ 0 3 0 33 0 0]
[ 0 0 2 0 14 0]
[ 0 0 0 0 0 15]]
Patience: 4 / 10
== Epoch: 15 | Classification Loss: 0.530127 | Representation DL Loss: 0.000000 | Accuracy: 96.8112% ==
== Epoch: 15 | Validation CLS Loss: 2.863460 | Validation Representation DL Loss: 0.000000 | F1 Score: 0.947036 == | Current accuracy: 0.948980 ==
[[22 0 0 0 0 0]
[ 1 77 0 0 3 0]
[ 0 2 36 0 2 0]
[ 0 2 0 30 0 0]
[ 0 0 0 0 13 0]
[ 0 0 0 0 0 8]]
Patience: 5 / 10
== Epoch: 16 | Classification Loss: 0.506789 | Representation DL Loss: 0.000000 | Accuracy: 97.7041% ==
== Epoch: 16 | Validation CLS Loss: 2.933983 | Validation Representation DL Loss: 0.000000 | F1 Score: 0.948378 == | Current accuracy: 0.948980 ==
[[34 0 0 0 0 0]
[ 1 66 0 0 5 0]
[ 0 2 19 0 1 0]
[ 0 0 0 35 0 0]
[ 0 0 1 0 21 0]
[ 0 0 0 0 0 11]]
Patience: 6 / 10
== Epoch: 17 | Classification Loss: 0.531937 | Representation DL Loss: 0.000000 | Accuracy: 96.0459% ==
== Epoch: 17 | Validation CLS Loss: 2.991309 | Validation Representation DL Loss: 0.000000 | F1 Score: 0.959388 == | Current accuracy: 0.964286 ==
[[29 0 0 0 0 0]
[ 0 58 0 2 3 0]
[ 0 0 36 0 0 0]
[ 0 0 0 41 0 0]
[ 0 0 2 0 14 0]
[ 0 0 0 0 0 11]]
Patience: 7 / 10
== Epoch: 18 | Classification Loss: 0.547258 | Representation DL Loss: 0.000000 | Accuracy: 95.7908% ==
== Epoch: 18 | Validation CLS Loss: 3.316063 | Validation Representation DL Loss: 0.000000 | F1 Score: 0.931193 == | Current accuracy: 0.943878 ==
[[34 0 0 0 0 0]
[ 0 68 0 0 0 0]
[ 0 6 34 0 0 0]
[ 0 1 0 27 0 0]
[ 0 1 3 0 7 0]
[ 0 0 0 0 0 15]]
Patience: 8 / 10
== Epoch: 19 | Classification Loss: 0.520411 | Representation DL Loss: 0.000000 | Accuracy: 97.1939% ==
== Epoch: 19 | Validation CLS Loss: 2.986099 | Validation Representation DL Loss: 0.000000 | F1 Score: 0.962358 == | Current accuracy: 0.969388 ==
[[36 0 0 0 0 0]
[ 2 65 0 0 1 0]
[ 0 0 28 0 1 0]
[ 0 1 0 34 0 0]
[ 0 0 1 0 11 0]
[ 0 0 0 0 0 16]]
Patience: 9 / 10
== Epoch: 20 | Classification Loss: 0.502342 | Representation DL Loss: 0.000000 | Accuracy: 98.0867% ==
== Epoch: 20 | Validation CLS Loss: 3.194363 | Validation Representation DL Loss: 0.000000 | F1 Score: 0.974049 == | Current accuracy: 0.979592 ==
[[38 0 0 0 0 0]
[ 0 54 0 0 2 0]
[ 0 0 30 0 2 0]
[ 0 0 0 45 0 0]
[ 0 0 0 0 17 0]
[ 0 0 0 0 0 8]]
Patience: 10 / 10
== Epoch: 21 | Classification Loss: 0.512222 | Representation DL Loss: 0.000000 | Accuracy: 97.3214% ==
== Epoch: 21 | Validation CLS Loss: 3.108909 | Validation Representation DL Loss: 0.000000 | F1 Score: 0.903065 == | Current accuracy: 0.908163 ==
[[23 0 0 0 0 0]
[ 0 68 2 3 1 0]
[ 0 0 29 0 0 0]
[ 0 4 0 34 0 0]
[ 0 0 8 0 12 0]
[ 0 0 0 0 0 12]]
Patience: 11 / 10
/data/wanh/CANAL/model.py:106: ImplicitModificationWarning: Trying to modify attribute `.obs` of view, initializing view as actual.
adata_i.obs['celltype'] = current_label_dict[i]
/data/wanh/ENTER/envs/pytorch/lib/python3.7/site-packages/anndata/_core/anndata.py:1785: FutureWarning: X.dtype being converted to np.float32 from float64. In the next version of anndata (0.9) conversion will not be automatic. Pass dtype explicitly to avoid this warning. Pass `AnnData(X, dtype=X.dtype, ...)` to get the future behavour.
[AnnData(sparse.csr_matrix(a.shape), obs=a.obs) for a in all_adatas],
example bank after updating:
AnnData object with n_obs × n_vars = 786 × 1000
obs: 'donor', 'cell_type1', 'library', 'organism', 'dataset_name', 'platform', 'organ', 'data_type', 'cell_ontology_class', 'cell_ontology_id', 'n_genes', 'n_counts', '__libsize__', 'baron_human_donor', 'age', 'gender', 'enge_donor', 'batch', 'muraro_donor', 'celltype', 'rank', 'stage'
var: 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
cell type composition of this example bank:
(array(['acinar', 'beta', 'delta', 'ductal', 'gamma', 'mesenchymal'],
dtype=object), array([166, 166, 153, 166, 76, 59]))
dataset composition from each stage of this example bank:
(array([1]), array([786]))
[6]:
dataset_stage_2 = "Enge"
data_path_stage_2 = './data/Pancreas/{}_train.h5ad'.format(dataset_stage_2)
adata_stage_2 = sc.read_h5ad(data_path_stage_2)
adata_stage_2 = adata_stage_2[np.array(adata_stage_2.obs['cell_type1']) != 'alpha']
cell_type_stage_2 = adata_stage_2.obs['cell_type1']
print(adata_stage_2)
print(np.unique(np.array(cell_type_stage_2), return_counts=True))
View of AnnData object with n_obs × n_vars = 998 × 1000
obs: 'donor', 'cell_type1', 'library', 'organism', 'dataset_name', 'platform', 'organ', 'data_type', 'cell_ontology_class', 'cell_ontology_id', 'n_genes', 'n_counts', '__libsize__', 'baron_human_donor', 'age', 'gender', 'enge_donor', 'batch', 'muraro_donor', 'celltype'
var: 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
uns: 'hvg', 'log1p'
(array(['acinar', 'beta', 'delta', 'ductal', 'mesenchymal'], dtype=object), array([332, 267, 55, 300, 44]))
[7]:
CANAL.train(experiments = experiments, pre_dataset = dataset_stage_1, dataset = dataset_stage_2,
adata = adata_stage_2, cell_type = cell_type_stage_2, current_stage = 2,
is_final_stage = False, ckpt_dir='./ckpts/', rehearsal_size=1000, SEED = seed)
current data: View of AnnData object with n_obs × n_vars = 998 × 1000
obs: 'donor', 'cell_type1', 'library', 'organism', 'dataset_name', 'platform', 'organ', 'data_type', 'cell_ontology_class', 'cell_ontology_id', 'n_genes', 'n_counts', '__libsize__', 'baron_human_donor', 'age', 'gender', 'enge_donor', 'batch', 'muraro_donor', 'celltype'
var: 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
uns: 'hvg', 'log1p'
(array(['acinar', 'beta', 'delta', 'ductal', 'mesenchymal'], dtype=object), array([332, 267, 55, 300, 44]))
model constructing begin!
new cell types: []
example bank for experience replay: AnnData object with n_obs × n_vars = 786 × 1000
obs: 'donor', 'cell_type1', 'library', 'organism', 'dataset_name', 'platform', 'organ', 'data_type', 'cell_ontology_class', 'cell_ontology_id', 'n_genes', 'n_counts', '__libsize__', 'baron_human_donor', 'age', 'gender', 'enge_donor', 'batch', 'muraro_donor', 'celltype', 'rank', 'stage'
var: 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
model constructing finished!
label train: [0 1 2 3 4 5] 6
label val: [0 1 2 3 4 5] 6
/data/wanh/ENTER/envs/pytorch/lib/python3.7/site-packages/torch/nn/_reduction.py:42: UserWarning: size_average and reduce args will be deprecated, please use reduction='mean' instead.
warnings.warn(warning.format(ret))
== Begin finetuning: | Incrmental stage | Current stage: 2 | CANAL | Dataset: Pancreas_novel_cells Enge ==
== Epoch: 1 | Classification Loss: 0.655114 | Representation DL Loss: 0.034605 | Accuracy: 91.9378% ==
== Epoch: 1 | Validation CLS Loss: 3.109473 | Validation Representation DL Loss: 0.080488 | F1 Score: 0.933107 == | Current accuracy: 0.957143 ==
[[105 0 0 0 0 0]
[ 0 70 3 0 5 0]
[ 0 1 30 0 0 0]
[ 0 0 0 91 4 0]
[ 0 0 0 0 18 0]
[ 0 1 0 1 0 21]]
Patience: 0 / 10
== Epoch: 2 | Classification Loss: 0.584491 | Representation DL Loss: 0.159628 | Accuracy: 94.8373% ==
== Epoch: 2 | Validation CLS Loss: 3.223272 | Validation Representation DL Loss: 0.177585 | F1 Score: 0.943006 == | Current accuracy: 0.960000 ==
[[87 0 0 0 0 0]
[ 0 82 0 0 0 0]
[ 0 4 37 0 0 0]
[ 0 3 1 95 0 0]
[ 0 0 3 0 13 0]
[ 0 1 0 2 0 22]]
Patience: 0 / 10
== Epoch: 3 | Classification Loss: 0.572462 | Representation DL Loss: 0.152405 | Accuracy: 95.6860% ==
== Epoch: 3 | Validation CLS Loss: 3.271531 | Validation Representation DL Loss: 0.143608 | F1 Score: 0.959979 == | Current accuracy: 0.968571 ==
[[ 85 0 0 0 0 0]
[ 2 69 0 0 1 0]
[ 0 4 32 0 0 0]
[ 0 2 0 112 2 0]
[ 0 0 0 0 15 0]
[ 0 0 0 0 0 26]]
Patience: 0 / 10
== Epoch: 4 | Classification Loss: 0.562595 | Representation DL Loss: 0.176586 | Accuracy: 95.4031% ==
== Epoch: 4 | Validation CLS Loss: 2.898163 | Validation Representation DL Loss: 0.240385 | F1 Score: 0.932569 == | Current accuracy: 0.962857 ==
[[ 74 0 0 0 0 0]
[ 0 92 0 0 0 0]
[ 0 6 38 0 0 0]
[ 0 0 0 100 0 0]
[ 0 1 4 0 10 0]
[ 0 1 0 1 0 23]]
Patience: 1 / 10
== Epoch: 5 | Classification Loss: 0.541776 | Representation DL Loss: 0.174463 | Accuracy: 96.8883% ==
== Epoch: 5 | Validation CLS Loss: 3.493633 | Validation Representation DL Loss: 0.243126 | F1 Score: 0.961770 == | Current accuracy: 0.985714 ==
[[117 0 0 0 0 0]
[ 2 74 0 0 0 0]
[ 0 0 28 0 0 0]
[ 0 0 0 108 0 0]
[ 0 0 2 0 7 0]
[ 0 0 0 1 0 11]]
Patience: 0 / 10
== Epoch: 6 | Classification Loss: 0.515143 | Representation DL Loss: 0.172311 | Accuracy: 97.6662% ==
== Epoch: 6 | Validation CLS Loss: 2.985641 | Validation Representation DL Loss: 0.164048 | F1 Score: 0.975710 == | Current accuracy: 0.985714 ==
[[94 0 0 0 0 0]
[ 0 91 0 0 0 0]
[ 0 1 43 0 0 0]
[ 0 1 0 88 0 0]
[ 0 0 1 0 13 0]
[ 0 2 0 0 0 16]]
Patience: 1 / 10
== Epoch: 7 | Classification Loss: 0.504798 | Representation DL Loss: 0.174250 | Accuracy: 98.2320% ==
== Epoch: 7 | Validation CLS Loss: 3.280770 | Validation Representation DL Loss: 0.154940 | F1 Score: 0.963710 == | Current accuracy: 0.974286 ==
[[94 0 0 0 0 0]
[ 0 99 0 0 3 0]
[ 0 3 36 0 0 0]
[ 1 0 0 74 0 0]
[ 0 0 0 0 19 0]
[ 0 0 0 2 0 19]]
Patience: 2 / 10
== Epoch: 8 | Classification Loss: 0.500768 | Representation DL Loss: 0.178268 | Accuracy: 98.0198% ==
== Epoch: 8 | Validation CLS Loss: 3.156156 | Validation Representation DL Loss: 0.192507 | F1 Score: 0.935961 == | Current accuracy: 0.965714 ==
[[90 0 0 0 0 0]
[ 0 77 0 0 1 0]
[ 0 4 52 0 0 0]
[ 0 0 0 88 0 0]
[ 0 0 5 0 11 0]
[ 0 0 0 2 0 20]]
Patience: 3 / 10
== Epoch: 9 | Classification Loss: 0.494693 | Representation DL Loss: 0.180622 | Accuracy: 98.3734% ==
== Epoch: 9 | Validation CLS Loss: 3.153236 | Validation Representation DL Loss: 0.117369 | F1 Score: 0.975951 == | Current accuracy: 0.985714 ==
[[ 88 0 0 0 0 0]
[ 0 88 0 0 1 0]
[ 0 2 39 0 0 0]
[ 0 0 0 100 0 0]
[ 0 0 0 0 12 0]
[ 0 0 0 2 0 18]]
Patience: 4 / 10
== Epoch: 10 | Classification Loss: 0.480812 | Representation DL Loss: 0.154725 | Accuracy: 98.9392% ==
== Epoch: 10 | Validation CLS Loss: 3.191809 | Validation Representation DL Loss: 0.125563 | F1 Score: 0.973008 == | Current accuracy: 0.985714 ==
[[ 94 0 0 0 0 0]
[ 0 95 0 0 1 0]
[ 0 2 29 0 0 0]
[ 0 0 0 100 0 0]
[ 0 0 0 0 10 0]
[ 0 0 0 2 0 17]]
Patience: 5 / 10
== Epoch: 11 | Classification Loss: 0.482663 | Representation DL Loss: 0.146409 | Accuracy: 98.7270% ==
== Epoch: 11 | Validation CLS Loss: 3.042729 | Validation Representation DL Loss: 0.126838 | F1 Score: 0.984880 == | Current accuracy: 0.994286 ==
[[ 95 0 0 0 0 0]
[ 0 77 0 0 1 0]
[ 0 0 41 0 0 0]
[ 0 0 0 108 0 0]
[ 0 0 0 0 9 0]
[ 0 0 0 1 0 18]]
Patience: 0 / 10
== Epoch: 12 | Classification Loss: 0.479892 | Representation DL Loss: 0.133567 | Accuracy: 98.5149% ==
== Epoch: 12 | Validation CLS Loss: 3.371669 | Validation Representation DL Loss: 0.108700 | F1 Score: 0.962714 == | Current accuracy: 0.980000 ==
[[99 0 0 0 0 0]
[ 0 83 0 0 2 0]
[ 0 3 49 0 0 0]
[ 0 0 0 87 0 0]
[ 0 0 0 0 10 0]
[ 0 0 0 2 0 15]]
Patience: 1 / 10
== Epoch: 13 | Classification Loss: 0.475289 | Representation DL Loss: 0.137088 | Accuracy: 98.7977% ==
== Epoch: 13 | Validation CLS Loss: 3.425863 | Validation Representation DL Loss: 0.121628 | F1 Score: 0.985746 == | Current accuracy: 0.988571 ==
[[95 0 0 0 0 0]
[ 0 87 0 0 1 0]
[ 0 3 42 0 0 0]
[ 0 0 0 83 0 0]
[ 0 0 0 0 17 0]
[ 0 0 0 0 0 22]]
Patience: 2 / 10
== Epoch: 14 | Classification Loss: 0.466741 | Representation DL Loss: 0.131327 | Accuracy: 99.5050% ==
== Epoch: 14 | Validation CLS Loss: 3.089997 | Validation Representation DL Loss: 0.129328 | F1 Score: 0.980976 == | Current accuracy: 0.988571 ==
[[110 0 0 0 0 0]
[ 0 81 0 0 0 0]
[ 0 1 25 0 0 0]
[ 0 0 0 97 0 0]
[ 0 0 0 0 14 0]
[ 0 0 0 3 0 19]]
Patience: 3 / 10
== Epoch: 15 | Classification Loss: 0.471988 | Representation DL Loss: 0.121874 | Accuracy: 99.0806% ==
== Epoch: 15 | Validation CLS Loss: 3.299883 | Validation Representation DL Loss: 0.117825 | F1 Score: 0.993588 == | Current accuracy: 0.997143 ==
[[85 0 0 0 0 0]
[ 0 80 0 0 1 0]
[ 0 0 50 0 0 0]
[ 0 0 0 98 0 0]
[ 0 0 0 0 15 0]
[ 0 0 0 0 0 21]]
Patience: 0 / 10
== Epoch: 16 | Classification Loss: 0.466107 | Representation DL Loss: 0.125688 | Accuracy: 99.4342% ==
== Epoch: 16 | Validation CLS Loss: 3.204759 | Validation Representation DL Loss: 0.122140 | F1 Score: 0.994993 == | Current accuracy: 0.997143 ==
[[96 0 0 0 0 0]
[ 0 86 0 0 0 0]
[ 0 0 41 0 0 0]
[ 0 0 0 88 0 0]
[ 0 0 0 0 18 0]
[ 0 0 0 1 0 20]]
Patience: 1 / 10
== Epoch: 17 | Classification Loss: 0.467420 | Representation DL Loss: 0.134502 | Accuracy: 99.6464% ==
== Epoch: 17 | Validation CLS Loss: 3.261277 | Validation Representation DL Loss: 0.118494 | F1 Score: 0.994948 == | Current accuracy: 0.994286 ==
[[ 95 0 0 0 0 0]
[ 0 100 0 0 0 0]
[ 0 2 48 0 0 0]
[ 0 0 0 73 0 0]
[ 0 0 0 0 13 0]
[ 0 0 0 0 0 19]]
Patience: 2 / 10
== Epoch: 18 | Classification Loss: 0.469532 | Representation DL Loss: 0.149825 | Accuracy: 99.5050% ==
== Epoch: 18 | Validation CLS Loss: 3.112838 | Validation Representation DL Loss: 0.128857 | F1 Score: 0.979060 == | Current accuracy: 0.988571 ==
[[101 0 0 0 0 0]
[ 0 76 0 0 4 0]
[ 0 0 42 0 0 0]
[ 0 0 0 90 0 0]
[ 0 0 0 0 18 0]
[ 0 0 0 0 0 19]]
Patience: 3 / 10
== Epoch: 19 | Classification Loss: 0.473967 | Representation DL Loss: 0.151132 | Accuracy: 98.8685% ==
== Epoch: 19 | Validation CLS Loss: 3.299162 | Validation Representation DL Loss: 0.142391 | F1 Score: 0.935499 == | Current accuracy: 0.968571 ==
[[97 0 0 0 0 0]
[ 0 77 5 0 0 0]
[ 0 2 38 0 0 0]
[ 0 0 0 99 0 0]
[ 0 0 3 0 6 0]
[ 0 0 0 1 0 22]]
Patience: 4 / 10
== Epoch: 20 | Classification Loss: 0.477590 | Representation DL Loss: 0.165486 | Accuracy: 98.7270% ==
== Epoch: 20 | Validation CLS Loss: 3.339000 | Validation Representation DL Loss: 0.251615 | F1 Score: 0.954600 == | Current accuracy: 0.965714 ==
[[94 0 0 0 0 0]
[ 0 85 0 0 0 0]
[ 0 10 26 0 0 0]
[ 0 0 0 98 0 0]
[ 0 0 0 0 14 0]
[ 0 0 0 2 0 21]]
Patience: 5 / 10
== Epoch: 21 | Classification Loss: 0.473712 | Representation DL Loss: 0.188792 | Accuracy: 99.1513% ==
== Epoch: 21 | Validation CLS Loss: 2.950457 | Validation Representation DL Loss: 0.243565 | F1 Score: 0.941742 == | Current accuracy: 0.951429 ==
[[98 0 0 0 0 0]
[ 0 71 13 0 0 0]
[ 0 2 33 0 0 0]
[ 0 0 0 93 0 0]
[ 0 0 1 0 15 0]
[ 0 0 0 1 0 23]]
Patience: 6 / 10
== Epoch: 22 | Classification Loss: 0.475351 | Representation DL Loss: 0.193461 | Accuracy: 99.2221% ==
== Epoch: 22 | Validation CLS Loss: 3.472855 | Validation Representation DL Loss: 0.143873 | F1 Score: 0.976173 == | Current accuracy: 0.985714 ==
[[100 0 0 0 0 0]
[ 0 73 0 0 1 0]
[ 0 2 43 0 0 0]
[ 0 0 0 98 0 0]
[ 0 0 0 0 12 0]
[ 0 0 0 2 0 19]]
Patience: 7 / 10
== Epoch: 23 | Classification Loss: 0.458011 | Representation DL Loss: 0.165682 | Accuracy: 99.5757% ==
== Epoch: 23 | Validation CLS Loss: 3.254958 | Validation Representation DL Loss: 0.162261 | F1 Score: 0.987201 == | Current accuracy: 0.985714 ==
[[ 71 0 0 0 0 0]
[ 0 86 0 0 0 0]
[ 0 5 49 0 0 0]
[ 0 0 0 107 0 0]
[ 0 0 0 0 10 0]
[ 0 0 0 0 0 22]]
Patience: 8 / 10
== Epoch: 24 | Classification Loss: 0.460796 | Representation DL Loss: 0.164778 | Accuracy: 99.2928% ==
== Epoch: 24 | Validation CLS Loss: 3.131350 | Validation Representation DL Loss: 0.215336 | F1 Score: 0.982760 == | Current accuracy: 0.988571 ==
[[101 0 0 0 0 0]
[ 0 92 0 0 0 0]
[ 0 3 32 0 0 0]
[ 0 0 0 90 0 0]
[ 0 0 1 0 17 0]
[ 0 0 0 0 0 14]]
Patience: 9 / 10
== Epoch: 25 | Classification Loss: 0.473259 | Representation DL Loss: 0.183562 | Accuracy: 99.2928% ==
== Epoch: 25 | Validation CLS Loss: 3.241163 | Validation Representation DL Loss: 0.176210 | F1 Score: 0.961656 == | Current accuracy: 0.965714 ==
[[92 0 0 0 0 0]
[ 0 89 0 0 0 0]
[ 0 10 34 0 0 0]
[ 0 0 0 85 0 0]
[ 0 0 0 0 12 0]
[ 0 0 0 2 0 26]]
Patience: 10 / 10
== Epoch: 26 | Classification Loss: 0.458024 | Representation DL Loss: 0.155457 | Accuracy: 99.7171% ==
== Epoch: 26 | Validation CLS Loss: 3.264828 | Validation Representation DL Loss: 0.135022 | F1 Score: 0.996631 == | Current accuracy: 0.997143 ==
[[91 0 0 0 0 0]
[ 0 94 0 0 0 0]
[ 0 1 33 0 0 0]
[ 0 0 0 87 0 0]
[ 0 0 0 0 18 0]
[ 0 0 0 0 0 26]]
Patience: 11 / 10
/data/wanh/CANAL/model.py:106: ImplicitModificationWarning: Trying to modify attribute `.obs` of view, initializing view as actual.
adata_i.obs['celltype'] = current_label_dict[i]
/data/wanh/ENTER/envs/pytorch/lib/python3.7/site-packages/anndata/_core/anndata.py:1785: FutureWarning: X.dtype being converted to np.float32 from float64. In the next version of anndata (0.9) conversion will not be automatic. Pass dtype explicitly to avoid this warning. Pass `AnnData(X, dtype=X.dtype, ...)` to get the future behavour.
[AnnData(sparse.csr_matrix(a.shape), obs=a.obs) for a in all_adatas],
example bank after updating:
AnnData object with n_obs × n_vars = 843 × 1000
obs: 'donor', 'cell_type1', 'library', 'organism', 'dataset_name', 'platform', 'organ', 'data_type', 'cell_ontology_class', 'cell_ontology_id', 'n_genes', 'n_counts', '__libsize__', 'baron_human_donor', 'age', 'gender', 'enge_donor', 'batch', 'muraro_donor', 'celltype', 'rank', 'stage'
var: 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
cell type composition of this example bank:
(array(['acinar', 'beta', 'delta', 'ductal', 'gamma', 'mesenchymal'],
dtype=object), array([166, 166, 166, 166, 76, 103]))
dataset composition from each stage of this example bank:
(array([1, 2]), array([495, 348]))
[8]:
dataset_stage_3 = "Baron_human"
data_path_stage_3 = './data/Pancreas/{}_train.h5ad'.format(dataset_stage_3)
adata_stage_3 = sc.read_h5ad(data_path_stage_3)
adata_stage_3 = adata_stage_3[np.array(adata_stage_3.obs['cell_type1']) != 'alpha']
cell_type_stage_3 = adata_stage_3.obs['cell_type1']
print(adata_stage_3)
print(np.unique(np.array(cell_type_stage_3), return_counts=True))
View of AnnData object with n_obs × n_vars = 5775 × 1000
obs: 'donor', 'cell_type1', 'library', 'organism', 'dataset_name', 'platform', 'organ', 'data_type', 'cell_ontology_class', 'cell_ontology_id', 'n_genes', 'n_counts', '__libsize__', 'baron_human_donor', 'age', 'gender', 'enge_donor', 'batch', 'muraro_donor', 'celltype'
var: 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
uns: 'hvg', 'log1p'
(array(['acinar', 'activated_stellate', 'beta', 'delta', 'ductal',
'endothelial', 'gamma', 'quiescent_stellate'], dtype=object), array([ 908, 268, 2367, 570, 1016, 241, 243, 162]))
[9]:
CANAL.train(experiments = experiments, pre_dataset = dataset_stage_2, dataset = dataset_stage_3,
adata = adata_stage_3, cell_type = cell_type_stage_3, current_stage = 3, is_final_stage = True,
ckpt_dir='./ckpts/', rehearsal_size=1000, SEED = seed)
current data: View of AnnData object with n_obs × n_vars = 5775 × 1000
obs: 'donor', 'cell_type1', 'library', 'organism', 'dataset_name', 'platform', 'organ', 'data_type', 'cell_ontology_class', 'cell_ontology_id', 'n_genes', 'n_counts', '__libsize__', 'baron_human_donor', 'age', 'gender', 'enge_donor', 'batch', 'muraro_donor', 'celltype'
var: 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
uns: 'hvg', 'log1p'
(array(['acinar', 'activated_stellate', 'beta', 'delta', 'ductal',
'endothelial', 'gamma', 'quiescent_stellate'], dtype=object), array([ 908, 268, 2367, 570, 1016, 241, 243, 162]))
model constructing begin!
new cell types: ['activated_stellate', 'endothelial', 'quiescent_stellate']
example bank for experience replay: AnnData object with n_obs × n_vars = 843 × 1000
obs: 'donor', 'cell_type1', 'library', 'organism', 'dataset_name', 'platform', 'organ', 'data_type', 'cell_ontology_class', 'cell_ontology_id', 'n_genes', 'n_counts', '__libsize__', 'baron_human_donor', 'age', 'gender', 'enge_donor', 'batch', 'muraro_donor', 'celltype', 'rank', 'stage'
var: 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
model constructing finished!
label train: [0 1 2 3 4 5 6 7 8] 9
label val: [0 1 2 3 4 5 6 7 8] 9
== Begin finetuning: | Final stage | Current stage: 3 | CANAL | Dataset: Pancreas_novel_cells Baron_human ==
/data/wanh/ENTER/envs/pytorch/lib/python3.7/site-packages/torch/nn/_reduction.py:42: UserWarning: size_average and reduce args will be deprecated, please use reduction='mean' instead.
warnings.warn(warning.format(ret))
== Epoch: 1 | Classification Loss: 1.044161 | Representation DL Loss: 0.089578 | Accuracy: 85.2797% ==
== Epoch: 1 | Validation CLS Loss: 2.562565 | Validation Representation DL Loss: 0.173877 | F1 Score: 0.530131 == | Current accuracy: 0.835106 ==
[[215 2 0 0 0 0 0 0 0]
[ 3 466 9 2 0 0 0 0 0]
[ 0 16 150 0 0 0 0 0 0]
[ 33 2 4 201 0 0 0 0 0]
[ 2 13 5 0 45 0 0 0 0]
[ 0 0 0 0 0 22 0 0 0]
[ 0 3 3 0 0 42 0 0 0]
[ 2 11 1 17 1 11 0 0 0]
[ 0 9 0 0 0 26 0 0 0]]
Patience: 0 / 10
== Epoch: 2 | Classification Loss: 0.832789 | Representation DL Loss: 0.456191 | Accuracy: 86.7536% ==
== Epoch: 2 | Validation CLS Loss: 3.191035 | Validation Representation DL Loss: 0.367353 | F1 Score: 0.800855 == | Current accuracy: 0.925532 ==
[[220 0 0 0 0 0 0 0 0]
[ 6 484 11 0 2 0 0 0 0]
[ 0 5 146 0 0 0 0 0 0]
[ 24 0 2 217 0 0 0 0 0]
[ 2 1 14 0 49 0 0 0 0]
[ 0 0 0 0 0 20 0 0 0]
[ 0 0 2 0 0 0 41 0 0]
[ 0 0 0 0 0 5 1 40 0]
[ 0 0 0 0 0 2 17 4 1]]
Patience: 0 / 10
== Epoch: 3 | Classification Loss: 0.697150 | Representation DL Loss: 0.426910 | Accuracy: 92.9894% ==
== Epoch: 3 | Validation CLS Loss: 3.301777 | Validation Representation DL Loss: 0.420670 | F1 Score: 0.864578 == | Current accuracy: 0.934650 ==
[[201 2 0 2 0 0 0 0 0]
[ 9 489 13 0 3 0 0 0 0]
[ 0 1 151 0 0 0 0 0 0]
[ 16 0 4 194 0 0 0 0 0]
[ 0 5 7 0 63 0 0 0 0]
[ 0 0 0 0 0 27 1 0 0]
[ 0 0 4 0 0 2 52 0 0]
[ 0 0 0 0 0 0 4 48 0]
[ 0 1 1 0 0 4 7 0 5]]
Patience: 0 / 10
== Epoch: 4 | Classification Loss: 0.635555 | Representation DL Loss: 0.454460 | Accuracy: 95.9373% ==
== Epoch: 4 | Validation CLS Loss: 3.271477 | Validation Representation DL Loss: 0.426472 | F1 Score: 0.932469 == | Current accuracy: 0.957447 ==
[[194 0 0 1 0 0 0 0 3]
[ 3 490 2 0 1 0 0 0 0]
[ 0 6 134 0 0 0 0 0 0]
[ 12 0 3 247 0 0 0 0 0]
[ 0 6 8 0 39 0 0 0 0]
[ 0 0 0 0 0 26 0 0 0]
[ 0 0 4 0 0 0 61 0 1]
[ 0 0 0 0 0 0 0 47 3]
[ 0 1 0 0 0 0 2 0 22]]
Patience: 0 / 10
== Epoch: 5 | Classification Loss: 0.605724 | Representation DL Loss: 0.433154 | Accuracy: 96.9388% ==
== Epoch: 5 | Validation CLS Loss: 3.499539 | Validation Representation DL Loss: 0.378533 | F1 Score: 0.945285 == | Current accuracy: 0.961246 ==
[[219 0 0 3 0 0 0 0 4]
[ 4 478 2 0 0 0 0 0 0]
[ 0 10 132 0 0 0 0 0 0]
[ 10 0 2 242 0 0 0 0 0]
[ 0 4 2 0 47 0 0 0 0]
[ 0 0 0 0 0 16 0 0 0]
[ 0 0 1 0 0 3 59 0 0]
[ 0 3 0 0 0 0 0 43 1]
[ 0 2 0 0 0 0 0 0 29]]
Patience: 0 / 10
== Epoch: 6 | Classification Loss: 0.594595 | Representation DL Loss: 0.473832 | Accuracy: 97.0711% ==
== Epoch: 6 | Validation CLS Loss: 3.803413 | Validation Representation DL Loss: 0.415562 | F1 Score: 0.940085 == | Current accuracy: 0.949848 ==
[[200 0 0 1 0 0 0 0 0]
[ 3 447 22 0 3 0 0 0 0]
[ 0 1 150 0 0 0 0 0 0]
[ 11 2 0 257 0 0 0 0 0]
[ 1 2 10 0 45 0 0 0 0]
[ 0 0 0 0 0 22 0 0 0]
[ 0 0 1 0 0 0 39 0 0]
[ 0 2 0 0 0 0 0 57 1]
[ 0 2 0 0 0 0 4 0 33]]
Patience: 1 / 10
== Epoch: 7 | Classification Loss: 0.575369 | Representation DL Loss: 0.404400 | Accuracy: 97.6190% ==
== Epoch: 7 | Validation CLS Loss: 3.288597 | Validation Representation DL Loss: 0.350269 | F1 Score: 0.936988 == | Current accuracy: 0.962766 ==
[[204 1 0 4 0 0 0 0 4]
[ 0 506 4 0 0 0 0 0 0]
[ 0 1 141 0 0 0 0 0 0]
[ 4 0 3 220 0 0 0 0 0]
[ 1 4 12 0 51 0 0 0 0]
[ 0 0 0 0 0 17 0 0 0]
[ 0 0 4 0 0 3 49 0 2]
[ 0 0 0 0 0 0 0 37 2]
[ 0 0 0 0 0 0 0 0 42]]
Patience: 0 / 10
== Epoch: 8 | Classification Loss: 0.559441 | Representation DL Loss: 0.363436 | Accuracy: 98.3371% ==
== Epoch: 8 | Validation CLS Loss: 3.625715 | Validation Representation DL Loss: 0.354042 | F1 Score: 0.929802 == | Current accuracy: 0.950608 ==
[[175 1 0 0 0 0 0 0 1]
[ 4 526 4 0 2 0 0 0 0]
[ 0 15 150 0 0 0 0 0 0]
[ 6 1 0 201 0 0 0 0 0]
[ 3 10 3 0 50 0 0 0 0]
[ 0 0 0 0 0 23 0 0 0]
[ 0 0 2 0 0 0 51 0 0]
[ 0 1 0 0 0 0 2 46 4]
[ 0 1 0 0 0 0 5 0 29]]
Patience: 1 / 10
== Epoch: 9 | Classification Loss: 0.556730 | Representation DL Loss: 0.358625 | Accuracy: 98.0537% ==
== Epoch: 9 | Validation CLS Loss: 3.439842 | Validation Representation DL Loss: 0.322663 | F1 Score: 0.970387 == | Current accuracy: 0.972644 ==
[[195 3 0 0 0 0 0 0 3]
[ 4 502 5 0 1 0 0 0 0]
[ 0 7 144 0 0 0 0 0 0]
[ 3 1 3 233 0 0 0 0 0]
[ 0 0 3 0 43 0 0 0 0]
[ 0 0 0 0 0 33 0 0 0]
[ 0 0 0 1 0 0 50 0 0]
[ 0 0 0 0 0 0 0 48 1]
[ 0 1 0 0 0 0 0 0 32]]
Patience: 0 / 10
== Epoch: 10 | Classification Loss: 0.543589 | Representation DL Loss: 0.330766 | Accuracy: 98.7528% ==
== Epoch: 10 | Validation CLS Loss: 3.388976 | Validation Representation DL Loss: 0.290416 | F1 Score: 0.957093 == | Current accuracy: 0.958207 ==
[[197 0 0 2 0 0 0 0 1]
[ 2 519 3 0 2 0 0 0 0]
[ 0 20 125 0 0 0 0 0 0]
[ 4 1 0 213 0 0 0 0 0]
[ 0 9 6 0 59 0 0 0 0]
[ 0 0 0 0 0 17 0 0 0]
[ 0 0 1 0 0 0 57 0 0]
[ 0 1 0 0 0 0 0 40 0]
[ 0 3 0 0 0 0 0 0 34]]
Patience: 1 / 10
== Epoch: 11 | Classification Loss: 0.537309 | Representation DL Loss: 0.320108 | Accuracy: 98.9229% ==
== Epoch: 11 | Validation CLS Loss: 3.707157 | Validation Representation DL Loss: 0.290019 | F1 Score: 0.959473 == | Current accuracy: 0.972644 ==
[[214 1 0 2 0 0 0 0 2]
[ 3 504 5 0 2 0 0 0 0]
[ 0 7 134 0 0 0 0 0 0]
[ 2 0 0 234 0 0 0 0 0]
[ 0 1 5 0 50 0 0 0 0]
[ 0 0 0 0 0 25 0 0 0]
[ 0 0 1 0 0 0 48 0 0]
[ 0 0 0 0 0 0 1 46 1]
[ 0 0 0 0 0 0 3 0 25]]
Patience: 2 / 10
== Epoch: 12 | Classification Loss: 0.532646 | Representation DL Loss: 0.290534 | Accuracy: 99.1119% ==
== Epoch: 12 | Validation CLS Loss: 3.558192 | Validation Representation DL Loss: 0.309710 | F1 Score: 0.967342 == | Current accuracy: 0.970365 ==
[[209 0 0 0 0 0 0 0 2]
[ 1 502 1 0 0 0 0 0 0]
[ 0 10 132 0 0 0 0 0 0]
[ 11 2 0 230 0 0 0 0 0]
[ 1 3 2 0 51 0 0 0 0]
[ 0 0 0 0 0 21 0 0 0]
[ 0 0 2 0 0 0 51 0 0]
[ 0 2 0 0 0 0 0 45 0]
[ 0 1 0 0 0 0 1 0 36]]
Patience: 3 / 10
== Epoch: 13 | Classification Loss: 0.535574 | Representation DL Loss: 0.289453 | Accuracy: 98.7906% ==
== Epoch: 13 | Validation CLS Loss: 3.470287 | Validation Representation DL Loss: 0.288024 | F1 Score: 0.959791 == | Current accuracy: 0.964286 ==
[[222 3 0 2 0 0 0 0 0]
[ 4 482 2 0 1 0 0 0 0]
[ 0 10 144 0 0 0 0 0 0]
[ 9 1 0 224 0 0 0 0 0]
[ 0 4 3 0 49 0 0 0 0]
[ 0 0 0 0 0 26 0 0 0]
[ 0 0 0 0 0 0 57 0 0]
[ 0 2 0 0 0 0 0 35 1]
[ 0 4 0 0 0 0 1 0 30]]
Patience: 4 / 10
== Epoch: 14 | Classification Loss: 0.527888 | Representation DL Loss: 0.287770 | Accuracy: 99.2252% ==
== Epoch: 14 | Validation CLS Loss: 3.247332 | Validation Representation DL Loss: 0.289546 | F1 Score: 0.975222 == | Current accuracy: 0.974924 ==
[[195 1 0 2 0 0 0 0 1]
[ 3 489 4 0 1 0 0 0 0]
[ 0 11 151 0 0 0 0 0 0]
[ 2 0 0 228 0 0 0 0 0]
[ 1 2 0 0 52 0 0 0 0]
[ 0 0 0 0 0 17 0 0 0]
[ 0 0 1 0 0 0 50 0 0]
[ 0 2 0 0 0 0 0 70 1]
[ 0 1 0 0 0 0 0 0 31]]
Patience: 0 / 10
== Epoch: 15 | Classification Loss: 0.531741 | Representation DL Loss: 0.278232 | Accuracy: 98.8662% ==
== Epoch: 15 | Validation CLS Loss: 3.454890 | Validation Representation DL Loss: 0.279549 | F1 Score: 0.958301 == | Current accuracy: 0.971884 ==
[[217 1 0 3 0 0 0 0 1]
[ 2 493 1 0 0 0 0 0 0]
[ 0 8 126 0 0 0 0 0 0]
[ 1 0 0 240 0 0 0 0 0]
[ 1 6 6 0 44 0 0 0 0]
[ 0 0 0 0 0 21 0 0 0]
[ 0 0 2 0 0 0 62 0 0]
[ 0 2 0 0 0 0 0 50 3]
[ 0 0 0 0 0 0 0 0 26]]
Patience: 1 / 10
== Epoch: 16 | Classification Loss: 0.527688 | Representation DL Loss: 0.286912 | Accuracy: 99.1308% ==
== Epoch: 16 | Validation CLS Loss: 3.519671 | Validation Representation DL Loss: 0.289830 | F1 Score: 0.979030 == | Current accuracy: 0.980243 ==
[[213 1 0 2 0 0 0 0 2]
[ 3 482 3 0 1 0 0 0 0]
[ 0 8 150 0 0 0 0 0 0]
[ 0 0 0 229 0 0 0 0 0]
[ 0 0 1 0 57 0 0 0 0]
[ 0 0 0 0 0 18 0 0 0]
[ 0 0 0 0 0 0 55 0 0]
[ 0 1 0 0 0 0 0 45 1]
[ 0 3 0 0 0 0 0 0 41]]
Patience: 0 / 10
== Epoch: 17 | Classification Loss: 0.531050 | Representation DL Loss: 0.294878 | Accuracy: 98.9985% ==
== Epoch: 17 | Validation CLS Loss: 3.303791 | Validation Representation DL Loss: 0.311983 | F1 Score: 0.968166 == | Current accuracy: 0.977204 ==
[[200 1 0 1 0 0 0 0 0]
[ 7 485 1 0 1 0 0 0 0]
[ 0 2 157 0 0 0 0 0 0]
[ 0 1 0 214 0 0 0 0 0]
[ 2 3 5 0 56 0 0 0 0]
[ 0 0 0 0 0 32 0 0 0]
[ 0 0 1 0 0 0 65 0 0]
[ 0 1 0 0 0 0 0 34 4]
[ 0 0 0 0 0 0 0 0 43]]
Patience: 1 / 10
== Epoch: 18 | Classification Loss: 0.530774 | Representation DL Loss: 0.302300 | Accuracy: 98.9229% ==
== Epoch: 18 | Validation CLS Loss: 3.640421 | Validation Representation DL Loss: 0.277686 | F1 Score: 0.954612 == | Current accuracy: 0.958967 ==
[[215 3 0 3 0 0 0 0 3]
[ 3 501 7 0 2 0 0 0 0]
[ 0 13 135 0 0 0 0 0 0]
[ 0 3 4 220 0 0 0 0 0]
[ 0 5 2 0 43 0 0 0 0]
[ 0 0 0 0 0 21 0 0 0]
[ 0 0 4 0 0 0 51 0 0]
[ 0 1 0 0 0 0 0 50 1]
[ 0 0 0 0 0 0 0 0 26]]
Patience: 2 / 10
== Epoch: 19 | Classification Loss: 0.531925 | Representation DL Loss: 0.308520 | Accuracy: 99.1497% ==
== Epoch: 19 | Validation CLS Loss: 3.591055 | Validation Representation DL Loss: 0.278387 | F1 Score: 0.969608 == | Current accuracy: 0.975684 ==
[[212 0 0 1 0 0 0 0 0]
[ 1 507 3 0 0 0 0 0 0]
[ 0 1 142 0 2 0 0 0 0]
[ 4 3 3 221 0 0 0 0 0]
[ 2 1 6 0 54 0 0 0 0]
[ 0 0 0 0 0 16 0 0 0]
[ 0 0 2 0 0 0 46 0 0]
[ 0 1 0 0 0 0 0 56 1]
[ 0 1 0 0 0 0 0 0 30]]
Patience: 3 / 10
== Epoch: 20 | Classification Loss: 0.529956 | Representation DL Loss: 0.304810 | Accuracy: 99.0552% ==
== Epoch: 20 | Validation CLS Loss: 3.356821 | Validation Representation DL Loss: 0.291033 | F1 Score: 0.955440 == | Current accuracy: 0.965046 ==
[[194 0 0 4 0 0 0 0 2]
[ 2 516 0 0 0 0 0 0 0]
[ 0 15 137 0 0 0 0 0 0]
[ 1 1 0 234 0 0 0 0 0]
[ 1 12 0 0 53 0 0 0 0]
[ 0 0 0 0 0 25 0 0 0]
[ 0 0 2 0 0 0 37 0 0]
[ 0 2 0 0 0 0 0 37 3]
[ 0 1 0 0 0 0 0 0 37]]
Patience: 4 / 10
== Epoch: 21 | Classification Loss: 0.538952 | Representation DL Loss: 0.309730 | Accuracy: 98.8095% ==
== Epoch: 21 | Validation CLS Loss: 3.520850 | Validation Representation DL Loss: 0.271492 | F1 Score: 0.971481 == | Current accuracy: 0.976444 ==
[[204 0 0 1 0 0 0 0 0]
[ 4 493 0 0 0 0 0 0 0]
[ 0 9 138 0 0 0 0 0 0]
[ 2 2 0 239 0 0 0 0 0]
[ 1 5 2 0 65 0 0 0 0]
[ 0 0 0 0 0 16 0 0 0]
[ 0 0 0 0 0 0 55 0 0]
[ 0 1 0 0 0 0 0 48 3]
[ 0 1 0 0 0 0 0 0 27]]
Patience: 5 / 10
== Epoch: 22 | Classification Loss: 0.529556 | Representation DL Loss: 0.299470 | Accuracy: 99.1497% ==
== Epoch: 22 | Validation CLS Loss: 3.384679 | Validation Representation DL Loss: 0.316501 | F1 Score: 0.936075 == | Current accuracy: 0.951368 ==
[[204 0 0 0 0 0 0 0 0]
[ 1 475 0 0 1 0 0 0 0]
[ 0 17 130 0 0 0 0 0 0]
[ 12 3 0 258 0 0 0 0 0]
[ 2 11 3 0 53 0 0 0 0]
[ 0 0 0 0 0 26 0 0 0]
[ 0 0 0 2 0 2 36 0 0]
[ 0 3 0 0 0 0 0 42 2]
[ 0 5 0 0 0 0 0 0 28]]
Patience: 6 / 10
== Epoch: 23 | Classification Loss: 0.529521 | Representation DL Loss: 0.305130 | Accuracy: 99.0930% ==
== Epoch: 23 | Validation CLS Loss: 3.620424 | Validation Representation DL Loss: 0.271665 | F1 Score: 0.959512 == | Current accuracy: 0.968845 ==
[[207 0 0 3 0 0 0 0 1]
[ 0 494 5 0 0 0 0 0 0]
[ 0 12 128 0 0 0 0 0 0]
[ 1 1 0 241 0 0 0 0 0]
[ 2 6 4 0 53 0 0 0 0]
[ 0 0 0 0 0 19 0 0 0]
[ 0 0 1 0 0 0 54 0 0]
[ 0 1 0 0 0 0 0 55 2]
[ 0 2 0 0 0 0 0 0 24]]
Patience: 7 / 10
== Epoch: 24 | Classification Loss: 0.531023 | Representation DL Loss: 0.310310 | Accuracy: 99.1497% ==
== Epoch: 24 | Validation CLS Loss: 3.683486 | Validation Representation DL Loss: 0.284018 | F1 Score: 0.972155 == | Current accuracy: 0.971125 ==
[[219 0 0 0 0 0 0 0 0]
[ 4 483 6 0 0 0 0 0 0]
[ 0 5 142 0 0 0 0 0 0]
[ 9 1 0 227 0 0 0 0 0]
[ 2 4 4 0 54 0 0 0 0]
[ 0 0 0 0 0 23 0 0 0]
[ 0 0 1 0 0 0 56 0 0]
[ 0 1 0 0 0 0 0 39 0]
[ 0 1 0 0 0 0 0 0 35]]
Patience: 8 / 10
== Epoch: 25 | Classification Loss: 0.530517 | Representation DL Loss: 0.282798 | Accuracy: 98.8095% ==
== Epoch: 25 | Validation CLS Loss: 3.482265 | Validation Representation DL Loss: 0.252784 | F1 Score: 0.970291 == | Current accuracy: 0.971884 ==
[[210 2 0 0 0 0 0 0 0]
[ 2 492 6 0 1 0 0 0 0]
[ 0 5 154 0 0 0 0 0 0]
[ 5 2 0 214 0 0 0 0 0]
[ 0 5 1 0 66 0 0 0 0]
[ 0 0 0 0 0 19 0 0 0]
[ 0 0 5 1 0 0 45 0 1]
[ 0 0 0 0 0 0 0 46 0]
[ 0 1 0 0 0 0 0 0 33]]
Patience: 9 / 10
== Epoch: 26 | Classification Loss: 0.526284 | Representation DL Loss: 0.283781 | Accuracy: 99.0930% ==
== Epoch: 26 | Validation CLS Loss: 3.358564 | Validation Representation DL Loss: 0.275235 | F1 Score: 0.961695 == | Current accuracy: 0.972644 ==
[[201 2 0 0 0 0 0 0 1]
[ 0 477 0 0 0 0 0 0 0]
[ 0 9 152 0 0 0 0 0 0]
[ 2 1 0 244 0 0 0 0 0]
[ 1 11 2 0 42 0 0 0 0]
[ 0 0 0 0 0 23 0 0 0]
[ 0 0 4 0 0 0 55 0 0]
[ 0 0 0 0 0 0 0 52 2]
[ 0 1 0 0 0 0 0 0 34]]
Patience: 10 / 10
== Epoch: 27 | Classification Loss: 0.519523 | Representation DL Loss: 0.284533 | Accuracy: 99.3953% ==
== Epoch: 27 | Validation CLS Loss: 3.534639 | Validation Representation DL Loss: 0.289244 | F1 Score: 0.979500 == | Current accuracy: 0.981003 ==
[[202 0 0 2 0 0 0 0 2]
[ 6 490 1 0 0 0 0 0 0]
[ 0 2 162 0 0 0 0 0 0]
[ 5 0 0 232 0 0 0 0 0]
[ 1 0 5 0 54 0 0 0 0]
[ 0 0 0 0 0 18 0 0 0]
[ 0 0 0 0 0 0 47 0 0]
[ 0 0 0 0 0 0 0 44 1]
[ 0 0 0 0 0 0 0 0 42]]
Patience: 0 / 10
== Epoch: 28 | Classification Loss: 0.519246 | Representation DL Loss: 0.275541 | Accuracy: 99.4520% ==
== Epoch: 28 | Validation CLS Loss: 3.718932 | Validation Representation DL Loss: 0.259004 | F1 Score: 0.975520 == | Current accuracy: 0.979483 ==
[[225 2 0 2 0 0 0 0 1]
[ 1 492 5 0 1 0 0 0 0]
[ 0 4 144 0 0 0 0 0 0]
[ 4 0 0 221 0 0 0 0 0]
[ 1 0 2 0 53 0 0 0 0]
[ 0 0 0 0 0 17 0 0 0]
[ 0 0 1 0 0 0 58 0 0]
[ 0 0 0 0 0 0 0 50 2]
[ 0 1 0 0 0 0 0 0 29]]
Patience: 1 / 10
== Epoch: 29 | Classification Loss: 0.518270 | Representation DL Loss: 0.276118 | Accuracy: 99.4898% ==
== Epoch: 29 | Validation CLS Loss: 3.553342 | Validation Representation DL Loss: 0.274948 | F1 Score: 0.938432 == | Current accuracy: 0.958207 ==
[[178 2 0 2 0 0 0 0 1]
[ 5 486 0 0 1 0 0 0 0]
[ 0 10 150 0 0 0 0 0 0]
[ 5 0 0 250 0 0 0 0 0]
[ 1 15 0 0 51 0 0 0 0]
[ 0 0 0 0 0 13 0 0 0]
[ 0 0 0 0 0 3 56 0 0]
[ 0 4 0 0 0 0 0 45 1]
[ 0 5 0 0 0 0 0 0 32]]
Patience: 2 / 10
== Epoch: 30 | Classification Loss: 0.516237 | Representation DL Loss: 0.254123 | Accuracy: 99.5087% ==
== Epoch: 30 | Validation CLS Loss: 3.558269 | Validation Representation DL Loss: 0.260434 | F1 Score: 0.970905 == | Current accuracy: 0.974164 ==
[[231 1 0 1 0 0 0 0 0]
[ 2 486 2 0 1 0 0 0 0]
[ 0 6 123 0 0 0 0 0 0]
[ 3 3 0 240 0 0 0 0 0]
[ 2 10 1 0 50 0 0 0 0]
[ 0 0 0 0 0 24 0 0 0]
[ 0 0 0 0 0 0 63 0 0]
[ 0 1 0 0 0 0 0 41 1]
[ 0 0 0 0 0 0 0 0 24]]
Patience: 3 / 10
== Epoch: 31 | Classification Loss: 0.514864 | Representation DL Loss: 0.256721 | Accuracy: 99.5654% ==
== Epoch: 31 | Validation CLS Loss: 3.670532 | Validation Representation DL Loss: 0.232568 | F1 Score: 0.969269 == | Current accuracy: 0.974164 ==
[[198 0 0 6 0 0 0 0 1]
[ 2 510 1 0 6 0 0 0 0]
[ 0 5 133 0 0 0 0 0 0]
[ 6 0 0 249 0 0 0 0 0]
[ 2 0 1 0 51 0 0 0 0]
[ 0 0 0 0 0 21 0 0 0]
[ 0 0 0 0 0 0 53 0 0]
[ 0 2 0 0 0 0 0 40 1]
[ 0 1 0 0 0 0 0 0 27]]
Patience: 4 / 10
== Epoch: 32 | Classification Loss: 0.514288 | Representation DL Loss: 0.253197 | Accuracy: 99.4709% ==
== Epoch: 32 | Validation CLS Loss: 3.492432 | Validation Representation DL Loss: 0.268612 | F1 Score: 0.979043 == | Current accuracy: 0.977204 ==
[[207 2 0 1 0 0 0 0 0]
[ 4 479 4 0 3 0 0 0 0]
[ 0 4 115 0 0 0 0 0 0]
[ 3 2 0 236 0 0 0 0 0]
[ 3 2 0 0 67 0 0 0 0]
[ 0 0 0 0 0 24 0 0 0]
[ 0 0 1 0 0 0 68 0 0]
[ 0 0 0 0 0 0 0 54 1]
[ 0 0 0 0 0 0 0 0 36]]
Patience: 5 / 10
== Epoch: 33 | Classification Loss: 0.513383 | Representation DL Loss: 0.248351 | Accuracy: 99.5465% ==
== Epoch: 33 | Validation CLS Loss: 3.521808 | Validation Representation DL Loss: 0.255846 | F1 Score: 0.965857 == | Current accuracy: 0.971125 ==
[[204 0 0 8 0 0 0 0 0]
[ 3 495 5 0 2 0 0 0 0]
[ 0 4 144 0 0 0 0 0 0]
[ 0 1 0 242 0 0 0 0 0]
[ 1 5 4 0 52 0 0 0 0]
[ 0 0 0 0 0 22 0 0 0]
[ 0 0 2 0 0 0 44 0 0]
[ 0 1 0 0 0 0 0 45 2]
[ 0 0 0 0 0 0 0 0 30]]
Patience: 6 / 10
== Epoch: 34 | Classification Loss: 0.511061 | Representation DL Loss: 0.245662 | Accuracy: 99.6599% ==
== Epoch: 34 | Validation CLS Loss: 3.580372 | Validation Representation DL Loss: 0.240557 | F1 Score: 0.973009 == | Current accuracy: 0.976444 ==
[[219 2 0 0 0 0 0 0 1]
[ 1 468 2 0 0 0 0 0 0]
[ 0 8 165 0 0 0 0 0 0]
[ 3 1 0 211 0 0 0 0 0]
[ 3 4 2 0 65 0 0 0 0]
[ 0 0 0 0 0 23 0 0 0]
[ 0 0 1 0 0 0 55 0 0]
[ 0 0 0 0 0 0 0 47 2]
[ 0 1 0 0 0 0 0 0 32]]
Patience: 7 / 10
== Epoch: 35 | Classification Loss: 0.512157 | Representation DL Loss: 0.239320 | Accuracy: 99.6410% ==
== Epoch: 35 | Validation CLS Loss: 3.849944 | Validation Representation DL Loss: 0.232048 | F1 Score: 0.973794 == | Current accuracy: 0.981003 ==
[[215 0 0 1 0 0 0 0 1]
[ 1 502 1 0 2 0 0 0 0]
[ 0 4 130 0 0 0 0 0 0]
[ 3 1 0 237 0 0 0 0 0]
[ 1 2 3 0 63 0 0 0 0]
[ 0 0 0 0 0 12 0 0 0]
[ 0 0 1 0 0 0 57 0 0]
[ 0 1 0 0 0 0 0 45 2]
[ 0 1 0 0 0 0 0 0 30]]
Patience: 8 / 10
== Epoch: 36 | Classification Loss: 0.510784 | Representation DL Loss: 0.244380 | Accuracy: 99.6599% ==
== Epoch: 36 | Validation CLS Loss: 3.457236 | Validation Representation DL Loss: 0.228220 | F1 Score: 0.965835 == | Current accuracy: 0.973404 ==
[[205 3 0 0 0 0 0 0 2]
[ 3 486 0 0 0 0 0 0 0]
[ 0 4 163 0 0 0 0 0 0]
[ 9 3 0 234 0 0 0 0 0]
[ 0 2 2 0 48 0 0 0 0]
[ 0 0 0 0 0 24 0 0 0]
[ 0 0 3 0 0 0 50 0 0]
[ 0 0 0 0 0 0 0 43 3]
[ 0 1 0 0 0 0 0 0 28]]
Patience: 9 / 10
== Epoch: 37 | Classification Loss: 0.508966 | Representation DL Loss: 0.235382 | Accuracy: 99.6788% ==
== Epoch: 37 | Validation CLS Loss: 3.499866 | Validation Representation DL Loss: 0.234173 | F1 Score: 0.975344 == | Current accuracy: 0.976444 ==
[[212 0 0 3 0 0 0 0 1]
[ 6 530 1 0 1 0 0 0 0]
[ 0 9 114 0 0 0 0 0 0]
[ 3 0 0 209 0 0 0 0 0]
[ 0 2 0 0 65 0 0 0 0]
[ 0 0 0 0 0 19 0 0 0]
[ 0 0 1 0 0 0 65 0 0]
[ 0 2 0 0 0 0 0 38 1]
[ 0 1 0 0 0 0 0 0 33]]
Patience: 10 / 10
== Epoch: 38 | Classification Loss: 0.509208 | Representation DL Loss: 0.234490 | Accuracy: 99.7543% ==
== Epoch: 38 | Validation CLS Loss: 3.660929 | Validation Representation DL Loss: 0.225370 | F1 Score: 0.978845 == | Current accuracy: 0.978723 ==
[[219 0 0 1 0 0 0 0 0]
[ 4 501 4 0 1 0 0 0 0]
[ 0 6 142 0 0 0 0 0 0]
[ 4 1 0 234 0 0 0 0 0]
[ 0 4 1 0 50 0 0 0 0]
[ 0 0 0 0 0 23 0 0 0]
[ 0 0 0 0 0 0 53 0 0]
[ 0 1 0 0 0 0 0 39 1]
[ 0 0 0 0 0 0 0 0 27]]
Patience: 11 / 10
predict cell types of the test data
predicted unknown cells will be annotated as “Unassigned”
[10]:
dataset_test = sc.read_h5ad("./data/Pancreas/Pancreas_test.h5ad")
print(dataset_test)
pred_cell_type = CANAL.predict(adata_predict = dataset_test, ckpt_dir = './ckpts/', experiments = experiments,
stage_num=3, dataset = dataset_stage_3, novel=True)
AnnData object with n_obs × n_vars = 2000 × 1000
obs: 'donor', 'cell_type1', 'library', 'organism', 'dataset_name', 'platform', 'organ', 'data_type', 'cell_ontology_class', 'cell_ontology_id', 'n_genes', 'n_counts', '__libsize__', 'baron_human_donor', 'age', 'gender', 'enge_donor', 'batch', 'muraro_donor', 'celltype', 'batch2'
var: 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
== Begin predicting after 3 fine-tuning stages: | Experiments: Pancreas_novel_cells ==
Annotation: ['acinar' 'beta' 'delta' 'ductal' 'gamma' 'mesenchymal'
'activated_stellate' 'endothelial' 'quiescent_stellate']
evaluate the performance of cell-type annotation and novel cell detection
[11]:
true_celltype = np.array(dataset_test.obs['cell_type1'])
CANAL.evaluation(pred_cell_type=pred_cell_type, true_celltype=true_celltype, novel_celltype='alpha')
Begin evaluation!
== H-score: 0.824017 ==|== Predict total accuracy: 0.841000 ==
== Predict known accuracy: 0.887336 ==|== Predict unknown accuracy: 0.769133 ==
[ ]: