Tutorial 2: run CANAL with data streams from various batches
[1]:
import os
os.chdir('/data/wanh/CANAL/')
import sys
sys.path.append('/data/wanh/CANAL/')
import argparse
from model import *
import time
import resource
construct the CANAL model
[2]:
experiments = 'Pancreas'
seed = 1
with open(f'./data/{experiments}/{experiments}_highly_gene_idx.csv') as csvfile:
spamreader = csv.reader(csvfile, delimiter=' ', quotechar='|')
highly_variable_idx = []
for row in spamreader:
highly_variable_idx.append(row[0])
highly_variable_idx = [int(i) for i in highly_variable_idx]
highly_variable_idx = np.array(highly_variable_idx)
[3]:
CANAL = CANAL_model(gpu_option='1')
stage 1
load the pre-processed dataset ‘’Muraro’’
[4]:
dataset_stage_1 = "Muraro"
data_path_stage_1 = './data/{}/{}_train.h5ad'.format(experiments, dataset_stage_1)
adata_stage_1 = sc.read_h5ad(data_path_stage_1)
cell_type_stage_1 = adata_stage_1.obs['cell_type1']
print(adata_stage_1)
print(np.unique(np.array(cell_type_stage_1), return_counts=True))
AnnData object with n_obs × n_vars = 1598 × 1000
obs: 'donor', 'cell_type1', 'library', 'organism', 'dataset_name', 'platform', 'organ', 'data_type', 'cell_ontology_class', 'cell_ontology_id', 'n_genes', 'n_counts', '__libsize__', 'baron_human_donor', 'age', 'gender', 'enge_donor', 'batch', 'muraro_donor', 'celltype'
var: 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
uns: 'hvg', 'log1p'
(array(['acinar', 'alpha', 'beta', 'delta', 'ductal', 'gamma',
'mesenchymal'], dtype=object), array([168, 613, 352, 153, 177, 76, 59]))
fine-tune the CANAL model for the first stage
[5]:
CANAL.train(experiments = experiments, pre_dataset = "None", dataset = dataset_stage_1,
adata = adata_stage_1, cell_type = cell_type_stage_1, current_stage = 1,
is_final_stage = False,ckpt_dir = './ckpts/', rehearsal_size = 1000,
highly_variable_idx=highly_variable_idx, SEED = seed)
current data: AnnData object with n_obs × n_vars = 1598 × 1000
obs: 'donor', 'cell_type1', 'library', 'organism', 'dataset_name', 'platform', 'organ', 'data_type', 'cell_ontology_class', 'cell_ontology_id', 'n_genes', 'n_counts', '__libsize__', 'baron_human_donor', 'age', 'gender', 'enge_donor', 'batch', 'muraro_donor', 'celltype'
var: 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
uns: 'hvg', 'log1p'
(array(['acinar', 'alpha', 'beta', 'delta', 'ductal', 'gamma',
'mesenchymal'], dtype=object), array([168, 613, 352, 153, 177, 76, 59]))
model constructing begin!
/data/wanh/CANAL/performer_pytorch/performer_pytorch.py:115: UserWarning: torch.qr is deprecated in favor of torch.linalg.qr and will be removed in a future PyTorch release.
The boolean parameter 'some' has been replaced with a string parameter 'mode'.
Q, R = torch.qr(A, some)
should be replaced with
Q, R = torch.linalg.qr(A, 'reduced' if some else 'complete') (Triggered internally at ../aten/src/ATen/native/BatchLinearAlgebra.cpp:2497.)
q, r = torch.qr(unstructured_block.cpu(), some = True)
/data/wanh/ENTER/envs/pytorch/lib/python3.7/site-packages/torch/nn/_reduction.py:42: UserWarning: size_average and reduce args will be deprecated, please use reduction='mean' instead.
warnings.warn(warning.format(ret))
model constructing finished!
label train: [0 1 2 3 4 5 6] 7
label val: [0 1 2 3 4 5 6] 7
== Begin finetuning: | Initial stage | Current stage: 1 | CANAL | Dataset: Pancreas Muraro ==
== Epoch: 1 | Classification Loss: 1.961912 | Representation DL Loss: 0.000000 | Accuracy: 4.5526% ==
== Epoch: 1 | Validation CLS Loss: 1.950393 | Validation Representation DL Loss: 0.000000 | F1 Score: 0.056088 == | Current accuracy: 0.110390 ==
[[ 0 0 8 0 0 0 25]
[ 0 0 21 0 0 0 109]
[ 0 0 31 0 0 0 34]
[ 0 0 9 0 0 0 21]
[ 0 0 22 0 0 0 4]
[ 0 0 5 0 0 0 7]
[ 0 0 9 0 0 0 3]]
Patience: 0 / 10
== Epoch: 2 | Classification Loss: 1.642274 | Representation DL Loss: 0.000000 | Accuracy: 37.5981% ==
== Epoch: 2 | Validation CLS Loss: 1.650223 | Validation Representation DL Loss: 0.000000 | F1 Score: 0.162927 == | Current accuracy: 0.392857 ==
[[ 0 35 2 0 0 0 0]
[ 0 111 1 0 0 0 0]
[ 0 68 5 0 0 0 0]
[ 0 34 0 0 0 0 0]
[ 0 2 17 0 0 0 8]
[ 0 15 1 0 0 0 0]
[ 0 0 4 0 0 0 5]]
Patience: 0 / 10
== Epoch: 3 | Classification Loss: 1.201346 | Representation DL Loss: 0.000000 | Accuracy: 68.5243% ==
== Epoch: 3 | Validation CLS Loss: 2.090522 | Validation Representation DL Loss: 0.000000 | F1 Score: 0.576238 == | Current accuracy: 0.766234 ==
[[ 37 0 0 0 0 0 0]
[ 0 107 2 0 0 0 0]
[ 5 4 55 0 1 0 0]
[ 0 0 34 0 0 0 0]
[ 3 0 1 0 28 0 0]
[ 2 3 7 0 0 0 0]
[ 0 0 10 0 0 0 9]]
Patience: 0 / 10
== Epoch: 4 | Classification Loss: 0.815416 | Representation DL Loss: 0.000000 | Accuracy: 85.0863% ==
== Epoch: 4 | Validation CLS Loss: 3.136152 | Validation Representation DL Loss: 0.000000 | F1 Score: 0.773316 == | Current accuracy: 0.883117 ==
[[ 31 0 0 0 0 0 0]
[ 0 125 2 0 0 0 0]
[ 2 2 63 0 0 0 0]
[ 0 0 7 16 0 0 0]
[ 2 0 2 0 23 0 0]
[ 1 13 1 4 0 0 0]
[ 0 0 0 0 0 0 14]]
Patience: 0 / 10
== Epoch: 5 | Classification Loss: 0.725605 | Representation DL Loss: 0.000000 | Accuracy: 89.7959% ==
== Epoch: 5 | Validation CLS Loss: 3.432527 | Validation Representation DL Loss: 0.000000 | F1 Score: 0.869056 == | Current accuracy: 0.905844 ==
[[ 29 0 0 0 0 0 0]
[ 0 108 1 0 0 0 0]
[ 4 5 65 0 1 0 0]
[ 0 0 4 29 0 0 0]
[ 0 0 0 0 31 0 0]
[ 2 1 0 11 0 8 0]
[ 0 0 0 0 0 0 9]]
Patience: 0 / 10
== Epoch: 6 | Classification Loss: 0.674437 | Representation DL Loss: 0.000000 | Accuracy: 91.9152% ==
== Epoch: 6 | Validation CLS Loss: 3.153286 | Validation Representation DL Loss: 0.000000 | F1 Score: 0.939253 == | Current accuracy: 0.957792 ==
[[ 29 0 0 0 0 0 0]
[ 0 118 0 0 0 0 0]
[ 3 1 66 0 1 3 0]
[ 0 0 0 23 0 0 0]
[ 2 0 0 0 31 0 2]
[ 0 0 0 1 0 17 0]
[ 0 0 0 0 0 0 11]]
Patience: 0 / 10
== Epoch: 7 | Classification Loss: 0.657222 | Representation DL Loss: 0.000000 | Accuracy: 93.0141% ==
== Epoch: 7 | Validation CLS Loss: 3.399523 | Validation Representation DL Loss: 0.000000 | F1 Score: 0.865206 == | Current accuracy: 0.909091 ==
[[ 28 0 0 0 0 0 0]
[ 0 110 3 0 0 0 0]
[ 3 3 61 0 1 1 0]
[ 0 5 2 29 0 0 0]
[ 4 0 0 0 32 0 0]
[ 1 2 3 0 0 4 0]
[ 0 0 0 0 0 0 16]]
Patience: 1 / 10
== Epoch: 8 | Classification Loss: 0.611189 | Representation DL Loss: 0.000000 | Accuracy: 95.2904% ==
== Epoch: 8 | Validation CLS Loss: 3.023797 | Validation Representation DL Loss: 0.000000 | F1 Score: 0.902009 == | Current accuracy: 0.931818 ==
[[ 32 0 0 0 0 0 0]
[ 0 117 1 0 0 0 0]
[ 2 2 54 0 3 1 0]
[ 0 0 1 25 0 0 0]
[ 3 0 1 0 37 0 3]
[ 0 1 0 3 0 10 0]
[ 0 0 0 0 0 0 12]]
Patience: 2 / 10
== Epoch: 9 | Classification Loss: 0.592466 | Representation DL Loss: 0.000000 | Accuracy: 96.4678% ==
== Epoch: 9 | Validation CLS Loss: 3.163720 | Validation Representation DL Loss: 0.000000 | F1 Score: 0.866828 == | Current accuracy: 0.918831 ==
[[ 34 0 0 0 0 0 0]
[ 0 117 0 3 0 0 0]
[ 3 5 52 2 1 1 0]
[ 0 0 0 23 0 0 0]
[ 1 0 1 0 37 0 0]
[ 1 0 0 7 0 5 0]
[ 0 0 0 0 0 0 15]]
Patience: 3 / 10
== Epoch: 10 | Classification Loss: 0.548137 | Representation DL Loss: 0.000000 | Accuracy: 97.6452% ==
== Epoch: 10 | Validation CLS Loss: 3.195732 | Validation Representation DL Loss: 0.000000 | F1 Score: 0.956919 == | Current accuracy: 0.964286 ==
[[ 32 0 0 0 0 0 0]
[ 0 122 0 0 0 0 0]
[ 1 2 66 0 1 3 0]
[ 0 0 0 23 0 0 0]
[ 2 0 1 0 30 0 0]
[ 0 1 0 0 0 14 0]
[ 0 0 0 0 0 0 10]]
Patience: 0 / 10
== Epoch: 11 | Classification Loss: 0.553055 | Representation DL Loss: 0.000000 | Accuracy: 96.8603% ==
== Epoch: 11 | Validation CLS Loss: 3.350313 | Validation Representation DL Loss: 0.000000 | F1 Score: 0.940164 == | Current accuracy: 0.941558 ==
[[ 32 0 0 0 0 0 0]
[ 0 114 0 0 0 0 0]
[ 1 4 59 0 2 3 0]
[ 0 0 2 24 0 0 0]
[ 0 0 2 0 33 0 0]
[ 1 3 0 0 0 19 0]
[ 0 0 0 0 0 0 9]]
Patience: 1 / 10
== Epoch: 12 | Classification Loss: 0.534972 | Representation DL Loss: 0.000000 | Accuracy: 97.6452% ==
== Epoch: 12 | Validation CLS Loss: 3.556324 | Validation Representation DL Loss: 0.000000 | F1 Score: 0.956248 == | Current accuracy: 0.974026 ==
[[ 38 0 0 0 0 0 0]
[ 0 111 0 0 0 0 0]
[ 1 1 59 0 0 1 0]
[ 0 0 0 35 0 0 0]
[ 2 0 0 0 31 0 0]
[ 0 0 0 3 0 10 0]
[ 0 0 0 0 0 0 16]]
Patience: 0 / 10
== Epoch: 13 | Classification Loss: 0.523263 | Representation DL Loss: 0.000000 | Accuracy: 98.3516% ==
== Epoch: 13 | Validation CLS Loss: 3.316058 | Validation Representation DL Loss: 0.000000 | F1 Score: 0.950711 == | Current accuracy: 0.970779 ==
[[ 33 0 0 0 0 0 0]
[ 0 133 0 0 0 0 0]
[ 1 0 55 0 0 1 0]
[ 0 0 0 26 0 0 0]
[ 2 0 0 0 24 0 0]
[ 1 3 0 1 0 12 0]
[ 0 0 0 0 0 0 16]]
Patience: 1 / 10
== Epoch: 14 | Classification Loss: 0.528360 | Representation DL Loss: 0.000000 | Accuracy: 97.8807% ==
== Epoch: 14 | Validation CLS Loss: 3.190031 | Validation Representation DL Loss: 0.000000 | F1 Score: 0.939733 == | Current accuracy: 0.961039 ==
[[ 24 0 0 0 0 0 0]
[ 0 122 1 0 0 0 0]
[ 0 1 71 0 2 2 0]
[ 0 0 2 25 0 0 0]
[ 2 0 0 0 29 0 0]
[ 0 0 0 2 0 9 0]
[ 0 0 0 0 0 0 16]]
Patience: 2 / 10
== Epoch: 15 | Classification Loss: 0.524607 | Representation DL Loss: 0.000000 | Accuracy: 98.1947% ==
== Epoch: 15 | Validation CLS Loss: 3.279603 | Validation Representation DL Loss: 0.000000 | F1 Score: 0.931640 == | Current accuracy: 0.961039 ==
[[ 30 0 0 0 0 0 0]
[ 0 109 1 0 0 0 0]
[ 2 2 75 0 2 1 0]
[ 0 0 1 28 0 0 0]
[ 0 0 0 0 38 0 0]
[ 2 0 0 1 0 5 0]
[ 0 0 0 0 0 0 11]]
Patience: 3 / 10
== Epoch: 16 | Classification Loss: 0.515277 | Representation DL Loss: 0.000000 | Accuracy: 98.6656% ==
== Epoch: 16 | Validation CLS Loss: 3.152055 | Validation Representation DL Loss: 0.000000 | F1 Score: 0.960131 == | Current accuracy: 0.957792 ==
[[ 25 0 0 0 0 0 0]
[ 0 116 1 0 0 0 0]
[ 1 6 60 0 0 1 0]
[ 0 0 0 31 0 0 0]
[ 0 0 1 0 34 0 0]
[ 1 1 0 1 0 18 0]
[ 0 0 0 0 0 0 11]]
Patience: 4 / 10
== Epoch: 17 | Classification Loss: 0.520274 | Representation DL Loss: 0.000000 | Accuracy: 98.2732% ==
== Epoch: 17 | Validation CLS Loss: 2.981735 | Validation Representation DL Loss: 0.000000 | F1 Score: 0.936793 == | Current accuracy: 0.964286 ==
[[ 40 0 0 0 0 0 0]
[ 0 128 1 0 0 0 0]
[ 1 1 59 0 1 0 0]
[ 0 0 1 27 0 0 0]
[ 1 0 0 0 28 0 0]
[ 1 2 0 2 0 7 0]
[ 0 0 0 0 0 0 8]]
Patience: 5 / 10
== Epoch: 18 | Classification Loss: 0.522617 | Representation DL Loss: 0.000000 | Accuracy: 98.1162% ==
== Epoch: 18 | Validation CLS Loss: 3.346835 | Validation Representation DL Loss: 0.000000 | F1 Score: 0.930889 == | Current accuracy: 0.944805 ==
[[ 36 0 0 0 0 0 0]
[ 0 117 0 0 0 0 0]
[ 2 5 49 0 1 1 0]
[ 0 0 0 34 0 0 0]
[ 1 0 1 0 29 0 0]
[ 0 0 0 6 0 14 0]
[ 0 0 0 0 0 0 12]]
Patience: 6 / 10
== Epoch: 19 | Classification Loss: 0.530727 | Representation DL Loss: 0.000000 | Accuracy: 98.1162% ==
== Epoch: 19 | Validation CLS Loss: 3.501268 | Validation Representation DL Loss: 0.000000 | F1 Score: 0.919102 == | Current accuracy: 0.948052 ==
[[ 31 0 0 0 0 0 0]
[ 0 121 1 0 0 0 0]
[ 0 3 71 1 2 1 0]
[ 0 0 0 37 0 0 0]
[ 2 0 1 0 19 0 0]
[ 1 1 0 3 0 8 0]
[ 0 0 0 0 0 0 5]]
Patience: 7 / 10
== Epoch: 20 | Classification Loss: 0.521382 | Representation DL Loss: 0.000000 | Accuracy: 98.3516% ==
== Epoch: 20 | Validation CLS Loss: 3.170170 | Validation Representation DL Loss: 0.000000 | F1 Score: 0.961344 == | Current accuracy: 0.967532 ==
[[ 42 0 0 0 0 0 0]
[ 0 117 0 0 0 0 0]
[ 1 2 43 0 0 0 0]
[ 0 0 0 42 0 0 0]
[ 4 0 0 0 30 0 0]
[ 1 2 0 0 0 15 0]
[ 0 0 0 0 0 0 9]]
Patience: 8 / 10
== Epoch: 21 | Classification Loss: 0.534004 | Representation DL Loss: 0.000000 | Accuracy: 97.4882% ==
== Epoch: 21 | Validation CLS Loss: 3.470222 | Validation Representation DL Loss: 0.000000 | F1 Score: 0.875722 == | Current accuracy: 0.928571 ==
[[ 25 0 0 0 0 0 0]
[ 0 124 2 0 0 0 0]
[ 1 1 74 0 1 3 0]
[ 0 1 1 16 0 0 0]
[ 4 0 1 0 30 0 1]
[ 0 6 0 0 0 5 0]
[ 0 0 0 0 0 0 12]]
Patience: 9 / 10
== Epoch: 22 | Classification Loss: 0.531479 | Representation DL Loss: 0.000000 | Accuracy: 97.5667% ==
== Epoch: 22 | Validation CLS Loss: 3.345588 | Validation Representation DL Loss: 0.000000 | F1 Score: 0.917604 == | Current accuracy: 0.951299 ==
[[ 30 0 0 0 0 0 0]
[ 0 131 2 0 0 0 0]
[ 1 3 63 0 1 0 0]
[ 0 0 0 22 0 0 0]
[ 0 0 0 0 30 0 2]
[ 0 3 0 3 0 12 0]
[ 0 0 0 0 0 0 5]]
Patience: 10 / 10
== Epoch: 23 | Classification Loss: 0.510365 | Representation DL Loss: 0.000000 | Accuracy: 98.6656% ==
== Epoch: 23 | Validation CLS Loss: 3.260308 | Validation Representation DL Loss: 0.000000 | F1 Score: 0.946963 == | Current accuracy: 0.948052 ==
[[ 36 0 0 0 0 0 0]
[ 0 113 0 0 0 0 0]
[ 2 5 55 0 0 2 0]
[ 0 0 0 24 0 0 0]
[ 4 0 0 0 33 0 0]
[ 0 2 0 1 0 20 0]
[ 0 0 0 0 0 0 11]]
Patience: 11 / 10
/data/wanh/CANAL/model.py:106: ImplicitModificationWarning: Trying to modify attribute `.obs` of view, initializing view as actual.
adata_i.obs['celltype'] = current_label_dict[i]
/data/wanh/ENTER/envs/pytorch/lib/python3.7/site-packages/anndata/_core/anndata.py:1785: FutureWarning: X.dtype being converted to np.float32 from float64. In the next version of anndata (0.9) conversion will not be automatic. Pass dtype explicitly to avoid this warning. Pass `AnnData(X, dtype=X.dtype, ...)` to get the future behavour.
[AnnData(sparse.csr_matrix(a.shape), obs=a.obs) for a in all_adatas],
example bank after updating:
AnnData object with n_obs × n_vars = 845 × 1000
obs: 'donor', 'cell_type1', 'library', 'organism', 'dataset_name', 'platform', 'organ', 'data_type', 'cell_ontology_class', 'cell_ontology_id', 'n_genes', 'n_counts', '__libsize__', 'baron_human_donor', 'age', 'gender', 'enge_donor', 'batch', 'muraro_donor', 'celltype', 'rank', 'stage'
var: 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
cell type composition of this example bank:
(array(['acinar', 'alpha', 'beta', 'delta', 'ductal', 'gamma',
'mesenchymal'], dtype=object), array([142, 142, 142, 142, 142, 76, 59]))
dataset composition from each stage of this example bank:
(array([1]), array([845]))
stage 2
load the pre-processed dataset ‘’Enge’’
[6]:
dataset_stage_2 = "Enge"
data_path_stage_2 = './data/{}/{}_train.h5ad'.format(experiments, dataset_stage_2)
adata_stage_2 = sc.read_h5ad(data_path_stage_2)
cell_type_stage_2 = adata_stage_2.obs['cell_type1']
print(adata_stage_2)
print(np.unique(np.array(cell_type_stage_2), return_counts=True))
AnnData object with n_obs × n_vars = 1782 × 1000
obs: 'donor', 'cell_type1', 'library', 'organism', 'dataset_name', 'platform', 'organ', 'data_type', 'cell_ontology_class', 'cell_ontology_id', 'n_genes', 'n_counts', '__libsize__', 'baron_human_donor', 'age', 'gender', 'enge_donor', 'batch', 'muraro_donor', 'celltype'
var: 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
uns: 'hvg', 'log1p'
(array(['acinar', 'alpha', 'beta', 'delta', 'ductal', 'mesenchymal'],
dtype=object), array([332, 784, 267, 55, 300, 44]))
fine-tune the CANAL model for the second stage
[7]:
CANAL.train(experiments = experiments, pre_dataset = dataset_stage_1, dataset = dataset_stage_2,
adata = adata_stage_2, cell_type = cell_type_stage_2, current_stage = 2,
is_final_stage = False, ckpt_dir='./ckpts/', rehearsal_size=1000, SEED = seed)
current data: AnnData object with n_obs × n_vars = 1782 × 1000
obs: 'donor', 'cell_type1', 'library', 'organism', 'dataset_name', 'platform', 'organ', 'data_type', 'cell_ontology_class', 'cell_ontology_id', 'n_genes', 'n_counts', '__libsize__', 'baron_human_donor', 'age', 'gender', 'enge_donor', 'batch', 'muraro_donor', 'celltype'
var: 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
uns: 'hvg', 'log1p'
(array(['acinar', 'alpha', 'beta', 'delta', 'ductal', 'mesenchymal'],
dtype=object), array([332, 784, 267, 55, 300, 44]))
model constructing begin!
new cell types: []
example bank for experience replay: AnnData object with n_obs × n_vars = 845 × 1000
obs: 'donor', 'cell_type1', 'library', 'organism', 'dataset_name', 'platform', 'organ', 'data_type', 'cell_ontology_class', 'cell_ontology_id', 'n_genes', 'n_counts', '__libsize__', 'baron_human_donor', 'age', 'gender', 'enge_donor', 'batch', 'muraro_donor', 'celltype', 'rank', 'stage'
var: 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
model constructing finished!
label train: [0 1 2 3 4 5 6] 7
label val: [0 1 2 3 4 5 6] 7
/data/wanh/ENTER/envs/pytorch/lib/python3.7/site-packages/torch/nn/_reduction.py:42: UserWarning: size_average and reduce args will be deprecated, please use reduction='mean' instead.
warnings.warn(warning.format(ret))
== Begin finetuning: | Incrmental stage | Current stage: 2 | CANAL | Dataset: Pancreas Enge ==
== Epoch: 1 | Classification Loss: 0.732645 | Representation DL Loss: 0.094611 | Accuracy: 90.7619% ==
== Epoch: 1 | Validation CLS Loss: 2.961814 | Validation Representation DL Loss: 0.213464 | F1 Score: 0.926786 == | Current accuracy: 0.955598 ==
[[ 80 0 0 0 0 0 0]
[ 1 173 1 0 0 5 0]
[ 0 4 90 2 0 0 0]
[ 0 3 1 40 0 0 0]
[ 0 1 0 0 83 0 0]
[ 0 5 0 0 0 10 0]
[ 0 0 0 0 0 0 19]]
Patience: 0 / 10
== Epoch: 2 | Classification Loss: 0.638844 | Representation DL Loss: 0.274230 | Accuracy: 94.7619% ==
== Epoch: 2 | Validation CLS Loss: 2.805388 | Validation Representation DL Loss: 0.231074 | F1 Score: 0.934943 == | Current accuracy: 0.967181 ==
[[ 78 0 0 0 0 0 0]
[ 0 195 0 0 0 1 0]
[ 0 1 76 4 0 0 0]
[ 0 1 0 33 0 0 0]
[ 0 2 0 0 94 0 0]
[ 0 8 0 0 0 10 0]
[ 0 0 0 0 0 0 15]]
Patience: 0 / 10
== Epoch: 3 | Classification Loss: 0.624132 | Representation DL Loss: 0.241587 | Accuracy: 95.1429% ==
== Epoch: 3 | Validation CLS Loss: 3.419496 | Validation Representation DL Loss: 0.199474 | F1 Score: 0.940456 == | Current accuracy: 0.965251 ==
[[103 0 0 0 0 0 0]
[ 0 174 0 0 0 1 0]
[ 2 0 81 0 0 0 0]
[ 0 5 5 32 0 0 0]
[ 0 0 0 0 76 0 0]
[ 0 5 0 0 0 12 0]
[ 0 0 0 0 0 0 22]]
Patience: 1 / 10
== Epoch: 4 | Classification Loss: 0.580929 | Representation DL Loss: 0.223328 | Accuracy: 96.5238% ==
== Epoch: 4 | Validation CLS Loss: 3.178293 | Validation Representation DL Loss: 0.196348 | F1 Score: 0.934727 == | Current accuracy: 0.963320 ==
[[ 88 0 0 0 0 0 0]
[ 1 178 0 0 0 0 0]
[ 0 4 73 2 0 0 0]
[ 0 2 2 38 0 0 0]
[ 0 0 0 0 88 0 0]
[ 0 8 0 0 0 10 0]
[ 0 0 0 0 0 0 24]]
Patience: 2 / 10
== Epoch: 5 | Classification Loss: 0.593464 | Representation DL Loss: 0.263157 | Accuracy: 95.9048% ==
== Epoch: 5 | Validation CLS Loss: 3.048083 | Validation Representation DL Loss: 0.237440 | F1 Score: 0.962671 == | Current accuracy: 0.963320 ==
[[113 0 0 0 0 0 0]
[ 0 157 2 0 0 0 0]
[ 1 5 75 0 0 0 0]
[ 0 5 2 35 0 0 0]
[ 0 3 0 0 82 0 0]
[ 0 1 0 0 0 13 0]
[ 0 0 0 0 0 0 24]]
Patience: 3 / 10
== Epoch: 6 | Classification Loss: 0.591397 | Representation DL Loss: 0.310375 | Accuracy: 95.8571% ==
== Epoch: 6 | Validation CLS Loss: 3.299119 | Validation Representation DL Loss: 0.239495 | F1 Score: 0.963631 == | Current accuracy: 0.976834 ==
[[ 92 0 0 0 0 0 0]
[ 1 187 1 0 0 0 0]
[ 0 2 81 0 0 0 0]
[ 0 2 4 23 0 0 0]
[ 0 0 0 0 87 0 0]
[ 0 2 0 0 0 13 0]
[ 0 0 0 0 0 0 23]]
Patience: 0 / 10
== Epoch: 7 | Classification Loss: 0.574342 | Representation DL Loss: 0.289868 | Accuracy: 96.2857% ==
== Epoch: 7 | Validation CLS Loss: 3.038925 | Validation Representation DL Loss: 0.279740 | F1 Score: 0.968957 == | Current accuracy: 0.982625 ==
[[ 99 0 0 0 0 0 0]
[ 1 184 0 0 0 0 0]
[ 0 0 75 0 0 0 0]
[ 0 2 3 25 0 0 0]
[ 0 0 0 0 86 0 0]
[ 0 3 0 0 0 16 0]
[ 0 0 0 0 0 0 24]]
Patience: 0 / 10
== Epoch: 8 | Classification Loss: 0.534463 | Representation DL Loss: 0.229033 | Accuracy: 98.0952% ==
== Epoch: 8 | Validation CLS Loss: 3.088756 | Validation Representation DL Loss: 0.373417 | F1 Score: 0.932029 == | Current accuracy: 0.940154 ==
[[ 78 0 0 0 0 0 0]
[ 0 186 0 0 0 0 0]
[ 0 6 53 25 0 0 0]
[ 0 0 0 41 0 0 0]
[ 0 0 0 0 92 0 0]
[ 0 0 0 0 0 10 0]
[ 0 0 0 0 0 0 27]]
Patience: 1 / 10
== Epoch: 9 | Classification Loss: 0.525990 | Representation DL Loss: 0.232932 | Accuracy: 98.2857% ==
== Epoch: 9 | Validation CLS Loss: 3.234226 | Validation Representation DL Loss: 0.214714 | F1 Score: 0.956892 == | Current accuracy: 0.974903 ==
[[107 0 0 0 0 0 0]
[ 0 171 3 0 0 0 0]
[ 0 2 77 1 0 0 0]
[ 0 4 0 32 0 0 0]
[ 0 0 0 0 85 0 0]
[ 0 3 0 0 0 8 0]
[ 0 0 0 0 0 0 25]]
Patience: 2 / 10
== Epoch: 10 | Classification Loss: 0.518713 | Representation DL Loss: 0.206362 | Accuracy: 98.4286% ==
== Epoch: 10 | Validation CLS Loss: 3.279357 | Validation Representation DL Loss: 0.233131 | F1 Score: 0.967129 == | Current accuracy: 0.978764 ==
[[ 96 0 0 0 0 0 0]
[ 1 170 1 0 0 0 0]
[ 0 1 96 0 0 0 0]
[ 0 0 6 33 0 0 0]
[ 0 0 0 0 75 0 0]
[ 0 0 2 0 0 11 0]
[ 0 0 0 0 0 0 26]]
Patience: 3 / 10
== Epoch: 11 | Classification Loss: 0.513857 | Representation DL Loss: 0.199755 | Accuracy: 98.6667% ==
== Epoch: 11 | Validation CLS Loss: 3.265217 | Validation Representation DL Loss: 0.179756 | F1 Score: 0.974266 == | Current accuracy: 0.976834 ==
[[ 93 0 0 0 0 0 0]
[ 0 164 2 0 0 0 0]
[ 0 2 80 0 0 0 0]
[ 0 6 0 37 0 0 0]
[ 0 0 0 0 89 0 0]
[ 0 2 0 0 0 21 0]
[ 0 0 0 0 0 0 22]]
Patience: 4 / 10
== Epoch: 12 | Classification Loss: 0.524200 | Representation DL Loss: 0.189691 | Accuracy: 98.0000% ==
== Epoch: 12 | Validation CLS Loss: 3.340358 | Validation Representation DL Loss: 0.204039 | F1 Score: 0.959553 == | Current accuracy: 0.959459 ==
[[ 92 0 0 0 0 0 0]
[ 3 178 0 0 0 0 0]
[ 2 5 77 0 0 0 0]
[ 0 10 0 37 0 0 0]
[ 0 0 0 0 85 0 0]
[ 0 1 0 0 0 11 0]
[ 0 0 0 0 0 0 17]]
Patience: 5 / 10
== Epoch: 13 | Classification Loss: 0.530708 | Representation DL Loss: 0.193137 | Accuracy: 97.7143% ==
== Epoch: 13 | Validation CLS Loss: 3.532589 | Validation Representation DL Loss: 0.166517 | F1 Score: 0.987495 == | Current accuracy: 0.986486 ==
[[115 0 0 0 0 0 0]
[ 2 163 1 0 0 0 0]
[ 0 1 82 0 0 0 0]
[ 0 1 2 36 0 0 0]
[ 0 0 0 0 87 0 0]
[ 0 0 0 0 0 9 0]
[ 0 0 0 0 0 0 19]]
Patience: 0 / 10
== Epoch: 14 | Classification Loss: 0.525352 | Representation DL Loss: 0.180527 | Accuracy: 98.1905% ==
== Epoch: 14 | Validation CLS Loss: 3.379144 | Validation Representation DL Loss: 0.169471 | F1 Score: 0.979416 == | Current accuracy: 0.984556 ==
[[100 0 0 0 0 0 0]
[ 1 172 1 0 0 0 0]
[ 0 0 88 0 0 0 0]
[ 0 2 3 30 0 0 0]
[ 0 0 0 0 81 0 0]
[ 0 1 0 0 0 19 0]
[ 0 0 0 0 0 0 20]]
Patience: 1 / 10
== Epoch: 15 | Classification Loss: 0.510894 | Representation DL Loss: 0.174968 | Accuracy: 98.4286% ==
== Epoch: 15 | Validation CLS Loss: 3.193695 | Validation Representation DL Loss: 0.171501 | F1 Score: 0.971803 == | Current accuracy: 0.974903 ==
[[101 0 0 0 0 0 0]
[ 1 185 3 0 0 0 0]
[ 0 4 80 0 0 0 0]
[ 0 0 4 25 0 0 0]
[ 0 0 0 0 90 0 0]
[ 0 1 0 0 0 16 0]
[ 0 0 0 0 0 0 8]]
Patience: 2 / 10
== Epoch: 16 | Classification Loss: 0.522997 | Representation DL Loss: 0.173566 | Accuracy: 97.8095% ==
== Epoch: 16 | Validation CLS Loss: 3.123750 | Validation Representation DL Loss: 0.176997 | F1 Score: 0.983112 == | Current accuracy: 0.982625 ==
[[ 83 0 0 0 0 0 0]
[ 1 198 1 0 0 0 0]
[ 0 2 84 0 0 0 0]
[ 0 5 0 32 0 0 0]
[ 0 0 0 0 78 0 0]
[ 0 0 0 0 0 11 0]
[ 0 0 0 0 0 0 23]]
Patience: 3 / 10
== Epoch: 17 | Classification Loss: 0.513260 | Representation DL Loss: 0.186763 | Accuracy: 98.0476% ==
== Epoch: 17 | Validation CLS Loss: 2.879964 | Validation Representation DL Loss: 0.168831 | F1 Score: 0.973845 == | Current accuracy: 0.978764 ==
[[ 77 0 0 0 0 0 0]
[ 0 200 2 0 0 0 0]
[ 0 3 76 0 0 0 0]
[ 0 3 1 35 0 0 0]
[ 0 0 0 0 88 0 0]
[ 0 2 0 0 0 14 0]
[ 0 0 0 0 0 0 17]]
Patience: 4 / 10
== Epoch: 18 | Classification Loss: 0.524881 | Representation DL Loss: 0.189891 | Accuracy: 97.9524% ==
== Epoch: 18 | Validation CLS Loss: 3.263474 | Validation Representation DL Loss: 0.244640 | F1 Score: 0.962683 == | Current accuracy: 0.969112 ==
[[ 86 0 0 0 0 0 0]
[ 7 200 0 0 0 0 0]
[ 1 2 71 0 0 0 0]
[ 0 3 0 32 0 0 0]
[ 0 0 0 0 83 0 0]
[ 0 2 0 1 0 13 0]
[ 0 0 0 0 0 0 17]]
Patience: 5 / 10
== Epoch: 19 | Classification Loss: 0.531891 | Representation DL Loss: 0.206761 | Accuracy: 97.5714% ==
== Epoch: 19 | Validation CLS Loss: 3.369882 | Validation Representation DL Loss: 0.187305 | F1 Score: 0.975497 == | Current accuracy: 0.974903 ==
[[ 82 0 0 0 0 0 0]
[ 2 188 1 0 0 0 0]
[ 1 2 83 1 0 0 0]
[ 0 6 0 33 0 0 0]
[ 0 0 0 0 81 0 0]
[ 0 0 0 0 0 17 0]
[ 0 0 0 0 0 0 21]]
Patience: 6 / 10
== Epoch: 20 | Classification Loss: 0.520505 | Representation DL Loss: 0.232034 | Accuracy: 98.1905% ==
== Epoch: 20 | Validation CLS Loss: 3.216591 | Validation Representation DL Loss: 0.212197 | F1 Score: 0.967152 == | Current accuracy: 0.976834 ==
[[108 0 0 0 0 0 0]
[ 1 192 1 0 0 0 0]
[ 0 1 63 0 0 0 0]
[ 0 6 1 34 0 1 0]
[ 0 0 0 0 83 0 0]
[ 0 1 0 0 0 13 0]
[ 0 0 0 0 0 0 13]]
Patience: 7 / 10
== Epoch: 21 | Classification Loss: 0.515391 | Representation DL Loss: 0.240733 | Accuracy: 98.3333% ==
== Epoch: 21 | Validation CLS Loss: 3.454315 | Validation Representation DL Loss: 0.246661 | F1 Score: 0.992885 == | Current accuracy: 0.990347 ==
[[117 0 0 0 0 0 0]
[ 3 164 1 0 0 0 0]
[ 0 0 84 0 0 0 0]
[ 0 0 1 37 0 0 0]
[ 0 0 0 0 85 0 0]
[ 0 0 0 0 0 8 0]
[ 0 0 0 0 0 0 18]]
Patience: 0 / 10
== Epoch: 22 | Classification Loss: 0.538312 | Representation DL Loss: 0.273456 | Accuracy: 97.5238% ==
== Epoch: 22 | Validation CLS Loss: 3.570201 | Validation Representation DL Loss: 0.231346 | F1 Score: 0.969802 == | Current accuracy: 0.976834 ==
[[115 0 0 0 0 0 0]
[ 1 166 0 0 0 0 0]
[ 0 0 68 0 0 0 0]
[ 0 10 0 32 0 0 0]
[ 0 0 0 0 88 0 0]
[ 0 1 0 0 0 13 0]
[ 0 0 0 0 0 0 24]]
Patience: 1 / 10
== Epoch: 23 | Classification Loss: 0.505255 | Representation DL Loss: 0.234257 | Accuracy: 98.6667% ==
== Epoch: 23 | Validation CLS Loss: 3.385605 | Validation Representation DL Loss: 0.200926 | F1 Score: 0.970653 == | Current accuracy: 0.971042 ==
[[ 96 0 0 0 0 0 0]
[ 1 176 3 0 0 0 0]
[ 2 1 84 2 0 0 0]
[ 0 6 0 31 0 0 0]
[ 0 0 0 0 87 0 0]
[ 0 0 0 0 0 18 0]
[ 0 0 0 0 0 0 11]]
Patience: 2 / 10
== Epoch: 24 | Classification Loss: 0.513569 | Representation DL Loss: 0.243295 | Accuracy: 98.3333% ==
== Epoch: 24 | Validation CLS Loss: 3.379432 | Validation Representation DL Loss: 0.242467 | F1 Score: 0.987711 == | Current accuracy: 0.990347 ==
[[ 90 0 0 0 0 0 0]
[ 0 190 1 0 0 0 0]
[ 0 1 81 3 0 0 0]
[ 0 0 0 28 0 0 0]
[ 0 0 0 0 97 0 0]
[ 0 0 0 0 0 15 0]
[ 0 0 0 0 0 0 12]]
Patience: 3 / 10
== Epoch: 25 | Classification Loss: 0.510514 | Representation DL Loss: 0.243732 | Accuracy: 98.4286% ==
== Epoch: 25 | Validation CLS Loss: 3.533717 | Validation Representation DL Loss: 0.206528 | F1 Score: 0.980295 == | Current accuracy: 0.976834 ==
[[ 72 0 0 0 0 0 0]
[ 3 184 1 0 0 0 0]
[ 2 2 80 0 0 0 0]
[ 0 4 0 40 0 0 0]
[ 0 0 0 0 92 0 0]
[ 0 0 0 0 0 18 0]
[ 0 0 0 0 0 0 20]]
Patience: 4 / 10
== Epoch: 26 | Classification Loss: 0.512440 | Representation DL Loss: 0.225025 | Accuracy: 98.0952% ==
== Epoch: 26 | Validation CLS Loss: 3.443067 | Validation Representation DL Loss: 0.221002 | F1 Score: 0.976119 == | Current accuracy: 0.992278 ==
[[ 80 0 0 0 0 0 0]
[ 0 188 0 0 0 0 0]
[ 0 1 88 1 0 0 0]
[ 0 0 0 38 0 0 0]
[ 0 0 0 0 94 0 0]
[ 0 1 0 1 0 7 0]
[ 0 0 0 0 0 0 19]]
Patience: 0 / 10
== Epoch: 27 | Classification Loss: 0.495099 | Representation DL Loss: 0.211242 | Accuracy: 99.0000% ==
== Epoch: 27 | Validation CLS Loss: 3.233916 | Validation Representation DL Loss: 0.234549 | F1 Score: 0.966947 == | Current accuracy: 0.982625 ==
[[103 0 0 0 0 0 0]
[ 1 171 1 0 0 0 0]
[ 0 0 73 0 0 0 0]
[ 0 0 4 39 0 0 0]
[ 0 0 0 0 90 0 0]
[ 0 0 3 0 0 11 0]
[ 0 0 0 0 0 0 22]]
Patience: 1 / 10
== Epoch: 28 | Classification Loss: 0.502362 | Representation DL Loss: 0.221395 | Accuracy: 98.5714% ==
== Epoch: 28 | Validation CLS Loss: 3.189550 | Validation Representation DL Loss: 0.203128 | F1 Score: 0.985530 == | Current accuracy: 0.988417 ==
[[ 93 0 0 0 0 0 0]
[ 1 178 0 0 0 0 0]
[ 0 0 68 0 0 0 0]
[ 0 5 0 29 0 0 0]
[ 0 0 0 0 94 0 0]
[ 0 0 0 0 0 20 0]
[ 0 0 0 0 0 0 30]]
Patience: 2 / 10
== Epoch: 29 | Classification Loss: 0.497482 | Representation DL Loss: 0.211709 | Accuracy: 98.7619% ==
== Epoch: 29 | Validation CLS Loss: 3.573305 | Validation Representation DL Loss: 0.200726 | F1 Score: 0.979131 == | Current accuracy: 0.994208 ==
[[ 95 0 0 0 0 0 0]
[ 0 172 1 0 0 0 0]
[ 0 0 81 0 0 0 0]
[ 0 0 0 48 0 0 0]
[ 0 0 0 0 84 0 0]
[ 0 0 2 0 0 7 0]
[ 0 0 0 0 0 0 28]]
Patience: 0 / 10
== Epoch: 30 | Classification Loss: 0.497388 | Representation DL Loss: 0.209115 | Accuracy: 98.8095% ==
== Epoch: 30 | Validation CLS Loss: 3.247670 | Validation Representation DL Loss: 0.192950 | F1 Score: 0.969476 == | Current accuracy: 0.988417 ==
[[ 96 0 0 0 0 0 0]
[ 0 183 1 0 0 0 0]
[ 0 0 84 0 0 0 0]
[ 0 0 2 40 0 0 0]
[ 0 0 0 0 86 0 0]
[ 0 2 1 0 0 8 0]
[ 0 0 0 0 0 0 15]]
Patience: 1 / 10
== Epoch: 31 | Classification Loss: 0.492352 | Representation DL Loss: 0.193179 | Accuracy: 98.8095% ==
== Epoch: 31 | Validation CLS Loss: 3.353072 | Validation Representation DL Loss: 0.193345 | F1 Score: 0.979558 == | Current accuracy: 0.978764 ==
[[ 87 0 0 0 0 0 0]
[ 1 169 1 0 0 0 0]
[ 0 2 88 0 0 0 0]
[ 0 6 1 37 0 0 0]
[ 0 0 0 0 94 0 0]
[ 0 0 0 0 0 22 0]
[ 0 0 0 0 0 0 10]]
Patience: 2 / 10
== Epoch: 32 | Classification Loss: 0.486709 | Representation DL Loss: 0.195091 | Accuracy: 99.0476% ==
== Epoch: 32 | Validation CLS Loss: 3.251515 | Validation Representation DL Loss: 0.178180 | F1 Score: 0.975382 == | Current accuracy: 0.978764 ==
[[100 0 0 0 0 0 0]
[ 1 185 1 0 0 0 0]
[ 0 3 72 0 0 0 0]
[ 0 5 0 40 0 0 0]
[ 0 0 0 0 75 0 0]
[ 0 1 0 0 0 9 0]
[ 0 0 0 0 0 0 26]]
Patience: 3 / 10
== Epoch: 33 | Classification Loss: 0.489627 | Representation DL Loss: 0.196400 | Accuracy: 98.8095% ==
== Epoch: 33 | Validation CLS Loss: 3.255376 | Validation Representation DL Loss: 0.170761 | F1 Score: 0.985829 == | Current accuracy: 0.988417 ==
[[106 0 0 0 0 0 0]
[ 0 185 1 0 0 0 0]
[ 0 0 88 0 0 0 0]
[ 0 4 1 31 0 0 0]
[ 0 0 0 0 77 0 0]
[ 0 0 0 0 0 7 0]
[ 0 0 0 0 0 0 18]]
Patience: 4 / 10
== Epoch: 34 | Classification Loss: 0.500685 | Representation DL Loss: 0.185241 | Accuracy: 98.3333% ==
== Epoch: 34 | Validation CLS Loss: 3.326653 | Validation Representation DL Loss: 0.181498 | F1 Score: 0.984720 == | Current accuracy: 0.988417 ==
[[ 93 0 0 0 0 0 0]
[ 1 199 0 0 0 0 0]
[ 0 0 75 0 0 0 0]
[ 0 3 1 34 0 0 0]
[ 0 0 0 0 75 0 0]
[ 0 1 0 0 0 18 0]
[ 0 0 0 0 0 0 18]]
Patience: 5 / 10
== Epoch: 35 | Classification Loss: 0.484148 | Representation DL Loss: 0.180605 | Accuracy: 99.0476% ==
== Epoch: 35 | Validation CLS Loss: 3.298595 | Validation Representation DL Loss: 0.174113 | F1 Score: 0.982748 == | Current accuracy: 0.990347 ==
[[100 0 0 0 0 0 0]
[ 1 190 0 0 0 0 0]
[ 0 0 90 0 0 0 0]
[ 0 1 1 32 0 0 0]
[ 0 0 0 0 77 0 0]
[ 0 1 1 0 0 14 0]
[ 0 0 0 0 0 0 10]]
Patience: 6 / 10
== Epoch: 36 | Classification Loss: 0.481255 | Representation DL Loss: 0.182213 | Accuracy: 99.3810% ==
== Epoch: 36 | Validation CLS Loss: 2.989938 | Validation Representation DL Loss: 0.176544 | F1 Score: 0.982944 == | Current accuracy: 0.982625 ==
[[ 96 0 0 0 0 0 0]
[ 2 174 3 0 0 0 0]
[ 0 0 76 0 0 0 0]
[ 0 4 0 29 0 0 0]
[ 0 0 0 0 94 0 0]
[ 0 0 0 0 0 16 0]
[ 0 0 0 0 0 0 24]]
Patience: 7 / 10
== Epoch: 37 | Classification Loss: 0.490625 | Representation DL Loss: 0.182296 | Accuracy: 98.9524% ==
== Epoch: 37 | Validation CLS Loss: 3.419080 | Validation Representation DL Loss: 0.175071 | F1 Score: 0.989412 == | Current accuracy: 0.990347 ==
[[ 84 0 0 0 0 0 0]
[ 0 179 1 0 0 0 0]
[ 0 0 79 0 0 0 0]
[ 0 4 0 35 0 0 0]
[ 0 0 0 0 94 0 0]
[ 0 0 0 0 0 20 0]
[ 0 0 0 0 0 0 22]]
Patience: 8 / 10
== Epoch: 38 | Classification Loss: 0.481338 | Representation DL Loss: 0.179012 | Accuracy: 99.0476% ==
== Epoch: 38 | Validation CLS Loss: 3.388458 | Validation Representation DL Loss: 0.173048 | F1 Score: 0.989224 == | Current accuracy: 0.992278 ==
[[100 0 0 0 0 0 0]
[ 0 169 2 0 0 0 0]
[ 0 0 79 0 0 0 0]
[ 0 1 0 44 0 0 0]
[ 0 0 0 0 88 0 0]
[ 0 1 0 0 0 12 0]
[ 0 0 0 0 0 0 22]]
Patience: 9 / 10
== Epoch: 39 | Classification Loss: 0.484925 | Representation DL Loss: 0.173560 | Accuracy: 99.0952% ==
== Epoch: 39 | Validation CLS Loss: 3.233103 | Validation Representation DL Loss: 0.177035 | F1 Score: 0.982181 == | Current accuracy: 0.978764 ==
[[ 85 0 0 0 0 0 0]
[ 2 192 1 0 0 0 0]
[ 0 3 83 0 0 0 0]
[ 0 5 0 38 0 0 0]
[ 0 0 0 0 75 0 0]
[ 0 0 0 0 0 17 0]
[ 0 0 0 0 0 0 17]]
Patience: 10 / 10
== Epoch: 40 | Classification Loss: 0.487681 | Representation DL Loss: 0.175791 | Accuracy: 98.9048% ==
== Epoch: 40 | Validation CLS Loss: 3.575808 | Validation Representation DL Loss: 0.171366 | F1 Score: 0.992162 == | Current accuracy: 0.990347 ==
[[ 91 0 0 0 0 0 0]
[ 2 168 1 0 0 0 0]
[ 0 0 90 0 0 0 0]
[ 0 2 0 41 0 0 0]
[ 0 0 0 0 89 0 0]
[ 0 0 0 0 0 15 0]
[ 0 0 0 0 0 0 19]]
Patience: 11 / 10
/data/wanh/CANAL/model.py:106: ImplicitModificationWarning: Trying to modify attribute `.obs` of view, initializing view as actual.
adata_i.obs['celltype'] = current_label_dict[i]
/data/wanh/ENTER/envs/pytorch/lib/python3.7/site-packages/anndata/_core/anndata.py:1785: FutureWarning: X.dtype being converted to np.float32 from float64. In the next version of anndata (0.9) conversion will not be automatic. Pass dtype explicitly to avoid this warning. Pass `AnnData(X, dtype=X.dtype, ...)` to get the future behavour.
[AnnData(sparse.csr_matrix(a.shape), obs=a.obs) for a in all_adatas],
example bank after updating:
AnnData object with n_obs × n_vars = 889 × 1000
obs: 'donor', 'cell_type1', 'library', 'organism', 'dataset_name', 'platform', 'organ', 'data_type', 'cell_ontology_class', 'cell_ontology_id', 'n_genes', 'n_counts', '__libsize__', 'baron_human_donor', 'age', 'gender', 'enge_donor', 'batch', 'muraro_donor', 'celltype', 'rank', 'stage'
var: 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
cell type composition of this example bank:
(array(['acinar', 'alpha', 'beta', 'delta', 'ductal', 'gamma',
'mesenchymal'], dtype=object), array([142, 142, 142, 142, 142, 76, 103]))
dataset composition from each stage of this example bank:
(array([1, 2]), array([506, 383]))
stage 3
load the pre-processed dataset ‘’Baron_human’’
[8]:
dataset_stage_3 = "Baron_human"
data_path_stage_3 = './data/{}/{}_train.h5ad'.format(experiments, dataset_stage_3)
adata_stage_3 = sc.read_h5ad(data_path_stage_3)
cell_type_stage_3 = adata_stage_3.obs['cell_type1']
print(adata_stage_3)
print(np.unique(np.array(cell_type_stage_3), return_counts=True))
AnnData object with n_obs × n_vars = 7951 × 1000
obs: 'donor', 'cell_type1', 'library', 'organism', 'dataset_name', 'platform', 'organ', 'data_type', 'cell_ontology_class', 'cell_ontology_id', 'n_genes', 'n_counts', '__libsize__', 'baron_human_donor', 'age', 'gender', 'enge_donor', 'batch', 'muraro_donor', 'celltype'
var: 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
uns: 'hvg', 'log1p'
(array(['acinar', 'activated_stellate', 'alpha', 'beta', 'delta', 'ductal',
'endothelial', 'gamma', 'quiescent_stellate'], dtype=object), array([ 908, 268, 2176, 2367, 570, 1016, 241, 243, 162]))
fine-tune the CANAL model for the third stage
[9]:
CANAL.train(experiments = experiments, pre_dataset = dataset_stage_2, dataset = dataset_stage_3,
adata = adata_stage_3, cell_type = cell_type_stage_3, current_stage = 3, is_final_stage = True,
ckpt_dir='./ckpts/', rehearsal_size=1000, SEED = seed)
current data: AnnData object with n_obs × n_vars = 7951 × 1000
obs: 'donor', 'cell_type1', 'library', 'organism', 'dataset_name', 'platform', 'organ', 'data_type', 'cell_ontology_class', 'cell_ontology_id', 'n_genes', 'n_counts', '__libsize__', 'baron_human_donor', 'age', 'gender', 'enge_donor', 'batch', 'muraro_donor', 'celltype'
var: 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
uns: 'hvg', 'log1p'
(array(['acinar', 'activated_stellate', 'alpha', 'beta', 'delta', 'ductal',
'endothelial', 'gamma', 'quiescent_stellate'], dtype=object), array([ 908, 268, 2176, 2367, 570, 1016, 241, 243, 162]))
model constructing begin!
new cell types: ['activated_stellate', 'endothelial', 'quiescent_stellate']
example bank for experience replay: AnnData object with n_obs × n_vars = 889 × 1000
obs: 'donor', 'cell_type1', 'library', 'organism', 'dataset_name', 'platform', 'organ', 'data_type', 'cell_ontology_class', 'cell_ontology_id', 'n_genes', 'n_counts', '__libsize__', 'baron_human_donor', 'age', 'gender', 'enge_donor', 'batch', 'muraro_donor', 'celltype', 'rank', 'stage'
var: 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
model constructing finished!
label train: [0 1 2 3 4 5 6 7 8 9] 10
label val: [0 1 2 3 4 5 6 7 8 9] 10
== Begin finetuning: | Final stage | Current stage: 3 | CANAL | Dataset: Pancreas Baron_human ==
/data/wanh/ENTER/envs/pytorch/lib/python3.7/site-packages/torch/nn/_reduction.py:42: UserWarning: size_average and reduce args will be deprecated, please use reduction='mean' instead.
warnings.warn(warning.format(ret))
== Epoch: 1 | Classification Loss: 0.897652 | Representation DL Loss: 0.042492 | Accuracy: 87.1570% ==
== Epoch: 1 | Validation CLS Loss: 3.369426 | Validation Representation DL Loss: 0.082840 | F1 Score: 0.548260 == | Current accuracy: 0.848639 ==
[[187 0 0 0 0 0 2 0 0 0]
[ 2 439 6 0 0 5 0 0 0 0]
[ 4 5 489 7 0 0 0 0 0 0]
[ 0 0 6 114 0 0 0 0 0 0]
[ 39 2 2 0 216 0 0 0 0 0]
[ 0 22 9 3 0 32 0 0 0 0]
[ 0 0 0 0 0 0 20 0 0 0]
[ 0 0 0 0 0 0 54 0 0 0]
[ 0 10 18 0 29 0 6 0 0 0]
[ 0 0 2 0 0 0 34 0 0 0]]
Patience: 0 / 10
== Epoch: 2 | Classification Loss: 0.770181 | Representation DL Loss: 0.238887 | Accuracy: 89.9576% ==
== Epoch: 2 | Validation CLS Loss: 3.496991 | Validation Representation DL Loss: 0.277476 | F1 Score: 0.763874 == | Current accuracy: 0.920635 ==
[[212 0 0 0 1 0 0 0 0 0]
[ 0 425 3 0 3 4 0 0 0 0]
[ 7 5 502 2 0 0 0 0 0 0]
[ 0 0 7 130 0 0 0 0 0 0]
[ 20 0 0 0 200 0 0 0 0 0]
[ 0 19 3 5 0 58 0 0 0 0]
[ 0 0 0 0 0 0 9 4 0 0]
[ 0 0 0 1 1 0 10 45 0 0]
[ 0 3 2 0 0 0 0 0 41 0]
[ 0 0 1 0 0 0 19 19 1 2]]
Patience: 0 / 10
== Epoch: 3 | Classification Loss: 0.684652 | Representation DL Loss: 0.308691 | Accuracy: 93.5219% ==
== Epoch: 3 | Validation CLS Loss: 3.752732 | Validation Representation DL Loss: 0.318060 | F1 Score: 0.907458 == | Current accuracy: 0.950113 ==
[[177 0 0 0 1 0 0 0 0 2]
[ 0 472 1 0 0 3 0 0 0 0]
[ 6 5 522 3 0 0 0 0 0 0]
[ 0 0 5 112 0 0 0 0 0 0]
[ 17 1 0 2 212 0 0 0 0 0]
[ 0 21 0 4 0 45 0 0 0 0]
[ 0 0 0 0 0 0 18 0 0 0]
[ 0 0 0 3 0 0 5 54 0 1]
[ 0 1 0 0 0 0 0 0 35 0]
[ 0 0 0 0 0 0 3 4 0 29]]
Patience: 0 / 10
== Epoch: 4 | Classification Loss: 0.648321 | Representation DL Loss: 0.319022 | Accuracy: 95.6577% ==
== Epoch: 4 | Validation CLS Loss: 3.773286 | Validation Representation DL Loss: 0.253966 | F1 Score: 0.890921 == | Current accuracy: 0.946712 ==
[[196 0 0 0 1 0 0 0 0 0]
[ 0 473 3 0 0 1 0 0 0 0]
[ 2 2 480 6 0 1 0 0 0 0]
[ 0 0 3 132 0 0 0 0 0 0]
[ 19 3 0 4 214 0 0 0 0 0]
[ 0 23 0 2 0 50 0 0 0 0]
[ 0 0 0 0 0 0 19 4 0 0]
[ 0 0 0 1 0 0 0 43 0 0]
[ 0 0 1 0 0 0 0 0 47 0]
[ 0 0 3 0 0 0 0 15 0 16]]
Patience: 1 / 10
== Epoch: 5 | Classification Loss: 0.631493 | Representation DL Loss: 0.338521 | Accuracy: 95.8557% ==
== Epoch: 5 | Validation CLS Loss: 3.629336 | Validation Representation DL Loss: 0.306584 | F1 Score: 0.956130 == | Current accuracy: 0.967687 ==
[[208 0 0 0 3 0 0 0 0 3]
[ 0 443 3 0 0 0 0 0 0 0]
[ 2 4 473 2 0 4 0 0 0 0]
[ 0 0 8 134 0 0 0 0 0 0]
[ 4 1 0 1 213 0 0 0 0 0]
[ 0 14 2 1 0 48 0 0 0 0]
[ 0 0 0 0 0 0 26 0 0 0]
[ 0 0 0 0 0 0 2 63 0 0]
[ 0 0 2 0 0 0 0 0 56 0]
[ 0 0 1 0 0 0 0 0 0 43]]
Patience: 0 / 10
== Epoch: 6 | Classification Loss: 0.608819 | Representation DL Loss: 0.326651 | Accuracy: 96.8741% ==
== Epoch: 6 | Validation CLS Loss: 3.729682 | Validation Representation DL Loss: 0.256002 | F1 Score: 0.965991 == | Current accuracy: 0.967687 ==
[[211 0 0 0 1 0 0 0 0 0]
[ 3 439 5 1 1 4 0 0 0 0]
[ 4 4 501 5 0 2 0 0 0 0]
[ 0 0 6 126 0 0 0 0 0 0]
[ 5 1 0 0 217 0 0 0 0 0]
[ 0 10 0 3 0 66 0 0 0 0]
[ 0 0 0 0 0 0 18 0 0 0]
[ 0 0 0 0 0 0 0 59 0 0]
[ 0 0 0 0 0 0 0 0 48 0]
[ 0 0 1 0 0 0 0 1 0 22]]
Patience: 1 / 10
== Epoch: 7 | Classification Loss: 0.584976 | Representation DL Loss: 0.289090 | Accuracy: 97.7935% ==
== Epoch: 7 | Validation CLS Loss: 3.903794 | Validation Representation DL Loss: 0.303166 | F1 Score: 0.941116 == | Current accuracy: 0.961451 ==
[[215 0 0 0 6 0 0 0 0 0]
[ 0 461 2 0 2 0 0 0 0 0]
[ 2 7 507 3 0 0 0 0 0 0]
[ 0 0 5 126 0 0 0 0 0 0]
[ 7 0 0 1 215 0 0 0 0 0]
[ 0 10 0 20 0 34 0 0 0 0]
[ 0 0 0 0 0 0 21 0 0 0]
[ 0 0 0 1 0 0 0 56 0 0]
[ 0 0 0 0 0 0 0 0 37 0]
[ 0 0 0 0 0 0 0 2 0 24]]
Patience: 2 / 10
== Epoch: 8 | Classification Loss: 0.578578 | Representation DL Loss: 0.271034 | Accuracy: 97.9915% ==
== Epoch: 8 | Validation CLS Loss: 3.763245 | Validation Representation DL Loss: 0.247214 | F1 Score: 0.958546 == | Current accuracy: 0.964853 ==
[[213 0 0 0 0 0 0 0 0 1]
[ 2 429 0 1 2 11 0 0 0 0]
[ 5 5 472 11 0 3 0 0 0 0]
[ 0 0 1 148 0 0 0 0 0 0]
[ 9 0 0 1 231 0 0 0 0 0]
[ 0 4 0 3 0 58 0 0 0 0]
[ 0 0 0 0 0 0 25 0 0 0]
[ 0 0 0 1 0 0 0 46 0 0]
[ 0 0 0 0 0 0 0 0 54 0]
[ 0 0 0 0 0 0 0 2 0 26]]
Patience: 3 / 10
== Epoch: 9 | Classification Loss: 0.567641 | Representation DL Loss: 0.262974 | Accuracy: 98.3027% ==
== Epoch: 9 | Validation CLS Loss: 3.825476 | Validation Representation DL Loss: 0.243286 | F1 Score: 0.969018 == | Current accuracy: 0.970522 ==
[[220 0 0 0 0 0 0 0 0 0]
[ 0 444 2 1 4 0 0 0 0 0]
[ 3 4 458 7 0 0 0 0 0 0]
[ 0 0 6 132 0 0 0 0 0 0]
[ 11 1 0 0 249 0 0 0 0 0]
[ 0 10 0 1 0 58 0 0 0 0]
[ 0 0 0 0 0 0 17 0 0 0]
[ 0 0 0 0 0 0 0 54 0 0]
[ 0 0 0 0 0 0 0 0 57 0]
[ 0 0 0 0 0 0 0 2 0 23]]
Patience: 0 / 10
== Epoch: 10 | Classification Loss: 0.566113 | Representation DL Loss: 0.247474 | Accuracy: 98.3168% ==
== Epoch: 10 | Validation CLS Loss: 4.053824 | Validation Representation DL Loss: 0.233484 | F1 Score: 0.965931 == | Current accuracy: 0.972222 ==
[[202 0 0 0 0 0 0 0 0 1]
[ 3 477 1 0 4 1 0 0 0 1]
[ 0 5 497 9 0 1 0 0 0 0]
[ 0 0 0 133 0 0 0 0 0 0]
[ 8 2 0 1 203 0 0 0 0 0]
[ 0 9 0 0 0 51 0 0 0 0]
[ 0 0 0 0 0 0 15 0 0 0]
[ 0 0 0 0 0 0 0 57 0 0]
[ 0 0 0 0 0 0 0 0 49 0]
[ 0 0 0 0 0 0 0 3 0 31]]
Patience: 0 / 10
== Epoch: 11 | Classification Loss: 0.558788 | Representation DL Loss: 0.237838 | Accuracy: 98.5431% ==
== Epoch: 11 | Validation CLS Loss: 3.754784 | Validation Representation DL Loss: 0.228295 | F1 Score: 0.968038 == | Current accuracy: 0.977324 ==
[[181 0 0 0 3 0 0 0 0 4]
[ 3 464 3 0 2 2 0 0 0 0]
[ 0 2 510 4 0 5 0 0 0 0]
[ 0 0 2 133 0 0 0 0 0 0]
[ 1 0 0 1 230 0 0 0 0 0]
[ 0 6 0 1 0 50 0 0 0 0]
[ 0 0 0 0 0 0 22 0 0 0]
[ 0 0 0 0 0 0 0 61 0 0]
[ 0 0 0 0 0 0 0 0 47 0]
[ 0 0 1 0 0 0 0 0 0 26]]
Patience: 0 / 10
== Epoch: 12 | Classification Loss: 0.551720 | Representation DL Loss: 0.226611 | Accuracy: 98.7694% ==
== Epoch: 12 | Validation CLS Loss: 3.678352 | Validation Representation DL Loss: 0.216834 | F1 Score: 0.967915 == | Current accuracy: 0.975057 ==
[[221 0 0 0 4 0 0 0 0 0]
[ 1 468 0 1 3 1 0 0 0 0]
[ 2 1 479 4 0 2 0 0 0 0]
[ 0 0 7 133 0 0 0 0 0 0]
[ 0 0 0 2 206 0 0 0 0 0]
[ 0 10 0 2 0 54 0 0 0 0]
[ 0 0 0 0 0 0 19 0 0 0]
[ 0 0 0 2 0 0 0 54 0 0]
[ 0 0 0 0 0 0 0 0 52 0]
[ 0 0 0 0 0 0 0 2 0 34]]
Patience: 1 / 10
== Epoch: 13 | Classification Loss: 0.548298 | Representation DL Loss: 0.211981 | Accuracy: 98.9250% ==
== Epoch: 13 | Validation CLS Loss: 3.758121 | Validation Representation DL Loss: 0.213382 | F1 Score: 0.970261 == | Current accuracy: 0.973923 ==
[[238 0 0 0 1 0 0 0 0 0]
[ 0 443 1 1 0 0 0 0 0 0]
[ 4 2 486 6 0 4 0 0 0 0]
[ 0 0 4 136 0 0 0 0 0 0]
[ 8 1 0 2 210 0 0 0 0 0]
[ 0 8 0 3 0 52 0 0 0 0]
[ 0 0 0 0 0 0 18 0 0 0]
[ 0 0 0 0 0 0 0 60 0 0]
[ 0 0 0 0 0 0 0 0 47 0]
[ 0 0 0 0 0 0 0 1 0 28]]
Patience: 2 / 10
== Epoch: 14 | Classification Loss: 0.544526 | Representation DL Loss: 0.210421 | Accuracy: 99.0099% ==
== Epoch: 14 | Validation CLS Loss: 3.543711 | Validation Representation DL Loss: 0.203992 | F1 Score: 0.969811 == | Current accuracy: 0.974490 ==
[[204 0 0 0 1 0 0 0 0 3]
[ 1 454 1 2 3 0 0 0 0 0]
[ 4 4 478 10 0 1 0 0 0 0]
[ 0 0 5 143 0 0 0 0 0 0]
[ 3 0 0 0 229 0 0 0 0 0]
[ 0 5 0 0 0 62 0 0 0 0]
[ 0 0 0 0 0 0 26 0 0 0]
[ 0 0 0 0 0 0 0 52 0 0]
[ 0 0 0 0 0 0 0 0 47 0]
[ 0 0 0 0 0 0 0 2 0 24]]
Patience: 3 / 10
== Epoch: 15 | Classification Loss: 0.545681 | Representation DL Loss: 0.208248 | Accuracy: 98.8826% ==
== Epoch: 15 | Validation CLS Loss: 3.947368 | Validation Representation DL Loss: 0.212520 | F1 Score: 0.980671 == | Current accuracy: 0.981859 ==
[[177 0 0 0 2 0 0 0 0 1]
[ 0 482 0 0 2 1 0 0 0 0]
[ 2 6 498 3 0 0 0 0 0 0]
[ 0 0 3 144 0 0 0 0 0 0]
[ 2 0 0 1 209 0 0 0 0 0]
[ 0 8 0 1 0 55 0 0 0 0]
[ 0 0 0 0 0 0 22 0 0 0]
[ 0 0 0 0 0 0 0 56 0 0]
[ 0 0 0 0 0 0 0 0 58 0]
[ 0 0 0 0 0 0 0 0 0 31]]
Patience: 0 / 10
== Epoch: 16 | Classification Loss: 0.541018 | Representation DL Loss: 0.206212 | Accuracy: 99.1938% ==
== Epoch: 16 | Validation CLS Loss: 3.793488 | Validation Representation DL Loss: 0.207475 | F1 Score: 0.979625 == | Current accuracy: 0.984694 ==
[[193 0 0 0 2 0 0 0 0 0]
[ 1 439 0 0 0 2 0 0 0 0]
[ 0 2 510 4 0 3 0 0 0 0]
[ 0 0 4 137 0 0 0 0 0 0]
[ 1 0 0 0 250 0 0 0 0 0]
[ 0 5 0 0 0 58 0 0 0 0]
[ 0 0 0 0 0 0 20 0 0 0]
[ 0 0 0 2 0 0 0 47 0 0]
[ 0 0 0 0 0 0 0 0 56 0]
[ 0 0 0 0 0 0 0 1 0 27]]
Patience: 0 / 10
== Epoch: 17 | Classification Loss: 0.544836 | Representation DL Loss: 0.210417 | Accuracy: 99.0665% ==
== Epoch: 17 | Validation CLS Loss: 3.774906 | Validation Representation DL Loss: 0.216999 | F1 Score: 0.961109 == | Current accuracy: 0.973356 ==
[[204 0 0 0 2 0 0 0 0 1]
[ 0 453 3 0 2 1 0 0 0 0]
[ 0 3 491 4 0 2 0 0 0 0]
[ 0 0 6 130 0 0 0 0 0 0]
[ 8 0 0 2 231 0 0 0 0 0]
[ 0 6 0 0 0 58 0 0 0 0]
[ 0 0 0 0 0 0 21 0 0 0]
[ 0 0 0 1 0 0 0 51 0 0]
[ 0 0 0 0 0 0 0 0 53 0]
[ 0 0 1 0 0 0 0 5 0 25]]
Patience: 1 / 10
== Epoch: 18 | Classification Loss: 0.553961 | Representation DL Loss: 0.224772 | Accuracy: 98.4866% ==
== Epoch: 18 | Validation CLS Loss: 3.846123 | Validation Representation DL Loss: 0.213684 | F1 Score: 0.949576 == | Current accuracy: 0.965986 ==
[[186 0 0 0 2 0 0 0 0 1]
[ 2 451 3 1 0 0 0 0 0 0]
[ 5 6 499 3 0 1 0 0 0 0]
[ 0 0 3 140 0 0 0 0 0 0]
[ 3 1 0 5 250 0 0 0 0 0]
[ 0 12 4 4 0 29 0 0 0 0]
[ 0 0 0 0 0 0 16 0 0 0]
[ 0 0 0 1 0 0 0 53 0 0]
[ 0 0 0 0 0 0 0 0 45 0]
[ 0 0 2 0 0 0 0 1 0 35]]
Patience: 2 / 10
== Epoch: 19 | Classification Loss: 0.552410 | Representation DL Loss: 0.231168 | Accuracy: 98.6846% ==
== Epoch: 19 | Validation CLS Loss: 3.748894 | Validation Representation DL Loss: 0.238518 | F1 Score: 0.974069 == | Current accuracy: 0.979592 ==
[[202 0 0 0 4 0 0 0 0 1]
[ 1 459 0 0 0 11 0 0 0 1]
[ 1 4 468 2 0 6 0 0 0 0]
[ 0 0 3 129 0 0 0 0 0 0]
[ 0 0 0 0 254 0 0 0 0 0]
[ 0 2 0 0 0 56 0 0 0 0]
[ 0 0 0 0 0 0 21 0 0 0]
[ 0 0 0 0 0 0 0 71 0 0]
[ 0 0 0 0 0 0 0 0 39 0]
[ 0 0 0 0 0 0 0 0 0 29]]
Patience: 3 / 10
== Epoch: 20 | Classification Loss: 0.553717 | Representation DL Loss: 0.244292 | Accuracy: 98.6280% ==
== Epoch: 20 | Validation CLS Loss: 4.026945 | Validation Representation DL Loss: 0.264095 | F1 Score: 0.946933 == | Current accuracy: 0.968254 ==
[[206 0 0 0 0 0 0 0 0 1]
[ 1 470 3 0 1 0 0 0 0 0]
[ 3 3 501 2 0 0 0 0 0 0]
[ 0 0 7 135 0 0 0 0 0 0]
[ 3 0 0 0 216 0 0 0 0 0]
[ 0 15 9 4 0 30 0 0 0 0]
[ 0 0 0 0 0 0 23 0 0 0]
[ 0 0 0 2 0 0 0 45 0 0]
[ 0 0 0 0 0 0 0 0 45 0]
[ 0 0 0 0 0 0 0 2 0 37]]
Patience: 4 / 10
== Epoch: 21 | Classification Loss: 0.551555 | Representation DL Loss: 0.245618 | Accuracy: 98.9109% ==
== Epoch: 21 | Validation CLS Loss: 3.907217 | Validation Representation DL Loss: 0.225754 | F1 Score: 0.967932 == | Current accuracy: 0.974490 ==
[[228 0 0 0 2 0 0 0 0 2]
[ 1 448 4 0 1 1 0 0 0 0]
[ 0 2 466 1 0 2 0 0 0 0]
[ 0 1 13 128 0 0 0 0 0 0]
[ 1 1 0 0 226 0 0 0 0 0]
[ 0 8 0 3 0 56 0 0 0 0]
[ 0 0 0 0 0 0 29 0 0 0]
[ 0 0 0 0 0 0 0 63 0 0]
[ 0 0 0 0 0 0 0 0 50 0]
[ 0 0 2 0 0 0 0 0 0 25]]
Patience: 5 / 10
== Epoch: 22 | Classification Loss: 0.551692 | Representation DL Loss: 0.243921 | Accuracy: 98.7694% ==
== Epoch: 22 | Validation CLS Loss: 3.935475 | Validation Representation DL Loss: 0.259029 | F1 Score: 0.971425 == | Current accuracy: 0.981859 ==
[[211 0 0 0 2 0 0 0 0 1]
[ 0 441 0 0 1 0 0 0 0 0]
[ 0 1 480 5 0 2 0 0 0 0]
[ 0 0 3 132 0 0 0 0 0 0]
[ 1 0 0 1 241 0 0 0 0 0]
[ 0 5 0 6 0 58 0 0 0 0]
[ 0 0 0 0 0 0 35 0 0 0]
[ 0 0 0 0 0 0 0 55 0 0]
[ 0 0 0 0 0 0 0 0 47 0]
[ 0 0 2 0 0 0 0 2 0 32]]
Patience: 6 / 10
== Epoch: 23 | Classification Loss: 0.546653 | Representation DL Loss: 0.238736 | Accuracy: 98.9533% ==
== Epoch: 23 | Validation CLS Loss: 3.684430 | Validation Representation DL Loss: 0.229921 | F1 Score: 0.976328 == | Current accuracy: 0.980726 ==
[[193 0 0 0 2 0 0 0 0 2]
[ 1 444 2 1 2 2 0 0 0 0]
[ 0 3 499 4 0 2 0 0 0 0]
[ 0 0 5 157 0 0 0 0 0 0]
[ 0 0 0 0 234 0 0 0 0 0]
[ 0 5 0 1 0 57 0 0 0 0]
[ 0 0 0 0 0 0 24 0 0 0]
[ 0 0 0 1 0 0 0 55 0 0]
[ 0 0 0 0 0 0 0 0 41 0]
[ 0 0 1 0 0 0 0 0 0 26]]
Patience: 7 / 10
== Epoch: 24 | Classification Loss: 0.548226 | Representation DL Loss: 0.242837 | Accuracy: 98.9675% ==
== Epoch: 24 | Validation CLS Loss: 3.870003 | Validation Representation DL Loss: 0.210899 | F1 Score: 0.983561 == | Current accuracy: 0.985261 ==
[[191 0 0 0 4 0 0 0 0 0]
[ 0 482 0 1 1 1 0 0 0 1]
[ 2 4 494 1 0 3 0 0 0 0]
[ 0 0 2 128 0 0 0 0 0 0]
[ 1 0 0 0 231 0 0 0 0 0]
[ 0 4 0 1 0 55 0 0 0 0]
[ 0 0 0 0 0 0 26 0 0 0]
[ 0 0 0 0 0 0 0 58 0 0]
[ 0 0 0 0 0 0 0 0 39 0]
[ 0 0 0 0 0 0 0 0 0 34]]
Patience: 0 / 10
== Epoch: 25 | Classification Loss: 0.540041 | Representation DL Loss: 0.235430 | Accuracy: 99.1655% ==
== Epoch: 25 | Validation CLS Loss: 3.968665 | Validation Representation DL Loss: 0.217216 | F1 Score: 0.969445 == | Current accuracy: 0.974490 ==
[[202 0 0 0 3 0 0 0 0 2]
[ 2 465 1 0 4 0 0 0 0 0]
[ 0 4 502 5 0 1 0 0 0 0]
[ 0 3 4 135 0 0 0 0 0 0]
[ 1 0 0 0 213 0 0 0 0 0]
[ 0 11 0 2 0 59 0 0 0 0]
[ 0 0 0 0 0 0 20 0 0 0]
[ 0 0 0 0 0 0 0 37 0 0]
[ 0 0 0 0 0 0 0 0 53 0]
[ 0 0 1 0 0 0 0 1 0 33]]
Patience: 1 / 10
== Epoch: 26 | Classification Loss: 0.540302 | Representation DL Loss: 0.230620 | Accuracy: 99.1938% ==
== Epoch: 26 | Validation CLS Loss: 3.945568 | Validation Representation DL Loss: 0.224271 | F1 Score: 0.969770 == | Current accuracy: 0.974490 ==
[[197 0 0 0 2 0 0 0 0 2]
[ 1 449 7 0 1 1 0 0 0 0]
[ 0 2 512 2 0 1 0 0 0 0]
[ 0 0 5 123 0 0 0 0 0 0]
[ 7 0 2 1 231 0 0 0 0 0]
[ 0 4 4 0 0 68 0 0 0 0]
[ 0 0 0 0 0 0 20 0 0 0]
[ 0 0 0 0 0 0 0 51 0 0]
[ 0 0 0 0 0 0 0 0 44 0]
[ 0 0 2 0 0 0 0 1 0 24]]
Patience: 2 / 10
== Epoch: 27 | Classification Loss: 0.536505 | Representation DL Loss: 0.224420 | Accuracy: 99.2362% ==
== Epoch: 27 | Validation CLS Loss: 3.690488 | Validation Representation DL Loss: 0.213966 | F1 Score: 0.980669 == | Current accuracy: 0.982993 ==
[[226 0 0 0 4 0 0 0 0 1]
[ 2 433 3 0 3 0 0 0 0 0]
[ 0 3 492 0 0 3 0 0 0 0]
[ 0 0 0 124 0 0 0 0 0 0]
[ 1 0 0 0 228 0 0 0 0 0]
[ 0 6 1 2 0 72 0 0 0 0]
[ 0 0 0 0 0 0 20 0 0 0]
[ 0 0 0 0 0 0 0 67 0 0]
[ 0 0 0 0 0 0 0 0 46 0]
[ 0 0 0 0 0 0 0 1 0 26]]
Patience: 3 / 10
== Epoch: 28 | Classification Loss: 0.533197 | Representation DL Loss: 0.213238 | Accuracy: 99.3918% ==
== Epoch: 28 | Validation CLS Loss: 4.037175 | Validation Representation DL Loss: 0.199539 | F1 Score: 0.969588 == | Current accuracy: 0.972789 ==
[[220 0 0 0 5 0 0 0 0 2]
[ 3 456 2 2 1 3 0 0 0 0]
[ 0 5 498 6 0 5 0 0 0 0]
[ 0 0 4 126 0 0 0 0 0 0]
[ 1 0 0 0 213 0 0 0 0 0]
[ 0 8 0 1 0 54 0 0 0 0]
[ 0 0 0 0 0 0 13 0 0 0]
[ 0 0 0 0 0 0 0 54 0 0]
[ 0 0 0 0 0 0 0 0 51 0]
[ 0 0 0 0 0 0 0 0 0 31]]
Patience: 4 / 10
== Epoch: 29 | Classification Loss: 0.530652 | Representation DL Loss: 0.209432 | Accuracy: 99.4908% ==
== Epoch: 29 | Validation CLS Loss: 3.959132 | Validation Representation DL Loss: 0.215437 | F1 Score: 0.972856 == | Current accuracy: 0.977324 ==
[[230 0 0 0 2 0 0 0 0 2]
[ 0 455 2 1 1 6 0 0 0 1]
[ 0 4 486 3 0 1 0 0 0 0]
[ 0 0 4 122 0 0 0 0 0 0]
[ 2 0 0 5 206 0 0 0 0 0]
[ 0 4 0 0 0 65 0 0 0 0]
[ 0 0 0 0 0 0 25 0 0 0]
[ 0 0 0 2 0 0 0 59 0 0]
[ 0 0 0 0 0 0 0 0 46 0]
[ 0 0 0 0 0 0 0 0 0 30]]
Patience: 5 / 10
== Epoch: 30 | Classification Loss: 0.530305 | Representation DL Loss: 0.213672 | Accuracy: 99.5332% ==
== Epoch: 30 | Validation CLS Loss: 3.678494 | Validation Representation DL Loss: 0.210690 | F1 Score: 0.969085 == | Current accuracy: 0.973923 ==
[[195 0 0 0 4 0 0 0 0 1]
[ 4 448 0 1 0 0 0 0 0 0]
[ 0 4 483 3 0 3 0 0 0 0]
[ 0 0 5 141 0 0 0 0 0 0]
[ 3 0 0 1 229 0 0 0 0 0]
[ 0 14 0 1 0 59 0 0 0 0]
[ 0 0 0 0 0 0 19 0 0 0]
[ 0 0 0 1 0 0 0 51 0 0]
[ 0 0 0 0 0 0 0 0 60 0]
[ 0 0 0 0 0 0 0 1 0 33]]
Patience: 6 / 10
== Epoch: 31 | Classification Loss: 0.530281 | Representation DL Loss: 0.212276 | Accuracy: 99.5191% ==
== Epoch: 31 | Validation CLS Loss: 3.838703 | Validation Representation DL Loss: 0.209954 | F1 Score: 0.966825 == | Current accuracy: 0.972789 ==
[[237 0 0 0 3 0 0 0 0 3]
[ 2 454 1 2 2 5 0 0 0 1]
[ 0 3 446 7 0 3 0 0 0 0]
[ 0 0 6 127 0 0 0 0 0 0]
[ 1 0 0 0 236 0 0 0 0 0]
[ 0 8 0 0 0 63 0 0 0 0]
[ 0 0 0 0 0 0 18 0 0 0]
[ 0 0 0 0 0 0 0 66 0 0]
[ 0 0 0 0 0 0 0 0 39 0]
[ 0 0 0 0 0 0 0 1 0 30]]
Patience: 7 / 10
== Epoch: 32 | Classification Loss: 0.527409 | Representation DL Loss: 0.206098 | Accuracy: 99.5474% ==
== Epoch: 32 | Validation CLS Loss: 3.758290 | Validation Representation DL Loss: 0.206586 | F1 Score: 0.974758 == | Current accuracy: 0.981293 ==
[[204 0 0 0 4 0 0 0 0 1]
[ 0 451 1 1 1 0 0 0 0 0]
[ 0 4 496 6 0 1 0 0 0 0]
[ 0 0 1 127 0 0 0 0 0 0]
[ 3 0 0 0 251 0 0 0 0 0]
[ 0 6 0 1 0 50 0 0 0 0]
[ 0 0 0 0 0 0 27 0 0 0]
[ 0 0 0 1 0 0 0 45 0 0]
[ 0 0 0 0 0 0 0 0 49 0]
[ 0 0 0 0 0 0 0 2 0 31]]
Patience: 8 / 10
== Epoch: 33 | Classification Loss: 0.527754 | Representation DL Loss: 0.211562 | Accuracy: 99.5191% ==
== Epoch: 33 | Validation CLS Loss: 3.877061 | Validation Representation DL Loss: 0.209572 | F1 Score: 0.980207 == | Current accuracy: 0.982993 ==
[[203 0 0 0 4 0 0 0 0 1]
[ 0 458 0 0 5 0 0 0 0 1]
[ 0 3 470 4 0 2 0 0 0 0]
[ 0 0 3 130 0 0 0 0 0 0]
[ 0 0 0 1 253 0 0 0 0 0]
[ 0 2 0 3 0 61 0 0 0 0]
[ 0 0 0 0 0 0 25 0 0 0]
[ 0 0 0 0 0 0 0 42 0 0]
[ 0 0 0 0 0 0 0 0 51 0]
[ 0 0 0 0 0 0 0 1 0 41]]
Patience: 9 / 10
== Epoch: 34 | Classification Loss: 0.526971 | Representation DL Loss: 0.197932 | Accuracy: 99.5615% ==
== Epoch: 34 | Validation CLS Loss: 3.889331 | Validation Representation DL Loss: 0.185381 | F1 Score: 0.980200 == | Current accuracy: 0.984694 ==
[[187 0 0 0 5 0 0 0 0 1]
[ 1 469 0 0 5 0 0 0 0 0]
[ 0 1 501 0 0 2 0 0 0 0]
[ 0 0 3 144 0 0 0 0 0 0]
[ 0 0 0 0 239 0 0 0 0 0]
[ 0 7 0 1 0 51 0 0 0 0]
[ 0 0 0 0 0 0 20 0 0 0]
[ 0 0 0 1 0 0 0 50 0 0]
[ 0 0 0 0 0 0 0 0 54 0]
[ 0 0 0 0 0 0 0 0 0 22]]
Patience: 10 / 10
== Epoch: 35 | Classification Loss: 0.525790 | Representation DL Loss: 0.193945 | Accuracy: 99.5474% ==
== Epoch: 35 | Validation CLS Loss: 3.863307 | Validation Representation DL Loss: 0.205481 | F1 Score: 0.971715 == | Current accuracy: 0.977324 ==
[[191 0 0 0 1 0 0 0 0 1]
[ 2 452 1 2 3 0 0 0 0 0]
[ 0 4 486 2 0 2 0 0 0 0]
[ 0 0 5 132 0 0 0 0 0 0]
[ 1 0 0 1 237 0 0 0 0 0]
[ 0 10 0 3 0 52 0 0 0 0]
[ 0 0 0 0 0 0 24 0 0 0]
[ 0 0 0 1 0 0 0 68 0 0]
[ 0 0 0 0 0 0 0 0 48 0]
[ 0 0 0 0 0 0 0 1 0 34]]
Patience: 11 / 10
load the unlabeled test data and apply CANAL to predict cell types
[10]:
dataset_test = sc.read_h5ad("./data/{}/{}_test.h5ad".format(experiments, experiments))
print(dataset_test)
pred_cell_type = CANAL.predict(adata_predict = dataset_test, ckpt_dir = './ckpts/', experiments = experiments,
stage_num=3, dataset = dataset_stage_3)
AnnData object with n_obs × n_vars = 2000 × 1000
obs: 'donor', 'cell_type1', 'library', 'organism', 'dataset_name', 'platform', 'organ', 'data_type', 'cell_ontology_class', 'cell_ontology_id', 'n_genes', 'n_counts', '__libsize__', 'baron_human_donor', 'age', 'gender', 'enge_donor', 'batch', 'muraro_donor', 'celltype', 'batch2'
var: 'highly_variable', 'means', 'dispersions', 'dispersions_norm'
== Begin predicting after 3 fine-tuning stages: | Experiments: Pancreas ==
Annotation: ['acinar' 'alpha' 'beta' 'delta' 'ductal' 'gamma' 'mesenchymal'
'activated_stellate' 'endothelial' 'quiescent_stellate']
evaluate the annotation performance
[11]:
print(np.unique(np.array(dataset_test.obs['cell_type1']),return_counts=True))
true_celltype = np.array(dataset_test.obs['cell_type1'])
CANAL.evaluation(pred_cell_type=pred_cell_type, true_celltype=true_celltype)
(array(['acinar', 'activated_stellate', 'alpha', 'beta', 'delta', 'ductal',
'endothelial', 'gamma', 'mesenchymal', 'quiescent_stellate'],
dtype=object), array([234, 16, 784, 415, 133, 295, 15, 67, 30, 11]))
Begin evaluation!
== Predict total accuracy: 0.960500 ==|== F1 Score: 0.946200 ==|== ARI: 0.923800 ==
Confusion matrix:
[[211 0 1 0 0 22 0 0 0 0]
[ 0 14 0 1 0 1 0 0 0 0]
[ 1 0 768 6 5 2 0 2 0 0]
[ 0 0 7 399 5 1 0 2 0 1]
[ 0 0 1 3 129 0 0 0 0 0]
[ 2 0 0 2 0 288 0 0 3 0]
[ 0 0 0 0 0 0 15 0 0 0]
[ 0 0 6 4 0 0 0 57 0 0]
[ 0 0 0 0 0 0 0 0 30 0]
[ 0 0 0 1 0 0 0 0 0 10]]
[ ]: