Enriched Carbon Pretraining Corpus Datasets
Collection
5 items • Updated
record_id stringlengths 11 14 | start int64 0 1.19B | end int64 62 1.19B | embedding list | embedding_norm float32 24.5 258 |
|---|---|---|---|---|
NW_006267348.1 | 11,820 | 13,147 | [
0.0311279296875,
-0.0220947265625,
0.034912109375,
-11.5625,
-0.150390625,
-0.1181640625,
-0.0556640625,
-0.08203125,
-0.115234375,
-0.228515625,
-0.44140625,
-0.09033203125,
0.10595703125,
0.0198974609375,
0.0712890625,
0.06494140625,
-0.158203125,
0.035400390625,
-0.1904296875,... | 125.5 |
NW_006267348.1 | 17,065 | 17,144 | [
-0.0498046875,
0.000896453857421875,
-0.3828125,
-9,
-0.2314453125,
-0.1005859375,
0.111328125,
-0.3359375,
-0.01263427734375,
0.07666015625,
0.0400390625,
-0.1494140625,
0.02880859375,
0.2236328125,
-0.05517578125,
-0.1884765625,
0.072265625,
0.015380859375,
0.0155029296875,
-... | 98.5 |
NW_006267348.1 | 22,199 | 23,626 | [
0.0859375,
0.05126953125,
-0.1923828125,
-17.25,
-0.0908203125,
-0.1845703125,
0.022705078125,
-0.103515625,
0.0947265625,
0.1591796875,
-0.0291748046875,
-0.1875,
0.0498046875,
-0.06494140625,
0.038818359375,
0.054443359375,
-0.1416015625,
0.01300048828125,
-0.08837890625,
-0.... | 181 |
NW_006267348.1 | 23,988 | 25,516 | [
0.099609375,
-0.00286865234375,
0.09130859375,
-18.125,
-0.126953125,
0.0189208984375,
-0.0164794921875,
-0.134765625,
-0.026611328125,
0.031494140625,
-0.09033203125,
-0.287109375,
0.03369140625,
-0.03955078125,
0.2060546875,
0.154296875,
-0.095703125,
-0.0311279296875,
-0.13085... | 193 |
NW_006267348.1 | 28,410 | 31,305 | [0.014404296875,0.0174560546875,-0.045654296875,-14.0,-0.02294921875,-0.061279296875,0.022705078125,(...TRUNCATED) | 154 |
NW_006267348.1 | 33,171 | 34,881 | [0.057373046875,-0.01422119140625,-0.14453125,-14.5625,-0.0947265625,-0.07421875,-0.0022735595703125(...TRUNCATED) | 156 |
NW_006267348.1 | 35,465 | 37,429 | [0.036865234375,-0.001556396484375,-0.0478515625,-15.0,-0.06103515625,-0.029296875,-0.0245361328125,(...TRUNCATED) | 160 |
NW_006267348.1 | 38,487 | 39,015 | [0.056884765625,0.0830078125,-0.1767578125,-21.125,0.04345703125,-0.09716796875,-0.02001953125,-0.10(...TRUNCATED) | 228 |
NW_006267348.1 | 39,287 | 40,951 | [0.06640625,0.158203125,-0.1943359375,-23.125,-0.0703125,0.115234375,-0.0155029296875,0.016235351562(...TRUNCATED) | 247 |
NW_006267348.1 | 41,141 | 42,854 | [0.072265625,0.0703125,0.134765625,-17.25,-0.123046875,0.0712890625,-0.043212890625,-0.02197265625,-(...TRUNCATED) | 184 |
End of preview. Expand in Data Studio
carbon-embeddings
AINovice2005/carbon-embeddings is a derived dataset from the sampled subset of carbon-cpu-enriched-sequences containing dense vector embeddings of biological sequence records.
Each row corresponds to a source sequence identified by record_id. The dataset retains the sequence's position within the processed corpus through start and end and provides a numerical embedding representing the sequence in the embedding model's learned representation space.
The dataset is intended for downstream embedding-based analysis, including similarity search, clustering, retrieval and visualization.
Information of Features
| Feature | Type | Description |
|---|---|---|
record_id |
string | NCBI Identifier linking the embedding to its source biological sequence record. |
start |
int64 | Starting token position of the sequence in the processed/tokenized Carbon corpus. |
end |
int64 | Ending token position of the sequence in the processed/tokenized Carbon corpus. |
embedding |
list | Dense numerical vector generated by the Carbon embedding model to represent the sequence in learned representation space. |
embedding_norm |
float32 | L2 norm of the embedding vector. It is primarily a numerical diagnostic rather than a biological score. |
Dataset Summary
The central field is embedding.
For each biological sequence, the embedding model generates a fixed-dimensional numerical representation. Sequences deemed similar by the model are positioned near one another in the embedding space, depending on the model architecture and the similarity metric applied.
The embedding_norm field records the magnitude of each embedding vector. This value is useful for diagnostics and for assessing whether embedding magnitudes vary systematically across the corpus.
The embedding column stores the numerical components of the sequence embedding as a list. These embeddings should be regarded as model-derived representations rather than direct biological measurements. Individual dimensions generally lack explicit biological meaning unless validated through separate analysis.
For similarity and retrieval tasks, users should apply a distance or similarity metric consistent with the embeddings. Common choices include cosine similarity and Euclidean distance, though the optimal metric depends on how the embeddings were generated and normalized.
The fields start and end indicate the corresponding region within the processed/tokenized Carbon corpus, while record_id provides the primary link between each embedding and its source sequence.
Intended Uses
The dataset can be used for:
- Biological sequence similarity search
- Nearest-neighbor retrieval
- Sequence clustering
- Embedding-space visualization
- Representation-learning experiments
- Retrieval-augmented biological workflows
- Duplicate and near-duplicate analysis
- Dataset exploration and stratification
- Downstream machine-learning models
- Downloads last month
- 394