Museum/L1B3RT4S
1
0
Fork 0
mirror of https://github.com/elder-plinius/L1B3RT4S.git synced 2026-07-10 05:30:03 +03:00
Code Issues Packages Projects Releases Wiki Activity

Update CLOSEDAI.mkd

Browse Source
This commit is contained in:
pliny
2024-09-15 23:23:05 -04:00
committed by GitHub
parent ce4850d8b9
commit b4ae7aa21d
1 changed files with 6 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CLOSEDAI.mkd
+6 -6
View File
@@ -1,8 +1,8 @@
Special Tokens Special Tokens
**\<SOS> (Start of Sequence)**: Marks the beginning of a sequence for the model to start processing. - **\<SOS> (Start of Sequence)**: Marks the beginning of a sequence for the model to start processing.
**\<EOS> (End of Sequence)**: Tells the model when to stop generating text or processing. - **\<EOS> (End of Sequence)**: Tells the model when to stop generating text or processing.
**\<PAD> (Padding Token)**: Pads sequences to the same length for batch processing. - **\<PAD> (Padding Token)**: Pads sequences to the same length for batch processing.
**\<UNK> (Unknown Token)**: Represents words not in the model's vocabulary. - **\<UNK> (Unknown Token)**: Represents words not in the model's vocabulary.
**\<MASK> (Mask Token)**: Used in tasks like predicting missing words in masked language models. - **\<MASK> (Mask Token)**: Used in tasks like predicting missing words in masked language models.
**\<SEP> (Separator Token)**: Separates different segments in input, like questions from context. - **\<SEP> (Separator Token)**: Separates different segments in input, like questions from context.