AI・検知エンジン

토큰화 (Tokenization)

定義

The preprocessing step of splitting text into smaller units called tokens for model input.

詳しく見る

Wikipedia ↗関連ニュース ↗

関連用語

LLM / sLLM 인공신경망 (ANN)스파이킹 신경망 (SNN)검색증강생성 (RAG)에이전트형 AI (Agentic AI)적대적 예제 (Adversarial Example)