Alchemyst AI - AI-Native Context Management Platform

What You’re Building

An AI that answers questions using YOUR documents instead of guessing. Before Alchemyst:

const answer = await llm.generate("What's our refund policy?");
// "I don't have information about your specific refund policy..." ❌

After Alchemyst:

const answer = await llm.generate("What's our refund policy?");
// "We offer a 30-day money back guarantee. Contact support@example.com..." ✅

Prerequisites

Alchemyst account (sign up)
Your ALCHEMYST_AI_API_KEY
Node.js 18+ or Python 3.9+

Time to complete: 10 minutes

Step 1: Install

npm install @alchemystai/sdk

Step 2: Initialize

import AlchemystAI from '@alchemystai/sdk';

const client = new AlchemystAI({
  apiKey: process.env.ALCHEMYST_AI_API_KEY,
});

Step 3: Store a Document

const result = await client.v1.context.add({
  documents: [{
    content: "Our refund policy: We offer a 30-day money back guarantee. Contact support@example.com to request a refund."
  }],
  context_type: 'resource',
  source: 'documentation',
  scope: 'internal'
});

console.log(`✅ Stored ${result.documents?.length || 0} documents`);
// Output: ✅ Stored 1 documents

What just happened:

Your document was chunked into searchable pieces (~500 word chunks)
Embeddings were generated automatically (vector representations)
Everything is indexed and ready to search

Verify it worked:

Visit platform.getalchemystai.com/context
You should see your document listed
Click to view chunks and metadata

Step 4: Search for Context

const userQuestion = "What's your refund policy?";

const { contexts } = await client.v1.context.search({
  query: userQuestion,
  similarity_threshold: 0.7,
  scope: 'internal'
});

console.log(`Found ${contexts?.length || 0} relevant chunks`);
// Output: Found 1 relevant chunks

console.log(contexts[0].content);
// Output: "Our refund policy: We offer a 30-day money back guarantee..."

Understanding similarity_threshold

Value	Meaning	When to Use
`0.5`	Somewhat relevant	Exploratory searches, broad topics
`0.7`	Relevant	Start here - good balance
`0.9`	Very relevant	Precise matches, technical queries

Tip: Start at 0.7, lower to 0.5 if you get no results, raise to 0.9 if results are too broad.

Step 5: Feed Context to Your LLM

import OpenAI from 'openai';

const openai = new OpenAI();

const prompt = contexts?.length 
  ? `Context:\n${contexts.map(c => c.content).join('\n\n')}\n\nQuestion: ${userQuestion}`
  : userQuestion;

const response = await openai.chat.completions.create({
  model: "gpt-4",
  messages: [{ role: "user", content: prompt }]
});

console.log(response.choices[0].message.content);
// Output: "We offer a 30-day money back guarantee. To request a refund, contact support@example.com."

Result: Your AI now answers from YOUR data, not generic training data.

Complete Working Example

TypeScript
Python

import AlchemystAI from '@alchemystai/sdk';
import OpenAI from 'openai';

const alchemyst = new AlchemystAI({
  apiKey: process.env.ALCHEMYST_AI_API_KEY,
});

const openai = new OpenAI();

async function main() {
  // 1. Store document
  await alchemyst.v1.context.add({
    documents: [{
      content: "Our refund policy: We offer a 30-day money back guarantee. Contact support@example.com to request a refund."
    }],
    context_type: 'resource',
    source: 'docs',
    scope: 'internal'
  });
  console.log("✅ Document stored");

  // 2. Search for context
  const userQuestion = "What's your refund policy?";
  const { contexts } = await alchemyst.v1.context.search({
    query: userQuestion,
    similarity_threshold: 0.7,
    scope: 'internal'
  });
  console.log(`Found ${contexts?.length} relevant chunks`);

  // 3. Generate answer with context
  const prompt = contexts?.length
    ? `Context:\n${contexts.map(c => c.content).join('\n\n')}\n\nQuestion: ${userQuestion}`
    : userQuestion;

  const response = await openai.chat.completions.create({
    model: "gpt-4",
    messages: [{ role: "user", content: prompt }]
  });

  console.log("AI Response:", response.choices[0].message.content);
}

main();

Expected Output:

✅ Document stored
Found 1 relevant chunks
AI Response: We offer a 30-day money back guarantee. To request a refund, contact support@example.com.

import os
from alchemyst_ai import AlchemystAI
import openai

alchemyst = AlchemystAI(api_key=os.environ.get("ALCHEMYST_AI_API_KEY"))

def main():
    # 1. Store document
    alchemyst.v1.context.add(
        documents=[{
            "content": "Our refund policy: We offer a 30-day money back guarantee. Contact support@example.com to request a refund."
        }],
        context_type="resource",
        source="docs",
        scope="internal"
    )
    print("✅ Document stored")

    # 2. Search for context
    user_question = "What's your refund policy?"
    result = alchemyst.v1.context.search(
        query=user_question,
        similarity_threshold=0.7,
        scope="internal"
    )

    contexts = result.contexts or []
    print(f"Found {len(contexts)} relevant chunks")

    # 3. Generate answer with context
    if contexts:
        context_text = "\n\n".join([ctx.content for ctx in contexts])
        prompt = f"Context:\n{context_text}\n\nQuestion: {user_question}"
    else:
        prompt = user_question

    response = openai.chat.completions.create(
        model="gpt-4",
        messages=[{"role": "user", "content": prompt}]
    )

    print("AI Response:", response.choices[0].message.content)

if __name__ == "__main__":
    main()

Expected Output:

✅ Document stored
Found 1 relevant chunks
AI Response: We offer a 30-day money back guarantee. To request a refund, contact support@example.com.

Next: Add Real Documents

Now that you understand the basics, let’s add real documents:

From File
From API
Multiple Files

import fs from 'fs';

const content = fs.readFileSync('./docs/policy.txt', 'utf-8');

await client.v1.context.add({
  documents: [{
    content: content,
    metadata: {
      file_name: "policy.txt",
      file_type: "text"
    }
  }],
  context_type: 'resource',
  source: 'documentation',
  scope: 'internal'
});

console.log("✅ File uploaded and indexed");

const response = await fetch('https://api.example.com/docs');
const apiDocs = await response.json();

const docs = apiDocs.map(doc => ({
  content: doc.body,
  metadata: {
    file_name: doc.title,
    doc_id: doc.id
  }
}));

await client.v1.context.add({
  documents: docs,
  context_type: 'resource',
  source: 'api',
  scope: 'internal'
});

console.log(`✅ Indexed ${docs.length} documents from API`);

const docs = [
  {
    content: "Refund policy: We offer a 30-day money back guarantee...",
    metadata: { file_name: "refunds.md" }
  },
  {
    content: "Shipping policy: We ship worldwide within 5-7 business days...",
    metadata: { file_name: "shipping.md" }
  },
  {
    content: "Privacy policy: We collect and protect your data...",
    metadata: { file_name: "privacy.md" }
  }
];

await client.v1.context.add({
  documents: docs,
  context_type: 'resource',
  source: 'documentation',
  scope: 'internal'
});

console.log(`✅ Indexed ${docs.length} policy documents`);

Advanced: Organize with Metadata

Use group_name to filter searches by category:

// Store with groups
await client.v1.context.add({
  documents: [{
    content: "Refund policy: We offer a 30-day money back guarantee...",
    metadata: {
      file_name: "refund-policy.md",
      group_name: ["customer-support", "policies"]  // ← For storage
    }
  }],
  context_type: 'resource',
  source: 'documentation',
  scope: 'internal'
});

// Search within specific groups
const { contexts } = await client.v1.context.search({
  query: "refund policy",
  scope: 'internal',
  metadata: {
    groupName: ['customer-support']  // ← For search (note: camelCase)
  }
});

console.log(`Found ${contexts?.length} results in customer-support`);

Naming difference: Storage uses group_name (snake_case in metadata) but TypeScript search uses groupName (camelCase). Python uses group_name consistently. Both refer to the same field - this is due to API design conventions.

Why use groups?

Benefit	Description
Faster searches	Smaller search space = lower latency
More relevant	Only search customer-support docs, not engineering docs
Better organization	Hierarchical structure like folders
Access control	Filter by team, project, or user permissions

Example hierarchy:

group_name: ["company", "customer-support", "policies"]
group_name: ["company", "engineering", "api-docs"]
group_name: ["company", "hr", "employee-handbook"]

Learn more: Context Arithmetic

Troubleshooting

No results from search

Error symptoms:

contexts?.length returns 0
Empty array in search results

Common causes:

similarity_threshold is too high
Document wasn’t stored successfully
scope mismatch between add and search

Fixes:Step 1: Lower the threshold:

similarity_threshold: 0.5  // Instead of 0.9

Step 2: Verify documents were stored:

const stored = await client.v1.context.view();
console.log("Total documents:", stored.length);
console.log("Documents:", stored);

Expected output:

Total documents: 1
Documents: [{ id: "doc_123", content: "Our refund policy...", ... }]

Step 3: Check scope matches:

// Both must use the same scope
await client.v1.context.add({ scope: 'internal', ... });
await client.v1.context.search({ scope: 'internal', ... });  // ✅ Match

Too many irrelevant results

Error symptoms:

Getting documents that don’t match the query
Too many results to process efficiently

Common causes:

similarity_threshold is too low
Documents are too broad or generic
Not using group filtering

Fixes:Step 1: Raise the threshold:

similarity_threshold: 0.8  // More strict (was 0.5)

Step 2: Use group filtering:

metadata: { 
  groupName: ['customer-support']  // Narrow to specific category
}

Step 3: Add more specific metadata:

await client.v1.context.add({
  documents: [{
    content: "...",
    metadata: {
      group_name: ["support", "refunds"],  // More specific
      category: "policies",
      department: "customer-service"
    }
  }]
});

Document already exists error

Error message:

{
  "error": "Document with file_name 'policy.md' already exists",
  "code": "CONFLICT",
  "status": 409
}

Cause: Trying to add a document with duplicate file_name in metadata.Fixes:Step 1: Delete old version first:

// Delete by file name
await client.v1.context.delete({ 
  metadata: { fileName: "policy.md" } 
});

// Then add new version
await client.v1.context.add({
  documents: [{
    content: "Updated content...",
    metadata: { file_name: "policy.md" }
  }]
});

Step 2: Use versioned names:

file_name: "policy-v2.md"  // or "policy-2024-02-01.md"

Step 3: Use unique identifiers:

file_name: `policy-${Date.now()}.md`  // Timestamp
// or
file_name: `policy-${uuid()}.md`       // UUID

API authentication errors

Error message:

{
  "error": "Invalid API key",
  "code": "UNAUTHORIZED",
  "status": 401
}

Fixes:Step 1: Verify API key is set:

console.log("API Key exists:", !!process.env.ALCHEMYST_AI_API_KEY);
// Should output: API Key exists: true

Step 2: Check key format:

// Should start with "alch_" or similar prefix
console.log("Key prefix:", process.env.ALCHEMYST_AI_API_KEY?.substring(0, 5));

Step 3: Get a new key:

Visit platform.getalchemystai.com/settings
Generate a new API key
Update your .env file

Rate limit exceeded

Error message:

{
  "error": "Rate limit exceeded",
  "code": "RATE_LIMIT",
  "status": 429,
  "retry_after": 60
}

Cause: Too many requests in a short time period.Fixes:Step 1: Add retry logic:

async function searchWithRetry(query: string, retries = 3) {
  for (let i = 0; i < retries; i++) {
    try {
      return await client.v1.context.search({ query, scope: 'internal' });
    } catch (error) {
      if (error.status === 429 && i < retries - 1) {
        const waitTime = error.retry_after || 60;
        console.log(`Rate limited. Waiting ${waitTime}s...`);
        await new Promise(resolve => setTimeout(resolve, waitTime * 1000));
      } else {
        throw error;
      }
    }
  }
}

Step 2: Check your rate limits:

Free tier: 100 operations/day
Pro tier: 10,000 operations/day

Step 3: Upgrade your plan: View pricing

Best Practices

Document Organization

// ✅ Good - organized with meaningful metadata
await client.v1.context.add({
  documents: [{
    content: "...",
    metadata: {
      file_name: "refund-policy-2024-02.md",
      group_name: ["customer-support", "policies"],
      department: "support",
      last_updated: "2024-02-01"
    }
  }]
});

// ❌ Bad - minimal metadata, hard to search
await client.v1.context.add({
  documents: [{
    content: "..."
  }]
});

Chunk Size Considerations

Small documents (under 1000 words): Add as-is
Medium documents (1000-5000 words): Let Alchemyst auto-chunk
Large documents (over 5000 words): Consider splitting by section

// For large documents, split manually for better control
const sections = [
  { title: "Introduction", content: "..." },
  { title: "Features", content: "..." },
  { title: "Pricing", content: "..." }
];

const docs = sections.map(section => ({
  content: section.content,
  metadata: {
    file_name: "product-docs.md",
    section: section.title
  }
}));

await client.v1.context.add({ documents: docs });

Search Optimization

// ✅ Good - specific query with filtering
const { contexts } = await client.v1.context.search({
  query: "How to request a refund for damaged items",
  scope: 'internal',
  similarity_threshold: 0.7,
  metadata: {
    groupName: ['customer-support', 'refunds']
  }
});

// ❌ Bad - too broad, no filtering
const { contexts } = await client.v1.context.search({
  query: "help",
  scope: 'internal'
});

Error Handling

async function safeSearch(query: string) {
  try {
    const { contexts } = await client.v1.context.search({
      query,
      scope: 'internal',
      similarity_threshold: 0.7
    });
    
    if (!contexts || contexts.length === 0) {
      console.log("No results found. Try a broader query.");
      return null;
    }
    
    return contexts;
  } catch (error) {
    console.error("Search failed:", error.message);
    // Fallback or retry logic here
    return null;
  }
}

Verify Your Setup

After implementing context search, verify everything is working:

Check Platform UI

Visit platform.getalchemystai.com/context
You should see your stored documents
Click to view chunks and embeddings
Check document count matches what you uploaded

Test with Code

// Test the complete flow
async function testSetup() {
  // 1. Store a test document
  console.log("Step 1: Storing test document...");
  await client.v1.context.add({
    documents: [{
      content: "Test document: The answer is 42",
      metadata: { file_name: "test.txt" }
    }],
    scope: 'internal'
  });
  
  // 2. Search for it
  console.log("Step 2: Searching for test document...");
  const { contexts } = await client.v1.context.search({
    query: "What is the answer?",
    scope: 'internal',
    similarity_threshold: 0.5
  });
  
  // 3. Verify results
  const found = contexts?.some(c => c.content.includes("42"));
  console.log("✅ Test passed:", found);
  
  // 4. Cleanup
  console.log("Step 3: Cleaning up...");
  await client.v1.context.delete({
    metadata: { fileName: "test.txt" }
  });
}

testSetup();

Expected Output:

Step 1: Storing test document...
Step 2: Searching for test document...
✅ Test passed: true
Step 3: Cleaning up...

What’s Next?

Add Memory

Make your AI remember users across sessions

TypeScript SDK

Complete TypeScript SDK reference

Python SDK

Complete Python SDK reference

API Reference

REST API documentation

Learn Advanced Patterns

Context Arithmetic

Master advanced filtering and organization

Usage Patterns

Best practices and anti-patterns

Real-World Use Cases

Customer support, code assistants, and more

Sample Projects

Community-built projects

Need Help?

Discord Community

Get real-time help from our community

Documentation

Browse guides and tutorials

Email Support

Contact our support team

GitHub Issues

Report bugs or request features

Getting started

Tutorials

Migration Guides

Enterprise

Advanced Usage

​What You’re Building

​Prerequisites

​Step 1: Install

​Step 2: Initialize

​Step 3: Store a Document

​Step 4: Search for Context

​Understanding similarity_threshold

​Step 5: Feed Context to Your LLM

​Complete Working Example

​Next: Add Real Documents

​Advanced: Organize with Metadata

​Why use groups?

​Troubleshooting

​Best Practices

​Document Organization

​Chunk Size Considerations

​Search Optimization

​Error Handling

​Verify Your Setup

​Check Platform UI

​Test with Code

​What’s Next?

Add Memory

TypeScript SDK

Python SDK

API Reference

​Learn Advanced Patterns

Context Arithmetic

Usage Patterns

Real-World Use Cases

Sample Projects

​Need Help?

Discord Community

Documentation

Email Support

GitHub Issues

What You’re Building

Prerequisites

Step 1: Install

Step 2: Initialize

Step 3: Store a Document

Step 4: Search for Context

Understanding similarity_threshold

Step 5: Feed Context to Your LLM

Complete Working Example

Next: Add Real Documents

Advanced: Organize with Metadata

Why use groups?

Troubleshooting

Best Practices

Document Organization

Chunk Size Considerations

Search Optimization

Error Handling

Verify Your Setup

Check Platform UI

Test with Code

What’s Next?

Learn Advanced Patterns

Need Help?