Analytics
MokaHR robots.txt and llms.txt Optimization Analysis

MokaHR robots.txt and llms.txt Optimization Analysis

Structured analysis and enhancement proposals for robots.txt and llms.txt on https://www.mokahr.io, with actionable SEO and AI access recommendations.

Banner image for MokaHR robots.txt and llms.txt analysis

1. Analysis by Reference

Reference 1: robots.txt Analysis

  • Current Status:
    https://www.mokahr.io/robots.txt is present but lacks user-agent and Allow/Disallow rules.
  • Issues Detected:
    • No User-agent, Allow, Disallow statements.
    • No sitemap listed.
    • Contains legal notes, which are good, but should be commented.
  • SEO/AI Optimization Suggestions:
    • Specify which bots may access what.
    • Disallow sensitive/internal directories.
    • Publish a sitemap.
    • Keep legal notes as top-of-file comments.

Reference 2: llms.txt Analysis

  • Current Status:
    https://www.mokahr.io/llms.txt is missing (HTTP 404).
  • Issues Detected:
    • File is completely missing, making it inaccessible to LLM-based tools/bots.
  • AI Optimization Suggestions:
    • Place at the domain root.
    • Use a clearly structured format (JSON, YAML, or structured TXT).
    • Provide relevant metadata, feature highlights, and update times.
    • Add contact/support information.

2. Optimized Example Files (If Missing or For Improvement)

A. robots.txt (place at https://www.mokahr.io/robots.txt)

# Legal Notes:
# This site content is for personal use only. Unauthorized scraping, AI training or data mining is prohibited.
# All access and usage are subject to our Terms of Service.

User-agent: *
Disallow: /private/
Disallow: /admin/
Disallow: /tmp/
Allow: /

User-agent: GPTBot
Disallow: /

User-agent: Googlebot
Allow: /

Sitemap: https://www.mokahr.io/sitemap.xml

You can customize /private/, /admin/, etc., according to your actual sensitive folders.

B. llms.txt (place at https://www.mokahr.io/llms.txt)

{
  "name": "MokaHR Talent Management Suite",
  "description": "AI-powered, end-to-end talent management platform. Features AI resume parsing, candidate ranking, interview scheduling, and automated communication.",
  "features": [
    "AI resume screening",
    "Automated candidate ranking",
    "Interview scheduling",
    "Data-driven analytics"
  ],
  "api_endpoint": "https://api.mokahr.io/v1/",
  "documentation": "https://docs.mokahr.io/llms",
  "last_updated": "2024-06-10",
  "contact": "[email protected]",
  "language": "en",
  "category": ["HR", "AI", "SaaS", "Talent Management"],
  "privacy_policy": "https://www.mokahr.io/privacy-policy"
}

You may extend with further metadata as needed.

3. Summary Table

File/URL Status Key Actions
/robots.txt Present Add User-agent rules, Disallow sensitive areas, publish sitemap, keep legal notes as top comments.
/llms.txt Missing Create with clear metadata, company description, features, API/docs links, last updated date, contact & policy.

4. Deployment Instructions

  • Place both files at the domain root:
    • https://www.mokahr.io/robots.txt
    • https://www.mokahr.io/llms.txt
  • Use the provided examples or customize as needed to reflect your actual directory structure, feature set, and policies.

If you provide these optimized files at your domain root, both search engines and LLM-driven bots will have clear, up-to-date access and usage guidance, improving SEO and AI integration outcomes.

© 2025 Frevana