Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/arvid-berndtsson/robots-txt-cron

https://github.com/arvid-berndtsson/robots-txt-cron

Science Score: 13.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ○
    .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (7.9%) to scientific vocabulary
Last synced: 10 months ago · JSON representation

Repository

Basic Info
  • Host: GitHub
  • Owner: arvid-berndtsson
  • License: mit
  • Language: TypeScript
  • Default Branch: main
  • Size: 36.1 KB
Statistics
  • Stars: 0
  • Watchers: 1
  • Forks: 0
  • Open Issues: 0
  • Releases: 0
Created over 1 year ago · Last pushed over 1 year ago
Metadata Files
Readme License

README.md

Robots.txt Analyzer Cron Job

A Cloudflare Worker that periodically triggers a cleanup operation via an API endpoint.

Overview

This worker is designed to run as a scheduled task (cron job) that calls a specific API endpoint with authentication. It's built using Cloudflare Workers and TypeScript.

Prerequisites

  • Node.js (latest LTS version recommended)
  • Wrangler CLI (Cloudflare Workers CLI)
  • A Cloudflare account

Setup

  1. Clone the repository: bash git clone https://github.com/arvid-berndtsson/robots-txt-cron.git cd robots-txt-cron

  2. Install dependencies: bash npm install

  3. Configure environment variables:

    • For local development:
      • Copy .dev.vars.example to .dev.vars
      • API_KEY: Your API authentication key
      • BASE_URL: The base URL of your API
    • For production:
      • Go to your GitHub repository settings
      • Navigate to "Settings" → "Secrets and variables" → "Actions"
      • Add "Repository secrets" for each environment variable

Development

To run the worker locally: bash npm run dev

To run tests: bash npm test

To build the project: bash npm run build

Deployment

Deploy to Cloudflare Workers: bash npm run deploy

Environment Variables

| Variable | Description | Required | |----------|-------------|----------| | APIKEY | Authentication key for the API (sensitive - set as repository secret) | Yes | | BASEURL | Base URL of the API endpoint | Yes |

License

MIT

Owner

  • Name: Arvid Berndtsson
  • Login: arvid-berndtsson
  • Kind: user
  • Location: Sweden
  • Company: @LimeTip

Based in Malmö, Sweden, I specialize in software development and digital sustainability. Founder of @LimeTip and Co-Founder of @merely-emissions

GitHub Events

Total
  • Push event: 2
  • Create event: 2
Last Year
  • Push event: 2
  • Create event: 2

Issues and Pull Requests

Last synced: over 1 year ago

All Time
  • Total issues: 0
  • Total pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Total issue authors: 0
  • Total pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 0
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 0
  • Pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
Pull Request Authors
Top Labels
Issue Labels
Pull Request Labels

Dependencies

package-lock.json npm
  • 279 dependencies
package.json npm
  • @cloudflare/vitest-pool-workers ^0.5.2 development
  • @cloudflare/workers-types ^4.20250109.0 development
  • typescript ^5.5.2 development
  • vitest 2.1.8 development
  • wrangler ^3.60.3 development