Marker API

[!IMPORTANT]

Marker API provides a simple endpoint for converting PDF documents to Markdown quickly and accurately. With just one click, you can deploy the Marker API endpoint and start converting PDFs seamlessly.

Features 🚀

• PDF to Markdown Conversion 📄 ➡️ 📝: Converts PDF documents to Markdown efficiently.
• Asynchronous Processing ⏳🔄: Supports both sync and async task processing.
• Distributed Architecture 🌐📡: Enables distributed processing using Celery and Redis.
• Monitoring 📊👁️: Integrated with Flower for monitoring the Celery tasks.
• Scalable 📈🛠️: Easily scalable with distributed workers.
• GPU/CPU Support 💻⚡: Supports both CPU and GPU for intensive processing.
• Multi-Language Support 🌍💬: Handles documents in any language.
• Advanced Formatting 🎨📋: Extracts tables, images, and code blocks accurately.
• Equation Handling ✏️➗: Converts most mathematical equations to LaTeX.
• Docker and Kubernetes Support 🐳🛠️: Full Docker support for both CPU and GPU setups, and Kubernetes (coming soon).

Comparison

| Original PDF | Marker-API | PyPDF | |--------------|------------|-------| | | | |

Here’s an improved version of the features section, along with the comparison table and setup details:

Simple Server vs. Distributed Server

| Criteria | Simple Server | Distributed Server | |----------------------------|---------------------------------------------|------------------------------------------------| | Use Case | Small scale, local deployments | Large scale, production environments | | Scalability | Limited (single server) | Highly scalable with multiple workers | | Performance | Synchronous (one file at a time) | Asynchronous with parallel task processing | | Setup Complexity | Easy (local or Docker setup) | Moderate (multiple services to manage) | | Monitoring | No built-in monitoring | Flower for task monitoring | | Failure Tolerance | Low (single server) | High (distributed worker nodes) | | Docker Support | Yes, CPU and GPU | Yes, CPU and GPU | | Resource Utilization | Limited to local resources | Efficient resource distribution with Celery | | Task Distribution | No task distribution | Uses Celery for task distribution | | Ideal For | Testing, small local deployments | Production environments with high traffic |

Installation and Setup

Simple Server Setup

Local Setup

1. Clone the repository: bash git clone https://github.com/adithya-s-k/marker-api cd marker-api

2. Set up the Python environment: bash cp .env.example .env poetry install # or pip install -e .

3. Run the server: bash python server.py --host 0.0.0.0 --port 8080

Docker Setup (Simple Server)

• For CPU: bash docker build -f docker/Dockerfile.cpu.server -t marker-api-cpu . docker run -p 8080:8080 marker-api-cpu

• For GPU: bash docker build -f docker/Dockerfile.gpu.server -t marker-api-gpu . docker run --gpus all -p 8080:8080 marker-api-gpu

Distributed Server Setup

The distributed server setup uses several services like FastAPI, Redis, Celery workers, and Flower, which are designed for large-scale deployments. You can either set it up locally or use Docker Compose for more seamless scaling.

Local Setup ⚙️

Step 1: Clone the repository and set up the environment

bash git clone https://github.com/adithya-s-k/marker-api cd marker-api cp .env.example .env poetry install # or pip install -e .

Step 2: Run Redis using Docker

Since Redis is essential for Celery to function, let's run Redis in a Docker container. This will ensure it's up and running on port 6379.

bash docker run -d -p 6379:6379 redis

Now, update your .env file with the correct Redis host URL:

bash REDIS_HOST=redis://localhost:6379/0

Step 3: Open three terminals for each service

You will need three separate terminals for Redis, Celery, and FastAPI:

• Terminal 1: Start a Celery worker

  bash celery -A marker_api.celery_worker.celery_app worker --pool=solo --loglevel=info

• Terminal 2: Start Flower for Celery task monitoring

  bash celery -A marker_api.celery_worker.celery_app flower --port=5555

• Terminal 3: Run FastAPI server

  bash python distributed_server.py --host 0.0.0.0 --port 8080

Adding More Celery Workers

To scale Celery workers manually, you can open more terminals and run the same Celery worker command:

bash celery -A marker_api.celery_worker.celery_app worker --pool=solo --loglevel=info

Each new terminal will spin up a new worker, allowing the system to handle more tasks concurrently.

Docker Compose Setup (Distributed Server) 🐳

For an even more seamless setup, you can use Docker Compose, which helps run all services within containers. Here’s how to do it.

Step 1: Start the distributed server with Docker Compose

Use Docker Compose to orchestrate all the services.

• Without Scaling:

  bash sudo docker-compose -f docker-compose.gpu.yml up --build

This command will start:

• FastAPI on port 8080.
• Redis for managing task queue messages.
• Celery Worker to handle background tasks.
• Flower on port 5556 to monitor the Celery tasks.

Step 2: Scale Celery Workers with Docker Compose

To scale up Celery workers, you can use the --scale option:

bash sudo docker-compose -f docker-compose.gpu.yml up --build --scale celery_worker=3

• This command will start 3 Celery workers, increasing the task processing capacity of the system.
• When you use scaling, Docker Compose will automatically spin up multiple instances of the celery_worker service.

Explanation of Scaling in Docker Compose

When running without scaling, you get one instance of the Celery worker service. However, adding the --scale celery_worker=3 flag creates three instances of the worker, meaning tasks will be processed concurrently by three separate workers, which improves the throughput and helps distribute the load across multiple workers.

Kubernetes Support

(Coming Soon)

Why Distributed?

The distributed server architecture offers several advantages over the simple server:

1. Scalability: By using Celery workers, the system can easily scale horizontally by adding more worker instances to handle increased load.

2. Reliability: The distributed system is more resilient to failures. If one worker crashes, others can continue processing tasks.

3. Non-blocking: Asynchronous processing allows the API to handle many requests concurrently without blocking, improving overall throughput.

4. Resource Management: The distributed architecture allows for better management of resources, particularly important for GPU utilization in PDF processing.

5. Monitoring: With tools like Flower, it's easier to monitor the system's performance and identify bottlenecks.

6. Flexibility: The distributed setup allows for more complex workflows and integration with other services.

By using the distributed server, you can process multiple PDFs in parallel, handle high loads more effectively, and provide a more responsive service to your users.

To Do

• [x] Create server
• [x] Add support for single PDF upload
• [x] Add support for multi PDF upload
• [x] Docker support and Skypilot support
• [x] Implement handling for multiple PDF uploads simultaneously.
• [x] Introduce a toggle mode to generate Markdown without including images in the output.
• [x] Enhance GPU utilization and optimize performance for efficient processing.
• [x] Implement dynamic adjustment of batch size based on available VRAM.
• [x] Add celery and flower for distributed queue and monitoring
• [x] Locust for load testing
• [x] Distibuted server for scaling workloads
• [ ] Clinet package init
• [ ] add K8s and helm chart
• [ ] Benchmarks from production load testing

Built With

• Marker
• FastAPI
• Celery
• Redis
• Flower
• Locust
• Streamlit

License

Marker-api is licensed under the GPL-3.0 license. See LICENSE for more information. The project uses Marker under the hood, which has a commercial license that needs to be followed. Here are the details:

Commercial Usage

Marker and Surya OCR Models are designed to be as widely accessible as possible while still funding development and training costs. Research and personal usage are always allowed, but there are some restrictions on commercial usage. The weights for the models are licensed under cc-by-nc-sa-4.0. However, this restriction is waived for any organization with less than $5M USD in gross revenue in the most recent 12-month period AND less than $5M in lifetime VC/angel funding raised. To remove the GPL license requirements (dual-license) and/or use the weights commercially over the revenue limit, check out the options provided. Please refer to Marker for more Information about the License of the Model weights

Acknowledgements

This project is a fork of marker project created by VikParuchuri.