add tests and readme

README.md

@@ -9,6 +9,210 @@ Fast Rust library for PDF to Markdown conversion with smart scanned vs text-base
 - **Structure Detection** - Headers (by font size), lists, code blocks (monospace fonts)
 - **CLI Tools** - `detect-pdf` and `pdf2md` binaries included
 
+## Installation
+
+Add to your `Cargo.toml`:
+
+```toml
+[dependencies]
+pdf-to-markdown = "0.1"
+```
+
+## Usage
+
+### Quick Start
+
+The simplest way to convert a PDF to Markdown:
+
+```rust
+use pdf_to_markdown::process_pdf;
+
+fn main() -> Result<(), Box<dyn std::error::Error>> {
+    let result = process_pdf("document.pdf")?;
+
+    match result.pdf_type {
+        pdf_to_markdown::PdfType::TextBased => {
+            println!("Markdown:\n{}", result.markdown.unwrap());
+        }
+        pdf_to_markdown::PdfType::Scanned => {
+            println!("PDF is scanned - OCR required");
+        }
+        _ => {}
+    }
+
+    Ok(())
+}
+```
+
+### PDF Type Detection
+
+Quickly detect if a PDF is text-based or scanned without full extraction:
+
+```rust
+use pdf_to_markdown::{detect_pdf_type, PdfType};
+
+fn main() -> Result<(), Box<dyn std::error::Error>> {
+    let result = detect_pdf_type("document.pdf")?;
+
+    println!("Type: {:?}", result.pdf_type);
+    println!("Pages: {}", result.page_count);
+    println!("Confidence: {:.0}%", result.confidence * 100.0);
+
+    if let Some(title) = result.title {
+        println!("Title: {}", title);
+    }
+
+    match result.pdf_type {
+        PdfType::TextBased => println!("Ready for text extraction"),
+        PdfType::Scanned => println!("Needs OCR"),
+        PdfType::ImageBased => println!("Mostly images"),
+        PdfType::Mixed => println!("Mix of text and images"),
+    }
+
+    Ok(())
+}
+```
+
+### Text Extraction
+
+Extract plain text from a PDF:
+
+```rust
+use pdf_to_markdown::extract_text;
+
+fn main() -> Result<(), Box<dyn std::error::Error>> {
+    let text = extract_text("document.pdf")?;
+    println!("{}", text);
+    Ok(())
+}
+```
+
+### Extract Text with Position Information
+
+Get text items with position data for advanced processing:
+
+```rust
+use pdf_to_markdown::{extract_text_with_positions, TextItem};
+use pdf_to_markdown::extractor::group_into_lines;
+
+fn main() -> Result<(), Box<dyn std::error::Error>> {
+    let items = extract_text_with_positions("document.pdf")?;
+
+    for item in &items {
+        println!("'{}' at ({}, {}) size={}",
+            item.text, item.x, item.y, item.font_size);
+    }
+
+    // Group items into lines
+    let lines = group_into_lines(items);
+    for line in lines {
+        println!("Line: {}", line.text());
+    }
+
+    Ok(())
+}
+```
+
+### Custom Markdown Conversion
+
+Convert text to Markdown with custom options:
+
+```rust
+use pdf_to_markdown::{to_markdown, MarkdownOptions};
+
+fn main() {
+    let text = "• First item\n• Second item\n\nconst x = 5;";
+
+    // With all detection enabled (default)
+    let md = to_markdown(text, MarkdownOptions::default());
+    println!("{}", md);
+
+    // Disable code detection
+    let opts = MarkdownOptions {
+        detect_headers: true,
+        detect_lists: true,
+        detect_code: false,
+        base_font_size: None,
+    };
+    let md = to_markdown(text, opts);
+    println!("{}", md);
+}
+```
+
+### Processing from Memory
+
+All functions have memory buffer variants for processing PDFs already in memory:
+
+```rust
+use pdf_to_markdown::{process_pdf_mem, detector::detect_pdf_type_mem};
+use pdf_to_markdown::extractor::{extract_text_mem, extract_text_with_positions_mem};
+
+fn main() -> Result<(), Box<dyn std::error::Error>> {
+    let buffer = std::fs::read("document.pdf")?;
+
+    // Process from memory
+    let result = process_pdf_mem(&buffer)?;
+
+    // Or detect only
+    let detection = detect_pdf_type_mem(&buffer)?;
+
+    // Or extract text
+    let text = extract_text_mem(&buffer)?;
+
+    Ok(())
+}
+```
+
+### Custom Detection Configuration
+
+Fine-tune the detection algorithm:
+
+```rust
+use pdf_to_markdown::detector::{detect_pdf_type_with_config, DetectionConfig};
+
+fn main() -> Result<(), Box<dyn std::error::Error>> {
+    let config = DetectionConfig {
+        max_pages_to_sample: 10,        // Sample more pages
+        min_text_ops_per_page: 5,       // Require more text operators
+        text_page_ratio_threshold: 0.8, // Stricter text classification
+    };
+
+    let result = detect_pdf_type_with_config("document.pdf", config)?;
+    println!("{:?}", result.pdf_type);
+
+    Ok(())
+}
+```
+
+## CLI Tools
+
+### pdf2md
+
+Convert a PDF to Markdown:
+
+```bash
+# Output to stdout
+pdf2md document.pdf
+
+# Output to file
+pdf2md document.pdf output.md
+
+# JSON output with metadata
+pdf2md document.pdf --json
+```
+
+### detect-pdf
+
+Detect PDF type without conversion:
+
+```bash
+# Human-readable output
+detect-pdf document.pdf
+
+# JSON output
+detect-pdf document.pdf --json
+```
+
 ## How Detection Works
 
 Instead of loading the entire PDF, we:
@@ -19,3 +223,35 @@ Instead of loading the entire PDF, we:
 4. Classify based on text operator presence
 
 This allows detecting 300+ page PDFs in milliseconds.
+
+## API Reference
+
+### Types
+
+| Type | Description |
+|------|-------------|
+| `PdfType` | Enum: `TextBased`, `Scanned`, `ImageBased`, `Mixed` |
+| `PdfProcessResult` | Full processing result with text, markdown, and metadata |
+| `PdfTypeResult` | Detection result with type, confidence, and page count |
+| `TextItem` | Text with position (x, y), font info, and page number |
+| `TextLine` | Group of `TextItem`s on the same line |
+| `MarkdownOptions` | Configuration for markdown conversion |
+| `DetectionConfig` | Configuration for PDF type detection |
+| `PdfError` | Error type: `Io`, `Parse`, `Encrypted`, `InvalidStructure` |
+
+### Functions
+
+| Function | Description |
+|----------|-------------|
+| `process_pdf(path)` | High-level: detect, extract, and convert |
+| `process_pdf_mem(buffer)` | Same as above, from memory |
+| `detect_pdf_type(path)` | Fast type detection |
+| `detect_pdf_type_mem(buffer)` | Type detection from memory |
+| `extract_text(path)` | Extract plain text |
+| `extract_text_mem(buffer)` | Extract text from memory |
+| `extract_text_with_positions(path)` | Extract text with coordinates |
+| `to_markdown(text, options)` | Convert text to markdown |
+
+## License
+
+MIT