https://docs.gathr.ai/gathr-unlimited/7.6.0/docs/gen-ai-in-gathr/data-parsing-processing/Recent content in Data Parsing & Processing onHugo -- gohugo.ioSun, 11 Dec 2022 19:37:55 +0530https://docs.gathr.ai/gathr-unlimited/7.6.0/docs/gen-ai-in-gathr/data-parsing-processing/binary-to-text-parser/Tue, 03 Jan 2023 21:01:49 +0530https://docs.gathr.ai/gathr-unlimited/7.6.0/docs/gen-ai-in-gathr/data-parsing-processing/binary-to-text-parser/Convert binary data into readable text, making it understandable and usable for further analysis or processing. Processor Configuration # Configure the processor parameters as explained below. Input Column # Select the column name for which binary data will be parsed. Is Input Base64 Encoded # Specify whether the input data is encoded in Base64 format. This helps the parser correctly interpret and convert the binary data encoded in Base64 into readable text during the parsing process.https://docs.gathr.ai/gathr-unlimited/7.6.0/docs/gen-ai-in-gathr/data-parsing-processing/character-splitter-processor/Sun, 11 Dec 2022 21:01:49 +0530https://docs.gathr.ai/gathr-unlimited/7.6.0/docs/gen-ai-in-gathr/data-parsing-processing/character-splitter-processor/The Character Splitter processor takes text document as input and breaks down the text into smaller chunks based on specific parameters. Below are the configuration details of the processor. Drop Non-Chunked Columns # All columns except for the chunked ones will be dropped from the output. Input Column # Select the column containing the text you want to split. Output Column # Specify the name of the column where the split text chunks will be stored.https://docs.gathr.ai/gathr-unlimited/7.6.0/docs/gen-ai-in-gathr/data-parsing-processing/ai-powered-pdf-parser/Sun, 11 Dec 2022 21:01:49 +0530https://docs.gathr.ai/gathr-unlimited/7.6.0/docs/gen-ai-in-gathr/data-parsing-processing/ai-powered-pdf-parser/The AI Parser Processor in Gathr efficiently extracts and organizes valuable information from PDF documents, enhancing data processing capabilities. Input Configurations # Input configurations for the AI Powered PDF Parser as explained below. Input Column # Provide the column containing the PDF data to be parsed. Base64 Encoded # Select if the input PDF data is encoded in Base64 format. Password-Protected # Select if the PDF is password protected. Fetch Password # Fetch password from either existing column or Inline of the PDF.https://docs.gathr.ai/gathr-unlimited/7.6.0/docs/gen-ai-in-gathr/data-parsing-processing/pdf_parser_processor/Sun, 11 Dec 2022 21:01:49 +0530https://docs.gathr.ai/gathr-unlimited/7.6.0/docs/gen-ai-in-gathr/data-parsing-processing/pdf_parser_processor/The PDF Parser Processor can convert binary PDF data into structured text. It enables seamless sectionization of PDF content, preparing it for integration with AI-powered chunking or storage in Vector Databases. This processor simplifies the extraction of valuable insights and data from PDF documents, facilitating enhanced data processing and analysis of your PDF documents. Input Configuration # This section allows you to configure how the input PDF data is processed. Configure the input configuration parameters as explained below.https://docs.gathr.ai/gathr-unlimited/7.6.0/docs/gen-ai-in-gathr/data-parsing-processing/recursive-splitter-processor/Sun, 11 Dec 2022 21:01:49 +0530https://docs.gathr.ai/gathr-unlimited/7.6.0/docs/gen-ai-in-gathr/data-parsing-processing/recursive-splitter-processor/The Recursive Splitter processor takes text document as input and splits the text progressively into smaller chunks using a list of separators. This approach allows handling various levels of granularity, ensuring the resulting chunks fit within size constraints while preserving context. Below are the configuration details of the processor. Drop Non-Chunked Columns # All columns except for the chunked ones will be dropped from the output. Input Column # Select the column containing the text you want to split.https://docs.gathr.ai/gathr-unlimited/7.6.0/docs/gen-ai-in-gathr/data-parsing-processing/token-splitter-processor/Sun, 11 Dec 2022 21:01:49 +0530https://docs.gathr.ai/gathr-unlimited/7.6.0/docs/gen-ai-in-gathr/data-parsing-processing/token-splitter-processor/The Token Splitter processor takes text document as input and splits the text into smaller chunks that fits within the maximum token limit while preserving the semantic coherence. The AutoTokenizer class from Transformers library (from Hugging Face) is used where you can load any tokenizer and create a tokenizer object which helps in generating the token-based chunks. Below are the configuration details of the processor. Drop Non-Chunked Columns # All columns except for the chunked ones will be dropped from the output.