<img height="1" width="1" style="display:none;" alt="" src="https://px.ads.linkedin.com/collect/?pid=1110556&amp;fmt=gif">
Skip to content
    September 16, 2025

    Powering RAG Pipelines With Automic Automation

    Key Takeaways
    • Find out how retrieval-augmented generation (RAG) improves generative AI output and reduces token use.
    • Discover how Automic Automation enables teams to automate RAG pipelines.
    • See how Automic Automation prepares customers for the AI revolution, combining cutting-edge technologies with proven capabilities.

    Technical developments surrounding generative artificial intelligence (GenAI) are advancing at blazing speed and are beginning to change our world for the better. This makes it all the more important to recognize the potential of this technology and leverage it sensibly to increase your competitiveness. Automic Automation is an automation platform that optimally equips you for the AI revolution—combining proven enterprise capabilities with the mind-blowing potential of GenAI.

    Automation platforms have always been an indispensable enabler of digital transformation and, as such, are ideally suited to channeling AI into orderly processes and harnessing it effectively. Automic Automation and GenAI prove to be an ideal combination on several levels: GenAI provides Automic users with helpful support in implementing new automation projects and it facilitates the smooth operation of automation processes already running in production. In addition, GenAI can also function as an integral part of automation workflows, enriching deterministic tasks with intelligent decision-making and enabling the development of entire workflows as AI agents.

    How retrieval-augmented generation (RAG) improves GenAI output

    Despite the rapid development of new large language models (LLMs), their inherent knowledge is limited at the time of their release. For example, an LLM published a week ago knows nothing about the new scientific study published yesterday or the latest version of a REST API released just the day before. Nor would it have visibility into the recently published knowledge base article or the latest update to the corporate travel policy. To harness the potential of GenAI, such information must be integrated into the context of a conversation with an LLM. The size of the context itself, the so-called context window, is also more or less limited depending on the model chosen. Although the technical limits are increasingly being expanded, the number of tokens continues to play at least a commercial role as a common metric for billing for corresponding services.

    A common approach to solving this problem is called "retrieval-augmented generation" (RAG). This method is used to identify relevant information from specific knowledge sources (retrieval), which is then integrated into the context of a query to an LLM (augmentation), thus enabling the LLM to produce a more accurate answer (generation). Instead of integrating all knowledge base articles into the context, the relevant parts of selected knowledge base articles are first identified based on the query to the LLM, and only these are used. By including only the truly relevant information in the request, the number of tokens consumed can be drastically reduced. Further, the accuracy and relevance of the response can also be significantly improved, and the likelihood of hallucinations can be reduced.

    ESD_FY25_Academy-Blog.Powering RAG Pipelines With Automic Automation.Featured NEW

    Automic Automation fuels RAG

    But how does such a system work in practice? Let's imagine the following scenario: At regular intervals, the contents of new documents must be transferred to a vector database, enabling users to employ an LLM to access the contents of their company documents. Before this, a few additional steps are usually necessary. For example, the text must first be extracted from PDF documents and then broken down into smaller pieces using special splitter algorithms, which can take overlaps into account to maintain semantic coherence. In the next step, vector representations of the individual text blocks are generated using a so-called embedding model. These blocks are then stored in a vector database. Upon closer inspection, this is a classic use case for automation. And indeed, such a "RAG pipeline" can be implemented quite easily with Automic Automation.

    ESD_FY25_Academy-Blog.Powering RAG Pipelines With Automic Automation.Figure-1

    Automic’s low-code approach makes your life easier

    First, you need to identify the data source and check the Automation Marketplace for the availability of a suitable integration with Automic Automation. Broadcom offers a wide range of integrations with third-party applications, which makes the creation of automated processes much easier, as they eliminate the need for specialized knowledge of third-party APIs.

    With the help of a suitable integration, the desired documents can be downloaded. Further processing of these documents is most easily accomplished with Python. In effect, there’s no way for data scientists to get around Python. This reality is reflected in the large selection of useful libraries. Automic Automation offers a new job type for this, the Python job. With this, Python becomes a first-class citizen. Automic Automation developers can rely on Python and popular libraries, such as Apache Tika and Langchain, to implement those parts of the RAG pipeline that are responsible for extracting text from the PDFs and splitting it into overlapping text blocks.

    ESD_FY25_Academy-Blog.Powering RAG Pipelines With Automic Automation.Figure-2

    Automic REST jobs also prove helpful, allowing REST API requests to be configured in a snap. In addition, the API’s responses can be evaluated for further processing. For example, models made available via the Ollama platform can be used to generate vector embeddings using the Ollama REST API. Creating a corresponding Automic REST job makes this step much easier. The Automic REST job can also be used to store the vector representation of the individual text blocks in any vector database that also provides a corresponding REST API, such as Chroma DB. Ultimately, the individual jobs are linked together in an Automic workflow, thus automating the RAG pipeline.

    RAG in action

    Implementing the RAG Pipeline in combination with LLMs opens up entirely new possibilities. With the new ASK_AI Automic script function, conversations with an LLM become an integral part of an Automic workflow. In this way, individual prompts can also be used for a similarity search in documents stored in the vector store. Relevant text blocks can be identified and added to the context of the conversation with the LLM. In this application scenario, a REST job can be used to initiate the similarity search. The result of this is ultimately sent to the configured LLM in an Automic automation script with the ASK_AI script function.

    Automic combines cutting-edge technologies with proven capabilities

    Automic Automation proves to be an ideal platform for implementing such scenarios. The wide range of integrations available on the Automation Marketplace allows for easy connection to third-party applications. The data from these applications can ultimately be made available in a vector store. Thanks to Automic Automation's low-code approach, individual building blocks can be developed as jobs and ultimately combined into an Automic workflow. Specialized job types are available, and thanks to Python jobs, more complex business logic can also be programmed and thus integrated into the automation workflow.

    Ultimately, Automic Automation also impresses customers with its proven capabilities for scheduling regular executions and its reliability in daily operations. GenAI and common design patterns such as RAG can be easily implemented with Automic Automation. It is therefore clear that Automic Automation is the right choice for combining cutting-edge technologies with proven capabilities, so teams can meet the challenges of the future.

    For more on Automic Automation’s recently released features, read the following blog: Democratizing Automation With the AI-Powered Filter Assistant and Seamless Python Scripting.

    Michael Grath

    Michael leads the Broadcom AOD Automation engineering organization, which has over 150 global team members. His responsibilities include Automic Automation, AutoSys, and Automation Analytics & Intelligence products and promoting innovation in product development through the use of generative AI.

    Other resources you might be interested in

    icon
    Blog April 14, 2026

    Announcing AutoSys 24.2: Accelerating Operations with Self-Service Agility and Automated Security

    Learn how AutoSys 24.2 helps reduce administrative bottlenecks, minimize security risks, and accelerate incident resolution.

    icon
    Blog April 10, 2026

    The Next Chapter for AutoSys: Moving Toward the Intelligent Control Plane

    Is Broadcom still investing in AutoSys? Yes! Learn about the V26 roadmap, which features MCP orchestration, AI job types, and AI-powered developer assistance.

    icon
    Course April 10, 2026

    Automic Automation: Upgrading to Version 26

    This course guides you through and demonstrates the process to upgrade Automic Automation from version 24 to version 26 on a Windows platform. The Unix upgrade is virtually the same.

    icon
    Course April 10, 2026

    Automic Automation: Integrated Database Maintenance

    See how Automic administrators can leverage the Integrated Database Maintenance suite to optimize their Automation Engine database for peak performance.

    icon
    Office Hours April 9, 2026

    Rally Office Hours: April 9, 2026

    Discover the latest Rally product updates, including new widgets and AI controls, plus upcoming events like the Big Room Planning workshop.

    icon
    Course April 8, 2026

    ValueOps ConnectALL: Synchronizing Rally and Jira

    Learn how to synchronize team-level data in Jira with program-level agile data in Rally using ConnectALL.

    icon
    Course April 6, 2026

    DX NetOps: Unified Collection Framework Install

    This course is designed to provide a clear, easy-to-follow guide for setting up and managing the Unified Collection Framework (UCF).

    icon
    Office Hours April 2, 2026

    Rally Office Hours: April 2, 2026

    This Rally Office Hours session highlights product tips for story parenting and queries, technical Q&A on API custom fields, and updates on upcoming AI and strategic portfolio management events.

    icon
    Video March 27, 2026

    Automic Automation Cloud Integrations: AWS Glue Automation Agent

    Broadcom's AWS Glue Automation Agent lets you easily execute AWS Glue jobs, monitor and manage them with your existing enterprise workload automation, as well as other cloud-native activities.