How to Kickstart Your Generative AI Journey

The statistics are sobering. According to a Gartner 2024 study, only 9% of organizations achieve AI maturity, with a staggering 52% of AI projects never making it to production. Yet the potential remains enormous: McKinsey estimates generative AI could automate up to 70% of employees’ daily tasks. Why do so many companies lag behind while others succeed? And where does your company stand in this equation?

The difference between success and failure isn’t about tech stack: it’s implementation strategy. Instead of chasing shiny demos, the 9% who make it follow a methodical, practical approach with clear building processes and goals.

We built this guide to go beyond theory, using insights that helped dozens of Stack AI clients navigate generative AI automation challenges. It’s a battle-tested blueprint for bridging the gap between AI hype and concrete ROI.

How to kickstart your generative AI journey

• Before you begin

• Step 1: build an AI search engine for all company data

• Step 2: build department-focused AI tools

• Step 3: build AI-powered bespoke tools

• Build it with Stack AI

Before you begin

To ensure consistent deployment and improvement of AI tools, assign ownership to the right people, prepare your infrastructure, and foster a culture of continuous improvement. Let’s explore the practical aspects.

Establish a team

To start your generative AI journey, you need three types of people.

An IT specialist. This is a straightforward pick if you have an IT team or a resident IT expert. This person will ensure everyone follows the technical constraints of your current tools, security policies, and performance standards. If you don’t have an IT specialist, choose someone with a technical background in software, programming, or app development.

Solution builders. Identify tech-savvy employees with direct contact with daily work challenges. Prioritize those who are self-starters, have no-code experience or complex SaaS apps, and who everyone turns to for help when a technical problem arises. They will develop prototypes within constraints set by your IT specialist.

Early adopters. Select a small group willing to provide feedback on prototypes and promote the tool with the rest of the team. They will work closely with solution builders to improve tools and help increase adoption when they’re ready for release.

Prepare your data infrastructure

AI works best with extensive context and existing data, generating more relevant answers. Ask your IT specialist to list all platforms storing company data, including:

• Cloud storage platforms like Microsoft SharePoint, OneDrive, Amazon S3, or Dropbox

• CRM platforms like Salesforce or HubSpot

• Data warehousing solutions like Snowflake

• Other data sources include SQL databases, on-premises storage, or legacy systems

Prepare to connect these data sources to your generative AI platform via API. In Stack AI, this can be done via no-code integration processes, making it accessible to skilled non-technical people.

Foster a culture of continuous learning and improvement

Like AI, humans use past experience and pattern recognition to solve problems. Creating a culture of continuous learning means providing training on AI and automation, so your teams can identify more opportunities to implement these technologies.

Book generative AI workshops, organize hackathons, and reward employees for taking initiative and following through.

Step 1: build an AI search engine for all company data

A chatbot interface for an AI search engine built with Stack AI.

Start with an AI search engine for your company data to help executives and senior management get quick insights and in-depth reports. This tool will be fundamental to track key initiatives and performance metrics. It will enhance your decision-making with digestible insights for a more data-driven approach.

If you’re building with Stack AI, you can start with the chat with knowledge base template. Due to data sensitivity and integration complexity, assign your IT specialist to connect all data sources to your generative AI platform, set up a retrieval-augmented generation (RAG) system, and build a chatbot-style interface.

For example, you can drop all the data sources on the project canvas and connect them to your favorite LLM. We recommend the all-rounder GPT-4o or Claude 3.7 Sonnet for its writing abilities. If you have high data privacy and security standards, use the Azure AI deployment of OpenAI models, a HIPAA-compliant solution.

Since this AI chatbot interface has access to all your company data, secure it with SSO, role-based access control (RBAC), or a strong password. Distribute it to a small group in your company and gather feedback on answer quality and accuracy.

Determine how often to sync your data sources with the chatbot. For rapidly changing data, like eCommerce and financial services, choose hourly updates. This is crucial to increase trust in the tool’s responses and improve your decisions.

Step 2: build department-focused AI tools

Department-based AI search engine

The architecture of a project using Salesforce CRM data and an Anthropic Claude model to generate accurate responses on sales data.

You have your company-wide AI search engine, but you can’t share it with all your internal teams due to potential sensitive data exposure. Instead, ask your IT specialist to create a collection of knowledge bases (KBs): organizing the data sources into containers holding only the relevant information for each team.

For example, if you create a knowledge base with only CRM data and product information from Microsoft SharePoint, you can expose this data to your sales team. Their tools won’t access any other company data.

When you have a collection of KBs sorted by department or team, involve your solution builders. Assign RBAC rules so they access only what they need and ask your IT specialist to guide them in building an AI chatbot for searching their team’s data.

If you’re using Stack AI, follow this guide on how to chat with Salesforce CRM data and adjust the data sources based on your infrastructure.

This is a beginner-level project that solution builders can tackle in a few hours, learning the platform's basics. With these foundational skills, they can move on to more complex projects we’ll cover in the next section.

Department-focused workflow automation

Next, browse the platform’s templates and see which can be adapted to each team’s workflows. Here’s a shortlist of what Stack AI offers:

• AI contract analyzer

• AI document comparison

• AI staff training assistant

• AI spreadsheet assistant

You can browse in-depth guides on building these specialized tools, giving your solution builders more hands-on experience:

• Chat with financial reports

• How to build a HIPAA-compliant AI chatbot

• How to automate KYC and due diligence with AI

• How to build a customer scoring tool with AI

• And more available in the Stack AI blog

Your solution builders can adapt promising templates to their work. At this point of their learning journey, they are exploring the platform and experimenting with the available features. This represents the middle of the learning curve, as they become intermediate-level users.

As they finish viable prototypes, share these tools with early adopters. Foster communication between them: this is the moment to test tools’ applicability. The ideal scenario is when a tool saves time, improves quality, and reduces costs.

Beyond providing feedback, early adopters are essential to propagate the tool to every team member. Resistance to change is an obstacle, with skepticism and fear around AI and automation adding extra pressure. Dispel these notions with data-driven facts on the tool’s benefits as they become available.

Keep communication open between teams, early adopters, and solution builders. Effective collaboration and feedback loops will help build the most powerful tools, increasing value and ROI in the long run.

Step 3: build AI-powered bespoke tools

A high-level view of the architecture of a Stack AI project for automating tender document analysis using 5 LLMs.

By this point, you have a chatbot that can answer all questions about your business, department-focused Q&A chatbots, and specialized workflow tools for common use cases.

Planning and designing bespoke tools

Bespoke tools automate critical workflows in your business, adapting perfectly to your needs. This requires planning due to scope, quality standards, and minimizing risk.

If you haven’t already, map out your core workflows. Measure their resource usage and productivity output. Involve your IT specialist and solution builders to brainstorm how to automate parts or all of these processes using AI features like:

• Summarization

• Tagging and metadata generation

• Information retrieval and question answering

• Multimodality (text, video, image, and audio inputs and outputs)

• Reasoning using a traditional LLM or a reasoning model like OpenAI o3

• Step-by-step prompt chaining transforms data across a pipeline to generate new content, in-depth analysis, or decisions

• Agentic tools can decide based on detected intent, retrieve relevant information, and interact with external systems

• AI agents, semi- or fully-autonomous systems that can set goals, interface with external systems, and iterate an action plan as they complete tasks

• Multi-agent systems, a group of AI agents that work together

Your solution builders should now be skilled enough to design and build prototypes that address the core parts of the target process. Focus on small tools first, validate results, and then expand them to cover more tasks. The IT specialist will check these solutions comply with technical and security policies and assist in deployment.

Extend existing department-level AI tools

In unique situations, existing department-level AI tools may be good candidates for an upgrade, so they can either connect with more data sources, push data to external systems, or run more complex logic.

Make an inventory of all active Stack AI tools in your business, understand their positive impact, and see how to amplify them.

To experiment with existing tools, ask the original solution builder of each tool to share the Stack AI project with others (different from the publishing URL for sharing the interface). This will create a copy of the nodes, architecture, and instructions, letting your team use the main tool as you brainstorm and experiment with it.

Chain multiple Stack AI projects

In other cases, improving automation might mean connecting a set of Stack AI projects together, feeding outputs from one as inputs to another.

Ask the original solution builders of each project to share all these projects and then connect them in the Stack AI canvas. You can add a project to the architecture the same way you add a node. You can pass data in and out of projects using the input and output handles in any configuration.

Identify human-in-the-loop positions strategically

What are the key points of the automation? Where can errors have the biggest impact? Where is the most challenging AI processing step?

As you answer these questions, set human-in-the-loop positions and define clear terms of what constitutes:

• Good responses that humans can safely approve to let the automation run

• Bad responses that require repeating the process, editing or discarding the results

Assign solution builders, early adopters, and team members to these positions, and use auditing tools to track success: a simple spreadsheet to log a human-in-the-loop inspection result is enough to gather data to improve the tool long-term.

Deploy, monitor, iterate

Build and implement small-scale tools, grow them as they succeed, and connect them with strategic human-in-the-middle positions to ensure quality.

If you see high success rates at specific points, experiment with pausing human-in-the-loop there to see its impact on the final output. This frees up time for optimizing more challenging automation issues.

Build it with Stack AI

Stack AI is more than a generative AI automation platform: it’s a partner in AI transformation. It does so with:

• Onboarding sessions. When signing up for the Enterprise plan, schedule an onboarding session to access resources, ask questions, and start your generative AI journey.

• Dedicated solution engineers. Your company will have a dedicated solution engineer to interface with your IT specialists and solution builders, identifying automation opportunities and suggesting improvements.

• In-person workshops. Stack AI delivers workshops to your team in your offices, helping everyone get started, upskill to intermediate level, and master advanced concepts.

• ROI tracking. The team tracks how much you’re making for every spent dollar to ensure AI adds value.

Start your generative AI journey

Turn AI promises into real results with the game plan in this article. Build AI search engines, department-focused tools, and bespoke AI solutions to automate tasks, increase productivity, and unlock ROI. Get started with a free Stack AI account.