David Moscatelli: Organizations are hesitant about public AI due to privacy concerns, local AI solutions are preferred in banking and healthcare, and the Go One device enhances on-premises AI scalability | TWIST

2 hours ago 27

Key Takeaways

Organizations are increasingly hesitant to use public AI providers due to data privacy concerns.
Go Abacus focuses on providing AI infrastructure tailored for regulated industries.
Banks and healthcare institutions prefer local AI solutions to maintain data privacy and control costs.
The Go One device enables on-premises AI by connecting to employee PCs and preconfiguring software.
The Go One device supports up to 2,000 concurrent users and can be scaled by daisy-chaining multiple devices.
Sentry’s AI debugging agent uses comprehensive data to identify root causes and suggest fixes.
Stress testing has significantly lowered the chances of system failure in Go Abacus’s solutions.
The Go One OS uses specialized language models trained on client-specific data for deterministic tasks.
Client models are trained during off-hours, with updated weights sent back nightly.
LLMs (Large Language Models) are essentially CSV files with weights and software to run those weights.
Go Abacus’s approach highlights the importance of privacy and cost control in AI deployment.
The scalability and flexibility of the Go One device make it suitable for large organizations.
Go Abacus’s solutions are designed to enhance software reliability and developer efficiency.
The company’s client-centric approach ensures adaptability in machine learning operations.
Understanding the structure of LLMs demystifies their operation and utility.

Guest intro

David Moscatelli is founder and CEO of Go Abacus, a company building secure, on-premises AI infrastructure for regulated industries like banking. He has achieved significant early traction with the Go1 device, a $250,000 box that enables banks to run AI models without sending data to cloud providers, and has already secured 1,600 pre-orders while hitting $1M in annual recurring revenue.

Why organizations are hesitant about public AI providers

Organizations are hesitant to use public AI providers due to data privacy concerns.

— David Moscatelli
Banks, credit unions, and hospitals are wary of sending data to public AI providers.
They are not interested in sending their data to any of the public AI providers.

— David Moscatelli
Concerns about data leaving their infrastructure drive the need for local AI solutions.
How do we use AI but locally within our network so none of our data leaves our infrastructure?

— David Moscatelli
Privacy concerns are a significant barrier to AI adoption in sensitive sectors.
The trend is moving towards localized AI solutions to address these privacy concerns.
Understanding these concerns is crucial for AI deployment in regulated industries.

Go Abacus’s focus on regulated industries

Goabacus aims to provide AI infrastructure specifically designed for regulated industries.

— David Moscatelli
The company targets a niche market with specific needs for privacy and compliance.
We’re AI infrastructure for regulated industries.

— David Moscatelli
Understanding the challenges faced by these industries is key to Go Abacus’s business model.
The focus on regulated industries highlights the company’s market positioning.
Go Abacus’s solutions are tailored to meet the compliance requirements of these sectors.
The company’s approach addresses the critical need for secure data management.
Go Abacus is at the forefront of AI deployment in sensitive environments.

Local AI solutions for banks and healthcare

Banks and healthcare institutions prefer to use AI locally to maintain data privacy and control costs.

— David Moscatelli
Local AI solutions help institutions avoid sending data to external providers.
How do we use AI locally within our network?

— David Moscatelli
Cost management is a significant factor in the preference for local AI solutions.
You want a fixed price.

— David Moscatelli
Understanding these preferences is crucial for AI providers targeting these sectors.
Local solutions address both privacy and cost concerns effectively.
The trend towards local AI solutions is growing in the banking and healthcare industries.

The Go One device and its capabilities

The Go One device enables on-prem AI by preconfiguring software and connecting to employee PCs.

— David Moscatelli
The device is designed to facilitate AI access within organizations.
We have a piece of hardware which is the Go One.

— David Moscatelli
The Go One device supports up to 2,000 concurrent users.
That device can support up to 2,000 concurrent users at one time.

— David Moscatelli
Scalability is achieved by daisy-chaining multiple devices.
You can chain them together to four six eight etcetera.

— David Moscatelli
The device’s scalability and flexibility make it suitable for large organizations.
Understanding the technical setup of the Go One device is crucial for enterprise environments.

Enhancing software reliability with AI debugging

Sentry’s AI debugging agent uses comprehensive data to identify root causes of problems and suggest fixes.

— David Moscatelli
The agent enhances software reliability and developer efficiency.
Sentry’s AI debugging agent uses all this data and context.

— David Moscatelli
Efficient problem resolution is crucial in software development.
The debugging agent’s capabilities highlight the role of AI in software maintenance.
Understanding the importance of efficient problem resolution is key for developers.
The agent’s use of comprehensive data ensures accurate problem identification.
AI debugging tools are becoming essential in modern software development.

The importance of stress testing in AI systems

The chances of failure in our systems are incredibly low due to stress testing.

— David Moscatelli
Stress testing ensures system robustness and reliability.
On the basis of our stress testing the chances of failure is incredibly low.

— David Moscatelli
Understanding stress testing methodologies is crucial for technology providers.
Stress testing is a critical component of system reliability.
The effectiveness of stress testing is emphasized in Go Abacus’s solutions.
Ensuring system robustness is essential for AI deployment in sensitive environments.
Stress testing methodologies impact the reliability of AI systems significantly.

Specialized language models in the Go One OS

The Go One OS uses a collection of specialized language models that are trained on client-specific data.

— David Moscatelli
The models perform deterministic tasks in banking and other sectors.
Each of those models specializes in a very specific task.

— David Moscatelli
The use of specialized models highlights the OS’s unique architecture.
Understanding the structure of the Go One OS is crucial for its application in banking.
The models’ specialization ensures efficiency in performing specific tasks.
The OS’s approach to model training is client-centric and adaptable.
Specialized models are becoming increasingly important in AI applications.

Client-centric model training in the Go One OS

The company trains its models on client data during off-hours and sends the updated weights back to the clients nightly.

— David Moscatelli
This approach ensures the models are always up-to-date with client data.
We batch train on their data and then we take the weights from that training.

— David Moscatelli
The client-centric approach highlights the adaptability of the Go One OS.
Understanding how client data is integrated into model training is crucial.
The nightly update of weights ensures the models remain relevant and accurate.
This approach emphasizes the importance of client-specific data in model training.
The Go One OS’s adaptability is a key feature for its application in various sectors.

Demystifying LLMs and their structure

An LLM is essentially a CSV file with weights and software to run those weights.

— David Moscatelli
This explanation simplifies the understanding of LLMs.
Hate to demystify for everyone that’s what it is.

— David Moscatelli
Understanding the structure of LLMs is valuable for their operation.
The explanation highlights the fundamental components of LLMs.
Simplifying the concept of LLMs aids in their application and understanding.
The demystification of LLMs is crucial for their broader adoption.
Understanding the operation of LLMs is essential for AI practitioners.