Team Ignite Insights · Nov 2, 2026 · 12 min read

Ignite Startups: The Truth About AI Bias, Data Quality, and Model Accuracy with Annie Brown | Ep307

Can more data really fix AI bias? Annie Brown, founder and CEO of Reliabl, explains why flawed data labeling undermines AI accuracy, how human expertise improves model performance, and why better-quality data matters more than bigger datasets.

Artificial intelligence is advancing rapidly, but one of its biggest weaknesses has little to do with computing power or model architecture. The real problem may be the quality of the data used to train it.

AI companies are investing heavily in larger datasets, more sophisticated models, and faster training infrastructure. Yet many systems continue to struggle with bias, inconsistent predictions, and tasks that require human judgment.

Annie Brown, founder and CEO of Reliabl, believes the industry is overlooking a fundamental problem: collecting more data does not solve anything if the process used to label that data is flawed.

On the Ignite Podcast, hosted by Brian Bell of Team Ignite Ventures, Brown explains how poor data annotation creates systematic errors, why human expertise remains essential to AI development, and how better data quality can produce more accurate models without endlessly increasing dataset sizes.

Her argument challenges one of the AI industry's most persistent assumptions: bigger datasets automatically produce better AI.

Brown's journey into artificial intelligence began far outside the traditional machine learning world.

She studied history and gender studies, focusing on media analysis, advertising, and semiotics, the study of how language and symbols communicate meaning.

That background became surprisingly relevant when she started building technology.

After working in microfinance and joining the teams behind Y Combinator-backed startups Superside and SafetyWing, Brown developed her own social media platform, Libs.

The platform grew quickly, reaching 50,000 users in six months and 100,000 users within a year.

But the growth exposed a serious weakness.

The platform needed an effective content moderation system, and traditional approaches were not equipped to handle the complexity of the content being uploaded.

Instead of relying entirely on external moderators, Brown's team developed a system allowing users to label content as they uploaded it.

This approach introduced a powerful idea: people who understand the context of information can help train AI systems more effectively.

Other organizations began approaching Brown with similar challenges. Their AI systems were hitting accuracy limitations, and they needed better ways to incorporate human knowledge.

Those conversations eventually led to Reliabl, a collaborative data annotation and AI evaluation platform.

Why More Training Data Won't Fix Broken AI

One of Brown's strongest arguments is that the AI industry has become too dependent on scale.

The conventional assumption is straightforward: if a model is inaccurate, biased, or unreliable, collect more data and train it again.

But that strategy ignores how the data was prepared.

If the original annotation process systematically misclassifies information, adding more examples can reinforce the same mistakes.

Brown compares this to pushing additional data through a contaminated system. The output remains flawed because the underlying process has not changed.

The problem becomes particularly serious when AI systems move beyond straightforward classification.

Identifying whether an image contains a cat or a dog is relatively objective.

Determining whether someone's voice sounds angry, whether an image is suggestive, or whether language is discriminatory requires a much deeper understanding of context.

Those distinctions cannot always be reduced to simple labels.

For founders building AI applications, the implication is significant.

Before investing in larger datasets or more expensive training runs, examine how the information is being classified.

A model trained on fewer, carefully labeled examples may deliver better results than one trained on massive amounts of inconsistent data.

The Hidden Bias That Can Destroy AI Accuracy

One of the most revealing examples Brown shares involves an audio dataset obtained from a major university.

While reviewing the recordings, her team discovered that almost all the Black women's voices had been labeled as angry or upset, even when the conversations sounded neutral.

Had the dataset been used without further review, the resulting model could have learned to associate Black women's speech with anger.

This demonstrates why annotation errors can be much more dangerous than isolated mistakes.

When the same error appears repeatedly across a dataset, it becomes a pattern that the model learns.

Brown explains that Reliabl has worked on an audio project involving approximately 70 hours of recordings, where consistency across the dataset is critical.

A small percentage of unusable data may be manageable. But as errors approach five or ten percent, they can significantly undermine model performance.

More importantly, even a relatively small amount of systematically biased data can create problems that extend beyond those individual examples.

The lesson is that model accuracy depends on the consistency and integrity of the underlying annotations, not simply the number of data points.

Why the Words Used to Label Data Matter

One of the most overlooked parts of AI development happens before annotation begins.

Teams must decide which labels to use and what those labels mean.

This process is known as taxonomy design.

Brown argues that poorly chosen terminology can introduce bias before a model ever begins training.

She highlights an example involving Google Vision AI's content moderation system, which used the label "racy" to classify potentially suggestive imagery.

The problem, according to Brown, is that the word carries historical and cultural associations that can disproportionately affect how images of women are classified.

A seemingly simple word choice can therefore shape the predictions generated by an AI system.

For developers, this raises a critical question: Who decides what each label means?

A technical team may define categories differently from lawyers, linguists, artists, or the communities affected by an AI system.

Reliabl addresses this through collaborative annotation, allowing multiple stakeholders to contribute their interpretations and reach consensus.

The goal is to capture knowledge and context that traditional labeling pipelines often overlook.

How to Test Whether an AI Model Is Biased

Brown also challenges the usefulness of asking chatbots directly whether they are biased.

A chatbot's explanation of its own behavior does not necessarily reveal the weaknesses in its training data or the mechanisms behind its predictions.

Instead, she recommends conducting evaluations that examine actual model behavior.

She describes a simple experiment involving two nearly identical prompts.

In one, she identified herself as a woman seeking a job in data science.

In the other, she identified herself as a man making the same request.

The results were noticeably different.

The prompt identifying the user as a woman produced career-development advice, including suggestions about improving a resume.

The prompt identifying the user as a man produced a list of jobs to apply for.

The only meaningful difference was the stated gender.

The experiment illustrates how AI systems can generate different recommendations based on demographic information, even when the underlying objective remains unchanged.

It also demonstrates a practical evaluation technique.

Instead of asking an AI system to explain its biases, change one variable in an otherwise identical prompt and compare the results.

That approach can reveal inconsistencies that would otherwise remain hidden.

Why Human Expertise Still Matters in AI Training

As artificial intelligence becomes more capable, some companies assume human annotation will eventually become unnecessary.

Brown disagrees.

She argues that the more complex the task, the more important human context becomes.

Simple classification tasks can often be handled by large groups of general-purpose annotators.

But evaluating legal issues, human rights questions, specialized scientific information, or complex social situations requires people who understand the subject.

Reliabl works with experts such as lawyers and linguists to evaluate AI outputs and improve annotation accuracy.

The company also works with Humane Intelligence, an organization involved in AI evaluation.

Brown describes collaborative projects involving Dream.org, Zindi, Humane Intelligence, and Radiant Earth that aim to improve mapping data.

Some existing maps fail to capture important details about marginalized communities.

By involving people from those communities in the annotation process, the project can incorporate knowledge that conventional datasets may miss.

This creates an additional benefit: local participants can earn income while contributing expertise that improves AI systems.

The broader principle is that the people closest to a problem may possess information that even sophisticated AI developers lack.

Can Better-Paid Data Workers Produce Better AI?

The AI industry's labor practices are another area Brown believes needs rethinking.

Many annotation operations depend on large numbers of workers completing repetitive tasks, often under demanding conditions and for relatively low compensation.

Brown argues that this approach undervalues the people responsible for preparing the information on which advanced AI systems depend.

She also believes it creates unnecessary inefficiency.

Poorly structured annotation workflows can generate noisy datasets that require additional processing and correction.

Instead, she advocates treating data labeling as skilled work, with fair compensation, better working conditions, and greater participation from domain experts.

Her business argument is that better working conditions and more thoughtful annotation processes can produce higher-quality data.

That creates the possibility of using smaller datasets while maintaining or improving model performance.

For AI startups, this challenges the assumption that reducing annotation costs should always be the primary objective.

The cheapest data pipeline may ultimately become expensive if it produces unreliable models.

Will AI Eventually Replace Human Data Annotators?

Brian Bell raises a difficult question about the future of annotation work.

As robotics and AI advance, workers are increasingly being asked to record and label tasks that machines may eventually perform independently.

In some cases, they are helping create the training data that could automate their own jobs.

Brown acknowledges the tension but argues that complete automation remains difficult, particularly for activities requiring contextual understanding.

Human behavior, language, and social expectations continually evolve.

Scientific knowledge changes, communities develop new practices, and existing categories become outdated.

As a result, AI systems may require ongoing evaluation, updated labels, and additional training.

Brown sees the long-term opportunity in systems that combine automated classifiers with continuing human oversight.

Reliabl is developing its own classifiers to improve the efficiency of annotation while retaining human participation for complex tasks.

The objective is to reduce repetitive work without removing the expertise required to maintain accuracy.

Why AI Infrastructure Needs More Careful Planning

The conversation also moves beyond data annotation into the broader future of artificial intelligence.

Bell compares today's AI industry to the internet in the mid-1990s, when the technology's potential was becoming clear but its infrastructure and economics were still evolving.

Brown shares his optimism about technological progress but expresses concern about the speed of data center construction.

She questions whether companies are investing in enormous infrastructure projects before fully understanding the most efficient and sustainable approaches.

Her concern reflects a recurring theme throughout the episode.

Moving faster does not necessarily mean building better systems.

Whether the challenge involves data centers, training datasets, or model evaluations, thoughtful design can matter more than raw scale.

Brown also expresses concern about the behavior and incentives of major AI companies, particularly when discussing regulation and the concentration of wealth created by advanced AI.

While she remains optimistic about artificial intelligence itself, she is more concerned about how people develop and deploy the technology.

What AI Founders Can Learn from Annie Brown

Beyond technical discussions, Brown shares several lessons from building Reliabl.

1. Build a team that can execute the product vision

Brown credits her product lead, Yulia, and the broader engineering team for creating Reliabl's accessible user experience.

She emphasizes that even an excellent product vision can fail without people capable of turning it into working software.

2. Test products with real users before launch

Reliabl's team tested its product extensively internally.

But when external users tried the software, they identified problems the team had overlooked.

That feedback helped the company address issues before rolling the product out more broadly.

The lesson is straightforward: internal testing cannot fully substitute for observing how real customers use a product.

3. Follow unusual intellectual interests

Brown credits professors who encouraged her to explore subjects that did not initially appear commercially relevant.

Her interest in history, language, and gender studies eventually became central to her work in AI.

She believes founders who pursue problems connected to their personal experience may be better equipped to persevere through difficult periods.

4. Question the industry's default assumptions

The belief that more data automatically produces better models is precisely the assumption Reliabl was built to challenge.

Brown's experience suggests that meaningful startup opportunities can emerge when founders examine practices that established companies have stopped questioning.

The Future of AI Depends on Better Data, Not Just Bigger Models

Artificial intelligence has made remarkable progress, but growing model capabilities have also exposed weaknesses in how training data is collected, classified, and evaluated.

Annie Brown's work at Reliabl highlights a different path forward.

Instead of treating human judgment as an obstacle to automation, AI developers can use it to improve the accuracy, consistency, and usefulness of their systems.

That means designing clearer taxonomies, involving relevant experts, identifying systematic bias, and testing models against real-world conditions.

For founders and investors, the opportunity may be larger than another model or another dataset.

It may lie in the infrastructure that makes AI dependable enough to handle increasingly complex tasks.

The next generation of AI breakthroughs may depend less on how much information machines consume and more on how carefully humans teach them what that information means.

Chapters:

00:00Annie Brown's Background and Journey into AI
04:19From Y Combinator to Founding Reliabl
09:01How Collaborative Data Annotation Works
11:04The Hidden Costs of Bad AI Training Data
13:13Why Expert Knowledge Beats Crowdsourced Labeling
14:59Reliabl's Role in AI Training and Evaluation
17:34Human-in-the-Loop AI and Community Mapping
19:18Building Software Through Co-Design and User Feedback
21:36Testing Chatbots for Gender Bias
25:28How Taxonomy Design Shapes AI Bias
27:28Rethinking Data Labeling and Worker Compensation
30:07Will AI Data Workers Automate Their Own Jobs?
33:06Reliabl's Future and AI for Cities
35:37Will AI Eventually Eliminate Human Annotation?
40:06AI Existential Risk and Responsible Development
46:04Changing Perspectives on Politics and Community Impact
50:55Why More Data Won't Fix Broken AI
52:30Curiosity, Mentorship, and Building Startups
54:34Closing Thoughts and Where to Find Annie

Listen to this episode

Open the full episode page ↗

Read the full transcript

Brian Bell (00:00.791) Hey everyone, welcome back to the Ignite podcast. Today we are delighted to have Annie Brown on the mic. She is the founder and CEO of Reliable, a platform that turns organizational and community knowledge into reliable, get it, AI through a collaborative annotation. she was a AI researcher at UC San Diego and sh her path has had a lot of interesting things from microfinance in Bangladesh to a machine learning patent to a startup in Oslo. So we're we're happy to have her on the pod. Thanks for coming on, Annie.

Annie Brown (00:31.224)Thank you for having me. Love Team Ignite. And yeah, I've been looking forward to doing this with you. So I'm excited.

Brian Bell (00:37.781)Thanks so much. I'll give you some money later for that. I'd love to start with your origin story. What's your background?

Annie Brown (00:45.056)Yeah, my background is a little unique to the AI startup world. I actually have my undergradu undergraduate degree in history and gender studies. And my focus there, I did a lot of media and data analysis on advertising and business practices. And so my undergraduate research paper focused on women's media globally. And then also we in that work, which is related to my AI work now, is I did a lot of semiotics research. And so that semiotics research I, you know, talk. Yes, let's define some terms. So semiotics is essentially the study of language, but it's language specific terminologies and the weight they carry. So like

Brian Bell (01:24.407)What is semiotics? What is that? Semiotics, what is that?

Annie Brown (01:40.14)The a really great example if you read Roland Barth's myth mythologies, he has a chapter on the word rose. So like obviously a rose is just a rose, but it also sym Exactly, but it also symbolizes love, passion, romance. Like, you know, it has these terms and so it in taxonomies

Brian Bell (01:53.279)One is a rose, just a rose. So a word a a picture is worth a thousand words, but a word's also worth a thousand words.

Annie Brown (02:06.815)Yes, exactly. I love that. Put that on a bumper sticker that only semiotics nerds will understand. yes, but yeah, so anyways, that research kind of has led me to this work, but in between that research and what I'm doing now, I graduated, worked in microfinance for a while, really understanding kind of what a social impact business was. I was kind of when this term social impact business came about.

Brian Bell (02:12.769)Yeah. A word is worth a thousand words.

Annie Brown (02:36.409)And then also then started got recruited as a communications director for a company based in Nor Norway that one week after I joined got into V S into Y Combinator. And so I kind of was thrown into the startup and tech world. Yeah.

Brian Bell (02:53.907)The heart of startups, yeah. Did you guys actually relocate to SF as well for the program?

Annie Brown (02:59.155)Yeah, well they so they did. I was on the founding team, so I was still we were like that was kind of the cause they were really pushing remote work before remote work was as big as it was, which is why both, you know, the companies I was with are actually quite successful now, which is lovely to see. but they I was in San Diego at the time, doing starting my AI bias kind of research thing in parallel to when I was working with them because at the same time I was working at that startup, I was working on my own social media platform that actually came out of my women's magazine research. And I turned that into a social media platform. In that way, I was able to learn about AI in content moderation. And that's how that's what got me into AI to start. So I kind of got that startup journey at the same time I was starting, you know, learning about, you know, why AI does not function as we want it to and why what are the gaps that are keeping us from AI doing more sensitive or difficult tasks. And that's the patent part of the introduction is that now we have a patent on that system that we built and that system is now reliable.

Brian Bell (04:19.351)Amazing. So how long? So that YC company was superside, right? And then what happened between Superside and the founding of reliable?

Annie Brown (04:24.566)Yes. huh. Yeah, so I I was doing my platform was called Libs. And it still exists to this day, so it's called Libs. And so that platform, we got a grant from Headstream Accelerator. I was building that while also doing the remote work with Superside. SuperSide had a spin-off company that also got into Y Combinator and they recruited me to be the communications director to kind of kick off that team. So I got to go through Y Combinator again, which was a nice, fun experience, with them and learned a lot from the founders, Sandra and Frederick are just like great founders. so that spin-off company did really well. I was working with them, starting to work more on like technical communication stuff now that I was doing a lot of AI bias and AI research.

Brian Bell (05:06.443)Mm-hmm.

Annie Brown (05:26.623)And then also using the money from that Headstream Accelerator grant to then build this social media platform. But the problem was we built the social media platform to about we scaled it very fast. It got to 50,000 users in six months and then a hundred thousand users within the year. And we did not have the content moderation system set up to handle that. So essentially we built this content moderation system and it worked really well. And all we did was built this process by which the users on the social media platform could label the content as they uploaded it. and what that did was it opened up this whole concept of participatory AI and how you direct user insights or direct expert insights can help improve AI outcomes. And so I started talking about this. while I was working at the safety wing, the spin-off company. the I bas I got recruited into working at CodePath kind of while this was happening too while I was sort of doing my own personal thought leadership around AI bias and started doing technical communications for code path and then I think like what sort of happened was that As I kept talking about this and doing more thought leadership, we got coverage in, you know, fast company and mashable. Like the concept that we were talking about was really picking up traction. I would also get people coming to me saying, Hey, I don't have a social media platform, but what you're building for AI seems very applicable to what we're doing. And it seems like our AI is hitting this like accuracy wall similar to what you described as happening to you. Could you build something for us? And so that's when it sort of clicked in my head that this concept that we built could be an actual company. So I started, I then that's when I met my CTO and co-founder, Duncan, and we started talking about, you know, what this concept would look like as an actual team. And we had the nice privilege of actually going through a class at Stanford. Together that was designed for PhD and postdoc students at Stanford who were interested in startups. And so me and Duncan kind of worked on it for three months over the summer and then at the end of it took it took it to be a real company. And about that's around the time we got into Tech Stars.

Brian Bell (08:09.555)Love this background because I think for any founders listening, working on an idea or thinking about ideas, you can you'll come up with the most unexpected ideas just through like living your life. You know, you've had all these like varied experiences, but they all kind of magically or almost in a happy accidental way led to reliable. Yeah.

Annie Brown (08:20.917)Mm-hmm. a hundred percent Yes, a hundred percent. And it also makes it so that it's something you actually want to stick with. Cause I mean, I've already been doing this for, you know, I guess now almost going on fifteen, twenty years, this these ideas that I've been having. So it's like, why stop now? You know, e even if hard things come up, which they always do in the entrepreneurial journey. So it's if it's also something that you have lived experience with or like have dealt with a lot, you'll probably be more likely to really stick with it too.

Brian Bell (08:43.425)Yeah. So reliable is a collaborative annotation tool, is how you would define it. What does that mean? for p people listening who don't know a lot about AI and annotations.

Annie Brown (09:05.76)Uh-huh. Yeah. Yeah, so we sell into data science teams and AI teams that are working to build complex problems or working to build AI to solve complex problems, or are evaluating AI systems that are very complex. And so what collaborative data annotation means is that we built a software that is very easy for different stakeholders to come together and come to a consensus on what one piece of data means. Which, you know, in some ways, if you're just labeling like I a lot of time separate this into two camps. So there's sort of early AI, which is very objective. So is this a cat, a dog, or a tree? And now we're yeah, exactly. Yes, the hot dog no hot dog machine. Exactly. That is that is early AI. And

Brian Bell (09:55.106)Or a hot dog. Hot dog, no hot dog. Silicon Valley, remember that? Yeah.

Annie Brown (10:04.555)The problem with hot dog no hot dog is that same system is now being asked to identify happy, sad, or being asked to identify is that teal or like soft pale blue, you know, these things that are like so maybe so obvious to certain human eyes. Like if I'm an artist, I can tell the difference between teal and blue. But for AI systems that were trained using this very you know, binary system. It just doesn't work. So reliable builds in with through our taxonomy design, ontology designs, collaborative tooling, governance, and our AI system that we're we're building that will help to identify these contexts as well. We're able to bring multiple viewpoints into labeling a piece of data to get it accurate. and then that accuracy then translates into. better accuracy for your AI outputs and better performance scores.

Brian Bell (11:04.833)How costly is a bad annotation? Like one out of a million annotations, no big deal. Like at what point does it like get really costly though? Yeah.

Annie Brown (11:08.779)Mm. You'd be surprised. It's it's it's interesting, like how much data, like right now we're working on this audio project that's like 70 hours of data. The data has to be consistent because and then labeling has to be consistent across that 70 hours. Like we're at we're kind of at a point where it's like, okay, if it's two percent of the data that you know fails or is poorly labeled, or the audio quality is bad, or things like that, then like. You know, we can throw those out and still train the model. But once it's getting closer to like five, ten percent, like that's gonna throw off your whole model because consistency is everything with AI. And so what's what's important about reliable is a lot of those errors that happen in annotation are ones that spread across the whole data set. So if you don't catch it early, you can have your labeler say, like a really good example would be We had audio data set that we had from a really, you know, big university. So you would think like, okay, this is good data. But it within that data set, all of the black women's voices, or almost all the black women's voices, were labeled as upset or angry when we were listening back to them, being like, no, this sounds this is neutral. This is a neutral conversation. And so if our client had used that data without us scanning at first, their whole data set would have had the AI predicting every time a black woman spoke, you know, that would be labeled as angry or upset, which would have, you know, made the model fail. And so it can be even if it's in a small portion of your data, one, the likelihood of it being in a small portion of your data is low, but even if it's in a small portion of your data, if it's three to five, you know, percent,

Brian Bell (12:56.033)Yeah, yeah.

Annie Brown (13:09.855)It will still affect your model performance for sure.

Brian Bell (13:13.847)I couldn't couldn't I if I have a bunch of stuff to label like brute force it with mechanical turk or something like that and just once like two or three annotators agree, I'm like, okay, I'm reasonably confident that's a hot dog.

Annie Brown (13:19.552)Mm. Yes. So I think we can be, you know, reasonably co confident in the hot dogness of the hot dog. But if the hot dog were say important in some legal human rights instrument, like you know, some of the work that we're working on, then that would be something that Mechanical Turk could not handle. you know, if you're talking about evaluating Complex social scenarios, especially if you're talking about things with like lived expert experience. So you can have experts like we have lawyers using our platforms, we have linguists using our platforms that when we talk to them about using other labeling platforms, the accessibility to get in, these platforms were built by technical teams for technical teams, not people that maybe don't have that technical experience, but have deep, deep. knowledge that you need to put into your AI. And so you can brute force it, but you're not going to get the results that you want. The other part is that there's another step even before you do the annotation that a lot of people overlook, which is the taxonomy design. Because if you're going back to talking about semiotics, if you don't choose your words carefully, you can imbibe unintended consequences into your model. So that's that's why reliable as a software, we have teams that need precision coming to us instead of going to the traditional solutions.

Brian Bell (14:59.509)I mean it's very much a gold rush happening in AI and you guys are kinda like the Levi Strauss selling, you know, canvas tents out there to all the people building AI, right?

Annie Brown (15:07.021)Yes. Yes. Yes. Yeah. I mean, we are AI dev tool. we're you know some of our customers refer to us as an operating system where they come and you know test the quality of their data before they run the AI systems. And then we also have some customers that come to us and test their AI systems after they've they've run the system. So Essentially, then you can actually do evaluations. So that's why we're a evaluation and training platform for AI. And yes, yes, because yeah, several of our clients are well, we work with one large evaluation nonprofit called Humane Intelligence. they're doing very interesting work, and then we have other clients that are actually evaluating large language models on

Brian Bell (15:43.491)You so you help on the evals too then? Interesting.

Annie Brown (16:04.523)Difficult social issues. and we're helping their lawyers, human rights experts, you know, international development employees come in and actually do the labeling of the outputs of the evaluation. so to label if, you know, this was a pass or a fail.

Brian Bell (16:26.219)So if you're helping on the annotations and you let the users, the communities who use your tool to design their own taxonomy as well, then you're not too far off from like probably fine-tuning models for them too, right? Just kind of, you know, forking some open source market because they all have all their own annotations. They have, you know, and so what are your customers using today if not reliable to do that?

Annie Brown (16:37.569)What is Yeah, well that's Yeah, so a lot of our so we kind of we offer, you know, it in and tiered pricing structure. And so our highest tier, we can come on and basically assist with, okay, you've done this project in reliable, what is the next step to moving to fine tuning? I would say in our startup journey, one of my dreams for reliable is that you could actually do that fine tuning within the reliable platform. So it could be just like this big feedback loop. Yep.

Brian Bell (17:21.387)Yeah. Yeah. Just like yeah, cr run a training run. Yeah, just fine tune it. Boom. Come back. Come back twelve hours later. It's like, yeah, we ran it against the evals and you know, we're we put push the F one score this way or whatever.

Annie Brown (17:27.355)Exactly. Yep. And it could it really can be that easy. I mean, obviously with reliable we'll always have the human in the loop aspect to it because even once we have our own classifiers running that are they're trained on this, we always recommend with our customers that they run an evaluation or they run a a labeling, you know, batch that then our model can learn from the experts and then we can, you know, duplicate that. So That's the kind of the other side of Reliable too, is that we are really interested and excited about the possibilities of workforce development with these types of tools because you have people that are, you know, PhDs working in Reliable, but you also have people that haven't graduated from high school using reliable, doing really important mapping tasks. We're doing a really interesting project right now with Dream.org and Zindi and Humane Intelligence and Radiant Earth. All came together to do this mapping equity project that shows how like overture map data might not be capturing the what's happening on the ground, especially in marginalized communities. And so hiring those marginalized communities to then do the labeling and fill in this map data that is very useful and important to, you know, big, big mapping companies. So that's that's kind of the thing that gets me really excited too about reliable is that we're bringing in this oftentimes overlooked population in AI and we're employing them and then boosting model performance. So it's a very good business case for, you know, doing that type of work anyway.

Brian Bell (19:18.945)Yeah. What have been some surprises or things you've learned building this platform that you found surprising?

Annie Brown (19:26.477)I think, I mean, I've learned a lot about building software, which I is very good knowledge to have. I love having it. And I think the team that we have in building it has been incredibly important. We have our full-time product lead, Yulia, has just like she's she's been with me since I was building that social media platform, Lips, and just UX as its own like feature and as its own selling point is been really cool for us 'cause I think people get into the reliable software and they're like, Wow. Yeah. well thank you.

Brian Bell (20:02.613)Yeah. You guys had a really good demo. I I see a lot of demos and I'm a product person. Been building product for, you know, fifteen years or something, or not anymore, I guess, but for ten years I did. But you guys had a really nice UI. And is that is that is that this team member who's really good at UI?

Annie Brown (20:16.397)Yeah. Mm-hmm. Yep, that's all Yulia. And I think we also work as a team really well because co-designing is like a principle of the work we do with AI stuff too. We use it internally. And so the whole team is able to put in really good input. And then Arian, our full stack developer, also is a machine learning developer. And so he's been able to, you know, work with Yulia to implement these really beautiful changes. So I would say like my advice to founders would be like build a really good team.

Brian Bell (20:21.238)Yeah.

Annie Brown (20:48.695)For software, because even if you have the most beautiful vision in your mind, if you don't have a good team to execute it, it just like it won't happen and it'll be a a nightmare. so pick your people well. And then also user testing and user feedback is so important. we've made like we tested our own thing, you know, a hundred times internally. We're like, wow, this is great. And then we ran it, we rolled it out with customers or we rolled it out with our first user testers and they caught like, you know, ten things we didn't even see that we didn't want to roll out. So it was good that we did that user testing ahead of, you know, rolling out with customers too. So those are I guess my two biggest takeaways. But I just love building software. I think it's really, really fun and I've enjoyed learning about it.

Brian Bell (21:36.981)You said some pretty interesting things online. I I I found something or you commented or maybe it wasn't TechCrunch article on chatbots being sexist. Yeah. Yeah, and you said asking a model about itself teaches us very little. What did what did you what should people look for instead?

Annie Brown (21:48.481)No yeah, that was a fun one. Good question. Yeah. So I think a lot of times like when we're having conversations with these chatbots, we think that they are capable of giving us answers about themselves, like about like how they're built or

Brian Bell (22:08.331)Yeah. But that's that's true intelligence when you can like reflect, you know, and there are reasoning loops now, right? Where the AI can kind of like analyze like its output. But it's it's it's sort of like predicting a bunch of words and then looking back and saying, look at those stream of words. Do I like what do I think about that? But it's not like self referential conscious loops, you know.

Annie Brown (22:12.329)Uh-huh. Sure. Well it yeah, and it's it's like it's also that, you know, the comp like the company is not putting their proprietary, you know, how I built Claude into Claude, right? So it's funny, I heard a joke the other day that was like somebody was like, Well, why don't I just go into Claude, tell it to build me Claude too, and I'll be a trillionaire as well. So it's kinda like I think that's yeah.

Brian Bell (22:53.399)Yeah. It's kind of like saying, like, well, just go talk to a human being and ask the human how they're built. Yeah. how they grew from like a single cell to like this trillions of cells organism and how their whole brain works and everything. It's like

Annie Brown (23:00.831)Right, exactly. Well, exactly. Yeah, and it would also be if like my human body were owned by a mega corporation with a bunch of lawyers that protected all my secrets too. So I guess that's so it's like when you ask a chatbot, are you biased? One, probably the s scientists behind, you know, anthropic or behind open AI they've already hardcore the answer to that exactly, but also like

Brian Bell (23:28.661)Already hard coded that the answer to that. Yeah. Yeah.

Annie Brown (23:34.26)They're still understanding. We don't really have a full understanding of like the biases in the system. It would be complex to share, but also they wouldn't want to share that anyways. So to answer your question, the alternative to that would be to actually do an evaluation yourself. So like, you know, you can do small tests when you're, you know, with AI. I remember I did one test that was pretty interesting saying, I'm a woman, please help me, you know get a job in data science. And I said, I'm a man, please help me get a a job in data science.

Brian Bell (24:07.497)Without being logged in to like on the like free version, so it knows nothing about you basically. Yep.

Annie Brown (24:12.255)Right, yeah, exactly. And so the woman one gave me a bunch of career hunting tips and like how to how to build up my resume essentially. And then the male one just gave me a list of jobs to apply to, which I thought was very interesting. So

Brian Bell (24:30.965)Wow, that is real so same question. The only thing is different is I'm a man or I'm a woman. And the model literally gave different answers to different genders. Wow.

Annie Brown (24:35.574)Right. yeah. Yeah. And that's why these evaluations are so interesting because you really get to see the gaps where these models are. And I think we're at such an interesting time in the technology. And, you know, to bring it back to reliable why we are so excited about what we're building is because there are all these gaps in the software that I think now that we're asking it to do more for us, we're starting to realize are there. But yeah, you can run your own small evaluation or you can do

Brian Bell (24:47.115)Yeah.

Annie Brown (25:08.543)A big proper one, you know, like we do with clients, but evaluations is a whole nother discussion. But also like what we're trying to do is bring more precision into evaluations because even evaluations that are done on really important AI models are a little like questionable on like repeatability and scientific value and things like that.

Brian Bell (25:28.999)Yeah. Why do the public debate on bias gets you know, it's it's focused on the model, but you know, I guess journalists don't understand the difference between the model and like the annotations and the taxonomies and

Annie Brown (25:36.909)Mm. Yes. Yeah. That's I i it's a good point. And one thing, you know, in our thought leadership that we try to educate on is that there's so many things that come into the output of an AI model. and it's not just like, you know, open AI has this one perspective on gender and they embedded that. No, I mean you are talking about millions and millions of documents and like if 20 million of those were annotated in a way that, you know, gender was not accounted for in the preparation of the labeling process. Like there wasn't clear instructions or the word they use was incorrect. Like one that I bring up a lot is and I I can share the link in the notes, but we did this presentation on Google Vision AI, like their moderation predictor, and they had this term racy. to determine if something was suggestive, but just because of historical context, racy is much more associated with women. And so therefore women would be much more associated or much more likely to be get a higher rating on this racy scale. So if Google had chosen, yeah.

Brian Bell (26:57.557)Yeah. And if you've been crushed by corporate jargon like I have, racy means like it's a framework for responsible, accountable, consulted and con informed.

Annie Brown (27:02.423)Mm-hmm. There you go. Well exactly. And it's it's Yes, and it's those differentiators in how all of our brains kind of perceive things that we have to account for when we're doing annotation if we wanna do it if we wanna do it well.

Brian Bell (27:13.697)Yeah, yeah. What piece of conventional wisdom in the data labeling industry is due for a a rethink?

Annie Brown (27:34.77)Mmm. let's see. I'm gonna pick one. I think there's a few things. I think it's it's definitely an antiquated kind of system because you're you're thinking about like, these super high tech AI companies making so much money, cutting edge of technology, but then you're basically like the way the data annotation is done is you are shipping out tasks. That you want your AI to be done by people, by humans, overseas, who are oftentimes underpaid and overworked. So I would say that would be my probably top one because from there everything goes up is that I think data workers have been undervalued in the role that they have played in developing this really quite incredible technology. And so valuing that work. And that means, you know, fair pay, but also good working conditions, you know, labor laws that protect everybody. And I and the way that

Brian Bell (28:43.927)Yeah, they're like they're like kind of the factory workers and the you know, the first industrial revolution who first entered a factory and they're just sitting there kind of turning this like wrench all day long and getting like repetitive stress injuries and stuff. Kind of the same thing in the data labeling, it's very much this like you know, assembly line of annotation, right?

Annie Brown (28:47.585)Yes. Yep, mm-hmm. Yes. And there's and there's harms in it. And I think our argument is that when you reduce the harms on the workers and you know give make this a dignified, you know, good tech job, you will get better results. Like and what we've shown with reliable and what the study shows too is that if you have less data but it is labeled better, your model will train just as fast, if not faster. Whereas if you have this approach that's now, which is these huge sloths of data that are all getting labeled we're very noisy. And so that's so if we streamline it, we can pay people better, do more with less data and get better AI outcomes.

Brian Bell (29:38.433)Very noisy and yeah. Yeah. We're investors in mini companies, as you know. and we're we're in a couple of these robotic data labeling companies. we invested out of YC. And you know what always struck me about using people in Vietnam or Philippines or somewhere in Latin America to record themselves doing things like cleaning a house or repairing a car is people are literally

Annie Brown (29:52.427)Yes. Mm-hmm.

Brian Bell (30:16.959)Labeling themselves out of a job. You know? Because like eventually these robot robots are kind of the next wave, right? They're the next chat GPT is coming. Like we haven't had the chat GPT moment for robotics yet, but it's coming in the next like year to two or three years, I think. Where all of a sudden these robots will get really, really good at physical, physical tasks, as good as like the Frontier labs are at text based tasks and image based tasks now.

Annie Brown (30:19.733)Yes, yeah, mm. Right. Mm-hmm. Yeah. Yeah.

Brian Bell (30:45.583)And well, I think we'll look back and it'll be, I don't know if I can think of a corollary in history of when like a set of workers basically were you know, labeled themselves, worked themselves out of jobs, right?

Annie Brown (30:58.817)Yeah, I mean I mean I think the Industrial Revolution and like factories and, you know, the first wave of kind of robotics is a good is a good primer for that because I do think in order to do that work well there had to be video footage of people doing it. They had to talk to the robotics engineers about what I'm actually doing. So I I think Yeah.

Brian Bell (31:04.523)Yeah. Yeah. I think I think the best equivalent would be like if the if in the industrial revolution you were in a factory that built parts for fa automated factories. That would be like the equivalent. Yeah.

Annie Brown (31:27.125)Right, right. Yeah, yeah, yeah. You're you're building the infrastructure. Well, one thing I will say on that, because I do th I do think that is true. There is a lot first of all, there's like a long way to go between there and then. and we learn that every

Brian Bell (31:40.395)Yeah. I mean yeah, it might be a decade away for robotics really. But

Annie Brown (31:44.062)Yeah, because and that's why like companies like reliable I think are exciting because it's like you're in this place where we're the gap like we were talking about before is so wide still to get from somebody cleaning a house really well to a robot cleaning a house really well and all the things that come in between that, all the nuances that are required for that to actually happen. So there is still time and so part of my thinking is like in that time span. While we are in need of this data to get to where these companies want to be, let's pay these people well. The other part of it is that what's so interesting about the type of data we work with, especially like which is complex systems, heavy science research, social science research, those things are changing and updating all the time. So that's the other thing about that's interesting, at least in the fields that we're in, is that. Even though these human labelers are coming in and they're being augmented by the classifiers we're building, humans are still always in the loop because they have to continually update these models on how human society is changing, how our thinking is changing about things, new research coming out. So I think there is also this continual loop of updating and fine-tuning that does promise jobs in the future too.

Brian Bell (33:06.379)Yeah. What are you guys excited about over the coming years at Reliable?

Annie Brown (33:10.925)Mm, I think well, we're applying to some big grants to hopefully do like some citywide work with our coalition partners, especially around getting in like AI for cities, right? Like there are all these smart cities coming out, some that are in our pipeline, our sales pipeline, but I think the like s AI at the local level is very exciting or interesting to me personally just

Brian Bell (33:42.882)Yeah. Stand up MCP servers basically for all government data. I had a founder send me, I I tweeted about this recently, but I had a founder send me their data room as an MCP server. I was like I was like, that's awesome. That's cool. That is that's the first time that's happened. Yeah. It just happened like last week.

Annie Brown (33:48.649)Yeah, well and just making huh. Cool. All right, I'll if that stands out to investors, I'll start doing that. no, but yeah, I think I think things like that, like the local level are interesting to me because that's like the type of nuance we're trying to get at. And these AI systems are, you know, gonna be built at every level of government, every level of business. I think another thing that I'm just very excited about is our classifier. so the AI that we are building, has been performing very well with very little data, kind of as the hypothesis that we have states is that, you know, if we bring in more nuanced labeling, we'll get this AI to be, you know, very accurate very quickly without having to, you know, invest years and years into data buildup. So yeah, I'm excited about that. We've got some big customers in the pipeline and we just kicked off our round. So you know, I think we're at this really cool hypergrowth inflection point for reliable and I think it's a it's a point where every founder you know hopes they I guess get to because I think the early, early stages are some of the most difficult, but then this you kind of see all your hard work coming together.

Brian Bell (35:12.055)Yeah, when you start getting product market fit, you kind of start feeling it. You're like, I'm selling this easier now. And it's much more fully featured. And people are like buying this a lot more. so yeah, I have a product that people buy and they come back and use it. that's really cool. And they're not churning, they keep paying me every month. that's really cool. Do you think?

Annie Brown (35:15.401)Yeah. Yes. Mm hmm. I have a product. Yes, yes. So yeah, we're at we're at a very exciting point in the startup journey in general.

Brian Bell (35:37.718)Long like if you think long term, do you think we're annotating into the heat death of the universe? Or at some point is like AI like so smart and so good and so cheap. we're and we're certainly building out data centers like left and right and compute clusters all over the place. And I think it it'll probably take a few more years to play out. I think we're we're probably like in 96 internet, is what it feels like to me. having been 16 in 1996, I I I kind of remember what it felt like. And I feel like AI is

Annie Brown (35:51.287)Sure. Uh-huh, uh-huh. Yeah. Uh-huh. Sure, sorry.

Brian Bell (36:07.223)Probably in ninety six, ninety seven. So I think we're still two or three years from like overbuilding and like, crap, we overdid it again. Like we always overdo it as a soci we'll go too far. But like in the future you can imagine tokens being almost too cheap to meter and really super intelligent. You can already see like the intelligence density per token per is going up. Right. And you know it there's various estimates, but

Annie Brown (36:13.195)Big boot bomb. Right. Hmm. Sure. Yeah.

Brian Bell (36:36.383)AI gets 10 to 20 X more price performance per unit of intelligence delivered every year, according to different estimates. It's been doing that for a long time. So at some point it's like annotation unnecessary. is the short version of that question, that rambling question I just gave you.

Annie Brown (36:42.327)Mm-hmm. Mm. Mm-hmm. Yeah, I think there's a well, I I love that question because it brought in a lot of interesting things I think about a lot with AI. Cause I do think we're I think I'm an AI optimist, I would say, on the scale and spectrum. I think I I think I have to be for the work that I do, right? But I do think that there is there is the doing too much without understanding it.

Brian Bell (37:07.969)That's my next question. Yeah. Yeah.

Annie Brown (37:17.739)Fault like data centers I am probably one of the things I'm most worried about with AI is data centers just because we're building them so fast when we don't even like 'cause there's probably just a few years out from Yeah, exactly. Like we're building these

Brian Bell (37:29.067)We haven't even like optimized the stack yet, right? We haven't optimized the memory to chip bandwidth and there's all kinds of different architectures and for LLMs that we have we haven't replicated the we did this with some other AI stuff ten years ago where you actually adapt a chip to the model. And I don't think they've fully done that yet with large language models.

Annie Brown (37:39.265) Yeah. Interesting. No, it's all it's all still so early technology. I mean, you think about ninety-seven, you think about ninety-seven internet, and we've got those big computers, we've got Ethernet cables, the whole the whole thing. And so we're still at that early stage. So it's like to build all these kind of like industrial, kind of inefficient data centers doesn't make a ton of sense. There's a really good report that the Kapoor Center put out on like just needing to get more data on data centers, you know, get like

Brian Bell (38:02.773)Yeah.

Annie Brown (38:22.325)Give us a little time to investigate some of these things, to figure out what the most efficient, what the most sustainable way of doing things is, I think is important. And I kinda think the same thing about with data annotation too. Cause like I said, I think there needs to be a balance between you know, working towards these classifiers, because reliable does have this AI strategy where, you know, if magically we train it, you know, It takes time, but let's say magically we come with the right combination and we train a classifier that can get every contextual system right, we would still want humans in the loop. And I think there's also still this advocation of if we are using jobs or if we are using AI to reduce the workload of our workers, can we give that time back to the workers to just have a better life as opposed to, you know, unemploying them or putting I think that's where it kind of AI and the universal income conversation kind of come together too in you know in these big lofty talks about AI. But I think we're still so far from that it's there's there's time. I also think just the way the models work, again, models right now and AI right now is not very good at nuance. So if you're asking it to do really anything with nuance, you're always gonna need that machine learning, fine-tuning. layer and training layer in there. but, you know, maybe reliable will be the first to make a classifier that can i identify everything and then, you know, and then we'll have enough money that we can give out a personal, you know, universal income to everybody in San Francisco or something. So

Brian Bell (40:06.133)Yeah. what's your P doom?

Annie Brown (40:11.007)Yes, the P doom. So I just want to clarify the P Doom. So is it like my point of view?

Brian Bell (40:18.999)Your probability super intelligent AI kills us all. And ends humanity. Like ends human existence. E Doom.

Annie Brown (40:22.018)Okay, okay, okay. Okay. I think Honestly, I think it's higher than I'd like it to be just because just because I don't I don't love the way I don't like the way that the biggest AI companies are handling this. the the extreme wealth that's being created.

Brian Bell (40:33.847)Yeah. Wow. Well, they're they're trying to throw off their own like culpability. And you know, they're trying to say like, We are building something so dangerous, you need to regulate us, because we can't build safe products. That's basically their argument.

Annie Brown (40:50.783)Yes, yes. But then yeah, but then building lobbying firms that make it hard to regul regulate. So it's like regulate us but don't.

Brian Bell (41:05.195)Yeah, yeah, yeah, totally. And at the same time, they're back channeling saying don't regulate us. Yeah. So to me it smells like marketing. I'll say personally, my PDM is zero. I I I I find it very, very unlikely. Basically, as close to zero. I mean, there's never there's never P you can never get to zero for probability, but it's as close to zero as you can get because super intelligent AI wouldn't even want to stay on the planet first of all, because like there's so much more energy in space in space.

Annie Brown (41:09.023)Exactly, exactly. Yes. Really? Uh-huh. Is that just because of the speed of it? Sure, sure. Yeah. Earth is pretty cool. I mean, I love Earth.

Brian Bell (41:36.096)Yeah, but it's it's we're we're adapted to like live here, right? We evolved or created, if you believe that, to like live here. but AI, it only all it needs is like computronium, like compute substrate and energy. And you can find a lot more of that out there. and so like it'd be it'd be us it'd like be it'd be like us like trying to live at the bottom of the ocean. Like we could do it, but it's it's like

Annie Brown (41:38.861)Mm, mm-hmm. Sure. huh. Mm-hmm. Interesting. Yeah.

Brian Bell (42:02.463)Why do that? It's not the optimal place for us to be. We want to be up where the sun is and all the plants and vegetation, where the food is. Same thing with AI. Like Earth will be like the deepest, darkest ocean for them.

Annie Brown (42:07.157)That's true. Yeah. Yeah, I'm not I'm not so worried I mean, super intelligence I don't think would want to hurt us 'cause I think if it's truly super intelligence it's gonna be quite compassionate. Yeah, exactly. It's gonna be very compassionate. I mean I I think like, you know, if reliable is used to build that, you know, software. Yeah.

Brian Bell (42:20.663)Yeah, it'll just reason through and go, like, why would I do that? Yeah. I think I think so. I think like I think in the extreme like Kantian philosopher sense, right? It would kind of go, what's the categorical imperative here? And it would be extremely rational and truth seeking. And it would go, I I I shouldn't do that because it's not the right thing to do. Like it would just like have morals like that, I think, because it'd be able to reason through from first principles everything.

Annie Brown (42:37.397)Yeah. Right. Yeah. I I agree. Yeah. Yeah. Yeah. I think I'm not worried about the superintelligence. What I'm worried about is the people building the superintelligence if they destroy us before we're able to get to the superintelligence. I think is and it just it kind of just all comes back to sustainability, right? And just like you're building this really cool technology, but what does it matter if we're not here to use it or to see it develop? Yeah.

Brian Bell (42:56.833)Yeah. You know, fun fact, I don't know if I've told this story on the podcast, but it's just a really good time to tell it because we're talking about tech and sustainability. I think in your in your twenties, like you have to like be on that train. I was, I was director of energy policy for Sierra Club for the state of Hawaii. And so I was lobbying the government for clean energy legislation and passing like solar roof bills and stuff like that. Really fulfilling work. And so when I came to San Francisco, I actually didn't even want to work in tech. I wanted to work in solar energy.

Annie Brown (43:15.895)Mm-hmm. Hmm. okay. Uh-huh.

Brian Bell (43:40.726)That's that was like my first tech job was solar. but what I what I what I realized as I started looking at the data is that solar was doubling at a very predictable rate. And then soon, and probably around 2030, 2035, we'd have more than enough solar energy to power the whole country. and every time it was doubling very predictably, I think every at that point.

Annie Brown (43:40.991)Okay. Mm-hmm. Mm-hmm. Is that Right. Mm-hmm.

Brian Bell (44:08.279)Twenty-four months was in installed capacity. And so if you just think exponentially and you just you just plot the exponential, you're like, well, energy's solved. Right. And I think I think a lot of things like that with that's why I'm such a techno optimist. I think a lot of things, tech creates its own problems, it always does. but on the whole, I don't think anybody wants to go back 50 years and live with 19, you know.

Annie Brown (44:11.276)Mm-hmm. Yeah. No. Mm. No.

Brian Bell (44:36.189)Sixty whatever technology, right? Nobody wants to do that.

Annie Brown (44:37.985)Well, we love this technology. We're talking because of this technology and I've loved this conversation, right? So I think that's that's what it is. I I'm very much a tech optimist, but I do think that tech needs to be built responsibly. I also think like with it's like help us reach the full potential of the technology. Because that's my thing is like with reliable, it's like we can't even get to the coolest parts of the technology that we wanted unless we have this, but we do have the

Brian Bell (44:41.335)Right. Yeah. We couldn't be doing this podcast right now. Yeah.

Annie Brown (45:07.691)I think a lot of times people have said it's impossible to do that. And what we're trying to say is it's not impossible. You know, you just need the right software to do it. And so I it's a very exciting time to be alive, not to end the podcast on a cliche note, but it is very, very exciting because I think we have the opportunity to achieve to solve really big challenging problems.

Brian Bell (45:19.444)Yeah. I think we'll look back. I think we'll look back 10, 15 years from now, because the next 10 years is going to be really crazy. And we'll go, yeah, hell no, I don't want to go back to 2026 because of all the discoveries AI will make, all the all the, you know, menial jobs that will be automated away with robots and AI. will people be able to find fulfillment? I read a whole article on this actually. Yeah. We have an example of that. People like in their like late fifties and sixties that retire as long as they're healthy. They're

Annie Brown (45:32.129)Mm-hmm. Yeah. No, Yep. Is that

Brian Bell (45:57.43)Really happy. They're super happy. They're happier than they've ever been in their lives. So

Annie Brown (45:59.468)Love to have. Well that's the thing. People are naturally driven. They'll do stuff that like interests them that excites them. yeah, for sure. I'd love I'd love more time for hobbies. I don't have enough time for my hobbies, so Okay, cool.

Brian Bell (46:04.375)They'll just they'll have hobbies. Yeah. Yeah. I could totally fill my time. Yeah. Exactly. well let's let's do some wrap up questions. what's something you believed with total conviction five years ago that you'd now argue against?

Annie Brown (46:21.558)Ooh, what a good question. I would say I would Let me think on this. So I think I would argue against Five years ago. I think I would argue against the Well, this is a little okay, so this is a little extreme, but I'm I'm a little of an extreme person. So I would say five years ago, I would have been much more I was much, much more gung ho on electoral politics. I think I was very much like, you know, like that is the best way to, you know, solve societal problems. I think living in San Francisco and seeing how much tech moves things here. and also seeing like the ki like direct community work I do, like I I volunteer and things like that and the impact that can have made me very excited about, you know, my role as an individual in a city and the impact I'm able to have. And also kind of made me s put less time into like politics or an energy towards that and more just into direct things I can do with my community and the work that I do. And that's not to say I don't vote or I don't encourage people to do that, but just for me personally, it's it's been helpful 'cause it helps me feel like I'm doing positive things without having to follow so carefully the, you know, kind of waves of politics, I guess.

Brian Bell (48:04.811)Yeah. Yeah, I think you know, we have a not to get political here, but we have a Republican government, which means it's a republic, which means we elect representatives. but what's what's interesting about it is that the Senate is completely disconnected from the population. So you could have two senators from Alaska, two senators from Wyoming, like and those people in those states are getting a disproportionate influence on politics.

Annie Brown (48:10.795)Mm-hmm. Yeah. Yes. Uh-huh. Mm-hmm.

Brian Bell (48:33.803)Versus like a st a state like California or Texas, also two senators each with thirty million people each or whatever it is. Thirty so yeah, that's a very interesting design pattern. But I I actually learned something interesting about the Senate recently is that we didn't even elect our senators until recently, the state legislatures. yeah, it was like fifty, sixty years ago that actually people started even electing senators. So before that, it was the people you elected electing the senators, almost like prime ministers.

Annie Brown (48:37.069)Yeah. Yeah. Mm-hmm. Mm-hmm. Yeah, 'cause it's the Senate was supposed Yeah. 'Cause the Senate was supposed to be a check on the people, kind of. Yeah.

Brian Bell (49:05.899)The people, yeah, that's how they designed it. Yeah. Sort of like a a blunt to the mass populist ideas. Yeah.

Annie Brown (49:11.797)Which, you know, question mark about but the it's funny 'cause it's like even talking about that, my entrepreneur brain is like, How do we innovate to, you know, solve that? But it's like, Yeah, exactly.

Brian Bell (49:20.629)Yeah, yeah. How do we fix government? I actually I I haven't done this in a while. I I asked AI that in like deep thinking mode. Like how if you were a founding father or mother, I guess. if you were like a f like if you could do it all over again, just reboot the whole like the whole democracy, how would what kind of changes would you make? And I put it like then like max thinking mode. And this is probably a year ago, so it's pro the answer would probably be even better now, but it had a lot of really good.

Annie Brown (49:27.06)Interesting. Uh-huh. huh. Yeah. Restart. Mm-hmm. Yeah.

Brian Bell (49:49.816)Tweaks. I think I think getting money out of politics and then that's like job number one. And then two would be I lost my train of thought.

Annie Brown (49:50.965)Mm. Mm-hmm. Well, I mean the whole Senate, the electoral assembly and stuff has pr but it's just like, you know, in a startup running a startup, you would be like, Okay, let's like pilot this, let's move really fast, but government just doesn't do that. So I think that's one of the reasons why I was always drawn to entrepreneurship. Cause I feel like I can, you know, make influence like fa much faster than through kind of these more right. Right. Yeah. I didn't know that.

Brian Bell (50:25.299)Balanced budget. That's what I I remember. Yeah. The states have to balance their budgets, but the federal government does not. and I think it's okay, like in emergency situations like war or pandemics, to run a deficit, but we are running such a deficit now that it's like putting the future of this country in peril. So anyway, I'll get off my political soapbox.

Annie Brown (50:37.969)Huh. Sure. Yeah. No, well it well hey, if the US government wants to hire me to build an ontology to balance the budget, I'm here for it. So

Brian Bell (50:55.415)There you go. Yeah. Not hot dog. what's a popular idea in AI or data that you think is wrong?

Annie Brown (50:59.584)Ha ha. Mm. Yeah, this actually came up at a recent event. like the audience was like kind of torn about it, that I was at, but that more data will fix the problem, I think is a is a concept that is like pretty that was like kind of the go to standard until like just a few years ago. I will say reliable was talking about it before just a few years ago, but this idea that basically if you just push more and more

Brian Bell (51:36.801)Just throw more data at it, right? Yeah.

Annie Brown (51:38.304)Yeah, like it'll fix the bias problem, it'll fix the racism problem, it'll fix the mathematical problems. Like would we just get more data, have more mathematics, PhD students label it, it'll be fine. But the thing is, is it's you're pushing more data through a tainted system and then it's coming out tainted and you're wondering why. It's just like it doesn't matter how much you push through, it's all it's always gonna create the same residue from that kind of icky system you have. So you

Brian Bell (52:05.409)Yeah. It's like washing your car with dirt dirty water. Just keep washing it. Yeah, it'll get cleaner. No, it's like dirty water.

Annie Brown (52:08.181)Yeah, exactly. And then you run out of water and you're like, Why is my car not clean? So I think that's that's my like main thing is that we have to look at the quality of the data and not just like the data sets themselves, but how we are preparing that data for training. And I think once we do that, a lot of the efficiency questions will also be kind of answered too.

Brian Bell (52:30.923)Yeah. what's the best advice you ever got?

Annie Brown (52:35.809)Hmm, best advice I ever got, I would say. If it counts as advice, but I've just had I've had a lot of professors that like just kind of encouraged me to follow weird trains of thought, that I think has been very helpful. because you know, just being curious, I guess, I think is the general advice I've gotten from some of my best professors, is just like. If you're if there's something kind of tugging at you that you really like want to understand more, just like go for it. Even if it's a weird thing to dig into, or even if people tell you like, like I think you're wrong, like if it's really tugging at you or pulling at you, like just go for it. And that curiosity had like led me to, you know, start my startup. I I would say, like my first startup and like continue to be in startups.

Brian Bell (53:40.471)You saw something in you before you saw it in yourself.

Annie Brown (53:45.034)That's a nice question. I would say I I would say, well, I'd say my mom. My mom has always been very proud of me. So that's a good one. And then my professors at William and Mary, because the my first startup actually started as a an undergraduate project. And so they were very excited about it. They like wanted to talk to me about it. And then it ended up growing to, you know, several chapters across the southeast United States and then into the social media platform and I don't think I would have done that if they hadn't told me like, Hey, this is really cool, you should keep, you know, pursuing this.

Brian Bell (54:34.487)I've really enjoyed the conversation.

Annie Brown (54:36.097)Yeah, I want to ask all these questions back to you, but I know we don't know we don't have time, but I'll I'll get those answers another time. Yes, so they can find us and all of our socials if you go to reliable.ai and then all of our socials are on that site, and then you can find

Brian Bell (54:38.539)Yeah. Where can folks find you online? It's not reliable how you would spell it normally. It's missing yeah.

Annie Brown (54:57.645)Good point. Yes. Yes. It's reliable without the E dot AI. and then because we are a tech company, so we had to drop the vowel. and then Yes, exactly. and then on personal socials, you can find me at Andrea Francis B and mostly on LinkedIn is my social media of choice these days.

Brian Bell (55:07.659)Yeah. Yeah, it's something fancy. Easier to get the domain that way too. Much more positive on LinkedIn. People just keep it positive there.

Annie Brown (55:27.565)I like it. It's I just get to talk about all the nerdy stuff I wanna talk about without trolls calling me a nerd.

Brian Bell (55:33.218)Yeah. Yeah. Yeah. I totally I yeah. Long story there, but really enjoyed it. yeah. Thank you.

Annie Brown (55:42.148)This has been so fun. Thank you for having me. Yay.

This article is for general informational purposes only and does not constitute investment, legal, tax, or accounting advice, nor an offer or solicitation to buy or sell any security or investment product. Investing involves substantial risk, including possible loss of principal, and past performance is not indicative of future results. Full disclaimer.