//

Women Leading The AI Industry: “The biases of our society emerge in the data that forms our AI systems.” with Sandra Carrico and Tyler Gallagher

The biases of our society emerge in the data that forms our AI systems. Companies that don’t hire women, will have data that encourages resume selection that excludes women. AI systems that act as gatekeepers, such as HR, loan access, and any imaginable background check can perpetuate established biases. As part of my series about […]


The biases of our society emerge in the data that forms our AI systems. Companies that don’t hire women, will have data that encourages resume selection that excludes women. AI systems that act as gatekeepers, such as HR, loan access, and any imaginable background check can perpetuate established biases.

As part of my series about the women who are leading the Artificial Intelligence industry, I had the pleasure of interviewing Sandra Carrico, the VP of Engineering and Chief Data Scientist at GLYNT. In her role of Chief Data Scientist, she invented and led the software development of the machine learning architecture and models for GLYNT’s state-of-the-art data extraction service. As VPE, she has led the engineering team that develops GLYNT’s engineering infrastructure and scalable services.


Thank you so much for doing this with us! Can you share with us the ‘backstory” of how you decided to pursue this career path?

My interest in AI originated in graduate school. I worked on an AI project that tried to use genetic algorithms to identify parts of speech. The math showed it should work, but at that time there weren’t enough compute cycles to support algorithm convergence. I had to put AI away for several decades. In 2007, when I heard Google was getting stunning results with AI, I knew it was time to return to AI. I’m fortunate that I have a supportive husband who loves math. For three years, we attended local classes together and learned how today’s systems work.

While I was at WattzOn, the predecessor to GLYNT, I was actively working on a variety of AI projects that improved our offerings, while also leading the engineering team. The primary problem for engineering was extracting data from utility bills at scale. That included finding detailed data elements such as the total amount due or the number of kilowatts used. As the VP of Engineering, I was concerned about the scalability of system. The system was state-of-the-art for the time, but nevertheless suffered one known limitation: The rules the programmers wrote to extract the data were brittle and broke easily with small changes to the documents.

One day about two years ago everything snapped into place in my mind: The business, the technology, and the opportunity. I focused on the only known limitation of the system, writing the rules to get the data. I recast that problem from programmers writing pattern-matching heuristics to machines finding heuristics. I knew machines learned heuristics better than humans could write them. No one had yet broadly applied machine learning to data extraction from documents and I knew there were several large businesses areas where manual data extraction persisted. GLYNT was born with a mission to extract data.

What lessons can others learn from your story?

Remember those projects that should have worked but failed. Those are valuable, because they indicate the limits of the current technology. When the limitation evaporates, the economy changes. Companies that take advantage of that change can address massive opportunities.

Can you tell our readers about the most interesting projects you are working on now?

I find my work with GLYNT engaging. In part, to achieve our technical success I developed a method called Mixed Formal Learning. I’m exploring further extensions of that method and its theoretical limits. I’m excited by the prospect of machine learning algorithms being applied to a much broader set of problems with dramatically lower data requirements.

None of us are able to achieve success without some help along the way. Is there a particular person who you are grateful towards who helped get you to where you are? Can you share a story about that?

I was fortunate to be at AT&T Bell Labs where I was a member of three major software areas: development, applied research, and research. Throughout my tenure, I was schooled in, and to some extent developed, a management style for production code that eventually led to agile and extreme programming. Bell Labs also taught me how find, evaluate, and lead applied research projects. As an individual contributor, I learned how to read academic papers and develop working software quickly. As a manager, I learned the social structure of development organizations that made the difference between the success and failure of projects.

What are the 5 things that most excite you about the AI industry? Why?

1. A Prosperous Future: I can clearly see a prosperous future enabled by AI. AI strips out the bloat, and will lead to deflation, where the costs of items converge to the costs of inputs plus the cost of design. The price of raw materials will out of necessity rise to control demand on our natural resources. Since the introduction of PCs in the 1970s, computers have become both more powerful and cheaper every year. With AI, I expect most items to follow a similar deflation path while improving in functionality and choice.

2. Smaller Amounts of Time Working: We worry deeply about work today, and perhaps rightfully so. The industrial revolution changed work from a daily task until you died to a 9-to-5 work week with weekends off and retirement. AI will further shorten time spent working, perhaps leading to work weeks of just four to six hours a day three to four days a week. Some of the free time will be dedicated to learning new technologies and how to continue to adapt to constant change. Videos, online classes, and distributed community education will likely become common methods to continue education in almost every field.

3. Freedom to Work Anywhere: Remote work will likely become more common and include an integrated lifestyle of work and family reminiscent of a far earlier time before the invention of the factory. That will spread prosperity more widely, and in those areas with high salaries, remote access to work will bring salaries down.

4. More Sustainable Food Production: Agriculture has been changing since the industrial revolution, and I expect to see even more dramatic changes. It’s already hard to find field labor for weeding, picking, and field preparation. Integrated robots will replace humans for these jobs over the next decade. We already have, or are close to having, the necessary vision and robotic dexterity.

What are the 5 things that concern you about the AI industry? Why?

1. Emotional Stress: AI will disrupt the current status quo. It will force people to adapt, and that will be emotionally painful as they feel the loss of old methods, even damaging and harmful ones. These new methods will bring prosperity, but one person’s cheaper goods is another’s lost wage.

2. Lifelong Learning: AI, and other technology will force lifetime learning on a population that might not find learning altogether fun and interesting.

3. Stress Due to Complexity: While AI will simplify tasks for which it is targeted, the implementation of that system will involve vast complexity. Today, many AI systems work but the implementers can’t explain exactly why they work. Eventually even the users of this technology will start to recognize the complexity of the systems. That complexity and lack of understand will bring uncertainty and angst. On the one hand, it will often work, but on the other hand, understanding and anticipating the failures may prove almost impossible.

4. Culture Shock: AI will rapidly change people’s environments and behaviors. The quickest people will adopt these new technologies, as already happens today, but at a far faster pace across more daily activities. Even with adoption, many people may develop a sense of not belonging to a society they no longer recognize. That might result in antisocial behavior and self-abusive habits such as various addictions.

5. Power Struggles: AI can enable the worst segments of society to leverage its power to oppress those with fewer resources and power. They can do that through force with robots or through pervasive surveillance and predictive techniques for manipulation. When used to power violent machines, AI is a force multiplier only limited by the raw inputs to create it. One evil person could construct an army of killing drones. More subtly, people themselves are fairly predictable once enough data is collected about an individual. Marketing organizations that manage to collect sufficient data can target hot buttons to strongly favor certain behaviors such as buying specific products. Personal targeting will prove more effective in swaying buying, voting, and other behaviors than the demographic targeting used today.

As you know, there is an ongoing debate between prominent scientists, (personified as a debate between Elon Musk and Mark Zuckerberg,) about whether advanced AI has the future potential to pose a danger to humanity. What is your position about this?

The biases of our society emerge in the data that forms our AI systems. Companies that don’t hire women, will have data that encourages resume selection that excludes women. AI systems that act as gatekeepers, such as HR, loan access, and any imaginable background check can perpetuate established biases.

What can be done to prevent such concerns from materializing? And what can be done to assure the public that there is nothing to be concerned about?

It’s essential to continue to allow the proliferation and democratization of AI. Technology as powerful as AI can, at the end of the day, be checked only with other AI technology. At some point, leaders may try to forbid the general population from acquiring AI tools as a way to consolidate more power. Freedom of thought and open access to AI for all at some basic level must prevail.

In addition, gatekeeper AI systems must be regulated to allow access by others in the community to test its predictions. End user agreements that forbid such investigation and that block the publication of any discovered anomalies must be made illegal.

How have you used your success to bring goodness to the world? Can you share a story?

I believe that GLYNT will prove to have the most social impact of anything I’ve done to date. Today a few large companies control the world’s data. The first group consists of large companies that produce data such as utilities for energy consumption data, financial institutions for financial data, and governments for regulatory data. The second group contains companies that infer data as part of a service such as Facebook or Google.

GLYNT, through its low-shot learning feature, democratizes access to data trapped in documents. The small number of supplied training data examples enables the user to map the data in the document to their own names for those fields. With labeled access to data, other AI- and non-AI-based services can form. These diverse companies and services will provide future jobs and gracefully reduce the concentration of power and wealth that exists today.

The AI within GLYNT depends on its infrastructure, which enables data extraction from millions of documents or more. That infrastructure plus the AI’s flexible support for many types of documents means GLYNT can readily serve people and companies at a reasonable cost.

As you know, there are not that many women in your industry. Can you share 3 things that you would you advise to other women in the AI space to thrive?

1. Learn every kind of math you can: AI at the end of the day is math, and control over AI and future developments rely on a strong command of applied and pure mathematics.

2. Learn to program: Programming frees you from dependency on others. There is no doubt that we are interdependent in society, but control of your future in some way depends on the ability to participate in the creation of these devices. AI is implemented by code that is based on mathematics.

3. Find your inner voice for validation: External validation looks backwards. To make outstanding contributions, it’s important to develop a sense of what could be and an internal method to accurately evaluate ideas independently and quickly. To quote Richard Hamming, the well-known computer scientist, in his famous lecture, “You and Your Research,” he said, “Why shouldn’t you do significant things in this one life, however you define significant?… I have to get you to drop modesty and say to yourself, ‘Yes, I would like to do first-class work.’ “

Can you advise what is needed to engage more women into the AI industry?

Anecdotally, I have heard that some highly competitive schools have developed a hostile environment for women. I don’t know how pervasive this problem is, but I don’t recall overt hostility when I attended university. The engineers worked together to learn the material and get the highest scores we could. Of course, people had biases, but they seemed to me derived from lack of experience, rather than an attempt to derail your progress. Once the bias was talked about, most of the attitudes adjusted. The general atmosphere was one of collaboration rather than cut-throat competition. We saw that kind of behavior in other majors, but not in engineering. The issues today seem more focused on excluding certain classes of people from the field.

I believe the grading methodology we had in engineering fostered a positive supportive environment. Professors didn’t fit the scores to a curve. They divided the class scores according to the clusters that formed in the performance scores. If everyone clumped at the top, everyone got A’s. That meant when we helped each other our group would ideally cluster together. If we did a good job helping each other, we all ended up in the A cluster.

For the classes in general, there were usually clear clusters of scores with good separation between the clusters. Many undergraduate classes typically contained three or more clusters. As the classes approached graduate level there were typically only two clusters.

What is your favorite “Life Lesson Quote”? Can you share a story of how that had relevance to your own life?

Richard Hamming said in“You and Your Research,” “I finally adopted what I called ‘Great Thoughts Time.’ When I went to lunch Friday noon, I would only discuss great thoughts after that.” That insight taught me to always carve out time to think deeply about problems I was facing in my field. The Great Thoughts Time tackled a deeper question, “What are the most important problems in my field?”. It was this discipline that led me back to AI after decades of keeping it on the back burner, and to a solution that became GLYNT

You are a person of great influence. If you could start a movement that would bring the most amount of good to the most amount of people, what would that be? You never know what your idea can trigger. 🙂

I think other communities could learn from Silicon Valley’s response to the deep technology downturn in 2000. During that time, there were no jobs posted. No one thought it was possible for jobs to evaporate. In response, the Valley’s companies worked with unemployed people, civic leaders, and various internal facilities teams to provide free spaces for continuing education and networking for new jobs. The spaces volunteered included corporate auditoriums, restaurants, and public spaces. The owners opened these areas to organizers for their use during periods when they would normally be empty or almost empty. For restaurants, the attendees purchased small goods such as coffee to help compensate the owner.

Whether employed or not, people held free public social events for networking, to explore the next areas for eventual job growth, and training. Volunteers taught classes or workshops to help people retool. The grass roots volunteerism gave it additional power, breaking down the divisions between leaders and participants. Some of the most successful efforts were initiated by unemployed members of the community. They had time to organize and publicize. The employed, who had less time, helped by donating spaces and other tangible goods.

The community’s attitude was also important. Everyone recognized they could have easily been the people that were unemployed. They weren’t special, just lucky. They also believed to the point of knowing that the downturn would end, and people would get new jobs. That optimism spread to those who could not find jobs and provided hope.

Much of that organizational infrastructure continues today in Silicon Valley. I benefited from it when I wanted to learn AI. Those classes were hosted at the hacker dojo and taught by volunteers Mike Bowles and Patricia Hoffman. Eventually Patricia moved on and Mike continued. As the recovery gained traction, Mike eventually asked for a small amount of tuition as compensation for his substantial efforts from those who could pay. Today, I continue to benefit from his weekly AI paper reading and the community that gathers at the hacker dojo. We can read and understand papers together at a level that would be impossible alone. As technology continues to change society, establishing this kind of infrastructure in communities will help foster resilience, establish new companies, and retrain people.

How can our readers follow you on social media?

This was very inspiring. Thank you so much for joining us!

Share your comments below. Please read our commenting guidelines before posting. If you have a concern about a comment, report it here.

You might also like...

Sign up for the Thrive Global newsletter

Will be used in accordance with our privacy policy.

Thrive Global
People look for retreats for themselves, in the country, by the coast, or in the hills . . . There is nowhere that a person can find a more peaceful and trouble-free retreat than in his own mind. . . . So constantly give yourself this retreat, and renew yourself.

- MARCUS AURELIUS

We use cookies on our site to give you the best experience possible. By continuing to browse the site, you agree to this use. For more information on how we use cookies, see our Privacy Policy.