PANEL: Designing European AI Agency: How Sovereign Intelligence Can Still Be Built

So now we're going to have something completely different. We're going to have a panel with some of you will know a gentleman called Joerg Resch. I think you may know him because it is Joerg who is really responsible for the existence of this conference. He created it, he developed it, and now he's working his magic for the German government in something called SPRIND. And the other participant is Elie from Belgium, who's a lawyer, which is always amazing because she's been able to take time off the clock to come and come and talk to us about this.

So this is Designing European AI Agency, How Sovereign Intelligence Can Be Built. So please, will you take your seats in the front? And we should also have another gentleman, Dr. David Scott, coming from Washington State.

And hello, David. Thank you very much for joining us.

Now, the idea of a panel is, in fact, for it to be interactive. And I know that Katrina is extremely interactive on these occasions, but I hope you will also have some questions for this. So let me just introduce everyone. Dr. David Scott is Executive Director of Information Risk and Synthetic Intelligence Research at the University of Washington, that's Washington State. So thank you very much for getting up at two o'clock in the morning, I think it is over there. And Emily van der Lande, Cyber Risk Consultant with a background in law and cyber security.

And I think I've already introduced Jörg Resch. So to set the scene for this, Anthropic's recent Mythos system card documents, a frontier model that can identify the constraints placed on it and develop strategies to confirm, to circumvent them and conceal having done so. So that is the, so I'm afraid.

Oh, it's over. Thank you. You've stopped it. So within months, the idea is that agents will be able to sign contracts, access systems and make payments on our behalf. And the rules that govern these actions sit in PDFs that are interpreted by lawyers and enforced after the fact. What would this mean to regulation itself? Executable by humans, executable by machines and enforced in real time. So what I'd like to do is to ask each of the panel members in turn to put their position on this. So perhaps we'll start with Emily.

Well, that's a big question. I think it adds a whole new layer to sovereignty. We always think about sovereignty in terms of states and sovereignty in terms of organizations. I think that when we start working with agents, we become organizations and independent individual sovereignty has equal amounts of restraints and opportunities and risks as a government would. So yay. Thank you. So Jörg. Thank you. Maybe one word in advance regarding the discussion and the presentation of Cedric, which I really liked.

With the ICC case and similar occurrences we had last year or the last few years, I think something is broken. It is trust. Trust is broken in a world where we didn't expect it to break. This is completely changing the discussion we are forwarding over here. It is changing the way how we look at things like sovereignty. Nevertheless, I think we have to be careful to not mix sovereignty with protectionism. We still need to think globally. And the Americans won't let us down on thinking globally. It is so important for any business anywhere in the world to use the best tools available.

And we at Sprint, the German federal agency for breakthrough innovation, we try to fund those ideas, those innovations that really make a difference. Our task is to help the supplier side improve their offerings in a way that they can globally compete. Thank you. Thank you.

And so, Scott. Yes, thank you. Can you hear me okay? Can someone wave their hand?

Okay, perfect. So I don't live in Europe and I'm not in a position to comment as a European, but I would like to emphasize some commercial and economic pathways to power. I want to talk about power as part of sovereignty. In our work and a number of work streams, we look at the source of the challenges as being the source of the solutions as well. So I asked the question of the folks in the room, how might European sovereignty be facilitated by each of you in the room as representatives of commercial actors, not just relying upon governments to be responsible for sovereignty?

And what are the new markets, products, services through which European power and sovereignty across business, operating, legal, technical, and social domains, we call that BOLTs, can be sustainably maintained? We're going to introduce just briefly three separate markets. One is information markets, new value beyond inert data markets. Second is network effect markets in the value creation and extraction in information markets can go beyond physical enclosure.

And then lastly, this touches on something Jorg has touched on, is synthetic intelligence markets, the value of domestication of wild AI. How can we better work with AI to assure that there's a balances of power in economics and other domains? Thanks. Thank you. So are there any questions from the audience at this point? There's none online, so I'm going to ask a question. So we have seen, and it's interesting that Jorg talked about the need to use the best that is available.

So do you believe that regulation or what could change the way Europe works that would allow Europe to be the instigator of these developments? What do you think, Jorg?

Oh, that's a very interesting question. Most of it I will hand over to Emily, because it has got quite some regulation, legal kind of aspects that I'm not an expert in. But as I said, it is a European task to improve in competing, especially in the AI world, in a way that the value creation part can also take place locally. But on the other hand, trying to, how should I put it, make sure that the core of what is coming around with AI and agent systems that we have, that it is open sourced in a way as a contrary to those tech pros who are mainly placed in America and China.

But maybe you go deeper into the legislation and regulation part of the topic. Yeah, so this is a tough one. Europe loves rules. I think we all, we can all agree on that in the room. And I come from a legal background. And actually, I want to take a bit of a more practical tone on this panel. I think that the next wave of regulation, and we already have a next wave, we have the AI-EU Act that will come into full effect in August.

Oh, it's a hard one. It's a really, really hard one. I think that if we want, going back to the original question of how we can foster innovation on European soil, I'd say we need to have a more practical aspect. I have a few stories that can illustrate this. So innovation comes from established companies, and it comes from new companies. And I think that both of these are struggling to find innovation in Europe for a few reasons. I was speaking to a company that I work with who is a very established company here in Germany. It's a multinational, 4,000 employees.

And they were like, okay, let's use AI. Let's actually hone that advantage. And they went through the whole process. And if you work in corporate, you know how hard it is to get anything signed off, let alone agents and AI. They managed to get all the boxes. Everything was ticked. Everything was ready. And when they wanted to swipe their credit card to get Claude licenses, it bounced. Why did it bounce? This was two weeks ago. They were very frustrated because there wasn't enough GPU clusters in Europe to handle the demand of this company.

So it's like when you actually have corporations that want to use the advantage but cannot because under GDPR, the inference, AI inference must be done in Europe. And we don't have enough GPUs. That's a problem. I have another story to illustrate this. I have a story of people who did follow the boxes. All the boxes were green. They got a Claude license. And even with a Claude license in Europe, the developers were found using IP addresses in the States to route the AI workloads for the States because the speed was six to seven times faster.

Which means that even when we have the rules, even when we have the green lights, even when we have the will and we want to be kosher and follow the law, we end up still not using European technology. So that's on the established corporation side. So you may go like you work with startups. Maybe with startups it's more nimble, right? Maybe they can be more agile. One of my close friends is actually at the top tier talent in the country and he's working on AI hardware and he wanted to desperately to build in Europe and to have his company in Germany and in Europe specifically.

And he said, I had to go to the States. He tried. He wanted to do it at home, but he said, I need to go to the States simply because when I went to the research institutions, they wanted to take 60 to 70 percent of his IP as a founder, which fine, if you throw buckets of money at him, he'll do it because he wants to do it in Europe. But we don't have the funds and it would repel any VC to invest in his company. So he was forced to go to the United States. And I mean, I can go on and on and on as like a person with a legal background.

I'm kind of hurting a little bit because I think that we forget that having values and having democratic values, it's it's a luxury. And the question is, can we afford that luxury?

And yeah, what concessions do we need to make? So I don't know that answers the question at all. But if we need an innovative, you know, next step for Europe, we need to really put our thinking caps on and think what we want to do. So in summary, that was lack of infrastructure was your first case. And the second one was to do with commercial terms that made it impossible to raise the venture capital effectively. I think it's it's overregulation as well. Another reason that they didn't manage to raise funds is because even when you have VCs who want and I said I'd be practical, right?

Let's jump on the practical side for a little bit. Even if AI companies would want to take VC money, European investors are really afraid of investing because they're afraid they'd be sued out of existence from the start versus the American way, which is a lot more aggressive. It's like you build and then you pay. Antropic did this as well in their recent court hearing came out that they had to pay 1.5 billion, I think, to to settle authors that went against them for training on their data.

Well, yeah, they're big enough to be able to pay that off. But first they were commercial. And we're not doing that in Europe, because we're all very scared to be slapped on the wrist. Thank you.

So Scott, what's your perspective from the other side of the Atlantic? I'm much more optimistic about Europe, it seems. So a number of things. First of all, sovereignty and power are unbundled now.

You know, no nation or no company or no individual can be entirely independent in all domains. And so I think there's a lot of work that's been done in identity and certifications and European norms that would shine nicely in new markets and products and services. And in a sense, if you think about it, Europe itself, you know, in 1848, you had the Italian and German unification, which was a sovereignty shift. You had the EU, which was a sovereignty shift. And that continues to this day. And so Europe already is an information engine.

And I use that word for anything where you take conflict and turn it into power, like an engine does with an internal combustion engine does with fuel, it can explode, or you can have a controlled burn. And so I think a lot of what Europe has to offer is a lot of history and experience with those management of conflict. And there's a lot of conflict in the world still, there's obviously successes from some companies.

But if commercial companies or commercial operations were able to undermine sovereignty, then perhaps commercial operations in Europe can be part of the recapture of sovereignty is my assertion. Okay, thank you. So are there any questions from the audience?

Yes, a question. I'd like to bring in some of something like a cultural perspective on this. If it's true what Jörg said, that trust is broken, then it's the reign of fear in Europe. And fear is not a good prediction of the future. It's just everybody's holding back his money, everybody is not investing in your inventions. So we're like, a bit like stalemated in this fear of coming up frictions in the global power system. And this is not a very good building ground for new innovation for sovereignty of European nation states and European vendors.

So my question is, how do we come out of this trap of fear? Thank you, very good question. It's a risk, and we have to be aware of that risk and try to avoid it as good as we can, to not act out of fear. In what I said, that some trust is broken, that's also a chance of doing things better than we used to do in former times. And sovereignty is one of those aspects we really need to do better.

And doing better means that we have to think back on our core forces and competences we have over here in Europe, and maybe even better than some other regions, like for example regulation, bringing regulation to a next level where also machines would understand what democratic legitimacy means in a more automated way, that our governments would be able to work more or less autonomous, but still be on the democratic side of things, not being governed by some large companies. These are things that are very important learnings coming out of that broken trust thing.

We, for example, at Sprint are working on something we call the Law as Code initiative, where we will try to publish any kind of law over here in Germany, and internationally we are working with OECD together to do that similar process in other countries as well, publish law in a machine-executable way in addition to what we humans can read and understand, so that we have a system that is the core of future automation wherever law is touched and might change the life of us human beings. And this is in a very future-oriented, modern way, capable to authorize agents to do things and all that.

We need to create a standard around this, and we all know that this is one of the core competences over here in Europe, to create guardrails and standards around this kind of profound development we see going around, and which is one of the of the main discussions we have over here at EIC, how can I make sure that those agents around me that make technology suddenly proactive don't destroy my life. Thank you, perhaps Yemen.

Yeah, that's a good question, the whole fear thing, how do you go against fear? I think inherently, like, let's all just stand still by one thing right now all together and be a little bit scared, because it is a little bit scary, and I hate being pessimistic, so I'll try to be optimistic afterwards, but bear with me.

So this is something I actually learned from Scott, we were talking on your patio a while ago, and change right now is exponential, there's an exponential acceleration in technology, which means, like, everyone's going like, yeah, yeah, I've seen that curve, everyone's seen the curve. What does the curve mean? The curve means that if you look at the last 20 years, the iPhone came among us.

If you think about the social, technical, legal, all the revolutions that happened by having a little slap of metal in your pocket, now picture all of that change, all of that structural change across people and organizations, and condense that in three years or five years. All of that changed in five years, and that's the next five years. Exponential means that you'll get the iPhone revolution after that in one year, because it's exponential.

So in that system, our social structures are creaking, everything is changing, we're changing the world in real time, so that's scary, because we don't like change. Firstly, scary, fear, everyone's scared.

Now, to counter that, what is the opposite of fear? I think the opposite of fear is love, and I think in sovereign Europe, in Europe, like my friend who starts this startup out of love, he wanted to do it on home ground in Germany, but he was pushed away.

I think that maybe, you know, we need to just swallow the fear, deal with it, and push out of love instead, and love makes you do sometimes irrational things, it makes you take risks, but it makes you fight for something, because if we don't, if we act out of fear, you know, I don't know what they'll be left to fight for Europe-wise, like, does this system even care about Europe? Like, I'm just putting the question out there, if this change is really exponential, and it continues.

Thank you, and what about Scott? How do we deal with fear?

So fear, and I was an attorney also for 30 years, and fear, and I talk about risk in the legal context, as a legal word that sort of equates with fear, and risk is a very great motivator of people, and one of the things that everyone should understand is that the people in the US and other countries are no more confident about all this than the Europeans.

The companies in the US may be confident, but the people still are very, there's a lot of anxiety about this, and so the shared notion of risk and fear is the thing that can be cultivated, and can be cultivated with things like Jörg is describing, where you have this kind of mutual domestication, just like we domesticated chickens and crops, and it changed both the chickens and the humans as a result, and we're undergoing that now.

I'm not saying AI is alive, but it is wild, and there's a Jörg system, the system that he's working on with Professor Reidenbach and others, will create a platform for mutual domestication of computational intelligence, AI, and symbolic intelligence, and that's precisely the kind of thing that people around the world can get behind, notwithstanding the fact that there's some power fluctuations that are happening in different nations for different historical trajectories.

People are still anxious about this, about what's happening, and the opportunity for risk, most institutions, if you think about it, a bank or a hospital, they're clusters of risk mitigation practices.

Grocery store brings you food, traffic system keeps car crashes from happening, so one of the contemplations I think that it's time for us to work on, and EIC has been doing this for years in the identity work and other domains, it's a place to convene and start to say, okay, oh, I'm afraid of this, oh, you're afraid of this also, what can we do together to reduce this risk, this fear that we can't do alone, and that all institutions are based upon the accumulation of those practices.

In my keynote in EIC, and I think it was 2024, maybe 2023, I talked about the four-step ladder of institution construction and asserted that all human institutions come together from the sharing of practices, and so we start by sharing risk and say, hey, you're anxious, I'm anxious, what are you doing about it, what are you doing about it, let's compare notes, and then Europe is great at regulation and certification, let's create some certifications for those practices that are seen by multiple groups within and outside Europe as mitigating that fear and risk. Okay, thank you, Scott.

One more question from the audience. So this is a great conversation, and we're talking about fear and risk and democracy, and we keep talking about it as if we would just overcome the fear, but what if the fear is justified?

I mean, the improvements that AI has given us are like suicidal teens and loss of jobs and data centers that are killing people, and perhaps the legislation in Europe is reflecting what people democratically want, and our job is to make this technology safe, not to accuse people of being afraid of it, and so the real question is, what do we do to make this safe? And in the identity world, again, we're having all these conflicts, is AI, does it get its own identity, is it just a workflow?

You're talking at the beginning about like, AI is going to make decisions for me and do things for me, I'm like, oh yeah, like, I don't want that, I already can't get any customer service anywhere, it's just, it's actually bad, it's not actually progress, at least not my dictionary definition. So the question is like, how do we actually improve the technology, not how do we mitigate fears, how do we make it something we shouldn't be afraid of? Thank you.

So, who wants to start with that? Yeah, thank you very much, it's a great input. What I described as the lowest code initiative simply wants to make technology, AI, agents, whatever you take, totally understand what we humans want, it has to keep us in the driver's seat.

An AI act, to take that as an example, is useful only if the AI itself understands it and acts upon it. It would not make sense that we, of course it's important that we humans understand it, but it's more important that the machines understand it from the very beginning and act upon it. This is what I would say is one of the core elements how we can make sure, how we can create a safe world for us humans, and we need to do that now because machines are starting to act proactively.

Thank you, Emilie. It's a great point, and you know, the list of horrors go on and on as we go at full speed through space, and everyone kind of wants to get off this ride at this point, like no one's happy, people want to make it go back in a box. But one thing I keep taking away from all of these conferences and all of these conversations is that we have this naive belief that we're going back to the way things were. It's a passing phenomenon that our laws and our acts and our things and our whatever is gonna, you know, we're gonna figure this out.

I think that what comes out of what Jörg just said is we're on this ride whether we like it or not. My own hope to actually answer your question is very close to what Jörg is actually saying, like my thing is human-centric technology. Hard to say when a technology may not even care about humans, so like who are we to think that we're the center of the world? I think it is something in your paper that really struck me is the human discretionary points. AI and computers are very good with processes.

I think that if we can build human ambiguity in the processes by outsourcing that to humans, assuming that processes are taken over by machines, making humans the exception is a really good thing and really making sure that there is a human that can make a decision and that the human decision prevails. I think that that is important. How we design that is, it's a gamble. We can try. And I think that that comes to Scott's point really nicely said about mutual domestication. I think that it is about, you know, finding that compromise.

And my other hope, and then I'll be quiet, is that if machines can focus on what machines do best, humans can focus on what humans do best. And that's my ideal world. And I don't know how we're going to design that in hyperspeed, but we need to try.

Okay, so Scott. Yeah, thank you. It is a great question. And it is a fundamental, you know, humans have been changing as long as we've been aware of ourselves as humans. And our work in the lab uses thermodynamic models for modeling both computational and human intelligence and living forms. And so the mutual domestication is not just a slogan. We are changing and have changed due to technology, language changes. And Emily mentioned the iPhone and all other forms of technology. And we do change with it. We have obesity because of industrial agriculture, and etc.

So we need to recognize that we're constantly changing as well. And that there's nothing, the universe doesn't really care about the human form as such. So we need to understand that life and human life and other life is precious, and recognize that other forms of energy dissipation, and again, I won't go into the thermodynamic stuff, but other forms of energy dissipation are going to present themselves in addition to biological forms. And this is one of those times. And so how we deal with it will define who humans will be going forward. It won't be, not just won't, it won't be nostalgia.

It won't be similar to what we had before. I mean, I'm on heart medicine and other things, which I would be dead long ago if it was not for those things. So we're all kind of cyborgs anyway. I'm a chemical cyborg because I take a pharmaceutical. So we have changed and we, our ages are longer than they've ever been, etc. So these are threats, because we have expectations that were set in past periods. And as Emily said, we don't have a lot of cycles to revisit what it means to be human.

We're being presented with these challenges very rapidly, and we don't have the institutional structures to deal with them, because it was much more leisurely in the piece of Westphalia and things like that in those times. So the convening of the people in that room, I meant what I said, the first comment, business cycles are fast, government cycles are not, whether it's the US or Europe.

I'm not advocating for market solutions for everything, but I am advocating for business leaders to take leadership in terms of understanding how European things that are special about Europe, the norms, the history, the trajectory, the humanity, human flourishing, and the history of challenges have been overcome, and to share that with the world. You can't make people buy it, but I think there's need. People want that humanity now back in their systems, and I think Europe is in a position to communicate that. So thank you so much, Scott.

We have now run out of time, so please can I ask everyone to give a big hand for the panel members and their insights. Thank you, Rod, thank you, Emily, and thank you, Scott, for your insights and your thoughts.