Open Source Summit Japan 2025: Transformation of Architecture

Publish Date
March 16, 2026
Share

Introduction: System Architecture, Not Model

While the race for "larger models" in the artificial intelligence world is giving way to a critical silence, the future of the sector was summarized in a single sentence at Open Source Summit Japan 2025: "The future of Agentic AI will not be defined by model scale, but by system architecture." 1

This determination is the most concrete indicator that the focus has shifted from just model training to the infrastructure that sustains and scales these models. The summit held in Toranomon Hills, Tokyo, proved how giants like Google, Fujitsu, and Honda have adopted open source as a "strategic foundation", and as a DevOps engineer, I had the opportunity to examine the new norms at the intersection of the artificial intelligence and infrastructure worlds on site.
This article analyzes the interconnection between the worlds of artificial intelligence and infrastructure, drawing on photographs I personally took at the event, the technical sessions I attended, and data compiled from the Linux Foundation's official program.

Key Themes

The technical discussions that stood out in the conference program and the presentations I examined on-site were gathered in the following main axes in terms of modern infrastructure management:

A. System Architecture and Hardware Disaggregation

Hardware Disaggregation: As detailed in Fujitsu's "Dynamic GPU Provisioning" session, moving beyond the traditional server structure, collecting CPU, GPU, and memory in a dynamic pool (Resource Pool) by removing them from physical cases.2
AI Factories: Transformation of data centers into production facilities that can respond to the variable demands of artificial intelligence workloads instead of static server farms.

B. Agentic Ops

Autonomous Infrastructure Management: As we saw in the demos on stage, the transition from the structure of tools like Terraform that "stop in case of error" to "Agent" based structures that can analyze the error and produce a solution.3
Continuous Feedback Loop: Continuous monitoring of not only the deployment of artificial intelligence models but also their performance in the live environment.5

C. Data Governance and Security

Federated Vector Access: According to the architectural diagrams shared by NTT and other presenters, structures that process data on site in distributed sources combine the results instead of moving it to a central lake.6
Policy-as-Code: Dynamic making of data access decisions at the moment of query by engines like OPA (Open Policy Agent). 6

D. Critical and Special Workloads

Game Server Orchestration: As shown in the Agones presentation of the Google Open Source team, management of stateful game servers on a global scale on Kubernetes.7
Space Grade Linux: Management of robotic arms on the ISS with open source software, exhibited in GITAI's presentation.8

Highlights from the Conference

A. System Architecture and Hardware Disaggregation

Liberation of Hardware: CoHDI

One of the most striking technical sessions of the conference was the speech titled "Dynamic GPU Provisioning in Kubernetes" presented by Jin Hase and Tsubasa Watanabe from the Fujitsu team. The architectural diagram shown on stage explained the CoHDI (Composable Disaggregated Infrastructure) project as follows:

Disaggregated Resource Pool: CPU, GPU, Memory, and NICs are removed from physical server cases and collected in a common pool.
PCIe/CXL Switch Fabric: These components are connected to each other via high-speed switches.
Kubernetes Integration: As seen in the diagram, Kubernetes requests hardware as if creating a Pod, and the system instantly assembles this hardware to create "Composed Baremetal" servers.2

So why is CoHDI important? This architecture has the potential to increase GPU usage efficiency from 30% to 80%.

Infrastructure Orchestration: Crossplane 2.0

Another important presentation at the event showed the Crossplane 2.0 architecture. The continuous monitoring of declarative definitions on Git by the Crossplane control loop and their transformation into real resources on AWS, Azure, or Google Cloud supported the vision of "self-healing" infrastructure. Also, Crossplane 2.0 could now be run in all Kubernetes environments, not just for cloud services, as in Crossplane v1. Thanks to the technology, it enabled every developer to install simple technologies without help. It was an impressive presentation in terms of showing the point where DevOps processes will come. 4

B. Agentic Ops

AgentOps: It is a continuous feedback engine that aims to close the reality gap between them by bringing together the development (inner loop) and application (outer loop) stages of artificial intelligence agents. This structure is defined as an AI platform engineering discipline that combines offline prepared data sets with observations from the live environment in a single ecosystem, enabling the detection of errors and the inclusion of real-world data back into test processes.5

Human vs. Agent: Annie Talvasto argued in her speech that machines will remain incomplete in DevOps processes as long as they do not have the information found in humans. For example, she mentioned that the system could be managed incompletely when you do not have even a small slack correspondence from a colleague. She mentioned in detail in her presentation that all this information should be provided to agents first. 3

Open Source in Space: ISS and Robotics

In the photos taken at the safety-critical systems session, images of the International Space Station (ISS) and GITAI robotic arm were reflected on the giant screen. Seeing that Linux is used in an environment where there is no luxury of making mistakes in the vacuum of space was proof of the reliability of open source.8

C. Data Governance and Security

Unified Vector Access Across Organizations and OPA: In large organizations, data is usually stored in different departments or partner companies, in different vector databases. Collecting this data in a single center is difficult due to legal compliance, data security, and ownership rights. The presentation aims to provide a "virtual integration" without copying the data.

vLLM: Used to transform complex metadata from data sources into meaningful and short natural language summaries. It also plays a role as a high-performance inference engine in the processes of understanding user queries and processing results.
Policy Decisions: Responsible for data governance and authorization. When a user initiates a query, OPA checks at runtime whether the user has permission to access that data source.
Milvus Detail: Acts as a "Semantic Catalog". It stores the technical and business metadata of data sources in all organizations in vector form. When a query comes, it performs the first stage search to determine which organization might have the relevant data.

This presentation proves how a secure and "smart" search infrastructure can be established between organizations without moving data to a central location by combining Milvus's scalable search capability, vLLM's language processing power, and OPA's flexible security policies.6

Context Engineering: Speaker Kevin Noel argues that the era of simply "writing a prompt and waiting for an answer" is closed, and instead, we have entered the era of context engineering.

LLMs are Stateless: Models do not remember even the previous sentence unless you remind them.
State Creation: For an artificial intelligence to give the correct answer, you need to present everything correctly to it at that "moment" (just-in-time). This is the art of bringing together past conversations, information from documents, and tool outputs.

The presentation shows that the following parts must be combined for a successful artificial intelligence response:

RAG (Relevant Information): Information pulled from in-house documents or databases.
Memory Management: User's past preferences and short-term conversation history.
External Tools: Data coming from APIs or calculation tools.
System Commands: Main rules determining the character and task of the artificial intelligence.

It is impossible to manually test such a complex structure (RAG + Memory + Tools). The speaker offers the following solution:

Automatic Evaluation: We must have artificial intelligence produce thousands of "question-answer" pairs (synthetic data) using the data we have.
Realistic Scenarios: We must measure the robustness of the system by creating synthetic users who ask incorrect or incomplete questions like real users.

Cost and Speed: We can score the success of the system (accuracy, irrelevance, hallucination) in minutes with synthetic data sets, instead of tests that would take weeks for humans. 10

D. Critical and Special Workloads

Game Server Orchestration( Agones): In the slide titled "How Agones works" by the Google Open Source team, a flow chart was shared over the infrastructure of the ARC Raiders game. In this diagram, it was clearly seen that after matchmaking, players were assigned to Pods called "Agones Game Server" and these servers were managed with a "binpacking" (compression) strategy on a global scale. This enabled developers to devote more time to game development without thinking about the infrastructure. It also proves how important a role Kubernetes plays as an open source in the game industry.7

Speed in Model Distribution: The presentation showed that models are now kept in OCI compatible registries like Docker images and directly connected (mounted) to the Pod like a disk thanks to the Model CSI Driver. This eliminates the time to copy GBs of data. Harbor's OCI registry is used for these operations.8

Inference Performance: According to the notes and visuals I took from the vLLM session, PagedAttention technology was the key to performance. This technology manages GPU memory with the virtual memory logic in operating systems, overcoming the "Key-Value Cache" bottleneck and multiplying the throughput. The presentation also mentions that DeepSpeed developed by Microsoft increases GPU efficiency by 4-6 times, providing high performance even on old hardware. It is emphasized in the presentation that the number of tokens produced per fee increases thanks to this. 9

Conclusion: Leaving Open Source Summit Japan 2025, I return with a vision that completely changes my perspective on my profession, not just with new technical knowledge in my pocket.

One of the points that impressed me the most throughout the event was the breadth of the coverage area of open source software (OSS). Today, the fact that the power of open source is trusted even in the most critical and advanced technology projects of humanity, such as the International Space Station (ISS), not just in websites or mobile applications, reminded me once again of the responsibility and potential of the work we do.

The most unforgettable moment of the summit for me was listening live to Linus Torvalds, one of the architects of the modern software world and the creator of Linux and Git. Hearing his approach to technology and his vision firsthand allowed me to understand the huge philosophy behind the basic tools we use.

The biggest lesson I learned from this summit was this: The software world is an endless ocean, open to development at any moment and constantly in motion. Being a part of this ecosystem means not only writing code but also constantly learning and contributing to the community. This dynamism of my profession motivates me incredibly. Now, it's time to continue my work much more determinedly and quickly with this inspiration! 👨‍💻✨

My visit to Japan was the realization of a curiosity that first took root through the vibrant culture of reading and knowledge-sharing encouraged at Galaksiya. My primary motivation for choosing this destination was to immerse myself in a culture and culinary heritage that I perceived to be fundamentally different—almost a world apart—from our own. I wanted to step out of my comfort zone and experience what it felt like to be an "outsider" in a land of such vast contrast, where every ritual and flavor seemed to belong to a different dimension.

However, as I navigated everything from Tokyo’s massive, neon-lit boulevards to the traditional, tucked-away eateries in narrow alleys, I experienced a profound realization. While the meticulous presentation of the food and the stoic silence of the streets initially felt foreign, I soon discovered a universal essence beneath that surface. Standing in the shadow of a modern skyscraper and watching someone find a moment of peace at an ancient shrine, I realized that the expression of human sincerity is the same there as it is in our own homeland. It was a powerful reminder that regardless of geography, humanity converges on the same fundamental values: kindness, labor, family, and the spirit of sharing.

Witnessing the deep-seated culture of respect and the "invisible harmony" behind a society that operates with such incredible precision transformed my observations into a broader lesson in humanity. Experiencing this cultural depth alongside such a unique vision helped me understand that the world is shaped not just by structures, but by human spirit and discipline. I return from this journey with an unforgettable insight into how we are all "one" even on the other side of the globe—and, of course, a few cherished photographs that capture the soul of the city.

See you at our next stop.

Emre İZMİR

Sr. DevOps Engineer
Open Source Summit Japan 2025 Impressions, Tokyo.