Robert Kelly (@DevOpsRobert) header
Robert Kelly (@DevOpsRobert) profile picture

VP of Innovation at Liatrio - I lead the enterprise technology AI & innovation efforts for our professional services organization.

Dallas, TX
Joined December 2022
Replying to @wholemars
I paid upfront with this hope or even expectation.
70
You may have joined the cool kids club building a software factory. So, are you "the foreman", or should you be working at a higher level now with the expectation the factory will change and adapt for the results you want? Most enterprise software factories I see are still being built on assumptions made six months ago. Right now, six months is a long time. Most try to give every team the same paved road. Engineering and architecture groups build it and expect everyone to use it. They standardize the a pipeline, enforce guardrails, and hope teams ship. It's the right goal, but the people and the work aren't all the same. Some products deploy daily. Others never need a build, just fast feedback. Teams work on different architectures that absolutely should be changing. And much more building now happens outside the engineering org, with business users prototyping ideas to show what should get built or creating personal products that previously wouldn't exist. There's been a 100x+ increase in internal, temporary products with no support model that only need to live a few weeks. That pattern isn't going away and may be the biggest need for the organization. A factory built around just one of those paths ignores the others. They may need to blend as business needs evolve, but a factory that can't change only serves people it was built for and won’t help the others. So the answer likely isn't one factory. It's a factory of factories, with an orchestration layer that knows which fits each idea and keeps adapting as the work and people change. It's more than CI/CD with a security scan and automatic bug fixes. It's how any idea becomes working software, whoever it comes from. You won't only be working with a ticket queue or through code changes. You'll reach it through a personal agent. Your agent will work with the foreman who can adjust the factory to your needs. Either way, you're dealing with something above the whole system, a higher abstraction than most factories built so far. We've talked about the human in the loop, then above the loop. I think it goes further. The human guides the system that runs the factory and its many value streams. You set direction, give the system context and access, and let it correct itself. As building gets faster, that context and access become the real constraint. This is how I'm working today on my team and it has been an amazing unlock. My factory is different than theirs and I work at a higher level of abstraction for the products I'm building. I believe what will last will be the factories where you can actually "talk" to a foreman and expect the work to get done. They need to work for you, adapt, keep listening, and change without retooling everything. The value of a factory will be its ability to mold to our needs. It's the intent and outcomes that matter, not the factory itself. What do you think? How are you building your factory today? Will it adjust and grow as everyone's needs change?
3
60
OpenCode reported a breach that affected a portion of their users. Regardless if you think you were in the 1.4% they reported as affected, make sure your team rotates their keys if using the harness. You should also consider moving your team behind a model gateway/proxy to help protect against this type of issue in the future. Using open harnesses isn't a bad thing, but it could mean there are more API keys for model providers at risk of being exfiltrated. I don't like using ANY API keys locally if we can avoid it. Using a model gateway (AI Gateway, Foundry, OpenRouter, etc.) helps mitigate issues like this because API keys aren't living with the harness and any keys can be rotated for all users easily. Related, our @liatrio team published a matrix report with many of the AI gateway features last month that you should check out: ai-gateway.liatr.io/ - Reach out any time if you'd like to chat about it! Stay safe out there!
2
3
169
2 years ago, a 5-7 person dev team was normal. As we adopted AI-native workflows, more than 4 people committing 2x code to the same codebase became challenging. Now, we can manage the same product codebase with 1-2 people and push 10x more code than 2 years ago. Here's the problem as I see it: Team structures don't change from the bottom up. Individuals on teams can't or won't say "our team should be smaller". And if they did, budgets were set, roles and resources have been allocated. There's no urgency from the top because the org is still struggling to see value they hoped to see. They don't realize that the value proposition, team structures, and productivity improvements are linked. We've created new bottlenecks and haven't adjusted to the new normal. I think they believe that making teams smaller means mass layoffs. It could, but it doesn't have to mean that. There's plenty of work to be done. It just means realigning and removing dependencies vs. managing them. If this change seems drastic or otherwise difficult for any reason other than org pressure or inertia, what is keeping your team or org from adjusting? What do you think? Is this doable in your org? If your org hasn't reset on new first principles with reduced overhead, what do you need to push this forward?
23
I'm sure you've heard the founders of the frontier labs and thought leaders throughout the industry talking about the potential need to slow the pace of AI. That's fine. It's a really important and compelling topic so they should have that discussion. Fortunately for us, AI is pretty dang good today and I don't think have to worry about anything but how we take advantage of what's available for everyone right now. We've got nothing but opportunity in advancing the adoption and positive change across the work we're doing. Even if AI stopped improving today, we've got plenty of great things to work with. Everyone can take advantage of agentic workflows, AI-native software development and improving processes across each role and job function. TL;DR here is we've got a lot of work to do and there's no reason to let things we can't control impact that.
19
Building software with agents running locally is really starting to feel "wrong". What's your take? We're getting to a point where just about everything can be built in cloud agents/remote sandboxes with clear visibility into what's being run, when, and what tools or access is in use. Remote agents can keep running when you close your laptop, allow for mobile/remote "prodding" if you desire, and can scale however you need. Sure local agents are fine and there's probably a place for them in some spots. But for enterprises or large orgs that need governance and observability, there's just so much potentially hidden with the local approach when there's another option. What's your remote agent or workflow of choice? Has your team moved to remote/cloud agents as the default? Or... What's the steel man against it? Do you have use cases that don't fit this approach? What's keeping you or your team from moving to cloud/remote agents for all software dev or other agentic workflows?
2
124
The [uncomfortable] truth is "best practices" have a shorter shelf life than ever before. The way we used AI to build software just 3 months ago is already out of date. And I can tell you why. For those of us who try to stay aligned to these new innovations, we may seem to follow a bit blindly behind them because we think it's an amazing time to be in tech and we "feel" that it's important. Yes, I might I shout pretty loudly that 𝐞𝐯𝐞𝐫𝐲𝐨𝐧𝐞 𝐬𝐡𝐨𝐮𝐥𝐝 𝐩𝐚𝐲 𝐚𝐭𝐭𝐞𝐧𝐭𝐢𝐨𝐧 and stay up to date, but that sounds like noise to everyone just trying to get work done. I'll admit that I couldn't clearly articulate the "why" here very well perviously. I wanted us to stay up to date because it was exciting and I "knew" it was important. But, after catching up at OpenAI Dev Day with the engineers working on how their models and harnesses are being trained and built intentionally, I think I can share a fairly simple argument on why. TL;DR 𝐓𝐡𝐞𝐫𝐞 𝐡𝐚𝐯𝐞 𝐛𝐞𝐞𝐧 𝐜𝐨𝐧𝐭𝐢𝐧𝐮𝐨𝐮𝐬 𝐢𝐦𝐩𝐫𝐨𝐯𝐞𝐦𝐞𝐧𝐭𝐬 𝐭𝐨 𝐛𝐨𝐭𝐡 𝐦𝐨𝐝𝐞𝐥𝐬 𝐚𝐧𝐝 𝐡𝐚𝐫𝐧𝐞𝐬𝐬𝐞𝐬. 𝐀𝐧𝐝 𝐭𝐡𝐞𝐬𝐞 𝐢𝐦𝐩𝐫𝐨𝐯𝐞𝐦𝐞𝐧𝐭𝐬 𝐢𝐧𝐟𝐥𝐮𝐞𝐧𝐜𝐞 𝐚𝐧𝐝 𝐢𝐦𝐩𝐚𝐜𝐭 𝐞𝐚𝐜𝐡 𝐨𝐭𝐡𝐞𝐫 𝐝𝐢𝐫𝐞𝐜𝐭𝐥𝐲. 1. Harnesses (i.e. Codex, Claude Code, etc.) are updated to take advantage of new model features as they're released. 2. Frontier models are being trained on these new workflows and harness features to improve their behavior. This cycle repeats, with each product being reinforced with updates to take advantage of new capabilities and efficiency improvements. Over the past 6-12 months, we've been trying to keep up: - we had to actively watch and manage context for our agents, skills and models. - we created clear plans, worked hard to break them down into small updates. - models were inefficient calling tools - we built custom skills and workflows to manage stepping through a problem. - models didn't understand what "done" looked like and needed to be reminded directly they weren't done with a task. - we built "loops", and harnesses added features like "/plan" and "/goal" to help us get through a complex scenario. None of these things were wrong, but every one of them has evolved and continues to evolve. Today, the latest models can work for 𝐃𝐀𝐘𝐒 on a complex task. Better context management and compaction handling is built in. Agents can orchestrate dozens of other agents and tasks. Are they perfect? Nope. But we need to understand how they work and how 𝐧𝐨𝐭 𝐭𝐨 𝐠𝐞𝐭 𝐢𝐧 𝐭𝐡𝐞𝐢𝐫 𝐰𝐚𝐲. Some of the workflows may persist, but the "help" we have been providing needs to adapt. Changing how we build is now part of the job, just as important as the software we're building. The new normal is keeping up. Models and harnesses will continue to improve. They're great now, but it's on us to make sure we're getting everything we can out of them, not wasting energy (or tokens!) on earlier "best practices". Any thoughts? I'd love to chat more about it!
2
3
168
Just when you thought it was safe to sleep on new model releases, a new class of "deterministic" models emerged last week and are already changing agentic workflows. TypeSafe just released Jev, the first public "System One" model, and it had the most day-1 adoption ever of any model release in Vercel AI Gateway to date. Quick points: Jev does not replace frontier models. A modern stack will soon use both types of models. The terms System 1 and System 2 are inspired from Daniel Kahneman, author of "Thinking, Fast and Slow" with the distinction between fast, intuitive System 1 thinking and slow, deliberate System 2 reasoning. Existing "System 2" models are for more open-ended/reasoning tasks: - Writing code - Summarizing content - Researching a topic - Planning tasks (.. etc. You know how you use models today ) Jev - "System 1" models - are for making structured decisions: - which model to route to for specific tasks - which tool to use and which not to use - did we actually get the answer we were looking for? >> And to get these answers 𝐅𝐀𝐒𝐓𝐄𝐑 and 𝐂𝐇𝐄𝐀𝐏𝐄𝐑 (comparatively almost free*) You send state and "typed" questions and you get back options, scores, or yes/no. I'm still trying to form an opinion around the claims that Jev "𝐜𝐚𝐧'𝐭 𝐡𝐚𝐥𝐥𝐮𝐜𝐢𝐧𝐚𝐭𝐞" its results assuming the parameters you supply are correct. More opinions to come as we put it through its paces, but the potential if used properly are pretty great. The differentiator is that it's fast enough to sit inside real software workflow paths. Estimated end-to-end latency is in the 70-500 ms range, with workflow tests vs Claude Fable 5.1 and GPT-6 Astra showing up to ~193x faster and ~444x cheaper on System 1 type tasks. A few examples of how it's being used already: - Route the right model for the job (higher frontier, lower-frontier, or small language model) before you spend your tokens - Gate a tool call before it runs (allow, ask a human, or deny) - Stop or steer an agent or workflow that's stuck or drifting off the task - Decide what belongs in agent context and what needs to be trimmed - Score every application trace or log entry for escalation or intervention - Triage a support ticket with several questions in one call (urgency, team, refund risk) - Split sales lead from lead intent before a seller/human sees it - Lint a PR for semantic policy issues in CI If you run agents, AI workflows or apps with decision logic, get this in front of your team so they can understand and research this new model class. Walk the use cases against what you already have and see where a typed decision approach belongs next to generation in other model calls or in your existing app workflows that use complex logic vs LLMs to make decisions. If you've already been looking at Jev, let me know what you think. Would love to chat more about this new approach! Check out TypeSafe and Vercel's blogs in comments
1
3
361
Messaging to adopt today: Measure the 𝐜𝐨𝐬𝐭 𝐩𝐞𝐫 𝐭𝐚𝐬𝐤, not 𝐜𝐨𝐬𝐭 𝐩𝐞𝐫 𝐭𝐨𝐤𝐞𝐧. Evals of bespoke tasks and workflows are how to effectively calculate AI cost & efficiency.
1
14
Software developers are spending more time with agents than humans. And the (uncomfortable) consensus is that they prefer it. Engineers love this new world of autonomy and productivity, but it also looks like it's creating an anti-pattern we've been trying to fix for more than a decade. So, why is that? And should we try to change it? It's hard to concede that one person working independently for hours with their team of agents is a better "team" than the team of software engineers we had a year ago. But that's what seems to be the case. Higher agency, more time to iterate and work through complex problems, and the ability to ship 10-100x more in the same amount of time is something nearly every software engineer has as always wanted. But this new level of productivity cuts against the grain of the org and systems that have been built over time to manage a 10th of the output we're seeing today. There's so much gravity being created from the inner loop in building software being so good that it's going to be extremely hard to walk back. It feels deeply wrong in some ways but might actually lead to the outcomes everyone wants. A few points to call out: 1. The current team dynamic has become distributed, asynchronous, and moved away from small, "XP" principles as we've moved to remote work as a default across the industry 2. Teams have grown in size in enterprises and have yet to reorganize as right-sized product teams. (not all are "just project teams" but many are still built this way) 3. The teams that exist today have been driven by program management, SAFe, and Scrum practices that were built to manage large teams and distributed org dependencies None of those things fit with the new way of working that's emerging with AI and agentic software development. They are able to deliver more as individual contributors working solo with teams of agents than anyone could have imagined earlier than 2 years ago. I don't think we can put the genie back in the bottle here (or try to, honestly) and our response should be to rethink how we all work together in this new world. That means there will need to some changes throughout the larger system to accommodate the new normal. -- Smaller teams (much smaller - 2-3 people max) -- More visibility of the work in progress (Summary of what was delivered vs. status updates) -- Move to outcomes and wider goals over a focus on breaking down work via process and tickets -- Systems thinking and product ownership vs. program management -- In-person work should be extremely collaborative but we should leave most of the week for the engineers to manage their agent's work vs. talk about the work/dependencies We need to lean into the autonomy and potential and build the systems around it. It’s going take some effort but also can't lose the magic of getting people in the room to brainstorm and build a common understanding of the challenge, so we need to build the new ways of working to support it.
2
96
Asking others to review our agent's PRs should be the exception, not the rule. If you want to review the code you and your agents write yourself, that's great. But the effort and time spent in additional human review needs to be focused on understanding a product domain or problems deeply, building standards, skills, and solid CI workflows with agentic review. If the agentic SDLC doesn't live up to our standards on day 1, that's not because AI isn't good enough. It's more than capable when given guidance and paired with a change that wouldn't otherwise give someone else on the team a headache to review. The caveat I'd add here is where software factories are taking tasks on their own from the backlog from start to finish. But even then, the system should hope to support anyone's ask eventually. Build in smart validation (read deterministic) around key logic or areas of the product that may need additional review. You may even be able to work your way out of that as your team sees repeatable success. It takes work, iteration, multiple models, and maybe some bespoke product-specific review agents. The most important thing to point out is that async "PR approval" has always been an anti-pattern. Bring XP practices back, pair early, review what you need to along the way, but we shouldn't settle for last year's process that just doesn't fit any more. Where's the challenge here? Do we need more or less review of AI-generated code at this point? Should anyone but the person who orchestrated the work be the gate for the code change? Why? If there are folks on the team who need guidance, code review isn't where that happens. We help them by building the tools and workflows that level up everyone on the team, including our agents.
1
3
156
Cursor / SpaceX AI launched their GitHub killer "Origin" today - just as GitHub was suffering a major outage. The timing wasn't planned but it was a bit ironic for sure. There are a few new entrants into the Agentic source control space but I think SpaceX AI is positioned extremely well here. It's going to be interesting as more platforms provide native experiences so I'm looking forward to having more options across the ecosystem. I have to assume that all the load GitHub has seen as agentic SWE has taken off has impacted most of us along the way. New solutions will lighten that load and give us more options going forward. I've been checking out the beta. You can start by syncing with GitHub as the source of truth or create your own repos directly in "codebase" via the Cursor admin dashboard. I've gotten a few repos sync'd and created and things are pretty smooth so far! What's your take? Are you interested in more options or more hopeful that GitHub can keep up with the scale? All paid plans can check it out in beta: cursor.com/origin
1
68
The ADE is the new IDE.
9
An AI native org has 4 components: - people who set strategy and bring taste - platforms with shared capabilities for integration, security, and governance - agents that handle execution - a shared context that makes everything the company knows accessible to the others What that changes is the shape of your day. You used to spend most of your time in meetings and executing, and now the shared context removes toil and agents execute. What's left is deciding what's worth doing and judging whether what we built together is the right thing.
2
20
Web browsers are a legacy technology. We’ve still got to support them because “humans”, but they are not meant for the agentic future.
26