Tag:

InfoQ

Tech

[Video Podcast] Improving Valkey with Madelyn Olson

by Chief Editor
written by Chief Editor

Valkey: The Rising Star in In-Memory Databases and Its Future Trajectory

The landscape of in-memory databases is rapidly evolving, and Valkey is quickly establishing itself as a significant player. Born from a community fork of Redis, Valkey offers a compelling alternative for developers seeking a high-performance, scalable, and open-source caching and messaging solution. This article delves into the origins of Valkey, its current capabilities, and potential future trends shaping its development and adoption.

From Redis Fork to Independent Force

Valkey’s story is rooted in a pivotal moment within the Redis community. In 2024, Redis shifted its licensing from the permissive BSD license to a dual SSPL and proprietary licensing model. This change prompted a group of core Redis contributors – including engineers from Alibaba, Amazon, Ericsson, Google, Huawei, and Tencent – to fork the code and establish Valkey under the Linux Foundation. This move ensured the continuation of an open-source, community-driven project, appealing to developers who prioritize freedom and collaboration.

Madelyn Olson, a Principal Software Development Engineer at Amazon and a key maintainer of the Valkey project, highlights the collaborative spirit behind Valkey’s creation. The initial team comprised six engineers, and the project has since garnered support from numerous managed service providers like Amazon ElastiCache, Google Cloud’s Memorystore, Aiven, and Percona.

Seamless Migration and Compatibility

One of Valkey’s key strengths is its compatibility with existing Redis deployments. Valkey aims to be a drop-in replacement for Redis open source 7.2, simplifying the migration process for developers. This compatibility extends to client libraries, meaning applications using redis-py or Spring Data Redis can seamlessly transition to Valkey without significant code changes. The ease of migration is a major draw for organizations looking to avoid vendor lock-in and maintain control over their data infrastructure.

Pro Tip: Many users report a remarkably smooth transition to Valkey, often described as simply clicking a button in managed service consoles like Amazon ElastiCache.

The Core: More Than Just a Hash Map

While often described as a “hash map over TCP,” Valkey’s capabilities extend far beyond simple key-value storage. It supports a variety of abstract data structures, including strings, lists, maps, sets, sorted sets, HyperLogLogs, bitmaps, streams, and spatial indices. This versatility makes Valkey suitable for a wide range of applications, from caching and session management to real-time analytics and message queuing.

The recent focus on performance improvements, detailed in Madelyn Olson’s QCon San Francisco 2025 presentation, demonstrates Valkey’s commitment to pushing the boundaries of in-memory database performance. These improvements center around a complete rebuild of the hash table, optimizing memory allocation and leveraging modern hardware capabilities.

Performance Gains Through Architectural Refinements

Valkey 8 introduced significant changes to the underlying hash table, focusing on reducing memory overhead and improving throughput. Key optimizations included embedding key data directly within the hash table structure and adopting a “SwissTable” approach to collision resolution, utilizing CPU cache lines more efficiently. These changes resulted in substantial memory savings – up to 40% in some cases – and maintained, or even improved, performance.

The team prioritized maintaining performance during these architectural changes, focusing on throughput as a primary metric. Valkey aims to deliver approximately a quarter of a million requests per second per core, with a capacity of 1.2 million requests per second on a single key.

Future Trends and Potential Developments

Several trends are likely to shape Valkey’s future development:

  • Enhanced Scalability: Continued improvements in horizontal scalability will be crucial for handling increasingly large datasets and high-throughput workloads.
  • Advanced Data Structures: Expanding the range of supported data structures will broaden Valkey’s applicability to new use cases.
  • Improved Observability: Enhanced monitoring and observability tools will be essential for managing and troubleshooting Valkey deployments in production environments.
  • Plugin Ecosystem Growth: The Rust-based plugin extensibility system offers a promising avenue for community contributions and feature expansion.
  • Edge Computing Integration: As edge computing gains traction, Valkey’s low latency and small footprint could make it an ideal choice for deploying caching and data processing logic closer to conclude-users.

Valkey’s Open Source Governance Model

Valkey operates under a vendor-neutral governance model, guided by a Technical Steering Committee (TSC) comprised of representatives from the founding organizations. While the TSC currently consists of the original six contributors, there are plans to expand it to include more community members, fostering a more inclusive and collaborative development process.

FAQ

Q: Is Valkey a direct replacement for Redis?
A: Valkey aims to be a drop-in replacement for Redis open source 7.2, offering seamless migration for many use cases.

Q: What programming languages are supported by Valkey?
A: Valkey supports a wide range of languages through existing Redis client libraries.

Q: What are the key performance benefits of Valkey?
A: Valkey offers high throughput, low latency, and efficient memory utilization, making it suitable for demanding applications.

Q: Is Valkey actively maintained?
A: Yes, Valkey is actively maintained by a dedicated team of engineers and a growing community of contributors.

Did you know? Ericsson is utilizing Valkey in telecommunications equipment, showcasing its potential in specialized and demanding environments.

Explore the Valkey blog for in-depth technical articles and updates. Join the Valkey Slack community to connect with other users and contributors.

0 comments
0 FacebookTwitterPinterestEmail
Tech

Expanding Swift from Apps to Services

by Chief Editor
written by Chief Editor

Swift Takes Center Stage: From Apple Platforms to Server-Side Powerhouse

Apple’s Swift programming language, initially unveiled in 2014, is rapidly evolving beyond its roots as a tool for building applications solely for Apple’s ecosystem. Recent developments reveal a strategic push to position Swift as a robust and versatile language for server-side development, powered by Apple Silicon and a focus on security and performance. This shift isn’t just about expanding Swift’s reach. it’s about fundamentally changing how Apple approaches service infrastructure.

The Rise of Swift for Services

For over eight years, developers both within and outside Apple have been quietly utilizing Swift to build and run services. The Vapor web framework, launched in 2016, demonstrated Swift’s potential beyond the user interface. Apple itself has leveraged Swift for critical infrastructure components, including iCloud Keychain, App Store processing pipelines, SharePlay file sharing, and most recently, Private Cloud Compute.

Private Cloud Compute is a prime example of this evolution. This innovative service, built on Apple Silicon, allows Apple Intelligence to scale its computational capacity while prioritizing user privacy. The architecture employs load balancers, Apple Silicon machines running inference services, and supporting services for deployment and transparency.

Security First: A Two-Tiered Approach

A core principle driving Swift’s adoption in services like Private Cloud Compute is security. Apple has implemented a unique two-tiered architecture: untrusted components, cryptographically prevented from accessing user data, and trusted components, which handle sensitive information. The trusted components must be verifiable from the silicon level up, necessitating a secure foundation.

This represents where Swift truly shines. Its design, coupled with the security features of Apple Silicon and secure boot infrastructure, provides the necessary trust anchor. Swift’s memory safety features are paramount in reducing exploitable vulnerabilities in network-facing services.

Swift’s Technical Advantages: Memory, Performance, and Interoperability

Swift offers several key technical advantages for service development. Unlike many traditional languages like Python, Ruby, or Go, Swift is natively compiled and doesn’t rely on a garbage collector. This results in significantly lower memory usage – a recent internal migration from Java to Swift reduced heap requirements from 32 gigabytes to under 256 megabytes for a high-request-rate service.

Performance is another critical benefit. Swift, built on LLVM, delivers both low latency and high throughput. The language eliminates common causes of high tail latencies, such as garbage collection spikes. Features like zero-cost abstractions allow developers to write efficient code without sacrificing safety or readability. For example, Swift’s copy-on-write semantics for collections enable powerful local reasoning and optimize memory usage.

Perhaps most impressively, Swift boasts exceptional interoperability. It seamlessly integrates with C and C++, allowing developers to leverage existing libraries and incrementally migrate codebases. Tools like jextract-swift and Java2Swift are bridging the gap between Swift and Java, enabling bidirectional operability and facilitating the reuse of code across different ecosystems.

Beyond the Core: Swift’s Expanding Ecosystem

Swift’s interoperability isn’t limited to lower-level languages. The ecosystem is growing to support cloud-native technologies, with libraries like gRPC Swift simplifying integration with microservice architectures. This allows for a phased approach to adoption, where Swift can be introduced as a latest component or library within an existing service.

Did you know? Swift’s value semantics – where each copy of a value is independent – remove “spooky action at a distance” and make code simpler to understand, a significant benefit for complex service architectures.

Principles for Swift Adoption

Apple recommends a pragmatic approach to adopting Swift in existing services. This includes starting with new components, libraries, or tools, replacing aging code, or incrementally rewriting performance-critical sections. Leveraging Swift’s interoperability allows for a gradual transition without disrupting existing workflows.

Pro Tip: Focus on areas where Swift’s strengths – memory safety, performance, and interoperability – provide the most significant value. Consider using Swift for tasks like parsing, network handling, or security-sensitive operations.

Getting Started with Swift

Getting started with Swift is straightforward. Swift.org provides installers and instructions for various platforms, including container images and devcontainer configurations. Exploring Java interoperability is a good starting point for those familiar with the Java ecosystem.

FAQ: Swift and Server-Side Development

Q: Is Swift suitable for large-scale server-side applications?
A: Yes. Apple has demonstrated Swift’s scalability with services like Private Cloud Compute, which handles significant computational load while prioritizing user privacy.

Q: What are the benefits of using Swift over other server-side languages?
A: Swift offers superior memory safety, performance, and interoperability, particularly when combined with Apple Silicon. It also eliminates the need for garbage collection, reducing latency and improving resource utilization.

Q: How easy is it to integrate Swift into existing Java-based systems?
A: Tools like jextract-swift and Java2Swift are making bidirectional interoperability between Swift and Java increasingly seamless, allowing for incremental adoption and code reuse.

Q: Where can I locate more information about Swift and server-side development?
A: Visit Swift.org for documentation, tutorials, and community resources.

What are your thoughts on Swift’s growing role in server-side development? Share your experiences and insights in the comments below!

0 comments
0 FacebookTwitterPinterestEmail
Tech

Team Building in the Brave New World: Transforming Software Engineering Culture and Leadership

by Chief Editor
written by Chief Editor

The Brave New World of AI and Software Engineering: Navigating the Future of Tech Teams

The tech landscape is rapidly transforming. Artificial intelligence (AI) isn’t just a buzzword anymore; it’s reshaping how software is built, how teams are structured, and what it means to be a software engineer. This article delves into the profound shifts AI is triggering, offering insights for leaders, engineers, and anyone interested in the future of technology.

Introducing AI-Augmented Teams: A New Paradigm

As Duncan Grazier, CTO of BuildOps, highlighted in a recent InfoQ podcast, we’re entering a “brave new world” where AI tools become integral members of software development teams. This isn’t just about using AI to write code snippets; it’s about AI contributing pull requests, automating code reviews, and even deploying software. This fundamental shift demands a re-evaluation of team structures, skillsets, and the very definition of a “software engineer.”

Did you know? The global AI market is projected to reach $1.8 trillion by 2030, according to Statista, showcasing the massive investment and growth in this area.

Incremental AI Adoption: A Practical Approach

Successfully integrating AI into software engineering isn’t about a complete overhaul; it’s about taking small, manageable steps. As Grazier suggests, start by providing access to AI tools in a safe environment. Focus on areas like test writing, where AI can automate repetitive tasks. Measure the impact – are engineers more productive? Is code quality improving? Gathering this data is key to informed decision-making.

Pro Tip: Start with pilot projects. Choose a specific, well-defined area where AI can provide immediate value, such as generating unit tests or automating code documentation. This reduces risk and builds confidence within the team.

Navigating the Cultural Shift: Impact on Engineers

The introduction of AI tools evokes a mix of excitement and apprehension among engineers. Some see it as a way to focus on more challenging problems, while others worry about their roles. Leaders must proactively address these concerns by showcasing the benefits: increased output, reduced mundane tasks, and the opportunity to work on complex, business-critical issues.

To understand the cultural implications, organizations must promote an environment of continuous learning. Conduct brown bag lunches, record training sessions, and provide hands-on workshops to showcase the value of AI and the benefits to individual team members. This will increase engagement and decrease the fear of the unknown.

The Future of Engineering Careers: Adapting to New Realities

The skills demanded of software engineers are evolving rapidly. The ability to break down complex problems, communicate effectively, and understand customer needs will become more critical than ever. AI will handle many of the routine coding tasks, shifting the focus towards higher-level thinking, problem-solving, and creative design. Engineers need to level up their non-coding skill set.

Career paths will likely be reshaped. Engineers may move up ladders faster, focusing on solving more complex problems and taking on leadership roles. The emphasis will shift from the mechanics of coding to the ability to understand and articulate solutions.

Redefining Leadership in the AI Era: The New Managerial Role

Leadership in software engineering must evolve. The engineering manager role is changing from the most senior technical person to someone able to leverage tools as well as people. Managers will need to prioritize communication, documentation, and the ability to explain complex concepts clearly. Coaching, the ability to mentor and guide others, becomes a crucial leadership competency.

Leaders will need to focus on the value that the team provides to the product and the business. Performance measurement will also change. The focus must be on how the team solves problems and how they deliver value to the organization.

Coaching and Skill Development: Preparing for the Future

Coaching is more important than ever. It’s essential to focus on building the non-coding, ‘soft’ skills such as communication and problem-solving that are crucial for success in the AI-augmented world. Technical professionals can benefit greatly by finding mentors and building relationships with leaders. These leaders have experiences and strategies to share about how to grow in the software engineering world.

Start developing these skills now. It’s about developing the ability to understand a problem, describe it, and collaborate. By doing so, you set your team up to be successful today and in the “brave new world” of tomorrow. Make sure your team focuses on building the core skills of communication and technical vision.

The Long-Term Implications: Unexplored Territory

The long-term impact of AI on the technology industry remains largely unknown. Will we see a significant shift in what it means to be a junior engineer? How will we attract and retain top talent in a rapidly evolving market? These are critical questions that leaders must consider.

Did you know? According to a study by McKinsey, the adoption of AI is accelerating across various industries, with a significant increase in investment and implementation.

Advice for Leaders: Core Values and Adaptive Thinking

The most important piece of advice for leaders is to be true to your core values. Know how you react to challenges and successes, and use that understanding to guide your approach. Promote clarity, transparency, and open communication to foster a culture of innovation and adaptability.

Frequently Asked Questions (FAQ)

Q: How can I get started with AI in software engineering?
A: Start small. Identify areas where AI can automate repetitive tasks, and experiment with AI tools in a safe environment.

Q: What skills will be most valuable for software engineers in the future?
A: Strong communication, problem-solving, and understanding of business needs will become increasingly important.

Q: How can I become a better leader in this new landscape?
A: Focus on developing your coaching skills, building strong communication, and promoting a culture of continuous learning.

Embrace the Transformation

The integration of AI in software engineering is not a threat; it’s a powerful opportunity. By understanding the trends, developing essential skills, and adapting to new realities, you can thrive in the brave new world of AI-augmented teams. The future of software development is here, and it’s time to embrace it.

Ready to learn more? Explore our other articles on AI, leadership, and software engineering trends. Share your thoughts in the comments below, and subscribe to our newsletter for the latest insights and updates!

0 comments
0 FacebookTwitterPinterestEmail
Tech

Productivity Through Play: Why Messing Around Makes Better Software Engineers

by Chief Editor
written by Chief Editor

Decoding the Future: Software Engineering Trends from a Leading Expert

Holly Cummins, a prominent figure in the software engineering world, recently sat down for an insightful podcast, sharing her perspectives on productivity, efficiency, and the evolving landscape of our industry. This article distills her key insights, offering a roadmap for engineers navigating the ever-changing tech terrain.

The Productivity Paradox: Rethinking How We Work

Cummins challenges the conventional wisdom that equates productivity with endless hours and relentless coding. She argues that true productivity stems from activities often dismissed as “unproductive” – moments of reflection, creative thinking, and even simple downtime. The core concept is about solving problems, not just generating lines of code.

Think about it: Many programmers find their “aha!” moments not at their desks, but in the shower, during a walk, or while doing chores. These activities allow the brain’s “default mode network” to activate, fostering creativity and problem-solving. The result? More effective solutions, not just more code.

Did you know? Studies show that taking short breaks can actually increase your focus and productivity. Try the Pomodoro Technique – 25 minutes of focused work, followed by a 5-minute break.

The Efficiency Trap: Beyond “Do More with Less”

Cummins urges us to redefine efficiency. It’s not about cramming in more, but achieving the same or better results with less effort. Using less resources – time, code, or effort – is where true efficiency lies.

Consider internal combustion engines: They have an efficiency ceiling, and operating them at peak efficiency shortens their lifespan. Similarly, pushing engineers to constant overwork leads to burnout and diminished returns. Sustainable practices are crucial.

Pro tip: Prioritize quality over quantity. Focus on writing clean, maintainable code, even if it takes a bit more time initially. The payoff in the long run is significant.

Navigating the “Do More With Less” Mandate

Cummins points out that the pressure to “do more with less” can be counterproductive. She highlights the importance of understanding and communicating the value of activities that are not immediately quantifiable, like taking breaks or engaging in professional development. This requires framing arguments in terms that resonate with business stakeholders, such as financial benefits and reduced errors.

Example: Research by the American Psychological Association shows that sleep deprivation leads to significant cognitive decline. By encouraging adequate rest, organizations can drastically improve employee performance.

Management vs. Technical Tracks: Forging a Sustainable Career Path

A major theme in the podcast revolved around the career paths available to software engineers. Cummins’s advice highlights the importance of organizations creating paths where expert engineers can excel without being forced into management roles.

This is often overlooked, and it can mean losing out on the skills of those that are great coders but not as effective in management. This requires recognizing and rewarding the value those individuals bring.

She also emphasized the need for professional development, whether on a management or technical path. Continuous learning and refinement are critical to long-term success.

The Future is Editor: Adapting to the AI-Driven Landscape

Cummins acknowledges the industry’s shift towards engineers taking on more of an “editor” role, managing code generation, rather than writing every line themselves. This shift requires a change in mindset and a willingness to adapt. Embracing AI-powered tools is not just a trend; it’s a new reality. To remain relevant, engineers need to focus on higher-level skills like architecture, design, and code oversight.

Related read: Dive deeper into this shift with our article on “The Rise of the AI-Enhanced Engineer” [Insert Internal Link to Relevant Article Here].

The Pillars That Endure: Timeless Truths in Software Engineering

While trends come and go, some fundamentals remain constant. Cummins emphasizes that despite rapid changes, the core challenges of software engineering—managing overhead, coordination, and cognitive load—persist. Applying Amdahl’s Law to both code and teams remains a valid strategy. In essence, software development, at its core, involves understanding and managing complexity.

External Link: Explore Fred Brooks’s “The Mythical Man-Month” for insights that still hold true [Insert External Link to the Book or Related Source Here].

Tackling Cognitive Load and Staying Ahead of the Curve

The constant influx of new tools and technologies inevitably increases cognitive load. To manage this, Cummins suggests focusing on removing unnecessary elements and choosing the correct tools for the job. Consider adopting principles like “shift left” cautiously, balancing its benefits with potential added complexity.

Moreover, continuously evaluate tools and methods. If something is causing more burden than benefit, it may be time to move on.

FAQ: Frequently Asked Questions About Software Engineering

How can I boost my productivity?

Prioritize breaks, embrace downtime for creative thinking, and focus on quality over quantity.

How do I communicate the importance of downtime to my manager?

Frame your arguments in financial terms, providing evidence-based insights from psychology and related fields.

What skills are most valuable for software engineers today?

Adaptability, critical thinking, AI integration, and the ability to manage code generation.

How can I avoid burnout?

Prioritize work-life balance, embrace recovery periods, and focus on sustainable work habits.

Question for readers: What are your biggest challenges in the evolving world of software engineering? Share your thoughts in the comments below!

0 comments
0 FacebookTwitterPinterestEmail
Tech

Building a Lightning Fast Firewall with Java & eBPF

by Chief Editor
written by Chief Editor
0 comments
0 FacebookTwitterPinterestEmail