Image for post
Image for post

Due to scalability and duplication issues, Kafka is engaged in the long process of breaking its dependency on ZooKeeper. In this article, we’ll look at how this will change the way that brokers communicate with each other and with Kafka as a whole.

Kafka and ZooKeeper today

When setting up a Kafka broker, there are three essential broker configurations that you need to define. The broker needs to be able to know and tell other system elements who it is (broker.id). It needs to know where to write topic data (log.dirs). And it needs to connect to ZooKeeper (zookeeper.connect) in order to join its…


Image for post
Image for post

Recently Aiven conducted a market study, and one question asked about the respondent’s company’s current cloud arrangements. Given that we’re quite familiar with the terrain, we were a bit startled that 61% of respondents stated that their data was stored in a private cloud. The same held true even in industries where we were pretty sure that couldn’t be the case.


Image for post
Image for post

SELECT 1; isn’t all it’s cracked up to be. As a test of verifying database availability, connecting to it and running one simple query is hardly a realistic workload. This simple operation can succeed while the server’s underlying disk system is having serious issues, or if backups have not been generated on schedule. In short, the database can respond to simple queries just fine but your service could still be down.

Aiven commits to 99.99% uptime, but we don’t calculate that figure based on simple availability checks. We believe our customers deserve better guarantees.

What Aiven does is use the…


Image for post
Image for post

Have you been to Venice? It’s soooo beautiful. There are hundreds of graceful bridges, countless churches and palaces from the awesome to the charming. The city’s canals, filled with iconic gondolas, stir the heart. If you pick your time right, you can even do your sightseeing without being mobbed by thousands of other tourists. In fact, outside of the tourist season, it’s a very quiet city.

Like… really quiet.

That’s because not a lot of people actually choose to live in Venice. The roads are uneven, the buildings aren’t readily wheelchair and pushchair accessible, the plumbing sucks and heating is…


Image for post
Image for post

Implementing your online, globally accessible service can seem daunting. Especially if your background is in the traditional industries, your first instinct may be to go DIY. You buy the hardware and software, set up the connections, install everything, and hope that everything comes together on launch day.

This is certainly a valid approach in certain circumstances. If you’re doing gene sequencing or storing sensitive surveillance data for the defence forces, your particular situation is rare or unique. Then it might make sense to build your own environment and store your own data at a location that no one can get…


Image for post
Image for post

When you’re setting up a microservice architecture, one of your first major decisions is whether to have the services communicate directly with each other, or whether to use a broker system. Since the broker model is more flexible and failure-resistant, it’s likely you’ve more or less decided to choose that. However, you may be concerned that the broker will become a bottleneck in the system if traffic is heavy.

Allow me to set your mind at ease by introducing you to Apache Kafka, a distributed, partitioned commit log service that acts as a messaging system but with a flair all…


Image for post
Image for post

Starting with Teams in June, we announced that we were building console features that would make account management a lot easier for large organizations. Teams was aimed at streamlining user management, and now we add a feature that makes invoicing easier: Billing Groups.

Combined, Teams and Billing Groups do for admin work what the Aiven platform does for cloud infrastructure: they make management efficient and truly scalable. In this post, we’ll discuss what problems the Billing Groups feature solves.

Why billing groups?

Before the Billing Groups feature, you would set up your billing by associating a payment card with a given project. While…


Image for post
Image for post

This tutorial uses Python and Pulumi to stand up a multiregion managed PostgreSQL service on Aiven. I default to using GCP as the underlying cloud provider, but this can be easily changed.

You could even use multiple public clouds with N number read replicas by changing a couple of variables in Pulumi. Let’s begin!

Requirements

The following must be installed on your local computer before you can follow along with this tutorial:

  • Python 3.7+
  • Pulumi 2.16+
  • Register for accounts with Pulumi & Aiven

Time and cost estimate

You’ll need no more than 15 minutes for this.

The costs to temporarily run this tutorial should be…


Image for post
Image for post

In retail, high network traffic is generally a good thing that means people are flocking to your site and buying your stuff. But the flip side is that times of heavy load really highlight every weakness in your system. The services become slow or unresponsive, the customers get irate, sysadmins field phone calls from execs demanding to know why their Cousin Alice couldn’t get through to buy that discounted power drill…

We’ve all been there, right? And none of us want to be there again. …


Image for post
Image for post

We have great news for administrators of Aiven for PostgreSQL and all application developers using it: Your life just got easier with PostgreSQL 13.

The new version of Postgres, one of the all-time greats of open source databases, includes housekeeping operations that keep your database size more manageable.

In this post, we’ll cover the most important improvements and how you can reap the benefits of them.

B-tree indexes are deduplicated

Duplicate entries — which occur when you index non-unique data but also when updating rows with unique indexes — are found in B-tree indexes more commonly than most people think. …

Aiven

Your database in the cloud, www.aiven.io

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store