On October of 2019, I attended RIPE79 in Rotterdam, which was a very good experience. I had a chance to meet new people, talk to people I already knew, exchange ideas, and discuss various topics. It’s a very good event, and Ι recommend it, if you can attend.
During one of the conversations I had there, ironically with another guy from Crete, Vasileios Kotronis, we discussed about how some companies pay large sums to obtain IPv4 space, yet some others, especially univerities, have vast and unutilized address space. And then we wondered, if there was a way to know how much IP Space is being used, and how much isn’t, and it’s just sitting there.
Well I’m glad to report that this problem can be considered… still unsolved.
This is not a blog post on how we solved it. It is very difficult, even
impossible I could argue, to really know this. For example, the United States
Department of Defence, has multiple allocations of
/8 networks, some of which
have never been advertised to the global BGP table. These networks never
appeared on the Internet. So no matter what clever method you try, I don’t
think it’s possible to measure their utilization.
One could go even further as to ask “What is utilized and what is unutilized
address space?“. It’s a fair question, which I don’t have the answer to. If I
/24 to a DHCP range, and it only sees 5-10 devices, how much of it
is untilized? All? None? Everything but 15 addresses? Anyways…
As I was sitting in Amsterdam’s Schiphol Airport, I remembered a very good blog post from Ben Cox, who mapped the entire Internet by first pinging each IP Address, and then creating a square image of the responses. You can find all the images in his blog post, which is worth checking out.
As you can see, representing each IP Address as a dot, in a 65536x65536 matrix,
220.127.116.11 goes to position
256*1 + 1*2, 256*3 + 1*4 is not good enough.
As it turns out, people aren’t made to be able to understand patterns,
“closeness”, and other things in such ways. This is where Hilbert Curves come
in. They are a mathematical way, where given a coordinate set
x,y like the
one above, you get back a different set of coordinates, which places elements
linearly close to each other in squares, close to each other, but in the 2D
Space. Again, go check out Ben’s wonderful explanation and animated GIFs, he’s
doing a better job than I could.
So as I was there, with time in my hands, due to an one and a half hour delay, I thought I should create a toolset to achieve the same output, or try and use the same as Ben did. As it turned out, thanks to the wonderful standard library of Go, and with some help from Google, it was pretty easy to implement everything myself, in this timeframe that I had, so I decided to write the code, therefore learning something in the process as well.
So I set on a journey to map the Greek Internet, using a methodology similar to the above. The goal was to create Hilbert Curve maps of Greek IP Space, but animated, with snapshots taken hourly, or even more frequently than that. Then, finally, create animated GIFs of the results.
This project involved three parts: obtaining the data, creating images from the data, and then, finally, animating it. So for simplicity, flexibility, and scalability, I split this into three parts.
The first thing I had to do was being able to
ping large blocks, such as
/16s, in a reasonable time frame, with reasonable stability (get similar or
the same results between scans), and output the results in a processable
format, such as CSV. By using the super easy
go-ping library, I was able to quickly
write a simple, 50-line or so program that accepted a
/16 and pinged every IP
Address, and wrote the results in a CSV file.
I went for a
/16, since I wanted to experiment, and I already had a target in
mind, my university. It was two
for loops, one for the
/24, and one for the individual address within it. At this point I was
quickly going through code, and didn’t spend much time on optimizing it for
other sizes of networks, I wanted something that worked, quickly. My code also
didn’t have any concurrency. It was running with one address at a time. This,
on a public WiFi network, tens of milliseconds away from the target, was
obviously too slow.
After adding some code to run pings in parallel, which was super easy thanks to
Go’s concurrency primitives, I ended up with a tool that accepted three command
b, to get the
/8 and the
/16 respectively, and
p, which is the number of parallel IPs that will be undergoing a scan. As you
can see, I still didn’t have the ability to scan anything other than a
and I couldn’t even spend the time parsing it from a
With this 97 line Go program written, I cross compiled it, from macOS to Linux,
and I sent it over one of my servers. I then set up a
cron job to run this
every hour, after measuring that it takes about 5 minutes to run, with 256
parallel IPs undergoing ping scans. It’s good enough.
As this was running, I still had time until my flight, so I obtained the CSV of
the first scan, and set out to create the second part of this tool, the imager.
This part took a little bit more than expected, so I had to continue this
in-flight as well, but it was easily done on time. The program created a
256x256 PNG, by reading the CSV file the scanner created, and then had a
condition: if the amount of ICMP replies received, is the same as the amount of
requests sent, set the pixel to green. Otherwise, set it to gray.
You may wonder why not just leave it black, and bother making it gray. Well, two reasons, one, it looks better, and two, if I had an error in the code, the default color is black, so I wouldn’t be able to easily tell the difference between offline and error.
And with that, tens of thousands of feet above Croatia, I got the first result:
Looking at this image, you can already tell quite a few things. Now I was lucky
to mostly know the actual situation, so predictions were easier, but more
dangerous to make. The top left part is the beginning of the IP Space. The
.1 of the
/24s. There you can see more compact greens, which means that
all the IP Space is tight. Hosts are sequential. Since it’s the first few
subnets, and it’s packed, I would guess these are point to point network links,
and servers. Indeed, PTR records can confirm this.
Two large but not tightly packet squares that appear, are
/22s, and they have
a lot of devices, but not so densly. Since this is noon, with a lot of people
there, my guess would be they are the WiFi subnets, with clients. Same for the
two more rectangles, which are
/23s. But by looking at this image you can’t
verify this. Luckily PTR records always are your friend.
Looking however at just a university’s IP space isn’t that interesting. So I
configured a few more
/16s to the scanners, of various types of companies,
and had them run hourly, for a period of two weeks.
Finally, today, I wrote the final part, the animator. It’s a simple 56-line Go
program that takes a bunch of PNGs, converts them to single-frame GIFs, and
then finally creates one large animated GIF. It was a really nice to write
program as it allowed me to finally see everything come together. It takes a
list of files, from
stdin for example, and then creates an animated GIF with
the content of the files, in this particular order.
As you can see, I tried to make these programs as flexible as possible, so in
the future I can use them for other things as well, without modifying the code.
For example, if I want to create a daily version, I can easily just
So, without further delay, here are the results, with some commentary in between them. They are hourly images, with each hour being one frame, over a veriod of two weeks.
We first start with the lab rat, the first choice of a
/16 owner that it all
As you can see, the data is much more rich now: you can easily draw better conclusions. For example, the subnets I guessed were the WiFi, indeed vary througout the day: as it gets closer to 09:00, they start to gain devices, they peak around noon, and then they are almost all gone. Moreover, in the weekends, there’s almost no use there at all.
Next up, another university, this time NTUA, in Athens. They too own a
so it’s nice to compare between the two universities:
Here you can see some similar patterns: continuously alive hosts, indicating servers, network equipment, workstations and labs, and WiFi. Looking at the WiFi parts of the image is like looking at lungs, inhaling and exhaling as the day progresses.
Moving from higher to lower education, I then picked the Greek High School Network, which is a national network that provides connectivity to schools all over Greece.
This is the first result that we will look at that is a victim of NAT. Due to the small amount of IPv4 Address Space, most access networks are behind NAT: they cannot afford to use public IPs for their users, and are all hidden behind one address. So looking at the results here isn’t particularly interesting, with the exception of some areas that could warrant further investigation.
GRNET is the Greek National Research and Education network, and is a state-run
company that provides services and connectivity to Greek universities, research
centers, and culture-related places. I tried to sample two of their
a test, to see if I will observe large differences between them.
Here’s the first, 18.104.22.168/16:
And the second, 22.214.171.124/16:
Since both networks, from what I know, are allocated further to academic
places, there’s not much uniformity here, especially in the second
some WiFi / Desktop networks are still visible, and you can also spot in the
first network two large pools of mostly-static devices. This is GRNET’s hosted
IaaS pools, to the best I can tell, Okeanos,
Now moving to commercial ISPs, undoubtedly the largest is OTE, and I think by
far. I have picked the first of their multiple
/16s, and by looking at the
PTR records, I could confirm that this was used as an access network for their
Here you can see that they use the entire
/16 equally. There’s no hot spots
like with the previous networks. This is, again, NATed IP Space, so each dot
represents a home or business customer’s DSL router. Now this is really really
sparse, so I imagine most CPEs provided by OTE block pings on their PPPoE
interface, and these few that respond are either older models, had their
settings changed by the user, or had the entire router replaced with a 3rd
Now if you spend some time looking at this image, you will observe something
weird. Entire areas go gray and disappear. There are not necessarily
aligned blocks, but usually are. I don’t know why that happens, but some
possible reasons can be outages, IDSes detecting and blocking the pings but
only for a few hours in two weeks, capacity and congestion issues, with packet
loss to these destinations, or many other things, but certainly it’s not an
issue with my code.. :P
Next up, moving to Vodafone Greece, which should by now include CYTA Hellas. They’re also a large commercial ISP, which provides DSL to their customers.
Vodafone is absolutely packed with customers. And they certainly do allow pings from their CPEs. This is by far the greenest GIF so far. I can spot an area that is empty for some time, and also a few large blocks that are less densly populated, but I am not quite sure what’s the difference with them. To be honest I didn’t look into the matter further.
Moving on, we have Wind Telecommunications, with one of their
/16s of what I
imagine is DSL customers, like OTE and Vodafone’s networks.
Wind is also doing quite nice, with many customers and highly utilized address space. There exist some gaps however, which is not clear how they are used. Are they IP space with firewalling? Not used? Something else? I don’t have an answer right now.
Finally, on the land of commercial DSL ISPs, we have Forthnet, the first Greek
ISP. Again, picking one of their
/16s, at random, to examine how it is being
They are clearly making use of the entire network as a PPPoE pool, without any special distinctions. Like most others in this list, their CPEs respond to ICMP requests normally.
Moving away from the commercial ISPs, onto COSMOTE, which is a commercial ISP, and more specifically the mobile part of OTE. This is the IP Space they use for NATing customers of their 4G network.
And yes, this is by far the most boring and grayest GIF of this blog post. No response, whatsoever. Since this is a mobile network, all these IPs should be on the provider’s equipment, and not on customer devices, and they made the decisions to not respond to pings. Fine..
While not a Greek company, and without any presence in Greece at all, Hetzner
is one of the largest IaaS and dedicated server providers in Europe. They are
based in Germany, and this is where they use the prefix above. They are
included here purely to see how a pure server block looks like, as we don’t
have any Cloud or server providers in Greece that I know of, and have filled an
/16 with customers yet.
As you can see, most of it is static, for the entire duration of the measurements. Not much change in a server-only prefix. Especially with a hefty setup fee. Sure, there exist some small changes here and there, probably of new servers being purchased, or old servers being returned, or maybe from their Cloud service, but it’s mostly static.
That’s all the results for today, and do stay tuned for more content similar to this, including the measurement of more networks, more countries, etc. I do not have a target yet, I am just coming up with potentially interesting ideas for measurements, and if I find any, I will definitely collect them and publish a new post. If you have any ideas, or can explain some of these behaviors above, feel free to contact me, so we can further investigate!