DOTLAN EveMaps

CCP FoxFour announced yesterday on the forum that there’s a CREST point incoming for getting information and states about incursions. Yeah! Finally! It’s been a long time since I’ve asked for exactly one of those information layers that are public available on the ingame map but not accessible outside.

The CREST endpoint is already available on SISI to test, so I digged out some of my prepared stuff (long time ago) and writing the background update daemon and the started with the initial frontend. While it’s not yet completed and of course the CREST endpoint not migrated to TQ, I wanted to share the current state and look with you here.

This is what you get from the CREST endpoint:

• Constellation and Staging System
• State (Established, Mobilizing, Withdrawing)
• Influence / System Control
• Boss has been spotted

With the ability to track incursions changes and influence over time you’ll get a nice history about all the incursions, where they happend and how they progressed over time. I only have to decide in which section I’ll pack the incursions stuff into, so I don’t overload the navigation bar or the site structure. Toughts?

Here is what the current implementation looks like (with random data):

Some of you may have already stumpled across this feature (in fact it’s already online for 7 month). You can overlay your current ingame standings on the you all your favorite online maps.

How does it work? 

If you’ve a API Key added to your user profile, you can select one character which standings (alliance + corp + personal) are getting used to be shown on the map like in the example image above. Your standings (of the selected char) are getting updated once every 24h as long you’ve been logged in on DOTLAN EveMaps during the last 7 days. Otherwise you can manually request an update on the standings page.

P.S.: I’ll try to introduce you to a couple features that have been added over the time but never got mentioned in the blog as an announcement.

5 years ago that I’ve integrated killmails from eve-kill.net into DOTLAN EveMaps. With the recent development that Karbowiak, Squizz and others from the Team EVSCO have put into their next-generation killboard zKillboard it was only a matter of time until I should switch to the better and more flexible API.

After a short examination I had some ideas that would fit perfectly into the theme and would make the integration more useful. Their new API allows me to walk more deeply through zKillboard’s whole database. Therefore the display isn’t limited anymore to the last 48h as it was before. You now get “endless scrolling”: You can now scroll down as far as you want, you even can select a specific date where you wanna start look at killmails of the selected system, alliance or corporation. For more details you can always hop over to zKillboard to get the full coverage for your favorite battle.

And with the new announced killmail cREST endpoint this all sounds very promising.

Links:

• https://evemaps.dotlan.net/system/B-R5RB/kills#2014-01-27
• https://evemaps.dotlan.net/system/Asakai/kills#2013-01-27
• https://zkillboard.com
• https://neweden-dev.com/ZKillboard_API

Couple days we reached my 100% goal for the donation drive (1.500€). The EVE Online community is really the best and I’ll try to do my best to provide a useful and stable service. In the mean time I’ve already ordered my replacement drives, ssds, etc and even opted to get more RAM (all ~2.500€). More memory is always better, ZFS loves more memory and it provides me more flexibility when playing around with other virtual servers.

In the beginning I’ve asked for opinions regarding the storage solution. At this point I was about 90% sure that I’ll go with the hardware raid controller (which I had experience with) and some SAS drives. But you all and some friends poked me in my back and pointed me into a different direction. ZFS! I’ve had used ZFS at work multiple times but always in combination with Solaris. ZFSonLinux has left a bad taste in my mind due to its first linux introduction zfs-fuse) and the license issue (the reasons why zfs is not part of the official kernel). The more I was reading and researching about performance tuning, integrity and features the more I felt comfortable with it. And not having to buy the 600€ HW Raid Controller offered me the possibility to buy more SSDs which I could use as ZIL/SLOG (write-log for sync) and L2ARC (Read Cache) to speed up the overall performance of the storage.

As mentioned yesterday on twitter: I got everything prepared and within 1.5 hours I could swap out the drives, insert the controller, do the cabling and do some basic tests. The migration of the storage itself is/was something that I wanted to do later after I’ve tested everything. Apart from some minor issues (not all of the ram got recognized (again, more on this later) and forgot to check the system clock) it all went smooth.

Read the rest of this entry »

Wow the donation drive is running pretty good. I really appreciate every single donation, so I want share you with my current thoughts with you. Right now I’m about 50/50 between dedicated hardware raid vs zfs. In the beginning I was leaning more towards the HW Raid solution cause I know them from work, but the more I dig deeper into ZFS docs or blog posts the more I’ve to think my setup.

It’s still some time to go and I don’t have to hurry up (since there’s no world going down tomorrow). When spending that much money I’ll and should take my time to carefully check my options and write them down to have an easier comparison between the 2 final solutions. Read the rest of this entry »

Failing hard drives aren’t fun at all, but when you detect them early enough there’s a good chance that the sword will pass you.

The latest failed/replaced drive in my server was in 2009 and thanks to raid1 nobody experienced any bigger outage. Now one of my very old drives starts to fail (I think after 6 years that’s okay).  Luckily it’s only one drive of the raid array for the host server and not the raid array where my VMs (evemaps included) are located.

But when I started thinking about what drives I should order to replace my old ones with, I started to think about the whole disk configuration. It’s easy to replace some drives with new ones and be happy, but maybe it’s worth taking some extra money and get some shiny drives with a dedicated raid controller (instead of using linux software raid) and add some ssd caching to squeeze out better IO performance to be ready for the future. Maybe this could help me to tackle my nightly IO performance issues as well when backups are being done. I’ve already tuned various bits and pieces, but as the data grows sometimes you just need bigger pipes to get everything done in time.

It’s already nearly 3 years ago since I’ve done the last major hardware upgrade. In the mean time I’ve added a 2nd CPU to make use of all my ram modules (which didn’t worked out, I should check this again).

My current idea would be to replace my system drive with new ones and move all VMs (evemaps included) on the new drive as well. The old 1tb sata drives should remain in the system for backups (rsnapshot) only.

• 2x 2TB WD SAS Drives (raid1)
• 1x 100GB Intel DC S3700 SSD
• Dedicated Raid Controller with Cache + BBU/Flash + SSD Cache Option

An alternative could be (to avoid the HW Controller)

• 2x 2TB WD SATA Raid Edition Drives (raid1) or other vendor
• 2x 100GB Intel DC S3700 SSD
• using software  for ssd caching (linux software module)

I’m still evaluating my possibilities, the deeper I look the more possibilities do I find. Sometimes i lean more towards a dedicated raid controller solution (from LSI) a bit later I find the software solution with ssd caching based on drivers like flashcache, enhanceIO, dm-cache, bcache, vcache, etc more flexible. Hard to say which suits better.

Any recommendations/thoughts?

As you can imagine a good configuration isn’t cheap. If you like the evemaps service, feel free to check the donation link on the right side, every cent helps me.

After the Odyssey expansion is already out for quite some time I finally found some time to update my database and remove all old static ice belts and replace them with Hidden Ice Belts.

But that’s not all. CCP even removed ice resources in 43 solar systems. I hope I got everything right.

Can somebody tell me what happened to systems that more then 1 static ice belt in the old systems. Do they have now 2 ice anomalies or did CCP replaced 2 static ice belts with 1 anomaly? Haven’t checked it out myself yet.

After some couple days preparing the database and testing the first results the integration into DOTLAN EveMaps is finally done. Some minor tweaks and graphs will follow someday, but the biggest thing (keeping track for all district changes and visualize them is completed.

What has changed?

• I removed the Home button in the top navigation to make some space for the new Districts button on the right. You can still click on the big DOTLAN logo or just use the universe page if you want to select a specific map
• A Background Daemon is collecting all the data provided by CCP’s new and first CREST Endpoint.
• I’m tracking: Ownership, Clones, CloneRate, CloneCapacity, CloneGenerationState, ReinforceState, AttackedState, LockedState, Surface Infrastructure Type
• All detected changes will be logged, stored and available in the live ticker (also as RSS Feed)

What pages are new?

• Regional District Listings + their states (currently there’s only one region filled with districts. Likely more in the future)
• Top Alliance/Corp District holder (sorted by complete System, complete Planets, District Counts)
• Changes listings
• System > Districts > Details
• Alliance: list of districts their own
• Corporation: list of districts their own

Screenshots

Links:

• Region: Molden Heath
• Top Alliances / Top Corporations
• Recent Ownership Changes
• Daily Changes
• Live Ticker

Please bear with me: I’ve never played DUST514 (yet) so I hope I got everything interpreted right and that my data visualization makes sense. Feel free to drop me some feedback or a donation below 🙂 I just hope that after releasing my DUST Integration CCP won’t pull back and remove everything again like it happend one week after the Dominion Release with the SovereigntyStatus API

Hope you enjoy the DUST514/Districts Integration:

As we speak I’m already working on the integration of Dust514 planetary districts. Since CCP released their first public CREST endpoint I started last week to prepare my backend and the database  to get the new shiny imported. The CREST results are already getting imported and changes are being tracked. I just have to work now on the visuals, presentation and integration of the Districts, their changes and the top holder into DOTLAN EveMaps.

Here are 2 work-in-progress screenshots:

More to come soon.

This feature is already 2 or 3 month old, but I want to introduce it to you and your daily hunting habits. It was at fanfest when Dabigredboat (Goons) and myself had a talk and he came up with an issue that he always had to compare the NPC Kills of the previous hour with the current NPC kills stats to determine where ratting has increased or decreased. That’s how the NPC Kills Delta overlay was born. So what exactly do you see?

Here’s an image (animiated gif) showing the difference of the normal NPC Kills overlay compared with the NPC Kills Delta overlay:

Using the NPC Delta overlay hunters can easily determine if ratting shifted into some else part of the space.

For example, you’re camping system/constellation XYZ it could be possible that the NPC Kills in your covered area are decreasing because pilots don’t want t get caught with their pants down and they’re shifting their PVE business into other areas of the space.

Good hunting!