DOTLAN EveMaps

Yeah it’s weekend and the ideas are coming, a solution is also already in mind, so why not doing it right now. It’s only a couple of hours ago since the last post dominion update release but this one is too good to be postponed.

Updates:

• Added a filter option for the Jump Range Tool. You can now also list highsec systems, which are in jump range. That way you can easily search for a good jump off point (for your Jump Freighter or Black Ops) from a secure high security system to your favorite destination system.
• Additionally I fixed a bug with the radar page which didn’t worked correctly for wormhole systems.
• Fixed some smaller bugs.

Warp Drive Active #50 – The Hotdrop Edition

This was kinda funny. Winterblink, known for his warp drive active podcasts (mostly together with Urban),  asked on twitter for some randoms chats over skype. Friendly as I am, I replied and we started to have a nice funny talk until he said

“This will kinda be a hot drop *BEEEEP* This call is beeing recored *BEEEEP* Alright, joining me this week on Warp…

Aaaaaah yes. I was his first hot drop victim, but in the end we had a nice talk. Well anytime again sir! Until next time.

Go out and listen:
http://www.warpdriveactive.com/2009/12/05/podcast-50-hot-drop/

After a few days playing around with CCP’s new Sovereignty Status API, I think it’s time for an update. Here we go.

• Structure Changes (live) for the last 48 hours (overall), per region or per alliance
• Structure Listing per region or per alliance
• Moved the API Updated to the bottom of the page (too many timestamps)
• Extended the Factionwarfare Live Ticker for the last 48 hours instead 24
• Fixed some bugs with maps and the jumps/npc/shipkill overlay numbers
• a couple of smaller fixes around the world
• Added a dropdown list to switch between the regions if you’re in the list view.
• Finally I also added a Support Page. Here you can find all informations how to contact me or show me your support by donating or buying GTCs etc.

That’s all for today: Now happy hunting for onlining TCU’s and happy donating 🙂

It’s finally done. The DOTLAN EveMaps Dominion Edition is ready to go. I hope you all have fun. If you like my work feel free to donate stuff to me: ISK, Minmatar Stuff (hint: I can finally fly Minmatar Black Ops and Caps o/) or just leave a comment here.

Let’s round up the new features and changes that happened in the last 4 month.

New Features and Changes

• IGB Integration
  • Support of the new javascript Ingame methods (for showInfo links, etc)
  • New Theme which is optimized for smaller browser windows
  • Added a ‘Radar’ page which displays the current region with the system highlighted where the user is currently in. (only for trusted pages)
• Zoomable/Moveable Region Map (svg only)
• Zoomable/Moveable Universe Map (svg only)
  • Integration in the Jump/-Range/Route Planer
  • Integration into Region/Alliance View
• Outposts
  • Added outposts details including ownership and name history
  • Added live outposts changes
  • Track outpost ownership/name changes
  • Reduced API calls from every day 24h to every hour!
• Faction Warfare
  • Added live changes (contested/uncontested/lost/recovered)
  • Added occupied overview (which alliance has occupied what and since when?
• Systems
  • List sovereignty structures and their states
  • Added Nearest Locations: List closest low/high/nullsec system or closest station with $service
  • Links to outpost details
  • List Agents in Space
• Alliances
  • Added long term statistics for members, corps, systems and outposts
  • Links to outpost details
  • Link to universe map with the systems highlighted the alliance is holding
  • New Alliance Ranking/Listing. You can select between sov holding, all and dead alliances
  • Alliance listing is limited to 100 alliances per page (including pages switcher)
• Sovereignty
  • UPDATE: Added sovereignty structures list which are not in a normal state (You can watch things onlining)
  • Added a live sovereignty ticker with all recent losts/gains
• Misc
  • Moved the unified search in the header section (available on every page)
  • Merged Alliance + Corporation + NPC into one section
  • sorted table column gets highlighted
  • Active page section gets highlighted (in the main navigation bar)
  • Agents in space will be linked to their right location/system
  • Many tweaks, fixes, updates to make DOTLAN EveMaps a better world to live in for us all. 🙂

Background system

• Switch from cron-based scripts to daemons
• Rewrite of all update scripts and daemons who’re keeping the database up2date
• Rewrite of API backend handlers
• Added cache clearing on cache groups

Database

• Complete change of all tables that are handling historical data
• All startDate/endDate fields (for historical data) switched from ‘date’ field type to ‘datetime’ to be compatible with dominion sovereignty changes
• With dominion all sovereignty Levels will be changed to level 0/1 expect 0-4, old sovereignty Levels will be preserved
• Added outpost history table
• Added summary table for kills/jumps in the last 1/3/24 hours (to reduce query run times)
• Added more historical sovereignty data (thanks to exi from #eve-dev) starting now with June 2007

And what’s coming after all that? Sure the work is still not finished with all the new shiny stuff which is coming out with dominion. I’ll focus more on the Sovereignty Status API and try to merge more data together.

Possible new stuff could be:

• Conflict View: Alliances which are fighting against each other (based on contested systems SBU from alliance X is attacking space hold by alliance Y, and so on.
• Better startpage with more likely customizable features and widgets
• Notification services (jabber/email)? (In the far far future)

During the rewrite of the backend (in preparation for the upcoming expansion), I made some very big enhancements to the database. Everything must be able to get updated every minute, because sovereignty can change anytime, but not only sovereignty can change anytime even outposts are playing ping-pong games or alliances are growing and disbanding anytime.

After the biggest part in the database conversion, fixing, and reimporting old data (from old stored xml api files) has been done, it was time to see what stuff I can create with the revamped and freshly imported data.

The outcome? Everything is getting LIVE (or at least with max 1hour refresh timers) and here are screenshots from some of the new features I announce for the dominion DOTLAN EveMaps release: Outpost history/details and faction warfare changes/informations.

Do I need a release party for the launch of the upcoming DOTLAN EveMaps release? 🙂

Everyone is waiting for the release of the upcoming expansions ‘Dominion’ which will revamp the sovereignty system. Tracking and visualizing the current sovereignty and territorial status/changes was and is still one of the major key parts of DOTLAN EveMaps. After waiting weeks to get response from the CCP Devs about upcoming changes and their connections with the API, I can finally say there’s light at the end of the tunnel. We got heard. While CCP Elerhino is preparing his Devblog which should come out in the next days, he and CCP Incognito already answered a few questions on the eveonline.com board and released a few new details.

What are the basic Sovereignty API changes?

• Sovereignty can change every minute and not just only after downtime. With this the cache timer will get reduced also to around 1h.
• Sovereignty Levels and Constellation Sovereignty are gone.
• corporationID will get added to the API to indicate which corporation owns the Territorial Claim Unit/System.
• There’ll be a new API SovereigntyStatus.xml.aspx which will include SBU, TCU, Outposts, Upgrades? in each system (quote: relevant informations)
• If the system index levels (strategy/military/industry) are getting added one day is unsure right now.
• The new APIs won’t be released on day one as they’re lacking behind the Sovereignty Changes.

What does this mean for DOTLAN EveMaps. Which parts are affected by this?

• Sovereignty can change every minute:
  The complete sovereignty database needs to be converted. Before Dominion, sov only changed once a day during the downtime. In the upcoming EveMaps version you can see sovereignty changes every hour (or even minutes, depending on future API’s changes or caching changes).
• Sovereignty Levels are gone:
  I’ll likely convert every single system from Sov 4, 3 or 2 to sovereignty 1. I know that after dominion there’ll be no sov levels at all. But keeping the database compatible with the past and the future will a big deal. I don’t really want to lose my historical data. Neither you.
• Sovereignty Status:
  If it comes true and CCP is releasing the SovereigntyStatus API soon after the dominion release, I’ll start tracking this API too. Right now I can’t really tell what the API will contains: Likely it will contains a list of the Territorial Claim Units (TCU) Sovereignty Blockade Units (SBU), Infrastructure Hubs and Outposts in their current state. That would be a really nice to show which systems are currently under attack/siege/whatever. *Dreaming* If they would add the hub upgrades to the API Export you can watch how your space is developed (or the one of your enemy). That would make it very easy to plan assaults against your enemies infrastructure. But right now we’ve to wait what CCP is including in their API and how long we’ve to wait for this.
• More APIs, more data, more, more, more

As I’m currently rewriting the whole DOTLAN EveMaps backend programs that keep track of updates, etc, etc. I found various things that I could also track and visualize and even fix.

Additional Changes:

• Faction warfware systems:
  Sorry for the past, but I never took a bigger look into the faction warfare API. Systems are getting contested and uncontested every few hours (depending on the successful or unsuccessful assaults of the militia). Basically the contested state will get updated very hour (API cache timer).
• Faction warfare stats:
  Maybe I’ll add them in also in a future release.
• Alliances changes:
  I’ll also track alliances changes every hour soon. You’ll directly see which corporations are joining/leaving your enemy (failcascades live!) or if CVA or BOB MK3 gets disbanded again.
• Outpost history:
  Similar to the alliance changes I’ll keep track of outpost stations changes. That includes corporation/alliance ownership (station ping pong) aswell as the funny station renaming battle (Youtube @ 5:07) between Goons and the GMs in 77S8-E. I still have a daily outpost naming history for the last 14 month.

I’m looking forward to for all the upcoming changes. btw. It seems the DOTLAN EveMaps Server is running stable again, apparently it looks like a bad memory configuration during the upgrade at the start of November caused the crashes.

If you’ve may noticed, the evemaps webserver is crashing every now and then. It looks like some background scripts, which are keeping the database up to date, are crashing and killing the server at the same time, leaving the server in an unmanageable state. No logfile outputs, no console output, nothing. The current solution is to destroy the virtual machine instance and start the server again, which is … not acceptable.

I’m thinking about installing a clean virtual machine and move evemaps to this or rewrite the maintenance scripts or compiling php from scratch. Right now I’m still searching for the root cause, which is hard to find without any usable logfiles or debug output or beeing able to reproduce the crash.

At least I’ve setup a monitoring system on a different computer which notifies my via jabber/email and later text message to my mobile, if the server crashed again. Maybe I’ll add a auto-restart script on the xen dom0.

Update

So far the server keeps crashing every day without any logfiles or error messages. After several tries to find the crash reason, webserver/php reinstall, code rewriting, I decided to move evemaps to their own single, fresh installed virtual server instance. Lets hope this helps. Otherwise I might consider compiling php from scratch rather then using a precompiled debian packge. I’ll also try to get to the datacenter and do some memtests with the current server setup this weekends.

Update 2

I’ll be in the datacenter around 19:30 GMT to exchange ram modules and do some memory checks (just in case). DOTLAN EveMaps will be down for around 30 to 45 minutes.

As I already announced via Twitter, I’m done with coding a full zoomable Eve Online universe map. It’s completely done in SVG and Javascript. Sorry IE Users, for now: get a real browser. Sure SVG and Javascript isn’t that fast compared with native java or flash code but I think in any way, it will be a good addition to the complete mapping solution.

Right now the map is completly zoomable and movable can display routes, jumps and jumpbridge maps and has the ability to set their focus on systems, constellations, regions or alliances. Based on that code I can create additional features in the future (depending on performance, presentation, etc).

I suppose, I should really make the release update on the live version of evemaps soon 🙂

… there’s much going on. After the first announcement of the new ingame browser around 2 month ago, I started working in the background to get everything prepared for the new expansion. The last weeks and days before the fanfest were even more work heavier, even during the flight to Island I was still coding to be able to present improved EveMaps version for the Ingame Browser to the devs. During the fanfest itself I was trying to gather more ideas which I could realize.

What did I’ve done in the background, what are the new features:

• Switched live and testing environment to subversion, for better development and code review (even I’m doing everything alone)
• EveMaps with theme support: light (current), dark (black background), igb (Ingame Browser, optimized for small  window sizes)
• Full IGB Browser Support
  • Regions, Constellation, Alliances, Corps, Stations, Agents, Moons, etc are will show an info icon right next to their name when hovering over it.
  • Systems are also get some additional Icons like a link to the Ingame Map, etc.
• Zoomable Region Maps
  • You can zoom into the normal regional and pan around like google maps.
  • Before you ask: No there’s no full zoomable universe map, maybe someday, maybe not.
• Long Term Alliances Stats
  • You can select a start/end date range and see the history for memberships, corps, systems and outposts. Great, if you want documenting some of the recent failure cascades *hint* 🙂
• Nearest Locations (per System)
  • In the systems details you’ll find a list which will show you the nearest (in jumps) 0.0/low/highsec system or the next station system with refining/repair/medical/research/factory slots.
• R&D Agents will show their research services.

Most of the features, which I’ve listed above, are currently tested with Firefox and Chrome. I’ll have to see I can somehow realize some of those functions with the Internet Explorer ….. no comment. Anyway, everything is still in development, needs more testing and tweaking. But basically that’s what you can expect with the Dominion release of EveMaps.

There’ll be more updates coming out for dominion, but atm I’ve to wait until CCP is realising more informations about the sovereignty system, their upgrades and the how everything is integrated and linked with the API.

Fly safe.

It’s hard to describe the Eve Online Fanfest with only a few words. But the most simple and best matching way would be to quote the answer of a CCP Dev when I asked him, what I could improve on the DOTLAN EveMaps.

“Keep Beeing Awesome!”

I’ve seen so many new things, met so many exciting people and talked about uncountable things in (mostly) the API/IGB roundtables that it’s hard to clear my head and realize that everything is already over. But I’ll try to summarize a few things.

(more…)

Got my coffee and already sitting in the train which leads to Frankfurt Airport. The final destination? South Africa ….. no I’m kidding that will have to wait. Sure it’s the Icelandair Flight to Iceland. Fanfest here I come!  I hope it will be an exciting weekend. If you wanna talk to me or just see and discuss the upcoming evemaps changes for the next expansion or suggestions just look for a guy with a (mostly black) shirt with dotlan logo on it 🙂 I’ve really some interesting upcoming changes (also in regards of the new ingame browser).

I’ve stressed my subversion system quite a lot with many changes. It’s a pain to clean all the small things from the last year.

btw. 1) Blogging, Surfing in the train while rolling with 300km/h is funny.

btw. 2) Fixing mysql performance issues short before bedtime is a bad idea. I only had around 4 to 4.5 hours of sleep. At least I could find 3 issues:
Note to myself: A table with only 700 entries and no index could lead to serious performance if combined in a bigger sql query.