Leaflet, ArcGIS Server Invalid SRS

I encoutered some problems with ArcGIS Server and Leaflet WMS api. Namely its because the API sends the "srs" parameter to ArcGIS server with the value EPSG3857 while ArcGIS server expects EPSG4326.

Basically these EPSG values refers to the spatial reference of the maps in which they are drawn, they are what determines where exactly the position is when you say a certain location like 1N 103E.

How can you discover this error? You can use firebug to trap what URL calls the webpage is calling the WMS server, then you paste the URLs into a new window there should be an xml telling you the error if there is any, if not then a picture with the portion of the map as specified in the URL should be displayed.

Found the solution in Google Groups bascially you need to setup the map upfront by stating what spatial reference it should use.
 e.g

var map=L.map('map',{crs:L.CRS.EPSG4326}).setView([51,0],13);

It's specified as a option to the map function.

Weblogic and Spring Log4jConfigListener [java.lang.IllegalStateException: Cannot set web app root system property when WAR file is not expanded]

If you are using Weblogic and Spring's Log4JConfigListener you might encounter a problem when in your testing envrionment when you deploy from your IDE to your Weblogic instance it seems ok.

But when you package it into a war file and deploy it. You get an
[java.lang.IllegalStateException: Cannot set web app root system property when WAR file is not expanded].

This is due to the Log4JConfigListener  requiring to look up the log4j.properties file and from a war file it can't do that.

You can either always deploy it in a exploded war file or simply adding the following to your weblogic.xml file.


         true


This should solve the problem.

InfoQ: Scaling Pintrest

Some things i learnt from the a InfoQ video about Scaling Pintrest.

1. Clustering is complicated

This changed my whole point of view about High Availability. The comment made about how your application is depending on some complicated algorithm for synchronising and making sure every node is in a correct state is bad because if anything goes wrong you will have a lot of trouble. Which is a problem with a lot of clustering solutions, they are complicated to setup and if anything goes wrong you have a hard time figuring out what's wrong, most of the time I just pray and reboot the servers hoping that everything works.

Compared that to Sharding where you know how your data is stored and partitioned (You wrote the algo). But the problem with Sharding is that your application will need to be smarter on how to find the data, perform joins and you need to have a robust plan for data migration. You will need to plan for future capacity when you are designing the Shard so you don't have to perform migration too often.

The video does show how they designed their sharding algo and considerations made.

2. Keep things simple in the technology stack

This has got to be the rule that everyone knows but damn hard to follow. The Pintrest guys started off with a complicated stack but eventually they settle down to well known technology in terms of maturity, stability and simplicity. They eventually just settled on MySQL, Redis and Memcache. Which are all well know and simple to understand systems.

3. Cache, Cache, Cache

For any large system, caching your reads is important for performance. (But you already know that :))

Agile Fluency

Nice Article from Martin Flower (or more specifically by Diana Larsen and James Shore) on Agile Fluency.

Good read on how to agile teams should evolve

http://martinfowler.com/articles/agileFluency.html

HBase schema design

The HBase reference documentation dosen't really say much about schema design and most of us coming from a relational back ground will need to change our way of thinking in order to swtich to a more column based kind of designs.

You need to look more at the types of queries that you will be needing to access your data rather that just nomarlise every thing to 3rd normal form as in traditional relational databases. And that itself might not be easy. I will be writing more as i try to design my current application's data base into a Hbase kind of design, and what are the challenges i find.

Below are some of the useful resources that helped me in understanding column based kind of designs.

http://ianvarley.com/coding/HBaseSchema_HBaseCon2012.pdf

http://traackit.blogspot.sg/2010/05/20090713-hbase-schema-design-case.html

Schema Design Guidelines and Case Studies
https://docs.google.com/viewer?a=v&q=cache:Oe0Kf9oQhO0J:india.paxcel.net:6060/LargeDataMatters/wp-content/uploads/2010/08/Schema-Design-Guidelines.doc+&hl=en&pid=bl&srcid=ADGEEShyf8WIXTdsWbYRTWWLHUgvYbWDa2AilTnm6xUWnaHDvpX6gARtzDNBx6GlsbE1hjdZoR3hYBYjk03zlwq8LLkA9VRYlLwihD4S75pxlMtitX640uwrQsOa0cC5K5JPGfhAfQIK&sig=AHIEtbR8TyqVgcc3WFGtpKWX7eZOZBpnFQ&pli=1

Varley Masters report (as referenced by HBase documentation)
http://ianvarley.com/UT/MR/Varley_MastersReport_Full_2009-08-07.pdf

Big List of 20 common bottles necks

All developers and implementors should take a look at this list of bottles necks from HighScalability.com

http://highscalability.com/blog/2012/5/16/big-list-of-20-common-bottlenecks.html.

Great stuff wish i have this list before i started work on any big systems.

Common Lisp the untold story

Nice history about the Common Lisp standardization process by Kent Pitman who wrote the Common Lisp Hyperspec.

http://www.nhplace.com/kent/Papers/cl-untold-story.html