troubleshooting SQLite issues in TileMill
Hi,
I've been creating a bunch of maps from the same sqlite source data, and using the attachDB feature to bring in new attributes. Always the same sqlite source spatial data, and usually the same attached dB (non-spatial). Trouble is, these projects with attached DB's seem... "sensitive", for lack of a better term. Each behaves differently.
For example, adding a new field to my attached table (a field not yet being used by the map), results in "No response from Server". The project no longer works. Yet an identical project using the same sqlite spatial data and attached db does work. One broken, one fine, same data.
The TileMill log is silent on the matter, in fact it never says anything but "Started". So what I'm wondering is, does TileMill have a debug mode, or verbose logging option, etc. ? Anything that might help users understand where TileMill is chocking on some data. And is there any way to clear a project cache/etc. that might help refresh data troubles?
My best solution to date has been to rebuild projects from scratch, which sometimes works.
Many thanks,
-Eric
TileMill 0.5.1
MacOS 10.7.1
Chrome 14.0.8xxx
Comments are currently closed for this discussion. You can start a new one.
Support Staff 2 Posted by Will White on 12 Oct, 2011 10:55 PM
Hi Eric,
You may be experiencing this issue. Try vacuuming the sqlite databases and loading the project again. If that doesn't work, can you look in
~/Library/Logs/DiagnosticReports/for a crash report from TileMill?Thanks,
Will
3 Posted by Eric Spry on 13 Oct, 2011 12:10 AM
Hey Will,
Thanks for the suggestions. I vacuumed both my spatial data and related table using BASE. Unfortunately, it didn't make a difference in the project - still no response from server, and a maxed out processor until I kill TileMill.
For the crash logs, I found loads for NODE - does that help? I had 9 today, so I zip'd them up in case it's useful for you to review.
-Eric
Support Staff 4 Posted by Will White on 13 Oct, 2011 01:05 AM
Thanks, Eric. I think these will help. I've added Dane to this discussion for his expertise in this area.
Support Staff 5 Posted by Dane on 13 Oct, 2011 09:10 PM
Eric,
yes, this issue should be fixed in mapnik (to be included in the next tilemill 0.6.0 release).
But, are you able to post a db that is experiencing this problem so I can take a look and confirm?
6 Posted by Eric Spry on 13 Oct, 2011 11:54 PM
Hi Dane,
Thanks, that is encouraging to hear. I can't post this data here, but I'll see if I can get it to you directly.
Getting back to my original query - are there any options for users to get more information on chasing these problems? I usually test in QGIS or other GIS package to validate - but SQLite is a little curious as it doesn't seem to enforce things like data types. Would you recommend that fields always have "types" (VARCHAR, INT, etc.), what other kinds of integrity checks make sense when the data is not behaving with TileMill.
I'm happy to hear there is a new update coming, but I was hoping to add a few tools of my own to diagnose these issues (debug mode, verbose logging, etc.).
Appreciate the help and I'll look into sending you that data.
-Eric
Support Staff 7 Posted by Dane on 15 Oct, 2011 01:21 AM
Eric,
Thanks for the effort in the data - see your email and will take a look as soon as I can next week.
Re: your original query - yes agree SQLite is quite "curious" - it's defaults are very forgiving but its errors also often very silent. Datatypes actually are independent in sqlite of how they are declared and rather they are handled though "type affinity" - http://www.sqlite.org/datatype3.html.
Regarding your own debugging tools - in your case the console was the right place to look. Those crashes are a direct result of this sqlite bug. However you also mention pegged CPU, which sounds potentially like another issue. For a pegged CPU/hung app the #1 thing to do ( I see you are on a mac, right?) is to open up Activity monitor, find the "node" process and sample it. The result will not make a ton of sense to you but if you post it back here I can help interpret it and it will show exactly what TileMill is doing when it is hung or not behaving correctly.
Eric Spry closed this discussion on 23 Jan, 2012 08:55 PM.