sij/mapnik
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
12666 commits 112 branches 95 tags 181 MiB
Commit graph

1680 commits

Author SHA1 Message Date
Mickey Rose
ec2c5ddbdc gdal: RAII + minor cleanup 2016-03-03 01:48:07 +01:00
lightmare
80d9aebb79 Merge pull request #3340 from lightmare/gdal-register-once 
call GDALAllRegister once, from one thread only
 2016-03-02 22:19:07 +01:00
Mickey Rose
d4566c28d2 gdal: only register drivers once 
- refs #3339, #3093
 2016-03-02 21:21:59 +01:00
Dane Springmeyer
72459b3140 Merge branch 'master' of github.com:mapnik/mapnik into faster-csv-compile 2016-03-02 11:10:23 -08:00
Mickey Rose
c21778fdfc merge mapnik-index::process_csv_file and csv_datasource::parse_csv 
- the function in plugin was already configurable via flags, and only
  contained two un-conditioned blocks that process_csv_file didn't have

- so I extracted the common parts into a separate function (in a class
  holding the flags and state), process_csv_file calls it with default
  flags, plugin sets them from params

- removed namespace ::detail, moving stuff that was used outside
  csv_utils to ::csv_utils, and the rest to ::csv_utils::detail
 2016-02-26 20:28:42 +01:00
artemp
f44b5ccfd9 always query upto num_features_to_query to determine data schema (currently 5) across all access modes (ref #3237 #3238) 2016-02-25 15:29:32 +01:00
Dane Springmeyer
08d7bf2152 no need for template usage for autodect_csv_flavour 2016-02-24 22:24:24 -08:00
Dane Springmeyer
f2782d0b67 [csv] move spirit usage to cpp 2016-02-24 17:42:51 -08:00
Dane Springmeyer
8ffee05048 Merge branch 'master' of github.com:mapnik/mapnik into faster-csv-compile 2016-02-24 16:42:23 -08:00
artemp
9fe049d8af CSV utils - strengthen quote detection logic + tests 2016-02-19 15:05:15 +01:00
artemp
6a9240f3d4 Revert "shape.input - add 'update-cache' optional parameter (default:true)" -- no much use as OS seems to apply it's own caching, doh .. 
This reverts commit f65c799475.
 2016-02-18 17:37:46 +01:00
artemp
f65c799475 shape.input - add 'update-cache' optional parameter (default:true) 2016-02-18 17:34:03 +01:00
artemp
b43ee6dd04 beef up has_index method to check if *.index file is valid (has expected header) (ref #3300) 2016-02-18 11:45:58 +01:00
artemp
cd5105785a csv_utils - fix istringstream initialiser by using explicit iterators pair std::string ctor 2016-02-18 11:35:14 +01:00
artemp
4c0781dbe9 remove redundant unique_ptr usage for local variable 2016-02-18 10:53:12 +01:00
artemp
15f905b272 csv - improved quote auto-detection logic which should catch more corner cases 2016-02-16 16:05:12 +01:00
Dane Springmeyer
85d4596769 move getline impl to separate header to avoid as much spirit inclusion 2016-02-15 19:06:11 -08:00
Mickey Rose
5d1fd56cef csv_grammar: make csv_white_space_skipper a primitive_parser 2016-02-15 15:17:26 +01:00
artemp
ca0c0e5888 csv_util - combine auto-decting separator/quote/newline into one method std::tuple<char, bool, char, char> autodect_csv_flavour(T & stream, std::size_t file_length) 2016-02-15 13:55:06 +01:00
Dane Springmeyer
e76a1d9a81 [csv] use faster version of csv_utils::parse_line in more places + fix error output 2016-02-01 12:23:26 -08:00
Dane Springmeyer
0b22bd87f7 shape_index_featureset: initialize members explicitly - remove debug print 2016-01-29 16:29:43 -08:00
Dane Springmeyer
32bbccb8fd fix #3276 2016-01-29 16:29:12 -08:00
Blake Thompson
b46f87b385 Merge branch 'master' into sanitize_errors 
Conflicts:
	include/mapnik/json/positions_grammar.hpp
 2016-01-07 08:34:52 -06:00
artemp
4c6632cf93 shapeindex : remove duplicate node defs + add default ctor 2016-01-04 13:19:16 +00:00
artemp
88a04a120f remove stderr 2016-01-04 11:56:01 +00:00
artemp
a2b3130d4b rebase to master (remove spirit-x3 changes) 2016-01-04 11:43:27 +00:00
Blake Thompson
dfa62c88d8 fix for santize address errors 2015-12-27 21:40:10 -06:00
Dane Springmeyer
8e6e74dfa2 suppress -Wshadow warning from ogr 2015-12-03 10:22:02 -05:00
Artem Pavlenko
1b6ca8bcd0 Merge pull request #3176 from mapycz/fix-warn 
csv.input: fix warning
 2015-11-23 10:46:10 +00:00
artemp
aee0149a1a GeoJSON - preserve feature ordering with both cache_features=true' and cache_features=false` (#3182) 2015-11-23 10:19:49 +00:00
Jiri Drbalek
7283b758a0 csv.input: fix warning 2015-11-23 08:54:13 +00:00
artemp
58998f8126 cleanup - remove unused vars 2015-11-20 17:45:10 +00:00
artemp
1eab1a3a28 CSV - change 'quote' auto-dection logic to handle mixed cases better 2015-11-20 17:10:55 +00:00
artemp
938702dc8b geojson.input - rely on expectation failure to select optional single Feature/Geometry parser 2015-11-18 14:43:40 +00:00
artemp
998164dc72 mapnik c++ style 2015-11-18 14:42:29 +00:00
Dane Springmeyer
7608040906 centralize warning suppression 2015-11-07 17:53:09 -08:00
Dane Springmeyer
da624b9ef4 update test-data to pull in 9f471a25e3 -refs #3160 2015-11-04 13:57:14 -08:00
artemp
133ca16d3a GeoJSON.input - skip empty geometries (#3156) 2015-10-29 16:54:35 +00:00
artemp
3de31c50ab correct exceptions text 2015-10-29 14:08:08 +00:00
Dane Springmeyer
35dccd4653 remove unused variable [skip ci] 2015-10-23 12:05:56 -07:00
artemp
158729ffa4 remove unreachable code block 2015-10-23 14:24:08 +01:00
artemp
aeaadd423d geojson.input - print line number 2015-10-22 18:32:23 +01:00
Dane Springmeyer
ca63ae0a96 csv: no need for newlines in exception strings 2015-10-22 10:22:58 -07:00
Dane Springmeyer
ab88fe48a4 csv: fixup error messages when geometry fails to parse 2015-10-22 10:10:23 -07:00
artemp
6effbedff3 csv.input/mapnik-index - validate geometry locator and throw early if invalid 2015-10-22 13:29:26 +01:00
Dane Springmeyer
a7e7f088a2 CSV plugin: trigger exception rather than crash when indexing error happens 2015-10-21 12:54:03 -07:00
artemp
c3ccff4f02 shape.input - add stream state check and calculate position limit 2015-10-21 16:33:12 +01:00
artemp
5cb2b8a77a geojson.input - create temp internal mapnik::feature_impl objects with id = -1 for clarity ref #3134 2015-10-21 11:14:13 +01:00
artemp
17ace8a3e8 shape.input - read shx_file_length in shape_featureset and avoid passing wrong file length by mistake ref #3136 2015-10-21 10:55:26 +01:00
artemp
19c1c0afd0 bug fix - shape_featureset now expects shx file length (!) ref #3136 2015-10-21 10:46:49 +01:00
artemp
ab2d86c617 shape.input: remove set_id() method as it's no longer required ref #1020 #1019 2015-10-20 14:18:17 +01:00
artemp
c952db979e use mapnik::value_integer for feature id's 2015-10-20 11:45:39 +01:00
artemp
c053682711 geojson.input - assign incremental feature_id in geojson_index_featureset and geojson_memory_index_featureset 
(ref #3134)
 2015-10-20 11:12:11 +01:00
artemp
26bf021af9 rename large_geojson_featureset -> geojson_memory_index_featurest to better reflect undelying implementations: geojson_index_featureset is most capable in terms of handling large input data with minimal memory requrements via utilising on disk spatial index (createed with mapnik-index utility) 2015-10-19 10:54:52 +01:00
Dane Springmeyer
8813e73cfc rename SHAPE_MEMORY_MAPPED_FILE to MAPNIK_MEMORY_MAPPED_FILE 2015-10-16 13:34:53 -07:00
artemp
4733c7ffc1 geojson.input - geometry_type for disk-index 2015-10-16 17:31:36 +01:00
artemp
942fb3c562 geojson.input - implement spatial disk index featureset 2015-10-16 13:15:20 +01:00
artemp
886ac23c64 geojson.input - force caching features and parse single feature/geometry when cache_features = false but extract_bounding_box fails (input is not FeatureCollection) 2015-10-15 17:35:46 +01:00
artemp
2906706a40 geojson.input - split parsing FeatureCollection and single Feature/Geometry into separate grammars 2015-10-15 15:12:16 +01:00
artemp
b798503623 shape.input - pass shx_file_length to shape_featureset 2015-10-15 11:49:18 +01:00
artemp
a73e8537d8 shape.input - add support for reading offset/record_length from *.shx in shape_featureset (no *.index) - ref #3126 2015-10-14 16:44:43 +01:00
artemp
8562d18cd0 geoson.input - throw an exception if input hasn't been fully consumed 2015-10-14 11:03:07 +01:00
Dane Springmeyer
324f70652b code cleanup / c++11 usage in plugins 2015-10-09 16:07:19 -07:00
Dane Springmeyer
5e8a009278 fix #3120 2015-10-09 15:27:34 -07:00
artemp
0bca3afa4e formatting 2015-10-09 12:49:33 +01:00
artemp
faeae5bc2e c++11 - more concise syntax 2015-10-09 11:26:39 +01:00
artemp
b5fe23ac28 inline extract_geometry 2015-10-09 11:26:10 +01:00
Dane Springmeyer
f7b125c90f use boolean_type (no functional change) [skip ci] 2015-10-07 17:53:32 -07:00
Dane Springmeyer
3bfd997b83 fix -Wsign-compare warning [skip ci] 2015-10-07 14:43:33 -07:00
Dane Springmeyer
6e84bde468 Add key_field_as_attribute (default:true) to control if key_field is added as feature attribute - refs #3115 2015-10-07 12:59:45 -07:00
artemp
b275bcb2c2 update to use latest grammar + use std:: qualifier for 'C' lib functions 2015-10-07 12:34:20 +01:00
artemp
e727fb3877 csv.input - read first line of data to update descriptor (disk-index) + remove stderr 
§
 2015-10-06 16:39:21 +01:00
artemp
d0cf7ad6f4 csv.input - implement get_geometry_type_impl for disk based index 2015-10-06 11:32:21 +01:00
artemp
2c8ad910d7 csv.input - remove redundant trim_copy 2015-10-06 10:00:42 +01:00
artemp
63c73b5057 add initial 'qoute' auto-detection + restore csv_test's 2015-10-05 16:56:33 +01:00
artemp
5c6c8ff7a3 csv.input - restore handling of inline headers and only one line of data without new line 2015-10-05 15:28:57 +01:00
artemp
287028d3e4 fix rebase conflict 2015-10-05 15:25:17 +01:00
artemp
178e39e19a make separator single character and simplify/optimise csv_grammar 2015-10-05 15:23:20 +01:00
bergwerkgis
6b20c8c9f2 fixes #3106 2015-10-05 09:55:20 +00:00
artemp
cbb40d0662 raster.input - throw if reader can't be created 2015-10-02 12:47:57 +01:00
artemp
1bccca9ff8 Merge remote-tracking branch 'origin/master' 
Conflicts:
	plugins/input/csv/csv_datasource.cpp
 2015-10-02 12:27:28 +01:00
artemp
1d320b7133 make quote single char 2015-10-02 12:20:54 +01:00
artemp
98ea1c5cd9 csv_utils::getline_csv - add 'quote' argument 2015-10-02 12:17:38 +01:00
Dane Springmeyer
336170c13c avoid excessive calling of std::ios::widen - refs #3101 2015-10-01 15:34:00 -07:00
Dane Springmeyer
3932cc51b3 add getline benchmark - refs #3101 2015-10-01 14:47:55 -07:00
Dane Springmeyer
2d59fca9f9 ensure consistent, ordered linking for plugins - closes #3105 2015-10-01 13:35:56 -07:00
Dane Springmeyer
05ade151a5 suppress yet more -Wshadow warnings from boost 2015-10-01 13:35:23 -07:00
Dane Springmeyer
3acec350f6 avoid more -Wshadow warnings from boost 2015-10-01 13:35:23 -07:00
artemp
be437eb6b0 add experimental getline_csv implementation which handles newline characters inside single/double quoted strings 2015-10-01 18:33:32 +01:00
artemp
a4e15b5a47 remove stderr 2015-10-01 15:34:49 +01:00
artemp
4273e37278 Merge branch 'master' into csv-disk-index 2015-10-01 15:26:47 +01:00
Dane Springmeyer
d70725b218 fix variable shadowing in extract_bounding_box_grammar 2015-09-30 15:42:42 -07:00
Dane Springmeyer
d6b60188e8 avoid -Wshadow errors from boost when compiling csv plugin 2015-09-30 14:11:16 -07:00
artemp
7a956651fe geojson - avoid passing invalid bounding boxes to boost::geometry::index 2015-09-30 12:50:53 +01:00
artemp
19897a3085 correct exception handling in main processing loop 2015-09-30 12:37:36 +01:00
artemp
848098baeb spatial_index/quad_tree - remove `operator>>' requirement, instead value_type stored must have standard layout for correct (de)serialisation + update unit test 2015-09-29 15:44:57 +01:00
artemp
49266fbd56 csv.input (disk-index) - add support for memory mapped files 2015-09-29 10:35:36 +01:00
artemp
21e6936f85 first cut at using csv_index_featureset 2015-09-25 17:50:24 +01:00
artemp
3c56379273 CSV - initial disk-index featureset 2015-09-25 16:47:03 +01:00
artemp
8b4ff85b0d fix - pass value by reference 2015-09-25 15:26:02 +01:00
artemp
99617ad71c spatial_index<...> - use operator>> for reading Value's + update shape_index_featureset 2015-09-25 10:43:10 +01:00
artemp
2e0486440c remove shp_index.hpp and update shape.input 2015-09-24 11:37:27 +01:00
artemp
c362702e8b remove unused erroneous type def 2015-09-18 09:42:28 +01:00
artemp
337525ef60 csv.input - don't attempt to build if boost < 1.56 2015-09-11 11:20:45 +02:00
artemp
8128a2b1ef topojson - remove 'invalid' geometry type to avoid default constructed geometries 2015-09-10 19:28:15 +02:00
artemp
5e0050d005 topojson - use separate local boolean to track first geometry 2015-09-10 15:35:25 +02:00
artemp
fc100be61e topojson : fix geometry index logic 2015-09-10 12:37:38 +02:00
artemp
826882b150 enable sorting + small refactor 2015-09-09 12:14:15 +02:00
artemp
0de6d36000 use memory mapped file by default for both index and data parsing (improves loading times by ~50%) - default on non-windows platforms 2015-09-09 11:56:21 +02:00
artemp
310dc968ea fix potential type mismatch 2015-09-09 11:08:00 +02:00
artemp
284ca099ea set input state to std::ios::failbit (only needed on linux ?) 2015-08-27 13:41:02 +02:00
artemp
8f37b30978 fix handling 'one line' + 'manual headers' + 'no newline' case 
update get_geometry_type  ( ref https://github.com/mapnik/mapnik/issues/3047)
 2015-08-27 13:07:51 +02:00
artemp
4f5bdf82c1 slightly better syntax (NOTE: 4000 seems arbitrary here) 2015-08-27 13:07:51 +02:00
artemp
3753d50b75 CSV - revive row_limit parameter + fix stderr 2015-08-25 15:05:04 +02:00
artemp
8709fb6f7c CSV - optimise parsing by providing num_columns hint 2015-08-24 16:35:32 +02:00
artemp
6c9257a915 add parse_line accepting iterator range and avoid string ctor 2015-08-24 15:41:04 +02:00
artemp
c9d1d51b8a simplify + factor out properties parsing logic 2015-08-24 14:13:13 +02:00
artemp
2477d8764e keep on untangling spaghetti 
* implement standalone ignore case equality to avoid copying
* fix various logic shortcommings
 2015-08-24 12:23:59 +02:00
artemp
5dead08ecc CSV - remove unused params and member vars 2015-08-24 09:30:57 +02:00
artemp
4babec802a CSV - implement spatial index access to features on disk + preserve support for inline data (work-in-progress) 2015-08-21 13:52:42 +02:00
artemp
4943cb4cf8 remove unused includes 2015-08-20 14:15:35 +02:00
artemp
93fcc0a783 cleanup 2015-08-19 15:24:38 +02:00
artemp
6c3d9bb2a2 CSV plug-in - refactor and bring some sanity, sigh .. 2015-08-19 12:04:56 +02:00
artemp
318a8217a7 work-in-progress 2015-08-17 15:27:17 +02:00
artemp
f7d1cf82a9 Merge branch 'master' into large_csv 2015-08-13 13:28:35 +02:00
Blake Thompson
161469ed63 Fixed an issue with fields over size of int32 in OGR plugin and added tests to cover this situation. 2015-08-05 13:47:17 -05:00
artemp
f9bd21eeb5 Merge branch 'svg-parser-errors' into large_csv 2015-07-31 11:18:55 +02:00
Dane Springmeyer
a43faf9017 Merge pull request #2961 from CartoDB/3.0.x.pgraster 
Render NODATA as transparent and clip highbit data
 2015-07-16 16:14:28 -07:00
Dane Springmeyer
2048be7b30 move occi, osm, rasterlite to non-core repo 84b0bfecb3 - refs #2980 2015-07-16 11:46:10 -07:00
Paul Ramsey
d144f97f4e Render NODATA as transparent and clip highbit data 
ala GDAL for greyscale outputs. Addresses #2661
 2015-07-08 08:29:53 -07:00
artemp
56678b8aad geojson.input - fix bug typo 2015-07-06 12:47:45 +02:00
artemp
9c7186e49e add `autodetect_newline' as a separate function 2015-06-19 13:30:00 +02:00
artemp
d893718343 Merge branch 'master' into large_csv 2015-06-18 13:06:28 +02:00
Dane Springmeyer
4fe3c87a84 tame a few more boost -Wsign-conversion warnings 2015-06-17 13:29:54 -07:00
artemp
8c6bf0eef6 update copyright notice 2015-06-16 12:49:16 +02:00
artemp
c428779e83 shape : make primitives parsers static + fix numeric parser to handle mapnik::value_integer (64-bit) 2015-06-16 12:01:11 +02:00
artemp
8bf82b717e OGR : support reading OFTInteger64/OFTInteger64List 2015-06-16 11:36:17 +02:00
Dane Springmeyer
b2c85e0c65 fix a slew of -Wsign-conversion, -Wconversion, -Wunused-argument warnings 2015-06-15 20:41:51 -07:00
Dane Springmeyer
0d86afff36 start using -Wsign-compare and -Wshadow + fix heaps of warnings - refs #2907 2015-06-12 14:51:18 -07:00
artemp
b228da7bc3 remove unused grammar + remove stderr 2015-06-11 10:27:32 +01:00
artemp
1a95f1753e remove dead code 2015-06-10 16:32:05 +01:00
artemp
40b963f9ad csv_grammar - handle various quotting options + disable csv_utils::fix_json_quoting(csv_line) 2015-06-10 13:40:55 +01:00
artemp
cb832c0964 format 2015-06-10 11:41:28 +01:00
artemp
989af2ea42 use csv_grammar to parse csv lines 2015-06-09 15:22:37 +01:00
artemp
35ff68a7ec refactor - detect_separator free func 2015-06-09 10:17:55 +01:00
artemp
f208717070 fix spelling 2015-06-04 17:21:08 +01:00
artemp
e0e46eb3a8 csv plugin : add file_length standalone helper, make file_length local variable 2015-06-04 17:21:08 +01:00
artemp
42e1adca84 c++11 - use std::bind 2015-06-03 14:00:14 +01:00
artemp
8fa314c418 follow up from be731f17ce - fix read_polyline (-fsanitize=address) 2015-06-02 17:50:10 +01:00
artemp
df4226a825 remove spurious std::move's (clang - moving a temporary object prevents copy elision [-Wpessimizing-move]) 2015-06-02 14:26:35 +01:00