sij/mapnik
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
12174 commits 112 branches 95 tags 181 MiB
Commit graph

1626 commits

Author SHA1 Message Date
Mickey Rose
c21778fdfc merge mapnik-index::process_csv_file and csv_datasource::parse_csv 
- the function in plugin was already configurable via flags, and only
  contained two un-conditioned blocks that process_csv_file didn't have

- so I extracted the common parts into a separate function (in a class
  holding the flags and state), process_csv_file calls it with default
  flags, plugin sets them from params

- removed namespace ::detail, moving stuff that was used outside
  csv_utils to ::csv_utils, and the rest to ::csv_utils::detail
 2016-02-26 20:28:42 +01:00
artemp
f44b5ccfd9 always query upto num_features_to_query to determine data schema (currently 5) across all access modes (ref #3237 #3238) 2016-02-25 15:29:32 +01:00
Dane Springmeyer
08d7bf2152 no need for template usage for autodect_csv_flavour 2016-02-24 22:24:24 -08:00
Dane Springmeyer
f2782d0b67 [csv] move spirit usage to cpp 2016-02-24 17:42:51 -08:00
Dane Springmeyer
8ffee05048 Merge branch 'master' of github.com:mapnik/mapnik into faster-csv-compile 2016-02-24 16:42:23 -08:00
artemp
9fe049d8af CSV utils - strengthen quote detection logic + tests 2016-02-19 15:05:15 +01:00
artemp
6a9240f3d4 Revert "shape.input - add 'update-cache' optional parameter (default:true)" -- no much use as OS seems to apply it's own caching, doh .. 
This reverts commit f65c799475.
 2016-02-18 17:37:46 +01:00
artemp
f65c799475 shape.input - add 'update-cache' optional parameter (default:true) 2016-02-18 17:34:03 +01:00
artemp
b43ee6dd04 beef up has_index method to check if *.index file is valid (has expected header) (ref #3300) 2016-02-18 11:45:58 +01:00
artemp
cd5105785a csv_utils - fix istringstream initialiser by using explicit iterators pair std::string ctor 2016-02-18 11:35:14 +01:00
artemp
4c0781dbe9 remove redundant unique_ptr usage for local variable 2016-02-18 10:53:12 +01:00
artemp
15f905b272 csv - improved quote auto-detection logic which should catch more corner cases 2016-02-16 16:05:12 +01:00
Dane Springmeyer
85d4596769 move getline impl to separate header to avoid as much spirit inclusion 2016-02-15 19:06:11 -08:00
Mickey Rose
5d1fd56cef csv_grammar: make csv_white_space_skipper a primitive_parser 2016-02-15 15:17:26 +01:00
artemp
ca0c0e5888 csv_util - combine auto-decting separator/quote/newline into one method std::tuple<char, bool, char, char> autodect_csv_flavour(T & stream, std::size_t file_length) 2016-02-15 13:55:06 +01:00
Dane Springmeyer
e76a1d9a81 [csv] use faster version of csv_utils::parse_line in more places + fix error output 2016-02-01 12:23:26 -08:00
Dane Springmeyer
0b22bd87f7 shape_index_featureset: initialize members explicitly - remove debug print 2016-01-29 16:29:43 -08:00
Dane Springmeyer
32bbccb8fd fix #3276 2016-01-29 16:29:12 -08:00
Blake Thompson
b46f87b385 Merge branch 'master' into sanitize_errors 
Conflicts:
	include/mapnik/json/positions_grammar.hpp
 2016-01-07 08:34:52 -06:00
artemp
4c6632cf93 shapeindex : remove duplicate node defs + add default ctor 2016-01-04 13:19:16 +00:00
artemp
88a04a120f remove stderr 2016-01-04 11:56:01 +00:00
artemp
a2b3130d4b rebase to master (remove spirit-x3 changes) 2016-01-04 11:43:27 +00:00
Blake Thompson
dfa62c88d8 fix for santize address errors 2015-12-27 21:40:10 -06:00
Dane Springmeyer
8e6e74dfa2 suppress -Wshadow warning from ogr 2015-12-03 10:22:02 -05:00
Artem Pavlenko
1b6ca8bcd0 Merge pull request #3176 from mapycz/fix-warn 
csv.input: fix warning
 2015-11-23 10:46:10 +00:00
artemp
aee0149a1a GeoJSON - preserve feature ordering with both cache_features=true' and cache_features=false` (#3182) 2015-11-23 10:19:49 +00:00
Jiri Drbalek
7283b758a0 csv.input: fix warning 2015-11-23 08:54:13 +00:00
artemp
58998f8126 cleanup - remove unused vars 2015-11-20 17:45:10 +00:00
artemp
1eab1a3a28 CSV - change 'quote' auto-dection logic to handle mixed cases better 2015-11-20 17:10:55 +00:00
artemp
938702dc8b geojson.input - rely on expectation failure to select optional single Feature/Geometry parser 2015-11-18 14:43:40 +00:00
artemp
998164dc72 mapnik c++ style 2015-11-18 14:42:29 +00:00
Dane Springmeyer
7608040906 centralize warning suppression 2015-11-07 17:53:09 -08:00
Dane Springmeyer
da624b9ef4 update test-data to pull in 9f471a25e3 -refs #3160 2015-11-04 13:57:14 -08:00
artemp
133ca16d3a GeoJSON.input - skip empty geometries (#3156) 2015-10-29 16:54:35 +00:00
artemp
3de31c50ab correct exceptions text 2015-10-29 14:08:08 +00:00
Dane Springmeyer
35dccd4653 remove unused variable [skip ci] 2015-10-23 12:05:56 -07:00
artemp
158729ffa4 remove unreachable code block 2015-10-23 14:24:08 +01:00
artemp
aeaadd423d geojson.input - print line number 2015-10-22 18:32:23 +01:00
Dane Springmeyer
ca63ae0a96 csv: no need for newlines in exception strings 2015-10-22 10:22:58 -07:00
Dane Springmeyer
ab88fe48a4 csv: fixup error messages when geometry fails to parse 2015-10-22 10:10:23 -07:00
artemp
6effbedff3 csv.input/mapnik-index - validate geometry locator and throw early if invalid 2015-10-22 13:29:26 +01:00
Dane Springmeyer
a7e7f088a2 CSV plugin: trigger exception rather than crash when indexing error happens 2015-10-21 12:54:03 -07:00
artemp
c3ccff4f02 shape.input - add stream state check and calculate position limit 2015-10-21 16:33:12 +01:00
artemp
5cb2b8a77a geojson.input - create temp internal mapnik::feature_impl objects with id = -1 for clarity ref #3134 2015-10-21 11:14:13 +01:00
artemp
17ace8a3e8 shape.input - read shx_file_length in shape_featureset and avoid passing wrong file length by mistake ref #3136 2015-10-21 10:55:26 +01:00
artemp
19c1c0afd0 bug fix - shape_featureset now expects shx file length (!) ref #3136 2015-10-21 10:46:49 +01:00
artemp
ab2d86c617 shape.input: remove set_id() method as it's no longer required ref #1020 #1019 2015-10-20 14:18:17 +01:00
artemp
c952db979e use mapnik::value_integer for feature id's 2015-10-20 11:45:39 +01:00
artemp
c053682711 geojson.input - assign incremental feature_id in geojson_index_featureset and geojson_memory_index_featureset 
(ref #3134)
 2015-10-20 11:12:11 +01:00
artemp
26bf021af9 rename large_geojson_featureset -> geojson_memory_index_featurest to better reflect undelying implementations: geojson_index_featureset is most capable in terms of handling large input data with minimal memory requrements via utilising on disk spatial index (createed with mapnik-index utility) 2015-10-19 10:54:52 +01:00
Dane Springmeyer
8813e73cfc rename SHAPE_MEMORY_MAPPED_FILE to MAPNIK_MEMORY_MAPPED_FILE 2015-10-16 13:34:53 -07:00
artemp
4733c7ffc1 geojson.input - geometry_type for disk-index 2015-10-16 17:31:36 +01:00
artemp
942fb3c562 geojson.input - implement spatial disk index featureset 2015-10-16 13:15:20 +01:00
artemp
886ac23c64 geojson.input - force caching features and parse single feature/geometry when cache_features = false but extract_bounding_box fails (input is not FeatureCollection) 2015-10-15 17:35:46 +01:00
artemp
2906706a40 geojson.input - split parsing FeatureCollection and single Feature/Geometry into separate grammars 2015-10-15 15:12:16 +01:00
artemp
b798503623 shape.input - pass shx_file_length to shape_featureset 2015-10-15 11:49:18 +01:00
artemp
a73e8537d8 shape.input - add support for reading offset/record_length from *.shx in shape_featureset (no *.index) - ref #3126 2015-10-14 16:44:43 +01:00
artemp
8562d18cd0 geoson.input - throw an exception if input hasn't been fully consumed 2015-10-14 11:03:07 +01:00
Dane Springmeyer
324f70652b code cleanup / c++11 usage in plugins 2015-10-09 16:07:19 -07:00
Dane Springmeyer
5e8a009278 fix #3120 2015-10-09 15:27:34 -07:00
artemp
0bca3afa4e formatting 2015-10-09 12:49:33 +01:00
artemp
faeae5bc2e c++11 - more concise syntax 2015-10-09 11:26:39 +01:00
artemp
b5fe23ac28 inline extract_geometry 2015-10-09 11:26:10 +01:00
Dane Springmeyer
f7b125c90f use boolean_type (no functional change) [skip ci] 2015-10-07 17:53:32 -07:00
Dane Springmeyer
3bfd997b83 fix -Wsign-compare warning [skip ci] 2015-10-07 14:43:33 -07:00
Dane Springmeyer
6e84bde468 Add key_field_as_attribute (default:true) to control if key_field is added as feature attribute - refs #3115 2015-10-07 12:59:45 -07:00
artemp
b275bcb2c2 update to use latest grammar + use std:: qualifier for 'C' lib functions 2015-10-07 12:34:20 +01:00
artemp
e727fb3877 csv.input - read first line of data to update descriptor (disk-index) + remove stderr 
§
 2015-10-06 16:39:21 +01:00
artemp
d0cf7ad6f4 csv.input - implement get_geometry_type_impl for disk based index 2015-10-06 11:32:21 +01:00
artemp
2c8ad910d7 csv.input - remove redundant trim_copy 2015-10-06 10:00:42 +01:00
artemp
63c73b5057 add initial 'qoute' auto-detection + restore csv_test's 2015-10-05 16:56:33 +01:00
artemp
5c6c8ff7a3 csv.input - restore handling of inline headers and only one line of data without new line 2015-10-05 15:28:57 +01:00
artemp
287028d3e4 fix rebase conflict 2015-10-05 15:25:17 +01:00
artemp
178e39e19a make separator single character and simplify/optimise csv_grammar 2015-10-05 15:23:20 +01:00
bergwerkgis
6b20c8c9f2 fixes #3106 2015-10-05 09:55:20 +00:00
artemp
cbb40d0662 raster.input - throw if reader can't be created 2015-10-02 12:47:57 +01:00
artemp
1bccca9ff8 Merge remote-tracking branch 'origin/master' 
Conflicts:
	plugins/input/csv/csv_datasource.cpp
 2015-10-02 12:27:28 +01:00
artemp
1d320b7133 make quote single char 2015-10-02 12:20:54 +01:00
artemp
98ea1c5cd9 csv_utils::getline_csv - add 'quote' argument 2015-10-02 12:17:38 +01:00
Dane Springmeyer
336170c13c avoid excessive calling of std::ios::widen - refs #3101 2015-10-01 15:34:00 -07:00
Dane Springmeyer
3932cc51b3 add getline benchmark - refs #3101 2015-10-01 14:47:55 -07:00
Dane Springmeyer
2d59fca9f9 ensure consistent, ordered linking for plugins - closes #3105 2015-10-01 13:35:56 -07:00
Dane Springmeyer
05ade151a5 suppress yet more -Wshadow warnings from boost 2015-10-01 13:35:23 -07:00
Dane Springmeyer
3acec350f6 avoid more -Wshadow warnings from boost 2015-10-01 13:35:23 -07:00
artemp
be437eb6b0 add experimental getline_csv implementation which handles newline characters inside single/double quoted strings 2015-10-01 18:33:32 +01:00
artemp
a4e15b5a47 remove stderr 2015-10-01 15:34:49 +01:00
artemp
4273e37278 Merge branch 'master' into csv-disk-index 2015-10-01 15:26:47 +01:00
Dane Springmeyer
d70725b218 fix variable shadowing in extract_bounding_box_grammar 2015-09-30 15:42:42 -07:00
Dane Springmeyer
d6b60188e8 avoid -Wshadow errors from boost when compiling csv plugin 2015-09-30 14:11:16 -07:00
artemp
7a956651fe geojson - avoid passing invalid bounding boxes to boost::geometry::index 2015-09-30 12:50:53 +01:00
artemp
19897a3085 correct exception handling in main processing loop 2015-09-30 12:37:36 +01:00
artemp
848098baeb spatial_index/quad_tree - remove `operator>>' requirement, instead value_type stored must have standard layout for correct (de)serialisation + update unit test 2015-09-29 15:44:57 +01:00
artemp
49266fbd56 csv.input (disk-index) - add support for memory mapped files 2015-09-29 10:35:36 +01:00
artemp
21e6936f85 first cut at using csv_index_featureset 2015-09-25 17:50:24 +01:00
artemp
3c56379273 CSV - initial disk-index featureset 2015-09-25 16:47:03 +01:00
artemp
8b4ff85b0d fix - pass value by reference 2015-09-25 15:26:02 +01:00
artemp
99617ad71c spatial_index<...> - use operator>> for reading Value's + update shape_index_featureset 2015-09-25 10:43:10 +01:00
artemp
2e0486440c remove shp_index.hpp and update shape.input 2015-09-24 11:37:27 +01:00
artemp
c362702e8b remove unused erroneous type def 2015-09-18 09:42:28 +01:00
artemp
337525ef60 csv.input - don't attempt to build if boost < 1.56 2015-09-11 11:20:45 +02:00