Merge branch 'master' of github.com:mapnik/mapnik into faster-csv-compile

CHANGELOG.md

@@ -6,6 +6,37 @@ Developers: Please commit along with changes.
 
 For a complete change history, see the git log.
 
+## 3.0.10
+
+Released: February 25, 2016
+
+(Packaged from 5c0d496)
+
+#### Summary
+
+ - The `shapeindex` command now has a `--index-parts` option. When used the index will be bigger
+   but will allow the Shapefile datasource to only parse polygon parts within the query bounds.
+ - WARNING: index files generated with this newer Mapnik are invalid for older versions of Mapnik.
+ - Any `.index` files accompanying a `.shp` must now be regenerated otherwise
+   it will be skipped. To avoid this problem you can delete the existing `.index` files, or ideally run `shapeindex` to recreate the `.index`. (https://github.com/mapnik/mapnik/pull/3300)
+   The trigger for this change was an optimization that required a new binary format for the shapefile indexes (https://github.com/mapnik/mapnik/pull/3217).
+ - Shapeindex - another fix for skipping `null` shapes (#3288)
+ - Fixed support for filter expressions starting with `not` (https://github.com/mapnik/mapnik/issues/3017)
+ - Ensure `mapped_memory_cache` acts as singleton across shared objects (#3306)
+ - Removed miniz support in PNG encoder (#3281)
+ - Added `-fvisibility=hidden -fvisibility-inlines-hidden` to default compiler flags
+ - Fixed parsing of SVG `PathElement` (https://github.com/mapnik/mapnik/issues/3225)
+ - JSON parsing now supports arbitrary (nested) attributes in `geometry`
+ - Support for rendering `dash-array` in SVGs
+ - SVG parser is now stricter (fails is all input is not parsable) (#3251)
+ - SVG parser now correctly handles optional separator `(,)` between multiple command parts
+ - Optimized parsing of `png` format string
+ - The `memory_datasource` now dynamically reports correct datasource type (vector or raster)
+ - Upgraded `mapbox::variant v1.1.0`
+ - Compare: https://github.com/mapnik/mapnik/compare/v3.0.9...v3.0.10
+
+
+
 ## 3.0.9
 
 Released: November 26, 2015
@@ -18,12 +49,12 @@ Released: November 26, 2015
  - Fixed mapnik.util.variant issue when compiling with gcc-5.x and SSO enabled by default (https://github.com/mapnik/mapnik/issues/3103) (via @nkovacs)
  - Fixed issue with complex scripts where some character sequences weren't rendered correctly (https://github.com/mapnik/mapnik/issues/3050) (via @jkroll20)
  - Revived postgis.input tests
- - JSON: geometry grammar has been refactored and optimized to have expectation points
+ - JSON: geometry grammar has been re-factored and optimized to have expectation points
  - Filled missing specializations for value_bool in `mapnik::value` comparison operators
  - `mapnik.Image` - fixed copy semantics implementation for internal buffer
  - JSON parsing: unified error_handler across all grammars
  - Improved unit test coverage
- - Raster scaling: fixed nodata handling, acurracy when working with small floats and clipping floats by \[0; 255\] (https://github.com/mapnik/mapnik/pull/3147)
+ - Raster scaling: fixed nodata handling, accuracy when working with small floats and clipping floats by \[0; 255\] (https://github.com/mapnik/mapnik/pull/3147)
  - Added [`code of conduct`](http://contributor-covenant.org)
  - GeoJSON plug-in is updated to skip feature with empty geometries
  - GeoJSON plug-in : ensure original order of features is preserved (fixed) (https://github.com/mapnik/mapnik/issues/3182)
@@ -41,9 +72,9 @@ Released: October 23, 2015
 
  - Renamed `SHAPE_MEMORY_MAPPED_FILE` define to `MAPNIK_MEMORY_MAPPED_FILE`. Pass `./configure MEMORY_MAPPED_FILE=True|False` to request
    support for memory mapped files across Mapnik plugins (currently shape, csv, and geojson).
- - Unified `mapnik-index` utility supporing GeoJSON and CSV formats
+ - Unified `mapnik-index` utility supporting GeoJSON and CSV formats
  - Increased unit test coverage for GeoJSON and CSV plugins
- - shape.input - refactor to support *.shx and improve handling various bogus shapefiles
+ - shape.input - re-factor to support *.shx and improve handling various bogus shapefiles
  - geojson.input - make JSON parser stricter + support single Feature/Geometry as well as FeatureCollection
  - maintain 'FT_LOAD_NO_HINTING' + support >= harfbuzz 1.0.5
  - geojson.input - implement on-disk-index support
@@ -109,7 +140,7 @@ Released: August 26, 2015
 
 #### Summary
 
-- CSV.input: plug-in has been refactored to minimise memory usage and to improve handling of larger input.
+- CSV.input: plug-in has been re-factored to minimise memory usage and to improve handling of larger input.
   (NOTE: [large_csv](https://github.com/mapnik/mapnik/tree/large_csv) branch adds experimental trunsduction parser with deferred string initialisation)
 - CSV.input: added internal spatial index (boost::geometry::index::tree) for fast `bounding box` queries (https://github.com/mapnik/mapnik/pull/3010)
 - Fixed deadlock in recursive datasource registration via @zerebubuth (https://github.com/mapnik/mapnik/pull/3038)
@@ -1200,7 +1231,7 @@ Released April 1, 2009
 
 - Plugins: Use memory mapped files for reading shape file (r628)
 
-- Core: Use streams to write images (i/o refactor) (r628) (#15)
+- Core: Use streams to write images (i/o re-factor) (r628) (#15)
 
 # Mapnik 0.5.1
 

Makefile

@@ -24,7 +24,8 @@ release:
 	git clone --depth 1 --branch v$${MAPNIK_VERSION} git@github.com:mapnik/mapnik.git $${TARBALL_NAME} && \
 	cd $${TARBALL_NAME} && \
 	git checkout "tags/v$${MAPNIK_VERSION}" && \
-	git submodule update --depth 1 --init && \
+	git submodule update --depth 100 --init && \
+	rm -rf deps/mapbox/variant/.git && \
 	rm -rf test/data/.git && \
 	rm -rf test/data/.gitignore && \
 	rm -rf test/data-visual/.git && \

appveyor.yml

@@ -12,7 +12,13 @@ os: Visual Studio 2015
 # limit clone to latest 5 commits
 clone_depth: 5
 
+services:
+  - postgresql94 #if changing this, also change PATH below
+
 install:
+  - SET PGUSER=postgres
+  - SET PGPASSWORD=Password12!
+  - SET PATH=C:\Program Files\PostgreSQL\9.4\bin\;%PATH%
   - scripts\build-appveyor.bat
 
 artifacts:

benchmark/bench_framework.hpp

@@ -10,6 +10,7 @@
 
 // stl
 #include <chrono>
+#include <cmath> // log10, round
 #include <cstdio> // snprintf
 #include <iostream>
 #include <set>
@@ -19,6 +20,12 @@
 
 namespace benchmark {
 
+template <typename T>
+using milliseconds = std::chrono::duration<T, std::milli>;
+
+template <typename T>
+using seconds = std::chrono::duration<T>;
+
 class test_case
 {
 protected:
@@ -92,7 +99,7 @@ inline int parse_args(int argc, char** argv, mapnik::parameters & params)
 
 inline void handle_common_args(mapnik::parameters const& params)
 {
-    if (auto severity = params.get<std::string>("log-severity")) {
+    if (auto severity = params.get<std::string>("log")) {
         if (*severity == "debug")
             mapnik::logger::set_severity(mapnik::logger::debug);
         else if (*severity == "warn")
@@ -102,7 +109,7 @@ inline void handle_common_args(mapnik::parameters const& params)
         else if (*severity == "none")
             mapnik::logger::set_severity(mapnik::logger::none);
         else
-            std::clog << "ignoring option --log-severity='" << *severity
+            std::clog << "ignoring option --log='" << *severity
                       << "' (allowed values are: debug, warn, error, none)\n";
     }
 }
@@ -134,6 +141,29 @@ inline int handle_args(int argc, char** argv, mapnik::parameters & params)
         }                                               \
     }                                                   \
 
+struct big_number_fmt
+{
+    int w;
+    double v;
+    const char* u;
+
+    big_number_fmt(int width, double value, int base = 1000)
+        : w(width), v(value), u("")
+    {
+        static const char* suffixes = "\0\0k\0M\0G\0T\0P\0E\0Z\0Y\0\0";
+        u = suffixes;
+
+        while (v > 1 && std::log10(std::round(v)) >= width && u[2])
+        {
+            v /= base;
+            u += 2;
+        }
+
+        // adjust width for proper alignment without suffix
+        w += (u == suffixes);
+    }
+};
+
 template <typename T>
 int run(T const& test_runner, std::string const& name)
 {
@@ -156,21 +186,43 @@ int run(T const& test_runner, std::string const& name)
         auto opt_min_duration = test_runner.params().template get<double>("min-duration", 0.0);
         std::chrono::duration<double> min_seconds(*opt_min_duration);
         auto min_duration = std::chrono::duration_cast<decltype(elapsed)>(min_seconds);
-        std::size_t loops = 0;
+        auto num_iters = test_runner.iterations();
+        auto num_threads = test_runner.threads();
+        auto total_iters = 0;
 
-        if (test_runner.threads() > 0)
+        if (num_threads > 0)
         {
-            using thread_group = std::vector<std::unique_ptr<std::thread> >;
+            std::mutex mtx_ready;
-            using value_type = thread_group::value_type;
+            std::unique_lock<std::mutex> lock_ready(mtx_ready);
-            thread_group tg;
+
-            for (std::size_t i=0;i<test_runner.threads();++i)
+            auto stub = [&](T const& test_copy)
             {
-                tg.emplace_back(new std::thread(test_runner));
+                // workers will wait on this mutex until the main thread
+                // constructs all of them and starts measuring time
+                std::unique_lock<std::mutex> my_lock(mtx_ready);
+                my_lock.unlock();
+                test_copy();
+            };
+
+            std::vector<std::thread> tg;
+            tg.reserve(num_threads);
+            for (auto i = num_threads; i-- > 0; )
+            {
+                tg.emplace_back(stub, test_runner);
             }
             start = std::chrono::high_resolution_clock::now();
-            std::for_each(tg.begin(), tg.end(), [](value_type & t) {if (t->joinable()) t->join();});
+            lock_ready.unlock();
+            // wait for all workers to finish
+            for (auto & t : tg)
+            {
+                if (t.joinable())
+                    t.join();
+            }
             elapsed = std::chrono::high_resolution_clock::now() - start;
-            loops = 1;
+            // this is actually per-thread count, not total, but I think
+            // reporting average 'iters/thread/second' is more useful
+            // than 'iters/second' multiplied by the number of threads
+            total_iters += num_iters;
         }
         else
         {
@@ -178,39 +230,25 @@ int run(T const& test_runner, std::string const& name)
             do {
                 test_runner();
                 elapsed = std::chrono::high_resolution_clock::now() - start;
-                ++loops;
+                total_iters += num_iters;
             } while (elapsed < min_duration);
         }
 
-        double iters = loops * test_runner.iterations();
-        double dur_total = std::chrono::duration<double, std::milli>(elapsed).count();
-        double dur_avg = dur_total / iters;
-        char iters_unit = ' ';
         char msg[200];
-
+        double dur_total = milliseconds<double>(elapsed).count();
-        if (iters >= 1e7) iters *= 1e-6, iters_unit = 'M';
+        auto elapsed_nonzero = std::max(elapsed, decltype(elapsed){1});
-        else if (iters >= 1e4) iters *= 1e-3, iters_unit = 'k';
+        big_number_fmt itersf(4, total_iters);
+        big_number_fmt ips(5, total_iters / seconds<double>(elapsed_nonzero).count());
 
         std::snprintf(msg, sizeof(msg),
-                "%-43s %3zu threads %4.0f%c iters %6.0f milliseconds",
+                "%-43s %3zu threads %*.0f%s iters %6.0f milliseconds %*.0f%s i/s\n",
                 name.c_str(),
-                test_runner.threads(),
+                num_threads,
-                iters, iters_unit,
+                itersf.w, itersf.v, itersf.u,
-                dur_total);
+                dur_total,
+                ips.w, ips.v, ips.u
+                );
         std::clog << msg;
-
-        // log average time per iteration, currently only for non-threaded runs
-        if (test_runner.threads() == 0)
-        {
-            char unit = 'm';
-            if (dur_avg < 1e-5) dur_avg *= 1e+9, unit = 'p';
-            else if (dur_avg < 1e-2) dur_avg *= 1e+6, unit = 'n';
-            else if (dur_avg < 1e+1) dur_avg *= 1e+3, unit = 'u';
-            std::snprintf(msg, sizeof(msg), " %4.0f%cs/iter", dur_avg, unit);
-            std::clog << msg;
-        }
-
-        std::clog << "\n";
         return 0;
     }
     catch (std::exception const& ex)

benchmark/run

@@ -6,8 +6,14 @@ source ./localize.sh
 
 BASE=./benchmark/out
 function run {
-    ${BASE}/$1 --threads 0 --iterations $3;
+    local runner="$BASE/$1 --log=none"
-    ${BASE}/$1 --threads $2 --iterations $(expr $3 / $2);
+    local threads="$2"
+    local iters="$3"
+    shift 3
+    $runner --threads 0 --iterations $iters "$@"
+    if test $threads -gt 0; then
+        $runner --threads $threads --iterations $((iters/threads)) "$@"
+    fi
 }
 run test_getline 30 10000000
 #run test_array_allocation 20 100000

include/mapnik/csv/csv_grammar.hpp

@@ -86,7 +86,7 @@ struct csv_line_grammar : qi::grammar<Iterator, csv_line(char, char), Skipper>
             ("\\\"", '\"')
             ("\"\"", '\"') // double quote
             ;
-        line = -lit("\n\r") >> column(_r1, _r2) % lit(_r1)
+        line = -lit("\r") >> -lit("\n") >> column(_r1, _r2) % lit(_r1)
             ;
         column = quoted(_r2) | *(char_ - lit(_r1))
             ;

include/mapnik/expression_node.hpp

@@ -31,6 +31,8 @@
 #include <mapnik/attribute.hpp>
 #include <mapnik/function_call.hpp>
 #include <mapnik/expression_node_types.hpp>
+// stl
+#include <memory>
 
 namespace mapnik
 {

include/mapnik/marker_helpers.hpp

@@ -36,7 +36,7 @@
 #include <mapnik/vertex_processor.hpp>
 #include <mapnik/renderer_common/apply_vertex_converter.hpp>
 #include <mapnik/renderer_common/render_markers_symbolizer.hpp>
-
+#include <mapnik/vertex_converters.hpp>
 // agg
 #include "agg_trans_affine.h"
 
@@ -158,84 +158,29 @@ void setup_transform_scaling(agg::trans_affine & tr,
                              attributes const& vars,
                              symbolizer_base const& sym);
 
+using vertex_converter_type = vertex_converter<clip_line_tag,
+                                               clip_poly_tag,
+                                               transform_tag,
+                                               affine_transform_tag,
+                                               simplify_tag,
+                                               smooth_tag,
+                                               offset_transform_tag>;
+
 // Apply markers to a feature with multiple geometries
-template <typename Converter, typename Processor>
+template <typename Processor>
-void apply_markers_multi(feature_impl const& feature, attributes const& vars, Converter & converter, Processor & proc, symbolizer_base const& sym)
+void apply_markers_multi(feature_impl const& feature, attributes const& vars,
-{
+                         vertex_converter_type & converter, Processor & proc, symbolizer_base const& sym);
-    using apply_vertex_converter_type = detail::apply_vertex_converter<Converter,Processor>;
-    using vertex_processor_type = geometry::vertex_processor<apply_vertex_converter_type>;
 
-    auto const& geom = feature.get_geometry();
-    geometry::geometry_types type = geometry::geometry_type(geom);
 
-    if (type == geometry::geometry_types::Point
+using vector_dispatch_type = vector_markers_dispatch<mapnik::label_collision_detector4&>;
-        || type == geometry::geometry_types::LineString
+using raster_dispatch_type = raster_markers_dispatch<mapnik::label_collision_detector4&>;
-        || type == geometry::geometry_types::Polygon)
-    {
-        apply_vertex_converter_type apply(converter, proc);
-        mapnik::util::apply_visitor(vertex_processor_type(apply), geom);
-    }
-    else
-    {
 
-        marker_multi_policy_enum multi_policy = get<marker_multi_policy_enum, keys::markers_multipolicy>(sym, feature, vars);
+extern template void apply_markers_multi<vector_dispatch_type>(feature_impl const& feature, attributes const& vars,
-        marker_placement_enum placement = get<marker_placement_enum, keys::markers_placement_type>(sym, feature, vars);
+                         vertex_converter_type & converter, vector_dispatch_type & proc, symbolizer_base const& sym);
+
+extern template void apply_markers_multi<raster_dispatch_type>(feature_impl const& feature, attributes const& vars,
+                         vertex_converter_type & converter, raster_dispatch_type & proc, symbolizer_base const& sym);
 
-        if (placement == MARKER_POINT_PLACEMENT &&
-            multi_policy == MARKER_WHOLE_MULTI)
-        {
-            geometry::point<double> pt;
-            // test if centroid is contained by bounding box
-            if (geometry::centroid(geom, pt) && converter.disp_.args_.bbox.contains(pt.x, pt.y))
-            {
-                // unset any clipping since we're now dealing with a point
-                converter.template unset<clip_poly_tag>();
-                geometry::point_vertex_adapter<double> va(pt);
-                converter.apply(va, proc);
-            }
-        }
-        else if ((placement == MARKER_POINT_PLACEMENT || placement == MARKER_INTERIOR_PLACEMENT) &&
-                 multi_policy == MARKER_LARGEST_MULTI)
-        {
-            // Only apply to path with largest envelope area
-            // TODO: consider using true area for polygon types
-            if (type == geometry::geometry_types::MultiPolygon)
-            {
-                geometry::multi_polygon<double> const& multi_poly = mapnik::util::get<geometry::multi_polygon<double> >(geom);
-                double maxarea = 0;
-                geometry::polygon<double> const* largest = 0;
-                for (geometry::polygon<double> const& poly : multi_poly)
-                {
-                    box2d<double> bbox = geometry::envelope(poly);
-                    double area = bbox.width() * bbox.height();
-                    if (area > maxarea)
-                    {
-                        maxarea = area;
-                        largest = &poly;
-                    }
-                }
-                if (largest)
-                {
-                    geometry::polygon_vertex_adapter<double> va(*largest);
-                    converter.apply(va, proc);
-                }
-            }
-            else
-            {
-                MAPNIK_LOG_WARN(marker_symbolizer) << "TODO: if you get here -> open an issue";
-            }
-        }
-        else
-        {
-            if (multi_policy != MARKER_EACH_MULTI && placement != MARKER_POINT_PLACEMENT)
-            {
-                MAPNIK_LOG_WARN(marker_symbolizer) << "marker_multi_policy != 'each' has no effect with marker_placement != 'point'";
-            }
-            apply_vertex_converter_type apply(converter, proc);
-            mapnik::util::apply_visitor(vertex_processor_type(apply), geom);
-        }
-    }
-}
 
 }
 

include/mapnik/markers_placement.hpp

@@ -29,7 +29,6 @@
 #include <mapnik/markers_placements/vertext_first.hpp>
 #include <mapnik/markers_placements/vertext_last.hpp>
 #include <mapnik/symbolizer_enumerations.hpp>
-#include <mapnik/util/variant.hpp>
 
 namespace mapnik
 {
@@ -38,70 +37,71 @@ template <typename Locator, typename Detector>
 class markers_placement_finder : util::noncopyable
 {
 public:
-    using markers_placement = util::variant<markers_point_placement<Locator, Detector>,
+    using basic_placement = markers_basic_placement<Locator, Detector>;
-                                            markers_line_placement<Locator, Detector>,
-                                            markers_interior_placement<Locator, Detector>,
-                                            markers_vertex_first_placement<Locator, Detector>,
-                                            markers_vertex_last_placement<Locator, Detector>>;
-
-    class get_point_visitor
-    {
-    public:
-        get_point_visitor(double &x, double &y, double &angle, bool ignore_placement)
-            : x_(x), y_(y), angle_(angle), ignore_placement_(ignore_placement)
-        {
-        }
-
-        template <typename T>
-        bool operator()(T &placement) const
-        {
-            return placement.get_point(x_, y_, angle_, ignore_placement_);
-        }
-
-    private:
-        double &x_, &y_, &angle_;
-        bool ignore_placement_;
-    };
 
     markers_placement_finder(marker_placement_e placement_type,
                              Locator &locator,
                              Detector &detector,
                              markers_placement_params const& params)
-        : placement_(create(placement_type, locator, detector, params))
+        : active_placement_(nullptr)
     {
+        switch (placement_type)
+        {
+        default:
+        case MARKER_POINT_PLACEMENT:
+            active_placement_ = construct(&point_, locator, detector, params);
+            break;
+        case MARKER_INTERIOR_PLACEMENT:
+            active_placement_ = construct(&interior_, locator, detector, params);
+            break;
+        case MARKER_LINE_PLACEMENT:
+            active_placement_ = construct(&line_, locator, detector, params);
+            break;
+        case MARKER_VERTEX_FIRST_PLACEMENT:
+            active_placement_ = construct(&vertex_first_, locator, detector, params);
+            break;
+        case MARKER_VERTEX_LAST_PLACEMENT:
+            active_placement_ = construct(&vertex_last_, locator, detector, params);
+            break;
+        }
+        // previously placement-type constructors (markers_*_placement)
+        // rewound the locator; reasons for rewinding here instead:
+        //  1) so that nobody is tempted to call now-virtual rewind()
+        //      in placement-type class constructors
+        //  2) it servers as a runtime check that the above switch isn't
+        //      missing cases and active_placement_ points to an object
+        active_placement_->rewind();
+    }
+
+    ~markers_placement_finder()
+    {
+        active_placement_->~basic_placement();
     }
 
     // Get next point where the marker should be placed. Returns true if a place is found, false if none is found.
     bool get_point(double &x, double &y, double &angle, bool ignore_placement)
     {
-        return util::apply_visitor(get_point_visitor(x, y, angle, ignore_placement), placement_);
+        return active_placement_->get_point(x, y, angle, ignore_placement);
     }
 
 private:
-    // Factory function for particular placement implementations.
+    basic_placement* active_placement_;
-    static markers_placement create(marker_placement_e placement_type,
-                                    Locator &locator,
-                                    Detector &detector,
-                                    markers_placement_params const& params)
-    {
-        switch (placement_type)
-        {
-        case MARKER_POINT_PLACEMENT:
-            return markers_point_placement<Locator, Detector>(locator,detector,params);
-        case MARKER_INTERIOR_PLACEMENT:
-            return markers_interior_placement<Locator, Detector>(locator,detector,params);
-        case MARKER_LINE_PLACEMENT:
-            return markers_line_placement<Locator, Detector>(locator,detector,params);
-        case MARKER_VERTEX_FIRST_PLACEMENT:
-            return markers_vertex_first_placement<Locator, Detector>(locator,detector,params);
-        case MARKER_VERTEX_LAST_PLACEMENT:
-            return markers_vertex_last_placement<Locator, Detector>(locator,detector,params);
-        default: // point
-            return markers_point_placement<Locator, Detector>(locator,detector,params);
-        }
-    }
 
-    markers_placement placement_;
+    union
+    {
+        markers_point_placement<Locator, Detector> point_;
+        markers_line_placement<Locator, Detector> line_;
+        markers_interior_placement<Locator, Detector> interior_;
+        markers_vertex_first_placement<Locator, Detector> vertex_first_;
+        markers_vertex_last_placement<Locator, Detector> vertex_last_;
+    };
+
+    template <typename T>
+    static T* construct(T* what, Locator & locator, Detector & detector,
+                        markers_placement_params const& params)
+    {
+        return new(what) T(locator, detector, params);
+    }
 };
 
 }

include/mapnik/markers_placements/basic.hpp

@@ -0,0 +1,153 @@
+/*****************************************************************************
+ *
+ * This file is part of Mapnik (c++ mapping toolkit)
+ *
+ * Copyright (C) 2016 Artem Pavlenko
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+ *
+ *****************************************************************************/
+
+#ifndef MAPNIK_MARKERS_PLACEMENTS_BASIC_HPP
+#define MAPNIK_MARKERS_PLACEMENTS_BASIC_HPP
+
+// mapnik
+#include <mapnik/box2d.hpp>
+#include <mapnik/symbolizer_enumerations.hpp>
+#include <mapnik/util/math.hpp>
+#include <mapnik/util/noncopyable.hpp>
+
+// agg
+#include "agg_basics.h"
+#include "agg_trans_affine.h"
+
+namespace mapnik {
+
+struct markers_placement_params
+{
+    box2d<double> size;
+    agg::trans_affine tr;
+    double spacing;
+    double max_error;
+    bool allow_overlap;
+    bool avoid_edges;
+    direction_enum direction;
+};
+
+template <typename Locator, typename Detector>
+class markers_basic_placement : util::noncopyable
+{
+public:
+    markers_basic_placement(Locator & locator, Detector & detector,
+                            markers_placement_params const& params)
+        : locator_(locator),
+          detector_(detector),
+          params_(params),
+          done_(false)
+    {
+        // no need to rewind locator here, markers_placement_finder
+        // does that after construction
+    }
+
+    markers_basic_placement(markers_basic_placement && ) = default;
+
+    virtual ~markers_basic_placement()
+    {
+        // empty but necessary
+    }
+
+    // Start again at first marker. Returns the same list of markers only works when they were NOT added to the detector.
+    virtual void rewind()
+    {
+        locator_.rewind(0);
+        done_ = false;
+    }
+
+    // Get next point where the marker should be placed. Returns true if a place is found, false if none is found.
+    virtual bool get_point(double &x, double &y, double &angle, bool ignore_placement) = 0;
+
+protected:
+    Locator & locator_;
+    Detector & detector_;
+    markers_placement_params const& params_;
+    bool done_;
+
+    // Rotates the size_ box and translates the position.
+    box2d<double> perform_transform(double angle, double dx, double dy) const
+    {
+        auto tr = params_.tr * agg::trans_affine_rotation(angle).translate(dx, dy);
+        return box2d<double>(params_.size, tr);
+    }
+
+    // Checks transformed box placement with collision detector.
+    // returns false if the box:
+    //  - a) isn't wholly inside extent and avoid_edges == true
+    //  - b) collides with something and allow_overlap == false
+    // otherwise returns true, and if ignore_placement == true,
+    //  also adds the box to collision detector
+    bool push_to_detector(double x, double y, double angle, bool ignore_placement)
+    {
+        auto box = perform_transform(angle, x, y);
+        if (params_.avoid_edges && !detector_.extent().contains(box))
+        {
+            return false;
+        }
+        if (!params_.allow_overlap && !detector_.has_placement(box))
+        {
+            return false;
+        }
+        if (!ignore_placement)
+        {
+            detector_.insert(box);
+        }
+        return true;
+    }
+
+    bool set_direction(double & angle) const
+    {
+        switch (params_.direction)
+        {
+            case DIRECTION_UP:
+                angle = 0;
+                return true;
+            case DIRECTION_DOWN:
+                angle = M_PI;
+                return true;
+            case DIRECTION_AUTO:
+                if (std::fabs(util::normalize_angle(angle)) > 0.5 * M_PI)
+                    angle += M_PI;
+                return true;
+            case DIRECTION_AUTO_DOWN:
+                if (std::fabs(util::normalize_angle(angle)) < 0.5 * M_PI)
+                    angle += M_PI;
+                return true;
+            case DIRECTION_LEFT:
+                angle += M_PI;
+                return true;
+            case DIRECTION_LEFT_ONLY:
+                angle += M_PI;
+                return std::fabs(util::normalize_angle(angle)) < 0.5 * M_PI;
+            case DIRECTION_RIGHT_ONLY:
+                return std::fabs(util::normalize_angle(angle)) < 0.5 * M_PI;
+            case DIRECTION_RIGHT:
+            default:
+                return true;
+        }
+    }
+};
+
+} // namespace mapnik
+
+#endif // MAPNIK_MARKERS_PLACEMENTS_BASIC_HPP

include/mapnik/markers_placements/interior.hpp

@@ -33,14 +33,8 @@ template <typename Locator, typename Detector>
 class markers_interior_placement : public markers_point_placement<Locator, Detector>
 {
 public:
-    markers_interior_placement(Locator &locator, Detector &detector, markers_placement_params const& params)
+    using point_placement = markers_point_placement<Locator, Detector>;
-        : markers_point_placement<Locator, Detector>(locator, detector, params)
+    using point_placement::point_placement;
-    {
-    }
-
-    markers_interior_placement(markers_interior_placement && rhs)
-        : markers_point_placement<Locator, Detector>(std::move(rhs))
-    {}
 
     bool get_point(double &x, double &y, double &angle, bool ignore_placement)
     {
@@ -51,7 +45,7 @@ public:
 
         if (this->locator_.type() == geometry::geometry_types::Point)
         {
-            return markers_point_placement<Locator, Detector>::get_point(x, y, angle, ignore_placement);
+            return point_placement::get_point(x, y, angle, ignore_placement);
         }
 
         if (this->locator_.type() == geometry::geometry_types::LineString)
@@ -73,20 +67,10 @@ public:
 
         angle = 0;
 
-        box2d<double> box = this->perform_transform(angle, x, y);
+        if (!this->push_to_detector(x, y, angle, ignore_placement))
-        if (this->params_.avoid_edges && !this->detector_.extent().contains(box))
         {
             return false;
         }
-        if (!this->params_.allow_overlap && !this->detector_.has_placement(box))
-        {
-            return false;
-        }
-
-        if (!ignore_placement)
-        {
-            this->detector_.insert(box);
-        }
 
         this->done_ = true;
         return true;

include/mapnik/markers_placements/line.hpp

@@ -35,29 +35,23 @@ template <typename Locator, typename Detector>
 class markers_line_placement : public markers_point_placement<Locator, Detector>
 {
 public:
-    markers_line_placement(Locator &locator, Detector &detector, markers_placement_params const& params)
+    using point_placement = markers_point_placement<Locator, Detector>;
-        : markers_point_placement<Locator, Detector>(locator, detector, params),
+    using point_placement::point_placement;
+
+    markers_line_placement(Locator & locator, Detector & detector,
+                           markers_placement_params const& params)
+        : point_placement(locator, detector, params),
             first_point_(true),
             spacing_(0.0),
             marker_width_((params.size * params.tr).width()),
             path_(locator)
     {
         spacing_ = params.spacing < 1 ? 100 : params.spacing;
-        rewind();
     }
 
-    markers_line_placement(markers_line_placement && rhs)
-        : markers_point_placement<Locator, Detector>(std::move(rhs)),
-          first_point_(std::move(rhs.first_point_)),
-          spacing_(std::move(rhs.spacing_)),
-          marker_width_(std::move(rhs.marker_width_)),
-          path_(std::move(rhs.path_))
-    {}
-
     void rewind()
     {
-        this->locator_.rewind(0);
+        point_placement::rewind();
-        this->done_ = false;
         first_point_ = true;
     }
 
@@ -70,7 +64,7 @@ public:
 
         if (this->locator_.type() == geometry::geometry_types::Point)
         {
-            return markers_point_placement<Locator, Detector>::get_point(x, y, angle, ignore_placement);
+            return point_placement::get_point(x, y, angle, ignore_placement);
         }
 
         double move = spacing_;
@@ -102,16 +96,10 @@ public:
                     {
                         continue;
                     }
-                    box2d<double> box = this->perform_transform(angle, x, y);
+                    if (!this->push_to_detector(x, y, angle, ignore_placement))
-                    if ((this->params_.avoid_edges && !this->detector_.extent().contains(box))
-                        || (!this->params_.allow_overlap && !this->detector_.has_placement(box)))
                     {
                         continue;
                     }
-                    if (!ignore_placement)
-                    {
-                        this->detector_.insert(box);
-                    }
                     return true;
                 }
             }

include/mapnik/markers_placements/point.hpp

@@ -23,148 +23,54 @@
 #ifndef MAPNIK_MARKERS_PLACEMENTS_POINT_HPP
 #define MAPNIK_MARKERS_PLACEMENTS_POINT_HPP
 
-#include <mapnik/box2d.hpp>
 #include <mapnik/geom_util.hpp>
 #include <mapnik/geometry_types.hpp>
-#include <mapnik/util/math.hpp>
+#include <mapnik/markers_placements/basic.hpp>
-#include <mapnik/label_collision_detector.hpp>
-#include <mapnik/symbolizer_enumerations.hpp>
-#include <mapnik/util/noncopyable.hpp>
-
-#include "agg_basics.h"
-#include "agg_trans_affine.h"
-
-#include <cmath>
 
 namespace mapnik {
 
-struct markers_placement_params
-{
-    box2d<double> size;
-    agg::trans_affine tr;
-    double spacing;
-    double max_error;
-    bool allow_overlap;
-    bool avoid_edges;
-    direction_enum direction;
-};
-
 template <typename Locator, typename Detector>
-class markers_point_placement : util::noncopyable
+class markers_point_placement : public markers_basic_placement<Locator, Detector>
 {
 public:
-    markers_point_placement(Locator &locator, Detector &detector, markers_placement_params const& params)
+    using basic_placement = markers_basic_placement<Locator, Detector>;
-        : locator_(locator),
+    using basic_placement::basic_placement;
-          detector_(detector),
-          params_(params),
-          done_(false)
-    {
-        rewind();
-    }
-
-    markers_point_placement(markers_point_placement && rhs)
-        : locator_(rhs.locator_),
-          detector_(rhs.detector_),
-          params_(rhs.params_),
-          done_(rhs.done_)
-    {}
-
-
-    // Start again at first marker. Returns the same list of markers only works when they were NOT added to the detector.
-    void rewind()
-    {
-        locator_.rewind(0);
-        done_ = false;
-    }
 
     // Get next point where the marker should be placed. Returns true if a place is found, false if none is found.
     bool get_point(double &x, double &y, double &angle, bool ignore_placement)
     {
-        if (done_)
+        if (this->done_)
         {
             return false;
         }
 
-        if (locator_.type() == geometry::geometry_types::LineString)
+        if (this->locator_.type() == geometry::geometry_types::LineString)
         {
-            if (!label::middle_point(locator_, x, y))
+            if (!label::middle_point(this->locator_, x, y))
             {
-                done_ = true;
+                this->done_ = true;
                 return false;
             }
         }
         else
         {
-            if (!label::centroid(locator_, x, y))
+            if (!label::centroid(this->locator_, x, y))
             {
-                done_ = true;
+                this->done_ = true;
                 return false;
             }
         }
 
         angle = 0;
-        box2d<double> box = perform_transform(angle, x, y);
 
-        if (params_.avoid_edges && !detector_.extent().contains(box))
+        if (!this->push_to_detector(x, y, angle, ignore_placement))
-        {
-            return false;
-        }
-        if (!params_.allow_overlap && !detector_.has_placement(box))
         {
             return false;
         }
 
-        if (!ignore_placement)
+        this->done_ = true;
-        {
-            detector_.insert(box);
-        }
-
-        done_ = true;
         return true;
     }
-
-protected:
-    Locator &locator_;
-    Detector &detector_;
-    markers_placement_params const& params_;
-    bool done_;
-
-    // Rotates the size_ box and translates the position.
-    box2d<double> perform_transform(double angle, double dx, double dy)
-    {
-        agg::trans_affine tr = params_.tr * agg::trans_affine_rotation(angle).translate(dx, dy);
-        return box2d<double>(params_.size, tr);
-    }
-
-    bool set_direction(double & angle)
-    {
-        switch (params_.direction)
-        {
-            case DIRECTION_UP:
-                angle = .0;
-                return true;
-            case DIRECTION_DOWN:
-                angle = M_PI;
-                return true;
-            case DIRECTION_AUTO:
-                angle = (std::fabs(util::normalize_angle(angle)) > 0.5 * M_PI) ? (angle + M_PI) : angle;
-                return true;
-            case DIRECTION_AUTO_DOWN:
-                angle = (std::fabs(util::normalize_angle(angle)) < 0.5 * M_PI) ? (angle + M_PI) : angle;
-                return true;
-            case DIRECTION_LEFT:
-                angle += M_PI;
-                return true;
-            case DIRECTION_LEFT_ONLY:
-                angle += M_PI;
-                return std::fabs(util::normalize_angle(angle)) < 0.5 * M_PI;
-            case DIRECTION_RIGHT_ONLY:
-                return std::fabs(util::normalize_angle(angle)) < 0.5 * M_PI;
-            case DIRECTION_RIGHT:
-            default:
-                return true;
-        }
-    }
 };
 
 }

include/mapnik/markers_placements/vertext_first.hpp

@@ -31,14 +31,8 @@ template <typename Locator, typename Detector>
 class markers_vertex_first_placement : public markers_point_placement<Locator, Detector>
 {
 public:
-    markers_vertex_first_placement(Locator &locator, Detector &detector, markers_placement_params const& params)
+    using point_placement = markers_point_placement<Locator, Detector>;
-        : markers_point_placement<Locator, Detector>(locator, detector, params)
+    using point_placement::point_placement;
-    {
-    }
-
-    markers_vertex_first_placement(markers_vertex_first_placement && rhs)
-        : markers_point_placement<Locator, Detector>(std::move(rhs))
-    {}
 
     bool get_point(double &x, double &y, double &angle, bool ignore_placement)
     {
@@ -49,7 +43,7 @@ public:
 
         if (this->locator_.type() == mapnik::geometry::geometry_types::Point)
         {
-            return markers_point_placement<Locator, Detector>::get_point(x, y, angle, ignore_placement);
+            return point_placement::get_point(x, y, angle, ignore_placement);
         }
 
         double x0, y0;
@@ -75,20 +69,10 @@ public:
             }
         }
 
-        box2d<double> box = this->perform_transform(angle, x, y);
+        if (!this->push_to_detector(x, y, angle, ignore_placement))
-        if (this->params_.avoid_edges && !this->detector_.extent().contains(box))
         {
             return false;
         }
-        if (!this->params_.allow_overlap && !this->detector_.has_placement(box))
-        {
-            return false;
-        }
-
-        if (!ignore_placement)
-        {
-            this->detector_.insert(box);
-        }
 
         this->done_ = true;
         return true;

include/mapnik/markers_placements/vertext_last.hpp

@@ -31,13 +31,8 @@ template <typename Locator, typename Detector>
 class markers_vertex_last_placement : public markers_point_placement<Locator, Detector>
 {
 public:
-    markers_vertex_last_placement(Locator &locator, Detector &detector, markers_placement_params const& params)
+    using point_placement = markers_point_placement<Locator, Detector>;
-        : markers_point_placement<Locator, Detector>(locator, detector, params)
+    using point_placement::point_placement;
-    {}
-
-    markers_vertex_last_placement(markers_vertex_last_placement && rhs)
-        : markers_point_placement<Locator, Detector>(std::move(rhs))
-    {}
 
     bool get_point(double &x, double &y, double &angle, bool ignore_placement)
     {
@@ -80,20 +75,10 @@ public:
             }
         }
 
-        box2d<double> box = this->perform_transform(angle, x, y);
+        if (!this->push_to_detector(x, y, angle, ignore_placement))
-        if (this->params_.avoid_edges && !this->detector_.extent().contains(box))
         {
             return false;
         }
-        if (!this->params_.allow_overlap && !this->detector_.has_placement(box))
-        {
-            return false;
-        }
-
-        if (!ignore_placement)
-        {
-            this->detector_.insert(box);
-        }
 
         this->done_ = true;
         return true;

include/mapnik/text/symbolizer_helpers.hpp

@@ -59,7 +59,7 @@ struct placement_finder_adapter
 
 };
 
-using vertex_converter_type = vertex_converter<clip_line_tag , transform_tag, affine_transform_tag, simplify_tag, smooth_tag>;
+using vertex_converter_type = vertex_converter<clip_line_tag, transform_tag, affine_transform_tag, simplify_tag, smooth_tag>;
 
 class base_symbolizer_helper
 {

include/mapnik/unicode.hpp

@@ -45,6 +45,10 @@ public:
 private:
     UConverter * conv_;
 };
+
+// convinience method
+void MAPNIK_DECL to_utf8(mapnik::value_unicode_string const& input, std::string & target);
+
 }
 
 #endif // MAPNIK_UNICODE_HPP

include/mapnik/value.hpp

@@ -24,805 +24,25 @@
 #define MAPNIK_VALUE_HPP
 
 // mapnik
+#include <mapnik/config.hpp>
 #include <mapnik/value_types.hpp>
 #include <mapnik/value_hash.hpp>
-#include <mapnik/util/conversions.hpp>
 #include <mapnik/util/variant.hpp>
 
-// stl
-#include <string>
-#include <cmath>
-#include <memory>
-
-#include <iosfwd>
-#include <cstddef>
-#include <new>
-#include <type_traits>
-
-// icu
-#include <unicode/unistr.h>
-#include <unicode/ustring.h>
 
 namespace mapnik {
 
 using value_base = util::variant<value_null, value_bool, value_integer,value_double, value_unicode_string>;
 
-inline void to_utf8(mapnik::value_unicode_string const& input, std::string & target)
-{
-    target.clear(); // mimic previous target.assign(...) semantics
-    input.toUTF8String(target); // this appends to target
-}
-
-namespace detail {
-
-namespace {
-template <typename T, typename U>
-struct both_arithmetic : std::integral_constant<bool,
-                                             std::is_arithmetic<T>::value &&
-                                             std::is_arithmetic<U>::value > {};
-
-struct equals
-{
-    static bool apply(value_null, value_unicode_string const& rhs)
-    {
-        return false;
-    }
-
-    template <typename T>
-    static auto apply(T const& lhs, T const& rhs)
-        -> decltype(lhs == rhs)
-    {
-        return lhs == rhs;
-    }
-};
-
-struct not_equal
-{
-    // back compatibility shim to equate empty string with null for != test
-    // https://github.com/mapnik/mapnik/issues/1859
-    // TODO - consider removing entire specialization at Mapnik 3.1.x
-    static bool apply(value_null, value_unicode_string const& rhs)
-    {
-        if (rhs.isEmpty()) return false;
-        return true;
-    }
-
-    template <typename T>
-    static auto apply(T const& lhs, T const& rhs)
-        ->decltype(lhs != rhs)
-    {
-        return lhs != rhs;
-    }
-};
-
-struct greater_than
-{
-    static bool apply(value_null, value_unicode_string const& rhs)
-    {
-        return false;
-    }
-
-    template <typename T>
-    static auto  apply(T const& lhs, T const& rhs)
-        ->decltype(lhs > rhs)
-    {
-        return lhs > rhs;
-    }
-};
-
-struct greater_or_equal
-{
-    static bool apply(value_null, value_unicode_string const& rhs)
-    {
-        return false;
-    }
-
-    template <typename T>
-    static auto apply(T const& lhs, T const& rhs)
-        ->decltype(lhs >= rhs)
-    {
-        return lhs >= rhs;
-    }
-};
-
-struct less_than
-{
-    static bool apply(value_null, value_unicode_string const& rhs)
-    {
-        return false;
-    }
-
-    template <typename T>
-    static auto apply(T const& lhs, T const& rhs)
-         ->decltype(lhs < rhs)
-    {
-        return lhs < rhs;
-    }
-};
-
-struct less_or_equal
-{
-    static bool apply(value_null, value_unicode_string const& rhs)
-    {
-        return false;
-    }
-
-    template <typename T>
-    static auto apply(T const& lhs, T const& rhs)
-        ->decltype(lhs <= rhs)
-    {
-        return lhs <= rhs;
-    }
-};
-
-}
-
-template <typename Op, bool default_result>
-struct comparison
-{
-    // special case for unicode_strings (fixes MSVC C4800)
-    bool operator() (value_unicode_string const& lhs,
-                     value_unicode_string const& rhs) const
-    {
-        return Op::apply(lhs, rhs) ? true : false;
-    }
-
-    //////////////////////////////////////////////////////////////////////////
-    // special case for unicode_string and value_null
-    //////////////////////////////////////////////////////////////////////////
-
-    bool operator() (value_null const& lhs, value_unicode_string const& rhs) const
-    {
-        return Op::apply(lhs, rhs);
-    }
-    //////////////////////////////////////////////////////////////////////////
-
-
-    // same types
-    template <typename T>
-    bool operator() (T lhs, T rhs) const
-    {
-        return Op::apply(lhs, rhs);
-    }
-
-    // both types are arithmetic - promote to the common type
-    template <typename T, typename U, typename std::enable_if<both_arithmetic<T,U>::value, int>::type = 0>
-    bool operator() (T const& lhs, U const& rhs) const
-    {
-        using common_type = typename std::common_type<T,U>::type;
-        return Op::apply(static_cast<common_type>(lhs),static_cast<common_type>(rhs));
-    }
-
-    //
-    template <typename T, typename U, typename std::enable_if<!both_arithmetic<T,U>::value, int>::type = 0>
-    bool operator() (T const& lhs, U const& rhs) const
-    {
-        return default_result;
-    }
-};
-
-template <typename V>
-struct add
-{
-    using value_type = V;
-    value_type operator() (value_unicode_string const& lhs ,
-                           value_unicode_string const& rhs ) const
-    {
-        return lhs + rhs;
-    }
-
-    value_type operator() (value_null const& lhs ,
-                           value_null const& rhs) const
-    {
-        return lhs;
-    }
-
-    value_type operator() (value_unicode_string const& lhs, value_null) const
-    {
-        return lhs;
-    }
-
-    value_type operator() (value_null, value_unicode_string const& rhs) const
-    {
-        return rhs;
-    }
-
-    template <typename L>
-    value_type operator() (L const& lhs, value_null const&) const
-    {
-        return lhs;
-    }
-
-    template <typename R>
-    value_type operator() (value_null const&, R const& rhs) const
-    {
-        return rhs;
-    }
-
-    template <typename L>
-    value_type operator() (L const& lhs , value_unicode_string const& rhs) const
-    {
-        std::string val;
-        if (util::to_string(val,lhs))
-            return value_unicode_string(val.c_str()) + rhs;
-        return rhs;
-    }
-
-    template <typename R>
-    value_type operator() (value_unicode_string const& lhs, R const& rhs) const
-    {
-        std::string val;
-        if (util::to_string(val,rhs))
-            return lhs + value_unicode_string(val.c_str());
-        return lhs;
-    }
-
-    template <typename T1, typename T2>
-    value_type operator() (T1 const& lhs, T2 const& rhs) const
-    {
-        return typename std::common_type<T1,T2>::type{ lhs + rhs };
-    }
-
-    value_type operator() (value_bool lhs, value_bool rhs) const
-    {
-        return value_integer(lhs + rhs);
-    }
-};
-
-template <typename V>
-struct sub
-{
-    using value_type = V;
-
-    value_type operator() (value_null const& lhs ,
-                           value_null const& rhs) const
-    {
-        return lhs;
-    }
-
-    value_type operator() (value_null, value_unicode_string const& rhs) const
-    {
-        return rhs;
-    }
-    value_type operator() (value_unicode_string const& lhs, value_null) const
-    {
-        return lhs;
-    }
-
-    template <typename R>
-    value_type operator() (value_unicode_string const& lhs, R const&) const
-    {
-        return lhs;
-    }
-
-    template <typename L>
-    value_type operator() (L const&, value_unicode_string const& rhs) const
-    {
-        return rhs;
-    }
-
-    template <typename L>
-    value_type operator() (L const& lhs, value_null const&) const
-    {
-        return lhs;
-    }
-
-    template <typename R>
-    value_type operator() (value_null const&, R const& rhs) const
-    {
-        return rhs;
-    }
-
-    template <typename T>
-    value_type operator() (T lhs, T rhs) const
-    {
-        return lhs - rhs ;
-    }
-
-    value_type operator() (value_unicode_string const&,
-                           value_unicode_string const&) const
-    {
-        return value_type();
-    }
-
-    template <typename T1, typename T2>
-    value_type operator() (T1 const& lhs, T2 const& rhs) const
-    {
-        return typename std::common_type<T1,T2>::type{ lhs - rhs };
-    }
-
-
-    value_type operator() (value_bool lhs, value_bool rhs) const
-    {
-        return value_integer(lhs - rhs);
-    }
-};
-
-template <typename V>
-struct mult
-{
-    using value_type = V;
-
-    value_type operator() (value_null const& lhs ,
-                           value_null const& rhs) const
-    {
-        return lhs;
-    }
-
-    value_type operator() (value_unicode_string const& lhs, value_null) const
-    {
-        return lhs;
-    }
-
-    value_type operator() (value_null, value_unicode_string const& rhs) const
-    {
-        return rhs;
-    }
-
-    template <typename L>
-    value_type operator() (L const& lhs, value_null const&) const
-    {
-        return lhs;
-    }
-
-    template <typename R>
-    value_type operator() (value_null const&, R const& rhs) const
-    {
-        return rhs;
-    }
-
-    template <typename R>
-    value_type operator() (value_unicode_string const& lhs, R const&) const
-    {
-        return lhs;
-    }
-
-    template <typename L>
-    value_type operator() (L const&, value_unicode_string const& rhs) const
-    {
-        return rhs;
-    }
-
-    template <typename T>
-    value_type operator() (T lhs, T rhs) const
-    {
-        return lhs * rhs;
-    }
-
-    value_type operator() (value_unicode_string const&,
-                           value_unicode_string const&) const
-    {
-        return value_type();
-    }
-
-    template <typename T1, typename T2>
-    value_type operator() (T1 const& lhs, T2 const& rhs) const
-    {
-        return typename std::common_type<T1,T2>::type{ lhs * rhs };
-    }
-
-    value_type operator() (value_bool lhs, value_bool rhs) const
-    {
-        return value_integer(lhs * rhs);
-    }
-};
-
-template <typename V>
-struct div
-{
-    using value_type = V;
-
-    value_type operator() (value_null const& lhs ,
-                           value_null const& rhs) const
-    {
-        return lhs;
-    }
-
-    value_type operator() (value_unicode_string const& lhs, value_null) const
-    {
-        return lhs;
-    }
-
-    value_type operator() (value_null, value_unicode_string const& rhs) const
-    {
-        return rhs;
-    }
-
-    template <typename L>
-    value_type operator() (L const& lhs, value_null const&) const
-    {
-        return lhs;
-    }
-
-    template <typename R>
-    value_type operator() (value_null const&, R const& rhs) const
-    {
-        return rhs;
-    }
-
-    template <typename T>
-    value_type operator() (T lhs, T rhs) const
-    {
-        if (rhs == 0) return value_type();
-        return lhs / rhs;
-    }
-
-    value_type operator() (value_bool lhs, value_bool rhs) const
-    {
-        if (rhs == 0) return lhs;
-        return value_integer(lhs) / value_integer(rhs);
-    }
-
-    value_type operator() (value_unicode_string const&,
-                           value_unicode_string const&) const
-    {
-        return value_type();
-    }
-
-    template <typename R>
-    value_type operator() (value_unicode_string const& lhs, R const&) const
-    {
-        return lhs;
-    }
-
-    template <typename L>
-    value_type operator() (L const&, value_unicode_string const& rhs) const
-    {
-        return rhs;
-    }
-
-    template <typename T1, typename T2>
-    value_type operator() (T1 const& lhs, T2 const& rhs) const
-    {
-        if (rhs == 0) return value_type();
-        using common_type = typename std::common_type<T1,T2>::type;
-        return common_type(lhs)/common_type(rhs);
-    }
-};
-
-template <typename V>
-struct mod
-{
-    using value_type = V;
-
-    template <typename T1, typename T2>
-    value_type operator() (T1 const& lhs, T2 const&) const
-    {
-       return lhs;
-    }
-
-    template <typename T>
-    value_type operator() (T lhs, T rhs) const
-    {
-        return lhs % rhs;
-    }
-
-    value_type operator() (value_unicode_string const&,
-                           value_unicode_string const&) const
-    {
-        return value_type();
-    }
-
-    value_type operator() (value_bool,
-                           value_bool) const
-    {
-        return false;
-    }
-
-    value_type operator() (value_double lhs, value_integer rhs) const
-    {
-        return std::fmod(lhs, static_cast<value_double>(rhs));
-    }
-
-    value_type operator() (value_integer lhs, value_double rhs) const
-    {
-        return std::fmod(static_cast<value_double>(lhs), rhs);
-    }
-
-    value_type operator() (value_double lhs, value_double rhs) const
-    {
-        return std::fmod(lhs, rhs);
-    }
-};
-
-template <typename V>
-struct negate
-{
-    using value_type = V;
-
-    template <typename T>
-    value_type operator() (T val) const
-    {
-        return -val;
-    }
-
-    value_type operator() (value_null val) const
-    {
-        return val;
-    }
-
-    value_type operator() (value_bool val) const
-    {
-        return val ? value_integer(-1) : value_integer(0);
-    }
-
-    value_type operator() (value_unicode_string const&) const
-    {
-        return value_type();
-    }
-};
-
-// converters
-template <typename T>
-struct convert {};
-
-template <>
-struct convert<value_bool>
-{
-    value_bool operator() (value_bool val) const
-    {
-        return val;
-    }
-
-    value_bool operator() (value_unicode_string const& ustr) const
-    {
-        return !ustr.isEmpty();
-    }
-
-    value_bool operator() (value_null const&) const
-    {
-        return false;
-    }
-
-    template <typename T>
-    value_bool operator() (T val) const
-    {
-        return val > 0 ? true : false;
-    }
-};
-
-template <>
-struct convert<value_double>
-{
-    value_double operator() (value_double val) const
-    {
-        return val;
-    }
-
-    value_double operator() (value_integer val) const
-    {
-        return static_cast<value_double>(val);
-    }
-
-    value_double operator() (value_bool val) const
-    {
-        return static_cast<value_double>(val);
-    }
-
-    value_double operator() (std::string const& val) const
-    {
-        value_double result;
-        if (util::string2double(val,result))
-            return result;
-        return 0;
-    }
-
-    value_double operator() (value_unicode_string const& val) const
-    {
-        std::string utf8;
-        val.toUTF8String(utf8);
-        return operator()(utf8);
-    }
-
-    value_double operator() (value_null const&) const
-    {
-        return 0.0;
-    }
-};
-
-template <>
-struct convert<value_integer>
-{
-    value_integer operator() (value_integer val) const
-    {
-        return val;
-    }
-
-    value_integer operator() (value_double val) const
-    {
-        return static_cast<value_integer>(rint(val));
-    }
-
-    value_integer operator() (value_bool val) const
-    {
-        return static_cast<value_integer>(val);
-    }
-
-    value_integer operator() (std::string const& val) const
-    {
-        value_integer result;
-        if (util::string2int(val,result))
-            return result;
-        return value_integer(0);
-    }
-
-    value_integer operator() (value_unicode_string const& val) const
-    {
-        std::string utf8;
-        val.toUTF8String(utf8);
-        return operator()(utf8);
-    }
-
-    value_integer operator() (value_null const&) const
-    {
-        return value_integer(0);
-    }
-};
-
-template <>
-struct convert<std::string>
-{
-    template <typename T>
-    std::string operator() (T val) const
-    {
-        std::string str;
-        util::to_string(str, val);
-        return str;
-    }
-
-    // specializations
-    std::string operator() (value_unicode_string const& val) const
-    {
-        std::string utf8;
-        val.toUTF8String(utf8);
-        return utf8;
-    }
-
-    std::string operator() (value_double val) const
-    {
-        std::string str;
-        util::to_string(str, val); // TODO set precision(16)
-        return str;
-    }
-
-    std::string operator() (value_bool val) const
-    {
-        return val ? "true": "false";
-    }
-
-    std::string operator() (value_null const&) const
-    {
-        return std::string();
-    }
-};
-
-struct to_unicode_impl
-{
-
-    template <typename T>
-    value_unicode_string operator() (T val) const
-    {
-        std::string str;
-        util::to_string(str,val);
-        return value_unicode_string(str.c_str());
-    }
-
-    // specializations
-    value_unicode_string const& operator() (value_unicode_string const& val) const
-    {
-        return val;
-    }
-
-    value_unicode_string operator() (value_double val) const
-    {
-        std::string str;
-        util::to_string(str,val);
-        return value_unicode_string(str.c_str());
-    }
-
-    value_unicode_string operator() (value_bool val) const
-    {
-        return value_unicode_string(val ? "true" : "false");
-    }
-
-    value_unicode_string operator() (value_null const&) const
-    {
-        return value_unicode_string();
-    }
-};
-
-struct to_expression_string_impl
-{
-    struct EscapingByteSink : U_NAMESPACE_QUALIFIER ByteSink
-    {
-        std::string dest_;
-        char quote_;
-
-        explicit EscapingByteSink(char quote)
-            : quote_(quote)
-        {}
-
-        virtual void Append(const char* data, int32_t n)
-        {
-            // reserve enough room to hold the appended chunk and quotes;
-            // if another chunk follows, or any character needs escaping,
-            // the string will grow naturally
-            if (dest_.empty())
-            {
-                dest_.reserve(2 + static_cast<std::size_t>(n));
-                dest_.append(1, quote_);
-            }
-            else
-            {
-                dest_.reserve(dest_.size() + n + 1);
-            }
-
-            for (auto end = data + n; data < end; ++data)
-            {
-                if (*data == '\\' || *data == quote_)
-                    dest_.append(1, '\\');
-                dest_.append(1, *data);
-            }
-        }
-
-        virtual void Flush()
-        {
-            if (dest_.empty())
-                dest_.append(2, quote_);
-            else
-                dest_.append(1, quote_);
-        }
-    };
-
-    explicit to_expression_string_impl(char quote = '\'')
-        : quote_(quote) {}
-
-    std::string operator() (value_unicode_string const& val) const
-    {
-        EscapingByteSink sink(quote_);
-        val.toUTF8(sink);
-        return sink.dest_;
-    }
-
-    std::string operator() (value_integer val) const
-    {
-        std::string output;
-        util::to_string(output,val);
-        return output;
-    }
-
-    std::string operator() (value_double val) const
-    {
-        std::string output;
-        util::to_string(output,val); // TODO precision(16)
-        return output;
-    }
-
-    std::string operator() (value_bool val) const
-    {
-        return val ? "true" : "false";
-    }
-
-    std::string operator() (value_null const&) const
-    {
-        return "null";
-    }
-
-    const char quote_;
-};
-
-
-} // namespace detail
-
 namespace value_adl_barrier {
 
-class value : public value_base
+class MAPNIK_DECL value : public value_base
 {
-    friend const value operator+(value const&,value const&);
+    friend MAPNIK_DECL value operator+(value const&,value const&);
-    friend const value operator-(value const&,value const&);
+    friend MAPNIK_DECL value operator-(value const&,value const&);
-    friend const value operator*(value const&,value const&);
+    friend MAPNIK_DECL value operator*(value const&,value const&);
-    friend const value operator/(value const&,value const&);
+    friend MAPNIK_DECL value operator/(value const&,value const&);
-    friend const value operator%(value const&,value const&);
+    friend MAPNIK_DECL value operator%(value const&,value const&);
 
 public:
     value() = default;
@@ -856,104 +76,32 @@ public:
         return *this;
     }
 
-    bool operator==(value const& other) const
+    bool operator==(value const& other) const;
-    {
+    bool operator!=(value const& other) const;
-        return util::apply_visitor(detail::comparison<detail::equals, false>(), *this, other);
+    bool operator>(value const& other) const;
-    }
+    bool operator>=(value const& other) const;
+    bool operator<(value const& other) const;
+    bool operator<=(value const& other) const;
 
-    bool operator!=(value const& other) const
+    value operator-() const;
-    {
-        return util::apply_visitor(detail::comparison<detail::not_equal, true>(), *this, other);
-    }
-
-    bool operator>(value const& other) const
-    {
-        return util::apply_visitor(detail::comparison<detail::greater_than, false>(), *this, other);
-    }
-
-    bool operator>=(value const& other) const
-    {
-        return util::apply_visitor(detail::comparison<detail::greater_or_equal, false>(), *this, other);
-    }
-
-    bool operator<(value const& other) const
-    {
-        return util::apply_visitor(detail::comparison<detail::less_than, false>(), *this, other);
-    }
-
-    bool operator<=(value const& other) const
-    {
-        return util::apply_visitor(detail::comparison<detail::less_or_equal, false>(), *this, other);
-    }
-
-    value operator- () const
-    {
-        return util::apply_visitor(detail::negate<value>(), *this);
-    }
 
     bool is_null() const;
 
-    template <typename T>
+    template <typename T> T convert() const;
-    T convert() const
-    {
-        return util::apply_visitor(detail::convert<T>(),*this);
-    }
 
-    value_bool to_bool() const
+    value_bool to_bool() const;
-    {
+    std::string to_expression_string(char quote = '\'') const;
-        return util::apply_visitor(detail::convert<value_bool>(),*this);
+    std::string to_string() const;
-    }
+    value_unicode_string to_unicode() const;
-
+    value_double to_double() const;
-    std::string to_expression_string(char quote = '\'') const
+    value_integer to_int() const;
-    {
-        return util::apply_visitor(detail::to_expression_string_impl(quote),*this);
-    }
-
-    std::string to_string() const
-    {
-        return util::apply_visitor(detail::convert<std::string>(),*this);
-    }
-
-    value_unicode_string to_unicode() const
-    {
-        return util::apply_visitor(detail::to_unicode_impl(),*this);
-    }
-
-    value_double to_double() const
-    {
-        return util::apply_visitor(detail::convert<value_double>(),*this);
-    }
-
-    value_integer to_int() const
-    {
-        return util::apply_visitor(detail::convert<value_integer>(),*this);
-    }
 };
 
-inline const value operator+(value const& p1,value const& p2)
+MAPNIK_DECL value operator+(value const& p1,value const& p2);
-{
+MAPNIK_DECL value operator-(value const& p1,value const& p2);
-    return value(util::apply_visitor(detail::add<value>(),p1, p2));
+MAPNIK_DECL value operator*(value const& p1,value const& p2);
-}
+MAPNIK_DECL value operator/(value const& p1,value const& p2);
-
+MAPNIK_DECL value operator%(value const& p1,value const& p2);
-inline const value operator-(value const& p1,value const& p2)
-{
-    return value(util::apply_visitor(detail::sub<value>(),p1, p2));
-}
-
-inline const value operator*(value const& p1,value const& p2)
-{
-    return value(util::apply_visitor(detail::mult<value>(),p1, p2));
-}
-
-inline const value operator/(value const& p1,value const& p2)
-{
-    return value(util::apply_visitor(detail::div<value>(),p1, p2));
-}
-
-inline const value operator%(value const& p1,value const& p2)
-{
-    return value(util::apply_visitor(detail::mod<value>(),p1, p2));
-}
 
 template <typename charT, typename traits>
 inline std::basic_ostream<charT,traits>&
@@ -972,36 +120,28 @@ inline std::size_t hash_value(value const& val)
 
 } // namespace value_adl_barrier
 
-using value_adl_barrier::value;
+using value = value_adl_barrier::value;
 
 namespace detail {
-
 struct is_null_visitor
 {
-    bool operator() (value const& val) const
+    bool operator()(value const& val) const
     {
         return val.is_null();
     }
 
-    bool operator() (value_null const&) const
+    bool operator()(value_null const&) const
     {
         return true;
     }
 
     template <typename T>
-    bool operator() (T const&) const
+    bool operator()(T const&) const
     {
         return false;
     }
 };
-
 } // namespace detail
-
-inline bool value::is_null() const
-{
-    return util::apply_visitor(mapnik::detail::is_null_visitor(), *this);
-}
-
 } // namespace mapnik
 
 // support for std::unordered_xxx

include/mapnik/version.hpp

@@ -25,7 +25,7 @@
 
 #define MAPNIK_MAJOR_VERSION 3
 #define MAPNIK_MINOR_VERSION 0
-#define MAPNIK_PATCH_VERSION 9
+#define MAPNIK_PATCH_VERSION 10
 
 #define MAPNIK_VERSION (MAPNIK_MAJOR_VERSION*100000) + (MAPNIK_MINOR_VERSION*100) + (MAPNIK_PATCH_VERSION)
 

include/mapnik/vertex_adapters.hpp

@@ -34,32 +34,10 @@ struct point_vertex_adapter
 {
     using value_type = typename point<T>::value_type;
 
-    point_vertex_adapter(point<T> const& pt)
+    point_vertex_adapter(point<T> const& pt);
-        : pt_(pt),
+    unsigned vertex(value_type * x, value_type * y) const;
-          first_(true) {}
+    void rewind(unsigned) const;
-
+    geometry_types type () const;
-    unsigned vertex(value_type * x, value_type * y) const
-    {
-        if (first_)
-        {
-            *x = pt_.x;
-            *y = pt_.y;
-            first_ = false;
-            return mapnik::SEG_MOVETO;
-        }
-        return mapnik::SEG_END;
-    }
-
-    void rewind(unsigned) const
-    {
-        first_ = true;
-    }
-
-    inline geometry_types type () const
-    {
-        return geometry_types::Point;
-    }
-
     point<T> const& pt_;
     mutable bool first_;
 };
@@ -68,110 +46,23 @@ template <typename T>
 struct line_string_vertex_adapter
 {
     using value_type = typename point<T>::value_type;
-    line_string_vertex_adapter(line_string<T> const& line)
+    line_string_vertex_adapter(line_string<T> const& line);
-        : line_(line),
+    unsigned vertex(value_type * x, value_type * y) const;
-          current_index_(0),
+    void rewind(unsigned) const;
-          end_index_(line.size())
+    geometry_types type () const;
-    {}
-
-    unsigned vertex(value_type * x, value_type * y) const
-    {
-        if (current_index_ != end_index_)
-        {
-            point<T> const& coord = line_[current_index_++];
-            *x = coord.x;
-            *y = coord.y;
-            if (current_index_ == 1)
-            {
-                return mapnik::SEG_MOVETO;
-            }
-            else
-            {
-                return mapnik::SEG_LINETO;
-            }
-        }
-        return mapnik::SEG_END;
-    }
-
-    void rewind(unsigned) const
-    {
-        current_index_ = 0;
-    }
-
-    inline geometry_types type () const
-    {
-        return geometry_types::LineString;
-    }
-
     line_string<T> const& line_;
     mutable std::size_t current_index_;
-    const std::size_t end_index_;
+   const std::size_t end_index_;
-
 };
 
 template <typename T>
 struct polygon_vertex_adapter
 {
     using value_type = typename point<T>::value_type;
-    polygon_vertex_adapter(polygon<T> const& poly)
+    polygon_vertex_adapter(polygon<T> const& poly);
-        : poly_(poly),
+    void rewind(unsigned) const;
-          rings_itr_(0),
+    unsigned vertex(value_type * x, value_type * y) const;
-          rings_end_(poly_.interior_rings.size() + 1),
+    geometry_types type () const;
-          current_index_(0),
-          end_index_((rings_itr_ < rings_end_) ? poly_.exterior_ring.size() : 0),
-          start_loop_(true) {}
-
-    void rewind(unsigned) const
-    {
-        rings_itr_ = 0;
-        rings_end_ = poly_.interior_rings.size() + 1;
-        current_index_ = 0;
-        end_index_ = (rings_itr_ < rings_end_) ? poly_.exterior_ring.size() : 0;
-        start_loop_ = true;
-    }
-
-    unsigned vertex(value_type * x, value_type * y) const
-    {
-        if (rings_itr_ == rings_end_)
-        {
-            return mapnik::SEG_END;
-        }
-        if (current_index_ < end_index_)
-        {
-            point<T> const& coord = (rings_itr_ == 0) ?
-                poly_.exterior_ring[current_index_++] : poly_.interior_rings[rings_itr_- 1][current_index_++];
-            *x = coord.x;
-            *y = coord.y;
-            if (start_loop_)
-            {
-                start_loop_= false;
-                return mapnik::SEG_MOVETO;
-            }
-            if (current_index_ == end_index_)
-            {
-                *x = 0;
-                *y = 0;
-                return mapnik::SEG_CLOSE;
-            }
-            return mapnik::SEG_LINETO;
-        }
-        else if (++rings_itr_ != rings_end_)
-        {
-            current_index_ = 0;
-            end_index_ = poly_.interior_rings[rings_itr_ - 1].size();
-            point<T> const& coord = poly_.interior_rings[rings_itr_ - 1][current_index_++];
-            *x = coord.x;
-            *y = coord.y;
-            return mapnik::SEG_MOVETO;
-        }
-        return mapnik::SEG_END;
-    }
-
-    inline geometry_types type () const
-    {
-        return geometry_types::Polygon;
-    }
-
 private:
     polygon<T> const& poly_;
     mutable std::size_t rings_itr_;
@@ -185,47 +76,10 @@ template <typename T>
 struct ring_vertex_adapter
 {
     using value_type = typename point<T>::value_type;
-    ring_vertex_adapter(linear_ring<T> const& ring)
+    ring_vertex_adapter(linear_ring<T> const& ring);
-        : ring_(ring),
+    void rewind(unsigned) const;
-          current_index_(0),
+    unsigned vertex(value_type * x, value_type * y) const;
-          end_index_(ring_.size()),
+    geometry_types type () const;
-          start_loop_(true) {}
-
-    void rewind(unsigned) const
-    {
-        current_index_ = 0;
-        end_index_ = ring_.size();
-        start_loop_ = true;
-    }
-
-    unsigned vertex(value_type * x, value_type * y) const
-    {
-        if (current_index_ < end_index_)
-        {
-            auto const& coord = ring_[current_index_++];
-            *x = coord.x;
-            *y = coord.y;
-            if (start_loop_)
-            {
-                start_loop_= false;
-                return mapnik::SEG_MOVETO;
-            }
-            if (current_index_ == end_index_)
-            {
-                *x = 0;
-                *y = 0;
-                return mapnik::SEG_CLOSE;
-            }
-            return mapnik::SEG_LINETO;
-        }
-        return mapnik::SEG_END;
-    }
-
-    inline geometry_types type () const
-    {
-        return geometry_types::Polygon;
-    }
-
 private:
     linear_ring<T> const& ring_;
     mutable std::size_t current_index_;
@@ -233,6 +87,11 @@ private:
     mutable bool start_loop_;
 };
 
+extern template struct MAPNIK_DECL point_vertex_adapter<double>;
+extern template struct MAPNIK_DECL line_string_vertex_adapter<double>;
+extern template struct MAPNIK_DECL polygon_vertex_adapter<double>;
+extern template struct MAPNIK_DECL ring_vertex_adapter<double>;
+
 template <typename T>
 struct vertex_adapter_traits {};
 

include/mapnik/vertex_converters.hpp

@@ -275,7 +275,7 @@ template <typename Dispatcher, typename... ConverterTypes>
 struct converters_helper;
 
 template <typename Dispatcher, typename Current, typename... ConverterTypes>
-struct converters_helper<Dispatcher,Current,ConverterTypes...>
+struct converters_helper<Dispatcher, Current, ConverterTypes...>
 {
     template <typename Converter>
     static void set(Dispatcher & disp, std::size_t state)

plugins/input/geojson/geojson_datasource.cpp

@@ -200,6 +200,20 @@ const mapnik::json::feature_grammar<base_iterator_type, mapnik::feature_impl> ge
 const mapnik::json::extract_bounding_box_grammar<base_iterator_type> geojson_datasource_static_bbox_grammar;
 }
 
+void geojson_datasource::initialise_descriptor(mapnik::feature_ptr const& feature)
+{
+    for ( auto const& kv : *feature)
+    {
+        auto const& name = std::get<0>(kv);
+        if (!desc_.has_name(name))
+        {
+            desc_.add_descriptor(mapnik::attribute_descriptor(name,
+                                                              mapnik::util::apply_visitor(attr_value_converter(),
+                                                                                          std::get<1>(kv))));
+        }
+    }
+}
+
 void geojson_datasource::initialise_disk_index(std::string const& filename)
 {
     // read extent
@@ -213,7 +227,7 @@ void geojson_datasource::initialise_disk_index(std::string const& filename)
     std::vector<value_type> positions;
     mapnik::util::spatial_index<value_type,
                                 mapnik::filter_in_box,
-                                std::ifstream>::query_first_n(filter, index, positions, 5);
+                                std::ifstream>::query_first_n(filter, index, positions, num_features_to_query_);
 
     mapnik::util::file file(filename_);
     if (!file.open()) throw mapnik::datasource_exception("GeoJSON Plugin: could not open: '" + filename_ + "'");
@@ -236,16 +250,7 @@ void geojson_datasource::initialise_disk_index(std::string const& filename)
         {
             throw std::runtime_error("Failed to parse geojson feature");
         }
-        for ( auto const& kv : *feature)
+        initialise_descriptor(feature);
-        {
-            auto const& name = std::get<0>(kv);
-            if (!desc_.has_name(name))
-            {
-                desc_.add_descriptor(mapnik::attribute_descriptor(name,
-                                                                  mapnik::util::apply_visitor(attr_value_converter(),
-                                                                                              std::get<1>(kv))));
-            }
-        }
     }
 }
 
@@ -285,20 +290,18 @@ void geojson_datasource::initialise_index(Iterator start, Iterator end)
                 if (geometry_index == 0)
                 {
                     extent_ = box;
-                    for ( auto const& kv : *f)
-                    {
-                        desc_.add_descriptor(mapnik::attribute_descriptor(std::get<0>(kv),
-                                                                          mapnik::util::apply_visitor(attr_value_converter(),
-                                                                                                      std::get<1>(kv))));
-                    }
                 }
                 else
                 {
                     extent_.expand_to_include(box);
                 }
                 values.emplace_back(box, std::make_pair(geometry_index,0));
+
+            }
+            if (geometry_index++ < num_features_to_query_)
+            {
+                initialise_descriptor(f);
             }
-            ++geometry_index;
         }
         // packing algorithm
         tree_ = std::make_unique<spatial_index_type>(values);
@@ -308,14 +311,16 @@ void geojson_datasource::initialise_index(Iterator start, Iterator end)
         // bulk insert initialise r-tree
         tree_ = std::make_unique<spatial_index_type>(boxes);
         // calculate total extent
+        std::size_t feature_count = 0;
         for (auto const& item : boxes)
         {
             auto const& box = std::get<0>(item);
             auto const& geometry_index = std::get<1>(item);
-            if (!extent_.valid())
+            if (!extent_.valid()) extent_ = box;
+            else extent_.expand_to_include(box);
+            if (feature_count++ < num_features_to_query_)
             {
-                extent_ = box;
+                // parse first N features to extract attributes schema.
-                // parse first feature to extract attributes schema.
                 // NOTE: this doesn't yield correct answer for geoJSON in general, just an indication
                 Iterator itr2 = start + geometry_index.first;
                 Iterator end2 = itr2 + geometry_index.second;
@@ -327,16 +332,8 @@ void geojson_datasource::initialise_index(Iterator start, Iterator end)
                 {
                     throw std::runtime_error("Failed to parse geojson feature");
                 }
-                for ( auto const& kv : *feature)
+
-                {
+                initialise_descriptor(feature);
-                    desc_.add_descriptor(mapnik::attribute_descriptor(std::get<0>(kv),
-                                                                      mapnik::util::apply_visitor(attr_value_converter(),
-                                                                                                  std::get<1>(kv))));
-                }
-            }
-            else
-            {
-                extent_.expand_to_include(box);
             }
         }
     }
@@ -399,12 +396,7 @@ void geojson_datasource::parse_geojson(Iterator start, Iterator end)
             if (geometry_index == 0)
             {
                 extent_ = box;
-                for ( auto const& kv : *f)
+
-                {
-                    desc_.add_descriptor(mapnik::attribute_descriptor(std::get<0>(kv),
-                                                                      mapnik::util::apply_visitor(attr_value_converter(),
-                                                                                                  std::get<1>(kv))));
-                }
             }
             else
             {
@@ -412,6 +404,10 @@ void geojson_datasource::parse_geojson(Iterator start, Iterator end)
             }
             values.emplace_back(box, std::make_pair(geometry_index,0));
         }
+        if (geometry_index < num_features_to_query_)
+        {
+            initialise_descriptor(f);
+        }
         ++geometry_index;
     }
     // packing algorithm
@@ -419,7 +415,7 @@ void geojson_datasource::parse_geojson(Iterator start, Iterator end)
 
 }
 
-geojson_datasource::~geojson_datasource() { }
+geojson_datasource::~geojson_datasource() {}
 
 const char * geojson_datasource::name()
 {
@@ -454,7 +450,7 @@ boost::optional<mapnik::datasource_geometry_t> geojson_datasource::get_geometry_
         std::vector<value_type> positions;
         mapnik::util::spatial_index<value_type,
                                     mapnik::filter_in_box,
-                                    std::ifstream>::query_first_n(filter, index, positions, 5);
+                                    std::ifstream>::query_first_n(filter, index, positions, num_features_to_query_);
 
         mapnik::util::file file(filename_);
 
@@ -494,7 +490,7 @@ boost::optional<mapnik::datasource_geometry_t> geojson_datasource::get_geometry_
     else if (cache_features_)
     {
         unsigned num_features = features_.size();
-        for (unsigned i = 0; i < num_features && i < 5; ++i)
+        for (unsigned i = 0; i < num_features && i < num_features_to_query_; ++i)
         {
             result = mapnik::util::to_ds_type(features_[i]->get_geometry());
             if (result)
@@ -519,7 +515,7 @@ boost::optional<mapnik::datasource_geometry_t> geojson_datasource::get_geometry_
         auto itr = tree_->qbegin(boost::geometry::index::intersects(extent_));
         auto end = tree_->qend();
         mapnik::context_ptr ctx = std::make_shared<mapnik::context_type>();
-        for (std::size_t count = 0; itr !=end &&  count < 5; ++itr,++count)
+        for (std::size_t count = 0; itr !=end &&  count < num_features_to_query_; ++itr,++count)
         {
             geojson_datasource::item_type const& item = *itr;
             std::size_t file_offset = item.second.first;

plugins/input/geojson/geojson_datasource.hpp

@@ -94,6 +94,7 @@ public:
     void initialise_index(Iterator start, Iterator end);
     void initialise_disk_index(std::string const& filename);
 private:
+    void initialise_descriptor(mapnik::feature_ptr const&);
     mapnik::datasource::datasource_t type_;
     mapnik::layer_descriptor desc_;
     std::string filename_;
@@ -103,6 +104,7 @@ private:
     std::unique_ptr<spatial_index_type> tree_;
     bool cache_features_ = true;
     bool has_disk_index_ = false;
+    const std::size_t num_features_to_query_ = 5;
 };
 
 

scripts/build-appveyor.bat

@@ -22,6 +22,8 @@ ECHO msvs_toolset^: %msvs_toolset%
 SET BUILD_TYPE=%configuration%
 SET BUILDPLATFORM=%platform%
 SET TOOLS_VERSION=%msvs_toolset%.0
+SET ICU_VERSION=56.1
+ECHO ICU_VERSION^: %ICU_VERSION%
 IF DEFINED APPVEYOR (ECHO on AppVeyor) ELSE (ECHO NOT on AppVeyor)
 ECHO ========
 

src/build.py

@@ -221,6 +221,7 @@ source = Split(
     warp.cpp
     css_color_grammar.cpp
     vertex_cache.cpp
+    vertex_adapters.cpp
     text/font_library.cpp
     text/text_layout.cpp
     text/text_line.cpp
@@ -257,6 +258,7 @@ source = Split(
     renderer_common/render_pattern.cpp
     renderer_common/render_thunk_extractor.cpp
     math.cpp
+    value.cpp
     """
     )
 

src/marker_helpers.cpp

@@ -23,7 +23,7 @@
 // mapnik
 #include <mapnik/marker_helpers.hpp>
 #include <mapnik/svg/svg_converter.hpp>
-
+#include <mapnik/label_collision_detector.hpp>
 #include "agg_ellipse.h"
 #include "agg_color_rgba.h"
 
@@ -158,5 +158,142 @@ void setup_transform_scaling(agg::trans_affine & tr,
     }
 }
 
+template <typename Processor>
+void apply_markers_single(vertex_converter_type & converter, Processor & proc,
+                          geometry::geometry<double> const& geom, geometry::geometry_types type)
+{
+    if (type == geometry::geometry_types::Point)
+    {
+        geometry::point_vertex_adapter<double> va(geom.get<geometry::point<double>>());
+        converter.apply(va, proc);
+    }
+    else if (type == geometry::geometry_types::LineString)
+    {
+        geometry::line_string_vertex_adapter<double> va(geom.get<geometry::line_string<double>>());
+        converter.apply(va, proc);
+    }
+    else if (type == geometry::geometry_types::Polygon)
+    {
+        geometry::polygon_vertex_adapter<double> va(geom.get<geometry::polygon<double>>());
+        converter.apply(va, proc);
+    }
+    else if (type == geometry::geometry_types::MultiPoint)
+    {
+        for (auto const& pt : geom.get<geometry::multi_point<double>>())
+        {
+            geometry::point_vertex_adapter<double> va(pt);
+            converter.apply(va, proc);
+        }
+    }
+    else if (type == geometry::geometry_types::MultiLineString)
+    {
+        for (auto const& line : geom.get<geometry::multi_line_string<double>>())
+        {
+            geometry::line_string_vertex_adapter<double> va(line);
+            converter.apply(va, proc);
+        }
+    }
+    else if (type == geometry::geometry_types::MultiPolygon)
+    {
+        for (auto const& poly : geom.get<geometry::multi_polygon<double>>())
+        {
+            geometry::polygon_vertex_adapter<double> va(poly);
+            converter.apply(va, proc);
+        }
+    }
+}
+
+template <typename Processor>
+void apply_markers_multi(feature_impl const& feature, attributes const& vars,
+                         vertex_converter_type & converter, Processor & proc, symbolizer_base const& sym)
+{
+    auto const& geom = feature.get_geometry();
+    geometry::geometry_types type = geometry::geometry_type(geom);
+
+    if (type == geometry::geometry_types::Point
+        ||
+        type == geometry::geometry_types::LineString
+        ||
+        type == geometry::geometry_types::Polygon)
+    {
+        apply_markers_single(converter, proc, geom, type);
+    }
+    else
+    {
+
+        marker_multi_policy_enum multi_policy = get<marker_multi_policy_enum, keys::markers_multipolicy>(sym, feature, vars);
+        marker_placement_enum placement = get<marker_placement_enum, keys::markers_placement_type>(sym, feature, vars);
+
+        if (placement == MARKER_POINT_PLACEMENT &&
+            multi_policy == MARKER_WHOLE_MULTI)
+        {
+            geometry::point<double> pt;
+            // test if centroid is contained by bounding box
+            if (geometry::centroid(geom, pt) && converter.disp_.args_.bbox.contains(pt.x, pt.y))
+            {
+                // unset any clipping since we're now dealing with a point
+                converter.template unset<clip_poly_tag>();
+                geometry::point_vertex_adapter<double> va(pt);
+                converter.apply(va, proc);
+            }
+        }
+        else if ((placement == MARKER_POINT_PLACEMENT || placement == MARKER_INTERIOR_PLACEMENT) &&
+                 multi_policy == MARKER_LARGEST_MULTI)
+        {
+            // Only apply to path with largest envelope area
+            // TODO: consider using true area for polygon types
+            if (type == geometry::geometry_types::MultiPolygon)
+            {
+                geometry::multi_polygon<double> const& multi_poly = mapnik::util::get<geometry::multi_polygon<double> >(geom);
+                double maxarea = 0;
+                geometry::polygon<double> const* largest = 0;
+                for (geometry::polygon<double> const& poly : multi_poly)
+                {
+                    box2d<double> bbox = geometry::envelope(poly);
+                    double area = bbox.width() * bbox.height();
+                    if (area > maxarea)
+                    {
+                        maxarea = area;
+                        largest = &poly;
+                    }
+                }
+                if (largest)
+                {
+                    geometry::polygon_vertex_adapter<double> va(*largest);
+                    converter.apply(va, proc);
+                }
+            }
+            else
+            {
+                MAPNIK_LOG_WARN(marker_symbolizer) << "TODO: if you get here -> open an issue";
+            }
+        }
+        else
+        {
+            if (multi_policy != MARKER_EACH_MULTI && placement != MARKER_POINT_PLACEMENT)
+            {
+                MAPNIK_LOG_WARN(marker_symbolizer) << "marker_multi_policy != 'each' has no effect with marker_placement != 'point'";
+            }
+            if (type == geometry::geometry_types::GeometryCollection)
+            {
+                for (auto const& g : geom.get<geometry::geometry_collection<double>>())
+                {
+                    apply_markers_single(converter, proc, g, geometry::geometry_type(g));
+                }
+            }
+            else
+            {
+                apply_markers_single(converter, proc, geom, type);
+            }
+        }
+    }
+}
+template void apply_markers_multi<vector_dispatch_type>(feature_impl const& feature, attributes const& vars,
+                                                        vertex_converter_type & converter, vector_dispatch_type & proc,
+                                                        symbolizer_base const& sym);
+
+template void apply_markers_multi<raster_dispatch_type>(feature_impl const& feature, attributes const& vars,
+                                                        vertex_converter_type & converter, raster_dispatch_type & proc,
+                                                        symbolizer_base const& sym);
 
 } // end namespace mapnik

src/renderer_common/render_markers_symbolizer.cpp

@@ -20,9 +20,9 @@
  *
  *****************************************************************************/
 
+#include <mapnik/label_collision_detector.hpp>
 #include <mapnik/svg/svg_storage.hpp>
 #include <mapnik/svg/svg_path_adapter.hpp>
-#include <mapnik/vertex_converters.hpp>
 #include <mapnik/marker_cache.hpp>
 #include <mapnik/marker_helpers.hpp>
 #include <mapnik/geometry_type.hpp>
@@ -39,14 +39,6 @@ struct render_marker_symbolizer_visitor
     using vector_dispatch_type = vector_markers_dispatch<Detector>;
     using raster_dispatch_type = raster_markers_dispatch<Detector>;
 
-    using vertex_converter_type = vertex_converter<clip_line_tag,
-                                                   clip_poly_tag,
-                                                   transform_tag,
-                                                   affine_transform_tag,
-                                                   simplify_tag,
-                                                   smooth_tag,
-                                                   offset_transform_tag>;
-
     render_marker_symbolizer_visitor(std::string const& filename,
                                      markers_symbolizer const& sym,
                                      mapnik::feature_impl & feature,
@@ -80,6 +72,15 @@ struct render_marker_symbolizer_visitor
         if (transform) evaluate_transform(geom_tr, feature_, common_.vars_, *transform, common_.scale_factor_);
         agg::trans_affine image_tr = agg::trans_affine_scaling(common_.scale_factor_);
 
+         // converter
+        vertex_converter_type converter(clip_box_,
+                                        sym_,
+                                        common_.t_,
+                                        prj_trans_,
+                                        geom_tr,
+                                        feature_,
+                                        common_.vars_,
+                                        common_.scale_factor_);
         boost::optional<svg_path_ptr> const& stock_vector_marker = mark.get_data();
 
         // special case for simple ellipse markers
@@ -107,14 +108,7 @@ struct render_marker_symbolizer_visitor
                                                      snap_to_pixels,
                                                      renderer_context_);
 
-            vertex_converter_type converter(clip_box_,
+
-                                            sym_,
-                                            common_.t_,
-                                            prj_trans_,
-                                            geom_tr,
-                                            feature_,
-                                            common_.vars_,
-                                            common_.scale_factor_);
             if (clip)
             {
                 geometry::geometry_types type = geometry::geometry_type(feature_.get_geometry());
@@ -152,15 +146,6 @@ struct render_marker_symbolizer_visitor
                                                      common_.vars_,
                                                      snap_to_pixels,
                                                      renderer_context_);
-
-            vertex_converter_type converter(clip_box_,
-                                            sym_,
-                                            common_.t_,
-                                            prj_trans_,
-                                            geom_tr,
-                                            feature_,
-                                            common_.vars_,
-                                            common_.scale_factor_);
             if (clip)
             {
                 geometry::geometry_types type = geometry::geometry_type(feature_.get_geometry());

src/unicode.cpp

@@ -60,4 +60,12 @@ transcoder::~transcoder()
 {
     if (conv_) ucnv_close(conv_);
 }
+
+
+void to_utf8(mapnik::value_unicode_string const& input, std::string & target)
+{
+    target.clear(); // mimic previous target.assign(...) semantics
+    input.toUTF8String(target); // this appends to target
+}
+
 }

src/value.cpp

@@ -0,0 +1,921 @@
+/*****************************************************************************
+ *
+ * This file is part of Mapnik (c++ mapping toolkit)
+ *
+ * Copyright (C) 2016 Artem Pavlenko
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+ *
+ *****************************************************************************/
+
+// mapnik
+#include <mapnik/value.hpp>
+#include <mapnik/value_types.hpp>
+#include <mapnik/util/conversions.hpp>
+
+// stl
+#include <cmath>
+#include <string>
+#include <type_traits>
+// icu
+#include <unicode/unistr.h>
+#include <unicode/ustring.h>
+
+namespace mapnik {
+
+namespace detail {
+
+namespace {
+template <typename T, typename U>
+struct both_arithmetic : std::integral_constant<bool,
+                                                std::is_arithmetic<T>::value &&
+                                                std::is_arithmetic<U>::value> {};
+
+struct equals
+{
+    static bool apply(value_null, value_unicode_string const& rhs)
+    {
+        return false;
+    }
+
+    template <typename T>
+    static auto apply(T const& lhs, T const& rhs)
+        -> decltype(lhs == rhs)
+    {
+        return lhs == rhs;
+    }
+};
+
+struct not_equal
+{
+    // back compatibility shim to equate empty string with null for != test
+    // https://github.com/mapnik/mapnik/issues/1859
+    // TODO - consider removing entire specialization at Mapnik 3.1.x
+    static bool apply(value_null, value_unicode_string const& rhs)
+    {
+        if (rhs.isEmpty()) return false;
+        return true;
+    }
+
+    template <typename T>
+    static auto apply(T const& lhs, T const& rhs)
+        -> decltype(lhs != rhs)
+    {
+        return lhs != rhs;
+    }
+};
+
+struct greater_than
+{
+    static bool apply(value_null, value_unicode_string const& rhs)
+    {
+        return false;
+    }
+
+    template <typename T>
+    static auto apply(T const& lhs, T const& rhs)
+        -> decltype(lhs > rhs)
+    {
+        return lhs > rhs;
+    }
+};
+
+struct greater_or_equal
+{
+    static bool apply(value_null, value_unicode_string const& rhs)
+    {
+        return false;
+    }
+
+    template <typename T>
+    static auto apply(T const& lhs, T const& rhs)
+        -> decltype(lhs >= rhs)
+    {
+        return lhs >= rhs;
+    }
+};
+
+struct less_than
+{
+    static bool apply(value_null, value_unicode_string const& rhs)
+    {
+        return false;
+    }
+
+    template <typename T>
+    static auto apply(T const& lhs, T const& rhs)
+        -> decltype(lhs < rhs)
+    {
+        return lhs < rhs;
+    }
+};
+
+struct less_or_equal
+{
+    static bool apply(value_null, value_unicode_string const& rhs)
+    {
+        return false;
+    }
+
+    template <typename T>
+    static auto apply(T const& lhs, T const& rhs)
+        -> decltype(lhs <= rhs)
+    {
+        return lhs <= rhs;
+    }
+};
+}
+
+template <typename Op, bool default_result>
+struct comparison
+{
+    // special case for unicode_strings (fixes MSVC C4800)
+    bool operator()(value_unicode_string const& lhs,
+                    value_unicode_string const& rhs) const
+    {
+        return Op::apply(lhs, rhs) ? true : false;
+    }
+
+    //////////////////////////////////////////////////////////////////////////
+    // special case for unicode_string and value_null
+    //////////////////////////////////////////////////////////////////////////
+
+    bool operator()(value_null const& lhs, value_unicode_string const& rhs) const
+    {
+        return Op::apply(lhs, rhs);
+    }
+    //////////////////////////////////////////////////////////////////////////
+
+    // same types
+    template <typename T>
+    bool operator()(T lhs, T rhs) const
+    {
+        return Op::apply(lhs, rhs);
+    }
+
+    // both types are arithmetic - promote to the common type
+    template <typename T, typename U, typename std::enable_if<both_arithmetic<T, U>::value, int>::type = 0>
+    bool operator()(T const& lhs, U const& rhs) const
+    {
+        using common_type = typename std::common_type<T, U>::type;
+        return Op::apply(static_cast<common_type>(lhs), static_cast<common_type>(rhs));
+    }
+
+    //
+    template <typename T, typename U, typename std::enable_if<!both_arithmetic<T, U>::value, int>::type = 0>
+    bool operator()(T const& lhs, U const& rhs) const
+    {
+        return default_result;
+    }
+};
+
+template <typename V>
+struct add
+{
+    using value_type = V;
+    value_type operator()(value_unicode_string const& lhs,
+                          value_unicode_string const& rhs) const
+    {
+        return lhs + rhs;
+    }
+
+    value_type operator()(value_null const& lhs,
+                          value_null const& rhs) const
+    {
+        return lhs;
+    }
+
+    value_type operator()(value_unicode_string const& lhs, value_null) const
+    {
+        return lhs;
+    }
+
+    value_type operator()(value_null, value_unicode_string const& rhs) const
+    {
+        return rhs;
+    }
+
+    template <typename L>
+    value_type operator()(L const& lhs, value_null const&) const
+    {
+        return lhs;
+    }
+
+    template <typename R>
+    value_type operator()(value_null const&, R const& rhs) const
+    {
+        return rhs;
+    }
+
+    template <typename L>
+    value_type operator()(L const& lhs, value_unicode_string const& rhs) const
+    {
+        std::string val;
+        if (util::to_string(val, lhs))
+            return value_unicode_string(val.c_str()) + rhs;
+        return rhs;
+    }
+
+    template <typename R>
+    value_type operator()(value_unicode_string const& lhs, R const& rhs) const
+    {
+        std::string val;
+        if (util::to_string(val, rhs))
+            return lhs + value_unicode_string(val.c_str());
+        return lhs;
+    }
+
+    template <typename T1, typename T2>
+    value_type operator()(T1 const& lhs, T2 const& rhs) const
+    {
+        return typename std::common_type<T1, T2>::type{lhs + rhs};
+    }
+
+    value_type operator()(value_bool lhs, value_bool rhs) const
+    {
+        return value_integer(lhs + rhs);
+    }
+};
+
+template <typename V>
+struct sub
+{
+    using value_type = V;
+
+    value_type operator()(value_null const& lhs,
+                          value_null const& rhs) const
+    {
+        return lhs;
+    }
+
+    value_type operator()(value_null, value_unicode_string const& rhs) const
+    {
+        return rhs;
+    }
+    value_type operator()(value_unicode_string const& lhs, value_null) const
+    {
+        return lhs;
+    }
+
+    template <typename R>
+    value_type operator()(value_unicode_string const& lhs, R const&) const
+    {
+        return lhs;
+    }
+
+    template <typename L>
+    value_type operator()(L const&, value_unicode_string const& rhs) const
+    {
+        return rhs;
+    }
+
+    template <typename L>
+    value_type operator()(L const& lhs, value_null const&) const
+    {
+        return lhs;
+    }
+
+    template <typename R>
+    value_type operator()(value_null const&, R const& rhs) const
+    {
+        return rhs;
+    }
+
+    template <typename T>
+    value_type operator()(T lhs, T rhs) const
+    {
+        return lhs - rhs;
+    }
+
+    value_type operator()(value_unicode_string const&,
+                          value_unicode_string const&) const
+    {
+        return value_type();
+    }
+
+    template <typename T1, typename T2>
+    value_type operator()(T1 const& lhs, T2 const& rhs) const
+    {
+        return typename std::common_type<T1, T2>::type{lhs - rhs};
+    }
+
+    value_type operator()(value_bool lhs, value_bool rhs) const
+    {
+        return value_integer(lhs - rhs);
+    }
+};
+
+template <typename V>
+struct mult
+{
+    using value_type = V;
+
+    value_type operator()(value_null const& lhs,
+                          value_null const& rhs) const
+    {
+        return lhs;
+    }
+
+    value_type operator()(value_unicode_string const& lhs, value_null) const
+    {
+        return lhs;
+    }
+
+    value_type operator()(value_null, value_unicode_string const& rhs) const
+    {
+        return rhs;
+    }
+
+    template <typename L>
+    value_type operator()(L const& lhs, value_null const&) const
+    {
+        return lhs;
+    }
+
+    template <typename R>
+    value_type operator()(value_null const&, R const& rhs) const
+    {
+        return rhs;
+    }
+
+    template <typename R>
+    value_type operator()(value_unicode_string const& lhs, R const&) const
+    {
+        return lhs;
+    }
+
+    template <typename L>
+    value_type operator()(L const&, value_unicode_string const& rhs) const
+    {
+        return rhs;
+    }
+
+    template <typename T>
+    value_type operator()(T lhs, T rhs) const
+    {
+        return lhs * rhs;
+    }
+
+    value_type operator()(value_unicode_string const&,
+                          value_unicode_string const&) const
+    {
+        return value_type();
+    }
+
+    template <typename T1, typename T2>
+    value_type operator()(T1 const& lhs, T2 const& rhs) const
+    {
+        return typename std::common_type<T1, T2>::type{lhs * rhs};
+    }
+
+    value_type operator()(value_bool lhs, value_bool rhs) const
+    {
+        return value_integer(lhs * rhs);
+    }
+};
+
+template <typename V>
+struct div
+{
+    using value_type = V;
+
+    value_type operator()(value_null const& lhs,
+                          value_null const& rhs) const
+    {
+        return lhs;
+    }
+
+    value_type operator()(value_unicode_string const& lhs, value_null) const
+    {
+        return lhs;
+    }
+
+    value_type operator()(value_null, value_unicode_string const& rhs) const
+    {
+        return rhs;
+    }
+
+    template <typename L>
+    value_type operator()(L const& lhs, value_null const&) const
+    {
+        return lhs;
+    }
+
+    template <typename R>
+    value_type operator()(value_null const&, R const& rhs) const
+    {
+        return rhs;
+    }
+
+    template <typename T>
+    value_type operator()(T lhs, T rhs) const
+    {
+        if (rhs == 0) return value_type();
+        return lhs / rhs;
+    }
+
+    value_type operator()(value_bool lhs, value_bool rhs) const
+    {
+        if (rhs == 0) return lhs;
+        return value_integer(lhs) / value_integer(rhs);
+    }
+
+    value_type operator()(value_unicode_string const&,
+                          value_unicode_string const&) const
+    {
+        return value_type();
+    }
+
+    template <typename R>
+    value_type operator()(value_unicode_string const& lhs, R const&) const
+    {
+        return lhs;
+    }
+
+    template <typename L>
+    value_type operator()(L const&, value_unicode_string const& rhs) const
+    {
+        return rhs;
+    }
+
+    template <typename T1, typename T2>
+    value_type operator()(T1 const& lhs, T2 const& rhs) const
+    {
+        if (rhs == 0) return value_type();
+        using common_type = typename std::common_type<T1, T2>::type;
+        return common_type(lhs) / common_type(rhs);
+    }
+};
+
+template <typename V>
+struct mod
+{
+    using value_type = V;
+
+    template <typename T1, typename T2>
+    value_type operator()(T1 const& lhs, T2 const&) const
+    {
+        return lhs;
+    }
+
+    template <typename T>
+    value_type operator()(T lhs, T rhs) const
+    {
+        return lhs % rhs;
+    }
+
+    value_type operator()(value_unicode_string const&,
+                          value_unicode_string const&) const
+    {
+        return value_type();
+    }
+
+    value_type operator()(value_bool,
+                          value_bool) const
+    {
+        return false;
+    }
+
+    value_type operator()(value_double lhs, value_integer rhs) const
+    {
+        return std::fmod(lhs, static_cast<value_double>(rhs));
+    }
+
+    value_type operator()(value_integer lhs, value_double rhs) const
+    {
+        return std::fmod(static_cast<value_double>(lhs), rhs);
+    }
+
+    value_type operator()(value_double lhs, value_double rhs) const
+    {
+        return std::fmod(lhs, rhs);
+    }
+};
+
+template <typename V>
+struct negate
+{
+    using value_type = V;
+
+    template <typename T>
+    value_type operator()(T val) const
+    {
+        return -val;
+    }
+
+    value_type operator()(value_null val) const
+    {
+        return val;
+    }
+
+    value_type operator()(value_bool val) const
+    {
+        return val ? value_integer(-1) : value_integer(0);
+    }
+
+    value_type operator()(value_unicode_string const&) const
+    {
+        return value_type();
+    }
+};
+
+// converters
+template <typename T>
+struct convert
+{
+};
+
+template <>
+struct convert<value_bool>
+{
+    value_bool operator()(value_bool val) const
+    {
+        return val;
+    }
+
+    value_bool operator()(value_unicode_string const& ustr) const
+    {
+        return !ustr.isEmpty();
+    }
+
+    value_bool operator()(value_null const&) const
+    {
+        return false;
+    }
+
+    template <typename T>
+    value_bool operator()(T val) const
+    {
+        return val > 0 ? true : false;
+    }
+};
+
+template <>
+struct convert<value_double>
+{
+    value_double operator()(value_double val) const
+    {
+        return val;
+    }
+
+    value_double operator()(value_integer val) const
+    {
+        return static_cast<value_double>(val);
+    }
+
+    value_double operator()(value_bool val) const
+    {
+        return static_cast<value_double>(val);
+    }
+
+    value_double operator()(std::string const& val) const
+    {
+        value_double result;
+        if (util::string2double(val, result))
+            return result;
+        return 0;
+    }
+
+    value_double operator()(value_unicode_string const& val) const
+    {
+        std::string utf8;
+        val.toUTF8String(utf8);
+        return operator()(utf8);
+    }
+
+    value_double operator()(value_null const&) const
+    {
+        return 0.0;
+    }
+};
+
+template <>
+struct convert<value_integer>
+{
+    value_integer operator()(value_integer val) const
+    {
+        return val;
+    }
+
+    value_integer operator()(value_double val) const
+    {
+        return static_cast<value_integer>(rint(val));
+    }
+
+    value_integer operator()(value_bool val) const
+    {
+        return static_cast<value_integer>(val);
+    }
+
+    value_integer operator()(std::string const& val) const
+    {
+        value_integer result;
+        if (util::string2int(val, result))
+            return result;
+        return value_integer(0);
+    }
+
+    value_integer operator()(value_unicode_string const& val) const
+    {
+        std::string utf8;
+        val.toUTF8String(utf8);
+        return operator()(utf8);
+    }
+
+    value_integer operator()(value_null const&) const
+    {
+        return value_integer(0);
+    }
+};
+
+template <>
+struct convert<std::string>
+{
+    template <typename T>
+    std::string operator()(T val) const
+    {
+        std::string str;
+        util::to_string(str, val);
+        return str;
+    }
+
+    // specializations
+    std::string operator()(value_unicode_string const& val) const
+    {
+        std::string utf8;
+        val.toUTF8String(utf8);
+        return utf8;
+    }
+
+    std::string operator()(value_double val) const
+    {
+        std::string str;
+        util::to_string(str, val); // TODO set precision(16)
+        return str;
+    }
+
+    std::string operator()(value_bool val) const
+    {
+        return val ? "true" : "false";
+    }
+
+    std::string operator()(value_null const&) const
+    {
+        return std::string();
+    }
+};
+
+struct to_unicode_impl
+{
+
+    template <typename T>
+    value_unicode_string operator()(T val) const
+    {
+        std::string str;
+        util::to_string(str, val);
+        return value_unicode_string(str.c_str());
+    }
+
+    // specializations
+    value_unicode_string const& operator()(value_unicode_string const& val) const
+    {
+        return val;
+    }
+
+    value_unicode_string operator()(value_double val) const
+    {
+        std::string str;
+        util::to_string(str, val);
+        return value_unicode_string(str.c_str());
+    }
+
+    value_unicode_string operator()(value_bool val) const
+    {
+        return value_unicode_string(val ? "true" : "false");
+    }
+
+    value_unicode_string operator()(value_null const&) const
+    {
+        return value_unicode_string();
+    }
+};
+
+struct to_expression_string_impl
+{
+    struct EscapingByteSink : U_NAMESPACE_QUALIFIER ByteSink
+    {
+        std::string dest_;
+        char quote_;
+
+        explicit EscapingByteSink(char quote)
+            : quote_(quote)
+        {
+        }
+
+        virtual void Append(const char* data, int32_t n)
+        {
+            // reserve enough room to hold the appended chunk and quotes;
+            // if another chunk follows, or any character needs escaping,
+            // the string will grow naturally
+            if (dest_.empty())
+            {
+                dest_.reserve(2 + static_cast<std::size_t>(n));
+                dest_.append(1, quote_);
+            }
+            else
+            {
+                dest_.reserve(dest_.size() + n + 1);
+            }
+
+            for (auto end = data + n; data < end; ++data)
+            {
+                if (*data == '\\' || *data == quote_)
+                    dest_.append(1, '\\');
+                dest_.append(1, *data);
+            }
+        }
+
+        virtual void Flush()
+        {
+            if (dest_.empty())
+                dest_.append(2, quote_);
+            else
+                dest_.append(1, quote_);
+        }
+    };
+
+    explicit to_expression_string_impl(char quote = '\'')
+        : quote_(quote) {}
+
+    std::string operator()(value_unicode_string const& val) const
+    {
+        EscapingByteSink sink(quote_);
+        val.toUTF8(sink);
+        return sink.dest_;
+    }
+
+    std::string operator()(value_integer val) const
+    {
+        std::string output;
+        util::to_string(output, val);
+        return output;
+    }
+
+    std::string operator()(value_double val) const
+    {
+        std::string output;
+        util::to_string(output, val); // TODO precision(16)
+        return output;
+    }
+
+    std::string operator()(value_bool val) const
+    {
+        return val ? "true" : "false";
+    }
+
+    std::string operator()(value_null const&) const
+    {
+        return "null";
+    }
+
+    const char quote_;
+};
+
+} // ns detail
+
+namespace value_adl_barrier {
+
+bool value::operator==(value const& other) const
+{
+    return util::apply_visitor(detail::comparison<detail::equals, false>(), *this, other);
+}
+
+bool value::operator!=(value const& other) const
+{
+    return util::apply_visitor(detail::comparison<detail::not_equal, true>(), *this, other);
+}
+
+bool value::operator>(value const& other) const
+{
+    return util::apply_visitor(detail::comparison<detail::greater_than, false>(), *this, other);
+}
+
+bool value::operator>=(value const& other) const
+{
+    return util::apply_visitor(detail::comparison<detail::greater_or_equal, false>(), *this, other);
+}
+
+bool value::operator<(value const& other) const
+{
+    return util::apply_visitor(detail::comparison<detail::less_than, false>(), *this, other);
+}
+
+bool value::operator<=(value const& other) const
+{
+    return util::apply_visitor(detail::comparison<detail::less_or_equal, false>(), *this, other);
+}
+
+value value::operator-() const
+{
+    return util::apply_visitor(detail::negate<value>(), *this);
+}
+
+value_bool value::to_bool() const
+{
+    return util::apply_visitor(detail::convert<value_bool>(), *this);
+}
+
+std::string value::to_expression_string(char quote) const
+{
+    return util::apply_visitor(detail::to_expression_string_impl(quote), *this);
+}
+
+std::string value::to_string() const
+{
+    return util::apply_visitor(detail::convert<std::string>(), *this);
+}
+
+value_unicode_string value::to_unicode() const
+{
+    return util::apply_visitor(detail::to_unicode_impl(), *this);
+}
+
+value_double value::to_double() const
+{
+    return util::apply_visitor(detail::convert<value_double>(), *this);
+}
+
+value_integer value::to_int() const
+{
+    return util::apply_visitor(detail::convert<value_integer>(), *this);
+}
+
+bool value::is_null() const
+{
+    return util::apply_visitor(mapnik::detail::is_null_visitor(), *this);
+}
+
+template <>
+value_double value::convert() const
+{
+    return util::apply_visitor(detail::convert<value_double>(), *this);
+}
+
+template <>
+value_integer value::convert() const
+{
+    return util::apply_visitor(detail::convert<value_integer>(), *this);
+}
+
+template <>
+value_bool value::convert() const
+{
+    return util::apply_visitor(detail::convert<value_bool>(), *this);
+}
+
+template <>
+std::string value::convert() const
+{
+    return util::apply_visitor(detail::convert<std::string>(), *this);
+}
+
+//
+value operator+(value const& p1, value const& p2)
+{
+    return value(util::apply_visitor(detail::add<value>(), p1, p2));
+}
+
+value operator-(value const& p1, value const& p2)
+{
+    return value(util::apply_visitor(detail::sub<value>(), p1, p2));
+}
+
+value operator*(value const& p1, value const& p2)
+{
+    return value(util::apply_visitor(detail::mult<value>(), p1, p2));
+}
+
+value operator/(value const& p1, value const& p2)
+{
+    return value(util::apply_visitor(detail::div<value>(), p1, p2));
+}
+
+value operator%(value const& p1, value const& p2)
+{
+    return value(util::apply_visitor(detail::mod<value>(), p1, p2));
+}
+
+} // namespace value_adl_barrier
+} // namespace mapnik

src/vertex_adapters.cpp

@@ -0,0 +1,213 @@
+/*****************************************************************************
+ *
+ * This file is part of Mapnik (c++ mapping toolkit)
+ *
+ * Copyright (C) 2016 Artem Pavlenko
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+ *
+ *****************************************************************************/
+
+#include <mapnik/vertex_adapters.hpp>
+#include <mapnik/geometry.hpp>
+#include <mapnik/geometry_types.hpp>
+#include <mapnik/vertex.hpp>
+
+namespace mapnik { namespace geometry {
+
+// point adapter
+template <typename T>
+point_vertex_adapter<T>::point_vertex_adapter(point<T> const& pt)
+    : pt_(pt),
+      first_(true) {}
+
+template <typename T>
+unsigned point_vertex_adapter<T>::vertex(value_type * x, value_type * y) const
+{
+    if (first_)
+    {
+        *x = pt_.x;
+        *y = pt_.y;
+        first_ = false;
+        return mapnik::SEG_MOVETO;
+    }
+    return mapnik::SEG_END;
+}
+
+template <typename T>
+void point_vertex_adapter<T>::rewind(unsigned) const
+{
+    first_ = true;
+}
+
+template <typename T>
+geometry_types point_vertex_adapter<T>::type () const
+{
+    return geometry_types::Point;
+}
+
+// line_string adapter
+template <typename T>
+line_string_vertex_adapter<T>::line_string_vertex_adapter(line_string<T> const& line)
+    : line_(line),
+      current_index_(0),
+      end_index_(line.size())
+{}
+
+template <typename T>
+unsigned line_string_vertex_adapter<T>::vertex(value_type * x, value_type * y) const
+{
+    if (current_index_ != end_index_)
+    {
+        point<T> const& coord = line_[current_index_++];
+        *x = coord.x;
+        *y = coord.y;
+        if (current_index_ == 1)
+        {
+            return mapnik::SEG_MOVETO;
+        }
+        else
+        {
+            return mapnik::SEG_LINETO;
+        }
+    }
+    return mapnik::SEG_END;
+}
+
+template <typename T>
+void line_string_vertex_adapter<T>::rewind(unsigned) const
+{
+    current_index_ = 0;
+}
+
+template <typename T>
+geometry_types line_string_vertex_adapter<T>::type() const
+{
+    return geometry_types::LineString;
+}
+
+template <typename T>
+polygon_vertex_adapter<T>::polygon_vertex_adapter(polygon<T> const& poly)
+    : poly_(poly),
+      rings_itr_(0),
+      rings_end_(poly_.interior_rings.size() + 1),
+      current_index_(0),
+      end_index_((rings_itr_ < rings_end_) ? poly_.exterior_ring.size() : 0),
+      start_loop_(true) {}
+
+template <typename T>
+void polygon_vertex_adapter<T>::rewind(unsigned) const
+{
+    rings_itr_ = 0;
+    rings_end_ = poly_.interior_rings.size() + 1;
+    current_index_ = 0;
+    end_index_ = (rings_itr_ < rings_end_) ? poly_.exterior_ring.size() : 0;
+    start_loop_ = true;
+}
+template <typename T>
+unsigned polygon_vertex_adapter<T>::vertex(value_type * x, value_type * y) const
+{
+    if (rings_itr_ == rings_end_)
+    {
+        return mapnik::SEG_END;
+    }
+    if (current_index_ < end_index_)
+    {
+        point<T> const& coord = (rings_itr_ == 0) ?
+            poly_.exterior_ring[current_index_++] : poly_.interior_rings[rings_itr_- 1][current_index_++];
+        *x = coord.x;
+        *y = coord.y;
+        if (start_loop_)
+        {
+            start_loop_= false;
+            return mapnik::SEG_MOVETO;
+        }
+        if (current_index_ == end_index_)
+        {
+            *x = 0;
+            *y = 0;
+            return mapnik::SEG_CLOSE;
+        }
+        return mapnik::SEG_LINETO;
+    }
+    else if (++rings_itr_ != rings_end_)
+    {
+        current_index_ = 0;
+        end_index_ = poly_.interior_rings[rings_itr_ - 1].size();
+        point<T> const& coord = poly_.interior_rings[rings_itr_ - 1][current_index_++];
+        *x = coord.x;
+        *y = coord.y;
+        return mapnik::SEG_MOVETO;
+    }
+    return mapnik::SEG_END;
+}
+
+template <typename T>
+geometry_types polygon_vertex_adapter<T>::type () const
+{
+    return geometry_types::Polygon;
+}
+
+// ring adapter
+template <typename T>
+ring_vertex_adapter<T>::ring_vertex_adapter(linear_ring<T> const& ring)
+    : ring_(ring),
+      current_index_(0),
+      end_index_(ring_.size()),
+      start_loop_(true) {}
+
+template <typename T>
+void ring_vertex_adapter<T>::rewind(unsigned) const
+{
+    current_index_ = 0;
+    end_index_ = ring_.size();
+    start_loop_ = true;
+}
+
+template <typename T>
+unsigned ring_vertex_adapter<T>::vertex(value_type * x, value_type * y) const
+{
+    if (current_index_ < end_index_)
+    {
+        auto const& coord = ring_[current_index_++];
+        *x = coord.x;
+        *y = coord.y;
+        if (start_loop_)
+        {
+            start_loop_= false;
+            return mapnik::SEG_MOVETO;
+        }
+        if (current_index_ == end_index_)
+        {
+            *x = 0;
+            *y = 0;
+            return mapnik::SEG_CLOSE;
+        }
+        return mapnik::SEG_LINETO;
+    }
+    return mapnik::SEG_END;
+}
+template <typename T>
+geometry_types ring_vertex_adapter<T>::type () const
+{
+    return geometry_types::Polygon;
+}
+
+template struct point_vertex_adapter<double>;
+template struct line_string_vertex_adapter<double>;
+template struct polygon_vertex_adapter<double>;
+template struct ring_vertex_adapter<double>;
+
+}}

test/data-visual

@@ -1 +1 @@
-Subproject commit f5623b5a312c58cd9c1926008fcf114944c769a3
+Subproject commit 93e70a5ae91b2bd4b8fc3912a7e6e4b017021b2b

test/unit/core/expressions_test.cpp

@@ -158,6 +158,16 @@ TEST_CASE("expressions")
     TRY_CHECK(eval(" [int] > 100 and [int] gt 100.0 and [double] < 2 and [double] lt 2.0 ") == true);
     TRY_CHECK(eval(" [int] >= 123 and [int] ge 123.0 and [double] <= 1.23456 and [double] le 1.23456 ") == true);
 
+    // empty string/null equality
+    TRY_CHECK(eval("[null] = null") == true);
+    TRY_CHECK(eval("[null] != null") == false);
+    TRY_CHECK(eval("[null] = ''") == false);
+    ///////////////////// ref: https://github.com/mapnik/mapnik/issues/1859
+    TRY_CHECK(eval("[null] != ''") == false); // back compatible - will be changed in 3.1.x
+    //////////////////////
+    TRY_CHECK(eval("'' = [null]") == false);
+    TRY_CHECK(eval("'' != [null]") == true);
+
     // regex
     // replace
     TRY_CHECK(eval(" [foo].replace('(\\B)|( )','$1 ') ") == tr.transcode("b a r"));

test/unit/datasource/geojson.cpp

@@ -663,20 +663,8 @@ TEST_CASE("geojson") {
                     auto ds = mapnik::datasource_cache::instance().create(params);
                     REQUIRE(bool(ds));
                     auto fields = ds->get_descriptor().get_descriptors();
-                    // TODO: this combo (cache_features==false and create_index==false)
+                    std::initializer_list<std::string> names = {"one", "two"};
-                    // exposes the case where not all field names are reported, which should
+                    REQUIRE_FIELD_NAMES(fields, names);
-                    // ideally be fixed, but how?
-                    if (cache_features == false && create_index == false) 
-                    {
-                        std::initializer_list<std::string> names = {"one"};
-                        REQUIRE_FIELD_NAMES(fields, names);
-                    } 
-                    else
-                    {
-                        std::initializer_list<std::string> names = {"one", "two"};
-                        REQUIRE_FIELD_NAMES(fields, names);
-                    }
-
                 }
                 // cleanup
                 if (create_index && mapnik::util::exists(filename + ".index"))