Notes, Tips and Tricks

Here you can find some of the notes I have taken as I study the code of Tangram ES. If you need to dig deep into the code-base, these notes may be useful for you.

Switching between MVT and GeoJSON tile sources

Press G.

main.cpp

case GLFW_KEY_G:
    static bool geoJSON = false;
    if (!geoJSON) {
        LOGS("Switching to GeoJSON data source");
        Tangram::queueSceneUpdate("sources.osm.type", "GeoJSON");
        Tangram::queueSceneUpdate("sources.osm.url", "https://vector.mapzen.com/osm/all/{z}/{x}/{y}.json");
    } else {
        LOGS("Switching to MVT data source");
        Tangram::queueSceneUpdate("sources.osm.type", "MVT");
        Tangram::queueSceneUpdate("sources.osm.url", "https://vector.mapzen.com/osm/all/{z}/{x}/{y}.mvt");
    }
    geoJSON = !geoJSON;
    Tangram::applySceneUpdates();
    break;

Data

Let's get started looking at the src/data directory. As you'd imagine, this is where we handle data--data for a tile, data sources, and the overall structure in which the data is organized.

tileData.h is the container for tile data. GeoJsonSource, MVTSource, and TopoJsonSource.

tileData.h is a collection of 3 structs.

struct Feature {
    Feature() {}
    Feature(int32_t _sourceId) { props.sourceId = _sourceId; }

    GeometryType geometryType = GeometryType::polygons;

    std::vector<Point> points;
    std::vector<Line> lines;
    std::vector<Polygon> polygons;

    Properties props;
};

struct Layer {

    Layer(const std::string& _name) : name(_name) {}

    std::string name;

    std::vector<Feature> features;

};

struct TileData {

    std::vector<Layer> layers;

};

Inside all of these sources, we add to the list of layers in the tileData. For example, in geoJsonSource.cpp we do this:

for (auto layer = document.MemberBegin(); layer != document.MemberEnd(); ++layer) {
    if (GeoJson::isFeatureCollection(layer->value)) {
        tileData->layers.push_back(GeoJson::getLayer(layer->value, projFn, m_id));
        tileData->layers.back().name = layer->name.GetString();
    }
}

Looking at topoJsonSource.cpp, we also do basically the same thing:

for (auto layer = objects.MemberBegin(); layer != objects.MemberEnd(); ++layer) {
    tileData->layers.push_back(TopoJson::getLayer(layer, topology, m_id));
}

Since tileData is really just struct containers, the creation of actual tileData is found in utility classes, as you see above.

Creating a Layer

GeoJson::getLayer has a callback function fed to it called projFn. This function projects the LatLngs to points within the coordinate space of the tile. The exact same function is also used for TopoJson::getLayer.

BoundingBox tileBounds(_projection.TileBounds(task.tileId()));
glm::dvec2 tileOrigin = {tileBounds.min.x, tileBounds.max.y*-1.0};
double tileInverseScale = 1.0 / tileBounds.width();

const auto projFn = [&](glm::dvec2 _lonLat){
    glm::dvec2 tmp = _projection.LonLatToMeters(_lonLat);
    return Point {
        (tmp.x - tileOrigin.x) * tileInverseScale,
        (tmp.y - tileOrigin.y) * tileInverseScale,
         0
    };
};

This coordinate space, as you can see, is simple:

  (0.0, 1.0) ---------- (1.0, 1.0)
            |          |             N
            ^ +y       |          W <|> E
            |          |             S
            |    +x    |
  (0.0, 0.0) ----->---- (1.0, 0.0)

TopoJson::getLayer gets a topology object which has already had points within the topology projected.

Layers and Styles

Being that we have layers in tile data, we need to study how they are styled and rendered as specified in scene.yml.

TileData is created by TileTask in TileTask::process.

void TileTask::process(TileBuilder& _tileBuilder) {

    auto tileData = m_source->parse(*this, *_tileBuilder.scene().mapProjection());

    if (tileData) {
        m_tile = _tileBuilder.build(m_tileId, *tileData, *m_source);
    } else {
        cancel();
    }
}

TileBuilder

TileBuilder.h is the class where we build the actual tile from the tile data with the StyleBuilder.

TileBuilder::build is the primary member function we care about that returns a Tile.

std::shared_ptr<Tile> TileBuilder::build(TileID _tileID, const TileData& _tileData, const DataSource& _source) {

    auto tile = std::make_shared<Tile>(_tileID, *m_scene->mapProjection(), &_source);

    tile->initGeometry(m_scene->styles().size());

    m_styleContext.setKeywordZoom(_tileID.s);

    for (auto& builder : m_styleBuilder) {
        if (builder.second)
            builder.second->setup(*tile);
    }

    for (const auto& datalayer : m_scene->layers()) {

        if (datalayer.source() != _source.name()) { continue; }

        for (const auto& collection : _tileData.layers) {

            if (!collection.name.empty()) {
                const auto& dlc = datalayer.collections();
                bool layerContainsCollection =
                    std::find(dlc.begin(), dlc.end(), collection.name) != dlc.end();

                if (!layerContainsCollection) { continue; }
            }

            for (const auto& feat : collection.features) {
                m_ruleSet.apply(feat, datalayer, m_styleContext, *this);
            }
        }
    }

    for (auto& builder : m_styleBuilder) {
        tile->setMesh(builder.second->style(), builder.second->build());
    }

    return tile;
}

This build is done during TileTask::process.

Note that builder.second is getting the value (as opposed to the key) of the m_styleBuilder, which is a hash map implemented via a class called fastmap.

scene.yaml

The most general-purpose, and useful (in my opinion) style file we have is the Cinnabar style. The scene.yaml for Cinnabar is pretty long and extensive, and this is my main reference for seeing how to do in-depth styling.

Mapzen has good documentation explaining how the scene file works.

Loading scene.yml

scene.yaml is loaded by void loadScene(const char* _scenePath). This is called explicity as well as in void initialize(const char* _scenePath).

void loadScene(const char* _scenePath) {
    LOG("Loading scene file: %s", _scenePath);

    auto sceneString = stringFromFile(setResourceRoot(_scenePath).c_str(), PathType::resource);

    // Copy old scene
    auto scene = std::make_shared<Scene>(*m_scene);

    if (SceneLoader::loadScene(sceneString, *scene)) {
        setScene(scene);
    }
}

This non-class function in turn calls SceneLoader::loadScene, and then the resultant scene object is set. The scene.yaml is loaded in parse.cpp within the yaml-cpp submodule. The yaml-cpp project is maintained by Mapzen tangrams. The actual loading is making a new root node object of the YAML document. It appears that yaml-cpp is a DOM style parser.

Applying scene.yaml

Once we have loaded scene.yaml, we need to apply it.

bool SceneLoader::loadScene(const std::string& _sceneString, Scene& _scene) {

    Node& root = _scene.config();

    if (loadConfig(_sceneString, root)) {
        applyConfig(root, _scene);
        return true;
    }
    return false;
}

SceneLoader::applyConfig is a long member function that does all of the work of setting up the Scene object. This function is paricularly relevent, because you can see how the yaml file is read and used to setup the scene object. It goes through all of the top-level scene elements, and these elements are specified in the Scene File documentation.

Loading a Source

SceneLoader::loadSource is where we setup our data sources. The documentation states that data sources can be tiled or non-tiled. In specfic, we can see that this support only applies currently for GeoJSON. If we are to make an OSM_XML source type, we'll want to follow the same mechanism for accepting tiled and non-tiled OSM sources.

Label Placement

https://mapzen.com/documentation/tangram/Filters-Overview/#label_placement

Fetching Tiles

View

The View is an object that represents the state of the viewport of the map. The view contains the set of visible tiles, and we call View::getVisibleTiles to get this information.

tangram.h

tangram.cpp is the primary API, and this is where View::getVisibleTiles is called. There is a bool update(float _dt) function that is called in a tight loop by the main function of the application. In this function, we call:

m_tileManager->updateTileSets(viewState, m_view->getVisibleTiles());

In the OS X GLFW application, bool update is called within

while (keepRunning && !glfwWindowShouldClose(main_window)) {
    ...
    Tangram::update(delta);
    Tangram::render();
    ...
}

It is exposed to Android in the JNI.

JNIEXPORT bool JNICALL Java_com_mapzen_tangram_MapController_nativeUpdate(JNIEnv* jniEnv, jobject obj, jfloat dt) {
    return Tangram::update(dt);
}

TileManager

Singleton container of TileSets. TileManager is a singleton that maintains a set of Tiles based on the current view into the map.

The TileManager is instantiated one time in the setup of the app in tangram.cpp

m_tileManager = std::make_unique<TileManager>(*m_tileWorker);

updateTileSets

Since TileManager::updateTileSets is called repeatedly by Tangram::update, this is a good first member function to look at.

First, it loops through all of the tile sets and updates them. TileManager::updateTileSet is a long and involved function. Several activities include: unsetting proxies, making sure all visible tiles are in the TileSet, prioritizing the tile to be loaded based on the distance from the center of the map, and making sure the download requests does not exceed the limit of downloads.

Then, TileManager::loadTiles is called. Here, we create the tasks, and we call

if (tileSet.source->loadTileData(std::move(task), m_dataCallback)) {
    ...
}

DataSource::loadTileData is the actual place where the HTTP request is made. startUrlRequest is a platform wrapper function that has your specific platform make the HTTP request.

bool DataSource::loadTileData(std::shared_ptr<TileTask>&& _task, TileTaskCb _cb) {

    std::string url(constructURL(_task->tileId()));

    // lambda captured parameters are const by default, we want "task" (moved) to be non-const,
    // hence "mutable"
    // Refer: http://en.cppreference.com/w/cpp/language/lambda
    return startUrlRequest(url,
            [this, _cb, task = std::move(_task)](std::vector<char>&& rawData) mutable {
                this->onTileLoaded(std::move(rawData), std::move(task), _cb);
            });

}

TileWorker

TileWorker is the wrapper for a thread that does the work of fetching a tile. The worker is constructed with a TileBuilder and a std::thread. This is done once during the setup of the app in Tangram::initialize.

m_tileWorker = std::make_unique<TileWorker>(MAX_WORKERS);

A lot happens in TileWorker::run. It takes the member m_mutex and creates a lock. It handles all of the tasks and removes them from the queue when necessary. There is a priority mechanism for removing the correct tile from the queue. Finally, it calls TileTask::process.

TODO: Figure out where TileWorker::run is called.

TileTask

TileTask::process takes TileBuilder and calls GeoJsonSource::parse.

GeoJsonSource

GeoJsonSource is constructed in SceneLoader::loadSource. This happens once.

GeoJsonSource::parse is the final stage where we create TileData from a TileTask. This is done for each and every tile.

results matching ""

    No results matching ""