9.17. Transformation Function Overview

9.17.1. Type Conversions

  • ::int or ::integer

  • ::long

  • ::float

  • ::double

  • ::boolean

  • ::r

  • toInt or toInteger

  • toLong

  • toFloat

  • toDouble

  • toBoolean

  • intToBoolean

9.17.2. String Functions

  • strip

  • stripPrefix

  • stripSuffix

  • stripQuotes

  • replace

  • removeChars

  • length

  • trim

  • capitalize

  • lowercase

  • regexReplace

  • concatenate

  • substring

  • toString

  • emptyToNull

  • printf

9.17.3. Date Functions

  • now

  • date

  • dateTime

  • basicIsoDate

  • isoDate

  • isoLocalDate

  • basicDateTime

  • isoDateTime

  • isoLocalDateTime

  • isoOffsetDateTime

  • basicDateTimeNoMillis

  • dateHourMinuteSecondMillis

  • millisToDate

  • secsToDate

  • dateToString

  • dateToMillis

9.17.4. Geometry Functions

  • point

  • pointM

  • multipoint

  • linestring

  • multilinestring

  • polygon

  • multipolygon

  • geometrycollection

  • geometry

  • projectFrom

9.17.5. ID Functions

  • stringToBytes

  • md5

  • murmur3_32

  • murmur3_64

  • murmurHash3

  • uuid

  • uuidZ3

  • uuidZ3Centroid

9.17.6. Math Functions

  • add

  • subtract

  • multiply

  • divide

  • mean

  • min

  • max

9.17.7. List and Map Functions

  • list

  • listItem

  • mapValue

  • parseList

  • parseMap

  • transformListItems

9.17.8. Encoding Functions

  • base64Encode

  • base64Decode

9.17.9. Control Functions

  • try

  • withDefault

  • require

9.17.10. State Functions

  • inputFilePath

  • lineNo

9.17.11. Functions defined using scripting languages

You can define functions using scripting languages that support JSR-223. This is currently tested with JavaScript only as it is natively supported in all JREs via the Nashorn extension. To define a JavaScript function for use in the converter framework, create a file with the .js extension and the function definition as the contents of the file. For instance, if you have defined a function such as

function hello(s) {
   return "hello: " + s;
}

you can reference that function in a transform expression as js:hello($2)

9.17.12. Installing Custom Scripts

Custom scripting functions are made available to GeoMesa comamnd line tools or distributed (map-reduce) ingest via including them on the classpath or setting a system property.

For local usage, geomesa defines the system property geomesa.convert.scripts.path to be a colon-separated list of script files and/or directories containing scripts. This system property can be set when using the command line tools by setting the CUSTOM_JAVA_OPTS environmental variable:

CUSTOM_JAVA_OPTS="-Dgeomesa.convert.scripts.path=/path/to/script.js:/path/to/script-dir/"

A more resilient method of including custom scripts is to package them as a JAR or ZIP file and add it to the GEOMESA_EXTRA_CLASSPATHS environmental variable. If using maven you can simply package them in a folder under src/main/resources/geomesa-convert-scripts/ which will create a folder in your jar file named geomesa-convert-scripts with the scripts inside. You can manually create a jar with this folder as well. An easier way is often to package them as a zip archive with a folder similary named geomesa-convert-scripts inside the archive containing the scripts:

$ unzip -l /tmp/scripts.zip
Archive:  /tmp/scripts.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
        0  2017-03-09 11:33   geomesa-convert-scripts/
       42  2017-03-09 11:33   geomesa-convert-scripts/my-script.js
---------                     -------
       42                     2 files

For either zip or jar files add them to the extra classpaths in your environment to make them available for the tools or map-reduce ingest:

GEOMESA_EXTRA_CLASSPATHS="/path/to/my-scripts.jar:/tmp/scripts.zip"

A example of ingest with a scripts on the classpath is below:

GEOMESA_EXTRA_CLASSPATHS="/tmp/scripts.zip:/path/to/my-scripts.jar" bin/geomesa-accumulo ingest -u <user-name>
-p <password> -s <sft-name> -C <converter-name> -c geomesa.catalog hdfs://localhost:9000/data/example.csv

You can also verify the classpath is properly configured with the tools:

GEOMESA_EXTRA_CLASSPATHS="/tmp/scripts.zip:/path/to/my-scripts.jar" bin/geomesa-accumulo classpath

9.17.13. CQL Functions

Most of the basic CQL functions are available as transformations. To use one, invoke it like a regular function, prefixed with the cql namespace. For example, you can use the CQL buffer function to turn a point into a polygon:

cql:buffer($1, 2.0)

For more information on the various CQL functions, see the GeoServer filter function reference.

9.17.14. JSON/Avro Transformations

See JSON Transform Functions and Avro Transform Functions.

9.17.15. Enrichment Functions

The converter framework provides a mechanism for setting an attribute based on a lookup from a cache. The cache can be a literal cache in the system or in an external system such as Redis.

  • cacheLookup