9.17. Transformation Function Overview¶

9.17.1. Type Conversions¶

::int or ::integer
::long
::float
::double
::boolean
::r
stringToInt or stringToInteger
stringToLong
stringToFloat
stringToDouble
stringToBoolean
intToBoolean

9.17.2. String Functions¶

strip
stripPrefix
stripSuffix
stripQuotes
replace
removeChars
length
trim
capitalize
lowercase
regexReplace
concatenate
substring
toString
emptyToNull
printf

9.17.3. Date Functions¶

now
date
dateTime
basicIsoDate
isoDate
isoLocalDate
basicDateTime
isoDateTime
isoLocalDateTime
isoOffsetDateTime
basicDateTimeNoMillis
dateHourMinuteSecondMillis
millisToDate
secsToDate
dateToString
dateToMillis

9.17.4. Geometry Functions¶

point
multipoint
linestring
multilinestring
polygon
multipolygon
geometrycollection
geometry
projectFrom

9.17.5. ID Functions¶

stringToBytes
md5
murmur3_32
murmur3_128
uuid
uuidZ3
uuidZ3Centroid
base64

9.17.6. Math Functions¶

add
subtract
multiply
divide
mean
min
max

9.17.7. List and Map Functions¶

list
mapValue
parseList
parseMap

9.17.8. Control Functions¶

try
withDefault
require

9.17.9. State Functions¶

inputFilePath
lineNo

9.17.10. Functions defined using scripting languages¶

You can define functions using scripting languages that support JSR-223. This is currently tested with JavaScript only as it is natively supported in all JREs via the Nashorn extension. To define a JavaScript function for use in the converter framework, create a file with the .js extension and the function definition as the contents of the file. For instance, if you have defined a function such as

function hello(s) {
   return "hello: " + s;
}

you can reference that function in a transform expression as js:hello($2)

9.17.11. Installing Custom Scripts¶

Custom scripting functions are made available to GeoMesa comamnd line tools or distributed (map-reduce) ingest via including them on the classpath or setting a system property.

For local usage, geomesa defines the system property geomesa.convert.scripts.path to be a colon-separated list of script files and/or directories containing scripts. This system property can be set when using the command line tools by setting the CUSTOM_JAVA_OPTS environmental variable:

CUSTOM_JAVA_OPTS="-Dgeomesa.convert.scripts.path=/path/to/script.js:/path/to/script-dir/"

A more resilient method of including custom scripts is to package them as a JAR or ZIP file and add it to the GEOMESA_EXTRA_CLASSPATHS environmental variable. If using maven you can simply package them in a folder under src/main/resources/geomesa-convert-scripts/ which will create a folder in your jar file named geomesa-convert-scripts with the scripts inside. You can manually create a jar with this folder as well. An easier way is often to package them as a zip archive with a folder similary named geomesa-convert-scripts inside the archive containing the scripts:

$ unzip -l /tmp/scripts.zip
Archive:  /tmp/scripts.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
        0  2017-03-09 11:33   geomesa-convert-scripts/
       42  2017-03-09 11:33   geomesa-convert-scripts/my-script.js
---------                     -------
       42                     2 files

For either zip or jar files add them to the extra classpaths in your environment to make them available for the tools or map-reduce ingest:

GEOMESA_EXTRA_CLASSPATHS="/path/to/my-scripts.jar:/tmp/scripts.zip"

A example of ingest with a scripts on the classpath is below:

GEOMESA_EXTRA_CLASSPATHS="/tmp/scripts.zip:/path/to/my-scripts.jar" bin/geomesa-accumulo ingest -u <user-name>
-p <password> -s <sft-name> -C <converter-name> -c geomesa.catalog hdfs://localhost:9000/data/example.csv

You can also verify the classpath is properly configured with the tools:

GEOMESA_EXTRA_CLASSPATHS="/tmp/scripts.zip:/path/to/my-scripts.jar" bin/geomesa-accumulo classpath

9.17.12. CQL Functions¶

Most of the basic CQL functions are available as transformations. To use one, invoke it like a regular function, prefixed with the cql namespace. For example, you can use the CQL buffer function to turn a point into a polygon:

cql:buffer($1, 2.0)

For more information on the various CQL functions, see the GeoServer filter function reference.

9.17.13. JSON/Avro Transformations¶

See JSON Transform Functions and Avro Transform Functions.

9.17.14. Enrichment Functions¶

The converter framework provides a mechanism for setting an attribute based on a lookup from a cache. The cache can be a literal cache in the system or in an external system such as Redis.

cacheLookup