Haystack / Project Haystack

What the Haystack API Actually Exposes

The Haystack REST API is a small fixed set of named operations over HTTP, and every one of them returns the same structure: a grid, meaning rows of tag/value records with metadata attached to the grid itself and to each column. Failures use that shape too. An error comes back as a grid whose metadata carries an err marker, a dis message and an errTrace tag, alongside an HTTP error status. A client that can parse one response can parse all of them, and the uniform shape makes the request failed a visibly different condition from the request worked and returned nothing, which is the distinction most integration debugging turns on.

Requests go out as GET with query parameters or POST with a grid body, and the encoding is negotiable. Zinc is the native text format, compact and fully typed; JSON exists for clients that would rather not write a parser; Trio and CSV exist for flat exchange. The formats op reports what a given server can read and write. The difference matters the moment someone exports a grid to CSV: Zinc and JSON carry Ref values, markers and unit-bearing numbers, and CSV flattens all three to text, so a round trip through a spreadsheet loses exactly the parts of the model that made it queryable.

Reads are expressed in Haystack filter syntax rather than SQL. A filter names tags that must all be present, so point and discharge and temp and sensor selects by combination rather than by name; paths walk relationships with ->, as in equipRef->siteRef->dis; values compare with equality and the usual relational operators (==, !=, <, , >=), and clauses combine with and, or and not. Tag names are case-sensitive and camelCase, there is no wildcard on tag names, and there is no full-text search over point names, so an untagged point is invisible to every filter no matter how descriptive its name is. The operations themselves are the short list below, and the ops op reports which of them a particular server actually implements.

• about: server name, product and module versions, project name and the server's own timezone. The cheapest call in the API and the right first one, because it answers whether anything is there and what version it is before any modeling question arises.
• ops: which operations this server implements. Implementations differ, so this is read rather than assumed from the specification.
• formats: which encodings the server can read and write, reported as mime types with read and write flags.
• read: retrieve entities by id or by filter, with an optional limit. The workhorse, and the op every analytics platform leans on.
• nav: walk the tree the server exposes one level at a time using navId. Useful before you know enough about the model to write a filter against it.
• hisRead: history for a single point over a range, expressed as a date span or a keyword such as today or yesterday.
• watchSub and watchPoll: subscribe to a set of points and poll for changes. The subscription carries a lease returned in the response metadata, and the client has to keep polling inside it or the watch is dropped.
• pointWrite: read or set a level of a writable point's priority array, in the BACnet style where the lowest-numbered level in play wins and a fallback level applies when nothing else holds the point. Called without a level, it returns the whole array, which is how you find out what is currently holding a point.
• invokeAction: call an action on an entity. The escape hatch, and the op that turns a read integration into one that can change plant behavior, so it deserves a deliberate decision rather than a default permission.

The Tags a Point Needs Before Anything Will Consume It

Analytics platforms rarely fail loudly on an under-tagged point. They skip it. The structural minimum is a point marker, an equipRef pointing at an equip, and that equip carrying a siteRef to a site, plus the id and dis that every entity has. A point with rich descriptive tags and no equipRef still exists and still reads, but it belongs to nothing, so every rule scoped by equipment type or by building passes over it. The coverage gap then shows up as a suspiciously clean fault report rather than as an error.

kind and unit decide whether arithmetic is possible at all. kind is Number, Bool or Str, and unit has to be a symbol from the Haystack unit database rather than free text, which is where Niagara integrations most often come up short: Niagara carries units in point facets, the mapping to Haystack symbols is not one to one, and points configured with no facet, or with a facet that has no Haystack equivalent, arrive unitless. A rule that compares a value against a threshold cannot run on a unitless number. The type mapping bites in the other direction too, since a Niagara multi-state enum maps to Str rather than Bool, and a rule written for a boolean command point will never match it.

tz is the point's timezone, and Haystack writes it as the city portion of the IANA name only: New_York, Los_Angeles, Denver, not America/New_York. Clients that send the full IANA string get a rejection or an unmatched zone. The tag does more work than it appears to, because hisRead ranges are interpreted in the point's tz and timestamps come back in it, so a point that inherited the host machine's timezone instead of the building's returns history shifted by whole hours. That gets reported as bad data rather than as a tagging error, and it survives every check that looks only at values.

• id and dis: a unique Ref and a human-readable name. dis is what the consuming platform displays, so leaving it as NumericWritable7 undoes a good part of the reason for tagging in the first place.
• sensor, cmd or sp: what the point is for. Most fault rules match the role marker before anything else, so a point carrying none of the three, or two of them, drops out of every rule that depends on the distinction.
• cur, his and writable: what the point supports, respectively a readable live value, an archive that hisRead can answer for, and a point that pointWrite applies to. These are capability claims about the point, not instructions to the station, so tagging a point his does not cause anything to begin archiving it.
• Quantity and location markers such as discharge, air, temp, chilled and water: what is being measured and where. None of them carries meaning alone; the combination does, which is why markers added one at a time to satisfy one rule tend to produce a model no other rule agrees with.
• geoAddr and area on the site record: required by anything that normalizes across buildings, such as energy per unit area or weather correlation. No amount of point-level tagging substitutes for them.

Where the Endpoint Runs, and What It Costs the Station

nHaystack is a Niagara module rather than an external gateway, so installing it is a platform operation and not an application one: the module goes into the Niagara modules directory on the host, the service is added to the station, and the station is restarted. The endpoint then appears on the station's existing web server under a haystack path rather than on a port of its own, which means the station's web service has to be enabled and running. A JACE engineered headless with web access switched off answers nothing until that changes. Module compatibility is the other gate: modules are built against a Niagara version, and current Niagara releases require modules to be signed by a certificate the platform trusts, so a build that ran on an older station is not automatically installable on a current one.

Authentication is Niagara's own, not a second user database. A client connects as a station user, and that user's permissions decide what the API can see and what it can write, which is the argument for giving a read-only integration a read-only user rather than the engineering account. The scheme is the part that usually goes wrong: newer Haystack versions specify SCRAM, older clients and older module builds speak HTTP Basic, and a mismatch presents as a 401 that no password change will fix. Because the endpoint rides the station's existing web server, there is no new listener and no separate certificate to manage, and the access rules already governing that web port govern the Haystack endpoint too, which is worth confirming rather than assuming when the consuming platform sits outside the building network.

Whether to expose per station or only at the Supervisor is a real architectural choice. A Supervisor endpoint gives the consuming platform one connection, one credential and one place where the model is maintained. Per-station endpoints avoid concentrating all of that traffic on the Supervisor, but they multiply credentials and endpoints and give the model somewhere to drift between stations. History usually settles it: a Supervisor can only answer hisRead for data that was actually archived up to it, so if the histories live on the controllers, a Supervisor-only endpoint hands the analytics platform live values and empty history.

The model costs the station something. The entity structure the service builds is held in the station and consumes station resources, and the service keeps a cache of the Niagara-to-Haystack mapping that has to be rebuilt after tagging changes before queries reflect them, which is the usual reason a freshly tagged point still fails to answer a filter. An external client holding a watch and polling it is continuous load on a device that was sized for its control job, not for serving data. On controllers already near their resource limits, the honest sequence is to check headroom before adding the service rather than after.

Telling the Failure Modes Apart

Nearly every Haystack integration problem presents the same way at the far end: the dashboard is empty, a site is missing, or a rule that works elsewhere returns nothing here. The work is telling apart causes that look identical from the client. Work outward from the connection to the model to the data, because each layer has a cheap test that either clears it or stops the search there.

about and ops answer the connection layer on their own. about proves the module is installed, the service is running and the web server is responding, and it reports the product and module versions any compatibility question will need. ops proves the operation your platform depends on is implemented in that build rather than merely present in the specification. Neither call depends on a single tag being correct, which is what makes them the first two to run when someone reports that the analytics platform cannot see a site.

• A 404 on the about op: the module is not installed on that station, the service was never added, or the station's web service is not running. No tagging question is in play yet.
• A 401 that a password reset does not fix: an auth scheme mismatch, a client speaking Basic to a station expecting SCRAM or the reverse, rather than a bad credential. The station's own user audit shows the attempt arriving, which separates it from a request that never reached the host.
• An unknown op, or an op absent from the ops grid: that build does not implement it. hisWrite and invokeAction are the common absentees, and no amount of client configuration adds them.
• Data that flows for a few minutes and then stops: the watch lease expired because the client stopped polling inside it. A fresh watchSub restores the flow at once, which is how it is told apart from a network drop.
• A read filter returning an empty grid while nav shows the points: tag names are case-sensitive, and the service's model cache may predate the tagging. A filter typed as Temp instead of temp and a point tagged after the last cache rebuild produce identical symptoms.
• Points that appear in one rule and not another: conflicting or missing role markers, typically a point carrying both sensor and cmd, or neither.
• equipRef values that no longer resolve: nav walks the tree normally and every join returns nothing. A consumer that cached ids has to re-read them after a rebuild that regenerated refs.
• hisRead returning an empty grid for a point whose live value reads fine: either the point has no history extension at all, or the query went to a Supervisor that never received the archive.
• History arriving shifted by whole hours: the tz tag, not the archive.
• pointWrite accepted but the equipment does not move: a higher priority level is already holding the point.