diff --git a/readme.md b/readme.md index 87f4887..6d09567 100644 --- a/readme.md +++ b/readme.md @@ -141,7 +141,7 @@ Everything goes over MQTT (port 1883 of `alex2mqtt.stormysdream.club`); the libr - **Discovery.** On `/discover` the library answers with one discovery object per device on `/discover_r`. The backend accepts one endpoint object per message and collects everything that arrives within 1 s for Alexa's discovery answer (up to 5 s for its proactive AddOrUpdate push), so all devices are published back to back from the next `loop()`. Each object sits on the heap (about 1 KB) until the MQTT client has sent it; when the client cannot take another one (free heap under 4 KB), the library prints `[Alex2ESP] discovery deferred at ` and sends the rest from `loop()` as the queue drains, for up to 5 s after the request. `[Alex2ESP] error: discovery gave up: N device(s) not announced` means those devices missed this answer - on the backend's proactive discovery that can remove them from Alexa until the next one. - **Directives.** The directive arrives as JSON on `//alexaDirective`. A directive larger than one TCP segment arrives in fragments, which are put together in one heap block that exists only until `loop()` has parsed it. `loop()` then fires `ReportState` or `Event` (and `DirectiveReceived`, if registered) with the directive: `directive["header"]`, `directive["endpoint"]`, `directive["payload"]`. One directive is handled at a time; a directive that arrives twice (a broker that mirrors its topics delivers every message twice) is handled once. - **Reports.** `send()` publishes the report on `//alexaResponce` at once. The backend waits 7 s for it, so answer from the event handler. Every property carries the board's UTC time as `timeOfSample`. -- **Limits.** A directive and a report (or the discovery object of one device) may be 2047 bytes each; `-DALEX2ESP_MAX_DIRECTIVE=` and `-DALEX2ESP_MAX_MESSAGE=` in `build_flags` change that. `send()` returns `false` when the report was not sent: no session with the broker, the MQTT client or the heap cannot take it, or it is too large. Nothing is ever sent truncated, and nothing is dropped without a line on Serial. +- **Limits.** A directive may be 2047 bytes (`ALEX2ESP_MAX_DIRECTIVE`), a report or the discovery object of one device 3071 (`ALEX2ESP_MAX_MESSAGE`). The second limit is the first plus 1024 unless it is set: an answer repeats the `correlationToken` of its directive, which is most of a large directive, and adds 140 to 170 bytes per property, so the largest directive can be answered with six properties. `-DALEX2ESP_MAX_DIRECTIVE=` and `-DALEX2ESP_MAX_MESSAGE=` in `build_flags` change the limits. `send()` returns `false` when the report was not sent: no session with the broker, the MQTT client or the heap cannot take it, or it is too large. Nothing is ever sent truncated, and nothing is dropped without a line on Serial. - **Serial output.** Every line of the library starts with `[Alex2ESP]`, a problem with `[Alex2ESP] error:`. `alexClient.setLogLevel(AlexaLogLevel::ERROR)` leaves only the problems, `AlexaLogLevel::NONE` nothing; the default, `AlexaLogLevel::INFO`, adds the session, discovery and one line per directive (`[Alex2ESP] ESP-01 <- Alexa.PowerController.TurnOn`). `AlexaLogLevel::DEBUG` (sizes and free heap per message) has to be compiled in with `-DALEX2ESP_LOG_MAX=3`; `-DALEX2ESP_LOG_MAX=0` compiles every line out. Credentials and correlation tokens are never printed. A sketch that defines a macro named `DEBUG`, `ERROR` or `INFO` cannot write the level of that name; it passes the number instead, for example `alexClient.setLogLevel(static_cast(3))` for `DEBUG`. Boards that run 1.1.0 or older keep working: the backend still publishes the token on `//alexaDirective_e` and serves the HTTP routes they use. @@ -243,12 +243,12 @@ For instance, to manually report the state of a PowerController, you can use the Behaviour changes: - Directives arrive over MQTT. The library subscribes to `/+/alexaDirective`, where Alex2MQTT has always published the whole directive, instead of fetching it over HTTP with the token from `//alexaDirective_e`. The HTTP detour dates from 2024, when a directive larger than one TCP segment reached the MQTT callback in pieces; the pieces are now put together by their offset and the total length, in one heap block that lives until `loop()` has parsed the directive. `loop()` no longer stalls for two HTTP round trips per directive. -- Reports leave over MQTT. `send()` publishes on `//alexaResponce` at once, where 1.1.0 queued the report for an HTTP POST from a later `loop()`. It returns `false` when there is no session with the broker, when the MQTT client or the heap cannot take the report, or when the report is over 2047 bytes (`ALEX2ESP_MAX_MESSAGE`); each case prints its reason. The 5-slot send queue is gone. +- Reports leave over MQTT. `send()` publishes on `//alexaResponce` at once, where 1.1.0 queued the report for an HTTP POST from a later `loop()`. It returns `false` when there is no session with the broker, when the MQTT client or the heap cannot take the report, or when the report is over 3071 bytes (`ALEX2ESP_MAX_MESSAGE`, by default 1024 more than the largest directive, so that every directive that is accepted can be answered); each case prints its reason. The 5-slot send queue is gone. - `timeOfSample` is the board's own time in UTC, for example `2026-09-28T13:05:09Z`. 1.1.0 sent the placeholder `{REPLACE_WITH_DATETIME}`, which only the backend's HTTP route replaced. `begin()` starts SNTP (`pool.ntp.org`, `time.nist.gov`) and no longer connects itself: `loop()` opens the MQTT session once the clock is set, or after 5 s without an answer, so the session comes up a few seconds later than before. `alexClient.setTimeSource(false)` before `begin()` leaves the clock to the sketch. `AddContextProp()` fills `timeOfSample` in when the property has none or carries the old placeholder. - A directive is parsed and handed to the sketch from `loop()`, one at a time. One that arrives while the previous one still waits for `loop()` is dropped, as is one over 2047 bytes (`ALEX2ESP_MAX_DIRECTIVE`); both print an error. - A directive that arrives twice is handled once: the Alex2MQTT broker currently delivers every message twice through a mirror. The `messageId`s of the last four directives are remembered; the repeat prints `repeated directive ... ignored`. - The subscription delivers the directives of every endpoint of the account. Those for endpoints of another board are recognised by their topic and neither buffered nor parsed. -- Discovery is answered from `loop()`, not inside the MQTT callback. A discovery object over 2047 bytes is refused with an error; the other devices are still announced. +- Discovery is answered from `loop()`, not inside the MQTT callback. A discovery object over 3071 bytes (`ALEX2ESP_MAX_MESSAGE`) is refused with an error; the other devices are still announced. - `getState()` stays `INITIALIZED` until the first connect, and becomes `CONNECTED` when the broker has acknowledged both subscriptions (1.1.0: the first of them). A refused subscription prints an error. - Serial output goes through one log with levels. `alexClient.setLogLevel()` takes `AlexaLogLevel::NONE`, `ERROR`, `INFO` (the default) or `DEBUG`; `-DALEX2ESP_LOG_MAX=<0..3>` in `build_flags` sets the highest level that is compiled in (default 2, `INFO`). This replaces the `Alex2ESP_DEBUG` define inside `AlexaUtils.cpp`. 1.1.0 printed the memory figures for every MQTT message and the whole directive, correlation token included, for every directive; both are gone. A sketch or a build flag that defines `DEBUG`, `ERROR` or `INFO` as a macro (`#define DEBUG 1`, `-DDEBUG`) still compiles: the library sets those macros aside where it declares its levels and passes a level by its number everywhere else. - `getDevice()` before `begin()` prints an error: the device would have no root topic. A second `begin()` is ignored with an error. @@ -257,7 +257,7 @@ Behaviour changes: Memory: `examples/basicLight.cpp` for a D1 mini takes 34,116 bytes of static RAM (1.1.0: 52,768) and 336,757 bytes of flash (1.1.0: 350,885), as PlatformIO reports them (espressif8266 4.2.1, Arduino core 3.1.2). The static RAM was the five 2 KB queue slots, three more 2 KB buffers and the two HTTP clients. SNTP and the time stamp are 1.8 KB of the flash figure. -Tests: `pio test -e native` in the repository runs 32 host tests of the receive and publish logic (reassembly of fragments, the two size limits, repeated directives, topics, time stamps). No board is needed. +Tests: `pio test -e native` in the repository runs 33 host tests of the receive and publish logic (reassembly of fragments, the two size limits, repeated directives, topics, time stamps). No board is needed. Boards that run 1.1.0 are not affected: the backend keeps the token topic and the HTTP routes. diff --git a/src/Alex2ESP.h b/src/Alex2ESP.h index ca44e50..84cb631 100644 --- a/src/Alex2ESP.h +++ b/src/Alex2ESP.h @@ -21,16 +21,12 @@ #include "AlexaBridgeLogic.h" #include "AlexaDevice.h" #include "AlexaInterface.h" +#include "AlexaLimits.h" #include "AlexaLog.h" #include "AlexaTransport.h" #include "AlexaUtils.h" #include -// Largest directive the bridge accepts, in bytes. Override with -DALEX2ESP_MAX_DIRECTIVE= in build_flags. -#ifndef ALEX2ESP_MAX_DIRECTIVE -#define ALEX2ESP_MAX_DIRECTIVE 2047 -#endif - enum class Alex2ESPState { UNINITIALIZED, // begin() has not been called diff --git a/src/AlexaLimits.h b/src/AlexaLimits.h new file mode 100644 index 0000000..af9c9b7 --- /dev/null +++ b/src/AlexaLimits.h @@ -0,0 +1,22 @@ +// The size limits of the bridge. Each is a default: -D= in build_flags overrides it for the whole +// project. The Arduino IDE has no per-sketch flags and builds with the defaults. +#ifndef ALEXA_LIMITS_H +#define ALEXA_LIMITS_H + +// Largest directive the bridge accepts, in bytes. Alex2MQTT publishes directives of 600 to 900 bytes; most of +// that is the correlationToken. +#ifndef ALEX2ESP_MAX_DIRECTIVE +#define ALEX2ESP_MAX_DIRECTIVE 2047 +#endif + +// Largest message the bridge publishes, in bytes of JSON: a report, or the discovery object of one device. +// +// The answer to a directive repeats its correlationToken, so it is as large as the directive plus 140 to 170 +// bytes for every property it reports. The default leaves 1024 bytes for the properties, six of them: a +// directive that was accepted can be answered. With a limit under ALEX2ESP_MAX_DIRECTIVE + 300 the answer of a +// lamp (two properties) to the largest directive is refused after the sketch has acted on the directive. +#ifndef ALEX2ESP_MAX_MESSAGE +#define ALEX2ESP_MAX_MESSAGE (ALEX2ESP_MAX_DIRECTIVE + 1024) +#endif + +#endif // ALEXA_LIMITS_H diff --git a/src/AlexaTransport.h b/src/AlexaTransport.h index 47ad4e1..4a5ea9b 100644 --- a/src/AlexaTransport.h +++ b/src/AlexaTransport.h @@ -6,12 +6,7 @@ #include #include #include - -// Largest message the bridge publishes, in bytes of JSON: a report or the discovery object of one device. -// Override with -DALEX2ESP_MAX_MESSAGE= in build_flags. -#ifndef ALEX2ESP_MAX_MESSAGE -#define ALEX2ESP_MAX_MESSAGE 2047 -#endif +#include "AlexaLimits.h" // "YYYY-MM-DDThh:mm:ssZ" and the terminating NUL #define ALEXA_TIMESTAMP_SIZE 21 diff --git a/test/test_bridge_logic/test_main.cpp b/test/test_bridge_logic/test_main.cpp index a4bfc7a..00af990 100644 --- a/test/test_bridge_logic/test_main.cpp +++ b/test/test_bridge_logic/test_main.cpp @@ -7,17 +7,19 @@ #include #include #include "AlexaBridgeLogic.h" +#include "AlexaLimits.h" typedef AlexaDirectiveBuffer::Result Result; #define ASSERT_RESULT(expected, actual) TEST_ASSERT_EQUAL_INT(static_cast(expected), static_cast(actual)) // A directive as Alex2MQTT publishes it on //alexaDirective -static std::string directive(const char *name, const char *messageId) +static std::string directive(const char *name, const char *messageId, + const std::string &correlationToken = "AAAAAAAAAQBnlqNbYnB0dHNmYW5zbGF0ZQ==") { return std::string("{\"header\":{\"namespace\":\"Alexa.PowerController\",\"name\":\"") + name + "\",\"payloadVersion\":\"3\",\"messageId\":\"" + messageId + - "\",\"correlationToken\":\"AAAAAAAAAQBnlqNbYnB0dHNmYW5zbGF0ZQ==\"}," + "\",\"correlationToken\":\"" + correlationToken + "\"}," "\"endpoint\":{\"endpointId\":\"ESP-01\",\"cookie\":{}},\"payload\":{}}"; } @@ -444,6 +446,74 @@ void test_document_that_ran_out_of_memory_is_not_published() TEST_ASSERT_EQUAL_UINT(0, length); // 0 tells the two reasons for TOO_LARGE apart } +// --- the limit of a directive and the limit of its answer --- + +// One property of a report as AlexaStatusMessage::AddProperty writes it +static void addProperty(JsonArray properties, const char *interfaceName, const char *name, const char *value) +{ + JsonObject property = properties.add(); + property["namespace"] = interfaceName; + property["name"] = name; + property["value"] = value; + property["timeOfSample"] = "2026-09-28T13:05:09Z"; + property["uncertaintyInMilliseconds"] = 0; +} + +// The Response of a lamp as AlexaStatusMessage builds it (src/AlexaStatusMessage.cpp): the correlationToken of the +// directive, the health of the endpoint and the power state +static void buildResponse(JsonDocument &answer, const JsonDocument &received) +{ + JsonObject header = answer["event"]["header"].to(); + header["namespace"] = "Alexa"; + header["name"] = "Response"; + header["payloadVersion"] = "3"; + header["messageId"] = "OQpZQ2l8Pr2f9kkS8g6ffwpx7bJgJARngGUEQ"; // 37 characters, as generateMessageId() returns + header["correlationToken"] = received["header"]["correlationToken"]; + answer["event"]["endpoint"]["endpointId"] = received["endpoint"]["endpointId"]; + answer["event"]["payload"].to(); + + JsonArray properties = answer["context"]["properties"].to(); + JsonObject health = properties.add(); + health["namespace"] = "Alexa.EndpointHealth"; + health["name"] = "connectivity"; + health["value"]["value"] = "OK"; + health["timeOfSample"] = "2026-09-28T13:05:09Z"; + health["uncertaintyInMilliseconds"] = 0; + addProperty(properties, "Alexa.PowerController", "powerState", "ON"); +} + +void test_largest_directive_can_be_answered() +{ + // The correlationToken is what makes a directive large, and the answer repeats it + std::string withoutToken = directive("TurnOn", ID_1, ""); + std::string largest = directive("TurnOn", ID_1, std::string(ALEX2ESP_MAX_DIRECTIVE - withoutToken.size(), 'T')); + TEST_ASSERT_EQUAL_UINT(ALEX2ESP_MAX_DIRECTIVE, largest.size()); + + AlexaDirectiveBuffer buffer(ALEX2ESP_MAX_DIRECTIVE); + ASSERT_RESULT(Result::COMPLETE, deliver(buffer, largest, 536)); + JsonDocument received; + TEST_ASSERT_TRUE(deserializeJson(received, buffer.data(), buffer.length()) == DeserializationError::Ok); + buffer.release(); + + JsonDocument answer; + buildResponse(answer, received); + size_t length = 0; + ASSERT_RESULT(AlexaSendResult::OK, AlexaBridgeLogic::checkMessage(answer, ALEX2ESP_MAX_MESSAGE, &length)); + // The answer is the larger of the two: a limit for messages as low as the limit for directives refuses it, + // after the sketch has switched the lamp + TEST_ASSERT_GREATER_THAN_UINT(ALEX2ESP_MAX_DIRECTIVE, length); + ASSERT_RESULT(AlexaSendResult::TOO_LARGE, AlexaBridgeLogic::checkMessage(answer, ALEX2ESP_MAX_DIRECTIVE, &length)); + + // The default leaves room for six properties + JsonArray properties = answer["context"]["properties"]; + addProperty(properties, "Alexa.BrightnessController", "brightness", "100"); + addProperty(properties, "Alexa.ColorTemperatureController", "colorTemperatureInKelvin", "2700"); + addProperty(properties, "Alexa.ToggleController", "toggleState", "ON"); + addProperty(properties, "Alexa.ToggleController", "toggleState", "OFF"); + TEST_ASSERT_EQUAL_UINT(6, properties.size()); + ASSERT_RESULT(AlexaSendResult::OK, AlexaBridgeLogic::checkMessage(answer, ALEX2ESP_MAX_MESSAGE, &length)); +} + // --- topics --- void test_endpoint_is_taken_from_the_directive_topic() @@ -564,6 +634,8 @@ int main(int, char **) RUN_TEST(test_empty_document_is_not_published); RUN_TEST(test_document_that_ran_out_of_memory_is_not_published); + RUN_TEST(test_largest_directive_can_be_answered); + RUN_TEST(test_endpoint_is_taken_from_the_directive_topic); RUN_TEST(test_other_topics_are_not_directive_topics);