enableAPI now defaults to 0 (it was 1). In 6.x every authenticated user could call any action
through /exec and run any script through /eval and /eval/action; at 0 such a call fails
with Api is disabled. It can be enabled by using setting enableAPI > 0. What stays allowed at 0:
- an action annotated
@@api- for an authenticated user, as1used to allow it; - an action annotated
@@noauth, and a request with a valid signature, as before; - any call of a user who has access to the
System.interpreternavigator form, since that user can already execute any code through the interpreter.
What an upgrading application can notice: an external system that calls actions over HTTP gets the
error above for every action without @@api. The fix is to annotate the actions the external
systems are meant to call with @@api; or to give enableAPI=1 to the role of the user they
authenticate as, through the per-role settings (Administration > System > Settings > Parameters).
The quickest way back is settings.enableAPI=1 globally (in lsfusion.properties,
conf/settings.properties or -Dsettings.enableAPI=1), which restores the 6.x behaviour - and with
it the arbitrary code execution over HTTP for every authenticated user, see
Authentication.
The order in which global events run is built at startup from the dependencies between them, and events with a dependency between them have always been ordered by it. For events with no dependency the order was whatever the traversal happened to produce, and the traversal started from the properties in the order the server had created them - an order that is not reproducible between starts, so the same logic could run two such events one way on one start and the other way on the next. The traversal now starts from an order that follows from the logic itself.
What an upgrading application can notice: two events with no dependency between them can run in a different order than they did before. That shows up where the application relied on their order without expressing it - two events writing the same property, or one reading what the other writes through something the platform does not see as a dependency.
There is no setting to go back, and nothing to go back to: the previous order was not one order but
a different one on each start. An application that needs a particular order states it with AFTER
(GOAFTER) in the event description block. Until it does, what used to appear and disappear across
restarts now behaves the same way every time, which is what makes it possible to find.
Inside a transaction TRUNCATE is the expensive way to empty a session (temporary) table: it holds an
exclusive lock on the relation until the commit, and on postgres such a lock suppresses the fast path
of the lock manager for the whole cluster for as long as it is held. A table returned inside a
transaction is now emptied with DELETE unless it holds more rows than
deleteFromInsteadOfTruncateForTempTablesInTransactionThreshold (100000 - a starting point, not a
measured optimum). Outside a transaction the old threshold (0, always TRUNCATE) still applies, so
what is chosen there changes only for an installation that raised it.
The rows an emptying leaves dead are counted per table until its storage is reset, and count towards
the same threshold, so a table emptied that way over and over goes back to TRUNCATE rather than
growing for the life of the connection.
Such a table is genuinely empty from the moment it is returned, so its name is handed out again right
away; it used to be kept out of the pool until the end of the transaction, which cost one extra
CREATE TEMPORARY TABLE per table shape per transaction.
What an upgrading application can notice: a pooled temporary table holds those dead rows until the
debt crosses the threshold, so its size on disk can exceed its row count. Set
deleteFromInsteadOfTruncateForTempTablesInTransactionThreshold to 0 for the previous behaviour.
subtractSeconds, secondsBetweenDates and toSeconds were declared INTEGER, but the expression
behind them (extract(epoch ...)) returns a fractional number on postgres, and nothing brought that
number back to the declared class. Two things followed. The value kept its fractional part - the
difference between 00:00:00.5 and 00:01:00 was 59.5, not 59 - which showed up only where the
value was finally written or read, and there it was rounded, so the property reported 60. And every
expression built on top of it was computed in that wider type, so subtractSeconds(from, to) / 60
divided as a fraction instead of the whole division its declared class implies.
These properties now truncate towards zero, and they are declared LONG - seconds between two dates
fit in INTEGER only up to about 68 years on postgres (on MSSQL DATEDIFF still computes the
difference in int, so the span it accepts is unchanged). getMilliSeconds, already LONG, truncates as well, and
extractWeek / extractDOWNumber bring their result back to INTEGER, the way the other
extract-based properties of Time already did.
What an upgrading application can notice: expressions over these properties are computed in whole
seconds now, so subtractSeconds(...) / 60 gives whole minutes (59 / 60 is 0) where it used to
give 0.98; wrap the operand in NUMERIC(...) where the fraction is what is wanted. An interval
shorter than a second is zero now rather than a fraction, which also holds for the durations the
system modules themselves compute (the ones SystemEvents sums and weights). A value that
was rounded on writing (59.5 became 60) is truncated now (59). Writing the result into a
property declared INTEGER keeps working, the value is narrowed on writing, but a property derived
from one of these has class LONG.
The check that looks for recursive properties used to start only from abstract properties, and it
did not follow the value an event writes (<- ... WHEN), only its condition. A property recalculated
from itself - directly or through other properties - therefore passed startup and only failed later,
the first time a session computed it.
The check now walks, from every property, the same dependencies the platform itself walks when it
computes one, and it runs during startup. (Its other half, the one that follows PREV, is a
different relation that nothing computes in that order, and it keeps looking only at abstract
properties, as before.) An application with such a cycle no longer starts: it fails with
Property ... is recursive. One of the paths: [...], naming the properties on the cycle and, for
those declared in .lsf, the position they are declared at.
dryRun runs the same check - it is the same task now, not a copy: it and the finalization of the
JSON properties of forms moved out of the database-side part of startup, which they never needed.
What an upgrading application can notice: logic that used to start can now be rejected at startup.
This only happens where the cycle is real, and such logic was already broken - it crashed the first
session that computed the property. But nothing computed it until then: the pass that would build the
dependencies of every property at startup is off by default (settings.disablePrereadCaches), so a
cyclic property that no one uses could sit in an application for years without being noticed. There
is no setting to switch the check off - the cycle has to be removed from the logic.
A GROUP LAST / PARTITION ordered by an indexed column used to degrade from an index lookup per
key to a full scan of the whole table as soon as the session held an unsaved change of the property
the aggregation is ordered by. The unsaved change is applied by substitution, so the ordering stopped
being the plain column, the planner saw no index for it, concluded that restricting the aggregation
to the requested keys buys nothing, and computed it for all keys over the whole history. Measured on
a 111M row table read for 25 keys: fractions of a second before the edit, minutes and over 8 GB of
temporary files after it - on a server with temp_file_limit set, a postgres error.
Such an ordering is in fact compiled per branch (the changed rows and the rest), and in each branch the ordering is the plain column again. The planner now takes that into account when estimating, so the per-key index plan is chosen as it is for a clean session.
What an upgrading application can notice: plans change for aggregations of this shape - ordered by a
column that has an index, read while the session holds unsaved changes of that column. There is no
dedicated setting; settings.noOrderTopSplit=true disables the order splitting itself and with it
this estimate, restoring the previous behaviour.
A React component can now draw the window the forms open in and the window the messages
appear in, and a DESIGN container's custom is read by the same rules a window's has
always been read by. Three things an upgrading application can notice:
- the client and the server must be upgraded together. The API version went to 392 because every window now carries what draws it, so a client of the previous version refuses to connect. There is no setting for this : deploy the pair;
- a form the server used to build can now be refused, with the message naming the
container. Three combinations are checked when the form is built : a
customliteral that is neither a React component name nor markup (a misspelled component name, which the browser used to draw as its own text),tabbedtogether withcustom(the client drew one of the two and ignored the other, and with acustomproperty it failed in the browser), and a React component paired with acustomproperty. The fix is to write what was meant : a component name, markup, or one oftabbed/custom; WINDOW ... CUSTOM 'literal' CUSTOM property()written in ONE statement keeps both halves. It used to drop the literal, so a window drawn from a template started empty until the property computed its first value ; it now shows the literal from the start, as two statements always did. A window written this way with a COMPONENT literal beside a property is refused at load, because a component draws the window itself.
A template a property computes now reports a place that names nothing, in the place and in the browser console. That changes what such a window or container shows - a message where there used to be a blank spot - but not whether the application starts.
Two situations that used to be tolerated now raise an error (see issue #1726):
- another session working inside a transaction - a session doing anything on an SQL session that another one (typically opened earlier in the same thread) already holds a transaction on. Its statements run in that transaction : they see its uncommitted data and disappear with its rollback, while the session itself believes it is outside any transaction. The extreme case is an apply, which never had a transaction of its own - its commit was physically nothing, so the data was silently lost, and the temp tables it created turned into the "relation t_N does not exist" errors of issue #1716. The service operations that do this knowingly (database synchronization, recalculations, checks) take it on with the setting below, the same way an application can;
- a user interaction inside a transaction - a dialog, a message or an input request
raised from a form or navigator action executed under
APPLY(an apply event, for instance). It used to be an assert, that is a log line in production, while the transaction and its locks were held for the user think time. Interactions from the contexts that can process them without a user (background tasks, the scheduler, the external API) are unaffected.
Symptoms after the upgrade : an action that previously "worked" now fails with
OTHER DATASESSION IN THE MIDDLE OF TRANSACTION IN THIS THREAD or
USER INTERACTION IN TRANSACTION. In both cases the flow was already broken - the
changes were being lost, or the transaction was hanging on a dialog - so the fix is to
move the work or the interaction out of the transaction.
Expect more of these than the flows you know about. Both situations used to be asserts, that is log lines in production, so they went unnoticed for as long as they existed; and the first one is now reported for ANY operation of the other session - a read, a write, a temp table - not only for its apply, which is simply the point where the damage used to become visible.
When either is intended, it can be allowed for a single stack, taking the consequences on knowingly :
pushSetting('allowUserInteractionInTransaction', 'true');
// ... the action that interacts inside the transaction ...
popSetting('allowUserInteractionInTransaction');
allowNestedTransaction works the same way for the other session's work. Both settings
also work globally : setting them restores the previous behavior (a log line instead of
the error), which is the quickest way back if the upgrade breaks a production flow -
the places to fix are then in the assert log, under the same messages.
Order-dependent aggregations and row picks (GROUP LAST / CONCAT / ordered CUSTOM,
TOP picks, the order-sensitive PARTITION types, GROUP AGGR / NAGGR / EQUAL on
duplicate BY keys, assignments with extra WHERE parameters, the FOR ... NEW
numbering) now implicitly append the object order as the ORDER tiebreak
(see issue #1700). Results that
previously depended on the SQL scan order become stable; queries with a total explicit
ORDER are unaffected. The compiled plans change slightly (longer aggregate / window
ORDER BY), and where the order was not total the picked row / concatenation order may
differ from what a particular database happened to return before - if a specific pick is
required, specify a total ORDER explicitly. There is no fallback setting: the previous
behavior was nondeterministic.
7.0 reworks two aspects of predicate push down planning:
- interval comparisons on keys (
a <= key,key <= b) participate in the join statistics (key compare joins), and interval keys are sourced by an iteration instead of being protected by hanging-key heuristics; - when a predicate is pushed into an aggregation, the joins that depend on the push target are no longer cut from the pushed condition wholesale: their depending arguments are virtualized (replaced with a fresh key), so the pushed condition keeps its key sources (see issue #1699).
Both changes affect query compilation globally: complex queries (filtered aggregations over intervals, nested aggregations, report totals and column footers) compile into different - usually better - plans. However, on large production volumes or in rare query shapes this may surface as:
- queries failing with the incorrect set operation error
(
Операция над множеством некорректна) that worked on 6.x; - performance degradation of specific complex queries.
The pre-7.0 planner behavior can be restored without downgrading, via settings.properties
(or -D JVM parameters):
settings.removeJoinCutBackwardCompatibility = true
settings.keyExprCompareJoinBackwardCompatibility = true
The first setting reverts the push down recursion guard to cutting the dependent joins wholesale, the second reverts the interval comparison handling. Setting both restores the pre-7.0 push down planning entirely. The settings can also be applied selectively to localize which of the two changes causes a particular problem. See the Working parameters article.
Until now an interval condition (prop >= a AND prop < b) was taken into account in the join
statistics only when the property was the first field of some index. A property standing
later in a composite index (say dateTime in INDEX device(m), dateTime(m)) was estimated as
if it were not indexed at all, so a filter like device(m) = value AND dateTime(m) >= from AND dateTime(m) < to was estimated as a full scan of the table even though the index covers it.
Such an interval is now taken into account for any field of a btree index. How much of the
index the database can actually use - which depends on the fields preceding that one being
bound - is a separate question, and it is answered where it was answered before: by the
access cost calculation, which walks the real index field by field and stops at the first
unbound one. Fields of LIKE / MATCH indexes other than the first are not affected.
Affected queries compile into different - usually better - plans; as with any statistics change this may surface as a performance degradation of a specific complex query.
settings.prefixIndexIntervalBackwardCompatibility = true
This one is read while the indexes are being registered at startup, so it has to be set before the server starts.
Order-dependent group operators (GROUP LAST / MAX / CONCAT, ordered PARTITION)
whose ORDER expression is a nullable property now emit an ORDER BY whose null placement
matches the index built for that column on PostgreSQL, so the aggregation is served by an
index scan instead of a full scan of the grouped partition plus a sort
(see issue #1727). Results are unchanged
(the aggregation already excludes null-ordered rows); only the plan changes, and existing
databases need no reindex. On large volumes this removes full-scan/spill behavior for such
aggregations. The change is result-safe and there is no fallback setting.