feat: adds reusable actions

[baywet]

This pull request adds action templates.

fixes #33
fixes #136
fixes #270

closes #238

This is another attempt to solve a scale limitation in the current specification. Action templates are better than the previous parameters proposal because:

• They do not introduce a matrix projection, people may use the template multiple times.
• They do not require "declaring" which environment variables to read in advance.
• They limit the string literal replacement to the templates, which does not impact how actions currently work.
• The template reference mechanism works like OpenAPI references.
• The template resolution mechanism can be isolated as a pre-processing step, and the resolved actions can then be fed to the existing code processing actions.
• It introduces a string literal replacement syntax which can be expanded upon for other sources or used at other places in the future.

The pull request is incomplete as it is, it's a draft, I want to collect feedback on the approach before making any further investments.

[ralfhandl]

I like the general direction.

[baywet]

Feedback from the meeting from both @lornajane and @ralfhandl :

• add a peer block to parameters for the environment variables, this allows to set defaults and "declare" which variables are used. But this declaration is optional.
• add a peer "actions" in the components, that's defined as everything except for the target. So people can specify the target on reference. Reusable actions may use action templates in turns.

[ralfhandl]

Mostly wording and capitalization

[baywet]

@ralfhandl @lornajane I pushed another update a couple of minutes ago. I wasn't happy about the whole reusable actions vs action templates kind of thing. After chatting with @mikekistler internally I realized we could simply define a an action template reference object as "you can override anything from the resolved template in the reference" like JSON schema does to some extent. And keep things extra simple.

Let me know what you think!

[baywet]

Sharing some number from the refactor on our internal repos: we're seeing about 30% reduction on average. Of course that number highly depends on how repetitive your Overlay document is.

[mikekistler]

Looks good. 👍

I think this is a valuable addition to Overlays and will help expand their adoption.

I wish we could fix some of the terminology quirks we have discussed -- in particular "Reusable Action Objects" -- but I don't want to make perfect the enemy of the good.

feedback implemented

[handrews]

A few minor comments and questions.

[lornajane]

I've got a lot of questions about the intent here (which I thought we'd already been through, so please accept my apologies for asking it again). I hadn't seen, or perhaps hadn't internalised, the use cases where target is getting rewritten in the reusable action as well as the action bit. I think this is beyond the scope of what I had previously seen and I'm uneasy about how approachable it is.

Also I think we use REQUIRED for fields that are required, rather than marking the ones that are not as Optional - so that needs to be consistent.

[lornajane]

I had thought it was not allowed to include the target. The target has to be with the action, but the action definition can be referred to as one of the reusable items from components. What's the use case for supporting action?

[baywet]

On this instance specifically, I think you've missed a few discussions, either directly here in the PR history, or on the Overlays call. I don't want to re-litigate the whole thing that late in the process. But at a high level:

• We believe that consistency is key, is there really any good reason to allow every field here but target?
• We believe that string interpolation + target are a powerful combination for advanced scenarios, that's actually lead to a huge reduction in my team's context "similar" actions.
• We believe we can also enable simpler scenarios (people who don't want to mess around with string interpolation), by allowing an override at the reference level.

Hopefully that's enough context, let me know if you still have questions or concerns.

[lornajane]

This change brings multiple features in at once, and I'm uneasy that we've missed the purpose of Overlays and leapt in a new "more is more" direction with this proposal which seems to bring a lot of complexity and also brings back many aspects that have previously been discarded as not a good fit - either for being too much change in a single leap, or just being out of scope for what Overlays is.

I'm supportive of actions being reusable to avoid repetition. I'm not in favour of configurable targets which would allow a matrix of difficult to diagnose or reason about operations. I've consistently expressed these views, I'm not sure it's fair to frame it as me having "missed" something.

[baywet]

I think I may have misunderstood your concern earlier. Based on your initial comment on the earlier proposal and from what I recall from our conversations, I had understood the main sticking point to be the matrix aspect. Essentially, I got a sense that "nobody wants to do math in overlays, it brings too much cognitive load". This is why I rewrote the thing entirely, to start with new design principles and avoid carrying this over.

I'll be honest — getting this feedback at this stage is frustrating, as the draft has been open for two months and this specific concern about target first came up 8 days ago. I recognize async communication is lossy, and I may have missed implicit signals, but it does make it harder to iterate when fundamental concerns surface this late.

Setting that aside, I'd like to better understand your concern about allowing string interpolation in target. Is it primarily about security, readability, invalid JSONPath after resolution, or overall spec complexity? If you can point to a concrete failure mode or example, that would help me think about alternatives.

The reason I've been pushing for interpolation in target/copy is that it enables patterns like the sidecar example. Another example would be cases where you have a dozen endpoints each with a different schema and want to introduce a status monitor for each, without relying on generic matching. I'm happy to update examples/test cases if they are not illustrative enough.

I'd also like to understand how you'd see this applying to copy, since I think consistency between the two matters.

I'm hoping this helps us narrow the disagreement and find a workable direction. :)

[lornajane]

I'm sorry I've been unable to schedule myself into the meetings but let me try to recap where I think we are:

• we're bringing in a few different features here
• one is reusable actions to avoid copy/pasting the same action for lots of target expressions
• we're also introducing parameters for those actions so that as well as allowing copy/paste, it's possible to adjust

All this is good so far, it's a big amount of change and does change the nature of Overlays as a project - but innovation is good and we have clear use cases and good examples that will help.

What I'm questioning here is the reusable targets. We've talked about interpolation in the target of an action object. I understood the reusable actions to be copy or updates with parameters, and parameters potentially being pulled into the description or target of the actual (not reusable) action. But seeing reusable targets, I had doubts and had assumed that further discussion was still welcome.

[baywet]

Hey @lornajane
Apologies. During the last Overlay meeting, I had committed to put together a concrete example of why I believe string interpolation support in targets is important. Health issues and other priorities came in the way of doing this.

Here is a gist that illustrates the difference in overlay experience, please have a look, and let me know whether it addresses your doubts regarding the usefulness of this aspect.

Essentially my concern is to enable reusable actions "components", where users only need to "fill in the parameters" and we get a consistent, easy to maintain, low thinking required experience from the users of the reusable actions.
I realized this maybe tailored to larger organizations, where large overlay documents are in use and where the people building the reusable actions and the one "referencing them" are not necessarily the same person. But I still think this is a genuine use case.