using ResultSelector in Map / Iterator in AWS step function : $.variable name is taken literally

Question

I'd like to filter output from each iteration of a Map step before collecting it into an array. However, when I use ResultSelector for this purpose, I am getting a verbatim result instead of content of the variable.

Each lambda in CallWorkerLambda returns a dictionary with element "output". I would like to take only this element (if I don't the IO pipeline gets overwhelmed with service parts of the returned message). To this end, I use "ResultSelector": "$.output" within the Iterator, and then "ResultPath": "$.output_array" in the outer Map step.

However, what I get as the resulting array looks like "output_array": ["$.output", "$.output", "$.output", ... ]

Below is the step code:

"ProcessPatch": {
      "Type": "Map",
      "Next": "Aggregate",
      "InputPath": "$",
      "ItemsPath": "$.taskdef",
      "MaxConcurrency": 0,
      "ResultPath": "$.output_array",
      "Parameters": {
        "Payload.$": "$$.Map.Item.Value",
        "algo_lambda.$": "$.staticdata.algo_lambda",
        "staticdata.$": "$.staticdata"
      },
      "OutputPath": "$",
      "Iterator": {
        "StartAt": "CallWorkerLambda",
        "States": {
          "CallWorkerLambda": {
            "Type": "Task",
            "Resource": "arn:aws:states:::lambda:invoke",
            "Parameters": {
              "FunctionName.$": "$.algo_lambda",
              "Payload": {
                "s.$": "$.Payload",
                "staticdata.$": "$.staticdata"
              }
            },
            "ResultSelector": "$.output",
            "End": true
          }
        }
      }
    }

Any ideas on what makes it fail to recognize $.variables and how to work around it?

question from:https://stackoverflow.com/questions/65840362/using-resultselector-in-map-iterator-in-aws-step-function-variable-name-is

Answer 1

The ResultSelector field lets you create a collection of key value pairs, where the values are static or selected from the state's result. So ResultSelector should be used like this:

"ResultSelector": {
    "ClusterId.$": "$.output.ClusterId",
    "ResourceType.$": "$.resourceType"
  },

But in your example it lacks key value pairs:

"ResultSelector": "$.output"