Filtering Arrays in DataWeave with filter

filter is one of the simplest DataWeave functions, but it becomes especially useful when integration payloads contain records that should not all continue to the next processing step. It is a good fit when the selection rule itself is part of the integration contract.

The basic idea is:

input array → evaluate each item → keep matching items → output array

Unlike map, filter does not primarily transform each item. It decides whether an item remains in the result.

Basic Example

Suppose an upstream system sends accounts with an active flag:

[
  { "id": "A1", "name": "Acme", "active": true },
  { "id": "A2", "name": "Global Media", "active": false },
  { "id": "A3", "name": "Northern Trail", "active": true }
]

To keep only active accounts:

%dw 2.0
output application/json
---
payload filter (account) -> account.active == true

Output:

[
  { "id": "A1", "name": "Acme", "active": true },
  { "id": "A3", "name": "Northern Trail", "active": true }
]

The result remains an array, containing only records for which the expression evaluates to true.

Filter and Then Transform

A common requirement is to select records and then reshape them.

Keep the two intentions visible:

payload
    filter (account) -> account.active == true
    map (account) -> {
        accountId: account.id,
        accountName: account.name
    }

This reads naturally:

1. keep active accounts;
2. transform the remaining accounts.

Trying to make map perform both selection and transformation often produces less readable code.

Combining Conditions

Suppose only active US accounts should continue:

payload filter (account) ->
    account.active == true and
    account.country == "US"

For more complex rules, formatting each condition on its own line makes the selection logic easier to review.

Filtering Numeric Values

Consider orders:

[
  { "orderId": "O1", "total": 250 },
  { "orderId": "O2", "total": 1250 },
  { "orderId": "O3", "total": 800 }
]

Keep orders worth at least 1000:

payload filter (order) -> (order.total default 0) >= 1000

Using default 0 here establishes how a missing total should behave. That is a business decision: another integration might reject a record with a missing total rather than treating it as zero.

Filtering Null Values

If an array itself contains null entries:

[
  "A",
  null,
  "B",
  null
]

You can remove them with:

payload filter ($ != null)

Output:

[
  "A",
  "B"
]

For business records, however, consider whether silently dropping incomplete data is appropriate. Sometimes the correct action is validation and error handling rather than filtering.

Filtering Blank Strings

Suppose the input is:

[
  "Salesforce",
  "",
  "  ",
  "MuleSoft"
]

A whitespace-aware filter can be written as:

payload filter (value) -> !isEmpty(trim(value default ""))

Output:

[
  "Salesforce",
  "MuleSoft"
]

This is normalization logic, not merely null handling. Be explicit about whether blank values should be removed, retained, or reported as invalid.

Filtering Nested Arrays

Suppose each account has contacts and only contacts with email addresses should be sent downstream:

payload map (account) -> {
    id: account.Id,
    name: account.Name,
    contacts: (account.Contacts default [])
        filter (contact) -> contact.Email != null
        map (contact) -> {
            id: contact.Id,
            email: contact.Email
        }
}

Here the outer map transforms accounts, while the nested filter selects eligible contacts.

Use a Named Function for Repeated Rules

When a filter condition becomes important business logic, give it a name.

%dw 2.0
output application/json
fun eligible(account) =
    account.active == true and
    account.country == "US" and
    (account.annualRevenue default 0) >= 1000000
---
payload filter eligible($)

A named function makes the main transformation easier to scan and gives the eligibility rule a clear boundary.

Do Not Use filter to Hide Bad Records

There is an important difference between:

record is intentionally out of scope

and:

record is malformed and cannot be processed

filter is a natural choice for the first case. The second case may require validation, logging, an Error Hospital, or another recovery mechanism.

Silently filtering malformed records can make production data loss difficult to detect.

Quick Decision Guide

Requirement	Typical approach
Keep matching array items	filter
Transform every array item	map
Select records and reshape them	filter then map
Remove null array entries	filter ($ != null)
Apply a reusable eligibility rule	named function + filter
Handle invalid records	validation/error handling rather than silently filtering

Practical Rule

Use filter when you can clearly answer:

Which records are intentionally allowed to continue?

Then use map separately to answer:

What should those records look like next?

Keeping selection and transformation as separate ideas usually makes DataWeave easier to understand and maintain.