Skip to main content

OpenTelemetry Remote Management Hash and Mask Rules

OpenTelemetry Remote Management Hash Rules​

A hash rule is a processing rule that allows you to replace an expression with a hash code generated for that value. Hashed data is completely hidden (obfuscated) before being sent to Sumo Logic. This can be very useful in situations where certain types of data must not leave your premises, such as credit card numbers and Social Security numbers. Each unique value will have a unique hash code.

The hash algorithm used is SHA-256.

Ingestion volume is calculated after the hash filter is applied. If the hash reduces the log size, the smaller size will be measured against ingestion limits.

note

Currently available for all standard STs, except Windows and Syslog.

How it works​

When you add a hash rule action to your processing rules, you need to provide two inputs:

  1. Expression: A regular expression that must contain exactly one capture group ( ). The string value matched by this capture group will be hashed using SHA-256. If multiple parts of the string need to be hashed, add additional hashing rules for them.

  2. Replacement Format: The formatted replacement string that will replace the matching string in the log. Use %s to refer to the hashed value from the SHA-256 function. The %s reference is mandatory and can only be used once.

Examples​

Hash a password​

For example, to hash the password Welcome123 from this log:

user=sumo password=Welcome123

You could use the following configuration:

Expression:

password=([A-Za-z0-9]+)

Replacement Format:

password=%s

Result:

  • Matching string: password=Welcome123
  • Capture group: Welcome123 (this value is hashed)
  • Output log: user=sumo password=<hashed_value>

Where <hashed_value> is the SHA-256 hash of Welcome123.

Hash member IDs​

To hash member IDs from this log:

2012-05-16 09:43:39,607 -0700 DEBUG [hostId=prod-cass-raw-8] [module=RAW] [logger=scala.raw.InboundRawProtocolHandler] [memberid=dan@demo.com] [remote_ip=98.248.40.103] [web_session=19zefhqy...] [session=80F1BD83AEBDF4FB] [customer=0000000000000005] [call=InboundRawProtocol.getMessages]

You could use the following configuration:

Expression:

memberid=([^\]]+)

Replacement Format:

memberid=%s

Resulting hashed log:

2012-05-16 09:43:39,607 -0700 DEBUG [hostId=prod-cass-raw-8] [module=RAW] [logger=scala.raw.InboundRawProtocolHandler] [memberid=906e9cc124c8e1085b10e1cec4cc6526f3637558be361d3b4bb54bb537e49a49] [remote_ip=98.248.40.103] [web_session=19zefhqy...] [session=80F1BD83AEBDF4FB] [customer=0000000000000005] [call=InboundRawProtocol.getMessages]
info

Any hashing expression should be tested and verified on a sample source file before being applied to your production logs.

Rules and limitations​

  • The regular expression must contain exactly one capture group enclosed in ( ). Values inside this capture group will be hashed. If multiple parts of the string need to be hashed, add additional hashing rules for them.

  • You can use an anchor to detect specific values in your logs. Only the value within the capture group will be hashed.

  • The hash algorithm is SHA-256 (MD5 is not supported for OpenTelemetry collectors).

  • Make sure you do not specify a regular expression that matches a full log line. Doing so will hash the entire log line.

  • The replacement format must include %s exactly once to reference the hashed value.

  • Do not unnecessarily match on more of the log than needed. Use precise regular expressions to ensure that only the intended sensitive information is hashed, not the surrounding context.

  • Each unique value will produce a unique hash code. The same input value will always produce the same hash output, allowing you to correlate occurrences while keeping the actual value hidden.

OpenTelemetry Remote Management Mask Rules​

note

This document does not cover masking logs for Windows source templates. For details on masking logs for Windows, refer to OpenTelemetry Remote Management Windows Source Template Mask Rules.

A mask rule is a processing rule that hides irrelevant or sensitive information from logs before they are ingested. When you create a mask rule, the selected expression will be replaced with a mask string before the data is sent to Sumo Logic. You can either specify a custom mask string or use the default "#####".

Ingestion volume is calculated after applying the mask filter. If the mask reduces the size of the log, the smaller size will be measured against ingestion limits. Masking is an effective method to reduce overall ingestion volume.

Examples​

Mask an email address​

For example, to mask the email address dan@demo.com from this log:

2018-05-16 09:43:39,607 -0700 DEBUG [hostId=prod-cass-raw-8] [module=RAW] [logger=scala.raw.InboundRawProtocolHandler] [auth=User:dan@demo.com] [remote_ip=98.248.40.103] [web_session=19zefhqy...] [session=80F1BD83AEBDF4FB] [customer=0000000000000005] [call=InboundRawProtocol.getMessages]

You could use the following filter expression:

auth=User:.*\.com

Using the masking string auth=User:AAA would produce the following result:

2018-05-16 09:43:39,607 -0700 DEBUG [hostId=prod-cass-raw-8] [module=RAW] [logger=scala.raw.InboundRawProtocolHandler] [auth=User:AAA] [remote_ip=98.248.40.103] [web_session=19zefhqy...] [session=80F1BD83AEBDF4FB] [customer=0000000000000005] [call=InboundRawProtocol.getMessages]

info

Any masking expression should be tested and verified with a sample source file before applying it to your production logs.

Mask credit card numbers​

You can mask credit card numbers from log messages using a regular expression within a mask rule. Once masked with a known string, you can then perform a search for that string within your logs to detect if credit card numbers may be leaking into your log files.

To mask credit card numbers in logs, you can use a masking filter with the following regular expression. The following regular expression can be used within a masking filter to mask American Express, Visa (16 digit only), Mastercard, and Discover credit card numbers:

((?:(?:4\d{3})|(?:5[1-5]\d{2})|6(?:011|5[0-9]{2}))(?:-?|\040?)(?:\d{4}(?:-?|\040?)){3}|(?:3[4,7]\d{2})(?:-?|\040?)\d{6}(?:-?|\040?)\d{5})

This regular expression covers instances where the number includes dashes, spaces, or is a solid string of numbers.

Samples include:

  • American Express. 3711-078176-01234  |  371107817601234  |  3711 078176 01234
  • Visa. 4123-5123-6123-7123  |  4123512361237123  |  4123 5123 6123 7123
  • Master Card. 5123-4123-6123-7123  |  5123412361237123  |  5123 4123 6123 7123
  • Discover. 6011-0009-9013-9424  |  6500000000000002  |  6011 0009 9013 9424

Rules and limitations​

  • Expressions that you want masked must be selected by the regular expression you given. And the masking string provided will mask whole of the string which is selected by the regular expression.

    For example, for this log message:

    { "reqHdr":{ "auth":"Basic ksoe9wudkej2lfj*jshd6sl.cmei=", "cookie":"$Version=0; JSESSIONID=6C1BR5DAB897346B70FD2CA7SD4639.localhost_bc; $Path=/" } }

    You would use the following as a mask expression to mask the auth parameter's token:

    "auth"\s*:\s*"Basic\s*[^"]+"

    Applying the masking string "auth":"#####", the log output will be:

    { "reqHdr": { "auth":"#####", "cookie":"$Version=0; JSESSIONID=6C1BR5DAB897346B70FD2CA7SD4639.localhost_bc; $Path=/" } }

  • Do not unnecessarily match on more of the log than needed. As seen in the previous example, avoid using overly broad expressions that could mask the entire log. This ensures that only the sensitive information is masked, not the whole log entry.

    (?s).*auth"\s*:\s*"Basic\s*([^"]+)".*(?s)

  • Avoid regular expressions that match an entire log line, as this will result in the entire line being masked.

  • To mask values spanning multiple lines, use the single-line modifier (?s). For example:

    auth=User\:(.*(?s).*session=.*?)\]
note
  • Masking utilizes the replace_pattern OTTL function. In this function:
    • Escape $ as $$ to bypass environment variable substitution logic.
    • Use $$$ to include a literal $.
  • When masking strings containing special characters like double quotes (") and backslashes (\), these characters will be escaped by a backslash when masking the logs.
Status
Legal
Privacy Statement
Terms of Use
CA Privacy Notice

Copyright © 2026 by Sumo Logic, Inc.