Sticky flag "y", searching at position

The flag y allows to perform the search at the given position in the source string.

To grasp the use case of y flag, and better understand the ways of regexps, let’s explore a practical example.

One of common tasks for regexps is “lexical analysis”: we get a text, e.g. in a programming language, and need to find its structural elements. For instance, HTML has tags and attributes, JavaScript code has functions, variables, and so on.

Writing lexical analyzers is a special area, with its own tools and algorithms, so we don’t go deep in there, but there’s a common task: to read something at the given position.

E.g. we have a code string let varName = "value", and we need to read the variable name from it, that starts at position 4.

We’ll look for variable name using regexp \w+. Actually, JavaScript variable names need a bit more complex regexp for accurate matching, but here it doesn’t matter.

• A call to str.match(/\w+/) will find only the first word in the line (let). That’s not it.
• We can add the flag g. But then the call str.match(/\w+/g) will look for all words in the text, while we need one word at position 4. Again, not what we need.

So, how to search for a regexp exactly at the given position?

Let’s try using method regexp.exec(str).

For a regexp without flags g and y, this method looks only for the first match, it works exactly like str.match(regexp).

…But if there’s flag g, then it performs the search in str, starting from position stored in the regexp.lastIndex property. And, if it finds a match, then sets regexp.lastIndex to the index immediately after the match.

In other words, regexp.lastIndex serves as a starting point for the search, that each regexp.exec(str) call resets to the new value (“after the last match”). That’s only if there’s g flag, of course.

So, successive calls to regexp.exec(str) return matches one after another.

Here’s an example of such calls:

let str = 'let varName'; // Let's find all words in this string
let regexp = /\w+/g;

alert(regexp.lastIndex);

let word1 = regexp.exec(str);
alert(word1[0]); // let (1st word)
alert(regexp.lastIndex);

let word2 = regexp.exec(str);
alert(word2[0]); // varName (2nd word)
alert(regexp.lastIndex);

let word3 = regexp.exec(str);
alert(word3);
alert(regexp.lastIndex);

We can get all matches in the loop:

let str = 'let varName';
let regexp = /\w+/g;

let result;

while (result = regexp.exec(str)) {
  alert( `Found ${result[0]} at position ${result.index}` );
  // Found let at position 0, then
  // Found varName at position 4
}

Such use of regexp.exec is an alternative to method str.matchAll, with a bit more control over the process.

Let’s go back to our task.

We can manually set lastIndex to 4, to start the search from the given position!

Like this:

let str = 'let varName = "value"';

let regexp = /\w+/g; // without flag "g", property lastIndex is ignored

regexp.lastIndex = 4;

let word = regexp.exec(str);
alert(word); // varName

Hooray! Problem solved!

We performed a search of \w+, starting from position regexp.lastIndex = 4.

The result is correct.

…But wait, not so fast.

Please note: the regexp.exec call starts searching at position lastIndex and then goes further. If there’s no word at position lastIndex, but it’s somewhere after it, then it will be found:

let str = 'let varName = "value"';

let regexp = /\w+/g;

// start the search from position 3
regexp.lastIndex = 3;

let word = regexp.exec(str);
// found the match at position 4
alert(word[0]); // varName
alert(word.index); // 4

For some tasks, including the lexical analysis, that’s just wrong. We need to find a match exactly at the given position at the text, not somewhere after it. And that’s what the flag y is for.

The flag y makes regexp.exec to search exactly at position lastIndex, not “starting from” it.

Here’s the same search with flag y:

let str = 'let varName = "value"';

let regexp = /\w+/y;

regexp.lastIndex = 3;
alert( regexp.exec(str) ); // null (there's a space at position 3, not a word)

regexp.lastIndex = 4;
alert( regexp.exec(str) ); // varName (word at position 4)

As we can see, regexp /\w+/y doesn’t match at position 3 (unlike the flag g), but matches at position 4.

Not only that’s what we need, there’s an important performance gain when using flag y.

Imagine, we have a long text, and there are no matches in it, at all. Then a search with flag g will go till the end of the text and find nothing, and this will take significantly more time than the search with flag y, that checks only the exact position.

In tasks like lexical analysis, there are usually many searches at an exact position, to check what we have there. Using flag y is the key for correct implementations and a good performance.

Common mistakes

• Skipping the small examples and then missing the exact rule that Sticky flag "y", searching at position depends on.
• Copying code without changing one value at a time to see which part controls the result.
• Treating similar-looking syntax or APIs as interchangeable before checking their edge cases.

Summary

• Sticky flag "y", searching at position gives you one more piece of the JavaScript mental model.
• The examples in this lesson are the quickest way to check whether the rule is clear.
• Revisit the key terms when you meet the same idea in later chapters.

Practice

• Rewrite one example from this lesson without looking at the original, then run it and compare the result.
• Change one input, operator, method call, or option in a code sample and predict what will happen before running it.
• Explain Sticky flag "y", searching at position in your own words as if you were reviewing it with another learner.

Keep learning

Continue with Methods of RegExp and String when you are ready for the next lesson.