Using OFFSET FETCH NEXT performing slow when actual row count is lower than requested

In SQL Server 2022 I have a stored procedure that retrieves a list of items. It performs well except when the actual row count of items is lower than specified in the FETCH NEXT statement.

Here is anonymized query:

WITHCTE_CORE_VIEW AS (    SELECT         T1.Column1,        T1.Column2,        T1.Column3,        T1.Column4,        T1.Column5,        T1.Column6,        T1.Column7,        T1.Column8,        T2.Column9 AS Column10,        T2.Column11 AS Column12,        T2.Column13 AS Column14,        T3.Column13 AS Column15,        T2.Column16,        T2.Column17    FROM     Table1 T1    INNER JOIN Table2 T2 ON T1.Column1 = T2.Column18 AND T2.Column19 = 2    INNER JOIN Table2 T3 ON T1.Column1 = T3.Column18 AND T3.Column19 = 1    LEFT JOIN @UserGroups T4 ON T2.Column11 = T4.Column20 AND T4.Column19 = 0    WHERE     T1.Column21 = 0     AND T1.Column19 = 0    AND T2.Column9 = @VarCompanyId     AND (T2.Column13 = @VarUserId OR T4.Column20 IS NOT NULL)),CTE_FILTER_STATUS AS (    SELECT value AS Column2 FROM OPENJSON(@VarFilterStatusesJson) WITH (value INT '$')),CTE_FILTER_TAGS AS (    SELECT T1.Column1    FROM     CTE_CORE_VIEW T1    INNER JOIN Table3 T5 ON T1.Column1 = T5.Column22    INNER JOIN @VarFilterOwnerTagIds T6 ON T5.Column23 = T6.Column20),CTE_TICKETS AS (SELECT    T1.Column1,    T1.Column10,    T1.Column12,    T1.Column14,    T1.Column15,    T1.Column16,    T1.Column17,    ROW_NUMBER() OVER (    ORDER BY        CASE WHEN @VarSortDesc = 1 AND @VarSortColumn = 'SortField1' THEN T1.Column4 END DESC,        CASE WHEN @VarSortDesc = 1 AND @VarSortColumn = 'SortField2' THEN T1.Column1 END DESC,        CASE WHEN @VarSortDesc = 1 AND @VarSortColumn = 'SortField3' THEN T7.Column24 + T7.Column25 END DESC,        CASE WHEN @VarSortDesc = 1 AND @VarSortColumn = 'SortField4' THEN T8.Column26 END DESC,        CASE WHEN @VarSortDesc = 1 AND @VarSortColumn = 'SortField5' THEN COALESCE(T9.Column24, T10.Column24, '') + COALESCE(T9.Column25, T10.Column25, '') + COALESCE(T10.Column27, '') + COALESCE(T10.Column28, '') END DESC,        CASE WHEN @VarSortDesc = 1 AND @VarSortColumn = 'SortField6' THEN COALESCE(T1.Column6, 5) END DESC,        CASE WHEN @VarSortDesc = 0 AND @VarSortColumn = 'SortField1' THEN T1.Column4 END,        CASE WHEN @VarSortDesc = 0 AND @VarSortColumn = 'SortField2' THEN T1.Column1 END,        CASE WHEN @VarSortDesc = 0 AND @VarSortColumn = 'SortField3' THEN T7.Column24 + T7.Column25 END,        CASE WHEN @VarSortDesc = 0 AND @VarSortColumn = 'SortField4' THEN T8.Column26 END,        CASE WHEN @VarSortDesc = 0 AND @VarSortColumn = 'SortField5' THEN COALESCE(T9.Column24, T10.Column24, '') + COALESCE(T9.Column25, T10.Column25, '') + COALESCE(T10.Column27, '') + COALESCE(T10.Column28, '') END,        CASE WHEN @VarSortDesc = 0 AND @VarSortColumn = 'SortField6' THEN COALESCE(T1.Column6, 5) END    ) RowNumberFROM     CTE_CORE_VIEW T1    INNER JOIN Table4 T8 ON T1.Column12 = T8.Column20    INNER JOIN Table5 T10 ON T1.Column15 = T10.Column1    LEFT JOIN Table6 T9 ON T10.Column1 = T9.Column13 AND T9.Column29 = T1.Column10    LEFT JOIN Table5 T7 ON T1.Column14 = T7.Column1WHERE    (        @VarFilterStatusesJson IS NULL OR         (            EXISTS (SELECT 1 FROM CTE_FILTER_STATUS T11 WHERE T11.Column2 = T1.Column2)        )    )    AND    (        @VarFilterNumber IS NULL OR        (            T1.Column7 LIKE '%'+ @VarFilterNumber +'%'        )    )    AND     (        @VarFilterTitle IS NULL OR        (            T1.Column8 LIKE '%'+ @VarFilterTitle +'%'        )    )    AND    (        @VarFilterSource IS NULL OR        (            T1.Column3 = @VarFilterSource        )    )    AND    (        @VarFilterPriority IS NULL OR        (            COALESCE(T1.Column6, 5) = @VarFilterPriority        )    )    AND    (        @VarFilterCreatedOnStart IS NULL OR        (            T1.Column5 >= @VarFilterCreatedOnStart        )    )    AND     (        @VarFilterCreatedOnEnd IS NULL OR        (            T1.Column5 <= @VarFilterCreatedOnEnd        )    )    AND     (        @VarFilterUpdatedOnStart IS NULL OR        (            T1.Column4 >= @VarFilterUpdatedOnStart        )    )    AND    (        @VarFilterUpdatedOnEnd IS NULL OR        (            T1.Column4 <= @VarFilterUpdatedOnEnd        )    )    AND    (        @VarFilterCustomerIdxJson IS NULL OR        (            EXISTS (SELECT 1 FROM @VarFilterCustomerIds T12 WHERE T12.Column20 = T1.Column15)        )    )    AND     (        @VarFilterOwnerExpertIdxJson IS NULL OR        (             EXISTS (SELECT 1 FROM @VarFilterOwnerExpertIds T13 WHERE T13.Column20 = T1.Column14)        )    )    AND    (        @VarFilterOwnerTeamIdxJson IS NULL OR        (             EXISTS (SELECT 1 FROM @VarFilterOwnerTeamIds T14 WHERE T14.Column20 = T1.Column12)        )    )    AND    (        @VarFilterOwnerCompanyIdxJson IS NULL OR        (            EXISTS (SELECT 1 FROM @VarFilterOwnerCompanyIds T15 WHERE T15.Column20 = T1.Column10)        )    )    AND    (        @VarFilterOwnerTagIdxJson IS NULL OR ```sql        EXISTS         (            SELECT 1 FROM CTE_FILTER_TAGS T16 WHERE T16.Column1 = T1.Column1        )    ))INSERT INTO @VarTicketIds (Column1, Column10, Column12, Column14, Column15, Column16, Column17, RowNumber, Count)SELECT     T1.Column1,    T1.Column10,    T1.Column12,    T1.Column14,    T1.Column15,    T1.Column16,    T1.Column17,    T1.RowNumber,    TicketCount.CountFROM CTE_TICKETS T1CROSS JOIN (SELECT COUNT(*) AS Count FROM CTE_TICKETS) AS TicketCountORDER BYT1.RowNumberOFFSET @VarPageStart ROWS FETCH NEXT @VarPagingSize ROWS ONLYOPTION(RECOMPILE);

Note following line:OFFSET @VarPageStart ROWS FETCH NEXT @VarPagingSize ROWS ONLYIf the rows that the query will return is greater or equal to @VarPagingSize then the query performs just fine. As soon as it is just 1 more item the query adds like 14 seconds to execute.

I believe that it's due to the fact that it just has to scan the whole index to determine if there is another item to grab. My question is - is there any way how to avoid this? For example, somehow when the item count is retrieved to use that somehow to limit the number of rows dynamically.