The source data represents interaction between people. Some are internal, others are external.
The internals are recorded in the Users
table (represented as the Users
CTE in this demonstration).
Each Entries
record is identified by an ID (ItemID
), the time of the interaction (Sequence
) and who performed the interaction (UserID
).
The goal is to have a single line per ItemID
with the following columns:
ItemID
- self explanatoryFirstSequence
-Sequence
value of first interaction (remember in reality this is a time-stamp)firstInternal
-UserID
of first user thatIsInternal
FirstInternalSequence
-Sequence
offirstInternal
CountPerFirstInternal
- A count of all interactions byfirstInternal
userCountAllInternal
- A count of all interaction with anyIsInternal
userCountAll
- Count of all interactions forItemID
LastSequence
- The last interaction forItemID
- allows to measure the 'age' of the interaction.
----- Demo source data BEGIN
WITH Users AS (
SELECT 'A' AS UserID UNION ALL
SELECT 'B' AS UserID
)
, Entries AS (
SELECT 10001 ItemID, 'X' AS UserID, 101 AS Sequence UNION ALL
SELECT 10001 ItemID, 'A' AS UserID, 102 AS Sequence UNION ALL
SELECT 10001 ItemID, 'X' AS UserID, 103 AS Sequence UNION ALL
SELECT 10001 ItemID, 'B' AS UserID, 104 AS Sequence UNION ALL
SELECT 10001 ItemID, 'X' AS UserID, 105 AS Sequence UNION ALL
SELECT 10001 ItemID, 'A' AS UserID, 106 AS Sequence
UNION ALL
SELECT 10020 ItemID, 'Y' AS UserID, 201 AS Sequence UNION ALL
SELECT 10020 ItemID, 'Y' AS UserID, 202 AS Sequence UNION ALL
SELECT 10020 ItemID, 'B' AS UserID, 203 AS Sequence UNION ALL
SELECT 10020 ItemID, 'Y' AS UserID, 204 AS Sequence UNION ALL
SELECT 10020 ItemID, 'A' AS UserID, 205 AS Sequence UNION ALL
SELECT 10020 ItemID, 'Y' AS UserID, 206 AS Sequence UNION ALL
SELECT 10020 ItemID, 'B' AS UserID, 207 AS Sequence UNION ALL
SELECT 10020 ItemID, 'B' AS UserID, 208 AS Sequence
UNION ALL
SELECT 10300 ItemID, 'A' AS UserID, 301 AS Sequence UNION ALL
SELECT 10300 ItemID, 'Z' AS UserID, 302 AS Sequence UNION ALL
SELECT 10300 ItemID, 'Z' AS UserID, 303 AS Sequence UNION ALL
SELECT 10300 ItemID, 'Z' AS UserID, 304 AS Sequence UNION ALL
SELECT 10300 ItemID, 'A' AS UserID, 305 AS Sequence UNION ALL
SELECT 10300 ItemID, 'Z' AS UserID, 306 AS Sequence UNION ALL
SELECT 10300 ItemID, 'A' AS UserID, 307 AS Sequence
)
----- Demo source data END
----- Code I am asking about
, Src AS (
SELECT
e.ItemID
, e.UserID
, e.Sequence
, CASE WHEN u.UserID IS NULL THEN 0 ELSE 1 END AS IsInternal
FROM Entries AS e
LEFT JOIN Users as u
ON u.UserID = e.UserID
)
, Src_UserID AS (
SELECT
*
, ROW_NUMBER() OVER (
PARTITION BY Src_UserID.ItemID, Src_UserID.IsInternal
ORDER BY Src_UserID.FirstUserSequence
) AS RC
FROM (
SELECT
src.ItemID
, src.IsInternal
, src.UserID
, COUNT(*) AS CountPerUser
, MIN(src.Sequence) AS FirstUserSequence
FROM src
GROUP BY src.ItemID, src.IsInternal, src.UserID
) as Src_UserID
)
, Src_Items AS (
SELECT
src.ItemID
, COUNT(*) AS CountAll
, SUM(IsInternal) AS CountAllInternal
, MIN(src.Sequence) AS FirstSequence
, MAX(src.Sequence) AS LastSequence
FROM src
GROUP BY src.ItemID
)
, Src_FirstInternal AS (
SELECT
src.ItemID
, src.UserID AS firstInternal
, src.CountPerUser AS CountPerFirstInternal
, MIN(src.FirstUserSequence) AS FirstInternalSequence
FROM Src_UserID AS src
WHERE src.IsInternal = 1 AND src.RC = 1
GROUP BY src.ItemID, src.IsInternal, src.UserID, src.CountPerUser
)
SELECT
s0.ItemID
, s0.FirstSequence
, s1.firstInternal
, s1.FirstInternalSequence
, s1.CountPerFirstInternal
, s0.CountAllInternal
, s0.CountAll
, s0.LastSequence
FROM Src_Items as s0
JOIN Src_FirstInternal AS s1
ON s1.ItemID = s0.ItemID
Difference between the Demo code and real life:
- Items are in tables and not
UNION ALL CTE
s - The list represented by
Entries
in the demo, in reality is 800K rows, and takes 8 seconds to retrieve. Sequence
column is actually a date.
Execution plan:
Inspecting the code in the Execution Plan, I see that most of the time is spent on SORT
, and it occurs 3 times.
Goal
I'm trying to get the above query to perform better. Running this on even on a limited set of results takes ages. I wonder if there is a better way of writing this query.
Entries
etc. in a#TempTable
instead of a giant CTE withUNION ALL
. \$\endgroup\$