Guessing game in NASM Win64 Assembly

Question

Still learning Assembly. This program is a simple guessing game to test most of the new things I've learned! Any advice and all topical comments on code optimization and conventions is appreciated!

Compiled as follows using VS2017 x64 Native Tools Command Prompt:

> nasm -g -fwin64 guess.asm

> cl /Zi guess.obj msvcrt.lib legacy_stdio_definitions.lib

guess.asm

;; bits 64
default rel

extern time, srand, rand
extern printf, scanf

%macro str 2
    %2: db %1, 0
%endmacro

; use DQ to define a quad-word, DD to define a double-word, DW to define a word, and DB to define a byte.
section .rdata          ; immutable predefined variables
    str "Enter your guess (1-100): ", prompt
    str "%u", scan_fmt
    str "ERROR! Input was not a number!", scan_fail
    str {"Too high!", 10}, too_high
    str {"Too low!", 10}, too_low
    str {"You guessed it!", 13, 10}, congrats

; use RESQ to reserve a number of quad-words, RESD to reserve a number of double-words, RESW to reserve a number of words, and RESB to reserve a number of bytes.
section .bss            ; mutable undefined variables
    target: resb 4      ; 4 bytes of storage (enough for a 32bit number)

section .text
    global main
main:
    stack_reserve: equ 40      ; 32 + 8
    sub     rsp, stack_reserve ; shadow space for callees + 8 bytes for stack alignment

    ;; calculate the random number
    xor     rax, rax
    mov     rcx, rax        ; clear rax and rcx
    call    time
    mov     rcx, rax
    call    srand
    call    rand

    ;; rand = (rand % 100) + 1
    xor     rdx, rdx        ; clear rdx
    mov     rcx, 100
    div     rcx
    inc     rdx
    mov     [target], rdx
    ;mov     rcx, scan_fmt
    ;call    printf
.check:
    lea     rcx, [prompt]
    call    printf

    ; memory from rsp+0..31 has potentially been stepped on by printf
    ; leave RSP where it is, ready for another call

    ;;; scanf into that 8-byte block of stack space above the shadow space, or into our *own* shadow space
    lea     rdx, [rsp+32]        ; stack addresses are normally 64-bit, can't get away with edx
    lea     rcx, [scan_fmt]
    mov     dword [rdx], 0       ; instead of error check, set n = 0 in case of I/O error
    call    scanf
    cmp     eax, 1               ; success = exactly 1 conversion
    jnz     .scanf_fail          ; error check

    mov     r8d, [rsp+32]        ; r8d: 32-bit unsigned number to be checked

    cmp     r8d, [target]
    jg      .too_high
    jl      .too_low

    lea     rcx, [congrats]
    call    printf
.end:
    ;xor     eax,eax              ; return 0
    movzx   eax, byte [rsp+32]   ; return EXIT_SUCCESS(0) or EXIT_FAILURE(1)
    add     rsp, stack_reserve
    ret
.too_high:
    lea     rcx, [too_high]
    call    printf
    jmp     .check
.too_low:
    lea     rcx, [too_low]
    call    printf
    jmp     .check
.scanf_fail:
    lea     rcx, [scan_fail]
    call    printf
    jmp     .end

Answer 1

The usage of target is not consistent:

target: resb 4      ; 4 bytes of storage (enough for a 32bit number)
...
mov     [target], rdx ; store 8 bytes

That's easy to improve, just use edx.

Using rand like this is not very random, it's some function of the time in seconds. Seconds are long, you could easily start the program twice in the same second. Maybe that's fine for a simple number guessing game. On not-too-old processors there is a simpler and more random way to get a random number though:

.rand_retry:
    rdrand eax
    jnc .rand_retry

But that doesn't work on some older processors (Intel processors older than Ivy Bridge, for example Nehalem or Core2, and AMD processors older than Excavator), and can actually be slower than calling rand. Seeding the random number generator with the TSC value is more portable and less predictable than using time:

rdtsc ; read TSC into edx:eax
mov ecx, eax
call srand
call rand

Since you asked for performance tips (this program is obviously limited by waiting for user input but you can still learn useful things from this):

This remainder calculation is inefficient:

mov     rcx, 100
div     rcx     ; 64bit div, quite slow
inc     rdx

There are other options. For example, for a 2³² input range (which rand normally does not deliver but rdrand does), a very similar-looking fixed-point scale can be used:

mov     ecx, 100
mul     ecx
inc     edx

The mul, combined with using the upper half of the result, effectively multiplies the input by 100 / 2³² so 0xFFFFFFFF maps to 99 and so on. This method can be adjusted for smaller ranges such as up to RAND_MAX by multiplying by a higher constant, namely 100 * 2³² / (RAND_MAX + 1)

The too low/too high cases could be less branchy and can share some code - admittedly they call an IO function anyway, but still. For example:

    cmp     r8d, [target]
    jne     .not_equal
...
.not_equal:
    lea     rcx, [too_high]
    lea     rdx, [too_low]
    ;; if "less than", use the too_low string
    cmovl   rcx, rdx
    call    printf
    jmp     .check