Skip to main content
added 97 characters in body
Source Link
RUser4512
  • 1.1k
  • 6
  • 21

I am running a simple OCaml program which opens a CSV file with a pseudo dict-reader and hashes "key" + "value" (where key and values are strings). Then some counts are evaluated on the hashes (but it is not really relevant for what follows).

After a quick look at the default OCaml profiler (gprof), I noticed that my program was mostly spending time in hashing elements (I don't know what caml_page_table_lookup does though).

Each sample counts as 0.01 seconds.
  %   cumulative   self              self     total           
 time   seconds   seconds    calls   s/call   s/call  name    
  8.27      4.12     4.12 97999951     0.00     0.00  caml_hash
  7.53      7.88     3.76 97999951     0.00     0.00  caml_hash_mix_string
  5.12     10.43     2.55 136894800     0.00     0.00  caml_page_table_lookup

The hashtables are seldomly called in my code :

let get_indices dict n = Hashtbl.fold (fun k v acc -> ((Hashtbl.hash (k^v)) mod n)  :: acc) dict [] ;;

And here :

let dict_reader file_path = 
    let line_stream = read_lines file_path in
    let header = split_line (Stream.next line_stream) in
    Stream.from
      (fun _ ->
         try Some (to_dict header (split_line (Stream.next line_stream))) with End_of_file -> None)

Is there a smart way to optimize these bottlenecks ? I feel like there is something redundant in evaluating hashes and storing values in hashtables...

I am running a simple OCaml program which opens a CSV file with a pseudo dict-reader and hashes "key" + "value" (where key and values are strings). Then some counts are evaluated on the hashes (but it is not really relevant for what follows).

After a quick look at the default OCaml profiler (gprof), I noticed that my program was mostly spending time in hashing elements (I don't know what caml_page_table_lookup does though).

Each sample counts as 0.01 seconds.
  %   cumulative   self              self     total           
 time   seconds   seconds    calls   s/call   s/call  name    
  8.27      4.12     4.12 97999951     0.00     0.00  caml_hash
  7.53      7.88     3.76 97999951     0.00     0.00  caml_hash_mix_string
  5.12     10.43     2.55 136894800     0.00     0.00  caml_page_table_lookup

The hashtables are seldomly called in my code :

let get_indices dict n = Hashtbl.fold (fun k v acc -> ((Hashtbl.hash (k^v)) mod n)  :: acc) dict [] ;;

And here :

let dict_reader file_path = 
    let line_stream = read_lines file_path in
    let header = split_line (Stream.next line_stream) in
    Stream.from
      (fun _ ->
         try Some (to_dict header (split_line (Stream.next line_stream))) with End_of_file -> None)

Is there a smart way to optimize these bottlenecks ?

I am running a simple OCaml program which opens a CSV file with a pseudo dict-reader and hashes "key" + "value" (where key and values are strings). Then some counts are evaluated on the hashes (but it is not really relevant for what follows).

After a quick look at the default OCaml profiler (gprof), I noticed that my program was mostly spending time in hashing elements (I don't know what caml_page_table_lookup does though).

Each sample counts as 0.01 seconds.
  %   cumulative   self              self     total           
 time   seconds   seconds    calls   s/call   s/call  name    
  8.27      4.12     4.12 97999951     0.00     0.00  caml_hash
  7.53      7.88     3.76 97999951     0.00     0.00  caml_hash_mix_string
  5.12     10.43     2.55 136894800     0.00     0.00  caml_page_table_lookup

The hashtables are seldomly called in my code :

let get_indices dict n = Hashtbl.fold (fun k v acc -> ((Hashtbl.hash (k^v)) mod n)  :: acc) dict [] ;;

And here :

let dict_reader file_path = 
    let line_stream = read_lines file_path in
    let header = split_line (Stream.next line_stream) in
    Stream.from
      (fun _ ->
         try Some (to_dict header (split_line (Stream.next line_stream))) with End_of_file -> None)

Is there a smart way to optimize these bottlenecks ? I feel like there is something redundant in evaluating hashes and storing values in hashtables...

Source Link
RUser4512
  • 1.1k
  • 6
  • 21

Optimize hash OCaml

I am running a simple OCaml program which opens a CSV file with a pseudo dict-reader and hashes "key" + "value" (where key and values are strings). Then some counts are evaluated on the hashes (but it is not really relevant for what follows).

After a quick look at the default OCaml profiler (gprof), I noticed that my program was mostly spending time in hashing elements (I don't know what caml_page_table_lookup does though).

Each sample counts as 0.01 seconds.
  %   cumulative   self              self     total           
 time   seconds   seconds    calls   s/call   s/call  name    
  8.27      4.12     4.12 97999951     0.00     0.00  caml_hash
  7.53      7.88     3.76 97999951     0.00     0.00  caml_hash_mix_string
  5.12     10.43     2.55 136894800     0.00     0.00  caml_page_table_lookup

The hashtables are seldomly called in my code :

let get_indices dict n = Hashtbl.fold (fun k v acc -> ((Hashtbl.hash (k^v)) mod n)  :: acc) dict [] ;;

And here :

let dict_reader file_path = 
    let line_stream = read_lines file_path in
    let header = split_line (Stream.next line_stream) in
    Stream.from
      (fun _ ->
         try Some (to_dict header (split_line (Stream.next line_stream))) with End_of_file -> None)

Is there a smart way to optimize these bottlenecks ?