1
\$\begingroup\$

I tried to find the correspondences between columns in two datasets, which is called schema matching. I implemented the algorithm like a c-style program so that I could implement it easily but I have noticed my logic is very bad.

Could someone look at it and guide me on how I can improve the stuff using plain-ruby methods?

I am not allowed to use cronjob or other background jobs, etc.

require 'csv'

imdb_csv = CSV.read("imdb.csv", :headers => true)
rotten_tomatoes_csv = CSV.read("rotten_tomatoes.csv", :headers => true)
g    = [["imdb.Name", "rt.Name"],["imdb.YearRange", "rt.Year"],["imdb.ReleaseDate", "rt.ReleaseDate"],["imdb.Director", "rt.Director"],["imdb.Creator", "rt.Creator"], ["imdb.Cast", "rt.Cast"],["imdb.Duration", "rt.Duration"],["imdb.RatingValue", "rt.RatingValue"],["imdb.Genre", "rt.Genre"],["imdb.Description", "rt.Description"]]

imdb = imdb_csv.to_a
rt = rotten_tomatoes_csv.to_a

def get_list(imdb, rt, g)
  imdb_rows = imdb.count
  imdb_columns = imdb[0].count
  rt_rows = rt.count
  rt_columns = rt[0].count
  list = []

  for j in 0..(imdb_columns-1) do
    d = 0
    for l in 0..(rt_columns-1) do
      sim_total = 0
      for i in 1..(imdb_rows-1) do

        str1 = imdb[i][j].nil? ? "" : imdb[i][j].delete(" \t\r\n")
        length1 = str1.nil? ? 0 : str1.length

        sim_single = 0

        for k in 1..(rt_rows-1) do

          str2 = rt[k][l].nil? ? "" : rt[k][l].delete(" \t\r\n")
          length2 = str2.nil? ? 0 : str2.length

          distance = 0
          m = 0

          while m < length1 and m < length2 do
            if str1[m] == str2[m]
              distance = distance + 1
            end
            m = m + 1
          end

          similarity = 0

          if ((distance == length1 and distance >= (length2/3)) or (distance == length2 and distance >= (length1/3))) and str1 != "" and str2 != ""
            similarity = 1
          end


          sim_single = sim_single + similarity


        end

        sim_total = sim_total + sim_single

      end


      if sim_total > d and sim_total > 0
        list << ["imdb.#{imdb[0][j]}", "rt.#{rt[0][l]}"]
        d = sim_total
      end

    end

    printf("===== %d", j)
  end
\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.