Wednesday, December 30, 2015

SQL, many small queries or one big queries?


When writing an application, it seems conceptually simpler to make many small queries. On a small scale, this seems to be okay. What about large scale applications? Would the application take a performance hit?

Here's some pseudocode that given a list of employee ids, gives you a list of employee names:

      //simple function that returns a name given an employee id
      func getName(var id) {
              data:= sqlQuery("SELECT employee.name FROM employee WHERE employee.id = " + id + ";")
              return data;
      }

      func Main(){
            Int[] ids := [1, 2 3]
            Str[] names := []
            ids.forEach id => do
                  names.add(getName(id))
            end
      }

For me, this seems clean and easy to read. Every time that you want an name, provide the method with an id. Simple. But does this solution scale? What if the list of ids was 50,000 names, or even 50 million? The overhead of the DB parsing and processing each request and potential network traffic make the idea that fewer queries are generally faster, make sense. This does make the application code a bit more complex


 func getName(Int[] ids) {
              sqlString:= ""
              ids.forEach id => do
                   sqlString += "or employee.id = " + id
              end
              Str[] names := sqlQuery("SELECT employee.name FROM employee WHERE employee.id =" + sqlString +";")
              return names ;
      }

 func Main(){
            Int[] ids := generateRandomIds(50000)
            Str[] names := getNames(ids)
         
      }


Here's a couple of posts that I read about the subject:

  • http://dba.stackexchange.com/questions/76973/what-is-faster-one-big-query-or-many-small-queries
  • https://technet.microsoft.com/en-us/library/ms190623(v=sql.105).aspx

No comments:

Post a Comment