Yup that's it. I found the Borax in the cleaning aisle. It was with the laundry detergent. After you make the ant killer you can use the borax for its intended purpose a laundry booster.
I used a plastic storage container. Combine 1 cup of Sugar half a cup of borax and just enough water to make it runnier than a paste, but not completely dissolved. Now soak a cotton ball in it. Set it out on a piece of paper or scrap cardboard, anything to protect your surface. The ants will be attracted to the cotton ball and by the next day you shouldn't have any ants. Just throw the cotton ball away after the ants have disappeared. I keep the mixture stored just in case any come back, but its been several days and I've only seen the random ant.
This is the borax that I used. Source