Typically, the first Doctor you watch...
...becomes "your Doctor". Tom Baker (The Fourth) is my Doctor, from my college viewing habits in the early 80's.
Matt's good, I liked him. Did a couple of my all time favorite episodes, namely "Vincent and the Doctor (fifth series, episode 10), which is both fun, scary, and has an ending so sweet that you better have tissues nearby. I'd tell you more, but spoilers!
Matt's Dr is wonky, but there is a good reason for that. All the doctors are eccentric as hell, but beyond that they all have rather distinct characteristics. That's part of the fun.
With a series that is fifty years old, starting in the middle is the only way to start, and the beginning of Matt's Doctor is a good place to start.
Hope you enjoy your travels with the Doctor.