I do the same sometimes, we were never able to figure out if it had a negative effect on the transmission though. From 1 to 2, it allows you to shift into it without the button being pressed. Then 2 to D4 and finally D4 to N is the same. Downshifting from D4 to 2 and 2 to 1 does require the button being pressed though (I only have downshifted to test and such). About 40,000 miles (55,000 total) on the car and no issues, my transmission shifts beautifully. But you never know...