It looks nice in my Matlab 2016b/Win10. Setting the Alpha blending enables the OpenGL renderer. If this causes troubles, try:
Does this help? Do you have the current versions of the drivers installed? Which OS are you using? Can you post a screenshot such that "looks jagged" gets more clear?