It's unlikely that the number of users pointing to the master as a preferred site is related to the performance issue that you're seeing. Uploads/Downloads can produce long running transactions, but they don't use much in the way of server resources. What's likely happening is that these CAD transactions are performing some server side operation on a large data set, which uses up the Method Servers available memory. Though, to know for sure what is causing the issue, you should get a series of thread dumps while the issue is occuring. Thread dumps will show you exactly what the MS is doing while the issue is occurring. You could also check the GC logs to see if the MS is going into full GC when the performance issue occurs.
