解ける happens to be the potential form of transitive 解く:
問題が解ける (present)、問題が解けた (past)
↑ intransitive "solve"
問題が(orを)解ける (present)、問題が(orを)解けた (past)
↑ potential form of transitive 解く "solve"
So grammatically speaking you can say:
やっと難しい問題が解けた。 The problem finally got solved / I could finally solve the problem
やっと難しい問題を解けた。 I could finally solve the problem
I know that using the verb 解かす with the を particle would probably be more correct
No, use 解く, not 解かす here:
問題を解く (present)、問題を解いた (past)
↑ transitive "solve"
You're right that とかす is transitive, but it means "melt" "dissolve" "comb out" etc. (溶かす, 融かす, 解かす, 梳かす etc.) You don't say 問題を解かす to mean "solve a problem".
So you can say:
やっと難しい問題を解いた。 I finally solved the problem
I know that transitive sentences normally uses を and intransitive uses が, but how "wrong" is it to use を... Would the other party, not be able to decipher what I am trying to say?
For example, if you said:
*パソコンを壊れた。
*ガソリンの値段を上がりました。
*お風呂を沸いた。
it'd depend on the context but native speakers would usually think that you meant to say:
パソコンが壊れた。 or パソコンを壊した。
ガソリンの値段が上がりました。
お風呂が沸いた。 or maybe お風呂を沸かした。