I know not what I should make of some things, if I were to equate polygamy with adultery.
[1] Then shall the kingdom of heaven be likened unto ten virgins, which took their lamps, and went forth to meet the bridegroom.
[2] And five of them were wise, and five were foolish.
[3] They that were foolish took their lamps, and took no oil with them:
[4] But the wise took oil in their vessels with their lamps.
[5] While the bridegroom tarried, they all slumbered and slept.
[6] And at midnight there was a cry made, Behold, the bridegroom cometh; go ye out to meet him.
[7] Then all those virgins arose, and trimmed their lamps.
[8] And the foolish said unto the wise, Give us of your oil; for our lamps are gone out.
[9] But the wise answered, saying, Not so; lest there be not enough for us and you: but go ye rather to them that sell, and buy for yourselves.
[10] And while they went to buy, the bridegroom came; and they that were ready went in with him to the marriage: and the door was shut.
[11] Afterward came also the other virgins, saying, Lord, Lord, open to us.
[12] But he answered and said, Verily I say unto you, I know you not.
[13] Watch therefore, for ye know neither the day nor the hour wherein the Son of man cometh.
I do see your point about the putting away of one wife while taking another. One problem, as I understand it, is that the second wife serves as a substitute for the first. While one in such circumstances might try to argue that both wives are legit, we all know such an argument would be disingenuous.
At its root, what is adultery about? That's the key question. It is about taking that which is the rightful property of one and giving it to another. This is why spiritual adultery is possible.
People tend to forget that the body of the husband
is the property of the wife, and hers is his. I have had friends express desire for married women. I ask them "What do you want with a man's body?" They seem averse to thinking about it that way.
So, understanding whose property is whose, let's look at divorce and remarriage: A man takes that which he does not own, and attempts to transfer it to an unlawful owner. Bogus! That's sin no matter how you try to excuse it.
There are several practical issues, and I wouldn't particularly recommend polygamy these days. I would say a man shouldn't even consider it except at the request of his first wive (the rightful owner). ...And that won't happen too often where I live!