Very nice, kwaw, you've good resources in these questions ...
The Visconti court "..was sharply divided into warring factions of humanists. On one side were Decembrio, Catone Sacco, Antonio da Rho and Lorenzo Valla. On the other side was a party led by the court poets, Antonio Panormita, Antonio Cremona, Francesco Mecenati and Francesco Filelfo..."
As far the relation between Decembrio and Filelfo is involved, it seems difficult to see the tension before 1440 ... actually it seems - at least at the surface - that Filelfo was invited cause Decembrio's activities. Or Filippo Maria in a mood of common sadism invited Filelfo just cause this tension to keep Decembrio under some stress and frustration?
Decembrio was political active and possibly ambitious (later he became a sort of president for the Ambrosian republic) ... things, which would have been watched with suspicious eyes by Filippo Maria.
That Filelfo - generally presented as a difficult character - wouldn't have accepted second place in Milan, is logical, once, that he had set his foot in it.
The tension between Milanese and Florentian scholars (Fuilelfo and Decembrio) are natural cause the war between Florence and Milan.
Panormita .... I stumbled about the following:
http://en.wikipedia.org/wiki/Antonio_Beccadelli
"He became a guest of Filippo Maria Visconti family at Pavia (1430-1433), where he completed his studies and entered the court of the Visconti. He would dedicate himself to philological studies, in particular to the tradition of Plautus.
In 1434, he entered the service of Alfons V of Aragon at Naples. Alfonso was a great patron of the arts, and in this city Beccadelli founded the academy Porticus Antoniana, later known as the Pontaniana, after Giovanni Pontano. At Naples, Beccadelli began a close friendship with Pontano, and introduced the young scholar to the royal chancery of King Alfonso.
Beccadelli and Alfonso shared a great love of culture, and Beccadelli accompanied Alfonso during the vicissitudes of the king's career. When Alfonso became a prisoner in the hands of Filippo Maria Visconti, Duke of Milan, in 1435, Alfonso persuaded his ferocious and crafty captor to let him go by making it plain that it was the interest of Milan not to prevent the victory of the Aragonese party in Naples. Beccadelli, with his former connection to the Milanese court, played a role in these negotiations."
As the situation of 1435 (Alfonso as prisoner played cards with Filippo Maria Visconti) are related to card-playing, this is a very interesting information.
Panormita wrote a very drastical erotic piece of literature, in which homosexual tendencies appear undisguided (the text is in the web somewhere) ca. 1421 and this text was dedicated to the early Cosimo de Medici (!), who in his late life developed a strong favour for "platonic ideas" and helped the young Marsilio Ficino to a villa in Carreggio near Florence, where the platonic academy was founded ...
In the biographical work of the Florentian book-trader Bisticci a story about the cardinal Cesarini is told (Cesarini died 1444, so this story likely relates to a time, when Pope Eugen was in Florence), when a young priest or monk was surprized reading a book of Panormita, which contained erotical pictures and was simply a porno book of early 15th century. The reading and possession of the book was condemned and MUST have led to excommunication - such were the times of Pope Eugen - , but Cesarini reacted mildly and only burnt the book (and was honoured by Bisticci for his understanding of human nature).
I wouldn't think, that there were early competition between Decembrio and Panormita, as it seems, that Decembrio in 1433 was not important enough ... (?) ... but perhaps you've better informations.
Lorenzo Valla ...
http://en.wikipedia.org/wiki/Lorenzo_Valla
was also only till 1433 in Milan (it seems that he was only at Milanese territory, Piaczenza and Pavia) and - as Panormita - didn't spend a long time there. He also left - as Panormita - to the court of Alfonso of Aragon
http://www.bautz.de/bbkl/v/valla_l.shtml
(another German source, which seems to better informed tells, that he arrived 1437 at Alfonso's court and was before occasionally in Milan'; the relation to Decembrio is described as close).
....
"Although the Duke {Filippo} was a bibliophile, for whom many kinds of manuscripts were written, including treatises on law and medicine and Italian translations of the Roman historians whose works he seems especially to have enjoyed, there is little evidence of an interest in books during the early years of his reign... "
This is really an interesting observation ...
It's hard to say, how important the relation from Filippo Maria Visconti to Martiano da Tortona was, who in his youth worked in a difficult time as his teacher and later in other services to him. Martiano da Tortona died 1425 ... and in 1426 Filippo Maria Visconti made the inventory of the library in 1426. This leaves the question, if Filippo Maria had his personal own intellectual connection to his own library by Martiano da Tortona ... and in 1426, when Martiano da Tortona was gone, it was necessary to make the inventory to get some informations, what treasures were still in it.
On the other side, this starting intellectual interests had some logical synchronity to the intellectual developments in Mantua (Vittoria da Feltre) and to Guarino 1429 in Ferrara.