Topic Modelling with Python and Jupyter Notebook

Script to change the amount of topics of the model:

%%time
# Build LDA model
number_of_topics = 40
lda_model = gensim.models.LdaModel(corpus=corpus,
                                   id2word=dictionary,
                                   num_topics=number_of_topics,
                                   update_every=20,
                                   passes=100,
                                   alpha='auto')
print("-"*50)
path = "./models_new/"
lda_model.save(path+'dispatch_1864_40.lda')

Topic 36 of 40 topics of a new lda_model

The year 1864 shows a high word count for deserters and prison.

topic40

More interesstingly is topic 31 that shows spikes in April and Nov 1864

‘Proclamation by the President of the Confederate States of America. – The Senate and House of Representatives of the Confederate States of America have signified their desire that a day may be recommended to the people, to be set apart and observed as a day of humiliation, fasting, and prayer, in the language following, to wit:” Reverently recognizing the Providence of God in the affairs of man, and gratefully remembering the guidance, support and deliverance granted to our patriot Fathers in the memorable war which resulted in the independence of the American Colonies, and now reposing in Him our supreme confidence and hope in the present struggle for Civil and Religions Freedom, and for the right to live under a Government of our own choice, and deeply impressed with the conviction that without Him nothing is strong, nothing wise, and nothing enduring; in order that the people of this Confederacy may have the opportunity at the same time of offering their adoration to the great Sovereign of the Universe, of penitently confessing their sins and strengthening their vows and purposes of amendment in humble reliance upon His Gracious and Almighty power:” The Congress of the Confederate States do resolve, That it be recommended to the people of these States that Friday, the 8th day of April next, be set apart and observed as a day of Humiliation, Fasting and Prayer, that Almighty God would so preside over our public counsels and authorities, that He would so inspire our armies and their leaders with wisdom, courage and perseverance, and so manifest Himself in the greatness of His goodness and majesty of His power, that we may be safety and successfully ld through the Chastening to which we are being subjected to the attainment of an honorable peace, so that while we enjoy the blessings of a free and happy Government, we may ascribe to Him the honor and the glory of our independence and prosperity.” A recommendation so congenial to the feelings of the people will receive their hearty concurrence; and it is grateful duty to the Executive to unite with their representatives in inviting them to meet in the Courts of the Most High. Recent events awaken fresh gratitude to the Supreme Ruler of nations. Our enemies have suffered repeated defeats, and a nefarious scheme to burn and plunder our Capital, and to destroy our civil Government by putting to death the chosen servants of the people, has been baffled and act at naught. Our armies have been strengthened; our finances promise rapid progress to a satisfactory condition, and our whole country is animated with a hopeful spirit and a fixed determination to achieve independence. In these circumstances it becomes us, with thankful hearts, to bow ourselves before the throne of the Most High, and while gratefully acknowledging so many mercies confess that our sins are people have justly exposed us to His chastisement. Let us recognize the sufferings which we have been called upon to endure, as administered by a fatherly hand for our improvement, and with resolute courage and patient endurance let us wait on Him for our deliverance. In furtherance of these objects, now, therefore, I Jefferson Davis, President of the Confederate States of America, do issue this my Proclamation, calling upon the people of the said States, in conformity with the desire expressed by their representatives to act apart Friday, the 8th day of April, as a day of Humiliation, Fasting and Prayer, and I do hereby invited them on that day to repair to their several places of public worship and beseech Almighty God to “ preside over our public counsels and so inspire our armies and leaders with wisdom, courage, and perseverance; and to be prominent Himself in the greatness of His in the majesty of His power, that we may blessings of an honorable peace and of free ; and that we, as a people, may ascribe all to the honor and glory of His name” Given under my head and the seal of the Confederate States of America, at the city of Richmond, on this 11th day of March, in the year our Lord one thousand eight hundred and sixty five. Jefferson Davis. By the President. J. P. Benjamin Secretary of State mh 14 – d14Page image of Daily Dispatch, Volume 26, Number 63, page 2. Richmond Dispatch. Monday morning… March 14, 1864.’

Prayers to God who is also responsible for the independence of the former American colony.

topic40