The EP mainly consists of acoustic folk punk and folk rock songs, with a touch of power pop.[3][4] While Grace has said that it is not a "pandemic album", the uncertainty of the pandemic inspired the EP's surprise release. Of her writing process she said "I've just been trying to stay sane through the past year and a half or whatever, so I've been in this mentality of like 'Just keep working. Just keep writing songs. Share them while you're excited about them so it feels relevant to you.'"[5] In a press release, Grace said "These are songs of late night madness and loneliness, orphan songs that came wandering in looking to feed like insects".[6]
In a review for NME, Justin McMahon praised Grace for returning to her folk punk roots and said that "Yesterday Part II" could pass for an Against Me! song.[7] Skyler Graham of Mxdwn said that "Laura Jane Grace has an aptitude for sharing these deeply personal and still relatable stories, and At War With The Silverfish finds the balance between experiences that are heartbreaking and universal."[8]