Public forms can't be deleted, since they're your pipe's start form.
What you can do, however, is disable the public form feature so that external users will no longer be able to access it using the public URL.
1. The first step for disabling the public form is clicking the "+" icon on the lower right corner of the screen to open the form;
2. Then, click on "share" to open the public form settings and click on the switch icon;
3. That's it! Your form is no longer public!