You can add a multi-line label with the following:
JLabel label = new JLabel("My label");
label.setText("<html>This is a<br>multline label!<br> Try it yourself!</html>");
From here, simply add the label to the frame using the add() method, and you're all set!