Example for generate RSA key

1. Use OpenSSL

Generate private key:
> openssl genrsa -out private.pem 2048
By default the format of output is PKCS#1-PEM

Generate public key:
> openssl rsa -in rsa_private.pem -pubout -out rsa_public.pem

From PEM to DER
> openssl rsa -in rsa_private.pem -outform der-out rsa_private.der

From PKCS#1 to PKCS#8
> openssl pkcs8 -topk8 -in rsa_private.pem -out pkcs8_private.pem

To signs the xxx.txt with hash algorithm SHA1 and save to xxx.sign
> openssl dgst -sha1 -out xxx.sign -sign rsa_private.pem xxx.txt

To verifies the xxx.sign file 
> openssl dgst -sha1 -verify public_key.pem -signature xxx.sign xxx.txt

2. Use Python

Generate key pair

import rsa

(pubkey, privkey) = rsa.newkeys(2048)
with open('public.pem', 'w') as f:
    f.write(pubkey.save_pkcs1().decode())
with open('private.pem', 'w') as f:
    f.write(privkey.save_pkcs1().decode())

Encrypt and Decrypt

with open('public.pem', 'r') as f:
    pubkey = rsa.PublicKey.load_pkcs1(f.read().encode())
with open('private.pem', 'r') as f:
    privkey = rsa.PrivateKey.load_pkcs1(f.read().encode())

plaintext = '0123456789'
ciphertext = rsa.encrypt(plaintext.encode(), pubkey)
print(rsa.decrypt(ciphertext, privkey).decode())

Signs and Verify

import base64

signature = rsa.sign(plaintext.encode(), privkey, 'SHA-256')
b64fmt = base64.b64encode(signature)
print(b64fmt)
try:
    signature = base64.b64decode(b64fmt)
    rsa.verify(plaintext.encode(), signature, pubkey)
except rsa.VerificationError:
    print('Verifiy error')
else:
    print('Verify succeed')

3. Use C with OpenSSL 1.1.1

    BIGNUM* bn = BN_new();
    BN_set_word(bn, RSA_3);  // exponent

    RSA* rsa = RSA_new();
    int ret = RSA_generate_key_ex(
        rsa,  /* pointer to the RSA structure */
        2048, /* number of bits for the key - 2048 is a good value */
        bn,   /* exponent allocated earlier */
        NULL  /* callback - can be NULL if progress isn't needed */
    ); 

    if (ret == 1)
        RSA_print_fp(stdout, rsa, 0);
    
    RSA_free(rsa);
    BN_free(bn);

4. What is a PEM,DER and PKCS8?

Privacy-Enhanced Mail (PEM) is a file formats for cryptographic material.
The PEM format is the DER format encoded in base64 with additional header and footer lines to be transported
The header and footer lines in the PEM format defines what type of PEM file it is.
RSA Private Key:

-----BEGIN RSA PRIVATE KEY-----
-----END RSA PRIVATE KEY-----

Public Key Certificate:

-----BEGIN CERTIFICATE-----
-----END CERTIFICATE-----

DER Distinguished Encoding Rules is a key file format for cryptographic data.
It's the binary ASN.1 encoding of the data (not in plain text otherwise it's a PEM.

PKCS8 is the eighth of the Public-Key Cryptography Standards (PKCS) and is a syntax for storing private key material. 
The private keys may be encrypted with a symmetric key algorithm. 

Difference between PKCS8 and PKCS1
PKCS #8 is a private key syntax for all algorithms and not just RSA. PKCS1 is primarily for using the RSA algorithm.
PKCS #8 also uses ASN.1 which identifies the algorithm in its structure.
Over time, while PKCS1 is still valid, PKCS #8 has become the standard syntax for storing private key information.