Set the value of "Attributes" in the appropriate key below to "70 01 00 20", you will then be able to rename and/or delete any special folders by right-clicking them. 

Recycle Bin
[HKEY_CLASSES_ROOT\CLSID\{645FF040-5081-101B-9F08-00AA002F954E}\ShellFolder] 

Search the [HKEY_CLASSES_ROOT\CLSID] key for any other folders you want to modify. 

Value Name: Attributes
Data Type: REG_BINARY
Data:(70 01 00 20=enable, 40 01 00 20=default)

 

Back to Windows Tweaks