You can do what you want but it might be very slow, especially for larger libraries. Calibre will need frequently to check the format file mod times. If you sort by the column then calibre must retrieve the mod time of every format file. With that proviso ...
Create a custom column based on other columns. Don't use the "acts like tags" variant. Set it to sort by date. Put the following into the template field:
Code:
program:
sublist(list_re(formats_modtimes('yyyy-MM-dd hh:mm:ss'), ',', '.*?:(.*)', '\1'), 0, 1, ',')