What are some unconventional ways programmers can improve their problem-solving skills outside of coding challenges and traditional learning resources?

Programming involves skills in language, algorithm, abstraction, hardware operations, compiler characteristics and creativity. What we typically mean by programming skill are the first three. Hence, unconventional ways encompass using hardware operations efficiently as well as providing hints to compilers so that it generates efficient code.

Knowing how hardware works including memory is enormously important in writing efficient code. Conventional wisdom of compiler should take care of proper code generation has its limits and magic actually happens beyond that limit. At the minimum, programmers should be aware and use following hardware specific optimizations. Application of these will need knowledge of compiler characteristics.

Use of vectorizations - most modern hardwares provide data parallel operations (SIMD). However, to take advantage of SIMD, programmer should know what not to use in a loop or how to refactor part of the code or how to provide hint to the compiler so that vectorized code is generated. Of course, modern compilers provide information on which loops are not vectorized and what can be done to vectorize. In some compliers, explicitly vector operations can be invoked. Result of vectorization is drastic improvement in speed, typically 3 to 8 times depending on vector width.

Use of SIMT - A level higher than vectorization is light-weight threads, typically GPU threads. To effectively use SIMT, programmers has to know how to reduce data transfer between CPU and GPU, when to use shared memory, how to use multi-level parallelism etc. Drastic improvement of typically 5 to 100 times in speed can result from SIMT use.

Adjacent memory use - A fundamental thing in CPU or GPU operation is memory caching. Programmers should know how to access memory in adjacent areas so that cache hit will be high. Programmers should know how to arrange data in multi-dimensional array or keep in just single dimension to ensure cache hit remains high.

Aligned memory access - If data elements are aligned (and intimated to compiler) as per natural width of processor reading, less activities are needed in processor to read such data. Programmers should know how to inform compiler to get aligned memory.

Other unusual ways to improve programming skill is to use often-neglected and long-forgotten hacks. A large number of these can be found in the book Hacker’s Delight by Henry S. Warren Jr. (https://en.wikipedia.org/wiki/Hacker%2527s_Delight). Bitwise operations are delight on their own rights but these can make sometimes programs very compact and very efficient.

In conclusion, improving programming skills beyond competitive coding should involve going to the basics of hardware operations and algorithmic diversities. To curious minds, the book Writing Efficient Programs by John Bentley (Writing efficient programs: | Guide books | ACM Digital Library) can still serve as a reminder of what we should consider while we write programs. Similarly, all the volumes of The Art of Computer Programming (The Art of Computer Programming - Wikipedia) sit in the shelf to remind how far someone can travel to make best use of the most appropriate algorithm for a problem at hand.