[JAVA] Basic knowledge of computer science and its practice that you should know for improving performance
Introduction
I would like to share this article with Qiita to contribute to the community. https://academy.tinybetter.com/Article/37407ec1-cd3a-50c8-2fa1-39f6ed3dae74/View
There are a lot of articles on the net, but the information is fragmented and it is difficult for beginners who started the program to get the whole picture. I don't see many articles that summarize the basic understanding and performance tips of computer science, so I summarized them.
Measure everything
The most important principle of performance tuning is measurement. I will summarize the tools for measurement.
■BenchmarkDotNet https://benchmarkdotnet.org/articles/overview.html A tool for measuring C # code. Available from Nuget. Use this instead of the Stopwatch class. Stopwatch cannot measure the effects of GC etc. correctly.
■ Visual Studio profiling tool https://docs.microsoft.com/ja-jp/visualstudio/profiling/?view=vs-2019 You can use Visual Studio's profiling tools to check the status of CPU, memory, threads, GC, LOH, and more. You can quickly find out which code of which method is the Critical Path by looking at the call tree.
■Fiddler https://www.telerik.com/fiddler It is a tool that captures the contents of the network. https://zappysys.com/blog/how-to-use-fiddler-to-analyze-http-web-requests/
■ Browser developer tools You can also capture the contents of your network with your browser's developer tools without having to install Fiddler. Fiddler seems to be more sophisticated, but if you just want to look it up easily, you can use the developer tools of your browser.
■Wireshark https://www.wireshark.org/download.html Software that can capture TCP / IP communication. https://knowledge.sakura.ad.jp/6286/ https://hldc.co.jp/blog/2020/04/15/3988/
It may not be very helpful in application development, but there may come a time when you want to take a deeper look at the communication. Let's remember.
■WinDbg https://docs.microsoft.com/ja-jp/windows-hardware/drivers/debugger/debugger-download-tools You can retrieve and analyze Windows dump files. You can get a stack trace and analyze which method is causing it. https://docs.microsoft.com/ja-jp/windows-hardware/drivers/debugger/debugging-using-windbg-preview https://docs.microsoft.com/en-us/windows-hardware/drivers/debugger/debugging-using-windbg [First steps of crash dump analysis](https://blog.engineer-memo.com/2012/06/24/%E3%82%AF%E3%83%A9%E3%83%83%E3%82 % B7% E3% 83% A5% E3% 83% 80% E3% 83% B3% E3% 83% 97% E8% A7% A3% E6% 9E% 90% E3% 81% AE% E3% 81% AF % E3% 81% 98% E3% 82% 81% E3% 81% AE% E4% B8% 80% E6% AD% A9 /)
I think the content of this article is generally correct, but there may be lies. Since C # has evolved and changed from the past, it is important not to swallow the information on the Internet, but to measure and confirm it by yourself.
Understand computers and networks
Understanding computer and network principles is essential to understanding the impact on performance. A computer consists of a CPU, memory, and a hard disk. The CPU is the computing process, and the memory and hard disk are the storage devices. Memory is further subdivided into several types.
Due to the improvement of CPU calculation processing, it is no longer possible to read and write to memory. As a result, a mechanism called cache memory has been prepared. https://ascii.jp/elem/000/000/563/563800/ The cache memory (primary) is the fastest, and the cache memory (secondary) → cache memory (tertiary) → stack memory → heap memory → SSD → hard disk, and so on. The hard disk has the largest capacity. Also, the price will be getting cheaper.
By the way, in order to send a certain character string by network communication, convert the character data to Byte, resolve the name with DNS, identify the target PC, and exchange with TCP / IP with three-way handshake. The process is to gradually increase the transmission amount in the Sliding Window. https://www.infraexpert.com/study/tcpip11.html https://ascii.jp/elem/000/000/619/619702/
Since the process is performed as described above, network communication is a very time-consuming process. Communication is finally converted to a voltage ON / OFF signal, and the voltage signal is lost due to interference from the outside such as radio waves on the cable, so TCP / IP detects it and sends it again. Will occur, so it will be slower. UDP is a little faster, but in any case, network communication is the slowest process in programming.
Understand priorities
Now that you have a better understanding of computers and networks, you will learn about priorities and basic approaches. We will discuss hot code paths, network calls, file IO, heap memory, threads, async, and more. The practical part will be explained using C # and .NET as examples this time, but the basic idea of network communication, heap memory, garbage collection is the same for Python, Java, etc.
Hot code path
The first thing you need to understand is the concept of hot code paths. The inside of iterative processing such as for statement is executed many times. For example, consider the following screen.
To create such a screen, you will have to write iterative processing with a for statement etc. in the following order. Multiple users → multiple dates → multiple tasks Conceptually it looks like this.
private void ShowCalendar()
{
List<UserRecord> userList = GetUserList(); //Network communication part 1
foreach (var user in userList)
{
var startDate = new DateTime(2020, 9, 12);
for (var i = 0; i < 21; i++)
{
var date = startDate.AddDays(i);
var taskList = GetTaskList(date); //Network communication part 2
foreach (var task in taskList)
{
CreateTaskTableCell(task);
}
}
}
}
private void CreateTaskTableCell(TaskRecord record)
{
//HOT PATH!!!!!!
// Get color of Task
var color = GetColorFromDatabase(); //Network communication part 3
}
Let's say it looks like this. There are three network communication codes.
The code inside the CreateTaskTableCell method is the hot code path. This method is called very often. For example, if you have 100 users, 21 days, and an average of 5 tasks per day, you'll be called 10500 times. If you write a code (network communication part 3) such as acquiring color data from the DB in it, if the DB access is 10 milliseconds, it will take 10 milliseconds x 10500 = 105 seconds. It will take less than two minutes for the page to appear. Network communication 2 is 2100 times, which is better than network communication 3, but this also needs improvement. Network communication # 1 is called only once, so you can leave it as it is.
The part of the code that is executed frequently is called the hot code path, and it is necessary to improve this code preferentially. This video https://www.youtube.com/watch?v=4yALYEINbyI At 12:10 20% of code is consuming 80% of resources. 4.0% of code is consuming 64% of resources. 0.8% of code consumes 51% of resources. Is saying. You can see how the hot code path is consuming resources.
Improve from the place with the greatest impact
The program code to note is often in the following order: ・ Network communication ・ Hard disk operation -Heap memory Apart from the above order, you also need to understand CPU, threads, async, exceptions, etc. and be careful to write good performance code.
Know the network communication code
Network communication is slow, so try to reduce the number of times as much as possible. It is important to know the method in which network communication is occurring in C #. The classes that are in network communication are listed below.
SqlConnection、SqlCommand
var cn = new SqlConnection();
var cm = new SqlCommand("insert into Table1.....");
cm.Connection = cn;
cn.Open(); //Is network communication occurring? Check it out.
cm.ExecuteNonQuery(); //Network communication has occurred!
HttpClient
var cl = new HttpClient();
var res = await cl.GetAsync("https://www.hignull.com"); //Network communication has occurred!
Google Calendar
var credential = new ServiceAccountCredential(
new ServiceAccountCredential.Initializer(serviceAccountEmail)
{
Scopes = new[] { CalendarService.Scope.Calendar }
});
var sv = new CalendarService(new BaseClientService.Initializer()
{
HttpClientInitializer = credential,
ApplicationName = "MyApp1",
});
var req = new EventsResouce.ListRequest(sv, "calendarID");
var eventList = req.Execute(); //Network communication has occurred!
Azure Blob Storage
CloudStorageAccount account = CloudStorageAccount.Parse("My connection string");
var cl = account.CreateCloudBlobClient();
var container = cl.GetContainerReference("My Container name");
var list = container.ListBlobs(); //Network communication has occurred!
Have you got a feel for it? Network communication is occurring when data is acquired by calling a DB server, an external WEB server, or an API of Saas (Google, Stripe, Office365, Blob). It's easy to get an image if you run the program on your own PC. If it is Google Calendar, the data stored in the computer of Google's data center will be acquired, so network communication will occur.
Furthermore, this data is probably stored on your hard disk (or SSD). Execution of these methods is very slow because the flow is that after performing network communication that takes a very long time, data is read from the hard disk, which also takes a long time, and the value is returned by network communication again.
Know the code for file operations
Next to the network is the code for file operations. Let's know some classes that do file operations.
var text = System.IO.File.ReadAllText("C:\\Doc\\MyMemo.txt");
When using the FileStream class, it is also an operation on the hard disk.
FileStream fs = new FileStream("C:\\Doc\\MyMemo1.txt", FileMode.Create);
StreamWriter sw = new StreamWriter(fs);
sw.WriteLine("WIFI Number 1234-5678-4563");
sw.Close();
fs.Close();
Understand heap memory
A table of contents on the tips needed to use heap memory efficiently.
・ Avoid unnecessary new
·boxing
・ String is immutable
・ StringBuilder
・ String.Intern
-Initialize List correctly
・ Be aware of lambda capture
-Use ArrayPool
-Use ValueTask instead of Task
・ Use Span
Avoid unnecessary new
private void Main()
{
List<UserRecord> userList = new List<UserRecord>();
userList = GetUserList();
}
private List<UserRecord> GetUserList()
{
List<UserRecord> userList = new List<UserRecord>();
// get user list...
return userList;
}
Unnecessary new wastes heap memory. Let's avoid it.
Avoid making both lowercase when comparing letters.
var s1 = "a";
var s2 = "A";
var isSameString = s1.ToLower() == s2.ToLower(); //Memory is wasted
Use Striing.Compare instead.
var s1 = "a";
var s2 = "A";
var isSameString = String.Compare(s1, s2, StringComparison.CurrentCultureIgnoreCase);
boxing
Boxing occurs when the value type is assigned to the Object type.
int x = 10;
object o = x; //Boxing has occurred!
Memory allocation occurs when boxing occurs.
Allocating heap memory is a time consuming process. (Network communication> Hard disk operation> Heap memory allocation) https://ufcpp.net/study/computer/MemoryManagement.html Avoid unnecessary boxing.
Casting a value type to an interface will cause boxing.
struct MyStruct : IComparer<int>
{
public int Compare(int x, int y)
{
return x.CompareTo(y);
}
}
private Main()
{
int s = new MyStruct();
IComparer<int> comparer = s; //Boxing has occurred!
s.Compare(0, 1);
}
If you can avoid it, avoid it.
Understand String
First and foremost, the lesser-written important thing is that String is a class but immutable. https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/builtin-types/reference-types
What does this mean? For example, if you have the following code
string s = "Hello";
s = "Hello World!";
You may have an image like the figure below as the behavior of the memory when writing this code.
Actually, allocate a new area in the heap memory as shown below.
This is why String is said to be immutable. If you want to change the String a certain number of times, use StringBuilder.
StringBuilder sb = new StringBuilder();
sb.Append("Hello");
sb.Append("World!");
In this example, you can set the value Hello World! From the beginning, but in reality, you can construct a query from the input value of the text box, construct a message to the user, etc., and use StringBuilder in such cases. Will be.
If you concatenate or change characters in String many times, the heap memory will be allocated and the previously used value (Hello in this example) will not be used, and the number of garbage collection will increase.
Use String.Intern
If you use the Intern method to intern a character string that is used many times in your application, you can reduce memory consumption and improve performance. For example, the word "add" on the add button would be a good target for internship.
Initialize List correctly
The constructor of a List class has an overload that receives capacity. This overload allows you to specify the size of the array provided internally. For example, if you have a task list page and you want to display 50 tasks per page, you can prevent the array from being regenerated by specifying 50 from the beginning. As a result, memory consumption can be reduced and processing time can be shortened.
List<TaskRecord> taskList = new List<TaskRecord>(50);
The source code of List
https://referencesource.microsoft.com/#mscorlib/system/collections/generic/list.cs
public class List<T> : IList<T>, System.Collections.IList, IReadOnlyList<T>
{
private const int _defaultCapacity = 4;
static readonly T[] _emptyArray = new T[0];
public List() {
_items = _emptyArray;
}
public List(int capacity) {
if (capacity < 0) ThrowHelper.ThrowArgumentOutOfRangeException(ExceptionArgument.capacity, ExceptionResource.ArgumentOutOfRange_NeedNonNegNum);
Contract.EndContractBlock();
if (capacity == 0)
_items = _emptyArray;
else
_items = new T[capacity];
}
//abridgement.............................
public void Add(T item) {
if (_size == _items.Length) EnsureCapacity(_size + 1);
_items[_size++] = item;
_version++;
}
private void EnsureCapacity(int min) {
if (_items.Length < min) {
int newCapacity = _items.Length == 0? _defaultCapacity : _items.Length * 2;
if ((uint)newCapacity > Array.MaxArrayLength) newCapacity = Array.MaxArrayLength;
if (newCapacity < min) newCapacity = min;
Capacity = newCapacity;
}
}
If you add an item and it exceeds the length of the internal array, the EnsureCapacity method is used to recreate the array. If you know the size, specify it from the beginning to avoid wasting memory.
Be aware of lambda capture
When you capture an external variable with a lambda expression, a class is implicitly defined and its instance is created.
public Action GetWriteLineAction(int number)
{
return () => Console.WriteLine(number);
}
It compiles like this:
public class __GeneratedClass
{
public int number;
public void GeneratedMethod()
{
Console.WriteLine(number);
}
}
public Action GetWriteLineAction(int number)
{
var c = new __GeneratedClass();
c.number = number;
return c.GeneratedMethod;
}
Capturing an external variable creates an instance and consumes heap memory. Be especially careful when using lambda expressions with hot code paths. In the latest C #, you can add a static keyword to a lambda expression to prohibit the capture of external variables, so if you want to prevent unintended capture of external variables, use it positively.
Avoid passing static methods to lambdas
Delegate static methods is slow. The reason is described in detail below ↓ https://ufcpp.net/study/csharp/functional/miscdelegateinternal/#static-method In addition, there are various test results such as slow delegate of static method. https://gist.github.com/ufcpp/b2e64d8e0165746effbd98b8aa955f7e
Consider the following code.
static void Main(string[] args)
{
var textList = new List<String>();
textList.Add(" "); //Empty
textList.Add("Higty");
//....Add some
textList.Where(String.IsNullOrWhiteSpace); //Static method
textList.Where(x => String.IsNullOrWhiteSpace( x ) ); //Surround with lambda
}
If you write a static method as an argument, it will actually be compiled as follows.
textList.Where(new Func<String, Boolean>(string.IsNullOrWhiteSpace));
Every time an instance of the Func class is created, it wastes memory. In addition, the call itself is very slow due to the delegate call being optimized towards the instance method.
If you surround it with lambda https://ufcpp.net/study/csharp/sp2_anonymousmethod.html#static
class __GeneratedClass
{
static Func<String, Boolean> _ActionCache = null;
internal Boolean __GeneratedMethod(String text)
{
return String.IsNullOrWhiteSpace(text);
}
}
static void Main(string[] args)
{
if (__GeneratedClass._ActionCache == null)
{
__GeneratedClass._ActionCache = __GeneratedClass.__GeneratedMethod;
}
textList.Where(__GeneratedClass._ActionCache);
}
It will be expanded like this, and the generation of useless instances will be zero. You can reduce wasted memory and reduce the number of garbage collections. Also, since it is an instance method call, the call will not be delayed.
Use ArrayPool
For example, when processing a file download or processing a file uploaded on a website, data is exchanged using Byte []. You may also use Byte [] in the buffer. By using ArrayPool, you can avoid allocating Byte [] to heap memory and use the already prepared Byte []. http://ikorin2.hatenablog.jp/entry/2020/07/25/113904 By avoiding allocation to heap memory, you can expect to reduce the number of garbage collections.
Use ValueTask instead of Task
https://www.buildinsider.net/column/iwanaga-nobuyuki/009 Performance can be improved if asynchronous methods are called over multiple layers.
Leverage Span and Stakalloc
You can improve performance and reduce memory usage by using Span
Understand GC (garbage collection)
In C #, there is a mechanism called garbage collection in which the .NET Framework (.NET Core) manages memory. https://docs.microsoft.com/ja-jp/dotnet/standard/garbage-collection/fundamentals Simply put, the heap memory is managed by dividing it into three generations, and the heap memory that is no longer referenced from anywhere is cleared to fill the free space and organize the memory. https://ufcpp.net/study/csharp/rm_gc.html https://ufcpp.net/study/computer/MemoryManagement.html?sec=garbage-collection#garbage-collection
As heap memory is consumed, garbage collection processing is executed at some point. This process is heavy. Try to reduce the number of times garbage collection occurs. In particular, the Gen2 collection process is very heavy. Let's reduce the number of GC occurrences by utilizing methods such as not wasting heap memory and reusing array data such as Byte [].
Tips to prevent the execution time of garbage collection from becoming long. https://docs.microsoft.com/en-us/previous-versions/dotnet/articles/ms973837(v=msdn.10)
Summary -If there are many allocations, the execution time will be long. -If there is a huge object (such as new Byte [10000]), the execution time will be long. -If there is a class in which objects refer to each other in a complicated manner, it takes time to verify that they are not referenced. -If there are many root objects, it takes time to verify that they are not referenced. -Carefully implement the creation of a class with a finalizer.
What is a finalizer?
https://docs.microsoft.com/ja-jp/dotnet/csharp/programming-guide/classes-and-structs/destructors If you have a finalizer, your generation will be automatically promoted by one generation. This means that there will be no Gen0 recovery. With a finalizer, it's about 300 times slower. ↓ https://michaelscodingspot.com/avoid-gc-pressure/ Also, if there is a problem with the finalizer implementation, the following problems will occur. https://troushoo.blog.fc2.com/blog-entry-56.html Let's watch out.
Learn about common memory leak patterns
https://michaelscodingspot.com/ways-to-cause-memory-leaks-in-dotnet/ https://michaelscodingspot.com/find-fix-and-avoid-memory-leaks-in-c-net-8-best-practices/
Event handler Anonymous method, lambda expression capture Static fields become the root of garbage collection About WPF Binding If the amount of data to be cached is large, memory will be insufficient and OutOfMemoryException will occur. About Week Reference Variables on the Stack of Threads (and Timers) that never end become the root of garbage collection
If a memory leaks, it will not be released, the available memory area will decrease, and garbage collection will occur frequently. Garbage collection is a heavy process that slows down overall application performance.
About exceptions
Raising an exception is a heavy process. Avoid raising unnecessary exceptions.
private Int32? GetNumber(String text)
{
try
{
var x = Int32.Parse(text);
return x;
}
catch (Exception ex)
{
return null;
}
}
There is a TryParse method instead of the Parse method.
private Int32? GetNumber(String text)
{
if (Int32.TryParse(text, out var x))
{
return x;
}
return null;
}
Class library to be careful about usage
First is the data access class.
Use DataReader rather than DataSet
DataSet and DataTable have various functions and are slow. If you just want to read the data, use DataReader.
Process multiple rows using Table-Valued Parameter or SqlBulkCopy
If you want to process multiple rows at once, use TVP (Table-Valued Parameter) or SqlBulkCopy. https://www.sentryone.com/blog/sqlbulkcopy-vs-table-valued-parameters-bulk-loading-data-into-sql-server TVP is faster for records of 1000 or less. This is because Bulk Insert takes some time to initialize. https://docs.microsoft.com/en-us/sql/relational-databases/tables/use-table-valued-parameters-database-engine?view=sql-server-ver15#BulkInsert For inserting more than 1000 records, use SqlBulkCopy to insert the data.
Calling a TVP stored from C # can be quite annoying. DbSharp makes it easy to automatically generate C # calling code. Introduction of DbSharp (C # source code automatic generation tool that executes stored procedure) Let's utilize it.
Pay attention to the dictionary key
GetHashCode is used to compare the keys in the Dictionary. Key comparisons are slow when using Strings or Enums as keys. http://proprogrammer.hatenadiary.jp/entry/2014/08/17/015345 It is a method to speed up when using Enum as a key ↓ https://stackoverflow.com/questions/26280788/dictionary-enum-key-performance/26281533 If possible, use Int32 etc. as a key.
use async, await
Don't use the Wait () method or the Result property. It will block the thread until the process is complete. Use async to free threads during latency. The released thread is used to perform other processing. Requests fly in parallel on websites, etc., and there is a wait for data acquisition from the DB. By making everything asynchronous, the thread does not just have to wait without doing anything, and it is always executing the processing of one of the requests, so the overall throughput is improved.
Use Stream
Data can be read and written in sequence by using Stream. If you can, use it.
Do not use HttpClient
According to the official documentation, HttpClient requires special usage. https://docs.microsoft.com/ja-jp/dotnet/api/system.net.http.httpclient?view=netcore-3.1 It is a reference article of proper usage ↓ https://qiita.com/superriver/items/91781bca04a76aec7dc0
DI and retry can be set easily by using HttpClientFactory. https://docs.microsoft.com/en-US/dotnet/architecture/microservices/implement-resilient-applications/use-httpclientfactory-to-implement-resilient-http-requests
Prevent memory fragmentation and Out Of Memory when using Socket
Byte [] is used for sending and receiving data on Socket.
private voi Send()
{
var buffer = new Byte[1024];
socket.BeginSend(buffer, ...);
}
The Byte [] passed to BeginSend is the I / O Completion Port, and data is written from the OS when I / O is completed. You need to ensure that the OS can write to this address without fail, and this buffer will be PINned to prevent the address from moving when processing garbage collection. http://ahuwanya.net/blog/post/Buffer-Pooling-for-NET-Socket-Operations
The problem is, for example, if you send data using a buffer of 1024 size, SendCallback will be executed 1000 times to send 1MB of data, 1000 instances of Byte [] will be generated, and all of them will be PINed. I will. The PINd object is not moved in garbage collection, which causes memory fragmentation. As a result, Out Of Memory occurs.
Reflection, Expression tree, dynamic, async, threads
In addition, we will introduce various speed-up tips.
Cache reflection
Reflection is a very slow process.
List<UserRecord> userList = GetUserList();
foreach (var user in userList)
{
PropertyInfo namePropertyInfo = Typeof(UserRecord).GetProperty("Name"); //Very slow!
String userName = (String)namePropertyInfo.GetValue(u);
}
Let's reduce the number of executions and avoid slowing down.
PropertyInfo namePropertyInfo = Typeof(UserRecord).GetProperty("Name");
List<UserRecord> userList = GetUserList();
foreach (var user in userList)
{
String userName = (String)namePropertyInfo.GetValue(u);
}
GetProperty, GetMethod, etc. are slow. Use the cache to avoid slowdowns. https://yamaokuno-usausa.hatenablog.com/entry/20090821/1250813986
Make good use of dynamic
If you use dynamic, the compiler will do the cache nicely to some extent ↓ https://ufcpp.net/study/csharp/misc_dynamic.html
Use Generic Type Caching
Generic Type Caching is faster because the map is complete at compile time. Let's use it when the Type can be determined statically. http://csharpvbcomparer.blogspot.com/2014/03/tips-generic-type-caching.html
Cache the compilation of the Expression tree
Compiling Expression is a heavy process. Make sure to cache the compiled delegate.
Make the Regex class compiled
Make the Regex class compiled with the following code. var rg= new Regex("[a-zA-Z0-9-]*", RegexOptions.Compiled); You can expect performance improvement by making it compiled.
Speed up Enum ToString
The Enum's ToString uses reflection internally. https://referencesource.microsoft.com/#mscorlib/system/type.cs,d5cd3cb0c6c2b6c1
public override String ToString()
{
return Enum.InternalFormat((RuntimeType)GetType(), GetValue());
}
//************************abridgement***************************
private void GetEnumData(out string[] enumNames, out Array enumValues)
{
Contract.Ensures(Contract.ValueAtReturn<String[]>(out enumNames) != null);
Contract.Ensures(Contract.ValueAtReturn<Array>(out enumValues) != null);
FieldInfo[] flds = GetFields(BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Static);
object[] values = new object[flds.Length];
string[] names = new string[flds.Length];
for (int i = 0; i < flds.Length; i++)
{
names[i] = flds[i].Name;
values[i] = flds[i].GetRawConstantValue();
}
// Insertion Sort these values in ascending order.
// We use this O(n^2) algorithm, but it turns out that most of the time the elements are already in sorted order and
// the common case performance will be faster than quick sorting this.
IComparer comparer = Comparer.Default;
for (int i = 1; i < values.Length; i++)
{
int j = i;
string tempStr = names[i];
object val = values[i];
bool exchanged = false;
// Since the elements are sorted we only need to do one comparision, we keep the check for j inside the loop.
while (comparer.Compare(values[j - 1], val) > 0)
{
names[j] = names[j - 1];
values[j] = values[j - 1];
j--;
exchanged = true;
if (j == 0)
break;
}
if (exchanged)
{
names[j] = tempStr;
values[j] = val;
}
}
enumNames = names;
enumValues = values;
}
It's very slow because it uses reflection. Here's how to speed it up ↓ https://qiita.com/higty/items/513296536d3b26fbd033
Expand the loop
You can speed up by expanding the loop. For example, suppose you want to increment all the values in an array by one.
var numberList = new Int32[40000];
for (var i = 0; i < numberList.Length; i++)
{
numberList[i] += 1;
}
Expand the loop and change it to handle the four elements at once.
var numberList = new Int32[40000];
for (var i = 0; i < 10000; i = i + 4)
{
numberList[i] += 1;
numberList[i + 1] += 1;
numberList[i + 2] += 1;
numberList[i + 3] += 1;
}
In the loop processing, the expression (i <numberList.Length) and increment (i ++) processing that verifies the condition of whether to repeat the processing each time is executed. By expanding, the above processing of 40,000 times becomes 10000 times, and the processing of evaluation and increment of the condition of 30,000 times is eliminated. If the number of elements in the array is not a multiple of 4, some tweaking is required, but loop unrolling can speed things up. You can get even faster by adding 10 or more instead of 4.
use for instead of foreach
Foreach is slightly slower than for because it accesses the Current property and calls the MoveNext method every time. It's negligible for most parts of the day, but if you're looking for very severe performance gains, consider using for. Especially when it is a combination of array and for, the compiler will do a lot of optimization.
Avoid LINQ-style writing and use foreach
Calls that connect LINQ-like method chains may be a little slower or much slower due to method calls. Refer to the exchange of comments on this page ↓ https://michaelscodingspot.com/performance-problems-in-csharp-dotnet/
LINQ style writing
var numbers = GetNumbers();
int total = numbers.Where(n => n % 2 == 0).Sum(n => n * 2);
return total;
Even with LINQ-style writing, the optimizer will optimize it to the maximum, but in most cases normal foreach is faster.
var numbers = GetNumbers();
int total = 0;
foreach (var number in numbers)
{
if (number % 2 == 0)
{
total += number * 2;
}
}
return total;
About copying Array
Source ↓ https://stackoverflow.com/questions/5099604/any-faster-way-of-copying-arrays-in-c https://marcelltoth.net/article/41/fastest-way-to-copy-a-net-array https://www.youtube.com/watch?time_continue=4&v=-H5oEgOdO6U&t=1689 According to this video, Array.CopyTo seems to be the fastest. The reason is that it calls the API of the OS directly.
Array secrets not in the documentation
https://www.codeproject.com/Articles/3467/Arrays-UNDOCUMENTED
Array loop is the fastest
int hash = 0;
for (int i=0; i< a.Length; i++)
{
hash += a[i];
}
Saving Length in a local variable is slow
int hash = 0;
int length = a.length;
for (int i=0; i< length; i++)
{
hash += a[i];
}
In the case of the code below, it is difficult to guarantee that the length variable has not been changed (it takes time to compile) considering changes from other threads. As a result, the boundary value check of Array works, which slows down. In the first example, it is confirmed that a.Length is within the length of the array, so the boundary value check is omitted and the intermediate code that optimizes the processing for each element of the array is generated, so it is faster. ..
Use fields instead of properties
Properties are actually methods. Method calls will be inserted when accessing properties. Even the cost of method calls can be unacceptable in environments where performance is critical. In such cases, consider making it a field instead of a property.
Pay attention to the index of the loop
If the index order of the double loop is incorrect, the memory locality of the CPU cannot be utilized well and the processing may be slowed down. https://raygun.com/blog/c-sharp-performance-tips-tricks/
Code that can take advantage of memory locality
for (int i = 0; i < _map.Length; i++)
{
for (int n = 0; n < _map.Length; n++)
{
if (_map[i][n] > 0)
{
result++;
}
}
}
Code that cannot be used well
for (int i = 0; i < _map.Length; i++)
{
for (int n = 0; n < _map.Length; n++)
{
if (_map[n][i] > 0)
{
result++;
}
}
}
The code above is about 8 times faster. The data used by the above code is placed closer to the memory and can be read at once.
Make the UI always responsive
Users are stressed when the UI is unresponsive. Let's execute the processing that takes a long time in the background thread, and make the UI thread always responsive. Use the Thread class or the async and await patterns.
Implement cache function
DB data, data acquired by network communication, data acquired by file operation, etc. are placed in memory or Redis to speed up access. Data with the following characteristics is suitable for caching. ・ Small amount of data ・ Frequently referred to ・ Not changed much If you cache too much data to keep it in memory, the memory will be exhausted. Caching less-referenced things wastes valuable memory space. If the data in the cache source is updated, the cache needs to be updated. Caching frequently updated data uses CPU and memory for the update process, so such data is not suitable for caching.
Based on the above, for example, the following data is appropriate as a cache target.
-Application text (the word "add" in the add button is unchanged) ・ Organization master (changed only once a year) ・ HTML of last month's sales data table (last month's sales data is unchanged) And so on.
public class CacheData
{
public static Dictionary<String, String> TextList = new Dictionary<String, String>();
}
Basically it will be implemented with static fields (or properties). Update this data when the original data is updated. In the case of WEB application, since it is accessed from multiple threads, it is necessary to manage conflicts with lock etc.
If you are scaling out your web app in the cloud, you can update the cache of all your web apps by sending notifications to multiple web apps using Redis Publish and Subscribe.
This article uses C # as an example to introduce the cache, but the cache mechanism itself is useful. The cache mechanism is used in various places such as CDN, DNS, and browser image files.
Debug builds are slow and release builds are fast
Don't forget to make it a release build.
bonus
I wrote some articles for those who are interested in how to create high performance applications. I tried to create task management-tools and services to improve from beginner level- Technology and knowledge you need to know to improve the performance of web applications I made the world's fastest Mapper library in C # (3 to 10 times faster than AutoMapper etc.) Functional UI design for programmers Advanced technical articles for becoming a world-class engineer
appendix
- Other quoted articles https://michaelscodingspot.com/performance-problems-in-csharp-dotnet/ https://devblogs.microsoft.com/dotnet/system-io-pipelines-high-performance-io-in-net/ https://raygun.com/blog/c-sharp-performance-tips-tricks/ https://www.c-sharpcorner.com/UploadFile/dacca2/5-tips-to-improve-performance-of-C-Sharp-code/ https://michaelscodingspot.com/avoid-gc-pressure/ https://michaelscodingspot.com/challenging-the-c-stringbuilder/